Temporal Task Queues: Designing Worker Pools for CPU, I/O, and GPU Workloads

A Temporal application can perform well in development and still develop serious performance problems in production.

The issue is often not the workflow logic or the Temporal service itself. It is the assumption that every Activity can run efficiently inside the same worker pool.

A payment API call, image-processing job, database query, and machine-learning inference task may all belong to the same business process, but they have very different execution requirements. Some consume CPU continuously. Others spend most of their time waiting on a network response. GPU workloads depend on scarce accelerator memory and may degrade when too many tasks run simultaneously.

When all of these workloads share one task queue and one worker configuration, the architecture forces incompatible work to compete for the same resources.

The result can include:

• CPU-heavy jobs delaying customer-facing API calls
• I/O-bound tasks leaving CPU capacity underused
• Expensive GPU workers processing ordinary workloads
• Background jobs competing with critical transactions
• Concurrency settings that work for one Activity but harm another
• Scaling policies that react to the wrong resource signals

Temporal task queues provide a way to solve this problem. By routing different classes of work to specialized worker pools, teams can tune compute, concurrency, autoscaling, hardware, and capacity guarantees around the work being executed.

What Are Temporal Task Queues?

A Temporal task queue is the routing layer between the Temporal service and the workers that execute Workflow Tasks or Activities.

When a Workflow schedules an Activity, the task is placed on a task queue. Workers configured to poll that queue can pick up and execute the task.

This creates a useful separation:

• Temporal coordinates when work should happen.
• Task queues determine where the work is routed.
• Workers provide the resources and application code that perform it.

The Temporal service does not need to know whether an Activity requires a standard CPU instance, a GPU, a particular programming language, or a high-memory node. It places the task on the selected queue, and the appropriate worker fleet handles the execution.

This makes task queues more than a delivery mechanism. They are an architectural boundary for controlling how different workloads consume resources.

Why a Single Temporal Task Queue Creates Performance Problems

Using one queue for every Activity may feel simpler at the beginning. There is only one worker deployment to manage, one concurrency setting, and one scaling policy.

That simplicity disappears as workloads become more varied.

Consider a worker pool processing three types of work:

1. Payment API calls that spend most of their time waiting on an external gateway
2. Document transformations that use a full CPU core during execution
3. Machine-learning inference that requires GPU memory

There is no single worker configuration that serves all three efficiently.

A high concurrency setting may work well for payment API calls because many requests can wait simultaneously. The same setting can cause severe CPU contention during document processing and GPU memory exhaustion during inference.

A low concurrency setting protects CPU and GPU workloads, but it leaves I/O workers underutilized and causes API tasks to wait unnecessarily.

A shared queue also makes scaling less precise. When task volume rises, the system cannot easily determine whether it needs more CPU workers, more network-oriented workers, or additional GPU capacity. It can only scale a mixed fleet.

The better approach is task queue specialization: divide work according to its execution characteristics and create worker pools designed for each category.

Designing Temporal Task Queues for CPU-Bound Activities

CPU-bound Activities spend most of their runtime performing computation rather than waiting on an external service.

Common examples include:

• Image processing
• Video encoding
• Encryption
• Compression
• Data aggregation
• Complex calculations
• Large document transformations

For this workload, throughput is constrained by available CPU cores.

Match Temporal worker concurrency to CPU capacity

Running more CPU-intensive Activities than the machine can process does not automatically increase throughput.

For example, if a worker has eight CPU cores, allowing sixteen compute-heavy Activities to run simultaneously means multiple tasks must compete for the same cores. The additional concurrency can increase context switching and execution time without completing more useful work.

A practical starting point is to align maximum Activity concurrency approximately with the number of available cores, then validate the setting under realistic load.

CPU-bound worker pools generally benefit from:

• Higher CPU allocations
• Lower, controlled concurrency
• More powerful instance types
• Dedicated task queues
• Scaling based on queue demand and execution latency

Example CPU task queue

A document-processing workflow might route computational Activities to:

document-processing-cpu

Workers polling this queue could run on compute-optimized Kubernetes nodes with eight cores and a carefully tested concurrency limit.

The Workflow remains responsible for coordinating the process. The specialized queue ensures the expensive transformation work runs on infrastructure designed for it.

Designing Temporal Worker Pools for I/O-Bound Activities

I/O-bound Activities spend much of their runtime waiting for another system.

Examples include:

• Calling a payment gateway
• Querying a database
• Reading from object storage
• Invoking an identity provider
• Sending an email
• Calling a shipping or logistics API
• Waiting for a third-party service response

During that waiting time, the worker may use very little CPU. This means a worker can often execute many I/O-bound Activities concurrently without saturating the machine.

Use higher concurrency carefully

I/O-bound worker pools can generally support higher concurrency than CPU-bound pools. The source architecture identifies two to five times the CPU core count as a possible starting range, subject to workload testing.

That does not mean concurrency should be increased without limits.

The actual capacity may be constrained by:

• Database connection pools
• External API rate limits
• Available memory
• File descriptors
• Network throughput
• SDK configuration
• Downstream service latency

Increasing worker concurrency can improve throughput, but it can also overwhelm the system the Activities are calling.

For I/O-bound Temporal task queues, teams should balance worker capacity with downstream capacity.

These worker pools usually benefit from:

• Moderate CPU allocation
• Higher concurrency
• Standard, cost-efficient instances
• Connection-pool monitoring
• Rate-limit awareness
• Task-queue-specific autoscaling

Example I/O task queue

A commerce platform could route external payment calls to:

payment-gateway-io

This queue could be served by workers configured for higher concurrency, while computational fraud-analysis tasks use a different worker pool.

The separation prevents slow payment-provider responses from occupying execution capacity intended for unrelated work.

Designing Dedicated Task Queues for GPU Workloads

GPU-bound Activities require a substantially different architecture.

Typical examples include:

• Machine-learning inference
• Video processing
• Scientific computing
• Computer vision
• Model-based document extraction
• Large-scale numerical operations

GPU capacity is expensive, specialized, and constrained by accelerator memory. Excessive parallel execution can create memory contention, out-of-memory failures, and lower throughput.

Keep GPU concurrency intentionally low

Unlike I/O worker pools, GPU workers often need highly constrained concurrency. A maximum concurrency of one or two may be appropriate for some workloads, depending on model size and accelerator capacity.

The correct setting must be established through workload testing because two models running on the same accelerator may have radically different memory requirements.

GPU-oriented Temporal worker pools should typically use:

• Dedicated GPU nodes
• Separate task queues
• Low concurrency
• Explicit resource allocation
• Maximum replica limits
• Queue-depth and latency monitoring

Avoid sending standard work to GPU workers

Without task queue separation, a GPU-backed worker could receive Activities that require no accelerator at all. This wastes expensive capacity and prevents GPU-dependent tasks from starting promptly.

A dedicated queue such as:

ml-inference-gpu

ensures only inference Activities reach the GPU fleet.

Meanwhile, API calls, preprocessing, and result persistence can use lower-cost CPU or I/O worker pools.

CPU vs. I/O vs. GPU Temporal Worker Pools

These values should be treated as design starting points, not universal defaults. The correct worker configuration depends on Activity duration, resource consumption, downstream limits, and workflow service-level objectives.

For teams preparing this architecture for production, Xgrid helps map Activity profiles to task queue boundaries, worker configurations, Kubernetes node pools, and scaling policies. Its Temporal Production Deployment Checklist can also help teams evaluate whether queue isolation, concurrency, monitoring, and capacity controls are ready before launch.

Review your Temporal task queue architecture with Xgrid →

Use Temporal Task Queues for Hardware Specialization

CPU and GPU are not the only infrastructure boundaries that matter.

Some Activities may require:

• High-memory nodes
• Local SSD storage
• Access to a private network
• Region-specific infrastructure
• Specialized processors
• Particular security controls

Dedicated task queues allow these Activities to reach workers deployed on the appropriate hardware.

For example:

• large-memory-processing
• private-network-database
• eu-residency-activities
• gpu-model-inference

This prevents teams from overprovisioning every worker to satisfy the most demanding Activity.

Instead of running all Activities on high-memory or GPU-enabled instances, the platform reserves expensive infrastructure for the work that actually needs it. This can significantly improve infrastructure efficiency.

Build Polyglot Workflows with Language-Specific Task Queues

Task queues can also separate execution by programming language.

A Workflow written in Go can schedule an Activity implemented by Python workers. The Workflow targets the queue polled by the Python worker fleet, and Temporal coordinates the execution through the shared workflow model.

This allows engineering teams to use:

• Go for high-concurrency orchestration
• Python for machine learning and data tooling
• Java for enterprise integrations
• TypeScript for services built around the JavaScript ecosystem

For example, a Go Workflow could coordinate an onboarding process while routing document-classification work to:

python-document-classification

A Python worker polls that queue, executes the model, and returns the result to the Workflow.

The architectural benefit is that teams can choose the right language for each task without replacing Temporal as the orchestration layer.

Separate High-Priority and Background Temporal Workloads

Workload type is only one way to specialize task queues. Business importance matters as well.

A customer-facing payment workflow should not compete for the same execution capacity as a nightly reporting job. Even when both workloads are I/O-bound, they may require different performance guarantees.

High-priority Temporal task queues

Critical queues may need:

• Guaranteed minimum worker capacity
• Spare capacity for sudden bursts
• Aggressive autoscaling
• Higher resource allocations
• Tighter schedule-to-start latency objectives

Examples include:

• Checkout
• Payment authorization
• Account access
• Order confirmation
• Time-sensitive fraud decisions

Lower-priority task queues

Background queues can often use:

• Shared worker capacity
• Conservative autoscaling
• Standard resource allocations
• Lower minimum replica counts
• Greater tolerance for queueing delay

Examples include:

• Report generation
• Historical data synchronization
• Nonurgent enrichment
• Batch maintenance
• Low-priority analytics

Priority separation ensures that a surge in background processing does not degrade workflows tied directly to revenue or customer experience.

Scale Temporal Worker Pools Independently

Once workloads are split across task queues, each worker fleet can scale according to its own demand.

A CPU-intensive queue might scale by adding compute-optimized replicas. An I/O queue might first increase concurrency and then add standard worker pods. A GPU queue might scale cautiously because new accelerator capacity is expensive and slower to provision.

Kubernetes supports this model well because each worker pool can be deployed independently with its own:

• Pod specification
• Resource requests and limits
• Node selector
• Minimum and maximum replicas
• Health checks
• Deployment strategy
• Autoscaling policy

Use schedule-to-start latency as a key scaling indicator because it reveals how long tasks wait before a worker begins executing them. CPU alone can miss I/O-bound saturation or worker-slot shortages.

For example:

• Rising latency on document-processing-cpu may require additional CPU workers.
• Rising latency on payment-gateway-io may signal insufficient concurrency or a slow downstream API.
• Rising latency on ml-inference-gpu may require more GPU replicas or stricter request prioritization.

Task-queue-level metrics make it possible to respond to the actual bottleneck rather than scaling the entire worker fleet indiscriminately.

Monitor Every Temporal Task Queue as a Separate Service

Creating specialized task queues without queue-specific monitoring leaves teams with incomplete visibility.

At minimum, track:

• Schedule-to-start latency
• Task queue depth
• Number of active pollers
• Worker health
• CPU and memory consumption
• Activity duration
• Activity failure rates
• Retry volume
• Downstream dependency latency

These metrics should be visible by task queue, not only as platform-wide averages.

A healthy background queue can hide a severe problem in a critical payment queue when metrics are aggregated. Queue-level dashboards and alerts show which worker fleet, workload, and business process require attention.

Common Temporal Task Queue Design Mistakes

Creating a task queue for every Activity

Specialization should follow meaningful execution boundaries. Hundreds of tiny queues create operational complexity without necessarily improving isolation.

Group Activities that share similar hardware, concurrency, priority, and scaling requirements.

Using one queue for the entire application

The opposite extreme forces incompatible work into one worker configuration and makes independent scaling impossible.

Mixing critical and background workloads

Customer-facing processes need capacity protection during traffic spikes. Priority-based queue separation prevents batch work from consuming that capacity.

Copying the same concurrency setting across workers

Concurrency should reflect whether the Activity is CPU-bound, I/O-bound, or accelerator-bound.

Scaling every queue on CPU utilization

Low CPU does not prove that a worker has spare Activity capacity. Schedule-to-start latency, queue depth, poller health, and downstream performance provide essential context.

Sharing GPU infrastructure with standard workloads

Dedicated queues protect expensive accelerator capacity and make utilization easier to understand.

Ignoring downstream service limits

An I/O worker pool can pick up large numbers of tasks while overwhelming the database or API it depends on. Worker concurrency and external capacity must be designed together.

A Practical Temporal Task Queue Design Process

Start by inventorying the Activities in the application.

For each Activity, document:

• Whether it is CPU-, I/O-, or GPU-bound
• Typical and peak duration
• CPU and memory consumption
• Required hardware
• Programming language
• External dependencies
• Business priority
• Acceptable schedule-to-start latency

Next, group Activities with similar execution requirements.

Create dedicated worker pools only where separation creates a clear benefit, such as different concurrency, hardware, priority, language, or autoscaling needs.

Then validate the architecture under realistic conditions. Test steady load, sudden traffic bursts, downstream slowdown, worker restarts, and mixed workload volume. Measure queue-specific latency rather than relying only on total throughput.

Temporal Task Queues Turn Routing Into a Scaling Strategy

Temporal task queues become most valuable when they are treated as intentional architectural boundaries rather than generic channels.

A production-ready design separates workloads so that:

• CPU-intensive Activities receive sufficient compute without excessive concurrency.
• I/O-bound Activities use higher concurrency without wasting CPU.
• GPU workloads run only on specialized infrastructure.
• Language-specific workers can use the best ecosystem for each task.
• Critical workflows retain capacity during traffic spikes.
• Every worker pool can scale and be monitored independently.

This structure improves performance, cost control, fault isolation, and operational clarity. It also prevents teams from responding to every slowdown by scaling an undifferentiated fleet.

Xgrid helps teams design production Temporal architectures around real workload behavior, including task queue boundaries, worker concurrency, Kubernetes deployment, autoscaling, observability, and priority isolation.

Talk to Xgrid about designing scalable Temporal task queues and worker pools →

FAQ: Temporal Task Queues

What is a Temporal task queue?

A Temporal task queue routes Workflow Tasks or Activity Tasks to workers polling that queue. It allows teams to control which worker fleet executes a particular category of work.

Should every Temporal Activity have its own task queue?

Usually not. Activities should be grouped when they share similar hardware, concurrency, scaling, language, and priority requirements. Creating too many queues can increase operational complexity.

Why separate CPU- and I/O-bound Activities?

CPU-bound Activities require controlled concurrency and sufficient compute. I/O-bound Activities spend more time waiting and can usually support higher concurrency. Separating them allows each worker pool to be tuned appropriately.

How should GPU workloads be handled in Temporal?

GPU Activities should generally use dedicated task queues and worker fleets running on GPU-enabled infrastructure. Concurrency should be limited according to accelerator memory and workload characteristics.

Can one Temporal Workflow use multiple task queues?

Yes. A Workflow can schedule different Activities on different task queues. This allows one business process to coordinate CPU, I/O, GPU, or language-specific work across specialized worker pools.