Temporal Worker Scaling: Why Schedule-to-Start Latency Matters More Than CPU

Temporal workflows can perform well in development and still slow dramatically after moving into production.

At first, the symptoms may look like a conventional infrastructure problem. Workflows take longer to complete. Activities appear delayed. Timeouts increase. Engineers inspect CPU, memory, network usage, and database performance. When nothing looks obviously wrong, the team adds more server capacity.

Yet workflow performance does not improve.

The underlying problem may have nothing to do with the Temporal Server or the amount of CPU available across the platform. Tasks may simply be waiting too long for an available worker to begin executing them.

That waiting time is measured through schedule-to-start latency, one of the most valuable signals for diagnosing and improving Temporal worker scaling.

CPU utilization tells you how busy a machine is. Schedule-to-start latency tells you whether the execution fleet can keep up with actual workflow demand. For teams operating Temporal in production, that distinction determines whether scaling decisions solve the real bottleneck or merely add cost.

Why Temporal Worker Scaling Fails in Production

A common Temporal production pattern begins with a successful proof of concept.

The initial workload is small, worker configurations are simple, and task queues have plenty of spare capacity. Workflows are picked up almost immediately, so performance appears healthy.

As adoption grows, more workflows begin sharing the same worker fleet. Activity volume increases. Traffic becomes uneven. Some workloads make fast API calls, while others perform CPU-intensive processing or wait on slow downstream services.

Eventually, tasks begin accumulating faster than workers can process them.

The workflows themselves remain durable. The Temporal Server continues scheduling tasks and preserving workflow state. But execution slows because the available workers cannot pick up tasks quickly enough.

One example in Xgrid’s production guide describes schedule-to-start latency rising from roughly 50 milliseconds to more than five seconds. The problem was traced to worker concurrency settings that did not match the workload. Once corrected, the degradation was resolved without redesigning the entire platform.

This is why worker scaling problems are so easy to misdiagnose. The platform may be functioning correctly while users still experience slow workflows.

How Temporal Server and Worker Capacity Differ

Understanding Temporal worker architecture begins with separating orchestration capacity from execution capacity.

The Temporal Server coordinates workflow execution

The Temporal Server maintains workflow state, persists event histories, schedules tasks, manages task queues, and coordinates execution timing.

It does not run the application’s business logic.

Whether the organization uses Temporal Cloud or a self-hosted deployment, the service acts as the orchestration layer. It decides what work needs to happen and makes that work available through task queues.

Temporal workers execute application code

Workers are separate application processes that poll task queues and execute Workflow Tasks or Activities.

Activities may include:

• Calling an external API
• Querying a database
• Processing a document
• Charging a payment method
• Sending a notification
• Running machine-learning inference
• Transforming or validating data

Workers provide the execution capacity required to turn scheduled tasks into completed work.

This separation is a central strength of Temporal. Workers can scale independently from the service that stores workflow state and coordinates execution. But it also means that increasing Temporal Server capacity will not fix a shortage of workers.

A deployment can have healthy server infrastructure and still experience severe delays because its execution fleet is undersized or incorrectly configured.

What Schedule-to-Start Latency Measures

Schedule-to-start latency measures the time between a task becoming available and a worker beginning to execute it.

In practical terms, it answers this question:

Once Temporal scheduled the task, how long did it wait before a worker was ready to pick it up?

A low value usually indicates that worker capacity is keeping pace with demand. Tasks are made available and collected quickly.

A rising value suggests that tasks are waiting in the queue because workers are unavailable, saturated, incorrectly configured, or not polling the right task queue.

• Below 100 milliseconds: worker capacity is generally keeping up with demand.
• Above 500 milliseconds: tasks may be queuing because execution capacity is insufficient.

These are not universal service-level objectives. A suitable threshold depends on the workflow’s business requirements and number of execution steps. A latency that is harmless for a background batch workflow may be unacceptable for a customer-facing payment flow.

Why CPU Utilization Can Hide Temporal Worker Bottlenecks

CPU remains useful, but it answers a different question.

CPU tells you how much computation a process or machine is currently performing. It does not directly tell you:

• How many tasks are waiting
• Whether enough workers are polling
• Whether task queues are configured correctly
• Whether worker concurrency matches the workload
• Whether one workload is blocking another
• Whether tasks are routed to the appropriate execution fleet

A worker can have low CPU usage while still being capacity-constrained.

For example, a worker handling external API calls may spend most of its time waiting on network responses. CPU usage remains low, but all available Activity slots may already be occupied. New tasks sit in the queue even though infrastructure dashboards show significant unused CPU.

The opposite can also happen. A CPU-intensive worker may operate near full CPU utilization while maintaining low schedule-to-start latency. If tasks are picked up promptly and throughput meets the required service level, high CPU usage may reflect efficient resource utilization rather than a scaling emergency.

The more useful interpretation is:

No single metric should operate in isolation. But for identifying whether worker capacity matches task demand, schedule-to-start latency is more direct than CPU.

Set Temporal Scaling Thresholds Around Workflow SLAs

The right latency threshold depends on the workflow’s end-to-end service-level objective.

Suppose a customer-facing workflow must complete within five seconds and runs ten Activities sequentially. An additional 500 milliseconds of queueing before each Activity could consume the entire five-second budget before the Activities perform any useful work.

For a background data synchronization that can finish within 30 minutes, the same delay may be insignificant.

Start with the required workflow completion time, then work backward:

1. Determine the end-to-end workflow SLA.
2. Estimate how many Tasks or Activities occur on the critical path.
3. Measure the normal duration of the Activities themselves.
4. Calculate how much queueing delay the workflow can tolerate.
5. Set warning and scaling thresholds below the point where users are affected.

The guide recommends example alerting thresholds such as a warning at 200 milliseconds sustained for two minutes, a critical alert at 500 milliseconds sustained for one minute, and a scaling trigger around 300 milliseconds. These should be treated as starting points and adjusted to actual workflow behavior.

Tune Temporal Worker Concurrency by Workload Type

Adding worker replicas is not the only scaling decision. Each worker also has concurrency limits that determine how much work it can execute simultaneously.

The right configuration depends on what the Activities do.

CPU-bound Temporal Activities

CPU-bound Activities spend most of their execution time performing computation.

Examples include:

• Image processing
• Encryption
• Data aggregation
• Complex calculations
• Compression
• Large data transformations

For these workloads, concurrency should generally remain close to available CPU capacity. Running substantially more concurrent Activities than available cores can increase context switching without improving throughput.

The better pattern is often:

• Higher CPU allocation
• Controlled concurrency
• Fewer, appropriately sized worker instances
• Dedicated task queues for computational workloads

I/O-bound Temporal Activities

I/O-bound Activities spend significant time waiting.

Examples include:

• API calls
• Database requests
• File operations
• Network requests
• Calls to payment or identity providers

Because one Activity can wait while another runs, these workers can often support concurrency beyond the number of CPU cores.

Concurrency around two to five times the CPU core count may provide a useful starting range for I/O-bound workloads. The correct value still depends on SDK behavior, connection pools, downstream rate limits, memory, and measured latency.

Increasing concurrency too aggressively can transfer the bottleneck downstream. A worker may pick up hundreds of tasks only to overwhelm a database or trigger external API rate limits.

GPU-bound Temporal Activities

GPU workloads require a different strategy.

Examples include:

• Machine-learning inference
• Video processing
• Scientific computing
• Specialized model execution

GPU memory and accelerator capacity are limited. Excessive concurrency can create memory contention and reduce performance. 

These workloads should typically use dedicated task queues and worker fleets running on appropriate hardware.

Separate Temporal Task Queues by Execution Profile

A single worker pool should not be expected to execute every workload efficiently.

Mixing CPU-heavy jobs, fast API calls, long-running requests, and GPU inference in one queue creates resource competition. The concurrency settings that work for one category may be wrong for another.

A stronger Temporal worker architecture separates task queues by execution characteristics.

For example:

• payment-api for I/O-bound payment calls
• document-processing for CPU-intensive transformations
• ml-inference for GPU-backed model execution
• customer-critical for high-priority transactions
• background-sync for lower-priority batch work

This enables each worker fleet to have its own:

• Instance type
• CPU and memory allocation
• Concurrency configuration
• Minimum replica count
• Autoscaling policy
• Cost profile
• Priority level

Task queue specialization also prevents business-critical workflows from competing with background work. The production guide recommends guaranteed minimum capacity and more aggressive scaling for high-priority queues, while lower-priority queues can tolerate longer schedule-to-start latency.

Use Kubernetes Autoscaling for Temporal Workers

Kubernetes is well suited to Temporal worker deployment because workers are stateless execution processes that can scale horizontally.

Running workers as pods provides:

• Automatic restart after crashes
• Declarative resource requests and limits
• Rolling deployments
• Health checks
• Replica management
• Access to specialized node pools
• Horizontal autoscaling

However, a default Horizontal Pod Autoscaler based only on CPU may repeat the same diagnostic mistake discussed earlier.

For many Temporal workloads, the better scaling signal is schedule-to-start latency.

An illustrative scaling policy might:

• Scale up when latency stays above the chosen SLA threshold
• Scale down only after latency remains low for a longer period
• Maintain minimum replicas for availability
• Cap maximum replicas to control cost and downstream pressure
• Use stabilization windows to prevent rapid scaling changes

The whitepaper proposes scaling up when schedule-to-start latency exceeds 500 milliseconds for two minutes and scaling down after it remains below 100 milliseconds for ten minutes. These values should be validated against the application’s own SLAs rather than copied blindly.

Why scale-down should be slower

Worker demand can change quickly. Scaling down immediately after a short quiet period risks removing capacity just before another burst.

A longer scale-down window helps prevent oscillation and gives the system time to confirm that demand has genuinely decreased.

Preserve minimum worker capacity

Scaling to zero may be appropriate for some noncritical queues, but customer-facing workflows usually need workers available immediately.

Maintaining a minimum number of replicas reduces cold-start delays and protects availability when a new traffic spike begins.

Monitor More Than Schedule-to-Start Latency

Although schedule-to-start latency is the primary worker-capacity signal, production monitoring should include supporting metrics.

Track:

• Schedule-to-start latency by task queue
• Workflow Task and Activity Task latency separately
• Task queue backlog
• Poller availability
• Worker CPU and memory
• Activity execution duration
• Workflow and Activity failure rates
• Retry volume
• Downstream dependency latency
• Rate-limit responses
• Worker restarts and deployment health

The key is to preserve task-queue-level visibility.

A platform-wide average can hide a serious bottleneck. One high-priority payment queue may be delayed while several low-volume queues remain healthy, making the aggregate metric look normal.

Alerts should identify the queue, worker deployment, workload type, and business process affected.

Diagnose High Schedule-to-Start Latency Systematically

When latency rises, adding replicas immediately may help, but it should not replace diagnosis.

Use this sequence:

Confirm that workers are polling

Check whether healthy worker processes are connected to the affected task queue.

A deployment issue, task queue naming mismatch, expired certificate, or network policy can leave tasks waiting even when worker pods appear healthy.

Review worker concurrency

Determine whether all execution slots are occupied.

Low CPU with full Activity concurrency suggests an I/O-bound workload that may support more concurrent executions. High CPU with full concurrency may require more replicas or larger instances.

Inspect task queue design

Check whether unrelated workloads are sharing one queue.

A long-running data-processing Activity may be delaying latency-sensitive customer operations.

Measure downstream dependencies

Workers may be available but remain occupied by slow external systems.

Scaling workers in that situation can increase pressure on the failing dependency. Concurrency limits, rate limiting, or circuit-breaking behavior may be more appropriate.

Check payload and execution design

Large payloads and bloated workflow histories can increase replay and persistence overhead. The worker-scaling problem may be compounded by workflows transporting large documents instead of passing external storage references.

Validate Temporal Worker Scaling Before Production

Worker architecture should be tested under realistic conditions rather than inferred from development traffic.

A useful load test should include:

• Expected steady-state volume
• Peak traffic
• Sudden bursts
• Slow downstream APIs
• Worker restarts
• Rolling deployments
• Mixed CPU and I/O workloads
• Queue prioritization
• Maximum expected Activity duration
• Autoscaling response time

Measure not only total throughput but also schedule-to-start latency at different percentiles.

A system may maintain acceptable average performance while its P99 latency grows enough to harm critical workflows.

For teams moving Temporal from proof of concept to production, Xgrid helps evaluate task queue design, worker concurrency, scaling metrics, Kubernetes deployment patterns, and failure behavior against real workload characteristics. Its Temporal Production Deployment Checklist can also help teams identify worker-capacity and observability gaps before those gaps become production incidents.

Review your Temporal worker architecture with Xgrid →

Common Temporal Worker Scaling Mistakes

Scaling the Temporal Server instead of workers

If tasks are waiting for execution capacity, increasing orchestration infrastructure does not solve the shortage.

Autoscaling only on CPU

CPU can miss queues blocked on I/O or workers that have exhausted concurrency slots.

Using one task queue for every workload

A single queue forces incompatible workloads to compete for the same worker resources and scaling policy.

Setting concurrency without measuring the workload

A copied concurrency value may be too high for CPU-intensive work and too low for I/O-bound work.

Ignoring downstream capacity

Adding workers can overwhelm databases, APIs, and rate-limited services unless their limits are included in the design.

Scaling down too aggressively

Rapid scale-down can create repeated cold starts and latency spikes during uneven traffic.

Monitoring only global averages

Healthy low-volume queues can hide a critical queue whose latency is rising rapidly.

Temporal Worker Scaling Should Follow Demand, Not Assumptions

Effective Temporal worker scaling is not about adding as much compute as possible. It is about ensuring that the right workers can begin the right tasks within the time required by the business process.

CPU utilization helps describe resource consumption. Schedule-to-start latency reveals whether scheduled work is actually reaching execution quickly enough.

The strongest production architecture combines:

• Schedule-to-start latency tied to workflow SLAs
• Workload-aware concurrency
• Separate task queues for different execution profiles
• Kubernetes autoscaling based on demand
• Supporting worker, queue, and dependency metrics
• Load tests that reflect real production behavior

When these controls are designed together, traffic spikes become manageable capacity events rather than customer-facing incidents. The whitepaper describes an environment where worker capacity scaled automatically during a sudden 50× increase in workflow volume, with the spike appearing in internal metrics before it produced customer complaints.

Xgrid helps teams turn this model into a production-ready Temporal worker architecture, from task queue and concurrency design to Kubernetes scaling, monitoring, and operational validation.

FAQ: Temporal Worker Scaling

What is a Temporal worker?

A Temporal worker is an application process that polls a task queue and executes Workflow Tasks or Activities. The Temporal Server coordinates and persists workflow state, while workers run the application’s business logic.

What is schedule-to-start latency in Temporal?

Schedule-to-start latency is the time between a task being scheduled and a worker beginning to execute it. Rising latency commonly indicates that workers are unavailable or lack enough capacity.

Why is CPU usage not enough for Temporal autoscaling?

CPU measures computational utilization, but workers may be saturated by I/O-bound Activities while CPU remains low. Schedule-to-start latency more directly shows whether tasks are waiting for execution capacity.

What causes high schedule-to-start latency?

Common causes include insufficient worker replicas, low concurrency, workers not polling the correct queue, slow downstream services occupying execution slots, mixed workloads, and unhealthy worker deployments.

How should concurrency be configured for Temporal workers?

Concurrency should match the workload. CPU-bound Activities usually need lower concurrency aligned with available cores, while I/O-bound Activities can often support higher concurrency. GPU workloads generally require tightly constrained parallel execution.