Can a processor access memory on another NUMA node?

Usually yes in the shared-memory NUMA systems discussed here, but the path and access cost differ from local memory.

Does NUMA mean every memory access is slow?

No. Many requests are served by caches, and local main-memory accesses avoid the remote path.

Should every application manually pin its threads?

Not necessarily. Operating systems and runtimes already make placement decisions. Manual changes should address measured behavior and be tested with the relevant workload.

AI Glossary / Data engineering and architecture / Intermediate

Non-uniform memory access

Definition

Non-uniform memory access, or NUMA, is a computer architecture in which the time required to access main memory depends partly on where that memory is located relative to the processor making the request. Memory associated with a nearby processor or node is usually faster to access than memory reached through an interconnect to another node.

What it means

NUMA systems make memory locality visible in performance. A program may be able to address memory across the machine, but all addresses are not equally close to the processor executing its work. Uniform-memory architectures instead provide broadly similar main-memory access costs across processors.

A NUMA node groups processors and associated memory resources. Topology depends on hardware; it is not safe to assume that one physical socket always means exactly one node. The operating system exposes topology information for planning execution and allocation.

How it works

When a processor requests data, caches may satisfy the request before main memory is involved. If main memory must be accessed, the location of the relevant memory affects the path taken. A remote access crosses the system interconnect to reach memory associated with another node.

The operating system schedules threads and manages memory allocation. Applications and runtime settings can influence those choices, for example by keeping workers close to the memory they use. Microsoft documents how local allocation can be preferred while allocation elsewhere may be necessary when local resources are insufficient.

Locality is not the only objective. Concentrating all work on one node can exhaust its memory bandwidth or leave other processors idle. Effective placement balances access cost, capacity, contention and the actual workload rather than blindly pinning everything to a single location.

Three practical examples

1. In-memory analytics: Workers scan different partitions of a large dataset. Placing each partition near the workers that read it can reduce remote memory traffic, provided the partitioning also distributes work reasonably.

2. AI preprocessing: Several workers transform training records before an accelerator consumes them. Poor placement can make workers repeatedly fetch remote memory even though the machine has spare local resources elsewhere.

3. Database service: A large cache spans multiple NUMA nodes. The database's allocation and scheduling policies influence which processors access which portions and how heavily the interconnect is used.

Think of it this way

Think of two workshops, each with its own storeroom. A worker can retrieve supplies from either storeroom, but visiting the other workshop takes longer. Keeping frequently used supplies nearby can save travel.

The analogy has limits. Computer caches may satisfy many requests without a main-memory trip, and multiple requests can overlap. A workshop picture also understates shared bandwidth and coherence mechanisms. It explains locality, not exact application speed.

Worked example

Suppose a fictional system has two NUMA nodes. Assume a main-memory access costs 80 nanoseconds locally and 140 nanoseconds remotely. A worker makes 1,000 accesses that reach main memory, with 750 local and 250 remote.

The summed access cost is 750 × 80 + 250 × 140 = 95,000 nanoseconds, giving an average of 95 nanoseconds per access. If the same accesses were all local, the sum would be 80,000 nanoseconds. Local placement removes 15,000 nanoseconds from this simplified accounting.

That is not a prediction of a 15,000-nanosecond reduction in application runtime. Real processors overlap requests, caches alter which requests reach memory and computation may dominate. Moving memory or workers also has costs. The calculation isolates one factor so the team can identify what to measure rather than promise an unsupported speedup.

Illustrative case study

Fictional case study: LocalMeadow runs a parallel data transformation on a two-node server. One initialization thread allocates and fills most of the data near its own node. Later, workers on both nodes process the dataset, and workers on the second node frequently access remote memory.

The team tests an alternative arrangement in which each worker initializes and processes its assigned partition under an explicitly checked allocation policy. It measures local and remote traffic, throughput and utilization for the full job.

Some partitions are much larger than others, so the first locality-focused version leaves workers unevenly loaded. LocalMeadow revises the partitioning and repeats the relevant comparison. The case shows that reducing remote access and balancing work are complementary design concerns, not interchangeable goals.

Common misconceptions and mistakes

A common mistake is reading total installed RAM as though every processor can access all of it at the same cost. Another is assuming that processor affinity alone guarantees local memory: the data may already reside elsewhere.

Do not equate NUMA with distributed computers that require explicit network messaging for every shared result. Also avoid assuming that remote accesses are always disastrous. Their effect depends on frequency, bandwidth, caching and overlap. A workload with little memory pressure may gain little from detailed placement changes, while a bandwidth-intensive workload may need careful measurement.

Business use

NUMA awareness can help organizations use large servers effectively for analytics, databases and AI support workloads. It explains why simply increasing thread counts may fail to improve throughput and why the same workload can behave differently after a deployment or scheduling change.

Teams should inspect the machine's actual topology, record memory and thread placement, and test representative workloads. Useful evidence includes throughput, remote memory traffic, node-level memory pressure and load balance. Compare the whole job, including initialization and data movement. The goal is a dependable improvement in service or processing time, not a lower remote-access counter at any cost.

Check your understanding

A thread is moved to node B, but most of its data remains on node A. Does moving the thread alone guarantee better memory locality?

Answer: No. Locality depends on both execution and data placement. The move may increase remote accesses unless the broader arrangement is appropriate.

Related terms

Shared memory
Cache coherence
Distributed memory
Memory locality
processor affinity
memory bandwidth
multiprocessor architecture
thread scheduling

Learn next

Inspect a server's NUMA topology and compare where a workload runs with where its memory is allocated. Form one placement hypothesis, then measure the complete workload before and after a controlled change.

Sources

Microsoft, “NUMA Support”: https://learn.microsoft.com/en-us/windows/win32/procthread/numa-support

Lawrence Livermore National Laboratory, introduction to parallel computing, shared-memory architectures: https://hpc.llnl.gov/documentation/tutorials/introduction-parallel-computing-tutorial

These sources support the locality model. Access times and LocalMeadow are fictional teaching examples; exact allocation policies and topology depend on the system.

Previous: Non-terminal symbols Next: Noncentered parameterization