AI Glossary / Data engineering and architecture / Advanced

Partitioned global address space

Definition

Partitioned global address space, or PGAS, is a parallel programming model that provides a global view of addressable data while retaining a distinction between portions associated with different processing resources. Data can be globally accessible without being equally close or equally cheap to access.

What it means

The name combines two ideas. Global means that the programming abstraction can identify data beyond one local memory region. Partitioned means that placement still matters: some data is associated with one process, locale or processing element, and other data is associated elsewhere.

This helps programmers express an operation on a distributed structure while reasoning about where the work should occur. It does not turn a cluster into a physically uniform memory machine. A remote reference may require network communication even when the source code looks compact.

PGAS is a family of approaches rather than one language or one consistency rule. A language may provide a global namespace; a library may provide operations on distributed memory. Details of allocation, visibility and completion belong to the particular system.

How it works

A program first establishes its participating resources and distributes data among them. Computation can then be arranged to operate mainly on nearby portions, with explicit or implicit communication when it needs another portion.

Chapel illustrates the idea using locales: a programmer can reason about the locations of variables and tasks while referring to lexically visible variables across locales. OpenSHMEM illustrates a library approach with communication operations for a PGAS environment. These are examples, not interchangeable APIs.

Correct execution still requires coordination. An update being initiated does not necessarily mean every dependent operation may immediately consume its result. The relevant completion and ordering rules must be followed.

Performance depends on the access pattern as well as the amount of data. A loop making many tiny remote requests can behave differently from one that processes local blocks and combines a small number of results.

Three practical examples

• A simulation assigns spatial blocks to different processing resources and exchanges the boundary information needed for the next calculation.
• A distributed array calculation processes each local portion before combining partial totals into a final result.
• A graph analysis places vertex information across resources and must consider the remote accesses caused by edges crossing those placements.

Think of it this way

Think of a research team whose catalog lists books held in several offices. Everyone can identify a book through the shared catalog, but retrieving a book from the next building is different from reaching for one on the nearby shelf.

The analogy describes visibility and locality. It does not define software permissions, memory consistency or the time a particular network operation will take. A global reference alone says little about those details.

Worked example

Counting local and remote reads

An invented array has twelve elements. Elements one through four are placed with resource A, five through eight with B, and nine through twelve with C. Each element is read once to compute a sum.

If A performs all twelve reads, four target its local portion and eight target remote portions. If each resource sums its own four elements, the initial element reads are all local. The three partial totals must still be combined, so communication has not disappeared.

This is a locality illustration, not a runtime forecast. Twelve small reads might be handled differently by different implementations, and creating tasks has a cost. The useful question is whether a placement and execution plan reduce expensive communication for the real workload while preserving the required result.

Illustrative case study

A fictional geospatial analytics team adopts a PGAS approach for a large raster calculation. Its first implementation lets every task request neighboring values one at a time, including values stored on other resources. The global view makes the code understandable, but remote request overhead becomes prominent.

The team reorganizes work around local tiles and handles the required boundary exchange in larger units. It compares the output with a trusted small reference calculation and measures communication alongside total runtime.

The lesson is that a convenient global abstraction does not remove locality from engineering decisions. The team documents where data resides and how dependencies are satisfied rather than describing remote memory as free.

Common misconceptions and mistakes

• Global addressability makes every access local. The model exposes or retains locality precisely because access costs can differ.
• PGAS automatically prevents races. Competing updates still need the coordination required by the chosen language or library.
• Every PGAS system has identical semantics. Allocation, completion, consistency and execution facilities vary across implementations.

Business use

PGAS is relevant to organizations developing demanding scientific, engineering or data-analysis applications. It offers one way to express computations spanning multiple memory domains without abandoning a global data perspective.

For an evaluation, describe the workload's data distribution, communication pattern and update dependencies. Compare equivalent outputs and problem sizes, not only attractive source-code examples. A simple distributed sum does not represent a graph workload with irregular remote access.

Maintainability also matters. Developers need to understand the abstraction's locality and ordering rules, and operational teams need visibility into communication and resource use. AI-generated code can compile while placing work badly or omitting a necessary completion step. Review those properties explicitly.

People also ask

Is PGAS the same as ordinary shared memory?

No. It provides a global abstraction while retaining partitioning and locality; remote access may involve communication.

Does PGAS eliminate message movement?

No. Data still moves between resources when needed, even if the programming interface hides some mechanics.

Must PGAS be built into a language?

No. Both languages and libraries can support PGAS approaches, with different facilities and obligations.

Check your understanding

In the twelve-element example, does performing the initial reads locally mean that the complete sum requires no communication?

Answer: No. Each resource obtains a local partial result, but those partial results must be combined. The change reduces remote element access rather than eliminating coordination altogether.

Related terms

• Parallel programming model
• Distributed memory
• Shared memory
• Locale (computer hardware)
• data locality
• locale
• one-sided communication
• memory consistency
• distributed array

Learn next

Read data locality and synchronization. Draw the placement of a small distributed array, mark every remote dependency and compare a central loop with local processing followed by a reduction.

More in Data engineering and architecture

• Parareal - Parareal is a numerical method that parallelizes the solution of time-dependent problems across several time intervals.
• Partial Dependency - A partial dependency is a functional dependency in which an attribute depends on a proper subset of a larger determinant, commonly part of a composite...
• Partition - In data engineering, a partition is a defined subdivision of a larger dataset or storage structure.
• Partitioning strategy - A partitioning strategy is the plan for dividing a dataset into manageable parts and maintaining those parts over time.
• Percept schema - A percept schema, in AI planning, is a structured description of what an agent can observe and under which conditions that observation becomes available.
• Personal cloud - A personal cloud is a collection of remotely accessible storage and related services organized around an individual's information and devices.

Sources

Chapel documentation, Locales primer: https://chapel-lang.org/docs/primers/locales.html. Primary explanation of the global namespace and reasoning about locality.

Chapel language specification, Locales: https://chapel-lang.org/docs/language/spec/locales.html. Primary description of processing and storage resources.

Oak Ridge National Laboratory, Introducing OpenSHMEM: SHMEM for the PGAS Community: https://impact.ornl.gov/en/publications/introducing-openshmem-shmem-for-the-pgas-community/. Primary research abstract for the library approach. The array arithmetic and geospatial case are original illustrations.

Previous: Partition Next: Partitioning strategy