Can a DAG contain parallel tasks?

Yes. Tasks without unmet dependencies between them can be candidates for parallel execution.

Does a daily repeated workflow violate acyclicity?

No. Repeating separate runs does not create a cycle within the dependency graph of one run.

Are all DAG tools interchangeable?

No. They can differ in task granularity, scheduling, state handling, and failure semantics.

AI Glossary / MLOps and deployment / Intermediate

DAG Processing Model

Definition

A DAG processing model represents a computation or workflow as a directed acyclic graph: nodes describe work, directed edges describe dependencies, and no dependency path loops back to its starting node. This structure makes it possible to identify a valid execution order and which independent tasks may be eligible to run in parallel.

What it means

Directed means that a dependency has a direction: one task must provide something or satisfy a condition before another proceeds. Acyclic means that following these directed relationships cannot bring you back to the same node.
For an AI pipeline, nodes might represent data preparation, validation, model training, and evaluation. The graph states their relationships, while an execution system supplies scheduling, resource management, status tracking, and other behavior.
The term can describe different levels of work. A data-processing engine may use a graph of computational operations, while a workflow orchestrator coordinates larger tasks. A diagram that looks similar does not mean the systems use identical execution or recovery semantics.

How it works

First define the tasks and their required inputs or prerequisite conditions. Add an edge for each real dependency, then check that these edges do not create a directed cycle.
A scheduler can identify tasks whose prerequisites are satisfied and submit them for execution when resources are available. Completion or failure changes which downstream tasks are eligible. The system's trigger rules determine whether a task requires upstream success or some other condition.
Retries, timeouts, persistence, and output tracking are additional execution features. The DAG structure helps express the relationships needed to use those features, but cannot guarantee them by itself. Tasks that may be retried need an output design that accounts for repeated execution.

Three practical examples

A training pipeline can require both feature preparation and data-quality checks to finish before model training begins.
A reporting workflow can compute independent regional summaries in parallel and combine them only after the required summaries are available.
A batch prediction workflow can separate extraction, transformation, inference, and publication so that failures can be associated with a specific stage and its dependants.

Think of it this way

Think of preparing a meal with several dishes. Chopping ingredients precedes cooking, while setting the table can happen independently. Serving waits for the required dishes. The analogy captures dependencies and parallel opportunities. A real workflow also needs machine-readable state, failure handling, and explicit rules for whether incomplete or failed branches permit later work.

Worked example

Consider an invented five-task pipeline. Task A ingests data in two minutes. After A, task B prepares features in four minutes and task C checks data quality in three minutes. Task D trains a model in five minutes, requiring both B and C to succeed. Task E evaluates the model in one minute after D.
The dependency edges are A to B, A to C, B to D, C to D, and D to E. Assume sufficient resources, fixed task durations, no failures, and negligible scheduling overhead.
A finishes at minute 2. B and C then run together: B finishes at minute 6 and C at minute 5. D starts at minute 6 and finishes at minute 11. E completes at minute 12.
The elapsed time is therefore 2 + max(4, 3) + 5 + 1 = 12 minutes. Executing all five tasks sequentially would take 2 + 4 + 3 + 5 + 1 = 15 minutes under the same simplified duration assumptions.
Now suppose only one worker is available. The graph still permits B and C to overlap logically, but the available resources may force sequential execution. If C fails, the assumed all-success rule blocks D even if B finishes. Neither concurrency nor successful completion follows from the graph shape alone.

Illustrative case study

In a fictional forecasting project, a pipeline retrains a model after every daily data import. Initially, its diagram shows training depending on feature generation but omits a dependency on data validation.
A review finds that training can begin while validation is still running. The team adds the missing edge and tests both successful and failed validation runs. It also records each run's input dataset so that a later investigation can connect outputs to the intended data.
The lesson is that a visually tidy DAG can still encode the wrong process. Correct dependencies and meaningful task outcomes matter more than the appearance of the diagram.

Common misconceptions and mistakes

Do not assume every task must run sequentially. Independent branches may run together when resources and scheduling policies permit.
Do not assume a graph automatically provides fault tolerance or complete lineage. Those require implemented state, metadata, and recovery behavior.
Do not add a backward edge to represent unlimited repetition and still call the resulting graph acyclic. Iterative work may happen inside a task, across separate workflow runs, or through a finite unrolling that preserves acyclicity.
Do not treat retries as harmless. Repeating a task that appends output or sends a message can duplicate effects unless the task and surrounding system account for that possibility.

Business use

A DAG makes operational dependencies reviewable. Teams can ask which stage delays completion, which failure blocks publication, and which outputs require rerunning after an input changes.
For planning, compare the longest dependent sequence with resource constraints and measured durations. For reliability, test failure paths, identify output ownership, and define what successful completion means. The resulting workflow should make the business process easier to inspect, not merely divide one opaque program into many smaller opaque tasks.

Check your understanding

Feature preparation and validation both follow ingestion, but training depends only on feature preparation. Must training wait for validation?

Answer: The stated graph does not require it. If validation is a prerequisite, that dependency must be represented and enforced.

Related terms

Directed acyclic graph
Critical path
workflow orchestration
task dependency
topological ordering
idempotency
data lineage
and pipeline

Learn next

Draw a small pipeline and label every dependency with its reason. Calculate its ideal elapsed time, then add one worker limit and one failed task. Explain how those changes affect eligibility, execution, and publication.

Sources

Sources: Hopsworks dictionary, DAG processing model: https://www.hopsworks.ai/dictionary/dag-processing-model Apache Airflow documentation, Dags, dependencies, runs, and control flow: https://airflow.apache.org/docs/apache-airflow/stable/core-concepts/dags.html

Previous: D-Separation Next: DAgger