AI Glossary / Data governance and quality / Intermediate

Data lifecycle

Definition

The data lifecycle is the sequence of activities through which data is planned, obtained, processed, used, maintained, shared, preserved, and eventually retired or disposed of where appropriate. Different frameworks divide these activities differently. The lifecycle is not necessarily a straight line: data may be corrected, reprocessed, reused, or moved between stages. Governance, documentation, quality, and security need attention across the lifecycle rather than only at collection or deletion.

What it means

A dataset has a history and an intended future. It may begin with a collection plan, become a cleaned analytical file, support a report or model, and later be archived for evidence or removed under a retention decision. Each transition can change its meaning, risks, or obligations. Looking at the lifecycle helps teams identify who is responsible at each point and what information must accompany the data when it moves.

How it works

Define the purpose and management plan before collection. Acquire data with appropriate provenance and permissions, then document transformations and quality checks. Use and share it within its established scope, maintaining versions and access controls. Preserve the data and context needed for continuing use, or apply an authorized disposition process when retention ends. Review dependencies such as backups, derived datasets, and external copies. These activities can overlap; a newly discovered error may send a published dataset back through processing and review.

Three practical examples

• Survey project: responses are planned, collected, checked, analyzed, and released with documentation while restricted source material follows its own retention rules.
• Customer operations: records are created, updated, used, and later reviewed for archival or deletion according to the organization's requirements.
• AI development: raw examples become labeled and versioned datasets, then training inputs and evaluation evidence, with traceability maintained between stages.

Think of it this way

Think of a document's journey from drafting through review, use, archiving, and eventual disposal. Different people handle it at different points, but its identity and meaning must remain understandable. The analogy is limited because data can branch into many copies and derived products, so there may be several concurrent lifecycles rather than one physical document moving along a single path.

Worked example

A fictional AI project acquires 2,000 support examples as dataset version V1. After review, 100 are excluded and the remaining 1,900 become a documented training candidate set. A later correction affects 20 source examples. Lifecycle management asks which labeled versions, training runs, evaluations, and stored copies depend on those examples. Simply replacing the current file does not update the history or establish whether a model must be rebuilt. The team records the correction, creates a new version, and follows its defined impact-review process. These numbers illustrate traceability and branching; they do not prescribe a universal rule about deleting or retraining models.

Illustrative case study

Fictional example: Cedar Research finishes a project but discovers that only the final chart has an assigned owner. The source data, processing scripts, and interpretation notes remain in temporary locations. The team creates a handover that identifies the preserved artifacts, their context, and the responsible custodian. This invented case demonstrates that project completion and data-lifecycle completion are different events, without imposing a particular retention duration.

Common misconceptions and mistakes

Do not assume the lifecycle ends when data is first used or published. Do not confuse it with a backup schedule or with the lifecycle of the application that stores it. Deleting a working copy does not necessarily remove archives, backups, or downstream derivatives. Also, preservation is more than retaining bits: context, formats, and access methods may be needed. Lifecycle stages vary across frameworks, so the value lies in covering responsibilities and transitions rather than memorizing one fixed list of stage names.

Business use

Lifecycle thinking reduces abandoned datasets, unexplained transformations, and unclear retirement decisions. For AI projects, it supports reproducibility and makes changes to source data easier to trace through derived artifacts. Establish ownership, review points, and evidence for important transitions. Align preservation and disposal with actual business, contractual, and legal requirements for the context. A generic lifecycle diagram does not decide those requirements, and a policy is useful only if systems and people can carry it out.

People also ask

Must data pass through every stage once in order?

No. Activities can overlap, repeat, and branch as data is corrected or reused.

Is archiving the same as backing up?

No. Archives support longer-term preservation and access goals, while backups primarily support recovery from loss or failure.

Does deleting a source file automatically resolve derived-data obligations?

No. Copies and derivatives need explicit consideration under the relevant policies and requirements.

Check your understanding

A dataset is published, then a labeling error is discovered. Is its lifecycle necessarily finished because publication already occurred?

Answer: No. Correction, versioning, notification, and impact review may be needed, and the lifecycle can return to processing and validation activities.

Related terms

• Data management plan
• Data collection
• Backup
• Data governance
• provenance
• lineage
• processing
• versioning
• data sharing
• preservation
• retention
• disposition
• archive

Learn next

map one dataset from its original source to its current consumers and retained copies. Identify the owner, evidence, and decision rule at each transition. Add a correction scenario and an end-of-retention scenario, checking whether the map covers derived outputs and dependencies rather than only the main file.

More in Data governance and quality

• Data entry - Data entry is the process of recording information in a computer system so it can be stored, processed, and used.
• Data exfiltration - Data exfiltration is the unauthorized transfer of information out of a system or controlled environment.
• Data integrity - Data integrity is the condition in which data remains consistent with its intended content and applicable rules, and is protected against improper...
• Data minimization principle - The data minimization principle requires personal data to be sufficient and relevant for a defined purpose while limiting it to what that purpose needs.
• Data Quality - Data quality is the degree to which data meets the requirements of its intended use.
• Data quality dimensions - Data quality dimensions are distinct aspects used to describe and assess data quality.

Sources

Sources: U.S. Geological Survey, Data Lifecycle, https://www.usgs.gov/data-management/data-lifecycle ; U.S. Geological Survey, The United States Geological Survey Science Data Lifecycle Model, https://pubs.usgs.gov/of/2013/1265/pdf/of2013-1265.pdf ; U.S. Geological Survey, Process, https://www.usgs.gov/data-management/process ; UK Government, Data Management, https://www.data.gov.uk/data-manual/data-management/ . Primary lifecycle and management-guidance excerpts consulted. USGS provides one research-oriented model, not a universal mandatory sequence; examples and retention scenarios are original and context-dependent.

Previous: Data leakage Next: Data loading