Can data be high quality for one use and poor for another?

Yes. Requirements differ by task, time, population, and consequence.

Does removing duplicates guarantee quality?

No. It addresses one kind of issue and can itself remove valid records if identity rules are wrong.

Can quality be measured automatically?

Some properties can, but factual accuracy and fitness for purpose often require external evidence and domain judgment.

AI Glossary / Data governance and quality / Beginner

Data Quality

Definition

Data quality is the degree to which data meets the requirements of its intended use. It is commonly assessed through properties such as accuracy, completeness, consistency, validity, uniqueness, and timeliness. Quality is contextual: a dataset suitable for a broad historical trend may be inadequate for a current operational decision. It is not simply the absence of blank cells or formatting errors, and it cannot be established by one universal score without defined criteria.

What it means

The practical question is whether users can rely on the data for a particular task, with understood limitations. A complete customer list can contain outdated addresses; a valid date can be assigned to the wrong event. Good quality work begins with the decision or service the data supports and identifies which failures would matter. It then measures those properties and communicates what is known, rather than treating an attractive table as evidence of reliability.

How it works

Define the intended use, critical fields, record grain, and quality rules. Profile the data to identify patterns, missingness, duplicates, and unusual values. Validate rules and compare selected facts with appropriate evidence where accuracy matters. Investigate causes, correct errors through controlled processes, and improve collection or transformation steps that create them. Monitor over time and document limitations. Quality work continues across the lifecycle; cleaning one extract does not automatically fix the source or prevent the same problem from recurring.

Three practical examples

Delivery operations: assess whether addresses are current and sufficiently complete for the actual delivery process.
Business reporting: check that transaction definitions and currencies are consistent before combining totals from different systems.
AI training: evaluate label meaning, duplication, coverage, provenance, and the relationship between the training population and the intended use.

Think of it this way

Think of ingredients selected for a particular recipe. Freshness, completeness, and suitability all matter, and an ingredient suitable for one dish may not fit another. The analogy is limited because data can be copied and reused without being consumed, while its meaning, provenance, and population coverage can change how suitable it is for each new use.

Worked example

A fictional dataset contains 1,000 customer records. Email is present in 980 records, so field completeness under that definition is 98%. Of the 980 supplied emails, 950 pass a format rule, giving about 96.94% format validity among populated values. Neither figure establishes that those addresses belong to the correct people or are currently usable. If a campaign requires only customers who opted into a particular communication, that eligibility rule is another separate requirement. The team reports the measures with their denominators and scope instead of calling the whole dataset “98% accurate.” The numbers illustrate how distinct checks answer distinct questions.

Illustrative case study

Fictional example: Birch Commerce repeatedly cleans duplicate customer records before monthly reporting. A review finds that an import process creates a new customer whenever a display name changes. The team corrects the identity-matching process and documents the remaining uncertain matches for review. This invented case illustrates addressing a cause rather than repeatedly treating symptoms. It makes no claim about a measured reduction in duplicates or a universally correct matching technique.

Common misconceptions and mistakes

Do not equate validity with accuracy, completeness with truth, or a successful import with suitability. Do not silently replace missing values with invented ones simply to improve a completeness metric. A high overall score can hide a serious problem in a critical field or subgroup. Also, more data is not automatically better: it can add irrelevant, stale, duplicated, or poorly documented records. Quality thresholds should reflect the use and consequences rather than a convenient arbitrary target.

Business use

Data quality affects customer experience, operational effort, confidence in reports, and AI behavior. Assign owners to important rules and prioritize issues by their impact on real decisions. Communicate uncertainty and known exclusions to consumers, including those using the data for a new purpose. For AI, statistical performance on one dataset does not replace checks on labels, coverage, and leakage. Keep quality evidence tied to a version and time so users know which dataset was actually assessed.

Check your understanding

A date field is populated in every row and all values match the expected format. Does that prove the dates are accurate?

Answer: No. Completeness and validity checks do not establish that each date correctly represents the event it is supposed to describe.

Related terms

Data quality dimensions
Accuracy
Completeness
Validity
Consistency
Data governance
uniqueness
timeliness
data profiling
data cleansing
provenance
fitness for purpose

Learn next

choose one dataset and one intended use. Define three critical quality rules with explicit denominators and evidence sources. Measure them, identify the most consequential limitation, and trace its likely cause. Document both the correction plan and the warning that current users need while the issue remains unresolved.

Sources

Sources: UK Government, The Government Data Quality Framework, https://www.gov.uk/government/publications/the-government-data-quality-framework/the-government-data-quality-framework ; IBM, What Is Data Quality?, https://www.ibm.com/think/topics/data-quality ; UK Government, Data Quality Issues Framework, https://www.gov.uk/government/publications/implement-a-data-quality-action-plan/data-quality-issues-framework ; The Effects of Data Quality on Machine Learning Performance, research abstract, https://arxiv.org/abs/2207.14529 . Primary guidance, vendor, and research excerpts consulted. Calculations and scenarios are original; the entry makes no universal claim that a fixed score establishes suitability.

Previous: Data proxy Next: Data quality claim