Does having no blank fields prove high accuracy?

No. Populated fields can still contain incorrect information.

Should every exception be deleted?

No. Investigate its meaning and cause before deciding how to handle it.

Is a database activity log a complete data audit?

No. It records selected events and does not by itself establish dataset quality.

AI Glossary / Data engineering and architecture / Advanced

Data auditing

Definition

Data auditing is a systematic examination of data and, where relevant, the processes that handle it against stated criteria. It gathers evidence about matters such as completeness, consistency, accuracy or adherence to an agreed procedure. The scope must be explicit: assessing dataset quality is different from examining database activity logs, although both may contribute to a broader review.

What it means

An audit asks what can be demonstrated about the data, rather than assuming that a successful import makes it suitable for use. A dataset might have every field populated but still contain incorrect values. It might also be accurate for an earlier period while unsuitable for a current decision. In an AI project, auditing can reveal missing categories, duplicated records or unexplained transformations before those issues are carried into training or retrieval. Findings should identify the evidence and the limits of what was examined.

How it works

Define the purpose, population, criteria and examination method before interpreting results. Tests might count missing values, inspect identifier duplication, compare related fields or reconcile selected records with a trusted source. Sampling and full-population checks provide different kinds of evidence. The UK Government Data Quality Framework distinguishes several quality dimensions and stresses fitness for the intended use. Record exceptions and investigate their causes, then verify any corrective action. A completed checklist is useful only when the tests actually address the stated question and their results can be traced.

Three practical examples

A customer-data team checks whether mandatory identifiers are missing and whether repeated identifiers represent legitimate multiple records.
An analytics team reconciles selected source transactions with their transformed reporting values.
An AI project examines whether a proposed dataset contains the expected categories and records unexplained gaps before training begins.

Think of it this way

Imagine inspecting a shipment against an order. Counting the boxes tests one thing; checking that the correct goods are inside tests another. An intact label does not prove the contents are right. Data auditing likewise separates evidence about presence, format and meaning. The inspection plan must reflect what the recipient needs. A quick count can be valuable, but describing it as a complete inspection would exaggerate what the evidence establishes.

Worked example

Consider an original dataset of 100 records. Eight lack a required region value, and five have a delivery date earlier than the order date. Three records fail both tests. The number failing at least one test is 8 + 5 − 3 = 10, so 90 pass both checks. Region completeness is 92 percent, while the date-rule pass rate is 95 percent. Neither percentage proves factual accuracy, and the ninety records passing both checks may still contain other errors. The calculation also assumes the date-order rule is appropriate for this dataset and that these counts cover all one hundred records rather than a sample.

Illustrative case study

A fictional distributor prepares delivery records for a forecasting project. An initial review reports a high field-completion rate, but a later reconciliation finds that some quantities use a different unit. The team adds an explicit unit check and traces the mismatch to one source export. It documents the affected records, corrects the transformation and repeats the relevant checks. The scenario illustrates why auditing should examine meaning and process as well as missing cells. It does not imply that passing those checks proves the resulting forecast will be accurate or suitable for every operational decision.

Common misconceptions and mistakes

Data auditing is not the same as automatically cleaning every flagged value. An exception can be legitimate, and an unexplained correction can remove useful evidence. Completeness is not accuracy, and a valid format does not prove a value is true. Another mistake is adding exception counts without accounting for overlap, which can exaggerate the number of affected records. Results from a selected sample should not be described as a full-population examination. Finally, a quality review does not automatically constitute a security audit, legal compliance assessment or certification against an external standard.

Business use

A well-scoped audit helps decide whether a dataset is fit for a proposed workflow and where improvement effort belongs. State the decision the review supports, select relevant criteria and retain evidence of findings. Prioritize issues by their consequences for that use rather than only by how easy they are to count. Assign responsibility for investigating and correcting problems, and distinguish an observed improvement from a proposed fix. For AI initiatives, connect findings to the particular training, evaluation or retrieval process instead of labeling data universally ready for AI.

Check your understanding

How many toy records failed at least one check?

Answer: Ten, after subtracting the three counted in both groups.

How many passed both checks?

Answer: Ninety.

Why could those ninety still be unsuitable?

Answer: The two tests did not examine every relevant property or establish factual accuracy.

Related terms

Data Quality
Validation
data profiling
reconciliation
audit trail
root cause analysis

Learn next

Choose two meaningful checks for a small dataset and record both individual failure counts and overlapping failures. Explain what each check demonstrates and what it leaves unknown. Then identify one independent source or process observation that could help investigate an exception without guessing a replacement value.

Sources

UK Government, The Government Data Quality Framework: https://www.gov.uk/government/publications/the-government-data-quality-framework/the-government-data-quality-framework The overlapping-exceptions calculation, shipment analogy and distributor case are original educational illustrations. The reference supports quality assessment and fitness for purpose; this entry does not claim to supply a formal audit or certification standard.

Previous: Data assimilation Next: Data augmentation