AI Glossary / Governance, ethics and safety / Beginner

Privacy

Definition

Privacy concerns people's interests in how information about them and their lives is observed, collected, inferred, used and shared. In AI, it includes managing the problems people may experience from data processing, even when the processing is technically secure.

What it means

Privacy is broader than keeping a password secret. A service can protect a database from intruders yet use its contents in ways that expose, classify or influence people beyond the intended purpose. An AI system can also infer information that a person never explicitly supplied.

The concept has social, ethical and legal dimensions, and no short definition settles every context. People's expectations depend on the setting, relationships and consequences. Legal rights and obligations vary by jurisdiction and activity, so a general privacy claim should not be treated as a universal compliance conclusion.

For AI projects, the useful starting point is the full information lifecycle. Consider the original data, intermediate representations, model outputs, logs, integrations and later reuse. A privacy problem may arise from ordinary authorized processing as well as from a security breach.

How it works

Map the people affected and the data actions involved. Identify what is collected, why it is needed, who can access it, what can be inferred and how long it remains available. Include people mentioned in documents or captured incidentally, not only account holders.

Assess possible consequences in context. Information may reveal a sensitive circumstance, enable unwanted profiling or be combined with other data to identify someone. A prediction can affect a person even if the inference is uncertain or wrong. Evaluate the purpose and downstream use as well as the confidentiality of storage.

Select measures that address the identified problems: reduce unnecessary collection, limit uses, control access, manage retention and test outputs or interfaces that may reveal information. Where suitable, privacy-enhancing techniques can provide additional protection under explicit assumptions. Maintain clear communication and meaningful ways to exercise applicable choices or rights. Reassess when new data sources, recipients or model capabilities change the situation.

Three practical examples

• A meeting assistant limits unnecessary retention of transcripts and explains which information is used for the requested summary.
• A workplace search tool checks document permissions when retrieving information and when presenting generated answers.
• A recommendation service examines whether inferred preferences reveal sensitive information through shared-device notifications.

Think of it this way

Think of telling a colleague something for the purpose of arranging a meeting. The fact that the colleague heard it does not mean every possible later use fits the original context. Appropriate handling depends on the relationship and purpose.

The analogy does not replace legal rules or technical protection. It illustrates why privacy concerns include the flow and use of information, not merely whether someone originally disclosed it.

Worked example

Secure storage, inappropriate disclosure

In an invented organization, 100 staff members can use an AI search assistant. A personnel document is legitimately accessible to five authorized staff. The source repository correctly enforces that restriction.

A separate indexing process copies the document into a store used by the assistant without preserving its permissions. If all 100 users can retrieve its contents through generated answers, the information is now exposed to 95 users outside the original access group.

Encryption of the index does not resolve that access mismatch if the application decrypts and serves the content to everyone. The review must follow authorization through indexing, retrieval and answer generation. These numbers illustrate a boundary failure, not a measured probability of harm.

Illustrative case study

Fictional case: Willow Coaching offers an assistant for personal learning notes. Its team focuses initially on secure storage. Later, a review finds that shared email summaries include inferred learning difficulties the user did not expect to be sent to another recipient.

The team examines why the inference is generated, whether it is necessary and how recipient choices are established. It revises the summary design and tests information flows across private notes and shared outputs. Security remains necessary, but it is no longer treated as the whole privacy assessment.

The case demonstrates a privacy issue arising from normal functionality. It does not report a real incident or offer a jurisdiction-specific legal conclusion.

Common misconceptions and mistakes

• Strong cybersecurity guarantees privacy. Security controls help, but authorized uses can still create privacy problems.
• Publicly available information has no privacy implications. Collection, combination and new uses can change its consequences.
• Removing names always resolves the issue. Linkability, inference and contextual information can remain relevant.

Business use

Businesses can make privacy concrete by connecting each data use to a justified purpose, responsible owner and operational controls. Review the actual product journey rather than relying only on a policy document. Include the consequences for individuals alongside organisational risks.

Procurement should examine data flows, retention, access, model training uses and downstream recipients. Ask what evidence supports a privacy claim and what assumptions limit it. A technique's name, a contractual phrase or a successful security test is not by itself a complete assessment of the AI system's privacy effects.

People also ask

Is privacy the same as confidentiality?

No. Confidentiality limits unauthorized disclosure; privacy also concerns collection, use, inference and effects on people.

Can a system affect privacy without storing names?

Yes. Other information or combinations can relate to or identify individuals.

Does consent solve every privacy problem?

No. The appropriateness and consequences of processing, along with applicable obligations, still need attention.

Check your understanding

An assistant keeps documents encrypted but reveals restricted facts in answers to users who lack source access. Has encryption alone protected privacy?

Answer: No. The information flow and authorization boundary are failing despite protection of stored data.

Related terms

• Personal data
• Privacy-by-design
• Privacy-enhancing technology
• data minimisation
• confidentiality
• privacy risk

Learn next

Read privacy risk and data minimization. Map one AI feature from collection to deletion, including inferred information and shared outputs, then identify a privacy problem that could occur without a security breach.

More in Governance, ethics and safety

• Predictive parity - Predictive parity is a group fairness criterion requiring equal precision, also called positive predictive value, across the groups being compared.
• Preference alignment - Preference alignment is the process of adjusting an AI system's behavior to better match specified preferences about its outputs or actions.
• Presentation Bias - Presentation bias is a distortion in behavior or resulting data caused by how information is displayed.
• Privacy attack - A privacy attack is an attempt to learn protected information by exploiting a system's data, behavior or accessible outputs.
• Privacy funnel - The privacy funnel is an information-theoretic approach to transforming data so that the released representation retains useful information while revealing...
• Privacy leakage - Privacy leakage occurs when information released by a system reveals something about protected data beyond the intended disclosure.

Sources

NIST Privacy Framework, Getting Started: https://www.nist.gov/privacy-framework/getting-started-0. NIST Privacy Framework Version 1.0: https://nvlpubs.nist.gov/nistpubs/CSWP/NIST.CSWP.01162020.pdf. Relevant returned material explains privacy risk across the data lifecycle and problems affecting individuals. The framework edition is identified explicitly; scenarios and workflow questions are original applications rather than legal guidance.

Previous: Priority queue Next: Privacy attack