Is opacity always deliberate?

No. Technical complexity and inadequate communication can create it without intentional concealment.

Can an interpretable model still be part of an opaque system?

Yes. The surrounding data sources, rules or human decisions may remain unclear.

Does reducing opacity prove fairness?

No. Better understanding can support assessment, but it does not establish the assessment's conclusion.

AI Glossary / Governance, ethics and safety / Intermediate

Opacity

Definition

Opacity is difficulty seeing or understanding how an AI system operates, reaches an output or affects a decision. It can arise from restricted information, technical complexity or a mismatch between available explanations and what a person needs to understand.

What it means

An opaque system is often called a black box, but that phrase can hide several different problems. A supplier may withhold documentation. A user may lack the technical knowledge to interpret released code. Even specialists may struggle to connect a complex learned representation to a meaningful explanation of one result.

These problems require different responses. Publishing code can reduce secrecy without making behavior understandable. A plain-language explanation can improve accessibility without proving that it faithfully describes the model.

Opacity is also relative to a question and audience. A service may clearly disclose that AI is used while remaining unclear about which information influenced a particular decision. Transparency about one aspect should not be mistaken for transparency about everything.

How it works

Identify what the person needs to know and where understanding breaks down. Relevant questions include what data is used, how a result was generated, which part of a workflow made the decision and how an error can be challenged.

Then determine whether the barrier is missing access, inadequate documentation, technical difficulty or the limits of an explanation method. Responses can include clearer records, appropriate review access, understandable interfaces or more suitable model designs.

Evaluate whether the response actually answers the intended question. A detailed architecture diagram may not explain an individual outcome, while a local feature chart may not describe overall behavior. Some uncertainty may remain even after substantial disclosure, so claims should state the scope of understanding achieved.

Three practical examples

A procurement team asks a supplier to distinguish withheld information from technical aspects that remain difficult to interpret even with access.
A user-facing service explains which stage used AI and which final action was taken by a human reviewer.
An auditor requests the input and model version associated with a disputed result because general product documentation cannot reconstruct that event.

Think of it this way

Think of a sealed machine cabinet. Opening the cabinet removes one barrier, but someone may still need a diagram and technical knowledge to understand the machinery. Even then, a brief observation may not explain an intermittent fault.

The analogy distinguishes access from understanding. Machine-learning behavior can involve statistical relationships rather than an easily traced mechanical chain, so disclosure alone may leave substantial interpretive limits.

Worked example

Three answers to three different questions

In an invented review, a supplier publishes the model's architecture, supplies a list of training-data categories and provides a local explanation for one prediction. These are three useful disclosures, but they answer different questions.

The architecture does not by itself show whether a particular record was used in training. The category list does not establish the exact cause of one prediction. The local explanation does not demonstrate that the model behaves fairly across all relevant groups.

The reviewer maps each requested claim to the evidence actually supplied. Where a claim remains unsupported, the gap is recorded rather than filled with a general transparent label. This exercise reduces opacity by making the boundaries of available information explicit.

Illustrative case study

Fictional company Elm Services tells customers that its AI review process is transparent because the model's name appears on every result. A complaint reveals that customers still cannot tell which records were considered or how to correct an outdated input.

The company documents the decision process, provides an appropriate route to review relevant records and explains the role of the model and staff. It also evaluates whether the explanation is understandable to the intended audience.

The changes address the practical questions raised by customers. Disclosing a model name remains useful, but it is no longer treated as a complete account of the process.

Common misconceptions and mistakes

Open-source code eliminates all opacity. Access does not guarantee comprehensibility or explain every learned relationship.
Complexity is the only cause. Secrecy and missing operational records can also matter.
An explanation makes a model trustworthy by itself. Faithfulness, performance and appropriate use still require evidence.
Every audience needs the same detail. Useful disclosure depends on the question and the person's role.

Business use

Businesses can use an opacity review to identify what prevents meaningful evaluation or challenge. This is valuable when procuring external models or explaining consequential workflows to staff and customers.

Specify the information needed for each purpose and the evidence that it answers the question. Where disclosure is limited, consider whether appropriate independent review or a different system design can address the gap. Avoid both unsupported claims of complete transparency and the assumption that complexity makes all scrutiny impossible. Useful accountability often begins with precise, answerable questions.

Check your understanding

A vendor releases source code but cannot identify which version produced a disputed output. Has every relevant form of opacity been removed?

Answer: No. Code access does not supply the missing operational history. Reconstructing the event requires appropriate version and input records as well.

Related terms

Learn next

Read interpretability and traceability. Choose one AI decision and write three questions a user, developer and reviewer would ask. Identify the evidence needed for each instead of using one generic transparency claim.

Sources

Jenna Burrell, How the machine thinks: Understanding opacity in machine learning algorithms: https://doi.org/10.1177/2053951715622512. NIST AI RMF, AI Risks and Trustworthiness: https://airc.nist.gov/airmf-resources/airmf/3-sec-characteristics/. The source distinctions inform this overview; the disclosure review and fictional customer case are original illustrations.

Previous: Ontology merging Next: Open access