Is every socio-technical system an AI system?

No. Transport networks, factories and many information services combine social and technical elements without AI.

Is this the same as human-in-the-loop?

Human-in-the-loop describes a particular participation pattern. Socio-technical analysis also examines institutions, incentives, affected people and wider interactions.

Who is responsible for the outcome?

Responsibilities should be explicitly assigned. Calling the outcome a system effect should not erase accountability for design, operation or decisions.

AI Glossary / Governance, ethics and safety / Advanced

Socio-technical system

Definition

A socio-technical system is a system whose behavior and outcomes arise from interacting social and technical elements. These include people, roles, organizational practices, incentives, software, data and equipment. An AI-enabled service is therefore more than its model: the way people develop, interpret, operate and respond to the technology helps determine what the service actually does.

What it means

Consider an AI tool that prioritizes incoming requests. Its performance depends on the model, but also on which requests are recorded, what the priority labels mean, how staff use the queue and whether urgent exceptions can be raised. A technically accurate score may have little benefit if nobody has time or authority to act on it.
The concept shifts attention to interactions without dismissing technical quality. It asks how components work together and what outcomes they produce for people. Two organizations can deploy the same model and obtain different results because their staffing, procedures, interfaces and objectives differ.
A socio-technical perspective also includes people affected by decisions who never directly use the software.

How it works

The analysis starts by defining the service, its purpose and its boundaries. Teams map who supplies information, who receives outputs, who makes decisions and who experiences their consequences. They examine the technologies and rules connecting these roles.
Next, they investigate how the process works in practice. A documented review step may be bypassed under workload pressure. An interface may conceal uncertainty. A target rewarding speed may discourage careful checks. These interactions can change outcomes even when the underlying model remains unchanged.
NIST's AI RMF discusses human roles, organizational influences and variations in human-AI performance. Its governance and mapping functions support understanding the operating context and assigning responsibilities. Evaluation then needs evidence about the combined workflow, alongside appropriate component tests. Changes to staffing, permissions or procedures can require reassessment just as model changes do.

Three practical examples

Customer service: A response assistant supplies drafts, but supervisors determine escalation rules and staff decide what to send. Accuracy, training, workload and access to source information all shape the customer's experience.
Workplace scheduling: A forecasting tool estimates demand, while staffing policies and employee availability determine the roster. A good forecast alone does not guarantee a workable or acceptable schedule.
Everyday navigation: A route recommendation interacts with road conditions, driver interpretation and local restrictions. The practical outcome depends on more than the calculated travel time.

Think of it this way

An orchestra's performance depends on instruments, musicians, the score, coordination and the setting. Improving one instrument does not guarantee a better performance if the musicians cannot hear each other. The analogy emphasizes interaction: a socio-technical system must be understood through how its parts work together, not just through a list of their individual qualities.

Worked example

A fictional service processes 1,000 requests. Its model produces 900 correct recommendations and 100 incorrect ones. In workflow A, reviewers correct 60 of the incorrect recommendations but mistakenly change 20 correct recommendations into incorrect decisions.
The final number correct is 900 + 60 - 20 = 940, giving 94% workflow accuracy. This differs from the model's 90% accuracy.
In workflow B, the same initial recommendations are reviewed under different conditions. Reviewers correct only 20 errors and mistakenly change 50 correct recommendations. The final number correct is 900 + 20 - 50 = 870, or 87%.
The arithmetic assumes one final decision per request and the stated changes. It does not establish why the workflows differ or prove that review is inherently beneficial or harmful. It shows why measuring the model alone cannot determine the combined result. Real evaluation should also consider error consequences, delays and affected groups.

Illustrative case study

Fictional company Alder Support pilots an AI assistant with experienced staff who have time to inspect source documents. The pilot appears useful. During expansion, the company assigns the tool to a much busier team and measures staff mainly by the number of conversations closed.
Some staff begin accepting drafts without checking the underlying instructions. Customer complaints increase even though the model version has not changed. The investigation examines workload, incentives, source visibility and escalation access as well as output quality.
The company revises the workflow and evaluates the resulting service before further expansion. Deployment changes can invalidate pilot assumptions. No single organizational change is guaranteed to solve the problem.

Common misconceptions and mistakes

“Socio-technical means nontechnical.” Technical performance remains essential; the concept adds the interactions that affect its practical use.
“Human error explains the failure.” That label may hide confusing interfaces, inadequate resources or conflicting incentives that made the error more likely.
“Adding a reviewer guarantees improvement.” Reviewers need relevant information, competence, time and authority, and the combined result still needs testing.
“The system ends at the application screen.” Downstream decisions and affected people can be central to understanding its impact.
“An unchanged model means unchanged risk.” The operating context can change independently of the model.

Business use

Map an AI project as a service before treating it as a software purchase. Identify the intended outcome, decision owners, affected people, dependencies and exception routes. Observe real work rather than relying only on process diagrams.
Use measures that reflect the final outcome as well as model quality. Track whether people can challenge outputs and whether corrections reach those responsible for improvement. Include operational staff and relevant subject experts in evaluation. When deployment conditions change, revisit the assumptions that supported the earlier approval.

Check your understanding

Why can the same model produce 94% or 87% workflow accuracy?

Answer: The reviewers' corrections and newly introduced errors differ.

Does the calculation show that review should be removed?

Answer: No. It shows the need to evaluate the particular combined process.

Why examine staff incentives?

Answer: They can influence how people use the tool and whether formal procedures are followed.

Related terms

AI governance
Automation bias
Human-AI interaction
human factors
human oversight
organizational context
systems thinking
end-to-end evaluation

Learn next

Draw the path from an input to a final decision for a familiar AI service. Identify handoffs and assumptions, then compare component measures with evidence about outcomes for users and affected people.

Sources

NIST AI RMF 1.0, Appendix C: AI Risk Management and Human-AI Interaction. https://airc.nist.gov/airmf-resources/airmf/appendices/app-c-ai-risk-management-and-human-ai-interaction/ NIST AI RMF 1.0, Core, governance and context mapping. https://airc.nist.gov/airmf-resources/airmf/5-sec-core/ Theofanos, Choong and Jensen, AI Use Taxonomy: A Human-Centered Approach. https://www.nist.gov/publications/ai-use-taxonomy-human-centered-approach The workflow calculations, practical examples and Alder Support case are original teaching illustrations.

Previous: Societal Bias Next: Soft actor-critic