AI Glossary / MLOps and deployment / Intermediate

Data virtualization

Definition

Data virtualization is an approach that presents data from different systems through a shared logical access layer. Users can query virtual tables or views without first building a complete central copy of every source. The underlying records can remain in their original systems while the layer coordinates access and combines results.

What it means

A business may keep customer details in one database, orders in another and product information in a third. A virtual view makes selected fields appear together to an authorized application. This simplifies access, but it does not make differences in meaning disappear. Customer identifiers, currencies, timestamps and definitions still require agreement. Virtualization is therefore an access and integration approach, not proof that the resulting data are correct, consistent or suitable for an AI model.

How it works

Connectors describe available sources, while metadata and mappings define the virtual objects exposed to users. A query planner decides which operations can run at the sources and which require combining data elsewhere. Filtering or aggregation can sometimes be pushed down to reduce transfer. Results still travel across connections; saying data stay at the source does not mean that no bytes move. Some products use caching or materialized results for performance, so freshness and storage behavior depend on configuration. Authentication and authorization must cover the virtual layer and underlying systems.

Three practical examples

• A sales dashboard joins account information with recent order totals without maintaining a second full copy of both operational databases.
• An AI team explores candidate training fields across several systems through a common query interface, then deliberately creates a versioned training extract.
• A service adviser sees selected inventory and customer records together. The view exposes only the fields needed for the task and preserves access restrictions for sensitive columns.

Think of it this way

Think of a library catalog that helps a reader find material in several branches. It offers one entry point without relocating every book. The comparison is limited: a data-virtualization engine can also retrieve, transform and join results, while a catalog alone does not make the books' contents mutually consistent or simultaneously available.

Worked example

Suppose a customer table contains C1 in the North region and C2 in the South region. An orders table contains two C1 orders worth 80 and 120, plus one C2 order worth 50. A virtual query joins on customer ID and groups by region, returning North = 200 and South = 50. The customer database may provide the region mapping while the order database first aggregates by customer. If C1 accidentally appears twice in the customer table, a naive join can duplicate its order total to 400. Virtualization has not caused the duplicate, but it has not automatically repaired it either. Validate key uniqueness and join meaning before trusting the apparently simple combined view.

Illustrative case study

Fictional case: Juniper Services wants a shared view of support cases and subscription accounts. It creates a virtual view for analysts and discovers that one system stores account closure dates in local time while the other uses UTC. The team documents a common time convention and tests a sample of boundary cases. For model training, it saves a dated extract with the query version and source timestamps. This fictional project illustrates why convenient live access and reproducible machine-learning datasets are separate requirements.

Common misconceptions and mistakes

Do not interpret virtualization as zero network traffic, zero storage or guaranteed real-time consistency. A query spanning sources may observe changes at different moments unless stronger coordination is available. Do not assume it replaces every data warehouse: repeated heavy joins may be better served by prepared data. A virtual view can also hide operational complexity from users while leaving that complexity for administrators. Source outages, schema changes and permission changes can still break dependent applications.

Business use

Data virtualization can reduce duplicated integration work and shorten exploration when data are distributed across systems. Evaluate its usefulness through query latency, source-system load, freshness, governance and maintenance effort. For AI use, establish whether each dataset is a live view or a frozen extract, and record the transformations that define features. Check cost under realistic concurrent demand, including the effect of analytical queries on operational applications that customers depend on.

People also ask

Is it the same as a virtual machine?

No. Virtual machines abstract computing environments; data virtualization abstracts access to data sources.

Does it remove the need for data cleaning?

No. Definitions, missing values, duplicates and conflicting identifiers still require explicit handling.

Can it support machine learning?

Yes, particularly discovery and access, but repeatable training often requires versioned data and recorded query logic.

Check your understanding

A live virtual view gives different training rows next week even though the query is unchanged. Is that necessarily a software fault?

Answer: No. The underlying sources may have changed. Reproducible experiments require a suitable snapshot or versioning strategy in addition to a stable query.

Related terms

• Materialized view
• federated query
• logical data layer
• data integration
• query pushdown
• data lineage
• snapshot

Learn next

Draw a proposed virtual view and identify each source, join key, refresh assumption and permission boundary. Test one small query against manually checked records. Then decide which consumers need current results and which need a stable historical version, because those needs imply different storage and freshness choices.

More in MLOps and deployment

• Continuous multi-output - Continuous multi-output describes a regression task in which each input example has more than one numerical target value.
• Curve-fitting compaction - Curve-fitting compaction represents a sequence or curve using a mathematical expression and its parameters instead of storing every original point.
• DAG Processing Model - A DAG processing model represents a computation or workflow as a directed acyclic graph: nodes describe work, directed edges describe dependencies, and no...
• Data-Centric ML - Data-centric ML is an approach to improving machine-learning systems through systematic work on the data they use.
• Data compression ratio - Data compression ratio compares the size of data before and after compression.
• Data compression symmetry - Data compression symmetry describes how similar the computational effort is for compressing and decompressing data.

Sources

Primary references: IBM documentation on creating virtual objects, https://www.ibm.com/docs/en/ws-and-kc?topic=data-creating-virtual-objects ; IBM Data Virtualization product description, https://www.ibm.com/products/watson-query . Primary documentation search excerpts consulted; full product-specific and editorial review remain pending. Implementation capabilities vary across systems.

Previous: Data version control Next: Data visualization