Monday, May 25, 2015

Data Virtualization Primer - The Architecture

This is the third in our Data Virtualization Primer Basics Series. I will cover the Data Virtualization architecture and components which are included in the presentation below. You can also check out the Data Virtualization Product Documentation.  We will also highlight the architecture in this article.

Our diagram to the right shows our Connect, Compose, Consume that highlights the Consumers, the Product and Sources. Data Virtualization gives us the Tools and Components to create, deploy, execute and monitor aggregated, federated data services.

The three main areas of Data Virtualization are:
  • Server - The server is positioned between business applications/Consumers and one or more data sources. An enterprise ready, scalable, managable, runtime for the Query Engine that runs inside JBoss AS that provides additional security, fault-tolerance, and administrative features.
  • Design Tools - The design tools are available to assist users in setting up Red Hat JBoss Data Virtualization for their desired data integration solution
  • Administration Tools - The administration tools are available for administrators to configure and monitor Red Hat JBoss Data Virtualization.

A JBoss Data Virtualization Server manages the following components:
  • Virtual Database - A virtual database (VDB) provides a unified view of data residing in multiple physical repositories. A VDB is composed of various data models and configuration information that describes which data sources are to be integrated and how. In particular, source models are used to represent the structure and characteristics of the physical data sources, and view models represent the structure and characteristics of the integrated data exposed to applications.
  • Access Layer - The access layer is the interface through which applications submit queries (relational, XML, XQuery and procedural) to the VDB via JDBC, ODBC or Web services.
  • Query Engine - When applications submit queries to a VDB via the access layer, the query engine produces an optimized query plan to provide efficient access to the required physical data sources as determined by the SQL criteria and the mappings between source and view models in the VDB. This query plan dictates processing order to ensure physical data sources are accessed in the most efficient manner.
  • Connector Architecture - Translators and resource adapters are used to provide transparent connectivity between the query engine and the physical data sources. A translator is used to convert queries into source-specific commands, and a resource adapter provides communication with the source.
The following design tools are available to assist users in setting up Red Hat JBoss Data Virtualization for their desired data integration solution:
  • Teiid Designer - Teiid Designer is a plug-in for JBoss Developer Studio, providing a graphical user interface to design and test virtual databases (VDBs).
  • Connector Development - The Connector Development Kit is a Java API that allows users to customize the connector architecture (translators and resource adapters) for specific integration scenarios.
  • WebUI - The WebUI allows building of Data Services through the browser so no local install is required.  This is developer preview and works with simple use cases currently
The following administration tools are available for administrators to configure and monitor Red Hat JBoss Data Virtualization.
  • AdminShell - AdminShell provides a script-based programming environment enabling users to access, monitor and control JBoss Data Virtualization.
  • Management Console - The Management Console provided by the Red Hat JBoss Enterprise Application Platform (EAP) is a web-based tool allowing system administrators to monitor and configure services deployed within a running JBoss EAP instance, including JBoss Data Virtualization.
  • Management CLI - The Management CLI (command-line interface) is provided by JBoss EAP to manage services deployed within a JBoss EAP instance. Operations can be performed in batch modes, allowing multiple tasks to be run as a group.
  • JBoss Operations Network - Red Hat JBoss Operations Network provides a single interface to deploy, manage, and monitor an entire deployment of Red Hat JBoss Middleware applications and services, including JBoss Data Virtualization.
  • Admin API - JBoss Data Virtualization includes a Java API ( org.teiid.adminapi ) that enables developers to connect to and configure JBoss Data Virtualization at runtime from within other applications.
  • Dashboard - The Dashboard builder allows connection to VDBs through the DV JDBC driver to visualize the data for testing and Business Analytics.

You can check out the product documentation as well at

Stayed tuned for the next Data Virtualization Primer topic!

Series 1 - The Basics
  1. Introduction
  2. The Concepts (SOAs, Data Services, Connectors, Models, VDBs)
  3. Architecture
  4. On Premise Server Installation
  5. JBDS and Integration Stack Installation
  6. WebUI Installation
  7. Teiid Designer - Using simple CSV/XML Datasources (Teiid Project, Perspective, Federation, VDB)
  8. JBoss Management Console
  9. The WebUI
  10. The Dashboard Builder
  11. OData with VDB
  12. JDBC Client
  13. ODBC Client
  14. DV on Openshift
  15. DV on Containers (Docker)