Data virtualization  takes a single data source, for example a set of data files, and allows multiple databases to share those data files as if each database has private sole access to those data files. The data files are kept coherent for each database by sharing the duplicate blocks while at the same time keeping modified blocks private to the database that made the change. Data virtualization for databases allows many copies of a database to share the majority of the underlying storage and allows copies to be provisioned in minutes for almost no storage overhead.

Data virtualization for databases accelerates enterprise application projects for businesses around the world by providing fast, flexible access to virtual data—dramatically lowering costs and complexity.

Enterprise applications constantly evolve to meet changing business demands, triggering expensive projects that overrun budgets and fall behind schedule. Data is the lifeblood of applications and has to be pumped across project environments. To make data management harder, applications are constantly on the move, migrating to new data centers, private clouds, public clouds, hybrid clouds, flash storage, and open platforms, creating complexity and fragmentation. With data growing constantly inside applications, managing data through project lifecycles gets harder and harder every day.

Data virtualization can be accomplished with an Agile Data Platform. An Agile Data Platform delivers the right data to the right team at the right time.

Today’s businesses deploy small armies of DBAs, storage admins, backup admins, and systems admins to make and move database copies across redundant hardware environments—dispiriting, rock-breaking work. IT teams struggle to deliver data, using storage, backup, and replication products designed without application projects in mind, an operational Frankenstein that leaves development teams waiting for data environments and drains businesses of millions dollars each year in lost productivity.

Agile Data requires a complete solution, an integrated set of technologies that can bridge the data divide. A complete solution like iPod and iTunes can revolutionize an industry. An integrated Agile Data Platform would  provide end-to-end, automated data delivery, eliminating delays for projects, frustration for developers, and lost business productivity.

An integrated Agile Data Platform would  non-disruptively collect data from enterprise applications, automatically versions all changes, and delivers fast, flexible access to virtual datasets for application development, ERP implementations, database upgrades, data center migration, application consolidation, BI and data warehouses, data protection, and innovation projects.

The Agile Data Platform virtualizes databases by sharing data blocks across all environments instead of making and moving full, physical copies.  An Agile Data Platform would consolidate dozens to hundreds of physical database copies into the space of a single copy and reduce the cost of the next incremental copy by over 100x, radically changing the economics of application projects.

Businesses today have to trade off quality, speed, and cost. An Agile Data Platform shrinks the cost of data access AND improves quality and speed, eliminating the tradeoff.

By versioning changes to data, an Agile Data Platform provides project teams with powerful, self-service data control—fast, simple data refresh, rollback, integration, and branching. With the Agile Data Platform provided by Delphix, customers like Facebook, RBS, P&G, Deutsche Bank, Comcast, New York Life, and Macy’s roll out application projects in 20 to 80% less time, helping IT move at business speed.

How It Works

Delphix connects non-disruptively to databases (the ubiquitous repository for enterprise data) and loads a compressed copy of the data into the Delphix Engine, shrinking the data by 3x on average. Inside the engine, the Delphix file system (DxFS) compresses data blocks within database files and filters out empty or temporary blocks, minimizing the data footprint.

After the initial data seeding, Delphix maintains synchronization by collecting changes and tracking all versions for as long as required (e.g. weeks or months). From any point in time, Delphix can open one or more virtual databases (VDBs) that can be used for development and other lifecycle environments.

For an average application, businesses maintain more than seven lifecycle environments for development, testing, QA, integration, training, pilots, operational reporting, production support, user acceptance, system validation, and sandboxes—not to mention redundant systems for backup, DR, and archiving logs. Instead of making and moving data copies over and over again, DxFS provides a virtualized view of databases by sharing the underlying data blocks across all environments and storing changes as new, unique blocks.

VDBs look and perform like normal, physical copies (e.g. users can add/drop tables, make schema changes, run reports against the data), but include powerful features designed to accelerate application projects like virtual branching and fast data rollback or refresh.

Virtualizing databases fundamentally changes application testing and quality. In order to minimize cost and complexity, many organization test their applications using stale data that may be days or months old, or use data subsets (non representative datasets) that can fail to test against a range of potential errors. With fast, automatic refresh and virtual databases that provide full, representative datasets, Delphix can dramatically improve the fidelity of QA and test environments.

Data Version Control

Developers have long used source code version control to track changes and work in parallel streams. Application projects that run on databases have to pair release versions with corresponding datasets— databases with the correct schemas and tables to enable application software to function properly. Databases, however, have traditionally been complex, slow, and hard to set up and maintain, which often forces application teams to settle with stale, partial, or shared data environments.
The Delphix Engine includes a second key technology component—the DataVisor, which provides efficient data synchronization (even across the WAN), full transactional consistency,integrated log shipping, and continuous versioning. With the DataVisor, a Delphix Engine can maintain synchronization with multiple source applications in near real time and automatically record and version all changes.
With a simple time slider, Delphix can quickly deliver a virtual version of a database at any point in time (down to the second or a specific transaction boundary). Instead of waiting weeks for data delivery and then running a test in QA, Delphix can reduce overall cycle times from weeks to hours, enabling faster testing and error detection.

Continuous versioning solves additional key challenges for enterprise applications. Many applications, like SAP, require data federation across multiple databases for data consistency; with the DataVisor Delphix can deliver multiple databases at the same point in time in just a few clicks and minutes.

Fine-grained version control allows developers to reset a database to perform multiple comparative tests (A/B tests), create a library of retained versions that coincide with software releases, and quickly rollback to a previous stage during complex data conversion and mapping cycles.

Data changes constantly in databases, making it impractical to know ahead of time when a specific version will be required. With the DataVisor, Delphix automatically records all versions, so if a DBA drops a table, it can be recovered in minutes from the most recent version available, minimizing downtime, data loss, and potential productivity or revenue impact.

Performance and Load

DxFS not only virtualizes databases by sharing data blocks in storage, it also shares data blocks in memory, enabling highly cost-effective performance for consolidated application workloads. On average, customers run 20 VDBs per Delphix Engine but can run up to 400 VDBs in some environments. Elasticity allows application teams to expand to environments as needed during project lifecycles, providing flexibility to adjust to changes in requirements.

For production or source applications, Delphix can actually reduce loads while maintaining synchronization. Delphix connects to source databases through standard APIs, so it does not sit between the production data path or require installation of any agents. VDB workloads only impact the Delphix Engine and have zero impact on source applications, allowing customers to offload reporting and batch workloads like BI extracts.

After the initial seeding, Delphix maintains synchronization by collecting incremental changes forever, eliminating all future full data loads from source environments, including repeated full backups, one of the largest loads on application systems and networks. In addition, the DataVisor applies changes to create “forever full” versions (a term coined by Gartner analyst Dave Russell) in the Delphix Engine, so versions can be made immediately available.

Deployment

Enterprise applications are notorious for long, complex, and risky implementation cycles. In contrast, Delphix deploys in minutes, can quickly synchronize data from a source, and delivers virtual data environments in a few clicks from that point forward. Since most enterprises take more than a week on average to deliver a single data environment, Delphix can accelerate application projects already in flight and begin to save time immediately.

The Delphix Engine comes packaged as a virtual appliance—software that can be installed anywhere a business needs: on physical infrastructure, in remote data centers, in private or public clouds, and outsourced environments. With the DataVisor, Delphix can efficiently maintain synchronization across all locations, providing flexibility for today’s application portfolios and a future-proof data delivery solution.

Delphix supports Oracle and Microsoft SQL Server databases and data warehouses on a range of operating systems (e.g. Linux, Windows, HP-UX, AIX, Solaris) and can run on any storage (e.g. EMC, NetApp, HDS, flash arrays, cloud storage, etc.). Published web services drive both a web management interface as well as the CLI, enabling seamless integration with existing automation workflows.

Different View

Most software vendors and IT organizations focus on today—the live, production application environment. Delphix focuses on tomorrow, delivering data across time and location to accelerate application lifecycles. While applications will continue to evolve and their locations will continue to change, data must be preserved across all current and future changes. Agile Data applies to all enterprise applications and unlocks data for the future of IT.