- Zero footprint architecture where no disruptive source/target agents are installed and with minimal administrative overhead.
- Comprehensive, flexible synchronization covering multi-topic, multi-partition data transfer needs dynamically propagate source schema changes, and replicated data filtering.
- Optimized cloud data integration combining high-performance multi-pathing and AES 256 WAN encryption.
This customer success story illustrates how Verizon centralized real-time data for data lake analytics using solutions from Attunity and Hortonworks.
The Challenge: Real-time Data Ingestion
“Getting data in real-time for our financial reports was very time-consuming and labor intensive,” said Arvind Rajagopalan, Global Technology Services (GTS) at Verizon. “We were using a multi-tier process to pull data from our SAP ERP and PeopleSoft systems and to integrate that into our Hadoop data lake. This approach was extremely challenging and our financial teams often had to wait a whole day to see data changes, which was not sustainable for the business.”
The Verizon GTS team had one PeopleSoft system, as well as two SAP ERP systems that delivered financial information and business transactions throughout the enterprise. Ingesting the PeopleSoft and SAP data into the data lake had these challenges:
- The GTS team used several separate utilities to capture source changes, which added unnecessary complexity and time while creating higher risk for error
- The alternative data replication solution from Oracle failed to support SAP pool and cluster tables, which was a key requirement to make the solution work. It also required a complex middle tier to populate the data lake, which added an additional layer of labor, time, and cost to the initiative
- Extra processing and delays were undermining the accuracy of daily reports
The Solution: Attunity Replicate and the Hortonworks Data Platform
Verizon’s GTS team began searching for solutions that could ingest data in real time, understand the SAP data dictionary and perform real-time change data capture (CDC) on the complex SAP pool and cluster tables. They tested Attunity Replicate with its CDC technology, which successfully moved Verizon’s SAP ECC data and PeopleSoft data in real-time into their Hortonworks data lake. The Verizon GTS team found Attunity Replicate to be simple, universal and fast, helping them to:
- Centralize all disparate ERP data into a Hortonworks data lake which improved financial insights and analytics across Verizon divisions
- Capture and translate data from a complex SAP format and easily export it via an intuitive user interface – completed 10 times faster compared to previously tried alternatives
- Replicate SAP data to all major targets, including databases, data warehouses, or the Hortonworks Data Platform (HDP)
- Continuously update the data lake using CDC technology to ensure the freshest data is available in real time for improved decision making
The Results: Successful Data Reporting and Analytics in Real-time
All divisional and corporate financial reporting/business intelligence (BI) teams now have access to ERP data in one place. As data changes, it is automatically uploaded to the data lake in less than an hour rather than overnight so that the most up-to-date data is available for daily reports.
The Verizon GTS team eliminates the middle-tier Oracle database, reducing maintenance time and cost, and easing the strain on valuable developers.
As more value is generated by delivering real-time insights and analytics in combining various data sets with the financials, the GTS team expects to discover new insights by bringing additional data sets to the data lake. This will allow Verizon to make more strategic decisions for different parts of the business and will accelerate key business initiatives.
To learn more: