Two big events are happening in Las Vegas this week. The first is Amazon’s yearly tech bonanza called AWS re:Invent and the second is our launch of Attunity for Data Lakes. But before I talk about Attunity’s news, let’s recap some of the AWS re:Invent 2018 highlights so far.
re:Invent Gets Bigger Every Year
If you’ve attended re:Invent in the past five years you won’t be surprised to learn just how big the event has gotten. This year’s conference spans seven different Las Vegas properties and hosts over 50,000 attendees. There are also over 2,000 breakout sessions listed in the event brochure covering everything from AI to XML (Did you see what I did there? AI to XML… A to Z… get it?) re:Invent is so massive that it can be difficult to navigate, especially if you’re looking to find a particular vendor. Attunity’s booth is 630 and is located just inside the main expo hall. Lucky for you I’ve annotated the map below, so there should be no excuse not to swing by and see us.
Attunity Launches Streaming Data Pipeline Solution for Data Lakes on Amazon Web Services
A Confluence of Serendipitous Events
This year’s event couldn’t have been more serendipitous. Amazon announced a major revamp of its data lake offerings and Attunity announced a complementary solution that kicks your AWS data lake initiatives into an even higher gear. We didn’t collude with Amazon to make those events coincide, but we “love it when a plan comes together.”
Our work began about 18 months ago when we started to hear stories from our customers about the current state of data lake technology. At the time, many were using Attunity Replicate to ingest and update data in their lakes. And while many customers had tremendous success running data lakes on AWS for new analytics initiatives, virtually all were experiencing growing pains as their implementations matured. Many had simply “hit the wall” when it came to data lake operational productivity and most lakes were teetering on the edge of becoming swamps.
We decided to investigate this phenomenon and discover whether the data lake stories were just a few isolated incidents or part of a larger market trend. The answer was clear after a few weeks of digging. Ironically data lake ROI starts to decline as implementations matured (typically productivity increases as you become more experienced with a technology). As more data entered the lake, many customers discovered that finding and extracting data for analytics took longer and longer. It was clear that they needed a solution to help them quickly and easily extract data sets that could be automatically updated and used further downstream for analytics.
It seemed like an opportunity we couldn’t pass up. Take our years of experience with data warehouse productivity solutions and apply this knowledge to data lakes. We chose to engineer a solution that helps increase data lake ROI by improving operational productivity, and a roadmap was developed that culminated in a general availability announcement at re:Invent 2018.
Attunity for Data Lakes
On Monday, November 26th, Attunity announced a new offering designed to automate real-time change data capture, data delivery and refinement for analytics. For more information on Attunity for Data Lakes, click here.
The Attunity for Data Lakes solution automates the creation and deployment of data pipelines that help data engineers successfully deliver a return on their existing investments. With Attunity’s no-code approach, data professionals can implement data pipelines in days, instead of months, ensuring the fastest time to insight for accurate and governed transactional data. In a nutshell, if you have or are looking to build a data lake on AWS, then you should consider adding the Attunity solution to your data lake arsenal.
Attunity for Data Lakes is a solution that establishes and automates the entire data lakes pipeline process from initial ingestion to data-mart provisioning. Also, metadata is added at every stage to ensure data lineage and improve data confidence. The concept can be visualized as follows:
The “Landing Zone” is where raw data is continually ingested into a data lake from a variety of data sources. The “Assemble Zone” is where data is standardized, repartitioned and merged into a transformation-ready store. The final zone, called the “Provisioning Zone” is where enriched data subsets are generated and ready for consumption by data analysts or scientists. Incidentally, the metadata visualizations in the solution promote confidence in the data sets because analysts can see where the data set originates. And similarly, data engineers can clearly see the data lineage for any data attribute, collection or set.
Attunity for Data Lakes is a great complement for any data lake that runs on AWS infrastructure, and helps data engineers improve the operational efficiency as well as increase ROI of their data lake. For more information regarding Attunity for Data Lakes and to watch a demo, visit the solution webpage here.
It’s definitely a busy week for Attunity. We are participating in AWS re:Invent 2018, one of the largest tech events of the year, and announced a fantastic new streaming data pipeline solution for data lakes that runs on Amazon Web Services. So if you’re planning a new data lake or looking to re:Invent your existing solution, then feel free to contact us with questions or to schedule a demo of this exciting new solution.