Automating Data Transformations
In the past decade, the Modern Data Stack emerged, evolved, and enabled many organizations to ingest, store, and analyze more data than ever.
Automation enabled efficiency and scale across the data cycle, with one exception: data transformations.
To transform data, many organizations still use inflexible GUI tools or rely on manual code-first solutions that require the work of highly skilled engineers. Neither of those options scales.
In this report, we present a way to combine the flexibility of code-first with the ease-of-use of GUI to enable scale through automation.
You will learn:
- How to optimize the transformation layer with Data Architecture as a Service (DAaaS) and Data as a Product (DaaP).
- How using metadata at the column level enables automation and revolutionizes data transformations.
- How to create trust in your data and data teams across your entire organization.
Table of Contents
- Today’s Modern Data Stack
- What Is the MDS?
- Managing Data as a Product (DaaP)
- Basic Terms and Concepts in the MDS
- Automation in the MDS
- A Renaissance in Data Transformation
- Why Data Transformations Matter
- Data Transformation: Existing Solutions
- SQL Plus Orchestration Tooling
- Code-First
- GUI-First
- Data Transformations: Finding the Golden Middle
- Hybrid Approaches
- Delivering Value with Data Transformations Through Automation
- Principles of Data Value
- Product-First
- Column-First
- Optimizing the Transformation Layer
- Enabling Analytics at Scale
- DAaaS
- Culture Shift
- Democratizing Data Transformation
- Implementation
- Principles of Data Value
About the Authors
