O’Reilly Report

Automating Data Transformations

In the past decade, the Modern Data Stack emerged, evolved, and enabled many organizations to ingest, store, and analyze more data than ever.

Automation enabled efficiency and scale across the data cycle, with one exception: data transformations.

To transform data, many organizations still use inflexible GUI tools or rely on manual code-first solutions that require the work of highly skilled engineers. Neither of those options scales.

In this report, we present a way to combine the flexibility of code-first with the ease-of-use of GUI to enable scale through automation.

You will learn:

  • How to optimize the transformation layer with Data Architecture as a Service (DAaaS) and Data as a Product (DaaP).
  • How using metadata at the column level enables automation and revolutionizes data transformations.
  • How to create trust in your data and data teams across your entire organization.


Table of Contents

  • Today’s Modern Data Stack
    • What Is the MDS?
    • Managing Data as a Product (DaaP)
    • Basic Terms and Concepts in the MDS
    • Automation in the MDS
  • A Renaissance in Data Transformation
    • Why Data Transformations Matter
    • Data Transformation: Existing Solutions
      • SQL Plus Orchestration Tooling
      • Code-First
      • GUI-First
    • Data Transformations: Finding the Golden Middle
      • Hybrid Approaches
  • Delivering Value with Data Transformations Through Automation
    • Principles of Data Value
      • Product-First
      • Column-First
    • Optimizing the Transformation Layer
      • Enabling Analytics at Scale
      • DAaaS
    • Culture Shift
      • Democratizing Data Transformation
      • Implementation

The truth about automation, repeated throughout history, is that it does not eliminate work. It merely shifts the type of work we do, often to something more challenging and enjoyable. If it now takes three analysts to build infrastructure instead of ten, the other seven will still have plenty to do—focusing on higher-level tasks like implementing new patterns, optimizing design, and tackling new creative problems that were previously consumed by manual process and day-to-day operations.

About the Authors

Satish Jayanthi
CTO and Co-Founder, Coalesce

As Co-Founder and CTO at Coalesce, Satish has designed and built the company’s data automation software. Prior, Satish was the Sr. Solutions Architect at WhereScape, a leading provider of data automation software, where he met his co-founder Armon.

Armon Petrossian
CEO and Co-Founder, Coalesce

Armon Petrossian is the CEO and Co-Founder of Coalesce. Prior to that, Armon was part of the founding team at WhereScape in North America, where he served as national sales manager for almost a decade.

Ready to try it? Get started today.

Experience Coalesce’s full feature set at no cost; up to 50 nodes, one user, and one workspace for free.

Request a demo

We’d love to walk you through the Coalesce data transformation tool within the context of your business and data transformation needs.

Request a product demo

Contact Us

We’re here to answer any questions you might have about the product, the team, potential partnerships, or anything you might be curious about!

Contact Us