Pipeline

Chain operations into a visual transformation pipeline.

Overview

The pipeline is the horizontal strip of cards at the bottom of the screen. Each card represents one transformation step that you have applied to your data. Together, the cards form a chain of SQL operations that transform your raw file into your desired output.

How It Works

Every time you apply an operation (Filter, Join, Add Column, etc.), a new card is added to the pipeline. Behind the scenes, each card generates a SQL statement. The pipeline composes all cards into a chain of SQL CTEs (Common Table Expressions), where each step feeds into the next.

-- Simplified view of how the pipeline builds SQL
WITH step_1 AS (
  SELECT * FROM raw_data WHERE status = 'active'
),
step_2 AS (
  SELECT *, revenue * 0.1 AS tax FROM step_1
),
step_3 AS (
  SELECT * FROM step_2 ORDER BY revenue DESC LIMIT 100
)
SELECT * FROM step_3

Pipeline Card Anatomy

Each card displays three pieces of information:

  • Operation name - the type of operation (e.g., Filter, Math, Join).
  • Summary - a brief human-readable description of what the step does.
  • Generated SQL - the exact SQL that will execute for this step.

Editing the Pipeline

You can interact with pipeline cards in three ways:

ActionHowWhat Happens
EditClick the cardRe-opens the operation panel with the card's current settings. Make changes and click Apply to update.
DeleteClick the X buttonRemoves the step entirely. Subsequent steps re-execute without it.

When you edit an earlier step, the pipeline automatically rebuilds from that step onward. All downstream cards re-execute with the updated data.

Natural Language Input

You can type a plain-English description of what you want into the pipeline input, and the AI will generate a SQL step for you. For example:

  • "Remove rows where revenue is negative"
  • "Add a column that calculates profit margin as revenue minus cost divided by revenue"
  • "Group by region and sum the sales column"

This requires an API key to be configured in Settings.

Tips

  • Preview the SQL at each step by clicking a card - this helps you understand and debug your transformations.
  • Start with Filter operations to reduce your data, then add Transform and Aggregate steps.
  • You can build complex, multi-step workflows that are easy to understand and modify.
  • See the Building a Pipeline guide for a step-by-step walkthrough.