Build a pipeline, share it as a link

No run button, and a count under every step

Nothing here waits for you to press anything. Change a dropdown, type in a box, drag a step to a new position, and the chain recomputes from the top. Text inputs are debounced so it does not thrash while you type, and that is the only concession.

What makes that useful rather than merely responsive is the line under each step:

1. Clean headers    7 headers rewritten · style: snake
2. Trim whitespace  23 cells trimmed · runs of spaces collapsed
3. Remove duplicates  1 duplicate dropped · 7 rows kept · matched on every column
4. Normalize dates  7 dates rewritten · format: iso · columns: close_date

Every step says what it did in the units that matter for that step. This is the thing that turns a pipeline from a black box into something you can reason about. A dedupe step reporting 4,182 duplicates dropped on a file you thought was clean is a finding. A trim step reporting 0 cells trimmed tells you the whitespace theory was wrong and to look elsewhere. Without the counts you would see only the final row count and have no idea which step accounted for the difference.

A step that fails stops the chain at that point and shows its error in place, so you know which step broke rather than being told the pipeline failed. The steps above it still show their results, because they still ran.

The calculated column, and why it is not eval

The expression step handles arithmetic, comparison, text and conditionals:

price * qty                         quantity times price
round(price * 1.2, 2)               with tax, to the penny
first .. " " .. last                joined with a space
if(amount > 1000, "large", "small") a bucket
upper(trim(`Email Address`))        normalized, backticks for spaces
coalesce(nickname, first_name)      first one that is not empty
left(order_date, 7)                 the month, for grouping

Functions include if, round, abs, floor, ceil, min, max, upper, lower, trim, len, concat, left, right, contains and coalesce. Column names with spaces go in backticks. Division by zero produces a blank rather than infinity, which is the honest CSV version of a spreadsheet's error cell.

This is a small hand-written parser, not eval, and that matters because a pipeline arrives as a link from someone else. The parser knows about numbers, strings, column names, the operators above and those sixteen function names, and nothing else. There is no path from an expression in a shared link to the page's JavaScript, to your clipboard, or to a network request. A column lookup reads own properties only, so an expression naming constructor gets an empty cell rather than a JavaScript function.

The competitor's equivalent accepts only + - * / ( ) over numeric columns: no functions, no comparisons, no string concatenation, no conditionals. That is the whole difference between a calculator and something you can build a bucket or a normalized email with.

Where a pipeline ends

The result at the bottom downloads as a CSV, or as a zip of files if the last step split it. Two other exits are more interesting.

Build a dashboard from this sends the finished result to the dashboard builder. That is the whole point of cleaning something: a chart of the raw export would have counted ACME Corp and acme corp as two customers.

Open in Data Explorer sends the result to the full application, which has the grid, the chart builder, SQL and its own pipeline. Under the button is a line saying how much of your chain has an equivalent there: most steps map exactly onto one of the app's own operations, a few arrive approximately, and two have no equivalent at all. The mapping is written down in src/quickconvert/pipeline/opMapping.ts with a caveat against every partial one, and a test keeps it honest as both sides change.

Whichever exit you take, the transformed data travels rather than the raw file, so you land looking at the result you just built.

Frequently Asked Questions

Related