CSV Summary

What you get back

Opening an unfamiliar export is mostly reconnaissance. How many rows are there really, once the trailing blank lines are gone? Which columns are worth grouping by? Is there a date, and how far does it run? Did somebody's join duplicate half the rows? Is the amount column full of numbers or full of numbers with currency symbols glued to them? Those questions have exact answers, they are cheap to compute, and there is no reason to answer them by scrolling.

This page answers all of them at once. The file is read into DuckDB compiled to WebAssembly, and then a fixed list of queries runs against it. Not a sample of it. Every row, every time, which is why the totals here are figures you can put in a report rather than estimates you have to caveat.

The report has five sections. Headline figures gives the row and column counts, the total and average of the leading metric, the number of missing values as a count and as a share of all cells, the number of exactly duplicated rows, and the date range if the file has a date column. What the numbers say gives three plain statements, each one a restatement of a figure already calculated: which group leads and by how much, the gap between the leading and trailing group, and whether the last period is above or below the first. Data quality lists every column that is missing values, every column that holds the same value in every row, and the count of repeated rows. Columns is one row per column with its DuckDB type, the role the engine inferred, its missing count, its distinct count and its minimum and maximum. Values shows the top values of up to three groupable columns with counts and percentage shares.

A worked example

Press Try an example and a twelve-row order file loads: a date, a region, a channel, a unit count and an amount, with one unit count deliberately blank. The summary comes back with twelve rows and five columns, a total amount, an average amount, one missing value at 1.67 percent of cells, zero duplicate rows, and a date range from 2024-01-08 to 2024-06-23.

The findings name the leading region with its share of the total, the gap to the trailing region, and the direction of travel across the months. The quality section says exactly which column is short a value and how many. The column table shows amount as DOUBLE with the role metric, region as VARCHAR with the role dimension, and order_date as DATE. The value sections show how the twelve rows split across region and across channel.

Now scale that up. Give it a hundred thousand rows and the shape of the output is identical, the numbers are still exact, and it still takes about a second once the engine is warm. The output is Markdown, so it goes straight into a pull request describing a data change, a ticket asking somebody why a column is half empty, or the top of the notebook you are about to write.

Why it never guesses

The one piece of inference here is deciding what each column is for, and the rules are deliberately dull. A numeric column is a metric unless its name reads like a key: id, code, reference, postcode, phone, year. Summing order_id produces a real number and a meaningless one, so it is not offered. A text column with a manageable number of distinct values is something to group by; a text column with a different value on nearly every row is an identifier and gets left alone. Dates are recognised by type, and by name when a CSV has stored them as text.

Every role the engine assigned is printed in the column table, so when it reads something the way you did not intend, you can see that immediately rather than wondering why a figure looks odd. And because the summary is built from the same intents the Ask page uses, anything you disagree with can be re-asked in words and re-checked against the SQL.

Where it fits

Reach for the CSV Profiler when the question is about the columns: what types are in there, what the ranges are, how many distinct values, where the nulls are hiding. Reach for this page when the question is about the file: what does it total, who leads, is it going up, and is it clean enough to report from. Reach for Ask CSV when you have a specific question and want the number with its working shown, or when you want to pin several answers into a report. Reach for the full app when the answer needs a pipeline: filters, joins, derived columns, charts you configure yourself.

Questions and answers

Related