Percentiles and summary statistics for a CSV

Drop in a CSV and every numeric column comes back with a full summary: count, min, p1, p5, p25, median, p75, p90, p95, p99, max, mean, sample standard deviation, interquartile range and skew. Values that are blank or unreadable are counted separately rather than quietly dropped. It runs in this tab, so nothing is uploaded anywhere.

Only interested in the extremes? Find the outliers instead

A mean on its own is close to useless

Ask any tool for a summary of a numeric column and most of them hand back an average. On a symmetric column that is a fair description. On the columns people actually keep in spreadsheets, it is often the single most misleading number available.

Money, duration, order size, session length, claim value, page weight: they all share a shape. Most of the mass sits low, a thin tail runs a long way right, and the mean is dragged into a region where almost no observation actually lives. Quote it in a meeting and somebody who knows the data will correct you. What you want instead is a handful of positions in the sorted column, plus one number describing how lopsided it is.

The median is the typical value in a way the mean is not, because moving one extreme observation further out does not move it at all.
p25 and p75 mark the middle half. The gap between them, the interquartile range, is the honest measure of how spread out a skewed column is, and it is what a box plot draws.
p90, p95 and p99 are the tail. Nearly every service level target is written against one of these, and a p99 of fourteen seconds is a fact about your worst users that the median cannot express.
p1 and p5 are the other tail, which usually gets ignored and is often where the data-entry problems live: the zero-value orders, the negative durations, the rows that should have been filtered out upstream.
The skew tells you in one number whether the mean is safe to quote.

All of that arrives from one drop, for every numeric column at once, with no configuration.

Worked example: four values, and where p25 lands

Percentile definitions are the reason two tools disagree about the same column, so here is exactly what this one does. Take tiny.csv:

Four values, so n is 4 and n minus 1 is 3. For p25 the position is 0.25 times 3, which is 0.75. That falls three quarters of the way between the value at index 0 and the value at index 1, so the answer is 10 plus 0.75 times (20 minus 10), which is 17.5. The median sits at position 1.5, halfway between 20 and 30, so 25. p75 sits at position 2.25, a quarter of the way from 30 to 40, so 32.5.

Notice that none of those three is a value that appears in the file. That is the interpolating convention doing its job, and it is the same one numpy and Excel's PERCENTILE.INC use. A tool that instead picks the nearest actual observation would report 20, 30 and 30, which are also defensible answers under a different definition. There is no universal right answer here, which is precisely why the page tells you which rule it applied.

Now something more realistic. Here is orders.csv with a blank and a value nobody can parse:

order_id,amount
1001,10
1002,20
1003,N/A
1004,40
1005,

The block for amount reads:

amount
  count         3
  missing       1
  not a number  1
  min           10
  p25           15
  median        20
  p75           30
  max           40
  mean          23.3333
  std (sample)  15.2753
  IQR           15
  skew          0.9352 (slightly skewed right)

Three numbers were summarized, not five, and the report says so on its own line before any statistic. The blank cell in row 1005 is counted as missing; the N/A in row 1003 is counted as not a number, which is a different problem with a different fix. A warning above the table names the column and the count as well, because a reader who skims past the header line still needs to know the denominator changed.

The mean of 10, 20 and 40 is 23.33 while the median is 20, and the skew of 0.935 explains the gap: one value is further from the middle than the others, on the high side. On three observations that number means very little, and the page says so in a warning rather than presenting it with a straight face.

Every field in the summary, and what it is for

count is how many values were actually used. Read it first. Everything below it is conditional on this number.
missing is blank cells, and not_a_number is non-blank cells that would not parse. Splitting them apart matters because they usually have different causes: the first is a gap in collection, the second is a placeholder string somebody typed.
min and max are the extremes as they are, unrounded. A negative minimum in a quantity column is a finding on its own.
p1, p5, p25, median, p75, p90, p95, p99 are the eight positions, all under the interpolating rule above.
mean is the plain arithmetic average, kept because it is what everybody asks for, and placed after the median so it is read second.
std is the sample standard deviation, dividing by n minus one.
IQR is p75 minus p25. Multiply it by 1.5 and add it to p75 to get the upper Tukey fence, which is the number the outlier page uses.
skew is the Fisher-Pearson coefficient with the sample adjustment, printed with a plain-English reading beside it in the report.

The Show control flips the panel between the aligned per-column report and the one-row-per-column table. The download is the CSV either way, because that is the artefact you cannot reproduce by copying what is on screen. Copy hands over whatever the panel is showing.

Which columns get summarized

Leave the column picker empty and every column that reads as numbers is included. A column qualifies when at least nine in ten of its non-blank values parse as a number, which is the same threshold the profiler uses for its numeric-looking note, so a column that appears in one tool's dropdown appears in the other's. That tolerance is deliberate: a price column carrying three "unknown" entries among four thousand rows is still a price column, and refusing to summarize it would be pedantic.

Tick specific columns and only those are summarized, whatever they look like. That override exists for the column that is 60 per cent numbers and 40 per cent junk, where the statistics on the numeric part are exactly what you want to see before deciding what to do about the rest. In that case the not_a_number count is the finding and the percentiles are context.

Columns you pick that turn out to hold no numbers at all are skipped, named in a warning, and do not stop the run. If nothing in the file is numeric the page says so in a sentence rather than handing back an empty table.

Gotchas worth knowing

A percentile on very few values is theatre. p99 of eleven observations is just the largest one wearing a hat. Below three values the page warns that the percentiles and the skew mean very little, and it is worth taking that seriously up to a few dozen.
Leading zeros keep a column out of the numbers. An account reference written 00417 is an identifier, and summarizing it would produce a mean nobody wants. Tick it explicitly if you really do want its statistics.
Currency symbols and thousands separators are not numbers here. $1,200.50 does not parse, and it lands in the not_a_number count. Strip the symbols first, and be aware that an unquoted comma in that value probably also broke the row into an extra column.
The skew needs three values and a non-zero spread. Below that it reports as not applicable rather than as zero, because zero would read as "symmetric" and that is a claim the data cannot support.
Nothing is abbreviated. A range from 2019 to 2023 prints as 2,019 and 2,023, never as "2k to 2k". Two different numbers must never render as the same string.
Nothing leaves the tab. The file is read and sorted by JavaScript in your browser. There is no upload endpoint on this page.

Frequently Asked Questions

Which percentile definition do you use?

Linear interpolation between the two neighbouring order statistics: the value sits at position p times (n minus 1) in the sorted column, and when that position falls between two values the answer is weighted between them. This is what numpy returns by default and what Excel's PERCENTILE.INC computes, so if you check the answer in either you get the same number. There are at least nine other defensible definitions, which is why tools disagree about the same column, and this page names the one it uses rather than leaving you to guess.

Why report p1 and p99 as well as p25 and p75?

Because the quartiles describe the middle and the tails are usually where the problem is. A latency column with a median of 120 milliseconds and a p99 of 14 seconds is a different service from one with a median of 120 and a p99 of 300, and the quartiles cannot tell them apart. p1 and p99 are where the tails actually begin. They are also the numbers most service level agreements are written against.

Is the standard deviation the sample one or the population one?

The sample standard deviation, dividing by n minus one. Your CSV is a sample of something, and that is what every spreadsheet and every statistics course means by the term. It also has to match: the fences on our outlier page are built from this same number, and a summary page that quietly used a different divisor would put the two tools into disagreement about the same column.

What does the skew number mean?

It is the Fisher-Pearson coefficient with the sample adjustment, the same value Excel's SKEW function and scipy's skew with bias set to false return. Zero means symmetric. Positive means a tail stretching to the right, which is the shape of nearly every money and duration column. Negative means the tail runs left. The page also puts it in words: below 0.5 in absolute terms is roughly symmetric, up to 1 is slightly skewed, past 1 is strongly skewed. It is the fastest way to know whether quoting a mean will mislead somebody.

What happens to values that are not numbers?

They are excluded from the statistics and counted in a column of their own. The output carries both missing (blank cells) and not_a_number (non-blank cells that would not parse, such as N/A or unknown), and a warning names the column and the count. A summary computed from 9,600 of a column's 10,000 values is a different fact from one computed on all 10,000, and printing the first as though it were the second is the main way this kind of tool misleads people.

How do I get the summary as data rather than a report?

Switch the Show control from Report to CSV. The download is the same either way: one row per column with count, missing, not_a_number, min, the eight quantiles, max, mean, std, iqr and skew. That file feeds straight back into a pivot, a chart, or a comparison against last month's summary of the same table.

How large a file can this page handle?

Up to about 200,000 rows or 20 MB, the whole file is parsed and summarized in the page itself, which is the fastest thing to do at that size. Past either of those two numbers the file is handed to DuckDB running in your tab instead: it reads the file column by column and returns eight quantiles and six moments per column, rather than holding ten million parsed values in memory the way the in-page path does. The answers are the same either way. The tests run both paths over the same file and require them to agree, down to the ninth decimal place and down to the wording of the warnings, because a reader who crosses a size threshold must not also cross into different numbers.

Does the file leave my computer?

No. This page has no upload endpoint. JavaScript in your tab reads the file, sorts each column and computes the statistics. Nothing is stored between visits, so reloading the page gives you an empty box again.

Summarize every column at once

Free, no account, no upload. One drop, and every numeric column comes back described properly.

Back to the summary