Avro to Parquet Converter

The two formats a data platform is built on, converted without a cluster. The Avro schema is read in the page and DuckDB writes the Parquet with real column types.

To convert Avro to Parquet, drop your .avro file above. The container's embedded schema is read in your browser, nested records flatten into dotted columns, each column is typed from its values and the declared schema, and DuckDB compiled to WebAssembly writes the Parquet file. Choose Snappy, Zstd or no compression.

Want to filter records before writing? Open the app

Two formats that solve different halves of the problem

Avro is row-oriented and built for writing. Records append cheaply, the schema travels with the data, and a consumer reading the stream in order is exactly what it is good at. That is why message pipelines land in Avro.

Parquet is columnar and built for reading. A query touching two fields out of forty reads two columns, and a repeated category value costs almost nothing because it is stored once per column chunk rather than once per record.

So the conversion is a lifecycle step rather than a fix: data arrives as Avro because it was streamed and is stored as Parquet because it will be queried. Doing it once, at the boundary, is worth it on every query afterwards.

What normally does this, and why not here

In production this is a Spark job, a Flink sink or a small script using pyarrow. All three are correct and all three assume a cluster or a Python environment.

That leaves a real gap for the one-off: a sample file to inspect, an extract a partner sent, a topic dump you want to explore before deciding whether to build anything. Standing up a Spark session for one file is a bad afternoon.

Everything here runs in the page. The Avro container is decoded by JavaScript and DuckDB compiled to WebAssembly writes the Parquet, so the whole conversion happens on a machine with no JVM, no Python and no cluster access.

The schema, and how it becomes column types

The Avro header names every field and its type, and those map onto Parquet directly:

long and int become BIGINT, so identifiers and counts stay exact.
double and float become DOUBLE.
boolean becomes BOOLEAN.
string becomes VARCHAR, which is what keeps a padded reference code intact.
A date logical type becomes DATE where the reader recognises it, which is what makes a date predicate push down at query time.
A union of null and a type becomes that type, nullable, which is how Avro spells an optional field.

The types are handed to DuckDB explicitly rather than left to be sniffed a second time, so the Parquet schema is exactly what the preview showed you and no column can be silently re-typed on the way out.

Flattening, and what it costs

Nested Avro records flatten into dotted columns. Parquet can represent nesting natively and this deliberately writes flat columns instead, which is worth being explicit about.

Flat columns are what query engines are fastest on, what every BI tool can see with no configuration, and what a person reading the schema can understand. Nested Parquet is more faithful to the Avro and materially harder to work with afterwards.

Array fields and nesting past four levels are kept as JSON text in a VARCHAR column, which DuckDB and most engines can still extract from. The row count always equals the record count, which every aggregate computed later depends on.

Nothing is uploaded, and that is the point

The hosted converters for these formats all want the file. That is a fine trade for a photograph and a poor one for a database extract, which is what these files almost always are: customer tables, transaction detail, event streams with identifiers in them.

Everything here runs inside the page. The file is read by JavaScript in your tab, the output is built there, and the download comes out of memory. Nothing is sent, stored or logged, and once the page has loaded the tool works with the network off.

The practical ceiling is your tab's memory rather than an upload quota. Past a hundred megabytes or so the full editor streams and is the better route.

Frequently Asked Questions

Do I need Spark, Java or pyarrow?

None of them. The Avro container is decoded by JavaScript and DuckDB compiled to WebAssembly writes the Parquet, both inside your browser tab. That is the gap this fills: the one-off conversion where standing up a cluster is disproportionate.

Where do the Parquet column types come from?

The Avro container's embedded schema. A long becomes BIGINT, a double becomes DOUBLE, a string becomes VARCHAR, and a union of null and a type becomes that type nullable. The types are declared to DuckDB explicitly so nothing is re-sniffed.

Is the nesting preserved as nested Parquet types?

No, it is flattened into dotted columns. Parquet supports nesting, and flat columns are what query engines optimise for and what BI tools can see without configuration. Arrays and deep nesting are kept as JSON text in a VARCHAR column.

How much smaller will the Parquet be?

Usually substantially, though it depends on the data. Avro stores each record's values together and Parquet stores each column together, which lets a repeated category value compress to almost nothing rather than being written once per record.

Will the row count match?

Exactly. Array fields are kept as text rather than being exploded into extra rows, because a row count that differs from the record count invalidates every aggregate computed on the file afterwards.

Is the file uploaded?

No. Both halves of the conversion run in your browser tab and the download comes from memory. Avro files are usually topic dumps or extracts, which is why this is the deciding factor for most people using it.

Move the stream into storage

Schema from the container, typed columns, compressed, all in your browser.

Back to the converter