Skip to content

Project

Apache Parquet

Apache Parquet is an open-source columnar storage file format optimized for compression and query efficiency, widely used in big data and analytics systems.

Known aliases

  • Parquet
  • Parquet 2.11

Current stories

buildOne report1 publisher

Iceberg 1.11.0 puts Spark, Flink and Hive file access behind one format registry

Iceberg 1.11.0 ships a File Format API that sends Spark, Flink, Hive and plain-Java reads and writes through one registry of format models. A dev.to walkthrough says the file format list stays fixed, so Vortex and Lance still need a change inside Iceberg.

Publishers:dev.to

Reality

Evidence55
Adoption
Not scored
Hype gap−10
Incentives
Not scored
Confidence50
buildOne report1 publisher

Infino bets agent builders will collapse their data stack into one SQL layer over Parquet

Infino, from OpenSearch veterans, launched a platform letting agents query one Parquet copy in object storage in place of four separate data systems. The pitch is fewer pipelines and one place to set access policy, and so far it rests on the CEO's account of how agents query.

Publishers:thenewstack.io

Reality

Evidence25
Adoption
Not scored
Hype gap+45
Incentives80
Confidence30
buildOne report1 publisher

QuerySurge 15 ties data-mapping documents to the tests that enforce them

QuerySurge 15 links data-mapping documents to the validation tests its AI generates from them and graphs which tests a mapping change affects. The company has not said how accurate those tests are, so full coverage can still include wrong tests.

Publishers:runtimewire.com

Reality

Evidence35
Adoption
Not scored
Hype gap+20
Incentives70
Confidence40
buildOne report1 publisher

Aurora's aurora_analytics extension prints DuckDB operators in its EXPLAIN output

A dev.to walkthrough found DuckDB's operator names in the EXPLAIN output of AWS's aurora_analytics extension, on a Parquet query that executed in 81 ms. That run hit a warm cache on one 500,000-row file, so it tells you which engine ran far more than how fast it will be on lake-scale data.

Publishers:dev.to

Reality

Evidence60
Adoption
Not scored
Hype gap+5
Incentives
Not scored
Confidence55
buildOne report1 publisher

Apache Iceberg moved lock-in up to the catalog layer, an AWS Community Builder argues

AWS Community Builder Aki argues Apache Iceberg made data portable while permissions, auditing, lineage and sharing stay tied to each catalog. The spec leaves the atomic commit to each catalog, so the catalog is now the choice that is hard to reverse.

Publishers:dev.to

Reality

Evidence55
Adoption
Not scored
Hype gap+10
Incentives30
Confidence50
buildOne report1 publisher

A Bootstrap stylesheet in the docs folder sank Iceberg's first 1.12.0 candidate

Reviewers logged 10,991 passing tests on Iceberg's first 1.12.0 candidate before rejecting it over two vendored web files. The change most likely to break a downstream build is a removed partition-statistics helper.

Publishers:dev.to

Reality

Evidence60
Adoption35
Hype gap+12
Incentives
Not scored
Confidence58
buildOne report1 publisher

Choosing Iceberg over Snowflake moves six governance components onto your build list

Keeping the Iceberg table format governance-unaware was a deliberate call. That leaves the catalog as the only thing standing on the path of every read. A dev.to guide lists six pieces an operator has to source above it.

Publishers:dev.to

Reality

Evidence42
Adoption25
Hype gap+22
Incentives74
Confidence45
productOne report1 publisher

DuckDB-Wasm writes a real .duckdb file with a WAL into the browser's OPFS

A one-line change to the open call leaves a real database file in the browser, with a write-ahead log and checkpoints that survive a restart. Getting it to persist depends on which DuckDB-Wasm build you installed.

Publishers:duckdb.org

Reality

Evidence64
Adoption
Not scored
Hype gap−8
Incentives58
Confidence57
buildOne report1 publisher

Deleting one Parquet file from your own bucket corrupts Snowflake's Iceberg catalog

A dev.to post argues that Snowflake-managed Iceberg tables let you keep the managed engine while owning the Parquet in your own bucket. The same post says the catalog stays Snowflake's, and that 50,000-inserts-per-second ingest should not move.

Publishers:dev.to

Reality

Evidence28
Adoption
Not scored
Hype gap+45
Incentives40
Confidence52
buildOne report1 publisher

Collapsing three copies into one Iceberg table moves the conflict into an HTTP 409

A GCP walkthrough put Spark, Trino and BigQuery on the same orders table through Google's Iceberg REST catalog. The 409 ConflictException it logged is the part of the demo a production pipeline has to plan for.

Publishers:dev.to

Reality

Evidence38
Adoption18
Hype gap+20
Incentives35
Confidence45
productOne report1 publisher

Keewano's $12M agent database asks for a second copy of your event stream

KeewanoDB keeps events in the order they arrived, grouped around a customer or device, so an agent can read one entity's history without an ETL job. Keewano expects the warehouse to stay where it is.

Publishers:siliconangle.com

Reality

Evidence33
Adoption14
Hype gap+34
Incentives79
Confidence56
buildOne report1 publisher

A million serverless briefings for $48 implies a Lambda rate of $0.000004 per GB-second

A dev.to walkthrough puts a quantized Llama 3.2 3B on up to 10,000 concurrent Lambdas and writes a million daily briefings for $48. The timing math is internally consistent. The per-GB-second price behind the $48 is unstated.

Publishers:dev.to

Reality

Evidence24
Adoption
Not scored
Hype gap+55
Incentives60
Confidence55
buildOne report1 publisher

Fabric mirroring bills continuously for a Snowflake source that changes three times a day

Microsoft documents no configurable mirroring window, and support confirmed that restarting a mirror reloads the whole table instead of resuming from the last CDC position. The estate here has roughly 2,500 tables.

Publishers:dev.to

Reality

Evidence60
Adoption35
Hype gap−10
Incentives30
Confidence55
buildOne report1 publisher

Splitting log templates from variables cut Uber's HDFS bill to $10K a year

Uber reported 169:1 on 200TB of Spark logs a day by separating static templates from variable fields before anything was written to HDFS. Whether that ratio transfers depends on how few call sites your lines come from.

Publishers:dev.to

Reality

Evidence45
Adoption32
Hype gap+20
Incentives
Not scored
Confidence48
buildOne report1 publisher

Vortex streams 13 Gbit/s of columnar data from S3 into GPU memory without an NVMe hop

The figure comes from a conference demo scanning a 4K video, and the same talk sets a target closer to 50 Gbit/s. The reason today's loaders still stage shards on disk turns out to be PyTorch's worker model.

Publishers:infoq.com

Reality

Evidence42
Adoption
Not scored
Hype gap+28
Incentives72
Confidence46
buildOne report1 publisher

Iceberg parks its kilobyte statistics in a Puffin sidecar the planner opens on demand

A distinct-value sketch is thousands of bytes and a delete bitmap has no fixed size, which is why Iceberg keeps both in a separate file, and why a bad join plan is a statistics problem rather than a rewriting one.

Publishers:dev.to

Reality

Evidence62
Adoption25
Hype gap+12
Incentives50
Confidence55
buildOne report1 publisher

Iceberg v3 moves the coordinate reference system out of the column comment and into the schema

Native geometry and geography types in format v3, with matching Parquet 2.11 logical types underneath, give a shape column its own bounding boxes and a CRS that travels with the data. The second half is the part worth migrating for.

Publishers:dev.to

Reality

Evidence42
Adoption28
Hype gap+18
Incentives58
Confidence48
buildOne report1 publisher

HFlow separates the black-frame measurement from the threshold that rejects the episode

Hebbian Robotics, a two-person YC Summer 2026 company, announced HFlow on August 19 and shipped it to PyPI the next day, betting that robot dataset checks and provenance are worth sharing between teams rather than rebuilding.

Publishers:runtimewire.com

Reality

Evidence57
Adoption6
Hype gap+9
Incentives73
Confidence52
buildOne report1 publisher

Six specifications decide whether an agent can move off the harness it was built on

Lock-in migrated from the model to the runtime that holds your tool grants, approval rules and learned state. Six specifications aim to make that runtime's contents portable, and half of them are one person's drafts.

Publishers:dev.to

Reality

Evidence36
Adoption24
Hype gap+27
Incentives74
Confidence42

Earlier coverage