Every Time-Series Database Benchmark Ever. 2024

Updated 10th September 2024
This page will gather ongoing time-series database benchmarks into one location.

The top benchmarks section will focus on time-series databases that are fast, provide time-series joins, time window aggregations and data compression, rather than pull in more general solutions.

Updated 10th September 2024

For me, there are three interesting happenings lately:

1. DuckDB/QuestDB are both now very competitive. (DuckDB in th last year moved 2.7->2.1 in clickbench. QuestDB 24.2->2.7. Lower = better)
2. ClickBench now includes many parquet/partitioned based timings showing the rise in popularity
3. The inclusion and rising popularity of embedded databases: chdb and duckdb

Clickbench

Clickbench is a large suite of benchmarks produced by clickhouse themselves. The focus is not time-series but a wide range of queries. They are being very transparent and open, i.e. on some queries they are beaten but the benchmarks only include open source choices.

Database	Relative Runtime (Lower = Better)
Umbra	×1.30
ClickHouse	×1.72
StarRocks	×1.84
Databend	×1.96
DuckDB	×2.07
QuestDB (partitioned)	×2.74
chDB	×5.04
chDB (Parquet, partitioned)	×5.17

Note query 23/28 were removed to get these numbers as the text/regex queries are not of great interest to us.

Top Time-series Benchmarks

To be considered a top benchmark, a combination of factors including repeatability, open-source, reputation, comprehensiveness, thoroughness were considered. This narrowed the top benchmarks to:

1. benchmarking 4 queries on 1.1 Billion NYC Taxi and Uber Trips - by Mark Litwintschik's. Uses different hardware but thorough writeup of each database features.
2. Clickbench - 30+ Databases with 42 queries benchmarked. Only includes open-source but highly reproducible.
3. STAC-M3 - Non-open costly commercial benchmark
4. H2O.ai - reproducible benchmarking of database-like operations in single-node environment.

Updated Aug 14, 2023

The latest updates were all vendor customized benchmarks for ingestion, each of the vendors benchmarks showed themselves as the fastest at ingestion: questdb, GridDB, TDEngine.

Taxi Ride Benchmarks

Results reproduced from Mark Litwintschik's excellent article.
Mark benchmarked 4 queries against a 1.1 Billion NYC Taxi Trips data set. It had 51 columns was 500 GB in size when in uncompressed CSV format

STAC-M3

STAC-M3 are highly commercial and results are not published or reproducible. Typically each vendor pays to perform a STAC-M3 benchmark and at the time they use the fastest available hardware humanly possible and a highly optimized configuration. making results extremely difficult to compare. They are however as far as we know, extremely thorough.

Below is a rough overview of the winners doing press releases by year:

h20.ai Reproducible Benchmarks

Aims to benchmark various database-like tools popular in open-source data science. It runs regularly against very latest versions of these packages and automatically updates. They provide this as a service to both developers of these packages and to users. You can find out more about the project in Efficiency in data processing slides and talk made by Matt Dowle on H2OWorld 2019 NYC conference.

All Other Time-series Benchmarks by Date

Rows marked in the below table mean they are produced by a vendor themselves and that you should assume they have chosen a benchmark to highlight their best possible performance. If you want a benchmark added please contact us.

Date	Benchmark	Summary	Databases
2023-05-16	Vendor TSBS Benchmark	1M rows per sec loading on customized TSBS benchmark by QuestDB. Vendor chosen customized benchmark.	GridDB, QuestDB, TimescaleDB, ClickHouse
2023-05-02	Vendor TSBS Benchmark	300K rows per sec loading on customized TSBS benchmark by GridDB. Vendor chosen customized benchmark.	GridDB, QuestDB, TimescaleDB
2023-04-14	H2O.ai Database-like Ops Benchmark	DuckDB updates existing H20.ai benchmark. Vendor chosen customized benchmark.	DuckDB vs Pandas vs Clickhouse
2023-02-20	Vendor TSBS Benchmark	Ingestion and query performance using customized TSBS Benchmark by TDEngine. Vendor chosen customized benchmark.	InfluxDB vs TimescaleDB vs. TDengine
2023-02-15	Academic Research	benchmarking specialized databases for high-frequency data kdb Paper says benchmark was to be open source but can't find it. Source requested from authors.	ClickHouse, InfluxDB, kdb+, TimeScaleDB
2023-01-25	ClickHouse vs PostgreSQL	Analyse billions of youtube video metrics. Clickhouse 5-10x faster than postgres for analytic queries	ClickHouse, PostgreSQL
2023-01-17	SigNoz	SigNoz is an observability platform based on ClickHouse. Here they perform 500GB log analysis mostly using aggregation queries. Note: SigNoz / Clickhouse won but this is a vendor self supplied benchmark.	ClickHouse, ElasticSearch, MinIO
2022-06-08	Academic Paper	SciTS: A Benchmark for Time-Series Databases in Scientific Experiments and IoTs "Click-House supports very high ingestion rates up to 1.3 million records...ClickHouse significantly outperforms other evaluated databases in the speed of data queries and shows reasonably low deviation in query latency."	ClickHouse, InfluxDB, TimescaleDB, and PostgreSQL
2022-05-26	QuestDB Blog	4Bn rows/sec query benchmark: Clickhouse vs QuestDB vs Timescale Vendor chosen benchmark to emphasise a very particular scenario.	Clickhouse, QuestDB, Timescale
2022-04-05	KX Blog	KX v DBOps Benchmark Results Vendor ran benchmark. Claims to have ran fastest 18/19 queries but no speed numbers published.	kdb+
2022-04-02	data-sleek blog	SingleStore vs. ClickHouse Benchmarks "SingleStore offers a much stronger solution than ClickHouse. The performance of the queries when joining tables is obvious — queries were 3-186x faster."	ClickHouse, SingleStore
2022-04-01	Masters Thesis	Evaluating ClickHouse as a Big Data Processing Solution for IoT-Telemetry	ClickHouse, ElasticSearch, Loki
2022-03-14	Altinity Blog	Evaluating Altinity ClickHouse vs Singlestore for loading 100b rows Vendor self-benchmark	ClickHouse, altinity, singlestore
2022-02-15	Gitlab CH Eval	Gitlab APM ClickHouse evaluation - to evaluate ClickHouse as a horizontally scalable datastore for o11y data (metrics, logs, traces). Chose ClickHouse	Clickhouse, CrateDB, MongoDB and TimescaleDB
2022-01-18	Apache Doris	Apache Doris self-published TPC-H benchmarks - Vendor-ran benchmark	Apache-Doris
2022-01-18	DB - Clickhouse vs kdb+	- Internal Deutsche Bank benchmark. Clickhouse fastest	kdb, clickhouse
2021-08-27	duckdb	Database Sorting Benchmark Duckdb wins their own chosen benchmark, clickhouse close.	duckdb, clickhouse, hyper, pandas
2021-07-08	alibabacloud	ClickHouse vs. Elasticsearch Clickhouse. "Elasticsearch is excellent in search scenarios where only a few records are filtered out by the WHERE clause. However, in analysis scenarios of large-scale data ... ClickHouse will have better concurrency performance due to its excellent column-storage mode and vectorized computing..."	ClickHouse, CrateDB, Druid, Hyper, MonetDB, PostgreSQL, SparkSQL, TimescaleDB
2021-04	brandonharris	ClickHouse vs. Redshift - Financial analysis Clickhouse faster.	ClickHouse, Redshift
2021-04	TSBS	Open Source Time Series Benchmark Suite - Started by TimeScaleDB Frustratingly they never seem to have published public results.	Akumuli, Cassandra, ClickHouse, CrateDB, InfluxDB, MongoDB, QuestDB, SiriDB, TimescaleDB, Timestream, VictoriaMetrics
2021	InfluxDB	Vendor selected benchmarks by influxDB themselves. Influx have chosen weak databases to compare against. Frustratingly their results whitepapers require email signup to see. They do claim to be 5x faster than MongoDB.	Elasticsearch, Cassandra, MongoDB, OpenTSDB, TimescaleDB
2020	mgbench	Open source query benchmarks for machine-generated log data by Andrew Crotty - Assistant Professor of Computer Science at Northwestern University. Clickhouse	ClickHouse, CrateDB, Druid, Hyper, MonetDB, PostgreSQL, SparkSQL, TimescaleDB
2020-09-08	Clickhouse vs Redshift	Performance for FinTech Risk Management.	ClickHouse, Redshift
2019-04-10	Measuring vertical scalability	Measuring vertical scalability for time series databases in Google Cloud by CEO of VictoriaMetrics. Expected result by vendor: "VictoriaMetrics provides the best vertical scalability for both data ingestion and querying. "	InfluxDB, TimescaleDB and VictoriaMetrics
XXXX	db-benchmarks.com	They claim to be publishing fair tests but the entire site and github repo is produced by manticore a vendor. I find it particularly distasteful to say "Fair" but not mention they work directly for one of the companies. Hence why I'm not linking to their site.	ClickHouse, Elastic, Manticore