Description Open Source OLAP Database originating from Yandex. Older proprietary closed source database only popular in Finance.
Architecture Columnar database - MergeTree Based Columnar database - Array Based
Released June 15, 2016 2003
Scaling Horizontally Scales to more machines. Vertically Scales to run on one big machine.
Language Mostly ANSI SQL Custom qSQL and language q
License Apache 2.0 Closed Source
Use Cases Real-time analytics, advertising, event logging, monitoring, IoT, data warehousing High-frequency trading, financial services, market data analysis.
See who uses kdb+.

Technical Features Compared

Tickers = Low Cardinality Strings Enum Symbols
Nanosecond Support Yes. DateTime64 Yes - timespan/timestamp
Asof Joins ASOF JOIN Aj
Window Joins No. wj
Latest Value on Time last_value with Window last
Previous / Next with offset No. Alternative is ROWS BETWEEN prev next
Moving Averages / Running Functions Window Functions mavg
Time Bars toStartOfMinute xbar
Generated Queries No. Functional Select
Python Integration Client Only PyKX customized pip + license

About ClickHouse and History

ClickHouse is an open-source column-oriented database management system (DBMS) developed by the Russian company Yandex. It is designed for analytical processing of large volumes of data, particularly for OLAP (Online Analytical Processing) workloads. ClickHouse is optimized for high-performance queries on data sets that range from gigabytes to petabytes in size.

ClickHouse Beginnings:

  1. Origin: ClickHouse originated within Yandex, one of the largest internet companies in Russia. Yandex needed a powerful and scalable database solution to handle the vast amounts of data generated by its various services, including search, advertising, and analytics.
  2. Development: ClickHouse was developed internally at Yandex to meet their specific requirements for fast and efficient analytics on large datasets.
  3. Open Sourcing: Recognizing the potential value of ClickHouse beyond its internal use, Yandex decided to release it as an open-source project in 2016.
  4. Community Growth: Since its open-source release, ClickHouse has gained popularity among companies seeking a high-performance and scalable database solution for analytics.

About kdb+ and History

Kdb+ is a high-performance, in-memory, column-oriented database developed by mostly one man: Arthur Whitney and based on APL.

  1. Foundation: Kx Systems, founded in 1993 by Arthur Whitney and Janet Lustgarten, created the kdb+ database system. Initially, it was developed to meet the demands of financial institutions for handling and analyzing large volumes of time-series data in real-time.
  2. Early Development: In the late 1990s and early 2000s, kdb+ gained popularity in the financial industry due to its speed and efficiency in processing and analyzing vast amounts of tick data and other financial data types. Its column-oriented architecture and integrated query language (q) were particularly well-suited for time-series analytics.
  3. See also full history of kdb+

What's next

ClickHouse at DB