Kdb+ Architecture and Administration

Learn industry best practices for architecting and maintaining an enterprise wide Kdb+ data solution. You will learn how to

  • Implement security
  • Load balance queries across multiple machines
  • Reduce storage costs using compression.
  • Recover from a crash (logging / checkpointing)
  • Speed up existing queries and best practices for structuring data

Contact us now to request Training

Course Content

Day 1

FunctionsRecap of Functions, Dictionaries, Lists etc
Best practices for writing modular code for large scale kdb systems.
TablesDefining, updating, deleting, adding columns, sorting.
Creating relationships between tables using enumeration and Foreign Keys
Inter-Process Communication Communication between servers, sending data (a)synchronously, buffering
Security / Stream processing - by overriding event handlers
Load Balancing - distributing real time and historical queries over multiple machines
Stitching together real-time / historical data using a Gateway
qSQLStandard SQL queries: "select col from table where ..", what's unique to qSQL

Day 2

JoinsCovers the standard SQL joins left-join, inner-join, union-join.
Specialized kdb joins: plus-join, equi-join, asof-join
Optimizing joins and queries using attributes
Multithreadingbuilt-in threading, Parallel processing. (Screen AvailableParallel screencast available)
Disk StorageText Output and Standard serialization - for saving small files
Splaying / Partitioning - For storing large amounts of historical data
Segments - For storing extremely large data sets and processing them efficiently in parallel
Advanced AnalyticsBuilding on our knowledge of qSQL and disk storage
Efficiently constructing analytics e.g. VWAP / TWAP

Day 3

Importing LibrariesCreating C DLL's and calling that code from kdb
C / Java Interfaces Calling kdb from c/Java (Screen AvailableJava API screencast available)
Logging / Check pointingCrash recovery, Transactions and rolling back
Memory Managementkdb's memory model, garbage collection (Screen Available Memory tutorial available)
CompressionHow can data be compressed, when to use it and which data to use it on.
Extremely large Databases with many coresWhat is happening below the covers,
Best practices for constructing map-reduce queries and edge cases to be careful of.


This class is a good fit if you have some experience in kdb+ before and are looking to increase your skills to the level where you could be responsible for administering complex kdb+ deployments.

Contact us now to request Training

Other courses we offer include:

If you're unsure which is more suitable please contact us.