New York kdb Training 2019
Spaces still available. Register now to learn Kdb+.

Kdb database limits and errors

Contents

  1. Table of Limits and Errors
  2. Limitations - Look at each in turn, see what it is and workarounds.
  3. Limits Summary

Table of Limit Errors

You may encounter these error messages when working in kdb, here's what they mean:

limit Attempt to create list longer than allowable maximum
params Too many parameters (8 max)
branch A branch (if;do;while;$[.;.;.]) more than 255 byte codes away
rank Invalid number of arguments
constants Too many constants (max 96)
globals Too many global variables (32 max)
locals Too many local variables (24 max)
mlim More than 999 nested columns in splayed table
glim `g# limit, kdb is currently limited to 99 concurrent `g#'s
stop A client query took longer than the time allowed.
abort The maximum memory that kdb was allowed, was exceeded.

There are also a number of architectural limitations namely:

  • 2 GB IPC object transfer size limit
  • Number of file handles permitted by OS limits all operations with on disk tables.
  • You can only have 56 enums

Limitations

'limit error

The limit error is either caused by trying to create too large a list (2 billion items usually) or trying to transfer an object to/from a kdb process that is over 2GB in size. e.g.:

The 2GB limit is because internally integers are used for storage and addressing and 2^31 is 2,147,483,648. You will sometimes see this error when retrieving a large of results with many columns. The only workaround is to transfer the table in multiple requests.

'constants 'locals 'global error

You have tried to declare too many variables at a given scope. Kdb stores functions as bytecode, this code is only built to handle reasonable limits on the number of variables. If we try to create too many variables at a given scope we get an error, e.g. too many function local variables:

The general solution is to encapsulate data into objects (lists, tables, dictionaries) rather than having many separate variables

'params error

Params error is caused when you try to create a function with more than 8 parameters. To us this sounds like a reasonable tradeoff to make functions readable. If you find your hitting this limit, try to group the arguments into cohedive objects (lists/dictionaries).

'branch error

This error is caused when a branch like an if condition, $ if-else condition, while, do etc. has too many statements in it. If you receive this error you should modularise your code into more functions, both to fix the error and to make your code more readable. (See our guide on q coding standards)

'rank error

Rank errors happen when you attempt to call a function and supply too many parameters. There is also a particularly tricky variation of this where you try to use default parameters to a qsql query. The parser gets confued and does not recognide them as parameters. As shown below:

'stop error

Using a -T setting, A maximum time allowed for a remote query can be configured.
In this example we call a function that exceeds that timeout and throws a stop error:

Server q -p 5000 -T 2

Client

'abort error

A query used more memory than the maximum memory permitted, as set using the -w kdb command line option.
An error is thrown and the kdb process is stopped.

Limits Summary

Many errors are caused by poorly written code, we recommend following the q language style guidelines which should eliminate most these problems. Other errors are reported due to OS limits, these are usually a hint that you are using kdb for a problem that it is not designed to solve. You can alter the OS to handle these cases but you may increasingly struggle to get the performance you want. That leaves a very few unfortunately tricky cases where kdb has sharp edges where it's basically user beware