kdb+ Command Line Options

Kdb+ provides a number of command line options, these allow database administrators to configure date handling, user security and to enforce memory/time limits on queries. Some command line options significantly change the characteristics of how kdb+ uses multiple CPUs or logging of data to disk and can result in significant performance improvements.

In this tutorial we will cover most the command line options that exist. We do not however have time to delve into the details of each. For the more complicated options including user permissions, logging, replication and mutli-threading, separate in-depth tutorials are available.

kdb+ Command Line Options

All Command Line Options

A list of all command line options is shown below. Typically we can view these settings from the kdb console using a slash or system command, in some cases we can also modify them.

Command Line Options
OptionDescriptionDefault
-bblock client write access.  
-c rows colsSet the rows and columns used for displaying data in the console.25 rows, 80 columns
-C rows colsSet the rows and columns used for displaying data in the web http interface.36 rows, 2000 columns
-llog updates to disk (asynchronously)  
-Llog updates to disk (synchronously)  
-o NSet the timezone offset from GMT 0
-p NSet port on which server listens.  
-p -NSet port on which server listens and use multi-threaded mode.  
-P NDisplay precision for floating point numbers.  
-r :host:portReplicate an existing kdb+ process  
-s NUse N slave threads for multi-threaded execution of queries.  
-t timerMillisSet the number of milliseconds between firing of the timer event. 0 - no events fired
-T timeoutSecondsTimeout in seconds for client queries 0 - No timeout
-u passwordFileUse a file to define usernames/passwords and restrict access to below current directory.  
-U passwordFileUse a file to define usernames/passwords.  
-w memoryLimitMBSet a memory limit of kdb+.  
-W Nstart of week as an offset from saturday. 2
-z Bformat used for "D"$ date parsing. 0 mm/dd/yyyy

Simple Configuration Options

Many of the command line options for kdb+ are for configuring date handling and output formatting. These settings can be specified at the command line or altered using a slash command, for example:

Kdb Timer

To see the effect of changing the timer, we need to override the .z.ts event handler. If we set the -t option to 2000 milliseconds whatever function is defined for .z.ts will be ran once every two seconds. To turn the timer off again we can set it to zero.

Query Timeouts and Memory Limits

OptionDescription
-T timeoutSecondsTimeout in seconds for client queries
-w memoryLimitMBSet a memory limit for the kdb+ process.

Using a -T setting, we can set a maximum time allowed for a remote query to complete. Here we use one kdb server process with a query time limit of 2 seconds and a client process to demonstrate a timeout:

Server q -p 5000 -T 2

Client


Similarly we can set the memory limit using -w. In the example below we create a function that creates increasingly large data structures until an error occurs:

Strictly speaking the space for the data needed was slightly less than the limit, however the actual memory being allocated internally by the memory manager exceeded the limit. When this occurs the kdb process exits. It's a good idea to ensure your kdb machines have more than enough RAM.

Setting Database User Permissions

OptionDescription
-bblock client write access.
-u passwordFileUse a file to define usernames/passwords and restrict access to below current directory.
-U passwordFileUse a file to define usernames/passwords.

Our tutorial on kdb+ user permissions covers this area in more detail, we will however quickly consider the command line options available. The -u/-U option allows specifying a file containing usernames and passwords which are then used to restrict which connections are permitted access to the kdb server. Here is an example user/password file:

users.pw File

Each line contains a single username:password pair. The password here is hashed using md5. Kdb+ has a builtin md5 function, using the md5 function we can see the passwords are "password123" and "column2" respectively.

Once we have started a server using the u command line option to specify a username/password file. Any time we open a handle to that server we must now supply a correct username and password otherwise we will get an access error.

Protected Server

Client

Logging and Replication

OptionDescription
-llog updates to disk (asynchronously)
-Llog updates to disk (synchronously)
-r :host:portReplicate an existing kdb+ process

Logging in kdb+ stores client queries to a .log file on disk. the use of logging means that if the machine crashes, power is lost or for any reason the process is killed, we can use the log file to replay the commands and restore the state of the process. Here's an example of a server logging client queries, that when restarted (again with -l) the previously created variables exist.

Logging Server

Client

Replication

Replication allows creating a kdb+ process that replicates the state of a master kdb+ process. All commands logged on the master are replayed on the replicated process aswell. Here we have issued a command from the a client q process to the master. When we then examine this variable on the secondary process, it has been set.

Master Logging Server

Replicating Server

Client

At this stage we want to be aware that logging and replication are supported by kdb+ command line options. There are a number of complex details to be aware of when using this facility, this will be looked at in much more detail in a later tutorial.

Mutlithreading in kdb+

OptionDescription
-p NSet port on which server listens.
-p -NSet port on which server listens and use multi-threaded mode.
-s NUse N slave threads for multi-threaded execution of queries.

-s Slave threads and Peach

A number of slave threads can be used to run calculations concurrently against multiple data items. To use this technique we simply replace our use of each with peach - the Parallel Each. For Example:

Further details of using slave threads are available in our kdb parallel processing with peach article.

Multi-Threaded Input Mode

Kdb+ normally processes all queries sequentially. If we start kdb with a negative port this instead uses concurrent threads for processing requests from separate handles. this however places a number of restrictions on what queries are acceptable. The main limitation is that client queries cannot modify global state. For certain usecases the limitations are more than compensated for by the increased performance. In our video we will demonstrate queries from multiple clients and how the result is returned significantly quicker using multi-threaded mode.