Kdb C API Query Subscribe Example

Pulse

Allows kdb developers to easily create real-time interactive dashboards direct from kdb data.

Examples of connecting, querying and subscribing using the C API for linux are available to download and are detailed below:

Useful Code KX Information
Query Kdb 2.6/2.8 from C in Linux
Query Kdb 3.0+ from C in Linux
C Connect / Query Code
Subscribing to Tickerplant data
Feedhandler and Further Coding

Download Example C Code

Example of Kdb C API Query

Useful Code KX Information

We will use the two methods shown below to connect and query kdb:
The Interfacing Kdb with C page on the code KX website details the full API specificiation.

Method Calls

Signature	Notes
int c = khpu("localhost", 1234,"myusername:mypassword");	Connect to a kdb server on the localhost port 1234.
result=k(handle,"([]a:til 10;b:reverse til 10;c:10?`4)",(K)0)	Send a synchronous message in string format to a server and return the result.

Querying 2.6/2.8 kdb from C in Linux

Code KX has a simple example of querying that relies on a q server to be running on port 12001.
To compile and run the example for kdb < version 3, do the following in linux:

svn export -r 1442 http://code.kx.com/svn/kx/kdb+/c/c/k.h
svn export -r 1442 http://code.kx.com/svn/kx/kdb+/l64/c.o
svn export http://code.kx.com/svn/contrib/wiki/csv.c
gcc -m64 csv.c c.o -lpthread
# start kdb on localhost port 12001
# q -p 12001 
./a.out

-lpthread - links in the posix threads library that is needed
-m64 - generates code for a 64 bit environment, setting int to 32 bits and long and pointer types to 64 bits
The svn uses -r as you need the old revisions of the library/header, these can also be downloaded qc-eg.zip and here

Querying 3.0 kdb from C in Linux

The main difference with kdb version 3.0+ is that we must make sure to specify the environment variable KXVER.
Kdb must be running on the local machine port 12001.

svn export http://code.kx.com/svn/kx/kdb+/c/c/k.h
svn export http://code.kx.com/svn/kx/kdb+/l64/c.o
svn export http://code.kx.com/svn/contrib/wiki/csv.c
gcc -m64 -DKXVER=3 csv.c c.o -lpthread
# start kdb on localhost port 12001
# q -p 12001 
./a.out

C Connect / Query Code

The major steps involved in kx's example csv.c to connect and query a kdb server are:

Connect to kdb server. khpu(host, port, "user:pass")
Send query synchronously. result=k(handle,"select...",(K)0)
Close handle. kclose(handle)
Check the result type then loop through rows/columns. using K.

..
#include"k.h" // essential to define kdb data objects and access them

int main(int argc,char*argv[]) {
    ...
    handle=khpu("localhost",12001,"myusername:mypassword"); // connect
	... // check connected
    result=k(handle,"([]a:til 10;b:reverse til 10;c:10?`4;d:{x#.Q.a}each til 10)",(K)0);
	... // check result type
    kclose(handle);
	...
	
	// make easier to access the data
    flip=ktd(result); // if keyed table, unkey it. ktd decrements ref count of arg.
    // table (flip) is column names!list of columns (data)
    columnNames=kK(flip->k)[0];
    columnData=kK(flip->k)[1];
    nCols=columnNames->n;
    nRows=kK(columnData)[0]->n;
    for(row=0;row<nRows;row++)  {
        for(col=0;col<nCols;col++) {
           // print value of row/col
            }
        }
        printf("\n");
    }
	...
    return 0;
}

Subscribing to Kdb tickerplant

See the subscriber.c in the C example code. This demonstrates connecting to a tickerplant and subscribing to the trade table. The steps are:

Connect
Subscribe to all syms on the trade table
loop forever, calling k() to receive incoming data.

Subscriber C Code

int main(int argc, char *argv[]) {

    // find host:port from args then connect
    int handle = khpu(host, port, "user:pass");

    // subscribe to trade table
    K r = k(handle, ".u.sub[`trade;`]", (K) 0);

    // process ticks
    while(1) {
        r = k(handle,(S)0);
        if(r) {
            if(r->t == 0) {
                // r is 3 item list of format (".u.upd"; `trade; updateTable)

                printf("\n%s update", kK(r)[1]->s); // table name symbol atom

                tbl = kK(r)[2]->k; 
                colNames = kK(tbl)[0];
                colData = kK(tbl)[1];

                printf(" rows: %lld cols: ", kK(colData)[0]->n);
                
                for(c=0; c < colNames->n; c++) {
                    printf("%s,", kS(colNames)[c]);
                }
            }
        }
        r0(r); // decrement reference count, free memory
    }
}

tbl is the latest table of trade data, this is a K object with ->t or type 98. We used ->k to access the underlying dictionary which contains the column names and data.

The dictionary tbl is an array with two items:
[0] is symbol list of column names
[1] is columns of data

Executing from Shell

wget -q http://www.timestored.com/kdb-guides/files/qc-subscriber.zip
unzip qc-subscriber.zip
cd qc-subscriber
gcc -m64 -DKXVER=3 subscriber.c c.o -lpthread
# kdb tickerplant is running on host FXtick01 on port 5010
./a.out FXtick01 5010

When you execute the above code at the linux terminal it will output the number of rows it is receiving and the column names

Feedhandler and Further Coding

The code shown above is for demonstration purposes only and is missing error handling, failover and graceful handling of many other scenarios. Writing production ready C subscribers and feedhandlers is tricky. If you would like further help with subscribing to kdb+tick tickerplants or creating feedhandlers we are available for kdb+ consulting.