Archive Page 8
November 5th, 2015 by John Dempster
Previously on our blog we had a lively debate about a possibly Open Sourced kdb+ , unfortunately kx now seems to be moving the opposite direction. In a recent announcement they are now restricting “32-bit kdb+ for non-commercial use only”. The timing is particularly unfortunate as:
Alternative (far less enterprise proven) solutions are available:
- MAN AHL have released Arctic an open source Market Data platform based on python and MongoDB
- Kerf Database – A DB aimed at the same market as kdb has now partnered with Briarcliff-Hall and is making greater sales inroads
This renewed interest in kdb alternatives hasn’t so far delivered a kdb+ killer but I fear in time it will.
July 1st, 2015 by Ryan Hamilton
Free kdb+ Twitter Feedhandler
Previously we showed a demo of us getting data from twitter into kdb, we are now open sourcing part of that work, allowing you to quickly get some real social data into kdb to play with.
If you want to try running the kdb twitter data feed visit our https://github.com/timeseries and see the twitter-kdb project. You can even download the jar straight from our releases page. Here’s an image of the command line version running:
You will need to setup API keys for twitter
May 2nd, 2015 by admin
An intellij keyword file is now available to provide syntax highlighting of kdb code in intellij:
q Code Intellij Highlighting
To install it copy this xml file to this directory:
C:\Users\USERNAME\.IdeaIC14\config\filetypes
Where USERNAME is obviously your username. Then restart intellij and open a .q file.
We’ve updated our notepad++ qlang.xml to provide code folding and highlighting of the .Q/.z namespaces.
May 2nd, 2015 by admin
We’ve now posted all source code from this website on our github kdb page.
Additionally we are open sourcing qunit, our kdb testing framework.
We look forward to receiving pull requests to fix our (hopefully few) bugs.
April 23rd, 2015 by admin
Since our last qStudio kdb+ IDE announcment we have added a lot of new features:
Bulk importing kdb server lists
There’s a lot of new features to allow supporting a huge number of servers efficiently:
- Support importing HUGE number of servers:
- 5000+ server connections are now supported
- To prevent massive memory use, the object tree for a server is no longer refreshed at startup only on connection.
- Allow specifying default username/password once for all servers
- Allow nested connection folders
- Add critical color option – servers with prod in name get highlighted in red
- Sort File Tree Alphabetically
- Numerous bugfixes including:
- Fix critical Mac bug that prevented launching in some instances
- Fix query cancelling
April 20th, 2015 by Ryan Hamilton
Benchmarking Smart Meter Data Analytics – I got forwarded this interesting paper that compares how quickly smart meter data can be analysed using
- a Relational Database
- Matlab
- An in-memory Column-Oriented database
- Two new NoSQL alternatives
Smart electricity grids, which incorporate renewable energysources such as solar and wind, and allow information sharingamong producers and consumers, are beginning to replace conventional power grids worldwide. Smart electricity meters are afundamental component of the smart grid, enabling automated collection of fine-grained (usually every 15 minutes or hourly) consumption data. This enables dynamic electricity pricing strategies,in which consumers are charged higher prices during peak timesto help reduce peak demand. Additionally, smart meter data analytics, which aims to help utilities and consumers understand electricity consumption patterns, has become an active area in researchand industry. According to a recent report, utility data analytics isalready a billion dollar market and is expected to grow to nearly 4billion dollars by year 2020
January 19th, 2015 by Ryan Hamilton
In a world overran with open source big data solutions is kdb+ going to be left behind? I hope not…
Every few weeks someone comes to me with a big data problem often with a half-done mongoDB/NoSQL solution that they “want a little help with”. Usually after looking at it I think to myself
“I could solve this in 20 minutes with 5 lines of q code”
But how do I tell someone they should use a system which may end up costing them £1,000s per core in licensing costs. Yes it’s great that there’s a free 32-bit trial version but the risk that you may end up needing the 64-bit is too great a risk.
Given the ever-increasing number of NoSQL solutions and in particular the rising popularity of Hadoop, R, python and MongoDB it’s not hard to see that open-source is taking over the world. Right now kdb+ still has the edge, that it’s faster, sleeker, sexier..but I don’t think that will save it in the long run. The power of open-source is that it let’s everyone contribute, witness the 100’s of libraries available for R, the 1000’s of javascript frameworks. The truly sad thing is that it’s not always the best underlying technology that wins. A 1000 amateurs creating a vibrant ecosystem of plug-ins, add-ons, tutorials… can beat other technologies through sheer force of numbers.
- APL was a great language yet it remains relegated to history while PHP flourishes.
- PostgreSQL was technically superior to MySQL yet MySQL is deployed everywhere
I believe kdb+ is the best solution to a large number of “big data” problems (small data to kdb+), When you stop and think, time-series data is everywhere, open sourcing kdb+ would open up entirely new sectors to kdb+ and I hope it’s a step kx take before it’s too late.
What do you think? Leave your comments below.
October 20th, 2014 by admin
Based on user requests we have released a number of new features with qStudio 1.36:
Download the latest ->qStudio<- now.
Dark Code Editor Themes
Which can be set under settings->preferences
Open Results and Charts in New Window
To expand a panel into a new window click the “pop-out” icon.
This will bring up the result in a new window:
UTF-8 Chinese Language Support
October 20th, 2014 by admin
Newry-based financial software firm First Derivatives has acquired a majority shareholding in big data analytics company Kx Systems for £36m (€44m).
KX has historically had a hard time penetrating markets outside finance, FD have a good sales team and previously acquired a marketing company in Philadelphia, hopefully this is the chance for kdb+ to go mainstream.
However it’s a worry that FD (First Derivatives) may increasingly “encourage” purchasing of the delta platform bundle rather than stand-alone kdb+. With the smaller margins outside of finance, will FD take a risk and open up the database. (Would FD have been a supporter of the 32-bit version becoming free for commercial use?) There’s a large number of individuals in off-shore locations that want to learn kdb+, FD could be incentivized to discourage that as it would hurt their consulting business.
It’s also interesting to consider companies that have already invested in KX technology and whether they will continue to do so
- Competing consulting firms that specialise in kdb+ won’t take this as good news.
- Panopticon/Datawatch based their visualization system on kdb+ (OEM license), they probably regret that now, given that their visualization software directly competes with FD’s dashboards.
- Companies that had used kdb+ as part of their trading platform stack may consider FD a competitor as they also offer a trading platform.
What do you think? Will this lead to wider adoption? a growing platform?
October 8th, 2014 by admin
kdb+ London Contract – £800 p/day
kdb+ belfast Citigroup Contract £400 p/day
Java Poland £150 p/day
