Archive for February, 2023

2018 – The Future of Tech in Banks – Solutions here now (part 2)

This article is (part 2) of a series. See the previous post on “2018 – The Future of Tech in Banks, particularly within Market Data“.

The previous article described a few problems with the current tech/finance structure in most banks. In the words of Jim Barksdale:

“there are two ways to make money. You can bundle, or you can unbundle.”

In this case we can see two possible solutions:

  • Horizontal Integration – Providing a bundled reliable layer e.g. AWS to solve your hardware needs
  • Vertical Integration – Providing a front to back solution, SAAS – Software-As-A-Service e.g. github hosting, third-party trading platform.

Horizontal Integration AWS – To Solve hardware layer

Consider the example of outsourcing: “AWS for hardware”, it makes 100% sense, there is very little customization or unknown capability with 95%+ of servers for application use within a bank being fairly standard. The area where this currently becomes problematic is high-performance and co-location, to cover those needs hybrid-cloud could help. The benefits and savings in other areas, security/reliability/costs can often out weight the drawbacks. In my opinion most internal cloud solutions will dissapear within the next few years.

Benefits of Bundling/Outsourcing

Solutions rely on the problem being well known/understood and that all inputs/outputs to the bound box can be well defined. They work by:

  • Preventing duplication of effort – Designed to be re-used
  • Reducing communication overhead – Everything within their box is a service with APIs or configurable. No meetings/Change tickets required (OK less. There will always be change tickets!).
  • Preventing misalignment and Misalignment of incentives -One entity is responsible for full delivery and if outsourced can be scaled up or down at little overhead/risk to the bank.

Vertical Integration – Outsourced Market Data-As-A-Service

An example of vertical integration would be “Market Data-As-A-Service”. If every bank has the same market data problem and we can get the benefits of buying that bundle, should we?
The danger is that it takes years to evolve to an “API” that covers 95% of the needs and even then you have to be careful that you don’t over allocate resources on something that the user actually has little value for. This is harder to know as an external entity as you don’t sit with the customer.

So given that banks have 3 options:
– Keep separate teams
– Use horizontal solutions
– Use vertical solutions
What should they do? When?

Ultimately it will be a combination of parts that evolve over time but if the problem is shared by all banks the solution is using off-the-shelf software eventually.

The market-data/feeds team should at all times be asking:

  • Should we build this?
  • Is this a problem specific to this bank that will add value?
  • Or is it a general problem where we can take advantage of economies of scale?

Conclusion

Outsourced Solutions for Market Data – currently make less sense. As:
– Even if we can outsource storage of market data, we need a way to store our own trade date and other internal data sets.
Column Oriented Storage  – is becoming a commoditized technology. A number of firms including the major ones such as AWS are bringing user friendliness, reliability and general availability of what used to be a niche technology.
– Over recent years, firms including HFT have captured a lot of value by having in-depth market data knowledge.

The market data teams should begin to learn redshift/google/AWS solutions for as they scale to all firms everywhere the savings are massive.

Open Solutions

So far we mostly considered outsourced commercial solutions to solve the common problems. That however is not the only approach. It would be possible to reap the same, if not more benefits from an open core model. e.g. An open source trading system, that every bank makes commits to improve only keeping closed source the parts that are uniquely valuable to their business. Unfortunately in practice so far this seems to be less viable as any entity that pushes adoption of the platform, realizes costs pushing it while not capturing much value. Whereas closed source, the company incurs the marketing costs but can get this back in licensing fees.

Pulse 0.13.5 Adds Sparklines and Dynamic HTML

Pulse – Real-time interactive Dashboards 0.13.5 adds Sparkline and Dynamic HTML support

  • Dynamic HTML – Full user control to generate HTML using template languages
  • Sparklines – Embed small charts within a table by specifying nested arrays.

DOWNLOAD NOW

Pulse is designed to provide real-time interactive dashboards so the underlying database has to be really fast. Pulse can support almost any data source, the question is which databases are worth supporting.

Time-series databases are the fastest growing database sector (image below).
The great news is that in the last few years there has been a a lot of interesting new entrants. So we’ve updated our past articles:

  1. Top Column-Oriented DatabasesDuckDB, Clickhouse and Doris are the new exciting entrants. Benchmark results in article.
  2. Top Time-Series Databases – Have exploded in popularity. QuestDB and TimeScale are the new entrants there.  Benchmark results in article.

Building kdb+ Trader Dashboards
Kdb+ Streaming
Pulse – kdb+ Streaming Subscriptions