{"id":112058,"date":"2026-03-07T14:30:31","date_gmt":"2026-03-07T14:30:31","guid":{"rendered":"https:\/\/www.timestored.com\/b\/?p=112058"},"modified":"2026-03-07T14:42:48","modified_gmt":"2026-03-07T14:42:48","slug":"the-client-side-data-pipeline","status":"publish","type":"post","link":"https:\/\/www.timestored.com\/b\/the-client-side-data-pipeline\/","title":{"rendered":"The Client Side Data Pipeline"},"content":{"rendered":"

Modern data tools look simple: run a query, show a table, maybe draw a chart.
\nBut under the hood there is usually a surprisingly complex data pipeline<\/strong> transforming raw query results into tables,
\ncharts, exports, and derived views.<\/p>\n

This diagram below is Pulse’s client-side data pipeline<\/strong> (adding the server side would double it).<\/p>\n

Why does a “simple” data tool need all this? Because the same query result gets consumed in fundamentally different ways:<\/p>\n

1. The table<\/strong>\u00a0must show data as it came from the database.
\n2. Users apply formatting<\/strong>\u00a0– decimal places, thousands separators, or advanced renderers like sparklines and heatmaps.
\n3. When they export to Excel<\/strong>, they want full numeric precision back (not the rounded display value), and sparklines replaced with raw arrays like `[2,4,5]`.
\n4. A time-series chart<\/strong> might append new rows to the end of a live stream. To handle that efficiently, we heavily optimize for the append-only case rather than re-rendering from scratch.
\n6. A Time-Series transform<\/strong> can compute indicators like MACD or Bollinger Bands on the raw result before it ever reaches the chart.<\/p>\n

Each of these is a different transformation of the same source data, and they all need to stay in sync when the query refreshes.<\/p>\n

A Concrete Example: One Query, Four Representations<\/h3>\n

Say you query a trades table:<\/p>\n

SELECT timestamp, symbol, price, volume FROM trades WHERE timestamp > now() - 01:00:00<\/pre>\n
That single result set flows through the pipeline to produce:<\/p>\n
– A live-updating table<\/strong>\u00a0with prices formatted as `$1,234.56` and volumes with thousand separators, updating in-place as new trades arrive via WebSocket.
\n– A time-series line chart<\/strong> of price by symbol, where the engine auto-pivots the long-form result (one row per trade) into wide-form (one column per symbol) – no `PIVOT` clause needed in your SQL.
\n– A Pine Script overlay<\/strong>\u00a0computing a 20-period moving average on price, injected as a new series before the chart renders.
\n– An Excel export<\/strong>\u00a0where the user gets raw timestamps (not the `HH:mm:ss` display format), full decimal precision, and no HTML artifacts.<\/p>\n
All four stay synchronized. When the next WebSocket push arrives with new rows, each branch of the pipeline updates – the table appends, the chart extends, the treemap re-aggregates – without re-running the query.<\/p>\n
Why Data Engineers Should Care<\/h3>\n
If you’ve built internal tools, you’ve probably hit the same tensions: display formatting vs. export fidelity, append-optimized rendering vs. full re-render, flat SQL results vs. hierarchical visualizations. These are data transformation problems, and the solutions look a lot like what you’d build in a backend ETL pipeline, just running at 60fps in the browser.<\/p>\n
The lesson: even “simple” analytics UIs hide a real data pipeline. And the more data-literate your users are, the more branches that pipeline grows.<\/p>\n
 <\/p>\n
\"\"<\/a><\/p>\n","protected":false},"excerpt":{"rendered":"
Modern data tools look simple: run a query, show a table, maybe draw a chart. But under the hood there is usually a surprisingly complex data pipeline transforming raw query results into tables, charts, exports, and derived views. This diagram below is Pulse’s client-side data pipeline (adding the server side would double it). Why does […]<\/p>\n","protected":false},"author":1,"featured_media":0,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"_bbp_topic_count":0,"_bbp_reply_count":0,"_bbp_total_topic_count":0,"_bbp_total_reply_count":0,"_bbp_voice_count":0,"_bbp_anonymous_reply_count":0,"_bbp_topic_count_hidden":0,"_bbp_reply_count_hidden":0,"_bbp_forum_subforum_count":0},"categories":[109],"tags":[],"_links":{"self":[{"href":"https:\/\/www.timestored.com\/b\/wp-json\/wp\/v2\/posts\/112058"}],"collection":[{"href":"https:\/\/www.timestored.com\/b\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/www.timestored.com\/b\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/www.timestored.com\/b\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/www.timestored.com\/b\/wp-json\/wp\/v2\/comments?post=112058"}],"version-history":[{"count":3,"href":"https:\/\/www.timestored.com\/b\/wp-json\/wp\/v2\/posts\/112058\/revisions"}],"predecessor-version":[{"id":112062,"href":"https:\/\/www.timestored.com\/b\/wp-json\/wp\/v2\/posts\/112058\/revisions\/112062"}],"wp:attachment":[{"href":"https:\/\/www.timestored.com\/b\/wp-json\/wp\/v2\/media?parent=112058"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/www.timestored.com\/b\/wp-json\/wp\/v2\/categories?post=112058"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/www.timestored.com\/b\/wp-json\/wp\/v2\/tags?post=112058"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}