<rss version="2.0" xmlns:atom="http://www.w3.org/2005/Atom">
<channel>
  <atom:link href="https://lubblabs.com/feed.xml" rel="self" type="application/rss+xml" />
  <title>Lubb Labs</title>
  <link>https://lubblabs.com</link>
  <description>Thoughtfully designed software that is reliable, maintainable, and built to last.</description>
  <language>en-US</language>
  <generator>Tableau v0.30.0</generator>
    <item>
       <title>Building GRapple (Part 1): Bringing Real-Time Race Data to Life with Elixir</title>
       <link>https://lubblabs.com/insights/building-grapple-part-1/</link>
       <pubDate>Thu, 30 Oct 2025 18:51:00 -04</pubDate>
       <guid>https://lubblabs.com/insights/building-grapple-part-1/</guid>
       <description><![CDATA[ <h2 dir="auto"><a href="#-the-starting-line" aria-hidden="true" class="anchor" id="-the-starting-line"></a>🟢 The Starting Line</h2><p dir="auto">It was October 15th and I was spending a few minutes scrolling my LinkedIn feed when something caught my attention — an announcement for the kickoff of <a href="https://hackthetrack.devpost.com/">Toyota Racing Development&#39;s Hack the Track hackathon</a>.</p><p dir="auto">I don’t know much about racing, but I’ve always loved cars since I was a boy. I looked into this hackathon some more and started to become increasingly interested. It was an opportunity to build something using a unique dataset from the world of motorsports.</p><p dir="auto">This was certainly something out of left field for me, but the more I learned about how racing teams deal with massive amounts of <strong>live-streamed data</strong> — from car sensors to weather systems — the more I realized something:</p><blockquote dir="auto"><p dir="auto">this was a perfect opportunity to push the <strong>BEAM’s real-time strengths</strong> into a completely new domain — motorsport telemetry.</p></blockquote><p dir="auto">After reading the contest description, one category stood out immediately:</p><p dir="auto"><strong>Real-Time Analytics</strong> — <em>Design a tool that simulates real-time decision-making for a race engineer.</em></p><p dir="auto">That was it. The perfect match for Elixir, Phoenix, and OTP.</p><p dir="auto">I hesitated for a night — it was a big leap into a domain I knew nothing about — but the next day, I joined the hackathon.</p><p dir="auto">Now the question was: <em>what would I build?</em></p><h2 dir="auto"><a href="#-making-sense-of-the-data-jungle" aria-hidden="true" class="anchor" id="-making-sense-of-the-data-jungle"></a>🧩 Making Sense of the Data Jungle</h2><p dir="auto">The organizers released gigabytes of CSVs: lap timing data, race results, track weather, and — most interestingly — <strong>telemetry data</strong> from the GR Cup race cars.</p><p dir="auto">At first glance, it was overwhelming. Millions of rows, hundreds of fields with names like <code>accx_can</code>, <code>pbrake_f</code>, <code>Laptrigger_lapdist_dls</code>. I didn’t even know what half the abbreviations meant.</p><p dir="auto">I started simple — using tools like <code>less</code>, <code>grep</code>, and Excel to poke around. Then it clicked: Each row wasn’t just a data point — it was <strong>a heartbeat from the car</strong>.</p><p dir="auto">Telemetry captures everything happening inside a racecar every fraction of a second: speed, throttle, brake pressure, gear changes, acceleration, GPS position.</p><p dir="auto">And if I could take those heartbeats and <strong>replay them live</strong>, I could recreate the rhythm of a race — not a static CSV, but a <em>living timeline of events</em>.</p><p dir="auto">That idea became the foundation of what I came to call <em>GRapple</em> (because it grapples with lots of data 🙂):</p><blockquote dir="auto"><p dir="auto">a real-time race replay, analytics, and strategy system powered by Elixir and OTP.</p></blockquote><h2 dir="auto"><a href="#️-turning-data-into-motion" aria-hidden="true" class="anchor" id="️-turning-data-into-motion"></a>⚙️ Turning Data into Motion</h2><p dir="auto">My first task was to sort the telemetry chronologically. That sounds simple, but the dataset wasn’t ordered, contained duplicates, and was over 11 million lines long, so I used Unix <code>sort</code> and <code>uniq</code> in wired that into Elixir for performance.</p><p dir="auto">Once sorted, I used <a href="https://github.com/dashbitco/nimble_csv"><code>NimbleCSV</code></a> to parse each lazily streamed event into well-defined structs. The core of it looks like this:</p><pre dir="ltr" class="lumis" style="color: #ebdbb2; background-color: #282828;"><code class="language-elixir" translate="no" tabindex="0"><div class="line" data-line="1"><span style="color: #928374;">@</span><span style="color: #928374;">doc</span> <span style="color: #928374;">&quot;&quot;&quot;
</div><div class="line" data-line="2">  Parses the given Telemetry Data CSV file into a lazy Stream of chronological TelemetryPoints.
</div><div class="line" data-line="3">  WARNING: this function can take a few minutes if the CSV file is large (GBs)
</div><div class="line" data-line="4">  &quot;&quot;&quot;</span>
</div><div class="line" data-line="5">  <span style="color: #d3869b;">@</span><span style="color: #d3869b;">spec </span><span style="color: #b8bb26; font-weight: bold;">parse_stream!</span><span style="color: #fe8019;">(</span><span style="color: #ebdbb2;">filename</span> <span style="color: #fe8019;">::</span> <span style="color: #ebdbb2;">String</span><span style="color: #fe8019;">.</span><span style="color: #b8bb26; font-weight: bold;">t</span><span style="color: #fe8019;">(</span><span style="color: #fe8019;">)</span><span style="color: #fe8019;">)</span> <span style="color: #fe8019;">::</span> <span style="color: #fe8019;">&lbrace;</span><span style="color: #b8bb26; font-weight: bold;">integer</span><span style="color: #fe8019;">(</span><span style="color: #fe8019;">)</span><span style="color: #fe8019;">,</span> <span style="color: #ebdbb2;">Enumerable</span><span style="color: #fe8019;">.</span><span style="color: #b8bb26; font-weight: bold;">t</span><span style="color: #fe8019;">(</span><span style="color: #fe8019;">)</span><span style="color: #fe8019;">&rbrace;</span>
</div><div class="line" data-line="6">  <span style="color: #fb4934;">def</span> <span style="color: #b8bb26; font-weight: bold;">parse_stream!</span><span style="color: #fe8019;">(</span><span style="color: #ebdbb2;">filename</span><span style="color: #fe8019;">)</span> <span style="color: #fb4934;">do</span>
</div><div class="line" data-line="7">    <span style="color: #ebdbb2;">cleaned_telemetry_file</span> <span style="color: #fe8019;">=</span> <span style="color: #b8bb26; font-weight: bold;">clean_file!</span><span style="color: #fe8019;">(</span><span style="color: #ebdbb2;">filename</span><span style="color: #fe8019;">)</span> <span style="color: #928374;"># remove dups</span>
</div><div class="line" data-line="8">    <span style="color: #ebdbb2;">sorted_telemetry_file</span> <span style="color: #fe8019;">=</span> <span style="color: #b8bb26; font-weight: bold;">sort_file_by_timestamp!</span><span style="color: #fe8019;">(</span><span style="color: #ebdbb2;">cleaned_telemetry_file</span><span style="color: #fe8019;">)</span>
</div><div class="line" data-line="9">    <span style="color: #ebdbb2;">telemetry_count</span> <span style="color: #fe8019;">=</span> <span style="color: #b8bb26; font-weight: bold;">num_data_points</span><span style="color: #fe8019;">(</span><span style="color: #ebdbb2;">sorted_telemetry_file</span><span style="color: #fe8019;">)</span>
</div><div class="line" data-line="10">
</div><div class="line" data-line="11">    <span style="color: #ebdbb2;">stream</span> <span style="color: #fe8019;">=</span>
</div><div class="line" data-line="12">      <span style="color: #ebdbb2;">File</span><span style="color: #fe8019;">.</span><span style="color: #b8bb26; font-weight: bold;">stream!</span><span style="color: #fe8019;">(</span><span style="color: #ebdbb2;">sorted_telemetry_file</span><span style="color: #fe8019;">)</span>
</div><div class="line" data-line="13">      <span style="color: #fe8019;">|&gt;</span> <span style="color: #ebdbb2;">TelemetryParser</span><span style="color: #fe8019;">.</span><span style="color: #b8bb26; font-weight: bold;">parse_stream</span><span style="color: #fe8019;">(</span><span style="color: #83a598;">skip_headers: </span><span style="color: #d3869b;">false</span><span style="color: #fe8019;">)</span>
</div><div class="line" data-line="14">      <span style="color: #fe8019;">|&gt;</span> <span style="color: #ebdbb2;">Stream</span><span style="color: #fe8019;">.</span><span style="color: #b8bb26; font-weight: bold;">map</span><span style="color: #fe8019;">(</span><span style="color: #fb4934;">fn</span> <span style="color: #ebdbb2;">row</span> <span style="color: #fe8019;">-&gt;</span>
</div><div class="line" data-line="15">        <span style="color: #fb4934;">try</span> <span style="color: #fb4934;">do</span>
</div><div class="line" data-line="16">          <span style="color: #ebdbb2;">TelemetryPoint</span><span style="color: #fe8019;">.</span><span style="color: #b8bb26; font-weight: bold;">from_row!</span><span style="color: #fe8019;">(</span><span style="color: #ebdbb2;">row</span><span style="color: #fe8019;">)</span>
</div><div class="line" data-line="17">        <span style="color: #fb4934;">rescue</span>
</div><div class="line" data-line="18">          <span style="color: #ebdbb2;">e</span> <span style="color: #fb4934;">in</span> <span style="color: #fe8019;">[</span><span style="color: #ebdbb2;">MatchError</span><span style="color: #fe8019;">,</span> <span style="color: #ebdbb2;">ArgumentError</span><span style="color: #fe8019;">]</span> <span style="color: #fe8019;">-&gt;</span>
</div><div class="line" data-line="19">            <span style="color: #ebdbb2;">Logger</span><span style="color: #fe8019;">.</span><span style="color: #b8bb26; font-weight: bold;">debug</span><span style="color: #fe8019;">(</span><span style="color: #b8bb26;">&quot;Skipping unparsable row: </span><span style="color: #fe8019;">#&lbrace;</span><span style="color: #b8bb26; font-weight: bold;">inspect</span><span style="color: #fe8019;">(</span><span style="color: #ebdbb2;">row</span><span style="color: #fe8019;">)</span><span style="color: #fe8019;">&rbrace;</span><span style="color: #b8bb26;"> (</span><span style="color: #fe8019;">#&lbrace;</span><span style="color: #ebdbb2;">Exception</span><span style="color: #fe8019;">.</span><span style="color: #b8bb26; font-weight: bold;">message</span><span style="color: #fe8019;">(</span><span style="color: #ebdbb2;">e</span><span style="color: #fe8019;">)</span><span style="color: #fe8019;">&rbrace;</span><span style="color: #b8bb26;">)&quot;</span><span style="color: #fe8019;">)</span>
</div><div class="line" data-line="20">            <span style="color: #fe8019;">nil</span>
</div><div class="line" data-line="21">        <span style="color: #fb4934;">end</span>
</div><div class="line" data-line="22">      <span style="color: #fb4934;">end</span><span style="color: #fe8019;">)</span>
</div><div class="line" data-line="23">      <span style="color: #928374;"># reject any rows that failed to be parsed</span>
</div><div class="line" data-line="24">      <span style="color: #fe8019;">|&gt;</span> <span style="color: #ebdbb2;">Stream</span><span style="color: #fe8019;">.</span><span style="color: #b8bb26; font-weight: bold;">reject</span><span style="color: #fe8019;">(</span><span style="color: #fe8019;">&amp;</span><span style="color: #b8bb26; font-weight: bold;">is_nil</span><span style="color: #fe8019;">/</span><span style="color: #fe8019;">1</span><span style="color: #fe8019;">)</span>
</div><div class="line" data-line="25">      <span style="color: #fe8019;">|&gt;</span> <span style="color: #b8bb26; font-weight: bold;">compute_deltas</span><span style="color: #fe8019;">(</span><span style="color: #fe8019;">)</span>
</div><div class="line" data-line="26">
</div><div class="line" data-line="27">    <span style="color: #fe8019;">&lbrace;</span><span style="color: #ebdbb2;">telemetry_count</span><span style="color: #fe8019;">,</span> <span style="color: #ebdbb2;">stream</span><span style="color: #fe8019;">&rbrace;</span>
</div><div class="line" data-line="28">  <span style="color: #fb4934;">end</span>
</div></code></pre><p dir="auto"><code>compute_deltas/1</code> does something interesting. It looks at the timestamps of two successive telemetry events, computes the delta of the next event, and stores this inside the struct for the current event. This way when we come to replay the telemetry event live, we will know exactly duration between any two events and can therefore simulate the race.</p><p dir="auto">Then came the fun part: I needed a process that would emit those events live at the same timing intervals they originally happened (using our deltas from before). Enter the <code>TelemetryReplayer</code>.</p><p dir="auto">At first, I was thinking to broadcast every telemetry row as a message on a Phoenix PubSub topic so other processes could subscribe, react, and compute metrics in real time. This would work great when I incorporate LiveView later on. But for now since I&#39;m still building the backend, I decided I would keep it simple and just have the <code>TelemetryReplayer</code> cast each telemetry event to its associated car GenServer (which I would implement next).</p><p dir="auto">At last, everything was starting to come together. I had gone from millions of rows of CSV files to well-defined data I could work with being streamed in realtime using the power of OTP. Seeing that first replay run felt surreal — after days of CSV wrangling, a virtual race had come to life in my terminal.</p><p dir="auto"><img src="/media/grapple-telemetry-replay.png" alt="TelemetryReplayer working in the terminal"/></p><p dir="auto">This was a truly motivating moment and I was ready to keep moving forward. It&#39;s not enough to replay the race live. If I could have one GenServer process per car consuming its telemetry events and keeping rolling metrics as its state, I could unlock a whole new world of possibilities when it comes to analytics and insight.</p><h2 dir="auto"><a href="#-teaching-grapple-to-think" aria-hidden="true" class="anchor" id="-teaching-grapple-to-think"></a>📊 Teaching GRapple to Think</h2><p dir="auto">Once the race replay worked, the next challenge was to give GRapple some <em>intuition</em>.</p><p dir="auto">I built a <code>MetricsServer</code>, one GenServer process per car, that tracks real-time statistics — speed, throttle, brake pressure, soon acceleration, steering, and much more.</p><p dir="auto">Each process consumes its car&#39;s telemetry events, and maintains rolling per-lap stats, smoothing raw signals with exponential moving averages (EMA) to highlight trends in driver behavior. Once a new lap starts, it stores the past lap in its own history. Each process is supervised by a <code>MetricsServerSupervisor</code> for fault-tolerance.</p><p dir="auto">This way, I could get the metrics for any car in the race at any given time by looking at the state of its associated running process. It also helps tremendously in reasoning about the system and keeps things simple: each process knows about its cars performance best. Using hand-crafted test CSV files, I started to see the system in action and it was mesmerizing. Data was flowing from CSV file to the <code>TelemetryReplayer</code> to a car&#39;s <code>MetricsServer</code> which would perform rolling metrics calculations all in real-time.</p><p dir="auto">And this was just the beginning. Eventually, I am hoping to implement another dedicated Analytics process that will query each <code>MetricsServer</code> and derive higher-level race insights.</p><h2 dir="auto"><a href="#-what-ive-learned-so-far" aria-hidden="true" class="anchor" id="-what-ive-learned-so-far"></a>🧠 What I&#39;ve Learned So Far</h2><p dir="auto">This project reminded me why I love Elixir and the BEAM.</p><ul><li dir="auto">The <a href="https://elixirschool.com/en/lessons/intermediate/concurrency">Actor Model</a> is perfect for modeling a race with multiple cars.</li><li dir="auto">OTP made concurrency <em>predictable</em>.</li><li dir="auto">Streams let me handle millions of rows lazily and efficiently.</li><li dir="auto">Elixir is the right tool for this job.</li></ul><p dir="auto">There&#39;s quite a bit until I get to the finish line with this project, but it&#39;s been refreshing stepping into the world of motorsports and a joy using Elixir so far.</p><h2 dir="auto"><a href="#-the-road-ahead" aria-hidden="true" class="anchor" id="-the-road-ahead"></a>🚀 The Road Ahead</h2><p dir="auto">In the next stage, I’ll start extending the <code>MetricsServer</code> to compute more metrics per car using the telemetry events. So far, I&#39;ve implemented speed, throttle behavior, and brake pressure rolling metrics, but will need to add acceleration, lateral G (for cornering insight), steering, lap timing, and (if time permits) lap section comparison for more insight. Eventually, I will connect it to <strong>AI models via</strong><a href="https://replicate.com/"><strong>Replicate</strong></a> to simulate a race engineer that can answer live questions about performance and even strategize race decisions.</p><p dir="auto">I&#39;ll be writing some more blog posts as I continue building <em>GRapple</em> for the Toyota Hack the Track 2025 challenge. Stay tuned.</p> ]]></description>
    </item>
  </channel>
</rss>
