John Davison
September 23, 2024

The LLM is Your Data Scientist

ChatGPT can be your companion Data Scientist, but only if you know how to ask it nicely… Here is a quick rundown of how ChatGPT helped me estimate missing heart rate training data from a few other training sessions, a simulated “supervised machine learning” solution.

The Problem

On September 14, 2024, I was out riding my bike. I typically track my heart rate zones as zone 2 & 3 cardio are basically the fountain of life. During this ride, my heart rate sensor, a polar h10, stopped gathering data, so I ended up with a 90 min session with no data, yikes!

What to do, what to do… if you have been following the whole “AI Thing” by now I’d “assume” (ass of you and me) that you you understood that LLMs were really good at observing and generating patterns. In fact, an LLM is “nothing more” than a canned observation of patterns capable of making more patterns, kind of like a self quilting quilt that had instead of a yarn structure, a structure made of all publicly known information and instead of knots in the quilt, the LMM produces any text output given some inputs.

Ok so back to my conundrum, how do I get my data if my sensor stopped recording?

The Plan

Well, I knew that I had other properly recorded sessions, so like a good data scientist, I knew that I had a few examples of “supervised training data”, ie, i knew there were some inputs, like, speed, distance, pace (and gps data) that produced some output → my heart rate. In this particular instance, because I used the polar flow app, my training session had no sensor HR data, but it did have the speed, distance and pace data…

Go Go Gadget Data Interpolation → “Supervised Learning” Without Running any Code !!!

  1. login to polar and get csv data of 3 complete and 1 incomplete rides
  2. me to chatgpt “hey i’m going to feed you csv training data and i want to estimate empty data”
  3. chatgpt: “ok, but your csv files have format errors so i’m confused”
  4. me: “ok, i cleaned up that junk try again”
  5. chatgpt: “ok here is your estimated data”
  6. me: “awesome, i’m lazy, can you chart it for me?”
  7. chatgpt: “here are your charts Mr. Perezoso (lazy)”
  8. me: “but the x axis, I don’t understand time data points, can you rechart in duration”
  9. chatgpt: “done”
  10. me: “thats fucking magic”
  11. chatgpt: “i’m just getting started…”

Chatgpt estimated the ride and projected around 36 minutes in zone 2, and 36 in zone 3 with a little leftover in zone 1, 4 and 5. If I look at that graph and squint, thinking tenderly about that ride, I’m like yep, an 80ish minute ride on mostly flat ground where I rode consistently in my training zone - this estimation is accurate enough for me cause all I care about is roughly speaking total duration per of zone 2,3 cardio and total duration of all zones.

The Result

**The Wow of this Routine is that **

  • I’m not a data scientist, I’m a stellar product engineer and I’ve taken some data for dummies classes but generally I don’t want to do any hands on data science
  • The way I was able to do this is by knowing the 80/20 of how supervised machine learning works
  • Took less than 30 minutes to create estimated data
  • What else is lurking our there for me…
  • ChatGPT didn’t actually use “machine learning” to do this, it used a matching function that calculated the average HR for a given speed/pace combination
  • My high level knowledge of how supervised learning caused me to make assumptions that ChatGPT could do this - without that high level knowledge, I would probably not have thought to do this

A critical take away

  • LLMs mirror and aid the things we know, if I didn’t know the basics of supervised machine learning, I would never have understood that ChatGPT could do this for me.
  • Knowing how to code is an incredible skill as the basics of computer science (in this case a csv) are the building blocks for the LLMs
  • The best kind of learning is slow, organic and primitive, slow is fast, fast is slow.
  • The winners in the future will be the people who take control of their time so that they can spend 90% learning, observing and 10% applying, doing

Artefacts

  • initial prompt → “hello and good morning, i have four csv files that each have training data about my cycling heart rate data, one of the files is missing heart rate data because my polar sensor broke, if I upload the files and tell you to examine the columns B Time, D Speed, E Pace and I Max Speed, C Heart Rate, in the file that has no HR data, can you make row by row entries that estimate the hr ?“
  • https://chatgpt.com/share/66f19437-2cfc-8004-abba-ee287b7c0620