🐍 Why your notebook is slower than your ideas


One CSV read cost me half a second

On September 23 at 10:00 am Eastern US time, I'm hosting a free webinar. In it, you'll found out the fastest route to becoming a Python quant:

  • Master the 20% of Python that drives 80% of quant results.
  • Build professional-grade backtests in under 30 minutes.
  • Automate execution by connecting Python directly to broker APIs.
  • Clean messy market data using the exact workflows used at J.P. Morgan.
  • Get the step-by-step roadmap to transition from “tutorial hell” to deploying real trading strategies.

It's based on my flagship course, Getting Started With Python for Quant Finance.

​Register for free here 👉​

Only the first 1,000 registrants will get a spot.

⭐️ ⭐️ ⭐️ ⭐️ ⭐️
" A top-tier introduction to Python and quant finance. "
– Zac T
​

You get maybe 40 minutes at the laptop after work.

Mine went like this. Download 30 years of daily bars, restart the kernel (the Python session running behind your notebook) because something was stuck in memory, re-read the CSV, then find a column parsed as text and restart again. Half a second per read isn't much on its own.

But I ran that loop most of one evening in October and never typed the moving-average crossover test I sat down to write.

Here are the numbers from a test I ran on my own laptop.

The file holds 3.8 million daily bars for the 500 companies currently in the S&P 500, 1994 through 2024, saved uncompressed with a date column, a ticker column, and five price and volume columns.

Current members only, so every company delisted since 1994 is missing from it. On an SSD with default settings and no dtype hints, pandas.read_csv averaged 458 milliseconds over three runs and polars.read_csv averaged 3.57 milliseconds.

I was paying that 458 milliseconds a dozen times an evening without once measuring it.

Time each step of your loader separately, the network download and the parse and the cleanup, then fix whichever one dominates.

Write the cleaned table to disk in a format that needs no parsing, and a restart costs you one read instead of a re-download.

That's the first piece of a system that pulls market data and keeps a clean copy on your machine.

Here's a Python example you can use now.

Assumes pd (pandas), pl (polars), and perf_counter from Python's time module are already imported, and that prices.csv sits in your working directory.

This prints two numbers in milliseconds, one for each library, on your file and your machine.

Wrap any other slow line in time_call the same way, a date parse or a group-by that averages rows bucket by bucket, and you'll see which one actually costs you the wait.

The last line writes the table to Parquet, a binary file format that stores each column separately, so the next restart reads binary instead of parsing 3.8 million lines of text.

Polars ran slower than pandas when I forward-filled missing closes grouped by ticker and sorted by date, both in eager mode, and I never worked out why, so time your own operations before you switch libraries for them.

[Content]

Your next steps

Time each step in your current loader separately, fix the slowest one, then write the result to Parquet and time that read too, because kernel startup is a separate cost you won't get back.

From there, write a script that pulls yesterday's bars, checks them against the last date already on disk, and rewrites the file (Parquet won't append in place, so you rewrite it or keep one file per year).

At 40 minutes a session, a month of evenings gets you a local price history you can query without waiting on a download.


Jason Strimpel, founder of PyQuant News

Say 👋 on X/Twitter, LinkedIn, or book a call with me. You can always reply to these emails. I check them all.

You got this email because you joined the PyQuant Newsletter or bought a product of mine.

You told me your email address is Reader.

Not right? Update your profile.

Want to stop getting these emails? No problem! Unsubscribe from all my emails.

Never investment advice. Use at your own risk. For educational purposes.

113 Cherry St #92768, Seattle, WA 98104

© 2026 PyQuant News, LLC

​

PyQuant Tips

For finance professionals, coders, and complete beginners. Automate your trading, investing, and data analysis with Python. No jargon, no fancy math, code you can use.

Read more from PyQuant Tips
Every strategy makes 1 of 3 bets: Use Python to tell you which one.

Sort any trading idea into 1 of 3 bets with 4 lines of Python Every strategy makes 1 of 3 bets: Use Python to tell you which one. My master's degree taught me MATLAB. Not Python. Since I didn't want to pay the $2,000 license fee, I watched every YouTube video I could find to teach myself. 96% of them were a complete waste of time. The other 4% buried me in strategy ideas. Statistical arbitrage. Trend and momentum. Pairs trading... Maybe that's you. A head full of trading ideas but nothing...

Use the workbook you already have.

Move from Excel to Python in 3 simple steps Use the workbook you already have. Most finance work still runs on Excel. When I was a derivatives trader, most the $7 billion operation was based on Excel. The process was the same: You build the model, drag the formulas down and refresh it every Monday. Eventually, the sheets got too big and wouldn't even open. The analysis was basic and not that interesting. An actuary in one of my courses said most of his analysis lived in Excel, and "it was...

You don't need a $30,000 terminal to build an AI research desk Let Python pick what matters. Let the agent explain it. A Bloomberg Terminal costs about $30,000 a year. So you do what everyone does: You paste news headlines into ChatGPT. Then a 10-K. Then a chart. Then another chat window. By Friday you have 40 chats, 12 browser tabs, and no idea if any of what ChatGPT came up with works. That's experimenting, which is fine, but the sooner you build a system, the sooner you get results. Every...