I’ve been prototyping an app where you can find your local past thunderstorms and watch the strikes play out at 3000x speed. I’m happy enough with it now that I’m publishing a first version of it: Lightning strikes in the Nordics & Baltics.

The app shows a map of Northern Europe. Under the map are two timeline charts. The bottom one is a heatmap of daily strikes over the past ~5 years (I’ll backfill more later) where the user can select a slice (24h/3d/etc). The top timeline chart shows the selection in greater detail. Both timelines adjust to the map view, so if you zoom in on a smaller region, the timelines will only count the strikes from within that area. That makes it possible to find storms from a local area.

The user can “play” the strikes inside the selected time span and get cool strike animations on the map. I doubt this app is truly useful for anything, but it’s nice to look at, and works as a tech demo. It was interesting to make and I’ve learned things along the way. There are still things I need to clean up or rewrite, but it’s at a place where other people can use it now.

During the summer I found myself wanting to look at where lightning had hit during thunderstorms. I didn’t really find any app that did this in a nice way, tho I didn’t spend many minutes searching. I guess it’s a bit of a niche, nerdy thing to want to see past lightning strikes on a map. A bit like looking at historic weather data in general. Weather apps generally only look into the future.

Process

When I learned that the Finnish Meteorological Institute (FMI) publishes lightning strike data for most of Northern Europe as open data, I knew I had to try to make something of it. It felt like a pretty self-contained, “easy” problem as long as I kept the features tight. But of course as usual, I underestimated all the little details that needed to be hashed out. I went down a bunch of different false paths before I arrived at the prototype I’m publishing today. I started out with a map that was just the outlines of each country. I pushed SVG points, representing the strikes, on the map and animated them with CSS. Unsurprisingly it didn’t work too well with thousands of concurrent strikes (even after refactoring to some smarter SVG), and the country outlines didn’t cut it when wanting to look at strikes somewhere far from the coast or borders.

Early in the process I realised I needed to group the strikes into storms, and just transfer the storm outlines to the browser for the “non-play” state. Showing all strikes all the time uses too many resources, both from the server’s and the browser’s perspective, hence I now show the semi-transparent red overlay representing the storm areas on the map. Playback still transfers individual strikes but that’s more manageable compared to always transferring strikes. The dynamic nature of the app ruled out pre-rendering images, which seems to be a common solution in other apps.

Having the storms and their polygons created some new ideas, like letting the user select individual storms and play just those, and I even had a Gantt-like timeline chart for the storms at one point, but with up to 40 concurrent storms it turned out pretty crazy, so I’ve scrapped all that at least for now.

Tech stack

I’m using Python to fetch the lightning strikes from FMI. They’re stored in an SQLite database that’s in WAL mode. The Python app doesn’t just store the strikes, it also groups them into storms according to some parameters, it generates GeoJSON storm polygons, and creates various indexes and tables that make it possible to quickly get the numbers needed for the timelines.

I chose Rust for the web app, which is new to me. I leaned heavily on LLMs to get going with Rust. You have to have a certain amount of discipline to make sure you understand what is going on and direct it properly. It can be a sort of turbo boost in getting started with and learning new things when used right. I think/hope that’s what I’ve done. I actually started with TypeScript and Deno. But once I realised I wanted to host this for other people as well, as fast as possible and without overpaying for the server, I switched to Rust and Axum.

The web app only reads the SQLite database. SQLite in WAL mode is a great fit for this purpose, as there’s a single writer, the Python app, and multiple readers: route handlers in the Rust app. The database is right there, in-process, no network hops, no write contention etc. The readers can do their queries freely while the writer is writing.

The web app is done with the help of Datastar. When the page loads in the browser it:

  • initializes the map and hooks up some events for the timelines
  • sets up some signals that hold frontend state, like the map pan and zoom positions, the timeline scroll positions etc
  • opens an SSE request to the backend that is kept open for the duration of the session

Every time the frontend state changes it’s sent to the server with a PUT. The SSE handler reacts to it and sends the needed HTML/SVG/GeoJSON/strike coordinates to the browser. The SSE handler is sort of like a game loop, it’s even got an FPS limiter to prevent doing unnecessary work when the state changes in quick succession. To make the app as light and as fast as possible the timelines use virtual scrolling: they only hold enough of the charts for the current view and a bit on the sides, so that there’s something there the instant the user scrolls.

This may seem like an unfamiliar approach to making web apps, but it’s actually incredibly freeing to drive the browser from where the data lives and not have to come up with a bunch of JSON APIs and abstractions. Simply put, the backend knows what’s going on, makes database queries, creates some HTML, tells the browser what to show. It results in less code, and it’s fast too.

The map is MapLibre with a couple of custom WebGL layers. They consume GeoJSON storm polygons and strike coordinates and make it pretty. MapLibre definitely was a great choice here.

Next

  • I want to refactor the web components to use the new Rocket web component API from the Datastar team. It looks perfect for this purpose and will streamline the “signals to MapLibre” code I have now.
  • Heatmap mode: I think a storm heatmap button/mode is needed. The red storm overlay doesn’t indicate the intensity of a storm and the play mode doesn’t give a proper overview of the strikes. A proper heatmap would be a good complement.
  • Speed control for the play animation.

Please try it out if it sounds interesting. I’m at the point where I’m looking for feedback!