skip to content
meistyr.tech/~luke
case study/004/in development

Project Signal

A community project built with a small group of developers: one player profile across games, replay stats, scrim finding and events. In early development.

read what led herenot yet public
role
Full Stack Engineer · Technical Lead
period
May 2026 — Present
status
In early development
01 — brief

What we're building

Project Signal is a gameplay statistics project being built by a small group of longtime members of the competitive gaming community.

02 — replay ingestion

Moving replay ingestion to Go

The replay parser turns each Rocket League replay into telemetry data. The NestJS API then takes that data in and writes it to the database in bulk. I'm designing a separate Go service to take over the ingestion and the database writes. Go can spread that work across CPU cores, and moving it out keeps the main API free to answer requests.

planned architecture

Select a part to see what it does and how it connects.

03 — decisions

The decisions behind it

PostgreSQL over MariaDB

Replay ingestion is moving to a pool of Go workers, and the jobs will queue in PostgreSQL. A job starts when someone acts on the site, so the API can notify the workers the moment it adds one, using Postgres’s LISTEN/NOTIFY. MariaDB has no equivalent, so its workers would have to keep checking the table. Keeping the queue in the same database also means a job and the data it belongs to will be saved in one transaction. A separate queue like Redis would be a second write that can fail on its own, leaving a job for data that was never saved.

cost

Leaving the database I had run in production for two years, and putting queue traffic on the same database the site uses.

Heartbeats over held locks

A worker could hold its job locked until it finishes, but a replay takes long enough to parse that the lock would stay open the whole time. Instead each worker will mark its job as running and record a heartbeat while it works. A job whose heartbeat goes quiet will be picked up by another worker, so a crashed worker won’t strand one.

cost

A slow worker looks the same as a dead one, so a job can run twice, and saving its results has to be safe to repeat. A malformed replay that crashes every worker needs an attempt limit, or it gets passed from worker to worker forever.

The API as the only way in

Everything outside the system goes through the API, the web app included, so nothing outside connects to the database and permissions are checked in one place. Services inside the system are different. The Go ingestion workers will read jobs from the database and write results to it directly, so heavy ingestion doesn’t wait on the API, and the API stays free to answer requests while ingestion runs.

cost

The workers will bypass the API’s checks. The database’s own constraints and a role that can only touch the ingestion tables will guard their writes. Each request from the site also takes an extra network hop.

04 — status
In early development. The full case study will be posted once people are using it.
next

Car Soccer City

League management platform for an established competitive league, in alpha with real stakeholders.