File downloader and digit analyzer
Submitted2026-07-27, one day
Drains a remote, rate-limited catalog of text files and reports digit statistics across them — built in a day, against the stack I most want to work in.
The service layer is split by responsibility, not by convenience.
Pacing, throttling, locks, the source API client, storage, digit analysis and identifiers each live in their own module and can be read alone.
Distributed politeness.
The external catalog allows one request per second and a hard batch cap of three. Both are enforced through Redis, so the limit holds across workers rather than per process.
An htmx interface that keeps its state.
A progress banner polls once a second, the table swaps in place so selections survive paging, and the digit breakdown loads in batches while the totals always reflect the entire selection rather than the visible page.
Tests that need nothing running.
respx mocks the source API; test settings swap Redis for local memory and PostgreSQL for in-memory SQLite. Zero network calls, no services to start.
Highlights
- A README with a configuration table, explicit constraints and a frank security note about what is and is not a secret
- Redis-backed request pacing and download locks
- A fully network-free test suite
Stack
Django 6 on ASGIhtmxCeleryRedisPostgreSQLnginxDocker ComposeuvicornwhitenoiseFlowerrespxi18n