Sumimi

Count kitties popping into my websites.

A self-hosted web analytics service that fits on a cheap VPS.

One executable and one DuckDB file. It serves its own tracker and dashboard, stores the last few years of your traffic in a fraction of the disk, and makes no outbound request of any kind — no account, no bill, nothing that phones home.

  • Rust + Axum + Tokio
  • DuckDB 2.0.0 embedded
  • One binary, one file
  • 1 vCPU / 512 MB RAM
  • Apache-2.0 licence

Why I built Sumimi

This started as a problem with my own server, not as an idea about a product. Here is the whole of it, because it is the reason every design decision in Sumimi looks the way it does.

  1. I have a small VPS — 1 GB of RAM, one core, a 20 GB disk — and for a long time it ran Umami for the traffic statistics of several small sites of mine. It is not a big machine. It is not supposed to be one. It is the box I already pay for.

  2. The first was the disk. It filled up. Umami keeps every page view in PostgreSQL — row after row, with thirty indexes on top — and the data is append-only: it grows forever and nothing is ever handed back. I was pruning by hand every few months, which is not a plan, it is a chore.

    The second was the dashboard. Every time I opened a panel there was a stutter, and after six months of history the stutter had become a wait: the page was aggregating half a year of rows on every click.

  3. Both problems had the same root. Analytics data is append-only: almost everything that arrives is new, and almost nothing is ever updated or deleted. That is exactly the shape of data that a columnar engine compresses well — store it as columns instead of rows, let the engine find the repetition, and write it to disk compressed.

    So the question stopped being "which analytics tool" and became "which storage engine". If the same disk could hold seven years of history instead of fifteen months, then with automatic backups and a rolling deletion policy on top, the service could sit there and watch those sites for years without me ever touching it again. It would stop being a chore and start being a status monitor for the sites I run.

  4. DuckDB — a columnar engine compiled into the binary, so the whole server is one file on disk with no database server to install, secure or upgrade. The measurements are in why DuckDB: on the same data it needs 50–85× less space than Umami on PostgreSQL, and the panel queries that took seconds take milliseconds.

Fifteen months of history became seven years on the same disk. The measured factor is larger than that — 50–85× depending on how repetitive your URLs and titles are — but the disk stopped being the thing I think about, and that was the whole point.

What it turned into

A single executable that is both the server and the analysis CLI. It speaks the Umami v2 HTTP API, so the official Umami dashboard and the existing Umami tracker snippets work against it unchanged — Sumimi is an independent open-source project and uses no Umami assets.

Where it fits

Two things that are usually a project of their own

On a cheap VPS, the two things that turn "copy one file onto a box" into a weekend are the certificate and the process management. Sumimi does both, so neither one is your problem.

And if you would rather not install anything at all, there is a container:

docker run -d --name sumimi \
  --restart unless-stopped \
  -p 3000:3000 -p 443:443 \
  -v sumimi-data:/data \
  -e SUMIMI_SSL_DOMAIN=stats.example.com \
  -e SUMIMI_SSL_EMAIL=you@example.com \
  ghcr.io/sumimi-dev/sumimi:latest

One command, one volume, one container — and Docker restarts it when it dies, which is the same job the supervisor does outside a container.

Design goal 1 — how much traffic a small box takes

The promise is a ceiling on the whole process, not just the database. The code pages, the collector, the queue, the pages the allocator keeps and DuckDB's own share all count, and the box runs out of all of them at once.

tierthe machineDuckDB's ceilingthe whole process
minimum1 vCPU / 512 MB128 MB256 MB
recommended1 vCPU / 1 GB256 MB512 MB

The table a deployment is sized from, measured on a 2 vCPU Linux box: the exact high-water mark, read by the server from its own /proc, with a fresh server for every row — because the promise is about the peak.

the rate offeredthe peak RSSthe engine's sharewhat it fits
5 000 /s110 MB2 MBthe minimum tier, with 146 MB to spare
10 000 /s115 MB2 MBthe minimum tier, with 141 MB to spare
20 000 /s (the tier's default)182 MB3 MBthe minimum tier, with 74 MB to spare
40 000 /s182 MB3 MBthe minimum tier — and the box is at its ceiling

The curve turned out to be flat

For a long time the rule was 68 MB + 7.8 MB per 1 000 requests a second — the peak grew with the traffic, because the allocator kept the pages of every request. Seven fixes later the curve is flat: 110 MB at 5 000 a second, 115 MB at 10 000, and 182 MB whether you offer 20 000 or 40 000. The memory is no longer what limits a small box. The CPU is — past what a core can parse, the traffic is refused with a Retry-After rather than queued, and the peak does not move.

One line of the environment file is worth more than every table row

MALLOC_MMAP_THRESHOLD_=65536 tells glibc to give the pages of a large block back to the operating system instead of keeping them. Measured on the same server, at the same rate, changing nothing else: −12% to −21% of the peak. The cost is a syscall per large block, which is about 5% of the throughput. It is a knob, not a free win — the Docker image sets it, and a bare-metal deployment should set it too.

Two honest caveats. The core is the limit before the memory is — past what a box can parse, the server refuses the traffic rather than queueing it, and the memory stays where it is. And the disk fills before either: 15 000 page views a second is 1.3 million views a day, which is about 50 GB a year of rows. Keep the history you want, and let sumimi prune and the retention window do the rest.

The whole method, the other machines and the numbers behind these tables are in the memory and capacity page.

The numbers that started it

The same traffic, stored two ways — Umami on PostgreSQL, and Sumimi on DuckDB. Both measured, on the same rows, with the same fields:

views a dayUmami + PostgreSQLSumimi + DuckDB
10 00017.3 GB / year0.20 GB / year
100 000173 GB / year2.0 GB / year
1 000 0001.73 TB / year20 GB / year

That is 85× less space on the marginal cost — the fair comparison, with the fixed overhead of an empty file taken out. On real site data, with real UUID identifiers, it is about 50×; the difference is how much your URLs and titles repeat. Either way the gap is not a detail: PostgreSQL spends 68% of its space on indexes on a table that is only ever appended to, and a columnar engine compresses the repetition instead.

The queries follow the storage. The same panels, the same data, two engines:

the panelDuckDBSQLite
summary8.3 ms234.7 ms
timeseries5.1 ms97.0 ms
top 50 pages1.4 ms70.9 ms
referrers2.1 ms68.7 ms
browsers3.8 ms120.3 ms
countries3.7 ms115.3 ms

That is the stutter, gone. The story of how each of these numbers was measured — and what the measurement traps were — is in why DuckDB and how much traffic a small box takes.

Where to go next

Sumimi / 数咪咪 is an independent open-source project. It is an open-source alternative to Umami and implements the Umami v2 HTTP API; no Umami assets are used. Licensed under the Apache-2.0 licence. Third-party names are referenced nominatively.