Watari: A Security Control That Was Never On

Nothing was failing. The application worked, the tests passed, the policies were syntactically correct, and any reviewer reading migration 0002 would have concluded that tenant isolation was enforced. It was not.

That gap — between a control existing and a control operating — is the most useful thing I have to write about on this project, so this post is mostly about that.

What the project is for

Watari is for SOC analysts, CSIRTs and CERTs — the people who work incidents rather than report on them. It sits where TheHive and DFIR-IRIS sit, and it is younger and less battle-tested than either. If you need something proven in production today, use those.

Three things it does differently, and the first is the subject of this post:

• Real multi-tenancy, enforced at the database rather than in application code, so a query that forgets its tenant filter returns nothing instead of returning someone else’s incident.
• OCSF-native ingestion. Alerts are OCSF 1.8.0 Detection Findings, so any compliant producer POSTs straight to the API with no translation layer.
• Built for looking at things. Entity graphs, ATT&CK matrices, swimlane timelines and geospatial views are first-class rather than bolted on.

One command gets you a populated instance:

git clone https://github.com/BlueSquadron/Watari.git
cd Watari
./bootstrap.sh

The audit that started it

I was writing the integration guide — the document that walks someone through authenticating, ingesting an alert, promoting it to a case. Writing documentation against a running system is the cheapest audit I know, because prose forces you to make claims, and a claim is a thing you can check.

The claim I could not verify was the isolation one. So I connected as one tenant and counted rows.

I got all of them.

Four causes, one landing

The application connected as a superuser. POSTGRES_USER, which the Postgres container entrypoint always creates as SUPERUSER — and superusers bypass RLS unconditionally. No amount of correct policy matters. So: a new watari_app role, NOSUPERUSER, NOBYPASSRLS, and deliberately not the table owner.

ENABLE exempts the table owner. Migrations run as the owner, and the owner was also the role serving requests, so even a non-superuser would have been exempt. Eighteen tables moved to FORCE ROW LEVEL SECURITY.

The tenant context was never set. This is the one worth dwelling on, and it gets its own section below.

The policies raised instead of filtering. Found by the test suite rather than by reading. The policies cast the session setting straight to UUID. A transaction-local setting does not revert to NULL when the transaction ends — it reverts to the empty string. So on a pooled connection, the next transaction evaluated ''::UUID and threw a type error. Wrapping it in NULLIF makes an absent context yield NULL, which filters the row.

Any one of those alone does nothing or breaks the application. The unprivileged role alone changes no behaviour. FORCE alone locks out migrations. The context alone returns zero rows everywhere. Which is why it landed as one change with one verification, and why I would push back on a reviewer asking me to split it.

The bug that was ordering, not logic

The third cause is the one I keep thinking about, because every individual file was correct.

Routers declared their database session parameter before their auth parameter. FastAPI resolves dependencies in declaration order, so get_db opened a session before get_current_user had worked out who was calling — and the function that applies the tenant context read state that the auth dependency had not populated yet. Seventy of the seventy-one authenticated routes were affected.

Reordering seventy signatures would have worked. It would also have meant that the seventy-first route, written next month by someone who had not read this commit, reintroduced the bug — and reintroduced it silently, because the failure mode is a query that returns too much rather than an error.

So the fix went the other way. get_current_user now applies the tenant context to the request’s session once it knows the caller. FastAPI caches the session dependency per request, so the endpoint receives the same, already-scoped session regardless of what order it declared its parameters in.

The part that is legitimately cross-tenant

Splitting the roles forced a question I had been avoiding: which work is genuinely allowed to see every tenant?

The answer turned out to be a short, nameable list — migrations, seeding, the Celery worker, audit writes, and authentication lookups. Authentication has to find a user before it knows which tenant they belong to, so it cannot be scoped by the thing it is about to establish. That list connects to ADMIN_DATABASE_URL; everything on the request path connects as the unprivileged role.

Being forced to enumerate that list was worth more than the isolation fix. A cross-tenant capability you have written down is a cross-tenant capability you can review.

The dependency that was already written

Same audit, second finding. X-API-Key returned 401 on every route. The OCSF ingestion the README leads with, and eight examples in the integration guide, could not work as documented.

Two causes, and both of them are a particular kind of embarrassing.

The permission check every route goes through depended on the bearer-token dependency, which raises before an API key is ever considered. And a unified dependency that tries the bearer token then falls back to the key already existed in the codebase, written for exactly this purpose, referenced by nothing.

Swapping the two dependencies covered all seventy routes with no per-router edits, and because the unified path already applied the tenant context, isolation behaves identically for either credential.

The second cause was worse: the service-account role mapped to an empty permission set, so a key would have been refused with 403 even once it authenticated. There was a comment explaining that permissions resolved from a column on the user row. The function takes only a role and does no such thing, and nothing ever populated that column. A comment describing behaviour the code does not have is worse than no comment, and I replaced it with a note about what per-account scoping would actually require.

Service accounts now carry the analyst permission set — enough to ingest a finding and work the resulting case, and deliberately not tenant admin, so a leaked ingestion key cannot manage users, tenants or tenant configuration. A test pins that grant to exactly the analyst set, so widening it later has to be somebody’s deliberate act rather than a side effect.

A few routes stayed on the bearer-only dependency on purpose: logout and change-own-password are session-bound and meaningless for a machine credential.

The unglamorous pass

Between those two fixes I spent a day on something with no user-visible result, and I would spend it again.

CONTRIBUTING.md told a newcomer to run the formatters. Running them on a clean checkout rewrote 129 files, because they had never been run over the codebase — so anyone following the instruction produced an enormous unrelated diff alongside their change, and no reviewer could separate the two. make test selected a directory that matched no files and exited before the real suites ran. make test-integration omitted an environment variable and then hid the resulting failure behind a shell ||, so it reported success while a test failed. npm run lint had never worked, because the ESLint config had never been committed. ruff check reported 67 findings on a clean tree, so a contributor could not tell which were theirs.

Every one of those is an invitation withdrawn. The project says contributions welcome, and then the first command in the guide produces a mess the contributor has to explain.

The related decision: bootstrap.sh now refuses to start against an environment file written before the RLS change. An old .env points the application at the schema owner, which bypasses every policy — the stack comes up, everything looks fine, and isolation is silently off. It detects that, prints the exact lines to add, and stops. Failing loudly on a known-dangerous configuration is worth more than starting successfully.

Where it is honestly weak

Alpha, v0.1.0, Apache-2.0. Python 3.12 and FastAPI on the backend, React 18 on the front. The schema will move before 1.0.

The known gaps are published in CONTRIBUTING.md rather than kept in my head, and the largest one is embarrassing in a useful way: there is no CI. A workflow running the linters and the test suite would have caught several of the 500s fixed in earlier pull requests before they were ever committed, and would stop them coming back. It is written down as the highest-value contribution available, which is where I would rather have it than in a private list of things I know.

Smaller and equally real: Leaflet marker icons 404 on the map tab, some dashboard bars render as hairlines at certain viewport widths, and the audit log page is empty after seeding even though the seed script claims to write entries.

The most useful thing anyone can do is run ./bootstrap.sh, click around, and open an issue for anything surprising. Setup friction is a bug. One external contributor has already landed a fix — a transaction rollback that prevented a poisoned session on a conflict — and that is the kind of change I most want to see, because it came from someone actually running the thing.