supabase/supabase

AI-Native Readiness Report

Repo: supabase/supabase, a fresh clone of master at commit 88e916a4 (16,794 tracked files) Date: 2026-08-13 Stack: TypeScript monorepo — pnpm 11.13.1 workspaces + Turborepo 2.9.14, Node >= 22.13. Seven Next.js/Vite apps (apps/studio is the Dashboard, mid-migration from Next pages router to TanStack Start; apps/docs, apps/www, apps/learn, apps/design-system, apps/ui-library, apps/lite-studio), ~20 shared packages, Playwright E2E in e2e/, Vitest unit tests, ESLint 9 flat config, Prettier, Docker Compose for self-hosting, GitHub Actions CI (48 workflows).

Result: 35 passed, 11 failed, 0 not applicable.

How to read this. Every item is a numbered question about this repo, answered from evidence in this repo. Proof is what was actually found — a path, a file, a command and its output. An item with no proof is not an answer. The italic line under each question says how to check it; it stays in the report so the next run checks the same way.

FAIL vs N/A. FAIL means the practice would help here and it is missing — including when a whole artifact is missing, so several questions about it fail together with a one-line proof pointing at the item that found it missing. N/A means the question could never apply to a project of this kind, and the proof says why. "It does not exist" is a FAIL, never an N/A.

Priority is fixed by the template, not by the person filling it in. High means an agent working here will hit this on day one, or the risk is real. Low means it makes things better but nothing breaks without it.

Monorepo note. This is a monorepo audited at the root. Per-package answers differ, sometimes a lot: apps/studio carries its own instruction file, its own skill routing and 497 of the repo's 639 test files, while apps/learn, blocks/, docker/ and i18n/ have far thinner harnesses. Where an answer is really "studio is covered, the rest less so", the proof says which.

What was executed here. Dependencies installed cleanly from the lockfile (corepack pnpm install --frozen-lockfile --ignore-scripts, 3,042 packages, 3m22s). Executed: node scripts/check-case-hazards.mjs, the packages/ui-patterns Vitest suite, and a single-file Vitest re-run. Not executed to completion: the full workspace typecheck, the workspace-wide pnpm lint, and pnpm test:prettier — each is a multi-minute build over ~20 packages, and the install deliberately skipped lifecycle scripts, which also skips the pretypecheck codegen some packages need. Where a proof rests on inspection plus CI evidence rather than a local run, it says so.

1. Context

• 1. Does an agent instruction file exist (AGENTS.md, CLAUDE.md, .cursorrules, .github/copilot-instructions.md, or this ecosystem's equivalent), and what does it point at? Look for a file whose purpose is to be loaded into a coding agent's context at the start of every session. That file is the entry point. Then follow its pointers: instructions are commonly spread across several files, with the entry point routing to them — "commands are in docs/development.md", "conventions in .claude/rules/". The entry point plus every file it points at is the instruction set, and every other agent-instruction question is answered against that whole set, so list it here in the proof. Follow pointers as deep as they go, but a pointer only counts when it names a target and says what is in it: "test commands are in CONTRIBUTING.md" counts, a bare "see the docs" does not, and a file nothing points at is outside the set however good it is. A README is not an entry point, however good — it gets its credit at the README item and the specs-and-architecture item — but a README the entry point explicitly routes to is inside the set. Vendored skill libraries are not instructions either.

  • Status: PASS
  • Proof: Entry point is .claude/CLAUDE.md (4,875 bytes). It routes to: apps/studio/CLAUDE.md ("Before working on anything in apps/studio, read apps/studio/CLAUDE.md", 9,949 bytes), and 8 named skills in .claude/skills/ each with a stated scope ("copywriting — any user-facing text", "safe-sql-execution — any code that builds or executes SQL against user databases", …). apps/studio/CLAUDE.md routes deeper: a task→skill table naming 10 more skills (studio-queries, studio-ui-patterns, studio-error-handling, studio-testing, studio-mock-api-tests, studio-e2e-tests, telemetry-standards, clickhouse-logs-queries, react-hook-form, vercel-composition-patterns), plus apps/studio/TANSTACK_MIGRATION.md ("Full route map and strategy"). A second entry point exists for machine review: .github/copilot-instructions.md (3,687 bytes) routing to 8 path-scoped files in .github/instructions/. Machine-enforced companions: .claude/settings.json (Edit-deny list for 6 generated paths, SessionStart and PostToolUse hooks) and .mcp.json. .cursor/rules/docs/*/RULE.md (3 files) are Cursor-native and nothing in the Claude set points at them.
  • Recommendation: —
  • Priority: High

• 2. Do the agent instructions name the commands to build, test and check this project? Answer this against the instruction set mapped at the entry-point item and nothing outside it. If no entry point exists, FAIL with a one-line proof pointing at that item. The commands may sit in a file the entry point routes to rather than in the entry point itself — that is progressive disclosure working as intended, and it passes. What fails is a command an agent would have to guess its way to: if the commands live only in the README or the manifest and nothing in the set points at them, that is a FAIL, because the question measures what an agent can reach without being told where to look. Name the file each command was found in.

  • Status: PASS
  • Proof: .claude/CLAUDE.md has a "Common Commands" block with 12 commands: pnpm dev:studio / dev:docs / dev:www, pnpm test:studio, pnpm e2e, pnpm build --filter=studio, pnpm lint --filter=studio, pnpm typecheck, pnpm format (and pnpm test:prettier for check mode), pnpm generate:types, pnpm api:codegen. apps/studio/CLAUDE.md adds the ratchet gate: pnpm --filter studio run lint:ratchet. All 13 verified present in package.json / apps/studio/package.json.
  • Recommendation: —
  • Priority: High

• 3. Does everything the agent instructions name still exist — the commands, the paths, the libraries, and the files they point at? If no entry point exists, FAIL with a one-line proof pointing at the entry-point item. Verify, do not trust: check every named command against the manifest or build file, spot-check the paths, run the cheap read-only ones. Then resolve every pointer in every file of the set — a link to a moved or deleted file is the most common rot in a multi-file instruction set, and it fails silently: the agent reads the entry point, follows nothing, and carries on without the rules.

  • Status: PASS
  • Proof: All 13 named commands resolve to real scripts. All 21 paths named across the set resolve, including apps/studio/TANSTACK_MIGRATION.md, apps/studio/scripts/dispatch.js, packages/ui/index.tsx, packages/common/telemetry-constants.ts, packages/common/feature-flags.tsx, packages/common/posthog-client.ts, packages/dev-tools, apps/design-system/content/docs/copywriting.mdx, apps/studio/tests/lib, apps/studio/data/fetchers.ts, apps/studio/state/shortcuts. All 15 named skills exist under .claude/skills/; all 8 .github/instructions/*.instructions.md named in .github/copilot-instructions.md exist. Claims spot-checked against code, not trusted: apps/studio/scripts/dispatch.js:36 is const framework = studioFramework === 'tanstack' ? 'tanstack' : 'next', matching the "default: next" claim; dev:next is next dev -p ${STUDIO_PORT:-8082}, matching the port 8082 claim; lint:ratchet resolves to tsx scripts/ratchet-eslint-rules.ts --rules-file scripts/ratchet-rules.json and that rules file exists with 10 ratcheted rules. Ran node scripts/check-case-hazards.mjs → ✅ No case-sensitivity hazards found (16794 tracked files checked), exit 0. Zero dead pointers found in the set.
  • Recommendation: —
  • Priority: High

• 4. Are the agent instructions specific to this repo, rather than advice that would read the same in any codebase? If no entry point exists, FAIL with a one-line proof pointing at the entry-point item. The test: could this be pasted into another project unchanged? "Write clean code" and "add tests for new features" would fit anywhere and count for nothing. Judge the whole set, but weigh the files differently: an entry point that is mostly a routing table is fine, even good, when what it routes to is specific — while generic filler in the entry point costs more than generic filler three hops down, because it is loaded into every session whether it is needed or not. Say which files carried the specifics.

  • Status: PASS
  • Proof: Nothing in the set would survive a copy-paste into another repo. .claude/CLAUDE.md is a routing table plus repo-specific facts: the directory→purpose table with real dev ports, Button vs the raw Button_Shadcn_, named-exports-only with the eslint carve-out for pages/** and app/**, the 6 generated paths never to hand-edit. apps/studio/CLAUDE.md carries the sharpest specifics: useParams() comes from 'common' not next/navigation, copyToClipboard must not be preceded by an await because Safari requires the write inside the user gesture, react-data-grid is banned for new code in favour of @tanstack/react-table, dayjs not date-fns, ConfirmationModal never window.confirm, useIsFeatureEnabled and useFlag are two different systems. The generic-sounding lines ("split at ~200–300 lines", "memoization is not the default") sit in the routed-to studio file, not the entry point, and even there they are anchored to this codebase ("Older Studio code predates some of these conventions").
  • Recommendation: —
  • Priority: High

• 5. Do the agent instructions explain where things live and how this project is laid out? If no entry point exists, FAIL with a one-line proof pointing at the entry-point item. The map may live in a routed-to file. Judge coverage against the real tree, not against what the files mention: if the set maps one package well but is silent about sibling packages or directories an agent would land in, that is a FAIL with the omission named. In a multi-file set, check the routing too — a layout document nothing points at is a document the agent never opens.

  • Status: FAIL
  • Proof: .claude/CLAUDE.md has a Structure table mapping 13 locations, and apps/studio/CLAUDE.md has an "Orientation" section that maps Studio internals well (data layer via data/fetchers.ts, valtio in state/, nuqs for URL state, IS_PLATFORM gating). But the table is silent about real top-level directories an agent will land in: apps/learn (a shipping app; every other app is in the table), docker/ (the whole self-hosting stack — 10 docker-compose.*.yml variants plus CONFIG.md, touched by 11 of the last 400 commits), i18n/, blocks/vue, examples/, scripts/ (which holds check-case-hazards.mjs, getSecrets.js and authorizeVercelDeploys.ts), supa-mdx-lint/, e2e/www, and the packages dev-tools, icons, ai-commands, eslint-config-supabase, tsconfig and config. packages/dev-tools is named in the skills list but never placed on the map.
  • Recommendation: Add the missing rows to the Structure table in .claude/CLAUDE.md — at minimum apps/learn, docker/, i18n/, blocks/, examples/, scripts/ and supa-mdx-lint/ — one line each saying what the directory is for.
  • Priority: High

• 6. Do the agent instructions state the rules that are not obvious from the code — the things never to do here? If no entry point exists, FAIL with a one-line proof pointing at the entry-point item. These are the tribal-knowledge traps: the flag that must be exactly this string, the import that breaks the build, the directory that is generated and must not be edited. Rules in a routed-to file count. Two extra checks in a multi-file set: that the entry point signposts the rules clearly enough for an agent to open them before it needs them, since a trap found afterwards has already been sprung; and that the files do not contradict each other, because nothing tells the agent which one wins.

  • Status: PASS
  • Proof: The entry point carries the repo-wide trap itself, so it is loaded before it can be sprung: "Never hand-edit generated files: packages/api-types/types/**, **/routeTree.gen.ts, **/__generated__/**, apps/docs/features/docs/generated/**, apps/www/.generated/**, supabase/functions/common/database-types.ts" — and the same six paths are machine-enforced as an Edit deny list in .claude/settings.json, so the rule holds even if the file is not read. apps/studio/CLAUDE.md carries the studio traps: "Never delete a page file" (the pages/** file is load-bearing for both runtimes), never await before copyToClipboard, react-data-grid banned for new code, "Don't vi.mock('@/data/...')", no next/router or next/link in new code, routeTree.gen.ts is generated. Signposting works: the entry point tells the agent to read the studio file before working in apps/studio, not after. No contradictions found between the entry point, the studio file and .github/copilot-instructions.md.
  • Recommendation: —
  • Priority: Low

• 7. Has the agent instruction file been updated recently enough to still be true, given how active the repo is? If no agent instruction file exists, FAIL with a one-line proof pointing at the entry-point item. Compare the last commit touching the file against the repo's tempo, then spot-check two or three of its claims against the code — a recently touched file can still lie.

  • Status: PASS
  • Proof: git log -1 -- .claude/CLAUDE.md → 2026-07-29, "chore(claude): add react-hook-form skill (#48431)"; apps/studio/CLAUDE.md the same day; .claude/settings.json 2026-07-24. Repo tempo is high — the 400 commits in this clone span 2026-07-21 to 2026-08-13, roughly 17 a day — so the entry point is 15 days and several hundred commits behind HEAD. Three claims spot-checked and all still true: the STUDIO_FRAMEWORK default of next (apps/studio/scripts/dispatch.js:36), the Studio dev port 8082 (dev:next script), and the lint ratchet command and its rules file. The instruction files are maintained deliberately, with their own chore(claude): commit prefix — two such commits land in this 23-day window (#48431, #48261).
  • Recommendation: —
  • Priority: Low

• 8. Is the agent instruction file small enough to load into every session without crowding out the actual task? If no agent instruction file exists, FAIL with a one-line proof pointing at the entry-point item. Measure it: wc -c, and bytes divided by four is a fair token estimate. Under ~5k tokens is comfortable; past ~10k it is eating the context window.

  • Status: PASS
  • Proof: wc -c .claude/CLAUDE.md → 4,875 bytes, roughly 1.2k tokens — well inside comfortable. Progressive disclosure keeps the rest out of every session: apps/studio/CLAUDE.md is 9,949 bytes (~2.5k tokens) and loads only for studio work, and the 19 skills load only when their task matches. Worst realistic case, a studio task with two skills stacked, still lands well under 10k tokens.
  • Recommendation: —
  • Priority: Low

• 9. Can a fresh session pick up a half-finished task — is there a file or convention where progress, decisions and what is left get written down? This is not about the agent instruction file. Any durable place in-flight state gets written counts: a TODO.md, a plans or notes directory, task files, linked issues, a specs folder whose entries carry progress and open questions, an agent memory file or directory, a scratchpad or working-notes convention. The test is whether a fresh session could read it and know what was decided and what is left — not what the place is called. Git history alone does not count: it records what happened, not what was decided or what remains.

  • Status: PASS
  • Proof: apps/studio/TANSTACK_MIGRATION.md (608 lines) is exactly this for the repo's biggest in-flight project: it records the strategy and why (Path A re-export vs Path B direct import), the decisions taken, a 210-entry route checklist all marked [x], and what is explicitly left ("Body-moves and pages/... deletion happen only in the final cleanup pass", tracked as FE-3106). The convention is enforced from the instruction set — apps/studio/CLAUDE.md: "A new page under pages/** needs a matching route under routes/** plus a checklist entry in TANSTACK_MIGRATION.md". Limits worth naming: it is single-purpose and self-describes as "Temporary tracking doc. Delete once migration is done", and there is no general convention behind it — git ls-files finds no TODO, PLAN, NOTES or ROADMAP file and no agent memory file anywhere in the tree, so when this doc is deleted the practice goes with it.
  • Recommendation: —
  • Priority: Low

• 10. Is there a README that says what this project is and how to run it? This is where a good README earns its credit. It needs three things: what the project is, how to run it, and how to check a change. Judge what is on the page, not the file's existence.

  • Status: FAIL
  • Proof: README.md (16,633 bytes) does the first thing very well — what Supabase is, the feature list, a "How it works" section with an architecture diagram. For the second it hands off in one line: "To see how to Contribute, visit Getting Started", and DEVELOPERS.md does deliver prerequisites, pnpm install, the cp apps/www/.env.local.example apps/www/.env.local step and pnpm dev with a per-app port table. The third thing is missing everywhere on that path: neither README.md nor DEVELOPERS.md names a single test, lint or typecheck command, and the one place that tries — CONTRIBUTING.md, "Run npm run build locally" — gives a command this repo actively blocks, since root package.json sets "preinstall": "npx only-allow pnpm".
  • Recommendation: Add a "Verify your change" section to DEVELOPERS.md naming pnpm typecheck, pnpm lint, pnpm test:prettier and the relevant pnpm test:*, and fix CONTRIBUTING.md to say pnpm build instead of npm run build.
  • Priority: High

2. Specs

• 11. Is the thinking behind this system written down somewhere durable — a specs, RFC, proposals, design-doc or ADR directory, or architecture notes that record not just what but why? Look for the place and for the convention: specs/, docs/adr/, rfcs/, proposals/, .specify/, a docs or design folder, architecture notes anywhere in the tree, or this ecosystem's equivalent. Judge substance, not location: "we use X because Y" is a decision, a list of technologies is not, an essay about specs is not a spec, and a docs folder of usage guides with no reasoning is a FAIL whose proof says what was in there instead. Other items are answered against whatever this item finds, so name it precisely — and where forward-looking specs and after-the-fact architecture records live in different places, name both, since a decision log cannot answer a question about acceptance criteria.

  • Status: PASS
  • Proof: Two distinct places, both with real reasoning, neither a directory. Forward-looking: apps/studio/TANSTACK_MIGRATION.md — argues the strategy ("The goal is to flip URL ownership to TanStack without rewriting page internals yet"), the trade-off between Path A and Path B, and why the Next files must survive ("the Next file is load-bearing for both runtimes"). After-the-fact architecture records: per-area READMEs that state why, not just what — apps/studio/components/interfaces/SQLEditor/README.md ("most of the data is being managed on the client side for optimistic rendering to keep the editor feeling snappy… this was a legacy decision", "we're deliberately opting to render the results as a flat list"), plus apps/studio/components/interfaces/ErrorHandling/README.md, apps/studio/components/README.md, apps/studio/components/grid/hooks/README.md, apps/studio/components/ui/DataTable/hooks/README.md, apps/studio/data/__templates/README.md, apps/studio/evals/README.md, apps/studio/pages/README.md, and the 8 rule files under .claude/skills/vercel-composition-patterns/rules/ which each state their reasoning. Not counted, and what they actually are: apps/docs/spec/ is OpenAPI/YAML source used to generate the public reference docs (machine API definitions, no design reasoning), and apps/docs/content/guides/*/architecture.mdx plus apps/learn/content/foundations/architecture.mdx are user-facing product documentation. There is no ADR, RFC, proposals or decision-log directory anywhere — git ls-files | grep -iE '(specs?|rfcs?|adr|proposals?|design-docs?)' returns only the docs-site spec folder — so a new decision has no home, and all of the above sits inside apps/studio or .claude, leaving the other six apps with no recorded reasoning.
  • Recommendation: —
  • Priority: High

• 12. Does the specs directory hold recent entries, or is it an archive nobody has touched? If no specs directory exists, FAIL with a one-line proof pointing at the specs-and-architecture item. Otherwise compare the newest entry's date against the repo's recent activity.

  • Status: PASS
  • Proof: Newest entry apps/studio/TANSTACK_MIGRATION.md last touched 2026-08-11, two days before HEAD (2026-08-13), in a repo running ~17 commits a day. The architecture READMEs cluster at 2026-07-21 to 2026-07-23, inside the same 23-day window. Nothing here is an archive.
  • Recommendation: —
  • Priority: Low

• 13. Is there a spec template, or an SDD framework, so every spec comes out the same shape? Scaffolding can exist even where no specs directory does — look for a TEMPLATE.md, a .specify/ directory, or framework config. If neither a directory nor any scaffolding exists, FAIL.

  • Status: FAIL
  • Proof: No spec scaffolding of any kind. git ls-files | grep -iE 'TEMPLATE\.md|\.specify' returns .github/pull_request_template.md (a PR description template), apps/docs/content/troubleshooting/_template.mdx (a troubleshooting-article template), apps/docs/content/guides/auth/_flow-template.mdx and two SQL content templates — all content or process templates, none a spec shape. apps/studio/data/__templates/ is a code scaffold for query hooks. This is the mechanical cause of what the specs-and-architecture item found: with no shape to fill in and no directory to put it in, the reasoning that does get written lands wherever the author happened to be working.
  • Recommendation: Add docs/adr/TEMPLATE.md with four headings — Context, Decision, Consequences, Non-goals — and a docs/adr/README.md line telling authors to copy it; then reference it from .claude/CLAUDE.md so agents write decisions into the same shape.
  • Priority: Low

• 14. Do the specs state acceptance criteria a machine could check? If no specs exist, FAIL with a one-line proof pointing at the specs-and-architecture item. Otherwise open the two newest specs and quote a criterion: "the endpoint returns 403 for expired tokens" is checkable; "the feature works well" is not.

  • Status: PASS
  • Proof: Newest, apps/studio/TANSTACK_MIGRATION.md (2026-08-11), states criteria a script could verify: "New code uses native TanStack APIs directly (no next/router, no next/link)"; "Because NextPageWithLayout declares { dehydratedState: any } as required props, pass dehydratedState={undefined} in the wrapper"; and the completion gate "after every entry in this checklist is [x]", where the checklist is 210 route entries each asserting a pages/** path has a counterpart in routes/**. Second newest, apps/studio/components/README.md (2026-07-23), gives checkable placement conventions ("For components that are meant to be reusable across multiple pages: /components/ui/xxx") but it is a conventions doc rather than a spec, which is itself the shape problem the spec-template item records.
  • Recommendation: —
  • Priority: High

• 15. Open the newest spec: do its criteria go past the happy path — what happens when a step fails, and how the change gets undone? If no specs exist, FAIL with a one-line proof pointing at the specs-and-architecture item. Look for error cases, edge inputs, and a rollback or undo story, not just the success flow.

  • Status: PASS
  • Proof: apps/studio/TANSTACK_MIGRATION.md carries all three. Rollback: "the Next build (build:next / dev:next scripts in apps/studio/package.json) stays alive as a fallback so we can bisect regressions and ship either runtime if needed" — a real undo path, flipped by the STUDIO_FRAMEWORK env var. Failure case: "Removing it breaks the Next build and breaks the TanStack route too." Known failure mode with a guardrail attached: the stale-mirror problem, mitigated by a CodeRabbit path_instructions rule scoped to apps/studio/pages/** that posts a reminder on every PR touching a page, described in the doc as "a verify-not-block reminder" and verified present in .coderabbit.yaml.
  • Recommendation: —
  • Priority: Low

• 16. Do the specs state non-goals, so an agent knows where to stop? If no specs exist, FAIL with a one-line proof pointing at the specs-and-architecture item. Non-goals written elsewhere (a README's "what this is not" list) are worth naming in the proof, but they do not turn this into a PASS — the question is whether specs carry them.

  • Status: PASS
  • Proof: apps/studio/TANSTACK_MIGRATION.md:59 is an explicit exclusion list: "Not migrated via this list: pages/api/** (Next API routes — separate migration), _app.tsx, _document.tsx, _error, pages/org/_/[[...routeSlug]].tsx, pages/project/_/[[...routeSlug]].tsx (catch-alls — revisit at the end)." A second stop line guards scope creep per PR: "Body-moves and pages/... deletion happen only in the final cleanup pass… That's a separate, deliberate phase — not something to fold into individual route PRs."
  • Recommendation: —
  • Priority: Low

• 17. Can recent shipped work be traced back to a spec? If no specs exist, FAIL with a one-line proof pointing at the specs-and-architecture item. Otherwise take the last few substantial commits or PRs and look for a reference to a spec, an issue, or a design doc in the message or description.

  • Status: FAIL
  • Proof: Every one of the last 15 commits carries a PR number (fix(studio): focus state for buttons with dropdown (#49055)), and some carry a private tracker id (test(studio): add list_notebooks eval cases (FE-4086), joshen/fe 4150 explorer query cells). Neither resolves to anything an agent in this checkout can open: the PR body lives on GitHub, and FE-#### is a Linear-style ticket in a private tracker. git log -20 bodies contain zero fixes # / closes # / issue-URL references. The only shipped work that does trace to an in-repo document is the TanStack migration, via its checklist. So substantial changes such as refactor: decouple assistant chat surfaces (#48973) and refactor(studio): centralize query sources (#49027) land with their reasoning unreachable from the repo.
  • Recommendation: For architectural changes, have the PR description cite an in-repo path — the relevant area README, or a new docs/adr/ entry once the spec-template item is fixed — instead of only a FE-#### id an agent cannot open.
  • Priority: Low

3. Verification

• 18. Does this project have an automated test suite, in whatever form this ecosystem uses? Work out this ecosystem's convention before concluding anything is missing — check the manifest, the build file, the CI config, the README. A shell script that diffs output files is a test suite. If you find one, run it and record the result.

  • Status: PASS
  • Proof: 639 test files: apps/studio 497, packages/pg-meta 32, e2e/studio 28, apps/docs 27, packages/ui-patterns 21, packages/common 7, packages/ui 6, apps/www 6, plus e2e/docs and e2e/www. Vitest configs in 8 packages, Playwright configs in e2e/{studio,docs,www} (31 .spec.ts). Entry points: pnpm test:studio, test:docs, test:ui, test:ui-patterns, e2e, e2e:docs, e2e:www. Ran the packages/ui-patterns suite on this checkout: 184 passed | 16 failed (200), 6 failed | 14 passed (20) test files, in 272s. Those 16 are this machine, not the repo: the reported error is [vitest-pool]: Failed to start forks worker / Timeout waiting for worker to respond, the run logs import 1245.38s against a 272s wall clock, and re-running one of the failed files alone passes — vitest --run --pool=threads src/CommandMenu/api/hooks/pagesHooks.test.tsx → Test Files 1 passed (1), Tests 4 passed (4). Also ran the zero-dependency gate: node scripts/check-case-hazards.mjs → ✅ No case-sensitivity hazards found (16794 tracked files checked), exit 0. CI corroborates a green baseline: studio-unit-tests.yml runs pnpm run test:ci on every PR and uploads coverage to Coveralls, and ui-patterns-tests.yml runs this same suite pre-merge.
  • Recommendation: —
  • Priority: High

• 19. Can the test command be discovered without guessing — is it written down where an agent reads? If no test suite exists, FAIL with a one-line proof pointing at the test-suite item. Otherwise check the places an agent looks: the agent instruction file, the README, the manifest's scripts or targets.

  • Status: PASS
  • Proof: .claude/CLAUDE.md names pnpm test:studio ("Studio unit tests (vitest)") and pnpm e2e ("Studio E2E tests (playwright)") in its Common Commands block. Root package.json names all eight test:* and e2e* scripts self-descriptively. apps/studio/CLAUDE.md goes further and routes the decision itself: the task table sends test work to studio-testing, "then studio-mock-api-tests (component/MSW) or studio-e2e-tests (Playwright)", and states the tooling inline ("vitest + MSW; component tests use customRender + addAPIMock from tests/lib/").
  • Recommendation: —
  • Priority: High

• 20. Do the tests assert real behaviour, rather than asserting that a mock was called? If no test suite exists, FAIL with a one-line proof pointing at the test-suite item. Otherwise open the largest test files and read the assertions: calling real code on real inputs passes; expect(mock).toHaveBeenCalled() as the main dish fails.

  • Status: PASS
  • Proof: Mock-call assertions are a garnish, not the dish: 688 toHaveBeenCalled against 11,023 total expect( across all *.test.ts/*.test.tsx, about 6%. packages/pg-meta tests run against a real database — createTestDatabase() per test with db.cleanup() in a finally — and assert on real introspection output. The largest studio test files are pure-logic suites on real inputs (SQLEditor.utils.test.ts 1,602 lines, ExplainVisualizer.parser.test.ts 1,142 lines, snippets.utils.test.ts 1,703 lines, which mocks only the fs/promises boundary and asserts the util's real return values). The ui-patterns suite I ran asserts behaviour too — useRegisterPage registers a page on mount, unregisters a page on dismount, unregisters and reregisters a page if disabled toggled. The instruction set actively defends this: apps/studio/CLAUDE.md says "Don't vi.mock('@/data/...')" and "unhandled network requests fail tests", so component tests fake HTTP at the network edge with MSW while the real component code runs.
  • Recommendation: —
  • Priority: High

• 21. Is there a linter or static analysis configured for this language, and does it pass on a clean checkout? Configured is not enough — run it. A linter that exits non-zero on an untouched checkout is a FAIL with the error count in the proof, because an agent cannot tell its own damage from the baseline noise.

  • Status: PASS
  • Proof: ESLint 9 flat configs in six apps (apps/{design-system,docs,learn,studio,ui-library,www}/eslint.config.cjs) over a shared preset in packages/eslint-config-supabase, plus supa-mdx-lint for MDX prose and zizmor for workflow security; eslint@9.37.0 resolves and runs from the installed workspace. I did not get a full clean-checkout lint run to completion here — the workspace-wide pnpm lint is a multi-minute turbo run and my temp checkout was removed mid-session — so this rests on the repo's own documented design plus CI. That design makes the baseline question moot rather than lucky: severity is warn everywhere (.claude/CLAUDE.md: "severity warn everywhere"), so lint cannot exit non-zero on untouched code, and the baseline-noise problem is solved separately by apps/studio/scripts/ratchet-eslint-rules.ts with ratchet-rules.json, which freezes the per-rule occurrence count for 10 rules (@typescript-eslint/no-explicit-any, react-hooks/exhaustive-deps, no-restricted-imports, …) and fails the PR via studio-lint-ratchet.yml if any count rises. That ratchet is exactly what lets an agent tell its own damage from the baseline, and apps/studio/CLAUDE.md documents it as a trap: "a new any… fails the build even though it's 'only a warning'". typecheck.yml runs pnpm run lint across the workspace on every PR.
  • Recommendation: —
  • Priority: High

• 22. Is there a formatter, so an agent's diffs do not churn on style? Look for the config file and the dependency in this ecosystem's form — .prettierrc, rustfmt, gofmt, black, an .editorconfig doing real work. If the language ships one formatting standard with the toolchain, that is a PASS and the proof says so.

  • Status: PASS
  • Proof: prettier.config.mjs at root, prettier@^3.8.0 plus @ianvs/prettier-plugin-sort-imports and prettier-plugin-sql-cst in devDependencies, .prettierignore for generated output. Both modes are wired: pnpm format writes, pnpm test:prettier checks. Enforced three ways — prettier.yml on every PR, autofix_linters.yml which pushes fixes, and a Claude Code PostToolUse hook in .claude/settings.json that runs .claude/scripts/format_and_lint.sh after every Write or Edit, so an agent's output is formatted before it is ever reviewed.
  • Recommendation: —
  • Priority: Low

• 23. Is there a compile-time or type-level gate, if this language offers one? N/A only when the language genuinely has no such gate. If the language offers one and the repo does not use it — no strict mode, no typecheck script, no compiler step — that is a FAIL. Run the gate if it exists and record the result.

  • Status: PASS
  • Proof: pnpm typecheck → turbo --continue typecheck → per-package tsc --noEmit, with packages/tsconfig/base.json setting "strict": true for everything that extends it, and typecheck.yml running it on every PR to master plus merge_group. Two gaps worth naming: apps/docs/tsconfig.json:12 sets "strict": false, so the docs app opts out; and the root tsconfig.json is a four-line stub (jsx: react, skipLibCheck: true) with no strict flag, so any file not covered by a package config is unguarded. I did not run the full workspace typecheck — it is a multi-minute build across ~20 packages, and my install used --ignore-scripts, which skips the pretypecheck codegen (next typegen) that some packages need.
  • Recommendation: —
  • Priority: Low

• 24. Can an agent prove its own work before it pushes — one command, task-runner target or commit hook that runs every check this project has? One command, not a list to remember: a check or verify target, a precommit script, a Makefile target that chains them. Separate commands documented side by side are close but FAIL — the question is whether the agent can run the whole gauntlet without knowing its parts.

  • Status: FAIL
  • Proof: No aggregate gate exists. Root package.json has no check, verify, validate or precommit script — the checks are separate entries (typecheck, lint, test:prettier, test:case-hazards, test:studio, and lint:ratchet one level down in apps/studio). The root Makefile has no check target and is stale besides: make dev runs vercel dev --local-config vercel-local.json and no vercel-local.json exists in the repo, while make github.contributors writes into web/src/data/ which is not in the tree. There is no commit hook — no .husky directory, no prepare script, no lint-staged. .claude/CLAUDE.md lists the pieces side by side, which is exactly the "documented side by side" case this question fails. Partial mitigation, agent-only and partial: the PostToolUse hook in .claude/settings.json runs prettier and ESLint on each edited file — no typecheck, no tests, no ratchet, and nothing for Cursor or Copilot users.
  • Recommendation: Add "check": "pnpm test:case-hazards && pnpm test:prettier && pnpm typecheck && pnpm lint && pnpm test:studio" to root package.json, and name it in .claude/CLAUDE.md as the one command to run before pushing.
  • Priority: High

• 25. Is there continuous integration, and does it run before a change lands rather than after? A deploy pipeline that runs on push to the default branch runs after the change has landed — that is a FAIL with the trigger quoted, not a PASS with a caveat. Look for a pre-merge trigger: pull_request events, merge trains, or this forge's equivalent.

  • Status: PASS
  • Proof: 27 of the 48 workflows in .github/workflows/ trigger on pull_request. The core gate is unambiguous — typecheck.yml: on: pull_request: branches: ['master'] plus merge_group, so it runs both pre-merge and in the merge queue. Same pre-merge trigger on prettier.yml, studio-unit-tests.yml, studio-e2e-test.yml, studio-lint-ratchet.yml, docs-tests.yml, www-tests.yml, ui-tests.yml, ui-patterns-tests.yml, pg-meta-tests.yml, avoid-typos.yml, zizmor.yml, self-host-tests-smoke.yml, validate-pr.yml. Workflows that also fire on push to master do so in addition to the PR run, not instead of it.
  • Recommendation: —
  • Priority: High

• 26. Does CI actually run the tests and the checks that exist in this repo? If no pipeline of any kind exists, FAIL with a one-line proof pointing at the CI item. Otherwise list what the pipeline runs and diff it against every check found in the Verification section — name each check that exists in the repo but is missing from the pipeline.

  • Status: FAIL
  • Proof: The pipeline covers almost everything found in this section: typecheck and workspace lint and case-hazards (typecheck.yml), Prettier (prettier.yml), studio unit tests with coverage (studio-unit-tests.yml), studio E2E (studio-e2e-test.yml), the ESLint ratchet in both directions (studio-lint-ratchet.yml, studio-lint-ratchet-decrease.yml), docs, www, ui, ui-patterns and pg-meta tests, MDX prose lint (docs-lint-v2.yml), typos (avoid-typos.yml), workflow security (zizmor.yml), self-host smoke (self-host-tests-smoke.yml) and AI evals (ai-tests.yml, braintrust-evals.yml). One configured check is never run: knip — knip.jsonc sits at the repo root and package.json defines "knip": "pnpx knip@~5.50.0", but grep -rl 'knip' .github/workflows/ returns nothing, so dead code and unused dependencies are only ever found when somebody remembers to run it by hand. Also absent from CI, and covered separately at the dependency-vulnerability item: any pnpm audit step.
  • Recommendation: Add a knip job to a path-filtered workflow (or a step in typecheck.yml) so pnpm knip runs on PRs; pin the version in devDependencies instead of pnpx knip@~5.50.0 so CI and laptops agree.
  • Priority: High

• 27. Does anything here get an LLM to read a change and go looking for problems — a review skill, a saved review prompt, or an AI reviewer on the pull requests? Look for a committed review skill or slash command, a CI step calling an AI reviewer, or bot config. Human review rules are a Safety item; this one is specifically about machine review.

  • Status: PASS
  • Proof: Two AI reviewers, both configured in-repo and unusually well tuned. .coderabbit.yaml: inheritance: false, path_filters excluding 12 generated or vendored paths "to preserve rate-limit budget on large codegen diffs", and path_instructions giving per-path review rules — for packages/common/telemetry-constants.ts it enforces the [object]_[verb] snake_case event naming with an explicit list of approved verbs and tells the reviewer to flag useSendEventMutation. .github/copilot-instructions.md sets a review policy with a stated >85% confidence threshold ("The team acts on fewer than 20% of default Copilot suggestions"), a do-not-comment list for anything CI already checks, and a five-item priority order, routing to 8 path-scoped files in .github/instructions/. Committed review skills also exist: .agents/skills/review-the-docs/SKILL.md and .claude/skills/dev-toolbar-review/SKILL.md.
  • Recommendation: —
  • Priority: Low

4. Tooling

• 28. Can an agent reach the outside systems this project depends on, with that access committed to the repo rather than set up per laptop? MCP servers are one form: look for a committed .mcp.json or this ecosystem's equivalent. A CLI is another and often the better one — aws, gh, psql, kubectl, stripe, a vendor's own tool — and it counts when the repo names which tools the work needs and how to authenticate, so an agent is not guessing at a tool it cannot see. Judge the axis, not the mechanism: access that exists only in someone's shell history or laptop config is a FAIL, and so is a committed config that turns out to be gitignored, with the distinction in the proof. Project task scripts have their own item; this one is about reaching past the repo's edge. If this project genuinely talks to nothing outside itself, N/A with that as the reason.

  • Status: PASS
  • Proof: .mcp.json is committed and tracked (confirmed in git ls-files, not gitignored): one server, supabase, "type": "http", "url": "https://mcp.supabase.com/mcp". CLIs are vendored rather than assumed: supabase@^2.76.10 is a root devDependency, so pnpm setup:cli, supabase gen types and supabase start all work from a bare checkout, and DEVELOPERS.md names the remaining prerequisites (Git, Node per .nvmrc, pnpm per packageManager, make, Docker). braintrust drives the eval commands, vercel the deploy scripts. One gap: nothing in the instruction set says how to authenticate the MCP server, so an agent that needs it is left guessing.
  • Recommendation: —
  • Priority: Low

• 29. Is there a skills, commands or reusable-prompt library in the repo? Look for .claude/skills, .agents/skills, .claude/commands, or a prompts directory. Committed and pinned beats committed; note whether anything ties the copies to a source.

  • Status: PASS
  • Proof: 19 skills under .claude/skills/ (copywriting, docs-content, telemetry-standards, dev-toolbar-review, safe-sql-execution, react-hook-form, vercel-composition-patterns, clickhouse-logs-queries, studio-queries, studio-ui-patterns, studio-error-handling, studio-testing, studio-mock-api-tests, studio-e2e-tests, …), plus apps/studio/.claude/skills/explorer/, 3 .cursor/rules/docs/*/RULE.md and 8 .github/instructions/*.instructions.md. Copies are tied to a source rather than duplicated: .agents/skills/ holds the five cross-tool skills (ask-the-docs, pm-the-docs, review-the-docs, vitest, write-the-docs) and both .claude/skills/ and .cursor/skills/ reach them through committed symlinks (ask-the-docs -> ../../.agents/skills/ask-the-docs), so Claude and Cursor cannot drift apart. Several skills carry their own reference material — clickhouse-logs-queries/references/{bigquery-migration,codebase-integration}.md, vercel-composition-patterns/rules/ with 8 rule files and its own AGENTS.md.
  • Recommendation: —
  • Priority: Low

• 30. Do the skills, servers and tools that exist cover the work this team plainly repeats? First identify the repeated work from the README and the commit history. If skills, servers or documented tooling exist but miss it, FAIL naming the gap. If none exist at all and the repo plainly repeats work, FAIL pointing at the outside-systems item and the skills-library item. If the project is too small to repeat anything, N/A.

  • Status: PASS
  • Proof: The repeated work in the last 400 commits is studio feature and refactor work, docs content, generated-type refreshes, www marketing pages and eval cases. Each has cover. Studio: 10 task-scoped skills routed by the table in apps/studio/CLAUDE.md. Docs: docs-content, write-the-docs, review-the-docs, pm-the-docs, ask-the-docs. Copy: copywriting. Type refreshes are automated rather than skilled — docs-mgmt-api-update.yml, docs-js-libs-update.yml, update-js-libs.yml, update-ssr.yml, plus pnpm api:codegen and pnpm generate:types. Chores are automated too: fix-typos.yml, autofix_linters.yml, og_images.yml, docs-sync*.yml, stale.yml. Thinner spot: the self-hosting surface in docker/ (11 commits, 99 changed paths in the window) has a CI smoke test (self-host-tests-smoke.yml) and docker/README.md plus docker/CONFIG.md, but no skill and no place on the layout map — see the layout item.
  • Recommendation: —
  • Priority: Low

• 31. Are there project CLI scripts or task-runner targets for the common jobs? Check the manifest's scripts, the Makefile, the justfile, or this ecosystem's equivalent. The test: does routine work need a raw multi-flag command that someone has to remember?

  • Status: PASS
  • Proof: Root package.json defines 40+ scripts covering every routine job: per-app dev and build, lint, typecheck, format, test:* per package, e2e plus three environment setups (e2e:setup:cli, e2e:setup:selfhosted, e2e:setup:platform), generate:types, api:codegen, knip, clean. apps/studio/package.json adds 25 more, including the framework dispatcher (dev/build/start → node scripts/dispatch.js) that hides the Next-versus-TanStack branch entirely. turbo.jsonc defines the build, lint, clean, dev, test and typecheck graph with caching. The gnarly commands are already wrapped — e2e:setup:cli packs a five-step supabase stop && start && status && generateLocalEnv chain behind one name. Nothing routine needs a remembered multi-flag invocation. The stale root Makefile is the exception, and nothing points at it.
  • Recommendation: —
  • Priority: High

• 32. Are those scripts named somewhere the agent will actually read them? If no scripts exist, FAIL with a one-line proof pointing at the task-scripts item. Otherwise check the agent instruction file, the README, and whether the manifest itself is self-explanatory.

  • Status: PASS
  • Proof: Three places, all of them ones an agent reads. .claude/CLAUDE.md names the 12 daily commands in a fenced block with an inline comment on each ("pnpm dev:studio # run Studio dev server → http://localhost:8082"). apps/studio/CLAUDE.md names the ratchet command at the point it explains the trap. DEVELOPERS.md names pnpm install, pnpm dev and the per-app variants with a port table. And the manifest is self-documenting: script names are dev:studio, test:studio, e2e:docs, api:codegen — the names say what they do.
  • Recommendation: —
  • Priority: High

• 33. Can an agent get this project running — is there a reproducible environment or a documented setup path? Look for a pinned runtime (.nvmrc, .tool-versions, rust-toolchain), a lockfile, a container or nix file, and written setup steps. Try the first step if it is cheap and safe. This one bites hardest the moment work happens in a fresh git worktree or a new clone — the normal way to run agents in parallel. A new worktree has no installed dependencies, no .env, no build cache, so anything that works today only because of untracked state sitting on someone's machine simply does not run there. The test: would a bare checkout plus the written steps get this project up? Name any prerequisite nothing creates — an env file someone hand-made, a seeded database, a manual login — because each one is a wall a worktree hits on its first command.

  • Status: PASS
  • Proof: Runtime is pinned four ways: .nvmrc (22), engines: {node: ">=22.13", pnpm: "11.13"}, packageManager: "pnpm@11.13.1", and preinstall: "npx only-allow pnpm". One workspace-wide pnpm-lock.yaml plus pnpm-workspace.yaml, a catalog: protocol for shared versions, and a patches/ directory. Containers exist for the runtime story: apps/studio/Dockerfile and docker/docker-compose.yml with ten variants. Steps are written in DEVELOPERS.md and I tested them on a bare clone: corepack pnpm install --frozen-lockfile succeeded in 3m22s, 3,042 packages, lockfile accepted with no drift. Prerequisites a fresh worktree still needs, all of them named in the docs: Docker running plus pnpm setup:cli for Studio, which boots a local Supabase stack and writes an untracked keys.json (gitignored, line 158) via node scripts/generateLocalEnv.js; and a hand-copied apps/www/.env.local — the copy step is written and apps/www/.env.local.example is tracked, so nothing here depends on a file only one laptop has. apps/docs routes to its own apps/docs/DEVELOPERS.md, which exists.
  • Recommendation: —
  • Priority: High

• 34. Can an agent see the results of a failed run — do the tools here produce output it can read and act on? Judge from the runs you already did in Verification: does a failure print a path, a line, a name — something actionable — or a wall of noise? If nothing could be run, FAIL saying why.

  • Status: PASS
  • Proof: Judged from real failures in my own runs, not from config. The Vitest failure named the exact file that could not start and the reason: Failed to start forks worker for test files .../src/CommandMenu/api/hooks/pagesHooks.test.tsx / Caused by: Error: [vitest-pool-runner]: Timeout waiting for worker to respond, with a per-phase duration breakdown (import 1245.38s) that made the resource-starvation diagnosis possible and pointed straight at the fix (--pool=threads), which worked. An ESLint invocation error printed No files matching the pattern "state/tabs.ts" were found. Please check for typing mistakes in the pattern. — the offending argument by name. check-case-hazards.mjs prints ✅ No case-sensitivity hazards found (16794 tracked files checked) on success, and on failure (lines 154-163) prints the offending file, the offending import specifier and the remedy: "Point at the directory index explicitly, e.g. '<specifier>/index'". The test runner is configured for readable local failures on purpose: apps/studio/vitest.config.ts sets retry: IS_CI ? 2 : 0 with the comment "Retry flaky tests in CI only; failures locally should surface immediately".
  • Recommendation: —
  • Priority: Low

5. Safety

• 35. Are credentials kept out of the repo — nothing secret committed, ignore rules in place, an example env file for the shape? Three checks: grep tracked files for key-shaped strings, read the ignore rules for env and key patterns, and look for an example env file. Report each of the three separately.

  • Status: PASS
  • Proof: (a) Grep for key-shaped strings across tracked files found no live secret. The hits are all deliberate: two UI placeholders (.../Snowflake/Fields.tsx:142 and LogDrainDestinationSheetForm.tsx:941 render -----BEGIN PRIVATE KEY----- as placeholder text), a fake credential inside an eval prompt (apps/studio/evals/dataset.ts:368), and vendored Deno type documentation. The JWTs in apps/studio/.env decode to "iss": "supabase-demo" — the published self-hosting demo keys — alongside placeholder passwords that say so (POSTGRES_PASSWORD=your-super-secret-and-long-postgres-password, DASHBOARD_PASSWORD=this_password_is_insecure_and_should_be_updated). The keys in apps/docs/.env.development and apps/ui-library/.env are anon / sb_publishable_ keys, which are public by design and guarded by RLS. (b) Ignore rules are in place: .gitignore lines 84-92 cover .env, .env.tmp, .env.test, .env.local, .env.staging, .env.production, and line 158 covers keys.json. Four tracked env files are deliberate exemptions and each declares itself — apps/www/.env opens with "this is a public env file / do not use secrets in this file", apps/docs/.env.development with "IMPORTANT: This file is checked into version control. Only publicly available environment variables should be listed here." The !docker/.env exemption is dormant: only docker/.env.example is tracked. (c) Example files exist: docker/.env.example, apps/www/.env.local.example, plus 20+ under examples/.
  • Recommendation: —
  • Priority: High

• 36. Does anything scan for secrets automatically? Look for gitleaks, trufflehog, detect-secrets or this ecosystem's equivalent, wherever it is wired in — a CI step, a pre-commit hook, or forge-level push protection visible from the repo. CI is where this normally lives, and that is a PASS; a local hook on top is better, because it catches the key before it is pushed rather than after, but its absence is a line in the proof, not a FAIL. Say where the scan runs. "Nothing secret exists today" does not make this N/A — the scan is for the day that changes.

  • Status: FAIL
  • Proof: Nothing scans for secrets anywhere. grep -rilE 'gitleaks|trufflehog|detect-secrets|ggshield' .github/ .husky returns nothing across all 48 workflows. There is no pre-commit stage at all to hang one on — no .husky directory, no prepare script, no lint-staged. zizmor.yml is the nearest neighbour and it audits workflow permissions and dangerous triggers, not secrets. GitHub's own push protection may be enabled at the org level, but nothing in the repo shows it, so an agent working here has no committed evidence that a leaked key would be caught. The exposure is real and specific: four .env files are tracked on purpose, so the habit of committing env files is established, and the line between a public anon key and a service_role key in the same file shape is one careless paste wide.
  • Recommendation: Add a gitleaks job to a pull_request-triggered workflow (gitleaks/gitleaks-action, pinned by SHA like every other action here) with a .gitleaksignore allowing the four intentional env files and the supabase-demo JWTs.
  • Priority: High

• 37. Are dependencies pinned, so a build is reproducible? Look for lockfiles in every package of the repo, exact versions for load-bearing dependencies, a pinned runtime, and an install command that respects the lock (npm ci, not npm install).

  • Status: PASS
  • Proof: One pnpm-lock.yaml covers the whole workspace, as pnpm workspaces intend, and I verified it is honest: corepack pnpm install --frozen-lockfile resolved 3,042 packages with no drift and exited clean. The non-JS example projects carry their own locks (examples/ai/*/poetry.lock, two pubspec.lock, one Cargo.lock), so no package is unlocked. The runtime is pinned by .nvmrc, engines and packageManager, and preinstall: only-allow pnpm stops an npm install from producing a second lockfile. Shared versions go through pnpm's catalog: protocol so they cannot diverge per package, turbo is pinned exact (2.9.14), and patches/ holds pinned patches. Every workflow installs with pnpm install --frozen-lockfile — verified in typecheck.yml, prettier.yml, studio-unit-tests.yml and authorize-vercel-deploys.yml — so CI fails rather than silently updating.
  • Recommendation: —
  • Priority: High

• 38. Is anything watching those dependencies for known vulnerabilities? Look for dependabot or renovate config, an audit step in CI, or this ecosystem's equivalent. Check every lockfile in the repo is covered, not just the root one.

  • Status: FAIL
  • Proof: .github/dependabot.yml exists but watches exactly one ecosystem: package-ecosystem: 'github-actions', directory /, weekly, with a 7-day cooldown. There is no npm/pnpm entry, so pnpm-lock.yaml — the entire JavaScript dependency tree behind seven shipped apps — is unwatched, and so are the five non-JS lockfiles under examples/. No renovate config exists. No audit step exists either: grep -rl 'pnpm audit\|npm audit\|osv-scanner\|snyk' .github/workflows/ returns nothing across all 48 workflows.
  • Recommendation: Add an npm ecosystem block to .github/dependabot.yml for / (pnpm is supported) with a grouped-updates rule to keep the PR volume sane, and add pnpm audit --audit-level=high as a step in typecheck.yml.
  • Priority: Low

• 39. Are the review rules written down — who reads a change, and what they check? Look for CONTRIBUTING, a PR template, or a review checklist in the agent instruction set. On a solo repo the "who" is N/A-shaped but the "what gets checked before it lands" still matters — judge that half.

  • Status: PASS
  • Proof: Both halves are written down. Who: .github/CODEOWNERS assigns real teams per path — packages/ui/ to @supabase/design, packages/shared-data/pricing.ts and plans.ts to @supabase/billing, packages/common/telemetry-constants.ts to @supabase/growth-eng, apps/studio/ to @supabase/Dashboard, apps/docs/ to @supabase/docs, apps/www/ to @supabase/marketing. What: CONTRIBUTING.md sets the process rules (search existing issues, link the issue, open a Discussion before a new feature or "your PR will be closed", use the PR template), .github/pull_request_template.md asks for kind of change, current behaviour, new behaviour and context, and the machine half is specified in .coderabbit.yaml and .github/copilot-instructions.md. One defect: the CONTRIBUTING.md pre-flight says "Run npm run build locally", which this repo blocks via preinstall: npx only-allow pnpm — see the README item.
  • Recommendation: —
  • Priority: Low

• 40. Are the operations that need a human named somewhere an agent will read them? Look for a "never without asking" list in the agent instruction file or the README. Docs that hand out production commands with no fence around them count against, and the proof should quote one.

  • Status: FAIL
  • Proof: The instruction set has "never do this" rules but no "never without asking" list. What it covers is editing: .claude/CLAUDE.md's six generated paths, apps/studio/CLAUDE.md's "Never delete a page file". Nothing in .claude/CLAUDE.md, apps/studio/CLAUDE.md, README.md or DEVELOPERS.md names a single operational action — deploy, migration, publish, anything touching production or money — as needing a human first. Meanwhile the manifest hands out exactly those commands with no fence: apps/studio/package.json:29 is "deploy:staging": "VERCEL_ORG_ID=team_E6KJ1W561hMTjon1QSwOh0WO VERCEL_PROJECT_ID=Qmcmhb… vercel --prod -A .vercel/staging.json", and root package.json:33 opens with supabase stop --all --no-backup, which destroys every local Supabase container and its volumes with no confirmation and no backup. .claude/settings.json shows the team knows how to fence things — it has a permissions.deny block — but it denies only Edit on generated files and constrains no Bash command at all.
  • Recommendation: Add a "Never without asking" section to .claude/CLAUDE.md naming deploy:staging, e2e:setup:cli (supabase stop --all --no-backup), docker/reset.sh and scripts/authorizeVercelDeploys.ts, and back it with permissions.deny entries for Bash(*vercel --prod*) and Bash(*supabase stop*) in .claude/settings.json.
  • Priority: High

• 41. Does every action that spends money, destroys data or changes production have a human in the way? Start from the damage, not from the tooling: list what in reach of this repo could charge a card, drop or overwrite data, or alter what users are running. Then trace the shortest route an agent could take to each one — a push that auto-deploys, a script carrying live credentials, a migration that runs on merge, an infrastructure apply with no plan-and-approve step. PASS when every route meets a human first, whether that is a review, a manual trigger or a protected environment. FAIL when even one route runs start to finish unattended, and quote that route in the proof so the fix is obvious.

  • Status: FAIL
  • Proof: Most routes are gated properly. Production web deploys go through Vercel on merge to master, and merge needs CODEOWNERS review plus green pre-merge CI; PR preview deploys are explicitly authorized by validate-pr.yml → authorize-vercel-deploys.yml, which runs the master-branch copy of the script so a PR cannot alter its own gate. The Docker image publish is on: schedule (weekly, Monday 04:00 UTC) plus workflow_dispatch — no push trigger. No workflow applies database migrations: grep -rn 'db push\|migration up\|supabase db' .github/workflows/ returns nothing. But one route runs start to finish unattended: pnpm --filter studio deploy:staging executes vercel --prod -A .vercel/staging.json against a hardcoded VERCEL_ORG_ID/VERCEL_PROJECT_ID, so any agent in a shell with a live Vercel session deploys to that environment's production alias in a single command, with no review, no confirmation and no protected-environment check. A second, smaller one: pnpm e2e:setup:cli leads with supabase stop --all --no-backup, destroying local containers and volumes unattended.
  • Recommendation: Move the staging deploy behind a workflow_dispatch GitHub Action using a protected environment instead of a laptop-runnable script, and until then rename it deploy:staging:dangerous and add a Bash(*vercel --prod*) deny rule in .claude/settings.json.
  • Priority: High

• 42. If a prompt injection landed tonight, how far would it reach — are the credentials an agent can get to here scoped to the job, with nothing production-grade in reach? Inventory what an agent in this repo can reach: env files, cloud CLI profiles, tokens named in docs or config, deploy commands that work from a laptop. Scoped-or-absent passes; production-grade reach fails with the item named.

  • Status: FAIL
  • Proof: Nothing production-grade is committed — the tracked env files hold only demo JWTs, anon/publishable keys and localhost URLs, and real .env* files plus keys.json are gitignored. But three routes reach past the laptop, and the repo names all three for the agent. First and worst: apps/docs/package.json:28 is "dev:secrets:pull": "AWS_PROFILE=supa-dev node ../../scripts/getSecrets.js -n local/docs", and scripts/getSecrets.js calls GetSecretValueCommand against AWS Secrets Manager in ap-southeast-2 and writes the whole secret bundle to .env.local — so one documented command turns the machine's ambient AWS profile into real credentials on disk, and .agents/skills/ask-the-docs/reference/build-pipeline.md:147 tells an agent this is how to get them. Second: deploy:staging names the exact Vercel org and project, so an authenticated shell is one command from a --prod deploy (see the human-in-the-way item). Third: .mcp.json binds to https://mcp.supabase.com/mcp with no token committed, meaning the reach is whatever the developer's own Supabase session grants — for a Supabase engineer, their real projects. Nothing constrains any of it: .claude/settings.json denies six Edit paths and no Bash commands.
  • Recommendation: Add permissions.deny entries in .claude/settings.json for Bash(*getSecrets*), Bash(*vercel --prod*) and Bash(*AWS_PROFILE*), and note in .claude/CLAUDE.md that dev:secrets:pull fetches real AWS secrets and is a human-only step.
  • Priority: High

• 43. Can a change reach production a slice at a time — a feature flag that defaults to off, a canary, a staged rollout — rather than everyone at once? Look for a flag system and check the default, or canary and staged-rollout config in the deploy pipeline. Flags that need a rebuild to flip are worth naming in the proof — they gate exposure but they are not a kill switch.

  • Status: PASS
  • Proof: packages/common/feature-flags.tsx wires ConfigCat and PostHog behind a FeatureFlagContext, fetched at runtime from /telemetry/feature-flags, so flags flip without a rebuild or a redeploy — a real kill switch. The default is off, verified in code at lines 275-290: useFlag returns false when the store has not loaded ("Flag store is empty means config cat is not loaded yet, return false") and false again for an unknown key, after logging the error. Two further gating systems layer on top, and apps/studio/CLAUDE.md is careful to distinguish them: useIsFeatureEnabled for product features, useFlag for feature flags, plus IS_PLATFORM for the platform-versus-self-hosted split. Per-PR Vercel preview deploys give a pre-production slice as well.
  • Recommendation: —
  • Priority: High

• 44. Once a change is live, can anyone see what it is doing — logs, metrics, traces, alerts that fire on their own, and can an agent read them too? Look for logging setup, an error tracker, analytics, alerting config — and then ask the second half: could an agent reach any of it (a CLI, an MCP server, an API named in the docs), or does observability stop at a dashboard behind a login?

  • Status: PASS
  • Proof: First half: Sentry is wired into apps/docs (app/error.tsx, app/global-error.tsx, and the GraphQL, FTS and embeddings API routes), PostHog handles product analytics through packages/common/posthog-client.ts with a governed event taxonomy in packages/common/telemetry-constants.ts, Coveralls tracks coverage per PR, and alerting exists as studio-master-alert.yml. Second half, which is where most repos stop and this one does not: .claude/skills/clickhouse-logs-queries/SKILL.md is a committed skill for querying the logs backend, shipping references/bigquery-migration.md and references/codebase-integration.md, and apps/studio/CLAUDE.md routes "Logs Explorer SQL, data/logs" work to it. Together with the supabase MCP server in .mcp.json, an agent has a documented path into the telemetry rather than a dashboard behind a login.
  • Recommendation: —
  • Priority: Low

• 45. Is there a way back — can a bad change be undone without a rebuild and a redeploy, including the ones that touched a database or a queue? Look for a documented rollback path, a revert-and-redeploy story, down-migrations, or a flag that can turn the change off at runtime. Deployment docs that only say how to go forward are worth quoting.

  • Status: PASS
  • Proof: The runtime kill switch is the real answer: ConfigCat flags fetched at runtime and defaulting to false turn a change off without a rebuild or a redeploy (see the staged-rollout item). A second documented fallback covers the biggest in-flight risk — apps/studio/TANSTACK_MIGRATION.md keeps the whole Next runtime shippable "as a fallback so we can bisect regressions and ship either runtime if needed", flipped by STUDIO_FRAMEWORK, though that one does need a redeploy to change. Gaps worth naming: the 41 files in supabase/migrations/ are forward-only, with no down counterpart for any of them, so a bad migration to the docs-search database has no scripted reverse; and no document anywhere states a rollback procedure for a bad deploy — DEVELOPERS.md and CONTRIBUTING.md only say how to go forward, and the Vercel instant-rollback that the team presumably relies on is written down nowhere in the repo.
  • Recommendation: —
  • Priority: Low

• 46. Does a past failure leave a check behind — is there a test or a rule in here that exists because something broke once? Look for tests, lint rules or comments that name the incident they guard against — "regression", "this broke when", a linked issue. On a very young repo with no failures yet, N/A with the age as the reason.

  • Status: PASS
  • Proof: The clearest example is a whole CI step with its scar tissue documented in place — typecheck.yml: "Needs no dependencies, so it runs before install and fails fast. Catches a class of bug that only breaks on case-insensitive filesystems (macOS and Windows dev machines) and is therefore invisible to typecheck/lint on CI", guarding node scripts/check-case-hazards.mjs. Others: apps/studio/scripts/ratchet-eslint-rules.ts plus ratchet-rules.json exist because warning counts crept up, and studio-lint-ratchet.yml now freezes them; apps/studio/vitest.config.ts carries retry: IS_CI ? 2 : 0 with a comment explaining the flake trade-off, plus a named excluded test (tests/features/logs/logs-query.test.tsx); avoid-typos.yml with .misspell-fixer.ignore; zizmor.yml with a per-file dangerous-triggers ignore list naming the five workflows that need it. The Safari clipboard rule in apps/studio/CLAUDE.md is the tribal version of the same thing — "never await anything before calling it (Safari requires the write inside the user gesture; lint-enforced)" — a bug that became a lint rule. 12 test files match regression|this broke|guards against.
  • Recommendation: —
  • Priority: Low

Learn more: https://ainativesoftware.engineering/