dose.wiki: codebase overview

01 The big picture in numbers

267

Published substance articles

233

Subjective effects

166

Trip reports

~1,400

TS/TSX files in src/

~700

Pipeline scripts in scripts/

The scripts/ directory holds around 700 script files, compared with around 1,400 TS/TSX files in src/, because pipelines produce most content and media. Scripts scrape source wikis, generate prose through OpenRouter, tie claims to real literature, and move replication media through Cloudflare R2. Editors review trip-report submissions before publication. Article text and citations have separate, ongoing review workflows; each article shows its review status. If you read only src/, you see the site but not the tooling that fills it.

02 System overview

Every system in the repository reads and writes Convex (a hosted realtime database with serverless functions). Content enters through scrapers, AI pipelines, and reader submissions. Editors change it in /dev. The public site reads it, and replication media is served from Cloudflare R2. The colors below keep the same meaning for the rest of this page.

03 Tech stack: what each tool is for

04 Repository map: where things live

Most of the code lives in four directories: src/ (the app), convex/ (the backend), scripts/ (the pipelines), and lib/ (server-side helpers shared between them). Everything else is documentation, assets, vendored dependencies, or configuration.

dose.wiki/
├── src/                      · the Next.js application (~1,400 files)
│   ├── app/                  · App Router: public pages, /dev, ~70 /api route handlers
│   ├── features/             · 15 domain modules; dev has ~400 TS/TSX files, including tests
│   ├── components/           · shared UI: layout, pages, ui primitives
│   ├── schema/               · Zod schemas validating article shapes
│   ├── data/                 · copy-block seeds, JSON artifacts, generated maps
│   └── theme/                · design tokens
├── convex/                   · the backend: schema.ts + one file per table
├── scripts/                  · pipelines (~700 script files; see section 08)
│   ├── parsers/  batch/  citations/  prepopulate/
│   ├── replications/  contributors/  chemistry/  legality/
│   ├── convex/  migrate/  seed/  build/  deploy/  reports/
│   └── lib/                  · shared CLI plumbing, OpenRouter client
├── lib/                      · server-side helpers shared by app + scripts
│   ├── convex/               · publicData.* readers: how pages read Convex
│   ├── next/                 · route loaders, copy blocks, host policies
│   ├── auth/  http/  og/  citations/  runtime/
├── vendor/openchemlib/       · vendored molecule-rendering fork
├── auth.ts  middleware.ts   · sign-in config + /dev gatekeeper
├── public/                   · molecule images, flags, favicons
├── docs/  notes-and-plans/   · architecture docs, ADRs, agent runbooks
├── .agents/skills/           · citation-pi + legality-pi workflows
└── README.md  ARCHITECTURE.md  CLAUDE.md  CONTEXT.md

The repository keeps two local skills: .agents/skills/citation-pi/ (the citation entailment audit) and .agents/skills/legality-pi/ (legality-source research). Other automated workflows are documented beside their owning scripts under scripts/ and docs/agents/, so executable behavior and operating guidance stay together.

05 Frontend: routes and feature modules

The code splits two ways. src/app/ holds thin route folders (one per URL, mostly data-fetching and metadata) plus ~70 API route handlers. src/features/ holds the rendering logic, grouped into 15 domain modules. A route file fetches from Convex and passes the data to a feature module.

One codebase builds two publications, dose.wiki and the Effect Index sister site, selected at build time by NEXT_PUBLIC_SITE_FLAVOR. The route tree therefore also holds Effect Index surfaces; which ones render depends on the flavor (src/config/siteFlavor.ts).

06 The Convex data layer: what is in the database

Most tables have a matching convex/<table>.ts module that exports their queries and mutations; convex/schema.ts declares all ~45 shapes. There are three groups: content, editorial, and workflow. Readers see the content. The editorial records control who can change it, hold what readers have submitted, and audit every change. The workflow tables feed the pipelines. Small sibling tables share a row below.

07 Four kinds of traffic, one database

Licensing is per dataset. The base data is CC0, but each export is "mixed": the substance index keeps TripSit's terms on its interactions field, the effect index references third-party replication media, and legacy trip reports keep their authors' rights (see the license page).

08 Scripts and pipelines: the third system

Alongside the site and the editor, scripts/ is a set of Node and Bun command-line programs, one subdirectory per workflow family, each exposed as an npm run command. Parsers turn scraped wiki text into structured data (scripts/parsers/). Prepopulation loads the high-confidence fields into Convex (scripts/prepopulate/), and batch generation drafts the prose sections through OpenRouter (scripts/batch/). Editors review the result in /dev. The citation and legality workflows research and check the sources supporting individual claims (scripts/citations/, scripts/legality/); a claim can remain uncited.

Scripts write to Convex as an administrator, behind gates. Where an editor save goes through an authenticated route handler, a script holds an admin token scoped to the one kind of write it performs (CONVEX_ADMIN_TOKEN_<INTENT>, resolved by scripts/lib/data-ops-run-context.mjs). Production writes share one gate, in that run context and the scripts/lib/production-write-command.mjs wrapper. A run is a dry run by default. A real write is permitted only with --write, an explicit TARGET_CONVEX_URL, a --confirm-write=<operation> phrase, and an --expected-deployment that must match the target.

The same module offers a pre-write backup and an audit log under scripts/data/audit-logs/, which the migration-style scripts use. npm run convex:doctor checks the environment before any of this (see AGENTS.md).

These write conventions also apply to the following workflow families:

• Citation entailment audit: scripts/citations/verdict-*, run by the citation-pi skill.
• Legality-source research: scripts/legality/, run by legality-pi.
• Replication intake and R2 media migration: scripts/replications/, the largest family (section 09).
• Contributor avatar tooling: scripts/contributors/.
• Molecule and chemistry audits: scripts/chemistry/.
• Convex migrations and exports: scripts/migrate/ and scripts/convex/.
• Build-time generators for social cards and theme CSS: scripts/build/.

The registry in scripts/lib/workflow-command-surface.mjs (with citation-command-surface.mjs beside it) is the index of runnable commands.

The content pipeline has its own page. This page stops at the architecture. How dose.wiki substance articles are made covers what the scrapers collect, how verbatim excerpts are pulled from them, the prompts each section is written from, how citations are proposed and then audited, and where people review. It publishes the extraction and section-generation prompt files unedited, but not the citation and legality workflow prompts. It also explains why the published section prompts may differ from the live versions used to generate an article.

09 The replications media platform

Replications (artist-made video, image, and audio recreations of subjective effects) are the one content type whose bytes do not live in Convex. Convex stores metadata; Cloudflare R2 stores media. Every media object sits under a content-addressed R2 key, a key derived from the file contents. convex/lib/replicationUrls.ts resolves public URLs from the REPLICATION_MEDIA_BASE_URL deployment variable; unset it and the site falls back to Convex Storage, which is the rollback switch.

Media arrives through scripts/replications/. Reviewed source media (for example the Reddit archive, following docs/agents/reddit-replication-intake.md) is uploaded to R2 under content-addressed keys and verified, then a production import writes the gallery metadata, taxonomy, and attribution rows to Convex. Provenance and artist identity are stored in the side tables from section 06, keeping the documents read when displaying the gallery small. Editors curate the result in the /dev Replication Studio: featured carousels, per-substance showcases, playlists, and duplicate reconciliation.

substanceGalleries pins and suppresses what each substance's Replication Showcase shows; replicationPlaylists are reusable ordered sets applied into a gallery draft; the Effect Index homepage carousel is an ordered list in siteConfig. All three are data, so curation changes ship without a deploy. The public gallery reads through lib/convex/publicData.replications.ts and two cacheable JSON endpoints (/api/replications/gallery and /showcase).

Rights metadata defaults to unknown, and every replication credits its artist; source posts and posters are retained losslessly in the attribution tables. The gallery's fair-use position is spelled out in its own copy (src/features/replications/replicationsFairUseCopy.ts) and on the license page.

10 Three flows, step by step

Flow A: a reader opens dose.wiki/lsd

Flow B: an editor saves an article in /dev

Flow C: a reader submits a trip report

11 Recurring vocabulary

Substance article

The central data object: one large structured document per drug, with ~15 sections (dosage, duration, pharmacology, legality, and more). Defined by Zod schemas in src/schema/, stored in substanceIndex.

VCode

A custom markup format used for subjective-effect article bodies, stored alongside a parsed AST and rendered by src/features/effects/.

Convex deployment

One production deployment (glad-minnow-656). All environments (local dev, previews, production) read and write it directly; the old dev deployment is retired. Treat every Convex write as a production write (see AGENTS.md).

Site flavor

One codebase, two publications: NEXT_PUBLIC_SITE_FLAVOR selects dose.wiki or the Effect Index at build time, and src/config/siteFlavor.ts owns every difference in nav, wordmark, and flavor-gated routes. Content and Convex data are shared.

Copy block

A keyed piece of editable site copy in the copyBlocks table, read through getCopy() with the checked-in JSON seed as fallback. Nearly all public prose, including this page, is copy blocks, edited in the /dev Copy Studio.

12 Where to start reading

A reasonable reading order for a first session:

1. README.md and ARCHITECTURE.md: the orientation docs from the maintainers. They cover routes, npm scripts, and system layout.
2. convex/schema.ts: the data model that everything else is built around. Skim the substanceIndex table definition first.
3. src/app/[slug]/page.tsx → lib/next/routeLoaders.substances.tsx → lib/convex/publicData.* → src/features/article/: Flow A from section 10. This chain covers the whole public read path.
4. middleware.ts + auth.ts: short files that show how /dev is protected and how roles work.
5. src/lib/http/protectedRouteOperation.ts + lib/http/rateLimitPolicy.ts: the shared handler for protected API operations (session role, named rate-limit bucket, payload cap). Public submission routes have separate abuse controls and do not require sign-in.
6. src/features/dev/save-orchestrator/ and src/app/api/save-to-convex/: Flow B. These files show how edits reach the database with an audit trail.
7. scripts/lib/workflow-command-surface.mjs: the registry of pipeline commands. From here you can find whichever pipeline (batch, citations, replications, migrate, sync) you need.