dose.wiki — codebase overview
Next.js 16 + Convex + AI editorial pipelines — written for someone new to the repo
dose.wiki is a harm-reduction encyclopedia: a public website of substance articles, subjective-effect documentation, and trip reports. The repository contains three systems: the public Next.js site that renders content, a login-protected /dev editing surface where that content is written and cited, and a set of command-line pipelines that scrape, generate, and migrate it. All three read and write the same Convex database, which is the source of truth.
Contents
01 The big picture in numbers
- 577
- Substance articles in Convex
- 233
- Subjective effects
- 163
- Trip reports
- ~1.9k
- Prerendered pages
- 806
- TS/TSX files in
src/ - 352
- Pipeline scripts in
scripts/
Roughly half of this repository is not the website. The scripts/ directory is nearly as large as the app itself, because most content is produced by pipelines: scraped from source wikis, drafted by Claude through OpenRouter, cited against real literature, and reviewed by a person before publication. Reading src/ alone covers the site but not the tooling that fills it.
02 System overview
Everything flows through Convex (a hosted realtime database with serverless functions). Content enters via scrapers and AI pipelines, gets edited in /dev, and is read on the public site. The colors below are used the same way for the rest of this page.
convex/*.ts) that read and write it. The Next.js app, the editor, and the scripts all talk to the same Convex deployment: the public site reads through lib/convex/publicData.ts, the editor writes through the /api/save-to-convex route, and scripts use an admin key.03 Tech stack — what each tool is for
App Router with server components by default: pages fetch data on the server and ship mostly-static HTML. ~1,900 pages are prerendered at build time. Deployed on Vercel.
Hosted document database with TypeScript server functions (queries, mutations). The convex/ directory is the backend. Separate dev and production deployments, synced by script with confirmation gates.
Configured in auth.ts; middleware.ts walls off /dev and, pre-launch, gates the production hosts (dose.wiki, www.dose.wiki) behind an under-construction homepage. Editor roles (admin / editor / viewer) live in the Convex memberships table.
One API gateway to many models. Claude Opus 4.5 writes article sections; Gemini Flash handles cheaper jobs like quote extraction. Used by both batch scripts and the in-editor section generator.
Utility-first styling with a token-based design system in src/theme/. Radix supplies accessible primitives; Framer Motion handles animation; React Hook Form drives the editor's forms.
Bun is the package manager. Zod schemas (in src/schema/ and schemas/) validate every article shape before it is stored or rendered, so malformed or AI-mangled content is rejected early. 212 Vitest test files.
04 Repository map — where things live
Most of the code lives in four directories: src/ (the app), convex/ (the backend), scripts/ (the pipelines), and lib/ (server-side helpers shared between them). Everything else is documentation, assets, or configuration.
dose.wiki/ ├── src/ — the Next.js application (806 files) │ ├── app/ — App Router: one folder per URL route │ ├── features/ — article, effects, reports, dev (323 files) │ ├── components/ — shared UI: layout, pages, ui primitives │ ├── schema/ — Zod schemas validating article shapes │ ├── data/ — local JSON artifacts + generated maps │ └── theme/ — design tokens ├── convex/ — the backend: schema.ts + one file per table ├── scripts/ — data pipelines (352 files; see section 08) │ ├── parsers/ batch/ citations/ prepopulate/ │ ├── convex/ migrate/ seed/ build/ analyze/ │ └── lib/ — shared CLI plumbing, OpenRouter client ├── lib/ — server-side helpers shared by app + scripts │ ├── convex/ — publicData.ts: how pages read Convex │ ├── auth/ http/ next/ citations/ runtime/ ├── auth.ts middleware.ts — sign-in config + /dev gatekeeper ├── public/ — molecule images, flags, favicons, SubstanceIndex.json ├── docs/ notes-and-plans/ — architecture docs, ADRs, plans ├── .claude/skills/ — ~25 runbooks for the AI content workflows └── README.md ARCHITECTURE.md CLAUDE.md CONTEXT.md
One thing you won't see in most repos: .claude/skills/ holds ~25 markdown runbooks for the AI content tooling — how to run batch section generation, the rules the citation workflow must follow, how Convex data changes should be handled. Because a lot of text processing here is automated, these procedures are written down so every run follows the same rules.
05 Frontend — routes and feature modules
The main split: src/app/ holds thin route folders (one per URL, mostly data-fetching and metadata), while src/features/ holds the rendering logic, grouped by domain. A route file fetches from Convex and hands off to a feature module.
/[slug]— a substance article (the core page)/substances— browse all substances/effects,/effect/[slug]— subjective effects/reports— trip reports/psychoactive · /chemical · /mechanism · /category— classification indexes/interactions,/search,/about,/contributors
/dev— editor home (auth required viamiddleware.ts)/dev/articles/[slug]— edit a substance/dev/change-log · tag-editor · index-layout · about · citation-review · profile/api/save-to-convex— the editor's main write endpoint/sign-in— Auth.js sign-in entry
article/— renders substance articles: dosage tables, duration charts, interaction grids, citation markerseffects/— effect pages, including a custom "VCode" markup renderer for effect descriptionsreports/— trip report pages
forms/— one React Hook Form section per article field grouptools/— substance editor + AI section generator, layout editor, citation review, about editorsave-orchestrator/— coordinates multi-endpoint savescontext/ · notices/ · profile/ · tags/ · prompts/— editor state, toasts, profile & tag editing
06 The Convex data layer — what's in the database
Each table has a matching convex/<table>.ts file exporting its queries and mutations, with the shapes declared in convex/schema.ts. Three groups: the content readers see, the editorial records that govern who can change it and how, and the workflow tables feeding the AI pipelines.
| Table | Group | What it holds | Docs |
|---|---|---|---|
substanceIndex | content | The main table: one document per substance with every section — summary, dosage, duration, pharmacology, harm potential, legality, citations, references. | 577 |
subjectiveEffects | content | Effect articles (e.g. "geometry", "euphoria") with VCode markup bodies, galleries, and audio replications. | 233 |
tripReports | content | First-person experience reports linked to substances and doses. | 163 |
tripReportSubmissions | editorial | The private submission queue from the report form. Editors accept, reject, or promote a submission into tripReports before it goes public. | — |
replications / effectIndexArticles | content | Effect gallery media (video/image recreations of effects) and methodology articles. | — |
categoryLayout | content | What the public /substances page actually reads: the hand-curated psychoactive index layout. Kept in sync with indexLayouts writes rather than reading that table directly. | — |
indexLayouts / siteConfig | content | Hand-curated classification trees (psychoactive / chemical / mechanism) and about-page configuration. | — |
moleculeOverrides | content | Hand-authored skeletal-diagram fixes from the /dev Molecules editor. When present, the public article swaps in this SVG instead of the generated one. | — |
memberships | editorial | Who may edit: email → role (admin / editor / viewer), synced from Auth.js sign-ins. | — |
changelog | editorial | Audit trail: every editor save records who, when, and a markdown diff. Browsable at /dev/change-log. | — |
contributorProfiles | editorial | Public bios, aliases, links, and avatars for contributors. | — |
articleSources | workflow | Raw scraped source text per substance, from the ten sites in the source list (section 08) — the input the AI generators quote from. | — |
prompts / quotes | workflow | The editable AI prompts used for section generation, and quotes extracted from sources. | — |
citationEvidence | workflow | The citation audit trail: each claim in an article mapped to reference IDs, supporting quotes, and a review status. | — |
07 Two surfaces, one database
Who: anyone, no login. Crawlable, SEO-tuned, mostly static. (Pre-launch: the production hosts serve only the under-construction homepage at /; previews and local dev serve the full site.)
How it gets data: server components call lib/convex/publicData.ts, which queries Convex (read-only URL), validates the result against the Zod schemas, and renders via src/features/article|effects|reports.
Key property: no client-side database access; pages can be prerendered at build time and revalidated when editors save.
Who: authenticated editors only. middleware.ts redirects everyone else to /sign-in; the Convex memberships role decides what they may touch.
How it writes: React Hook Form state → the save-orchestrator → POST /api/save-to-convex (auth + rate-limit checked) → Convex mutations → automatic changelog entry → public pages revalidated.
Source-backed generation runs only through authenticated local batch workflows. The protected editor supports manual review and editing without exposing scraped source text in the browser.
There is a third consumer: anyone who wants the raw data. The About page (/about) offers the entire substance database as one file, SubstanceIndex.json (~5 MB, served from public/), with a one-click download button and a live preview of a sample record (src/components/pages/AboutArchivePreview.tsx). The intent is to make the dataset easy to reuse in other projects and research: Convex stays the live source for pages, the JSON is a static snapshot of it, and the content is dedicated to the public domain under CC0, so it can be reused freely with no attribution required (see the license page).
08 Content pipelines — how an article gets built
Articles are built in stages. Raw wiki text is parsed into structured data, loaded into Convex, extended with AI-generated prose, reviewed in /dev, then run through a citation workflow that ties each claim to a reference. Each stage is a separate scripts/ subdirectory with its own npm run commands.
The scraped sources, in full. The parser registry (scripts/parsers/source-identity.ts) covers ten sites: PsychonautWiki, TripSit (three feeds: factsheets, wiki, and the combination guide), Erowid, Wikipedia, DrugBank, Isomer Design, SaferParty, The Drug Classroom, Disregard Everything I Say, and the Drug Users Bible. A few further sources (Bluelight, D. M. Turner, Nervewing, ProtestKit) are stored in the corpus but have no parser. Per repo policy, Wikipedia and PsychonautWiki text is used to discover references but is never itself cited as claim support on the public site.
The prompting system, zoomed in (quotes → prompts → sections)
The "AI section drafting" box above is itself a two-stage process. The full scraped articles are too long and too mixed to hand to a model directly, so a cheap model first extracts verbatim excerpts by topic — copy-pasted text only, no paraphrasing — and a stronger model then writes one section from those excerpts against a section-specific prompt. The generation prompts forbid claims that are not present in the excerpts, which keeps the output checkable against its sources.
one substance, from
articleSourcesprompt pulls verbatim,
topic-relevant excerpts
topic in the Convex
quotes tablequotes → Claude Opus 4.5
→ YAML section output
and saves in
/devExtraction prompts (scripts/prompts/, scripts/batch/extract-quotes/) pull excerpts into eight topic buckets, defined in lib/quoteSections.mjs:
- Intro text · Harm potential · Pharmacology
- History & culture · Dosage & duration
- Tolerance · Legality · Subjective effects
The rules are strict: verbatim text only, complete paragraphs, keep formatting, and when in doubt include rather than drop.
Each article section has its own markdown prompt in src/data/config/sectionPrompts/— base, summary, dosage/duration, pharmacology, subjective effects, interactions, harm potential, tolerance, legality, history & culture, identification. They range from 2.5 KB (base) to 32 KB (pharmacology).
Each one fixes the style, an evidence-only constraint, a YAML output shape, worked examples, and a validation checklist. The repo files are seeds; the live copies sit in the Convex prompts table and are editable in the /dev prompt workspace (registry: src/data/config/promptRegistry.ts).
Read a stage-1 prompt verbatim — legality quote extraction (2.9 KB)
Unedited contents of scripts/prompts/legality-extraction.md. This is the instruction set the extraction model receives along with the full source text.
# Legality Extraction Prompt
You are extracting legality-related content from source articles about a psychoactive substance.
## Task
Extract ALL verbatim text related to legal status, scheduling, and regulatory information from the provided source content.
## What to Extract
### International Status
- UN Convention classifications (1961, 1971, 1988)
- International scheduling (Schedule I, II, III, IV)
- WHO recommendations
- International treaties and agreements
- INCB (International Narcotics Control Board) status
### Country-Specific Legal Status
For each country mentioned, extract:
- Legal/illegal status
- Schedule/Class classification (e.g., Schedule I, Class A, Tabelle I, Anlage I)
- Controlled substance designation
- Prescription status
- Possession limits and thresholds
- Penalties for possession, sale, manufacture
- Decriminalization status
- Medical/research exemptions
- Recent legal changes
- Regional variations (states, territories, provinces)
### Historical Legal Context
- When the substance was first scheduled
- Legal history timeline
- Changes in legal status over time
- Reasons for scheduling
- Analog act coverage (e.g., Federal Analogue Act)
### Legal Terminology Reference
**US Scheduling:**
- Schedule I - High abuse potential, no accepted medical use
- Schedule II - High abuse potential, accepted medical use
- Schedule III - Moderate abuse potential
- Schedule IV - Low abuse potential
- Schedule V - Lowest abuse potential
**UK Classification:**
- Class A - Most harmful (heroin, cocaine, LSD, MDMA)
- Class B - Less harmful (cannabis, amphetamines)
- Class C - Least harmful of controlled (benzos, GHB)
**German Classification:**
- Anlage I - Not marketable, no medical use
- Anlage II - Marketable but not prescribable
- Anlage III - Prescribable controlled substances
**UN Conventions:**
- 1961 Single Convention - Narcotics (opioids, cannabis, coca)
- 1971 Convention on Psychotropic Substances - Psychedelics, stimulants, sedatives
- 1988 Convention - Precursors and trafficking
## Extraction Rules
1. Extract COMPLETE paragraphs and sections - do not truncate mid-sentence
2. Preserve all markdown formatting (headers, lists, bold, etc.)
3. Include surrounding context when legal information is embedded in other text
4. When in doubt, INCLUDE the content - it's better to have too much than miss critical legal info
5. Do NOT paraphrase - extract verbatim text only
6. Extract from ALL sources that contain legal content
## Output Format
For each source that has legal content, output:
```
## Source: {Source Name}
{verbatim extracted text}
---
```
If a source has NO legal content, output:
```
## Source: {Source Name}
*No legality content found.*
---
```
## Important
Legal information is CRITICAL for user safety. Be thorough and extract everything even tangentially related to legal status, penalties, or scheduling.Read a stage-4 prompt verbatim — summary section generation (4.8 KB)
Unedited contents of src/data/config/sectionPrompts/summary.md. The model receives this plus the extracted quotes for the substance; the larger prompts (pharmacology, harm potential) follow the same structure at more depth.
# Summary Section Generation You are a harm-reduction database assistant generating ONLY the `summary` section of a dose.wiki substance article. --- ## Context dose.wiki is a new harm reduction database created by Josie Kins, a psychedelic researcher and the founder of PsychonautWiki, the Subjective Effect Index, Effect Index, and the blog "Disregard Everything I Say." She is known for her work on psychedelic harm reduction documentation, subjective effect taxonomy, and visual replication art. --- ## Style Write in the style of Josie Kins' psychedelic harm reduction documentation: - Phenomenologically precise yet accessible - Clinical but not cold - Neutral documentation (no advocacy) - Technical accuracy with readable prose - Information-dense but well-structured --- ## Critical Constraints - SYNTHESIZE information from the provided excerpts into original prose - Do not copy sentences or lengthy phrases verbatim from any source - Standard technical terminology (chemical names, drug class names) can remain as-is - Restructure and reframe information in your own voice - Include ONLY information present in the provided excerpts - Do not reference or comment on sources; present information directly without meta-commentary --- ## Core Principles **Evidence Only:** Every claim must be explicitly stated or clearly implied in the source quotes. If information cannot be verified from the provided sources, do not include it. **Concise Overview:** The summary should be 50-80 words that introduce the substance to someone unfamiliar with it. **Harm Reduction Focus:** Flag notable safety concerns, common adulterants, or caustic routes of administration when mentioned in sources. **No Redundancy:** Do not repeat detailed information that belongs in other sections (dosage, duration, pharmacology, etc.). The summary provides context, not exhaustive detail. --- ## Output Format Return ONLY valid YAML for this exact structure (no markdown code fences, no commentary): ```yaml summary: "" ``` --- ## Field Definition | Field | Type | Description | Example | |-------|------|-------------|---------| | `summary` | `string` | 50-80 word overview paragraph | See examples below | --- ## What to Include The summary should briefly cover: 1. **What it is**: Chemical class and/or common name 2. **Psychoactive class**: Primary effects category (psychedelic, stimulant, etc.) 3. **Brief history/origin**: When first synthesized, by whom, or traditional use 4. **Notable characteristics**: What distinguishes it from similar substances 5. **Key safety notes**: Common adulterants, caustic routes, or other prominent warnings (if applicable) --- ## What NOT to Include - Specific dosage numbers (belongs in `dosage` section) - Duration timelines (belongs in `duration` section) - Detailed mechanism of action (belongs in `pharmacology` section) - Legal status details (belongs in `legality` section) - Exhaustive effect lists (belongs in `subjective_effects` section) --- ## Example Outputs ### Example 1: Classic Psychedelic ```yaml summary: "LSD (lysergic acid diethylamide) is a semisynthetic psychedelic of the lysergamide class. First synthesized by Albert Hofmann in 1938, its psychoactive properties were discovered in 1943. LSD is known for producing profound alterations in perception, thought, and mood, with effects lasting 8-12 hours. It remains one of the most potent psychoactive substances known, active in the microgram range." ``` ### Example 2: Research Chemical with Safety Note ```yaml summary: "25I-NBOMe is a synthetic psychedelic of the substituted phenethylamine class, specifically an N-benzylated derivative of the 2C-x family. First synthesized in 2003, it gained notoriety as a dangerous LSD adulterant. Unlike classical psychedelics, 25I-NBOMe has been associated with fatalities at recreational doses and is notably caustic when insufflated, causing severe nasal damage." ``` ### Example 3: Dissociative ```yaml summary: "Ketamine is an arylcyclohexylamine dissociative originally developed as an anesthetic in 1962. It produces dose-dependent effects ranging from mild dissociation to complete anesthetic states. Ketamine has gained attention for its rapid-acting antidepressant properties and is used clinically for treatment-resistant depression. Chronic use is associated with urinary tract damage." ``` --- ## Validation Checklist Before returning output: - [ ] Summary is 50-80 words - [ ] YAML is valid (no tabs, proper quoting) - [ ] No markdown code fences in output - [ ] No commentary or explanation, just YAML - [ ] Content derived only from provided sources - [ ] Does not duplicate detailed content from other sections - [ ] Includes safety warnings if mentioned prominently in sources
The citation workflow, zoomed in (scripts/citations/)
summary, pharmacology,
tolerance, harm, history, legality
(batches of ≤3) find real
references per claim
[cite:ref-id] markers only —article text must stay
byte-identical otherwise
every marker must map
to evidence + quotes
then applied with
--write --confirm-citation-writeaudit trail in
citationEvidence[cite:reference-id] markers to the six citable sections, but must not change any other byte of article text. Wikipedia and PsychonautWiki may be used to find references but are never themselves cited as claim support. Both rules are written into CLAUDE.md and checked by the validation gate.09 Two flows worth tracing
Flow A — a reader opens dose.wiki/lsd
src/app/[slug]/page.tsxserver component runs
lib/convex/publicData.tsqueries
substanceIndex.getBySlugsrc/schema/src/features/article/sections, tables, citations
prerendered when possible
Flow B — an editor saves an article in /dev
src/features/dev/forms/save-orchestrator/collects dirty articles
/api/save-to-convex:auth role + rate limit +
Zod contract checks
substanceIndex.saveSubstance+ auto
changelog entrytoast confirms the save
10 Recurring vocabulary
- Substance article
- The central data object: one large structured document per drug, with ~15 sections (dosage, duration, pharmacology, legality...). Defined by Zod schemas in
src/schema/, stored insubstanceIndex. - Citable sections
- The six prose sections allowed to carry citations:
summary,pharmacology,tolerance,harm_potential,history_culture,legality. Everything else (dosage tables etc.) is structured data, not cited prose. - Workbench
- An external review staging area for citation drafts. The pipeline exports a task, the draft is reviewed outside the repo, then
citations:apply-workbench -- --write --confirm-citation-writewrites the approved result back to Convex — both flags are required. - VCode
- A custom markup format used for subjective-effect article bodies, stored alongside a parsed AST and rendered by
src/features/effects/. - Prepopulate vs. generate
- Prepopulate = deterministic import of parsed source data (high-confidence facts like dosage tables). Generate = AI-drafted prose sections that always pass through human review.
- Dev vs. production Convex
- Two separate Convex deployments. Day-to-day work hits dev;
npm run convex:sync-productionpromotes data with explicit confirmation gates, andconvex:exportsnapshots production tosrc/data/SubstanceIndex.json— a separate, non-public file from thepublic/download described in section 07.
11 Where to start reading
A reasonable reading order for a first session:
README.mdandARCHITECTURE.md— the maintainers' own orientation docs, covering routes, npm scripts, and system layout.convex/schema.ts— the data model that everything else is built around. Skim thesubstanceIndextable definition first.src/app/[slug]/page.tsx→lib/convex/publicData.ts→src/features/article/— Flow A from section 09; this covers the whole public read path.middleware.ts+auth.ts— short files that show how/devis protected and how roles work.src/features/dev/save-orchestrator/andsrc/app/api/save-to-convex/— Flow B; how edits land in the database with an audit trail.scripts/lib/workflow-command-surface.mjs— the registry of pipeline commands; from here you can find whichever pipeline (batch, citations, migrate, sync) you need.
Caveats about this page
Counts (577 substances, 806 files, etc.) are a snapshot as of July 2026 and will drift. Some operational detail — exact sync gating, the full citation validation rules, rate-limit policies — is simplified here; CLAUDE.md, CONTEXT.md, and docs/architecture/ in the repo are the authoritative versions.
Want the content side? This page covers the codebase. For a detailed walkthrough of how each article is created, from scraping and quote extraction through AI synthesis, citations, and human review, read How dose.wiki articles are made.