Jamie BruntonandClaude Opus 5
1ac6ad22ad
A new Reddit account cannot start chats until Reddit considers it established, and nothing in the app previously did anything about that. Account growth posts a small number of AI-written comments per day on active threads in an operator-chosen list of communities, per account, so the history that lifts the hold actually accumulates. Growth is account-scoped rather than part of the shared outreach funnel: karma, comment history and pacing all belong to one identity. It runs as the lowest-priority work in the account runtime, after inbound replies, and deliberately keeps running while an account is held out of outbound messaging. Comments publish without operator review, so the guards are the feature: - Both screening stages fail closed. A deterministic gate rejects links, contact solicitation, user tags, filler and any text that reveals the generator; the classifier must actively approve with confidence, and an outage stalls growth rather than publishing unreviewed text. - Thread text reaches the model only as fenced untrusted data, since Reddit posts do contain instructions aimed at automated readers. - The daily allowance is adaptive, not configured: it starts at two per day and rises only while comments survive and earn upvotes. - Posted comments are swept at ~15m/2h/24h with one batched info.json request, plus a single signed-out probe that detects a comment the public cannot see. Two consecutive removals, an invisible comment, or a captcha challenge pauses the account until an operator clears it. A failed probe never pauses anything. - One comment per thread across every owned account, enforced by a unique index, because two of our accounts on one post is the easiest pattern to correlate. - A comment interrupted mid-post is abandoned, never retried: we cannot know whether the write landed and a duplicate cannot be withdrawn. - Reddit's refusals are authoritative — a ban disables that community for that account, a throttle backs off at the scope Reddit named, and a signed-out response invalidates the session. Comments are submitted through the account's own browser session against the old.reddit form endpoint, which returns machine-readable error codes and needs no credentials beyond the cookies already imported. API-first: /api/growth exposes state, settings, the community list and pause/resume, all verified against a local instance. Route responses are parsed with the shared contracts in tests so the server-side mirror cannot drift from what the dashboard expects. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
146 lines
8.6 KiB
Markdown
146 lines
8.6 KiB
Markdown
# Software architecture
|
|
|
|
## System shape
|
|
|
|
RedditBot is one Node.js process with four major surfaces:
|
|
|
|
1. A composition root that loads configuration, migrates SQLite, restores accounts, and wires services.
|
|
2. Per-account runtimes that own browser, Matrix, repositories, limits, conversation automation, and sync state.
|
|
3. A shared durable candidate funnel that discovers and screens once, then assigns ready work to an available account.
|
|
4. An authenticated Express/React operator surface with HTTP snapshots, typed SSE events, and a least-privilege agent API/MCP adapter.
|
|
|
|
```mermaid
|
|
flowchart LR
|
|
OP["Operator or agent"] --> WEB["Express API and React dashboard"]
|
|
WEB --> CTL["Persisted bot control"]
|
|
WEB --> DB[("SQLite WAL database")]
|
|
WEB --> EVT["Typed realtime event hub"]
|
|
|
|
IDX["src/index.ts"] --> BOOT["bootstrap application container"]
|
|
BOOT --> DB
|
|
BOOT --> MAN["Account runtime manager"]
|
|
BOOT --> WEB
|
|
|
|
MAN --> A1["Account runtime A"]
|
|
MAN --> A2["Account runtime B"]
|
|
A1 --> REDDIT["Reddit JSON and identity APIs"]
|
|
A2 --> REDDIT
|
|
A1 --> MATRIX["Reddit Matrix chat"]
|
|
A2 --> MATRIX
|
|
A1 --> AI["OpenRouter"]
|
|
A2 --> AI
|
|
|
|
DISC["Shared discovery loop"] --> DB
|
|
DB --> SCREEN["Shared screening and preparation"]
|
|
SCREEN --> DB
|
|
DB --> SEND["Account-aware dispatch"]
|
|
SEND --> A1
|
|
SEND --> A2
|
|
```
|
|
|
|
## Startup and lifecycle
|
|
|
|
`src/index.ts` creates the logger and database, runs migrations, loads database-backed configuration (or imports legacy `config.yaml` once), calls `bootstrap()`, and hands the resulting `AppContainer` to `runMainLoop()`.
|
|
|
|
`src/bootstrap.ts` is the composition root. It intentionally contains wiring rather than business decisions. It creates repositories, the fallback bootstrap browser/session, shared dependencies, the web server, and `RedditAccountRuntimeManager`. Enabled verified accounts are restored at startup. Each account runtime verifies both the Reddit cookie identity and Matrix identity before becoming usable.
|
|
|
|
If account runtimes exist, `runMainLoop()` selects the durable multi-account path. A legacy single-runtime path remains for compatibility/testing, but new behavior should target the account-runtime model.
|
|
|
|
SIGINT/SIGTERM abort shared loops, stop account runtimes, stop the web server, close browser contexts, and close SQLite. Long-lived loops use abort signals and non-overlapping schedules so shutdown does not leave concurrent work behind.
|
|
|
|
## Ownership boundaries
|
|
|
|
### Process-global
|
|
|
|
- SQLite connection and migrations.
|
|
- persisted bot running/paused state;
|
|
- durable candidate funnel;
|
|
- Reddit request coordinator (shared across runtime construction);
|
|
- recipient history/reservations across accounts;
|
|
- Express server, dashboard assets, event stream, and operational jobs;
|
|
- configured subreddit catalog and application policy;
|
|
- shared operator-owned exchange library, classification metadata, and outbound-use ledger;
|
|
|
|
### Account-scoped
|
|
|
|
- verified Reddit and Matrix identity;
|
|
- persistent Playwright browser profile;
|
|
- posts, messages, outcomes, activity, outbox, and conversation state repositories;
|
|
- growth settings, community list, comment history, and adaptive comment allowance;
|
|
- Matrix chat cache and sync cursor;
|
|
- media index and files;
|
|
- rate limiter and invitation cooldown;
|
|
- response checker and conversation automation;
|
|
- AI usage attribution.
|
|
|
|
Account IDs use the verified Matrix-shaped identity. A runtime is immutable with respect to identity: refreshing a session replaces that account runtime instead of mutating it in place. Operations for one account ID are serialized; different account runtimes may operate concurrently.
|
|
|
|
## Architectural layers
|
|
|
|
| Layer | Location | Responsibility |
|
|
| --- | --- | --- |
|
|
| Composition/lifecycle | `src/index.ts`, `src/bootstrap.ts`, `src/main-loop.ts` | Build, start, coordinate, pause, and shut down the application. |
|
|
| Domain services | `src/services/` | Screening, preparation, delivery, chat, conversation policy, outcomes, sync, and account lifecycle. |
|
|
| Integrations | `src/reddit/`, `src/ai/`, `src/browser/`, `src/services/chat/matrix/` | Reddit HTTP, OpenRouter, Playwright, and Matrix protocols. |
|
|
| Persistence | `src/database/`, `src/config/persistence.ts` | Migrations, account-scoped repositories, candidate leases, runtime/config state. |
|
|
| Web/API | `src/web/` | Auth, routes, snapshots, jobs, API catalog, static assets, activity and SSE. |
|
|
| Shared transport | `packages/api-contracts/` | Strict Zod contracts used by server and dashboard. |
|
|
| Operator UI | `apps/dashboard/` | React routes, feature slices, queries, mutations, realtime repair, and presentation. |
|
|
| Agent adapter | `src/mcp/` | Stdio MCP tools backed by the HTTP agent API. |
|
|
|
|
## Core design decisions
|
|
|
|
### Durable funnel, ephemeral workers
|
|
|
|
Candidate state and leases live in SQLite. Discovery, screening, and dispatch loops are replaceable workers. On restart, leases are recovered and due retry work is promoted rather than relying on an in-memory queue.
|
|
|
|
### Shared discovery, account-specific delivery
|
|
|
|
A healthy authenticated runtime may scan and prepare a candidate, but the chosen delivery account owns the final post/message rows and remote send. Prepared data is materialized for the recipient account immediately before delivery. This avoids duplicated screening while preserving account ownership and foreign keys.
|
|
|
|
### Snapshots are authoritative; events accelerate them
|
|
|
|
HTTP GET responses are the authoritative dashboard state. Typed SSE events invalidate or repair query caches and provide fast feedback, but reconnects fall back to fresh snapshots. The event hub provides IDs and bounded replay rather than making the UI depend on an uninterrupted stream.
|
|
|
|
### Contracts at process boundaries
|
|
|
|
External configuration, HTTP requests/responses, SSE events, session imports, and persisted JSON are validated with Zod or narrow parsers. Database rows and service-private types should not accidentally become public API shapes.
|
|
|
|
### Dependency injection over module mocking
|
|
|
|
Services accept dependencies through constructors or narrow callbacks. Tests use in-memory SQLite, fake clocks/fetchers, and service seams instead of relying on fragile ESM namespace mocks.
|
|
|
|
## Repository map
|
|
|
|
```text
|
|
apps/dashboard/ React operator console
|
|
packages/api-contracts/ Shared Zod HTTP/SSE contracts
|
|
src/ai/ OpenRouter client and prompt/model fallback behavior
|
|
src/browser/ Playwright context and Reddit cookie identity
|
|
src/config/ Runtime schemas, tone profiles, SQLite persistence
|
|
src/database/ Migrations and repositories
|
|
src/mcp/ Agent-facing MCP adapter over HTTP
|
|
src/reddit/ Reddit JSON client and shared request scheduler
|
|
src/services/ Domain and account-runtime services
|
|
src/web/ Express auth, API routes, jobs, events, static host
|
|
tests/ Backend, database, contract, web, and dashboard tests
|
|
extensions/reddit-session-exporter/ Local Chrome extension for verified session dumps
|
|
scripts/ API documentation generation and local helpers
|
|
docs/ Maintainer and agent documentation
|
|
```
|
|
|
|
## Where to extend the system
|
|
|
|
- New transport field: define it in `packages/api-contracts`, return it from the route, consume it in the dashboard, and add contract/route/UI tests.
|
|
- New API operation: add a focused router, register it in `src/web/server.ts`, describe it in `src/web/api-catalog.ts`, regenerate `docs/api.md`, and add authorization tests.
|
|
- New persisted state: add an append-only migration, expose it through a small repository, and test fresh plus upgraded databases.
|
|
- Shared operator media: keep it under `MediaLibraryService`/`MediaLibraryRepository`; never merge it with account-owned received media.
|
|
- New account behavior: place it in the account runtime or a service injected into it; do not add mutable cross-account identity to a singleton.
|
|
- New account-growth behavior: keep it account-scoped and lowest-priority, fail closed before anything is published, and make every outcome visible in the growth audit trail.
|
|
- New candidate stage: make transitions durable and idempotent, expose counts/reasons, and define lease/retry behavior.
|
|
- New dashboard feature: add a feature slice and route, use shared schemas and query keys, and treat HTTP as authoritative.
|
|
|
|
## Known design pressure
|
|
|
|
`src/bootstrap.ts` remains a large composition root and the legacy single-runtime path duplicates parts of the multi-account delivery flow. Changes should reduce duplication through small factories/policies without hiding operational state or merging account-owned dependencies into process-global singletons.
|