142 lines
8.2 KiB
Markdown
142 lines
8.2 KiB
Markdown
# Software architecture
|
|
|
|
## System shape
|
|
|
|
RedditBot is one Node.js process with four major surfaces:
|
|
|
|
1. A composition root that loads configuration, migrates SQLite, restores accounts, and wires services.
|
|
2. Per-account runtimes that own browser, Matrix, repositories, limits, conversation automation, and sync state.
|
|
3. A shared durable candidate funnel that discovers and screens once, then assigns ready work to an available account.
|
|
4. An authenticated Express/React operator surface with HTTP snapshots, typed SSE events, and a least-privilege agent API/MCP adapter.
|
|
|
|
```mermaid
|
|
flowchart LR
|
|
OP["Operator or agent"] --> WEB["Express API and React dashboard"]
|
|
WEB --> CTL["Persisted bot control"]
|
|
WEB --> DB[("SQLite WAL database")]
|
|
WEB --> EVT["Typed realtime event hub"]
|
|
|
|
IDX["src/index.ts"] --> BOOT["bootstrap application container"]
|
|
BOOT --> DB
|
|
BOOT --> MAN["Account runtime manager"]
|
|
BOOT --> WEB
|
|
|
|
MAN --> A1["Account runtime A"]
|
|
MAN --> A2["Account runtime B"]
|
|
A1 --> REDDIT["Reddit JSON and identity APIs"]
|
|
A2 --> REDDIT
|
|
A1 --> MATRIX["Reddit Matrix chat"]
|
|
A2 --> MATRIX
|
|
A1 --> AI["OpenRouter"]
|
|
A2 --> AI
|
|
|
|
DISC["Shared discovery loop"] --> DB
|
|
DB --> SCREEN["Shared screening and preparation"]
|
|
SCREEN --> DB
|
|
DB --> SEND["Account-aware dispatch"]
|
|
SEND --> A1
|
|
SEND --> A2
|
|
```
|
|
|
|
## Startup and lifecycle
|
|
|
|
`src/index.ts` creates the logger and database, runs migrations, loads database-backed configuration (or imports legacy `config.yaml` once), calls `bootstrap()`, and hands the resulting `AppContainer` to `runMainLoop()`.
|
|
|
|
`src/bootstrap.ts` is the composition root. It intentionally contains wiring rather than business decisions. It creates repositories, the fallback bootstrap browser/session, shared dependencies, the web server, and `RedditAccountRuntimeManager`. Enabled verified accounts are restored at startup. Each account runtime verifies both the Reddit cookie identity and Matrix identity before becoming usable.
|
|
|
|
If account runtimes exist, `runMainLoop()` selects the durable multi-account path. A legacy single-runtime path remains for compatibility/testing, but new behavior should target the account-runtime model.
|
|
|
|
SIGINT/SIGTERM abort shared loops, stop account runtimes, stop the web server, close browser contexts, and close SQLite. Long-lived loops use abort signals and non-overlapping schedules so shutdown does not leave concurrent work behind.
|
|
|
|
## Ownership boundaries
|
|
|
|
### Process-global
|
|
|
|
- SQLite connection and migrations.
|
|
- persisted bot running/paused state;
|
|
- durable candidate funnel;
|
|
- Reddit request coordinator (shared across runtime construction);
|
|
- recipient history/reservations across accounts;
|
|
- Express server, dashboard assets, event stream, and operational jobs;
|
|
- configured subreddit catalog and application policy.
|
|
|
|
### Account-scoped
|
|
|
|
- verified Reddit and Matrix identity;
|
|
- persistent Playwright browser profile;
|
|
- posts, messages, outcomes, activity, outbox, and conversation state repositories;
|
|
- Matrix chat cache and sync cursor;
|
|
- media index and files;
|
|
- rate limiter and invitation cooldown;
|
|
- response checker and conversation automation;
|
|
- AI usage attribution.
|
|
|
|
Account IDs use the verified Matrix-shaped identity. A runtime is immutable with respect to identity: refreshing a session replaces that account runtime instead of mutating it in place. Operations for one account ID are serialized; different account runtimes may operate concurrently.
|
|
|
|
## Architectural layers
|
|
|
|
| Layer | Location | Responsibility |
|
|
| --- | --- | --- |
|
|
| Composition/lifecycle | `src/index.ts`, `src/bootstrap.ts`, `src/main-loop.ts` | Build, start, coordinate, pause, and shut down the application. |
|
|
| Domain services | `src/services/` | Screening, preparation, delivery, chat, conversation policy, outcomes, sync, and account lifecycle. |
|
|
| Integrations | `src/reddit/`, `src/ai/`, `src/browser/`, `src/services/chat/matrix/` | Reddit HTTP, OpenRouter, Playwright, and Matrix protocols. |
|
|
| Persistence | `src/database/`, `src/config/persistence.ts` | Migrations, account-scoped repositories, candidate leases, runtime/config state. |
|
|
| Web/API | `src/web/` | Auth, routes, snapshots, jobs, API catalog, static assets, activity and SSE. |
|
|
| Shared transport | `packages/api-contracts/` | Strict Zod contracts used by server and dashboard. |
|
|
| Operator UI | `apps/dashboard/` | React routes, feature slices, queries, mutations, realtime repair, and presentation. |
|
|
| Agent adapter | `src/mcp/` | Stdio MCP tools backed by the HTTP agent API. |
|
|
|
|
## Core design decisions
|
|
|
|
### Durable funnel, ephemeral workers
|
|
|
|
Candidate state and leases live in SQLite. Discovery, screening, and dispatch loops are replaceable workers. On restart, leases are recovered and due retry work is promoted rather than relying on an in-memory queue.
|
|
|
|
### Shared discovery, account-specific delivery
|
|
|
|
A healthy authenticated runtime may scan and prepare a candidate, but the chosen delivery account owns the final post/message rows and remote send. Prepared data is materialized for the recipient account immediately before delivery. This avoids duplicated screening while preserving account ownership and foreign keys.
|
|
|
|
### Snapshots are authoritative; events accelerate them
|
|
|
|
HTTP GET responses are the authoritative dashboard state. Typed SSE events invalidate or repair query caches and provide fast feedback, but reconnects fall back to fresh snapshots. The event hub provides IDs and bounded replay rather than making the UI depend on an uninterrupted stream.
|
|
|
|
### Contracts at process boundaries
|
|
|
|
External configuration, HTTP requests/responses, SSE events, session imports, and persisted JSON are validated with Zod or narrow parsers. Database rows and service-private types should not accidentally become public API shapes.
|
|
|
|
### Dependency injection over module mocking
|
|
|
|
Services accept dependencies through constructors or narrow callbacks. Tests use in-memory SQLite, fake clocks/fetchers, and service seams instead of relying on fragile ESM namespace mocks.
|
|
|
|
## Repository map
|
|
|
|
```text
|
|
apps/dashboard/ React operator console
|
|
packages/api-contracts/ Shared Zod HTTP/SSE contracts
|
|
src/ai/ OpenRouter client and prompt/model fallback behavior
|
|
src/browser/ Playwright context and Reddit cookie identity
|
|
src/config/ Runtime schemas, tone profiles, SQLite persistence
|
|
src/database/ Migrations and repositories
|
|
src/mcp/ Agent-facing MCP adapter over HTTP
|
|
src/reddit/ Reddit JSON client and shared request scheduler
|
|
src/services/ Domain and account-runtime services
|
|
src/web/ Express auth, API routes, jobs, events, static host
|
|
tests/ Backend, database, contract, web, and dashboard tests
|
|
extensions/reddit-session-exporter/ Local Chrome extension for verified session dumps
|
|
scripts/ API documentation generation and local helpers
|
|
docs/ Maintainer and agent documentation
|
|
```
|
|
|
|
## Where to extend the system
|
|
|
|
- New transport field: define it in `packages/api-contracts`, return it from the route, consume it in the dashboard, and add contract/route/UI tests.
|
|
- New API operation: add a focused router, register it in `src/web/server.ts`, describe it in `src/web/api-catalog.ts`, regenerate `docs/api.md`, and add authorization tests.
|
|
- New persisted state: add an append-only migration, expose it through a small repository, and test fresh plus upgraded databases.
|
|
- New account behavior: place it in the account runtime or a service injected into it; do not add mutable cross-account identity to a singleton.
|
|
- New candidate stage: make transitions durable and idempotent, expose counts/reasons, and define lease/retry behavior.
|
|
- New dashboard feature: add a feature slice and route, use shared schemas and query keys, and treat HTTP as authoritative.
|
|
|
|
## Known design pressure
|
|
|
|
`src/bootstrap.ts` remains a large composition root and the legacy single-runtime path duplicates parts of the multi-account delivery flow. Changes should reduce duplication through small factories/policies without hiding operational state or merging account-owned dependencies into process-global singletons.
|