docs: describe React dashboard migration state

This commit is contained in:
jjbrunton committed 2026-08-08 15:32:16 +01:00
1 parent dcf2a1200c
commit 8824da3f15
6 files changed
+194 -95

No files matched your search

+11 -11
View File
@@ -27,22 +27,21 @@ The RedditBot web dashboard exposes a REST API for monitoring and controlling th
**Content-Type**: `application/json` (except SSE)
**Response Format**: JSON
**Authentication**: Dashboard session cookie or bearer token
**Authentication**: Required for cutover; not yet enforced by the checked-in Express host
---
## Authentication
The dashboard, API, SSE streams, and private media require the token configured
through `WEB_DASHBOARD_TOKEN` (preferred) or `web.authToken`. Interactive users
exchange the token for an HttpOnly, SameSite=Strict session cookie at
`POST /auth/session`. API clients may instead send
`Authorization: Bearer <token>`.
The React cutover requires a single Express authentication boundary covering
the dashboard assets, `/api`, both SSE streams, debug routes, and private media.
That boundary and its token/session configuration are not present in the
checked-in server yet. Until they land and their security tests pass, bind the
dashboard to a trusted network and do not expose port 3000 publicly.
`GET /auth/login` and `POST /auth/session` are the only unauthenticated
dashboard routes. A rejected API request returns HTTP 401 with
`{"error":"Dashboard authentication required"}`. Debug endpoints are absent
unless `web.debugEnabled` is explicitly enabled.
The intended contract is an HttpOnly, SameSite=Strict dashboard session for
interactive users and bearer-token access for API clients. Treat those details
as a migration requirement, not as proof of current protection.
---
@@ -1001,7 +1000,8 @@ the complete validated event envelope:
Clients may reconnect with `Last-Event-ID`; the server replays retained events
after that ID in order. The stream sends heartbeats, bounds buffered events and
slow-client queues, and requires the same dashboard authentication as the API.
slow-client queues. Applying the same authentication boundary as the API is a
pending cutover gate.
Current event families include bot status, message sent or failed,
conversation updated, reply received, sync lifecycle and progress, rate-limit
+66 -23
View File
@@ -41,7 +41,7 @@
| Response Monitoring | Background service checks inbox for user replies |
| Chat Automation | Reddit chat scraping via shadow DOM traversal, auto-reply support |
| Media Management | Download, cache, deduplicate, and sync media from conversations |
| Web Dashboard | Real-time monitoring with SSE log streaming, analytics, and controls |
| Web Dashboard | React operator console with typed SSE events, analytics, and guarded controls |
---
@@ -101,9 +101,9 @@
│
▼
┌─────────────────────────────────────────────────────────┐
│ Web Dashboard (Express) │
│ React Dashboard + Express API/SSE host │
│ │
│ 25+ API endpoints │ SSE log streaming │ Analytics │
│ Typed contracts │ SSE events │ Analytics │
│ Runtime config │ Bot controls │ Media gallery │
└─────────────────────────────────────────────────────────┘
```
@@ -161,8 +161,14 @@ RedditBot/
├── Dockerfile # Container build definition
├── package.json # Node.js dependencies & scripts
├── tsconfig.json # TypeScript configuration
├── apps/
│ └── dashboard/ # React, TypeScript, React Router, and Vite
│ ├── src/ # Shell, routes, feature slices, API clients
│ └── dist/ # Generated production assets (not committed)
├── packages/
│ └── api-contracts/ # Shared Zod HTTP and realtime schemas
├── public/
│ └── index.html # Single-file web dashboard (HTML+CSS+JS)
│ └── index.html # Temporary legacy dashboard fallback
├── scripts/
│ ├── inspect-chat-dom.ts # Debug: inspect chat DOM structure
│ └── login.ts # Manual browser login helper
@@ -196,7 +202,9 @@ RedditBot/
│ ├── retry.ts # Exponential backoff wrapper (28 lines)
│ └── image.ts # Image download + base64 + MIME validation (34 lines)
└── web/
└── server.ts # Express dashboard + SSE streaming (794 lines)
├── server.ts # Express API, SSE, and dashboard host
├── dashboard-assets.ts # React-build selection with legacy fallback
└── events/ # Typed event hub and Express SSE adapter
```
---
@@ -206,6 +214,7 @@ RedditBot/
### Core Services
#### MonitorService (`src/services/monitor.ts`)
Discovers new image posts across configured subreddits.
- Shuffles subreddit order each cycle
@@ -214,6 +223,7 @@ Discovers new image posts across configured subreddits.
- Returns the single newest valid post per cycle
#### ProcessorService (`src/services/processor.ts`)
Core processing pipeline for each discovered post.
1. Downloads image via `ImageService`
@@ -224,6 +234,7 @@ Core processing pipeline for each discovered post.
6. Handles AI refusals gracefully (marks post, tries next)
#### StrategySelector (`src/services/strategy-selector.ts`)
Implements explore/exploit A/B testing for message variants.
- Tracks response rates per strategy variant
@@ -232,6 +243,7 @@ Implements explore/exploit A/B testing for message variants.
- Selects best variant based on weighted response rates
#### ResponseChecker (`src/services/response-checker.ts`)
Background service that monitors for user replies.
- Navigates Reddit inbox via Playwright
@@ -240,6 +252,7 @@ Background service that monitors for user replies.
- Stores successful message/response pairs for few-shot learning
#### ChatService (`src/services/chat.ts`)
Most complex service - manages Reddit chat interactions.
- Scrapes `chat.reddit.com` using JavaScript injection (shadow DOM traversal)
@@ -251,6 +264,7 @@ Most complex service - manages Reddit chat interactions.
- Parameter: `myUsername` is hardcoded (should be from config)
#### MediaService (`src/services/media.ts`)
Media download, caching, and indexing.
- Downloads images from chat conversations
@@ -260,6 +274,7 @@ Media download, caching, and indexing.
- Tracks download status per conversation
#### MediaSyncService (`src/services/media-sync.ts`)
Periodic background media synchronization.
- Runs incremental syncs (top 10 chats) or full syncs (all conversations)
@@ -268,6 +283,7 @@ Periodic background media synchronization.
- Exposed via web dashboard endpoints
#### AutoReplyService (`src/services/auto-reply.ts`)
Automated reply to Reddit chat conversations.
- Checks for unread messages in cached conversations
@@ -279,6 +295,7 @@ Automated reply to Reddit chat conversations.
### External Integrations
#### RedditApi (`src/reddit/api.ts`)
Axios-based Reddit API client.
- Fetches posts from subreddits (`/r/{name}/new.json`)
@@ -288,6 +305,7 @@ Axios-based Reddit API client.
- Uses Playwright browser cookies for authenticated requests
#### OpenRouterService (`src/ai/openrouter.ts`)
OpenRouter API client using OpenAI-compatible SDK.
- `analyzeImage()`: Multi-model image analysis with fallback chain
@@ -298,6 +316,7 @@ OpenRouter API client using OpenAI-compatible SDK.
- Refusal detection via heuristic patterns
#### Browser Client (`src/browser/client.ts`)
Playwright persistent context manager.
- Browser session persistence (cookies/storage)
@@ -306,6 +325,7 @@ Playwright persistent context manager.
- Debug screenshot capture on login failure
#### MessagePage (`src/browser/message-page.ts`)
Reddit DM compose page automation.
- Fills recipient/subject/body fields
@@ -316,6 +336,7 @@ Reddit DM compose page automation.
### Data Layer
#### Database Client (`src/database/client.ts`)
Better-SQLite3 database initialization.
- Creates database at configured path
@@ -324,6 +345,7 @@ Better-SQLite3 database initialization.
- Safe column-addition for schema evolution
#### Repositories (`src/database/repositories.ts`)
Three repository classes abstracting SQLite operations.
- **PostsRepository**: Processed post CRUD, mark sent, update analysis
@@ -333,6 +355,7 @@ Three repository classes abstracting SQLite operations.
### Utilities
#### Logger (`src/utils/logger.ts`)
Winston structured logging.
- JSON format for log files
@@ -340,6 +363,7 @@ Winston structured logging.
- File logging: `error.log` and combined log
#### RateLimiter (`src/utils/rate-limiter.ts`)
Token-bucket rate limiting.
- p-queue with concurrency=1
@@ -348,6 +372,7 @@ Token-bucket rate limiting.
- Timer status exposed to dashboard
#### Retry (`src/utils/retry.ts`)
Exponential backoff wrapper.
- p-retry with configurable retries (default 3)
@@ -355,6 +380,7 @@ Exponential backoff wrapper.
- Logging on retries and failures
#### Image (`src/utils/image.ts`)
Image processing utilities.
- Download via axios
@@ -364,15 +390,33 @@ Image processing utilities.
### Web Dashboard
#### WebServer (`src/web/server.ts`)
Express server with 25+ API endpoints.
Express server with API route modules and SSE transport.
- Stats, activity, messages, chats, conversations, media endpoints
- Bot controls (start/stop)
- SSE realtime log streaming
- Typed resumable events at `/api/events/stream`
- Temporary legacy log stream at `/api/logs/stream`
- Analytics: strategy, subreddit, daily trends
- Runtime config management (GET/PUT)
- Debug endpoints
#### React dashboard (`apps/dashboard`)
- React Router routes Overview, Conversations, Media, Performance, Operations,
and Settings.
- TanStack Query owns authoritative HTTP snapshots and mutation state.
- Feature-local clients validate responses with schemas from
`packages/api-contracts`.
- Operations consumes typed dashboard events and invalidates affected query
keys; cached content remains visible if the stream reconnects.
- Vite writes production assets to `apps/dashboard/dist`.
The asset resolver prefers the React build and can fall back to
`public/index.html`. Wiring that resolver into authenticated Express static and
private-media hosting is still a migration gate; the legacy file has not been
removed.
---
## Data Flow
@@ -597,7 +641,8 @@ All endpoints are served by the Express server at `http://localhost:{port}`.
| Method | Endpoint | Description |
|---|---|---|
| GET | `/api/logs/stream` | SSE realtime log stream |
| GET | `/api/events/stream` | Typed resumable SSE event stream |
| GET | `/api/logs/stream` | Temporary legacy SSE log stream |
| GET | `/api/logs` | Recent logs (HTTP) |
| GET | `/api/timers` | Rate limiter timer status |
| POST | `/api/debug/inspect-dom` | Debug: inspect chat DOM structure |
@@ -744,16 +789,14 @@ Configuration and database creation use factory functions:
### Observer Pattern
SSE (Server-Sent Events) for realtime log streaming:
SSE (Server-Sent Events) carries validated dashboard events:
```typescript
// Logger writes to a shared buffer
logger.info("Message sent to user123");
// The transport-agnostic hub assigns IDs and retains a bounded replay window.
eventHub.publish({ type: "message.sent", payload: { /* validated fields */ } });
// WebServer subscribes clients via SSE
app.get("/api/logs/stream", (req, res) => {
// Push new log entries to connected clients
});
// Express adapts the hub to resumable SSE.
router.get("/events/stream", broadcaster.events);
```
### Circuit Breaker / Retry
@@ -827,7 +870,7 @@ class RateLimiter {
The project includes full Docker support:
- **`Dockerfile`** - Multi-stage build with Node.js
- **`Dockerfile`** - Playwright image with browser, VNC, and frontend/server build tooling
- **`docker-compose.yml`** - Service orchestration
- **`.dockerignore`** - Excludes unnecessary files
@@ -849,7 +892,7 @@ Deployment documentation is provided in `DOKPLOY_DEPLOY.md`.
1. Set up `config.yaml` with target subreddits and settings
2. Set environment variables (or use `.env` file)
3. Run `npm install` to install dependencies
4. Run `npm run build` to compile TypeScript
4. Run `npm run build` to compile the server and Vite dashboard
5. Run `scripts/login.ts` once to establish browser session
6. Start the bot (via `npm start` or Docker)
@@ -859,10 +902,8 @@ Deployment documentation is provided in `DOKPLOY_DEPLOY.md`.
### Critical
1. **No Tests** - Zero test coverage. Add unit tests for services, especially:
- `StrategySelector` (complex algorithm)
- `ChatService` (shadow DOM logic)
- `OpenRouterService` (model fallback logic)
1. **Dashboard security cutover** - Wire authenticated React asset, API, SSE,
debug, and private-media hosting before exposing the dashboard externally.
2. **No README.md** - No project documentation for new developers.
@@ -881,9 +922,11 @@ Deployment documentation is provided in `DOKPLOY_DEPLOY.md`.
### Medium Priority
7. **Single-Page Dashboard** - `public/index.html` is ~2000+ lines with embedded CSS/JS. Consider a proper frontend framework.
7. **Legacy dashboard fallback** - `public/index.html` remains during migration.
Remove it only after authenticated React hosting and feature-parity gates pass.
8. **WebServer Size** - 794 lines. Consider splitting routes into separate files.
8. **Settings route** - The React route is still a placeholder; runtime and
access settings have not yet moved into validated route-backed forms.
9. **Config Mutability** - Runtime config updates via `PUT /api/config` modify the in-memory config object. Consider a proper config service with validation on updates.
+28 -3
View File
@@ -670,7 +670,32 @@ How the bot automatically replies to chat messages.
## Dashboard Data Flow
How data flows from services to the web dashboard.
How data flows from services to the React operator console.
### Current React model
1. Feature-local API clients request authoritative snapshots from `/api/*`.
2. Shared Zod schemas in `packages/api-contracts` validate HTTP responses and
realtime envelopes at runtime.
3. TanStack Query caches snapshots, preserves them during background refresh,
and owns mutation pending/error/success state.
4. Operations opens `/api/events/stream`. Each typed event invalidates the
affected query keys, such as stats, timers, conversations, media, or
operations activity.
5. Browser `EventSource` reconnects automatically. The client marks cached
state stale while reconnecting instead of blanking the UI.
```text
services/repositories -> Express /api snapshots -> Zod -> TanStack Query -> React routes
| ^
+-> realtime event hub -> /api/events/stream --------+
```
### Legacy fallback model
The diagram below describes `public/index.html`, which remains only as a
compatibility fallback until Express is switched to authenticated React static
hosting and private-media protection is verified.
```
┌─────────────────────────────────────────────────────────────────┐
@@ -698,7 +723,7 @@ How data flows from services to the web dashboard.
│ ├─► Update in-memory config │
│ └─► Return updated config │
│ │
│ ──── SSE LOG STREAMING ────────────────────────────────────── │
│ ──── LEGACY SSE LOG STREAMING ─────────────────────────────── │
│ │
│ GET /api/logs/stream │
│ ├─► Client connects │
@@ -709,7 +734,7 @@ How data flows from services to the web dashboard.
│ │ └─► Data: { level, message, timestamp } │
│ └─► On disconnect: unsubscribe │
│ │
│ ──── FRONTEND (public/index.html) ─────────────────────────── │
│ ──── LEGACY FALLBACK (public/index.html) ──────────────────── │
│ │
│ JavaScript polls API endpoints: │
│ ├─► /api/stats every 5 seconds │
+25 -27
View File
@@ -23,8 +23,8 @@ Complete guide for deploying RedditBot in various environments.
| Requirement | Version | Notes |
|---|---|---|
| Node.js | 18+ | LTS recommended |
| npm | 9+ | Or yarn/pnpm |
| Node.js | 20.19+ or 22.12+ | Required by the locked Vite toolchain |
| npm | Bundled with Node | Use `npm ci` for lockfile-exact installs |
| OpenRouter API Key | - | Sign up at [openrouter.ai](https://openrouter.ai) |
| Reddit Account | - | With DM permissions enabled |
@@ -110,16 +110,21 @@ This will:
### 4. Start the Bot
```bash
# Development (with TypeScript execution)
# Terminal 1: backend with TypeScript execution
npm run dev
# Production (compiled JavaScript)
# Terminal 2: Vite dashboard; proxies /api to port 3000
npm run dashboard:dev
# Production: compile server and dashboard, then start
npm run build
npm start
```
### 5. Access Dashboard
Open http://localhost:3000 in your browser.
Open the Vite URL (normally http://localhost:5173) during frontend development.
Production and legacy-fallback checks use http://localhost:3000.
---
@@ -145,28 +150,18 @@ docker run -d \
reddit-bot:latest
```
### Dockerfile Structure
### Dockerfile behavior
```dockerfile
# Build stage
FROM node:18-alpine AS builder
WORKDIR /app
COPY package*.json ./
RUN npm ci
COPY . .
RUN npm run build
The image is based on the Playwright Jammy image rather than Alpine because the
runtime needs Chromium and its system libraries. It installs Xvfb, x11vnc,
noVNC, Fluxbox, and Websockify for interactive Reddit login; installs the locked
Node dependencies; copies the repository; and runs `npm run build`.
# Production stage
FROM node:18-alpine
WORKDIR /app
COPY --from=builder /app/dist ./dist
COPY --from=builder /app/node_modules ./node_modules
COPY --from=builder /app/config.yaml ./config.yaml
COPY --from=builder /app/package.json ./package.json
EXPOSE 3000
CMD ["node", "dist/index.js"]
```
That build produces both `dist/` for the Express server and
`apps/dashboard/dist/` for Vite assets. Ports 3000 (dashboard/API) and 6080
(noVNC) are exposed, and `docker/entrypoint.sh` starts the display services and
application. This is a single-stage image and retains development dependencies
needed by the current build/runtime layout.
---
@@ -271,9 +266,12 @@ app:
### Security
1. **Network Isolation**: Run behind a reverse proxy with authentication
1. **Network Isolation**: Keep the dashboard on a trusted network during the
migration cutover
2. **Environment Variables**: Never commit credentials; use secrets management
3. **Dashboard Access**: Restrict to internal network or add auth
3. **Dashboard Access**: Authenticated Express hosting for React assets, API,
SSE, debug routes, and private media is a required cutover gate, not a
currently proven guarantee
4. **Browser Data**: Protect `data/browser/` directory
### Performance
+37 -21
View File
@@ -26,8 +26,12 @@ RedditBot/
├── tsconfig.json # TypeScript config
├── Dockerfile # Container build
├── docker-compose.yml # Docker Compose
├── apps/
│ └── dashboard/ # React + Vite operator console
├── packages/
│ └── api-contracts/ # Shared Zod transport contracts
├── public/
│ └── index.html # Web dashboard (single file)
│ └── index.html # Temporary legacy dashboard fallback
├── scripts/
│ ├── login.ts # Manual login helper
│ └── inspect-chat-dom.ts # DOM inspection debug script
@@ -60,8 +64,7 @@ RedditBot/
│ ├── rate-limiter.ts # Rate limiting
│ ├── retry.ts # Retry wrapper
│ └── image.ts # Image utilities
└── web/
└── server.ts # Express dashboard server
└── web/ # Express API, events, and dashboard hosting
```
---
@@ -101,16 +104,29 @@ npm run login
### 5. Start Development Server
```bash
# Development mode (hot reload with tsx)
# Terminal 1: backend reload with tsx
npm run dev
# Production mode (compiled JS)
# Terminal 2: Vite frontend with /api proxied to port 3000
npm run dashboard:dev
# Production build: server plus apps/dashboard/dist
npm run build && npm start
```
### 6. Access Dashboard
Open http://localhost:3000
During frontend development, open the URL printed by Vite (normally
http://localhost:5173). The Express dashboard remains on http://localhost:3000
for production/fallback verification.
Useful frontend checks:
```bash
npm run dashboard:typecheck
npm run dashboard:build
npx vitest run tests/dashboard
```
---
@@ -277,12 +293,14 @@ setInterval(() => {
}, intervalMs);
```
### 4. Add to Web Dashboard (Optional)
### 4. Add to the Web API and Dashboard (Optional)
If the service needs API endpoints, add them in `src/web/server.ts`:
If the service needs an API, add a focused router under `src/web/routes/` and
mount it from `src/web/server.ts`. Define its transport schema in
`packages/api-contracts`; do not expose repository rows directly.
```typescript
server.app.get('/api/new-service', async (req, res) => {
router.get('/new-service', async (req, res) => {
try {
const result = await newService.execute(req.query.input as string);
res.json(result);
@@ -292,6 +310,9 @@ server.app.get('/api/new-service', async (req, res) => {
});
```
Add frontend behavior under `apps/dashboard/src/features/<feature>/` and drive
it with Testing Library and MSW tests under `tests/dashboard/`.
### 5. Update Config (If Needed)
Add new configuration options to:
@@ -498,17 +519,12 @@ Reddit frequently updates their `faceplate-*` components. Shadow DOM traversal c
**Mitigation**: Use `scripts/inspect-chat-dom.ts` to debug when selectors change.
### No Test Coverage
### Dashboard Migration Boundary
The project has no automated tests. Any changes should be manually tested.
The React feature slices live in `apps/dashboard`, but `public/index.html`
remains as a compatibility fallback. Do not delete it until authenticated
Express hosting, private-media protection, and browser parity checks are proven.
**Recommendation**: Add tests for critical paths:
- StrategySelector algorithm
- ChatService shadow DOM parsing
- OpenRouterService model fallback
### Single-File Dashboard
The entire dashboard is in `public/index.html` (~2000+ lines). Making changes requires editing a single large file.
**Recommendation**: Consider migrating to a proper frontend framework for maintainability.
The repository has automated service, contract, component, and transport tests.
Manual browser verification is still required for Reddit DOM changes and the
final dashboard cutover.
+27 -10
View File
@@ -3,8 +3,24 @@
## Outcome
Replace the inline dashboard in `public/index.html` with a typed, accessible React
application without disrupting the bot worker or its Express API. Express remains
the authenticated API, SSE, private-media, and static-asset host.
application without disrupting the bot worker or its Express API. At cutover,
Express will remain the authenticated API, SSE, private-media, and static-asset
host.
## Current status
The React application and production build are present. The current routes are
Overview, Conversations (including route-addressable detail), Media,
Performance, Operations, and a Settings placeholder. Shared HTTP and realtime
schemas live in `packages/api-contracts`, and `/api/events/stream` supplies the
typed SSE transport.
`npm run build` compiles the server and writes the Vite application to
`apps/dashboard/dist`. `src/web/dashboard-assets.ts` can select that build and
fall back to `public/index.html`, but the checked-in Express server has not yet
completed that authenticated static-hosting cutover. Until it does, the legacy
page remains the served fallback and the dashboard must not be exposed to an
untrusted network.
The migration is complete only when the legacy dashboard can be removed without
losing a documented capability and the full test, type-check, production-build,
@@ -34,19 +50,20 @@ The application shell provides a responsive sidebar or drawer with these routes:
| `/media` | Private blurred-by-default gallery and sync controls |
| `/performance` | Trends, strategies, subreddits, and successful examples |
| `/operations` | Domain activity, background jobs, and diagnostics |
| `/operations/logs` | Filterable bounded realtime technical console |
| `/settings/*` | Monitoring, limits, behaviour, AI, and access settings |
| `/settings` | Placeholder for runtime and access settings |
The shell always surfaces bot state, dry-run state, Reddit session state,
realtime connection state, cooldowns, active sync progress, notifications, and
the guarded start/stop control.
Overview surfaces bot state, dry-run state, cooldowns, timers, and the guarded
start/stop control. Operations owns connection health, sync state, activity,
and the bounded technical console.
## Realtime model
HTTP snapshots are authoritative. A typed SSE event stream updates or invalidates
the relevant query cache. Every event has an ID, type, occurrence timestamp, and
validated payload. The stream supports bounded replay and `Last-Event-ID` resume.
After reconnection the client refetches authoritative snapshots to repair gaps.
Relevant events invalidate TanStack Query keys so the next HTTP read remains
authoritative. Connection loss marks cached state stale while `EventSource`
reconnects.
Required event families are bot status, message sent or failed, conversation
updated, reply received, sync lifecycle and progress, rate-limit state, Reddit
@@ -91,8 +108,8 @@ The legacy dashboard remains available until replacement behavior is proven.
- No production frontend or backend code uses `any` or `console.log`.
- Mutations display the confirmed server outcome and do not rely on optimistic
success for operational controls.
- Production builds are deterministic and the Docker image serves only generated
frontend assets plus authenticated private content.
- Production builds are deterministic. Serving only generated frontend assets
plus authenticated private content remains a cutover gate.
## Cutover evidence