jjbrunton
633bb0ec04
Split monolithic chat service into modular structure with Matrix client support, extract web server into per-resource route modules, introduce bootstrap/main-loop separation, and add comprehensive vitest coverage across services, repositories, and utilities. Adds project docs and CLAUDE.md agent instructions.
515 lines
11 KiB
Markdown
515 lines
11 KiB
Markdown
# Development Guide
|
|
|
|
Guide for contributing to RedditBot development.
|
|
|
|
---
|
|
|
|
## Table of Contents
|
|
|
|
1. [Project Structure](#project-structure)
|
|
2. [Development Setup](#development-setup)
|
|
3. [Coding Standards](#coding-standards)
|
|
4. [Adding a New Service](#adding-a-new-service)
|
|
5. [Adding a New Strategy Variant](#adding-a-new-strategy-variant)
|
|
6. [Debugging](#debugging)
|
|
7. [Common Tasks](#common-tasks)
|
|
8. [Known Issues](#known-issues)
|
|
|
|
---
|
|
|
|
## Project Structure
|
|
|
|
```
|
|
RedditBot/
|
|
├── config.yaml # Main configuration
|
|
├── package.json # Dependencies and scripts
|
|
├── tsconfig.json # TypeScript config
|
|
├── Dockerfile # Container build
|
|
├── docker-compose.yml # Docker Compose
|
|
├── public/
|
|
│ └── index.html # Web dashboard (single file)
|
|
├── scripts/
|
|
│ ├── login.ts # Manual login helper
|
|
│ └── inspect-chat-dom.ts # DOM inspection debug script
|
|
└── src/
|
|
├── index.ts # Entry point / orchestrator
|
|
├── config/
|
|
│ └── index.ts # Config loader + Zod schema
|
|
├── database/
|
|
│ ├── client.ts # SQLite setup + migrations
|
|
│ └── repositories.ts # Repository classes
|
|
├── reddit/
|
|
│ └── api.ts # Reddit API client
|
|
├── ai/
|
|
│ └── openrouter.ts # OpenRouter AI client
|
|
├── browser/
|
|
│ ├── client.ts # Playwright browser manager
|
|
│ ├── message-page.ts # DM compose automation
|
|
│ └── media-downloader.ts # Media download
|
|
├── services/
|
|
│ ├── monitor.ts # Post discovery
|
|
│ ├── processor.ts # Processing pipeline
|
|
│ ├── response-checker.ts # Response monitoring
|
|
│ ├── strategy-selector.ts # A/B testing
|
|
│ ├── chat.ts # Chat scraping + messaging
|
|
│ ├── media.ts # Media management
|
|
│ ├── media-sync.ts # Media sync scheduler
|
|
│ └── auto-reply.ts # Auto-reply service
|
|
├── utils/
|
|
│ ├── logger.ts # Winston logger
|
|
│ ├── rate-limiter.ts # Rate limiting
|
|
│ ├── retry.ts # Retry wrapper
|
|
│ └── image.ts # Image utilities
|
|
└── web/
|
|
└── server.ts # Express dashboard server
|
|
```
|
|
|
|
---
|
|
|
|
## Development Setup
|
|
|
|
### 1. Install Dependencies
|
|
|
|
```bash
|
|
npm install
|
|
```
|
|
|
|
### 2. Configure
|
|
|
|
Copy and edit `config.yaml`:
|
|
|
|
```bash
|
|
cp config.yaml config.yaml.backup # Backup existing
|
|
# Edit config.yaml with your settings
|
|
```
|
|
|
|
### 3. Set Environment Variables
|
|
|
|
```bash
|
|
export REDDIT_USERNAME="your-username"
|
|
export REDDIT_PASSWORD="your-password"
|
|
export OPENROUTER_API_KEY="sk-or-v1-your-key"
|
|
```
|
|
|
|
### 4. Initial Login
|
|
|
|
```bash
|
|
# Open browser for manual login
|
|
npm run login
|
|
```
|
|
|
|
### 5. Start Development Server
|
|
|
|
```bash
|
|
# Development mode (hot reload with tsx)
|
|
npm run dev
|
|
|
|
# Production mode (compiled JS)
|
|
npm run build && npm start
|
|
```
|
|
|
|
### 6. Access Dashboard
|
|
|
|
Open http://localhost:3000
|
|
|
|
---
|
|
|
|
## Coding Standards
|
|
|
|
### TypeScript
|
|
|
|
- **Strict mode**: No `any` types (use `unknown` with type guards)
|
|
- **Interfaces**: Define interfaces for all data structures
|
|
- **Null checks**: Use optional chaining (`?.`) and nullish coalescing (`??`)
|
|
- **Async/Await**: Use async/await, not `.then()` chains
|
|
|
|
```typescript
|
|
// Good
|
|
interface PostData {
|
|
id: string;
|
|
subreddit: string;
|
|
author: string;
|
|
}
|
|
|
|
async function fetchPost(id: string): Promise<PostData> {
|
|
const response = await api.get(`/posts/${id}`);
|
|
return response.data;
|
|
}
|
|
|
|
// Bad
|
|
async function fetchPost(id: any): Promise<any> {
|
|
return api.get(`/posts/${id}`).then(r => r.data);
|
|
}
|
|
```
|
|
|
|
### Logging
|
|
|
|
Use `winston` logger, never `console.log`:
|
|
|
|
```typescript
|
|
// Good
|
|
this.logger.info("Processed post {id}", { id: post.id });
|
|
this.logger.warn("AI refusal detected", { postId: post.id });
|
|
this.logger.error("Failed to send DM", { error: err.message });
|
|
|
|
// Bad
|
|
console.log("Processed post", post.id);
|
|
```
|
|
|
|
### Error Handling
|
|
|
|
Use try/catch with structured error handling:
|
|
|
|
```typescript
|
|
try {
|
|
await this.processPost(post);
|
|
} catch (err) {
|
|
this.logger.error("Post processing failed", {
|
|
postId: post.id,
|
|
error: err instanceof Error ? err.message : String(err)
|
|
});
|
|
// Don't throw - allow other posts to be processed
|
|
}
|
|
```
|
|
|
|
### Service Structure
|
|
|
|
Follow single responsibility:
|
|
|
|
```typescript
|
|
class NewService {
|
|
constructor(
|
|
private config: Config,
|
|
private dependency: ExistingService,
|
|
private logger: Logger
|
|
) {}
|
|
|
|
async doSomething(input: string): Promise<Result> {
|
|
// Validate input
|
|
if (!input) {
|
|
throw new Error("Input is required");
|
|
}
|
|
|
|
// Do work
|
|
const result = await this.dependency.process(input);
|
|
|
|
// Return result
|
|
return result;
|
|
}
|
|
}
|
|
```
|
|
|
|
### File Organization
|
|
|
|
- One class per file (except repositories which share one file)
|
|
- Services in `src/services/`
|
|
- Utilities in `src/utils/`
|
|
- External integrations in their own folders (`src/ai/`, `src/reddit/`, `src/browser/`)
|
|
|
|
---
|
|
|
|
## Adding a New Service
|
|
|
|
### 1. Create the Service File
|
|
|
|
```typescript
|
|
// src/services/new-service.ts
|
|
import { Config } from '../config';
|
|
import { Logger } from '../utils/logger';
|
|
|
|
interface NewServiceDeps {
|
|
config: Config;
|
|
logger: Logger;
|
|
// other dependencies
|
|
}
|
|
|
|
interface Result {
|
|
success: boolean;
|
|
data?: any;
|
|
error?: string;
|
|
}
|
|
|
|
export class NewService {
|
|
constructor(private deps: NewServiceDeps) {}
|
|
|
|
async execute(input: string): Promise<Result> {
|
|
const { config, logger } = this.deps;
|
|
|
|
logger.info("NewService executing", { input });
|
|
|
|
// Implementation here
|
|
|
|
return { success: true };
|
|
}
|
|
}
|
|
```
|
|
|
|
### 2. Add to Dependencies Interface
|
|
|
|
If the service needs other services, add them to the interface:
|
|
|
|
```typescript
|
|
interface NewServiceDeps {
|
|
config: Config;
|
|
logger: Logger;
|
|
redditApi: RedditApi;
|
|
openrouter: OpenRouterService;
|
|
}
|
|
```
|
|
|
|
### 3. Wire Up in main()
|
|
|
|
In `src/index.ts`, add initialization:
|
|
|
|
```typescript
|
|
const newService = new NewService({
|
|
config,
|
|
logger,
|
|
redditApi,
|
|
openrouter
|
|
});
|
|
|
|
// If it needs to run periodically:
|
|
setInterval(() => {
|
|
newService.execute(someInput).catch(err => {
|
|
logger.error("NewService error", { error: err.message });
|
|
});
|
|
}, intervalMs);
|
|
```
|
|
|
|
### 4. Add to Web Dashboard (Optional)
|
|
|
|
If the service needs API endpoints, add them in `src/web/server.ts`:
|
|
|
|
```typescript
|
|
server.app.get('/api/new-service', async (req, res) => {
|
|
try {
|
|
const result = await newService.execute(req.query.input as string);
|
|
res.json(result);
|
|
} catch (err) {
|
|
res.status(500).json({ error: (err as Error).message });
|
|
}
|
|
});
|
|
```
|
|
|
|
### 5. Update Config (If Needed)
|
|
|
|
Add new configuration options to:
|
|
- `config.yaml`
|
|
- `src/config/index.ts` Zod schema
|
|
- Configuration documentation
|
|
|
|
---
|
|
|
|
## Adding a New Strategy Variant
|
|
|
|
### 1. Add to config.yaml
|
|
|
|
```yaml
|
|
strategyVariants:
|
|
- name: "existing_variant"
|
|
prompt: "Existing prompt..."
|
|
enabled: true
|
|
|
|
- name: "new_variant"
|
|
prompt: |
|
|
Write a friendly message that references the image content.
|
|
Be specific and genuine. Keep it under 2 sentences.
|
|
enabled: true
|
|
```
|
|
|
|
### 2. No Code Changes Needed
|
|
|
|
The `StrategySelector` automatically picks up new variants from the config. The algorithm will:
|
|
|
|
1. Include the new variant in random exploration
|
|
2. Track its response rate once messages are sent
|
|
3. Promote it to exploitation if it has good stats
|
|
|
|
### 3. Monitor Performance
|
|
|
|
Check performance via the dashboard or API:
|
|
|
|
```bash
|
|
curl http://localhost:3000/api/analytics/by-strategy
|
|
```
|
|
|
|
---
|
|
|
|
## Debugging
|
|
|
|
### Browser Debugging
|
|
|
|
Run with visible browser:
|
|
|
|
```yaml
|
|
# config.yaml
|
|
browser:
|
|
headless: false
|
|
```
|
|
|
|
Or set environment variable:
|
|
|
|
```bash
|
|
BROWSER_HEADLESS=false npm run dev
|
|
```
|
|
|
|
### DOM Inspection
|
|
|
|
Use the debug endpoint to inspect Reddit's shadow DOM:
|
|
|
|
```bash
|
|
curl -X POST http://localhost:3000/api/debug/inspect-dom
|
|
```
|
|
|
|
Or run the standalone script:
|
|
|
|
```bash
|
|
npx tsx scripts/inspect-chat-dom.ts
|
|
```
|
|
|
|
### Logging
|
|
|
|
Adjust log level by modifying `src/utils/logger.ts`:
|
|
|
|
```typescript
|
|
const logger = winston.createLogger({
|
|
level: 'debug', // Change from 'info' to 'debug'
|
|
// ...
|
|
});
|
|
```
|
|
|
|
### Database Inspection
|
|
|
|
Open the database with SQLite viewer:
|
|
|
|
```bash
|
|
# Install sqlite3 if needed
|
|
brew install sqlite3
|
|
|
|
# Open database
|
|
sqlite3 data/reddit-bot.db
|
|
|
|
# Query
|
|
SELECT * FROM sent_messages LIMIT 5;
|
|
```
|
|
|
|
### Network Debugging
|
|
|
|
Enable axios debug:
|
|
|
|
```typescript
|
|
// In src/reddit/api.ts
|
|
import axios from 'axios';
|
|
axios.defaults.debug = true;
|
|
```
|
|
|
|
---
|
|
|
|
## Common Tasks
|
|
|
|
### Add a New Subreddit
|
|
|
|
```yaml
|
|
# config.yaml
|
|
reddit:
|
|
subreddits:
|
|
- gonewild
|
|
- ratemyboobs
|
|
- new_subreddit_name # Add here
|
|
```
|
|
|
|
### Change AI Model
|
|
|
|
```yaml
|
|
# config.yaml
|
|
openrouter:
|
|
model: "new-model-name"
|
|
fallbackModels:
|
|
- "fallback-model-1"
|
|
- "fallback-model-2"
|
|
```
|
|
|
|
### Adjust Rate Limits
|
|
|
|
```yaml
|
|
# config.yaml
|
|
rateLimit:
|
|
minDelayMs: 45000 # Increase delay
|
|
maxMessagesPerHour: 15 # Reduce cap
|
|
jitterMs: 20000 # More jitter
|
|
```
|
|
|
|
### Enable Dry Run
|
|
|
|
```yaml
|
|
# config.yaml
|
|
dryRun: true # Logs all actions without sending
|
|
```
|
|
|
|
Or via dashboard: `PUT /api/config` with `{"dryRun": true}`
|
|
|
|
### Add Database Column
|
|
|
|
```typescript
|
|
// In src/database/client.ts
|
|
function runMigrations(db: Database) {
|
|
// ... existing migrations ...
|
|
|
|
// Check if column exists
|
|
const tableInfo = db.prepare(`PRAGMA table_info(sent_messages)`).all();
|
|
const hasNewColumn = tableInfo.some((col: any) => col.name === 'new_column');
|
|
|
|
if (!hasNewColumn) {
|
|
db.exec("ALTER TABLE sent_messages ADD COLUMN new_column TEXT");
|
|
}
|
|
}
|
|
```
|
|
|
|
### Add New API Endpoint
|
|
|
|
```typescript
|
|
// In src/web/server.ts
|
|
import { Request, Response } from 'express';
|
|
|
|
server.app.get('/api/new-endpoint', async (req: Request, res: Response) => {
|
|
try {
|
|
const result = await someService.doSomething();
|
|
res.json({ success: true, data: result });
|
|
} catch (err) {
|
|
res.status(500).json({ error: (err as Error).message });
|
|
}
|
|
});
|
|
```
|
|
|
|
---
|
|
|
|
## Known Issues
|
|
|
|
### Hardcoded Username in ChatService
|
|
|
|
`ChatService` has `myUsername = 'Upset_Marsupial1006'` hardcoded in JavaScript evaluation scripts. This should be parameterized from config.
|
|
|
|
**Fix needed**: Pass username to `page.evaluate()` calls.
|
|
|
|
### Shadow DOM Fragility
|
|
|
|
Reddit frequently updates their `faceplate-*` components. Shadow DOM traversal code may break without warning.
|
|
|
|
**Mitigation**: Use `scripts/inspect-chat-dom.ts` to debug when selectors change.
|
|
|
|
### No Test Coverage
|
|
|
|
The project has no automated tests. Any changes should be manually tested.
|
|
|
|
**Recommendation**: Add tests for critical paths:
|
|
- StrategySelector algorithm
|
|
- ChatService shadow DOM parsing
|
|
- OpenRouterService model fallback
|
|
|
|
### Single-File Dashboard
|
|
|
|
The entire dashboard is in `public/index.html` (~2000+ lines). Making changes requires editing a single large file.
|
|
|
|
**Recommendation**: Consider migrating to a proper frontend framework for maintainability.
|