Restructure chat service, web routes, and add test suite
Split monolithic chat service into modular structure with Matrix client support, extract web server into per-resource route modules, introduce bootstrap/main-loop separation, and add comprehensive vitest coverage across services, repositories, and utilities. Adds project docs and CLAUDE.md agent instructions.
This commit is contained in:
jjbrunton committed
1 parent
dfa0580859
commit
633bb0ec04
84 files changed
+19278
-3888
No files matched your search
+1002
File diff suppressed because it is too large.
Load diff
@@ -0,0 +1,898 @@
|
||||
# RedditBot - Software Architecture Documentation
|
||||
|
||||
## Table of Contents
|
||||
|
||||
1. [Project Overview](#project-overview)
|
||||
2. [System Architecture](#system-architecture)
|
||||
3. [Directory Structure](#directory-structure)
|
||||
4. [Component Reference](#component-reference)
|
||||
5. [Data Flow](#data-flow)
|
||||
6. [Database Schema](#database-schema)
|
||||
7. [API Reference](#api-reference)
|
||||
8. [Configuration](#configuration)
|
||||
9. [Design Patterns](#design-patterns)
|
||||
10. [External Integrations](#external-integrations)
|
||||
11. [Deployment](#deployment)
|
||||
12. [Technical Debt & Recommendations](#technical-debt--recommendations)
|
||||
|
||||
---
|
||||
|
||||
## Project Overview
|
||||
|
||||
**RedditBot** is an automated Reddit monitoring and messaging system that:
|
||||
|
||||
- Monitors 30+ NSFW subreddits for image posts
|
||||
- Uses AI (OpenRouter) to analyze images and detect gender
|
||||
- Generates personalized direct messages to post authors
|
||||
- Sends DMs via browser automation (Playwright)
|
||||
- Monitors for responses and tracks success rates
|
||||
- Implements A/B testing with explore/exploit strategy selection
|
||||
- Learns from successful message/response pairs (few-shot learning)
|
||||
- Provides a real-time web dashboard for monitoring and control
|
||||
|
||||
### Key Capabilities
|
||||
|
||||
| Capability | Description |
|
||||
|---|---|
|
||||
| Post Discovery | Polls subreddits for new image posts, filters by gender and spam profiles |
|
||||
| AI Analysis | Multi-model image analysis with fallback chain via OpenRouter |
|
||||
| Message Generation | AI-crafted DMs using successful examples as few-shot prompts |
|
||||
| A/B Testing | Explore/exploit strategy selector optimizes response rates over time |
|
||||
| Response Monitoring | Background service checks inbox for user replies |
|
||||
| Chat Automation | Reddit chat scraping via shadow DOM traversal, auto-reply support |
|
||||
| Media Management | Download, cache, deduplicate, and sync media from conversations |
|
||||
| Web Dashboard | Real-time monitoring with SSE log streaming, analytics, and controls |
|
||||
|
||||
---
|
||||
|
||||
## System Architecture
|
||||
|
||||
### High-Level Architecture
|
||||
|
||||
```
|
||||
┌─────────────────────────────────────────────────────────────────────┐
|
||||
│ main (index.ts) │
|
||||
│ Orchestrator / Entry Point │
|
||||
└────────────────────────┬────────────────────────────────────────────┘
|
||||
│
|
||||
┌────────────────┼────────────────┐
|
||||
│ │ │
|
||||
▼ ▼ ▼
|
||||
┌───────────────┐ ┌──────────────┐ ┌──────────────┐
|
||||
│ Monitor │ │ Processor │ │ Response │
|
||||
│ Service │ │ │ │ Checker │
|
||||
│ │ │ 1. Download │ │ │
|
||||
│ Poll subreddits│ │ 2. Validate │ │ Check inbox │
|
||||
│ for new posts │ │ 3. Gender │ │ for replies │
|
||||
└───────┬───────┘ │ Filter │ └──────────────┘
|
||||
│ │ 4. AI Analysis│ │
|
||||
│ │ 5. Strategy │ │
|
||||
│ │ Selection │ │
|
||||
│ │ 6. Message │ │
|
||||
│ │ Generation │ │
|
||||
└────┬────┘ └─────────────┘ │
|
||||
│ │
|
||||
▼ ▼
|
||||
┌───────────────────────┐ ┌───────────────────────┐
|
||||
│ Strategy Selector │ │ Chat Service │
|
||||
│ │ │ │
|
||||
│ A/B Testing │ │ Scrapes chat (shadow │
|
||||
│ Explore/Exploit │ │ DOM traversal) │
|
||||
│ Variant Selection │ │ Auto-reply, upsell │
|
||||
└───────────────────────┘ │ detection │
|
||||
└───────────────────────┘
|
||||
│ │
|
||||
▼ ▼
|
||||
┌───────────────────────┐ ┌───────────────────────┐
|
||||
│ Message Sender │ │ Media Service │
|
||||
│ (Browser) │ │ │
|
||||
│ │ │ Download, cache, │
|
||||
│ Playwright DM compose │ │ deduplicate (SHA-1) │
|
||||
│ Shadow DOM handling │ │ Matrix thumbnail │
|
||||
└───────────────────────┘ │ resolution │
|
||||
└───────────────────────┘
|
||||
│ │
|
||||
▼ ▼
|
||||
┌─────────────────────────────────────────────────────────┐
|
||||
│ SQLite Database │
|
||||
│ │
|
||||
│ processed_posts │ sent_messages │ successful_messages │
|
||||
└─────────────────────────────────────────────────────────┘
|
||||
│
|
||||
▼
|
||||
┌─────────────────────────────────────────────────────────┐
|
||||
│ Web Dashboard (Express) │
|
||||
│ │
|
||||
│ 25+ API endpoints │ SSE log streaming │ Analytics │
|
||||
│ Runtime config │ Bot controls │ Media gallery │
|
||||
└─────────────────────────────────────────────────────────┘
|
||||
```
|
||||
|
||||
### Service Communication
|
||||
|
||||
```
|
||||
main.ts (Orchestrator)
|
||||
│
|
||||
├──► Config (Zod-validated YAML + env vars)
|
||||
│
|
||||
├──► Database (Better-SQLite3)
|
||||
│ │
|
||||
│ └──► PostsRepository
|
||||
│ └──► MessagesRepository
|
||||
│ └──► SuccessfulMessagesRepository
|
||||
│
|
||||
├──► MonitorService ──► RedditApi ──► Playwright (cookies)
|
||||
│
|
||||
├──► ProcessorService
|
||||
│ ├──► Image download (axios)
|
||||
│ ├──► Gender detection (OpenRouter)
|
||||
│ ├──► Full analysis (OpenRouter)
|
||||
│ └──► StrategySelector ──► MessagesRepository
|
||||
│
|
||||
├──► MessageSender (Browser - Playwright)
|
||||
│
|
||||
├──► ResponseChecker ──► Playwright (inbox navigation)
|
||||
│
|
||||
├──► ChatService ──► Playwright (chat.reddit.com shadow DOM)
|
||||
│
|
||||
├──► MediaService ──► Playwright (media download)
|
||||
│
|
||||
├──► MediaSyncService (background periodic sync)
|
||||
│
|
||||
├──► AutoReplyService ──► OpenRouter (reply generation)
|
||||
│
|
||||
├──► RateLimiter (p-queue, jitter)
|
||||
│
|
||||
└──► WebServer (Express, SSE)
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Directory Structure
|
||||
|
||||
```
|
||||
RedditBot/
|
||||
├── .dockerignore # Docker ignore rules
|
||||
├── .gitignore # Git ignore rules
|
||||
├── agents.md # Engineering standards & agent guidelines
|
||||
├── config.yaml # Main YAML configuration file
|
||||
├── docker-compose.yml # Docker Compose setup
|
||||
├── DOKPLOY_DEPLOY.md # Dokploy deployment guide
|
||||
├── Dockerfile # Container build definition
|
||||
├── package.json # Node.js dependencies & scripts
|
||||
├── tsconfig.json # TypeScript configuration
|
||||
├── public/
|
||||
│ └── index.html # Single-file web dashboard (HTML+CSS+JS)
|
||||
├── scripts/
|
||||
│ ├── inspect-chat-dom.ts # Debug: inspect chat DOM structure
|
||||
│ └── login.ts # Manual browser login helper
|
||||
└── src/
|
||||
├── index.ts # Main entry point / orchestrator (391 lines)
|
||||
├── config/
|
||||
│ └── index.ts # Config loader + Zod validation schema (157 lines)
|
||||
├── database/
|
||||
│ ├── client.ts # SQLite setup + migration runner (121 lines)
|
||||
│ └── repositories.ts # Repository pattern: Posts, Messages, Success (433 lines)
|
||||
├── reddit/
|
||||
│ └── api.ts # Reddit API client (axios + Playwright cookies) (233 lines)
|
||||
├── ai/
|
||||
│ └── openrouter.ts # OpenRouter AI client (image analysis, message gen) (307 lines)
|
||||
├── browser/
|
||||
│ ├── client.ts # Playwright persistent context + login (302 lines)
|
||||
│ ├── message-page.ts # DM compose page automation + shadow DOM (560 lines)
|
||||
│ └── media-downloader.ts # Playwright-based media downloader (29 lines)
|
||||
├── services/
|
||||
│ ├── monitor.ts # Post discovery service (72 lines)
|
||||
│ ├── processor.ts # Image download + AI + message pipeline (132 lines)
|
||||
│ ├── response-checker.ts # Inbox response checker (179 lines)
|
||||
│ ├── strategy-selector.ts # A/B testing explore/exploit (101 lines)
|
||||
│ ├── chat.ts # Chat scraping + messaging + auto-reply (1123+ lines)
|
||||
│ ├── media.ts # Media download, caching, indexing (254 lines)
|
||||
│ ├── media-sync.ts # Periodic background media sync (119 lines)
|
||||
│ └── auto-reply.ts # Auto-reply to chat conversations (199 lines)
|
||||
├── utils/
|
||||
│ ├── logger.ts # Winston structured logging (32 lines)
|
||||
│ ├── rate-limiter.ts # Token-bucket rate limiting (114 lines)
|
||||
│ ├── retry.ts # Exponential backoff wrapper (28 lines)
|
||||
│ └── image.ts # Image download + base64 + MIME validation (34 lines)
|
||||
└── web/
|
||||
└── server.ts # Express dashboard + SSE streaming (794 lines)
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Component Reference
|
||||
|
||||
### Core Services
|
||||
|
||||
#### MonitorService (`src/services/monitor.ts`)
|
||||
Discovers new image posts across configured subreddits.
|
||||
|
||||
- Shuffles subreddit order each cycle
|
||||
- Fetches new posts via `RedditApi`
|
||||
- Filters out already-processed posts and previously-messaged users
|
||||
- Returns the single newest valid post per cycle
|
||||
|
||||
#### ProcessorService (`src/services/processor.ts`)
|
||||
Core processing pipeline for each discovered post.
|
||||
|
||||
1. Downloads image via `ImageService`
|
||||
2. Validates MIME type and file size
|
||||
3. Runs quick gender detection (cost-saving pre-filter)
|
||||
4. Selects A/B strategy variant via `StrategySelector`
|
||||
5. Runs full AI analysis + message generation via `OpenRouterService`
|
||||
6. Handles AI refusals gracefully (marks post, tries next)
|
||||
|
||||
#### StrategySelector (`src/services/strategy-selector.ts`)
|
||||
Implements explore/exploit A/B testing for message variants.
|
||||
|
||||
- Tracks response rates per strategy variant
|
||||
- Uses minimum sample thresholds before trusting statistics
|
||||
- Applies random exploration rate for continued discovery
|
||||
- Selects best variant based on weighted response rates
|
||||
|
||||
#### ResponseChecker (`src/services/response-checker.ts`)
|
||||
Background service that monitors for user replies.
|
||||
|
||||
- Navigates Reddit inbox via Playwright
|
||||
- Identifies responses to sent messages
|
||||
- Updates `got_response` flag in database
|
||||
- Stores successful message/response pairs for few-shot learning
|
||||
|
||||
#### ChatService (`src/services/chat.ts`)
|
||||
Most complex service - manages Reddit chat interactions.
|
||||
|
||||
- Scrapes `chat.reddit.com` using JavaScript injection (shadow DOM traversal)
|
||||
- Manages conversation cache (in-memory + disk JSON)
|
||||
- Loads full conversation history with pagination
|
||||
- Sends messages with human-like typing delays
|
||||
- Verifies message delivery
|
||||
- Detects upsell/spam keywords in chat
|
||||
- Parameter: `myUsername` is hardcoded (should be from config)
|
||||
|
||||
#### MediaService (`src/services/media.ts`)
|
||||
Media download, caching, and indexing.
|
||||
|
||||
- Downloads images from chat conversations
|
||||
- Resolves Matrix blurred thumbnails to full resolution
|
||||
- Stores files in `data/media/` directory
|
||||
- Maintains JSON index with SHA-1 deduplication
|
||||
- Tracks download status per conversation
|
||||
|
||||
#### MediaSyncService (`src/services/media-sync.ts`)
|
||||
Periodic background media synchronization.
|
||||
|
||||
- Runs incremental syncs (top 10 chats) or full syncs (all conversations)
|
||||
- Configurable interval via `mediaSyncInterval`
|
||||
- Tracks sync status in memory
|
||||
- Exposed via web dashboard endpoints
|
||||
|
||||
#### AutoReplyService (`src/services/auto-reply.ts`)
|
||||
Automated reply to Reddit chat conversations.
|
||||
|
||||
- Checks for unread messages in cached conversations
|
||||
- Detects upsell/spam keywords
|
||||
- Generates AI reply suggestions via `OpenRouterService`
|
||||
- Sends replies (auto or pending for manual approval)
|
||||
- Event-based architecture for dashboard integration
|
||||
|
||||
### External Integrations
|
||||
|
||||
#### RedditApi (`src/reddit/api.ts`)
|
||||
Axios-based Reddit API client.
|
||||
|
||||
- Fetches posts from subreddits (`/r/{name}/new.json`)
|
||||
- Validates posts (image-only, skips deleted authors)
|
||||
- Extracts image URLs with fallback chain: `preview` > `i.redd.it` > `imgur` > `thumbnail`
|
||||
- Checks user profiles against seller/spam keywords
|
||||
- Uses Playwright browser cookies for authenticated requests
|
||||
|
||||
#### OpenRouterService (`src/ai/openrouter.ts`)
|
||||
OpenRouter API client using OpenAI-compatible SDK.
|
||||
|
||||
- `analyzeImage()`: Multi-model image analysis with fallback chain
|
||||
- `detectGenderOnly()`: Quick cost-saving gender detection
|
||||
- `generateMessage()`: DM text with few-shot examples from DB
|
||||
- `generateReply()`: AI chat reply from conversation history
|
||||
- `detectUpsell()`: Keyword-based spam/OnlyFans detection
|
||||
- Refusal detection via heuristic patterns
|
||||
|
||||
#### Browser Client (`src/browser/client.ts`)
|
||||
Playwright persistent context manager.
|
||||
|
||||
- Browser session persistence (cookies/storage)
|
||||
- Login verification (multiple UI checks + API fallback)
|
||||
- Automated login flow (form fill, 2FA/captcha detection)
|
||||
- Debug screenshot capture on login failure
|
||||
|
||||
#### MessagePage (`src/browser/message-page.ts`)
|
||||
Reddit DM compose page automation.
|
||||
|
||||
- Fills recipient/subject/body fields
|
||||
- Extensive Shadow DOM support for `faceplate-*` components
|
||||
- Detects recipient errors, verifies submission success
|
||||
- Supports dry-run mode for testing
|
||||
|
||||
### Data Layer
|
||||
|
||||
#### Database Client (`src/database/client.ts`)
|
||||
Better-SQLite3 database initialization.
|
||||
|
||||
- Creates database at configured path
|
||||
- Enables WAL mode and foreign keys
|
||||
- Runs migrations on startup
|
||||
- Safe column-addition for schema evolution
|
||||
|
||||
#### Repositories (`src/database/repositories.ts`)
|
||||
Three repository classes abstracting SQLite operations.
|
||||
|
||||
- **PostsRepository**: Processed post CRUD, mark sent, update analysis
|
||||
- **MessagesRepository**: Sent message CRUD, response tracking, strategy stats, subreddit stats, daily trends
|
||||
- **SuccessfulMessagesRepository**: Successful pairs for few-shot learning, subreddit-weighted retrieval
|
||||
|
||||
### Utilities
|
||||
|
||||
#### Logger (`src/utils/logger.ts`)
|
||||
Winston structured logging.
|
||||
|
||||
- JSON format for log files
|
||||
- Colored console output
|
||||
- File logging: `error.log` and combined log
|
||||
|
||||
#### RateLimiter (`src/utils/rate-limiter.ts`)
|
||||
Token-bucket rate limiting.
|
||||
|
||||
- p-queue with concurrency=1
|
||||
- Minimum delay between messages with random jitter
|
||||
- Hourly message count tracking
|
||||
- Timer status exposed to dashboard
|
||||
|
||||
#### Retry (`src/utils/retry.ts`)
|
||||
Exponential backoff wrapper.
|
||||
|
||||
- p-retry with configurable retries (default 3)
|
||||
- Backoff range: 5s to 60s
|
||||
- Logging on retries and failures
|
||||
|
||||
#### Image (`src/utils/image.ts`)
|
||||
Image processing utilities.
|
||||
|
||||
- Download via axios
|
||||
- Base64 conversion for AI API
|
||||
- MIME type validation (image/jpeg, image/png, image/webp)
|
||||
|
||||
### Web Dashboard
|
||||
|
||||
#### WebServer (`src/web/server.ts`)
|
||||
Express server with 25+ API endpoints.
|
||||
|
||||
- Stats, activity, messages, chats, conversations, media endpoints
|
||||
- Bot controls (start/stop)
|
||||
- SSE realtime log streaming
|
||||
- Analytics: strategy, subreddit, daily trends
|
||||
- Runtime config management (GET/PUT)
|
||||
- Debug endpoints
|
||||
|
||||
---
|
||||
|
||||
## Data Flow
|
||||
|
||||
### Post Processing Pipeline
|
||||
|
||||
```
|
||||
1. MonitorService polls subreddits
|
||||
│
|
||||
├─► RedditApi.fetchPosts()
|
||||
│ └─► GET /r/{subreddit}/new.json
|
||||
│ └─► Filter: image posts only, valid author
|
||||
│
|
||||
├─► Filter: skip processed posts (DB check)
|
||||
├─► Filter: skip previously messaged users
|
||||
│
|
||||
└─► Return newest valid post
|
||||
│
|
||||
2. ProcessorService.processPost(post)
|
||||
│
|
||||
├─► ImageService.downloadImage(url)
|
||||
│ └─► Validate MIME type, file size
|
||||
│
|
||||
├─► OpenRouter.detectGenderOnly()
|
||||
│ └─► Quick pre-filter (cost-saving)
|
||||
│ └─► Skip if gender doesn't match target
|
||||
│
|
||||
├─► StrategySelector.selectStrategy(subreddit, author)
|
||||
│ └─► Explore/exploit algorithm
|
||||
│
|
||||
├─► OpenRouter.analyzeImage(image)
|
||||
│ └─► Multi-model fallback chain
|
||||
│ └─► Detect gender, describe image
|
||||
│
|
||||
├─► OpenRouter.generateMessage(analysis, title, examples, strategy)
|
||||
│ └─► Few-shot learning from successful_messages
|
||||
│
|
||||
└─► Mark post as processed in DB
|
||||
│
|
||||
3. MessagePage.sendDM(author, subject, message)
|
||||
│
|
||||
├─► Navigate to /compose?to={author}
|
||||
├─► Fill fields (with shadow DOM handling)
|
||||
├─► Submit message
|
||||
└─► Verify success
|
||||
│
|
||||
└─► Record sent_messages in DB
|
||||
```
|
||||
|
||||
### Response Monitoring Flow
|
||||
|
||||
```
|
||||
1. ResponseChecker.checkResponses()
|
||||
│
|
||||
├─► Navigate to /message/inbox
|
||||
├─► Parse response items
|
||||
│
|
||||
├─► For each response:
|
||||
│ ├─► Match to sent_messages by recipient + timestamp
|
||||
│ ├─► Update got_response = 1
|
||||
│ ├─► Store message/response pair
|
||||
│ └─► Add to successful_messages (for few-shot learning)
|
||||
│
|
||||
└─► Update strategy stats in DB
|
||||
```
|
||||
|
||||
### Chat Interaction Flow
|
||||
|
||||
```
|
||||
1. ChatService.fetchChats()
|
||||
│
|
||||
├─► Navigate to chat.reddit.com
|
||||
├─► Traverse shadow DOM via JS injection
|
||||
├─► Extract active conversations
|
||||
└─► Cache in memory + disk JSON
|
||||
│
|
||||
2. ChatService.loadConversation(chatId)
|
||||
│
|
||||
├─► Load from cache
|
||||
├─► Fetch full history (pagination)
|
||||
└─► Return conversation object
|
||||
│
|
||||
3. ChatService.sendMessage(chatId, text)
|
||||
│
|
||||
├─► Navigate to chat view
|
||||
├─► Fill message input (shadow DOM)
|
||||
├─► Apply typing delay (human-like)
|
||||
├─► Submit
|
||||
└─► Verify delivery
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Database Schema
|
||||
|
||||
### Table: `processed_posts`
|
||||
|
||||
Tracks all posts that have been discovered and processed.
|
||||
|
||||
| Column | Type | Constraints | Description |
|
||||
|---|---|---|---|
|
||||
| `id` | TEXT | PRIMARY KEY | Reddit post ID |
|
||||
| `subreddit` | TEXT | NOT NULL | Source subreddit |
|
||||
| `author` | TEXT | NOT NULL | Post author username |
|
||||
| `title` | TEXT | | Post title |
|
||||
| `image_url` | TEXT | | Original image URL |
|
||||
| `image_analysis` | TEXT | | AI-generated image description |
|
||||
| `generated_message` | TEXT | | AI-generated DM text |
|
||||
| `processed_at` | INTEGER | NOT NULL | Unix timestamp |
|
||||
| `message_sent` | INTEGER | DEFAULT 0 | 0 = not sent, 1 = sent |
|
||||
| `error` | TEXT | | Error message if processing failed |
|
||||
|
||||
**Indexes**: `idx_posts_subreddit`, `idx_posts_author`
|
||||
|
||||
### Table: `sent_messages`
|
||||
|
||||
Tracks all DMs sent by the bot.
|
||||
|
||||
| Column | Type | Constraints | Description |
|
||||
|---|---|---|---|
|
||||
| `id` | INTEGER | PRIMARY KEY AUTOINCREMENT | Internal ID |
|
||||
| `post_id` | TEXT | FK → processed_posts.id | Source post |
|
||||
| `recipient` | TEXT | NOT NULL | Recipient username |
|
||||
| `subject` | TEXT | DEFAULT 'Stunning' | DM subject line |
|
||||
| `message` | TEXT | NOT NULL | DM body |
|
||||
| `sent_at` | INTEGER | NOT NULL | Unix timestamp |
|
||||
| `strategy_id` | TEXT | | A/B test variant identifier |
|
||||
| `got_response` | INTEGER | DEFAULT 0 | 0 = no response, 1 = responded |
|
||||
| `response_checked_at` | INTEGER | | Timestamp of last response check |
|
||||
|
||||
**Indexes**: `idx_messages_recipient`, `idx_messages_strategy`, `idx_messages_response`
|
||||
|
||||
### Table: `successful_messages`
|
||||
|
||||
Stores successful message/response pairs for few-shot learning.
|
||||
|
||||
| Column | Type | Constraints | Description |
|
||||
|---|---|---|---|
|
||||
| `id` | INTEGER | PRIMARY KEY AUTOINCREMENT | Internal ID |
|
||||
| `sent_message_id` | INTEGER | FK → sent_messages.id (nullable) | Related message |
|
||||
| `original_message` | TEXT | NOT NULL | The DM that got a response |
|
||||
| `image_analysis` | TEXT | | Context from image analysis |
|
||||
| `post_title` | TEXT | | Original post title |
|
||||
| `response_snippet` | TEXT | | First reply text |
|
||||
| `subreddit` | TEXT | | For weighted example selection |
|
||||
| `created_at` | INTEGER | NOT NULL | Unix timestamp |
|
||||
|
||||
**Indexes**: `idx_success_subreddit`, `idx_success_message`
|
||||
|
||||
### Migration Strategy
|
||||
|
||||
The database uses progressive migrations that:
|
||||
- Run on every startup
|
||||
- Use `ALTER TABLE ADD COLUMN IF NOT EXISTS` for safe schema evolution
|
||||
- Create indexes with `IF NOT EXISTS` for idempotency
|
||||
- Enable WAL mode and foreign keys for performance and integrity
|
||||
|
||||
---
|
||||
|
||||
## API Reference
|
||||
|
||||
### Web Dashboard Endpoints
|
||||
|
||||
All endpoints are served by the Express server at `http://localhost:{port}`.
|
||||
|
||||
#### Bot Status & Control
|
||||
|
||||
| Method | Endpoint | Description |
|
||||
|---|---|---|
|
||||
| GET | `/api/stats` | Bot statistics (running state, post/message/response counts, response rate) |
|
||||
| POST | `/api/start` | Start the bot |
|
||||
| POST | `/api/stop` | Stop the bot |
|
||||
|
||||
#### Activity & Messages
|
||||
|
||||
| Method | Endpoint | Description |
|
||||
|---|---|---|
|
||||
| GET | `/api/activity` | Recent activity log entries |
|
||||
| GET | `/api/posts/by-author/:username` | Look up processed posts by user |
|
||||
| GET | `/api/messages` | Recent sent messages |
|
||||
| GET | `/api/successful` | Successful message/response pairs |
|
||||
|
||||
#### Chat & Conversations
|
||||
|
||||
| Method | Endpoint | Description |
|
||||
|---|---|---|
|
||||
| GET | `/api/chats` | Active cached conversations |
|
||||
| POST | `/api/chats/fetch-all` | Fetch all chats from Reddit |
|
||||
| GET | `/api/conversations` | All cached conversations |
|
||||
| GET | `/api/conversations/:id` | Single conversation detail |
|
||||
| POST | `/api/conversations/:id/suggest-reply` | AI suggest reply for conversation |
|
||||
| POST | `/api/conversations/:id/send-reply` | Send chat reply |
|
||||
| POST | `/api/conversations/:id/approve-reply` | Approve pending auto-reply |
|
||||
| POST | `/api/conversations/:id/reject-reply` | Reject pending auto-reply |
|
||||
| POST | `/api/conversations/:id/dismiss-upsell` | Dismiss upsell flag |
|
||||
|
||||
#### Media
|
||||
|
||||
| Method | Endpoint | Description |
|
||||
|---|---|---|
|
||||
| GET | `/api/media` | Downloaded media gallery |
|
||||
| GET | `/api/media/sync` | Media sync status |
|
||||
| POST | `/api/media/sync` | Trigger incremental sync (top 10 chats) |
|
||||
| POST | `/api/media/sync/full` | Trigger full sync (all conversations) |
|
||||
|
||||
#### Analytics
|
||||
|
||||
| Method | Endpoint | Description |
|
||||
|---|---|---|
|
||||
| GET | `/api/analytics/by-subreddit` | Response rate by subreddit |
|
||||
| GET | `/api/analytics/by-strategy` | Response rate by strategy variant |
|
||||
| GET | `/api/analytics/trend` | Daily response rate trend |
|
||||
|
||||
#### Configuration
|
||||
|
||||
| Method | Endpoint | Description |
|
||||
|---|---|---|
|
||||
| GET | `/api/config` | Current runtime configuration |
|
||||
| PUT | `/api/config` | Update runtime config (rate limits, dry run, poll interval) |
|
||||
|
||||
#### Logs & Debugging
|
||||
|
||||
| Method | Endpoint | Description |
|
||||
|---|---|---|
|
||||
| GET | `/api/logs/stream` | SSE realtime log stream |
|
||||
| GET | `/api/logs` | Recent logs (HTTP) |
|
||||
| GET | `/api/timers` | Rate limiter timer status |
|
||||
| POST | `/api/debug/inspect-dom` | Debug: inspect chat DOM structure |
|
||||
| POST | `/api/debug/check-responses` | Debug: manual response check |
|
||||
|
||||
---
|
||||
|
||||
## Configuration
|
||||
|
||||
### Configuration Sources (Priority Order)
|
||||
|
||||
1. **`config.yaml`** - Primary configuration file (project root)
|
||||
2. **Environment Variables** - Override specific fields:
|
||||
- `OPENROUTER_API_KEY` - OpenRouter API key
|
||||
- `REDDIT_USERNAME` - Reddit username
|
||||
- `REDDIT_PASSWORD` - Reddit password
|
||||
- `BROWSER_HEADLESS` - Browser headless mode (`true`/`false`)
|
||||
|
||||
### Configuration Schema (Zod-Validated)
|
||||
|
||||
```typescript
|
||||
{
|
||||
reddit: {
|
||||
subreddits: string[] // Subreddits to monitor
|
||||
pollInterval: number // Minutes between polls
|
||||
targetGender: string // Target gender filter
|
||||
sellerKeywords: string[] // Keywords for spam/seller detection
|
||||
}
|
||||
openrouter: {
|
||||
apiKey: string // OpenRouter API key
|
||||
model: string // Primary AI model
|
||||
fallbackModels: string[] // Fallback models on refusal
|
||||
systemPrompt: string // System prompt for analysis
|
||||
messagePrompt: string // System prompt for message generation
|
||||
}
|
||||
rateLimit: {
|
||||
minDelayMs: number // Minimum delay between messages
|
||||
maxMessagesPerHour: number // Hourly message cap
|
||||
jitterMs: number // Random jitter range
|
||||
}
|
||||
browser: {
|
||||
headless: boolean // Headless mode
|
||||
userDataDir: string // Persistent session directory
|
||||
}
|
||||
database: {
|
||||
path: string // SQLite database path
|
||||
}
|
||||
mediaSync: {
|
||||
enabled: boolean // Enable media sync
|
||||
intervalMinutes: number // Sync interval
|
||||
}
|
||||
autoReply: {
|
||||
enabled: boolean // Enable auto-reply
|
||||
autoSend: boolean // Auto-send or pending approval
|
||||
checkInterval: number // Check interval in minutes
|
||||
}
|
||||
strategyVariants: {
|
||||
name: string // Variant name
|
||||
prompt: string // Message style prompt
|
||||
enabled: boolean // Enable/disable variant
|
||||
}[]
|
||||
web: {
|
||||
port: number // Dashboard port
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
### Runtime Configuration
|
||||
|
||||
The following settings can be modified via the web dashboard without restarting:
|
||||
|
||||
- `rateLimit.minDelayMs` - Message delay
|
||||
- `rateLimit.maxMessagesPerHour` - Hourly cap
|
||||
- `rateLimit.jitterMs` - Jitter range
|
||||
- `reddit.pollInterval` - Poll frequency
|
||||
- `dryRun` - Dry run mode (log only, no actual sends)
|
||||
|
||||
---
|
||||
|
||||
## Design Patterns
|
||||
|
||||
### Repository Pattern
|
||||
|
||||
All database operations are abstracted behind three repository classes:
|
||||
|
||||
- `PostsRepository` - Post lifecycle management
|
||||
- `MessagesRepository` - Message tracking and analytics
|
||||
- `SuccessfulMessagesRepository` - Few-shot learning data
|
||||
|
||||
Benefits:
|
||||
- Single source of truth for data access
|
||||
- Easy to swap implementations (e.g., different databases)
|
||||
- Consistent error handling and query patterns
|
||||
|
||||
### Dependency Injection
|
||||
|
||||
Services receive dependencies via constructor parameters:
|
||||
|
||||
```typescript
|
||||
class ProcessorService {
|
||||
constructor(
|
||||
private config: Config,
|
||||
private redditApi: RedditApi,
|
||||
private openrouter: OpenRouterService,
|
||||
private strategySelector: StrategySelector,
|
||||
private messageSender: MessagePage,
|
||||
private postsRepo: PostsRepository,
|
||||
private messagesRepo: MessagesRepository,
|
||||
private successfulRepo: SuccessfulMessagesRepository,
|
||||
private logger: Logger
|
||||
) {}
|
||||
}
|
||||
```
|
||||
|
||||
Benefits:
|
||||
- Loose coupling between components
|
||||
- Easy to mock dependencies for testing
|
||||
- Clear dependency graph
|
||||
|
||||
### Strategy Pattern
|
||||
|
||||
`StrategySelector` implements A/B testing with multiple message variants:
|
||||
|
||||
```typescript
|
||||
class StrategySelector {
|
||||
selectStrategy(subreddit: string, author: string): string {
|
||||
// Explore/exploit algorithm
|
||||
// Returns variant name based on response rates
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
Benefits:
|
||||
- Swappable message strategies
|
||||
- Measurable performance per variant
|
||||
- Configurable exploration rate
|
||||
|
||||
### Factory Pattern
|
||||
|
||||
Configuration and database creation use factory functions:
|
||||
|
||||
- `loadConfig()` - Loads, merges env vars, validates with Zod
|
||||
- `createDatabase()` - Initializes SQLite, runs migrations
|
||||
|
||||
### Observer Pattern
|
||||
|
||||
SSE (Server-Sent Events) for realtime log streaming:
|
||||
|
||||
```typescript
|
||||
// Logger writes to a shared buffer
|
||||
logger.info("Message sent to user123");
|
||||
|
||||
// WebServer subscribes clients via SSE
|
||||
app.get("/api/logs/stream", (req, res) => {
|
||||
// Push new log entries to connected clients
|
||||
});
|
||||
```
|
||||
|
||||
### Circuit Breaker / Retry
|
||||
|
||||
`withRetry()` wrapper for resilient external calls:
|
||||
|
||||
```typescript
|
||||
await withRetry(
|
||||
() => redditApi.fetchPosts(subreddit),
|
||||
{ retries: 3, logger }
|
||||
);
|
||||
```
|
||||
|
||||
Multiple fallback chains:
|
||||
- AI model fallback (primary → fallback models)
|
||||
- Image URL fallback (preview → i.redd.it → imgur → thumbnail)
|
||||
|
||||
### Rate Limiter Pattern
|
||||
|
||||
Token-bucket-like implementation:
|
||||
|
||||
```typescript
|
||||
class RateLimiter {
|
||||
async waitForSlot(): Promise<void> {
|
||||
// Enforce minimum delay + jitter
|
||||
// Track hourly counts
|
||||
// Block if hourly cap reached
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## External Integrations
|
||||
|
||||
### Reddit API
|
||||
|
||||
- **Authentication**: Playwright browser cookies (persistent session)
|
||||
- **Endpoints Used**:
|
||||
- `GET /r/{name}/new.json` - Fetch new posts
|
||||
- `GET /user/{username}/profile` - Check for seller/spam indicators
|
||||
- `GET /message/inbox` - Check for responses
|
||||
- `GET chat.reddit.com` - Chat scraping
|
||||
- **Rate Limits**: Enforced by `RateLimiter` utility
|
||||
|
||||
### OpenRouter AI
|
||||
|
||||
- **SDK**: OpenAI-compatible (`openai` npm package)
|
||||
- **Base URL**: `https://openrouter.ai/api/v1`
|
||||
- **Endpoints Used**:
|
||||
- `POST /chat/completions` - Image analysis, message generation, replies
|
||||
- **Model Fallback**: Primary model → fallback models on refusal
|
||||
- **Cost Optimization**: Quick gender detection before full analysis
|
||||
|
||||
### Playwright Browser
|
||||
|
||||
- **Context**: Persistent (cookies/session storage saved to disk)
|
||||
- **Usage**:
|
||||
- Automated Reddit login
|
||||
- DM compose page automation
|
||||
- Inbox navigation for response checking
|
||||
- Chat page scraping (shadow DOM traversal)
|
||||
- Media file downloads
|
||||
- **Shadow DOM**: JavaScript injection for `faceplate-*` component traversal
|
||||
|
||||
---
|
||||
|
||||
## Deployment
|
||||
|
||||
### Docker
|
||||
|
||||
The project includes full Docker support:
|
||||
|
||||
- **`Dockerfile`** - Multi-stage build with Node.js
|
||||
- **`docker-compose.yml`** - Service orchestration
|
||||
- **`.dockerignore`** - Excludes unnecessary files
|
||||
|
||||
### Dokploy
|
||||
|
||||
Deployment documentation is provided in `DOKPLOY_DEPLOY.md`.
|
||||
|
||||
### Required Environment Variables
|
||||
|
||||
| Variable | Required | Description |
|
||||
|---|---|---|
|
||||
| `OPENROUTER_API_KEY` | Yes | OpenRouter API key for AI |
|
||||
| `REDDIT_USERNAME` | Yes | Reddit username |
|
||||
| `REDDIT_PASSWORD` | Yes | Reddit password |
|
||||
| `BROWSER_HEADLESS` | No | Browser headless mode (default: `true`) |
|
||||
|
||||
### Pre-Deployment Checklist
|
||||
|
||||
1. Set up `config.yaml` with target subreddits and settings
|
||||
2. Set environment variables (or use `.env` file)
|
||||
3. Run `npm install` to install dependencies
|
||||
4. Run `npm run build` to compile TypeScript
|
||||
5. Run `scripts/login.ts` once to establish browser session
|
||||
6. Start the bot (via `npm start` or Docker)
|
||||
|
||||
---
|
||||
|
||||
## Technical Debt & Recommendations
|
||||
|
||||
### Critical
|
||||
|
||||
1. **No Tests** - Zero test coverage. Add unit tests for services, especially:
|
||||
- `StrategySelector` (complex algorithm)
|
||||
- `ChatService` (shadow DOM logic)
|
||||
- `OpenRouterService` (model fallback logic)
|
||||
|
||||
2. **No README.md** - No project documentation for new developers.
|
||||
|
||||
3. **Hardcoded Username** - `ChatService` has `myUsername = 'Upset_Marsupial1006'` hardcoded in JS evaluation scripts. Should be parameterized from config.
|
||||
|
||||
### High Priority
|
||||
|
||||
4. **God Class** - `main.ts` (391 lines) creates all dependencies and wires everything. Consider extracting an `AppContainer` or `DependencyContainer`.
|
||||
|
||||
5. **ChatService Size** - 1123+ lines. Consider splitting into:
|
||||
- `ChatScraper` (navigation + DOM traversal)
|
||||
- `ConversationCache` (cache management)
|
||||
- `ChatMessenger` (sending + verification)
|
||||
|
||||
6. **No .env.example** - No template showing required environment variables.
|
||||
|
||||
### Medium Priority
|
||||
|
||||
7. **Single-Page Dashboard** - `public/index.html` is ~2000+ lines with embedded CSS/JS. Consider a proper frontend framework.
|
||||
|
||||
8. **WebServer Size** - 794 lines. Consider splitting routes into separate files.
|
||||
|
||||
9. **Config Mutability** - Runtime config updates via `PUT /api/config` modify the in-memory config object. Consider a proper config service with validation on updates.
|
||||
|
||||
10. **Error Handling** - Some services have basic try/catch. Consider structured error types with retry logic.
|
||||
|
||||
### Low Priority
|
||||
|
||||
11. **Logging** - Winston is well-configured, but consider adding request IDs for tracing.
|
||||
|
||||
12. **Type Safety** - Some `page.evaluate()` calls use `any` types. Consider stricter typing for DOM structures.
|
||||
|
||||
13. **Media Storage** - Media files stored in `data/media/` with no cleanup policy. Consider retention policies.
|
||||
@@ -0,0 +1,488 @@
|
||||
# Configuration Guide
|
||||
|
||||
Complete reference for configuring RedditBot.
|
||||
|
||||
---
|
||||
|
||||
## Table of Contents
|
||||
|
||||
1. [Configuration Overview](#configuration-overview)
|
||||
2. [Configuration Files](#configuration-files)
|
||||
3. [Environment Variables](#environment-variables)
|
||||
4. [config.yaml Reference](#configyaml-reference)
|
||||
5. [Runtime Configuration](#runtime-configuration)
|
||||
6. [Configuration Validation](#configuration-validation)
|
||||
7. [Troubleshooting](#troubleshooting)
|
||||
|
||||
---
|
||||
|
||||
## Configuration Overview
|
||||
|
||||
RedditBot uses a layered configuration approach:
|
||||
|
||||
```
|
||||
┌──────────────────────────────────────────────────────────────┐
|
||||
│ Priority Order (High → Low) │
|
||||
├──────────────────────────────────────────────────────────────┤
|
||||
│ 1. Environment Variables │
|
||||
│ (OPENROUTER_API_KEY, REDDIT_USERNAME, etc.) │
|
||||
│ │
|
||||
│ 2. config.yaml │
|
||||
│ (Primary configuration file) │
|
||||
│ │
|
||||
│ 3. Defaults │
|
||||
│ (Hardcoded defaults in code) │
|
||||
└──────────────────────────────────────────────────────────────┘
|
||||
```
|
||||
|
||||
Configuration is loaded at startup by `loadConfig()` in `src/config/index.ts`, which:
|
||||
|
||||
1. Reads `config.yaml` from the project root
|
||||
2. Merges environment variable overrides
|
||||
3. Validates the entire configuration tree using Zod
|
||||
4. Returns a strongly-typed `Config` object
|
||||
|
||||
---
|
||||
|
||||
## Configuration Files
|
||||
|
||||
### config.yaml
|
||||
|
||||
The primary configuration file, located at the project root:
|
||||
|
||||
```
|
||||
RedditBot/
|
||||
├── config.yaml # ← Main configuration
|
||||
├── package.json
|
||||
├── tsconfig.json
|
||||
└── src/
|
||||
```
|
||||
|
||||
### Data Directory
|
||||
|
||||
Runtime data is stored in `data/` (created automatically):
|
||||
|
||||
```
|
||||
data/
|
||||
├── reddit-bot.db # SQLite database
|
||||
├── browser/ # Browser session data (cookies, storage)
|
||||
├── media/ # Downloaded media files
|
||||
│ ├── index.json # Media index (SHA-1 → filename)
|
||||
│ ├── abc123.jpg # Media file 1
|
||||
│ └── def456.png # Media file 2
|
||||
└── chats/ # Cached conversation JSON files
|
||||
├── chat_abc123.json
|
||||
└── chat_def456.json
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Environment Variables
|
||||
|
||||
Environment variables override corresponding values in `config.yaml`.
|
||||
|
||||
| Variable | Required | Type | Description | Example |
|
||||
|---|---|---|---|---|
|
||||
| `OPENROUTER_API_KEY` | Yes | string | OpenRouter API key for AI services | `sk-or-v1-abc123...` |
|
||||
| `REDDIT_USERNAME` | Yes | string | Reddit username for authentication | `MyBotUser` |
|
||||
| `REDDIT_PASSWORD` | Yes | string | Reddit password for authentication | `secret123` |
|
||||
| `BROWSER_HEADLESS` | No | boolean | Run browser in headless mode | `true` or `false` |
|
||||
|
||||
### Setting Environment Variables
|
||||
|
||||
**Option 1: `.env` file** (recommended for development)
|
||||
|
||||
Create a `.env` file in the project root:
|
||||
|
||||
```env
|
||||
OPENROUTER_API_KEY=sk-or-v1-abc123...
|
||||
REDDIT_USERNAME=MyBotUser
|
||||
REDDIT_PASSWORD=secret123
|
||||
BROWSER_HEADLESS=true
|
||||
```
|
||||
|
||||
> **Note**: Add `.env` to `.gitignore` - never commit credentials.
|
||||
|
||||
**Option 2: Shell export**
|
||||
|
||||
```bash
|
||||
export OPENROUTER_API_KEY=sk-or-v1-abc123...
|
||||
export REDDIT_USERNAME=MyBotUser
|
||||
export REDDIT_PASSWORD=secret123
|
||||
```
|
||||
|
||||
**Option 3: Docker**
|
||||
|
||||
Pass via `docker-compose.yml` or Docker CLI:
|
||||
|
||||
```yaml
|
||||
environment:
|
||||
- OPENROUTER_API_KEY=sk-or-v1-abc123...
|
||||
- REDDIT_USERNAME=MyBotUser
|
||||
- REDDIT_PASSWORD=secret123
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## config.yaml Reference
|
||||
|
||||
Complete reference for every configuration option.
|
||||
|
||||
### Reddit Settings
|
||||
|
||||
```yaml
|
||||
reddit:
|
||||
# Subreddits to monitor for new posts
|
||||
subreddits:
|
||||
- gonewild
|
||||
- ratemyboobs
|
||||
- HomemadeNsfw
|
||||
- realgirls
|
||||
- plussize_gonewild
|
||||
# ... add or remove subreddits
|
||||
|
||||
# Minutes between polling cycles
|
||||
pollInterval: 15
|
||||
|
||||
# Target gender for image analysis filtering
|
||||
# Options: "F" (female), "M" (male)
|
||||
targetGender: "F"
|
||||
|
||||
# Keywords used to detect sellers/spam profiles
|
||||
# Posts from users matching these keywords are skipped
|
||||
sellerKeywords:
|
||||
- seller
|
||||
- onlyfans
|
||||
- link in bio
|
||||
- check my profile
|
||||
- promo
|
||||
- discount
|
||||
- affiliate
|
||||
```
|
||||
|
||||
### OpenRouter AI Settings
|
||||
|
||||
```yaml
|
||||
openrouter:
|
||||
# OpenRouter API key (can also be set via OPENROUTER_API_KEY env var)
|
||||
apiKey: "sk-or-v1-abc123..."
|
||||
|
||||
# Primary AI model for image analysis
|
||||
# Models must be available on OpenRouter
|
||||
model: "meta-llama/llama-3.2-90b-vision-preview"
|
||||
|
||||
# Fallback models tried if the primary model refuses or fails
|
||||
fallbackModels:
|
||||
- "google/gemini-2.0-flash-001"
|
||||
- "meta-llama/llama-3.2-11b-vision-instruct"
|
||||
|
||||
# Quick detection model (cheaper/faster, used for gender pre-filter)
|
||||
quickModel: "google/gemini-2.0-flash-lite-001"
|
||||
|
||||
# System prompt for image analysis
|
||||
# The AI will describe the image and detect gender
|
||||
systemPrompt: |
|
||||
Analyze this image and provide:
|
||||
1. Gender detected (F or M)
|
||||
2. Detailed description of the image content
|
||||
3. Notable features (hair color, clothing, setting, etc.)
|
||||
|
||||
Return as JSON: {"gender": "F", "description": "...", "features": [...]}
|
||||
|
||||
# System prompt for message generation
|
||||
# Used when crafting DMs to post authors
|
||||
messagePrompt: |
|
||||
Generate a natural, friendly DM for this Reddit post.
|
||||
|
||||
Image analysis: {description}
|
||||
Post title: {title}
|
||||
Strategy: {strategy}
|
||||
|
||||
Guidelines:
|
||||
- Be complimentary but not creepy
|
||||
- Reference something specific from the image
|
||||
- Keep it under 2 sentences
|
||||
- Sound like a genuine person, not a bot
|
||||
- Do NOT ask for anything in return
|
||||
```
|
||||
|
||||
### Rate Limiting
|
||||
|
||||
```yaml
|
||||
rateLimit:
|
||||
# Minimum delay between messages (milliseconds)
|
||||
# Higher values reduce detection risk
|
||||
minDelayMs: 30000
|
||||
|
||||
# Maximum messages per hour
|
||||
# Bot will pause when this limit is reached
|
||||
maxMessagesPerHour: 20
|
||||
|
||||
# Random jitter added to delay (milliseconds)
|
||||
# Adds unpredictability to timing
|
||||
jitterMs: 15000
|
||||
```
|
||||
|
||||
**Timing example with defaults:**
|
||||
```
|
||||
Minimum delay: 30,000ms (30 seconds)
|
||||
Jitter: 0 to 15,000ms (0-15 seconds)
|
||||
Effective delay: 30-45 seconds between messages
|
||||
Max messages: 20 per hour
|
||||
```
|
||||
|
||||
### Browser Settings
|
||||
|
||||
```yaml
|
||||
browser:
|
||||
# Run browser in headless mode (no UI)
|
||||
# Set to false for debugging
|
||||
headless: true
|
||||
|
||||
# Directory for persistent browser data
|
||||
# Cookies and session storage are saved here
|
||||
userDataDir: "./data/browser"
|
||||
```
|
||||
|
||||
**Browser session persistence:**
|
||||
|
||||
```
|
||||
data/browser/
|
||||
├── Default/
|
||||
│ ├── Cookies # Reddit session cookies
|
||||
│ ├── Local Storage # Local storage data
|
||||
│ └── Preferences # Browser preferences
|
||||
├── Login Data # Saved login info
|
||||
└── Preferences # Browser state
|
||||
```
|
||||
|
||||
> **Important**: Keep this directory intact. Deleting it requires re-login.
|
||||
|
||||
### Database Settings
|
||||
|
||||
```yaml
|
||||
database:
|
||||
# Path to SQLite database file
|
||||
path: "./data/reddit-bot.db"
|
||||
```
|
||||
|
||||
The database is auto-created on first run. Migrations run automatically on startup.
|
||||
|
||||
### Media Sync Settings
|
||||
|
||||
```yaml
|
||||
mediaSync:
|
||||
# Enable periodic media synchronization
|
||||
enabled: true
|
||||
|
||||
# Interval between sync cycles (minutes)
|
||||
intervalMinutes: 30
|
||||
```
|
||||
|
||||
**Sync behavior:**
|
||||
- **Incremental sync** (automatic): Downloads media from top 10 conversations
|
||||
- **Full sync** (manual via dashboard): Downloads media from all conversations
|
||||
|
||||
### Auto-Reply Settings
|
||||
|
||||
```yaml
|
||||
autoReply:
|
||||
# Enable automatic chat replies
|
||||
enabled: true
|
||||
|
||||
# Auto-send replies (true) or mark as pending for approval (false)
|
||||
autoSend: false
|
||||
|
||||
# Minutes between checking for new messages
|
||||
checkInterval: 10
|
||||
```
|
||||
|
||||
**Reply modes:**
|
||||
|
||||
| autoSend | Behavior |
|
||||
|---|---|
|
||||
| `true` | Bot sends AI-generated replies automatically |
|
||||
| `false` | Bot suggests replies, user approves via dashboard |
|
||||
|
||||
### Strategy Variants (A/B Testing)
|
||||
|
||||
```yaml
|
||||
strategyVariants:
|
||||
# Variant 1: Compliment-based
|
||||
- name: "compliment"
|
||||
prompt: |
|
||||
Write a friendly compliment about the person's appearance.
|
||||
Be genuine and specific. Keep it short and natural.
|
||||
enabled: true
|
||||
|
||||
# Variant 2: Question-based
|
||||
- name: "question"
|
||||
prompt: |
|
||||
Write a friendly message that includes a question.
|
||||
Engage them in conversation naturally.
|
||||
enabled: true
|
||||
|
||||
# Variant 3: Casual observation
|
||||
- name: "observation"
|
||||
prompt: |
|
||||
Write a casual observation about the photo or setting.
|
||||
Sound like you're commenting to a friend.
|
||||
enabled: true
|
||||
```
|
||||
|
||||
### Web Dashboard Settings
|
||||
|
||||
```yaml
|
||||
web:
|
||||
# Port for the web dashboard
|
||||
port: 3000
|
||||
```
|
||||
|
||||
### Global Settings
|
||||
|
||||
```yaml
|
||||
# Global dry run mode
|
||||
# When true, all actions are logged but not actually performed
|
||||
dryRun: false
|
||||
|
||||
# Number of successful message examples to include in prompts
|
||||
fewShotExamples: 3
|
||||
|
||||
# Minimum responses needed before trusting strategy stats
|
||||
minStrategySamples: 10
|
||||
|
||||
# Exploration rate for A/B testing (0.0 to 1.0)
|
||||
# 0.15 = 15% of messages use random variants (exploration)
|
||||
explorationRate: 0.15
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Runtime Configuration
|
||||
|
||||
The following settings can be modified via the web dashboard without restarting the bot:
|
||||
|
||||
| Setting | Endpoint | Type | Description |
|
||||
|---|---|---|---|
|
||||
| `rateLimit.minDelayMs` | `PUT /api/config` | number | Message delay |
|
||||
| `rateLimit.maxMessagesPerHour` | `PUT /api/config` | number | Hourly cap |
|
||||
| `rateLimit.jitterMs` | `PUT /api/config` | number | Jitter range |
|
||||
| `reddit.pollInterval` | `PUT /api/config` | number | Poll frequency |
|
||||
| `dryRun` | `PUT /api/config` | boolean | Dry run mode |
|
||||
|
||||
### Updating Runtime Config
|
||||
|
||||
Via dashboard:
|
||||
1. Open the settings modal
|
||||
2. Modify desired values
|
||||
3. Click "Save"
|
||||
|
||||
Via API:
|
||||
```bash
|
||||
curl -X PUT http://localhost:3000/api/config \
|
||||
-H "Content-Type: application/json" \
|
||||
-d '{
|
||||
"rateLimit": {
|
||||
"minDelayMs": 45000,
|
||||
"maxMessagesPerHour": 15,
|
||||
"jitterMs": 20000
|
||||
},
|
||||
"dryRun": true
|
||||
}'
|
||||
```
|
||||
|
||||
> **Note**: Runtime changes are in-memory only. They are lost when the bot restarts. To make changes persistent, update `config.yaml`.
|
||||
|
||||
---
|
||||
|
||||
## Configuration Validation
|
||||
|
||||
All configuration is validated using Zod at startup. Invalid configuration will cause the bot to exit with a descriptive error.
|
||||
|
||||
### Validation Rules
|
||||
|
||||
| Field | Validation |
|
||||
|---|---|
|
||||
| `reddit.subreddits` | Must be non-empty array of strings |
|
||||
| `reddit.pollInterval` | Must be positive integer |
|
||||
| `reddit.targetGender` | Must be "F" or "M" |
|
||||
| `openrouter.apiKey` | Must be non-empty string |
|
||||
| `openrouter.model` | Must be non-empty string |
|
||||
| `openrouter.fallbackModels` | Must be non-empty array |
|
||||
| `rateLimit.minDelayMs` | Must be positive integer |
|
||||
| `rateLimit.maxMessagesPerHour` | Must be positive integer |
|
||||
| `rateLimit.jitterMs` | Must be non-negative integer |
|
||||
| `browser.headless` | Must be boolean |
|
||||
| `browser.userDataDir` | Must be non-empty string |
|
||||
| `database.path` | Must be non-empty string |
|
||||
| `mediaSync.enabled` | Must be boolean |
|
||||
| `mediaSync.intervalMinutes` | Must be positive integer (if enabled) |
|
||||
| `autoReply.enabled` | Must be boolean |
|
||||
| `autoReply.autoSend` | Must be boolean |
|
||||
| `autoReply.checkInterval` | Must be positive integer (if enabled) |
|
||||
| `strategyVariants` | Must be array with name + prompt |
|
||||
| `web.port` | Must be integer 1-65535 |
|
||||
| `dryRun` | Must be boolean |
|
||||
|
||||
### Common Validation Errors
|
||||
|
||||
**Error**: `Invalid config: openrouter.apiKey must be a string`
|
||||
**Fix**: Set `OPENROUTER_API_KEY` environment variable or add `apiKey` to `config.yaml`
|
||||
|
||||
**Error**: `Invalid config: reddit.subreddits must be a non-empty array`
|
||||
**Fix**: Add at least one subreddit to `reddit.subreddits` in `config.yaml`
|
||||
|
||||
**Error**: `Invalid config: browser.userDataDir must be a string`
|
||||
**Fix**: Ensure `browser.userDataDir` is set in `config.yaml`
|
||||
|
||||
---
|
||||
|
||||
## Troubleshooting
|
||||
|
||||
### Configuration Not Loading
|
||||
|
||||
**Symptom**: Bot fails to start with validation errors.
|
||||
|
||||
**Steps**:
|
||||
1. Verify `config.yaml` is in the project root
|
||||
2. Check YAML syntax (use a YAML linter)
|
||||
3. Verify environment variables are set correctly
|
||||
4. Check bot logs for specific validation errors
|
||||
|
||||
### Credentials Not Working
|
||||
|
||||
**Symptom**: Browser login fails.
|
||||
|
||||
**Steps**:
|
||||
1. Verify `REDDIT_USERNAME` and `REDDIT_PASSWORD` are correct
|
||||
2. Check that the account is not locked or requires 2FA
|
||||
3. Try manual login via `npm run login`
|
||||
4. Ensure `BROWSER_HEADLESS=false` for debugging login issues
|
||||
|
||||
### AI Not Working
|
||||
|
||||
**Symptom**: Image analysis fails or returns errors.
|
||||
|
||||
**Steps**:
|
||||
1. Verify `OPENROUTER_API_KEY` is valid
|
||||
2. Check API key has sufficient balance
|
||||
3. Verify `openrouter.model` is available on OpenRouter
|
||||
4. Check fallback models are set correctly
|
||||
5. Review logs for specific AI errors
|
||||
|
||||
### Browser Session Expired
|
||||
|
||||
**Symptom**: Bot stops with "Browser session expired" error.
|
||||
|
||||
**Steps**:
|
||||
1. Delete `data/browser/` directory
|
||||
2. Run `npm run login` to re-authenticate
|
||||
3. Restart the bot
|
||||
|
||||
### Port Already in Use
|
||||
|
||||
**Symptom**: Dashboard fails to start with "Port already in use".
|
||||
|
||||
**Steps**:
|
||||
1. Change `web.port` in `config.yaml`
|
||||
2. Or kill the process using the port: `lsof -i :3000`
|
||||
@@ -0,0 +1,875 @@
|
||||
# Data Flow Documentation
|
||||
|
||||
Complete documentation of data flows through the RedditBot system.
|
||||
|
||||
---
|
||||
|
||||
## Table of Contents
|
||||
|
||||
1. [Overview](#overview)
|
||||
2. [Post Processing Flow](#post-processing-flow)
|
||||
3. [Response Monitoring Flow](#response-monitoring-flow)
|
||||
4. [Chat Interaction Flow](#chat-interaction-flow)
|
||||
5. [Strategy Selection Flow](#strategy-selection-flow)
|
||||
6. [Media Sync Flow](#media-sync-flow)
|
||||
7. [Auto-Reply Flow](#auto-reply-flow)
|
||||
8. [Dashboard Data Flow](#dashboard-data-flow)
|
||||
9. [Error Recovery Flows](#error-recovery-flows)
|
||||
10. [Data Lifecycle](#data-lifecycle)
|
||||
|
||||
---
|
||||
|
||||
## Overview
|
||||
|
||||
RedditBot processes data through several interconnected pipelines:
|
||||
|
||||
```
|
||||
┌─────────────┐ ┌──────────────┐ ┌─────────────┐
|
||||
│ Reddit API │────►│ Monitor │────►│ Processor │
|
||||
│ (Posts) │ │ Service │ │ Service │
|
||||
└─────────────┘ └──────────────┘ └──────┬──────┘
|
||||
│
|
||||
┌──────────────┐ ▼
|
||||
│ Response │ ┌──────────────┐
|
||||
│ Checker │◄────│ OpenRouter │
|
||||
└──────────────┘ │ (AI) │
|
||||
└──────────────┘
|
||||
│
|
||||
┌──────────────┐ ▼
|
||||
│ Chat │ ┌──────────────┐
|
||||
│ Service │◄────│ Message │
|
||||
└──────────────┘ │ Sender │
|
||||
└──────────────┘
|
||||
│
|
||||
┌──────────────┐ ▼
|
||||
│ Media │ ┌──────────────┐
|
||||
│ Service │◄────│ SQLite DB │
|
||||
└──────────────┘ └──────────────┘
|
||||
│
|
||||
┌──────────────┐ │
|
||||
│ Dashboard │◄──────────┘
|
||||
│ (Express) │
|
||||
└──────────────┘
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Post Processing Flow
|
||||
|
||||
The primary data flow for discovering, analyzing, and messaging about new posts.
|
||||
|
||||
### Stage 1: Discovery
|
||||
|
||||
```
|
||||
┌─────────────────────────────────────────────────────────────────┐
|
||||
│ MonitorService │
|
||||
│ │
|
||||
│ 1. Shuffle subreddit order │
|
||||
│ 2. For each subreddit: │
|
||||
│ ┌─────────────────────────────────────────────────────┐ │
|
||||
│ │ RedditApi.fetchPosts(subreddit) │ │
|
||||
│ │ │ │
|
||||
│ │ HTTP: GET /r/{subreddit}/new.json?limit=100 │ │
|
||||
│ │ │ │
|
||||
│ │ Response: { │ │
|
||||
│ │ data: { │ │
|
||||
│ │ children: [ │ │
|
||||
│ │ { │ │
|
||||
│ │ data: { │ │
|
||||
│ │ id: "abc123", │ │
|
||||
│ │ subreddit: "gonewild", │ │
|
||||
│ │ author: "user123", │ │
|
||||
│ │ title: "Summer vibes", │ │
|
||||
│ │ url: "https://...", │ │
|
||||
│ │ preview: { reddits: [...] }, │ │
|
||||
│ │ thumbnail: "https://...", │ │
|
||||
│ │ post_hint: "image", │ │
|
||||
│ │ over_18: true │ │
|
||||
│ │ } │ │
|
||||
│ │ } │ │
|
||||
│ │ ] │ │
|
||||
│ │ } │ │
|
||||
│ │ } │ │
|
||||
│ └─────────────────────────────────────────────────────┘ │
|
||||
│ │
|
||||
│ 3. Filter posts: │
|
||||
│ ├─► post_hint === "image" │
|
||||
│ ├─► author exists and is not deleted │
|
||||
│ ├─► image_url can be extracted │
|
||||
│ └─► Author not in seller/spam list │
|
||||
│ │
|
||||
│ 4. Cross-reference with DB: │
|
||||
│ ├─► PostsRepository.getById(postId) → skip if exists │
|
||||
│ └─► MessagesRepository.getByAuthor(author) → skip if sent │
|
||||
│ │
|
||||
│ 5. Return newest valid post │
|
||||
└─────────────────────────────────────────────────────────────────┘
|
||||
```
|
||||
|
||||
### Stage 2: Image Processing
|
||||
|
||||
```
|
||||
┌─────────────────────────────────────────────────────────────────┐
|
||||
│ ProcessorService │
|
||||
│ │
|
||||
│ 1. Download Image │
|
||||
│ ┌─────────────────────────────────────────────────────┐ │
|
||||
│ │ ImageService.downloadImage(url) │ │
|
||||
│ │ │ │
|
||||
│ │ HTTP: GET {image_url} │ │
|
||||
│ │ │ │
|
||||
│ │ Response: Binary image data │ │
|
||||
│ │ │ │
|
||||
│ │ Validation: │ │
|
||||
│ │ ├─► MIME type: image/jpeg | png | webp │ │
|
||||
│ │ ├─► File size: < 10MB │ │
|
||||
│ │ └─► Not empty/broken │ │
|
||||
│ └─────────────────────────────────────────────────────┘ │
|
||||
│ │
|
||||
│ 2. Convert to base64 │
|
||||
│ ┌─────────────────────────────────────────────────────┐ │
|
||||
│ │ Buffer.toString('base64') │ │
|
||||
│ │ │ │
|
||||
│ │ Result: "data:image/jpeg;base64,/9j/4AAQSkZJR..." │ │
|
||||
│ └─────────────────────────────────────────────────────┘ │
|
||||
└─────────────────────────────────────────────────────────────────┘
|
||||
```
|
||||
|
||||
### Stage 3: Gender Detection (Pre-Filter)
|
||||
|
||||
```
|
||||
┌─────────────────────────────────────────────────────────────────┐
|
||||
│ OpenRouterService │
|
||||
│ │
|
||||
│ 1. detectGenderOnly(imageBase64) │
|
||||
│ ┌─────────────────────────────────────────────────────┐ │
|
||||
│ │ HTTP: POST https://openrouter.ai/api/v1/chat/ │ │
|
||||
│ │ completions │ │
|
||||
│ │ │ │
|
||||
│ │ Request: { │ │
|
||||
│ │ model: "fast-model", │ │
|
||||
│ │ messages: [ │ │
|
||||
│ │ { │ │
|
||||
│ │ role: "user", │ │
|
||||
│ │ content: [ │ │
|
||||
│ │ { type: "text", text: "Is this F or M?" }, │ │
|
||||
│ │ { type: "image_url", image_url: {...} } │ │
|
||||
│ │ ] │ │
|
||||
│ │ } │ │
|
||||
│ │ ] │ │
|
||||
│ │ } │ │
|
||||
│ │ │ │
|
||||
│ │ Response: { │ │
|
||||
│ │ choices: [{ message: { content: "F" } }] │ │
|
||||
│ │ } │ │
|
||||
│ └─────────────────────────────────────────────────────┘ │
|
||||
│ │
|
||||
│ 3. Compare with config.targetGender │
|
||||
│ ├─► Match: continue to full analysis │
|
||||
│ └─► No match: SKIP post (log: "Gender mismatch, skipping") │
|
||||
└─────────────────────────────────────────────────────────────────┘
|
||||
```
|
||||
|
||||
### Stage 4: Strategy Selection
|
||||
|
||||
```
|
||||
┌─────────────────────────────────────────────────────────────────┐
|
||||
│ StrategySelector │
|
||||
│ │
|
||||
│ 1. Get strategy statistics from DB │
|
||||
│ ┌─────────────────────────────────────────────────────┐ │
|
||||
│ │ MessagesRepository.getStrategyStats() │ │
|
||||
│ │ │ │
|
||||
│ │ Result: { │ │
|
||||
│ │ variant_a: { sent: 50, responded: 15, rate: 0.30 },│
|
||||
│ │ variant_b: { sent: 30, responded: 12, rate: 0.40 },│
|
||||
│ │ variant_c: { sent: 5, responded: 1, rate: 0.20 } │
|
||||
│ │ } │ │
|
||||
│ └─────────────────────────────────────────────────────┘ │
|
||||
│ │
|
||||
│ 2. Apply exploration/exploit algorithm │
|
||||
│ ├─► Random roll < explorationRate → pick random variant │
|
||||
│ └─► Otherwise → pick highest response rate (min samples) │
|
||||
│ │
|
||||
│ 3. Return selected variant name │
|
||||
│ Example: "variant_b" │
|
||||
└─────────────────────────────────────────────────────────────────┘
|
||||
```
|
||||
|
||||
### Stage 5: Full AI Analysis
|
||||
|
||||
```
|
||||
┌─────────────────────────────────────────────────────────────────┐
|
||||
│ OpenRouterService │
|
||||
│ │
|
||||
│ 1. analyzeImage(imageBase64) │
|
||||
│ ┌─────────────────────────────────────────────────────┐ │
|
||||
│ │ HTTP: POST https://openrouter.ai/api/v1/chat/ │ │
|
||||
│ │ completions │ │
|
||||
│ │ │ │
|
||||
│ │ Request: { │ │
|
||||
│ │ model: "primary-model", │ │
|
||||
│ │ messages: [ │ │
|
||||
│ │ { │ │
|
||||
│ │ role: "system", │ │
|
||||
│ │ content: config.openrouter.systemPrompt │ │
|
||||
│ │ }, │ │
|
||||
│ │ { │ │
|
||||
│ │ role: "user", │ │
|
||||
│ │ content: [ │ │
|
||||
│ │ { type: "text", text: "Describe this image" },│
|
||||
│ │ { type: "image_url", image_url: {...} } │ │
|
||||
│ │ ] │ │
|
||||
│ │ } │ │
|
||||
│ │ ] │ │
|
||||
│ │ } │ │
|
||||
│ │ │ │
|
||||
│ │ Response: { │ │
|
||||
│ │ choices: [{ │ │
|
||||
│ │ message: { │ │
|
||||
│ │ content: '{"gender":"F","description":"..."}' │ │
|
||||
│ │ } │ │
|
||||
│ │ }] │ │
|
||||
│ │ } │ │
|
||||
│ └─────────────────────────────────────────────────────┘ │
|
||||
│ │
|
||||
│ 2. Check for refusal │
|
||||
│ ├─► If refused → try fallback models │
|
||||
│ └─► If all failed → mark post with error, skip │
|
||||
│ │
|
||||
│ 3. Parse and return ImageAnalysis │
|
||||
│ { gender: "F", description: "Young woman, blonde hair..." }│
|
||||
└─────────────────────────────────────────────────────────────────┘
|
||||
```
|
||||
|
||||
### Stage 6: Message Generation
|
||||
|
||||
```
|
||||
┌─────────────────────────────────────────────────────────────────┐
|
||||
│ OpenRouterService │
|
||||
│ │
|
||||
│ 1. Retrieve few-shot examples │
|
||||
│ ┌─────────────────────────────────────────────────────┐ │
|
||||
│ │ SuccessfulMessagesRepository.getExamples( │ │
|
||||
│ │ subreddit, count: 3 │ │
|
||||
│ │ ) │ │
|
||||
│ │ │ │
|
||||
│ │ Result: [ │ │
|
||||
│ │ { │ │
|
||||
│ │ original_message: "Hey, you look amazing...", │ │
|
||||
│ │ response_snippet: "Thank you so much!", │ │
|
||||
│ │ subreddit: "gonewild" │ │
|
||||
│ │ }, │ │
|
||||
│ │ // ... more examples │ │
|
||||
│ │ ] │ │
|
||||
│ └─────────────────────────────────────────────────────┘ │
|
||||
│ │
|
||||
│ 2. Generate message │
|
||||
│ ┌─────────────────────────────────────────────────────┐ │
|
||||
│ │ HTTP: POST https://openrouter.ai/api/v1/chat/ │ │
|
||||
│ │ completions │ │
|
||||
│ │ │ │
|
||||
│ │ Request: { │ │
|
||||
│ │ model: "primary-model", │ │
|
||||
│ │ messages: [ │ │
|
||||
│ │ { role: "system", content: messagePrompt }, │ │
|
||||
│ │ { role: "user", content: [ │ │
|
||||
│ │ `Image: ${imageAnalysis.description}`, │ │
|
||||
│ │ `Post title: ${postTitle}`, │ │
|
||||
│ │ `Strategy: ${selectedVariant}`, │ │
|
||||
│ │ `Examples: ${JSON.stringify(examples)}` │ │
|
||||
│ │ ] │ │
|
||||
│ │ } │ │
|
||||
│ │ ] │ │
|
||||
│ │ } │ │
|
||||
│ └─────────────────────────────────────────────────────┘ │
|
||||
│ │
|
||||
│ 3. Return generated message │
|
||||
│ "Hey, I came across your post and..." │
|
||||
└─────────────────────────────────────────────────────────────────┘
|
||||
```
|
||||
|
||||
### Stage 7: DM Sending
|
||||
|
||||
```
|
||||
┌─────────────────────────────────────────────────────────────────┐
|
||||
│ MessagePage │
|
||||
│ │
|
||||
│ 1. Navigate to compose page │
|
||||
│ ┌─────────────────────────────────────────────────────┐ │
|
||||
│ │ page.goto("https://www.reddit.com/compose?to= │ │
|
||||
│ │ user123") │ │
|
||||
│ └─────────────────────────────────────────────────────┘ │
|
||||
│ │
|
||||
│ 2. Fill recipient field │
|
||||
│ ┌─────────────────────────────────────────────────────┐ │
|
||||
│ │ page.evaluate(() => { │ │
|
||||
│ │ const input = document.querySelector( │ │
|
||||
│ │ 'faceplate-autocomplete input' │ │
|
||||
│ │ ); │ │
|
||||
│ │ input.value = "user123"; │ │
|
||||
│ │ input.dispatchEvent(new Event('input', │ │
|
||||
│ │ { bubbles: true })); │ │
|
||||
│ │ }); │ │
|
||||
│ └─────────────────────────────────────────────────────┘ │
|
||||
│ │
|
||||
│ 3. Fill subject field ("Stunning") │
|
||||
│ 4. Fill body field (generated message) │
|
||||
│ 5. Click send button │
|
||||
│ 6. Verify success (check for confirmation) │
|
||||
│ │
|
||||
│ Result: true (sent) or false (failed) │
|
||||
└─────────────────────────────────────────────────────────────────┘
|
||||
```
|
||||
|
||||
### Stage 8: Database Recording
|
||||
|
||||
```
|
||||
┌─────────────────────────────────────────────────────────────────┐
|
||||
│ Database │
|
||||
│ │
|
||||
│ 1. INSERT INTO processed_posts │
|
||||
│ VALUES ( │
|
||||
│ 'abc123', -- post_id │
|
||||
│ 'gonewild', -- subreddit │
|
||||
│ 'user123', -- author │
|
||||
│ 'Summer vibes', -- title │
|
||||
│ 'https://...', -- image_url │
|
||||
│ '{"gender":"F",...}', -- image_analysis │
|
||||
│ 'Hey, I came across...', -- generated_message │
|
||||
│ 1716234567, -- processed_at │
|
||||
│ 1, -- message_sent │
|
||||
│ NULL -- error │
|
||||
│ ); │
|
||||
│ │
|
||||
│ 2. INSERT INTO sent_messages │
|
||||
│ VALUES ( │
|
||||
│ NULL, -- id (auto) │
|
||||
│ 'abc123', -- post_id │
|
||||
│ 'user123', -- recipient │
|
||||
│ 'Stunning', -- subject │
|
||||
│ 'Hey, I came across...', -- message │
|
||||
│ 1716234567, -- sent_at │
|
||||
│ 'variant_b', -- strategy_id │
|
||||
│ 0, - - got_response │
|
||||
│ NULL -- response_checked_at │
|
||||
│ ); │
|
||||
└─────────────────────────────────────────────────────────────────┘
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Response Monitoring Flow
|
||||
|
||||
How the bot detects and records user replies to sent DMs.
|
||||
|
||||
```
|
||||
┌─────────────────────────────────────────────────────────────────┐
|
||||
│ ResponseCheckerService │
|
||||
│ │
|
||||
│ 1. Navigate to inbox │
|
||||
│ ┌─────────────────────────────────────────────────────┐ │
|
||||
│ │ page.goto("https://www.reddit.com/message/inbox") │ │
|
||||
│ └─────────────────────────────────────────────────────┘ │
|
||||
│ │
|
||||
│ 2. Parse inbox items │
|
||||
│ ┌─────────────────────────────────────────────────────┐ │
|
||||
│ │ Extract from page: │ │
|
||||
│ │ [ │ │
|
||||
│ │ { │ │
|
||||
│ │ from: "user456", │ │
|
||||
│ │ subject: "Re: Stunning", │ │
|
||||
│ │ snippet: "Thank you so much! 😊", │ │
|
||||
│ │ timestamp: 1716234000, │ │
|
||||
│ │ link: "/message/xyz789" │ │
|
||||
│ │ }, │ │
|
||||
│ │ // ... more messages │ │
|
||||
│ │ ] │ │
|
||||
│ └─────────────────────────────────────────────────────┘ │
|
||||
│ │
|
||||
│ 3. Match responses to sent messages │
|
||||
│ ┌─────────────────────────────────────────────────────┐ │
|
||||
│ │ For each inbox item: │ │
|
||||
│ │ ├─► MessagesRepository.getByRecipient(from) │ │
|
||||
│ │ ├─► Find unmatched message within time window │ │
|
||||
│ │ ├─► Update: got_response = 1 │ │
|
||||
│ │ ├─► Update: response_checked_at = now │ │
|
||||
│ │ └─► Store in successful_messages │ │
|
||||
│ └─────────────────────────────────────────────────────┘ │
|
||||
│ │
|
||||
│ 4. Update strategy statistics │
|
||||
│ ┌─────────────────────────────────────────────────────┐ │
|
||||
│ │ MessagesRepository.updateStrategyStats( │ │
|
||||
│ │ strategy_id, got_response: true │ │
|
||||
│ │ ) │ │
|
||||
│ └─────────────────────────────────────────────────────┘ │
|
||||
│ │
|
||||
│ 5. Log results │
|
||||
│ └─► "Found 3 responses, updated 2 strategies" │
|
||||
└─────────────────────────────────────────────────────────────────┘
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Chat Interaction Flow
|
||||
|
||||
How the bot interacts with Reddit chat conversations.
|
||||
|
||||
```
|
||||
┌─────────────────────────────────────────────────────────────────┐
|
||||
│ ChatService │
|
||||
│ │
|
||||
│ ──── PHASE 1: FETCH CHATS ────────────────────────────────── │
|
||||
│ │
|
||||
│ 1. Navigate to chat page │
|
||||
│ page.goto("https://chat.reddit.com") │
|
||||
│ │
|
||||
│ 2. Traverse shadow DOM for conversations │
|
||||
│ ┌─────────────────────────────────────────────────────┐ │
|
||||
│ │ page.evaluate(() => { │ │
|
||||
│ │ // JS injection to pierce shadow DOM │ │
|
||||
│ │ const roots = document.querySelectorAll( │ │
|
||||
│ │ 'faceplate-chat-list' │ │
|
||||
│ │ ); │ │
|
||||
│ │ const chats = []; │ │
|
||||
│ │ roots.forEach(root => { │ │
|
||||
│ │ const shadow = root.shadowRoot; │ │
|
||||
│ │ const items = shadow.querySelectorAll( │ │
|
||||
│ │ '.chat-item' │ │
|
||||
│ │ ); │ │
|
||||
│ │ items.forEach(item => { │ │
|
||||
│ │ chats.push({ │ │
|
||||
│ │ id: item.dataset.chatId, │ │
|
||||
│ │ participant: item.querySelector( │ │
|
||||
│ │ '.username' │ │
|
||||
│ │ ).textContent, │ │
|
||||
│ │ lastMessage: item.querySelector( │ │
|
||||
│ │ '.message' │ │
|
||||
│ │ ).textContent, │ │
|
||||
│ │ timestamp: item.dataset.timestamp │ │
|
||||
│ │ }); │ │
|
||||
│ │ }); │ │
|
||||
│ │ }); │ │
|
||||
│ │ return chats; │ │
|
||||
│ │ }); │ │
|
||||
│ └─────────────────────────────────────────────────────┘ │
|
||||
│ │
|
||||
│ 3. Cache results │
|
||||
│ └─► Save to memory + disk JSON │
|
||||
│ │
|
||||
│ ──── PHASE 2: LOAD CONVERSATION ──────────────────────────── │
|
||||
│ │
|
||||
│ 1. Check cache │
|
||||
│ ├─► If cached and fresh: return from memory │
|
||||
│ └─► If not: fetch from disk or Reddit │
|
||||
│ │
|
||||
│ 2. Fetch full history (pagination) │
|
||||
│ ┌─────────────────────────────────────────────────────┐ │
|
||||
│ │ For each page of messages: │ │
|
||||
│ │ ├─► Navigate to chat view │ │
|
||||
│ │ ├─► Extract messages from shadow DOM │ │
|
||||
│ │ ├─► Scroll to load more (infinite scroll) │ │
|
||||
│ │ └─► Repeat until all loaded │ │
|
||||
│ └─────────────────────────────────────────────────────┘ │
|
||||
│ │
|
||||
│ 3. Return conversation object │
|
||||
│ { │
|
||||
│ chatId: "abc123", │
|
||||
│ participants: ["user456", "bot"], │
|
||||
│ messages: [ │
|
||||
│ { author: "user456", text: "Hey!", timestamp: ... }, │
|
||||
│ { author: "bot", text: "Hi there!", timestamp: ... }, │
|
||||
│ // ... more messages │
|
||||
│ ], │
|
||||
│ lastUpdated: 1716234567 │
|
||||
│ } │
|
||||
│ │
|
||||
│ ──── PHASE 3: SEND MESSAGE ───────────────────────────────── │
|
||||
│ │
|
||||
│ 1. Navigate to chat view │
|
||||
│ 2. Fill message input (shadow DOM) │
|
||||
│ 3. Apply typing delay (human-like) │
|
||||
│ ┌─────────────────────────────────────────────────────┐ │
|
||||
│ │ const typingDelay = random(500, 2000); │ │
|
||||
│ │ await page.waitForTimeout(typingDelay); │ │
|
||||
│ │ │ │
|
||||
│ │ // Simulate typing character by character │ │
|
||||
│ │ for (const char of message) { │ │
|
||||
│ │ input.value += char; │ │
|
||||
│ │ await page.waitForTimeout(random(30, 100)); │ │
|
||||
│ │ } │ │
|
||||
│ └─────────────────────────────────────────────────────┘ │
|
||||
│ 4. Submit message │
|
||||
│ 5. Verify delivery (check for sent indicator) │
|
||||
│ │
|
||||
│ Result: true (delivered) or false (failed) │
|
||||
└─────────────────────────────────────────────────────────────────┘
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Strategy Selection Flow
|
||||
|
||||
How A/B test variants are selected and updated.
|
||||
|
||||
```
|
||||
┌─────────────────────────────────────────────────────────────────┐
|
||||
│ StrategySelector │
|
||||
│ │
|
||||
│ SELECT STRATEGY │
|
||||
│ ──────────────────────────────────────────────────────────── │
|
||||
│ │
|
||||
│ Input: subreddit, author │
|
||||
│ │
|
||||
│ 1. Get stats from DB │
|
||||
│ ┌─────────────────────────────────────────────────────┐ │
|
||||
│ │ SELECT strategy_id, │ │
|
||||
│ │ COUNT(*) as sent, │ │
|
||||
│ │ SUM(got_response) as responded │ │
|
||||
│ │ FROM sent_messages │ │
|
||||
│ │ GROUP BY strategy_id │ │
|
||||
│ └─────────────────────────────────────────────────────┘ │
|
||||
│ │
|
||||
│ 2. Calculate response rates │
|
||||
│ variant_a: 15/50 = 30% │
|
||||
│ variant_b: 12/30 = 40% │
|
||||
│ variant_c: 1/5 = 20% │
|
||||
│ │
|
||||
│ 3. Filter by minimum samples │
|
||||
│ minSamples = 10 │
|
||||
│ variant_c excluded (only 5 sends) │
|
||||
│ │
|
||||
│ 4. Explore vs Exploit │
|
||||
│ random() < explorationRate (e.g., 0.15)? │
|
||||
│ ├─► Yes: Random pick from enabled variants │
|
||||
│ └─► No: Pick highest rate (variant_b at 40%) │
|
||||
│ │
|
||||
│ Output: selected variant name │
|
||||
│ │
|
||||
│ ──── UPDATE STATS ────────────────────────────────────────── │
|
||||
│ │
|
||||
│ Input: strategy_id, got_response (boolean) │
|
||||
│ │
|
||||
│ 1. Record in sent_messages │
|
||||
│ UPDATE sent_messages │
|
||||
│ SET got_response = 1, │
|
||||
│ response_checked_at = now │
|
||||
│ WHERE id = ? │
|
||||
│ │
|
||||
│ 2. If successful: store in successful_messages │
|
||||
│ INSERT INTO successful_messages │
|
||||
│ VALUES (original_message, image_analysis, post_title, │
|
||||
│ response_snippet, subreddit, now) │
|
||||
│ │
|
||||
│ 3. Update strategy statistics │
|
||||
│ (reflected in next selectStrategy call) │
|
||||
└─────────────────────────────────────────────────────────────────┘
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Media Sync Flow
|
||||
|
||||
How media files are downloaded and synchronized from chat conversations.
|
||||
|
||||
```
|
||||
┌─────────────────────────────────────────────────────────────────┐
|
||||
│ MediaSyncService │
|
||||
│ │
|
||||
│ ──── TRIGGER ──────────────────────────────────────────────── │
|
||||
│ │
|
||||
│ Scheduled: every N minutes (config.mediaSync.intervalMinutes) │
|
||||
│ Manual: POST /api/media/sync (incremental) │
|
||||
│ Manual: POST /api/media/sync/full (full) │
|
||||
│ │
|
||||
│ ──── INCREMENTAL SYNC ────────────────────────────────────── │
|
||||
│ │
|
||||
│ 1. Get top 10 conversations from ChatService │
|
||||
│ 2. For each conversation: │
|
||||
│ ├─► Check if media already downloaded (index lookup) │
|
||||
│ ├─► If not: download media │
|
||||
│ │ └─► MediaService.downloadMedia(chatId) │
|
||||
│ └─► Update index │
|
||||
│ │
|
||||
│ ──── FULL SYNC ────────────────────────────────────────────── │
|
||||
│ │
|
||||
│ 1. Get ALL conversations from ChatService │
|
||||
│ 2. For each conversation: │
|
||||
│ ├─► Download any missing media │
|
||||
│ └─► Update index │
|
||||
│ │
|
||||
│ ──── MEDIA DOWNLOAD (MediaService) ────────────────────────── │
|
||||
│ │
|
||||
│ 1. Extract media URLs from conversation messages │
|
||||
│ 2. For each URL: │
|
||||
│ ├─► Check if Matrix thumbnail │
|
||||
│ │ └─► Resolve to full resolution URL │
|
||||
│ ├─► Download via Playwright │
|
||||
│ ├─► Compute SHA-1 hash │
|
||||
│ ├─► Check if already exists (dedup) │
|
||||
│ ├─► Save to data/media/{hash}.{ext} │
|
||||
│ └─► Update index JSON │
|
||||
│ │
|
||||
│ Output: Array of saved file paths │
|
||||
└─────────────────────────────────────────────────────────────────┘
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Auto-Reply Flow
|
||||
|
||||
How the bot automatically replies to chat messages.
|
||||
|
||||
```
|
||||
┌─────────────────────────────────────────────────────────────────┐
|
||||
│ AutoReplyService │
|
||||
│ │
|
||||
│ Scheduled: every N minutes (config.autoReply.checkInterval) │
|
||||
│ │
|
||||
│ 1. Get cached conversations │
|
||||
│ 2. For each conversation: │
|
||||
│ │ │
|
||||
│ ├─► Check for new/unread messages │
|
||||
│ │ └─► If no new messages: continue │
|
||||
│ │ │
|
||||
│ ├─► Detect upsell/spam keywords │
|
||||
│ │ ├─► "onlyfans", "link in bio", "check my..." │
|
||||
│ │ ├─► If detected: flag conversation, skip reply │
|
||||
│ │ └─► If not: continue │
|
||||
│ │ │
|
||||
│ ├─► Generate AI reply suggestion │
|
||||
│ │ ┌─────────────────────────────────────────────────┐ │
|
||||
│ │ │ OpenRouter.generateReply(conversationHistory) │ │
|
||||
│ │ │ │ │
|
||||
│ │ │ Request: { │ │
|
||||
│ │ │ messages: [ │ │
|
||||
│ │ │ { role: "system", content: replyPrompt }, │ │
|
||||
│ │ │ { role: "user", content: conversation } │ │
|
||||
│ │ │ ] │ │
|
||||
│ │ │ } │ │
|
||||
│ │ │ │ │
|
||||
│ │ │ Response: "Thanks for reaching out! 😊" │ │
|
||||
│ │ │ │ │
|
||||
│ │ │ Reply: "Thanks for reaching out! 😊" │ │
|
||||
│ │ └─────────────────────────────────────────────────┘ │
|
||||
│ │ │
|
||||
│ ├─► If autoSend = true: │
|
||||
│ │ └─► ChatService.sendMessage(chatId, reply) │
|
||||
│ │ └─► Log: "Auto-sent reply to {user}" │
|
||||
│ │ │
|
||||
│ └─► If autoSend = false: │
|
||||
│ └─► Mark as pending (visible in dashboard) │
|
||||
│ └─► User can approve/reject via dashboard │
|
||||
│ │
|
||||
│ Output: Summary of actions taken │
|
||||
│ "Checked 15 conversations: 3 new messages, 1 reply sent" │
|
||||
└─────────────────────────────────────────────────────────────────┘
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Dashboard Data Flow
|
||||
|
||||
How data flows from services to the web dashboard.
|
||||
|
||||
```
|
||||
┌─────────────────────────────────────────────────────────────────┐
|
||||
│ WebServer (Express) │
|
||||
│ │
|
||||
│ ──── REST API ─────────────────────────────────────────────── │
|
||||
│ │
|
||||
│ GET /api/stats │
|
||||
│ ├─► Read from: │
|
||||
│ │ ├─► Bot running state (in-memory) │
|
||||
│ │ ├─► PostsRepository.countProcessed() │
|
||||
│ │ ├─► MessagesRepository.countSent() │
|
||||
│ │ ├─► MessagesRepository.countResponded() │
|
||||
│ │ └─► Calculate response rate │
|
||||
│ └─► Return: { running, processed, sent, responded, rate } │
|
||||
│ │
|
||||
│ GET /api/messages │
|
||||
│ └─► MessagesRepository.getRecent(limit: 50) │
|
||||
│ │
|
||||
│ GET /api/analytics/by-strategy │
|
||||
│ └─► MessagesRepository.getStrategyStats() │
|
||||
│ │
|
||||
│ PUT /api/config │
|
||||
│ ├─► Validate new config values │
|
||||
│ ├─► Update in-memory config │
|
||||
│ └─► Return updated config │
|
||||
│ │
|
||||
│ ──── SSE LOG STREAMING ────────────────────────────────────── │
|
||||
│ │
|
||||
│ GET /api/logs/stream │
|
||||
│ ├─► Client connects │
|
||||
│ ├─► Send any pending log entries │
|
||||
│ ├─► Subscribe to logger buffer │
|
||||
│ ├─► Push new entries as they arrive │
|
||||
│ │ └─► Event: log │
|
||||
│ │ └─► Data: { level, message, timestamp } │
|
||||
│ └─► On disconnect: unsubscribe │
|
||||
│ │
|
||||
│ ──── FRONTEND (public/index.html) ─────────────────────────── │
|
||||
│ │
|
||||
│ JavaScript polls API endpoints: │
|
||||
│ ├─► /api/stats every 5 seconds │
|
||||
│ ├─► /api/messages every 10 seconds │
|
||||
│ ├─► /api/timers every 2 seconds │
|
||||
│ └─► SSE /api/logs/stream (persistent connection) │
|
||||
│ │
|
||||
│ Updates DOM in real-time: │
|
||||
│ ├─► Stats cards (processed, sent, responded, rate) │
|
||||
│ ├─► Activity feed (scrolling list) │
|
||||
│ ├─► Message viewer (click to expand) │
|
||||
│ ├─► Chat browser (conversation list) │
|
||||
│ ├─► Media gallery (image grid) │
|
||||
│ └─► Rate limiter timers (countdown) │
|
||||
└─────────────────────────────────────────────────────────────────┘
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Error Recovery Flows
|
||||
|
||||
### AI Refusal Recovery
|
||||
|
||||
```
|
||||
1. OpenRouter.analyzeImage() returns refusal
|
||||
│
|
||||
├─► Detect refusal via heuristic patterns
|
||||
│ ├─► "cannot analyze"
|
||||
│ ├─► "against policy"
|
||||
│ └─► "unable to process"
|
||||
│
|
||||
├─► Try next model in fallback chain
|
||||
│ ├─► OpenRouter.fallbackModels[0]
|
||||
│ ├─► OpenRouter.fallbackModels[1]
|
||||
│ └─► ...
|
||||
│
|
||||
└─► If all models refuse:
|
||||
├─► Mark post as processed with error
|
||||
├─► Log: "All models refused to analyze post {id}"
|
||||
└─► Continue to next post
|
||||
```
|
||||
|
||||
### Network Error Recovery
|
||||
|
||||
```
|
||||
1. RedditApi.fetchPosts() throws error
|
||||
│
|
||||
└─► withRetry() wrapper handles recovery:
|
||||
├─► Retry 1: wait 5s
|
||||
├─► Retry 2: wait 15s
|
||||
├─► Retry 3: wait 45s
|
||||
└─► After 3 failures:
|
||||
├─► Log error
|
||||
├─► Skip this subreddit
|
||||
└─► Continue to next subreddit
|
||||
```
|
||||
|
||||
### Browser Session Recovery
|
||||
|
||||
```
|
||||
1. BrowserClient.verifySession() returns false
|
||||
│
|
||||
├─► Check for expired cookies
|
||||
│
|
||||
├─► Attempt automated re-login
|
||||
│ ├─► BrowserClient.login()
|
||||
│ └─► If success: continue
|
||||
│
|
||||
└─► If re-login fails:
|
||||
├─► Log error: "Browser session expired"
|
||||
├─► Stop bot
|
||||
└─► Require manual login (scripts/login.ts)
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Data Lifecycle
|
||||
|
||||
### Post Data
|
||||
|
||||
```
|
||||
Discovered (MonitorService)
|
||||
│
|
||||
├─► Stored in processed_posts (pending)
|
||||
│
|
||||
├─► Processed (ProcessorService)
|
||||
│ ├─► image_analysis updated
|
||||
│ ├─► generated_message updated
|
||||
│ └─► message_sent updated
|
||||
│
|
||||
└─► Archived (retention policy TBD)
|
||||
└─► Kept indefinitely for analytics
|
||||
```
|
||||
|
||||
### Message Data
|
||||
|
||||
```
|
||||
Generated (ProcessorService)
|
||||
│
|
||||
├─► Stored in sent_messages (got_response = 0)
|
||||
│
|
||||
├─► Response checked (ResponseCheckerService)
|
||||
│ ├─► got_response = 1
|
||||
│ └─► response_checked_at set
|
||||
│
|
||||
├─► If responded:
|
||||
│ └─► Copied to successful_messages (for learning)
|
||||
│
|
||||
└─► Retained indefinitely for analytics
|
||||
```
|
||||
|
||||
### Learning Data
|
||||
|
||||
```
|
||||
Successful pair created (ResponseCheckerService)
|
||||
│
|
||||
├─► Stored in successful_messages
|
||||
│ ├─► original_message
|
||||
│ ├─► image_analysis
|
||||
│ ├─► post_title
|
||||
│ ├─► response_snippet
|
||||
│ └─► subreddit (for weighted selection)
|
||||
│
|
||||
├─► Retrieved for few-shot learning
|
||||
│ └─► Included in OpenRouter.generateMessage() prompt
|
||||
│
|
||||
└─► Retained indefinitely (grows bot intelligence)
|
||||
```
|
||||
|
||||
### Chat Data
|
||||
|
||||
```
|
||||
Conversation discovered (ChatService)
|
||||
│
|
||||
├─► Cached in memory
|
||||
├─► Persisted to disk JSON
|
||||
│
|
||||
├─► Messages added as they arrive
|
||||
│
|
||||
├─► Media downloaded (MediaService)
|
||||
│ └─► Stored in data/media/
|
||||
│
|
||||
└─► Retained until manually cleared
|
||||
└─► No automatic cleanup (TBD)
|
||||
```
|
||||
|
||||
### Media Data
|
||||
|
||||
```
|
||||
Media URL discovered
|
||||
│
|
||||
├─► Downloaded (MediaService)
|
||||
│ ├─► Matrix thumbnails resolved
|
||||
│ ├─► SHA-1 hash computed
|
||||
│ └─► Saved to data/media/{hash}.{ext}
|
||||
│
|
||||
├─► Indexed in data/media/index.json
|
||||
│
|
||||
├─► Deduplicated (hash check before download)
|
||||
│
|
||||
└─► Retained until manually cleared
|
||||
└─► No automatic cleanup (TBD)
|
||||
```
|
||||
@@ -0,0 +1,550 @@
|
||||
# Database Schema
|
||||
|
||||
Complete reference for the SQLite database schema, migrations, and query patterns.
|
||||
|
||||
---
|
||||
|
||||
## Table of Contents
|
||||
|
||||
1. [Overview](#overview)
|
||||
2. [Database File](#database-file)
|
||||
3. [Schema](#schema)
|
||||
4. [Indexes](#indexes)
|
||||
5. [Migrations](#migrations)
|
||||
6. [Query Patterns](#query-patterns)
|
||||
7. [Repository Methods](#repository-methods)
|
||||
8. [Data Retention](#data-retention)
|
||||
|
||||
---
|
||||
|
||||
## Overview
|
||||
|
||||
RedditBot uses SQLite (via `better-sqlite3`) for persistent storage. The database tracks processed posts, sent messages, and successful message/response pairs.
|
||||
|
||||
**Features**:
|
||||
- WAL mode for better concurrent read performance
|
||||
- Foreign key constraints enabled
|
||||
- Progressive migrations for schema evolution
|
||||
- Repository pattern for data access
|
||||
|
||||
---
|
||||
|
||||
## Database File
|
||||
|
||||
**Default path**: `./data/reddit-bot.db`
|
||||
|
||||
Configured in `config.yaml`:
|
||||
|
||||
```yaml
|
||||
database:
|
||||
path: "./data/reddit-bot.db"
|
||||
```
|
||||
|
||||
The database file is auto-created on first run.
|
||||
|
||||
---
|
||||
|
||||
## Schema
|
||||
|
||||
### Table: `processed_posts`
|
||||
|
||||
Tracks all posts that have been discovered and processed.
|
||||
|
||||
```sql
|
||||
CREATE TABLE processed_posts (
|
||||
id TEXT PRIMARY KEY, -- Reddit post ID
|
||||
subreddit TEXT NOT NULL, -- Source subreddit
|
||||
author TEXT NOT NULL, -- Post author username
|
||||
title TEXT, -- Post title
|
||||
image_url TEXT, -- Original image URL
|
||||
image_analysis TEXT, -- AI-generated JSON description
|
||||
generated_message TEXT, -- AI-generated DM text
|
||||
processed_at INTEGER NOT NULL, -- Unix timestamp
|
||||
message_sent INTEGER DEFAULT 0, -- 0 = not sent, 1 = sent
|
||||
error TEXT -- Error message if processing failed
|
||||
);
|
||||
```
|
||||
|
||||
**Column Details**:
|
||||
|
||||
| Column | Type | Constraints | Description |
|
||||
|---|---|---|---|
|
||||
| `id` | TEXT | PRIMARY KEY | Reddit post ID (e.g., "abc123") |
|
||||
| `subreddit` | TEXT | NOT NULL | Source subreddit name |
|
||||
| `author` | TEXT | NOT NULL | Post author username |
|
||||
| `title` | TEXT | | Post title (nullable if unavailable) |
|
||||
| `image_url` | TEXT | | Original image URL |
|
||||
| `image_analysis` | TEXT | | JSON string from AI analysis |
|
||||
| `generated_message` | TEXT | | The DM text that was generated |
|
||||
| `processed_at` | INTEGER | NOT NULL | Unix timestamp when processed |
|
||||
| `message_sent` | INTEGER | DEFAULT 0 | 0 = not sent, 1 = successfully sent |
|
||||
| `error` | TEXT | | Error message if processing failed |
|
||||
|
||||
**Example Row**:
|
||||
|
||||
```json
|
||||
{
|
||||
"id": "abc123",
|
||||
"subreddit": "gonewild",
|
||||
"author": "user123",
|
||||
"title": "Summer vibes",
|
||||
"image_url": "https://i.redd.it/abc123.jpg",
|
||||
"image_analysis": "{\"gender\":\"F\",\"description\":\"Young woman...\"}",
|
||||
"generated_message": "Hey, you look amazing...",
|
||||
"processed_at": 1716234567,
|
||||
"message_sent": 1,
|
||||
"error": null
|
||||
}
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
### Table: `sent_messages`
|
||||
|
||||
Tracks all DMs sent by the bot.
|
||||
|
||||
```sql
|
||||
CREATE TABLE sent_messages (
|
||||
id INTEGER PRIMARY KEY AUTOINCREMENT,
|
||||
post_id TEXT NOT NULL,
|
||||
recipient TEXT NOT NULL,
|
||||
subject TEXT DEFAULT 'Stunning',
|
||||
message TEXT NOT NULL,
|
||||
sent_at INTEGER NOT NULL,
|
||||
strategy_id TEXT,
|
||||
got_response INTEGER DEFAULT 0,
|
||||
response_checked_at INTEGER,
|
||||
FOREIGN KEY (post_id) REFERENCES processed_posts(id)
|
||||
);
|
||||
```
|
||||
|
||||
**Column Details**:
|
||||
|
||||
| Column | Type | Constraints | Description |
|
||||
|---|---|---|---|
|
||||
| `id` | INTEGER | PRIMARY KEY AUTOINCREMENT | Internal ID |
|
||||
| `post_id` | TEXT | NOT NULL, FK | Source post ID |
|
||||
| `recipient` | TEXT | NOT NULL | Recipient username |
|
||||
| `subject` | TEXT | DEFAULT 'Stunning' | DM subject line |
|
||||
| `message` | TEXT | NOT NULL | DM body text |
|
||||
| `sent_at` | INTEGER | NOT NULL | Unix timestamp when sent |
|
||||
| `strategy_id` | TEXT | | A/B test variant name |
|
||||
| `got_response` | INTEGER | DEFAULT 0 | 0 = no response, 1 = responded |
|
||||
| `response_checked_at` | INTEGER | | Timestamp of last response check |
|
||||
|
||||
**Example Row**:
|
||||
|
||||
```json
|
||||
{
|
||||
"id": 1,
|
||||
"post_id": "abc123",
|
||||
"recipient": "user123",
|
||||
"subject": "Stunning",
|
||||
"message": "Hey, you look amazing...",
|
||||
"sent_at": 1716234567,
|
||||
"strategy_id": "compliment",
|
||||
"got_response": 1,
|
||||
"response_checked_at": 1716238167
|
||||
}
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
### Table: `successful_messages`
|
||||
|
||||
Stores successful message/response pairs for few-shot learning.
|
||||
|
||||
```sql
|
||||
CREATE TABLE successful_messages (
|
||||
id INTEGER PRIMARY KEY AUTOINCREMENT,
|
||||
sent_message_id INTEGER,
|
||||
original_message TEXT NOT NULL,
|
||||
image_analysis TEXT,
|
||||
post_title TEXT,
|
||||
response_snippet TEXT,
|
||||
subreddit TEXT,
|
||||
created_at INTEGER NOT NULL,
|
||||
FOREIGN KEY (sent_message_id) REFERENCES sent_messages(id)
|
||||
);
|
||||
```
|
||||
|
||||
**Column Details**:
|
||||
|
||||
| Column | Type | Constraints | Description |
|
||||
|---|---|---|---|
|
||||
| `id` | INTEGER | PRIMARY KEY AUTOINCREMENT | Internal ID |
|
||||
| `sent_message_id` | INTEGER | FK, nullable | Related sent message ID |
|
||||
| `original_message` | TEXT | NOT NULL | The DM that got a response |
|
||||
| `image_analysis` | TEXT | | Context from image analysis |
|
||||
| `post_title` | TEXT | | Original post title |
|
||||
| `response_snippet` | TEXT | | First reply text |
|
||||
| `subreddit` | TEXT | | For weighted example selection |
|
||||
| `created_at` | INTEGER | NOT NULL | Unix timestamp |
|
||||
|
||||
**Example Row**:
|
||||
|
||||
```json
|
||||
{
|
||||
"id": 1,
|
||||
"sent_message_id": 1,
|
||||
"original_message": "Hey, you look amazing...",
|
||||
"image_analysis": "{\"gender\":\"F\",\"description\":\"...\"}",
|
||||
"post_title": "Summer vibes",
|
||||
"response_snippet": "Thank you so much! 😊",
|
||||
"subreddit": "gonewild",
|
||||
"created_at": 1716238167
|
||||
}
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Indexes
|
||||
|
||||
### `processed_posts` Indexes
|
||||
|
||||
```sql
|
||||
CREATE INDEX idx_posts_subreddit ON processed_posts(subreddit);
|
||||
CREATE INDEX idx_posts_author ON processed_posts(author);
|
||||
```
|
||||
|
||||
**Purpose**: Fast lookup by subreddit (for processing stats) and by author (for skip-checking).
|
||||
|
||||
---
|
||||
|
||||
### `sent_messages` Indexes
|
||||
|
||||
```sql
|
||||
CREATE INDEX idx_messages_recipient ON sent_messages(recipient);
|
||||
CREATE INDEX idx_messages_strategy ON sent_messages(strategy_id);
|
||||
CREATE INDEX idx_messages_response ON sent_messages(got_response);
|
||||
```
|
||||
|
||||
**Purpose**:
|
||||
- `idx_messages_recipient`: Fast lookup when matching responses
|
||||
- `idx_messages_strategy`: Analytics by strategy variant
|
||||
- `idx_messages_response`: Filter for responded/not-responded
|
||||
|
||||
---
|
||||
|
||||
### `successful_messages` Indexes
|
||||
|
||||
```sql
|
||||
CREATE INDEX idx_success_subreddit ON successful_messages(subreddit);
|
||||
CREATE INDEX idx_success_message ON successful_messages(sent_message_id);
|
||||
```
|
||||
|
||||
**Purpose**:
|
||||
- `idx_success_subreddit`: Weighted example selection by subreddit
|
||||
- `idx_success_message`: Link back to sent message
|
||||
|
||||
---
|
||||
|
||||
## Migrations
|
||||
|
||||
Migrations are defined in `src/database/client.ts` and run automatically on every startup.
|
||||
|
||||
### Migration Logic
|
||||
|
||||
```typescript
|
||||
// Pseudo-code for migration process
|
||||
function runMigrations(db: Database) {
|
||||
// Enable features
|
||||
db.exec("PRAGMA journal_mode=WAL;");
|
||||
db.exec("PRAGMA foreign_keys=ON;");
|
||||
|
||||
// Create tables if they don't exist
|
||||
db.exec(`CREATE TABLE IF NOT EXISTS processed_posts (...)`);
|
||||
db.exec(`CREATE TABLE IF NOT EXISTS sent_messages (...)`);
|
||||
db.exec(`CREATE TABLE IF NOT EXISTS successful_messages (...)`);
|
||||
|
||||
// Create indexes if they don't exist
|
||||
db.exec(`CREATE INDEX IF NOT EXISTS idx_posts_subreddit ON processed_posts(subreddit)`);
|
||||
// ... more indexes
|
||||
|
||||
// Safe column additions (for schema evolution)
|
||||
db.exec(`ALTER TABLE processed_posts ADD COLUMN error TEXT`);
|
||||
}
|
||||
```
|
||||
|
||||
### Migration Safety
|
||||
|
||||
- All `CREATE TABLE` statements use `IF NOT EXISTS`
|
||||
- All `CREATE INDEX` statements use `IF NOT EXISTS`
|
||||
- Column additions use `ALTER TABLE ADD COLUMN IF NOT EXISTS` pattern
|
||||
- Migrations are idempotent (safe to run multiple times)
|
||||
|
||||
### Adding New Columns
|
||||
|
||||
When adding new columns to existing tables:
|
||||
|
||||
```typescript
|
||||
// In src/database/client.ts
|
||||
const tableInfo = db.prepare(`PRAGMA table_info(processed_posts)`).all();
|
||||
const hasErrorColumn = tableInfo.some((col: any) => col.name === 'error');
|
||||
|
||||
if (!hasErrorColumn) {
|
||||
db.exec("ALTER TABLE processed_posts ADD COLUMN error TEXT");
|
||||
}
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Query Patterns
|
||||
|
||||
### Post Processing
|
||||
|
||||
**Check if post was already processed**:
|
||||
|
||||
```sql
|
||||
SELECT id FROM processed_posts WHERE id = ?;
|
||||
```
|
||||
|
||||
**Mark post as processed**:
|
||||
|
||||
```sql
|
||||
INSERT OR REPLACE INTO processed_posts
|
||||
(id, subreddit, author, title, image_url, image_analysis,
|
||||
generated_message, processed_at, message_sent, error)
|
||||
VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?, ?);
|
||||
```
|
||||
|
||||
**Mark post as sent**:
|
||||
|
||||
```sql
|
||||
UPDATE processed_posts
|
||||
SET message_sent = 1
|
||||
WHERE id = ?;
|
||||
```
|
||||
|
||||
**Update post with analysis**:
|
||||
|
||||
```sql
|
||||
UPDATE processed_posts
|
||||
SET image_analysis = ?, generated_message = ?
|
||||
WHERE id = ?;
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
### Message Tracking
|
||||
|
||||
**Record a sent message**:
|
||||
|
||||
```sql
|
||||
INSERT INTO sent_messages
|
||||
(post_id, recipient, subject, message, sent_at, strategy_id)
|
||||
VALUES (?, ?, 'Stunning', ?, ?, ?);
|
||||
```
|
||||
|
||||
**Get recent messages**:
|
||||
|
||||
```sql
|
||||
SELECT * FROM sent_messages
|
||||
ORDER BY sent_at DESC
|
||||
LIMIT ?;
|
||||
```
|
||||
|
||||
**Update response status**:
|
||||
|
||||
```sql
|
||||
UPDATE sent_messages
|
||||
SET got_response = 1, response_checked_at = ?
|
||||
WHERE id = ?;
|
||||
```
|
||||
|
||||
**Check if user was previously messaged**:
|
||||
|
||||
```sql
|
||||
SELECT COUNT(*) FROM sent_messages WHERE recipient = ?;
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
### Strategy Analytics
|
||||
|
||||
**Get strategy statistics**:
|
||||
|
||||
```sql
|
||||
SELECT
|
||||
strategy_id,
|
||||
COUNT(*) as sent,
|
||||
SUM(got_response) as responded
|
||||
FROM sent_messages
|
||||
WHERE strategy_id IS NOT NULL
|
||||
GROUP BY strategy_id;
|
||||
```
|
||||
|
||||
**Get response rate by subreddit**:
|
||||
|
||||
```sql
|
||||
SELECT
|
||||
p.subreddit,
|
||||
COUNT(*) as sent,
|
||||
SUM(s.got_response) as responded
|
||||
FROM sent_messages s
|
||||
JOIN processed_posts p ON s.post_id = p.id
|
||||
GROUP BY p.subreddit;
|
||||
```
|
||||
|
||||
**Get daily response trend**:
|
||||
|
||||
```sql
|
||||
SELECT
|
||||
DATE(sent_at, 'unixepoch') as date,
|
||||
COUNT(*) as sent,
|
||||
SUM(got_response) as responded
|
||||
FROM sent_messages
|
||||
GROUP BY DATE(sent_at, 'unixepoch')
|
||||
ORDER BY date DESC
|
||||
LIMIT 30;
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
### Few-Shot Learning
|
||||
|
||||
**Get successful examples for a subreddit**:
|
||||
|
||||
```sql
|
||||
SELECT
|
||||
original_message,
|
||||
image_analysis,
|
||||
post_title,
|
||||
response_snippet
|
||||
FROM successful_messages
|
||||
WHERE subreddit = ?
|
||||
ORDER BY created_at DESC
|
||||
LIMIT 3;
|
||||
```
|
||||
|
||||
**Store successful pair**:
|
||||
|
||||
```sql
|
||||
INSERT INTO successful_messages
|
||||
(sent_message_id, original_message, image_analysis,
|
||||
post_title, response_snippet, subreddit, created_at)
|
||||
VALUES (?, ?, ?, ?, ?, ?, ?);
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Repository Methods
|
||||
|
||||
### PostsRepository
|
||||
|
||||
```typescript
|
||||
class PostsRepository {
|
||||
/** Check if a post was already processed */
|
||||
getById(id: string): ProcessedPost | undefined
|
||||
|
||||
/** Mark a post as processed */
|
||||
insert(post: ProcessedPostInsert): void
|
||||
|
||||
/** Mark a post as having a message sent */
|
||||
markSent(id: string): void
|
||||
|
||||
/** Update post with AI analysis results */
|
||||
updateAnalysis(id: string, analysis: string, message: string): void
|
||||
|
||||
/** Get total count of processed posts */
|
||||
count(): number
|
||||
|
||||
/** Get count by subreddit */
|
||||
countBySubreddit(subreddit: string): number
|
||||
}
|
||||
```
|
||||
|
||||
### MessagesRepository
|
||||
|
||||
```typescript
|
||||
class MessagesRepository {
|
||||
/** Record a sent message */
|
||||
insert(message: MessageInsert): number
|
||||
|
||||
/** Get recent sent messages */
|
||||
getRecent(limit: number): SentMessage[]
|
||||
|
||||
/** Get messages by recipient */
|
||||
getByRecipient(recipient: string): SentMessage[]
|
||||
|
||||
/** Update response status */
|
||||
markResponded(id: number, checkedAt: number): void
|
||||
|
||||
/** Get strategy statistics */
|
||||
getStrategyStats(): StrategyStat[]
|
||||
|
||||
/** Get response rate by subreddit */
|
||||
getSubredditStats(): SubredditStat[]
|
||||
|
||||
/** Get daily response trend */
|
||||
getDailyTrend(days: number): DailyTrend[]
|
||||
|
||||
/** Get total count of sent messages */
|
||||
count(): number
|
||||
|
||||
/** Get count of responded messages */
|
||||
countResponded(): number
|
||||
}
|
||||
```
|
||||
|
||||
### SuccessfulMessagesRepository
|
||||
|
||||
```typescript
|
||||
class SuccessfulMessagesRepository {
|
||||
/** Store a successful message/response pair */
|
||||
insert(pair: SuccessfulMessageInsert): void
|
||||
|
||||
/** Get examples for few-shot learning */
|
||||
getExamples(subreddit: string, count: number): Example[]
|
||||
|
||||
/** Get total count */
|
||||
count(): number
|
||||
|
||||
/** Get recent examples */
|
||||
getRecent(limit: number): SuccessfulMessage[]
|
||||
}
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Data Retention
|
||||
|
||||
### Current Behavior
|
||||
|
||||
All data is retained indefinitely. There is no automatic cleanup or retention policy.
|
||||
|
||||
| Table | Retention | Notes |
|
||||
|---|---|---|
|
||||
| `processed_posts` | Indefinite | Grows with each post processed |
|
||||
| `sent_messages` | Indefinite | Grows with each message sent |
|
||||
| `successful_messages` | Indefinite | Grows with each response received |
|
||||
|
||||
### Estimated Growth
|
||||
|
||||
With default settings (20 messages/hour, 24/7 operation):
|
||||
|
||||
| Timeframe | Posts | Messages | Responses (est. 30%) |
|
||||
|---|---|---|---|
|
||||
| 1 day | ~480 | ~288 | ~86 |
|
||||
| 1 week | ~3,360 | ~2,016 | ~605 |
|
||||
| 1 month | ~14,400 | ~8,640 | ~2,592 |
|
||||
| 1 year | ~175,200 | ~105,120 | ~31,536 |
|
||||
|
||||
### Recommended Cleanup (Future)
|
||||
|
||||
Consider implementing automated cleanup:
|
||||
|
||||
```sql
|
||||
-- Archive old processed posts
|
||||
CREATE TABLE processed_posts_archive AS
|
||||
SELECT * FROM processed_posts
|
||||
WHERE processed_at < ?;
|
||||
|
||||
-- Delete archived posts
|
||||
DELETE FROM processed_posts
|
||||
WHERE processed_at < ?;
|
||||
|
||||
-- Keep last N months of data
|
||||
```
|
||||
|
||||
> **Note**: Successful messages should NEVER be deleted as they are used for few-shot learning.
|
||||
@@ -0,0 +1,413 @@
|
||||
# Deployment Guide
|
||||
|
||||
Complete guide for deploying RedditBot in various environments.
|
||||
|
||||
---
|
||||
|
||||
## Table of Contents
|
||||
|
||||
1. [Prerequisites](#prerequisites)
|
||||
2. [Local Development](#local-development)
|
||||
3. [Docker Deployment](#docker-deployment)
|
||||
4. [Docker Compose](#docker-compose)
|
||||
5. [Dokploy Deployment](#dokploy-deployment)
|
||||
6. [Production Considerations](#production-considerations)
|
||||
7. [Monitoring & Maintenance](#monitoring--maintenance)
|
||||
8. [Troubleshooting](#troubleshooting-1)
|
||||
|
||||
---
|
||||
|
||||
## Prerequisites
|
||||
|
||||
### Required
|
||||
|
||||
| Requirement | Version | Notes |
|
||||
|---|---|---|
|
||||
| Node.js | 18+ | LTS recommended |
|
||||
| npm | 9+ | Or yarn/pnpm |
|
||||
| OpenRouter API Key | - | Sign up at [openrouter.ai](https://openrouter.ai) |
|
||||
| Reddit Account | - | With DM permissions enabled |
|
||||
|
||||
### Recommended
|
||||
|
||||
| Requirement | Notes |
|
||||
|---|---|
|
||||
| 1 GB RAM | Minimum for browser automation |
|
||||
| 1 CPU core | More for higher message throughput |
|
||||
| 5 GB disk | For database, media, and browser data |
|
||||
|
||||
---
|
||||
|
||||
## Local Development
|
||||
|
||||
### 1. Clone and Install
|
||||
|
||||
```bash
|
||||
git clone <repository-url>
|
||||
cd RedditBot
|
||||
npm install
|
||||
```
|
||||
|
||||
### 2. Configure
|
||||
|
||||
Create `config.yaml`:
|
||||
|
||||
```yaml
|
||||
reddit:
|
||||
subreddits:
|
||||
- gonewild
|
||||
pollInterval: 15
|
||||
targetGender: "F"
|
||||
sellerKeywords:
|
||||
- seller
|
||||
- onlyfans
|
||||
|
||||
openrouter:
|
||||
apiKey: "sk-or-v1-your-key-here"
|
||||
model: "meta-llama/llama-3.2-90b-vision-preview"
|
||||
fallbackModels:
|
||||
- "google/gemini-2.0-flash-001"
|
||||
|
||||
rateLimit:
|
||||
minDelayMs: 30000
|
||||
maxMessagesPerHour: 20
|
||||
jitterMs: 15000
|
||||
|
||||
browser:
|
||||
headless: true
|
||||
userDataDir: "./data/browser"
|
||||
|
||||
database:
|
||||
path: "./data/reddit-bot.db"
|
||||
|
||||
web:
|
||||
port: 3000
|
||||
|
||||
dryRun: true # Start in dry run mode
|
||||
```
|
||||
|
||||
Set environment variables:
|
||||
|
||||
```bash
|
||||
export REDDIT_USERNAME="your-username"
|
||||
export REDDIT_PASSWORD="your-password"
|
||||
```
|
||||
|
||||
### 3. Initial Login
|
||||
|
||||
Run the login script to establish a browser session:
|
||||
|
||||
```bash
|
||||
npm run login
|
||||
```
|
||||
|
||||
This will:
|
||||
1. Open a browser window
|
||||
2. Navigate to Reddit login
|
||||
3. Wait for you to complete login (including 2FA if enabled)
|
||||
4. Save cookies to `data/browser/`
|
||||
|
||||
### 4. Start the Bot
|
||||
|
||||
```bash
|
||||
# Development (with TypeScript execution)
|
||||
npm run dev
|
||||
|
||||
# Production (compiled JavaScript)
|
||||
npm start
|
||||
```
|
||||
|
||||
### 5. Access Dashboard
|
||||
|
||||
Open http://localhost:3000 in your browser.
|
||||
|
||||
---
|
||||
|
||||
## Docker Deployment
|
||||
|
||||
### Build the Image
|
||||
|
||||
```bash
|
||||
docker build -t reddit-bot .
|
||||
```
|
||||
|
||||
### Run the Container
|
||||
|
||||
```bash
|
||||
docker run -d \
|
||||
--name reddit-bot \
|
||||
-p 3000:3000 \
|
||||
-v $(pwd)/data:/app/data \
|
||||
-e OPENROUTER_API_KEY=sk-or-v1-your-key \
|
||||
-e REDDIT_USERNAME=your-username \
|
||||
-e REDDIT_PASSWORD=your-password \
|
||||
-e BROWSER_HEADLESS=true \
|
||||
reddit-bot:latest
|
||||
```
|
||||
|
||||
### Dockerfile Structure
|
||||
|
||||
```dockerfile
|
||||
# Build stage
|
||||
FROM node:18-alpine AS builder
|
||||
WORKDIR /app
|
||||
COPY package*.json ./
|
||||
RUN npm ci
|
||||
COPY . .
|
||||
RUN npm run build
|
||||
|
||||
# Production stage
|
||||
FROM node:18-alpine
|
||||
WORKDIR /app
|
||||
COPY --from=builder /app/dist ./dist
|
||||
COPY --from=builder /app/node_modules ./node_modules
|
||||
COPY --from=builder /app/config.yaml ./config.yaml
|
||||
COPY --from=builder /app/package.json ./package.json
|
||||
|
||||
EXPOSE 3000
|
||||
CMD ["node", "dist/index.js"]
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Docker Compose
|
||||
|
||||
### docker-compose.yml
|
||||
|
||||
```yaml
|
||||
version: '3.8'
|
||||
|
||||
services:
|
||||
reddit-bot:
|
||||
build: .
|
||||
container_name: reddit-bot
|
||||
ports:
|
||||
- "3000:3000"
|
||||
volumes:
|
||||
- ./data:/app/data
|
||||
environment:
|
||||
- OPENROUTER_API_KEY=${OPENROUTER_API_KEY}
|
||||
- REDDIT_USERNAME=${REDDIT_USERNAME}
|
||||
- REDDIT_PASSWORD=${REDDIT_PASSWORD}
|
||||
- BROWSER_HEADLESS=true
|
||||
restart: unless-stopped
|
||||
healthcheck:
|
||||
test: ["CMD", "wget", "--spider", "http://localhost:3000/api/stats"]
|
||||
interval: 30s
|
||||
timeout: 10s
|
||||
retries: 3
|
||||
start_period: 60s
|
||||
```
|
||||
|
||||
### Start with Compose
|
||||
|
||||
```bash
|
||||
# Set environment variables
|
||||
cp .env.example .env
|
||||
# Edit .env with your credentials
|
||||
|
||||
# Build and start
|
||||
docker compose up -d --build
|
||||
|
||||
# View logs
|
||||
docker compose logs -f reddit-bot
|
||||
|
||||
# Stop
|
||||
docker compose down
|
||||
|
||||
# Restart
|
||||
docker compose restart
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Dokploy Deployment
|
||||
|
||||
Dokploy is a self-hosted platform for deploying applications. See `DOKPLOY_DEPLOY.md` for detailed instructions.
|
||||
|
||||
### Quick Start
|
||||
|
||||
1. **Create Application**: In Dokploy dashboard, create a new Docker application
|
||||
2. **Set Environment Variables**:
|
||||
- `OPENROUTER_API_KEY`
|
||||
- `REDDIT_USERNAME`
|
||||
- `REDDIT_PASSWORD`
|
||||
- `BROWSER_HEADLESS`
|
||||
3. **Mount Volume**: Mount `./data` for persistent storage
|
||||
4. **Set Port**: Map container port 3000 to host port
|
||||
5. **Deploy**: Push to repository and trigger deployment
|
||||
|
||||
### Dokploy Configuration
|
||||
|
||||
```yaml
|
||||
# Dokploy-specific settings
|
||||
app:
|
||||
type: docker
|
||||
repository: <your-repo>
|
||||
branch: main
|
||||
dockerfilePath: Dockerfile
|
||||
|
||||
environment:
|
||||
OPENROUTER_API_KEY: $OPENROUTER_API_KEY
|
||||
REDDIT_USERNAME: $REDDIT_USERNAME
|
||||
REDDIT_PASSWORD: $REDDIT_PASSWORD
|
||||
BROWSER_HEADLESS: "true"
|
||||
|
||||
ports:
|
||||
- container: 3000
|
||||
host: 3000
|
||||
|
||||
volumes:
|
||||
- source: reddit-bot-data
|
||||
target: /app/data
|
||||
type: bind
|
||||
|
||||
restartPolicy: unless-stopped
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Production Considerations
|
||||
|
||||
### Security
|
||||
|
||||
1. **Network Isolation**: Run behind a reverse proxy with authentication
|
||||
2. **Environment Variables**: Never commit credentials; use secrets management
|
||||
3. **Dashboard Access**: Restrict to internal network or add auth
|
||||
4. **Browser Data**: Protect `data/browser/` directory
|
||||
|
||||
### Performance
|
||||
|
||||
1. **Browser Headless**: Always use headless mode in production
|
||||
2. **WAL Mode**: SQLite WAL mode is enabled by default for concurrent reads
|
||||
3. **Rate Limiting**: Adjust based on Reddit account age and trust level
|
||||
4. **Message Volume**: Start low (10-20/hour) and increase gradually
|
||||
|
||||
### Reliability
|
||||
|
||||
1. **Auto-restart**: Use `restart: unless-stopped` in Docker
|
||||
2. **Health Checks**: Monitor `/api/stats` endpoint
|
||||
3. **Logging**: Review logs regularly for errors
|
||||
4. **Backups**: Regularly backup `data/reddit-bot.db`
|
||||
|
||||
### Monitoring
|
||||
|
||||
1. **Dashboard**: Check web dashboard for real-time stats
|
||||
2. **Response Rate**: Monitor response rates per strategy/subreddit
|
||||
3. **Error Rate**: Watch for AI refusals, send failures
|
||||
4. **Browser Health**: Verify session isn't expired
|
||||
|
||||
### Scaling
|
||||
|
||||
RedditBot is designed as a single-instance application. For scaling:
|
||||
|
||||
1. **Multiple Accounts**: Run separate instances per Reddit account
|
||||
2. **Subreddit Split**: Distribute subreddits across instances
|
||||
3. **Centralized DB**: Consider moving to PostgreSQL for multi-instance
|
||||
|
||||
---
|
||||
|
||||
## Monitoring & Maintenance
|
||||
|
||||
### Daily Checks
|
||||
|
||||
- [ ] Dashboard shows bot is running
|
||||
- [ ] Response rate is reasonable (>10%)
|
||||
- [ ] No repeated errors in logs
|
||||
- [ ] Browser session is active
|
||||
|
||||
### Weekly Tasks
|
||||
|
||||
- [ ] Review analytics (strategy performance, subreddit performance)
|
||||
- [ ] Check disk usage (media files, database)
|
||||
- [ ] Verify backup is working
|
||||
- [ ] Review successful messages for quality
|
||||
|
||||
### Monthly Tasks
|
||||
|
||||
- [ ] Backup database
|
||||
- [ ] Review and update strategy variants
|
||||
- [ ] Check for OpenRouter model updates
|
||||
- [ ] Review and update seller keywords
|
||||
- [ ] Verify browser session is still valid
|
||||
|
||||
### Backup Procedure
|
||||
|
||||
```bash
|
||||
# Stop the bot
|
||||
docker compose stop
|
||||
|
||||
# Backup database
|
||||
cp data/reddit-bot.db data/reddit-bot.db.backup.$(date +%Y%m%d)
|
||||
|
||||
# Backup browser session (optional)
|
||||
tar czf data/browser.backup.$(date +%Y%m%d).tar.gz data/browser/
|
||||
|
||||
# Restart the bot
|
||||
docker compose start
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Troubleshooting
|
||||
|
||||
### Bot Won't Start
|
||||
|
||||
**Symptom**: Bot exits immediately on startup.
|
||||
|
||||
**Steps**:
|
||||
1. Check logs: `docker compose logs reddit-bot`
|
||||
2. Verify `config.yaml` syntax
|
||||
3. Verify environment variables are set
|
||||
4. Check for Zod validation errors in logs
|
||||
|
||||
### Browser Login Fails
|
||||
|
||||
**Symptom**: "Browser session expired" or login fails.
|
||||
|
||||
**Steps**:
|
||||
1. Delete browser data: `rm -rf data/browser/`
|
||||
2. Re-run login: `npm run login`
|
||||
3. Check Reddit account is not locked
|
||||
4. Try `BROWSER_HEADLESS=false` for debugging
|
||||
|
||||
### AI Analysis Fails
|
||||
|
||||
**Symptom**: All messages show "AI refusal" or analysis errors.
|
||||
|
||||
**Steps**:
|
||||
1. Verify API key is valid and has balance
|
||||
2. Check model availability on OpenRouter
|
||||
3. Try changing to a different model
|
||||
4. Review logs for specific error messages
|
||||
|
||||
### High Error Rate
|
||||
|
||||
**Symptom**: Many messages fail to send.
|
||||
|
||||
**Steps**:
|
||||
1. Check rate limiting (reduce maxMessagesPerHour)
|
||||
2. Increase minDelayMs and jitterMs
|
||||
3. Check for IP bans (try different network)
|
||||
4. Review Reddit account standing
|
||||
|
||||
### Dashboard Not Accessible
|
||||
|
||||
**Symptom**: Cannot reach http://localhost:3000.
|
||||
|
||||
**Steps**:
|
||||
1. Verify bot is running (`docker compose ps`)
|
||||
2. Check port is not in use: `lsof -i :3000`
|
||||
3. Verify firewall allows port 3000
|
||||
4. Check web.port in config.yaml
|
||||
|
||||
### Database Issues
|
||||
|
||||
**Symptom**: Database errors or corruption.
|
||||
|
||||
**Steps**:
|
||||
1. Stop the bot
|
||||
2. Backup the database
|
||||
3. Try opening with SQLite viewer to check integrity
|
||||
4. If corrupted, restore from backup
|
||||
5. Consider enabling automatic backups
|
||||
@@ -0,0 +1,514 @@
|
||||
# Development Guide
|
||||
|
||||
Guide for contributing to RedditBot development.
|
||||
|
||||
---
|
||||
|
||||
## Table of Contents
|
||||
|
||||
1. [Project Structure](#project-structure)
|
||||
2. [Development Setup](#development-setup)
|
||||
3. [Coding Standards](#coding-standards)
|
||||
4. [Adding a New Service](#adding-a-new-service)
|
||||
5. [Adding a New Strategy Variant](#adding-a-new-strategy-variant)
|
||||
6. [Debugging](#debugging)
|
||||
7. [Common Tasks](#common-tasks)
|
||||
8. [Known Issues](#known-issues)
|
||||
|
||||
---
|
||||
|
||||
## Project Structure
|
||||
|
||||
```
|
||||
RedditBot/
|
||||
├── config.yaml # Main configuration
|
||||
├── package.json # Dependencies and scripts
|
||||
├── tsconfig.json # TypeScript config
|
||||
├── Dockerfile # Container build
|
||||
├── docker-compose.yml # Docker Compose
|
||||
├── public/
|
||||
│ └── index.html # Web dashboard (single file)
|
||||
├── scripts/
|
||||
│ ├── login.ts # Manual login helper
|
||||
│ └── inspect-chat-dom.ts # DOM inspection debug script
|
||||
└── src/
|
||||
├── index.ts # Entry point / orchestrator
|
||||
├── config/
|
||||
│ └── index.ts # Config loader + Zod schema
|
||||
├── database/
|
||||
│ ├── client.ts # SQLite setup + migrations
|
||||
│ └── repositories.ts # Repository classes
|
||||
├── reddit/
|
||||
│ └── api.ts # Reddit API client
|
||||
├── ai/
|
||||
│ └── openrouter.ts # OpenRouter AI client
|
||||
├── browser/
|
||||
│ ├── client.ts # Playwright browser manager
|
||||
│ ├── message-page.ts # DM compose automation
|
||||
│ └── media-downloader.ts # Media download
|
||||
├── services/
|
||||
│ ├── monitor.ts # Post discovery
|
||||
│ ├── processor.ts # Processing pipeline
|
||||
│ ├── response-checker.ts # Response monitoring
|
||||
│ ├── strategy-selector.ts # A/B testing
|
||||
│ ├── chat.ts # Chat scraping + messaging
|
||||
│ ├── media.ts # Media management
|
||||
│ ├── media-sync.ts # Media sync scheduler
|
||||
│ └── auto-reply.ts # Auto-reply service
|
||||
├── utils/
|
||||
│ ├── logger.ts # Winston logger
|
||||
│ ├── rate-limiter.ts # Rate limiting
|
||||
│ ├── retry.ts # Retry wrapper
|
||||
│ └── image.ts # Image utilities
|
||||
└── web/
|
||||
└── server.ts # Express dashboard server
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Development Setup
|
||||
|
||||
### 1. Install Dependencies
|
||||
|
||||
```bash
|
||||
npm install
|
||||
```
|
||||
|
||||
### 2. Configure
|
||||
|
||||
Copy and edit `config.yaml`:
|
||||
|
||||
```bash
|
||||
cp config.yaml config.yaml.backup # Backup existing
|
||||
# Edit config.yaml with your settings
|
||||
```
|
||||
|
||||
### 3. Set Environment Variables
|
||||
|
||||
```bash
|
||||
export REDDIT_USERNAME="your-username"
|
||||
export REDDIT_PASSWORD="your-password"
|
||||
export OPENROUTER_API_KEY="sk-or-v1-your-key"
|
||||
```
|
||||
|
||||
### 4. Initial Login
|
||||
|
||||
```bash
|
||||
# Open browser for manual login
|
||||
npm run login
|
||||
```
|
||||
|
||||
### 5. Start Development Server
|
||||
|
||||
```bash
|
||||
# Development mode (hot reload with tsx)
|
||||
npm run dev
|
||||
|
||||
# Production mode (compiled JS)
|
||||
npm run build && npm start
|
||||
```
|
||||
|
||||
### 6. Access Dashboard
|
||||
|
||||
Open http://localhost:3000
|
||||
|
||||
---
|
||||
|
||||
## Coding Standards
|
||||
|
||||
### TypeScript
|
||||
|
||||
- **Strict mode**: No `any` types (use `unknown` with type guards)
|
||||
- **Interfaces**: Define interfaces for all data structures
|
||||
- **Null checks**: Use optional chaining (`?.`) and nullish coalescing (`??`)
|
||||
- **Async/Await**: Use async/await, not `.then()` chains
|
||||
|
||||
```typescript
|
||||
// Good
|
||||
interface PostData {
|
||||
id: string;
|
||||
subreddit: string;
|
||||
author: string;
|
||||
}
|
||||
|
||||
async function fetchPost(id: string): Promise<PostData> {
|
||||
const response = await api.get(`/posts/${id}`);
|
||||
return response.data;
|
||||
}
|
||||
|
||||
// Bad
|
||||
async function fetchPost(id: any): Promise<any> {
|
||||
return api.get(`/posts/${id}`).then(r => r.data);
|
||||
}
|
||||
```
|
||||
|
||||
### Logging
|
||||
|
||||
Use `winston` logger, never `console.log`:
|
||||
|
||||
```typescript
|
||||
// Good
|
||||
this.logger.info("Processed post {id}", { id: post.id });
|
||||
this.logger.warn("AI refusal detected", { postId: post.id });
|
||||
this.logger.error("Failed to send DM", { error: err.message });
|
||||
|
||||
// Bad
|
||||
console.log("Processed post", post.id);
|
||||
```
|
||||
|
||||
### Error Handling
|
||||
|
||||
Use try/catch with structured error handling:
|
||||
|
||||
```typescript
|
||||
try {
|
||||
await this.processPost(post);
|
||||
} catch (err) {
|
||||
this.logger.error("Post processing failed", {
|
||||
postId: post.id,
|
||||
error: err instanceof Error ? err.message : String(err)
|
||||
});
|
||||
// Don't throw - allow other posts to be processed
|
||||
}
|
||||
```
|
||||
|
||||
### Service Structure
|
||||
|
||||
Follow single responsibility:
|
||||
|
||||
```typescript
|
||||
class NewService {
|
||||
constructor(
|
||||
private config: Config,
|
||||
private dependency: ExistingService,
|
||||
private logger: Logger
|
||||
) {}
|
||||
|
||||
async doSomething(input: string): Promise<Result> {
|
||||
// Validate input
|
||||
if (!input) {
|
||||
throw new Error("Input is required");
|
||||
}
|
||||
|
||||
// Do work
|
||||
const result = await this.dependency.process(input);
|
||||
|
||||
// Return result
|
||||
return result;
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
### File Organization
|
||||
|
||||
- One class per file (except repositories which share one file)
|
||||
- Services in `src/services/`
|
||||
- Utilities in `src/utils/`
|
||||
- External integrations in their own folders (`src/ai/`, `src/reddit/`, `src/browser/`)
|
||||
|
||||
---
|
||||
|
||||
## Adding a New Service
|
||||
|
||||
### 1. Create the Service File
|
||||
|
||||
```typescript
|
||||
// src/services/new-service.ts
|
||||
import { Config } from '../config';
|
||||
import { Logger } from '../utils/logger';
|
||||
|
||||
interface NewServiceDeps {
|
||||
config: Config;
|
||||
logger: Logger;
|
||||
// other dependencies
|
||||
}
|
||||
|
||||
interface Result {
|
||||
success: boolean;
|
||||
data?: any;
|
||||
error?: string;
|
||||
}
|
||||
|
||||
export class NewService {
|
||||
constructor(private deps: NewServiceDeps) {}
|
||||
|
||||
async execute(input: string): Promise<Result> {
|
||||
const { config, logger } = this.deps;
|
||||
|
||||
logger.info("NewService executing", { input });
|
||||
|
||||
// Implementation here
|
||||
|
||||
return { success: true };
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
### 2. Add to Dependencies Interface
|
||||
|
||||
If the service needs other services, add them to the interface:
|
||||
|
||||
```typescript
|
||||
interface NewServiceDeps {
|
||||
config: Config;
|
||||
logger: Logger;
|
||||
redditApi: RedditApi;
|
||||
openrouter: OpenRouterService;
|
||||
}
|
||||
```
|
||||
|
||||
### 3. Wire Up in main()
|
||||
|
||||
In `src/index.ts`, add initialization:
|
||||
|
||||
```typescript
|
||||
const newService = new NewService({
|
||||
config,
|
||||
logger,
|
||||
redditApi,
|
||||
openrouter
|
||||
});
|
||||
|
||||
// If it needs to run periodically:
|
||||
setInterval(() => {
|
||||
newService.execute(someInput).catch(err => {
|
||||
logger.error("NewService error", { error: err.message });
|
||||
});
|
||||
}, intervalMs);
|
||||
```
|
||||
|
||||
### 4. Add to Web Dashboard (Optional)
|
||||
|
||||
If the service needs API endpoints, add them in `src/web/server.ts`:
|
||||
|
||||
```typescript
|
||||
server.app.get('/api/new-service', async (req, res) => {
|
||||
try {
|
||||
const result = await newService.execute(req.query.input as string);
|
||||
res.json(result);
|
||||
} catch (err) {
|
||||
res.status(500).json({ error: (err as Error).message });
|
||||
}
|
||||
});
|
||||
```
|
||||
|
||||
### 5. Update Config (If Needed)
|
||||
|
||||
Add new configuration options to:
|
||||
- `config.yaml`
|
||||
- `src/config/index.ts` Zod schema
|
||||
- Configuration documentation
|
||||
|
||||
---
|
||||
|
||||
## Adding a New Strategy Variant
|
||||
|
||||
### 1. Add to config.yaml
|
||||
|
||||
```yaml
|
||||
strategyVariants:
|
||||
- name: "existing_variant"
|
||||
prompt: "Existing prompt..."
|
||||
enabled: true
|
||||
|
||||
- name: "new_variant"
|
||||
prompt: |
|
||||
Write a friendly message that references the image content.
|
||||
Be specific and genuine. Keep it under 2 sentences.
|
||||
enabled: true
|
||||
```
|
||||
|
||||
### 2. No Code Changes Needed
|
||||
|
||||
The `StrategySelector` automatically picks up new variants from the config. The algorithm will:
|
||||
|
||||
1. Include the new variant in random exploration
|
||||
2. Track its response rate once messages are sent
|
||||
3. Promote it to exploitation if it has good stats
|
||||
|
||||
### 3. Monitor Performance
|
||||
|
||||
Check performance via the dashboard or API:
|
||||
|
||||
```bash
|
||||
curl http://localhost:3000/api/analytics/by-strategy
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Debugging
|
||||
|
||||
### Browser Debugging
|
||||
|
||||
Run with visible browser:
|
||||
|
||||
```yaml
|
||||
# config.yaml
|
||||
browser:
|
||||
headless: false
|
||||
```
|
||||
|
||||
Or set environment variable:
|
||||
|
||||
```bash
|
||||
BROWSER_HEADLESS=false npm run dev
|
||||
```
|
||||
|
||||
### DOM Inspection
|
||||
|
||||
Use the debug endpoint to inspect Reddit's shadow DOM:
|
||||
|
||||
```bash
|
||||
curl -X POST http://localhost:3000/api/debug/inspect-dom
|
||||
```
|
||||
|
||||
Or run the standalone script:
|
||||
|
||||
```bash
|
||||
npx tsx scripts/inspect-chat-dom.ts
|
||||
```
|
||||
|
||||
### Logging
|
||||
|
||||
Adjust log level by modifying `src/utils/logger.ts`:
|
||||
|
||||
```typescript
|
||||
const logger = winston.createLogger({
|
||||
level: 'debug', // Change from 'info' to 'debug'
|
||||
// ...
|
||||
});
|
||||
```
|
||||
|
||||
### Database Inspection
|
||||
|
||||
Open the database with SQLite viewer:
|
||||
|
||||
```bash
|
||||
# Install sqlite3 if needed
|
||||
brew install sqlite3
|
||||
|
||||
# Open database
|
||||
sqlite3 data/reddit-bot.db
|
||||
|
||||
# Query
|
||||
SELECT * FROM sent_messages LIMIT 5;
|
||||
```
|
||||
|
||||
### Network Debugging
|
||||
|
||||
Enable axios debug:
|
||||
|
||||
```typescript
|
||||
// In src/reddit/api.ts
|
||||
import axios from 'axios';
|
||||
axios.defaults.debug = true;
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Common Tasks
|
||||
|
||||
### Add a New Subreddit
|
||||
|
||||
```yaml
|
||||
# config.yaml
|
||||
reddit:
|
||||
subreddits:
|
||||
- gonewild
|
||||
- ratemyboobs
|
||||
- new_subreddit_name # Add here
|
||||
```
|
||||
|
||||
### Change AI Model
|
||||
|
||||
```yaml
|
||||
# config.yaml
|
||||
openrouter:
|
||||
model: "new-model-name"
|
||||
fallbackModels:
|
||||
- "fallback-model-1"
|
||||
- "fallback-model-2"
|
||||
```
|
||||
|
||||
### Adjust Rate Limits
|
||||
|
||||
```yaml
|
||||
# config.yaml
|
||||
rateLimit:
|
||||
minDelayMs: 45000 # Increase delay
|
||||
maxMessagesPerHour: 15 # Reduce cap
|
||||
jitterMs: 20000 # More jitter
|
||||
```
|
||||
|
||||
### Enable Dry Run
|
||||
|
||||
```yaml
|
||||
# config.yaml
|
||||
dryRun: true # Logs all actions without sending
|
||||
```
|
||||
|
||||
Or via dashboard: `PUT /api/config` with `{"dryRun": true}`
|
||||
|
||||
### Add Database Column
|
||||
|
||||
```typescript
|
||||
// In src/database/client.ts
|
||||
function runMigrations(db: Database) {
|
||||
// ... existing migrations ...
|
||||
|
||||
// Check if column exists
|
||||
const tableInfo = db.prepare(`PRAGMA table_info(sent_messages)`).all();
|
||||
const hasNewColumn = tableInfo.some((col: any) => col.name === 'new_column');
|
||||
|
||||
if (!hasNewColumn) {
|
||||
db.exec("ALTER TABLE sent_messages ADD COLUMN new_column TEXT");
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
### Add New API Endpoint
|
||||
|
||||
```typescript
|
||||
// In src/web/server.ts
|
||||
import { Request, Response } from 'express';
|
||||
|
||||
server.app.get('/api/new-endpoint', async (req: Request, res: Response) => {
|
||||
try {
|
||||
const result = await someService.doSomething();
|
||||
res.json({ success: true, data: result });
|
||||
} catch (err) {
|
||||
res.status(500).json({ error: (err as Error).message });
|
||||
}
|
||||
});
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Known Issues
|
||||
|
||||
### Hardcoded Username in ChatService
|
||||
|
||||
`ChatService` has `myUsername = 'Upset_Marsupial1006'` hardcoded in JavaScript evaluation scripts. This should be parameterized from config.
|
||||
|
||||
**Fix needed**: Pass username to `page.evaluate()` calls.
|
||||
|
||||
### Shadow DOM Fragility
|
||||
|
||||
Reddit frequently updates their `faceplate-*` components. Shadow DOM traversal code may break without warning.
|
||||
|
||||
**Mitigation**: Use `scripts/inspect-chat-dom.ts` to debug when selectors change.
|
||||
|
||||
### No Test Coverage
|
||||
|
||||
The project has no automated tests. Any changes should be manually tested.
|
||||
|
||||
**Recommendation**: Add tests for critical paths:
|
||||
- StrategySelector algorithm
|
||||
- ChatService shadow DOM parsing
|
||||
- OpenRouterService model fallback
|
||||
|
||||
### Single-File Dashboard
|
||||
|
||||
The entire dashboard is in `public/index.html` (~2000+ lines). Making changes requires editing a single large file.
|
||||
|
||||
**Recommendation**: Consider migrating to a proper frontend framework for maintainability.
|
||||
@@ -0,0 +1,854 @@
|
||||
# Service Catalog
|
||||
|
||||
Detailed documentation for each service in the RedditBot codebase.
|
||||
|
||||
---
|
||||
|
||||
## Table of Contents
|
||||
|
||||
1. [MonitorService](#monitorservice)
|
||||
2. [ProcessorService](#processorservice)
|
||||
3. [StrategySelector](#strategypicker)
|
||||
4. [ResponseChecker](#responsechecker)
|
||||
5. [ChatService](#chatservice)
|
||||
6. [MediaService](#mediaservice)
|
||||
7. [MediaSyncService](#mediasyncservice)
|
||||
8. [AutoReplyService](#autoreplyservice)
|
||||
9. [RedditApi](#redditapi)
|
||||
10. [OpenRouterService](#openrouterservice)
|
||||
11. [BrowserClient](#browserclient)
|
||||
12. [MessagePage](#messagepage)
|
||||
13. [RateLimiter](#ratelimiter)
|
||||
|
||||
---
|
||||
|
||||
## MonitorService
|
||||
|
||||
**File**: `src/services/monitor.ts`
|
||||
**Lines**: 72
|
||||
**Responsibility**: Discover new image posts across configured subreddits
|
||||
|
||||
### Purpose
|
||||
|
||||
Polls configured subreddits at regular intervals, discovers new image posts, filters out already-processed posts and previously-messaged users, and returns the single newest valid post for processing.
|
||||
|
||||
### Public Methods
|
||||
|
||||
```typescript
|
||||
class MonitorService {
|
||||
constructor(
|
||||
private redditApi: RedditApi,
|
||||
private postsRepo: PostsRepository,
|
||||
private messagesRepo: MessagesRepository,
|
||||
private config: Config
|
||||
)
|
||||
|
||||
/**
|
||||
* Discover the next post to process.
|
||||
* Returns null if no valid posts are available.
|
||||
*/
|
||||
discoverNextPost(): Promise<Post | null>
|
||||
}
|
||||
```
|
||||
|
||||
### Algorithm
|
||||
|
||||
1. Shuffle subreddit order (randomize polling order each cycle)
|
||||
2. For each subreddit:
|
||||
- Fetch new posts via `RedditApi.fetchPosts()`
|
||||
- Filter: image posts only
|
||||
- Filter: author not deleted
|
||||
- Filter: post not already in `processed_posts`
|
||||
- Filter: author not previously messaged (or meets re-message criteria)
|
||||
3. Return the newest valid post across all subreddits
|
||||
4. If no valid posts found, return `null`
|
||||
|
||||
### Dependencies
|
||||
|
||||
- `RedditApi` - Fetch posts from Reddit
|
||||
- `PostsRepository` - Check for already-processed posts
|
||||
- `MessagesRepository` - Check for previously-messaged users
|
||||
- `Config` - Subreddit list, poll interval, target gender
|
||||
|
||||
### Error Handling
|
||||
|
||||
- Skips subreddits that return errors (continues to next)
|
||||
- Logs errors at `warn` level
|
||||
- Returns `null` on any unrecoverable error
|
||||
|
||||
---
|
||||
|
||||
## ProcessorService
|
||||
|
||||
**File**: `src/services/processor.ts`
|
||||
**Lines**: 132
|
||||
**Responsibility**: Core processing pipeline for discovered posts
|
||||
|
||||
### Purpose
|
||||
|
||||
Orchestrates the full processing pipeline for a single discovered post: image download, gender detection, AI analysis, strategy selection, message generation, and DM sending.
|
||||
|
||||
### Public Methods
|
||||
|
||||
```typescript
|
||||
class ProcessorService {
|
||||
constructor(
|
||||
private config: Config,
|
||||
private redditApi: RedditApi,
|
||||
private openrouter: OpenRouterService,
|
||||
private strategySelector: StrategySelector,
|
||||
private messageSender: MessagePage,
|
||||
private postsRepo: PostsRepository,
|
||||
private messagesRepo: MessagesRepository,
|
||||
private successfulRepo: SuccessfulMessagesRepository,
|
||||
private logger: Logger
|
||||
)
|
||||
|
||||
/**
|
||||
* Process a single discovered post through the full pipeline.
|
||||
*/
|
||||
processPost(post: Post): Promise<void>
|
||||
}
|
||||
```
|
||||
|
||||
### Pipeline Steps
|
||||
|
||||
```
|
||||
1. Download image
|
||||
└─► ImageService.downloadImage(url)
|
||||
└─► Validate MIME type (image/jpeg, image/png, image/webp)
|
||||
└─► Validate file size
|
||||
└─► Convert to base64
|
||||
|
||||
2. Quick gender detection (cost-saving pre-filter)
|
||||
└─► OpenRouter.detectGenderOnly(image)
|
||||
└─► Skip if gender doesn't match targetGender
|
||||
|
||||
3. Select A/B strategy variant
|
||||
└─► StrategySelector.selectStrategy(subreddit, author)
|
||||
|
||||
4. Full AI analysis
|
||||
└─► OpenRouter.analyzeImage(image)
|
||||
└─► Detect gender, describe image
|
||||
└─► Handle refusal (mark post, try next)
|
||||
|
||||
5. Generate message
|
||||
└─► OpenRouter.generateMessage(analysis, title, examples, strategy)
|
||||
└─► Few-shot examples from SuccessfulMessagesRepository
|
||||
|
||||
6. Send DM
|
||||
└─► MessagePage.sendDM(author, "Stunning", message)
|
||||
└─► Handle failure (log, don't mark as sent)
|
||||
|
||||
7. Record in database
|
||||
└─► Mark post as processed
|
||||
└─► Mark message_sent = 1 if successful
|
||||
└─► Store generated_message for reference
|
||||
```
|
||||
|
||||
### Error Handling
|
||||
|
||||
- AI refusal: Post is marked as processed with error, skipped
|
||||
- Image download failure: Post is marked with error, skipped
|
||||
- DM send failure: Post is marked as not sent, logged
|
||||
- All errors logged with context (post ID, subreddit, author)
|
||||
|
||||
### Cost Optimization
|
||||
|
||||
- Quick gender detection costs less than full analysis
|
||||
- Non-matching genders are skipped before expensive analysis
|
||||
- AI refusal detection prevents wasted tokens on rejected prompts
|
||||
|
||||
---
|
||||
|
||||
## StrategySelector
|
||||
|
||||
**File**: `src/services/strategy-selector.ts`
|
||||
**Lines**: 101
|
||||
**Responsibility**: A/B testing with explore/exploit algorithm
|
||||
|
||||
### Purpose
|
||||
|
||||
Selects message variants (strategies) for each post using an explore/exploit algorithm based on historical response rates. Optimizes for maximum response rate over time while continuing to explore under-tested variants.
|
||||
|
||||
### Public Methods
|
||||
|
||||
```typescript
|
||||
class StrategySelector {
|
||||
constructor(
|
||||
private config: Config,
|
||||
private messagesRepo: MessagesRepository
|
||||
)
|
||||
|
||||
/**
|
||||
* Select the best strategy variant for a given subreddit/author.
|
||||
*/
|
||||
selectStrategy(subreddit: string, author: string): string
|
||||
}
|
||||
```
|
||||
|
||||
### Algorithm
|
||||
|
||||
1. **Get strategy stats** from `MessagesRepository`:
|
||||
- Response count per variant
|
||||
- Sent count per variant
|
||||
- Response rate per variant
|
||||
|
||||
2. **Apply minimum sample threshold**:
|
||||
- If a variant has fewer than `minSamples` sends, treat it as untested
|
||||
- Untested variants are excluded from exploitation
|
||||
|
||||
3. **Explore vs Exploit decision**:
|
||||
- With probability `explorationRate`, select a random variant (explore)
|
||||
- Otherwise, select the variant with the highest response rate (exploit)
|
||||
|
||||
4. **Fallback**:
|
||||
- If no variants have enough data, select randomly from enabled variants
|
||||
|
||||
### Configuration
|
||||
|
||||
```yaml
|
||||
strategyVariants:
|
||||
- name: "variant_a"
|
||||
prompt: "Compliment-based message..."
|
||||
enabled: true
|
||||
- name: "variant_b"
|
||||
prompt: "Question-based message..."
|
||||
enabled: true
|
||||
```
|
||||
|
||||
### Dependencies
|
||||
|
||||
- `Config` - Strategy variants, exploration rate, min samples
|
||||
- `MessagesRepository` - Response rate statistics
|
||||
|
||||
---
|
||||
|
||||
## ResponseChecker
|
||||
|
||||
**File**: `src/services/response-checker.ts`
|
||||
**Lines**: 179
|
||||
**Responsibility**: Monitor inbox for user replies
|
||||
|
||||
### Purpose
|
||||
|
||||
Background service that navigates the Reddit inbox via Playwright, identifies responses to bot-sent messages, updates the database, and stores successful message/response pairs for few-shot learning.
|
||||
|
||||
### Public Methods
|
||||
|
||||
```typescript
|
||||
class ResponseCheckerService {
|
||||
constructor(
|
||||
private browser: BrowserClient,
|
||||
private messagesRepo: MessagesRepository,
|
||||
private successfulRepo: SuccessfulMessagesRepository,
|
||||
private logger: Logger
|
||||
)
|
||||
|
||||
/**
|
||||
* Check inbox for responses to sent messages.
|
||||
*/
|
||||
checkResponses(): Promise<void>
|
||||
}
|
||||
```
|
||||
|
||||
### Algorithm
|
||||
|
||||
1. Navigate to `/message/inbox` via Playwright
|
||||
2. Parse inbox items (messages, conversations)
|
||||
3. For each response item:
|
||||
- Extract recipient username and message text
|
||||
- Match to `sent_messages` by recipient + timestamp proximity
|
||||
- Update `got_response = 1` and `response_checked_at`
|
||||
- Store the message/response pair in `successful_messages`
|
||||
4. Update strategy statistics (response rates)
|
||||
|
||||
### Dependencies
|
||||
|
||||
- `BrowserClient` - Playwright browser context
|
||||
- `MessagesRepository` - Update response flags
|
||||
- `SuccessfulMessagesRepository` - Store learning pairs
|
||||
- `Logger` - Activity logging
|
||||
|
||||
---
|
||||
|
||||
## ChatService
|
||||
|
||||
**File**: `src/services/chat.ts`
|
||||
**Lines**: 1123+
|
||||
**Responsibility**: Reddit chat scraping, messaging, and conversation management
|
||||
|
||||
### Purpose
|
||||
|
||||
The most complex service in the codebase. Manages Reddit chat interactions including scraping active conversations, loading conversation history, sending messages with human-like delays, detecting upsell/spam, and managing conversation caches.
|
||||
|
||||
### Public Methods
|
||||
|
||||
```typescript
|
||||
class ChatService {
|
||||
constructor(
|
||||
private browser: BrowserClient,
|
||||
private config: Config,
|
||||
private logger: Logger
|
||||
)
|
||||
|
||||
/** Fetch active conversations from Reddit */
|
||||
fetchChats(): Promise<ChatSummary[]>
|
||||
|
||||
/** Load full conversation history */
|
||||
loadConversation(chatId: string): Promise<Conversation>
|
||||
|
||||
/** Send a message in a conversation */
|
||||
sendMessage(chatId: string, text: string): Promise<boolean>
|
||||
|
||||
/** Get cached conversations */
|
||||
getCachedConversations(): Conversation[]
|
||||
|
||||
/** Check for new messages in conversations */
|
||||
checkForUpdates(): Promise<Conversation[]>
|
||||
|
||||
/** Detect upsell/spam in a message */
|
||||
detectUpsell(message: string): boolean
|
||||
}
|
||||
```
|
||||
|
||||
### Shadow DOM Handling
|
||||
|
||||
Reddit uses custom `faceplate-*` web components with shadow DOM. Standard CSS selectors cannot pierce these boundaries. The service uses JavaScript injection via `page.evaluate()`:
|
||||
|
||||
```javascript
|
||||
// Traverse shadow roots to find chat elements
|
||||
function traverseShadowRoots(root) {
|
||||
const elements = root.querySelectorAll('.some-class');
|
||||
const shadowChildren = root.querySelectorAll('*::shadow, ::deep');
|
||||
// ... recursive traversal
|
||||
}
|
||||
```
|
||||
|
||||
### Conversation Cache
|
||||
|
||||
Conversations are cached in two layers:
|
||||
|
||||
1. **In-memory**: Fast access for active conversations
|
||||
2. **Disk JSON**: Persistence across restarts at `data/chats/`
|
||||
|
||||
Cache structure:
|
||||
```json
|
||||
{
|
||||
"chatId": "abc123",
|
||||
"participants": ["user1", "bot"],
|
||||
"messages": [
|
||||
{ "author": "user1", "text": "Hey!", "timestamp": 1234567890 }
|
||||
],
|
||||
"lastUpdated": 1234567890,
|
||||
"hasNewMessages": false
|
||||
}
|
||||
```
|
||||
|
||||
### Human-Like Messaging
|
||||
|
||||
Messages are sent with:
|
||||
- Typing delay simulation (random delay before "typing" starts)
|
||||
- Variable typing speed
|
||||
- Random delays between messages
|
||||
- Delivery verification
|
||||
|
||||
### Known Issues
|
||||
|
||||
- **Hardcoded username**: `myUsername = 'Upset_Marsupial1006'` is hardcoded in JS evaluation scripts. Should be parameterized from `Config`.
|
||||
- **Size**: 1123+ lines suggests this should be split into smaller services.
|
||||
|
||||
### Dependencies
|
||||
|
||||
- `BrowserClient` - Playwright browser context
|
||||
- `Config` - Rate limits, username (should be), check intervals
|
||||
- `Logger` - Activity logging
|
||||
|
||||
---
|
||||
|
||||
## MediaService
|
||||
|
||||
**File**: `src/services/media.ts`
|
||||
**Lines**: 254
|
||||
**Responsibility**: Media download, caching, and indexing
|
||||
|
||||
### Purpose
|
||||
|
||||
Downloads images from chat conversations, resolves Matrix blurred thumbnails to full resolution, stores files locally, and maintains a deduplicated index.
|
||||
|
||||
### Public Methods
|
||||
|
||||
```typescript
|
||||
class MediaService {
|
||||
constructor(
|
||||
private config: Config,
|
||||
private browser: BrowserClient,
|
||||
private logger: Logger
|
||||
)
|
||||
|
||||
/** Download media from a conversation */
|
||||
downloadMedia(chatId: string): Promise<string[]>
|
||||
|
||||
/** Get media index */
|
||||
getIndex(): MediaIndex
|
||||
|
||||
/** Get a specific media file path */
|
||||
getPath(hash: string): string | null
|
||||
}
|
||||
```
|
||||
|
||||
### Deduplication
|
||||
|
||||
Files are deduplicated using SHA-1 hashing:
|
||||
|
||||
```
|
||||
SHA-1(image_bytes) → data/media/{hash}.{ext}
|
||||
```
|
||||
|
||||
The index maps hashes to file paths:
|
||||
```json
|
||||
{
|
||||
"abc123def456": "abc123def456.jpg",
|
||||
"789xyz012abc": "789xyz012abc.png"
|
||||
}
|
||||
```
|
||||
|
||||
### Matrix Thumbnail Resolution
|
||||
|
||||
Reddit's Matrix system uses blurred placeholder thumbnails. The service resolves these to full resolution by:
|
||||
|
||||
1. Detecting Matrix thumbnail URLs
|
||||
2. Extracting the full-resolution URL from the response
|
||||
3. Downloading the full-resolution image
|
||||
|
||||
### Dependencies
|
||||
|
||||
- `Config` - Media directory path
|
||||
- `BrowserClient` - Playwright for Matrix resolution
|
||||
- `Logger` - Download activity
|
||||
|
||||
---
|
||||
|
||||
## MediaSyncService
|
||||
|
||||
**File**: `src/services/media-sync.ts`
|
||||
**Lines**: 119
|
||||
**Responsibility**: Periodic background media synchronization
|
||||
|
||||
### Purpose
|
||||
|
||||
Runs periodic background syncs to download media from chat conversations. Supports both incremental (top 10 chats) and full (all conversations) sync modes.
|
||||
|
||||
### Public Methods
|
||||
|
||||
```typescript
|
||||
class MediaSyncService {
|
||||
constructor(
|
||||
private config: Config,
|
||||
private mediaService: MediaService,
|
||||
private chatService: ChatService,
|
||||
private logger: Logger
|
||||
)
|
||||
|
||||
/** Start periodic sync */
|
||||
start(): void
|
||||
|
||||
/** Stop periodic sync */
|
||||
stop(): void
|
||||
|
||||
/** Trigger incremental sync */
|
||||
syncIncremental(): Promise<void>
|
||||
|
||||
/** Trigger full sync */
|
||||
syncFull(): Promise<void>
|
||||
|
||||
/** Get sync status */
|
||||
getStatus(): SyncStatus
|
||||
}
|
||||
```
|
||||
|
||||
### Sync Modes
|
||||
|
||||
| Mode | Description | Use Case |
|
||||
|---|---|---|
|
||||
| Incremental | Top 10 most recent conversations | Regular background sync |
|
||||
| Full | All conversations | Initial setup, catch-up |
|
||||
|
||||
### Dependencies
|
||||
|
||||
- `Config` - Sync interval, enabled flag
|
||||
- `MediaService` - Actual download logic
|
||||
- `ChatService` - Conversation list
|
||||
- `Logger` - Sync progress
|
||||
|
||||
---
|
||||
|
||||
## AutoReplyService
|
||||
|
||||
**File**: `src/services/auto-reply.ts`
|
||||
**Lines**: 199
|
||||
**Responsibility**: Automated reply to chat conversations
|
||||
|
||||
### Purpose
|
||||
|
||||
Monitors cached conversations for unread messages, detects upsell/spam, generates AI reply suggestions, and either auto-sends or marks as pending for manual approval.
|
||||
|
||||
### Public Methods
|
||||
|
||||
```typescript
|
||||
class AutoReplyService {
|
||||
constructor(
|
||||
private config: Config,
|
||||
private chatService: ChatService,
|
||||
private openrouter: OpenRouterService,
|
||||
private logger: Logger
|
||||
)
|
||||
|
||||
/** Start auto-reply checking */
|
||||
start(): void
|
||||
|
||||
/** Stop auto-reply checking */
|
||||
stop(): void
|
||||
|
||||
/** Check for unread messages and generate replies */
|
||||
checkAndReply(): Promise<void>
|
||||
}
|
||||
```
|
||||
|
||||
### Workflow
|
||||
|
||||
```
|
||||
1. Check cached conversations for unread messages
|
||||
│
|
||||
├─► Skip if no new messages
|
||||
│
|
||||
├─► Detect upsell/spam keywords
|
||||
│ └─► Flag conversation, skip reply
|
||||
│
|
||||
├─► Generate AI reply suggestion
|
||||
│ └─► OpenRouter.generateReply(conversation history)
|
||||
│
|
||||
├─► If autoSend: send reply immediately
|
||||
│ └─► ChatService.sendMessage()
|
||||
│
|
||||
└─► If !autoSend: mark as pending
|
||||
└─► Dashboard shows pending replies for approval
|
||||
```
|
||||
|
||||
### Dependencies
|
||||
|
||||
- `Config` - Enabled flag, autoSend flag, check interval
|
||||
- `ChatService` - Conversation access, message sending
|
||||
- `OpenRouterService` - Reply generation
|
||||
- `Logger` - Activity logging
|
||||
|
||||
---
|
||||
|
||||
## RedditApi
|
||||
|
||||
**File**: `src/reddit/api.ts`
|
||||
**Lines**: 233
|
||||
**Responsibility**: Reddit API client
|
||||
|
||||
### Purpose
|
||||
|
||||
Axios-based client for interacting with Reddit's public API. Handles post fetching, user profile checking, and authenticated requests using Playwright browser cookies.
|
||||
|
||||
### Public Methods
|
||||
|
||||
```typescript
|
||||
class RedditApi {
|
||||
constructor(
|
||||
private browser: BrowserClient,
|
||||
private logger: Logger
|
||||
)
|
||||
|
||||
/** Fetch new posts from a subreddit */
|
||||
fetchPosts(subreddit: string, limit?: number): Promise<Post[]>
|
||||
|
||||
/** Check if a user is a seller/spam */
|
||||
checkUserProfile(username: string): Promise<UserProfile>
|
||||
|
||||
/** Make authenticated request using browser cookies */
|
||||
authenticatedRequest(url: string): Promise<any>
|
||||
}
|
||||
```
|
||||
|
||||
### Image URL Resolution
|
||||
|
||||
Fallback chain for extracting image URLs from post data:
|
||||
|
||||
```
|
||||
1. post.preview.reddits[0].url (Reddit CDN preview)
|
||||
2. post.url (i.redd.it direct)
|
||||
3. post.thumbnail (if not self/spam)
|
||||
4. Scan post.selftext for imgur/image links
|
||||
```
|
||||
|
||||
### Seller/Spam Detection
|
||||
|
||||
Checks user profile for keywords:
|
||||
- "seller"
|
||||
- "onlyfans"
|
||||
- "link in bio"
|
||||
- Other configured `sellerKeywords`
|
||||
|
||||
### Dependencies
|
||||
|
||||
- `BrowserClient` - Browser cookies for authentication
|
||||
- `Logger` - Request logging
|
||||
|
||||
---
|
||||
|
||||
## OpenRouterService
|
||||
|
||||
**File**: `src/ai/openrouter.ts`
|
||||
**Lines**: 307
|
||||
**Responsibility**: OpenRouter AI client
|
||||
|
||||
### Purpose
|
||||
|
||||
Client for the OpenRouter API (OpenAI-compatible SDK). Provides image analysis, message generation, reply suggestions, and upsell detection.
|
||||
|
||||
### Public Methods
|
||||
|
||||
```typescript
|
||||
class OpenRouterService {
|
||||
constructor(private config: Config, private logger: Logger)
|
||||
|
||||
/** Full image analysis with model fallback */
|
||||
analyzeImage(imageBase64: string): Promise<ImageAnalysis>
|
||||
|
||||
/** Quick gender detection (cost-saving) */
|
||||
detectGenderOnly(imageBase64: string): Promise<string>
|
||||
|
||||
/** Generate DM message */
|
||||
generateMessage(
|
||||
imageAnalysis: ImageAnalysis,
|
||||
postTitle: string,
|
||||
strategy: string
|
||||
): Promise<string>
|
||||
|
||||
/** Generate chat reply */
|
||||
generateReply(conversationHistory: Message[]): Promise<string>
|
||||
|
||||
/** Detect upsell/spam keywords */
|
||||
detectUpsell(message: string): boolean
|
||||
}
|
||||
```
|
||||
|
||||
### Model Fallback Chain
|
||||
|
||||
```
|
||||
Primary model (config.openrouter.model)
|
||||
│
|
||||
└─► If refusal detected:
|
||||
│
|
||||
└─► Try each fallback model (config.openrouter.fallbackModels)
|
||||
│
|
||||
└─► If all fail: throw error
|
||||
```
|
||||
|
||||
### Refusal Detection
|
||||
|
||||
Heuristic patterns to detect when AI models refuse to process prompts:
|
||||
|
||||
```typescript
|
||||
const refusalPatterns = [
|
||||
/cannot.*analyze/i,
|
||||
/unable.*process/i,
|
||||
/against.*policy/i,
|
||||
/cannot.*view/i,
|
||||
// ...
|
||||
];
|
||||
```
|
||||
|
||||
### Few-Shot Learning
|
||||
|
||||
`generateMessage()` retrieves successful message/response pairs from the database and includes them as examples in the prompt:
|
||||
|
||||
```
|
||||
"Here are examples of messages that got responses:
|
||||
- Example 1: {original_message} → {response_snippet}
|
||||
- Example 2: ...
|
||||
|
||||
Generate a similar message for this image..."
|
||||
```
|
||||
|
||||
### Dependencies
|
||||
|
||||
- `Config` - API key, model selection, prompts
|
||||
- `MessagesRepository` - Successful examples for few-shot learning
|
||||
|
||||
---
|
||||
|
||||
## BrowserClient
|
||||
|
||||
**File**: `src/browser/client.ts`
|
||||
**Lines**: 302
|
||||
**Responsibility**: Playwright persistent context manager
|
||||
|
||||
### Purpose
|
||||
|
||||
Manages the Playwright browser session with persistent cookies/storage. Handles automated login, session verification, and 2FA/captcha detection.
|
||||
|
||||
### Public Methods
|
||||
|
||||
```typescript
|
||||
class BrowserClient {
|
||||
constructor(private config: Config, private logger: Logger)
|
||||
|
||||
/** Initialize browser context */
|
||||
initialize(): Promise<void>
|
||||
|
||||
/** Verify browser session is active */
|
||||
verifySession(): Promise<boolean>
|
||||
|
||||
/** Perform automated login */
|
||||
login(): Promise<void>
|
||||
|
||||
/** Get page context for operations */
|
||||
getPage(): Promise<Page>
|
||||
|
||||
/** Close browser */
|
||||
close(): Promise<void>
|
||||
}
|
||||
```
|
||||
|
||||
### Login Flow
|
||||
|
||||
```
|
||||
1. Navigate to Reddit login page
|
||||
2. Fill username and password
|
||||
3. Submit form
|
||||
4. Check for 2FA prompt
|
||||
└─► If present: wait for manual input (with timeout)
|
||||
5. Check for captcha
|
||||
└─► If present: log warning, wait
|
||||
6. Verify login success:
|
||||
└─► Check for user menu indicator
|
||||
└─► Check for profile avatar
|
||||
└─► Fallback: API request to /api/me
|
||||
7. Save cookies to persistent directory
|
||||
8. On failure: capture debug screenshot
|
||||
```
|
||||
|
||||
### Session Persistence
|
||||
|
||||
- Cookies and storage are saved to `config.browser.userDataDir`
|
||||
- Subsequent launches restore the session automatically
|
||||
- Session expires when Reddit cookies expire (requires re-login)
|
||||
|
||||
### Dependencies
|
||||
|
||||
- `Config` - Credentials, headless mode, user data dir
|
||||
- `Logger` - Login progress, errors
|
||||
|
||||
---
|
||||
|
||||
## MessagePage
|
||||
|
||||
**File**: `src/browser/message-page.ts`
|
||||
**Lines**: 560
|
||||
**Responsibility**: Reddit DM compose page automation
|
||||
|
||||
### Purpose
|
||||
|
||||
Automates the Reddit DM compose page: filling recipient/subject/body fields, handling shadow DOM components, submitting messages, and verifying success.
|
||||
|
||||
### Public Methods
|
||||
|
||||
```typescript
|
||||
class MessagePage {
|
||||
constructor(
|
||||
private browser: BrowserClient,
|
||||
private config: Config,
|
||||
private logger: Logger
|
||||
)
|
||||
|
||||
/** Send a DM to a recipient */
|
||||
sendDM(recipient: string, subject: string, body: string): Promise<boolean>
|
||||
|
||||
/** Check if recipient field shows an error */
|
||||
hasRecipientError(page: Page): Promise<boolean>
|
||||
}
|
||||
```
|
||||
|
||||
### Shadow DOM Handling
|
||||
|
||||
Reddit's compose form uses `faceplate-*` web components:
|
||||
|
||||
```javascript
|
||||
// Fill shadow DOM input fields
|
||||
page.evaluate(({ selector, value }) => {
|
||||
const shadowInput = document.querySelector(`${selector} input`);
|
||||
if (shadowInput) {
|
||||
shadowInput.value = value;
|
||||
shadowInput.dispatchEvent(new Event('input', { bubbles: true }));
|
||||
}
|
||||
}, { selector: 'faceplate-textarea[name="body"]', value: body });
|
||||
```
|
||||
|
||||
### Dry Run Mode
|
||||
|
||||
When `config.dryRun` is enabled:
|
||||
- All fields are filled as normal
|
||||
- Submit button is NOT clicked
|
||||
- Message is logged but not sent
|
||||
- Returns `true` (simulates success)
|
||||
|
||||
### Dependencies
|
||||
|
||||
- `BrowserClient` - Playwright page access
|
||||
- `Config` - Dry run mode
|
||||
- `Logger` - Send activity
|
||||
|
||||
---
|
||||
|
||||
## RateLimiter
|
||||
|
||||
**File**: `src/utils/rate-limiter.ts`
|
||||
**Lines**: 114
|
||||
**Responsibility**: Token-bucket rate limiting
|
||||
|
||||
### Purpose
|
||||
|
||||
Enforces rate limits on bot actions (primarily message sending) to avoid detection and bans. Uses a combination of minimum delay, random jitter, and hourly caps.
|
||||
|
||||
### Public Methods
|
||||
|
||||
```typescript
|
||||
class RateLimiter {
|
||||
constructor(config: RateLimitConfig)
|
||||
|
||||
/** Wait until a slot is available */
|
||||
waitForSlot(): Promise<void>
|
||||
|
||||
/** Get remaining time until next slot */
|
||||
getTimeUntilSlot(): number
|
||||
|
||||
/** Get current hourly count */
|
||||
getHourlyCount(): number
|
||||
|
||||
/** Reset hourly counter */
|
||||
resetHourlyCounter(): void
|
||||
}
|
||||
```
|
||||
|
||||
### Rate Limiting Strategy
|
||||
|
||||
```
|
||||
1. Check hourly cap:
|
||||
└─► If maxMessagesPerHour reached: wait until hour rolls over
|
||||
|
||||
2. Enforce minimum delay:
|
||||
└─► lastActionTime + minDelayMs + random(0, jitterMs)
|
||||
|
||||
3. Track action timestamp
|
||||
4. Increment hourly counter
|
||||
```
|
||||
|
||||
### Dependencies
|
||||
|
||||
- `Config` - Rate limit settings
|
||||
- `p-queue` - Concurrency control
|
||||
Reference in new issue
Block a user