122 lines
11 KiB
Markdown
122 lines
11 KiB
Markdown
# Runtime flows
|
|
|
|
## Candidate funnel
|
|
|
|
The production multi-account path is three independent loops backed by `candidate_queue`.
|
|
|
|
```mermaid
|
|
stateDiagram-v2
|
|
[*] --> discovered: listing intake
|
|
discovered --> screening: leased by screening worker
|
|
screening --> ready: eligibility and preparation pass
|
|
screening --> retry_wait: transient failure
|
|
screening --> rejected: terminal eligibility or AI failure
|
|
ready --> sending: leased to available account
|
|
sending --> sent: Matrix confirms conversation
|
|
sending --> retry_wait: account cooldown or transient failure
|
|
sending --> rejected: terminal delivery result
|
|
retry_wait --> discovered: screening retry due
|
|
retry_wait --> ready: delivery retry due
|
|
discovered --> expired: outside configured recent window
|
|
ready --> expired: outside configured recent window
|
|
```
|
|
|
|
### 1. Discovery
|
|
|
|
The discovery loop rotates through enabled subreddits and authenticated runtimes. A runtime fetches Reddit `/new` data using the process-shared `RedditRequestCoordinator`. Intake deduplicates post IDs and authors, excludes processed/contacted history, stores the full post snapshot, source type, weight, and timestamps, and logs exact intake counts.
|
|
|
|
The queue grows to a demand-based target rather than a fixed backlog. The target is the smaller of configured `reddit.maxReadyCandidates` and the work that authenticated running accounts can still send within the recent-post window, accounting for rolling hourly capacity and minimum message spacing. Intake admits only the current deficit, newest Reddit posts first, pauses at zero outbound capacity, and reports fetched posts left unevaluated because the funnel is full. Reddit request pacing, not a tight application poll, remains the primary request limiter.
|
|
|
|
### 2. Screening and preparation
|
|
|
|
A screening worker leases one discovered candidate. `MonitorService.screenCandidate()` applies cheap rules before expensive/networked rules:
|
|
|
|
1. Global processed-post and recipient-history checks.
|
|
2. For chat sources, self-declared gender and adult age parsing.
|
|
3. Seller evidence from the post, profile/about, and recent activity.
|
|
4. Source-specific preparation in `ProcessorService`.
|
|
|
|
For image posts, preparation downloads and validates the image, detects target gender, performs model analysis, selects an image strategy, and generates an opener. For chat posts, it selects a chat strategy and generates from the post text. Both paths apply outbound safety checks and return either a prepared immutable result or an explicit skip reason.
|
|
|
|
Terminal results become rejected candidates. Rate limits and transient failures become retryable candidates. Prepared candidates become ready; the persisted snapshot means a different account can deliver without repeating AI work.
|
|
|
|
### 3. Dispatch
|
|
|
|
An account is eligible for new outreach only when its runtime is running, both identities verify, it has no actionable reply backlog, there is no invitation cooldown, and its rate limiter permits a send. Available accounts are ordered by their current hourly send count.
|
|
|
|
Matrix cooldowns retain the operation that triggered them. A room-creation or invitation limit pauses new outreach but does not suppress replies in existing rooms; a limit returned while sending into an existing room pauses those sends as well.
|
|
|
|
Screening, retry promotion, dispatch, and the operator's active-queue view are ordered by the Reddit post creation time, newest first; source priority is only a tie-breaker. The dispatch worker leases the newest ready candidate for an account, materializes the processed post in that account's repository, applies the account rate limiter, checks dry-run, and atomically claims the recipient globally. `ChatService.startConversationDetailed()` returns one of:
|
|
|
|
- `sent`: persist the message and outcome metadata, commit recipient history, update activity, and mark the candidate sent;
|
|
- `terminal`: release the recipient reservation and reject the candidate;
|
|
- `retryable`: release/defer so another healthy account can claim it soon.
|
|
|
|
Only Matrix confirmation makes a send successful. A healthy HTTP process, room identifier lookup, or attempted request does not.
|
|
|
|
## Reddit request and cooldown flow
|
|
|
|
Every runtime shares one `RedditRequestCoordinator`, because Reddit can enforce limits by origin as well as account. JSON requests are serialized with a minimum interval; failed requests consume spacing too. A 429 extends the global cooldown using the server delay plus a buffer. Broad 403 behavior can also result in coordinated backoff. A listing 404 is treated as terminal for that subreddit and persisted as `enabled=false`.
|
|
|
|
Do not add local retries around this layer without preserving the shared schedule. Per-account bursts defeat the coordinator.
|
|
|
|
## Reply-first conversation automation
|
|
|
|
Each account runtime refreshes recent chats and processes actionable inbound conversations oldest-first before it can accept new outreach. After the initial conservative startup scan, the persisted latest Matrix message event plus the worker's prior-scan watermark is authoritative: a newly observed inbound event newer than the confirmed opener remains actionable even if Matrix's presentation-only unread notification count has already returned to zero. On startup, read-marked rooms without a pending reply claim are not revived. Durable event claims, conversation policy state, and a later outbound event prevent duplicate replies.
|
|
|
|
Outgoing Matrix events are attributed against the recorded opener, durable automated-reply fingerprints, and confirmed exchange-library event records. If synchronization observes an unmatched outgoing message, the conversation becomes operator-owned before another reply can be generated. An unmatched outgoing message also supersedes, rather than confirms, a pending automated claim; only the claimed reply text can reconcile that claim as sent.
|
|
|
|
```mermaid
|
|
flowchart TD
|
|
IN["Unread inbound Matrix event"] --> VERIFIED{"Confirmed after a recorded opener?"}
|
|
VERIFIED -- no --> IGNORE["Ignore as unverified"]
|
|
VERIFIED -- yes --> CLAIM{"Durable event claim exists?"}
|
|
CLAIM -- sent --> IGNORE
|
|
CLAIM -- pending --> RECON["Reconcile against Matrix history"]
|
|
CLAIM -- no --> STATE["Load or snapshot conversation policy"]
|
|
STATE --> STOP{"Manual, seller, image, boundary, unsafe, or turn limit?"}
|
|
STOP -- yes --> HANDOFF["Persist terminal/human-required outcome"]
|
|
STOP -- no --> GEN["Generate policy-specific reply"]
|
|
GEN --> SAFE{"Outbound checks pass?"}
|
|
SAFE -- no --> HANDOFF
|
|
SAFE -- yes --> PERSIST["Persist intended reply claim"]
|
|
PERSIST --> WAIT["Wait for the account rate limiter"]
|
|
WAIT --> RECHECK{"Refresh Matrix; claimed event still latest and safe?"}
|
|
RECHECK -- no --> SUPERSEDE["Mark claim superseded; persist boundary/handoff if present"]
|
|
RECHECK -- yes --> SNAPSHOT["Persist intended conversation state"]
|
|
SNAPSHOT --> SEND["Matrix send"]
|
|
SEND --> MARK["Mark claim sent and record latency"]
|
|
```
|
|
|
|
The inbound event key, intended reply, claim status, and conversation state make retries idempotent. Immediately after any limiter wait, automation refreshes Matrix and revalidates the latest inbound event, boundaries, images, seller evidence, policy status, and the one-media-request limit. Newer state makes the old claim terminally `superseded`; it is never sent. If the process dies after Matrix accepted a reply, reconciliation finds the outbound event and marks the old claim sent instead of sending it again.
|
|
|
|
Legacy conversations may make at most one optional autonomous media request, including a request already present in the opener. The hybrid policy retains its single rapport-gated media ask. Deterministic, sequence-aware refusal checks run before generation and again at send time; generated media asks must pass the same non-explicit, non-coercive validator.
|
|
|
|
## Chat and media synchronization
|
|
|
|
`ChatService` communicates with Reddit's Matrix homeserver, maintains an account-scoped SQLite-backed cache, resolves `t2_` identities to Reddit usernames, and emits sync progress/new-reply callbacks. Startup chooses full or delta sync from persisted cursor state. Operators can explicitly run tracked sync jobs through the versioned API.
|
|
|
|
`MediaService` resolves Matrix media, downloads it into the persistent data directory, and writes an account-scoped media index. `MediaSyncService` schedules incremental work only while the account is authenticated. Private media is served behind the same authorization boundary as the dashboard and APIs.
|
|
|
|
The process-global exchange library is separate. Dashboard uploads are decoded and re-encoded with `sharp`, which validates the actual JPEG/PNG/WebP contents, applies orientation, and removes EXIF/location/camera metadata. OpenRouter then assigns overlapping anatomy tags plus SFW/NSFW classification. Failed or low-confidence classifications remain disabled until the operator corrects them.
|
|
|
|
For each latest inbound text event, `MediaLibraryExchangeService` detects direct or indirect image requests before normal reply generation. Exact requests require the matching tag. Generic requests use the conversation's completed library-send count to broaden the eligible safety tier and randomly choose an unused asset. A unique durable reservation on account + conversation + asset prevents reuse, while account + conversation + inbound event makes restart handling idempotent.
|
|
|
|
After the normal reply limiter waits, automation refreshes Matrix and rechecks that the same inbound request is still latest and that no image, boundary, seller signal, or other terminal state has appeared. It then uploads the private file to Matrix, sends an `m.image` event with a stable transaction ID, and only after confirmation sends the optional reciprocal text. If the image is confirmed but the text is not, the asset remains consumed and the conversation is handed to the operator rather than risking a duplicate.
|
|
|
|
Conversation performance is derived from the complete attributed inbound transcript. Engagement cannot become a prompt-learning example while automation is active, and a later decline, boundary, or upsell removes an earlier example before future prompts load it.
|
|
|
|
## Dashboard data and events
|
|
|
|
Routes assemble snapshots from repositories and injected services. `LogBroadcaster`/the realtime event hub publishes typed events for bot state, sends, failures, conversations, replies, sync, sessions, rate limits, and logs. The React client validates incoming events, invalidates matching TanStack Query keys, and reconnects with replay support. Cached screens remain visible but stale during a disconnect.
|
|
|
|
## Failure classification
|
|
|
|
Use three categories consistently:
|
|
|
|
- Terminal: the candidate/action is unsuitable or impossible; persist a reason and do not retry automatically.
|
|
- Retryable: service limits, temporary auth/network errors, or account cooldown; preserve the candidate and retry time.
|
|
- Human-required: policy or safety cannot continue autonomously; stop automation and surface an attention item.
|
|
|
|
Avoid converting errors into generic skips. Operational diagnosis depends on preserving which stage failed and why.
|