Add conversation image replacement

This commit is contained in:
jjbrunton committed 2026-08-14 12:56:00 +01:00
1 parent 2500f0a23c
commit 8523873831
18 files changed
+812 -3

No files matched your search

+2
View File
@@ -82,6 +82,8 @@ an administrator credential.
| `POST` | `/api/conversations/{id}/send-reply` | Send a conversation reply | admin | yes |
| `GET` | `/api/conversations/{id}/shareable-media` | List every enabled exchange-library image not yet shared in this conversation | read | no |
| `POST` | `/api/conversations/{id}/share-media` | Send a selected exchange-library image | admin | yes |
| `GET` | `/api/conversations/{id}/image-replacement-options` | List green-screen sources and conversation-owned replacement targets | read | no |
| `POST` | `/api/conversations/{id}/generate-image-replacement` | Composite a selected conversation image into a green-screen source | admin | yes |
| `POST` | `/api/conversations/start` | Start a new conversation | admin | yes |
| `POST` | `/api/conversations/{id}/read` | Advance the operator read watermark | admin | yes |
| `POST` | `/api/conversations/{id}/unread` | Return a conversation to the unread queue | admin | yes |
+3 -1
View File
@@ -23,7 +23,7 @@ stateDiagram-v2
### 1. Discovery
The discovery loop rotates through enabled subreddits and authenticated runtimes. A runtime fetches Reddit `/new` data using the process-shared `RedditRequestCoordinator`. In the default `post-author` targeting mode those posts enter intake directly. An image subreddit configured for `comment-author` instead reads top-level comments from at most five newly listed posts, batches previously unseen eligible comment text through the reaction classifier, and creates candidates only for role-alignment verdicts at or above 0.8 confidence. A bounded in-process cache prevents unchanged comments from repeatedly consuming model calls. The candidate ID, author, timestamp, and permalink come from the comment while the original post image and title remain attached as preparation context; the usual image screen later verifies the depicted gender against `reddit.targetGenders`. Intake then deduplicates candidate IDs and authors, excludes processed/contacted history, stores the full snapshot, source type, weight, and timestamps, and logs exact intake counts.
The discovery loop rotates through enabled subreddits and authenticated runtimes. A runtime fetches Reddit `/new` data using the process-shared `RedditRequestCoordinator`. In the default `post-author` targeting mode those posts enter intake directly. An image subreddit configured for `comment-author` instead reads top-level comments from at most five newly listed posts and batches previously unseen eligible comment text through the reaction classifier. Candidates require a role-alignment verdict at or above 0.8 confidence; when `reddit.targetGenders` is non-empty, the expressed gender perspective must also be included. An `unknown` perspective fails closed unless explicitly targeted. A bounded in-process cache prevents unchanged comments from repeatedly consuming model calls. The candidate ID, author, timestamp, and permalink come from the comment while the original post image and title remain attached as preparation context; the usual image screen later verifies the depicted gender against the same target list. Intake then deduplicates candidate IDs and authors, excludes processed/contacted history, stores the full snapshot, source type, weight, and timestamps, and logs exact intake counts.
The queue grows to a demand-based target rather than a fixed backlog. The target is the smaller of configured `reddit.maxReadyCandidates` and the work that authenticated running accounts can still send within the recent-post window, accounting for rolling hourly capacity and minimum message spacing. Intake admits only the current deficit, newest Reddit posts first, pauses at zero outbound capacity, and reports fetched posts left unevaluated because the funnel is full. Reddit request pacing, not a tight application poll, remains the primary request limiter.
@@ -130,6 +130,8 @@ After the normal reply limiter waits, automation refreshes Matrix and rechecks t
The conversation UI also supports an explicitly confirmed manual choice from this same eligible set. Its read endpoint reports request-matched options without exposing filesystem paths. The send endpoint refreshes the conversation inside the owning account's rate limiter, rechecks the latest inbound request and safety boundaries, reserves the selected asset in the same durable ledger, and uses the same Matrix image-plus-reciprocal delivery path. A successful manual share marks automation as user-overridden.
Green-screen replacement is a generation-only conversation flow. The operator selects a source already held in the consent-controlled exchange library and a target identified by the server as either the triggering post image or a downloaded attachment in that conversation. `ImageReplacementService` detects the largest green area at bounded resolution, derives its perspective quadrilateral, cover-fits and warps the target, composites through the actual green pixels so foreground occlusions remain, and carries a bounded amount of the source lighting into the insert. The JPEG is returned as a private data-URL preview; it is not persisted, classified, added to the share ledger, or sent automatically.
Conversation performance is derived from the complete attributed inbound transcript. Engagement cannot become a prompt-learning example while automation is active, and a later decline, boundary, or upsell removes an earlier example before future prompts load it.
## Dashboard data and events
+2
View File
@@ -78,6 +78,8 @@ Exchange-library uploads require an explicit consent checkbox before the raw ima
Conversation reply composers include a collapsed saved-image picker. It lists only server-approved assets for the latest inbound request, keeps thumbnails blurred until revealed, and requires selection plus a dedicated confirmation dialog before sending. The server remains authoritative and may reject a choice if the conversation changes before the rate-limited send executes.
The composer also has a separate green-screen creation panel. Its source picker uses consent-controlled exchange-library assets; its target picker is derived server-side from the triggering post image and the attachments already loaded into that conversation. Source, target, and generated preview are blurred independently. Generation returns an authenticated in-memory JPEG preview/download and never adds it to the library or sends it to Matrix, so delivery remains a separate explicit operator action.
The conversation inbox assigns every row to exactly one primary workflow queue: **Needs attention**, **Waiting**, **Sellers**, **Snoozed**, **Done**, **Deleted**, or **Archived**. Removed-account history belongs only to Archived, deleted recipients belong only to Deleted, and seller/off-platform outcomes belong only to Sellers, so none can leak into actionable work. Unread state, manual ownership, and received images are independent badges and secondary toggles rather than competing queues. The default queue is Needs attention. Suspended profiles are not treated as deleted.
## Feature structure
+2 -1
View File
@@ -20,7 +20,7 @@ This is an ownership map, not an API dump. Read the owning implementation and te
| Module | Owns | Important constraints |
| --- | --- | --- |
| `services/monitor.ts` | Subreddit ordering, post/comment candidate fetching, reaction/age/gender/seller screening, rejection summaries | Comment reactions fail closed at the confidence and target-gender gates; cheap checks run first; preserve exact rejection/deferred counts; listing 404 disables the subreddit. |
| `services/monitor.ts` | Subreddit ordering, post/comment candidate fetching, reaction/age/gender/seller screening, rejection summaries | Comment reactions fail closed at the confidence gate and, when configured, the expressed-gender gate; cheap checks run first; preserve exact rejection/deferred counts; listing 404 disables the subreddit. |
| `reddit/api.ts` | Reddit JSON calls, post/comment parsing, seller/profile evidence, username resolution, error types | All reads must use the coordinator; redact error metadata. |
| `reddit/request-coordinator.ts` | Process-wide Reddit request spacing and cooldown | Space failed requests too; 429 cooldown is global. |
| `services/processor.ts` | Source-specific AI preparation and outbound safety | Returns prepared or explicit skip; performs no remote chat send. |
@@ -70,6 +70,7 @@ Verification is the safety mechanism that matters most. Posted comments are swep
| `services/media-sync.ts` | Periodic authenticated media sync and operator-visible completion status | Stop while auth is unavailable and on runtime shutdown; do not report partial work as success. |
| `services/media-library.ts` | Shared operator uploads, image validation/re-encoding, classification state, and private files | Require consent confirmation; received media never enters this store. |
| `services/media-library-exchange.ts` | Inbound request detection, progressive selection, reservation, and image-first exchange | Never offer proactively; exact tags do not substitute; no asset reuse per conversation. |
| `services/image-replacement.ts` | Green-screen detection, perspective fitting, compositing, and private preview generation | Sources come from the consent-controlled library; targets resolve from authoritative conversation data; generation never sends. |
| `database/media-library-repository.ts` | Shared asset metadata and durable account/conversation send audit | Reserve before remote send; uncertain sends stay consumed. |
## Persistence