60 lines
4.2 KiB
Markdown
60 lines
4.2 KiB
Markdown
# Human-led conversation learning
|
|
|
|
## Purpose
|
|
|
|
The loop learns from the decision that caused a useful outcome, not merely from the last AI message before it. Manual interventions are first-class evidence because an operator may repair tone, timing, boundaries, or a media invitation before a recipient shares an image.
|
|
|
|
## Pipeline and gates
|
|
|
|
```text
|
|
authoritative Matrix timeline
|
|
-> event attribution (exact > fingerprint > inferred)
|
|
-> outcome trajectory and causal credit
|
|
-> identifier-redacted decision examples
|
|
-> maturity / review / invalidation gates
|
|
-> account-scoped, quota-controlled retrieval
|
|
-> shadow audit or injected prompt context
|
|
-> offline evaluation record
|
|
```
|
|
|
|
Capture and use are independent. The default configuration captures and audits retrieval in shadow mode, but cannot affect generated replies. Injection requires both `generationEnabled: true` and `shadowMode: false`.
|
|
|
|
## Attribution and credit
|
|
|
|
Authoritative Matrix event IDs link openers, automated claims, dashboard manual sends, and exchange-library sends. Fingerprints support legacy backfill only. An unmatched outbound event is human-owned for the automation safety boundary, but inferred learning is held back when exact attribution is required.
|
|
|
|
Credit classes are:
|
|
|
|
- `human-created-success`: the human introduced the successful decision.
|
|
- `human-improved-success`: AI attempted the step and the human replaced or improved it.
|
|
- `human-completed-ai-trajectory`: the human intervention helped complete an AI-started exchange without directly making the media ask.
|
|
- `autonomous-success`: the bounded automated policy made the attributed ask.
|
|
- `recipient-initiated-media`: media arrived without an attributed ask.
|
|
- `uncertain`: provenance is insufficient for causal use.
|
|
|
|
Image receipt does not override a decline, boundary, or upsell outcome. Those outcomes invalidate the trajectory and all derived decisions. Operator exclusion is sticky across re-sync; explicit inclusion can reverse review/exclusion, but cannot resurrect an invalidated safety outcome.
|
|
|
|
## Retrieval and evaluation
|
|
|
|
Retrieval stays within the owning account, excludes the current room, filters confidence, prefers the same policy/stage/community, and fills separate human and autonomous quotas. Stored context is bounded and redacted. The generation ledger stores selected decision IDs, mode, policy version, and an optional prompt hash—not prompt text.
|
|
|
|
`npm run learning:eval -- --db <path> --policy <version>` versions the current eligible dataset and enforces hard gates for leaked identifiers, invalid decision states, and empty outputs. Evaluation is deliberately local and deterministic. More subjective pairwise or model-graded evaluation can be added behind the same saved-run contract.
|
|
|
|
## Infrastructure and framework boundary
|
|
|
|
No new Dokploy service is required. SQLite, the existing backend, Matrix synchronization, OpenRouter client, and React dashboard are sufficient. Deployment needs only the normal application image plus a database backup before the migration; it does not require a vector database, queue, object store, or hosted tracing system.
|
|
|
|
The repository/provider/evaluation interfaces are the framework seams. A hosted platform such as Langfuse or an experiment tool can later receive generation and evaluation records, and DSPy-style optimization or model fine-tuning can consume exported reviewed examples. None is placed in the online reply path until volume, evaluation quality, and operational need justify the extra infrastructure.
|
|
|
|
## Rollout
|
|
|
|
1. Deploy with capture enabled, generation disabled, and shadow mode enabled.
|
|
2. Run a full chat sync to backfill authoritative histories.
|
|
3. Review inferred and fingerprint-attributed trajectories in `/learning`.
|
|
4. Let examples mature and run the offline evaluation against a backup/non-production copy.
|
|
5. Compare human, autonomous, and recipient-initiated cohorts; do not blend them into the opener strategy metric.
|
|
6. Enable generation while retaining shadow mode to exercise selection/auditing.
|
|
7. Only after review and evaluation, turn shadow mode off for a small account/policy cohort; rollback is setting it true again.
|
|
|
|
Starting, restarting, deploying, or enabling the live bot remains a separate explicit operational action.
|