Three intertwined fixes for the torn-write that corrupted the prod
data/chat-cache.json mid-session:
- ChatCache.save / persistCooldown now write via temp file + rename.
rename(2) is atomic on the same filesystem, so a crashed or
concurrent writer can never leave NULs in the middle of the file.
A small serialising write queue stops two concurrent saves from
racing to the same temp name.
- ChatCache throws if constructed without a filePath under NODE_ENV=test.
The old default (./data/chat-cache.json) silently routed test
writes into the production cache — exactly how the corruption
happened. Future test that forgets to pass a tmp path fails loudly
instead of stomping data.
- Cooldown / delta-media / username-persist tests now pass an
explicit tmp ChatCache so the new guard is satisfied.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
For chats where we sent an opening message but the recipient never
accepted, /sync drops the room with no heroes and the recipient is
in state as m.room.member with membership:'invite' — not 'join'.
chat.reddit.com's web UI labels these rooms with the invitee anyway,
because it reads the full room state.
- roomToActiveChat now accepts 'invite' membership in addition to
'join' when scanning member-join state events.
- enrichUsernames pass 2 hits /_matrix/client/v3/rooms/{id}/state
first (cheap, returns all member events including invites) and
only falls back to /messages history if /state yields nothing.
- Added MatrixClient.getRoomFullState to back that lookup.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
api.reddit.com/api/user_data_by_account_ids returns 404 without an
OAuth token — which is why every chat in the dashboard sat on
t2_xxx or unknown. The same path on www.reddit.com (with a .json
suffix) is served unauthenticated and returns the expected user
map. Switch the resolver to that host so enrichUsernames actually
succeeds.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Reddit's homeserver does not expose /_matrix/client/v3/rooms/{id}/joined_members
(it returns 'page not found'), so the previous fallback was always a 404.
Two replacements:
- roomToActiveChat: include timeline event senders as candidate MXIDs
alongside heroes / member-join state. /sync omits heroes for idle
rooms but always carries full sender MXIDs in any timeline events
it does return, so we recover a t2 ID whenever the delta sees a
message.
- enrichUsernames: split into three passes — apply known mappings
first (so re-created delta chats get upgraded without a restart),
then page /messages 'b' once per still-unknown room to mine a t2
from the recipient's history, then resolve any remaining raw t2s
via the bulk user_data endpoint. Failed history lookups are cached
in unresolvableRooms so we don't retry every sync.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Reddit's homeserver returns 404 from /_matrix/client/v3/rooms/{id}/joined_members
for rooms we've left, been kicked from, or never fully joined — which is
roughly every chat that wasn't accepted by the recipient. The previous
enrichUsernames pass retried these on every delta sync, flooding the
dashboard log with debug-level 404s. Track failures in an in-memory
unresolvable set so each room is tried at most once per process.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Reddit's homeserver sometimes returns a room in /sync with no
m.heroes and no joined m.room.member state events, leaving us with
no t2 ID to resolve and the chat stuck on 'unknown'. enrichUsernames
now hits /_matrix/client/v3/rooms/{roomId}/joined_members for any
chat whose username is 'unknown' or a bare MXID, stamps the t2 ID,
and lets the existing resolver fill in the real Reddit username.
Also treats a cached 'unknown' username as stale in roomToActiveChat
so subsequent deltas re-attempt the lookup instead of pinning it.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
enrichUsernames() was only invoked once at startup, so any rooms that
arrived via a later delta /sync stayed pinned to their raw t2_xxx
identifier — which the dashboard then rendered as a bare Matrix-style
ID. Call enrichUsernames() at the tail of syncOnce so new rooms get
resolved to real Reddit usernames before the UI reads them.
Also pass the unformatted username to openConversation() and format
the modal title in the frontend, so lookups by username keep working
after the display sanitisation.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Adds an inline subreddit editor on the dashboard that lets you add,
remove, and edit (tone, type, weight) the subreddit list, with changes
persisted to config.yaml on disk so they survive restarts.
Backend
- src/config/persistence.ts: round-trips config.yaml via
YAML.parseDocument, mutating existing row nodes in place so their
trailing inline `# ...` comments survive an edit; new rows are
appended as flow-style maps to match the on-disk idiom; removed rows
are dropped. Default values ('default' tone, 'image' type) are
omitted on write so the file stays clean. Atomic via tmp+rename.
- src/web/routes/config.ts: GET /api/config now also returns the
current subreddit list and the available tone keys. PUT /api/config
accepts a `subreddits` field, validates with the existing
SubredditConfigSchema, rejects duplicates, persists, then mutates
ctx.config.reddit.subreddits in place so the processor picks up the
new tone immediately without a restart.
Frontend
- public/index.html: replaces the read-only subreddit chip list in the
Configuration snapshot section with a grid of editable rows (name,
type, tone dropdown, weight, remove). + Add / Save buttons with
inline status. Tones are sourced from /api/config so the catalog
drives the dropdown.
Tests
- tests/config-persistence.test.ts: covers in-place mutation comment
preservation, removal, appends, reorder, default omission, and
atomic write cleanup.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Introduces a tone-profile system that lets the processor adapt outgoing
message style per-subreddit (and per-flair) instead of using one tone
for everything. Profiles act as a "tone mic" layered after the base
prompt and strategy override, so they shape voice without redefining
the whole message.
- src/config/tone-profiles.ts defines the catalog (default, dominant,
submissive, playful, direct, ...) and a resolveTone() helper that
falls back to the default profile for unknown keys.
- config.yaml maps subreddits to tone keys; the schema in
src/config/index.ts validates it.
- src/services/processor.ts threads the resolved tone into the prompt
build, picking based on subreddit and optional link flair.
- src/reddit/api.ts adds link_flair_text to RedditPost so the
processor can use it for tone selection.
- src/ai/openrouter.ts accepts and forwards the tone-shaped prompt.
- public/index.html surfaces the active tone in the dashboard.
- Tests cover the catalog, config validation, and processor integration.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Closes the noVNC-unavailable gap: when Dokploy's reverse proxy strips
WebSocket upgrades for noVNC, there's no way to do a one-time
interactive Reddit login in the deployed container. Without that,
extractMatrixCredentials inside the container hits the /chat/ login
wall and never sniffs a bearer.
Add POST /api/debug/matrix-credentials that accepts a credentials JSON
body, persists it to data/matrix-credentials.json, and hot-swaps it
into the running ChatService via setMatrixClient. Also export
saveCredentials from the chat module so the route can reuse the same
cache-file logic as the in-container sniff.
scripts/seed-matrix-creds.ts opens a headed Chromium on the dev's
machine, waits for an interactive Reddit login (detected by polling
/api/me.json from inside the page), opens www.reddit.com/chat/, sniffs
the first authed matrix.redditspace.com request, resolves user_id via
/whoami, and POSTs the result to BOT_URL/api/debug/matrix-credentials.
End-to-end seeding without ever touching the container.
Usage:
BOT_URL=https://red.apps.jbrunton.co.uk tsx scripts/seed-matrix-creds.ts
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
chat.reddit.com now redirects to www.reddit.com/chat/, and that SPA was
bouncing automated logins to the /login/ wall (confirmed via the new
matrix-debug.png endpoint — the captured screenshot shows the login UI,
not the chat surface, despite isLoggedIn() returning true seconds
earlier). Going direct to the canonical URL means the session cookie set
by the automated /login form-fill is applied to the very first request
instead of only after a cross-subdomain redirect, which is where it was
getting dropped. Switch goto from awaited domcontentloaded on the old
URL to a non-awaited domcontentloaded on the new URL so the request
listener can resolve the moment the SPA fires its first authed call to
matrix.redditspace.com.
Adds GET /api/debug/cookies as a temporary diagnostic — returns cookies
visible to www.reddit.com, chat.reddit.com, and matrix.redditspace.com
(values hashed by default; ?reveal=1 to dump raw). Lets us compare what
the browser holds vs. what /chat/ rejects if this doesn't fix it.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
The dashboard's /api/logs buffer only ever saw lines explicitly pushed
via broadcaster.push, so winston output (including the Matrix
token-extractor's timeout diagnostics) was invisible from the deployed
container. Attach a winston transport that forwards every entry into the
LogBroadcaster before the rest of bootstrap runs, so the matrix sniff's
`finalUrl=... hosts=[...]` line shows up in /api/logs.
Also serve data/matrix-debug.png (the screenshot the extractor drops on
timeout) at /api/debug/matrix-debug.png so we can see what
chat.reddit.com looked like in the headed browser without shell access
to the Dokploy host.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
The upstream openresty proxy in front of Dokploy strips the
WebSocket Upgrade header, which kills noVNC. Publishing noVNC
directly on a host port (16080) avoids the proxy entirely.
Auth is still gated by VNC_PASSWORD. The dashboard continues
to go through Traefik on its existing domain.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
A ./data bind mount points at the deploy directory, which Dokploy
rebuilds on every redeploy — wiping the SQLite db, the persistent
browser session, and the cached Matrix credentials. That forced the
slow re-login + chat.reddit.com token sniff on every container start.
Switch to a named docker volume (reddit_bot_data) so the data
directory survives rebuilds.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
If chat.reddit.com sniffing fails, fall through with a placeholder
credential set so the post-monitoring/processor side of the bot
(and the web UI) stay up. Chat features are disabled until the
next successful token refresh.
Also dump diagnostics on sniff timeout: final page URL, every host
the page touched (with request counts + whether any carried an
Authorization header), and a full-page screenshot to data/matrix-debug.png.
This makes the three failure modes (page didn't load / loaded but
never hit Matrix / hit Matrix without auth) distinguishable from logs.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
A bot crash leaves /tmp/.X99-lock behind; the next container start
hits "Server is already active for display 99" and the entrypoint
previously fell through with only a warning, so Chromium then died
with "Missing X server or $DISPLAY" in a tight crash loop. Remove
the stale lock up-front and fail hard if Xvfb still won't come up,
so the failure mode is "container won't start" rather than "bot
crashes forever".
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Adds a durable mapping so the UI never has to show raw t2_xxx IDs or
MXIDs just because the in-memory cache is cold after a restart.
ChatService seeds from the table on construction and writes through
every resolution path; cached chats whose username is still t2_xxx
get upgraded from the DB on loadCache.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
On Dokploy, traffic is routed through Traefik via the configured
domain — publishing host ports caused a collision with another
service already bound to 3000 on the same host. Switch to `expose`
so the ports are reachable on the internal Docker network only.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
The Xvfb readiness loop in entrypoint.sh used xdpyinfo, which isn't
in the Playwright base image. On busy hosts the loop fell through
before Xvfb was ready, so Chromium launched with no X server and
crashed with "Missing X server or \$DISPLAY".
Add x11-utils (provides xdpyinfo) and extend the readiness loop to
10 s with explicit logging.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Chromium in the container was failing with ERR_INTERNET_DISCONNECTED
when navigating to Reddit. Override container DNS to public resolvers
so the browser can reach reddit.com / chat.reddit.com.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
jjbrunton
