diff --git a/.agents/learnings/promoted/2026-08-31-expo-prebuild-native-regression.md b/.agents/learnings/promoted/2026-08-31-expo-prebuild-native-regression.md new file mode 100644 index 0000000..34a3031 --- /dev/null +++ b/.agents/learnings/promoted/2026-08-31-expo-prebuild-native-regression.md @@ -0,0 +1,66 @@ +--- +id: 2026-08-31-expo-prebuild-native-regression +date: 2026-08-31 +status: promoted +scope: apps/mobile native projects (ios/, android/) and expo prebuild +evidence: v1.0.8 SDK 54 to 56 upgrade, commits 604eec3, 7aca39a, 6bd04a6, 79233f5, b8cb2add +destination: apps/mobile/scripts/release-preflight.mjs, apps/mobile/scripts/release-preflight.test.mjs, and docs/releasing.md +review_after: 2026-11-30 +--- + +## Symptom + +`expo prebuild --clean` during the SDK 54 to 56 upgrade (and the +@bacons/apple-targets 0.1 to 5.0 bump) silently regressed four checked-in +native invariants. Each regression was caught only by a release gate or by +manual diffing against the last shipped tag (3fc6f8a, v1.0.7), not by +prebuild itself. + +## Reproduction + +Run `expo prebuild --clean` in apps/mobile against the upgraded SDK and diff +the regenerated ios/ and android/ projects against the previously committed +projects. + +## Cause + +1. Widget identity: apple-targets 5.x derives PRODUCT_BUNDLE_IDENTIFIER from + the target folder name, changing com.sauci.app.LiveDrawWidget to + com.sauci.app.widget, and repointed CODE_SIGN_ENTITLEMENTS to the + gitignored ios/.targets/LiveDrawWidget/generated.entitlements. +2. Android manifest lost the tools:node="remove" attributes on + READ_MEDIA_IMAGES and READ_MEDIA_VIDEO, so the built AAB regained + Play-forbidden permissions (verified via bundletool dump on the rejected + build 50 AAB). +3. iOS Info.plist gained dev-launcher-only keys NSBonjourServices + (_expo._tcp), NSLocalNetworkUsageDescription, and RCTMetroPort. +4. android/gradle.properties reset to stock -Xmx2048m + -XX:MaxMetaspaceSize=512m, causing Metaspace OOM in kspReleaseKotlin/CMake + under RN 0.85. + +Separately, a package absent from the new SDK's bundledNativeModules +(expo-av) has no compatible release and fails the iOS archive (EXAV.h +imports removed ExpoModulesCore headers). + +## Remediation + +Fixed by restoring widget identity and tracking +targets/widget/generated.entitlements with a pinned bundleIdentifier +(604eec3); restoring manifest tools:node="remove" attributes (7aca39a); +removing the dev-launcher-only Info.plist keys (6bd04a6); raising +gradle.properties memory to 4096m/1024m (79233f5); and migrating expo-av to +expo-video/expo-audio (b8cb2add). + +Extend apps/mobile/scripts/release-preflight.mjs with executable checks for +items 2 and 3 (item 1 is already covered by the existing LiveDrawWidget +preflight check; item 4 hard-fails the build by itself). Add an "After +native regeneration" checklist to docs/releasing.md. + +## Evidence + +`node apps/mobile/scripts/release-preflight.mjs` fails if the tracked +AndroidManifest.xml lacks tools:node="remove" on READ_MEDIA_IMAGES or +READ_MEDIA_VIDEO, or if ios/Sauci/Info.plist contains NSBonjourServices, +NSLocalNetworkUsageDescription, or RCTMetroPort. +`node --test apps/mobile/scripts/release-preflight.test.mjs` covers the new +checks. diff --git a/.agents/learnings/promoted/2026-08-31-local-eas-build-preconditions.md b/.agents/learnings/promoted/2026-08-31-local-eas-build-preconditions.md new file mode 100644 index 0000000..f6fa84d --- /dev/null +++ b/.agents/learnings/promoted/2026-08-31-local-eas-build-preconditions.md @@ -0,0 +1,55 @@ +--- +id: 2026-08-31-local-eas-build-preconditions +date: 2026-08-31 +status: promoted +scope: apps/mobile local EAS release builds +evidence: v1.0.8 local EAS build session, commit d8bedea +destination: docs/releasing.md +review_after: 2026-11-30 +--- + +## Symptom + +Local EAS release builds failed before native compilation for environment +reasons unrelated to the app source, and a successful iOS build left the +tracked source in a state that fails release-preflight afterward. + +## Reproduction + +Run `eas build --local` from a non-interactive shell without first exporting +ANDROID_HOME/ANDROID_SDK_ROOT/JAVA_HOME: fails with "SDK location not +found" because non-interactive shells do not load the user profile. Run it +with a non-empty EAS_LOCAL_BUILD_WORKINGDIR: fails with "Workingdir is not +empty". Run it outside apps/mobile: eas cannot find the project. Pipe the +build log through `tail`: the full log needed for diagnosis is lost. After a +successful iOS build, EAS auto-increment updates both Info.plists but not +the widget target's CURRENT_PROJECT_VERSION in the pbxproj, so +release-preflight fails until it is synced. + +## Cause + +`eas` must be installed as a global CLI (scripts call bare `eas`; eas-cli +23.0.0 verified). Non-interactive shells used for automation do not source +the profile that normally exports the Android/Java environment variables. +EAS_LOCAL_BUILD_WORKINGDIR must not exist or be empty. EAS iOS +auto-increment writes plist versions but does not touch the widget target's +pbxproj CURRENT_PROJECT_VERSION, so it drifts from the app's build number by +one increment after every successful build. + +## Remediation + +Before a local EAS build: verify `eas` is on PATH, explicitly export +ANDROID_HOME, ANDROID_SDK_ROOT, and JAVA_HOME (JDK 17), ensure +EAS_LOCAL_BUILD_WORKINGDIR does not exist or is empty, and run from +apps/mobile. Capture the full build log to a file rather than piping through +`tail`. After a successful build, sync the widget's CURRENT_PROJECT_VERSION +to the new build number (commit d8bedea, sed 46 to 47) and sync +ios/Podfile.lock from the successful build's working directory when +dependencies changed, since the archive's pod install resolution is +authoritative and local CocoaPods here is broken. + +## Evidence + +docs/releasing.md's local-build section documents the environment +preconditions and the post-build widget CURRENT_PROJECT_VERSION and +Podfile.lock sync steps; `npm run release:preflight` passes after the sync. diff --git a/.agents/learnings/promoted/2026-08-31-npm-lockfile-divergence.md b/.agents/learnings/promoted/2026-08-31-npm-lockfile-divergence.md new file mode 100644 index 0000000..79b256f --- /dev/null +++ b/.agents/learnings/promoted/2026-08-31-npm-lockfile-divergence.md @@ -0,0 +1,55 @@ +--- +id: 2026-08-31-npm-lockfile-divergence +date: 2026-08-31 +status: promoted +scope: root package.json, package-lock.json, apps/web dependencies, CI +evidence: v1.0.8 lockfile investigation, commits 90cde4a, 9609a90 +destination: docs/development/dependencies.md +review_after: 2026-11-30 +--- + +## Symptom + +CI and the API Docker image (node 20.19.4/npm 10.8.2, per .nvmrc) broke +invisibly while a lockfile regenerated on this dev machine (node 26/npm +11.19) passed local `npm ci`. Pushes to staging bypass required status +checks, so the red CI ran unnoticed for hours. + +## Reproduction + +Regenerate package-lock.json with npm 11 on a wildcard-range dependency +(react-native-worklets depends on `@react-native/metro-config: "*"`), then +run `npm ci` under node 20.19.4/npm 10.8.2. npm 10 requires 0.85.3 at that +tree position; npm 11 resolves/hoists 0.87.x there. `npm ci` fails with +"Invalid: lock file's ... does not satisfy". The reverse direction also +fails: a lock regenerated by npm 10 fails `npm ci` under npm 11 with missing +platform-optional binaries (@next/swc-linux-x64-gnu breaking `next build`, +@rollup/rollup-linux-x64-gnu breaking the API Docker tsup build, lightningcss +variants breaking local npm ci), matching the npm/cli#4828-style +optional-dependency resolution gap. + +## Cause + +Two different npm major versions in active use (CI/Docker on npm 10, this +dev machine on npm 11) resolve wildcard/range dependencies to different +versions at the same tree position, and each npm's own platform-optional +binary selection is not portable to the other npm's lockfile. + +## Remediation + +Pin the wildcard-range dependency via a root package.json override +(`"@react-native/metro-config": "0.85.3"`, alongside the existing +rollup/react-native/expo pins). Pin apps/web `@next/swc` linux/darwin +binaries as direct optionalDependencies (must be bumped together with any +`next` version bump). Regenerate the lock and validate it both ways: local +`npm ci` plus `npm run verify:fast`, and +`docker run --platform linux/amd64 node:20.19.4 ... npm ci` (the +`--platform` flag is mandatory; Apple Silicon Docker otherwise runs arm64 +and the platform-binary check is meaningless). After any push that bypassed +required status checks, check `gh run list --branch staging`. + +## Evidence + +commits 90cde4a and 9609a90 add the override and optionalDependencies pins +and record a lockfile that passes both the amd64-emulated `npm ci` under +node 20.19.4 and local `npm ci` plus `npm run verify:fast`. diff --git a/AGENTS.md b/AGENTS.md index a1122af..dc14b5c 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -76,6 +76,7 @@ environment limitation; do not call it a passing product check. |---|---| | System boundaries and data flow | `docs/architecture/system.md` | | Local services, ports, fixtures | `docs/development/local-stack.md` | +| Dependency and lockfile discipline | `docs/development/dependencies.md` | | Test layers and required gates | `docs/testing/strategy.md` | | Database migration safety | `docs/operations/database-migrations.md` | | Agent roles and effort routing | `docs/agents/routing.md` | diff --git a/docs/development/dependencies.md b/docs/development/dependencies.md new file mode 100644 index 0000000..459581f --- /dev/null +++ b/docs/development/dependencies.md @@ -0,0 +1,76 @@ +# Dependency and lockfile discipline + +## Two npm realities + +CI runners and the API Docker image use the node version pinned in +`.nvmrc` (node 20.19.4 with npm 10.8.2). A developer machine can run a newer +npm (for example npm 11). npm 10 and npm 11 resolve wildcard or range +dependencies differently at the same place in the dependency tree, and each +npm's platform-optional binary selection is not portable to a lockfile +regenerated by the other npm. A lockfile that passes `npm ci` under one npm +major version can fail it under the other with either an "Invalid: lock +file's ... does not satisfy" error or missing platform-optional binaries +(for example `@next/swc-linux-x64-gnu` breaking `next build`, +`@rollup/rollup-linux-x64-gnu` breaking the API Docker `tsup` build, or a +missing `lightningcss` variant breaking local `npm ci`). + +## Pin wildcard-range dependencies with root overrides + +When a transitive dependency declares a wildcard or wide range (for example +`react-native-worklets` depending on `@react-native/metro-config: "*"`), pin +the exact version the app needs in the root `package.json` `overrides` block +rather than leaving it to whichever npm resolved the lockfile last: + +```json +"overrides": { + "rollup": "4.54.0", + "react-native": "0.85.3", + "expo": "~56.0.0", + "@react-native/metro-config": "0.85.3" +} +``` + +## Pin platform-optional binaries that must travel with the framework version + +`apps/web/package.json` pins the `@next/swc` linux and darwin binaries as +direct `optionalDependencies` so npm does not have to infer them per +platform: + +```json +"optionalDependencies": { + "@next/swc-linux-x64-gnu": "15.5.24", + "@next/swc-linux-x64-musl": "15.5.24", + "@next/swc-darwin-arm64": "15.5.24" +} +``` + +These versions must be bumped together with any `next` version bump, or the +pinned binaries will drift from the framework version and `npm ci` or +`next build` can fail on the platform that needs the stale binary. + +## Validate a regenerated lockfile both ways + +After changing `overrides` or any dependency version, regenerate +`package-lock.json` and validate it under both npm majors before committing: + +```bash +# Local npm (matches this machine's npm major) +npm ci +npm run verify:fast + +# The CI/Docker npm major, emulated on Apple Silicon +docker run --rm --platform linux/amd64 -v "$PWD:/repo" -w /repo node:20.19.4 npm ci +``` + +The `--platform linux/amd64` flag is mandatory on Apple Silicon: without it, +Docker runs the arm64 image and the platform-binary check proves nothing. + +## After any bypassed push + +Pushes to `staging` bypass required status checks. After any such push, check +`gh run list --branch staging` to confirm CI is actually green rather than +assuming it ran and passed. + +Do not pipe `npm` commands through `tail` when diagnosing a lockfile or build +failure; it can swallow the command's exit code and the detail needed to find +the root cause. diff --git a/docs/index.md b/docs/index.md index 2744e05..26df3e9 100644 --- a/docs/index.md +++ b/docs/index.md @@ -8,6 +8,7 @@ historical notes. Agents should load only the section relevant to their task. - [System architecture](architecture/system.md) - [Client data freshness](architecture/client-data-freshness.md) - [Local full-stack development](development/local-stack.md) +- [Dependency and lockfile discipline](development/dependencies.md) - [Testing strategy](testing/strategy.md) - [Database migration safety](operations/database-migrations.md) - [Backend deployment and branching](operations/backend-deployments.md) diff --git a/docs/releasing.md b/docs/releasing.md index 4b9e71c..d6c38e9 100644 --- a/docs/releasing.md +++ b/docs/releasing.md @@ -249,6 +249,23 @@ and dependency installation failures occur before native compilation: fix and preflight those inputs without starting another build. Make one meaningful build per newly proven input state. +Before starting: confirm `eas` is installed and on PATH (scripts call bare +`eas`); explicitly export `ANDROID_HOME`, `ANDROID_SDK_ROOT`, and `JAVA_HOME` +(JDK 17) rather than relying on a shell profile, because non-interactive shells +used for automation do not load it ("SDK location not found" otherwise); ensure +`EAS_LOCAL_BUILD_WORKINGDIR` does not exist or is empty ("Workingdir is not +empty" otherwise); and run every command from `apps/mobile`. Capture the full +build log to a file; do not pipe it through `tail`, which drops the detail +needed to diagnose a failure. + +After a successful iOS build, EAS auto-increment updates both app and widget +Info.plists but not the widget target's `CURRENT_PROJECT_VERSION` in the +pbxproj, so `release:preflight` fails until it is synced to the new build +number (for example `sed -i '' 's/CURRENT_PROJECT_VERSION = 46;/CURRENT_PROJECT_VERSION = 47;/' ios/Sauci.xcodeproj/project.pbxproj` +for both Debug and Release). If native dependencies changed, also sync +`ios/Podfile.lock` from the successful build's working directory: the archive's +`pod install` resolution is authoritative when local CocoaPods is broken. + Report provider progress by phase: packaging, native compilation, submission queue, or store processing. Do not describe every failure as a rebuild. @@ -292,6 +309,24 @@ verify it, and commit it before starting the release from a clean worktree. Use 5.x, a non-clean prebuild against an already-generated iOS project fails while updating the widget's Xcode target. +**After native regeneration:** `expo prebuild --clean` can silently regress +checked-in native invariants (confirmed during the SDK 54 to 56 upgrade). Before +committing a regenerated project: + +- Diff pbxproj signing and identity keys (PRODUCT_BUNDLE_IDENTIFIER, + CODE_SIGN_ENTITLEMENTS) against the last release tag; apple-targets can derive + a widget's bundle identifier from its folder name instead of the pinned config. +- Semantic-diff Info.plist and entitlements files against the last release tag; + prebuild can reintroduce dev-launcher-only keys (NSBonjourServices, + NSLocalNetworkUsageDescription, RCTMetroPort) and drop widget entitlements. +- Check android/gradle.properties for a reset `org.gradle.jvmargs` (prebuild + resets it to the stock -Xmx2048m/512m, which OOMs release Kotlin/CMake tasks). +- Run `npm run release:preflight`; it enforces the Android manifest media + permission removal and rejects the dev-launcher-only Info.plist keys. +- Treat `expo-doctor` unmaintained-package warnings as release blockers on an + SDK upgrade: a package absent from the new SDK's bundledNativeModules can fail + the iOS archive with no compatible release available. + **Out of memory:** ```bash # Increase Gradle memory