From 7e79cca552a767aec7d4edcad3ae8a7edf4df300 Mon Sep 17 00:00:00 2001 From: help4bis Date: Fri, 10 Apr 2026 18:37:16 +1000 Subject: [PATCH 1/7] feat: init claude-notes directory --- help4bis-claude-notes/.gitkeep | 0 1 file changed, 0 insertions(+), 0 deletions(-) create mode 100644 help4bis-claude-notes/.gitkeep diff --git a/help4bis-claude-notes/.gitkeep b/help4bis-claude-notes/.gitkeep new file mode 100644 index 0000000..e69de29 From c6f997b42d70c5f80147eae76c5add360d740cdf Mon Sep 17 00:00:00 2001 From: help4bis Date: Fri, 10 Apr 2026 18:38:03 +1000 Subject: [PATCH 2/7] docs: session decisions 2026-04-10 --- help4bis-claude-notes/decisions.md | 23 +++++++++++++++++++++++ 1 file changed, 23 insertions(+) create mode 100644 help4bis-claude-notes/decisions.md diff --git a/help4bis-claude-notes/decisions.md b/help4bis-claude-notes/decisions.md new file mode 100644 index 0000000..ad92c65 --- /dev/null +++ b/help4bis-claude-notes/decisions.md @@ -0,0 +1,23 @@ +### 2026-04-10 18:00 — Enable Olm E2EE in web build +- **Decision:** Re-enabled Olm encryption in index.html with synchronous load + Olm.init() before Flutter boots, graceful fallback if Olm fails +- **Rationale:** 45 of 88 rooms have Megolm encryption. MatrixSdkDatabase persists Olm keys in IndexedDB resolving /keys/upload 400 errors +- **Files affected:** web/index.html, lib/main.dart, app_config.dart, auth_repository.dart, pubspec.yaml +- **Status:** Working — device keys uploaded, messages encrypt/decrypt correctly + +### 2026-04-10 17:25 — Deploy target is app2.m8chat.au NOT app.m8chat.au +- **Decision:** app.m8chat.au = PRODUCTION (old FluffyChat). app2.m8chat.au = dev/test for m8chat-app2 +- **Rationale:** User corrected after accidental production deploy. Production restored from backup. +- **Status:** All future deploys to app2.m8chat.au + +### 2026-04-10 17:15 — Lazy-load Jitsi external_api.js +- **Decision:** Moved from index.html to dynamic load in jitsi_web_service.dart +- **Rationale:** Brave blocks cross-origin scripts at page load +- **Status:** Done + +### 2026-04-10 17:10 — Add MessageType.encrypted with lock icon UX +- **Files:** message_model.dart, chat_repository.dart, message_bubble.dart, room_preview.dart +- **Status:** Done + +### 2026-04-10 17:05 — Fix web metadata to M8Chat branding +- **Files:** web/index.html, web/manifest.json +- **Status:** Done From ae61c06252143ff06cb19cc55eb7367ffc217ea2 Mon Sep 17 00:00:00 2001 From: help4bis Date: Fri, 10 Apr 2026 18:38:42 +1000 Subject: [PATCH 3/7] docs: current state 2026-04-10 --- help4bis-claude-notes/current-state.md | 51 ++++++++++++++++++++++++++ 1 file changed, 51 insertions(+) create mode 100644 help4bis-claude-notes/current-state.md diff --git a/help4bis-claude-notes/current-state.md b/help4bis-claude-notes/current-state.md new file mode 100644 index 0000000..96f0666 --- /dev/null +++ b/help4bis-claude-notes/current-state.md @@ -0,0 +1,51 @@ +## M8Chat App2 — Current State (2026-04-10) + +Custom Flutter Matrix chat client (v1.2.0) deployed to app2.m8chat.au for testing. +Production (app.m8chat.au) still runs the old FluffyChat fork — DO NOT deploy there. + +### What works +- **Chat messaging** — rooms list, timeline, send/receive, reactions, replies, file upload, typing indicators +- **E2EE (Olm/Megolm)** — Olm loads in browser, device keys upload, messages encrypt/decrypt between app2 and Element +- **Spaces** — expandable space list with child rooms +- **User search** — search Matrix user directory, start DMs +- **Jitsi conferencing** — lazy-loaded iframe, no-login required +- **LiveKit video/audio calls** — JWT fetch works, WebRTC connects, video+audio publish to LiveKit SFU +- **Session persistence** — credentials in flutter_secure_storage, events in MatrixSdkDatabase (IndexedDB) +- **Brave browser** — app loads and works (Olm included, graceful fallback if WASM blocked) + +### What is broken / incomplete +- **Video call rendering — SCRAMBLED VIDEO from Element X**: Root cause identified. Element X sends media with `encryption: GCM` (AES-GCM frame encryption). App2 sends `encryption: NONE`. The LiveKit Flutter SDK needs E2EE frame encryption enabled to decrypt Element X video. This is the #1 priority fix. +- **Incoming call detection**: App listens for old m.call.invite events. Element X sends MSC3401 call.member state events. Incoming calls from Element X to app2 are not detected. +- **Avatar 404s**: ~10 room avatars return 404 on legacy /_matrix/media/v3/download/ path. Server may require authenticated media. +- **Rooms list spam**: debugPrint in rooms_repository.dart dumps all 15 rooms to console on every sync. Needs removal. +- **Old ghost devices**: 20 devices for @try user, 5 without keys (from pre-Olm builds). Should be cleaned up server-side. +- **Red exclamation mark in Element**: Try device is unverified (no cross-signing). Cosmetic — requires manual verification or Phase 2 cross-signing. +- **Profile screen**: E2EE status shows dynamically but notifications/cross-signing are placeholders. + +### Key file paths +- **Project root**: /srv/wp-dev/pwa-sites/m8chat-app2/ +- **Web entry**: web/index.html (Olm load + fallback + Flutter bootstrap) +- **App config**: lib/core/config/app_config.dart (server URLs, version 1.2.0) +- **Matrix client**: lib/core/network/matrix_client.dart (MatrixSdkDatabase provider) +- **Auth flow**: lib/core/auth/auth_notifier.dart + lib/features/auth/data/auth_repository.dart +- **LiveKit service**: lib/features/calls/data/livekit_service.dart (JWT fetch, MatrixRTC state events) +- **Call screen**: lib/features/calls/presentation/call_screen.dart (video rendering) +- **Call controller**: lib/features/calls/presentation/call_controller.dart (state machine) +- **Chat repo**: lib/features/chat/data/chat_repository.dart (timeline, send, react, redact) +- **Jitsi service**: lib/features/jitsi/data/jitsi_web_service.dart (lazy script load) +- **Build output**: build/web/ (flutter build web --release) + +### Server infrastructure (chat.m8chat.au) +- Synapse 1.151.0 (healthy, 86 active users) +- LiveKit 1.10.1 + lk-jwt-service (both healthy) +- Element Web 1.12.15, Element Call +- PostgreSQL on host (user: synapse_user, pass: kijPt4cPWkoaZk8BVm, db: synapse) +- LiveKit WSS URL: wss://matrix.m8chat.au/livekit/ +- JWT endpoint: https://matrix.m8chat.au/_matrix/livekit/jwt/sfu/get + +### Deployment +- **Target**: app2.m8chat.au (NOT app.m8chat.au) +- **SSH**: m8chat@app.m8chat.au:2233 (key: ~/.ssh/m8chat_prod), path: public_html/app2.m8chat.au/ +- **Build**: cd /srv/wp-dev/pwa-sites/m8chat-app2 && flutter build web --release +- **Deploy**: rsync -avz --delete --exclude='.htaccess' --exclude='.ftpquota' -e "ssh -p 2233 -i ~/.ssh/m8chat_prod" build/web/ m8chat@app.m8chat.au:public_html/app2.m8chat.au/ +- **Backup on server**: public_html/app2.m8chat.au_backup_20260410_*.tar.gz From 72020bb9b1a2e5211cdd4de09112a76c7d24fee9 Mon Sep 17 00:00:00 2001 From: help4bis Date: Fri, 10 Apr 2026 18:39:03 +1000 Subject: [PATCH 4/7] docs: remaining work 2026-04-10 --- help4bis-claude-notes/remaining-work.md | 46 +++++++++++++++++++++++++ 1 file changed, 46 insertions(+) create mode 100644 help4bis-claude-notes/remaining-work.md diff --git a/help4bis-claude-notes/remaining-work.md b/help4bis-claude-notes/remaining-work.md new file mode 100644 index 0000000..ee443c7 --- /dev/null +++ b/help4bis-claude-notes/remaining-work.md @@ -0,0 +1,46 @@ +## Remaining Work — Priority Order + +### P0 — LiveKit E2EE frame encryption (BLOCKING video calls) +- [ ] Enable AES-GCM frame encryption in LiveKit Flutter SDK to match Element X +- Element X publishes media with `encryption: GCM`, app2 publishes `encryption: NONE` +- Without matching encryption, remote video renders as scrambled garbage +- Check livekit_client docs for `E2EEOptions` / `FrameCryptorFactory` on web +- May need to pass shared key material derived from the MatrixRTC session +- **Start here next session**: read livekit_client 2.4.0 source for E2EE setup on web + +### P1 — Incoming call detection from Element X +- [ ] Element X sends MSC3401 `org.matrix.msc3401.call.member` state events (already in hidden events list) +- [ ] App currently listens for old `m.call.invite` events in matrixrtc_repository.dart +- [ ] Need to detect call.member state changes and show incoming call overlay +- The state event format includes `memberships[]` with `foci_active[].type: livekit` + +### P2 — Cleanup and polish +- [ ] Remove debug logging from rooms_repository.dart (debugPrint of all rooms on every sync) +- [ ] Fix avatar 404s — switch to authenticated media API (/_matrix/client/v1/media/) +- [ ] Clean up 20 ghost devices for @try user on server (DELETE via admin API) +- [ ] Remove diagnostic Olm logging from index.html (console.log statements) +- [ ] Remove debug prints from auth_repository.dart (encryptionEnabled logging) + +### P3 — Voice calling +- [ ] Test voice-only calls (same LiveKit flow, just no video track) +- [ ] Likely same E2EE issue as video — fix P0 first + +### P4 — Cross-signing (Phase 2) +- [ ] Device verification to remove red exclamation in Element +- [ ] Profile screen hooks for verify/cross-sign + +### Build and deploy commands +```bash +cd /srv/wp-dev/pwa-sites/m8chat-app2 +dart analyze --fatal-infos +flutter build web --release +rsync -avz --delete --exclude='.htaccess' --exclude='.ftpquota' \ + -e "ssh -p 2233 -i ~/.ssh/m8chat_prod" \ + build/web/ m8chat@app.m8chat.au:public_html/app2.m8chat.au/ +``` + +### Next session should start with +1. Read these claude-notes +2. Research LiveKit E2EE on Flutter web — check `livekit_client` package for `E2EEOptions`, `FrameCryptor`, `KeyProvider` +3. Check if Element X derives the E2EE key from the MatrixRTC session or uses a separate mechanism +4. Implement and test — the scrambled video fix is the most impactful single change From a01c527157501d3dd5ac8e6ca87ec6693afa9b42 Mon Sep 17 00:00:00 2001 From: help4bis Date: Sat, 11 Apr 2026 03:53:46 +1000 Subject: [PATCH 5/7] claude-notes: update decisions.md (2026-04-11 03:53) --- help4bis-claude-notes/decisions.md | 48 ++++++++++++++++++++++++++++++ 1 file changed, 48 insertions(+) diff --git a/help4bis-claude-notes/decisions.md b/help4bis-claude-notes/decisions.md index ad92c65..30d7005 100644 --- a/help4bis-claude-notes/decisions.md +++ b/help4bis-claude-notes/decisions.md @@ -21,3 +21,51 @@ ### 2026-04-10 17:05 — Fix web metadata to M8Chat branding - **Files:** web/index.html, web/manifest.json - **Status:** Done + +### 2026-04-11 02:00 — Add Jitsi Conference tab to bottom nav +- **Decision:** Added 4th tab (videocam icon) to authenticated bottom nav so logged-in users can join Jitsi conferences +- **Rationale:** Jitsi join was only accessible on the unauthenticated welcome screen. Logged-in users could never reach it. +- **Files affected:** rooms_screen.dart (v1.3.0), conference_tab.dart (new), jitsi_screen.dart (Back→/rooms) +- **Status:** Done and deployed + +### 2026-04-11 03:00 — LiveKit E2EE frame encryption attempt +- **Decision:** Added CallE2EEManager for m.rtc.encryption_keys to-device key exchange + E2EEOptions on LiveKit Room +- **Rationale:** Element X encrypts video frames with AES-GCM. Without decryption, video is scrambled. +- **Files affected:** call_e2ee.dart (new), livekit_service.dart (v1.4.0) +- **Status:** PARTIALLY WORKING — E2EE worker requires SharedArrayBuffer which needs COOP/COEP headers. Headers added to .htaccess on 2026-04-11. Key exchange protocol (m.rtc.encryption_keys) implemented but interop with Element X NOT YET VERIFIED. + +### 2026-04-11 04:00 — Web auth: fresh login every session (no session persistence) +- **Decision:** Removed session restore entirely for web. Every visit requires login. No access token stored. +- **Rationale:** Security — web app runs on untrusted/shared devices. Can't assume same user. Different from future APK which will persist sessions. +- **Files affected:** auth_notifier.dart (v2.0.0), secure_storage.dart (v2.0.0), auth_repository.dart (v2.0.0), app_config.dart (v2.0.0) +- **Status:** Done. Removed StoredCredentials, restoreSession(), storage key constants. Device ID persistence also removed (caused keys/upload 400 because Olm account not persisted). + +### 2026-04-11 04:30 — SSSS + cross-signing + key backup bootstrap +- **Decision:** Implemented full security setup dialog using Matrix SDK Bootstrap class +- **Rationale:** Users need to create/restore SSSS to decrypt old messages. Replaces "Phase 2" placeholder. +- **Files affected:** security_setup_dialog.dart (new), key_restore_dialog.dart (new), profile_screen.dart (v1.4.0) +- **Status:** Done. User 'try' successfully ran security setup. Recovery key generated. + +### 2026-04-11 05:00 — Fix authenticated media (Synapse 1.151.0) +- **Decision:** Switched from frozen /_matrix/media/v3/download/ to /_matrix/client/v1/media/download/ with access_token query param +- **Rationale:** Synapse 1.151.0 froze the old media endpoint. All room avatars returned 404. +- **Files affected:** mxc_url.dart (v2.0.0) +- **Status:** Done — avatars load correctly now + +### 2026-04-11 06:00 — Add Help tab + Account management info +- **Decision:** Added 5th bottom nav tab (Help) with expandable help cards +- **Rationale:** Users need in-app guidance for encrypted messages, security setup, account management +- **Files affected:** help_tab.dart (new), rooms_screen.dart (v1.3.0) +- **Status:** Done. Contains: encrypted messages fix guide, getting started, conference guide, video call issues, account management (links to m8chat.au), privacy statement. + +### 2026-04-11 07:00 — COOP/COEP headers for E2EE workers +- **Decision:** Added Cross-Origin-Opener-Policy and Cross-Origin-Embedder-Policy to .htaccess +- **Rationale:** LiveKit E2EE frame encryption uses Web Workers that need SharedArrayBuffer +- **Files affected:** .htaccess on app2.m8chat.au (production server) +- **Status:** Headers verified active. E2EE re-enabled in code. NOT YET TESTED with a live call. + +### 2026-04-11 07:00 — Fix hangup call termination +- **Decision:** Changed _clearCallMemberEvent to send {} instead of {'memberships': []} +- **Rationale:** Element X sends {} when ending calls (verified from DB). Our {'memberships': []} wasn't being recognised. +- **Files affected:** livekit_service.dart +- **Status:** Deployed. NOT YET TESTED. \ No newline at end of file From 93442790f3ae03f6dc9476d3be175d41cf167760 Mon Sep 17 00:00:00 2001 From: help4bis Date: Sat, 11 Apr 2026 03:54:26 +1000 Subject: [PATCH 6/7] claude-notes: update current-state.md (2026-04-11 03:54) --- help4bis-claude-notes/current-state.md | 98 +++++++++++++++----------- 1 file changed, 55 insertions(+), 43 deletions(-) diff --git a/help4bis-claude-notes/current-state.md b/help4bis-claude-notes/current-state.md index 96f0666..3997316 100644 --- a/help4bis-claude-notes/current-state.md +++ b/help4bis-claude-notes/current-state.md @@ -1,51 +1,63 @@ -## M8Chat App2 — Current State (2026-04-10) +## M8Chat App2 — Current State (2026-04-11) -Custom Flutter Matrix chat client (v1.2.0) deployed to app2.m8chat.au for testing. -Production (app.m8chat.au) still runs the old FluffyChat fork — DO NOT deploy there. +Custom Flutter Matrix chat client deployed to app2.m8chat.au. Replaces the FluffyChat fork (app.m8chat.au). Built on Matrix Dart SDK 0.33.0, LiveKit for video, Olm for E2EE. + +### What was built/changed this session + +1. **Jitsi Conference tab** — 4th bottom nav tab for joining conf.m8chat.au meetings +2. **Help tab** — 5th bottom nav tab with troubleshooting guides, account management links +3. **Auth rewrite (v2.0.0)** — fresh login every session on web, no credential persistence +4. **SSSS/cross-signing/key backup** — full security setup dialog (Bootstrap flow) +5. **Key restore dialog** — enter recovery key to download Megolm keys from backup +6. **Authenticated media (v2.0.0)** — switched to /_matrix/client/v1/media/download/ (Synapse 1.151.0 froze old endpoint) +7. **LiveKit E2EE** — CallE2EEManager for m.rtc.encryption_keys key exchange, COOP/COEP headers +8. **Hangup fix** — sends {} to clear call.member state event +9. **Cleaned up 24 ghost devices** for try user (DB direct delete) ### What works -- **Chat messaging** — rooms list, timeline, send/receive, reactions, replies, file upload, typing indicators -- **E2EE (Olm/Megolm)** — Olm loads in browser, device keys upload, messages encrypt/decrypt between app2 and Element -- **Spaces** — expandable space list with child rooms -- **User search** — search Matrix user directory, start DMs -- **Jitsi conferencing** — lazy-loaded iframe, no-login required -- **LiveKit video/audio calls** — JWT fetch works, WebRTC connects, video+audio publish to LiveKit SFU -- **Session persistence** — credentials in flutter_secure_storage, events in MatrixSdkDatabase (IndexedDB) -- **Brave browser** — app loads and works (Olm included, graceful fallback if WASM blocked) +- Login/logout flow (fresh every session) +- Room list with avatars (authenticated media endpoint) +- Chat messaging (text) +- Encrypted message display (lock icon for undecryptable) +- Security setup (SSSS + cross-signing + key backup creation) +- Key restore from backup +- Jitsi conference joining (Conference tab) +- Help tab with troubleshooting content +- Spaces view +- Profile with security settings -### What is broken / incomplete -- **Video call rendering — SCRAMBLED VIDEO from Element X**: Root cause identified. Element X sends media with `encryption: GCM` (AES-GCM frame encryption). App2 sends `encryption: NONE`. The LiveKit Flutter SDK needs E2EE frame encryption enabled to decrypt Element X video. This is the #1 priority fix. -- **Incoming call detection**: App listens for old m.call.invite events. Element X sends MSC3401 call.member state events. Incoming calls from Element X to app2 are not detected. -- **Avatar 404s**: ~10 room avatars return 404 on legacy /_matrix/media/v3/download/ path. Server may require authenticated media. -- **Rooms list spam**: debugPrint in rooms_repository.dart dumps all 15 rooms to console on every sync. Needs removal. -- **Old ghost devices**: 20 devices for @try user, 5 without keys (from pre-Olm builds). Should be cleaned up server-side. -- **Red exclamation mark in Element**: Try device is unverified (no cross-signing). Cosmetic — requires manual verification or Phase 2 cross-signing. -- **Profile screen**: E2EE status shows dynamically but notifications/cross-signing are placeholders. +### What's broken / untested +- **Video calls (E2EE interop)** — COOP/COEP headers added, E2EE re-enabled, but NOT YET TESTED. May still show scrambled video if key exchange doesn't interop with Element X's m.rtc.encryption_keys format. +- **Hangup termination** — Changed to send {} but NOT YET TESTED whether Element X recognises it. +- **Incoming call detection** — Not implemented. No MSC3401 state event listener. +- **Avatar 404s in cached/old data** — sync_persistence_service caches URLs; stale URLs may linger in IndexedDB +- **Ghost devices accumulate** — each web login creates a new device. No cleanup mechanism. +- **Rooms provider fires excessively** — console shows room list logged 15+ times per sync cycle (performance) +- **Noto font warning** — emoji characters in room names can't render (missing font asset) ### Key file paths -- **Project root**: /srv/wp-dev/pwa-sites/m8chat-app2/ -- **Web entry**: web/index.html (Olm load + fallback + Flutter bootstrap) -- **App config**: lib/core/config/app_config.dart (server URLs, version 1.2.0) -- **Matrix client**: lib/core/network/matrix_client.dart (MatrixSdkDatabase provider) -- **Auth flow**: lib/core/auth/auth_notifier.dart + lib/features/auth/data/auth_repository.dart -- **LiveKit service**: lib/features/calls/data/livekit_service.dart (JWT fetch, MatrixRTC state events) -- **Call screen**: lib/features/calls/presentation/call_screen.dart (video rendering) -- **Call controller**: lib/features/calls/presentation/call_controller.dart (state machine) -- **Chat repo**: lib/features/chat/data/chat_repository.dart (timeline, send, react, redact) -- **Jitsi service**: lib/features/jitsi/data/jitsi_web_service.dart (lazy script load) -- **Build output**: build/web/ (flutter build web --release) +| File | Version | Purpose | +|------|---------|---------| +| lib/core/auth/auth_notifier.dart | 2.0.0 | Auth state machine — fresh login, no restore | +| lib/core/auth/secure_storage.dart | 2.0.0 | Minimal — only stores device ID (currently unused) | +| lib/core/config/app_config.dart | 2.0.0 | Server URLs, app name/version | +| lib/shared/utils/mxc_url.dart | 2.0.0 | Authenticated media URL resolution | +| lib/features/calls/data/livekit_service.dart | 1.4.0 | LiveKit connect/disconnect with E2EE | +| lib/features/calls/data/call_e2ee.dart | 1.0.0 | E2EE key exchange via m.rtc.encryption_keys | +| lib/features/profile/presentation/security_setup_dialog.dart | 1.0.0 | SSSS Bootstrap driver | +| lib/features/profile/presentation/key_restore_dialog.dart | 1.0.0 | Recovery key input + loadAllKeys | +| lib/features/help/presentation/help_tab.dart | 1.1.0 | In-app help content | +| lib/features/jitsi/presentation/conference_tab.dart | 1.0.0 | Jitsi meeting join UI | +| lib/features/rooms/presentation/rooms_screen.dart | 1.3.0 | 5-tab bottom nav | -### Server infrastructure (chat.m8chat.au) -- Synapse 1.151.0 (healthy, 86 active users) -- LiveKit 1.10.1 + lk-jwt-service (both healthy) -- Element Web 1.12.15, Element Call -- PostgreSQL on host (user: synapse_user, pass: kijPt4cPWkoaZk8BVm, db: synapse) -- LiveKit WSS URL: wss://matrix.m8chat.au/livekit/ -- JWT endpoint: https://matrix.m8chat.au/_matrix/livekit/jwt/sfu/get +### Server configuration +- **COOP/COEP headers** added to ~/public_html/app2.m8chat.au/.htaccess +- Synapse 1.151.0 on chat.m8chat.au (PostgreSQL, SQLite was wrong assumption) +- LiveKit 1.10.1 on chat.m8chat.au +- lk-jwt-service for MatrixRTC JWT at /_matrix/livekit/jwt/sfu/get -### Deployment -- **Target**: app2.m8chat.au (NOT app.m8chat.au) -- **SSH**: m8chat@app.m8chat.au:2233 (key: ~/.ssh/m8chat_prod), path: public_html/app2.m8chat.au/ -- **Build**: cd /srv/wp-dev/pwa-sites/m8chat-app2 && flutter build web --release -- **Deploy**: rsync -avz --delete --exclude='.htaccess' --exclude='.ftpquota' -e "ssh -p 2233 -i ~/.ssh/m8chat_prod" build/web/ m8chat@app.m8chat.au:public_html/app2.m8chat.au/ -- **Backup on server**: public_html/app2.m8chat.au_backup_20260410_*.tar.gz +### Known technical debt +- call_e2ee.dart imported but E2EE interop with Element X not verified +- secure_storage.dart exists but device ID save/load is unused (removed to fix keys/upload 400) +- key_restore_dialog.dart is separate from security_setup_dialog.dart — could be consolidated +- Rooms provider rebuilds too frequently (every sync tick) \ No newline at end of file From 255e7a29f69b502e039edf79505f87c7dcebf208 Mon Sep 17 00:00:00 2001 From: help4bis Date: Sat, 11 Apr 2026 03:55:01 +1000 Subject: [PATCH 7/7] claude-notes: update remaining-work.md (2026-04-11 03:55) --- help4bis-claude-notes/remaining-work.md | 90 +++++++++++++++---------- 1 file changed, 55 insertions(+), 35 deletions(-) diff --git a/help4bis-claude-notes/remaining-work.md b/help4bis-claude-notes/remaining-work.md index ee443c7..abeeecb 100644 --- a/help4bis-claude-notes/remaining-work.md +++ b/help4bis-claude-notes/remaining-work.md @@ -1,46 +1,66 @@ -## Remaining Work — Priority Order +## Remaining Work — M8Chat App2 (2026-04-11) -### P0 — LiveKit E2EE frame encryption (BLOCKING video calls) -- [ ] Enable AES-GCM frame encryption in LiveKit Flutter SDK to match Element X -- Element X publishes media with `encryption: GCM`, app2 publishes `encryption: NONE` -- Without matching encryption, remote video renders as scrambled garbage -- Check livekit_client docs for `E2EEOptions` / `FrameCryptorFactory` on web -- May need to pass shared key material derived from the MatrixRTC session -- **Start here next session**: read livekit_client 2.4.0 source for E2EE setup on web +### P0 — Must test immediately (next session) +- [ ] **Test video call with E2EE + COOP/COEP** — COOP/COEP headers are now on the server, E2EE is re-enabled in code. Start a call from app2 → Element X mobile. Watch console for: + - `[E2EE] Received encryption key(s) from @user` — means key exchange worked + - `worker error` — means SharedArrayBuffer still not available (COOP/COEP not working) + - Scrambled video — key exchange format mismatch with Element X +- [ ] **Test hangup termination** — end call from app2, verify Element X sees call ended +- [ ] **If E2EE worker still crashes**: the COOP/COEP headers may break other things (e.g. loading cross-origin resources like Jitsi external_api.js, cached images). Test all features after enabling COEP. -### P1 — Incoming call detection from Element X -- [ ] Element X sends MSC3401 `org.matrix.msc3401.call.member` state events (already in hidden events list) -- [ ] App currently listens for old `m.call.invite` events in matrixrtc_repository.dart -- [ ] Need to detect call.member state changes and show incoming call overlay -- The state event format includes `memberships[]` with `foci_active[].type: livekit` +### P1 — Important fixes +- [ ] **Incoming call detection** — MSC3401 state event listener. When Element X starts a call, app2 should show an incoming call UI. Currently nothing happens. +- [ ] **Ghost device cleanup** — Each web login creates a new device. Need either: (a) periodic admin API cleanup, (b) logout deletes the device, or (c) cron job to purge old M8Chat devices. +- [ ] **Rooms provider performance** — Room list rebuilds on every sync tick (15+ times visible in console). Should debounce or only rebuild on actual changes. +- [ ] **Device ID for APK/iOS** — When building native apps, re-enable session persistence and device ID reuse. The secure_storage.dart scaffold exists but is currently neutered for web. -### P2 — Cleanup and polish -- [ ] Remove debug logging from rooms_repository.dart (debugPrint of all rooms on every sync) -- [ ] Fix avatar 404s — switch to authenticated media API (/_matrix/client/v1/media/) -- [ ] Clean up 20 ghost devices for @try user on server (DELETE via admin API) -- [ ] Remove diagnostic Olm logging from index.html (console.log statements) -- [ ] Remove debug prints from auth_repository.dart (encryptionEnabled logging) +### P2 — Quality improvements +- [ ] **Noto emoji font** — Add NotoColorEmoji font asset to render emoji in room names (currently shows squares) +- [ ] **Consolidate key restore dialogs** — key_restore_dialog.dart and security_setup_dialog.dart overlap. The security setup already does key restore. +- [ ] **Voice-only calling** — LiveKit supports audio-only but no UI for it yet +- [ ] **Cross-signing verification UX** — The Bootstrap flow creates cross-signing keys but there's no UI for verifying other users' devices (emoji comparison) -### P3 — Voice calling -- [ ] Test voice-only calls (same LiveKit flow, just no video track) -- [ ] Likely same E2EE issue as video — fix P0 first +### P3 — Future features +- [ ] **Push notifications** (Phase 2) +- [ ] **Room search** (Phase 2 — placeholder in AppBar) +- [ ] **Message reactions** (partial — display works, sending not implemented) +- [ ] **File/image upload** (attachment button exists, handler not wired) +- [ ] **Read receipts** (partial implementation) -### P4 — Cross-signing (Phase 2) -- [ ] Device verification to remove red exclamation in Element -- [ ] Profile screen hooks for verify/cross-sign +### Blockers / dependencies +- **E2EE video interop** depends on: (1) COOP/COEP working, (2) m.rtc.encryption_keys format matching Element X. If format doesn't match, need to capture what Element X actually sends (check to-device events on Synapse during a test call). +- **COEP may break cross-origin loads** — CachedNetworkImage, Jitsi external API, and avatar images from matrix.m8chat.au may fail with `require-corp`. May need `credentialless` instead of `require-corp`, or add `crossorigin` attributes. -### Build and deploy commands +### Commands to resume ```bash cd /srv/wp-dev/pwa-sites/m8chat-app2 -dart analyze --fatal-infos + +# Build flutter build web --release -rsync -avz --delete --exclude='.htaccess' --exclude='.ftpquota' \ - -e "ssh -p 2233 -i ~/.ssh/m8chat_prod" \ - build/web/ m8chat@app.m8chat.au:public_html/app2.m8chat.au/ + +# Deploy +rsync -avz --delete --exclude='.htaccess' \ + -e "ssh -i ~/.ssh/m8chat_prod -p 2233" \ + build/web/ m8chat@app.m8chat.au:~/public_html/app2.m8chat.au/ + +# Analyse +dart analyze lib/ + +# Regenerate Riverpod/Freezed code +dart run build_runner build --delete-conflicting-outputs + +# Check Synapse DB +ssh -i ~/.ssh/m8chat_key -p 2233 m8chat_help4bis@chat.m8chat.au \ + "PGPASSWORD=kijPt4cPWkoaZk8BVm psql -h localhost -U synapse_user -d synapse" + +# Check deployed .htaccess (COOP/COEP headers) +ssh -i ~/.ssh/m8chat_prod -p 2233 m8chat@app.m8chat.au \ + "cat ~/public_html/app2.m8chat.au/.htaccess" ``` -### Next session should start with -1. Read these claude-notes -2. Research LiveKit E2EE on Flutter web — check `livekit_client` package for `E2EEOptions`, `FrameCryptor`, `KeyProvider` -3. Check if Element X derives the E2EE key from the MatrixRTC session or uses a separate mechanism -4. Implement and test — the scrambled video fix is the most impactful single change +### What next session should start with +1. `/catchup` to load these notes +2. Test video call: app2 → Element X. Paste console output. +3. If E2EE works → commit everything to Gitea +4. If E2EE fails → check COEP compatibility issues, may need `credentialless` mode +5. If hangup works → move to P1 (incoming call detection) \ No newline at end of file