ben-ryder / athena Goto Github PK

Right now after an update is pushed the old PWA site can still be cached. There is no mechanism to inform the user of updates.

At minimum I should try and find a way to automatically invalidate the cache and update after a production deploy, but long term I may want to make this user facing so they can choose to update when ready.

Right now I just do doc.text = newText when doing content updates, but this isn't good for supporting offline/concurrent edits as it just replaces all the text.

I'm thinking that I could use https://www.npmjs.com/package/diff to compare the old and new string values when an edit is made and then use A.splice to apply each change. I think this would involve migrating to the Next API too:

• https://automerge.org/docs/the_js_packages/#using-the-next-api
• https://automerge.org/docs/documents/#javascript-language-mapping

Doing it this way would allow me to continue using the basic codemirror setup I've got with value and onChange using a basic string type. I could also then contain this functionality within the state actions, and the frontend can be simple as it can just pass the full updated text. This diff processing could be done once on save, and/or on some sort of debounce'd autosave which can automatically save changes after x seconds of inactivity.
I think this will work ok for me here because my use case is not real-time collaborative sync, but is single user cross-device sync with offline support.

When a modal is closed the focus returns to the top of the page.
It should ideally return to the same point that it was at when the modal was opened.

Investigate ways of offering speech to text:

• It would have to be fully locally, not reliant on an API. I could maybe use whiper.cpp wasm?
• It would have to work accurately enough to be useful.
• The added file size of the app would have to not be crazy.

The order of items (when ordered by name) is "test 1", "test 10" , "test 2", it should ideally be "test 1", "test 2" , "test 10".

This is pretty low priority as you can fix this by just doing: "test 01", "test 02" , "test 10", so I may not ever bother actually fixing this.

There are places where I'm doing obj.list = obj.list.filter() to update and remove items from lists but I should make these updates more granular by using indexOf, deleteAt and push as this makes the changes easier to sync and less likely to conflict: https://automerge.org/docs/documents/lists/.

Problem

Right now this application is "server first", meaning that the server holds the single source of truth and data is updated via HTTP requests.
This works fine, but will not be ideal for offline support going forwards. If I wanted to support offline usage I would probably have to implement some sort of rudimentary sync system which would store a log of changes locally, then calculate the difference when online and push these changes. This would have a number of problems including conflict issues if multiple clients make offline changes then attempt to push these changes later.

There's also the general question of data ownership too, especially for a notes app where offline use and owning your
data is of paticualr importance.

Solution

I should look at changing this application around, so it works primarily as as a local first, offline first web app (using service worker etc?) which saves and reads data to a local store (such as IndexDB, in-browser sqlite etc?).
Data could then be synced via a CRDT system such as automerge with changes being communicated via a technology like websockets rather than a REST API.
I could then still use a server as always online data sync client/backup and a central way to communicate between different clients.

Links to more info:

• Automerge - Likely the JS library I'd use for managing data sync
• CRDTs - General info about CRDTs (Conflict-Free Distrubuted Data Types)
• Local First - An article about local first application architecture
• Actual is an open source budgeting app, built in a local-first way using CRDT (GitHub)
• Logux a websocket framework with CRDT like behaviour, intergrates with Redux

Implementation Ideas

• Use same React app
• Redux provides application state management
• Redux application state maps to an automerge document, changes to redux state can be mapped to automerge changes and sent via websocket to server.
• Server can still use REST to initially authenticate users - and maybe first time data sync/dumps? - but websockets can then handle the CRDT based change sync.
• Potentially all clients should sync via the server. So a client makes a change, then sends this change to the server. The server then merges that change with it's own copy of the data, then broadcasts that change out as it's own change to all clients connected on the websocket.
• In the React app, local first support could be implemented with either some sort of Redux persistence library or more likely through a mixture of browser storage (IndexDB for content, localStorage for user details & keys still)
• To potentially improve performace, I think character level changes in note content (body, title, description etc) should not always count as syncable changes, but be batched together. So if a user makes changes to a notes body, maybe a change would be registered every x minutes or if they close/navigate away from the content. If the user changes the title, that is a single change etc etc.
• Each user vault is a single automerge document, content (such as notes, tags etc) are split using automerge tables.
• Redux stores one document/application state at a time but local persistent storage can store multiple vaults & maybe even multiple users?
• The sync system will then only act against for the currently active user & vault, even if there are locally persisted changes previously made in other vaults or by other users
• Logux looks like a good option, but I fear relying on it might cause library lock in and it would mean that I'm not learning and practising an in depth use of websockets, CRDTs etc

Description

Designs

Using automerge-repo is the recommended usage for Automerge and I should investigate if it fits my use case here.

It is similar to what I've begun to implement in https://github.com/ben-ryder/local-first-backend which supports syncing changes via websocket, broadcast channel and REST API and persisting changes in IndexDB in the browser and Postgres on the server.
My main use case for rolling my own sync and storage system was the desire for zero knowledge encryption, where an encryption key derived from a user password is used to encrypt content on the users local device before it is sent to the server.
At the time I created this system, my research led me to conclude that this was the only option I had to support zero knowledge encryption becuase the Automerge sync protocol relied on the server acting as a peer and being able to process the Automerge docuements and changes. I need to...

1. Check if I still need to roll my own storage/sync system
2. See if I can reduce the amount of custom code required by implementing my own Automerge network and storage adapters but still using the automerge-repo system.

When typing on mobile, somtimes each character will cause previous typed characters to appear and backspace will not work.
This seems to only happen when ading a new note, and editing a notes seems to be effected less.

Allow multiple "vaults" which are seperate collections of notes, tasks etc.

When using popups or opening modals, every now and again these actions will completely crash the browser and/or current tab.
There are no JS errors or anything - as far as I can see - everything just crashes.
After a few refreshes or random code changes the issues then go away.

This effects both the dev server and production builds of the app, and looks to be an issue across devices