The integration of a query profiler in SignalDB is proposed to enhance the database's performance analysis capabilities. This profiler would enable developers to monitor and evaluate the performance of various queries executed within SignalDB. It should provide detailed insights such as execution time, memory usage, and frequency of query calls, enabling developers to identify and optimize performance bottlenecks effectively. The profiler would also support tracking the impact of queries on the overall application performance, helping in fine-tuning the data retrieval and manipulation processes. Additionally, the query profiler should be designed to be minimally invasive, ensuring that its operation does not significantly impact the performance it is meant to measure. This feature is crucial for developers who need to ensure optimal performance and efficiency of their applications, particularly those with complex data handling requirements.

This issue lists Renovate updates and detected dependencies. Read the Dependency Dashboard docs to learn more.

This repository currently has no open or pending branches.

Detected dependencies

• Check this box to trigger a request for Renovate to run again on this repository

ts export const posts = new Collection({ persistence: createFilesystemAdapter('./collections/posts.json'), reactivity: reactivityAdapter, });
I'm using solidjs and I don't know if I can access the file or not.

import { uuidv7 } from 'uuidv7'
import { PersistentCollection, Collection, createFilesystemAdapter, createIndex } from 'signaldb'


const users = new Collection({
  persistence: createFilesystemAdapter('./users.json'),
  indices: [createIndex('id')],
})

users.on('persistence.error', (...argv) => {
  console.log('error:', argv)
})

users.on('added', (...argv) => {
  console.log('added:', argv)
})

users.insert({id: uuidv7(), name: 'qwerty', age: 1})
users.insert({id: uuidv7(), name: 'qwerty', age: 2})
users.insert({id: uuidv7(), name: 'qwerty', age: 3})

how do I save a collection?

The issue came up in #491
Currently it's necessary to wait until the persistence.init event was emitted on the collection before making any changes to data. Otherwise the modification will be discarded and overwritten by the loaded data of the persistence adapter.
In terms of developer experience, it would be better if the collection would queue up any changes until the persistence.init event was emitted and then apply them.

Seems like there's a mistake here on the import. It should be importing from "solid-js", not "solidj-s/dist/solid"

I confirmed that removing the /dist/solid fixes it.

Notice no errors:

https://github.com/adamhaile/S

https://mobx.js.org/README.html

https://github.com/Riim/cellx

I have a weird issue after upgrading from 0.11.0. It works fine in 0.11.0, but 0.12.1 and 0.13.0 fails.

When building my angular project I get this:

✖ Compiling with Angular sources in Ivy partial compilation mode.
node_modules/signaldb/dist/types/EventEmitter.d.ts:2:19 - error TS2430: Interface 'EventEmitter<T>' incorrectly extends interface 'EventEmitter'.
  Types of property 'emit' are incompatible.
    Type '<U extends keyof T>(event: U, ...args: Parameters<T[U]>) => boolean' is not assignable to type '(type: string | number, ...args: any[]) => boolean'.
      Types of parameters 'event' and 'type' are incompatible.
        Type 'string | number' is not assignable to type 'keyof T'.
          Type 'number' is not assignable to type 'keyof T'.
            Type 'number' is not assignable to type 'string | symbol'.

2 declare interface EventEmitter<T extends Record<string | symbol, any>> {
                    ~~~~~~~~~~~~
node_modules/signaldb/dist/types/EventEmitter.d.ts:6:15 - error TS2415: Class 'EventEmitter<T>' incorrectly extends base class 'EventEmitter'.

6 declare class EventEmitter<T> extends BaseEventEmitter {
                ~~~~~~~~~~~~

I have been trying to figure out why it occurs, because to me it seems there have been no changes to that file (signaldb/dist/types/EventEmitter.d.ts) since 0.11.0.

The implementation of auto fetching in SignalDB is proposed to enhance its data handling capabilities. This feature involves registering queries when they are executed and unregistering them once they leave the reactive context. When a query is registered, a callback function will be invoked, allowing data to be loaded from an external system into SignalDB's in-memory database. After the data loading is complete, the reactive context will re-execute, and the query will return the newly fetched data. Additionally, a reactive function on the collection, such as isPreloading(), will be implemented to indicate the loading state. This feature aims to optimize data retrieval processes, particularly in scenarios requiring dynamic data loading from external sources, while maintaining the reactivity and performance efficiency of web applications using SignalDB.

Minimal repro here.

The development of a data replication protocol for SignalDB is aimed at achieving efficient, reliable, and speedy data synchronization over http and websockts. Key features of this protocol should include a streamlined replication process, supporting concurrent push and pull operations, and a changestream mechanism for continuous and incremental data updates. Additionally, client-side conflict resolution is essential for decentralizing the process, thereby simplifying backend requirements and enhancing the database's autonomy. The protocol must be robust and fault-tolerant, capable of handling intermittent connectivity and large datasets with high-frequency updates. It should support offline operations with seamless re-syncing once connectivity is restored. Furthermore, the architecture should be extensible, allowing for future enhancements such as accommodating complex data types or integrating with various backend systems. Overall, this protocol will focus on ensuring that SignalDB offers a highly efficient, reliable, and user-friendly replication mechanism for modern client-side applications.

Inspiration:

• https://gist.github.com/pesterhazy/3e039677f2e314cb77ffe3497ebca07b
• https://watermelondb.dev/docs/Implementation/SyncImpl
• pubkey/rxdb#3883

Discussed in #765

async register(onChange) {
      websock.onMessage({items, changes} => {
        void onChange({items, changes})
      }
    },

The implementation of an Appwrite persistence adapter is crucial for integrating SignalDB with Appwrite's backend services. This adapter will enable SignalDB to interface directly with Appwrite's server-side database, offering developers the flexibility to use SignalDB's client-side features while leveraging Appwrite's robust data storage, user authentication, and management capabilities. The goal is to create a seamless connection that allows for efficient data synchronization and manipulation between SignalDB and Appwrite. This integration is particularly valuable for applications that require secure, scalable, and easily manageable data storage solutions, making the most of both SignalDB's client-side efficiency and Appwrite's server-side robustness.

When using ReplicatedCollection with LocalStorageAdapter for persistence I am experiencing what I what describe as a timing issue. For the most part it work, but now and then the collection just empties itself.

I have tried to add a lot of logging to get to the bottom of it. Here is how it looks when it works fine.

You'll see that I am saving a new entity. Before I put it in the Collection I log the number of entities already in the collection. There are also log statements for when persistence.pullCompleted occurs, together with the current number of entities in the collection.

This is an example for when it breaks.

Notice that persistence.pullCompleted happens twice, even if my pull method is only called once. Also in the last example the number of entities is 0, on the first persistence.pullCompleted event - before my pull method has returned a LoadResponse.

The development of a Supabase persistence adapter for SignalDB is proposed to extend SignalDB's compatibility with Supabase as a backend service. This adapter will enable developers to leverage Supabase's real-time database capabilities, including its robust authentication, authorization, and real-time synchronization features, in conjunction with SignalDB's client-side functionalities. The integration aims to provide a seamless bridge between SignalDB's client-side operations and Supabase's cloud-hosted database, facilitating a more efficient and scalable data handling experience. This enhancement will particularly benefit applications requiring real-time data updates and those utilizing Supabase for their backend infrastructure

I'm dealing with a situation where I insert a skeleton row into a ReplicatedCollection. A skeleton row is a bare minimum that the server needs like an id and a name. The collection calls its push hook and the skeleton row gets sent to the server. The server then notifies all clients that a new row was created. This new row includes other fields added by the server like createdAt, updatedAt, and createdBy. The collection's onChange gets called with { added: [newCompleteRow], modified: [], removed: [] }. My issue happens because the complete row added by onChange does not replace the initial skeleton row, and I end up with 2 rows with the same id.

There are a few possible solutions to my problem.

First, I could figure out if the current client is the one that initially inserted the skeleton row and swap the row (with modified) instead of simply adding it. I would rather not meddle with the backend.

The second option is that when onChange gets called with added rows, they should be upserted into the collection by the row's id. If a row is truly new it gets inserted otherwise it swaps the skeleton row.

A third option is to let the persistence adapter/replicated collection's push hook have access to the collection object so I can manually findOne by id and swap the row if it already exists.

Any idea how to go about this?

Before v1.0.0 can be released it would be good to restructure the documentation, to make it more clear and structured, so that users can easily find what they're searching and make the initial setup faster.

I will collected my ideas in this issue. Feel free to recommend additional approaches.

ToDo:

• Add an API reference with all exported classes/functions
• Add guides for other frameworks (svelte, solid, vue, angular, etc.)
• The getting started page should link to the guides
• Add usage of replication possibilities to the getting started page

https://rxjs.dev/

I am using signal db with react, I have one collection defined as follows -

import { PersistentCollection } from 'signaldb'

export const collectionChats = new PersistentCollection('collection-chats')

I am consuming this collection in a nested route, in my application and I have a delete chats button on the main route, when I delete the chats in the main route, my nested route component is not re-rendering here is the code for my main route -

return (
    <div className="w-full p-9 pb-0 flex flex-col gap-4 h-full">
      <div className="flex justify-between">
        <Tabs defaultValue="account" className="w-[400px]">
          <TabsList className="grid w-full grid-cols-2">
            <NavLink to="videos">
              Videos
            </NavLink>
            <NavLink to="videos/chat">
              Chat
            </NavLink>
          </TabsList>
        </Tabs>
        <Button
          onClick={() => {
            collectionChats.updateOne(
              { id: params.collectionId },
              {
                $set: { chats: [] },
              }
            );
          }}
          variant="destructive"
        >
          <Trash2Icon className="h-[1.2rem] mr-2 w-[1.2rem] dark:-bg-white bg-dark" /> Delete Chat
        </Button>
      </div>

      <div className="w-full h-full pb-4 overflow-hidden">
        <Outlet />
      </div>
    </div>
  );

I am consuming the collectionChats inside the chat route which is a nested route -

const chatMessages = collectionChats.findOne({ id: params.collectionId });

I am expecting that when I click on the delete button and set the chats array to empty for a particular collectionId, my chat route should re-render and all the chats should disappear.

if you insert new document with custom id, then delete this document, then try to add new document with this id again you will get error:

Item with same id already exists

const document = { id: 'test', text: 'Hello' }
DB.insert(document);
DB.removeOne({ id: document.id })
DB.insert(document); // <- error

Looks like when removing item the idIndex isn't cleared for this id, but with custom identifiers i think this should happen cause in my scenario i use some id from other source to use it as id in signaldb and cant remove/add this id twice, actually after page refresh idIndex doesn't contain any index and this is more like bug.

signaldb 0.8.6

Fig 1

Fig 2

Hi Maxnowack , I can across signaldb yesterday when researching on client side database and I must say you have done a great work here.
Why going through the documentation, I discovered that the link to Fig1 from Fig 2 is broken. I wish to contribute to this project and I will appreciate it if I can start from fixing the broken link.

The creation of a Firebase persistence adapter for SignalDB is essential to integrate SignalDB with Firebase's cloud-based database and services. This adapter will enable developers to connect SignalDB's client-side database functionalities with Firebase's scalable backend, including its real-time database, authentication, and storage services. The integration is designed to facilitate seamless data synchronization and storage between the client and Firebase, offering a robust solution for applications that demand real-time data updates and cloud-based data persistence. This Firebase adapter aims to simplify backend operations for developers using SignalDB, making it easier to build dynamic, responsive web applications with efficient data handling and synchronization capabilities.

var cs = null, us = cs;
const W = /* @__PURE__ */ Zt(us);
function as(t) {
  let e = null;
  return {
    async register(n) {
      if (typeof window < "u")
        throw new Error("Filesystem adapter is not supported in the browser");
      await W.promises.access(t).then(() => !0).catch(() => !1) || await W.promises.writeFile(t, "[]"), W.watch(t, { encoding: "utf8" }, () => {
        n();
      });
    },

W.promises.access
Explain how this can bring back the promise

https://github.com/xania/state

https://vuejs.org/guide/extras/reactivity-in-depth.html

Import to esm does not work
Rename it index.es.js to index.es.mjs and "import": "./dist/index.es.mjs" in package.json

I was testing signaldb in my solid-js project, but I noticed that adding an element to the collection triggered the effect.

import { Collection } from 'signaldb';
import solidReactivityAdapter from 'signaldb-plugin-solid'
import { createEffect } from 'solid-js';


const posts = new Collection({
  reactivity: solidReactivityAdapter,
});

createEffect(() => {
  console.log(posts.find({ author: 'John' }).count());
});

// ---------------------- INSERTS ---------------------------

(async () => {
  await posts.insert({ id: crypto.randomUUID(), author: 'John' });
  await new Promise((r) => setTimeout(r, 1000));
  await posts.insert({ id: crypto.randomUUID(), author: 'Kevin' });
  await new Promise((r) => setTimeout(r, 1000));
  await posts.insert({ id: crypto.randomUUID(), author: 'David' });
  console.log('Finish inserts');
})();

Testing the same example but with vue and @maverick-js/signals works as expected.

Then check in the documentation to make a custom adapter and add some console.log. It turns out that it never calls the “create” function.

import { Collection } from 'signaldb';
//import solidReactivityAdapter from 'signaldb-plugin-solid'
import { createEffect } from 'solid-js';

const solidReactivityAdapter = createReactivityAdapter({
  create: () => {
    console.log('Create');
    const [depend, rerun] = createSignal(0);
    return {
      depend: () => {
        console.log('Depend');
        depend();
      },
      notify: () => {
        console.log('Notify');
        rerun(depend() + 1);
      },
    };
  },
  isInScope: undefined,
  onDispose: (callback) => {
    onCleanup(callback);
  },
});

const posts = new Collection({
  reactivity: solidReactivityAdapter,
});

createEffect(() => {
  console.log(posts.find({ author: 'John' }).count());
});

// ---------------------- INSERTS ---------------------------

(async () => {
  await posts.insert({ id: crypto.randomUUID(), author: 'John' });
  await new Promise((r) => setTimeout(r, 1000));
  await posts.insert({ id: crypto.randomUUID(), author: 'Kevin' });
  await new Promise((r) => setTimeout(r, 1000));
  await posts.insert({ id: crypto.randomUUID(), author: 'David' });
  console.log('Finish inserts');
})();

I don't know what could be happening, I also leave a link to stackblitz you can try to replicate my experience.

If objects in collections contains date objects, then they change to strings when read from a persistence adapter.

This is true for Opfs, Localstorage and Filesystem. All of them uses plain JSON.stringify and JSON.parse, which will convert Dates to strings.

A simple solution could be to add a reviver function to JSON.parse:

function dateReviver(key, value) {
   // handle dates
}

// Parse the JSON string with the reviver function
const parsedObj = JSON.parse(jsonString, dateReviver);

Better maybe even encapsulate it in a new class, that all persistence adapters can use.

Suggested solution

I suggest that serializing dates to json, should be resolve in an object like this:

{
  "createdAt": {
    "$type": "Date",
    "data": "2024-08-01T12:00:00Z"
  }
}

Doing that requires a replacer method for JSON.stringify, and a reviver for JSON.parse. Here follows a pseudo example.

stringify

function dateReplacer(key, value) {
  if (this[key] instanceof Date) {
    return { $type: 'Date', data: this[key].toISOString() };
  }
  return value;
}

const obj = {
  createdAt: new Date('1990-01-01')
};

const jsonString = JSON.stringify(obj, dateReplacer);

parse

function dateReviver(key, value) {
  if (value && typeof value === 'object' && value.$type === 'Date') {
    return new Date(value.data);
  }
  return value;
}

const parsedObj = JSON.parse(jsonString, dateReviver);

https://github.com/modderme123/reactively

The development of Async Collections in SignalDB is aimed at enhancing the database's flexibility and performance, particularly in scenarios where data persistence without in-memory storage is essential. These Async Collections would enable SignalDB to interact with asynchronous storage mechanisms like IndexedDB more efficiently, allowing for data operations without necessarily storing data in the client's memory. This feature is especially relevant for applications that handle large datasets or require extended data persistence beyond the session scope. Furthermore, Async Collections could facilitate the outsourcing of query execution to Web Workers, thereby offloading intensive data processing tasks from the main thread. This approach would significantly improve the application's responsiveness and overall performance, especially in web applications with complex data processing requirements. The introduction of Async Collections in SignalDB aims to provide developers with more options for data management, catering to a broader range of application architectures and performance needs.