Woudl be nice if we could do that using puppeteer

interface Node {
  id: ID!
}

type Note implements Node {
  id: ID! # graphql conform id
  documentId: ID! # human readable id
  title: String!
  content: String!
  access: NoteAccess
}

enum NoteAccess {
  public
  private
}

type Query {
  node(id: ID!): Node
  note(documentId: ID!): Note
}

query noteDetail($documentId: ID!) @live {
  note(documentId: $documentId) {
    id
    title
    content
  }
}

Variables

{
 "documentId": "my-first-note"
}

Query.note(documentId:"my-first-note")

liveQueryStore.invalidate(`Query.note(documentId:"my-first-note")`)

Let's say the access was updated from private to public. The Query.note field returns null if the viewer is not allowed to see it. But later the access is changed, so the note should be pushed to the client. There is currently no efficient way for invalidating only a field with a specific argument. The solution rn would be to invalidate Query.note which will cause any query (even those that are unaffected) to be scheduled for re-execution. With the argument narrowing more efficient invalidations would be possible.

graphql/graphql-spec#746

As @maraisr pointed out it might be possible to support defer/stream mixed with the live directive.

Hi, I'm currently having a very strange problem. I'm having differing execution results for the same server implementation of the live query with different clients. Specifically, a difference between a production and development version of my client code. On the development version, everything works just fine 100% of the time. However, on my production code, there is a pattern of live query results not being sent back to the client. After throwing in a lot of debugging statements, the only difference between the execution of the two requests that I can find, is that on my production version of the client, this function is being run and it's releasing the identifiers used to alert of a change, even though it should still be watching those identifiers.

I'm really stumped on this one, since both clients should be sending identical requests, and I could not find anything in this codebase which would change upon setting the environment to production rather than development.

Stale nodes that are no longer displayed a first removed after the query component (that ONCE referenced them) unmounts. Ideally, we want those removed after they are no longer referenced in the query.

This is an performance issue for long-running applications.

From taking a quick look it seems like the library must provide an api for hooking into the flow
https://github.com/enisdenjo/graphql-transport-ws/blob/65e073d9136ed79804461147137afd19ee995c81/src/server.ts#L538

The approach of calling liveQueryStore.triggerUpdate("Query.users") might not scale that well. It could return different results for different users. It does not address the unique field argument combinations.

Resource-Based Updates

If a user with id fgahex changes we ideally only want to update queries that select the given user.

Object types that have an id field could be automatically collected during query execution (and updated during re-execution).

Query

query user @live {
  me {
    __typename
    id
    login
  }
}

Execution result:

{
  "data": {
    "__typename": "User",
    "id": "fgahex",
    "login": "testuser"
  }
}

Will result in the following entry for the selected resources of that query: User.fgahex.
An update for all the live queries that select the given user could be scheduled via liveQueryStore.triggerUpdate("User.fgahex").

Things to figure out:

What is the best way of collecting resources?

This would require sth. like a middleware for each resolver that gathers the ids and attaches them to the result (as extensions?) that the liveQueryStore can pick up (maybe this?).

Another solution would be to push this to the user so he has to register it manually, by calling some function that is passed along the context object?

The information could also be extracted from the query response. However, that would have the implications that it needs at least the id extracted for each record (or even id + __typename for GraphQL servers that do not expose unique ids across object types). Since most client-side frameworks encourage using unique ids for identifying Objects anyways that shouldn't be that big of a limitation. An additional drawback would be that the whole result tree must be traversed.

Pros:

• The server automatically keeps track of the resources a query selects.
• Does work for resources that got replaced with objects of another id (by calling triggerUpdate for the previous resource).

Cons:

• Big queries result in a lot of strings that are stored additionally in memory
• Does only work for object types that have an id property.
• Does probably not play that well with a list type.
• Does not solve the field argument issue.

I thought about allowing to provide a buildResourceIdentifier function to the extensions field of a field resolver. That would easily allow adding custom resource identifiers/ events in user-land for invalidating a query.

const GraphQLQueryType = new GraphQLObjectType<Root>({
  name: "Query",
  fields: {
    todos: {
      type: new GraphQLNonNull(
        new GraphQLList(new GraphQLNonNull(GraphQLTodoType))
      ),
      extensions: {
        liveQuery: {
          // return string or string array?
          buildResourceIdentifier: (_root, _args, context) => `UserTodos:${context.viewer.id}`
        }
      },
      resolve: (root, args, context) => Array.from(root.todos.values()),
    },
  },
});

e.g. https://github.com/n1ru4l/graphql-live-queries/blob/eb6297ae677c5ee186d15d130a014a3340d9e703/packages/in-memory-live-query-store/src/InMemoryLiveQueryStore.ts#L40

We should get some inspiration from other server implementations :)

aka

export function execute(
  args: ExecutionArgs,
): PromiseOrValue<
  ExecutionResult | AsyncIterableIterator<AsyncExecutionResult>
>;
export function execute(
  schema: GraphQLSchema,
  document: DocumentNode,
  rootValue?: any,
  contextValue?: any,
  variableValues?: Maybe<{ [key: string]: any }>,
  operationName?: Maybe<string>,
  fieldResolver?: Maybe<GraphQLFieldResolver<any, any>>,
  typeResolver?: Maybe<GraphQLTypeResolver<any, any>>,
): PromiseOrValue<
  ExecutionResult | AsyncIterableIterator<AsyncExecutionResult>
>;

When publishing a new value through the live query observable list updates cause the whole list to re-render. Also, the old records seem not to be evicted properly from the relay store.

Hey @n1ru4l thanks for all your hard work on these libraries!

I've come across an issue where I'm seeing a load of empty patches, e.g.: {"revision":92}, which the client isn't too happy about, https://github.com/n1ru4l/graphql-live-query/blob/main/packages/graphql-live-query-patch/src/createApplyLiveQueryPatch.ts#L42-L44.

The generatePatch function returns undefined in the case where previousValue is the same as currentValue. I believe there should be a check in place to not yield valueToPublish in those cases.

Can't believe it has been nearly 5 years since this video https://www.youtube.com/watch?t=1090&v=ViXL0YQnioU where Lee Byron presents the idea of the @live directive.

{
  feed {
    stories {
      author { name }
      message
      likeCount @live
    }
  }
}

Really cool to follow your progress on live queries, but I've been wondering why you went with the approach of annotating the entire query vs being able to annotate the individual fields or fragments?

Are you planning, at some point, to support @live per field or fragment, or what is the rationale of annotating the whole query?

Keep up the great work!

Seeing some posts about subscriptions on Reddit about game development, I wanna explore options for building multiplayer games with live queries 😉

I had this idea that is actually possible to allow returning AsyncIterators from resolvers without waiting for it to land in graphql-js. The only downside would be that typings in user code could be wrong (Promise<T> | T expected instead of AsyncIterator<T>) 😅. However, that could be easily solved with patch-package.

The idea is to wrap each field resolver (as in #94). The wrapper will then check whether the returned value is a plain value, a Promise, or an AsyncIterator. In the case of plain value and Promise it will just forward it. In case of an AsyncIterator it will (1) get the first value and resolve a Promise with it and (2) register that the live query returns an AsyncIterator for a given query path and then forwards patches to the client once new data is returned from the AsyncIterator.

the simplest example is a TodoMVC app that that uses live queries and relay.

CSS: https://github.com/tastejs/todomvc-app-css
The Backend should use plain graphql-js for keeping things as simple as possible.

Tracking updates for potentially huge paginated GraphQL connections is hard and can not be done with the implementation inside this repository.

The current way of doing this is a subscription (with super complicated pub-sub handlers), with manual cache update handlers on the frontend.

For a project, I came up with the following subscription solution for a connection of notes.

Ideally, we would want to avoid a lot of this boilerplate and make connection updates live without much hassle. I thought of having a specific directive especially for connection such as liveConnection, but did not work out the details yet. The idea is that it behaves a bit differently than the live directive and in case more items are fetched the servercan then based on that directive check which items on the client would be affected.

type Query {
  notes(first: Int, after: String, filter: NotesFilter): NoteConnection!
}

type NoteConnection {
  edges: [NoteEdge!]!
  pageInfo: PageInfo!
}

type NoteEdge {
  cursor: String!
  node: Note!
}

type Note implements Node {
  id: ID!
  documentId: ID!
  title: String!
  content: String!
  contentPreview: String!
  createdAt: Int!
  viewerCanEdit: Boolean!
  viewerCanShare: Boolean!
  access: String!
  isEntryPoint: Boolean!
  updatedAt: Int!
}

type NotesUpdates {
  """
  A node that was added to the connection.
  """
  addedNode: NotesConnectionEdgeInsertionUpdate
  """
  A note that was updated.
  """
  updatedNote: Note
  """
  A note that was removed.
  """
  removedNoteId: ID
}
type NotesConnectionEdgeInsertionUpdate {
  """
  The cursor of the item before which the node should be inserted.
  """
  previousCursor: String
  """
  The edge that should be inserted.
  """
  edge: NoteEdge
}

type Subscription {
  notesUpdates(
    filter: NotesFilter
    endCursor: String!
    hasNextPage: Boolean!
  ): NotesUpdates!
}

The implementation on the frontend then could look similar to this (Full code can be found here):

const subscription =
  requestSubscription <
  tokenInfoSideBar_NotesUpdatesSubscription >
  (environment,
  {
    subscription: TokenInfoSideBar_NotesUpdatesSubscription,
    variables: {
      filter: props.showAll ? "All" : "Entrypoint",
      endCursor: data.notes.pageInfo.endCursor,
      hasNextPage: data.notes.pageInfo.hasNextPage,
    },
    updater: (store, payload) => {
      console.log(JSON.stringify(payload, null, 2));
      if (payload.notesUpdates.removedNoteId) {
        const connection = store.get(data.notes.__id);
        if (connection) {
          ConnectionHandler.deleteNode(
            connection,
            payload.notesUpdates.removedNoteId
          );
        }
      }
      if (payload.notesUpdates.addedNode) {
        const connection = store.get(data.notes.__id);
        if (connection) {
          const edge = store
            .getRootField("notesUpdates")
            ?.getLinkedRecord("addedNode")
            ?.getLinkedRecord("edge");
          // we need to copy the fields at the other Subscription.notesUpdates.addedNode.edge field
          // will be mutated when the next subscription result is arriving
          const record = store.create(
            // prettier-ignore
            `${data.notes.__id}-${edge.getValue("cursor")}-${++newEdgeIdCounter.current}`,
            "NoteEdge"
          );

          record.copyFieldsFrom(edge);

          if (payload.notesUpdates.addedNode.previousCursor) {
            ConnectionHandler.insertEdgeBefore(
              connection,
              record,
              payload.notesUpdates.addedNode.previousCursor
            );
          } else if (
            // in case we don't have a previous cursor and there is no nextPage the edge must be added the last list item.
            connection?.getLinkedRecord("pageInfo")?.getValue("hasNextPage") ===
            false
          ) {
            ConnectionHandler.insertEdgeAfter(connection, record);
          }
        }
      }
    },
  });

const TokenInfoSideBar_NotesUpdatesSubscription = graphql`
  subscription tokenInfoSideBar_NotesUpdatesSubscription(
    $filter: NotesFilter!
    $endCursor: String!
    $hasNextPage: Boolean!
  ) {
    notesUpdates(
      filter: $filter
      endCursor: $endCursor
      hasNextPage: $hasNextPage
    ) {
      removedNoteId
      updatedNote {
        id
        title
        isEntryPoint
      }
      addedNode {
        previousCursor
        edge {
          cursor
          node {
            id
            documentId
            title
          }
        }
      }
    }
  }
`;

This issue lists Renovate updates and detected dependencies. Read the Dependency Dashboard docs to learn more.

Rate-Limited

These updates are currently rate-limited. Click on a checkbox below to force their creation now.

• fix(deps): update dependency @graphql-tools/utils to v8.13.1
• chore(deps): update dependency todomvc-app-css to v2.4.3
• chore(deps): update babel monorepo (@babel/core, @babel/preset-env, @babel/preset-typescript)
• chore(deps): update dependency @testing-library/jest-dom to v5.17.0
• chore(deps): update dependency @types/node to v18.19.31
• chore(deps): update dependency classnames to v2.5.1 (classnames, @types/classnames)
• chore(deps): update dependency prettier to v2.8.8
• chore(deps): update dependency @types/node to v20
• chore(deps): update dependency globby to v14
• chore(deps): update dependency husky to v9
• chore(deps): update dependency lint-staged to v15
• chore(deps): update dependency patch-package to v8
• chore(deps): update dependency prettier to v3
• chore(deps): update dependency vite to v5
• chore(deps): update dependency ts-node to v10.9.2
• fix(deps): update dependency @repeaterjs/repeater to v3.0.5
• fix(deps): update dependency fast-json-patch to v3.1.1
• fix(deps): update dependency graphql-ws to v5.16.0
• fix(deps): update dependency ioredis to v5.4.1
• fix(deps): update dependency puppeteer to v18.2.1
• fix(deps): update dependency ws to v8.16.0 (ws, @types/ws)
• fix(deps): update socket.io packages to v4.7.5 (socket.io, socket.io-client)
• fix(deps): update dependency puppeteer to v22
• 🔐 Create all rate-limited PRs at once 🔐

Open

These updates have all been created already. Click a checkbox below to force a retry/rebase of any.

• fix(deps): update dependency @graphql-tools/utils to v10
• fix(deps): update dependency @graphql-yoga/node to v3
• chore(deps): update dependency @repeaterjs/repeater to v3.0.5
• chore(deps): update dependency husky to v8.0.3
• chore(deps): update dependency vite to v3.2.10
• chore(deps): update dependency @apollo/client to v3.10.1
• chore(deps): update dependency @changesets/changelog-github to v0.5.0
• chore(deps): update dependency @changesets/cli to v2.27.1
• chore(deps): update dependency @types/react-dom to v18.3.0
• chore(deps): update dependency semver to v7.6.0
• chore(deps): update dependency typescript to v4.9.5
• chore(deps): update dependency bob-the-bundler to v7
• chore(deps): update dependency chalk to v5
• chore(deps): update dependency tsc-watch to v6
• chore(deps): update dependency typescript to v5
• chore(deps): update dependency urql to v4
• chore(deps): update graphql-codegen (major) (@graphql-codegen/cli, @graphql-codegen/gql-tag-operations-preset)
• chore(deps): update jest packages (major) (@testing-library/jest-dom, @types/jest, babel-jest, jest, ts-jest)
• chore(deps): update react monorepo to v18 (major) (react, react-dom)
• chore(deps): update relay packages (major) (@types/react-relay, @types/relay-runtime, babel-plugin-relay, react-relay, relay-compiler, relay-runtime)
• chore(deps): update testing library (major) (@testing-library/react, @testing-library/user-event)
• fix(deps): replace dependency fastify-static with @fastify/static 5.0.0
• fix(deps): update dependency fastify to v3.29.5
• fix(deps): update dependency express to v4.19.2 (express, @types/express)
• fix(deps): update dependency fastify to v4
• Click on this checkbox to rebase all open PRs at once

Detected dependencies

• Check this box to trigger a request for Renovate to run again on this repository

The readme says:

For scaling horizontally the independent InMemoryLiveQueryStore instances could be wired together via a PubSub system such as Redis.

Could you give an example for how to do this?

graphql-ws does the same, so why need to install two packages for this 🤔

I had this first draft in mind. Wanna incorporate the GraphQL logo as well

It would be nice if you could define whether a query should be live or not via a Boolean query argument. The main benefit I see is that the operation could then be used for SSR and streaming the results afterward.

unless could also be a valid option.

query users($isClient: Boolean = false) @live(if: $isClient) {
  users(first: 10) {
    id
    login
  }
}

const Foo = () => {
  const isSSR = useIsSSR();
  const [data] = useQuery(UsersQuery, { isClient: !isSSR })
  ...
}

In order to showcase and proof that @n1ru4l/socket-io-graphql-client works with apollo-client, adding the example app using apollo would be awesome :)

Some systems only want to allow access to the GraphQL layer in case some authentication has happened.

A library such as https://www.npmjs.com/package/fast-json-patch could be used to only send patches over the network for altering the existing result on the client.

https://github.com/n1ru4l/graphql-live-queries/blob/master/packages/example/server/src/registerGraphQLLayer.ts

should be its own package similar to the network interface for the client (so it can be used in other projects without copy paste):

https://github.com/n1ru4l/graphql-live-queries/blob/master/packages/socket-io-graphql-network-interface/src/createSocketIOGraphQLNetworkInterface.ts

The invalidate method is an implementation detail of the InMemoryLiveQueryStore

am using graphql-tools to build my schema with makeExcutableSchema but when GraphQLLiveDirective is added to the schema using the schema directives I get this error.

this is a portion of my code

schema.ts

import { GraphQLSchema, specifiedDirectives } from 'graphql';
import { makeExecutableSchema, IResolvers } from 'apollo-server-express';
import { GraphQLLiveDirective } from '@n1ru4l/graphql-live-query';
import { typeDefs as scalarTypeDefs, resolvers as scalarResolvers } from 'graphql-scalars';
import { logger, SchemaData } from 'utils';
import * as data from './build';

const {
  typeDefs, Mutation, Query, Subscription, Directives, Resolvers,
  // @ts-ignore
} = Object.values(data as SchemaData)
  .reduce<SchemaData>((result: SchemaData, x: SchemaData): SchemaData => ({
    Mutation: { ...result.Mutation, ...x.Mutation },
    Query: { ...result.Query, ...x.Query },
    Subscription: { ...result.Subscription, ...x.Subscription },
    // @ts-ignore
    typeDefs: [...result.typeDefs, x.typeDefs],
    Directives: { ...result.Directives, ...x.Directives },
    Resolvers: { ...result.Resolvers, ...x.Resolvers },
  }),
{
  typeDefs: [...scalarTypeDefs],
  Mutation: {},
  Query: {},
  Subscription: {},
  Directives: {},
  Resolvers: { ...scalarResolvers },
} as SchemaData);

const schemaDirectives = {
  ...Directives,
};

const resolvers: IResolvers = {
  Mutation, Query, Subscription, ...Resolvers,
};

const schema: GraphQLSchema = makeExecutableSchema({
  logger: {
    log: e => {
      if (typeof e === 'string') {
        logger.info(e);
      }
      logger.error(e);
    },
  },
  allowUndefinedInResolve: false,
  resolverValidationOptions: {
    requireResolversForNonScalar: true,
    requireResolversForArgs: true,
    requireResolversForResolveType: true,
    // requireResolversForAllFields: false,
  },
  inheritResolversFromInterfaces: true,
  resolvers,
  typeDefs,
  schemaDirectives: [GraphQLLiveDirective]
});

export { schema };

and index.ts

import 'module-alias/register';
import { registerSocketIOGraphQLServer,  } from '@n1ru4l/socket-io-graphql-server';
import { Server as IOServer, Socket } from 'socket.io';
import express from 'express';
import http from 'http';
import { schema } from 'schema';
import { ApolloServer } from 'apollo-server-express';
import { checkJwt, pubClient, subClient, logger, mongoose } from 'utils';
import { InMemoryLiveQueryStore } from '@n1ru4l/in-memory-live-query-store';
import { createAdapter } from 'socket.io-redis';

const app = express();
const httpServer = http.createServer(app);

const socketServer = new IOServer(httpServer, {
  cors: {
    origin: ['http://localhost:3000'],
    methods: ['GET', 'POST'],
    allowedHeaders: ['client-name', 'Authorization'],
    credentials: true,
  },
  // path: '/socketio-graphql'
});

socketServer.adapter(createAdapter({ pubClient, subClient }));

const liveQueryStore = new InMemoryLiveQueryStore();

registerSocketIOGraphQLServer({
  socketServer,
  /* getParameter is invoked for each incoming operation and provides all values required for execution. */
  getParameter: async ({
    /* Socket.io instance of the client that executes the operation */
    socket,
  }) => ({
    execute: liveQueryStore.execute,

    /* The parameters used for the operation execution. */
    graphQLExecutionParameter: {
      /* GraphQL schema used for exection (required) */
      schema,
      /* root value for execution (optional) */
      rootValue: {},
      /* context value for execution (optional) */
      contextValue: await (async () => {
        const user = await checkJwt(socket.handshake?.auth?.token);
        return {
          user,
          socket,
          liveQueryStore,
          mongoose,
          logger
        }
      })(),
    },
  }),
});

socketServer.on('connect', (socket: Socket) => console.log({ id: socket.id, token: socket.handshake.auth}));

const apolloServer = new ApolloServer({
  schema,
  context: async ({ req }) => ({
    user: req?.user,
    mongoose,
    logger,
    req,
  }),
  introspection: true,
});

apolloServer.applyMiddleware({ app })

httpServer.listen(4000, () => console.log('listening'));

secondly @n1ru4l/socket-io-graphql-server does not export pubsub. so is there an example guy cause the todo app does not have an example of subscription. but in the readme it says
A layer for serving a GraphQL schema via a socket.io server. Supports Queries, Mutations, Subscriptions and Live Queries.
so how is the subscription suppose to work without pubsub over socket.io.

am I missing something. please help

The patches generated for lists are currently pretty BAD. If there was a possibility for giving JSON patch a hint via the id of an object that could help with generating more efficient patches (as it then can identify a single object as removed from a list instead of all items as changed).

Related: Starcounter-Jack/JSON-Patch#91 Starcounter-Jack/JSON-Patch#92

Some clients such as urql won't trigger a rerender on the components if the new executionResult published by an applyLiveQueryPatchInflator wrapped source was mutated instead of cloned.

benjamine/jsondiffpatch#249

The issue is that the @n1ru4l/graphql-live-query-patch-jsondiffpatch package doesn't function in-browser due to importing everything from the jsondiffpatch package within this file, which makes the browser try to import the chalk dependency, which throws an error in the browser because it's trying to access process which doesn't exist outside of node.

The simplest solution would be disallowing @defer/@stream in a @LiVe query.

There is an error with this repository's Renovate configuration that needs to be fixed. As a precaution, Renovate will stop PRs until it is resolved.

Location: renovate.json
Error type: Invalid JSON (parsing failed)
Message: Syntax error: expecting end of expression or separator near "] "post

we need documentation :)

Is there a possibility of adding a debounce strategy (with an upper limit) ? E.g. debounce changes for 50ms with a hard limit of 500ms. The diff would be between the value at the start of the debounce period (since thats the version matching the client) and the value at the end of the period.

As a use case I'm building an api to a queueing engine, and certain queries can get chatty at high concurrency. Most of the values I'm monitoring are gauges/counters and other entities for which from a dashboard perspective intermediate values (within a certain granularity) are not important.

should be as easy as copying the logic from https://github.com/n1ru4l/graphql-bleeding-edge-playground over here!

In case there is no patch we don't need to send anything over the wire!

Or an AsyncIterator? 🤔

Suggestions:

1. The exports field shouldn't stripped from the published package.json and no index.mjs file is included. @n1ru4l I found that you raised this issue earlier, (though not sure why it was closed) kamilkisiela/bob#16
2. Don't bundle. Publish as commonjs unbundled with both commonjs and ESM entrypoints. This can also allow consumers to deep import files rather than via their named exports. This should be encouraged in the documentation. The entrypoints can remain mostly for static analysis.
3. As an alternative, perhaps consider perhaps only publishing ESM. https://gist.github.com/sindresorhus/a39789f98801d908bbc7ff3ecc99d99c