gatsbyjs / gatsby-graphql-toolkit Goto Github PK

View Code? Open in Web Editor NEW

79.0 79.0 20.0 2.63 MB

TypeScript 99.89% JavaScript 0.11%

gatsby-graphql-toolkit's People

Contributors

Stargazers

Watchers

Forkers

jbampton feedm3 fabe madsmadsdk ivome xtuden-com jmolivas p0intbl4nc kara-todd syfxlin aarondunphy aaronadamsca isabella232 graysonhicks imotor-code freshegguk nrandell soumendudas22 leksat

gatsby-graphql-toolkit's Issues

Invalid object Node Ids create an unclear error message

Invalid nodes sometimes fail with the message with id "[object Object]" is nullish.

This seems to occur when the Node ID is an object rather than an integer or string.

Duplicate nodes overwrite unique nodes, resulting in missing Gatsby GraphQL nodes.

I'm using gatsby-source-craft but a lot of the entires nodes are getting overwritten when it creates the Gatsby GraphQL layer. So for example, gatsby-source-craft uses gatsby-graphql-toolkit to bring down all of the data from the Craft servers GraphQL: I have 3,494 entries of a certain type, during the fetchAllNodes (I think) part of sourceAllNodes, some nodes are replaced with duplicates of each other, so that when createNode is called with the same id, I end up with only 1,842 nodes in the Gatsby GraphQL.

I've checked the generated GraphQL from gatsby-graphql-toolkit, when I run it in the Craft CMS GraphiQL playground, it returns all 3,494 nodes with unique ids and fields/data. I was unable to comment out schemaCustomization without the project breaking completely.

Issue in gatsby-source-craft

I'm at a loss with how to proceed currently, I can't find exactly where the duplication/overwriting of the nodes occurs.

compileGatsbyFragments: cannot use __typename on field of interface type

Say we have the following custom fragment:

fragment Example on MyInterface {
  __typename
  someField
}

The compileNodeQueries will transform it to something like this for node query:

fragment Example on MyInterface {
  remoteTypeName: __typename
  someField
}

And compileGatsbyFragments will transform it to:

fragment Example on PrefixMyInterface {
  remoteTypeName
  someField
}

The problem though is that the field remoteTypeName is not added to PrefixMyInterface and as a result, Gatsby query fails with an error.

Fails to execute LIST_ query with NoPagination

Following error message is provided:

Error: Failed to execute query LIST_WATCHES.
    at Object.paginate (/Users/rubendebruijn/schaap_citroen_v2/node_modules/gatsby-graphql-source-toolkit/src/source-nodes/fetch-nodes/paginate.ts:60:13)
    at processTicksAndRejections (node:internal/process/task_queues:96:5)
    at fetchNodeList (/Users/rubendebruijn/schaap_citroen_v2/node_modules/gatsby-graphql-source-toolkit/src/source-nodes/fetch-nodes/fetch-lists.ts:61:20)
    at Object.fetchAllNodes
(/Users/rubendebruijn/schaap_citroen_v2/node_modules/gatsby-graphql-source-toolkit/src/source-nodes/fetch-nodes/fetch-lists.ts:36:24)
    at Object.createNodes
(/Users/rubendebruijn/schaap_citroen_v2/node_modules/gatsby-graphql-source-toolkit/src/source-nodes/node-actions/create-nodes.ts:13:20)
    at async Promise.all (index 0)
    at sourceAllNodes (/Users/rubendebruijn/schaap_citroen_v2/node_modules/gatsby-graphql-source-toolkit/src/source-nodes/source-all-nodes.ts:20:3)
    at Object.exports.sourceNodes (/Users/rubendebruijn/schaap_citroen_v2/gatsby-node.js:99:3)
    at runAPI (/Users/rubendebruijn/schaap_citroen_v2/node_modules/gatsby/src/utils/api-runner-node.js:462:16)

As described in the guide, i implemented a PaginationAdapter that doesn't require variables. There is no pagination implemented in the BE i connect with.

Still, i would love to automaticly generate the fragments with generateDefaultFragments This issue doesn't seem to appear when i write a fragment manually and generate it with readOrGenerateDefaultFragments

Approach:

const NoPagination = {
  name: "NoPagination",
  expectedVariableNames: [],
  start() {
      return {
          variables: {},
          hasNextPage: true,
      };
  },
  next() {
      return {
          variables: {},
          hasNextPage: false,
      };
  },
  concat(result) {
      return result;
  },
  getItems(pageOrResult) {
      return pageOrResult;
  },
}

async function createSourcingConfig(gatsbyApi) {
  const execute = createDefaultQueryExecutor(`https://mywatchapi.com/graphql`);
  const schema = await loadSchema(execute)

const gatsbyNodeTypes = [
  {
    remoteTypeName: `Watch`,
    queries: `
      query LIST_WATCHES {
        watches { 
            ..._WatchId_ 
         }
      }
      fragment _WatchId_ on Watch {
          __typename 
          id 
      }
    `,
  },
]

  const fragments = await generateDefaultFragments({ schema, gatsbyNodeTypes })
  
  const documents = compileNodeQueries({
    schema,
    gatsbyNodeTypes,
    customFragments: fragments,
  })

  return {
    gatsbyApi,
    schema,
    execute,
    gatsbyTypePrefix: `Example`,
    gatsbyNodeDefs: buildNodeDefinitions({ gatsbyNodeTypes, documents }),
    paginationAdapters: [NoPagination],
  }
}

Expected

sourceAllNodes will run succesfuly, while the fragment generated automaticly

Actual

sourceAllNodes runs, but no nodes were created and error is thrown

Feat: display warning when there are non-critical errors in the response

We should display a warning in the default execute implementation when there are errors along with data:

{
  "errors": [
    {
      "message": "Some non-critical error."
    }
  ],
  "data": {
    "list": []
  },
}

(GraphQL spec treats responses without data key as completely failed. If data key exists in the response, then errors are effectively warnings)

gatsby 4 - cannot call `createTypes` in the `sourceNodes` API

Are there plans to support gatsby 4 on the way ?

https://v4.gatsbyjs.com/docs/reference/release-notes/migrating-from-v3-to-v4/#disallow-schema-related-apis-in-sourcenodes

0.5.0 Introduces regression when compiling queries

after upgrading the compiled queries are quite strange and apply fragments on types that I don't expect to see there

example 0.4.0:

query LIST_accordionListItem($limit: Int, $skip: Int) {
  accordionListItemCollection(limit: $limit, skip: $skip) {
    total
    items {
      remoteTypeName: __typename
      ...AccordionListItemId
      ...AccordionListItem
    }
  }
}

fragment AccordionListItemId on AccordionListItem {
  remoteTypeName: __typename
  sys {
    id
  }
}

fragment AccordionListItem on AccordionListItem {
  sys {
    ... on Sys {
      id
      spaceId
      environmentId
      publishedAt
      firstPublishedAt
      publishedVersion
    }
  }
  entryTitle
  title
  description
  content {
    remoteTypeName: __typename
    ... on PaxTimeTable {
      remoteTypeName: __typename
      sys {
        id
      }
    }
    ... on TextAndMediaLayout {
      remoteTypeName: __typename
      sys {
        id
      }
    }
    ... on PageContentTextSection {
      remoteTypeName: __typename
      sys {
        id
      }
    }
  }
}

after 0.5.0:

query LIST_accordionListItem($limit: Int, $skip: Int) {
  accordionListItemCollection(limit: $limit, skip: $skip) {
    total
    items {
      remoteTypeName: __typename
      ...AccordionListItemId
      ...PageContentAccordionList__itemsCollection__0__items__0
      ...AccordionListItem
      ...SupportFormLogistics__emailTemplate__0
    }
  }
}

fragment AccordionListItemId on AccordionListItem {
  remoteTypeName: __typename
  sys {
    id
  }
}

fragment PageContentAccordionList__itemsCollection__0__items__0 on AccordionListItem {
  remoteTypeName: __typename
  sys {
    remoteTypeName: __typename
    id
  }
}

fragment AccordionListItem on AccordionListItem {
  sys {
    remoteTypeName: __typename
    ... on Sys {
      id
      spaceId
      environmentId
      publishedAt
      firstPublishedAt
      publishedVersion
    }
  }
  entryTitle
  title
  description
  content {
    remoteTypeName: __typename
    ... on PaxTimeTable {
      remoteTypeName: __typename
      sys {
        remoteTypeName: __typename
        id
      }
    }
    ... on TextAndMediaLayout {
      remoteTypeName: __typename
      sys {
        remoteTypeName: __typename
        id
      }
    }
    ... on PageContentTextSection {
      remoteTypeName: __typename
      sys {
        remoteTypeName: __typename
        id
      }
    }
  }
}

fragment SupportFormLogistics__emailTemplate__0 on Entry {
  remoteTypeName: __typename
  sys {
    remoteTypeName: __typename
    id
  }
}

Cannot query field

I am running into an issue where everything works great in development but I get the following error in production:

There was an error in your GraphQL query:
Cannot query field "cmsSeries" on type "Query".

I am not quite sure where this error is coming from since everything works as it should in development and also the build schema has all the correct types.

I am using [email protected] and the [email protected].

feat: display error on 4xx/5xx http error codes

I copy/pasted the wrong access token while trying things and was confused for a while why it wasn't working 🤦

I'd assume the toolkit would at least display the http errors for these and perhaps also just quit altogether?

wagtail grapple support

Hi,

I'm trying to get wagtail-grapple to support your toolkit. See https://wagtail-grapple.readthedocs.io/en/latest/

There are a few issues I have related to generating cache when using the lib.

   const nodeInterface = schema.getType(`PageInterface`)
    const possibleTypes = schema.getPossibleTypes(nodeInterface)

    const gatsbyNodeTypes = possibleTypes.map((type) => {
        let query = `
   query LIST_${pluralize(type.name).toUpperCase()} ($limit: PositiveInt, $offset: PositiveInt) {
 
      pages(limit: $limit, offset: $offset) {
      remoteId: id
        ..._${type.name}Id_
        
      }
    
    }
    fragment _${type.name}Id_ on ${type.name} {
         remoteId: id
        __typename
        id
    }
    `;

This will generate a fragment:

fragment BlogPage on BlogPage {
  remoteId: id
  path
  depth
  numchild
  translationKey
  title
  draftTitle
  slug
  contentType
  live
  hasUnpublishedChanges
  urlPath
  seoTitle
  showInMenus
  searchDescription
  goLiveAt
  expireAt
  expired
  locked
  lockedAt
  firstPublishedAt
  lastPublishedAt
  latestRevisionCreatedAt
  aliasOf {
    ... on Page {
      remoteTypeName: __typename
      remoteId: id
      remoteId: id
    }
  }
  author
  date
  body {
    ... on StreamFieldBlock {
      id
      blockType
      field
      rawValue
      value
    }
    ... on CharBlock {
      id
      blockType
      field
      rawValue
      value
    }
    ... on TextBlock {
      id
      blockType
      field
      rawValue
      value
    }
    ... on EmailBlock {
      id
      blockType
      field
      rawValue
      value
    }
    ... on IntegerBlock {
      id
      blockType
      field
      rawValue
      value
    }
    ... on FloatBlock {
      id
      blockType
      field
      rawValue
      value
    }
    ... on DecimalBlock {
      id
      blockType
      field
      rawValue
      value
    }
    ... on RegexBlock {
      id
      blockType
      field
      rawValue
      value
    }
    ... on URLBlock {
      id
      blockType
      field
      rawValue
      value
    }
    ... on BooleanBlock {
      id
      blockType
      field
      rawValue
      value
    }
    ... on DateBlock {
      id
      blockType
      field
      rawValue
      value
    }
    ... on TimeBlock {
      id
      blockType
      field
      rawValue
      value
    }
    ... on DateTimeBlock {
      id
      blockType
      field
      rawValue
      value
    }
    ... on RichTextBlock {
      id
      blockType
      field
      rawValue
      value
    }
    ... on RawHTMLBlock {
      id
      blockType
      field
      rawValue
      value
    }
    ... on BlockQuoteBlock {
      id
      blockType
      field
      rawValue
      value
    }
    ... on ChoiceBlock {
      id
      blockType
      field
      rawValue
      value
      choices {
        ... on ChoiceOption {
          key
          value
        }
      }
    }
    ... on StreamBlock {
      id
      blockType
      field
      rawValue
      blocks {
        remoteTypeName: __typename
      }
    }
    ... on StructBlock {
      id
      blockType
      field
      rawValue
      blocks {
        remoteTypeName: __typename
      }
    }
    ... on StaticBlock {
      id
      blockType
      field
      rawValue
      value
    }
    ... on ListBlock {
      id
      blockType
      field
      rawValue
      items {
        remoteTypeName: __typename
      }
    }
    ... on EmbedBlock {
      id
      blockType
      field
      rawValue
      value
      url
      embed
      rawEmbed
    }
    ... on PageChooserBlock {
      id
      blockType
      field
      rawValue
      page {
        ... on PageInterface {
          remoteTypeName: __typename
          remoteId: id
        }
      }
    }
    ... on DocumentChooserBlock {
      id
      blockType
      field
      rawValue
      document {
        ... on DocumentObjectType {
          id
          collection {
            ... on CollectionObjectType {
              id
              path
              depth
              numchild
              name
              descendants {
                remoteTypeName: __typename
              }
              ancestors {
                remoteTypeName: __typename
              }
            }
          }
          title
          file
          createdAt
          fileSize
          fileHash
          url
          tags {
            ... on TagObjectType {
              id
              name
            }
          }
        }
      }
    }
    ... on ImageChooserBlock {
      id
      blockType
      field
      rawValue
      image {
        ... on CustomImage {
          id
          collection {
            ... on CollectionObjectType {
              id
              path
              depth
              numchild
              name
              descendants {
                remoteTypeName: __typename
              }
              ancestors {
                remoteTypeName: __typename
              }
            }
          }
          title
          file
          width
          height
          createdAt
          focalPointX
          focalPointY
          focalPointWidth
          focalPointHeight
          fileSize
          fileHash
          src
          url
          aspectRatio
          sizes
          tags {
            ... on TagObjectType {
              id
              name
            }
          }
          rendition {
            ... on CustomImageRendition {
              id
              file
              width
              height
              image {
                remoteTypeName: __typename
              }
              title
              createdAt
              focalPointX
              focalPointY
              focalPointWidth
              focalPointHeight
              fileSize
              fileHash
              src
              url
              aspectRatio
              sizes
              collection {
                ... on CollectionObjectType {
                  id
                  path
                  depth
                  numchild
                  name
                  descendants {
                    remoteTypeName: __typename
                  }
                  ancestors {
                    remoteTypeName: __typename
                  }
                }
              }
              tags {
                ... on TagObjectType {
                  id
                  name
                }
              }
              rendition {
                remoteTypeName: __typename
              }
              srcSet
              customRenditionProperty
            }
          }
          srcSet
          image640
          image800
          image1024
        }
      }
    }
    ... on SnippetChooserBlock {
      id
      blockType
      field
      rawValue
      snippet
    }
  }
  pageType
  url
  remoteParent: parent {
    ... on PageInterface {
      remoteTypeName: __typename
      remoteId: id
    }
  }
  remoteChildren: children {
    ... on PageInterface {
      remoteTypeName: __typename
      remoteId: id
    }
  }
  siblings {
    ... on PageInterface {
      remoteTypeName: __typename
      remoteId: id
    }
  }
  nextSiblings {
    ... on PageInterface {
      remoteTypeName: __typename
      remoteId: id
    }
  }
  previousSiblings {
    ... on PageInterface {
      remoteTypeName: __typename
      remoteId: id
    }
  }
  descendants {
    ... on PageInterface {
      remoteTypeName: __typename
      remoteId: id
    }
  }
  ancestors {
    ... on PageInterface {
      remoteTypeName: __typename
      remoteId: id
    }
  }
}

generates the needed pages. But I have a few things that don't work correctly:

The lib adds a value key, which conflicts because the interface classes overlap that are generated.

 Error: Failed to execute query LIST_BLOGPAGES.
  Errors:
  Fields "value" conflict because they return conflicting types String! and Int!. Use different aliases on the fields to fetch both if this was intentional.
  Fields "value" conflict because they return conflicting types String! and Float!. Use different aliases on the fields to fetch both if this was intentional.
  Fields "value" conflict because they return conflicting types String! and Float!. Use different aliases on the fields to fetch both if this was intentional.

StreamBlocks have multiple blocks

 ... on StreamBlock {
      id
      blockType
      field
      rawValue
      blocks {
        remoteTypeName: __typename

      }
    }

StreamBlock.blocks has multiple interfaces, but not recognize.

Currently I use regex to fix the missing keys

Nullable lists are transformed into non-nullable lists

The problem

A bug in the gatsby-graphql-toolkit transforms a nullable list from the API into a non-nullable list within the internal Gatsby GraphQL API. Additionally, if the list entry is non-nullable it will become nullable. The null operator gets flipped between the list and the entry.

This makes using nullable lists impossible. As soon as the API will deliver null instead of an empty array, Gatsby fails with an error during sourcing saying this is an invalid value (although it is not).

Example

Let's say the Graphql API looks like this:

type Type {
  list: [Entry!]
}

If we query the __schema field, we get the following result:

 {
  "name": "list",
  "type": {
    "kind": "LIST",
    "name": null,
    "ofType": {
      "kind": "NON_NULL",
      "name": null,
      "ofType": {
        "name": "Entry",
        "kind": "OBJECT",
        "ofType": null
      }
    }
  }
},

This is all correct. Now we take the gatsby-graphql-toolkit to add the Type to our Gatsby store. This will transform the list into a wrong type:

type GatsbyType {
  list: [GatsbyEntry]!
}

With the __schema result:

{
  "name": "list",
  "type": {
    "kind": "NON_NULL",
    "name": null,
    "ofType": {
      "kind": "LIST",
      "name": null,
      "ofType": {
        "name": "GatsbyEntry",
        "kind": "OBJECT",
        "ofType": null
      }
    }
  }
}

LIST does not fetch all data

I am happy to provide more information and might not be using the right terminology but the gatsby-source-craft fails to gather all the entries - it gathers all but one on two of the LIST queries.

The query gatsby-source-craft fetches, when ran through the GraphQL on my CraftCMS returns the right amount of entries, but the plugin - which uses gatsby-graphql-toolkit - fails to return all the entries.

Any suggestions ?

Add ability to map custom scalar types

At the moment, any custom scalar type is replaced with the most generic JSON type. We should allow you to define a map of remoteScalarType => gatsbyScalarType in the config.

Technically it is also possible now by overriding toolkit's types with schema customization. But it is tedious as it has to be done per-field (so if you have fields of type Timestamp on 100 different types, you will have to add 100 type definitions manually).

Cannot query field on type "Query"

I have used the https://github.com/vladar/gatsby-graphql-toolkit-examples/tree/master/craft-cms example and seem to run into an issue...

in the craft-cms graphiql editor, the following graphql query:

query {
  globalSet {
    id
  }
}

returns this data:

{
  "data": {
    "globalSet": {
      "id": "70"
    }
  }
}

But in gatsby, using the gatsby-graphql-toolkit to create the data layer, the same query

const query = useStaticQuery(graphql`
  query {
    globalSet {
      id
    }
  }
`);

results in the following error:

There was an error in your GraphQL query:

Cannot query field "globalSet" on type "Query".

If you don't expect "globalSet" to exist on the type "Query" it is most likely a
 typo. However, if you expect "globalSet" to exist there are a couple of
solutions to common problems:

- If you added a new data source and/or changed something inside
gatsby-node/gatsby-config, please try a restart of your development server.
- You want to optionally use your field "globalSet" and right now it is not used
 anywhere.

It is recommended to explicitly type your GraphQL schema if you want to use
optional fields.

File: src/components/molecules/autocomplete/autocomplete.jsx:157:9

See our docs page for more info on this error:
https://gatsby.dev/creating-type-definitions

I have gone through the Troubleshooting instructions and believe that there is an issue with the schema customization logic, which is why I'm filing this issue.

I hope there is someone who can assist! Been trying to resolve this for a couple of weeks.

Using:
Gatsby 5.8.0
Gatsby Graphql Source Toolkit 2.0.4

Accept a function as `gatsbyTypePrefix` .

I'm working with Sanity Graphql, some remote schema types use TypeName and others use SanityTypeName, how can I achieve all generated Gatsby types say SanityRemoteTypeName, providing a gatsbyTypePrefix = "Sanity" will cause some types to become SanitySanityRemoteTypeName and not providing any gatsbyTypePrefix will result in some fields being RemoteTypeName.

So I think gatsbyTypePrefix option should accept a function that will get passed the RemoteTypeName and it will check if it's already prefixed with Sanity then do nothing otherwise prefix with Sanity

success fetching Nodes, but no Gatsby nodes created.

The schema is created and there are no errors, but no gatsby nodes created.
Trying the compiled queries on remote graphql works just fine.

Toolkit throws TypeError if query returns a non-iterable object

Problem

Our Gatsby project consumes numerous queries from our single graphql endpoint. The various queries return either an Array, or a non-iterable object. See example below:

// Array datatype
type: [Posts!]!

// non-iterable object
type: LocationSettings!

The toolkit properly handles the queries which return iterable objects (i.e. Array). However, the toolkit fails to properly handle non-iterable objects, by throwing a TypeError. I have an extract of the error stack-trace below:

warn TypeError: partialNodes is not iterable
    at fetchNodeList (/gatsby/node_modules/gatsby-graphql-source-toolkit/src/source-nodes/fetch-nodes/fetch-lists.ts:64:24)
    at processTicksAndRejections (internal/process/task_queues.js:93:5)
    at Object.fetchAllNodes (/gatsby/node_modules/gatsby-graphql-source-toolkit/src/source-nodes/fetch-nodes/fetch-lists.ts:36:24)
    at Object.createNodes (/gatsby/node_modules/gatsby-graphql-source-toolkit/src/source-nodes/node-actions/create-nodes.ts:13:20)
    at async Promise.all (index 0)
    at sourceAllNodes (/gatsby/node_modules/gatsby-graphql-source-toolkit/src/source-nodes/source-all-nodes.ts:20:3)
    at Object.exports.sourceNodes (/gatsby/website/plugins/gatsby-source-odunet/gatsby-node.js:69:5)
    at runAPI (/gatsby/node_modules/gatsby/src/utils/api-runner-node.js:434:16)

Proposed resolution

The stack trace shows that the error is thrown in the code block below. The toolkit expects partialNodes to be iterable. It is therefore unable to handle queries that return non-iterable objects, e.g. type: LocationSettings! above.

gatsby-graphql-toolkit/src/source-nodes/fetch-nodes/fetch-lists.ts

Lines 61 to 73 in 2b705fb

 for await (const page of paginate(context, plan)) { 

 const partialNodes = plan.adapter.getItems(page.fieldValue) 

 for (const node of partialNodes) { 

 if (!node || node[typeNameField] !== remoteTypeName) { 

 // Possible when fetching complex interface or union type fields 

 // or when some node is `null` 

 continue 

 } 

 // TODO: run in parallel? 

 yield addPaginatedFields(context, nodeDefinition, node) 

 } 

 }

A possible fix would introduce a conditional statement to check the partialNodes type, and properly handle it. I have a sample solution below which will replace the block of code above:

 if (Array.isArray(partialNodes)) {
    for (const node of partialNodes) {
      if (!node || node[typeNameField] !== remoteTypeName) {
        // Possible when fetching complex interface or union type fields
        // or when some node is `null`
        continue
      }
      // TODO: run in parallel?
      yield addPaginatedFields(context, nodeDefinition, node)
    }
 } else if (partialNodes && partialNodes[typeNameField] === remoteTypeName) {
     yield addPaginatedFields(context, nodeDefinition, partialNodes)
 }

Feat: a way to avoid certain references

Problem/Motivation

I was experimenting with Drupal. In most cases the toolkit was just working, which is super cool. Thanks for the great work!

Now I met the first issue when tried to add content translation support. In Drupal, content translations aren't separate entities, they don't have own IDs, just language codes differ. The idea was to fetch all translations of a Drupal entity into a single Gatsby node. However, in Drupal schema, a content entity and it's translations have the same GraphQL type.

Probably better to explain with screenshots :)

Here is my original fragment:

Both entities and entityTranslations fields have [NodeArticle] type. The toolkit's sees the entityTranslations type and replaces the original fields with a reference. This leads to two things:

entityTranslations field's sub-fields are moved to entities field.
Content translations are fetched with separate NODE_ queries. Drupal serve all of them in the default language.

Proposed resolution

Add something like @noReference directive:

fragment NodeArticle on NodeArticle {
  entityId
  entityTranslations @noReference {
    entityLabel
    body {
      summaryProcessed
      processed
    }
    fieldTags {
      entity {
        ...TaxonomyTermTags
      }
    }
    fieldImage {
      entity {
        ...MediaImage
      }
    }
  }
}

Does this proposal sound good?

Stale data invalidation with delta syncing

GraphQL schema doesn't convey full information about data interdependencies and relations. So it is possible to have some pieces of data stale with delta sync. Quick example:

type Category {
  title: String
  totalCommentCount: Int
}

type Post {
  category: Category
  commentCount: Int
  comments: [Comment]
}

Say some comment is deleted and we are notified about it via a DELETE event. Three things are stale now:

Reference to this comment in Post.comments field
Post.commentCount field is also stale as it has to be decremented on comment delete
Category.totalCommentCount - same

When this happens there should be a way to update specific fields of related nodes. So in addition to UPDATE and DELETE events (targeting individual nodes), we need UPDATE_FIELDS event that can notify us about specific node fields that had changed.

This event would generate dynamic new NODE_ query that will only fetch the mentioned fields and update the node. So it will look something like this:

{
  eventName: "UPDATE_FIELDS",
  remoteTypeName: "Post",
  remoteId: { __typename: "Post", id: "1" },
  fields: ["commentCount", "comments"]
},
{
  eventName: "UPDATE_FIELDS",
  remoteTypeName: "Category",
  remoteId: { __typename: "Category", id: "1" },
  fields: ["totalCommentCount"]
},

Additionally maybe allow fields to be a fragment, i.e.:

{
  eventName: "UPDATE_FIELDS",
  remoteTypeName: "Post",
  remoteId: { __typename: "Post", id: "1" },
  fields: `fragment ChangedFields on Post { commentCount, comments { __typename } }`
},

Technically we can update related nodes fully. But that can be pretty heavy in some cases, so more targeted updates can be useful in performance-sensitive scenarios.

Perf: interfaces on non-node types produce huge queries

Take this schema:

interface Foo {
  foo: String
  bar: Int
}

type Foo1 implements Foo {
  foo: String
  bar: Int
  baz1: String
}

#...

type Foo99 implements Foo {
  foo: String
  bar: Int
  baz99: String
}

type Problem {
  foo: Foo
  foo2: Foo
}

Now if we try to generate a fragment for the type Problem it will inline every single FooX type (assuming they are non-node types):

fragment Problem on Problem {
  foo {
    ... on Foo1 {
      foo
      bar
      baz1
    }
    # ...
    ... on Foo99 {
      foo
      bar
      baz99
    }
  }

  foo2 {
    ... on Foo1 {
      foo
      bar
      baz1
    }
    # ...
    ... on Foo99 {
      foo
      bar
      baz99
    }
  }
}

If there are many fields of this type - things can get pretty wild. One idea of how to fix this - create a separate fragment for each non-node interface type:

fragment Foo on Foo {
  foo
  bar
    ... on Foo1 {
     baz1
    }
    # ...
    ... on Foo99 {
      baz99
    }
}
fragment Problem on Problem {
  foo {
    ...Foo
  }
  foo2 {
    ...Foo
  }
}

Complains about null for non-null type

I am getting the following error message:

Cannot return null for non-nullable field CMCategory.nameDe.

However, none of the returned CMCategory from the origin graphql server returns null for this field.

A reproduction sample is available at
https://github.com/rburgst/gatsby-graphql-toolkit-bug

This is the server schema

  type Query {
    getCategories: [Category]!
    getConcerts: [Concert]!
  }

  type Category {
    id: String!
    nameDe: String!
    nameEn: String!
    displayOrder: Int!
    isSubcategoryOf: Category
  }
  type Concert {
    id: ID!
    category: Category
    secondCategory: Category
    titleDe: String!
    titleEn: String!
  }

To replicate do the following

start the dev server
```
cd server
pnpm install
pnpm dev
```
In another shell start a gatsby build
```
cd gatsby
yarn install
yarn build
```

Expected

the build runs through

Actual

The following error is thrown


 ERROR #85925  GRAPHQL

There was an error in your GraphQL query:

Cannot return null for non-nullable field CMCategory.nameDe.

The field "CMCategory.nameDe." was explicitly defined as non-nullable via the schema customization API (by yourself or a plugin/theme). This means that this field is not optional and you have to define a value. If this is not your
desired behavior and you defined the schema yourself, go to "createTypes" in gatsby-node.js. If you're using a plugin/theme, you can learn more here on how to fix field types:
https://www.gatsbyjs.com/docs/reference/graphql-data-layer/schema-customization#fixing-field-types

   1 | fragment ConcertCategoryFragment on CMCategory {
   2 |   remoteId
>  3 |   nameDe
     |   ^
   4 |   nameEn
   5 |   isSubcategoryOf {
   6 |     remoteId
   7 |     nameDe
   8 |     nameEn
   9 |   }
  10 | }
  11 |

GraphQL 16 compatibility

Make the toolkit compatible with GQL 16.

The only problem I see is this visitor, which is causing TypeError: visitFn.call is not a function:

gatsby-graphql-toolkit/src/compile-node-queries/ast-transformers/remove-unused-fragments.ts

Lines 11 to 42 in b395feb

 return { 

 enter: { 

 FragmentSpread: node => { 

 currentSpreads.push(node.name.value) 

 }, 

 }, 

 leave: { 

 OperationDefinition: node => { 

 if (!node.name?.value) { 

 throw new Error("Every query must have a name") 

 } 

 definitionSpreads.set(node.name.value, currentSpreads) 

 currentSpreads = [] 

 }, 

 FragmentDefinition: node => { 

 definitionSpreads.set(node.name.value, currentSpreads) 

 currentSpreads = [] 

 }, 

 Document: node => { 

 const operations = node.definitions.filter(isOperation) 

 const operationNames = operations.map(op => op.name?.value) 

 const usedSpreads = new Set( 

 operationNames.reduce(collectSpreadsRecursively, []) 

 ) 

 const usedFragments = node.definitions.filter( 

 node => isFragment(node) && usedSpreads.has(node.name.value) 

 ) 

 return GraphQLAST.document([...operations, ...usedFragments]) 

 }, 

 }, 

 }

Support for enter and leave objects was dropped in GraphQL 16 (graphql/graphql-js#2957), so the visitor would just need to be rewritten in this format:

visit(ast, {
  Kind: {
    enter(node) {
      // enter the "Kind" node
    }
    leave(node) {
      // leave the "Kind" node
    }
  }
})

I've patched this locally, and otherwise everything seems to be working fine.

Import interfaces from source schema

Is it possible to define an interface in my source schema and properly reflect it in the Gatsby schema?

Example:

interface Person {
  name: String!
}

type Employee implements Person {
  name: String!
  department: String!
}

type Client implements Person {
  name: String!
  company: String!
}

Both Employee and Client are nodes that are sourced to Gatsby, but the interface is dropped entirely. If a field returns the Person type, the interface will be created and correctly applied to Employee and Client, but it won't contain any fields except remoteTypeName.

I would be great to be able to run a reusable fragment against an interface:

fragment Person {
  name
}

But if i'm not mistaken, this is not possible right now?

Reporter in fetch-lists.ts causes significant slowdown

I have a lot of types and a lot of nodes to be fetched and looks like this kind of reporting slows down the fetching process.

When I remove the reporter locally, I manage to source the data for 10x the time. This might be that my laptop does not have the best hardware but it's definitely faster without the reporter.

Maybe we should find a better way to log what is happening in that function, possibly provide an api to log things if one wants to.

https://github.com/gatsbyjs/gatsby-graphql-toolkit/blob/0ece5cec5c7998150dc3eb360015846c5fa233bf/src/source-nodes/fetch-nodes/fetch-lists.ts#L21&L22

Provide a way to skip generation of certain fields

The fragment generation step is very useful but my schema is very big and I don't use everything from it so I would like to skip generating some fields. Our cms graphql api is limited to a certain amount of query complexity and the generated queries for me are always too large to query from and manually fixing them is quite the task 💯

The way of setting default values for arguments is super nice way to also maybe do this?

I am happy to make the changes if it's agreed this would be useful :)

Is there a way to do Automatic Persisted Queries with this toolkit?

Hi @vladar,

@smthomas mentioned this repo in the weekly Drupal/Gatsby meeting. I'm curious if you have a recommendation for how to do Automatic Persisted Queries with WordPress and Drupal leveraging this toolkit. I work at Pantheon and I think we'll be able to show significant improvements in build times if the majority of queries are cachable in our CDN.

I've experimented with using Automatic Persisted Queries in WordPress using gatsby-source-graphql and https://github.com/Quartz/wp-graphql-persisted-queries by overriding createLink:

createLink: pluginOptions => {
   return createPersistedQueryLink({useGETForHashedQueries: true}).concat(createHttpLink({ uri: `https://${process.env.DATA_SOURCE_URL}/graphql`, fetch: fetch }))
}

It seemed to work but I'm looking for guidance from someone who knows more about GraphQL around best practices using these newer plugins.

Support lists with a single value in NODE_ queries

Some schemas do not provide any means to fetch a single node. Instead, they allow you to query a list of nodes by id (but returning a list with a single element). E.g.

query NODE_FOO {
  queryFoo(input: { id: $id }) {
    ...idFragment
  }
}

and the return value could be:

{
  data: {
    queryFoo: [ { id: 'foo' } ]
  }
}

The toolkit must support this and extract { id: 'foo' } as the node value. Right now it erroneously extracts [ { id: 'foo' } ] as a node value.

Sourcing non-list data

Hi, I have an issue which I can't find any docs for .. I have a non-list type that just returns a set of site settings so does not have an ID or anything like that to filter on... what the best way to add this. do I need to get it into readOrGenerateDefaultFragments somehow?

if I do something like this:

    {
      remoteTypeName: `Settings`,
      queries: `
 
        query GET_SETTINGS {
          settings {
             ..._SettingsId_ 
          }
       }
  
        fragment _SettingsId_ on Settings { __typename  }
 
      `,
    },

Gatsby picks up the Shema shape but will not return any data??

Select GraphQL errors thrown while creating source nodes

While creating source nodes, a GraphQL error is thrown for some, but not all, node types

Failed to execute query LIST_faqs_faqs_Entry.
Errors:
Fragment "faqs_faqs_Entry" is never used.

The error is thrown when creating the pagination adapters:
- paginate.ts:60 Object.paginate [gatsby-source-craftcms]/[gatsby-graphql-source-toolkit]/src/source-nodes/fetch-nodes/paginate.ts:60:13

I'm attaching the generated graphql document for that node.

faqs_faqs_Entry.graphql.zip

Toolkit version: 0.30

Support variables within input objects

The toolkit doesn't support this at the moment:

query LIST_FOO {
  foo(input: { page: $page }) { ..._FooId_ }
}

only supports this:

query LIST_FOO {
  foo(input: $input) { ..._FooId_ }
}

It fails to add variable declarations in this case and also fails to discover the field to paginate.

	for await (const page of paginate(context, plan)) {
	const partialNodes = plan.adapter.getItems(page.fieldValue)

	for (const node of partialNodes) {
	if (!node \|\| node[typeNameField] !== remoteTypeName) {
	// Possible when fetching complex interface or union type fields
	// or when some node is `null`
	continue
	}
	// TODO: run in parallel?
	yield addPaginatedFields(context, nodeDefinition, node)
	}
	}

	return {
	enter: {
	FragmentSpread: node => {
	currentSpreads.push(node.name.value)
	},
	},
	leave: {
	OperationDefinition: node => {
	if (!node.name?.value) {
	throw new Error("Every query must have a name")
	}
	definitionSpreads.set(node.name.value, currentSpreads)
	currentSpreads = []
	},
	FragmentDefinition: node => {
	definitionSpreads.set(node.name.value, currentSpreads)
	currentSpreads = []
	},
	Document: node => {
	const operations = node.definitions.filter(isOperation)
	const operationNames = operations.map(op => op.name?.value)

	const usedSpreads = new Set(
	operationNames.reduce(collectSpreadsRecursively, [])
	)
	const usedFragments = node.definitions.filter(
	node => isFragment(node) && usedSpreads.has(node.name.value)
	)
	return GraphQLAST.document([...operations, ...usedFragments])
	},
	},
	}

gatsbyjs / gatsby-graphql-toolkit Goto Github PK

gatsby-graphql-toolkit's People

Contributors

Stargazers

Watchers

Forkers

gatsby-graphql-toolkit's Issues

Following error message is provided:

Approach:

Expected

Actual

The problem

Example

Problem

Proposed resolution

Problem/Motivation

Proposed resolution

Expected

Actual

Recommend Projects

Recommend Topics

Recommend Org