Since we are going to recommend creating plenty of queries and uses it locally, we need to provide some way to clone Query.

Proposed API

Let any Query has a method __.clone that will create a clone of the Query, but we won't allow using directly, because of AST-awared tools and problems with methods. We can add operator clone that will accept any Query and call its internal clone method.

API can changes 🤗

Suggested API:

retry(query, { times, delay, filter, mapParams, otherwise })
rertry(mutation, { times, delay, filter, mapParams, otherwise })

I guess, we can deprecate old form and remove it a little bit later 🤗

Provide a complementary package with io-ts contract applier.

[Vite](vite-link) uses ESBuild under the hood, which does not allow to use its internal AST in the plugins. To apply custom transformations to the code one must use either [Babel](vite-babel-plugin link) or [SWC](vite-swc-plugin link), which are allowing custom AST-transformations.

cc @AlexandrHoroshih

The problem

fn in connectQuery accepts data from source-Query and returns parameters for target-Query. It disallows us to extend the semantic of the operator.

Solution

If fn returns object { params: Params } we will be able to extend it with other settings of target-Query. E.g., it allows us in future to add placeholder formulation in the same operator.

Now we manually add showcases to a particular page, it would be nice to have one config for showcase which will be used to inject a link to it to every related page.

Something like this 👇

// showcase/react-create-query/docs.ts

export default {
  name: "React + createQuery",
  articles: [
    "/api/factories/create-query/",
    "/integrations/react/use-query/"
  ]
}

Do we need to create integration for https://github.com/ianstormtaylor/superstruct as a Contract provider?

We have private implementation of createGraphQLQuery factory in Aviasales, this factory could not be ported to Farfetched because it has plenty of compromises related to our app. Hoverer, it would be nice to write a recipe about it in docs.

Installation instruction takes a lot of screen space 👇

It would be nice to replace it with single block with switcher pnpm/npm/yarn.

PR in Vitepress

nx does not allow using jest 28 which is required to write tests with real Fetch API, so we are using whatwg-fetch.

After nrwl/nx#10857 will be merged, we should upgrade nx, jest and remove whatwg-fetch from the repo.

It leads to broken build in master. We have to add something like pnpm i --lockfile-only before commit.

https://vitepress.vuejs.org/guide/theme-team-page

Releases

v.0.1 Samet Nangshe

v.0.2 Laem Promthep

• @farfetched/core
  • createJsonMutation
  • createMutation
  • handle stale after mutation
  • postpone mutation (analogue of connectQuery)
  • optimistic update operator
  • automagic refetch
  • Retry API
  • initial for createHeadlessQuery
• @farfetched/react
  • useMutation

v.0.3 cache

Declarative cache

• Other query-libs comparison
• @farfetched/cache
  • cacheQuery
  • memoryCache
  • persistentCache
• dev.to announce
• testing guide
• additional showcase for caching

v.0.4 GraphQL

• Update comparison
• @farfetched/graphql
  • createGraphQLQuery
  • createGraphQLMutation
  • optional auto-batching
  • import, parsing, and usage of schema
  • choose library to handle GraphQL-stuff
  • automatic optimistic update for Query based on Mutation and schema
• dev.to announce
• additional showcase for GraphQL

v.0.5 DevTools

• Project identity
• Design of DevTools
• Choose UI-engine (forest? solid?)
• #42
• dev.to announce
• talk

v.0.6 lists

Paginations and infinite scroll

v.0.7 solid

• @farfetched/solid

v.0.8 triggers

Mark data as stale and re-fetch on some declarative triggers.

• Trigger API
• @farfetched/web-api
  • internet connectivity after lost
  • tab focus
• timers
• explicit triggers
• migrate Mutation API to Triggers API, migrate connectQuery to Triggers API

v.0.9 polling

Get updates from the server

• WebSocket
• Server Sent Events
• HTTP Polling

v.0.10 REST

• createREST — return set of queries and mutations for typical REST API

v.0.11 Suspense

• @farfetched/react
  • Suspense
• dev.to announce
• Showcase with SSR and Suspense

v.1

Stable release 🎉

🤷

Now result events (done.error, done.success, done.skip) have weird naming, I suggest we should rename it.

1. done.success -> end.success
2. done.error -> end.failure
3. done.skip -> end.skip
4. done.finally -> end.finally

I've though about a lot about APIs for compatibility with chainRoute from Atomic Router. It's some conclusions 👇

Query is useless as a chainRoute source

Farfetched build around the idea of queries connection, it assumes that to load all information of a page it will be necessary to execute plenty of queries. So, simple chainRoute({ route, ...query.compatibleProtocol }) works only for single query, which is impossible in the real application.

Query as Effect note

It means, cast Query to Effect has no sense to make Farfecthed compatible with Atomic Router.

cc @sergeysova

Query Chain

As I discovered, single query has no sense as source of chainQuery (or other method to postpone routing until data fetching). So, we need to subscribe router on the whole chain of queries. It means, data fetching with first queries should be started after some start event, and some all-done-event should be fired after all queries successfully loads.

For example 👇

const profileQuery = createQuery()
const settingsQuery = createQuery()
const privacyQuery = createQuery()

connectQuery({ source: profileQuery, target: [settingsQuery, privacyQuery] })

In this application, profile page should be opened only after all three queries successfully done. It can be represented with something like this:

// It is not API proposal, just example

const profileLoadedRoute = chainRoute({
  route: profileRoute,
  beforeOpen: {
    effect: queryChain({ startAt: profileQuery }),
  },
});

In this case, queryChain could return Effect. Start of the Effect will start the first Query in the chain, .done-event will represent successfully load of profileQuery, settingsQuery and privacyQuery.

Further steps

We have to collect some real-world feedback and feedback from Effector Committee before implementing this proposal.

Now any Query accepts Contract as a validation abstraction to check response. Contract is a completely static structure, it has to be defined in write-time (when developer writs code). In some cases, it is not enough for validation.

Case

In this application, we have to make two queries and join results for displaying all data 👇

const infoQuery = createQuery<void, { id: number, name: string}, unkown>()
const contentQuery = createQuery<number, { [id]: { image: string } }, unkonw>()

connectQuery({
  source: { info: infoQuery },
  fn: ({ info }) {
    return info.id;
  },
  target: contentQuery
})

const $compundData = combine(
  { info: infoQuery.$data, content: contentQuery.$data }
  ({ info, content }) => ({...info, ...content[info.id]})
)

As we can see, contentQuery returns dictionary. What if the dictionary does not include required id? It is impossible to validate it with static Contract.

Proposal

I purpose to extend factories config with the shared field new overload:

validate: TwoArgsSourcedField<Data, Params, null | string[], ValidationSource>

where:

• Data is data from the Query, it is already validated against Contract
• Params is Query parameters
• null | string[] is a result of validation, in case of null response is valid, and otherwise strings will be passed as validation errors
• ValidationSource is any external Store, which can be used in validation process

This field won't change Query result type, so it will be pretty easy to add it as a optional parameter.

Further steps

Let's postpone this proposal until initial public release.

After effector/effector@31b7b55 effector/babel-plugin supports @farfetched/core as a factory by default. After its release, we have to add a note to documentation.

Also, it would be nice to add @farfetched/core as a factory by default to @effector/swc-plugin and add note as well.

It would be really nice to provide ESM-build as well 🚀

Operator to alter data in Query based on :

• params of the Mutation to perform optimistic update
• result of the Mutation to perform pessimistic update

In some cases, it's important to reset the whole state of the Query.

It is necessary to use some code transformation tools in SSR and tests, so we have to write an instruction about it and add link to https://farfetched.pages.dev/recipes/ssr.html and https://farfetched.pages.dev/recipes/testing.html

➡️ https://farfetched.pages.dev/recipes/sids.html

Now we are using @nrwl/js:library to generate new package, but after it's execution it's necessary to make some edits in the generated project:

1. add publish to targets:

"publish": {
  "executor": "@nrwl/workspace:run-commands",
  "options": {
    "command": "node tools/scripts/publish.mjs core"
  },
  "dependsOn": [
    {
      "projects": "self",
      "target": "build"
    }
  ]
}

2. add typetest to targets:

"typetest": {
  "executor": "./tools/executors/tsd:tsd",
  "inputs": ["{projectRoot}/**/*.type_spec.ts", "{projectRoot}/**/*.ts"]
}

3. add size to targets:

"size": {
  "executor": "./tools/executors/size-limit:size-limit",
  "options": {
    "limit": "15 kB",
    "outputPath": "dist/packages/core"
  },
  "dependsOn": [
     {
      "projects": "self",
      "target": "build"
    }
  ]
}

4. mark test-utils as implicitDependencies.

It would be nice to hide all this work under custom generator.

Following this comment.

This is not an issue, but I was just curious about this use case, and how it could potentially be solved with farfetched.

For example, we have page with 5 widgets, each one fetches data, so, we have 5 data requests. All data available only for authenticated user. User logs into application, doing their things, then went to have lunch. After an hour they come back and wants to check data on those widgets. Clicks on a navigation link, page is opened, 5 requests are sent, but OH NO, authentication session has been expired, and all 5 requests have received error 401.

There was a requirement in one of my previous projects, so this is not some fancy user case out of nowhere. And the requirement was following:

• All 5 widgets continue to show "loading state"
• Sign-in form pops up, like modal window in overlay
• When user types their username and password and hits enter, sign-in form closes
• And all 5 widgets shows actual data

So, we need to postpone all failed requests somewhere in a internal queue, without showing errors to user, and without notifying widgets, that their requests has received responses (so they will continue to show "loading state" like still waiting for response). Then, after session successfully renewed — take all postponed failed requests out of the internal queue and retry them all. Upon receiving successful responses widgets will show data. Like nothing happen. From each widget's point of view it was just a loooooong answer for the request. Widgets have no idea, that requests were actually failed and retried.

I'm pretty sure this could be implemented differently, but this is like I did this, literally. It was long ago, I was young, and it was angular.js though :) Maybe you will suggest different approach to implement this kind of requirement.

It could be other cases, like with tokens. For example:
Request receives error 401 in response. This means access token has been expired. We took refresh token, and requests new access token. Upon receiving new access token — retry failed request. All automated and without letting UI to even know there was some trouble.

I've noticed that in real applications, custom solutions for .$failed and .$succeed are used as frequently as .$pending.

We have to add warnings about current API stability to the website.

• https://farfetched.pages.dev/api/

PR in Vitepress

This API will provide a declarative way to set up reties for particular Query or Mutation.

Use case

HTTP request could fail by many reasons:

• not a good time for a client — e.g. user could go through underpass and lost their connectivity
• not a good time for a server — e.g. temporary lost connectivity between services on server

Restrictions

Check on particular error

Requests with some errors could be retried, some could not.

The API have to provide a way to distinguish particular error before starting retry.

Dynamic timeout

If the error was caused by server problems, it could be dangerous to retry the request immediately — it could lead to "internal DDoS".

The API have to provide a way to define retry interval dynamically as a Sourced field.

Dynamic number of retries

Same as timeout.

Parameters modification

In some systems, it is important to tell the server that the current request is a retry.

The API have to provide a way to modify Query/Mutation parameters on every retry.

API proposal

Based on the provided use cases and restrictions, I purpose to add new operator retry with the following API:

function retry({
  source: 
    | Query<Params, ..., Error>
    | Mutation<Params, ..., Error>
    | Array<Query<Params, ..., Error> | Mutation<Params, ..., Error>>,
  filter: (error: Error) => boolean,
  timeout: TwoArgsSourcedField<Params, Meta /* retry number, etc. */, number, ExternalTimeoutSource>,
  retries: Sourced<Params, number, ExternalRetriesSource>,
  mapParams?: TwoArgsSourcedField<Params, Meta /* retry number, etc. */, number, ExternalMappingSource>,
})

Usage example

retry({
  source: locationQuery,
  filter: isNetworkError,
  timeout: (params, { retryNumber }) => retryNumber * 1000,
  retries: 10,
})

A bunch of sample-s and createEffect-s (from Effector) in @farfetched/solid lead to memory leak. We have to create it inside withRegion and clean all connections by clearNode on Solid's onCleanup hook.

It should mirror Effector's createStore serialize option, it will be passed as is.

Release-action publishes all packages to http://npmjs.com/. It would be nice to create GitHub releases and GitHub package registry.

Now connectQuery({source, target}) creates connection (sic!) between source and target. It could be useful in cases then you need to perform few api calls. For example, I want to receive userId first and download user avatar next. It works fine for user profile page but causes extra query in other pages where avatar is not needed. For such precise control, I need to use enable field.

I want to show another approach:

const userAvatarQuery = connectQuery(source: userIdQuery, target: avatarQuery)

Here if I want to receive user avatar I need to explicitly call userAvatarQuery. So we don't create a connection between source and target in the application graph (we did indeed, but in the form of userAvatarQuery).

userAvatarQuery knows information about parents (source and target) and call them on own start. Ofc we can skip calling userIdQuery if it was already called.

For me, such api looks more clear. Also, it's easier to control a bunch of dependent api's.

const userEmailQuery = connectQuery(source: userIdQuery, target: emailQuery);
const userAvatarQuery = connectQuery(source: userIdQuery, target: avatarQuery);
const userInfoFromEmailQuery = connectQuery(source: userEmailQuery, target: infoFromEmailQuery);

Here we can call userInfoFromEmailQuery and all parent tree will be called step by step. This is how we can call a long chain of dependent queries.

Now, to correct SSR, users have to set up some import overrides on bundler level:

• effector-react -> effector-react/scope
• effector-solid -> effector-solid/scope

Why 'on bundler level'?

Built-in effector/babel-plugin can do it, but it is impossible to use in our case. In common setup, babel/swc transforms only application code and does not touch node_modules. However, @farfetched/solid and @farfetched/react has imports from effector-solid/effector-react and now it should be replaced. Only bundler has access to these imports. So, it leads to poor DX.

Solution

effector/effector#765
effector/effector#766

Query is an entity for receiving data from the remote source.
Mutation in an entity for sending data to the remote source.

Use case

Sometimes, it is necessary to send some command to the server and possibly change remote state:

• user login
• entity CRUD
• etc.

So

We have to introduce some abstraction for this kind of operations.

After first public release, we have to apply to DocSearch Open-Source program and add search to the website.

http://docsearch.algolia.com/

It is impossible now, because Algolia requires repository to be public.

Our roadmap is an issue now, I assume, it would be better to has it as a page on the website.

There are no reasons to make it mandatory actually 🤗

Sometimes it is important to know that all retry attempts are failed and run some logic because of it. I suppose, we can add field fallback?: Event<SomeMeta> | Effect<SomeMeta, any, ant> to retry operator to cover this case.

We have to install real package from built (with link or something like this) in some common envs (Vite, CRA, etc.) to ensure that modules resolve correctly.

Blocks #90 🚨

Now, it's impossible to pass single Query as source in connectQuery, it is required to wrap in the object. We can introduce simplified form.

Add a piece of documentation about testing — babel-plugin, jest setup, etc.

Now result events (done.error, done.success, done.skip) do not provide information about parameters of Query, we have to change its API:

1. done.success: Event<Data> -> Event<{ params: Params, data: Data}>
2. done.error: Event<Error> -> Event<{ params: Params, error: Error }>
3. done.skip: Event<void> -> Event<{ params: Params }>
4. done.finally: Event<void> -> Event<{ params: Params }>

On this early stage of library development, I do not think we should add *Data analogues of these events without parameters. Let's add it if real users will request it.

Let's talk about typical usage of Query in the UI 👇

// It is abtract UI-lib, both Solid and React should be supported
function UserInfo() {
  const [user, { error }] = useQuery(userQuery)

  if (error && isInvalidDataError(error)) return <p>API returnas invalid data</p>
  else if (error && isNetworkError(error)) return <p>No internet, sorry<p>
  else if (error) return <p>Something went wrong</p>

  return <p>Data: {data}</p>
}

The problem

It looks like a lot of boilerplate code in common cases. As I notices, many developers can skip precise error handling due to poor DX.

Solution

I purpose to introduce components ShowError to handle it in a declarative boilerplate-free way:

function UserInfo() {
  const [user, { error }] = useQuery(userQuery)

  if (error) return (
    <ShowError error={error}>
      <ErrorCase is={isInvalidDataError}>{({ message }) => `Invalid data: ${message}`}</ErrorCase>
      <ErrorCase is={isNetworkError}>{({ code }) => `Netword error: ${code}`}</ErrorCase>
      <ErrorCase otherwise>Something went wrong</ErrorCase>
    </ShowError>
  )

  return <p>Data: {data}</p>
}

Further steps

We have to collect some real-world feedback and feedback from Effector Committee before implementing this proposal.

Provide a complementary package for zod to use as contract applier.

Currently, 18 node is not supported by Cloudflare:

#101
cloudflare/pages-build-image#1

So we need to wait Cloudflare image update, and after we can add .nvmrc file

We can create ESLint plugin for Farfetched to enforce the best practices.

It will be great to create a standalone static site that collects changelogs for all packages and renders it as a page. Something like https://changelog.effector.dev