The cacheTime option name may be not clear enough. It should be renamed into cacheExpirationTime (old option could be also maintained with a warning to avoid breaking changes, and it could be removed in future major versions)

A github workflow should be created for running mutation tests

The problem

Selectors theorically isolate views about the need to know its own data dependencies, but in some scenarios you want to clean a selector cache. If you use the cleanCache method, only the selector cache is cleaned, and it will be recomputed, but its dependencies will remain cached, so data will not be really refreshed. Then, you need to know which are the dependencies of a selector and clean them manually, which "breaks" the philosophy of the library.

Consider the next example, in which we have defined a selector for requesting books and authors from a REST API, then add an authorName property to the books, and then return the data:

export const booksWithAuthorName = new Selector(
  books, // Axios provider requesting data from a REST API
  authors, // Axios provider requesting data from a REST API
  (booksResponse, authorsResponse) => {
    return booksResponse.map(book => ({
      ...book,
      authorName: authors.find(author => author.id === book.author)
    }));
  }
);

Now, suposse we are using this selector in a React component, and we want to refresh data every ten seconds while the component is rendered.

THIS EXAMPLE DOESN'T WORK. With this code, only the selector cache is cleaned, so data from the selector is recalculated, but the books and authors providers are still cached and requests are not repeated:

import React, { useEffect } from "react";
import { useData } from "@data-provider/react";

import { booksWithAuthorName } from "../data/books";

const Books = () => {
  const books = useData(booksWithAuthorName);

  useEffect(() => {
      const interval = setInterval(() => {
        // Cleans the cache of the selector
        booksWithAuthorName.cleanCache();
      }, 10000);
      clearBooksInterval = () => clearInterval(interval);
    }
    return clearBooksInterval;
  }, [books]);

  return (
    <ul>
      {
        books.map((book) => {
          return (
            <li key={`book-${book.id}`}>{book.title} - {book.authorName}</li>
          );
        });
      }
    </ul>
  );
};

export default Books;

Proposed solution

Add a new cleanDependenciesCache method to Selector. Using it in the previous example, the providers cache would be also cleaned without need to know which are the selector dependencies.

It is desirable to install Fossa in the repository and display a badge in the README file in order to ensure the compliance of dependencies licenses.

It is desirable to add a cacheTime option to the Provider Class. This option may define the maximum time the cache is valid, and, next time the provider is read, it would be ignored.

With this method, the cache is not cleaned automatically under the hood, it will only be ignored when the read method is dispatched and the time has expired. So, for example, if you have a React component connected to a data provider using the @data-provider/react useData hook, the data will not be refreshed until you rerender the component, because the cleanCache event is not being triggered.

Another method could be useful too: cleanCacheInterval.

With this method, the cache is automatically cleaned every defined interval, and the cleanCache will be triggered, so every alive connected component or dependant selector will be automatically read again, and the data will be refreshed without the need to manually dispatch the read method.

The package has a great unit test coverage, but it is desirable to add e2e tests.

Tests should include testing the usage of the library in a node.js package, a front-end application built with Webpack, and a front-end application loading the library in "umd" format. Front-end applications should be tested using Cypress.

Having e2e tests will improve the CI/CD, allowing Greenkeeper to update dependencies automatically in a more secure way.

Currently, the selector interface is different for defining dependencies than for defining the last function, which can be considered the "selector" itself, because it receives all of the results of the dependencies. But dependencies receive also the results from the previous dependencies, but in a different format (in a single argument, as an array of results, while the last function receives them as different arguments).

Well, as the selector function is also able to return another dependency, there is no a big difference between the rest of dependencies, and it may be hard to understand why the last function receives the dependencies results in one format, and the other ones in another. This was not intentional in a first moment, but the evolution of the library, allowing selectors to return another selectors, have taking us to this point.

In order to simplify the interface, and for a better comprehension (I hope), here is a proposal in which there are no distinction between the arguments that all functions in a Selector receive. All functions in a Selector will receive previous results as different arguments, ordered by the order of the dependencies itself.

Note also that the query argument have been moved to the first place to avoid making its position dependant of the number of previous dependencies.

const booksOfAuthor = new Selector(
  (query) => authorsModel.query(query),
  (query, authorData) => books.queries.byAuthorId(authorData.id),
  (query, authorData, booksData) => {
    return booksData.map(book => {
      return {
         ...bookData,
         authorName: authorData.name
      };
    });
  }
);

Sometimes you need to clean all dependencies of a Selector, except some specific ones. (A usual use case is when a dependency of the selector is an origin requesting user data, that could be reused by a lot of other Selectors. Then, you don't wan't to clean that cache, because it will recalculate lots of selectors unnecessarily).

To achieve this, the cleanDependenciesCache method could accept an options argument. That argument could be an object containing an except property, that should be an array containing those origins or selectors that you don't want to be cleaned.

selector.cleanDependenciesCache({
   except: [origin]
})

In some scenarios could be desirable to modify configuration only for certain invocations to the provider methods. (For example, when passing an "idempotency-key" as an http header using "mercury-api" read method)

This options could be passed as a parameter to the "read" method, and should overwrite instance options only for that execution. For other methods, such as "create" or "update", options could be passed as second argument.

It is desirable to allow to cancel the promise of read method. A deeper analysis is required to decide how to implement it, taking into account that addons should be the responsible of cancelating the promises (and it is not possible in every types of origins)

The problem

Currently data-provider has three state properties: data, loading and error. In some scenarios you need to know if the data has been loaded any time, not only if its being loaded right now. This is because you want to show the loading indicator the first time the data is being loaded, but not next times the data is being loaded again because the cache was invalidated, as it may result in undesirables screen "flickerings" in case the component is "alive" while the cache is cleaned.

This problem is described in the data-provider react tutorial, and it is proposed to be solved using a double condition before rendering the loading indicator:

Solution proposed in the data-provider documentation:

import React from "react";
import { useData, useLoading } from "@data-provider/react";

import { todosFiltered, updateTodo } from "../data/todos";
import TodoList from "../components/TodoList";

const FilteredTodoList = ({ showCompleted }) => {
  const todosProvider = todosFiltered.query({ completed: showCompleted });
  const todos = useData(todosProvider);
  const loading = useLoading(todosProvider);

  if (loading && !todos) {
    return <div>Loading...</div>;
  }
  return (
    <TodoList todos={todos} onTodoClick={updateTodo} />
  );
};

export default FilteredTodoList;

Note the expression (loading && !todos) that we are using to handle the loading state. Data Provider informs to us when the todos are being loadded, but we don't want to show the "loading..." indicator every time data is loading. It will be shown only the first time the data is being loaded, while todos collection is empty. Rest of times, the current todos state will be shown until the todos are fetched again, and then replaced. In this way we avoid unwanted flickerings in the UI.

Problem of this solution

Apart that this solution is too much "verbose", as this is a very common scenario, and you'll need to repeat the double condition in a lot of components, it will only work when the data is not empty. If the data returned by the selector or provider is empty, it will result in rendering the loading indicator every time the cache is cleaned, so it will result in undesired flickerings in the UI.

New Proposed solution

To add a new property loaded to the state, which would be set as true the first time the data is loaded, and never modified again (until the "resetState" method is called)

This property should be exposed also by ui bindings addons. For example, in the case of @data-provider/react, a new hook useLoaded could be exposed. A fourth property loaded could also be returned by the current useDataProvider hook.

It is desirable to accept a function as argument of the config method. The provided function should receive current configuration and return the new one.

Selectors are not cleaning instantly its cache when a dependency is cleaned using the force option. They should clean their cache instantly when any of their dependencies is cleaned using the force option.

To Reproduce
Configure the cleanCacheThrottle option in all providers and selectors to 5 seconds. Set an interval to clean a selector cache for each two seconds. The cleanCache method of the selector will be throttled then, so its cache will be cleaned each 5 seconds. Then, call to the cleanCache method of one of the selector's dependencies using the force option. The selector's dependency should also be cleaned instantly, but it remains throttled.

Pnpm and Nx.dev versions should be upgraded

It is desirable to publish a scaffold for creating data-provider addons. It could be done using yeoman, with a npm package named create-data-provider-addon, or both. More research in order to find the best solution is needed.

The problem

When a selector is queried recursively, the resultant query is the extension of all objects used as "query".

const queriedSelector = mySelectorInstance
  .query({foo: "var"})
  .query({foo2:"var2"});

console.log(queriedSelector.queryValue) // -> {foo:"var", foo2: "var2"}

In providers created with the Axios addon, this behavior changes, as the addon defines its own "queryMethod", and the result of applying recurrent queries is the extension of the queryParams and urlParams properties of the query objects, not the extension of the objects itself (it applies almost "deep" extend, instead of "extend"):

const queriedAxiosProvider = mySelectorInstance
  .query({urlParams: { foo:"var"}})
  .query({urlParams: {foo2:"var2"}});

console.log(queriedAxiosProvider.queryValue) // -> {urlParams: {foo:"var", foo2: "var2"}};

This difference in selector and providers query method is a pain point when you want to use a Selector as a proxy of an Axios provider, as you don't have the possibility to use recursive queries:

const proxySelector = new Selector(
  (query) => axiosProvider.query(query),
  (response) => response.results
)

// THIS WORKS ----- Examples using an unique query
axiosProvider
  .query({ queryString: { author: 1, sort: "asc" }})
  .read();
// Request to "/foo-url?author=1&sort=asc", then return response;

proxySelector
  .query({ queryString: { author: 1, sort: "asc" }})
  .read();
// Request to "/foo-url?author=1&sort=asc" (from cache), then return "results" property of the response;


// THIS DOESN'T WORK ----- Examples using recurrent queries
axiosProvider
  .query({ queryString: { author: 1 }})
  .query({ queryString: { sort: "asc" }})
  .read();
// Same request than the first one. Shared cache.

proxySelector
  .query({ queryString: { author: 1 }})
  .query({ queryString: { sort: "asc" }})
  .read();
/* Request to "/foo-url?sort=asc", then return "results" property of the response.
   The first `queryString` property is being overwritten
   because the selectors applies a simple extend for recurrent queries. */

Proposed solution

A getChildQueryMethod option can be added to Selectors configuration (same property name than the exposed in Provider Class for allowing addons to define its own behavior):

const proxySelector = new Selector(
  (query) => axiosProvider.query(query),
  (response) => response.results,
  {
    getChildQueryMethod: (newQuery, currentQuery) => {
      return {
        queryString: {
          ...currentQuery.queryString,
          ...newQuery.queryString,
        },
        urlParams: {
          ...currentQuery.urlParams,
          ...newQuery.urlParams,
        },
      };
    }
  }
);

// Now, all previous examples would work

Currently providers handler does not have a cleanDependenciesCache method. It is desirable to expose the method also for groups of providers.

It is desirable to provide an option to disable the automatic cache clean when an update, create or delete method finish successfully.

Prismic client should be uptated to latest version

There is an error with this repository's Renovate configuration that needs to be fixed. As a precaution, Renovate will stop PRs until it is resolved.

Location: renovate.json
Error type: The renovate configuration file contains some invalid settings
Message: Invalid configuration option: node

Currently headers can be set only through configuration. A deeper analysis is required in order to allow to set them also when calling to the read method, for example.

Currently selectors and origins can be created only instantiating classes directly. Some developers may prefer to use methods instead, so it would be useful to expose a selector method and to define a guideline for addons to do the same, so calling to functions could be used instead of intantiating classes using New.

import { select } from "@data-provider/core";
import { books } from "./books";
import { authors } from "./authors";

export const booksWithAuthorName = select(
  books,
  authors,
  (query, books, authors) => {
     return books.map(book => {
        ...book,
        authorName: authors.find(author => author.id === book.author).name
     });
  }
)

It is desirable to add a useLoadedSuspense hook that should suspense the render of the component until the provider has been loaded.

This issue lists Renovate updates and detected dependencies. Read the Dependency Dashboard docs to learn more.

Awaiting Schedule

These updates are awaiting their schedule. Click on a checkbox to get an update now.

• chore(deps): update dependency fs-extra to v11.1.1

Rate-Limited

These updates are currently rate-limited. Click on a checkbox below to force their creation now.

• chore(deps): update dependency cross-fetch to v3.1.8
• chore(deps): update dependency husky to v8.0.3
• chore(deps): update dependency json-server to v0.17.3
• chore(deps): update dependency prettier to v2.8.8
• chore(deps): update dependency redux to v4.2.1
• chore(deps): update dependency start-server-and-test to v1.15.4
• chore(deps): update dependency typescript to v4.9.5
• chore(deps): update pnpm/action-setup action to v2.2.4
• chore(deps): update babel monorepo (@babel/core, @babel/eslint-parser, @babel/plugin-syntax-flow, @babel/plugin-transform-react-jsx, @babel/preset-env, @babel/preset-react, @babel/preset-typescript)
• chore(deps): update dependency @cypress/webpack-preprocessor to v5.17.1
• chore(deps): update dependency @mocks-server/main to v3.12.0
• chore(deps): update dependency @typescript-eslint/parser to v5.61.0
• chore(deps): update dependency babel-loader to v8.3.0
• chore(deps): update dependency commander to v9.5.0
• chore(deps): update dependency deepmerge to v4.3.1
• chore(deps): update dependency eslint to v8.44.0
• chore(deps): update dependency eslint-config-prettier to v8.8.0
• chore(deps): update dependency lint-staged to v13.2.3
• chore(deps): update dependency react-redux to v8.1.1
• chore(deps): update dependency rollup to v2.79.1
• chore(deps): update dependency serve to v14.2.0
• chore(deps): update dependency sinon to v15.2.0
• chore(deps): update dependency webpack to v5.88.1
• chore(deps): update stryker-js monorepo to v6.4.2 (@stryker-mutator/core, @stryker-mutator/jest-runner)
• fix(deps): update dependency axios to v1.4.0
• fix(deps): update dependency axios-retry to v3.5.1
• chore(deps): update andstor/file-existence-action action to v2
• chore(deps): update dependency @mocks-server/admin-api-paths to v4
• chore(deps): update dependency @mocks-server/admin-api-paths to v5
• chore(deps): update dependency @mocks-server/main to v4
• chore(deps): update dependency @rollup/plugin-babel to v6
• chore(deps): update dependency @rollup/plugin-commonjs to v23
• chore(deps): update dependency @rollup/plugin-commonjs to v24
• chore(deps): update dependency @rollup/plugin-commonjs to v25
• chore(deps): update dependency @rollup/plugin-json to v5
• chore(deps): update dependency @rollup/plugin-json to v6
• chore(deps): update dependency @rollup/plugin-node-resolve to v14
• chore(deps): update dependency @rollup/plugin-node-resolve to v15
• chore(deps): update dependency @testing-library/cypress to v9
• chore(deps): update dependency @testing-library/react to v13
• chore(deps): update dependency @testing-library/react to v14
• chore(deps): update dependency babel-loader to v9
• chore(deps): update dependency babel-plugin-module-resolver to v5
• chore(deps): update dependency commander to v10
• chore(deps): update dependency commander to v11
• chore(deps): update dependency cross-fetch to v4
• chore(deps): update dependency cypress to v10
• chore(deps): update dependency cypress to v11
• chore(deps): update dependency cypress to v12
• chore(deps): update dependency cypress-fail-fast to v7
• chore(deps): update dependency prettier to v3
• chore(deps): update dependency rollup to v3
• chore(deps): update dependency start-server-and-test to v2
• chore(deps): update dependency typescript to v5
• chore(deps): update dependency wait-on to v7
• chore(deps): update jest monorepo to v28 (major) (babel-jest, jest)
• chore(deps): update jest monorepo to v29 (major) (babel-jest, jest)
• chore(deps): update nrwl monorepo to v14 (major) (@nrwl/cli, @nrwl/eslint-plugin-nx, @nrwl/tao, @nrwl/workspace)
• chore(deps): update nrwl monorepo to v15 (major) (@nrwl/cli, @nrwl/eslint-plugin-nx, @nrwl/tao, @nrwl/workspace)
• chore(deps): update nrwl monorepo to v16 (major) (@nrwl/eslint-plugin-nx, @nrwl/tao, @nrwl/workspace)
• chore(deps): update nrwl/nx-set-shas action to v3
• chore(deps): update stryker-js monorepo to v7 (major) (@stryker-mutator/core, @stryker-mutator/jest-runner)
• fix(deps): update dependency prismic-javascript to v3
• fix(deps): update dependency query-string to v8
• 🔐 Create all rate-limited PRs at once 🔐

Open

These updates have all been created already. Click a checkbox below to force a retry/rebase of any.

• chore(deps): update nrwl monorepo to v13.10.6 (@nrwl/cli, @nrwl/eslint-plugin-nx, @nrwl/tao, @nrwl/workspace)

Detected dependencies

• Check this box to trigger a request for Renovate to run again on this repository

Describe the bug
When having an error response, such as 400, 401 or 500, the error received in the catch, or in the component, has only the response body.

To Reproduce
In this selector for example

export const mySuperSelector = new Selector(
  {
    source: myAwesomeOrigin,
    query: query => query,
    catch: e => {
	  // e is only response body error, and you can't access to status or other properties
      return false;
    }
  },
  data => data,
  false
);

Expected behavior
You should be able to access full response error object.

It is desirable to provide a method to clean current value, loading and error "states".

It should be available in all different "instances", queried or not, and should be available also in the "instances" object, for cleaning all states at a time.

E2E tests should be added to the example React application

When the cleanCacheThrottle option is set to a provider, options of the cleanCache and cleanDependenciesCache methods are not being passed to throttled invocations.

E2E tests should be added to ensure the stability of the integration with Prismic API.

Hello, I need to manage Axios interceptor with this library, is there any way to do that?
Thanks

Hey friend,

Thank you for publishing this great module

I've worked with React-Admin before, and i search a lot for something that manages the api calls and cache

Now i thing i found it

But it needs to improve more

Some of the most important features that your plugin needed, are:

1. Cache invalidate timeout
2. Relation resolver
3. Basic CRUD methods support (Optional custom methods)

E2E tests should be added

It is desirable to add more details about current resource state when a change or changeAny event is emitted.
A new data property could be added to the emitted object in the case of changeAny event. This property should contain current values of loading, error and value.

For change events, that currently are emitting only the method name, this should be replaced by an object containing method and data properties, but this change implies a breaking change in the event interface, and this fact has to be taken into account before making the change ¿Does it really improve the interface enough to assume the risk of a breaking change? Maybe it has to implemented in the next major version.

Description

When dependencies of a Selector are taking some time to load, and the cache of one of them is cleaned with more frequency than the time that all dependencies take to be resolved, then the Selector never resolves the data, it remains "loading" forever.

For example, if dependencies A,B,C are declared in sequency and each one take 2 second to be resolved, the Selector should be resolved in 6 seconds. But, if you execute cleanCache on the the A dependency each 4 seconds (as in a theorical polling scenario), then the Selector never resolves.

Cause:

Selectors are waiting for the new data of a dependency when its cleanCache method is called. They actively listen to the cleanCache method of all of its dependencies. If any of them cleans the cache, then it stops reading dependencies and start again in order to have the latest data available.

Possible solution:

A restartReadOnCleanedDependenciesMaxTime option could be added to the selectors, so, when it reaches this max time, it will not stop reading the rest of dependencies. It could have a default value of 5 seconds, as this is a very unusual scenario.

E2E tests should be added to the example of implementing origin addons

The option cleanCacheThrottle should also affect to the cleanDependenciesCache method.

It is desirable to expose a "setData" method, so addons could update the data in the state directly. Doing this, addons could implement an "optimistic" behavior, for example.

The expirationTime option is not needed since the core provides the cacheTime one.

Current behaviour

Now, we can only filter by "document.type", but we have come across the situation where we have different content based on the same custom type. It this case, it would be nice if we can filter directly by tag, instead of doing it by querying by document.type and then filtering by tag in client side.

Implementation ideas
In _prismicQuery method we should add a new option to filter by tag.

As a cleanCacheThrottle option was recently added (issue #140), it is desirable to add a force option to the cleanCache method, for those scenarios in which the throttle should be ignored because the cache has to be cleaned instantly.

The method could accept options as an object, so the forced invocation could look like:

provider.cleanCache({ force: true });

If the cache is forced to be cleaned, the throttle time should be reseted and any pending invocation should be cancelled.

It is desirable to implement some method to allow addons to define its own base tag. That tag should be added to all addon instances, and each instance could continue adding its own tags as now.

All selectors should have a default tag named selector in order to allow to configure them together.

Hello there!

My name is Ana. I noted that you use the mutation testing tool strykerjs in the project.
I am a postdoctoral researcher at the University of Seville (Spain), and my colleagues and I are studying how mutation testing tools are used in practice. With this aim in mind, we have analysed over 3,500 public GitHub repositories using mutation testing tools, including yours! This work has recently been published in a journal paper available at https://link.springer.com/content/pdf/10.1007/s10664-022-10177-8.pdf.

To complete this study, we are asking for your help to understand better how mutation testing is used in practice, please! We would be extremely grateful if you could contribute to this study by answering a brief survey of 21 simple questions (no more than 6 minutes). This is the link to the questionnaire https://forms.gle/FvXNrimWAsJYC1zB9.

Drop me an e-mail if you have any questions or comments ([email protected]). Thank you very much in advance!!

Travis CI is no longer free: https://blog.travis-ci.com/2020-11-02-travis-ci-new-billing, so builds has to be migrated to a new CI, for instance GitHub Actions.

Sometimes the caches are cleaned for many different reasons in a short period of time, which could result in calls to the read method too much close in the time, maybe even milliseconds.
This can happen, for example, when a same provider is a dependency of many selectors whose method cleanDependenciesCache is called at a time.

It is desirable to add options allowing to set a debounce to the cleanCache method. Based on the Lodash debounce features, it should have the trailing and leading options set as true, so the first invocation would not be delayed, and, if more invocations are received before the waiting time ends, it would result in a new invocation when the wait time finishes, so no "refreshes" would be lost.

The new option could be named as cleanCacheDebounceTime, and it should also affect to the cleanDependenciesCache method.