Giter Club home page Giter Club logo

strapi-middleware-cache's Introduction

Strapi LRU Caching middleware - Legacy

A cache middleware for the headless CMS strapi.io

NO LONGER MAINTAINED

THIS PROJECT IS DEPRECATED

Since the release of Strapi v4, this project has been parked in favor of the new Strapi Plugin Rest Cache which was forked from this repository. Maintainers have now moved there, and we recommend you switch to the new and improved plugin. A big thank you to all those that contributed to this middleware, it has served many.

TOC

Strapi V4 users: https://strapi-community.github.io/strapi-plugin-rest-cache/

How it works

This middleware caches incoming GET requests on the strapi API, based on query params and model ID. The cache is automatically busted everytime a PUT, PATCH, POST, or DELETE request comes in.

Supported storage engines

  • Memory (default)
  • Redis

Important: Caching must be explicitely enabled per model

Installing

Using npm

npm install --save strapi-middleware-cache

Using yarn

yarn add strapi-middleware-cache

Version 1 compatibility

โš ๏ธ Important: The middleware has gone through a full rewrite since version 1, and its configuration may not be fully compatible with the old v1. Make sure to (re)read the documentation below on how to use it ๐Ÿ‘‡

Requirements

Since 2.0.1:

  • strapi 3.4.0
  • node 14

See 1.5.0 for strapi < 3.4.0

Setup

For Strapi stable versions, add a middleware.js file within your config folder

e.g

touch config/middleware.js

To use different settings per environment, see the Strapi docs for environments.

You can parse environment variables for the config here as well if you wish to, please see the Strapi docs for environment variables.

Enable the cache middleware by adding the following snippet to an empty middleware file or simply add in the settings from the below example:

module.exports = ({ env }) => ({
  settings: {
    cache: {
      enabled: true,
    },
  },
});

Starting the CMS should now log the following

$ strapi develop
[2021-02-26T07:03:18.981Z] info [cache] Mounting LRU cache middleware
[2021-02-26T07:03:18.982Z] info [cache] Storage engine: mem

Configure models

The middleware will only cache models which have been explicitely enabled. Add a list of models to enable to the module's configuration object.

e.g

// config/middleware.js
module.exports = ({ env }) => ({
  settings: {
    cache: {
      enabled: true,
      models: ['review'],
    },
  },
});

Starting the CMS should now log the following

$ strapi develop
[2021-02-26T07:03:18.981Z] info [cache] Mounting LRU cache middleware
[2021-02-26T07:03:18.982Z] info [cache] Storage engine: mem
[2021-02-26T07:03:18.985Z] debug [cache] Register review routes middlewares
[2021-02-26T07:03:18.986Z] debug [cache] POST /reviews purge
[2021-02-26T07:03:18.987Z] debug [cache] DELETE /reviews/:id purge
[2021-02-26T07:03:18.987Z] debug [cache] PUT /reviews/:id purge
[2021-02-26T07:03:18.987Z] debug [cache] GET /reviews recv maxAge=3600000
[2021-02-26T07:03:18.988Z] debug [cache] GET /reviews/:id recv maxAge=3600000
[2021-02-26T07:03:18.988Z] debug [cache] GET /reviews/count recv maxAge=3600000

Configure the storage engine

The module's configuration object supports the following properties

Example

With Redis Sentinels

// config/middleware.js

/**
 * @typedef {import('strapi-middleware-cache').UserMiddlewareCacheConfig} UserMiddlewareCacheConfig
 */

module.exports = ({ env }) => ({
  settings: {
    /**
     * @type {UserMiddlewareCacheConfig}
     */
    cache: {
      enabled: true,
      type: 'redis',
      models: ['review'],
      redisConfig: {
        sentinels: [
          { host: '192.168.10.41', port: 26379 },
          { host: '192.168.10.42', port: 26379 },
          { host: '192.168.10.43', port: 26379 },
        ],
        name: 'redis-primary',
      },
    },
  },
});

With Redis Cluster

// config/middleware.js

/**
 * @typedef {import('strapi-middleware-cache').UserMiddlewareCacheConfig} UserMiddlewareCacheConfig
 */

module.exports = ({ env }) => ({
  settings: {
    /**
     * @type {UserMiddlewareCacheConfig}
     */
    cache: {
      enabled: true,
      type: 'redis',
      models: ['review'],
      redisConfig: {
        cluster: [
          { host: '192.168.10.41', port: 6379 },
          { host: '192.168.10.42', port: 6379 },
          { host: '192.168.10.43', port: 6379 },
        ],
      },
    },
  },
});

Running the CMS will output the following

$ strapi develop
[2021-02-26T07:03:18.981Z] info [cache] Mounting LRU cache middleware
[2021-02-26T07:03:18.982Z] info [cache] Storage engine: mem
[2021-02-26T07:03:18.985Z] debug [cache] Register review routes middlewares
[2021-02-26T07:03:18.986Z] debug [cache] POST /reviews purge
[2021-02-26T07:03:18.987Z] debug [cache] DELETE /reviews/:id purge
[2021-02-26T07:03:18.987Z] debug [cache] PUT /reviews/:id purge
[2021-02-26T07:03:18.987Z] debug [cache] GET /reviews recv maxAge=3600000
[2021-02-26T07:03:18.988Z] debug [cache] GET /reviews/:id recv maxAge=3600000
[2021-02-26T07:03:18.988Z] debug [cache] GET /reviews/count recv maxAge=3600000

Per-Model Configuration

Each route can hold its own configuration object for more granular control. This can be done simply by replacing the model strings in the list by object.

On which you can set:

  • Its own custom maxAge
  • Headers to include in the cache-key (e.g the body may differ depending on the language requested)

e.g

// config/middleware.js

/**
 * @typedef {import('strapi-middleware-cache').UserModelCacheConfig} UserModelCacheConfig
 */

module.exports = ({ env }) => ({
  settings: {
    cache: {
      enabled: true,
      type: 'redis',
      maxAge: 3600000,
      models: [
        /**
         * @type {UserModelCacheConfig}
         */
        {
          model: 'reviews',
          headers: ['accept-language']
          maxAge: 1000000,
          routes: [
            '/reviews/:slug',
            '/reviews/:id/custom-route',
            { path: '/reviews/:slug', method: 'DELETE' },
          ]
        },
      ]
    }
  }
});

Running the CMS should now show the following

$ strapi develop
[2021-02-26T07:03:18.981Z] info [cache] Mounting LRU cache middleware
[2021-02-26T07:03:18.982Z] info [cache] Storage engine: mem
[2021-02-26T07:03:18.985Z] debug [cache] Register review routes middlewares
[2021-02-26T07:03:18.986Z] debug [cache] POST /reviews purge
[2021-02-26T07:03:18.987Z] debug [cache] DELETE /reviews/:id purge
[2021-02-26T07:03:18.987Z] debug [cache] PUT /reviews/:id purge
[2021-02-26T07:03:18.987Z] debug [cache] GET /reviews recv maxAge=1000000 vary=accept-language
[2021-02-26T07:03:18.988Z] debug [cache] GET /reviews/:id recv maxAge=1000000 vary=accept-language
[2021-02-26T07:03:18.988Z] debug [cache] GET /reviews/count recv maxAge=1000000 vary=accept-language
[2021-02-26T07:03:18.990Z] debug [cache] GET /reviews/:slug maxAge=1000000 vary=accept-language
[2021-02-26T07:03:18.990Z] debug [cache] GET /reviews/:id/custom-route maxAge=1000000 vary=accept-language
[2021-02-26T07:03:18.990Z] debug [cache] DELETE /reviews/:slug purge

Single types

By default, the middleware assumes that the specified models are collections. Meaning that having 'post' or 'posts' in your configuration will result in the /posts/* being cached. Pluralization is applied in order to match the Strapi generated endpoints.

That behaviour is however not desired for single types such as homepage which should remain singular in the endpoint (/homepage)

You can mark a specific model as being a single type by using the singleType boolean field on model configurations

e.g

// config/middleware.js
module.exports = ({ env }) => ({
  settings: {
    cache: {
      enabled: true,
      models: [
        {
          model: 'footer',
          singleType: true,
        },
      ],
    },
  },
});

Running the CMS should now show the following

$ strapi develop
[2021-02-26T07:03:18.981Z] info [cache] Mounting LRU cache middleware
[2021-02-26T07:03:18.982Z] info [cache] Storage engine: mem
[2021-02-26T07:03:18.985Z] debug [cache] Register review routes middlewares
[2021-02-26T07:03:18.986Z] debug [cache] PUT /footer purge
[2021-02-26T07:03:18.987Z] debug [cache] DELETE /footer purge
[2021-02-26T07:03:18.987Z] debug [cache] GET /review recv maxAge=3600000

Authentication

By default, cache is not looked up if Authorization or Cookie header are present. To dissable this behaviour add hitpass: false to the model cache configuration

You can customize event further with a function hitpass: (ctx) => true where ctx is the koa context of the request. Keep in mind that this function is executed before every recv requests.

e.g

// config/middleware.js
module.exports = ({ env }) => ({
  settings: {
    cache: {
      enabled: true,
      models: [
        {
          model: 'footer',
          hitpass: false,
          singleType: true,
        },
      ],
    },
  },
});

Etag support

By setting the enableEtagSupport to true, the middleware will automatically create an Etag for each payload it caches.

Further requests sent with the If-None-Match header will be returned a 304 Not Modified status if the content for that url has not changed.

Clearing related cache

By setting the clearRelatedCache to true, the middleware will inspect the Strapi models before a cache clearing operation to locate models that have relations with the queried model so that their cache is also cleared (this clears the whole cache for the related models). The inspection is performed by looking for direct relations between models and also by doing a deep dive in components, looking for relations to the queried model there too.

Cache entry point

Koa Context

By setting the withKoaContext configuration to true, the middleware will extend the Koa Context with an entry point which can be used to clear the cache from within controllers

// config/middleware.js
module.exports = ({ env }) => ({
  settings: {
    cache: {
      enabled: true,
      withKoaContext: true,
      models: ['post'],
    },
  },
});

// controller

module.exports = {
  async index(ctx) {
    ctx.middleware.cache; // A direct access to the Cache API
  },
};

IMPORTANT: We do not recommend using this unless truly necessary. It is disabled by default as it goes against the non-intrusive/transparent nature of this middleware.

Strapi Middleware

By setting the withStrapiMiddleware configuration to true, the middleware will extend the Strapi middleware object with an entry point which can be used to clear the cache from anywhere (e.g., inside a Model's lifecycle hook where ctx is not available).

// config/middleware.js
module.exports = ({ env }) => ({
  settings: {
    cache: {
      enabled: true,
      withStrapiMiddleware: true,
      models: ['post'],
    },
  },
});

// model

module.exports = {
  lifecycles: {
    async beforeUpdate(params, data) {
      strapi.middleware.cache; // A direct access to the Cache API
    },
  },
};

IMPORTANT: We do not recommend using this unless truly necessary. It is disabled by default as it goes against the non-intrusive/transparent nature of this middleware.

Cache API

/**
 * @typedef {import('strapi-middleware-cache').CacheStore} CacheStore
 * @typedef {import('strapi-middleware-cache').MiddlewareCacheConfig} MiddlewareCacheConfig
 * @typedef {import('strapi-middleware-cache').ModelCacheConfig} ModelCacheConfig
 * @typedef {import('strapi-middleware-cache').CustomRoute} CustomRoute
 */

const cache = {
  /**
   * @type {CacheStore}
   */
  store,

  /**
   * @type {MiddlewareCacheConfig}
   */
  options,

  /**
   * Clear cache with uri parameters
   *
   * @param {ModelCacheConfig} cacheConf
   * @param {{ [key: string]: string; }=} params
   */
  clearCache,

  /**
   * Get related ModelCacheConfig
   *
   * @param {string} model
   * @param {string=} plugin
   * @returns {ModelCacheConfig=}
   */
  getCacheConfig,

  /**
   * Get related ModelCacheConfig with an uid
   *
   * uid:
   * - application::sport.sport
   * - plugins::users-permissions.user
   *
   * @param {string} uid
   * @returns {ModelCacheConfig=}
   */
  getCacheConfigByUid,

  /**
   * Get models uid that is related to a ModelCacheConfig
   *
   * @param {ModelCacheConfig} cacheConf The model used to find related caches to purge
   * @return {string[]} Array of related models uid
   */
  getRelatedModelsUid,

  /**
   * Get regexs to match all ModelCacheConfig keys with given params
   *
   * @param {ModelCacheConfig} cacheConf
   * @param {{ [key: string]: string; }=} params
   * @param {boolean=} wildcard
   * @returns {RegExp[]}
   */
  getCacheConfRegExp,

  /**
   * Get regexs to match CustomRoute keys with given params
   *
   * @param {CustomRoute} route
   * @param {{ [key: string]: string; }=} params
   * @param {boolean=} wildcard
   * @returns {RegExp[]}
   */
  getRouteRegExp,
};

Admin panel interactions

The strapi admin panel uses a separate rest api to apply changes to records, e.g /content-manager/explorer/application::post.post the middleware will also watch for write operations on that endpoint and bust the cache accordingly

strapi-middleware-cache's People

Contributors

bartzy avatar benjenkyn avatar blefevre avatar dependabot[bot] avatar derrickmehaffy avatar joacimastrom avatar meck93 avatar patrixr avatar rubenmkindapp avatar rubms avatar sgarcia-g avatar stafyniaksacha avatar tomekzar avatar v1nn1k avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

strapi-middleware-cache's Issues

Cant seem to be caching, no logs to show if its caching

Hi,

First all! Thanks a lot for the lib. I have been looking for one and I hope the official strapi team starts looking at this ๐Ÿ‘ .

I am trying to get it working on my project. I have set up the middleware and add the model.

module.exports = ({ env }) => ({
  settings: {
    cache: {
      enabled: true,
      models: ['ratings']
    },
  },
});

I don't see the message.
image

Anything that I am doing wrong? Following is the debug info:
image

Please let me know if you need further info and grateful if you can help :)

Cheers,
Jeremy

Changes to bust function introduced in v1.0.9 break admin cache busting

When editing a collection type in the admin cache busting isn't working anymore because the pattern variable in the bust function isn't using pluralize since v1.0.9, which makes it so that the pattern isn't matching the cache keys anymore. Everything works fine for single types though, since the pluralize function is avoided in this context (as it should).

v1.0.8:

const pattern     = new RegExp(`^/${pluralize(model)}`);

v1.0.9

const pattern     = new RegExp(`^/${model}`);

Using a bit of your code from function bustAdmin, I'm able for figure out if the pattern should be pluralized or not (This works for both single types and collection types):

const bust = (model) => async (ctx, next) => {
  const modelCfg = _.find(toCache, (cfg) => {
    return cfg.model === (cfg.singleType ? model : pluralize(model));
  });

  const pattern     = new RegExp(`^/${modelCfg.model}`);
  const keys        = await cache.keys() || [];

  await next();

  if (!_.inRange(ctx.status, 200, 300)) return;

  await Promise.all(
    _.filter(keys, k => pattern.test(k))
      .map(k => cache.del(k))
  );
}

Strapi-middleware-cache is not updating/busting the Redis cache accordingly on Strapi v3.0.6

Description: Strapi-middleware-cache is not updating/busting the Redis cache accordingly on Strapi v3.0.6.

Issue: We are using Strapi-middleware-cache plugin and updated our middleware.js accordingly.
We are using Redis storage for caching collections. We want to bust/flush & update the cache accordingly for each update/create/delete any collection's entries.

But we found the Redis cache is getting bust/flush only with Redis server restart (each time we had to manually restart Redis).

We are following this documentation: https://strapi.io/blog/caching-in-strapi-strapi-middleware-cache

Please see our middleware.js code below:

module.exports = ({ env }) => ({
  settings: {
    cache: {
      enabled: true,
      type: "redis",
      maxAge: 3600000,
      logs: true,
      populateContext: true,
      populateStrapiMiddleware: true,
      enableEtagSupport: true,
      models: ["cars", "tests"],
      redisConfig: {
        host: "127.0.0.1",
        port: 6379,
      },
    },
  },
});

And we updated the cars controller accordingly on this file: api/cars/controllers/cars.js

module.exports = {
  async index(ctx) {
    console.log("Inside cars controller--------");
    ctx.middleware.cache.store;
    await ctx.middleware.cache.bust({ model: "cars" });
  },
};

Actual Issue: Cache is not getting flush/bust automatically on create/update/delete operations. We found the Redis cache is getting bust/flush only with Redis server restart, each time when any entry is update/deleted/created, we had to manually restart Redis.

Expected: We want to bust/flush & update the cache accordingly for each update/create/delete operations for any collection's entries. We don't need to restart Redis server to flush/update the cache.

System:
OS: MAC OS Catalina 10.15.7
Strapi: Strapi v3.0.6
strapi-middleware-cache: "^1.5.0"
redis_version: v6.2.5

Add option to bypass serving cache

Hi!

I was wondering if you would consider adding an option to bypass serving the cache entry when matched in the cache store. I'm working on implementing front-end cache using the ETag and If-None-Match headers. I've already added that layer to Strapi and in addition I'd like to leverage the cache store from this middleware by using the cache entry to create the ETag instead of querying the database every time.

Cache not evicted when publishing or unpublishing

Reproduction Steps:

  1. Create a new item e.g. a new page under pages
  2. Get the list of pages e.g. /pages, the new unpublished item is missing as expected
  3. Publish the new page
  4. Get the list of pages /pages, the newly published item is missing

The same is true when changing state from published to unpublished. The cache is not evicted and as a result the item is still visible in the list eventhough it is not published

The workaround for this issue is to publish and then modify in order to evict the cache

Cacheconf doesn't match with config

The config I've setup in config/middleware.js does not match the config returned by getCacheConfig

I have this model setup in my config:

{
  model: "contracts",
  hitpass: false,
  injectDefaultRoutes: false,
  routes: [
    {
      path: "/contracts/:contractType/:countryLocale",
      method: "GET",
      paramNames: ["contractType", "countryLocale"],
    },
  ],
}

Calling getCacheConfig("contracts") returns the following, notice how the paramNames does not match

{
  singleType: false,
  hitpass: false,
  injectDefaultRoutes: false,
  headers: [],
  maxAge: 3600000,
  model: 'contracts',
  routes: [
    {
      path: '/contracts/:contractType/:countryLocale',
      method: 'GET',
      paramNames: [ 'contractType' ]
    }
}

Bust cache from Models - Lifecycle hooks

I have a custom API which get's the data from different content types. The inbuilt mechanism doesn't automatically refreshes the cache on custom routes if the content type data is changed. As per documents ctx is required to bust the cache.

await ctx.middleware.cache.bust({ model: 'posts' });

But ctx is not available in models. How can I manually bust the cache from models life cycle hooks like beforeDelete

Caching doesn't seem to be working

Hey!

Firstly, many thanks for this very useful package! ๐Ÿ‘

I've added the following code in at config/middleware.js

module.exports = ({ env }) => ({
    settings: {
      cache: {
        enabled: true,
        models: ['grid'], 
      }
    }
  })

I can see the mounting and the using storage engine logs but I have no log from the caching routes.
I also tried with grids, nothing happens.

Here are the logs:

$ strapi develop
[2020-11-23T20:27:40.633Z] debug [Cache] Mounting LRU cache middleware
[2020-11-23T20:27:40.634Z] debug [Cache] Storage engine: mem

Missing this log :
debug [Cache] Caching route /grids/* [maxAge=3600000]

Any idea?

Many thanks :)

Is it possible to set maxAge forever?

I have a MongoDB server holding data which I don't change much. Most queries are to read. Is it possible to set maxAge for lru cache forever, thus the lru cache will never expire by itself?

how to set related models

strapi: 3.6.6
strapi-middleware-cache: 2.1.0

i have city and building models, i want to clear building cache when i update or delete city model.
in building/models/building/settings.json, i have set {attributes: {city: {model: "city"}}}
in middleware.js, i have set {clearRelatedCache: true}, it's not clear building cache when i edit or delete city model.

document said (this clears the whole cache for the related models), is there any way to set related models?

same key used when using _where query param

Environment:

  • node: v16.13.0
  • npm: 7.17.0
  • storage engine: redis

Hi, I'am having a request from the front with _where query param but in the cache its stored like object and all the _where requests will be the same.

When I have investigated the redis instance found in key _where[object Object]
I guess you are missing toString() to evaluate the _where object otherwise all the requests that uses _where will be the same for different objects.

Manage Redis disconnections and re-connections

Hello ๐Ÿ‘‹

We are using the library and so far is working great, but we came up with a possible scenario: If the connection with Redis is lost for whatever reason (maybe the cache goes down, or the connection pipe gets broken), then the library will try to fetch data from Redis and will get stuck forever (which is far from ideal). It would be great if the library could handle this case.

We think that this could be avoided by doing two things:

  1. Change the status flag to use the status property of the ioredis' client.
  2. Including a cache timeout, just in case.

We have some work in progress in a fork, so I will open a draft PR, but I would like to hear your thoughts on this one.

Keep up the good work with the library! :D

GraphQL support

I'm using Strapi with the GraphQL plugin. Is there any chance it will get supported? GraphQL is used with POST.

let me know how to delete or update cache for the model

The cache doesn't return the updated data.
I used the default memory cache and following is my cache config in/config/middleware.js
cache: {
enabled: true,
enableEtagSupport: true,
enableXCacheHeaders: true,
withKoaContext: true,
withStrapiMiddleware: true,
clearRelatedCache: true,
maxAge: 3.154e10,
models: [
{
model: 'sqore-posts',
hitpass: false,
singleType: true,
routes: ['/sqore-posts/:slug', '/sqore-posts'],
},
],
},

After updating the content, updated data is not from API and only old cache data when calling /sqore-posts/:slug.

So I tried to clear or update the cache of model "sqore-posts" in the sqore-posts model.
but I don't know how to clear or update the cache.
I used this code in /models/sqore-posts.js.

afterUpdate: async (entry) => {
    console.log('---------------sqore-posts afterUpdate')
    const cacheConf = await strapi.middleware.cache.getCacheConfig('sqore-posts')
    await strapi.middleware.cache.clearCache(cacheConf)
},

I am looking for your help.
Thanks.

Key collision when using the same database across installs

I am running half-a-dozen Strapi apps off a DigitalOcean droplet with Redis locally installed for caching purposes. I noticed today that two of those sites were showing each other's Strapi data in their front-end client applications. I quickly narrowed this down to an issue with key collisions in Redis. I was able to resolve the issue by using the redisConfig option to specify a db for each Strapi app. But I thought I would call it to your attention anyway in case this was an issue you were not aware of and might have an idea of how to resolve without needing to manually specify a different Redis database for each application. Thanks!

Rewrite tests for v2

Due the re-write of of the middleware to version 2, the old tests had to be disabled

The latest working versions of the tests is here

Here's an overview

 Caching
    Collection types
      โœ“ should let uncached requests go through
      โœ“ following request should be cached
      โœ“ following request should have response status 304
      โœ“ caches based on headers
      โœ“ doesnt cache models not present in the config
      โœ“ busts the cache on a POST resquest
      โœ“ busts the cache of a related model (via direct relation) on a POST resquest
      โœ“ busts the cache of a related model (via relation in component) on a POST resquest
      โœ“ busts the cache on an admin panel POST resquest
      โœ“ busts the cache on an admin panel POST publishing change
      โœ“ busts the cache on a PUT resquest
      โœ“ busts the cache of a related model (via direct relation) on a PUT resquest
      โœ“ busts the cache of a related model (via relation in component) on a PUT resquest
      โœ“ busts the cache on an admin panel PUT resquest
      โœ“ busts the cache on an admin panel PUT publishing change
      โœ“ busts the cache on a DELETE resquest
      โœ“ busts the cache of a related model (via direct relation) on a DELETE resquest
      โœ“ busts the cache of a related model (via relation in component) on a DELETE resquest
      โœ“ busts the cache on an admin panel DELETE resquest
      โœ“ busts the cache on an admin panel DELETE publishing change
      when an ID is specified on a POST request
        โœ“ doesn't bust the cache for other IDs
      when an ID is specified on a PUT request
        โœ“ doesn't bust the cache for other IDs
      when an ID is specified on a DELETE request
        โœ“ doesn't bust the cache for other IDs
    Single types
      โœ“ caches non pluralized endpoints
      โœ“ doesnt cache pluralized endpoints
      โœ“ busts the cache on a POST resquest
      โœ“ busts the cache on an admin panel POST resquest
      โœ“ busts the cache on a PUT resquest
      โœ“ busts the cache on an admin panel PUT resquest
      โœ“ busts the cache on a DELETE resquest
      โœ“ busts the cache on an admin panel DELETE resquest

Ideally the tests would be an end-to-end setup which spins up an actual Strapi instance to test against.

cc @stafyniaksacha

Once the model is cached, auth is neglected

Hi,

I integrated the strapi middleware cache to my project and added the model like categories. Below is my cache settings in middleware.js under config
cache: {
enabled: true,
type: 'redis',
maxAge: 3600000,
models: [
{
model: 'Categories',
maxAge: 1000000,
}
],
populateStrapiMiddleware: true,
logs:true
}
First time when GET /Categories api is invoked it asked for bearer token. Next time when it is cached authentication is totally ignored. If I remove auth header no forbidden message is shown instead data is returned giving access to anybody with api. Am I missing any steps? How to overcome this problem so that cache middleware is not invoked before auth is verified.

Modify settings to support custom object

So in looking at your code, it would appear you are specifically pulling out config options that may not always apply.

Looking through the ioredis config options (and what I am currently testing) is what is known as the Redis HA config or Sentinel system: https://github.com/luin/ioredis#sentinel

While it's a tad difficult to see in this example screenshot:

image

Basically there are 3 servers, running 6 services:

Server 1: Redis Server + Sentinel (Master)
Server 2: Redis Server + Sentinel (Slave)
Server 3: Redis Server + Sentinel (Slave)

When configuring the ioredis for the cache, we would simply pass the Sentinel addresses like:

var redis = new Redis({
  sentinels: [
    { host: "192.168.10.41", port: 26379 },
    { host: "192.168.10.42", port: 26379 },
    { host: "192.168.10.43", port: 26379 },
  ],
  name: "redis-primary",
});

And thus there would be no root host, port, pass, ect and would be contained within the sentinels array. Likewise for the name as sentinels can manage multiple Redis "clusters" (tad confusing here I know)

I will play around locally and see if I can find a solution to submit a PR unless you already have some ideas. Personally I think just simply having a config option for redisConfig: {} to allow someone to use whatever they want would be better then having the config options in the root of the settings object.

FYI I am also testing on the latest Strapi next tagged release which is what will be 3.0.0-rc.1 when they release it (Release Candidate for the Stable)

Functions not available in lifecycle-methods

Hi, it seems like strapi.middleware.cache is not being initialized as specified in the readme. This is my config:

cache: {
      enabled: true,
      maxAge: 3600000,
      populateStrapiMiddleware: true,
      models: ["contract"]
}

but trying to call functions in lifecycle through strapi.middleware.cache just returns xx is not a function
image

Here is the model-file in question:

module.exports = {
  lifecycles: {
    afterUpdate: async (result) => {
      const { cache } = strapi.middleware
      console.log("Cache: ", cache);
      console.log(cache.getCacheConfig("contract"));

Am I missing something?

Failing Redis connectivity when using DEV Redis

Description: Failing Redis connectivity when using DEV Redis

Steps to reproduce:

  1. Install this Library on the Strapi app
  2. To connect this library to Redis - Use Per-Model configuration mentioned on strapi-middleware-cache NPM docs
  3. Try to connect Strapi app to any DEV instance of Redis server (stop your local Redis server on your machine)
  4. Run the app
  5. Throws below error:
error [Cache] Redis connection failed
[2021-05-14T16:38:20.740Z] error Error: connect ECONNREFUSED 127.0.0.1:6379

So though my Strapi app is connected to DEV Redis but this middleware is still trying to connect to local Redis (127.0.0.1:6379) by default and since my local Redis is stopped, it throws above connectivity error.

We are using below Per-Model Configuration mentioned on https://www.npmjs.com/package/strapi-middleware-cache

module.exports = ({ env }) => ({
  settings: {
    cache: {
      enabled: true,
      type: 'redis',
      maxAge: 3600000,
      models: [
          'cars',
          'articles',
          'test',
          'trucks'
        ]
    }
  }
});

Notes: This middleware is working fine when Strapi app is connected to local Redis server. Issue is only when Strapi app is connected to DEV Redis

Expected Behavior: This middleware should work fine though we use Local/DEV/UAT/PROD Redis server's
Please let us know what configuration we need to pass to this middleware to use the correct Redis what Strapi is connected to?

Issues with busting related cache

Hi!

I'm having issues with cache for related models not being busted.
Workflow is as follows:

  1. Query /regions/germany, /regions/germany?& is added to cache
  2. Change other model which has one-to-one relation with regions through strapi-admin, which should bust the cache
  3. Relations cache is busted with wildcard as we don't know params for related entries.

Issue is that in https://github.com/patrixr/strapi-middleware-cache/blob/master/lib/utils/getRouteRegExp.js#L27

  // wildcard: clear all routes
  if (wildcard) {
    let pattern = route.path;
    for (const paramName of route.paramNames) {
      pattern = pattern.replace(`:${paramName}`, '([^/]+)');
    }

    return [new RegExp(`^${pattern}\\?`)];
  }

pattern.replace(':${paramName}', '([^/]+)'); should be changed to pattern.replace(paramName, '([^/]+)'); as paramName already contains :.

Or we can map through paramNames and remove : and later add it when replacing.

I prepared PR with the needed fix #77 @patrixr

And many thanks for the great lib, it helped us to decrease loading times significantly!

Faulty logic in param matching for getRouteRegExp

The matching of params when creating the regex is breaking since params extracted from the route will have format [":param1", ":param2"] while the ones added wont have the : since they are likely used as:

getCacheConfRegExp(cacheConf, {
  param1,
  param2,
});

Even if you explicitly name the params

getCacheConfRegExp(cacheConf, {
  ":param1": param1,
  ":param2": param2,
});

the pattern-replace will fail in the loop a few lines down since it then tries to match ::param1 which doesn't exist in the route

pattern = pattern.replace(`:${paramName}`, params[paramName]);

How to connect to RedisLabs

Hello, thank you kindly for putting together this package. I'm very new to redis and I'm having a hard time connect to the redis instance I set up on Redislabs

My /config/middleware.js files looks like this (some values have been changed):

  settings: {
    cache: {
      enabled: true,
      type: 'redis',
      maxAge: 3600000,
      models: ['posts'],
      redisConfig: {
        url: 'redis://redis-5555.c8.us-east-1-4.ec2.cloud.redislabs.com:5555',
        name: 'content',
        password: 'secret_password',
      },
    },
  },
});

But I keep getting the following error:

Error: connect ECONNREFUSED 127.0.0.1:6379 at TCPConnectWrap.afterConnect [as oncomplete]

Using the admin does not purge the cache

Example:

[2022-04-03T11:17:27.434Z] info [cache] Mounting LRU cache middleware
[2022-04-03T11:17:27.435Z] info [cache] Storage engine: mem
[2022-04-03T11:17:27.441Z] debug [cache] Register dealers routes middlewares
[2022-04-03T11:17:27.442Z] warn [cache] POST /dealers has no matching endpoint, skipping
[2022-04-03T11:17:27.443Z] warn [cache] DELETE /dealers/:id has no matching endpoint, skipping
[2022-04-03T11:17:27.443Z] debug [cache] PUT /dealers/:id purge
[2022-04-03T11:17:27.445Z] debug [cache] GET /dealers recv maxAge=3600000
[2022-04-03T11:17:27.445Z] debug [cache] GET /dealers/:id recv maxAge=3600000
[2022-04-03T11:17:27.474Z] debug [cache] Register admin routes middlewares
[2022-04-03T11:17:27.474Z] debug [cache] POST /content-manager/single-types/:model/actions/publish purge-admin
[2022-04-03T11:17:27.474Z] debug [cache] POST /content-manager/single-types/:model/actions/unpublish purge-admin
[2022-04-03T11:17:27.474Z] debug [cache] POST /content-manager/collection-types/:model purge-admin
[2022-04-03T11:17:27.475Z] debug [cache] POST /content-manager/collection-types/:model/:id/actions/publish purge-admin
[2022-04-03T11:17:27.475Z] debug [cache] POST /content-manager/collection-types/:model/:id/actions/unpublish purge-admin
[2022-04-03T11:17:27.475Z] debug [cache] POST /content-manager/collection-types/:model/actions/bulkDelete purge-admin
[2022-04-03T11:17:27.476Z] debug [cache] PUT /content-manager/single-types/:model purge-admin
[2022-04-03T11:17:27.476Z] debug [cache] PUT /content-manager/collection-types/:model/:id purge-admin
[2022-04-03T11:17:27.476Z] debug [cache] DELETE /content-manager/single-types/:model purge-admin
[2022-04-03T11:17:27.476Z] debug [cache] DELETE /content-manager/collection-types/:model/:id purge-admin

[2022-04-03T11:19:13.438Z] debug [cache] GET /dealers MISS
[2022-04-03T11:19:13.441Z] debug GET /dealers (178 ms) 200
[2022-04-03T11:19:20.462Z] debug [cache] GET /dealers HIT
[2022-04-03T11:19:20.465Z] debug GET /dealers (4 ms) 200
[2022-04-03T11:19:23.846Z] debug PUT /dealers/5fbc20181740b2002cf9deb4 (326 ms) 200
[2022-04-03T11:19:25.357Z] debug [cache] GET /dealers MISS
[2022-04-03T11:19:25.360Z] debug GET /dealers (191 ms) 200
[2022-04-03T11:19:28.698Z] debug GET /content-manager/collection-types/application::dealer.dealer/5fb79f7ebcd3da002c163063 (250 ms) 200
[2022-04-03T11:19:32.228Z] debug PUT /content-manager/collection-types/application::dealer.dealer/5fb79f7ebcd3da002c163063 (480 ms) 200
[2022-04-03T11:19:34.019Z] debug [cache] GET /dealers HIT
[2022-04-03T11:19:34.020Z] debug GET /dealers (2 ms) 200

using PUT does purge the cache, but changing an entry in the UI does not :(
Reverting back to 1.5, it does purge from the UI.

Node: 14.18.0
Strapi: 3.6.0

Api pluralization does use the same logic as Strapi v3

Strapi has some grammatical logic around pluralization for its apis so words like "elsewhere" or "offsite" don't end up having s appended to the end of it. Ideally this app would utilize the same code to pluralize its apis

strapi 4?

Hello!

Is strapi cache compatible with strapi 4?

Bust cache of specific endpoint when PUT/DELETE request with other endpoint

I have a model with 3 fields containing {id(auto-generated), username, email}

I have modified the default GET endpoint to "username" instead of "id" to search specific record based on username like
GET /profiles/someusername

But for PUT/DELETE I'm using "id" as the endpoint PUT/DELETE /profiles/random_generated_id

Whenever I use PUT/DELETE, data changes in the database, the previously cached data of different endpoint is not busting. I'm thinking of extending those PUT/DELETE API and manually deleting the specific endpoint's cache.

Help me get out of this situation.

[cache] the store is not ready when using redis

I'm trying to use it in redis mode but the log keeps saying
warn [cache] the store is not ready
However, I can connect to that redis with redis-cli -a <host> -p <port> -a <password>

here's my config

module.exports = ({env}) => ({
  settings:{
    audit:{
      enabled: true
    },
    cache:{
      enabled: true,
      logs: true,
      maxAge: 3600000,
      clearRelatedCache: true,
      withStrapiMiddleware: true,
      type: "redis",
      models: [...],
      redisConfig:{
        host: host,
        port: port,
        password: password
      }

    }
  }
})

Post endpoints for the related entities don't bust the cache.

Let consider this relation: A Book can have multiple authors
Steps:

  1. My endpoint for fetching the books is HTTP://localhost:1337/books. Data is being cached here. This endpoint also fetches the authors data from the authors collection.
  2. Go to the admin panel and update any author from that book.
  3. Call the get books endpoint, it brings data from the cache but the updated author info is not available.
  4. Go to the admin panel and update any book
  5. Now the cache has been busted.

How can this be achieved? Is there a way to manually bust the cache via another endpoint just like graphql has exposed its playground? Help needed.

PS: I am using strapi with mongo.

clearRelatedCache causes crash when model contains file collection

When clearRelatedCache is enabled and a model contains a field which is a file collection, the following crash occurs:

TypeError: Cannot read property 'kind' of undefined
      at /Users/foo/myproject/node_modules/strapi-middleware-cache/lib/index.js:195:51
      at /Users/foo/myproject/node_modules/lodash/lodash.js:4967:15
      at baseForOwn (/Users/foo/myproject/node_modules/lodash/lodash.js:3032:24)
      at /Users/foo/myproject/node_modules/lodash/lodash.js:4936:18
      at Function.forEach (/Users/foo/myproject/node_modules/lodash/lodash.js:9410:14)
      at clearCache (/Users/foo/myproject/node_modules/strapi-middleware-cache/lib/index.js:194:13)
      at async /Users/foo/myproject/node_modules/strapi-middleware-cache/lib/index.js:248:9
      at async bustAdmin (/Users/foo/myproject/node_modules/strapi-middleware-cache/lib/index.js:318:11)
      at async /Users/foo/myproject/node_modules/strapi/lib/middlewares/responses/index.js:9:9
      at async cors (/Users/foo/myproject/node_modules/@koa/cors/index.js:95:16)
      at async /Users/foo/myproject/node_modules/strapi/lib/middlewares/logger/index.js:56:11
      at async /Users/foo/myproject/node_modules/strapi/lib/middlewares/responseTime/index.js:17:9
      at async /Users/foo/myproject/node_modules/strapi/lib/services/metrics/middleware.js:29:5
      at async /Users/foo/myproject/node_modules/strapi/lib/Strapi.js:325:9

In my project I have a field called imageGallery which looks like this:

    "imageGallery": {
      "collection": "file",
      "via": "related",
      "allowedTypes": [
        "images"
      ],
      "plugin": "upload",
      "required": false,
      "pluginOptions": {}
    }

I think the problem is in the getRelatedModels function, line 102ff:

    if (attr.collection) {
      relatedModels[attr.collection] = models[attr.collection];
    } else if (attr.model && attr.model !== 'file') {
      relatedModels[attr.model] = models[attr.model];
    }

In the first case, when attr.collection is true, I think it should also check that the collection != 'file'

Need help on enableEtagSupport - etag not showing

I got the issue when setting enableEtagSupport to be true but no etag included in the response header
What else do I need to do

below are my settings

cache: { enabled: true, type: 'mem', enableEtagSupport: true, enableXCacheHeaders: true, maxAge: CACHE_AGE_MEDIUM, models: [...], ...}

Override 127.0.0.1 host

Hi,

Is it possible to override the hostname of the strapi-middleware-cache plugin? I am using Redis container in my docker-compose.yml and by default Strapi calls host 127.0.0.1 instead of my container name. I would like replace 127.0.0.1 by redis

Because I get this error:

error [cache] redis connection failed {"errno":-111,"code":"ECONNREFUSED","syscall":"connect","address":"127.0.0.1","port":6379}

I tried to add REDIS_HOST env variable but same problem

Cache not busted from admin anymore

Hi,

With the latest version of Strapi (3.4.1) I noticed the cache isn't busted anymore when saving/updating an entry in the CMS.
I did notice in the source (https://github.com/patrixr/strapi-middleware-cache/blob/master/lib/index.js#L189) that the admin REST endpoint refer to /content-manager/explorer/:scope and that Strapi uses /content-manager/collection-types/:scope.

I'm not fully sure if these routes have changed in the latest version of Strapi, but I think this prevents the busting of the cache as those do not match anymore.

TypeError: Cannot read property 'kind' of undefined

i'm getting 500 errors submitting posts to strapi because this extension seems to be choking on the unique id field

[2021-08-02T03:03:18.800Z] debug GET /content-manager/collection-types/application::posts.posts?page=1&pageSize=20&_sort=date:DESC (97 ms) 200
[2021-08-02T03:03:24.820Z] debug POST /content-manager/uid/generate (88 ms) 200
[2021-08-02T03:03:26.405Z] debug POST /content-manager/uid/generate (48 ms) 200
[2021-08-02T03:03:26.950Z] debug POST /content-manager/uid/check-availability (35 ms) 200

  TypeError: Cannot read property 'kind' of undefined
      at /workspace/node_modules/strapi-middleware-cache/lib/index.js:195:51
      at /workspace/node_modules/lodash/lodash.js:4967:15
      at baseForOwn (/workspace/node_modules/lodash/lodash.js:3032:24)
      at /workspace/node_modules/lodash/lodash.js:4936:18
      at Function.forEach (/workspace/node_modules/lodash/lodash.js:9410:14)
      at clearCache (/workspace/node_modules/strapi-middleware-cache/lib/index.js:194:13)
      at processTicksAndRejections (internal/process/task_queues.js:95:5)
      at async /workspace/node_modules/strapi-middleware-cache/lib/index.js:248:9
      at async bustAdmin (/workspace/node_modules/strapi-middleware-cache/lib/index.js:318:11)
      at async /workspace/node_modules/strapi/lib/middlewares/responses/index.js:9:9
      at async cors (/workspace/node_modules/@koa/cors/index.js:95:16)
      at async /workspace/node_modules/strapi/lib/middlewares/logger/index.js:56:11
      at async /workspace/node_modules/strapi/lib/middlewares/responseTime/index.js:17:9
      at async /workspace/node_modules/strapi/lib/services/metrics/middleware.js:29:5
      at async /workspace/node_modules/strapi/lib/Strapi.js:325:9

[Feature Request] Add support for Redis Clusters

Hello o/

First off thank you for creating this middleware! I have been testing it for the new Strapi rc versions (working great by the way) and I was wondering if you would be interested to add support for the redis-clustr as I believe this will be very helpful for larger installations of Strapi in a cluster and thus allowing all of them to share the cache.

Please do let me know what you think!

[FR] Add option to disable initial logging

While doing some testing on my own project, I noticed the initial logging could quickly get out of hand (for reference I have 90-ish models in my project, and am only caching about 20% of them)

image

I'm thinking it might be better to add some kind of config option to disable the verbosity (or at least decrease it) of the logging.

beta custom path cache issue

I am testing out the beta branch as I have a custom route that is currently being cached forever despite updates.

Use case

My courses/:id/children route computes the children of the current course (id). So it is dependent of any course that are children of this item.

Issue

This custom children route only has GET implemented. So the cache is never cleared despite the underlying children that have been updated.

What I need

I want to bust the cache of the children route upon any of the courses are updated/deleted/posted. In essence I want to just clear all children cache anytime any course is changed. I don't need fine grain control.

Alternatives

Looking at the beta docs, it seems I might be able to call the cache object directly within the model hooks. This seems reasonable. Why is the warning about using this feature there?

https://github.com/patrixr/strapi-middleware-cache/tree/beta#strapi-middleware

Redis connection failing when using this middleware to connect to redis server running on DEV/UAT

Description: Redis connection failing when using a separate redis server

This middleware working fine on local with running local redis server.

But I have a separate redis instance running on DEV/UAT/PROD and i connect to it through a different library (not ioredis)

I have used this below configuration:
// config/middleware.js

module.exports = ({ env }) => ({
  settings: {
    cache: {
      enabled: true,
      type: 'redis',
      maxAge: 3600000,
      models: [
        {
          model: 'listings',
          maxAge: 1000000,
        }
      ]
    }
  }
});

My DEV/UAT/PROD redis has a different type: redis-something.

How can i make this middleware work with a different redis server?

Support Redis Clusters (Distributed/Partitioned)

So in my research and initial testing with clusters: #3

There seems to be an issue with redis-lru in which clusters are failing to set cache values due to how that package handles keys and pipelines. I'm currently looking at alternatives but we could try to reach out the package author about it.

I did see the following issue (from 2018) that seems slightly related: facundoolano/redis-lru#11

Single type has many relation data update doesn't bust the cache

Hello, thanks for this great package! I noticed that if I have a single type model with hasMany relation, updating data in the related model doesn't bust this single model cache.

That's how my relation looks like.
image

I wonder, is there any way to make it work? Or it would require changes in the middleware code itself?

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.