Reduce google translation API calls about matters-server HOT 5 CLOSED

Since the article won't be changed after published, I think long-time (months or years) cache or even directly store translated versions in the database may good choices.

Personally, I more prefer the latter:

1. The memory of ElasticCache eaten by static contents is a waste;
2. ElasticCache might be flushed at any time;

AWS S3 is also an option, but it may take more cost on implementation and billing (I/O, Network, Storage, etc.).

from matters-server.

For Caching Strategies, I think Lazy Loading is good for us, which means old & new articles don't need to be prefetched:

1. Return translated data if exists in DB
2. Retrieve translated data from Google Translate and store it in DB, then return

from matters-server.

For Caching Strategies, I think Lazy Loading is good for us, which means old & new articles don't need to be prefetched

Agreed. objectCache directive is already using a simple lazy loading strategy: it tries to get data from cache, and if cache misses, it gets from original source, which is Google translation, and then stored in cache.

The delay in SSR query we saw yesterday were from a high peak in cache misses, which result in hitting translation rate limit. But it looks like the cache has been filled quite fast and most frequent viewed articles are cached now.

Personally, I more prefer the latter:

The memory of ElasticCache eaten by static contents is a waste;
ElasticCache might be flushed at any time;
AWS S3 is also an option, but it may take more cost on implementation and billing (I/O, Network, Storage, etc.).

Yes, the additional implementation cost is a good point. We will always need lazy loading cache strategy, and might want to expand its usage, so that we can guarantee a fast return for frequent viewed content.

If we decide to use a month or years long cache, maybe there is no need of disk storage, since Google translation call will be reduced a lot. However this way we will need to implement better cache purge or update mechanism, and handle rate limit issue (for example by returning null first) in case we need to update cache schemas 🤔

from matters-server.

Another thought is that, since we need to query language for each article every time, we can store language in database first.

from matters-server.

Implemented with #1149

from matters-server.