Introduce Redis to handle resources about pasta HOT 3 OPEN

But regarding memory consumption, indexes are not a major issue. It's the pickle files (i.e., commit caches), and the way how python duplicates memory when doing things in parallel. If we want to benefit from databases, then a persistent, non-relational in-memory database (e.g., key-value store) would probably be the best choice. So yes, redis could be a promising candidate.

from pasta.

Indexes are not just for simulating a database. They give a human-readable overview and allow manipulation with standard tooling.

However, I agree that we can use databases for replacing the pickle-hell that we currently have. There might be another student that will evaluate this topic (hopefully soon).

from pasta.

Indexes are not just for simulating a database. They give a human-readable overview and allow manipulation with standard tooling.

A human readable overview could be a view into the database.

The choice of a database will also depend on whether the data that we need to store can fit in memory

from pasta.