Serialization to disk about salsa HOT 7 OPEN

A very wise observation from: rust-lang/rfcs#1317 (comment)

In a strictly on-demand setting (IDE, not a compiler), serialization to disk creates more problems than it solves.

from salsa.

In a strictly on-demand setting (IDE, not a compiler), serialization to disk creates more problems than it solves.

Note that some popular IDEs like Visual Studio actually use a disk database. VS migrated a while ago from a custom format to a SQLite database: https://devblogs.microsoft.com/cppblog/introducing-c-experimental-editor-tools/.

from salsa.

Hi! This would be a desirable feature for me. Is this being worked on?

Not trying to rush you, just trying to evaluate how suitable this library is for my use-case. Thank you. :)

from salsa.

from salsa.

I think serialization should be generally opt-in:

• maybe at salsa::database level:
  rather coarse, without lazy loading or transparent spilling, useful for "whole session" store/load and short-term-running scenarios
• or even per query:
  fine-grained, with lazy loading and maybe transparent spilling, useful to reduce RAM usage in long-term-running scenarios

I think I already have a kind of usage scenario ("scenario" as in "salsa is currently not used, but I investigate potential usages") in zs-filecrawler.

QueryGroup 1: 
  file_content(filepath) <-- hash_data(filepath)
  ^-[maybe lazy input]      --> association [filepath -> hash_of_file_data]

QueryGroup 2:
  hash2file(hash)    <-- call_hook(hash)
  ^-[input, from QG1]   --> implicit association [hash -> done(hook return value)]

from salsa.

A similar, but different feature is to allow to transparently spill rarely used values to disk.

IntelliJ relies on similar feature heavily: when you open a multi-million line project with lots of dependencies, indices become really huge.

Note that this is a significantly different setup from rustc, which operates on a crate at a time, and has a reasonable natural cap on the amount of data it must process simultaneously.

from salsa.

Thank you

from salsa.