Limit cache size about librespot HOT 14 CLOSED

Comment by plietar
Friday Jul 29, 2016 at 20:04 GMT

This is definitely something I'd like to have, but requires quite a bit of work.
At the moment, caching is just a matter of placing the file in the cache directory, and trying to open it when loading.

This requires some bookkeeping to keep track of what files are in the cache, their size, along with an eviction algorithm.

These could be used as inspiration, although librespot doesn't need something as sophisticated :
https://wiki.mozilla.org/Necko/Cache
https://www.chromium.org/developers/design-documents/network-stack/disk-cache

from librespot.

Comment by mfonville
Tuesday Nov 29, 2016 at 14:16 GMT

For the moment, users that face issues with it, they could make a simple cron-job that checks the atime of files tp clean-up.

from librespot.

Comment by juriansluiman
Wednesday Feb 22, 2017 at 13:22 GMT

@mfonville how is this possible? If I run find /tmp/librespot/files/* -atime +2 -exec rm {} \; it crashes librespot. The only way I can get it to work is to stop the librespot service, clear up the files and start it again. If I do this, the music will stop when playing, so it doesn't seem to be the most ideal solution yet :)

from librespot.

Comment by plietar
Wednesday Feb 22, 2017 at 14:33 GMT

@juriansluiman Deleting files while librespot is running should work.
Can you run librespot with RUST_BACKTRACE=1 and --verbose and post the log please ? What platform are you running on ?

from librespot.

Comment by juriansluiman
Wednesday Feb 22, 2017 at 15:10 GMT

@plietar sorry, you get forget my comment. I was having another issue with librespot so it stopped working, but just tried again and it does continue to play. My bad!

from librespot.

Comment by michaelherger
Tuesday May 02, 2017 at 04:34 GMT

I've modified librespot to disable the audio file-cache. See pull request #181.

from librespot.

Any progress on this @sashahilton00 ?

from librespot.

Wondering the same. I just discovered that librespot is taking up 41.8GB of data.

That is a lot for something that should just stream audio!

from librespot.

For a simple algorithm, would it make sense to evict the oldest cached files, either by creation or modification date, when the cache grows beyond a certain size?
As long some code gets added to fix this issue, I'm not sure it needs to be optimal or perfect.

It's definitely better than having a library be responsible for taking up any available diskspace on a long enough timescale.

from librespot.

Nope, no progress. We did discuss it a little in another issue, I can't remember which one exactly, will check when I get home. Ideally we would separate the cache logic out, as there are a few use cases/requirements, eg. Limiting cache by size and/or date, or disable the cache all together.

I can appreciate that in the meantime though it would be better to have something rather than nothing, so if someone does want to throw together a PR that implements a basic cache eviction function, we'll look at getting it merged. If no one wants to, I'll try and give it a go at a later date, but currently I have limited time, and there are a couple of other features that take priority imo, such as retrieving files from the CDN or session reconnection.

from librespot.

I have an old branch with a simple LRU based file cache implementation with a prescribed limit, can hunt things up if someone would like to work on it..

from librespot.

I'm completely new to this project, but I'm happy to take a stab if someone has a partial implementation.

from librespot.

I have deployed the following systemd units:

# /etc/systemd/system/spotifyd-cache-clean.timer
[Unit]
Description=Daily clean audio file cache of spotifyd/librespot

[Timer]
OnCalendar=4:00
Persistent=true

[Install]
WantedBy=timers.target

# /etc/systemd/system/spotifyd-cache-clean.service
[Unit]
Description=Clean audio file cache of spotifyd/librespot

[Service]
User=spotifyd
ExecStart=/usr/bin/find /var/cache/spotifyd/files -type f -atime +30 -print -delete

Obviously depends on systemd and the filesystem correctly tracking access times,
then deletes all cached files not accessed for the last 30 days

from librespot.

While that's a good temporary hack for some, it's not really a solution.

Firstly, because it isn't cross-platform (even if it were to be included with librespot, which exists where systemd does not) and secondly, because it's not a POSIX-compatible invocation of find.

It's also not okay for libraries or programs to take up arbitrary disk space without ways of cleaning up after themselves.

from librespot.