New install doesn't update if dir2cast is added after the media files about dir2cast HOT 27 CLOSED

Thanks. I understand your situation a lot better now. I really appreciate you taking the time to add details to the bug report.

I will come up with a solution for this.

from dir2cast.

@gityopr or @helmut72 if you'd care to try the #58 to see if it fixes your problem? if so, i'll merge this and add it to the official release

from dir2cast.

Looks good to me. Adding, deleting, adding again, deleting again, using touch before adding a 3rd time... all behaves like expected. Great, thanks!

from dir2cast.

Can you give me the output of ls -l ? it updates based on the modification timestamps of the file, so that needs to be newer than dir2cast

from dir2cast.

Thanks for the quick reply!

pi@rpi:~/podcast-server/kdbookecast $ ls -l
total 2164536
-rw-rw-r-- 1 pi pi      1203 Dec 21 18:34  502.html
-rwxr--r-- 1 pi pi   3767269 Dec 21 18:44 'A Full Life by Paulo Bacigalupi.m4a'
-rw-r--r-- 1 pi pi    295773 Dec 21 18:39 'Carl Sagan - The Demon-Haunted World (Cary Elwes, Seth MacFarlane).jpg'
-rwxrw-r-- 1 pi pi 498811479 Dec 21 18:34 'Carl Sagan - The Demon-Haunted World (Cary Elwes, Seth MacFarlane).mp3'
-rwxrw-r-- 1 pi pi 544397046 Dec 21 18:35  colorless-tsukuru-tazaki-and.mp3
-rw-rw-r-- 1 pi pi        96 Dec 21 18:34  composer.json
-rwxrwxr-x 1 pi pi      8908 Dec 21 18:35  dir2cast.ini
-rw-rw-r-- 1 pi pi     66709 Dec 21 18:34  dir2cast.php
drwxrwxr-x 2 pi pi      4096 Dec 21 18:34  getID3
-rwx-w---- 1 pi pi 406827669 Dec 21 18:35 'How Beautiful We Were.mp3'
-rwxrw-r-- 1 pi pi     22074 Dec 21 18:35  image.jpg
-rw-rw-r-- 1 pi pi       780 Dec 21 18:35  index.html
-rw-r--r-- 1 pi pi     31116 Dec 21 18:39 'Klara and the Sun.jpg'
-rwxrw-r-- 1 pi pi 587253078 Dec 21 18:36 'Klara and the Sun.m4a'
-rw-r--r-- 1 pi pi    115349 Dec 21 18:39 'Love Is an Ex-Country.jpg'
-rwxrw-r-- 1 pi pi 174817435 Dec 21 18:36 'Love Is an Ex-Country.m4a'
drwxr-xr-x 2 pi pi      4096 Dec 21 18:39  temp

I touch'd "A Full Life by Paulo Bacigalupi.m4a" since that was the one I was testing. That caused the script to pick it up and it appeared in my client apps shortly after. However, pasting a new file which has a newer modification date into the directory didn't update it.

pi@rpi:~/podcast-server/kdbookecast $ ls -la
total 2171220
drwxrwxr-x  4 pi pi      4096 Dec 22 08:19  .
drwxr-xr-x 14 pi pi      4096 Dec 20 22:20  ..
-rw-rw-r--  1 pi pi      1203 Dec 21 18:34  502.html
-rwxr--r--  1 pi pi   3767269 Dec 22 08:13 'A Full Life by Paulo Bacigalupi.m4a'
-rw-r--r--  1 pi pi   6824161 Dec 22 08:19 'Alien (Preview).mp3'
-rw-r--r--  1 pi pi    295773 Dec 21 18:39 'Carl Sagan - The Demon-Haunted World (Cary Elwes, Seth MacFarlane).jpg'
-rwxrw-r--  1 pi pi 498811479 Dec 21 18:34 'Carl Sagan - The Demon-Haunted World (Cary Elwes, Seth MacFarlane).mp3'
-rwxrw-r--  1 pi pi 544397046 Dec 21 18:35  colorless-tsukuru-tazaki-and.mp3
-rw-rw-r--  1 pi pi        96 Dec 21 18:34  composer.json
-rwxrwxr-x  1 pi pi      8908 Dec 21 18:35  dir2cast.ini
-rw-rw-r--  1 pi pi     66709 Dec 21 18:34  dir2cast.php
-rwxrw-r--  1 pi pi      6148 Dec 21 18:39  .DS_Store
drwxrwxr-x  2 pi pi      4096 Dec 21 18:34  getID3
-rwx-w----  1 pi pi 406827669 Dec 21 18:35 'How Beautiful We Were.mp3'
-rwxrw-r--  1 pi pi     22074 Dec 21 18:35  image.jpg
-rw-rw-r--  1 pi pi       780 Dec 21 18:35  index.html
-rw-r--r--  1 pi pi     31116 Dec 21 18:39 'Klara and the Sun.jpg'
-rwxrw-r--  1 pi pi 587253078 Dec 21 18:36 'Klara and the Sun.m4a'
-rw-r--r--  1 pi pi    115349 Dec 21 18:39 'Love Is an Ex-Country.jpg'
-rwxrw-r--  1 pi pi 174817435 Dec 21 18:36 'Love Is an Ex-Country.m4a'
drwxr-xr-x  2 pi pi      4096 Dec 22 08:14  temp

This is the situation with the folder now, the server lists "A Full Life....m4a" but not "Alien (Preview).mp3". Here's the output of ls -l inside the temp folder:

pi@rpi:~/podcast-server/kdbookecast $ ls -l temp/
total 40
-rw-r--r-- 1 pi pi  268 Dec 21 18:39 '1205de5de8d7ad1fc2ac4079e146bfc1__Carl Sagan - The Demon-Haunted World (Cary Elwes, Seth MacFarlane).mp3__data'
-rw-r--r-- 1 pi pi  261 Dec 21 18:39 '29fb0b475b5c5980f6afcd5e08ce3f22__Klara and the Sun.m4a__data'
-rw-r--r-- 1 pi pi  104 Dec 22 08:14 '513726ad29a49b5aa9324880afb8c615__A Full Life by Paulo Bacigalupi.m4a__data'
-rw-r--r-- 1 pi pi  109 Dec 21 18:39 '93ec60a54c683cd5d310584ff36cc430__How Beautiful We Were.mp3__data'
-rw-r--r-- 1 pi pi 1241 Dec 21 18:39 '95822048accbb8e847e963dd005f5508__Love Is an Ex-Country.m4a__data'
-rw-r--r-- 1 pi pi   77 Dec 22 08:14 'ae5150c0a97439f7a4327499bf2aea36__._A Full Life by Paulo Bacigalupi.m4a__data'
-rw-r--r-- 1 pi pi 9793 Dec 22 08:27  bf49df1dd863708a9101bb03671c5556__config_www.xml
-rw-r--r-- 1 pi pi  313 Dec 21 18:39  cfb467a3fad5b73a1e9808c74ac9b5ed__colorless-tsukuru-tazaki-and.mp3__data

Thanks for your help!

Edit: Thought the code button would make it prettier, fixed that!

from dir2cast.

Try touching all of them in sequence, as they're mostly older than dir2cast. I wonder if copying them is preserving the date from the source folder? Then wait. Few mins before adding more

from dir2cast.

I think i see a separate bug, it shouldn't be including the . file for A Full Life

the config_www.xml temp file (i assume that's the feed itself) looks like it contains everything, is it just being browser cached? do you need to do a hard refresh in the browser?

dir2cast is designed to wait a bit after you update so that it isn't adding broken MP3s to the feed (as you copy them in), but waiting 5 mins solves that - or you can fiddle with that in the config

from dir2cast.

basically if that .xml temp file updates (and contains all the files) then the problem is probably caching in your browser.

the following two settings apply to your use case

; *** THESE SHOULD BE LEFT AS THEY ARE - unless you have a good reason. ***

; Number of seconds for which the cache file is guaranteed valid. You can 
; only specify this in the dir2cast.ini that is in the same folder as 
; dir2cast.php. Defaults to 5
;MIN_CACHE_TIME = 5

; Time-to-live (Expiry time) of the feed
; This defaults to 60 minutes
;TTL = 60

a new feed is valid of a minimum of 5 seconds before it will even try to regenerate. Also, it declares the feed as being current for an hour so feed readers may not refresh it for an hour unless you force them to. This includes HTTP caches and browsers, potentially also nginx depending how it's configured. (tacking ?saidsa junk on the end usually bypasses this if you want to test.)

Anyway, you can reduce these if you want.

there is also a URL ?force=password where you can make it refresh anyway, that's if the cache files are definitely not right.

; The password to use on the 'force=<password>' part of the URL in order to 
; force the RSS feed to be regenerated (bypassing and replacing the cached 
; copy). If this is empty then this feature is disabled. 
;FORCE_PASSWORD =

from dir2cast.

Thanks, touching all the media files and confirming the .xml in that folder did the trick. I knew it would be something silly! Thank you.

from dir2cast.

Still technically a bug though (I didn't intend for it not to work properly if you copied dir2cast in AFTER your media files),, so I will try to improve the experience here!

from dir2cast.

Yeah something is definitely up with the stock swag docker container and dir2cast. Tried spinning up a new webserver on a Debian VM instead of a Raspberry Pi. I also waited about an hour to add files to the directory.

The server works great but dir2cast needs that reset link or clearing the temp folder to update. Does it rely on some PHP extensions or anything like that?

from dir2cast.

Yeah something is definitely up with the stock swag docker container and dir2cast. Tried spinning up a new webserver on a Debian VM instead of a Raspberry Pi. I also waited about an hour to add files to the directory.

The server works great but dir2cast needs that reset link or clearing the temp folder to update. Does it rely on some PHP extensions or anything like that?

Only the XML extension which you must have if it worked at all. It does rely on the file modification times so if your file system is FAT32 or something instead of ext3 or ext4 it might not work. I’ve never tried it on fat32

from dir2cast.

I have been working on some more tests to see if I can reproduce and fix the problem. Thanks for reporting it.

from dir2cast.

i have done some work on the caching which may solve your problem, would love to know if they do

from dir2cast.

would also be useful if you could run the test suite for me on your Pi

from dir2cast.

Sorry for the late reply, I've been away from this project because of the holidays. How do I test the test suite?

from dir2cast.

To run the test suite,

• ssh into your Pi like usual
• I assume you have PHP already installed, but if not apt install php
• do pecl install xdebug
• make sure composer is installed (not sure if you can get that through apt but if you can't, these instructions seem reasonable
• git clone dir2cast onto your Pi from GitHub
• from inside the test folder (cd test), run composer install
• finally, run ./run.sh

from dir2cast.

Hi @gityopr hoping to hear back from you, or perhaps give instructions so I can have a go :) but I have a hunch it's a twist specific to your particular combination and I might not be able to replicate it

from dir2cast.

I'm sorry for the delay on this! I believe part of the issue is that I'm running my webserver out of a docker container. SWAG by Linuxserver. Last time I looked at this I had a lot of trouble figuring out how to get composer running in that container. I'll give it a bit more work this weekend.

Thanks for not closing this issue!

from dir2cast.

Last time I looked at this I had a lot of trouble figuring out how to get composer running in that container.

I would have thought you'd do something like this:

• run the container normally to start the web server (assuming the container name is now swag and docker ps shows it's running)
• docker exec -it swag /bin/bash - this will open a shell into the container
• cd /config - from the example docs i've seen, this seems to be the path inside the container where the files are mounted in
• cd ... - find where the dir2cast checkout is
• then follow the instructions from there, inside the container.

from dir2cast.

I have the same problem. Downloading a video with youtube-dl-server use the date of video. Also touch the file doesn't help. Even deleting a video still shows the video in my Podcast app. Solution is deleting dir2casts temp folder. Fow now I will run a 3rd container with a cronjob deleting the temp folder, if file count has been changed.

from dir2cast.

@helmut72 can you try setting MIN_CACHE_TIME=0 and MIN_FILE_AGE=0 in dir2cast.ini, it should effectively disable the cache altogether

from dir2cast.

I would still like to know more about the filesystem and the file dates that you have which are causing this behaviour - ideally a test case that I can work with so that I can reproduce the problem?

from dir2cast.

Hi, I use a standard Ubuntu 22.04 with Docker installed from docker.com, Apache with PHP. Nothng special with one exception: to reduce wear on a Raspberry I have set noatime option in fstab for ages, so I do it also on my server with SSD.
I have moved my collection from one folder to the video folder of dir2cast. File dates are different. From 2019 to may 2022. Then I have tested if dir2cast recognize them. Can test your suggested settings after work.

from dir2cast.

Thank you, I appreciate the feedback. I have an idea for a fix which will enable this to work with your use cases.

I did not realise how popular it would be to copy "old" files into a podcast folder (I don't do this personally), but I will add some code to refresh the feed if the number of files changes.

from dir2cast.

can you try setting MIN_CACHE_TIME=0 and MIN_FILE_AGE=0 in dir2cast.ini

This fix one problem, using touch file.mp4 add the file in my Podcast app. But deleting a file it is still shown in my Podcast app. Also files with old dates won't be added.

I did not realise how popular it would be to copy "old" files into a podcast folder

When someone uploaded a video on Youtube in 2019, file date becomes 2019 on my harddrive. I use youtube-dl-server to get my content easily.

Also try wget:
wget https://jsoncompare.org/LearningContainer/SampleFiles/Video/MP4/sample-mp4-file.mp4

File date is 2021 on the harddrive. dir2cast never find it.

from dir2cast.

v1.33 has been released which incorporates these changes. Thanks for your help!

from dir2cast.