ryochan7 / alphagpo Goto Github PK

The populate-unknown-podcasts command relies on episode data linked to a Podcast to determine if the feed is considered unparsed. That works fine for most feeds. However, a growing problem that has come up is people linking feeds that are restricted to paying members via HTTP Authentication. Obviously mygpo will not be able to parse those feeds so the current check and the old blank title check will fail for determining whether a podcast is known. There needs to be some flag to tell mygpo to skip a feed otherwise the populate-unknown-podcasts command will just try to parse it and fail every time the command runs. A flag will also be useful for keeping the feed off the Directory section of the site.

It might be time to start reviewing the automated pull requests from dependabot and update this project's dependencies. Most of the other problems with this project have been dealt with so there is time to check package upgrades and make sure the project does not break. For the time being, mygpo will definitely still target Django 3.x rather than attempting to upgrade to Django 4.x.

The update interval for Podcast feed updates is too long for my liking. It seems like the project tries to find an average time interval between episodes and then use that as the time interval for updating the feed info. It is too long for my liking particularly for podcasts that release multiple episodes per week.

Need to probably set an upper interval threshold and maybe change the main interval algorithm.

I have mentioned this in the mygpo issues tracker but I should make a small post about it here as well. The alphagpo public instance will be closing July 31st. I can no longer afford the $10 per month to keep the public instance of this fork running. Also, the Android podcatcher AntennaPod no longer plans to support mygpo for podcast syncing so there is far less of a need for this project, or the upstream mygpo project, to even exist. Also, this repo and the alphagpo-feedservice repo will be set to read-only archived at the end of the month.

It has been good getting back into playing with Django. Learning Django again was part of the motivation for wanting to get involved with mygpo initially.

This problem is more of a generic Django usage problem than anything specific to mygpo. I want to provide some way to provide Django to create default Category and CategoryTag objects from some sort of fixture data as a part of the initial project setup. I thought using the included Django fixture support would do what I want but it includes too much for my use case; fixture support seems mainly useful for full data backup rather than re-creating data with updated datetime values and other dynamic values.

Right now, the prod instance of mygpo has applicable Category and CategoryTag instances corresponding with the currently supported iTunes podcast categories. Fixture support can dump the current data to a JSON file. Natural keys are used rather than hard-coded keys. However, I do not like that the created and modified field values would be fixed to the time the original instances were added to the original database. It would probably be better to create model instances in Python so those fields can be updated with datetime values for when an object was created on the current database.

There seems to be more feeds that are either invalid or just don't have any episodes logged; YouTube feeds would be one example. Currently, the only way to specify that a feed should be skipped is if the feed requires HTTP authentication in order to access it. There needs to be a different way to flag feeds to be skipped from future parsing.

Still have not found out if there is a way to automate adding categories for use with the Directory section. Although, the original site Directory section is mostly non-functional and many category titles are just wrong like android;. Some old categories, like Bbc, don't look like they were added from podcast feed info but added manually instead.

Also, trying to add a Category from the Django admin results in 500 errors.

Search has not worked on my instance since the start. Seen some messages about search indexes but it didn't matter to me before. With more feeds now indexed on the instance, it might be a good idea to actually figure out how to get the app to create search indexes and get the search feature working. The search feature mostly works on the official mygpo instance.

Trying to pull up aggregate client stats results in 500 errors. Not even sure what data is supposedly given on those pages. Need to fix some views in the administration package.

This is mainly for reference as the biggest task is already done. When a client submits a subscription for a podcast feed that is unknown, very few elements of the podcast feed are added to the database. mygpo uses a blank title field to denote that a podcast entity has not been populated. There are already a couple of ways to have mygpo parse a feed and populate a podcast object. There was no way that I had seen to do it for a group of feeds.

I ended up changing that with commit c62df87 with the addition of a new management command populate-unknown-podcasts. Just going to run it manually every once in a while although a cron job could be set up to perform the task. That should make the Untitled podcast problem from the original repo far less of a problem.

One future consideration is regarding maybe adding a couple of indexes to the Podcasts table. For the management command, the latest_episode_timestamp field is checked and a podcast is considered unknown if no timestamp is populated. The query could end up being slow once a lot of podcasts are added to the table without using an index.

Many of the tasks assigned in the project use the Celery provided shared_task decorator. Many of the decorator calls use an outdated run_every argument that no longer applies. There are other options for setting up periodic tasks with the current project and Celery 5.x. Going to work on editing tasks little by little.

Probably want to set up Fabric or a Makefile for helping run repetitive tasks on the project. If nothing else, I should show the screen commands I am using to start and stop the Django project when I run it.

https://www.fabfile.org/