Comments (13)
thrillfall
commented on September 15, 2024
Thanks for debugging this. Please provide a test that is going green with your implementation
from nextcloud-gpodder.
thawn
commented on September 15, 2024
I have the same error. Has the fix been implemented?
from nextcloud-gpodder.
thrillfall
commented on September 15, 2024
The default identifier is guid. URL is prone to redundancy. The solution might fix your situation but will break it for everyone else.
We need a test that reproduces the exception as to assess this solutions effects
from nextcloud-gpodder.
mf8hd
commented on September 15, 2024
$this->episodeActionRepository->findByEpisodeUrl($episodeActionEntity->getGuid(), $userId)->getId();
If i read the code above correctly we query the database collumn 'episode' [via the function findByEpisodeUrl()] for a guid [getGuid()]. That can´t be right!
My hack was to query the collumn 'episode' for an episode url. If that´s wrong, then maybe it´s the other way round:
$this->episodeActionRepository->findByGuid($episodeActionEntity->getGuid(), $userId)->getId();
and the line
$this->episodeActionRepository->deleteEpisodeActionByEpisodeUrl($episodeActionEntity->getGuid(), $userId);
should read something like this
$this->episodeActionRepository->deleteEpisodeActionByGuid($episodeActionEntity->getGuid(), $userId);
Does that make any sense ?
from nextcloud-gpodder.
thrillfall
commented on September 15, 2024
please replicate your current condition in a test (see existing tests for howto set on up) so we can assess the outcome of the proposed changes.
from nextcloud-gpodder.
mf8hd
commented on September 15, 2024
that´s the difficult part, since my problem ist gone. I´ll try to replicate it ...
from nextcloud-gpodder.
thawn
commented on September 15, 2024
I will do my best to create such a test case. However, I will need some help:
Could you please point me to what would be a good comparable existing test case?
Also, could you please help me with extracting the debugging information out of the android antenna pod app and out of my nextcloud server (I am admin for that nextcloud instance)?
from nextcloud-gpodder.
thawn
commented on September 15, 2024
FYI, the problem still persists on my setup, so I should be able to get the required info for a test case.
from nextcloud-gpodder.
mf8hd
commented on September 15, 2024
About:
This is a way to log the JSON data of /apps/gpoddersync/episode_action/create POST requests on the nextcloud side of things
Versions:
NextCloud 27.1.4, PHP 8.2, NGINX, GPodder Sync 3.8.2, AntennaPod/3.2.0, curl/7.74.0
Caution:
Check all the paths, usernames, commands, etc. and change them to fit your environment !
Overview, aka the short version:
- Enable nextcloud debug logging
- Create the logfile (with write permission for the webserver user !)
- Put logging code in
EpisodeActionController.php. (That´s the entry point for/apps/gpoddersync/episode_action/createhttp requests.) - Restart webserver and php
- Create json file with test data
- Create nextcloud testuser, with an app password
- Test the setup with curl, testuser and file with test data
- You should find the test data in the logfile
- Sync with AntennaPod and bad data
- Bad request logged in nextcloud logfile -> Success
Enable nextcloud logging:
Change or insert these lines in nextclouds config/config.php:
'log_type' => 'file',
'logfile' => '/var/log/nextcloud.log',
'loglevel' => '0',
'log.syslog.format' => '[%reqId%][%remoteAddr%][%user%][%app%][%method%][%url%][%exception%][%data%] %message%',
Create the logfile /var/log/nextcloud.log with sufficient permissions for the webserver:
sudo bash
echo hello > /var/log/nextcloud.log
chown www-data.root /var/log/nextcloud.log
sudo tail -f /var/log/nextcloud.log|grep gpoddersync
Patch /apps-external/gpoddersync/lib/Controller/EpisodeActionController.php
Insert the two lines
use function OCP\Log\logger;
and
logger('gpoddersync')->debug(json_encode($episodeActionsArray));
as shown below.
<?php
declare(strict_types=1);
namespace OCA\GPodderSync\Controller;
use OCA\GPodderSync\Core\EpisodeAction\EpisodeActionSaver;
use OCA\GPodderSync\Db\EpisodeAction\EpisodeActionRepository;
use OCP\AppFramework\Controller;
use OCP\AppFramework\Http\JSONResponse;
use OCP\IRequest;
// *** DEBUG MF8HD ***
use function OCP\Log\logger;
class EpisodeActionController extends Controller {
private EpisodeActionRepository $episodeActionRepository;
private string $userId;
private EpisodeActionSaver $episodeActionSaver;
protected $request;
public function __construct(
string $AppName,
IRequest $request,
$UserId,
EpisodeActionRepository $episodeActionRepository,
EpisodeActionSaver $episodeActionSaver
) {
parent::__construct($AppName, $request);
$this->episodeActionRepository = $episodeActionRepository;
$this->userId = $UserId ?? '';
$this->episodeActionSaver = $episodeActionSaver;
$this->request = $request;
}
/**
*
* @NoAdminRequired
* @NoCSRFRequired
*
* @return JSONResponse
*/
public function create(): JSONResponse {
$episodeActionsArray = $this->filterEpisodesFromRequestParams($this->request->getParams());
// *** DEBUG MF8HD ***
logger('gpoddersync')->debug(json_encode($episodeActionsArray));
// *** DEBUG MF8HD: The exception occures in the next line ***
$this->episodeActionSaver->saveEpisodeActions($episodeActionsArray, $this->userId);
return new JSONResponse(["timestamp" => time()]);
}
/**
* @NoAdminRequired
* @NoCSRFRequired
*
* @param int $since
* @return JSONResponse
*/
public function list(int $since = 0): JSONResponse {
$episodeActions = $this->episodeActionRepository->findAll($since, $this->userId);
$untypedEpisodeActionData = [];
foreach ($episodeActions as $episodeAction) {
$untypedEpisodeActionData[] = $episodeAction->toArray();
}
return new JSONResponse([
"actions" => $untypedEpisodeActionData,
"timestamp" => time()
]);
}
/**
* @param array $data
* @return array $episodeActionsArray
*/
public function filterEpisodesFromRequestParams(array $data): array {
return array_filter($data, "is_numeric", ARRAY_FILTER_USE_KEY);
}
}
Restart webserver and php:
systemctl stop nginx php8.2-fpm
systemctl start nginx php8.2-fpm
Create json testfile gptest.json with the following test content:
[
{
"podcast": "https://www.omnycontent.com/d/playlist/9c074afa-3313-47e8-b802-a9f900789975/09af2160-238f-48b2-b20b-ad4b00ebd8e7/b86dddc1-67a5-41c2-a13c-ad4b00ebd8f5/podcast.rss",
"episode": "https://chrt.fm/track/383EG2/traffic.omny.fm/d/clips/9c074afa-3313-47e8-b802-a9f900789975/09af2160-238f-48b2-b20b-ad4b00ebd8e7/c9fd6328-74a5-4836-b94c-b0a1012e511b/audio.mp3?utm_source=Podcast&in_playlist=b86dddc1-67a5-41c2-a13c-ad4b00ebd8f5",
"guid": "c9fd6328-74a5-4836-b94c-b0a1012e511b",
"action": "play",
"timestamp": "2023-10-24T16:45:42",
"started": 1321,
"position": 1321,
"total": 1321
},
{
"podcast": "https://www.omnycontent.com/d/playlist/9c074afa-3313-47e8-b802-a9f900789975/09af2160-238f-48b2-b20b-ad4b00ebd8e7/b86dddc1-67a5-41c2-a13c-ad4b00ebd8f5/podcast.rss",
"episode": "https://chrt.fm/track/383EG2/traffic.omny.fm/d/clips/9c074afa-3313-47e8-b802-a9f900789975/09af2160-238f-48b2-b20b-ad4b00ebd8e7/c9fd6328-74a5-4836-b94c-b0a1012e511b/audio.mp3?utm_source=Podcast&in_playlist=b86dddc1-67a5-41c2-a13c-ad4b00ebd8f5",
"guid": "c9fd6328-74a5-4836-b94c-b0a1012e511b",
"action": "delete",
"timestamp": "2023-10-24T16:45:44"
},
{
"podcast": "https://www.omnycontent.com/d/playlist/9c074afa-3313-47e8-b802-a9f900789975/09af2160-238f-48b2-b20b-ad4b00ebd8e7/b86dddc1-67a5-41c2-a13c-ad4b00ebd8f5/podcast.rss",
"episode": "https://chrt.fm/track/383EG2/traffic.omny.fm/d/clips/9c074afa-3313-47e8-b802-a9f900789975/09af2160-238f-48b2-b20b-ad4b00ebd8e7/c9fd6328-74a5-4836-b94c-b0a1012e511b/audio.mp3?utm_source=Podcast&in_playlist=b86dddc1-67a5-41c2-a13c-ad4b00ebd8f5",
"guid": "c9fd6328-74a5-4836-b94c-b0a1012e511b",
"action": "delete",
"timestamp": "2023-10-24T16:45:44"
}
]
Create testuser in nextcloud:
Create testuser USERNAME
Create app password for testuser USERNAME
Test with curl:
curl -u USERNAME:APP-PASSWORD-FOR-USERNAME-AND-CURL -X POST -H "Content-Type: application/json" -d @gptest.json https://YOUR.SERVER.DOMAIN/YOUR-NEXTCLOUD/index.php/apps/gpoddersync/episode_action/create
Result in /var/log/nextcloud.log:
{"reqId":"Z1a1c6Z4cIj6HaQusiBI","level":0,"time":"2023-12-05T10:02:17+00:00","remoteAddr":"127.0.0.1","user":"USERNAME","app":"gpoddersync","method":"POST","url":"/owncloud/index.php/apps/gpoddersync/episode_action/create","message":"[{\"podcast\":\"https:\\/\\/www.omnycontent.com\\/d\\/playlist\\/9c074afa-3313-47e8-b802-a9f900789975\\/09af2160-238f-48b2-b20b-ad4b00ebd8e7\\/b86dddc1-67a5-41c2-a13c-ad4b00ebd8f5\\/podcast.rss\",\"episode\":\"https:\\/\\/chrt.fm\\/track\\/383EG2\\/traffic.omny.fm\\/d\\/clips\\/9c074afa-3313-47e8-b802-a9f900789975\\/09af2160-238f-48b2-b20b-ad4b00ebd8e7\\/c9fd6328-74a5-4836-b94c-b0a1012e511b\\/audio.mp3?utm_source=Podcast&in_playlist=b86dddc1-67a5-41c2-a13c-ad4b00ebd8f5\",\"guid\":\"-c9fd6328-74a5-4836-b94c-b0a1012e511b\",\"action\":\"play\",\"timestamp\":\"2023-10-24T16:45:42\",\"started\":1321,\"position\":1321,\"total\":1321},{\"podcast\":\"https:\\/\\/www.omnycontent.com\\/d\\/playlist\\/9c074afa-3313-47e8-b802-a9f900789975\\/09af2160-238f-48b2-b20b-ad4b00ebd8e7\\/b86dddc1-67a5-41c2-a13c-ad4b00ebd8f5\\/podcast.rss\",\"episode\":\"https:\\/\\/chrt.fm\\/track\\/383EG2\\/traffic.omny.fm\\/d\\/clips\\/9c074afa-3313-47e8-b802-a9f900789975\\/09af2160-238f-48b2-b20b-ad4b00ebd8e7\\/c9fd6328-74a5-4836-b94c-b0a1012e511b\\/audio.mp3?utm_source=Podcast&in_playlist=b86dddc1-67a5-41c2-a13c-ad4b00ebd8f5\",\"guid\":\"-c9fd6328-74a5-4836-b94c-b0a1012e511b\",\"action\":\"delete\",\"timestamp\":\"2023-10-24T16:45:44\"},{\"podcast\":\"https:\\/\\/www.omnycontent.com\\/d\\/playlist\\/9c074afa-3313-47e8-b802-a9f900789975\\/09af2160-238f-48b2-b20b-ad4b00ebd8e7\\/b86dddc1-67a5-41c2-a13c-ad4b00ebd8f5\\/podcast.rss\",\"episode\":\"https:\\/\\/chrt.fm\\/track\\/383EG2\\/traffic.omny.fm\\/d\\/clips\\/9c074afa-3313-47e8-b802-a9f900789975\\/09af2160-238f-48b2-b20b-ad4b00ebd8e7\\/c9fd6328-74a5-4836-b94c-b0a1012e511b\\/audio.mp3?utm_source=Podcast&in_playlist=b86dddc1-67a5-41c2-a13c-ad4b00ebd8f5\",\"guid\":\"-c9fd6328-74a5-4836-b94c-b0a1012e511b\",\"action\":\"delete\",\"timestamp\":\"2023-10-24T16:45:44\"}]","userAgent":"curl/7.74.0","version":"27.1.4.1","data":{"app":"gpoddersync"}}
Conclusion:
You should see the postcast episodes that prevent syncing (and several others) in the nextcloud logfile before the exeption occures. Then you can put the suspicious episode data (reformated !) in the gptest.json file and try again with curl.
You can check the database table oc_gpodder_episode_action in your nextcloud database for the podcast episodes that made it into the database.
from nextcloud-gpodder.
mf8hd
commented on September 15, 2024
Meanwhile:
I can reproduce the error/exception with the following json testfile:
[
{
"podcast": "https://podcast1",
"episode": "https://episode1",
"guid": "guid1",
"action": "play",
"timestamp": "2023-10-24T16:45:42",
"started": 1321,
"position": 1321,
"total": 1321
},
{
"podcast": "https://podcast1",
"episode": "https://episode1",
"guid": "guid2",
"action": "play",
"timestamp": "2023-10-24T16:45:42",
"started": 1321,
"position": 1321,
"total": 1321
},
{
"podcast": "https://podcast1",
"episode": "https://episode2",
"guid": "guid1",
"action": "play",
"timestamp": "2023-10-24T16:45:42",
"started": 1321,
"position": 1321,
"total": 1321
},
{
"podcast": "https://podcast1",
"episode": "https://episode2",
"guid": "guid2",
"action": "play",
"timestamp": "2023-10-24T16:45:42",
"started": 1321,
"position": 1321,
"total": 1321
}
]
The testfile contains all permutations of two episodes and two guids for one podcast.
The exception vanishes when i use the hack from my initial post (with getEpisode()).
Now i´ll try to make a working version with getGuid().
from nextcloud-gpodder.
thawn
commented on September 15, 2024
@mf8hd thanks a lot for the detailed information! I will use that to get the log data for my setup - however I won't have time before the weekend. Happy coding - it sounds like you are getting close to a solution :-)
from nextcloud-gpodder.
LinAGKar
commented on September 15, 2024
My impression is that @mf8hd's original solution is actually the correct one. That function (deleteConflictingEpisodeAction) will be hit when you're trying to add an episode action with a guid the already exists in the DB, and a URL that also already exists but on a different row. So it I understand things correctly, in the above example, what will happen is:
For the first item:
No item with guid 'guid1' and episode 'https://episode1' exists. So it inserts them as row 1.
For the second item:
A row with episode 'https://episode1' already exists, but there is no row with guid 'guid2'. So it finds row 1 by episode, and changes its guid to 'guid2'.
For the third item:
No item with guid 'guid1' or episode 'https://episode2' exists. So it inserts them as row 2.
For the fourth item:
A row with episode 'guid2' already exists. So it finds row 1 by episode and tries to change its episode to 'https://episode2'. But there is already a different row with that episode. So deleteConflictingEpisodeAction is called in order to delete the conflicting row, by episode. And it takes the guid value from the item, and tries to search for it in the episode column, which of course doesn't find a match. Thus it gets NULL, and then tries to call getId on NULL.
If I understand things correctly, the function should be using getEpisode, in order to work as intended. In which case it would find row 2 by episode, delete it, and then be able to change episode on row 1 to 'https://episode2'.
The test testDoNotFailToUpdateEpisodeActionByGuidIfThereIsAnotherWithTheSameValueForEpisodeUrl doesn't trip because the item it tries to save at https://github.com/thrillfall/nextcloud-gpodder/blob/main/tests/Integration/EpisodeActionSaverGuidBackwardCompatibilityTest.php#L84C84-L84C100 has the same value for the guid and episode, so it ends up working correctly anyway.
from nextcloud-gpodder.
thrillfall
commented on September 15, 2024
please replicate your current condition in a test (see existing tests for howto set on up) so we can assess the outcome of the proposed changes.
in a pull request please. Thanks
from nextcloud-gpodder.
Related Issues (20)
- login problem from antennapod HOT 4
- Add support for Gpodder client HOT 3
- Failed sync with antennaPod: "make sure the cursor is initialized correctly" HOT 2
- AntennaPod seemingly receives 302 / HTML response, sync fails because response not JSON HOT 6
- link on https://apps.nextcloud.com/apps/gpoddersync to `Ask questions or discuss` is dead HOT 1
- Installation fails: App with id gpoddersync has invalid signature HOT 3
- Cannot Open Any Podcasts in Repod HOT 2
- Access/Export Podcast history HOT 1
- Call to a member function get() on null Error HOT 1
- Parsing Error HOT 1
- [solved] Newbie needs helps to start syncing HOT 2
- GPodder not updated HOT 8
- Correctly handle events like `DOWNLOAD` or `DELETE`
- Make GPodder Sync Nextcloud Hub 8 (v 29) compatible HOT 9
- Avoid multiple entires for same podcast
- GPodder Sync Install Error on NC 29 HOT 1
- Unkown podcast after syncing HOT 1
- duplicate key value violates unique constraint "gpodder_episode_user_id"
- Document Authentication
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from nextcloud-gpodder.