Retrieve doesn't work with Media branch about spotweb HOT 52 CLOSED

Looks like same issue as reported here #58 (comment) #58 (comment)

from spotweb.

It's indeed the same problem, also tried the fixes that are proposed in that post, but no avail...

from spotweb.

That's really weird.
I'll look into this, maybe one of the latest commits broke something.

Not sure, but weird indeed

from spotweb.

Did you use git clone https://github.com/spotweb/spotweb.git when installing?

from spotweb.

No, I downloaded the whole Media branch to my desktop (Download ZIP). I'm running it on a Windows PC with XAMPP installed.

from spotweb.

Ok, i also am running the media branch and downloaded the ZIP too, but i'm not seeing these issue's.

I'll try to do another shadow install later on the evening to see if i can reproduce it.

from spotweb.

I installed it using this steps:

1. Download ZIP of Media Branch
2. Extracted ZIP to webdirectory of XAMPP
3. Created a new database in phpMyAdmin
4. Went to http://127.0.0.1/install.php
5. Completed the installation, if I go to http://127.0.0.1 I see my fresh SpotWeb installation
6. I open an elevated command prompt in de SpotWeb folder and execute the following command: php retrieve.php

Then I get the errors.
Am I doing something wrong?

from spotweb.

Just to let you know, I tried the fork of RikSmith, and here everything seems to be running fine, running the retriever now without errors so far.

from spotweb.

I'm installing Xampp right now, downloaded the ZIP.. let's see.

from spotweb.

Maybe an Unix vs Win line endings problem

from spotweb.

Could be, that's why i'm cross checking 2 new install's , 1 on Xampp, 1 on Wamp and 1 on my unix box using the same ZIP and see if there is actually a problem with the source. but my guess is there is none because i'm already running this.

Xampp/Mediabranch ZIP, replicated the issue as above (still no clue)
Wamp/Mediabranch ZIP, replicated the issue as above (still no clue)

Warning: PDO::quote() expects parameter 1 to be string, array given in C:\wamp\www\lib\dbeng\dbeng_pdo.php on line 175

Call Stack:
0.0000 312904 1. {main}() C:\wamp\www\retrieve.php:0
0.0600 4900888 2. Services_Retriever_Base->perform() C:\wamp\www\retrieve.php:155
2.8200 4925816 3. Services_Retriever_Base->loopTillEnd() C:\wamp\www\lib\services\Retriever\Services_Retriever_Base.php:319
8.1800 15163696 4. Services_Retriever_Spots->process() C:\wamp\www\lib\services\Retriever\Services_Retriever_Base.php:227
8.2200 15164824 5. Dao_Base_Spot->matchSpotMessageIds() C:\wamp\www\lib\services\Retriever\Services_Retriever_Spots.php:216
8.2200 15165640 6. dbeng_abs->arrayValToIn() C:\wamp\www\lib\dao\Base\Dao_Base_Spot.php:601
136.2560 15636608 7. dbeng_pdo->safe() C:\wamp\www\lib\dbeng\dbeng_abs.php:122
136.2560 15636656 8. PDO->quote() C:\wamp\www\lib\dbeng\dbeng_pdo.php:175

SpotWeb v0.68.29.32 on PHP v5.5.12 crashed

Fatal error occured retrieving reports:
42000: 1064: You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near '
,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,' at line 1 (SQLSTATE[42000]: Syntax error or access violation: 1064 Y
ou have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ',,,,,,,,,,,,,,,,
,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,' at line 1)

#0 C:\wamp\www\lib\dbeng\dbeng_pdo.php(146): dbeng_pdo->exec('SELECT messagei...', Array)
#1 C:\wamp\www\lib\dao\Base\Dao_Base_Spot.php(606): dbeng_pdo->arrayQuery('SELECT messagei...')
#2 C:\wamp\www\lib\services\Retriever\Services_Retriever_Spots.php(216): Dao_Base_Spot->matchSpotMessageIds(Array)
#3 C:\wamp\www\lib\services\Retriever\Services_Retriever_Base.php(227): Services_Retriever_Spots->process(Array, '2', 5003, 1429119380.9903)
#4 C:\wamp\www\lib\services\Retriever\Services_Retriever_Base.php(319): Services_Retriever_Base->loopTillEnd('0', '5000')
#5 C:\wamp\www\retrieve.php(155): Services_Retriever_Base->perform()
#6 {main}

Next up, brand new install using Mediabranch ZIP on my Unix box.

from spotweb.

After installing the ZIP on my Unix box i got the same issue replicated.
After some searching i found and fixed the problem..

5c0afee

from spotweb.

Please give me some feedback if the issue has been resolved, i'm able to retrieve on Xampp, Wamp and Unix-box

z:\wamp\bin\php\php5.5.12\php.exe retrieve.php
Last retrieve at 01/01/70 01:00:00
Retrieving new Spots from server secure.server.com...
Appr. Message count: 2405577
First message number: 2
Last message number: 2405579
Current article number: 2

Retrieving 2 till 5003 (parsed: 5001, in DB: 0, signed: 0, invalid: 5001, rtntn.skip: 0, mod: 0, full: 0, total: 5001) in 2.09 seconds
Retrieving 5003 till 10004 (parsed: 5001, in DB: 0, signed: 0, invalid: 5001, rtntn.skip: 0, mod: 0, full: 0, total: 5001) in 3.45 seconds
Retrieving 10004 till 15005 (parsed: 5001, in DB: 0, signed: 0, invalid: 5001, rtntn.skip: 0, mod: 0, full: 0, total: 5001) in 2.93 seconds
Retrieving 15005 till 20006 (parsed: 5001, in DB: 0, signed: 0, invalid: 5001, rtntn.skip: 0, mod: 0, full: 0, total: 5001) in 1.72 seconds
Retrieving 20006 till 25007 (parsed: 5001, in DB: 0, signed: 0, invalid: 5001, rtntn.skip: 0, mod: 0, full: 0, total: 5001) in 2.11 seconds
Retrieving 25007 till 30008 (parsed: 5001, in DB: 0, signed: 0, invalid: 5001, rtntn.skip: 0, mod: 0, full: 0, total: 5001) in 1.74 seconds
Retrieving 30008 till 35009 (parsed: 5001, in DB: 0, signed: 0, invalid: 5001, rtntn.skip: 0, mod: 0, full: 0, total: 5001) in 2.15 seconds
Retrieving 35009 till 40010 (parsed: 5001, in DB: 0, signed: 0, invalid: 5001, rtntn.skip: 0, mod: 0, full: 0, total: 5001) in 3.52 seconds
Retrieving 40010 till 45011 (parsed: 5001, in DB: 0, signed: 0, invalid: 5001, rtntn.skip: 0, mod: 0, full: 0, total: 5001) in 2.13 seconds

No more error. ;-)

from spotweb.

error fixed but created new error

:/var/www/spotweb# php retrieve.php
Last retrieve at Wed 15 Apr 2015 06:00:03 PM EDT
Retrieving new Spots from server ***.
Appr. Message count: 1163777
First message number: 2052169
Last message number: 3215946
Current article number: 3215936

Retrieving 3215936 till 3215947 (parsed: 10, in DB: 1, signed: 10, invalid: 0, rtntn.skip: 0, mod: 0, full: 0, total: 11) in 0.11 seconds
Processed a total of 11 spots
Finished retrieving spots.

Creating collections starting from 506523, PHP Notice: Undefined variable: val in /var/www/spotweb/lib/dbeng/dbeng_abs.php on line 109
PHP Notice: Undefined index: in /var/www/spotweb/lib/dbeng/dbeng_abs.php on line 109
PHP Notice: Undefined variable: val in /var/www/spotweb/lib/dbeng/dbeng_abs.php on line 109
PHP Notice: Undefined index: in /var/www/spotweb/lib/dbeng/dbeng_abs.php on line 109
PHP Notice: Undefined variable: val in /var/www/spotweb/lib/dbeng/dbeng_abs.php on line 109
PHP Notice: Undefined index: in /var/www/spotweb/lib/dbeng/dbeng_abs.php on line 109
PHP Notice: Undefined variable: val in /var/www/spotweb/lib/dbeng/dbeng_abs.php on line 109
PHP Notice: Undefined index: in /var/www/spotweb/lib/dbeng/dbeng_abs.php on line 109
PHP Notice: Undefined variable: val in /var/www/spotweb/lib/dbeng/dbeng_abs.php on line 109
PHP Notice: Undefined index: in /var/www/spotweb/lib/dbeng/dbeng_abs.php on line 109
, done
Calculating how many spots are new, done.
Last retrieve at Wed 15 Apr 2015 06:53:20 PM EDT
Retrieving new comments from server ***
Appr. Message count: 7980154
First message number: 5124185
Last message number: 13104339
Current article number: 13104196

Retrieving 13104196 till 13104340, found 142 comments in 0.2 seconds
PHP Notice: Undefined variable: val in /var/www/spotweb/lib/dbeng/dbeng_abs.php on line 109
PHP Notice: Undefined variable: val in /var/www/spotweb/lib/dbeng/dbeng_abs.php on line 109
PHP Notice: Undefined variable: val in /var/www/spotweb/lib/dbeng/dbeng_abs.php on line 109
PHP Notice: Undefined variable: val in /var/www/spotweb/lib/dbeng/dbeng_abs.php on line 109
PHP Notice: Undefined variable: val in /var/www/spotweb/lib/dbeng/dbeng_abs.php on line 109
PHP Notice: Undefined variable: val in /var/www/spotweb/lib/dbeng/dbeng_abs.php on line 109
PHP Notice: Undefined variable: val in /var/www/spotweb/lib/dbeng/dbeng_abs.php on line 109
PHP Notice: Undefined variable: val in /var/www/spotweb/lib/dbeng/dbeng_abs.php on line 109
PHP Notice: Undefined variable: val in /var/www/spotweb/lib/dbeng/dbeng_abs.php on line 109
PHP Notice: Undefined variable: val in /var/www/spotweb/lib/dbeng/dbeng_abs.php on line 109
PHP Notice: Undefined variable: val in /var/www/spotweb/lib/dbeng/dbeng_abs.php on line 109
PHP Notice: Undefined variable: val in /var/www/spotweb/lib/dbeng/dbeng_abs.php on line 109
PHP Notice: Undefined variable: val in /var/www/spotweb/lib/dbeng/dbeng_abs.php on line 109
PHP Notice: Undefined variable: val in /var/www/spotweb/lib/dbeng/dbeng_abs.php on line 109
PHP Notice: Undefined variable: val in /var/www/spotweb/lib/dbeng/dbeng_abs.php on line 109
PHP Notice: Undefined variable: val in /var/www/spotweb/lib/dbeng/dbeng_abs.php on line 109
PHP Notice: Undefined variable: val in /var/www/spotweb/lib/dbeng/dbeng_abs.php on line 109
PHP Notice: Undefined variable: val in /var/www/spotweb/lib/dbeng/dbeng_abs.php on line 109
PHP Notice: Undefined variable: val in /var/www/spotweb/lib/dbeng/dbeng_abs.php on line 109
PHP Notice: Undefined variable: val in /var/www/spotweb/lib/dbeng/dbeng_abs.php on line 109
PHP Notice: Undefined variable: val in /var/www/spotweb/lib/dbeng/dbeng_abs.php on line 109
PHP Notice: Undefined variable: val in /var/www/spotweb/lib/dbeng/dbeng_abs.php on line 109
PHP Notice: Undefined variable: val in /var/www/spotweb/lib/dbeng/dbeng_abs.php on line 109
PHP Notice: Undefined variable: val in /var/www/spotweb/lib/dbeng/dbeng_abs.php on line 109
PHP Notice: Undefined variable: val in /var/www/spotweb/lib/dbeng/dbeng_abs.php on line 109
PHP Notice: Undefined variable: val in /var/www/spotweb/lib/dbeng/dbeng_abs.php on line 109
PHP Notice: Undefined variable: val in /var/www/spotweb/lib/dbeng/dbeng_abs.php on line 109
PHP Notice: Undefined variable: val in /var/www/spotweb/lib/dbeng/dbeng_abs.php on line 109
PHP Notice: Undefined variable: val in /var/www/spotweb/lib/dbeng/dbeng_abs.php on line 109
PHP Notice: Undefined variable: val in /var/www/spotweb/lib/dbeng/dbeng_abs.php on line 109
PHP Notice: Undefined variable: val in /var/www/spotweb/lib/dbeng/dbeng_abs.php on line 109
PHP Notice: Undefined variable: val in /var/www/spotweb/lib/dbeng/dbeng_abs.php on line 109
PHP Notice: Undefined variable: val in /var/www/spotweb/lib/dbeng/dbeng_abs.php on line 109
PHP Notice: Undefined variable: val in /var/www/spotweb/lib/dbeng/dbeng_abs.php on line 109
PHP Notice: Undefined variable: val in /var/www/spotweb/lib/dbeng/dbeng_abs.php on line 109
PHP Notice: Undefined variable: val in /var/www/spotweb/lib/dbeng/dbeng_abs.php on line 109
PHP Notice: Undefined variable: val in /var/www/spotweb/lib/dbeng/dbeng_abs.php on line 109
PHP Notice: Undefined variable: val in /var/www/spotweb/lib/dbeng/dbeng_abs.php on line 109
PHP Notice: Undefined variable: val in /var/www/spotweb/lib/dbeng/dbeng_abs.php on line 109
PHP Notice: Undefined variable: val in /var/www/spotweb/lib/dbeng/dbeng_abs.php on line 109
PHP Notice: Undefined variable: val in /var/www/spotweb/lib/dbeng/dbeng_abs.php on line 109
PHP Notice: Undefined variable: val in /var/www/spotweb/lib/dbeng/dbeng_abs.php on line 109
PHP Notice: Undefined variable: val in /var/www/spotweb/lib/dbeng/dbeng_abs.php on line 109
PHP Notice: Undefined variable: val in /var/www/spotweb/lib/dbeng/dbeng_abs.php on line 109
PHP Notice: Undefined variable: val in /var/www/spotweb/lib/dbeng/dbeng_abs.php on line 109
PHP Notice: Undefined variable: val in /var/www/spotweb/lib/dbeng/dbeng_abs.php on line 109
PHP Notice: Undefined variable: val in /var/www/spotweb/lib/dbeng/dbeng_abs.php on line 109
PHP Notice: Undefined variable: val in /var/www/spotweb/lib/dbeng/dbeng_abs.php on line 109
PHP Notice: Undefined variable: val in /var/www/spotweb/lib/dbeng/dbeng_abs.php on line 109
PHP Notice: Undefined variable: val in /var/www/spotweb/lib/dbeng/dbeng_abs.php on line 109
PHP Notice: Undefined variable: val in /var/www/spotweb/lib/dbeng/dbeng_abs.php on line 109
PHP Notice: Undefined variable: val in /var/www/spotweb/lib/dbeng/dbeng_abs.php on line 109
PHP Notice: Undefined variable: val in /var/www/spotweb/lib/dbeng/dbeng_abs.php on line 109
PHP Notice: Undefined variable: val in /var/www/spotweb/lib/dbeng/dbeng_abs.php on line 109
PHP Notice: Undefined variable: val in /var/www/spotweb/lib/dbeng/dbeng_abs.php on line 109
PHP Notice: Undefined variable: val in /var/www/spotweb/lib/dbeng/dbeng_abs.php on line 109
PHP Notice: Undefined variable: val in /var/www/spotweb/lib/dbeng/dbeng_abs.php on line 109
PHP Notice: Undefined variable: val in /var/www/spotweb/lib/dbeng/dbeng_abs.php on line 109
PHP Notice: Undefined variable: val in /var/www/spotweb/lib/dbeng/dbeng_abs.php on line 109
PHP Notice: Undefined variable: val in /var/www/spotweb/lib/dbeng/dbeng_abs.php on line 109
PHP Notice: Undefined variable: val in /var/www/spotweb/lib/dbeng/dbeng_abs.php on line 109
PHP Notice: Undefined variable: val in /var/www/spotweb/lib/dbeng/dbeng_abs.php on line 109
PHP Notice: Undefined variable: val in /var/www/spotweb/lib/dbeng/dbeng_abs.php on line 109
PHP Notice: Undefined variable: val in /var/www/spotweb/lib/dbeng/dbeng_abs.php on line 109
PHP Notice: Undefined variable: val in /var/www/spotweb/lib/dbeng/dbeng_abs.php on line 109
PHP Notice: Undefined variable: val in /var/www/spotweb/lib/dbeng/dbeng_abs.php on line 109
PHP Notice: Undefined variable: val in /var/www/spotweb/lib/dbeng/dbeng_abs.php on line 109
PHP Notice: Undefined variable: val in /var/www/spotweb/lib/dbeng/dbeng_abs.php on line 109
PHP Notice: Undefined variable: val in /var/www/spotweb/lib/dbeng/dbeng_abs.php on line 109
PHP Notice: Undefined variable: val in /var/www/spotweb/lib/dbeng/dbeng_abs.php on line 109
PHP Notice: Undefined variable: val in /var/www/spotweb/lib/dbeng/dbeng_abs.php on line 109
PHP Notice: Undefined variable: val in /var/www/spotweb/lib/dbeng/dbeng_abs.php on line 109
PHP Notice: Undefined variable: val in /var/www/spotweb/lib/dbeng/dbeng_abs.php on line 109
PHP Notice: Undefined variable: val in /var/www/spotweb/lib/dbeng/dbeng_abs.php on line 109
PHP Notice: Undefined variable: val in /var/www/spotweb/lib/dbeng/dbeng_abs.php on line 109
PHP Notice: Undefined variable: val in /var/www/spotweb/lib/dbeng/dbeng_abs.php on line 109
PHP Notice: Undefined variable: val in /var/www/spotweb/lib/dbeng/dbeng_abs.php on line 109
PHP Notice: Undefined variable: val in /var/www/spotweb/lib/dbeng/dbeng_abs.php on line 109
PHP Notice: Undefined variable: val in /var/www/spotweb/lib/dbeng/dbeng_abs.php on line 109
PHP Notice: Undefined variable: val in /var/www/spotweb/lib/dbeng/dbeng_abs.php on line 109
PHP Notice: Undefined variable: val in /var/www/spotweb/lib/dbeng/dbeng_abs.php on line 109
PHP Notice: Undefined variable: val in /var/www/spotweb/lib/dbeng/dbeng_abs.php on line 109
PHP Notice: Undefined variable: val in /var/www/spotweb/lib/dbeng/dbeng_abs.php on line 109
PHP Notice: Undefined variable: val in /var/www/spotweb/lib/dbeng/dbeng_abs.php on line 109
PHP Notice: Undefined variable: val in /var/www/spotweb/lib/dbeng/dbeng_abs.php on line 109
PHP Notice: Undefined variable: val in /var/www/spotweb/lib/dbeng/dbeng_abs.php on line 109
PHP Notice: Undefined variable: val in /var/www/spotweb/lib/dbeng/dbeng_abs.php on line 109
PHP Notice: Undefined variable: val in /var/www/spotweb/lib/dbeng/dbeng_abs.php on line 109
PHP Notice: Undefined variable: val in /var/www/spotweb/lib/dbeng/dbeng_abs.php on line 109
PHP Notice: Undefined variable: val in /var/www/spotweb/lib/dbeng/dbeng_abs.php on line 109
PHP Notice: Undefined variable: val in /var/www/spotweb/lib/dbeng/dbeng_abs.php on line 109
PHP Notice: Undefined variable: val in /var/www/spotweb/lib/dbeng/dbeng_abs.php on line 109
PHP Notice: Undefined variable: val in /var/www/spotweb/lib/dbeng/dbeng_abs.php on line 109
PHP Notice: Undefined variable: val in /var/www/spotweb/lib/dbeng/dbeng_abs.php on line 109
PHP Notice: Undefined variable: val in /var/www/spotweb/lib/dbeng/dbeng_abs.php on line 109
PHP Notice: Undefined variable: val in /var/www/spotweb/lib/dbeng/dbeng_abs.php on line 109
PHP Notice: Undefined variable: val in /var/www/spotweb/lib/dbeng/dbeng_abs.php on line 109
PHP Notice: Undefined variable: val in /var/www/spotweb/lib/dbeng/dbeng_abs.php on line 109
PHP Notice: Undefined variable: val in /var/www/spotweb/lib/dbeng/dbeng_abs.php on line 109
PHP Notice: Undefined variable: val in /var/www/spotweb/lib/dbeng/dbeng_abs.php on line 109
PHP Notice: Undefined variable: val in /var/www/spotweb/lib/dbeng/dbeng_abs.php on line 109
PHP Notice: Undefined variable: val in /var/www/spotweb/lib/dbeng/dbeng_abs.php on line 109
PHP Notice: Undefined variable: val in /var/www/spotweb/lib/dbeng/dbeng_abs.php on line 109
PHP Notice: Undefined variable: val in /var/www/spotweb/lib/dbeng/dbeng_abs.php on line 109
PHP Notice: Undefined variable: val in /var/www/spotweb/lib/dbeng/dbeng_abs.php on line 109
PHP Notice: Undefined variable: val in /var/www/spotweb/lib/dbeng/dbeng_abs.php on line 109
PHP Notice: Undefined variable: val in /var/www/spotweb/lib/dbeng/dbeng_abs.php on line 109
Processed a total of 142 comments
Finished retrieving comments.

Last retrieve at Wed 15 Apr 2015 06:53:21 PM EDT
Retrieving new reports from server ***...
Appr. Message count: 44118
First message number: 94703
Last message number: 138821
Current article number: 138819

Retrieving 138819 till 138822, found 3 reports in 0.05 seconds
PHP Notice: Undefined variable: val in /var/www/spotweb/lib/dbeng/dbeng_abs.php on line 109
PHP Notice: Undefined variable: val in /var/www/spotweb/lib/dbeng/dbeng_abs.php on line 109
Processed a total of 3 reports
Finished retrieving reports.

Non-fatal: Updating black/whitelist failed, most likely unreachable!
Expiring debuglog entries, if any, done.
Starting to create statistics
Finished creating statistics

from spotweb.

Updated dbeng_abs.php, running retrieve now, will let you know the results.

from spotweb.

I updated it. Same problem continued to exist

from spotweb.

Ok, but retrieve works.. i'll look into this later today

from spotweb.

Try now:

041181f

from spotweb.

Errors are gone!
Thanks

from spotweb.

Perfect!.

Closing the issue,

from spotweb.

Please reopen, this is not the correct fix as far as i can tell

from spotweb.

Currently when I run retrieve I get following errors:

Retrieving 1956879 till 1961880

getHeader(): Failed to retrieve article: [email protected]

getHeader(): Failed to retrieve article: [email protected]

getBody(): Failed to retrieve article: [email protected]

Warning: gzinflate(): data error in C:\XAMPP\htdocs\spotweb-media\lib\services\N
ntp\Services_Nntp_SpotReading.php on line 134

Warning: gzinflate(): data error in C:\XAMPP\htdocs\spotweb-media\lib\services\N
ntp\Services_Nntp_SpotReading.php on line 134

Warning: gzinflate(): data error in C:\XAMPP\htdocs\spotweb-media\lib\services\N
ntp\Services_Nntp_SpotReading.php on line 134

Warning: gzinflate(): data error in C:\XAMPP\htdocs\spotweb-media\lib\services\N
ntp\Services_Nntp_SpotReading.php on line 134

Fatal error: Maximum execution time of 120 seconds exceeded in C:\XAMPP\htdocs\s
potweb-media\lib\services\Nntp\Services_Nntp_Engine.php on line 73

from spotweb.

I also set the max_execution_time in my php.ini to 5000, and put ini_set('max_execution_time', 5000); at the 2nd line of retrieve.php.

from spotweb.

@Zeagle88

Try this:
Settings -> Performance -> Retrieve increment

Lower this if you get timeouts during retrieve

Set to 500 for example and then see what happens.

from spotweb.

Changed the Retrieve Increment to 500, running retrieve now, will let you know the result.

from spotweb.

So far no crashes, only a lot of getBody() and getHeader() errors, but he's still going.

from spotweb.

@Zeagle88

Which USP provider do you have?

from spotweb.

:)

from spotweb.

Still getting a timeout...

from spotweb.

Wow, 600 seconds to retrieve 500 articles is waaayyy to long.

1. your DB could be corrupted,
2. Or your USP has issues..
3. Or you recently changed to a different USP?

from spotweb.

Could you also show me what your settings are under retrieve?

from spotweb.

I did a clean install, new database...
SunnyUsenet was always my USP, never changed.
The retrieval of spots with the master-branch works very fast, +/- 30seconds for 5000 spots, only in the media-branch it's slow.

I know that SunnyUsenet has a retention of 900 days, but if I put 900 in the retention settings, when I run the retrieve, I get a timeout (after 5000 seconds) at "Removing Spot information which is beyond retention period"

from spotweb.

If you want I can truncate all the tables of my DB and start over, maybe this will fix it?

from spotweb.

I see, but you are also retrieving full spots and full comments, this is severly sloweing down the retrieval..

Try disabeling those and try again.

from spotweb.

Also the prefetch of nzbs and images slows down retrieval.. read the wiki for performance tips.

from spotweb.

And i see you are trying to retrieve from 1-11-2009 !!

Try setting this to 1-1-2015 first.. then backfill

from spotweb.

Truncated the tables and did a fresh install.
These are my current Retrieve settings:

This is my retrieve now:

It's weird that the master-branch runs smooth with full spots & comments, and the media-branch doesn't...

from spotweb.

That's more like it.. ;-)

never done a side-by-side comparison of both master and media, but i'm interested in the results if anyone has time,

from spotweb.

Well, these are my settings for the master-branch:

With these settings, the retrieve works as fast the retrieve from the media-branch with the following settings:

from spotweb.

hmm...

What's the use of the collections?

from spotweb.

Trying to match similar files, collection of series, for example grouping of S01E01, E02, E03 etc etc..

My max-execution_time is set to 600 (PHP5.3.28).. and it's working just fine.

Could you try to run recreate-collections.php ?

from spotweb.

In my php.ini, I set max_execution_time=5000, but for some reason it still times out after 120 seconds, but not for everything, sometimes it waits 5000 seconds before timeout.

Running recreate-collections.php now.

from spotweb.

Create a file called "phpinfo.php" and place this in your root folder of spotweb with this contents:

Then browse to http://spotweb/phpinfo.php and look for max_execution_time, what does it say?

Seems like either your settings are not applying to your spotweb root or you are using the wrong path to php.. if you typ "which php" from bash? what reply do you get?

from spotweb.

The recreate-collections.php finished without errors, running retrieve again and now he's retrieve all the comments.

from spotweb.

After a while when retrieving comments, I still get a timeout for 120 seconds.

from spotweb.

That not possible because you max_execution_time is set to 5000.

So either your using the wrong command for php using a different php.ini.

from spotweb.

That's weird because in my spotweb-media folder is no php.ini file... I'll look in my folders if I find another php.ini file.

from spotweb.

check in your xampp folders
you will need to have php.ini in /cli folder

On Thu, Apr 30, 2015 at 2:35 PM, Jonathan [email protected] wrote:

That's weird because in my spotweb-media folder is no php.ini file... I'll
look in my folders if I find another php.ini file.

—
Reply to this email directly or view it on GitHub
#60 (comment).

from spotweb.

Searched in my whole XAMPP directory for php.ini and found one. This is the one that I changed the max_execution_time from 120 to 5000.

from spotweb.

max_exitcution time more than 600 (5 mins) is more than enough

change your Retrieve increment to 2000 instead of 5000

On Thu, Apr 30, 2015 at 3:36 PM, Jonathan [email protected] wrote:

[image: image]
https://cloud.githubusercontent.com/assets/11771445/7413741/825c9db4-ef4e-11e4-8654-99aab63d1015.png
Searched in my whole XAMPP directory for php.ini and found one. This is
the one that I changed the max_execution_time from 120 to 5000.

—
Reply to this email directly or view it on GitHub
#60 (comment).

from spotweb.

I can change the retrieve increment to 2000, but that doesn't fix the problem that I get timeouts after 120sec :)

from spotweb.

i would blame your USP i get my comments in general in less than 0.5 sec

On Thu, Apr 30, 2015 at 4:00 PM, Jonathan [email protected] wrote:

I can change the retrieve increment to 2000, but that doesn't fix the
problem that I get timeouts after 120sec :)

—
Reply to this email directly or view it on GitHub
#60 (comment).

from spotweb.