Comments (7)
I could not replicate that, but the latest commit 43aa262 should at least make it fail gracefully and not stop the scraping process.
I'll leave this open for now - let me know if you figure a way to consistently replicate that or if it fails with a specific book in particular.
from blinkist-scraper.
It's still failing. :\
I am able to download few books 1 or 2 at a time then process gets stopped with this error, then I have rerun the command to scrape more books.
Traceback (most recent call last):
File "C:\Users\Fang Yuan\Downloads\blinkist-scraper-master\scraper.py", line 229, in scrape_book_audio
audio_request_json = audio_request.json();
File "C:\Python38\lib\site-packages\requests\models.py", line 888, in json
return complexjson.loads(
File "C:\Python38\lib\json_init_.py", line 357, in loads
return _default_decoder.decode(s)
File "C:\Python38\lib\json\decoder.py", line 337, in decode
obj, end = self.raw_decode(s, idx=_w(s, 0).end())
File "C:\Python38\lib\json\decoder.py", line 355, in raw_decode
raise JSONDecodeError("Expecting value", s, err.value) from None
json.decoder.JSONDecodeError: Expecting value: line 1 column 1 (char 0)During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "main.py", line 66, in
audio_files = scraper.scrape_book_audio(driver, book_json, args.language)
File "C:\Users\Fang Yuan\Downloads\blinkist-scraper-master\scraper.py", line 239, in scrape_book_audio
print(f'[!] Received malformed json data: {audio_request_json.text}')
UnboundLocalError: local variable 'audio_request_json' referenced before assignment
from blinkist-scraper.
It's still failing. :\
I am able to download few books 1 or 2 at a time then process gets stopped with this error, then I have rerun the command to scrap more books.
Traceback (most recent call last):
File "C:\Users\Fang Yuan\Downloads\blinkist-scraper-master\scraper.py", line 229, in scrape_book_audio
audio_request_json = audio_request.json();
File "C:\Python38\lib\site-packages\requests\models.py", line 888, in json
return complexjson.loads(
File "C:\Python38\lib\json__init__.py", line 357, in loads
return _default_decoder.decode(s)
File "C:\Python38\lib\json\decoder.py", line 337, in decode
obj, end = self.raw_decode(s, idx=_w(s, 0).end())
File "C:\Python38\lib\json\decoder.py", line 355, in raw_decode
raise JSONDecodeError("Expecting value", s, err.value) from None
json.decoder.JSONDecodeError: Expecting value: line 1 column 1 (char 0)
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "main.py", line 66, in
audio_files = scraper.scrape_book_audio(driver, book_json, args.language)
File "C:\Users\Fang Yuan\Downloads\blinkist-scraper-master\scraper.py", line 239, in scrape_book_audio
print(f'[!] Received malformed json data: {audio_request_json.text}')
UnboundLocalError: local variable 'audio_request_json' referenced before assignment
I am getting the same error when scraping the second book on a new clone on ca0d913.
from blinkist-scraper.
I might found the cause:
The error occurs after accepting the cookies inside the ChromeDriver window.
It seems there is some interference, maybe with some json data on side-loaded sites, just guessing ...
When not interacting with the Chrome window, the program seems to work like expected.
from blinkist-scraper.
My bad, latest commit 426d5bd should fix the handling of the exception. Again, it will not fix the bug but at least it should handle it without stopping the program and provide some more information about what's failing.
@Wunderharke Interesting! I never touch the Chrome window while scraping anyway, since it's all automated - just let it do its job. @mikan01 can you confirm if that makes it better for you too?
from blinkist-scraper.
My bad, latest commit 426d5bd should fix the handling of the exception. Again, it will not fix the bug but at least it should handle it without stopping the program and provide some more information about what's failing.
@Wunderharke Interesting! I never touch the Chrome window while scraping anyway, since it's all automated - just let it do its job. @mikan01 can you confirm if that makes it better for you too?
Thank you, now the problem seems to be fixed. :)
from blinkist-scraper.
@mikan01,
If this issue has been resolved would you please consider closing it?
Thanks!
from blinkist-scraper.
Related Issues (20)
- Chromedriver issues on raspberry - unknown error: DevToolsActivePort file doesn't exist HOT 6
- Missing "uncategorized" category from scraper, thus not retrieving all books HOT 8
- reCaptcha not taking HOT 1
- Stuck in Cloudflare hCaptcha loop. HOT 15
- Captha taking longer than expected HOT 12
- Error in proxy2 when scrapping HOT 1
- Downloads blinks in German even with --language en specified HOT 2
- Cloudflare blocking scraping HOT 40
- Infinite captcha HOT 2
- Not all Blinkist books can be Downloaded HOT 1
- Script Throws Exception for audio download of blinks HOT 1
- 'NoneType' object has no attribute 'split'
- string indices must be integers
- blinkist-scraper no longer working - attribute error!? HOT 5
- Timed out receiving message from renderer: 298.991
- infinite captcha: bug or scraper not working any more? HOT 3
- script doesn't work HOT 4
- It doesn't work HOT 1
- Some books are not scraped HOT 4
- Can't get past Captcha HOT 11
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from blinkist-scraper.