Comments (1)
Hi, welcome to the scrapy world, it's a fun journey!
First thing I notice is that right before the last ERROR line you should have the INFO like:
[fb] INFO: Parse function called on https://mbasic.facebook.com/DonaldTrump
So make sure you gave the appropriate page name. Also sometimes the bot gets stuck because Facebook fingerprints the browser and tries to block the new scrapy device. Although I've written some code in the parse_home
function to bypass this behavior, sometimes it doesn't work well. A simple workaround is to log in via your traditional web browser once and everything should work fine.
Also check your mailbox, sometimes fb sends you an email saying that an unknown device has been trying to access without permission.
from fbcrawl.
Related Issues (20)
- no module named urllib2
- . HOT 1
- [scrapy.spidermiddlewares.httperror] INFO: Ignoring response <404 https://mbasic.facebook.com/gettingstarted/groups/2296031677301785>: HTTP status code is not handled or not allowed
- INFO: Ignoring response <404
- Error from VMWare Windows Instance HOT 1
- Spider error processing HOT 11
- Page scraped, clicking on "more"! ERROR HOT 3
- How do I run this
- "IndentationError: Unexpected Indent" when running scrapy, blank csv file HOT 2
- after finishing successfully, it results an empty csv file fbcrawl.py HOT 1
- scraping multiple pages by adding text file with page links to be extracted
- Empty page comments file HOT 2
- DNSLookupError: DNS lookup failed: no results for hostname lookup: mbasic.facebook.com.
- IndexError: list index out of range in comments.py line63 HOT 1
- "Show More" apparently not working. HOT 6
- The crawler has stopped working! HOT 2
- Ignoring response 404 - getting empty csv HOT 1
- Comments crawler not parsing pages correctly HOT 1
- Cannot login
- No <form> element found with HOT 1
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from fbcrawl.