comments crawl fail with IndexError: list index out of range about fbcrawl HOT 6 OPEN

Ok bois, I had the same errors while crawling comments from a specific post.

How I got around this was to use "-a post=" instead of "-a page=".

I learned about the new feature from this: #27
It is working for me now :)

from fbcrawl.

I have the same problem

from fbcrawl.

I'm stucked with the same problem too. Can't figure out the solution yet.

from fbcrawl.

How I got around this was to use "-a post=" instead of "-a page=".

It was still throwing uncaught error, I had to do like this -a page="" -a post="FULL_POST_PATH"

Note there will still be KeyError: 'flag' after that but that error is atleast caught.

from fbcrawl.

I was getting the same error as the OP. I tried ngbtri's suggestion of using "post" instead of "page" but now I'm getting a different error(see below). I tried ishandutta2007's suggestion but I get another error:

*** With -a page="" -a post="https://mbasic..."

2019-10-05 08:56:47 [twisted] CRITICAL:
Traceback (most recent call last):
File "/usr/lib/python3.7/site-packages/twisted/internet/defer.py", line 1418, in _inlineCallbacks
result = g.send(result)
File "/usr/lib/python3.7/site-packages/scrapy/crawler.py", line 85, in crawl
self.spider = self._create_spider(*args, **kwargs)
File "/usr/lib/python3.7/site-packages/scrapy/crawler.py", line 108, in _create_spider
return self.spidercls.from_crawler(self, *args, **kwargs)
File "/usr/lib/python3.7/site-packages/scrapy/spiders/init.py", line 50, in from_crawler
spider = cls(*args, **kwargs)
File "/home/redacted/github/fbcrawl/fbcrawl/spiders/comments.py", line 24, in init
raise AttributeError('You need to specifiy only one between post and page')
AttributeError: You need to specifiy only one between

*** With -a post instead of -a page

2019-10-05 08:51:42 [scrapy.spidermiddlewares.httperror] INFO: Ignoring response <404 https://mbasic.facebook.com/login/%22https:/mbasic.facebook.com/story.php?story_fbid=redacted&id=redacted%22>: HTTP status code is not handled or not allowed
2019-10-05 08:51:42 [scrapy.core.engine] INFO: Closing spider (finished)

from fbcrawl.

Ok bois, I had the same errors while crawling comments from a specific post.

How I got around this was to use "-a post=" instead of "-a page=".

I learned about the new feature from this: #27
It is working for me now :)

Thanks this worked for me too

from fbcrawl.