Crawl posts image as well? about fbcrawl HOT 8 CLOSED

what did you mean by "html hierarchy path", can't you get it using WebDeveloper tools? like press F12, click on "Inspect" highlight the facebook post image, take the image URL from there ?... clean it up and insert into the code

from fbcrawl.

@mnba If you are crawling for one image sure this approach may work. But I would like to crawl the complete images (full resolutions ones). Since this is a public repo, I also want to hear how does the author wish to handle list of string within CSV columns (maybe separated using tabs within img columns?)

from fbcrawl.

you can use the built in scrapy image extraction functions.

https://docs.scrapy.org/en/latest/topics/media-pipeline.html

from fbcrawl.

I have completed the function in my branch, Thanks everyone

from fbcrawl.

how cralwer image from facebook fanpage?
Can you hlep me? @theblackcat102

from fbcrawl.

@hkbk Sorry for the late reply, you can checkout my fork version of fbcrawl at this line. Basically an iterative sequence to find all the post image ( if exists ). Or you can try my fbcrawl with the postgresql part commented out on a fanpage link and see if it works?

from fbcrawl.

@hkbk Sorry for the late reply, you can checkout my fork version of fbcrawl at this line. Basically an iterative sequence to find all the post image ( if exists ). Or you can try my fbcrawl with the postgresql part commented out on a fanpage link and see if it works?

• @theblackcat102 : basically crawler public and private facebook's group but we checkpoint from Facebook. Can you for an idea to pass checkpoint? thanks.

from fbcrawl.

@hkbk can you explain what does checkpoint from Facebook means?

from fbcrawl.