The Verge site config does not pull images about ftr-site-config HOT 14 CLOSED

Could you provide an example. I just tried the first article that loaded and go this result (plenty of images):

http://ftr.fivefilters.org/makefulltextfeed.php?url=www.theverge.com%2F2017%2F4%2F1%2F14969400%2Fsci-fi-fantasy-books-recommendations-april-2017

from ftr-site-config.

I'll try to explain.

Even in the link you provided, if you visit the verge page, you'll notice that the header image or the first image (however you want to call it) doesn't load. Now for a lot of pages, this is usually the only image for the entire post.

Take a look at this page, it loads no image, but the verge website certainly has one:

http://ftr.fivefilters.org/makefulltextfeed.php?url=http%3A%2F%2Fwww.theverge.com%2Fcircuitbreaker%2F2017%2F3%2F31%2F15129708%2Fapple-usb-c-accessories-cables-adaptors-discount-macbook-pro

from ftr-site-config.

Ah, I see what you mean. Yes, this is an issue which we hope to improve. The main problem is that these feature images are often outside the main body element. It's possible to include them with custom rules (I'll try to add one for theverge.com) but the ideal solution would be something a little smarter that can try to detect them.

A few versions ago we actually added code to Full-Text RSS that would look for the og:image meta element and insert that into the start of the extracted article if and only if the extracted HTML contained no image elements. I need to see why it's not working for the example you provided, as it should really be including this image.

from ftr-site-config.

I've tried this using varying xpath patterns but the output remains the same. Another feed which has a similar issue is that of nytimes.com

For some weird reason even when I select the topmost of div elements, the main image (placeholder image) is always skipped. Same with this.

from ftr-site-config.

Just updated the site config for The Verge, so this issue should be fixed for this site if you try the links above again.

This line in the site config

body: //div[contains(@class, 'c-entry-content') or contains(@class, 'c-entry-hero__image')]

was changed to

body: //picture[contains(@class, 'c-picture')] | //div[contains(@class, 'c-entry-content') or contains(@class, 'c-entry-hero__image')]

from ftr-site-config.

Thanks a lot. For some weird reason, my own installation, v 3.5 doesn't show images even with the latest config update. Strange.

from ftr-site-config.

Could you test if this works with older versions? I tried your config with 2 older versions and they didn't work.

Thanks again. Appreciate the help.

from ftr-site-config.

No time at the moment to test older versions. But I can't see why it'd be an issue. Might have something to do with the parser being used or the lazy image replacement. My suggestion is try enabling debug on our hosted version and your own version and compare the results.

from ftr-site-config.

Great suggestion. Appreciate all the help. @fivefilters

from ftr-site-config.

I solved it. Turning off the html5php parser resolved the issue. Must be something with my system. @fivefilters

from ftr-site-config.

I'll just put this link here. The same thing happens with NyTimes too. The header image is missing.

http://ftr.fivefilters.org/makefulltextfeed.php?url=https%3A%2F%2Fmobile.nytimes.com%2F2017%2F03%2F30%2Ftechnology%2Fuber-waymo-levandowski.html&max=1

from ftr-site-config.

Fixed for that too: https://github.com/fivefilters/ftr-site-config/blob/master/mobile.nytimes.com.txt

from ftr-site-config.

Thank you @fivefilters ! I'm horribly sorry, but the image still doesn't show up:

http://ftr.fivefilters.org/makefulltextfeed.php?url=https%3A%2F%2Fwww.nytimes.com%2F2017%2F03%2F28%2Fmagazine%2Fnone-of-us-are-safe-from-getting-owned.html%3Fpartner%3Drss%26emc%3Drss%26_r%3D0&max=1

from ftr-site-config.

It does now, yay!

Actually sometimes it does, sometimes it doesn't. Weird.

from ftr-site-config.