andrejgajdos / link-preview-generator Goto Github PK

Hi there, firstly, I love this link preview generator. Your blog post spells it out very cleanly.

Anyway, it works beautifully locally but then when we tried to launch it live (in our docker) it errors out.

We don't get the error locally, do we really have to add a bunch of packages like this article suggests?

https://medium.com/@ssmak/how-to-fix-puppetteer-error-while-loading-shared-libraries-libx11-xcb-so-1-c1918b75acc3

This is a minor bug but h2 tags are never used as a title.
See the code below from index.js, where both parts parse the h1 header :

const h1 = document.querySelector("h1").innerHTML;
if (h1 != null && h1.length > 0) {
  return h1;
}
const h2 = document.querySelector("h1").innerHTML;
if (h2 != null && h2.length > 0) {
  return h2;
}

I'm not sure how useful it is to get h2 tags, but I thought it couldn't hurt to fix it.

Is there a way to use this in a front-end application? E.g. using webpack and ES6 imports? The current package seems to only work in node.

Do

I have added this in one of my APIs with this logic

const { link } = req.body;
	try {
		const previewData = await linkPreviewGenerator(link, [
			'--no-sandbox',
			'--disable-setuid-sandbox'
		  ]);
		return res.json(previewData);
	} catch (error) {
		console.error(error);
		return res.status(500).send(error.message);
	}

And In the Dockerfile, I have added all the dependencies Puppeteer will require to launch chrome.
with this code:

RUN apt-get update \
    && apt-get install -yq --no-install-recommends \
	ca-certificates fonts-liberation gconf-service libappindicator1 \
	libasound2 libatk-bridge2.0-0 libatk1.0-0 libc6 libcairo2 libcups2 \
	libdbus-1-3 libexpat1 libfontconfig1 libgbm1 libgcc1 libgconf-2-4 libgdk-pixbuf2.0-0 \
	libglib2.0-0 libgtk-3-0 libnspr4 libnss3 libpango-1.0-0 \
	libpangocairo-1.0-0 libstdc++6 libx11-6 libx11-xcb1 \
	libxcb1 libxcomposite1 libxcursor1 libxdamage1 libxext6 \
	libxfixes3 libxi6 libxrandr2 libxrender1 libxss1 libxtst6 lsb-release wget xdg-utils

But when I call the API, it is throwing URL is not defined

I would love, if you can help mw with this.

Hello, i have trying get images preview of url products. I used this package in my project and works with Ebay, Mercadolibre, Newegg but with Amazon not get image. Example of image preview Amazon:

I would like you to help me, since most of the product urls come from this website. Thanks!

src is never defined in

paragraphs isn't defined either at

Thanks for the efforts and work you gave to make this.

I'm facing a isssue when i send google meet link as url to the function and the error is :
(node:15759) [fs-extra-WARN0003] Warning: fs.realpath.native is not a function. Is fs being monkey-patched? please can you provide any insight to this .

The above image shows that the basic linkedin page isn't accessible.
Is this because they implemented some anti-scraping technique?

Thx!

could you provide the heroku code found here https://link-preview-generator.herokuapp.com/ ?
Best,

This is all very new to me. I saw you deployed the demo to Heroku, but it seems that to use this as an API for a mobile app in production, it would have to be deployed to someplace like AWS. I looked at the documentation for npm and I couldn't find out how this would be used to create a business-use API. Can you direct me in the right direction?

I want to get a favicon as preview data. Can you merge a PR if I implement the following?

// index.js
const getFavicon = async (page, uri) => { // New
  ...
  return favicon
}

module.exports = async (
  uri,
  puppeteerArgs = [],
  puppeteerAgent = "facebookexternalhit/1.1 (+http://www.facebook.com/externalhit_uatext.php)",
  executablePath
) => {
  puppeteer.use(pluginStealth());
  ...
  const obj = {};
  obj.title = await getTitle(page);
  obj.description = await getDescription(page);
  obj.domain = await getDomainName(page, uri);
  obj.img = await getImg(page, uri);
  ojb.favicon = await getFavicon(page, uri) // New

  await browser.close();
  return obj;
};

Hi,

thanks for your work so far.
I've trouble with quite some links and I thought you might could have a look. Thanks!

https://en.bem.info/methodology/
(node:300075) UnhandledPromiseRejectionWarning: TypeError [ERR_INVALID_URL]: Invalid URL: /methodology/

https://www.der-informationsdesigner.de/agentur-blog/allgemein/persuasive-design-psychologie-im-webdesign
(node:300075) UnhandledPromiseRejectionWarning: Error: Evaluation failed: Error [ERR_TLS_CERT_ALTNAME_INVALID]: Hostname/IP does not match certificate's altnames: Host: der-informationsdesigner.de. is not in the cert's altnames: DNS:*.goserver.host, DNS:goserver.host

https://web.dev/one-line-layouts/
(node:300075) UnhandledPromiseRejectionWarning: TimeoutError: Navigation timeout of 30000 ms exceeded

This dependency was not found:
* ws in ./node_modules/puppeteer/lib/WebSocketTransport.js
To install it, you can run: npm install --save ws

used in Vuejs project
even after installing ws , still same error!

Thanks for your library. It works well, just this link: https://yarnpkg.com doesn't.

Here is some of the stack trace:
Error: Evaluation failed: ReferenceError: uri is not defined\n at <anonymous>:48:37\n at Array.forEach (<anonymous>)\n at <anonymous>:46:14\n at ExecutionContext._evaluateInternal (./node_modules/puppeteer/lib/ExecutionContext.js:122:13)\n at runMicrotasks (<anonymous>)\n at processTicksAndRejections (node:internal/process/task_queues:94:5)\n at async ExecutionContext.evaluate (./node_modules/puppeteer/lib/ExecutionContext.js:48:12)\n at async getImg (./node_modules/link-preview-generator/index.js:18:15)\n at async module.exports (./node_modules/link-preview-generator/index.js:176:13)

I'm not sure if it's link-preview-generator or puppeteer which needs a fix. The error also appears in the provided heroku demo

Thanks for sharing this handy preview generator. It works on most links, but a few are failing:

https://www.bloomberg.com/news/articles/2020-03-27/trump-threatens-to-force-gm-to-move-faster-on-ventilators

Error: Evaluation failed: TypeError: Cannot read property 'src' of undefined
at puppeteer_evaluation_script:50:22
at ExecutionContext._evaluateInternal ((...)\node_modules\puppeteer\lib\ExecutionContext.js:122:13)
at process._tickCallback (internal/process/next_tick.js:68:7)
-- ASYNC --
at ExecutionContext. ((...)\node_modules\puppeteer\lib\helper.js:111:15)
at DOMWorld.evaluate ((...)\node_modules\puppeteer\lib\DOMWorld.js:112:20)
at process._tickCallback (internal/process/next_tick.js:68:7)
-- ASYNC --
at Frame. ((...)\node_modules\puppeteer\lib\helper.js:111:15)
at Page.evaluate ((...)\node_modules\puppeteer\lib\Page.js:860:43)
at Page. ((...)\node_modules\puppeteer\lib\helper.js:112:23)
at getImg ((...)\node_modules\link-preview-generator\index.js:18:26)
at module.exports ((...)\node_modules\link-preview-generator\index.js:174:19)
at process._tickCallback (internal/process/next_tick.js:68:7)

I am using it in Windows 10 nodejs and got the following error while CPU is heavily busy

(node:5176) MaxListenersExceededWarning: Possible EventEmitter memory leak detected. 11 exit listeners added. Use emitter.setMaxListeners() to increase limit
(node:5176) MaxListenersExceededWarning: Possible EventEmitter memory leak detected. 11 SIGINT listeners added. Use emitter.setMaxListeners() to increase limit
(node:5176) MaxListenersExceededWarning: Possible EventEmitter memory leak detected. 11 SIGTERM listeners added. Use emitter.setMaxListeners() to increase limit
(node:5176) MaxListenersExceededWarning: Possible EventEmitter memory leak detected. 11 SIGHUP listeners added. Use emitter.setMaxListeners() to increase limit
(node:5176) UnhandledPromiseRejectionWarning: Error: Evaluation failed: TypeError: Cannot read property 'innerHTML' of null
at puppeteer_evaluation_script:14:44

Demos are not working.

After running:

const linkPreviewGenerator = require("link-preview-generator");
const previewData = linkPreviewGenerator("https://www.youtube.com/watch?v=8mqqY2Ji7_g");
console.log(previewData);
Promise { }

This hangs out here:

(node:12888) UnhandledPromiseRejectionWarning: TimeoutError: Navigation timeout of 30000 ms exceeded at C:\lpg\node_modules\puppeteer\lib\cjs\puppeteer\common\LifecycleWatch er.js:106:111

Any ideas of how to make it work?

Hello
i try to use your package but i got this error :
"Evaluation failed: TypeError: Cannot read property 'innerHTML' of null\n at puppeteer_evaluation_script:14:44" when trying this url:
https://www.laprovence.com/article/om/6063890/om-leonardo-balerdi-a-tire-daile.html

thanks for your help

I am sorry but it is not working for

"https://www.bloomberg.com/news/articles/2020-06-20/europe-will-have-a-digital-tax-no-matter-what-says-breton?srnd=premium-europe"

Possible to fix it?

Update the npm package (https://www.npmjs.com/package/link-preview-generator)

I am trying to generating an amazon product URL preview with the image.
Your library has been very helpful to get me the image of the product but review and rating not required in my and they are appearing.
Is there any workaround for that.

I just want the product image only.

Hey, Is there any ready-to-use files for shared hosting so we can make our own link preview?

Hi Everyone, i have tried this with Walmart products but this package doesn't work with link that are CAPTCHA protected. Any solution for this issue? Thanks

Link: https://www.walmart.com/ip/Best-Foods-Mayonnaise-Real-Mayo-Gluten-Free-Sandwich-Spread-Rich-in-Omega-3-ALA-30-oz/10291731?athcpid=10291731&athpgid=AthenaGlassHomepage__gm__-1.0&athcgid=null&athznid=bs&athieid=v0&athstid=CS098&athguid=CoDW1pljBsF1OIvZ0aIZV3FqYst9a2Gugrcu&athancid=null&athposb=5&athena=true

I can't use this with error

/root/bot/node_modules/puppeteer/lib/cjs/puppeteer/node/BrowserRunner.js:193
            reject(new Error([
                   ^

Error: Failed to launch the browser process!
[0729/104939.603734:ERROR:zygote_host_impl_linux.cc(90)] Running as root without --no-sandbox is not supported. See https://crbug.com/638180.


TROUBLESHOOTING: https://github.com/puppeteer/puppeteer/blob/main/docs/troubleshooting.md

    at onClose (/root/bot/node_modules/puppeteer/lib/cjs/puppeteer/node/BrowserRunner.js:193:20)
    at Interface.<anonymous> (/root/bot/node_modules/puppeteer/lib/cjs/puppeteer/node/BrowserRunner.js:183:68)
    at Interface.emit (node:events:539:35)
    at Interface.close (node:internal/readline/interface:529:10)
    at Socket.onend (node:internal/readline/interface:258:10)
    at Socket.emit (node:events:539:35)
    at endReadableNT (node:internal/streams/readable:1345:12)
    at processTicksAndRejections (node:internal/process/task_queues:83:21)

Node.js v17.9.0

Hey,

First of all, thank you for this great library.
I use your demo website to generate preview for my website and it seams to be down since a couple of days. The website is not really par of this repository but it was a way to notice you.

Alex