UnhandledErrorOrRejection about secret-agent HOT 29 CLOSED

@GoldLine1337 SecretAgent spawns processes for both sockets and the Replay UI when it launches. Looking into this a bit, it sounds like on Unix-like processes, the parent nodejs process will allocate a matching amount of memory to spawned child processes.

Could you help me get a picture of where you're running this:

• What kind of RAM footprint is the machine you're running on?
• Do you have swap space on the machine? https://stackoverflow.com/a/32444643

from secret-agent.

I'm sure what do you mean but im on windows and I have 8gb of ram
Unable to connect to CoreTab { error: 'Error: spawn ENOMEM', context: {} }
2x UnhandledErrorOrRejection

from secret-agent.

Ah, ok. Should be plenty. How much free memory is on your machine?

If possible, would you be willing to share your session database? It's generated by SecretAgent in your temp dir (%windir%.secret-agent[newest file]). The path should look something like: C:\Users\[USERNAME]\AppData\Local\Temp\tmp-13152HMljuIU3YKR0\.secret-agent\23423423523453453453454345.db

from secret-agent.

30% free memory
I could get it up to 50% free
https://easyupload.io/wxqunk
sessions.db

from secret-agent.

you want all the dbs?

from secret-agent.

from secret-agent.

Looks like that would be b3d3faa0-3fb0-11eb-a460-2fe0571dee00.db if you're experiencing crashes on all of them.

from secret-agent.

https://easyupload.io/x8s8jd

from secret-agent.

Thanks, could you also share which windows version and nodejs version you're using?

from secret-agent.

Windows 10 and nodejs v12.8.0

from secret-agent.

@GoldLine1337 I'm having trouble reproducing, but I'm seeing some issues with resources not being cleaned up properly. Are you seeing this unhandled error on every attempt? Or did it work a few times before it stopped working?

from secret-agent.

I see this error everytime I run secret-agent it doesnt work at all

from secret-agent.

@GoldLine1337 I've fixed some issues with Replay, but I don't think they're causing your main problem. Do you see this after a clean reboot as well?

And is this same computer able to run a basic puppeteer example? (asking because puppeteer also needs to launch Chromium, which is where your code is breaking)

from secret-agent.

Yes puppeteer runs perfectly!
const SecretAgent = require('secret-agent');

(async () => {
const agent = new SecretAgent({ humanEmulatorId: 'ghost' });
await agent.goto('https://example.org');
const title = await agent.document.title;
const intro = await agent.document.querySelector('p').textContent;
await agent.close();

console.log('Retrieved from https://example.org', {
title,
intro,
});
})();
I have tried the example code it also doesnt work.

from secret-agent.

Very weird. Which version of puppeteer did you test with? And I don't think I asked which version you're using.

I'm gonna compare their launch of chrome with ours.

from secret-agent.

Could you try two experiments for me also:

1. Try to add to your example above the require process.env.SA_SHOW_REPLAY = false;
2. In your puppeteer test, would you try to see if you experience launch issues when using pipes (puppeteer.launch({pipe: true});)

from secret-agent.

Works with pipes also version is [email protected]

from secret-agent.

Ok. Were you able to try the first experiment as well?

process.env.SA_SHOW_REPLAY = 'false';
const SecretAgent = require('secret-agent');

(async () => {
const agent = new SecretAgent({ humanEmulatorId: 'ghost' });
await agent.goto('https://example.org');
const title = await agent.document.title;
const intro = await agent.document.querySelector('p').textContent;
await agent.close();

console.log('Retrieved from https://example.org', {
title,
intro,
});
})();
```

from secret-agent.

after adding that false parameter
I dont get this error
2020-12-18T21:23:34.524Z WARN [C:\Users\GoldLine~HQ.000\node_modules@secret-agent\client\lib\ScriptInstance] Unable to connect to CoreTab { error: 'Error: spawn ENOMEM', context: {} }

but I get
2020-12-18T21:23:34.528Z ERROR [C:\Users\GoldLineHQ.000\node_modules@secret-agent\core\index] UnhandledErrorOrRejection { clientError: 'Error: spawn ENOMEM', context: {} }
2020-12-18T21:23:34.529Z ERROR [C:\Users\GoldLineHQ.000\node_modules@secret-agent\core\index] UnhandledErrorOrRejection { clientError: 'Error: spawn ENOMEM', context: {} }

from secret-agent.

OK, I don't think those told us much :(. Can we try one more experiment?

Try adding the launch arguments to Chromium in Puppeteer (these are the ones we're using in SecretAgent).

puppeteer.launch({
   args: [
 '--headless', 
 '--hide-scrollbars', 
 '--mute-audio',
  '--disable-background-networking',
  '--enable-features=NetworkService,NetworkServiceInProcess',
  '--disable-background-timer-throttling',
  '--disable-backgrounding-occluded-windows',
  '--disable-breakpad',
  '--disable-client-side-phishing-detection',
  '--disable-component-extensions-with-background-pages',
  '--disable-default-apps',
  '--disable-dev-shm-usage',
  '--disable-extensions',
  '--disable-features=TranslateUI,site-per-process',
  '--disable-hang-monitor',
  '--disable-ipc-flooding-protection',
  '--disable-prompt-on-repost',
  '--disable-renderer-backgrounding',
  '--disable-sync',
  '--disable-gpu',

  '--force-color-profile=srgb',
  '--use-gl=swiftshader-webgl',
  '--use-gl=swiftshader',
  '--use-gl=osmesa',

  '--metrics-recording-only',
  '--no-first-run',

  '--enable-automation',
  '--remote-debugging-pipe',

  '--password-store=basic',
  '--use-mock-keychain',
  '--ignore-certificate-errors',
  '--allow-running-insecure-content',

  '--lang=en-US,en;q=0.9',
  '--window-size=1920,1080',
  '--window-position=0,0',

  // don't leak private ip
  '--force-webrtc-ip-handling-policy=default_public_interface_only',
  // Use proxy for localhost URLs
  '--proxy-bypass-list=<-loopback>',
  '--no-startup-window',
]
})

from secret-agent.

It works can I disable those with secret-agent?

from secret-agent.

Meaning that crashes it? Could you help figure out which one is crashing you?

from secret-agent.

These are the differences between Secret Agent and Puppeteer launch args:

If I had to guess, it's going to be the webgl ones.

  '--disable-features=TranslateUI,site-per-process',
  '--disable-gpu',
  '--use-gl=swiftshader-webgl',
  '--use-gl=swiftshader',
  '--use-gl=osmesa',

  '--ignore-certificate-errors',
  '--allow-running-insecure-content',

  '--lang=en-US,en;q=0.9',
  '--window-size=1920,1080',
  '--window-position=0,0',

  // don't leak private ip
  '--force-webrtc-ip-handling-policy=default_public_interface_only',
  // Use proxy for localhost URLs
  '--proxy-bypass-list=<-loopback>',
  '--no-startup-window',

from secret-agent.

still works is secret-agent always headless?

from secret-agent.

It runs headless by default, and Replay provides a UI to scrub back and forth with your script. There's a flag to turn on the browser if you need it. We need to have Chrome running though ;)

It looks like puppeteer 2.1.1 is on Chrome 80. Could you try [email protected]? That version runs on Chrome83. Trying to narrow down the differences.

Thanks for your help troubleshooting

from secret-agent.

@GoldLine1337 Were you able to try out puppeteer 3?

from secret-agent.

from secret-agent.

Should I close this out? Are you done trying? I haven't found any way to reproduce this, unfortunately.

from secret-agent.

from secret-agent.