Comments (29)
@GoldLine1337 SecretAgent spawns processes for both sockets and the Replay UI when it launches. Looking into this a bit, it sounds like on Unix-like processes, the parent nodejs process will allocate a matching amount of memory to spawned child processes.
Could you help me get a picture of where you're running this:
- What kind of RAM footprint is the machine you're running on?
- Do you have swap space on the machine? https://stackoverflow.com/a/32444643
from secret-agent.
I'm sure what do you mean but im on windows and I have 8gb of ram
Unable to connect to CoreTab { error: 'Error: spawn ENOMEM', context: {} }
2x UnhandledErrorOrRejection
from secret-agent.
Ah, ok. Should be plenty. How much free memory is on your machine?
If possible, would you be willing to share your session database? It's generated by SecretAgent in your temp dir (%windir%.secret-agent[newest file]). The path should look something like: C:\Users\[USERNAME]\AppData\Local\Temp\tmp-13152HMljuIU3YKR0\.secret-agent\23423423523453453453454345.db
from secret-agent.
30% free memory
I could get it up to 50% free
https://easyupload.io/wxqunk
sessions.db
from secret-agent.
you want all the dbs?
from secret-agent.
from secret-agent.
Looks like that would be b3d3faa0-3fb0-11eb-a460-2fe0571dee00.db
if you're experiencing crashes on all of them.
from secret-agent.
from secret-agent.
Thanks, could you also share which windows version and nodejs version you're using?
from secret-agent.
Windows 10 and nodejs v12.8.0
from secret-agent.
@GoldLine1337 I'm having trouble reproducing, but I'm seeing some issues with resources not being cleaned up properly. Are you seeing this unhandled error on every attempt? Or did it work a few times before it stopped working?
from secret-agent.
I see this error everytime I run secret-agent it doesnt work at all
from secret-agent.
@GoldLine1337 I've fixed some issues with Replay, but I don't think they're causing your main problem. Do you see this after a clean reboot as well?
And is this same computer able to run a basic puppeteer example? (asking because puppeteer also needs to launch Chromium, which is where your code is breaking)
from secret-agent.
Yes puppeteer runs perfectly!
const SecretAgent = require('secret-agent');
(async () => {
const agent = new SecretAgent({ humanEmulatorId: 'ghost' });
await agent.goto('https://example.org');
const title = await agent.document.title;
const intro = await agent.document.querySelector('p').textContent;
await agent.close();
console.log('Retrieved from https://example.org', {
title,
intro,
});
})();
I have tried the example code it also doesnt work.
from secret-agent.
Very weird. Which version of puppeteer did you test with? And I don't think I asked which version you're using.
I'm gonna compare their launch of chrome with ours.
from secret-agent.
Could you try two experiments for me also:
- Try to add to your example above the require
process.env.SA_SHOW_REPLAY = false;
- In your puppeteer test, would you try to see if you experience launch issues when using pipes (
puppeteer.launch({pipe: true});
)
from secret-agent.
Works with pipes also version is [email protected]
from secret-agent.
Ok. Were you able to try the first experiment as well?
process.env.SA_SHOW_REPLAY = 'false';
const SecretAgent = require('secret-agent');
(async () => {
const agent = new SecretAgent({ humanEmulatorId: 'ghost' });
await agent.goto('https://example.org');
const title = await agent.document.title;
const intro = await agent.document.querySelector('p').textContent;
await agent.close();
console.log('Retrieved from https://example.org', {
title,
intro,
});
})();
```
from secret-agent.
after adding that false parameter
I dont get this error
2020-12-18T21:23:34.524Z WARN [C:\Users\GoldLine~HQ.000\node_modules@secret-agent\client\lib\ScriptInstance] Unable to connect to CoreTab { error: 'Error: spawn ENOMEM', context: {} }
but I get
2020-12-18T21:23:34.528Z ERROR [C:\Users\GoldLineHQ.000\node_modules@secret-agent\core\index] UnhandledErrorOrRejection { clientError: 'Error: spawn ENOMEM', context: {} }HQ.000\node_modules@secret-agent\core\index] UnhandledErrorOrRejection { clientError: 'Error: spawn ENOMEM', context: {} }
2020-12-18T21:23:34.529Z ERROR [C:\Users\GoldLine
from secret-agent.
OK, I don't think those told us much :(. Can we try one more experiment?
Try adding the launch arguments to Chromium in Puppeteer (these are the ones we're using in SecretAgent).
puppeteer.launch({
args: [
'--headless',
'--hide-scrollbars',
'--mute-audio',
'--disable-background-networking',
'--enable-features=NetworkService,NetworkServiceInProcess',
'--disable-background-timer-throttling',
'--disable-backgrounding-occluded-windows',
'--disable-breakpad',
'--disable-client-side-phishing-detection',
'--disable-component-extensions-with-background-pages',
'--disable-default-apps',
'--disable-dev-shm-usage',
'--disable-extensions',
'--disable-features=TranslateUI,site-per-process',
'--disable-hang-monitor',
'--disable-ipc-flooding-protection',
'--disable-prompt-on-repost',
'--disable-renderer-backgrounding',
'--disable-sync',
'--disable-gpu',
'--force-color-profile=srgb',
'--use-gl=swiftshader-webgl',
'--use-gl=swiftshader',
'--use-gl=osmesa',
'--metrics-recording-only',
'--no-first-run',
'--enable-automation',
'--remote-debugging-pipe',
'--password-store=basic',
'--use-mock-keychain',
'--ignore-certificate-errors',
'--allow-running-insecure-content',
'--lang=en-US,en;q=0.9',
'--window-size=1920,1080',
'--window-position=0,0',
// don't leak private ip
'--force-webrtc-ip-handling-policy=default_public_interface_only',
// Use proxy for localhost URLs
'--proxy-bypass-list=<-loopback>',
'--no-startup-window',
]
})
from secret-agent.
It works can I disable those with secret-agent?
from secret-agent.
Meaning that crashes it? Could you help figure out which one is crashing you?
from secret-agent.
These are the differences between Secret Agent and Puppeteer launch args:
If I had to guess, it's going to be the webgl ones.
'--disable-features=TranslateUI,site-per-process',
'--disable-gpu',
'--use-gl=swiftshader-webgl',
'--use-gl=swiftshader',
'--use-gl=osmesa',
'--ignore-certificate-errors',
'--allow-running-insecure-content',
'--lang=en-US,en;q=0.9',
'--window-size=1920,1080',
'--window-position=0,0',
// don't leak private ip
'--force-webrtc-ip-handling-policy=default_public_interface_only',
// Use proxy for localhost URLs
'--proxy-bypass-list=<-loopback>',
'--no-startup-window',
from secret-agent.
still works is secret-agent always headless?
from secret-agent.
It runs headless by default, and Replay provides a UI to scrub back and forth with your script. There's a flag to turn on the browser if you need it. We need to have Chrome running though ;)
It looks like puppeteer 2.1.1 is on Chrome 80. Could you try [email protected]? That version runs on Chrome83. Trying to narrow down the differences.
Thanks for your help troubleshooting
from secret-agent.
@GoldLine1337 Were you able to try out puppeteer 3?
from secret-agent.
from secret-agent.
Should I close this out? Are you done trying? I haven't found any way to reproduce this, unfortunately.
from secret-agent.
from secret-agent.
Related Issues (20)
- aws lambda support? HOT 5
- feature-request: provide interface for MITM Proxy HOT 5
- MITM Socket Server is spawned even if `SA_DISABLE_MITM` is used
- example for waiting for a form submit HOT 8
- failed to install `secret-agent` on Mac
- how to set proxy url and get raw html content HOT 6
- Certificate verification failed HOT 1
- EDIT: waitForFileChooser does not work when using userProfile with 2+ sessionStorage domains HOT 6
- feature request: support css string query in waitForElement HOT 4
- TypeError: Cannot read properties of null (reading 'type') HOT 3
- Fingerprint doesn't change HOT 4
- Chrome version updates? HOT 4
- no webgl support in headed mode, Windows VPS. HOT 4
- UnhandledErrorOrRejection
- Problem with cookieStorage and Twitter HOT 1
- SecretAgent is Moving to @ulixee/hero!
- Download path directory HOT 1
- Download path directory change HOT 1
- Clicking on a canvas inside an iframe HOT 6
- TimeoutError: Timeout waiting for child-tab-created
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from secret-agent.