Desktop and Android

Shows up as an empty gap in the waterfalls: https://www.webpagetest.org/result/170905_FC_743699f299f879e2a955cd03e18363b0/1/details/#waterfall_view_step1

Are they generated through VisualMetrics, meaning the video is also wrong? Is there a way to keep/get hold of the original video? I can see the path in the log but I guess it is moved to the server?

The screenshot looks ok:

As I feared, the auto-updating container is pretty fragile.
For example, our agents stopped working at some point, because the npm update command resulted in errors like:

module.js:472
    throw err;
    ^

Error: Cannot find module 'semver'
    at Function.Module._resolveFilename (module.js:470:15)
    at Function.Module._load (module.js:418:25)
    at Module.require (module.js:498:17)
    at require (internal/module.js:20:19)
    at Object.<anonymous> (/usr/lib/node_modules/npm/lib/utils/unsupported.js:2:14)
    at Module._compile (module.js:571:32)
    at Object.Module._extensions..js (module.js:580:10)
    at Module.load (module.js:488:32)
    at tryModuleLoad (module.js:447:12)
    at Function.Module._load (module.js:439:3)

So the whole agent stopped working.

Beside the actual error, this shows that the current update mechanism is very fragile to update problems.

As suggested in my last comment in #19, I would love to switch to a container-based update mechanism which would require a regular build trigger for the image (i.e. nightly).

@pmeenan: Is it possible for you to set up such an automated docker hub build trigger?

Explicitly specified and should support:

• local traffic-shaping for when tethered (rndis or openvpn)
• remote dummynet server
• shelling out to arbitrary script (like current agent)

Hi Pat,
and thanks for adding Mac OS X support, that is great for doing fixes and PRs :) I've started last week getting Docker to work on my Mac but it was a pain since traffic shaping will not work and some configure issues that's been talked about in other issue.

When I try to start my agent on my Mac I get env: python\r and I fixed it by changing newlines with dos2unix wptagent.py but I guess there's a better way to do it so it works out of the box?

Best
Peter

Trying to do a docker build of the latest dockerfile and I am getting the below:

You are using pip version 8.1.1, however version 9.0.1 is available.
You should consider upgrading via the 'pip install --upgrade pip' command.
---> 96276f9afba8
Removing intermediate container 3fc8c7a5d7c3
Step 7/13 : COPY .git /wptagent/.git
lstat .git: no such file or directory

What can I do about that?

The requests are decoded but missing the "start" time (some tweaks needed to the moz log parsing) so they don't show up in the waterfall.

i.e. https://www.webpagetest.org/result/170905_FC_743699f299f879e2a955cd03e18363b0/1/details/#waterfall_view_step1

If usbmuxd isn't running, install all of the necessary components and start it.

This is happening with the latest version of wptagent, alongside ImageMagick with legacy tools on Windows O/S.

There is no test result data.

I get the following stack trace from the wptagent.py:

15:47:55.835 - Connect to dev tools websocket Error: [Errno 10061] No connection
could be made because the target machine actively refused it
15:48:11.118 - Unhandled exception in test run: 'video_file'
Traceback (most recent call last):
File "C:\wptagent-new\wptagent\wptagent.py", line 117, in run_single_test
browser.run_task(self.task)
File "C:\wptagent-new\wptagent\internal\chrome_desktop.py", line 112, in run_t
ask
DevtoolsBrowser.run_task(self, task)
File "C:\wptagent-new\wptagent\internal\devtools_browser.py", line 162, in run
_task
self.on_stop_recording(task)
File "C:\wptagent-new\wptagent\internal\chrome_desktop.py", line 134, in on_st
op_recording
DesktopBrowser.on_stop_recording(self, task)
File "C:\wptagent-new\wptagent\internal\desktop_browser.py", line 261, in on_s
top_recording
if os.path.isfile(task['video_file']):
KeyError: 'video_file'
SUCCESS: Sent termination signal to the process "chrome.exe" with PID 2344.
SUCCESS: Sent termination signal to the process "chrome.exe" with PID 3800.
SUCCESS: Sent termination signal to the process "chrome.exe" with PID 3848.
ERROR: The process "chrome.exe" not found.

As we migrate our iOS test setup to use this WPT Agent we would like to use the remote traffic-shaping option of the agent which you documented here.

Can you give as some hints on how your setup works? E.g.: do you use a specific piece of hardware to run the FreeBSD bridge?

I'm happy to create some documentation or even a walkthrough as we set up everything.

Hey,
with our Samsung Galaxy S7 we experience an issue with reverse tethering. When the phone is restarted, Chrome throws an ERR_INTERNET_DISCONNECTED, although we can execute ping commands via adb shell.
We need to connect and disconnect the phone to a WiFi once after every reboot in order to have a working connection with Chrome.

Do you experience similar issues or know a workaround?

The iWptBrowser app already supports rotating the browser, just need to plumb the support through the agent and the UI.

Use ideviceinstaller to automatically updated the browser from the server.

Hi,
I just wanted to ask what the current status or plans for iOS support are?
Are the official iOS test agents already running with this code, or with the old one?

Thanks for your work!

Hello,

I use wptagent docker image with docker-compose and I notice that this generates defunct processes.

webpagetest-agent1:
image: webpagetest/agent
environment:
SERVER_URL: http://webpagetest/work/
LOCATION: agent1
KEY: key1
NAME: Docker Test
cap_add:
- NET_ADMIN
networks:
- net1

root 5681 5345 0 Oct13 ? 00:00:00 [chrome] <defunct>
root 5682 5345 0 Oct13 ? 00:00:00 [nacl_helper] <defunct>
root 5991 5345 0 Oct13 ? 00:00:03 [chrome] <defunct>
root 6285 5345 0 Oct13 ? 00:00:00 [chrome] <defunct>
root 6286 5345 0 Oct13 ? 00:00:00 [nacl_helper] <defunct>

Maybe, is it possible to improve this?

For main-thread activity and Time to Interactive measurements (already captured, just needs to be processed).

Environment:
macOS Sierra 10.12.4
Linux Headless agent

Using latest wptagent and latest wptserver from repo.

Can I disable traffic shaping? It's not really necessary for my current purposes.
modprobe doesn't exist on mac - the equivalent would be kextutil (I guess)

http://10.0.1.19:32771 "GET /work/getwork.php?f=json&shards=1&location=Local-WPTDriver&pc=Docker&version=170502.200558&screenwidth=1920&screenheight=1200&freedisk=1.117 HTTP/1.1" 200 216
10:59:49.080 - Job: {"processResults":1,"runs":2,"bwIn":0,"plr":0,"latency":0,"url":"https://www.lensway.se","Test ID":"170503_J5_2","Capture Video":1,"IPAddr":"172.17.0.1","bwOut":0,"orientation":"default","lighthouseTrace":1,"timeline":1,"timelineStackDepth":0,"browser":"Chrome"}
10:59:49.081 - [{'record': True, 'command': 'navigate', 'target': u'https://www.lensway.se'}]
10:59:49.093 - Preparing browser
10:59:49.093 - Terminating all instances of chrome
chrome: no process found
10:59:49.098 - Flushing DNS
dnsmasq: unrecognized service
rndc: neither /etc/bind/rndc.conf nor /etc/bind/rndc.key was found
10:59:49.307 - Clearing profile /wptagent/work/Docker/browser.170503_J5_2.1
10:59:49.308 - /opt/google/chrome/chrome --disable-background-networking --no-default-browser-check --no-first-run --process-per-tab --new-window --disable-infobars --disable-translate --disable-notifications --disable-desktop-notifications --disable-save-password-bubble --allow-running-insecure-content --disable-component-update --disable-background-downloads --disable-add-to-shelf --disable-client-side-phishing-detection --disable-datasaver-prompt --disable-default-apps --disable-domain-reliability --safebrowsing-disable-auto-update --disable-background-timer-throttling --disable-sync --host-resolver-rules="MAP cache.pack.google.com 127.0.0.1","MAP clients1.google.com 127.0.0.1" --window-position="0,0" --window-size="1024,768" --remote-debugging-port=9222 --user-data-dir="/wptagent/work/Docker/browser.170503_J5_2.1" --disable-gpu --no-sandbox
10:59:49.335 - Starting new HTTP connection (1): localhost
10:59:49.336 - Connect to dev tools Error: HTTPConnectionPool(host='localhost', port=9222): Max retries exceeded with url: /json (Caused by NewConnectionError('<requests.packages.urllib3.connection.HTTPConnection object at 0x7f0e513d3a10>: Failed to establish a new connection: [Errno 111] Connection refused',))
[819:824:0503/105949.448093:ERROR:bus.cc(427)] Failed to connect to the bus: Failed to connect to socket /var/run/dbus/system_bus_socket: No such file or directory
10:59:49.838 - Starting new HTTP connection (1): localhost
10:59:49.839 - http://localhost:9222 "GET /json HTTP/1.1" 200 374
10:59:49.840 - Dev Tools tabs: [{"description":"","title":"New Tab","url":"chrome://newtab/","webSocketDebuggerUrl":"ws://localhost:9222/devtools/page/f17c2297-71ff-4743-8746-a04d2eb82283","type":"page","id":"f17c2297-71ff-4743-8746-a04d2eb82283","devtoolsFrontendUrl":"/devtools/inspector.html?ws=localhost:9222/devtools/page/f17c2297-71ff-4743-8746-a04d2eb82283"}]
10:59:49.841 - Connect to dev tools websocket Error: [Errno 111] Connection refused
10:59:49.842 - Devtools connected
10:59:49.842 - Sending: {"params":{},"id":1,"method":"Network.clearBrowserCache"}
10:59:49.842 - DevTools websocket connected
10:59:49.844 - {"id":1,"result":{}}
10:59:49.844 - Sending: {"params":{},"id":2,"method":"Network.clearBrowserCookies"}
10:59:49.846 - {"id":2,"result":{}}
10:59:49.846 - Sending: {"params":{"returnByValue":true,"expression":"navigator.userAgent"},"id":3,"method":"Runtime.evaluate"}
10:59:50.114 - {"id":3,"result":{"result":{"type":"string","value":"Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/58.0.3029.96 Safari/537.36"}}}
10:59:50.114 - Sending: {"params":{"url":"about:blank"},"id":4,"method":"Page.navigate"}
10:59:50.282 - {"id":4,"result":{"frameId":"979.1"}}
10:59:50.282 - Waiting for Idle...
10:59:50.889 - Configuring traffic shaping: 0/0 - 0 ms, 0.00% plr
10:59:50.890 - sudo tc qdisc add dev ifb0 root handle 1:0 netem delay 0ms
Cannot find device "ifb0"
10:59:50.899 - Resetting traffic shaping
Cannot find device "ifb0"
10:59:50.908 - DevTools websocket disconnected

I can not reproduce it on my Mac but I guess you can turn it off with privacy.trackingprotection.enabled https://wiki.mozilla.org/Security/Tracking_protection

I'll do a untested PR with that :)

Best
Peter

• Script Commands that will be translated into exec:
  • click (clickAndWait)
  • selectValue
  • sendClick
  • sendKeyDown
  • setInnerHTML
  • setInnerText
  • setValue
  • submitForm
• Other Script Commands:
  • requiredRequest
  • setDOMRequest
  • waitForJSDone (change semantics to console log message)
  • overrideHost (depends on support being added to dev tools)
  • if/else/endif

The console log is already captured, it just needs to be written out in the format that the UI expects.

I'm trying to get wptagent to run on Linux with a private server and ran into an issue where the agent is polling the wrong server URL for work.
Instead of sending requests to /work/getwork.php they are sent to /getwork.php resulting in server errors.

I fixed it locally by changing the path at internal/webpagetest.py:get_test. I'm not sure if this is a generic issue, or said path has changed in the latest WPT server.

As a separate run from the main tests (both desktop and Android)

The traffic-shaping feature doesn't work on Mac under Docker directly right now. However, even when submitting a test with "Connection: Native connection (No traffic shaping)", the agent still tries to set up something related to it and fails the run early.

10:22:50.818 - http://192.168.65.1:4001 "GET /work/getwork.php?f=json&shards=1&location=Test&pc=7010319d4402&screenwidth=1920&screenheight=1200&freedisk=55.338 HTTP/1.1" 200 236
10:22:50.819 - Job: {"processResults":1,"runs":3,"bwIn":0,"plr":0,"latency":0,"url":"https:\/\/test.wikipedia.org\/wiki\/Sandbox","Test ID":"170914_XY_3","Capture Video":1,"IPAddr":"172.17.0.1","bwOut":0,"orientation":"default","fvonly":1,"timeline":1,"lighthouseTrace":1,"timelineStackDepth":0,"browser":"Chrome"}
10:22:50.819 - [{'record': True, 'command': 'navigate', 'target': u'https://test.wikipedia.org/wiki/Sandbox'}]
10:22:50.820 - Preparing browser
10:22:50.820 - Terminating all instances of chrome
10:22:50.826 - Flushing DNS
dnsmasq: unrecognized service
rndc: neither /etc/bind/rndc.conf nor /etc/bind/rndc.key was found
10:22:50.843 - Clearing profile /wptagent/work/7010319d4402/browser.170914_XY_3.1
10:22:50.844 - /opt/google/chrome/chrome --disable-background-networking --no-default-browser-check --no-first-run --process-per-tab --new-window --disable-infobars --disable-translate --disable-notifications --disable-desktop-notifications --disable-save-password-bubble --allow-running-insecure-content --disable-component-update --disable-background-downloads --disable-add-to-shelf --disable-client-side-phishing-detection --disable-datasaver-prompt --disable-default-apps --disable-domain-reliability --safebrowsing-disable-auto-update --disable-background-timer-throttling --host-resolver-rules="MAP cache.pack.google.com 127.0.0.1","MAP clients1.google.com 127.0.0.1" --window-position="0,0" --window-size="1024,768" --remote-debugging-port=9228 --user-data-dir="/wptagent/work/7010319d4402/browser.170914_XY_3.1" --disable-gpu --no-sandbox about:blank
10:22:50.847 - Starting new HTTP connection (1): localhost
10:22:50.848 - Connect to dev tools Error: HTTPConnectionPool(host='localhost', port=9228): Max retries exceeded with url: /json (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f0af59f0410>: Failed to establish a new connection: [Errno 111] Connection refused',))
[1581:1590:0914/102250.965725:ERROR:bus.cc(395)] Failed to connect to the bus: Failed to connect to socket /var/run/dbus/system_bus_socket: No such file or directory

(chrome:1581): LIBDBUSMENU-GLIB-WARNING **: Unable to get session bus: Unknown or unsupported transport 'disabled' for address 'disabled:'
10:22:51.351 - Starting new HTTP connection (1): localhost
10:22:51.353 - http://localhost:9228 "GET /json HTTP/1.1" 200 2189
10:22:51.354 - Dev Tools tabs: [{"description":"","title":"CryptoTokenExtension","url":"chrome-extension:\/\/kmendfapggjehodndflmmgagdbamhnfd\/_generated_background_page.html","webSocketDebuggerUrl":"ws:\/\/localhost:9228\/devtools\/page\/f90cfffc-50d5-40f7-adbd-2d4f5d6a5527","type":"background_page","id":"f90cfffc-50d5-40f7-adbd-2d4f5d6a5527","devtoolsFrontendUrl":"\/devtools\/inspector.html?ws=localhost:9228\/devtools\/page\/f90cfffc-50d5-40f7-adbd-2d4f5d6a5527"},{"description":"","title":"Google Network Speech","url":"chrome-extension:\/\/neajdppkdcdipfabeoofebfddakdcjhd\/_generated_background_page.html","webSocketDebuggerUrl":"ws:\/\/localhost:9228\/devtools\/page\/9eb6a2b1-cdf9-4186-bfd7-71ec5fbd09b3","type":"background_page","id":"9eb6a2b1-cdf9-4186-bfd7-71ec5fbd09b3","devtoolsFrontendUrl":"\/devtools\/inspector.html?ws=localhost:9228\/devtools\/page\/9eb6a2b1-cdf9-4186-bfd7-71ec5fbd09b3"},{"description":"","title":"Feedback","url":"chrome-extension:\/\/gfdkimpbcpahaombhbimeihdjnejgicl\/_generated_background_page.html","webSocketDebuggerUrl":"ws:\/\/localhost:9228\/devtools\/page\/0ac98e70-c662-46d0-8013-30d290dedc68","type":"background_page","id":"0ac98e70-c662-46d0-8013-30d290dedc68","devtoolsFrontendUrl":"\/devtools\/inspector.html?ws=localhost:9228\/devtools\/page\/0ac98e70-c662-46d0-8013-30d290dedc68"},{"description":"","title":"Google Hangouts","url":"chrome-extension:\/\/nkeimhogjdpnpccoofpliimaahmaaome\/background.html","webSocketDebuggerUrl":"ws:\/\/localhost:9228\/devtools\/page\/89bf1205-755c-498b-bb52-288634ab5788","type":"background_page","id":"89bf1205-755c-498b-bb52-288634ab5788","devtoolsFrontendUrl":"\/devtools\/inspector.html?ws=localhost:9228\/devtools\/page\/89bf1205-755c-498b-bb52-288634ab5788"},{"description":"","title":"about:blank","url":"about:blank","webSocketDebuggerUrl":"ws:\/\/localhost:9228\/devtools\/page\/3b53b6d4-4fec-4a32-a20c-303d020236c3","type":"page","id":"3b53b6d4-4fec-4a32-a20c-303d020236c3","devtoolsFrontendUrl":"\/devtools\/inspector.html?ws=localhost:9228\/devtools\/page\/3b53b6d4-4fec-4a32-a20c-303d020236c3"}]
10:22:51.356 - DevTools websocket connected
10:22:51.356 - Devtools connected
10:22:51.357 - Sending: {"params":{"returnByValue":true,"expression":"[window.innerWidth, window.innerHeight]"},"id":1,"method":"Runtime.evaluate"}
10:22:51.365 - {"id":1,"result":{"result":{"type":"object","value":[1024,702]}}}
10:22:51.365 - Sending: {"params":{"returnByValue":true,"expression":"window.devicePixelRatio"},"id":2,"method":"Runtime.evaluate"}
10:22:51.366 - {"id":2,"result":{"result":{"type":"number","value":1,"description":"1"}}}
10:22:51.367 - Sending: {"params":{},"id":3,"method":"Network.clearBrowserCache"}
10:22:51.370 - {"id":3,"result":{}}
10:22:51.370 - Sending: {"params":{},"id":4,"method":"Network.clearBrowserCookies"}
10:22:51.387 - {"id":4,"result":{}}
10:22:51.388 - Sending: {"params":{"returnByValue":true,"expression":"navigator.userAgent"},"id":5,"method":"Runtime.evaluate"}
10:22:51.396 - {"id":5,"result":{"result":{"type":"string","value":"Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/61.0.3163.79 Safari/537.36"}}}
10:22:51.396 - Sending: {"params":{"waitForDebuggerOnStart":true,"autoAttach":true},"id":6,"method":"Target.setAutoAttach"}
10:22:51.396 - Sending: {"params":{},"id":7,"method":"Target.getTargets"}
10:22:51.398 - {"id":7,"result":{"targetInfos":[{"targetId":"f90cfffc-50d5-40f7-adbd-2d4f5d6a5527","type":"background_page","title":"CryptoTokenExtension","url":"chrome-extension://kmendfapggjehodndflmmgagdbamhnfd/_
10:22:51.398 - {u'url': u'chrome-extension://kmendfapggjehodndflmmgagdbamhnfd/_generated_background_page.html', u'type': u'background_page', u'attached': False, u'targetId': u'f90cfffc-50d5-40f7-adbd-2d4f5d6a5527', u'title': u'CryptoTokenExtension'}
10:22:51.398 - {u'url': u'chrome-extension://neajdppkdcdipfabeoofebfddakdcjhd/_generated_background_page.html', u'type': u'background_page', u'attached': False, u'targetId': u'9eb6a2b1-cdf9-4186-bfd7-71ec5fbd09b3', u'title': u'Google Network Speech'}
10:22:51.399 - {u'url': u'chrome-extension://gfdkimpbcpahaombhbimeihdjnejgicl/_generated_background_page.html', u'type': u'background_page', u'attached': False, u'targetId': u'0ac98e70-c662-46d0-8013-30d290dedc68', u'title': u'Feedback'}
10:22:51.399 - {u'url': u'chrome-extension://nkeimhogjdpnpccoofpliimaahmaaome/background.html', u'type': u'background_page', u'attached': False, u'targetId': u'89bf1205-755c-498b-bb52-288634ab5788', u'title': u'Google Hangouts'}
10:22:51.399 - {u'url': u'about:blank', u'type': u'page', u'attached': True, u'targetId': u'3b53b6d4-4fec-4a32-a20c-303d020236c3', u'title': u'about:blank'}
10:22:51.399 - Sending: {"params":{"url":"data:text\/html,%3Chtml%3E%0D%0A%3Chead%3E%0D%0A%3Cstyle%3E%0D%0Abody%20%7Bbackground-color%3A%20white%3B%20margin%3A%200%3B%7D%0D%0A%23o%20%7Bwidth%3A100%25%3B%20height%3A%20100%25%3B%20background-color%3A%20%23DE640D%3B%7D%0D%0A%3C%2Fstyle%3E%0D%0A%3Cscript%3E%0D%0Awindow.addEventListener%28%27beforeunload%27%2C%20function%28%29%20%7B%0D%0A%20%20var%20o%20%3D%20document.getElementById%28%27o%27%29%0D%0A%20%20o.parentNode.removeChild%28o%29%3B%0D%0A%7D%29%3B%0D%0A%3C%2Fscript%3E%0D%0A%3C%2Fhead%3E%0D%0A%3Cbody%3E%3Cdiv%20id%3D%27o%27%3E%3C%2Fdiv%3E%3C%2Fbody%3E%0D%0A%3C%2Fhtml%3E"},"id":8,"method":"Page.navigate"}
10:22:51.400 - {"id":6,"result":{}}
10:22:51.403 - {"id":8,"result":{"frameId":"1687.1"}}
10:22:51.403 - Waiting for Idle...
10:22:52.016 - Configuring traffic shaping: 0/0 - 0 ms, 0.00% plr
10:22:52.016 - sudo tc qdisc add dev ifb0 root handle 1:0 netem delay 0ms
Cannot find device "ifb0"
10:22:52.025 - Resetting traffic shaping
Cannot find device "ifb0"
10:22:52.034 - DevTools websocket disconnected
10:22:52.035 - Starting new HTTP connection (1): localhost
10:22:52.036 - http://localhost:9228 "GET /json/close/3b53b6d4-4fec-4a32-a20c-303d020236c3 HTTP/1.1" 200 17
10:22:52.040 - Stopping browser
10:22:52.040 - Terminating all instances of chrome
[0914/102252.045258:ERROR:nacl_helper_linux.cc(310)] NaCl helper process running without a sandbox!
Most likely you need to configure your SUID sandbox correctly
10:22:52.051 - Waiting up to 30 seconds for chrome to exit
10:22:52.088 - Test run time: 1.269 sec
10:22:52.088 - Uploading result
10:22:52.089 - Uploading result zip
10:22:52.089 - http://192.168.65.1:4001/work/workdone.php?run=1&cached=0&pc=7010319d4402&location=Test&error=Error+configuring+traffic-shaping&id=170914_XY_3&
10:22:52.117 - http://192.168.65.1:4001 "POST /work/workdone.php?run=1&cached=0&pc=7010319d4402&location=Test&error=Error+configuring+traffic-shaping&id=170914_XY_3& HTTP/1.1" 200 0
10:22:52.118 - [{'record': True, 'command': 'navigate', 'target': u'https://test.wikipedia.org/wiki/Sandbox'}]
10:22:52.119 - Preparing browser
10:22:52.119 - Terminating all instances of chrome

Environment:

# Server
docker run -p 4001:80 --rm webpagetest/server

# Agent
docker run --rm \
-e SERVER_URL="http://192.168.65.1:4001/work/" \
-e LOCATION="Test" \
--cap-add=NET_ADMIN \
webpagetest/agent

Steps to reproduce:

1. http://localhost:4001/
2. Submit test with connection set to native.
3. Run result is empty, agent console showing above error.

Preferably from on-device but worst-case at least when tethered, collect stats from host machine

Hi Pat,
been trying out WPT agent and seems like tests take a long time to run if we push them after each other.

I have case where we test 9 URLs, we push them to the server, running 1 run per URL, then when we got the result back we run 9 more URLs and then after them 9 more. They all run on the same (AWS) Linux agent, testing both Chrome & Firefox (and emulated mobile). The first batch of 9 URLs takes something like 6 minutes, but when we starts with the next it usually starts to take time, so the 27 URLs can take 1 - 1 1/2 hour to run. But they should be finished in 18 min or so (with a couple of minutes overhead at most right?).

I suspect it could be the wait_for_idle function (do the timeout really work?). When I was demoing WPTAgent for my teammates I got into problems that the agent never could starts since it wasn't idle but that was ok since I'm running on my local machine. Right now we use the AWS images I haven't checked the logs, but I guess I can check if I can login to the machine.

Maybe this is something you already seen on WebPageTest.org Linux agents?

Best
Peter

Hi I'm not sure if Firefox and IE are supported yet but I'm getting errors when trying to run tests on these browsers (Chrome is working fine).

Firefox and IE:
The console output of wptagent keeps repeating this message:
"Connect to dev tools Error: HTTPConnectionPool(host='localhost',port=9223): Max retries exceeded with url: /json (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x0358C970>: Failed to establish a new connection: [Errno 10061] No connection could be made because the target machine actively refused it',))"

Chrome (FYI):
The console output of wptagent displays this message:
"Connect to dev tools websocket Error: [Errno 10061] No connection could be made because the target machine actively refused it"

Firefox:
The browser loads but it won't navigate to the test url.

IE11:
The browser loads but it always navigates to this url (I'm guessing it's bing.com because that's the default search engine on IE):
"https://www.bing.com/search?q=--disable-background-networking+--no-default-browser-check+--no-first-run+--process-per-tab+--new-window+--disable-infobars+--disable-translate+--disable-notifications+--disable-desktop-notifications+--disable-save-password-bubble+--allow-running-insecure-content+--disable-component-update+--disable-background-downloads+--disable-add-to-shelf+--disable-client-side-phishing-detection+--disable-datasaver-prompt+--disable-default-apps+--disable-domain-reliability+--safebrowsing-disable-auto-update+--disable-background-timer-throttling+--host-resolver-rules%3D%22MAP+cache.pack.google.com+127.0.0.1%22,%22MAP+clients1.google.com+127.0.0.1%22+--window-position%3D%220,0%22+--window-size%3D%221024,768%22+--remote-debugging-port%3D9224+--user-data-dir%3D%22C%3A%5Cwptagent-new%5Cwptagent%5Cwork%5CWPT-Test22%5Cbrowser.170721_DW_3.1%22+--no-sandbox+about%3Ablank&src=IE-TopResult&FORM=IETR02&conversationid="

I added Firefox and IE11 to browsers.ini as follows:

[IE11]
exe=C:/Program Files (x86)/Internet Explorer/iexplore.exe

[Firefox]
exe=C:/Program Files (x86)/Mozilla Firefox/firefox.exe

Hi Pat,
seen we are running Chrome 60 on our agent I guess the agents aren't autoupdating the browser right? What do you think is the best way to handle it in the future: Run on a Linux (AWS or whatever) and then upgrade ourself or will that be built in the agent so we can the AWS images?

Best
Peter

websocket connection issue. Not able to connect websocket.

In my web application first browser make a socket connection then all http requests sending across this websocket. But when i run my application through WPT browser its not creating a socket connection.

When trying to run a test with the wptagent on linux, I'm getting devtools connection errors:
Connect to dev tools Error: HTTPConnectionPool(host='localhost', port=9222): Max retries exceeded with url: /json (Caused by NewConnectionError('<requests.packages.urllib3.connection.HTTPConnection object at 0x7fe0a80da0d0>: Failed to establish a new connection: [Errno 111] Connection refused',))

These errors repeat themselves and the run never completes.

Pointers on how to further debug this would be appreciated :)

Same as catchpoint/WebPageTest#934 (comment) but for a wptagent on Windows 10.

Hi Pat,
sorry for adding all the issues, I'm testing all our tests now on the agent to see if it would work to switch.

One of our tests access one URL and then the other:

logData 0
navigate https://en.wikipedia.org/wiki/Edward_VI_of_England

// Erase the current HTML to make the video easier to understand
exec document.body.innerHTML = "";
sleep	30

logData 1
navigate https://en.wikipedia.org/wiki/Elizabeth_I_of_England

The problem is that the first URL is picked up in the waterfall (as the second request) - http://wpt.wmftest.org/result/170904_MC_AB/1/details/#waterfall_view_step1:

Best
Peter

Looks like the devtools.screenshot method of recording is hard-coded to a 450 frame limit which can be as short as 7.5 seconds at 60fps.

Video capture for Chrome should be moved to use the same methods as for Firefox (and be independent of Chrome itself).

Hi,

we are getting multiple occurrences of the following error messages in /var/log/syslog after starting the wptagent on our raspberry pi 3 running Raspbian Stretch Lite released 2017-09-07 with a Samsung S7 attached via USB:

Nov  7 12:26:56 raspberrypi kernel: [ 3425.359044] eth0: hw csum failure
Nov  7 12:26:56 raspberrypi kernel: [ 3425.359053] CPU: 0 PID: 0 Comm: swapper/0 Not tainted 4.9.41-v7+ #1023
Nov  7 12:26:56 raspberrypi kernel: [ 3425.359058] Hardware name: BCM2835
Nov  7 12:26:56 raspberrypi kernel: [ 3425.359073] [<8010fb3c>] (unwind_backtrace) from [<8010c058>] (show_stack+0x20/0x24)
Nov  7 12:26:56 raspberrypi kernel: [ 3425.359087] [<8010c058>] (show_stack) from [<80455f84>] (dump_stack+0xd4/0x118)
Nov  7 12:26:56 raspberrypi kernel: [ 3425.359102] [<80455f84>] (dump_stack) from [<806282dc>] (netdev_rx_csum_fault+0x44/0x48)
Nov  7 12:26:56 raspberrypi kernel: [ 3425.359116] [<806282dc>] (netdev_rx_csum_fault) from [<8061af08>] (__skb_checksum_complete+0xb4/0xb8)
Nov  7 12:26:56 raspberrypi kernel: [ 3425.359132] [<8061af08>] (__skb_checksum_complete) from [<80691110>] (tcp_v4_rcv+0x4fc/0xc10)
Nov  7 12:26:56 raspberrypi kernel: [ 3425.359148] [<80691110>] (tcp_v4_rcv) from [<806698dc>] (ip_local_deliver_finish+0xe4/0x330)
Nov  7 12:26:56 raspberrypi kernel: [ 3425.359163] [<806698dc>] (ip_local_deliver_finish) from [<8066a140>] (ip_local_deliver+0x54/0xe4)
Nov  7 12:26:56 raspberrypi kernel: [ 3425.359177] [<8066a140>] (ip_local_deliver) from [<80669d5c>] (ip_rcv_finish+0x234/0x48c)
Nov  7 12:26:56 raspberrypi kernel: [ 3425.359191] [<80669d5c>] (ip_rcv_finish) from [<8066a4ec>] (ip_rcv+0x31c/0x55c)
Nov  7 12:26:56 raspberrypi kernel: [ 3425.359205] [<8066a4ec>] (ip_rcv) from [<80625d54>] (__netif_receive_skb_core+0x2b4/0xbc0)
Nov  7 12:26:56 raspberrypi kernel: [ 3425.359218] [<80625d54>] (__netif_receive_skb_core) from [<8062862c>] (__netif_receive_skb+0x20/0x7c)
Nov  7 12:26:56 raspberrypi kernel: [ 3425.359232] [<8062862c>] (__netif_receive_skb) from [<806286b4>] (netif_receive_skb_internal+0x2c/0xa4)
Nov  7 12:26:56 raspberrypi kernel: [ 3425.359245] [<806286b4>] (netif_receive_skb_internal) from [<80628750>] (netif_receive_skb+0x24/0x98)
Nov  7 12:26:56 raspberrypi kernel: [ 3425.359263] [<80628750>] (netif_receive_skb) from [<7f3b6514>] (ifb_ri_tasklet+0xf4/0x29c [ifb])
Nov  7 12:26:56 raspberrypi kernel: [ 3425.359281] [<7f3b6514>] (ifb_ri_tasklet [ifb]) from [<80123250>] (tasklet_action+0x74/0x10c)
Nov  7 12:26:56 raspberrypi kernel: [ 3425.359294] [<80123250>] (tasklet_action) from [<8010169c>] (__do_softirq+0x18c/0x3cc)
Nov  7 12:26:56 raspberrypi kernel: [ 3425.359307] [<8010169c>] (__do_softirq) from [<80122cd8>] (irq_exit+0x10c/0x168)
Nov  7 12:26:56 raspberrypi kernel: [ 3425.359320] [<80122cd8>] (irq_exit) from [<801736b8>] (__handle_domain_irq+0x70/0xc4)
Nov  7 12:26:56 raspberrypi kernel: [ 3425.359333] [<801736b8>] (__handle_domain_irq) from [<8010150c>] (bcm2836_arm_irqchip_handle_irq+0xa8/0xac)
Nov  7 12:26:56 raspberrypi kernel: [ 3425.359347] [<8010150c>] (bcm2836_arm_irqchip_handle_irq) from [<8071a47c>] (__irq_svc+0x5c/0x7c)
Nov  7 12:26:56 raspberrypi kernel: [ 3425.359352] Exception stack(0x80c01f18 to 0x80c01f60)
Nov  7 12:26:56 raspberrypi kernel: [ 3425.359359] 1f00:                                                       00000000 ba35c470
Nov  7 12:26:56 raspberrypi kernel: [ 3425.359371] 1f20: 00000000 00000000 80c00000 80c0312c 00000001 80c03198 80c15f08 80c15f08
Nov  7 12:26:56 raspberrypi kernel: [ 3425.359382] 1f40: babffa80 80c01f74 80c040a4 80c01f68 80108a54 80108a58 60000013 ffffffff
Nov  7 12:26:56 raspberrypi kernel: [ 3425.359397] [<8071a47c>] (__irq_svc) from [<80108a58>] (arch_cpu_idle+0x34/0x4c)
Nov  7 12:26:56 raspberrypi kernel: [ 3425.359411] [<80108a58>] (arch_cpu_idle) from [<80719bc4>] (default_idle_call+0x34/0x48)
Nov  7 12:26:56 raspberrypi kernel: [ 3425.359423] [<80719bc4>] (default_idle_call) from [<80161df0>] (cpu_startup_entry+0xe4/0x160)
Nov  7 12:26:56 raspberrypi kernel: [ 3425.359438] [<80161df0>] (cpu_startup_entry) from [<80714450>] (rest_init+0x84/0x88)
Nov  7 12:26:56 raspberrypi kernel: [ 3425.359453] [<80714450>] (rest_init) from [<80b00d28>] (start_kernel+0x3b4/0x3c0)

Steps to reproduce:

• Configure passwordless sudo for user pi with sudo visudo
• sudo apt-get update
• sudo apt-get install python2.7 python-dev python-pip imagemagick ffmpeg cgroup-tools libjpeg-dev traceroute git android-tools-adb
• sudo pip install dnspython monotonic pillow psutil requests ujson
• git clone https://github.com/WPO-Foundation/wptagent.git
• python wptagent/wptagent.py --android --server http://path.to.wpt.server.de/work/ --location SamsungGalaxyS7

This looks like some kernel problem but maybe you still have an idea how it is caused.

Both explicit IP config and dhcp (maybe prefer dhcp)

Hi,

I was trying to run the WPT Agent on a Raspberry Pi 3 (Model B V1.2) with Raspbian Stretch Lite (release date: 2017-09-07, kernel version: 4.9) for an iPhone 7 with iOS 11. During the setup I encountered some stumbling blocks.

The source of the problem seem to be libimobiledevice and ios-webkit-debug-proxy. Recently there were some issues related to iOS 11 in both of the projects. E.g.:

• libimobiledevice/libimobiledevice#510
• google/ios-webkit-debug-proxy#231

Following the hints given in those issues, I compiled the latest version of libimobiledevice and ios-webkit-debug-proxy directly from the corresponding repositories.
After replacing the binaries within WPT Agent with the freshly compiled ones, measurements worked fine with an iPhone running iOS 11.

But I'm still not sure about the exact source of the problem. Which Debian version do you use for your Pi and did you encounter similar problems?

Hey Pat,
we use a script like this to login the user:

// The login page
logData 0
navigate https://en.wikipedia.org/w/index.php?title=Special:UserLogin&returnto=Main+Page

// Log in the user
setValue        name=wpName     <%WPT_USER>
setValue        name=wpPassword <%WPT_USER_PASSWORD>
submitForm      name=userlogin

// Erase the current HTML to make the video easier to understand
exec document.body.innerHTML = "";
sleep	1

// This is the URL that we want to measure as a logged in user
logData 1
navigate https://en.wikipedia.org/wiki/Facebook

And on Linux/Firefox the login URL gets picked up in the waterfall:

http://wpt.wmftest.org/result/170904_0B_9H/1/details/#waterfall_view_step1:

On Firefox on Windows the old agent it looks ok:

Best
Peter

Already tried a stand-alone VPN extension/app and usbmux but the extension can not create a listening socket that the outside can connect to.

Ideas to try:

1. VPN extension connect to iWptBrowser on localhost which proxies the packets over usbmuxd
2. Do something like Simplert where it advertises a USB accessory and send data over that

Firefox Nightly 56 just added a window.performance.timing.timeToNonBlankPaint (Firefox bug 1377251)
that is akin to Chrome's window.chrome.loadTimes().firstPaintTime. And IE and Edge have window.performance.timing.msFirstPaint (MSDN).

wptagent can use Firefox's timeToNonBlankPaint and IE/Edge's msFirstPaint to report pageData["firstPaint"] like it does for chromeTimes["firstPaintTime"] (in page_data.js#L25-L32).

Note that Firefox's timeToNonBlankPaint is not standard and requires that the Firefox user profile set the about:config pref dom.performance.time_to_non_blank_paint.enabled = true. Also timeToNonBlankPaint, unlike the other performance.timing properties, is the millisecond duration since performance.timing.navigationStart, not a timestamp relative to the UNIX epoch.

CC @digitarald

http://wpt.wmftest.org/result/170904_3D_7P/1/details/#waterfall_view_step1

http://wpt.wmftest.org/result/170904_Q4_7D/1/details/#waterfall_view_step1

Hello,

If I look documentation of webpagetest api, I found blockAds option.

blockAds: Boolean, block ads defined by http://adblockplus.org

When I use this option with waptagent, it does not seem to work. Can you implement it?

I use latest wptagent docker image.

I tested with both Firefox and Chrome and got:
startedDateTime": "1970-01-01T00:00:00.000+00:00"

See this test for an example.

Android and desktop (downloads in particular)