zpeters / speedtest Goto Github PK

Hello,
i am using your speedtest to messure my internetspeed (obvious) and hand it's output over to nagios (a monitoring system).
From time to time my uploadtest takes really long (my provider isn't really qualified sometimes...), so the nagios client will give back a timeout, but at this point the download speedtest was already done and I am loosing this number. Also the nagios will do a quicker recheck and this causes an unnecessary load.

At this point i am getting a critial mail from my nagios system and can't see why.
I like to split the messure of up- and download in two commands. So I can change notification time and recheck times differently.

Is there a way you could implement this?

I REALLY like the idea of this tool, I've been looking for a way to keep Comcast honest. After I thought I had the config correct and created a scheduled task on Windows... I kept getting odd results for the download speed. It wasn't until I changed to another server that I got the right result again. Example:

C:>C:\temp\SpeedTest\speedtest-64-v0.07.2-xcompile.exe -s 3165
3165 | Georgia Institute of Technology (Atlanta, GA, United States)
Testing latency...
2015/04/03 18:07:28 Testing download speed
.........
2015/04/03 18:07:49 Testing upload speed
.....
Ping: 96.81 ms | Download: 42.09 Mbps | Upload: 10.37 Mbps

C:>C:\temp\SpeedTest\speedtest-64-v0.07.2-xcompile.exe -s 3165
3165 | Georgia Institute of Technology (Atlanta, GA, United States)
Testing latency...
2015/04/03 18:09:37 Testing download speed
.........
2015/04/03 18:09:45 Testing upload speed
.....
Ping: 23.80 ms | Download: 574.53 Mbps | Upload: 10.72 Mbps

*** Changing to another server, then changing back doesn't seem to fix the issue. Example - I previously used server 5027 | AT&T (Tucker, GA, United States), and I am still getting erroneous results even after trying server 3165. The download speed doesn't even seem to be a factor off (like 10x). Changing to another server I get the same affect - first works, second doesn't:
C:>C:\temp\SpeedTest\speedtest-64-v0.07.2-xcompile.exe -s 1767
1767 | Comcast (Atlanta, GA, United States)
Testing latency...
2015/04/03 18:15:40 Testing download speed
.........
2015/04/03 18:16:07 Testing upload speed
.....
Ping: 47.40 ms | Download: 35.67 Mbps | Upload: 11.56 Mbps

C:>C:\temp\SpeedTest\speedtest-64-v0.07.2-xcompile.exe -s 1767
1767 | Comcast (Atlanta, GA, United States)
Testing latency...
2015/04/03 18:16:19 Testing download speed
.........
2015/04/03 18:16:29 Testing upload speed
.....
Ping: 20.20 ms | Download: 621.10 Mbps | Upload: 11.53 Mbps

Its almost like the random data it should be downloading isn't random anymore and is getting cached.... which led me to check and in fact, if I turn off my web cache in my router, the speeds return to normal.... so methinks I need to put a rule into my router unless you can have the tool download random data like testmy.net. ;) Thanks!

I compile and run it in my RT-AC68U router (2 core ARM, 256M ram), than It got out of memory:

admin@RT-AC68U-7DB0:/tmp/home/root# /tmp/mnt/sda1/asus/speedtest
github.com/zpeters/speedtest -- unofficial cli for speedtest.net
2015/08/10 10:43:10 Testing download speed
.......fatal error: runtime: out of memory

What can I do, if i want to reduce the memory of this program used.

Look into using https://github.com/codegangsta/cli

for some reason the --rc switch disables reporting

automatically checking for latest version was hitting api limits. Turned this into an optional switch feature

Feedback from [email protected]

https://github.com/zpeters/speedtest
Is very neat and certainly is in the spirit of https://github.com/sivel/speedtest-cli

Though I find that neither are super-reliable above a few hundred Mbps. Likely because they do not sustain a transfer for more than a second. (Ookla will transfer for 10s or more even on connections of 500Mbps and above).

Ookla uses a custom server and claim this allows better tests in excess of 200Mbps
(See http://www.ookla.com/support/a22990987/NetGauge-HTTP-Legacy-Fallback)
Client is flash (unless on mobile).

Though not described, the new test protocol does not look too complicated (though huge disclaimer, I have not figured the ins and outs of it).

Attached are some captures of testing on an 80Mbps connection (only download phase).

• Speedtest starts 15 TCP streams bt client and testserver over course of test.
• First starts at 8.2s, next 3 start 0.6s later, next 8 start 0.6s later
  • First may be test stream
  • Or may simply space out each connection start
• All connections terminate at roughly same time -- 10s after first stream started
  Inline image 2

Most streams look like format:
HI
HELLO 2.2 2014-06-01.01
DOWNLOAD 729604
DOWNLOAD JABCDEF...

The first stream started has a slightly different format:
HI
HELLO 2.2 2014-06-01.01
PING 1441239250094
PONG 1441239250175
PING 1441239250104
PONG 1441239250188
PING 1441239250167
PONG 1441239250250
PING 1441239250178
PONG 1441239250258
PING 1441239250231
...
HI
HELLO 2.2 2014-06-01.01
DOWNLOAD 1000000
DOWNLOAD JABCDEF...

Ping-pong timestamp is correct unix time including milliseconds.

Observations

• Once the ping-pong stops, never returns to ping-pong
• In some streams, download number changes
  • May represent current throughput
    745959 mean 74.5959 Mbps? So server can capture data if test aborts?
  • May represent something else
• Do not see close signal for stream
  • Client simply closes tcp-stream/socket to end transfer?
• Client only receives ABCDEFGHIJ from server
  • At least from this testserver (Acelx)

And that's all I know (at the moment).

So it seems to me it may be possible to connect to these testservers and get a long, 10s TCP stream (or several) and this would make your test more robust (by having 10s+ of duration).

Ookla claims this works up to several Gbps: http://blog.ookla.com/2011/06/16/breaking-the-1-gbps-barrier-over-cable/

Other option, I suppose, would be to download the random JPGs many times in parallel.

• Here you are at the mercy of the speedtest testserver's http server
• Run several captures, its bring your own http server
  • And also, bring your own PHP/ASP/JSP

Anyway an interesting thing. Very cool project, I'll use it in <200M locations. If you get bored and want to connect to the new Ookla servers, now you've got some data.

https://github.com/briandowns/spinner

Research this

Create an output format similar to the "-q" switch with will just output the data in CSV (or some user specified format) for easy data collection. Use fine grain units for better accuracy

Some functions need to be moved to their own modules

Cannot get servers list from speedtest.net: 'Cannot unmarshal XML'

Do I need to install go-env to make it work or just chmod +x speedtest and ./speedtest it?

The server is centos 6.5 and I got the linux arm64 version, my server is on aws. Thanks!

A week ago i started using git flow. Once bugs are fixed i need to do a "proper" release, this may bump version numbers.

Review switches and cleanup

Add wercker?

https://app.wercker.com/sessions/new

Might be easier to manager

Right now speedtest cli uses an averaging algorithm for determining upload and download speeds. I'd like to add a "max" and possibly others to get the results closer to peoples expectations

Zachs-MacBook-Pro:speedtest zachpeters$ ./speedtest -p
4471 | Mississippi State University (Mississippi State, MS, United States)
Ping (Lowest): 32.81 ms
Zachs-MacBook-Pro:speedtest zachpeters$ ./speedtest -p -s 4471
4471 | Mississippi State University (Mississippi State, MS, United States)
Ping (Lowest): 0.00 ms
Zachs-MacBook-Pro:speedtest zachpeters$

Add license verbiage

speedtest.go - downloadTest()

Hello,

I'm using your speedtest command line tool for windows. Thanks for your work.

It looks like I have some problems with the latest builds, expecially with the -r and -s options. Until a few days ago I was using a 32 bit release (v0.7 if I remember correctly) and it was working good. Now I've upgrade to the latest 32 (v.0.7.5) and 64 bit (v0.8.1-2) builds and I have problems.

I've attached some screenshots where you can see the output I get with the different versions (latest x64, latest x86, v0.7 x86).
This was tested on XP x64, but I've also tried on a win7 x64 virtual machine and it's the same.

Please let me know if I can help and thanks for your work.

Best regards.

Numerous reports and observations of inaccurate latency. Sometimes latency is several hundred ms, testing through other means shows it should up < 100.

when loading speedtest-config.php or speedtest-servers-static.php is shows a blank page

but after debugging speedtest.net, it appears you need to load the page with the extra variable of x=RANDOMNUMBER

e.g: http://c.speedtest.net/speedtest-config.php?x=1442587869539
e.g: http://c.speedtest.net/speedtest-servers-static.php?x=1442587869539

or load from these urls without the x variable:
https://www.speedtest.net/speedtest-config.php
https://www.speedtest.net/speedtest-servers-static.php

sthttp.go GetLatency

In the scenario where we are checking multiple servers we want to "silently" skip servers that fail (ie are offline) but not kill execution. Currently we are setting their latency to 1 minute which will force them to drop to the end of our list. Seems like there should be a cleaner way to do this

Hi,

I'm trying to use speedtest to analyse my internet-bandwidth over a longer period of time. So I'm using the tool through a Windows-schedule, in command-line mode. The output is put in a output text-file.

I'm trying to use the output-file with a visualisation-tool.

The problem I have is that the lines in the output-file are not separated by a (As in a normal text-file). But by

Is there any way of having that changed ?

Many thanks

Eddy

Hi,

I don't know if it is possible but can you add an option to choose between wired or wireless connection to make the speed test ?

Why do I want to choose wireless connection ? Just to tell my ISP (on Twitter) that their hardware is crappy.

Imports like ./debug fail when fetching packages with go get. Example:

[vendion@Baldr][~]% go get github.com/zpeters/speedtest
/home/vendion/gocode/src/github.com/zpeters/speedtest/speedtest.go:19:2: local import "./debug" in non-local package

Instead those imports should be defined as github.com/zpeters/speedtest/debug that way when a user runs go get github.com/zpeters/speedtest it will know where to fetch those other packages. Another option if you don't need to support Go 1.4 or older you can make those other packages an "internal dependency" by creating an internal directory and moving the debug, print, settings, sthttp, tests under that directory then the import line can be changed to speedtest/internal/debug and so on.

Also small nitpick but instead of having multiple import statements in your source code, it is more idiomatic in Go to have one import statement and group standard lib packages, internal/vendored packages, and 3rd party packages separating them by a new line. See the Imports section of the Code Review Comments page of the Go wiki for details. It is recommended to install goimports and either set your editor to run that on save or create a git hook that runs it when commiting changes or before pushing. Goimports handles everything that gofmt does plus manages your import section adding new packages you use, and removing unused packages as well as correctly groups them.

speedtest.go - uploadTest

Needed for easier management of debugging logging and possibly logging fatal crashes to pager duty

move to https://github.com/mgutz/logxi/issues

[14:40] # ./speedtest -d
2013/07/07 14:40:24 Debugging on...
Loading config from speedtest.net
2013/07/07 14:40:24 Getting config...
Getting servers list...2013/07/07 14:40:24 Getting servers...
2013/07/07 14:40:24 Found 2191 servers...
(2191) found
2013/07/07 14:40:24 Finding 3 closest servers...
Finding fastest server...2013/07/07 14:40:24 Testing latency: Cleveland, Ohio (Cleveland State University)
Run 0 took: 10.988ms
Run 1 took: 2.71ms
Run 2 took: 2.295ms
2013/07/07 14:40:24 Total runs took: 15.993ms
2013/07/07 14:40:24 Testing latency: Detroit, MI (Comcast)
Run 0 took: 48.581ms
Run 1 took: 24.022ms
Run 2 took: 20.911ms
2013/07/07 14:40:24 Total runs took: 93.514ms
2013/07/07 14:40:24 Testing latency: Detroit, MI (Wayne State University)
2013/07/07 14:40:24 Error: Get http://speedtest.wayne.edu/speedtest/latency.txt: dial tcp 141.217.6.42:80: connection refused
panic: Error: Get http://speedtest.wayne.edu/speedtest/latency.txt: dial tcp 141.217.6.42:80: connection refused

goroutine 1 [running]:
log.Panicf(0x5d1f84, 0x6f7272450000000a, 0x7f46d4752a90, 0x100000001, 0x7f46d4752a90, ...)
/usr/lib/go/src/pkg/log/log.go:314 +0xb3
speedtest/misc.E(0xf840224cc0, 0xf840224c00)
/home/hhs8/speedtest-cli/speedtest/src/speedtest/misc/misc.go:13 +0xa7
speedtest/sthttp.GetFastestServer(0x3, 0xf840585000, 0x9c400000003, 0x0, 0x0, ...)
/home/hhs8/speedtest-cli/speedtest/src/speedtest/sthttp/sthttp.go:189 +0x29d
main.main()
/home/hhs8/speedtest-cli/speedtest/src/speedtest/speedtest.go:115 +0x21f

goroutine 2 [syscall]:
created by runtime.main
/build/buildd/golang-1/src/pkg/runtime/proc.c:221

goroutine 3 [syscall]:
syscall.Syscall6()
/build/buildd/golang-1/src/pkg/syscall/asm_linux_amd64.s:40 +0x5
syscall.EpollWait(0xf800000006, 0xf8400b30c0, 0xa0000000a, 0xffffffff, 0xc, ...)
/usr/lib/go/src/pkg/syscall/zerrors_linux_amd64.go:1781 +0xa1
net.(_pollster).WaitFD(0xf8400b30b0, 0xf8400b21c0, 0x0, 0x0, 0x0, ...)
/usr/lib/go/src/pkg/net/fd_linux.go:146 +0x110
net.(_pollServer).Run(0xf8400b21c0, 0x0)
/usr/lib/go/src/pkg/net/fd.go:236 +0xe4
created by net.newPollServer
/usr/lib/go/src/pkg/net/newpollserver.go:35 +0x382

goroutine 4 [chan receive]:
net.(_pollServer).WaitRead(0xf8400b21c0, 0xf840078360, 0xf840057030, 0xb, 0x1, ...)
/usr/lib/go/src/pkg/net/fd.go:268 +0x73
net.(_netFD).Read(0xf840078360, 0xf8400c0000, 0x100000001000, 0xffffffff, 0xf840059210, ...)
/usr/lib/go/src/pkg/net/fd.go:428 +0x1ec
net.(_TCPConn).Read(0xf840072330, 0xf8400c0000, 0x100000001000, 0x9a000000000, 0x0, ...)
/usr/lib/go/src/pkg/net/tcpsock_posix.go:87 +0xce
bufio.(_Reader).fill(0xf8400b2240, 0xf840001d70)
/usr/lib/go/src/pkg/bufio/bufio.go:77 +0xf0
bufio.(_Reader).Peek(0xf8400b2240, 0xf800000001, 0xf8400af101, 0x0, 0x0, ...)
/usr/lib/go/src/pkg/bufio/bufio.go:102 +0xbc
net/http.(_persistConn).readLoop(0xf84009a2a0, 0x0)
/usr/lib/go/src/pkg/net/http/transport.go:521 +0xab
created by net/http.(*Transport).getConn
/usr/lib/go/src/pkg/net/http/transport.go:382 +0x5df

goroutine 5 [chan receive]:
net.(_pollServer).WaitRead(0xf8400b21c0, 0xf840078750, 0xf840057030, 0xb, 0x1, ...)
/usr/lib/go/src/pkg/net/fd.go:268 +0x73
net.(_netFD).Read(0xf840078750, 0xf840403000, 0x100000001000, 0xffffffff, 0xf840059210, ...)
/usr/lib/go/src/pkg/net/fd.go:428 +0x1ec
net.(_TCPConn).Read(0xf8402f4710, 0xf840403000, 0x100000001000, 0xecc00000000, 0x0, ...)
/usr/lib/go/src/pkg/net/tcpsock_posix.go:87 +0xce
bufio.(_Reader).fill(0xf8402adb80, 0xf8403accd0)
/usr/lib/go/src/pkg/bufio/bufio.go:77 +0xf0
bufio.(_Reader).Peek(0xf8402adb80, 0xf800000001, 0xf840287101, 0x0, 0x0, ...)
/usr/lib/go/src/pkg/bufio/bufio.go:102 +0xbc
net/http.(_persistConn).readLoop(0xf84009ae40, 0x0)
/usr/lib/go/src/pkg/net/http/transport.go:521 +0xab
created by net/http.(*Transport).getConn
/usr/lib/go/src/pkg/net/http/transport.go:382 +0x5df

goroutine 6 [chan receive]:
net.(_pollServer).WaitRead(0xf8400b21c0, 0xf840078870, 0xf840057030, 0xb, 0x1, ...)
/usr/lib/go/src/pkg/net/fd.go:268 +0x73
net.(_netFD).Read(0xf840078870, 0xf8403ea000, 0x100000001000, 0xffffffff, 0xf840059210, ...)
/usr/lib/go/src/pkg/net/fd.go:428 +0x1ec
net.(_TCPConn).Read(0xf8402f40b0, 0xf8403ea000, 0x100000001000, 0xefe00000000, 0x0, ...)
/usr/lib/go/src/pkg/net/tcpsock_posix.go:87 +0xce
bufio.(_Reader).fill(0xf8400b2200, 0xf8403ac5f0)
/usr/lib/go/src/pkg/bufio/bufio.go:77 +0xf0
bufio.(_Reader).Peek(0xf8400b2200, 0xf800000001, 0xf8400aff01, 0x0, 0x0, ...)
/usr/lib/go/src/pkg/bufio/bufio.go:102 +0xbc
net/http.(_persistConn).readLoop(0xf84009a000, 0x0)
/usr/lib/go/src/pkg/net/http/transport.go:521 +0xab
created by net/http.(*Transport).getConn
/usr/lib/go/src/pkg/net/http/transport.go:382 +0x5df

Review speedtest's description of their testing methodology. Currently we are using a rather "dumb" way of testing bandwidth, we are just measuring start to finish time on downloads of known sizes

• Add a switch to list servers
• Add an option to specify which server to test from

Add "thank yous" to readme

I noticed if I run the exe (0.8.4-b) with no command line switches the output does not tell me which server is selected for the speedtest. If I run with the -s switch the output is the same but does include the server, but then I have to specify which server to use. I think it would make sense for the output without command line switches to include the server selected for the test.

Culley Morrow [email protected]
Oct 27

to me
I ran across your CLI speedtest tool on the Kaseya forums and it definitely has some promise. Like others I am not sure about the reliability of the results. This is with the most current speedtest-32-v0.8.4-a-9-g8644fff.exe.

I am on a LAN within an ISP’s main office. The server I am testing against is about 20 miles away and has a 100 down, 50 up pipe to it which is shared with the rest of the building. The web version gets a faster result consistently.

Culley Morrow
Oct 28 (13 days ago)

to me
I used the –l command to find server number 3571 which is in our main CO less than 20 miles away. It should be the same server which I chose in both web and CLI. I ran it again this morning and I see the same results. This is a hardwire connection where the only bottleneck is the Cisco routers with 100mbps between my office and the main backbone routers.

Culley Morrow
MTA Tech Express
Systems/Network Admin II
Phone: 907.761.2682

From: Zach Peters [mailto:[email protected]]
Sent: Tuesday, October 27, 2015 3:16 PM
To: Culley Morrow [email protected]
Subject: Re: Inaccurate speeds?

Previously we were using ?x=5p33dy for a string, this might be getting blocked or cached. I'm randomizing it

Not entirely sure how this should be controlled but it would be nice if there was a we to set certain limits on the amount of data downloaded/uploaded to make it more appropriate to different connection times. ie. cable 30d/3u, T1 1d/1u, etc.

I'm running this tool now every 15 minutes since about 2 months. Mostly my upload-speeds are between 2043 and 3520 (about 1850 runs of the command-file).

However, I have around 450 entries with upload-speeds between 268435 and 805306 kbps. For me these upload-speeds are impossible.

Any idea what can be done on this ?

I guess there has been a change to the cli package:

On exit, the speedtest program is outputting:

DEPRECATED Action signature. Must be cli.ActionFunc. This is an error in the application. Please contact the distributor of this application if this is not you. See https://github.com/codegangsta/cli/blob/master/CHANGELOG.md#deprecated-cli-app-action-signature

This can be fixed by adding a:

Os.Exit(0)

at the end of the runTest function

sorry, I'm not sure how to do a pull request...

Add git commit number for easier issue tracking

Diablo Lew to me
8:26 AM
Good day!
Is it possible to conclude only the download speed and ping ?
Thank you!

Create a 32bit linux release as you have only made a 64bit version :)

I started on some tests but only about 5% - 10% of the code is covered. This needs a lot of love

Observations and reports show reported upload speeds are up to 50% of speedtest.net reported

I also noticed the new behaviour, I mean that with "-r" option the whole string is printed at the end of the command, while with older releases the fields used to spawn in realtime one after another during the command execution.
Personally, I find the old behaviour to be better, because it allows you to see pieces of information before than with the new behaviour.

[email protected]