lbenitez000 / trparse Goto Github PK

Hello, I've just found a bug whilst using your module:

# Parse probes data: <name> | <(IP)> | <rtt> | 'ms' | '*'
        probes_data = match_hop[2].split()
        # Get rid of 'ms': <name> | <(IP)> | <rtt> | '*'
        probes_data = filter(lambda s: s.lower() != 'ms', probes_data)

        i = 0
        while i < len(probes_data):
            # For each hop parse probes
            name = None
            ip = None
            rtt = None
            anno = ''

Here (line 118 of trparse.py), you try to iterate through the result of a filter, which is returning me:

File "/usr/local/lib/python3.6/site-packages/trparse.py", line 118, in loads
    while i < len(probes_data):
TypeError: object of type 'filter' has no len()

So to fix this I had to simply modify this on line 115:
From : probes_data = filter(lambda s: s.lower() != 'ms', probes_data) to probes_data = list(filter(lambda s: s.lower() != 'ms', probes_data)) .

It all works perfectly fine now, thanks for creating this module, it make my code much cleaner and my life easier so I thought I'd help you with that.

Cheers.

Consider the following example:

traceroute to vialink.com.br (52.33.65.153), 64 hops max, 52 byte packets
 7  * * *

Here, we'll have three probes. Each one will have rtt = None and ip = None.

Now, in this example:

traceroute to vialink.com.br (52.33.65.153), 64 hops max, 52 byte packets
25  * * 52.93.14.61 (52.93.14.61)  218.370 ms

We will have three probes here. The first two will have rtt = None and ip = None. The last one will have rtt = 218.370 and ip = 52.93.14.61.

Lastly:

traceroute to vialink.com.br (52.33.65.153), 64 hops max, 52 byte packets
10  xe-0-0-0.96.jr03-napbr-db.durand.com.br (200.170.83.97)  20.741 ms  *  16.571 ms

We will also have three probes. The second one will have rtt = None (since it is a timeout) but it will have ip = '200.170.83.97'. This happens because the add_probe method always use the last probe info (name/ip) if the new one has no IP:

    def add_probe(self, probe):
        """Adds a Probe instance to this hop's results."""
        if self.probes:
            probe_last = self.probes[-1]
            if not probe.ip:
                probe.ip = probe_last.ip
                probe.name = probe_last.name
        self.probes.append(probe)

I think this behavior is inconsistent because the same probe can have or not IP depending on the position it is found. So I think a proper behavior is to always use ip = None and name = None in timeout probes. If we consider we never know the host (name/ip) which produced the timeout (can be another host different than the previous one), it makes sense to leave ip = None and name = None in timeout probes.

Hello,

it seems that parsing of % traceroute -n; output isn't supported which is a pity :/

Traceback (most recent call last):
  File "traceroute.py", line 28, in <module>
    main()
  File "traceroute.py", line 24, in main
    traceroute = trparse.loads(out)
  File "venv/local/lib/python2.7/site-packages/trparse.py", line 135, in loads
    rtt = float(probes_data[i+2])
ValueError: invalid literal for float(): 108.170.245.49

When parsing the following traceroute, the parser throw exception, since probe name contains "/".

traceroute to 142.116.179.108 (142.116.179.108) from 10.8.180.143, 32 hops max, 40byte packets
1 10.8.180.2 (10.8.180.2) 1.000 ms 0.000 ms 0.000 ms
2 10.255.255.253 (10.255.255.253) 0.000 ms 0.999 ms 2.000 ms
3 8.42.102.51 (8.42.102.51) 0.000 ms 0.999 ms 0.000 ms
4 8.42.102.24 (8.42.102.24) 0.000 ms 0.000 ms 1.000 ms
5 xe-1-0-2.er1.ord2.us.zip.zayo.com (64.125.36.245) 0.000 ms 1.001 ms 0.000 ms
6 ae6.cr1.ord2.us.zip.zayo.com (64.125.31.172) 0.999 ms 0.000 ms 1.000 ms
7 ae15.er1.ord7.us.zip.zayo.com (64.125.31.85) 0.998 ms 0.999 ms 1.002 ms
8 zayo-bellca.ord7.us.zip.zayo.com (64.125.13.202) 8.001 ms 8.998 ms 8.002 ms
9 tcore4-chicagocp_0-10-0-0.net.bell.ca (64.230.79.86) 26.000 ms 22.000 ms 20.000 ms
10 tcore3-toronto21_hundredgige1-2-0-0.net.bell.ca (64.230.79.160) 25.000 ms 22.999 ms 19.000 ms
11 agg1-toronto21_ae1.net.bell.ca (64.230.104.243) 20.001 ms 20.002 ms 20.998 ms
12 lnsm1-toronto46_1/0.100.net.bell.ca (64.230.53.205) 19.998 ms 20.000 ms 19.001 ms
13 lnsm1-toronto46-142-116-179-108.internet.virginmobile.ca (142.116.179.108) 24.999 ms 24.003 ms 23.000 ms

Example that works (although ASN should belong to a probe, not to a hop):

 9  [AS19905] xe-0-1-2.2028.jr01vr-napbr-fly.fly.com.br (177.93.238.14)  21.291 ms  21.860 ms  21.067 ms
10  [AS22356] xe-0-0-0.96.jr03-napbr-db.durand.com.br (200.170.83.97)  15.772 ms  736.220 ms  23.648 ms

Example that do not work:

11  [AS19905] ae0.900.jr02-tvt-spo.durand.com.br (177.190.108.50)  26.945 ms
    [AS3356] 4.16.168.34 (4.16.168.34)  192.924 ms  187.665 ms
12  [AS16509] 52.95.54.156 (52.95.54.156)  181.465 ms
    [AS16509] 52.95.55.2 (52.95.55.2)  185.864 ms  184.844 ms

To generate that output, I run the command traceroute -a vialink.com.br in MACOS X.

This works:

traceroute to 187.45.161.65 (187.45.161.65), 64 hops max, 52 byte packets
 1  192.168.0.1 (192.168.0.1)  1.359 ms  0.816 ms  1.408 ms
 [...] 
 5  65-161-45-187.static.in-addr.vialink.com.br (187.45.161.65)  12.930 ms  32.327 ms  10.240 ms

It returns the following Traceroute object:

Traceroute for 187.45.161.65 (187.45.161.65)

  1 192.168.0.1 (192.168.0.1) 1.359 ms
    192.168.0.1 (192.168.0.1) 0.816 ms
    192.168.0.1 (192.168.0.1) 1.408 ms

  [...]

Where self.dest_name = 187.45.161.65 and self.dest_ip = 187.45.161.65.

But when the output to parse has no headers, it doesn't work:

 1  192.168.0.1 (192.168.0.1)  1.359 ms  0.816 ms  1.408 ms
 [...]
 5  65-161-45-187.static.in-addr.vialink.com.br (187.45.161.65)  12.930 ms  32.327 ms  10.240 ms

In the current implementation, trparse parses the output, but it interprets the first line (in this case, a hop) as the header, and it returns the following Traceroute object (which is wrong):

Traceroute for 192.168.0.1 (192.168.0.1)

  1 192.168.0.1 (192.168.0.1) 1.359 ms
    192.168.0.1 (192.168.0.1) 0.816 ms
    192.168.0.1 (192.168.0.1) 1.408 ms

  [...]

Note the first line: 192.168.0.1 (192.168.0.1) is not the correct address/ip (it's the first probe name/IP instead)!

Outputs without header make sense because traceroute prints the first line (header) to stderr (instead of to stdout). So, if the user only captures stdout, the string to be parsed won't have the headers.

Some examples:

$ traceroute 187.45.161.65 >/dev/null
traceroute to 187.45.161.65 (187.45.161.65), 64 hops max, 52 byte packets
$ traceroute 187.45.161.65 2>/dev/null
 1  192.168.0.1 (192.168.0.1)  3.386 ms  1.059 ms  0.886 ms
 [...]
 5  65-161-45-187.static.in-addr.vialink.com.br (187.45.161.65)  19.414 ms  18.871 ms  10.390 ms
$ traceroute 187.45.161.65 2>&1
traceroute to 187.45.161.65 (187.45.161.65), 64 hops max, 52 byte packets
 1  192.168.0.1 (192.168.0.1)  3.386 ms  1.059 ms  0.886 ms
 [...]
 5  65-161-45-187.static.in-addr.vialink.com.br (187.45.161.65)  19.414 ms  18.871 ms  10.390 ms

I think one of the following options should be considered:

Option 1: Make explicit that we do not know the destination address / destination ip, like:

Traceroute for unknown address

  1 192.168.0.1 (192.168.0.1) 1.359 ms
  [...]

Or:

Traceroute for None (None)

  1 192.168.0.1 (192.168.0.1) 1.359 ms
  [...]

Option 2: Require the header and throw a parse error instead of wrongly parsing the output.

message = "Expected header. Got: '{}'".format(lines[0])
raise Exception(message)

Any idea why the destination IP address from the header and the source IP addresses from the probes are not populating?

Input:

traceroute6 to turner-tls.map.fastly.net (2a04:4e42:200::323) from 2600:1700:bab0:d40:985:f00a:98bd:f142, 5 hops max, 12 byte packets
 1  * * *
 2  2001:506:6000:11b:71:156:212:143  27.635 ms  20.383 ms  23.438 ms
 3  * * *
 4  2001:1890:ff:ff08:12:242:117:16  20.118 ms  20.327 ms  21.213 ms
 5  * * *

output (i'm converting to a dict/JSON object:

{"destination_ip": null, "destination_name": "turner-tls.map.fastly.net", "hops": [{"hop": "1", "probes": [{"annotation": null, "asn": null, "ip": null, "name": null, "rtt": null}]}, {"hop": "2", "probes": [{"annotation": null, "asn": null, "ip": null, "name": null, "rtt": "27.635"}, {"annotation": null, "asn": null, "ip": null, "name": null, "rtt": "20.383"}, {"annotation": null, "asn": null, "ip": null, "name": null, "rtt": "23.438"}]}, {"hop": "3", "probes": [{"annotation": null, "asn": null, "ip": null, "name": null, "rtt": null}]}, {"hop": "4", "probes": [{"annotation": null, "asn": null, "ip": null, "name": null, "rtt": "20.118"}, {"annotation": null, "asn": null, "ip": null, "name": null, "rtt": "20.327"}, {"annotation": null, "asn": null, "ip": null, "name": null, "rtt": "21.213"}]}, {"hop": "5", "probes": [{"annotation": null, "asn": null, "ip": null, "name": null, "rtt": null}]}]}

When running a traceroute with asn numbers, the parser fails:

Command line: traceroute -a 8.8.8.8
Output:
' 1 [AS0] 10.118.104.1 (10.118.104.1) 2.819 ms 3.276 ms 1.772 ms\n 2 [AS12849] 213.57.80.185 (213.57.80.185) 4.293 ms 4.027 ms 5.672 ms\n 3 * * [AS12849] bb-h1-b5.15-pt.hotnet.net.il (213.57.0.129) 11.136 ms\n 4 [AS12849] core-213-57-3-150.hfa.hotnet.net.il (213.57.3.150) 57.411 ms\n [AS12849] core-213-57-0-150.pt.hotnet.net.il (213.57.0.150) 55.538 ms 55.365 ms\n 5 [AS15169] 216.239.46.140 (216.239.46.140) 55.946 ms\n [AS15169] 216.239.47.98 (216.239.47.98) 54.371 ms\n [AS15169] 216.239.59.78 (216.239.59.78) 54.784 ms\n 6 [AS15169] 216.239.57.188 (216.239.57.188) 55.054 ms\n [AS15169] 209.85.243.131 (209.85.243.131) 57.285 ms\n [AS15169] 209.85.247.101 (209.85.247.101) 54.662 ms\n 7 [AS15169] 66.249.95.226 (66.249.95.226) 63.427 ms\n [AS15169] 72.14.233.246 (72.14.233.246) 65.639 ms\n [AS15169] 66.249.95.226 (66.249.95.226) 64.876 ms\n 8 [AS15169] 108.170.234.139 (108.170.234.139) 63.365 ms\n [AS15169] 209.85.249.14 (209.85.249.14) 60.498 ms\n [AS15169] 108.170.234.139 (108.170.234.139) 63.207 ms\n 9 * * *\n10 * [AS15169] google-public-dns-a.google.com (8.8.8.8) 63.369 ms 63.766 ms\n'

Error:
>>> tr = trparse.loads(out) Traceback (most recent call last): File "<stdin>", line 1, in <module> File "/Library/Python/2.7/site-packages/trparse.py", line 135, in loads rtt = float(probes_data[i+2]) ValueError: could not convert string to float: (213.57.0.129)

Hi,

I am trying to do the following command in Python:
traceroute -A google.es | cat > test.txt

So, what I've done is the following:

tracerouteProcess = subprocess.Popen(["traceroute","-A","google.es"]. stdout=subprocess.PIPE) f = open("test.txt", "w")

Sending the output from traceroute to a file
using cat because the OS of host (SELinux) where the script will be executed
does not allow directly redirecting the traceroute output to a file.

catProcess = subprocess.Popen(["cat"], stdin=tracerouteProcess.stdout, stdout=f) catProcess.communicate() string = f.read() traceroute = trparse.loads(string)

And it raises the following error:

dest_name = match_dest.group(1)
AttributeError: 'NoneType' object has no attribute 'group'

I will appreciate some kind of example to see what I am doing wrong.

Thanks in advance

Hello! When traceroute outputs asterisks (timeout) I get :
'NoneType' object has no attribute 'group'

trparse fails to deal with hostnames of the form 24-104-71-65-ip-static.hfc.comcastbusiness.net (it fails because the hostname starts with a digit). This appears to be due to the RE_PROBE_NAME regex and I've tried to fix it simply by changing the first part of the regex from:

^([a-zA-Z][a-zA-z0-9\.-]+)$...

to:

^([a-zA-Z0-9][a-zA-z0-9\.-]+)$...

But then after some tests, now this new regex also matches against RTT data, such as 0.248 causing more problems.

There is probably some way to match hostnames better without mucking around too much and I'll update if I figure something out (but apparently it can get messy).