Hey,

I just found out that when you setup the hyperglass-agent certificates they are not working until 2hrs after the inital setup if you have the CEST timezone configured.

I'f I check the generated cert it looks like this:

root@lg:/etc/hyperglass# openssl x509 -in certs/c02.fra.de.as49697.net.pem -text -noout
Certificate:
    Data:
        Version: 3 (0x2)
        Serial Number:
            3c:01:bd:6d:b3:9e:79:c7:87:f4:f8:5d:76:cf:c3:a5:18:bd:1b:3c
        Signature Algorithm: sha256WithRSAEncryption
        Issuer: CN = c02.fra.de.as49697.net, O = hyperglass
        Validity
            Not Before: Oct  7 13:01:53 2020 GMT
            Not After : Oct  7 13:01:53 2022 GMT

But 13:01:53 2020 GMT means 15:01:53 CEST and currently it's only 13:01:53 CEST :D

Feature Description

Option in one of the config files to limit the number of routers that can be selected in the hyperglass main page.

Expected behavior

hyperglass.yaml file option to set the number of devices that can be selected.

The error output from hyperglass-agent setup:

❌ Method Not Allowed

I know it's in progress, but wanted to point out some specific things to look at.

On this page, the command for ubuntu/debian is sudo apt install -y python3.6-dev python3-pip. Debian does not have a package called python3.6-dev. Ubuntu does, but only for 18.04. For debian, the appropriate command is sudo apt install -y python3-dev python3-pip. For ubuntu I guess it depends on the version? Idk, I don't use ubuntu.

Lower down is the install command: pip3 install --user hyperglass-agent. This is more of a question, but I'm curious why install as the user vs as system. The docs for hyperglass proper do not have this flag (pip3 install hyperglass).

On the next page (Setup), under the NTP warning at the top, there's a small typo: "Before you get to far" should be "Before you get too far".

BIRD expects a different input format when searching for BGP AS Path and BGP Community.
To keep hyperglass and hyperglass-agent consistent while using other NOS models, the input should be translated for BIRD.

Examples:

• BGP AS Path
  • User entered input: _1234_
    Bird command: show route all where bgp_path ~ [= * 1234 * =]
  • User entered input: _1234$
    Bird command: show route all where bgp_path ~ [= * 1234 =]
  • User entered input: ^1234_
    Bird command: show route all where bgp_path ~ [= 1234 * =]

With that _ within the input needs to be translated to *, while both ^ and $ should be translated into a space. The actual command also needs to be changed to show route all where bgp_path ~ [= {bird_target} =]

• BGP Community
  • User entered input: 1234:100
    Bird command: show route all where (1234,100) ~ bgp_community
  • User entered input: 1234:100:200
    Bird command: show route all where (1234,100,200) ~ bgp_community

With that : needs to be translated into ,. The actual command also needs to be changed to show route all where ({bird_target}) ~ bgp_community

Version: 0.1.0 (Latest from pip)

Behavior: When use ping or traceroute on bird nodes, execution of ping or traceroute command is successful. But it will return not-found error to hyperglass.

Reproduce:

• Configure hyperglass-agent to use bird.
• Start a ping or traceroute from hyperglass.

Related Code:
output += await parser( raw=raw_output, query_data=query, not_found=params.not_found_message )

Possible Solution:

• Remove parse_bird_output when performing ping or traceroute.
• Add parsers for ping and traceroute.

hyperglass-agent version: 0.1.6

When generating a certificate with hyperglass agent, certain addresses (e.g. RFC1918, 127.x.x.x, or 169.254.x.x) addresses are not included in the discovered local address list.
Any of these addresses should be perfectly fine for exposing hyperglass-agent under them.

When I try to run "pip3 install --user hyperglass-agent" it errors out specifying

Command "/usr/bin/python3 -u -c "import setuptools, tokenize;__file__='/tmp/pip-install-h7_0f_17/uvloop/setup.py';f=getattr(tokenize, 'open', open)(__file__);code=f.read().replace('\r\n', '\n');f.close();exec(compile(code, __file__, 'exec'))" install --record /tmp/pip-record-6yc0xiwa/install-record.txt --single-version-externally-managed --compile --user --prefix=" failed with error code 1 in /tmp/pip-install-h7_0f_17/uvloop/.

Specifically it states

`x86_64-linux-gnu-gcc -pthread -Wno-unused-result -Wsign-compare -DNDEBUG -g -fwrapv -O2 -Wall -g -fstack-protector-strong -Wformat -Werror=format-security -g -fwrapv -O2 -g -fstack-protector-strong -Wformat -Werror=format-security -Wdate-time -D_FORTIFY_SOURCE=2 -fPIC -I/usr/include/python3.7m -I/tmp/pip-install-h7_0f_17/uvloop/vendor/libuv/include -c uvloop/loop.c -o build/temp.linux-x86_64-3.7/uvloop/loop.o -O2
x86_64-linux-gnu-gcc: fatal error: Killed signal terminated program cc1
compilation terminated.
error: command 'x86_64-linux-gnu-gcc' failed with exit status 1

Failed building wheel for uvloop
`

OS: Ubuntu 19.10
Python Version: 3.7.5
routing: BIRD2

In the course of today's hyperglass installation, the following error occurred when querying any BGP route of the BIRD router:

The query completed, but no results were found.

System environment

Hyperglass operating system: Docker container with alpine Linux Version 3.9
Hyperglass version: 1.0.0b64
Hyperglass python version: 3.6.12

Hyperglass-agent operating system: Ubuntu Linux 18.04 (Bionic Beaver)
Hyperglass-agent version: 0.1.6
Hyperglass-agent python version: 3.6.12
BIRD Version: 1.6.3

Setup

The queries are sent from the Hyperglass (installed in a Docker Container) encrypted to the hyperglass agent installed on a BIRD router / Ubuntu Linux 18.04 (Bionic Beaver). The connection between hyperglass and hyperglass-agent is established and working (ping and traceroute are working).

After a bit of reading code, I think I found the error in nos_utils/bird.py

Assuming the bird output generated with birdc "show route all where 8.8.8.0/24 ~ net like the following, the parse_bird_output() in bird.py mentioned above returns an empty newline.

BIRD data:

BIRD 1.6.3 ready.
0.0.0.0/0          via 1.2.3.4 on wan [router01 00:00:00 from 1.2.3.4] * (100/?) [i]
    Type: BGP unicast univ
    BGP.origin: IGP
    BGP.as_path:
    BGP.next_hop: 1.2.3.4
    BGP.local_pref: 100
                via 1.2.3.4 on wan [router02 00:00:00 from 1.2.3.4] (100/?) [i]
    Type: BGP unicast univ
    BGP.origin: IGP
    BGP.as_path:
    BGP.next_hop: 1.2.3.4
    BGP.local_pref: 100

Local test script:

Tested the parse_bird_output() locally, with the following:

#!/usr/bin/env python3

data: str = """BIRD 1.6.3 ready.
0.0.0.0/0          via 1.2.3.4 on wan [router01 00:00:00 from 1.2.3.4] * (100/?) [i]
    Type: BGP unicast univ
    BGP.origin: IGP
    BGP.as_path:
    BGP.next_hop: 1.2.3.4
    BGP.local_pref: 100
                via 1.2.3.4 on wan [router02 00:00:00 from 1.2.3.4] (100/?) [i]
    Type: BGP unicast univ
    BGP.origin: IGP
    BGP.as_path:
    BGP.next_hop: 1.2.3.4
    BGP.local_pref: 100
"""

def parse_bird_output(raw):
    """Parse raw BIRD output and return parsed output.
    Arguments:
        raw {str} -- Raw BIRD output
        query_data {object} -- Validated query object
        not_found {str} -- Lookup not found message template
    Returns:
        str -- Parsed output
    """

    def remove_ready(lines):
        for line in lines:
            if not re.match(r".*(BIRD \d+\.\d+\.?\d* ready\.).*", line):
                yield line.strip()

    raw_split = re.split(r"(Table)", raw.strip())

    if not raw_split:
        lines = 'Not found !'
    else:
        lines = raw_split

    output = "\n".join(remove_ready(lines))
    return output

print(*parse_bird_output(data))

Log output:

2020-12-17 18:19:21.793 | DEBUG    | hyperglass_agent.execute:run_query:40 - Query: query_type='bgp_route' vrf='default' afi='ipv4_default' source=IPv4Address('1.2.3.4') target='8.8.8.0/24'
2020-12-17 18:19:21.794 | DEBUG    | hyperglass_agent.execute:run_query:55 - Formatted Command: birdc "show route all where 8.8.8.0/24 ~ net"
2020-12-17 18:19:22.674 | DEBUG    | hyperglass_agent.execute:run_query:70 - Parser: parse_bird_output
2020-12-17 18:19:22.677 | DEBUG    | hyperglass_agent.nos_utils.bird:parse_bird_output:87 - Parsed output:

2020-12-17 18:19:22.679 | DEBUG    | hyperglass_agent.api.web:query_entrypoint:98 - Query Output:

Disclaimer: duplicate of thatmattlove/hyperglass-frr/issues/1

FRRouting support for getting a Command Line Interface command output in JSON format is getting better and better over time.

E.g. for hyperglass-frr is show bgp [ ipv4 | ipv6 ] unicast [ prefix | route ] json

Will avoid screen-scraping and well-known data-format is used as data-source when querying FRRouting for information.

frrouting-A# show bgp ipv4 unicast 172.20.230.0/25
BGP routing table entry for 172.20.230.0/25
Paths: (1 available, best #1, table default)
  Local
    0.0.0.0 from 0.0.0.0 (192.0.2.10)
      Origin IGP, metric 0, localpref 0, weight 32768, valid, sourced, local, bestpath-from-AS Local, best (First path received)
      Community: graceful-shutdown
      AddPath ID: RX 0, TX-All 2 TX-Best-Per-AS 0
      Advertised to: 192.0.2.11
      Last update: Sat Feb 22 03:43:28 2020

frrouting-A# show bgp ipv4 unicast 172.20.230.0/25 json 
{
  "prefix":"172.20.230.0\/25",
  "paths":[
    {
      "aspath":{
        "string":"Local",
        "segments":[
        ],
        "length":0
      },
      "origin":"IGP",
      "med":0,
      "metric":0,
      "localpref":0,
      "weight":32768,
      "valid":true,
      "sourced":true,
      "local":true,
      "bestpath":{
        "bestpathFromAs":0,
        "overall":true,
        "selectionReason":"First path received"
      },
      "community":{
        "string":"graceful-shutdown",
        "list":[
          "gracefulShutdown"
        ]
      },
      "addpathRxId":0,
      "addpathTxId":2,
      "addpathTxIdAll":2,
      "addpathTxIdBestPerAS":0,
      "advertisedTo":{
        "192.0.2.11":{
          "hostname":"frrouting-B"
        }
      },
      "lastUpdate":{
        "epoch":1582343008,
        "string":"Sat Feb 22 03:43:28 2020\n"
      },
      "nexthops":[
        {
          "ip":"0.0.0.0",
          "afi":"ipv4",
          "metric":0,
          "accessible":true,
          "used":true
        }
      ],
      "peer":{
        "peerId":"0.0.0.0",
        "routerId":"192.0.2.10"
      }
    }
  ]
}

frrouting-A# find json 
<snip>
  (view)  show [ip] bgp <view|vrf> all [<ipv4|ipv6> [<unicast|multicast|vpn|labeled-unicast|flowspec>]] [json]
  (view)  show [ip] bgp [<ipv4|ipv6> [unicast]] neighbors <A.B.C.D|X:X::X:X|WORD> received prefix-filter [json]
  (view)  show [ip] bgp [<view|vrf> VIEWVRFNAME] [<ipv4|ipv6> [<unicast|multicast|vpn>]] route-leak  [json]
  (view)  show [ip] bgp [<view|vrf> VIEWVRFNAME] [<ipv4|ipv6> [<unicast|multicast|vpn|labeled-unicast|flowspec>]]          [cidr-only          |dampening <flap-statistics|dampened-paths>          |community [AA:NN|local-AS|no-advertise|no-export          ]
  (view)  show [ip] bgp [<view|vrf> VIEWVRFNAME] [<ipv4|ipv6> [<unicast|multicast|vpn|labeled-unicast|flowspec>]] neighbors <A.B.C.D|X:X::X:X|WORD> <advertised-routes|received-routes|filtered-routes> [route-map WORD] [json]
  (view)  show [ip] bgp [<view|vrf> VIEWVRFNAME] [<ipv4|ipv6> [<unicast|multicast|vpn|labeled-unicast|flowspec>]] neighbors <A.B.C.D|X:X::X:X|WORD> <flap-statistics|dampened-routes|routes> [json]
  (view)  show [ip] bgp [<view|vrf> VIEWVRFNAME] [<ipv4|ipv6> [<unicast|multicast|vpn|labeled-unicast|flowspec>]] large-community [<AA:BB:CC> [exact-match]] [json]
  (view)  show [ip] bgp [<view|vrf> VIEWVRFNAME] [<ipv4|ipv6> [<unicast|multicast|vpn|labeled-unicast|flowspec>]] large-community-list <(1-500)|WORD> [exact-match] [json]
  (view)  show [ip] bgp [<view|vrf> VIEWVRFNAME] [<ipv4|ipv6> [<unicast|multicast|vpn|labeled-unicast|flowspec>]] summary [failed] [json]
  (view)  show [ip] bgp [<view|vrf> VIEWVRFNAME] [<ipv4|ipv6> [<unicast|multicast|vpn|labeled-unicast|flowspec>]]<A.B.C.D|A.B.C.D/M|X:X::X:X|X:X::X:X/M> [<bestpath|multipath>] [json]
  (view)  show [ip] bgp [<view|vrf> VIEWVRFNAME] [<ipv4|ipv6> flowspec] detail [json]
  (view)  show [ip] bgp [<view|vrf> VIEWVRFNAME] [<ipv4|ipv6>] neighbors [<A.B.C.D|X:X::X:X|WORD>] [json]
</snip>

I am unable to find any way to make the hyperglass agent listen only on a single IP address.

I do not wish to expose the hyperglass-agent HTTP service to the internet proper and have it listen only on our OOB interface, but also do not wish to run a firewall on our FRR box, as iptables etc. would massively impact performance.

Is there any way to make the agent listen only on a specific address/interface?

Thanks in advance! :)