faucetsdn / faucet Goto Github PK

With the routing updates, faucet_yaml.dot file needs to be updated to reflect current state. A possible option is to automatically generate list of possible options from those in the tests and/or source code.

So we have a bit of a dilemma with Gauge and InfluxDB support regarding datatypes.

OFP_PORT_STATS structs as defined by the OpenFlow spec contain uint64 values for each statistic.

InfluxDB uses int64 to store integers internally which leaves us with a large range of values that can't be stored currently, also due to issue #353 gauge will crash when it tries to store a value that doesn't fit in influx.

Maybe time to switch to float64?

It would be good to have a few lines in the README describing the dependencies / install requirements.

When I create a Switch in mininet with dpid=0000000000000000, ryu's app show me a random dpid like 51515168712262, when the last dpid is repeated.

        for i in range(2):
            self.HostList.append(self.addHost('h' + str(i)))
            self.SwitchList.append(self.addSwitch('s' + str(i),dpid='000000000000000'+str(i)))

You need to avoid this value, starting from 1:

        for i in range(1,3):
            self.HostList.append(self.addHost('h' + str(i)))
            self.SwitchList.append(self.addSwitch('s' + str(i),dpid='000000000000000'+str(i)))

Currently, the flood manager builds output action lists to implement VLAN flooding, one list for every ingress port, outputting to all ports except the ingress port (since it is incorrect to flood out the ingress port).

We should really use groups instead which would scale better (smaller action list, also group OFPGT_ALL automatically exclude outputting to ingress port so only one output entry is needed per VLAN - see https://www.opennetworking.org/images/stories/downloads/sdn-resources/onf-specifications/openflow/openflow-spec-v1.3.3.pdf section 5.7.1).

The controller should export, via some means, when it re/learns a host on a port, so that an external system can react.

So at the moment doing BGP routing with multiple DPs is quite annoying. Officially faucet doesn't support this and you get the following error when you do:

AssertionError: DPs ovs-nznog, at-x930-2 sharing a BGP speaker VLAN is unsupported

Now there is a workaround you can use to make faucet happy here, you can definite two vlans (e.g mynetwork_layer2 and mynetwork_layer3) with the same VLAN ID and push these out to different DPIDs so that routing happens in the right place.

Unfortunately I have discovered a limitation of this workaround to do with how faucet pushes VLANs to DPIDs. We assume all VLANs live on all DPIDs and populate the DP.vlans dictionary with all VLANs that are configured in faucet.yaml (this happens in config_parser.py in _dp_parser_v2()). What this means is that we have an ordering problem for the dps list in faucet.yaml. The DPID with the routing VLAN (mynetwork_layer3) must come first in faucet.yaml, if a DPID with the layer 2 version (mynetwork_layer2) we trip the assert on line 171 of config_parser.py.

Now the upshot of this is when you're using this multi-DP routing workaround is that you can't use the "include" keyword in faucet.yaml to include other config files as this implicitly reorders the dps list which can trigger the assert if you have a DPID with a name that sorts about your DPID with the routing VLAN.

Now I don't quite have enough time to think about the best way of solving any of these problems, but thought I'd write it down so we remember for later.

Currently if a port has tagged and untagged vlans, the tagged vlan will overwrite the untagged rule.

This could be fixed by increasing the priority of the tagged vlan rules. But more importantly the switch really shouldnt accept tagged packets on ports with native vlans. We should require explicit configuration for QnQ when needed.

I'm running both faucet and gauge (last commit version 20c473c). When I exit from mininet, with exit command, gauge crash, but faucet is still running.
To reproduce the issue, you can run gauge and mininet as follows:
ryu-manager --verbose --ofp-tcp-listen-port=6663 ~/faucet/src/ryu_faucet/org/onfsdn/faucet/gauge.py
sudo mn --controller=remote,ip=127.0.0.1,port=6663 --topo linear,3
After that, write exit command on mininet terminal.
My gauge.yaml config is the following:

faucet_configs:
- '/etc/ryu/faucet/faucet.yaml'
watchers:
port_status_poller:
type: 'port_state'
dps: ['s1','s2','s3','s4','s5','s6','s7','s8']
interval: 10
db: 'influx'

port_stats_poller:
    type: 'port_stats'
    dps: ['s1','s2','s3','s4','s5','s6','s7','s8']
    interval: 10
    db: 'influx'

port_table_poller:
    type: 'flow_table'
    dps: ['s1','s2','s3','s4','s5','s6','s7','s8']
    interval: 20
    #db: 'influx'
    db: 'ft_file'

dbs:

ft_file:
    type: 'text'
    file: 'flow_table.JSON'
influx:
    type: 'influx'
    influx_db: 'faucet'
    influx_host: '127.0.0.1'
    influx_port: 8086
    influx_user: ''
    influx_pwd: ''
    influx_timeout: 10

This is gauge.log:
Mar 15 17:32:22 gauge INFO DPID 1 (0x1) up
Mar 15 17:32:22 gauge INFO DPID 2 (0x2) up
Mar 15 17:32:22 gauge INFO DPID 3 (0x3) up
Mar 15 17:32:23 gauge.port_state INFO port 1 deleted
Mar 15 17:32:23 gauge.port_state INFO port 1 deleted
Mar 15 17:32:23 gauge.port_state INFO port 1 deleted
Mar 15 17:32:23 gauge.port_state INFO port 2 deleted
Mar 15 17:32:23 gauge.port_state INFO port 2 deleted
Mar 15 17:32:24 gauge.port_state INFO port 3 deleted
Mar 15 17:32:24 gauge.port_state INFO port 2 deleted
Mar 15 17:32:24 gauge.exception ERROR Unhandled exception, killing RYU
Traceback (most recent call last):
File "/home/nboettcher/Downloads/faucet/src/ryu_faucet/org/onfsdn/faucet/util.py", line 51, in __koe
func(*args, **kwargs)
File "/home/nboettcher/Downloads/faucet/src/ryu_faucet/org/onfsdn/faucet/gauge.py", line 99, in handler_connect_or_disconnect
watcher.stop()
TypeError: stop() takes exactly 2 arguments (1 given)

Is a problem with watcher.stop(), but I don't know how to fix it.

Source code can be better structured for easier understanding. Given that this is built as an application for Ryu Controller, I would put the faucet and gauge apps under bash $FAUCET_ROOT/ryu/app directory and appropriately name them as per https://github.com/osrg/ryu/tree/master/ryu/app.

I would also create bash ryu/tests directory and top level bash doc directory for consistency.

Due to how we parse our version number in setup.py we break python3 compat:

root@6b45c889c8f6:/tmp# pip3 install ~/faucet/
Processing /root/faucet
    Complete output from command python setup.py egg_info:
    Traceback (most recent call last):
      File "<string>", line 1, in <module>
      File "/tmp/pip-5apv5jb1-build/setup.py", line 9, in <module>
        readme_contents = open(os.path.join(setup_dir, 'README.rst')).read()
      File "/usr/lib/python3.5/encodings/ascii.py", line 26, in decode
        return codecs.ascii_decode(input, self.errors)[0]
    UnicodeDecodeError: 'ascii' codec can't decode byte 0xe2 in position 15736: ordinal not in range(128)

DP Class has "influxdb_stats" variable which is never used. This should be removed.

So here's a great idea: if we can somehow run faucet in a mode where it just parses the config then exits returning either a success or error code depending on whether the config is valid or not then operators can use this as part of an ansible deploy to decide if it's safe to reload faucet or not.

Have assigned to Chris because he offered to code the feature for me in IRC 😄

The documentation in src/docs/INSTALL is out-of-date, because it references "upstart" when that no longer works and is no longer necessary. See the following thread for a recommendation from Shivaram for the proper install process:

https://lists.geant.org/sympa/arc/faucet-users/2017-03/msg00009.html

The src/docs/INSTALL documentation should be updated to match the current expected install process. At the same time, the documentation in README.rst should also be updated, as it describes another method of running (using ryu-manager) that requires setting environment variables.

At the moment we only support configuring multiple datapath elements via running separate instances of Faucet with different configuration files (on separate ports)

This makes it annoying to manage multiple OpenFlow-enabled switches since we need to assign separate TCP ports for each to connect back to our multiple Faucet instances.

In original Valve we had the ability to list multiple DPIDs in the configuration file that would be configured (with different numbers of ports and different vlan assignments).

I think this change will require a change to the config file format since there doesn't appear to be an easy way to list additional datapaths in there.

FAUCET currently supports multiple dataplanes as ships-in-the-night (ie. each dataplane is controlled as a separate switch, even when they are configured in one file and controlled by a single FAUCET controller).

We should support arbitrary "stacking", allowing cross connection of dataplanes, potentially in a complex way (eg fat tree, Clos).

At the moment if we get a bad response from a database backend (i.e influxdb) gauge will crash with no error.

If we get the username/password wrong we will crash, if we get an error from influxdb on a query (such as bad datatype) we will crash.

We should add some error handling to the database connection and queries.

I see use of "valve" used in README, etc. Would it be possible to clean up terminology or explain how valve relates to Faucet?

I'm running faucet (latest master) under python 3.6. I encountered two issues related to the Python 3.x port that are easy to fix.

Issue 1 occurs at start up.

Traceback (most recent call last):
  File "/home/mininet/venv36/bin/ryu-manager", line 11, in <module>
    load_entry_point('ryu==4.12', 'console_scripts', 'ryu-manager')()
  File "/home/mininet/venv36/lib/python3.6/site-packages/ryu/cmd/manager.py", line 81, in main
    services.extend(app_mgr.instantiate_apps(**contexts))
  File "/home/mininet/venv36/lib/python3.6/site-packages/ryu/base/app_manager.py", line 509, in instantiate_apps
    self._instantiate(app_name, cls, *args, **kwargs)
  File "/home/mininet/venv36/lib/python3.6/site-packages/ryu/base/app_manager.py", line 495, in _instantiate
    app = cls(*args, **kwargs)
  File "/home/mininet/code/faucet/src/ryu_faucet/org/onfsdn/faucet/faucet.py", line 189, in __init__
    self.config_file, self.logname)
  File "/home/mininet/code/faucet/src/ryu_faucet/org/onfsdn/faucet/config_parser.py", line 35, in dp_parser
    config_hashes, dps = _config_parser_v2(config_file, logname)
  File "/home/mininet/code/faucet/src/ryu_faucet/org/onfsdn/faucet/config_parser.py", line 148, in _config_parser_v2
    top_confs['vlans'])
  File "/home/mininet/code/faucet/src/ryu_faucet/org/onfsdn/faucet/config_parser.py", line 85, in _dp_parser_v2
    dp.sanity_check()
  File "/home/mininet/code/faucet/src/ryu_faucet/org/onfsdn/faucet/dp.py", line 141, in sanity_check
    assert isinstance(self.dp_id, (int, long))
NameError: name 'long' is not defined

Issue 2 occurs with ICMPv6 neighbor discovery resolving. I configured an IPv6 faucet_vip and then ping'd it from host fc00::1. A while later, it will log a traceback to the console when the ND request is sent to check if fc00::1 is still there.

Mar 20 13:52:00 faucet.valve DEBUG    Packet_in DPID 1 (0x1) src:92:06:7e:ac:62:d6 in_port:1 vid:2040
Mar 20 13:52:00 faucet.valve INFO     Responded to ND solicit for fc00::10:10:10:1 to fc00::1 (92:06:7e:ac:62:d6)
Mar 20 13:52:00 faucet.valve INFO     learned 2 hosts on vlan 2040
Mar 20 13:52:00 faucet.valve INFO     Adding new route fc00::1/128 via fc00::1 (92:06:7e:ac:62:d6)
Mar 20 13:52:00 faucet.valve DEBUG    Packet_in DPID 1 (0x1) src:92:06:7e:ac:62:d6 in_port:1 vid:2040
Mar 20 13:52:01 faucet.valve DEBUG    Packet_in DPID 1 (0x1) src:92:06:7e:ac:62:d6 in_port:1 vid:2040
Mar 20 13:52:02 faucet.valve DEBUG    Packet_in DPID 1 (0x1) src:92:06:7e:ac:62:d6 in_port:1 vid:2040
Mar 20 13:52:02 faucet.valve INFO     learned 2 hosts on vlan 2040
Mar 20 13:52:03 faucet.valve DEBUG    Packet_in DPID 1 (0x1) src:92:06:7e:ac:62:d6 in_port:1 vid:2040
Mar 20 13:52:04 faucet.valve DEBUG    Packet_in DPID 1 (0x1) src:92:06:7e:ac:62:d6 in_port:1 vid:2040
Mar 20 13:52:04 faucet.valve INFO     learned 2 hosts on vlan 2040
Mar 20 13:52:05 faucet.valve DEBUG    Packet_in DPID 1 (0x1) src:92:06:7e:ac:62:d6 in_port:1 vid:2040
Mar 20 13:52:06 faucet.valve DEBUG    Packet_in DPID 1 (0x1) src:92:06:7e:ac:62:d6 in_port:1 vid:2040
Mar 20 13:52:06 faucet.valve INFO     learned 2 hosts on vlan 2040
Mar 20 13:53:06 faucet.valve INFO     expiring host f2:5f:74:ae:72:0b from vlan 2040
Mar 20 13:53:06 faucet.valve INFO     1 recently active hosts on vlan 2040
Mar 20 13:55:44 faucet.valve DEBUG    Packet_in DPID 1 (0x1) src:f2:5f:74:ae:72:0b in_port:2 vid:2040
Mar 20 13:55:44 faucet.valve INFO     learned 2 hosts on vlan 2040
Mar 20 13:57:10 faucet.valve INFO     expiring host 92:06:7e:ac:62:d6 from vlan 2040
Mar 20 13:57:10 faucet.valve INFO     1 recently active hosts on vlan 2040
Mar 20 14:00:26 faucet.valve INFO     resolving fc00::1
Mar 20 14:00:29 faucet.valve INFO     resolving fc00::1 retry 2 (last attempt was 3s ago)
Mar 20 14:00:36 faucet.valve INFO     resolving fc00::1 retry 3 (last attempt was 7s ago)

Note: there is no additional logging in the faucet_exception.log or faucet.log, and the process does NOT terminate.

Faucet: Exception occurred during handler processing. Backtrace from offending handler [resolve_gateways] servicing event [EventFaucetResolveGateways] follows.
Traceback (most recent call last):
  File "/home/mininet/venv36/lib/python3.6/site-packages/ryu/base/app_manager.py", line 290, in _event_loop
    handler(ev)
  File "/home/mininet/code/faucet/src/ryu_faucet/org/onfsdn/faucet/faucet.py", line 385, in resolve_gateways
    flowmods = valve.resolve_gateways()
  File "/home/mininet/code/faucet/src/ryu_faucet/org/onfsdn/faucet/valve.py", line 1112, in resolve_gateways
    ofmsgs.extend(self.ipv6_route_manager.resolve_gateways(vlan, now))
  File "/home/mininet/code/faucet/src/ryu_faucet/org/onfsdn/faucet/valve_route.py", line 342, in resolve_gateways
    ip_gw, faucet_vip, vlan, ports))
  File "/home/mininet/code/faucet/src/ryu_faucet/org/onfsdn/faucet/valve_route.py", line 109, in _neighbor_resolver
    vid, faucet_vip, ip_gw)
  File "/home/mininet/code/faucet/src/ryu_faucet/org/onfsdn/faucet/valve_route.py", line 602, in _neighbor_resolver_pkt
    self.faucet_mac, vid, faucet_vip.ip, ip_gw)
  File "/home/mininet/code/faucet/src/ryu_faucet/org/onfsdn/faucet/valve_packet.py", line 182, in nd_request
    nd_mac = ipv6_link_eth_mcast(dst_ip)
  File "/home/mininet/code/faucet/src/ryu_faucet/org/onfsdn/faucet/valve_packet.py", line 151, in ipv6_link_eth_mcast
    mcast_mac = ':'.join(['%02X' % ord(x) for x in mcast_mac_bytes])
  File "/home/mininet/code/faucet/src/ryu_faucet/org/onfsdn/faucet/valve_packet.py", line 151, in <listcomp>
    mcast_mac = ':'.join(['%02X' % ord(x) for x in mcast_mac_bytes])
TypeError: ord() expected string of length 1, but int found

I replaced the offending code with a call to:

def _bytes_to_mac(value):
    if isinstance(value, str):
        return ':'.join(['%02X' % ord(x) for x in value])
    return ':'.join(['%02X' % x for x in value])

We should no longer support the "v1" standalone configuration type, as it can support only one dataplane, and we expect only "v2" and later on new installations. Tests currently support "v1" only and need to be migrated also.

The function import_config reads the DPID from the hw_switch_config.yaml and stores it in decimal.
But the function get_config_header attempts to read it as a hex and converts it to decimal again and writes it into the faucet.yaml file.

I've noticed a few places where Faucet uses field masks for an exact match on an IPv4 or IPv6 address. It doesn't seem to hurt anything, but I think it's unnecessary.

---
type:            FLOW_MOD
time:            1491349340.242006000
xid:             0xFFA1AC8F
version:         0x04
conn_id:         6
src:             127.0.0.1:6653
dst:             127.0.0.1:33754
msg:             
  cookie:          0x000000005ADC15C0
  cookie_mask:     0x0000000000000000
  table_id:        0x03
  command:         ADD
  idle_timeout:    0x0000
  hard_timeout:    0x0000
  priority:        0x23AB
  buffer_id:       NO_BUFFER
  out_port:        0x00000000
  out_group:       0x00000000
  flags:           [  ]
  match:           
    - field:           ETH_TYPE
      value:           0x0806
    - field:           VLAN_VID
      value:           0x17F8
    - field:           ARP_TPA
      value:           10.10.10.1
      mask:            255.255.255.255     # Don't need mask on exact match
  instructions:    
    - instruction:     APPLY_ACTIONS
      actions:         
        - action:          OUTPUT
          port_no:         CONTROLLER
          max_len:         0x0080
    - instruction:     GOTO_TABLE
      table_id:        0x06
---
type:            FLOW_MOD
time:            1491349340.242018000
xid:             0xFFA1AC91
version:         0x04
conn_id:         6
src:             127.0.0.1:6653
dst:             127.0.0.1:33754
msg:             
  cookie:          0x000000005ADC15C0
  cookie_mask:     0x0000000000000000
  table_id:        0x04
  command:         ADD
  idle_timeout:    0x0000
  hard_timeout:    0x0000
  priority:        0x23AB
  buffer_id:       NO_BUFFER
  out_port:        0x00000000
  out_group:       0x00000000
  flags:           [  ]
  match:           
    - field:           ETH_TYPE
      value:           0x0800
    - field:           VLAN_VID
      value:           0x17F8
    - field:           IP_PROTO
      value:           0x01
    - field:           IPV4_SRC
      value:           10.0.0.0
      mask:            255.0.0.0
    - field:           IPV4_DST
      value:           10.10.10.1
      mask:            255.255.255.255    # Don't need mask for exact match
  instructions:    
    - instruction:     APPLY_ACTIONS
      actions:         
        - action:          OUTPUT
          port_no:         CONTROLLER
          max_len:         0x0080
---
type:            FLOW_MOD
time:            1491349341.206877000
xid:             0xFFA1ACD6
version:         0x04
conn_id:         6
src:             127.0.0.1:6653
dst:             127.0.0.1:33754
msg:             
  cookie:          0x000000005ADC15C0
  cookie_mask:     0x0000000000000000
  table_id:        0x04
  command:         ADD
  idle_timeout:    0x0000
  hard_timeout:    0x0000
  priority:        0x23AB
  buffer_id:       NO_BUFFER
  out_port:        0x00000000
  out_group:       0x00000000
  flags:           [  ]
  match:           
    - field:           ETH_TYPE
      value:           0x0800
    - field:           VLAN_VID
      value:           0x17F8
    - field:           IPV4_DST
      value:           10.0.0.1
      mask:            255.255.255.255    # Don't need mask for exact match
  instructions:    
    - instruction:     APPLY_ACTIONS
      actions:         
        - action:          SET_FIELD
          field:           ETH_SRC
          value:           '0e:00:00:00:00:01'
        - action:          SET_FIELD
          field:           ETH_DST
          value:           'ce:1d:40:de:0d:8e'
        - action:          DEC_NW_TTL
    - instruction:     GOTO_TABLE
      table_id:        0x06
---
type:            FLOW_MOD
time:            1491349340.250914000
xid:             0xFFA1ACBA
version:         0x04
conn_id:         6
src:             127.0.0.1:6653
dst:             127.0.0.1:33754
msg:             
  cookie:          0x000000005ADC15C0
  cookie_mask:     0x0000000000000000
  table_id:        0x05
  command:         ADD
  idle_timeout:    0x0000
  hard_timeout:    0x0000
  priority:        0x240B
  buffer_id:       NO_BUFFER
  out_port:        0x00000000
  out_group:       0x00000000
  flags:           [  ]
  match:           
    - field:           ETH_TYPE
      value:           0x86DD
    - field:           VLAN_VID
      value:           0x17F8
    - field:           IP_PROTO
      value:           0x3A
    - field:           IPV6_DST
      value:           'fc00::10:10:10:1'
      mask:            'ffff:ffff:ffff:ffff:ffff:ffff:ffff:ffff'     # Don't need mask for exact match
    - field:           ICMPV6_TYPE
      value:           0x80
  instructions:    
    - instruction:     APPLY_ACTIONS
      actions:         
        - action:          OUTPUT
          port_no:         CONTROLLER
          max_len:         0x0080

The YAML output above is produced using my oftr tool:
https://launchpad.net/~byllyfish/+archive/ubuntu/oftr

I'm using latest master (2017-04-04). I have two faucet_vips: [ "10.10.10.1/8", "fc00::10:10:10:1/64" ]

Input ACLs are supported; also support ACLs on output.

Today, when JSON object is received by CouchDB from Gauge, we get only Switch DPID. Can we get the Switch name too?

Also, Faucet creates OF pipelines and pushes them into the switch. When tables and flow info is sent to CouchDB, the JSON object contains Table numbers but not a friendly name. Not having this means one has to look at the pipeline to derive the name. Hence, request table name as programmed by the pipeline (VLAN_ACL, PORT_ACL, FLOOD, etc) be sent with TableID to CouchDB.
Another option here is that the table id-name mapping can be sent separately and stored in a different table which is refreshed if pipeline is changed.

In summary, both of the above features require name mapping for switch and table id be available in the Couch database so that applications have easy access to them for presentation to users.

Since commit 8c62149 we have introduced a regression with Faucet routing and versions of Ryu <v4.9 which don't support the enable_ipv6 argument.

Stack trace is here:

ubuntu@faucet:~$ sudo start-faucet
loading app /usr/local/lib/python2.7/dist-packages/ryu_faucet/org/onfsdn/faucet//faucet.py
Generating grammar tables from /usr/lib/python2.7/lib2to3/Grammar.txt
Generating grammar tables from /usr/lib/python2.7/lib2to3/PatternGrammar.txt
loading app ryu.controller.ofp_handler
instantiating app None of DPSet
creating context dpset
instantiating app /usr/local/lib/python2.7/dist-packages/ryu_faucet/org/onfsdn/faucet//faucet.py of Faucet
API method core.start called with args: {'router_id': '127.0.0.2', 'label_range': (100, 100000), 'waiter': <ryu.lib.hub.Event object at 0x7f9c90a56d90>, 'local_as': 9727, 'bgp_server_port': 9179, 'refresh_max_eor_time': 0, 'refresh_stalepath_time': 0}
API method network.add called with args: {'prefix': '202.74.195.254/22', 'next_hop': '202.74.195.254'}
API method network.add called with args: {'prefix': '172.16.0.3/24', 'next_hop': '172.16.0.3'}
API method network.add called with args: {'prefix': '2401:7000:69:4000::ffff/64', 'next_hop': '2401:7000:69:4000::ffff'}
API method network.add called with args: {'prefix': '2401:7000:69:43e8::f/124', 'next_hop': '2401:7000:69:43e8::f'}
API method network.add called with args: {'prefix': 'fe80::20d:88ff:fe00:aa/64', 'next_hop': 'fe80::20d:88ff:fe00:aa'}
API method network.add called with args: {'prefix': '2401:7000:69:4000::fffe/128', 'next_hop': '2401:7000:69:4000::fffe'}
API method network.add called with args: {'prefix': '2401:7000:69::/64', 'next_hop': '2401:7000:69:43e8::3'}
Traceback (most recent call last):
File "/usr/local/bin/ryu-manager", line 11, in
load_entry_point('ryu==4.8.1', 'console_scripts', 'ryu-manager')()
File "/usr/local/lib/python2.7/dist-packages/ryu/cmd/manager.py", line 91, in main
services.extend(app_mgr.instantiate_apps(**contexts))
File "/usr/local/lib/python2.7/dist-packages/ryu/base/app_manager.py", line 509, in instantiate_apps
self._instantiate(app_name, cls, *args, **kwargs)
File "/usr/local/lib/python2.7/dist-packages/ryu/base/app_manager.py", line 495, in _instantiate
app = cls(*args, **kwargs)
File "/usr/local/lib/python2.7/dist-packages/ryu_faucet/org/onfsdn/faucet/faucet.py", line 121, in init
self._reset_bgp()
File "/usr/local/lib/python2.7/dist-packages/ryu_faucet/org/onfsdn/faucet/faucet.py", line 201, in _reset_bgp
bgp_speakers[vlan] = self._create_bgp_speaker_for_vlan(vlan)
File "/usr/local/lib/python2.7/dist-packages/ryu_faucet/org/onfsdn/faucet/faucet.py", line 186, in _create_bgp_speaker_for_vlan
enable_ipv6=True)
TypeError: neighbor_add() got an unexpected keyword argument 'enable_ipv6'

I am using the git cloned version of faucet as of 3/29/2017. Faucet logs shows:

Mar 29 16:46:18 faucet.exception ERROR    Unhandled exception, killing RYU
Traceback (most recent call last):
  File "/usr/local/lib/python2.7/dist-packages/faucet/valve_util.py", line 52, in __koe
    func(*args, **kwargs)
  File "/usr/local/lib/python2.7/dist-packages/faucet/faucet.py", line 420, in _packet_in_handler
    pkt = packet.Packet(msg.data)
  File "/usr/local/lib/python2.7/dist-packages/ryu/lib/packet/packet.py", line 67, in __init__
    self._parser(parse_cls)
  File "/usr/local/lib/python2.7/dist-packages/ryu/lib/packet/packet.py", line 76, in _parser
    proto, cls, rest_data = cls.parser(rest_data)
  File "/usr/local/lib/python2.7/dist-packages/ryu/lib/packet/icmpv6.py", line 130, in parser
    msg.data = cls_.parser(buf, offset)
  File "/usr/local/lib/python2.7/dist-packages/ryu/lib/packet/icmpv6.py", line 367, in parser
    options.append(option)
MemoryError

I don't have instructions to replicate it yet. It is happening on my machine :-( Will investigate more and add to this bug report. But, I did want to make sure it is recorded.

All the tests that use tcpdump_helper() were failing in my environment. By adding some logging to tcpdump_helper, I discovered that tcpdump was returning an error message:

tcpdump: ethernet addresses supported only on ethernet/FDDI/token ring/802.11/ATM LANE/Fibre Channel

The tcpdump_helper function is not specifying an interface using -i, so tcpdump is using the lowest numbered active interface. On ubuntu/trusty with libpcap 1.5.3, a netfilter interface is first. On the docker image (ubuntu/xenial with libpcap 1.7.4), the mininet host's eth0 interface is first, so it works fine.

I changed the tcpdump_helper code to explicitly specify the host's default interface, and mininet tests now run successfully on trusty. The code now asserts tcpdump_started.

I also noticed that you're testing the version of ladvd with "-v". That's the "verbose" option. You want the "-h" option.

Here is the patch:

diff --git a/tests/faucet_mininet_test.py b/tests/faucet_mininet_test.py
index e99ce7a..0ff744a 100755
--- a/tests/faucet_mininet_test.py
+++ b/tests/faucet_mininet_test.py
@@ -83,7 +83,7 @@ EXTERNAL_DEPENDENCIES = (
      r'pylint (\d+\.\d+).\d+,', "1.6"),
     ('curl', ['--version'], 'libcurl',
      r'curl (\d+\.\d+).\d+', "7.3"),
-    ('ladvd', ['-v'], 'ladvd',
+    ('ladvd', ['-h'], 'ladvd',
      r'ladvd version (\d+\.\d+)\.\d+', "1.1"),
 )
 
@@ -224,10 +224,9 @@ class FaucetTest(faucet_mininet_test_base.FaucetTestBase):
 
     def tcpdump_helper(self, tcpdump_host, tcpdump_filter, funcs=[],
                        timeout=10, packets=2):
-        tcpdump_out = tcpdump_host.popen(
-            'timeout %us tcpdump -e -n -U -v -c %u %s' % (
-                timeout, packets, tcpdump_filter),
-            stderr=subprocess.STDOUT)
+        tcpdump_cmd = 'timeout %us tcpdump -i %s -e -n -U -v -c %u %s' % (
+            timeout, tcpdump_host.intf().name, packets, tcpdump_filter)
+        tcpdump_out = tcpdump_host.popen(tcpdump_cmd, stderr=subprocess.STDOUT)
         popens = {tcpdump_host: tcpdump_out}
         tcpdump_started = False
         tcpdump_txt = ''
@@ -235,12 +234,14 @@ class FaucetTest(faucet_mininet_test_base.FaucetTestBase):
             if host == tcpdump_host:
                 if tcpdump_started:
                     tcpdump_txt += line.strip()
-                else:
+                elif re.search('tcpdump: listening on ', line):
                     # when we see tcpdump start, then call provided functions.
-                    if re.search('tcpdump: listening on ', line):
-                        tcpdump_started = True
-                        for func in funcs:
-                            func()
+                    tcpdump_started = True
+                    for func in funcs:
+                        func()
+                else:
+                    print('tcpdump_helper: %s' % line)
+        self.assertTrue(tcpdump_started)
         return tcpdump_txt
 
     def bogus_mac_flooded_to_port1(self):

I am trying to run gauge

I can see switch s1 is added in both faucet and gauge controller, but I am not getting any stats from switch
my gauge log

Feb 26 01:57:20 gauge  INFO     DPID 1 (0x1) up
Feb 26 01:57:35 gauge  INFO     port stats request timed out for s1
Feb 26 01:57:40 gauge  INFO     port stats request timed out for s1

My VM IP: 192.168.16.129, 192.168.171.131
my config files:

• faucet.yaml

version: 2
vlans:
    4090:
        name: "trusted network"
        unicast_flood: True
dps:
    s1:
        dp_id: 1
        description: "Switch"
        interfaces:
            1:
                native_vlan: 4090
                name: "port1.0.1"
            2:
                native_vlan: 4090
                name: "port1.0.2"

• gauge.yaml

version: 2
faucet_configs:
    - '/etc/ryu/faucet/faucet.yaml'
watchers:
    port_state_poller:
        type: 'port_state'
        dps: ['s1']
        db: 'influx'
        interval: 5

    port_stats_poller:
        type: 'port_stats'
        dps: ['s1']
        db: 'influx'
        interval: 5
dbs:
    influx:
        type: 'influx'
        influx_db: 'faucet'
        influx_host: 'localhost'
        influx_port: 8086
        influx_user: 'faucet'
        influx_pwd: '123'
        influx_timeout: 10

Steps I followed:

• Start controller
  sudo ryu-manager --ofp-tcp-listen-port=6633 faucet/src/ryu_faucet/org/onfsdn/faucet/faucet.py
• Start mininet and add gauge controller
  sudo mn --mac --controller=remote,ip=192.168.16.129
sh ovs-vsctl set-controller s1 tcp:192.168.16.129:6633 tcp:192.168.16.129:6663
• Start gauge controller
  sudo ryu-manager --verbose --ofp-tcp-listen-port=6663 faucet/src/ryu_faucet/org/onfsdn/faucet/gauge.py

Always getting port stats request timed out for s1

Hi,

I am doing the experiment with my mini-server running Lagopus as an open flow switch. I want to implement the Vlan into my network topology by using faucet.py through ryu controller and it doesn't seem to work really well. However, if i use OVS and create the network topology with mininet, the faucet.py work just fine. So my question is how can i get the faucet.py works with Lagopus ?

Vlan flow.pdf

Thanks
Panha

I'm looking at modifying the pipeline around ACLs, and maybe a couple of other things. Which has raised the question:

To what extent should we avoid using more and more openflow tables?

So should we be trying to use as few as possible, even when that isnt particularly logical, for example with the vlan table we are also filtering out BPDUs. I would prefer to have an initial filtering table, to keep things logically separated. But if this makes things significantly harder for hardware vendors, or would cause a considerable decrease in performance with ovs I think we probably ought to compromise (as these changes are somewhat cosmetic).

I could also see an upside to this by reducing the number of fields certain tables will need to match on.

So is anyone aware of issues with switch implementations when using more tables? (and by more tables, I am talking about in the order of maybe 20 at the absolute most).

At the moment when you pip install ryu-faucet you get a lot of files in the config directory /etc/ryu/faucet/:

ubuntu@faucet:~/faucet$ ls /etc/ryu/faucet/
faucet_demo_step1.yaml faucet_demo_step2.yaml faucet_demo_step3.yaml faucet_Lagopus.yaml faucet_ovs.yaml faucet.yaml faucet_ZodiacFX.yaml gauge.yaml

And unfortunately faucet.yaml and gauge.yaml are installed as data files (which means when you pip uninstall ryu-faucet these files will be deleted as well even if they have user changes).

I propose we install a /etc/ryu/faucet/faucet.yaml-dist and /etc/ryu/faucet/gauge.yaml-dist which are fully featured configuration examples which users can copy into the appropriate file names and make their changes so that PIP doesn't step on your toes and delete them.

Another solution may also be to flag them as configuration files that PIP will assume that user changes will be made to these files and it won't delete them. I know you can do that with DEB files but I'm not familiar enough with PIP to know if the same concept exists?

For robustness we probably shouldn't crash. Someone may try and reload Faucet into a new config with placeholder datapaths and find their instance crashes.

Traceback (most recent call last):
File "/usr/local/bin/ryu-manager", line 11, in
load_entry_point('ryu==4.8.1', 'console_scripts', 'ryu-manager')()
File "/usr/local/lib/python2.7/dist-packages/ryu/cmd/manager.py", line 91, in main
services.extend(app_mgr.instantiate_apps(**contexts))
File "/usr/local/lib/python2.7/dist-packages/ryu/base/app_manager.py", line 509, in instantiate_apps
self._instantiate(app_name, cls, *args, **kwargs)
File "/usr/local/lib/python2.7/dist-packages/ryu/base/app_manager.py", line 495, in _instantiate
app = cls(*args, **kwargs)
File "/usr/local/lib/python2.7/dist-packages/ryu_faucet/org/onfsdn/faucet/faucet.py", line 105, in init
self.config_file, self.logname)
File "/usr/local/lib/python2.7/dist-packages/ryu_faucet/org/onfsdn/faucet/config_parser.py", line 58, in dp_parser
config_hashes, dps = _dp_parser_v2(conf, config_file, logname)
File "/usr/local/lib/python2.7/dist-packages/ryu_faucet/org/onfsdn/faucet/config_parser.py", line 253, in _dp_parser_v2
for port_num, port_conf in ports_conf.iteritems():
AttributeError: 'NoneType' object has no attribute 'iteritems'

At the moment we enforce a few default drop flows in _add_default_drop_flows() which is in valve.py.

There are scenarios where you wouldn't want faucet antispoof etc so we should make these optional.

faucet/src/ryu_faucet/org/onfsdn/faucet/ryu-faucet.conf

If so, what is the purpose?

@shivarammysore - CouchDB support in Gauge has no tests. In particular, it does not handle CouchDB being down (if CouchDB is down Gauge will crash).

CouchDB should get tests in the same way as InfluxDB is tested (ie the most basic down/up test).

instantiating app /home/faucet/faucet/src/ryu_faucet/org/onfsdn/faucet/gauge.py of Gauge
Traceback (most recent call last):
File "/usr/local/bin/ryu-manager", line 9, in
load_entry_point('ryu==4.11', 'console_scripts', 'ryu-manager')()
File "/usr/local/lib/python2.7/dist-packages/ryu/cmd/manager.py", line 81, in main
services.extend(app_mgr.instantiate_apps(**contexts))
File "/usr/local/lib/python2.7/dist-packages/ryu/base/app_manager.py", line 509, in instantiate_apps
self._instantiate(app_name, cls, *args, **kwargs)
File "/usr/local/lib/python2.7/dist-packages/ryu/base/app_manager.py", line 495, in _instantiate
app = cls(*args, **kwargs)
File "/home/faucet/faucet/src/ryu_faucet/org/onfsdn/faucet/gauge.py", line 77, in init
watcher = watcher_factory(conf)(conf, self.logname)
File "/home/faucet/faucet/src/ryu_faucet/org/onfsdn/faucet/watcher.py", line 455, in init
self.setup()
File "/home/faucet/faucet/src/ryu_faucet/org/onfsdn/faucet/watcher.py", line 106, in setup
self.switch_database, exists = self.conn.create(self.conf.switches_doc)
File "/home/faucet/faucet/src/ryu_faucet/org/onfsdn/faucet/nsodbc.py", line 91, in create
self.database[db_name] = DatabaseCouch(self.conn.create(db_name))
File "/usr/local/lib/python2.7/dist-packages/couchdb/client.py", line 207, in create
self.resource.put_json(name)
File "/usr/local/lib/python2.7/dist-packages/couchdb/http.py", line 570, in put_json
**params)
File "/usr/local/lib/python2.7/dist-packages/couchdb/http.py", line 585, in _request_json
headers=headers, **params)
File "/usr/local/lib/python2.7/dist-packages/couchdb/http.py", line 581, in _request
credentials=self.credentials)
File "/usr/local/lib/python2.7/dist-packages/couchdb/http.py", line 289, in request
conn = self.connection_pool.get(url)
File "/usr/local/lib/python2.7/dist-packages/couchdb/http.py", line 507, in get
conn.connect()
File "/usr/lib/python2.7/httplib.py", line 836, in connect
self.timeout, self.source_address)
File "/usr/local/lib/python2.7/dist-packages/eventlet/green/socket.py", line 69, in create_connection
raise err
socket.error: [Errno 111] ECONNREFUSED

Ryu 4.10 has just been released and our test suite doesn't run because it parses 4.10 as version 4.1 which is less than our required 4.9.

I think we want to use this function to parse python versions:

https://packaging.pypa.io/en/latest/version/#packaging.version.parse

At the moment we tend to crash when using entire-line comments in faucet.yaml.

Say for example you want to document an ACL rule:

    # Allow unicast ipv6 from fe80::/64
    - rule:
        dl_type: 0x86dd
        nw_src: fe80::/64
        actions:
            allow: 1

Pyyaml will parse and remove everything after (and including) the # which means the config parse will get fed a blank line with spaces on it.

This tends to give us the following crash:

Traceback (most recent call last):
File "/usr/local/bin/ryu-manager", line 9, in
load_entry_point('ryu==4.9', 'console_scripts', 'ryu-manager')()
File "/usr/local/lib/python2.7/dist-packages/ryu/cmd/manager.py", line 81, in main
services.extend(app_mgr.instantiate_apps(**contexts))
File "/usr/local/lib/python2.7/dist-packages/ryu/base/app_manager.py", line 509, in instantiate_apps
self._instantiate(app_name, cls, *args, **kwargs)
File "/usr/local/lib/python2.7/dist-packages/ryu/base/app_manager.py", line 495, in _instantiate
app = cls(*args, **kwargs)
File "/usr/local/lib/python2.7/dist-packages/ryu_faucet/org/onfsdn/faucet/faucet.py", line 105, in init
self.config_file, self.logname)
TypeError: 'NoneType' object is not iterable

I think there may be some serious issues around the thread safety of the parallel tests.

But on three separate environments the docker tests fail with a fresh pull from master. I have seen different sets of tests fail and if you run the tests singly or in serial they pass.

I think these issues may have existed for some time. I doubt it relates to any of the new code. But I haven't been able to find the cause.

Right now, Valve reloads all flows (reprovisions the entire pipeline) on a reload request. It should reload just affected flows (eg ACL, or to move a port into a different VLAN), if it can, with a full restart as a last resort.

There are a couple of places where you hard-wire the system python interpreter. This presents a problem when using a virtualenv, because my system python doesn't have the loaded dependencies.

(venv27):~/code/faucet$ grep -r "/usr/bin/python" *
faucet/check_faucet_config.py:#!/usr/bin/python
setup.py:#!/usr/bin/python
tests/test_check_config.py:#!/usr/bin/python
tests/faucet_mininet_test.py:#!/usr/bin/python
tests/test_config.py:#!/usr/bin/python
tests/test_valve.py:#!/usr/bin/python
tests/faucet_mininet_test_base.py:#!/usr/bin/python
tests/faucet_mininet_test_util.py:#!/usr/bin/python

faucet-1967 gauge-1967 
*** Starting 3 switches
s0b s0c s0d ...
*** Waiting for switches to connect
__main__.FaucetAPITest.test_api (subunit.RemotedTestCase)
__main__.FaucetAPITest.test_api ... FAIL
error: __main__.FaucetAPITest.test_api [
Traceback (most recent call last):
  File "/home/mininet/faucet/tests/faucet_mininet_test_base.py", line 212, in tearDown
    0, os.path.getsize(os.environ['FAUCET_EXCEPTION_LOG']))
  File "/usr/lib/python2.7/genericpath.py", line 57, in getsize
    return os.stat(filename).st_size
OSError: [Errno 2] No such file or directory: '/tmp/faucet-tests-y6Typi/FaucetAPITest-test_api-JN9Gab/faucet-exception.log'
faucet-1967 gauge-1967 
*** Starting 3 switches
s0b s0c s0d ...
*** Waiting for switches to connect
__main__.FaucetAPITest.test_api (subunit.RemotedTestCase)
__main__.FaucetAPITest.test_api ... FAIL
error: __main__.FaucetAPITest.test_api [
Traceback (most recent call last):
  File "/home/mininet/faucet/tests/faucet_mininet_test_base.py", line 212, in tearDown
    0, os.path.getsize(os.environ['FAUCET_EXCEPTION_LOG']))
  File "/usr/lib/python2.7/genericpath.py", line 57, in getsize
    return os.stat(filename).st_size
OSError: [Errno 2] No such file or directory: '/tmp/faucet-tests-y6Typi/FaucetAPITest-test_api-JN9Gab/faucet-exception.log'
]
*** Creating network
*** Adding controller
*** Adding hosts:
u131 u132 u141 u142 
*** Adding switches:
s13 s14 
*** Adding links:
(s13, s14) (u131, s13) (u132, s13) (u141, s14) (u142, s14) 
*** Configuring hosts
u131 u132 u141 u142 
*** Starting controller
faucet-1964 gauge-1964 
*** Starting 2 switches
s13 s14 ...
*** Waiting for switches to connect

On a fresh Ubuntu 16.04 server install, i get the above error when trying to run "sudo ./faucet_mininet_test.py". Where the test fails to progress for over 30 minutes.

I've installed all the dependencies that the test requests and cloned/installed mininet

commit d528da4
mininet: 2.3.0d1
required binary/library ryu-manager --version version is 4.11
required binary/library ovs-vsctl --version version is 2.5
required binary/library tcpdump -h version is 4.9
required binary/library nc present (nc from the netcat-openbsd)
required binary/library vconfig present (the VLAN you are talking about)
required binary/library fuser -V version is 22.21
required binary/library mn --version version is 2.3
required binary/library exabgp --version version is 3.4
required binary/library pip show influxdb version is 4.0
required binary/library pylint --version version is 1.6
required binary/library curl --version version is 7.47
required binary/library ladvd -v version is 1.1

@mbruyere

We need to support Umbrella (http://www.touix.net/en/content/tousix-project-technical-overview).

Umbrella essentially hard codes broadcast to unicast (where unicast destinations are all known in advance, having been provisioned). This can be accomplished with ACLs.

However we also need the failover functionality of groups (we will support a port failover option using groups).

I am doing the experiment with my mini-server running Lagopus as an open flow switch. I want to implement the Vlan into my network topology by using faucet.py throw ryu controller and it doesn't work. However, if i use OVS and create the network topology with mininet, the faucet.py work just fine. So my question is how can i get the faucet.py works with Lagopus ?

Thanks
Panha

When a change is made to the config file, and the reload is triggered by sending a SIGHUP, faucet removes rules from the eth_src_table.

However with the permanent_learn flag set, a MAC address that has been learnt on the permanent_learn port will not be re-learnt, and all it's packets will be dropped.

There is a rule for antispoofing here:
https://github.com/REANNZ/faucet/blob/master/faucet/valve_host.py#L121
that is installed for hosts on permanent_learn ports, which matches packets with the same MAC address, from any in_port and drops them. This means that as the MAC is no longer learned (rule was deleted on config reload) it is unable to be re-learnt.

From what I can tell,
https://github.com/REANNZ/faucet/blob/master/faucet/valve.py#L702
and
https://github.com/REANNZ/faucet/blob/master/faucet/valve.py#L395-L401
is the code that deletes the rules from eth_src_table. and is matching on the in_port, which the antispoofing rule does not have.

Table 3 before:

cookie=0x5adc15c0, duration=195.089s, table=3, table=3, n_packets=148, n_bytes=14504, priority=9098,in_port=1,dl_vlan=100,dl_src=08:00:27:00:03:02 actions=goto_table:6
 cookie=0x5adc15c0, duration=195.050s, table=3, n_packets=147, n_bytes=14406, hard_timeout=300, priority=9098,in_port=2,dl_vlan=100,dl_src=52:54:00:12:35:02 actions=goto_table:6
 cookie=0x5adc15c0, duration=195.089s, table=3, n_packets=0, n_bytes=0, priority=9097,dl_vlan=100,dl_src=08:00:27:00:03:02 actions=drop
 cookie=0x5adc15c0, duration=196.030s, table=3, n_packets=0, n_bytes=0, priority=0 actions=drop

Table 3 after config reload:

 cookie=0x5adc15c0, duration=423.878s, table=3, n_packets=8, n_bytes=784, priority=9097,dl_vlan=100,dl_src=08:00:27:00:03:02 actions=drop
 cookie=0x5adc15c0, duration=424.819s, table=3, n_packets=0, n_bytes=0, priority=0 actions=drop
 cookie=0x5adc15c0, duration=424.819s, table=3, n_packets=2, n_bytes=196, priority=9000 actions=CONTROLLER:128,goto_table:6

From Chris:

==
I think we definitely will need to support multiple fibs per box. These should be entirely independently of each other (eg. should be able to share address ranges). And they each should be able to forward and receive traffic from multiple arbitrary vlans.

I dont think they should necessarily be able to forward and receive traffic from the same vlan, I suspect this is a pointless use case that will make life very difficult for us.

This is to fit with our philosophy of focusing on the underlying physical necessities, without sweating about how people will want to use them (eg. how we dont have trunk ports).

So off the top of my head I would think we could configure this like this:

dps:
..dp1:
....dpid: 1
....interfaces:
........if1:
..........tagged_vlans: [v1,v2,v3]
vlans:
..v1:
....vid: 1
....faucet_vips: [192.168.1.1/24]
..v2:
....vid: 2
....faucet_vips: [192.168.2.1/24]
..v3:
....vid: 3
....faucet_vips:[192.168.0.1/16]
routers:
..r1:
....vlans: [v1, v2]
..r2
....vlans: [v3]

And then r1 can route between 192.168.1.0/24 and 192.168.2.0/24 and r2 can deal with overlapping addresses in its vlan without a worry.

How the fibs get populated is up to something other than faucet, so if they act as vrfs or virtual routers is someone elses concern. But you can just use bgp to throw the routes into there that you want to have appear there (we should also add an 'add_route' method to the api, so people can do this that way).

Then for PBR you can have some kind of action that is jump_to_fib or something.

I think bgp should be on a separate ryu app. It will be a central point to peer with all neighbors and handle bgp logic. Faucet will be responsible for converting and install fib entries.
If bgp is not needed, we just need not to run bgp app.
I'm now working on it.

routes are missed with the add_resolved_routes method.
if multiple routes share the same gateway, only the first one is added to eth_src_table.

Currently, each VLAN has its own BGP configs and thus, its own BGP speaker.
But Ryu seems not to support two speakers in the same PC.
I run a test and got this message:
Traceback (most recent call last):
File "/usr/local/lib/python2.7/dist-packages/ryu/services/protocols/bgp/api/base.py", line 204, in call
return call(**kwargs)
File "/usr/local/lib/python2.7/dist-packages/ryu/services/protocols/bgp/api/core.py", line 37, in start
raise RuntimeConfigError('Current context has to be stopped to start '
RuntimeConfigError: 200.1 - Current context has to be stopped to start a new context.

ryu.services.protocols.bgp.rtconf.base.RuntimeConfigError: 200.1 - Current context has to be stopped to start a new context.

I think Faucet should have a single BGP config that works for all VLANs, making a VLAN an IP interface.

I am touching up my change that does most of this. I just wanted to check a couple of things through with you guys before I change too much more.

Firstly this is kind of a big change. Here is an overview:

The main goal is to have consistency between how faucet and gauge are configured. I migrate gauge to using yaml, and make the database configuration actually configurable. I also try to make gauge and faucet more consistent in how the code is structured.

The gauge config looks something like this:

valve_configs:
        - "faucet1.yaml"
        - "/etc/ryu/faucet/faucet2.yaml"
pollers:
        flow_table_poller:     #the name of the poller
                gauge_type: "flow_table"
                dps: ["switch-1", "switch-2", "switch-3"]
                interval: 60
                db: "flow_table_file"
        port_state_poller:
                gauge_type:"port_state"
                dps: ["office-switch-1"]
                db: "influx"
dbs:
        influx:
               type: "influx"
                db: "faucet"
                host: "localhost"
                port: 8086
        flow_table_file:
                type: "text"
               output_file: "gauge_flow_table.txt"

For consistencies sake I would like to change it so all of the configuration for faucet key's things by names rather than by the id number (eg port number, vid or dp_id). We can do this easily without breaking existing configs. My plan is that we check if a dp_id is configured elsewhere. If not then the key is the dp_id. So this will not break existing configs so doesnt need a new config version.

So the config for valve would look more like:

---
   version: 2

   dps:
      test-switch-1:
           dp_id: 0x000000000002
           interfaces:
               1:
                   native_vlan: 2040
                   acl_in: 1
       test-switch-2:
           dp_id: 0x000000000002
           interfaces:
               port_1:
                   of_number: 1
                   native_vlan: 2040
                   acl_in: 1

   vlans:
       "dev VLAN":
           vid: 2040

   acls:
       1:
           - rule:
               nw_dst: "172.0.0.0/8"
               dl_type: 0x800
               allow: 1

           - rule:
               dl_type: 0x0806
               allow: 1

           - rule:
               nw_dst: "10.0.0.0/16"
               dl_type: 0x800
               allow: 0

And just a reminder, this will work so that the name used for the key will be the default value for the numeric id of the object (if it has one) and as the name of the object. Either of these can be rewritten, but I think we should try to deprecate that usage.

I add a file called parser.py this handles parsing configs for faucet and gauge. It reads in yaml files, builds configs and calls the factories and return fully built valve/gauge objects.

I added a file which I am currently calling poller.py (I would like a better name for this since it refers to both pollers and event loggers---actually come to think of it maybe logger is a better generic name). This has all the code for the loggers and pollers and the main gauge class sits in its own thread.

So I guess the most important question I have is about how we would like routers to be configured going forward. I feel like these should not be tied to vlans, since you might want to route between vlans, but perhaps the ryu bgp design means that makes sense?

And does anyone have any issues with this?