napalm-automation / napalm-yang Goto Github PK

Bindings for napalm based on YANG models

License: Apache License 2.0

Makefile 0.01% Python 99.50% Jupyter Notebook 0.14% HTML 0.35% Roff 0.01%

napalm-yang's Introduction

napalm-yang

Project archived

Unfortunately I don't have the cycles to keep maintaining this project so as of now it's officially abandoned. If someone wants to fork the project and maintain it don't hesitate to contact me to add a link to the fork and direct existing and new users to the new fork. However, I'd recommend checking the following alternative:

yangify

Yangify has a cleaner interface that is easier to maintain and happens to be orders of magnitude faster and also more memory efficient.

If you are an existing user of napalm-yang and are worried about this announcement and/or want help migrating to some other project don't hesitate to contact me.

Documentation

Visit RTD for the documentation.

napalm-yang's People

Contributors

Stargazers

Watchers

napalm-yang's Issues

Cannot access key when processing list named "property"

When attempting to process a list attribute named "property", you cannot use "property_key" to reference the key of the dictionary without the parser returning an error:

Parsing attribute: /components/component[name='TempSensor12/2']/properties/property[name='overheat-threshold']/state/name
Traceback (most recent call last):
  File "test.py", line 30, in <module>
    config.parse_state(native=[json.dumps(plat)], profile=['eos'])
  File "/home/bewing/napalm-yang/napalm_yang/base.py", line 251, in parse_state
    parser.parse()
  File "/home/bewing/napalm-yang/napalm_yang/parser.py", line 75, in parse
    self._parse(self._yang_name, self.model, self.mapping[self._yang_name])
  File "/home/bewing/napalm-yang/napalm_yang/parser.py", line 81, in _parse
    self._parse_container(attribute, model, mapping)
  File "/home/bewing/napalm-yang/napalm_yang/parser.py", line 104, in _parse_container
    self._parse(k, v, mapping[v._yang_name])
  File "/home/bewing/napalm-yang/napalm_yang/parser.py", line 83, in _parse
    self._parse_list(attribute, model, mapping)
  File "/home/bewing/napalm-yang/napalm_yang/parser.py", line 136, in _parse_list
    self._parse(key, obj, element_mapping)
  File "/home/bewing/napalm-yang/napalm_yang/parser.py", line 81, in _parse
    self._parse_container(attribute, model, mapping)
  File "/home/bewing/napalm-yang/napalm_yang/parser.py", line 104, in _parse_container
    self._parse(k, v, mapping[v._yang_name])
  File "/home/bewing/napalm-yang/napalm_yang/parser.py", line 81, in _parse
    self._parse_container(attribute, model, mapping)
  File "/home/bewing/napalm-yang/napalm_yang/parser.py", line 104, in _parse_container
    self._parse(k, v, mapping[v._yang_name])
  File "/home/bewing/napalm-yang/napalm_yang/parser.py", line 83, in _parse
    self._parse_list(attribute, model, mapping)
  File "/home/bewing/napalm-yang/napalm_yang/parser.py", line 136, in _parse_list
    self._parse(key, obj, element_mapping)
  File "/home/bewing/napalm-yang/napalm_yang/parser.py", line 81, in _parse
    self._parse_container(attribute, model, mapping)
  File "/home/bewing/napalm-yang/napalm_yang/parser.py", line 104, in _parse_container
    self._parse(k, v, mapping[v._yang_name])
  File "/home/bewing/napalm-yang/napalm_yang/parser.py", line 81, in _parse
    self._parse_container(attribute, model, mapping)
  File "/home/bewing/napalm-yang/napalm_yang/parser.py", line 104, in _parse_container
    self._parse(k, v, mapping[v._yang_name])
  File "/home/bewing/napalm-yang/napalm_yang/parser.py", line 85, in _parse
    self._parse_leaf(attribute, model, mapping)
  File "/home/bewing/napalm-yang/napalm_yang/parser.py", line 145, in _parse_leaf
    self.extra_vars, None, self.bookmarks)
  File "/home/bewing/napalm-yang/napalm_yang/helpers.py", line 97, in resolve_rule
    rule[k] = template(v, **kwargs)
  File "/home/bewing/napalm-yang/napalm_yang/helpers.py", line 124, in template
    return template.render(**kwargs)
  File "/home/bewing/wat/lib/python2.7/site-packages/jinja2/environment.py", line 1008, in render
    return self.environment.handle_exception(exc_info, True)
  File "/home/bewing/wat/lib/python2.7/site-packages/jinja2/environment.py", line 780, in handle_exception
    reraise(exc_type, exc_value, tb)
  File "<template>", line 1, in top-level template code
jinja2.exceptions.UndefinedError: 'property_key' is undefined

This can be worked around by using "parent_key", if "property" is the parent.

Implement `is_alive` method

See details in the parent issue napalm-automation/napalm-base#137

Create tool to dynamically build dummy parsers/translators

Pin napalm-base driver

See details in the parent issue napalm-automation/napalm-base#107

Filtering capabilities for the getters

When working with heavy outputs, it proves very important when the filtering means retrieving from the device only what's needed not the complete bulk, thus much faster.

Parsing fab, reth and st interfaces

It seems like for juniper srx's the fab, reth and st interfaces are not supports to parse the config.

No handlers could be found for logger "napalm-yang"
Traceback (most recent call last):
  File "test.py", line 11, in <module>
    running_config.parse_config(native=[config], profile=["junos"])
  File "/Users/jemershaw/workspace/devices/tmp/lib/python2.7/site-packages/napalm_yang/base.py", line 225, in parse_config
    parser.parse()
  File "/Users/jemershaw/workspace/devices/tmp/lib/python2.7/site-packages/napalm_yang/parser.py", line 85, in parse
    self._parse(self._yang_name, self.model, self.mapping[self._yang_name])
  File "/Users/jemershaw/workspace/devices/tmp/lib/python2.7/site-packages/napalm_yang/parser.py", line 91, in _parse
    self._parse_container(attribute, model, mapping)
  File "/Users/jemershaw/workspace/devices/tmp/lib/python2.7/site-packages/napalm_yang/parser.py", line 132, in _parse_container
    self._parse(k, v, mapping[v._yang_name])
  File "/Users/jemershaw/workspace/devices/tmp/lib/python2.7/site-packages/napalm_yang/parser.py", line 93, in _parse
    self._parse_list(attribute, model, mapping)
  File "/Users/jemershaw/workspace/devices/tmp/lib/python2.7/site-packages/napalm_yang/parser.py", line 178, in _parse_list
    self._parse(key, obj, element_mapping)
  File "/Users/jemershaw/workspace/devices/tmp/lib/python2.7/site-packages/napalm_yang/parser.py", line 91, in _parse
    self._parse_container(attribute, model, mapping)
  File "/Users/jemershaw/workspace/devices/tmp/lib/python2.7/site-packages/napalm_yang/parser.py", line 132, in _parse_container
    self._parse(k, v, mapping[v._yang_name])
  File "/Users/jemershaw/workspace/devices/tmp/lib/python2.7/site-packages/napalm_yang/parser.py", line 91, in _parse
    self._parse_container(attribute, model, mapping)
  File "/Users/jemershaw/workspace/devices/tmp/lib/python2.7/site-packages/napalm_yang/parser.py", line 132, in _parse_container
    self._parse(k, v, mapping[v._yang_name])
  File "/Users/jemershaw/workspace/devices/tmp/lib/python2.7/site-packages/napalm_yang/parser.py", line 95, in _parse
    self._parse_leaf(attribute, model, mapping)
  File "/Users/jemershaw/workspace/devices/tmp/lib/python2.7/site-packages/napalm_yang/parser.py", line 194, in _parse_leaf
    value = self.parser.parse_leaf(mapping["_process"], self.bookmarks)
  File "/Users/jemershaw/workspace/devices/tmp/lib/python2.7/site-packages/napalm_yang/parsers/base.py", line 99, in parse_leaf
    result = getattr(self, method_name)(m, data)
  File "/Users/jemershaw/workspace/devices/tmp/lib/python2.7/site-packages/napalm_yang/parsers/jsonp.py", line 116, in _parse_leaf_map
    return mapping['map'][v.lower()]
KeyError: u'fab'

Me = stuck

I am running into an issue with the compliance report, I bet it's me, but I am not able to identify what I am doing wrong.

Following the tutorial: https://github.com/napalm-automation/napalm-yang/blob/develop/interactive_demo/tutorial.ipynb, under the "Compliance Report" paragraph.

I paste this in my Python console:

>>> data = {
...     "interfaces": {
...         "interface":{
...             "Et1": {
...                 "config": {
...                     "mtu": 9000
...                 },
...             },
...             "Et2": {
...                 "config": {
...                     "mtu": 1500
...                 }
...             }
...         }
...     }
... }
>>> # We load a dict for convenience, any source will do
... config = napalm_yang.base.Root()
>>> config.add_model(napalm_yang.models.openconfig_interfaces())
>>> config.load_dict(data)
>>> report = config.compliance_report("/home/admin/validate.yml")
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/usr/local/lib/python2.7/dist-packages/napalm_yang/base.py", line 289, in compliance_report
    return validate.compliance_report(self, validation_file=validation_file)
  File "/usr/local/lib/python2.7/dist-packages/napalm_base/validate.py", line 158, in compliance_report
    actual_results = getattr(cls, getter)(**kwargs)
  File "/usr/local/lib/python2.7/dist-packages/napalm_yang/base.py", line 181, in to_dict
    r = _to_dict(v, filter)
  File "/usr/local/lib/python2.7/dist-packages/napalm_yang/base.py", line 319, in _to_dict
    result = _to_dict_container(element, filter)
  File "/usr/local/lib/python2.7/dist-packages/napalm_yang/base.py", line 333, in _to_dict_container
    r = _to_dict(v, filter)
  File "/usr/local/lib/python2.7/dist-packages/napalm_yang/base.py", line 321, in _to_dict
    result = _to_dict_list(element, filter)
  File "/usr/local/lib/python2.7/dist-packages/napalm_yang/base.py", line 342, in _to_dict_list
    for k, v in element.items():
AttributeError: 'YANGBaseClass' object has no attribute 'items'
>>>
>>> with open('/home/admin/validate.yml') as fopen:
...     print(fopen.read())
...
---
- to_dict:
    _kwargs:
        filter: true
    interfaces:
        interface:
            Et1:
                config:
                    mtu: 9000
            Et2:
                config:
                    mtu: 9000
            _mode: strict

>>>

I am running napalm-yang 0.0.6. Didin't want to bother @dbarrosop, as this isn't an emergency & perhaps I'm not the only one stuck at this point.
Any feedback is much appreciated, thanks!

Issue parsing ietf-pim

Ran into issues attempting to build models from the draft IETF PIM yang models. napalm.py is not parsing several keywords correctly:

presence
choice

parse_config extension question

Hi !
I'm still studying openconfig combined with ansible and napalm. I fully support the conclusion of this article as to use napalm as a transition mechanism (waiting for the vendor to support the OC models completely ). I tried various example and wrote based on the network-instance model the following example of a router with ISIS enabled:

result I would like to achieve when getting config from a Junos device
{
    "network-instances": {
        "network-instance": {
            "global": {
                "name": "global", 
                "protocols": {
                    "protocol": {
                        "isis 2200": {
                            "identifier": "isis", 
                            "isis": {
                                "interfaces": {
                                    "interface": {
                                        "ge-0/0/3.0": {
                                            "interface-id": "ge-0/0/3.0", 
                                            "config": {
                                                "circuit-type": "POINT_TO_POINT"
                                            }
                                        }, 
                                        "ge-0/0/2.0": {
                                            "interface-id": "ge-0/0/2.0", 
                                            "config": {
                                                "circuit-type": "POINT_TO_POINT"
                                            }
                                        }, 
                                        "ge-0/0/1.0": {
                                            "interface-id": "ge-0/0/1.0", 
                                            "config": {
                                                "circuit-type": "POINT_TO_POINT"
                                            }
                                        }
                                    }
                                }, 
                                "global": {
                                    "config": {
                                        "net": [
                                            "49.4225.0000.0000.5555.00"
                                        ]
                                    }
                                }, 
                                "levels": {
                                    "level": {
                                        "1": {
                                            "level-number": "1", 
                                            "config": {
                                                "metric-style": "WIDE_METRIC", 
                                                "enabled": true
                                            }
                                        }
                                    }
                               }
                            }, 
                            "name": "2200"
                        }
                    }
                }
            }
        }
    }

I used napalm-yang to retrieve ISIS config with model "openconfig_network_instance" from devices:

...
model.add_model(napalm_yang.models.openconfig_network_instance)
...

And got the following result with (model.translate_config(profile=['junos']):

{
    "network_instances": {
        "network-instance": {
            "global": {
                "name": "global", 
                "config": {
                    "type": "DEFAULT_INSTANCE", 
                    "enabled": True
                }, 
                "protocols": {
                    "protocol": {
                        "isis isis": {
                            "name": "isis", 
                            "identifier": "isis"
                        } 
                    }
                }
            } 
        }
    }
}

while the config on the the Junos vmx is:

mgmt@gn4-net-core-5> show configuration protocols isis 
level 1 wide-metrics-only;
level 2 wide-metrics-only;
interface ge-0/0/0.0 {
    point-to-point;
}
interface ge-0/0/1.0 {
    point-to-point;
}
interface ge-0/0/2.0 {
    point-to-point;
}
interface ge-0/0/3.0 {
    point-to-point;
}
interface lo0.0;

interfaces {
    lo0 {
            family iso {
                address 49.4225.0000.0000.5555.00;
            }
        }
    }
}

protocols {
    isis {
        level 1 wide-metrics-only;
         interface ge-0/0/0.0 {
            point-to-point;
        }
        interface ge-0/0/1.0 {
            point-to-point;
        }
        interface ge-0/0/2.0 {
            point-to-point;
        }
        interface ge-0/0/3.0 {
            point-to-point;
        }
        interface lo0.0;
    }
}

Clearly parse_config needs some extensions to be implemented. So the question is:
Is there a documented/recommended process that I can follow so that Junos -> OC and OC -> Junos for parse_config can take into account the "full OC model" ?

(Sorry if it is a stupid/silly question that has been previously answered, in that case I'd love to have some pointers :-) )

Thanks for your great job !
All the best,

Frederic

Add tests for nxos parsing

requirements.txt format breaks my conda version

The format pyangbind>=0.8.0<1.0.0 seems to cause a parsing error, I believe pyangbind>=0.8.0,<1.0.0 is expected. I've tested locally and this seems to fix my issue, PR inbound :)

Unify config/state parsers?

Unify config/state parsers? Just the YAML files, keep get_ separated.

Get napalm-yang out put in ietf format.

I am facing one issue while validating yang output from napalm yang against openconfig yang model for interfaces in ODL.

java.lang.IllegalStateException: Schema node with name ge-0/0/7 wasn't found under (http://openconfig.net/yang/interfaces?revision)interface.
  at org.opendaylight.yangtools.yang.data.codec.gson.JsonParserStream.resolveNamespace(JsonParserStream.java:318)
  at org.opendaylight.yangtools.yang.data.codec.gson.JsonParserStream.read(JsonParserStream.java:219)
  at org.opendaylight.yangtools.yang.data.codec.gson.JsonParserStream.read(JsonParserStream.java:238)
  at org.opendaylight.yangtools.yang.data.codec.gson.JsonParserStream.read(JsonParserStream.java:238)
  at org.opendaylight.yangtools.yang.data.codec.gson.JsonParserStream.parse(JsonParserStream.java:100)
  at com.luminanetworks.lsc.ext.jsonrpc.impl.JsonRPCTx.read(JsonRPCTx.java:186)
  at org.opendaylight.netconf.sal.restconf.impl.BrokerFacade.readDataViaTransaction(BrokerFacade.java:536)
  at org.opendaylight.netconf.sal.restconf.impl.BrokerFacade.readConfigurationData(BrokerFacade.java:172)
  at org.opendaylight.netconf.sal.restconf.impl.RestconfImpl.readConfigurationData(RestconfImpl.java:715)
  at org.opendaylight.netconf.sal.restconf.impl.StatisticsRestconfServiceWrapper.readConfigurationData(StatisticsRestconfServiceWrapper.java:97)
  at org.opendaylight.netconf.sal.rest.impl.RestconfCompositeWrapper.readConfigurationData(RestconfCompos



How can I convert this or get output in standard Json RPC format or IETF format?

parse_conf problem with junos profile related to vlan-id

Hi following napalm-ansible issue #110 discussion. (This is definitely a Junos problem.)

Thanks !
Frederic

 <configuration junos:commit-seconds="1513679950" junos:commit-localtime="2017-12-19 10:39:10 UTC" junos:commit-user="mgmt">
            <interfaces>
                <interface>
                    <name>ge-0/0/0</name>
                    <unit>
                        <name>0</name>
                        <description>R1@ge-0/0/0.0-&gt;R2-ge-0/0/0.0</description>
                        <family>
                            <inet>
                                <address>
                                    <name>172.16.12.1/24</name>
                                </address>
                            </inet>
                            <iso>
                            </iso>
                            <mpls>
                            </mpls>
                        </family>
                    </unit>
                </interface>
                <interface>
                    <name>ge-0/0/1</name>
                    <unit>
                        <name>0</name>
                        <description>R1@ge-0/0/1.0-&gt;R3@ge-0/0/0.0</description>
                        <family>
                            <inet>
                                <address>
                                    <name>172.16.13.1/24</name>
                                </address>
                            </inet>
                            <iso>
                            </iso>
                            <mpls>
                            </mpls>
                        </family>
                    </unit>
                </interface>
                <interface>
                    <name>ge-0/0/2</name>
                    <unit>
                        <name>0</name>
                        <description>R1@ge-0/0/2.0-&gt;R5@ge-0/0/0.0</description>
                        <family>
                            <inet>
                                <address>
                                    <name>172.16.15.1/24</name>
                                </address>
                            </inet>
                            <iso>
                            </iso>
                            <mpls>
                            </mpls>
                        </family>
                    </unit>
                </interface>
                <interface>
                    <name>ge-0/0/3</name>
                    <vlan-tagging/>
                    <unit>
                        <name>0</name>
                        <vlan-id>0</vlan-id>
                        <family>
                            <inet>
                                <address>
                                    <name>172.16.16.1/24</name>
                                </address>
                            </inet>
                        </family>
                    </unit>
                    <unit>
                        <name>6</name>
                        <vlan-id>6</vlan-id>
                        <family>
                            <inet>
                                <address>
                                    <name>192.168.16.1/24</name>
                                </address>
                            </inet>
                        </family>
                    </unit>
                </interface>
                <interface>
                    <name>fxp0</name>
                    <unit>
                        <name>0</name>
                        <family>
                            <inet>
                                <address>
                                    <name>10.134.250.31/9</name>
                                </address>
                            </inet>
                        </family>
                    </unit>
                </interface>
                <interface>
                    <name>lo0</name>
                    <unit>
                        <name>0</name>
                        <family>
                            <inet>
                                <address>
                                    <name>172.16.1.1/32</name>
                                </address>
                            </inet>
                            <iso>
                                <address>
                                    <name>49.4225.0000.0000.1111.00</name>
                                </address>
                            </iso>
                        </family>
                    </unit>
                    <unit>
                        <name>4225</name>
                        <family>
                            <inet>
                                <filter>
                                    <input>
                                        <filter-name>FWF-ACCEPT-ALL</filter-name>
                                    </input>
                                </filter>
                                <address>
                                    <name>1.1.1.1/32</name>
                                </address>
                            </inet>
                        </family>
                    </unit>
                </interface>
            </interfaces>
    </configuration>

next-hop not being parsed in BGP Junos

Hello, I think I found a problem with next-hop on Junos. I want to announce a static route on BGP, so I configure it inside the routing-options section, here is how it looks:

routing-options {                       
    static {                            
        route 192.168.4.0/24 next-hop 192.168.4.0;
    }                                   
    autonomous-system 200;              
}

The idea is to populate a network-instance model from the device, change the config and then merge the new configuration back to the device. When doing this I get this error.

Traceback (most recent call last):
  File "openConfigEx.py", line 79, in <module>
    d.load_merge_candidate(config=config2)
  File "/usr/local/lib/python2.7/dist-packages/napalm/junos/junos.py", line 227, in load_merge_candidate
    self._load_candidate(filename, config, False)
  File "/usr/local/lib/python2.7/dist-packages/napalm/junos/junos.py", line 217, in _load_candidate
    raise MergeConfigException(e.errs)
napalm.base.exceptions.MergeConfigException: [{'source': None, 'message': 'mgd: statement has no contents; ignored', 'bad_element': 'route 192.168.4.0/24', 'severity': 'warning', 'edit_path': '[edit routing-options static]'}]

I started getting this error when adding the static route in routing-options. I noticed as well that the configuration parsed from the device does not contain the next-hop information, this can be seen bellow in the cofig parsed from the device as well as in the translation of this config to Junos native configuration.

I'm working with a Junos 12.1 VM and napalm-yang version 0.1.0.

Device configuration

## Last commit: 2018-06-21 10:23:11 UTC by root
version 12.1X47-D15.4;
system {
    host-name r2;
    root-authentication {
        encrypted-password "$1$nq.N1UsY$JxA/ESAj3KuXseXE597gg0"; ## SECRET-DATA
        ssh-rsa "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCzFChCRM5cviOavo6Bta1gUHjdWraB9ZLWAwPhkSr9lkj8qqfLpvxch7ADsL5AUO1cox6Se9oyWRZQutEbZTW+pYPnR7W05LzbVpDURfc85BuYOHWVdWUjiB764fslEUvH+0NkeAWO/Hag6AKPYluATGM3vkRZLjPxNLtR4D0M5rfN3EmvBq1EhY8OKhBY+ELxmny0UW+hj69IyVEwD5WU/eo5OYWr2JbEBJ2TA7T4YQ8bQ0ZaVu1G5CGIR/NH8TOBh/4BhNWhj4/knSgyU+nupRvgjSf8dFspSN68udhCxByJerXFb2suwJuVsb9TN6yrAsUYqoDjphwjnQPC8Erb vagrant"; ## SECRET-DATA
    }
    login {
        user vagrant {
            uid 2000;
            class super-user;
            authentication {
                ssh-rsa "ssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEA6NF8iallvQVp22WDkTkyrtvp9eWW6A8YVr+kz4TjGYe7gHzIw+niNltGEFHzD8+v1I2YJ6oXevct1YeS0o9HZyN1Q9qgCgzUFtdOKLv6IedplqoPkcmF0aYet2PkEDo3MlTBckFXPITAMzF8dJSIFo9D8HfdOV0IAdx4O7PtixWKn5y2hMNG0zQPyUecp4pzC6kivAIhyfHilFR61RGL+GPXQ2MWZWFYbAGjyiYJnAmCP3NOTd0jMZEnDkbUvxhMmBYSdETk1rRgm+R4LOzFUGaHqHDLKLX+FIPKcF96hrucXzcWyLbIbEgE98OHlnVYCzRdK8jlqm8tehUc9c9WhQ== vagrant insecure public key"; ## SECRET-DATA
            }                           
        }                               
    }                                   
    services {                          
        ssh {                           
            root-login allow;           
        }                               
        netconf {                       
            ssh;                        
        }                               
        web-management {                
            http {                      
                interface ge-0/0/0.0;   
            }                           
        }                               
    }                                   
    syslog {                            
        user * {                        
            any emergency;              
        }                               
        file messages {                 
            any any;                    
            authorization info;         
        }                               
        file interactive-commands {     
            interactive-commands any;   
        }                               
    }                                   
    license {                           
        autoupdate {                    
            url https://ae1.juniper.net/junos/key_retrieval;
        }                               
    }                                   
}                                       
interfaces {                            
    ge-0/0/0 {                          
        unit 0 {                        
            family inet {               
                dhcp;                   
            }                           
        }                               
    }                                   
    ge-0/0/1 {                          
        unit 0 {                        
            family inet {               
                address 192.168.1.2/24; 
            }                           
        }                               
    }                                   
    ge-0/0/2 {                          
        unit 0 {                        
            family inet {               
                address 192.168.4.1/24; 
            }                           
        }                               
    }                                   
    lo0 {                               
        unit 0 {                        
            family inet {               
                address 2.2.2.2/32;     
            }                           
        }                               
    }                                   
}                                       
routing-options {                       
    static {                            
        route 192.168.4.0/24 next-hop 192.168.4.0;
    }                                   
    autonomous-system 200;              
}                                       
protocols {                             
    bgp {                               
        group external-peers {          
            type external;              
            export send-direct;         
            neighbor 192.168.1.1 {      
                peer-as 100;            
            }                           
        }                               
    }                                   
}                                       
policy-options {                        
    policy-statement send-direct {      
        term accept {                   
            from protocol [ static bgp direct ];
            then accept;                
        }                               
        term reject {                   
            then reject;                
        }                               
    }                                   
}                                       
security {                              
    forwarding-options {                
        family {                        
            inet6 {                     
                mode packet-based;      
            }                           
            mpls {                      
                mode packet-based;      
            }                           
        }                               
    }                                   
}

Configuration parsed

{
    "network_instances": {
        "network-instance": {
            "global": {
                "config": {
                    "enabled": True, 
                    "type": "DEFAULT_INSTANCE"
                }, 
                "name": "global", 
                "protocols": {
                    "protocol": {
                        "bgp bgp": {
                            "bgp": {
                                "global": {
                                    "config": {
                                        "as": 200
                                    }
                                }, 
                                "neighbors": {
                                    "neighbor": {
                                        "192.168.1.1": {
                                            "config": {
                                                "neighbor-address": "192.168.1.1", 
                                                "peer-as": 100, 
                                                "peer-group": "external-peers"
                                            }, 
                                            "neighbor-address": "192.168.1.1"
                                        }
                                    }
                                }
                            }, 
                            "identifier": "bgp", 
                            "name": "bgp"
                        }, 
                        "static static": {
                            "identifier": "static", 
                            "name": "static", 
                            "static-routes": {
                                "static": {
                                    "192.168.4.0/24": {
                                        "config": {
                                            "prefix": "192.168.4.0/24"
                                        }, 
                                        "prefix": "192.168.4.0/24"
                                    }
                                }
                            }
                        }
                    }
                }
            }
        }
    }
}

Configuration translation to Junos

<configuration>
  <protocols>
    <bgp>
      <group>
        <name>external-peers</name>
        <neighbor>
          <name>192.168.1.1</name>
          <peer-as>100</peer-as>
        </neighbor>
      </group>
      <group>
        <name>external-peers</name>
        <neighbor>
          <name>10.100.100.4</name>
          <peer-as>100</peer-as>
        </neighbor>
      </group>
    </bgp>
  </protocols>
  <routing-options>
    <autonomous-system>
      <as-number>200</as-number>
    </autonomous-system>
  </routing-options>
  <routing-options>
    <static>
      <route>
        <name>192.168.4.0/24</name>
      </route>
    </static>
  </routing-options>
</configuration>

My code

from napalm_base import get_network_driver
import napalm_yang, salt.client, ast

import json

def use_real_devices():
    junos_configuration = {
        'hostname': '127.0.0.1',
        'username': 'root',
        'password': 'Juniper',
        'optional_args': {'port': 2232}
    }

    eos_configuration = {
        'hostname': '127.0.0.1',
        'username': 'vagrant',
        'password': 'vagrant',
        'optional_args': {'port': 12443}
    }

    junos = get_network_driver("junos")
    junos_device = junos(**junos_configuration)

    eos = get_network_driver("eos")
    eos_device = eos(**eos_configuration)
    return junos_device, eos_device

def pretty_print(dictionary):
    print(json.dumps(dictionary, sort_keys=True, indent=4))

junos_device, eos_device = use_real_devices()

#__________________________________________________________________________________

with junos_device as d:
    candidate = napalm_yang.base.Root()
    candidate.add_model(napalm_yang.models.openconfig_network_instance)
    candidate.parse_config(device=d)
    pretty_print(candidate.get(filter=True))

    neighbor = candidate.network_instances.network_instance["global"].protocols.protocol["bgp bgp"].bgp.neighbors.neighbor.add("10.100.100.4")

    neighbor.config.neighbor_address = "10.100.100.4"
    neighbor.config.peer_as = 100
    neighbor.config.peer_group = "external-peers"
    pretty_print(candidate.get(filter=True))
    
    running = napalm_yang.base.Root()
    running.add_model(napalm_yang.models.openconfig_network_instance)
    running.parse_config(device=d)

    config2 = candidate.translate_config(profile=junos_device.profile, merge=running)
    print config2
    d.load_merge_candidate(config=config2)
    print(d.compare_config())
    d.discard_config()

Parsing interface config and state into the same model doesn't work.

If I try to parse the interface config of a device and then parse the interface state of the same device into the same model, I get a traceback that essentially says the interface is already defined in the model.

In [14]: interface_oc_model.parse_config(device=device)
In [15]: interface_oc_model.parse_state(device=device)
/mnt/tim/venvs/napalm/lib/python2.7/site-packages/napalm_yang/base.pyc in parse_state(self, device, profile, native, attrs)
    258         for v in attrs:
    259             parser = Parser(v, device=device, profile=profile, native=native, is_config=False)
--> 260             parser.parse()
    261
    262     def translate_config(self, profile, merge=None, replace=None):

/mnt/home/tim/venvs/napalm/lib/python2.7/site-packages/napalm_yang/parser.py in parse(self)
     85         if "parent" not in self.bookmarks:
     86             self.bookmarks["parent".format(self._yang_name)] = self.native
---> 87         self._parse(self._yang_name, self.model, self.mapping[self._yang_name])
     88
     89     def _parse(self, attribute, model, mapping):

/mnt/home/tim/venvs/napalm/lib/python2.7/site-packages/napalm_yang/parser.py in _parse(self, attribute, model, mapping)
     91
     92         if model._is_container in ("container", ):
---> 93             self._parse_container(attribute, model, mapping)
     94         elif model._yang_type in ("list", ):
     95             self._parse_list(attribute, model, mapping)

/mnt/home/tim/venvs/napalm/lib/python2.7/site-packages/napalm_yang/parser.py in _parse_container(self, attribute, model, mapping)
    129                 parser.parse()
    130             else:
--> 131                 self._parse(k, v, mapping[v._yang_name])
    132
    133         # Restoring state

/mnt/home/tim/venvs/napalm/lib/python2.7/site-packages/napalm_yang/parser.py in _parse(self, attribute, model, mapping)
     93             self._parse_container(attribute, model, mapping)
     94         elif model._yang_type in ("list", ):
---> 95             self._parse_list(attribute, model, mapping)
     96         else:
     97             self._parse_leaf(attribute, model, mapping)

/mnt/home/tim/venvs/napalm/lib/python2.7/site-packages/napalm_yang/parser.py in _parse_list(self, attribute, model, mapping)
    151
    152             try:
--> 153                 obj = model.add(key)
    154             except KeyError as e:
    155                 if "is already defined as a list entry" in e.message and \

/mnt/home/tim/venvs/napalm/lib/python2.7/site-packages/pyangbind/lib/yangtypes.pyc in add(self, *args, **kwargs)
    747
    748       if k in self._members:
--> 749         raise KeyError("%s is already defined as a list entry" % k)
    750       if self._keyval and keyargs is None:
    751         if k is None:

KeyError: u'Port-Channel8 is already defined as a list entry'

Is this expected?

KeyError: 'path-setup-protocol' when translating network_instances

Summary:

I think network-instances.yaml is missing ´path-setup-protocol´ branch under /network-instances/network-instance[name='default']/mpls/lsps/unconstrained-path.

Details:

I'm trying to write an IOS translaor for static routes. I copied dir openconfig-network-instance translator from dummy to ios, and filled just the metadata section for now:

---
metadata:
    processor: TextTranslator
    root: true

And I populate just the most basic config object with a single network instance. This is the code:

import logging
logging.basicConfig()
logging.getLogger().setLevel(logging.DEBUG)

from napalm_base import get_network_driver
import napalm_yang
from napalm_yang.models import openconfig_network_instance

config = napalm_yang.base.Root()
config.add_model(openconfig_network_instance())

ni = config.network_instances.network_instance.add('default')
ni.config.type='DEFAULT_INSTANCE'

config.translate_config(profile=['ios'])

The last line raises a KeyError:

INFO:napalm-yang:Finding parser for openconfig-network-instance:network-instances
DEBUG:napalm-yang:Found on profile: ios, napalm_yang/mappings/ios/translators/openconfig-network-instance/network-instances.yaml
DEBUG:napalm-yang:Translating attribute: /network-instances
DEBUG:napalm-yang:Parsing attribute: /network-instances/network-instance
DEBUG:napalm-yang:Translating attribute: /network-instances/network-instance
DEBUG:napalm-yang:Translating network-instance default
DEBUG:napalm-yang:Translating attribute: /network-instances/network-instance[name='default']
DEBUG:napalm-yang:Parsing attribute: /network-instances/network-instance[name='default']/tables
DEBUG:napalm-yang:Translating attribute: /network-instances/network-instance[name='default']/tables
DEBUG:napalm-yang:Parsing attribute: /network-instances/network-instance[name='default']/tables/table
DEBUG:napalm-yang:Translating attribute: /network-instances/network-instance[name='default']/tables/table
DEBUG:napalm-yang:Parsing attribute: /network-instances/network-instance[name='default']/fdb
DEBUG:napalm-yang:Translating attribute: /network-instances/network-instance[name='default']/fdb
DEBUG:napalm-yang:Parsing attribute: /network-instances/network-instance[name='default']/fdb/mac-table
DEBUG:napalm-yang:Translating attribute: /network-instances/network-instance[name='default']/fdb/mac-table
DEBUG:napalm-yang:Parsing attribute: /network-instances/network-instance[name='default']/fdb/mac-table/entries
DEBUG:napalm-yang:Translating attribute: /network-instances/network-instance[name='default']/fdb/mac-table/entries
DEBUG:napalm-yang:Parsing attribute: /network-instances/network-instance[name='default']/fdb/mac-table/entries/entry
DEBUG:napalm-yang:Translating attribute: /network-instances/network-instance[name='default']/fdb/mac-table/entries/entry
DEBUG:napalm-yang:Parsing attribute: /network-instances/network-instance[name='default']/fdb/state
DEBUG:napalm-yang:Skipping attribute: openconfig-network-instance:fdb
DEBUG:napalm-yang:Parsing attribute: /network-instances/network-instance[name='default']/fdb/config
DEBUG:napalm-yang:Translating attribute: /network-instances/network-instance[name='default']/fdb/config
DEBUG:napalm-yang:Parsing attribute: /network-instances/network-instance[name='default']/fdb/config/mac-aging-time
DEBUG:napalm-yang:Translating attribute: /network-instances/network-instance[name='default']/fdb/config/mac-aging-time
DEBUG:napalm-yang:Parsing attribute: /network-instances/network-instance[name='default']/fdb/config/maximum-entries
DEBUG:napalm-yang:Translating attribute: /network-instances/network-instance[name='default']/fdb/config/maximum-entries
DEBUG:napalm-yang:Parsing attribute: /network-instances/network-instance[name='default']/fdb/config/mac-learning
DEBUG:napalm-yang:Translating attribute: /network-instances/network-instance[name='default']/fdb/config/mac-learning
DEBUG:napalm-yang:Parsing attribute: /network-instances/network-instance[name='default']/name
DEBUG:napalm-yang:Translating attribute: /network-instances/network-instance[name='default']/name
DEBUG:napalm-yang:Parsing attribute: /network-instances/network-instance[name='default']/afts
DEBUG:napalm-yang:Translating attribute: /network-instances/network-instance[name='default']/afts
DEBUG:napalm-yang:Parsing attribute: /network-instances/network-instance[name='default']/afts/aft
DEBUG:napalm-yang:Translating attribute: /network-instances/network-instance[name='default']/afts/aft
DEBUG:napalm-yang:Parsing attribute: /network-instances/network-instance[name='default']/interfaces
DEBUG:napalm-yang:Translating attribute: /network-instances/network-instance[name='default']/interfaces
DEBUG:napalm-yang:Parsing attribute: /network-instances/network-instance[name='default']/interfaces/interface
DEBUG:napalm-yang:Translating attribute: /network-instances/network-instance[name='default']/interfaces/interface
DEBUG:napalm-yang:Parsing attribute: /network-instances/network-instance[name='default']/mpls
DEBUG:napalm-yang:Translating attribute: /network-instances/network-instance[name='default']/mpls
DEBUG:napalm-yang:Parsing attribute: /network-instances/network-instance[name='default']/mpls/lsps
DEBUG:napalm-yang:Translating attribute: /network-instances/network-instance[name='default']/mpls/lsps
DEBUG:napalm-yang:Parsing attribute: /network-instances/network-instance[name='default']/mpls/lsps/constrained-path
DEBUG:napalm-yang:Translating attribute: /network-instances/network-instance[name='default']/mpls/lsps/constrained-path
DEBUG:napalm-yang:Parsing attribute: /network-instances/network-instance[name='default']/mpls/lsps/constrained-path/named-explicit-paths
DEBUG:napalm-yang:Translating attribute: /network-instances/network-instance[name='default']/mpls/lsps/constrained-path/named-explicit-paths
DEBUG:napalm-yang:Parsing attribute: /network-instances/network-instance[name='default']/mpls/lsps/constrained-path/named-explicit-paths/named-explicit-path
DEBUG:napalm-yang:Translating attribute: /network-instances/network-instance[name='default']/mpls/lsps/constrained-path/named-explicit-paths/named-explicit-path
DEBUG:napalm-yang:Parsing attribute: /network-instances/network-instance[name='default']/mpls/lsps/constrained-path/tunnels
DEBUG:napalm-yang:Translating attribute: /network-instances/network-instance[name='default']/mpls/lsps/constrained-path/tunnels
DEBUG:napalm-yang:Parsing attribute: /network-instances/network-instance[name='default']/mpls/lsps/constrained-path/tunnels/tunnel
DEBUG:napalm-yang:Translating attribute: /network-instances/network-instance[name='default']/mpls/lsps/constrained-path/tunnels/tunnel
DEBUG:napalm-yang:Parsing attribute: /network-instances/network-instance[name='default']/mpls/lsps/unconstrained-path
DEBUG:napalm-yang:Translating attribute: /network-instances/network-instance[name='default']/mpls/lsps/unconstrained-path
DEBUG:napalm-yang:Parsing attribute: /network-instances/network-instance[name='default']/mpls/lsps/unconstrained-path/path-setup-protocol
---------------------------------------------------------------------------
KeyError                                  Traceback (most recent call last)
<ipython-input-9-22cfd64f337a> in <module>()
----> 1 config.translate_config(profile=['ios'])

/home/matej/Projects/napalmyang/napalm-yang/napalm_yang/base.pyc in translate_config(self, profile, merge, replace)
    277             other_replace = getattr(replace, k) if replace else None
    278             translator = Translator(v, profile, merge=other_merge, replace=other_replace)
--> 279             result.append(translator.translate())
    280 
    281         return "\n".join(result)

/home/matej/Projects/napalmyang/napalm-yang/napalm_yang/translator.pyc in translate(self)
     51             return ""
     52         self._translate(self._yang_name, self.model, self.mapping[self._yang_name],
---> 53                         self.translation, self.other)
     54         return self.translator.post_processing(self)
     55 

/home/matej/Projects/napalmyang/napalm-yang/napalm_yang/translator.pyc in _translate(self, attribute, model, mapping, translation, other)                                                                                                                                 
     58 
     59         if model._is_container in ("container", ):
---> 60             self._translate_container(attribute, model, mapping, translation, other)
     61         elif model._yang_type in ("list", ):
     62             self._translate_list(attribute, model, mapping, translation, other)

/home/matej/Projects/napalmyang/napalm-yang/napalm_yang/translator.pyc in _translate_container(self, attribute, model, mapping, translation, other)                                                                                                                       
     92                 translator.translate()
     93             else:
---> 94                 self._translate(v._yang_name, v, mapping[v._yang_name], et, other_attr)
     95 
     96     def _translate_list(self, attribute, model, mapping, translation, other):

/home/matej/Projects/napalmyang/napalm-yang/napalm_yang/translator.pyc in _translate(self, attribute, model, mapping, translation, other)                                                                                                                                 
     60             self._translate_container(attribute, model, mapping, translation, other)
     61         elif model._yang_type in ("list", ):
---> 62             self._translate_list(attribute, model, mapping, translation, other)
     63         else:
     64             self._translate_leaf(attribute, model, mapping, translation, other)

/home/matej/Projects/napalmyang/napalm-yang/napalm_yang/translator.pyc in _translate_list(self, attribute, model, mapping, translation, other)                                                                                                                            
    132             self.bookmarks["parent"] = et
    133 
--> 134             self._translate(attribute, element, mapping, et, other_element)
    135 
    136         # Restore state

/home/matej/Projects/napalmyang/napalm-yang/napalm_yang/translator.pyc in _translate(self, attribute, model, mapping, translation, other)                                                                                                                                 
     58 
     59         if model._is_container in ("container", ):
---> 60             self._translate_container(attribute, model, mapping, translation, other)
     61         elif model._yang_type in ("list", ):
     62             self._translate_list(attribute, model, mapping, translation, other)

/home/matej/Projects/napalmyang/napalm-yang/napalm_yang/translator.pyc in _translate_container(self, attribute, model, mapping, translation, other)                                                                                                                       
     92                 translator.translate()
     93             else:
---> 94                 self._translate(v._yang_name, v, mapping[v._yang_name], et, other_attr)
     95 
     96     def _translate_list(self, attribute, model, mapping, translation, other):

/home/matej/Projects/napalmyang/napalm-yang/napalm_yang/translator.pyc in _translate(self, attribute, model, mapping, translation, other)                                                                                                                                 
     58 
     59         if model._is_container in ("container", ):
---> 60             self._translate_container(attribute, model, mapping, translation, other)
     61         elif model._yang_type in ("list", ):
     62             self._translate_list(attribute, model, mapping, translation, other)

/home/matej/Projects/napalmyang/napalm-yang/napalm_yang/translator.pyc in _translate_container(self, attribute, model, mapping, translation, other)                                                                                                                       
     92                 translator.translate()
     93             else:
---> 94                 self._translate(v._yang_name, v, mapping[v._yang_name], et, other_attr)
     95 
     96     def _translate_list(self, attribute, model, mapping, translation, other):

/home/matej/Projects/napalmyang/napalm-yang/napalm_yang/translator.pyc in _translate(self, attribute, model, mapping, translation, other)
     58 
     59         if model._is_container in ("container", ):
---> 60             self._translate_container(attribute, model, mapping, translation, other)
     61         elif model._yang_type in ("list", ):
     62             self._translate_list(attribute, model, mapping, translation, other)

/home/matej/Projects/napalmyang/napalm-yang/napalm_yang/translator.pyc in _translate_container(self, attribute, model, mapping, translation, other)
     92                 translator.translate()
     93             else:
---> 94                 self._translate(v._yang_name, v, mapping[v._yang_name], et, other_attr)
     95 
     96     def _translate_list(self, attribute, model, mapping, translation, other):

/home/matej/Projects/napalmyang/napalm-yang/napalm_yang/translator.pyc in _translate(self, attribute, model, mapping, translation, other)
     58 
     59         if model._is_container in ("container", ):
---> 60             self._translate_container(attribute, model, mapping, translation, other)
     61         elif model._yang_type in ("list", ):
     62             self._translate_list(attribute, model, mapping, translation, other)

/home/matej/Projects/napalmyang/napalm-yang/napalm_yang/translator.pyc in _translate_container(self, attribute, model, mapping, translation, other)
     92                 translator.translate()
     93             else:
---> 94                 self._translate(v._yang_name, v, mapping[v._yang_name], et, other_attr)
     95 
     96     def _translate_list(self, attribute, model, mapping, translation, other):

KeyError: 'path-setup-protocol'

Trunked Vlans not being translated with the to_dict function

I'm trying to use the to_dict function to translate parse config into a format I can save to a file or pass to ansible, but the to_dict function is not translating the contents of the trunk-vlans section. I'm using the latest napalm_yang develop branch.

I have the following data in openconfig_interfaces model

{
    "config": {
        "description": "to-test2",
        "enabled": True,
        "type": "ieee8023adLag"
    },
    "ethernet": {
        "switched-vlan": {
            "config": {
                "access-vlan": 1,
                "interface-mode": "TRUNK",
                "native-vlan": 990,
                "trunk-vlans": [
                    110,
                    901
                ]
            }
        }
    },
    "name": "Port-Channel1",
    "routed-vlan": {
        "ipv4": {
            "config": {
                "enabled": False
            }
        }
    }
}

But this is what I see if I try to print out that data with the to_dict function. Notice the trunk-vlans section shows pyangbind object instead of the list of trunked vlans.

In [18]: yang.to_dict(filter=True)['interfaces']['interface']['Port-Channel1']
Out[18]:
{'config': {'description': u'to-test2',
  'enabled': True,
  'type': u'ieee8023adLag'},
 'ethernet': {'switched_vlan': {'config': {'access_vlan': 1,
    'interface_mode': u'TRUNK',
    'native_vlan': 990,
    'trunk_vlans': '<pyangbind.lib.yangtypes.YANGBaseClass object at 0x7fbbef93f590>'}}},
 'name': u'Port-Channel1',
 'routed_vlan': {'ipv4': {'config': {'enabled': False}}}}

Now I have a work around for this but it is specific to trunk-vlans. What I've done is change the _to_dict_leaf function in napalm_yang.base to look like this and I get what I would expect and the resulting dict is now loadable.

def _to_dict_leaf(element, filter):
    value = None
    if element._changed() or not filter:
        if element._yang_name == "trunk-vlans":
            value = []
            for i in element._list:
                value.append(i)
        else:
            try:
                value = ast.literal_eval(element.__repr__())
            except Exception:
                value = element.__repr__()

    return value

I haven't worked with any other models so far that will have a list in it like trunk-vlans, so this will do for me for now. But I'm wondering what would be the best solution for other sections that are of the yang union type. Right now I'm only doing this for the trunk-vlans out of an abundance of caution. Maybe the right solution is to just drop that if statement, I'm not sure.

translate_config merge for containers with _process: unnecesary

Let's consider the EOS translations for static-routes in develop branch.

Also running config has a static route:

running = napalm_yang.base.Root()
running.add_model(napalm_yang.models.openconfig_network_instance)
running.load_dict({'network_instances': {'network_instance': {u'global': {'config': {'enabled': True,
     'type': u'DEFAULT_INSTANCE'},
    'name': u'global',
    'protocols': {'protocol': {u'static static': {'identifier': u'static',
       'name': u'static',
       'static_routes': {'static': {u'2.2.2.2/32': {'config': {'prefix': u'2.2.2.2/32'},
          'next_hops': {'next_hop': {u'10.1.1.9': {'config': {'next_hop': u'10.1.1.9'},
             'index': u'10.1.1.9'}}},
          'prefix': u'2.2.2.2/32'}}}}}}}}}}
)

And candidate config has a different static route:

candidate = napalm_yang.base.Root()
candidate.add_model(napalm_yang.models.openconfig_network_instance)
candidate.load_dict({'network_instances': {'network_instance': {u'global': {'config': {'enabled': True,
     'type': u'DEFAULT_INSTANCE'},
    'name': u'global',
    'protocols': {'protocol': {u'static static': {'identifier': u'static',
       'name': u'static',
       'static_routes': {'static': {u'3.3.3.3/32': {'config': {'prefix': u'3.3.3.3/32'},
          'next_hops': {'next_hop': {u'10.1.1.9': {'config': {'next_hop': u'10.1.1.9'},
             'index': u'10.1.1.9'}}},
          'prefix': u'3.3.3.3/32'}}}}}}}}}}
)

And we want to merge candidate into running:

configuration = candidate.translate_config(profile=[profile], merge=running)

The new route gets added, but the old one didn't get removed:

>>> print configuration
ip route 3.3.3.3/32 10.1.1.9

Same issue with replace:

>>> configuration = candidate.translate_config(profile=[profile], replace=running)
>>> print configuration
no ip route 3.3.3.3/32 10.1.1.9
ip route 3.3.3.3/32 10.1.1.9

With logging turned on you see this:

INFO:napalm-yang:Defaulting static: 2.2.2.2/32

But the translator for EOS has this:

static:
    _process: unnecessary

So nothing happens. All the action is hidden under next-hop, but the framework doesn't descend that far - which is fine.

Ideally we could modify the mappings to something like this:

...
static:
    _process:
      - mode: container
        key_value: "ip route {{ static_key }}"
        negate: "no ip route {{ static_key }}\n"
        end: "\n"
    next-hops:
        _process: unnecessary
        next-hop:
            _process:
              - mode: container
                key_value: " {{ next_hop_key }}"
                negate: " {{ next_hop_key }}\n"
...

This looks nice and more maintainable than the current approach. But that doesn't work for static routes with multiple next-hops, you'll get this:

ip route 3.3.3.3/32 10.1.1.9 10.1.1.2

instead of

ip route 3.3.3.3/32 10.1.1.9
ip route 3.3.3.3/32 10.1.1.2

My suggestion is to add another attribute to mode: container that would repeat key_value for each child. Kind of the opposite of flat: true in text parser. Something like:

...
static:
    _process:
      - mode: container
        key_value: "ip route {{ static_key }}"
        negate: "no ip route {{ static_key }}\n"
        repeat_for_children: true
        end: "\n"
    next-hops:
        _process: unnecessary
        next-hop:
            _process:
              - mode: container
                key_value: " {{ next_hop_key }}"
                negate: " {{ next_hop_key }}\n"
...

What do you think?

(wanted to make a POC but I'm still struggling with the internal logic of translators/parsers)

subinterface name in Junos

I have a question regarding the subinterface name in Junos.
I can see a subinterface name is just the index

            "ge-0/0/0": {
                "config": {
                    "enabled": true,
                    "name": "ge-0/0/0",
                    "type": "ethernetCsmacd"
                },
                "name": "ge-0/0/0",
                "subinterfaces": {
                    "subinterface": {
                        "0": {
                            "config": {
                                "description": "R1@ge-0/0/0.0->R2-ge-0/0/0.0",
                                "enabled": true,
                                "name": "0"    <===== ***
                            },

I was wondering if the name should include the parent interface like:

                "subinterfaces": {
                    "subinterface": {
                        "0": {
                            "config": {
                                "description": "R1@ge-0/0/0.0->R2-ge-0/0/0.0",
                                "enabled": true,
                                "name": "ge-0/0/0.0"   <===== ***
                            },

In EOS and IOS the subinterface names contains the parent interface name

                "subinterfaces": {
                    "subinterface": {
                        "1": {
                            "config": {
                                "description": "another subiface",
                                "enabled": true,
                                "name": "GigabitEthernet2.1" 
                "subinterfaces": {
                    "subinterface": {
                        "1": {
                            "config": {
                                "description": "another subiface",
                                "enabled": false,
                                "name": "Ethernet2.1"

Could anyone confirm which is the correct output ? thanks

`_changed` left as `False` when leaf is set to it's default value

Hello,
apparently, if you set an attribute to it's default, _changed is left unmodified. What's the rationale behind that behavior? Is there any way to know the user set the value in that case? It would be interesting to have get(filter=True)(or some other argument) return any value the user might have tried to set even if it was the default value.

Thanks
David

XMLParser Leaf map is missing default argument

Can we get a default attribute supported for mode: map in XMLParser? TextParser has it :)

EOS Interface regex seems to be too greedy

Following the fantastic tutorial (https://github.com/napalm-automation/napalm-yang/blob/develop/interactive_demo/tutorial.ipynb), I ran into an issues with parsing the config of my deployed switches. I kept getting a keyerror being thrown like this one below

File "/Users/tim/napalm_source/napalm-yang/napalm_yang/base.py", line 225, in parse_config
parser.parse()
File "/Users/tim/napalm_source/napalm-yang/napalm_yang/parser.py", line 85, in parse
self._parse(self._yang_name, self.model, self.mapping[self._yang_name])
File "/Users/tim/napalm_source/napalm-yang/napalm_yang/parser.py", line 91, in _parse
self._parse_container(attribute, model, mapping)
File "/Users/tim/napalm_source/napalm-yang/napalm_yang/parser.py", line 130, in _parse_container
self._parse(k, v, mapping[v._yang_name])
File "/Users/tim/napalm_source/napalm-yang/napalm_yang/parser.py", line 93, in _parse
self._parse_list(attribute, model, mapping)
File "/Users/tim/napalm_source/napalm-yang/napalm_yang/parser.py", line 172, in _parse_list
self._parse(key, obj, element_mapping)
File "/Users/tim/napalm_source/napalm-yang/napalm_yang/parser.py", line 91, in _parse
self._parse_container(attribute, model, mapping)
File "/Users/tim/napalm_source/napalm-yang/napalm_yang/parser.py", line 130, in _parse_container
self._parse(k, v, mapping[v._yang_name])
File "/Users/tim/napalm_source/napalm-yang/napalm_yang/parser.py", line 91, in _parse
self._parse_container(attribute, model, mapping)
File "/Users/tim/napalm_source/napalm-yang/napalm_yang/parser.py", line 130, in _parse_container
self._parse(k, v, mapping[v._yang_name])
File "/Users/tim/napalm_source/napalm-yang/napalm_yang/parser.py", line 95, in _parse
self._parse_leaf(attribute, model, mapping)
File "/Users/tim/napalm_source/napalm-yang/napalm_yang/parser.py", line 184, in _parse_leaf
value = self.parser.parse_leaf(attribute, mapping["_process"], self.bookmarks)
File "/Users/tim/napalm_source/napalm-yang/napalm_yang/parsers/base.py", line 101, in parse_leaf
result = self._parse_leaf_default(attribute, m, data)
File "/Users/tim/napalm_source/napalm-yang/napalm_yang/parsers/text_tree.py", line 113, in _parse_leaf_default
return super()._parse_leaf_default(attribute, mapping, data)
File "/Users/tim/napalm_source/napalm-yang/napalm_yang/parsers/jsonp.py", line 116, in _parse_leaf_default
d = mapping['map'][d.lower()]
KeyError: u'port-channel5'

The interface that is trying to be parsed is port-channel51

It appears that regex regexp: "(?P<value>(\\w|-)*)\\d+" is too greedy in the file napalm_yang/mappings/eos/parsers/config/openconfig-interfaces/interfaces.yaml

load_dict behavior

Ensure load_dict behaves this way and add tests:

with overwrite=False, merge all data. For example:

d1 = { "a": {"aa": 1}}
d2 = {"a": {"ab" 2}, "b": 3}

becomes:

{ "a": {"aa": 1, "ab": 2}, "b": 3}

With overwrite=True existing keys are replaced so same structures become instead:

{ "a": {"ab": 2}, "b": 3}

Step 12 in Tutorial On Junos Device Not Functioning

Bug Report

$ python --version
Python 2.7.12

$ pip list | grep napalm-yang
napalm-yang       0.0.7

As I was working through the interactive_demo, was able to get step 11 (Populating the model from a device) working fine when connecting to a live EX4550. However, for step 12 (Populating from a file), I could not get to work when the configuration file from the exact same device was manually created using the commands show configuration | display xml | save [FILENAME] and then downloading to my local machine.

In the NetworkToCode Slack channel, it was requested that I provide the differences between my file and the tutorial's junos.config file. However, I don't want to break any rules my company has in place by sharing the entire configuration of the switch.

I did a comparison and mentioned that the first tag in my file was the <rpc-reply> which was missing in the tutorial's demo file. As suggested, this was removed and the spacing correctly altered and attempted again, but this only produced python errors.

In a direct-message @dbarrosop said he could probably replicate the issue himself. I'd love to be able to help, so please let me know if I can do further testing and/or provide specific sections of the configuration that I can "scrub" confidential data from.

Thanks,
Brady

Discussion: Complex models (oc-platform)

While working on #38 and the openconfig-platform YANG model, I ran into significant difficulties attempting to use the yaml-configured parsers to transform the data models from the devices to the YANG data model. While this may only come into play in extremely complex models, is there a desire to have a standardized way to handle this?

Do we want to propose more base methods designed to return OC-standard dicts, and use load_dict()?

Start adding private, non-base methods to drivers to do the same?

Create a "python" parser that will take the output of the commands and run a method defined in napalm_yang.helpers?

I did write up a procedural method to process EOS inventory, temperature, and transceiver data into a pretty complete model: https://gist.github.com/bewing/3c1a35781cbc6bef4dd88f6f60f69b01

Allow pure python parsers for complex models

Or when the device does not have capabilities to retrieve the operational data in a bulk.
They can also exist as a private method under the drivers, but we have agreed that we could investigate this alternative to avoid a dependency loop.

How can you negate a config item?

I may be dense but I can't seem to find any documentation about how to perform a no <something>. Specifically in this case I'm just trying to wipe the description on a port. I've tried setting it to False/None and an empty string, False/None just get converted to strings and an empty string fails entirely with the cli complaining about an incomplete command.

----------
          ID: interfaces_config
    Function: napalm_yang.managed
      Result: False
     Comment: Cannot execute "load_merge_candidate" on 192.168.255.2 as admin. Reason: Error [1002]: CLI command 9 of 10 'description' failed: invalid command [incomplete command (at token 1: None)]!

I am running with:

    - napalm-yang==0.1.0
    - pyang==1.7.4
    - pyangbind==0.8.0

I can think of a few other cases we would need to no <something> though, is this a supported feature?

fetching interface config when using groups on Junos

Hi,
not sure if you have noticed already, but the following rpc command seems to be more efficient than ' , specially when you use groups as we do.

junos/parsers/config/openconfig-interfaces
interfaces.yaml
metadata:
processor: XMLParser
execute:
- method: _rpc
kwargs:
get: '<get-configuration inherit="inherit"> </get-configuration>'

BR
Al

Unify yaml files renaming translator/parsers into just processors

FileNotFoundError: [WinError 206] Nom de fichier ou extension trop long: 'C:\\Users\\user\\AppData\\Local\\Temp\\9\\pip-build-71ldn97i\\napalm-yang\\napalm_yang/models/openconfig/network_instances/network_instance/mpls/lsps/constrained_path/named_explicit_paths/named_explicit_path/explicit_route_objects/explicit_route_object/config/'

Hello,

I am facing an issue (WinError 206) installing napalm-yang package on Windows7 with Python 3.6.1.

FileNotFoundError: [WinError 206] Nom de fichier ou extension trop long: 'C:\\Users\\user\\AppData\\Local\\Temp\\9\\pip-build-71ldn97i\\napalm-yang\\napalm_yang/models/openconfig/network_instances/network_instance/mpls/lsps/constrained_path/named_explicit_paths/named_explicit_path/explicit_route_objects/explicit_route_object/config/

Full log

C:\Users\user>pip install napalm-yang 
Collecting napalm-yang
  Downloading https://files.pythonhosted.org/packages/c3/bb/65f552c8dea409db2a706e15384bdb0a88af67e74bcc05a13941b6588306
/napalm-yang-0.1.0.tar.gz (4.0MB)
    100% |████████████████████████████████| 4.0MB 6.6MB/s
Exception:
Traceback (most recent call last):
  File "c:\users\user\appdata\local\programs\python\python37-32\lib\site-packages\pip\basecommand.py", line 215, in main
    status = self.run(options, args)
  File "c:\users\user\appdata\local\programs\python\python37-32\lib\site-packages\pip\commands\install.py", line 324, in run
    requirement_set.prepare_files(finder)
  File "c:\users\user\appdata\local\programs\python\python37-32\lib\site-packages\pip\req\req_set.py", line 380, in prepare_files
    ignore_dependencies=self.ignore_dependencies))
  File "c:\users\user\appdata\local\programs\python\python37-32\lib\site-packages\pip\req\req_set.py", line 620, in _prepare_file
    session=self.session, hashes=hashes)
  File "c:\users\user\appdata\local\programs\python\python37-32\lib\site-packages\pip\download.py", line 821, in unpack_url
    hashes=hashes
  File "c:\users\user\appdata\local\programs\python\python37-32\lib\site-packages\pip\download.py", line 663, in unpack_http_url
    unpack_file(from_path, location, content_type, link)
  File "c:\users\user\appdata\local\programs\python\python37-32\lib\site-packages\pip\utils\__init__.py", line 605, in unpack_file
    untar_file(filename, location)
  File "c:\users\user\appdata\local\programs\python\python37-32\lib\site-packages\pip\utils\__init__.py", line 553, in untar_file
    ensure_dir(path)
  File "c:\users\user\appdata\local\programs\python\python37-32\lib\site-packages\pip\utils\__init__.py", line 83, in ensure_dir
    os.makedirs(path)
  File "c:\users\user\appdata\local\programs\python\python37-32\lib\os.py", line 221, in makedirs
    mkdir(name, mode)
FileNotFoundError: [WinError 206] Nom de fichier ou extension trop long: 'C:\\Users\\user\\AppData\\Local\\Temp\\9\\pip-build-71ldn97i\\napalm-yang\\napalm_yang/models/openconfig/network_instances/network_instance/mpls/lsps/constrained_path/named_explicit_paths/named_explicit_path/explicit_route_objects/explicit_route_object/config/'

Could you help me to fix it or find a workaround?

Regards,
Ludovic

Create merge config from the `diff` result

Compare two instrances of a model

And return a dict with the differences.

Exception: I don't have any data to operate with

Hi,
Used napalm-yang from standard pip install and faced the fxp bug. So switched to napalm-yang develop and get this error:

No handlers could be found for logger "napalm-yang"
Traceback (most recent call last):
File "./napalm-yang-01.py", line 17, in
model.parse_config(device=d,profile=['junos'])
File "/home/ansible/playbooks/napalm-yang/napalm_yang/base.py", line 240, in parse_config
parser = Parser(v, device=device, profile=profile, native=native, is_config=True)
File "/home/ansible/playbooks/napalm-yang/napalm_yang/parser.py", line 48, in init
raise Exception("I don't have any data to operate with")
Exception: I don't have any data to operate with

python test script is standard:

#! /usr/bin/python

import napalm
import napalm_yang

from pprint import *

driver = napalm.get_network_driver('junos')

model=napalm_yang.base.Root()
model.add_model(napalm_yang.models.openconfig_interfaces)
model.add_model(napalm_yang.models.openconfig_vlan)

#pprint(napalm_yang.utils.model_to_dict(model))

with driver(hostname="gn4-net-core-5", username="<user>", password="<password>) as d:
    model.parse_config(device=d,profile=['junos'])

With previous napalm-yang module (0.0.7) , I got:

No handlers could be found for logger "napalm-yang"
Traceback (most recent call last):
File "./napalm-yang-01.py", line 17, in
model.parse_config(device=d,profile=['junos'])
File "/home/ansible/playbooks/napalm-yang/napalm_yang/base.py", line 240, in parse_config
parser = Parser(v, device=device, profile=profile, native=native, is_config=True)
File "/home/ansible/playbooks/napalm-yang/napalm_yang/parser.py", line 48, in init
raise Exception("I don't have any data to operate with")
Exception: I don't have any data to operate with
ansible@gn4-ansible:~~/playbooks$ export PYTHONPATH=~~/playbooks/napalm-yang/^C
ansible@gn4-ansible:/playbooks$ unset PYTHONPATH
ansible@gn4-ansible:/playbooks$ ./napalm-yang-01.py
No handlers could be found for logger "napalm-yang"
Traceback (most recent call last):
File "./napalm-yang-01.py", line 17, in
model.parse_config(device=d,profile=['junos'])
File "/usr/local/lib/python2.7/dist-packages/napalm_yang/base.py", line 225, in parse_config
parser.parse()
File "/usr/local/lib/python2.7/dist-packages/napalm_yang/parser.py", line 85, in parse
self._parse(self._yang_name, self.model, self.mapping[self._yang_name])
File "/usr/local/lib/python2.7/dist-packages/napalm_yang/parser.py", line 91, in _parse
self._parse_container(attribute, model, mapping)
File "/usr/local/lib/python2.7/dist-packages/napalm_yang/parser.py", line 132, in _parse_container
self._parse(k, v, mapping[v._yang_name])
File "/usr/local/lib/python2.7/dist-packages/napalm_yang/parser.py", line 93, in _parse
self._parse_list(attribute, model, mapping)
File "/usr/local/lib/python2.7/dist-packages/napalm_yang/parser.py", line 178, in _parse_list
self._parse(key, obj, element_mapping)
File "/usr/local/lib/python2.7/dist-packages/napalm_yang/parser.py", line 91, in _parse
self._parse_container(attribute, model, mapping)
File "/usr/local/lib/python2.7/dist-packages/napalm_yang/parser.py", line 132, in _parse_container
self._parse(k, v, mapping[v._yang_name])
File "/usr/local/lib/python2.7/dist-packages/napalm_yang/parser.py", line 91, in _parse
self._parse_container(attribute, model, mapping)
File "/usr/local/lib/python2.7/dist-packages/napalm_yang/parser.py", line 132, in _parse_container
self._parse(k, v, mapping[v._yang_name])
File "/usr/local/lib/python2.7/dist-packages/napalm_yang/parser.py", line 95, in _parse
self._parse_leaf(attribute, model, mapping)
File "/usr/local/lib/python2.7/dist-packages/napalm_yang/parser.py", line 194, in _parse_leaf
value = self.parser.parse_leaf(mapping["_process"], self.bookmarks)
File "/usr/local/lib/python2.7/dist-packages/napalm_yang/parsers/base.py", line 99, in parse_leaf
result = getattr(self, method_name)(m, data)
File "/usr/local/lib/python2.7/dist-packages/napalm_yang/parsers/jsonp.py", line 116, in _parse_leaf_map
return mapping['map'][v.lower()]
KeyError: u'fxp'

Thanks for your help !

ios and eos parser parsing interface error in special case

When I parsing the following configuration for interface model, I hit error:

interface Loopback101
 description loopback for vrf green
 vrf forwarding green
 ip address 172.16.0.1 255.255.255.255
!

  File "/home/xxx/virtenvs/napalm-yang/local/lib/python2.7/site-packages/napalm_yang-0.1.0-py2.7.egg/napalm_yang/parsers/jsonp.py", line 118, in _parse_leaf_default
    d = mapping["map"][d.lower()]
KeyError: u'loopback10'

It is due to the "regexp" match rule:

        config:
            _process: unnecessary
            name:
                _process: unnecessary
            type:
                _process:
                    - pre: "{{ interface_key }}"
                      regexp: "(?P<value>(\\w|-)*)\\d+"
                      map:
                          fastethernet: ethernetCsmacd
                          gigabitethernet: ethernetCsmacd
                          management: ethernetCsmacd
                          loopback: softwareLoopback
                          port-channel: ieee8023adLag
                          vlan: l3ipvlan

Enable travis

Create OC object from dict

Allow manipulating OC objects by passing a dict that gets merged into it.

Parsing Interface state of an Arista Fails

When I try to parse the state of an Arista device I get the following traceback

Traceback (most recent call last):
  File "state.py", line 10, in <module>
    running_config.parse_state(device=device)
  File "/Users/tim/napalm-yang/napalm_yang/base.py", line 260, in parse_state
    parser.parse()
  File "/Users/tim/napalm-yang/napalm_yang/parser.py", line 81, in parse
    self.native = self.parser.init_native(self.native)
  File "/Users/tim/napalm-yang/napalm_yang/parsers/jsonp.py", line 28, in init_native
    resp.append(json.loads(k))
  File "/usr/local/Cellar/python/2.7.13/Frameworks/Python.framework/Versions/2.7/lib/python2.7/json/__init__.py", line 339, in loads
    return _default_decoder.decode(s)
  File "/usr/local/Cellar/python/2.7.13/Frameworks/Python.framework/Versions/2.7/lib/python2.7/json/decoder.py", line 364, in decode
    obj, end = self.raw_decode(s, idx=_w(s, 0).end())
TypeError: expected string or buffer

I believe this is because the _execute_methods function in parser.py for the Arista gets the results back as list and then appends it to the result list. If I take the first element of the list that is returned by doing something like this in _execute_methods it parses.

if isinstance(r, list):
    r = r[0]
result.append(r)

But that doesn't seem to be the right solution otherwise you couldn't specify multiple commands to run in the yaml state file

RTD link is invalid

Hi guys,

The RTD link in the repo README points to a non-existing URL, looks like it should be:

http://napalm.readthedocs.io/en/latest/yang.html

Make sure interface extensions and BGP models compile

Python 3 support

See details in the parent issue napalm-automation/napalm#199

parse_config returns nothing for network_instance model in Junos

Hello, I'll put you in context, I'm trying to populate a model with the BGP configuration from a Junos (JUNOS 12.1X47-D15.4) router that I set up with Vagrant and Virtualbox, here the tutorial. To populate the model I parse the configuration from the device using the network_instance model but this is only returning an empty set of curly brackets: {}.

I tried populating the interfaces model by the same method and it works just fine. At the beginning I thought that the network_instance profile was not yet implemented for Junos, but I checked here and I saw that some parts are implemented, e.g. the part for the getting the as-number, and the router-id.

I tried populating the model the same way but with an EOS router, also implemented with Vagrant and Virtualbox, and in both cases, with the interfaces and the network_instance model, it worked perfectly. I compared the network_instance profiles for EOS and Junos and noticed that for both the same parts were implemented, which brings me to my question, why is the parse_config function returning no information for the network_instance model in Junos if the profile is already implemented?

Maybe I'm forgetting or not noticing something and this is not really a problem, but I don't know what else to do to make this work or if this is not supposed to work yet. I hope some one can take a look at it.

Thanks for reading,
Elkin

MORE CONTEXT
1. Here the code I'm using:

from napalm_base import get_network_driver
import napalm_yang, salt.client, ast

import json

def use_real_devices():
    junos_configuration = {
        'hostname': '127.0.0.1',
        'username': 'root',
        'password': 'Juniper',
        'optional_args': {'port': 2231}
    }

    eos_configuration = {
        'hostname': '127.0.0.1',
        'username': 'vagrant',
        'password': 'vagrant',
        'optional_args': {'port': 12443}
    }

    junos = get_network_driver("junos")
    junos_device = junos(**junos_configuration)

    eos = get_network_driver("eos")
    eos_device = eos(**eos_configuration)
    return junos_device, eos_device

def pretty_print(dictionary):
    print(json.dumps(dictionary, sort_keys=True, indent=4))

junos_device, eos_device = use_real_devices()

#__________________________________________________________________________________

with junos_device as d:
    candidate = napalm_yang.base.Root()
    candidate.add_model(napalm_yang.models.openconfig_network_instance)
    candidate.parse_config(device=d)
    pretty_print(candidate.get(filter=True))

2. Router Config:

## Last commit: 2018-06-15 08:14:10 UTC by root
version 12.1X47-D15.4;
system {
    host-name r1;
    root-authentication {
        encrypted-password "$1$nq.N1UsY$JxA/ESAj3Ku"; ## SECRET-DATA
        ssh-rsa "ssh-rsa AAAAB3...
    }
    login {
        user vagrant {
            uid 2000;
            class super-user;
            authentication {
                ssh-rsa "ssh-rsa AAAAB3...
            }                           
        }                               
    }                                   
    services {                          
        ssh {                           
            root-login allow;           
        }                               
        netconf {                       
            ssh;                        
        }                               
        web-management {                
            http {                      
                interface ge-0/0/0.0;   
            }                           
        }                               
    }                                   
    syslog {                            
        user * {                        
            any emergency;              
        }                               
        file messages {                 
            any any;                    
            authorization info;         
        }                               
        file interactive-commands {     
            interactive-commands any;   
        }                               
    }                                   
    license {                           
        autoupdate {                    
            url https://ae1.juniper.net/junos/key_retrieval;
        }                               
    }                                   
}                                       
interfaces {                            
    ge-0/0/0 {                          
        unit 0 {                        
            family inet {               
                dhcp;                   
            }                           
        }                               
    }                                   
    ge-0/0/1 {                          
        unit 0 {                        
            family inet {               
                address 192.168.1.1/24; 
            }                           
        }                               
    }                                   
    ge-0/0/2 {                          
        unit 0 {                        
            family inet {               
                address 192.168.2.1/24; 
            }                           
        }                               
    }                                   
    lo0 {                               
        unit 0 {                        
            family inet {               
                address 1.1.1.1/32;     
            }                           
        }                               
    }                                   
}                                       
routing-options {                       
    autonomous-system 100;              
}                                       
protocols {                             
    bgp {                               
        group external-peers {          
            type external;              
            export send-direct;         
            neighbor 192.168.1.2 {      
                peer-as 200;            
            }                           
            neighbor 192.168.2.3 {      
                peer-as 300;            
            }                           
        }                               
    }                                   
}                                       
policy-options {                        
    policy-statement send-direct {      
        term 1 {                        
            from protocol direct;       
            then accept;                
        }                               
    }                                   
}                                       
security {                              
    forwarding-options {                
        family {                        
            inet6 {                     
                mode packet-based;      
            }                           
            mpls {                      
                mode packet-based;      
            }                           
        }                               
    }                                   
}

Real output from a device for the interfaces model

Could you provide a gist with the output for the interfaces model from a real device?

Parsing network statement in ospf

Trying to parsing ios configuration, ospf configuration is as following:

router ospf 65003
 router-id 192.168.0.3
 passive-interface Loopback0
 network 10.1.0.4 0.0.0.3 area 0
 network 10.1.0.8 0.0.0.3 area 1.2.3.4
 network 10.1.0.16 0.0.0.3 area 1.2.3.4
 network 192.168.0.3 0.0.0.0 area 0

In the parser, we need to use IP address and mask fields to find out which interface is.

We need to find a solution to support this.

Real output for the acls model from a real device

Could you provide a gist with the output from a real device for the acls model?

print_model and print_data as class methods

It would be nice to those two functions as class methods:

https://github.com/napalm-automation/napalm-yang/blob/dbarrosop/poc/test.py#L7
https://github.com/napalm-automation/napalm-yang/blob/dbarrosop/poc/test.py#L28

Unable to parse model with attribute "max"

When attempting to parse openconfig-platform via parse_state(), an error is thrown when processing the "max" key:

Finding parser for openconfig-platform:components
Found on profile: eos, /home/bewing/napalm-yang/napalm_yang/mappings/eos/parsers/state/openconfig-platform/components.yaml
Parsing attribute: /components
Parsing attribute: /components/component
Parsing attribute: /components/component
Parsing element component[xcvrSlots_52]
Parsing attribute: /components/component[name='xcvrSlots_52']
Parsing attribute: /components/component[name='xcvrSlots_52']/name
Parsing attribute: /components/component[name='xcvrSlots_52']/subcomponents
Parsing attribute: /components/component[name='xcvrSlots_52']/subcomponents
Parsing attribute: /components/component[name='xcvrSlots_52']/subcomponents/subcomponent
Parsing attribute: /components/component[name='xcvrSlots_52']/subcomponents/subcomponent
Parsing attribute: /components/component[name='xcvrSlots_52']/transceiver
Skipping attribute: openconfig-platform-transceiver:xcvrSlots_52
Finding parser for openconfig-platform-transceiver:transceiver
Couldn't find parsing file: /home/bewing/napalm-yang/napalm_yang/mappings/eos/parsers/state/openconfig-platform-transceiver/transceiver.yaml
Parsing attribute: /components/component[name='xcvrSlots_52']/state
Parsing attribute: /components/component[name='xcvrSlots_52']/state
Parsing attribute: /components/component[name='xcvrSlots_52']/state/version
Parsing attribute: /components/component[name='xcvrSlots_52']/state/version
Parsing attribute: /components/component[name='xcvrSlots_52']/state/name
Parsing attribute: /components/component[name='xcvrSlots_52']/state/name
Parsing attribute: /components/component[name='xcvrSlots_52']/state/temperature
Parsing attribute: /components/component[name='xcvrSlots_52']/state/temperature
Parsing attribute: /components/component[name='xcvrSlots_52']/state/temperature/avg
Parsing attribute: /components/component[name='xcvrSlots_52']/state/temperature/avg
Parsing attribute: /components/component[name='xcvrSlots_52']/state/temperature/instant
Parsing attribute: /components/component[name='xcvrSlots_52']/state/temperature/instant
Parsing attribute: /components/component[name='xcvrSlots_52']/state/temperature/max
Traceback (most recent call last):
  File "test.py", line 15, in <module>
    config.parse_state(d, profile=['eos'])
  File "/home/bewing/napalm-yang/napalm_yang/base.py", line 251, in parse_state
    parser.parse()
  File "/home/bewing/napalm-yang/napalm_yang/parser.py", line 73, in parse
    self._parse(self._yang_name, self.model, self.mapping[self._yang_name])
  File "/home/bewing/napalm-yang/napalm_yang/parser.py", line 79, in _parse
    self._parse_container(attribute, model, mapping)
  File "/home/bewing/napalm-yang/napalm_yang/parser.py", line 102, in _parse_container
    self._parse(k, v, mapping[k])
  File "/home/bewing/napalm-yang/napalm_yang/parser.py", line 81, in _parse
    self._parse_list(attribute, model, mapping)
  File "/home/bewing/napalm-yang/napalm_yang/parser.py", line 133, in _parse_list
    self._parse(key, obj, element_mapping)
  File "/home/bewing/napalm-yang/napalm_yang/parser.py", line 79, in _parse
    self._parse_container(attribute, model, mapping)
  File "/home/bewing/napalm-yang/napalm_yang/parser.py", line 102, in _parse_container
    self._parse(k, v, mapping[k])
  File "/home/bewing/napalm-yang/napalm_yang/parser.py", line 79, in _parse
    self._parse_container(attribute, model, mapping)
  File "/home/bewing/napalm-yang/napalm_yang/parser.py", line 102, in _parse_container
    self._parse(k, v, mapping[k])
  File "/home/bewing/napalm-yang/napalm_yang/parser.py", line 79, in _parse
    self._parse_container(attribute, model, mapping)
  File "/home/bewing/napalm-yang/napalm_yang/parser.py", line 102, in _parse_container
    self._parse(k, v, mapping[k])
KeyError: 'max_'

NullHandler logging

Should logging be done through the NullHandler instead of the StreamHandler?

The recommendation is here: https://docs.python.org/3/howto/logging.html#configuring-logging-for-a-library

napalm-yang/napalm_yang/helpers.py

Line 26 in e981fd7

ch = logging.StreamHandler(stream)

Using NullHandler would eliminate the No handlers could be found for logger "napalm-yang" messages.

oc-interfaces parser failing on Junos fxp interface

$ sudo salt vmx1 napalm_yang.parse "['models.openconfig_interfaces']" config=True
vmx1:
    The minion function caused an exception: Traceback (most recent call last):
      File "/usr/lib/python2.7/dist-packages/salt/minion.py", line 1468, in _thread_return
        return_data = executor.execute()
      File "/usr/lib/python2.7/dist-packages/salt/executors/direct_call.py", line 28, in execute
        return self.func(*self.args, **self.kwargs)
      File "/usr/lib/python2.7/dist-packages/salt/utils/napalm.py", line 393, in func_wrapper
        return func(*args, **kwargs)
      File "/usr/lib/python2.7/dist-packages/salt/modules/napalm_yang_mod.py", line 356, in parse
        root.parse_config(**parser_kwargs)
      File "/usr/local/lib/python2.7/dist-packages/napalm_yang/base.py", line 225, in parse_config
        parser.parse()
      File "/usr/local/lib/python2.7/dist-packages/napalm_yang/parser.py", line 85, in parse
        self._parse(self._yang_name, self.model, self.mapping[self._yang_name])
      File "/usr/local/lib/python2.7/dist-packages/napalm_yang/parser.py", line 91, in _parse
        self._parse_container(attribute, model, mapping)
      File "/usr/local/lib/python2.7/dist-packages/napalm_yang/parser.py", line 132, in _parse_container
        self._parse(k, v, mapping[v._yang_name])
      File "/usr/local/lib/python2.7/dist-packages/napalm_yang/parser.py", line 93, in _parse
        self._parse_list(attribute, model, mapping)
      File "/usr/local/lib/python2.7/dist-packages/napalm_yang/parser.py", line 178, in _parse_list
        self._parse(key, obj, element_mapping)
      File "/usr/local/lib/python2.7/dist-packages/napalm_yang/parser.py", line 91, in _parse
        self._parse_container(attribute, model, mapping)
      File "/usr/local/lib/python2.7/dist-packages/napalm_yang/parser.py", line 132, in _parse_container
        self._parse(k, v, mapping[v._yang_name])
      File "/usr/local/lib/python2.7/dist-packages/napalm_yang/parser.py", line 91, in _parse
        self._parse_container(attribute, model, mapping)
      File "/usr/local/lib/python2.7/dist-packages/napalm_yang/parser.py", line 132, in _parse_container
        self._parse(k, v, mapping[v._yang_name])
      File "/usr/local/lib/python2.7/dist-packages/napalm_yang/parser.py", line 95, in _parse
        self._parse_leaf(attribute, model, mapping)
      File "/usr/local/lib/python2.7/dist-packages/napalm_yang/parser.py", line 194, in _parse_leaf
        value = self.parser.parse_leaf(mapping["_process"], self.bookmarks)
      File "/usr/local/lib/python2.7/dist-packages/napalm_yang/parsers/base.py", line 99, in parse_leaf
        result = getattr(self, method_name)(m, data)
      File "/usr/local/lib/python2.7/dist-packages/napalm_yang/parsers/jsonp.py", line 116, in _parse_leaf_map
        return mapping['map'][v.lower()]
    KeyError: u'fxp'

$ sudo salt vmx1 net.cli "show configuration interfaces fxp0 | display set"
vmx1:
    ----------
    comment:
    out:
        ----------
        show configuration interfaces fxp0 | display set:

            set interfaces fxp0 unit 0 family inet address 10.0.0.15/24
    result:
        True

The XML at that level is the following:

    <interfaces>
        <interface>
            <name>fxp0</name>
            <unit>
                <name>0</name>
                <family>
                    <inet>
                        <address>
                            <name>10.0.0.15/24</name>
                        </address>
                    </inet>
                </family>
            </unit>
        </interface>
    </interfaces>

Very strange that the exception complains about the fxp key, when the interface name is called fxp0.

Logs:

[DEBUG   ] Parsing attribute: /interfaces
[DEBUG   ] Parsing attribute: /interfaces/interface
[DEBUG   ] Parsing attribute: /interfaces/interface
[DEBUG   ] Parsing element interface[fxp0]
[DEBUG   ] Parsing attribute: /interfaces/interface[name='fxp0']
[DEBUG   ] Parsing attribute: /interfaces/interface[name='fxp0']/routed-vlan
[DEBUG   ] Skipping attribute: openconfig-vlan:fxp0
[INFO    ] Finding parser for openconfig-vlan:routed-vlan
[DEBUG   ] Found on profile: junos, /usr/local/lib/python2.7/dist-packages/napalm_yang/mappings/junos/parsers/config/openconfig-vlan/routed-vlan.yaml
[DEBUG   ] Parsing attribute: /interfaces/interface[name='fxp0']/routed-vlan
[DEBUG   ] Parsing attribute: /interfaces/interface[name='fxp0']/routed-vlan/state
[DEBUG   ] Parsing attribute: /interfaces/interface[name='fxp0']/routed-vlan/config
[DEBUG   ] Parsing attribute: /interfaces/interface[name='fxp0']/routed-vlan/config
[DEBUG   ] Parsing attribute: /interfaces/interface[name='fxp0']/routed-vlan/config/vlan
[DEBUG   ] Parsing attribute: /interfaces/interface[name='fxp0']/routed-vlan/config/vlan
[DEBUG   ] Parsing attribute: /interfaces/interface[name='fxp0']/routed-vlan/ipv4
[DEBUG   ] Skipping attribute: openconfig-if-ip:routed-vlan
[INFO    ] Finding parser for openconfig-if-ip:ipv4
[DEBUG   ] Found on profile: junos, /usr/local/lib/python2.7/dist-packages/napalm_yang/mappings/junos/parsers/config/openconfig-if-ip/ipv4.yaml
[DEBUG   ] Parsing attribute: /interfaces/interface[name='fxp0']/routed-vlan/ipv4
[DEBUG   ] Parsing attribute: /interfaces/interface[name='fxp0']/routed-vlan/ipv4/neighbors
[DEBUG   ] Parsing attribute: /interfaces/interface[name='fxp0']/routed-vlan/ipv4/neighbors
[DEBUG   ] Parsing attribute: /interfaces/interface[name='fxp0']/routed-vlan/ipv4/state
[DEBUG   ] Parsing attribute: /interfaces/interface[name='fxp0']/routed-vlan/ipv4/unnumbered
[DEBUG   ] Parsing attribute: /interfaces/interface[name='fxp0']/routed-vlan/ipv4/unnumbered
[DEBUG   ] Parsing attribute: /interfaces/interface[name='fxp0']/routed-vlan/ipv4/config
[DEBUG   ] Parsing attribute: /interfaces/interface[name='fxp0']/routed-vlan/ipv4/config
[DEBUG   ] Parsing attribute: /interfaces/interface[name='fxp0']/routed-vlan/ipv4/config/enabled
[DEBUG   ] Parsing attribute: /interfaces/interface[name='fxp0']/routed-vlan/ipv4/config/enabled
[DEBUG   ] Parsing attribute: /interfaces/interface[name='fxp0']/routed-vlan/ipv4/config/mtu
[DEBUG   ] Parsing attribute: /interfaces/interface[name='fxp0']/routed-vlan/ipv4/config/mtu
[DEBUG   ] Parsing attribute: /interfaces/interface[name='fxp0']/routed-vlan/ipv4/addresses
[DEBUG   ] Parsing attribute: /interfaces/interface[name='fxp0']/routed-vlan/ipv4/addresses
[DEBUG   ] Parsing attribute: /interfaces/interface[name='fxp0']/routed-vlan/ipv6
[DEBUG   ] Skipping attribute: openconfig-if-ip:routed-vlan
[INFO    ] Finding parser for openconfig-if-ip:ipv6
[ERROR   ] Couldn't find parsing file: /usr/local/lib/python2.7/dist-packages/napalm_yang/mappings/junos/parsers/config/openconfig-if-ip/ipv6.yaml
[DEBUG   ] Parsing attribute: /interfaces/interface[name='fxp0']/state
[DEBUG   ] Parsing attribute: /interfaces/interface[name='fxp0']/ethernet
[DEBUG   ] Skipping attribute: openconfig-if-ethernet:fxp0
[INFO    ] Finding parser for openconfig-if-ethernet:ethernet
[ERROR   ] Couldn't find parsing file: /usr/local/lib/python2.7/dist-packages/napalm_yang/mappings/junos/parsers/config/openconfig-if-ethernet/ethernet.yaml
[DEBUG   ] Parsing attribute: /interfaces/interface[name='fxp0']/name
[DEBUG   ] Parsing attribute: /interfaces/interface[name='fxp0']/name
[DEBUG   ] Parsing attribute: /interfaces/interface[name='fxp0']/subinterfaces
[DEBUG   ] Parsing attribute: /interfaces/interface[name='fxp0']/subinterfaces
[DEBUG   ] Parsing attribute: /interfaces/interface[name='fxp0']/subinterfaces/subinterface
[DEBUG   ] Parsing attribute: /interfaces/interface[name='fxp0']/subinterfaces/subinterface
[DEBUG   ] Parsing element subinterface[0]
[DEBUG   ] Parsing attribute: /interfaces/interface[name='fxp0']/subinterfaces/subinterface[index='0']
[DEBUG   ] Parsing attribute: /interfaces/interface[name='fxp0']/subinterfaces/subinterface[index='0']/index
[DEBUG   ] Parsing attribute: /interfaces/interface[name='fxp0']/subinterfaces/subinterface[index='0']/index
[DEBUG   ] Parsing attribute: /interfaces/interface[name='fxp0']/subinterfaces/subinterface[index='0']/vlan
[DEBUG   ] Skipping attribute: openconfig-vlan:0
[INFO    ] Finding parser for openconfig-vlan:vlan
[DEBUG   ] Found on profile: junos, /usr/local/lib/python2.7/dist-packages/napalm_yang/mappings/junos/parsers/config/openconfig-vlan/vlan.yaml
[DEBUG   ] Parsing attribute: /interfaces/interface[name='fxp0']/subinterfaces/subinterface[index='0']/vlan
[DEBUG   ] Parsing attribute: /interfaces/interface[name='fxp0']/subinterfaces/subinterface[index='0']/vlan/state
[DEBUG   ] Parsing attribute: /interfaces/interface[name='fxp0']/subinterfaces/subinterface[index='0']/vlan/config
[DEBUG   ] Parsing attribute: /interfaces/interface[name='fxp0']/subinterfaces/subinterface[index='0']/vlan/config
[DEBUG   ] Parsing attribute: /interfaces/interface[name='fxp0']/subinterfaces/subinterface[index='0']/vlan/config/vlan-id
[DEBUG   ] Parsing attribute: /interfaces/interface[name='fxp0']/subinterfaces/subinterface[index='0']/vlan/config/vlan-id
[DEBUG   ] Parsing attribute: /interfaces/interface[name='fxp0']/subinterfaces/subinterface[index='0']/state
[DEBUG   ] Parsing attribute: /interfaces/interface[name='fxp0']/subinterfaces/subinterface[index='0']/ipv4
[DEBUG   ] Skipping attribute: openconfig-if-ip:0
[INFO    ] Finding parser for openconfig-if-ip:ipv4
[DEBUG   ] Found on profile: junos, /usr/local/lib/python2.7/dist-packages/napalm_yang/mappings/junos/parsers/config/openconfig-if-ip/ipv4.yaml
[DEBUG   ] Parsing attribute: /interfaces/interface[name='fxp0']/subinterfaces/subinterface[index='0']/ipv4
[DEBUG   ] Parsing attribute: /interfaces/interface[name='fxp0']/subinterfaces/subinterface[index='0']/ipv4/neighbors
[DEBUG   ] Parsing attribute: /interfaces/interface[name='fxp0']/subinterfaces/subinterface[index='0']/ipv4/neighbors
[DEBUG   ] Parsing attribute: /interfaces/interface[name='fxp0']/subinterfaces/subinterface[index='0']/ipv4/state
[DEBUG   ] Parsing attribute: /interfaces/interface[name='fxp0']/subinterfaces/subinterface[index='0']/ipv4/unnumbered
[DEBUG   ] Parsing attribute: /interfaces/interface[name='fxp0']/subinterfaces/subinterface[index='0']/ipv4/unnumbered
[DEBUG   ] Parsing attribute: /interfaces/interface[name='fxp0']/subinterfaces/subinterface[index='0']/ipv4/config
[DEBUG   ] Parsing attribute: /interfaces/interface[name='fxp0']/subinterfaces/subinterface[index='0']/ipv4/config
[DEBUG   ] Parsing attribute: /interfaces/interface[name='fxp0']/subinterfaces/subinterface[index='0']/ipv4/config/enabled
[DEBUG   ] Parsing attribute: /interfaces/interface[name='fxp0']/subinterfaces/subinterface[index='0']/ipv4/config/enabled
[DEBUG   ] Parsing attribute: /interfaces/interface[name='fxp0']/subinterfaces/subinterface[index='0']/ipv4/config/mtu
[DEBUG   ] Parsing attribute: /interfaces/interface[name='fxp0']/subinterfaces/subinterface[index='0']/ipv4/config/mtu
[DEBUG   ] Parsing attribute: /interfaces/interface[name='fxp0']/subinterfaces/subinterface[index='0']/ipv4/addresses
[DEBUG   ] Parsing attribute: /interfaces/interface[name='fxp0']/subinterfaces/subinterface[index='0']/ipv4/addresses
[DEBUG   ] Parsing attribute: /interfaces/interface[name='fxp0']/subinterfaces/subinterface[index='0']/ipv4/addresses/address
[DEBUG   ] Parsing attribute: /interfaces/interface[name='fxp0']/subinterfaces/subinterface[index='0']/ipv4/addresses/address
[DEBUG   ] Parsing element address[10.0.0.15]
[DEBUG   ] Parsing attribute: /interfaces/interface[name='fxp0']/subinterfaces/subinterface[index='0']/ipv4/addresses/address[ip='10.0.0.15']
[DEBUG   ] Parsing attribute: /interfaces/interface[name='fxp0']/subinterfaces/subinterface[index='0']/ipv4/addresses/address[ip='10.0.0.15']/ip
[DEBUG   ] Parsing attribute: /interfaces/interface[name='fxp0']/subinterfaces/subinterface[index='0']/ipv4/addresses/address[ip='10.0.0.15']/ip
[DEBUG   ] Parsing attribute: /interfaces/interface[name='fxp0']/subinterfaces/subinterface[index='0']/ipv4/addresses/address[ip='10.0.0.15']/state
[DEBUG   ] Parsing attribute: /interfaces/interface[name='fxp0']/subinterfaces/subinterface[index='0']/ipv4/addresses/address[ip='10.0.0.15']/config
[DEBUG   ] Parsing attribute: /interfaces/interface[name='fxp0']/subinterfaces/subinterface[index='0']/ipv4/addresses/address[ip='10.0.0.15']/config
[DEBUG   ] Parsing attribute: /interfaces/interface[name='fxp0']/subinterfaces/subinterface[index='0']/ipv4/addresses/address[ip='10.0.0.15']/config/ip
[DEBUG   ] Parsing attribute: /interfaces/interface[name='fxp0']/subinterfaces/subinterface[index='0']/ipv4/addresses/address[ip='10.0.0.15']/config/ip
[DEBUG   ] Parsing attribute: /interfaces/interface[name='fxp0']/subinterfaces/subinterface[index='0']/ipv4/addresses/address[ip='10.0.0.15']/config/prefix-length
[DEBUG   ] Parsing attribute: /interfaces/interface[name='fxp0']/subinterfaces/subinterface[index='0']/ipv4/addresses/address[ip='10.0.0.15']/config/prefix-length
[DEBUG   ] Parsing attribute: /interfaces/interface[name='fxp0']/subinterfaces/subinterface[index='0']/ipv4/addresses/address[ip='10.0.0.15']/config/secondary
[DEBUG   ] Skipping attribute: napalm-if-ip:config
[INFO    ] Finding parser for napalm-if-ip:secondary
[ERROR   ] Couldn't find parsing file: /usr/local/lib/python2.7/dist-packages/napalm_yang/mappings/junos/parsers/config/napalm-if-ip/secondary.yaml
[DEBUG   ] Parsing attribute: /interfaces/interface[name='fxp0']/subinterfaces/subinterface[index='0']/ipv4/addresses/address[ip='10.0.0.15']/vrrp
[DEBUG   ] Parsing attribute: /interfaces/interface[name='fxp0']/subinterfaces/subinterface[index='0']/ipv4/addresses/address[ip='10.0.0.15']/vrrp
[DEBUG   ] Parsing attribute: /interfaces/interface[name='fxp0']/subinterfaces/subinterface[index='0']/ipv6
[DEBUG   ] Skipping attribute: openconfig-if-ip:0
[INFO    ] Finding parser for openconfig-if-ip:ipv6
[ERROR   ] Couldn't find parsing file: /usr/local/lib/python2.7/dist-packages/napalm_yang/mappings/junos/parsers/config/openconfig-if-ip/ipv6.yaml
[DEBUG   ] Parsing attribute: /interfaces/interface[name='fxp0']/subinterfaces/subinterface[index='0']/config
[DEBUG   ] Parsing attribute: /interfaces/interface[name='fxp0']/subinterfaces/subinterface[index='0']/config
[DEBUG   ] Parsing attribute: /interfaces/interface[name='fxp0']/subinterfaces/subinterface[index='0']/config/index
[DEBUG   ] Parsing attribute: /interfaces/interface[name='fxp0']/subinterfaces/subinterface[index='0']/config/index
[DEBUG   ] Parsing attribute: /interfaces/interface[name='fxp0']/subinterfaces/subinterface[index='0']/config/enabled
[DEBUG   ] Parsing attribute: /interfaces/interface[name='fxp0']/subinterfaces/subinterface[index='0']/config/enabled
[DEBUG   ] Parsing attribute: /interfaces/interface[name='fxp0']/subinterfaces/subinterface[index='0']/config/name
[DEBUG   ] Parsing attribute: /interfaces/interface[name='fxp0']/subinterfaces/subinterface[index='0']/config/name
[DEBUG   ] Parsing attribute: /interfaces/interface[name='fxp0']/subinterfaces/subinterface[index='0']/config/description
[DEBUG   ] Parsing attribute: /interfaces/interface[name='fxp0']/subinterfaces/subinterface[index='0']/config/description
[DEBUG   ] Parsing attribute: /interfaces/interface[name='fxp0']/hold-time
[DEBUG   ] Parsing attribute: /interfaces/interface[name='fxp0']/hold-time
[DEBUG   ] Parsing attribute: /interfaces/interface[name='fxp0']/config
[DEBUG   ] Parsing attribute: /interfaces/interface[name='fxp0']/config
[DEBUG   ] Parsing attribute: /interfaces/interface[name='fxp0']/config/enabled
[DEBUG   ] Parsing attribute: /interfaces/interface[name='fxp0']/config/enabled
[DEBUG   ] Parsing attribute: /interfaces/interface[name='fxp0']/config/description
[DEBUG   ] Parsing attribute: /interfaces/interface[name='fxp0']/config/description
[DEBUG   ] Parsing attribute: /interfaces/interface[name='fxp0']/config/type
[DEBUG   ] Parsing attribute: /interfaces/interface[name='fxp0']/config/type
[WARNING ] The minion function caused an exception

Whats involved to add new models?

Hi folks,

I've recently been looking into projects to potentially assist with translating between device native configurations and a set of yang models, to which this library looks more or less perfect, however I note that you have python class models for a set of the openconfig models, and no apparent mechanism by which to add new models.

My query is, are these classes I'm seeing in your repository the output from pyangbind? Are there additional steps that need to be taken in order to leverage this library with models not included with it?

If so, would someone mind outlining the basic steps? IIRC, output from pyangbind is a single file with all the generated output, whereas your models appear much nicer structured?