Giter Club home page Giter Club logo

cisco-gnmi-python's Introduction

cisco-gnmi-python

License Code style: black

This library wraps gNMI functionality to ease usage with Cisco implementations in Python programs. Derived from openconfig/gnmi.

This is not an officially supported Cisco product. This library is intended to serve as a gNMI client reference implementation and streamline development with Cisco products.

Usage

pip install cisco-gnmi
python -c "import cisco_gnmi; print(cisco_gnmi)"
cisco-gnmi --help

This library covers the gNMI defined Capabilities, Get, Set, and Subscribe RPCs, and helper clients provide OS-specific recommendations. A CLI (cisco-gnmi) is also available upon installation. As commonalities and differences are identified between OS functionality this library will be refactored as necessary.

Several examples of library usage are available in examples/. The cisco-gnmi CLI script found at src/cisco_gnmi/cli.py may also be useful.

It is highly recommended that users of the library learn Google Protocol Buffers syntax to significantly ease usage. Understanding how to read Protocol Buffers, and reference gnmi.proto, will be immensely useful for utilizing gNMI and any other gRPC interface.

cisco-gnmi CLI

Since v1.0.5 a gNMI CLI is available as cisco-gnmi when this module is installed. Capabilities, Get, rudimentary Set, and Subscribe are supported. The CLI may be useful for simply interacting with a Cisco gNMI service, and also serves as a reference for how to use this cisco_gnmi library. CLI usage is documented at the bottom of this README in CLI Usage.

ClientBuilder

Since v1.0.0 a builder pattern is available with ClientBuilder. ClientBuilder provides several set_* methods which define the intended Client connectivity and a construct method to construct and return the desired Client. There are several major methods involved here:

    set_target(...)
        Specifies the network element to build a client for.
    set_os(...)
        Specifies which OS wrapper to deliver.
    set_secure(...)
        Specifies that a secure gRPC channel should be used.
    set_secure_from_file(...)
        Loads certificates from file system for secure gRPC channel.
    set_secure_from_target(...)
        Attempts to utilize available certificate from target for secure gRPC channel.
    set_call_authentication(...)
        Specifies username/password to utilize for authentication.
    set_ssl_target_override(...)
        Sets the gRPC option to override the SSL target name.
    set_channel_option(...)
        Sets a gRPC channel option. Implies knowledge of channel options.
    construct()
        Constructs and returns the built Client.

Initialization Examples

ClientBuilder can be chained for initialization or instantiated line-by-line.

from cisco_gnmi import ClientBuilder

builder = ClientBuilder('127.0.0.1:9339')
builder.set_os('IOS XR')
builder.set_secure_from_target()
builder.set_call_authentication('admin', 'its_a_secret')
client = builder.construct()

# Or...

client = ClientBuilder('127.0.0.1:9339').set_os('IOS XR').set_secure_from_target().set_call_authentication('admin', 'its_a_secret').construct()

Using an encrypted channel automatically getting the certificate from the device, quick for testing:

from cisco_gnmi import ClientBuilder

client = ClientBuilder(
    '127.0.0.1:9339'
).set_os('IOS XR').set_secure_from_target().set_call_authentication(
    'admin',
    'its_a_secret'
).construct()

Using an owned root certificate on the filesystem:

from cisco_gnmi import ClientBuilder

client = ClientBuilder(
    '127.0.0.1:9339'
).set_os('IOS XR').set_secure_from_file(
    'ems.pem'
).set_call_authentication(
    'admin',
    'its_a_secret'
).construct()

Passing certificate content to method:

from cisco_gnmi import ClientBuilder

# Note reading as bytes
with open('ems.pem', 'rb') as cert_fd:
    root_cert = cert_fd.read()

client = ClientBuilder(
    '127.0.0.1:9339'
).set_os('IOS XR').set_secure(
    root_cert
).set_call_authentication(
    'admin',
    'its_a_secret'
).construct()

Usage with root certificate, private key, and cert chain:

from cisco_gnmi import ClientBuilder

client = ClientBuilder(
    '127.0.0.1:9339'
).set_os('IOS XE').set_secure_from_file(
    root_certificates='rootCA.pem',
    private_key='client.key',
    certificate_chain='client.crt',
).set_call_authentication(
    'admin',
    'its_a_secret'
).construct()

Client

Client is a very barebones class simply implementing capabilities, get, set, and subscribe methods. It provides some context around the expectation for what should be supplied to these RPC functions and helpers for validation.

Methods are documented in src/cisco_gnmi/client.py.

NXClient

NXClient inherits from Client and provides several wrapper methods which aid with NX-OS gNMI implementation usage. These are subscribe_xpaths, and the removal of get and set as they are not yet supported operations. These methods have some helpers and constraints around what is supported by the implementation.

Methods and usage examples are documented in src/cisco_gnmi/nx.py.

XEClient

XEClient inherits from Client and provides several wrapper methods which aid with IOS XE gNMI implementation usage. These are delete_xpaths, get_xpaths, set_json, and subscribe_xpaths. These methods have some helpers and constraints around what is supported by the implementation.

Methods and usage examples are documented in src/cisco_gnmi/xe.py.

XRClient

XRClient inherits from Client and provides several wrapper methods which aid with IOS XR gNMI implementation usage. These are delete_xpaths, get_xpaths, set_json, and subscribe_xpaths. These methods have some helpers and constraints around what is supported by the implementation.

Methods and usage examples are documented in src/cisco_gnmi/xr.py.

gNMI

gRPC Network Management Interface (gNMI) is a service defining an interface for a network management system (NMS) to interact with a network element. It may be thought of as akin to NETCONF or other control protocols which define operations and behaviors. The scope of gNMI is relatively simple - it seeks to "[define] a gRPC-based protocol for the modification and retrieval of configuration from a target device, as well as the control and generation of telemetry streams from a target device to a data collection system. The intention is that a single gRPC service definition can cover both configuration and telemetry - allowing a single implementation on the target, as well as a single NMS element to interact with the device via telemetry and configuration RPCs".

gNMI is a specification developed by OpenConfig, an operator-driven working-group. It is important to note that gNMI only defines a protocol of behavior - not data models. This is akin to SNMP/MIBs and NETCONF/YANG. SNMP and NETCONF are respectively decoupled from the data itself in MIBs and YANG modules. gNMI is a control protocol, not a standardization of data. OpenConfig does develop standard data models as well, and does have some specialized behavior with OpenConfig originating models, but the data models themselves are out of the scope of gNMI.

Development

Requires Python and utilizes pipenv for environment management. Manual usage of pip/virtualenv is not covered. Uses black for code formatting and pylint for code linting. black is not explicitly installed as it requires Python 3.6+.

Get Source

git clone https://github.com/cisco-ie/cisco-gnmi-python.git
cd cisco-gnmi-python
# If pipenv not installed, install!
pip install --user pipenv
# Now use Makefile...
make setup
# Or pipenv manually if make not present
pipenv --three install --dev
# Enter virtual environment
pipenv shell
# Work work
exit

Code Hygiene

We use black for code formatting and pylint for code linting. hygiene.sh will run black against all of the code under gnmi/ except for protoc compiled protobufs, and run pylint against Python files directly under gnmi/. They don't totally agree, so we're not looking for perfection here. black is not automatically installed due to requiring Python 3.6+. hygiene.sh will check for regular path availability and via pipenv, and otherwise falls directly to pylint. If black usage is desired, please install it into pipenv if using Python 3.6+ or separate methods e.g. brew install black.

# If using Python 3.6+
pipenv install --dev black
# Otherwise...
./hygiene.sh

Recompile Protobufs

If a new gnmi.proto definition is released, use update_protos.sh to recompile. If breaking changes are introduced the wrapper library must be updated.

./update_protos.sh

CLI Usage

The below details the current cisco-gnmi usage options. Please note that Set operations may be destructive to operations and should be tested in lab conditions.

cisco-gnmi --help
usage:
cisco-gnmi <rpc> [<args>]

Supported RPCs:
capabilities
subscribe
get
set

cisco-gnmi capabilities 127.0.0.1:57500
cisco-gnmi get 127.0.0.1:57500
cisco-gnmi set 127.0.0.1:57500 -delete_xpath Cisco-IOS-XR-shellutil-cfg:host-names/host-name
cisco-gnmi subscribe 127.0.0.1:57500 -debug -auto_ssl_target_override -dump_file intfcounters.proto.txt

See <rpc> --help for RPC options.


gNMI CLI demonstrating library usage.

positional arguments:
  rpc         gNMI RPC to perform against network element.

optional arguments:
  -h, --help  show this help message and exit

Capabilities

This command will output the CapabilitiesResponse to stdout.

cisco-gnmi capabilities 127.0.0.1:57500 -auto_ssl_target_override

Usage

cisco-gnmi capabilities --help
usage: cisco-gnmi [-h] [-os {None,IOS XR,NX-OS,IOS XE}]
                  [-root_certificates ROOT_CERTIFICATES]
                  [-private_key PRIVATE_KEY]
                  [-certificate_chain CERTIFICATE_CHAIN]
                  [-ssl_target_override SSL_TARGET_OVERRIDE]
                  [-auto_ssl_target_override] [-debug]
                  netloc

Performs Capabilities RPC against network element.

positional arguments:
  netloc                <host>:<port>

optional arguments:
  -h, --help            show this help message and exit
  -os {None,IOS XR,NX-OS,IOS XE}
                        OS wrapper to utilize. Defaults to IOS XR.
  -root_certificates ROOT_CERTIFICATES
                        Root certificates for secure connection.
  -private_key PRIVATE_KEY
                        Private key for secure connection.
  -certificate_chain CERTIFICATE_CHAIN
                        Certificate chain for secure connection.
  -ssl_target_override SSL_TARGET_OVERRIDE
                        gRPC SSL target override option.
  -auto_ssl_target_override
                        Use root_certificates first CN as
                        grpc.ssl_target_name_override.
  -debug                Print debug messages.

Output

[cisco-gnmi-python] cisco-gnmi capabilities redacted:57500 -auto_ssl_target_override
Username: admin
Password:
WARNING:root:Overriding SSL option from certificate could increase MITM susceptibility!
INFO:root:supported_models {
  name: "Cisco-IOS-XR-qos-ma-oper"
  organization: "Cisco Systems, Inc."
  version: "2019-04-05"
}
...

Get

This command will output the GetResponse to stdout. -xpath may be specified multiple times to specify multiple Paths for the GetRequest.

cisco-gnmi get 127.0.0.1:57500 -os "IOS XR" -xpath /interfaces/interface/state/counters -auto_ssl_target_override

Usage

cisco-gnmi get --help
usage: cisco-gnmi [-h] [-xpath XPATH]
                  [-encoding {JSON,BYTES,PROTO,ASCII,JSON_IETF}]
                  [-data_type {ALL,CONFIG,STATE,OPERATIONAL}] [-dump_json]
                  [-os {None,IOS XR,NX-OS,IOS XE}]
                  [-root_certificates ROOT_CERTIFICATES]
                  [-private_key PRIVATE_KEY]
                  [-certificate_chain CERTIFICATE_CHAIN]
                  [-ssl_target_override SSL_TARGET_OVERRIDE]
                  [-auto_ssl_target_override] [-debug]
                  netloc

Performs Get RPC against network element.

positional arguments:
  netloc                <host>:<port>

optional arguments:
  -h, --help            show this help message and exit
  -xpath XPATH          XPaths to Get.
  -encoding {JSON,BYTES,PROTO,ASCII,JSON_IETF}
                        gNMI Encoding.
  -data_type {ALL,CONFIG,STATE,OPERATIONAL}
                        gNMI GetRequest DataType
  -dump_json            Dump as JSON instead of textual protos.
  -os {None,IOS XR,NX-OS,IOS XE}
                        OS wrapper to utilize. Defaults to IOS XR.
  -root_certificates ROOT_CERTIFICATES
                        Root certificates for secure connection.
  -private_key PRIVATE_KEY
                        Private key for secure connection.
  -certificate_chain CERTIFICATE_CHAIN
                        Certificate chain for secure connection.
  -ssl_target_override SSL_TARGET_OVERRIDE
                        gRPC SSL target override option.
  -auto_ssl_target_override
                        Use root_certificates first CN as
                        grpc.ssl_target_name_override.
  -debug                Print debug messages.

Output

[cisco-gnmi-python] cisco-gnmi get redacted:57500 -os "IOS XR" -xpath /interfaces/interface/state/counters -auto_ssl_target_override
Username: admin
Password:
WARNING:root:Overriding SSL option from certificate could increase MITM susceptibility!
INFO:root:notification {
  timestamp: 1585607100869287743
  update {
    path {
      elem {
        name: "interfaces"
      }
      elem {
        name: "interface"
      }
      elem {
        name: "state"
      }
      elem {
        name: "counters"
      }
    }
    val {
      json_ietf_val: "{\"in-unicast-pkts\":\"0\",\"in-octets\":\"0\"...

Set

Please note that Set operations may be destructive to operations and should be tested in lab conditions. Behavior is not fully validated.

Usage

cisco-gnmi set --help
usage: cisco-gnmi [-h] [-update_json_config UPDATE_JSON_CONFIG]
                  [-replace_json_config REPLACE_JSON_CONFIG]
                  [-delete_xpath DELETE_XPATH] [-no_ietf] [-dump_json]
                  [-os {None,IOS XR,NX-OS,IOS XE}]
                  [-root_certificates ROOT_CERTIFICATES]
                  [-private_key PRIVATE_KEY]
                  [-certificate_chain CERTIFICATE_CHAIN]
                  [-ssl_target_override SSL_TARGET_OVERRIDE]
                  [-auto_ssl_target_override] [-debug]
                  netloc

Performs Set RPC against network element.

positional arguments:
  netloc                <host>:<port>

optional arguments:
  -h, --help            show this help message and exit
  -update_json_config UPDATE_JSON_CONFIG
                        JSON-modeled config to apply as an update.
  -replace_json_config REPLACE_JSON_CONFIG
                        JSON-modeled config to apply as a replace.
  -delete_xpath DELETE_XPATH
                        XPaths to delete.
  -no_ietf              JSON is not IETF conformant.
  -dump_json            Dump as JSON instead of textual protos.
  -os {None,IOS XR,NX-OS,IOS XE}
                        OS wrapper to utilize. Defaults to IOS XR.
  -root_certificates ROOT_CERTIFICATES
                        Root certificates for secure connection.
  -private_key PRIVATE_KEY
                        Private key for secure connection.
  -certificate_chain CERTIFICATE_CHAIN
                        Certificate chain for secure connection.
  -ssl_target_override SSL_TARGET_OVERRIDE
                        gRPC SSL target override option.
  -auto_ssl_target_override
                        Use root_certificates first CN as
                        grpc.ssl_target_name_override.
  -debug                Print debug messages.

Output

Let's create a harmless loopback interface based from openconfig-interfaces.yang.

config.json

{
    "openconfig-interfaces:interfaces": {
        "interface": [
            {
                "name": "Loopback9339"
            }
        ]
    }
}
[cisco-gnmi-python] cisco-gnmi set redacted:57500 -os "IOS XR" -auto_ssl_target_override -update_json_config config.json
Username: admin
Password:
WARNING:root:Overriding SSL option from certificate could increase MITM susceptibility!
INFO:root:response {
  path {
    origin: "openconfig-interfaces"
    elem {
      name: "interfaces"
    }
  }
  message {
  }
  op: UPDATE
}
message {
}
timestamp: 1585715036783451369

And on IOS XR...a loopback interface!

...
interface Loopback9339
!
...

Subscribe

This command will output the SubscribeResponse to stdout or -dump_file. -xpath may be specified multiple times to specify multiple Paths for the GetRequest.

cisco-gnmi subscribe 127.0.0.1:57500 -os "IOS XR" -xpath /interfaces/interface/state/counters -auto_ssl_target_override

Usage

cisco-gnmi subscribe --help
usage: cisco-gnmi [-h] [-xpath XPATH] [-interval INTERVAL]
                  [-mode {TARGET_DEFINED,ON_CHANGE,SAMPLE}]
                  [-suppress_redundant]
                  [-heartbeat_interval HEARTBEAT_INTERVAL]
                  [-dump_file DUMP_FILE] [-dump_json] [-sync_stop]
                  [-sync_start] [-encoding {JSON,BYTES,PROTO,ASCII,JSON_IETF}]
                  [-os {None,IOS XR,NX-OS,IOS XE}]
                  [-root_certificates ROOT_CERTIFICATES]
                  [-private_key PRIVATE_KEY]
                  [-certificate_chain CERTIFICATE_CHAIN]
                  [-ssl_target_override SSL_TARGET_OVERRIDE]
                  [-auto_ssl_target_override] [-debug]
                  netloc

Performs Subscribe RPC against network element.

positional arguments:
  netloc                <host>:<port>

optional arguments:
  -h, --help            show this help message and exit
  -xpath XPATH          XPath to subscribe to.
  -interval INTERVAL    Sample interval in seconds for Subscription. Defaults
                        to 10.
  -mode {TARGET_DEFINED,ON_CHANGE,SAMPLE}
                        SubscriptionMode for Subscription. Defaults to SAMPLE.
  -suppress_redundant   Suppress redundant information in Subscription.
  -heartbeat_interval HEARTBEAT_INTERVAL
                        Heartbeat interval in seconds.
  -dump_file DUMP_FILE  Filename to dump to. Defaults to stdout.
  -dump_json            Dump as JSON instead of textual protos.
  -sync_stop            Stop on sync_response.
  -sync_start           Start processing messages after sync_response.
  -encoding {JSON,BYTES,PROTO,ASCII,JSON_IETF}
                        gNMI Encoding. Defaults to whatever Client wrapper
                        prefers.
  -os {None,IOS XR,NX-OS,IOS XE}
                        OS wrapper to utilize. Defaults to IOS XR.
  -root_certificates ROOT_CERTIFICATES
                        Root certificates for secure connection.
  -private_key PRIVATE_KEY
                        Private key for secure connection.
  -certificate_chain CERTIFICATE_CHAIN
                        Certificate chain for secure connection.
  -ssl_target_override SSL_TARGET_OVERRIDE
                        gRPC SSL target override option.
  -auto_ssl_target_override
                        Use root_certificates first CN as
                        grpc.ssl_target_name_override.
  -debug                Print debug messages.

Output

[cisco-gnmi-python] cisco-gnmi subscribe redacted:57500 -os "IOS XR" -xpath /interfaces/interface/state/counters -auto_ssl_target_override
Username: admin
Password:
WARNING:root:Overriding SSL option from certificate could increase MITM susceptibility!
INFO:root:Dumping responses to stdout as textual proto ...
INFO:root:Subscribing to:
/interfaces/interface/state/counters
INFO:root:update {
  timestamp: 1585607768601000000
  prefix {
    origin: "openconfig"
    elem {
      name: "interfaces"
    }
    elem {
      name: "interface"
      key {
        key: "name"
        value: "Null0"
      }
    }
    elem {
      name: "state"
    }
    elem {
      name: "counters"
    }
  }
  update {
    path {
      elem {
        name: "in-octets"
      }
    }
    val {
      uint_val: 0
    }
  }
...

Licensing

cisco-gnmi-python is licensed as Apache License, Version 2.0.

Issues

Open an issue :)

Related Projects

  1. openconfig/gnmi
  2. google/gnxi
  3. Telegraf Cisco gNMI Plugin

cisco-gnmi-python's People

Contributors

cprecup avatar marcsello avatar miott avatar remingtonc avatar skkumaravel avatar suv27 avatar tahigash avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

cisco-gnmi-python's Issues

replace issue in client.py

client.py

I believe the below code should be request.replace.extend(replaces)

if replaces:
request.replaces.extend(replaces)

Thanks,
Mahendra

The grpc channel is not closing.

If the TSL/authentication parameters are not valid, the channel remains open retrying the authentication. The channel is not accessible from the subclasses of Client and without having access to the "Channel.close" method or having the ability to delete the channel from memory, the only way to completely shutdown is to exit the Python interpreter.

Deprecate Python 2 support?

This would simplify some aspects of workflow. Need for automation projects which are exclusively Python 2.

Handle empty XPath appropriately

An empty XPath should not result in a Path with any PathElems.

cat get_all.txt
path: <
>
type: CONFIG
encoding: JSON_IETF
I think you can remove elem entirely

Suggest examples in the README (or Wiki).

I could use an example of how to formulate a Subscription.

Suggestion for documentation:
For the README or WiKi, I'd suggest something like the following:

Working example of OpenConfig Streaming Telemetry on Aceess Point.

from cisco_gnmi import ClientBuilder

builder = ClientBuilder('192.168.0.5:8080')
builder.set_secure_from_target()
builder.set_ssl_target_override('openconfig.mojonetworks.com')
builder.set_call_authentication('admin', 'admin')
client = builder.construct()

paths = [client.parse_xpath_to_gnmi_path('/access-points/access-point[hostname="test-ap.example.net"]/radios/radio[id="0"]/state/rx-noise-channel-utilization/')]
print(client.get(paths))

You may also want to provide a pointer in your documentation that list-key values must be quoted due to your xpath parsing. Even if the list-key is an int (like the radio id in the above example).

NX-OS DME break in Path parsing

Path parsing e.g. /sys/intf/phys-[eth1/33] does not appropriately handle paths. Need to make Path parsing more modular to account for format differences that are not XPath based.

subscription {
  path {
    origin: "DME"
    elem {
      name: "sys"
    }
    elem {
      name: "intf"
    }
    elem {
      name: "phys-"
    }
    elem {
      key {
        key: "eth1"
        value: ""
      }
    }
  }
  mode: SAMPLE
  sample_interval: 10000000000
}

Recieving EOF grpc message with "-no_stop" SubscribeRequest

When utilizing the -no_stop option to continually dump, the RPC is terminating with EOF.

python3 subscribe_dump.py -xpath '/access-points/access-point[hostname="albano-house.example.net"]/radios/radio[id="0"]/state/rx-noise-channel-utilization/' '192.168.0.31:8080' -os None
Username: admin
Password:
INFO:root:Connecting to 192.168.0.31:8080 as None ...
WARNING:root:Overriding SSL option from certificate could increase MITM susceptibility!
INFO:root:Subscribing to /access-points/access-point[hostname="albano-house.example.net"]/radios/radio[id="0"]/state/rx-noise-channel-utilization/ ...
INFO:root:{'update': {'timestamp': '1587091884807946240', 'update': [{'path': {'elem': [{'name': 'access-points'}, {'name': 'access-point', 'key': {'hostname': 'albano-house.example.net'}}, {'name': 'radios'}, {'name': 'radio', 'key': {'id': '0'}}, {'name': 'state'}, {'name': 'rx-noise-channel-utilization'}]}, 'val': {'uintVal': '0'}}]}}
ERROR:root:Stopping due to exception!
Traceback (most recent call last):
  File "subscribe_dump.py", line 56, in main
    for message in client.subscribe_xpaths(**sub_args):
  File "/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/site-packages/grpc/_channel.py", line 364, in __next__
    return self._next()
  File "/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/site-packages/grpc/_channel.py", line 347, in _next
    raise self
grpc._channel._Rendezvous: <_Rendezvous of RPC that terminated with:
	status = StatusCode.UNKNOWN
	details = "EOF"
	debug_error_string = "{"created":"@1587091885.760523000","description":"Error received from peer","file":"src/core/lib/surface/call.cc","file_line":1036,"grpc_message":"EOF","grpc_status":2}"
>
INFO:root:Writing to gnmi_sub.json ...

And gnmi_sub.json does contain that first response as in the output above.

This particular Target is an Arista Access Point.

Support to handle data size > 4mb in get responses

When we perform a gnmi get request, the default size handles only 4mb response size. We need to handle datasize > 4mb in responses.

options = [('grpc.max_receive_message_length', 1610241024)]

Thanks,
Mahendra

Autogen documentation from source

The modules are relatively well documented with numpy style. Should be able to use sphinx with sphinx.ext.napoleon to autogen more readable docs.

Related to #40 but separating from examples given uncertainty.

Tracking in doc-autogen.

pipenv shell
mkdir docs
cd docs
sphinx-quickstart
cd ..
sphinx-apidoc -f -o docs/source src/cisco_gnmi
cd docs
make html
exit

Add tests

Tests will actually be helpful here.

More easily expose messages

It's currently difficult to determine exactly what messages are being constructed. Should develop message building infra and have clients communicate using it.

Need mechanism to update/replace based on xpath

Current way of doing the update/replace operation requires user to provide the payload in the json format. While this seems ok for update operation, for replace operation user would have to provide the entire payload, though that is not being replaced.

For example, if user needs to replace an ip address on an interface on NX-OS, and if if the box had 48 interfaces, the user would have to create the payload which would include all the interfaces on the device with their corresponding info, even though the intent is to replace ip address for only interface.

Providing a mechanism to replace based on xpath would help resolve this.
something of the below format
--xpath_update /interfaces/interface[name=eth1/47]/@ip.json

On NX-OS, if we do not provide the entire payload, we would hit the below error. I believe, it could be the same for other platforms too.

File "/ws/mgangaia-sjc/pyats-latest/lib/python3.6/site-packages/cisco_gnmi/nx.py", line 194, in set_json
return self.set(prefix=prefix, updates=updates, replaces=replaces)
File "/ws/mgangaia-sjc/pyats-latest/lib/python3.6/site-packages/cisco_gnmi/client.py", line 223, in set
response = self.service.Set(request, metadata=self.default_call_metadata)
File "/ws/mgangaia-sjc/pyats-latest/lib/python3.6/site-packages/grpc/_channel.py", line 549, in call
return _end_unary_response_blocking(state, call, False, None)
File "/ws/mgangaia-sjc/pyats-latest/lib/python3.6/site-packages/grpc/_channel.py", line 466, in _end_unary_response_blocking
raise _Rendezvous(state, None, None, deadline)
grpc._channel._Rendezvous: <_Rendezvous of RPC that terminated with:
status = StatusCode.INTERNAL
details = "descr:internal processing error [error:operation-failed msg:Cannot delete object of class:l1PhysIf path:/config/interfaces][error:operation-failed msg:Cannot delete object of class:l1PhysIf Commit Failed path:/config/interfaces]"
debug_error_string = "{"created":"@1595898351.637576743","description":"Error received from peer","file":"src/core/lib/surface/call.cc","file_line":1039,"grpc_message":"descr:internal processing error [error:operation-failed msg:Cannot delete object of class:l1PhysIf path:/config/interfaces][error:operation-failed msg:Cannot delete object of class:l1PhysIf Commit Failed path:/config/interfaces]","grpc_status":13}"

Thanks,
Mahendra

IOS XE/gRPC Certificate Usage

Trying to reconcile how to use certs with gRPC and IOS XE specifically. I believe the use case presented with IOS XE is more comprehensive than IOS XR and NX-OS thus this is a good exercise if not frustrating.

Following IOS XE 16.12.x gNMI documentation.

>>> client = ClientBuilder('x').set_secure_from_file('../scripts/certs/client.crt', '../scripts/certs/client.key', '../scripts/certs/rootCA.pem').set_ssl_target_override().set_call_authentication('x', 'x').construct()
WARNING:root:Overriding SSL option from certificate could increase MITM susceptibility!
>>> print(client.capabilities())
E1008 14:49:11.385280000 4671555008 ssl_transport_security.cc:690]     Invalid private key.
E1008 14:49:11.385306000 4671555008 ssl_security_connector.cc:112]     Handshaker factory creation failed with TSI_INVALID_ARGUMENT.
E1008 14:49:11.385314000 4671555008 secure_channel_create.cc:132]      Failed to create secure subchannel for secure name 'x:9339'
E1008 14:49:11.385322000 4671555008 secure_channel_create.cc:50]       Failed to create channel args during subchannel creation.
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/Users/remcampb/Development/projects/cisco-gnmi-python/src/cisco_gnmi/client.py", line 112, in capabilities
    response = self.service.Capabilities(message)
  File "/Users/remcampb/.local/share/virtualenvs/cisco-gnmi-python-QTeA_bEB/lib/python3.7/site-packages/grpc/_channel.py", line 604, in __call__
    return _end_unary_response_blocking(state, call, False, None)
  File "/Users/remcampb/.local/share/virtualenvs/cisco-gnmi-python-QTeA_bEB/lib/python3.7/site-packages/grpc/_channel.py", line 506, in _end_unary_response_blocking
    raise _Rendezvous(state, None, None, deadline)
grpc._channel._Rendezvous: <_Rendezvous of RPC that terminated with:
	status = StatusCode.UNAVAILABLE
	details = "Empty update"
	debug_error_string = "{"created":"@1570571351.385334000","description":"Failed to pick subchannel","file":"src/core/ext/filters/client_channel/client_channel.cc","file_line":3876,"referenced_errors":[{"created":"@1570571351.385331000","description":"Empty update","file":"src/core/ext/filters/client_channel/lb_policy/pick_first/pick_first.cc","file_line":200,"grpc_status":14}]}"
>

Potentially helpful: grpc/grpc#9593

Blocks #9

Python 2 Support

String comparisons are broken. Need to resolve for Python 2 usage.

Return data in useful format for Get

Currently the returns for Get (and all other methods for that matter) return the proto response classes. This has several implications - must understand the structure of the protos, must know encoding, must know what to access where... We can probably simplify this for the end user.

Freeze grpcio for stability.

Let's pin grpcio to a specific version in setup.py. We can monitor it for important fixes or new features and then upgrade after analysis of backward compatibility and stability.

Validate deletion of leaflist entry on nxos

The below format is the supported xpath to delete a leaflist entry on nxos.
Please check if we could use the below xpath to perform deletion.

/network-instances/network-instance[name=foo]/config/enabled-address-families[.=IPV4]

Thanks,
Mahendra

set_json Path should be as specific as possible

IOS XR/IOS XE set_json Path generation only goes one element deep, we should instead make this traverse the structure until multiple keys are identified and use that longest prefix as XPath.

Filing as a bug because in a replace operation this could create a significant impact on device.

ssl.get_server_certificate failure with IOS XE/NX-OS

Following documentation at https://www.cisco.com/c/en/us/td/docs/ios-xml/ios/prog/configuration/1610/b_1610_programmability_cg/gnmi_protocol.html#id_67108

Using the generated certificates yields:

python test_xe.py
Traceback (most recent call last):
  File "test_xe.py", line 14, in <module>
    print(client.capabilities())
  File "/home/remcampb/development/cisco-gnmi-python/src/cisco_gnmi/client.py", line 93, in capabilities
    response = self.service.Capabilities(message, metadata=self._gen_metadata())
  File "/home/remcampb/.local/share/virtualenvs/cisco-gnmi-python-z1MRTrKn/local/lib/python2.7/site-packages/grpc/_channel.py", line 565, in __call__
    return _end_unary_response_blocking(state, call, False, None)
  File "/home/remcampb/.local/share/virtualenvs/cisco-gnmi-python-z1MRTrKn/local/lib/python2.7/site-packages/grpc/_channel.py", line 467, in _end_unary_response_blocking
    raise _Rendezvous(state, None, None, deadline)
grpc._channel._Rendezvous: <_Rendezvous of RPC that terminated with:
	status = StatusCode.UNAVAILABLE
	details = "failed to connect to all addresses"
	debug_error_string = "{"created":"@1569363736.047551305","description":"Failed to pick subchannel","file":"src/core/ext/filters/client_channel/client_channel.cc","file_line":3818,"referenced_errors":[{"created":"@1569363736.047542320","description":"failed to connect to all addresses","file":"src/core/ext/filters/client_channel/lb_policy/pick_first/pick_first.cc","file_line":395,"grpc_status":14}]}"
>

Attempting to download and use certificate from device:

python test_xe.py
Traceback (most recent call last):
  File "test_xe.py", line 16, in <module>
    client = XEClient('...', attempt_implicit_secure=True).with_authentication('...', '...')
  File "/home/remcampb/development/cisco-gnmi-python/src/cisco_gnmi/client.py", line 78, in __init__
    self.as_secure(root_from_target=True, target_name_from_root=True)
  File "/home/remcampb/development/cisco-gnmi-python/src/cisco_gnmi/base.py", line 113, in as_secure
    root_certificates = get_cert_from_target(self.target_netloc)
  File "/home/remcampb/development/cisco-gnmi-python/src/cisco_gnmi/base.py", line 80, in get_cert_from_target
    (self.target_netloc.hostname, self.target_netloc.port)
  File "/usr/lib/python2.7/ssl.py", line 1007, in get_server_certificate
    with closing(context.wrap_socket(sock)) as sslsock:
  File "/usr/lib/python2.7/ssl.py", line 353, in wrap_socket
    _context=self)
  File "/usr/lib/python2.7/ssl.py", line 601, in __init__
    self.do_handshake()
  File "/usr/lib/python2.7/ssl.py", line 830, in do_handshake
    self._sslobj.do_handshake()
ssl.SSLEOFError: EOF occurred in violation of protocol (_ssl.c:590)

OpenSSL too has issues.

openssl s_client -state -connect $IP:$PORT
CONNECTED(00000003)
SSL_connect:before/connect initialization
SSL_connect:SSLv2/v3 write client hello A
SSL_connect:unknown state
depth=1 CN = rootCA
verify error:num=19:self signed certificate in certificate chain
SSL_connect:unknown state
SSL_connect:unknown state
SSL_connect:unknown state
SSL_connect:unknown state
SSL_connect:unknown state
SSL_connect:unknown state
SSL_connect:unknown state
SSL_connect:unknown state
SSL_connect:unknown state
SSL_connect:failed in unknown state
139800832919192:error:140790E5:SSL routines:ssl23_write:ssl handshake failure:s23_lib.c:177:

Might be related to SNI?

XPath predicate parsing requires string value

Per #40 XPath predicate exprs currently must be strings. Must investigate:

  1. Current XML XPath regex result, does it require a string?
  2. May YANG XPath predicate values be types other than strings?
  3. Easy fix, then scope something more robust like a state machine. pyang just added some xpath code, might be able to disaggregate.

Intelligent Path origin

Determine how to cleanly express the origin and "smart" determination of what the origin should be per XPath.

The check_configs function only works with one update config.

The "name" parameter is a hardcoded string called from create_updates so it will always go to this if:

        def check_configs(name, configs):
            if isinstance(name, string_types):
                logging.debug("Handling %s as JSON string.", name)
                try:
                    configs = json.loads(configs)
                except:
                    raise Exception("{name} is invalid JSON!".format(name=name))
                configs = [configs]

The "configs" variable can only be one dict for json.loads unless you put it in a list, but if you do, the configs list is then deposited into another list.

Parse XPath filters to Path

Currently ignores (unwillingly) XPath filters which could otherwise be expressed in Path and will ultimately break if filters are provided.

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.