akarneliuk / pygnmi Goto Github PK

View Code? Open in Web Editor NEW

125.0 13.0 43.0 1.45 MB

The pure Python implementation of the gNMI client.

Home Page: https://training.karneliuk.com

License: BSD 3-Clause "New" or "Revised" License

Python 100.00%

automation gnmi grpc network network-automation pygnmi python telemetry

pygnmi's Introduction

pyGNMI: Python gNMI client

This repository contains pure Python implementation of the gNMI client to interact with the network functions.

Usage

Sample code example:

# Modules
from pygnmi.client import gNMIclient

# Variables
host = ('169.254.255.64', '57400')

# Body
if __name__ == '__main__':
    with gNMIclient(target=host, username='admin', password='admin', insecure=True) as gc:
         result = gc.get(path=['openconfig-interfaces:interfaces', 'openconfig-acl:acl'])

    print(result)

Also integration with Nornir is supported (refer to examples).

Video tutorial

Watch the detailed explanation how to use pyGNMI in our YouTube channel.

All gNMI RPCs supported

Capabilities
Get
Set
Subscribe

Supported operation modes

insecure gRPC channel (without encryption)
secure gRPC channel (with encryption and authentication based on certificate)

Tested Network Operating Systems (NOS)

Arista EOS
Nokia SR OS
Cisco IOS XR
Juniper JUNOS
Nokia SRLinux
Cisco NX-OS

Network Operating Systems (NOS) in test

Broadcom SONiC

License

By using the pyGNMI tool you agree with the license.

Contributors

Dev Log

Release 0.8.14: - Number of minor bug fixes and improvements. - Adding support of prefix to pygnmicli. - Adding error propagation from child thread to main thread <#142> - Changes to u_val <#144> - Adding Master Arbitration support for Set <#146> - Adding bytes_val and leaflist_val with string_val parsing <#151>

Release 0.8.13: - Number of minor bug fixes and improvements.

Release 0.8.12: - Fixed operation of no_qos_marking flag for pygnmicli.

Release 0.8.11: - Previous release introduced break for telemetry in Juniper due to inconsistency of communicated encoudings in Capabilities() and what is really supported in Subscribe().

Release 0.8.10: - Automatic detection of supported encoding and using it where applicable (e.g., in subscribe2 method). - Possibility to remove qos from Subscribe for platforms, which doesn't support it (e.g., Juniper).

Release 0.8.9:

Default value for encoding everywhere is set to None.
Method capabilities() now is called as part of connect() to collect supported encoding as part of session establishing.
If encoding is not specified by user, then it is auto-set based on the list collected via capabilites() with json having the first priority follwed by json_ietf.

Release 0.8.8:

Added new argument -e / --encoding to pygnmicli to specify the encoding, which overrides the default one. Fix for Issue 58.
Fixed minor bug with encoding handling inside get() and subscribe2() methods.
Simplified the code.

Release 0.8.7:

Fixed bug, when returned json_val or json_ietf_val is not processed correctly if the value is empty string. Fix for Issue 58.

Release 0.8.6:

Fixed minor issue with establishing insecure channel.
Fixed bug with inabillity to specify prefix in Subscribe messages for subscribe2() method.
Important: It is recommended to use method subscribe2() instead of subscribe() for building telemetry collectors with pygnmi as this method is further developed and throroughle tested. The method subscribe() will be deprecated in future releases.
Functionality qos is now properly supported in subscribe2() method.

Release 0.8.5:

Fixed some issues with telemetry representation with pygnmicli.

Release 0.8.4:

Change logic of setting default values for some parameters to improve user experience.
Added token authentication to pygnmicli.

Release 0.8.3:

Changed behaviour of subscribe2() to RPC to avoid adding the empty Extension field for no extensions presenting. Fix for Issue 83.
Uppdated documentation with examples in GitHub.
Added support of History extensions to pygnmicli.

Release 0.8.2:

Implemented History Extension.
Implemented handling of corner case, where --skip-verify was failing trying to parse certificate, which doesn't have CN and SARs.

Release 0.8.1:

Removed the need for --no-binary=protobuf for operation.

Release 0.8.0:

Important: potentially breaking change. The dependency is moved from grpcio-tools to protobuf, which as a standalone package has a much newer serion.
Spec is rebuilt and updated to support gNMI of version 0.8.0.

Release 0.7.5:

Amended the logic of ONCE telemetry mode to automatically terminate on receiving {"sync_response": True} message.

Release 0.7.4:

Feature skip_verify is now stabilised and doesn't require subject alternative names any more.

Release 0.7.3:

Amended the logic of target functionality to be more inline with gNMI Reference.

Release 0.7.2:

Minor bug fixing in the skip_verify logic. Impotant: for this feature to work, you need at least one subject alternative name filed (DNS, IP address, email, - any will work). It also doesn't matter which value it has, but at least one item shall present.

Release 0.7.1:

Added new argument skip_verify to gNMIclient, which removes a need to set the override argument manually. However, the latter one still stays for the backward compatibility.
Changed default values for arguments username and password from None to "", as with token-based authentication they don't need to be specified.
Added new argument target to gNMIclient.get(), gNMIclient.set(), and gNMIclient.subscribe2() methods. If provided, it adds target key to Path() per GNMI Specification 2.2.2.1.

Release 0.7.0:

Added authentication with Token using Authorization: Bearer TOKEN, where TOKEN is a variable provided as gNMIclient(token=TOKEN) key (needed for Arista CVP).
Added functionality to change GRPC_SSL_CIPHER_SUITES dynamically to HIGH value (needed for Nokia SR OS).

Release 0.6.9:

Adding new documentation for mutual TLS feature.

Release 0.6.8:

Minor bug-fixing.

Release 0.6.7:

Added new show_diff key to gNMIclient object (supported values print and get). When applied, it shows the changes happened to all keys following XPath from all arguments to Set() RPC at the network devices. It is so fair tailored to OpenConfig YANG modules as it uses some architectural principles of OpenConfig YANG module to re-construct XPath.
Added an optional timeout to connect() method.
Minor bug-fixing.

Release 0.6.6:

Minor bug-fixing.

Release 0.6.5:

Implemented prefix and timestamp in SetResponse message.
Implemented alias and atomic in Notification message.
Minor bug-fixing.

Release 0.6.4:

Minor bug-fixing.

Release 0.6.3:

Implemented prefix key in the Update message.
Added possibility to provide password in STDIN rather than key.
Minor bug-fixing.

Release 0.6.2:

Added support of keepalive timer for gRPC session to prevent automatic closure each 2 hours.
Fixed issue with Subscribe RPC not sending delete notification in case of a path is removed from the node.
Added the CLI based tool.
Minor bug-fixing.

Release 0.6.1:

Added support of origin per RFC7951.
Added timeout to the initial setup useful for long-living connections.
Minor bug-fixing.

Release 0.6.0:

Significant improvements in telemetry capabilities of the pygnmi. Now you can use subscribe2 method by simply providing the a corredponding dictionary at input and all modes (STREA, ONCE, POLL) are working correctly.
Function telemetryParser is now automatically used inside subscribe2.
Telemetry is now implemeted using threading.
Added new unit tests with pytest and added code coverage with coverage.py.

Release 0.5.3:

Minor improvements and bug fixing.
Full coverage of unit tests for all operations (Capabilities, Get, Set(Update, Replace, Delete), Subscribe) and all notations of GNMI Path.

Release 0.5.2:

Minor bug fixing.
First release with unit tests.

Release 0.5.1:

Added example for non-blocking iterator for telemetry.
Added the extra support for Juniper TLS certificates.
Fixed regexp warnings.
Changed the logging functionality.
Enabled Unix domain socket.
Added close()
Many thanks for all contributors to make this release happen.

Release 0.5.0:

Added possibility to extract certificate from the destination network function.

Release 0.4.8:

Added documentation in module regading supported the different paths naming conventions. Supported options: yang-module:container/container[key=value], /yang-module:container/container[key=value], /yang-module:/container/container[key=value], /container/container[key=value]

Release 0.4.6:

Fixed gNMI Path issue.

Release 0.4.6:

Replaced the sys.exit with raising exceptions.
Minor bug fix.
Brought the gNMI path to the canonical format: /origin:element1/element2....
Added possibility to omit the YANG module name, as some vendors doesn't include that in the request per their gNMI implementation: /element1/element2....

Release 0.4.5:

Minor bug fix.

Release 0.4.4:

Minor bug fix.

Release 0.4.3:

Added possibility to modify the timeout (default value is 5 seconds) for the session using gnmi_timeout key for gNMIclient class.

Release 0.4.2:

Modified the path generation to comply with gNMI Path encoding conventions.
Fixed the problem debug output, where the requests where not printed in case of response failing.

Release 0.4.1:

Minor bug fix.

Release 0.4.0:

Added support for Juniper JUNOS
Fixed the issue with override for PKI-based certificates

Release 0.3.12:

Minor bug fix.

Release 0.3.11:

Minor bug fix.

Release 0.3.10:

Renamed the debug mode. Add argument debug=True upon object creation to see the Protobuf messages.

Release 0.3.9:

Added functionality to list the full the device configuration in case the path is empty: get(path[]).

Release 0.3.8:

Merged the proposal how to implement TLS with override for Cisco IOS XR (tested for Cisco IOS XR, to be tested for other vendors yet)
Merged examples with TLS

Release 0.3.7:

Added the argument encoding as an extra key to Set operation

Release 0.3.6:

Added the argument encoding to Get operation

Release 0.3.5:

Added the example for Nornir Integration
Added the topology diagram
Added links to the video tutorial

Release 0.3.4:

Added the close method to gNMIClient class for those, who doesn't use with ... as ... context manager.

Release 0.3.3:

Added the functionality to pass gRPC messages to the code execution

Release 0.3.2:

Minor bugs fixed.

Release 0.3.1:

Minor bugs fixed.
Added examples of gNMI operations.

Release 0.3.0:

Added new function telemetryParser, which converts Protobuf messages in Python dictionary.
Fixed the errors with the telemetry parsing.

Release 0.2.7:

Modified core so that telemetry is working in once and stream mode.

Release 0.2.6:

Added alpha version of the Subscribe operation.

Release 0.2.5:

Added typing hints.

Release 0.2.4:

Minor bugfixing.

Release 0.2.3:

Added support for IPv6 transport (now you can connect to the network function over IPv6).

Release 0.2.2:

Added conversion of the collected information over the gNMI into a Python dictionary for Set operation.

Release 0.2.1:

Fixing the bugs with improper Protobuf paths generation.
Now all Set operations (delete, replace, and update) are working properly.

Releast 0.2.0:

Added the Set operation from gNMI specification.

Releast 0.1.9:

Added the property datatype='all' to the get() request. The values are per the gNMI specification: all, config, state, operatonal.

Release 0.1.8:

Added conversion of the collected information over the gNMI into a Python dictionary for Get operation.

Release 0.1.7:

Changing packages modules.

Release 0.1.6:

Restructuring internal context.

Release 0.1.5:

Minor bugfixing.

Release 0.1.4:

Minor bugfixing.

Release 0.1.3:

Minor bugfixing.

Release 0.1.2:

The gNMIClient is recreated as context manger.
Tests with Nokia SR OS done, the module is working nice for insecure channel.

Release 0.1.1:

Added the Get operation out of gNMI specification.

Release 0.1.0:

The first release.

pygnmi's People

Contributors

Stargazers

Watchers

pygnmi's Issues

Unable to receive streaming responses to Subscribe

The 'Subscribe' request in 'stream' mode is (supposed to be) a long-lived subscription, returning responses from the server as they arrive.

For some reason, the current implementation only seems to return the first response from the server - see https://github.com/akarneliuk/pygnmi/blob/master/pygnmi/client.py#L674

gNMI inactivity timeout?

Hi Anton,
Are you aware of any inactivity timeouts in the code?
I have a gNMI subscription that can go for hours or days without producing a message and it appears disconnect/timeout after 2 hours. See below.

This is a log from the Juniper router. It seems to send its last message at 20:26:22 and then receives a "call cancellation" at 22:26:42 (almost exactly 2 hours) from the gNMI client.

This means that when the next message has to be sent from the Juniper, there is no gNMI subscription in place and the message is lost.

Sep 16 20:26:22 ProcessInPendingCall: Pushing tag for RPC: /gnmi.gNMI/Subscribe with tag id: request_1239761477into input pending call queue
Sep 16 22:26:42 ProcessInPendingCall: Processing pending call for RPC: /gnmi.gNMI/Subscribe with tag id: request_1239761477
Sep 16 22:26:42 ProcessInPendingCall: Received call cancellation for RPC: /gnmi.gNMI/Subscribe
Sep 16 22:26:42 DelCallTag: Tag id request_1239761477 is removed from tag map for client nso
Sep 16 22:26:42 ProcessCall: Removing the cancelled call details for tag: request_261625440
Sep 16 22:26:42 ~GrpcCall:Invoking destructor
Sep 16 22:26:42 DecrRefCount: rpc request is /gnmi.gNMI/Subscribe
Sep 16 22:26:42 DecrRefCount: Reducing Ref count for Package gnmi
Sep 16 22:26:42 ~GrpcToJapi:Invoking destrutor
Sep 16 22:26:42 DelCallTag: Tag id request_261625440 is removed from tag map for client nso
Sep 16 22:26:42 GetPeerConnInfo: getpeername for client fd 15 failed with error: Bad file descriptor
Sep 16 22:26:42 IsClientConnected: getpeername for the client ipv6:::ffff:xxxxxxxxx failed with error: Bad file descriptor
Sep 16 22:26:42 ~GrpcClientManager: Invoking destructor for client nso
Sep 16 22:26:42 ~GrpcClientManager:Termination sent to handler thread
Sep 16 22:26:42 DeleteChannel: Deleting entries in tag map for client ipv6:::ffff:xxxxxx:26647
Sep 16 22:26:42 ~GrpcClientManager: Cleaning out the channel map for client nso
Sep 16 22:26:42 GrpcHandleCall:Grpc handler cq is shutdown
Sep 16 22:26:42 DeleteChannel: Tag map is empty for client ipv6:::ffff:xxxxxx:26647
Sep 16 22:26:42 GrpcHandlerThreadCleanup:Executing grpc handler thread exit handler
Sep 16 22:26:42 DestroyClientInfo: peer ipv6:::ffff:xxxxxxx:26647 deleted from client map
Sep 16 22:26:45 jsd_config_read:Reading configuration...
Sep 16 22:26:45 readGrpcConfig:Inside ConfigManager::readGrpcConfig
Sep 16 22:26:45 readGrpcConfig:grpc ssl server knob is configured
Sep 16 22:26:45 readGrpcConfig: No change in certificate as crc before(32075) and after(32075) match
Sep 16 22:26:45 readGrpcConfig:mutual-authentication knob is configured
Sep 16 22:26:45 readGrpcConfig:request-response grpc is unchanged
Sep 16 22:26:45 readConfig:Notification service is not configured
Sep 16 22:26:45 readConfig:Notification service is not configured
Sep 16 22:26:45 jsd_sensor_config_read:jsd_sensor_config_read Invoked.
Sep 16 22:26:45 jsd_config_read_streaming_servers:No streaming servers available, delete all old servers.
Sep 16 22:26:45 jsd_config_read_export_profiles:No export profiles to read..
Sep 16 22:26:45 jsd_config_read_sensors:No sensors avaialble to read, delete all old sensors.
Sep 16 22:26:45 jsd_config_read_sensors:DAX walk failed for jsd_config_single_sensor.

pygnmi 0.7.1 reporting an incorrect version (0.7.0)

arista@devbox:~$ pip freeze | grep pygnmi
pygnmi==0.7.1

arista@devbox:~$ pip show pygnmi
Name: pygnmi
Version: 0.7.1
Summary: Pure Python gNMI client to manage network functions and collect telemetry.
Home-page: https://github.com/akarneliuk/pygnmi
Author: Anton Karneliuk
Author-email: [email protected]
License: bsd-3-clause
Location: /home/arista/.local/lib/python3.9/site-packages
Requires: dictdiffer, cryptography, grpcio-tools, grpcio
Required-by:

arista@devbox:~$ cat test2.py 
from pprint import pprint as pp
import json
from pygnmi.client import gNMIclient

with open("token.tok") as f:
    TOKEN = f.read().strip('\n')

host = ('192.168.0.5', '443')
with gNMIclient(target=host, token=TOKEN, skip_verify=False) as gc:
    result = gc.capabilities()

pp(result['gnmi_version'])

arista@devbox:~$ python test2.py 
'0.7.0'
arista@devbox:~$

Is it expected?

Nokia: minor mgmt_core: Unknown element

Hi there! First of all, thanks for this library: it looks awesome!

You know I've been trying what I think is the simplest example, as you show in the main page of the project:

# modules
from pygnmi.client import gNMIclient, telemetryParser

# variables
host =    {
        "ip_address": "192.168.122.12",
        "nos": "nokia-sros",
        "port": 57400,
        "username": "admin",
        "password": "admin",
    }

paths = ['openconfig-interfaces:show router interface']

with gNMIclient(
        target   = (host['ip_address'],host['port']), 
        username = host['username'],
        password = host['password'],
        insecure = True
        ) as gc:
    result = gc.get(path=paths)

After that, I get an error:

Host: 192.168.122.12:57400
Error: MINOR: MGMT_CORE #2201: /show router interface - Unknown element
CRITICAL:root:Host: 192.168.122.12:57400, Error: MINOR: MGMT_CORE #2201: /show router interface - Unknown element
ERROR:root:Internal Python error in the inspect module.
Below is the traceback from this internal error.

In the router I do have grpc enabled, and so gnmi.

A:rtr2# show system grpc       
===============================================================================
gRPC Server
===============================================================================
Administrative State      : Enabled
Operational State         : Up

Supported services
-------------------------------------------------------------------------------
gNMI Version              : 0.4.0
RibApi Version            : 1.0.0

Nevertheless my guess is there is something wrong either with the path variable of some configuration missing in the router. All I want to do is to get the list of interfaces as you can see in the path variable.

Could you help me with this? Thanks!!

Lucas

pygnmi doesn't send anything over the wire

Test code:

from pygnmi.client import gNMIclient
if name == 'main':
host = ("172.16.18.9", "57400")
paths = ['openconfig-interfaces:interfaces', 'openconfig-network-instance:network-instances']
with gNMIclient(target=host, username="xxxx", password="xxxxxxx", insecure=True) as gc:
response = gc.capabilities()
print (response)

When I run:

(virtual3.9) root@squad3_1:~/virtual3.9# python pygnmitest.py
Traceback (most recent call last):
File "/root/virtual3.9/pygnmitest.py", line 7, in
with gNMIclient(target=host, username="xxxx", password="xxxxxxx", insecure=True) as gc:
File "/root/virtual3.9/lib/python3.9/site-packages/pygnmi/client.py", line 94, in enter
return self.connect()
File "/root/virtual3.9/lib/python3.9/site-packages/pygnmi/client.py", line 146, in connect
self.wait_for_connect(timeout)
File "/root/virtual3.9/lib/python3.9/site-packages/pygnmi/client.py", line 156, in wait_for_connect
grpc.channel_ready_future(self.__channel).result(timeout=timeout)
File "/root/virtual3.9/lib/python3.9/site-packages/grpc/_utilities.py", line 139, in result
self._block(timeout)
File "/root/virtual3.9/lib/python3.9/site-packages/grpc/_utilities.py", line 85, in _block
raise grpc.FutureTimeoutError()
grpc.FutureTimeoutError

I have tried this in a python 3.8 and 3.9 venv on 2 different Ubuntu VMs. Same result.

If I tcpdump the NIC, I see no traffic to the host on port 57400.

Thanks,
Serge

Long running connections and is_alive method

Hi,

Thank you for creating this great library.

I am in the process of integrating it with Salt Nornir Proxy Minion and planning to run long-live connections to devices, as a result would be good to have liveability detection of the connections, something like is_alive method.

To be honest, not sure if long-running connection is a good way to use gNMI protocol. To your opinion, are there any potential issues with that or it is better to use gNMI similar to RESTCONF - each set/get request is atomic and reinitiates connection to device using with ... as ..: constructs?

use yang prefix in first path element?

Hi
looking at this line it seems impossible to specify the path element to contain a yang prefix, as the element before : will be promoted to origin.

Consider the following OC path that specifies a fully qualified top level element by setting yang prefix in the first element:

/oc-netinst:network-instances/...

In this path oc-netinst is a yang prefix of a module and not the origin. It is expected that this path will translate to a path with a first path element to be oc-netinst:network-instances with no origin set.

Problem connecting to SROS device when using encryption

I have the same problem as the user had I case #59.
When I look at a wireshark of the data that is send with gnmic that is working it is using tls 1.2, but when I use pygnmi it is using TLS 1.

It works when I add the os value in case #59 (os.environ["GRPC_SSL_CIPHER_SUITES"] = "HIGH"), but is there not a way where pygnmi can negotiate the tls version with the devices, so we do not have to change this for what device we are talking to?

Error getting capabilities from gnxi-simulators

I am trying to test with gnmi simulators. I am getting below error. Same simulator returns successful response with gnmi_cli command line request.

Simulator is from https://github.com/onosproject/gnxi-simulators
gnmi simulator is running in docker on port 10162.
Here is the code I am using to get capabilities.

-----code starts here --->
#!/usr/bin/env python
//# Modules
from pygnmi.client import gNMIclient
///# Body
if name == "main":
gc = gNMIclient(target=("127.0.0.1", 10162), username="admin",password="admin", insecure=True)
result = gc.capabilities()
print(f"127.0.0.1 : {result}\n\n")
----- code ends here --->

--- Error message starts here -----

$ python3 capabilities.py
Traceback (most recent call last):
File "/home/test/.local/lib/python3.8/site-packages/pygnmi/client.py", line 181, in capabilities
gnmi_message_response = self.__stub.Capabilities(gnmi_message_request, metadata=self.__metadata)
AttributeError: 'gNMIclient' object has no attribute '_gNMIclient__stub'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
File "capabilities.py", line 10, in
result = gc.capabilities()
File "/home/test/.local/lib/python3.8/site-packages/pygnmi/client.py", line 222, in capabilities
except grpc._channel._InactiveRpcError as err:
AttributeError: module 'grpc' has no attribute '_channel'

pygnmi subscribe not sending delete notification in case of a path is removed from the node

Hi @akarneliuk
Hope you are doing good !
I was testing EDT for oc-platform and I am seeing that pygnmi subscribe is not sending delete field for a path removed from node and is only sending the updates . can we fix this ?

https://github.com/openconfig/reference/blob/master/rpc/gnmi/gnmi-specification.md#3523-sending-telemetry-updates

RP/0/RP0/CPU0:BH1_P2A5#0/RP0/ADMIN0:Apr  6 00:39:50.782 IST: bh_cardmgr[2655]: %INFRA-BH_CARDMGR_DRIVER-5-PLUGGABLE_REMOVAL : Location: 0/0/0/2, Serial #: INL20190037  
RP/0/RP0/CPU0:Apr  6 00:39:50.787 IST: osa_driver[251]: %PKT_INFRA-FM-4-FAULT_MINOR : ALARM_MINOR :PROV-INPROGRESS :DECLARE :HundredGigECtrlr0/0/0/2:  
RP/0/RP0/CPU0:Apr  6 00:39:51.259 IST: osa_driver[251]: %PKT_INFRA-FM-4-FAULT_MINOR : ALARM_MINOR :PROV-INPROGRESS :CLEAR :HundredGigECtrlr0/0/0/2:  
RP/0/RP0/CPU0:Apr  6 00:39:53.799 IST: osa_driver[251]: %PKT_INFRA-FM-2-FAULT_CRITICAL : ALARM_CRITICAL :IMPROPRMVL :DECLARE :Optics0/0/0/2:  
RP/0/RP0/CPU0:Apr  6 00:39:53.800 IST: osa_driver[251]: %PKT_INFRA-FM-3-FAULT_MAJOR : ALARM_MAJOR :SYNCLOSS :DECLARE :HundredGigECtrlr0/0/0/2:  


{'sync_response': True}

{'update': {'update': [{'path': 'state/name', 'val': '0/0-OpticsContainer0/0/0/2'}, {'path': 'state/type', 'val': 'openconfig-platform-types:PORT'}, {'path': 'state/id', 'val': '8718'}, {'path': 'state/location', 'val': '0/0'}, {'path': 'state/description', 'val': 'Pluggable Optical Module Container'}, {'path': 'state/removable', 'val': False}, {'path': 'state/empty', 'val': True}, {'path': 'state/parent', 'val': '0/0-OpticsCtlr0/0/0/2'}, {'path': 'state/openconfig-platform-ext:entity-id', 'val': 8718}], 'timestamp': 1617649790785000000, 'prefix': 'components/component[name=0/0-OpticsContainer0/0/0/2]'}}
{'update': {'update': [], 'timestamp': 1617649790788000000, 'prefix': 'components/component[name=0/0-Optics0/0/0/2]'}}
{'update': {'update': [{'path': 'state/name', 'val': '0/0-OpticsContainer0/0/0/2'}, {'path': 'state/type', 'val': 'openconfig-platform-types:PORT'}, {'path': 'state/id', 'val': '8718'}, {'path': 'state/location', 'val': '0/0'}, {'path': 'state/description', 'val': 'Pluggable Optical Module Container'}, {'path': 'state/removable', 'val': False}, {'path': 'state/empty', 'val': False}, {'path': 'state/parent', 'val': '0/0-OpticsCtlr0/0/0/2'}, {'path': 'state/openconfig-platform-ext:entity-id', 'val': 8718}], 'timestamp': 1617649810870000000, 'prefix': 'components/component[name=0/0-OpticsContainer0/0/0/2]'}}
{'update': {'update': [{'path': 'state/name', 'val': '0/0-Optics0/0/0/2'}, {'path': 'state/type', 'val': 'openconfig-platform-types:TRANSCEIVER'}, {'path': 'state/id', 'val': '90713'}, {'path': 'state/location', 'val': '0/0'}, {'path': 'state/description', 'val': 'Cisco 100G QSFP28 LR4-S Pluggable Optics Module'}, {'path': 'state/mfg-name', 'val': 'CISCO-INNOLIGHT'}, {'path': 'state/hardware-version', 'val': 'ES1'}, {'path': 'state/serial-no', 'val': 'INL20190037'}, {'path': 'state/part-no', 'val': '10-3146-01'}, {'path': 'state/removable', 'val': True}, {'path': 'state/oper-status', 'val': 'openconfig-platform-types:ACTIVE'}, {'path': 'state/empty', 'val': False}, {'path': 'state/parent', 'val': '0/0-OpticsContainer0/0/0/2'}, {'path': 'state/allocated-power', 'val': 7}, {'path': 'state/openconfig-platform-ext:entity-id', 'val': 90713}], 'timestamp': 1617649810870000000, 'prefix': 'components/component[name=0/0-Optics0/0/0/2]'}}

gNMI and path with both origin and YANG namespaces

Hi @akarneliuk ,

I need to generate paths that have both an origin, and a YANG namespace in the top-level element (and in fact, in subsequent elements too). Those paths look like the second example in https://www.cisco.com/c/en/us/td/docs/ios-xml/ios/prog/configuration/1610/b_1610_programmability_cg/gnmi_protocol.html#id_84819:

path {
  origin: "rfc7951"
  elem {
    name: "openconfig-interface:interfaces"
  }
  elem {
    name: "interface"
    key {
      key: "name"
      value: "Loopback111"
    }
  }
}

The current code in pygnmi can't generate the path above, due to the way it parses path elements that contain :.

Unfortunately, Schema path encoding conventions for gNMI doesn't specify how to encode the origin of the path. One possible way to do this is to look at the first path element only, and if that element contains a :, then set the origin from that. The XPATH for the path I quoted above would look like: rfc7951:openconfig-interfaces:interfaces/interface[name=Loopback111], or (maybe more explicit): rfc7951:/openconfig-interfaces:interfaces/interface[name=Loopback111]. I have some code that does just that, see the draft PR #41 I'm opening at the same time. The problem with that approach is that this doesn't quite conform to XPATH specification.

Another possible way is to add an extra optional argument to get, subscribe, etc., that specifies the origin. If that argument is unset, then guess the origin from the XPATH (ie. current behaviour). If that argument is set, then it is used for the gNMI path, and : in the path elements is not treated specially. Ie. to get the path I quoted above, you would have to call gnmi_path_generator("/openconfig-interfaces:interfaces/interface[name=Loopback111]", origin="rfc7951"). The downside of this approach is that it requires more changes and the API becomes a bit more complex. ~~I haven't tried to write that approach yet~~ Draft of that second approach: https://github.com/fperrin/pygnmi/commits/fp-gnmi-path-origin

What do you think?

val.uint_val values are not returned in pygnmi response, when using get

Hi Anton,
I am encountering an issue, when using get with an path which returns val.uint_val.
For val.json_ietf_val everything works for fine, but is just ignored.

gNMI response:

notification {
update {
path {
elem {
name: "interfaces"
}
elem {
name: "interface"
key {
key: "name"
value: "Management1"
}
}
elem {
name: "state"
}
elem {
name: "counters"
}
elem {
name: "out-unicast-pkts"
}
}
val {
uint_val: 15578
}
}
}

INFO||tE4_gnmiConnect.py||136||gnmi response {'notification': [{'timestamp': 0, 'update': [{'path': 'interfaces/interface[name=Management1]/state/counters/out-unicast-pkts'}]}]}

looking at the code, it looks to me, that uint_val is missing for get:

                            if update_msg.HasField('val'):
                                if update_msg.val.HasField('json_ietf_val'):
                                    update_container.update({'val': json.loads(update_msg.val.json_ietf_val)})

                                elif update_msg.val.HasField('json_val'):
                                    update_container.update({'val': json.loads(update_msg.val.json_val)})

                                elif update_msg.val.HasField('ascii_val'):
                                    update_container.update({'val': json.loads(update_msg.val.ascii_val)})

                                elif update_msg.val.HasField('bytes_val'):
                                    update_container.update({'val': json.loads(update_msg.val.bytes_val)})

                                elif update_msg.val.HasField('proto_bytes'):
                                    update_container.update({'val': json.loads(update_msg.val.proto_bytes)})

where asm it seems to be corrrect for the telemetry parser.

I think we have to copy the lines for those values from telemetry parser to the get section. I can take over that task and prepare a pull request.

what do you think?

pygnmi subscribe request API does not allow for multiple streaming requests

As explained at https://github.com/cisco-ie/cisco-gnmi-python/blob/master/src/cisco_gnmi/client.py#L235 the 'subscribe' request is a streaming request - multiple subscribe requests from the client can be queued

With the current pygnmi code structure, this is not possible

Failing to install pygnmi on CentOS with 'ascii' codec can't decode error

Hi,

As stated in subject, failing to install pygnmi on Centos with Python 3.6.8 with this error:

[root@host /]# python3 -m pip install pygnmi --upgrade
Requirement already satisfied: pygnmi in /usr/local/lib/python3.6/site-packages (0.5.3)
Collecting pygnmi
  Using cached pygnmi-0.6.7.tar.gz (29 kB)
  Preparing metadata (setup.py) ... error
  ERROR: Command errored out with exit status 1:
   command: /usr/bin/python3 -c 'import io, os, sys, setuptools, tokenize; sys.argv[0] = '"'"'/tmp/pip-install-vttklldz/pygnmi_6a28cc3abbfb4c57a7bb4ef96aa200a1/setup.py'"'"'; __file__='"'"'/tmp/pip-install-vttklldz/pygnmi_6a28cc3abbfb4c57a7bb4ef96aa200a1/setup.py'"'"';f = getattr(tokenize, '"'"'open'"'"', open)(__file__) if os.path.exists(__file__) else io.StringIO('"'"'from setuptools import setup; setup()'"'"');code = f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, __file__, '"'"'exec'"'"'))' egg_info --egg-base /tmp/pip-pip-egg-info-s5mrssho
       cwd: /tmp/pip-install-vttklldz/pygnmi_6a28cc3abbfb4c57a7bb4ef96aa200a1/
  Complete output (7 lines):
  Traceback (most recent call last):
    File "<string>", line 1, in <module>
    File "/tmp/pip-install-vttklldz/pygnmi_6a28cc3abbfb4c57a7bb4ef96aa200a1/setup.py", line 4, in <module>
      long_description = fh.read()
    File "/usr/lib64/python3.6/encodings/ascii.py", line 26, in decode
      return codecs.ascii_decode(input, self.errors)[0]
  UnicodeDecodeError: 'ascii' codec can't decode byte 0xc3 in position 1864: ordinal not in range(128)
  ----------------------------------------
WARNING: Discarding https://files.pythonhosted.org/packages/7e/51/0985a330bd13318518f391edbf8797b7dbbf2003e3ff78161a197a02a295/pygnmi-0.6.7.tar.gz#sha256=b346c43ef1780badf420cb7f06827a6ce6b555fa3af0f21d286be551badb2868 (from https://pypi.org/simple/pygnmi/). Command errored out with exit status 1: python setup.py egg_info Check the logs for full command output.

Was able to fix the issue by specifying the encoding to use for opening readme file in setup.py:

[root@host pygnmi]# cat setup.py 
from distutils.core import setup

with open('README.rst', encoding="utf-8") as fh:
    long_description = fh.read()

after adding encoding="utf-8" can install with no issues:

[root@host pygnmi]# python3 -m pip install .
Processing /root/pygnmi
  Preparing metadata (setup.py) ... done
Requirement already satisfied: grpcio in /usr/local/lib64/python3.6/site-packages (from pygnmi==0.6.7) (1.43.0)
Requirement already satisfied: grpcio-tools in /usr/local/lib64/python3.6/site-packages (from pygnmi==0.6.7) (1.43.0)
Requirement already satisfied: kthread in /usr/local/lib/python3.6/site-packages (from pygnmi==0.6.7) (0.2.2)
Requirement already satisfied: cryptography in /usr/local/lib64/python3.6/site-packages (from pygnmi==0.6.7) (36.0.1)
Collecting dictdiffer
  Downloading dictdiffer-0.9.0-py2.py3-none-any.whl (16 kB)
Requirement already satisfied: cffi>=1.12 in /usr/local/lib64/python3.6/site-packages (from cryptography->pygnmi==0.6.7) (1.15.0)
Requirement already satisfied: six>=1.5.2 in /usr/local/lib/python3.6/site-packages (from grpcio->pygnmi==0.6.7) (1.16.0)
Requirement already satisfied: setuptools in /usr/local/lib/python3.6/site-packages (from grpcio-tools->pygnmi==0.6.7) (59.6.0)
Requirement already satisfied: protobuf<4.0dev,>=3.5.0.post1 in /usr/local/lib64/python3.6/site-packages (from grpcio-tools->pygnmi==0.6.7) (3.19.1)
Requirement already satisfied: pycparser in /usr/local/lib/python3.6/site-packages (from cffi>=1.12->cryptography->pygnmi==0.6.7) (2.21)
Building wheels for collected packages: pygnmi
  Building wheel for pygnmi (setup.py) ... done
  Created wheel for pygnmi: filename=pygnmi-0.6.7-py3-none-any.whl size=32631 sha256=faa39de9c4938710810842707e4ca93d897f980fa8e6e99b85ce00ec76f2126b
  Stored in directory: /tmp/pip-ephem-wheel-cache-ixnhdp84/wheels/bc/83/38/7f21b3a28050665ce96fec2ff33c00c61381e6e1f71ce7df3b
Successfully built pygnmi
Installing collected packages: dictdiffer, pygnmi
  Attempting uninstall: pygnmi
    Found existing installation: pygnmi 0.5.3
    Uninstalling pygnmi-0.5.3:
      Successfully uninstalled pygnmi-0.5.3
Successfully installed dictdiffer-0.9.0 pygnmi-0.6.7
WARNING: Running pip as the 'root' user can result in broken permissions and conflicting behaviour with the system package manager. It is recommended to use a virtual environment instead: https://pip.pypa.io/warnings/venv
[root@host pygnmi]#

Unable to connect to Unix domain socket

At https://github.com/akarneliuk/pygnmi/blob/master/pygnmi/client.py#L61 the code assumes the target consists of a host:port string.

I have an application that needs to connect locally, as in grpc.insecure_channel('unix:///some/socket/path')
I don't control the server path, so I cannot make it look like '/x/y/z:port'

grpc.FutureTimeoutError Failure

Hi team,

Please find the gnmic capabilities of my NVDIA switch and the script . When I execute i got the below failure but gnmic manual execution is working.

Can you please help?

gNMI enabled
Port 9339
mode using "--skip-verify capabilities"

gnmic Capabilities:

kasinath@npbsrv01:~$ gnmic -a :9339 -u cumulus -p aviz123 --skip-verify capabilities
gNMI version: 0.7.0
supported models:

nvidia-wjh, NVIDIA, 1.0.1
nvidia-if-ethernet-counters-ext, NVIDIA, 1.0.0
openconfig-interfaces, OpenConfig, 2.3.2
openconfig-if-ethernet, OpenConfig, 2.9.0
openconfig-if-ethernet-ext, OpenConfig, 0.1.1
openconfig-system, OpenConfig, 0.5.0
openconfig-lldp, OpenConfig, 0.2.1
openconfig-platform, OpenConfig, 0.13.0
supported encodings:
JSON

kasinath@npbsrv01:~$

gnmic Version:

root@sqa01:/home/kasinath/Automation# gnmic version
version : 0.26.0
commit : ba387ba
date : 2022-06-29T06:52:10Z
gitURL : https://github.com/karimra/gnmic
docs : https://gnmic.kmrd.dev
root@sqa01:/home/kasinath/Automation#

Script:

root@sqa01:/home/kasinath/Automation# more test_gnmi.py
#!/usr/bin/env python3

Modules

from pygnmi.client import gNMIclient

Variables

host1 = {
"ip_address": "",
"port": 9339,
"username": "cumulus",
"password": "aviz123"
}

Body

if name == "main":
with gNMIclient(target=(host1["ip_address"], host1["port"]), username=host1["username"], password=host1["password"], insecure=True) as
gc:
result = gc.capabilities()

    print(result)

root@sqa01:/home/kasinath/Automation#

Execution failure:

root@sqa01:/home/kasinath/Automation# python3 test_gnmi.py
Traceback (most recent call last):
File "test_gnmi.py", line 17, in
with gNMIclient(target=(host1["ip_address"], host1["port"]), username=host1["username"], password=host1["password"], insecure=True) as gc:
File "/usr/local/lib/python3.8/dist-packages/pygnmi/client.py", line 103, in enter
return self.connect()
File "/usr/local/lib/python3.8/dist-packages/pygnmi/client.py", line 225, in connect
self.wait_for_connect(timeout)
File "/usr/local/lib/python3.8/dist-packages/pygnmi/client.py", line 235, in wait_for_connect
grpc.channel_ready_future(self.__channel).result(timeout=timeout)
File "/usr/local/lib/python3.8/dist-packages/grpc/_utilities.py", line 139, in result
self._block(timeout)
File "/usr/local/lib/python3.8/dist-packages/grpc/_utilities.py", line 85, in _block
raise grpc.FutureTimeoutError()
grpc.FutureTimeoutError
root@sqa01:/home/kasinath/Automation#

pygnmi installation successful:

root@sqa01:/home/kasinath# pip install pygnmi
Collecting pygnmi
Downloading pygnmi-0.8.8.tar.gz (30 kB)
Requirement already satisfied: cryptography in /usr/lib/python3/dist-packages (from pygnmi) (2.8)
Collecting dictdiffer
Downloading dictdiffer-0.9.0-py2.py3-none-any.whl (16 kB)
Collecting grpcio
Downloading grpcio-1.47.0-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (4.5 MB)
|████████████████████████████████| 4.5 MB 39.9 MB/s
Collecting protobuf
Downloading protobuf-4.21.5-cp37-abi3-manylinux2014_x86_64.whl (408 kB)
|████████████████████████████████| 408 kB 41.2 MB/s
Requirement already satisfied: six>=1.5.2 in /usr/lib/python3/dist-packages (from grpcio->pygnmi) (1.14.0)
Building wheels for collected packages: pygnmi
Building wheel for pygnmi (setup.py) ... done
Created wheel for pygnmi: filename=pygnmi-0.8.8-py3-none-any.whl size=32744 sha256=5c388dfec571999d5975fcb12f22e2c99db55a56aaaca153d168930b3183d3b2
Stored in directory: /root/.cache/pip/wheels/d5/56/76/e2698d9a887f61437c7e46940e75b062a022bff4608c6a7471
Successfully built pygnmi
Installing collected packages: dictdiffer, grpcio, protobuf, pygnmi
Successfully installed dictdiffer-0.9.0 grpcio-1.47.0 protobuf-4.21.5 pygnmi-0.8.8
root@sqa01:/home/kasinath#

Thanks
Kasi

Default to json_ietf if the target only supports that

I am working with IOS-XR 7.2.2. It doesn't accept the "json" encoding, it requires "json_ietf". This is clear from the capabilities response:

>>> gc.capabilities()["supported_encodings"]
['json_ietf', 'ascii', 'proto']

However, if I don't specify the encoding explicitly, pygnmi still chooses json by default, and the request fails:

>>> gc.get(path=['openconfig-interfaces:interfaces'])
GRPC ERROR Host: 192.168.0.2:57400, Error: gNMI: unsupported get-request encoding: JSON
Traceback (most recent call last):
...

Therefore, I have to set the encoding explicitly on every request, which is inconvenient (and easy to forget):

         result = gc.get(path=['openconfig-interfaces:interfaces'],
                         encoding='json_ietf')

AFAICS from source, get() has a hard-coded default of json for the encoding argument.

There is a choose_encoding() function, but any explicit value of requested_encoding passed through takes precedence - therefore, the encoding json will always be chosen.

I think this can be improved. Suggestion:

in get(), make the default encoding argument argument None
in choose_encoding, choose whichever of 'json' or 'json_ietf' exists in the capabilities

To be backwards-compatible, you'd need to prefer 'json' over 'json_ietf'. Perhaps pass a list of default_encodings to choose_encoding, instead of a single value? Try them in turn, and pick the first that is supported.

--insecure vs --skip-verify and options inside gNMIclient

greetings! I am trying to use pygnmi for connecting to a cisco Nexus 9000 series switch. and with the gnmic tool for command line i can get my commands to work as long as i use "--skip-verify" in my command string. I am trying to accomplish the same thing in a python script and am running into errors. In the client in the library, there doesn't appear to be an equivalent to "--skip-verify" as an arg in the client function.

and that brings me to my question. in the case of this. Does "insecure=True" act the same as "--skip-verify" in the gnmic tool?

i guess what i am really looking for is how, in the python client, do i also do untrusted TLS command as an arg?

i.e. working command for cisco 9k switch:
gnmic -a 192.168.1.100:6030 -u admin -p admin --skip-verify get --path /lldp/state/enabled

i.e. NOT working command for cisco 9k switch:
gnmic -a 192.168.1.100:6030 -u admin -p admin --insecure get --path /lldp/state/enabled

and here is my python snippet:

with gNMIclient(target=host, username='admin', password='admin',insecure=True, gnmi_timeout=2) as gc:

also this library is truly amazing, it was a no brainer to start using this as my automation go to for my growing presence of grpc stuff

keep up the great work!!!

Code assumes the use of 'with', should make this clear in the examples

The 'with' construct in Python calls the enter and exit methods. The library assumes that, to a point that things only work using with: ...

grpc.FutureTimeoutError()

Hi @akarneliuk ,

This could quite possibly be me doing something wrong as I am new to gNMI configuration. Nevertheless, I get the following error upon trying a basic get request using pygnmi. Here are some of my details.

Target: Nokia SR Linux which supports gNMI.
local host : 
(gnmi) [root@grpc gnmi]# python3 --version
Python 3.6.8
(gnmi) [root@grpc gnmi]# python3 -m pip list
Package                  Version
------------------------ ---------
cachetools               4.2.1
certifi                  2020.12.5
chardet                  4.0.0
google                   3.0.0
google-api-core          1.26.2
google-api-python-client 2.0.2
google-auth              1.28.0
google-auth-httplib2     0.1.0
googleapis-common-protos 1.53.0
grpcio                   1.36.1
grpcio-tools             1.36.1
httplib2                 0.19.0
idna                     2.10
packaging                20.9
pip                      21.0.1
protobuf                 3.15.6
pyasn1                   0.4.8
pyasn1-modules           0.2.8
pygnmi                   0.4.5
pyparsing                2.4.7
pytz                     2021.1
requests                 2.25.1
rsa                      4.7.2
setuptools               54.1.2
six                      1.15.0
soupsieve                2.2.1
uritemplate              3.0.1
urllib3                  1.26.4
wheel                    0.36.2

The error is as follows:-

(gnmi) [root@grpc gnmi]# python3
Python 3.6.8 (default, Nov 16 2020, 16:55:22)
[GCC 4.8.5 20150623 (Red Hat 4.8.5-44)] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> from pygnmi.client import gNMIclient
>>> host = ('172.20.20.19', '57400')
>>> with gNMIclient(target=host, username='admin', password='admin', insecure=True) as gc:
...     result = gc.get(path=['interface'])
...
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/root/gnmi/lib/python3.6/site-packages/pygnmi/client.py", line 51, in __enter__
    grpc.channel_ready_future(self.__channel).result(timeout=self.__gnmi_timeout)
  File "/root/gnmi/lib64/python3.6/site-packages/grpc/_utilities.py", line 140, in result
    self._block(timeout)
  File "/root/gnmi/lib64/python3.6/site-packages/grpc/_utilities.py", line 86, in _block
    raise grpc.FutureTimeoutError()
grpc.FutureTimeoutError

I can hit the same target using the openconfig gnmi_get cli which is written in go.

token auth

Hello
would like to use pygnmi with token authentication but I dont see this option?
token based auth https://grpc.io/docs/guides/auth/
here's the implementations of 2 gnmi clients karimra/gnmic#412
Thank you!

Stop publishing `dist` packages under git VCS

I see your git repository has dist directory containing various distributions of this package.
My advice is to stop doing this. There is no reason for keeping distributions under version control at all.
pypi is supposed to serve all the versions of this package, not git.
If you want to keep "old" versions - use tags, e.g. git tag 0.6.3. If you need to create an old package, you can check out the repository from a specific version and then just call setup.py sdist or bdist_wheel.

breaking change in 0.8

Hello @akarneliuk

arista@devbox:~$ cat t3.py 
"""
pygnmi script
"""
from pygnmi.client import gNMIclient

TELEMETRY_REQUEST = {
                         'subscription': [
                             {
                                 'path': '/inventory/state/device/device-id'
                             }
                         ],
                         'mode': 'once',
                         'encoding': 'json'
                     }

with open("token.tok") as f:
    TOKEN = f.read().strip('\n')

if __name__ == "__main__":
     with gNMIclient(target=('192.168.0.5', '443'), token=TOKEN, skip_verify=True) as gconn:
         inventory = gconn.subscribe2(subscribe=TELEMETRY_REQUEST)
         for device in inventory:
                print(device)
arista@devbox:~$

working

arista@devbox:~$ pip install pygnmi==0.7.5

arista@devbox:~$ python t3.py 
ssl_target_name_override is applied, should be used for testing only!
{'update': {'update': [{'path': 'inventory/state/device[device-id=leaf4]/device-id', 'val': 'leaf4'}], 'timestamp': 1658653200000000000, 'prefix': ''}}
{'update': {'update': [{'path': 'inventory/state/device[device-id=host2]/device-id', 'val': 'host2'}], 'timestamp': 1658653200000000000, 'prefix': ''}}
{'update': {'update': [{'path': 'inventory/state/device[device-id=spine1]/device-id', 'val': 'spine1'}], 'timestamp': 1658653200000000000, 'prefix': ''}}
{'update': {'update': [{'path': 'inventory/state/device[device-id=leaf2]/device-id', 'val': 'leaf2'}], 'timestamp': 1658653200000000000, 'prefix': ''}}
{'update': {'update': [{'path': 'inventory/state/device[device-id=leaf3]/device-id', 'val': 'leaf3'}], 'timestamp': 1658653200000000000, 'prefix': ''}}
{'update': {'update': [{'path': 'inventory/state/device[device-id=spine2]/device-id', 'val': 'spine2'}], 'timestamp': 1658653200000000000, 'prefix': ''}}
{'update': {'update': [{'path': 'inventory/state/device[device-id=host1]/device-id', 'val': 'host1'}], 'timestamp': 1658653200000000000, 'prefix': ''}}
{'update': {'update': [{'path': 'inventory/state/device[device-id=leaf1]/device-id', 'val': 'leaf1'}], 'timestamp': 1658653200000000000, 'prefix': ''}}
{'update': {'update': [{'path': 'inventory/state/device[device-id=cvx01]/device-id', 'val': 'cvx01'}], 'timestamp': 1658653200000000000, 'prefix': ''}}
{'sync_response': True}

not working

arista@devbox:~$ pip install pygnmi==0.8.2

arista@devbox:~$ python t3.py 
ssl_target_name_override is applied, should be used for testing only!
Exception in thread Thread-5:
Traceback (most recent call last):
  File "/usr/local/lib/python3.9/threading.py", line 973, in _bootstrap_inner
    self.run()
  File "/usr/local/lib/python3.9/threading.py", line 910, in run
    self._target(*self._args, **self._kwargs)
  File "/home/arista/.local/lib/python3.9/site-packages/pygnmi/client.py", line 940, in enqueue_updates
    for update in subscription:
  File "/home/arista/.local/lib/python3.9/site-packages/grpc/_channel.py", line 426, in __next__
    return self._next()
  File "/home/arista/.local/lib/python3.9/site-packages/grpc/_channel.py", line 826, in _next
    raise self
grpc._channel._MultiThreadedRendezvous: <_MultiThreadedRendezvous of RPC that terminated with:
        status = StatusCode.UNIMPLEMENTED
        details = "Extension not supported"
        debug_error_string = "{"created":"@1658653512.927644730","description":"Error received from peer ipv4:192.168.0.5:443","file":"src/core/lib/surface/call.cc","file_line":966,"grpc_message":"Extension not supported","grpc_status":12}"
>
^CTraceback (most recent call last):
  File "/home/arista/t3.py", line 22, in <module>
    for device in inventory:
  File "/home/arista/.local/lib/python3.9/site-packages/pygnmi/client.py", line 1004, in __next__
    result = self.next()
  File "/home/arista/.local/lib/python3.9/site-packages/pygnmi/client.py", line 1016, in next
    return self._next_update(timeout=None)
  File "/home/arista/.local/lib/python3.9/site-packages/pygnmi/client.py", line 1090, in _next_update
    return self._get_one_update(timeout=timeout)
  File "/home/arista/.local/lib/python3.9/site-packages/pygnmi/client.py", line 969, in _get_one_update
    return telemetryParser(self._updates.get(block=True, timeout=timeout))
  File "/usr/local/lib/python3.9/queue.py", line 171, in get
    self.not_empty.wait()
  File "/usr/local/lib/python3.9/threading.py", line 312, in wait
    waiter.acquire()
KeyboardInterrupt

gnmi history extension

Hello
I would like to use pygnmi with these 2 diff use cases
https://github.com/openconfig/reference/blob/master/rpc/gnmi/gnmi-history.md

retrieve data at a specific time in the past
request all updates applied to a data tree between two specified times.

but I dont see these options.
Is it already supported?
Many thanks in advance for your help!

Timeout option to pygnmicli

Small feature request: add a timeout flag to pygnmicli.

It currently uses the fixed default (5 seconds) from the gNMIclient class.

PyGNMI proto files collide with PyATS yang.connector gnmi ?

Hello,

After upgrading PyATS/Genie form version 22.1 to version 22.5 started getting this error while running "ping" api for Cisco IOS XE device:

            Traceback (most recent call last):
              File "/usr/local/lib/python3.9/site-packages/nornir/core/task.py", line 99, in start
                r = self.task(self, **self.params)
              File "/usr/local/lib/python3.9/site-packages/nornir_salt/utils/yangdantic.py", line 47, in wrapper
                return self.function(*args, **kwargs)
              File "/usr/local/lib/python3.9/site-packages/nornir_salt/plugins/tasks/pyats_genie_api.py", line 66, in pyats_genie_api
                result = getattr(device.api, api)(**kwargs)
              File "src/genie/conf/base/api.py", line 176, in genie.conf.base.api.API.__getattr__.wrapper_match
              File "src/genie/conf/base/api.py", line 162, in genie.conf.base.api.API.__getattr__.wrapper_match
              File "src/genie/conf/base/api.py", line 244, in genie.conf.base.api.API.get_api
              File "/usr/lib64/python3.9/importlib/__init__.py", line 127, in import_module
                return _bootstrap._gcd_import(name[level:], package, level)
              File "<frozen importlib._bootstrap>", line 1030, in _gcd_import
              File "<frozen importlib._bootstrap>", line 1007, in _find_and_load
              File "<frozen importlib._bootstrap>", line 986, in _find_and_load_unlocked
              File "<frozen importlib._bootstrap>", line 680, in _load_unlocked
              File "<frozen importlib._bootstrap_external>", line 850, in exec_module
              File "<frozen importlib._bootstrap>", line 228, in _call_with_frames_removed
              File "/usr/local/lib/python3.9/site-packages/genie/libs/sdk/apis/iosxe/mpte_utils/mpte_utils.py", line 8, in <module>
                from yang.connector.gnmi import Gnmi
              File "/usr/local/lib/python3.9/site-packages/yang/connector/__init__.py", line 22, in <module>
                from .gnmi import Gnmi
              File "/usr/local/lib/python3.9/site-packages/yang/connector/gnmi.py", line 16, in <module>
                from cisco_gnmi import ClientBuilder
              File "/usr/local/lib/python3.9/site-packages/cisco_gnmi/__init__.py", line 27, in <module>
                from .client import Client
              File "/usr/local/lib/python3.9/site-packages/cisco_gnmi/client.py", line 30, in <module>
                from . import proto
              File "/usr/local/lib/python3.9/site-packages/cisco_gnmi/proto/__init__.py", line 25, in <module>
                from . import gnmi_pb2_grpc
              File "/usr/local/lib/python3.9/site-packages/cisco_gnmi/proto/gnmi_pb2_grpc.py", line 4, in <module>
                from . import gnmi_pb2 as gnmi__pb2
              File "/usr/local/lib/python3.9/site-packages/cisco_gnmi/proto/gnmi_pb2.py", line 19, in <module>
                from . import gnmi_ext_pb2 as gnmi__ext__pb2
              File "/usr/local/lib/python3.9/site-packages/cisco_gnmi/proto/gnmi_ext_pb2.py", line 19, in <module>
                DESCRIPTOR = _descriptor.FileDescriptor(
              File "/usr/local/lib64/python3.9/site-packages/google/protobuf/descriptor.py", line 1024, in __new__
                return _message.default_pool.AddSerializedFile(serialized_pb)
            TypeError: Couldn't build proto file into descriptor pool!
            Invalid proto descriptor for file "gnmi_ext.proto":
              gnmi_ext.proto: A file with this name is already in the pool.

Found this old github issue:
protocolbuffers/protobuf#3002

Turns out I have PyGNMI==0.6.9 installed on the same system and interesting enough after uninstalling PyGNMI package, PyATS api calls start working fine.

Could it be that proto files created by PyGNMI somehow overlap with proto files created by PyATS yang connector, could we implement some sort of shielding mechanism like put all PyGNMI proto files in dedicated directory as per this comment

At this point we have two options. Use pure python protobuf package, or set package directive in our proto files. I think this covers everything.

pygnmicli sub

Hello @akarneliuk
I am not able to use pygnmicli with subscribe-once subscribe-stream subscribe2
I already provided output examples in issue #85
using subscribe2 on the same setup with Python pygNMI works

sub `once` doesnt close the session

Hello

arista@devbox:~$ pip freeze | grep pyg
pygnmi==0.7.4
arista@devbox:~$

would like to use pygnmi with the sub rpc with once.

arista@devbox:~$ cat test7.py 
"""
pygnmi script
"""
from pygnmi.client import gNMIclient

TELEMETRY_REQUEST = {
                         'subscription': [
                             {
                                 'path': '/inventory/state/device/device-id'
                             }
                         ],
                         'mode': 'once',
                         'encoding': 'json'
                     }

with open("token.tok") as f:
    TOKEN = f.read().strip('\n')

if __name__ == "__main__":
     with gNMIclient(target=('192.168.0.5', '443'), token=TOKEN, skip_verify=True) as gconn:
         inventory = gconn.subscribe2(subscribe=TELEMETRY_REQUEST)
         for device in inventory:
                print(device)

It works
but it doesnt close the session as you can see below ... (so I did CNTL + C to stop it)

rista@devbox:~$ python test7.py 
ssl_target_name_override is applied, should be used for testing only!
{'update': {'update': [{'path': 'inventory/state/device[device-id=cvx01]/device-id', 'val': 'cvx01'}], 'timestamp': 1657867500000000000, 'prefix': ''}}
{'update': {'update': [{'path': 'inventory/state/device[device-id=leaf3]/device-id', 'val': 'leaf3'}], 'timestamp': 1657867500000000000, 'prefix': ''}}
{'update': {'update': [{'path': 'inventory/state/device[device-id=leaf4]/device-id', 'val': 'leaf4'}], 'timestamp': 1657867500000000000, 'prefix': ''}}
{'update': {'update': [{'path': 'inventory/state/device[device-id=host1]/device-id', 'val': 'host1'}], 'timestamp': 1657867500000000000, 'prefix': ''}}
{'update': {'update': [{'path': 'inventory/state/device[device-id=spine1]/device-id', 'val': 'spine1'}], 'timestamp': 1657867500000000000, 'prefix': ''}}
{'update': {'update': [{'path': 'inventory/state/device[device-id=spine2]/device-id', 'val': 'spine2'}], 'timestamp': 1657867500000000000, 'prefix': ''}}
{'update': {'update': [{'path': 'inventory/state/device[device-id=leaf1]/device-id', 'val': 'leaf1'}], 'timestamp': 1657867500000000000, 'prefix': ''}}
{'update': {'update': [{'path': 'inventory/state/device[device-id=leaf2]/device-id', 'val': 'leaf2'}], 'timestamp': 1657867500000000000, 'prefix': ''}}
{'update': {'update': [{'path': 'inventory/state/device[device-id=host2]/device-id', 'val': 'host2'}], 'timestamp': 1657867500000000000, 'prefix': ''}}
{'sync_response': True}









^CTraceback (most recent call last):
  File "/home/arista/test7.py", line 22, in <module>
    for device in inventory:
  File "/home/arista/.local/lib/python3.9/site-packages/pygnmi/client.py", line 990, in __next__
    return self.next()
  File "/home/arista/.local/lib/python3.9/site-packages/pygnmi/client.py", line 996, in next
    return self._next_update(timeout=None)
  File "/home/arista/.local/lib/python3.9/site-packages/pygnmi/client.py", line 1066, in _next_update
    return self._get_one_update(timeout=timeout)
  File "/home/arista/.local/lib/python3.9/site-packages/pygnmi/client.py", line 959, in _get_one_update
    return telemetryParser(self._updates.get(block=True, timeout=timeout))
  File "/usr/local/lib/python3.9/queue.py", line 171, in get
    self.not_empty.wait()
  File "/usr/local/lib/python3.9/threading.py", line 312, in wait
    waiter.acquire()
KeyboardInterrupt

arista@devbox:~$

gnmic equivalent command (it is OK i.e it stops by itself i.e no need to CNTL + C)

arista@devbox:~$ gnmic -a 192.168.0.5:443 --mode=once subscribe --path /inventory/state/device/device-id --token=`cat token.tok` --skip-verify

fyi pygnmi using sub and stream instead of once is OK

"""
pygnmi script
"""

from pygnmi.client import gNMIclient

TELEMETRY_REQUEST = {
                         'subscription': [
                             {
                                 'path': '/inventory/state/device/device-id',
                                 'mode': 'target_defined'
                             }
                         ],
                         'mode': 'stream',
                         'encoding': 'json'
                     }

with open("token.tok") as f:
    TOKEN = f.read().strip('\n')

if __name__ == "__main__":
    with gNMIclient(target=('192.168.0.5', '443'), token=TOKEN, skip_verify=True) as gconn:
        inventory = gconn.subscribe2(subscribe=TELEMETRY_REQUEST)
        for device in inventory:
            print(device)

ERROR:root:Conversion of gNMI paths to the Protobuf format failed on multiple key path

Failed deleting this path:
nokia-conf:configure/policy-options/prefix-list[name=PL_DDOS_MITIGATE]/prefix[ip-prefix=1.2.5.0/24][type=through]

ip-prefix and type, both are required.

path = "nokia-conf:configure/policy-options/prefix-list[name=PL_DDOS_MITIGATE]/prefix[ip-prefix=1.2.5.0/24][type=through]"
gc.set(delete=[path])

returns:
ERROR:root:Conversion of gNMI paths to the Protobuf format failed

add the `target` support in PyGNMI

CVP is the Arista management tool. CVP has a database with the data streamed from the EOS devices.
CVP supports gNMI so we can sub on CVP to OC paths streamed by the device.
Here's the doc

Here's an example using gNMIc.
-a is the cvp address and --target is the device SN

to sub to the BGP YANG container in the cvp database (cvp address is 192.168.0.5) for the device spine1:

gnmic -a 192.168.0.5:443 subscribe --path "openconfig:/network-instances/network-instance/protocols/protocol/bgp/" --token=$token --target=spine1_SN --skip-verify

Mutual TLS (mTLS) support

Do you support mTLS and do you have any working examples using both client and server certs? I've tried various combinations of this without success.

with gNMIclient(target=host, username='admin', password='password',
path_root='/certs/ca.cert.pem',
path_key='/certs/ca.nopass.key',
path_cert='/certs/client.crt', override='theSubject') as gc:

UPDATE w/ CLI using ASCII doesn't work

When trying UPDATE cmds with path CLI on Arista using ASCII failures occur because the jsdon.dumps() is leaving the "" wrapped around the cmds:

Error: failed to apply: Invalid input (at token 0: '"interface'): CLI command 2 of 4 '"interface Management 1"' failed: invalid command
CLI Commands:
1 "interface Management 1"
2 "description TEST"

Updates to strip the first/last char worked:

$ git diff
diff --git a/pygnmi/client.py b/pygnmi/client.py
index 6deb0a9..28437aa 100644
--- a/pygnmi/client.py
+++ b/pygnmi/client.py
@@ -468,7 +468,7 @@ class gNMIclient(object):
                         elif encoding == 'proto':
                             replace_msg.append(Update(path=u_path, val=TypedValue(proto_bytes=u_val)))
                         elif encoding == 'ascii':
-                            replace_msg.append(Update(path=u_path, val=TypedValue(ascii_val=u_val)))
+                            replace_msg.append(Update(path=u_path, val=TypedValue(ascii_val=u_val[1:-1])))
                         elif encoding == 'json_ietf':
                             replace_msg.append(Update(path=u_path, val=TypedValue(json_ietf_val=u_val)))
 
@@ -494,7 +494,7 @@ class gNMIclient(object):
                         elif encoding == 'proto':
                             update_msg.append(Update(path=u_path, val=TypedValue(proto_bytes=u_val)))
                         elif encoding == 'ascii':
-                            update_msg.append(Update(path=u_path, val=TypedValue(ascii_val=u_val)))
+                            update_msg.append(Update(path=u_path, val=TypedValue(ascii_val=u_val[1:-1])))
                         elif encoding == 'json_ietf':
                             update_msg.append(Update(path=u_path, val=TypedValue(json_ietf_val=u_val)))
 
diff --git a/setup.py b/setup.py
old mode 100644
new mode 100755

No target specified in prefix

Here is my code

from pygnmi.client import gNMIclient

host = ('120.108.58.112', '8080')

with gNMIclient(target=host, username='', password='', insecure=True, debug=True) as gc:
     result = gc.get(path=['sonic-port:sonic-port/PORT/PORT_LIST[ifname=Ethernet2]'], encoding='json_ietf')

print(result)

Error:

Exception: <_InactiveRpcError of RPC that terminated with:
status = StatusCode.UNIMPLEMENTED
details = "No target specified in prefix"

How to configure the prefix attribute?

Telemetry streaming for specified time interval

Thanks for the Telemetry streaming library.
Telemetry streaming is not timebound. Request to Support stream output for specified time and close the channel Gracefully.
Ex: Stream for 2 mins and close the channel gracefully

Normalize order of keys in gNMI paths

Per the gNMI specs, keys are unordered. With the current implementation, the following can be received in a single subscribe update event:

network-instance[name=overlay]/bgp-rib/ipv4-unicast/rib-in-out/rib-in-post/routes[neighbor=192.168.127.3][prefix=10.10.10.10/32]/last-modified
network-instance[name=overlay]/bgp-rib/ipv4-unicast/rib-in-out/rib-in-post/routes[prefix=10.10.10.10/32][neighbor=192.168.127.3]/tie-break-reason

(also note the missing '/' at the beginning of the path)

The Telemetryparser should apply a consistent ordering of keys (e.g. sort them alphabetically) such that applications can more easily process these events ( in my case, apply regex matching )

Support custom user-agent string (and other gRPC options)

gRPC supports the "grpc.primary_user_agent" option to set the User Agent header sent in API calls.
pygnmi should accept this and other grpc options as init parameters

Token based authentication for pygnmicli

https://grpc.io/docs/guides/auth/
issue #63
I did not find this option in pygnmicli

same thing for issue #67

arista@devbox:~$ pygnmicli
usage: pygnmicli [-h] -t TARGET -u USERNAME [-p PASSWORD] [-c PATH_CERT] [-k PATH_KEY] [-r PATH_ROOT] [-O OVERRIDE] [-i] [--skip-verify]
                 [-o {capabilities,get,set-update,set-replace,set-delete,subscribe-stream,subscribe-poll,subscribe-once,subscribe2}] [-x GNMI_PATH [GNMI_PATH ...]]
                 [--gnmi-path-target GNMI_PATH_TARGET] [-d [{all,config,operational,state}]] [-f FILE] [-D] [-C {get,print,}] [--ext-history-range-start EXT_HISTORY_RANGE_START]
                 [--ext-history-range-end EXT_HISTORY_RANGE_END] [--ext-history-snapshot-time EXT_HISTORY_SNAPSHOT_TIME]
pygnmicli: error: the following arguments are required: -t/--target, -u/--user
arista@devbox:~$

Is it something you would like to add?
If yes, would you like 2 diff issues?

Support multithreaded clients

As shown at https://stackoverflow.com/questions/52993976/grpc-python-support-multithreading-on-client-and-server/53090254, the suggested structure for multithreaded client applications:

global channel
per-thread stub instance

The way gNMIClient is currently structured, a channel and a stub are always allocated together.
Perhaps using Thread local storage (see e.g. https://www.bogotobogo.com/python/Multithread/python_multithreading_Thread_Local_Specific_Data.php) for the stub could work, together with separating channel instantiation from stub instantiation

Each method call could do 'getStub' which would create a thread-local instance if not already existing, on demand

skip_verify in 0.7.1

This is working

pygnmi 0.7.1
token based auth
no username and no password
skip_verify instead of override

arista@devbox:~$ pip freeze | grep gnm
pygnmi==0.7.1

arista@devbox:~$ cat test2.py 
from pprint import pprint as pp
import json
from pygnmi.client import gNMIclient

with open("token.tok") as f:
    TOKEN = f.read().strip('\n')

host = ('192.168.0.5', '443')
with gNMIclient(target=host, token=TOKEN, skip_verify=False) as gc:
    result = gc.capabilities()

pp(result) 
arista@devbox:~$

but using skip_verify=True it doesnt work.
It should be the opposite isnt it?

path format and root format

Hi Anton,

I verified the pygnmi path syntax and I am not sure, whether the pygnmi path generator implementation matches the gnmi spec ...

in client.py docstring the path is specified_
path = ['yang-module:container/container[key=value]', 'yang-module:container/container[key=value]', ..]

in https://github.com/openconfig/reference/blob/master/rpc/gnmi/gnmi-path-conventions.md I see following spec: ... the root path / is encoded as a zero length array (slice) of PathElem messages. Example declarations in several languages: Python: path = [] Note this is not the same as a path consisting of a single empty string element. A human-readable path can be formed by concatenating elements of the prefix and path using a / separator, and preceded by a leading / character. ....

in this respect I think we should verify the pygmni implementation:

a single root path / should be allowed. The current pygnmi implementation incorrectly converts "/" to "path { elem {} elem {} }", should be changed to "path []"
leading / character must be allowed in path. This is not the case with current implementation. E.g. the current pygnmi implementation incorrectly converts "/interfaces/interface" to "path { elem {} elem { name: "interfaces"} elem { name: "interface } }", should be changed to "path { elem { name: "interfaces"} elem { name: "interface } }""
both two options must allow origin specification: e.g. "openconfig:/" or "openconfig:interfaces:/interfaces/interface"

@anton, I am happy to change the path generator model accordingly, test against Juniper and Arista and send a pull request ... let me know ...

best regards Stefan

timeout for pygnmi subscribe generator

Hi Anton,

many thanks for providing this library.
Actually this request is more a question than an issue ...

I consider using pygnmi for a testing environment and for this use case I would need a timeout for subscribe generator.

I have implemented a wrapper class (based on kthreads and queues) and that serve my needs
https://github.com/slieberth/gnmi_subscribe_timeout/blob/main/gnmi_subscribe_timeout.py

Do you think a timeout for the pygnmi subscribe generator?
( I can contribute, if you wish ... )

beste regards
Stefan

pygnmi no longer installable via pip since 0.8.0

The new requirements of 0.8.0 seem to break pip. 'protobuf --no-binary=protobuf' does not seem to be a valid dependency in setup.py.

quoth:/tmp$ python3 -m venv pygnmi-venv
quoth:/tmp$ . pygnmi-venv/bin/activate
(pygnmi-venv) quoth:/tmp$ pip install pip -U >/dev/null
(pygnmi-venv) quoth:/tmp$ pip --version
pip 22.2 from /tmp/pygnmi-venv/lib/python3.8/site-packages/pip (python 3.8)
(pygnmi-venv) quoth:/tmp$ pip install pygnmi
Collecting pygnmi
  Using cached pygnmi-0.8.0.tar.gz (21 kB)
  Preparing metadata (setup.py) ... error
  error: subprocess-exited-with-error
  
  × python setup.py egg_info did not run successfully.
  │ exit code: 1
  ╰─> [1 lines of output]
      error in pygnmi setup command: 'install_requires' must be a string or list of strings containing valid project/version requirement specifiers; Invalid requirement, parse error at "'--no-bin'"
      [end of output]
  
  note: This error originates from a subprocess, and is likely not a problem with pip.
error: metadata-generation-failed

× Encountered error while generating package metadata.
╰─> See above for output.

note: This is an issue with the package mentioned above, not pip.
hint: See above for details.

gnmi enhancement to support json_ietf encoding and work when json_ietf_val is bytes string

Can we support different encodings for get? I think we just need a new argument (already made local change and it works).
On top of that, when json_ietf_val is a bytes string, we hit json.decoder.JSONDecodeError exception. Example:

json_ietf_val = b'DIRECTLY_CONNECTED'

This could be a simple fix as well, which I've made locally and it works.

Thank you for the awesome library!

pygnmi not returning the entire config of devicewhen path is empty

Hey @akarneliuk
Hope you are doing good . I was checking if pygnmi could return the entire config of the box when path is set as empty .
When I am trying with gnmi_cli client code with path as empty and type as 1 I get the entire config from the Cisco IOS XR device but when I try from pygnmi its returning empty . could you pls look into that ?

//from pygmi

raw_data = gc.get(path=[], encoding='json_ietf',datatype='config')

error {
}

{}

//from gnmi_cli client :

path: <>
encoding: 4
type: 1


bash@bgl-ads-4729 gnmi_open_source>./gnmi_client_with_uname -a 10.127.60.177:57400 -insecure -insecure_username test2 -insecure_password cisco123 -proto "$(cat get-config.proto)" -get
notification: <
  timestamp: 1611122182616479629
  update: <
    path: <
    >
    val: <
      json_ietf_val: "{\"Cisco-IOS-XR-man-xml-ttyagent-cfg:netconf\":{\"agent\":{\"tty\":{\"enable\":[null]}}},\"Cisco-IOS-XR-infra-syslog-cfg:syslog\":{\"console-logging\":{\"logging-level\":\"debug\"},\"host-server\":{\"vrfs\":{\"vrf\":[{\"vrf-name\":\"default\",\"ipv4s\":{\"ipv4\":[{\"address\":\"10.105.57.55\",\"ipv4-severity-port\":{\"severity\":\"debug\",\"port\":514}}]}}]}}},\"Cisco-IOS-XR-infra-infra-clock-linux-cfg:clock\":{\"time-zone\":{\"time-zone-name\":\"IST\",\"area-name\":\"Asia/Kolkata\"}},\"Cisco-IOS-XR-call-home-cfg:call-home\":{\"active\":[null],\"contact-smart-licensing\":true,\"profiles\":{\"profile\":[{\"profile-name\":\"CiscoTAC-1\",\"active\":[null],\"methods\":{\"method\":[{\"method\":\"email\",\"enable\":false},{\"method\":\"http\",\"enable\":true}]}}]}},\"Cisco-IOS-XR-ip-tcp-cfg:ip\":{\"cinetd\":{\"services\":{\"telnet\":{\"vrfs\":{\"vrf\":[{\"vrf-name\":\"default\",\"ipv4\":{\"tcp\":{\"maximum-server\":100}}}]}}}}},\"Cisco-IOS-XR-aaa-lib-cfg:aaa\":{\"Cisco-IOS-XR-aaa-locald-cfg:usernames\":{\"username\":[{\"ordering-index\":0,\"name\":\"test\",\"usergroup-under-usernames\":{\"usergroup-under-username\":[{\"name\":\"root-lr\"},{\"name\":\"cisco-support\"}]},\"secret\":{\"type\":\"type5\",\"secret5\":\"$1$YZ4z$UjIVzjqOgKsP.IEz31RFP.\"}},{\"ordering-index\":1,\"name\":\"test1\"},{\"ordering-index\":2,\"name\":\"test3\"},{\"ordering-index\":3,\"name\":\"root\",\"usergroup-under-usernames\":{\"usergroup-under-username\":[{\"name\":\"root-lr\"},{\"name\":\"cisco-support\"}]},\"secret\":{\"type\":\"type10\",\"secret10\":\"$6$dyOzw1jqDzxnDw1.$5/O9cXLjfw168BufPz33afDZP/ndMC7Ec1PKe01nTlxv/MtMrzAB4iHAjj11fA2GFJLxN4l5YWxAK5Rl7PJPL.\"}}]},\"accountings\":{\"accounting\":[{\"type\":\"exec\",\"listname\":\"default\",\"type-xr\":\"start-stop\",\"method1\":\"server-group\",\"server-group-name1\":\"TACACS_ALL\"},{\"type\":\"commands\",\"listname\":\"default\",\"type-xr\":\"start-stop\",\"method1\":\"server-group\",\"server-group-name1\":\"TACACS_ALL\"}]},\"Cisco-IOS-XR-aaa-locald-cfg:server-groups\":{\"Cisco-IOS-XR-aaa-tacacs-cfg:tacacs-server-groups\":{\"tacacs-server-group\":[{\"server-group-name\":\"TACACS_ALL\",\"private-servers\":{\"private-server\":[{\"ordering-index\":0,\"ip-address\":\"10.106.201.176\",\"port-number\":49,\"key\":{\"text\":\"09584B1A0D0C19155A5E57\",\"encrypt-type\":\"type7\"},\"timeout\":5}]}}]}},\"authorizations\":{\"authorization\":[{\"type\":\"exec\",\"listname\":\"default\",\"method1\":\"server-group\",\"method2\":\"local\",\"server-group-name1\":\"TACACS_ALL\"},{\"type\":\"commands\",\"listname\":\"default\",\"method1\":\"server-group\",\"server-group-name1\":\"TACACS_ALL\"}]},\"authentications\":{\"authentication\":[{\"type\":\"login\",\"listname\":\"default\",\"method1\":\"server-group\",\"method2\":\"local\",\"server-group-name1\":\"TACACS_ALL\"}]}},\"Cisco-IOS-XR-ip-domain-cfg:ip-domain\":{\"vrfs\":{\"vrf\":[{\"vrf-name\":\"default\",\"name\":\"cisco.com\",\"servers\":{\"server\":[{\"order\":0,\"server-address\":\"72.163.128.140\"}]}}]}},\"Cisco-IOS-XR-snmp-agent-cfg:mib\":{\"Cisco-IOS-XR-snmp-ifmib-cfg:interface-mib\":{\"stati

grpc keep alive feature not working

In line 67/68 of client.py, I think it should read:

if 'keepalive_time_ms' in kwargs:
self.configureKeepalive(**kwargs)

Otherwise:

TypeError: configureKeepalive() got an unexpected keyword argument 'interval_ms'

Passing encoding type as json_ietf_val in pygNMI

Hi,
I am trying to use pygnmi on Cisco IOS-XR device which works on json_ietf_val
The gNMI client is giving below error marked in BOLD below
How can we pass encoding as json_ietf_val

Pls suggest the possible paramter

from pygnmi.client import gNMIclient
host = ('10.127.60.177', '57400')
with gNMIclient(target=host, username='test2', password='cisco123', insecure=True) as gc:
... result = gc.get(path=['openconfig-interfaces:interfaces'])
...
Host: 10.127.60.177:57400
Error: gNMI: unsupported get-request encoding: JSON

Problems with SSL certificates with missing common name and alternative name

If the common name is empty for the certificate, pygnmi fails due to this line.

>>> ssl_cert_deserialized
<Certificate(subject=<Name(<<redacted>>)>, ...)>
>>> ssl_cert_deserialized.subject.get_attributes_for_oid(x509.oid.NameOID.COMMON_NAME)
[]

I bypassed this using an if-else block, manually setting ssl_target_name_override. But this time pygnmi failed due to alternative name.

>>> ssl_cert_subject_alt_names = ssl_cert_deserialized.extensions.get_extension_for_oid(x509.oid.ExtensionOID.SUBJECT_ALTERNATIVE_NAME)
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/redacted/redacted/.virtualenvs/redacted/lib/python3.8/site-packages/cryptography/x509/extensions.py", line 125, in get_extension_for_oid
    raise ExtensionNotFound("No {} extension was found".format(oid), oid)
cryptography.x509.extensions.ExtensionNotFound: No <ObjectIdentifier(oid=2.5.29.17, name=subjectAltName)> extension was found

I tried this with a second certificate we use on some other devices and it failed with the same error.

I'm not sure what is missing here but I am using the same certificates with other gnmi tools in production so I don't think there's something wrong with them.

FutureTimeoutError connecting to SR-OS over TLS

Hi,

I just started working with gNMI and TLS using a SR-OS platform. The basic setup seems to be working, as gnmic returns the correct output when reaching the test device. This is also working without specifying the --tls-ca cert but with --skip-verify:

gnmic -a [hostname] -u [user] -p [passwd] get --path /state/system/platform --tls-ca certificate.ca-bundle
[
  {
    "source": "[source]",
    "timestamp": 1651586070618194298,
    "time": "2022-05-03T15:54:30.618194298+02:00",
    "updates": [
      {
        "Path": "state/system/platform",
        "values": {
          "state/system/platform": "7750 SR-7"
        }
      }
    ]
  }
]

But, using pyGNMI from the same machine, same connection data, is not working for me:

if __name__ == '__main__':
    with gNMIclient( target=host, username=username, password=password, 
        path_root=path_root_cert, 
        debug=True
        ) as gc:

Traceback (most recent call last):
  File "/gRPC_tests.py", line 37, in <module>
    with gNMIclient( target=host, username=username, password=password, 
  File "/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/site-packages/pygnmi/client.py", line 94, in __enter__
    return self.connect()
  File "/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/site-packages/pygnmi/client.py", line 146, in connect
    self.wait_for_connect(timeout)
  File "/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/site-packages/pygnmi/client.py", line 156, in wait_for_connect
    grpc.channel_ready_future(self.__channel).result(timeout=timeout)
  File "/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/site-packages/grpc/_utilities.py", line 139, in result
    self._block(timeout)
  File "/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/site-packages/grpc/_utilities.py", line 85, in _block
    raise grpc.FutureTimeoutError()

Tried in some other ways, without insecure parameter, etc. Debug flag does not give more info. I can see traffic exchange between my machine and the gNMI server.

FYI: Unencrypted connections (w/o using TLS) worked just fine.

Thanks!

Info regarding creating the config for delete operation using pyangbind

Hi @akarneliuk ,
I am trying to generate config for delete operation using pyangbind but some how I am not able to do it
getting exception as Exception: Conversion of gNMI paths to the Protobuf format failed
do you have a working example of delete operation via pygNMI which is working with pyangbind ?
pls suggest

In [5]: intf=ifmgr.interface_configurations.interface_configuration.add(active="act",interface_name="Optics0/2/0/0")                                                                                    

In [6]: print(pybindJSON.dumps(ifmgr,mode="ietf"))                                                                                                                                                      
{
    "Cisco-IOS-XR-ifmgr-cfg:interface-configurations": {
        "interface-configuration": [
            {
                "active": "act",
                "interface-name": "Optics0/2/0/0"
            }
        ]
    }
}

In [7]: pybindJSON.dump(ifmgr,"test.json",mode="ietf") 
   ...: fn=open("test.json") 
   ...: data=json.load(fn) 
   ...: set_list = [(k, v) for k, v in data.items()]                                                                                                                                                    

In [8]:     with gNMIclient(target=host, username='test2',password='cisco123', path_cert='/Volumes/avpathak/BossHogg/openconfig/ems_BH_P2A4.pem',override='ems.cisco.com',debug=True) as gc: 
   ...:         result = gc.set(delete=set_list) 
   ...:         print(result) 
   ...:                                                                                                                                                                                                 
Conversion of gNMI paths to the Protobuf format failed
---------------------------------------------------------------------------
TypeError                                 Traceback (most recent call last)
~/anaconda3/lib/python3.7/site-packages/pygnmi/client.py in set(self, delete, replace, update, encoding)
    399                 try:
--> 400                     del_protobuf_paths = [gnmi_path_generator(pe) for pe in delete]
    401 

~/anaconda3/lib/python3.7/site-packages/pygnmi/client.py in <listcomp>(.0)
    399                 try:
--> 400                     del_protobuf_paths = [gnmi_path_generator(pe) for pe in delete]
    401 

~/anaconda3/lib/python3.7/site-packages/pygnmi/path_generator.py in gnmi_path_generator(path_in_question)
     16     if path_in_question:
---> 17         if re.match(r'.*?\[.+?=.+?\].*?', path_in_question):
     18             split_list = re.findall(r'.*?\[.+?=.+?\].*?', path_in_question)

~/anaconda3/lib/python3.7/re.py in match(pattern, string, flags)
    172     a Match object, or None if no match was found."""
--> 173     return _compile(pattern, flags).match(string)
    174 

TypeError: expected string or bytes-like object

During handling of the above exception, another exception occurred:

Exception                                 Traceback (most recent call last)
<ipython-input-8-fb313a024e28> in <module>
      1 with gNMIclient(target=host, username='test2',password='cisco123', path_cert='/Volumes/avpathak/BossHogg/openconfig/ems_BH_P2A4.pem',override='ems.cisco.com',debug=True) as gc:
----> 2     result = gc.set(delete=set_list)
      3     print(result)
      4 

~/anaconda3/lib/python3.7/site-packages/pygnmi/client.py in set(self, delete, replace, update, encoding)
    402                 except:
    403                     logger.error(f'Conversion of gNMI paths to the Protobuf format failed')
--> 404                     raise Exception (f'Conversion of gNMI paths to the Protobuf format failed')
    405 
    406             else:

Exception: Conversion of gNMI paths to the Protobuf format failed

In [9]:

junos - device config example to work with pygnmi?

Hello,

i'm attempting to use pygnmi to get some data from a Juniper MX204, and I am running into issues. I suspect it may be with my router config, that I might be missing something to get things to work, but so far deciphering their docs has been..painful. I wonder if you had an example configuration you used to validate against for Juniper.

relevant snippet of my simple python code based on examples:

if __name__ == '__main__':
    with gNMIclient(target=host, username=USERNAME, password=PASSWORD, insecure=True, debug=True) as gc:
        caps = gc.capabilities()

        result_oc_intf = gc.get(path=['openconfig-interfaces:interfaces'], encoding="json_ietf")
        print()
        print("openconfig-interfaces:interfaces -")
        pprint(result_oc_intf)

Getting capabilities works, and openconfig-interfaces is listed in the supported models.

I get the following error:

...output truncated showing all of the capabilities...
------------------------------------------------
gNMI request:
------------------------------------------------
path {
  origin: "openconfig-interfaces"
  elem {
    name: "interfaces"
  }
}
encoding: JSON_IETF

------------------------------------------------
Host: 172.17.252.51:32767
Error: Error in retrieval for the given config path
CRITICAL:root:Host: 172.17.252.51:32767, Error: Error in retrieval for the given config path

on the Juniper MX side i've configured the following only so far, with no success:

set system schema openconfig unhide
set system services extension-service request-response grpc clear-text address 172.17.252.51
set system services extension-service request-response grpc clear-text port 32767

thanks.

akarneliuk / pygnmi Goto Github PK

pygnmi's Introduction

pyGNMI: Python gNMI client

Usage

Video tutorial

All gNMI RPCs supported

Supported operation modes

Tested Network Operating Systems (NOS)

Network Operating Systems (NOS) in test

License

Contributors

Dev Log

pygnmi's People

Contributors

Stargazers

Watchers

Forkers

pygnmi's Issues

gNMI response:

gnmic Capabilities:

gnmic Version:

Script:

Modules

Variables

Body

Execution failure:

pygnmi installation successful:

working

not working

Recommend Projects

Recommend Topics

Recommend Org