ciscotestautomation / unicon.plugins Goto Github PK

View Code? Open in Web Editor NEW

21.0 16.0 42.0 3.06 MB

pyATS Unicon Plugins

Home Page: https://developer.cisco.com/pyats/

License: Apache License 2.0

Makefile 0.30% Python 99.70%

pyats unicon plugin cisco netdevops test-automation network-automation

unicon.plugins's Introduction

Unicon Plugins

Unicon is a package aiming to provide a unified connection experience to network devices through typical command-line management interface. By wrapping the underlying session (eg, telnet, ssh), Unicon provides:

direct and proxied connections through any common CLI interface (telnet, ssh, serial etc)
power of expect-like programming without having to deal with low-level logic
multi-vendor support through an agnostic API interface
seamless handling of CLI modes (eg, enable, configure, admin-configure mode)
rejected commands, command error detections
value-add statful services (specific to the platform)

and is extensible: platform supports and services are implemented via open-source plugins.

Unicon is the standard, go-to CLI connection implementation for Cisco pyATS framework.

This package was initially developed internally in Cisco, and is now release to the general public starting late 2017 through Cisco DevNet.

Development Mode

To start developing plugins for Unicon, clone this repository into your pyATS virtual environment, and run make develop:

bash$ cd ~/pyats
bash$ git clone https://github.com/CiscoTestAutomation/unicon.plugins
bash$ cd unicon.plugins
bash$ make develop

Support & Community

See https://developer.cisco.com/docs/pyats/#!license-support page for details.

unicon.plugins's People

Contributors

Stargazers

Watchers

unicon.plugins's Issues

unable to login to an `iosxr` and `sros` device

Hi,

Thank you for making such a library available. Here is my experience with it:

in regards to junos, all looking good.

in regards to sros, I am waiting for this PR to land in pip, (but even without the asterisk) with log_stdout enable:

In [69]: uni_conn = unicon.Connection(hostname="x.x.x.x", credentials={"default": {"username": os.environ["USR_NETOPS"], "password": o
    ...: s.environ["PASS_NETOPS"]}}, os="sros", learn_hostname=True, start=["ssh {}".format("x.x.x.x")], )

2021-03-21 12:37:00,953: %UNICON-INFO: +++ x.x.x.x logfile /tmp/x_x_x_x-20210321T123700952.log +++

2021-03-21 12:37:00,953: %UNICON-INFO: +++ Unicon plugin sros +++

In [70]: uni_conn.connect()
...
[email protected]'s password: 

2021-03-21 12:37:05,601: %UNICON-INFO: +++ connection to spawn: ssh -l __ x.x.x.x, id: 140305151131360 +++

2021-03-21 12:37:05,602: %UNICON-INFO: connection to x.x.x.x

A:a_hostname#

The connection hangs and never returns (although successfully logins).

in regards to iosxr, it doesn't recognise the RSP/CPU prompt (although it a library from Cisco)

In [71]: uni_conn = unicon.Connection(hostname="x.x.x.x", credentials={"default": {"username": os.environ["USR_NETOPS"], "password":
    ...: os.environ["PASS_NETOPS"]}}, os="iosxr", learn_hostname=True, start=["ssh {}".format("x.x.x.x")], )

2021-03-21 12:44:39,433: %UNICON-INFO: +++ x.x.x.x logfile /tmp/x_x_x_x-20210321T124439432.log +++

2021-03-21 12:44:39,433: %UNICON-INFO: +++ Unicon plugin iosxr +++

In [72]: uni_conn.connect()
Unable to negotiate with x.x.x.x port 22: no matching cipher found. Their offer: aes128-cbc,aes192-cbc,aes256-cbc,3des-cbc


2021-03-21 12:44:42,854: %UNICON-INFO: +++ connection to spawn: ssh -l __ x.x.x.x, id: 140305378210048 +++

2021-03-21 12:44:42,855: %UNICON-INFO: connection to x.x.x.x

---------------------------------------------------------------------------
OSError                                   Traceback (most recent call last)
~/.local/lib/python3.8/site-packages/unicon/eal/backend/pty_backend.cpython-38-x86_64-linux-gnu.so in unicon.eal.backend.pty_backend.RawPtySpawn._read()

OSError: [Errno 5] Input/output error

The above exception was the direct cause of the following exception:

EOF                                       Traceback (most recent call last)
~/.local/lib/python3.8/site-packages/unicon/statemachine/statemachine.cpython-38-x86_64-linux-gnu.so in unicon.statemachine.statemachine.StateMachine.go_to()

~/.local/lib/python3.8/site-packages/unicon/statemachine/statetransition.cpython-38-x86_64-linux-gnu.so in unicon.statemachine.statetransition.AnyStateTransition.do_transitions()

~/.local/lib/python3.8/site-packages/unicon/eal/backend/pty_backend.cpython-38-x86_64-linux-gnu.so in unicon.eal.backend.pty_backend.RawSpawn.expect()

~/.local/lib/python3.8/site-packages/unicon/eal/backend/pty_backend.cpython-38-x86_64-linux-gnu.so in unicon.eal.backend.pty_backend.RawSpawn.read_update_buffer()

~/.local/lib/python3.8/site-packages/unicon/eal/backend/pty_backend.cpython-38-x86_64-linux-gnu.so in unicon.eal.backend.pty_backend.RawSpawn.read()

~/.local/lib/python3.8/site-packages/unicon/eal/backend/pty_backend.cpython-38-x86_64-linux-gnu.so in unicon.eal.backend.pty_backend.RawPtySpawn._read()

EOF: ('Unable to read. Connection closed or not available', OSError(5, 'Input/output error'))

The above exception was the direct cause of the following exception:

StateMachineError                         Traceback (most recent call last)
~/.local/lib/python3.8/site-packages/unicon/bases/connection.cpython-38-x86_64-linux-gnu.so in unicon.bases.connection.Connection.connect()

~/.local/lib/python3.8/site-packages/unicon/bases/routers/connection_provider.cpython-38-x86_64-linux-gnu.so in unicon.bases.routers.connection_provider.BaseSingleRpConnectionProvider.connect()

~/.local/lib/python3.8/site-packages/unicon/bases/routers/connection_provider.cpython-38-x86_64-linux-gnu.so in unicon.bases.routers.connection_provider.BaseSingleRpConnectionProvider.establish_connection()

~/.local/lib/python3.8/site-packages/unicon/statemachine/statemachine.cpython-38-x86_64-linux-gnu.so in unicon.statemachine.statemachine.StateMachine.go_to()

StateMachineError: Failed while bringing device to "any" state

The above exception was the direct cause of the following exception:

ConnectionError                           Traceback (most recent call last)
<ipython-input-72-f0137236f6fa> in <module>
----> 1 uni_conn.connect()

~/.local/lib/python3.8/site-packages/unicon/bases/connection.cpython-38-x86_64-linux-gnu.so in unicon.bases.connection.Connection.connect()

ConnectionError: failed to connect to x.x.x.x
Failed while bringing device to "any" state

While the actual prompt is

[email protected]'s password: 


RP/0/RSP0/CPU0:a_hostname#

Would you be so kind to advise if I am doing somthing wrong?

Connection to Linux Terminal Server Fail

Hello Team,

I am trying to ssh to linux terminal server (Production Server) , and i am facing problem with it

i am writing python code , same code iam using on linux VM and its successful

i am suspecting the the prompt of the server is the one making the problem as it like @ #

i tried with prompt_recovery = True , still the same

code as follows

from unicon import Connection

server = Connection(hostname='',
start=['ssh -q -o LogLevel=error '],
os='linux',
credentials={'default': {'username': '', 'password': ''}},
learn_hostname = True,
debug = True,
)

server.connect()
output = server.execute('date')
print(output)

Log File

(env) myuser@testVM:/var/www/test$ python connectNet.py

2023-02-17 14:03:53,554: %UNICON-INFO: +++ logfile /tmp/-20230217T140353554.log +++

2023-02-17 14:03:53,554: %UNICON-INFO: +++ Unicon plugin linux (unicon.plugins.linux) +++

2023-02-17 14:03:53,554: %UNICON-DEBUG: +++ initializing context +++

2023-02-17 14:03:53,555: %UNICON-DEBUG: +++ initializing state_machine +++

2023-02-17 14:03:53,555: %UNICON-DEBUG: +++ initializing services +++

2023-02-17 14:03:53,555: %UNICON-DEBUG: adding service send : <unicon.plugins.generic.service_implementation.Send object at 0x7fd8e9b08cd0>

2023-02-17 14:03:53,555: %UNICON-DEBUG: adding service sendline : <unicon.plugins.generic.service_implementation.Sendline object at 0x7fd8e9b08e20>

2023-02-17 14:03:53,555: %UNICON-DEBUG: adding service transmit : <unicon.plugins.generic.service_implementation.Send object at 0x7fd8e9b08df0>

2023-02-17 14:03:53,555: %UNICON-DEBUG: adding service receive : <unicon.plugins.generic.service_implementation.ReceiveService object at 0x7fd8e9b08e80>

2023-02-17 14:03:53,555: %UNICON-DEBUG: adding service receive_buffer : <unicon.plugins.generic.service_implementation.ReceiveBufferService object at 0x7fd8e9b08e50>

2023-02-17 14:03:53,555: %UNICON-DEBUG: adding service expect : <unicon.plugins.generic.service_implementation.Expect object at 0x7fd8e9b08d60>

2023-02-17 14:03:53,556: %UNICON-DEBUG: adding service log_user : <unicon.plugins.generic.service_implementation.LogUser object at 0x7fd8e9b08eb0>

2023-02-17 14:03:53,556: %UNICON-DEBUG: adding service execute : <unicon.plugins.linux.service_implementation.Execute object at 0x7fd8e9b08ee0>

2023-02-17 14:03:53,556: %UNICON-DEBUG: adding service ping : <unicon.plugins.linux.service_implementation.Ping object at 0x7fd8e9b08f10>

2023-02-17 14:03:53,556: %UNICON-DEBUG: adding service expect_log : <unicon.plugins.generic.service_implementation.ExpectLogging object at 0x7fd8e9b08f70>

2023-02-17 14:03:53,556: %UNICON-DEBUG: adding service sudo : <unicon.plugins.linux.service_implementation.Sudo object at 0x7fd8e9b08f40>

2023-02-17 14:03:53,556: %UNICON-DEBUG: adding service trex_console : <unicon.plugins.linux.service_implementation.TrexConsole object at 0x7fd8e9b192e0>

2023-02-17 14:03:53,557: %UNICON-DEBUG: spawn session 'ssh -l -q -o LogLevel=error ' is being launched.

2023-02-17 14:03:53,559: %UNICON-DEBUG: Subprocess 231116 launched as 'ssh -l -q -o LogLevel=error '

2023-02-17 14:03:53,560: %UNICON-INFO: +++ connection to spawn: ssh -l -q -o LogLevel=error , id: 140569610393296 +++

2023-02-17 14:03:53,560: %UNICON-INFO: connection to
Password:

2023-02-17 14:03:54,168: %UNICON-DEBUG: <<< Unicon Got :: '\rPassword: '

2023-02-17 14:03:54,168: %UNICON-DEBUG: *** Pattern List ::

2023-02-17 14:03:54,168: %UNICON-DEBUG: '.+$'

2023-02-17 14:03:54,168: %UNICON-DEBUG:

2023-02-17 14:03:54,168: %UNICON-DEBUG: *** Pattern Matched :: '.+$'

2023-02-17 14:03:54,168: %UNICON-DEBUG: <unicon.eal.MatchMode mode_id=0, mode_name='search multi-line re.S'>

2023-02-17 14:03:54,169: %UNICON-DEBUG:

2023-02-17 14:03:54,192: %UNICON-DEBUG: Statement matched: '[pattern='^.*[Pp]assword( for )?(\S+)?: ?$', action=password_handler, args=None, loop_continue=True, continue_timer=False, trim_buffer=True]'

2023-02-17 14:03:54,192: %UNICON-DEBUG: >>> Unicon Sending :: '\r'

Number of session: 3/6

@ #

2023-02-17 14:03:54,935: %UNICON-DEBUG: <<< Unicon Got :: '\r\nNumber of session: 3/6\r\n\r\n@ # '

2023-02-17 14:03:54,935: %UNICON-DEBUG: *** Pattern List ::

2023-02-17 14:03:54,935: %UNICON-DEBUG: 'Escape character is .*\n'

2023-02-17 14:03:54,935: %UNICON-DEBUG: '^.*RETURN to get started'

2023-02-17 14:03:54,936: %UNICON-DEBUG: 'Are you sure you want to continue connecting $yes/no(/\[fingerprint\])?$'

2023-02-17 14:03:54,936: %UNICON-DEBUG: '^.*(Connection refused|Connection reset by peer|Broken pipe|Network is down|closed by remote host)'

2023-02-17 14:03:54,936: %UNICON-DEBUG: 'Received disconnect from .*:'

2023-02-17 14:03:54,936: %UNICON-DEBUG: 'Hit Enter to proceed:'

2023-02-17 14:03:54,936: %UNICON-DEBUG: '^(.*?)Press Ctrl\+x to Exit the session'

2023-02-17 14:03:54,936: %UNICON-DEBUG: '^(.*?)Connected.'

2023-02-17 14:03:54,936: %UNICON-DEBUG: '^.?(%\w+(-\S+)?-\d+-\w+|Guestshell destroyed successfully|%Error opening tftp:\/\/255\.255\.255\.255).$'

2023-02-17 14:03:54,936: %UNICON-DEBUG: '^.?Press any key to continue\..?$'

2023-02-17 14:03:54,936: %UNICON-DEBUG: '^.*[Pp]ermission denied'

2023-02-17 14:03:54,936: %UNICON-DEBUG: '^.*Login incorrect'

2023-02-17 14:03:54,936: %UNICON-DEBUG: '^.*([Uu]sername|[Ll]ogin): ?$'

2023-02-17 14:03:54,937: %UNICON-DEBUG: '^.*[Pp]assword( for )?(\S+)?: ?$'

2023-02-17 14:03:54,937: %UNICON-DEBUG: '^.Enter passphrase for key .?:\s*?'

2023-02-17 14:03:54,937: %UNICON-DEBUG: '^.?(?P[-\w]+)\s?([-\w\]/~~:\.\d ]+)?([>\$~~%#\]])\s(\x1b\S+)?$'

2023-02-17 14:03:54,937: %UNICON-DEBUG: '^(.?)(?Ptrex>\s)$'

2023-02-17 14:03:54,937: %UNICON-DEBUG: '^(.*?([>\$~~%\]]|\] # |[^#\\s]#|~~ #|~~/|^admin:|^#|~~\s?#\s?)\s?(\x1b\S+)?)$'

2023-02-17 14:03:54,937: %UNICON-DEBUG:

2023-02-17 14:03:54,937: %UNICON-DEBUG: *** Pattern Matched :: '^.?(?P[-\w]+)\s?([-\w\]/~~:\.\d ]+)?([>\$~~%#\]])\s(\x1b\S+)?$'

2023-02-17 14:03:54,937: %UNICON-DEBUG: <unicon.eal.MatchMode mode_id=1, mode_name='search only last line'>

2023-02-17 14:03:54,938: %UNICON-DEBUG: *** Match Groups:: ('', None, '#', None)

2023-02-17 14:03:54,938: %UNICON-DEBUG: Statement matched: '[pattern='^.?(?P[-\w]+)\s?([-\w]/~~:.\d ]+)?([>$~~%#]])\s(\x1b\S+)?$', action=update_cur_state, args={'transition': <unicon.statemachine.statetransition.AnyStateTransition object at 0x7fd8e9b08ca0>, 'state': learn_hostname}, loop_continue=False, continue_timer=False, trim_buffer=True]'

2023-02-17 14:03:54,939: %UNICON-DEBUG: Learned hostname(s) : ''.

2023-02-17 14:03:54,940: %UNICON-INFO: Learned hostname:

2023-02-17 14:03:54,940: %UNICON-DEBUG: >>> Unicon Sending :: '\r'

@ #

2023-02-17 14:05:05,967: %UNICON-DEBUG: spawn session 'ssh -l -q -o LogLevel=error ' has been closed.
Traceback (most recent call last):
File "src/unicon/statemachine/statemachine.py", line 737, in unicon.statemachine.statemachine.StateMachine.go_to
File "src/unicon/statemachine/statetransition.py", line 216, in unicon.statemachine.statetransition.StateTransition.do_transitions
File "src/unicon/eal/dialogs.py", line 476, in unicon.eal.dialogs.Dialog.process
File "src/unicon/eal/dialog_processor.py", line 326, in unicon.eal.dialog_processor.SimpleDialogProcessor.process
File "src/unicon/eal/dialog_processor.py", line 285, in unicon.eal.dialog_processor.SimpleDialogProcessor.timeout_handler
unicon.core.errors.TimeoutError: timeout occurred:
timeout value: 60
last_command: '\r'
pattern: ['Escape character is .\n', '^.RETURN to get started', 'Are you sure you want to continue connecting $yes/no(/\[fingerprint\])?$', '^.(Connection refused|Connection reset by peer|Broken pipe|Network is down|closed by remote host)', 'Received disconnect from .:', 'Hit Enter to proceed:', '^(.?)Press Ctrl\+x to Exit the session', '^(.?)Connected.', '^.?(%\w+(-\S+)?-\d+-\w+|Guestshell destroyed successfully|%Error opening tftp:\/\/255\.255\.255\.255).$', '^.?Press any key to continue\..?$', '^.[Pp]ermission denied', '^.Login incorrect', '^.([Uu]sername|[Ll]ogin): ?$', '^.[Pp]assword( for )?(\S+)?: ?$', '^.Enter passphrase for key .?:\s*?', '^(.*?([>\$~~%\]]|\] # |[^#\\s]#|~~ #|~~/|^admin:|^#|~~\s?#\s?)\s?(\x1b\S+)?)$']
buffer:'\r\n@ # '

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
File "src/unicon/bases/connection.py", line 784, in unicon.bases.connection.Connection.connect
File "src/unicon/bases/linux/connection_provider.py", line 62, in unicon.bases.linux.connection_provider.BaseLinuxConnectionProvider.connect
File "src/unicon/bases/linux/connection_provider.py", line 117, in unicon.bases.linux.connection_provider.BaseLinuxConnectionProvider.establish_connection
File "src/unicon/statemachine/statemachine.py", line 740, in unicon.statemachine.statemachine.StateMachine.go_to
unicon.core.errors.StateMachineError: Failed while bringing device to "shell" state

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
File "connectNet.py", line 21, in
server.connect()
File "src/unicon/bases/connection.py", line 791, in unicon.bases.connection.Connection.connect
unicon.core.errors.ConnectionError: failed to connect to
Failed while bringing device to "shell" state

Parallel connection to multiple devices login failure

Hello team,

I am using pyats/unicon to connect to a large testbed of devices in parallel using python threading library.

I have noticed that has I increase the parallelism of my connections, more and more devices fail to login.

The pattern is always the same:
Unicon matches the Escape character pattern Escape character is '^]'. and sends a couple of carriage returns. These carriage returns cause the Username to never be inserted on the router.
My understanding is that is should hold from sending those carriage returns because there is a Username pattern Username: which should match variable ESCAPE_CHAR_PROMPT_PATTERN:

https://github.com/CiscoTestAutomation/unicon.plugins/blob/master/src/unicon/plugins/generic/settings.py

        # Pattern to avoid sending 'enter' after Escape character pattern is seen
        self.ESCAPE_CHAR_PROMPT_PATTERN = r'.*(User Access Verification|sername:\s*$|assword:\s*$|login:\s*$|The highlighted entry will)'

However, maybe because of the parallelism, there is a context switch and the username pattern is seen later then it should and we end up sending the carriage returns, resulting in a failed authentication.

Is there any parameter that can help in this situation? Perhaps somehow delaying the carriage returns?

Logs are below:

Connected to 10.4.147.1.

Escape character is '^]'.

2024-03-04` 09:24:21,737: %UNICON-DEBUG: Statement matched: '[pattern='Escape character is .*\n', action=escape_char_callback, args=None, loop_continue=True, continue_timer=False, trim_buffer=True]'

2024-03-04 09:24:22,113: %UNICON-DEBUG: >>> Unicon Sending (target=PAN_ASR920_PEN_HUB):: '\r'

2024-03-04 09:24:22,666: %UNICON-DEBUG: >>> Unicon Sending (target=PAN_ASR920_PEN_HUB):: '\r'

Username:

2024-03-04 09:24:23,931: %UNICON-DEBUG: <<< Unicon Got (target=PAN_ASR920_PEN_HUB):: '\r\n\r\n\r\nUsername:'`
2024-03-04 09:24:23,982: %UNICON-DEBUG: *** Pattern Matched (target=PAN_ASR920_PEN_HUB):: '^.*([Uu]sername|[Ll]ogin): ?$'
2024-03-04 09:24:23,983: %UNICON-DEBUG: >>> Unicon Sending (target=PAN_ASR920_PEN_HUB):: 'Renato.Barros\r'
% Authentication failed
Username:

Can not connect to Nexus switch with VDC - connection hangs

Here's the log:

2021-07-15 19:22:12,063: %UNICON-INFO: +++ initializing handle +++

2021-07-15 19:22:12,075: %UNICON-INFO: connection to 0055-router-7018-name-replaced
ssh -l username 10.192.255.23
Role changed to: netops-user/PRD_NETOPS
Z Access to this device or the attached
networks is prohibited without express written permission.
Violators will be prosecuted to the fullest extent of both civil
and criminal law.
ZPassword: 
Cisco Nexus Operating System (NX-OS) Software
TAC support: http://www.cisco.com/tac
Copyright (c) 2002-2016, Cisco Systems, Inc. All rights reserved.
The copyrights to certain works contained in this software are
owned by other third parties and used and distributed under
license. Certain components of this software are licensed under
the GNU General Public License (GPL) version 2.0 or the GNU
Lesser General Public License (LGPL) Version 2.1. A copy of each
such license is available at
http://www.opensource.org/licenses/gpl-2.0.php and
http://www.opensource.org/licenses/lgpl-2.1.php

0055-router-7018-name-replaced# 

2021-07-15 19:22:16,162: %UNICON-INFO: We may be on a VDC, checking
show vdc


Switchwide mode is m1 f1 m1xl f2 m2xl f2e f3 

vdc_id  vdc_name                          state               mac                 type        lc      
------  --------                          -----               ----------          ---------   ------  
2       0055-router-7018-name-replaced          active              04:c5:a4:e8:37:c2   Ethernet    m1 m1xl m2xl 


0055-router-7018-name-replaced# 

2021-07-15 19:22:16,332: %UNICON-INFO: Current VDC 0055-router-7018-name-replaced

2021-07-15 19:22:16,333: %UNICON-INFO: +++ initializing handle +++



0055-router-7018-name-replaced#

Exception:

Issue with the parser show interface

Traceback (most recent call last):
  File "src/genie/cli/commands/parser.py", line 339, in genie.cli.commands.parser.ParserCommand.parse
  File "src/genie/conf/base/device.py", line 523, in genie.conf.base.device.Device.parse
  File "src/genie/conf/base/device.py", line 554, in genie.conf.base.device.Device._get_parser_output
TypeError: device is not connected, output must be provided.

Commenting out lines 49-56 in /unicon/plugins/nxos/connection_provider.py fixes the issue, so it has something to do with the renaming of the hostname.

SROS plugin does not work correctly with json output

It is possible to get configuration as json string via cli with command:
"admin show configuration json ....".
Currently the mdcli prompt regex breaks the output if there is json array in the response.

In release 21.x the context part seems be updated to include the path not just the brackets without any content.
Like [/] if in the root level.
So that probably would make easier to match correctly with regex.

In order to keep backward compatibility, I think that the current regex could be modified to not allow carriage return or new line in the context part. Like this:

--- a/src/unicon/plugins/sros/patterns.py
+++ b/src/unicon/plugins/sros/patterns.py
@@ -9,6 +9,6 @@ class SrosPatterns(UniconCorePatterns):
         super().__init__()
         self.continue_connect = r'Are you sure you want to continue connecting \(yes/no(/\[fingerprint\])?\)'
         self.permission_denied = r'^Permission denied, please try again\.\s?$'
-        self.mdcli_prompt = r'^(.*?)\[.*\][\r\n]+[AB]:.*@%N#\s?$'
+        self.mdcli_prompt = r'^(.*?)\[(?![\r\n]).*\][\r\n]+[AB]:.*@%N#\s?$'
         self.classiccli_prompt = r'^(.*?)\*?[AB]:%N(>.*)?#\s?$'
         self.discard_uncommitted = r'Discard uncommitted changes\? \[y,n\]'

Any forum or discord channel?

Is there a forum or discord channel to discuss development and change to this project?

sros md-cli: premature end when issuing command

Hi,

And thank you for making unicon available. I encountered an issue when trying to dump the configuration of a sros router using md-cli. More specifically,

the connection object is defined like:

uni_conn = unicon.Connection(
    hostname="some-name",
    os="sros",
    start=["{} {}".format("ssh", "x.x.x.x")],
    learn_hostname=True,
    credentials={
        "default": {
            "username": os.environ["NETWORK_USERNAME"],
            "password": os.environ["NETWORK_PASSWORD"],
        }
    },
    init_config_commands=[],
    log_stdout=True,
    debug=True,
    connection_timeout=10,
    overwrite_settings=False,
    settings={"DEFAULT_CLI_ENGINE": "mdcli"},
)

The command issued is:
(the purpose is to dump the router's configuration)

uni_conn.execute("admin show configuration flat | no-more", timeout=10)

But, the output stops being produced, the last lines look like this:

...
    configure { policy-options as-path-group "BOGON_ASNS_IN" }
    configure { policy-options as-path-group "BOGON_ASNS_IN" entry 1 }
    configure { policy-options as-path-group "BOGON_ASNS_IN" entry 1 expression ".* 0 .*" }
    configure { policy-options as-path-group "BOGON_ASNS_IN" entry 2 }
    configure { policy-options as-path-group "BOGON_ASNS_IN" entry 2 expression ".* 23456 .*" }
    configure { policy-options as-path-group "BOGON_ASNS_IN" entry 3 }
    configure { policy-options as-path-group "BOGON_ASNS_IN" entry 3 expression ".*

While the router's configuration is:

...
    configure { policy-options as-path-group "BOGON_ASNS_IN" }
    configure { policy-options as-path-group "BOGON_ASNS_IN" entry 1 }
    configure { policy-options as-path-group "BOGON_ASNS_IN" entry 1 expression ".* 0 .*" }
    configure { policy-options as-path-group "BOGON_ASNS_IN" entry 2 }
    configure { policy-options as-path-group "BOGON_ASNS_IN" entry 2 expression ".* 23456 .*" }
    configure { policy-options as-path-group "BOGON_ASNS_IN" entry 3 }
    configure { policy-options as-path-group "BOGON_ASNS_IN" entry 3 expression ".* [64496-64511] .*" }
    configure { policy-options as-path-group "BOGON_ASNS_IN" entry 4 }
...

It seems there is a problem with the [ character.

What the debug option prints out is:

\r\n    configure { policy-options as-path-group "BOGON_ASNS_IN" }\r\n    configure { policy-options as-path-group "BOGON_ASNS_IN" entry 1 }\r\n    configure { policy-options as-path-group "BOGON_ASNS_IN" entry 1 expression ".* 0 .*" }\r\n    configure { policy-options as-path-group "BOGON_ASNS_IN" entry 2 }\r\n    configure { policy-options as-path-group "BOGON_ASNS_IN" entry 2 expression ".* 23456 .*" }\r\n    configure { policy-options as-path-group "BOGON_ASNS_IN" entry 3 }\r\n    configure { policy-options as-path-group "BOGON_ASNS_IN" entry 3 expression ".* ',)


2022-10-29 19:46:43,132: %UNICON-DEBUG: Statement matched: '[pattern='^(.*?)\[.*\][\r\n]+[AB]:.*@some-name#\s?$', action=None, args=None, loop_continue=False, continue_timer=False, trim_buffer=True]'
2022-10-29 19:46:43,206: %UNICON-DEBUG: Statement matched: '[pattern='^(.*?)\[.*\][\r\n]+[AB]:.*@some-name#\s?$', action=None, args=None, loop_continue=False, continue_timer=False, trim_buffer=True]'
2022-10-29 19:46:43,211: %UNICON-DEBUG: >>> Unicon Sending (target=some-name):: '\r'
2022-10-29 19:46:43,229: %UNICON-DEBUG: <<< Unicon Got (target=some-name):: '\r\n'
2022-10-29 19:46:43,229: %UNICON-DEBUG: *** Pattern List (target=some-name):: 
2022-10-29 19:46:43,229: %UNICON-DEBUG: 	'.+$'
2022-10-29 19:46:43,230: %UNICON-DEBUG: 
2022-10-29 19:46:43,230: %UNICON-DEBUG: *** Pattern Matched (target=some-name):: '.+$'
2022-10-29 19:46:43,230: %UNICON-DEBUG: <unicon.eal.MatchMode mode_id=0, mode_name='search multi-line re.S'>
2022-10-29 19:46:43,230: %UNICON-DEBUG: 
[]
A:user_awesome@some-name# 
2022-10-29 19:46:43,230: %UNICON-DEBUG: <<< Unicon Got (target=some-name):: '\r\n\r\n[]\r\nA:user_awesome@some-name# '
2022-10-29 19:46:43,230: %UNICON-DEBUG: *** Pattern List (target=some-name):: 
2022-10-29 19:46:43,230: %UNICON-DEBUG: 	'^(.*?)\\*?[AB]:some-name(>.*)?#\\s?$'
2022-10-29 19:46:43,230: %UNICON-DEBUG: 	'^(.*?)\\[.*\\][\\r\\n]+[AB]:.*@some-name#\\s?$'
2022-10-29 19:46:43,230: %UNICON-DEBUG: 
2022-10-29 19:46:43,230: %UNICON-DEBUG: *** Pattern Matched (target=some-name):: '^(.*?)\\[.*\\][\\r\\n]+[AB]:.*@some-name#\\s?$'
2022-10-29 19:46:43,230: %UNICON-DEBUG: <unicon.eal.MatchMode mode_id=0, mode_name='search multi-line re.S'>
2022-10-29 19:46:43,230: %UNICON-DEBUG: *** Match Groups:: ('\r\n\r\n',)
2022-10-29 19:46:43,230: %UNICON-DEBUG: Statement matched: '[pattern='^(.*?)\[.*\][\r\n]+[AB]:.*@some-name#\s?$', action=update_cur_state, args={'transition': <unicon.statemachine.statetransition.AnyStateTransition object at 0x7f4e9ffa94f0>, 'state': mdcli}, loop_continue=False, continue_timer=False, trim_buffer=True]'
2022-10-29 19:46:43,237: %UNICON-DEBUG: >>> Unicon Sending (target=some-name):: '\x1a'
[]
A:user_awesome@some-name# 
2022-10-29 19:46:43,259: %UNICON-DEBUG: <<< Unicon Got (target=some-name):: '\r\n\r\n[]\r\nA:user_awesome@some-name# '
2022-10-29 19:46:43,259: %UNICON-DEBUG: *** Pattern List (target=some-name):: 
2022-10-29 19:46:43,259: %UNICON-DEBUG: 	'Discard uncommitted changes\\? \\[y,n\\]'
2022-10-29 19:46:43,259: %UNICON-DEBUG: 	'^(.*?)\\[.*\\][\\r\\n]+[AB]:.*@some-name#\\s?$'
2022-10-29 19:46:43,259: %UNICON-DEBUG: 
2022-10-29 19:46:43,260: %UNICON-DEBUG: *** Pattern Matched (target=some-name):: '^(.*?)\\[.*\\][\\r\\n]+[AB]:.*@some-name#\\s?$'
2022-10-29 19:46:43,260: %UNICON-DEBUG: <unicon.eal.MatchMode mode_id=0, mode_name='search multi-line re.S'>
2022-10-29 19:46:43,260: %UNICON-DEBUG: *** Match Groups:: ('\r\n\r\n',)
2022-10-29 19:46:43,260: %UNICON-DEBUG: Statement matched: '[pattern='^(.*?)\[.*\][\r\n]+[AB]:.*@some-name#\s?$', action=None, args=None, loop_continue=False, continue_timer=False, trim_buffer=True]'

Would you be so kind to look into it and advise what is going wrong?

un-desirable config commit (and not documented?) upon connection with a `iosxr` device

Hi,

when unicon connects to an iosxr device, it commits the following configuration by default:

Building configuration...
!! IOS XR Configuration 5.3.4
logging console disable
line console
 exec-timeout 0 0
 absolute-timeout 0
 session-timeout 0
!
line default
 exec-timeout 0 0
 absolute-timeout 0
 session-timeout 0
!
end

Is there a way to disable this step? I cannot find where that is documented.

PS. the same question was asked in #17 but the issue was closed...

unicon HVRP self.config_prompt wrong regex

There is a regex mistake for hvrp's unicon patterns.py file

Path: myvenv/lib/python3.8/site-packages/unicon/plugins/hvrp/patterns.py
$ character is missing at the end of the regex self.config_prompt

        self.config_prompt = r'^\[.*\]'

We have an error when we try to configure a Huawei N8000 device with pyATS

In [3]: device.configure(['interface GigabitEthernet0/7/2','shutdown'])

2022-09-29 14:46:16,117: %UNICON-INFO: +++  with via 'cli': config +++
system-view
Enter system view, return user view with return command.
[~N1-NE8000-PEAG-1]
---------------------------------------------------------------------------
TimeoutError                              Traceback (most recent call last)

TimeoutError: timeout occurred:
              timeout value: 30
              last_command: 'system-view\r'
             pattern: ['^\\[.*\\]']
             buffer:'system-view\r\nEnter system view, return user view with return command.\r\n[~N1-NE8000-PEAG-1]'

If we add $, it works

        self.config_prompt = r'^\[.*\]$'

In [6]: device.configure(['interface GigabitEthernet0/7/2','shutdown'])

2022-09-29 14:29:03,854: %UNICON-INFO: +++  with via 'cli': config +++
system-view
Enter system view, return user view with return command.
[~N1-NE8000-PEAG-1]interface GigabitEthernet0/7/2
[~N1-NE8000-PEAG-1-GigabitEthernet0/7/2]shutdown
[*N1-NE8000-PEAG-1-GigabitEthernet0/7/2]commit
[~N1-NE8000-PEAG-1-GigabitEthernet0/7/2]return
<N1-NE8000-PEAG-1>

Syslog messages interupting connection

Hi,

I've noticed that when Unicon sees a syslog message (console session) it stops and waits, in the example below, there was a RESTCONF connection made at the time Unicon has connected to device. Any actions stop from there on, connection will eventually timeout.

I'm guessing this has something to do with: SYSLOG_WAIT which by default is 1, I couldn't find any documentation around this value, could you please elaborate?

2022-11-23 11:33:17,885: %UNICON-INFO: +++ DUT-ISR4331-K9 logfile /opt/autom8_app/autom8_app/logs/pyats/DUT-ISR4331-K9.log +++

2022-11-23 11:33:17,885: %UNICON-INFO: +++ Unicon plugin iosxe (unicon.plugins.iosxe) +++
Trying 10.113.128.141...
Connected to 10.113.128.141.
Escape character is '^]'.


2022-11-23 11:33:17,914: %UNICON-INFO: +++ connection to spawn: telnet 10.113.128.141 10012, id: 140555820049808 +++

2022-11-23 11:33:17,914: %UNICON-INFO: connection to DUT-ISR4331-K9

Login: automation
Password:

2022-11-23 11:33:17,964: %UNICON-INFO: Executing post credential command: sendline()



Nov 23 11:33:46.605 GMT: %DMI-5-AUTH_PASSED: R0/0: dmiauthd: User 'automation' authenticated successfully from 10.113.128.165:0  for rest over http. External groups: PRIV15
Username:automation
Password:

Ask For Credentials - Script Fails When Large Number Of Devices

I am using the ask feature to get the password info at the beginning of the script so that I do not have to worry about storing it in a text, csv, or yaml file. I ran the exact same script with the username and password in the YAML file for 100 devices and it is successful. When I remove all of the username and passwords and put the below info in script, it fails at different devices. When I reduced the number of devices in the script, it was successful. I would think this is a Unicon issue because I am not even using pyATS to run the script, just python3 and unicon. I can also say I am using a Unicon Plugin aos device which I wrote the plugin for so it would be good to test if large number of Cisco devices fail with same type of script. The script I am running logs into a device, runs a show command, and moves on to the next device.
testbed:
credentials:
default:
password: "%ASK{Password:}"
username: TESTUSER
enable:
password: "%ASK{Enable_Password:}"

Inconsistent output format for "echo $?" command on Linux

Hello
function post_service in The Linux Execute class expects the output of the "echo $?" command to end with a carriage return character followed by a newline character (\r\n) on Linux systems.
But in the Linux system that I tested. Neither sh nor bash returns such a pattern. I think this pattern belongs to Windows. What type of shell or operating system was considered in this source code for which such regex was written?

con.sendline('echo $?')
try:
      match = con.expect(r'\r\n\d+', timeout=timeout)

Source code: unicon.plugins/linux/service_implementation.py#L61

Reload is inconsistent

Sending e.g device.reload() doesn't always produce expected result. It's sitting and waiting for something at the reload confirm prompt and eventually timeouts and prompt recovery kicks in which effectively sends a new line and allows for reload to proceed.

YAML:

test_bed = """
devices:
  $HOSTNAME:
    os: iosxe
    platform: iosxe
    type: router
    credentials:
      default:
        username: $USERNAME
        password: $PASSWORD
      terminal_server:
        username: $USERNAME
        password: $PASSWORD
      fallback:
        username: cisco
        password: cisco
    connections:
      a:
        protocol: telnet
        ip: $CONSOLE_IP
        port: $CONSOLE_PORT
        login_creds: [terminal_server, default, fallback]
        arguments:
          connection_timeout: 120
          mit: True
          cred_action:
            terminal_server:
              post: sendline()
        settings:
          SENDLINE_AFTER_CRED: terminal_server
          ESCAPE_CHAR_CHATTY_TERM_WAIT: 0.5
          ESCAPE_CHAR_PROMPT_WAIT: 1
          ESCAPE_CHAR_CHATTY_TERM_WAIT_RETRIES: 15
          EXEC_TIMEOUT: 30
          CONFIG_TIMEOUT: 30
          CONFIG_TRANSITION_WAIT: 0.5
          RELOAD_TIMEOUT: 420
          RELOAD_WAIT: 300
          POST_RELOAD_WAIT: 60
          RELOAD_RECONNECT_ATTEMPTS: 5
          SLEEP_PRE_LAUNCH: 0.2
          POST_DISCONNECT_WAIT_SEC: 0
          GRACEFUL_DISCONNECT_WAIT_SEC: 0.2
          CONSOLE_TIMEOUT: 120
"""

LOG output:

2022-11-28 11:47:58,156: %UNICON-INFO: +++ DUT-C1117-4PM logfile /opt/autom8_app/autom8_app/logs/pyats/DUT-C1117-4PM.log +++

2022-11-28 11:47:58,156: %UNICON-INFO: +++ Unicon plugin iosxe (unicon.plugins.iosxe) +++
Trying 80.194.79.121...


2022-11-28 11:47:58,169: %UNICON-INFO: +++ connection to spawn: telnet 80.194.79.121 10014, id: 140694475231472 +++

2022-11-28 11:47:58,169: %UNICON-INFO: connection to DUT-C1117-4PM
Connected to 80.194.79.121.
Escape character is '^]'.

Login: automation
Password:

2022-11-28 11:47:58,282: %UNICON-INFO: Executing post credential command: sendline()



DUT-C1117-4PM#

2022-11-28 11:47:58,396: %UNICON-INFO: Learned hostname(s): 'DUT-C1117-4PM'.

2022-11-28 11:47:58,460: %UNICON-INFO: +++ DUT-C1117-4PM with via 'a': enable +++

2022-11-28 11:47:58,526: %UNICON-INFO: +++ DUT-C1117-4PM with via 'a': executing command 'write erase' +++
write erase
************************************************************************************************************
Erasing Nvram will not clear license trust code.
************************************************************************************************************
Erasing the nvram filesystem will remove all configuration files! Continue? [confirm]
[OK]
Erase of nvram: complete
DUT-C1117-4PM#

2022-11-28 11:48:00,922: %UNICON-INFO: +++ DUT-C1117-4PM with via 'a': reload +++

2022-11-28 11:48:00,985: %UNICON-INFO: +++ reloading DUT-C1117-4PM  with reload_command 'reload' and timeout is 420 seconds +++
reload

System configuration has been modified. Save? [yes/no]: n
WARNING:
Boot variable either does not exist or buffer is too small
This may impact autoboot of the router. Proceed with caution
Do you wish to proceed with reload anyway[confirm]

2022-11-28 11:55:00,991: %UNICON-WARNING: Timeout of 420 seconds has been reached.
Prompt Recovery has commenced. Total timeout occurs in 100 seconds.

2022-11-28 11:55:00,994: %UNICON-INFO: Sending prompt recovery command: b'\r'

Proceed with reload? [confirm]
2022-11-28 11:55:11,007: %UNICON-INFO: Sending prompt recovery command: b'\x15'

2022-11-28 11:55:21,028: %UNICON-INFO: Sending prompt recovery command: b'\x1a'

2022-11-28 11:55:31,036: %UNICON-INFO: Sending prompt recovery command: b'\r'
Rom image verified correctly