Giter Club home page Giter Club logo

ansible-pyats's Introduction

ansible-pyats

ansible-genie is a implementation of the pyATS network testing framework in an Ansible role. It contains modules, filters, and tasks:

  • Run a command and get structured output
  • "snapshot" the output of a command and save it to a file
  • Compare the current output of a command to a previous "snapshot"

Installation

First, install the Python dependencies:

$ pip install pyats genie
<snip>
Installing collected packages: pyats, genie
Successfully installed genie-19.9 pyats-19.9.2

pyATS and Genie require Python >=3.4.

Manual

For manual installation, you can just clone the repository into your ANSIBLE_ROLES_PATH:

$ git clone https://github.com/CiscoDevNet/ansible-pyats "${ANSIBLE_ROLES_PATH:-roles}/ansible-pyats"
Cloning into 'roles/ansible-pyats'...
remote: Enumerating objects: 83, done.
remote: Counting objects: 100% (83/83), done.
remote: Compressing objects: 100% (56/56), done.
remote: Total 83 (delta 28), reused 56 (delta 12), pack-reused 0
Unpacking objects: 100% (83/83), done.

Ansible Galaxy

If you are using Ansible Galaxy, you can use this role by adding the following to your requirements.yml:

- src: https://github.com/CiscoDevNet/ansible-pyats
  scm: git
  name: ansible-pyats

Next, install your Galaxy dependencies:

$ ansible-galaxy install -r requirements.yml -p "${ANSIBLE_ROLES_PATH:-roles}"

Modules

  • pyats_parse_command: Run a command on a remote device and return the structured output

Filters

  • pyats_parser: provides structured data from unstructured command output
  • pyats_diff: provides the difference between two data structures

Example Playbooks

Run a command and retrieve the structured output

- hosts: router
  connection: network_cli
  gather_facts: no
  roles:
    - ansible-pyats
  tasks:
    - pyats_parse_command:
        command: show ip route bgp
      register: output

    - debug:
        var: output.structured

Snapshot the output of a command to a file

- hosts: router
  connection: network_cli
  gather_facts: no
  roles:
    - ansible-pyats
  tasks:
    - include_role:
        name: ansible-pyats
        tasks_from: snapshot_command
      vars:
        command: show ip route
        file: "{{ inventory_hostname }}_routes.json"

Role Variables

  • command: the command to run on the device
  • file: the name of the file in which to store the command "shapshot"

Compare the output of a command with a previous snapshot

- hosts: router
  connection: network_cli
  gather_facts: no
  roles:
    - ansible-pyats
  tasks:
    - include_role:
        name: ansible-pyats
        tasks_from: compare_command
      vars:
        command: show ip route
        file: "{{ inventory_hostname }}_routes.json"

Role Variables

  • command: the command to run on the device
  • file: the name of the file in which to store the command "shapshot"

Using the pyats_parser filter directly

- hosts: router
  connection: network_cli
  gather_facts: no
  roles:
    - ansible-pyats
  tasks:
    - name: Run command
      cli_command:
        command: show ip route
      register: cli_output
    
    - name: Parsing output
      set_fact:
        parsed_output: "{{ cli_output.stdout | pyats_parser('show ip route', 'iosxe') }}"

Using the pyats_diff filter directly

- name: Diff current and snapshot
  set_fact:
    diff_output: "{{ current_output | pyats_diff(previous_output) }}"

Change ACL configuration and compare before and after configs using genie_config_diff

---

- hosts: cisco
  gather_facts: no
  connection: network_cli

  tasks:
    - name: collect config (before)
      ios_command:
        commands:
          - show run
      register: result_before

    - name: load new acl into device
      ios_config:
        lines:
          - permit ip host 192.168.114.1 any
          - permit ip host 192.168.114.2 any
          - permit ip host 192.168.114.3 any
        parents: ip access-list extended test
        save_when: modified

    - name: collect config (after)
      ios_command:
        commands:
          - show run
      register: result_after

    - name: debug
      debug:
        msg: "{{ result_before.stdout[0] | genie_config_diff(result_after.stdout[0], mode='add', exclude=exclude_list) }}"

  vars:
    exclude_list:
      - (^Using.*)
      - (Building.*)
      - (Current.*)
      - (crypto pki certificate chain.*)

The argument mode can be add (displays added commands in result_after), remove (displays removed commands in result_after), or modified (displays modified commands). If mode argument is not specified, added, removed, and modified commands are displayed.
The argument exclude means command lists which is excluded when comparing before and after configs. In the playbook example above, variable excluded_list, which is defined as the play variable, is used.

Other examples

        msg: "{{ result_before.stdout[0] | genie_config_diff(result_after.stdout[0]) }}"
        msg: "{{ result_before.stdout[0] | genie_config_diff(result_after.stdout[0], mode='remove') }}"
        msg: "{{ result_before.stdout[0] | genie_config_diff(result_after.stdout[0], exclude=exclude_list) }}"

The result of example playbook

PLAY [cisco] **********************************************************************************

TASK [collect config (before)] ****************************************************************
ok: [test3]

TASK [load new acl into device] ***************************************************************
ok: [test3]

TASK [collect config (after)] *****************************************************************
ok: [test3]

TASK [debug] **********************************************************************************
ok: [test3] => {
    "msg": [
        "ip access-list extended test:",
        "+ permit ip host 192.168.114.1 any: ",
        "+ permit ip host 192.168.114.2 any: ",
        "+ permit ip host 192.168.114.3 any: "
    ]
}

PLAY RECAP ************************************************************************************
test3                      : ok=4    changed=1    unreachable=0    failed=0    skipped=0    rescued=0    ignored=0   

Compare show commands using genie_parser_diff

This filter can compare the output of show commands parsed by Genie parser. The arguments mode and exclude also can be used.

    - name: debug
      debug:
        msg: "{{ sh_int_parsed_before | genie_parser_diff(sh_int_parsed_after, mode='modified', exclude=exclude_list) }}"
        
  vars:
    exclude_list:
      - (.*in_octets.*)
      - (.*in_pkts.*)
      - (.*out_octets.*)
      - (.*out_pkts.*)

License

Cisco Sample License

ansible-pyats's People

Contributors

rschmied avatar stevenca avatar supertylerc avatar tachashi avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

ansible-pyats's Issues

ImportError: No module named genie.libs.parser.utils

Hi I am receiving this issue when running the playbook with pyats_parse_command

Already installed genie.metaparser without success.

TASK [pyats_parse_command] ***************************************************************************************************************************************************************************
task path: ///nexus/franquezaj/pyats_test.yaml:8
<10.131.177.117> attempting to start connection
<10.131.177.117> using connection plugin network_cli
<10.131.177.117> local domain socket does not exist, starting it
<10.131.177.117> control socket path is /home/dpko.un.org/franquezaj-su/.ansible/pc/174c8aa783
<10.131.177.117> local domain socket listeners started successfully
<10.131.177.117> loaded cliconf plugin ios from path /usr/local/lib/python3.6/site-packages/ansible/plugins/cliconf/ios.py for network_os ios
<10.131.177.117>
<10.131.177.117> local domain socket path is /home/dpko.un.org/franquezaj-su/.ansible/pc/174c8aa783
<10.131.177.117> ESTABLISH LOCAL CONNECTION FOR USER: franquezaj-su
<10.131.177.117> EXEC /bin/sh -c '( umask 77 && mkdir -p "echo /home/dpko.un.org/franquezaj-su/.ansible/tmp/ansible-local-33632kydch6pg/ansible-tmp-1594909721.7643833-114462321998527" && echo ansible-tmp-1594909721.7643833-114462321998527="echo /home/dpko.un.org/franquezaj-su/.ansible/tmp/ansible-local-33632kydch6pg/ansible-tmp-1594909721.7643833-114462321998527" ) && sleep 0'
<10.131.177.117> Attempting python interpreter discovery
<10.131.177.117> EXEC /bin/sh -c 'echo PLATFORM; uname; echo FOUND; command -v '"'"'/usr/bin/python'"'"'; command -v '"'"'python3.7'"'"'; command -v '"'"'python3.6'"'"'; command -v '"'"'python3.5'"'"'; command -v '"'"'python2.7'"'"'; command -v '"'"'python2.6'"'"'; command -v '"'"'/usr/libexec/platform-python'"'"'; command -v '"'"'/usr/bin/python3'"'"'; command -v '"'"'python'"'"'; echo ENDFOUND && sleep 0'
<10.131.177.117> EXEC /bin/sh -c '/usr/bin/python && sleep 0'
Using module file /home/reponeg/nexus/franquezaj/roles/ansible-pyats/library/pyats_parse_command.py
<10.131.177.117> PUT /home/dpko.un.org/franquezaj-su/.ansible/tmp/ansible-local-33632kydch6pg/tmp1d7xb3e5 TO /home/dpko.un.org/franquezaj-su/.ansible/tmp/ansible-local-33632kydch6pg/ansible-tmp-1594909721.7643833-114462321998527/AnsiballZ_pyats_parse_command.py
<10.131.177.117> EXEC /bin/sh -c 'chmod u+x /home/dpko.un.org/franquezaj-su/.ansible/tmp/ansible-local-33632kydch6pg/ansible-tmp-1594909721.7643833-114462321998527/ /home/dpko.un.org/franquezaj-su/.ansible/tmp/ansible-local-33632kydch6pg/ansible-tmp-1594909721.7643833-114462321998527/AnsiballZ_pyats_parse_command.py && sleep 0'
<10.131.177.117> EXEC /bin/sh -c '/usr/bin/python /home/dpko.un.org/franquezaj-su/.ansible/tmp/ansible-local-33632kydch6pg/ansible-tmp-1594909721.7643833-114462321998527/AnsiballZ_pyats_parse_command.py && sleep 0'
<10.131.177.117> EXEC /bin/sh -c 'rm -f -r /home/dpko.un.org/franquezaj-su/.ansible/tmp/ansible-local-33632kydch6pg/ansible-tmp-1594909721.7643833-114462321998527/ > /dev/null 2>&1 && sleep 0'
The full traceback is:
Traceback (most recent call last):
File "/home/dpko.un.org/franquezaj-su/.ansible/tmp/ansible-local-33632kydch6pg/ansible-tmp-1594909721.7643833-114462321998527/AnsiballZ_pyats_parse_command.py", line 102, in
_ansiballz_main()
File "/home/dpko.un.org/franquezaj-su/.ansible/tmp/ansible-local-33632kydch6pg/ansible-tmp-1594909721.7643833-114462321998527/AnsiballZ_pyats_parse_command.py", line 94, in _ansiballz_main
invoke_module(zipped_mod, temp_path, ANSIBALLZ_PARAMS)
File "/home/dpko.un.org/franquezaj-su/.ansible/tmp/ansible-local-33632kydch6pg/ansible-tmp-1594909721.7643833-114462321998527/AnsiballZ_pyats_parse_command.py", line 40, in invoke_module
runpy.run_module(mod_name='ansible.modules.pyats_parse_command', init_globals=None, run_name='main', alter_sys=True)
File "/usr/lib64/python2.7/runpy.py", line 176, in run_module
fname, loader, pkg_name)
File "/usr/lib64/python2.7/runpy.py", line 82, in _run_module_code
mod_name, mod_fname, mod_loader, pkg_name)
File "/usr/lib64/python2.7/runpy.py", line 72, in _run_code
exec code in run_globals
File "/tmp/ansible_pyats_parse_command_payload_zFifX8/ansible_pyats_parse_command_payload.zip/ansible/modules/pyats_parse_command.py", line 16, in
ImportError: No module named genie.libs.parser.utils
fatal: [10.131.177.117]: FAILED! => {
"ansible_facts": {
"discovered_interpreter_python": "/usr/bin/python"
},
"changed": false,
"module_stderr": "Traceback (most recent call last):\n File "/home/dpko.un.org/franquezaj-su/.ansible/tmp/ansible-local-33632kydch6pg/ansible-tmp-1594909721.7643833-114462321998527/AnsiballZ_pyats_parse_command.py", line 102, in \n _ansiballz_main()\n File "/home/dpko.un.org/franquezaj-su/.ansible/tmp/ansible-local-33632kydch6pg/ansible-tmp-1594909721.7643833-114462321998527/AnsiballZ_pyats_parse_command.py", line 94, in _ansiballz_main\n invoke_module(zipped_mod, temp_path, ANSIBALLZ_PARAMS)\n File "/home/dpko.un.org/franquezaj-su/.ansible/tmp/ansible-local-33632kydch6pg/ansible-tmp-1594909721.7643833-114462321998527/AnsiballZ_pyats_parse_command.py", line 40, in invoke_module\n runpy.run_module(mod_name='ansible.modules.pyats_parse_command', init_globals=None, run_name='main', alter_sys=True)\n File "/usr/lib64/python2.7/runpy.py", line 176, in run_module\n fname, loader, pkg_name)\n File "/usr/lib64/python2.7/runpy.py", line 82, in _run_module_code\n mod_name, mod_fname, mod_loader, pkg_name)\n File "/usr/lib64/python2.7/runpy.py", line 72, in _run_code\n exec code in run_globals\n File "/tmp/ansible_pyats_parse_command_payload_zFifX8/ansible_pyats_parse_command_payload.zip/ansible/modules/pyats_parse_command.py", line 16, in \nImportError: No module named genie.libs.parser.utils\n",
"module_stdout": "",
"msg": "MODULE FAILURE\nSee stdout/stderr for the exact error",
"rc": 1
}

How specify variables custom for devices

Hi, i would to use this module to extract the inventory for cisco c9200 e c9300 devices.

In genie project, i opened the issue CiscoTestAutomation/genieparser#129 and i should to specify these variables to executing correctly the parsing.

custom:
  abstraction:
    order:
    - os
    - platform
os: iosxe
type: iosxe
platform: 'c9300'

How i can specify the same variables in the playbook when i call the role ?

Thanks
Gerardo

ERROR! A worker was found in a dead state

I am testing simple "show ntp peer-status" with pyats filter command. Here is my playbook:

- name: Play to test command module against NX OS devices
  hosts: leafs
  connection: network_cli
  vars: 
    ansible_python_interpreter: '/usr/bin/env python'
  roles:
    - ansible-pyats
  tasks:
    - name: gather NTP peer status
      cli_command:
        command: 'show ntp peer-status'
      register: ntp_stats
            
    - name: print out ntp-peer status
      debug:
        msg: "{{ ntp_stats.stdout | pyats_parser('show ntp peer-status','nxos') }}"

the module was able to successfully login, gather the ouput, but failed when I passed it through the parser "pyats_parser".

Below is the task output with errors at the end

ok: [10.255.130.142] => changed=false 
  invocation:
    module_args:
      answer: null
      check_all: false
      command: show ntp peer-status
      newline: true
      prompt: null
      sendonly: false
  stdout: |-
    Total peers : 1
    * - selected for sync, + -  peer mode(active),
    - - peer mode(passive), = - polled in client mode
        remote                                 local                                   st   poll   reach delay   vrf
    -----------------------------------------------------------------------------------------------------------------------
    +10.255.0.1                              0.0.0.0                                  2   16       1   0.00058management
  stdout_lines: <omitted>
objc[87625]: +[__NSCFConstantString initialize] may have been in progress in another thread when fork() was called.
objc[87625]: +[__NSCFConstantString initialize] may have been in progress in another thread when fork() was called. We cannot safely call it or ignore it in the fork() child process. Crashing instead. Set a breakpoint on objc_initializeAfterForkError to debug.
ERROR! A worker was found in a dead state

My ansible version info

ansible [core 2.11.1] 
  config file = /Users/zhangp/development/aci-ansible/ansible.cfg
  configured module search path = ['/Users/zhangp/.ansible/plugins/modules', '/usr/share/ansible/plugins/modules']
  ansible python module location = /Users/zhangp/development/aci-ansible/.venv/lib/python3.7/site-packages/ansible
  ansible collection location = /Users/zhangp/.ansible/collections:/usr/share/ansible/collections
  executable location = /Users/zhangp/development/aci-ansible/.venv/bin/ansible
  python version = 3.7.9 (default, Jun 20 2021, 18:47:05) [Clang 12.0.0 (clang-1200.0.32.27)]
  jinja version = 2.11.3
  libyaml = True

Support for more than 100 Ports

Looking at the output for the PyATS config command, we encountered the following:

"interface Ethernet1/98": {},
"interface Ethernet1/99": {},
"interface Vlan1": {
"description Mandatory": {},
"no ip redirects": {},
"no ipv6 redirects": {}
},

As this is a Cisco Nexus 93360YC-FX2, the ports go up to 96 but an additional 12 Fiber ports make it up to 108.

Error when parsing output for command: "show running-config vrf {{ vrf }} | sec '^vrf'"

When using the pyats_parse_command Ansible module with the command show running-config vrf {{ vrf }} | sec '^vrf', the following exception is raised:

The full traceback is:
  File "/tmp/ansible_pyats_parse_command_payload_eairedkh/__main__.py", line 92, in main
    parsed_output = device.parse(module.params['command'], output=response)
  File "src/genie/conf/base/device.py", line 552, in genie.conf.base.device.Device.parse
  File "src/genie/conf/base/device.py", line 588, in genie.conf.base.device.Device._get_parser_output

fatal: [switch_1]: FAILED! => {
    "changed": false,
    "invocation": {
        "module_args": {
            "answer": null,
            "command": "show running-config vrf myVRF | sec '^vrf'",
            "prompt": null,
            "sendonly": false
        }
    },
    "msg": "Unable to parse output for command 'show running-config vrf myVRF | sec '^vrf'' (cli() got an unexpected keyword argument 'output')"

I've tested using the identical command with the pyATS/Genie Python library within a Python script and it is able to parse the output successfully.

Model: Cisco Nexus9000
NXOS: version 7.0(3)I7(3)

Support Multiple Commands

It would be great if this role and its tasks supported multiple commands at once. An example playbook:

---

- name: Check for any differences after two minutes
  hosts: all
  vars:
    commands:
      - show version
      - show ip bgp summary
      - show ip route
  roles:
    - ansible-pyats
  tasks:
    - include_role:
        name: ansible-pyats
        tasks_from: snapshot_command
      vars:
        command: "{{ commands }}"
        file: "{{ inventory_hostname }}.json"
    - pause:
        minutes: 2
    - include_role:
        name: ansible-pyats
        tasks_from: compare_command
      vars:
        command: "{{ commands }}"
        file: "{{ inventory_hostname }.json"

Something similar to this is possible with NAPALM, which accepts a list of commands, or with ios_commands, which also accepts a list of commands. However, I don't think it's currently possible with the modules in this repo. It would be great if these modules supported multiple commands. The old with_items and loop patterns appear to be severely discouraged (see the apt module, for example), so it would be best to not have to resort to those options.

Add installation hint into Readme

Aloha,
im ist a beginner to Ansible and network automation.
So it wasn't clear for me, how to "install" this Ansible filter.
Only after I tried the clay584/parse_genie filter, I understood, that the "installation has to be just git clone this repo into a valid Ansible role directory.
Maybe a hint in the readme for this would help beginners?
Cheers
Marcel

How to specify exclude keys in playbook?

Hi. I have two questions to use genie.py.

1, Could I use genie_parser and genie_diff with log files?
for example:

- name: Read in snaphot file
  set_fact:
    snapshot_data1: "{{ lookup('file', before_snapshot, errors='ignore') }}"
    snapshot_data2: "{{ lookup('file', after_snapshot, errors='ignore') }}"

- name: before parsing output
  set_fact:
    before_parsed_output: "{{ before_snapshot | pyats_parser(command, 'nxos') }}"

- name: after parsing output
  set_fact:
    after_parsed_output: "{{ after_snapshot | pyats_parser(command, 'nxos') }}"

- name: Diff current and snapshot
  set_fact:
    diff_output: "{{ before_parsed_output | pyats_diff(after_parsed_output, exclude=exclude_list) }}"

- name: debug
  copy:
    content: "{{ diff_output }}"
    dest: "./log/genie_diff.txt"

This playbook is using 'show ip route vrf all'.
Once I run this playbook, I got an error message which is "AttributeError: 'dict' object has no attribute 'split'".

Is it because of using log files? Do I need to execute command and parsing by using 'stdout'?

2, How can I exclude unnecessary keys from the diff result?
Now I'm using genie.py as following below.
I commented out a part of genie.py and using it because of an error an error occured as I mentioned.

#config1 = Config(output1)
        #config1.tree()
        #dict1 = config1.config
#
        #config2 = Config(output2)
        #config2.tree()
        #dict2 = config2.config

        dd = Diff(output1, output2, mode=mode, exclude=exclude)
        dd.findDiff()
        #diff = str(dd)
        #diff_list = diff.split('\n')
        return dd

And I create a exclude list.

exclude_list:
  - updated

But it could not work.

I also tried "- vrf", "- routes" and "- next_hop", but "- next_hop" were not work well.

What I want to actual exclude key is 'updated'.
How can I do that?

Thanks!

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.