Giter Club home page Giter Club logo

ansible-role-voms-client's Introduction

EGI VOMS Client

Docker Repository on Quay

General information

About VOMS and VOs

This is an Ansible role which configures VOMS clients. VOMS is a web service for managing membership of Virtual Organisations. VOMS clients are necessary to obtain authorisation (in the form of short-lived proxies) for interacting with specific services, based on VO membership. VOMS clients are set of command-line utilities which send authenticated requests to the relevant VOMS server in order to request authorisation from them.

In order to use the VOMS client, an individual needs to

  • have a personal x.509 certificate
  • be registered in the VO they want to get an authorisation for

VOMS clients are typically installed on the User Interface or Worker Node profiles.

Configuration

Configuration of the VOMS clients is done with a few files:

  1. .lsc files
  2. vomses files

See the VOMS documentation for more detailed information.

For every VO you wish to interact with, the relevant configuration needs to be in place. This can be quite a time-consuming task, especially in cases where a site administrator does not know a-priori which VOs to configure.

To make life bearable, we take a data-driven approach.

The necessary data is available via the EGI Operations Portal API, which is used in this role as a data source. This allows us to configure all VOs registered in the Operations Portal in one foul swoop. Two approaches could be taken to generating the configuration:

  1. configuration from raw data pulled from Lavoisier at Ansible runtime
  2. configuration from filtered data pulled from Lavoisier prior to Ansible runtime.

In the former approach, a well-crafted json_query could be used to iterate over the data returned from Lavoisier. The query in this case needs to reflect the complexity and structure of the data object returned by Lavoisier, which cannot be assumed to return an array of consistent data. In the latter approach, a much simpler is used to iterate over a cached data object, which has been filtered to exclude items which do not contain the relevant information. This cached data can be easily created by a simple python script - files/create_clean_vo_data.py which reads the role vars and creates a local cache of data. The data format has been chosen to be YAML so that we can add it to the repository and keep track of changes - this would be difficult with JSON, due to the lack of lines.

We have opted for the latter (see 4215026e18c) for the following reasons:

  1. It is easier to maintain a well-documented script than a complex json query.
  2. It is easier to read a well-documented script than a complex json query
  3. If the role is added as a dependency to playbooks (as will certainly be the case, since the voms clients are used all over the place), the data needs to be present.

There is however the drawback that the data in the repo can quickly become out of synch with the actual data on Lavoisier. This could happen either by individuals editing the cache by hand, or by the maintainer not running the script when necessary. The only way to overcome this is to maintain a strong test suite.

Updating the VO data

In order to update the VO data using files/create_clean_vo_data.py, an authentication token is required to interact with the EGI Operations Portal API.

The token can be generated by accessing, while being authenticated via EGI Check-in, the Operations Portal API documentation page, follow the instructions on the page, and then export the token into the environment before running files/create_clean_vo_data.py.

It is possible to test that the token is working using a curl call:

# Exporting Operations Portal API token
$ export OPS_PORTAL_API_TOKEN='...'
# Testing an API call using curl
$ curl -X GET "https://operations-portal.egi.eu/api/vo-voms/json" \
    -H "Accept: application/json" \
    -H "X-API-Key: $OPS_PORTAL_API_TOKEN"

Once the curl call is confirmed to work, it's possible to use the provided script:

# Exporting Operations Portal API token
$ export OPS_PORTAL_API_TOKEN='...'
# Updating the VO data
$ ./files/create_clean_vo_data.py

Testing

The role is tested with molecule for the following scenarios:

Tests cover unit and integration tests, but not functional tests, since a personal certificate is necessary for using the VOMS client. Specific tests included are:

  • presence of binary executables
  • presence of configuration directories
  • contents of configuration files for selected VOs

Requirements

See requirements.txt.

Role Variables

Role variables kept on defaults/main.yml include:

  • prerequisites - the prerequisite packages on an OS-basis
  • voms_dir, vomses_dir - directory locations on the target host which contain the voms information
  • lavoisier - the lavoisier framework endpoints necessary for extracting the data necessary to populate the configuration files.

There is no need to change the default variables.

Dependencies

Dependencies are not explicitly declared in the metadata, but this role depends on the UMD role:

- { role: EGI-Foundation.umd, release: 4 }

Example Playbook

- hosts: servers
  roles:
    - { role: EGI-Foundation.umd, release: 4 }
    - { role: EGI-Foundation.voms-client }

License

Apache-2.0

Author Information

See AUTHORS.md

ansible-role-voms-client's People

Contributors

brucellino avatar dependabot[bot] avatar enolfc avatar gwarf avatar

Stargazers

 avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar

ansible-role-voms-client's Issues

Malformed LSC file during the generation of the proxy

Hi,

During the creation of a proxy (with any VO) I have the following error message:
org.italigrid.voms.VOMSError: LSC fiel parsing error: Malformed LSC file (vo=vo.grif.fr, host=grid12.lal.in2p3.fr): No distinguished name entries found

Thabn you.
Cheers, Giuseppe

Implement a means to filter the VoVoms JSON array based on "bad" VOs

Some VOs do not have vomses.
The json array passed by lavoisier contains all VOs, some of which we know the task will fail on, since they have a nonstandard structure - this causes the json_query lookup to fail.
At the moment (pre v0.1.0), we have a dirty hack to run the configure tasks across known-good slices of the array. It would be nice to pass known-bad items to the query in order to exclude them.

Prepare release v0.1.0

A branch has been created for release v0.1.0-rc.

We need to check the following before merging it into master and tagging the release:

  • Community Health check
  • Builds passing
  • Test coverage described and implemented
  • All necessary documentation is in place
  • Example realistic playbook provided
  • Repository enabled in Zenodo
  • Role enabled in Galaxy
  • Authors file up to date

Missing entries in VOMS Certs JSON

The VOMS certs json given by http://cclavoisier01.in2p3.fr:8080/lavoisier/voms-certificates?accept=json has some entries which do not follow the general structure:

{
    "expiry": "Sat May 04 22:57:06 CEST 2019",
    "host": "voms.fnal.gov",
    "X509Cert": [{
      "DN": ["/DC=org/DC=opensciencegrid/O=Open Science Grid/OU=Services/CN=voms1.fnal.gov"]
    }, {
      "CA_DN": ["/DC=org/DC=cilogon/C=US/O=CILogon/CN=CILogon OSG CA 1"]
    }, {
      "X509PublicKey": ["-----BEGIN CERTIFICATE-----\nMIIEQDCCAyigAwIBAgIFAQAAwo0wDQYJKoZIhvcNAQELBQAwaDETMBEGCgmSJomT8ixkARkWA29y\nZzEXMBUGCgmSJomT8ixkARkWB2NpbG9nb24xCzAJBgNVBAYTAlVTMRAwDgYDVQQKEwdDSUxvZ29u\nMRkwFwYDVQQDExBDSUxvZ29uIE9TRyBDQSAxMB4XDTE4MDQwNDIwNTIwNloXDTE5MDUwNDIwNTcw\nNlowfjETMBEGCgmSJomT8ixkARkWA29yZzEfMB0GCgmSJomT8ixkARkWD29wZW5zY2llbmNlZ3Jp\nZDEaMBgGA1UEChMRT3BlbiBTY2llbmNlIEdyaWQxETAPBgNVBAsTCFNlcnZpY2VzMRcwFQYDVQQD\nEw52b21zMS5mbmFsLmdvdjCCASIwDQYJKoZIhvcNAQEBBQADggEPADCCAQoCggEBANrCRdvstr6i\nc7JWeH9v9pTl9TWcNnhjUsqpmfLDsdHnmoW1JYC6JFXCZTpFbS2Fz0c9Q7xgP2M9/WBgHzauHdIN\nwLqJy3oxVa0KIN+gt/NxfwxO5Jy76zY1eg8esI0ZilQjT13GOJsyxNr+0CmP9+I+WOgTXTd/RVG5\njpZyNO7+tH+uUuuIR/hXJyWPxd7xqhxOJ1A9IK3N34WR2p2lQD7nXz2AogFFULmh0EV/xftzkk4G\nRixKU7SZtCd+rYI1Z9NvQosrISm0mOXWORRC/bfovKwLWWmibeWXMbJd+8mC53mQFZRuaLd4m7ph\nKqkkXH/3/RIUZy5qCnJb5UBtuPECAwEAAaOB2jCB1zAMBgNVHRMBAf8EAjAAMA4GA1UdDwEB/wQE\nAwIEsDAmBgNVHSAEHzAdMA0GCysGAQQBgpE2AQYCMAwGCiqGSIb3TAUCAgEwNwYDVR0fBDAwLjAs\noCqgKIYmaHR0cDovL2NybC5jaWxvZ29uLm9yZy9jaWxvZ29uLW9zZy5jcmwwHQYDVR0lBBYwFAYI\nKwYBBQUHAwIGCCsGAQUFBwMBMDcGA1UdEQQwMC6CDnZvbXMxLmZuYWwuZ292gg12b21zLmZuYWwu\nZ292gQ10aW1tQGZuYWwuZ292MA0GCSqGSIb3DQEBCwUAA4IBAQCSSCQ3ptGUM+j5Cp/AHJu0Cw1v\nJQp+eVJGjdShCdm/HK6jaqbNmNggqU2uurcGplnZEON4iaKVuT6vFEWbG0O2yluMCAdaqSJBws5m\ni55kuja4IbsJ8KeSjQlTkrszU3VNQofa4gcNWSql+VYjxggDrnsAunL8lG6CfSHXFDX23zx1yfEk\nw8syQXYZevUq8dky1tNMHCspMY7g9cA+UOCbHvE0xBMdQOU08R4PFpNMP4/AskCadui+ObFqomzB\nVDr5D3FGurRSVNyHXm5FAiIxQcf2sDf9jZRk11jCyjOigZzOoMNOHDHzFlrmCl2NVxklAfEyq6rU\nt2CVYeQq3CJN\n-----END CERTIFICATE-----\n"]
    }, {
      "SerialNumber": ["4295017101"]
    }]
}

Instead, they have errors:

{
    "entries": [{
      "key": "key",
      "entry": ["glow-voms.cs.wisc.edu"]
    }, {
      "key": "ERROR",
      "entry": ["fr.in2p3.lavoisier.interfaces.error.InitializationException: Exception raised for view 'voms-certificate-url' [java.net.SocketTimeoutException: connect timed out]"]
    }]
  }

This causes a task which asks for X509Cert to fail.

I could implement a little filter to exclude entries which do not follow the right structure, but I was wondering whether this data can be cleaned by passing different options to lavoisier.

Have an inner and outer loop for VOs and associated VOMS servers.

There is not a 1-1 mapping between VOs and their VOMS servers. Sometimes VOs have no VOMS server, and sometimes they have more than 1. Currently, the task which configures the vomses file for the VO has a loop over VO, and has two issues:

  1. The task fails when there is no VOMSES configuration for the VO
  2. The task only configures the first VOMS found, and does not do all of them.

This should be fixed by having an outer loop over VOs and an inner loop over VOMSes.

Good luck ๐Ÿค”

Write a python3 compatible cache script

The python script to cache data from Lavoisier is only python2 compatible and will give errors in python3, as pointed out by @enolfc in #12

Either update the script to be compatible for both environments, or add a new one which will work for python3.

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.