openshift / elliott Goto Github PK

License: Apache License 2.0

Dockerfile 0.29% Python 99.65% Makefile 0.06%

elliott's Introduction

This repository is deprecated, please go to the new location at https://github.com/openshift-eng/art-tools/tree/main/elliott

elliott's People

Contributors

Stargazers

Watchers

elliott's Issues

Fix dev container kinit failed

[dev@dbaf7d0152d0 elliott]$ kinit shiywang
kinit: Invalid UID in persistent keyring name while getting default ccache

Mojo doc reference needs to be updated

Mojo reference in https://github.com/openshift/elliott/blob/master/README.md

The page you tried to load was originally found on Mojo, but it was contained in an area marked as not needed in The Source.
It will not be migrated.

elliot get - "--json" flag is missing

According to the documentation, the output of the get subcommand can be formatted in JSON using the --json flag, but this flag is missing.

 > elliott get --json 40392
Usage: elliott get [OPTIONS] ADVISORY
Try "elliott get -h" for help.

Error: no such option: --json

Unable to install on Fedora 37 due to incompatible version of `libgit2`

I tried to install elliott using pip install rh-elliott (as mentioned in the README), but it first errored out due to missing git2.h when building pygit2.
After installing libgit2-devel, the command errored out again with a different error:

      generating cffi module 'build/temp.linux-x86_64-cpython-311/pygit2._libgit2.c'                                                                                                                                
      creating build/temp.linux-x86_64-cpython-311                                                                                                                                                                  
      building 'pygit2._pygit2' extension                                                                                                                                                                           
      creating build/temp.linux-x86_64-cpython-311/src                                                                                                                                                              
      gcc -Wsign-compare -DDYNAMIC_ANNOTATIONS_ENABLED=1 -DNDEBUG -O2 -fexceptions -g -grecord-gcc-switches -pipe -Wall -Werror=format-security -Wp,-D_FORTIFY_SOURCE=2 -Wp,-D_GLIBCXX_ASSERTIONS -fstack-protector-
strong -m64 -mtune=generic -fasynchronous-unwind-tables -fstack-clash-protection -fcf-protection -D_GNU_SOURCE -fPIC -fwrapv -O2 -fexceptions -g -grecord-gcc-switches -pipe -Wall -Werror=format-security -Wp,-D_FO
RTIFY_SOURCE=2 -Wp,-D_GLIBCXX_ASSERTIONS -fstack-protector-strong -m64 -mtune=generic -fasynchronous-unwind-tables -fstack-clash-protection -fcf-protection -D_GNU_SOURCE -fPIC -fwrapv -O2 -fexceptions -g -grecord
-gcc-switches -pipe -Wall -Werror=format-security -Wp,-D_FORTIFY_SOURCE=2 -Wp,-D_GLIBCXX_ASSERTIONS -fstack-protector-strong -m64 -mtune=generic -fasynchronous-unwind-tables -fstack-clash-protection -fcf-protecti
on -D_GNU_SOURCE -fPIC -fwrapv -fPIC -I/usr/local/include -I/usr/include/python3.11 -c src/blob.c -o build/temp.linux-x86_64-cpython-311/src/blob.o                                                                 
      In file included from src/blob.h:34,                                                                                                                                                                          
                       from src/blob.c:30:                                                                                                                                                                          
      src/types.h:36:2: error: #error You need a compatible libgit2 version (1.1.x)                                                                                                                                 
         36 | #error You need a compatible libgit2 version (1.1.x)                                                                                                                                                  
            |  ^~~~~                                                                                                                                                                                                
      error: command '/usr/bin/gcc' failed with exit code 1                                                                                                                                                         
      [end of output]                                                                                                                                                                                               
                                                                                                                                                                                                                    
  note: This error originates from a subprocess, and is likely not a problem with pip.                                                                                                                              
  ERROR: Failed building wheel for pygit2                                                                                                                                                                           
Failed to build pygit2                                                                                                                                                                                              
ERROR: Could not build wheels for pygit2, which is required to install pyproject.toml-based projects

Indeed, the version of libgit2-devel is currently 1.3.1: https://pkgs.org/download/libgit2-devel

I ended up installing the development version of elliott by running pip install git+https://github.com/openshift/elliott@master, but thought I'd let you know.

Deprication message when loading a yaml file

 elliott --group cnv-1.4-cpaas-mvp --data-path product find-builds -k image --attach 40392
14:26:59 /productization/lib/python2.7/site-packages/elliottlib/runtime.py:143: YAMLLoadWarning: calling yaml.load() without Loader=... is deprecated, as the default Loader is unsafe. Please read https://msg.pyyaml.org/load for full details.

Be able to specify customized user instead of hardcoded "dev"

Once VScode support this microsoft/vscode-remote-release#46
we can use buildArgs to specify our own user like USER=$(id -un) instead of dev.

When switching to RHSA set the appropriate Impact in the description section

Currently we have hard coded the impact in the description to [[Important]]. We should set that to the 'highest_impact' as calculated in the attach_cve_flaws_cli method.

SSLError: HTTPSConnectionPool(host='errata.devel.redhat.com', port=443): Max retries exceeded with url

when I execute find-bugs this morning to the advisory I created.

➜  ~ elliott --group=openshift-3.9 find-bugs --auto  --add 41339
...
..
2019-04-17 11:16:21,621 INFO Using branch from group.yml: rhaos-3.9-rhel-7
Searching for bugs with target release(s): 3.9.0, 3.9.z
Found bugs: 1698136, 1693320, 1614291, 1653267, 1613280, 1686266, 1693036, 1686947, 1420235, 1678372, 1463717, 1550015, 1688642, 1508107, 1429788, 1645143
Adding 16 bugs to advisory 41339
Error: Erratum 41339: idsfixed: Bug #1614291 The bug is filed already in RHBA-2019:0758.
Bug #1653267 The bug is filed already in RHBA-2018:3748.

My first question it seems this bug has been attached to another advisory, so we should skip it ?
@smunilla

if so, then I hit command like this, and error out:

➜  ~ elliott --group=openshift-3.9 find-bugs --id 1698136 --id 1693320  --id 1613280 --id 1686266 --id 1693036 --id 1686947 --id 1420235 --id 1678372 --id 1463717 --id 1550015 --id 1688642 --id 1508107 --id 1429788 --id  1645143 --add 41339
/home/shiywang/.config/elliott/settings.yaml
2019-04-17 11:28:40,572 INFO Data clone directory already exists, checking commit sha
2019-04-17 11:28:41,980 INFO https://github.com/openshift/ocp-build-data.git is already cloned and latest
/usr/lib/python2.7/site-packages/elliottlib/runtime.py:143: YAMLLoadWarning: calling yaml.load() without Loader=... is deprecated, as the default Loader is unsafe. Please read https://msg.pyyaml.org/load for full details.
  tmp_config = Model(yaml.load(group_yml.format(**replace_vars)))
2019-04-17 11:28:42,039 INFO Using branch from group.yml: rhaos-3.9-rhel-7
Adding 14 bugs to advisory 41339
Traceback (most recent call last):
  File "/usr/bin/elliott", line 854, in <module>
    main()
  File "/usr/bin/elliott", line 843, in main
    cli(obj={})
  File "/usr/lib64/python2.7/site-packages/click/core.py", line 764, in __call__
    return self.main(*args, **kwargs)
  File "/usr/lib64/python2.7/site-packages/click/core.py", line 717, in main
    rv = self.invoke(ctx)
  File "/usr/lib64/python2.7/site-packages/click/core.py", line 1137, in invoke
    return _process_result(sub_ctx.command.invoke(sub_ctx))
  File "/usr/lib64/python2.7/site-packages/click/core.py", line 956, in invoke
    return ctx.invoke(self.callback, **ctx.params)
  File "/usr/lib64/python2.7/site-packages/click/core.py", line 555, in invoke
    return callback(*args, **kwargs)
  File "/usr/lib64/python2.7/site-packages/click/decorators.py", line 64, in new_func
    return ctx.invoke(f, obj, *args, **kwargs)
  File "/usr/lib64/python2.7/site-packages/click/core.py", line 555, in invoke
    return callback(*args, **kwargs)
  File "/usr/bin/elliott", line 425, in find_bugs
    advs.commit()
  File "/usr/lib/python2.7/site-packages/errata_tool/erratum.py", line 946, in commit
    self._write()
  File "/usr/lib/python2.7/site-packages/errata_tool/erratum.py", line 885, in _write
    r = self._put(url, data=pdata)
  File "/usr/lib/python2.7/site-packages/errata_tool/connector.py", line 189, in _put
    verify=self.ssl_verify)
  File "/usr/lib/python2.7/site-packages/requests/api.py", line 131, in put
    return request('put', url, data=data, **kwargs)
  File "/usr/lib/python2.7/site-packages/requests/api.py", line 60, in request
    return session.request(method=method, url=url, **kwargs)
  File "/usr/lib/python2.7/site-packages/requests/sessions.py", line 533, in request
    resp = self.send(prep, **send_kwargs)
  File "/usr/lib/python2.7/site-packages/requests/sessions.py", line 646, in send
    r = adapter.send(request, **kwargs)
  File "/usr/lib/python2.7/site-packages/requests/adapters.py", line 514, in send
    raise SSLError(e, request=request)
requests.exceptions.SSLError: HTTPSConnectionPool(host='errata.devel.redhat.com', port=443): Max retries exceeded with url: /api/v1/erratum/41339 (Caused by SSLError(SSLError("bad handshake: SysCallError(-1, 'Unexpected EOF')",),))

I think this is more like a issue of https://github.com/red-hat-storage/errata-tool instead of elliot.
or maybe errata is really an old antique can't even handle ~10 connections ?
maybe we should add an limit on our --id parameters, like if more than 10 then please do this spereately ?

BTW if we support adding multiple bugs maybe sth like --add 1234, 3456, 7890 is better than --id 1234 --id 3456 --id 7890 ?

Python 3 support

In starting to look at elliott I noticed a number of possible issues with python3 support in the code base. Below are a few that I saw so far.

tox missing py3 target (available in the python-3 branch)
setup.py is missing test deps
print() issue
urlparse error
unicode
absolute vs relative imports
iteritems
dict values() is view not a list in python3
contextlib.next was replace by ExitStack (in a unittest)
unittests and elliottlib.assertions.FileNotFoundError and ChildProcessError are not working in py3

I did see pull request #82 started looking at this

Some spots for better error checking

Hi @smunilla, long time user here -- I love your product! I'm writing today to point out two areas I think we could improve error checking and reporting.

Bugzilla isn't logged in. It'd be nice if we could notice when a bugzilla call gives the unauthorized error

xmlrpclib.Fault: <Fault 102: 'You are not authorized to access bug #1605844. To see this bug, you must first log in to an account with the appropriate permissions.'>

This weird connection reset error could be caught and retried I think:

requests.exceptions.ChunkedEncodingError: ('Connection broken: error("(104, \'ECONNRESET\')",)', error("(104, 'ECONNRESET')",))

Both of these happened while running:

./elliott --group=openshift-4.1 repair-bugs --noop --use-default-advisory rpm --auto

xmlrpclib.Fault: <Fault -32601: "The requested method 'Bug.get_bugs' was not found.">

[jenkins@buildvm elliott]$ elliott --group=openshift-4.1 repair-bugs --from MODIFIED --to ON_QA --auto -a 38252
2019-04-29 23:01:06,154 INFO Cloning config data from [email protected]:openshift/ocp-build-data.git
2019-04-29 23:01:07,552 INFO Using branch from group.yml: rhaos-4.1-rhel-7
Fetching Erratum(errata_id=38252)
Getting bugs for advisory
Fetching data for 819 bugs: Hold on a moment, we have to grab each one
[***************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************]
[****************Traceback (most recent call last):
  File "/bin/elliott", line 974, in <module>
    main()
  File "/bin/elliott", line 963, in main
    cli(obj={})
  File "/usr/lib64/python2.7/site-packages/click/core.py", line 764, in __call__
    return self.main(*args, **kwargs)
  File "/usr/lib64/python2.7/site-packages/click/core.py", line 717, in main
    rv = self.invoke(ctx)
  File "/usr/lib64/python2.7/site-packages/click/core.py", line 1137, in invoke
    return _process_result(sub_ctx.command.invoke(sub_ctx))
  File "/usr/lib64/python2.7/site-packages/click/core.py", line 956, in invoke
    return ctx.invoke(self.callback, **ctx.params)
  File "/usr/lib64/python2.7/site-packages/click/core.py", line 555, in invoke
    return callback(*args, **kwargs)
  File "/usr/lib64/python2.7/site-packages/click/decorators.py", line 64, in new_func
    return ctx.invoke(f, obj, *args, **kwargs)
  File "/usr/lib64/python2.7/site-packages/click/core.py", line 555, in invoke
    return callback(*args, **kwargs)
  File "/bin/elliott", line 891, in repair_bugs
    raw_bug_list)
  File "/usr/lib64/python2.7/multiprocessing/pool.py", line 250, in map
    return self.map_async(func, iterable, chunksize).get()
  File "/usr/lib64/python2.7/multiprocessing/pool.py", line 554, in get
    raise self._value
xmlrpclib.Fault: <Fault -32601: "The requested method 'Bug.get_bugs' was not found.">

AttributeError: 'module' object has no attribute 'full_load'

(venv) [lmilbaum@lmilbaum ~]$ elliott list
Traceback (most recent call last):
  File "/home/lmilbaum/.virtualenvs/venv/bin/elliott", line 722, in <module>
    main()
  File "/home/lmilbaum/.virtualenvs/venv/bin/elliott", line 711, in main
    cli(obj={})
  File "/home/lmilbaum/.virtualenvs/venv/lib/python2.7/site-packages/click/core.py", line 764, in __call__
    return self.main(*args, **kwargs)
  File "/home/lmilbaum/.virtualenvs/venv/lib/python2.7/site-packages/click/core.py", line 717, in main
    rv = self.invoke(ctx)
  File "/home/lmilbaum/.virtualenvs/venv/lib/python2.7/site-packages/click/core.py", line 1134, in invoke
    Command.invoke(self, ctx)
  File "/home/lmilbaum/.virtualenvs/venv/lib/python2.7/site-packages/click/core.py", line 956, in invoke
    return ctx.invoke(self.callback, **ctx.params)
  File "/home/lmilbaum/.virtualenvs/venv/lib/python2.7/site-packages/click/core.py", line 555, in invoke
    return callback(*args, **kwargs)
  File "/home/lmilbaum/.virtualenvs/venv/lib/python2.7/site-packages/click/decorators.py", line 17, in new_func
    return f(get_current_context(), *args, **kwargs)
  File "/home/lmilbaum/.virtualenvs/venv/bin/elliott", line 83, in cli
    cli_args=kwargs)
  File "/home/lmilbaum/.virtualenvs/venv/lib/python2.7/site-packages/dotconfig/__init__.py", line 74, in __init__
    data = yaml.full_load(f)
AttributeError: 'module' object has no attribute 'full_load'

pygit2 version is too new so that won't pass compilation

    Running setup.py install for pygit2: finished with status 'error'
    Complete output from command /usr/bin/python2 -u -c "import setuptools, tokenize;__file__='/tmp/pip-install-1RqAJw/pygit2/setup.py';f=getattr(tokenize, 'open', open)(__file__);code=f.read().replace('\r\n', '\n');f.close();exec(compile(code, __file__, 'exec'))" install --record /tmp/pip-record-hXLQpv/install-record.txt --single-version-externally-managed --compile --user --prefix=:
    running install
    running build
    running build_py
    creating build
    creating build/lib.linux-x86_64-2.7
    creating build/lib.linux-x86_64-2.7/pygit2
    copying pygit2/utils.py -> build/lib.linux-x86_64-2.7/pygit2
    copying pygit2/submodule.py -> build/lib.linux-x86_64-2.7/pygit2
    copying pygit2/_run.py -> build/lib.linux-x86_64-2.7/pygit2
    copying pygit2/credentials.py -> build/lib.linux-x86_64-2.7/pygit2
    copying pygit2/repository.py -> build/lib.linux-x86_64-2.7/pygit2
    copying pygit2/__init__.py -> build/lib.linux-x86_64-2.7/pygit2
    copying pygit2/refspec.py -> build/lib.linux-x86_64-2.7/pygit2
    copying pygit2/ffi.py -> build/lib.linux-x86_64-2.7/pygit2
    copying pygit2/config.py -> build/lib.linux-x86_64-2.7/pygit2
    copying pygit2/remote.py -> build/lib.linux-x86_64-2.7/pygit2
    copying pygit2/settings.py -> build/lib.linux-x86_64-2.7/pygit2
    copying pygit2/blame.py -> build/lib.linux-x86_64-2.7/pygit2
    copying pygit2/errors.py -> build/lib.linux-x86_64-2.7/pygit2
    copying pygit2/_build.py -> build/lib.linux-x86_64-2.7/pygit2
    copying pygit2/index.py -> build/lib.linux-x86_64-2.7/pygit2
    creating build/lib.linux-x86_64-2.7/pygit2/decl
    copying pygit2/decl/proxy.h -> build/lib.linux-x86_64-2.7/pygit2/decl
    copying pygit2/decl/refspec.h -> build/lib.linux-x86_64-2.7/pygit2/decl
    copying pygit2/decl/common.h -> build/lib.linux-x86_64-2.7/pygit2/decl
    copying pygit2/decl/errors.h -> build/lib.linux-x86_64-2.7/pygit2/decl
    copying pygit2/decl/transport.h -> build/lib.linux-x86_64-2.7/pygit2/decl
    copying pygit2/decl/attr.h -> build/lib.linux-x86_64-2.7/pygit2/decl
    copying pygit2/decl/buffer.h -> build/lib.linux-x86_64-2.7/pygit2/decl
    copying pygit2/decl/strarray.h -> build/lib.linux-x86_64-2.7/pygit2/decl
    copying pygit2/decl/merge.h -> build/lib.linux-x86_64-2.7/pygit2/decl
    copying pygit2/decl/clone.h -> build/lib.linux-x86_64-2.7/pygit2/decl
    copying pygit2/decl/checkout.h -> build/lib.linux-x86_64-2.7/pygit2/decl
    copying pygit2/decl/remote.h -> build/lib.linux-x86_64-2.7/pygit2/decl
    copying pygit2/decl/revert.h -> build/lib.linux-x86_64-2.7/pygit2/decl
    copying pygit2/decl/index.h -> build/lib.linux-x86_64-2.7/pygit2/decl
    copying pygit2/decl/oid.h -> build/lib.linux-x86_64-2.7/pygit2/decl
    copying pygit2/decl/submodule.h -> build/lib.linux-x86_64-2.7/pygit2/decl
    copying pygit2/decl/repository.h -> build/lib.linux-x86_64-2.7/pygit2/decl
    copying pygit2/decl/config.h -> build/lib.linux-x86_64-2.7/pygit2/decl
    copying pygit2/decl/types.h -> build/lib.linux-x86_64-2.7/pygit2/decl
    copying pygit2/decl/blame.h -> build/lib.linux-x86_64-2.7/pygit2/decl
    copying pygit2/decl/describe.h -> build/lib.linux-x86_64-2.7/pygit2/decl
    copying pygit2/decl/pack.h -> build/lib.linux-x86_64-2.7/pygit2/decl
    copying pygit2/decl/diff.h -> build/lib.linux-x86_64-2.7/pygit2/decl
    copying pygit2/decl/net.h -> build/lib.linux-x86_64-2.7/pygit2/decl
    copying pygit2/decl/graph.h -> build/lib.linux-x86_64-2.7/pygit2/decl
    copying pygit2/decl/stash.h -> build/lib.linux-x86_64-2.7/pygit2/decl
    running build_ext
    generating cffi module 'build/temp.linux-x86_64-2.7/pygit2._libgit2.c'
    creating build/temp.linux-x86_64-2.7
    building '_pygit2' extension
    creating build/temp.linux-x86_64-2.7/src
    gcc -pthread -fno-strict-aliasing -O2 -g -pipe -Wall -Werror=format-security -Wp,-D_FORTIFY_SOURCE=2 -Wp,-D_GLIBCXX_ASSERTIONS -fexceptions -fstack-protector-strong -grecord-gcc-switches -specs=/usr/lib/rpm/redhat/redhat-hardened-cc1 -specs=/usr/lib/rpm/redhat/redhat-annobin-cc1 -m64 -mtune=generic -fasynchronous-unwind-tables -fstack-clash-protection -fcf-protection -D_GNU_SOURCE -fPIC -fwrapv -DNDEBUG -O2 -g -pipe -Wall -Werror=format-security -Wp,-D_FORTIFY_SOURCE=2 -Wp,-D_GLIBCXX_ASSERTIONS -fexceptions -fstack-protector-strong -grecord-gcc-switches -specs=/usr/lib/rpm/redhat/redhat-hardened-cc1 -specs=/usr/lib/rpm/redhat/redhat-annobin-cc1 -m64 -mtune=generic -fasynchronous-unwind-tables -fstack-clash-protection -fcf-protection -D_GNU_SOURCE -fPIC -fwrapv -fPIC -I/usr/local/include -I/usr/include/python2.7 -c src/blob.c -o build/temp.linux-x86_64-2.7/src/blob.o
    In file included from src/blob.h:34,
                     from src/blob.c:30:
    src/types.h:36:2: error: #error You need a compatible libgit2 version (v0.28.x)
       36 | #error You need a compatible libgit2 version (v0.28.x)
          |  ^~~~~
    src/types.h:237:5: error: unknown type name 'git_mailmap'
      237 |     git_mailmap *mailmap;
          |     ^~~~~~~~~~~
    src/blob.c:183:5: warning: initialization of 'Py_ssize_t (*)(PyObject *, Py_ssize_t,  void **)' {aka 'long int (*)(struct _object *, long int,  void **)'} from incompatible pointer type 'int (*)(PyObject *, Py_buffer *, int)' {aka 'int (*)(struct _object *, struct bufferinfo *, int)'} [-Wincompatible-pointer-types]
      183 |     (getbufferproc)Blob_getbuffer,
          |     ^
    src/blob.c:183:5: note: (near initialization for 'Blob_as_buffer.bf_getreadbuffer')
    error: command 'gcc' failed with exit status 1
    ----------------------------------------
Command "/usr/bin/python2 -u -c "import setuptools, tokenize;__file__='/tmp/pip-install-1RqAJw/pygit2/setup.py';f=getattr(tokenize, 'open', open)(__file__);code=f.read().replace('\r\n', '\n');f.close();exec(compile(code, __file__, 'exec'))" install --record /tmp/pip-record-hXLQpv/install-record.txt --single-version-externally-managed --compile --user --prefix=" failed with error code 1 in /tmp/pip-install-1RqAJw/pygit2/

openshift / elliott Goto Github PK

elliott's Introduction

elliott's People

Contributors

Stargazers

Watchers

Forkers

elliott's Issues

Recommend Projects

Recommend Topics

Recommend Org