Giter Club home page Giter Club logo

azure-pipelines-agent's Introduction

I'm in the Scalable Algorithms group at Sandia National Labs working on distributed and GPU-accelerated sparse matrix operations.

Carl's GitHub stats

Reach me at <github username without last two letters>@sandia.gov.

azure-pipelines-agent's People

Contributors

cwpearson avatar

Stargazers

avatar avatar avatar avatar avatar avatar avatar

Watchers

avatar avatar

Forkers

joaander pabloinigoblasco arvindmits

azure-pipelines-agent's Issues

Lower CPU priority for

  1. create a user that the docker agent will run as. This is done in the Dockerfile
RUN useradd -ms /bin/bash azp
USER azp
WORKDIR /home/azp
  1. use /etc/security/limits.conf to raise the niceness of that user
RUN echo azp soft priority 20 >> /etc/security/limits.conf

Allow manager to map static data into containers

To map large graph data into a test environment

  1. the manager could map it as a read-only volume
  2. the manager could add an environment variable saying that the large test data was available and where it was
  3. If that environment variable exists, the test could run, otherwise it could be skipped

`python manager.py ...` exits with 404 

python manager.py [redacted]] https://dev.azure.com/c3srdev amd64-ubuntu1604-cuda100 --volume ~/graph/:/data/graph:PANGOLIN_GRAPH_DIR

testing for nvidia-docker
nvidia-docker looks good
autodetected docker image cwpearson/azp-cuda-agent:amd64-ubuntu1604-cuda102
pulling cwpearson/azp-cuda-agent:amd64-ubuntu1604-cuda102
Traceback (most recent call last):
  File "/home/pearson/.local/share/virtualenvs/azure-pipelines-agent-fYRwYNLf/lib/python3.6/site-packages/docker/api/client.py", line 261, in _raise_for_status
    response.raise_for_status()
  File "/home/pearson/.local/share/virtualenvs/azure-pipelines-agent-fYRwYNLf/lib/python3.6/site-packages/requests/models.py", line 940, in raise_for_status
    raise HTTPError(http_error_msg, response=self)
requests.exceptions.HTTPError: 404 Client Error: Not Found for url: http+docker://localhost/v1.35/images/create?tag=amd64-ubuntu1604-cuda102&fromImage=cwpearson%2Fazp-cuda-agent

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "manager.py", line 147, in <module>
    client.images.pull(DOCKER_IMAGE)
  File "/home/pearson/.local/share/virtualenvs/azure-pipelines-agent-fYRwYNLf/lib/python3.6/site-packages/docker/models/images.py", line 444, in pull
    repository, tag=tag, stream=True, **kwargs
  File "/home/pearson/.local/share/virtualenvs/azure-pipelines-agent-fYRwYNLf/lib/python3.6/site-packages/docker/api/image.py", line 414, in pull
    self._raise_for_status(response)
  File "/home/pearson/.local/share/virtualenvs/azure-pipelines-agent-fYRwYNLf/lib/python3.6/site-packages/docker/api/client.py", line 263, in _raise_for_status
    raise create_api_error_from_http_exception(e)
  File "/home/pearson/.local/share/virtualenvs/azure-pipelines-agent-fYRwYNLf/lib/python3.6/site-packages/docker/errors.py", line 31, in create_api_error_from_http_exception
    raise cls(e, response=response, explanation=explanation)
docker.errors.NotFound: 404 Client Error: Not Found ("manifest for cwpearson/azp-cuda-agent:amd64-ubuntu1604-cuda102 not found")

$ docker version
Client: Docker Engine - Community
 Version:           19.03.1
 API version:       1.39 (downgraded from 1.40)
 Go version:        go1.12.5
 Git commit:        74b1e89
 Built:             Thu Jul 25 21:21:05 2019
 OS/Arch:           linux/amd64
 Experimental:      false

Server: Docker Engine - Community
 Engine:
  Version:          18.09.7
  API version:      1.39 (minimum version 1.12)
  Go version:       go1.10.8
  Git commit:       2d0083d
  Built:            Thu Jun 27 17:23:02 2019
  OS/Arch:          linux/amd64
  Experimental:     false

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.