gawbul / docker-sge Goto Github PK

Dockerfile to build a container with SGE installed

Home Page: https://registry.hub.docker.com/u/gawbul/docker-sge/

License: MIT License

Shell 100.00%

docker-sge's Introduction

Hi there, I'm @gawbul (Steve) 👋

I am a Computational Scientist, and M.Sc. student in Computer Science in the Department of Computer Science at the University of York. I am interested in Bioinformatics, Evolution, and Genomics. I am the founder of pyOpenSci. I work as a freelance Consultant, and a Staff Site Reliability Engineer - where I focus on solving technical problems relevant to large-scale distributed systems. I have a B.Sc. (Hons) in Biomedical Sciences, a PGcert in Research Training, and a Ph.D. in Computational Genomics from the University of Hull (as part of the EvoHull Group) focusing on questions within Bioinformatics, Comparative Genomics and Molecular Evolution. My current research interests centre around questions relating to Deep Learning with a particular focus on it's applications in Bioinformatics, Genomics and Precision Medicine.

- 🔭 I’m currently working on ...

Pathlength a ray tracing program for calculating resolution and sensitivity in crustacean eyes (reniform reflecting superposition compound eyes).

A comparative genomic analyses of introns in several species of fish using publicly available data from the Ensembl genome project.

- 🌱 I’m currently learning ...

Trying to learn ~~too many things~~ Go, ~~all about the Science of Exercise~~, various topics within SRE from Google including Site Reliability Engineering: Measuring and Managing Reliability and working towards a number of AWS certifications, utilising resources such as the DevOps on AWS Specialization. I am also interested in learning more about technologies such as Kubernetes and Kafka. As part of this I am undertaking the Certified Kubernetes Administrator (CKA) exam via the Linux Foundation, and the Confluent Certified Administrator for Apache Kafka (CCAAK) exam via Confluent.

- 👯 I’m looking to collaborate on ...

Any interesting problems and challenges within a variety of different fields that are amenable to being solved using methods within the broader scope of Computational Science.

- 🤔 I’m looking for help with ...

Developing my theoretical understanding of Software Engineering and learning more about Data Science, Machine Learning and AI with a particular focus on biological, biomedical and clinical applications. I am also developing my theoretical knowledge of networking and cyber security.

- 💬 Ask me about ...

Anything relating to Computational Genomics (Bioinformatics, Comparative Genomics and Molecular Evolution), DevOps and Site Reliability Engineering as well as Cloud, High-Performance, and Scientific Computing.

- 📫 How to reach me: ...

- 😄 Pronouns: ...

he/him/his.

- ⚡ Fun fact: ...

I worked as a Security Operative and Bouncer for over 11 years. I am a pugilist, martial artist and yogi.

docker-sge's People

Contributors

Stargazers

Watchers

Forkers

ajdavid robsyme brspurri pgruenbacher tedm stevekm jerias huozhihui yh646492956 aalhendi1707 lanpang

docker-sge's Issues

qsub returns an error message

I've launched the container like this:

docker run --rm -it gawbul/docker-sge:latest bash

Then I've tried to submit a job:

# echo 'echo Hello' | qsub
Unable to run job: unable to send message to qmaster using port 6444 on host "9818fcbe520e": can't resolve host name
Exiting.

The command '/bin/sh -c apt-get upgrade -y' returned a non-zero code: 100

Trying to build the image from the included Docker file (using my fork), I keep getting this error:

$ docker build -t stevekm/docker-sge .
The command '/bin/sh -c apt-get upgrade -y' returned a non-zero code: 100

Any ideas? Did some Google'ing and included some tests to make sure network access was working inside the container, could figure out what is causing the error. Full output of docker build is here:

https://github.com/stevekm/docker-sge/blob/a30cfb3f687358a63d9f13faff31aa9992fedaa6/build_log.txt

Unable to initialize environment because of error: cell directory "/opt/sge/default" doesn't exist

Get the following when running # qsub:

Unable to initialize environment because of error: cell directory "/opt/sge/default" doesn't exist
Exiting.

izpack link no longer exists

The wget link to izpack 4.3.5 no longer works as codehaus no longer exists.

As a work around I found the following link to work as a replacement
RUN wget -c http://download.jboss.org/jbosstools/updates/requirements/izpack/4.3.5/IzPack-install-4.3.5.jar

Error in apt-get install -y for required packages

Getting the following error:

Processing triggers for libgdk-pixbuf2.0-0:amd64 (2.30.7-0ubuntu1) ...
Errors were encountered while processing:
 dictionaries-common
 aspell
 aspell-en
 hunspell-en-us
 libenchant1c2a:amd64
 libwebkitgtk-3.0-0:amd64
 enchant
 libyelp0
 yelp
 gnome-user-guide
 zenity
 unity-control-center
 unity-control-center-signon
 indicator-bluetooth
E: Sub-process /usr/bin/dpkg returned an error code (1)
2014/12/12 15:45:17 The command [/bin/sh -c apt-get update && apt-get -y install wget darcs git mercurial tcsh build-essential automake autoconf openssl libssl-dev munge libmunge2 libmunge-dev libjemalloc1 libjemalloc-dev db5.3-util libdb-dev libncurses5 libncurses5-dev libpam0g libpam0g-dev libpacklib-lesstif1-dev libmotif-dev libxmu-dev libxpm-dev hwloc libhwloc-dev openjdk-7-jre openjdk-7-jdk ant ant-optional javacc junit libswing-layout-java libxft2 libxft-dev libreadline-dev vim tmux man gawk] returned a non-zero code: 100

Automate inst_sge Grid Engine Installation

./inst_sge -m -s -x -csp currently requires user input to configure. Need to investigate how to automate.

error: unable to send message to qmaster using port 6444 on host "66be2f8db7c1": can't resolve host name

Hostname changes during docker build and settings written to /opt/sge reference wrong hostname.

sgeadmin's job is not allowed to run in any queue

Having trouble getting jobs to run inside the container. I tried using the sgeadmin account as described in the README, but it is not working.

Downloaded & ran the container from Docker hub:

$ docker pull gawbul/docker-sge
$ docker run -it --rm gawbul/docker-sge login -f sgeadmin

Inside the container:


sgeadmin@c61cc2576e48:~$ echo "echo Running test from $HOSTNAME" | qsub
Unable to run job: warning: sgeadmin's job is not allowed to run in any queue
Your job 1 ("STDIN") has been submitted
Exiting.
sgeadmin@c61cc2576e48:~$ qstat
job-ID  prior   name       user         state submit/start at     queue                          slots ja-task-ID
-----------------------------------------------------------------------------------------------------------------
      1 0.00000 STDIN      sgeadmin     qw    12/07/2017 00:58:19                                    1

The job appears to wait in the qstat queue indefinitely.

Multiple instances (different IPs) running docker-sge?

Hi there,

I have a very fundamental question about this repo.
I have the docker-sge image running on an AWS machine with 32 cores, and with the help of this repo, I have a 32 node GridEngine (which is fantastic). Call this one instance 1.

However, I would like to add another AWS instance (completely different server and IP) to SGE (so there are a total of 64 CPUs being dedicated to SGE). Is there any way to run another docker-sge image (on another instance entirely: instance 2) and let the master (instance 1) know about the new instance (instance 2)?

Thanks!