pgm / flock Goto Github PK
View Code? Open in Web Editor NEWa lightweight library for managing scatter/gather jobs run on a job scheduler such as SGE or LSF
License: Other
a lightweight library for managing scatter/gather jobs run on a job scheduler such as SGE or LSF
License: Other
See error below.
ec2-54-90-73-70.compute-1.amazonaws.com] run: mkdir -p /data2/runs/20150308-152040
[ec2-54-90-73-70.compute-1.amazonaws.com] out: mkdir: cannot create directory `/data2': Permission denied
[ec2-54-90-73-70.compute-1.amazonaws.com] out:
Fatal error: run() received nonzero return code 1 while executing!
Requested: mkdir -p /data2/runs/20150308-152040
Executed: /bin/bash -l -c "mkdir -p /data2/runs/20150308-152040"
Aborting.
Disconnecting from [email protected]... done.
I created the /data2/run directory by hand and launched a flock job from the UI. Here is the stack trace.
[ec2-54-90-73-70.compute-1.amazonaws.com] run: mkdir -p /data2/runs/20150308-152432
[ec2-54-90-73-70.compute-1.amazonaws.com] put: /tmp/tmp_XqrFi -> /data2/runs/20150308-152432/config
ERROR:paramiko.transport:Secsh channel 0 open FAILED: Connection refused: Connect failed
Got exception, reconnecting and retrying call
Traceback (most recent call last):
File "/home/vagrant/flock/ui/sshxmlrpc.py", line 87, in call
result = self._exec_call(args, kwargs)
File "/home/vagrant/flock/ui/sshxmlrpc.py", line 80, in _exec_call
result = xmlrpcclient.getattr(self.method_name)(_args, *_kwargs)
File "/usr/lib/python2.7/xmlrpclib.py", line 1224, in call
return self.*send(self.__name, args)
File "/usr/lib/python2.7/xmlrpclib.py", line 1578, in __request
verbose=self.__verbose
File "/usr/lib/python2.7/xmlrpclib.py", line 1264, in request
return self.single_request(host, handler, request_body, verbose)
File "/usr/lib/python2.7/xmlrpclib.py", line 1292, in single_request
self.send_content(h, request_body)
File "/usr/lib/python2.7/xmlrpclib.py", line 1439, in send_content
connection.endheaders(request_body)
File "/usr/lib/python2.7/httplib.py", line 969, in endheaders
self._send_output(message_body)
File "/usr/lib/python2.7/httplib.py", line 829, in _send_output
self.send(msg)
File "/usr/lib/python2.7/httplib.py", line 791, in send
self.connect()
File "/home/vagrant/flock/ui/sshxmlrpc.py", line 18, in connect
self.sock = self.ssh_transport._open_ssh_channel(dest_address, source_address)
File "/home/vagrant/flock/ui/sshxmlrpc.py", line 47, in _open_ssh_channel
source_address)
File "build/bdist.linux-x86_64/egg/paramiko/transport.py", line 740, in open_channel
raise e
ChannelException: (2, 'Connect failed')
ERROR:paramiko.transport:Secsh channel 0 open FAILED: Connection refused: Connect failed
Disconnecting from [email protected]... done.
Traceback (most recent call last):
File "remoteExec.py", line 120, in
transfer_config_and_submit(sys.argv[2:])
File "remoteExec.py", line 92, in transfer_config_and_submit
submit_to_wingman(host, key_filename, target_dir, timestamp, remote_config, params)
File "remoteExec.py", line 96, in submit_to_wingman
service.run_submitted(target_dir+"/files", timestamp, remote_config, json.dumps(params))
File "/home/vagrant/flock/ui/sshxmlrpc.py", line 92, in __call
result = self._exec_call(args, kwargs)
File "/home/vagrant/flock/ui/sshxmlrpc.py", line 80, in _exec_call
result = xmlrpcclient.getattr(self.method_name)(_args, *_kwargs)
File "/usr/lib/python2.7/xmlrpclib.py", line 1224, in call
return self.__send(self.__name, args)
File "/usr/lib/python2.7/xmlrpclib.py", line 1578, in __request
verbose=self.__verbose
File "/usr/lib/python2.7/xmlrpclib.py", line 1264, in request
return self.single_request(host, handler, request_body, verbose)
File "/usr/lib/python2.7/xmlrpclib.py", line 1292, in single_request
self.send_content(h, request_body)
File "/usr/lib/python2.7/xmlrpclib.py", line 1439, in send_content
connection.endheaders(request_body)
File "/usr/lib/python2.7/httplib.py", line 969, in endheaders
self._send_output(message_body)
File "/usr/lib/python2.7/httplib.py", line 829, in _send_output
self.send(msg)
File "/usr/lib/python2.7/httplib.py", line 791, in send
self.connect()
File "/home/vagrant/flock/ui/sshxmlrpc.py", line 18, in connect
self.sock = self.ssh_transport._open_ssh_channel(dest_address, source_address)
File "/home/vagrant/flock/ui/sshxmlrpc.py", line 47, in _open_ssh_channel
source_address)
File "build/bdist.linux-x86_64/egg/paramiko/transport.py", line 740, in open_channel
paramiko.ssh_exception.ChannelException: (2, 'Connect failed')
Got the following error while starting a cluster
Running plugin deadmansSwitch
*** WARNING - Verifying that domain hoc-heartbeats is accessible with non-admin credentials
tmpV5pOIh 100% |||||||||||||||||||||||||||||||||||||| Time: 00:00:00 2.32 M/s
*** WARNING - Adding cronjob for checking deadmans switch on <Node: hoc-master (i-30f64fc0)>
Running plugin installWingmanService
!!! ERROR - Error occured while running plugin 'installWingmanService':
!!! ERROR - Unhandled exception occured
Traceback (most recent call last):
File "/usr/local/lib/python2.7/dist-packages/StarCluster-0.9999-py2.7.egg/starcluster/cli.py", line 274, in main
sc.execute(args)
File "/usr/local/lib/python2.7/dist-packages/StarCluster-0.9999-py2.7.egg/starcluster/commands/start.py", line 282, in execute
save_config_on_master=self.opts.config_on_master)
File "/usr/local/lib/python2.7/dist-packages/StarCluster-0.9999-py2.7.egg/starcluster/cluster.py", line 1852, in start
save_config_on_master=save_config_on_master)
File "", line 2, in _start
File "/usr/local/lib/python2.7/dist-packages/StarCluster-0.9999-py2.7.egg/starcluster/utils.py", line 112, in wrap_f
res = func(_arg, *_kargs)
File "/usr/local/lib/python2.7/dist-packages/StarCluster-0.9999-py2.7.egg/starcluster/cluster.py", line 1876, in _start
self.setup_cluster(save_config_on_master)
File "/usr/local/lib/python2.7/dist-packages/StarCluster-0.9999-py2.7.egg/starcluster/cluster.py", line 1885, in setup_cluster
self._setup_cluster(save_config_on_master)
File "", line 2, in _setup_cluster
File "/usr/local/lib/python2.7/dist-packages/StarCluster-0.9999-py2.7.egg/starcluster/utils.py", line 112, in wrap_f
res = func(_arg, *_kargs)
File "/usr/local/lib/python2.7/dist-packages/StarCluster-0.9999-py2.7.egg/starcluster/cluster.py", line 1899, in _setup_cluster
self.run_plugins()
File "/usr/local/lib/python2.7/dist-packages/StarCluster-0.9999-py2.7.egg/starcluster/cluster.py", line 1918, in run_plugins
nodes=nodes)
File "/usr/local/lib/python2.7/dist-packages/StarCluster-0.9999-py2.7.egg/starcluster/cluster.py", line 1946, in run_plugin
func(*args)
File "/home/ubuntu/.starcluster/plugins/sc_flock_plugins.py", line 142, in run
temp_wingman_script.write(wingman_script_template(flock_home=self.flock_home, run_dir=self.run_dir))
TypeError: 'Template' object is not callable
!!! ERROR - Oops! Looks like you've found a bug in StarCluster
!!! ERROR - Crash report written to: /home/ubuntu/.starcluster/logs/crash-report-2800.txt
!!! ERROR - Please remove any sensitive data from the crash report
!!! ERROR - and submit it to [email protected]
A declarative, efficient, and flexible JavaScript library for building user interfaces.
๐ Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. ๐๐๐
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google โค๏ธ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.