ioam / lancet Goto Github PK
View Code? Open in Web Editor NEWLaunch jobs, organize the output, and dissect the results
Home Page: http://ioam.github.io/lancet/
License: BSD 3-Clause "New" or "Revised" License
Launch jobs, organize the output, and dissect the results
Home Page: http://ioam.github.io/lancet/
License: BSD 3-Clause "New" or "Revised" License
The current git HEAD seems to have trouble running the basic example given on the lancet homepage:
>>> import lancet
>>> integers = lancet.Range('integer', 100, 115)
RuntimeError: maximum recursion depth exceeded while calling a Python object
/Users/user/Code/lancet/lancet/core.pyc in __len__(self)
431 return DataFrame(self.specs) if DataFrame else "Pandas not available"
432
--> 433 def __len__(self): return len(self.specs)
434
435
/usr/local/lib/python2.7/site-packages/param/parameterized.pyc in __get__(self, obj, objtype)
372 # Parameterized class); objtype is never None
373
--> 374 if not obj:
375 result = self.default
376 else:
/Users/user/Code/lancet/lancet/core.pyc in __len__(self)
431 return DataFrame(self.specs) if DataFrame else "Pandas not available"
432
--> 433 def __len__(self): return len(self.specs)
434
435
/usr/local/lib/python2.7/site-packages/param/parameterized.pyc in __get__(self, obj, objtype)
372 # Parameterized class); objtype is never None
373
--> 374 if not obj:
375 result = self.default
376 else:
It would be really helpful if jobs launched via GridEngine could list the job number in the job name first, such that it is viewable via qstat
. Currently although it is included in the job name, qstat usually truncates it.
I'd really appreciate this change because I will often launch large parameter searches with jobs that can run up to 48 hours and hog 8 CPU's and realize relatively early that certain parameters won't work out. Rather than wasting the resources I then cancel these jobs selectively.
In the past I have had to work out which job-IDs correspond to which parameters by looking at the first or last job and counting from there. This is obviously insane (and error-prone) when there are >100 jobs and becomes even more difficult when some jobs have already completed. I'd therefore hugely appreciate if we could prepend the job number to the job name submitted to qsub, making identifying specific jobs much easier.
Currently the job name is built like this:
job_name = "%s_%s_tid_%d" % (self.batch_name, self.job_timestamp, tid)
Producing this qstat
output:
job-ID prior name user state submit/start at queue slots ja-task-ID
-----------------------------------------------------------------------------------------------------------------
7817272 0.00941 divisive_g s1048519 r 07/22/2014 20:10:03 ecdf@eddie491 2
7817306 0.00941 divisive_g s1048519 t 07/22/2014 23:43:15 ecdf@eddie320 2
....
I suggest changing it to this:
job_name = "t%d_%s_%s" % (tid, self.batch_name, self.job_timestamp)
Output:
job-ID prior name user state submit/start at queue slots ja-task-ID
-----------------------------------------------------------------------------------------------------------------
7817272 0.00941 t0_divisiv s1048519 r 07/22/2014 20:10:03 ecdf@eddie491 2
7817306 0.00941 t1_divisiv s1048519 t 07/22/2014 23:43:15 ecdf@eddie320 2
....
While reading your interesting paper and experimenting with lancet, I tried the following:
from lancet import Args, Range
params = Args(arg1=1.0) * Range('arg2', 1, 3, steps=3)
ap = params - Args(arg1=1.0, arg2=2.0)
But this doesn't work. I can imagine scenarios where this is just as natural as extending a parameter space with addition. Was wondering why this was not supported.
Now that noarch is generally well supported, we should make this pure-Python package be a noarch conda package to avoid ultra-confusing problems like holoviz/datashader#532 due to having no Lancet package for a particular Python version.
For example:
launcher = Launcher(...)
output = launcher()
results = [float(stdout.read()) for stdout in output.stdout]
Ideally the output should contain enough information that it should be very easy to generate HoloMaps
or pandas dataframes from this output without having to write a lot of code.
python setup.py develop
does not work out of the box with Python-2.7.x and Lancet.
Launcher currently re-launches everything when called again. Instead Launcher should only run previous runs if the earlier invocation failed (with a non-zero exit status) or explicitly asked to force-rerun. This would also make it easy to change the parameters and add additional parameters ranges to explore without having to worry about re-running everything.
A declarative, efficient, and flexible JavaScript library for building user interfaces.
๐ Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. ๐๐๐
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google โค๏ธ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.