mesos / chronos Goto Github PK

View Code? Open in Web Editor NEW

4.4K 289.0 529.0 7.31 MB

Fault tolerant job scheduler for Mesos which handles dependencies and ISO8601 based schedules

Home Page: http://mesos.github.io/chronos/

License: Apache License 2.0

Shell 0.35% JavaScript 13.47% HTML 0.17% Scala 81.90% Ruby 3.80% Dockerfile 0.31%

chronos mesos job-scheduler scheduled-jobs fault-tolerance cron crontab cron-jobs mesos-framework chronos-scheduler

chronos's Introduction

Apache Mesos Repository Has Moved

Apache Mesos is now a Top-Level Apache project, and we've moved the codebase. The downloads page explains the essential information, but here's the scoop:

Please check out the source code from Apache's git repostory:

git clone https://git-wip-us.apache.org/repos/asf/mesos.git

or if you prefer GitHub, use the GitHub mirror:

git clone git://github.com/apache/mesos.git

For issue tracking and patches, we use Apache-maintained infrastructure including the JIRA issue tracker instead of the GitHub one, and review board for patches instead of pull requests.

Other information including documentation and a getting started guide are available on the Mesos website: http://mesos.apache.org

Thanks!

-- The Mesos developers

chronos's People

Contributors

Stargazers

Watchers

Forkers

florianleibert enachb brianm bscott obsdeck leiqzhang brownjohnf pochadri snouhaud jametong bhuvangu skny5 domenicosolazzo danielym pdeyhim inthecloud247 vfulco mailmahee jmtobar anapsix craigbeck twocngdagz hshoff arkadiuszsz sralmai staabm siliconcow dharmeshkakadia rafaltrojniak c4urself fizx rgahagan solidsnack kasin jeid64 arron-green prashantgupta lanphan dsrinivasan cniclsh nikolairoman guigra riywo c4ssio ngeorgieff xuanhan863 josephmartz bijaybisht kriswuollett imaxxs cb372 softprops deniskin82 cruncherbigdata isantoso tpetr anismiles dhardy92 libin robzienert bosky101 lucianocheng ortoo harishd pombredanne andrewglenn tonypakati oulan joliny kedyyan perryhau sxhao viadeo egooyiri airclear austinbgibbons alteryx tw4dl trutch yilab tarnfeld postpcera jwoolf kelmi heera stelcheck joebos asher mschober yiyujia di-stars jy4618272 h4ck3rm1k3 th0br0 zenlambda pecbali chengweiv5 eiriklv preachermanx berngp

chronos's Issues

can't install mesos

Ubuntu 12.10.

/home/fred/bin/mesos/
Trying to create /home/fred/bin/mesos/
sed: can't read : No such file or directory
fred@daneel-olivaw:~/bin/airbnb-chronos-98b5599/bin$ sudo ./installer.bash

Welcome to the interactive chronos installation. This script will first install mesos and then chronos.

Depending on your hardware and internet connection, this script might take 15 - 25 minutes as we're compiling mesos from source.
If you run into any issues, please check the FAQ in the chronos repo.
Do you have mesos installed already? Type 'yes' or 'no' followed by [ENTER]:
no
Type the target directory (absolute path) where you would like mesos to be installed followed by [ENTER]:
/usr/bin/mesos
Trying to create /usr/bin/mesos
sed: can't read : No such file or directory

zookeeper / chronos error logs

2013-07-18 16:03:46,602 - INFO [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:2181:NIOServerCnxn@777] - Client attempting to establish new session at /10.1.8.138:43635
2013-07-18 16:03:46,604 - INFO [SyncThread:0:FileTxnLog@199] - Creating new log file: log.b9
2013-07-18 16:03:46,622 - INFO [SyncThread:0:NIOServerCnxn@1580] - Established session 0x13ff35fbbe50000 with negotiated timeout 5000 for client /10.1.8.138:43635
2013-07-18 16:03:46,651 - INFO [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:2181:NIOServerCnxn$Factory@251] - Accepted socket connection from /10.1.8.138:43636
2013-07-18 16:03:46,651 - INFO [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:2181:NIOServerCnxn@777] - Client attempting to establish new session at /10.1.8.138:43636
2013-07-18 16:03:46,652 - INFO [SyncThread:0:NIOServerCnxn@1580] - Established session 0x13ff35fbbe50001 with negotiated timeout 5000 for client /10.1.8.138:43636
2013-07-18 16:03:47,494 - INFO [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:2181:NIOServerCnxn$Factory@251] - Accepted socket connection from /10.1.8.138:43637
2013-07-18 16:03:47,495 - INFO [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:2181:NIOServerCnxn@777] - Client attempting to establish new session at /10.1.8.138:43637
2013-07-18 16:03:47,496 - INFO [SyncThread:0:NIOServerCnxn@1580] - Established session 0x13ff35fbbe50002 with negotiated timeout 10000 for client /10.1.8.138:43637
2013-07-18 16:03:47,498 - INFO [ProcessThread:-1:PrepRequestProcessor@411] - Got user-level KeeperException when processing sessionid:0x13ff35fbbe50002 type:create cxid:0x51e84a26 zxid:0xfffffffffffffffe txntype:unknown reqpath:n/a Error Path:/mesos Error:KeeperErrorCode = NodeExists for /mesos
2013-07-18 16:05:23,991 - INFO [ProcessThread:-1:PrepRequestProcessor@411] - Got user-level KeeperException when processing sessionid:0x13ff35fbbe50001 type:create cxid:0x51e84a2a zxid:0xfffffffffffffffe txntype:unknown reqpath:n/a Error Path:/airbnb Error:KeeperErrorCode = NodeExists for /airbnb
2013-07-18 16:05:23,992 - INFO [ProcessThread:-1:PrepRequestProcessor@411] - Got user-level KeeperException when processing sessionid:0x13ff35fbbe50001 type:create cxid:0x51e84a2b zxid:0xfffffffffffffffe txntype:unknown reqpath:n/a Error Path:/airbnb/service Error:KeeperErrorCode = NodeExists for /airbnb/service
2013-07-18 16:05:23,993 - INFO [ProcessThread:-1:PrepRequestProcessor@411] - Got user-level KeeperException when processing sessionid:0x13ff35fbbe50001 type:create cxid:0x51e84a2c zxid:0xfffffffffffffffe txntype:unknown reqpath:n/a Error Path:/airbnb/service/chronos Error:KeeperErrorCode = NodeExists for /airbnb/service/chronos
2013-07-18 16:05:23,994 - INFO [ProcessThread:-1:PrepRequestProcessor@411] - Got user-level KeeperException when processing sessionid:0x13ff35fbbe50001 type:create cxid:0x51e84a2d zxid:0xfffffffffffffffe txntype:unknown reqpath:n/a Error Path:/airbnb/service/chronos/state Error:KeeperErrorCode = NodeExists for /airbnb/service/chronos/state

build error chronos - maven errors

Hello airbnb and others,

I have been looking at chronos and have been pretty excited about getting it running.

I have tried installing Chronos multiple times following directions and have not had success.

So I decided to rebuild everything from scratch, including zookeeper and mesos.

I downloaded zokeeper-3.4.4. (If I use included zookeeper I cannot get mesos to connect.)
Extracted to /opt/zoo
I setup conf file:
tickTime=2000
initLimit=10
syncLimit=5
dataDir=/opt/zoo/data
dataLogDir=/opt/zoo/log
clientPort=2181

I create /opt/zoo/data and log

I run bin/zkServer.sh start and all is working.
netstat -an | grep 2181
tcp 0 0 0.0.0.0:2181 0.0.0.0:* LISTEN
tcp 0 0 127.0.0.1:2181 127.0.0.1:54670 ESTABLISHED

I extract mesos-0.12.0-incubating.tar.gz to /opt/mesos
cd /opt/mesos
./bootstrap
./configure --with-webui --disable-perftools (If I use included zookeeper I cannot get mesos to connect.)
make
make install

(I also perform the same on a second server.)

On first server I execute:
/usr/local/sbin/mesos-master --zk=zk://localhost:2181/mesos
/usr/local/sbin/mesos-slave --master=zk://localhost:2181/mesos

I http to server and success!! I see master and 1 slave.

I hop on to the second box and execute:
/usr/local/sbin/mesos-slave --master=zk://192.168.1.100:5050

This also connects as slave! (I'm not sure if this is the proper way to setup 2nd server, but we will come back as this is the farthest I have gotten.)

Now to Chronos
Download zip file and extract to /opt/chronos-master
cd chronos-master
maven package
Results :

Tests in error:
PersistenceStoreSpec » Throwable UnsupportedClassVersionError: org/apache/meso...
JobUtilsSpec » Throwable UnsupportedClassVersionError: org/apache/mesos/state/...

Tests run: 36, Failures: 0, Errors: 2, Skipped: 0

[INFO] ------------------------------------------------------------------------
[INFO] BUILD FAILURE
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 2:33.751s
[INFO] Finished at: Mon Jul 22 18:39:24 EDT 2013
[INFO] Final Memory: 38M/524M
[INFO] ------------------------------------------------------------------------
[ERROR] Failed to execute goal org.apache.maven.plugins:maven-surefire-plugin:2.14.1:test (default-test) on project chronos: There are test failures.
[ERROR]
[ERROR] Please refer to /opt/chronos-master/target/surefire-reports for the individual test results.
[ERROR] -> [Help 1]
[ERROR]
[ERROR] To see the full stack trace of the errors, re-run Maven with the -e switch.
[ERROR] Re-run Maven using the -X switch to enable full debug logging.
[ERROR]
[ERROR] For more information about the errors and possible solutions, please read the following articles:
[ERROR] [Help 1] http://cwiki.apache.org/confluence/display/MAVEN/MojoFailureException

so I run:
mvn clean install -U
same results.

If I try using installer script it ends in build failures.

Both systems:
uname -a
Linux CHN01 2.6.32-358.14.1.el6.x86_64 #1 SMP Mon Jun 17 15:54:20 EDT 2013 x86_64 x86_64 x86_64 GNU/Linux
2 CPU
4 GB RAM
20 GB HDD (Only chronos running here.)

Thoughts?
Thanks!

Failed to execute goal org.scalastyle:scalastyle-maven-plugin:0.2.0:check (default)

When installing chronos with mvn install the build fails with:

Failed to execute goal org.scalastyle:scalastyle-maven-plugin:0.2.0:check (default)

Using installer.bash I get the sed error at #4

Deleting all jobs doesn't work

In the README it says you can send a DELETE to /scheduler/jobs, but when you try and do it you get an error 405 Method Not Allowed. Looking through JobManagementResource.scala, the only DELETE path is to delete a singular job. Is deleting all jobs not implemented yet or am I just not doing it right? I see in TaskManagementResource the ability to delete all tasks currently running which is maybe what the README is talking about?

curl -L -X DELETE nyvimeochrono:4400/scheduler/jobs -v

About to connect() to nyvimeochrono port 4400 (#0)
Trying 192.168.7.249... connected
Connected to nyvimeochrono (192.168.7.249) port 4400 (#0)

DELETE /scheduler/jobs HTTP/1.1
User-Agent: curl/7.21.4 (universal-apple-darwin11.0) libcurl/7.21.4 OpenSSL/0.9.8r zlib/1.2.5
Host: nyvimeochrono:4400
Accept: /

< HTTP/1.1 405 Method Not Allowed
< Date: Thu, 13 Jun 2013 15:57:28 GMT
< Allow: GET,OPTIONS,HEAD
< Content-Length: 0
<
Connection #0 to host nyvimeochrono left intact
Closing connection #0

Chronos jobs can't be renamed

Once a job is created, it can not be renamed.

The reason for this is that the job graph uses the job name as the primary key for the job.

We need to implement a fix for job renaming by either a) changing the primary key to be independent of the job name or b) allow changing the name by changing all references to the job name to the new job name.

I create a job and I loose it when restart chronos.

when I run a job the following command line:
sudo curl-L-X GET chronos: 4400/scheduler/jobs. Then I can see the work I have scheduled.
My problem is that when I restart chronos the job disappears.

Thanks,

Error 405 Method Not Allowed

hello ,

what is the authorization scheme used by chronos , please ?

trying to manually start a job as documented in https://github.com/airbnb/chronos#manually-starting-a-job .

https://gist.github.com/bentrigat/5505642

Chronos Install fails on Ubuntu 12.04.2 LTS

I am attempting to install Chronos on Ubuntu 12.04.2 LTS and so far have been unsuccessful.

I am following the instructions the install Chronos on Amazon Linux instructions with a few minor tweaks (just documenting them here in case they should be incorporated into the README).

First I had to change the command export JAVA_HOME=/usr/lib/jvm/java-1.6.0-openjdk-1.6.0.0.x86_64/ to export JAVA_HOME=/usr/lib/jvm/java-1.6.0-openjdk-i386.

Second, I got errors when trying to make mesos. I found a fix at http://mail-archives.apache.org/mod_mbox/incubator-mesos-dev/201303.mbox/%3CCA+8RcoRzFA7e_PMuFOchnS12AXRe24nWu_WvKHKHroHwqqty1A@mail.gmail.com%3E and had to change line 278 of mesos/src/files/files.cpp from length = std::min(length, sysconf(_SC_PAGE_SIZE) * 16); to length = std::min(length, (ssize_t) sysconf(_SC_PAGE_SIZE) * 16);

I'm at the step where I execute "mvn package". My Maven Version is:

Apache Maven 3.0.4
Maven home: /usr/share/maven
Java version: 1.6.0_27, vendor: Sun Microsystems Inc.
Java home: /usr/lib/jvm/java-6-openjdk-i386/jre
Default locale: en_US, platform encoding: UTF-8
OS name: "linux", version: "3.2.0-38-generic-pae", arch: "i386", family: "unix"

Below is the truncated output. Any suggestions or bug fixes I can make to get this going?

Tests run: 0, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 88.866 sec
Running com.airbnb.scheduler.jobs.JobSchedulerIntegrationTest
Mar 25, 2013 9:36:59 AM com.airbnb.scheduler.graph.JobGraph addVertex
WARNING: Adding vertex:job1
Mar 25, 2013 9:36:59 AM com.airbnb.scheduler.graph.JobGraph addVertex
WARNING: Current number of vertices:1
Mar 25, 2013 9:36:59 AM com.airbnb.scheduler.jobs.JobScheduler$$anonfun$registerJob$3 apply
INFO: Persisting job:job1
Mar 25, 2013 9:36:59 AM com.airbnb.scheduler.jobs.JobScheduler addSchedule
INFO: Adding schedule for time:1:00:00 AM UTC
Mar 25, 2013 9:36:59 AM com.airbnb.scheduler.jobs.JobScheduler iteration
INFO: Checking schedules with time horizon:PT2H
Mar 25, 2013 9:36:59 AM com.airbnb.scheduler.jobs.JobScheduler next
INFO: Calling next for stream: R5/2012-01-01T00:00:00.000Z/P1D, jobname: job1
Mar 25, 2013 9:36:59 AM com.airbnb.scheduler.jobs.JobScheduler next
INFO: Task ready for scheduling: 2012-01-01T00:00:00.000Z
Mar 25, 2013 9:36:59 AM com.airbnb.scheduler.jobs.JobScheduler com$airbnb$scheduler$jobs$JobScheduler$$scheduleStream
INFO: Scheduling:job1
Mar 25, 2013 9:36:59 AM com.airbnb.scheduler.jobs.TaskManager scheduleDelayedTask
INFO: Scheduling task 'ct:job1:1325376000000:0' with delay: '0'
Mar 25, 2013 9:36:59 AM com.airbnb.scheduler.jobs.JobScheduler com$airbnb$scheduler$jobs$JobScheduler$$scheduleStream
INFO: Saving updated job:ScheduleBasedJob(R4/2012-01-02T00:00:00.000Z/P1D,job1,CMD,PT2H,0,0,,,2,,,,false)
Mar 25, 2013 9:36:59 AM com.airbnb.scheduler.jobs.JobScheduler next
INFO: Calling next for stream: R4/2012-01-02T00:00:00.000Z/P1D, jobname: job1
Mar 25, 2013 9:36:59 AM com.airbnb.scheduler.jobs.JobScheduler iteration
INFO: Checking schedules with time horizon:PT2H
Mar 25, 2013 9:36:59 AM com.airbnb.scheduler.jobs.JobScheduler next
INFO: Calling next for stream: R4/2012-01-02T00:00:00.000Z/P1D, jobname: job1
Mar 25, 2013 9:36:59 AM com.airbnb.scheduler.jobs.ScheduledTask call
INFO: Triggering: 'job1'
Mar 25, 2013 9:36:59 AM com.airbnb.scheduler.jobs.TaskManager removeTaskFutureMapping
INFO: Removing task mapping
Mar 25, 2013 9:36:59 AM com.airbnb.scheduler.graph.JobGraph addVertex
WARNING: Adding vertex:FOO
Mar 25, 2013 9:36:59 AM com.airbnb.scheduler.graph.JobGraph addVertex
WARNING: Current number of vertices:1
Mar 25, 2013 9:36:59 AM com.airbnb.scheduler.jobs.JobScheduler$$anonfun$registerJob$3 apply
INFO: Persisting job:FOO
Mar 25, 2013 9:36:59 AM com.airbnb.scheduler.jobs.JobScheduler addSchedule
INFO: Adding schedule for time:12:05:01 AM UTC
Mar 25, 2013 9:36:59 AM com.airbnb.scheduler.jobs.JobScheduler iteration
INFO: Checking schedules with time horizon:PT5M
Mar 25, 2013 9:36:59 AM com.airbnb.scheduler.jobs.JobScheduler next
INFO: Calling next for stream: R/2012-01-01T00:00:00.000Z/PT1M, jobname: FOO
Mar 25, 2013 9:36:59 AM com.airbnb.scheduler.jobs.JobScheduler run
INFO: Starting run loop for JobScheduler. CurrentTime: 2013-03-25T13:36:59.882Z
Mar 25, 2013 9:36:59 AM com.airbnb.scheduler.jobs.JobScheduler run
INFO: No longer running.
Mar 25, 2013 9:36:59 AM com.airbnb.scheduler.graph.JobGraph getExecutableChildren
INFO: Dependents: []
Mar 25, 2013 9:36:59 AM com.airbnb.scheduler.graph.JobGraph getExecutableChildren
INFO: Dependents: []
Mar 25, 2013 9:36:59 AM com.airbnb.scheduler.jobs.JobScheduler handleFailedTask
WARNING: Task of job: FOO failed.
Mar 25, 2013 9:36:59 AM com.airbnb.scheduler.jobs.JobScheduler handleFailedTask
WARNING: Retrying job: FOO, attempt: 0
Tests run: 0, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.488 sec
Running com.airbnb.scheduler.jobs.graph.JobGraphSpec
Mar 25, 2013 9:37:00 AM com.airbnb.scheduler.graph.JobGraph addVertex
WARNING: Adding vertex:A
Mar 25, 2013 9:37:00 AM com.airbnb.scheduler.graph.JobGraph addVertex
WARNING: Current number of vertices:1
Mar 25, 2013 9:37:00 AM com.airbnb.scheduler.graph.JobGraph addVertex
WARNING: Adding vertex:B
Mar 25, 2013 9:37:00 AM com.airbnb.scheduler.graph.JobGraph addVertex
WARNING: Current number of vertices:2
Mar 25, 2013 9:37:00 AM com.airbnb.scheduler.graph.JobGraph addVertex
WARNING: Adding vertex:A
Mar 25, 2013 9:37:00 AM com.airbnb.scheduler.graph.JobGraph addVertex
WARNING: Current number of vertices:1
Mar 25, 2013 9:37:00 AM com.airbnb.scheduler.graph.JobGraph addVertex
WARNING: Adding vertex:B
Mar 25, 2013 9:37:00 AM com.airbnb.scheduler.graph.JobGraph addVertex
WARNING: Current number of vertices:2
Mar 25, 2013 9:37:00 AM com.airbnb.scheduler.graph.JobGraph addVertex
WARNING: Adding vertex:C
Mar 25, 2013 9:37:00 AM com.airbnb.scheduler.graph.JobGraph addVertex
WARNING: Current number of vertices:3
Mar 25, 2013 9:37:00 AM com.airbnb.scheduler.graph.JobGraph addVertex
WARNING: Adding vertex:A
Mar 25, 2013 9:37:00 AM com.airbnb.scheduler.graph.JobGraph addVertex
WARNING: Current number of vertices:1
Mar 25, 2013 9:37:00 AM com.airbnb.scheduler.graph.JobGraph addVertex
WARNING: Adding vertex:A
Mar 25, 2013 9:37:00 AM com.airbnb.scheduler.graph.JobGraph addVertex
WARNING: Adding vertex:A
Mar 25, 2013 9:37:00 AM com.airbnb.scheduler.graph.JobGraph addVertex
WARNING: Current number of vertices:1
Mar 25, 2013 9:37:00 AM com.airbnb.scheduler.graph.JobGraph addVertex
WARNING: Adding vertex:A
Mar 25, 2013 9:37:00 AM com.airbnb.scheduler.graph.JobGraph addVertex
WARNING: Adding vertex:A
Mar 25, 2013 9:37:00 AM com.airbnb.scheduler.graph.JobGraph addVertex
WARNING: Current number of vertices:1
Mar 25, 2013 9:37:00 AM com.airbnb.scheduler.graph.JobGraph addVertex
WARNING: Adding vertex:B
Mar 25, 2013 9:37:00 AM com.airbnb.scheduler.graph.JobGraph addVertex
WARNING: Current number of vertices:2
Mar 25, 2013 9:37:00 AM com.airbnb.scheduler.graph.JobGraph addVertex
WARNING: Adding vertex:C
Mar 25, 2013 9:37:00 AM com.airbnb.scheduler.graph.JobGraph addVertex
WARNING: Current number of vertices:3
Mar 25, 2013 9:37:00 AM com.airbnb.scheduler.graph.JobGraph addVertex
WARNING: Adding vertex:D
Mar 25, 2013 9:37:00 AM com.airbnb.scheduler.graph.JobGraph addVertex
WARNING: Current number of vertices:4
Mar 25, 2013 9:37:00 AM com.airbnb.scheduler.graph.JobGraph addVertex
WARNING: Adding vertex:A
Mar 25, 2013 9:37:00 AM com.airbnb.scheduler.graph.JobGraph addVertex
WARNING: Current number of vertices:1
Mar 25, 2013 9:37:00 AM com.airbnb.scheduler.graph.JobGraph addVertex
WARNING: Adding vertex:B
Mar 25, 2013 9:37:00 AM com.airbnb.scheduler.graph.JobGraph addVertex
WARNING: Current number of vertices:2
Mar 25, 2013 9:37:00 AM com.airbnb.scheduler.graph.JobGraph addVertex
WARNING: Adding vertex:C
Mar 25, 2013 9:37:00 AM com.airbnb.scheduler.graph.JobGraph addVertex
WARNING: Current number of vertices:3
Mar 25, 2013 9:37:00 AM com.airbnb.scheduler.graph.JobGraph addVertex
WARNING: Adding vertex:D
Mar 25, 2013 9:37:00 AM com.airbnb.scheduler.graph.JobGraph addVertex
WARNING: Current number of vertices:4
Mar 25, 2013 9:37:00 AM com.airbnb.scheduler.graph.JobGraph addVertex
WARNING: Adding vertex:E
Mar 25, 2013 9:37:00 AM com.airbnb.scheduler.graph.JobGraph addVertex
WARNING: Current number of vertices:5
Mar 25, 2013 9:37:00 AM com.airbnb.scheduler.graph.JobGraph getExecutableChildren
INFO: Dependents: [B,C]
Mar 25, 2013 9:37:00 AM com.airbnb.scheduler.graph.JobGraph getExecutableChildren
INFO: Dependents: [D]
Mar 25, 2013 9:37:00 AM com.airbnb.scheduler.graph.JobGraph getExecutableChildren
INFO: Dependents: [E]
Mar 25, 2013 9:37:00 AM com.airbnb.scheduler.graph.JobGraph getExecutableChildren
INFO: Dependents: [B,C]
Mar 25, 2013 9:37:00 AM com.airbnb.scheduler.graph.JobGraph getExecutableChildren
INFO: Dependents: []
Mar 25, 2013 9:37:00 AM com.airbnb.scheduler.graph.JobGraph getExecutableChildren
INFO: Dependents: [E,D]
Mar 25, 2013 9:37:00 AM com.airbnb.scheduler.graph.JobGraph addVertex
WARNING: Adding vertex:A
Mar 25, 2013 9:37:00 AM com.airbnb.scheduler.graph.JobGraph addVertex
WARNING: Current number of vertices:1
Mar 25, 2013 9:37:00 AM com.airbnb.scheduler.graph.JobGraph addVertex
WARNING: Adding vertex:B
Mar 25, 2013 9:37:00 AM com.airbnb.scheduler.graph.JobGraph addVertex
WARNING: Current number of vertices:2
Mar 25, 2013 9:37:00 AM com.airbnb.scheduler.graph.JobGraph addVertex
WARNING: Adding vertex:C
Mar 25, 2013 9:37:00 AM com.airbnb.scheduler.graph.JobGraph addVertex
WARNING: Current number of vertices:3
Tests run: 0, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.443 sec
Running com.airbnb.scheduler.jobs.ScheduleStreamSpec
Tests run: 0, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.001 sec
Running com.airbnb.scheduler.jobs.TaskManagerSpec
Tests run: 0, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.003 sec
Running com.airbnb.scheduler.jobs.JobUtilsSpec
09:37:00.951 [specs2.DefaultExecutionStrategy1] INFO  org.apache.zookeeper.ZooKeeper - Client environment:zookeeper.version=3.3.3-1203054, built on 11/17/2011 05:47 GMT
09:37:00.967 [specs2.DefaultExecutionStrategy1] INFO  org.apache.zookeeper.ZooKeeper - Client environment:host.name=deldrid1-VirtualBox
09:37:00.968 [specs2.DefaultExecutionStrategy1] INFO  org.apache.zookeeper.ZooKeeper - Client environment:java.version=1.6.0_27
09:37:00.968 [specs2.DefaultExecutionStrategy1] INFO  org.apache.zookeeper.ZooKeeper - Client environment:java.vendor=Sun Microsystems Inc.
09:37:00.968 [specs2.DefaultExecutionStrategy1] INFO  org.apache.zookeeper.ZooKeeper - Client environment:java.home=/usr/lib/jvm/java-6-openjdk-i386/jre
09:37:00.968 [specs2.DefaultExecutionStrategy1] INFO  org.apache.zookeeper.ZooKeeper - Client environment:java.class.path=/home/deldrid1/chronos/target/test-classes:/home/deldrid1/chronos/target/classes:/home/deldrid1/.m2/repository/org/scala-lang/scala-library/2.9.1/scala-library-2.9.1.jar:/home/deldrid1/.m2/repository/org/apache/mesos/mesos/0.12.0-SNAPSHOT_JDK1.7/mesos-0.12.0-SNAPSHOT_JDK1.7.jar:/home/deldrid1/.m2/repository/com/google/guava/guava/12.0/guava-12.0.jar:/home/deldrid1/.m2/repository/com/google/code/findbugs/jsr305/1.3.9/jsr305-1.3.9.jar:/home/deldrid1/.m2/repository/com/google/inject/guice/3.0/guice-3.0.jar:/home/deldrid1/.m2/repository/javax/inject/javax.inject/1/javax.inject-1.jar:/home/deldrid1/.m2/repository/aopalliance/aopalliance/1.0/aopalliance-1.0.jar:/home/deldrid1/.m2/repository/com/google/inject/extensions/guice-servlet/3.0/guice-servlet-3.0.jar:/home/deldrid1/.m2/repository/com/google/protobuf/protobuf-java/2.4.1/protobuf-java-2.4.1.jar:/home/deldrid1/.m2/repository/org/joda/joda-convert/1.2/joda-convert-1.2.jar:/home/deldrid1/.m2/repository/joda-time/joda-time/2.1/joda-time-2.1.jar:/home/deldrid1/.m2/repository/com/yammer/dropwizard/dropwizard-core/0.6.1/dropwizard-core-0.6.1.jar:/home/deldrid1/.m2/repository/com/sun/jersey/jersey-core/1.15/jersey-core-1.15.jar:/home/deldrid1/.m2/repository/com/sun/jersey/jersey-server/1.15/jersey-server-1.15.jar:/home/deldrid1/.m2/repository/asm/asm/3.1/asm-3.1.jar:/home/deldrid1/.m2/repository/com/sun/jersey/jersey-servlet/1.15/jersey-servlet-1.15.jar:/home/deldrid1/.m2/repository/com/yammer/metrics/metrics-core/2.2.0/metrics-core-2.2.0.jar:/home/deldrid1/.m2/repository/com/yammer/metrics/metrics-servlet/2.2.0/metrics-servlet-2.2.0.jar:/home/deldrid1/.m2/repository/com/yammer/metrics/metrics-jetty/2.2.0/metrics-jetty-2.2.0.jar:/home/deldrid1/.m2/repository/com/yammer/metrics/metrics-logback/2.2.0/metrics-logback-2.2.0.jar:/home/deldrid1/.m2/repository/com/yammer/metrics/metrics-jersey/2.2.0/metrics-jersey-2.2.0.jar:/home/deldrid1/.m2/repository/com/yammer/metrics/metrics-annotation/2.2.0/metrics-annotation-2.2.0.jar:/home/deldrid1/.m2/repository/com/fasterxml/jackson/core/jackson-databind/2.1.1/jackson-databind-2.1.1.jar:/home/deldrid1/.m2/repository/com/fasterxml/jackson/core/jackson-annotations/2.1.1/jackson-annotations-2.1.1.jar:/home/deldrid1/.m2/repository/com/fasterxml/jackson/core/jackson-core/2.1.1/jackson-core-2.1.1.jar:/home/deldrid1/.m2/repository/com/fasterxml/jackson/jaxrs/jackson-jaxrs-json-provider/2.1.1/jackson-jaxrs-json-provider-2.1.1.jar:/home/deldrid1/.m2/repository/com/fasterxml/jackson/module/jackson-module-jaxb-annotations/2.1.1/jackson-module-jaxb-annotations-2.1.1.jar:/home/deldrid1/.m2/repository/com/fasterxml/jackson/dataformat/jackson-dataformat-yaml/2.1.1/jackson-dataformat-yaml-2.1.1.jar:/home/deldrid1/.m2/repository/com/fasterxml/jackson/datatype/jackson-datatype-guava/2.1.1/jackson-datatype-guava-2.1.1.jar:/home/deldrid1/.m2/repository/net/sourceforge/argparse4j/argparse4j/0.2.2/argparse4j-0.2.2.jar:/home/deldrid1/.m2/repository/org/slf4j/slf4j-api/1.7.2/slf4j-api-1.7.2.jar:/home/deldrid1/.m2/repository/org/slf4j/jul-to-slf4j/1.7.2/jul-to-slf4j-1.7.2.jar:/home/deldrid1/.m2/repository/ch/qos/logback/logback-core/1.0.7/logback-core-1.0.7.jar:/home/deldrid1/.m2/repository/ch/qos/logback/logback-classic/1.0.7/logback-classic-1.0.7.jar:/home/deldrid1/.m2/repository/org/slf4j/log4j-over-slf4j/1.7.2/log4j-over-slf4j-1.7.2.jar:/home/deldrid1/.m2/repository/org/eclipse/jetty/jetty-server/8.1.8.v20121106/jetty-server-8.1.8.v20121106.jar:/home/deldrid1/.m2/repository/org/eclipse/jetty/orbit/javax.servlet/3.0.0.v201112011016/javax.servlet-3.0.0.v201112011016.jar:/home/deldrid1/.m2/repository/org/eclipse/jetty/jetty-continuation/8.1.8.v20121106/jetty-continuation-8.1.8.v20121106.jar:/home/deldrid1/.m2/repository/org/eclipse/jetty/jetty-servlet/8.1.8.v20121106/jetty-servlet-8.1.8.v20121106.jar:/home/deldrid1/.m2/repository/org/eclipse/jetty/jetty-security/8.1.8.v20121106/jetty-security-8.1.8.v20121106.jar:/home/deldrid1/.m2/repository/org/eclipse/jetty/jetty-http/8.1.8.v20121106/jetty-http-8.1.8.v20121106.jar:/home/deldrid1/.m2/repository/org/eclipse/jetty/jetty-io/8.1.8.v20121106/jetty-io-8.1.8.v20121106.jar:/home/deldrid1/.m2/repository/org/eclipse/jetty/jetty-util/8.1.8.v20121106/jetty-util-8.1.8.v20121106.jar:/home/deldrid1/.m2/repository/org/hibernate/hibernate-validator/4.3.0.Final/hibernate-validator-4.3.0.Final.jar:/home/deldrid1/.m2/repository/javax/validation/validation-api/1.0.0.GA/validation-api-1.0.0.GA.jar:/home/deldrid1/.m2/repository/org/jboss/logging/jboss-logging/3.1.0.CR2/jboss-logging-3.1.0.CR2.jar:/home/deldrid1/.m2/repository/com/fasterxml/jackson/datatype/jackson-datatype-joda/2.1.1/jackson-datatype-joda-2.1.1.jar:/home/deldrid1/.m2/repository/com/yammer/dropwizard/dropwizard-scala_2.9.1/0.6.1/dropwizard-scala_2.9.1-0.6.1.jar:/home/deldrid1/.m2/repository/com/fasterxml/jackson/module/jackson-module-scala/2.1.1/jackson-module-scala-2.1.1.jar:/home/deldrid1/.m2/repository/org/scalastuff/scalabeans/0.3/scalabeans-0.3.jar:/home/deldrid1/.m2/repository/com/thoughtworks/paranamer/paranamer/2.3/paranamer-2.3.jar:/home/deldrid1/.m2/repository/com/yammer/metrics/metrics-scala_2.9.1/2.2.0/metrics-scala_2.9.1-2.2.0.jar:/home/deldrid1/.m2/repository/com/bazaarvoice/dropwizard/dropwizard-configurable-assets-bundle/0.1.9/dropwizard-configurable-assets-bundle-0.1.9.jar:/home/deldrid1/.m2/repository/net/sf/jgrapht/jgrapht/0.8.3/jgrapht-0.8.3.jar:/home/deldrid1/.m2/repository/com/twitter/common/zookeeper/client/0.0.18/client-0.0.18.jar:/home/deldrid1/.m2/repository/com/twitter/common/base/0.0.46/base-0.0.46.jar:/home/deldrid1/.m2/repository/com/twitter/common/util-system-mocks/0.0.35/util-system-mocks-0.0.35.jar:/home/deldrid1/.m2/repository/com/twitter/common/net-util/0.0.38/net-util-0.0.38.jar:/home/deldrid1/.m2/repository/commons-codec/commons-codec/1.6/commons-codec-1.6.jar:/home/deldrid1/.m2/repository/com/twitter/common/quantity/0.0.35/quantity-0.0.35.jar:/home/deldrid1/.m2/repository/org/apache/zookeeper/zookeeper/3.3.4/zookeeper-3.3.4.jar:/home/deldrid1/.m2/repository/log4j/log4j/1.2.15/log4j-1.2.15.jar:/home/deldrid1/.m2/repository/com/twitter/common/zookeeper/candidate/0.0.31/candidate-0.0.31.jar:/home/deldrid1/.m2/repository/com/twitter/common/zookeeper/group/0.0.31/group-0.0.31.jar:/home/deldrid1/.m2/repository/com/twitter/common/util/0.0.61/util-0.0.61.jar:/home/deldrid1/.m2/repository/com/twitter/common/util-executor-service-shutdown/0.0.23/util-executor-service-shutdown-0.0.23.jar:/home/deldrid1/.m2/repository/com/twitter/common/jdk-logging/0.0.23/jdk-logging-0.0.23.jar:/home/deldrid1/.m2/repository/com/twitter/common/stats/0.0.61/stats-0.0.61.jar:/home/deldrid1/.m2/repository/com/twitter/common/stat-registry/0.0.9/stat-registry-0.0.9.jar:/home/deldrid1/.m2/repository/com/twitter/common/stat/0.0.9/stat-0.0.9.jar:/home/deldrid1/.m2/repository/com/twitter/common/stats-provider/0.0.34/stats-provider-0.0.34.jar:/home/deldrid1/.m2/repository/com/twitter/common/application-action/0.0.41/application-action-0.0.41.jar:/home/deldrid1/.m2/repository/com/twitter/common/util-sampler/0.0.31/util-sampler-0.0.31.jar:/home/deldrid1/.m2/repository/com/twitter/common/collections/0.0.42/collections-0.0.42.jar:/home/deldrid1/.m2/repository/commons-lang/commons-lang/2.4/commons-lang-2.4.jar:/home/deldrid1/.m2/repository/org/apache/commons/commons-email/1.2/commons-email-1.2.jar:/home/deldrid1/.m2/repository/javax/mail/mail/1.4.1/mail-1.4.1.jar:/home/deldrid1/.m2/repository/javax/activation/activation/1.1/activation-1.1.jar:/home/deldrid1/.m2/repository/org/apache/commons/commons-math3/3.1.1/commons-math3-3.1.1.jar:/home/deldrid1/.m2/repository/junit/junit/4.8.1/junit-4.8.1.jar:/home/deldrid1/.m2/repository/org/mockito/mockito-all/1.9.5/mockito-all-1.9.5.jar:/home/deldrid1/.m2/repository/org/specs2/specs2_2.9.1/1.12.4/specs2_2.9.1-1.12.4.jar:/home/deldrid1/.m2/repository/org/specs2/specs2-scalaz-core_2.9.1/6.0.1/specs2-scalaz-core_2.9.1-6.0.1.jar:/home/deldrid1/.m2/repository/org/apache/maven/plugins/maven-surefire-report-plugin/2.13/maven-surefire-report-plugin-2.13.jar:/home/deldrid1/.m2/repository/org/apache/maven/maven-project/2.0.9/maven-project-2.0.9.jar:/home/deldrid1/.m2/repository/org/apache/maven/maven-settings/2.0.9/maven-settings-2.0.9.jar:/home/deldrid1/.m2/repository/org/apache/maven/maven-profile/2.0.9/maven-profile-2.0.9.jar:/home/deldrid1/.m2/repository/org/apache/maven/maven-artifact-manager/2.0.9/maven-artifact-manager-2.0.9.jar:/home/deldrid1/.m2/repository/org/apache/maven/maven-repository-metadata/2.0.9/maven-repository-metadata-2.0.9.jar:/home/deldrid1/.m2/repository/org/apache/maven/wagon/wagon-provider-api/1.0-beta-2/wagon-provider-api-1.0-beta-2.jar:/home/deldrid1/.m2/repository/org/apache/maven/maven-plugin-registry/2.0.9/maven-plugin-registry-2.0.9.jar:/home/deldrid1/.m2/repository/org/apache/maven/maven-model/2.0.9/maven-model-2.0.9.jar:/home/deldrid1/.m2/repository/org/apache/maven/maven-plugin-api/2.0.9/maven-plugin-api-2.0.9.jar:/home/deldrid1/.m2/repository/org/apache/maven/reporting/maven-reporting-api/3.0/maven-reporting-api-3.0.jar:/home/deldrid1/.m2/repository/org/apache/maven/reporting/maven-reporting-impl/2.1/maven-reporting-impl-2.1.jar:/home/deldrid1/.m2/repository/commons-validator/commons-validator/1.2.0/commons-validator-1.2.0.jar:/home/deldrid1/.m2/repository/commons-beanutils/commons-beanutils/1.7.0/commons-beanutils-1.7.0.jar:/home/deldrid1/.m2/repository/commons-digester/commons-digester/1.6/commons-digester-1.6.jar:/home/deldrid1/.m2/repository/commons-logging/commons-logging/1.0.4/commons-logging-1.0.4.jar:/home/deldrid1/.m2/repository/xml-apis/xml-apis/1.0.b2/xml-apis-1.0.b2.jar:/home/deldrid1/.m2/repository/org/apache/maven/doxia/doxia-sink-api/1.1.4/doxia-sink-api-1.1.4.jar:/home/deldrid1/.m2/repository/org/apache/maven/doxia/doxia-logging-api/1.1.4/doxia-logging-api-1.1.4.jar:/home/deldrid1/.m2/repository/org/apache/maven/doxia/doxia-core/1.1.4/doxia-core-1.1.4.jar:/home/deldrid1/.m2/repository/xerces/xercesImpl/2.9.1/xercesImpl-2.9.1.jar:/home/deldrid1/.m2/repository/org/apache/httpcomponents/httpclient/4.0.2/httpclient-4.0.2.jar:/home/deldrid1/.m2/repository/org/apache/httpcomponents/httpcore/4.0.1/httpcore-4.0.1.jar:/home/deldrid1/.m2/repository/org/apache/maven/doxia/doxia-decoration-model/1.1.4/doxia-decoration-model-1.1.4.jar:/home/deldrid1/.m2/repository/org/apache/maven/doxia/doxia-site-renderer/1.1.4/doxia-site-renderer-1.1.4.jar:/home/deldrid1/.m2/repository/org/apache/maven/doxia/doxia-module-xhtml/1.1.4/doxia-module-xhtml-1.1.4.jar:/home/deldrid1/.m2/repository/org/apache/maven/doxia/doxia-module-fml/1.1.4/doxia-module-fml-1.1.4.jar:/home/deldrid1/.m2/repository/org/codehaus/plexus/plexus-i18n/1.0-beta-7/plexus-i18n-1.0-beta-7.jar:/home/deldrid1/.m2/repository/org/codehaus/plexus/plexus-utils/3.0.8/plexus-utils-3.0.8.jar:/home/deldrid1/.m2/repository/org/apache/maven/shared/maven-shared-utils/0.2/maven-shared-utils-0.2.jar:/home/deldrid1/.m2/repository/org/mozilla/rhino/1.7R4/rhino-1.7R4.jar:/home/deldrid1/.m2/repository/com/google/javascript/closure-compiler/r2388/closure-compiler-r2388.jar:/home/deldrid1/.m2/repository/args4j/args4j/2.0.16/args4j-2.0.16.jar:/home/deldrid1/.m2/repository/org/json/json/20090211/json-20090211.jar:/home/deldrid1/.m2/repository/org/apache/ant/ant/1.8.2/ant-1.8.2.jar:/home/deldrid1/.m2/repository/org/apache/ant/ant-launcher/1.8.2/ant-launcher-1.8.2.jar:/home/deldrid1/.m2/repository/com/googlecode/jarjar/jarjar/1.1/jarjar-1.1.jar:/home/deldrid1/.m2/repository/org/apache/maven/plugins/maven-checkstyle-plugin/2.9.1/maven-checkstyle-plugin-2.9.1.jar:/home/deldrid1/.m2/repository/org/apache/maven/maven-artifact/2.2.1/maven-artifact-2.2.1.jar:/home/deldrid1/.m2/repository/org/apache/maven/shared/maven-doxia-tools/1.2.1/maven-doxia-tools-1.2.1.jar:/home/deldrid1/.m2/repository/commons-io/commons-io/1.4/commons-io-1.4.jar:/home/deldrid1/.m2/repository/org/codehaus/plexus/plexus-container-default/1.0-alpha-9/plexus-container-default-1.0-alpha-9.jar:/home/deldrid1/.m2/repository/classworlds/classworlds/1.1-alpha-2/classworlds-1.1-alpha-2.jar:/home/deldrid1/.m2/repository/org/codehaus/plexus/plexus-resources/1.0-alpha-7/plexus-resources-1.0-alpha-7.jar:/home/deldrid1/.m2/repository/org/codehaus/plexus/plexus-interpolation/1.13/plexus-interpolation-1.13.jar:/home/deldrid1/.m2/repository/org/codehaus/plexus/plexus-velocity/1.1.8/plexus-velocity-1.1.8.jar:/home/deldrid1/.m2/repository/com/puppycrawl/tools/checkstyle/5.5/checkstyle-5.5.jar:/home/deldrid1/.m2/repository/antlr/antlr/2.7.7/antlr-2.7.7.jar:/home/deldrid1/.m2/repository/commons-beanutils/commons-beanutils-core/1.8.3/commons-beanutils-core-1.8.3.jar:/home/deldrid1/.m2/repository/commons-cli/commons-cli/1.2/commons-cli-1.2.jar:/home/deldrid1/.m2/repository/com/google/collections/google-collections/1.0/google-collections-1.0.jar:/home/deldrid1/.m2/repository/org/apache/velocity/velocity/1.5/velocity-1.5.jar:/home/deldrid1/.m2/repository/oro/oro/2.0.8/oro-2.0.8.jar:/home/deldrid1/.m2/repository/commons-collections/commons-collections/3.2.1/commons-collections-3.2.1.jar:
09:37:00.969 [specs2.DefaultExecutionStrategy1] INFO  org.apache.zookeeper.ZooKeeper - Client environment:java.library.path=/usr/lib/jvm/java-6-openjdk-i386/jre/lib/i386/client:/usr/lib/jvm/java-6-openjdk-i386/jre/lib/i386:/usr/lib/jvm/java-6-openjdk-i386/jre/../lib/i386:/usr/java/packages/lib/i386:/usr/lib/i386-linux-gnu/jni:/lib/i386-linux-gnu:/usr/lib/i386-linux-gnu:/usr/lib/jni:/lib:/usr/lib
09:37:00.969 [specs2.DefaultExecutionStrategy1] INFO  org.apache.zookeeper.ZooKeeper - Client environment:java.io.tmpdir=/tmp
09:37:00.969 [specs2.DefaultExecutionStrategy1] INFO  org.apache.zookeeper.ZooKeeper - Client environment:java.compiler=<NA>
09:37:00.969 [specs2.DefaultExecutionStrategy1] INFO  org.apache.zookeeper.ZooKeeper - Client environment:os.name=Linux
09:37:00.970 [specs2.DefaultExecutionStrategy1] INFO  org.apache.zookeeper.ZooKeeper - Client environment:os.arch=i386
09:37:00.970 [specs2.DefaultExecutionStrategy1] INFO  org.apache.zookeeper.ZooKeeper - Client environment:os.version=3.2.0-38-generic-pae
09:37:00.970 [specs2.DefaultExecutionStrategy1] INFO  org.apache.zookeeper.ZooKeeper - Client environment:user.name=deldrid1
09:37:00.970 [specs2.DefaultExecutionStrategy1] INFO  org.apache.zookeeper.ZooKeeper - Client environment:user.home=/home/deldrid1
09:37:00.975 [specs2.DefaultExecutionStrategy1] INFO  org.apache.zookeeper.ZooKeeper - Client environment:user.dir=/home/deldrid1/chronos
Tests run: 0, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.029 sec

Results :

Tests run: 0, Failures: 0, Errors: 0, Skipped: 0

[INFO] ------------------------------------------------------------------------
[INFO] BUILD FAILURE
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 1:47.413s
[INFO] Finished at: Mon Mar 25 09:37:02 EDT 2013
[INFO] Final Memory: 15M/52M
[INFO] ------------------------------------------------------------------------
[ERROR] Failed to execute goal org.apache.maven.plugins:maven-surefire-plugin:2.13:test (default-test) on project chronos: Execution default-test of goal org.apache.maven.plugins:maven-surefire-plugin:2.13:test failed: There was an error in the forked process
[ERROR] org.apache.maven.surefire.testset.TestSetFailedException: java.lang.ArrayIndexOutOfBoundsException: 0; nested exception is java.lang.ArrayIndexOutOfBoundsException: 0
[ERROR] java.lang.ArrayIndexOutOfBoundsException: 0
[ERROR] at org.apache.maven.surefire.report.SmartStackTraceParser.rootIsInclass(SmartStackTraceParser.java:176)
[ERROR] at org.apache.maven.surefire.report.SmartStackTraceParser.getString(SmartStackTraceParser.java:131)
[ERROR] at org.apache.maven.surefire.common.junit4.JUnit4StackTraceWriter.smartTrimmedStackTrace(JUnit4StackTraceWriter.java:73)
[ERROR] at org.apache.maven.surefire.booter.ForkingRunListener.encode(ForkingRunListener.java:328)
[ERROR] at org.apache.maven.surefire.booter.ForkingRunListener.encode(ForkingRunListener.java:312)
[ERROR] at org.apache.maven.surefire.booter.ForkingRunListener.toString(ForkingRunListener.java:258)
[ERROR] at org.apache.maven.surefire.booter.ForkingRunListener.testError(ForkingRunListener.java:131)
[ERROR] at org.apache.maven.surefire.common.junit4.JUnit4RunListener.testFailure(JUnit4RunListener.java:111)
[ERROR] at org.junit.runner.notification.RunNotifier$4.notifyListener(RunNotifier.java:100)
[ERROR] at org.junit.runner.notification.RunNotifier$SafeNotifier.run(RunNotifier.java:41)
[ERROR] at org.junit.runner.notification.RunNotifier.fireTestFailure(RunNotifier.java:97)
[ERROR] at org.specs2.reporter.JUnitReporter$class.org$specs2$reporter$JUnitReporter$$notifyResult(JUnitReporter.scala:75)
[ERROR] at org.specs2.reporter.JUnitReporter$$anonfun$executeFragment$1.apply(JUnitReporter.scala:55)
[ERROR] at org.specs2.reporter.JUnitReporter$$anonfun$executeFragment$1.apply(JUnitReporter.scala:46)
[ERROR] at org.specs2.reporter.DefaultExecutionStrategy$class.executeWithBarrier$1(ExecutionStrategy.scala:102)
[ERROR] at org.specs2.reporter.DefaultExecutionStrategy$$anonfun$executeConcurrently$1$$anonfun$apply$2.apply(ExecutionStrategy.scala:104)
[ERROR] at org.specs2.reporter.DefaultExecutionStrategy$$anonfun$executeConcurrently$1$$anonfun$apply$2.apply(ExecutionStrategy.scala:104)
[ERROR] at org.specs2.internal.scalaz.concurrent.Promise$Unfulfilled$.fulfill(Promise.scala:100)
[ERROR] at org.specs2.internal.scalaz.concurrent.Promise$Done.eval(Promise.scala:127)
[ERROR] at org.specs2.internal.scalaz.concurrent.Promise$$anonfun$1.apply(Promise.scala:15)
[ERROR] at org.specs2.internal.scalaz.concurrent.Promise$$anonfun$1.apply(Promise.scala:15)
[ERROR] at org.specs2.internal.scalaz.concurrent.Actor$$anonfun$3.apply(Actor.scala:31)
[ERROR] at org.specs2.internal.scalaz.concurrent.Actor$$anonfun$3.apply(Actor.scala:25)
[ERROR] at org.specs2.internal.scalaz.concurrent.Effects$$anon$1$$anonfun$1.apply(Effect.scala:13)
[ERROR] at org.specs2.internal.scalaz.concurrent.Effects$$anon$1$$anonfun$1.apply(Effect.scala:13)
[ERROR] at org.specs2.internal.scalaz.concurrent.Effect$$anonfun$$bang$1.apply$mcV$sp(Effect.scala:8)
[ERROR] at org.specs2.internal.scalaz.concurrent.Effect$$anonfun$$bang$1.apply(Effect.scala:8)
[ERROR] at org.specs2.internal.scalaz.concurrent.Effect$$anonfun$$bang$1.apply(Effect.scala:8)
[ERROR] at org.specs2.internal.scalaz.concurrent.StrategyLow$$anon$3$$anon$4.call(Strategy.scala:35)
[ERROR] at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334)
[ERROR] at java.util.concurrent.FutureTask.run(FutureTask.java:166)
[ERROR] at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1146)
[ERROR] at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
[ERROR] at java.lang.Thread.run(Thread.java:679)
[ERROR] -> [Help 1]
[ERROR] 
[ERROR] To see the full stack trace of the errors, re-run Maven with the -e switch.
[ERROR] Re-run Maven using the -X switch to enable full debug logging.
[ERROR] 
[ERROR] For more information about the errors and possible solutions, please read the following articles:
[ERROR] [Help 1] http://cwiki.apache.org/confluence/display/MAVEN/PluginExecutionException
deldrid1@deldrid1-VirtualBox:~/chronos$ mvn - version
[INFO] Scanning for projects...
[INFO]                                                                         
[INFO] ------------------------------------------------------------------------
[INFO] Building chronos 1.0-SNAPSHOT
[INFO] ------------------------------------------------------------------------
[INFO] ------------------------------------------------------------------------
[INFO] BUILD FAILURE
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 0.701s
[INFO] Finished at: Mon Mar 25 09:40:52 EDT 2013
[INFO] Final Memory: 2M/15M
[INFO] ------------------------------------------------------------------------
[ERROR] Unknown lifecycle phase "-". You must specify a valid lifecycle phase or a goal in the format <plugin-prefix>:<goal> or <plugin-group-id>:<plugin-artifact-id>[:<plugin-version>]:<goal>. Available lifecycle phases are: validate, initialize, generate-sources, process-sources, generate-resources, process-resources, compile, process-classes, generate-test-sources, process-test-sources, generate-test-resources, process-test-resources, test-compile, process-test-classes, test, prepare-package, package, pre-integration-test, integration-test, post-integration-test, verify, install, deploy, pre-site, site, post-site, site-deploy, pre-clean, clean, post-clean. -> [Help 1]
[ERROR] 
[ERROR] To see the full stack trace of the errors, re-run Maven with the -e switch.
[ERROR] Re-run Maven using the -X switch to enable full debug logging.
[ERROR] 
[ERROR] For more information about the errors and possible solutions, please read the following articles:
[ERROR] [Help 1] http://cwiki.apache.org/confluence/display/MAVEN/LifecyclePhaseNotFoundException

cap deploy added & mvn package error

Hi guys,

I forked the chronos repo and added a Capistrano deploy sequence to make it easier to debug EC2 installation, I followed the steps in your "install for amazon" section of the readme with the 1.7 openjdk.

Getting errors on the mvn package step, full logs in the below links, they are similar for both distros.

Logfiles:
https://github.com/c4ssio/chronos/blob/master/err/install_fedora.err
https://github.com/c4ssio/chronos/blob/master/err/install_debian.err

https://github.com/c4ssio/chronos/blob/master/err/mvn_x_fedora.err
https://github.com/c4ssio/chronos/blob/master/err/mvn_x_debian.err
(these 2 are similar)

echo $JAVA_HOME:
/usr/lib/jvm/java-1.7.0-openjdk-1.7.0.25.x86_64/ (Fedora)
/usr/lib/jvm/java-1.7.0-openjdk-amd64/ (Debian)

lmk if you've seen this before, or if there is another revision of chronos I should be using -- I'm on master branch.

Here's what I did:

clone c4ssio/chronos.git
set ENV variables:
export EC2_PRIVATE_KEY=/path/to/pk
export EC2_FEDORA_ROOT=ec2-user
export EC2_FEDORA_DNS=
export EC2_DEBIAN_ROOT=ubuntu
export EC2_DEBIAN_DNS=
cd chronos
cap <fedora/debian> install:all

This runs through the steps described in your docs for installing on an Amazon instance. I amended the ubuntu packages to include build-essential and replace the default-jdk w openjdk-7.

I started two Large EC2 instances, 1 with RHEL 6.4 and 1 with Ubuntu 12.04.

mvn package failed on test com.airbnb.scheduler.jobs.JobSchedulerIntegrationTest

I tried just following the instruction in the README, but mvn package failed on test phase. Any idea?

$ lsb_release -a
No LSB modules are available.
Distributor ID: Ubuntu
Description:    Ubuntu 12.04.2 LTS
Release:    12.04
Codename:   precise

$ mvn package -X
Apache Maven 3.0.4
Maven home: /usr/share/maven
Java version: 1.7.0_25, vendor: Oracle Corporation
Java home: /usr/lib/jvm/java-7-openjdk-amd64/jre
Default locale: en_US, platform encoding: UTF-8
OS name: "linux", version: "3.2.0-40-virtual", arch: "amd64", family: "unix"

(snip)

Tests run: 3, Failures: 1, Errors: 0, Skipped: 0, Time elapsed: 0.003 sec <<< FAILURE!
JobScheduler should::A job creates a failed task and then a successful task from a synchronous job(com.airbnb.scheduler.jobs.JobSchedulerIntegrationTest)  Time elapsed: 0.001 sec  <<< FAILURE!
org.specs2.reporter.JUnitReporter$$anon$1: '0' is not equal to '1' expected:<[1]> but was:<[0]>
    at com.airbnb.scheduler.jobs.JobSchedulerIntegrationTest$$anonfun$1$$anonfun$apply$19.apply(JobSchedulerIntegrationTest.scala:31)
    at com.airbnb.scheduler.jobs.JobSchedulerIntegrationTest$$anonfun$1$$anonfun$apply$19.apply(JobSchedulerIntegrationTest.scala:14)


Results :

Failed tests:
   '0' is not equal to '1' expected:<[1]> but was:<[0]>

Tests run: 36, Failures: 1, Errors: 0, Skipped: 0

[INFO] ------------------------------------------------------------------------
[INFO] BUILD FAILURE
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 24.246s
[INFO] Finished at: Tue Aug 06 03:00:15 UTC 2013
[INFO] Final Memory: 19M/314M
[INFO] ------------------------------------------------------------------------
[ERROR] Failed to execute goal org.apache.maven.plugins:maven-surefire-plugin:2.14.1:test (default-test) on project chronos: There are test failures.
[ERROR]
[ERROR] Please refer to /home/ubuntu/chronos/target/surefire-reports for the individual test results.
[ERROR] -> [Help 1]

Other resources than disk,memory,cpu

I would like to run jobs on specific nodes. To achieve this it would be nice to add resources (like libraries, db-connection,...) to a node. That would make the job run only on those nodes having the necessary resources.

Benoît

Jobs which failed after being force run shouldn't be disabled.

I recently addad a patch here which may disable jobs after N failures:

5a8cdb9

However, we shouldn't disable jobs that fail after being force run.

Some questions and the webUI looks flaky

I am exploring chronos and the webUI looks a little flaky to me. Not sure if its due to some config (I am using local_scheduler.yml for now).

"destroy" a job does not work. (see screenshot, I tried to destroy it, but its not removed). Tried running it with a local zk instance too. Still does not work.
Icons are not showing up properly
I also tried curl -L -X DELETE localhost:8080/scheduler/jobs but it gives a 405. So how can I enable the full functionality of the webUI?

Also, I have a question about how to use chronos.

Say I need to trigger a bunch of jobs (and handle their retries if they fail etc). How can I get started with setting up the "clients/agents?" on those machines?
What configuration is recommended if I need to run just 1 node for chronos and it will manage jobs on few machines (<10).

Thanks,
-Utkarsh

On Chrome, shrinking the window artifacts the results_header div, resulting in overflow.

Here's an image.

No such issue exists on Safari. This is probably not worth fixing, but I thought I'd add it for completion.

Chronos set datetime correctly when you launch

Hello,

When chronos start, it doesn't use the correct timezone
;

so mesos and zookeeper are the correct time.

Can you tell me how I can change the timezone ?

Best regards.

I already have zookeeper running but chronos still does not work

I have installed chronos and zookeeper running in my server but chronos still does not keep the jobs.
Thanks

2197 1 0 14:18 pts/0 00:00:03 java -Dzookeeper.log.dir=. -Dzookeeper.root.logger=INFO,CONSOLE -cp /home/rbernal/mesos/third_party/zookeeper-3.3.4/bin/../build/classes:/home/rbernal/mesos/third_party/zookeeper-3.3.4/bin/../build/lib/.jar:/home/rbernal/mesos/third_party/zookeeper-3.3.4/bin/../zookeeper-3.3.4.jar:/home/rbernal/mesos/third_party/zookeeper-3.3.4/bin/../lib/log4j-1.2.15.jar:/home/rbernal/mesos/third_party/zookeeper-3.3.4/bin/../lib/jline-0.9.94.jar:/home/rbernal/mesos/third_party/zookeeper-3.3.4/bin/../lib/commons-lang-2.4.jar:/home/rbernal/mesos/third_party/zookeeper-3.3.4/bin/../lib/commons-collections-3.2.jar:/home/rbernal/mesos/third_party/zookeeper-3.3.4/bin/../lib/commons-cli-1.1.jar:/home/rbernal/mesos/third_party/zookeeper-3.3.4/bin/../lib/apache-rat-tasks-0.6.jar:/home/rbernal/mesos/third_party/zookeeper-3.3.4/bin/../lib/apache-rat-core-0.6.jar:/home/rbernal/mesos/third_party/zookeeper-3.3.4/bin/../src/java/lib/.jar:/etc/zookeeper: -Dcom.sun.management.jmxremote -Dcom.sun.management.jmxremote.local.only=false org.apache.zookeeper.server.quorum.QuorumPeerMain /etc/zookeeper/zoo.cfg
54322 3116 2464 0 15:28 pts/1 00:00:00 grep --color=auto -i zookeeper

Failed to execute goal org.apache.maven.plugins:maven-surefire-plugin:2.13:test (default-test) on project chronos: There are test failures

trying to install chronos, running into a mvn build failure

mvn version:

))=3 mvn -version
Apache Maven 3.0.5 (r01de14724cdef164cd33c7c8c2fe155faf9602da; 2013-02-19 07:51:28-0600)
Maven home: /.../apache-maven-3.0.5
Java version: 1.7.0_11, vendor: Oracle Corporation
Java home: /auto/tadev/tp64/java/jdk1.7.0_11/jre
Default locale: en_US, platform encoding: ANSI_X3.4-1968
OS name: "linux", version: "2.6.18-308.4.1.el5", arch: "amd64", family: "unix"

....
WARNING: Task of job: FOO failed.
Apr 05, 2013 1:09:09 PM com.airbnb.scheduler.jobs.JobScheduler handleFailedTask
WARNING: Retrying job: FOO, attempt: 0
Tests run: 2, Failures: 1, Errors: 0, Skipped: 0, Time elapsed: 0.071 sec <<< FAILURE!
JobScheduler should::A job creates a failed task and then a successful task from a synchronous job(com.airbnb.scheduler.jobs.JobSchedulerIntegrationTest) Time elapsed: 0.033 sec <<< FAILURE!
org.specs2.reporter.JUnitReporter$$anon$1: '0' is not equal to '1' expected:<[1]> but was:<[0]>
at com.airbnb.scheduler.jobs.JobSchedulerIntegrationTest$$anonfun$1$$anonfun$apply$11.apply(JobSchedulerIntegrationTest.scala:31)
at com.airbnb.scheduler.jobs.JobSchedulerIntegrationTest$$anonfun$1$$anonfun$apply$11.apply(JobSchedulerIntegrationTest.scala:14)

Running com.airbnb.scheduler.state.PersistenceStoreSpec
Apr 05, 2013 1:09:09 PM com.airbnb.scheduler.state.MesosStatePersistenceStore persistJob
INFO: Persisting job 'sample-name' with data 'ScheduleBasedJob(R1/2012-01-01T00:00:01.000Z/PT1M,sample-name,sample-command,PT1H,1,0,fooexecutor,args,2,,,,false)'
Apr 05, 2013 1:09:09 PM com.airbnb.scheduler.state.MesosStatePersistenceStore persistData
INFO: State J_sample-name does not exist yet. Adding to state
Apr 05, 2013 1:09:09 PM com.airbnb.scheduler.state.MesosStatePersistenceStore persistData
INFO: State update successful: true
Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.024 sec
Running com.airbnb.scheduler.api.SerDeTest
Tests run: 2, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.031 sec

Results :

Failed tests:
'0' is not equal to '1' expected:<[1]> but was:<[0]>

Tests run: 35, Failures: 1, Errors: 0, Skipped: 0

Problems Installing on Mac OS 10.8.4

While installing using installer.bash received (full trace not shown): https://gist.github.com/msukmanowsky/8be86ee881498f7e3acd

$ java -version
java version "1.7.0_25"
Java(TM) SE Runtime Environment (build 1.7.0_25-b15)
Java HotSpot(TM) 64-Bit Server VM (build 23.25-b01, mixed mode)
$ javac -version
javac 1.7.0_25

Any ideas?

ERROR org.apache.zookeeper.server.NIOServerCnxn: Thread Thread[main,5,main] died

When start chronos:

INFO [2013-05-06 13:52:00,545] org.apache.zookeeper.ClientCnxn: Opening socket connection to server localhost/127.0.0.1:28281
INFO [2013-05-06 13:52:00,733] org.apache.zookeeper.server.NIOServerCnxn: Accepted socket connection from /127.0.0.1:55408
INFO [2013-05-06 13:52:00,741] org.apache.zookeeper.ClientCnxn: Socket connection established to localhost/127.0.0.1:28281, initiating session
INFO [2013-05-06 13:52:00,772] org.apache.zookeeper.server.NIOServerCnxn: Client attempting to establish new session at /127.0.0.1:55408
INFO [2013-05-06 13:52:00,774] org.apache.zookeeper.server.persistence.FileTxnLog: Creating new log file: log.1
INFO [2013-05-06 13:52:00,885] org.apache.zookeeper.ClientCnxn: Session establishment complete on server localhost/127.0.0.1:28281, sessionid = 0x13e7a1cfc8e0000, negotiated timeout = 20000
INFO [2013-05-06 13:52:00,890] org.apache.zookeeper.server.NIOServerCnxn: Established session 0x13e7a1cfc8e0000 with negotiated timeout 20000 for client /127.0.0.1:55408
Failed to load native Mesos library at /home/rbernal/airbnb-chronos-8ad3f69/lib/org/apache/mesos/mesos/
ERROR [2013-05-06 13:52:01,272] org.apache.zookeeper.server.NIOServerCnxn: Thread Thread[main,5,main] died
! java.lang.UnsatisfiedLinkError: /home/rbernal/airbnb-chronos-8ad3f69/lib/org/apache/mesos/mesos: /home/rbernal/airbnb-chronos-8ad3f69/lib/org/apache/mesos/mesos: cannot read file data: Is a directory

When I trying to connect:

INFO [2013-05-06 13:54:50,335] org.apache.zookeeper.server.NIOServerCnxn: Accepted socket connection from /172.16.1.222:3098
WARN [2013-05-06 13:54:50,339] org.apache.zookeeper.server.NIOServerCnxn: Exception causing close of session 0x0 due to java.io.IOException: Len error 1195725856
INFO [2013-05-06 13:54:50,340] org.apache.zookeeper.server.NIOServerCnxn: Closed socket connection for client /172.16.1.222:3098 (no session established for client)

Info:

rbernal@ubuntu1204:~$ lsb_release -a
Distributor ID: Ubuntu
Description: Ubuntu 12.04 LTS
Release: 12.04

rbernal@ubuntu1204:~$ java -version
java version "1.7.0_17"
Java(TM) SE Runtime Environment (build 1.7.0_17-b02)
Java HotSpot(TM) 64-Bit Server VM (build 23.7-b01, mixed mode)

rbernal@ubuntu1204:~$ mvn -version
Apache Maven 3.0.5 (r01de14724cdef164cd33c7c8c2fe155faf9602da; 2013-02-19 11:51:28-0200)
Maven home: /opt/apache-maven-3.0.5
Java version: 1.7.0_17, vendor: Oracle Corporation
Java home: /usr/lib/jvm/jdk1.7.0/jre
Default locale: en_US, platform encoding: UTF-8
OS name: "linux", version: "3.2.0-26-virtual", arch: "amd64", family: "unix"

Disabled jobs need UI widget

When a job is disabled in the chronos UI, there needs to be some sort of UI hint that the job is disabled. It's non-obvious at a glance which jobs are disabled.

job submitted from the CLI are stored with the wrong timestamp

in the same minute:

$ date -u
Thu May 2 23:23:38 UTC 2013

$ curl -L -H 'Content-Type: application/json' -X POST -H 'Content-Type: application/json' -d '{"schedule":"R/2013-05-02T23:23:30Z/PT24H","name":"testjob24","command":"ls -aRtl /tmp >> /tmp/ben/f","epsilon":"PT15M","owner":"[email protected]","async":false}' localhost:4400/scheduler/iso8601

but the job record is for tomorrow May 03 ...

$ curl -L -X GET localhost:4400/scheduler/jobs[{"schedule":"R/2013-05-03T23:23:30.000Z/PT24H","name":"testjob24","command":"ls -aRtl /tmp >> /tmp/ben/f","epsilon":"PT15M","successCount":0,"errorCount":0,"executor":"","executorFlags":"","retries":2,"owner":"[email protected]","lastSuccess":"","lastError":"","async":false}]

Backing up and restoring Chronos jobs

Hey, I want to make a bunch of feature requests to the project and was wondering if I should just make separate issues and ask for them? I want to add the features myself but wanted to get your inputs before I implemented them.

I'm looking for the proper way to backup and restore Chronos jobs. Right now we use a script I wrote https://github.com/jeid64/chronos_backup that gets the jobs from the API then then restores them later. I was wondering if we should keep doing this or look into backing up Zookeeper data.

scala-maven-plugin:3.1.0 issues

I am having issues installing Chronos.
I believe they're related to scala-maven-plugin:3.1.0.
My environment:

-Running CentOS release 6.4 (Final)

root@eucaclc chronos-master]# /usr/bin/java -version
java version "1.7.0_19"
OpenJDK Runtime Environment (rhel-2.3.9.1.el6_4-x86_64)
OpenJDK 64-Bit Server VM (build 23.7-b01, mixed mode)
[root@eucaclc chronos-master]# mvn -version
Apache Maven 3.0.5 (r01de14724cdef164cd33c7c8c2fe155faf9602da; 2013-02-19 08:51:28-0500)
Maven home: /usr/local/maven
Java version: 1.7.0_19, vendor: Oracle Corporation
Java home: /usr/lib/jvm/java-1.7.0-openjdk-1.7.0.19.x86_64/jre
Default locale: en_US, platform encoding: UTF-8
OS name: "linux", version: "2.6.32-358.6.1.el6.x86_64", arch: "amd64", family: "unix"

Getting the following error:

[INFO] --- scala-maven-plugin:3.1.0:compile (scala-compile-first) @ chronos ---
[INFO] /home/chronos/chronos-master/src/main/java:-1: info: compiling
[INFO] /home/chronos/chronos-master/src/main/scala:-1: info: compiling
[INFO] Compiling 36 source files to /home/chronos/chronos-master/target/classes at 1368735434011
[WARNING] /home/chronos/chronos-master/src/main/scala/com/airbnb/notification/MailClient.scala:53: warning: non variable type-argument String in type pattern Option[String] is unchecked since it is eliminated by erasure
[WARNING] case (to : String, subject : String, message : Option[String]) => {
[WARNING] ^
[ERROR] /home/chronos/chronos-master/src/main/scala/com/airbnb/scheduler/config/MainModule.scala:35: error: value setFailoverTimeout is not a member of org.apache.mesos.Protos.FrameworkInfo.Builder
[ERROR] possible cause: maybe a semicolon is missing before `value setFailoverTimeout'?
[ERROR] .setFailoverTimeout(config.failoverTimeoutSeconds).setUser(config.user).build()
[ERROR] ^
[ERROR] /home/chronos/chronos-master/src/main/scala/com/airbnb/scheduler/config/ZookeeperModule.scala:14: error: object state is not a member of package org.apache.mesos
[ERROR] import org.apache.mesos.state.ZooKeeperState
[ERROR] ^
[ERROR] /home/chronos/chronos-master/src/main/scala/com/airbnb/scheduler/state/MesosStatePersistenceStore.scala:11: error: object state is not a member of package org.apache.mesos
[ERROR] import org.apache.mesos.state.{InMemoryState, State}
[ERROR] ^
[ERROR] /home/chronos/chronos-master/src/main/scala/com/airbnb/scheduler/state/Migration.scala:12: error: object state is not a member of package org.apache.mesos
[ERROR] import org.apache.mesos.state.ZooKeeperState
[ERROR] ^
[WARNING] one warning found
[ERROR] four errors found
[INFO] ------------------------------------------------------------------------
[INFO] BUILD FAILURE
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 11.872s
[INFO] Finished at: Thu May 16 16:17:22 EDT 2013
[INFO] Final Memory: 17M/281M
[INFO] ------------------------------------------------------------------------
[ERROR] Failed to execute goal net.alchim31.maven:scala-maven-plugin:3.1.0:compile (scala-compile-first) on project chronos: wrap: org.apache.commons.exec.ExecuteException: Process exited with an error: 1(Exit value: 1) -> [Help 1]
[ERROR]
[ERROR] To see the full stack trace of the errors, re-run Maven with the -e switch.
[ERROR] Re-run Maven using the -X switch to enable full debug logging.
[ERROR]
[ERROR] For more information about the errors and possible solutions, please read the following articles:
[ERROR] [Help 1] http://cwiki.apache.org/confluence/display/MAVEN/MojoExecutionException

Success don't appear to be incremented

Os : Mountain Lion 10.8.3
Installer Method : bin/installer.sh

Issue : Success counts don't appear to be incremented in the ui.

Steps :

I have a file named /opt/mesos/makit.sh with perms 777 and content

#!/bin/bash
mkdir /opt/mesos/foo

I force this job to be run manually.

This script works as indicated in the below logs but the success count is not incremented as shown in the screenshot

c-d53c802d5a95
I0323 16:57:09.055505 2670112768 slave.cpp:762] Got registration for executor 'Task ct:Mkit:1364072224796:0 (/opt/mesos/m...)' of framework chronos
I0323 16:57:09.056035 2670112768 slave.cpp:820] Flushing queued tasks for framework chronos
Registered executor on valhall.local
Starting task ct:Mkit:1364072224796:0
sh -c '/opt/mesos/makit.sh'
I0323 16:57:09.064097 2638692352 slave.cpp:842] Status update: task ct:Mkit:1364072224796:0 of framework chronos is now in state TASK_RUNNING
I0323 16:57:09.064705 2671722496 master.cpp:1022] Status update from slave(1)@172.16.90.1:57512: task ct:Mkit:1364072224796:0 of framework chronos is now in state TASK_RUNNING
I0323 16:57:09.066112 2673332224 slave.cpp:739] Got acknowledgement of status update for task ct:Mkit:1364072224796:0 of framework chronos
WARN [2013-03-23 20:57:09,065] com.airbnb.scheduler.mesos.MesosJobFramework: Task with id 'ct:Mkit:1364072224796:0' RUNNING.
Waited on process 89018, returned status 0
I0323 16:57:09.074038 2638692352 slave.cpp:842] Status update: task ct:Mkit:1364072224796:0 of framework chronos is now in state TASK_FINISHED
I0323 16:57:09.074358 2673332224 master.cpp:1022] Status update from slave(1)@172.16.90.1:57512: task ct:Mkit:1364072224796:0 of framework chronos is now in state TASK_FINISHED
I0323 16:57:09.074504 2673332224 master.hpp:296] Removing task with resources cpus=1; mem=1; disk=1 on slave 201303231527-22679724-57512-86332-0
I0323 16:57:09.074784 2671185920 hierarchical_allocator_process.hpp:542] Recovered cpus=1; mem=1; disk=1 (total allocatable: cpus=4; mem=1020; ports=[31000-32000]; disk=138748) on slave 201303231527-22679724-57512-86332-0 from framework chronos
INFO [2013-03-23 20:57:09,074] com.airbnb.scheduler.mesos.MesosJobFramework: Task with id 'ct:Mkit:1364072224796:0' FINISHED
INFO [2013-03-23 20:57:09,075] com.airbnb.scheduler.mesos.MesosJobFramework: Asynchronous task: 'ct:Mkit:1364072224796:0', not updating job-graph.
I0323 16:57:09.075306 2638692352 slave.cpp:739] Got acknowledgement of status update for task ct:Mkit:1364072224796:0 of framework chronos
0:0:0:0:0:0:0:1 - - [23/Mar/2013:20:57:09 +0000] "GET /scheduler/graph/csv HTTP/1.1" 200 69 4 4
0:0:0:0:0:0:0:1 - - [23/Mar/2013:20:57:09 +0000] "GET /scheduler/jobs HTTP/1.1" 200 331 4 4

Chronos redirects to localhost:4400 instead of leader node when multiple nodes are in use

Hey! Thanks for Chronos, it's pretty awesome software. I'm running into a small problem at the moment. I have multiple mesos masters all using zookeeper and I have Chronos setup to use Zookeeper to choose a Chronos leader node. I'm attempting to run Chronos on multiple nodes and they all seem to elect a leader and wait for that leader to become unavailable, but when I access a standby node at port 4400, I get redirected to localhost:4400.

I've attached a gist containing my Chronos config and the output from the start of a Chronos standby node. It looks like something in Zookeeper might be causing this. INFO [2013-05-30 14:40:36,013] com.airbnb.scheduler.config.ZookeeperModule: Using hostname:localhost

https://gist.github.com/jeid64/5678377

Last Success Not Updated After Job seems to repair itself

For kicks, I entered an echo "hi bob" job into the command prompt and scheduled it to run every minute with a 15 minute epsilon for retries. The first trial failed because I forgot a " at the end of my echo. After fixing this and saving the job, it ran again about a minute later.

The last success attribute in the job view never updated, and the 'Last' status column in the job list still says 'Failed' next to the job. Is there something obvious being missed here?

Debugging Info:

Command to run:

/usr/local/java/bin/java -cp /opt/chronos/target/chronos-1.0-SNAPSHOT.jar -XX:+HeapDumpOnOutOfMemoryError -XX:HeapDumpPath=/opt/chronos -server -Xms512m -Xmx512m com.airbnb.scheduler.Main server /opt/chronos/config/scheduler.yml

scheduler.yml file:

scheduleHorizonSeconds: 120
zookeeperTimeoutMs: 10000
master: zk://etl-01.mxstg:2181,etl-02.mxstg:2181,etl-03.mxstg:2181/masters

mailUser: "AWSACCESSKEY"

mailPassword: "AWSSECRET"

mailServer: "email-smtp.us-east-1.amazonaws.com:25"

mailFrom: "[email protected]"

failureRetryDelay: 60000

Amount of memory, in MiB, per Mesos task

mesosTaskMem: 1024

Number of CPUs per Mesos task

mesosTaskCpu: 1.0

Amount of disk space, in MiB, required per Mesos task

mesosTaskDisk: 1024

http:
rootPath: /scheduler/*
adminPort: 4401
port: 4400

logging:

The default level of all loggers. Can be OFF, ERROR, WARN, INFO, DEBUG, TRACE, or ALL.

level: WARN

loggers:

"com.airbnb.scheduler": TRACE

file:

  # If true, write log statements to a file.
  enabled: true

  # Do not write log statements below this threshold to the file.
  threshold: ALL

  # The file to which current statements will be logged.
  currentLogFilename: /tmp/chronos-1.log

  # When the log file rotates, the archived log will be renamed to this and gzipped. The
  # %d is replaced with the previous day (yyyy-MM-dd). Custom rolling windows can be created
  # by passing a SimpleDateFormat-compatible format as an argument: "%d{yyyy-MM-dd-hh}".
  archivedLogFilenamePattern: /tmp/example-%d.log.gz

  # The number of archived files to keep.
  archivedFileCount: 10

  # The timezone used to format dates. HINT: USE THE DEFAULT, UTC.
  timeZone: UTC

-----END FILE-----

Mesos version: 0.12.0

Java -version output:

java version "1.7.0_21"
Java(TM) SE Runtime Environment (build 1.7.0_21-b11)
Java HotSpot(TM) 64-Bit Server VM (build 23.21-b01, mixed mode)

install script dependencies need to be documented

The installer.bash build script requires that libcurl, and python_dev be installed, as well as JAVA_HOME being set. This should be documented.

Also, if the install fails, mesos is recloned from Git. This is inefficient.

Launch chronos : java.net.ConnectException: Connection refused !!

Hello,

I have this problem when i launch chronos :

WARN [2013-06-06 15:24:59,879] com.google.common.cache.CacheBuilder: ignoring weigher specified without maximumWeight
INFO [2013-06-06 15:24:59,930] com.airbnb.scheduler.Main$: ---------------------
INFO [2013-06-06 15:24:59,930] com.airbnb.scheduler.Main$: Initializing chronos.
INFO [2013-06-06 15:24:59,930] com.airbnb.scheduler.Main$: ---------------------
INFO [2013-06-06 15:25:00,000] com.airbnb.scheduler.config.MainModule: Wiring up the application
INFO [2013-06-06 15:25:00,243] com.airbnb.scheduler.config.ZookeeperModule: localhost:2181
INFO [2013-06-06 15:25:00,245] com.airbnb.scheduler.config.ZookeeperModule: Using external zookeeper.
INFO [2013-06-06 15:25:00,272] com.airbnb.scheduler.config.ZookeeperModule: localhost:2181
INFO [2013-06-06 15:25:00,372] com.airbnb.scheduler.config.ZookeeperModule: Providing MesosStatePersistenceStore
WARN [2013-06-06 15:25:00,488] org.apache.zookeeper.ClientCnxn: Session 0x0 for server null, unexpected error, closing socket connection and attempting reconnect
! java.net.ConnectException: Connection refused
! at sun.nio.ch.SocketChannelImpl.checkConnect(Native Method) ~[na:1.6.0_27]
! at sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:592) ~[na:1.6.0_27]
! at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1146) ~[chronos-1.0-SNAPSHOT.jar:na]

I use the sample sample_scheduler.yml .

Thanks.

Ubuntu
chronos@CHRONOS:/chronos/chronos-master/config$ uname -a
Linux CHRONOS 3.5.0-32-generic #53~precise1-Ubuntu SMP Wed May 29 20:33:37 UTC 2013 x86_64 x86_64 x86_64 GNU/Linux

Java
chronos@CHRONOS:/chronos/chronos-master/config$ java -version
java version "1.6.0_27"
OpenJDK Runtime Environment (IcedTea6 1.12.5) (6b27-1.12.5-0ubuntu0.12.04.1)
OpenJDK 64-Bit Server VM (build 20.0-b12, mixed mode)

Thks.

Exception thrown when submitting a shell based job

I'm running the latest version of Chronos from master and v0.12.0 of Mesos on the following configuration:

Centos 6.2 x86_64
Linode 1GB RAM instance
- Xen VM

I am using the local_scheduler_nozk.yml Chronos config. I realize that this will prevent my jobs from persisting if Chronos is restarted but I am just trying to get this setup before I move on to a full install of Zookeeper.

I built Mesos v0.12.0 separately as the build process via the Chronos installer would fail. Mesos evidently requires a specific version of libunwind for 64-bit boxes.

In any case I have a Mesos master/slave running with Chronos registered against the master Mesos as a valid framework user. Whenever I submit a task via the Chronos UI or via the RESTful API I get an HTTP status code of 500 with this in the logs:

WARN  [2013-07-21 23:15:13,111] com.airbnb.scheduler.api.Iso8601JobResource: Exception while serving request
! java.lang.UnsatisfiedLinkError: org.apache.mesos.state.ZooKeeperState.__fetch(Ljava/lang/String;)J
! at org.apache.mesos.state.ZooKeeperState.__fetch(Native Method) ~[chronos-1.0-SNAPSHOT.jar:na]
! at org.apache.mesos.state.ZooKeeperState.fetch(ZooKeeperState.java:77) ~[chronos-1.0-SNAPSHOT.jar:na]
! at com.airbnb.scheduler.state.MesosStatePersistenceStore.persistData(MesosStatePersistenceStore.scala:143) ~[chronos-1.0-SNAPSHOT.jar:na]
! at com.airbnb.scheduler.state.MesosStatePersistenceStore.persistJob(MesosStatePersistenceStore.scala:64) ~[chronos-1.0-SNAPSHOT.jar:na]
! at com.airbnb.scheduler.jobs.JobScheduler$$anonfun$registerJob$3.apply(JobScheduler.scala:172) ~[chronos-1.0-SNAPSHOT.jar:na]
! at com.airbnb.scheduler.jobs.JobScheduler$$anonfun$registerJob$3.apply(JobScheduler.scala:168) ~[chronos-1.0-SNAPSHOT.jar:na]
...

Any idea why this is happening? I'm wondering if this has anything to do with how libmesos.so was linked against that version of libunwind that I used.

Asynchronous job updates should return a failure code if the taskid is wrong

Hey, currently in TaskManagementResource.scala there is no check if an id is right or wrong. So if you send anything to /scheduler/job/id, as long as it has a statuscode of 0 or 1, you'll always get back a 204 response.

Build Failure on Ubuntu 12.04 64bit

I tried to install the chronos on Ubuntu 12.04 which has been updated.
I searched on Google but the result doesn't help. So I just wonder if there is a bug or I have done something wrong.

Here is part of the output

Tests run: 2, Failures: 1, Errors: 0, Skipped: 0, Time elapsed: 0.128 sec <<< FAILURE!
JobScheduler should::A job creates a failed task and then a successful task from a synchronous job(com.airbnb.scheduler.jobs.JobSchedulerIntegrationTest)  Time elapsed: 0.039 sec  <<< FAILURE!
org.specs2.reporter.JUnitReporter$$anon$1: '0' is not equal to '1' expected:<[1]> but was:<[0]>
    at com.airbnb.scheduler.jobs.JobSchedulerIntegrationTest$$anonfun$1$$anonfun$apply$11.apply(JobSchedulerIntegrationTest.scala:31)
    at com.airbnb.scheduler.jobs.JobSchedulerIntegrationTest$$anonfun$1$$anonfun$apply$11.apply(JobSchedulerIntegrationTest.scala:14)

Running com.airbnb.scheduler.jobs.Iso8601ExpressionParserSpec
Tests run: 3, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.04 sec
Running com.airbnb.scheduler.state.PersistenceStoreSpec
Apr 23, 2013 9:45:15 PM com.airbnb.scheduler.state.MesosStatePersistenceStore persistJob
INFO: Persisting job 'sample-name' with data 'ScheduleBasedJob(R1/2012-01-01T00:00:01.000Z/PT1M,sample-name,sample-command,PT1H,1,0,fooexecutor,args,2,,,,false)'
Apr 23, 2013 9:45:15 PM com.airbnb.scheduler.state.MesosStatePersistenceStore persistData
INFO: State J_sample-name does not exist yet. Adding to state
Apr 23, 2013 9:45:15 PM com.airbnb.scheduler.state.MesosStatePersistenceStore persistData
INFO: State update successful: true
Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.039 sec
Running com.airbnb.scheduler.api.SerDeTest
Tests run: 2, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.052 sec

Results :

Failed tests: 
   '0' is not equal to '1' expected:<[1]> but was:<[0]>

Tests run: 35, Failures: 1, Errors: 0, Skipped: 0

[INFO] ------------------------------------------------------------------------
[INFO] BUILD FAILURE
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 21.220s
[INFO] Finished at: Tue Apr 23 21:45:15 PDT 2013
[INFO] Final Memory: 17M/66M
[INFO] ------------------------------------------------------------------------
[ERROR] Failed to execute goal org.apache.maven.plugins:maven-surefire-plugin:2.13:test (default-test) on project chronos: There are test failures.
[ERROR] 
[ERROR] Please refer to /usr/local/chronos/target/surefire-reports for the individual test results.
[ERROR] -> [Help 1]
[ERROR] 
[ERROR] To see the full stack trace of the errors, re-run Maven with the -e switch.
[ERROR] Re-run Maven using the -X switch to enable full debug logging.
[ERROR] 
[ERROR] For more information about the errors and possible solutions, please read the following articles:
[ERROR] [Help 1] http://cwiki.apache.org/confluence/display/MAVEN/MojoFailureException

I found this post #21 and tried to run mvn clean and then mvn package, but still failed when I run mvn package

the maven and java I used is as following:

# java -version
java version "1.7.0_15"
OpenJDK Runtime Environment (IcedTea7 2.3.7) (7u15-2.3.7-0ubuntu1~12.04.1)
OpenJDK 64-Bit Server VM (build 23.7-b01, mixed mode)
# javac -version
javac 1.7.0_15
# mvn -version
Apache Maven 3.0.4
Maven home: /usr/share/maven
Java version: 1.7.0_15, vendor: Oracle Corporation
Java home: /usr/lib/jvm/java-7-openjdk-amd64/jre
Default locale: en_US, platform encoding: UTF-8
OS name: "linux", version: "3.5.0-27-generic", arch: "amd64", family: "unix"

Thank you!

Failure to find org.apache.mesos:mesos:jar:0.0.14-SNAPSHOT in twitter-repo (http://maven.twttr.com/)

How can we solve that?
Changing to version 0.12.0-incubating seems to work.

Mesos install retry issue

If mesos fails to install for any reason (like missing package), retrying installer.bash will fail to retry the mesos install and result in the following error:

Updating mesos source...
~/mesos/src ~/airbnb-chronos-1e4174f
HEAD is now at 3758114 Added success logging for resource collection, and
~/airbnb-chronos-1e4174f
Installed mesos in: /xyz
~/mesos ~/airbnb-chronos-1e4174f
Could not determine mesos version. Try reinstalling mesos. Aborting.

This can be fixed by removing the mesos folder it populated from git, but that git checkout can take a long time to clone every time.

unable to run job on cluster but ok using local_scheduler_nozk.yml

hello ,
am able to run 1 simple job (simple ls) when i use the provided local_scheduler_nozk.yml but when i switch to a 2 nodes cluster config (1 node with mesos master+chronos+zookeeper and 1 node as a simple mesos slave) the job is registered but never run.
no trace of running in the logs (although using TRACE level) and no output for my job on any of the 2 nodes cluster , the log dont show launching of the job nor TASK_RUNNING or TASK_FAILURE i expected either.
see gist for details about the job 'test7'.
note i see the job in the chronos ui and it does not show either success nor error.
also i made sure to use UTC timestamp ;)
i also see the mesos master and slave in the mesos ui.
/tmp/mesos/slave on the mesos-master/chronos server is empty
the mesos tests at https://github.com/apache/mesos/blob/trunk/docs/Home.md all pass (java/python/c++).
tried to submit from both chronos web ui and the chronos/curl cli to no avail.

please see https://gist.github.com/bentrigat/5503859

fontawesome loading is broken

Get this error when I load chronos:
Failed to load resource: the server responded with a status of 404 (Not Found) http:/host:8080/scripts/styles/vendor/fontawesome//fonts/vendor/fontawesome/fontawesome-webfont.woff?v=3.0.1

How can I fix this? Due to my lack of expertise of fancy javascript frameworks chronos uses, I am not able to figure out where the invalid url is actually being constructed.

My guess is somewhere here: app.build.js or package.json

If you can point me in the right direction, I can submit a pull request.

Chronos's assets won't optimize if server has low RAM.

Hey, I didn't really know if this was an issue or not but it would be nice if it was stated somewhere. I was getting Stackoverflow errors on my VM at the optimization part of the installer.sh script on my VM that had a gig of RAM. I was wondering why my Chronos compile seemed to work besides that, but I wouldn't get anything on the webpage. 0 assets would be put into the JAR with low RAM and there isn't a check in the make script to see if the optimization worked or not. Also, I was slowly stepping up the RAM on the VM and slowly more and more assets would be put into the JAR, so it depends on the RAM amount how much succeeds.

Mesos installer script fails on Mac OSX 10.8

Two items:

The github address for mesos needs to be updated to incubator-mesos
After doing that, the following fails:

Switched to a new branch 'snapshot'
Building mesos source...
~/src ~/GitHub/chronos
./bootstrap: line 16: autoreconf: command not found

This can be resolved with installation of more tools, but it is probably best to have separate install sections for linux vs mac to help guide people through this type of stuff.

build failed for chronos

Using Sun Java 7. Mesos is already installed. This is the message I get:
Could not resolve dependencies for project com.airbnb:chronos:jar:1.0-SNAPSHOT: Could not find artifact org.apache.mesos:mesos:jar:0.14.0 in twitter-repo (http://maven.twttr.com/)

fail to compile

Hi,

I am trying to compile using eclipse juno and maven2 plug-in. I'm new to maven, so I may be doing something stupid. Error is below.

Thanks for your help,
James

[ERROR] Failed to execute goal net.alchim31.maven:scala-maven-plugin:3.0.1:compile (scala-compile-first) on project chronos: wrap: org.apache.commons.exec.ExecuteException: Process exited with an error: 1(Exit value: 1) -> [Help 1]
org.apache.maven.lifecycle.LifecycleExecutionException: Failed to execute goal net.alchim31.maven:scala-maven-plugin:3.0.1:compile (scala-compile-first) on project chronos: wrap: org.apache.commons.exec.ExecuteException: Process exited with an error: 1(Exit value: 1)
at org.apache.maven.lifecycle.internal.MojoExecutor.execute(MojoExecutor.java:217)
at org.apache.maven.lifecycle.internal.MojoExecutor.execute(MojoExecutor.java:153)
at org.apache.maven.lifecycle.internal.MojoExecutor.execute(MojoExecutor.java:145)
at org.apache.maven.lifecycle.internal.LifecycleModuleBuilder.buildProject(LifecycleModuleBuilder.java:84)
at org.apache.maven.lifecycle.internal.LifecycleModuleBuilder.buildProject(LifecycleModuleBuilder.java:59)
at org.apache.maven.lifecycle.internal.LifecycleStarter.singleThreadedBuild(LifecycleStarter.java:183)
at org.apache.maven.lifecycle.internal.LifecycleStarter.execute(LifecycleStarter.java:161)
at org.apache.maven.DefaultMaven.doExecute(DefaultMaven.java:320)
at org.apache.maven.DefaultMaven.execute(DefaultMaven.java:156)
at org.apache.maven.cli.MavenCli.execute(MavenCli.java:537)
at org.apache.maven.cli.MavenCli.doMain(MavenCli.java:196)
at org.apache.maven.cli.MavenCli.main(MavenCli.java:141)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:601)
at org.codehaus.plexus.classworlds.launcher.Launcher.launchEnhanced(Launcher.java:290)
at org.codehaus.plexus.classworlds.launcher.Launcher.launch(Launcher.java:230)
at org.codehaus.plexus.classworlds.launcher.Launcher.mainWithExitCode(Launcher.java:409)
at org.codehaus.plexus.classworlds.launcher.Launcher.main(Launcher.java:352)
Caused by: org.apache.maven.plugin.MojoExecutionException: wrap: org.apache.commons.exec.ExecuteException: Process exited with an error: 1(Exit value: 1)
at scala_maven.ScalaMojoSupport.execute(ScalaMojoSupport.java:350)
at org.apache.maven.plugin.DefaultBuildPluginManager.executeMojo(DefaultBuildPluginManager.java:101)
at org.apache.maven.lifecycle.internal.MojoExecutor.execute(MojoExecutor.java:209)
... 19 more
Caused by: org.apache.commons.exec.ExecuteException: Process exited with an error: 1(Exit value: 1)
at org.apache.commons.exec.DefaultExecutor.executeInternal(DefaultExecutor.java:346)
at org.apache.commons.exec.DefaultExecutor.execute(DefaultExecutor.java:149)
at org.apache.commons.exec.DefaultExecutor.execute(DefaultExecutor.java:136)
at scala_maven_executions.JavaMainCallerByFork.run(JavaMainCallerByFork.java:92)
at scala_maven.ScalaCompilerSupport.compile(ScalaCompilerSupport.java:108)
at scala_maven.ScalaCompilerSupport.doExecute(ScalaCompilerSupport.java:64)
at scala_maven.ScalaMojoSupport.execute(ScalaMojoSupport.java:342)
... 21 more

Installer Aborts While Running Mesos Tests

I ran ./bin/installer.bash on Fedora 18 and it runs successfully until the Mesos test are executed. At that point, there is some sort of a failure and the installer terminates. Here is the output:

make[3]: Leaving directory `/opt/mesos/src/build/src'
make  check-TESTS
make[3]: Entering directory `/opt/mesos/src/build/src'
==================
All 0 tests passed
==================
make[3]: Leaving directory `/opt/mesos/src/build/src'
make[2]: Leaving directory `/opt/mesos/src/build/src'
make[1]: Leaving directory `/opt/mesos/src/build/src'
Making check in ec2
make[1]: Entering directory `/opt/mesos/src/build/ec2'
make[1]: Nothing to be done for `check'.
make[1]: Leaving directory `/opt/mesos/src/build/ec2'
Making check in hadoop
make[1]: Entering directory `/opt/mesos/src/build/hadoop'
make[1]: Nothing to be done for `check'.
make[1]: Leaving directory `/opt/mesos/src/build/hadoop'
Source directory: /opt/mesos/src
Build directory: /opt/mesos/src/build
Note: Google Test filter = -Cgroups*:FsTest*:Proc*
[==========] Running 210 tests from 39 test cases.
[----------] Global test environment set-up.
[----------] 10 tests from MasterTest
[ RUN      ] MasterTest.TaskRunning
[       OK ] MasterTest.TaskRunning (53 ms)
[ RUN      ] MasterTest.ShutdownFrameworkWhileTaskRunning
[       OK ] MasterTest.ShutdownFrameworkWhileTaskRunning (19 ms)
[ RUN      ] MasterTest.KillTask
[       OK ] MasterTest.KillTask (17 ms)
[ RUN      ] MasterTest.StatusUpdateAck
[       OK ] MasterTest.StatusUpdateAck (28 ms)
[ RUN      ] MasterTest.RecoverResources
E0325 12:11:47.706445 11336 slave.cpp:1155] Failed to unwatch executor default of framework 201303251211-16777343-37435-11316-0000: Not watched
[       OK ] MasterTest.RecoverResources (2035 ms)
[ RUN      ] MasterTest.FrameworkMessage
[       OK ] MasterTest.FrameworkMessage (16 ms)
[ RUN      ] MasterTest.MultipleExecutors
[       OK ] MasterTest.MultipleExecutors (18 ms)
[ RUN      ] MasterTest.MasterInfo
[       OK ] MasterTest.MasterInfo (20 ms)
[ RUN      ] MasterTest.MasterInfoOnReElection
[       OK ] MasterTest.MasterInfoOnReElection (8 ms)
[ RUN      ] MasterTest.MasterLost
[       OK ] MasterTest.MasterLost (5 ms)
[----------] 10 tests from MasterTest (2243 ms total)

[----------] 1 test from WhitelistFixture
[ RUN      ] WhitelistFixture.WhitelistSlave
[       OK ] WhitelistFixture.WhitelistSlave (49 ms)
[----------] 1 test from WhitelistFixture (51 ms total)

[----------] 5 tests from FrameworksManagerTestFixture
[ RUN      ] FrameworksManagerTestFixture.AddFramework
[       OK ] FrameworksManagerTestFixture.AddFramework (15 ms)
[ RUN      ] FrameworksManagerTestFixture.RemoveFramework
[       OK ] FrameworksManagerTestFixture.RemoveFramework (20 ms)
[ RUN      ] FrameworksManagerTestFixture.ResurrectFramework
[       OK ] FrameworksManagerTestFixture.ResurrectFramework (2 ms)
[ RUN      ] FrameworksManagerTestFixture.ResurrectExpiringFramework
[       OK ] FrameworksManagerTestFixture.ResurrectExpiringFramework (2 ms)
[ RUN      ] FrameworksManagerTestFixture.ResurrectInterspersedExpiringFrameworks
[       OK ] FrameworksManagerTestFixture.ResurrectInterspersedExpiringFrameworks (11 ms)
[----------] 5 tests from FrameworksManagerTestFixture (56 ms total)

[----------] 1 test from FrameworksManagerTest
[ RUN      ] FrameworksManagerTest.CacheFailure
E0325 12:11:48.929335 11338 frameworks_manager.cpp:216] Error getting framework info from underlying storage: Fake Caching Error
E0325 12:11:48.930222 11336 frameworks_manager.cpp:216] Error getting framework info from underlying storage: Fake Caching Error
[       OK ] FrameworksManagerTest.CacheFailure (10 ms)
[----------] 1 test from FrameworksManagerTest (11 ms total)

[----------] 4 tests from LevelDBStateTest
[ RUN      ] LevelDBStateTest.GetSetGet
[       OK ] LevelDBStateTest.GetSetGet (45 ms)
[ RUN      ] LevelDBStateTest.GetSetSetGet
[       OK ] LevelDBStateTest.GetSetSetGet (73 ms)
[ RUN      ] LevelDBStateTest.GetGetSetSetGet
[       OK ] LevelDBStateTest.GetGetSetSetGet (52 ms)
[ RUN      ] LevelDBStateTest.Names
[       OK ] LevelDBStateTest.Names (21 ms)
[----------] 4 tests from LevelDBStateTest (192 ms total)

[----------] 4 tests from ZooKeeperStateTest
[ RUN      ] ZooKeeperStateTest.GetSetGet
[       OK ] ZooKeeperStateTest.GetSetGet (952 ms)
[ RUN      ] ZooKeeperStateTest.GetSetSetGet
[       OK ] ZooKeeperStateTest.GetSetSetGet (240 ms)
[ RUN      ] ZooKeeperStateTest.GetGetSetSetGet
[       OK ] ZooKeeperStateTest.GetGetSetSetGet (259 ms)
[ RUN      ] ZooKeeperStateTest.Names
[       OK ] ZooKeeperStateTest.Names (188 ms)
[----------] 4 tests from ZooKeeperStateTest (1659 ms total)

[----------] 5 tests from SlaveStateFixture
[ RUN      ] SlaveStateFixture.CreateExecutorDirectory
[       OK ] SlaveStateFixture.CreateExecutorDirectory (1 ms)
[ RUN      ] SlaveStateFixture.format
[       OK ] SlaveStateFixture.format (1 ms)
[ RUN      ] SlaveStateFixture.parse
[       OK ] SlaveStateFixture.parse (2 ms)
[ RUN      ] SlaveStateFixture.CheckpointSlaveID
[       OK ] SlaveStateFixture.CheckpointSlaveID (2 ms)
[ RUN      ] SlaveStateFixture.CheckpointFrameworkPID
[       OK ] SlaveStateFixture.CheckpointFrameworkPID (1 ms)
[----------] 5 tests from SlaveStateFixture (16 ms total)

[----------] 3 tests from GarbageCollectorTest
[ RUN      ] GarbageCollectorTest.Restart
[       OK ] GarbageCollectorTest.Restart (1059 ms)
[ RUN      ] GarbageCollectorTest.ExitedExecutor
E0325 12:11:52.319907 11337 slave.cpp:1155] Failed to unwatch executor default of framework 201303251211-16777343-37435-11316-0000: Not watched
[       OK ] GarbageCollectorTest.ExitedExecutor (1045 ms)
[ RUN      ] GarbageCollectorTest.DiskUsage
E0325 12:11:53.377125 11335 slave.cpp:1155] Failed to unwatch executor default of framework 201303251211-16777343-37435-11316-0000: Not watched
[       OK ] GarbageCollectorTest.DiskUsage (1059 ms)
[----------] 3 tests from GarbageCollectorTest (3167 ms total)

[----------] 8 tests from ResourceOffersTest
[ RUN      ] ResourceOffersTest.ResourceOfferWithMultipleSlaves
[       OK ] ResourceOffersTest.ResourceOfferWithMultipleSlaves (56 ms)
[ RUN      ] ResourceOffersTest.TaskUsesNoResources
[       OK ] ResourceOffersTest.TaskUsesNoResources (10 ms)
[ RUN      ] ResourceOffersTest.TaskUsesInvalidResources
[       OK ] ResourceOffersTest.TaskUsesInvalidResources (22 ms)
[ RUN      ] ResourceOffersTest.TaskUsesMoreResourcesThanOffered
[       OK ] ResourceOffersTest.TaskUsesMoreResourcesThanOffered (13 ms)
[ RUN      ] ResourceOffersTest.ResourcesGetReofferedWhenUnused
[       OK ] ResourceOffersTest.ResourcesGetReofferedWhenUnused (27 ms)
[ RUN      ] ResourceOffersTest.ResourcesGetReofferedAfterTaskInfoError
[       OK ] ResourceOffersTest.ResourcesGetReofferedAfterTaskInfoError (22 ms)
[ RUN      ] ResourceOffersTest.Request
[       OK ] ResourceOffersTest.Request (45 ms)
[ RUN      ] ResourceOffersTest.TasksExecutorInfoDiffers
[       OK ] ResourceOffersTest.TasksExecutorInfoDiffers (22 ms)
[----------] 8 tests from ResourceOffersTest (230 ms total)

[----------] 11 tests from FaultToleranceTest
[ RUN      ] FaultToleranceTest.SlaveLost
[       OK ] FaultToleranceTest.SlaveLost (8 ms)
[ RUN      ] FaultToleranceTest.SlavePartitioned
[       OK ] FaultToleranceTest.SlavePartitioned (27 ms)
[ RUN      ] FaultToleranceTest.SchedulerFailover
[       OK ] FaultToleranceTest.SchedulerFailover (8 ms)
[ RUN      ] FaultToleranceTest.FrameworkReliableRegistration
[       OK ] FaultToleranceTest.FrameworkReliableRegistration (23 ms)
[ RUN      ] FaultToleranceTest.FrameworkReregister
[       OK ] FaultToleranceTest.FrameworkReregister (35 ms)
[ RUN      ] FaultToleranceTest.SchedulerFailoverStatusUpdate
[       OK ] FaultToleranceTest.SchedulerFailoverStatusUpdate (28 ms)
[ RUN      ] FaultToleranceTest.ForwardStatusUpdateUnknownExecutor
[       OK ] FaultToleranceTest.ForwardStatusUpdateUnknownExecutor (40 ms)
[ RUN      ] FaultToleranceTest.SchedulerFailoverFrameworkMessage
pure virtual method called
terminate called without an active exception
/home/admin/Downloads/airbnb-chronos-141eefb/bin/install_mesos.bash: line 33: 11316 Aborted                 (core dumped) ./bin/mesos-tests.sh --gtest_filter=-Cgroups*:FsTest*:Proc*
[root@localhost airbnb-chronos-141eefb]#

Logging output of job running

We're interested in logging the output from our Chronos jobs and sending it to the Chronos master and seeing with the WebUI what's going on with the job. One of the methods we were thinking about would be to PUT to /schedule/task/id with an option parameter called something like statusOutput that could have a string that we could see from the ui. We would then like to be able to look back through a jobs history and see all the updates. Any ideas?

Build Failure

trying to install chronos

...................

Apr 16, 2013 4:14:28 PM com.airbnb.scheduler.jobs.JobScheduler next
INFO: Calling next for stream: R4/2012-01-02T00:00:00.000Z/P1D, jobname: job1
Tests run: 2, Failures: 1, Errors: 0, Skipped: 0, Time elapsed: 0.223 sec <<< FAILURE!
JobScheduler should::A job creates a failed task and then a successful task from a synchronous job(com.airbnb.scheduler.jobs.JobSchedulerIntegrationTest) Time elapsed: 0.214 sec <<< FAILURE!
org.specs2.reporter.JUnitReporter$$anon$1: '0' is not equal to '1' expected:<[1]> but was:<[0]>
at com.airbnb.scheduler.jobs.JobSchedulerIntegrationTest$$anonfun$1$$anonfun$apply$11.apply(JobSchedulerIntegrationTest.scala:31)
at com.airbnb.scheduler.jobs.JobSchedulerIntegrationTest$$anonfun$1$$anonfun$apply$11.apply(JobSchedulerIntegrationTest.scala:14)

Running com.airbnb.scheduler.api.SerDeTest
Tests run: 2, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.003 sec

Results :

Failed tests:
'0' is not equal to '1' expected:<[1]> but was:<[0]>

Tests run: 35, Failures: 1, Errors: 0, Skipped: 0

[INFO] ------------------------------------------------------------------------
[ERROR] BUILD FAILURE
[INFO] ------------------------------------------------------------------------
[INFO] There are test failures.

Please refer to /home/rbernal/airbnb-chronos-8ad3f69/target/surefire-reports for the individual test results.
[INFO] ------------------------------------------------------------------------
[INFO] For more information, run Maven with the -e switch
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 55 minutes 49 seconds
[INFO] Finished at: Tue Apr 16 16:14:35 UYT 2013
[INFO] Final Memory: 89M/241M
[INFO] ------------------------------------------------------------------------

defaultJobOwner not taking effect

I'm configuring the installation of Chronos, and by default when creating a new job, the owner is set to "[email protected]."

According to the docs (https://github.com/airbnb/chronos/tree/master/config#defaultjobowner), we should be able to override this by setting the defaultJobOwner config setting in the yml config file. That does not appear to be working...

The contents of my config file are below:

root@ip-10-1-0-245:/opt/chronos# cat /etc/chronos/config.yml 
mailUser: "[secret]"
mailPassword: "[secret]"
mailServer: "email-smtp.us-east-1.amazonaws.com:25"
mailFrom: "[email protected]"
mailSslOn: true
defaultJobOwner:  "[email protected]"
scheduleHorizonSeconds: 60
master: local

http:

    adminPort: 4401
    port: 4400
    rootPath: "/scheduler/*"

logging:

  # The default level of all loggers. Can be OFF, ERROR, WARN, INFO, DEBUG, TRACE, or ALL.
  level: INFO

How to setup chronos cluster?

I shall begin by saying that this is all new to me, so forgive me in advance if this seems simple and I could not figure it out.

So basically, from my understanding, I have installed Mesos/Chronos/Zookeeper.
I believe everything has been installed under /usr/local on my RedHat Linux platform.

MESOS

/usr/local/mesos/
/usr/local/mesos/build

[root@HOSTA local]# ls -l /usr/local/bin/
total 3908
-rw-r--r-- 1 root root 1229 Jul 16 10:26 mesos-build-env.sh
-rwxr-xr-x 1 root root 1218932 Jul 16 08:57 mesos-local
-rwxr-xr-x 1 root root 1296097 Jul 16 08:57 mesos-log
-rwxr-xr-x 1 root root 1475130 Jul 16 08:57 mesos-mesos

[root@HOSTA local]# ls -l /usr/local/sbin/
total 4872
-rwxr-xr-x 1 root root 320 Jul 16 08:57 mesos-daemon.sh
-rwxr-xr-x 1 root root 3346880 Jul 16 08:57 mesos-master
-rwxr-xr-x 1 root root 1609468 Jul 16 08:57 mesos-slave
-rwxr-xr-x 1 root root 872 Jul 16 08:57 mesos-start-cluster.sh
-rwxr-xr-x 1 root root 848 Jul 16 08:57 mesos-start-masters.sh
-rwxr-xr-x 1 root root 832 Jul 16 08:57 mesos-start-slaves.sh
-rwxr-xr-x 1 root root 570 Jul 16 08:57 mesos-stop-cluster.sh
-rwxr-xr-x 1 root root 682 Jul 16 08:57 mesos-stop-masters.sh
-rwxr-xr-x 1 root root 668 Jul 16 08:57 mesos-stop-slaves.sh

Mesos config file

[HOSTA]$ cat /usr/local/var/mesos/conf/masters
zk=zk://10.1.8.138:2181/mesos
master=10.1.8.138:5050

CHRONOS

chronos home => /usr/local/chronos

chronos config

[root@HOSTA chronos]# cat config/nemo_scheduler.yml
scheduleHorizonSeconds: 60
master: zk://10.1.8.138:2181/mesos
zookeeperServers: 10.1.8.138:2181
zookeeperStateZnode: "/airbnb/service/chronos/state"
zookeeperLeaderZnode: "/airbnb/service/chronos/leader"
zookeeperCandidateZnode: "/airbnb/service/chronos/candidate"
zookeeperTimeoutMs: 5000
user: abc
failureRetryDelay: 60000
mesosTaskMem: 1024
mesosTaskCpu: 1.0
mesosTaskDisk: 1024
http:
rootPath: /scheduler/*
adminPort: 4401
port: 4400

Start chronos:

/usr/bin/java -cp "$CHRONOS_HOME"/target/chronos*.jar com.airbnb.scheduler.Main server "$CHRONOS_HOME"/config/nemo_scheduler.yml

ZOOKEEPER

/usr/local/zookeeper

zookeeper config

[root@HOSTA zookeeper]# cat conf/zoo.cfg
tickTime=2000
initLimit=10
syncLimit=5
dataDir=/var/zookeeper
clientPort=2181
server.1=10.1.8.138:2888:3888

Overview

So my question is, how do I get chronos up and running with zookeeper and mesos.
At this time, I first start zookeeper manually.
Then I start mesos-master followed by mesos-slave.
Finally I kick off chronos.
Some things appear to be working. For example, I actually get to the chronos UI:
http://10.1.8.138:4400/
I can also put a job in successfully, but when forcing job, it does not appear to be executing.

INFO [2013-07-18 14:09:06,427] com.airbnb.scheduler.api.JobManagementResource: Manually triggering job:111111111111111111111111111111111111111
DEBUG [2013-07-18 14:09:06,427] com.airbnb.scheduler.jobs.TaskManager: Adding task 'ct:111111111111111111111111111111111111111:1374156546427:0' to queue
10.1.197.39 - - [18/Jul/2013:14:09:06 +0000] "PUT /scheduler/job/111111111111111111111111111111111111111 HTTP/1.1" 204 0 2 2
INFO [2013-07-18 14:09:42,276] com.airbnb.scheduler.jobs.JobScheduler: Size of streams: 1
INFO [2013-07-18 14:09:42,277] com.airbnb.scheduler.jobs.JobScheduler: Checking schedules with time horizon:PT60S
INFO [2013-07-18 14:09:42,277] com.airbnb.scheduler.jobs.JobScheduler: Calling next for stream: R/2013-07-19T09:19:05.000Z/PT24H, jobname: 111111111111111111111111111111111111111
TRACE [2013-07-18 14:09:42,277] com.airbnb.scheduler.jobs.JobScheduler: Recurrences: '-1', next date: 'R/2013-07-19T09:19:05.000Z/PT24H'
DEBUG [2013-07-18 14:09:42,277] com.airbnb.scheduler.jobs.JobScheduler: Filtering out empty streams
10.1.197.39 - - [18/Jul/2013:14:09:52 +0000] "GET /scheduler/graph/csv HTTP/1.1" 200 35 1 1
10.1.197.39 - - [18/Jul/2013:14:09:52 +0000] "GET /scheduler/jobs HTTP/1.1" 200 218 2 2

My goal is to have a cluster of mesos servers with Chronos on the front end, and zookeeper for persistence.
This will be used to handle our batch jobs across different servers.
At the moment, I am having a tough time simply getting this one host up and running.
I have yet to move on to a secondary server.
What am I missing or doing wrong?
I feel like I have asked for a lot here, but some help is better than nothing.

Someone please assist!! Thanks!

This sounds like exactly what I need. I can't see a script in the repo that does this. Can you please point me in the right direction?

Timezone support?

Hi,

I didn't see anything in the configuration or docs about specifying a timezone.

Is it possible to configure timezones for jobs?

(Say if you had jobs that were spread across machines in several colos in different timezones).

Cheers,
Victor

Build failure on Amazon Linux

I'm running Amazon Linux and the chronos tests fail. Don't ask me why :)

Here is part of my build output:

Apr 21, 2013 12:05:51 AM com.airbnb.scheduler.graph.JobGraph addVertex
WARNING: Current number of vertices:2
Apr 21, 2013 12:05:51 AM com.airbnb.scheduler.graph.JobGraph addVertex
WARNING: Adding vertex:C
Apr 21, 2013 12:05:51 AM com.airbnb.scheduler.graph.JobGraph addVertex
WARNING: Current number of vertices:3
Tests run: 7, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.265 sec
Running com.airbnb.scheduler.state.PersistenceStoreSpec
Apr 21, 2013 12:05:51 AM com.airbnb.scheduler.state.MesosStatePersistenceStore persistJob
INFO: Persisting job 'sample-name' with data 'ScheduleBasedJob(R1/2012-01-01T00:00:01.000Z/PT1M,sample-name,sample-command,PT1H,1,0,fooexecutor,args,2,,,,false)'
Apr 21, 2013 12:05:51 AM com.airbnb.scheduler.state.MesosStatePersistenceStore persistData
INFO: State J_sample-name does not exist yet. Adding to state
Apr 21, 2013 12:05:51 AM com.airbnb.scheduler.state.MesosStatePersistenceStore persistData
INFO: State update successful: true
Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.13 sec

Results :

Failed tests: 
   '0' is not equal to '1' expected:<[1]> but was:<[0]>

Tests run: 35, Failures: 1, Errors: 0, Skipped: 0

Additional info:

$ java -version
java version "1.6.0_24"
OpenJDK Runtime Environment (IcedTea6 1.11.9) (amazon-57.1.11.9.52.amzn1-x86_64)
OpenJDK 64-Bit Server VM (build 20.0-b12, mixed mode)

$ mvn -v
Apache Maven 3.0.5 (r01de14724cdef164cd33c7c8c2fe155faf9602da; 2013-02-19 13:51:28+0000)
Maven home: /opt/maven
Java version: 1.7.0_19, vendor: Oracle Corporation
Java home: /usr/lib/jvm/java-1.7.0-openjdk-1.7.0.19.x86_64/jre
Default locale: en_US, platform encoding: UTF-8
OS name: "linux", version: "3.4.37-40.44.amzn1.x86_64", arch: "amd64", family: "unix"

Thanks for your suggestions!