Onyx

What is it?

• a masterless, cloud scale, fault tolerant, distributed computation system
• written in Clojure, for Clojure
• batch and stream processing hybrid
• exposes an information model for the description and construction of distributed workflows
• enabled by hardware advances in the last 10 years
• Competes against Storm, Cascading, Map/Reduce, Dryad, Apache Sqoop, Twitter Crane, etc

What would I use this for?

• Realtime event stream processing
• Continuous computation
• Extract, transform, load
• Data transformation à la map-reduce
• Data cleaning
• Data ingestion and storage medium transfer

Installation

Available on Clojars:

[com.mdrogalis/onyx "0.5.3"]

Quick Start Guide

Feeling impatient? Hit the ground running ASAP with the onyx-starter repo and walkthrough.

User Guide 0.5.3

• What does Onyx offer?
• Concepts
• Environment
• Hardware
• APIs
• Constraints
• Internal Design
• Peer Configuration
• Information Model
• Functions
• Flow Conditions
• Scheduling Jobs and Tasks
• Event Subscription
• Error Handling
• Plugins
• HornetQ Internal Plugin
• Reliability Guarantees
• Logging
• Performance Tuning
• Examples
• Frequently Asked Questions

API Docs 0.5.3

Code level API documentation can be found here.

Official plugin listing

Official plugins are vetted by Michael Drogalis. Ensure in your project that plugin versions directly correspond to the same Onyx version (e.g. onyx-core-async version 0.5.3 goes with onyx version 0.5.3). Fixes to plugins can be applied using a 4th versioning identifier (e.g. 0.5.3.1).

• onyx-hornetq
• onyx-datomic
• onyx-sql
• onyx-kafka
• onyx-core-async

Generate plugin templates through Leiningen with onyx-plugin.

Offical Dashboard

You can run a dashboard to monitor Onyx cluster activity, found here.

lib-onyx

lib-onyx is a library created to support extra functionality in Onyx. It provides pluggable functionality such as in-memory streaming joins, automatic message retry, and interval-based actions.

Release Notes

You can find the lastest major release notes here.

Need help?

Check out the Onyx Google Group.

Want the logo?

Feel free to use it anywhere. You can find a few different versions here.

Running the tests

A simple lein midje will run the full suite, which takes about 15-20 minutes on my quad-core MacBook Pro.

Contributing

Contributions are welcome. Please fork the repository and send a pull request to the master branch.

Branching

Onyx uses a similiar branching strategy to Clojure itself. Onyx uses semantic versioning, and each minor version gets its own branch. All work is done on master or feature branches and dropped into a major.minor.x branch when it's time to cut a new release. Pull requests into the master branch are welcome.

Commit rights

Anyone who has a patch accepted may request commit rights. Please do so inside the pull request post-merge.

Contributor list

• Michael Drogalis
• Owen Jones
• Bruce Durling
• Malcolm Sparks
• Lucas Bradstreet
• Bryce Blanton
• David Rupp

Performance Benchmarks

At the time of writing this, I do not have any performance benchmarks to publish. Creating a correct, useful benchmark is extremely difficult. I'm working on it - hang tight.

Author

This project is authored by Michael Drogalis. You can get me directly at (mjd3089.at.rit.edu) if needed.

License

Copyright © 2015 Michael Drogalis

Distributed under the Eclipse Public License, the same as Clojure.

Profiler

YourKit supports open source projects with its full-featured Java Profiler. YourKit, LLC is the creator of YourKit Java Profiler and YourKit .NET Profiler, innovative and intelligent tools for profiling Java and .NET applications.