charwliu / fiware-cygnus Goto Github PK

#Cygnus

(cygnus-common)
(cygnus-ngsi)

##Welcome This project is part of FIWARE, being part of the Cosmos Ecosystem.

Cygnus is a connector in charge of persisting certain sources of data in certain configured third-party storages, creating a historical view of such data.

Internally, Cygnus is based on Apache Flume, a technology addressing the design and execution of data collection and persistence agents. An agent is basically composed of a listener or source in charge of receiving the data, a channel where the source puts the data once it has been transformed into a Flume event, and a sink, which takes Flume events from the channel in order to persist the data within its body into a third-party storage.

Cygnus is designed to run a specific Flume agent per source of data.

Current stable release is able to persist the following sources of data in the following third-party storages:

• NGSI-like context data in:
  • HDFS, the Hadoop distributed file system.
  • MySQL, the well-know relational database manager.
  • CKAN, an Open Data platform.
  • MongoDB, the NoSQL document-oriented database.
  • FIWARE Comet (STH), a Short-Term Historic database built on top of MongoDB.
  • Kafka, the publish-subscribe messaging broker.
  • DynamoDB, a cloud-based NoSQL database by Amazon Web Services.
  • PostgreSQL, the well-know relational database manager.
  • CartoDB, the database specialized in geolocated data.
• Twitter data in:
  • HDFS, the Hadoop distributed file system.

##Further documentation The per agent Quick Start Guide provides a good documentation summary.

Nevertheless, both the Installation and Administration Guide and the User and Programmer Guide shown within this documentation for each agent cover more advanced topics.

The per agent Flume Extensions Catalogue completes the available documentation for Cygnus.

Of special interest are the Contributing Guidelines if your aim is to extend Cygnus.

Finally, please check our Apiary Documentation if you want to know how to use our API methods for Cygnus.

##Licensing Cygnus is licensed under Affero General Public License (GPL) version 3. You can find a copy of this license in the repository.

##Reporting issues and contact information Any doubt you may have, please refer to the Cygnus Core Team.