Microsoft patterns & practices

This project focuses on using Apache Storm/Trident with Java. For guidance on using .NET without Storm, see the companion Data Pipeline Guidance.

The two primary concerns of this project are:

• Facilitating cold storage of data for later analytics. That is, translating the chatty stream of events into chunky blobs.

• Demonstrate how to use OpaqueTridentEventHubSpout and Apache Storm/Trident to store Microsoft Azure Eventhub messages to Microsoft Azure Blob exactly-once.

• Architecture Overview
• Getting Started
• Create Java Topology Project eventhub-blobwriter from Scratch
• Design Considerations and Technical How-To

• Performance Resut: The performance result will be published once we finishes the performance test.

• Using Zookeeper to store the state: The current sample stores state in Redis Cache. We plan to replace that with Zookeeper.