vacek-cobra / parallec-samples Goto Github PK

• Sample Applications
• Sample Spark Server

Sample Applications demonstrate how to use parallec.io library, such as aggregate APIs and publish to Elastic Search and Kafka.

Each file is independent with a main function and can be run directly. (TCP/HTTP Async API requires the sample HTTP/TCP servers to start in advance. Sample servers included here are independent executable files too).

More comprehensive examples are available in the test cases. Such as cancel tasks. Capacity aware scheduler and etc.

####HTTP

• Basic HttpBasicMinimumApp.java: 10 lines minimum example of hitting 3 websites.

• Async HttpBasicAsyncRunProgressPollingApp.java: Use async mode to run a parallel task, and then poll the progress and show an aggregation on status code.

• Aggregate to Elastic Search Http100WebAggregateToElasticSearchApp.java: Hitting 100 common websites to get these status code to elastic search, and visualized in Kibana in 20 lines. Usage of response context to pass elastic search client demo video

• Aggregate to Kafka Http100WebAggregateToKafkaApp.java: Hitting 100 common websites to get these status code to kafka. Usage of response context to pass kafka client. You may use the included Kafka Receiver to validated. Check pom.xml to see which version of kafka we use. Simple fork this and change HOSTNAME to point to your kafka server to work. Have tested with our kafka server and receiver.

• Filter Response with Regular Expression Http3WebAgrregateToElasticSearchMinApp.java: Usage of FilterRegex.

• Request Template HttpDiffRequestsDiffServersApp.java: Different requests to different target URLs. Request template.

• Request Template HttpDiffRequestsSameServerApp.java: Different requests to same target server. Request template. setReplaceVarMapToSingleTargetSingleVar().

• Auto Progress Polling and Async APIs HttpAsyncApiPollableJob.java: demos to handle async APIs with auto progress polling. Task level concurrency control. (require starts the Sample Web Server with Async API first)

####Set Target Hosts Please refer to the documentation to set target hosts differently.

###Usage

You may simple fork the project, or copy and paste indivisual files after getting the correct dependencies

Maven

<dependency>
	<groupId>io.parallec</groupId>
	<artifactId>parallec-core</artifactId>
	<version>0.9.3</version>
</dependency>

Gradle

compile 'io.parallec:parallec-core:0.9.3'

#####Screenshots

Executing Http3WebAgrregateToElasticSearchMinApp.java, visualized in Kibana.

With elasticsearch-1.3.4 and kibana-3.1.2

##Sample Spark Server

Sample Single File Web Server in Spark with Parallec . Require JDK 1.8+ due to Spark server.

• SparkServer: http://sparkjava.com

####Build & Run ######Build:

Fork the project and run:

mvn clean compile assembly:single

######Run: In folder: parallec-samples/sample-spark-server/target: (if not conduct ping, no need of sudo)

sudo java -jar parallec-sample-spark-server-0.9.0-jar-with-dependencies.jar

The APIs tries to get target hosts from a local file in the same path.

 get("/http/:filename"
 get("/ssh/:filename/:concurrency/:showDetail"

targetHostFile location:

• If in IDE, put into the same folder as the pom.xml.
• If run as jar. just same folder as the executable jar file.

APIs:

• localhost:4567/ssh/targetHostFile/200/true (require update login user/password)
• localhost:4567/ping/targetHostFile ; need root
• localhost:4567/http/targetHostFile : use elastic search

######Sample Output

http://localhost:4567/http/targetHostFile

Parallec: completed HTTP and sent to elastic search 
:3 Servers in 0.575 seconds. Results:  Results: [200 OK COUNT: 3 ]:
	www.parallec.io
	www.jeffpei.com
	www.ebay.com
###################################

At 2015.11.01.10.15.47.398-0800

Thanks for trying Parallec.io. Please submit a git issue for any questions you have.

Yuanteng (Jeff) Pei

Your name here

Code licensed under Apache License v2.0

© 2015 eBay Software Foundation