| tags | projects | |
|---|---|---|
|
You’ll set up Spring XD (eXtreme Data), create a stream to monitor a live twitter feed, and pipe it into a file.
If you are on a Mac there are some simple instructions to Install Homebrew.
If you don’t have a Mac or you aren’t interested in installing Homebrew, then you can skip that and jump right to Installing Spring XD.
There are several package managers available for Mac OS X, but one of the most popular is Homebrew. And we have strong support for Homebrew! If you haven’t already, use these steps to set it up.
ruby -e "$(curl -fsSL https://raw.github.com/mxcl/homebrew/go)"
This installation script will provide extra information.
Homebrew has a base set of "formulas" to install many tools. It also supports 3rd party sets called "taps". Pivotal has a tap that provides several tools including Spring XD, which you’ll see in the next section.
If you are using a Mac with Homebrew, the process is pretty simple.
brew tap pivotal/tap && brew install springxd
That’s it! It might take a little bit of time to install.
If you aren’t using Homebrew or you’re on a different platform:
-
Download Spring XD 1.2.0.RELEASE.
-
Move it to your preferred folder and unzip it. For convenience, we will refer to this location as
$XD_INSTALL_DIR
To start Spring XD:
$XD_INSTALL_DIR/xd/bin/xd-singlenode
|
Note
|
singlenode is Spring XD’s simplest mode and more suited for demonstration purposes. See https://github.com/spring-projects/spring-xd/wiki for more options and details. |
You should see something like this:
_____ __ _______
/ ___| (-) \ \ / / _ \
\ `--. _ __ _ __ _ _ __ __ _ \ V /| | | |
`--. \ '_ \| '__| | '_ \ / _` | / ^ \| | | |
/\__/ / |_) | | | | | | | (_| | / / \ \ |/ /
\____/| .__/|_| |_|_| |_|\__, | \/ \/___/
| | __/ |
|_| |___/
1.2.0.RELEASE eXtreme Data
Started : SingleNodeApplication
Documentation: https://github.com/spring-projects/spring-xd/wiki
In another terminal, start Spring XD’s management shell:
$XD_INSTALL_DIR/shell/bin/xd-shell
This will produce a prompt:
_____ __ _______
/ ___| (-) \ \ / / _ \
\ `--. _ __ _ __ _ _ __ __ _ \ V /| | | |
`--. \ '_ \| '__| | '_ \ / _` | / ^ \| | | |
/\__/ / |_) | | | | | | | (_| | / / \ \ |/ /
\____/| .__/|_| |_|_| |_|\__, | \/ \/___/
| | __/ |
|_| |___/
eXtreme Data
1.2.0.RELEASE | Admin Server Target: http://localhost:9393
Welcome to the Spring XD shell. For assistance hit TAB or type "help".
xd:>
Inside Spring XD’s shell create a twitter stream:
xd:> stream create --name twittersearchjava --definition "twittersearch --consumerKey=afes2uqo6JAuFljdJFhqA --consumerSecret=0top8crpmd1MXGEbbgzAwVJSAODMcbeAbhwHXLnsg --query='java' | file" --deploy
Here you are creating a stream which consists of a source and a sink. In addition to creating a stream definition, You are also deploying the stream using the --deploy option.
-
The stream is named twittersearchjava.
-
The source is twittersearch, using credentials required by the Twitter API, querying on the token java.
-
The results are piped into the file sink, which defaults to /tmp/xd/output/[streamName].out
In another terminal:
$ cd /tmp/xd/output $ tail -f twittersearchjava.out
Spring XD is capturing live data from twitter about java and writing it to twittersearchjava.out. You should see the tweets rendered as JSON. While Spring XD runs and the stream is deployed, the file sink will continue to grow as it accumulates more data. You can stop the stream by undeploying it from the Spring XD shell:
xd:>stream undeploy twittersearchjava
Congratulations! You’ve just installed Spring XD and created a stream to capture a live twitter feed and pipe it into a file.
React
Typescript
Django
Laravel
Facebook
Microsoft
Google
Alibaba
D3
Tencent