deepakddixit / monarch Goto Github PK

Overview Documentation Contributing

Monarch is a robust, in-memory computing platform that provides operational intelligence to both user (external) apps and data science driven analytics (internal) apps. It is based on [Apache Geode] (http://geode.apache.org/), adding several capabilities for analytical data workloads.

Please visit the documentation section.

This project welcomes new contributors. If you would like to help by adding new features, enhancements or fixing bugs, check out the contributing guidelines.

You acknowledge that your submissions to this repository are made pursuant the terms of the Apache License, Version 2.0 (http://www.apache.org/licenses/LICENSE-2.0.html) and constitute "Contributions," as defined therein, and you represent and warrant that you have the right and authority to do so. You can reach us on the group.

You can download and install the latest version of Monarch from the Monarch Release page. Refer to the documentation for installation steps.

If you would like to build Monarch from source, refer to the documentation on building from source.

NOTE: In the document refers to the full directory path in which Monarch s/w is installed.

Go to /examples directory and run the runExamples.sh script

$ ./examples/runExamples.sh -i <ampool_home>

At the end of the running you should see the following message:

Stopping Ampool
All examples are running
$

This implies the installation is correctly done, else please contact the [email protected]

Ampool System/Cluster for all occurrences is based on a locator-server configuration of Apache Geode, referred here as Ampool cluster. Thus an Ampool cluster consist of one or more locators and one or more servers connected in certain topology.

Locator is an important component of Ampool cluster. Locator maintains and provides up to date topology information to clients and servers. Locator guides clients to servers for data. There can be more than one locator in a cluster. Multiple locators in a cluster are recommended to insure continued availability in case a locator machine failure.

Servers hold and maintain the data for a table, which internally is split into a number of buckets based on partition configuration. Servers communicate to maintain bucket balance and consistency.

Ampool provides an interactive command line interface for the Ampool cluster management and monitoring called MASH (Memory Analytics Shell). It provides multiple commands to manage the cluster.

To start MASH, type the following on command prompt

$ <ampool-home>/bin/mash
mash>

The mash prompt above implies you are in MASH shell. Command help can be used to obtain the list all the supported commands

mash>help

To get details of a specific command type help followed by the specific command e.g. start sever command, start locator command.

mash>help start server
mash>help start locator

There are two services (the Ampool Locator and Ampool Server) that need to be started before you start using Ampool. The recommended order of starting the two services is Locator first followed by Servers. Similarly recommended way to stop the two services is in the reverse order.

The locator can be started using MASH start locator command as shown below.

mash>start locator  --name=locator1  --dir=<full path to pre-created locator data directory>
--properties-file=<ampool-home>/config/ampool_locator.properties
--hostname-for-clients=localhost

Below are more commonly used options when starting the locator. For the complete list of supported options you can refer help start locator command.

• dir: Directory in which the Locator will be started and run. This directory needs to exist a priori. The default is the current directory from which MASH is launched.
• hostname-for-clients : Hostname or IP address that will be sent to clients so they can connect to this Locator. The default is the bind-address of the Locator.
• Port: The port to be used for communication by the locator. If not specified, the default port 10334 is used.
• properties-file:The additional set of properties can be passed when starting the locator. For more details see /config/ampool_locator.properties file.

Please use help command for more detailed description and list of parameters i.e.

mash>help start locator

The Server can be started using MASH start server command as shown below.

mash>start server  --name=server1  --server-port=40404  --locators=localhost[10334]
--dir= <full path to pre-created locator data directory>
 --properties-file=<ampool-home>/config/ampool_server.properties

Below are more commonly used options when starting the servers. For the complete list of supported options you can refer help start server command.

• dir : Directory in which the Cache Server will be started and run. This directory needs to
• locators : Sets the list of Locators used by the Ampool Server to join the appropriate Ampool cluster.
• properties-file : The additional set of properties can be passed when starting the server. For more details see /config/ampool_server.properties file.
• server-port: The port that the distributed system’s server sockets in a client-server topology will listen on. The default server-port is 40404. Please use help command for more detailed description and list of parameters i.e.

mash>help start server

The Ampool services (both locator and server) can be stopped using MASH commands. It is recommended that the servers are stopped before the locator. Assuming the above listed Locator and Sever are running, you can use below command to stop a single server. 

mash>stop server --name=server1

And the below command can be used to stop the locator service. 

mash>stop locator --name=locator

f you want to stop all the members (except the locator) in the Ampool cluster, you can use the shutdown command. But, stopping all members may cause data-loss if the disk-persistence is not enabled for the created tables. 

mash>shutdown

To include the locator in the shutdown use 

mash>shutdown --include-locators=true

Let us say you have 5 machines (Machine_0 to Machine_4) and a Client machine on which you want to create an Ampool cluster and run the Ampool services (i.e. one locator and 4 servers) using MASH commands.

Assuming the above machine names and Ampool package is loaded on each of the machines including client machine, follow the process as below:

1. Log into Machine_0 and start Locator_0 through MASH shell using start locator command.

mash>start locator  --name=Locator_0

2. Log into Machine_1 and start Server_1 through MASH shell using start server command as foloows:

mash>start server  --name=Server_1  --server-port=40404  --locators=Machine_0[10334]

3. Log into Machine_2 and start Server_2 through MASH shell using start server command as follows:

mash>start server  --name=Server_2  --server-port=40404  --locators=Machine_0[10334]

4. Log into Machine_3 and start Server_3 through MASH shell using start server command as follows:

mash>start server  --name=Server_3  --server-port=40404  --locators=Machine_0[10334]

5. Log into Machine_4 and start Server_4 through MASH shell using start server command as follows:

mash>start server  --name=Server_4  --server-port=40404  --locators=Machine_0[10334]

Your Ampool Cluster is all set. Now you can connect to the cluster through Client Machine using MASH as follows:

mash>connect --locator=Machine_0[10334]

Now you are ready to play around with other MASH commands that will help you create tables, put and update data, get data, delete tables etc. as described in next session.

To list a few commonly used MASH commands let us attempt to do the following:

1. Start an Ampool Cluster with one Locator and two Servers on local host.
2. Connect to the Ampool Cluster.
3. List members of the cluster.
4. Create an Ordered table and call it TableOrdered.
5. Create an Unordered table and call it TableUnordered.
6. Create an Immutable table and call it TableImmutable.
7. List the tables.
8. Describe tables.
9. Insert a few rows into Ordered table i.e. TableOrdered.
10. Insert a few rows into Unordered table i.e. TableUnordered.
11. Append a few rows into Immutable table i.e. TableImmutable.
12. Get a complete row entry with specific Key from a table to verify the entries inserted above.
13. Update a specific column of a row in the table.
14. Delete a complete row entries with specific Key from the table.
15. Full table scan on tables.
16. Scan all entries within a specified range (applicable only for Ordered table).
17. Delete a table.
18. Stop Servers.
19. Stop Locator.
20. Exit MASH.

NOTE: Only the usage of the commands is shown. For details on individual commands and complete list of parameters, use the help command in MASH.

mash>start locator --name=locator_1
mash>start server --name=server_1 –-server-port=40404
mash>start server --name=server_2 –-server-port=40405

mash>connect --name=locator_1

mash>list members

This will display list containing the locator_1, server_1 and server_2

Creating tables with --columns option with create a table with all column types as BINARY. In order to create a table with supported column types please use --schema-json option. It is applicable to all types of tables.

create table --name=TableOrdered --type=ORDERED_VERSIONED --columns=ID,NAME,AGE,SEX,DESIGNATION

create table --name=TableUnordered --type=UNORDERED --columns=ID,NAME,AGE,SEX,DESIGNATION

create table --name=TableImmutable --type=IMMUTABLE --schema-json='{"ID":"INT", "NAME":"STRING", "AGE":"INT", "GENDER":"STRING","DESIGNATION":"STRING"}'

mash>list tables

This will display list containing TableOrdered and TableUnordered

mash> describe table –name=TableOrdered
mash> describe table –name=TableUnordered
mash> describe table –name=TableImmutable

This above two will give a complete description of TableOrdered and TableUnordered

To insert into tables with columns having BINARY type, --value option can be used. It converts the provided string data to BINARY. If the table was defined with column types other than BINARY, then please --value-json option that will convert column values to the respective data types. This options is applicable to all types of tables including tput and tappend commands.

mash> tput --key=1 --value=ID=1,NAME=Andy,AGE=34,SEX=M,DESIGNATION=PM
--table= TableOrdered

mash> tput --key=2 --value=ID=2,NAME=Avinash,AGE=38,SEX=M,DESIGNATION=ARCH
--table= TableOrdered

mash> tput --key=3 --value=ID=3,NAME=Deepa,AGE=26,SEX=F,DESIGNATION=SE
--table= TableOrdered

mash> tput --key=4 --value=ID=4,NAME=Sri,AGE=32,SEX=M,DESIGNATION=SSE
--table= TableOrdered

mash> tput --key=5 --value=ID=5,NAME=Avani,AGE=36,SEX=F,DESIGNATION=PL
--table= TableOrdered

mash> tput --key=6 --value=ID=6,NAME=Satya,AGE=28,SEX=F,DESIGNATION=SE
--table= TableOrdered

mash> tput --key=7 --value=ID=7,NAME=Manish,AGE=34,SEX=M,DESIGNATION=ARCH
--table= TableOrdered

mash> tput --key=8 --value=ID=8,NAME=Shankar,AGE=32,SEX=M,DESIGNATION=TL
--table= TableOrdered

mash> tput --key=1 --value=ID=1,NAME=Raghu,AGE=34,SEX=M,DESIGNATION=DIR
--table= TableUnordered

mash> tput --key=2 --value=ID=2,NAME=Bali,AGE=27,SEX=M,DESIGNATION=SE
--table= TableUnordered

mash> tput --key=3 --value=ID=3,NAME=Suman,AGE=26,SEX=F,DESIGNATION=SE
--table= TableUnordered

mash> tput --key=4 --value=ID=4,NAME=Danny,AGE=32,SEX=M,DESIGNATION=SSE
--table= TableUnordered

mash> tput --key=5 --value=ID=5,NAME=Kiran,AGE=31,SEX=F,DESIGNATION=SSE
--table= TableUnordered

mash> tput --key=6 --value=ID=6,NAME=Fransis,AGE=33,SEX=F,DESIGNATION=MNG
--table= TableUnordered

mash> tput --key=7 --value=ID=7,NAME=Arvind,AGE=34,SEX=M,DESIGNATION=ADM
--table= TableUnordered

mash> tput --key=8 --value=ID=8,NAME=Hari,AGE=32,SEX=M,DESIGNATION=FIN
--table= TableUnordered

mash>tappend --table=TableImmutable --value-json='{"ID":"1", "NAME":"Joey", "AGE":"33", "GENDER":"M","DESIGNATION":"ARCH"}'
mash>tappend --table=TableImmutable --value-json='{"ID":"2", "NAME":"Monica", "AGE":"38", "GENDER":"F","DESIGNATION":"ARCH"}'
mash>tappend --table=TableImmutable --value-json='{"ID":"3", "NAME":"Janice", "AGE":"29", "GENDER":"M","DESIGNATION":"HR"}'

mash>tget --key=2 --table=TableOrdered

This display all the column and their values for row whose key = 2 from TableOrdered.

mash>tget --key=7 --table=TableOrdered

This display all the column and their values for row whose key = 7 from TableOrdered.

mash>tget --key=4 --table=TableUnordered

This display all the column and their values for row whose key = 4 from TableUnordered.

mash>tget --key=6 --table= TableUnordered

This display all the column and their values for row whose key = 6 from TableUnordered.

Here the same tput command is used. If the specified key already exists, then tput acts like an update else it creates a new entry in the table. For example.

mash> tput --key=6 --value=NAME=Raj,AGE=35 --table=TableOrdered

Here row with key=6 exist in TableOrdered and therefore the above command only updates the specified column values i.e. NAME=Raj and AGE=35. One can verify the change made using tget –key=6 –table=TableOrdered command.

mash> tput --key=11 --value=ID=11,NAME=Zhaid,AGE=25 --table=TableUnordered

Here row with key=11 does not exist in TableUnordered and therefore the above command will create a new row entry in TableUnordered table with specified key and column values. One can verify the change made using tget --key=11 --table=TableUnordered command. If the table was created by specifying the column types using JSON format (see help on create table command), one can specify the values in JSON format too. For example:

mash> tput --key=11 –value-json=”{‘ID’:’11’,’NAME’:’Zhaid’,’AGE’:’25’}” --table=TableUnordered

This is applicable to only Ordered and Unordered tables.

mash> tdelete --key=8 --table=TableOrdered

Here the complete row with key=8 will be permanently deleted from TableOrdered

mash> tdelete --key=3 --table=TableUnordered

Here the complete row with key=3 will be permanently deleted from TableUnordered.

mash>tscan --table=TableOrdered
mash>tscan --table=TableUnordered
mash>tscan --table=TableImmutable

mash>tscan --table=TableOrdered --startkey=2 --stopkey=7

This display all the rows starting with key falls in the range [startkey, stopkey) range. Note the stopkey is excluded from the range. NOTE: Key being a STRING object, string comparison is used for determining if a key falls in the range or outside the range.

mash>delete table --name=TableOrdered
mash>delete table --name=TableUnordered
mash>delete table --name=TableImmutable

The deletion can be verified by using list tables command. You’ll find the deleted tables now missing from the list.

mash>stop server --name=server_1
mash>stop server --name=server_2

mash>stop locator --name=locator_1

mash>exit

You can find documentation for Monarch client API under 

<ampool-home>/javadoc directory.

Few examples demonstrating the use of Ampool Java client APIs can be found at location: /examples directory.