Provide more Documentation for the Frontend about pgobserver HOT 10 CLOSED

Yes, I would like to. First off, sorry for the delayed response, seems this Issue went unnoticed.

For the python frontend in general you need to put a copy of the config file "pgobserver.conf" into the users home folder that will run the frontend, naming it ".pgobserver.conf".

For the frontend, configure the database section properly and maybe change your port. That should be sufficient.

For starting it, you may simply call the "run.sh" script.

Please add some error/problem reports, so we can try to solve it.

Br,
Jan

from pgobserver.

Hello Jan,

I am afraid I am in the same situation. The Python part I have running now, but the configuration is not clear to me and I need some more assistance.
From your setup documentation:

Configure .pgobserver.conf to match your setup
    set database where to store data
    configure usernames and passwords  <-- which users?
    set hostgroup ( for gatherer )   <-- please explain
    possible remove/change the additional filter "default_schema_filter" for schemas where you want your sprocs to be found. our default is scan only schemas with _api or _data    <-- please explain
Create an unprivileged PostgreSQL user on the database you want to monitor
Add entry to monitor_data.hosts table to include the databases you want to monitor   <-- which information do I need to supply in this table?
    use host_group to decide which gatherer monitors which cluster  <-- what is a 'host_group'?

Build a single jar including dependencies using

mvn clean verify assembly:single <-- how to apply this?

Please understand that Java is not a hobby of mine, but I do like to use this functionality.

Kr

Jan Robertson

from pgobserver.

Ok, I will try to answer this quickly, feel free to ask for more help if needed.

You need to configure the two users in the config file: one user used by the "gatherer" Java app to connect to the monitored databases. The other user is for the Python front end to connect to the database where the metrics are stored by the "gatherer".

"host_group" in the config file should match the group column set in host table. when the gatherer starts, he will connect to all configured databases for that particular host_group. this way if you run multiple environments you can influence which gatherer connects where.

the "default_schema_filter" you may leave empty ( will change this in template ). We do not monitor everything. whatever you configure there will just be appended to the SQL statement to select functions, so you are free to filter by schema/function name in any way you need.

as far as the hosts table goes, I believe the column names should tell you what to put where, and for the one config column it has a proper default set.

To "apply" the "mvn .." you need to have Maven installed + Java JDK ;-) then you run this command and you basically get one big "jar" file you can run via run.sh

Just for our information: Would providing a Dockerfile/Image have helped you?

Br,
Jan

from pgobserver.

Hello Jan,

still some confusion.

I see a "host_group"-table and a "hosts"-table in the database I created.
In the config file there is a "gather_group and no "host_group". Where
do I enter the databases I want to be monitored?

The users in the config file, are they users at Linux-level or in the
database ?

What do you mean by a "a Dockerfile/Image"?
Maybe you can show me a simple configured config file with the content
of a hosts-table .

Thanks for your effort anyway.

KR

Jan

On 09/30/2014 09:53 AM, Jan Mussler wrote:

Ok, I will try to answer this quickly, feel free to ask for more help
if needed.

You need to configure the two users in the config file: one user used
by the "gatherer" Java app to connect to the monitored databases. The
other user is for the Python front end to connect to the database
where the metrics are stored by the "gatherer".

"host_group" in the config file should match the group column set in
host table. when the gatherer starts, he will connect to all
configured databases for that particular host_group. this way if you
run multiple environments you can influence which gatherer connects where.

the "default_schema_filter" you may leave empty ( will change this in
template ). We do not monitor everything. whatever you configure there
will just be appended to the SQL statement to select functions, so you
are free to filter by schema/function name in any way you need.

as far as the hosts table goes, I believe the column names should tell
you what to put where, and for the one config column it has a proper
default set.

To "apply" the "mvn .." you need to have Maven installed + Java JDK
;-) then you run this command and you basically get one big "jar" file
you can run via run.sh

Just for our information: Would providing a Dockerfile/Image have
helped you?

Br,
Jan

—
Reply to this email directly or view it on GitHub
#8 (comment).

from pgobserver.

Host and Host Group are merely UI grouping, e.g. we group our customer database shards into one host group.

The host_gather_group column in monitor_data.hosts makes the connection to the gather_group in the .conf config file. The Java gatherer will collect metrics from the hosts that have monitor_data.hosts.host_grather_group = config-file.gather_group. In most scenarios you just put "host1" in there, which is the default for the table. thus all DBs are monitored from the same Jav app.

filling the hosts table requires you to set: host_name,host_port,host_user,host_password,host_db,host_enabled

there other fields are filled by default. after the insert you can modify eh host_settings json to the wished intervals.

As far as docker goes: docker.io ;-) but that is unrelated to any configuration issues. was more about building/deploying.

from pgobserver.

Hello Jan,

just ran:

|mvn clean verify assembly:single

result:

[INFO] ------------------------------------------------------------------------
[INFO] BUILD SUCCESS
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 2:14.646s
[INFO] Finished at: Thu Oct 02 13:06:44 CEST 2014
[INFO] Final Memory: 11M/45M
[INFO] ------------------------------------------------------------------------

|
|

Next:

java -jar target/PGObserver-Gatherer-1.0-SNAPSHOT-jar-with-dependencies.jar

|result:

Oct 02, 2014 1:07:24 PM de.zalando.pgobserver.gatherer.GathererApp main
INFO: Config{database=de.zalando.pgobserver.gatherer.config.Database@312737, logfiles={liveuserfilter=[postgres]}, frontend={port=8080}}
Oct 02, 2014 1:07:24 PM de.zalando.pgobserver.gatherer.GathererApp main
INFO: Connection to db:localhost using user: pgobserver_gatherer
Oct 02, 2014 1:07:25 PM de.zalando.pgobserver.gatherer.Host scheduleGatheres
INFO: Settings for Host pgobserverhost[pgobserver]
Load: 300 Seconds
Sprocs: 300 Seconds
Table IO: 600 Seconds
Table Stats: 600 Seconds

Oct 02, 2014 1:07:25 PM de.zalando.pgobserver.gatherer.Host scheduleGatheres
SEVERE: Adding Executor for Host:pgobserverhost
Oct 02, 2014 1:07:25 PM de.zalando.pgobserver.gatherer.Host scheduleGatheres
INFO: Schedule SprocGather for pgobserverhost[pgobserver]
Oct 02, 2014 1:07:25 PM de.zalando.pgobserver.gatherer.AGatherer schedule
SEVERE: Schedule: Interval 300 for Host: pgobserverhost[pgobserver]
Oct 02, 2014 1:07:25 PM de.zalando.pgobserver.gatherer.Host scheduleGatheres
INFO: Schedule LoadGather for pgobserverhost[pgobserver]
Oct 02, 2014 1:07:25 PM de.zalando.pgobserver.gatherer.AGatherer schedule
SEVERE: Schedule: Interval 300 for Host: pgobserverhost[pgobserver]
Oct 02, 2014 1:07:25 PM de.zalando.pgobserver.gatherer.Host scheduleGatheres
INFO: Schedule TableIO for pgobserverhost[pgobserver]
Oct 02, 2014 1:07:25 PM de.zalando.pgobserver.gatherer.AGatherer schedule
SEVERE: Schedule: Interval 600 for Host: pgobserverhost[pgobserver]
Oct 02, 2014 1:07:25 PM de.zalando.pgobserver.gatherer.Host scheduleGatheres
INFO: Schedule TableStats for pgobserverhost[pgobserver]
Oct 02, 2014 1:07:25 PM de.zalando.pgobserver.gatherer.AGatherer schedule
SEVERE: Schedule: Interval 600 for Host: pgobserverhost[pgobserver]
Oct 02, 2014 1:07:25 PM org.restlet.ext.simple.SimpleServerHelper start
INFO: Starting the Simple [HTTP/1.1] server on port 8182
Oct 02, 2014 1:07:26 PM de.zalando.pgobserver.gatherer.AGatherer run
INFO: [pgobserverhost[pgobserver]] started after interval 300 s
Oct 02, 2014 1:07:31 PM de.zalando.pgobserver.gatherer.SprocGatherer gatherData
SEVERE:
org.postgresql.util.PSQLException: The connection attempt failed.
at org.postgresql.core.v3.ConnectionFactoryImpl.openConnectionImpl(ConnectionFactoryImpl.java:152)

and a lot more info and errors

When I look in the browser at: localhost:8182 the page I get is:

|
|
|

the result of:

localhost:8080

is "unable to connect"

Do you have any idea what's going wrong?

Br

Jan

On 10/01/2014 03:23 PM, Jan Mussler wrote:

Host and Host Group are merely UI grouping, e.g. we group our customer
database shards into one host group.

The host_gather_group column in monitor_data.hosts makes the
connection to the gather_group in the .conf config file. The Java
gatherer will collect metrics from the hosts that have
monitor_data.hosts.host_grather_group = config-file.gather_group. In
most scenarios you just put "host1" in there, which is the default for
the table. thus all DBs are monitored from the same Jav app.

filling the hosts table requires you to set:
host_name,host_port,host_user,host_password,host_db,host_enabled

there other fields are filled by default. after the insert you can
modify eh host_settings json to the wished intervals.

As far as docker goes: docker.io ;-) but that is unrelated to any
configuration issues. was more about building/deploying.

—
Reply to this email directly or view it on GitHub
#8 (comment).

from pgobserver.

It seems to me you have not properly configured the credentials for the database you want to monitor.

looking at the :8182 result it will show you a successful gathering of data when last_persist is set.

8080 unreachable is probably because of the frontend not running?

from pgobserver.

Hello Jan,

some posts ago you asked if a docker image would be helpful. YES that would be very helpful! Do you think you could upload an image on docker hub?

Would be great!
Sanni

from pgobserver.

Looks like there have been some recent updates on the frontend section of the documentation, but is there a Docker image as per the request? Didn't see any references to one in the README ... I can help here BTW

from pgobserver.

Dockerfiles are now actually there, both in "frontend" and "backend" folders and for testing purposes there is also a Vagrant setup which utilizes them behind the scenes. Details here.

And also documentation in general has been improved some months ago so I think we can close this ticket for now.

from pgobserver.