I'd love some instructions that work for Debian squeeze. I know people say it can work, but I can't get it to work.

This is what I did:

echo "deb http://stable.packages.cloudmonitoring.rackspace.com/debian-squeeze-x86_64 cloudmonitoring main" > /etc/apt/sources.list.d/rackspace-monitoring-agent.list

curl https://monitoring.api.rackspacecloud.com/pki/agent/linux.asc | sudo apt-key add -

sudo apt-get update

sudo apt-get install rackspace-monitoring-agent

I get the following error:

insserv: warning: script 'K01nova-agent' missing LSB tags and overrides
insserv: script mongoshard: service mongoshard already provided!
insserv: warning: script 'nova-agent' missing LSB tags and overrides
insserv: warning: script 'mongos' missing LSB tags and overrides
Stopping rackspace-monitoring-agent: rackspace-monitoring-agent failed!

A little help?

Create documentation on the READ.ME for server-side configuration.

Explain features, attributes, parameters, and requirements for server-side configuration.

We have a thought that the list of host info types reported by the agent to the AEP should only be the types supported by this agent on this OS. Discuss.....

README is invalid for FreeBSD 10 installation. You need to go with gmake instead of make to properly start the build.

We should emit metrics from each agent through the AEPs to do extra monitoring:
memory usage
cpu time
process age
something else

• Write Unit Tests for HUP support (logrotation)
• Write StdoutFileWriter

https://github.com/rphillips/virgo-logging

Goal: Create a website for Virgo Agent Project to explain what this project is about, how it can be valuable for other agent writers, and how to contribute to the project.

The target audience for document should be the developers who wish to learn about the project, use it to build other agents, or contribute to the project. This would not include the documentation on how the features are surfaced through any particular product, like Rackspace Cloud Monitoring. Those documentation should be managed by the vendors themselves.

Repo for doc: https://github.com/virgo-agent-toolkit/docs

Suggested Layout:
A. Documentation

1. Introduction (getting started)
2. Advanced topics (Troubleshooting and debugging)
3. Automation (how to use automation with the agent)

B. Version Information (see current and past version info)

1. Release notes
   C. Resources
2. Link to List of Agent Check Types on Rackspace Documentation - documentation SPECIFIC to the project and its developers here.
   * API docs(arguments, return values) of functions that lives in the code base, including checks.
   * Differences or 'features' that the rackspace-monitoring-agent adds to virgo
   * Common pitfalls/gotchas while developing a new check type
   * Generic troubleshooting for connecting and using versions of rackspace-monitoring-agent that
3. List of Agent Plug-ins (not supported)
4. Link to other agent related features, like the capability to pick up configuration files and push to the server
   D. Get Involved
5. How to Add example configs and YAML files
6. How to Improve Documentation
7. How to Add a plug-in

E. Developer Resources

1. Architecture Diagram
2. Plug-in Architecture
3. Coding style
4. About Luvit (likely links to other resources)
5. About Lua (likely links to other resources)

Inspiration: https://collectd.org/wiki/index.php/Main_Page

Currently agent checks don't make it easy to get the metrics out with type and unit information. This metadata is stored in various ways and there's no way to programmatically extract it from all checks afaik.
https://github.com/racker/virgo/blob/master/check/apache.lua#L97
https://github.com/racker/virgo/blob/master/check/disk.lua#L59
It would be cool if all checks supported a method to return the metric metadata in a consistent format.

Create a GUI for Windows to perform the same task as the --setup option when running the agent from the command-line.

Ideally this can be accessed from a Start Menu item if reconfiguration is needed and will be run during the MSI installer.

Hi,

Trying to find any documentation but ... non found. Am I missing it?

Tx

In order to ease configuration and help customers who do not have access to the public Internet, we could fall back to the service net endpoints for API and AEP.

Initial thoughts are to use the xenstore vm-data to determine if the agent is on a Rax cloud host.

TL;DR: division in virgo has some design issues to address. I'm gonna proceed with a little hacky way for now, but we need to think about the design for long-term run.

Currently all lua scripts, monitoring-related or not, are in virgo instead virgo-base. This is fine for people who just wanna deploy their own monitoring system, but not good for people who wanna develop their own agent based on virgo-base, e.g., a cloudkeep(https://github.com/cloudkeep) agent that uses FUSE to provide keys to applications. The lua files in virgo need to be divided into monitoring-related part (which stays in virgo) and general part (which should be moved into virgo-base.

Here's a diagram I drew while reading:

Red nodes are the ones with monitoring logic. Some of them are easy to deal with, like /check/*.lua, /schedule/scheduler.lua, since they are independent modules that can be simply isolated from the core part. The challenging part is the MonitoringAgent -> ConnectionStream -> AgentClient -> AgentProtocolConnection path. Each of them more of less has something to do with monitoring.

@robert-chiniquy and I thought about providing a base type for each of them, which has core logic such as handshake.hello (authentication) and heartbeat, and use dependency injection to determine what type to use. When users need to extend something, they simply sub-class the base type and inject the type to proper place.

However, the idea above has issues since there are too many levels. Suppose we need to add a new RPC call, helloworld, which uses a new message type helloworld.param. The message type is not a big issue. But we also need to add the handler for helloworld RPC method, which is defined in /protocol/connection.lua. However, it's in the bottom level. In order to support that, the type needs to be specified somehow from top down.

For now, so we are gonna add an argument to constructors, which has types that needs to be overwritten. Then when instantiating objects, if a type is defined for that object, use that type; other wise, use the default one. But this is not a good design in long-term. We probably still need to refactor those scripts in the future for a cleaner design.

number of total files open on the system.

https://github.com/virgo-agent-toolkit/rackspace-monitoring-agent/pull/459/files

I got a uuid that looks like this: f5a39211-3b09-4103-c1a8-

@rphillips ideas?

Instead of using an external luvit binary to run tests zip up the tests and run via the entry point method.

This is a second issue from #489.

I probably wasn't obvious and clear enough in my original pull request description, but there are actually two issues in play here. First one is logging issue which has been fixed in #489 and the second one is the actual crash dump upload issue which I haven't been able to track down yet.

Every time I restart the agent it tries to upload some crash dump files, but it either fails with socket hung up or ECANCELED error.

Second (ECANCELED) error seems to be related to the retried request.

Mon Jan  6 18:42:05 2014 WRN: POST to nil:nil failed for /var/lib/rackspace-monitoring-agent/rackspace-monitoring-agent-crash-report-ponies.dmp with status: ? and error: socket hang up.
Mon Jan  6 18:42:05 2014 DBG: retrying download 1 more times.
Mon Jan  6 18:42:05 2014 WRN: POST to nil:nil failed for /var/lib/rackspace-monitoring-agent/rackspace-monitoring-agent-crash-report-ponies.dmp with status: ? and error: ECANCELED, operation canceled.

If I manually try to send a crash dump from that server using cURL it works just fine.

Could it be some weird socket pooling / re-use issue going on in Luvit since it posts crash dump to the same IP address which is also used to talk to the agent endpoint?

Edit: Here is a screenshot from a tcp dump capture. All those RST's at the end look kinda weird... Might be related?

The non-debug logging level prints information regularly. Reduce this to just errors and warnings.

When constructing these objects, sigar is called to populate the object. This should actually be populated within the run function to allow everything to be async. A better fix would be to run these in a subprocess.

In #636 and https://github.com/racker/ele/pull/2630 we've been talking of expanding the handshake message to better accommodate the new and growing feature list field. The suggestions are tending toward simplify the handshake and breakout the features.

The current handshake (handshake.hello), https://github.com/virgo-agent-toolkit/virgo-base-agent/blob/master/libs/connection.lua#L262, contains the auth token, monitoring id, agent (application) name, versions, and the feature set.

We could reduce the handshake down (again) to more of a plain authenticate if we move the monitoring id and the features into a secondary message (something akin to handshake.identify). This way the hello authenticates and versions the connection while the identify binds and sets up features for this agent.

/var/run/{agent}/uuid I guess?

Move the config logic into lua land. Right now it's in C and exported as virgo.config. This does not fit in the newer model.

The task is to read config files in lua and available within virgo-base.

Now that Cloud Intelligence is fully functional, should we change the following prompt (the line after "Select Option") in agent setup to mention it?

In order to execute checks, the agent must be associated with a Cloud Monitoring Entity.

Please select the Entity that corresponds to this server:
  1. Create a new Entity for this server (not supported by Rackspace Cloud Control Panel)
  2. Do not associate with an Entity

Select Option (e.g., 1, 2): 1
Creating an entity does not work with the Rackspace Cloud Control Panel. Really create an entity? (yes/no) yes

This is for the README update. The website update is discussed here: #551

The target audience for document should be the developers who wish to learn about the project, use it to build other agents, or contribute to the project. This would not include the documentation on how the features are surfaced through any particular product, like Rackspace Cloud Monitoring. Those documentation should be managed by the vendors themselves.

Topics to address:
A. What is (the agent)
B. Key Concepts
C. How to Install and Configure
D. How to use it (the agent)
E. Supported OSes
F. Sub-section of specific topics link to:

1. Check Types and their metrics
2. Plug-ins
   G. Debugging and Troubleshooting
   H. Tests
   I. Contacts (e.g. IRC and mailing list)

Inspiration:
https://github.com/etsy/statsd

I need to look into this. dtruss shows monitoring.zip opened multiple times. Should just require holding state within the module loader.

Currently both rackspace-monitoring-agent and make test target uses a same bundle (rackspace-monitoring-agent-bundle.zip) built from make_bundle target. It would be nice to have a test bundle target just for testing, which contains tests related stuff, and a release bundle that only have lua scripts required to run. This way we can keep the released bundle clean/minimal.

This could show all apt-get packages (on Ubuntu) which have un-applied updates. Even cooler would be if it could track which stream they come from (i.e. security vs. general updates).

When I was playing with ninja it was complaining that "all" was an ambiguous target. Paul mentioned virgo has a all.gyp which is a bit weird anyways. Perhaps we are doing something wrong with gyp?

There is a wide variety of data available using the HostInfo system within the agent. People have expressed interest in using HostInfo data as check data.

Pros:

• HostInfo data can then be treated as metrics and stored long-term
• Changes to HostInfo has the potential to be alarmed on

Cons:

• Some HostInfo data does not map well to a check format

Using

./configure --prefix=/usr

causes the make install to fail. Using a normal

make install

with no DESTDIR set works just fine, but for some reason the Makefile wants to put the prefix before the DESTDIR, so running

make DESTDIR=/home/wgiokas/pkg/virgo-git install

when the prefix is set to /usr causes it to try

install -d /usr//home/wgiokas/pkg/virgo-git//usr/bin

Also, when the prefix is set to / and I run the same make install command as before, it still tries to install to

///home/wgiokas/pkg/virgo-git//usr/bin

To conclude (tl;dr):

1. what the help says (it defaults to a prefix of /usr, not /usr/local)
2. the prefix should be after the DESTDIR
3. the prefix should overwrite whatever is setting /usr

Thank you,
kaictl

The GC support shouldn't block the eventloop if there are not any pending events.

It should be centos-5-x86_64

/exe/1.1.0-27/centos-5.11-x86_64-rackspace-monitoring-agent-1.1.0-27.sig

The agent should be able to report which custom plugin files are available for it to run. This would be a list of executable files in /usr/lib/rackspace-monitoring-agent/plugins

The Apache check does not display the status of the virtual hosts. http://blog.e-shell.org/132
An alternative would be to enhance the apache check to have the virtual hosts status. The risk is the variation of the number of metrics, and how to map the host name as part of the metrics locator.

I've encountered condition today when agent had more then 500 established connections to the London endpoint.

We should put some kind of sanity check to the agent to make sure there at most 1 established connection per endpoint.

On a side note, we should also put limit inside the endpoint.

FreeBSD 10 release has ditched GCC (completely - it's not even in the base system anymore) in favour of LLVM/CLANG. Because of that build on this target OS fails with this error (after some time build was running):

--- CUT ---
COPY /opt/virgo-0.1.9/out/Debug/jit
ACTION _opt_virgo_0_1_9_base_bundle_gyp_bundle_h_target_bundle /opt/virgo-0.1.9/out/Debug/obj/gen/bundle.h
TOUCH /opt/virgo-0.1.9/out/Debug/obj.target/base/bundle.zip.embed.stamp
TOUCH /opt/virgo-0.1.9/out/Debug/obj.target/base/bundle.zip.stamp
AR(target) /opt/virgo-0.1.9/out/Debug/obj.target/base/deps/libarchive.a
AR(target) /opt/virgo-0.1.9/out/Debug/obj.target/base/deps/liblua_sigar.a
LINK(target) /opt/virgo-0.1.9/out/Debug/minilua
lockf: g++: No such file or directory
gmake[1]: *** [/opt/virgo-0.1.9/out/Debug/minilua] Error 1
gmake[1]: *** Waiting for unfinished jobs....
gmake[1]: Leaving directory `/opt/virgo-0.1.9/out'
gmake: *** [all] Error 2
--- CUT ---

Installing GCC is not a proper solution. There is a reason why FreeBSD 10 release ditched GCC and there is a reason why you get whole pile of warnings while trying to install GCC on this distro from ports.

Add code to load a new monitoring.zip file and load it while running.

Config option to exit instead of loading new monitoring.zip

Currently there are two main challenges with agent checks and plugins

• There is no authoritative source other than the plugin-contrib repo
• We would like to build more agent checks but that means the agent will only become bigger

We should add a native agent check that inspects resource use by a program (1 or more processes).
It should:

1. Take a regex (or similar pattern) that can be used to match a command as a parameter to the check
2. The check should find all processes matching the pattern supplied, and aggregate metrics across them (ie, we don't report metrics for each individual process).
   In terms of metrics, we should ideally report at least CPU and memory use, as well as how many processes were aggregated.
   Lets try to make this work on Windows too.

This is just an idea at this point.

http://www.rackspace.com/knowledge_center/article/access-slow-query-and-general-logs-for-cloud-databases