Parameter to export raw chart data is missing from command line about nmonvisualizer HOT 13 CLOSED

Do you want to output the chart data so that you can recreate the exact same chart somewhere else?
Or, do you want to output the raw data for the entire parsed data set?

from nmonvisualizer.

I would like to use the tool for parsing, the visualization is secondary. I have to process the nmon created data. So, I want insert the whole data set into a database.

from nmonvisualizer.

The current commit has a --rawdata parameter in ReportGenerator. This will create a rawdata directory and output all the parsed files as CSV.

However, if you are doing something with the parsed data, it may be easier (and faster) to write your own code to go from DataSets directly to the database. Start by looking at subclassing NMONVisualizerApp. There's also an 'apijar' target in the Ant build file that will create a smaller API only jar file you can use to with the custom code.

from nmonvisualizer.

Thanks for the change.
There is a small problem:

• the field separator is comma, the decimal separator is comma, too. In this case, the values should be between "-s. (Or: How can I set the field separator/number format?)
• there should be the same amount of data in each record: my output had 6980 commas in the header, 587 in the first row, etc. The count must be the same for all rows. - ,, must written for no data.

from nmonvisualizer.

I would need the command line parameters to the separate processes. I think, the tool parses this info, too. How can I get this into a file? -> createMasterDetailProcessFile()

from nmonvisualizer.

There was a major bug in CSVWriter when outputting data if there was missing data for a type (i.e. a process). Hopefully this clears up all the issues you are seeing. The output should not include commas in the numbers even before this fix, but some of the fields were being concantenated, so that may be what you are seeing. Please confirm that I have fixed it.

NMONVisualizerCmdLine.createMasterDetailProcessFile(), that was a custom function for a one-off program. I think that if you really want to do custom processing, you should subclass NMONVisualizerApp, maybe using the CmdLine program as an example.

Regardless, I went ahead and added code so that the process data is also written out with the raw data. Look for _process.csv. You can associate the pid in that file with the pid that is output in the raw data file - see the header <process_name> () . Note that processes without pids are the aggregate data for all processes with the same name.

from nmonvisualizer.

Comment/fix request with minor severity: the csv-writing classes should have a static variable storing the field delimiter. (If you have ; instead of , ms excel can open the file automatically) This separator is currently hardcoded on many-many places.
Major bug:
The decimal separator is comma. - The same as my system setting. Anyway: this is not ok, the output is not to parse. See: http://stackoverflow.com/questions/8742645/decimalformat-depending-on-system-settings

I solved the problem:
CSVWriter.java[37]:
symbols.setNaN("");
symbols.setDecimalSeparator('.');

from nmonvisualizer.

If the value of a field contains " or , the output is not well formatted. To avoid this, please escape these characters.

from nmonvisualizer.

I completely forgot about I18LN issues and the decimal separator. In the latest commit, I forced the separator to '.' and also escape the strings when needed. Does that output a valid CSV for you?

If the CSV is valid, can you open it in Excel? This also seems to be a localization issue in that Excel uses different separators. If Excel does not work, then I can change CSVWriter to output a separator that is not ','.

from nmonvisualizer.

Great job, almost done. ;)
Rules of handling spec. characters in CSV:

• if the value contains spec. characters ( field separator or double quotes) ->
  a.) double double quotes
  b.) surround value with double quotes

Example:
a.)
original: I said: "this" (value contains double quotes)
expected: I said: """this""" (3 double qoutes)
(current output: I said: ""this"" )

b.)
original: I said it, Joe (value contains field separator)
expected: "I said it, Joe" (value is double quoted)

See: http://stackoverflow.com/questions/10451842/how-to-escape-comma-and-double-quote-at-same-time-for-csv-file

from nmonvisualizer.

It should be outputting 3 double quotes:

StringBuilder appendable= new StringBuilder(24);
String toEscape = ""test"";
toEscape = toEscape.replaceAll(""", """"");
appendable.append(""");
appendable.append(toEscape);
appendable.append(""");
System.out.println(appendable);

output is: """test"""

from nmonvisualizer.

Yes, it should be. ;)
But, the fact is:
Input row:
UARG,T0003,10092736, 1,perl_node, 1 , elink2, ipme2,/pkg/IPMRS_ENTW/elink2/EventLink/base/lib/perl_node -n "ERROR_ROUTER" -d -c /pkg/IPMRS_ENTW/elink2/EventLink/base/control/55/1185/control/4/config

Output in the processes.csv:
10092736,perl_node,2014-10-22,15:42:36,2014-10-22,15:43:36,""/pkg/IPMRS_ENTW/elink2/EventLink/base/lib/perl_node -n ""ERROR_ROUTER"" -d -c /pkg/IPMRS_ENTW/elink2/EventLink/base/control/55/1185/control/4/config""

Another thing: if I want only the csv in the output ( --nodata --nosummary --rawdata --chartdata ), it shoud be ok, (Current situation: error message: "--nodata and --nosummary were specifed but no custom chart definitions (-d or -a) were given")
If at least one output is given, it is ok)

Thanks.

from nmonvisualizer.

Are you sure that doesn't work in excel?

I put "/pkg/IPMRS_ENTW/elink2/EventLink/base/lib/perl_node -n ""ERROR_ROUTER"" -d -c /pkg/IPMRS_ENTW/elink2/EventLink/base/control/55/1185/control/4/config" into a CSV file via a text editor.
Then I open that file in Excel and it displays /pkg/IPMRS_ENTW/elink2/EventLink/base/lib/perl_node -n "ERROR_ROUTER" -d -c /pkg/IPMRS_ENTW/elink2/EventLink/base/control/55/1185/control/4/config
That's what I expect.

Regarding the command line flags, the latest commit should allow you to only get CSV output.

from nmonvisualizer.