Posemo Developer Manual: Output Modules

This manual is about writing output modules (frontend connectors) for Posemo.

The default output module just gives a JSON data structure. For other build-in output modules see the modules in lib/PostgreSQL/SecureMonitoring/Output.

Overview

Posemo can write output to any frontend. The default output is a JSON file with all check results including critical/warning, some global meta data, some meta data for each host and some meta data (e.g. how to display graphs, names and titles) per check.

Some fontends like check_mk need some extra (global) "metrics" configuration, which depend on the Checks, which can not included in the result. This may be infos about titles, graphs, a list of all checks, … this infos may be generated too by the output module.

Output Modules

Output Modules are Moose Roles which must implement at least one method: generate_output. The role will be consumed by PostgreSQL::SecureMonitoring::Run (or maybe any subclass).

Output Modules may have any number of Moose attributes, which a user can use as command line options.

The generate_output should set the output attribute (a string); it can be set directly or via adding content.

# Set everything
$self->output( $complete_output_string );

# Or add something (default: empty string, so you can add from begin):
$self->add_output( $partly_output_string );

Posemo gives you helpers for looping over all results or hosts. Usually you should use the buildin method and only write your needed callbacks, instead if looping by yourself. See "Iterate over all results" below.

Global Metrics or Graph Info

Some implementations (e.g. check_mk) need some global metrics or graph info file.

You should may collect all infos in the generate_output method inside your output module and write some extra files. See CheckMk.pm as example.

Simple Example

The output code may be simple or complex. In the simplest form, it converts the result data structure into another format and returns it. Here is the variant for JSON, the default output module:

has pretty => ( is => "ro", isa => "Bool", default => 0, );

sub generate_output
   {
   my $self            = shift;
   my $complete_result = shift;

   my $json = JSON->new->pretty( $self->pretty );
   $self->output( $json->encode( $complete_result) );

   return;
   }

This defines a new attibute pretty, which can be set via command line. It is a flag, which is forwarded to the JSON converter. Then the JSON encoder is called, that's all.

The Result

The result is consists of three parts: global meta data, host meta data, check results for each host.

It is one big data structure and as usual, hash keys are in random order and very likely not in the order of the following example.

Example data structure

{
message        => 'PostgreSQL Secure Monitoring version v0.6.0, running on host Chefkoch-plus.local at Fri Jun  1 16:40:14 2018'
posemo_version => 'v0.6.0',
runtime        => '0.109699878692627',
hostname       => 'Chefkoch-plus.local',
error_count    => 0,
configfile     => 't/conf/simple.conf',
global_id      => 'Simple test',
result         => [
                  {
                   host      => 'localhost',
                   name      => 'localhost',
                   hostgroup => '_GLOBAL',
                   database  => '_posemo_tests',
                   user      => '_posemo_tests'
                   port      => '15432',
                   results   => [
                                 {
                                   check_name     => 'Writeable',
                                   description    => 'Database is writeable before timeout'
                                   result_unit    => 's',
                                   return_type    => 'bool',
                                   result_type    => 'float',
                                   columns        => [ 'writeable' ],
                                   status         => 0,
                                   row_type       => 'single',
                                   result         => 0.00235891342163086,
                                   warning_level  => 3,
                                   critical_level => 5,
                                 },
                                 # […] more checks
                                ],
                   },
                   # […] more hosts
               ], # End result list
}

Global meta data

The global meta data contains the following values as outer key/value pairs in the result hash. An output module may use these values for any purpose.

message

A string with a human readable short message.
posemo_version

Version information, taken from $VERSION in PostgreSQL::SecureMonitoring.
runtime

The complete runtime (excluding Perl startup time) in seconds.
hostname

The hostname of the machine where Posemo is running.
error_count

Number of (hard) errors, e.g. connection problems. Should be 0.

A hard error occurs when a check dies. This can happen when a check dies, e.g. when it can't get a connection to the database.
configfile

Path to the config file.
global_id

global_id from the config file. An optional ID which can be set per config file.
result

Not a global meta data, but on the same level: ArrayRef with results for all hosts. See below.

Host meta data

The value of the result key in the first level is an array reference of hash references, each element of the array represents one host.

Most values are informal values and may be used by a frontend to display some (additional) infos. The most important value is the name, which should be used to identify the server.

host

Hostname or IP address of the destination host. This address was used for connecting and is given by the config file. Output modules usually should use the name (see next item) to identify the machine.
name

Name of the host. Given by the config file and may be any string. Default (when not given in the config): the same as host.

The administrator may use the name to specify which host was meant, e.g. when the connection address is different from the name usually used by the host of if the connecting host is an IP address.

Therefore: output modules really should use the name and not host to identify the machine.
hostgroup

An administrator may group several servers to host groups (see config manual for details). The value for this key contains the name of the host group. Or _GLOBAL, when no no host group is used (for this host).

Output modules can use this for grouping hosts too.
database

Name of the connected database.
user

Name of the connecting user.
port

The port used for connection.
results

ArrayRef with results for each check configured for this host. See below.

Check results

The results key for each host (see above) contains an ArrayRef of HashRefs with the results for each check for this host.

Each check gives some informational values, some information about the result type, information about how to display the result and other things configured by the check module itself. You find additional information about each key in the documentation for writing check modules.

Non optional values

check_name

The name of the check. Default: derived from the class name, but may be changed by the check itself.

May be used as headline.
description

A desciption of the check, given by the check.

May be used as (sub) headline.
row_type

The row type of the result, depending on the return value and the has_multiline_result attribute of the check module.

Possible values:
- single
  
  A single result value. Usually a number or boolean flag. result contains only one single value. As example see the checks SlaveLag, AllocatedBuffers or Writeable.
- list
  
  The result is a list of values. Therefore the key result contains a array reference. As example see CheckpointTime.
- multiline
  
  A complex result, the SQL function returns multiple lines. Therefore the key result contains a array reference of array references.
  
  Usually each first column contains a title like a database name or table name. The first row usually contains a column with the title !TOTAL and summary values of all other rows.
  
  As example see the checks DBSize, Transactions or Activity.
result

The actual result value(s), according to row_type. These are raw values directly from PostgreSQL. A frontend module must distinguish all row_types and use the given values. There may be some hints how to display the values in graphs (see keywords below).

Usually a frontend module should display a multiline result in one graph per row; and usually the best way is to display all on one page, but not all frontends can handle a dynamic number of graphs per service.

When there is a hard error, the result is undef.
result_type

Data type of the result; as SQL types.

Per default it is the same as return_type.

When the row_type is multiline or list, then usually all values should have this type, but in multiline the first column is a name (of a database or table or something else).
return_type

The SQL return type, defined by the check. Usually a frontend module should check the result_type instead.
result_unit

The unit of the result. Default: empty.

At the moment, there is no complete list of result units. The buildin checks use the following units:
```
s   seconds
ms  milliseconds
%   percent
```
This is work in progress and new units may be added or the existing ones may change etc.

An output module should use a lookup table or hash to rewrite it to the used frontend monitoring system.
columns

The value of this key containt an arrayref with a list of the names for each column.

The names are directly taken from the SQL result and depend on the individual check. They may be used as name/description for each graph element in the output.

status

The warning/critical status. May contain the following values:

0: STATUS_OK        Everything is OK
1: STATUS_WARNING   Warning level reached
2: STATUS_CRITICAL  Critical level reached
3: STATUS_UNKNOWN   unknown
 
 The names above are constants in the C<PostgreSQL::SecureMonitoring::Checks> Module, which can be importet:
 
 use PostgreSQL::SecureMonitoring::Checks qw(:status);

Values about thresholds and errors (optional)

The following keys may exist in the result, depending on the check and its result, and are about warning/critical thresholds or errors.

warning_level

Check configuration: a numerical value for the warning level. From config file or a default may be set by the check.

critical_level

Check configuration: a numerical value for the critical level. From config file or a default may be set by the check.

lower_is_worse

Check configuration. Hint, if a lower value is worse than a higher value.

e.g. a lower cache hit ratio is worse than a higher one.

critical

Flag: result has reached critical level. (0: false; 1: true)

warning

Flag: result has reached warning level. (0: false; 1: true)

message

String with an optional message. When warning/critical level is reached, there is usually a message.

error

String with error message when there was a fatal error (e.g. connection problem).

In this case, the state and result values are unknown.

Values for graphs and display (optional)

The following keys may exist in the result, depending on the check and its configuration.

min_value and max_value

Configured by the check itself or via config file.

Usable as a hint for the displaying frontend, e.g. for percent values, these contain 0 and 100.

Default: none.

result_is_counter

Configured by the check itself.

A flag indicating if the result is a counter, i.e. an (ever) raising value, like accumulated time or I/O. The frontend should display the rate by timerange (usually seconds).

Default: off/disabled.

graph_type

A check module can define a graph type for the frontend. Valid values are: line, area, stacked_area.

Output modules should handle this, forward it to the frontend.

graph_mirrored

A flag, indicating that the graph should be mirrored at the null level. Usually this is used for input/output graphs or similar. For instance, it is used for committed/rolled back transactions in the Transactions check.

Iterate over all results

If you must touch the results to perform some work (e.g. collect each single result or host in a flat list), use the following. It may help a lot to write output modules. You should not lopp over all hosts or results by yourself, you should call the method and write some callbacks.

You can call the loop_over_all_results method to loop over all single results. This iterates over all results and calls methods in your output module for each host, check and single result.

Call it as:

$self->loop_over_all_results($complete_result);

where $complete_result is the result (input) parameter of the output method.

This calls the following methods in your module, if it exists:

for_each_host          ($host_result)
for_each_check         ($host_result, $check_result)
for_each_row_result    ($host_result, $check_result, $row)
for_each_single_result ($host_result, $check_result, $single_result)

TODO

TODO: More Documentation here. You may look at the CheckMK output module for an example, or in Run.pm for the code!

Recommendations for frontends

TODO.

...

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Posemo Developer Manual: Output Modules

Overview

Output Modules

Global Metrics or Graph Info

Simple Example

The Result

Example data structure

Global meta data

Host meta data

Check results

Non optional values

Values about thresholds and errors (optional)

Values for graphs and display (optional)

Iterate over all results

TODO

Recommendations for frontends

FilesExpand file tree

OutputModules.pod

Latest commit

History

OutputModules.pod

File metadata and controls

Posemo Developer Manual: Output Modules

Overview

Output Modules

Global Metrics or Graph Info

Simple Example

The Result

Example data structure

Global meta data

Host meta data

Check results

Non optional values

Values about thresholds and errors (optional)

Values for graphs and display (optional)

Iterate over all results

TODO

Recommendations for frontends