Anomaly detection Engine for Linux Logs (ADE)

Utilities to support different anomaly scorers and new anomaly scorers - utilities

utilities is a bash script which invokes the correct Java class after setting up the environment needed. utilities provides a request to

check if a flowlayout.xml has the correct format.
print out a textual description of a specific model. The textual description contains a summary of statistical information captured during train.

ADE is shipped with a flowlayout.xml file that has been tuned to work for most enterprise Linux systems. Use these request if you make changes to the flowlayout.xml. You might choose to change the flowlayout.xml to

to tailor the existing scorers to more closely match the type of system you are analyzing
to apply new statistical techniques to detect differ types of anomalies
to change the Java class(s) that are used to write ADE results to a database, a file in the file system, ....

Usage

Use utilities when defining a different collection of scores, different parameters for the scorers, or a different order to apply the scorers. utilities provides the following requests

return to standard out the version of the ADE code
return to standard out the version of the database schema that is expected by the ADE code
check the syntax of a flowlayout.xml file
writes to a file a textual description of the statistical information used to create a model

Notes

The flowlayout.xml controls how upload, analyze, and train process Linux logs. The following behavior of ADE is controled by flowlayout.xml:

The time duration which is used (interval)

The statistical methods that are used to determine if a message is anomalous and any parameters which the statistical method needs

The order that statistical methods are applied and how the results of different statistical methods are combined

The statistical methods that are used to determine if the collection of messages within an interval are unusual

The Java class(s) which are used to write out the results of ADE to either a database, to a file in the file system, ....

For simplicity within ADE the contents of flowlayout.xml is called the meta model.

ADE does not need to be connected to a JDBC complaint database for the following requests to be processed:

return the version of ADE code

return the version of the database schema

check the synatx of a flowlayout.xml file

Options

Option	Parameter	Request type
-b	none	Print version of database schema used by ADE code
-d	name of model file (.bin)	Print textual summary of model contained in .bin ilfe
-f	name of flowlayout.xml	Verify syntax of flowlayout.xml and print out results
-h	none	Print out help information
-o	name of output file	Name of file to contain textual description of model
-v	none	Print version of the ADE code

Restrictions

The version of the ADE code being used to print out the textual information about the model must match the version of the ADE code used to create the model.

ADE