3. Create Custom Signature¶

Signatures are what HAWK uses to process incoming logs to normalize the data. The following information will help you create new custom signature sets or edit existing ones.

The signatures get processed on the HAWK engines. HAWK eyeCon 5.0 comes with 71 signature rule sets. The default HAWK signatures can be found on the HAWK engine located at /usr/local/hawk/rules Any custom signature should be located in /usr/local/hawk/rules-local. Signature rule sets are broken out into individual files by vendor and vendor type.

Note

If you want to make changes to an existing HAWK default signature it should be moved into the rules-local directory. This is because on update the rules directory is overwritten with new changes.

First we need to define each of the key items that make up a signature rule set.

Naming Convention:¶

All signatures have a unified naming convention. logType_DeviceType_Vendor_VendorSpecific.hwk

Log Type: sl (syslog), or snmp (Simple Network Management Protocal)
Device Type: sw (switch), fw (firewall), ids (intrusion detection system), router (router), wap (wireless access point), etc
Vendor: Cisco, fortinet, juniper, mcafee, dell, etc
Vendor Specific: asa, pix, force10

Payload:¶

Payload is any log information sent to HAWK eyeCon.

Example Payload:

<86>Nov 14 00:38:46 192.168.2.3 sshd[30933]: pam_unix(sshd:session): session opened for user bob by (uid=0)

RuleName:¶

Name given to the signature rule set.

RuleDetails:¶

Details description of signature rule set.

RuleVersion:¶

Current version of signature rule set.

RuleKey:¶

Signature rule set key. Must be unique.

SNMPRule:¶

True or False this signature set is for SNMP trap payloads.

Host Classification:¶

If a payload matches the trigger and matches a rule the Host Classification is set. List of Host Classifications can be found here listed under os_type_name column.

Example HostClassification:

HostClassification="Linux Operating System"

Triggers:¶

Triggers are something unique in the log message that if matches it will process the payload using the rules in that file.

Example Triggers:

Trigger="<([0-9]{1,8})>(.*)[ ]+(.*)[ ]+([0-9]{1,2}):([0-9]{1,2}):([0-9]{1,2}) (.*) (.*)"
Trigger="<([0-9]{1,8})>([^ ]+):.*"
Trigger="([0-9]+) HAWKUnixLog:"

PreRules:¶

Pre-rules are to extract information that exist in every Payload. This includes, resource name, IP address source and destination, IP port number source and destination, correlation_username, etc.

Example PreRule:

BeginPreRule
        InfoMatch="HAWKRelay ([0-9]+\.[0-9]+\.[0-9]+\.[0-9]+)"
        Source="resource_name: $1;"
EndPreRule

RuleGroup:¶

Rule Groups are used to help optimize the signatures. If it matches the Rule Group it will then check each rule under its group.

Example RuleGroup:

BeginGroup
        GroupMatch="(login|ssh|authenticate|su|sudo)"
        BeginRule
                HID=""
                Alert=""
                Priority=""
                AlertType=""
                AlertIndex=""
                PayloadMatch=""
        EndRule
        {More Rules here}
EndGroup

Rule:¶

Rules if matched assign information to the event. Rules are made up of several fields.

Example Rule:

BeginRule
        HID="HID7113"
        Alert="Valid User Session Closed - SSH"
        Priority="3"
        AlertType="Attempted Authentication"
        AlertIndex="0"
        Ignore="True"
        PayloadMatch="sshd\[[\d]+\]: pam_unix.*session closed for user"
        InfoMatch="closed for user ([^ ]+)"
        Source="correlation_username: $1; audit_logoff: success;"
EndRule

HID:¶

HID is the HAWK ID. You can find a list of HAWK ID descriptions here .

Alert:¶

Alert is the name you want to give to this event. Typically this is the HAWK ID description.

Priority:¶

Priority can be 1 through 5. 1 being the highest Priority and 5 being the lowest Priority.

AlertType:¶

Alert Type is the type of event. You can find a list of Alert Types here .

AlertIndex:¶

AlertIndex is used to give rules order of process. If you have a catch all you want to have a AlertIndex=”10”. Normal rules should have an AlertIndex=”0”. Higher the index number the later it will get processed.

Ignore:¶

Ignore is a boolean value: True or False. If set to False (default) event will be instereted into the Data Tier. If set to True the event will be discarded and not instered into the Data Tier.

PayloadMatch:¶

Is a regular expression if it matches then this rule is applied to the event.

NotPayloadMatch:¶

Is a regular expression if matches then this rule is not applied to the event.

InfoMatch:¶

Is a regular expression if matches then you can pull information out of it and assign it to column key variables.

Source:¶

Source is where you match Column keys to values. You can find a list of Event Columns here .

RegexPal:¶

RegexPal.com Is an online tool you can use to test regular expressions to insure that it’s working as expected.

Overview:¶

A general overview of how the payload is processed by HAWK Signature rules.

3.1. Workflow for Existing Signatures¶

Obtain a payload that is not being parsed correctly by HAWK engines.

<86>Nov 14 00:38:46 192.168.2.3 sshd[30933]: pam_unix(sshd:session): session opened for user bob by (uid=0)

Determine the existing HAWK signature that should be edited. In this example this is a Linux SSH login so we will be working with sl_unix.hwk file.
Move the HAWK signature file from /usr/local/hawk/rules to /usr/local/hawk/rules-local.
Verify the trigger matches the payload using Regexpal.com.

Determine if an existing rule needs to be edited to match or if a new rule needs to be created.

5a. If an existing rule needs to be edited follow the following steps:

Verify the appropriate HID that matches the event. You can find a list of HAWK ID descriptions here

Using the HID description for the Alert.

Verify the appropriate AlertType. You can find a list of Alert Types here

Verify the AlertIndex equales “0” for a normal rule or “5” or “10” for a catch all.

Using regexpal.com verify PayloadMatch string matches log message.

If you need to pull information out of the log and assign it to HAWK columns use InfoMatch. Using regular expression set InfoMatch to match the string with specific information you are needing.

Using Source you can set columns to values or assigned variables from infomatch. In this example, We are setting column ip_dport with value 22, ip_proto column to value 6, audit_account_validation column with the value success, and correlation_username with the value from with-in the InfoMatch line.

Note

Anything in parentheses () is assigned to a number variable counting left to right.

BeginRule
       HID="HID7113"
       Alert="Valid User Session Opened - SSH"
       Priority="3"
       AlertType="Attempted Authentication"
       AlertIndex="0"
       PayloadMatch="sshd"
       PayloadMatch=": session opened for user"
       InfoMatch="sshd.* session opened for user ([^ ]+)"
       Source="ip_dport: 22; ip_proto: 6; correlation_username: $1; audit_account_validation: success;"
EndRule

5b. If a new rule needs to be created follow the following steps:

First you need to find the appropriate HID that matches the event. You can find a list of HAWK ID descriptions here

Using the HID description for the Alert.

Set the appropriate AlertType. You can find a list of Alert Types here

Set the AlertIndex to equal “0”.

Using regualar expression set PayloadMatch to match something unique within the log message.

If you need to pull information out of the log and assign it to HAWK columns use InfoMatch. Using regular expression set InfoMatch to match the string with specific information you are needing.

Using Source you can set columns to values or assigned variables from InfoMatch. In this example, We are setting column ip_dport with value 22, ip_proto column to value 6, audit_account_validation column with the value success, and correlation_username with the value from with-in the InfoMatch line.

Note

Anything in parentheses () is assigned to a number variable counting left to right.

BeginRule
       HID="HID7113"
       Alert="Valid User Session Opened - SSH"
       Priority="3"
       AlertType="Attempted Authentication"
       AlertIndex="0"
       PayloadMatch="sshd"
       PayloadMatch=": session opened for user"
       InfoMatch="sshd.* session opened for user ([^ ]+)"
       Source="ip_dport: 22; ip_proto: 6; correlation_username: $1; audit_account_validation: success;"
EndRule

Note

If you would like any changes you made to be included in the default HAWK signature sets please e-mail support@hawkdefense.com a copy of the signature set. After review they will be included in the next release.

3.2. Workflow for New Signatures¶

Obtain the payload(s) that are needed to be parsed by HAWK engines.

<189>Dec 19 19:21:13.546: switch: %NAME1-A %IFMGR-5-ASTATE_UP: Changed interface Admin state to up: Te 0/0
<189>Dec 19 19:21:13.546: switch: %NAME1-A %IFMGR-5-OSTATE_DN: Changed interface state to down: Te 0/46
<189>Dec 19 19:21:13.546: switch: %NAME1-A %IFMGR-5-OSTATE_UP: Changed interface state to up: Te 0/5

<189>Dec 19 19:21:13.546: switch: %NAME1-A %SEC-3-AUTHENTICATION_FAILURE: Authentication failure on vty0 (192.168.2.200) for method "local" user "root"
<189>Dec 19 19:21:13.546: switch: %NAME1-A %SEC-3-LOGIN_FAILURE: Login failure for user root on line vty0 ( 192.168.2.200 )
<189>Dec 19 19:21:13.546: switch: %NAME1-A %SEC-5-LOGIN_SUCCESS: Login successful for user Admin on line vty0 ( 192.168.15.140 )
<189>Dec 19 19:21:13.546: switch: %NAME1-A %SEC-5-LOGOUT: Exec session is terminated for user Admin on line vty0 ( 192.168.15.140 )
<189>Dec 19 19:21:13.546: switch: %NAME1-A %SEC-5-LOGOUT: Exec session is terminated on console
<189>Dec 19 19:21:13.546: switch: %NAME1-A %SEC-5-USER_ACC_CREATION_SUCCESS: User account "Admin" created or modified by default from console successfully

Create an HAWK signature file here /usr/local/hawk/rules-local. In this example, We are working with Dell Force10 switch. The name will be sl_sw_dell_force10.hwk
Enter the required information for RuleName, RuleDetails, RuleVersion, RuleKey, and SNMP.

RuleName="HAWK Syslog Dell Force10 Switch"
RuleDetails="HAWK Syslog Dell Force10 Switch compilation ruleset. Release Date: 12/26/2014"
RuleVersion="1.00"
RuleKey="DELL"
SNMPRule="False"

Enter the Host Classification.

HostClassification="Generic Switch"

Create a trigger that will match all the payloads using Regexpal.com. In this example, We will set Trigger=”^<[d]{1,8}>[^ ]+ [d]+ [d]+:[d]+:[d]+.[d]+: [^ ]+ %[^ ]+”

Create any necessary Pre-Rules. In this example, The resource name “%NAME1-A” is in the log. Using a Pre-Rule you can collect this information.

BeginPreRule
        InfoMatch="^<[\d]{1,8}>[^ ]+ [\d]+ [\d]+:[\d]+:[\d]+.[\d]+: [^ ]+ %([^ ]+) %[^ ]+"
        Source="resource_name: $1;"
EndPreRule

Note

Anything in parentheses () is assigned to a number variable. Counting left to right.

Create any necessary Rule Groups to optimize the signatures. In this example there are several %SEC and %IFMGR logs that could be grouped together.

BeginGroup
       GroupMatch="%SEC"
EndGroup

BeginGroup
       GroupMatch="%IFMGR"
EndGroup

The next step is to create the necessary rules. In this example we are looking at the following log:

<189>Dec 19 19:21:13.546: switch: %NAME1-A %SEC-3-AUTHENTICATION_FAILURE: Authentication failure on vty0 (192.168.2.200) for method "local" user "root"``

.

First you need to find the appropriate HID that matches the event. You can find a list of HAWK ID descriptions here

Using the HID description for the Alert.

Set the appropriate AlertType. You can find a list of Alert Types here

Set the AlertIndex to equal “0”.

Using regualar expression set PayloadMatch to match something unique within the log message.

If you need to pull information out of the log and assign it to HAWK columns use InfoMatch. Using regular expression set InfoMatch to match the string with specific information you are needing.

Using Source you can set columns to values or assigned variables from InfoMatch. In this example, We are setting audit_login column with the value failure, correlation_username with the value from with-in the InfoMatch line, and ip_src with the value from with-in the InfoMatch line.

Note

Anything in parentheses () is assigned to a number variable counting left to right. Also, even if you are escaping parentheses they still count. This is why $3 was used and not $2. $2 was assigned to the escaped parentheses.

BeginGroup
       GroupMatch="%SEC"
       BeginRule
               HID="HID7022"
               Alert="Attempted Authentication Failure"
               Priority="3"
               AlertType="Attempted Authentication"
               AlertIndex="0"
               PayloadMatch="%SEC-3-LOGIN_FAILURE:"
               InfoMatch="Login failure for user ([^ ]+) on line [^ ]+ \( ([\d]+\.[\d]+\.[\d]+\.[\d]+) \)"
               Source="audit_login: failure; correlation_username: $1; ip_src: $3;"
       EndRule
EndGroup

Continue to repeat step 6 until all the necessary rules have been added.
Finally, it may be necessary to add a catch all at the bottom to insure you collect all information for devices matching the trigger.

BeginRule
       HID="HID4200"
       Alert="Switch General Message"
       Priority="5"
       AlertType="Miscellaneous Information"
       AlertIndex="10"
       PayloadMatch=".*"
EndRule

The end result should look similar to the example below:

#
#       HAWK Network Defense, Inc.
#       Copyright 2007, All Rights Reserved
#       [email protected]
#
#

RuleName="HAWK Syslog Dell Force10 Switch"
RuleDetails="HAWK Syslog Dell Force10 Switch compilation ruleset. Release Date: 12/26/2014"
RuleVersion="1.00"
RuleKey="DELL"
SNMPRule="False"


HostClassification="Generic Switch"


Trigger="^<[\d]{1,8}>[^ ]+ [\d]+ [\d]+:[\d]+:[\d]+.[\d]+: [^ ]+ %[^ ]+"


BeginPreRule
        InfoMatch="^<[\d]{1,8}>[^ ]+ [\d]+ [\d]+:[\d]+:[\d]+.[\d]+: [^ ]+ %([^ ]+) %[^ ]+"
        Source="resource_name: $1;"
EndPreRule

BeginGroup
        GroupMatch="%SEC"
        BeginRule
                HID="HID4204"
                Alert="Switch Attempted Authentication via local"
                Priority="3"
                AlertType="Attempted Authentication"
                AlertIndex="0"
                PayloadMatch="%SEC-3-AUTHENTICATION_FAILURE:"
                InfoMatch="Authentication failure on [^ ]+ \(([\d]+\.[\d]+\.[\d]+\.[\d]+)\) for method "local" user "([^ ]+)""
                Source="audit_login: failure; ip_src: $2; correlation_username: $3;"
        EndRule

        BeginRule
                HID="HID7022"
                Alert="Attempted Authentication Failure"
                Priority="3"
                AlertType="Attempted Authentication"
                AlertIndex="0"
                PayloadMatch="%SEC-3-LOGIN_FAILURE:"
                InfoMatch="Login failure for user ([^ ]+) on line [^ ]+ \( ([\d]+\.[\d]+\.[\d]+\.[\d]+) \)"
                Source="audit_login: failure; correlation_username: $1; ip_src: $3;"
        EndRule

        BeginRule
                HID="HID7117"
                Alert="Attempted Authentication Success"
                Priority="4"
                AlertType="Attempted Authentication"
                AlertIndex="0"
                PayloadMatch="%SEC-[5|6]-LOGIN_SUCCESS:"
                InfoMatch="Login successful for user ([^ ]+) on line [^ ]+ \( ([\d]+\.[\d]+\.[\d]+\.[\d]+) \)"
                Source="audit_login: success; correlation_username: $1; ip_src: $3;"
        EndRule

        BeginRule
                HID="HID7157"
                Alert="Authentication Logoff Successful"
                Priority="4"
                AlertType="Attempted Authentication"
                AlertIndex="0"
                PayloadMatch="%SEC-[5|6]-LOGOUT:"
                InfoMatch="session is terminated for user ([^ ]+) on line [^ ]+ \( ([\d]+\.[\d]+\.[\d]+\.[\d]+) \)"
                Source="audit_logoff: success; correlation_username: $1; ip_src: $3;"
        EndRule

        BeginRule
                HID="HID7157"
                Alert="Authentication Logoff Successful"
                Priority="4"
                AlertType="Attempted Authentication"
                AlertIndex="0"
                PayloadMatch="%SEC-5-LOGOUT:"
                InfoMatch="session is terminated on console"
                Source="audit_logoff: success;"
        EndRule

        BeginRule
                HID="HID4205"
                Alert="Switch User Created/modified"
                Priority="4"
                AlertType="Miscellaneous Information"
                AlertIndex="0"
                PayloadMatch="%SEC-5-USER_ACC_CREATION_SUCCESS:"
                InfoMatch="User account "([^ ]+)" created or modified by [^ ]+ from [^ ]+ successfully"
                Source="audit_user_change: success; correlation_username: $1;"
        EndRule
EndGroup


BeginGroup
        GroupMatch="%IFMGR"
        BeginRule
                HID="HID4201"
                Alert="Switch Detected Interface Changed State to Down"
                Priority="4"
                AlertType="Miscellaneous Information"
                AlertIndex="0"
                PayloadMatch="interface state to down"
        EndRule

        BeginRule
                HID="HID4202"
                Alert="Switch Detected Interface Changed State to Up"
                Priority="4"
                AlertType="Miscellaneous Information"
                AlertIndex="0"
                PayloadMatch="interface state to up"
        EndRule

        BeginRule
                HID="HID4201"
                Alert="Switch Detected Interface Changed State to Down"
                Priority="4"
                AlertType="Miscellaneous Information"
                AlertIndex="0"
                PayloadMatch="interface [^ ]+ state to down"
        EndRule

        BeginRule
                HID="HID4202"
                Alert="Switch Detected Interface Changed State to Up"
                Priority="4"
                AlertType="Miscellaneous Information"
                AlertIndex="0"
                PayloadMatch="interface [^ ]+ state to up"
        EndRule
EndGroup

BeginRule
        HID="HID4200"
        Alert="Switch General Message"
        Priority="5"
        AlertType="Miscellaneous Information"
        AlertIndex="10"
        PayloadMatch=".*"
EndRule

Note

If you would like any changes you made to be included in the default HAWK signature sets please e-mail support@hawkdefense.com a copy of the signature set. After review they will be included in the next release.