7. Event Collection and Correlation Engine¶

An engine Service Account user must be defined before proceeding with this section. Please see User Management on steps creating a user. Make sure this user has all Access Control Rights enabled.

The following commands are needed to install the HAWK engine:

user@host:# yum install hawkcore hawk-ece hawk-ece-rules

7.1. Configure hawk-analyticsd¶

Edit hawk-analyticsd.cfg file.

#!HAWK
# Hawk SysLog Configuration File
# .sample hawk-analyticsd.cfg file
#

# Unique Name
HawkName="HAWK5-ECE-01"

# Authenticated Data Store
#HAWKUrl="https://admin:password@hawk5-server3:8080/API/1.1"
HAWKUrl="https://admin:password@hawk5-server3:8080/API/1.1"

# SSL Configuration
# Toggle SSL Peer Verification
HTTPSSLVerifyPeer="False"
HTTPSSLVerifyHost="False"

BalancerHost="tcp://127.0.0.1:40010"

# Memcache atomic counter configuration
MemcacheConfig="127.0.0.1:11211"

# User Definition
User="root"
Group="root"

# Number of threads to be used for normalization
NormalizationThreadCount=4

### Queue Configuration

#WriteToDiskCompression="False"

# Maximum amount of queue threads we want to startup
QueueThreadCount=4

# Maximum amount of time event statistics should be tracked
EventCacheTimeOut=28800

# Enable DNS Resolution (slower insertion)
EnableDNS="True"

GeoIPFile="/opt/hawk/etc/GeoLiteCity.dat"

#
# Verbosity - Verbosity of our Engine
#       0 - Only log errors, and warning
#       1 - include the above plus information
#       2 - include the above plus debugging
#
Verbosity="1"

# LogSource
#       LOGFILE
#       SYSLOG
LogSource="Logfile"
LogFile="/var/log/hawk/hawk-analyticsd.log"


EnableAggregation="True"

AggregationRule="alert_name, ip_src, ip_dst, ip_proto, ip_dport, correlation_username, target_username, audit_login"

AggregationTimeWindow="5"

CacheStoreDb="/opt/hawk/analytics"

# Format:
# map, replace: index_field[,...]

# Map Process ID to Application Name
CacheMapReplace="pid, app: group_name, resource_addr"

# Map Dport to Application name, inherit and update if possible
CacheMapReplace="ip_dport, app: group_name, ip_dst"

HAWKName¶: Specifies a unique name for the engine, which is used for access control, as well as scalability and availability. Example value: HAWK-ENGINE-01 where the unique name will be seen in the HAWK Resource Manager
HAWKUrl:¶: Connection string to connect to HAWK API. Username and Password should be the service account you setup in the HAWK UX. Also, included the IP address or Hostname of the HAWK API.
HTTPSSLVerifyPeer:¶: Set to False if using a self-signed SSL certificate.
HTTPSSLVerifyHost:¶: Set to False if using a self-signed SSL certificate.
BalancerHost:¶: Enter the IP address or Hostname where hawk-balancerd is installed. If leveraging a clustered environment list each one on a separate line.
MemcacheConfig:¶: Enter the Memcached server IP address or Hostname. If using a memcached pool separate IP addresses or Hostnames by a comma ‘,’.
User:¶: System user hawk-analyticsd should run as.
Group:¶: System group hawk-analyticsd should run as.
NormalizationThreadCount:¶: Specifies the number of parallel threads to normalize events.
WriteToDiskCompression:¶: Enable compression on local disk storage of JSON file archive. Default is True
QueueThreadCount:¶: Specifies the number of parallel threads to correlate, score, and write the events to the specified datastores.
EventCacheTimeOut:¶: Specify the maximum amount of time event statistics should be tracked (in seconds). Default is 20
EnableDNS:¶: Specify whether or not to perform DNS lookup requests during the event storage process. This has the potential to significantly degrate write performance. Default: True
GeoIPFile:¶: Provide the location to our localized GeoIP lookup dataset.
Verbosity:¶: Provide the requested verbosity threshold to increase or decrease the volume of log output.
LogSource:¶: Specify the logging output option, Examples: LogFile or Syslog are both available options
LogFile:¶: Specify the destination output log file for logging if LogSource has been specified as LogFile.
EnableAggregation:¶: Specifies whether or not to enable data aggregation support.
AggregationRule:¶: Each AggregationRule specifies a strict record search for the fields provided. If one of the fields does not exist, the aggregation rule will be skipped.
AggregationTimeWindow:¶: Specifies in the number of seconds, how long to aggregate the events from the time of arrival, until the time for correlation.
CacheStoreDb:¶: Local database to be used for hawk-analyticsd cache.
CacheMapReplace:¶: Rules to cache event information to be used to perform document enrichment.
HTTPProxy:¶: To communicate with the data tier via a Proxy. i.e. (HTTPProxy=”Proxy_server:8080”)

Note

To get more detailed information about hawk-analyticsd. user@host:# man hawk-analyticsd.cfg

7.2. Configure hawk-balancerd¶

Note

Typically its not required to change any default settings for hawk-balancerd.

Edit the hawk-balancerd configuration file.

user@host#: vi /opt/hawk/etc/hawk-balancerd.cfg

#!HAWK
#
# Hawk Balancer Configuration File
# .sample hawk-balancer.cfg file
#

# User Definition
User="root"
Group="root"

# Local ip and port to bind to for receiving messages for brokering
BindHost="tcp://0.0.0.0:40010"

#
# Verbosity - Verbosity of our Engine
#       0 - Only log errors, and warning
#       1 - include the above plus information
#       2 - include the above plus debugging
#
Verbosity="1"

# LogSource
#       LOGFILE
#       SYSLOG
LogSource="Logfile"
LogFile="/var/log/hawk/hawk-balancerd.log"

User:¶: System user hawk-eventd should run as.
Group:¶: System group hawk-eventd should run as.
BindHost:¶: IP address or Hostname where hawk-balancerd should bind to.
Verbosity:¶: Provide the requested verbosity threshold to increase or decrease the volume of log output.
LogSource:¶: Specify the logging output option, Examples: LogFile or Syslog are both available options
LogFile:¶: Specify the destination output log file for logging if LogSource has been specified as LogFile.

7.3. Configure hawk-pulsed¶

Edit the hawk-pulsed configuration file.

user@host#: vi /opt/hawk/etc/hawk-pulsed.cfg

#!HAWK
#
# Hawk Pulse Configuration File
# .sample hawk-pulsed.cfg file
#

# Unique Name
HawkName="HAWK-ECE-01"

# Authenticated Data Store
HAWKUrl="https://username:password@server1:8080/API/1.1"

# SSL Configuration
# Toggle SSL Peer Verification
HTTPSSLVerifyPeer="False"
HTTPSSLVerifyHost="False"

HTTPCredentialSecret="example-secret"
HTTPCredentialSecret="example-secret"

# Hosts responsible for saving our results
ForwardHost="tcp://127.0.0.1:40010"

# User Definition
# So we're not running as root
User="root"
Group="root"

#
# Resource Configuration
#
# Resource Threads for processing each resource.
ResourceThreadCount=3
# Polling Timeout in seconds
ResourcePollTimeout=5

#
# Verbosity - Verbosity of the Hawk Engine
#       0 - Only log errors, and warning
#       1 - include the above plus information
#       2 - include the above plus debugging

Verbosity="1"

#
LogSource="LogFile"
LogFile="/var/log/hawk/hawk-pulsed.log"

HAWKName¶: Specifies a unique name for the engine, which is used for access control, as well as scalability and availability. Example value: HAWK-ENGINE-01 where the unique name will be seen in the HAWK Resource Manager
HAWKUrl:¶: Connection string to connect to HAWK API. Username and Password should be the service account you setup in the HAWK UX. Also, included the IP address or Hostname of the HAWK API.
HTTPSSLVerifyPeer:¶: Set to False if using a self-signed SSL certificate.
HTTPSSLVerifyHost:¶: Set to False if using a self-signed SSL certificate.
HTTPCredentialSecret:¶: Specifies the pre-determined shared key secret used for decrypting credentials from the API.
ForwardHost:¶: Enter the Local IP address or Hostname of hawk-balancerd.
User:¶: System user hawk-eventd should run as.
Group:¶: System group hawk-eventd should run as.
ResourceThreadCount:¶: Specifies the number of parallel threads for processing each resource.
ResourcePollTimeout:¶: Specifies the timeout in seconds for polling data from each resource.
Verbosity:¶: Provide the requested verbosity threshold to increase or decrease the volume of log output.
LogSource:¶: Specify the logging output option, Examples: LogFile or Syslog are both available options
LogFile:¶: Specify the destination output log file for logging if LogSource has been specified as LogFile.
HTTPProxy:¶: To communicate with the data tier via a Proxy. i.e. (HTTPProxy=”Proxy_server:8080”)

Note

To get more detailed information about hawk-analyticsd. user@host:# man hawk-pulsed.cfg

7.4. Configure hawk-syslogd¶

Note

Typically its not required to change any default settings for hawk-syslogd.

Edit the hawk-syslogd configuration file.

user@host#: vi /opt/hawk/etc/hawk-syslogd.cfg

#!HAWK
#
# Hawk SysLog Configuration File
# .sample hcslogd.cfg file
#

# User Definition
User="root"
Group="root"

# Syslog Configuration
LogHost="udp://0.0.0.0:514"
LogHost="tcp://0.0.0.0:514"
LogHost="ssl://0.0.0.0:8514"

# Hosts responsible for saving our results
ForwardHost="tcp://127.0.0.1:40010"

# SSL Configuration
SSLLease="1024"
SSLCrt="/opt/hawk/etc/ssl.crt"
SSLCsr="/opt/hawk/etc/ssl.csr"
SSLKey="/opt/hawk/etc/ssl.key"
SSLSecret="example-secret"
SSLSubject="CN=hawkdefense.com/O=HAWK Network Defense, Inc./C=US/ST=TX/L=Dallas"

#
# Verbosity - Verbosity of our Engine
#       0 - Only log errors, and warning
#       1 - include the above plus information
#       2 - include the above plus debugging
#
Verbosity="1"
# LogSource
#       LOGFILE
#       SYSLOG
LogSource="Logfile"
LogFile="/var/log/hawk/hawk-syslogd.log"

User:¶: System user hawk-eventd should run as.
Group:¶: System group hawk-eventd should run as.
LogHost:¶: Specifies parameters for binding on specific addresses and protocols for syslog event feeds.
ForwardHost:¶: Enter the Local IP address or Hostname of hawk-balancerd.
Verbosity:¶: Provide the requested verbosity threshold to increase or decrease the volume of log output.
SSLLease:¶: Specify the amount of days our generated SSL certificate is valid for.
SSLCrt:¶: Specify the destination to store our SSL certificate file.
SSLCsr:¶: Specify the destination to store our SSL certificate request.
SSLKey:¶: Specify the destination to store our SSL keyfile.
SSLSecret:¶: Specify the SSL passphrase used for generating the SSL certificate.
SSLSubject:¶: Specify the SSL certificate parameters for certificate generation.
LogSource:¶: Specify the logging output option, Examples: LogFile or Syslog are both available options
LogFile:¶: Specify the destination output log file for logging if LogSource has been specified as LogFile.

Note

To get more detailed information about hawk-analyticsd. user@host:# man hawk-syslogd.cfg

7.5. Configure hawk-eventd¶

hawk-eventd should be configured any location hawk-data (API) is configured.

Edit the hawk-eventd configuration file.

user@host#: vi /opt/hawk/etc/hawk-eventd.cfg

#!HAWK
#
# Hawk Event Daemon Configuration File
User="root"
Group="root"

HAWKUrl="https://admin:password@hawk5-server3:8080/API/1.1"

# Toggle SSL Peer Verification
HTTPSSLVerifyPeer="False"
HTTPSSLVerifyHost="False"

Mode="HTTP"

LogDirectory="/opt/hawk/events"
# System Configuration
QueueThreadCount=4
Verbosity="1"

User:¶: System user hawk-eventd should run as.
Group:¶: System group hawk-eventd should run as.
HAWKUrl:¶: Connection string to connect to HAWK API. Username and Password should be the service account you setup in the HAWK UX. Also, included the IP address or Hostname of the HAWK API.
HTTPSSLVerifyPeer:¶: Set to False if using a self-signed SSL certificate.
HTTPSSLVerifyHost:¶: Set to False if using a self-signed SSL certificate.
Mode:¶: Enter the mode hawk-eventd should run as. On data tier mode should be set to “Messages”. If on engine tier mode should be set to “HTTP”
LogDirectory:¶: Location of event files saved by hawk-analyticsd.
QueueThreadCount:¶: Amount of threads to be used to process event files.
Verbosity:¶: Provide the requested verbosity threshold to increase or decrease the volume of log output.
HTTPProxy:¶: To communicate with the data tier via a Proxy. i.e. (HTTPProxy=”Proxy_server:8080”)

7.6. Restarting Services¶

After you are done configuring the HAWK engine the services must be restarted for the changes to take effect.

user@host:# service hawk-balancerd restart

user@host:# service hawk-analyticsd restart

user@host:# service hawk-pulsed restart

user@host:# service hawk-syslogd restart

user@host:# service hawk-eventd restart