Inter-Bro Communication 0.1



Inter-Bro Communications
Using built in functionality, it is possible to have multiple bro instances exchange event information with one another and to allow for non-bro clients to interact with the system as well.  Some of this is described elsewhere.  Here we will describe how to configure cross bro communication both in terms of clear text and ssl communication.

The basic functionality is controlled by the policy files remote.bro, listen-clear.bro and listen-ssl.bro.  It is possible (and recommended) not to change these files, but to use created policy files to maintain a local configuration. It is important to realize that the design of the protocol is not strictly assumed to be in the traditional client-server model.  While this is the most familiar design pattern typically used, it is not necessary in that each side can act as both server and client.

The role of each of the before mentioned files is as follows:

remote.bro:
This file is the primary configuration mechanism and is used to define which bros the current instance should connect to and which events should be asked for or allowed to be shared.  It also holds the definition of which ports should be used for ssl and clear text communications.  These are defined in:

redef Remote::destinations += {
     ["host_10"] = [$host = 10.10.10.10, $events = /.*/, $connect=T, $retry = 10 secs, $ssl=T],
};

const default_port_ssl = 47756/tcp &redef;
const default_port_clear = 47757/tcp &redef;
 
The fields described in the destinations section are as follows:

host: the host that will be connected to, or will be connecting from depending on the 'connect' flag.

events: a regular expression defining the types of events that will be exchanged.

connect: a directional flag, indicating if you will be connecting to an external host and requesting events ("T"), or that you expect to recieve requests ("F").

retry: the delay between reconnection attempts if the connection is broken.

ssl: flag for decifding if ssl should be used or not.  See below for setting up ssl options.


In general the side connecting to and asking for events will be configured with this, while the listening side of the conversation not filter outgoing events.  If the environmental requirements support the need to filter on the receving end, that it can be done but determining configuration errors is made more complicated.

listen-ssl.bro:
This file contains information used for authentication and encryption for an ssl enabled connection.  It is highly suggested that this mechanism be used in any non-test system since the placement of non-authenticated nodes within the fabric could cause significant problems both in terms of stability and intergity of the overall mechanism.  The location of the bro instance and CA public certificates is defined in bro.init via empty placeholders, but needs to be set up in the local policy before it can be used.  If the host that is runing the bro instance already has a properly installed and configured Broccoli library, you may just want to use those cretificates unless local policy.  The most significant things that need configuring are:

redef ssl_ca_certificate = "/bro/etc/ca_cert.pem";
redef ssl_private_key = "/bro/etc/bro.pem";
redef ssl_passphrase = "my d0g has Fl33z";

The ssl passphrase need not be entered here, but you will have to be around to type it in on bro restart if you do not.

listen-clear.bro:
There is not much to configure from here, except the IP address that the listening socket is bound to, which is defined in:

const listen_if_clear = 0.0.0.0 &redef;


Sample Configuration 1:
Here is an example used for connecting Broccoli instrumented sshd servers to a centeral collector.  The Broccoli needs to be appropriatly configured - see Christian's page for details on the sshd side,  and the  bro ought to be configured like:

# To prevent requesting sshd events from any peering Bro that connects,
# here is a list of our sshds. List the IP addresses of the hosts your
# sshds are running on here:
#
redef Remote::destinations += {
        ["sshd1"] = [$host = 10.11.12.13, $events = /sensor_sshd.*/, $connect=F, $ssl=T],
};

If the ssl/clear text port is on a non-standard number, it can be changed via default_port_ssl or default_port_clear .

Sample Configuration 2:
Two bro instances, bro1 and bro2.  Bro1 will be treated as a sensor, and bro2 will be treated like a collector.  All communication will be in the clear since on this network we re all friends...

Bro1: sensor for ftp based events
@load listen-clear.bro, optionally setting the default_port_clear if there is a collision with some other pre-existing service

Bro2: collector for ftp based events
@load remote.bro with the following configuration for Remote::destinations

# connect to ftp sensor and gather the following events:
#
# ftp_unexpected_conn_violation
# ftp_sensitive_file
# ftp_reply
# ftp_request
# ftp_unexpected_conn
# ftp_binary_response
# ftp_excessive_filename
#
redef Remote::destinations += {
        ["ftp-sensor"] = [$host = 10.11.12.14, $events = /ftp_*/, $connect=F, $ssl=T],
};

Debugging
The default location for most connection information is comm.log and remote.log.  A sample connection is analyzed here for a complete transfer of event information between sensor and analyzer:

Sensor:
comm.log:

1126645547 (parent) communication started, parent pid is 73482, child pid is 73483
1126645548 (child)  listening on 0.0.0.0:44332 (ssl)
1126645588 (child)  [#10000/10.100.101.102:4241] accepted SSL connection
1126645588 (parent) [#10000/10.100.101.102:4241] added peer
1126645588 (parent) [#10000/10.100.101.102:4241] peer connected
1126645588 (parent) [#10000/10.100.101.102:4241] phase: version
1126645588 (parent) [#10000/10.100.101.102:4241] phase: handshake
1126645588 (parent) [#10000/10.100.101.102:4241] registered for event ftp_unexpected_conn_violation
1126645588 (parent) [#10000/10.100.101.102:4241] registered for event software_version_change
1126645588 (parent) [#10000/10.100.101.102:4241] registered for event login_input_line
1126645588 (parent) [#10000/10.100.101.102:4241] registered for event net_weird
1126645588 (parent) [#10000/10.100.101.102:4241] registered for event software_unparsed_version_found
(...)
1126645588 (parent) [#10000/10.100.101.102:4241] registered for event pm_attempt_set
1126645588 (parent) [#10000/10.100.101.102:4241] request for unknown event login_fail_pair
1126645588 (parent) [#10000/10.100.101.102:4241] phase: sync
1126645588 (parent) [#10000/10.100.101.102:4241] sending full state
1126645588 (parent) [#10000/10.100.101.102:4241] done sending full state
1126645588 (parent) [#10000/10.100.101.102:4241] phase: running
1126645607 (parent) parent statistics: pending=0 bytes=2K/402K chunks=12/2960 io=5/913 bytes/io=0.43K/0.44K events=0/1476
1126645607 (parent) child statistics: [0] pending=0 bytes=0K/0K chunks=0/0 io=0/0 bytes/io=NaNK/NaNK

remote.log:

1126645588.599792 10.100.101.102/4241 connection established.


Analyzer:
comm.log:

1126645546 (child)  [#1/10.200.201.202:44332] connected
1126645546 (parent) [#1/10.200.201.202:44332] peer connected
1126645546 (parent) [#1/10.200.201.202:44332] phase: version
1126645546 (parent) [#1/10.200.201.202:44332] phase: handshake
1126645546 (parent) [#1/10.200.201.202:44332] phase: sync
1126645546 (parent) [#1/10.200.201.202:44332] phase: running
1126645584 (parent) parent statistics: pending=0 bytes=816K/2K chunks=5909/16 io=888/4 bytes/io=0.92K/0.61K
events=2891/0
1126645584 (parent) child statistics: [0] pending=0 bytes=0K/0K chunks=0/0 io=0/0 bytes/io=NaNK/NaNK
1126645644 (parent) parent statistics: pending=0 bytes=1964K/2K chunks=13645/16 io=1930/4 bytes/io=1.02K/0.61K events=6758/0
1126645644 (parent) child statistics: [0] pending=0 bytes=0K/0K chunks=0/0 io=0/0 bytes/io=NaNK/NaNK
1126645704 (parent) parent statistics: pending=0 bytes=3074K/2K chunks=21055/16 io=3021/4 bytes/io=1.02K/0.61K events=10462/0

remote.log:

1126645546.115779 10.200.201.202/44332 connection established.
1126645546.116053 10.200.201.202/44332 requesting events matching /^?(.*)$?/
1126645546.116405 10.200.201.202/44332 accepting state
1126645546.116471 10.200.201.202/44332 requesting synchronized state

What is going on here is:

Once the handshake completes, the most interesting log entrys for this are the statistics information.  What I usually watch are the numbers of events reported (last column).  The ratio represents events reviewed/events sent (hence the sensor side looks like 0/n and the analyzer side looks like n/0).  If you have two way event passing going on, then there should be some n/m rather than 0.