BroFabric 0.1

Overview
The BroFabric is a framework for multiple bro instances (nodes) to share event and notice information with one another.  In the current form, the notion of a central analyzer node and remote sensor nodes is built in to the design, but this is not a limitation of the underlying (bro intercommunication) software.

There are three basic components to the fabric:

BroShell: as described elsewhere, this will be used to query the state of the various nodes interactively, and do simple debugging.  The policy side changes to the shell will be added to the individual policy files (ie the fabric based components will be in brofabric.bro), while the shell software will be modified for the additional functionality.  See below for more details.

BroFabric: The fabric layer is designed to contain static information about members of the current operational fabric.  Information (such as local IP ranges) are also held in this layer.  The basic idea is to separate the dynamic event data from what might be thought of as a naming or service data layer.

MetaEventLayer: This layer contains all of the policy that is designed to operate on dynamic data provided by the individual node hosts.  Examples of this might be distributed NOTICE analysis.  Again, see below and attached policy for more details.


BroFabric
With the fabric we will assume that any host making a remote connection to the node described as the sensors interested in joining the fabric.  Adding additional registration (etc.) is trivial and can be implemented if the need arises. 

Fabric members are indexed by their event_peer id number, which should be unique per host.  As a strawman data structure, it currently looks like:

type fabric_member: record {
        EP: event_peer;         # information about the individual event peer
        domain: set[subnet];    # the addresses associated with the new fabric member
};

As data is determined to be useful per fabric entity, it can be added to this structure and functions created to extract it (see return_node_address() later as an example).  When a remote host (dis)connects, we use remote_connection_established and remote_connection_closed to load and unload hosts from the list of fabric members.  Ie:

event remote_connection_established(p: event_peer)
        {
        print("remote_connection_established");
        local FM: fabric_member;

        if ( p$id !in fabric_member_container && fabric_center )
                { # we need to add this
                FM$EP = p;
                fabric_member_container[p$id] = FM;
                # further information will be loaded in via events
               
                NOTICE([$note=FabricMemberAdd,
                        $msg=fmt("%0.6f Added id=%s, name=%s",
                        network_time(), p$id, p$descr)]);

                }
        }

Here 'fabric_center' is just a boolean which tells the current bro instance if it ought to keep track of connections and disconnects from itself.  Information can then be added to the fabric via additional events which touch structures within the fabric_member record.  For example, the local subnets of the remote bro instance can be added via register_local_address:

event register_local_address(p: event_peer, loc_sub: set[subnet])
        {
        local s: subnet;
        local FM: fabric_member;

        if ( p$id in fabric_member_container && fabric_center)
                {
                FM = fabric_member_container[p$id];

                for ( s in loc_sub )
                        {
                        add FM$domain[s];
                        }
                }
        }

Information can be extracted from the fabric via regular functions such as return_node_address :

function return_node_address(id:count) : addr
        {
        # here we are given one of the node id's and we return it's
        # IP address
        local tmpFM: fabric_member;
        local retAddr: addr;

        if ( id in fabric_member_container )
                {
                tmpFM = fabric_member_container[id];
                retAddr = tmpFM$EP$host;
                }
        else
                {
                # error condition
                retAddr = 0.0.0.0;
                }

        return retAddr;
        }


MetaEventLayer
The metaevent layer (which will be renamed when I come up with something better) is the component that actually does most of the work.  The initial proof of concept for this layer is explored in distribute.bro and has been fleshed out more in eventwrapper.bro.  The general idea for this is to look at collections of events from different sources and to identify when the same IP address has been seen by multiple sources.

In eventwrapper.bro, Notices are looked at rather than discrete events in order to minimize the overhead brought on by the communication layer.  In addition, most of the things that are interesting are not event driven, but are the byproduct of local function analysis.  In order to access these notices (such as AddressScan), the notice_action event is used to transfer all notices that the sensor bros are processing rather than events.  This is orders of magnitude less communication than would otherwise be required.  If the notice is part of a defined list, then it is further processed.  The default list of 'interesting' notices is:

global interestingNotices = {
        "TRWAddressScan",
        "AddressScan",
        "AddressDropped",
        "PortScan",
        "SensitivePortmapperAccess",
        "LowPortTrolling",
        "OutboundCallback",
        "SensitiveConnection",
        "LandMineDetect",
} &redef;

With analysis notices happening when a given notice has been seen by more than source_sensor_pairs_exceeded sensors, or if the IP has been seen more than source_IP_threshold times.  See the code for more details.


BroShell
Several changes have been made in the broshell code to support this functionality.  Broshell infrastructure is broken up into the c++ code in the shell itself, and the event driven code on the bro side.  Event code relating to shell functionality has been added into the individual component side code, so that fabric related events for broshell are located in brofabric.bro rather than broshell.bro.  This is designed to simplify the broshell code.

The new commands in broshell include 'show fabric' and 'show notice <>' which look like:

--> show fabric
{
        [1] = host:131.243.64.97 id: 1 port: 44332/tcp islocal: F descr: ,
        [3] = host:127.0.0.1 id: 10000 port: 4477/tcp islocal: F descr: ,
        [2] = host:128.55.14.206 id: 2 port: 44332/tcp islocal: F descr:
}
-->
and
--> show notice TRWAddressScan
{
        [1] = 222.186.68.211 seen 3 times for TRWAddressScan 1: 131.243.64.97 2: 128.55.14.206 local:127.0.0.1,
        [2] = 219.153.32.12 seen 3 times for TRWAddressScan 1: 131.243.64.97 2: 128.55.14.206 local:127.0.0.1
}
-->

Currently the data formatting is rather crude, and indicates the need for additional functionality in the broshell itself.  For more details on the basic operation and command set of the broshell, see: http://www.nersc.gov/~scottc/software/bro/broshell.html  


Download
[NOTE: all of this is alpha quality - there is a great deal of debugging code here since this is still proof of concept]

Broshell Source Code : shell.cc
Broshell Policy : broshell.bro

Additional Policy:
brofabric.bro
eventwrapper.bro