General Thoughts
Rule organization
Mapping Alert Types
Signature Metadata
Looking at Snort Logs

Notes on errors from stock 2.0 and 2.1 rule sets


General thoughts

We ought to design the snort2bro interface based on the digestion of the most recent set of snort rules available.  This is based on the idea that this is what users are going to do, or at least what the users expect that they ought to do.  We can do usefulness sig analysis along with this, but users want to be able to run stupid sigs...

Rule Organization
There are currently sigs in 2.1.X rule set that will kill s2b (see below) - this ought to be fixable.  Also, there are rules types that should not be run (insert duplicate sid's etc) which will cause instability in bro.  There are general, non-rule, files that come with the rule set.  These need to be cleaned out and not processed since s2b can not deal with them... I am thinking about some sort of wrapper script around s2b to keep all this mess away.

The wrapper script should also make sure that there is only a single instance of any sid

There is an implicit structure/organization within the rule tarball that ought to be exploited.  The rule set is already sorted into functionally named files that contain the associated individual rules. The types are:


attack-responses.rules  finger.rules      nntp.rules       scan.rules         web-client.rules
backdoor.rules          ftp.rules         oracle.rules     shellcode.rules    web-coldfusion.rules
bad-traffic.rules       icmp.rules        other-ids.rules  smtp.rules         web-frontpage.rules
chat.rules              imap.rules        p2p.rules        snmp.rules         web-iis.rules
ddos.rules              info.rules        policy.rules     sql.rules          web-misc.rules
deleted.rules           local.rules       pop2.rules       telnet.rules       web-php.rules
dns.rules               misc.rules        pop3.rules       tftp.rules         x11.rules
dos.rules               multimedia.rules  porn.rules       virus.rules
experimental.rules      mysql.rules       rpc.rules        web-attacks.rules
exploit.rules           netbios.rules     rservices.rules  web-cgi.rules


There are several that can be ignored immediately - in particular are the porn rules.  I do not want to sell this as a tool to enforce HR policy.  If others feel strongly about this I do not have needlessly ideological point to make.


Mapping Alert Types
The mass of data may be filtered via mappings between snort's classification type, and bro's SigAction type.  They are both:
Table: Snort Default Classifications
Classtype Description Priority
attempted-admin Attempted Administrator Privilege Gain high
attempted-user Attempted User Privilege Gain high
shellcode-detect Executable code was detected high
successful-admin Successful Administrator Privilege Gain high
successful-user Successful User Privilege Gain high
trojan-activity A Network Trojan was detected high
unsuccessful-user Unsuccessful User Privilege Gain high
web-application-attack Web Application Attack high
attempted-dos Attempted Denial of Service medium
attempted-recon Attempted Information Leak medium
bad-unknown Potentially Bad Traffic medium
denial-of-service Detection of a Denial of Service Attack medium
misc-attack Misc Attack medium
non-standard-protocol Detection of a non-standard protocol or event medium
rpc-portmap-decode Decode of an RPC Query medium
successful-dos Denial of Service medium
successful-recon-largescale Large Scale Information Leak medium
successful-recon-limited Information Leak medium
suspicious-filename-detect A suspicious filename was detected medium
suspicious-login An attempted login using a suspicious username was detected medium
system-call-detect A system call was detected medium
unusual-client-port-connection A client was using an unusual port medium
web-application-activity access to a potentially vulnerable web application medium
icmp-event Generic ICMP event low
misc-activity Misc activity low
network-scan Detection of a Network Scan low
not-suspicious Not Suspicious Traffic low
protocol-command-decode Generic Protocol Command Decode low
string-detect A suspicious string was detected low
unknown Unknown Traffic low

Bro's related data types:

type SigAction: enum {
        SIG_IGNORE,     # ignore this sig. completely (even for scan detection)
        SIG_QUIET,      # process, but don't report individually
        SIG_FILE,       # write into the signatures.log file
        SIG_FILE_BUT_NO_SCAN,   # as SIG_FILE, but ignore for scan processing
        SIG_LOG,        # log it and write into the signatures.log file
        SIG_LOG_PER_ORIG,       # log once per originator
        SIG_LOG_ONCE,   # log once and then never again
        SIG_LOG_NO_WORM,        # log if not originated by a known worm-source
        SIG_COUNT_PER_RESP,     # count per dest. and log if threshold reached
        SIG_SUMMARY,    # don't log, but generate per-orig summary
};

# Actions for a signature.
const signature_actions: table[string] of SigAction =  {

        ["sid-526"] = SIG_IGNORE,
                # sid-526 BAD TRAFFIC data in TCP SYN packet

} &redef &default = SIG_LOG;

Here we may be able to augment the SigAction types (if needed) and create a mapping based on the provided classification type.  This may be slightly difficult in that a file will need be written which augments the value of the SigAction table.

Signature Metadata
The signature metadata file will be the location where s2b picks up additional information for bro to translate the raw snort sig to the stateful bro sig.

Record contents will be similar to:

sid#,additional_filter1, additional_filter2, action, notes/url?

details will be fleshed out as we begin working on this

Looking at Snort Logs
Logs from currently operational snort applications can be used to get an idea about the volume of logs that will be problematic, particularly with regard to false positives.  Taking 2 months of logs we see a mix of interesting and low value noise:
 
   1 [**] [105:1:1] spp_bo: Back Orifice Traffic detected (key: 31337) [**]
   1 [**] [1:1228:3] SCAN nmap XMAS [**]
   1 [**] [1:1444:2] TFTP Get [**]
   1 [**] [1:1497:6] WEB-MISC cross site scripting attempt [**]
   1 [**] [1:1694:3] ORACLE alter table attempt [**]
   1 [**] [1:1948:1] DNS zone transfer UDP [**]
   1 [**] [1:1975:3] FTP DELE overflow attempt [**]
   1 [**] [1:2066:2] WEB-MISC Lotus Notes .pl script source download attempt [**]
   1 [**] [1:2127:1] WEB-CGI ikonboard.cgi access [**]
   1 [**] [1:2179:1] FTP PASS format string attempt [**]
   1 [**] [1:542:8] CHAT IRC nick change [**]
   1 [**] [1:823:4] WEB-CGI cvsweb.cgi access [**]
   1 [**] [1:987:9] WEB-IIS .htr access [**]
   2 [**] [1:1013:6] WEB-IIS fpcount access [**]
   2 [**] [1:1141:7] WEB-MISC handler access [**]
   2 [**] [1:1147:5] WEB-MISC cat%20 access [**]
   2 [**] [1:1242:6] WEB-IIS ISAPI .ida access [**]
   2 [**] [1:1309:6] WEB-CGI zsh access [**]
   2 [**] [1:1561:4] WEB-MISC ?open access [**]
   2 [**] [1:1629:3] OTHER-IDS SecureNetPro traffic [**]
   2 [**] [1:1643:4] WEB-CGI db2www access [**]
   2 [**] [1:1721:3] WEB-CGI adcycle access [**]
   2 [**] [1:1958:4] RPC sadmind TCP PING [**]
   2 [**] [1:2180:1] P2P BitTorrent announce request [**]
   2 [**] [1:2212:1] WEB-CGI imageFolio.cgi access [**]
   2 [**] [1:228:1] DDOS TFN client command BE [**]
   2 [**] [1:585:6] RPC portmap sadmind request UDP [**]
   2 [**] [1:877:5] WEB-CGI rksh access [**]
   2 [**] [1:966:6] WEB-FRONTPAGE .... request [**]
   3 [**] [1:1233:7] WEB-CLIENT Outlook EML access [**]
   3 [**] [1:1399:7] WEB-PHP PHP-Nuke remote file include attempt [**]
   3 [**] [1:1672:6] FTP CWD ~ attempt [**]
   3 [**] [1:1682:3] ORACLE all_source access [**]
   3 [**] [1:249:3] DDOS mstream client to handler [**]
   3 [**] [1:729:4] VIRUS OUTBOUND .scr file attachment [**]
   3 [**] [1:997:5] WEB-IIS asp-dot attempt [**]
   4 [**] [1:1023:7] WEB-IIS msadcs.dll access [**]
   4 [**] [1:1149:9] WEB-CGI count.cgi access [**]
   4 [**] [1:1767:3] WEB-MISC search.dll access [**]
   5 [**] [1:1997:1] WEB-PHP read_body.php access attempt [**]
   5 [**] [1:221:1] DDOS TFN Probe [**]
   6 [**] [1:1122:4] WEB-MISC /etc/passwd [**]
   6 [**] [1:1616:4] DNS named version attempt [**]
   6 [**] [1:2162:1] VIRUS OUTBOUND .hta file attachment [**]
   7 [**] [1:1777:2] FTP EXPLOIT STAT * dos attempt [**]
   7 [**] [1:1990:1] CHAT MSN user search [**]
   7 [**] [1:2172:1] VIRUS OUTBOUND .com file attachment [**]
   7 [**] [1:793:4] VIRUS OUTBOUND .vbs file attachment [**]
   8 [**] [113:3:1] (spp_frag2) TTL Limit Exceeded (reassemble) detection [**]
   8 [**] [1:1529:9] FTP SITE overflow attempt [**]
   8 [**] [1:2229:1] WEB-PHP viewtopic.php access [**]
   8 [**] [1:2307:2] WEB-PHP PayPal Storefront arbitrary command execution attempt [**]
   9 [**] [1:1229:5] FTP CWD ... [**]
   9 [**] [1:1668:5] WEB-CGI /cgi-bin/ access [**]
  10 [**] [1:255:8] DNS zone transfer TCP [**]
  11 [**] [1:2152:1] WEB-PHP test.php access [**]
  13 [**] [1:1377:10] FTP wu-ftp bad file completion attempt [ [**]
  13 [**] [1:1877:2] WEB-CGI printenv access [**]
  14 [**] [1:1676:3] ORACLE select union attempt [**]
  15 [**] [1:1463:5] CHAT IRC message [**]
  15 [**] [1:2002:1] WEB-PHP external include path [**]
  16 [**] [1:1285:5] WEB-IIS msdac access [**]
  16 [**] [1:1437:3] MULTIMEDIA Windows Media audio download [**]
  18 [**] [1:1301:6] WEB-PHP admin.php access [**]
  18 [**] [1:1653:3] WEB-CGI campus access [**]
  19 [**] [1:1623:6] FTP invalid MODE [**]
  20 [**] [1:1478:3] WEB-CGI swc access [**]
  20 [**] [1:1882:9] ATTACK-RESPONSES id check returned userid [**]
  20 [**] [1:2087:2] SMTP From comment overflow attempt [**]
  22 [**] [116:58:1] (snort_decoder): Experimental Tcp Options found [**]
  23 [**] [1:862:6] WEB-CGI csh access [**]
  24 [**] [1:2160:1] VIRUS OUTBOUND .exe file attachment [**]
  24 [**] [1:251:1] DDOS - TFN client command LE [**]
  24 [**] [1:524:7] BAD-TRAFFIC tcp port 0 traffic [**]
  26 [**] [1:1156:4] WEB-MISC apache DOS attempt [**]
  27 [**] [1:2183:1] SMTP Content-Transfer-Encoding overflow attempt [**]
  32 [**] [1:1637:4] WEB-CGI yabb access [**]
  33 [**] [116:57:1] (snort_decoder): Obsolete TCP Options found [**]
  33 [**] [1:868:6] WEB-CGI rsh access [**]
  35 [**] [1:1067:5] WEB-MISC net attempt [**]
  35 [**] [1:2256:2] RPC sadmind query with root credentials attempt UDP [**]
  36 [**] [1:853:6] WEB-CGI wrap access [**]
  40 [**] [1:839:4] WEB-CGI finger access [**]
  41 [**] [1:1881:4] WEB-MISC bad HTTP/1.1 request, Potentially worm attack [**]
  43 [**] [1:1992:2] FTP LIST directory traversal attempt [**]
  46 [**] [1:1684:3] ORACLE all_tab_columns access [**]
  54 [**] [1:1448:4] MISC MS Terminal server request [**]
  54 [**] [1:865:5] WEB-CGI ksh access [**]
  55 [**] [1:1681:3] ORACLE all_views access [**]
  57 [**] [1:1260:6] WEB-MISC long basic authorization string [**]
  64 [**] [1:1680:3] ORACLE all_constraints access [**]
  69 [**] [1:1226:2] X11 xopen [**]
  74 [**] [1:1243:8] WEB-IIS ISAPI .ida attempt [**]
  88 [**] [1:939:5] WEB-FRONTPAGE posting [**]
  92 [**] [1:2181:1] P2P BitTorrent transfer [**]
  96 [**] [1:1213:4] WEB-MISC backup access [**]
  96 [**] [1:2178:1] FTP USER format string attempt [**]
 100 [**] [119:3:1] (http_inspect) U ENCODING [**]
 102 [**] [1:1683:3] ORACLE all_tables access [**]
 108 [**] [1:1551:3] WEB-MISC /CVS/Entries access [**]
 133 [**] [1:1778:2] FTP EXPLOIT STAT ? dos attempt [**]
 137 [**] [1:525:5] BAD-TRAFFIC udp port 0 traffic [**]
 168 [**] [1:1686:3] ORACLE dba_tablespace access [**]
 171 [**] [1:721:4] VIRUS OUTBOUND .pif file attachment [**]
 173 [**] [1:1112:4] WEB-MISC http directory traversal [**]
 174 [**] [1:1807:2] WEB-MISC Chunked-Encoding transfer attempt [**]
 176 [**] [1:504:4] MISC source port 53 to <1024 [**]
 181 [**] [1:952:5] WEB-FRONTPAGE author.exe access [**]
 198 [**] [1:1867:1] MISC xdmcp info query [**]
 209 [**] [1:1564:4] WEB-MISC login.htm access [**]
 211 [**] [1:498:4] ATTACK-RESPONSES id check returned root [**]
 229 [**] [1:1991:1] CHAT MSN login attempt [**]
 229 [**] [1:990:5] WEB-IIS _vti_inf access [**]
 230 [**] [1:962:6] WEB-FRONTPAGE shtml.exe access [**]
 270 [**] [1:884:8] WEB-CGI formmail access [**]
 282 [**] [1:1631:4] CHAT AIM login [**]
 288 [**] [1:2091:2] WEB-IIS WEBDAV nessus safe scan attempt [**]
 300 [**] [1:528:4] BAD-TRAFFIC loopback traffic [**]
 384 [**] [1:1687:3] ORACLE dba_tables access [**]
 387 [**] [1:2161:1] VIRUS OUTBOUND .doc file attachment [**]
 469 [**] [1:1256:7] WEB-IIS CodeRed v2 root.exe access [**]
 475 [**] [119:7:1] (http_inspect) IIS UNICODE CODEPOINT ENCODING [**]
 524 [**] [1:971:3] WEB-IIS ISAPI .printer access [**]
 529 [**] [1:1326:3] EXPLOIT ssh CRC32 overflow NOOP [**]
 560 [**] [1:1519:6] WEB-MISC apache ?M=D directory list attempt [**]
 629 [**] [1:621:3] SCAN FIN [**]
 637 [**] [1:523:4] BAD-TRAFFIC ip reserved bit set [**]
 665 [**] [1:522:1] MISC Tiny Fragments [**]
 706 [**] [1:1288:5] WEB-FRONTPAGE /_vti_bin/ access [**]
 750 [**] [1:885:6] WEB-CGI bash access [**]
 790 [**] [1:1627:3] BAD-TRAFFIC Unassigned/Reserved IP protocol [**]
 909 [**] [116:54:1] (snort_decoder): Tcp Options found with bad lengths [**]
 932 [**] [1:2129:2] WEB-IIS nsiislog.dll access [**]
1141 [**] [1:895:5] WEB-CGI redirect access [**]
1149 [**] [1:540:8] CHAT MSN message [**]
1176 [**] [1:1070:6] WEB-MISC WebDAV search access [**]
1190 [**] [1:1493:4] WEB-MISC RBS ISP /newuser access [**]
1543 [**] [1:882:4] WEB-CGI calendar access [**]
1687 [**] [1:1633:4] CHAT AIM receive message [**]
1689 [**] [1:1632:4] CHAT AIM send message [**]
1724 [**] [1:628:3] SCAN nmap TCP [**]
1763 [**] [1:983:6] WEB-IIS unicode directory traversal attempt [**]
2158 [**] [1:1748:4] FTP command overflow attempt [**]
2310 [**] [1:872:6] WEB-CGI tcsh access [**]
2580 [**] [1:1002:5] WEB-IIS cmd.exe access [**]
4242 [**] [1:1201:7] ATTACK-RESPONSES 403 Forbidden [**]
8837 [**] [119:12:1] (http_inspect) APACHE WHITESPACE (TAB) [**]
10233 [**] [119:2:1] (http_inspect) DOUBLE DECODING ATTACK [**]
13035 [**] [116:55:1] (snort_decoder): Truncated Tcp Options [**]
16957 [**] [106:2:1] (spp_rpc_decode) Multiple RPC Records [**]
16991 [**] [106:4:1] (spp_rpc_decode) Incomplete RPC segment [**]
18819 [**] [1:1042:6] WEB-IIS view source via translate header [**]
35058 [**] [1:469:1] ICMP PING NMAP [**]
43800 [**] [119:13:1] (http_inspect) NON-RFC HTTP DELIMITER [**]
55590 [**] [1:1432:4] P2P GNUTella GET [**]
55744 [**] [1:474:1] ICMP superscan echo [**]
98543 [**] [1:620:6] SCAN Proxy Port 8080 attempt [**]
99046 [**] [1:615:5] SCAN SOCKS Proxy attempt [**]
144096 [**] [119:16:1] (http_inspect) OVERSIZE CHUNK ENCODING [**]
174496 [**] [1:618:5] SCAN Squid Proxy attempt [**]
194539 [**] [119:4:1] (http_inspect) BARE BYTE UNICODE ENCODING [**]

All the various web related events are highlighted in red to demonstrate the prevalence of this type - in this case 'helping out' the web based rules would significantly help to reduce the volume of alerts.  In addition, a significant percent of the remaining alerts are either scanning events or weird events (which we can ignore).

There are several rules that are not run operationally (mostly ICMP informational rules), and many classes of rules are removed from service.  When these are not, the logs/alerts are orders of magnitude larger (...insert later...)




Notes on s2b errors

note, any non-rule file in the direct must be removed or .'d, or the snort2bro will die on non-rule analysis

For 2.0

Skipped options:

byte_test
Test a byte field against a specific value (with operator). Capable of testing binary values or converting representative byte strings to their binary equivalent and testing them.

http://www.snort.org/docs/snort_manual/node14.html#SECTION003511000000000000000

stateless
The flow rule option is used in conjunction with TCP stream reassembly (see Section ). It allows rules to only apply to certain directions of the traffic flow.

This allows rules to only apply to clients or servers. This allows packets related to $HOME_NET clients viewing web pages to be distinguished from servers running the $HOME_NET.

The established keyword will replace the flags: A+ used in many places to show established TCP connections.

Options: stateless - trigger regardless of the state of the stream processor ( useful for packets that are designed to cause machines to crash )

http://www.snort.org/docs/snort_manual/node15.html#SECTION00369100000000000000

window
The ack keyword is used to check for a specific TCP window size.

http://www.snort.org/docs/snort_manual/node15.html#SECTION003612000000000000000

byte_jump
The byte_jump option is used to grab some number of bytes, convert them to their numeric representation, jump the doe_ptr up that many bytes (for further pattern matching/byte_testing). This will allow relative pattern matches to take into account numerical values found in network data.

http://www.snort.org/docs/snort_manual/node14.html#SECTION003512000000000000000

Misc:
Warning in ./rules2.0/exploit.rules, line 30: Can't negate patterns with more than one character



For 2.1

pathological error w/ icmp-info.rules.  Looks like dealing with a lt/gt cgeck (icode:>0) make s2b die

(same as 2.0 plus)

threshold
Event Thresholding can be used to reduce the number of logged alerts for noisy rules. This can be tuned to significantly reduce false alarms, and it can also be used to write a newer breed of rules. Thresholding commands limit the number of times a particular event is logged during a specified time interval.

http://www.snort.org/docs/snort_manual/node18.html


pcre
The pcre keyword allows rules to be written using perl compatible regular expressions. For more detail on what can be done via a pcre regular expression, check out the PCRE web site http://www.pcre.org

http://www.snort.org/docs/snort_manual/node14.html#SECTION003510000000000000000

isdataat
Verify that the payload has data at a specified location, optionally looking for data relative to the end of the previous content match.

http://www.snort.org/docs/snort_manual/node14.html#SECTION00359000000000000000