NSM-Console (Network Security Monitoring Console) is a framework for performing analysis on packet capture files. It implements a modular structure to allow for an analyst to quickly write modules of their own without any programming language experience. Using these modules a large amount of pcap analysis can be performed quickly using a set of global (as well as per-module) options. It aims to be simple to run and easy to understand without a lot of learning time.

The recently posted NSM-Console whitepaper is a good place to start if you want an introduction to NSM-Console.

You can download NSM-Console 0.6 from the project page.

Here are the highlights for the changes in this version, check the TODO and CHANGELOG for a full list of changes.

The dump command
The dump command lets you dump either the payload (in ascii format), or the full packet (with the -f flag) to a file, you can specify ranges like 1-20, 5,7,9-* and 1-* just like you can with the print command, extremely useful for picking and choosing packets to dump to a file. Here’s an example:

nsm> dump 1-5,10-* file.txt
Writing (append) packet(s) 1 through 5 from /Users/hinmanm/data.pcap to file.txt...
Writing (append) packet(s) 10 through * from /Users/hinmanm/data.pcap to file.txt...

And another of the full dump

nsm> dump -f 20,56-59 full.out
New args: 20,56-59 full.out
Dumping full packet, not just payload...
Writing (append) packet(s) 20 through 20 from /Users/hinmanm/data.pcap to full.out...
Writing (append) packet(s) 56 through 59 from /Users/hinmanm/data.pcap to full.out...

Performance improvements for harimau and checkip
The Harimau module, as well as the checkip command, now use wget (if it’s installed) instead of Ruby’s HTTP request library, this results is a much much faster query time. Another performance tweak was that reverse dns lookups for most modules is now turned off by default because it was taking extremely long for large pcaps

New modules: clamscan, foremost, trace-summary
New modules have been included in this release, clamscan, which scans extracted files (from either the tcpxtract or foremost modules) for viruses, foremost, which is another file extractor and trace-summary, which provides an extremely nice summary of traffic in the pcap

NSM-Console handle’s gzip’d files
You can now specify a gzip’d file directly to NSM-Console, it will attempt to decompress the file into a temporary file when it is loaded. Note that directories of gzip’d files is still not supported entirely.

Bro-ids and Snort changes
Bro-IDS has been split into 2 modules, one for protocol analysis and the other for generating content streams.
Snort has had its community and emerging rules updated to the latest revisions.

As always, bugfixes and minor enhancements, check out the full changelog for a list of what’s changed. If you have any requests for additional features or modules, be sure to leave a command and let me know

Also, if anyone out there wants to see what my ~/.nsmcrc looks like, you can find it here.

Let’s start with the wiring, at a local electronics store, I purchased 4 RJ-45 wiring plugs, I probably shouldn’t have bought solder-less ones, but I didn’t feel like buying a board to solder them to. Anyhow, 2 of the ports will be used for entry and exit, the other two for taps. In this case, we need 2 extra ports so that inbound data is passed through one port, outbound data is passed through the other port.

Set up the wiring as shown in this wiring diagram (credit goes to the Snort team for the diagram):

Personally, I split open a network cable and used the wires inside just so the color coding could be correct, that’s probably the easiest way to wire the ports.

After wiring the ports, you should be able to test that data passed from one host port to the other host port is unchanged, below is a picture of the tap I created. Yes, I know it’s very messy, the box I bought for it didn’t fit the way I wanted.

The next thing to do it connect the two ports (labeled “tap 1″ and “tap 2″ in the picture above) to 2 NICs in the machine of your choice. I’m using FreeBSD to manage the bridge. If you want to monitor outbound and inbound traffic separately, you’re done, just start tcpdump on the interface and you should be able to see all the traffic.

If you want to monitor both outbound and inbound traffic on the same interface, you’ll need to bridge the interfaces. You can accomplish this in FreeBSD with the following:

shell> ifconfig bridge create
shell> ifconfig bridge0 addm ed0 addm ed1 monitor up
shell> tcpdump -i bridge0
(or run snort/bro-ids/argus/etc on interface bridge0)

In this case, my network cards are ed0 and ed1, if you had different network interfaces, substitute them instead. You don’t need to assign an address to the bridge interface, since the only wires that are connected are the receive wires, so it wouldn’t transmit through the taps if it wanted to. For more advanced bridging, check out the FreeBSD manual on bridging.

I should note though, that you’ll need a 3rd network card in the monitoring machine if you want to remotely manage the machine.

Firstly, you can download NSM-Console version 0.5 here:
http://writequit.org/projects/nsm-console/files/nsm-console-0.5.tar.gz

Mirror here:
https://secure.redsphereglobal.com/data/dakrone/files/nsm-console-0.5.tar.gz

Like always, let’s go over some of the new features in this release:

Alias command
You can now alias a command whatever else you would like to, the syntax is the same as regular bash alias syntax, for instance, here are my aliases from my ~/.nsmcrc:
alias ls = list
alias ll = list
alias serv = e cat /etc/services | grep
So as an example, if I wanted to look up a service port, now I just type “serv 5190” and see if /etc/services has an entry for that port. (I have a habit of hitting ‘ll’ or ‘ls’ all the time, so now at least they’re useful)

Additional modules: flowtime and harimau
I added a couple of modules, the first is flowtime, which is a packet timeliner that I wrote about in this post. The second is the Harimau module, which will query the Harimau watchlist for all the IPs in a pcap file and print out the matching entries. Thanks go to Spoonfork and the Security.org.my team for the awesome tool.
Note: flowtime won’t work out of the box in Hex unless you install Argus version 3 (not version 2, which is what Hex comes with) as well as symlink ‘ploticus’ to ‘pl’ somewhere in your path.

Checkip command
Speaking of the Harimau watchlist, it has also been integrated as an NSM-Console command. You can see an example here:
nsm> checkip 209.177.146.34
209.177.146.34,www.emergingthreats.net/rules/bleeding-botcc.rules,botcc,2008-02-05 00:03:10

Module improvements
The snort module now uses the ac-bnfa search algorithm, which should help on systems with lower amounts of RAM (*cough* like my own). In addition, the bro-ids module now actually generates many more helpful reports and actually performs intrusion detection instead of just generating flow content. Some modules have been added to categories to make them easier to toggle.

Other minor improvements
Toggle handles multiple module names, space separated
All NSM-Console errors finally go to STDERR instead of STDOUT
Help command is much more readable and supports argument to get help about a particular command.
~./nsmcrc is read extremely quietly now, so it doesn’t fill up the screen
Bugfixes.

You can read the entire changelog here.

As always, please please please let me know if you have any comments, criticisms or suggestions Feel free to email me or leave a comment below.

• Snort module with community rules
  • self-contained snort module will all the community rules and configuration file, this’ll generate alerts into a file after reading the pcap file. I wasn’t sure whether to use community or bleeding edge rules, it’s still easy to point the snort module to your own snort.conf file and do it that way.
• Exec command will do substitution now on the following variables:
  • ${PCAP_FILE}
  • ${PCAP_BASE}
  • ${MODULE_DIR}
  • ${OUTPUT_DIR}
  • This’ll let you do something like “exec tcpdump -X -n -r ${PCAP_FILE}“
  • In addition, exec now logs all the commands run into the regular logfile
• The ‘logfile’ command, real simple, just specifies a new logfile
• Whitespace is handled much much better, there were a lot of bugs with whitespace being handled correctly for the “set” command (among others), it should be handled much better now.
• Category loading now handles non-files much better, before, if you left a “CVS” directory in the categories folder, it would read it but when it went to do a “toggle all”, it would error out, this has been fixed.
• Lots of bugfixes

You can grab the new version here:

http://writequit.org/projects/nsm-console/files/nsm-console-0.3-DEVEL.tar.gz

It’s definitely stable enough for daily use, highly recommended over the older versions. I’m still hoping to get cvs-web interface up to be able to browse the code.

• aimsnarf
• ngrep (gif/jpg/pdf/exe/pe/ne/elf/3pg/torrent)
• tcpxtract
• tcpflow
• chaosreader
• bro-IDS
• snort
• tcpdstat
• capinfos
• tshark
• argus
• ragator
• racount
• rahosts
• hash (md5 & sha256)
• ra
• honeysnap
• p0f
• pads
• fl0p
• iploc
  
• foremost – thanks shadowbq!
• flowgrep
• tcptrace
• tcpick
• flowtime
• flowtag
• harimau
• clamscan
  

Think of any other useful modules? Leave me a comment and let me know!

P.S. I’m also brainstorming for some pcap/real-time network visualization tools, stay tuned!