Firstly, scholar01 has created a ‘flowtag’ module for NSM-Console to use Chris Lee’s  excellent Flowtag software for categorizing and tagging network flow for a packet capture. Thanks for the submission scholar01!

Secondly, JohnQPublic has created a ‘clamscan’ module to in order to scan the files extracted by either tcpxtract or foremost for viruses. The clamscan module uses the popular open-source antivirus ClamAV software. Thanks JohnQPublic!

Both of these modules have been committed into NSM-Console’s code, and while only flowtag is included in the 0.5 release, you can try them out by checking NSM-Console out of SVN with the following command:

svn co http://svn.security.org.my/trunk/rawpacket-root/usr/home/analyzt/rp-NSM/nsm-console nsm-console

Note that the majority of the code I commit to svn is stable enough for regular usage, it just doesn’t undergo the regular testing that the point-releases do before they are released.

Thanks to both authors for submitting modules, they’re now included in the ‘credits’ command.

You can get the screencast here (right-click and download, don’t stream):

NSM-Console project page.

It’s under 10 mb and clocks in at 7 minutes and 14 seconds.

If you’re looking to make your own module, I highly recommend first reading this README file (found in the modules directory) and watching the screencast.

If you do make a module and would like it included with the NSM-Console distribution, let me know by sending me an email or leaving a comment.

The version of NSM-Console used in the screencast is verion 0.4

• aimsnarf
• ngrep (gif/jpg/pdf/exe/pe/ne/elf/3pg/torrent)
• tcpxtract
• tcpflow
• chaosreader
• bro-IDS
• snort
• tcpdstat
• capinfos
• tshark
• argus
• ragator
• racount
• rahosts
• hash (md5 & sha256)
• ra
• honeysnap
• p0f
• pads
• fl0p
• iploc
  
• foremost – thanks shadowbq!
• flowgrep
• tcptrace
• tcpick
• flowtime
• flowtag
• harimau
• clamscan
  

Think of any other useful modules? Leave me a comment and let me know!

P.S. I’m also brainstorming for some pcap/real-time network visualization tools, stay tuned!

Behold! I present NSM-Console! (read more about it here, watch a screencast here)

Download the framework here.
Keep in mind this framework only includes 3 modules (mostly used just for testing)

NSM-Console in a small (< 500 1000 1500 lines) framework for running nsm modules. Essentially, it’s a framework for running things (but we don’t call it that because it sounds like it wasn’t any work :P). Here’s the breakdown:

Users will run the ‘nsm’ program to enter into a console, by which they can select modules that have options to run against a packet capture file. Let’s talk a little bit about how it works. When nsm starts, it looks in the directory given by $MODULE_DIR (hardcoded at the moment, will be moved later) for all directories that follow the naming convention “<name>.module”, where <name> is the name of the module. Following so far?

For explanation’s sake, let’s call our module “aimsnarf”, after the aim-sniffing program I wrote a while back. So inside the module directory, there is another directory called “aimsnarf.module”. Inside the aimsnarf.module directory, there are a few files that define how our module will act. First is a file just called “aimsnarf”, this is the file that dictates commands to be run in the framework. It can have multiple commands separated by lines in the file. (If the module were called “bathtub”, the file would be called “bathtub” also, it’s named after the module name). Let’s take a look at an example line from the aimsnarf.module/aimsnarf file:

aimsnarf -r ${PCAP_FILE} > ${OUTPUT_DIR}/${OUTPUT_FILE}

In this line, you can see some variables that get changed out when the module is executed from nsm console. ${PCAP_FILE} is always set to the pcap file being analyzed, ${OUTPUT_DIR} will be set to the module-specific directory output will be directed into (set by nsm console), the last option, ${OUTPUT_FILE} is not set by the nsm console, it is a user-defined variable. But where is it defined? Let’s take a look at the next file: aimsnarf.module/defaults:

OUTPUT_FILE=${PCAP_BASE}.aim

The defaults file defines the user-settable options for the module, in this case, there is only 1, the “OUTPUT_FILE” option. The defaults file also provides a default value for that option (in this case, ${PCAP_BASE}.aim). The default value is provided so the module can be run without having to set every option. ${PCAP_BASE} is another variable that is provided by the nsm console, it is the basename of the pcap file (so it the pcap file was “~/pcap/file.pcap”, the ${PCAP_BASE} would be “file.pcap”).

The next file is aimsnarf.module/description, this file provides a 1-line description of the module to be listed when the user lists all the modules. Here is the ‘description’ for the aimsnarf module:

Extracts AIM messages from a pcap file

Simple eh? Still following? Okay, there is one more file, the aimsnarf.module/info file, this file provides a more detailed explanation of the module, it should also define what each option means, here is what aimsnarf’s ‘info’ file looks like:

Aimsnarf is a program to extract aim conversations from a pcap file

The following options are enabled:
OUTPUT_FILE (default $PCAP_DATA}.aim)
This option specifies the output file.

So how does this all work together? Well, first, you would start the NSM console you would see the following screen:

You can see that the aimsnarf module has been loaded (as well as a couple of others) and a prompt has been presented. From here there are a few different options, here’s the output of the ‘help’ command:

I think the help is pretty self-explanatory, hopefully it makes sense to everyone else as well.

Let’s run the aimsnarf module on a pcap file as an example. First, we need to specify a pcap file to work on with the ‘file’ command, so we type:

nsm> file /Users/hinmanm/data.pcap
Setting ${PCAP_FILE} = /Users/hinmanm/data.pcap
Setting ${PCAP_BASE} = data.pcap

Next, we will specify a different output directory (the default is ‘output’) using the ‘output’ command:

nsm> output data-output
Setting ${OUTPUT_DIR} = data-output

Next, let’s make sure the only module that is going to run is the aimsnarf module:

nsm> toggle none
All modules turned off.
nsm> toggle aimsnarf
aimsnarf module turned on.

And make sure our options are set correctly for the aimsnarf module:

nsm> options aimsnarf
Global options:
-----------------------------------
${PCAP_FILE}: /Users/hinmanm/data.pcap
${PCAP_BASE}: data.pcap
${OUTPUT_DIR}: data-output
${MODULE_DIR}: modules
Options for module aimsnarf:
-----------------------------------
${OUTPUT_FILE} = ${PCAP_BASE}.aim

That looks good to me! Now, let’s run and see what happens:

nsm> run
Executing analysis...
===> module aimsnarf running...
Creating directory data-output/aimsnarf
Result: true
--> aimsnarf -r /Users/hinmanm/data.pcap > data-output/aimsnarf/data.pcap.aim
/Library/Ruby/Site/1.8/universal-darwin9.0/pcap.bundle: warning: do not use Fixnums as Symbols
/Library/Ruby/Site/1.8/universal-darwin9.0/pcap.bundle: warning: do not use Fixnums as Symbols
/Library/Ruby/Site/1.8/universal-darwin9.0/pcap.bundle: warning: do not use Fixnums as Symbols
Result: true
===> module aimsnarf finished.
===> module hash skipped.
===> module ngrep skipped.

Looks like it ran the command! (ignore the warnings, they’re generated by aimsnarf, not nsm) Simple enough, right?

So here’s the real question: why write something like this? Well, I believe this tool will help to perform large amounts of analysis (think 20-30 plugins instead of 3) on packet capture files without having to manually manage each tool that does analysis. Instead of having to change a hardcoded script, you can now edit (and create) modules to do exactly what you need, but still having the flexibility of global varaibles (ie: ${OUTPUT_DIR}) and module specific variables (ie: ${OUTPUT_FILE} in aimsnarf). Being able to toggle each module on and off allows you to do a flexible amount of analysis on a file on the fly. In the future I hope to be able to extend the framework to run on a directory of files, allowing you to automate running a vast number of pcap analysis tools on a large number of pcap files very easily.

While this is not a replacement for running each command individually (and it shouldn’t be), I hope this tool will be useful to people. I hope to be able to make it useful for the Hex liveCD as well

Questions? Comments? Leave me a comment below or send me an email!