For the next couple of blog posts, I’ve decided to describe some of the software we use (and have developed) at Sonian. We have a few Clojure repos pushed to github that have documentation that is poor-to-none. I wanted to start with our main communication tool these days: our IRC server, Subrosa. We originally were using Skype to communicate, but finally got fed-up with stability and usability, we wanted to switch to something with a little bit more control. Dan Larkin had been working on an IRC server we could use for hosting our main topic chats, we switched from Skype in part because of these reasons:

• Logging (Skype’s logs are all in-process, so there was no easy way to aggregate and share logs if a person had never been in a room)
• Topic chats – Since we’re a decentralized team, we split a lot of our work into discussions rooms specific to the story being worked on so we can chat about it while the work goes on. With Skype, you cannot join a room that you haven’t been invited to (if you took over a story for someone else for example), with IRC we are free to drop in and out of the rooms however we’d like (we trade off a little with no private rooms, but it is a work-server, not really for private conversations)
• Full control over the sever
• A culture of many clients and configuration options – Each of us can connect to IRC however we’d like, set up notification exactly as we’d like and set up the variety of tools already built around IRC (bouncers, etc)

Switching has been very nice, we run an SSL-enabled IRC server where we do all our of development chat and haven’t looked back at Skype (which we still use, but no longer for the majority of our development chatter).

Below the covers

Let’s go a little more in-depth with the Subrosa code. Subrosa is built on top of Netty to make use of asynchronous communication between the clients (called channels in the code) and the server. Netty uses a pipeline of handlers for each message (either sent or received). Each handler will process the message, then pass it up the pipeline until there are no more handlers. Messages that are sent and received are handled by ChannelUpstreamHandlers and ChannelDownstreamHandlers, which are added to the pipeline in the add-irc-pipeline! function:

The most interesting handler that is associated is the (message-stage #'message-handler), which handles the commands that are sent once a user is connected. This calls the dispatch-message function in subrosa.commands that handles any commands that come across the channel. The commands themselves live in commands.clj.  Let’s look at probably the most common command, privmsg:
There are two types of commands, authorized and unauthorized commands (defcommand and defcommand* respectively). Authorized commands require the user to have sent the USER and NICK commannds as well as PASS (if required by the server). The bulk of the work in this command comes from helper functions that determine whether the message was sent to a room (room-for-name recipient) or a user (channel-for-nick recipient), then sends the command to the room or the specified user. Also of note are the hooks that are manually called (privmsg-room-hook and privmsg-nick-hook). This allows Subrosa to support plugins for additional features.

The database

Subrosa uses a custom in-memory database built around a ref to keep track of its users, channels and rooms. It acts like a key-value store for Clojure maps that allows for some interesting automatic indexing between fields. The database was previously built around clojure.contrib.datalog, but switched to a custom DB with the 0.7 version. Every time a user joins the server, a map is put in the DB mapping the Netty channel to the user’s nick (so that messages can be sent to that user’s nick). Rooms are also kept in the database to allow a message to be sent to each nick (therefore each channel) in a given room. Check out the tests for a quick look at how it’s used.

Hooks and Plugins

Subrosa uses a hooks system (manually invoked from each defcommand) to allow for supporting plugins. Hooks are actually how commands are implemented, the defcommand and defcommand* functions are macros that add a hook with the “cmd” tag. There are no limit to the number of hooks, at the time of this writing, there are 9 hooks:

• privmsg-room-hook
• privmsg-nick-hook
• join-hook
• part-hook
• quit-hook
• nick-hook
• topic-hook
• notice-room-hook
• notice-nick-hook

Hooks can easily be added (there are no rules about them) and invoked by giving a tag and a function to be called when the hook is called:

(add-hook ::myhooktag 'my-awesome-hook (fn [& args] (println "I got called!")))

And called using run-hook:

(run-hook 'my-awesome-hook "Subrosa" "is" "awesome")

This is exactly how the logging plugin works (and the /catchup command added in my fork of Subrosa). Automatic logging solves our first main problem with Skype, which is that it is difficult to get conversation logs if you aren’t involved in the chat from the beginning. That about wraps up a basic overview of Subrosa and how (some of) it works. I encourage you to check out the Subrosa github repo and peruse the code. Hopefully this may inspire you to use it as your own server, or fork the project on github and add more commands to it (it’s missing a lot of RFC commands still). Thanks (and pull requests) go to Dan Larkin for getting this project up and awesome (and for letting me write about it).