Anyone that might have read my blog for a long time might remember this post about how I do GTD (Getting To Done) on my machine(s). Well, I decided it’s been quite a while since I’ve talked about that, so I’ll go through some of my workflow, tools and quirks.

If you read my blog soley for security content, you can skip this post Personally I like to read other people’s workflows/setup to see if there is an interesting way I can fit useful tools into my own setup. Anyhow, on with the details

I have 3 machines that I primarily use. My main machine is a MacBook Pro with an attached Dell LCD monitor running OSX (what else?), the second machine is a Dell Optiplex tower running Ubuntu 8.04. The third machine is a Dell 610 laptop that dual-boots Hex and Windows. Here’s a picture of my default setup on the machines:

OSX

Ubuntu

Hex

Most of my work is done on the MacBook (my personal machine, not supplied by work), I use the Ubuntu machine for tasks that span multiple days, so I can leave a screen term logged in and attach/detach whenever I need to take my laptop home. The Hex machine is mainly used for security purposes, packet analysis, nsm-console development testing, etc.

I like to keep a pretty clean desktop, clutter really tends to irritate me. I try as hard as I can to separate everything that would be on the desktop into either a “papers”, “media”, “code” or “misc” folder. In the previous post about GTD, I mentioned that we use the Mantis ticketing system to keep track of administration tickets. We still use Mantis, just recently upgraded, you can check out a shot here:

If my firefox looks a little strange, it’s because I use the vimperator extension to give firefox Vim-like keybindings and layout. It helps someone who spends a large amount of time in a vim editor to transition quickly between text editing and web-browsing. Speaking of text editing, I absolutely love MacVim and really recommend it to anyone using OSX who likes Vim:

I’ve also transitioned from Mutt to Thunderbird, mostly because Mutt’s 2-way IMAP settings are not so great if you have to check mail between multiple computers. I use the pre-release muttator extension to give Thunderbird mutt-like keybindings:

I use the standard GTD folder naming, which mostly consists of “Inbox”, “Archive” and “Do” folders. (I hope the idea behind each folder is obvious). I still use Gmail for personal email, but can’t access it from inside the work network over POP3 or IMAP due to firewall restrictions, so I use the web client.

Instead of using iCal for meetings, like I was doing, I moved to using Google Calendar, which is aided by the notifier for events, so I don’t have to set up reminders for meetings. It also lets me share my wife’s calendar and see what she’s up to As well as google calendar, I also use gReader to browse my 222 RSS subscriptions, nothing else comes close as far as I’m concerned (yay more vim-like keybindings).

I do still use a moleskine, but lately I’ve transitioned to the next larger notebook (5′ x 7′), so that I can write out larger notes and more information (writting assembly notation in a tiny notebook gets annoying). I make use of the same sort of TODO list as in my last post ( – for an incomplete task, + for a completed task ). I take it to all meetings I go to and doodle in it sometimes I still use the smaller book for lists and small notes.

One of the things I’ve found is that despite having a widescreen 2nd monitor, the ‘wide’ part doesn’t really help any for what I do, so I flipped the monitor up to make it vertical, which is really helpful for using IDA, since I can see more instructions at once, here’s the malware I’ve working on at the moment:

Since I normally have at least 1 VM running, as well as mail, 1 or 2 browsers, X11, Adium, MacVim and any other number of things, I cut down on RAM usage by using cmus (hurray more vim keybindings) for a music playing instead of iTunes (which is such a memory hog for simple things). I also use irssi instead of a graphical client for IRC since it’s a little lighter weight:

I rigged Quicksilver up to the cmus-remote command-line utility so I can have system-wide keybindings for player control, pretty much the other thing I use iTunes for anymore is heavy music folder management (I do like that feature). I spend a great deal of time at the command-line, if you’re interested in configuration files, you can find my zshrc, screenrc, vimrc, gvimrc and nsmcrc at dotfiles.org.

Finally, here’s an overview of my basic whole-desktop setup:

My GTD process has kind of evolved into a stripped down version of the multiple-step instructions from some of the more famous practitioners (receive email, process until inbox has 0 messages, perform actions). I make up a large amount of time by unifying my tools around a certain look and feel (Vim-keybindings for everything), allowing my brain not to have to shift gears for different tasks. By keeping a strict set of rules on which application goes where (as in for virtual spaces), I always know where I can find what I am looking for (application wise). In my opinion, this adds to my efficiency more than any particular tool could have.

Well, that’s about it, hopefully you found something interesting in this post. If you have a neat tool or helpful suggestion, leave a comment below (especially if it has Vim keybindings )

“I have a small update. I’ve made the number of backups of each
filesystem easier to handle by replacing the hardcoded number with a
variable. I also added some comments so everybody should be able adjust
both the path of the snapshot directory and the number of backups easily.

I decided to show you the disaster scenario for which this script is
being used.

Let’s take a simple server with the following ZFS file systems:

—
root@static:/> zfs list
NAME                   USED   AVAIL   REFER  MOUNTPOINT
tank                   8,78G  101G   28,5K   /export
tank/backup            4,93G  101G   27,5K   /export/backup
tank/backup/snapshots  4,93G  101G   4,93G   /export/backup/snapshots
tank/backup/sysdata    68K    101G   41,5K   /export/backup/sysdata
tank/repository        205M   101G   205M    repository/packages
tank/zones             3,65G  101G   25,5K   /export/zones
tank/zones/ffxi-sites  3,65G  101G   3,63G   /export/zones/ffxi-sites
root@static:/>
—

tank/backup: usual file system, but with compression=on. Very useful for
snapshots, compression rate is 1.66:1
tank/backup/snapshots: The snapshot directory I use in the scripts
tank/backup/sysdata: That’s a backup directory in which essential system
data is stored. Most important: the complete contents of /etc/zones and
perhaps some stuff like /etc/netmasks, /etc/nsswitch.conf, whatever
could be of importance during reconstruction of the host.
tank/repository/packages: That’s where I keep my packages, scripts,
whatever. Just a repository which is shared (sharenfs=ro,anon=o) and
re-mounted in the local zone as `/var/spool/pkg` … makes things easier.
tank/zones/ffxi-sites: That’s the zonepath of the zone `ffxi-sites-zone`.

I bet it’s getting interesting now, because I’ll explain you how to
backup this zone using the script … and how to reconstruct it from
scratch, on another hardware with different network drivers and such.

Okay. I make backups of tank/backup/sysdata (we need the contents of
/etc/zones for reconstruction), tank/repository/packages and, of
course, the entire local zone itself by backing up
tank/zones/ffxi-sites. Use my script for it, e.g. by executing it for
each file system in your crontab and sending it to a backup server.

This will result in two copies. the local copy in
/export/backup/snapshots (you can edit this path in the script) and one
on a remote server. We us the local copy in case someone shot the
database in the local zone or whatever. And the remote copy is needed in
case of a necessary re-installation of the entire server.

Btw: My private ISP only offers plain FTP for accessing backup servers.
FTP isn’t supported by the backup script. I make the local backups
between 1 an 3 am, and I use the following (imperfect) script at 5am to
transfer all backups to the FTP server:

—
#!/bin/bash
HOST='<ip of ftp server>'
USER='<user>'
PASSWD='<pass>'

BACKUPDIR="/export/backup/snapshots"

if [ ! -d $BACKUPDIR ]; then
echo "Backup Directory doesn't exist"
exit 1
fi

cd $BACKUPDIR

# cleanup ftp server, kamikaze style

ftp -n $HOST << END_CLEANUP > /dev/null
quote USER $USER
quote PASS $PASSWD
mdel *.zfs
bye
END_CLEANUP

# transfer all ZFS images to the ftp server

#for FILE in `ls -rt1 *.zfs`; do
# ftp -n $HOST << END_SCRIPT
# quote USER $USER
# quote PASS $PASSWD
# binary
# put $FILE
# bye
# END_SCRIPT
#done

#for FILE in `ls -rt1 *.zfs`; do
ftp -n $HOST << END_SCRIPT >/dev/null
quote USER $USER
quote PASS $PASSWD
binary
mput *.zfs
# put $FILE
quit
END_SCRIPT
#done

exit 0
—

This script isn’t recommended for production use, there’s a plain text
password in it. I only use it because the ftp server is firewalled and
cannot be accessed from other servers using this login – and if someone
hacked into my machine, he won’t need the backups anyway.

Okay, now disaster happens. We have to reconstruct *everything*. So
insert the Solaris DVD into the drive, install Solaris as usual, apply
the Recommend Patches cluster, and so on. Finally, we’re ready create
the ZFS pool again, we create tank/backup/snapshots and copy the ZFS
images from our remote server backup into this directory. We have our
local copies back.

Now: deploy the three filesystems using `zfs receive`, e.g. `zfs receive
tank/backup/sysdata < tank_backup_sysdata-070611-033000.zfs`

We have our zone configuration back now. We `cd` to
/export/backup/sysdata. Then we’re going to copy the `index` file plus
the ?.xml` files back to /etc/zones, replacing the default ones.

We still have to fix the network interface, which changed due to the
(imaginary) new hardware we have to use now. Enter the zone
configuration using `zonecfg`, e.g.:

—-
root@static:/> zonecfg -z ffxi-sites-zone
zonecfg:ffxi-sites-zone> info
zonename: ffxi-sites-zone
zonepath: /export/zones/ffxi-sites
autoboot: false
pool: pool_default
limitpriv:
net:
address: <your zone ip>
physical: rtls0
rctl:
name: zone.cpu-shares
value: (priv=privileged,limit=10,action=none)
attr:
name: comment
type: string
value: ffxi-sites
zonecfg:ffxi-sites-zone>
—

We have to change the “physical” entry to something else, let’s say
we’re using a X4100 now and so we need `e1000g0` instead of `rtls0`.
`ifconfig -a` shows us the device name.

Type `select net address=<your zone ip>`. Then `set pyhsical=e1000g0`.
Then `end`.

We still have to commit the changes. We do this by typing `commit` and
then we `exit`.

All we have to do now is `zoneadm boot ffxi-sites-zone` and we’re online
again, without having to deploy the zone, too.

Okay, done. We’re online again.

Only continue reading if you want to join some unsupported “hacking
procedures”.

Q: What to do in case our zonepath changed, or we want it to be changed?
A: That’s easy. Grab your `vi` and edit /etc/zones/index. Ignore the “DO
NOT EDIT” warning, that’s for girls only. See below on what to do.

Q: I only want to clone a zone, what to do now?
A: `zfs send <source filesystem> | zfs receive <destination
filesystem>`. Or clone and promote a filesystem, your choice. Then grab
you `vi` and edit /etc/zones/index again. Change the IP like we I shows
you earlier before you boot the cloned zone. And don’t forget chmod 700
your new zonepath.

(Matthew Says: Another way to clone as zone, depending on your patch level, can be seen further down here: http://uadmin.blogspot.com/2006/08/day-in-life-of-solaris-11-admin.html)

Let’s have a look at the index file:

—
root@static:/> cat /etc/zones/index
# Copyright 2004 Sun Microsystems, Inc. All rights reserved.
# Use is subject to license terms.
#
# ident "@(#)zones-index 1.2 04/04/01 SMI"
#
# DO NOT EDIT: this file is automatically generated by zoneadm(1M)
# and zonecfg(1M). Any manual changes will be lost.
#
global:installed:/
ffxi-sites-zone:installed:/export/zones/ffxi-sites:6cffc060-de2f-c972-f548-f36320bcfccf
root@static:/>
—

“ffxi-sites-zone” is the name of my zone. enter the name of your clone here.
“installed” is the zone’s status. Our filesystems contains a bootable
zone, so “installed” is okay. If you configured a new zone manually and
didn’t install it using `zoneadm` yet, it’ll be “configured” … then
deploy a new, bootable zonepath and set the mode to “installed”. I use
this because I use completely installed zone templates for deploying new
zones, that’s much faster than the official (and supported) `zoneadm -z
<zone> install` way.
/export/zones/ffxi-sites is the zonepath. you may change it. Just make
sure the new zonepath exists and it has mode 700. The latter is very
important.

Make sure you made a backup of your orginal before editing it manually.
You edit it on your own risk.

The entry for a cloned zone may look like this:

—
ffxi-sites2-zone:installed:/export/zones/ffxi-sites2:6cffc060-de2f-c972-f548-f36320bcfccf
—

Okay, and now forget what I told you, because Sun won’t give you the
least bit of support if something goes wrong The fact that I use
such methods for production use won’t help you much if you have a typo
where you shouldn’t have made one.

Hope you had some fun.”

Thanks for the submission Ralf!

#!/bin/bash
# backup_zfssnap.sh, (c) 2007 ralf [dot] ramge [at] webde [dot] de

BACKUPDIR="/export/backup/snapshots"
DSTAMP=`date '+%y%m%d-%H%M%S'`
FILESYS=$1
DEST=$2
REPLICA=$3
BACKUPNAME=`echo $FILESYS | sed 's/\//_/g'`
BACKUPFILE=$BACKUPNAME"-"$DSTAMP".zfs"
SNAPSHOT=$FILESYS"@backup-"$DSTAMP

if [ ! -d $BACKUPDIR ]; then
echo "Backup Directory doesn't exist"
exit 1
fi

cd $BACKUPDIR

# Check here if we have 7 backup files, create them if we don't
COUNT_FILES=`ls -1 $BACKUPNAME* | wc -l`
if [ $COUNT_FILES -le 1 ]; then
for COUNT in 1 2 3 4 5 6 7
do
if [ ! -f $BACKUPNAME"-000000-00000"$COUNT".zfs" ]; then
touch $BACKUPNAME"-000000-00000"$COUNT".zfs"
sleep 1
fi
done
fi

# Check here that we have less than 8 backup files
COUNT_FILES=`ls -1 $BACKUPNAME* | wc -l`
if [ $COUNT_FILES -gt 7 ]; then
# echo "More than 7 backup files exist"
# exit 1
while [ $COUNT_FILES -gt 7 ]
do
OLDEST_BACKUP_FILE=`ls -rt1 $BACKUPNAME* | head -1`
rm $OLDEST_BACKUP_FILE
let COUNT_FILES=COUNT_FILES-1
done
fi

# Find the oldest backup file to delete
OLDEST_BACKUP_FILE=`ls -rt1 $BACKUPNAME* | head -1`

# Create the snapshot
zfs snapshot $SNAPSHOT

# Create a filesystem image in the local backup directory
zfs send $SNAPSHOT > $BACKUPDIR"/"$BACKUPFILE

# Check for $2 and, if exists, create a second copy on a remote host for tape archival
if [ ! -z $2 ]; then
`zfs send $SNAPSHOT | ssh root@$2 "cat >$BACKUPDIR/$BACKUPFILE"`
fi

# Check for $3 and, if exists, mirror the filesystem on the remote host
if [ ! -z $3 ]; then
`ssh root@$2 "zfs receive $3 < $BACKUPDIR/$BACKUPFILE"`
fi

# Check for $4 and, if exists, clone and promote the filesystem on the remote host
if [ ! -z $4 ]; then
`ssh root@$2 "zfs clone $SNAPSHOT $4; sleep 30; zfs promote $4"`
fi

# Get the trash out of the house
rm $OLDEST_BACKUP_FILE
if [ ! -z $2 ]; then
ssh root@$2 "rm $BACKUPDIR/$OLDEST_BACKUP_FILE"
fi

SNAPLIST=`zfs list -H | grep $FILESYS | grep @backup | cut -f1`
for i in $SNAPLIST; do
zfs destroy $i
done

# Exit cleanly
exit 0

Thanks for the submission Ralf! (I changed your email address in the script comments so you wouldn’t get spam)