I started out writing the script in Perl, which at this point is the scripting language I know the best, I then decided that now is as good a time as any to learn Ruby (which I’ve been interested in for a while now), so I re-wrote the entire program in Ruby (first *useful* script I’ve actually written in Ruby). Then, one day at work I was told that I needed to extend the program to support 5000 simultaneous threads doing disk I/O. I thought about this for a while and (after talking with my friend Jon about it) decided on using Java, as the threading was much more robust (something I had problems with using Ruby and Perl). Well, I’ve got working versions of all 3 programs and I thought I’d share my perspective on the pro’s and con’s of each one:

Java pro’s:

• Most robust thread implementation of the 3 languages
• Handles SMP much better than ruby
• Code is portable with minimum requirements to run
• OO language (a bigger pro to actual developers who this matters more to)

Java con’s:

• JVM overhead (not really that much nowadays)
• More difficult to read due to Java’s extreme verbosity
• Requires jdk 1.5+ (1.4 is still the only actual “supported” JDK in my company)

Ruby pro’s:

• Most readable code of all 3 (shortest too)
• I got to learn Ruby
• Ruby implementation available for most platforms
• More OO than Perl (not that I used OO…)

Ruby con’s:

• Ruby only took advantage of 1 of my CPU cores (Java used both)
• Ruby is slower than Perl (maybe one day they’ll be just as fast?)
• Almost no one in my department has heard of Ruby

Perl pro’s:

• Super-easy to install with ActiveState for windows, comes default with most *nix
• Super-easy to install the required module: perl -MCPAN -e shell ; install File::Random
• Allows fine-grain tuning of thread parameters (adjustable thread stack size)

Perl con’s:

• If you don’t have threaded perl, gotta reinstall (/cry @ Solaris)
• Least readable code (unless you loooovvve punctuation)
• Perl doesn’t like me spawning millions of threads and detaching all of them

Overall, since I need code that’s portable to multiple platforms easily, while allowing for very large amounts of IO, I’ll probably stick to the java version (which was renamed CIOSim [Customer I/O Simulator] because you actually pronounce it :P), followed by the Ruby version (so easy to write), and then the Perl version, which, actually has the largest amount of features.

I haven’t written all the features into each version yet (except for the Perl one), but, if you’d like to take a look at them, here they are:

Java version
Ruby version
Perl version

Next tool I need to write, I’ll probably be looking at Ruby

Anyone out there use anything different for sysadmin tools? Python? Lisp? Assembler? Leave a comment and let me know

./phfos.pl -d <dir> -r -v --min=2 --max=3 -n 1000

We’re actually guessing that after allocating too many threads, since we aren’t immediately exiting the thread after reading (what –readstop does), we allocate past the amount of stack space perl allocated us, giving us the EXC_BAD_ACCESS (0x0001) error.

Well, how do we fix that? Turns out perl has an option to set individual stack sizes per thread, so if you decrease the size of the stack per thread, we could allocate more threads. Only 1 problem, on OSX

perl -e'use threads; print(threads->get_stack_size(), "\n")'

Gives you:

Can't locate auto/threads/get_stack_s.al in @INC (@INC contains: /System/Library/Perl/5.8.8/darwin-thread-multi-2level /System/Library/Perl/5.8.8 /Library/Perl/5.8.8/darwin-thread-multi-2level /Library/Perl/5.8.8 /Library/Perl /Network/Library/Perl/5.8.8/darwin-thread-multi-2level /Network/Library/Perl/5.8.8 /Network/Library/Perl /System/Library/Perl/Extras/5.8.8/darwin-thread-multi-2level /System/Library/Perl/Extras/5.8.8 /Library/Perl/5.8.6 /Library/Perl/5.8.1 .) at -e line 1

Anyone else can read about setting stack sizes for perl scripts here (look for the THREAD STACK SIZE section)

Now I need to somehow figure out how I’m going to set thread stack size without being able to see what the values are…

Suggestions?