Tutorial: Finding the OEP of an Upacked binary file

February 25, 2008

…because all the other tutorials I’ve been able to find on this subject are not so easy to read.

This is going to be a long post, but hey, at least it’ll have lots of pictures!

Alright, in this tutorial I’m going to attempt to explain how to find the OEP (Original Entry Point) of a binary executable that has been packed with the Upack/WinUpack packer. I just recently learned this myself, so please excuse any errors this tutorial might have. In this tutorial, the following tools are used:

Sadly, I haven’t figured out how to get this working the same way in Ollydbg yet, but perhaps for a later tutorial. Alright, let’s dive right in. In this example, I’ll be using the “calc.exe” application (Windows calculator) that I packed with WinUpack. The first thing to do is load the file into PEiD to try and determine what kind of packer was used on the file:

0peid.png

You can see in the red square above that this file was packed with WinUpack 0.39 final, which is good because that’s what this tutorial is about ;)

The next thing to do is load the executable into IDA-pro, IDA will complain about the file, but for the most part these complaints can be ignored, just click “ok” and “yes” until you see something similar to below:

1idastart.png

Here, you can see the start of the file, the line we’re most interested in is (in this file), the line at 0100101F, which is a "push dword ptr [esi+34h]” instruction. Select this line and hit F2 to toggle a breakpoint on this line. The line should highlight red just like the picture below:

2pushbp.png

Now, press F9 to run the file until it encounters the breakpoint. When IDA does encounter it, the display will change to the following style, take note of the IDA-view ESP box highlighted in red below, that’s what’s going to be important coming up:

3running.png

Right click on the ESP address (in this case 0007FFC0) and set a breakpoint, we want to set a hardware breakpoint or size 4 on this address, see the picture below for what I mean:

4hwbp.png

After setting it, the line will highlight red like this:

5hwbp2.png

When the breakpoint has been set, continue running the file by hitting ‘F9′ again. When IDA hits the breakpoint, you’ll see the following two windows:

6afterhwbp.png

Notice the red box, this is our new OEP: 01012475. Write this number down somewhere (or remember it). Now that we know our OEP, we need to dump the file and fix the imports, the first step to dump the file is to open up LordPE and select the running calc.exe process, as seen below:

7lordpe.png

Right-click on the file and click on “dump full”, as you see here (I saved my file as calc-dump.exe):

8lordpedump.png

Here you can see a comparison of the sizes of the packed and unpacked files:

9sizecmp.png

Let’s run the file and see what we get, oops, looks like we get the error below:

10iatmissing.png

This means the import table is all messed up, we’ll have to fix it. In order to do that, fire up ImpRec and point it at the active calc.exe process, in the OEP box, enter the OEP we found earlier (just the offset, which in this case is 12475):

11imprecnewoep.png

Then, click on “IAT Autosearch”, you should see a message like this:

12autosearch.png

Click on “Get Imports” and the window should be filled with a list of imports found in the file, like this:

13importsfound.png

In order to fix the dump, click on “Fix Dump” and select the file you dumped earlier (in my case it was calc-dump.exe), ImpRec will fix the dump and save a new file, if it works correctly, you should see these messages in the log:

14fixdump.png

Let’s compare these size of all 3 of these files:

15sizecmp2.png

And now, try and run the file:

16running.png

Hurray! It worked! From here you can do everything you need with the file, since it is no longer packed. I hope this helps someone, it certainly helped me understand unpacking a little bit better. Now I can get to unpacking that malware I captured earlier…

3 Comments to "Tutorial: Finding the OEP of an Upacked binary file"

  1. Aleksey F. wrote:

    Great job, Olly instructions should be about the same. Load it up, ignore all the warnings, single step until you see the ESP change (turns red), right click on ESP and select “Follow in Dump”, highlight the first 4 bytes in the hex dump window, right click on them, select Breakpoint -> Hardware on access ->Dword, run, when it breaks it should be on the OEP or jump to OEP. This description of course is very generic. I actually prefer to do this kind of stuff in Olly, I didn’t even know you can step through in IDA, so thanks!

  2. upack wrote:

    […] not so easy to read. This is going to be a long post, but hey, at least it??ll have lots of picturhttp://writequit.org/blog/?p=150ABF Freight System, Inc – Wikipedia, the free encyclopediaFor a typical U-Pack move 3, ABF delivers […]

  3. :wq - blog » Blog Archive » Example malware unpacking and analysis: part 1, unpacking wrote:

    […] reverse engineering has always been incredibly interesting to me and I noticed that ever since my OEP finding tutorial for UPACK, I’ve also gotten a lot of google searches for “how to reverse malware” and other […]

 
Powered by Wordpress and MySQL. Theme by Shlomi Noach, openark.org