Please consider donating:


Using DBI for solving Reverse Engineering 101 – Newbie Contest from eLearnSecurity


Last weekend I had some time so I wanted to have a look at a reversing challenge which you can find here:

Reverse Engineering 101 Contest Steps

  1. Get the exe to be hacked
  2. Break it open and start exploring.
  3. The only rule for the challenge is that it has to be solved by creating a valid key file named eLearnSecurity.dat. No patching is allowed! Test your dat file using the exe.

Download the exe here:

Static analysis with IDA

A big giveaway is the following rule:

“The only rule for the challenge is that it has to be solved by creating a valid key file named eLearnSecurity.dat”.

That means the binary is reading that file sooner or later. So let’s open the binary in IDA and check the imports. Maybe we find something like ReadFile.


So here the binary reads the file:


We also can see that after the function GetFileSize there’s a compare against 10h which means, the size of the file is 10h = 16 bytes.

Scrolling down a little bit we can find the crypto algorithm right after reading the file:


Since ESI points to the buffer where the file was written in memory, the file should contain 4 dwords like this:


which is represented in memory by this buffer:


So the algorithm is:

dword4 ^ dword3 ^ dword2 ^ dword1 ^ key1 ^ key2 ^ key3 ^ key4 ^ key5 == 4C833425

From now on it’s pure mathematics and some fiddling…………..

We can influence dword1 to dword4, key1 to key5 are computed at compile time, run time or whatever. We don’t have to care about that at the moment (getting back to key5 soon!) since they are constant for each run of the program, that means key1 ^ key2 ^ key3 ^ key4 ^ key5 is a constant value for each run of the binary.

And this means the following:

dword4 ^ dword3 ^ dword2 ^ dword1 ^ const_Value == 4C833425

So we only need to know the value of const_Value at runtime!

But how to get it?

Well, the check is done here after the crypto algorithm:

.text:0x004014D9 cmp ecx, 0x4C833425 


ecx = dword4 ^ dword3 ^ dword2 ^ dword1 ^ const_Value

We know: A ^ 0 = A

So if we provide a file with only null bytes then at the time the compare happens we will have in ecx:

ecx = 0 ^ 0 ^ 0 ^ 0 ^ const_Value = const_Value 

So we just have to provide a file containing only null bytes, start the binary in a debugger, set a breakpoint at that compare, and check what’s in ecx, right?

Well, it’s not that easy here as we will see soon! But anyway, as an exercise let’s try that out…

Dynamic analysis with Immunity

Starting the binary in a debugger, you are landing here:


Unfortunately it’s horrible to step through the binary since there is a fair amount of anti-debug techniques in place, like this one:


Of course, we can set a BP on the exception handler, and continue stepping through the binary. But we don’t want to lose time on anti-debugging. So let’s search for the ReadFile API:


Double-click on the first ReadFile entry:


Ok, what now??? Because we never get here when running the binary in a debugger due to anti-debug stuff, I suggest to place instructions for an infinite loop here (EB FE), to run the binary and then to attach a debugger to it.

Here are the steps:

[*] Patch the binary with an infinite loop:

CTRL-E, write down the original bytes which we are changing now (85 C0) to EB FE:


[*] Then save the patched file:

  1. Copy to executable -> all modifications
  2. Save file: “elearnsecurity_eh-net_re_challenge2013_patched.exe”

[*] Now run the patched binary and attach a debugger to its process:


Important: wait around 10 – 15 seconds before attaching the debugger …..the anti-debugging stuff is running now……

[*] Then start ImmunityDebugger or OllyDbg, attach to the process, press F9 to start and then press F12 to pause the application:

-> you land our our loop (EB FE):

[*] Set a BP here (F2) and restore the original bytes :


[*] Now we can step through the crypto algorithm (F8) till the compare instruction:


So here is ECX = A50DA5E3.

But I’m sorry, this is worth nothing since key5 – the value at DWORD PTR DS:[41B2C4] – is generated by some kind of checksum during runtime. If you patch the binary its value is different. Ouch. So if we change even more bytes in the same routine in that binary we receive

ECX = 2AF4A100.

What now?

Well, checking the rules again we recognize: no patching is allowed!

So anyway we are not allowed to patch the binary therefore we need to find another way to solve it – DBI to the rescue !!!

Analyzing with DBI (PIN)

I then wrote a small PIN Tool which traces all compare instructions and logs the values of ECX into a logfile.

pin –t PINtrace_INS.dll -- E:\work\elearnsecurity_eh-net_re_challenge2013\elearnsecurity_eh-net_re_challenge2013.exe

(you can find the source code for the pin tool at the bottom of this post)

The logfile will look like this:

0x00406F8A cmp ecx, dword ptr [0x41a1a0] ECX:0x0041A0C8
0x00406F8A cmp ecx, dword ptr [0x41a1a0] ECX:0x0041A0C8
0x00406F8A cmp ecx, dword ptr [0x41a1a0] ECX:0x0041A0C8
0x00411BA4 cmp ecx, eax ECX:0x0018F70C
0x004041D4 cmp ecx, dword ptr [0x419680] ECX:0x6C6DFEF0
0x00406F8A cmp ecx, dword ptr [0x41a1a0] ECX:0x0041A0C8
0x00407012 cmp ecx, ebx ECX:0x000000FF
x00407012 cmp ecx, ebx ECX:0x000000FE
0x00407012 cmp ecx, ebx ECX:0x000000FD
0x00407012 cmp ecx, ebx ECX:0x000000FC
0x00407012 cmp ecx, ebx ECX:0x000000FB
0x00407012 cmp ecx, ebx ECX:0x000000FA

Almost at the end of the logfile we find the result for the important compare instruction:

0x004014D9 cmp ecx, 0x4c833425 ECX:0xCF6FC79D 

Note: ECX is different than in our analysis with ImmunityDebugger.

So this means:

const_Value = 0xCF6FC79D

-> dword4 ^ dword3 ^ dword2 ^ dword1 ^ 0xCF6FC79D = 0x4C833425

-> dword4 ^ dword3 ^ dword2 ^ dword1 = 0x83ECF3B8

(0xCF6FC79D ^ 0x4C833425 = 0x83ECF3B8)

If we deliberately choose

dword1= 0, dword2 = 0, dword3 = 0


dword4 = 0x83ECF3B8

Ok, let’s construct the appropriate dat-file with our favorite hex editor (mind the little endian nuisance):


Run the binary and voila:



If you spend enough time you could solve this challenge by pure reversing but sometimes it’s just pita, especially when heavy anti-debugging techniques are at work.

Sometimes you just need a small piece of information out of the runtime context. Here DBI is really the best solution. DBI or Dynamic Binary Instrumentation (note: it doesn’t mean “Dull But Important”) is an easy and fast way to overcome reversing barriers like anti-debugging techniques or other obstacles. Other typical usage examples are taint analysis, tracing, unpacking code, examining memory or registers, fuzzing, capturing code, and so on. So my recommendation to each reverser is: please have a look at DBI.

Thanks !

My big thanks go to Peter (corelanc0d3r) for giving me the insight of DBI. Also I’d like to thank Carlos (m0n0sapiens) for teaching RE to the world. You are both great guys I appreciate very much!!


Source code


#include "pin.H"
/* ===================================================================== */
/* Global Variables */
/* ===================================================================== */
std::ofstream TraceFile;

/* ===================================================================== */
/* Commandline Switches */
/* ===================================================================== */
KNOB<string> KnobOutputFile(KNOB_MODE_WRITEONCE, "pintool", "o", "trace.out", "specify trace file name");

/* ===================================================================== */
/* Print Help Message */
/* ===================================================================== */

INT32 Usage()
cerr <<
"This tool logs instructions and register values\n"
"which you can determine in the source code.\n"
cerr << KNOB_BASE::StringKnobSummary();
cerr << endl;
return -1;

/* ===================================================================== */
/* Logging function */
/* ===================================================================== */

VOID logCall(const string *s, CONTEXT * ctxt)
char ss [14]; // EIP
sprintf (ss, "0x%p \n",EIP);
char sp [14]; // ECX
sprintf (sp, "0x%p \n",ECX);
TraceFile.write(ss, 12);
TraceFile.write(s->c_str(), s->size());
TraceFile.write(" ECX: ",10);
TraceFile.write(sp, 12);

/* ======================================================================== */
/* Function to determine if an address is part of the main executable image */
/* ======================================================================== */

bool isPartOfMainImage(ADDRINT address)
for (IMG img=APP_ImgHead(); IMG_Valid(img); img = IMG_Next(img))
if (IMG_IsMainExecutable(img))
for (SEC sec=IMG_SecHead(img); SEC_Valid(sec); sec = SEC_Next(sec))
if (address >= SEC_Address(sec) && address < SEC_Address(sec) + SEC_Size(sec))
return true;
return false;

/* ===================================================================== */
/* instrumentation function */
/* ===================================================================== */

void Eval(INS ins, void *v)
/* instruction to look for */
string findString = "cmp ecx"; // .text:004014D9 cmp ecx, 4C833425h
ADDRINT address = INS_Address(ins);
if ( INS_Disassemble(ins).find (findString) != string::npos && isPartOfMainImage(address) )
std::string asminstruction; 
asminstruction = INS_Disassemble(ins).c_str();
INS_InsertCall(ins, IPOINT_BEFORE, (AFUNPTR)logCall, IARG_PTR, new string(asminstruction), IARG_CONTEXT, IARG_END);

/* ===================================================================== */
/* Wrap up log when all is said and done */
/* ===================================================================== */

VOID Fini(INT32 code, VOID *v)
TraceFile << "#eof" << endl;

/* ===================================================================== */
/* Main */
/* ===================================================================== */

int main(int argc, char *argv[])
INS_AddInstrumentFunction(Eval, 0);
PIN_AddFiniFunction(Fini, 0);
return 0;

© 2013 – 2021, Corelan Team (fancy). All rights reserved.

One Response to Using DBI for solving Reverse Engineering 101 – Newbie Contest from eLearnSecurity

Corelan Training

We have been teaching our win32 exploit dev classes at various security cons and private companies & organizations since 2011

Check out our schedules page here and sign up for one of our classes now!


Want to support the Corelan Team community ? Click here to go to our donations page.

Want to donate BTC to Corelan Team?

Your donation will help funding server hosting.

Corelan Team Merchandise

You can support Corelan Team by donating or purchasing items from the official Corelan Team merchandising store.

Protected by Copyscape Web Plagiarism Tool

Corelan on Slack

You can chat with us and our friends on our Slack workspace:

  • Go to our facebook page
  • Browse through the posts and find the invite to Slack
  • Use the invite to access our Slack workspace
  • Categories