A Word for OllyDbg
September 11th, 2005
OllyDbg is a very nice tool for debugging other people’s code. While I definitely still prefer WinDbg in most situations, OllyDbg is great for stepping through assembly.
I had a good reason to use it last week. I have an old VB6 application that needs to interop with new .NET applications. The VB6 code is using an ancient COM library for encryption; this library is fairly opaque in regards to what it is actually doing.
Without giving away too many details, there’s no way I can move the old code to over to anything else. I wish the original author had just imported the advapi32 Crypt* functions, but it’s too late for that now.
So, needing to decrypt data coming from the VB6 side, I was left with a few choices. I could just use COM Interop and reference the old library in my new code. But I wasn’t really happy with that, mostly because of the complexity that it adds to deployment.
I was familiar enough with the Crypto API to know that the COM library couldn’t be doing anything too complicated. This is where OllyDbg comes in.
I stepped through the library call in assembly, stopping when it made Crypt* calls. I got the parameters from the stack, and wrote them all down.
From there, I wrote a quick C++ console app to test out recreating the calls and decrypting some sample data. As it turned out, the specific algorithms that the COM library was using weren’t exposed in System.Security.Cryptography, so the C# version I ended up with had to use P/Invoke with advapi32.
But anyway, I got rid of an annoying dependency. A very satisfying hack. You can read about an even better one here—Lee Holmes uses OllyDbg to crack a program to run as a non-admin.
