Anti-debugging techniques have been known for a long time. One way to detect the presence of debuggers (and also DBI frameworks like Pin, emulators, etc) is by measuring the time taken to execute a piece of code and comparing it against a maximum tolerated amount of time. This way we can detect if our code is being debugged/instrumented/emulated by detecting the latency introduced by the instrumentation tool.
In this Reverse Engineering StackExchange thread you can find a set of time-related functions that are commonly used in Windows environments in order to detect the presence of debuggers using time-checking strategies:
- RDTSC instruction
- RDPMC instruction
You can find more details on these time-based antidebugging tricks on “The Ultimate Anti-Debugging Reference” by Peter Ferrie [PDF].
So a few weeks ago I found one more way to perform time-based debugger detection; the (mildly) interesting thing is that it doesn’t use functions that are directly related to time stuff, so it can be a bit more subtle than those well-known time functions.
The technique is based on the Windows Kernel Transaction Manager, which can be used to implement both transacted file operations and transacted Registry operations.