Malware analysis: Gargoyle, a memory scanning evasion technique

Gargoyle is a memory analysis evasion technique that uses return-oriented programming (RoP) for hiding all of a program’s executable code in non-executable memory when it is inactive, and temporarily mark it executable to do some work at a pre-defined interval (every 15 second, in the PoC).

Return-oriented Programming (RoP) attacks

Return-oriented programming is an exploit technique that allows an attacker gains control of the call stack to hijack program control flow and then executes carefully chosen machine instruction sequences that are already present in the machine’s memory, called “gadgets“.

Each gadget typically ends in a return instruction and is located in a subroutine within the existing program and/or shared library code.

Chained together, these gadgets allow an attacker to perform arbitrary operations on a machine employing defenses that thwart simpler attacks.


The code, written by Josh Lospinoso, uses Windows Asynchronous Procedure Calls to woke up the RoP job.

Asynchronous Procedure Calls on Windows

An asynchronous procedure call (APC) is a function that executes asynchronously in the context of a particular thread.

When an APC is queued to a thread, the system issues a software interrupt.
The next time the thread is scheduled, it will run the APC function.

APCs can be generated by the system (kernel-mode APCs) or by an application (user mode APCs).

How Gargoyle works

Gargoyle writes a payload into read/write only memory, then using an Asynchronous Procedure Call based on a timer that calls a RoP gadget to invoke VirtualProtectEx function to change protections to RWX.
With this new attribute, the code area is executable so, Gargoyle execute the payload and after it again calls VirtualProtectEx to return to RW protections, in order to further evade detection.

Here a great Infographic by Josh Lospinoso:

Really hard to discover

Gargoyle takes advantage of the fact that live memory analysis has a cost that a program must reside in executable memory (less than 10% of total memory on most systems) meant an analyst could safely limit analysis there in order to obtain better performance then a full memory scan.

Using Gargoyle, the code is read-only during most of its life and the APC call used to wake Gargoyle is not something that can be trivially enumerated: a challenge to malware analyst!

Luckily, currently is only a proof of concept and no use of the technique has been seen in the wild.

However, some tools/techniques can be successful used in order to detect and analyse RoP attacks:

  • ROPEMU: a framework to analyze, dissect and decompile complex code-reuse attacks. It allows to dump a complex RoP execution trace into an ELF file.
  • can perform a stack checking functon against the memory dump.
  • Eh’Trace: a binary tracing tool for Windows.
  • RoPGuard: runtime prevention of return-oriented programming attacks

References and further readings


This site uses Akismet to reduce spam. Learn how your comment data is processed.