Malware Obfuscation Part 2

Introduction#

In this post I explore several malware obfuscation techniques used to evade antivirus detection. It’s part of my ongoing Malware Development series where I discover the world of malware development.

⚠️ Warning: This content is for educational and defensive security research purposes only. Do not use these techniques on systems or networks you do not own or have explicit permission to test.

Windows applicatons#

Windows application make use of dynamic-link libraries (DLL). These are libraries in Microsoft Windows that can contain code, functions, data and resources. The kernel32.dll is one of core DLL files in Windows. It handles core functions such as memory management, file input/output, and process/thread creation.

Previously we saw that shellcode is loaded into memory through VirtualAlloc. VirtualAlloc is one the function from the kernel32.dll as seen on the image below. VirtualAlloc was imported to launch the shellcode in the malware. Antivirus solutions tend to analayse indicators such as VirtualAlloc in DLLs since malware uses these functions.

Function call obfuscation#

Because some functions are commonly used by malware and may be blacklisted, malware developers use function call obfuscation to hide the usage of these functions. The Windows API functions GetModuleHandle and GetProcAddress enables us to hide the external functions of DLLs.

• GetProcAddress: Retrieves the address of an exported function in the DLL.
• GetModuleHandle: Retrieves a module handle for the specified module. The module must have been loaded by the calling process. For example, loads the DLL.

By calling the GetProcAddress, it is possible to retrieve the exported function from the specified DLL during runtime. This means that the function does not appear in the PE’s static import table. By encrypting the string of the function, it will be possible to avoid static analysis of the binary.

The GetProcAddress function call provides a function pointer which type is described as the following in the Microsoft docs:

LPVOID VirtualAlloc(
  [in, optional] LPVOID lpAddress,
  [in]           SIZE_T dwSize,
  [in]           DWORD  flAllocationType,
  [in]           DWORD  flProtect
);

In the C++ code, a matching type for the function type needs to be created:

typedef LPVOID(WINAPI* VirtualAlloc_t )(
    LPVOID,
    SIZE_T,
    DWORD,
    DWORD
);

VirtualAlloc_t  pVirtualAlloc = (VirtualAlloc_t) GetProcAddress(GetModuleHandle("kernel32.dll"), "VirtualAlloc");

However running strings on the binary still reveals VirtualAlloc, which means it is detectable by static analysis.

XOR encrpytion#

To hide the string VirtualAlloc, malware developers make use of XOR (exclusive OR) encryption. XOR encryption is a symmetric encryption algorithm by performing a bitwise XOR operation on a plaintext with a secret key. It works as the following:

1. Each character of the string is represented as a byte.
2. A bitwise XOR is performed between the plaintext byte and the key byte:

0 0 1 1 (plain text byte)
0 1 0 1 ( key byte)
------- ( XOR )
0 1 1 0 (encrypted byte)

3. Result: sequence of encrypted byes that no longer contains the plainttext string.

By encrypting the string VirtualAlloc at compile time and decrypting it at runtime, it is possible to hide the string for static analysis of the the binary. The following Python script uses XOR encryption to encrypt a string and outputs the result as a C++ byte array, ready to be embeded in the source code of the malware.

def xor_encrypt_to_cpp_array(text, key):
    encrypted_bytes = []

    for i in range(len(text)):
        encrypted_byte = ord(text[i]) ^ ord(key[i % len(key)])
        encrypted_bytes.append(encrypted_byte)

    # format as C++ array
    cpp_array = ", ".join(f"0x{b:02x}" for b in encrypted_bytes)

    print("unsigned char data[] = {")
    print("    " + cpp_array)
    print("};")


# Example usage
text = "Hello"
key = "key"

xor_encrypt_to_cpp_array(text, key)

The following C++ function decrypts the XOR-encrypted byte array at runtime with using the same key (symmetric) as defined in the Python code.

void xorDecrypt(const unsigned char* data, char* output, size_t len, const char* key) {
    size_t keyLen = strlen(key);

    for (size_t i = 0; i < len; i++) {
        output[i] = data[i] ^ key[i % keyLen];
    }

    output[len] = '\0'; // important: null terminate
}

Succesful function call obfuscation#

The following code demontrates a complete call to get the function pointer of VirtualAlloc using XOR decryption:

 unsigned char cVirtualAlloc[] = { 0x25,.... HEX values };
 const unsigned int cVirtualAllocLen = sizeof(cVirtualAlloc);

 const char* key = "secret";
 char decrypted[cVirtualAllocLen + 1]; // +1 for null terminator
 
 xorDecrypt(cVirtualAlloc, decrypted, cVirtualAllocLen, key);

 VirtualAlloc_t pVirtualAlloc = (VirtualAlloc_t)GetProcAddress(GetModuleHandle("kernel32.dll"), decrypted);

VirusTotal#

After compiling the binary and uploading it to VirusTotal, 7 out of 72 anti-virus solutions flagged it as malicious.

VirtualAlloc does not appear in the PE’s static import table, meaning that the function is resolved through runtime.

Conclusion#

Most antivirus solutions detect functions such as VirtualAlloc as malicious. In this blog post demonstrated how malware developers use function call and XOR string obfuscation to evade static analysis of a binary. In the next part of the series, sandbox evasion techniques will be explored.