Windows system call parameter analysis

Posted on January 22, 2018 by Martin Messer

Due to its introspection capabilities, the Cyberus virtualization platform is able to analyze Windows system calls. In this article we demonstrate how simple it is to extract system call parameters out of a running windows machine with Python using the Tycho API.


In order to access this feature via Python, we need the following prerequisites:

  • The typical Cyberus Setup. If you are not familiar with it, please have a look at our blog article Fun with Python and Tycho.
  • We need an executable on the analysis system whose system calls we can intercept. For demonstration purposes we decide to use the open source sandbox and analysis detection tool pafish.

Intercepting system calls

To analyze Windows system calls there are two possibilities:

  1. Adding breakpoints into the Windows libraries, just before they perform a syscall instruction.
  2. Letting the CPU interrupt the instruction flow on actual syscall instructions.

The first method can easily be detected by malware samples, because breakpoints are visible manipulations in memory. Breaking on syscall instructions is not possible with usual debuggers, but the Cyberus virtualization platform provides such a mechanism.

Please note that we will stick to the debugger terminology of “setting breakpoints on system calls” even if we are not really writing breakpoint instructions over syscall instructions.

Get Arguments of NtCreateFile

Let’s assume that the system call NtCreateFile is the most interesting one for our purposes when analyzing pafish.exe. We will show the Python Tycho API workflow by detecting this system call and carving out its arguments.

First, we open a python shell on the developer laptop and import some additional libraries to get system call breakpoints and system call analysis functionalities. In the next step we connect to Tycho service and open the pafish process.

Attention: In contrast to previous posts, we have currently not started the process yet. This time, we first instruct Tycho to open the process handle. The process handle will start to work as soon as the process is started.

To get the first occurrence of NtCreateFile, we have to pause pafish on startup and wait until it is scheduled the first time.

The break_on_start line tells Tycho to stop pafish.exe even before it executes its first instruction. In order to see when it “clicks in”, we run a loop that sleeps until pafish.exe is scheduled for the first time and paused.

At this point, we are ready to launch the pafish.exe executable on our analysis box. pafish.exe does usually open a command line window on startup and prints its output. In our case, the window will not show up because Tycho interrupts it before it executes its first instruction. Lets look at the Windows Task Manager, to ensure that pafish.exe is already started:

Pafish was already started, but is currently paused by Tycho Python API.
Pafish was already started, but is currently paused by Tycho Python API.

We can see that the pafish.exe process burns some CPU cycles without being visible. This is a side effect of our method to interrupt user threads without cooperation from the Windows kernel.

Back at the developer laptop, we see that the wait loop has terminated. We can now start to configure system call breakpoints and system call interpretation. At first, we acquire a system call breakpoint from our process handle object. Then, we configure the breakpoint in that way that it interrupts pafish.exe at the next NtCreatefile system call. Afterwards, we just let the process continue so we can wait for it to call NtCreateFile:

After returning from wait_for_breakpoint, we hold a handle object thread_handle that refers to the user thread of the pafish.exe process that tripped onto our system call breakpoint.

On the analysis box, we can see that the pafish console window popped up and printed some output already:

Pafish paused during system call breakpoint hit.
Pafish paused during system call breakpoint hit.

While we claim that the Cyberus virtualization platform is invisible to guest processes, the attentive reader might have seen that pafish prints the CPU brand string >>> Running on SuperNOVA <<<. This string comes from our internal test version of the virtualization platform binary and is not visible in the product binary, where only the original CPU brand string is visible.

Having a handle object to the interrupted thread, we can now analyze the system call’s parameters. We can use the interpret_syscall function, which we have imported already, to carve out all parameters of this system call. This function uses the guest thread’s register- and stack state to automatically access all relevant information.

When we call interpret_syscall, Tycho will access the relevant pieces of the thread’s registers and stack memory, in order to provide us with a complete list of argument values:

Of course we get no return value as well as no output parameter values, because this system call has not been executed, yet.

In order to get these, we can use interpret_execute_syscall, which executes the system call and pauses the thread immediately after it returns from kernel space:

Have a look at the FileHandle parameter: NtCreateFile assumes the address of a pointer to a file handle, which it can then initialize with the handle value of the newly created file. Here we see that the address (value 582376) is dereferenced and the file handle value (value 172) is shown, too.

This is another built in feature of the Tycho Python API system call interpretation engine: If a parameter within a set of previously configured pointer types, it will automatically be inspected, too. We will give more information on that in upcoming blog posts.


As we can see, the Python Tycho API enables us to carve all parameters out of system calls with roughly 10 lines of Python code!

The system call interpretation feature of Tycho can do the following:

  • interrupt system calls on by letting the CPU break on the actual syscall instruction. This is a completely invisible way to breakpoint on system calls.
  • Read all parameters out of the system call before it is executed by the Windows kernel.
  • Break upon return from the syscall instruction and also read output parameters as well as the return value.
  • The user can manipulate input/output parameters as well as the system call’s return value.
  • Of course, it is also possible to completely deflect the system call and fake the kernel’s return values.

While for demonstration purposes we concentrated on the NtCreateFile system call, a complete set of low level system calls is available. It is also possible to add custom handlers to system call events that carve information more selectively.

What Next

Stay tuned for the next article, which will demonstrate how to list all files and devices that an application accesses during its life time.

Other nice ideas about what to do with Tycho system call interpretation:

  • We can allow or disallow applications to access certain file system and/or Windows registry paths by deflecting the right system calls whenever they contain path argument we don’t allow.
  • With the same strategy, we can easily whitelist/blacklist which hosts a TCP application is allowed to access.

Tycho can also be integrated with other tools. One prominent example is Cuckoo Sandbox, which can be combined with the Cyberus virtualization platform. The value of this combination lies in the indetectability, as Cuckoo can just use the Tycho system call interpretation feature instead of relying on a detectable user space agent or kernel space module. We will post more information on that in the future.