Wednesday, February 24, 2016

Windows 32-bit structured exception handling example for GCC

I recently spent a few hours learning about structured exception handling (SEH) in 32-bit (i686) Microsoft Windows. The canonical article documenting this seems to be A Crash Course on the Depths of Win32™ Structured Exception Handling by Matt Pietrek in 1997.

After reading the first 1600 words of the article, I came across a code sample (Figure 3) that shows the most basic way one can use structured exception handling. The code uses C++ and a little bit of inline assembly to add its own exception handler to the beginning of the linked list of exception handlers that is stored in the Thread Environment Block. It then generates an segmentation fault (by writing to address 0) in order to demonstrate that the exception handler can run, fix the problem, and tell the OS to resume and try the problematic instruction again.

This example did not compile in my favorite C++ development environment for Windows, which is MSYS2. MSYS2 comes with a mingw-w64 GCC compiler. I don't know much about x86 assembly but I managed to port the example and get it to run. The main thing I had to change was the syntax of the inline assembly. Here is the code:


/* Basic demonstration of Win32 structured exception handling (SEH).

This code is based on Figure 3 (MYSEH.CPP) from this article:

    https://www.microsoft.com/msj/0197/exception/exception.aspx

Originally written by Matt Pietrek for the January 1997 Microsoft
Systems Journal.

Modified in 2016 by David E. Grayson to work with gcc.
*/

#include <windows.h>
#include <stdio.h>

DWORD scratch = 10;

EXCEPTION_DISPOSITION __cdecl
_except_handler(
    struct _EXCEPTION_RECORD * ExceptionRecord,
    void * EstablisherFrame,
    struct _CONTEXT * ContextRecord,
    void * DispatcherContext)
{
    // Indicate that we made it to our exception handler.
    printf("Hello from an exception handler\n");  fflush(stdout);

    // Change EAX in the context record so that it points to some
    // place where we can successfully write.
    ContextRecord->Eax = (DWORD_PTR)&scratch;

    // Tell the OS to restart the faulting instruction.
    return ExceptionContinueExecution;
}

int main()
{
    __asm__(                        // Build EXCEPTION_REGISTRATION record:
        "push    %0\n"              // Bytes 4 to 7: Address of handler function
        "push    %%fs:0\n"          // Bytes 0 to 3: Address of previous handler
        "movl    %%esp, %%fs:0\n"   // Install new EXECPTION_REGISTRATION
        :
        : "r" ((DWORD_PTR)_except_handler)
        );

    __asm__(
        "movl    $0, %eax\n"        // Zero out EAX
        "movl    $1, (%eax)\n"      // Write to EAX to deliberately cause a fault
        );

    printf("After writing!\n"); fflush(stdout);

    __asm__(                        // Remove our EXCEPTION_REGISTRATION record
        "movl    (%esp), %eax\n"    // Get pointer to previous record
        "movl    %eax, %fs:0\n"     // Install previous record
        "addl    $8, %esp\n"        // Clean our EXCEPTION_REGISTRATION off stack
        );

    printf("scratch = %d\n", scratch);

    return 0;
}

I compiled the code in a MinGW-w64 32-bit Shell, with /mingw32/bin/g++, using the command g++ -g3 -O0 myseh.cpp. When I run a.exe, the output is:

Hello from an exception handler
After writing!
scratch = 1

So this shows that our exception handler really did work!

Using less assembly

The code above worked, but I would not use it in production because it contains inline assembly that does sketchy things to the stack and registers without telling the compiler about it. We can make it a little bitter by using functions and structs provided by Windows headers to manipulate the thread environment block (which were probably not available in 1997 when Matt Pietrek wrote his article).


/* Basic demonstration of Win32 structured exception handling (SEH).

This code is based on Figure 3 (MYSEH.CPP) from this article:

    https://www.microsoft.com/msj/0197/exception/exception.aspx

Originally written by Matt Pietrek for the January 1997 Microsoft
Systems Journal.

Modified in 2016 by David E. Grayson to work with gcc and use less assembly.
*/

#include <windows.h>
#include <winternl.h>
#include <stdio.h>

DWORD scratch = 10;

EXCEPTION_DISPOSITION __cdecl
_except_handler(
    struct _EXCEPTION_RECORD * ExceptionRecord,
    void * EstablisherFrame,
    struct _CONTEXT * ContextRecord,
    void * DispatcherContext)
{
    // Indicate that we made it to our exception handler.
    printf("Hello from an exception handler\n");  fflush(stdout);

    // Change EAX in the context record so that it points to some
    // place where we can successfully write.
    ContextRecord->Eax = (DWORD_PTR)&scratch;

    // Tell the OS to restart the faulting instruction.
    return ExceptionContinueExecution;
}

int main()
{
    NT_TIB * teb = (NT_TIB *)NtCurrentTeb();

    // Install our handler at the beginning of the list.
    EXCEPTION_REGISTRATION_RECORD reg;
    reg.prev = teb->ExceptionList;
    reg.handler = (PEXCEPTION_ROUTINE)_except_handler;
    teb->ExceptionList = &reg;

    __asm__(
        "movl $0, %%eax\n"        // Zero out EAX
        "movl $1, (%%eax)\n"      // Write to EAX to deliberately cause a fault
        ::: "memory", "eax"
        );

    printf("After writing!\n"); fflush(stdout);

    // Remove our handler from the list.
    teb->ExceptionList = reg.prev;

    printf("scratch = %d\n", scratch);

    return 0;
}

We have to use two casts in the code above. First, we cast the output of NtCurrentTeb() because the struct type it points to by default is not very useful and just has a lot of fields with names like Reserved1. Second, we cast _except_handler because there is a disagreement about whether the exception handler should use the __cdecl calling convention or __stdcall. Both calling conventions seem to work, but since the calling convention of our _except_handler function does not match the calling convention of the handler member of the struct, we need to use a type-cast to satisfy the type-checking the GCC does at compile time. I have noticed cases like this in the past, where I had to change the calling convention of something to get it to compile with GCC, but presumably it would compile fine with the Microsoft C compiler. The function prototype of _except_handler in Microsoft's headers uses __cdecl, while the type definition of EXCEPTION_ROUTINE uses NTAPI, which means __stdcall.

What about 64-bit?

Apparently structured exception handling for 64-bit Windows is done in a totally different way, so you can't easily adapt the code above to work in 64-bit mode. More info about x64 Structured Exception Handling is available from OSR Online.

What's the point?

Microsoft's C compiler emits code similar to what I have above when you use the __try, __except, and __finally keywords. GCC does not yet support these keywords, but Clang has partial support for them. I have heard (but not verified) that WebKit uses SEH, so if we want to compile WebKit with GCC, someone needs to add SEH to GCC first. Or maybe we could compile WebKit but we would have to somewhat disable its sandboxing features. I haven't actually tried to compile WebKit with GCC yet.

Friday, October 9, 2015

Tuesday, June 2, 2015

USB CDC ACM braindump

I just dumped a lot of stuff I know about USB CDC ACM devices into a StackOverflow answer. Check it out if you want to learn more about the standard way to make a USB device that has a virtual serial port.

Friday, May 29, 2015

Ways to return a string from a C function

I am working on a C library and I need to have a flexible, easy, efficient way to return strings to the user. See my gist on this topic: https://gist.github.com/DavidEGrayson/db13d38cbe5c95db64c4.

Monday, March 2, 2015

Non-bitcoin assets on the block chain are not interesting

I frequently hear people in the Bitcoin community say that money is just the first application of the Bitcoin block chain, and that they want to use it for tracking other assets like land and securities. That thinking is misguided and I would like to explain why.

First of all, let's talk about the data in the block chain. The payload data for the block chain is mainly just a bunch of blobs of structured binary data called transactions. Each transaction has a list of inputs and a list of outputs, and it specifies how much value is stored inside each output. The value inside each output is just a positive integer which represents how many bitcoins are in that output. Each transaction output also has a script which determines who can unlock the output, and the typical script will contain something called a Bitcoin address. The rest of the data in the block chain, and all the software that runs on the nodes in the Bitcoin network, is designed to fulfill one purpose: it establishes a decentralized consensus on the set of unspent transaction outputs. It does this in a way that anyone can use the network, but nearly no one can modify its data maliciously.

The block chain, and the servers that maintain it, know nothing about real estate or stocks. So when someone tells you they have developed a way to represent non-bitcoin assets on the block chain, what they are saying is that they have invented a set of rules for how to interpret the data in the block chain in order to add extra information to it about asset ownership.

These rules that they invented are not enforced in the block chain. Therefore, if you are an owner of a non-bitcoin block chain asset, then you have to trust some third party to actually give value to your asset. That third party could arbitrarily change the rules of the game some day and declare that your assets are not valid, in a very similar manner to how PayPal or Coinbase can arbitrarily freeze your account. The block chain doesn't care, so the only recourse you would have is to complain loudly in public and/or file a lawsuit. There is nothing you can actually do by yourself in order to maintain control of that asset. Therefore, that kind of asset is inferior to an actual, real bitcoin, and it is not a particularly interesting thing to talk about.

The only benefit I can see to these kind of schemes is that there would be more transparency in the ownership and transfer of these assets. But you don't need the block chain to achieve that. Imagine this alternative system instead: the central authority that everyone is trusting could simply maintain a centralized ledger themselves. They could publish a new version of the ledger on their website every hour, signed with their company's PGP key. The asset holders could use ECDSA cryptography, or any kind of authentication mechanism that the company implements, in order to authenticate themselves and issue orders for transferring the asset. This kind of system is totally possible with today's technology, and it would be cheaper to implement than a block chain-based system because you wouldn't have to download tens of gigabytes of data or pay fees to Bitcoin miners.

So why does everyone keep talking about non-bitcoin assets on the block chain and writing software and protocols for it? There are people who have spent a lot more time thinking about this than I have, so maybe I am missing something. Please enlighten me in the comments.

TLDR: The bitcoin network establishes a decentralized consensus on the set of unspent transaction outputs. It doesn't know anything about non-bitcoin assets, so you'd have to trust a third party in order to actually hold such an asset. If you are trusting a third party, there is no real reason to use the block chain.

Sunday, February 15, 2015

Ruby blocks with brackets are parsed differently

In Ruby, you can write blocks using either brackets or with the do/end keywords. Check out this code that gives surprising results:


a = []
 
a.concat a.map { |b| }  # => []
 
a.concat a.map do |b|
end
# => no implicit conversion of Enumerator into Array (TypeError)

It turns out that the Ruby parser actually cares which notation you use for your blocks, and that affects which method will receive the block. In the code above, a block written with brackets gets passed to the last method in the statement. A block written with do/end gets passed to the first method, which causes the code example above to break.

You can even use both types of blocks in the same statement, and they go to different methods:


def f(x)
  puts 'f got block' if block_given?
end
 
def g(x)
  puts 'g got block' if block_given?
end
 
def h
  puts 'h got block' if block_given?
end
 
f g h { } do
end
# f got block
# h got block

I used to think that the different notations for blocks were just equivalent ways of writing the same thing, and I said that do/end should be used if and only if the block is multi-line. Now it seems like there might be legitimate reasons to break that rule, or to come up with a different rule.

Monday, September 22, 2014

Introducing RPicSim

Check out the post I recently wrote on the Pololu blog: Introducing RPicSim. It talks about a tool we made for using Ruby, RSpec, and test-driven development to aid the development of PIC microcontroller firmware.