I finally got a binary compiled, forcing a 32 bit build by adding
add_definitions(-m32 -O2)
to all three CMakeList.txts, and
set(CMAKE_EXE_LINKER_FLAGS "-m32")
to that in PenumbraOverture. It needs some 32 bit compiler stuff installed to actually work, on Debian the packages are libc6-dev-i386 and g++-4.4-multilib (for stubs-32.h and a 32 bit crtbegin.o). The resulting binary seems to run ok, but sometimes dies on exit with:
(0) : fatal error C9999: *** exception during compilation ***
Cg compiler terminated due to fatal errorSegmentation fault
When adding -march=native to the compiler flags, it dies when trying to enter the "real" game, maybe with less optimization it wouldn't crash at all.