Penumbra Overture, frequent switch to my desktop

[flcn27]

While playing Overture the game frequently switches to the desktop for me. Sometimes I can't go back and need to restart the game. This is a real immersion breaker when things were just getting creepy.
Any idea how I can prevent this from happening? The only other program I'm running is Steam + Steam in-game overlay. Thanks in advance.

EDIT: As I'm writing this, I can hear a dog killing me while I was sent back to the desktop. It seems to be a problem with my mouse though because I had a similar problem in Amnesia. Sometimes in Amnesia my mouse cursor would instantly move to the upper left corner and then I could no longer move it around, forcing me to restart the game.
Now in Penumbra I end up at the desktop sometimes when I double click with my mouse (to select an item in my inventory) or sometimes when I use the left mouse button once (to click through a note or something).
I even had the same problem in The Elder Scrolls IV: Oblivion, sometimes when I clicked an item in my inventory I would also end up at the desktop. So, any idea how can I fix this?