Both are in a way similar but also very different.
To me Penumbra was more eerie and there was more heart attacks, but Amnesia was more distressing and had an uncertain feeling all the time. Can't really say which one is scarier, since they're both scary in their own ways and both also scare me. Stories and atmoshperes are awesome.
Though Amnesia has more likable characters in my opinion. In Amnesia I like everyone, but in Penumbra I like everyone except Clarence. Every time he opened his mouth it made my temple vein twitch. The idea of having someone inside your head was good, but I didn't just like his voice. It reminded me of a drunken bum. And I really DO hate drunken bums. It was just purely irritating, not scary. D: