Upgrading my graphics card?

[Your Computer]

Well, like I said, including RAM and such. I also screwed up. I mixed the nVidia Tesla 6gb with the nVidia GTX 660 Ti 3gb. My bad about that bit.

I'd assume with a GTX 660 + at least 16gb to compliment it nicely and I'd assume he has something between a 4-8 core computer (plus if he had two monitors and never unplugged them), you'd want at least 800, especially if you run it on Max SSAO, AA, (and all other highest graphics quality). For example, and this happened recently, my girlfriends computer busted her supply due to her over powered system AND she elected to play BF3 (yeah, I know, that's WAY more intensive than Amnesia, but still).

Then again, I'm assuming he has a stronger computer, PS is probably good enough. I was just suggesting to check into it.

The GTX 660 Ti consumes about 50 watts more than the GTX 660. The Telsa consumes about 100 watts more than the GTX 660. According to the Dell website, he should be running on a 460 watt power supply, with 8 gigs of RAM, a i7-3770 (quad core) and all common peripherals. 800 watts would be for SLI configurations (with room to spare, depending on the amount of graphic cards).

I am only going to use one monitor, its resolution is 1440x900. I want to play a lot of high graphic games. For some reason I can only play GTA IV on medium settings with the screen resolution down to 1280x720, which keeps me at 40 FPS. Would the GTX 660 2 GBs be able to play it max settings?

You should be able to play at "max" settings, 1080p or more (2GB is generally recommended for resolutions higher than 1080p), and get decent performance.