I don't know man, I think your just not playing the right custom stories. I think many do often aim to create the most immersive experience they can, its just that they fail for lack of ability in one respect or another (which isn't a problem of course, being the amateurs we are).
I think the nature of it being a 'challenge' or 'competition' would impact very negatively on the quality of the content released. I appreciate you may not mean a 'timed' challenge, but even so that push to get something competitive out in time to meet the challenge goes against what is required to make a very immersive and engaging experience for the player. Amnesia had a delayed release, and even then was criticised for the ending seeming 'rushed' (I wouldn't know, I'm too pathetic to complete it
). A real nice, polished, immersive and engaging experience takes an awful lot of time and care to be produced, particularly for modders who have the freedom (time) to be crazy-perfectionist about it.
People are currently taking time to make the kind of quality experiences you are looking for, I assure you, and they will continue to do so. If you want to see more stuff like that, by all means help create it. In my opinion however, a community challenge is not the sort of activity to get the kind of results you are looking for - such things are better suited for making the most intricate 10 min survival or puzzle game using the engine, rather than long term engaging stories to shit your pants to.
EDIT: However don't let my pessimism shit all over your ideas bro
go for it! It would certainly be interesting to see what would come of it