I assume that by custom content you mean 3D models, textures and animations. Like Acies said, those are created with for example 3ds Max, Blender or Maya. (My personal recommendation would be Maya, since Max has some problems with the export of animations and I never used Blender)
From those programs, the models and animations are exported with the open COLLADA plugin. Textures are mostly created with Photoshop or a similar program and exported as .dds with the help of the Nvidia plugin.
When you have your models and textures exported, you use Frictionals own tools (material editor and model editor respectively) to create the .mat and .ent files you can then use in the level editor.
That's it for a general overview ^^ For more details on a specific subject you can also check the
wiki.