JKO's Guide to compiling Penumbra

[JKO]

Hi and welcome ladies and gentleman to the
JKO's Guide to compiling Penumbra!

I've decided to take the time to sit down and write it, even if I'm a total stranger here right now, for multiple reasons. First, there is very few documentations out there, which makes it hard for newbies to learn the engine (which I believe have a great potential for a modding community). Second, most of what you need to get everything working has been said on the forums, but they are scarce, hard to find and some can be misleading or hard to understand. I apologize if I makes mistakes in this text as English is not my first language (I am French-Canadian). I must also point out that I am not an expert and that I have very limited knowledge of the engine right now, so if you know more than me and see something that could be fixed or done differently, please advise.

This guide aims to help new users from the ground up to get the game working under Windows, with Visual Studio 2010. I might do other guide for different IDE and different OS, if more people needs help for those (From what I can see right now, the problems mostly comes from Windows and VS10).
So, let's get started!

Getting Started

First, obviously, you need to have a version of Visual Studio installed. For this guide I will be using Visual Studio 2010, but it should be quite the same thing for the other versions as well.

Downloading Git

Now you need the main source files to get you started. The 3 main sources required are downloadable through Git, a kind of SVN service. Since we are on Windows, you will need msysgit. You can download it here (Select Git-1.7.0.2-preview20100309.exe)

Install it in his own directory, like you would install any software. Remember where it is installed, we will come back to it shortly.
Now you need a folder to contain all the files you are going to download. Create a folder named "OuvertureSource" in a location convenient for you.
Now we will download the 3 main folders with Git: HPL1engine, OALWrapper and Penumbra. To do so, open the Start menu and run Cmd with the search bar or the run utility.
When opened, go to the \git\cmd folder, e.g: cd C:\Program Files (x86)\Git\cmd (cd is the Change directory command in DOS, in case you didn't know) Now that you are there, you will need to get the appropriate download links for Git. This is where they are, but I'm going to give the links here anyway.

Downloading the main source files

To download HPL1engine, make sure you are currently in the \git\cmd directory in cmd and type the following:

Code: (Select All)

git clone git://github.com/FrictionalGames/HPL1Engine.git

This will download the folder as it is right now on GitHub to your computer. You are making a "clone" of it.

To download OALWrapper:

Code: (Select All)

git clone git://github.com/FrictionalGames/OALWrapper.git

To download Penumbra:

Code: (Select All)

git clone git://github.com/FrictionalGames/PenumbraOverture.git

You NEED all three, they are ESSENTIAL.
You will also need the dependencies folder, but you don't need Git to get it. You can download it as a .zip here. (Like the other 3, it is essential)

Setting up the main source files

When they will all be downloaded, you need to put them all together in the "OuvertureSource" folder you have made earlier. The three ones you got with git (HPL1Engine, OALWrapper, PenumbraOverture) are in the \git\cmd folder. Copy them into OuvertureSource (It is wise to copy them instead of cutting them, so if you mess something up you won't have to re-download them, you will have the backups at hand. But this choice is yours.) When it is done, unzip dependencies.zip in OuvertureSource.
You will also need GLaux.lib, which is missing from dependencies at the moment. You can download it here. Once downloaded, extract it to OvertureSource\dependencies\lib\win32\ .

You will also need to create a fifth folder, "redist" (Do it). When you will compile penumbra.exe, later in this guide, this is where it will go. You will have to copy the content of the game in this folder, or it won't work (The game won't find any of the game files). If you have a very small hard drive and really can't afford to have a duplicate of the game, you can simply take the generated penumbra.exe to your existing game folder, but this method have a lot of disadvantages.
If you bought the game on Steam, you need to copy the content of "C:\...\Steam\steamapps\common\penumbra overture\redist\" in the new redist folder you created in OuvertureSource. If not, then search to where you installed it and do the same thing. You might also want to delete penumbra.exe in OuvertureSource/redist/ after it is done (The copied one, not the original), though it is not really necessary.

In the end, the five folders in OuvertureSource should look like so:

Code: (Select All)

OvertureSource/ (You create it)
   dependencies/ (Unzipped from the download)
   HPL1Engine/ (Acquired from Git)
   OALWrapper/ (Acquired from Git)
   PenumbraOverture/ (Acquired from Git)
   redist/ (With all the game contents)

Happiness! We now have pretty much everything we need! It's time for step 2:

Projects corrections and buildings

This is where most attempts fails. The problem is that the the VS project files are not set-up properly. I think it is not entirely their fault however, as Urkle said himself, he is not a windows user and had to hack the files manually. But the good news it that I know how to fix this! Simply follow my instructions step by step and you should not run into any problem. There is some conditions however that I might have missed since I already had done a working compilation before doing the test ones and Visual Studio might have remembered something, preventing me to debug it properly, but I believe that it is not the case. If something bad happen, like a missing GLaux.lib for instance, tell me, and I'll adjust this guide properly. Before we compile penumbra however, we need two files: HPL.lib, generated by the codes of HPL1engine, and OALWrapper.lib, generated by the OALWrapper codes. They are both used in the codes of Penumbra, so we will need to build them beforehand. Now let's get working!

Setting and building HPL1engine

This is the main engine of the game. If you need to modify core functions, like graphics of physics, this is where you want to go. But even if you don't want to do anything with it right now, you need to build your version of HPL.lib .

First, open HPL.sln, in the HPL1engine folder. VS10 will ask you if you want to convert it, accept this request. When it will all be done, click once on the name "HPL" in the Solution Explorer sidebar. This will make sure we are changing the options of the right thing. In the upper toolbar, change the build option from debug to release (It is not really important with HPL.lib, but will be with OALWrapper.lib, so this is just for consistency).

Now, from the menu bar, go in Project -> HPL Properties (Or it might simply say Properties) (You can also go there with Alt+F7). The HPL Property Pages window will appear. Go to Configuration Properties -> VC++ Directories. There you will find "Include Directories". To make it shorter, I will use "Include something" for the rest of the guide, it refers there. Click on the paths on the right of "Include Directories" and a little arrow should have appeared, click it and select <Edit>. To add a new include line, simply click on the little new folder icon and select the folder you want to include. Here you must include the "C:\...\OuvertureSource\dependencies\include\AL" folder. Simply browse to where it is on your computer and select it. Click OK.

Then you want HPL.lib to compile directly into ..\HPL1engine\lib\ . That way, you won't have to move it when you will compile it, and this is where Penumbra will check for it later on. To do so, simply go to Configuration Properties -> General. There you will find "Output Directory". Click on the path. It works the same way as the Includes, but you have to click <Browse> instead. Select your ..\OvertureSource\HPL1engine\lib\ folder and the new path should look like ".\lib". When this is done, click Apply in the bottom of the window and click Ok to close it.

You should now be ready to build HPL.lib! In the menu bar, go to Build -> Build Solution (Or simply F7). Let the compiler do it's magic, but it might take a while. When it will be one, it will pop a freshly built HPL.lib in ../HPL1engine/lib/ . You don't need to move it, Penumbra will know it's there and take it automatically.

Setting and building OALWrapper

I don't really know what this one does, other than being a C++ wrap of Open AL, although I admit I didn't searched very far. (I'd like some precision on it if possible).

This one is slightly harder to configure and is the most frequent source of pain, but again, follow my instructions and you will be fine.
First you will need to add a folder in OALWrapper: "lib". This is where Penumbra is going to look, and it is nonexistent by default (?!). Simply create the folder and continue onto the next step.
Now, again, open OALWrapper.sln. Convert it when asked to. Click on OALWrapper in the Solution Explorer to make sure we are configuring the right thing, and set the debug bar on Release (This is very important! Somehow an OALWrapper.lib made in debug won't work later on, so make sure it is on Release!) When it's done, go to the Properties (Alt+F7).

First go to Configuration Properties -> General and make sure "Target Extension" is set to ".lib". While still in General, change the "Output Directories" to the lib folder you just created (You can change it the same way you changed the includes in HPL, but select <browser> instead. Make sure it is the lib folder you made inside OALWrapper!) Now when you will build OALWrapper.lib this is where it will go, and where Penumbra will look.

Now go in the includes (VC++ Directories) and add both "dependencies/include/AL" and "OALWrapper/include" (Do it with the new folder button, to make sure you get the right paths)
Now, very important step, go in Configurations Properties -> C/C++ -> Preprocessor and add WIN32 to "Preprocessor Definitions". This will tell Visual Studio to compile OALWrapper the Windows Way™ (If you don't, you'll still get the file, but it won't be usable, at least under Windows).

Here you go! Click Apply and close the window, you are now ready to build it. Like with HPL.lib, go to Build -> Build Solution, or press F7. If everything is done correctly, it should build without error and pop a freshly built OALWrapper.lib in OvertureSource/OALWrapper/lib/, and just like HPL.lib, you don't need to move it.
Otherwise, if you run into any error, make sure you did everything properly, then tell me what's wrong, and I'll try to update the guide.

The fun part: Compiling Penumbra!

So, if you have done and build both HPL.lib and OALWrapper.lib properly, you are now ready for the part you have been waiting for a while now: Compiling Penumbra. This is done from the third folder: PenumbraOverture (Obviously) This is what you will be mostly modifying, since it's the "actual game". From there you can change the menus, the items, the monsters, etc. Luckily, It doesn't require a lot of configurations, and you should get the game up and running in no time!

Setting up Penumbra's Properties

First, open Penumbra.sln, and like the others before, convert it. Again, click on its name in Solution Explorer to activate it. This time though, leave it's debug option to Debug. Go in Properties (Alt+F7). Now head to Configuration Properties -> General and set "Output Directory" to "..\redist". If you followed my instructions earlier you should have this folder in OvertureSource along with all the game content directly in it. Now go in Configuration Properties -> Debugging (right under General) and set "Working Directory" to "..\redist" again. This will allow you to use the debug functions from Visual Studio and run the game directly from it.
You also need to go in VC++ Directories and add "dependencies/include/AL" to the includes. When it's done, click Apply and close the window.

Now it's the moment of truth, Will it Compile? It should! If you want to compile it only, go in Build and select Compile. If you want to build an run it, click on the green arrow in the debug bar or click "Start Debugging" under Debug or simply press F5. The compilation will take a while (Don't worry, this is because it's a first time compilation, the other ones will be much shorter).

The aftermath

Now hold your breath and see what happens. If it says the compilations failed because of errors, and it is because of an alu.h missing, you didn't include "dependencies/include/AL" correctly.
If it just crashes and display a "Windows Fatal Error", you probably did something wrong with OALWrapper.lib, try to do it again.
If the window does open but hang, and an error message like "Unable to load PointLight2D", this means you don't have the game contents with your version of penumbra.exe in OuvertureSource/redist, go and add them.

But if you did everything properly, the penumbra window should open and shortly after, display "Loading...", followed with the pre-menu. If so, cry manly tears of joy as you have now successfully build you own version of Penumbra: Overture for the first time! You are now able to modify it as you please, and show the developers that they where right to release the source codes of Penumbra: Overture!

Additional Files

Here are all the downloads and links, if you prefer a list:

msysgit Download
Frictional Games main GitHub
HPL1engine Git
OALWrapper Git
PenumbraOverture Git
Dependencies Zip file
GLaux.lib Standalone download

Current Revision of the Guide

Code: (Select All)

23/07/2010: First copy of the guide.
             Added HPL.lib proper Output Directory
             Added instructions for GLaux.lib

Lasts notes

As I said earlier, this guide is at a very early stage. If you see any error in it, or if you are sure to have followed my instructions properly but still encounters problems, please tell me immediately, so I could fix it here ASAP. If some parts are unclear, under-explained or simply in a bad format, please inform me of it, and I'll make it clearer. Your feedback is greatly appreciated.

Thank you for reading this guide and I truly hope it has been helpful!

[Urkle]

Very nice guide and thanks for writing this up.. Any "changes" to the Project files you with to contribute, If you have a github account just commit them to a fork of the project and send me a pull-request. OR if you do not have a github account, simply use the git format-patch command to generate a patch and post it here on the forums. I would very-much like to get it so that it can be compiled out-of-the-box on ALL platforms.

[nofsky]

I haven't really had time (or the will) to try compiling the source before but with this guide I thought I'd at least give it a go.

I'm using VS2010 Express with the Platfrom SDK installed (i think that's needed), Windows 7 64-bit.

Compiling HPL1Engine:
Getting loads of warnings about unsafe functions, mostly sprintf. This can be ignored, I'm sure.
Some other minor warnings.
It compiled to .\build\Release instead of .\lib so I had to change that in the solution properties.
Other than that, everything appeared to compile fine.

Compiling OALWrapper:
Getting a few warnings about unsafe functions again.
3 warnings about something being declared deprecated (alutLoadWAVFile & alutUnloadWAV).
But it seems to compile properly.

Compiling PenumbraOverture:
Fatal Error, can't find GLaux.lib. Fortunately I got this file from another thread here a while ago. I placed it in dependencies\lib\win32.
After that it compiled fine.

Running the game (in debug):
Everything works fine! I played from the start until the first save spot (in the office). No problems at all.


Thanks for taking your time to write this guide! Very much appreciated.

[JKO]

[...] Any "changes" to the Project files you with to contribute, If you have a github account just commit them to a fork of the project and send me a pull-request. OR if you do not have a github account, simply use the git format-patch command to generate a patch and post it here on the forums. I would very-much like to get it so that it can be compiled out-of-the-box on ALL platforms.

I could do it and would like to very much, but there is a slight problem: Not everyone is using Visual Studio 2010. Many are still with 2008, and if I change the project files to VS10, they would be unable to open it! Is it possible to have two version of the "same" project in the revision? (one at VS10, one lower)

Running the game (in debug):
Everything works fine! I played from the start until the first save spot (in the office). No problems at all.

Thanks for taking your time to write this guide! Very much appreciated.

This result is exactly why I did that! Thank you.
I will add the GLaux.lib part since it seems to be a problem (I already had it installed). I'll also fix the HPL.lib not going in the right folder, forgot it
Also, these warnings are normal, they (Frictional Games) used functions Visual Studio doesn't like (The sprintf's are C functions to print in strings. In C++ you're "supposed" to use flux instead, that's why). Unless you replace them all, there's nothing you can do.

[nofsky]

There's a problem with the PenumbraOverture solution. Both the Debug and Release configuration will use the settings for Release, so you aren't actually making a debug build when you pick that.

I changed the Debug configuration to really use the Debug settings with the Configuration Manager. I had to copy/paste some settings from the Release to the Debug configuration:
C/C++ > General > Additional Include Directories
Linker > General > Output File
Linker > General > Additional Library Directories
Linker > Input > Additional Dependencies

Trying to compile it after this resulted in errors like this:
HPL.lib(World3D.obj) : error LNK2038: mismatch detected for '_ITERATOR_DEBUG_LEVEL': value '0' doesn't match value '2' in AttackHandler.obj

I'm not sure how to fix this. I tried making a debug build of HPL1Engine but that made no difference. It should be possible to make a debug build of the game, right?

It still works fine using the Release settings, though.

[JKO]

I tried it like you said, by copy pasting those 4 into debug and switching to debug. I also cannot compile, but not for the same reason, I get :
unexpected end of file while looking for precompiled header. Did you forget to add '#include "StdAfx.h"' to you source?
and
Cannot open precompiled header file: 'build/debug\Penumbra.pch': No such file or directory.

I conclude that there might be some error or differences between Release and Debug that cause a conflict. I had the problem you have before, the ITERATOR thing, this is the reason I specify to compile OALWrapper.lib on release, because on debug it gives this error. But I believe you can turn it off with Configuration Properties -> Linker -> General -> Enable incremental linking (Say No), but I haven't tested that.

I'll keep playing with the options until I can get it to work.

[nofsky]

I forgot to mention that, I got the precompiled header problem too, but I just disabled the setting so it matches the release configuration. I'm not sure what use a precompiled header is anyway, faster compiling? It doesn't seem to be needed.

I tried disabling Incremental Linking but that didn't seem to make a difference. I still get the same error.

[JKO]

I made a lot of progress. First, you need to compile HPL in debug mode. This while give you his little brother, HPLd.lib (for HPL debug I suppose). Then, you also need OALWrapper.lib in debug mode (I took the liberty to rename it OALWrapperd.lib, to make it like HPLd.lib in the penumbra files). Then, I manage to get penumbra running in debug mode, I even see the firsl "Loading...". But after a bit, Visual Studio gives me an error message:

Code: (Select All)

Debug Assertion Failed!

Program:
..\OuvertureSource\PenumbraOuverture\..\redist\Penumbra.exe
File: c:\program files (x86)\microsoft visual studio 10.0\vc\include\vector
Line:933

Expression: "Standard C++ Libraries Out of Range"&&0 (It gave me vector out of range earlier)

For more information on how your program can cause an assertion failure, see the Visual C++ documentation on asserts.

(Press Retry to debug the application)
<Abort> <Retry> <Ignore>

I think the problem comes from OALWrapper (It ALWAYS comes from it!). When I build it in Release mode it is 15,216Kb, but when I build it in Debug mode it is 3,921Kb, that's a significant difference. And if I try the Release build of OALWrapper in penumbra(debug), I get the iterator error (Which I believe means "you're mixing Release with Debug and that's bad", but I'm not sure)

By the way, to get the debugs running, make sure you copy/paste the different paths to the libs / includes as well. I did more that what you showed and It worked. Also make sure to corret the paths manually for the .\lib\OALWrapper.lib and .\lib\OALWrapperd.lib (That you make youself, as I said). You'll also need to make Penumbra load HPLd and OALWrapperd while in debug mode (Linker -> Input(debug)).
I've tested the Iterator off thing and it doesn't do anything, indeed.

(Don't give up, we'll get the Debug mode working!)

[MulleDK19]

Thank you!

I finally got it compiled without crappy lighting and crashes.


Btw, if you have both VS2010 and VS2008 installed, OALWrapper will open in VS2008! So for OALWrapper, make sure to right click > Open with... > Visual Studio 2010, or you'll get linking problems when compiling Penumbra.

[Tomy]

I compiled it all, but when I start Penumbra.exe it gives me an error: "Couldn't load image: player_crosshair_cross". But there's no such file at all even in my original Penumbra/redist directory.