organizing include and lib paths

[ZeHa]

Hello!

I'm about to buy a laptop in a few days or weeks, and of course I want to get it development-ready and might use it very extensively for that.

But until now, I always found it a pain to organize my development stuff, especially when it comes to include and lib paths. Of course it's manageable, but all too often you end up with some messy directories and always wish you had done it the right way from the beginning ;)

So I wondered how others solve this. How are your directories organized? What's a wonderful and easy solution to this? Many libraries also follow different principles, some of them have got a direct "include" and "lib" folder, but others don't or have it named differently. Often it takes some time and some unsatisfied linker errors, until you've got everything up and running. Then again, you wonder if you should use <sdl.h> or <SDL/sdl.h> in your code, which also leads to overthink your directory policy...

So how do you do it?

[luggage]

I use dev studio but tend not to use the tools->options directory settings if I can and instead place them in the project settings for each project. I reference external libs in relative directories rather than absolute. That way I can quickly save off a buildable version of the entire thing.

dev\mygame\code
dev\mygame\data
dev\mygame\libs\
dev\mygame\include\

The libs\include dirs only contain external libraries. Then if I zip up 'mygame' that will just be fine for extracting and building at a later data. Only thing I don't stick in there is directx which I leave in the tools\options settings.

Not sure if it's the best way but it keeps things neat, tidy and I can quickly make backups that I can trust to rebuild without having to scour the net for old versions of libraries.

[ZeHa]

Hey that's actually a good idea... never thought about that ;)
That way it's really easy to exchange it via USB stick or something. Did you experience any disadvantages (except for maybe "hdd space wasting", which doesn't count on today's hdd sizes :D )

EDIT: How do you organize inside "include"? Do you just throw all your libs together (and hope for no name conflicts), or do you make sub-folders like SDL, boost etc and then #include <sdl/sdl.h>?

[luggage]

Up till now it's just the wasted hard drive space. Inside the include dir I guess it would make sense to put the things in further sub directories but the libs I rely on are only ever a .lib and .h.

One of the other benefits of using the project settings for the include and lib paths in your workspace is you can quickly add a new build setting to test out a new version of libs and what not.

[ggambett]

I use a unix-like convention in Windows, and of course, in Linux. It's pretty natural for Linux, so I'll describe my Windows setup.

c:\devel\libs\lib (all the .lib files)
c:\devel\libs\include (.h files, in subdirectories:)
c:\devel\libs\include\SDL
c:\devel\libs\include\GameLib (our ingeniously named game framework)
c:\devel\projects\game1
c:\devel\projects\game2
c:\devel\projects\GameLib (building this copies the appropriate files to the lib and include dirs)

I also point at this using the project settings as Scott says. Additionally I use relative paths (ie ..\..\libs\include) for ease of moving things around.

[GolfHacker]

This is how I'm organized:

Projects/Base
Projects/COTS/boost
Projects/COTS/freeimage
Projects/COTS/ogg-vorbis
Projects/COTS/openal
Projects/COTS/SDL
Projects/COTS/SDL_mixer
Projects/DirkDashing
Projects/FashionCentsDeluxe

The Base directory contains the code for the core game engine, which I am reusing across all games. This includes low-level OpenGL rendering code, audio engine, user interface library, standard types, generic application framework, etc. Bug fixes, enhancements, and changes to base code are automatically picked up by all my games.

The COTS directory lets me keep all of the third-party libraries and headers in one place (which is especially important on Linux, where I build my own custom versions of these libraries to deliver with my game).

The other project directories are specific to each game, and contain the game code, game assets, design docs, etc.

[jimflip]

Use a the following standard(ish) SVN layout


/game/trunk
/game/tags
/game/branches

Then
/game/trunk/header
/game/trunk/source
/game/trunk/data
/game/trunk/depend
/game/trunk/make
/game/trunk/doc

[Bad Sector]

I use dev studio but tend not to use the tools->options directory settings if I can and instead place them in the project settings for each project.

There where i work, we use the tools->options directory settings in Visual Studio for everything that is not within the game engine (the engine itself is broken in many small libraries and we use relative paths for in-project configurations). However for external stuff, like FreeType, we put the directories in tools->options. This way of configuration has the advantage of letting anyone use the organization method he prefers (f.e. personally i use a method similar to ggambett, but a co-developer has his libraries all over the place :-) - well, at least that looks like to me, it makes sense to him :-P) and by using only relative paths in the project files, we can checkin and checkout the whole engine from anywhere - or keep different versions here and there for small tests without having to do anything with absolute paths (like we used to at the past where it was a pain to do some stuff...). Again this adds to "personal organization beliefs" because i prefer to work from c:\theseis while another co-worker works from his My Documents folder :-).

Of course using this method, one can also work with the game from Linux ;-) (which was yet another reason to use this scheme).

[ZeHa]

/game/trunk/depend

So you also include all your dependancies (includes and libs) into your project folder? Or is the "depend"-folder only for things like DLLs?

[luggage]

badsector - we used to work with that method at a development house a few years ago but it was more trouble than it was worth. If you had to use someone else's machine to do any work or update their libraries you'd be faced with any old type of structure.

That's why we went with everything relative. Doesn't matter if you want to work in C:\coding or your My Documents folder or whatever takes your fancy - it just means all the external libraries will all be in the same place and the latest version of them can also be checked into source control.

One 'Get Latest Version' later and you have an entire project ready to build rather than having to install a list of external libraries and set up all your include\lib paths.

It overcame the problem of having two versions of an external lib on a machine at once. We finished a project with a 1.2 version of a lib then started a new project with a version 2.0. A few months later we needed to do a spot of work on the old project for language translations. Relative paths for everything solved this.

[Spaceman Spiff]

The general arrangements described here are what I use at home and are how we organize things at work. ex: \dev\projects\thisgame\etc

We also use relative paths in the project settings for includes, which as mentioned, avoids problems with multiple versions of a lib or SDK lying around. We do keep the 3rd party stuff and SDKs (the \inc and \lib at least) under source control also.

Finally, since you are getting a laptop for development, may I go off topic and suggest getting a 7200 RPM hard drive upgrade; It will make a big difference in compile times - best thing I ever did to out laptops, and really easy to do on most laptops these days.

[ZeHa]

Finally, since you are getting a laptop for development, may I go off topic and suggest getting a 7200 RPM hard drive upgrade; It will make a big difference in compile times - best thing I ever did to out laptops, and really easy to do on most laptops these days.

Well to be honest, I don't even know the RPM right now, but I assume it's not 7xxx but 5xxx. I never really complained about compile times, but I'll keep it in mind :)

[Spaceman Spiff]

Well to be honest, I don't even know the RPM right now, but I assume it's not 7xxx but 5xxx. I never really complained about compile times, but I'll keep it in mind :)

Laptop Hard Drives come in 3 different rotational speeds these days: 4200rpm, 5400rpm, and 7200rpm. The hard drives in 'desktop' computers are almost always 7200rpm drives.

When compiling and building projects, it has been my observation that seek times matter more than raw transfer rates; for every file compiled you have all the individual include files being processed (yes, I'm ignoring precompiled headers for sake of discussion, but they only help the issue partly).

Every #include<xyz.h> (or opens an .obj output, hits the MFT or scans a directory) results in the CPU waiting on the disk drive to find and read (or write) that file. The actual time spent reading a .h file and transferring it to the CPU is tiny ( < 1ms) compared to the time spent seeking the head to the correct track and waiting for the disk to spin the correct sector(s) under the heads.

According to a bunch manufacturers specs, the typical average latency for laptop drives is 4.2ms, 5.6ms and 7.2ms at 7200, 5400 and 4200rpm respectively.

A factor usually overlooked is swap file/OS activity which there seems to be plenty of when compiling big projects. On a laptop, you will probably find that the hard drive speed makes big difference as to how everything feels, not just compile times.

here is a good review of some of the latest notebook drives www.tomshardware.com/... (http://www.tomshardware.com/2007/04/23/notebook_hdds_deluxe/) with some good benchmark tables.

An idea to try for those of us using laptops would be to use a RAM disk to hold projects and all the libs/includes.

Ok, I’ve went waaay off track here, so I’ll shut up now.

[svero]

I map a few drives... one for code S: and one for work stuff w: (pointing to my documents)

Then I have..

s:\bin - output dir for builds
s:\games - all the game projects. Has subdirs like aargon, ratmaster etc
s:\engine - game engine stuff. Subdirs like kernel, sprite, leveledit etc..
s:\common - common non game specific code. Has subdirs like math,stl, etc

And...

w:\dev\proj - data dir for "proj" game
w:\dev\proj2 - data dir for proj2 game

and so on..

Note that s:\bin has all builds ... debug versions are written out as aargond vs aargon in msft fasion

[Huge]

I think it is important to do something like this:

dev/include/blah.h
dev/src/game1/game.cpp
dev/src/game2/game2.cpp
dev/lib

rather than this:

dev/game1/include/blah.h

Since it implies your includes are for more than one project/game, and that you are anticipating good reuse. Also you only need 1 extra include path.
If there is something that is only for game1, not game2 (or future game3) then that could obviously go under game1.

For thirdparty stuff, I stick the code under "src/thirdparty", and mess with the project to make its output go to dev/lib. I then copy the header files to the include directory - in a subdirectory if there is more than 1 ".h".

At some point "thirdparty" becomes "system" (eg directx) and gets done as most of you say with an additional include path. I guess this is a matter of taste.

[Garthy]

I do my primary development under Linux, so what I do won't map to Windows directly, but might be of use.

I tend to develop in:

<home>/projects/<project name>/<mode>/<os>/<variant>/<subproject>

with:

<home>: My home dir.
<project name>: Name of project.
<mode>: Either development, release, or optimised, depending on build type.
<os>: Operating system.
<variant>: Any variants on the project, ie. Flick Silver Edition for E.V.E. Paradox.
<subproject>: A folder for each subproject used. One will be the code for that project, and the others will be support libraries I am working on at the same time.

I keep all the code in-sync and handle code tracking using subversion. I keep the repository in another path altogether.

In each subproject folder there is a folder for each component of the particular subproject (eg. particular graphics and utility libraries in my cross-game libraries). This means that I include ("-I") each project folder using relative paths and #include in the form:

#include "component/header.h"

I keep common large resources for a game that aren't controlled by subversion (eg. textures, static windows exes) in:

<home>/projects/<project name>

I keep common large resources between games in:

<home>/projects

Basically each thing lives in the highest-level directory (shortest path) that makes sense for it.

I make heavy use of symlinks, tab completion, and aliases so that I don't have to remember or type in these long paths.

In actual use there are many exceptions to the above rules, but they are mostly due to legacy reasons- ie. I got it wrong initially and haven't yet fixed it up. The actual directory names I use aren't the same; I've changed them for the purposes of this post.

Hope this helps. :)

[GolfHacker]

Egad, that sounds like a lot of work!

Rather than keep multiple copies of code in sync, I write one single code base for all operating systems. The code base uses cross-platform libraries and SDKs, like SDL, OpenGL, OpenAL, etc, and the code I write is also cross-platform. I use very little OS-specific code, but when I need it, I confine it all to a single class.

One code base per project, with OS-specific code isolated to a single class, makes it a lot easier to maintain. And a lot easier to build on.

[jimflip]

/game/trunk/depend

So you also include all your dependancies (includes and libs) into your project folder? Or is the "depend"-folder only for things like DLLs?

Ahh well spotted, actually thats only on the engine project. We place in there other libs and headers that we "depend" on, it's to make things a little less confusing when we make branches and tags otherwise tags/branches then need links to other branches/tags of sub libs, which is still possible but can be a bit overkill if we just need a developer of a game to use a certain version of the engine.

[Bad Sector]

Btw, i forgot to comment on my organization method :-P.

First of all i work mainly from Linux. Sometimes from Windows, but mainly on linux. I have a shared "Big" partition (200MB - hey, when i got that drive 250MB it was big :-P) and there is a /projects directory. From Linux it's symlinked to /home/badsector/projects and using an ext2 driver (the partition is ext3, but the driver can see it as ext2 - i just have to try to not crash Windows too often :-) i map this partition to K:, so from Windows the path is K:\Projects.

When it comes to 3rd party libraries, i try to use as minimum as possible. Usually it boils down to SDL and libjpeg or libpng - i barely use anything more. For these libraries, i just install them system wide in Linux and in my MinGW installation in Windows. For my libraries, i just put them in the "projects" directory just like everything else. When i want to use a library of mine, i use relative paths. For example, Nikwi (projects/nikwideluxe) uses UndeadScript (projects/us), so i put ../us/include in include path and ../us/lib in library path.

With the exception of very simple libraries, most of my projects have this structure:

project_dir/src - source code and private header files
project_dir/include/library_name - public header files for libraries
project_dir/lib - library files
project_dir/bin - binary files

If a project needs platform specific code, the contents of the src are broken in common and platform specific. Usually win32 and linux: :-)

project_dir/src/common - common code
project_dir/src/win32 - win32 specific code
project_dir/src/linux - linux specific code

If the project has some tools and subprojects, i put them in the project_dir or if there are more than one or two, in a special 'tools' (or any other appropriate) directory:

project_dir/editor - this, or...
project_dir/tools/editor - ...this

In a few (two or three) cases, i had to break things even more. That's actually the case with my Undead 3D Game Engine which is broken in separate libraries, each one having one of the above structures:

undead/umath/... - math library
undead/urenderer/ - renderer
undead/ugfx/ - low level graphics functions
undead/uio/ - I/O functions

and other stuff. Each "subproject" of the above uses relative paths like if it was a normal project. Of course, Undead uses some of the other libraries (such as badcfg (http://www.slashstone.com/more/libbadcfg)) which is outside of the undead dir.