.h driven development?

[ZeHa]

Hello,

at the moment I'm planning a bigger engine project, and most of it is in my head and some of it already on paper, but here the problem starts: on paper, I make rough drawings with boxes, arrows, lines and some text, but I'm not listing every method. That's because you simply forget some of them, and also there are methods you just don't want to note because you think it's clear anyway. But during implementing, I often notice that there are some things I haven't thought about enough and then I have to change some things which can be pretty ugly as all of you should know :D

So for this time, I'd like to try a different approach, but I'm not sure yet if it would be a good idea: My plan is to start coding all the header files (or at least a big enough amount that represents a somehow closed unit), exactly as they would appear in the final project - and of course in the end, they will. That way I'd write down every method and every variable, and if I should see that something won't work, I can simply change it. I'm also free to delete or replace complete header files. Since nothing is implemented yet, there's always the possibility to alter anything. And if some big chunk is planned, where I'm sure there won't be any more changes, I could even start implementing that chunk already and continue headering the rest of the project afterwards.

For me, there would be the following advantages:
- Changes in concept are quickly made (as already mentioned)
- I know they will be used in the final product, which means I'll really type every method signature (instead of always thinking "it's just a concept")
- It feels a little bit like starting to code already
- With every header file adding up, I should be able to see if something works or something simply won't work.
- I won't have to do anything twice (because I don't have to draw every little piece on paper first and note everything down)
- Implementing should be like a breeze, as most of the concept will be done, I won't have to write any header files anymore and there shouldn't be some big surprising changes after all

On the other hand, I fear one (big) disadvantage:
- It could be, that when I start implementing, I'll have to implement a huge mountain of code until I can make some useful test runs.

Now I'm asking you if anyone has already done his project development like this, and if yes, what were your experiences? And if not, what would you think about the idea? Are there any more disadvantages?

[Nikster]

Isn't this what UML is for ? many can generate code as well in most languages.

[mot]

This way you will spend months coding blindfold and having nothing that works. I suggest designing a fairly minimal API (ie. open window, render cube, play sound), writing some example code that uses the API and then growing it and refactoring until you're satisfied.

Coding .h files won't tell you much about dependencies and usability of the API.

Oh, and yes, I tried to do this and it didn't work. I got distracted before getting to the .c part :)

Oh, and one more thing - I think that whatever great engine you're planning to do, there are ten other attempts to do the same on Sourceforge. Make a game instead!

[Jamie W]

I like the idea, and I think it'd work well if you can somehow put together just a skeleton base functionality (or definition of one, with your .h files). Then flesh it out later in stages.

I'd be cautious of mapping out the entire finished thing in .h files, I think a more balanced approach would work better in practice.

1. Create .h files for all the core functionality of your project.
2. Create some .c files, to get something up and running.
3. Go back to the .h files, fleshing out more project.
4. More .c files.
etc etc.

I very much like the idea of getting stuff completed in stages.

I guess at the end of the day, you'd just have to try it, and see if it works for you, and modify your approach as you see fit in future.

[zoombapup]

You could go the whole hog and write unit test frameworks for all the classes and then the classes :)

I usually do what youre suggesting and write stub interfaces for all my classes and make sure the system compiles (i.e. that all forward references etc work).

There is merit in the concept of refactoring code. Dont expect your first pass at ANY code to be ideal. Make it modular enough that you can replace or refactor and not kill anything (so for instance, I use an event based system to reduce dependancies).

[ggambett]

I don't know. I don't like your idea. I always keep my code in "working" state. Incomplete, with bugs maybe, but always compilable and always working.

I usually start with a vague idea in my head about how to implement what I want to implement. Then I start building it block by block, constantly compiling and testing. If your initial idea is relatively clear, you're a good programmer, and you refactor all the time, you should end up with a complete, bug free, elegant implementation.

[ZeHa]

Isn't this what UML is for ? many can generate code as well in most languages.
Yes, but if I would really create UML diagrams, I would face the same problems as on paper - I wouldn't note down EVERY method. And I wouldn't like to generate code out of it, since I want to write my header files by myself somehow...

This way you will spend months coding blindfold and having nothing that works.
That's as well my biggest concern... and I think it would become true very likely...

Oh, and yes, I tried to do this and it didn't work.
:rolleyes: hehe

Oh, and one more thing - I think that whatever great engine you're planning to do, there are ten other attempts to do the same on Sourceforge. Make a game instead!
Well for my special game it's hard to find an engine that's reliable and exactly what I want - and of course, I plan to do the engine flexible enough to create further games with it. On the other hand you're right... creating a game is more important, but I just want to make that engine. I'll learn very much with it and I've got something to show off afterwards (for job applications or something like that).

1. Create .h files for all the core functionality of your project.
2. Create some .c files, to get something up and running.
3. Go back to the .h files, fleshing out more project.
4. More .c files.
etc etc.
Well in fact that was also what I was thinking about... whenever a good portion has finished, that can be regarded as a unit and seems to be well-planned, I'd start implementing it.

You could go the whole hog and write unit test frameworks for all the classes and then the classes
Well I never did that so far - and I don't think I would like it - just like drawing those UML diagrams : D

Make it modular enough that you can replace or refactor and not kill anything (so for instance, I use an event based system to reduce dependancies).
Modularity is one big goal of my engine anyway, so perhaps the approach might work with this type of project?

Then I start building it block by block, constantly compiling and testing. If your initial idea is relatively clear, you're a good programmer, and you refactor all the time, you should end up with a complete, bug free, elegant implementation.
The problem I see with that is that as soon as there are cpp implementations, you (or at least I :D ) are less likely to refactor something completely. Instead I try to make some minor changes that will fix my problem, but perhaps I wouldn't re-create a complete class. When dealing just with header files, it's usually no problem to do heavy refactoring.


Well, thanks for your answers. There were some interesting points, and it seems that there are some pros and cons, though heavier balanced onto these cons, and the only one who really tried that idea said "it didn't work" :)
I think I might try something like Jamie suggested, but with the smallest units possible (of course still big enough to get properly planned). Also, there are some initialization methods or complete systems like loggers and consoles that should be possible to be implemented rightaway along with developing the rest, and even if it proves to be a bad implementation, it's separated enough to exchange it completely if necessary.

I won't start coding this week, so I'd still like to discuss it a little, so if you have any more suggestions, tips, hints or "forget its", please post them :)

[scratchdisk]

I agree with the other posters ... you don't want to try to think about the entire engine at once. Once you have your paper box diagram complete, Use unit test driven development to bring up small amounts of functionality at a time and iterate on it.

[Jamie W]

Well, thanks for your answers.

Thanks to you also, for bringing up this subject. I'd been thinking about this anyway, and it's good to discuss.

I think a balance is always good. There are benefits to prototyping everything (or a large core chunk of everything) before you impliment, it's like thinking a thing through, considering the implications and potential problems, before you start to impliment it.

On the other hand it's also very good to have your project in a tangiable working state (rather than just some abstract concept).

By the way guys, what is 'refactoring'?

[ZeHa]

http://en.wikipedia.org/wiki/Code_refactoring :D

(...assuming your question wasn't rhetorical ;) )

[Nikster]

Yes, but if I would really create UML diagrams, I would face the same problems as on paper - I wouldn't note down EVERY method. And I wouldn't like to generate code out of it, since I want to write my header files by myself somehow...


Well if you added in your main post you wanted it to be as painful as possible, I would have suggested tacking some upside down drawing pins to each of your keys :)

[ZeHa]

:D

Okay that was a good one, but seriously, I just don't like generated code - I'm very religious about code formatting and code styling and overall maintaining so that I just wouldn't accept a generated header file.

Yes, it's possible to let it be formatted automatically in my prefered style, but still, I feel more comfortable in having my own files (typed with those pinned keys if necessary :D ) and in addition to this, I'm faster in writing the header files than in drawing some quirky UML diagram ;)

[Nikster]

Well, I guess it depends on how you're going to do it, will it be OO ? I mean, you don't have to use the code generated from UML, heck, sometimes I just use UMLPad to get menu flows working as you can visually see it, doing that in code becomes a nightmare to track dependencies and how the flow generally works and you can quickly see any flaws in the system.

I'll be suprised if by just generating prototypes in a header will make it any easier than just starting with a main() and working from there, rather than laying out the whole prototypes (unless it's a tiny project of course), and the only reason I use UML over pencil/paper, is the canvas size and the ability to easily change relationships.

I'd be interested if you did take this route how many times you have to rewrite/factor the header :)

[Jamie W]

http://en.wikipedia.org/wiki/Code_refactoring :D

(...assuming your question wasn't rhetorical ;) )

Nope, wasn't rhetorical! I'd never heard of it before.

Having just briefly read the Wiki entry, I can see I'd been doing that all along, my code being perfectly neat and mess free (*sniggers*).

;)

[Jamie W]

Well if you added in your main post you wanted it to be as painful as possible, I would have suggested tacking some upside down drawing pins to each of your keys :)

Or coding it on a ppppowerbook (http://www.hot.ee/ppppowerbook/).

[GolfHacker]

Yes, but if I would really create UML diagrams, I would face the same problems as on paper - I wouldn't note down EVERY method. And I wouldn't like to generate code out of it, since I want to write my header files by myself somehow...

I disagree - you don't face the same problems, if you do it properly. It sounds like you're drawing class diagrams on paper. That's a good start, but its only half the story. Using UML, you should then create some sequence diagrams to capture the control flow between objects that work together to accomplish something. Class diagrams show structural relationships, but sequence diagrams show dynamic behavior between objects instantiated from those classes (which is the very issue you're struggling with).

For example, you might create a sequence diagram titled "Render Sprite", and the diagram would show how the objects work together to accomplish this. As you're adding objects and flow arrows to the diagram, you're adding methods to the classes that the objects instantiate. Note, you're not adding all the possible methods (like every get/set), but you're adding the important methods that you need to do the work, and you're nailing down their function signatures. You're essentially diagramming the critical code paths, and it is easy to change your logic and flow without having to refactor a lot of code. You may even decide to change your class structure as you go, and that's ok - you haven't written any code yet, so changes are cheap.

When you're satisfied with your design, hit the code generation button and you'll get not only header files but also code files with stubbed function calls. In theory, it should compile as-is with minimal changes, but of course it won't do anything yet. Then just start implementing pieces of it using an iterative approach, as scratchtip and others have pointed out - the difference is, you're iteratively implementing the pieces within an established framework, so you have a good roadmap you're working toward and you shouldn't be making major revisions to the overall framework with each new iteration.

As for the formatting of generated code, this is easy to fix. Just reformat the header file however you want, either by hand or write a quick perl script.

[ZeHa]

Well it sounds interesting. But unfortunately, I never really understood sequence diagrams, and I never found anything useful to read. There seem to be some topics where you just can't find a proper entry point in the internet. One of them is Docbook, the other one is UML ;)

Perhaps you've got some nice sequence diagram tutorial for me? If I know how it works, I might really use it, but until now it has always been some black box for me.

(A little offtopic, but one thing that I didn't get either was with class diagrams: there seem to be some different arrow shapes, but some of them actually have overloaded meanings, depending on if it's OOA and OOD - that's just horrible)

[TunaBreeze]

Whenever I'm at the stage you're in I always draw it out on paper then create a mock .h file where I make prototypes of all the functions, and global variables. I then review what I have on paper and what I have in code, make any adjustments, refinements, or additions, before I make the function definitions.

[GolfHacker]

Zeha, here are some good online UML articles to start with:

UML Basics (Part 1)
http://www-128.ibm.com/developerworks/rational/library/769.html

Class diagrams
http://www.objectmentor.com/resources/articles/umlClassDiagrams.pdf
http://sparxsystems.com.au/resources/uml2_tutorial/uml2_classdiagram.html
http://www-128.ibm.com/developerworks/rational/library/content/RationalEdge/sep04/bell/

Sequence diagrams
http://sparxsystems.com.au/resources/uml2_tutorial/uml2_sequencediagram.html
http://www-128.ibm.com/developerworks/rational/library/3101.html (focus on the Basics)

And here's my favorite UML book (for tutorial and reference):
The Unified Modeling Language User Guide (http://www.amazon.com/Unified-Modeling-Language-User-Guide/dp/0201571684) by Booch, Rumbaugh, and Jacobson

[GolfHacker]

Here are some good UML tools I recommend:

Enterprise Architect (commercial):
http://www.sparxsystems.com.au/products/ea.html

Umbrello (a very good open source UML modeling tool):
http://uml.sourceforge.net/index.php

I currently use EA, because that's what I also use at my full-time job, but I have used Umbrello in the past. Both are good tools, from my experience.

[Huge]

I like to start with the header files too, but I do not like to write a whole "engine" at a time.
Also, I make a distinction between "global" header files and "local" ones. The global ones go in /dev/include, and you include them like this:
#include <xxx/yyy.h>
and the local ones go in the source directory with the cpp code, and you include them like this:
#include "zzz.h"
This way, other directories can not "see" other local header files - this gives you the freedom to hack away, knowing you only need to check things in the current directory. Gives you the best of both worlds - overall good design, with locally efficient coding.
Also, I like to make the gobal header files "interfaces" where possible. That is pure-virtual, or inline-stubbed-out-virtual functions. Then, in the cpp, implement this inline using java-style class (everything inside the class {} definition). This makes you write better code because it is very very easy to pull a bit of common code into a separate function, since you don't have to change a header file.
This method also solves the problem of "friend" classes. Simply make everything in you class public! Before you flame me, this works because nothing outside the cpp file can "see" your definition. Then, put your related/helper classes in the same cpp file as your interface implemenation. Problem solved. Again, this helps you coding because you are much more likely to pull helper classes out if you don't have to worry about access restrictions.

Anyhow, getting back to original post, this is also good for design, because your interfaces will always compile, if not link/do anything.