Evolutionary vs Throwaway Prototyping

[Valen]

I'm about to start working on my next game, and I'm trying to decide what the best approach is to reduce potential risk. When I was in college I took some classes that dealt with the System Development Life Cycle. Most of those models are of course too bulky for development of small games, but I think RAD models like Prototyping can be useful. We must know as quickly as possible whether a game idea is good or not, which is where prototyping comes in.

As many of you probably know, there're two primary types of prototyping -- evolutionary and throwaway. In evolutionary prototyping, a prototype is created with the possibility of being turned into the final product. If it gets the green light it's cleaned up and finished. In throwaway prototyping, a prototype is created as quickly as possible (sometimes in a different language than the final product) just to see the product's viability, and if it gets the green light the prototype is totally discarded and the project begins from scratch. The advantage to evolutionary prototyping is that there isn't a duplication of effort, but you could end up with a not-so-great code architecture. And a throwaway prototype can probably be created faster, but is only good to test an idea.

So what I'm wondering is, what's the best way to go for the kind of development that we do? Does it make sense to create a throwaway prototype or would this be too much wasted effort? If you've done prototyping, what was your overall experience?

[Rainer Deyke]

I much prefer evolutionary prototyping. Getting the prototype to run at all often takes a lot of effort that would be duplicated with throwaway prototyping. I also feel that throwaway prototyping teaches bad coding habits. If there's something wrong with the architecture of the prototype (which is usually the case), this can be fixe by refactoring.

[JaffaMused]

Both approaches can work well depending on your situation.

If you need to quickly test your game design (without much emphasis on actual implementation) I'd reccomend going the throw away route and use something high level like visual basic - that can quickly prove that your "line up the things of the same color" game concept works.

You may even be able to test the design by letting a bunch of people at it and get some feedback. If the game design sucks you can move on with only a few days invested in real effort, where as with the evolutionary approach, you might have wasted months to get this far.

On the flip side, if your game design is more demanding, you'd probably be better off prototyping it as if it were the final implementation, keeping your code reuse high. However, be very aware of feature creep - it's pretty much inevitable and will mess-up your best intentions of keeping the code clean.

* In short, if you just want to get an idea about the game mechanics AND the game is simple - make a quick throwawy prototype. Otherwise make your prototype as the "first playable" milestone of your entire project.

(I once got to test 5 different simple game designs in a week using VB to implement a prototype of each design - we quickly knew which ones weren't going to make it as final products) - Just my 2 cents, hope it helps.

[Valen]

...make your prototype as the "first playable" milestone of your entire project.

This is what I've been leaning towards. My game doesn't have simple mechanics, so I think the best way to do it in my case is to make it playable as fast as possible to get a general idea of whether it'll work. I just don't want to sacrifice architecture in the process.

[princec]

I never prototype! I just see what happens and go with the flow. Occasionally I rewrite bits, but not often these days.

Cas

[cliffski]

for puzzle games throwaway is probably fine, as the development lifecycle is shorter. If you are doing something like an RPG or an RTS or *shudder* a Sim game (like me), then you arent going to be able to knock up anything in a few days or weeks anyway.
I find that the best approach is to have a comprehensive library with as much reusable stuff as you can, this rapidly cuts your dev time.
Over the years (and through various games) I've gradually added new code to my library. I'm not as good at this as others, but at least I know have all my sound, input and rendering code in the library, as well as helper classes for stuff like reading ini files, translating text etc.
The one thing I fall down on is not having a reusbale cross-project GUI library for stuff like buttons etc.

[princec]

The GUI stuff is damned hard to design in a reusable fashion I'm already on my 4th iteration of trying to get it right.

Cas

[paulm]

As people have said, it really depends on what you're doing of course. My personal methodology is one of 'hack, design and code', but I'm not the greatest of coders by any means.

I've found the Game in a Day events at GarageGames, lovingly referred to as GIDs, very helpful. I've not managed to complete a game in that time, simply because of my lack of experience, but the experienced developers have done very well. You'd be surprised what you can get done in a day, especially if you have a lot of code lying around you can reuse.

So I wouldn't underestimate the power of throwaway prototyping, even for large projects. Thinking in terms of day forces you to cut out a lot of features and really get down to the core of the prototype, which is certainly not a bad thing. If you keep your scope of gameplay narrow enough (and as long as you're a half-decent coder), you can prototype a lot of stuff in a very short amount of time.

I believe speed is one of the tenets of being a successful indie, and I think prototyping in this intensive manner is great for flexing your programming muscles as well as getting you into a speedy mindset.

Cheers,
Paul.

[Mithril Studios]

As others have said, it really depends on what you want the model to do for you.

An evolutionary model (like the incremental one you bring up) is usually used when you already know what the requirements are for the application. This type of model permits you to do project tasks like manage the risk associated with the development process (e.g. quitting at a stable place when it gets too costly), or regularly release new features to a core application.

A prototyping model is usually used when you don't know what the requirements are up front. Instead of developing to a feature/release schedule, you are developing to find out what's the good, bad, and ugly.

This is what I've been leaning towards. My game doesn't have simple mechanics, so I think the best way to do it in my case is to make it playable as fast as possible to get a general idea of whether it'll work. I just don't want to sacrifice architecture in the process.

Perhaps instead of coding to find out if it's playable or not, why don't you see if you can set up the game using a whiteboard, or pen and paper, or miniature figures. Abstract the majority of the complexity away and use something which takes almost no time to set up and test if the core is fun or not.

Anthony

[Diodor Bitan]

I think the philosophy behind both evolutionary coding and prototyping is the same. You try, you evaluate, you adapt. This is how I code, and although I don't prototype much (code with the intent and the certainty to throw the code away) I think prototypes can be useful, depending on the problem.

The real antitethical aproach is to plan everything ahead. This is impossible for me. I try to look ahead obviously, and I think a lot about what my game should be like in the end, but the numbers of questions and ideas and problems that turn up make it impossible to plan except in very general terms.

[serg3d]

Perhaps instead of coding to find out if it's playable or not, why don't you see if you can set up the game using a whiteboard, or pen and paper, or miniature figures. Abstract the majority of the complexity away and use something which takes almost no time to set up and test if the core is fun or not.
Anthony

Wouldn't work if the core itself complex , like in simulators, racers, non-static puzzle games etc.
From the other hand I don't see much difference between evolutionary and throwaway prototypes. During the evolution I throw away a lot of code and conepts, enough for several prototypes to fill.

[Coyote]

I think Evolutionary Prototyping evolved (*g*) from throwaway prototyping, due to the reality that too often things that were intended to be JUST PROTOTYPES find their way into the final code. It takes a lot of discipline to throw everything away - and such discipline is often missplaced.

And in Evolutionary Prototyping, you still end up throwing away entire modules and chunks of code. So the two just kinda blend together. So I always assume I'm doing evolutionary prototyping with some parts getting altogether replaced.

I'm a HUGE fan of evolutionary prototyping. Mainly because *IT WORKS* and too many other development paradigms break down horribly and seem to succeed only in spite of themselves. Waterfall variants being a prime example, but that's kind of like shooting fish in a barrel.

IMO, there's a fundamental difference that many people still don't get between most forms of engineering and software development. If civil engineering was like software engineering, then requirements for bridges would change almost daily, they'd be required to deal with variable loads between 1 ton and 20,000 tons, cross everything from a 50' ditch to a half-mile-wide river, be made of of variable, locally-available materials, and handle anywhere from 1 lane to 12 lanes of traffic under wildly varying climate conditions. And the process currently used for designing and building bridges would be completely useless --- as is the best efforts to bring similar kinds of processes into software engineering.

The thing that works so fabulously well with rapid prototyping (either methodology) is that it easily accomodates changes (something the 'XP' - eXtreme Programming - just took to the extreme), it allows for MUCH earlier feedback (and design changes) - which means much cheaper & easier changes.

[mathgenius]

The pragmatic programmers use the term "Tracer Bullets" to describe this evolutionary aproach:
http://www.artima.com/intv/tracer.html
They talk about throwaway code being used when there are very specific questions being asked, eg. can i reach 60fps with such and such a technique? But when the target is moveing, the tracer bullets provide a feedback loop so you can see how you are doing.

It seems to me, any game idea is going to get modified and hacked apart over and over so much, that the evolutionary aproach is certainly the way to go.

Simon.

[Coyote]

Thanks for the link. I agree 100%. I'm going back and reading some of the articles. But it sounds like these guys know exactly what they are talking about.

[Mithril Studios]

Really, honest there *is* a difference.

Just for clarification, Evolutionary Prototyping is a Prototyping process model, while Evolutionary processes are a collection of different models - like Microsoft's Incremental model, the Spiral model, etc.

So with that said, I realized I misread Valen's post, and I thought he was talking about choosing between a Prototyping and Evolutionary process.

I guess no matter whether you use evolutionary or throwaway, you are going to end up redoing your code at some point. I think evolutionary would be better suited when you have a large superset of req's, and you need to flesh out the details, while throwaway would be good for almost no req's, and you have to work a lot with an external client to get them.

I'm a HUGE fan of evolutionary prototyping. Mainly because *IT WORKS* and too many other development paradigms break down horribly and seem to succeed only in spite of themselves. Waterfall variants being a prime example, but that's kind of like shooting fish in a barrel.

I agree with you for the most part here. There's the other side of the coin though - it's a huge pain to try to manage any kind of prototyping project, and the waterfall processes really shine there. (I worked as a subcontractor for the US govt for 4 years. I know how fun *those* suckers are.)

Or, if you have a project where the requirements are determined and locked up front (say for an iteration of the Space Shuttle flight software), you don't need a prototyping process. Again, it really depends on what the goals of the project are.

@ Coyote
I've been meaning to explore XP more -- can you make any recommendations on it (books, etc.?)


Anthony

[Coyote]

If you have really rigid & rarely changing specifications, then yeah... Waterfall WILL work. A space shuttle or missile guidance system is a textbook example.

As for XP - "eXtreme Programming Explained" is the big one. There are others in the series (Applied and Installed, I think) but I can't vouch for their usefulness.

Another one is called (I think) "Extreme Programming Refactored," and it's actually (I understand) a critique somewhat against XP. I haven't read it, so I can't recommend it, though I think a balanced look at XP is probably well worth it if you are thinking of using it.

I'm not a big advocate of XP, personally, but there are a few practices (or at least attitudes) of XP that I respect highly, and the six months I worked in an XP environment were at least enlightening. The biggest thing was NOT TO BE AFRAID OF CODE and, as they say, "embrace change."