Tuesday, February 05, 2008
Frequent Demos Kill Software Development
Okay, today's post is gonna be a little more on the technical / development side o' the fence, and not even game-specific. Sorry.
A friend of mine, who had served several years in the military, once told me, "No battle-ready unit ever passed inspection, and no inspection-ready unit ever passed battle." I have no idea if that's remotely true or not, but the point was that the traits that get measured in an inspection were very different - and perhaps even contrary to - the traits required to succeed in actual combat.
I wrote some time ago about the root of some of my tendencies to procrastinate, the "local maxima problem." The problem comes from the fact that many times, in programming, you have to break things to fix them. The "agile programming" folks like to invoke a technique called "refactoring" - you basically throw out the old code and make a whole new version. No big deal - or at least it shouldn't be. In fact, it should be happening all the time.
The problem is that during the time that this is happening, apparent progress might halt or even appear to go backwards. From the perspective of an outsider, at the end of two days of frantic labor, you've got something that looks exactly the same as it did before. Oh, sure, as a developer you know that this new, improved version is ten times faster, much easier to maintain, and got rid of four bugs that were practically insurmountable in the old code.
Now, as a programmer, I should be in that mindset. Totally. But I'm not. And some things happened at Ye Olde Day Job that reminded me of why I'm usually not... and may be the source of some of my development hangups.
You go like crazy to hit a demo milestone. This is true both in the games biz and much of pro software development. And this, I believe is a good thing. There's another saying in software (and I think hardware, too) that "if it weren't for trade shows we'd never get anything done." It forces an integration pass, acts as a huge motivator, and can really help a team get the important things done.
But in doing that, you may throw in some crap code for the demo that's pure prototype to help the end-user see all what you've been working on. Maybe it's some slapped-together front-end menu, or whatever. You show it to the powers that be - your own upper management, or maybe your publisher. You decide on changes. And then - ideally - you go back to your desk, and proceed to rip certain aspects of it apart to refine and refactor and improve the project as a whole.
But lo and behold, after a certain point in the project, said Powers That Be become a lot more demanding and random. They suddenly want to see the latest and greatest version ten days AFTER a milestone. This is sorta like them coming by for you to take them on a test drive in your car while you've got the engine pulled out and on the garage floor Yet Again.
And this makes your publisher or upper management people grouchy. What the heck are you doing? The software was working just fine a week and a half ago, why is everything crashing and broken now?
So you become paranoid about yanking anything - which means real progress slows down because you are afraid to make any move unless you are sure you can get it back into a demo-able state within two hours' notice.
Sure, one might say - there's CVS - or whatever source-control system you are working on. Just make sure that everything is working perfectly before you check it back in. Great idea. But often impractical in a collaborative effort with a whole bunch of cross-dependencies. Never mind the treacherousness of trying to make a build and run it untested on some other computer besides a dev box in the middle of development, without having done a reasonable integration pass (and if you happen to do automated integration testing every time someone checks code into source control, give yourself a gold star... but you are probably still painfully aware of how dangerous it can be even doing that).
So - yeah. Bottom line - if you want to shoot development progress, fill it full of lead, and leave it bleeding at the side of the road in a toxic pool of virulent disease-ridden slime, perform frequent spot-inspections and impromptu demos.
That little bit of aversion therapy lives with me still, even as an indie answerable to nobody. I still find myself paranoid about retreating from the dead-end of local maxima.
(Vaguely) related bits of ranting
* Fighting Procrastination: The Local Maxima Problem
* Productivity Tip: The List
* Embrace Code!
Labels: productivity, programming
- posted by The Rampant Coyote @ 12:55 AM (Permalink)
Comments:
Links to this post:
<< Home
So imagine what it is like to be working on an MMO that has people playing, and people expecting weekly patches. My life is your nightmare, apparently!
-Herb-
Linkrealms
-Herb-
Linkrealms
# posted by 
: February 05, 2008 9:54 AM
: February 05, 2008 9:54 AM
Ouch! Yeah, I guess you kinda-sorta are answering to your people in beta right now, going through exactly the same process.
So --- uh --- I guess that explains why LinkRealms didn't ship in 2007... :)
So --- uh --- I guess that explains why LinkRealms didn't ship in 2007... :)
Hi Rampant Coyote!
Ok, i've read that (and many other articles you wrote before) and now i'd like your advice on this subject.
I'm an Indie wannabe (not indie yet, working on it - game industry day job but still young) so i have a sellable game project in progress.
I recently setup a roadmap that you can have a quick look there.
(More details in the dev-blog .)
So, you can see the milestones planified (but the dates can move, we will review the roadmap each month). What i planned was that on the first milestone i gather few people to test the version (just to check technical things) and for the next milestones i send a version to those people to make them play (an "alpha test" community you could say). I really need tests because it's a multiplayer oriented game (STR).
Now i read your article and i know what you're speaking of (as i work in the industry and worked before in embedded softwares). On my dayjob we use a double roadmap, one for the versions to show to the client , one for internal developpement. But in practice you get the same problem because the client roadmap is more important than the other one.
Then, do you have any advice in my home-made case (like the MMO case anonymous was speaking of) where you have to realease "demo" or test version early?
Thanks for your attention - sorry if my english is not perfect, not english native Xp
Ok, i've read that (and many other articles you wrote before) and now i'd like your advice on this subject.
I'm an Indie wannabe (not indie yet, working on it - game industry day job but still young) so i have a sellable game project in progress.
I recently setup a roadmap that you can have a quick look there.
(More details in the dev-blog .)
So, you can see the milestones planified (but the dates can move, we will review the roadmap each month). What i planned was that on the first milestone i gather few people to test the version (just to check technical things) and for the next milestones i send a version to those people to make them play (an "alpha test" community you could say). I really need tests because it's a multiplayer oriented game (STR).
Now i read your article and i know what you're speaking of (as i work in the industry and worked before in embedded softwares). On my dayjob we use a double roadmap, one for the versions to show to the client , one for internal developpement. But in practice you get the same problem because the client roadmap is more important than the other one.
Then, do you have any advice in my home-made case (like the MMO case anonymous was speaking of) where you have to realease "demo" or test version early?
Thanks for your attention - sorry if my english is not perfect, not english native Xp
Breaking working code really is a pain!
You touched on using source control, but it occured to me, maybe we should fork right after a demo build?
That way we always have the (old) working version to show if required, as well as a backup if we mess up the refactoring.
You touched on using source control, but it occured to me, maybe we should fork right after a demo build?
That way we always have the (old) working version to show if required, as well as a backup if we mess up the refactoring.
I had a path finding algorithm build for this game that was ugly as sin, but fast. For months when I got bored I would work on debugging it. After wasting who knows how much time, I replaced it in a few hours, and the replacement was almost as fast, didn't require path smoothing, and didn't crash the system. There were two other times during this project I would have saved time just rebuilding.
However, there are two times I almost rebuilt, but found I had a typo that was throwing everything off.
However, there are two times I almost rebuilt, but found I had a typo that was throwing everything off.
@Joshua - Branching for a milestone build is pretty common practice, so ... yeah. Though that is a pain in the neck all of its own.
The problem is when said Powers That Be don't WANT to see the milestone, but want to see the current build. Or see whatever improvements were made since the milestone / demo. And give you only a couple of hours' notice.
Then you start getting paranoid.
@Klaim - I looked over the roadmap briefly, and it looks fine AFAICT. An okay plan now is better than a perfect one in two months, and it's far better than having none at all. Get feedback early and often, but... as we've found... not TOO often. :)
As far as releasing a demo in advance of the full version --- I really don't know anything about that. I'm actually kinda worried about that with Frayed Knights. I'm not really assuming it's a "demo" so much as it's a "public feedback version," really. It's serving nearly zero marketing purposes ... and I'm worried it will actually be of negative marketing value.
The problem is when said Powers That Be don't WANT to see the milestone, but want to see the current build. Or see whatever improvements were made since the milestone / demo. And give you only a couple of hours' notice.
Then you start getting paranoid.
@Klaim - I looked over the roadmap briefly, and it looks fine AFAICT. An okay plan now is better than a perfect one in two months, and it's far better than having none at all. Get feedback early and often, but... as we've found... not TOO often. :)
As far as releasing a demo in advance of the full version --- I really don't know anything about that. I'm actually kinda worried about that with Frayed Knights. I'm not really assuming it's a "demo" so much as it's a "public feedback version," really. It's serving nearly zero marketing purposes ... and I'm worried it will actually be of negative marketing value.
Funny you post this today; I just handed off the demo of my own project at work. Sure enough, there's a bit of code in there I'm not happy about.
I got to thinking: I should have forked this code a week ago, back before the rush really set in. Then after the demo, we'd have to re-do all the rush job fixes we've done in the last week. It'd be something planned on, so we'd have to keep good notes of what the problems were (and the fixes, if they were good fixes)
I got to thinking: I should have forked this code a week ago, back before the rush really set in. Then after the demo, we'd have to re-do all the rush job fixes we've done in the last week. It'd be something planned on, so we'd have to keep good notes of what the problems were (and the fixes, if they were good fixes)
Thanks Coyote ^^
"I'm actually kinda worried about that with Frayed Knights. I'm not really assuming it's a "demo" so much as it's a "public feedback version," really. It's serving nearly zero marketing purposes ... and I'm worried it will actually be of negative marketing value."
I feel the same. That's why I think the demo version we plan de release for IGF will be available only to testers and IGF-Contest judges.
When is the deadline for Frayed Knights? Wasn't it a project for a contest, if a remember well?
"I'm actually kinda worried about that with Frayed Knights. I'm not really assuming it's a "demo" so much as it's a "public feedback version," really. It's serving nearly zero marketing purposes ... and I'm worried it will actually be of negative marketing value."
I feel the same. That's why I think the demo version we plan de release for IGF will be available only to testers and IGF-Contest judges.
When is the deadline for Frayed Knights? Wasn't it a project for a contest, if a remember well?
The FK "Pilot Episode" - the "feedback demo" as I was mentioning - will be available on or about April 1st. I'll also have sign-ups in the next week or so for anybody wanting to participate in alpha tests during March. That'll be on the forums, most likely - it's easier to keep track of people that way.
And yeah, it's in the Dream Games competition, which ends at the end of April. I entered the first chapter of FK for the competition.
And yeah, it's in the Dream Games competition, which ends at the end of April. I entered the first chapter of FK for the competition.
Good article. I also work a day job making games for publishers, and make my own indie stuff in my spare time. I've come to the conclusion that you cannot do your best work when you are working with someone else's money - you simply have to give them what they ask for, which is not "your best work", but "good enough to release on schedule". It's only when you are entirely funding a project yourself that you have the freedom to be truly creative. This is why Valve games are good - no publisher pressure. This is also why I love making free games - no self imposed monetary pressure.
The best advice I've read on this topic is don't make the demo look done.
As for CVS, ouch. The new distributed version control system are so much better at merging branches that it becomes practical to do all new developments in branches and only commit fully working code to the mainline. I highly recommend you at least try the bzr five minute tutorial.
As for CVS, ouch. The new distributed version control system are so much better at merging branches that it becomes practical to do all new developments in branches and only commit fully working code to the mainline. I highly recommend you at least try the bzr five minute tutorial.
# posted by : February 05, 2008 10:14 PM
: February 05, 2008 10:14 PM
I wish I had control over our source control solution, but... no dice. But that's outside my control, and I don't feel an overwhelming need to fight that particular battle.
That article on demos, BTW, was awesome! Thank you for the link!
monoRAIL - I agree. In fact, that was my reason for going indie in the first place... to make my own games, the ones I was really passionate about. Not that I didn't LOVE working on several games in the past and devote a lot of time and energy to them. But there's something a lot more deeply satisfying about working on "your own" game. Or one that you have managed to make your own.
Post a Comment
That article on demos, BTW, was awesome! Thank you for the link!
monoRAIL - I agree. In fact, that was my reason for going indie in the first place... to make my own games, the ones I was really passionate about. Not that I didn't LOVE working on several games in the past and devote a lot of time and energy to them. But there's something a lot more deeply satisfying about working on "your own" game. Or one that you have managed to make your own.
Links to this post:
<< Home
