Just got this comment in a previous post and thought I’d respond here rather than there.
So.
WHAT were you guys thinking, to go around saying it was ready to ship?
Because, y’know, it wasn’t. As just about every review and forum thread is acknowledging, some more heatedly than others. (PC Gamer: “You should not buy it.” Uh, wow. I’ve NEVER seen them be that blunt before.)
Does Stardock have an actual QA department? Is it organizationally independent of development? Are there good lines of communication between QA and the rest of the company? Do they write test plans? Do they run them? How can they test this game and not encounter the problems that were present?
Or is the only testing done by developers in their spare time?
How does something like this happen?
-Rollory
First off, welcome to the site! I don’t think I’ve seen you around here before. I’ve got some free games you can try, and if you’re here for the Name That Game! feature, I’ll be posting a new entry later today.
Second, allow me to answer your questions and respond to your statements in the order they were presented.
So.
Nice to meet you too!
WHAT were you guys thinking, to go around saying it was ready to ship?
Brad’s now infamous statement (which you can read here) was made in anger and exhaustion. Brad explains the situation (and apologizes) here.
Because, y’know, it wasn’t. As just about every review and forum thread is acknowledging, some more heatedly than others. (PC Gamer: “You should not buy it.” Uh, wow. I’ve NEVER seen them be that blunt before.)
PCGamer’s most recent article about elemental states “I’m glad Stardock are patching Elemental so quickly after its disastrous early launch, and I’m relieved to finally have the game in a playable state.”
Does Stardock have an actual QA department?
Yes.
Is it organizationally independent of development?
Yes.
Are there good lines of communication between QA and the rest of the company?
Yes. We use Skype chat channels to stay in constant touch and they use Jing to quickly post screenshots and movies of problems so that we developers can see what they are seeing.
Do they write test plans? Do they run them?
Yes and yes.
How can they test this game and not encounter the problems that were present?
And now you have raised my ire. As a former tester I can tell you that testers find bugs; they do not fix them. If bugs exist in a final product they are not the fault of the testers; they are the fault of the developers. Of course they saw the issues. They are not idiots, and I resent your suggestion that they are.
Or is the only testing done by developers in their spare time?
No. Although most of us developers do play the Impulse version in our spare time and keep track of any problems we find.
How does something like this happen?
At last, you have asked a truly salient question (although I’m sure you intended it to be rhetorical.)
“This” happened because it was the lesser evil. Stardock simply does not have the clout to release a retail game during Christmas. Our choices were to launch on August 24th or push the game back to February of next year. Pushing back would have had disastrous consequences for Stardock because of the partnerships we had made and the forfeiture of our retail space.
I’m sure you’re thinking, “Well, releasing an incomplete, buggy game is also going to have disastrous consequences!” And thus you’ve hit the crux. We were in a bind, and chose the lesser evil – to release on time and then work like the dickens to get the game to the state we and the players want (instead of, you know, sleeping like most people who have just shipped a game do). Yes, a lot of people have already had a negative initial reaction to the game. There’s nothing we can do about that. But Stardock has a reputation for continually improving their games over the months and years after its release, and we’re continuing that tradition by improving Elemental as quickly as we can and turning it into the game it deserves to be.
-Rollory
Again, welcome to the site, and I’m sure we’ll have lots of spirited debates in the future!
As a fellow game developer, I can totally relate to what Anthony says. Sometimes, bugs slip through the cracks (especially after major overhauls on already solid systems that have to be altered for one reason or another.) People who are quick to criticize have rarely ever made a game before, nor do they know the intricacies, effort and time (i.e. YEARS!) it can take to ship a product.
Devs understand that you paid for their product. If something breaks in game they won’t leave you hanging. They will fix it.
If you really want to help a developer fix a potentially good game, submit bug reports to customer service instead of trolling their blogs.
FWIW, I’ve been doing QA for a living for about 10 years now, the first three or so in gaming, actually. Have always had a great relationship with my devs, _and_ management. My job is to make sure they know exactly what the actual current state of the software is and what the consequences of releasing it in that state will be, that way they’re never taken aback by customer reactions, and can make appropriate plans for the future.
I do have the impression Stardock and/or Mr. Wardell was somewhat taken aback in this case.
You seem to be saying that there was a good internal sense of the product’s problems, and an awareness of the likely consequences. I am surprised at this. For one thing, I’m pretty sure I read in one of Brad Wardell’s messages that a lot of the problems encountered by customers had been unknown to Stardock prior to the release (he talks about how good it is to get so much bug data from the customers). For another, there seems to have been absolutely no hint of it before the actual release, which led directly to a conflict between expectations and actual results. I realize the risks of saying “yeah we’re releasing something buggy, might not want to buy it yet” but at the same time Stardock seems to have a pretty solid reputation among its fans for ongoing patching and support – so in this case, if there’d been a bit of a heads-up, the reaction might have been more along the lines of “doesn’t matter, Stardock’s good for it” and ongoing buzz. (Star Ruler and Dwarf Fortress come to mind as games that recently have done essentially this – the Star Ruler guys ran out of cash and openly admitted they were selling the game to fund ongoing work on it; Tarn Adams stated openly that DF 0.31 was going to be a buggy piece of crap for some months and people shouldn’t expect too much out of it. In neither case does it seem to have harmed them.)
At the very least, in the case of a necessarily bad release, I would expect a message on the forums or somewhere, something like “NOTE: We are aware that the game is currently not stable on certain systems, as well as of errors with game mechanics such-and-such. Please see [link] for a listing of the most serious current issues, and in which patch we expect to include fixes for them.” – all present and in place BEFORE the first copy is sold, so that the moment the first irate customer goes looking for answers, he can’t help but find them. That’s just basic project management there; you prioritize known issues, set up a schedule for dealing with them, pad it a bit, and let the customers know so they don’t explode at you. I haven’t bought the game, so maybe this did happen, but I haven’t seen any mention of it.
Also, you mentioned Christmas, but I’ve also seen elsewhere a reference to another window in February. Are sales really that much worse in February than now? If not, the only way I can see justifying _not_ waiting another couple months (and using the time for additional content and/or touching up some of the “blandness” that I have seen referenced in some cases – spell effects for example seems to be a not uncommon complaint) is if Stardock really needed the money. But I’ve always had the impression Stardock was on a solid financial footing and did the games sort of as extras.
“(although I’m sure you intended it to be rhetorical.)”
Actually, it was the preceding set of questions that were potentially rhetorical. (I _have_ seen places where the answer to those would be other than what you would expect, but I am an optimist; I always hope not to see such foolishness again.) It’s fascinating to me to see these sorts of screwups and to understand how they happen. I’ve been involved in a couple bad launches myself, so far I haven’t ever gone through the same disaster twice, or indeed repeated someone else’s disaster yet.
“If you really want to help a developer fix a potentially good game, submit bug reports ”
I charge $30 an hour for that.
May I add that this does not seem to fully jibe with your comments in the OP:
forums.elementalgame.com/394855/page/2/#2753014
Man, talk about being between a rock and a hard place.
The good news I think, is that a lot of people have plenty of faith that the game will receive updates soon enough make it into what you had really wanted to ship. I mean that in a good way – I once worked at a place where post-ship patches were shot-down by the publisher.
I’ve said everything I can/want to about Elemental’s launch and current status. Rollory, you are free to continue to pile on, but it’s not going to affect the game as it currently is or what it will become in the future.
in any case i’m sure there’s always going to be scheduling problems with game release; it’s practically unavoidable from what i understand given the nature of these publishing companies and their marketing tactics. i’ll always be curious to know what really most games would be like if the dev’s had half the time they wanted to work on a game, hell even a single game component. good luck in any case.
“shiny”
there is -> there are
“shiny”
[…] rather than a genuine mistake, and that Brad is lying to avoid being exposed as a hypocrite, see here. I genuinely feel for the guy, but the fact that Brad Wardell is actually being given points for […]
Anthony,
Fascinating insights. Could you clarify this: “Pushing back would have had disastrous consequences for Stardock because of the partnerships we had made and the forfeiture of our retail space.”
To a run of the mill game consumer and Indie advocate, can you put this in layman’s terms? Does that mean Stardock couldn’t sell Elemental in Walmart?
Tyler, the way I understand it (things work differently on this side of the pond), the retail market is, like many other aspects of the games business, broken, and the publishers actually rent shelf space from the retailers.
I think you are beeing a little to hard against Rollory given that all he knows is what Frogboys says, i.e. “we had no idea the game was buggy or bad”.
You seem to confirm that Frogboy is just plain lying on the homepage.