What I Learned From James Petruzzi
May 15, 2012 16 Comments
Long before I asked James Petruzzi, developer of Take Arms and 48 Chambers, to do his excellent Tales from the Dev Side editorial for my site, I sought out his help for a planned article that never really panned out. Although that didn’t come to pass, the hours Brian and I spent talking with James completely altered my perception of how certain Xbox Live Indie Games should be judged. Before that conversation, I didn’t appreciate the absurd difficulty and almost unbelievable sounding limitations that Xbox Live Indie Game developers are saddled with. As someone who has never developed games, I couldn’t grasp just how hard it was. Mind you, I was (and really still am) new to the XBLIG scene. I was told that XNA was one of the simplest development tools many long-time indie developers had worked with. So it was like “well if that’s the case, why is putting online in your game such a big deal?”
Well, obviously I was wrong. I quietly backed away from my “games should have online functions” policy. Sure, I will still say that games can benefit from online play, or having online leaderboards, but I’m not going to let that be the focus of any review, which I had done in the past.
I’ve reviewed multiple games with online functions, and about two months ago, Brian and myself came to a realization: not a single online XBLIG we’ve played has ever been without some really serious glitches. That is without exception. It is universally true. Most of these games I review shortly after their release, and it’s not unusual for me to have to accept a review code to give to someone else to test the online feature because of the lack of other active players. This is only time I do accept review tokens. The code is given to someone else, while my copy is purchased by me.
Bug Ball was the game that created a change in my online review policy. It’s a good game, but networking issues greatly hampered its online playability.
I have a reputation as being the harshest critic on the XBLIG scene, and I’ve certainly earned it. I’ve been told I’m overly brutal, too nit-picky, and sometimes even mean. That might all be true, but there is one thing you can’t deny: I’m fair. Every game I review starts with a clean slate.
Back in February, a developer requested that I play their latest game, Bug Ball. A review code was provided, which I gave to Brain and his roommate. We really enjoyed the game, but unfortunately, it was riddled with multiple glitches related to online play. Characters would disappear from one player’s screen, the ball would disappear from one player’s screen, or sometimes the game would just stop working on one of our sides. I believe this was the first online game I reviewed following my conversation with James, and thus it was the first time I was aware that the developer had no way of knowing that these kind of glitches were happening. After all, they could not truly test the game over Xbox Live.
Brian and I talked about it, and we both decided that if I was to publish a review noting the glitches and how it ruined the experience for us, it would eliminate my right to claim that, no matter what I’m accused of, I’m always fair. Because slamming a game for issues a developer could not possibly have been aware of would not have been fair. Thus, we decided it was time for a change in Indie Gamer Chick policy. I contacted the developer and told them what issues we had, and that I would hold off on my review until they had a chance to fix the problems. Shortly there after, I added this policy to my FAQ.
I am often asked if I could help playtest games, or join the AppHub. I’ve had more than a dozen people generously offer to stake my XNA membership fee. But it’s not something I’m interested in, nor is it something I think I should be doing. As a critic, I feel it’s important that I stay separate from the development process. Although I understand that developers do want honest feedback in their games before they reach the marketplace, and I really do sympathize for them when they can’t get that, it shouldn’t come from me. Doing so would compromise the entire point of my site.
I had a lot of fun playing Spectrangle360, but multiple issues with online play has caused my review of it to be delayed while its developer works to figure out what is going wrong.
But, I am willing to help once the game reaches the marketplace. I am aware that, for many games, I’m the first person that will play it once it goes on sale. Since I’ve never talked about this policy outside my FAQ, I want to lay it out here. It goes as follows.
What I will do.
- I will contact the developer and list all glitches related to the networking parts of their game, explaining as clearly as I can what happened, both on my end and on the end of whoever my playing partner was.
- I can take any follow-up questions asking for clarification if necessary.
- I will leave it up to the developer whether they want me to go forward with writing the review immediately or if they would like me to hold off on it until they have a chance to fix the game.
- If the developer asks for me to hold off on the review, I will not count that as their Second Chance with the Chick, and they retain the right to request a second review once the original review is published and further patches are added to the game.
What I won’t do.
- I’m not willing to try an re-create any issues I come across for the developer. Besides, I usually play the game long enough to see the same glitch happen multiple times. Once the game returns to development, it’s up to them to figure out how to test it.
- I’m not willing to test the game with the developer to try to set off the issues. Again, once I’ve sent the information back to the developer, I consider the game to be back in development, which I should have no part of.
- I’m not willing to continue to play the game some more to try to find even more issues. Once the game is in the market and thus playable by the developer on the network it was designed for, they should be busy themselves looking for issues. Asking me to do your work for you takes time away from me being able to play games from other developers who are eager to get their games reviewed here.
- Once the developer tells me they’ve fixed the problems and are ready for the game to be replayed for its review, I will not inform them of any further glitches that come up. The game will be reviewed as is, and any further fixes will have to use up your Second Chance with the Chick. So make sure that when you tell me the game is ready, you’ve tested it thoroughly and are sure it’s as ready as it can be.
By the way, I certainly hope nothing here or in James’ Tales from the Dev Side discourages developers from trying to add online components to their games. Yes, doing so is extremely challenging, and maybe even not worth the effort. However, if you came to the scene looking to challenge yourself, why sell yourself short? It’s almost like what John F. Kennedy said of going to the moon. You choose to put online in your games. Not because it is easy, but because it is hard. A developer who can make a game with online play on Xbox Live Indie Games is a skilled developer indeed.
I have nothing but respect for the Xbox Live Indie Game community, and I’m always willing to offer advice when someone wants it. I know a lot of you wish I was willing to help more in the development process, and given how crappy the playing testing and peer review system you guys have to deal with is, I can’t blame you. Because I feel that doing so is a conflict of interest, I regretfully have to turn you down. But, when it comes to online play, I am willing to lend you a teeny tiny hand. I’m still the same Indie Gamer Chick I’ve always been. I call it like I see it. I’ve absolutely demolished games here. I show no mercy. But with online XBLIGs, I’m willing to cut you some slack and give you a chance to make things better. Why? Because it’s the right thing to do.
Indie Gamer Chick 
I think your change is a very enlightened one. It is more difficult to playtest games on XBLIG than it is on, say, PC for release on Steam. The PC is obviously a very open platform and you can work hard and get interested people to playtest with you for months on end.
I’ve seen only the very popular developers manage to get a community of other XBLIGers to get online and try something 16 player (like James Silva’s ZP2K9).
The other big stopping point for multiplayer is that you can go through all that effort and have no community, and that’s not a complaint about XBLIG. That’s just a reality that we are small and independent.
I think devs should focus on quality online coop experiences that will build a community of fans to your game. Try to make a game that doesn’t end (easy to say, hard to do). Random but high quality levels, worlds, zones, whatever they are for your game is a great way to keep people playing your game and more importantly FINDING other people to play with.
“A developer who can make a game with online play on Xbox Live Indie Games is a skilled developer indeed.”
This is really tough. A developer who is working with a blindfold and without a net… is that skill? Luck? Or is it just not trying to do too much? We could argue that a developer who makes it work was smart or skilled to recognize the limitations, but a master craftsman also has master tools, and we just don’t have those with XBLIG.
XNA Game Studio and the XNA Framework (the code stuff) is a joy to work with. It is very productive, and you can get high quality games running in a reasonably short amount of time, but we haven’t been given the tools that games are going to need now or very very soon (quality network test and debug cycle and quite frankly access to outside servers so we can create more complex data driven games). I’m not saying I expect MSFT to do any of this unless their hand is forced by the success of indie development on say… the PS4 or WiiU.
MSFT really allowed something special to happen with XBLIG, and they had their sales sights set too high. Now disillusioned, they keep the lights running but aren’t building more tools so devs can do great things.
So are you saying that what guys like Discord James or James Silva accomplished with their games was based more on luck than skill? Come on now.
Otherwise, I agree with you. Actually, there was one game that did play almost entirely glitch free online. Unfortunately, it was Hearts of Men, which had the dumbest concept for online multiplayer. Thinking back, I honestly don’t think any glitch happened at all with that one. So they did well with it. But the concept was beyond stupid.
Yeah, keeping an online community going is super hard. Focusing on making games with coop gameplay isn’t really the answer, some games just don’t work coop (like fighting games, competitive puzzle games, etc.)
There are a few tricks devs can do:
-support “join in progress”, a lot of games only let player join while the game is running a Lobby screen.
-make sure the game supports the “invite” system so people can play with their friends list.
Thank you very much for the mention. As to the patch if anyone has purchased Spectrangle360 we are working to get it done, please be patient with us. We are sorry if the online play has been spoiled by the issues IndieGamerChick is refereeing too.
I have to say, when Spectrangle worked glitch free, myself and Brian enjoyed it very much. The problem was, of the 10 rounds we played, 8 had a glitch that caused our screens to be different. The mechanics worked the same, that part didn’t glitch, but the pieces appeared wrong on the screen, which made playing the game difficult. For example, if I lined up a blue piece to a blue piece, on Brian’s end it might look like I lined it up to a red piece, which is impossible. This happened in 8 out of 10 games.
When this is fixed, the game will get a positive review, because it is a GOOD game. No doubt about it. It’s fun. But it needs to be fixed. Honestly, I would maybe even suggest pulling it from the market until the patch is ready. You don’t want word of mouth to spread that the game is broken. Not everyone will know that it CAN be fixed, or that it is actively being fixed, and it could hurt overall sales. BUT, seek the advice of other developers first. Don’t take my word on it. I would ask James what he thinks.
Enjoyed this article and the one linked below.
Regarding network play on XBLIG. It is all very true. It is why I spent 5 chapters covering networking topics in my XNA Game Studio 3.0 Unleashed book. I expected that content alone to set my book apart from the rest. I think it did, but the problem is no one cared.
As was noted in this post:
https://indiegamerchick.com/2012/05/10/tales-from-the-dev-side-making-a-multiplayer-xblig-by-james-petruzzi/
Making multiplayer games is hard. It has to be done from the beginning and can’t easily be added in at the end.
The reason no one cared about networking being covered in my book is because it is hard and folks are not making much money on XBLIG as it is, much less if they double their development time with networking code.
I tried to offset this with some example code. The code is now open source on CodePlex:
http://xnags30unleashed.codeplex.com/
So I’d encourage folks that are serious about creating multiplayer games in XBLIG to check out those chapters of the book and if nothing else, use the code from Chapters 28 or 29 depending on the type of game being created.
It is a lot of work, but requires for folks to actually be playing your game at the same time. It is hard enough getting folks to spend money on XBLIG games that don’t have achievements. It is even harder trying to find folks to play with online.
Anyway, I hope this will help folks who are working on a game and are adding or want to add multiplayer to their games. Code is free to use under MsPL.
Thanks,
Chad Carter
http://about.me/chadcarter
Great book Chad! I’ve read it several times and reference it frequently.
With XNA memberships we’re tasked with peer reviewing one another’s (should it be ‘, or just s?) games and performing technical tasks like pulling controllers, memory cards, etc., but as you mention in the article, even we don’t have access to outside servers to test most multiplayer related content.
It’s next to impossible doing any organizing on the App Hub site itself. Only way I think it’d even be possible is to organize an official Peer Review online session through Twitter. Even still, what a pain. Everyone’s in different timezones and have their own stuff to deal with.
Oddly I have had some very limited experience with XNA and a few other game development things in my time and well all I can say about XNA is I could get no where due to me knowing next to nothing about coding.
The whole system does worry me as the peer review thing has had instances (and I believe many more not said about) with the peer review system being slightly abused, these are only rumours but I’ve heard horror stories of some studios being messaged and told they’d only pass their game on peer review if they went and passed another game before, undermining the peer review process as a game shouldn’t pass if it has issues but shouldn’t be failed when its fine. Also some studios seemingly get a game out in days (Moe Mekuri 2.5 being a good example) while others do tend to languish stuck in peer review for long amounts of time. I think the fact some companies aren’t too interested even at this early level with producing something working when they could simply push out something to earn some money is a real problem when the system itself does depend on these people.
oddly from following free lance games for a bit I know there they are on about trying to set up something to get developers to peer review one another more and actually stop this practice of passing games just to get yours out and actually point out issues etc, as with some indie games I’m just shocked at the state I end up playing them in (even just the trials having major issues) I tend not to be playing the games right on release and I’ve seen some games really broken even after having to download a supposed patched version.
I have to say I can see where you’re coming from on the multiplayer point, and while personally I don’t tend to care about online multiplayer, as while some games are very good the amount of people who buy and regularly play the multiplayer for a lot of indie games can be quite small. That’s not to say it can’t add to the game but I do tend to think offline multiplayer can make a bigger difference with gamespot actually doing their series “It came from xbox live” it gives a good chance for offline multiplayer to be showcased. The other great benefit of having some kind of multiplayer means in a party setting it can get some playtime there as a bit of a novelty game. It really does seem like wasted potential to see single player racing games with no multiplayer mode at all.
I am also dubious about the future of XBLIG as while MS have said they want to add kinect support etc to the XNA kit they don’t seem to be doing much to keep it ticking over as recently there have been numerous things about sales figures breaking and some developers not getting paid till a month or so later than they were meant to.
Pingback: What I Learned From James Petruzzi « Indie Gamer Chick | Game Review Guide
If a developer is going to release a game with online play, the NEED to test it. You can’t give them a pass because testing it is difficult. If testing is too hard, then don’t include multiplayer / online features. Testing alone is completely possible. 2 Xboxes, 2 XNA memberships, 2 Gold Accounts. Need more people? Find some friends. That said, it’s difficult and can be costly, especially when purchasing extra memberships or hardware. But that’s not a good excuse for not testing it.
I did online high scores for the original Hypership Out of Control on XBLIG. It was a complete PITA arranging people to get together and test out aspects of the game and to track down bugs. I’d say just online scores easily accounted for an extra month of development time. And while cool for the first month or so, the peer to peer model (the only model we have) for online scores doesn’t work well in absence of large numbers of users.
Based on the extra dev time, associated costs, and minor return on investment, I’ve deemed online features simply not worth the effort on XBLIG.
The only thing I didn’t like about adding online multiplayer to my game was debugging, running the game on my xbox and three PCs at once (praise be to old laptops!), seeing one of the players dropping out, checking which PC it was, HOPING it wasn’t the old Compaq with the Celeron CPU…
“After all, they could not truly test the game over Xbox Live.”
(2 developer accounts) + (2 XBOX) = truly test
I think she (or they) were talking about the issues they had when it came to having multiple (like 10 ) players in a game at once.
I could be mistaken, however.
Pingback: Chasm | Indie Gamer Chick