Friday, 9 September 2011

Game Testing FAQ, Beta Testing analysis

Stephen Reid, Senior Online Community Manager, posted the Game Testing FAQ on the official forums.

Beta Testing Weekends began last weekend, to get more people than ever before testing Star Wars: The Old Republic. We invited a number of testers in to experience the game this past weekend, and will continue to invite more testers, in greater numbers, as we move towards launch.

While the number of those invited will increase over time and while we plan to invite as many people as possible who are signed up for Game Testing to test, there are no guarantees that everyone who applies will be invited.

It is also important to note that not every weekend going forward will be a Beta Testing Weekend. In September, we're hoping for one more Beta Testing Weekend, which will be larger in invite size than the last.

Now, onto some FAQs and answers:

What's the difference between Early Game Access, Beta Testing Weekends, Guild Testing and Game Testing?

They are four different ways to get access to play The Old Republic. Details are as follows:

Early Game Access: Available to everyone who pre-orders any edition of Star Wars: The Old Republic and redeems their Pre-Order Code. Essentially a head-start, it will begin before the formal release date/launch day of the game. The final release client will be used to play. Exact duration will be revealed closer to our release date.

(Edited to reflect that Early Game Access is not another phase of testing; apologies for any confusion. )

Beta Testing Weekends: Large scale, short-term testing. We will be inviting registered testers to play the game in greater invite numbers than ever before. The first Beta Testing Weekend started on September 2nd. They will continue beyond September and will not necessarily occur every weekend.

Game Testing: The ongoing testing program. We continue to invite those who have registered for Game Testing into the program on a regular basis. Those selected will generally test the game for an extended period of time, as opposed to a Beta Testing Weekend.

Guild Testing: We are selecting guilds who are registered on our Guild HQ for inclusion into general Game Testing (see above). Invites to a guild do not necessarily extend to all guild members.

As part of Beta Testing Weekends, will the Game Testing Agreement (specifically, the Non-Disclosure Agreement) be lifted or modified?

No. The Game Testing Agreement is still fully in effect for Beta Testing Weekends. As before, you may tell others that you are part of the Game Testing Program (or that you have been invited to a Beta Testing Weekend). However, you may not reveal any other details about your game testing experience, including any aspect of the testing process, any details of the client, or any assets from the game (including text). Even after the Game Testing Weekend is over you may not detail your experiences.

Any breaches of the Game Testing Agreement at any time will result in your potential immediate removal from Game Testing and/or prevent you from being invited to any future testing.

Will Beta Testing Weekend invites be sent in the usual way Game Testing invites are sent?

Yes. Those invited will receive an email to the email address they have registered on their SWTOR account. The email will come from no-reply@bioware.swtor.com. Ensure you have that email address registered in your email whitelist so that the email does not go into your spam folder. If you think you have deleted the email by mistake, you can check to see if you have been invited by logging in to the SWTOR.com web site at www.swtor.com/tester/. If you are unsure of an email's authenticity, we strongly advise you to login to the website to confirm. If you are still unsure, please email support@swtor.com

After a Beta Testing Weekend is over, will I get re-invited to a future Beta Testing Weekend?

At this time, we are not planning to re-invite those who are invited to a Beta Testing Weekend to future Beta Testing Weekends. This may change in the future.

I previously participated in Game Testing, but am no longer participating. Does that omit me from being re-selected, either for a Beta Testing Weekend, or Game Testing generally?

Currently, if you have previously been selected to test Star Wars: The Old Republic, you will normally not be re-selected for future testing. (This applies whether you were selected previously for Game Testing, a Beta Testing Weekend, or any other testing access.) That may change closer to launch, but we want to get as many fresh eyes on the game as possible.

Will future Beta Testing Weekends be three days long?

Most Beta Testing Weekends will last roughly from a Friday evening to Sunday evening. In some cases, we may choose to extend the time available at our discretion.

I got an invite to a Beta Testing Weekend, but can't test during that weekend due to other commitments. Can I postpone my invite to another weekend?

You may choose, if you wish, to decline a Beta Testing Weekend invite. If you do this, you will be put back into the general 'pool' for potential future selection. However, your selection in the future is not guaranteed; you will be picked randomly.

Will those invited to Beta Testing Weekends be playing on the same servers as those in Game Testing?

For the first weekend, those invited will be playing on a separate server. This will probably be the format for future Beta Testing Weekends, but if our requirements change, we may invite people to specific servers (or potentially allow them to choose from multiple servers).

Will extra time be given to download the SWTOR game client before a Beta Testing Weekend starts?

Yes. Invites to Beta Testing Weekends are generally sent out a few days before the weekend to allow time to download the client. However, because of internet download speeds and the invite schedule, we cannot guarantee you will have the game client downloaded and installed before the weekend starts.

The size of the game client is currently approximately 27Gb.

Can I use the same game client in testing and then at launch?

We cannot guarantee that you will not have to re-download and re-install the game client at any point before the start of Early Game Access. Periodically addditional content patches may be required during any period of Game Testing.

Do those invited to Beta Testing Weekends get access to Forums to talk about their experience and give feedback?

Yes. Specific Forums for Beta Testing Weekends will be opened from when invites are sent out for that weekend, until a short period of time after the weekend's testing is over. These will only be accessible to those invited to the specific Beta Testing Weekend.

Are invites to Game Testing still random?

Yes. Those invited to Game Testing - of any kind - are still currently randomly selected from those who have opted-in to be eligible for Game Testing.

There used to be a 'System Scan' function on the website as part of Game Testing sign-up, and now it's gone. How does this affect me?

We no longer require potential Game Testing applicants to perform a 'system scan'. If you have previously scanned your PC and have since upgraded it, don't be concerned. You're still eligible for selection to Game Testing.

When does Game Testing for EU-based players start?

Game Testing for EU-based players is imminent, and will include localized French and German versions of the game. Future phases of Game Testing for EU players will happen on EU-based servers. Beta Testing Weekends will also begin specifically for EU players in the same timeframe.

As EU players are being invited now and other players have been testing for longer, will feedback from EU players make a difference to the game at launch?

Yes. Every piece of feedback from a tester is potentially useful. Even if a piece of feedback is not acted upon before launch, it may be used to change the game in some way in the future.

As part of Guild Testing, will guilds be invited to Beta Testing Weekends?

Guilds are generally invited to the main Game Testing program. Individual guild members, through random selection, may be invited to Beta Testing Weekends, but guilds will not be selected for inclusion in Beta Testing Weekends as part of Guild Testing.

I have infractions on my Forums account. Does that prevent me from being invited to Game Testing?

No, it does not, although accruing enough infractions could lead to a Forums ban. If you are banned from the Forums, you would also be removed from Game Testing.

I pre-ordered The Old Republic; does that allow me access to Game Testing or Beta Testing Weekends?

At this time, pre-ordering The Old Republic and redeeming your Pre-Order Code ensures you gain Early Game Access.

I live outside of the initial launch countries for Star Wars: The Old Republic; will I be able to take part in Game Testing in any way?

We're hoping to extend Game Testing to many more countries before launch, but we have no estimate on exactly when that will happen.

Also posted shortly after, was an update about Beta Testing Weekend number one.

Hello everyone. Now that the first-ever Beta Testing Weekend is over we wanted to give you a bit of insight into how it went and what our next steps are. If you haven't read our FAQ about Beta Testing Weekends, you might want to take a look at that in conjunction with this.

For our first Beta Testing Weekend, we invited a large number of testers to play Star Wars: The Old Republic during Labor Day Weekend (a US holiday). While the number of testers invited was larger than any previous invite - many thousands - the number was still a fraction of what we're planning for later weekends.

What were the objectives of the weekend?

Primarily this was a small scale simulation of a launch scenario. That means, bringing on a large number of players, starting with the invite process, then proceeding through download, install, patching and logging on to a fresh new server.

We were looking to have large numbers of players on the game's Origin Worlds, like a launch day; we were looking to test our limits on PCU (peak concurrent users); and obviously we were testing to see how all this 'felt' to new players and whether they enjoyed the experience.

On top of that, as always, we were seeking feedback on the game itself - how it feels, how it plays, what people like and dislike.

How did it go?

On balance, extremely well. Every goal we had was met or exceeded, and we had no major downtime. This is good news for us, as now we can move forward in future Testing Weekends and increase the overall number of invites.

There were hitches, as you may know. We had an issue introduced into our invite process that caused a number of people to become 'stuck' in specific database state. That was fixed by our web team as fast as possible, and while that delayed the initial download for some, we got as many as we could into the game. We are thoroughly testing that process to ensure it doesn't happen again - it was in fact the very first time we've had invite issues since Game Testing began.

Here are a few random quotes from four different testers within their Forums:
“I absolutely had fun. I found myself still finding enemies and attacking long after I had fulfilled the requirements of the mission simply because it was fun to just blast my enemies. You guys honestly put my jaw on the floor with the amazing world you built... and the look of everything. Video's had me skeptical but being in game washed away any doubts I had. I think you guys have nailed Star Wars for what it is..a galaxy far far away and a long time ago..enough said.”

"I really can't wait to see how the story unfolds. In most MMOs I play, I pick up the quest without even reading it but with these quests I am listening to the amazing voice acting and building my own story."

"Words alone can’t express how epic the voiceover is. Normally I end up making fun of the voiceovers in other games but this one caught me off guard. In addition, the in-game music and ambient sounds was also amazing. I didn’t feel overwhelmed with the background/ambient sound and the in-game music matched the current situation very well. Like I said before, overall this is and will be an amazing game. I just purchased the Collector’s Edition."

"Overall, I would say I had a lot of fun. In addition to the quests and PvP, I would sometimes stop and just look around at the planet or station I’m on. The amount of detail that was put into the environment was incredible. Well done."
What's next?

We're rolling out a major new build for The Old Republic very soon, and we're waiting until that build is fully tested and ready to go before we invite people to another Beta Testing Weekend. We're still hopeful that will happen in September, but we have no guarantees.

Thanks for reading, and I hope this gave you some insight into this weekend's events!
Some things were of interest to me in here. First the option to "turn down" your Beta Weekend invite doesn't postpone it, you have the same chance of getting in again as you did before. So erm, if you get invited a weekend you can't play, that's going to be a hard call. I would likely turn it down and hope for another slot, but only if I couldn't suddenly cancel all my weekend plans of course.

The system scan thing is interesting, this means that they are not looking at hardware specs to choose players for the weekends. That means the invites are truly random from those who applied. I guess the client software has some sort of system scanner in it, and will upload details of the systems being tested on.

The fact that "The first Beta Testing Weekend started on September 2nd. They will continue beyond September and will not necessarily occur every weekend" is very good news, and means there is hope for EUers to get in.

Chris Collins then posted about when EU players will get into the Beta Weekends (or not):

Just to clarify - there is still no change on the post I made earlier in the week on this. We're still going to be inviting a large batch of EU testers to the General Testing service (on the US environment), so there will be EU players testing the game prior the EU environments being ready for testing.
This still does not answer when the EU environment will be ready for Beta Weekends, although it seems likely quite a while away, perhaps in October. Still, given that the Beta Weekends will be running in October, we have a chance to get in.

No comments:

Post a Comment