Suggestions -How can we improve the BETA test experience for you?

[mgilmartin]

We are interested in finding out how we can improve our BETA test process. Please post your suggestions here.

Mike GIlmartin
ATARI, INC

[Andyindy]

Like you said on a post in another thread, you should involve the date the tester applyed for the Beta Test.

I'll keep my idea's coming when I get them.

*Andy*

[Andyindy]

There has been alot talk about having a chat room for the Beta Testers.

Most of us agree that it should be passworded and that it would be a good idea.

*Andy*

[W4R(H1LD]

I agree that a chat room would be a great idea. Then we could arrange times during the week that a rep from atari might be able to drop in for discussions live via chat. :)

[Andyindy]

That would be good so then we could have easier contact with the developers so we could give them better feedback.

*Andy*

[Pyros]

Ingame bug reports the link directly to the bug report on our betatests.net account would help.
Also, as suggested, beta tester chating and scheduled chats with people on the support team, or QA team would also help our input and bugs be heard.
Another suggestion would be making the community more self efficient, by making members of the community who have proved themselves in helping others and testing "VIP community members", giving them access to some of the mod functions for the beta board so they can keep things organized and help the online Atari staff keep this section organized, as it will probably become more chaotic as more and more betas become available.

[Andyindy]

I mean it will make it alot easier to explain a bug or something to some over cht because if they don't understand somethign you say they can just say so and you can try explaining it another way.

I like the idea of the VIP members so that they can help sort out the message boards. I mean Gilly and the Atari staff would have to find out who the best and most relaible people are so that they can trust us. But I think that is a good idea, well done Pyros.

*Andy*

[MaineManiac]

If my CD key worked I could actually do some testing...

As it is right now, my testing expereince sucks. :)

[Ameth]

Some sort of interface help would be nice. /help doesnt get anything

Cant figure out why "chat" box does not have chat but instead tells me how close the mobs are and how much I and they hit for. Isnt "chat" for conversing with the other people in "MMORPG"?

For those of us whos acceptance letter got religated and deleted in the "spam" folders... better instructions for use of game. I didnt find out I was accepted till I checked my beta status. The download and install instructions are great... but if you dont know about the web page to log into... the game icon does squat. Very frustrating.

Also, is it normal for the logon page to not recognized new characters? I played for 2 hours and logged off the game, yet the web page claims I have no characters :(

Lastly, since the bug report page is down... how do we send in our feedback? Email support?

[Rikkir]

I think a chat board would be a great idea. It would be nice to be able to chat with other beta test to help issues you may be having or to help other those experiences you may have had in game or system wise. I am not sure what the problem is on Tazzon but the secondary chat room would be great.

Rikkir

[girl_1]

(Warning - complex suggestion below)

When we register for betatesting you do get some demographics on applicants. What would be cool is if you could make sub-groups of people. You could group people by playing time (the hours they generally play) as well as - age, location, system specs. You could even assign one person in a group as a moderator - to organize and parse specific information that you wanted to distribute to testers or - to collect data from them.

Why would this be useful ?

for the testers it would be useful because it would give some continuity over a longer period of time. it would be easier to have someone you're familiar with to check bugs and problems against. it would make it more fun (if the other people in your group are reasonable and not nuts). it might put a damper on the people that flame/or cause problems in game.

for atari it would be useful because it would let you test specific things (such as how well the game plays when all people are located in a close area or how well the game plays on people that all have the same video card) and it would let you more quickly validate hardware specific problems. it would also be handy to help manage and collect information.

How would you do it ?

Like I said above you already collect location, previous game history, and hardware specific info. Sort through your db and make groupings (maybe add a checkbox on the reg form for people that would like to be grouped in the future) by features that would be useful - and assign one person in each group to contact/manage their sub group (heh - you could go the extra step and give them cool names, and maybe acknowledge the most active groups somehow but that's icing)

[Pyros]

girl_1: The already do almost just that. That's when when you sign up, you need to put hardware, your qualifications, etc. This way, if they need to test the game's compatability on a specific card (say Radeon 9700), they can draw users from the pool that has registered their video card as a Radeon 9700.
While they're not technically groups, don't have leaders, and still have random selections, it's a step forward in your idea. Testers who help more get more points, and therefore are put higher on the beta priority list. It's sort of like your idea, but not quite at that level yet.

[Kadin]

Well first off the big complaint I keep seeing is this..

<x - name> I have been on this list for ever and it still says pending.

So maybe send an email saying "Sorry you were not selected for (x-phase) of product testing we are beginning (y-Phase) in <blank> time. If you would like to wait for this phase please let us know ( or update your beta app)”

The people willing to test that phase would stay around and others wouldnt. Might also help some of the tons of record sorting that is having to be done and on a personal level let people know you haven’t forgotten them.

Pyros
I have tested everything from Games to Business Software. I think you have a great idea there it would help everyone regardless what level they choose to test and report at.

[[HA]Fodder4U]

^
I totally agree and am a bit peeved everyday I log in and still see pending next to the betas I signed up for.

[Andyindy]

But if you think about it. They get thousands of applications for the beta. If they had to go through every single one and change there options it may take a while.

*Andy*

[Kadin]

Andyindy

I think if i were them i would ask them to log in and change their own options, this would aslo let them know who instrested still.

But as for the email its self have it db generated :)

[Pyros]

Originally posted by Kadin
Andyindy

I think if i were them i would ask them to log in and change their own options, this would aslo let them know who instrested still.

But as for the email its self have it db generated :)

Let's say they didn't make it in phase 1, and want to go to phase 2, they need to log into their account, go to their beta application, and select "sign me up for phase 2". As you said, this would seperate the active members from the non-active ones. This, and the automated "rejection" e-mail is one I fully support. A solid idea. Good thinking.

[patag]

I would suggest you have acceptance for specific stages of the game and accept and deny people at once.

When I beta tested PlanetSide for SoE, they staged the beta test in exactly that manner. If you got accepted in the Alpha testing, they sent you accept / deny email. If you were denied, then you would be automatically be enrolled in the next stage until either a) you were accepted or b) the game was released.

Having something say pending for ever doesn't help you or the person wanting into the beta. People need the constant feedback, image a file download without a little status bar saying what % is finished. I'm pretty sure when people face the unknown, they turn away quickly.

Other suggestions: select and invite people who already have fan sites up and running. Look at it this way: they're already promoting your game, why not honor those who are spending their time and money attempting to build a community around your game. I say this with a slanted view of course, since I run http://horizons.universalrealms.com (shameless plug, but hey!). :)

And it may not make you wealthy, but accepting me into beta would give you worlds of great karma.

[Pyros]

Originally posted by patag
Other suggestions: select and invite people who already have fan sites up and running. Look at it this way: they're already promoting your game, why not honor those who are spending their time and money attempting to build a community around your game. I say this with a slanted view of course, since I run http://horizons.universalrealms.com (shameless plug, but hey!). :)


Do they not do this already? I thought they have a fansite section/press section that gives these members priority.

[Halsy]

#1. Make certain your beta testers can access beta boards immediately.

#2. Manuals and information. I hate guessing how everything is supposed to work in a game. If you're admitting people to a beta you should have a functional manual.

[patag]

Originally posted by Pyros
Do they not do this already? I thought they have a fansite section/press section that gives these members priority.

I filled that out on my application, but I didn't get the impression that would queue my application any differently. If so, who do I need to talk to in order to get my key? :cool:

[Pyros]

Originally posted by patag
I filled that out on my application, but I didn't get the impression that would queue my application any differently. If so, who do I need to talk to in order to get my key? :cool:

I'd suggest mailing betatestadmin@atari.com and mentioning you run a fansite. They might be interested to have you join.

[Kadin]

Pyros

I think there are some great idea's floating around on how to keep that word "pending" from killing you every day when you check up on the beta sight ( yep ya might have guessed im in "pending" mode for horizons) . But i want to expand here alittle more.

((Ingame bug reports the link directly to the bug report on our betatests.net account would help. ))

I think this is the best idea yet, maybe i can expand on here. I liked the idea of tracking the bug reports to an account but if ya could add some feed back to that ( ie. little note from bug team that says "hey can you remember tell me what you were doing when this happened next time. ) etc. would go along way.


PS. patag, Nice web site :D

[Silver_Dragon27]

I haven't gotten into a beta yet, but from previous beta experiences I can suggest one thing.

Streamline and simplify. I know we are helping test a product of yours but I have seen too often where the process of reporting bugs is needlessly complex.

Reporting most bugs should take a minimum amount of time and should be possible ingame. If you can report a bug in under 30 seconds you will encourage more reports, vs a process that takes 5-10 minutes and requires logging out of the game.

I would also throw in a vote for a scheduled chat, maybe by invite only for two way communication with devs and testers.

[Andyindy]

Yeah I agree. I havent been accepted for a beta yet either but I know I will do a good job of it.

*Andy*

[Vetrik]

A listing of current bugs and ways to recreate them (if known) to help in testing is my suggestion

[Andyindy]

Thats what your meant to send to the dev team anyway.

*Andy*

[Vetrik]

A list of all bugs reported from everyone though, not just ones you have sent in yourself

[Andyindy]

I see what you mean, but I don't see how that helps.

*Andy*

[Vetrik]

Well if you knew of a certain bug already you could then try and recreate it different ways, maybe narrowing down what might be causing it. Developers might not be working on that bug atm, but when they do it will provide them with more information and it might help them fix it quicker.

just my suggestion though

[Pyros]

Originally posted by Vetrik
Well if you knew of a certain bug already you could then try and recreate it different ways, maybe narrowing down what might be causing it. Developers might not be working on that bug atm, but when they do it will provide them with more information and it might help them fix it quicker.

just my suggestion though

That's a good suggestion. Often, when bugs happen, they can be recreated more then once. But, users aren't going to know if they did it a different way, as they don't know that it's been done before.

[Andyindy]

Oh I get it now. So you find out what is causing it and you recreate it to see if you can find out what is exactly causing the problem.

*Andy*

[Lochlan]

I agree with some of the suggestions here. I have thus far only been involved in the application to be a beta tester, so I only have experience with that part.

Therefore, the suggestion I agree with most strongly, I will clarify here: It would be terrific, if there was more communication to your pending testers. Letting us know whether we made each round of beta testing via auto generated email would be very much appreciated. (Even if we didn't make the latest wave of beta testers, it would help very much to definitely know if we've made it or not without having to keep checking the site, only to see the "pending" status message.)

Thanks for listening! :)

~Loch

[Xilimyth]

Personally, I actually liked BETAing with the developers, or at least representatives of that said developers. It only happened to me once in the Anarachy Online beta, but it was interesting to be rping to tell how I felt about a certain ability.

That's just me though :)

[Merilae]

Beta testers are a panicky lot :) When they get their email acceptance notices, they're in a hurry to get started. Activation notices should have a link to a clear set of instructions with solutions to common problems previous testers have encountered (browser version info, ftp site addy, when to expect the cd in the mail, setting up an account and what to do if your credit info isn't accepted etc.)

I haven't been accepted to a beta yet and can't reallly comment on the instructions people are receiving, but I've seen a few posts regarding difficulties and confusion regarding activating the Horizons beta test.

Hammering out bugs with the account sign up process before release and making sure that your future live players can sign in easily is very important.

I'd also like to see info regaring how many beta accounts are being mailed out during the week and an explanation of the selection process. If only to cut down on the amount of posts asking "why haven't I been selected yet?"

Good communication, even as early as beta, goes a long way with the tester base and can only help "word of mouth" sales. I find informed customers have a lot more patience with game issues (servers down, problems logging in etc.) Even though beta testers can be annoying at times..they're not just people looking to play free and/or help test, they're a major part of the future customer base.

I'm impressed with the system Atari is setting up for their beta tests. I like to see my status regarding beta sign ups (ok I admit, I'd rather see "active" rather than "pending" next to my Horizons request :D). It prevents us from wondering if our beta applications got lost out in cyberworld :D I also foresee the possiblity for Atari to develop a "trusted" base of reliable testers. Which can only help us all!

WTG Atari

[Xilimyth]

ok I admit, I'd rather see "active" rather than "pending" next to my Horizons request ). It prevents us from wondering if our beta applications got lost out in cyberworld

Seeing that word is something I too would kill for. :) While the "luck of the draw" is annoying at times, it does give everyone a fair chance right?

Grr, stupid fair chances :cry:

[Andyindy]

ha ha. I'd also kill to see accepted instead of pending.

*Andy*

[Xilimyth]

I still have faith that if we offer to have our Incoporeal Souls incorporated into the game to help power the monster AI we MAY still have a chance at squeezing in. What do you think Andy? :D

[Andyindy]

Sounds good to me. ha ha :D

*Andy*

[proudclod]

Andy, the problem is that, you can kill a person to see active, but you still wont be able to play....:P Anyways....I would think betatesters should be able to have a special privelage of gettin a discount on games, I recently enrolled in the u2xmp beta test, and i just love it, but the problem is, i wont be able to afford unreal 2+ XMP since i dont have it to start with. If we would get special deals, or discounts or something like that, i would gladly buy both, but for now, im out of luck...:( Also, i dont mind waiting for beta tests, but As its been an issue for allot of us, i would like to know when the beta is over, so i dont have to sign on everyday to see pending, ive been signed up for a month, and a friend of mine, already got on the horizons beta, i know its all random, but i would like to know when the beta is over....Thats all :P(a month isnt a long time, but still im an RPG/MMORPG fanatic)

[Xilimyth]

Well, with Horizons apparently being released in November, there's still time, so don't give up hope. And if your friend gets on your nerves, ignore em. Just like I did ;)

[Andyindy]

Ha ha. Yeah I know. It's annoying seeing Pending all the time and then you find out from someone on the forum that the beta has finished.

*Andy*

[Xilimyth]

The BETA's finished?? *clenches his heart*

[orris]

quote:
--------------------------------------------------------------------------------
Originally posted by patag
Other suggestions: select and invite people who already have fan sites up and running. Look at it this way: they're already promoting your game, why not honor those who are spending their time and money attempting to build a community around your game. I say this with a slanted view of course, since I run http://horizons.universalrealms.com (shameless plug, but hey!).

--------------------------------------------------------------------------------



Do they not do this already? I thought they have a fansite section/press section that gives these members priority.

Not sure how Atari is, but if it were me deciding, having a fan page would put you on the NO list....atleast through the first few phases of beta. Someone that already has the means to reach thousands of gamers could really hurt the community perseption about the game. Also, I saw many pictures and stories about SWG beta from a friends Guild site...you had to be a member of the guild to access it...but with over 100 members in that guild(all of them of course MMORPGs) that info spread like wildfire. None of the other members were under any type of Non-discloser aggreement.
Having a fan site as a beta member doesnt really help for good publicity until after the NDA is dropped.
Just my 2 cents.

[Pyros]

Originally posted by orris
Not sure how Atari is, but if it were me deciding, having a fan page would put you on the NO list....atleast through the first few phases of beta. Someone that already has the means to reach thousands of gamers could really hurt the community perseption about the game. Also, I saw many pictures and stories about SWG beta from a friends Guild site...you had to be a member of the guild to access it...but with over 100 members in that guild(all of them of course MMORPGs) that info spread like wildfire. None of the other members were under any type of Non-discloser aggreement.
Having a fan site as a beta member doesnt really help for good publicity until after the NDA is dropped.
Just my 2 cents.

While this is usually a concern, for the more open phases, they try to add more webmasters in, as nothing is free publicity like a page that reaches thousands of gamers.

[r3b00t]

It would be unfair if they would put you on the No list just because you run a fansite or you're a member of the press. I'm a member of the press and I don't know why they should put me on the no list... I've beta tested already some games from EA and Microsoft and I never leaked anything.

I haven't been selected by atari yet and can't login anymore, always "Cannot find server". But it would be nice to put fansites or members from the press on a special list and when they do there job good maybe send them some extra exclusive new screens. I think everybody that runs a fan/gamingsite would be happy with new material. Or grant them the right the make a preview about the beta after a couple of months. I would really like to do something for my site with the beta's I get but I'll keep myself to the NDA. Giving permission for some things later on the betatesting would be nice.

[Xilimyth]

I don't know how Atari works internally, but perhaps with Members of the Press, there's a seperate NDA that is more strict or something.

I know BETAs generally have a harder time containing info from those two groups (fan sites and press), so they may need to know for security's sake.

[r3b00t]

I don't know why the security should be strickt for press. I work with companies as Nintendo, Sony, Microsoft... Atari requests more dan 70.000 visitors each month but the site isn't online so we don't have any support from atari yet. But as a member of the press I wouldn't risk the site and my reputation. I still have to work with these companies. Why risk the support of a company? I think that getting review and preview material and news from atari is more important than some screens and a preview you use while violating the NDA.

[Pyros]

Originally posted by r3b00t
I don't know why the security should be strickt for press. I work with companies as Nintendo, Sony, Microsoft... Atari requests more dan 70.000 visitors each month but the site isn't online so we don't have any support from atari yet. But as a member of the press I wouldn't risk the site and my reputation. I still have to work with these companies. Why risk the support of a company? I think that getting review and preview material and news from atari is more important than some screens and a preview you use while violating the NDA.

Some users leak it out anyways. Unofficial demos, or leaked betas usually come from members of the press who are dishonest and leak it out. I run a GameSpy fansite, and the publishers of the game, Microsoft, have no problem disclosing information to me. It really depends on how early the game is in development.

[abarickman]

Originally posted by MaineManiac
If my CD key worked I could actually do some testing...

As it is right now, my testing expereince sucks. :)
me to, sort of playing a game, but its not done yet.

[r3b00t]

It should be nice if you want to select a beta you could click on the gametitle and in a pop-up you get some information like the numer of betatesters allowed, in what fase the beta is, a description of the game, de requirement systemspecs, ... If you click on the game to request the beta the script checks your profile for your system specs so it can see if you are allowed for it. If your system specs are too low you get a message that you can't apply because you don't meet the requirements. This way it would be easier to a person from a list because you are quite sure he/she has te right system requirements. In your profile you can see a list of games you already betatested + what bugs you found and what suggestions you make. You could also find more info about what happened with it, like it got fixed by the developers or they worked around it, ... This way you know what you achieved. Haven't betatested before for atari so I don't know if such thing already existst.

When you apply for a betatest you can only apply if the right info is fulled in in your profile. This way your profile has to be ok en not empty to get in it. Once more less things to check by the betatestadmin. To select people you might use the date they've been waiting. When you put in what betatests they already done you can put in the date on which they can select. You can give new people a change for later beta's and more experienced people for the earlier builds. Because you have a date in which they registered if they haven't tested a beta yet, and you have the date from their last beta (also finish date) you can select people based on the time they've been waiting.

[Dalsa]

This sure got all the worms out of the wood work tehehe.. Lots of new faces. I think the time people have been waiting to beta test should some how effect their chances of being selected, not because I have been here while, I've actually only been here a week at most, but because I've read many a post from guys who have been here months and not been selected and others from people who get selected the day after. It seems rather unfair, I would prefer to wait a matter of months knowing I will get a place one day, than waiting without knowing either way.

Just my bit :haha: