Forums Kongregate

How did they do it?

9 posts

Flag Post

Okay, so over in Barrens Chat tonight, we’re in the middle of a debate. An intelligent one, surprising ly. It’s not about how they chose the beta testers, it’s why they chose them that way. If an admin who knows why would like to step in, please do. We’ve thrown around a lot of ideas, each with a huge flaw in it, and that includes by level. So we would like an admin to please help clear up the fog for us.

 
Flag Post

There are threads here that discuss this issue already, with some comments from admins. Browse the first page or two of this forum.

 
Flag Post

To be blunt: It was the only way they could do that is realistic, isn’t biased, and still manages to put in people more likely to activly test.

Random would likely put in quite a few people who signed up and left.

Admin picks would be too biased, and in all likelyhood the admins and mods don’t know all 20k beta testers. Plus active members who would test well, but are disliked by the community would be screwed.

A complicated system involving points, forum activity, mod warning history, etc. that chooses perfect testers from all levels would be impossible to set up.

So the only reasonable option is level based pickings.

 
Flag Post

Some threads, for reference:

[Kongai] Beta Test Out?

The way of choosing the testers!

New Round of Kongai Beta Invites!

[Kongai] Beta Test Round 2 Invites

 
Flag Post

Here’s how the dabate goes, sit back and relax for this one:

hiperson134: Anyways, I’ll admit that the way they choose people is flawed, but how else would they do it?

Boometh: Forum behavior?

hiperson134: but not everyone at Kong goes to the forums. I’ve been going to Kong for almost…6 months now, and I rarely go to the forums

Boometh: Not everyone tries leveling up either..

hiperson134: I think chat behavior would be better than forum behavior

hiperson134: chat behavior would probly be even better than by level

hiperson134: heh, they probably had this same exact discussion when they decided how to pick the people.

skyfallsdown: No, because people like me who never bother with chat would be excluded

hiperson134: then there’s the next problem. Where does one go after seeing that idea is flawed?

Boometh: Well I don’t bother with the leveling system and I got excluded.

BleedingBlade: both are good points

imbrium: nothing would make everyone happy7

skyfallsdown: But if you’re trying to get people to test a game for you, the best way to do it is get people who have proven themselves to be good gamers.

Radec: or obsessive gamers.

hiperson134: I half agree and half disagree with that sky

imbrium: level system is probably the easiest way to tell how much time ppl spend on the site in general

hiperson134: I disagree because people can be great at games that don’t have badges, thus they don’t get as mch points

Boometh: Well actuy, some good gamers are pretty good yet don’t bother with achievements.

BleedingBlade: not nessicarily a guy hacked his prof and hes lvl8 and he joined about 2 hours ago

hiperson134: boometh, how long have you been going to Kong?

Boometh: Since june 2007!

Boometh: And I’m level 12..

BleedingBlade: woah

imbrium: but even if you don’t bother with them, you’re bound to get them at some point

imbrium: i joined in december 07. and am level 8

Boometh: Not exactly, imbrium.

BleedingBlade: i joined nov 07

hiperson134: See, there’s another thing. It’s now obvious that there are people who are lower levels, yet have been going to Kong since it’s very early times

Radec: I’ve been here since june 2007, and I’m 14. alot of peopel dont spend a ton of time here

BleedingBlade: they should check the join date of your acc

Boometh: I don’t bother with them and I don’t get them at some point.

hiperson134: I joined Sept 13 and I’m at level 24. That’s mostly because I like things that challenge me, but I’ve got nowhere to talk when it comes to being a lower level

Radec: just get the ones for games you actually really enjoy playing

skyfallsdown: To hiperson: sure, you may be good at games and not deal with bagdes, but you can’t tell that they are in anyway good since they haven’t proven themselves. I’m not saying you are all bad games, just they have no way in measuring your skill.

imbrium: shrugs well. people are bound to get excluded. it’s the nature of life.

hiperson134: Now I think the way that bbleedingblade said it may be the best way

Radec: i agree with imbrium in this instance. whining about being excluded doesn’t make anythign better. the game will come out eventually

hiperson134: People who joined long ago obvious have a great amount of experiance with the website, and thus should be able to get to the stuff that is blocked off for most people before others

BleedingBlade: ia lso think there should be a timer thing on your acc that tells you how much time you have been on the site. unnessicary but appreciated by some

hiperson134: *obviously

Radec: the only time being excluded from such an event where b****ing would be appropriate is if they happen to choose nothing but irritating beta testers who ruin the game with their inane ideas, but that rarely happens anyways

KakkoiBishounen: I saw paragraphs. What happened?

hiperson134: We’re debating on how Kongai testers should be chosen. I think it’s coming to a wrap-up soon

hiperson134: Now, for people that have been here a long time, but do not go for many badges, check out my suggestion here: http://www.kongregate.com/forums/1/topics/13?page=47

imbrium: not really agreeing or disagreeing with anything but— join date of the site isn’t much of an indicator of anything

imbrium: i join things and then don’t return for months or years.

skyfallsdown: No, it’s not

Boometh: I know, thats a good point!

KakkoiBishounen: Well, here’s my two cents.

hiperson134: basically, it’s an idea that rewards you for milestones including amount of time you’ve had your account, good behavior, etc.

imbrium: perhaps a thing like join date plus level

KakkoiBishounen: I was a beta tester for SF:DG (Starfighter: Disputed Galaxy) and frankly I wished that some people were more professional. I think that choosing should be based on who has the most time and who can be the most responsible for his or her actions.

KakkoiBishounen: That way, testing could be done efficiently and with minimal to no problems.

KakkoiBishounen: But that’s just my idea.

KakkoiBishounen: How they’d measure that is way beyond my thought process. I didn’t think that far just yet.

hiperson134: Okay, we’ve so far we’ve decided by level, by forum behavior, by chat behavior, and join date are all bad ideas, anything I missed?

Radec: well apparently there’s more to the conversation…

KakkoiBishounen: Well there was my idea on free time and responsibility.

KakkoiBishounen: I mean if nobody had free time, what can get done?

skyfallsdown: None of these are bad ideas, but almost impossible to implement.\

KakkoiBishounen: Meaning there’s a possibility, yes?

skyfallsdown: Of what? Measuring free time and responsibility? No.

hiperson134: With the suggestion I made, people can get points, just by being good, following chat guidlines, haveing your account active for a long period of time, things like that.

KakkoiBishounen: Well this would be the decision of the observations of individual mods, staff, etc.

KakkoiBishounen: *based on the observations.

hiperson134: Boometh has been going here for a year, and has he been awarded anyhting for faithfulness to the site? no. Is that fair? no

KakkoiBishounen: Well, how do you measure faith?

skyfallsdown: But how do you measure responsibility when your only encounter with a person is on a chat log?

hiperson134: I meant that he’s been coming to Kong for a long time and keeps coming back. That’s what I meant by faithfulness

Radec: neither have I, hiperson. the difference here is that I’m not whining about it. I’m sure there are more effective ways to select beta testers, but maybe you should bring that up to the mods and admins themselves

KakkoiBishounen: Based on their actions. Do they stop fights or do they egg them on? Do they promote cheating or do they try to discourage it.

skyfallsdown: Hiper: it’s a free gaming site, you shouldn’t have to expect anything besides games.

Radec: exactly.

skyfallsdown: Kakkoi, that’s not measuring it at all. What if, wheil stopping fights on this chat, they’re not picking up their kids for school? It’s impossible to measure responsibility at all without meeting them face to face.

KakkoiBishounen: That would be two different situations.

KakkoiBishounen: In this case the only concern is how they dedicate themselves to the beta. If they obvoiusly don’t have time then they wouldn’t care.

Radec: I swear… I think this is the first time in… forever that the Barrens Chat was almost intelligent…

BleedingBlade: ……………..

BleedingBlade: im takin that as a compliment

Radec: where’s the ranting? the idiotic combacks and insults?

hiperson134: This is the first time I’ve ever been to Barrens Chat.

(cuts to the last half of the debate)

skyfallsdown: Kakkoi, you’re talking about dedication, which is a whole other thing.

KakkoiBishounen: There’s going to be that what if, but the main concern is getting the project done as quickly and efficiently as possible. That way it would speed up the release. Now for that instance, would the experienced people who don’t have free time…

KakkoiBishounen: Be able to get it done or would the people who can invest time be able to?

KakkoiBishounen: Think of this analogy.

skyfallsdown: The best way to measure dedication than, if getting people who have tried to get as manny badges as possible — ie, their level.

BleedingBlade: depneds on the rate of the first

KakkoiBishounen: How do you choose?

skyfallsdown: if = is**

BleedingBlade: depends on the rate of the experienced ppl

hiperson134: by the experiance of the employees

KakkoiBishounen: The most loyal? What if they don’t have time? The most experienced? Same thing. Their knowledge would help, but can they invest the time? The best bet is to find the common ground among people who have time.

KakkoiBishounen: But as we’ve said, that’s a pretty hard find.

Radec: well, it would most likely be a good idea as well to include some experienced employees, if for nothing more than to help ensure that everyone else is still working towards the main goal.

skyfallsdown: Kakkoi, your analogy doesn’t make sense. If it’s a job, than your time dedication is dependent on your income, so you’d obvious put effort into it.

Radec: having alot of time doesn’t necessarily mean they’re inclined to do anything.

Radec: take interns for example.

skyfallsdown: They are looking for volunteers here – they are not getting paid, so they won’t be as willing to go for a long amount of time playing a game

KakkoiBishounen: My point is from the view of the person hiring.

KakkoiBishounen: Not the person working.

KakkoiBishounen: Now it’d be nice if we had some experienced people who do have time, as Radec mentioned. Yes, it would be useful to have someone there to make sure people know what they’re doing.

Boometh: Experienced in what?

Radec: hopefully no one from Barrens Chat when it’s like it is normally XD

Boometh: I am more of a “The Van” person my self..

KakkoiBishounen: I’m normally here nowadays, actually.

hiperson134: I’m technically from Sanctuary

Radec: figured.

Radec: no one from barrens usually ahs an attention span that good.

skyfallsdown: I’m gonna repeat Boom: experience in what?

Radec: Hopefully, experience in knowing what makes and breaks a game.

Radec: all those little inner workings and balance checks to ensure fun multiplayer.

KakkoiBishounen: Experience in what they’re doing, obviously. Yes I understand that the beta is a one time thing, but people who have been involved in other betas for other things would know what they are doing or at least what to test for.

skyfallsdown: But isn’t it just as important to have people who just play games, and give a response from that?

hiperson134: As I sit here and watch you guys throw points at each other, seemingly getting nowhere, I think, “How did the admins choose the way to do it?”

KakkoiBishounen: Everyone’s input is important, but preferably it’d be from those who know how to and can present it more maturely.

Radec: yeah, we aren’t getting anywhere.

KakkoiBishounen: I would like to know myself.

hiperson134: I find it obvious they went through the same thing we’re going through now. How did they make their final decision?

Radec: it’s not really my intention to get anywhere right now. I’m just blowin’ time.

KakkoiBishounen: Someone wanna post a “How did they do it” thread?

This is how this thread came to be. We then continued on by talking about how people from the ages of 14 to 18 were this calm and able to keep their heads in chat.

 
Flag Post
I'm no admin, but a while back I wrote a post that Emily Greer—co–founder of Kongregate.com—"seemed to like":http://www.kongregate.com/forums/1/topics/5899?page=3#posts-101521, and I also wrote a more recent, higher quality answer to the _"What criteria is being used to select beta testers?"_ question in my Kongai FAQ. You can view the FAQ answer "here":http://www.kongregate.com/forums/1/topics/3965?page=2#posts-112908. Also, I looked over the first post I mentioned (the one Emily seemed to like) and saw how terribly unpolished it was. It pains me to leave something so unpolished, so I went over it with a fresh editing pass since I thought you might like to read it. You can view the original, terribly unpolished post "here":http://www.kongregate.com/forums/1/topics/5899?page=3#posts-101463 (don't do it, it burns!), and you can view the freshly edited version (that smells like oven-baked cookies) after the line break below. Keep in mind that it was written *4 March* 2008. _— Bruce_ PS. Kudos on the copy/paste job you did with the chat discussion (above); I know how time consuming that can be. Darn the lack of a "select all" feature! PPS. If or when you get into the beta, don’t let _Milskidasith_ fool you with his innocent _“I only have 3 cards; let’s have a random match!”_ routine. He’ll take down your first card quicker then you can say, “why did I switch out when his obvious move was intercept?” and proceed to torture you with an Ashi stalemate and insane player vs player challenge. [/inside joke] -----------------------------------------------------

I still disagree with the tester choosing method and the card game name but I will shut my mouth because I might lose it, Matrix-style. :x

_“Tell me, Mr. Anderson… What good is a Kongai if you’re unable to… speak?”_ [insert Smith-style head motion] ;) Seriously though, the Kongregate staff only want to let in a limited amount of people so their server doesn’t go splat. As such, the quickest, easiest way to do this is to go by level. They’re also trying to be fair by adding those who contribute and if you include “plays a hell of a lot of Kongregate”, I’d say it’s pretty fair. From what I can extrapolate from the info they’ve given so far, they’re using this method because soon they’re going to be inviting a lot more people in. I think they’re being a bit hush-hush about this because (A) they have a PR team (ie. the devils subordinates… kidding!), (B) they don’t want to say they’ll do one thing then have to take a different course of action. Also, right now, I think they’re interested more in the raw data the testing produces then absolute quality of testing. Yes, this is a shame, since I’m sure a lot of us would contribute to the beta test quite nicely, but oh well. Perhaps we’re overestimating how much we can contribute? But before you go all “oh noes!” on us, take this into consideration: It took about a week to move from 50 to 100 testers. They have 20,000 people who signed up for the beta, apparently. Eventually they’ll have to let in all 2 million of the people who use Kongregate.com (I’m quoting figures from Emily here). It may be a little while longer, especially if you’re a low level, but you’ll definitely get to play soon enough. You’ll still be able to make the same suggestions and provide the same feedback when you do. The staff will probably be less receptive to feedback since they’ll be getting more of it and they won’t have such “trusted” members testing (for lack of a better word), but they will address anything really good or necessary, I’m sure. *To those who really, really want to get into the beta:* If that is your only goal (to speak nothing of this thing we call “life”), you need to level up. As far as I can see, that will increase your chances to get in. It’s unclear whether or not you’ll be let in after an “acceptance round” if you just made it to the acceptable level after it, but there’s probably a good chance you’ll get into the next round. May the levels be with you. ;)
 
Flag Post

Roflz i agree

Darn the lack of a “select all” feature!

 
Flag Post

Hi guys,

I agree that our selection method leaves lots of great regular Kongregate users out of the beta because they haven’t focused their energy on earning points and leveling up. I didn’t feel that pure seniority of who got here first was a good way to choose, since someone might have joined a year ago but only visits Kong once or twice a month would be favored over someone who joined six months ago and is here every day. Anything that involves a qualitative judgment of behavior/contribution (whether chat, comments, or forum posts) is just not feasible — we simply don’t have time to look through what would be hundreds of thousands (or even millions) of messages to judge them and decide who’s going to be in the next batch of beta testers.

Level is definitely an imperfect proxy for activity on Kongregate in that lots of highly engaged users are relatively low level. But high level people all are pretty active on Kongregate, so it’s the best proxy we have.

 
Flag Post

Thank you to all. Yes I’m in it, but we started throwing our ideas around, then got into a debate about it. Since we ruled out everything, we decided it’d be best to see what the forums had to say about it. So thanks for explaining everything quickly and clearly.

P.S. Yeah, there really, REALLY needs to be a “select all” feature. My hand was hurting after that.
P.P.S.The reason part of it is bold is because there was an asterik that I, unknowingly, left in, thus causing the bold until the next asterik.
P.P.P.S. Even though I’m n already, it doesn’t mean I can’t have a voice for the people who aren’t.