All Gens Suspect Tests / Tier Finalisation

[Disaster Area]

Before we can restart our tiering project, one of the things we need to do is combine all of our current up-to-date information on tiering into one to two stickied threads on the topic. (Another thing is to confirm that the banworthiness guidelines are adequate) Besides that, though, we also need to address suspect tests & tier finalisation. Notwithstanding when to vote on whether there should be any suspects / when to carry out a suspect test, I think we also need some sort of standard to make the vote legitimate. To do so, we need to ensure that:
- the voter pool is significant in size (we can / should debate how large, but I think probably at least 10 people. This can maybe vary in generation and be lowered when doing lower tiers?)
- the voters in the voter pool have significant, relevant experience (we need to decide what consists of significant, relevant experience, and how, using that knowledge, we can determine or restrict the voter pool)

(Note: by saying a tier is "final" it doesn't mean we can never change it, but it means that there are no obvious suspects that haven't been already tested, and so the tier can then have a season)

I think that this is an important step in ensuring our tiers are more legitimate, rather than the limited focus of a small group of people.

Furthermore, I think that the Kyogre ban in ADV 1U and all of the Pokemon bans besides Mega Ray in ORAS 1U should be initially repealed and the process restarted (in the case of ORAS 1U, once the current season is finished). I will maintain ORAS 1U as a separate metagame in the meantime (I'm unsure what name I want to give it yet, if u have any ideas holla)

 

[Ortheore]

I think I've already expressed my opinion in previous threads but I'll restate it here anyway.

I'm opposed to the concept of "tier finalisation" as I think none of the tiers are so broken as to be unplayable (with the exception of any MRay meta), so we ought to just jump straight in. Adding a testing phase only creates another obstacle to adding the corresponding season, which offers not only a delay, but also a chance for interest to peter out, as any testing phase is likely to draw a relatively small crowd, exactly the opposite of what's intended. Furthermore, it's questionable whether it'd even get things right- significant numbers of players would be expected to make decisions with only a dozen or so games of experience under their belt, which is ridiculous, especially for something that's "final". Lastly, I don't think it's necessary to have the tiers "finalised" before hosting seasons, as players are either accustomed to playing tiers that can potentially change, or are at least able to adapt.

I also think that 1U should be treated differently to lower tiers, as although I oppose "finalisation", 1U's ruleset is nonetheless far more enduring than the more ephemeral lower tiers, so bans should be treated with much greater gravity.

==============

Questions surrounding suspect tests that I can think of (note that all my answers are applicable to 1U, lower tiers I will mention later):

• How do we know when players want one or a tier needs it? Probably we would need to have multiple players in charge of monitoring discussion (more than one so that bias doesn't cause issues). I'm gonna say the season host and at least one player who's at a high level
  
• What should be the minimum timeframe between suspects? First of all, time passed should only count if we're hosting tours for that tier- having a dormant tier for 6 months is not good enough. Ideally I would like to see the tier being played for at least a year before the first suspect, assuming that significant portions of the playerbase have no prior experience in the tier. The interval for subsequent suspects can be shortened to 6 months, as the playerbase would be assumed to be adapting existing knowledge rather than building from scratch. Note that this isn't a hard restriction, suspects can be called early, but would have much higher ban thresholds. And lastly, I'm aware that these intervals are much longer than most people are accustomed to, so even though I think that that's really the way we ought to go, I am a bit flexible on these values.
  
• How many players do we need for a suspect? Couple notable metrics here- total number of players, number of eligible voters and number of votes cast. I think that the total number of players and the number of votes cast should both be considered here. Votes cast is obviously a matter of sample size, but I think the size of the playerbase should also be considered, even if a significant portion of them don't get a vote due to eligibility criteria. The reason is that it can be assumed that with more players comes a greater overall level of competition, whereas a smaller group, even if they're elite, could potentially miss things or be perceived as being less legitimate due to the size of their group. As far as actual values go, if we don't have enough players for a typical MT (i.e. 16), then that should be a red flag. In terms of actual votes cast, I'm not too fussed, nine is a minimum imo, but if we set the bar higher that's also acceptable
  
• What eligibility criteria should we implement? Hmm, we have plenty of options here. We can provide options for playing lots of matches or winning lots of matches, performing well in tours, and these criteria can even coexist imo. I'm not settled on a best option (or set of options), but I just want to throw it out there that a benchmark to consider is having players' scores in the past season or two >= the score of someone who consistently makes it out of the first round. Doesn't work so well in my tour format though.
  
• What ban thresholds should be used? I think for regular suspects the threshold for a ban should be 60%. 50% is too low imo, as it becomes too easy to implement bans when they're theoretically supposed to be only done if absolutely necessary and furthermore, it might only take a small demographic shift to end up with a majority of players disagreeing with a given ban, a scenario I think is unacceptable. 60% provides a nice buffer against this. If a suspect is called early (based on what I said in the timeframe section), then I'm going to propose 75% or 80% as the threshold (maybe 75%?)

Lower tiers are a very different matter to what I've described above. They're much less enduring and there's more of a focus on getting things done. I think it's perfectly acceptable for suspects to start after say, six weeks to two months of play, and although I'd prefer 60% for a ban, 50% is also acceptable. As for deciding tier boundaries, I propose we scrap the tier council thing, since its only real role was to act as a tiebreaker. Instead, we can simply have an arbitrary policy of deciding ties, where if something gets 50% it either drops or gets included in the tier. I personally prefer having it drop because fewer bans, but I don't really mind either way. And in terms of eligibility, simply requiring X games played should be sufficient