Summary:Talk:Task force/Wikipedia Quality/Benefits of having "trusted / high quality" user recognition

Starter thread / A couple of conversation starters

Philippe opened the taskforce asking: "What have we agreed on in terms of quality? Where is the community in terms of the quality discussion. What do we NOT agree on? What have we not discussed about quality, as a community? What sort of information would be useful, in terms of helping us think this through?"

Discussion initially moved to the Five Pillars, newcomer guidance, and the nature of reliable sourcing, before branching out into other threads. These points are summarized in their own threads.

Thoughts to date

Starting point

FT2 followed up an earlier post which stated that "creating an official usergroup of trusted (or good quality, or senior) content editor might be the single biggest step towards helping here that we can make":

Suppose a community process existed for recognizing editors who are trusted in their content work. This means (eg) they consistently work well on content, edit neutrally, don't edit war, collaborate instead of fillibuster, make generally good edits, cite well, improve content, debate issues instead of attacking personalities, and have a good general ability. No "extra rights" but...

1. They have an investment in extra standing (reputation-wise), which is valuable and a source of reward. It will tend to be guarded and incentivize people.
2. We can patrol articles and edit wars easier by highlighting these users in the article history (valuable information).
3. Not all editors will want or seek adminship, this is a parallel way to recognize those who edit content. If it can be made something "everyone should aim for" we might have many thousands of editors flagged this way, a strong impetus for a quality based community.
4. In an entrenched area or difficult edit war, Arbcom or the community can now say "any trusted content editor may edit the article. Others = talk page only". Many many trusted content editors, so no real POV or "too narrow editors" issue here. Anyone who wants to edit the mainspace and can get community agreement they edit content well, may join in (others only on the talk page). Instant stability, good decisions, consensus, and quality, on problem and edit war articles! No harm done, no bias added, articles still edited by a wide pool that anyone can join.
5. (added later) While wiki isn't a "profession" this provides a way users can stretch their skills and a means of self evaluation and development as editors. A "recognized wiki-editor" qualification would also be good for ethos.

If we only have 2 - 4 recommendations, this might be one with profound scope to help in many ways - stability, quality, entrenched edit wars, experienced editor enjoyment, and incentive to gain good content editing skills and edit well.

General discussion

Randomran felt it was a good idea if de-politicized and factional gaming could be avoided. Multiple FAs and a good track record of civility and consensus building would be good criteria.

Piotrus felt it made some good points but expressed concern that a user who was reliable in one area might have trouble in another, or that users editing in divisive areas would be unable to gain such standing due to mud slinging by other editors and cliques. He suggested that once a large number of trusted/senior editors were appointed, the task could be left to these users (who would presumably be less prone to mud-slinging): "In other words - I can trust the quality editors to make quality decisions, but I am increasingly disappointed with flaming and mistrust-sawing comments from "the peanut gallery" in various discussions I see".

FT2 responded: 1/ it would also need an effective and hard-to-game removal process; 2/ "appropriate recognition and self management where they have COI or other problems/strong views...[the] idea implies good editorship on articles the user cares about, too"; 3/ pure self selection can encourage divisiveness, where we're aiming for mass involvement and good standards. A two-step process might resolve the issue.

Bhneihouse liked the "overall feel" and wanted to further refine it.

Non-anonymous experts

Piotrus noted that giving non-anonymous experts extra standing could be another (or complimentary) approach. FT2 noted that experts don't always make desirable editors and have the same learning curve as any other person on how to edit appropriately:

"This is about recognizing people who know how to edit and are accepted as good at doing so. Nurture those, and [everyone benefits] including for experts":

"I'd tell the expert that (like joining any new project) they need to learn how to edit Wikipedia, which will be different from how they edit their own papers. But they can edit freely, and (since they are bright and used to scientific collaboration) they'll surely be recognized as a trusted content editor for our purposes nice and quickly, if they take a few minutes to understand how we work here. In fact I'd make that part of the "New user wizard" ("Do you have formal credentials in any field you plan to edit?" + guidance)."

Bhneihouse suggested a list of known credentials may be useful for screening.

Sue Gardner's post

Sue Gardner (WMF) stated that this would be a useful and good idea, for both new editors (to identify trusted editors) and for not-new editors. She expressed a strong preference that such recognition should be assessed automatically rather than manually to address scaling and "popularity contests", and save time.

FT2 felt automation could help and could cut down the cases needing review, but that the benefits flowed from the idea that "some users can generally be trusted to do right, in article and article discussions, of any kind. Those are the users we want in this pool, because once identified, they provide a large population of experienced content writers not needing much guidance or checking, and capable of being given heavy disputes to put into good editorial order... The benefits here flow from their acknowledged trust to do right (broadly speaking) on any content matter, up to and including self-management of bias, interaction style [etc]". He felt this could not be assessed without human involvement.

Bhneihouse agreed about non-automation, and agrered with Piotrus and Sue on "popularity contests". She noted that "Perhaps a pivotal point in quality control is how Wikipedia "approves" and trusts editors? Perhaps another pivotal point is the actual "structure" of this process?"

Sue Gardner commented in response that this would be

"a marker/label for people who are particularly trusted to have good judgment. Probably these would be people who've been around for a while, and understand the policies well, who are reasonable and thoughtful. I think that's a great idea. I think new editors would really appreciate being able to tell at-a-glance if an editor they didn't know was someone they should trust and listen to..."

She also noted the distinction would greatly help newcomers, who could understand the editing better and seek reputable advice, as well as helping those who deal with editorial behaviour and disputes. It would extend the usual network of trust, which usually doesn't scale well. She did not like the term "trusted editor" though (implies others are not trusted) and suggested "senior editor".

She also worried that the system for gaining the standard would be gameable or (per Piotrus) lock out editors in controversial areas. She felt the decision should be made by "thoughtful... experienced people" and editing criteria, not by simple voting, and perhaps a "trusted team" to identify such users. She concluded it was a good idea she would like to see work.

How would such users be selected?

Piotrus suggested a user should show quality content (1 FA/5 GAs/50 DYKs), and be trusted only in areas he has a reputation (WikiProject based), and this would be hard to game or disrupt. FT2 stated that writing certain content would not necessarily correlate to trusting their editorial approach generally, would not show neutrality or good conduct on their "pet subjects" or appropriate talk page approaches. It could at best be evidence.

Randomran agreed some processes are "vulnerable to whim and personal opinion" and saw FA as one of the few that is hard to game as an indication that a user understands quality writing, adding later that "The human component can be there as a screen, as a veto, but people should really be judged by accomplishments that the widest number of Wikipedians cannot deny".

FT2 noted that good content writers contain a "fair proportion of users who couldn't meet the kind of role we're talking about" and suggested two possible approaches:

1. Specific criteria (user presents a portfolio meeting set criteria; objections must meet defined and evidenced criteria too). It's prescriptive and somewhat gameable but much harder to game. (details)
2. a 2 stage highly automated process whereby users only needed 50% community approval (hard to game and mass participation) and a 75% "trusted user" approval (high standards, veto, allows existing senior editors to see any community views and concerns), both parts held via SecurePoll for efficiency and to prevent "popularity contests". (details, also discussed below)

Woodwalker was "not against formalizing the status of quality users" but was concerned that expertise in one area might not equate to expertise in another. FT2 highlighted that a generally well reputed and skilled editor could be more of an asset in the contexts under discussion due to issues of foibles and bias, collaboration and mass editing skills and the non-expert knowing how to help others make the best of their input (including experts). "If we're assessing what kind of editor can be broadly trusted to work on all kinds of difficult articles unsupervised... in a proper way... [then these qualities] will get you that person... a PhD won't". (link). Woodwalker agreed but suggested not calling them experts if they were not, and preferred a criteria-based approach. Piotrus suggested looking at their activity record, and only considering concerns from the last 6 months (the criteria had included a 9 month cutoff).

FT2 noted a panel would have the same issues of gaming and politics, and that we should trust the wider community; it is easier since all issues (including most alliances and gaming) would be visible and public. Rather than trying to be perfect on selection, create a method that is 95% valid but "slightly able to be gamed", along with a "clear and standardized removal process" and "some kind of scrutineers panel [for] cases claimed to be grossly affected by bias and canvassing, or where the results don't reflect appropriately on the user". Woodwalker agreed that community not panel was appropriate, and Piotrus agreed that addressing "popularity contests" was very important indeed. Philippe endorsed designing for 90% ("good enough"), and Bhneihouse stated the two stage process sounded good and process now needed adding to handle the exceptions.

Further discussion on the 2 stage proposal

FT2 described the latter as "a hybrid of enwiki Mediation Committee's nomination method (filter[s] good quality users and operates historically with no drama whatsoever) and a modification of the SecurePoll tool already in place... a bit more involved than 100% automation, but it is simple (once set up) and keeps almost all the benefits of automation, all the benefits of user involvement, and very little of the drawbacks of either, when merged." Its design goals were stated as "automation, low gameability, simplicity of experience to users, very low scope for politicking/dramatizing/popularity contests, and low time needed by participants".

Some discussion took place on its talk page:

Randomran stated he had concerns it can be gamed and a threshold is needed to weed out bad applicants and prevent fillibustering.

FT2 noted that this would not help a clique, because all the community can do is 1/ stop someone getting 50% (which is still an easy level for a decent editor in such circumstances) and 2/ raise concerns, and filibustering doesn't work because it isn't a "debate". Community concerns will be publicly visible after the 1st stage and inform the second, but the 2nd stage is not easily influenced by partisan cliques because its constituency is editors already considered to be high quality and of good judgement.

Piotrus agreed this "sounds plausible and indeed should not be very easy to game".

There was some discussion how to bootstrap the process (the users who will operate the 2nd stage at the start) - Randomran suggested using 2+ FA writers. FT2 stated it was a once-off issue and needed the highest quality of content editors to get a good start; he suggested using the subset of FA/GA writers who have also passed RFA (the latter attesting to other areas of trust, awareness and judgement). Since RFA often requires content work this is a substantial pool of FA/GA writers and "probably enough" to start it off.

Randomran agreed that 2+ FA writers who had also passed RFA would be a strong pool, but (being devils advocate) was concerned that it might be taken as a cabal ("I'm not sure this is a bad thing in practice. But in principle, a lot of people just hate cabals"); that it might still just reflect popularity; and might still exclude editors in controversial areas. He felt it could perhaps be strengthened against these issues.