Summary:Talk:Task force/Improve Wikipedia's Quality Task Force/Communal enabling of quality

Starting point

FT2 observed that '"originally no thought was given to basics like practicalities of 'consensus' and decision-making, the very wide spectrum of editors' views, etc. Quality improvement recommendations may have limited effect, if communities don't simultaneously have better ways to handle large scale decision-making and differences of opinion".

Specifically, if communities "[lacked] good ways to make large scale decisions and [handle] large scale divisions of opinion... the mechanism to choose [useful options] doesn't exist in the first place". On enwiki this affects (for example) disagreements about reliability of sources, balance of mainstream views, editor processes, deployment of new proposed tools, proposals for ways to handle conflict and disruption better. quality related matters don't stall under such problems as 'too difficult to get a clear decision' or 'vocal fringe'. Other difficult decisions (around content, processes, self-governance, improvement, etc) discourage many people from even trying to propose improvements, or can waste a lot of volunteer energy.

A possible solution was noted, to ensure that communities have theior attention drawn to it, so that this will be taken more seriously as a quality factor:

"[We can tell communities, i]f you focus on it, and figure a way to improve consensus seeking on big decisions in your wiki, then you can do quality improvement easier yourselves. If that's not a problem holding back your wiki now, then judging by bigger projects it will be in future."

Bhneihouse's view

Bhneihouse asked how to poll contributors and make them stakeholders in the process; "what are the decision rules? ". She noted that the online community is generally "more independent and less rule oriented", top-down rules don't work, and "no matter how hard you work, many will still resist change, even when they can fully see the benefit". There are two main approaches - mandate rules (and those who can work with them will turn up), or, involve the community in creating value based rules that take into account their preferences and current methodologies. She assumed there would be a "play book" to guide handling if someone "doesn't play nice".

Piotrus' view

Piotrus thought the point valid but saw the main problems as being in different areas. (Piotrus' essay "Why good users leave and why civility is key"). He saw the core problem as being:

"[G]ood, productive, experienced users leave due to attacks and stressful incidents; quality relates to number of editors; easily reachable users are mostly reached (geeks who think it's fun); other groups are harder to reach; as a result we are burning through experienced users faster than we are adding to them and quality will suffer [...] Conclusion: we need to understand why editors are leaving, address the issues causing them to leave, reach out to those who left and try to bring them back, and support plans to tap new pools of editors".

Randomran's view (from the community health taskforce)

Quality is highly correlated with the number of editors working on an article, and that the loss of editors risks making even basic maintenance an issue (per Ortega (file)). He felt there is "a real systemic problem with how disputes are settled" and notes Ortega's concern about "consensus-building falling off and more people pushing their personal opinion":

"And yes, those two person debates escalate into WikiProject discussions or RFCs or policy discussions, which attract entire cartels of opinion pushers. And sadly, many of those opinion pushers are not interested in consensus building. They've actually realized that they can accomplish more by filibustering debate than by compromising. So you keep on having the same debate over and over, hoping to make progress, only to see it fall apart because the different extremes unified in their desire to keep the debate going forever. So not only does quality never get a decisive resolution... you actually lose good contributors who just got sick of trying to mediate between the different fanatics."

Woodwalker's view

We don't have a sound definition of "quality" (Ortega uses Featured Articles which does not really capture quality for a project, nor is it valid on many projects). He later amended this to note "[The data] only shows featured articles have been edited by many users, not that content with high quality was generally edited by many users" (ie this is correlation, not necessarily causation).
We assume number of editors equates to growth decline in quality. Quality editors matter, but many editors are not quality editors and their contributions have much less impact on quality, and some users actually detract from quality.
Editors have a lifetime and will leave. They tend as a group to be " not so good at communicating with others", an issue fostered by anonymity and exploited by other projects such as Citizendium.
Quality editors may be better communicators but equally may burn out sooner (due to lower tolerance for poor communication).
Projects tend to foster "maintenance" editors and respect these (essential for adminship etc too), but users adding quality have it less easy, and may have less influence and less final say in decisions. Another negative factor.
Numbers of quality and "maintenance" editors can grow if help functions are improved (eg via wizards as discussed)
Quality users are often found in isolation at small projects, surrounded by maintenance users and vandals. When they leave, the quality they added is at risk of being destroyed again.

Woodwalker therefore sees the key questions as:

"How can we ensure quality users get more respect and/or influence in communities (and in that way prolong their life time)?"
"How can we get quality users at local projects out of their isolation?"
"[And] we may want to explore the questions how to decrease the amount of vandal users and increase the amount of maintenance users too, since this increases quality as well."

General discussion followed:

The best content is often written by one (or few) users and is easily eroded

Piotrus has written about 20 Featured Articles. On most he was the sole editor (or sole substantive editor), and collaborative FAs and GAs ("good articles") are the tiny exception. Randomran concurs but notes sometimes small "tag teams" of a few editors do form around an article.

Yaroslav Blanter states after a certain quality is reached, further edits mostly reduce quality. Protecting the quality that is achieved is a major issue.

Setting and enforcing fair expectations matters, especially in retaining capable users

Randomran notes that core editors on enwiki suddenly vanish, suggesting they became fed up (described by Piotrus as "That's it, I'm outta here!") rather than loosing time and interest. Bhneihouse concurs too, that "discord" is a major cause of loss, and that experts are likely to have zero tolerance for unnecessary conflict, adding: "I think it likely that is a common thread for the exact people we want to attract and retain as contributors and editors. So the whole supermajority/consensus issue is probably more important than I originally thought". FT2 concurred noting it applies to "most mature and capable users" and warning that many who have zero tolerance may consider themselves worthy of exception and not accept others having zero tolerance of their own behaviour: [As] said elsewhere, even experts need induction/newcomer handling, "This is how we work, these are the expectations..." Bhneihouse agreed aboutletting people know the expectations up front: "Let people know up front what the expectations are, let them know that Wikipedia takes protecting their rights and contributions seriously, and we will attract and keep the mature and capable users".

How to help "quality users" and "quality content"

Piotrus provided his analysis.

10% of Wikipedia editors produce 90% of content. Overall, the number of editors does matter, but the number of high end contributors of course matters even more.
Quality users don't contribute to wiki-projects because they want to change policy, but because they want to show the world their knowledge. What all wiki-users crave for is some sort of 'applause'. This is why positive reinforcement is important, and negative reinforcement is so bad. He feels this is the major cause for editors vanishing.
Maintenance editors are crucial for stability, content creators are crucial for growth. Creating new content is usually more contentious than retaining existing stable content.
Quality users could get more respect and influence by giving "special powers" to non-anonymous editors. This would also help them not to be overruled by anonymous but experienced POV pushers and could help improve Wikipedia's standing. Such users should not be allowed to be disruptive, but should be much better protected from (successful) harassment and given positive reinforcement and support.
(Blatant) vandals are not an issue, POV pushers are: "They are harder to identify as disruptive, yet they are the ones primarily responsible for creating bad atmosphere and driving others out of this project. In my experience, it is those type of editors that are primarily responsible for driving good editors away, and I am not seeing any signs the Wikipedia system is able to deal with this issue. In fact, the danger to quality in the future is not only the loss of contributors - it is the possibly shifting proportion between good content editors and POV pushers"

Woodwalker agrees that "rewarding or at least protecting non-anonymous users is a very good idea, as long as constructive critique remains possible ", although noting that anonymous users can be quality users too. He notes "it is easier to 200 times revert a vandal than it is to remove POV from one long article" and that "the difference in the amount of 'award' the system gives for maintenance (enough) and editing content (too little) is a problem".

Woodwalker adds further that erosion (quality work gradually being degraded by well meaning maintenance edits is a problem; good content needs active management. Smaller communities suffer disproportionately from this problem and small project quality content is at higher risk of erosion. It's also a concern that if such users stop editing, the smaller non-English project often lacks anyone to watch quality on its best articles, and they degrade. Although degradation at smaller projects is slower, it is ultimately far more destructive of quality too. Outreach to quality editors (especially on smaller projects) and helping them to work on larger projects too, may be beneficial and could extend their editing enjoyment and lifetime.

FT2 felt that having "a way to formally recognize "trusted [or senior] content editors" is actually the one biggest change that could help" (on these and other related issues), since the taskforce would be producing only 2 - 4 recommendations. Reasons and discussion were in another thread and met broad positive support.

Yaroslav Blanter stated that interwiki mobility (translation and dissemination) of good and featured articles would be an immediate and valuable aide to quality. Users willing to translate or improve wording are easy to find but may not know good content exists elsewhere.

Consensus and community

Bhneihouse felt consensus seeking was a "hard sell", and focus should be on "understanding what users do, how they use this creation we call Wikipedia, and what they are really looking to get out of it, both the experience and the information... If you [reach] for a true consensus... you will derail the project in its entirety". She felt a supermajority was often enough and that one underlying question is "what is Wikipedia growing up to be?"

Slrubenstein felt that "the anarchic nature of the Wikipedia community has both advantages and disadvantages" and should not be abandoned. He concurs there is a difference between topics where clear mainstream consensus exists and where it does not, but feels that consensus processes are not the major problem. Policies about "disruptive editors/editing" can help address those, though they can take days or weeks ("it really isn't a long time").

Randomran (providing input from the community health taskforce) feels by contrast consensus seeking is a core problem, and is interested that NPOV/V/NOR are seen as viable way to settle many disputes. He sees many users not understanding or caring about them, possibly due to learning curve issues; "Maybe one really effective thing that the board could do is give backing to the core content policies? It could actually help".

BarryN (Bridgespan Group) feels this is a valuable thread, and that "getting consensus for change in a well-established community is the hardest part... it is rare that there is an emergent consensus from the diffuse community that change is needed [yet] a top-down change mandate rarely works either unless there is a huge crisis". He sees consensus on the "vision" but "a growing consensus that the way the community is engaging around article quality is becoming an obstacle both in terms of the ability to reach high quality consensus on articles (too much 'last person standing wins') and on acceptance/cultivation of new [capable/expert] editors". He feels change is more likely to come from many small changes rather than "[one] big bang shift in the culture".