Communal enabling of quality

Starting point

FT2 observed that '"originally no thought was given to basics like practicalities of 'consensus' and decision-making, the very wide spectrum of editors' views, etc. Quality improvement recommendations may have limited effect, if communities don't simultaneously have better ways to handle large scale decision-making and differences of opinion".

Specifically, if communities "[lacked] good ways to make large scale decisions and [handle] large scale divisions of opinion... the mechanism to choose [useful options] doesn't exist in the first place". On enwiki this affects (for example) disagreements about reliability of sources, balance of mainstream views, editor processes, deployment of new proposed tools, proposals for ways to handle conflict and disruption better. quality related matters don't stall under such problems as 'too difficult to get a clear decision' or 'vocal fringe'. Other difficult decisions (around content, processes, self-governance, improvement, etc) discourage many people from even trying to propose improvements, or can waste a lot of volunteer energy.

A possible solution was noted, to ensure that communities have theior attention drawn to it, so that this will be taken more seriously as a quality factor:

"[We can tell communities, i]f you focus on it, and figure a way to improve consensus seeking on big decisions in your wiki, then you can do quality improvement easier yourselves. If that's not a problem holding back your wiki now, then judging by bigger projects it will be in future."

Bhneihouse's view

Bhneihouse asked how to poll contributors and make them stakeholders in the process; "what are the decision rules? ". She noted that the online community is generally "more independent and less rule oriented", top-down rules don't work, and "no matter how hard you work, many will still resist change, even when they can fully see the benefit". There are two main approaches - mandate rules (and those who can work with them will turn up), or, involve the community in creating value based rules that take into account their preferences and current methodologies. She assumed there would be a "play book" to guide handling if someone "doesn't play nice".

Piotrus' view

Piotrus thought the point valid but saw the main problems as being in different areas. (Piotrus' essay "Why good users leave and why civility is key"). He saw the core problem as being:

"[G]ood, productive, experienced users leave due to attacks and stressful incidents; quality relates to number of editors; easily reachable users are mostly reached (geeks who think it's fun); other groups are harder to reach; as a result we are burning through experienced users faster than we are adding to them and quality will suffer [...] Conclusion: we need to understand why editors are leaving, address the issues causing them to leave, reach out to those who left and try to bring them back, and support plans to tap new pools of editors".

Randomran's view (from the community health taskforce)

Quality is highly correlated with the number of editors working on an article, and that the loss of editors risks making even basic maintenance an issue (per Ortega (file)). He felt there is "a real systemic problem with how disputes are settled" and notes Ortega's concern about "consensus-building falling off and more people pushing their personal opinion":

"And yes, those two person debates escalate into WikiProject discussions or RFCs or policy discussions, which attract entire cartels of opinion pushers. And sadly, many of those opinion pushers are not interested in consensus building. They've actually realized that they can accomplish more by filibustering debate than by compromising. So you keep on having the same debate over and over, hoping to make progress, only to see it fall apart because the different extremes unified in their desire to keep the debate going forever. So not only does quality never get a decisive resolution... you actually lose good contributors who just got sick of trying to mediate between the different fanatics."

Woodwalker's view

1. We don't have a sound definition of "quality" (Ortega uses Featured Articles which does not really capture quality for a project, nor is it valid on many projects). He later amended this to note "[The data] only shows featured articles have been edited by many users, not that content with high quality was generally edited by many users" (ie this is correlation, not necessarily causation).
2. We assume number of editors equates to growth decline in quality. Quality editors matter, but many editors are not quality editors and their contributions have much less impact on quality, and some users actually detract from quality.
3. Editors have a lifetime and will leave. They tend as a group to be " not so good at communicating with others", an issue fostered by anonymity and exploited by other projects such as Citizendium.
4. Quality editors may be better communicators but equally may burn out sooner (due to lower tolerance for poor communication).
5. Projects tend to foster "maintenance" editors and respect these (essential for adminship etc too), but users adding quality have it less easy, and may have less influence and less final say in decisions. Another negative factor.
6. Numbers of quality and "maintenance" editors can grow if help functions are improved (eg via wizards as discussed)
7. Quality users are often found in isolation at small projects, surrounded by maintenance users and vandals. When they leave, the quality they added is at risk of being destroyed again.

Woodwalker therefore sees the key questions as:

1. "How can we ensure quality users get more respect and/or influence in communities (and in that way prolong their life time)?"
2. "How can we get quality users at local projects out of their isolation?"
3. "[And] we may want to explore the questions how to decrease the amount of vandal users and increase the amount of maintenance users too, since this increases quality as well."

General discussion followed:

The best content is often written by one (or few) users and is easily eroded

Piotrus has written about 20 Featured Articles. On most he was the sole editor (or sole substantive editor), and collaborative FAs and GAs ("good articles") are the tiny exception. Randomran concurs but notes sometimes small "tag teams" of a few editors do form around an article.

Yaroslav Blanter states after a certain quality is reached, further edits mostly reduce quality. Protecting the quality that is achieved is a major issue.

Setting and enforcing fair expectations matters, especially in retaining capable users

Randomran notes that core editors on enwiki suddenly vanish, suggesting they became fed up (described by Piotrus as "That's it, I'm outta here!") rather than loosing time and interest. Bhneihouse concurs too, that "discord" is a major cause of loss, and that experts are likely to have zero tolerance for unnecessary conflict, adding: "I think it likely that is a common thread for the exact people we want to attract and retain as contributors and editors. So the whole supermajority/consensus issue is probably more important than I originally thought". FT2 concurred noting it applies to "most mature and capable users" and warning that many who have zero tolerance may consider themselves worthy of exception and not accept others having zero tolerance of their own behaviour: [As] said elsewhere, even experts need induction/newcomer handling, "This is how we work, these are the expectations..." Bhneihouse agreed aboutletting people know the expectations up front: "Let people know up front what the expectations are, let them know that Wikipedia takes protecting their rights and contributions seriously, and we will attract and keep the mature and capable users".

How to help "quality users" and "quality content"

Piotrus provided his analysis.

1. 10% of Wikipedia editors produce 90% of content. Overall, the number of editors does matter, but the number of high end contributors of course matters even more.
2. Quality users don't contribute to wiki-projects because they want to change policy, but because they want to show the world their knowledge. What all wiki-users crave for is some sort of 'applause'. This is why positive reinforcement is important, and negative reinforcement is so bad. He feels this is the major cause for editors vanishing.
3. Maintenance editors are crucial for stability, content creators are crucial for growth. Creating new content is usually more contentious than retaining existing stable content.
4. Quality users could get more respect and influence by giving "special powers" to non-anonymous editors. This would also help them not to be overruled by anonymous but experienced POV pushers and could help improve Wikipedia's standing. Such users should not be allowed to be disruptive, but should be much better protected from (successful) harassment and given positive reinforcement and support.
5. (Blatant) vandals are not an issue, POV pushers are: "They are harder to identify as disruptive, yet they are the ones primarily responsible for creating bad atmosphere and driving others out of this project. In my experience, it is those type of editors that are primarily responsible for driving good editors away, and I am not seeing any signs the Wikipedia system is able to deal with this issue. In fact, the danger to quality in the future is not only the loss of contributors - it is the possibly shifting proportion between good content editors and POV pushers"

Woodwalker agrees that "rewarding or at least protecting non-anonymous users is a very good idea, as long as constructive critique remains possible ", although noting that anonymous users can be quality users too. He notes "it is easier to 200 times revert a vandal than it is to remove POV from one long article" and that "the difference in the amount of 'award' the system gives for maintenance (enough) and editing content (too little) is a problem".

Woodwalker adds further that erosion (quality work gradually being degraded by well meaning maintenance edits is a problem; good content needs active management. Smaller communities suffer disproportionately from this problem and small project quality content is at higher risk of erosion. It's also a concern that if such users stop editing, the smaller non-English project often lacks anyone to watch quality on its best articles, and they degrade. Although degradation at smaller projects is slower, it is ultimately far more destructive of quality too. Outreach to quality editors (especially on smaller projects) and helping them to work on larger projects too, may be beneficial and could extend their editing enjoyment and lifetime.

FT2 felt that having "a way to formally recognize "trusted [or senior] content editors" is actually the one biggest change that could help" (on these and other related issues), since the taskforce would be producing only 2 -  4 recommendations. Reasons and discussion were in another thread and met broad positive support.

Yaroslav Blanter stated that interwiki mobility (translation and dissemination) of good and featured articles would be an immediate and valuable aide to quality. Users willing to translate or improve wording are easy to find but may not know good content exists elsewhere.

Consensus and community

Bhneihouse felt consensus seeking was a "hard sell", and focus should be on "understanding what users do, how they use this creation we call Wikipedia, and what they are really looking to get out of it, both the experience and the information... If you [reach] for a true consensus... you will derail the project in its entirety". She felt a supermajority was often enough and that one underlying question is "what is Wikipedia growing up to be?"

Slrubenstein felt that "the anarchic nature of the Wikipedia community has both advantages and disadvantages" and should not be abandoned. He concurs there is a difference between topics where clear mainstream consensus exists and where it does not, but feels that consensus processes are not the major problem. Policies about "disruptive editors/editing" can help address those, though they can take days or weeks ("it really isn't a long time").

Randomran (providing input from the community health taskforce) feels by contrast consensus seeking is a core problem, and is interested that NPOV/V/NOR are seen as viable way to settle many disputes. He sees many users not understanding or caring about them, possibly due to learning curve issues; "Maybe one really effective thing that the board could do is give backing to the core content policies? It could actually help".

BarryN (Bridgespan Group) feels this is a valuable thread, and that "getting consensus for change in a well-established community is the hardest part... it is rare that there is an emergent consensus from the diffuse community that change is needed [yet] a top-down change mandate rarely works either unless there is a huge crisis". He sees consensus on the "vision" but "a growing consensus that the way the community is engaging around article quality is becoming an obstacle both in terms of the ability to reach high quality consensus on articles (too much 'last person standing wins') and on acceptance/cultivation of new [capable/expert] editors". He feels change is more likely to come from many small changes rather than "[one] big bang shift in the culture".

Without splitting the discussion, there's an unspoken aspect we might also need to consider if no other group is doing so.

Originally in 2001, no thought was given to basics like practicalities of "consensus" and decision-making, the very wide spectrum of editors' views, etc.

As well as recommendations on improving quality, we may need to briefly consider recommendations to help the communities be better placed to avoid the major issues that have prevented quality improvement in the past.

Rationale: Quality can't be easily improved without some attention to the way editors make decisions. Quality improvement recommendations may have limited effect, if communities don't simultaneously have better ways to handle large scale decision-making and differences of opinion.

I read this as empowerment and consideration for the processes people already utilize. Is this what you meant?

I just interviewed to sit on a local library board. For a year I have been talking to people who work at the library about their jobs and after my interview continued to do so. I realized that the board could strategize forever about what the library should do but without knowing what it actually already does and how people do what they do, the board couldn't draft livable strategies that were going to be owned by the people who worked there. After all, it is the people doing the job that make or break anything. I.e., if the decision rule is consensus and someone decides to stall it, is there a process for evaluating the situation and bypassing the consensus rule?

So the questions would seem to be: how do we poll contributors about what does and doesn't work for them, and about their degrees of tolerance for changes? How do we make them stakeholders in the process? What are the decision rules for this community?

Not quite. The last sentence sums it up - Quality improvement recommendations may have limited effect, if communities don't simultaneously have better ways to handle large scale decision-making and differences of opinion.

1. the online community that participates tends to be more independent and less rule oriented, in general. 2. top down rules don't work. the rules have to be based on shared community values. 3. when rules are in place, there is always a "playbook" to refer to when disputes arise, and a code of conduct to follow. 4. thus, to get independent souls to collaborate in concert with minimal upset, create a set of rules that takes into account shared community values, how they do what they do, how they want to do what they want to do, and yet keeps the end goal in sight. 5. no matter how hard you work, many will still resist change, even when they can fully see the benefit.

so there are basically two main choices: mandate the rules regardless of community values and the people who agree with the rules of decision making will show up, hopefully, and continue to participate. -OR- involve the community in creating value based rules that take into account their preferences and current methodologies and you have a better shot of them playing nice with others.

I am certain there are a number of change management people out there who could contribute their opinions on this. I guess I assumed that QC/CQI (continuance quality improvement) was going to be implemented hand in hand with a play book so there would be guidelines when inevitably someone doesn't play nice.

Misses the point a bit.

The community doesn't have good ways to make large scale decisions and large scale divisions of opinion, so the mechanism to choose such options you name, doesn't exist in the first place.

It's a "chicken and egg" problem. Starting from basics we might:

1. Ensure that attention's drawn to it so that it's taken more seriously as a quality factor, which means...
2. ... a clear consensus may form that communities do need a way to make good decisions on a large scale...
3. ...the communities can then formally reach consensus on how they want to make their large scale decisions going forward.
4. This would mean that in future, quality related matters don't stall under such problems as "too difficult to get a clear decision" or "vocal fringe that's unclear how much weight to give them".
5. It also would mean that other difficult decisions eg around content, processes, self-governance, improvement, and other major issues, can also be easier and less stressful too, which discourages many people from even trying to propose improvements or wastes a lot of volunteer energy.

Valid points, but are we sure that the community decision processes are having trouble? I do see problems with the community, but in other areas. --Piotrus 03:52, 24 November 2009 (UTC)

Piotrus, where do you see problems in the community, in what areas, and what would be your suggestions to address these concerns?

Keeping it short, see my mini-essay here. Bottom line: I've seen many good, productive editors - including several academics - leave the project. Most of them were relatively experienced Wikipedians, who got fed up with various personal attacks and otherwise stressful incidents. Hence, a major problem we are facing can be summarize as follows: 1) quality is proportional to the number of editors 2) we have mostly exhausted the low-hanging fruits (techy geeks who find editing encyclopedia fun) - as in they have already became Wikipedians 3) we are having trouble attracting other groups (but that is what the usability and other task forces are taking care of, partially) 4) we are running into a problem that we are burning through our existing contributors (they are getting fed up with the project and leaving for less stressful and more rewarding tasks) 5) at this point ([1]) we have reached a stage where we are loosing contributors faster than gaining new ones 6) thus, quality may be and is likely to suffer due to a diminishing number of dedicated editors 7) my conclusion: we need to understand why editors are leaving, address the issues causing them to leave, reach out to those who left and try to bring them back, and support plans to tap new pools of editors. --Piotrus 03:59, 25 November 2009 (UTC)

Just checking in from the community health task force... Piotrus is onto something. Ortega actually said that quality is highly correlated with the number of editors working on an article, and that the loss of editors risks making even basic maintenance an issue. At our task force, we are trying very hard to figure out what is causing editor burnout and how to stop it.

A theme that might be relevant to your task force are disputes and hostility. I'm not sure how much of it is just a change in culture, as Wikipedia probably attracts different personalities in 2009 than it did in 2001. But I think there's a real systemic problem with how disputes are settled. At least, I hope it's systemic, because that's the only kind of problem we can have control over. We can't change peoples' personalities, but we CAN change the system so that contrary and disruptive personalities aren't empowered.

Which brings it back to the concern about consensus-building. Ortega suggests that there is a problem with consensus-building falling off, and more people pushing their personal opinion. I suspect that consensus building is falling off simply because of scale, if nothing else. It's easy for 2 or 3 people to agree about how to change a couple of sentences on an article. But it's hard for 50 or so people to come together and figure out the best way to cover an entire topic area. And yes, those two person debates escalate into WikiProject discussions or RFCs or policy discussions, which attract entire cartels of opinion pushers. And sadly, many of those opinion pushers are not interested in consensus building. They've actually realized that they can accomplish more by filibustering debate than by compromising. So you keep on having the same debate over and over, hoping to make progress, only to see it fall apart because the different extremes unified in their desire to keep the debate going forever.

So not only does quality never get a decisive resolution... you actually lose good contributors who just got sick of trying to mediate between the different fanatics.

Just my perspective from the community health task force. Don't let me interfere with your work... only trying to help.

Thanks Piotrus, that was insightful. I already had your 'morsels of wisdom' among my bookmarks and read them some time ago (I think they're really good). And Randomran thanks too, I've put your project on my watchlist and will follow your discussions.

I'd like to point out that:

• We still don't have a sound definition of quality to start with. Ortega uses the FA status as objective measure and then combines this with the number of people editing an article. However, in my experience this can only have real value at the English Wikipedia and some other really large projects. (At smaller projects the number of people editing an article is not always a positive factor when an article reaches FA status. There are not enough quality contributors at these projects to make teams. At smaller projects, one quality contributor can write/translate an FA article while ten unknowing but well meaning others can destroy it);
• We are now implicitly assuming that the number of total editors is equal to the growth/decline in quality at a project. I think it is not that simple. There are 'quality editors', and their number does matter. There are other editors who mean well but don't improve quality a lot, their number doesn't matter that much. There are 'vandals' (including other users that decrease quality of content such as POV-pushers), their number does matter since they destroy quality. There are editors mainly involved in maintenance (including bots), they decrease the amount of damage done to quality by vandalism and thus matter too.
• The risk that a certain editor leaves the project exists, the average user account has a certain life time.
• Let's face it, the average Wikimedia contributor is an individualistic, technically well developed person, often with a liberal attitude about social interactions. He (I dare guess most Wikimedia users are males) knows about computers, but is not so good at communicating with others. This bad effect is strengthened by the choice/option to stay anonymous at Wikimedia projects (Citizendium tries to exploit this weak point of Wikipedia).
• The number of quality editors and 'maintenance editors' can grow when the help functions of the project are improved. We've explored this in our discussion above (practical example: idea of wizards).
• I assume most users that grow into 'maintenance users' more or less fit the profile I sketched above. I dare assume that many of the best 'quality users' don't fit this profile though. They know less about computers (they know about the areas where they are adding quality instead), are generally more civilized/communicative and thus have a bigger risk of burning out and leaving. Even though we need both types of user for quality, the balance will slowly develop in favour of the 'maintenance users' (because they have a higher 'life time' expectancy).
• We end up with the situation that most of the users at a project are primarily involved in maintenance. They hold the best positions to have their ways too, because doing maintenance is a requirement for being an admin/bureacrat/etc (this does matter to have your way, see Piotrus' essays on wikipolitics and why good users leave).
• The process thus builds environments in which it is very easy to do maintenance and the most active maintenance users are highly respected. Adding quality isn't easy though, because there is simply little thought/political influence left for quality users. The type of user adding quality is confronted with bad behaviour often and often doesn't have the final say about content either. No wonder they leave sooner...
• The influx of new users of all types is probably related to the size and thus the growth of a project, but let's assume it is constant for now. At larger projects like wp-en, the influx of new quality users is at a constant high level, at smaller projects, it is at a constant low level. This means that quality users are often found in isolation at small projects, surrounded by maintenance users and vandals. When they leave, the quality they added is at risk of being destroyed again.
• This is imho one of the biggest problems our task group has to find solutions for. The questions are: 1) How can we ensure quality users get more respect and/or influence in communities (and in that way prolong their life time)? 2) How can we get quality users at local projects out of their isolation? Besides, we may want to explore the questions how to decrease the amount of vandal users and increase the amount of maintenance users too, since this increases quality as well.

I see I made a mistake about Ortega's comments on FA status. The statistics do show that the number of editors increases the chance an article becomes FA. I think it is an objective measure, yet not representative. It only shows featured articles have been edited by many users, not that content with high quality was generally edited by many users.

Yeah, it's a correlation and he's careful to note that it requires further investigation.

Having written ~20 FAs - and this is my subjective experience - in vast majority of cases I was doing the work alone. Other editors helped a little, particularly with improving by non-native English, and of course there were hundreds of tiny additions/improvements over time. Did the article become better with more contributors editing it? Yes. But if I were to quantify it, I'd say that if on average my contribution was 100, than the contribution of 100 other editors was 0.1 each (*100 meant they improved the article from 100 to 110). In other words, in my experience as a FA (and GA, and DYK) creator, one user does most of the work, others do only a little bit that even if added together does not exceed the contributions of a single, main creator (that said, I am familiar with exceptions to this rule - I've seen FA/GA/DYKs which were truly collaborative - but they are, I again stress this, exceptions to what I've usually observed).

My experience with FA's is that actually after the certain level of quality has been achieved (for instance the article has been selected as FA) the majority of subsequent edits are actually detrimental and substandard - they only make the article worse and lower the quality. In my opinion, protection of the quality id one of the major issues we need to discuss at some point.

I'd say the same thing. Usually I'm working on a FA by myself, or at most in a "tag team". Maybe one or two peer reviewers will contribute a pile of copy edits. The correlation is likely attributed to the fact that FAs have usually existed for a much longer time, and something that's been around that long is bound to have more editors. Also, a FA is likely to be a high-attention topic, so it wouldn't be surprising to see more readers *and* more contributors.

But I think the broader point linking community health to quality is valid.

Ortega notes that "core" editors are definitely the main contributors to featured articles. And I think there is widespread agreement (although no conclusive evidence) that good editors are burning out. The closest thing I've found to definitive evidence: Ortega studied all the different language Wikipedias and found that "core" editors slowly fall out of the core, and fade away with time. But on the English Wikipedia, "core" editors go from being highly active to suddenly disappearing. I think that's the difference between someone who left Wikipedia because they were just running out of time and interest, versus someone who left Wikipedia because they had suddenly gotten fed up.

I understand the reason for sudden disappearances quite well. Fading users may be losing interest or get busy with RL and so on - I expect that's normal, and I am sure we have our share of those on en Wikipedia as well; vanishing ones are those who get sufficiently antagonized by negative reinforcement (read: PAs, harassment, etc.) that they say "that's it, I am outta here". --Piotrus 19:56, 26 November 2009 (UTC)

Just a personal perspective about online discord, I hate to say how long I have been online, (LOL) and I have literally left a ton of communities because of the discord factor, because things like my clients or my business or my daughter are just more important than watching people arguing or having them diss my opinion, whether valid or invalid. And I think it likely that is a common thread for the exact people we want to attract and retain as contributors and editors. So the whole supermajority/consensus issue is probably more important than I originally thought, having had more time to think about it. I believe we need subject matter experts who are dedicated to different sections who can help vet and edit. Those subject matter experts are likely to have the same kind of zero tolerance I have for unnecessary conflict. Just mho.

Agree 100%, this is likely a very major cause.

Important observation #1: This isn't just experts, it's most mature and capable users.

Important observation #2: "They are likely to have zero tolerance for unnecessary conflict"... yes, but also may not be entirely happy that others will have zero tolerance for theirs too. A lot of people expect their standards and others' to differ (usually leniently to themselves!). Same thing said elsewhere, even experts need induction/newcomer handling, "This is how we work, these are the expectations..."

some of this is, of course, going to be cultural. Across the en-wiki, we have how many countries/cultures speaking English? and even in the US, we have so many different ways that people interact, specific to geographic locations.

I think these factors, especially when dealing with a wiki that encompasses a large culturally diverse population, can be mitigated by the kinds of policies we have been discussing. let people know up front what the expectations are, let them know that Wikipedia takes protecting their rights and contributions seriously and we will attract and keep the mature and capable users. (I wasnt just meaning experts, btw.) I think the ideas we have been tossing around of creating a kind of incubator for a wiki prior to stamping it "launched" may also attract the kind of contributors we are talking about.

Valid points. Few comments:

• if we want to define/operationalize quality, I suggest we do it in a new section (how about the one I started few days ago, which raises the same issues in Q1-Q2 below?)
• there is certainly a major difference between highly productive editors and less productive editors; as some past studies have shown, 10% of Wikipedia editors produce 90% of content (more or less). Overall, the number of editors does matter, but the number of high end contributors of course matters even more.
• I don't think that editors "grow" into maintenance editors (gnomes). I think some chose to become them early from the start. In any case, maintenance editors are crucial for stability, content creators are crucial for growth. Both groups must be nurtured, but I would agree that content creators are at a higher risk of burning out, as creating content is probably more controversial than maintaining it (albeit I can be wrong, as I can think of some examples of large wiki conflicts about maintenance issues - but they could be exception to the rule).
• Your two questions:
  • 1) How can we ensure quality users get more respect and/or influence in communities (and in that way prolong their life time)?
    • • My answer: tough question :) One solution: encourage and give special powers to non-anonymous users (as described here). This would also address a major quality issue: anonymous experts getting overruled by anonymous (but more active, or experienced) POV pushers. This is indeed a major strength of Citizendium. Other: careful analysis of this. One one hand, content creators should not be given a carte blanche to be disruptive. On the other hand, I've seen examples where editors who have created vast amount of content were harassed, successfully, by editors who have done very few edits, yet admins and arbcom in their investigations did not seem to care about that difference. Solution 3: positive reinforcement. When faced with bad atmosphere and harassment, it is important to show editors that we still respect them and care - yet wikilove is much rarer than wikistress (for example, arbcoms commonly warn/admonish/sanction editors; very rarely they also recognize that those editors do something right).
  • 2) How can we get quality users at local projects out of their isolation?
    • • I am not sure I understand your question. Editors edit what they want to do, I know some editors whom I'd like to see more active in the community, but they don't care about anything but their little field. In the end, as long as they are happy in it, is this such a problem?
  • 3) Besides, we may want to explore the questions how to decrease the amount of vandal users and increase the amount of maintenance users too, since this increases quality as well.
    • • Regular vandals are a non-issue. The problem are those editors: POV pushers, or however we want to call them. They are harder to identify as disruptive, yet they are the ones primarily responsible for creating bad atmosphere and driving others out of this project. In my experience, it is those type of editors that are primarily responsible for driving good editors away, and I am not seeing any signs the Wikipedia system is able to deal with this issue. In fact, the danger to quality in the future is not only the loss of contributors - it is the possibly shifting proportion between good content editors and POV pushers. --Piotrus 19:54, 25 November 2009 (UTC)

@Piotrus (point for point):

• I started a new section (see above). I think the topic you started is the right way to proceed, but I think we should separate the definition of quality from Philippe's questions.
• I agree.
• Initially, most constructive users are prepared to do maintenance, though there is a group which is only interested in writing too. Actually, I didn't mean only gnomes, the 'classical' gnome type is supposed to be low-profile and uninterested in wikipolitics. There are a lot of users who do a lot of maintenance, don't add content, yet work themselves up to admins and have high social statuses. The thing is, just like it is easier to start a new article than it is to revise an existing article, it is easier to 200 times revert a vandal than it is to remove POV from one long article. Many users that start doing some maintenance will do more and more, for this reason. I'm not saying that doing maintenance is a bad thing, at the contrary. I just think the difference in the amount of 'award' the system gives for maintenance (enough) and editing content (too little) is a problem if we want to have projects growing faster.
• 1) I agree rewarding or at least protecting non-anonymous users is a very good idea, as long as constructive critique remains possible. This could at least increase the amount of quality content added by this group. Anonymous users can be 'quality users' too though.
• 2) I'm sorry, I forgot to give my arguments. I have two: my theory of wiki-erosion and my assumption that quality users are motivated by constructive cooperation.
  • Wiki-erosion is the reduction of quality by maintenance edits (or the absence of maintenance reverts), because the community's maintenance users don't have enough specialist knowledge. As an FA writer (we all have more or less compatible wikiprofiles, that's perhaps why we're in this task force together :) ) I know this natural erosion rate from watching my own FA being victims to erosion every so often. The 'natural' rate of wiki-erosion is related to the community size: the smaller the community, the smaller the chance specialist knowledge to identify erroneous edits exists. Thus, specialist/quality content at small projects has a higher risk of being 'eroded' than at larger projects. Wiki-erosion decreases the quality of the content to a level related with the size of the community. We can prevent this by identifying isolated quality users and reaching out to them, to help them move quality content to places where the risk of erosion is smaller. From my own experience, I know the example of Pelex, who mainly edits the Slovakian Wikipedia. He understands other languages but finds writing in English more difficult. I encouraged him to translate some of his articles to wp-en and helped to correct grammatical errors. There must be many users of this type (I only understand so many languages and my specialist knowledge is limited to geology so I can't identify most of them). At the moment, the Slovakian Wikipedia contains some nice material about geology not available in other languages. The language barrier keeps this quality content in isolation. Unlike its big English brother, the Slovakian Wikipedia doesn't get translated often (furthermore, translation of quality content requires a translator familiar with the subject) so the chance that this content will stay isolated is higher too. Imagine the specialist user stops editing after some time, the content is then in the hands of the maintenance users of the Slovakian Wikipedia. Due to the natural rate of wiki-erosion, errors will slip into it.
  • There is one more reason why we should get isolated quality users out of their isolation: I think it prolongs their life time. Let me cite Ortega again: he expresses some surprise at the fact that departure rates aren't related to policy and hints that it is probably rather related to the general style of social interaction. The fact doesn't surprise me at all. Quality users don't contribute to wiki-projects because they want to change policy, but because they want to show the world their knowledge. What all wiki-users crave for is some sort of 'applause' (all users are, in a way, like trolls, they seek some form of attention). If you wouldn't have written an extensive answer, I wouldn't have given this reply or at least I would have become less interested. The process of adding content to projects works the same way. What all wiki-users hate is the opposite of constructive cooperation: non-constructive cooperation, obstructive or rude behaviour and fights - according to your own model these things make users radicalise or leave.
• Third and last point ( :-) ). I don't agree vandals are a non-issue. Regular vandals are because they form a small group and will easily get caught. Being non-vandalistic is the easiest quality requirement to recognize and projects have been successful in battling vandalism (though I occasionally see something slipping through the net). Looking at the 'content-related factors of quality' I've identified, I'd say other forms of non-encyclopaedicity are the next easiest to identify (for example, a list of unknown family members is added to a biography). NPOV and verifiability are yet harder to identify, yet the most difficult is balance. This is maybe the reason that no project has a guideline about balance alone. The type of user you refer to ('true believers') mainly reduces NPOV, verifiability and balance. I agree with you: these are bigger problems than vandalism and we should focus on them.

Randomran said it for me - "there's a real systemic problem with how disputes are settled... Ortega suggests that there is a problem with consensus-building falling off, and more people pushing their personal opinion. I suspect that consensus building is falling off simply because of scale, if nothing else. It's easy for 2 or 3 people to agree about how to change a couple of sentences on an article. But it's hard for 50 or so people to come together and figure out the best way to cover an entire topic area. And yes, those two person debates escalate into WikiProject discussions or RFCs or policy discussions, which attract entire cartels of opinion pushers. And sadly, many of those opinion pushers are not interested in consensus building. They've actually realized that they can accomplish more by filibustering..."

Woodwalker also - "This is imho one of the biggest problems our task group has to find solutions for... 1) How can we ensure quality users get more respect and/or influence in communities (and in that way prolong their life time)? 2) How can we get quality users at local projects out of their isolation?"

I'd add a third aspect: big problems often need outside help -- and on enwiki we have problems so difficult, so extended in some areas, that by the time you exclude involved users, people who will be accused of bias, users unwilling to endure attacks, and only users with very high experience and capability, there's nobody (including its Arbcom) able, capable and willing to address these.

As our mandate is to produce a few recommendations, we need to think hard, what one change here is most likely to succeed and make the biggest incremental difference. I think having a way to formally recognize "trusted content editors" (noted before) is actually the one biggest change that could help. I post below, my thoughts why.

I would not disagree with saying that content creation is not valued high enough, but as a major content creator, I may be somewhat biased here :>

Anonymous users certainly can be high quality contributors. The problem is two fold: 1) when anonymous users quarrel with/harass non-anonymous experts 2) when anonymous experts lack incentive to become non-anonymous, while the fact that we have non-anonymous, verifiable experts would boost Wikipedia standing in the eyes of its readers.

Wiki-erosion occurs everywhere; you make an interesting point that it would be larger in smaller communities than in the larger ones - I am not necessarily convinced by it. In smaller communities, there is also less editor who can contribute to erosion. You are assuming that as community grows, the ratio of experts to non-experts improves - I am not necessarily convinced it is the case. That said, I do think we should be encouraging content translation and editor migration into English Wikipedia (the biggest, the best...). Invinting quality editors from lesser project will improve our quality, but in the end, its a zero sum game (they write for us - they don't write for their projects).

"Quality users don't contribute to wiki-projects because they want to change policy, but because they want to show the world their knowledge. What all wiki-users crave for is some sort of 'applause'". Certainly. This is why positive reinforcement is important, and negative reinforcement is so bad (IMHO, the major cause for vanishing of editors).

If we classify POV-pushers as vandals, than I agree, vandals are a problem. I meant the the "wheely on wheels" or "YOU SUCK" type of vandals are a non-issue; I think we agree on that :) Discussing the type of advanced vandals, I think there are two groups, both dangerous to quality, but one of them much more than the others. The first group are POV pushers who don't engage in harassment of their opponents, the other group does. The first group can be controlled by the community, as other than being slightly annoying they don't cause other editors to leave. It is the other group - the "true believers" who POV push and harass their opponents, often resulting in their burning out and leaving - that is a serious problems (as they cause quality contributors to disappear). --Piotrus 20:08, 26 November 2009 (UTC)

@Piotrus: about wiki-erosion: it is indeed also dependent on the amount of 'bad edits', on the project size. However, this only determines the erosion rate. So, at the English Wikipedia the erosion rate is higher than on the Greek Wikipedia, because there are more edits (on average of course, having patrolled both projects for vandalism I can assure you there's more vandalism at wp-el than at wp-en at some specific times of the day ;-) ). Sadly, this fact doesn't help. I assume Wiki-erosion is a hyperbolic function with time. It levels the quality of content to a certain level. That level is (in my assumption) directly related to the amount of specialist knowledge in a community. Therefore, even though it will take (on average) much more time to erode quality at wp-sk, the erosion will in the end be way more destructive than at wp-en.

Concerning the translations: I once made a suggestion on Meta a couple of years ago, but as with all Meta suggestions it did not go anywhere. I think translations of good and featured articles should be encouraged by all means. The problem is mostly to monitor the quality (not all projects, including en.wp, maintain high standards for FA and GA, and not all really good articles get selected since the authors do not bother about the status). We may think about bot monitoring the size of the articles, and if an article has been greatly improved and not reversed (to exclude vandalism) it gets checked manually and makes it to the list of articles for translation. This could be a Meta-wide list with individual projects encouraging the translation. Usually there are manu users willing to translate articles, they are just not sure where there is quality stuff.

Consensus is a hard sell. Rarely will every person agree with every other person on any one point, much less a complex web of information or processes.

I do not think we need to be focusing on creating consensus. I think we need to focus on understanding what users do, how they use this creation we call Wikipedia, and what they are really looking to get out of it, both the experience and the information. I also think there is a point where a supermajority agrees and that may be good enough. If you are reaching for a true concensus, I think you will derail the project in its entirety.

I want to clarify that I am NOT saying that some people's opinions do not matter. Every good policy "crafter" will tell you that compromise breeds forward motion. Unfortunately, the "audience" for Wikipedia may not like compromise. So the challenge is to identify stakeholders who will accept the compromise as long as the majority of their issues are dealt with, and sell the compromise to others. Perhaps the notion of compromise needs to be built into the framework rather than the concept of consensus.

Wouldn't it be awful if each community crafted their own policies and made their own decisions without functioning within a cohesive framework? I see the process we are engaging in as a certain amount of codification rather than handing a carte blanche invitation to everyone who has an idea to propose it. There has to be a certain amount of self-censoring that people need to be encouraged to do to ensure the system itself isn't overloaded with information for niche markets.

So in a way, that question leads to "what is Wikipedia growing up to be?"

@Bhneihouse: the 'supermajority' is actually what I mean when I use the word consensus at Wikipedia. ;-)

Anyway, the compromise is not ours to seek, Wikipedia being a tertiary source only. Wikipedia should just reflect the consensus and compromises of others, and if that doesn't exist, neutrally describe all viewpoints (even that seems very difficult sometimes :-) ). We have the principles of encyclopeadicity, neutrality and balance anchored in the guidelines of most projects.

A problem raised is that many users don't understand or don't want to understand these principles. Before we go on discussing that, we'll have to ask ourselves if this is an important problem to discuss here and now.

What I probably should have asked is how did what I wrote miss the point?

Also, what problems have you seen that lead you to believe that decision making among the communities is a key area to address? If you could share examples we could all learn from them.

Note to all: please give examples so we can all see through your eyes. Getting everyone up to speed and sitting at a similar table, if not the same table is preferable.

On enwiki, I get the strong impression that discussions touching on quality have at times been productive, but at other times held up by inadequate decision making structures that haven't quite scaled to that community. Getting a proposal considered - even if likely to benefit quality - is inordinately difficult. Part of that is because it's unstructured and hard to evaluate balance, also because discussion varies so much that there is no easy way to assess "balance of good points made" -- the points for (and how strongly held) and the points against (and how strongly held), so it becomes a sort of "guess your own vote" based vaguely on numbers and column inches.

A major step to quality would be to find a way to help communities make their own quality decisions easier, in future. That would have a strong long term effect. (The old "teach people to fish, rather than giving them fish" thing).

We have an opportunity here to say to all our communities, "Look, folks, a reason quality improvement decisions may not be made as fast and well as it could, is that communal discussions themselves haven't ever been designed for consensus on this scale. If you focus on it, and figure a way to improve consensus seeking on big decisions in your wiki, then you can do quality improvement easier yourselves. If that's not a problem holding back your wiki now, then judging by bigger projects it will be in future."

Issues this applies to on enwiki, for example, that it might apply to on other projects: - disagreements about reliability of sources, balance of mainstream views, new and amended editor processes, deployment of new proposed tools, proposals for ways to handle conflict and disruption better...

... all kinds of things, because they all work on consensus, and we haven't ever looked at that in its own right, as an underlying factor in how wikis operate.

That's where this post came from. make of it what you will :)

Just in case you missed it, I checked in from the community health task force with a comment. Some of us are definitely investigating the connection between community health and disputes. My comment is upthread.

My apologies, I have been away and sick and am joining the discussion late. I think FT2 raises an important point. My view may be slightly different than his in this matter - I think the anarchic nature of the Wikipedia community has both advantages and disadvantages. I do not thin that we should abandon this. What I mean is, I think that the anarchic aspect of the community is one thing that distinguishes Wikipedia from other projects and should be retained, and I also mean that any approach would have plusses and minuses, and in this case, I am content with the pluses and minuses in the anarchic community.

I think Woodwalker introduces a separate point. I agree that our articles should report a consensus "out in the real world" when appropriate. For example, the article on evolution is based on a consensus among life scientists, the article on gravity is based on a consensus among physicists, and so on. But I think FT2 is raising a different matter. Namely, in some articles there may be disagreement among Wikipedia editors as to what the consensus of x (biologists, physicists, Christians, etc), or may disagree as to whose consensus should be included (for example, the evolution article does not cover the consensus among creationists). In these matters, we hope that Wikipedia editors themselves can reach a consensus.

I know AN/I, ArbCom, and many other Wikipedians are tired of the difficulties in achieving consensus among editors working on some articles. I respect FT2's interest in exploring possible mechanisms to help in consensus-building. Personally, I do not think this is the major problem facing us.

Personally, I think that the core content policies (NPOV, V and NOR) are in most cases adequate external criteris that editors can refer to in trying to forge a consensus. I think the the DE policy is essential to help sanction editors who refuse to participate in a consensus-building way (namely, meaning you respond in good faith to other people's views, and are attentive to their good faith responses to your views). I think in most cases a relatively stable consensus can be forged given time - sometimes hours, but yes, sometimes weeks. I know weeks can sound like a long time to a Wikiholic, but it really isn't.

Am I missing the point? I don't mind good faith disagreement, but I do care to make sure I am actually responding to the issue FT2 raised. Slrubenstein 13:48, 8 December 2009 (UTC)

I actually think that consensus is a pretty core problem. At least for community health. I know you guys are focusing on quality, so maybe it's less important for you. (I'm sort of checking in from the other task force whenever our discussions overlap.)

But I think it's really interesting that you think NPOV, V, and NOR can settle a lot of issues. I actually think that you're right in a lot of cases. A lot of the consensus problems actually fall apart because people either don't understand or don't care about those. (You were right to point out that it takes time for a newbie to understand them.) Someone insists "I'm not neutral? This is an important issue among [whatever group] and Wikipedia is full of [other group] bias!" Or "it's verifiable in www.johnny'swebsite.tv!!!" Or "it's verifiable, but I'm not going to look for sources, so just look at the google hits." The problem gets worse among Wikilawyers though -- then they actually continue to insist that they're meeting NPOV and V and NOR. They will wikilawyer "there is no consensus that johnny'swebsite is unreliable." ... and then there's "ignore all rules".

Maybe one really effective thing that the board could do is give backing to the core content policies? It could actually help in a lot of discussions.

Great thread and I know the community health team is wrestling with this as well. Change is hard and getting consensus for change in a well-established community is the hardest part. When I think of the work I've done with clients over the years, it is rare that there is an emergent consensus from the diffuse community that change is needed. In addition, a top-down change mandate rarely works either unless there is a huge crisis.

My read on where Wikimedia is today is that there is a strong consensus about the purpose Wikimedia serves (the vision) and a growing consensus that the way the community is engaging around article quality is becoming an obstacle both in terms of the ability to reach high quality consensus on articles (too much "last person standing wins") and on acceptance/cultivation of new editors with expertise to contribute to existing articles as well as new areas.

I wonder if the strategy to start to tackle these challenges is one of many small innovations/experiments, some that might take hold and others that might fail. For example, what if a small group of editors developed a new approach to consensus building on an article. They might set out some simple guidelines and then commit to follow those guidelines. They might also volunteer to facilitate discussions on articles where there are tough disputes. Over time, these guidelines and behaviors would evolve (get improved) and they might provide an alternative way to engage.

I could imagine a bunch of better ideas than this, but I think the "solution" will come from many small changes by many people that add up to a major culture change rather than a big bang shift in the culture.

Thanks again for the continued great dialogue on this issue.

--BarryN 18:15, 8 December 2009 (UTC)

Thanks Barry. Those are good points for community health. We're discussing how we form consensus and make decisions. Adding/changing roles (e.g.: facilitators) in our organizational structure is another thing we're discussing.

Those two areas are high on my list, personally. We'll see what other task force members think.

I'm in favor of facilitators. On general principle. :)

Philippe, Facilitator

I'll be even more general. I'm in favor of ANYTHING that will produce a consensus when other options have been tried and have failed.