Findings, conclusions and recommendations by Piotrus (as of 26 Nov)

Starting point

Piotrus summed up some rough agreed points on quality editor departure and negative reinforcement (discouragement) and his early conclusions and recommendations:

1. Internal wiki processes with regards to improving content quality (eg assessment/FA/GA) are functioning well are not in need of any serious reform.
2. Number of editors correlates with article quality, and quality is threatened by a decline in editorship. In his view this is "the biggest danger to the project's quality and very survival is this decline, and recommendations should focus on stopping and reversing it".
3. We should endorse solutions to increase editor uptake and decrease editor burnout or discouragement
4. We should gather more information to survey departed editors, software to facilitate easy project-wide surveys in future, issue a call for academic studies on the observed decline in users; and encourage a peer-review outlet on Wikipedia research.
5. People "edit Wikipedia because it if fun and otherwise personally rewarding"; negative discouragement must be strongly countered and limited (better than at present) to allow quality to be established.
6. Many editors leave after attacks; those attacking tend to remain. It is rare that civility is properly enforced.

Specific recommendations:

1. "we need to be more active with dealing with personal attacks and other civility violations. Telling editors to grow thicker skin is resulting in them leaving the project... [The] "Personal attacks noticeboard" needs to be recreated, and admins must be coached to treat personal attacks and other civ[ility] violations as seriously as 3RR".
2. Editors who edit non-anonymously should be as well protected as BLP subjects. (Even one unfounded or exaggerated accusation against them under their real name could be a concern in some cases)
3. More surgical remedies are needed than simple bans. Blocking users for a few substandard edits, when in fact their overall pattern is a concern, is common but sub-optimal. "[S]olutions like article / topic bans and probations, 1RR restrictions, civility restrictions, talk page bans, and such should be used more often. That said, an important qualification to their use is whether the editor sanctioned [sincerely recognizes] s/he did something wrong..."
4. Harsher remedies may be a factor in sudden departures; more mentoring and a lot more focus on positive enforcement (recognition, awards, mediation, counselling, etc) would help.
5. "It is very rare for [enwiki] ArbCom to recognize that editors who were in some ways disruptive are also constructive. Often, FoF/remedies will treat an editor who did few errors harshly, ignoring his good contributions and issuing a generalizing findings that he was disruptive, damaging his wikicareer, and ignoring his attempts to reform. I think it is imperative that ArbCom reevaluates its approach to editors and in particular, tries to be more constructive than destructive, in line with our policy that sanctions should be preventative, not punitive."

General discussion

FT2 agreed with "the sentiments" but was concerned that suggestions at this level would be very limited in terms of ability to change culture, and would quickly become "yesterdays news" -- "I like the ideas you have, but for my money we need to focus on what would make the most significant change[s] to progress quality, that once set up achieves the most, with littlest communal 'pushing'... anything that relies on telling a large number of users "do this instead" ("this" = some ongoing behavior change) is doomed to failure -- it'll be old news in a week, 99% won't hear in the first place, culture won't be changed by it, new users won't hear it"

Piotrus affirmed that "the biggest problem facing us is the erosion to due to hostility/incivility/crude sanctions, and our recommendations need to be very clear on the need to decrease negative reinforcement and increase positive ones" and that the suggestions were intended to guide specific (mainly positive reinforcement and support based) solutions.

FT2 was concerned that merely "saying nice things" and "too narrow proposals" would not help much for a user on the way out, and suggestions directed at Arbcom would not be of much impact either -- he felt we should focus on the community and identify recommendations with a permanent and pervasive effect that does not rely on a general statement "telling people what to do".

Piotrus was concerned that this focused on getting new editors but ignored the need to prevent erosion of established editors.

Bhneihouse wanted to see a bigger picture and felt this was too detailed at this point in time. Piotrus suggested the more general suggestions should be retained with a view to ways to make them practical; which was generally agreed.

It seems we have a rough agreement (and some early academic studies to back it up) that quality editors are leaving due to lack of positive reinforcement and significant amount of negative reinforcement (personal attacks, harassment, etc.). As such my early conclusions and recommendations are:

Conclusion 1: based on my own observations as well as extensive academic literature on Wikipedia's (variously defined) quality (available here - search for string "quality" for sample works), I believe that the internal wiki processes with regards to improving content quality (with initiatives like article assessment, FAs, GAs, and so on) are functioning well are not in need of any serious reform.

Conclusion 2: there is an obvious positive correlation between the number of editors and article's quality (and obviously, number - see also conclusions and recommendations of Task force/Expanding Content).

Conclusion 3: quality (as well as content creation) are being threatened by a recent decline in the number of active contributors). As such, the biggest danger to the project's quality and very survival is this decline, and recommendations should focus on stopping and reversing it.

General recommendation, set 1 (encouraging focus on methods of dealing with the decline in active contributors):

• GR1.1: endorse solutions to gain more editors (Task force/Reader Conversion)
• GR1.2: endorse solutions to prevent editors from burning out and being burned out (Task force/Community Health)

General recommendation, set 2 (gathering more information):

• GR 2.1: we should endorse the Proposal:Survey Former users, a survey to analyze in depth reasons editors are leaving
  • GR 2.1.1: we should propose that the Foundation prioritizes creating methods for easy and regular implementation of project-wide surveys (2008 General Survey is the only general survey carried out; it is not inconciveable to say that if surveys were done more often, current decline of editors might have been anticipated and prevented)
• GR. 2.2: we should issue a call for academic studies regarding the observed decline in users, the Foundation could encourage creation of a dedicated track for related research on Wikimania conference
  • GR 2.2.1: on that note, it would be useful to improve the academic standing of Wikimania conference, by encouraging creation of a peer-review outlet on Wikipedia research, tied but not limited to the conference (see also related discussion on wiki-research-l)

Hypothesis 1: a reasonable hypothesis for a decline in editors is that it the decline is a result of editors burning out and leaving due to negative reinforcement (unfriendly editing atmosphere stemming from personal attacks, harassment, battlegrounds). In other words: we edit Wikipedia because it if fun and otherwise personally rewarding (as proven by numerous studies of editor's motivations). When others make it stresfull to edit it, we leave. Conclusion 4: Negative reinforcement must be counteracted and limited under current, unacceptable levels in order for project's quality to improve (or even remain constant).

Specific recommendations, set 1 (crafting solutions that aim at reducing negative reinforcement):

• SR 1.1: we need to be more active with dealing with personal attacks and other civility violations. Telling editors to grow thicker skin is resulting in them leaving the project - few editors like to edit in an unfriendly atmosphere, and many quality contributors are real life experts not used to being flamed and harassed. Personal attacks noticeboard needs to be recreated, and admins must be coached to treat personal attacks and other civ-violations as seriously as 3RR.
  • Personal observation 1: I have witnessed many good editors and major content creators leave the project after being subject to personal attacks, such as accusations of academic dishonesty or nationalism (and I've been tempted to leave the project myself due to being a vitctim of such attacks). The editors responsible for such accusations - usually much less productive in content creation or even maintenance - are still here, unpunished and unrepentant, often targeting others...
  • Personal observation 2: it is very rare for an editor to enforce civility properly. Recently I've witnessed an excellent example of good civility enforcement by Manning Barlett clerking EEML arbcom case (disclaimer: I am involved in that case) and I think it could be a great case study of how CIV should be enforced
• SR 1.2: editors who chose to edit non-anonymously should be protected with the same level of responce as the subjects of BLP articles (essay), although ideally the same level of protection should be extended to all editors
  • Personal observation 3: I was told by several editors that they refuse to edit non-anonymously because they are afraid their peers will see a comment on Wikipedia where somebody accuses them (with their real name) of being [insert PA]. Some of them stopped editing anonymously for that reason, on the off chance their anonymity would be compromised.
• SR 1.3: at the same time, while coming down strongly on uncivil editors, we need to develop and encourage the use of more surgical remedies than simple bans. Blocking editors from editing for few problematic edits, when their overall pattern of their contributions is not problematic, is a common but substandard option (essay), as while preventing possible disruption, it also prevents quality content creation at the same time and sends the editors a message they are not welcomed here. As such, Isolutions like article / topic bans and probations, 1RR restrictions, civility restrictions, talk page bans, and such should be used more often. That said, an important qualification to their use is whether the editor sanctioned is recognizing s/he did something wrong and promises to behave better (and whether it can be assumed they are being honest - repeated offenders should get harsher treatment, of course). Unrepentant believers in self-perfection ("true believers") are not needed here, but editors willing to deradicalize should be helped and encouraged.
  • Hypothesis SR 1.3: Incidentally, I think that en Wikipedia tends to favor blocks and bans over more surgical remedies due to being dominated by editors from United States, whose current system strongly favors incarceration for even slight crimes (if in doubt, err on the side of imprisoning/banning, don't worry about mentoring). Harsher then necessary remedies on en Wikipedia may be partially responsible for dominance of vanishing editors over fading ones, as slapped editors leave en Wikipedia, where on other projects they would be more likely to be approached with mentorship or similar, reeducation/deradicalization arrangement

Hypothesis 2: building on hypothesis 1 and conclusion 4, a reasonable hypothesis is that more positive reinforcement can counteract negative reinforcement. Conclusion 5: positive reinforcement should be encouraged beyond the current levels found on Wikipedia.

Specific recommendations, set 2 (crafting solutions that aim at increasing positive reinforcement):

• SR 2.1: it is difficult to be specific in saying spread WikiLove :) Initiatives that recognize editors, give them awards, etc. should be promoted. Perhaps the Foundation can use some of its funds to recognize top editors with some awards (plaques, etc.), or - perhaps a better or complimentary solution - hire a few staffers who would be full-time morale officers - giving out barnstars, relaunching ESPERANZA, etc.? Similarly, we could use more editors dedicated to mediating, mentoring and even counseling. We don't pay for content creation, but maybe we should pay for making the editors feel welcome.
• SR 2.2: in my analysis of ArbCom findings (which I've seen close and personal several times), I find that it is very rare for ArbCom to recognize that editors who were in some ways disruptive are also constructive. Often, FoF/remedies will treat an editor who did few errors harshly, ignoring his good contributions and issuing a generalizing findings that he was disruptive, damaging his wikicareer, and ignoring his attempts to reform. I think it is imperative that ArbCom reevaluates its approach to editors and in particular, tries to be more constructive than destructive, in line with our policy that sanctions should be preventative, not punitive.
  • SR 2.2.1: this can be applied to blocks and other solutions issued by admins as well, in conjunctions with SR 1.3: when issuing a restriction, try to say something nice about that user, and try to encourage him to reform.
  • Hypotheses SR 2.2: it is likely that the attitude of ArbCom (assume guilty until proven innocent, harsher sanctions are better than surgical, no need to be constructive / recognize bad behavior, focus on punishment over prevention) trickle down to admins and the entire community, making reconciliation/deradiralization/civil disagreements more difficult, and contributing to editors leaving with the feeling that they have been treated unjustly and too harshly by the system

Uff. That's it for now, let me know if this makes sense :) --Piotrus 21:46, 26 November 2009 (UTC)

Interesting, but (for me) a bit limiting. It seems to jump from a rather obvious statement about joiners,leavers, and +/- reinforcement, to conclude we have now understood what needs doing. The points are fair, but they feel very limited in style. Saying "improve support, survey leavers, nuance and improve sanctions" somehow seems limited. That's not to minimize the points, but it doesn't feel "hands on" enough. What specific changes would have great impact?

Bear in mind the Foundation/Task Force will state "these things need doing". If they are general, like "Improve civility" or "Improve sanctions", then it's an instant fail, because we won't have enough ability to say how, and a week later it'll be yesterday's news. So look for specific changes that, if stated, have a very high likelihood of leading to real direct improvements, and changes that will persist and grow.

• "Survey ex users" probably won't. We can probably guess the main issues and reasons quite accurately right now; it's unlikely to be the best use of our recommendations in terms of "knowledge gained" or "changes made".
• "Improve Arbcom sanctions" probably won't - wait 6 months and watch to see if the Arbcom of 2010 is really going to invent a new approach.
• "Tell people to be nicer to newcomers" probably won't - you have the inertia of many newcomers and "tell people to be nice" has a very low leverage.

By contrast:

• Set up trusted users? Yes - because once set up, that spreads out and influences many things positivelky, without further "pushing".
• Give new users a range of wizards to guide them? Yes - because once set up an ongoing and visible improvement in new user knowhow and understanding (including dealing with slightly advanced matters like edit warriors and processes) becomes likely without further "pushing".

I like the ideas you have, but for my money we need to focus on what would make the most significant change to progress quality, that once set up achieves the most, with littlest communal 'pushing'. That's the key.

I am not disputing your recommendations, in fact I have no problem endorsing them. I do however think that the biggest problem facing us is the erosion to due to hostility/incivility/crude sanctions, and our recommendations need to be very clear on the need to decrease negative reinforcement and increase positive ones.

I also fully agree that specifics are better than generalities. Once we agree on generalities, we can discuss various specific solutions. Note that my analysis above has quite a few of those (recreate PAIN, create 3-strike rule for CIV violations just like for reverting, hire people to specialize in positive reinforcements, have arbcom say "nice" things to prevent editors from feeling crushed by the system, etc.). Here's another idea: community-reviews of editors focusing on positive reinforcement (with constructive critique if needed), targeting stressed editors, those who left or got restricted/banned (with recommendations for more surgical sanctions). This would serve as a community effort to retain/reclaim editors - one could nominate those who departed/are close to leaving, they would see they are recognized and needed, and hopefully they'd stay/come back.--Piotrus 22:35, 26 November 2009 (UTC)

I wasn't denigrating yours either; the sentiments are good, I think as specific suggestions they're at the wrong level. Specifically, anything that relies on telling a large number of users "do this instead" ("this" = some ongoing behavior change) is doomed to failure -- it'll be old news in a week, 99% won't hear in the first place, culture won't be changed by it, new users won't hear it. So I'm thinking in terms of stuff where those sorts of things aren't of concern. Again not as criticism, but so you can see my thinking

3 strike for civility -- what's civil and what isn't is a common hot point as it is. Suggesting that communities form a "clean start" view on acceptable interaction and conduct to other users, gain consensus, and then stick by it from there on, would be better... but if we said that, the odds of it happening are low.

I'd swap all the nice things Arbcom could say, for better tools to address the problems sooner in the first place.

I'm trying to think of users who left, where a "nice things to say" panel or board would do much. At that point it's about their real life (busy, stress, etc), their real life interests (discovered second life or got a girl/boyfriend), etc. A lot isn't under our control. What we need to do is make it more satisfying and something they can develop, not just tell them "we (really do) love you". Especially as a "Oops! We heard you were pissed off and disillusioned so we wanna say we love you". That's not going to help more users on the way out.

I agree that some things are hard to change, due to organizational interia and so such.

Making the ArbCom faster - sure. Would more arbitrators help? You have first hand experience with this, IIRC :)

I guess my recommendations is heavily connected to my experiences. Not only personal, but that of observing WPPOLAND for 5 years. Our recruitment ratio = editors leaving ratio, and with few exceptions, most editors leaving left because they were fed up with unpleasant atmosphere (personal attacks, harassment). Would you disagree that this is a reason most editors leave (mind you, I am talking about leaving, not limiting activity due to RL issues)? And would you disagree that that preventing editors from leaving should be our top priority? --Piotrus 03:27, 27 November 2009 (UTC)

Yes. But (again) I see "fixing Arbcom" (as a conclusion and route) as a complete dead end in the context of this discussion. It's a tiny part of quality intended only for where communities fail. Here, we prioritize "how to make success in these areas much more likely -- without Arbcom".

Assume that anything that relies on persuading a group of users to have different outlooks will fail (see elsewhere for why). Also having worked on Arbcom I can tell you it's the wrong place to be focusing. They're under intense strain, and they cut stuff to the bone and even then it's barely keeping up. We need to focus on the community, and ways to make specific changes that will have a large effect even if this report becomes "yesterdays news" and almost nobody reads it. Instead, engage the mass of people who do edit, and those who don't, to resolve it, by guiding and informing those efforts better.

Assume bulk persuading people "don't do this, do that" is a fail. Instead, make it so that many people doing "this not that" becomes almost inevitable due to simple human nature. Much easier :)

• If you set up newcomer guidance they will take benefit from it, continually, even if the report is ignored.
• If you hammer the basics of quality and highlight improvements on every medium, emphasizing you CAN fix it, more people will, continually, even if the report is ignored.
• If you try and tell Arbcom "be nicer" or "do this not that" it'll last approximately until the next 50 emails arrive; they're firefighting already.

Sorry to harp on it, but it's key to success.

It's not that I disagree with you; it's that your solutions seem to focus on getting new editors - which is fine, but I think we need to figure out how to retain them (and recover lost ones), or we will just keep burning through them until eventually there will be nobody (old, young) left to recruit, and all we will have is a bunch of flaming trolls creating poisonous atmosphere :(

Ah, minor correction perhaps. I'm looking at all aspects, but mostly, what is the most we can do for quality, with 2 - 4 recommendations, that will have strong persistent effects. I think I have said clearly that this includes improving the environment for longer standing users, giving them further skills and help. So we agree there. Where we've differed isn't in what's desirable, but that goals like "Make Arbcom be nicer" or "Set up 3 x CIV as a limit" are in suggestion terms, dead ends. Too narrow, too detailed, unlikely to gain the level of culture change needed, unlikely to have the pervasive effect a task like this is looking for. We agree on goals. I just say those specific approaches are too "slim". Hope that clarifies.

This all makes sense but I fear that the detail level is way too far along for the process. I would like to see more big picture, focused thinking that addresses the infrastructure rather than negotiating exact details at this point. It is good to keep thinking about this and typing it in, however could you focus that same energy and brain power on an assessment of what exists, followed by looking at the whole and then component pieces to assess what works in a Quality framework?

Not dissing anyone, just trying to redirect a bit.

At the very least I suggest we don't discard the more general recommendations; perhaps somebody can make them more practical.

practical, yes. That is my destination as well.