Diminishing retention rates

Nowhere in your study I see anything about a test to see whether or not the same kind of people (age, gender, social background, IQ, education, working environment, accessibility to scientific resources) are signing up for Wikimedia projects like wikipedia. If the userbase changes, then changing the editor or the accessibility across different platforms makes no sense. If you statistically correct for all the factors that might influence your results, you might find that the kind of people that primarily signed up before 2005 still have a high retention rate, while the new kind of people (hypothesized to be teenagers and adolescents) have a relatively low retention rate but contribute to the overall results due to the vast numbers. Redtails 13:05, 11 March 2011 (UTC)

One anecdote does not a study make - but I may just be a counterexample to your hypothesis. Being an internet oldtimer, I appreciate the anonymity, so won't give full vital statistics. But I am not a teenager. I started fixing grammatical and clarity issues, created an account so I could create pages - then only created two, both translations from other wikipedias. I entered the wikicup - then never bothered to contribute. Part of that was because life got busy around about that time - but mostly I've become disillusioned before ever going through the normal honeymoon period.

The policies look good as written - but only appear to apply to some people. Many admins are clearly exempt from most, if not all, of the civility policies. "I've never heard of it", "I don't like it" and "it's not mainstream American" are primary reasons for article deletion. "No sources in English is another popular reason for deletion.

I haven't personally been bitten by this kind of thing. But it has produced a chilling atmosphere.

More in another thread, if I have time. This isn't the right place to point out that elephant, since it will be less noticeable here than in its own thread.

Kobnach

There are quite a few problems with Wikipedia today that make it virtually unusable for new contributors. Many of those have been touched on here already, but a few have not, or not to the point I think they should be:

1) The notability guidelines are a ticking timebomb. The first issue is that they are listed as guidelines, but are treated as hard and fast rules by most of the admins/editors. The worst of it is that the admins don't even seem to want to follow the letter of the guidelines anyways! Often "I never heard of this" is the justification and there is no recourse for a new contributor to surmount this. I know that there has been considerable discussion surrounding new procedures for dealing with this, as WP is want to do, but I think that's wrong headed.

1a) Admins/editors first need to be re-educated on what the word "guideline" actually means. I believe that this is the first problem that creates a situation where new users end up bullied by self-righteous admins who don't like to have their word questioned.

1b) Notability is a dumb guideline for the most part anyways. It affects wikipedia very little to have long-tail articles of limited interest. Disk space is virtually unlimited these days. If all that's allows on wikipedia are articles about things that everybody already knows about, then there's no point to the entire project! The point of something like wikipedia is to aggregate and summarize information on things people don't know!

1c) Admin counter arguments to getting rid of the guideline are always absurd (e.g. "we don't need to be seeing articles about people's goldfish", there's other, dedicated wikis for that kind of thing). Let's get real here. That's not the problem. The problem is when little known, to perhaps the general public, but perfectly notable within specific cultures and sub-cultures are removed.

Telling people to go put their trivia on dedicated wikis is pointless and insulting.

The recent mass programming language deletion gaffaw is a fantastic example of this. The abuse and dismissal of new users by established admins in the resulting discussion thread was inexcusable.

I wasn't a super active contributer before then, mostly because navigating past admins was virtually impossible and more thankless than bothering to contribute. But that incident has sworn me off of contributing completely until this is resolved.

2) There is no reason in the world for article length to be constrained to encyclopedia length articles. Like 1b above, if it takes a great deal of information to aggregate and summarize effectively all of the major points on a topic, then that's what it takes. The admin/editors seem to think they are working on an actual print encyclopedia for some reason. Digital space is effectively unlimited. It echoes arguments about the application of Latin linguistic rules to English. It's non-sequitur and moronic.

Arguments against this are often similarly absurd to the above (e.g. "we don't want to just replicate the internet here"). Well that's not what's happening. Get over it.

3) New user participation requires such a high bar to navigate, the deluge of inside acronyms, rules, guidelines, procedures and practices makes it daunting and virtually impossible for most people to work within the system. Established admins and editor are likewise de facto immune from all of this and operate at their whim. New users can't be smacked with a rule and then observe admin/editors behave contrary to that same rule and expect a good outcome.

4) Deleting things should be hard -- very very hard. It's not. It's very very easy.

The reward mechanism for new users contributing is very simple. You see your words and contributions showing up in Wikipedia immediately and globally, you can call your friends and family up and point them to it. You get the sense of satisfaction of having made a meaningful contribution to the world. That's the feedback loop.

Seeing hours of writing or editing revert seconds or minutes after submitting them is a tremendous turnoff. Nobody wants to give up their free time for something that goes immediately in the trash.

All users should be forced to try and improve submissions long before any deletion can happen. Only outright vandalism should be grounds for fast deletion. The vast vast majority of deletions are because the deleters are too lazy to improve what they don't like (despite flags for non-neutral POV or notability or some other B.S.). Deletionists should be considered the same as vandals. In my experience most deletionists are habitual and sociopathic. They do it because they can't create, so they destroy what other people have spent their time on.

If a user were to log on and just null out the content from page after page, that's how we'd consider it. Why do we tolerate deletionism?

If you see something in a non-neutral POV? Edit it! Bad grammar? Fix it! Disorganized writing? Fix it! That used to be the norm on wikipedia prior to 2005. Today, anything and everything just gets deleted. There is no incentive to bother to contribute because some overzealous bureaucratic pinhead comes along and deletes what you've written.

Anybody who deletes too frequently or too many times needs to be reviewed. Again, the recent programming language debacle should offer a textbook case of people up and down the bureaucratic hierarchy destroying knowledge instead of contributing through fixing.

Hear, hear. And I didn't even know about a mass deletion of programming languages. People who think they are improving the quality of Wikipedia by getting swathes of articles deleted are not thinking in terms of the end user - the person who wants to look something up. All too often, discussions revolve around editors, not actual users. The whole point of an encyclopedia is to contain recondite information and the whole point of an online encyclopedia is to integrate it into the matrix of more familiar knowledge. Not to provide yet another way for people to massage their egos online.

I've reached the conclusion there are far too many admins. Hence the prevalent mindset that deletion doesn't matter because "it can always be undeleted". Which shows total unawareness of the fact the editors who could improve it can't even see it or what's wrong with it. Only admins can. Somewhere admins went from being janitors to being sysops and prefects, like everywhere else online, and it's contrary to and detrimental to the spirit of the project. But these days that's who they promote to admin, and it's next to impossible to de-sysop someone.

As a quite experienced editor, I absolutely agree with the IP. There is an increasing divide between the inclusionist feel of most WP readers and casual editors and the hardcore deletionist attitude of most regular editors around. And the new page patrollers are often too much trigger-happy with CSD and PROD. Heck, the very existence of PROD is insane: basically it's a mechanism of the kind "I put a timebomb here; if nobody notices, this will get deleted".

It's also useful to consider the view of outside journalists who write about Wikipedia's increasingly visible deletion problem. I don't recall ever reading anyone outside Wikipedia complaining about finding too many non-notable topics here. But I have read people complaining about information that vanishes. To a typical reader, Wikipedia's notability requirements would seem like pure nonsense if the reader became aware of them. The reader doesn't care about what anybody else considers "notable" - if the reader wants to read about topic X, it's notable to the reader.

If Wikipedia wants to continue with its campaign of suckering new users into creating content which then gets destroyed, Wikipedia should forfeit its tax exempt status. Is there any other charitable organization that destroys as much intellectual property as Wikipedia? Deleting people's good faith work is not "charitable" in any sense I can recognize. The public is not asking to be protected from running across the occasional obscure topic on Wikipedia. Neither are many of Wikipedia's monetary donors demanding a smaller Wikipedia.

To the I.P.: I concur with your equating vandals and deletionists. (We might call them "vandeletionists".) Actually vandals are far less of a problem, because we have an arsenal of tools to combat them. But we have no tools to combat deletionists. In keeping with the principle that "bad money drives out good", deletionists have an automatic advantage over inclusionists, because inclusionists have no way to put points on the board. The best possible outcome for the inclusionist is a temporary draw, since an article can be nominated for deletion an unlimited number of times. A deletionist only has to delete one article, one time to (potentially) drive an inclusionist away from Wikipedia in disgust, whereas there doesn't seem to be anything an inclusionist can do that drives a deletionist away from Wikipedia. The only defense against deletionists is to work very, very hard to defend each article against deletion. But this only increases the damage done when deletionists win. You don't want to gamble your time to improve an article unless you are confident you will win. It's like the second law of thermodynamics automatically favors the deletionist.

To level the contest, we must have some way to delete the deletionists. For example:

• A cap on the total number of articles that can be deleted per year, and caps on the maximum number of deletion nominations from individual editors, to force deletionists to be selective.
• Escalating blocks against editors who only contribute to article deletion. It should be necessary for an editor to substantially improve at least ten articles for every article the editor helps to destroy.
• Some sort of penalty for proposing a deletion request that fails, and for voting in favor of deleting articles that are kept. Currently all the penalties are against constructive editors who must gamble their time hoping that their edits will stick. Inclusionism is expensive, deletionism is cheap. There is no penalty for making vexatious deletion requests. That's backwards. We're treating good-faith editors as if they are vandals.
• A statute of limitations. Deletionists should have six months maximum from the time of article creation to propose deletion on grounds of notability, after which the article defaults to "keep". If an article did not bother anyone for six months by being insufficiently notable, then it is insufficiently bothersome to get rid of ever.
• Proactive review of new articles to assure editors the articles will not be deleted later. Currently there is no easy way to know whether any given article will get zapped at any time in the future. Articles should be stamped "deletion proof" so editors can edit them with confidence.
• Requiring deletionists to provide credible evidence that shows net harm to Wikipedia from keeping an article. Offending a deletionist's sense of tidiness or importance is not such evidence, rather anything that offends deletionists making them lose interest in destroying content provides net benefit to Wikipedia.

While Wikipedia's strategizers are trying to analyze the falling number of editors, could we have some analysis of the strengthening hold of deletionists? Why aren't deletionists getting discouraged and giving up? What editors are spending the most time nominating articles for deletion, and voting for other articles to be deleted?

Maybe a relatively small number of Wikipedia editors are responsible for most of the editor retention problem. We should track the fate of editors whose work was deleted. Did they lose the confidence to gamble more of their time and effort on the uncertain outcome of editing? It would be interesting to identify the deletionist who holds the record for driving away the largest number of constructive contributors, if that could be worked out from the data.

Deletionism on Wikipedia appears to be a natural evolution of the features that make reverting vandalism as cheap or cheaper than vandalizing. The immune system that protects a wiki from vandalism can also mutate into an auto-immune disease against legitimate content, since contributions form a continuum from brilliant prose to well-intentioned-but-not-so-good down to vandalism. The need to revert vast amounts of vandalism without concern for the vandals creates the habits and culture in which we can act unilaterally and without apology or accountability against good-faith editors whose knowledge of the rules is incomplete or interpreted differently.

Diseases of the immune system are some of the most insidious diseases known to medicine. These will probably be among the last diseases to be cured. The challenge posed to Wikipedia by deletionism - as currently practiced - may be similar. Wikipedia might be turning out to be pretty good at attracting and retaining the type of users who will eventually kill the project.

An editor whose primary interest is deleting articles never loses. The deletionist either wins, or ties, since the only penalty when an article is kept is that deletionist immediately finds more articles to try to delete. How could a deletionist ever get discouraged? The satisfaction gained by inflicting pain on other people is never ending. It's a sociopath's dream job, perfectly legal and even applauded by fellow sociopaths who have found their stable niche. The deletionist invests very little to delete an article, whereas the contributors can have many hours of their labor at stake. If the article remains, it doesn't threaten the deletionist's interests in any way, nor create any obstacles to deleting other articles. Deletionists can simply ignore whatever they don't want to see on Wikipedia, as can anyone else. Wikipedia doesn't get any harder to search or navigate as the number of articles increases.

No constructive editor really wants to confront the deletionist, because the deletionist has a safe perch from which to examine all the constructive editor's edits, and try to delete those. The fact that most editors don't even look at WP:AFD gives deletionists a huge advantage. There is only a seven-day window in which to contest a deletion nomination, or improve the article, but no time limit on nominating articles for deletion. It's like the asymmetry between an attacker, who can choose the time and place to attack, vs. the defender who has to guard everything at all times.

The problem with watching WP:AfD for over-zealous deletionists is that over 95% of the nominations are justified. I consider myself an inclusionist, & even I have nominated a couple of articles for deletion. Crap (by anyone's definition) is added, & if it slips by New Pages Patrol it remains in Wikipedia until someone happens upon it.

On the other hand, I remember a certain person commenting that giving an article the benefit of the doubt & keeping it doesn't really harm anything. If it isn't notable or important, no one will ever bother to read it. I consider that one of the wisest things he has ever written about Wikipedia, but wonder if he would agree with this statement now.

Someone suggested a different thread that new articles can be kept in a lower-quality state until improved. As you say, nothing happens if a bad article is not deleted. I would add that keeping it on WP and monitoring page views would give evidence of just how notable the topic is, which after a period of grace (say a year) could be used in assessing whether the article is notable enough or needs to be deleted according to current policies (I wouldn't oppose changing the policy though). I don't see why we need speedy deletions and proposed deletion if we have other ways of fighting vandalism.
After reading Teratomis' reply, I'd like to add to the list of mechanisms to "delete deletionists":

• When an article is proposed for deletion all contributors to that article should be notify in order to appeal the proposal.
• Experienced inclusionist editors should be made voluntarily available to newcomers who which to contest an deletion proposal.
  

That's my two cents. Asinthior 14:59, 17 May 2011 (UTC)

You make an excellent point w/r to the external users of WP. Nobody outside of WP ever writes that WP has too many non-notable topics. However, the deletionist issue is something discussed ad naseum (and has been for almost half a decade with little done to counter it).

The most recent case has just come up today is the deletion argument surrounding one of the most notable gaming websites, http://en.wikipedia.org/wiki/Old_Man_Murray

Reading the AFD discussion (yes, that's right, the 2nd nomination), http://en.wikipedia.org/wiki/Wikipedia:Articles_for_deletion/Old_Man_Murray_(2nd_nomination)#Old_Man_Murray

The arguments are the same, some deletionist flags it, some overzealous editor approves it, and in the middle, a useless discussion over the semantics of notability generally centered around people who think that for something to be notable it must appear in a highly respected academic journal and be referenced several dozen times.

Respondents usually reply with a laundry list of notable references to the article, none of which, no matter how notable or significant will ever be accepted by the abusive deletionist and admin clique and the article will end up deleted.

This kind of nonsense needs to stop. But alas, it's been going on for 5 or 6 years with absolutely no response from WM.

Here's a typical thread (from the AfD discussion for OMM, but you can find almost this same discussion verbatim in most AfD discussions -- bolded emphasis is mine). IMHO editors like this don't help. By simply ignoring any evidence presented, they establish bizarre and insurmountable standards that no person could ever surmount -- least of all newbies. How many notable references does the page need? One? Six? What exactly does "significant" mean? Who knows? But an admin will be more than happy to berate a newbie and send them links to hundreds of pages of guidelines, rule and policies to keep the newbie busy and out of their hair.

Delete Fails WP:WEB for lack of significant (or pretty much any) coverage in reliable third-party sites. Of the nine references provided in the article, five are to Old Man Murray itself, and the others are to blogs (and even they do not provide "significant" coverage, just mentions). Google News Archive finds a few passing mentions in Wired Magazine, otherwise nothing. --MelanieN (talk) 15:42, 22 February 2011 (UTC)

Is typing a word into google archive honestly a standard you want to use for deleting articles? Worm4Real (talk) 22:14, 26 February 2011 (UTC)

• Bottom line, yes. It's the standard we have here, for lack of a better. It doesn't have to be the news archive, it can be Google Books or other search engines. Or it can be reliable sources not found in any of those places, but cited in the article. Currently there aren't any. I did my best to find what Wikipedia requires, which is significant coverage by independent reliable sources, before I !voted "delete". Your numerous comments here have done nothing so far to change my mind; in fact you would change more minds if you would keep your comments less hostile. You obviously feel very strongly about this site, but if you want to keep its page on Wikipedia you simply have to come up with reliable sources to support your insistence that it is notable by Wikipedia's definition. If there are no such sources, then the article can't stay here, no matter how strongly you and your friends feel about it. --MelanieN (talk) 00:40, 27 February 2011 (UTC)

Honestly I don't even feel strongly about the site, it just kind of amazes me to see people do a few no effort web searches and declare that they could find no references to the website, then to see those people ignore references when presented with them. The only complaint presented with anything anyone not from wikipedia has said is that they're not from wikipedia. Worm4Real (talk) 01:43, 27 February 2011 (UTC)

4) Deleting things should be hard -- very very hard. It's not. It's very very easy. Yes I definitely agree with this. I've been contributing one or another for about 5 years now and I still find certain guidelines arcane. The jargon is forbidding and I have little understanding of all the template thingummies. Ultimately an encyclopedia seeks to encompass all knowledge, including what might seem irrelevant or trivial to others. There is no doubt a worthy desire for Wikipedia not to be a 2nd Guiness Book of Records, but on the other hand Wikipedia is not constrained by physical limits, so there is very little cause to limit the number of articles. On the other article themselves should be created with the user in mind. A short introduction to the subject is desirable, but again there is no reason to limit the length of the disquisition. Augusta2 00:43, 13 March 2011 (UTC) 00:38, 13 March 2011 (UTC)

I find the chart confusing--plotting quantity against percentages is not easy to interpret.

What I see is normal growth over time. I would expect the absolute number of editors to rise, and the retention rate to decrease.

By it's nature, editing is a 'as I have time' thing, and it competes with other activities.

I'd like to know retention numbers, and how retention is measured--since it is possible for someone to be very active in bursts, which can leave gaps in participation of over 1 month.

Making a WYSIG editor will be useful--learning how to use the editing system has a moderate learning curve which undoubtedly prevents some participation.

The ratio appears to be asymptotic.

I don't think anyone is saying that the trends are particularly unnatural in the evolution in the project. It's just that we're seeing negative effects from these trends, whatever their origin, and that we think the Foundation and community at large can do things about it to reverse the trend and be more open again.

The red line in the chart represents percentage of 1-year retention. The red line should be as close to a straight horizontal line as possible. That would mean that overtime, the same percentage of people stay for a year or longer in WP. What you see in the chart is that over time stay at WP for a year or more.

The blue line is telling us how many active editors there are in WP, showing a sharp increase during the first half of the evaluated period and a slow decrease trend during the second period. While the total number of active editors in WP has increased over time, starting in 2007 this number has started to decline slowly. But most importantly, this line is here to give us context. It shows that while total number or active editors increased, the percentage of them staying for at least a year has decreased. That means that if we would draw a green line showing the total number of editors that leave WP, i.e. that become inactive editors, that line would be much, much higher than the blue one.

In other words, on March 2007, when WP had the highest number of active editors (around fifty-five thousand) and was keeping 15% of them for a year or longer, WP had lost eighty-five thousand editors. What this is telling us is that there is a core of fifteen thousand to ten thousand active editors that stay in WP, while all new comers leave after a year. In other words the destiny of WP is at the hands of 10% of the editors. The rest just leave.

To answer one of your specific questions:

• As by the chart, I interpret 1-year retention as a user that stops making contributions after one year. So if he would contribute today, and then again in 2 years, he would not be in this category.

Asinthior 15:52, 17 May 2011 (UTC)

This is a great discussion with many excellent ideas. Perhaps the creator could create a summary. It seems to me that the strongest ideas here are about making the deletion process more difficult to initiate; and imposing some sort of penalty on deletions that fail; and marking editors who delete more than they create. I also like building a "deletion proof" form for readers to check if they find the article truly worthwhile.