Looking ahead to recommendations: Ask Wikimedia to commit to more data collection

Jump to navigation Jump to search

Looking ahead to recommendations: Ask Wikimedia to commit to more data collection

I thought as part of our final submission to the WMF we might also add something that's a bit out of (or above? meta?) our immediate Task Force remit.

I would submit to them that they should implement a program of tracking user data much more closely. I think this would greatly help any future strategy work and could also help the projects.

For example, if Wikimedia had data on the top 100 reverters, page (speedy) deleters, people who quote certain policies most often, that might lead us to a number of users who could do with some "be nice" education. Data on what percentage a user's contribs are on talk pages, articles, include citations, and any number of a vast array of metrics could be very handy. If we could draw up league tables I think users could be targeted for any number of behaviour-changing messages. It would also enable the system to classify users according to their habits.

Similarly, if Wikimedia itself compiled more data on users (as opposed to the situation now, where this task force relies on some snapshots and samples), we would be in a better position to work on strategy.

So my suggestion is that our Task Force adds a section on our final submission that calls for a commitment to data collection and use to benefit future strategy processes. It wouldn't have to be a detailed request, it could be just a very broad recommendation of a couple of lines without specifying any particular data to be collected. What do you think?

Bodnotbod18:34, 23 November 2009

I think this is useful. Wikipedia needs more performance indicators. Not that they need to be publicly available to everyone at all times. But this task force is exactly the kind of time when you would want these kinds of stats. I'd prefer them to be more anonymous... but the ideas are there: the number of reverts, the activity at ArbCom, the activity at admin noticeboards, and so on...

Randomran18:59, 23 November 2009

Well, anon stats would certainly be a start :o)

It does leave the question of what we do about "being nice". I have a feeling that whatever recommendations we (or other TFs) make about reducing community hostility is going to have to be carried out by editors on the projects. That's to say strategies/policies can help with this, but it will probably have to be seen through by identifying instances of unfriendliness and influencing individual users too. So that makes me think that users who have a harsh approach should be identifiable in the data so that they can be offered a friendly word to moderate themselves.

Bodnotbod19:58, 23 November 2009

That's an interesting idea. But before we ask what do to about hostility, I think we also need to ask why there is so much more hostility now. What is the cause of the hostility? It's possible that it's an editor-by-editor problem, but I get the feeling it is much more systemic than that.

Randomran20:21, 23 November 2009

If I were to go on my own personal experience I would say one factor is "so much to do, so little time". I suspect I've been guilty of not being so nice to people as I could have been because I have a set of tasks in mind and I want to complete them. An example of a task might be "review all edits since the last time I looked at my watchlist". If I see that that will take some time and I come across a rogue edit, I'm far more likely to revert (or other action) and leave a terse, short-hand message than give any thought to the impact on the person who made the edit.

To be nice takes considerably more time than it takes just to "get things done". I hope I am rarely uncivil. But I suspect I could be taken as brusque, curt, abrupt and it's all about racing to get stuff done in my case.

However, that's just a personal report from me, sample size one, so whether this accounts for a useful proportion of hostility is unknown.

Bodnotbod20:45, 23 November 2009

That actually describes me too. "Too busy getting things done to be nice." Not that I'm nasty, but I don't think "hey, this guy could be a newbie, and might need some help learning things." If so, then the solution is helping people to manage their time, and showing people that integrating newbies needs to be a priority.

Another reason for hostility, IMO, is that the project has just gotten bigger. It's the difference between a small town and New York City. In a small town, everyone kind of shares the same values, and knows each other, so they're not likely to have long and drawn out hostilities. They have disputes, but they can settle them themselves. But New York City is so big, and diverse, and anonymous, so people have more disagreements, and don't feel the need to be nice to someone they'll probably never see again. That's why NYC has a whole bunch of different community programs to handle its growth: groups for immigrants, workplace groups to welcome new employees, orientation leaders to integrate new students. The small town is its own community group. In NYC, you have to create hundreds.

... I just had that insight right now. I don't like to pat myself on the back, but I really like the analogy. :) I actually think it reinforces what you were saying about too little time. Randomran 21:06, 23 November 2009 (UTC)

Randomran21:06, 23 November 2009

< nods >

One of my other lines of thinking to improve "community health" was to empower and publicise Wikiprojects. If people on a Wikiproject take newbies who edit "their" articles under their wing perhaps this would lead to a culture of people knowing each other more. So to take your analogy of The Big City, this would give Wikipedia distinct neighbourhoods where people could have more of a sense of community.

I also thought that perhaps there could be a "Wikiprojects Charter" that members of Wikiprojects sign up to whenever they join a Wikiproject. So it could have, say, ten statements the user implicitly or explicitly agrees to and one of those points could be "I will welcome and encourage new editors".

One way to empower Wikiprojects would be to give them better communication tools. This is where the idea of social features could come into play. Perhaps Wikiprojects could have a more dynamic hub showing activity on articles that fall within the project, tools to communicate with all project members, ways of seeing what members are doing.

I guess the downsides are potential Wikiproject Wars where a sense of being on a "side" is fostered and the groups start to lock horns where there is an overlap on articles. For example the Wikiproject for a particular nation might run into conflict with Wikiproject military history because a patriotic view of a battle runs counter to a Western historian view.

Bodnotbod15:52, 24 November 2009

Since my best experience on Wikipedia was because someone plopped a WikiProject box on my user page (and invited me to remove it if I didn't want it, or join in if I appreciated it), I'm a huge fan of WikiProjects. I probably wouldn't have made the transition from new user to frequent user without a WikiProject. It's easy to feel lost in a huge city. But if you know the people in your neighborhood, or even just in your building, you have that support. You have people who can help you learn the ropes, settle disputes, discuss standards, help you with articles, and even give you a pat on the back now and then.

The problem is what you said, where you get into group wars. I can't say I've had too many bad experiences when WikiProjects collide, because usually they care a common commitment to basic premises like verify things and don't push a point of view. (POV problems are sometimes challenging, at least projects agree in principal to be neutral.) The problem is with less formal groups. Groups who pour in from off-wiki like a website or email list. Or groups without any headquarters who kind of track each other's contributions and maybe send the occasional email or user page message. These groups are not organized around improving content, but around pushing an agenda. Formal WikiProjects are usually trying to reach a consensus. But the informal groups are trying to win a debate -- and once they realize that Wikipedia does not have "winners", they dedicate themselves to filibustering the debate, making Wikipedia a more hostile and unproductive place.

A lot of this is leading us back to a good idea: social features. But it also leads us back to the hostility problem. Even with the primitive social features that Wikipedia already has, it's possible to coordinate drama and battles. We need to reconcile the contradictory goals of trying to make it easier to coordinate and help one another, while simultaneously making it harder for people to form gangs and mobs. That means having rules and guidelines where good coordination is allowed and promoted, but bad coordination is put to an end -- without sending the bad apples to pursue their crusade "underground".

This is a tough challenge. But if we pull it off, it will improve community health exponentially. I think we've found the heart of our challenge.

Randomran22:42, 24 November 2009

Ah, it's gratifying to hear that about Wikiprojects. I personally have had no involvement with them apart from sensing their presence from the boxes you see on talk pages. It's good to hear that at least one of them is doing some kind of outreach by placing boxes on user pages. Perhaps we could consider some way of making it easier for Wikiprojects to issue some kind of invite.

I have also seen FloNight suggesting elsewhere that we build on what is already on Wikimedia, which would make Wikiprojects a good target for recommendations.

All that said, I keep having to remind myself every other day "this is not just about the Wikipedias"... so I don't know whether WikiProject cultures exist on other Wikimedia projects. Similarly, FloNight has made me aware that there's already quite a culture of "contests" on en:Wikipedia which might make a good foundation for the "rewarding editors" theme, but whether contests exist on other projects I don't know either.

I'll write myself a To Do note to go around all the English projects and ask on the various Village Pumps and see if I can find out. I have been feeling a bit stuck as to what to do if I'm feeling a bit too tired to read the last 48 hours, so that will give me a meaningful task for tomorrow.

Bodnotbod02:31, 25 November 2009

Yeah, there's so much to read. Sometimes you have to take a break from reading dense reports, and sometimes it's good to just google around, or try to write down some ideas yourself. I'm sure that we can expand our ideas so they're relevant to all the Wikimedia projects, although I think our big community health challenge is really about figuring out why growth has stabilized and how to fix that.

I've spent a lot of time trying to find data that helps us drill down into the problem. But I'm almost positive that it's going to end up reinforcing the broad solutions you're working on. I'm hoping to shift my effort from data gathering to discussing the actual problems very soon.

Randomran04:03, 25 November 2009
 
 
 
Edited by another user.
Last edit: 04:00, 1 December 2009

I definitely think we should make it more visible to experienced editors when they are dealing with newbies. Especially, I think when someone registered creates their first article, it might make sense to automatically tag it with an assessment that points out it is a newbie editor's first article. Alternatively, when someone tries to save a deletion or notability tag on a first-time article or an article created by an editor still in their first 15 days, we could throw up a nudge in the preview, sort of like the warning when a reference is inline without a reference/ tag, or give an error like you can set it to give if you don't provide a comment. Something that reminds the editor not to bite the newbies - you could include a check to see if they (or anyone) posted a welcome tag on the editor's page or other guidance before nominating their work for possible deletion.

The challenge is that we don't want to make editing harder for people doing the good work/slog of deleting stupid vandalism, but we want to encourage people to be nice to well-meaning newbies. If a skeleton article is created, for instance, we might suggest an editor give it at least 10 minutes and a note to the creator before promoting it for speedy deletion due to lack of notability - lack of content in a first draft and lack of notability are not the same thing. hmm. We might alternatively prompt a newbie user to tag a draft - is there an {{earlydraft]] tag that indicates a user is actively working on an article and asks that it not be evaluated yet?

Polite/nice process cannot be enforced, but it can be encouraged, and we can create infrastructure that helps that.

--Sorry, this was me. I didn't realize I wasn't logged in. Netmouse 04:00, 1 December 2009 (UTC)

99.139.137.17503:26, 1 December 2009
 
 

Personally, I think the main driver of hostility was the skyrocketing popularity of Wikipedia between 2003 and 2005. I think that prior to 2003, essentially the Wikipedia editing community was puttering away doing a good job writing good articles, mostly pretty much ignored by the outside world. Then in 2003-ish, readership skyrocketed. That resulted in:

  • A huge influx of vandals;
  • A huge influx of POV-pushers and self-promoters;
  • A huge influx of new editors who wanted to help but didn't know how;
  • A ton of fairly stupid and ill-informed media coverage, much of it making fun of Wikipedia, or hostile to it;
  • A huge wave of interest in commercializing/monetizing Wikipedia (business proposals, etc., many of them clueless)

I think it's hard to overstate the importance of that enormous surge in popularity, and how influential it was in shaping Wikimedia community attitudes and behaviours. I wasn't editing Wikipedia at the time, but I've talked to, and observed, lots of people who were, and I think I can imagine what it was like for them. I think they felt beleagured, like an enormous tidal wave of input was crashing over them, much of it threatening Wikipedia's integrity and quality, and there weren't enough of them to successfully manage it. And I think they felt a heightened sense of personal accountability: Wikipedia was starting to be more influential, to matter more. Which I think made editors even more worried about maintaining its quality.

I believe that sudden, unprecedented surge in popularity created an atmosphere of insularity and suspicion of outsiders that continues to this day. And we are still beleagured: the problem hasn't gone away -- Bodnotbod and Randomran attribute their own brusqueness/curtness/abruptness to being too busy, which I think is very true for many editors. It's a chicken and egg problem: if we are too busy to be welcoming, we will never get the help we need to become less busy. So, we need to upwards-prioritize bringing in new people -- we need to make an investment in friendliness :-)

I'll say too that there are obviously other factors at play here, in addition to the popularity surge. Super-fast: the Wikimedia projects are edited mainly by geeks, and geeks aren't known for being particularly warm and welcoming. Online culture in general tends to be sparky, anonymity drives sparkiness, and text-based communications tend generally to lack warmth. Communicating across languages sometimes, I think, leads people to overstate their points for clarity. We have no really good mechanisms for sanctioning / kicking out jerks. And the people who edit Wikimedia projects are super-smart; I believe a certain amount of hostility isn't intended as such, but is experienced by the participants as pleasurable intellectual debate. Plus, editors are mostly male, and men are not as socialized as women to be sensitive to social cues, and to care about social harmony.

So, there are lots of contributing factors. Most of which I think can be alleviated by broadening the contributor base.

Sue Gardner21:32, 26 November 2009
 
 
 

We need to look and see if another group is already making this recommendation. It is broad and something that most task forces might include. There might be a different task force that has a more direct task of making recommendation about data collection. I'll look and see.

FloNight♥♥♥18:37, 24 November 2009

That's a good point. If we're supposed to identify 2-4 high leverage areas, we may not want to waste our breath on what's already out there. But that said, we don't want something this important to fall through the cracks, and we'll want to throw our weight behind any other task force that thinks this is a good idea.

Randomran22:17, 24 November 2009
 

Most strongly agree. I've been arguing for a global Wikimedia survey since 2005; we finally got one in 2008 and that's it :( And even the data from it, as far as I know, is still not available to other researchers (despite some early promises). We need a regular set of yearly surveys, with data available to interested researches and a way for the community to propose questions to be asked, and a way to do other, more specific surveys when need arises (like Proposal:Survey Former users). --Piotrus 20:14, 25 November 2009 (UTC)

Piotrus20:14, 25 November 2009