Finding and interviewing ex-editors

That's a great idea! I know the holidays can be too busy to make time. But let me know what your schedule is. I'm free except on the 24th and 25th. Maybe tonight or tomorrow, if you get this in time? Let me know.

IRC would be good.

Sorry, I just this now. Are you available sometime later today? I will be in the strategy IRC channel to chat. This shouldn't take more than 10 minutes.

Thanks!

hey, need to do some christmas shopping after work. but I'm popping in now, if you're around.

Here is a draft of the online survey. There are a number of kinks to be worked out, but this should give folks a feel for what the survey will look like.

Please let me know if you have any final suggestions. We're hoping to send the first set of emails by end of Tuesday.

Three form issues:

• Question 3: The number of edits (at the very beginning) -- 100 to 999 is pretty broad. We could afford to break this up into two ranges, if not three. (e.g.: 100 to 199, 200 to 499, 500 to 999)
• Question 5, 8, 8a: "Check any that apply". It's really important that we get people to pick their top three, instead of just "any". I know some people will click more, but we want to avoid people clicking everything. There's a good chance that they all apply, to some degree.
• Question 5 AND 8a: We need an "other", just in case.
• ... could we throw in an optional "tell us your username" thing at the end? Could be useful for follow up.

A few substantive issues:

• Question 6: This question about "was there anything that needed correction" is too obvious. Almost everyone will say YES. It would be more useful to phrase it in a way that emphasizes that this was how much they personally wanted to contribute. Of course anyone could find something else to do. But was it something that they thought was important? Something they thought that they had a special set of skills that made them really suitable to fix it?
• Question 8: Re: complexity, we should flat out test a null hypothesis: writing a good article is hard work, and maybe people leave when they realize it's more than they can handle. This is distinct from maintaining articles. "Yes, writing an encyclopedic article is difficult and time consuming."
• Question 9: on the agree/disagree questions... I think the very first one is already addressed in the earlier question about why they left, and later in the agree/disagree form when we ask them if they would come back when their personal life permits it. I'd just drop it and shorten it. But if there's room, I'd also like to know if people would agree/disagree that "the community is too lenient on disruptive editors". Just a thought.

You also have some numbering issues. But I used the numbers I saw in the current survey.

All in all it's very good though. We did a great job, IMO.

So there are a few issues that I haven't gotten around to with Limesurvey (our survey tool):

• Numbering -- I just need to get around to this.
• "Check all that apply" -- I don't remember if I've mentioned this before, but this is the default text that Limesurvey uses for multiple choice options is "Check all that apply." I haven't found an easy way to suppress this, but I will do a little more digging tonight.
• Regarding the "Other" option -- yes, I'll need to make sure we have these in the final version where it's appropriate. There is another issue, however, with Limewire which affects the questions with multiple "yes/no" answers. Limesurvey defaults "other + <please specify free text>" as the last option and I don't think there is a way to change this. So for Yes/No questions, the answer choices may be "Yes 1, Yes 2, Yes 3. . .No, Other (please specify). . ." where what we really want is "Yes 1, Yes 2, Yes 3, Yes, Other (please specify), No." I'm not sure there's way around this with Limewire. The only way (I think) we can have "other" grouped with the "yes" answers is if the "other" choice doesn't have the "please specify" field. (Sorry if this wasn't the best explanation).

In response to your other points:

• Q3: I know the response buckets are a bit broad, but we're trying to keep this consistent with Ed Chi's (researcher from PARC) buckets.
• Q6: This question has been a tricky one to get right. I see your point -- the question is phrased as a yes/no. What if we changed the phrasing to ". . .how much missing or incorrect content. . ."? This doesn't help us unearth the personal motivation aspect, but Q5 (Why did you stop contributing to Wikipedia?) should help us with that.
• Q8: Agreed. I think the phrasing you used above is really good -- eliminates possibility of a user selecting this choice because they had difficulties with the interface. I may be splitting hairs, but the subtle change is a good one.

Thanks again for the thoughts. The changes should be up on the wiki page in an hour or so.

Howie

Hey Howie,

I tweaked some wording for clarity. I also added an answer about feedback and appreciation (focusing specifically on the "reward" hypothesis) as a criticism of the community.

I also took a crack at question four, so it wasn't such a no-brainer. Everyone will say that Wikipedia has a lot of missing and incorrect content. Maybe if we focus on "important" content we'll get a more interesting answer. (I'm still not fully happy with this question. I'd really like to find out if we lost an editor who could have been really helpful, or an editor that had more or less hit their peak. Not sure how to phrase it.)

Everything else looks pretty much solid though.

Getting the "check the top three answers" is key, though. Very important. If we have to slip "select three" into the actual question phrasing, then do it, even if it looks sloppy.