Talk:Task force/Community Health/Former administrators survey
|Thread title||Replies||Last modified|
|Status update?||5||00:40, 17 September 2010|
|Who are we targetting?||2||08:24, 22 July 2010|
|How do we measure how much of the downtrend is self-perpetuating?||3||23:02, 20 July 2010|
|The survey is finished: final comments due||21||22:09, 20 July 2010|
|Separate health issues?||18||22:06, 27 September 2010|
|my last minute changes||12||01:27, 24 June 2010|
|Questions to Drop||10||00:58, 19 June 2010|
|Status?||3||16:45, 15 June 2010|
|Time||2||06:28, 13 June 2010|
|Other languages||15||19:52, 12 June 2010|
|Question 12||0||14:23, 7 June 2010|
|"Inactive" is better than "Former"||3||21:17, 29 May 2010|
|Questions||5||07:07, 21 May 2010|
Last edit: 19:56, 19 August 2010
It's been two months since this thing was declared finished. So has it gone out yet? Who's responsible for tabulating the responses?
Me either. We may have been too late and gotten lost as they wound the whole strategy project down.
Blink, Blink, Blink.
I wonder if anything palatable come out from all this.
I will only feel satisfied when we start discussion on "Implementation" in respective wiki.
Sorry for the lack of update.
The plan is still to do this, but the issue on our end is resourcing: right now, all available resources are fully deployed. As we're building up our staffing model, which is in the very near term, I think we'll be able to re-tackle this, but I've commandeered most of the Community staff for the fundraiser.
Sorry. I'll not let it slide, but feel free to poke me.
Investigating the least sustainable measure of the largest project is in the same priority leagues as fundraising, isn't it? Having a viable theory of the causes of admin attrition is certainly necessary for optimizing the choice of solutions. Having said that, I would certainly prefer the version of the survey which measures more independent variables than fewer.
Currently this just talks of inactives and those who've hoisted a retired template and stopped editing. I'm rather hoping that we are going to exclude:
- Blocked and or banned users
- former admins on the list of admins who can only be re-adminned by Arbcom or a fresh RFA
- Known duplicates
- Former staff members who had the admin bit as part of their work account
And I hope we include:
- Former admins who could have the bit back on request and are not still editing.
- Former admins who could have the bit back on request and are still editing. However we might need a slight alteration to the wording for this group.
Its also worth studying change over time to get a handle on the likelihood of people coming back, a quick glance at http://en.wikipedia.org/w/index.php?title=Wikipedia:List_of_administrators/P-Z&diff=prev&oldid=372869052 shows that the pattern is far more complex than a simple 1% retirements per month, instead we have people flitting in and out of activity and returning from breaks as long as 3 years.
Point of clarification... I'm not sure there ARE any former staff members who had the admin bit as part of their work account. Maybe I'm wrong on that though. :) Staff with a demonstrated need get staff rights, which are very different from the admin bit... but there might be someone from early on that I don't know about that got the admin bit. :)
I guess it depends on what you are measuring, and how you do this selection. http://en.wikipedia.org/wiki/Wikipedia:ADMINSTATS currently has 2015 accounts that have made at least one logged admin action. Whilst http://en.wikipedia.org/wiki/User:WereSpielChequers/RFA_by_month only covers 1913 admins appointed either by RFA or by Jimbo - some of the 102 difference will be bots, but I'm sure we haven't yet had 102 admin bots. Also I can remember one long departed staff member being desysoped because he'd only received the bit as an employee and he'd now long departed, so I suspect the staff rights level is from the last five years or so.
Of course if those individuals used Wikimedia email accounts for their user email then the point is moot.
Can anyone think of a way to measure how many administrators are leaving because the reduced ratio of admins to users makes it harder to be an admin?
I tried to address this. I can't think of anything more to add, other than the renaming part. I'll leave that to someone who can represent the Foundation in distributing the survey questionnaires.
I'm not sure yet that the declining numbers of active admins are making it harder to be an admin. I think there is a huge amount of slack, the number of things I get involved in because users come to my page and ask for an admin action is tiny compared to the number of admin actions I take because I have a long watchlist and go to places like category speedy deletions. If and when I take on something that involves little interaction with others I can go for many hours of editing without interactions.
I've no doubt there will be some tipping point where people hunt down admins who seem to be logged in in order to get someone blocked or a page protected. But at present we just see the occasional request at AN/I. My worry is that when we do reach that tipping point we could quickly have more admins giving up the mop or spending more time using non-admin alt accounts.
To all who have worked on or commented on the survey:
The former administrators survey is finished. In the next 48 hours please make any final comments. On Monday, if no reason to delay has been raised, we will turn it over to the WMF for any final editing, then proceed with distribution.
That's been done, actually. This survey was modeled on the former editors' survey, to see how former administrators responses vary.
That survey was actually for casual contributors (defined as users who had between 10-99 lifetime edits). I think it would be worthwhile doing a survey of active contributors (say, users who have between 100-999 lifetime edits, maybe more). I think a comparison of casual, active, and admins would be really interesting.
Noraft, would it be possible to squeeze in a question that asks if the person had admin tools or other higher level permissions (such as oversight or checkuser, crat) on multiple WMF projects? And if the person had other responsibilities (such as a member of M edition Committee or leader of a Wikiproject). My interest is to see if people are over committing by seeking too many tools/roles and then burning out.
Otherwise, it all looks good. :-)
As long as we can do it as a multiple choice (which I think we can), I don't think it would add too much to the respondents' answer load.
Randomran, do you think you can squeeze in a question or an answer that gets at the heart of FloNight's query?
Hey, I tried to give it a shot with the true and false question here:
- "I had administrator duties on multiple Wikimedia projects."
We can always tweak that question if we want to be broader about "i took on too much".
We also have a few other questions about whether admin duties caused them to spend more time on wikipedia overall, and whether it caused them to spend less time making articles.
Sorry to jump into the discussion late. I actually just found out about this effort earlier today. I think it's great that we're trying to learn more about the community through surveys.
I know I'm past the deadline, but I was wondering if folks think it would be helpful to include a question around how these Admins spent their time on Wikipedia. For example, something along the lines of "As an admin, how did you distribute your time on Wikipedia? a) Editing articles b) Moderating discussions, c) Working on Policy, d) Fighting Vandalism, etc."
The formulation of the questions and answers obviously needs work, I'm wondering what people think of the idea. There's currently nothing in the survey that helps us understand what activities these admins recall being engaged in, and whether certain types of activities appear to correlate with other measures taken in the survey.
I think it's an interesting question, but I'm worried that a lot of people would have trouble figuring out their time. There isn't really an average month. Some would know off hand, but a lot would just be guessing and going off of their gut feeling. (e.g.: "Yeah, it felt like I spent 90% of my time on these stupid disputes!") We already have a bunch of relative/subjective questions about whether they left because they spend too much time on vandalism, or settling disputes, or if they missed writing articles, etc.
If we want exact numbers, it might be better to just grab a bunch of admins and their contribution history, and then just break up the edits based on namespace (discussing policy in the WP space? editing in the article space? and so on). That said, if you think it's super important, it might not be too late to ask. But I think it would be hard to write a question (and hard to give a straight answer).
Yeah, I was struggling with how to word the question as well. I think capturing edits on the different namespaces would be a good idea. I think the data you get would be very interesting -- I think it would be very revealing to take a sample of admins and then break out their behavior along namespace. But this data doesn't give us insight into how admins perceive they're spending their time. it would be good to have both the qualitative and the quantitive.
One way we might be able to phrase the question is to have users select the top x (probably 3 or 5) areas they spent their time on. It's not perfect, but it helps with the issue you correctly raised about users not knowing off hand and guessing.
Part of the reason I'd like to get this data is that almost all (if not all) of the information we have about our admins is anecdotal. For example, we know there are admins that spend a lot of time on vandalism, policy, editing, etc., but we have no clue as to which of these areas is the biggest, most taxing for admins, etc. It would be great to start adding a little more color to this picture.
I know I'm coming in late, so it's really up to the team to decide whether to entertain this. I think it would be nice, but I don't want to derail things.
If we can find the right way to phrase it, it's only a question of adding in one more question. But keep in mind we already probe at it from a qualitative point of view:
- Why did you become an admin?
- I was fighting vandalism, and thought admin tools could help
- I was interested in deletion discussions, and thought admin tools could help
- I was assisting with disputes or incivility, and thought admin tools could help
- Did being an admin influence you to leave?
- Yes. Administrator duties pulled me away from editing and building articles.
- Yes. There was too much vandalism.
- Yes. Frequent requests from other editors were very demanding.
- Yes. There were too many conflicts and disputes.
- Please read the following statements and select all that you believe are TRUE:
- Administrator duties stopped me from spending significant time on building articles.
- After I became an admin, I spent significantly less time building articles
- After I became an admin, I spent significantly more time on Wikipedia overall
So we're kind of probing the qualitative "what did you spend your time on" stuff.
If you think we need a specific question for this -- pick the top three -- we can still add it. Or if you think we can get at it with a few more true/false questions, we can do that too. Or we can just leave it. I don't know what the foundation is doing with the survey yet, so we have time to change it if absolutely necessary.
Asking former administrators how they spent their time will not tell you how they spent their time; it will tell you how they remember spending their time, which statistics shows doesn't match up with reality. If you want to know how they spent their time, we should grab that from contribution histories.
One more comment on the survey questions. For the Former Contributors Survey, we didn't employ question logic. So users that selected "I haven't stopped contributing" to the question "Why did you stop contributing. . ." still got the question "Did X have an impact on your decision to stop contributing".
So for this survey, questions 9, 10, and 12 should ideally be re-worded if a respondent answers "I haven't stopped contributing" to question 4. We'll have to take a look at how Limesurvey can implement this logic. Ideally, only questions 9, 10, and 12 would change depending on the answer to question 4, but I'm not sure if Limesurvey can do that. For example, the Limesurvey may force us to fork the survey at that point. That may not be a bad thing since it could make it easier to analyze the responses of those who consider themselves as having left vs. those who haven't. On the other hand, analyzing the aggregate data may require a little more work.
Are there any reasons that a separate health issues survey would not be more prone to self-selection and other biases than a couple of unobtrusive general health questions with different default answers unified in this survey?
What are the specific other merits to a seperate health issues survey, if any?
Does anyone believe that a seperate health issues survey would be seen as more respectful?
Please answer these questions before finalizing this survey.
The survey is complete, was finalized a week ago, and has already been turned over to WMF for distribution. However, I will answer the questions anyway:
1) No, there are no "reasons that a separate health issues survey would not be more prone to self-selection and other biases than a couple of unobtrusive general health questions with different default answers unified in this survey." But we're not interested in including them for the previously mentioned reasons. 2) I don't see much merit to doing one, period, whether separately or unified. I think a separate health issues survey is unnecessary and not in line with current research goals of volunteers working on the admin survey. 3) I think respect doesn't hinge on whether or not it is separate. Anything can be written respectfully or not. But this is moot for the reasons given above.
Which are the "previously mentioned reasons" which have taken priority over avoiding bias?
What is your opinion on reverting the disputed thread summary without comments?
The idea of including a question about exercise is supported by the peer reviewed secondary literature, photographic evidence, statistical evidence, good survey design, and respect for administrators as human beings.
I am happy to consider this the "Former" survey and leave it at that, trusting the Foundation will make a good decision, but I think it is wrong that people are summarizing my comments contrary to the facts.
There's just not a chance in hell that we lost 150 administrators in 2 years to the sudden onset of obesity, or obesity related illnesses. Obesity has been a problem since long before that. That's the causal link that's missing for your hypothesis. Whether you're a statistician or not, you should already know that a theory with no causal link is junk science. It's absurd that Americans can be suffering from obesity for decades and suddenly all drop dead in the span of two years. It's so absurd that I'm convinced that this is a joke gone horribly bad and you're enjoying wasting everyone's time. You haven't convinced the few people working on this survey. And now that you've drawn more attention to it, you've only gained more detractors.
If it makes you feel any better, let's pretend you're right that the "community health" metaphor should be taken literally, and huge portions of the community coincidentally started suffering from physical ailments in 2007. We still have questions in the survey that allows contributors to say that "I stopped contributing because of something that happened in my life: it had nothing much to do with Wikipedia." We also have a series of open ended questions that allow contributors to tell us why they left, which will allow them to talk about health problems. In the bizarre event that you're right, it's not like we've taken away the chance for them to tell us, assuming they are able to get out of bed.
At this stage, we are trying to isolate contributing factors. Nobody has claimed that there is any one solitary cause. If you admit obesity has long been a problem, why do you want to omit survey questions concerning exercise?
Your sarcasm and hyperbole isn't very professional, but it's better than repeatedly reverting without comment as you had been.
There are always biases in every survey, and there are sources of error in every scientific experiment. That's why people use control are comparator groups. For example, in the former contributors survey conveniently worked out where half the respondents said they had good experiences at Wikipedia and just didn't have time to keep editing, with the other half talking about specific bad experiences that at least partially motivated them to leave. We learned more comparing these two groups than just using the top line numbers.
See the diagram on the right. Your theory of admin flight has no merit, respectfully.
As a statistician, I urge you to refrain from attempting to exclude legitimate independent variables while claiming that you have some means of controlling for sources of error after having done so. There is no such means of control available when independent variables are completely excluded from a poll.
Nobody has yet put forth a single reason, legitimate or otherwise, that health is not a factor in admin attrition, in the face of obesity being in the top two preventable causes of death and abundant photographic evidence that meetup attendees are above average weight. No amount of reversion will make up for shoddy statistical workmanship.
Just to say that you are too late and that the survey was already turned to the WMF.
And yes thank to your crappy argumentation, i doubt you will convince the Foundation to include you question as a last minute change.
It happens that some persons from the Foundation do read our discussion threads and see by themselves how much your argumentation suck.
Note: We can leave your "historical" content as it is "historical" because the Foundation will not fellow suit.
As I can see that the opposition on this issue has nothing left than a willingness to revert solid evidence without comment and resort to ad hominem attacks, I have asked Philippe Beaudette, who claims that he has "not insignificant" experience with survey design, to a public debate on this topic.
I am closely familiar with the applied statistical mathematics inherent in survey design, and the peer reviewed secondary medical literature on the topic and I look forward to seeing them upheld.
Sorry for being so absent lately. Life has been busy.
I made a lot of changes at the end. Let me try to summarize them:
- I tried to make the survey as consistent as possible with the other one, where relevant. Comparability is good.
- (The only exception is the difficulty of the work stuff. The emphasis here is now more on the difficulty of the admin work.)
- I tried to shorten the survey by putting more stuff in the "true/false" category. One direct question is often more powerful than a pile of questions.
- I expanded the questions around administrators, and tried to find neutral language that isn't too leading.
Having worked a lot with the other survey, I think this will get us to the right place. But if there's anything that seemed like a bad call, let me know and we can talk it out.
Q1 "roughly how many edits" (edits, or edits+actions? Some users may do hundreds of actions and few edits)
I looked at that, and replaced it with a quick true or false question:
- After I became an admin, I spent significantly less time building articles (True/False)
I don't feel strongly about this one. I was just trying to keep the survey as simple as possible. I'm not sure admins would be able to tell us with any accuracy how their edits are split, which is why I thought it would be more direct to just ask "so, admin took you away from writing articles, right?"
Do I understand the goal of this question correctly?
Inevitably having extra areas to work in reduces time in existing areas, all other things being equal.
Add to it, the simple question "Was that a good or bad thing?" or some such.
I like the changes. Just one quibble. The following two answers were removed from item 10:
- Yes. I was bothered by the number of editors requesting administrative actions from me.
- Yes. I had trouble with editors dissatisfied with administrative actions I took (such as blocks).
And I assume they were intended to be accounted for by
- Administrator duties were too much work.
However, as an administrator is a "free agent" working on the areas of Wikipedia he or she chooses to, they might not consider it a "duty" to respond to editor requests. Also, the second answer (trouble with editors dissatisfied with administrative actions) doesn't seem to be accounted for by anything, unless I missed something.
There's a response in 10...
- Yes. Frequent requests from other editors were very demanding.
I took the other one out, just because I was trying to keep things short. I also think we hit the "complainers" issue with the responses "Yes. Problems with the community made it difficult to be an administrator", and "Yes. I felt that I was often working alone, with little feedback or support."
It wouldn't be terrible to slip it back in though. We could even change the wording of one of those responses (problems with the community, or working alone) to include it.
It depends on if you think the "complainers" issue is a different enough problem that it warrants pulling it out as separate.
Maybe its just a conceptual thing for me. "Frequent requests from other editors were very demanding" has a little ambiguity: were the frequent requests demanding, or were the editors making the requests demanding. Also it doesn't have that gut level trigger for me. "I was bothered" does. Its less diplomatic, more how people relate to the world. When someone is pissing me off, I don't think "He's very demanding," what pervades is a feeling of being disturbed/bothered/frustrated. Experience tells me you'll get a more accurate response set using "feeling words" for feeling issues. If the coach yells at you for playing poorly, and you quit the team, and I ask you "Why'd you quit the team?" are you more likely to say "The coach was very demanding," or "The coach made me angry"? Feelings are more salient, in my opinion.
Anyway, whichever way you choose to go on it, after this issue is dispensed with, I'm ready to hand it off to WMF.
I see your point. On surveys, people might have different reactions to different things. Someone might look at "the requests bothered me" and say "well, no, I wasn't bothered. but it was just too much work." Someone else might look at "too many requests" and say "well, no, there weren't too many... but they were just annoying and i didn't want to do them." It's tricky to balance between something general enough that a lot of people would consider checking it off, but not so general that literally every person would check it off. One thing I try to avoid is telling people how they feel (e.g.: X was making me angry/tired/frustrated/annoyed), but other than that it's more art than science.
It's a pretty quick change if you think it's an important difference.
You've reminded me to go back to basics. The way we used to do it when I was doing survey research was to stay general: "I experienced negative feelings such as anger, irritation, or anxiety as a result of..." Anyway, I understand the survey has departed, so I'm sure we're in good hands with the Foundation.
I turned a critical eye on the survey, and to get it down to a more "completable" length, I propose dropping the following four questions. The longer the survey takes to complete, the less people will complete it. Given that completion rates and survey length are inversely proportional, any arguments as to why these questions should be kept should justify why the question is so important we're willing to accept a lower response rate to include it. Thanks!
7. What were your most rewarding experiences with Wikipedia? Why were they rewarding?
8. What were your worst experiences with Wikipedia? (Remember, all answers will be kept anonymous and no action will be taken. Don't mention names if you prefer not to. This is for survey purposes only.)
14. Looking back how do you see your time as an administrator and Wikimedian?
15. Is there anything you would have done differently as an administrator? Or wanted others to do differently?
I would definitely drop the last two. But the best/worst experience questions were some of the most insightful questions about what was causing users to leave. I think a small tweak could turn them into the most insightful questions for administrators: "what was your most rewarding experience as an administrator" and "what was your worst experience as an administrator"?
I've been busy but I promise to take a look at the survey and look at questions that can be set aside sometime this week.
If Q7 and Q8 are important, after cutting Q14 and Q15, I don't think there's much more we can trim. I propose we move forward with the remaining questions.
I'm really still very uncomfortable with the answers around exercise. The information gained from those will not be significant: it'll be a red herring without other physical data to back them up. Every bit of experience that I have in survey design (which is not insignificant) says those are bad statements that will give nothing but inconclusive data and should be removed.
Sorry, I meant to pull them (having argued against them before), but forgot. My bad!
Why are the many uploaded photographs of people who come to meet-ups not sufficient for other physical data? As someone who could benefit, according to actuarial tables, to drop 30 pounds, I am in better shape than the average meet-up attendee, visually speaking.
What exactly do you mean by "a red herring"? I've heard and seen that phrase used too many times to mean, "I don't have any logical argument here, but I feel uncomfortable."
I think it is a shame to ignore this issue. If you feel it is so important to prevent an inquiry as to whether simple lethargy could be a problem in the administrator community, then would you at least do those of us who need to be in better shape the courtesy of saying why you feel uncomfortable about it?
Philippe, I'll take a crack at answering this question.
There are several reasons why questions about exercise are inappropriate for this survey.
1) The general thrust of the survey is why administrators leave. The multiple choice answers account for "I needed more exercise" with the "real life isssues" answer. 2) Determining which real life issues prompted them to leave is beyond the scope of this survey. 3) One or two possible answers related to exercise are insufficient to determine how/why health issues relate to attrition. The data garnered would not be enough to draw any useful conclusions. An inquiry related to health would require a survey all its own. 4) We are attempting to maintain a somewhat parallel structure with the former editors survey, which did not include exercise as a factor. (This is the least important reason of the four given)
That said, I also want to say that Philippe did not indicate that he thought it was "important to prevent an inquiry as to whether simple lethargy could be a problem in the administrator community." I venture a guess though, that he does think it is important to produce a survey that measures what it is designed to measure, is focused, and is short enough to garner the most respondents possible. If you think a health survey is important, get some consensus for it, some WMF buy-in, and write one.
What's the status of this survey? Let me know if you need some guidance on actually putting together the survey and distributing it.
I'm sorry, Eekim. I haven't been giving this survey my full attention. I promise to time-prioritize it higher. I expect to be done with it in about a week, and we definitely need WMF help distributing it.
As a former administrator of one of the wikis i tried to answer the questions. It took me 15-25 minutes. A bit long. Also, some of the answers indicate a not so clear purpose of this survey, just as if it was originally aimed at former editors, while later adapted to former admins.
Yes, I agree that the survey needs to be a little shorter. The former user survey involved 13 questions. 5 were basic biographic information (e.g.: when did you start?), and 8 questions were of any substance. Even then, a lot of people didn't finish.
We have to work within those limits. We'll get to it.
As a former admin, do you think we were asking the right questions? Why do you think administrators are leaving? (Is it the same reason you're leaving?)
Do you think this survey may be done in other languages than english?
I started translating the questions. How many translators do you need? May I change the questions in order to adapt the survey to my own community?
If possible, I would like to see the survey done in other languages for other WMF projects other than WP English. I don't think that we should make the assumption that all wikis are facing the same set of issues that are causing attrition of admin and contributors.
"I don't think that we should make the assumption that all wikis are facing the same set of issues that are causing attrition of admin and contributors."
- I think that there lots of issues in common and lots of issues that are community dependent and I think that it's important to identify this two sets. The issues in the first set must be addressed globally and the second set of issues must be addressed locally.
I think they will to be in the same questionnaire, but my point here is that we don't need to have the same questionnaire for all wikis.
Actually, that's going to be pretty important if we're going to be able to make any valid comparison between the different language projects. We won't be able to learn from each other's mistakes or successes. We're not going to be able to figure out what pressures make certain Wikipedias harder than others. It's just going to be a bunch of separate surveys. That would be big missed opportunity.
Please try to put yourself in the shoes of someone getting this survey. Wouldn't you rather be referred to as an "inactive" than a "former" administrator? I recommend renaming.
I don't know if you are planning to use Google Docs Forms to do this but if you want to those instructions are here.
In all seriousness and honesty, I am concerned about lethargy due to obesity as a potential cause of administrator attrition. Americans, and people from all developed countries face this as a real risk, with deaths due to obesity likely to eclipse those due to tobacco as the deadliest preventable cause of death, if they haven't already. Can anyone think of some way(s) to ask this, such as maybe a separate, completely anonymous survey offered through a separate short mnemonic URL included with the survey message to show that it's not going to be personally identifiable?
I think you're right about inactive rather than former. I'd have to see some consensus on the obesity thing before I considered it. If you're going to get into health, then you should really get into health and ask a lot of health and lifestyle related questions. That's a whole other line of research...
You don't need to ask if the guy was overweighted, just include the option "I left Wikipedia because I felt I need to do more outdoor activities."
I've added a couple questions particular to administrators. I don't have time to do it this moment, but questions five and nine should be modified to fit our target respondents (administrators) better.
I'm sending a link to the survey to the Wikipedia English Functionaries mailing list for comments and suggestions. I'm directing them to this talk page to add their thoughts.
A couple of questions here are unnecessary if they give a username. "how many edits did you make" 1/ they may not know, 2/ asks them to think more, 3/ they may get it very wrong, 4/ we have that data (and in more detail) in house. Ditto "How many times nominated, who nominated you, etc we have that data. Or mark them "if you have given your username then ignore this question".
Q4 Extra options "It seemed a natural progression to my existing involvement" and "Administrators can influence things for the better". Reword last one (which nobody will say "yes" to as worded!) as "I wanted to have more status or power within the community" which is the same thing but has a chance of some people deciding it applies to them.
Q5 Extra options "Natural progression as time moved on and my focus of attention changed online or in the real world" and "Casual interest that faded over time".
Q9 "Did the difficulty of the work"... this implied assumption work was difficult, they may not feel that's accurate.
Q10 better "Did any negative experience with the community..."
Thanks again for starting this up! I think that in order to figure out some good questions, we should actually try to form hypotheses about why they left (... basically, guess.) Once we have a list of potential reasons that they decided to leave, we can try to figure some good questions to get at them.
Reasons for leaving:
- They just lost interest, or other interests/demands came up that took them away. (Null hypothesis.)
- They got the tools, and felt they had nothing else left to accomplish, so they left. ("I won the game" hypothesis.)
- They burnt out from managing all the community conflicts. (We may need to flesh this out further, because that could mean a lot of things.)
- They burnt out due to policing vandalism.
... anything else you can think of?
- I know that some admin get tired of the constant requests that fill up their talk pages: Can you unblock me? Why did you take X action? Users A B and C are bothering me. Ad infinitum.
- Some of them "flame out," that is, get involved in a dispute where they feel either the community isn't backing them up (and they deserve it because of their service/reputation/whatever), or because they're tired of the unending incivility. We might think about taking a look at Wikipedia:List of administrators/Inactive and surveying some of their talk pages for clues about what went wrong.
- Others leave because of dissatisfaction with uneven enforcement of rules, opposing consensuses, etc.