open-ended questions - a gold mine of data

The survey was designed with several open-ended questions that permitted editors to express themselves freely. This will be hard to quantify. But I suggest two things:

• We have some person or group go through the actual surveys, and figure out a way to get a representative description of what people were actually talking about.
• We use some kind of information parsing tool like this one to look for commonly used words. (Or phraselets, ideally.)

It will be important to parse the information first, though. Don't just want to jump in and read all the surveys. Want to look at editors who gave specific answers, to understand what those answers meant. Want to look at editors who had a specific number of edits, to see how new users and experienced users differed.

Yep, we're gathering those as well. We have to be careful how we use those, because we promised that they would be shared in the aggregate only, so figuring out how to handle that will be important, in order to keep our commitment to the respondents.

That's not too hard. Just pick a constant (e.g.: say, all the editors with more than 1000 edits, or all the editors who thought that complexity was a factor in why they left) and pick a random sample of 20 surveys. From those 20 surveys, aggregate their answers about their best/worst experience, and the "miscellaneous question" at the end. From there, we could easily look for common themes, as well as significant differences.