Summary:Talk:Task force/Wikipedia Quality/Thoughts to date

Starting point

FT2's impressions at 26 November.

Points arising:

Categorizing the community - newcomers need guidance (wizards, hand-holding, won't know norms), experienced users are already involved so for this group we learn what demotivates and discourages and how to encourage uptake of other tasks. Featured content writers tend to work independently.
Consensus seeking needs improvement if it is to be viable long term.
Quality is not easy to metricate (beyond "substandard"), though ideas have been proposed. Metrics should mirror what we want people to pay attention to (tags, cite ratios, promotions to higher ratings, stability, time taken to resolve identified issues, etc). There isn't a way to capture reader concerns on articles and thereby focus editorial attention.

Questions and issues:

Highs or lows - should we emphasize raising fewer articles to featured level, or should we emphasize the need for all articles to quickly reach some kind of "reasonable/good" level? Does the world assess us more by featured content (clear successes) or poor content (clear failures)?
Effective community - we don't know how to make a large community operate consensus effectively.
Specific tasks - we have specific tasks where we need to get more input, but in a volunteer community they don't get enough attention.
Better guidance - we have specific demands for editors and increasingly high expectations, but need to better guide editors (the anarchic "go figure it yourself" is limiting).
Locking in good content - we haven't worked out how to "lock in" good content, or review lost content. So articles erode unless curated by users who actively watch for poor edits. The watching process is affected by OWNership/tendentiousness/attrition by POV warriors and user departure.

Shortlist for possible 2 - 4 recommendations:

Guiding and handholding newcomers: We need to guide newcomers better, and provide more for experienced users, not just "figure it yourself". For newcomers we need wizards, guidance, "question marks" you can click to see an explanatory popup of a key community term, automated user explanatory notes, etc
Recognition and enhancement for established users: A "Good Editor" standard, more insight into other wiki-specialisms, masterclasses, better help in disputes, etc. Migrating newcomers to established users is half the deal; keeping them enjoying their editing is the other half. We need to value them - and act like it, and show it.
Reader feedback: "Completing the loop" by allowing review and feedback and ways for editors to be notified where to pay attention.
Focus on the "low lying fruit": Focus on articles that are visibly substandard, as a starting point. Set defined quality baselines that articles should aim to reach within 1 - 3 months of creation, and metrics and information that specifically focus on that gap (measuring crudely via tags and clickable user feedback for issues). That's a useful standard for readers, much easier to measure, they're easy to fix, and this affects far more articles than any other likely targeted change. It's also easy to incentivize authors to fix such issues, and feeds through to GA/FA articles and editorship.
Key tasks under-attended: We need to solve the problem that key tasks get too little attention. Some kind of "task manager" feed where simple matters get added (missing cites for facts, bias queries etc) and which users and readers could filter based on their preferred areas, interests, or articles they are reading/watching. ("This article has requests for help that match your filters, do you want to read them?" may get a much better response than a mere "citation needed".)
Consensus mechanisms: Draw communities attention to the overriding importance of deciding to improve their own consensus-building and difference-resolving methods, as a way to empower themselves on quality.
Quality lock-in and erosion: We need to find a better way to "lock in" quality, against erosion, users who habitually watch articles but then leave, long term steady flow of poor/misinformed editors or deliberate edit warriors or people with long term agendas, etc. Flagged revisions is one way but a number of users object to it. How else?
Automated crude quality ratings: If users can see their article is rated at 3.2, and what's causing it to get that grade, they have a clearer incentive to get it to grade 4 or 5, and patrolers can focus on poor articles to improve the long tail at the bottom. It needs to just reflect basics that matter, as a spur to mediocre or poor articles and to inform and encourage their authors or readers. ("This article is only rated at 2.1 for quality. Click to see if you can help Wikipedia with any of these issues". Leveraging engagement is crucial)
Interwiki workflow: We need to improve the ability of knowledge to flow between wikis. Being able to see the quality ratings of the same article on other wikis (not just their existence) is one possible way.

Bottom line questions - What changes have the most significant effects long term? Which things most impede other changes?

General discussion

Randomran thinks it's a good summary but stepping back to the bigger questions will also help: "Why has quality not improved at the rate that we hope it would? What are the biggest barriers to quality on Wikipedia?"

He feels that teaching newcomers, improve consensus mechanisms and defining quality better are all valuable. But also "ask people why they think it's so hard to make Wikimedia projects into environment of quality. The recommendations will be much more persuasive (and effective) if they're matched with big problems."

Piotrus agrees with the suggested recommendations. He comments that in his view Wikipedia is judged predominantly by the places it fails or falls short, not by its highs (more below), but that consensus wouldn't be a problem if good faith and civility (conduct norms) were followed. He singles out "Recognition and enhancement for established users" and "Focus on the low lying fruit" as "very valid", but "Quality erosion" may need a closer look because sometimes it's the quality standards that change, not the editorial context.

Is Wikipedia judged more by its successes, or its failures?

Piotrus states that in his view Wikipedia is judged more by poor content and failures than by successes. "Many readers don't really distinguish between GA or FA; they are rather annoyed/confused by poorly written (or non-existent) articles". There is general consensus on this as a key point.

FT2 states that reducing clearly substandard items is a universally recognized core means of quality improvement:

"The wiki probably [has] 2 million articles we could easily get pile-on help to improve to a recognized and reasonable quality baseline with ease.

We have hundreds of thousands of members of the public who'd love us to make [it] so smooth they'd take to it as a fish to water, knowing when they needed guidance or ideas on improvement it was "just there".

We can make huge strides by addressing the easy but vast majority cases... [they are] very amenable to automation, they scale, and they encourage incremental improvement in other ways..."

He notes we would still need to support established users and address low quality ones, but GA/FA will flourish due to these other actions.

A baseline quality standard

Randomran feels what we may need (and is being described) is a "basic 'safe enough to eat' quality standard". He asks what this gets us and how we address articles that don't meet the baseline.

FT2 agrees, describing it as a communally agreed "baseline for quality". (Example criteria). What it gets us - Much easier to drive or promote basic improvements, educates new users who write them, hence likely to make good inroads, automatically, on a large scale, engaging the wider public, and relevant to what critics most notice; What about sub-baseline articles - they would be hit hard (next section)

How baseline standard can be attained

FT2 describes his view:

"We set up automated systems, "Help fix this!" buttons when someone views an article, feeds for individual substandard issues, "Fix a random issue" button, everything we darn can, and drive like hell that EVERYONE can help fix basics in articles, readers, people who've never used Wikipedia before. ANYONE.

"You can look up a citation, here's how!" ...
"You can check a fact, or if a statement/section is fairly tagged, here's how!" ...
"You just wrote an article, and I noticed some improvements that will help it stay on Wikipedia. Here are the top 2 items!" ...
"This article has requests for help that match your filters, do you want to read them?" ...
"This article is only rated at 2.1 for quality. Click to see if you can help Wikipedia with any of these issues" ...

We push like hell for it, using automated methods, to get this kind of work automatic. That's what we do."