Summary:Talk:Task force/Improve Wikipedia's Quality Task Force/A couple of conversation starters

From Strategic Planning
Opening

Philippe opened the taskforce asking: "What have we agreed on in terms of quality? Where is the community in terms of the quality discussion. What do we NOT agree on? What have we not discussed about quality, as a community? What sort of information would be useful, in terms of helping us think this through?"

Discussion initially moved to the Five Pillars, newcomer guidance, and the nature of reliable sourcing, before branching out into other threads.

The "Five Pillars" and newcomers

Woodwalker noted the Five Pillars of Wikipedia (Verifiable, Neutral, Balanced, Findable, Encyclopedic) were a sound basis but could be improved and better promoted, and it is highly demotivating when some users don't understand these requirements. FT2 asked "whether it's appropriate and fair - to newcomers and the project - to throw users unfamiliar with encyclopedia writing into the project, without clear guidance", noting we make high demands of their learning curve and standards. He had started writing a "newcomers manual".

Hillgentleman thought the 5 pillars were commonsense; Woodwalker felt they were not obvious to many users and that educating newcomers was a significant drain on time for experienced editors. He suggested that guidelines, 'newcomers manuals', etc should be more visible and easy to find, which in turn allows experienced users to assume they are known and treat such users "with less patience". Woodwalker also noted tools such as Flagged Revisions whose effect on quality was unknown and needed further examination, and suggested looking at "the sharing of all information and initiatives among projects" - while local communities made their own choices they should be aware of the choices and work of others (eg in manuals, tools, etc). Yaroslav Blanter added (later) that perhaps "completeness" should be considered a pillar.

Wizards

FT2 noted that an enwiki user has written a "new article wizard" (updated v2.0). He observed that other organizations seeking mass use of powerful tools educate people as they go - "it's worth noting the effort they put into interfaces, wizards, help systems, and newcomer guidance" (with an "advanced" setting to disable these functions). This was strongly supported by other taskforce members and examples considered or proposed, with Piotrus suggesting such initiatives be prioritized.

Examples and uses were considered. Woodwalker suggested wizards to help remove bad content or deal with POV and other problem text; FT2 suggested a "report a problem" suggestion wizard (via link/icon) on every page, and suggested other examples where wizards might be viable included new account creation (names and multiple accounts), edit war detection, insufficient/absent citation detection and advice, controversial topic editing, and requesting/proposing page deletion. He commented that we should not "demand users read most policies, or hit them with a stick for not doing so. Instead, guide them when they have actual need for the information... The popup doesn't need to say everything covered in the policy, just what's relevant to the action they're doing". He proposed a Wizard extension to allow communities to design and customize wizards with little help from WMF.

Sources

Primary/secondary/tertiary sources were discussed next. Woodwalker noted that these terms were deemed ambiguous (primary sources per academia and Wikipedia especially, differ in meaning): "In real life, most peer-reviewed scientific papers are considered primary sources. I believe many wikipedians consider them to be secondary". He commented that "hundreds of references to primary sources" even in featured articles is a "plague" at enwiki, makes editing more difficult, and wrongly implies that the content is trustworthy. He contrasted the guidance at enwiki that wasn't working well with the possibly overstrict guidance at dewiki which (he felt) does generally work well. FT2 gave examples of different sources as a discussion point; Bhneihouse was still "at a loss" on how the terms were being used and commented good information can come from these different kinds of source. Woodwalker felt this isue, although valid, would not lead to anything in the top 2 -  4 recommendations.

Sjc noted that "[Lack of critical consideration and t]he current obsession with citing sources has led to rafts and reams of citations which fulfill policy criteria at the marked expense of quality", a view endorsed by FT2 and Woodwalker. The latter noted only "specialist users" should use primary sources; others unfamiliar with the necessary approach and rigor should be "actively discouraged" from doing so.

Bhneihouse advised working from standard terms, which would help with the academic community. She suggested identifying tools and documents to focus upon, such as the Wikipedia articles and project pages on primary/secondary/tertiary sources. Woodwalker suggested the German version of that page. Piotrus agreed they were important but the communities were "doing fine" and other areas were a more important priority.

Slrubenstein (at the end of the thread) adds that a "functional definition" of sources is easier:

  • Wikipedia "provides people with a balanced, proportionate, and accurate account of the state of knowledge on a particular topic. It is not a means for forwarding new analytic or synthetic arguments, or interpretations" [tertiary]
  • "Wikipedia is based on... reliable sources that DO present new analysis, synthesis, or interpretation, but are considered reliable according to the criteria of specialists in that topic." [secondary]
  • These in turn "are based on primary sources, that is, whatever it is that they are analyzing, synthesizing, explaining, or interpreting."
Quality v. editability

Woodwalker drew the conversation back to its starting point: "most larger and medium-sized communities" have agreed their requirements for quality, general approach, and page layout, but "there seems to be some friction between the anyone-can-edit principle and the quality principle, and our communities often don't agree where to draw the line". He wanted to know more about "the relation between quality and editability", and suggested looking at ways to reduce friction between them (including already-mentioned wizards).

Bhneihouse suggested that on an encyclopedia "anyone can contribute to" perhaps users could be paired (less experienced with more) to help learning, if the guidance is good then people can teach each other. She also noted different demographic groups (eg children) may have different workflow approaches to bear in mind.

Assessing expert consensus

Bhneihouse stated there was a problem with defining "neutrality" as the broad consensus of experts/specialists in contentious areas, or "balance" in terms of proportion to importance given by experts. In some areas there is no such consensus or an important counterpoint exists, and in others: "[T]he issue here is that the experts may or may not be handling an issue or the parts of an issue in proportion to it's impact, i.e. Nozick on Rawls (Nozick so criticizes Rawls without understanding his argument clearly that his treatise is disproportionate) or dissemination of information about H1N1 (swine) flu -- many experts actually have misinformation, unless you are referencing the CDC or NIH itself you are probably quoting the wrong information". She felt the definition was good but needed care in application.

Woodwalker noted: "If a debate is raging in the literature, the fact should be pointed out. Giving each view proportionally requires knowing the relative importance of the view. There are ways to 'weigh' scientific literature: citation indexes and impact factors.. [if in doubt] we should still try to be as balanced and neutral as possible and cite the literature as precisely as possible. In a couple of years the debate may be decided and another user can then remove the refuted views easily." He added that creating easy-to-use guidance about sources in specialist fields would need experts from those areas.

Bhneihouse noted not all debates happen in the literature; limiting sources to formal literature might misrepresent some areas: "How do we handle information that comes from what is currently considered non-standard channels that academics already accept?". Woodwalker commented the debates might happen there but they were not accepted as references per se. Bhneihouse agreed (verifiability, reliability) and asked if we should document what was and was not appropriate source material from scientific inquiry. She raised a concern that "non academics may have trouble with academic guidelines, no matter how sensible they may be. How do we convey academic standards to non academics in a way they can understand?", which in turn led to a further discussion.

Meta-discussion of the above, and taskforce focus

FT2 opined in reply to the previous question that "we don't". He stated:

"We set up a better editing environment, with better odds of more success, and changes that will probably catalyze and feed through into such areas and issues... and then focus on getting our basic raw editors and basic "not fit to eat" articles up to a basic standard. best odds all round. We can't afford to do much more... that has to be our first focus... then general improvements for established editors, and we can debate what else we can squeeze in. However much we value experts, and suffer when they leave, they leave because of general issues with the editing environment and poor editorship and disputes, all general issues. Focus on those, which also affect everyone else too. Don't try to...fix specialist issues with academic sourcing. Wikipedia's standards and community aren't yet at a point where these... are a "top 5" focus... We have to get the basics needed for improvement. that means specialisms such as experts, FAs and the like are not a priority (this time around). Ruthless but hard choices."

Woodwalker felt while this was not controversial and all articles needed bringing up to a "plateau" (ie, a general basic standard), there is "a serious problem growing inside the larger projects concerning citation of (academic?) literature" with "hundreds of references" that upon review are "simply unsound" - they are irrelevant, contradict or don't confirm the article text they are intended to support, or are otherwise inappropriate. This undermines efforts to be trustworthy in education and academia. He blamed in part the "cite needed" template, which encouraged users to trawl for any web page stating the point, but often without checking or full understanding. He expressed concerns given we "simply don't have enough expert users" to check the cites involved. He suggested as an interim approach a guideline not to use a scientific source unless one understands it oneself.

FT2 suggested a separate category and sourcing guideline for topics where "the topic and availability of references means that core topic material should be cited primarily from strict and limited sources". He suggested a process for doing this, including selection, handling by small groups of editors and with bot help if required, and wizards and feeds to "push these along", and noted this showed the power of suggestions made elsewhere, which could "open whole new quality doorways across the wiki" to help expert topics as well as basic/beginner issues. (Proposal) This concept (ie a Citation/Citation Check wizard) was strongly supported by Woodwalker who made further suggestions [1].

Final words

Slrubenstein notes that Wikipedia has grown. he states that "More people contribute... than are possible to socialize into the Wikipedia way", and that:

"People could easily argue their interpretations of 'NPOV' or 'vandal' or 'troll' and... "global" consensus... is now practically impossible to achieve. I think the number of newbies who do not understand or care about our core policies seriously degrades the quality of articles and increases the number of conflicts. I believe the problem is ignorance, not bad will. So I agree that early mentoring or very user-friendly wizards or tutorials is a great idea. This is a very constructive suggestion"