A couple of conversation starters

Opening

Philippe opened the taskforce asking: "What have we agreed on in terms of quality? Where is the community in terms of the quality discussion. What do we NOT agree on? What have we not discussed about quality, as a community? What sort of information would be useful, in terms of helping us think this through?"

Discussion initially moved to the Five Pillars, newcomer guidance, and the nature of reliable sourcing, before branching out into other threads.

The "Five Pillars" and newcomers

Woodwalker noted the Five Pillars of Wikipedia (Verifiable, Neutral, Balanced, Findable, Encyclopedic) were a sound basis but could be improved and better promoted, and it is highly demotivating when some users don't understand these requirements. FT2 asked "whether it's appropriate and fair - to newcomers and the project - to throw users unfamiliar with encyclopedia writing into the project, without clear guidance", noting we make high demands of their learning curve and standards. He had started writing a "newcomers manual".

Hillgentleman thought the 5 pillars were commonsense; Woodwalker felt they were not obvious to many users and that educating newcomers was a significant drain on time for experienced editors. He suggested that guidelines, 'newcomers manuals', etc should be more visible and easy to find, which in turn allows experienced users to assume they are known and treat such users "with less patience". Woodwalker also noted tools such as Flagged Revisions whose effect on quality was unknown and needed further examination, and suggested looking at "the sharing of all information and initiatives among projects" - while local communities made their own choices they should be aware of the choices and work of others (eg in manuals, tools, etc). Yaroslav Blanter added (later) that perhaps "completeness" should be considered a pillar.

Wizards

FT2 noted that an enwiki user has written a "new article wizard" (updated v2.0). He observed that other organizations seeking mass use of powerful tools educate people as they go - "it's worth noting the effort they put into interfaces, wizards, help systems, and newcomer guidance" (with an "advanced" setting to disable these functions). This was strongly supported by other taskforce members and examples considered or proposed, with Piotrus suggesting such initiatives be prioritized.

Examples and uses were considered. Woodwalker suggested wizards to help remove bad content or deal with POV and other problem text; FT2 suggested a "report a problem" suggestion wizard (via link/icon) on every page, and suggested other examples where wizards might be viable included new account creation (names and multiple accounts), edit war detection, insufficient/absent citation detection and advice, controversial topic editing, and requesting/proposing page deletion. He commented that we should not "demand users read most policies, or hit them with a stick for not doing so. Instead, guide them when they have actual need for the information... The popup doesn't need to say everything covered in the policy, just what's relevant to the action they're doing". He proposed a Wizard extension to allow communities to design and customize wizards with little help from WMF.

Sources

Primary/secondary/tertiary sources were discussed next. Woodwalker noted that these terms were deemed ambiguous (primary sources per academia and Wikipedia especially, differ in meaning): "In real life, most peer-reviewed scientific papers are considered primary sources. I believe many wikipedians consider them to be secondary". He commented that "hundreds of references to primary sources" even in featured articles is a "plague" at enwiki, makes editing more difficult, and wrongly implies that the content is trustworthy. He contrasted the guidance at enwiki that wasn't working well with the possibly overstrict guidance at dewiki which (he felt) does generally work well. FT2 gave examples of different sources as a discussion point; Bhneihouse was still "at a loss" on how the terms were being used and commented good information can come from these different kinds of source. Woodwalker felt this isue, although valid, would not lead to anything in the top 2 -  4 recommendations.

Sjc noted that "[Lack of critical consideration and t]he current obsession with citing sources has led to rafts and reams of citations which fulfill policy criteria at the marked expense of quality", a view endorsed by FT2 and Woodwalker. The latter noted only "specialist users" should use primary sources; others unfamiliar with the necessary approach and rigor should be "actively discouraged" from doing so.

Bhneihouse advised working from standard terms, which would help with the academic community. She suggested identifying tools and documents to focus upon, such as the Wikipedia articles and project pages on primary/secondary/tertiary sources. Woodwalker suggested the German version of that page. Piotrus agreed they were important but the communities were "doing fine" and other areas were a more important priority.

Slrubenstein (at the end of the thread) adds that a "functional definition" of sources is easier:

• Wikipedia "provides people with a balanced, proportionate, and accurate account of the state of knowledge on a particular topic. It is not a means for forwarding new analytic or synthetic arguments, or interpretations" [tertiary]
• "Wikipedia is based on... reliable sources that DO present new analysis, synthesis, or interpretation, but are considered reliable according to the criteria of specialists in that topic." [secondary]
• These in turn "are based on primary sources, that is, whatever it is that they are analyzing, synthesizing, explaining, or interpreting."

Quality v. editability

Woodwalker drew the conversation back to its starting point: "most larger and medium-sized communities" have agreed their requirements for quality, general approach, and page layout, but "there seems to be some friction between the anyone-can-edit principle and the quality principle, and our communities often don't agree where to draw the line". He wanted to know more about "the relation between quality and editability", and suggested looking at ways to reduce friction between them (including already-mentioned wizards).

Bhneihouse suggested that on an encyclopedia "anyone can contribute to" perhaps users could be paired (less experienced with more) to help learning, if the guidance is good then people can teach each other. She also noted different demographic groups (eg children) may have different workflow approaches to bear in mind.

Assessing expert consensus

Bhneihouse stated there was a problem with defining "neutrality" as the broad consensus of experts/specialists in contentious areas, or "balance" in terms of proportion to importance given by experts. In some areas there is no such consensus or an important counterpoint exists, and in others: "[T]he issue here is that the experts may or may not be handling an issue or the parts of an issue in proportion to it's impact, i.e. Nozick on Rawls (Nozick so criticizes Rawls without understanding his argument clearly that his treatise is disproportionate) or dissemination of information about H1N1 (swine) flu -- many experts actually have misinformation, unless you are referencing the CDC or NIH itself you are probably quoting the wrong information". She felt the definition was good but needed care in application.

Woodwalker noted: "If a debate is raging in the literature, the fact should be pointed out. Giving each view proportionally requires knowing the relative importance of the view. There are ways to 'weigh' scientific literature: citation indexes and impact factors.. [if in doubt] we should still try to be as balanced and neutral as possible and cite the literature as precisely as possible. In a couple of years the debate may be decided and another user can then remove the refuted views easily." He added that creating easy-to-use guidance about sources in specialist fields would need experts from those areas.

Bhneihouse noted not all debates happen in the literature; limiting sources to formal literature might misrepresent some areas: "How do we handle information that comes from what is currently considered non-standard channels that academics already accept?". Woodwalker commented the debates might happen there but they were not accepted as references per se. Bhneihouse agreed (verifiability, reliability) and asked if we should document what was and was not appropriate source material from scientific inquiry. She raised a concern that "non academics may have trouble with academic guidelines, no matter how sensible they may be. How do we convey academic standards to non academics in a way they can understand?", which in turn led to a further discussion.

Meta-discussion of the above, and taskforce focus

FT2 opined in reply to the previous question that "we don't". He stated:

"We set up a better editing environment, with better odds of more success, and changes that will probably catalyze and feed through into such areas and issues... and then focus on getting our basic raw editors and basic "not fit to eat" articles up to a basic standard. best odds all round. We can't afford to do much more... that has to be our first focus... then general improvements for established editors, and we can debate what else we can squeeze in. However much we value experts, and suffer when they leave, they leave because of general issues with the editing environment and poor editorship and disputes, all general issues. Focus on those, which also affect everyone else too. Don't try to...fix specialist issues with academic sourcing. Wikipedia's standards and community aren't yet at a point where these... are a "top 5" focus... We have to get the basics needed for improvement. that means specialisms such as experts, FAs and the like are not a priority (this time around). Ruthless but hard choices."

Woodwalker felt while this was not controversial and all articles needed bringing up to a "plateau" (ie, a general basic standard), there is "a serious problem growing inside the larger projects concerning citation of (academic?) literature" with "hundreds of references" that upon review are "simply unsound" - they are irrelevant, contradict or don't confirm the article text they are intended to support, or are otherwise inappropriate. This undermines efforts to be trustworthy in education and academia. He blamed in part the "cite needed" template, which encouraged users to trawl for any web page stating the point, but often without checking or full understanding. He expressed concerns given we "simply don't have enough expert users" to check the cites involved. He suggested as an interim approach a guideline not to use a scientific source unless one understands it oneself.

FT2 suggested a separate category and sourcing guideline for topics where "the topic and availability of references means that core topic material should be cited primarily from strict and limited sources". He suggested a process for doing this, including selection, handling by small groups of editors and with bot help if required, and wizards and feeds to "push these along", and noted this showed the power of suggestions made elsewhere, which could "open whole new quality doorways across the wiki" to help expert topics as well as basic/beginner issues. (Proposal) This concept (ie a Citation/Citation Check wizard) was strongly supported by Woodwalker who made further suggestions [1].

Final words

Slrubenstein notes that Wikipedia has grown. he states that "More people contribute... than are possible to socialize into the Wikipedia way", and that:

"People could easily argue their interpretations of 'NPOV' or 'vandal' or 'troll' and... "global" consensus... is now practically impossible to achieve. I think the number of newbies who do not understand or care about our core policies seriously degrades the quality of articles and increases the number of conflicts. I believe the problem is ignorance, not bad will. So I agree that early mentoring or very user-friendly wizards or tutorials is a great idea. This is a very constructive suggestion"

What have we agreed on in terms of quality? Where is the community in terms of the quality discussion. What do we NOT agree on? What have we not discussed about quality, as a community? What sort of information would be useful, in terms of helping us think this through?

A lot of questions. The first question is related to the first key question for the quality task force. In short: we have the 5 pillars (at least most projects do). Somewhere between these pillars (following links to individual guidelines) five core requirements for content can be found: it should be verifiable, neutral, balanced, findable and encyclopaedic (at least for the -pedia projects, other projects have similar requirements). These words have been used for several different purposes. What I mean is:

• Verifiable means content is based on sources that can be trusted. Sources should preferably (1) be primary, (2) be as recent as possible, in that order. The order is to reduce the anecdotal way of referencing that is a plague in Wikipedia.
• Neutral means the content reflects the broad consensus among experts/specialists in literature.
• Balanced means sections of the content are proportional to the importance given to it by those experts.
• Findable means the reader can find the content easily. This is where the wiki structure plays its unique role (categories, links, navigation templates, etc).
• Encyclopaedic means it is not written in a format different from what is expected in an encyclopaedia. Other projects (wiktionary, wikiquote, wikibooks, etc) have instead of this a similar requirement of format.

These five core requirements can imho still be worked out in more detail and should really be better advertised and implemented. This has to be done if we want to attract users that add quality and readers that search for quality. One of the most demotivating factors for quality users is when they experience ongoing obstruction from users that don't understand these requirements. Woudloper 20:37, 17 November 2009 (UTC)

A related question I've asked is whether it's appropriate and fair – to them and the project – to dump users unfamiliar with encyclopedia writing into the project, without some clear-cut "newcomers' guide".

(I started writing a newcomer's manual to try and help on this).

There was a time that anyone who wasn't a vandal and clicked "edit here" was fine, because it wasn't that big and standards weren't that demanding, but now we're a lot more demanding in 2 ways: we demand they pick up with a fast learning curve, and we set higher and more detailed standards.

We still want users to just click and edit. Can we reconcile these wishes?

Re: whether it's appropriate and fair – to them and the project – to dump users unfamiliar with encyclopedia writing into the project, without some clear-cut "newcomers' guide".

It depends on what you want from your new editors and why you want to let everybody have the freedom to edit right away. As the English wikipedia has already taken shape and refined many times over, newcomers may

• keep the information updated
• fix errors
• write new articles.

Basically the 5 pillars are just common sense. (You don't say things loudly unless you have a reference.) So why not? the wikipedia community is still evolving, and rightly so. Hillgentleman 21:50, 17 November 2009 (UTC)

The 5 pillars are just a start. They seem common sense, I agree. Unfortunately, for a lot of users they are not. In some cases because they are newcomers, in some because they don't understand, in some because they don't want to understand. Most of these users need to be contacted personally to be told about things like the 5 pillars, even if such guidelines are linked in welcome messages. The need to be constantly helping and educating others about the basics takes valuable time from experienced users. This problem is present both at the larger and smaller projects, though the amount of patience with and the ways of contacting newcomers differ enormously. Tackling it can be done in two conventional ways:

• Make guidelines, 'newcomers manuals', etc more visible. They should be easy to find. A link should be present when a new or unregistered user edits a project, preferably close to the 'save' button.
• If this is done, it can be assumed new users should know the guidelines when editing. They can therefore be treated with less patience, which saves time for experienced users.

Apart from this, new ways of editing have been introduced at several larger projects. An example are flagged revisions, a system which filters contributions of experienced users from those of the inexperienced. In its most strict sense flagged revisions can even be used to make contributions of inexperienced users temporary invisible (except for themselves). As of yet, the influence of such systems on quality is not yet clear to me. I think their exact influence on projects where they were introduced should still be examined into more detail.

Apart from this another open door: the sharing of all information and initiatives among projects could be better. If a manual is written at a certain project, other projects should be made aware of its existence. Someone, perhaps the foundation, has to play a more active role in this, perhaps by posting an 'up for translation' message at local message boards or village pumps. Of course not in an authoritarian way. The choice is made by local communities, but they should be made aware of the choices of others.

Someone on enwiki has written a "new article wizard" (and updated v2.0). If people are educated as they go (in sensible size doses) then it could pay off in quality terms. If we want quality articles, and we want mass editorship, then one starting point is that most editors won't be used to this kind of writing or the standards, policies and arcana that have evolved. Bringing them sufficiently up to speed - ensuring whatever they do is guided - is a common way to resolve that, in the "real world". If you look at companies that want wide usage of complex programs for example (Microsoft's a good example), it's worth noting the effort they put into interfaces, wizards, help systems, and "newcomer guidance". There's usually an "advanced" setting for users who don't need/want that.

Wizzards are indeed a very useful and potentially time-saving tool. Your example is for adding good content. However, wizzards can also be used for instructing how to remove bad content. Above I tried to define content quality by five requirements (it's just a try). What could help is making such wizzards for all of these requirements, then translate them for all projects. For example 'how to deal with POV/unencyclopaedic/biased/etc content'.

A "Report a problem with this page/article" link or icon on every page, that leads to a pop-up wizard for advising how to handle problems? I'd say "yes" to that. It's something we could do with the present-day setup.

One of the wikis (polish, maybe?) has something like this. Might be worth checking into.

@FT2: it is a good idea. Of course there should first be a wizard before there can be a link to it though. @Philippe: I checked the Polish wp but couldn't find something like it. Perhaps Piotrus can help us out.

Love, Love, Love the new article wizard....

The new article wizard is certainly a step in the right direction, as far as tapping the pools of less tech-savvy editors go. Such initiatives should be prioritized, and the Wizard should be included in the more friendly (usability wise) Beta being developed. --Piotrus 19:57, 25 November 2009 (UTC)

I'd like to see the same principles elsewhere, with generic "wizard hooks" in the code, and a "Wizard:" namespace so users can actually code up the wizard contents for these hooks on-wiki:

• New account wizard, asks users "These are the issues with name choice, please choose a name"... asks "have you already got an account or used one in the past" and notifies them the basics of sock policy"...

• Edit war wizard, detects 3+ reverts (not quite 3RR) and advises users "I see you've reverted this page a few times recently, are you sure?" -> Yes / Help me! / Cancel my edit

• Citation wizard, "I see you've added a chunk of text (or new article) but there aren't any citations in it. Would you like help in adding citations so other users can check your edits are verifiable?"

• Controversial topic wizard, advises users "You're editing a controversial topic. Would you like some hints and tips first?"

• Deletion wizard: [if we gave users a "delete" button even if not admins] -> "You've clicked "delete", but you aren't an administrator. Wikipedia has a number of deletion tools and processes. Would you like guidance on the appropriate one to use, and on making your request?"

• ... etc ...

In other words, don't demand users read most policies, or hit them with a stick for not doing so. Instead, guide them when they have actual need for the information.

So the links for a deletion wizard might be buttons for "Help me!" and "Cancel request", and also a link for "View policy summary" which opens a short popup that includes a link to "View full policy". The popup doesn't need to say everything covered in the policy, just what's relevant to the action they're doing.

(I also wouldn't make this "ad hoc". I'd design a proper "Mediawiki Wizards" extension, so that all that would be needed in future is a bugzilla request: "Please add a wizard hook for X, that provides parameters P, Q, R". The conditions it fires (parameter based) and the actual wizard pages, are then configurable on-wiki, either via a "Wizard:" namespace, or stand-alone like AbuseFilter.)

May be we should add completeness of the information as a requirement: The information about the subject of the article should be as complete as possible. (Not may be formulated in a proper way; I only mean relevant information, but this is a discussion point).

err, just checking in. But I think you meant that references should be secondary, or third-party. Primary sources open Wikipedia to a flood of personal websites, press releases, and advertisements. Stuff that is less reliable because it's not independent.

@Randomran: I see what you mean. When I wrote "sources that can be trusted", I didn't mean personal websites or advertisements. Information from such sources doesn't fulfil the requirements of being encyclopaedic, neutral or balanced anyway.

Yeah, sorry, I was just fixated on the word "primary". Using primary sources opens the floodgates to articles written from the most biased perspectives. I know it's just semantics, though, because everything else you wrote makes perfect sense.

It suddenly occurred to me that 'primary source' is often used with a different meaning at Wikipedia. In real life, most peer-reviewed scientific papers are considered primary sources (1+3*). I believe many wikipedians consider them to be secondary. I had the real-life definition in mind when I used the word 'primary'.

The guideline at wp-en is actually recommendable, however it is sadly tucked away as a subsection of the original research guideline. Nevertheless, it got translated literally to a number of other projects (I found translations of an older version of the English text in the Spanish, Czech and Dutch guidelines on original research). The equally recommendable German guideline is shorter and very clear. It forbids the use of primary sources of any kind (1+2+3*)and states that Wikipedia content should entirely be based on secondary literature (3*+4).

I personally don't object to the use of 'primary sources' when they are peer-reviewed (1*+3*) and used with care by a user specialized in the field. For factual information (the population of Wyoming, the dimensions of a certain crystal system) I think even non-peer-reviewed primary sources (2+3*)can be used, since they are often the only way to accomplish verifiability (in these two examples: the U.S. census and the mindat.org crystallographic database can respectively be used as refs). However, I think MOST of Wikipedia/Wikiversity/Wikibooks content should be based on a balanced combination of secondary sources (3*+4) to be really reliable. Primary sources (1+2+3*) should only be used when there is no way around it, for example when a certain number or fact is quoted in an infobox template. Using hundreds of references to primary sources (1+2+3*) in (even featured -example) articles is a plague at wp-en. Such a sheer number of unsound sources has two unwanted effects:

• It makes editing more difficult, and
• It wrongly introduces the suggestion that the content can be trusted.

Somehow the guideline at wp-en isn't working well, while the very strict guideline at wp-de (in my eyes: too strict) works generally well (there are exceptions too). I have no explanation for this. Wild guesses: English admins don't dear to implement their own guideline or perhaps the thing is indeed tucked away too far?

Terminology's going to get in the way here. Can you edit the above, so it's clear when you're meaning:

• A primary source in the sense of "a personal or eye witness account, or the original writing" - one which represents the words of its author alone.
• A writing as above, but from a source usually recognized to be reliable despite no formal peer review (government and certain major organizations' official/formal records, etc)
• A source such as a peer reviewed paper that represents the view of its author but has some kind of credible review
• A source where an author discusses and analyzes various of the above dispassionately, to provide coverage of a topic.

I numbered your four types and put red numbers in my last reply. A star means "part of type ...". I feel we should find better terminology/definitions first if we want to continue this discussion. But do we? Our task force could for example end up giving advice on the creation of wizards/guidelines for every source type we can identify ('when to cite a peer-reviewed paper about history'). I'm not sure if this discussion leads to one of the 2-4 most effective recommendations this task force could give for improving quality though.

Often seemingly self-evident primary sources are not reliable. In fact very often the converse is the case. They are explicitly unreliable, erroneous, biased, slanted, POV or downright lies and propaganda. Even basic level history students are taught that in evaluating evidence one has to be critical of its provenance, its intention and the careful handling of all such testimony as corroboration. The current obsession with citing sources has led to rafts and reams of citations which fulfill policy criteria at the marked expense of quality.

Yes.

That was exactly the point I tried to make in the other thread at this talk page. The use of primary sources should be discouraged when the contributor doesn't have extensive knowledge about a subject. In practice, that means only specialist users should be using primary sources. Other users should be actively discouraged to do so.

Back to the original questions by Philippe: most larger and medium-sized communities I know have discussed requirements for quality of content, more general ones (encyclopaedicity, neutrality, bias, etc) and page-specific ones (structure, style, lay-out, etc). There seems to be some friction between the anyone-can-edit principle and the quality principle, and our communities often don't agree where to draw the line. I think it would be handy to know more about the relation between quality and editability before we can address the problem. What we can do in the meantime is think about ways to reduce the friction between these two principles (an example is the idea of wizzards).

1. I am still at a loss about primary and secondary sources and how people are defining them. I would prefer that Wikipedia standardize its terminology to research terminology. A primary source is the "person" who creates the knowledge, i.e. Einstein created e=mc2. A secondary source is someone who "digests" the information and uses it usually as a part of another project, i.e. a scientist explaining energy. Good information, if properly vetted especially when peer reviewed, can come from both sources.

2. At the beginning of the thread, Phillipe wrote:

 * Neutral means the content reflects the broad consensus among experts/specialists in literature.

this could present a problem when dealing with issues where there is no broad consensus among experts/specialists or when the counterpoint is important to discuss, i.e. the issue of vaccination of children and the potential risk to them.

3. Also:

   * Balanced means sections of the content are proportional to the importance given to it by those experts.

again, the issue here is that the experts may or may not be handling an issue or the parts of an issue in proportion to it's impact, i.e. Nozick on Rawls (Nozick so criticizes Rawls without understanding his argument clearly that his treatise is disproportionate) or dissemination of infomration about H1N1 (swine) flu -- many experts actually have misinformation, unless you are referencing the CDC or NIH itself you are probably quoting the wrong information, and the CDC is a primary source -- so if you don't use a primary source then Wikipedia is SOL (excuse the parlance) for accurate information. In general I think the idea of balance as defined is good however we have to be careful as we define what balance really is and where it comes from.

3. Let's remember where the basis for Wikipedia came from as we go through this process -- something that everyone can do/something that everyone can contribute to. We want to find a way that the average Joe can use Wikipedia yet keep it current and vetted. Perhaps the focus shouldn't just be Mrs. Rose the 60 year old horticulturist teaching herself how to format in Wikipedia so she can contribute about roses, or getting her to teach herself how to vet her sources. Perhaps people with less experience can be paired with people with more experience, and then those people can teach others and so on. As long as there are solid, easily accessible guidelines then one person becomes two becomes four becomes...

LOVE the article wizard concept. I will try it in the next few days. WOW.

And how about we test some of this on children? My daughter is 12 and I know these guys have a slightly different workflow that older folks. Let's make sure we include from elementary school up in our mindset as we move this forward.

1 -> I agree, if we want to discuss this into detail we have to adapt or invent better terminology. FT2 gave 4 types of sources, we could easily add more. These types can again be split into peer-reviewed or not, etc. But see 3.

2 -> If a debate is raging in the literature, the fact should be pointed out. Giving each view proportionally requires knowing the relative importance of the view. There are ways to 'weigh' scientific literature: citation indexes and impact factors give a relative importance to each paper and magazine (there are disadvantages to this, but it is as objective as possible). If it's not possible to know what the balance between the different POVs is, we should imho still try to be as balanced and neutral as possible and cite the literature as precisely as possible. In a couple of years the debate may be decided and another user can then remove the refuted views easily. Another simple rule of thumb that should be used with care: 'when in doubt, don't'.

3 -> I personally have expertise with literature about natural sciences. I am able to write a manual/wizzard 'how to use and cite scientific literature about natural sciences'. Your examples are from medical sciences, and I cannot judge them. If we want to create easy-to-use manuals about every type of source, we would need an expert from each area.

I think we need to work from "standard" terms that already exist. I think that defining or redefining sources will lead to vast amounts of confusion in vetting information and in Wikipedia being taken seriously by the academic community. It's a bit like redefining what the word "red" means. Sure we can do it, but is there much point to it?

I agree about issues that are hot topics. However, not all debates happen in the literature. Some debates happen across many types of sources -- from listservs to personal email correspondence to lectures and discussions. So I don't think constraining ourselves to the literature does certain topics justice. I agree about assessing ways to weight relative importance however, if the sources are more difficult to track, that throws off the calculations. For example: Mothering magazine recently shared a link that stated the Canadian government was ceasing certain flu vaccines because it was shown that getting a seasonal flu vaccine shot made a person more succeptible to the H1N1 flu. This was passed out on a listserv. I contacted a friend in public health who contacted a colleague in Canada who had worked for the World Health Organization. He debunked the Mothering link in its entirety. If we are only tracking the literature, then how would I share this? In scholarly research, emails, conversations and interviews are all citation worthy and can be included. How do we handle information that comes from what is currently considered non-standard channels that academics already accept?

I don't know that every type of source needs its own manual, or that every type of information needs its own type of manual. In academia, there are various codified standards, but there are similarities between all of them. Do we want to consider citing differently per subject based on different types of content, i.e. MLA vs APA? I dont think so. So why would we have different formats or criteria for different types of information? That would create incredible confusion.

Perhaps this is where we need to go for definitions:

http://en.wikipedia.org/wiki/Primary_source

and where we need to ensure our documentation exists.

I would suggest focusing on editing this one document, and this one:

http://en.wikipedia.org/wiki/Secondary_source

and making these documents part of the basis for our quality initiative.

I would also suggest that we tighten up this document:

http://en.wikipedia.org/wiki/Wikipedia:No_original_research#Primary.2C_secondary.2C_and_tertiary_sources

We have been talking around ideas, which is good, but it would also be good to start identifying the tools we need to leverage and create better quality. If we start a list of documents that will become the core of our effort, we can start to identify how and where we are going to implement our quality initiatives. Then later, we can hammer out exact definitions of primary, secondary and the like.

Anyone else have "pet" documents that should be on this list?

I'd certainly recommend the German version of that guideline. The French version is very limited compared to the English and German versions. If you don't understand German I can post a Google-translation somewhere.

Reliable Sources is my pet one, but honestly I think they are fine. Tweaking them will not result in anything major, and I feel that those policies have been developing well anyway. I suggest we agree that they are important and need to be further improved, but also that the community has been doing fine job on those and we should focus on other issues (like diminishing pool of contributors and community health issues, with their implications for quality). --Piotrus 19:59, 25 November 2009 (UTC)

I'd like to yes. The definitions at wp-en are a good start but certainly not sufficient. As our conversation above showed, the terms primary and secondary source are perhaps too confusing to use.

Emails, private conversations and other unpublished sources may be used by researchers, but they are certainly not accepted as reference in the scientific magazines (at least not in my field). I'm not sure if we should use them, if the literature itself prohibits it. This is in any case highly specialistic content. It is only likely to be an issue for our largest projects such as wp-en or wp-es.

I'll give an example why I think it would be good to have a manual for using scientific publications in natural sciences as a source. Any paper will have a certain structure (abstract-intro-method-results-discussion). Depending on the place in the paper, information may be a primary or secondary source. Alerting the user on how to read and use a paper could decrease the amount of unsound references, which is imho a problem at all larger projects.

I agree about emails and conversations, as they can get outside the realm of verifiability and reliability. So, are you proposing that we have different standards/manuals for each of a number of different subjects? Also, if we are following scientific journals and scientific principles then we perhaps should follow scientific principles of the definitions of primary and secondary research. Perhaps a matrix of what we will and will not utilize from scientific inquiry is in order?

While I do not argue with your points, my concern is that non academics may have trouble with academic guidelines, no matter how sensible they may be. How do we convey academic standards to non academics in a way they can understand?

Being a bit controversial -- we don't.

We set up a better editing environment, with better odds of more success, and changes that will probably catalyze and feed through into such areas and issues... and then focus on getting our basic raw editors and basic "not fit to eat" articles up to a basic standard. best odds all round. We can't afford to do much more, I think that has to be our first focus.

If we have a focus left over, then general improvements for established editors, and we can debate what else we can squeeze in. However much we value experts, and suffer when they leave, they leave because of general issues with the editing environment and poor editorship and disputes, all general issues. Focus on those, which also affect everyone else too. Don't try to make special "expert only" stuff, or fix specialist issues with academic sourcing. Wikipedia's standards and community aren't yet at a point where these specialist issues are a "top 5" focus.

Apologies if this sounds controversial or such. Discuss if needed, but after thinking through how to make best use of an opportunity like this, that's more and more where it seems to come down to. We have to get the basics needed for improvement. that means specialisms such as experts, FAs and the like are not a priority (this time around). Ruthless but ... sigh... hard choices.

@FT2: it doesn't sound controversial. Yet some projects, such as wp-en and wp-de, are certainly at an academic level in some areas. I agree these projects should focus on areas that are behind, but there's also the principle that writing a new article is five times less work than rewriting old stuff. I think there is a serious problem growing inside the larger projects concerning citation of (academic?) literature. At the English Wikipedia, longer articles often have hundreds of references to scientific papers. Following these refs, I often find they are simply unsound.

• They refer to papers that describe another thing;
• They refer to papers that try to disprove the very information they should verify;
• They refer to papers that are secondary sources for the information they should verify;
• They refer to papers that don't even contain the information they should verify.

I can live with the secondary/primary problem for the moment, but the large-scale malcitation at wp-en is a serious problem that undermines the project's intention to become a trustworthy source in education or academics.

Let's call it 'the curse of the {{citation needed}}-template'. Instead of citing the source used for writing the article, users tend to search for any citation on the internet that verifies the information in the article, just to get rid of the {{citation needed}}-template. They find abstracts of scientific papers (access to scientific literature on the internet is often limited to the abstracts) and cite without understanding or reading what the papers are about. It's a way of being lazy. Instead of going to a library, the contributor uses Google, reads diagonally through an abstract and voilà he has a ref. Anyone can see this method will never create an encyclopaedic source useful for academic use.

I agree that this isn't our top priority, and I agree the projects should first get to a plateau, a general level of quality, before they try to reach an academic level. Yet imagine the titanic task of having to check thousands of references for their value once the project is there. It's also a difficult task tat requires specialist knowledge about the subject. Most likely it's not going to happen at all. We simply don't have enough expert users in our communities that would be ready to help.

I don't like the idea of having different guidelines for normal and specialist contributors. The solution I'd rather see is simple. We could suggest to implement a guideline not to use a source when one does not understand it well. Only use a scientific paper as a source when you have a thorough knowledge of the subject yourself. If you don't have that specialistic knowledge, go to a library, search for a textbook (as recent and summarizing as possible - a truly sound secondary source) and use that instead. Such a guideline would be a very simple and effective way to make Wikipedia more acceptable for academic use.

Maybe a simple solution would be to have a category and sourcing guidelines ("Category:Articles whose core content should primarily be from current peer reviewed sources"), for those topics where the topic and availability of references means that core topic material should be cited primarily from strict and limited sources.

Users can agree by consensus if an article should move to that category. Bots can also be used to identify and suggest articles where most cites are already from a list of peer reviewed or other highly reputed sources, for review.

This fits within normal handling. Then for such articles, a bot reviews these every while and highlights those cites which are more than 6(?) 12(?) months from publication so our academic experts can check these and update them or ensure they're still currently valid.

So... essentially isn't this something easily done already, without needing the weight of a taskforce and quality project behind it? Any small group of users could propose a mechanism of this kind and the low level of bot, feed, or other infrastructure needed to support it.

The twist is, we can then use the processes being discussed elsewhere to heavily "push" this along. We can ask readers to ("Help Wikipedia! This topic has 3 cites more than 6 months old. Please update them if they're out of date for the field!") or put these onto a feed or user filters, for users interested in the topics. Cites are tagged with the date of publication so a Citation Check Wizard can ask on editing, "I notice you're about to add a 18 month old cite to this topic. Are you sure this still represents latest knowledge in the field?". And so on. Think about it :)

It also shows how the seemingly simple recommendations we've been discussing can also be used to improve quality in other ways and places - including even specialisms like expert sourcing, which was never previously considered. That's the kind of pervasive effect that we can aim for. Notice we were considering wizards and notifications in a "basic quality" and "new user" context, but once added they can open whole new quality doorways across the wiki.

I just read your post in the other thread and this Citation Wizard, wow, I really like the idea! An independent Citation Check Wizard would be great too.

The problem I wanted to point out in my last post (08:20, 27 November 2009) wasn't the problem of outdated sources (Actually, 18 months seems little time for me, 10 to 20 years is already quite good for scientific papers). It's the problem of wrong citations due to a lack of understanding in the subject (see the list of examples I gave in my last post - they are unfortunately quite common). The citation wizard could help, for example by giving advice depending on expertise:

• You are:
  • An expert for this subject, knowing well the consensus in the literature -> either use primary sources (scientific papers) or secondary sources (textbooks), depending on how general the information is.
  • Just interested but knowing a lot about the subject -> use secondary sources (textbooks), only use primary sources (scientific papers) if you are sure what you're doing.
  • Just searching for a source that can replace the stupid template! -> try to find a secondary source (textbook or neutral website) which has the same subject as this article or section.

I think this is a very valuable point. I think that Wikipedia has expanded exponentially, meaning that more people contribute each montht than are possible to socialize into the Wikipedia way. When I first registered, I got into plenty of conflicts utnil I really "got" NPOV. There are articles I wrote or contributed to back then that I later had to revamp to comply with NOR. But when I registered Wikipedia was dominated by a relatively small comunity of people who really understood the issues raised by NPOV and to which NPOV was a response. Back then, too, the number of people active on the listserve was relatively small but highly representative of the then-small Wikipedia community. People could easily argue their interpretations of "NPOV" or "vandal" or "troll" and reach the kind of "global" consensus FT2 above points out is now practically impossible to achieve.

I think the number of newbies whho do not understand or care about our core policies seriously degrades the quality of articles and increases the number of conflicts. I believe the problem is ignorance, not bad will. So I agree that early mentoring or very user-friendly wizards or tutorials is a great idea. This is a very constructive suggestion.

As to sources ... I think this is something to be decided by the comunity via those who work on NOR or adjunct policies.

But my personal view is that it is easier to explain them backwards:

Wikipedia is a tertiary source: it provides people with a balanced, proportionate, and accurate account of the state of knowledge on a particular topic. It is not a means for forwarding new analytic or synthetic arguments, or interpretations.

Wikipedia is based on secondary sources. These are reliable sources that DO present new analysys, synthesis, or interpretation, but are considered reliable according to the criteria of specialists in that topic.

Secondary sources are based on primary sources, that is, whatever it is that they are analyzing, synthesizing, explaining, or interpreting.

I see this is a kind of functionalist way of defining sources, i.e. as functions of one another. One value of this approach is, it explains why something can be both a brimary and a secondary source. For example, when Marx or Freud were originally publishing their books, they were secondary sources and in some cases we can still use them as secondary sources. But since they have died, many schools of interpretation of their works have developed, and there is a vast body of scholarship analyzing or interpreting Freud (e.g. Juliet Mitchell, Leo Bersani, Jacques Lacan) or Marx (e.g. Althusser, Thompson, etc.) and in some articles, it is these interpreters who are the secondary sources, and marx or freud the primary sources. Slrubenstein 14:09, 8 December 2009 (UTC)