A couple of conversation starters
Back to the original questions by Philippe: most larger and medium-sized communities I know have discussed requirements for quality of content, more general ones (encyclopaedicity, neutrality, bias, etc) and page-specific ones (structure, style, lay-out, etc). There seems to be some friction between the anyone-can-edit principle and the quality principle, and our communities often don't agree where to draw the line. I think it would be handy to know more about the relation between quality and editability before we can address the problem. What we can do in the meantime is think about ways to reduce the friction between these two principles (an example is the idea of wizzards).
1. I am still at a loss about primary and secondary sources and how people are defining them. I would prefer that Wikipedia standardize its terminology to research terminology. A primary source is the "person" who creates the knowledge, i.e. Einstein created e=mc2. A secondary source is someone who "digests" the information and uses it usually as a part of another project, i.e. a scientist explaining energy. Good information, if properly vetted especially when peer reviewed, can come from both sources.
2. At the beginning of the thread, Phillipe wrote:
* Neutral means the content reflects the broad consensus among experts/specialists in literature.
this could present a problem when dealing with issues where there is no broad consensus among experts/specialists or when the counterpoint is important to discuss, i.e. the issue of vaccination of children and the potential risk to them.
3. Also:
* Balanced means sections of the content are proportional to the importance given to it by those experts.
again, the issue here is that the experts may or may not be handling an issue or the parts of an issue in proportion to it's impact, i.e. Nozick on Rawls (Nozick so criticizes Rawls without understanding his argument clearly that his treatise is disproportionate) or dissemination of infomration about H1N1 (swine) flu -- many experts actually have misinformation, unless you are referencing the CDC or NIH itself you are probably quoting the wrong information, and the CDC is a primary source -- so if you don't use a primary source then Wikipedia is SOL (excuse the parlance) for accurate information. In general I think the idea of balance as defined is good however we have to be careful as we define what balance really is and where it comes from.
3. Let's remember where the basis for Wikipedia came from as we go through this process -- something that everyone can do/something that everyone can contribute to. We want to find a way that the average Joe can use Wikipedia yet keep it current and vetted. Perhaps the focus shouldn't just be Mrs. Rose the 60 year old horticulturist teaching herself how to format in Wikipedia so she can contribute about roses, or getting her to teach herself how to vet her sources. Perhaps people with less experience can be paired with people with more experience, and then those people can teach others and so on. As long as there are solid, easily accessible guidelines then one person becomes two becomes four becomes...
LOVE the article wizard concept. I will try it in the next few days. WOW.
And how about we test some of this on children? My daughter is 12 and I know these guys have a slightly different workflow that older folks. Let's make sure we include from elementary school up in our mindset as we move this forward.
1 -> I agree, if we want to discuss this into detail we have to adapt or invent better terminology. FT2 gave 4 types of sources, we could easily add more. These types can again be split into peer-reviewed or not, etc. But see 3.
2 -> If a debate is raging in the literature, the fact should be pointed out. Giving each view proportionally requires knowing the relative importance of the view. There are ways to 'weigh' scientific literature: citation indexes and impact factors give a relative importance to each paper and magazine (there are disadvantages to this, but it is as objective as possible). If it's not possible to know what the balance between the different POVs is, we should imho still try to be as balanced and neutral as possible and cite the literature as precisely as possible. In a couple of years the debate may be decided and another user can then remove the refuted views easily. Another simple rule of thumb that should be used with care: 'when in doubt, don't'.
3 -> I personally have expertise with literature about natural sciences. I am able to write a manual/wizzard 'how to use and cite scientific literature about natural sciences'. Your examples are from medical sciences, and I cannot judge them. If we want to create easy-to-use manuals about every type of source, we would need an expert from each area.
I think we need to work from "standard" terms that already exist. I think that defining or redefining sources will lead to vast amounts of confusion in vetting information and in Wikipedia being taken seriously by the academic community. It's a bit like redefining what the word "red" means. Sure we can do it, but is there much point to it?
I agree about issues that are hot topics. However, not all debates happen in the literature. Some debates happen across many types of sources -- from listservs to personal email correspondence to lectures and discussions. So I don't think constraining ourselves to the literature does certain topics justice. I agree about assessing ways to weight relative importance however, if the sources are more difficult to track, that throws off the calculations. For example: Mothering magazine recently shared a link that stated the Canadian government was ceasing certain flu vaccines because it was shown that getting a seasonal flu vaccine shot made a person more succeptible to the H1N1 flu. This was passed out on a listserv. I contacted a friend in public health who contacted a colleague in Canada who had worked for the World Health Organization. He debunked the Mothering link in its entirety. If we are only tracking the literature, then how would I share this? In scholarly research, emails, conversations and interviews are all citation worthy and can be included. How do we handle information that comes from what is currently considered non-standard channels that academics already accept?
I don't know that every type of source needs its own manual, or that every type of information needs its own type of manual. In academia, there are various codified standards, but there are similarities between all of them. Do we want to consider citing differently per subject based on different types of content, i.e. MLA vs APA? I dont think so. So why would we have different formats or criteria for different types of information? That would create incredible confusion.
Perhaps this is where we need to go for definitions:
http://en.wikipedia.org/wiki/Primary_source
and where we need to ensure our documentation exists.
I would suggest focusing on editing this one document, and this one:
http://en.wikipedia.org/wiki/Secondary_source
and making these documents part of the basis for our quality initiative.
I would also suggest that we tighten up this document:
We have been talking around ideas, which is good, but it would also be good to start identifying the tools we need to leverage and create better quality. If we start a list of documents that will become the core of our effort, we can start to identify how and where we are going to implement our quality initiatives. Then later, we can hammer out exact definitions of primary, secondary and the like.
Anyone else have "pet" documents that should be on this list?
I'd certainly recommend the German version of that guideline. The French version is very limited compared to the English and German versions. If you don't understand German I can post a Google-translation somewhere.
Reliable Sources is my pet one, but honestly I think they are fine. Tweaking them will not result in anything major, and I feel that those policies have been developing well anyway. I suggest we agree that they are important and need to be further improved, but also that the community has been doing fine job on those and we should focus on other issues (like diminishing pool of contributors and community health issues, with their implications for quality). --Piotrus 19:59, 25 November 2009 (UTC)
I'd like to yes. The definitions at wp-en are a good start but certainly not sufficient. As our conversation above showed, the terms primary and secondary source are perhaps too confusing to use.
Emails, private conversations and other unpublished sources may be used by researchers, but they are certainly not accepted as reference in the scientific magazines (at least not in my field). I'm not sure if we should use them, if the literature itself prohibits it. This is in any case highly specialistic content. It is only likely to be an issue for our largest projects such as wp-en or wp-es.
I'll give an example why I think it would be good to have a manual for using scientific publications in natural sciences as a source. Any paper will have a certain structure (abstract-intro-method-results-discussion). Depending on the place in the paper, information may be a primary or secondary source. Alerting the user on how to read and use a paper could decrease the amount of unsound references, which is imho a problem at all larger projects.
I agree about emails and conversations, as they can get outside the realm of verifiability and reliability. So, are you proposing that we have different standards/manuals for each of a number of different subjects? Also, if we are following scientific journals and scientific principles then we perhaps should follow scientific principles of the definitions of primary and secondary research. Perhaps a matrix of what we will and will not utilize from scientific inquiry is in order?
While I do not argue with your points, my concern is that non academics may have trouble with academic guidelines, no matter how sensible they may be. How do we convey academic standards to non academics in a way they can understand?
Being a bit controversial -- we don't.
We set up a better editing environment, with better odds of more success, and changes that will probably catalyze and feed through into such areas and issues... and then focus on getting our basic raw editors and basic "not fit to eat" articles up to a basic standard. best odds all round. We can't afford to do much more, I think that has to be our first focus.
If we have a focus left over, then general improvements for established editors, and we can debate what else we can squeeze in. However much we value experts, and suffer when they leave, they leave because of general issues with the editing environment and poor editorship and disputes, all general issues. Focus on those, which also affect everyone else too. Don't try to make special "expert only" stuff, or fix specialist issues with academic sourcing. Wikipedia's standards and community aren't yet at a point where these specialist issues are a "top 5" focus.
Apologies if this sounds controversial or such. Discuss if needed, but after thinking through how to make best use of an opportunity like this, that's more and more where it seems to come down to. We have to get the basics needed for improvement. that means specialisms such as experts, FAs and the like are not a priority (this time around). Ruthless but ... sigh... hard choices.
@FT2: it doesn't sound controversial. Yet some projects, such as wp-en and wp-de, are certainly at an academic level in some areas. I agree these projects should focus on areas that are behind, but there's also the principle that writing a new article is five times less work than rewriting old stuff. I think there is a serious problem growing inside the larger projects concerning citation of (academic?) literature. At the English Wikipedia, longer articles often have hundreds of references to scientific papers. Following these refs, I often find they are simply unsound.
- They refer to papers that describe another thing;
- They refer to papers that try to disprove the very information they should verify;
- They refer to papers that are secondary sources for the information they should verify;
- They refer to papers that don't even contain the information they should verify.
I can live with the secondary/primary problem for the moment, but the large-scale malcitation at wp-en is a serious problem that undermines the project's intention to become a trustworthy source in education or academics.
Let's call it 'the curse of the {{citation needed}}-template'. Instead of citing the source used for writing the article, users tend to search for any citation on the internet that verifies the information in the article, just to get rid of the {{citation needed}}-template. They find abstracts of scientific papers (access to scientific literature on the internet is often limited to the abstracts) and cite without understanding or reading what the papers are about. It's a way of being lazy. Instead of going to a library, the contributor uses Google, reads diagonally through an abstract and voilà he has a ref. Anyone can see this method will never create an encyclopaedic source useful for academic use.
I agree that this isn't our top priority, and I agree the projects should first get to a plateau, a general level of quality, before they try to reach an academic level. Yet imagine the titanic task of having to check thousands of references for their value once the project is there. It's also a difficult task tat requires specialist knowledge about the subject. Most likely it's not going to happen at all. We simply don't have enough expert users in our communities that would be ready to help.
I don't like the idea of having different guidelines for normal and specialist contributors. The solution I'd rather see is simple. We could suggest to implement a guideline not to use a source when one does not understand it well. Only use a scientific paper as a source when you have a thorough knowledge of the subject yourself. If you don't have that specialistic knowledge, go to a library, search for a textbook (as recent and summarizing as possible - a truly sound secondary source) and use that instead. Such a guideline would be a very simple and effective way to make Wikipedia more acceptable for academic use.
Maybe a simple solution would be to have a category and sourcing guidelines ("Category:Articles whose core content should primarily be from current peer reviewed sources"), for those topics where the topic and availability of references means that core topic material should be cited primarily from strict and limited sources.
Users can agree by consensus if an article should move to that category. Bots can also be used to identify and suggest articles where most cites are already from a list of peer reviewed or other highly reputed sources, for review.
This fits within normal handling. Then for such articles, a bot reviews these every while and highlights those cites which are more than 6(?) 12(?) months from publication so our academic experts can check these and update them or ensure they're still currently valid.
So... essentially isn't this something easily done already, without needing the weight of a taskforce and quality project behind it? Any small group of users could propose a mechanism of this kind and the low level of bot, feed, or other infrastructure needed to support it.
The twist is, we can then use the processes being discussed elsewhere to heavily "push" this along. We can ask readers to ("Help Wikipedia! This topic has 3 cites more than 6 months old. Please update them if they're out of date for the field!") or put these onto a feed or user filters, for users interested in the topics. Cites are tagged with the date of publication so a Citation Check Wizard can ask on editing, "I notice you're about to add a 18 month old cite to this topic. Are you sure this still represents latest knowledge in the field?". And so on. Think about it :)
It also shows how the seemingly simple recommendations we've been discussing can also be used to improve quality in other ways and places - including even specialisms like expert sourcing, which was never previously considered. That's the kind of pervasive effect that we can aim for. Notice we were considering wizards and notifications in a "basic quality" and "new user" context, but once added they can open whole new quality doorways across the wiki.
I just read your post in the other thread and this Citation Wizard, wow, I really like the idea! An independent Citation Check Wizard would be great too.
The problem I wanted to point out in my last post (08:20, 27 November 2009) wasn't the problem of outdated sources (Actually, 18 months seems little time for me, 10 to 20 years is already quite good for scientific papers). It's the problem of wrong citations due to a lack of understanding in the subject (see the list of examples I gave in my last post - they are unfortunately quite common). The citation wizard could help, for example by giving advice depending on expertise:
- You are:
- An expert for this subject, knowing well the consensus in the literature -> either use primary sources (scientific papers) or secondary sources (textbooks), depending on how general the information is.
- Just interested but knowing a lot about the subject -> use secondary sources (textbooks), only use primary sources (scientific papers) if you are sure what you're doing.
- Just searching for a source that can replace the stupid template! -> try to find a secondary source (textbook or neutral website) which has the same subject as this article or section.