A couple of conversation starters
1 -> I agree, if we want to discuss this into detail we have to adapt or invent better terminology. FT2 gave 4 types of sources, we could easily add more. These types can again be split into peer-reviewed or not, etc. But see 3.
2 -> If a debate is raging in the literature, the fact should be pointed out. Giving each view proportionally requires knowing the relative importance of the view. There are ways to 'weigh' scientific literature: citation indexes and impact factors give a relative importance to each paper and magazine (there are disadvantages to this, but it is as objective as possible). If it's not possible to know what the balance between the different POVs is, we should imho still try to be as balanced and neutral as possible and cite the literature as precisely as possible. In a couple of years the debate may be decided and another user can then remove the refuted views easily. Another simple rule of thumb that should be used with care: 'when in doubt, don't'.
3 -> I personally have expertise with literature about natural sciences. I am able to write a manual/wizzard 'how to use and cite scientific literature about natural sciences'. Your examples are from medical sciences, and I cannot judge them. If we want to create easy-to-use manuals about every type of source, we would need an expert from each area.
I think we need to work from "standard" terms that already exist. I think that defining or redefining sources will lead to vast amounts of confusion in vetting information and in Wikipedia being taken seriously by the academic community. It's a bit like redefining what the word "red" means. Sure we can do it, but is there much point to it?
I agree about issues that are hot topics. However, not all debates happen in the literature. Some debates happen across many types of sources -- from listservs to personal email correspondence to lectures and discussions. So I don't think constraining ourselves to the literature does certain topics justice. I agree about assessing ways to weight relative importance however, if the sources are more difficult to track, that throws off the calculations. For example: Mothering magazine recently shared a link that stated the Canadian government was ceasing certain flu vaccines because it was shown that getting a seasonal flu vaccine shot made a person more succeptible to the H1N1 flu. This was passed out on a listserv. I contacted a friend in public health who contacted a colleague in Canada who had worked for the World Health Organization. He debunked the Mothering link in its entirety. If we are only tracking the literature, then how would I share this? In scholarly research, emails, conversations and interviews are all citation worthy and can be included. How do we handle information that comes from what is currently considered non-standard channels that academics already accept?
I don't know that every type of source needs its own manual, or that every type of information needs its own type of manual. In academia, there are various codified standards, but there are similarities between all of them. Do we want to consider citing differently per subject based on different types of content, i.e. MLA vs APA? I dont think so. So why would we have different formats or criteria for different types of information? That would create incredible confusion.
Perhaps this is where we need to go for definitions:
http://en.wikipedia.org/wiki/Primary_source
and where we need to ensure our documentation exists.
I would suggest focusing on editing this one document, and this one:
http://en.wikipedia.org/wiki/Secondary_source
and making these documents part of the basis for our quality initiative.
I would also suggest that we tighten up this document:
We have been talking around ideas, which is good, but it would also be good to start identifying the tools we need to leverage and create better quality. If we start a list of documents that will become the core of our effort, we can start to identify how and where we are going to implement our quality initiatives. Then later, we can hammer out exact definitions of primary, secondary and the like.
Anyone else have "pet" documents that should be on this list?
I'd certainly recommend the German version of that guideline. The French version is very limited compared to the English and German versions. If you don't understand German I can post a Google-translation somewhere.
Reliable Sources is my pet one, but honestly I think they are fine. Tweaking them will not result in anything major, and I feel that those policies have been developing well anyway. I suggest we agree that they are important and need to be further improved, but also that the community has been doing fine job on those and we should focus on other issues (like diminishing pool of contributors and community health issues, with their implications for quality). --Piotrus 19:59, 25 November 2009 (UTC)
I'd like to yes. The definitions at wp-en are a good start but certainly not sufficient. As our conversation above showed, the terms primary and secondary source are perhaps too confusing to use.
Emails, private conversations and other unpublished sources may be used by researchers, but they are certainly not accepted as reference in the scientific magazines (at least not in my field). I'm not sure if we should use them, if the literature itself prohibits it. This is in any case highly specialistic content. It is only likely to be an issue for our largest projects such as wp-en or wp-es.
I'll give an example why I think it would be good to have a manual for using scientific publications in natural sciences as a source. Any paper will have a certain structure (abstract-intro-method-results-discussion). Depending on the place in the paper, information may be a primary or secondary source. Alerting the user on how to read and use a paper could decrease the amount of unsound references, which is imho a problem at all larger projects.
I agree about emails and conversations, as they can get outside the realm of verifiability and reliability. So, are you proposing that we have different standards/manuals for each of a number of different subjects? Also, if we are following scientific journals and scientific principles then we perhaps should follow scientific principles of the definitions of primary and secondary research. Perhaps a matrix of what we will and will not utilize from scientific inquiry is in order?
While I do not argue with your points, my concern is that non academics may have trouble with academic guidelines, no matter how sensible they may be. How do we convey academic standards to non academics in a way they can understand?
Being a bit controversial -- we don't.
We set up a better editing environment, with better odds of more success, and changes that will probably catalyze and feed through into such areas and issues... and then focus on getting our basic raw editors and basic "not fit to eat" articles up to a basic standard. best odds all round. We can't afford to do much more, I think that has to be our first focus.
If we have a focus left over, then general improvements for established editors, and we can debate what else we can squeeze in. However much we value experts, and suffer when they leave, they leave because of general issues with the editing environment and poor editorship and disputes, all general issues. Focus on those, which also affect everyone else too. Don't try to make special "expert only" stuff, or fix specialist issues with academic sourcing. Wikipedia's standards and community aren't yet at a point where these specialist issues are a "top 5" focus.
Apologies if this sounds controversial or such. Discuss if needed, but after thinking through how to make best use of an opportunity like this, that's more and more where it seems to come down to. We have to get the basics needed for improvement. that means specialisms such as experts, FAs and the like are not a priority (this time around). Ruthless but ... sigh... hard choices.
@FT2: it doesn't sound controversial. Yet some projects, such as wp-en and wp-de, are certainly at an academic level in some areas. I agree these projects should focus on areas that are behind, but there's also the principle that writing a new article is five times less work than rewriting old stuff. I think there is a serious problem growing inside the larger projects concerning citation of (academic?) literature. At the English Wikipedia, longer articles often have hundreds of references to scientific papers. Following these refs, I often find they are simply unsound.
- They refer to papers that describe another thing;
- They refer to papers that try to disprove the very information they should verify;
- They refer to papers that are secondary sources for the information they should verify;
- They refer to papers that don't even contain the information they should verify.
I can live with the secondary/primary problem for the moment, but the large-scale malcitation at wp-en is a serious problem that undermines the project's intention to become a trustworthy source in education or academics.
Let's call it 'the curse of the {{citation needed}}-template'. Instead of citing the source used for writing the article, users tend to search for any citation on the internet that verifies the information in the article, just to get rid of the {{citation needed}}-template. They find abstracts of scientific papers (access to scientific literature on the internet is often limited to the abstracts) and cite without understanding or reading what the papers are about. It's a way of being lazy. Instead of going to a library, the contributor uses Google, reads diagonally through an abstract and voilà he has a ref. Anyone can see this method will never create an encyclopaedic source useful for academic use.
I agree that this isn't our top priority, and I agree the projects should first get to a plateau, a general level of quality, before they try to reach an academic level. Yet imagine the titanic task of having to check thousands of references for their value once the project is there. It's also a difficult task tat requires specialist knowledge about the subject. Most likely it's not going to happen at all. We simply don't have enough expert users in our communities that would be ready to help.
I don't like the idea of having different guidelines for normal and specialist contributors. The solution I'd rather see is simple. We could suggest to implement a guideline not to use a source when one does not understand it well. Only use a scientific paper as a source when you have a thorough knowledge of the subject yourself. If you don't have that specialistic knowledge, go to a library, search for a textbook (as recent and summarizing as possible - a truly sound secondary source) and use that instead. Such a guideline would be a very simple and effective way to make Wikipedia more acceptable for academic use.
Maybe a simple solution would be to have a category and sourcing guidelines ("Category:Articles whose core content should primarily be from current peer reviewed sources"), for those topics where the topic and availability of references means that core topic material should be cited primarily from strict and limited sources.
Users can agree by consensus if an article should move to that category. Bots can also be used to identify and suggest articles where most cites are already from a list of peer reviewed or other highly reputed sources, for review.
This fits within normal handling. Then for such articles, a bot reviews these every while and highlights those cites which are more than 6(?) 12(?) months from publication so our academic experts can check these and update them or ensure they're still currently valid.
So... essentially isn't this something easily done already, without needing the weight of a taskforce and quality project behind it? Any small group of users could propose a mechanism of this kind and the low level of bot, feed, or other infrastructure needed to support it.
The twist is, we can then use the processes being discussed elsewhere to heavily "push" this along. We can ask readers to ("Help Wikipedia! This topic has 3 cites more than 6 months old. Please update them if they're out of date for the field!") or put these onto a feed or user filters, for users interested in the topics. Cites are tagged with the date of publication so a Citation Check Wizard can ask on editing, "I notice you're about to add a 18 month old cite to this topic. Are you sure this still represents latest knowledge in the field?". And so on. Think about it :)
It also shows how the seemingly simple recommendations we've been discussing can also be used to improve quality in other ways and places - including even specialisms like expert sourcing, which was never previously considered. That's the kind of pervasive effect that we can aim for. Notice we were considering wizards and notifications in a "basic quality" and "new user" context, but once added they can open whole new quality doorways across the wiki.
I just read your post in the other thread and this Citation Wizard, wow, I really like the idea! An independent Citation Check Wizard would be great too.
The problem I wanted to point out in my last post (08:20, 27 November 2009) wasn't the problem of outdated sources (Actually, 18 months seems little time for me, 10 to 20 years is already quite good for scientific papers). It's the problem of wrong citations due to a lack of understanding in the subject (see the list of examples I gave in my last post - they are unfortunately quite common). The citation wizard could help, for example by giving advice depending on expertise:
- You are:
- An expert for this subject, knowing well the consensus in the literature -> either use primary sources (scientific papers) or secondary sources (textbooks), depending on how general the information is.
- Just interested but knowing a lot about the subject -> use secondary sources (textbooks), only use primary sources (scientific papers) if you are sure what you're doing.
- Just searching for a source that can replace the stupid template! -> try to find a secondary source (textbook or neutral website) which has the same subject as this article or section.