Problematic articles
FT2 you misunderstand my reference. I am not being dismissive of other articles. Nor am I identifying the articles to which I refer as those specifically in the sciences.
As a practical heuristic, you may be correct that i am talking about disciplines that privilege peer-reviewed articles. In effct, that may be what I was talking about. But I was deliberately speaking in a more abstract way, in order to be more inclusive.
The point is that quality articles are balanced. The question is, how do we recognize when an article has achieved balance. And my point is that different kinds of editors will focus on different kinds of balance.
My point is simply to distinguish between two kinds of standards for recognizing an NPOV/balanced article.
There are some topics where debates among specialists are the same as the debates among non-specialists. In these cases, it does not matter whetherone is a specialist or a non-specialist, either kind of person can tell whether the different sides of the debate are being represented.
But there are other topics where debates among specialists are quite diferent from debates among non-specialists. As an example I suggest "race and intelligence." For most non-specialists the debate is whether intelligence is nature or nurture. In this debate there are two sides. But for specialists, this is not the debate. Sociologists do not deny that a portion of intelligence is inherited. But they are focussing on how different aspects of th environment, at specific times in a child's development, may affect IQ scores. Among sociologists, there are a number of debates most of which have more than two, maybe even several, sides. Similarly, geneticists who study IQ scores of twins do not deny that a portion of intelligence is environmental. In fact, one of the major debates right now is how the fetal environment of twins may affect intelligence. This is not a nature-nurture debate, it is more complicated than that.
So when it comes to race and intelligence, we can just ask a bunch of regular editors if they feel the article is balanced. My bet is, if the article says that some people believe that differences in average IQ between Blacks abd Whites is natural, and some people believe it is caused by different environments, and some people believe it is a combination of the two, then a groups of average editors will agree that the article complis with NPOV and is balanced.
But if you ask a bunch of sociologists and geneticists if they think the article is balanced, I bet they would be speechless. They would say that the article doesn't even talk about the actual debates scientists are really arguing about.
If the article discusses the different debates among sociologists as to what aspects of the environemtn are most important, why scores on tsts begin to diverge at a certain time, and so on, and if it discusses the differences in fetal environment for identical or fraternal twins and how that might affect intelligence, then sociologist and geneticists can judge whether the debates are bing represented in a balanced way and whether any fring views are being given too much weight.
But if you asked a bunch of average editors whether the article is balanced, they simply cannot give a well-informed answer. In these cases, average editors can judge the clarity of the article, its style and presentation, but they just cannot jusge whether it is balanced.
We would need experts on the topic to judge.
Is the key here peer-reviewed journal articles? I honestly do not know. maybe. But maybe not I would have to think about other examples. Certainly, the one example I gave is one where research is presented in peer-reviewed journals.
But do you remember when Jon Stewart put an end to CNN's "Crossfire?" He was the boy you finaly said that the emperor has no clothes. Of course, FOX news, and the O'Reilly Factor continue to claim to be fair and balanced. We all know what is going on, right? These shows cover politics and public policy and always assume there are two sides, right and left, so if they have someone from the right speak on - say - health care reform, and someone from the left speak on health care reform, then they claim to be "rair and balanced." The problem here is not just that FOX favors the right in all sorts of little ways. The problem is that there are many political and public policy issues where there are not just two views, left and right.
So this leads me to think that my point is not just limited to issues covered by peer-reviewed journal articles. I think Wikipedia has a lot of articles on political or public policy issues, and also articles on history (e.g. the article on Fascism, which is often mired in conflict) where people think there are just two views. It could be "left versus right" or it could be "religious versus atheist." There are all sorts of articles in which regular people (the pool for most editors) may actually have a personal stake (they are devout Christians or they are atheists) and they see certain issues in terms of two sides (you agree with me or you do not) so when they work on articles they think there are just two views, and if both views are represented, the article is balanced.
But in fact this just often is not the case, and many people just are not aware of the other sides. In some cases this is very simple to explain - some people who edit the Jesus article really believe that all editors are either Christian or atheists and that these are the two views that need to be represented, equally. But there is a community of scholars - trained in history, religion Biblical studies, etc - who look at the New Testament neither as Christians nor as atheists but as historians, and they have their own set of debates. I cannot tell you how many times I have had to explain to some other editor that X is not representing a Christian POV or an Atheits POV but the POV of an historian.
So I am identifying topics where there are popular debates and also academic debates.
But this is not just popular versus academic. When it comes to health care reform, it is not just a matter of Democrat versus Republican, and it is not just (Democrat versus Republican) versus (academic) - there is the insurance indusry, the AMA, and a host of public policy experts who are having their own debates, who are not necessarily academic.
FT2, I agree with you that balance is one criteria for a quality article. But the qustion is, who can assess whether an article is balanced or not? That depends on where the debates are located. If the only debate is in Congress, it may be a debate between democrats and republicans. If the debate is public (let's say, over abortion) there can be a debate between right and left, between religious and atheist. I think that with many of these debates the average editor, or a random selection of editors, can recognize whether the article is balanced.
But if the debates are located in academica, or in public policy circles, the average Wikipedian may simply not be able to judge whether the article is balanced. or worse, the average Wikipedian might try to reshape an academic debate so that it has onoy two side (left versus right) when in fact it has, say, five sides.
So I hope I have explained my concerns clearly. my concerns may apply to a small percentage of Wikipedia articles, but if college or high-school students are relying on Wikipedia these are the articles they will be looking at. And high school and college students are not qualified to recognize whether the articles are balanced, because thse readers do not know all the major debates and the different sides of the debates. And the average Wikipedia editor isn't qualified either.
So I agree with you that such articles have to be balanced but the mechanisms we may rely on for the vast majority of our articles won't work here.
To be constructive, can we attempt to make a classification to discriminate between these situations?
I spelled out my concerns anecdotally, because they arise out of concrete situations I have been in. I think the first question is, do they resonate with others? If so I would welcome someone else's attempt to operationalize the distinction or begin a typology.
For me the basic point remains this: there are some topics that are so complex and dependent on specialist knowledge that no group of average editors, no matter how well they understand our NPOV policy, can judge whether the article is balanced - represents all significant views proportionately and without giving weight to fringe views. We need to keep this in mind if we make "balance" a criteria of quality.
I agree with this basic point, but my problem is - can we make it constructive? Can we somehow define the class of articles which will be flagged "hey, this article is very much vulnerable: it is easy to make it look neutral, but the real neutrality can only be checked by experts coming from two or more groups with different backgrounds"? If we can, I would say: fine, let us tag these articles and FOR THEM the baseline quality would include this tag instead of NPOV checking (or the flag removed if it had been checked), and otherwise satisfy the same criteria.
An observation, and an admittedly minor suggestion. Such articles are often sources of conflict because experts and non-experts have different views as to how to achieve NPOV. Fora start, how about creating a new template. Right now we have Template:POV
As a small first step, how about a new template that says something like, "This article has neutrality problems that require the ettention of experts in the field?"
Other suggestions for wording, welcome.
Fine by me. My lengthy reply above was in response to FT2 - he was critical about one gross distinction I made between scholarly and non-scholarly articles so he is more attuned to or sensitive to the full range of types of Wikipedia articles. Along the lines of your wish to be more constructive, maybe FT2 has some ideas about how to relate my point to a more precise and accurate taxonomy of Wikipedia articles ....
I've worked on some featured quality articles, and I'd definitely agree that everything you said is part of our highest quality standards. The question is if that's a baseline, or is that the highest possible standard? Because as much as I hate to say it, CNN is basically consumable. No respectable academic would say they were neutral in the sense that they cover all respected points of view and separate fact from fiction. But in the sense that they present two sides and usually don't pick one of the other, they are neutral.
I think we may need to discuss what we mean by "baseline" anyway. A phrase that comes by is "safe enough to eat". Not fine dining safe, or even organic foods safe. It might mean "McDonald's safe". "Won't put you in the hospital safe".
I think more specifically, we want to know what we're going to actually DO when we have the baseline.
Is it just a way to mark articles for readers, to give them adequate warning? Is it a way to mark articles that -- if they can't reach the baseline -- should never have been added to the encyclopedia in the first place?
No, I think FA are way above the baseline quality we are trying to define. Baseline is indeed to mark an ok article, but not necessarily a high-quality article. Articles which are below the baseline quality are in some way problematic.
Well, articles below the baseline includes stubbs and a good deal of other articles. I think we need to balance two diferent stakeholders: readers who expect a certain level of quality, and yes, we need more refined ways of communicating to readers how accurate or complete an article is. Editors (meaning, people who actually edit) are the other stakeholders and ideally - and in the old days - for them Wikipedia was always a work in progress, articles were always in some need of improvement. We need to communicate different kinds of information to different stakeholders.
Well, pal, I am with you on that!
Wikiprojects currently have their "A, B, C" system.
Are you suggesting delegating to them an overhaul of how these marks are determined with benchmarks?
Or are you suggesting refining the GA system?
I can't imagine serious opposition to some proposal along these lines, from anyone.
Either way would be fine with me. In practice, we probably need to combine both, since it is unreasonable to vote for every stub to determine it is a stub. After flagged revisions have been introduced, editors are actually entrusted by the community to decide which articles are sub-standard and which are ok, and we can tie the baseline quality criterion with this level - then an editor decides whether the article is of baseline quality. May be for some other levels we may use trusted users as discussed in another tread,; for some problematic cases we may even need real experts. I believe some of this stuff can be delegated to the Wikimedia-wide-projects, but I do not think it is a good idea if the projects decide for instance what is baseline quality.
Something I raised in another thread: only featured articles are determined by consensus. Every other rating scale is a matter of going into the talk page and changing it yourself. (Or in the case of "Good Article" and "A-Class", asking one other person to do it for you.)
I think this is a great idea. But the logistics are tricky.
- Who is going to sign off on "baseline quality"?
- How do we ensure that the person/people signing off have correctly understood and applied the baseline standard?
- What does "baseline quality" mean, if it's only half way to "featured quality" (comprehensive, thoroughly researched, neutral not just in tone but in substance)?
I'm stumped, to be quite honest. But I want to see this work.
Maybe "baseline quality" can mean: complies (to the best of our knowledge) with NPOV, V, and NOR; if an academic topic perhaps draws on a certain minimum number of sources, or we can have a checklist - identifies major debates, identifies key historical moments?
One thing I am starting to think is, how can the Wikiprojects be of use? Well, one criteria for baseline quality could be: to give the relevant wikiproject a foothold or foundation for taking it to the next step. This would involve in effect giving wikiprojects assignments. But I think it is perfectly reasonable, for example, to ask Wikiproject to develop a bibliography of works published by major university presses or in the leading journals in the past five years. Just having such a bibloigraphy would help us know how complete the article is, and provide future editors with something "to do" (i.e. start reading through that bibliography to add to the article). This would not be a full-scale "good article" or "featured article" drive, it would be something short of it - but still something, something we have been lacking (a systematic algorithm of stages through which to move an article, with "to do" lists at critical moments e.g. first develop a bibliopgraphy; then review the literature; then revisit NPOV issues.
Just an idea.
I think baseline quality is best described through qualifiers and exceptions.
- "Not entirely referenced, but the lead is verified in reliable sources."
- "Not a comprehensive survey of the subject, but presents at least two perspectives in a neutral tone."
- "Not complete, but stable."
- "Does not violate what Wikipedia is not."
If we're going to use a consensus-based process (e.g.: ask a WikiProject) to judge the baseline, we may as well just make it a nomination-based process like a "featured article". Someone nominates the article for the baseline, and a few experienced editors go through the article to check it. That already takes a long time. Once you start asking people to compile entire bibliographies to judge the baseline, we may as well ask them to write a featured article. You're asking the evaluators to do the work of the editors, which is going to make this a huge burden in practice.
You misunderstand me. I di dnot say that someone has to compile a complete bibliography to make an article base-line. I said that whatever makes an article base-line it should be enough that the appropriate Wikiproject can move on to the next step, one of which could be coming upo with a bibliography from the past five years (which is not "complete") - a topic has to be clearly established and defined to make that possible. Isn't it obvious that for people to come up with a bibliography (as a basis for the next step, more research) the bibliography cannot have hundreds of items? This means that a clearly defined encyclopedic topic must first be established. ANd I consider this to be one criteria for "base-line."
I see nothing wrong with defining grades of quality not just in terms of what the article fulfills, but what future research thus becomes definable.
I have no objection to the other criteria you suggest.
Oh, that makes sense. Once you have the baseline, there would definitely be additional work to be done. If the baseline is as comprehensive as an expanded dictionary definition, the next step would be finding the research to flesh it out.
Evaluating the baseline is still the challenge though. Even the featured article process takes literally weeks, often a month or more. It's easy to see that this will make it impossible to evaluate more than a few dozen articles per year, in a pool of millions. That's just as bad as coming up with a baseline quality standard that can just be stamped on by any passerby who likes the article.
I agree with you. That's why I think a checklist like
- no clear NPOV violations
- no clear V or RS violations
- no clear NOR violation
is a start. You point to two extremes but i am sure there is a middle ground.