Ensuring high quality sources where needed, especially science/academic topics (narrow focus)

Ensuring high quality sources where needed, especially science/academic topics (narrow focus)

Edited by author.
Last edit: 08:38, 13 December 2009
Themes and ideas from Archive 1:
  1. Obsession with "citing from anything" rather than citing from high quality; leads to a "plague" of poor quality or even plainly incorrect citations.
  2. How to better handle it when experts disagree or there isn't a clear obvious "mainstream view"?
  3. Quantity of cites isn't an issue; quality and poor usage (especially to make a point) is. Citing is massively gameable by POV warriors.
Possible approaches:
  1. Bibiliography of high quality cites? Or preferred sources in given topic areas? (Note that some reliable sources mix high and low quality material)
  2. Tagging articles that need it or where sufficient sources exist, as needing high quality cites only, then using software (bots, wizards etc) to patrol and check cites are appropriate, up to date, current knowledge, etc.
  3. Obtaining access for some subgroup of editors to subscription services (newspapers, journals, etc) - must be a limited pool, should not appear "elitist".
  4. Clarify terminology to match mainstream or be consistent (primary/secondary/tertiary)
  5. Concept: Citation Check wizard and comments.
FT2 (Talk | email)08:28, 13 December 2009

Also noting the idea of WikiCite - worth reading.

(Implementation notes - broadly, it involves a table of "indexed facts", a bit like a wiki-bibilography but fact based not source based. Tag these facts and allow searching by text, topic, tags, categories etc to make them findable: most people understand tagging these days. Then every fact asserted in Wikipedia should have a cite backing it. Those facts stated that don't, feed into "help Wikipedia!" in the user/reader interface for later addition.)

FT2 (Talk | email)08:32, 13 December 2009
 

If I may, I'd like to shortly address the problem of "citing from anything" again. Slrubenstein has given an interesting and long comment on my essay about the definition of quality at my talkpage at meta. His central point was that he misses "accuracy" in my definition. I didn't reply yet, but I think it's not really absent. The two of us mean the same but use different words.

Accuracy is partly included in what I call "verifiability". We should not just use (the right) sources for content, we also use them for the structure, balance and even titles of our articles. For general subjects such secondary sources can include traditional encyclopaedia; more specialized subjects need secondary scientific sources. Most battleground subjects like global warming, the history of Israel, evolution, etc. are actually fields of scientific research too. There are often good secondary scientific sources for these subjects, even if the "true believers" on both sides will often not accept them. However, a wiki-community should not reinvent the wheel. No original research, no citation needed in every second sentence. The community should study the secondary sources, their titles, chapter subdivisions and the balance in attention given to different views. This should be the normal approach, yet it is taken rarely.

Woodwalker10:56, 15 December 2009
 

I think we should stat very clearly that printed peer-reviewed sources have priority over websites, even if they are not accessible for everybody. On the other hand, this of course invites problematic users to make up references of citing real articles for the info which is not contained over there. This behaviour is disruptive and should be labeled as such.

Yaroslav Blanter19:36, 17 December 2009
 

I already placed this above, but now think it is more appropriate here - sorry, maybe I still don't understand the liquid thread system. here goes:

I think we need to be honest about two ways that sources are used - consider it a "push model" versus a "pull model." In Wikipedia, I think most editors use a "push model" - they have a view they wish to include so they then go about lookin for reliable sources they can use to support their view. I don't like this but see no way to stop if, and many of FT2 and others' suggestions may help us develop a better set of criteria for "reliable sources" - someone on another threa, I apologize that I do not remember who, singled out better sources and I cannot agree more. So I value FT2's approach here and will see if I can add anything to it.

For now I want to point out that this is not the approach most academics take. This other approach ("pull") is really a combination of approaches. One starts with identifying the best sources, and then finding out what views are represented in these sources. Thompson's web of knowledge has one way for ranking peer-reviewed journals, there is at least one other company that produces the same statistics but calculated in a different way. The key statistics are impact factor and immediacy index. Immediacy index basically measures how soon it is before other articles are published that cite articles published in this journal. Impact factor has to do with the total times articles from the journal are cited elsewhere. So one way to go is to identify which journals for a given discipline, subdiscipline, or topic, score highest on impact factor and immediacy index, and give greater weight to articles published in those journals, regardless of what view (e.g. their interpretation of why the Maya empire collapsed, or whether Franco was a "fascist").

We can easily ask people on different Wikiprojects to investigate the journals on their topic o discipline and annualy revise th impact/immediacy stats, and actively encourage editors working on WP articles to seek out journal articles from those journals and from those articles decide what the contents of the WP article should be or how views should be weighted.

Another approach is to start with some articles from major journals or books from major university presses and see what books or articles are regularly or most frequently citd in other books and articles, as a way of inferring wich books and articles are most important.

This approach does not start with a view in search of a source, it starts with sources in search of views. But in my experience this is how most academics research literature reviews, they try to find out what books and articles everyone else cites, and give more weight to articls in the most prestigious journals ... and then, whatever those books and articles are writing about, what issues they are debating, what are the dominant sides in any debate, that then determines what gos into a lit. review.

I think a big problem with Wikipedia is that many editors do not know how to research in this way or do not have the resources (books and journals readily available at their library). Perhaps we can get the Wikiprojects to help in encouraging people to research for articles this way. Any good university library has the Thompson web of knowledge database and perhaps one or two others that provide impact factors and immediacy indexes so why not start a project of: ranking journals, and making sure that Wikipedia has articles on all the top ranked journals and provides these two stats - the databases today will also show what the stats were in previous years. By the way, these indexes are only for the natural and social sciences. I am not sure why they do not cover the humanities but my guess is that the journals are far more specialized with lower readerships and perhaps the raw data is insufficient to calculate significant stats.

The thing is, there are LOTS of peer-reviewed journals; among academics, their rejection rates and where they dtand on these indexes are the real measures of prestige. My point is NOT to ban the use of cites from low-scoring journals. My point is to encourage editors to turn to high-ranking journals with an unbiased eye to discover what researchers are saying on a topic without looking for a particular view they already wish to put in an article.

Slrubenstein15:24, 20 December 2009
 

I concur with Slrubenstein here. His push-pull idea is interesting.

Comparing the way articles on Wikipedia are constructed with the way real (traditional) encyclopaedists work, there is a difference that leads to bias in Wikipedia content. The right way to construct an article would be:

  1. Consider which sources the body of the article should be based on. Try to find at least 5 of such sources (in the end, many other sources can be used as references to verify more specific statements).
  2. Write down and compare the structure these sources use to present the information and then decide what the structure of the article should be.
  3. For controversial subjects, examine what weight the >5 main sources give different views, the Wikipedia article should reflect this.
  4. Only then we are ready to start writing!

Instead, Wikipedia articles are often an add-what-you-like project. This way of working leads to bias and unclarity.

Woodwalker16:37, 22 December 2009
 

This is a good algorithm - there may be othe equally good ones. So what we are doing is developing algorithms editors can follow in working on articles. This is taking us VERY far from the original injunction, "BE BOLD," but I think that this is a good move to make. If we want to have a higher quality encyclopedia, we need to instruct some people in how to do research. We cannot force anyone to follow any of these algorithms, but we can make them available.

Here is a VERY "bold" idea. Make people go through a workshop before they can become registered users. The workshop need not be long or hard. We can shoot for a fifteen minute to half-hour process. In it people would have to read core policies (NPOV, V and NOR) and one or two or three of these research algorithms and then answer and pass a short and simple quiz. When I served on my university's Human Subject's Committee, there was a computerized program that did just this: it explained the reasons that Congress passed Human Subjects legislation, what the core elements of the legislation is, and what the basic tasks of HS committees is. I had to take an extraordinarily simple quiz that basically showed I had read what was on the computer screen (a lot of it through animation!).

I know this is very bold, but I think it is reasonable to make sure people have read and understand the basics of three core content policies and perhaps these research algorithms too, before letting them register since being a registered user is increasingly a threshold required for editing many articles.

I prefer this approach over creating new hierarchies of expertise. Wikipedia remains the encyclopedia anyone can edit, because ANYONE can become a registered user. But given the trust we put in people's edits, it is fair to ask people to go through a short tutorial before registring.

I emphasize I am talking about a bsic ans simple tutorial.

I think this would do a LOT to improve quality just by ensuring a community of registered users with some higher level of self-consciousness of what it means to edit an encyclopedia.

Slrubenstein18:14, 22 December 2009

Actualy, this is smth I have never thought about but it seems like a very good idea - I mean, not necessarily making people to go through the workshops but generally (a) stating that quality contribution typically require certain skills, which are obvious to people knowing what the academic research is but may be less obvious to others; (b) providing opportunities to learn this culture - via workshops or other means; may be even live workshops organized by chapters.

Yaroslav Blanter20:11, 22 December 2009
 

I'm as sympathetic as anyone to the idea that "not everyone can be a helpful editor". But I think making people go through a 15 minute workshop just to create an account is a bad idea. It's far too restrictive. Anyone in business will tell you that you don't want to create hurdles at the signup stage. You want to get them in the door, enjoying whatever it is you have to offer, as quickly as possible.

A *voluntary* workshop is a good idea though. Even a workshop that could be required to go from "average editor" to "editor who has the power to participate in more serious work" -- like working on featured articles, or creating new articles. That's worth discussing too.

But we can't screw up the experience for the first 15 minutes. That's the most critical moment for getting a volunteer's support.

Randomran21:32, 22 December 2009
 

Well, I have already prooised we create a stage between "registered users" and "admins" (for page protection purposes) - if going through this tutorial were required of the new proposed stage, i would say fine.

But I am not so quick to back down concerning regisered users. Remember, we are not talk about someone's first contact, for most people that is as an anonymous editor and this does not stop people for being anonymous editors.

In the meantime, I daresay we have more registered users than we need. Or I should say, we have more registered users who degrade quality than enhance it - they do notget our basic policies and shirk from doing research. Even ifmy suggestion alienated half of them, even two thirds of them, if the rmaining, who registered, when through the 15 minute tutorial I suggested, Wikipedia would be a far better place than it is today.

Folks, is quality a problemn or is it not?

If it is, don't we need to make soe changes?

Slrubenstein22:09, 22 December 2009
 

@Slrubenstein

Well the answer is more how to achieve building of Good Quality Article in collaborative & enjoyable way.

Good Quality Article starts with Verifiability, Neutral Point of View, No Original Research and No Systemic or Cultural Bias that lead to "Reference-Driven Content" editing style which is one of "least enjoyable" way to build an article.

Collaborative because over reliance on a handful of "Key Contributors" will turn into weakness if few of them leave Wikipedia. Moreover there is also the Quality Article "Unkeep", the more Quality Article an editor create and more time he/she most devote to maintaining those articles quality level which is less time to create more quality content. So more people involved and will lessen each individual burden.

Enjoyable because contributions to Wikipedia is voluntary, we should be careful with exigences and other requirements. Setting the hurdle too high will just have people quit and not coming back for a long time.

For how i read the current trends of this Task Force it's leaning on more constraints and more reliance of "Experts" with near no acknowledgment of the fun factor.

KrebMarkt08:22, 23 December 2009
 

So what? KrebMarket, no proposal here is going to prohibit most editors from editing WP the way they have been editing. If most people want the "fun factor" and edit the way you say, well, they will continue doing that.

Doing things that way has produced the encyclopedia we have. This encyclopedia has its virtues. It also has its flaws. This group is tasked to address one of its flaws. If the old way of doing things worked, we wouldn't have these flaws. Ergo, we need a new way of doing things.

But has anyone suggested that the new way displace the old way? Why does it have to be either-or? Why not this and that?

I did not see you active before but if you are on the task force, by all means, propose something you think would be better.

Slrubenstein13:43, 23 December 2009
 

I like the idea of the 15 min workshop, but not as a requirement to become an editor. It's possible to have it as a requirement to become a 'senior editor' though.

Woodwalker16:26, 23 December 2009
 

This would go along with creating a level of protection that is below admin but above registered user.

Slrubenstein16:54, 23 December 2009
 

The senior editor idea could work. Wikipedia is not supposed to be elitist or restrictive, but it IS a meritocracy in the sense that good edits are more likely to survive the bad ones (and the same with good editors). There might be objections to putting hard limits on user activity. But it is much more desirable to ask editors to "prove" and "earn" their merit, and certain privileges that come with it.

I think the workshop is a decent idea for a senior editor. But it depends on what privileges we're giving to senior editors. If we're actually entrusting them with certain tools, or giving them a little more weight in certain situations, then they need to actually have a track record of action. It's one thing to take a workshop on what you're supposed to do. It's another thing to actually do it.

Randomran18:16, 23 December 2009
 

I view the workshop or tutorial as an entry-device only, just a first step. I don't object to making it a requirement for senior editor but I still see nothing wrong with making it a requrement for all future registered users. The on-line tutorial I did for Human Subjects Committee was pretty easy and about 20 minutes and I really don't see many potential registered users being turned away from it. Yeah, it is a screening device, but that is not its main function its main function is to ensure that the growing mass of registered users have a basic understanding of NPOV, V and NOR, and the most basic understanding of research. Given Wikipedia's claims to be the first point source of knowledge, and our appreciation of quality problems, and the effect of registered users, just by sheer weight, I think this is justified. It sends a message about what is important, it heightens awareness and sensitivity - not for all, but it will, for many.

I agree in porinciple about track-record but I am very concerned about any reliable rmeasure. There are people with 40,000 edits who are crappy editors. I have never been involved in a GA or FA - does this make me a bad editor? We have lots of metrics we could use, I am just saying I don't really trust the.

And I don't really trust the kind of vetting process we have for admins - I am still wary of setting up a hierarchy at Wikipedia. So I think we need to discuss more, to what uses such a hierarchy would be put. And as ourselves if the same effects cannot be achieved through other means.

For example, my idea of a kind of protection that requires three editors to agree for an edit - and we can refine this, say three editors who have been watching the article for at least six months. Such a kind of protection requires no greater hierarchy than this extremely localized one (people who have been watching the article for some time), the real point is to encourage consensus based edits by people who have demonstrated an interest in and commitment to the article. If it means a newbie 9even one with a PhD) presenting it on the talk page and there being some discussion, well - why not do this? Won't this ensure a higher level of quality AND stability? Slrubenstein 18:26, 23 December 2009 (UTC)

Slrubenstein18:26, 23 December 2009
 

@Slrubenstein

Sorry to reply you this late, liquid thread hate me :(

Before going further i have to provide some evidences that i have a minimum of understanding in Reliable Source and Quality article building so my opinion won't be dismissed. Tired to pass for the village idiot.

As member of En anime/manga project my specialty is Reliable Source hunting whatever they are primaries, secondaries or tertiaries. I usually provide French RS resources and go as far as translating reviews into English. Furthermore i have sometime "Wall of references" duty. I think the number of references i put in articles should be over 1000.

Quality article writing wise i participated to one of the few GAs with near-zero English reliable source proving again that not-in-English doesn't mean not notable and not a subject for good article. I also took part of a GA-rescue. Related to quality, i happened to get involved in the A class at en:WP:ASSESSMENT leading to en:Wikipedia talk:WikiProject Council/Assessment working group which didn't produce anything worth mention.

My editing stance can be found here

Now back to current discussion subject which is ensuring the quality of the sources used.

Depending of the nature of the projects and the subject of the article possible avenue of reliable sources will differ greatly ranging from academic paper to blog post.

Sources have two aspects Credibility and Weight.

Credibility is always limited to an area of expertise.

Weight is related how much importance should be given to the "Point of View" expressed by the source.

Reading the proposals made i should ask, what is your definition of a "high quality source" ? A Credible Source with a lot of weight in its area of expertise?

Here few ideas:

  • Most projects have a list of reliable sources that fall within their scope. We should make a global database out of those lists so it could be used by a wider population of editors.
  • In the same way we should also report non reliable source to this database, we often find non RS during peer reviews or GA reviews.
  • Bolder we can also exchange the list of RS between the different language projects which would also help editors fighting the systemic bias.

What can't be automated is the "weight" given to a source. It's each editor responsibility to balance the weight of the various Credible opinion on a subject to be the closest to NPOV.

KrebMarkt18:54, 23 December 2009
 

KrebMarkt is right that there are already a bunch of lists at different WikiProjects. Compiling them together is a good idea. But I stand behind the idea of turning that list into searchable results, in the form of a specialized web search engine.

Yes, in an ideal world, people would be searching books too. But being able to quickly see how a concept has been covered in the BBC or even CNN would be a huge improvement. The biggest problem with quality isn't that people are misusing or misweighting good sources -- that's an important problem, but it's rare. The biggest problem with 90% of articles is that they aren't using any sources at all (or that they've become a soapbox to rehash what someone said on some blog).

Randomran19:02, 23 December 2009
 

A such search engine will help especially for fact checking.

Downside is editors putting putting pieces of "opinion" from various RS without thinking the weight issue.

During a peer review an editor asked why XYZ reviews should be mentioned and given weight. Someone answered by saying that those reviews came RS websites figuring in the anime/manga project list of RS. Using list of RS is great but understanding why those sources ended in the list is even better.

About 90% of articles without sources, we should do something to make people understand that it isn't an acceptable norm. Once during an Afd i found an editor arguing that a not sourced article isn't a big deal and like many others ones. I wanted to slap him.

KrebMarkt19:55, 23 December 2009
 

I value Kreb Mrkt's comments. I do not fully agree with Randomran.

It is a shame we do not have stats on how many pages people hit. I think there are articles requiring specialized knowldedge that are relative stale becaus emost people do not care. (the Lorenz equations?)

I think there are articles that require specialized knowldedge and draw fring thinkers, but the dedication of three or four people who have been watching if for years reverts what crap is introduced immediately the the article is virutally stable (evolution).

There are articles that rely on fan knowledge and are not sourced (TV shows, movies).

Then there are articles that are controvesial, and either go through periods of intesne instability when o9lder editors are wrn down, or are stable due to a few thankless people. Examples: race, race and IQ, Jesus.

It is these latter articles I care about most because they are of interst to many people (een if they themselves are a small portion of our aticles) and because they are often shit.

And these artiecles are very well sourced. But sources are used like weapons and there is no real neutraility, in research or in writing.

How do we address ths=ese articles? I conceed that they mayu constistute a minority, but we dare not underestimate their impotamtce as I bet these are freuently visdbu vii

Slrubenstein20:14, 23 December 2009

Actually, this is smth I though about. I believe there are three large groups of articles.

  • Articles of common interest: just common stuff, bios, events, cities etc. They should be well-sourced, and it is usually not a problem to find sources. If the sources somehow can not be found, it means the article may be not notable.
  • Specialized articles. These typically can only be written by experts, and experts have access to sources. The problem is not that we do not have sources but that we so not have experts.
  • Fan-driven articles: movies, games ets. Sources here are problematic, and at some point a general decision should be made by communities what happens to these articles/topics.
Yaroslav Blanter22:06, 24 December 2009

Referring to the below discussion, I am not talking here of NPOV, this is a separate issue.

Yaroslav Blanter22:07, 24 December 2009
 

Yea,

I have somewhat the same articles grouping in mind ;)

I can reply questions about the Fan-driven articles sources, if needed as i do articles sourcing for the En anime/manga project so i know what Fan-driven issues can be.

KrebMarkt22:46, 24 December 2009

I have good experience with our local projects on anime/manga and also on computer games. In my experience the most problematic are (currently) fantasy books and related stuff.

Yaroslav Blanter22:53, 24 December 2009
 
 

I think those are almost two different problems.

One class of article is low quality because it was substantially produced by one or two editors, with everyone else just adding a random statement at a time. No one has ever thought about researching it or organizing it or focusing it on what is relevant. The other class of article is low quality because there's too much activity -- it's constantly attracting new editors who all have an opinion, not to mention the cabals who have a stake in keeping out anything they disagree with. If we had four recommendations, I might spend two on trying to improve the stubs and low activity articles, with another two recommendations just for the high activity articles that suffer from controversies.

I might even focus a little more on the stubs and low activity articles, and put three recommendations on those. First, it's more widely desired in the community. People might disagree about how to resolve the Israel/Palestine or Race/IQ articles, but most good editors will agree that an unsourced article is a bad thing. Secondly, a controversial article will at least be well written. The dispute may let systemic bias slip in, but you'll find that these articles are pretty well written with a lot of sources. The low-activity articles are unmonitored, meaning you either get stubs that haven't had substantial work in months, or articles where people add all kinds of garbage with no effort to monitor it for quality control.

The low activity articles need peer reviewed sources. That would be a huge step up from the unsourced stuff, and the stuff sourced to blogs and personal websites.

The high activity articles need something else. When you have a well written article that has sources, but it still suffers from major problems with bias, you need an authority to step in. An expert. A mediator. An arbitrator. An editorial board. A neutral third opinion.

Randomran21:18, 23 December 2009
 

The lack of minimum sourcing hurts the outsiders perception of Wikipedia but won't bring turmoils inside it save for some contested subjects.

The weight issues are permanent headaches for the community and it's beyond the quality of the source issue. Concerned articles are often well written just that they are section that need balancing and editors gun slugging RS to see which PoV will prevail.

Slrubenstein is right by dividing article by type. A science stub and a fiction stub won't have the same evolution and needs in term of sourcing.

I think it will be necessary to make a proposal focused on "Science & technology, Expert & academic sources" while in term of rough numbers it's not the biggest chunk of Wikipedia it's critical if we really want to rally the support of educative communities which from my personal knowledge is in France neutral leaning negative with the option to whack students who copy-paste Wikipedia content.

For Fiction articles the mentioned global RS sources database would help a lot especially many, many websites/blog/others on TV shows, anime, manga, sci-fi, etc... Line of the RS not RS is more blurry in this area. Things like this person reviews comics in a well know RS website like The Comics Journal but also makes additional comments and comics reviews on its personal blog. So do we count his blog reviews as RS materials? Does this person carry its "expert" stamp outside its RS website?

A pity we don't have someone that can talk about the sourcing of BLP articles.

@Slrubenstein Thanks, i just wanted clarify while i may have "weird" reactions to some proposals but i'm still experienced & concerned by the discussed issues.

KrebMarkt22:44, 23 December 2009
 

Well, I think Randomran deserves credit also. I have been watching the article "Race and Intelligence" since I got here, and Race as well. One reason I care about these articles is because I believe - with no data, I admit - that these are topics of high interest to people, and they get many readers. In the general public, there is a "nature versus nurture" debate. Among scientists (social scientists as well as life scientists) there are debates, but not nature versus nurture. I think this is a case where we could use more experts, not because there is a paucity of scholarly sources, but because the editors who add these sources are partisan in a debate I do not think scientists themselves are having. This leads to real distortions. The result is that a very small number of people have to keep a sharp eye on the Race article, to keep it stable. And it has been fairly stable, for a while, but in part because of the few keeping sharp eye. And "race and Intelligence" keeps going in and out of mediation. There is no consensus as to what are considered fringe views among scientists.

These are just two examples but for me they are exemplary of quality issues because when I think of high school or college students using Wikipedia, I think of the damage that can be done if they rely on Race and Intelligence.

I think that there are similar problems in the social sciences and humanities: many articles (postmodernism, the Glorious Revolution, Emile Durkheim) have citations, even good citations. But these citations do not represent the current state of research or scholarly debate on these issues. The articles have little OR, have verifiable sources, appear NPOV although perhaps with some weasel words. They just are not great articles because we do not have enough editors who know a lot on these topics, OR because we have editors who make contributions but do not know how to do good research (i.e. identify the major sources) or use sources well (i.e. organize different views into debates, rather than just pasting together different views.

I agree with Randomran that there may be two separate problems. In the past he has been skilful at deviding one thread into two. Randomran, do you want to do that here? I think both of us are bringign up important issues, but keeping them in one thread may not be productive.

I agree with KrebMrkt's point bout BLPs. At En WP, there was a robust discussion at the WP:Areas for Reform page, on BLP issues, you may wish to review.

Slrubenstein23:35, 23 December 2009
 

If nothing else, I try to provide focus, and figure out what the real point of disagreement is about. It seems we are indeed talking about two different problems. Might be a good idea to start a thread on either (or both) of them.

Randomran06:40, 24 December 2009
 

How would you name the two? Would you mind starting a second thread?

Slrubenstein09:48, 24 December 2009
 

Not sure, but this data might give us some guidance:

  1. Most articles have a few editors
  2. A few articles have the most editors
  3. Most articles are redirects
  4. Ignoring redirects, the median length of an article is 1500 bytes. (e.g.: this or this)

Seems like we have two problems:

  • How do we improve the quality of the majority of articles that are 1500 bytes *or less*?
  • How do we improve the quality of the few "hyperactive" and controversial articles?
Randomran19:32, 24 December 2009
 

Q1:

  • Putting more forward those <1.5KB articles similar to what we discussed here
  • Awards for editors who edited X uniques articles during the year something similar to project assessment awards with different level of awards and run every year.

Those proposals will increase the numbers of contributors on <1.5KB articles while it won't guaranty an upgrade in quality, it will at least increase drastically the probability to have a <1.5KB article meeting a subject "expert" editor. Yea, that just rolling more often the dice hopping to roll a 6.

Q2

  • Flagged revisions
  • Mediation, arbitration, any solution ending by -ion.

That technically not within Quality scope but the "Forced" consensus/compromise as discussed in Task Force/Health is the most suited tool in some deadlocks.

KrebMarkt20:04, 24 December 2009

As far as I am concerned these issues are withing the Quality scope (as soon as they concern quality of course, not per se).

Yaroslav Blanter22:09, 24 December 2009
 

Thanks for this analysis.

I wish to address only the second question: "How do we improve the quality of the few "hyperactive" and controversial articles?"

My own experience is that these controversies are on-going or very destabilizing for a few reasons, each of which might lead us to different actions:

  • many editors, how make occassional edits or only a few edits to these articles, have done little or no research on the topic. Many do not understand the nuances of NPOV. Better informed editors - I mean, newbies better-socialized into the values of the community, might help
  • when those editors who contribute the most and most frequently are in an intractable conflict, mediation often fails because we do not have a robust mediation process. Arbitration is useless because it does not deal with content. We need a more robust mediation process.
  • intractable conflicts are usually among a half a dozen or so people. I think that if there were simply more well-informed people working on these articles, it might be possible for editors to work towards a consensus. Recruiting more experts might help.
Slrubenstein13:44, 25 December 2009

I am not sure whether my experience would help but anyway. In Russian Wikipedia we had persistent conflicts between Armenian and Azeri editors. To the point that they could start fighting in just any article. At some point, we got a serious arbitration request half a year ago (I was then an Arbitration Committee member and had to deal with this case - we only serve 6 months, not 2 years as in en.wp). What we introduced in the end was to create a mediation committee whose members are acceptable to both sides and who have the last word. If any editor is unhappy with any edit in an article related in some way to Armenia or Azerbaijan, he/she files a case which the mediators decide on. Additionally, we indicated what sources are automatically recognized as authoritative and which need to be decided by the mediators, and also we prohibited edit warring: not more than one edit per user per article per 24 hours. The system is in place for about 5 months, and we feel it much easier, we did not have any major issues since, though initially, of course, some users had to be blocked.

Yaroslav Blanter18:26, 25 December 2009

Btw this is much easier than POV in articles related to religion etc. These POV pushers just have to be indefinitely blocked, nothing else helps.

Yaroslav Blanter18:28, 25 December 2009
 
 

@Slrubenstein You should get a look at Binding mediated consensus decisionmaking on Wikipedia.

That for indecisiveness & non desire of compromise.

KrebMarkt15:23, 25 December 2009
 

Some people who are part of an intractable conflict are also at the limits of good faith. "Good faith" means you think you honestly believe you're improving Wikipedia. But someone can honestly believe they're improving Wikipedia when they're pushing the limits of original research, or weighting towards a POV that they consider "the most reasonable". I'm actually working with the community health task force to look into the effectiveness of binding mediation, so I can't really say anything decisive about it. But I'm concerned that some people really aren't there to "negotiate" in good faith, and that they have a vested interest in avoiding reason. ... That said, improved decision making could help some of the time.

Randomran17:05, 25 December 2009
 

There are two aspects:

  • Some editors are comfortable with "no result/no compromise" thinking that over time they will have the consensus to impose their "view". What's funny is that often both side of the discussion think this way.
  • Some editors are really PoV pushers who feaster on aggravating edit conflict and strife. They know that their view won't pass and a comprise will be found but until then they will drag their feet and use the edit conflict to make their view soundly know.
KrebMarkt19:05, 25 December 2009
 

Well, a good example is right here [1]. I have worked on an article on Jesus reinterpreted in the light of what we know of Jewish culture and history at the time. There is a section reviewing current explanations as to why Christianity broke away from Judaism. It also includes an account of the most cutting edge research by a scholar named Boyarin. User Ari has recently come over deleting a section saying it is dubious, and calling for a reduction in the space devoted to Boyarin as his views are marginal. I reverted his deletion and left a note at talk in which I said that of course he thinks these views are marginal (anyone who deletes views does) but this is why we have an NPOV policy and instead of deleting views he does not like, he should add other views he feels are neglected. In the past few days we have had a little revert war. He continues to delete content that I drew from reading books assigned in university courses on the topic. Although he claims these views are dubious and marginal, he has not added any of those other views he seems to think are right. And he insists that I insulted him. How do I deal with this problem?

One reason it is hard to resolve is that very few Wikipedians know this literature, have researched it, and watch the page. If five or six people who were knowledgable in Early Christian History, Jewish history, Biblical studies watched this page perhaps we could have a real discussion on how to improve the article, incorporating views currently neglected, clarifying views already present. With five or six knowledgable editors we could have a real conversation on which are the mainstream views, minority views, and so on. But we just do not have enough people.

So with only two editors it turns into a silly revert war.

I also continue to be confused about how Ari feels I insulted him. I suspect he misunderstands NPOV (which in my understsanding demands that we include views we do not like, because it is inevitable that editors all acting in good faith will have incompatible understandings of which views are notable and which are not. For me a corollary of this is to add content rather than delete. But Ari deletes, and claims I insulted him.

Sorry to bring in a personal example but to me this is a classic example of those small number of articles that could be better and often are not for lack of adequate attention by enough editors who (1) are well-informed and (2) understand NPOV and (3) committed to consensus-based editing

Slrubenstein23:11, 25 December 2009