Proposal talk:Text or Syntax driven Charts, Diagrams, Graphs and more

From Strategic Planning

Forgetting that IE doesn't support SVG, it supports VML

you can't put the graphs directly into SVG: it will be necessary to put them into an intermediary that will, on detection of the browser, result in SVG for non-IE browsers, VML for IE browsers, and a PNG for older browsers and slower machines (server-side).

see e.g. GWTCanvas or the pyjamas port http://pyjs.org/examples/gwtcanvas/output/GWTCanvasDemo.html for an API which is capable of converting into either format.

NOTE: allowing creation of graphs in a FULL programming language is NOT recommended - that runs the risk of elitism. the whole point is to have the graphs in a human-readable, average-human-intelligable format. GWTcanvas and its pyjs port are purely mention for reference to demonstrate that the level of abstraction between an API to a browser-specific rendering implementation (VML and SVG) is perfectly possible.

Diagrams: Like text or like PNGs?

If diagrams are 'like PNGs' then there is less opportunity for collaborative improvement than if they are 'like text'. GraphViz, GnuPlot, PIC Language, and EasyTimelines provide concise human readable textual formats for diagrams. SVG by contrast is fine grained and machine oriented - closer to a PNG than to a piece of text in how well it supports collaborative edits. The concise formats, however, can and do generate to the SVG format.

Unifying through SVG and AJAX

Each of the textual graphing syntaxes has within it a subroutine library where each subroutine generates multiple SVG elements. These subroutine libraries do not interoperate. They are only exposed to users through custom little languages.

AJAX techniques (client side) could be used instead. Text for a diagram could use subroutines from any of the libraries, for example putting 'market share' pie charts along a time line.

The subroutine library plays a role similar to a css style sheet. Users generally do not specify every colour and line width. They provide textual data with less custom formatting detail than they would if attempting to fully specify a diagram in SVG.

so, you're talking about deploying e.g. GChart http://code.google.com/p/gchart or the pyjamas port, http://pyjs.org/examples/gcharttestapp and providing a means whereby what gets displayed is controlled, even client-side, by fetching the text of the diagram? i like that idea - a lot. particularly because GChart and the pyjamas port both support GWTCanvas (and the pyjamas port of GWTCanvas). here's the downside: if you have an older browser, or do not have a computer with significant resources, the display of graphs is absolutely horrendous. firefox 2 and IE6 take _minutes_ to display complex graphs. no, that's not seconds - _minutes_. John did his best to cater for limitations in FF2 but there's only so much that can be done. so - great idea, and easily achievable, too. downside: definitely requires a modern browser engine (Opera 9+; FF 3+; Safari 3+; IE7+; Android; iPhone; Nokia S60 etc.) Lkcl 20:55, 11 October 2009 (UTC)[reply]

XML to Diagrams

Two examples of existing Wikimedia adaptations:

  • OpenStreetMaps has an XML representation which is then converted to SVG (by osmarender) for rendering to .png. For their slippy-maps interface (large maps like google maps) the conversion from XML to SVG to png is done server side.
    • There is a javascript osmarenderer that runs in firefox and goes from XML to SVG and displays in the browser using SVG. It's a little slower than the server's version. It is relatively new (about a year old) and has not been optimised yet.
  • WikiPathways has an XML representation which is converted to .png server side. As with open streetmaps, to view you just need a browser that supports .png. Registered users can view and edit the pathways graphically using an editor written in java, updating the underlying XML. Editing works on Firefox, IE and Opera using the same code. On saving, the .png image is regenerated server side so viewers without the java editor can still view it.

These suggest to me that we could use different domain specific XML formats and then use XSLT that can convert them to SVG and thence an SVG renderer to get to .png. As for WikiPathways, Wikimedia would offer those .pngs for read only access. XML would be available for editing, initially in 3rd party apps. User:JamesCrook

Generalising the idea in Proposal:Inline_SVG_preference users could have preferences for NOT using the .pngs and doing the rendering client side for each format supported. SVG is a special case of XML diagram with an empty XSLT transform.

  • One underlying format supports IE (has no SVG) and Firefox (has SVG)
  • Editing is via a graphical editor. You don't have to be a programmer to edit a diagram.
  • Accepts that there are many custom standards. Use a standard mechanism (XSLT) to support the different standards.
  • Speed is addressed. Older browsers can view rapidly using the .pngs.
  • Based on existing reasonably large real-world extensions of Wikimedia.

Custom Markup to Diagrams

An example of another Wikimedia adaptation:

  • WikiTex supports new tags types, e.g <music> with custom what-to-draw instructions inside them. This is already implemented for music, chemical structures, chess games, graphviz and gnuplot.

Discussion suggests that a similar syntax to image insertion should be used, so that the editable diagram has a page of its own. Also that it should be borne in mind that these custom syntaxes may (in time) need WYSIWYG editors. User:JamesCrook

  • Supports arbitrary custom formats. These domain specific formats already are mature, e.g graphviz and XyMTeX.
  • Possible downside is that these renderings are supported by diverse executables rather than by one single underlying mechanism. This will require more security analysis than using XML+XSLT to SVG to .png.

Impact?

Some proposals will have massive impact on end-users, including non-editors. Some will have minimal impact. What will be the impact of this proposal on our end-users? -- Philippe 01:12, 4 September 2009 (UTC)[reply]

Impact on Server Load

Servers will be serving more and larger .pngs, particularly if supporting OpenStreetMaps type diagrams. We can push more of that work client side as browsers get smarter - with (in time) the benefit of smarter diagrams too.

Impact on Article Quality

Richer web pages. Many more diagrams. Picture is worth a thousand words. Particularly valuable in areas where there are (potentially) many diagrams 'of the same kind' (maps, charts, chemical-pathways, circuits, protein structures, process diagrams).

Impact on Spam and Vandalism avoidance

Can scan XML for spammy urls etc, but undoubtedly more difficult to auto detect spam in diagrams than in text. One can hope that it is no worse than spam and vandalism in pngs.

Impact on Verifiability and Copyright Violation Checking

Need to look to what OpenStreetMaps and WikiPathways are doing here. In text user edits are usually in one 'run', and it is perhaps easier to spot unsourced data. In diagrams and the underlying XML one user's changes may be in lots of small pieces.

Impact on Barrier to Entry

  • Diagrams are likely to lower the barrier for entry for editing for non-native English speakers.
  • Initially many diagrams are likely to be in the more technical areas. These are likely to have a high barrier to entry (for editing).
Retrieved from "https://strategy.wikimedia.org/w/index.php?title=Proposal_talk:Text_or_Syntax_driven_Charts,_Diagrams,_Graphs_and_more&oldid=53670"