The Development of XML
(Excerpt from "XML in a Nutshell" by Elliotte Rusty Harold & W. Scott Means)
XML is a child of the Standard Generalized Markup Language (SGML). The language that eventually became known as SGML was invented in the 1970s by Charles F. Goldfarb, Ed Mosher and Ray Lorie at IBM, and developed by several hundred people around the world until it was finally adopted as ISO Standard 8879 in 1986. SGML was intended to solve many of the problems that XML now solves, and in the same way. It was, and remains, a semantic and structural markup language for text documents. SGML is extraordinarily powerful and achieved some success in the US military, government, aerospace and other fields that needed to manage technical documents tens of thousands of pages long efficiently.
SGML’s greatest success was HTML, an application of SGML. However, HTML does not come close to offering the full power of SGML itself. It restricts authors to a finite set of tags designed to describe web pages in a presentation-oriented way. As a result, it is really little more than a traditional markup language used by web browsers. HTML is not well suited to much beyond designing web pages. You would not use HTML, for example, to exchange data between incompatible databases or to send updated product catalogues to retailers. HTML defines web pages, and does so very well, but nothing else.
SGML was unquestionably the better choice for other applications that took advantage of the Internet but were not simply web pages intended for people to read. The problem was that SGML is complicated, very complicated. The official SGML specification runs to more than 150 highly technical pages. It covers many special cases and unlikely scenarios. SGML is so complex that hardly any software implements it completely. Programs that implemented, or claimed to implement, different subsets of SGML were often incompatible with one another. A particular feature regarded as essential in one program was considered unnecessary in another and omitted altogether.
In 1996, Jon Bosak, Tim Bray, C. M. Sperberg-McQueen, James Clark and several others began working on a “lite” version of SGML. This version retained SGML’s power but discarded many features that, over the preceding 20 years of use, had proved redundant, too difficult to implement, confusing to end users or simply not very useful. The result was XML 1.0, released in February 1998. It was an immediate success. Many developers who knew they needed a structural markup language but could not come to terms with SGML’s complexity happily switched to XML. It was used in every conceivable field, from court records to pig farming.
And XML 1.0 was only the beginning. The next standard was Namespaces in XML, an attempt to allow markup from different XML applications to be used in the same document without conflicts. For example, a web page about books could contain a title element referring to the page title as well as title elements referring to the titles of books, without the two interfering with each other.
Next came the eXtensible Stylesheet Language (XSL), an XML application designed to transform other XML documents into a form that could be viewed in web browsers. XSL soon split into XSL Transformations (XSLT) and XSL Formatting Objects (XSL-FO). XSLT became a general-purpose language for transforming one XML document into another, whether for display as a web page or for entirely different purposes. XSL-FO is an XML application for describing the layout of both printed pages and web pages, approaching PostScript in power and expressiveness.
XSL is not the only way to prepare XML documents for presentation, however. Cascading Style Sheets (CSS) were already being used in HTML documents when XML was invented and proved suitable for use with XML as well. With CSS Level 2, the W3C made preparing XML documents for presentation an explicit goal of CSS. The existing Document Style Sheet and Semantics Language (DSSSL), with its origins in the SGML world, was also adapted to prepare XML documents for print and use on the Web.
The eXtensible Linking Language (XLink) initially defined more powerful linking constructs for connecting XML documents in a hypertext network, making HTML’s A tag seem as though its name stood for “anaemic”. XLink also split into two separate standards: XLink for describing connections between documents, and XPointer for addressing individual parts of an XML document. At this point, it was noticed that both XPointer and XSLT were developing sophisticated but unfortunately incompatible syntaxes that performed exactly the same task: identifying particular elements in an XML document. The addressing portions of both specifications were therefore separated out and combined in a third specification, XPath. A little later, another part branched off from XLink and grew into XInclude, a syntax for constructing complex documents by combining individual documents and document fragments.
Another piece of the puzzle was a uniform interface for accessing the contents of an XML document from a Java, JavaScript or C++ program. The simplest API simply treated the document as an object containing other objects. Work was already under way, both inside and outside the W3C, to define such a Document Object Model (DOM) for HTML. Extending these efforts to cover XML was therefore not difficult.
Outside the W3C, David Megginson, Peter Murray-Rust, Tim Bray and other members of the xml-dev mailing list noticed that third-party XML parsers were compatible in terms of the documents they could check, but not in terms of their APIs. This led to the development of the Simple API for XML (SAX). SAX2 was released in 2000, aiming to provide better configurability, namespace support and a cleaner API.
One surprise during XML’s development was that developers adopted it more for record-like structures, such as serialised objects or database tables, than for the narrative, text-oriented structures for which SGML had originally been used. DTDs worked very well for narrative structures but reached their limits with the record-like structures that were actually being developed. The main problems were seen as the inability to assign types to data and the fact that DTDs are not themselves XML documents. A number of companies and individuals began working on schema languages to address these shortcomings. Many of these proposals were submitted to the W3C, which formed a working group to combine the best ideas from all these languages into something intended to be more than the sum of its parts. In 2001, this group published version 1.0 of the W3C XML Schema Language. Unfortunately, the language proved excessively complex and inaccessible. As a result, numerous developers went back to the drawing board to devise cleaner, simpler and more elegant languages, such as RELAX NG and Schematron.
Eventually, it became clear that XML 1.0, XPath, the W3C’s XML Schema language, SAX and DOM all work with similar conceptual models of an XML document, but with subtle differences. For example, XPath and SAX regard CDATA sections merely as syntactic sugar, whereas DOM treats them differently from ordinary text nodes. The W3C’s XML Core Working Group therefore began working on an “XML Information Set” on which all these standards could build and to which they could refer.
As increasingly valuable XML documents began to be transmitted over the Internet, the need to secure and authenticate these transmissions became apparent. In addition to existing mechanisms built into the underlying protocols, such as SSL or HTTP Digest authentication, formats were developed to protect the XML documents themselves throughout their lifetime, rather than only during transmission. XML Encryption, a standardised XML syntax for encrypting digital content, addresses the need for confidentiality. XML Signature, a joint IETF and W3C standard for digitally signing content and embedding those signatures in XML documents, addresses authentication. Because digital signatures and encryption algorithms are defined in terms of byte sequences rather than XML data models, XML Signature and XML Encryption are based on Canonical XML. This is a standardised serialisation format that removes insignificant differences between documents, such as whitespace inside tags or whether attribute values are enclosed in double or single quotation marks.
Throughout all this, the core XML 1.0 specification remained unchanged. All this new functionality was layered on top of XML 1.0 rather than changing its foundations. This is a testament to XML’s solid design and strength. However, XML 1.0 was itself based on Unicode 2.0, and as Unicode continued to evolve and added new scripts such as Mongolian, Khmer and Burmese, XML fell behind. This was the main reason for the publication of XML 1.1 in early 2004. It should be noted, however, that XML 1.1 offers little of interest to developers working in English, Spanish, Japanese, Russian, French, German, Dutch or many other languages already supported by Unicode 2.0.
Undoubtedly, many new extensions to XML remained to be invented. Even this rich collection of specifications deals only with technologies central to XML. Much development work remains to be done, and is already taking place at an increasing pace in XML applications such as SOAP, SVG, XHTML, MathML, Atom, XForms, WordprocessingML and thousands more. XML has proved to be a solid foundation for many different technologies.