Skip to main content

Code documentation

Development tools

Code Structure

Techniques and Standards

How To

Functional Info

Background Info

JMRI: XML Schema Usage

JMRI uses XML for a number of purposes: to hold decoder definitions, for its persistance system for configuration and panel information, and to create parts of the web site from other files. This page describes how we specify the content of those files using XML schema.

For examples (not a tutorial!) on the structure of our schema, see the examples page.

The current schema can be seen online in the schema directory. The most commonly used one is the layout.xsd schema for panel files. See below on how it's organized.

Access to Schema Definitions

JMRI uses XML Schema to define the format of its files.

Those XML Schema may need to be available to the program when it reads the files, as they define the default values of missing attributes and other needed information.

In the JMRI distributions, these are stored in the xml/schema directory. Note that they are not stored in each directory alongside the XML files. There are just too many locations to keep such a set of schema definition files up to date. JMRI itself, via the jmri.jmrit.XmlFile class, provides support for locating those files when the XML parser needs them.

Modifying JMRI Schema

This section talks about how to handle small changes to existing schema, for example added or removing an attribute or element. For more extensive changes, including creating entirely new types or new file formats, see the next section on "Developing JMRI Schema".

Every time you change what JMRI writes (and therefore reads) in an XML file, you must also do the following:

  1. You need to change the code that does the reading and writing.
  2. You need to change the schema file(s) so that the XML format can be properly checked.
  3. You have to provide new test XML files to make sure nothing has been broken, and in some cases have to adjust old ones.

Please don't skip the latter steps. They matter a lot to the long-term stability of JMRI code.

If possible, it's best to make changes by addition, so that existing files can continue to be read unchanged. JMRI strongly values backward compatibility, where any newer version of JMRI can still load and use a file written by an older version.

If you can make a change that's just an addition, then the process is:

  1. Change your code
  2. Add the new elements and attributes to the most recent version of the schema file(s).
  3. Run "ant headlesstest" to make sure that the older files already present in the tests can still be processed. Fix anything that breaks. (You may discover at this point that you didn't actually make a backward compatible change, in which case either improve that or see the section on "Schema Versioning" below).
  4. Create a test file with the new content. Ideally, this won't require using the screen, so it can actually be loaded and stored as part of headlesstest. In that case, put your file in a "load" test subdirectory (see below). At a minimum, though, put your file in a "verify" test subdirectory so that your schema changes will be tested. For more on this, see below.

Schema Versioning

"Versioning" schema allows us to have different schema for older and newer files. This lets newer versions of JMRI continue to check and read files that were written by older versions of JMRI. This backward compatibility is an important JMRI feature which we don't want to lose.

In practical terms, versioning consists of having multiple-but-related versions of the schema definition files which are labeled by the first JMRI version that can read them.

When do you need to create a new version?

In either case, it's important to include sufficient test files that the unit tests catch any problems with the new and old schema. See the test section below.

Note that the unlabeled schema is the primordial, oldest, now-obsolete schema. For example layout.xsd is older than layout-2-9-6.xsd, and therefore not to be used for new files anymore. Don't assume that layout.xsd is the default for new files!

Checking JMRI Schema

It's important that the JMRI schema definitions be kept semantically correct. If we let too many problems build up, we'll eventually have a lot of back-fitting to do. The W3C online schema validation tool is a very good tool for checking that changes to the JMRI schema are still technically correct. You should check your changes with it before commiting them to the repository. Unfortunately, it doesn't seem to check compliance with nested schema elements, e.g. from DocBook (see below) or JMRI schema, but it's still a very useful check.

Using the JMRI "Validate XML File" tool in the "Debug" menu to validate a .xml file ("instance file") that uses your new or updated schema is an important check of both. Use it often during development. You can also use it from the command line via e.g.:


./runtest.csh jmri/jmrit/XmlFileValidateAction xml/decoders/0NMRA.xml

For a quick file check, Linux and Mac OS X users can validate from the command line with e.g.


 cd xml
 xmllint -schema schema/aspecttable.xsd -noout signals/sample-aspects.xml
xmllint can't check schema files themselves, unfortunately, because their schema isn't something it can handle.

Your schema docs should point to our standard stylesheet in their head matter:


<?xml-stylesheet href="schema2xhtml.xsl" type="text/xsl"?>
Stylesheets turn XML code, like this schema, into a human-readable form when the XML is parsed and displayed by a browser. For an example of that, click on this link to the aspecttable.xsd schema file. Our standard stylesheet is pretty basic. It just shows basic structure. If anybody knows of a better stylesheet, we can certainly switch to it.

JUnit testing

You should also add a JUnit test that checks a typical file. There are three kinds of checks that can be done:
  1. You should always have a class that validates a typical file against the schema.

    That's done by having a SchemaTest class in your test-tree package (see e.g. test/jmri/configurexml/SchemaTest.java for an example) that checks all the XML files kept there. If that's in place, just put a copy of a (new) typical XML file in the existing "verify" subdirectory.

    To more extensively check your schema, you can check that it fails XML files that you think are not valid. There are lots of ways to be not valid, and you don't need to check them all, but if there something specific you want to be sure of, put an example of that in the "invalid" subdirectory. Those files are expected to fail for some specific reason. You should document that reason via comments in the file itself so your colleagues can figure it out later.

    If there's no "verify" subdirectory, create one and add it to the end of the SchemaTest class in that package. If there's no SchemaTest class, create one by replicating an existing one, see link above. Don't forget to add it to PackageTest so it gets invoked!

  2. If you're working on configurexml files (panel files), and your new code isn't involved in the actual display of panels (e.g. can run as part of headlesstest), you should add a test that a sample file can be loaded and stored back successfully. Beyond just checking the schema, this also checks that all the load and store plumbing is working. (We run these tests headless as part of Jenkins, so please don't add tests that pop screen windows; they'll just cause errors).

    That's done by having a LoadAndStoreTest class in your test-tree package (see e.g. test/jmri/configurexml/LoadAndStoreTest.java for an example) that checks all the XML files kept there. If that's in place, just put a copy of a (new) typical XML file in the existing "load" subdirectory.

    If there's no "load" subdirectory, create one and add it to the end of the LoadAndStoreTest class in that package. If there's no LoadAndStoreTest class, create one by replicating an existing one, see link above. Don't forget to add it to PackageTest so it gets invoked!

    When LoadAndStoreTest runs, it loads the files in the load directory one-by-one, storing each back out to the "temp" directory within the local preferences directory, and then compares the input and output files. Sometimes this load-and-store process results in something that's in a different order, or contains more info (e.g. attributes missing from the input file are written with default values in the output file). If the comparison fails, but the output file is still OK when you manually inspect it, copy that output file to the "loadref" directory (create it if needed) within your test package. See test/jmri/configurexml/loadref for an example. LoadAndStoreTest will compare to files it finds here, instead of to the original file in the "load" subdirectory.

    If your changes to the code cause this testing to fail with older versions of the file, do not change the older file version. Instead, either put an updated output reference in the "loadref" directory, version the schema to allow the older file to continue to load, or improve your code. Backward compatibility is important!

  3. You can also add a custom JUnit test that reads your sample file and makes sure that the proper objects have been created, that they have the correct data and state, etc. This could be anything from a "load and check that new beans exist in the new manager" to something quite more extensive.
At a minimum, please do the schema checks. They're easy, and will save lots of trouble in the future. If your new features don't involve displaying on the screen, adding load-and-store checks is also valuable, and isn't so hard.

Note: Do not remove or modify any existing XML file checks. Those keep old versions of the files working! If your new code and/or schema breaks the processing of the existing files, you need to either fix your code or version the schema to allow multiple formats to coexist.

Developing JMRI Schema

For some examples of the XML schema structures described here, see the separate XML Schema Examples page/

Our preferred organization for XML schema is based on the structure of the underlying code: A particular *Xml class is the unit of reuse.

Lots of classes descend from jmri.configurexml.XmAdapter: (see Javadoc)

By convention, provide <xsd:annotation><xsd:appinfo> element containing the class name that reads/writes the element:


    <xs:annotation>
        <xs:documentation>
        Some human readable docs go here
        </xs:documentation>
        <xs:appinfo>
            <jmri:usingclass configurexml="false">jmri.managers.DefaultSignalSystemManager</jmri:usingclass>
        </xs:appinfo>
    </xs:annotation>

The Venetian Blinds Pattern

We are moving toward structuring our XML using the "Venetian Blinds pattern". In this style, the top level elements that are written by classes have types defined for them. Any elements that are within those are defined anonymously, within those elements. For an example of this, see the types/sensors.xsd file, which defines a type for the "sensors" element written for SensorManagers. Within that, there is included a definition of a "Sensor" element, and a "comment" element within that.

This limits the number of types, and keeps the schema files roughly aligned with the classes that do the reading and writing.

There are a few items (elements and attribute groups) that extend across multiple types. They are defined in the types/general.xsd file.

More information on XML schema design patterns is available at DeveloperWorks and the Oracle Java site.

On Elements vs Attributes

When defining how to store new classes or updates to classes, the basic questions are: JMRI XML originally leaned heavily to attributes due to limitations in the JDOM library. These limitations are long gone, and we're now moving toward using elements in the proper way.

Available Defined Types

The JMRI schema provides a large number of pre-defined data types. These (generally) check their content, and will be maintained in the future as valid content changes, so it's best to use these where possible instead of defining your own.

A partial list of the pre-defined types:

systemNameType
System names, to eventually be tightened up to a real test of validity
userNameType
User names, not including the empty name
nullUserNameType
User names, with an empty value allowed
beanNameType
Either user or system name
turnoutStateType
closed, thrown
signalColorType
red, yellow, etc
trueFalseType
true, false
yesNoType
yes, no
yesNoMaybeType
yes, no, maybe
For others, browse the general types schema.

External Standards and Future Work

The OASIS collaboration defines a number of schema and schema elements that have become well-known standards. Were possible, we should use those standard elements to improve inter-operability. The first ones of interest are: Learning to use these will require some work, as we can't assume that computers using JMRI have internet access, so can't just reference the entire schema as remote entities.

Copyright, Author and Revision Information

For various reasons, we need to move to DocBook format for Copyright, Author and Revision information in our XML files (instance files).

Sample XML:


  <db:copyright>
        <db:year>2009</db:year>
        <db:year>2010</db:year><
        db:holder>JMRI</db:holder></db:copyright>

  <db:authorgroup>
    <db:author>
        <db:personname><db:firstname>Sample</db:firstname><db:surname>Name</db:surname></db:personname>
        <db:email>name@com.domain</db:email>
    </db:author>    
  </db:authorgroup>

  <db:revhistory>
    <db:revision>
        <db:revnumber>1</db:revnumber>
        <db:date>2009-12-28</db:date>
        <db:authorinitials>initials</db:authorinitials>
    </db:revision>    
  </db:revhistory>

Sample schema description: (But see the real one, which is provided in schema/docbook)


    <xs:element ref="docbook:copyright" minOccurs="1" maxOccurs="1" >
      <xs:annotation><xs:documentation>
      DocBook element(s) providing copyright information in standard form.
      Must be present.
      </xs:documentation></xs:annotation>
    </xs:element>

    <xs:element ref="docbook:authorgroup" minOccurs="1" maxOccurs="unbounded" >
      <xs:annotation><xs:documentation>
      DocBook element(s) describing the authors in standard form
      </xs:documentation></xs:annotation>
    </xs:element>

    <xs:element ref="docbook:revhistory" minOccurs="1" maxOccurs="unbounded" >
      <xs:annotation><xs:documentation>
      DocBook element(s) describing the revision history in standard form
      </xs:documentation></xs:annotation>
    </xs:element>