Keep tests easy to read
A label such as Test 17 says little when it fails. Column 703 is AAA immediately explains the expected behavior. Useful scenario labels and file names make a growing suite easier to maintain.
Not every test needs a separate file. Repeated expectations can be shared when they genuinely apply to several cases. The following example combines an exact expected result with reusable checks.
Reuse shared expectations
Our function normalizes a label by fixing spacing and converting the text to uppercase. The test uses a variable for the input and imports additional expectations:
examples/08-organization/labels.xspec
<?xml version="1.0" encoding="UTF-8"?>
<x:description xmlns:x="http://www.jenitennison.com/xslt/xspec"
xmlns:f="urn:example:labels" stylesheet="labels.xsl">
<x:import href="shared.xspec"/>
<x:scenario label="Normalize an editorial label">
<x:variable name="input" select="' xml publishing '"/>
<x:call function="f:label"><x:param select="$input"/></x:call>
<x:expect label="Normalize internal whitespace" select="'XML PUBLISHING'"/>
<x:like label="A non-empty uppercase string"/>
</x:scenario>
</x:description>The shared expectations are in this file:
examples/08-organization/shared.xspec
<?xml version="1.0" encoding="UTF-8"?>
<x:description xmlns:x="http://www.jenitennison.com/xslt/xspec"
xmlns:xs="http://www.w3.org/2001/XMLSchema" stylesheet="labels.xsl">
<x:scenario label="A non-empty uppercase string" shared="yes">
<x:expect label="Return a non-empty string" result-type="xs:string"
test="string-length($x:result) gt 0"/>
<x:expect label="Use uppercase characters" result-type="xs:string"
test="$x:result eq upper-case($x:result)"/>
</x:scenario>
</x:description>x:import includes the file. x:like uses a shared scenario at the required point. The exact comparison with XML PUBLISHING checks the full result. The shared checks additionally require a non-empty uppercase string.
A general check that some output exists would be too weak here. Shared tests should supplement the concrete expectation, not replace it. And if two functions have different whitespace rules, the same normalization test will not suit both.
Nested scenarios can also share preparation. Keep it clear which parameters come from the outer scenario and which are changed inside. The resulting call should still be easy to understand when reading the test.
Run a smaller selection while debugging
focus lets you concentrate on a scenario while others are set aside. pending marks a test that should not run yet. Both can be useful during development.
examples/08-organization/focus-pending.demo.xspec
<?xml version="1.0" encoding="UTF-8"?>
<!-- Development-only illustration. Never part of the release test set. -->
<x:description xmlns:x="http://www.jenitennison.com/xslt/xspec"
xmlns:f="urn:example:labels" stylesheet="labels.xsl">
<x:scenario label="Inspect one scenario" focus="Debugging this scenario">
<x:call function="f:label"><x:param select="'xml'"/></x:call>
<x:expect label="Uppercase the label" select="'XML'"/>
</x:scenario>
<x:scenario label="Document a future requirement" pending="Awaiting editorial decision">
<x:call function="f:label"><x:param select="'xml'"/></x:call>
<x:expect label="The future contract is not implemented" select="'XML!'"/>
</x:scenario>
</x:description>This file is a demonstration. Otherwise a report could look successful even though a large part of the tests never ran.
The package runner checks the selected suites and their imports for leftover focus and pending markers. A marker blocks release verification, and the results must not contain pending expectations. The deliberately defective initial example is not hidden; it is run separately.
Do not change expected output just to make a test pass
A stylesheet change breaks a test. Updating the expected file to match the new output is quick, but first answer this question: is the new output intentional, or has the change introduced a bug?
When the requirement changes, the expected output can change too. Record why. When the requirement stays the same, the code usually needs fixing instead.
For large publishing documents, combine a few complete reference files with many small, focused examples. A missing emphasis element is then less likely to be overlooked among hundreds of unrelated output changes.