Rather than relying solely on exceptions back to nose/test framework - add option (--structured_data_report_file) to run_tests.sh that causes a bunch of detailed data to be dumped to the specified file in a very structured way. Includes full to_dict of the job from the API which in turn includes job metrics, command-line, job's standard error and outputs (instead of the test frameworks), as well as the tool inputs, and exceptions broken out for tool execution versus output checking.
Its all indexed in the file by the test id (without the actual test toolbox depending on knowing the test id) - so one could pair this information with the XUnit output to produce much more detailed breakdowns of the tests.
John Chilton
887bc44a41