123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960616263646566676869707172737475767778798081828384858687888990919293949596979899100101102103104105106107108109110111112113114115116117118119120121122123124125126127128129130131132133134135136137138139140141142143144145146147148149150151152153154155156157158159160161162163164165166167168169170171172173174175176177178179180181182183184185186187188189190191192193194195196197198199200201202203204205206207208209210211212213214215216217218219220221222223224225226227228229230231232233234 |
- [/
- / Copyright (c) 2003 Boost.Test contributors
- /
- / Distributed under the Boost Software License, Version 1.0. (See accompanying
- / file LICENSE_1_0.txt or copy at http://www.boost.org/LICENSE_1_0.txt)
- /]
- [section:log_formats Log formats]
- The __UTF__ supports several log formats:
- * [link boost_test.test_output.log_formats.log_human_readable_format HRF]: human readable format
- * [link boost_test.test_output.log_formats.log_xml_format XML]: an machine interpretable log format
- * [link boost_test.test_output.log_formats.log_junit_format JUNIT]: a standardized log format
- understandable by automated tools such as Continuous Builds
- [h4 Design]
- The following functionalities are supported by the logging framework:
- * each logger manages its own log level. The rationale is that some log format are meant for
- automated processing, and by design need to carry all the information that will later be digested
- by a visualization tool.
- * several log format may be active at the same time. The rationale is that the user might want to
- see a non-exhaustive log in his terminal using a human friendly format, while having a detailed
- full log in a file with a format dedicated to automated processing.
- * each logger /indicates/ its default output stream. some logger may prefer to output to one of the standard
- stream while other may prefer output to a file.
- [note The logger indicates the default output stream in case the default should be used. ]
- [h4 Defaults]
- By default the active log level threshold is set to
- * [link boost_test.test_output.log_formats.test_log_output "non fatal error messages"] and the test log output
- is generated in [link boost_test.test_output.log_formats.log_human_readable_format human readable format].
- * [link boost_test.test_output.log_formats.test_log_output "general information"] for
- [link boost_test.test_output.log_formats.log_junit_format JUNIT] log format
- The active log level threshold and the output format can be configured
- at runtime during a test module invocation and at compile time from within a test module using the
- [link boost_test.test_output.logging_api test log public interfaces]. The behavior is logger specific though.
- [/ -------------------------------------------------------------------------------------------------- ]
- [section Test log output]
- The test log is produced during the test execution. All entries in the test log are assigned a particular log
- level. Only the entries with level that exceeds the ['active log level threshold] actually
- appear in the test log output. Log levels are arranged by the 'importance' of the log entries. Here is
- the list of all levels in order of increasing 'importance':
- [#test_log_output_table]
- [table:id_messages Messages
- [
- [Notifications]
- [Meaning]
- ]
- [
- [Success]
- [This category includes messages that provide information on successfully passed assertions]
- ]
- [
- [Test tree traversal]
- [This category includes messages that are produced by the __UTF__ core and indicate which test suites/cases are currently being executed or skipped]
- ]
- [
- [General information]
- [This category includes general information messages produced in most cases by a test module author using the
- macro __BOOST_TEST_MESSAGE__]
- ]
- [
- [Warning]
- [This category includes messages produced by failed `WARNING` level assertions]
- ]
- [
- [Non fatal error]
- [This category includes messages produced by failed `CHECK` level assertions]
- ]
- [
- [Uncaught C++ exceptions]
- [This category includes messages that are produced by the __UTF__ and provide detailed information on the C++
- exceptions uncaught by the test case body.
- ]
- ]
- [
- [Non-fatal system error]
- [This category includes messages that are produced by the __UTF__ itself and provides information about caught
- non-fatal system error. For example it includes messages produced in the case of test case timeout or if
- floating point values calculation errors are caught.
- ]
- ]
- [
- [Fatal system error]
- [This category includes messages produced by failed require level assertions and by the __UTF__ itself in case of
- abnormal test case termination.]
- ]
- ]
- [note
- The active log level works namely as threshold, not as selector. For the given active log level threshold, all
- test log entries with ['importance] higher than threshold are enabled and all test log entries with
- ['importance] below threshold are disabled.
- ]
- In addition to the levels described above the test log defines two special log levels. The current log level can
- be set to:
- * All messages[br]
- If active log level threshold is set to this value, all test log entries appear in the output. In practice
- this is equivalent to setting the active log level threshold to ['success information messages]
- * Nothing[br]
- If the active log level threshold is set to this value, none of test log entries appear in the output. This log level
- is used to execute a ['silent] test that doesn't produce any test log and only generates a result code indicating
- whether test failed or passed.
- [endsect] [/ test log output]
- [section:log_human_readable_format HRF: Human readable log format]
- The human readable log format is designed to closely match an errors description produced by the Microsoft family
- of C++ compilers. This format allows jumping to the error location, if test module output is redirected into IDE
- output window. The rest of the log messages are designed to produce the most human friendly description of the
- events occurring in test module. This is a default format generated by test modules.
- Here the list of events along with corresponding message and the condition that has to be satisfied for it to appear
- in the output.
- [table
- [
- [Event]
- [Condition]
- [Output]
- ]
- [ [On testing start]
- [threshold != log_nothing]
- [`Running <total number of test cases> test case(s) ...`] ]
- [ [On testing start]
- [threshold != log_nothing, show_build_info is set]
- [
- [pre Platform: $BOOST_PLATFORM
- Compiler: $BOOST_COMPILER
- STL : $BOOST_STDLIB
- Boost : $BOOST_VERSION]] ]
- [ [On abnormal testing termination]
- [threshold <= log_messages]
- [`Test is aborted`] ]
- [ [On test unit start]
- [threshold <= log_test_units]
- [`Entering test <test unit type> <test unit name>`] ]
- [ [On test unit end]
- [threshold <= log_test_units; testing time is reported only if elapsed time is more than 1 us.]
- [`Leaving test <test unit type> <test unit name>; testing time <value>`] ]
- [ [On skipped test unit]
- [threshold <= log_test_units]
- [`Test <test unit type> <test unit name> is skipped`] ]
- [ [On uncaught C++ exception]
- [threshold <= log_cpp_exception_errors. Checkpoint message is reported only if provided]
- [`unknown location(0): fatal error in <test case name>: <explanation> <last checkpoint location>: last checkpoint: <checkpoint message>`] ]
- [ [On resumable system error]
- [threshold <= log_system_errors. Checkpoint message is reported only if provided]
- [`unknown location(0): fatal error in <test case name>: <explanation> <last checkpoint location>: last checkpoint: <checkpoint message>`] ]
- [ [On fatal system error]
- [threshold <= log_fatal_errors. Checkpoint message is reported only if provided]
- [`unknown location(0): fatal error in <test case name>: <explanation> <last checkpoint location>: last checkpoint: <checkpoint message>`] ]
- [ [On passed test assertion]
- [threshold <= log_successful_tests]
- [`<assertion location>: info: check<assertion expression> passed`] ]
- [ [On failed WARNING level test assertion]
- [threshold <= log_warnings]
- [`<assertion location>: warning in <test case name>: condition <assertion description> is not satisfied`]]
- [ [On failed CHECK level test assertion]
- [threshold <= log_all_errors]
- [`<assertion location>: error in <test case name>: check <assertion description> failed`] ]
- [ [On failed REQUIRE level test assertion]
- [threshold <= log_fatal_errors]
- [`<assertion location>: fatal error in <test case name>: critical check <assertion description> failed`]]
- [ [On test log message]
- [threshold <= log_messages]
- [`<Message content>`]]
- ]
- The level of details concerning the error message depends on the [link boost_test.testing_tools testing tool] producing the log entry.
- [endsect] [/ human readable report]
- [/ -------------------------------------------------------------------------------------------------- ]
- [section:log_xml_format XML log format]
- This log format is designed for automated test results processing. The test log output XML schema depends on the
- active log level threshold.
- [endsect] [/section:log_xml_format ]
- [/ -------------------------------------------------------------------------------------------------- ]
- [section:log_junit_format JUNIT log format]
- The [@http://junit.org/ JUNIT format] is log format supported by a wide range of Continuous Build/Integration tools.
- This format defaults its log level to [link test_log_output_table `General information`] and its default stream to a file named after
- [link boost_test.tests_organization.test_tree.master_test_suite master test suite].
- The logger will attempt to not overwrite any existing output file, which is also usually understood by Continuous Build tools.
- This format is in fact both a log and a report format: most of the Continuous Build tools will summarize
- the content of a JUNIT file and show an overview of the failing/succeeding tests of a module (report format)
- while letting the user inspect the detailed logs (log format).
- [caution The minimal log-level for the JUnit logger is [link test_log_output_table `non fatal error`].
- Any set-up to higher log level will default to that minimal log-level.]
- [note Until Boost 1.64, the log level was previously defaulting to `success` and was causing a heavy load
- on the logging part in some circumstances.]
- [endsect] [/section:log_junit_format ]
- [endsect]
|