Details
- Reviewers
bcooksley - Group Reviewers
Sysadmin - Commits
- R857:5c1eb058934c: Match test not run due to executable not found to jenkins <failure>. This will…
Diff Detail
- Repository
- R857 CI System Tooling
- Branch
- catchnotfoundskippedtests
- Lint
No Linters Available - Unit
No Unit Test Coverage - Build Status
Buildable 11308 Build 11326: arc lint + arc unit
Ran locally on a ctest xml file containing
<Test Status="notrun"> <Name>plugin-btbrowser_test</Name> <Path>./addons/backtracebrowser/autotests</Path> <FullName>./addons/backtracebrowser/autotests/plugin-btbrowser_test</FullName> <FullCommandLine></FullCommandLine> <Results> <NamedMeasurement type="numeric/double" name="Processors"> <Value>1</Value> </NamedMeasurement> <NamedMeasurement type="text/string" name="Completion Status"> <Value>Unable to find executable</Value> </NamedMeasurement> <NamedMeasurement type="text/string" name="Command Line"> <Value></Value> </NamedMeasurement> <Measurement> <Value>Unable to find executable: btbrowser_test</Value> </Measurement> </Results> </Test>
and got as result
<testcase classname="projectroot.addons.backtracebrowser.autotests" name="plugin_btbrowser_test"> <failure><![CDATA[Unable to find executable: btbrowser_test]]></failure> <system-out><![CDATA[Unable to find executable: btbrowser_test]]></system-out> </testcase>
Done in the build dir of kate (with a add_test() manipulated to not find the executable) with ctest -T Test and xsltproc /path/to/ci-tooling/templates/ctesttojunit.xsl Testing/20190427-2217/Test.xml
<testsuite name="CTest" tests="8" failures="1" errors="0" skipped="1">
fixed now to
<testsuite name="CTest" tests="8" failures="2" errors="0" skipped="0">
This diff itself looks fine, my only question is around the rationale for this change.
Mind going into the background of how CTest can setup a test but not know how to find the binary?
It's a combination of the effort to make tests runable without installation in ECM >= 5.38 [1] and an incomplete add_test() syntax (add_test(footest footest) vs. add_test(NAME footest COMMAND footest).
Thanks for review.
As @heikobecker said. Example:
add_executable(btbrowser_test ${BtBrowserSrc}) add_test(plugin-btbrowser_test btbrowser_test) # fail, no exe btbrowser_test in same build dir for find_package(ECM 5.38) and newer
This will build fine, only fail at the point when tests are run and the command is executed.
The reason is in the old signature of add_test where the command argument is taken verbatim for the current build dir. While the newer signature (with NAME & COMMAND) also takes a cmake target for the command argument: "If <command> specifies an executable target (created by add_executable()) it will automatically be replaced by the location of the executable created at build time.". So the test binary as generated into the toplevel bin/ directory will be properly called only with the new signature.
add_executable(btbrowser_test ${BtBrowserSrc}) add_test(NAME plugin-btbrowser_test COMMAND btbrowser_test) # no fail, btbrowser_test is interpreted as target, executable name with path to be called resolved from it
Thanks for the explanations, this definitely looks like something we should be catching then.