Troubleshooting
In case the Boost Unit Test Adapter is not discovering tests, it is suggested to check if the adapter is actually being called by Visual Studio so as to discover the tests. This can be verified by checking for the presence of a particular log type in the section Test of the Output window. The log that needs to be looked for, is the one similar to the snippet shown below, indicating that the Logger has been initialized.
If the presence of this log entry is not found, there could be an issue with the adapter registration in Visual Studio. The registered adapters can be listed via the Developer Command Prompt for the specific Visual Studio version being utilized. For instance, for Visual Studio 2015, Developer Command Prompt for VS2015 needs to be utilized as shown below.
Upon issuing the command vstest.console.exe /ListDiscoverers /UseVsixExtensions:true the listing for Boost Test Adapter should appear as shown in the above snippet. If no listing for the Boost Test Adapter appears or any of the Supported File Types (dll or exe) is/are missing, the Boost Test Adapter will not be called by Visual Studio to discover tests.
Test adapter registration may vary from one Visual Studio version to the other. It is recommended that the extension is installed and properly tested for correct installation on supported versions.
In case of such a scenario, the user is requested to open an issue in the relevant sections on GitHub providing a dump of the console as shown in this guide, stating which is the Visual Studio version utilized and the Boost Unit Test Adapter version that can be obtained from the Extensions and Updates form in the Visual Studio IDE.
The test adapter tries to launch the user's test module in order to discover tests. If the user's module fails to load (e.g. because it depends on DLLs which are not in the PATH), an error will be logged in the 'Test' section of the output pane (generally, the error manifests as discovery failure due to a non-zero process exit code). If such an error occurs, the easiest way to diagnose the source of the problem is to try launching the test module outside the adapter.
In case a .runsettings file is being used in which configuration in relation to the code coverage is present (similar to the one shown in the below snippet),
<?xml version="1.0" encoding="utf-8"?>
<!-- File name extension must be .runsettings -->
<RunSettings>
<DataCollectionRunSettings>
<DataCollectors>
<DataCollector friendlyName="Code Coverage" uri="datacollector://Microsoft/CodeCoverage/2.0" assemblyQualifiedName="Microsoft.VisualStudio.Coverage.DynamicCoverageDataCollector, Microsoft.VisualStudio.TraceCollector, Version=11.0.0.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a">
<Configuration>
<CodeCoverage>
<!-- Match assembly file paths: -->
<ModulePaths>
<Include>
<ModulePath>.*\.dll$</ModulePath>
<ModulePath>.*\.exe$</ModulePath>
</Include>
<Exclude>
<ModulePath>.*boosttestadapter\.dll$</ModulePath>
</Exclude>
</ModulePaths>
<!-- Match fully qualified names of functions: -->
<!-- (Use "\." to delimit namespaces in C# or Visual Basic, "::" in C++.) -->
<Functions>
<Exclude>
<Function>^boost::.*</Function>
<Function>^UnitTestUnitTest::.*</Function>
<Function>^.*::PrintDataStructure</Function>
<!--TestCases Exclusion-->
</Exclude>
</Functions>
<!-- Match attributes on any code element: -->
<Attributes>
<Exclude>
<!-- Don’t forget "Attribute" at the end of the name -->
</Exclude>
</Attributes>
<!-- Match the path of the source files in which each method is defined: -->
<Sources>
<Exclude>
<Source>.*\\atlmfc\\.*</Source>
<Source>.*\\vctools\\.*</Source>
<Source>.*\\public\\sdk\\.*</Source>
<Source>.*\\microsoft sdks\\.*</Source>
<Source>.*\\vc\\include\\.*</Source>
<Source>.*\\Microsoft Visual Studio 10.0\\VC\\include\\crtdbg\.h</Source>
</Exclude>
</Sources>
<!-- Match the company name property in the assembly: -->
<CompanyNames>
<Exclude>
<CompanyName>.*microsoft.*</CompanyName>
</Exclude>
</CompanyNames>
<!-- We recommend you do not change the following values: -->
<UseVerifiableInstrumentation>True</UseVerifiableInstrumentation>
<AllowLowIntegrityProcesses>True</AllowLowIntegrityProcesses>
<CollectFromChildProcesses>True</CollectFromChildProcesses>
<CollectAspDotNet>False</CollectAspDotNet>
</CodeCoverage>
</Configuration>
</DataCollector>
</DataCollectors>
</DataCollectionRunSettings>
</RunSettings>
Visual studio will request the Boost Unit Test adapter to run the tests in code coverage mode, irrespective whether the user requested the code coverage analysis or not. This behaviour will occur even if the specific Visual Studio version being utilized does not have support for code coverage analysis (the feature is only available in Visual Studio Enterprise). This will have the effect (as documented in Analyzing code coverage for boost tests) where the standard output, the standard error and the memory leak information will not be made available. The suggested workaround is to make use of a different .runsettings configuration file, not containing a code coverage configuration section, whilst executing code not for coverage analysis purposes.
Once the module is successfully compiled, Visual Studio will send us the module for test inspection. In the case of --list_content discovery, we will try to execute the module to identify tests. If the program fails to start correctly, a crash report window may appear based on your defined program. This window will remain active until the user interact with it or it will automatically close based on the configured <DiscoveryTimeoutMilliseconds>. This issue is common if something fails on program startup - in the case of Boost Test, this might be common if static/global variables trigger exceptions on construction (e.g. calling abort(), explicitly throwing exceptions).
Inside the Test Explorer window, double clicking on the tests or using the F12 key should allow you to navigate the tests. In some cases doing so may result in the message 'Cannot open the following file xxx.cpp' in the Visual Studio status bar. This issue is caused by relative paths. To fix it, the Use Full Paths setting under C++ -> Advanced in the project's Property Page must be set to Yes (/FC), as shown in the snippet below.
This issue occurs since Boost.Test makes use of the __FILE__ macro whose value depends on the /FC compilation switch (implicitly defined when using debug options e.g. /ZI). Using /FC will force the __FILE__ macro to expand to a fully qualified path, thus allowing the test explorer to correctly navigate to the test case.
If you identify any issue with the Boost Unit Test Adapter, feel free to open a new issue. Ideally, the Boost library version number and the Boost Unit Test Adapter version number are listed in the issue report so that it makes it easier to reproduce the issue.