Unit Tests Are Executed During Integration Test Runs

[rogercoll]

Component(s)

No response

Describe the issue you're reporting

While modifying the CI to include sudoers execution for integration tests, I noticed that we are using non-exclusive Go build tags to differentiate between unit and integration tests.

The CI runs the following make targets for each commit:

• make gotest GROUP=
• make gointegration GROUP=...

If we run the equivalent locally and just targeting one component for a reduced output:

Unit test

$ make --no-print-directory -C receiver/filestatsreceiver test
/home/hello/opentelemetry-collector-contrib/.tools/gotestsum --rerun-fails=1 --format testname --packages="./..." -- -race -count=1 -timeout 600s -parallel 4 --tags=""
PASS Test_Config_Validate/valid (0.00s)
PASS Test_Config_Validate/missing_include_pattern (0.00s)
PASS Test_Config_Validate (0.00s)
PASS TestComponentFactoryType (0.00s)
PASS TestComponentConfigStruct (0.00s)
PASS internal/metadata.TestMetricsBuilderConfig/default (0.00s)
PASS TestComponentLifecycle/metrics-shutdown (0.00s)
PASS TestComponentLifecycle/metrics-lifecycle (0.00s)
PASS TestComponentLifecycle (0.00s)
PASS Test_Scrape (0.00s)
PASS Test_Scrape_All (0.00s)
PASS internal/metadata.TestMetricsBuilderConfig/all_set (0.00s)
PASS internal/metadata.TestMetricsBuilderConfig/none_set (0.00s)
PASS internal/metadata.TestMetricsBuilderConfig (0.00s)
PASS internal/metadata.TestResourceAttributesConfig/default (0.00s)
PASS internal/metadata.TestResourceAttributesConfig/all_set (0.00s)
PASS internal/metadata.TestResourceAttributesConfig/none_set (0.00s)
PASS internal/metadata.TestResourceAttributesConfig (0.00s)
PASS internal/metadata.TestMetricsBuilder/default (0.00s)
PASS internal/metadata.TestMetricsBuilder/all_set (0.00s)
PASS internal/metadata.TestMetricsBuilder/none_set (0.00s)
PASS internal/metadata.TestMetricsBuilder/filter_set_include (0.00s)
PASS internal/metadata.TestMetricsBuilder/filter_set_exclude (0.00s)
PASS internal/metadata.TestMetricsBuilder (0.01s)
PASS internal/metadata.TestResourceBuilder/default (0.00s)
PASS internal/metadata.TestResourceBuilder/all_set (0.00s)
PASS internal/metadata.TestResourceBuilder/none_set (0.00s)
PASS internal/metadata.TestResourceBuilder (0.00s)
PASS .
PASS internal/metadata

DONE 28 tests in 1.671s

Integration test

$ make --no-print-directory -C receiver/filestatsreceiver mod-integration-test
running go integration test ./... in /home/hello/opentelemetry-collector-contrib/receiver/filestatsreceiver
/home/hello/opentelemetry-collector-contrib/.tools/gotestsum --rerun-fails=1 --format testname --packages="./..." -- -race -timeout 360s -parallel 4 -tags=integration,""
PASS Test_Config_Validate/valid (0.00s)
PASS Test_Config_Validate/missing_include_pattern (0.00s)
PASS Test_Config_Validate (0.00s)
PASS TestComponentFactoryType (0.00s)
PASS TestComponentConfigStruct (0.00s)
PASS TestComponentLifecycle/metrics-shutdown (0.00s)
PASS TestComponentLifecycle/metrics-lifecycle (0.00s)
PASS TestComponentLifecycle (0.00s)
PASS Test_Integration (3.00s)
PASS Test_Scrape (0.00s)
PASS Test_Scrape_All (0.00s)
PASS . (cached)
PASS internal/metadata.TestMetricsBuilderConfig/default (0.00s)
PASS internal/metadata.TestMetricsBuilderConfig/all_set (0.00s)
PASS internal/metadata.TestMetricsBuilderConfig/none_set (0.00s)
PASS internal/metadata.TestMetricsBuilderConfig (0.01s)
PASS internal/metadata.TestResourceAttributesConfig/default (0.00s)
PASS internal/metadata.TestResourceAttributesConfig/all_set (0.00s)
PASS internal/metadata.TestResourceAttributesConfig/none_set (0.00s)
PASS internal/metadata.TestResourceAttributesConfig (0.00s)
PASS internal/metadata.TestMetricsBuilder/default (0.00s)
PASS internal/metadata.TestMetricsBuilder/all_set (0.00s)
PASS internal/metadata.TestMetricsBuilder/none_set (0.00s)
PASS internal/metadata.TestMetricsBuilder/filter_set_include (0.00s)
PASS internal/metadata.TestMetricsBuilder/filter_set_exclude (0.00s)
PASS internal/metadata.TestMetricsBuilder (0.01s)
PASS internal/metadata.TestResourceBuilder/default (0.00s)
PASS internal/metadata.TestResourceBuilder/all_set (0.00s)
PASS internal/metadata.TestResourceBuilder/none_set (0.00s)
PASS internal/metadata.TestResourceBuilder (0.00s)
PASS internal/metadata (cached)

DONE 29 tests in 0.188s

The total number of executed tests just differs by one, this is because unit test files are also included in the "integrations" execution. Files without any build tags (default files) will also be included when using the Go's -tags= flag.

Do we want to keep running unit tests among the integration ones? Could we remove the "unit" test execution from the CI if they are already run?

If all the unit test files are modified to include a negative tag (//go:build !integration), only the actual integration tests are run with the corresponding make target:

 make --no-print-directory -C receiver/filestatsreceiver mod-integration-test
running go integration test ./... in /home/hello/opentelemetry-collector-contrib/receiver/filestatsreceiver
/home/hello/opentelemetry-collector-contrib/.tools/gotestsum --rerun-fails=1 --format testname --packages="./..." -- -race -timeout 360s -parallel 4 -tags=integration,""
PASS Test_Integration (3.00s)
PASS . (cached)
EMPTY internal/metadata

DONE 1 tests in 0.185s