java_test "broken" with no guidance how to fix #1017

pmbethe09 · 2016-03-05T16:11:51Z

191d980

Apparently can use --legacy_bazel_java_test but there was no clear guidance how to fix rules of the form:
java_test(srcs = glob(["*Test.java"]), ...) with multiple Tests in one rule.

The text was updated successfully, but these errors were encountered:

damienmg · 2016-03-05T16:13:41Z

What's your test suite class? You add a main_class before didn't you, just change this main_class for a test_class should works.

damienmg · 2016-03-06T18:21:29Z

Note to myself: fix the relnotes when doing the next bazel release

damienmg · 2016-03-06T18:22:52Z

Also the link to the ml announcement: https://groups.google.com/forum/#!topic/bazel-discuss/r5Qbn1c7eg4

pmbethe09 · 2016-03-07T15:09:45Z

I never used a TestSuite or main_class before. The previous behavior "just worked".

See attached mini example.
bazel test javatests/... --legacy_bazel_java_test
works just fine. But without the legacy flag:

"Class not found: [tests]"

Example.zip

damienmg · 2016-03-07T15:36:47Z

The glob in the example resolve to only 1 file which is why the legacy behavior where finding the rule. The new behavior use the name of the rule to find the main class (compared to using the srcs if only one file is present like for java_binary). This is the behavior we use since the beginning at Google.

pmbethe09 · 2016-03-07T15:39:54Z

Add another class or 2 to blah and the result is still the same.
On Mar 7, 2016 10:37 AM, "Damien Martin-Guillerez" [email protected]
wrote:

The glob in the example resolve to only 1 file which is why the legacy
behavior where finding the rule. The new behavior use the name of the rule
to find the main class (compared to using the srcs if only one file is
present like for java_binary). This is the behavior we use since the
beginning at Google.

—
Reply to this email directly or view it on GitHub
#1017 (comment).

damienmg · 2016-03-08T01:43:52Z

humm maybe we just take the first one, would be weird. We definitely don't create a test suite out of it.

damienmg · 2016-03-08T01:44:09Z

(PS: the glob has to resolve to multiple files)

pmbethe09 · 2016-03-08T02:51:46Z

I don't know why it worked, but I can tell you that it did and picked up all of the tests in the glob.
See attached updated example -- run it then cat bazel-out/local_darwin-fastbuild/testlogs/javatests/tests/test.log
(or change target to linux)

I see
"
JUnit version 4.12
.Bar
.Foo

Time: 0.004

OK (2 tests)
"

It appears that org.junit.runner.JUnitCore was added as the main_class when none was specified, and that picked up all of the tests in the first jar 'javatests' jar file on the classpath.
This was easily be confirmed via experimentation/printlns

However, adding this class as the test_class and running without the '--legacy_bazel_java_test', gets me to a new failure "

initializationError(org.junit.runner.JUnitCore)
java.lang.Exception: No runnable methods
"

pmbethe09 · 2016-03-08T02:51:57Z

and attachment
Example.zip

damienmg · 2016-03-08T03:19:19Z

The test_class should be a test class. not the JUnitCore runner.

Oh yes the legacy behavior allow multiple test file in the srcs. Humm... not sure we should do anything there.

pmbethe09 · 2016-03-09T03:17:58Z

Easy enough to write a genrule to take my SRCS and generate
an org.junit.runners.Suite, but as I wrote, sad that this just worked
before, and now I have to add a layer of my own.

On Mon, Mar 7, 2016 at 10:19 PM, Damien Martin-Guillerez <
[email protected]> wrote:

The test_class should be a test class. not the JUnitCore runner.

Oh yes the legacy behavior allow multiple test file in the srcs. Humm...
not sure we should do anything there.

—
Reply to this email directly or view it on GitHub
#1017 (comment).

pmbethe09 · 2016-03-21T15:00:01Z

If anyone else runs into this -- I created a junit_tests skylark rule which assumes the srcs are all Junit tests, and creates a native.java_test using a generated test_class.
(attached)
junit.txt

Baseline: 19b5675 Incompatible changes: - Skylark rules that are available from their own repository will now issue a warning when accessed through @bazel_tools. - Set --legacy_bazel_java_test to off by default. java_test will now have a slightly different behaviour, correctly emitting XML file but, as a downside, it needs correct declaration of the test suite (see #1017). - Labels in .bzl files in remote repositories will be resolved relative to their repository (instead of the repository the Skylark rule is used in). - Renamed proto_java_library to java_proto_library. The former is now deprecated and will print out a warning when used. - android_sdk now compiles android_jack on the fly from android_jar, which means android_jar must be a jar and android_jack is now deprecated. The Jack tools (jack, jill, resource_extractor) must be specified. - Any project that depended on the objc_options rule will be broken. Can be fixed by adding attrs (infoplists,copts) directly to rules depending on the options. - .aidl files correctly require import statements for types defined in the same package and the same android_library. New features: - Experimental Windows support is available. - Experimental support for writing remote repository rules in Skylark is available. - iOS ipa_post_processor attribute allows for user-defined IPA edits. - Adds a to_json method to Skylark structs, providing conversion to JSON format. - Native python rule can depend on skylark rule as long as skylark rule provides 'py' provider. - When using both --verbose_failures and --sandbox_debug, Bazel prints instructions how to spawn a debugging shell inside the sandbox. - add flag --sandbox_add_path, which takes a list of additional paths as argument and mount these paths to sandbox. Important changes: - @androidsdk//:org_apache_http_legacy added for the legacy Apache classes for android sdk version 23 and above. - Genrules correctly work when used with bazel run. - When namespace-sandbox is run with the -D (debug) flag and inside a terminal, it spawns a shell inside the sandbox to aid in debugging when the sandboxed command fails. - Added --artifact to workspace generator for generating workspace and build file rules from artifact coodrinates. - Specifying --experimental_android_resource_shrinking on the command line will enable a resource shrinking pass on android_binary targets that already use Proguard. - J2ObjC updated to 1.0.1 release. - Added "root_symlinks" and "symlinks" parameters to Skylark runfiles() method. - You can no longer use objc_binary targets for the xctest_app attribute of an ios_test rule. - Enable overriding jsonnet binaries and stdlib for Jsonnet rules. - mount target of /etc/resolv.conf if it is a symlink. - Tests that failed to build because execution was halted no longer print their status. - Bazel warns if a cc rule's includes attribute contains up-level references that escape its package. - Add repository_ctx.download and repository_ctx.download_and_extract function.

Baseline: 19b5675 Incompatible changes: - Skylark rules that are available from their own repository will now issue a warning when accessed through @bazel_tools. - Set --legacy_bazel_java_test to off by default. java_test will now have a slightly different behaviour, correctly emitting XML file but, as a downside, it needs correct declaration of the test suite (see bazelbuild#1017). - Labels in .bzl files in remote repositories will be resolved relative to their repository (instead of the repository the Skylark rule is used in). - Renamed proto_java_library to java_proto_library. The former is now deprecated and will print out a warning when used. - android_sdk now compiles android_jack on the fly from android_jar, which means android_jar must be a jar and android_jack is now deprecated. The Jack tools (jack, jill, resource_extractor) must be specified. - Any project that depended on the objc_options rule will be broken. Can be fixed by adding attrs (infoplists,copts) directly to rules depending on the options. - .aidl files correctly require import statements for types defined in the same package and the same android_library. New features: - Experimental Windows support is available. - Experimental support for writing remote repository rules in Skylark is available. - iOS ipa_post_processor attribute allows for user-defined IPA edits. - Adds a to_json method to Skylark structs, providing conversion to JSON format. - Native python rule can depend on skylark rule as long as skylark rule provides 'py' provider. - When using both --verbose_failures and --sandbox_debug, Bazel prints instructions how to spawn a debugging shell inside the sandbox. - add flag --sandbox_add_path, which takes a list of additional paths as argument and mount these paths to sandbox. Important changes: - @androidsdk//:org_apache_http_legacy added for the legacy Apache classes for android sdk version 23 and above. - Genrules correctly work when used with bazel run. - When namespace-sandbox is run with the -D (debug) flag and inside a terminal, it spawns a shell inside the sandbox to aid in debugging when the sandboxed command fails. - Added --artifact to workspace generator for generating workspace and build file rules from artifact coodrinates. - Specifying --experimental_android_resource_shrinking on the command line will enable a resource shrinking pass on android_binary targets that already use Proguard. - J2ObjC updated to 1.0.1 release. - Added "root_symlinks" and "symlinks" parameters to Skylark runfiles() method. - You can no longer use objc_binary targets for the xctest_app attribute of an ios_test rule. - Enable overriding jsonnet binaries and stdlib for Jsonnet rules. - mount target of /etc/resolv.conf if it is a symlink. - Tests that failed to build because execution was halted no longer print their status. - Bazel warns if a cc rule's includes attribute contains up-level references that escape its package. - Add repository_ctx.download and repository_ctx.download_and_extract function.

ittaiz · 2016-05-18T13:23:09Z

@damienmg does this mean you can't have a simple test target which runs all tests in the package?

alltom · 2016-08-07T14:36:04Z

It seems like the documentation still needs to be updated. http://www.bazel.io/docs/be/java.html#java_test_examples has an example that just globs *.java, and from what I understand in this thread (and trying it on my own), that no longer works.

darkcrawler01 · 2016-12-01T11:45:57Z

Please update the documentation

wcurrie · 2018-11-18T05:29:05Z

Looking at how bazel uses java_test internally, the approach appears to be:

define a junit Suite that finds tests by classpath scanning. eg AllTests
build it using a java_library rule. eg test_runner
include test_runner as a dep in individual java_test rules, listing the Suite from (1) as the test_class

bazel/src/test/java/com/google/devtools/build/lib/BUILD

Lines 165 to 180 in f4023b9

    
           java_test( 
        
               name = "cmdline_test", 
        
               size = "small", 
        
               srcs = glob(["cmdline/*.java"]), 
        
               tags = [ 
        
                   "foundations", 
        
               ], 
        
               test_class = "com.google.devtools.build.lib.AllTests", 
        
               deps = [ 
        
                   ":guava_junit_truth", 
        
                   ":test_runner", 
        
                   ":testutil", 
        
                   "//src/main/java/com/google/devtools/build/lib/cmdline", 
        
                   "//src/main/java/com/google/devtools/build/lib/vfs:pathfragment", 
        
               ], 
        
           )

damienmg added the under investigation label Mar 6, 2016

damienmg self-assigned this Mar 6, 2016

damienmg closed this as completed Mar 21, 2016

taishi8117 mentioned this issue Jun 16, 2016

#816 compile heron with bazel 0.2.3 apache/incubator-heron#869

Merged

wcurrie unassigned damienmg Nov 18, 2018

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

java_test "broken" with no guidance how to fix #1017

java_test "broken" with no guidance how to fix #1017

pmbethe09 commented Mar 5, 2016

damienmg commented Mar 5, 2016

damienmg commented Mar 6, 2016

damienmg commented Mar 6, 2016

pmbethe09 commented Mar 7, 2016

damienmg commented Mar 7, 2016

pmbethe09 commented Mar 7, 2016

damienmg commented Mar 8, 2016

damienmg commented Mar 8, 2016

pmbethe09 commented Mar 8, 2016

pmbethe09 commented Mar 8, 2016

damienmg commented Mar 8, 2016

pmbethe09 commented Mar 9, 2016

pmbethe09 commented Mar 21, 2016

ittaiz commented May 18, 2016

alltom commented Aug 7, 2016

darkcrawler01 commented Dec 1, 2016

wcurrie commented Nov 18, 2018

java_test "broken" with no guidance how to fix #1017

java_test "broken" with no guidance how to fix #1017

Comments

pmbethe09 commented Mar 5, 2016

damienmg commented Mar 5, 2016

damienmg commented Mar 6, 2016

damienmg commented Mar 6, 2016

pmbethe09 commented Mar 7, 2016

damienmg commented Mar 7, 2016

pmbethe09 commented Mar 7, 2016

damienmg commented Mar 8, 2016

damienmg commented Mar 8, 2016

pmbethe09 commented Mar 8, 2016

pmbethe09 commented Mar 8, 2016

damienmg commented Mar 8, 2016

pmbethe09 commented Mar 9, 2016

pmbethe09 commented Mar 21, 2016

ittaiz commented May 18, 2016

alltom commented Aug 7, 2016

darkcrawler01 commented Dec 1, 2016

wcurrie commented Nov 18, 2018