Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Using sparkledriver with certain dependencies, causes the tests to fail #5

Closed
dotemacs opened this issue Jul 19, 2017 · 6 comments
Closed

Using sparkledriver with certain dependencies, causes the tests to fail #5

dotemacs opened this issue Jul 19, 2017 · 6 comments

Comments

@dotemacs
Copy link
Contributor

dotemacs commented Jul 19, 2017 •
edited
Loading

This might be a fault of my own, but I'm not sure how to use sparkledriver within a project.

I added it to a real project and started to experience issues. So to reproduce the issue, I created a sample repo to show you what I mean.

Adding it to :dependencies, like here:

https://github.com/dotemacs/glitter-truck/blob/master/project.clj#L7

the simple tests pass just fine:

https://github.com/dotemacs/glitter-truck/blob/master/test/glitter_truck/core_test.clj

But including any of the dependencies which I have in the real project, such as these (that are commented out, in with-loads-of-deps branch):

https://github.com/dotemacs/glitter-truck/blob/with-loads-of-deps/project.clj#L7-L9

the tests fail with:

Click to expand stacktrace $ lein test

lein test glitter-truck.core-test
SLF4J: Failed to load class "org.slf4j.impl.StaticLoggerBinder".
SLF4J: Defaulting to no-operation (NOP) logger implementation
SLF4J: See http://www.slf4j.org/codes.html#StaticLoggerBinder for further details.
[2017-07-18T23:47:41.846][Instance 1][Port 49990] Jul 19, 2017 12:47:41 AM com.sun.webkit.network.URLLoader doRun
[2017-07-18T23:47:41.846][Instance 1][Port 49990] WARNING: Unexpected error
[2017-07-18T23:47:41.846][Instance 1][Port 49990] java.io.IOException: org.apache.commons.codec.digest.DigestUtils.sha1Hex(Ljava/lang/String;)Ljava/lang/String;: http://clojure.org/
[2017-07-18T23:47:41.846][Instance 1][Port 49990] at com.machinepublishers.jbrowserdriver.StreamConnection.exec(StreamConnection.java:367)
[2017-07-18T23:47:41.846][Instance 1][Port 49990] at com.machinepublishers.jbrowserdriver.StreamConnection.getResponseCode(StreamConnection.java:447)
[2017-07-18T23:47:41.846][Instance 1][Port 49990] at com.sun.webkit.network.URLLoader.receiveResponse(URLLoader.java:420)
[2017-07-18T23:47:41.846][Instance 1][Port 49990] at com.sun.webkit.network.URLLoader.doRun(URLLoader.java:164)
[2017-07-18T23:47:41.846][Instance 1][Port 49990] at com.sun.webkit.network.URLLoader.lambda$run$94(URLLoader.java:129)
[2017-07-18T23:47:41.846][Instance 1][Port 49990] at java.security.AccessController.doPrivileged(Native Method)
[2017-07-18T23:47:41.846][Instance 1][Port 49990] at com.sun.webkit.network.URLLoader.run(URLLoader.java:128)
[2017-07-18T23:47:41.847][Instance 1][Port 49990] at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
[2017-07-18T23:47:41.847][Instance 1][Port 49990] at java.util.concurrent.FutureTask.run(FutureTask.java:266)
[2017-07-18T23:47:41.847][Instance 1][Port 49990] at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
[2017-07-18T23:47:41.847][Instance 1][Port 49990] at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
[2017-07-18T23:47:41.847][Instance 1][Port 49990] at java.lang.Thread.run(Thread.java:745)
[2017-07-18T23:47:41.847][Instance 1][Port 49990] Caused by: java.lang.NoSuchMethodError: org.apache.commons.codec.digest.DigestUtils.sha1Hex(Ljava/lang/String;)Ljava/lang/String;
[2017-07-18T23:47:41.847][Instance 1][Port 49990] at com.machinepublishers.jbrowserdriver.HttpCache.getEntry(HttpCache.java:87)
[2017-07-18T23:47:41.847][Instance 1][Port 49990] at org.apache.http.impl.client.cache.CacheInvalidator.getEntry(CacheInvalidator.java:137)
[2017-07-18T23:47:41.847][Instance 1][Port 49990] at org.apache.http.impl.client.cache.CacheInvalidator.flushInvalidatedCacheEntries(CacheInvalidator.java:86)
[2017-07-18T23:47:41.847][Instance 1][Port 49990] at org.apache.http.impl.client.cache.BasicHttpCache.flushInvalidatedCacheEntriesFor(BasicHttpCache.java:361)
[2017-07-18T23:47:41.847][Instance 1][Port 49990] at org.apache.http.impl.client.cache.CachingExec.flushEntriesInvalidatedByRequest(CachingExec.java:420)
[2017-07-18T23:47:41.847][Instance 1][Port 49990] at org.apache.http.impl.client.cache.CachingExec.execute(CachingExec.java:265)
[2017-07-18T23:47:41.848][Instance 1][Port 49990] at org.apache.http.impl.execchain.ProtocolExec.execute(ProtocolExec.java:184)
[2017-07-18T23:47:41.848][Instance 1][Port 49990] at org.apache.http.impl.client.InternalHttpClient.doExecute(InternalHttpClient.java:184)
[2017-07-18T23:47:41.848][Instance 1][Port 49990] at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:82)
[2017-07-18T23:47:41.848][Instance 1][Port 49990] at com.machinepublishers.jbrowserdriver.StreamConnectionClient.execute(StreamConnectionClient.java:175)
[2017-07-18T23:47:41.848][Instance 1][Port 49990] at com.machinepublishers.jbrowserdriver.StreamConnection.exec(StreamConnection.java:333)
[2017-07-18T23:47:41.848][Instance 1][Port 49990] ... 11 more
[2017-07-18T23:47:41.848][Instance 1][Port 49990]

lein test :only glitter-truck.core-test/simplest-test

FAIL in (simplest-test) (core_test.clj:8)
status code
expected: (= 200 (-> (make-browser) (fetch! "http://clojure.org") status-code))
actual: (not (= 200 499))
[2017-07-18T23:47:43.013][Instance 2][Port 50001] Jul 19, 2017 12:47:43 AM com.sun.webkit.network.URLLoader doRun
[2017-07-18T23:47:43.013][Instance 2][Port 50001] WARNING: Unexpected error
[2017-07-18T23:47:43.013][Instance 2][Port 50001] java.io.IOException: org.apache.commons.codec.digest.DigestUtils.sha1Hex(Ljava/lang/String;)Ljava/lang/String;: http://clojure.org/
[2017-07-18T23:47:43.013][Instance 2][Port 50001] at com.machinepublishers.jbrowserdriver.StreamConnection.exec(StreamConnection.java:367)
[2017-07-18T23:47:43.013][Instance 2][Port 50001] at com.machinepublishers.jbrowserdriver.StreamConnection.getResponseCode(StreamConnection.java:447)
[2017-07-18T23:47:43.013][Instance 2][Port 50001] at com.sun.webkit.network.URLLoader.receiveResponse(URLLoader.java:420)
[2017-07-18T23:47:43.013][Instance 2][Port 50001] at com.sun.webkit.network.URLLoader.doRun(URLLoader.java:164)
[2017-07-18T23:47:43.013][Instance 2][Port 50001] at com.sun.webkit.network.URLLoader.lambda$run$94(URLLoader.java:129)
[2017-07-18T23:47:43.013][Instance 2][Port 50001] at java.security.AccessController.doPrivileged(Native Method)
[2017-07-18T23:47:43.013][Instance 2][Port 50001] at com.sun.webkit.network.URLLoader.run(URLLoader.java:128)
[2017-07-18T23:47:43.013][Instance 2][Port 50001] at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
[2017-07-18T23:47:43.013][Instance 2][Port 50001] at java.util.concurrent.FutureTask.run(FutureTask.java:266)
[2017-07-18T23:47:43.014][Instance 2][Port 50001] at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
[2017-07-18T23:47:43.014][Instance 2][Port 50001] at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
[2017-07-18T23:47:43.014][Instance 2][Port 50001] at java.lang.Thread.run(Thread.java:745)
[2017-07-18T23:47:43.014][Instance 2][Port 50001] Caused by: java.lang.NoSuchMethodError: org.apache.commons.codec.digest.DigestUtils.sha1Hex(Ljava/lang/String;)Ljava/lang/String;
[2017-07-18T23:47:43.014][Instance 2][Port 50001] at com.machinepublishers.jbrowserdriver.HttpCache.getEntry(HttpCache.java:87)
[2017-07-18T23:47:43.014][Instance 2][Port 50001] at org.apache.http.impl.client.cache.CacheInvalidator.getEntry(CacheInvalidator.java:137)
[2017-07-18T23:47:43.014][Instance 2][Port 50001] at org.apache.http.impl.client.cache.CacheInvalidator.flushInvalidatedCacheEntries(CacheInvalidator.java:86)
[2017-07-18T23:47:43.014][Instance 2][Port 50001] at org.apache.http.impl.client.cache.BasicHttpCache.flushInvalidatedCacheEntriesFor(BasicHttpCache.java:361)
[2017-07-18T23:47:43.014][Instance 2][Port 50001] at org.apache.http.impl.client.cache.CachingExec.flushEntriesInvalidatedByRequest(CachingExec.java:420)
[2017-07-18T23:47:43.014][Instance 2][Port 50001] at org.apache.http.impl.client.cache.CachingExec.execute(CachingExec.java:265)
[2017-07-18T23:47:43.014][Instance 2][Port 50001] at org.apache.http.impl.execchain.ProtocolExec.execute(ProtocolExec.java:184)
[2017-07-18T23:47:43.014][Instance 2][Port 50001] at org.apache.http.impl.client.InternalHttpClient.doExecute(InternalHttpClient.java:184)
[2017-07-18T23:47:43.015][Instance 2][Port 50001] at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:82)
[2017-07-18T23:47:43.015][Instance 2][Port 50001] at com.machinepublishers.jbrowserdriver.StreamConnectionClient.execute(StreamConnectionClient.java:175)
[2017-07-18T23:47:43.015][Instance 2][Port 50001] at com.machinepublishers.jbrowserdriver.StreamConnection.exec(StreamConnection.java:333)
[2017-07-18T23:47:43.015][Instance 2][Port 50001] ... 11 more
[2017-07-18T23:47:43.015][Instance 2][Port 50001]

lein test :only glitter-truck.core-test/simplest-test

ERROR in (simplest-test) (NativeConstructorAccessorImpl.java:-2)
matching line
expected: (= "I hope you find Clojure's combination of facilities elegant, powerful, practical and fun to use." (foo))
actual: org.openqa.selenium.NoSuchElementException: Element not found or does not exist.
For documentation on this error, please visit: http://seleniumhq.org/exceptions/no_such_element.html
Build info: version: '2.53.0', revision: '35ae25b1534ae328c771e0856c93e187490ca824', time: '2016-03-15 10:43:46'
System info: host: 'machine.local', ip: '192.168.1.100', os.name: 'Mac OS X', os.arch: 'x86_64', os.version: '10.11.6', java.version: '1.8.0_102'
Driver info: driver.version: unknown
For documentation on this error, please visit: http://seleniumhq.org/exceptions/no_such_element.html
Build info: version: '2.53.0', revision: '35ae25b1534ae328c771e0856c93e187490ca824', time: '2016-03-15 10:43:46'
System info: host: 'machine.local', ip: '192.168.1.100', os.name: 'Mac OS X', os.arch: 'x86_64', os.version: '10.11.6', java.version: '1.8.0_102'
Driver info: driver.version: JBrowserDriver
at sun.reflect.NativeConstructorAccessorImpl.newInstance0 (NativeConstructorAccessorImpl.java:-2)
sun.reflect.NativeConstructorAccessorImpl.newInstance (NativeConstructorAccessorImpl.java:62)
sun.reflect.DelegatingConstructorAccessorImpl.newInstance (DelegatingConstructorAccessorImpl.java:45)
java.lang.reflect.Constructor.newInstance (Constructor.java:423)
com.machinepublishers.jbrowserdriver.Util.handleException (Util.java:131)
com.machinepublishers.jbrowserdriver.JBrowserDriver.findElementsByXPath (JBrowserDriver.java:769)
sun.reflect.NativeMethodAccessorImpl.invoke0 (NativeMethodAccessorImpl.java:-2)
sun.reflect.NativeMethodAccessorImpl.invoke (NativeMethodAccessorImpl.java:62)
sun.reflect.DelegatingMethodAccessorImpl.invoke (DelegatingMethodAccessorImpl.java:43)
java.lang.reflect.Method.invoke (Method.java:498)
clojure.lang.Reflector.invokeMatchingMethod (Reflector.java:93)
clojure.lang.Reflector.invokeInstanceMethod (Reflector.java:28)
sparkledriver.core$find_by_xpath_STAR_.invokeStatic (core.clj:127)
sparkledriver.core$find_by_xpath_STAR_.invoke (core.clj:124)
glitter_truck.core$foo.invokeStatic (core.clj:8)
glitter_truck.core$foo.invoke (core.clj:4)
glitter_truck.core_test$fn__345.invokeStatic (core_test.clj:14)
glitter_truck.core_test/fn (core_test.clj:6)
clojure.test$test_var$fn__7983.invoke (test.clj:716)
clojure.test$test_var.invokeStatic (test.clj:716)
clojure.test$test_var.invoke (test.clj:707)
clojure.test$test_vars$fn__8005$fn__8010.invoke (test.clj:734)
clojure.test$default_fixture.invokeStatic (test.clj:686)
clojure.test$default_fixture.invoke (test.clj:682)
clojure.test$test_vars$fn__8005.invoke (test.clj:734)
clojure.test$default_fixture.invokeStatic (test.clj:686)
clojure.test$default_fixture.invoke (test.clj:682)
clojure.test$test_vars.invokeStatic (test.clj:730)
clojure.test$test_all_vars.invokeStatic (test.clj:736)
clojure.test$test_ns.invokeStatic (test.clj:757)
clojure.test$test_ns.invoke (test.clj:742)
user$eval85$fn__136.invoke (form-init5469215561029139845.clj:1)
clojure.lang.AFn.applyToHelper (AFn.java:156)
clojure.lang.AFn.applyTo (AFn.java:144)
clojure.core$apply.invokeStatic (core.clj:648)
clojure.core$apply.invoke (core.clj:641)
leiningen.core.injected$compose_hooks$fn__19.doInvoke (form-init5469215561029139845.clj:1)
clojure.lang.RestFn.applyTo (RestFn.java:137)
clojure.core$apply.invokeStatic (core.clj:646)
clojure.core$apply.invoke (core.clj:641)
leiningen.core.injected$run_hooks.invokeStatic (form-init5469215561029139845.clj:1)
leiningen.core.injected$run_hooks.invoke (form-init5469215561029139845.clj:1)
leiningen.core.injected$prepare_for_hooks$fn__24$fn__25.doInvoke (form-init5469215561029139845.clj:1)
clojure.lang.RestFn.applyTo (RestFn.java:137)
clojure.lang.AFunction$1.doInvoke (AFunction.java:29)
clojure.lang.RestFn.invoke (RestFn.java:408)
clojure.core$map$fn__4785.invoke (core.clj:2646)
clojure.lang.LazySeq.sval (LazySeq.java:40)
clojure.lang.LazySeq.seq (LazySeq.java:49)
clojure.lang.Cons.next (Cons.java:39)
clojure.lang.RT.boundedLength (RT.java:1749)
clojure.lang.RestFn.applyTo (RestFn.java:130)
clojure.core$apply.invokeStatic (core.clj:648)
clojure.test$run_tests.invokeStatic (test.clj:767)
clojure.test$run_tests.doInvoke (test.clj:767)
clojure.lang.RestFn.applyTo (RestFn.java:137)
clojure.core$apply.invokeStatic (core.clj:646)
clojure.core$apply.invoke (core.clj:641)
user$eval85$fn__148$fn__179.invoke (form-init5469215561029139845.clj:1)
user$eval85$fn__148$fn__149.invoke (form-init5469215561029139845.clj:1)
user$eval85$fn__148.invoke (form-init5469215561029139845.clj:1)
user$eval85.invokeStatic (form-init5469215561029139845.clj:1)
user$eval85.invoke (form-init5469215561029139845.clj:1)
clojure.lang.Compiler.eval (Compiler.java:6927)
clojure.lang.Compiler.eval (Compiler.java:6917)
clojure.lang.Compiler.load (Compiler.java:7379)
clojure.lang.Compiler.loadFile (Compiler.java:7317)
clojure.main$load_script.invokeStatic (main.clj:275)
clojure.main$init_opt.invokeStatic (main.clj:277)
clojure.main$init_opt.invoke (main.clj:277)
clojure.main$initialize.invokeStatic (main.clj:308)
clojure.main$null_opt.invokeStatic (main.clj:342)
clojure.main$null_opt.invoke (main.clj:339)
clojure.main$main.invokeStatic (main.clj:421)
clojure.main$main.doInvoke (main.clj:384)
clojure.lang.RestFn.invoke (RestFn.java:421)
clojure.lang.Var.invoke (Var.java:383)
clojure.lang.AFn.applyToHelper (AFn.java:156)
clojure.lang.Var.applyTo (Var.java:700)
clojure.main.main (main.java:37)
Caused by: org.openqa.selenium.NoSuchElementException: Element not found or does not exist.
For documentation on this error, please visit: http://seleniumhq.org/exceptions/no_such_element.html
Build info: version: '2.53.0', revision: '35ae25b1534ae328c771e0856c93e187490ca824', time: '2016-03-15 10:43:46'
System info: host: 'machine.local', ip: '192.168.1.100', os.name: 'Mac OS X', os.arch: 'x86_64', os.version: '10.11.6', java.version: '1.8.0_102'
Driver info: driver.version: unknown
at com.machinepublishers.jbrowserdriver.ElementServer.validate (ElementServer.java:212)
com.machinepublishers.jbrowserdriver.ElementServer.access$000 (ElementServer.java:61)
com.machinepublishers.jbrowserdriver.ElementServer$1.perform (ElementServer.java:155)
com.machinepublishers.jbrowserdriver.AppThread$Runner.run (AppThread.java:91)
com.sun.javafx.application.PlatformImpl.lambda$null$173 (PlatformImpl.java:295)
java.security.AccessController.doPrivileged (AccessController.java:-2)
com.sun.javafx.application.PlatformImpl.lambda$runLater$174 (PlatformImpl.java:294)
com.machinepublishers.glass.ui.monocle.RunnableProcessor.runLoop (RunnableProcessor.java:94)
com.machinepublishers.glass.ui.monocle.RunnableProcessor.run (RunnableProcessor.java:53)
java.lang.Thread.run (Thread.java:745)

Ran 1 tests containing 2 assertions.
1 failures, 1 errors.
Tests failed.

Then I tried to setup project.clj like this, with sparkledriver under :test profile:

(defproject glitter-truck "0.1.0-SNAPSHOT"
  :description "FIXME: write description"
  :url "http://example.com/FIXME"
  :license {:name "Eclipse Public License"
            :url "http://www.eclipse.org/legal/epl-v10.html"}
  :dependencies [[org.clojure/clojure "1.8.0"]
                 ;; [ring "1.6.2"]
                 ;; [compojure "1.6.0"]
                 ;; [crypto-random "1.2.0"]
                 ]
  :profiles {:test {:dependencies [[sparkledriver "0.1.8"]]}})

Running the test like this, it passes, but uncommenting one of the dependencies, it fails again, with the same error.

Is there something else that needs to be done to use sparkledriver within a project?
Am I missing something really obvious?
@dotemacs
Copy link
Contributor Author

Just for completeness:

  • I don't have any firewalls that would play havoc with the external HTTP calls
  • my environment:

OS: OSX 10.11.6

$ java -version
java version "1.8.0_102"
Java(TM) SE Runtime Environment (build 1.8.0_102-b14)
Java HotSpot(TM) 64-Bit Server VM (build 25.102-b14, mixed mode)

@jackrusher
Copy link
Owner

jackrusher commented Jul 19, 2017 •
edited
Loading

@dotemacs Ok, I was able to reproduce this only with crypto-random (the other libraries caused no bother). My guess is that crypto-random pulls in a different version of org.apache.commons.codec.digest.DigestUtils that doesn't supply the sha1Hex method, which is causing the jBrowserDriver's cache implementation to throw a java.lang.NoSuchMethodError exception.

@dotemacs
Copy link
Contributor Author

@jackrusher thanks for that.

Running lein deps :tree gave me some insight.

Bothring & compojure require crypto-random as a dependency.

But I do experience the failure when I have just ring or just compojure included. Not sure why it passes for you.

I wonder what is it that you do or have setup that doesn't trigger this behaviour for ring & compojure?

Are you using the example repo that I've quoted above or are you using those libraries in some other project?

Thanks

@dotemacs
Copy link
Contributor Author

By adding the following:

[commons-codec "1.7"]

dependency, the tests pass even if you include all three libraries: ring, compojure & crypto-random.

I went with 1.7 as the lowest version that it passed with. Obviously going with a higher version ensures that tests pass also.

Should this be included as a dependency in some way in order to prevent other potential users from stumbling upon this issue ? I'm happy to do a pull request but not sure what you think about it.

Thanks

@dotemacs
Copy link
Contributor Author

Or maybe just a note in the README ?

Happy to do the PR whichever way you want to go.

@jackrusher
Copy link
Owner

I checked out your repo and just ran lein test for each combination of deps. No idea why they all fail for you. Do you have something relevant in your ~/.lein/profiles.clj, maybe?

It looks like the current (highly transitive) dependency for jBrowserDriver is [commons-codec "1.9"], so I'd recommend that one in order to avoid any weirdness upstream. I'm not sure about adding it in sparkledriver, but I would definitely take a PR for the README mentioning the problem/solution.

@dotemacs dotemacs mentioned this issue Jul 19, 2017
Merged
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
2 participants
@dotemacs @jackrusher