Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add httpConnectTimeout and httpReadTimeout to google config #3974

Merged
merged 17 commits into from
Jun 5, 2023

Conversation

bentsherman
Copy link
Member

Close #3973

@pditommaso
Copy link
Member

Many tests fails with this error

Unable to load cache item
java.lang.IllegalStateException: Unable to load cache item
	at net.sf.cglib.core.internal.LoadingCache.createEntry(LoadingCache.java:79)
	at net.sf.cglib.core.internal.LoadingCache.get(LoadingCache.java:34)
	at net.sf.cglib.core.AbstractClassGenerator$ClassLoaderData.get(AbstractClassGenerator.java:119)
	at net.sf.cglib.core.AbstractClassGenerator.create(AbstractClassGenerator.java:294)
	at net.sf.cglib.proxy.Enhancer.createHelper(Enhancer.java:480)
	at net.sf.cglib.proxy.Enhancer.createClass(Enhancer.java:337)
	at org.spockframework.mock.runtime.CglibMockFactory.createMock(CglibMockFactory.java:34)
	at org.spockframework.mock.runtime.ProxyBasedMockFactory.create(ProxyBasedMockFactory.java:45)
	at org.spockframework.mock.runtime.JavaMockFactory.createInternal(JavaMockFactory.java:58)
	at org.spockframework.mock.runtime.JavaMockFactory.create(JavaMockFactory.java:38)
	at org.spockframework.mock.runtime.CompositeMockFactory.create(CompositeMockFactory.java:44)
	at org.spockframework.lang.SpecInternals.createMock(SpecInternals.java:50)
	at org.spockframework.lang.SpecInternals.createMockImpl(SpecInternals.java:305)
	at org.spockframework.lang.SpecInternals.createMockImpl(SpecInternals.java:295)
	at org.spockframework.lang.SpecInternals.MockImpl(SpecInternals.java:108)
	at nextflow.cloud.google.lifesciences.GoogleLifeSciencesHelperTest.$spock_initializeFields(GoogleLifeSciencesHelperTest.groovy:36)
Caused by: java.lang.ExceptionInInitializerError
	at java.base/java.lang.Class.forName(Class.java:495)
	at java.base/java.lang.Class.forName(Class.java:474)
	at net.sf.cglib.core.ReflectUtils.defineClass(ReflectUtils.java:467)
	at net.sf.cglib.core.AbstractClassGenerator.generate(AbstractClassGenerator.java:339)
	at net.sf.cglib.proxy.Enhancer.generate(Enhancer.java:492)
	at net.sf.cglib.core.AbstractClassGenerator$ClassLoaderData$3.apply(AbstractClassGenerator.java:96)
	at net.sf.cglib.core.AbstractClassGenerator$ClassLoaderData$3.apply(AbstractClassGenerator.java:94)
	at net.sf.cglib.core.internal.LoadingCache$2.call(LoadingCache.java:54)
	at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:317)
	at net.sf.cglib.core.internal.LoadingCache.createEntry(LoadingCache.java:61)
	... 15 more
Caused by: java.lang.IllegalStateException: You are currently running with version 2.2.0 of google-api-client. You need at least version 1.31.1 of google-api-client to run version 1.31.5 of the Cloud Life Sciences API library.
	at com.google.common.base.Preconditions.checkState(Preconditions.java:534)
	at com.google.api.client.util.Preconditions.checkState(Preconditions.java:113)
	at com.google.api.services.lifesciences.v2beta.CloudLifeSciences.<clinit>(CloudLifeSciences.java:44)
	... 25 more

@bentsherman bentsherman requested a review from pditommaso May 30, 2023 19:51
@bentsherman
Copy link
Member Author

@pditommaso this one is ready to merge

It turns out the API changes were made somewhere between 2.9 and 2.22 . Since google-cloud-storage 2.9 is already an implicit dependency, I just removed it from the build file, tests are passing.

@pditommaso
Copy link
Member

But some specific API are used from that library, right? If so it's better to include an explicit deps

@bentsherman bentsherman force-pushed the 3973-google-storage-http-timeout branch from 8abe53c to caea2ba Compare May 31, 2023 16:47
@bentsherman
Copy link
Member Author

Fair enough, I tested with the explicit dependency and it also works, see the second-to-last commit. I tried to trick github by force-removing the last commit, but it's running the tests again anyway

pditommaso and others added 9 commits June 5, 2023 10:24
This commits adds the support for AWS Simple Email Service as an alternative
transport for sending notification emails from nextflow.

The email is sent via AWS SES when using the nf-amazon plugin, and:
- the NXF_ENABLE_AWS_SES=true environment variable is set
- or, not `mail.smtp` setting is provided and the AWS_REGION or AWS_DEFAULT_REGION is set
  in the launching environment

Signed-off-by: Paolo Di Tommaso <[email protected]>
Signed-off-by: Paolo Di Tommaso <[email protected]>
This commit fixed the access to public AWS S3 buckets when no AWS credentials 
are available in the hosting environment. 

It uses a custom credentials provider that fallbacks to `AnonymousAWSCredentials` 
class when no creds are detected. 

Signed-off-by: Ben Sherman <[email protected]>
Signed-off-by: Paolo Di Tommaso <[email protected]>
Co-authored-by: Paolo Di Tommaso <[email protected]>
Signed-off-by: Paolo Di Tommaso <[email protected]>
@pditommaso pditommaso merged commit 49fa15f into master Jun 5, 2023
@pditommaso pditommaso deleted the 3973-google-storage-http-timeout branch June 5, 2023 14:53
@pditommaso pditommaso added this to the 23.10.0 milestone Jun 14, 2023
abhi18av pushed a commit to abhi18av/nextflow that referenced this pull request Oct 28, 2023
…w-io#3974)



Signed-off-by: Paolo Di Tommaso <[email protected]>
Signed-off-by: Ben Sherman <[email protected]>
Co-authored-by: Paolo Di Tommaso <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Google batch timeout - add google.batch.storage.httpRequestTimeout configuration option
2 participants