Allow setting S3 certificate authority (CA) list

[effnish]

Is your feature request related to a problem? Please describe.

Our minio cluster is served over https using a certificate signed by an internal CA. If I understand the code correctly, the S3 state-store compiles in the Mozilla CA list and can't be overridden.

Describe the solution you'd like

It would be very helpful if I could set the common SSL_CERT_FILE environment variable or another config parameter to have RisingWave use the specified file or the system CA list for the S3 connector.

Describe alternatives you've considered

I'm currently running the software with a custom Traefik route and a hostAlias in Kubernetes to proxy the connection. e.g. RisingWave (http) -> Traefik -> (https) Minio

Additional context

No response

[tabVersion]

#17826 is trying to fix the issue. Are you willing to help with testing?

[effnish]

I'm not sure if #17826 addresses this issue. I'm completely ignorant of the code structure, but I think src/object_store/src/object/s3.rs#L660 is what was tripping me up.

let conn = hyper_rustls::HttpsConnectorBuilder::new()
            .with_webpki_roots() // <-- prevents using a self-signed CA cert for Minio as object store for RisingWave itself
            .https_or_http()
            .enable_all_versions()
            .wrap_connector(http);

[tabVersion]

I'm not sure if #17826 addresses this issue. I'm completely ignorant of the code structure, but I think src/object_store/src/object/s3.rs#L660 is what was tripping me up.

let conn = hyper_rustls::HttpsConnectorBuilder::new()
            .with_webpki_roots() // <-- prevents using a self-signed CA cert for Minio as object store for RisingWave itself
            .https_or_http()
            .enable_all_versions()
            .wrap_connector(http);

Oh, I got the problem. Do you mean you are using a s3 service which needs a custom cert to connect to? cc @wcy-fdu

[effnish]

Yes, sorry I was clear before. We're running Minio as a separate deployment in k8s that's enforcing all connections are over https. The certificate it serves is signed by a custom/internal CA certificate. The .with_webpki_roots() call only trusts the certificate authorities that are shipped by Mozilla and ignores the server's CA store and other overrides.

When I configure the following in helm, I get certificate validation errors:

state_store:
  minio:
    enabled: true
    endpoint: https://minio.example.internal
    bucket: risingwave

[effnish]

This looks to be resolved by #18011