Concurrrent request issue in 3.15.3

[jcjveraa]

When we cause multiple SOAP requests 'in parallel' (fire off a number of requests at a rate of a few per second - not too fast as we can do it manually) we notice that our SOAPHandler has some strange issues.

What do we do in the SOAPHandler:
We add some custom headers with text content to the SOAP message in that handler using (in pseudocode to give an indication of what we are doing) SoapHeader.addHeaderElement(new QName(...)) and on that element element.setTextContent(content) - nothing too special.

Expected result
Headers are added 'independently' for each requests, no interaction between SOAP calls.

Actual result
As soon as we trigger some (low, few per second) limit we get a pair of messages:

1. a message where the headers are not added at all
2. a message where the header is added in duplicate.

In the logging:

2025-01-13 15:52:38,579 ERROR [my.company.our.ClientClass] (executor-thread-1)!: jakarta.xml.ws.soap.SOAPFaultException: 1100 A header representing a Message Addressing Property is not present. (Reason: the required header element ourCustom:Header is absent)
...
2025-01-13 15:52:38,590 ERROR [my.company.our.ClientClass] (executor-thread-2) !: jakarta.xml.ws.soap.SOAPFaultException: 1101 A header representing a Message Addressing Property is not valid and the message cannot be processed. (Reason: there is a greater than expected number of the specified header element ourCustom:Header)
...

What I can see is that each is on a seperate executor-thread, if that gives any indication. I also notice that the errors are 11 ms apart, while the 'entering of requests' is done via me clicking a button in our GUI so there is no way that I can fire them off so quickly.

We add the SoapHandler via quarkus.cxf.client.clientName.handlers=MyHandler property - at first we did it via manually adding a handlerchain on the service but that did not make a difference.

For emphasis: with non-concurrent requests it all works fine, but with slightly higher rates there seems to be 'cross talk'.

Extra info
What we are doing, as you can see in the error logs, is mostly related to the adressing feature https://cxf.apache.org/docs/featureslist.html. Perhaps that can be done more efficiently using the Features capabilities mentioned here, but we are working of off an existing implementation that did it manually and for me the documentation on how to use the adressing feature is not entirely clear https://docs.quarkiverse.io/quarkus-cxf/dev/user-guide/interceptors-features-handlers/cxf-interceptors-and-features.html

[jcjveraa]

After reading https://docs.quarkiverse.io/quarkus-cxf/dev/user-guide/advanced-client-topics/dynamic-client-configuration.html I changed the scope of the service in which the CXFClient is injected to @RequestScoped, but that didn't matter unfortunately.

Note that there is nothing dynamic about our configuration, but as this looks to me like some concurrency issue I thought it worth a shot.

[ppalaga]

Thanks for the report, @jcjveraa! Yeah, any race conditions need to be taken seriously. Would you mind putting together a minimal reproducer?

[jcjveraa]

Thanks for the quick response again Peter.
I'll need to make some time to try and make a reproducer. The flow in short is: Resource class (web endpoint) -> Service class -> Data Access class (which makes the call with a CXFClient - we are essentially querying a remote database using a SOAP call).

In the meantime I've been able to confirm its a race condition in the client somewhere, as wrapping the code with lock solves the issue completely.

Again in pseudocode, this makes the issue 'not repeatable anymore'.

@ApplicationScoped
@Startup
class myDao {
   // ...
   @CXFClient("myClient")
   WebService service;

   private final ReentrantLock lock = new ReentrantLock();
   // ...
   public ResultObject getResult() {
     lock.lock();
     ResultObject result = this.service.getResult();
     lock.unlock();
     return result;
   }
}

I forgot to mention earlier, but our SOAPHandler is effectively stateless/purely functional - it only contains some final strings set when the handler is constructed that are used to implement WS Addressing, nothing “dynamic”.