[BUG] BWC failure on backward compatibility test

[andy-k-improving]

What is the bug?

This is served as a follow up ticket for #3168 to gather all the progress and information related to the backward compatibility test failure on CI pipeline.

How can one reproduce the bug?
Steps to reproduce the behavior:

1. To trigger the Run backward compatibility tests

What is the expected behavior?
To have backward compatbility test passed on the main branch for Github CI-Runner.

What is your host/environment?
N/A

Do you have any screenshots?
https://github.com/opensearch-project/sql/actions/runs/12144751620/job/33864844886

Do you have any additional context?
As the test is executed against openserach-min distribution, which only Linux distribution is available (Not Mac and WIndows)
Contributor is encouraged to equipped with a Ubuntu machine or a equivalent virtual environment.