[RLlib] relax gymnasium requirement

[duburcqa]

Description

Would it be possible to relax gymnasium requirement to gymnasium >= 0.26.3, <=0.28.1 ? This is necessary to be compatible with stable-baseline3. See DLR-RM/stable-baselines3#1452.

Use case

No response

[ArturNiederfahrenhorst]

Hi @duburcqa ,

That's not possible "just like that".
I have an open draft PR that attempts to upgrade to 0.27.x I believe.
We'll definitely upgrade at some point.
Could you specify how urgent this is and what the use case is?

[duburcqa]

That's not possible "just like that".
I have an open draft PR that attempts to upgrade to 0.27.x I believe.

Ok I see, it is a bad news but not surprising...

Could you specify how urgent this is and what the use case is?

Nothing especially urgent. It just does not exist any version of ray[rllib] and stable-baselines3 that are compatible with each other, at least for python 3.11. This incompatibility makes it difficult for package maintainers of gym environments to write unit tests and provide examples for the most common learning frameworks.

[ArturNiederfahrenhorst]

@Rohan138 Can you take a look? I have an open PR for reference
#32999

[duburcqa]

Any update on this point ?

[Rohan138]

@duburcqa I see that stable-baselines3 is now pinned to the latest gymnasium==0.28.1 release; we will try to update to the same in #32999, however, this will take time. In the meanwhile, I would recommend installing ray and stable-baselines3 in separate environments, since stable-baselines3 does not work with previous versions of gymnasium.

[duburcqa]

Thank you for the update. I could create separate environments but it does not really solve the issue because I'm maintaining a library offering a suite of learning environment and having to be compatible with multiple versions of gym that seems to be partial incompatible with each other is less than ideal.

[Rohan138]

I'll get this PR merged soon, should update to gymnasium=0.28.1. #35698

Thank you for raising this issue!

[duburcqa]

Any update ?

[Rohan138]

@duburcqa See the PR in #35698; we're currently trying to figure out a mujoco versioning issue that's breaking our tests when we try to upgrade.