Skip to content

v0.28.0
Compare
Choose a tag to compare
@pseudo-rnd-thoughts pseudo-rnd-thoughts released this 24 Mar 17:26
· 409 commits to main since this release
v0.28.0
8c167b8
This commit was created on GitHub.com and signed with GitHub’s verified signature. The key has expired.
GPG key ID: 4AEE18F83AFDEB23
Expired
Learn about vigilant mode.

v0.28.0 Release notes

This release introduces improved support for the reproducibility of Gymnasium environments, particularly for offline reinforcement learning. gym.make can now create the entire environment stack, including wrappers, such that training libraries or offline datasets can specify all of the arguments and wrappers used for an environment. For a majority of standard usage (gym.make(”EnvironmentName-v0”)), this will be backwards compatible except for certain fairly uncommon cases (i.e. env.spec and env.unwrapped.spec return different specs) this is a breaking change. See the reproducibility details section for more info.
In v0.27, we added the experimental folder to allow us to develop several new features (wrappers and hardware accelerated environments). We’ve introduced a new experimental VectorEnv class. This class does not inherit from the standard Env class, and will allow for dramatically more efficient parallelization features. We plan to improve the implementation and add vector based wrappers in several minor releases over the next few months.
Additionally, we have optimized module loading so that PyTorch or Jax are only loaded when users import wrappers that require them, not on import gymnasium.

Reproducibility details

In previous versions, Gymnasium supported gym.make(spec) where the spec is an EnvSpec from gym.spec(str) or env.spec and worked identically to the string based gym.make(“”). In both cases, there was no way to specify additional wrappers that should be applied to an environment. With this release, we added additional_wrappers to EnvSpec for specifying wrappers applied to the base environment (TimeLimit, PassiveEnvChecker, Autoreset and ApiCompatibility are not included as they are specify in other fields).
This additional field will allow users to accurately save or reproduce an environment used in training for a policy or to generate an offline RL dataset. We provide a json converter function (EnvSpec.to_json) for saving the EnvSpec to a “safe” file type however there are several cases (NumPy data, functions) which cannot be saved to json. In these cases, we recommend pickle but be warned that this can allow remote users to include malicious data in the spec.

import gymnasium as gym

env = gym.make("CartPole-v0")
env = gym.wrappers.TimeAwareObservation(env)
print(env)  
# <TimeAwareObservation<TimeLimit<OrderEnforcing<PassiveEnvChecker<CartPoleEnv<CartPole-v0>>>>>>
env_spec = env.spec
env_spec.pprint()
# id=CartPole-v0
# reward_threshold=195.0
# max_episode_steps=200
# additional_wrappers=[
# 	name=TimeAwareObservation, kwargs={}
# ]

import json
import pickle

json_env_spec = json.loads(env_spec.to_json())
pickled_env_spec = pickle.loads(pickle.dumps(env_spec))
recreated_env = gym.make(json_env_spec)
print(recreated_env)  
# <TimeAwareObservation<TimeLimit<OrderEnforcing<PassiveEnvChecker<CartPoleEnv<CartPole-v0>>>>>>
# Be aware that the `TimeAwareObservation` was included by `make`

To support this type of recreation, wrappers must inherit from gym.utils.RecordConstructorUtils to allow gym.make to know what arguments to create the wrapper with. Gymnasium has implemented this for all built-in wrappers but for external projects, should be added to each wrapper. To do this, call gym.utils.RecordConstructorUtils.__init__(self, …) in the first line of the wrapper’s constructor with identical l keyword arguments as passed to the wrapper’s constructor, except for env. As an example see the Atari Preprocessing wrapper
For a more detailed discussion, see the original PRs - #292 and #355

Other Major Changes

  • In Gymnasium v0.26, the GymV22Compatibility environment was added to support Gym-based environments in Gymnasium. However, the name was incorrect as the env supported Gym’s v0.21 API, not v0.22, therefore, we have updated it to GymV21Compatibility to accurately reflect the API supported. #282
  • The Sequence space allows for a dynamic number of elements in an observation or action space sample. To make this more efficient, we added a stack argument which can support which can support a more efficient representation of an element than a tuple, which was what was previously supported. #284
  • Box.sample previously would clip incorrectly for up-bounded spaces such that 0 could never be sampled if the dtype was discrete or boolean. This is fixed such that 0 can be sampled in these cases. #249
  • If jax or pytorch was installed then on import gymnasium both of these modules would also be loaded causing significant slow downs in load time. This is now fixed such that jax and torch are only loaded when particular wrappers is loaded by the user. #323
  • In v0.26, we added parameters for Wrapper to allow different observation and action types to be specified for the wrapper and its sub-environment. However, this raised type issues with pyright and mypy, this is now fixed through Wrapper having four generic arguments, [ObsType, ActType, WrappedEnvObsType, WrappedEnvActType]. #337
  • In v0.25 and 0.v26 several new space types were introduced, Text, Graph and Sequence however the vector utility functions were not updated to support these spaces. Support for these spaces has been added to the experimental vector space utility functions: batch_space, concatenate, iterate and create_empty_array. #223
  • Due to a lack of testing the experimental stateful observation wrappers (FrameStackObservation, DelayObservation and TimeAwareObservation) did not work as expected. These wrappers are now fixed and testing has been added. #224

Minor changes

  • Allow the statistics of NormalizeX wrappers to be disabled and enabled for use during evaluation by @raphajaner in #268
  • Fix AttributeError in lunar_lander.py by @DrRyanHuang in #278
  • Add testing for docstrings (doctest) such that docstrings match implementations by @valentin-cnt in #281
  • Type hint fixes and added __all__ dunder by @howardh in #321
  • Fix type hints errors in gymnasium/spaces by @valentin-cnt in #327
  • Update the experimental vector shared memory util functions by @pseudo-rnd-thoughts in #339
  • Change Gymnasium Notices to Farama Notifications by @jjshoots in #332
  • Added Jax-based Blackjack environment by @balisujohn in #338

Documentation changes

  • Fix references of the MultiBinary and MultiDiscrete classes in documentation by @Matyasch in #279
  • Add Comet integration by @nerdyespresso in #304
  • Update atari documentation by @pseudo-rnd-thoughts in #330
  • Document Box integer bounds by @mihaic in #331
  • Add docstring parser to remove duplicate in Gymnasium website by @valentin-cnt in #329
  • Fix a grammatical mistake in basic usage page by @keyb0ardninja in #333
  • Update docs/README.md to link to a new CONTRIBUTING.md for docs by @mgoulao in #340
  • MuJoCo/Ant clarify the lack of use_contact_forces on v3 (and older) by @Kallinteris-Andreas in #342

What's Changed

Thank you to our new contributors in this release: @Matyasch, @DrRyanHuang, @nerdyespresso, @khoda81, @howardh, @mihaic, and @keyb0ardninja.

Full Changelog: v0.27.1...v0.28.0

Contributors

mihaic, howardh, and 11 other contributors
Assets 2
Loading