Address sample_sequence_length greater than min_length_time_axis

[mickvangelderen]

Based on #43, please review that first and if it wasn't merged yet, please review this PR per commit.

The goal of this PR is to make the tests not generate flashbax warnings and to prevent code from being added that would trigger a flashbax warning.

I am not sure that the suggested changes in the last commit are right.

I am unsure why min_length_time_axis=min_length // add_batch_size + 1 is used in the trajectory buffer source code, particularly the + 1. I am also unsure why the min_length fixture is defined as:

@pytest.fixture
def min_length(sample_batch_size: int) -> int:
    return int(sample_batch_size + 1)

I could understand if it was defined in terms of add_batch_size, but it is not.

[SimonDuToit]

I think the + 1 in min_length_time_axis=min_length // add_batch_size + 1 is just to be conservative. This matches how max_length_time_axis is calculated.
The min_length fixture is defined to be viable, since it has to be at least as big as sample_size.

[SimonDuToit]

I'm not sure why its necessary to replace min_length with add_batch_size. Otherwise it makes sense, please just see my comment on the merge conflict.

[SimonDuToit]

What is the reasoning for this change?

[mickvangelderen]

max_length_time_axis = 3, min_length_time_axis = 1, add_batch_size = 75, sample_sequence_length = 2, period = 1, max_size = None

    def validate_trajectory_buffer_args(
        max_length_time_axis: Optional[int],
        min_length_time_axis: int,
        add_batch_size: int,
        sample_sequence_length: int,
        period: int,
        max_size: Optional[int],
    ) -> None:
        """Validate the arguments of the trajectory buffer."""
    
        validate_size(max_length_time_axis, max_size, add_batch_size)
    
        if max_size is not None:
            max_length_time_axis = max_size // add_batch_size
    
        if sample_sequence_length > min_length_time_axis:
>           warnings.warn(
                "`sample_sequence_length` greater than `min_length_time_axis`, therefore "
                "overriding `min_length_time_axis`"
                "to be set to `sample_sequence_length`, as we need at least `sample_sequence_length` "
                "timesteps added to the buffer before we can sample.",
                stacklevel=1,
            )
E           UserWarning: `sample_sequence_length` greater than `min_length_time_axis`, therefore overriding `min_length_time_axis`to be set to `sample_sequence_length`, as we need at least `sample_sequence_length` timesteps added to the buffer before we can sample.

The warning checks that sample_sequence_length <= min_length_time_axis. The flat buffer creates a trajectory buffer with sample_sequence_length = 2 and min_length_time_axis = min_length // add_batch_size + 1 = 1 // (1 + 10) + 1 = 0 + 1 = 1. When calling create trajectory buffer you need to ensure that min_length >= add_batch_size hence the change. Perhaps it is not the right solution.

[SimonDuToit]

This is a sensible change.