Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Allow systems to execute with a fixed timestep #125

Closed
reidbhuntley opened this issue Aug 11, 2020 · 16 comments
Closed

Allow systems to execute with a fixed timestep #125

reidbhuntley opened this issue Aug 11, 2020 · 16 comments
Labels
A-ECS Entities, components, systems, and events C-Feature A new feature, making something new possible

Comments

@reidbhuntley
Copy link
Contributor

In order to have objects move at the same speed regardless of FPS while still keeping a game's logic deterministic, we need some way to let users mark systems to be run with a fixed timestep. (See the classic Gaffer on Games article here)

Currently a user could achieve this by creating an accumulator resource inside a system and running the system's logic multiple times per call based on that accumulator. However, this stops working correctly when you try it with multiple systems at once. If systems A and B should both run with a fixed timestep, and B is dependent on A, then they should always run one after another (ABABAB); but using this method would first make A run a few times, then B run a few times (AAA...BBB...), so their order would depend on the amount of time in the accumulator, and thus be nondeterministic.

To solve this, I think we should create a second scheduler and push systems to there that a user wants to run with a fixed timestep. On each frame that entire schedule would be executed some number of times based on the amount of time in an accumulator, and it would execute before the main scheduler. This approach of running fixed-timestep logic before other logic is how other engines like Godot and Unity have handled this issue.

Additionally, we'd need some way in the API to configure what the timestep actually is, as well as a way for systems to read that value, ideally through the Time resource. A field containing a float between 0 and 1 indicating how far through the current timestep we are would also be helpful, since this would allow rendered objects to update faster than the fixed timestep through the use of interpolation.

@cart
Copy link
Member

cart commented Aug 11, 2020

Agreed. This is a "must have" feature so we should take the time to get it right.

I think we could approach this from a few angles:

  1. Add support for multiple schedules in Apps and give each schedule its own run() function (which could then perform the time step work).
  2. Have one global schedule and allow for "sub schedules" via resources and stages:
  • Ex: create a new "physics" stage in the global schedule, add a PhysicsSchedule resource, then add this system to the "physics" stage:
fn physics_step(world: &mut World, resources: &mut Resources) {
  let schedule = resources.get::<PhysicsSchedule>();
  schedule.run(world, resources);
}
  1. Use an "accumulator" resource like you mentioned and let systems determine when something is executed

(2) and (3) are both limited by the global schedule update, but i do like how (2) builds on top of the existing feature set without adding new things. (1) allows for fully decoupling schedules from each other, which does seem valuable.

@karroffel karroffel added A-ECS Entities, components, systems, and events C-Feature A new feature, making something new possible labels Aug 12, 2020
@reidbhuntley
Copy link
Contributor Author

I think (3) is unsuitable for the reasons I gave above. (2) is definitely worth looking into, especially if there are more use cases for the "sub schedule" concept. If not, (1) may be the best way to go due to how simple it could allow the API to be.

@tristanpemble
Copy link
Contributor

tristanpemble commented Aug 12, 2020

I believe Unity DOTS is taking a somewhat similar approach to 2 by allowing system groups to loop: ComponentSystemGroup::UpdateCallback

Relevant forum post describing the design in a little bit more detail

@lwansbrough
Copy link
Contributor

I know amethyst_network uses a SimulationTime resource which covers the use case for network physics, etc.

@tinco
Copy link

tinco commented Aug 28, 2020

I just spent a couple days exploring option 1. It's more complicated than I thought it would be. The API definitely is rather simple, app's have multiple schedules, each schedule has a runner. But the internals are more complex. I went for each schedule has its own executor, which means the schedules run fully independent of eachother, but it also means the parallel stages contend for access to the World and Resources, I did this through a RwLock but as I'm finishing this there seem to be a serious drawback.

A RwLock let's all the systems run in parallel, but any time a stage finishes, it will try to get write access on the World and Resources (to run the commands) which will immediately block all systems until the current systems are done running, until the finished stage has done its deeds. I'm not exactly sure how this would go in practice, maybe it all goes fine because most time is spent in the systems, and not as much time is spent in finishing up stages. It feels a bit awkward because in a game engines avoiding randomness in timing is very important, and this introduces some randomness. Maybe this would be solved by a more advanced Resources system where there might be finer granularity locking on the resources.

If we go option 2, it will mean we'll dynamically change the schedule so that the stages are interleaved at their respectively correct rates. This is very attractive if we're thinking of sequential stage execution, but parallel stages seem much more interesting to me, so it's actually possible get some work done in these stages. If the stages run in parallel then we actually get the exact same situation as in option 1.

Just to paint a common game scenario:

We have a animation+render loop running at 60fps. We have a gamelogic loop running at 30fps. While the animation+render loop is interpolating animations and sending stuff to the graphics card, the game logic loop is running a physics step, and then running the game logic systems.

I agree that option 3 does not seem to be able to adequately express this. I must say I'm not fully clear on how you would express this with option 2.

After stepping back and writing this comment, considering option 2 has the same drawbacks, I think option 1 does seem to be most attractive so I'm gonna finish this up and hopefully I'll have something to show that could be a basis for this feature.

@thebluefish
Copy link
Contributor

thebluefish commented Sep 25, 2020

I have been contemplating a few approaches to this problem. Here is something I have worked out for a separate Physics tick rate via a singleton PhysicsSchedule, system runner, and PhysicsRate component.

use bevy::{prelude::*, ecs::Schedule};
pub struct Physics;

pub struct PhysicsRate(f64);
pub struct PhysicsSchedule(f64, Schedule); // accumulator, schedule

static mut PHYSICS_SCHEDULE: Option<PhysicsSchedule> = None;

pub fn get_schedule<'a>() -> &'a mut PhysicsSchedule {
    unsafe {
        match &mut PHYSICS_SCHEDULE {
            Some(schedule) => schedule,
            None => {
                PHYSICS_SCHEDULE = Some(PhysicsSchedule(0.0, Schedule::default()));
                get_schedule()
            }
        }
    }
}

fn physics_runner(world: &mut World, resources: &mut Resources) {
    unsafe {
        match &mut PHYSICS_SCHEDULE {
            Some(PhysicsSchedule(acc, schedule)) => {
                let delta = resources.get::<Time>().unwrap().delta_seconds_f64;
                let rate = resources.get::<PhysicsRate>().unwrap().0;
                *acc += delta;
                while *acc >= rate {
                    *acc -= rate;
                    schedule.run(world, resources);
                }
            },
            _ => {
                panic!("schedule must exist before running it!")
            }
        };
    }
}

impl Plugin for Physics {
    fn build(&self, app: &mut AppBuilder) {
        unsafe {
            let schedule = get_schedule();
            schedule.1.add_stage("update");
            schedule.1.add_system_to_stage("update", phys_system.system());
        }

        // rate of 30 ticks per second
        app.add_resource(PhysicsRate(1.0 / 30.0))
            .add_stage_after("event_update", "physics_runner")
            .add_system_to_stage("physics_runner", physics_runner.thread_local_system());
    }
}

fn phys_system(rate: Res<PhysicsRate>) {
    println!("physics! delta: {}", rate.0);
}

My concern with this approach is how I would succinctly integrate 3rd party plugins. It seems that I would probably need to duplicate the plugin's build function to run its systems on my custom schedule.

@Ratysz
Copy link
Contributor

Ratysz commented Sep 25, 2020

@thebluefish, why is PhysicsSchedule a global variable instead of a resource?

@thebluefish
Copy link
Contributor

thebluefish commented Sep 25, 2020 via email

@Ratysz
Copy link
Contributor

Ratysz commented Sep 25, 2020

been fighting the borrow checker there

Yeah, it gets a bit gnarly, but it works if the resource is Arc<Mutex<PhysicsSchedule>>, and you get it like this in the system

    let physics = resources
        .get::<Arc<Mutex<PhysicsSchedule>>>()  // Option<Ref<Arc<Mutex<PhysicsSchedule>>>>
        .unwrap()  // Ref<Arc<Mutex<PhysicsSchedule>>>
        .deref()  // Arc<Mutex<PhysicsSchedule>>
        .clone();  // Clones the Arc, which lets the compiler infer that `resources` isn't borrowed anymore.
    let mut physics = physics.lock().unwrap();  // MutexGuard<PhysicsSchedule>

For what it's worth, the approach I've been using in similar situations is to implement System on the type containing the schedule. That way the schedule is not stored in resources, but indeed in the parent schedule.

@thebluefish
Copy link
Contributor

For what it's worth, the approach I've been using in similar situations is to implement System on the type containing the schedule. That way the schedule is not stored in resources, but indeed in the parent schedule.

Good call, this worked out great for my case.

@smokku
Copy link
Member

smokku commented Oct 28, 2020

@thebluefish, I see you've encapsulated your approach in https://github.com/thebluefish/bevy_contrib_schedules
Thanks.

@klaatu01
Copy link

@tinco where did you get to with this? Happy to try and pickup where you left off.

@tinco
Copy link

tinco commented Nov 12, 2020

hey @klaatu01 I don't recommend it. I abandoned it because it just seems like a bad idea to me now. Especially since @cart recently had this proud post that bevy is now lock free.

What I did adds RwLocks all over the place. Anyway here's the code:

master...tinco:multiple-schedulers

If I'd try now I would really carefully read how the lock free system works and find out how this would fit in there. I bet the new architecture opens up new possibilities.

@tinco
Copy link

tinco commented Nov 12, 2020

This is the API I made work on my branch:

use bevy::{app::ScheduleRunnerPlugin, prelude::*};
use std::time::Duration;

// This example shows multiple schedules running at the same time. So you can have
// the default schedule at your target screen framerate, and for example your
// physics game loop at a fixed 30 fps.
fn main() {
    // this app has the default schedule running at 1 fps and a second schedule
    // running at 2 fps.
    App::build()
        .add_plugin(ScheduleRunnerPlugin::run_loop(Duration::from_secs_f64(
            1.0,
        )))
        .add_schedule("faster", ScheduleRunnerPlugin::run_loop(Duration::from_secs_f64(
            2.0,
        )))
        .add_system(hello_world_system.system())
        .add_system_to_schedule("faster", zippy_system.system())
        .run();
}

fn hello_world_system() {
    println!("hello world");
}

fn zippy_system() {
    println!("zip");
}

@klaatu01
Copy link

@tinco thanks so much for sharing. I've been looking into this quite a bit, so even just seeing where you got to is very useful!

If I'd try now I would really carefully read how the lock free system works and find out how this would fit in there. I bet the new architecture opens up new possibilities.

I must have missed this, I will need to take a look. Thanks!

@reidbhuntley
Copy link
Contributor Author

This is now possible thanks to #1021 !

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
A-ECS Entities, components, systems, and events C-Feature A new feature, making something new possible
Projects
None yet
Development

No branches or pull requests

10 participants