-
-
Notifications
You must be signed in to change notification settings - Fork 3.7k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Dynamic Systems and Components #623
Dynamic Systems and Components #623
Conversation
crates/bevy_scene/src/scene.rs
Outdated
if let Some(component_registration) = | ||
component_registry.get(&match type_info.id() { | ||
bevy_ecs::ComponentId::RustTypeId(id) => id, | ||
_ => todo!("Handle external ids in component registry"), |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This may need to be fixed before merging. We have to find out how we want to manage the scene registry when it comes to external types. I don't have a full understanding of how it works currently so I didn't implement anything yet
I think I'm going to prioritize figuring out the dynamic component and systems approach before I try to fix this.
Just an FYI, @cart probably don't waste time reviewing this yet ( even if you get time ) as I found some stuff that might need to change a little as I keep going. I'll mark the PR as not a draft when it's ready for review. |
d6c3844
to
7e44b37
Compare
f4ca66f
to
d9e07fb
Compare
OK, this is still probably a little rough around the edges, but I think the concept is laid out enough to get some review by anybody who is able to look at this. If any community members want to comment on this it'd be good to get as much feedback as possible before Carter has a chance to look at him to save him some time if there are glaring issues. 😄 |
1f901da
to
3dccbc9
Compare
Hey, CI is passing now. :) I just ran my new benchmarks on this PR and the result is ( PR in blue, master in red ): The frame time and the CPU cycles differences are within the noise noise range on my machine ( my laptop with a web browser running on it ), but the CPU instructions are always stable pretty much within 0.01% when there are no changes so this PR clearly represents an increase of at least 1.85% increase in CPU instructions for the benchmark games. I was hoping to avoid slowing down non-scripted games, such as the current benchmark games, at all, but it looks like the new design has a slight overhead. Not sure, how much that means though. |
a362d2e
to
c4b591e
Compare
I've now successfully demonstrated both dynamic systems and dynamic_components summing up, I think, all that is necessary to allow 3rd party implementations of scripting systems for Bevy! 🎉 The examples demonstrating both dynamic systems and dynamic components are very limited in scope and do not prove that there aren't bugs when doing queries over multiple archetypes or all kinds of other variables, but for the most part I've built on the existing Bevy logic wherever possible and there is as little new logic as possible, which should help for review and stability. I'm going to start working on a language agnostic scripting system on top of these new features. This will stay separate from Bevy, but it will help stress test these new features once I get it going. |
Awesome work! I'm curious to hear what @Ralith would think about upstreaming these changes. |
I'm open to the idea, though I'd need to do a proper review. The usecase is well-justified. A few initial thoughts:
Could this be folded into
Maybe a custom |
Oh, that's perfect. I had missed the
That's a good idea, I'll try that out. I'll will add that I'm pretty sure that with these changes the ECS bench iteration speed is not decreased when using static systems. As far as the iteration speed of dynamic systems ( or, probably more accurately, stateful queries ), the last I tried the ECS bench with a dynamic system the iteration speed when from ~1µs to ~100µ compared to the non-dynamic version. I'm not sure how accurate that benchmark was, though. At this point I figured that we can be reasonably sure that we haven't lost a, so far, noticeable amount of performance for non-scripted systems. If we notice scripted systems, or stateful queries or something are slow, we can try to optimize it later after it is actually working, being that the non-dynamic features already there don't appear slower because of the new features. |
Using a custom hash implementation and benchmarking just the hash with criterion yields a small performance increase:
Does this look right for a hash implementation? I'm not sure if there is a better way to do this that avoids the #[derive(Debug, Clone)]
pub enum ComponentId {
RustTypeId(std::any::TypeId),
ExternalId(u64),
}
impl Hash for ComponentId {
fn hash<H: Hasher>(&self, state: &mut H) {
match self {
ComponentId::RustTypeId(id) => {
id.hash(state);
state.write_u8(0);
}
ComponentId::ExternalId(id) => {
state.write_u64(*id);
state.write_u8(1);
}
}
}
} Edit: Opened a forum topic just to get more feedback. Also it doesn't effect the CPU instructions measurement noticeably for the asteroids and breakout benchmarks. |
OK, I got help on the forum that said that the hashes are allowed to collide, so we can totally get rid of any discriminant between impl Hash for ComponentId {
fn hash<H: Hasher>(&self, state: &mut H) {
match self {
ComponentId::RustTypeId(id) => {
id.hash(state);
}
ComponentId::ExternalId(id) => {
state.write_u64(*id);
}
}
}
}
Hey that also brings down the CPU instructions a little bit in the game benchmarks. Still a little higher, but not as much as before. |
ecd08b7
to
a383a18
Compare
311b639
to
07a44c0
Compare
07a44c0
to
9111e06
Compare
OK, I did the cleanup I needed to do and went over the code one more time and I think it's ready for review, now. |
68e588f
to
40a70b4
Compare
Supporting this behavior (in particular, dynamically adding systems) has some painful downstream limitations on future optimizations: see #1205 for an example, but I would expect to see more examples in areas that touch on scheduler logic. This is a super cool batch of changes though, and I'd love to find a compromise position! Before we merge this, can we guarantee that systems are only being added at a specific time (e.g. between frames), and maybe emit some sort of event so then any of the more advanced optimizations will know when to rebuild? |
Actually, this PR doesn't handle letting you add systems after starting the app, so it won't effect the scheduling performance in that respect. For instance, in the dynamic systems example, we add the dynamic system when building the app, just like we typically do in a Bevy project: bevy/examples/ecs/dynamic_systems.rs Line 151 in 40a70b4
The primary concern of this PR was to allow dynamically adding systems without having to recompile not necessarily without having to restart. Now I could see wanting to add systems in-between frames or something like that being useful, but that would be separate from these core changes. I'm planning on coming back to these changes and, apparently, doing a massive rebase to resolve the merge conflicts, probably sometime within the next couple months. |
@cart I saw you mention somewhere around here that you were working on an ECS internal rewrite that would make integrations like this scripting stuff easier. I'm maybe going to find time to finish this back up, but not immediately, and I thought maybe it would be good to wait for whatever reorganization you were doing so that I don't get in the way or have to deal with more merge conflicts later. If that's right, and I didn't misinterpet something somewhere ( I can't find that comment I thought I saw ), could you let me know what that rewrite is done so that I can take another look at this once I get time? |
Sure thing! I should be ready for a draft pr in the next couple of days. The relevant changes:
|
Awesome! That sounds like it tackles all of my pain points in implementing this! |
I'm in the process of create a plugin system that uses bevy ecs, this is my current thoughts.
*To support mutable components would require the client to pass back mutated components and having the host to mutate the component with the new data. *Note I'm planing on using bincode to serialize all components, therefor I belive the TypeInfo is |
@Defman That sounds reasonable. The cool part about what I was trying to do with this PR is that it simply handles adding the ability to add dynamic systems and components, and it doesn't care how you use that ability. The thing that sounds possibly inefficient ( to my untrained ears ) about your plan is that it sounds like you'll essentially be serializing the entire piece of the world that the plugin wants to access for every query, and I'm not sure how well that would scale. It also just depends on how performant it needs to be for your game. Granted, I'm not sure there are any better easy options. You might be able to try to take slices of bytes for the components in your query and casting those bytes to a |
@zicklag I too have some concerns with performance, but for now it's more of a proof of concept. What I really want is to know if I should implement my own Query struct, since the native one requires access to the world. Note I have thought about using repr(C), but since I'm using wasm for the client I would have to copy the bytes and fix pointers. Not to mention the layout may differ between multiple clients and host, due to 32 vs 64bit. |
I'm not sure yet. It sounds like a lot will change with the internal rewrite, so you might want to wait to see what that looks like once it's done. I'm waiting for that before I pick this PR back up again. |
Yup! Nothing else major in the pipeline that would overlap with this work. |
Thanks for the pointer @alice-i-cecile! I might be able to get to re-visiting this stuff in a week or two. I haven't checked out Bevy's new ECS updates yet so I'm going to spend some time understanding that before I do much. Thanks for the link to the note. 👍 I'm fairly certain the exact contents of this PR are pretty much trash at this point, but I'm totally fine with that as most of my work might already be done/getting done by @BoxyUwU! Quick summary of what I know now:
I'm totally happy letting @BoxyUwU take care of the dynamic components and then adding the Dynamic systems myself, which BTW really doesn't have to come with Bevy, so maybe that doesn't even mean a Bevy PR, I can just add it to my scripting system crate. I've got enough to work on from the FFI and language abstraction layer for the scripting system so if you guys handle the Bevy side of it that would be great! As far as the notes on hackmd.io I really like the idea of dogfooding the static API on top of the dynamic API. 🎉 |
In fact, I'm going to close this as any new effort should really be done in new PR's anyway and things have changed quite a bit. Any more discussion on this could move to #142. |
Related to: #32
Resolves: #142
This PR will attempt to establish the ability to create systems and components that have been determined at runtime instead of compile time. I'm going to try to implement this in one PR because I won't be sure about the sufficiency of the design until I actually get a working example of a dynamically registered system and component ( which I will include with this PR ).
I'm going to try to attack this problem one step at a time:
Steps
Non-Rust Component Ids
Status: Completed in commit: "Implement Custom Component Ids"
Currently bevy_hecs uses
std::any::TypeId
to uniquely identify the component IDs, but this does not allow us to define components that have a non-Rust origin. The first commit in the PR has migrated all of the internals of the bevy ECS to use a newComponentId
instead that is defined as:Establish Dynamic Queries
Status: Completed in commit: "Implement Dynamic Systems"
This adds a new state type parameter to the
Fetch
trait. This allows compile-time constructed queries like to be constructed withState = ()
and runtime constructed queries with the state parameter set to aDynamicComponentQuery
.Establish Dynamic Component Insertion
Status: Completed in commit: "Add Dynamic Component Support"
This adds a new implementation of the
Bundle
trait,RuntimeBundle
which can be used to insert untyped components at runtime by providing a buffer of bytes and the required type information.Create Examples
Status: Completed in respective commits
We also have new examples for dynamic_systems and dynamic_components.
Remaining Work
The biggest thing necessary at this point is a solid review. There's a lot of code changed, but thankfully not a ton of new logic. The bulk of the changes are repetitive changes required to add the necessary type parameters and such for the new
Fetch
`State type parameter.Otherwise, there are a couple of
todo!()
's inbevy_scene
because I don't know enough about the Bevy scene architecture to integrate scenes with external components yet. I think that could reasonably be left to a separate PR, but I'm fine either way.