sandermertens / flecs Goto Github PK
View Code? Open in Web Editor NEWA fast entity component system (ECS) for C & C++
Home Page: https://www.flecs.dev
License: MIT License
A fast entity component system (ECS) for C & C++
Home Page: https://www.flecs.dev
License: MIT License
Default type construction by signature:
int ecs_type_parse(ecs_world_t *world, ecs_type_t* type, const char *sig);
User defined type construction by something:
int ecs_type_parse_{smth}(ecs_world_t *world, ecs_type_t* type, {smth});
Function modifies given type
pointer and returns non-zero error code.
To improve productivity and to allow flecs to be used together with external modeling tools, modules, systems, components and entities should be described in a format external to the source code. This will enable generation of boiler plate source code, validation of the source code against a model, and generating runnable applications that import modules and instantiate data.
The generator should be developed as a bake driver so code generation is integrated with the build process, and the flecs definitions can be created in the project.json file. The definition should contain:
The definition should contain all relevant data, like which components a system is interested in and which components an entity has. In addition, it would be a nice to have if entities can specify component values (which would require reflection).
@SanderMertens In the context of dynamically loading a module, when unloading the module from memory, would like to unregister the associated systems that were previously registered by the respective module. One example use case that this would facilitate, is dynamically 'upgrading' a module ( unloading the old one, and importing the new one ). Understood there is probably some hefty business logic in supporting this, but perhaps a System Unregister like API or leveraging ecs_delete() to take the appropriate actions if the entity is a system? Thoughts?
Hi
From what I read, reactive behaviors (OnAdded, On*...) are executed during ecs_* methods.
But can there be behavior for executing those events at a specified order?
Here is how my systems structured while I work with Entitas:
Input feature
LeftMouseDown
etc)...
Shooting feature
LeftMouseDown
added, creates new entity-bullet)...
...
Collision feature
Collided
on bullet)Collided
being added, adds Destroyed
component)...
...
...
Visual feature
Asset
component, spawns visual representation, links them somehow)Destroyed
component OR removed Asset
component, removes visual representation)RemoveDestroyedEntitiesSystem (reacts on added Destroyed
, removes entity)
There can be a lot of different logic between marking an entity as Destroyed
and actually removing such entity. For example, some system could remove Destroyed
component before the entity would be removed.
Those systems are executed at that specified order.
Can there be a feature to delay the execution of reactive systems?
What would be the best way to have some systems update with deltaTime, eg ecs_progress
and other systems update at stable tick intervals eg. for physics the deltaTime would be fixed to some predetermined value and likely called more than once in a frame depending on certain factors.
I'm not sure if what I'm asking is possible but thinking about this from what I know about the library so far maybe ecs_progress
could be updated to allow me to specify which systems / features to progress. I'm not even sure ecs_progress
would be the best place to specify something like this, as I'm sure there is a certain amount of overhead with that call and calling multiple times in a frame is maybe too costly. But it certainly seems like the most convenient place.
The work around right now is that each system that cares about having a fixed timestep needs to track this themselves.
A common pattern for applications dealing with hierarchies is to access the data in a depth-first way. As asked by @teksoc-1 in #63 :
@SanderMertens On the subject of Flecs hierarchies, is there a public API to get all the children of a given parent? I poked around and wasn't seeing it, but seemed like a natural API to have, unless I'm missing something.
@SanderMertens Can't recall if you mentioned this before, or if it is related to an existing issue, but it appears an assertion failure "assert(world->magic == ECS_WORLD_MAGIC)" on reactive systems from a multi stage system.
Sample code to reproduce:
#define NUM_THREADS 4
#define TRIGGER_ASSERTION_FAILURE 1
typedef int DummyComponent;
void
ReactiveDummySystem( ecs_rows_t * rows )
{
ecs_os_log( "%s", __FUNCTION__ );
}
void
PeriodicDummySystem( ecs_rows_t * rows )
{
ecs_os_log( "%s", __FUNCTION__ );
ECS_COLUMN_COMPONENT( rows, DummyComponent, 1 );
for ( unsigned int i = 0; i < rows->count; i++ )
{
DummyComponent dummyComp = 0;
ecs_set_ptr( rows->world, rows->entities[ i ], DummyComponent, &dummyComp );
}
}
int main( int argc, char ** argv )
{
int rc = -1;
ecs_world_t * world = ecs_init();
// Platform specific threading, etc.
set_api();
ecs_set_target_fps( world, 1 );
ecs_set_threads( world, NUM_THREADS );
ECS_COMPONENT( world, DummyComponent );
ECS_SYSTEM( world, PeriodicDummySystem, EcsOnUpdate, DummyComponent );
#if TRIGGER_ASSERTION_FAILURE
ECS_SYSTEM( world, ReactiveDummySystem, EcsOnSet, DummyComponent );
#endif
ecs_new_w_count( world, DummyComponent, NUM_THREADS );
ecs_progress( world, 0 );
rc = ecs_fini( world );
return rc;
}
Following the getting started instructions, the following command fails:
bake new my_app -t flecs
with the following error:
error template 'flecs' not found.
This is happening on MacOS 0.14.1.
This will allow not mess with C++
type system, need to replace all necessary typedef
via struct
.
See. https://floooh.github.io/2018/06/17/handles-vs-pointers.html
Hi @SanderMertens,
I recently discovered flecs and i've been exploring your other work as well.
What are your thoughts on networking(ipc)?
Furthermore, bake and corto look really interesting as well. I'm curious if you've ever done any simulation testing at large and/or distributed scales?
In comparison, platforms like SpatialOS are enabling a lot more game devs & designers to work at a larger scale via data oriented approaches. But, I feel like there's another layer of resolution that is yet to be obtained and it collapses a LOT of problem spaces... Mainly in the realm of Game/Data Engineering, Design, Analytics, Machine Learning and Business Insights.
I'm still experimenting so i'll see how far this rabbit hole goes with zmq, corto, flecs and dear imgui :)
(extending windows right click with contextual: bake/flecs sub-commands was a worthwhile effort!)
From @teksoc-1's comment in #21 :
@SanderMertens Do you have any particular suggestions for implementing client/server patterns in Flecs i.e. Module X provides some service say for example HandleRequestSystem(Request) where the implementation may either add a Response component to entities, or generate new entities with Response components - service side implementation is really orthogonal to the 'problem'. This 'problem' that we see is ensuring the correct client (system) only receives the appropriate filtered Responses for the Requests that it made - doesn't care about any other Responses to Requests it didn't generate. Our naive approach to this problem is to leverage unique Tags that we would add to entities, and thus filtering for MyResponseSystem(Response, MyUniqueTag) ... is there a better way to do this, are we not thinking about this the ECS way, etc?
By @teksoc-1:
When doing some test cases of deleting entities and having OnRemove systems registered with components of the deleted entities, the entities seem to not get deleted, and will see them again next iteration. If I do not register OnRemove systems, the entities get deleted as expected, however, when the OnRemove Systems are applied ( and respectively called as expected ), the entities do not get deleted.
An operation is required that can bulk-add and bulk-remove components from entities that match a certain pattern. This operation will be more efficient than calling ecs_add
or ecs_remove
on individual entities.
@SanderMertens Have been meaning to mention this for awhile now, and just chalked it up to something in our environment set up, but when writing to the _ecs_os_api
pointer to the ecs_os_api
global data structure declared with the const
qualifier, it crashes inside ecs_set_api_defaults()
on macOS (clang compiler). The pointer assignment naturally bypasses the compiler checks, but at run time crashes on the fact that we are modifying a data structure with the const
qualifier. Not sure if you are able to reproduce, if there is some compiler/platform specific implementation with your set up that is not triggering this condition?
Describe the bug
When the _ecs_field
function is called with a zero-size (which bypasses type size checks) the returned pointer is not correctly offset by the row index. This is because the passed in size was used to calculate the row offset. Instead, the function should use the size that is specified in the table column.
To Reproduce
Call _ecs_field(rows, 0, 1, 0)
and _ecs_field(rows, 0, 1, 1)
and verify that the returned pointers are the same.
Expected behavior
The pointers should not be the same.
Currently we have read-write(mutable) access to component data via ECS_COLUMN
, to make all systems work together we split it into phases but if systems count increases we run out of phases, and get loose control of running order within phase, by knowing systems RO/RW component access we can build better scheduling(producer-consumer data-flow graph?).
As bonus we disallow users to change immutable data.
In system definition.
at: https://github.com/SanderMertens/ecs_graphics/blob/master/src/main.c#L31
we defined read-only EcsInput
component data access:
ECS_SYSTEM(world, MoveSquare, EcsOnUpdate, Immutable.EcsInput, Square.EcsPosition2D);
In system function.
at: https://github.com/SanderMertens/ecs_graphics/blob/master/src/main.c#L5
we implement read-only EcsInput
component data access:
void MoveSquare(ecs_rows_t *rows) {
const EcsInput *input = ecs_column_ro(rows, EcsInput, 1);
//or, ECS_COLUMN_IMMUTABLE(rows, EcsInput, input, 1);
EcsPosition2D *position = ecs_shared_rw(rows, EcsPosition2D, 2);
//or, ECS_SHARED(rows, EcsPosition2D, position, 2);
int x_v = input->keys[ECS_KEY_D].state || input->keys[ECS_KEY_RIGHT].state;
x_v -= input->keys[ECS_KEY_A].state || input->keys[ECS_KEY_LEFT].state;
int y_v = input->keys[ECS_KEY_S].state || input->keys[ECS_KEY_DOWN].state;
y_v -= input->keys[ECS_KEY_W].state || input->keys[ECS_KEY_UP].state;
position->x += x_v * MOVE_SPEED;
position->y += y_v * MOVE_SPEED;
}
Hi, I noticed you have a reflection library and I was wondering if that means this ECS library is currently capable of having components / systems created from a scripting engine such as v8/JavaScript or Lua.
This feature would be very useful for allowing people to create mods for your game, and add data via components and logic via systems. Minecraft Bedrock (which was created with EnTT) uses an ECS in their scripting API for example: https://minecraft.gamepedia.com/Bedrock_Edition_beta_scripting_documentation
I was also wondering about the viability of compiling to WebAssembly via Emscripten and interfacing from JS.
Currently the code does not compile yet on Windows because:
I don't code on Windows most of the time, so any help would be appreciated!
ut_*
function with ecs_os_api_*
callbacks.flecs.bake
module. It will setup os api within module import function.bake.utils
dependency from all modules use os api instead.For some kinds of components, a user may want to determine how a component value from a temporary stage is merged with the main stage. Currently flecs simply overwrites the component, but in some cases this is undesirable.
For example, an application may have two threads that both fill a dynamic buffer in a component, where after merging, the resulting component should contain the union of the two buffers.
A similar use case needs to be addressed for when an application performs an ecs_set
, in which case the previous component value is replaced with a new one. In some cases, the application may need access to the previous value before replacing it, for example to clean up resources.
To support these use cases, flecs will support lifecycle callbacks for components. The callbacks envisioned are:
Topics to cover:
Hey :D
When working with Unity ECS I found Dynamic buffers to be really usefull:
https://docs.unity3d.com/Packages/[email protected]/manual/dynamic_buffers.html
What do you think about it?
For example to handle collisions, there can be multiple collision hits at same frame. And to not put gameplay logic into collision detection we would separate dealing damage based on those hits into another system. And a bunch more usecases.
I was trying to do "bake clone https://github.com/SanderMertens/flecs" on WSL, and was hit with a bunch of "unknown type name 'uint64_t/uint32_t/uintptr_t/etc'". Is there something additional with the setup for WSL that I missed?
Could add a custom userdata to system callback action?
Currently applications can hardcode the number of threads with ecs_set_threads
. An application may however want express that it wants to run as many threads as there are cores, which could be expressed with
ecs_set_threads(world, -1);
For flecs to learn about how many cores the current system has, a new operation can be added to os_api that returns the number of cores.
Currently it is not possible in Flecs to provide values for entities in bulk. Usecases where data is restored from an external data source therefore require relatively expensive ecs_set
calls to populate the component data.
To facilitate faster insertion of data a new API call is needed that can create/populate a table with data from buffers provided by the application, so that data can be copied with a memcpy per component.
A code example:
ecs_entity_t entity_buffer = {2000, 3000, 4000, 5000};
Position pos_buf = {{10, 20}, {30, 40}, {50, 60}, {70, 80}};
Velocity vel_buf = {{1, 2}, {3, 4}, {5, 6}, {7, 8}};
ecs_new_w_data(world, &(ecs_table_data_t){
.row_count = 4,
.column_count = 2,
.entities = entity_buffer,
.components = {ecs_entity(Position), ecs_entity(Velocity)},
.columns = {pos_buf, vel_buf}
});
While it is currently possible to load data asynchronously and insert it into ECS in the EcsOnLoad
stage, this requires data to be entered one entity at a time into the framework. There is currently no efficient way to directly insert component data in batches, which means that the best thing an application can do is to use ecs_get_ptr
or ecs_set
.
To make efficient loading of large amounts of data possible, flecs should accept application buffers for component data (BYOB). An application could manually provide the buffers of a custom flecs archetype. These buffers could be initialized asynchronously, and when done, inserted into ECS with a single, cheap operation.
A few limitations would apply:
An example API of this feature could look like this:
ecs_entity_t my_entity_buf[] = { ... };
Position my_position_buf[] = { ... };
Velocity my_velocity_buf[] = { ... };
ecs_type_t byob_type = ecs_new_unmanaged(world, "Position, Velocity");
ecs_set_unmanaged(world, byob_type, 0, my_entity_buf);
ecs_set_unmanaged(world, byob_type, 1, my_position_buf);
ecs_set_unmanaged(world, byob_type, 2, my_velocity_buf);
A function needs to be implemented that deletes all entities with a specified type. The function will clear any table that has the specified components.
In systems, the ecs_shared
function is used to return a pointer to a shared component. Systems should not modify shared components, as this could introduce race conditions when the same shared component is accessed by different threads.
To enforce this, ecs_shared
should return a const pointer. Right now it returns a normal pointer.
During the OnLoad phase the application is expected to only insert data into the store. To speed up load operations, systems in the OnLoad phase will not be staged.
Future versions of Flecs will make this configurable (#65)
Fix public API issues. So far, the following have been identified:
Flecs should not contain platform-specific code, and should instead allow an application to provide functions that fulfill platform-specific functionality like creating threads, condition variables and application termination.
Currently when using threads, Flecs splits up the workload of a system into multiple jobs by assigning each job total_matched_entities / thread_count
entities. It does this for every system not in EcsOnLoad
and EcsOnStore
.
This design is an all-or-nothing approach, and typically an application has systems that can either be ran on N threads or one thread. To let the scheduler know how it should schedule systems, an application should be able to provide scheduling parameters.
This could be done through an EcsSchedulingParams
component, which could look something like this (design not yet finalized):
ECS_SYSTEM(world, Sys, EcsOnUpdate, Position,
SYSTEM.EcsSchedulingParams = {.bind_to_threads = [0, -1]);
In the above example, 0
could refer to the main thread, a non-zero value to a worker thread, and -1
could refer to all worker threads. Additionally, applications could specify whether a system shared threads with other systems, and whether a system wants to run without any other systems running.
@SanderMertens Can you please confirm this - when applying multi thread execution via ecs_set_threads(), seeing a crash in ecs_schedule_jobs() from a null dereference of 'table' in line 182.
Note - not using bake, but rather using ecs_os_set_api() per documentation if not using bake; previously this was working with my set up - thinking this may have been recently introduced in one of the recent fixes? Using the same test code as was previously used for creation/deletion of multiple entities.
Currently a system only gets access to the columns of a table that are in its signature. There are cases however where a system needs to access all data in a table, for example when creating a snapshot of component data.
An API is needed that can provide access to the table columns in a system. This would constitute out of two parts:
The following snippet illustrates how a system could iterate columns of a table dynamically:
void MySystem(ecs_rows_t *rows) {
ecs_type_t table_type = ecs_table_type(rows);
uint32_t column_count = ecs_vector_count(table_type);
ecs_entity_t *components = ecs_vector_first(table_type);
for (int i = 0; i < column_count; i ++) {
const void *column_data = ecs_table_column(rows, i);
EcsComponent *component = ecs_get_ptr(rows->world, components[i], EcsComponent);
uint32_t element_size = component->size;
uint32_t element_count = rows->count;
// .. do whatever
}
}
Hi, first of all this is a very interesting project and I've been keeping an eye on it.
My question is simple, how is organized the memory layout?
Would you like to add more error handling for return values from functions like the following?
To make embedding flecs easier in legacy code, being able to compile it for C89 would be beneficial.
By @teksoc-1:
I was using the profiler tool in Xcode, to see memory behavior when creating N entities and deleting the same N entities in while ecs_progress() loop, I observed memory consumption continue to grow and what appeared to be a leak? A little further digging, it seemed that perhaps the staged columns for the data stage where not being freed; I applied a small patch of code in merge_commits() under stage.c to iterate over the staged columns of the data stage and apply ecs_vector_free() do the .data of each staged column for the merged entities. While that seemed to fix the perceived memory leak ( at least to the extent that it gets a nice happy green check mark under Xcode instruments, and I don't visually see the memory continue to grow ), I have no idea if that is even correct, or any unintended side effects I'm not considering, or if there even was a memory leak in the first place ( again, something wrong with my setup? ).
when bake clone SanderMertens/flecs-systems-civetweb
:
flecs-systems-civetweb\src\civetweb.c(641): error C2027: use of undefined type 'timespec'
I wanted to look at physics module, but there is 404
Same for others D:
Query handle:
struct ecs_query_t
{
ecs_system_column_t* columns;
uint32_t column_count;
};
Legacy query construction by signature:
int ecs_query_parse(ecs_world_t *world, ecs_query_t* query, const char *sig);
User defined query construction by something:
int ecs_query_parse_{smth}(ecs_world_t *world, ecs_query_t* query, {smth});
Function modifies given query
pointer and returns non-zero error code.
I know CMake has problems, but it is kind of a standard now. Bake is the only thing that keeps me from using reflecs right now, because I don't want to install/learn yet another build tool, even if it is theoretically superior.
Providing CMakeLists.txt would help adoption IMO.
Currently the Flecs API defines a fixed set of phases (EcsPreUpdate
, EcsOnUpdate
, etc), to which systems can be assigned. These phases are designed to be generic enough to support most scenarios, however if an application needs more (or fewer) phases, the only thing applications can do to day is revert to manual systems. When and how manual systems are ran is fully up to the application, which means that there isn't much Flecs can do to ensure interoperability between such systems.
Additionally, the semantics associated with a phase might influence which features of Flecs are used (threading, staging etc). The current phases may be doing too little or too much for an application.
To address this problem, a user should be able to define custom phases, arrange them in a custom order, and assign custom semantics to each custom phase. The terminology used is:
Custom phases/pipelines should be functionally equivalent to the builtin phases in Flecs, so that there are no adverse effects to using custom phases vs. the builtin Flecs phases. An application should be able to define/run multiple pipelines in a single application.
I've been evaluating this library as one possible solution for a project, but I noticed there are quite a lot of warnings when compiling with visual studio 2017.
Examples:
flecs\src\column_system.c(281): warning C4146: unary minus operator applied to unsigned type, result still unsigned
flecs\src\column_system.c(934): warning C4244: '=': conversion from 'uint64_t' to 'uint32_t', possible loss of data
flecs\src\column_system.c(1001): warning C4244: '+=': conversion from 'double' to 'float', possible loss of data
flecs\src\entity.c(1050): warning C4018: '<': signed/unsigned mismatch
flecs\src\entity.c(300): warning C4090: 'function': different 'const' qualifiers
flecs\src\type.c(1068): warning C4244: 'initializing': conversion from 'ecs_entity_t' to 'int', possible loss of data
etc...
Would you be interested in a push request if I keep using this library?
I find I need something like ecs_rows_t *ecs_get_rows(ecs_world_t *world, ecs_type_t system)
, to update a different table, based on table associated with the calling system.
IOW system table used to update another system table.
My current workaround is to use ecs_run
to find ecs_entity_t
and ecs_get_ptr
for each component. But the manual explicitly says to limit use of ecs_get*
.
When a task (a system which does not subscribe for any entities) is registered with the EcsOnLoad phase, it is not executed. The following code reproduces the issue:
typedef int DummyComponent;
void
Task( ecs_rows_t * rows )
{
printf("Task executed\n");
}
int main( int argc, char ** argv )
{
ecs_world_t * world = ecs_init();
ECS_COMPONENT( world, DummyComponent );
ECS_SYSTEM( world, Task, EcsOnLoad, .DummyComponent );
printf("Run systems\n");
ecs_progress( world, 0 );
printf("Done\n");
return ecs_fini(world);
}
Hey!
Awesome project
I'm interested in how you would approach making rolling back a few (10 Max) frames for changing inputs and resimulation
I'm not that familiar with C.
What I was thinking is if we could allocate ten times as much memory as usually used and at the end of each frame copy all memory related to world to free/oldest block
When it's time to rollback - we clear memory of current frame and copying from storage block. (Or we could start using that storage block as current world)
Another way is store inverted changes then if
-entity was deleted then we store command to create entity with all components
-component changed then we store command to set component value to old value
-etc
Thanks :D
Add example projects that just demonstrate Flecs features to the repository. These examples should not rely on external modules, but should be used to concisely demonstrate a single feature (all other examples / demo's should be stored in flecs-hub).
When calling ecs_quit
from a system, ecs_progress
should return 0 so that the main loop is signalled that the application should stop, but currently this does not work when it is called from a worker thread.
A declarative, efficient, and flexible JavaScript library for building user interfaces.
๐ Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. ๐๐๐
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google โค๏ธ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.