Giter Club home page Giter Club logo

ecez's Introduction

Test Zig changes watch

ECEZ - An archetype based ECS API

This is a opinionated WIP ECS (Entity Component System) API for Zig.

Try it yourself!

Requirements

The master branch zig compiler

Steps

Run the following commands

# Clone the repo
git clone https://github.com/Avokadoen/ecez.git
# Run tests
zig build test
# Run GOL example, use '-Denable-tracy=true' at the end to add tracy functionality
zig build run-game-of-life 

Zig documentation

You can generate the ecez API documentation using zig build docs. This will produce some web resources in zig-out/doc/ecez. You will need a local server to serve the documentation since browsers will block resources loaded by the index.html. The simplest solution to this is just using a basic python server:

python -m http.server 8000 -d ecez/zig-out/doc/ecez # you can then access the documentation at http://localhost:8000/#ecez.main 

Features

As mentioned, the current state of the API is very much Work in Progress (WIP). The framework is to some degree functional and can be played with. Current implemented features are:

Compile time based and type safe API

Zig's comptime feature is utilized to perform static reflection on the usage of the API to validate usage and report useful messages to the user (in theory :)).

    // The Storage simply store entites and their components and expose a query API
    const Storage = ecez.CreateStorage(.{
        Health, 
        Attributes,
        Chest,
        Weapon,
        Blunt,
        Sharp,
        // ...
    }, .{});

    // Scheduler can dispatch systems on multiple threads
    const Scheduler = ecez.CreateScheduler(
        Storage,
        .{
            ecez.Event("update_loop", .{
                // Here AttackSystems is a struct with multiple functions which will be registered
                AttackSystems,
                MoveSystem,
                // ...
            }, .{}),
            ecez.Event("on_mouse_click", .{FireWandSystem}, .{MouseArg}),
        },
    )

    var storage = try Storage.init(testing.allocator, .{});
    defer storage.deinit();

    var scheduler = Scheduler.init();
    defer scheduler.deinit();

    scheduler.dispatchEvent(&storage, .update_loop, .{}, .{});

    // Dispatch event can take event "scoped" arguments, like here where we include a mouse event.
    // Events can also exclude components when executing systems. In this example we will not call
    // "FireWandSystem" on any entity components if the entity has a MonsterTag component.
    scheduler.dispatchEvent(&storage, .on_mouse_click, .{@as(MouseArg, mouse)}, .{ MonsterTag });

    // Events/Systems execute asynchronously
    // You can wait on specific events ...
    scheduler.waitEvent(.update_loop);
    scheduler.waitEvent(.on_mouse_click);
    // .. or all events
    scheduler.waitIdle();

Special system arguments

Systems can have arguments that have a unique semantical meaning:

  • Entity - give the system access to the current entity
  • EventArgument - data that is relevant to an triggered event
  • SharedState - data that is global to the world instance
  • Queries - the same queries described below
  • InvocationCount - how many times the system has been executed for the current dispatch
  • Storage.StorageEditQueue - Any storage type has a StorageEditQueue which can be included as a system argument pointer allowing storage edits
Examples

Example of EventArgument

    const MouseMove = struct { x: u32, y: u32,  };
    const OnMouseMove = struct {
        // We see the argument annotated by EventArgument 
        // which hints ecez that this will be supplied on dispatch
        pub fn system(thing: *ThingThatCares, mouse: ecez.EventArgument(MouseMove)) void {
            thing.value = mouse.x + mouse.y;
        }
    };

    const Scheduler = ecez.CreateScheduler(
        Storage,
        .{
            // We include the inner type of the EventArgument when we register the event
            ecez.Event("onMouseMove", .{OnMouseMove}, MouseMove),
        },
    )
    
    // ...

    // As the event is triggered we supply event specific data
    scheduler.dispatchEvent(&storage, .onMouseMove, MouseMove{ .x = 40, .y = 2 }, .{});

Example of SharedState

    const OnKill = struct {
        pub fn system(health: Health, kill_counter: *ecez.SharedState(KillCounter)) void {
            health = 0;
            kill_counter.count += 1;
        }
    };

    const Storage = ecez.CreateStorage(
        .{
            // ... Components
        },
        .{
            KillCounter,
        }
    );
    const Scheduler = ecez.CreateScheduler(
        Storage,
        .{
            ecez.Event("onKill", .{OnKill}, .{}),
        }
    );

    var storage = try Storage.init(allocator, .{KillCounter{ .value = 0 }});

shared state can also be pointers:

    const OnKill = struct {
        pub fn system(health: Health, kill_counter: ecez.SharedState(*KillCounter)) void {
            health = 0;
            // notice special variable to access pointer
            kill_counter.ptr.count += 1;
        }
    };

    const Storage = ecez.CreateStorage(
        .{
            // ... Components
        },
        .{
            *KillCounter,
        }
    )
    const Scheduler = ecez.CreateScheduler(
        Storage,
        .{
            ecez.Event("onKill", .{OnKill}, .{}),
        }
    );

    var counter = KillCounter{ .value = 0 };
    var storage = try Storage.init(allocator, .{&counter});

    // ..

Example of Entity

    const System = struct {
        pub fn system(entity: Entity, health: Health) void {
            // ...
        }
    };

Example of Query

    const QueryActiveColliders = StorageStub.Query(
        struct{
            // !entity must be the first field if it exist!
            entity: Entity, 
            position: Position,
            collider: Collider,
        },
        // exclude type
        .{ InactiveTag },
    ).Iter;

    const System = struct {
        // Very bad brute force collision detection with wasted checks (it will check previously checked entities)
        // Hint: other_obj.skip + InvocationCount could be used here to improve the checks ;)
        pub fn system(
            entity: Entity, 
            position: Position, 
            collider: BoxCollider, 
            other_obj: *QueryActiveColliders,
        ) ecez.ReturnCommand {
            while (other_colliders.next()) |other_collider| {
                const is_colliding = // ....;
                if (is_colliding) {
                    return .@"break";
                }              
            }
            return .@"continue";
        }
    };

Example of InvocationCount

    const System = struct {
        pub fn system(health: HealthComponent, count: InvocationCount) void {
            // ...
        }
    };

Example of Storage.StorageEditQueue

const Storage = ecez.CreateStorage(
    .{
        // ... Components
    },
    .{}
);
const System = struct {
    // Keep in mind it *must* be a pointer
    pub fn system(entity: Entity, health: HealthComponent, storage_edit: *Storage.StorageEditQueue) void {
        if (health.value <= 0) {
            storage_edit.queueSetComponent(entity, RagdollComponent);
        }
    }
};

const Scheduler = ecez.CreateScheduler(
    Storage,
    .{
        ecez.Event("onUpdate", .{
            System,
            ecez.FlushEditQueue, // FlushEditQueue will ensure any queued storage work will start & finish 
        }, .{}),
    }
);

You can have multiple queries in a single system, and have systems with only query parameters.

Both SharedState and EventArgument can be mutable by using a pointer

System return values

Systems have two valid return types: void and ecez.ReturnCommand.

ReturnCommand is an enum defined as followed:

/// Special optional return type for systems that allow systems exit early if needed
pub const ReturnCommand = enum {
    /// System should continue to execute as normal
    @"continue",
    /// System should exit early
    @"break",
};

System restrictions

There are some restrictions to how you can define systems:

  • If the system takes the current entity argument, then the entity must be the first argument
  • Components must come before special arguments (event data and shared data, but not entity)
    • Event data and shared data must come after any component or entity argument

Implicit multithreading of systems

When you trigger a system dispatch or an event with multiple systems then ecez will schedule this work over multiple threads. This has implications on how you use systems. You can use the DependOn function to communicate order of system execution.

Example:

    const Scheduler1 = ecez.CreateScheduler(
        Storage,
        .{
            ecez.Event("update_loop", .{
                // here we see that 'CalculateDamage', 'PrintHelloWorld' and 'ApplyDrag'
                // can be executed in parallel
                CalculateDamage,
                PrintHelloWorld,
                // Apply drag reduce velocity over time
                ApplyDrag,
                // Move moves all entities with a Postion and Velocity component. 
                // We need to make sure any drag has been applied 
                // to a velocity before applying velocity to the position. 
                // We also have to make sure that a new "hello world" is visible in the 
                // terminal as well because why not :)                       
                ecez.DependOn(Move, .{ApplyDrag, PrintHelloWorld}),
            }, .{}),
        },
    );

    // You can also use structs with systems for DependOn
    const Scheduler2 = ecez.CreateScheduler(
        Storage,
        .{
            ecez.Event("update_loop", .{
                // Submit all combat systems in a single struct
                CombatSystemsStruct,
                // Submit all physics systems in a single struct
                PhysicsSystemsStruct,
                // Submit all AI systems, wait for Physics and Combat to complete
                ecez.DependOn(AiSystemsStruct, .{PhysicsSystemsStruct, CombatSystemsStruct}),
            }, .{}),
        },
    );
---
title: System execution sequence
---
sequenceDiagram
    % our systems
    participant calculateDamage
    participant printHelloWorld
    participant applyDrag
    participant move

    par move blocked 
        par execute unblocked systems
            calculateDamage->>calculateDamage: 
            printHelloWorld->>printHelloWorld: 
            applyDrag->>applyDrag: 
        end
        applyDrag->>move: applyDrag done
        printHelloWorld->>move: printHelloWorld done
    end
    move->>move: run

Multithreading is done through zjobs

zjobs is as the name suggest a job based multithreading API.

Queries

You can query the storage instance for components and filter out instances of components that are paired with unwanted components.

Example

const Storage = ecez.CreateStorage(.{
    Monsters,
    HappyTag,
    SadTag,
    AngryTag,
    SickTag,
    HealthyTag
    // ...
}, .{});

var storage = try Storage.init(allocator, .{});

// .. some construction of your entites

const include = ecez.include;

// we want to iterate over all Monsters, HappyTag and HealthyTag components grouped by entity,
// we filter out all monsters that might have the previously mentioned components if they also have 
// a SadTag or SickTag attached to the same entity
var happy_healhy_monster_iter = Storage.Query(
    // notice that Monster components will be mutable through pointer semantics
    // we query our result by submitting the type we want the resulting items to be
    struct{
        // these are our include types
        entity: Entity,
        monster: *Monster,
        happy_tag: HappyTag,
        healthy: HealthyTag,
    },
    // excluded types are submitted in a tuple of types
    .{SadTag, SickTag}
).submit(&storage);

while (happy_healhy_monster_iter.next()) |happy_healhy_monster| {
    // these monsters are not sick or sad so they become more happy :)
    happy_healhy_monster.monster.mood_rating += 1;
}

happy_healhy_monster_iter.reset();
happy_healhy_monster_iter.skip(5);
if (happy_healhy_monster_iter.next()) |fifth_happy_monster| {
    if (happy_healhy_monster.entity == entity_we_are_looking_for) {
        // the 5th monster becomes extra happy! 
        happy_healhy_monster.monster.mood_rating += 1;
    }
}

Serialization through the ezby format

ecez uses a custom byte format to convert storages into a slice of bytes.

There is a loose spec: ezby spec

Example

const ecez = @import("ecez");
const ezby = ecez.ezby;

// ... create storage and some entities with components ...

// serialize the storage into a slice of bytes
const bytes = try ezby.serialize(StorageType, testing.allocator, storage, .{});
defer testing.allocator.free(bytes);

// nuke the storage for fun
storage.clearRetainingCapacity();

// restore the storage state with the slice of bytes which is ezby encoded
try ezby.deserialize(StorageType, &storage, bytes);

Tracy integration using ztracy

ztracy

The codebase has integration with tracy to allow both the library itself, but also applications to profile using a tracy client. There is also a wrapper allocator called TracyAllocator which allows tracy to report on memory usage if the application opts in to it. The extra work done by tracy is of course NOPs in builds without tracy!

Demo/Examples

Currently the project has one simple example in the example folder which is an implementation of conway's game of life which also integrate tracy

Test Driven Development

The codebase also utilize TDD to ensure a certain level of robustness, altough I can assure you that there are many bugs to find yet! ;)

Planned

Please see the issues for planned features.

ecez's People

Contributors

avokadoen avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar

ecez's Issues

Runtime Type Information for (some) Archetypes

Typed Archetypes must be interchangeable with Byte array based archetypes.
This is needed so that entities can have dynamic archetype types. And so the archetypes themselves can be "upgraded" to static archetypes when needed

If you upgrade or downgrade a entity's archetype then
the information required to move the entity is only known at run time

Specialized parameters for systems

Systems should be able to request specialized variables:

  • Entity
  • *World (?) not sure I want to give this footgun
    • world can be supplied with some specialized type wrapper to ensure safe operations on the world state i.e deferring operations to the end of the dispatch
  • Custom user data #12 solves this usecase
  • Delta time #12 solves this usecase

Related issue: #12

anytype event data for events

triggerEvent should take an aditional event context parameter for events that require additional context

i.e: user input

WorldInFlight argument

A system should be allowed to have an argument before component argument of type WorldInFlight which will expose some functions to mutate the world in a thread safe manner and can later be used to perform queries in systems (this will have to be carefully implemented to only do query requests once per system ...)

Current planned functions WorldInFlight functions:

  • setEntityType: move an entity to new archetype (only valid entity should be current system call entity)
  • query: get an iterator which can give resulting components or archetypes (single threaded, readonly)
  • more?

Related issue: #14

Cache system for archetypes used by a ECS system

Currently each dispatch will allocate by calling getTypeSubsets()

A cache should be utilized instead by either

  1. Gradually store system archetypes as archetypes are created
  2. Store the result of getTypeSubsets until new archetypes are created which should invalidate systems affect by introducing a new archetype

document more in readme

  • events
  • systems
  • components
  • archetypes
  • world creation
  • specialized arguments
    • SharedState
    • EventArgument
  • synchronization
    • waitDispatch
    • waitEvent
    • dependOn (TODO: not yet implemented) #23

Increase type safety by enforcing explicit intent

ecez can increase type safety by enforcing users to explicitly mark types based on usage and also extend supported types
i.e Vector type and others

This will also make it easier to reason with types in the implementation

Example:

const Position = ecez.Component(@Vector(f32, 3));
const Velocity = ecez.Component(@Vector(f32, 3));
const Health = ecez.Component(f32);

const Colors = ecez.SharedState(struct{
 // red
 // ....
});

Events

Implement a way of registering events to a world

Refactor archetypes to utilize comptime more

Types are currently runtime based which has a range of problems for performance including query overhead, allocations etc ..

Making type information utilize comptime type will improve some aspects of this and should resolve issue #10

Shared state mechanism

State that is not connected to the entity should be able to be expressed and shared across systems

Suggested API:

// writing to shared state should be allowed and thread safe 
// scheduler will avoid dispatching these in parallel
fn systemWithSharedStateMutate(a: A, window: Shared(*WindowHandle)) void {
    window_api.setCursor(.default);
    _ = a;
}

// read only shared state allow scheduler to do these in parallel!
fn systemWithSharedStateConst(a: A, color: Shared(CommonColors)) void {
    a.color = color.green;
}

var world = WorldBuilder().WithSystems(
    systemWithSharedStateMutate,
    systemWithSharedStateConst,
).WithSharedState(
    WindowHandle,
    CommonColors,
).init(allocator);

world.registerShared(WindowHandle, 123);

try world.dispatch(); // error.UndefinedSharedState 

Expand on readme

Current readme is enough for now, but should be expanded on when/if the feature set becomes practical for external usage

implement Systems

Systems in ecez should be compile time known in order to allow the compiler to do any optimizations that are enabled by this. This is of course the alternative to using function pointers

In order to make it easier to register larger quantity of systems, a combination of functions and structs should be valid input to a register system call. Example:

const archetype_ab_systems = @import("ab_functions.zig");

fn anotherSystem(a: *ComponentA, c: ComponentC) void  {
    a.value += c.value
}

pub fn main() !void {
    // .... init api

    // here we register all functions that are visible 
    // from archetype_ab_systems.X (ignores values and nested function definitions)
    api.registerSystem(archetype_ab_systems);
    // here we add a single function
    api.registerSystem(anotherSystem);
}

It might be problematic to compile time embed complex systems because it removes some scheduling possibilities. Future efforts should be made to used function pointers for more complex systems, and schedule them dynamically

Benchmark and consider AOS instead of SOA

SOA is usually the better option for performance (citation needed)

But in an archetype based system we can expect systems utilizing archetype A would want to utilize most components per entity. Maybe this is something users should be able to control themself, because my hunch is that it might be very case by case based

Suggested syntax:

const World = WorldBuilder().WithArchetypes(
    Archetype.A, // default is SOA
    AOS(Archetype.B), // Force B to be stored as AOS
    SOA(Archetype.C), // Explicitly use SOA storage
);

Resolve issue with zig self-hosted

There are compile issues in the scheduled daily zig build because the default compiler is now the self hosted one.
The code should only need minor tweaks in theory, but in practice there are some bugs in the self hosted which are not present in stage1

queries & Iterators

Ability to iterate entities and components (which is an alternative to implicit iteration with systems)

Remove runtime marker for SharedState structs

Because of issues with the zig compiler, generated SharedState structs gets an added variable to their type which exist on runtime. This is bad for many reasons and the added tag should only exist on compile time.

Also make sure to just ptrCast/bitCast values when this issue is resolved instead of copying struct between types ...

Support packed components

Currently all components must be 1*N byte aligned which exclude packed structs

Suggestion is to replace array of u8 to a tagged union which has a u1, u8, u16, u32, u64 variants for more specialized storage depending on component type.

Will wait to implement this until some benchmarking can be done to make sure it does not slow down things too much

API Design: define entities by archetype directly

Building on the features made in the comptime branch we can design the API to utilize compile time more and enabling these features to be merged in master. Merging comptime branch in master will resolve issue #27

The user will have to define archetypes i.e:

const Player = struct {
   // Note variable name is not important here
   hp: Health,
   pos: Position,
   vel:  Velocity,
};

const StaticObject = {
   Pos: Position,
};

These archetypes are then stored in a comptime hash map

The API will not allow dynamically adding or removing components from an entity, but changing the entity type will be valid in any direction (or maybe this should be restricted?)

This will like mentioned allow the library to move a lot more over to compile time and hopefully make planned queries mostly compile time based as well and add the functionality for type queries i.e give me all Players

It will also make entity construction very fast since the user will always specify the full component list at creation i.e

world.setEntityType(entity, Player{...});

Improve CI

Use alpine instead of ubuntu
Curl zig master instead of installing snap and using snap to install zig master

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.