Giter Club home page Giter Club logo

automata's Introduction

⦿ | Automata | ⦿ ⦿ ⦿ ⦿

Spawn a system of concurrent, distributed, fault tolerant, and highly available intelligent agents for coordinated and/or uncoordinated action in one or many environments with no central point of failure. This project is Open Source. License

Project Mission

The Automata Project combines the state-of-the-art AI control techniques with the latest research in autonomous decentralized systems, providing AI designers a flexible framework for creating valuable emergent properties in real world systems.

This project is in the Alpha stage and not ready for production systems. We need Contributors to get to 1.0. We are eager for your contributions and very happy you found yourself here! Please join the slack channel and/or reach out to [email protected] if interested or with any questions. Here are our contributing guidelines and get up to speed on the wiki.

Roadmap

Note: This will be updated often. The direction of the project will change as the work evolves. We welcome any of your thoughts about roadmapping.

See the current milestones here.

Usage

Testing is currently happening using the mock sequence in world/mock_world/mock_automata/mock_seq_1.ex. This is currently our canonical example of how to design a behavior tree (one type of Automaton), and eventually communicates with other heterogeneous agents that you define in world/<mock_world>/<mock_automata>/<mock_automaton>.

Currently, you can run the mock sequence in an iex shell as follows:

iex -S mix
iex(1)> Automata.spawn()

and tests can be run with debugging capabilities as follows:

MIX_ENV=test iex -S mix test test/unit/automaton/automaton_test.exs:23

where you are running a specific context/it block containing line 23.

Implementation Overview

Technologies

  • Elixir & OTP provide the primitives for robust concurrent, fault-tolerant, highly available, self-healing distributed systems. The main point of the Erlang model is an application that can be expected to run forever, as stated by the language designer. Talk about unstoppable agents! Read more about the many benefits of Elixir and OTP.

  • Heterogeneous agents equipped with reasoning and knowledge components provide AI architects a selection of graphical, reinforcement learning, and neuroevolution based orchestrated agent constellations for a variety of real world environments.

  • Why and How Elixir? Elixir for world class communication & control fault tolerance and distribution. Nx is used for numerical computation. We primarily focus on unsupervised learning and architecture search. Evolutionary architecture novelty search (EANS) evolves populations of feed-forward neural networks without computationally expensive backpropagation and uses novelty to explore the search space. Mixture of Experts (MoE) Deep Learning is on the roadmap.

The Automata framework has three abstract semantic layers:

Environment (State Space Representations)

The maps of the territory. Without good maps, any explorer could easily get lost. To fit the information available to the problem at hand, we separate representations into distinct layers with well defined boundaries.

Reasoning (Decision Making, Action Selection)

All agents have some formulation of action selection, otherwise they would never achieve their goals. To keep agent decision making correct, we fit the mode of reasoning to the representation.

Knowledge (Memory, Local & Global Histories)

Interchangeable memory modules across short term working memory, blackboards, and knowledge bases.

Read the wiki and/or the docs for more about the technologies underlying Automata.

Requirements

Autonomy is the capacity of agent(s) to achieve a set of coordinated goals by their own means (without human intervention) adapting to environment variations.

It combines five complementary aspects:

  1. Perception e.g. interpretation of stimuli, removing ambiguity/vagueness from complex input data and determining relevant information based on context/strata/motif specific communications
  2. Reflection e.g. building/updating a faithful environment run-time model
  3. Goal management e.g. choosing among possible goals the most appropriate ones for a given configuration of the environment model
  4. Planning to achieve chosen goals
  5. Self-adaptation e.g. the ability to adjust behavior through learning and reasoning and to change dynamically the goal management and planning processes.

Note that the five aspects are orthogonal. The first two aspects deal with “understanding” the map of the environment. The third and the forth aspects deal with autonomy of decision. Self adaptation ensures adequacy of decisions with respect to the environment map. See MMLC2.

A system is defined as an Autonomous Decentralized System (ADS) if the following two properties are satisfied:

  1. Autonomous Controllability: Even if any subsystem fails, is repaired, and/or is newly added, the other subsystems can continue to manage themselves and function.

  2. Autonomous Coordinability: Even if any subsystem fails, is repaired, and/or is newly added, the other subsystems can coordinate their individual objectives among themselves and can function in a coordinated fashion.

Features

Functional Features

User Defined Agents

Planned builtin and/or custom third-party agent types include:

  • Graphical Model Based

  • Reinforcement Learning (RL) Based

    • Bandits
    • td-learning, q-learning, actor-critic
    • Deep Learning, Gann
    • DEC-POMDP (Decentralized Partially Observable Markov Decision Process)
  • Neuroevolution

    • TWEANN (Typology and Weight Evolving Artificial Neural Network)

A Concurrent, Scalable Blackboard Knowledge System

The central problem of artificial intelligence is how to express the knowledge needed in order to create intelligent behavior. — John McCarthy, M.I.T/Stanford Professor, Creator of Lisp

  • A global blackboard that can coordinate automata without being a central point of failure.
  • Individual automaton blackboards readable by all automata, writeable by owning automaton

Meta Level Control

  • Meta-level control to support agent interaction, any potential network reorganization. Meta-level control is the ability of an agent to optimize its long term performance by choosing and sequencing its deliberation and execution actions appropriately. 2

Neuromorphic Computing

  • potentially bringing the code to the data rather than the other way around.

Performance Features

  • Learn more about the performance features of Automata

Applications

  • Trading Systems
  • Pandemic Testing Drone Units
  • Patient Monitoring & Care Systems
  • Swarm Intelligence / Distributed Robotics
  • Intelligent agents with soft realtime multi-dimensional sensory, perception, intuition, and action capabilities
  • Multi-Agent Reinforcement Learning (MARL)
  • Mixture of Experts (MoE) Deep Learning Control Systems
  • Blockchain Smart Contract Systems
  • A Mega-constellation of satellites
  • 3D Printing and Factory Automation
  • Product Automation & Analytics
  • Augmented, Virtual, Mixed Reality Backends
  • Smart Home / IOT Systems
  • High-Speed Rail Systems (Japan has an ADS railway that learns)
  • Chatbot & Game AI's (esp. MMOG user/npc backends)
  • QA Testing (BT's are particularly suited to combinatorial testing)
  • ? (choose your adventure)

API

defmodule MyAutomaton do
  use Automaton,
    # required
    type: :behavior_tree,

    # the remainder are required with type :behavior_tree.
    node_type: :selector,

    # the heartbeat for this node(subtree), in milliseconds
    # the default is 50ms (mimics the average human brain perception cycle time)
    # heartbeat adaption as meta-level(automata) action, to be changed at runtime
    tick_freq: 50, # 50ms

    # excluded for execution nodes
    # list of child control/action(execution) nodes
    # these run in order for type :selector and :sequence nodes and in parallel
    # for type :parallel, and in a user-defined dynamic order for :priority
    children: [ChildAction1, ChildSequence1, ChildAction2]


    @doc """
    Required with `type: :behavior_tree`
      - Called every tick, must return `{:ok, status}` or `{:error, reason}`

    ## Reactively and Proactively Change the World
      > ie.. effect the current environment in phases using
        either actuator *effectors* or via *communication* with
        other agents and/or internal/external systems
    """
    def update do
      {:ok, status}
    end
end

Example

Below is a simplified hypothetical example of a :behavior tree sequence control node(subtree) for an autonomous "Forex Trader". The first two leafs are condition nodes, and the last two are action nodes.

automata trader sequence diagram

References
  1. Multi-Agent Meta-Level Control

Installation

If available in Hex, the package can be installed by adding Automata to your list of dependencies in mix.exs:

def deps do
  [
    {:automata, "~> 0.1.0"}
  ]
end

Authors

See also the list of contributors who participated in this project.

License

This project is licensed under the Apache 2.0 License - see the License.md file for details or License

automata's People

Contributors

kiote avatar paultannenbaum avatar upstarter avatar venomnert avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

automata's Issues

Turn WorldConfig into a struct.

In lib/automata/automata.ex.

So we can use embedded schema without a database, and eventually probably also make automaton_config a struct too

Validate user opts to the `use Automaton` macro.

Let's add required fields validation, error handling
for user_opts in Automaton.Config.Parser using the Ecto.Schema validation protocol

The NodeTypeError for user_opts[:node_type] is just a string message now, we should make some real error types.

Tick/Update tree runs out of order

Right now with the else clause here commented out, when you run iex -S mix, the tree runs once in the correct order, but when you uncomment the else clause, it runs out of order. I am assuming this is because Process.send_after runs async. But this will not work with BT updates

Should we forget about ticking the entire tree(subtree) and just store the currently running node somewhere as recommended in some BT literature, or should we try to get this working so we have more fine grained control of subtree tick frequencies? One could imagine scenarios in which a control node should update at a high or low frequency relative to others.

Create an `assets/` directory

  1. Move all images from the root dir into an assets dir

  2. Search and replace all links to images and replace with new assets/ path

Set up project to be able to perform dialyzer and docs mix tasks

It is desirable to be able to run both the dialyzer and docs mix tasks after every change to automata. They should be run before a PR for a change is requested.

To effect this, dialyxer and ex-doc needs to be added as dependencies to mix.ex

See issues #6, #7 and #9

This should probably be done earlier than later in the project's life. It is my understanding that in a couple of weeks (end of Feb?) there might be some significant changes to automata although this should not affect changing automatas dependencies.

Please read the online help for Elixir's dialyzer and docs. tasks.

setup basic unit testing for `lib/core/control/automaton.ex`

Setup a good skeleton with elixir testing best practices would be a big step forward for the project.

See the BTSK (behavior tree starter kit) for examples of whats important to test for the abstract stuff in the API. That would be a great starting point.

Handle user update asynchronously

Find a way to pass a function to state in user-defined update function so that the update can be run async as a Task so subsequent ticks will return :bh_running until the task completes (which sets the resulting status). This makes the agent reactive. Or is there a better more elixir way to do this? I'm thinking similar to this:

  def handle_cast({:run, fun}, state) do
    Task.async(fun) # sends a message back to the automaton when completed
    {:noreply, state}
  end

  # handle_info/2 receives generic messages from the Task processes
  def handle_info({_task, {:ok, result}}, state) do
    new_state = %{state | status: result}
    {:noreply, new_state}
  end

  def handle_info({_task, {:error, reason}}, state) do
    state = %{state | status: :bh_failure, fail_reason: reason}
    {:noreply, state}
  end

First Test for the Mock Sequence

Create simple behavioral test for the MockSeq1. Test that it runs all of it's children, one after the other in time, where no successor runs before a predecessor (since we aim to be fully async).

We can either use espec or the elixir builtin testing library. We can setup the TestMockSeq1 to have more children (simple actions would be best) for now until we get factories or fixtures.

Add Factories / Fixtures

It would be nice to have Factories similar to FactoryBot for ruby/rspec. Is there a good library for elixir? If not, can we setup fixtures?

Define protocols and data flows for control processes

right now lib/core/control/behavior.ex has the basic API.

TODO:

  1. what are the functions that the user will have to define and/or override? Can we get away with just having them define the update function.

  2. Once we know this we can decide how the params will be passed, perhaps as a map or keyword list.

Run a basic mock tree and tick and update the nodes.

Does the NodeSupervisor become the root source of the tick traversal? Or is it in the root user-defined node?

Is it feasible to have a per node tick_freq rather than a tick_freq for the entire user-defined tree? Seems doable but we need to think through all implications and determine if this is the way to go.

Performance Audit

  1. Determine best way to increase performance by starting subtree nodes as needed or some other such mechanism to avoid starting the whole BT on application start.

  2. Determine the best way to prevent ticking the whole tree when started and running. Perhaps a zipper tree can be used to store the tree at the root and only update currently running node as long as it is running otherwise re-update entire tree.

Convert all start_link calls to use SHA hashes as names, via the Registry "via tuple".

Using atoms as name option in start_link is bad for scalability. Since these are not garbage collected, using them will eventually cause the BEAM to run out of memory. Lets use unique SHA hashes as names using the via tuple contract provided by Registry.

Use unique content hashes for names of nodes and eventually Distributed Hash Tables for resource discovery (like kademlia). SHA hashes provide versioning and avoid name collisions. For example, reusing an action would be unique from another version even though the user used the same module. Two versions of same module can do different things also. Each version runs in separate process. No namespace collisions. One could have thousands of the same module running as processes. This is a very good thing for programming large systems.

Typespecs for currently defined functions.

In order to provide type checking we should go through and add typespecs for the currently defined functions so we get nice compiler errors and extra documentation from them.

Write Tests for AutomatonServer

We need tests for this as it is one of the trickier logics. Not great for beginners but good first issue for a senior dev.

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.