Giter Club home page Giter Club logo

neurorobot's Introduction

SpikerBot app by Backyard Brains


Summary
A neurorobot is a robot controlled by a computer model of a biological brain. Backyard Brains is developing a neurorobot for education that enables students to perform computational neuroscience investigations by designing brains and observing their behavior (Harris et al., 2020). Students use the NeuroRobot app to design and simulate biologically-based neural networks connected to mobile robots with sensors (camera, microphone, distance sensor) and actuators (speaker, motors, lights).

Installation
To install the SpikerBot app on a Windows PC, download this repository, unpack SpikerBot_Installer.zip, and run the installer SpikerBot_Installer.exe. Because this is an unsigned app, you will need to navigate a blue warning message by clicking 'More info' then 'Run Anyway' in order to install the app. The app also runs natively in Matlab on all platforms.

How the app works
The NeuroRobot app has 3 modes of operation: Startup, Runtime and Design. Startup is the main menu. Runtime is the brain simulation engine where you can see what the brain is observing, hearing and doing. This is also where you provide dopamine rewards to modulate its motivation and learning. Design is where you modify the structure of the brain. Click anywhere in the brain-shaped area to add a neuron or synapse. Select the orange box next to the eyes, microphone or whiskers to send sensory information to specific neurons in the brain. Select a neuron and click the ‘Axon’ button to extend an axon to the wheels, the speaker or another neuron.

The simulated neurons are designed to model the spiking (aka firing) of biological neurons. Thus, they can be quiet or fire regularly or in bursts, and can respond in different ways to different synaptic inputs. Synaptic connections between neurons have a “weight” (-100 to 100 mV) that represents the strength of the synapse. Every time a sending (“presynaptic”) neuron fires a spike, the weight of the synapse is applied to the receiving (“postsynaptic”) neuron’s membrane potential. To reliably trigger a spike, a synapse should have a weight of 25 mV or more. To test this, connect a highly active neuron to several quiet neurons, use different synaptic weights, and examine the different rates of spiking produced.

Synaptic connections can be plastic. This means that if a sending and a receiving neuron are active at the same time, a synapse connecting them will grow stronger. In other words, neurons that fire together, wire together. Some synapses are plastic only in the presence of a dopamine reward.

The NeuroRobot app collects camera and microphone data continuously. Within this data, it can detect simple features such as color and pitch, and complex data such as objects and words. To make a neuron respond to a sensory feature, select the orange square next to the relevant sensor, then select the target neuron. To make a neuron produce movement or sound, select the neuron first, then the orange square of the desired speaker or motor.

The basal ganglia allows vertebrate organisms to select particular actions in particular situations. Specifically, the basal ganglia disinhibits (i.e. activates) one group of neurons, associated with one specific behavior, at a time. Dopamine rewards make the currently selected basal ganglia neuron group and its associated behavior stay selected longer (higher "motivation") and increases their likelihood of being selected in similar situations in the future. Inputs to basal ganglia neurons strongly influence how long the currently selected neuronal group stays selected. Neurons belonging to a particular basal ganglia group are identified by the “ID” variable, by dashed lines emanating from "Striatal" neurons, and (optionally) by color.

neurorobot's People

Contributors

djordje-jovic avatar gagegreg avatar ilya-muromets avatar stanislavmircic avatar vertebot avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

neurorobot's Issues

Microsoft Win32 SAPI

Hi all,

I'm on an old MacBook Pro running High Sierra and trying to run neurorobot.m with just my webcam and no robot, with the following settings:

%% Settings
rak_only = 0;
camera_present = 1;
use_webcam = 1;
hd_camera = 1;
use_cnn = 0; % requires gpu
use_rcnn = 0;
grey_background = 1;
vocal = 1; % custom sound output
supervocal = 1;
brain_gen = 0; % brain build
pulse_period = 0.1; % in seconds
matlab_audio_rec = 1;
audio_th = 2;
microcircuit = 0;

However, when I do, I get this message:

>> neurorobot
---------
Error using tts (line 33)
Microsoft Win32 SAPI is required.

Error in neurorobot (line 196)
        this_wav = tts(this_word,'Microsoft David Desktop - English (United
        States)',[],16000);

Is there a way I should be installing Microsoft SAPI? And which version? Alternatively, is there a setting I can change so that it won't rely in SAPI? Thanks!

It's a me, Mario!

Sadly the Mario brain is giving me this error. :(

Unknown synapse type
Unknown synapse type
Unknown synapse type

Brace indexing is not supported for variables of this type.

Error in draw_brain (line 274)
                        plot_contact_synapses(nneuron, ncontact, 4) = text(x2b, y2b, audio_out_names{neuron_tones(nneuron)}, 'fontsize', bfsize - 4, 'verticalalignment', 'middle', 'horizontalalignment', 'center', 'FontWeight', 'bold');

Error in update_brain_name_edit (line 59)
    draw_brain
 
Error while evaluating UIControl Callback.

Corrupted some *.mat files

Corrupted some *.mat files

There is an issue with corrupted *.mat files at some point. It is because those files are marked as large files and in that case it is necessary to use git lfs while pushing those files via git. It is working with Terminal on macOS, but it seems an issue with GitHub Desktop app.
Pay attention to this in the future.

Crash when Karl brain is selected

If I am connected to RAK and I select "Karl" brain:
simulation will run but it will crash on line 11 in update_bran.m:
vis_I(nneuron) = vis_I(nneuron) + sum(vis_pref_vals(these_prefs, ncam));

because
"these_prefs" that is loaded from brain mat file is 1x9 logical array and
"vis_pref_vals" is 6x2 double
when "ncam" is 2 logical indexing goes out of array "vis_pref_vals" bounds with error:

The logical indices in position 1 contain a true value outside of the array bounds.

Attempt to connect to RAK without WiFi causes hard crash

If I try to connect to the RAK module using the C++ library but forget to connect to the module's WiFi first I get this output:

openStreams >> SharedMemory >>> opened
SharedMemory >>> init
openStreams >> VideoAndAudioObtainer >>> opened

then Matlab freezes. It's a hard freeze: to close and restart matlab I have to click the 'End now' button in the 'Matlab not responding' dialog box twice.

Can we change it so the app aborts the connection attempt if OS is not connected to a WiFi beginning with "LTH" (RAK standard) or add a try/catch or timeout or something so I can just get a quick error/reminder instead of crash?

Some brains have neurons connected to themselves

Some brains have neurons connected to themselves

List of brains with issue:

  • Chopin
  • Mocho
  • Momo
  • Opportunity
  • Reservoir
  • Vomba

In matrix: connectome for brains above, there are neurons which have some value more than zero for the same column number (e.g. row 3, column 3). which means it is connected to it self.

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.