srobo / astoria Goto Github PK

This is a meta-ticket to track the various parts required to implement astwifid.

• Design #52
• Implementation #53
• Testing

We should automatically test that the various managers and consumers can talk to each other correctly

We should print this so that we can see what version of code students are running

I was just looking at htop on a robot and noticed that astwifid was idling at around 17% CPU. We should look into why.

e.g 2021-01-30 19:27:08 astctl-10971672-b38c-4724-90ac-83b61bd7a50b INFO astctl-0788fa8e-3be1-4465-88e4-a262ff43581d v0.1.0 - Set a metadata attribute.

• Ugly
• Not useful for CLI tools

there's some variation in line wrapping between files – do you have opinions on whether we should hard-wrap docs or not (and if so, at how many columns)? it's already inconsistent in the existing docs, so don't worry about changing anything this pr, but possibly something to consider

I'm not really sure where I sit on this. I'll open an issue and we can discuss this evening.

Originally posted by @trickeydan in #20 (comment)

Create a replacement for astdiskd (astdiskd-mock) which can be used to manually add/remove fake disks pointing to a directory on the filesystem.

Solutions:

• Add repo containing those packages to build process
• Override package names in debian folder

I: dh_python3 pydist:224: Cannot find package that provides dbus_next. Please add package that provides it to Build-Depends or add "dbus_next python3-dbus-next" line to debian/py3dist-overrides or add proper dependency to Depends by hand and ignore this info.
I: dh_python3 pydist:224: Cannot find package that provides gmqtt. Please add package that provides it to Build-Depends or add "gmqtt python3-gmqtt" line to debian/py3dist-overrides or add proper dependency to Depends by hand and ignore this info.

probably good to document exactly what happens in this case, though doesn't have to be done in this pr

Originally posted by @sersorrel in #19 (comment)

When trying to kill astctl usercode log with ^C it doesn't quit on its own.

It would be useful to be able to get an individual attr

We need to be able to get a marker offset from metadata disks to allow offsets to be set for a game.

Doesn't exit on ctrl+c.

Likely caused by the while True not responding to the caught Signal.

Consumers (at least) should search for astoria.toml in /etc/srobo/astoria.toml

• Data Components
  • Data Component class
  • Consumer Base Class
  • Manager Base Class
• astprocd messages
• MQTTWrapper
• astctl
  • astctl event
  • astctl list-disks
• Managers
  • astdiskd
  • astprocd
    • UsercodeLifecycle / Execution
  • astmetad

We should use a common class for commands to reduce code overlap.

The class should also handle the timeout issue

We have decided that we no longer want bundle.toml to be used inside the bundle file.

We want to cache wifi_{ssid,psk,region}

Error message:

2021-10-23 19:28:45 astdiskd INFO Static disk 1e24acb1-0169-4516-862e-4334dded6ff3 mounted (/home/dan/Code/sr/sr-robot3/examples)
2021-10-23 19:29:29 astdiskd INFO Static disk static-1e24acb1-0169-4516-862e-4334dded6ff3 unmounted (/home/dan/Code/sr/sr-robot3/examples)
2021-10-23 19:29:29 astdiskd ERROR Task exception was never retrieved
future: <Task finished name='Task-121' coro=<StateManager._register_request.<locals>._handler() done, defined at /home/dan/.cache/pypoetry/virtualenvs/sr.robot3-y247SDwh-py3.9/lib/python3.9/site-packages/astoria/common/manager.py:73> exception=RuntimeError('dictionary changed size during iteration')>
Traceback (most recent call last):
  File "/home/dan/.cache/pypoetry/virtualenvs/sr.robot3-y247SDwh-py3.9/lib/python3.9/site-packages/astoria/common/manager.py", line 76, in _handler
    response = await handler(req)
  File "/home/dan/.cache/pypoetry/virtualenvs/sr.robot3-y247SDwh-py3.9/lib/python3.9/site-packages/astoria/managers/astdiskd/static.py", line 106, in handle_remove_all_static_disks
    for uuid, path in self.disks.items():
RuntimeError: dictionary changed size during iteration
^C2021-10-23 19:30:15 astdiskd INFO Halting

At the moment the hostapd config option for bridge is commented out. As we'll need to make use of it later, we should have it be populated by astwifid.

We need to be able to set and compare the following extra attributes:

• WiFi region (what are valid values for this?)
• WiFi Client / AP mode (i.e connect to arena)
• WiFi enabled / disabled - we may want to disable the wifi.

These are defined in the Metadata schema.

We currently store test zip files, whilst this is useful to have, it is also incredibly difficult to look at exactly what code the unit tests are running.

We should zip these before running the unit tests in a temp directory. Binary files in git isn't fun.

The output should be run through a sort first

Check if info.yaml or wifi.yaml (yml too)are present in the robot.zip, if so the reject the zip and print an error stating that this is an incompatible (old) zip.

May be worth checking for overlay files too.

Might be worth doing this at the same time as #54

If a second process connects to the broker with the same client ID, it breaks the existing client.

It should raise an exception on the new client.

This behaviour is important as we don't want multiple instances of state managers.

We should maintain compatibility with this. It is currently main.py.

It would be nice if this was configurable between robot.py and main.py, or even to run languages other than Python.

We will need to design the various states that astwifid can be in.

e.g client mode / AP mode.

It will need to listen to information from astmetad to listen for changes in credentials, region and robot mode (i.e comp / dev)

The design should be added to the design section of the documentation and must include a state machine diagram.

The Raspberry Pi WiFi driver doesn't support 802.11w which is required for WPA3.
There should be a configuration option for Astoria which allows WPA3 to be disabled.

We need a systemd unit for each of the state manager services.

• systemd units
• add astoria user
• config file into /etc/astoria.toml (needs #37)

TODO: Get this into the log file.

I think that would require sufficient changes that we should do it in a separate PR

Originally posted by @trickeydan in #59 (comment)

• Running astoria
• Writing new State Managers / Consumers
• Architecture / State Diagrams
• Code documentation

Published by a state manager or consumer to /astoria/event/event-name (retain=False)

Potential Format

class EventData(pydantic.BaseModel):
	event_name: str
	triggering_component: str (Client name of triggering component)
	data: str  # Must always be a string

Use Cases

• Broadcast of logs in real-time from astprocd (#22)
• Start button trigger (#23)

If a manager crashes when handling a request, it will never respond to the requestor.

We need to add a timeout to handle this, to prevent things getting stuck.

Example: astctl usercode kill hangs if astprocd crashes.

Introduced in 832ad05

If waiting on multiple dependencies, there is a possible race condition.

• foo depends on bar and bees
• foo starts and waits for bar and bees
• bar starts
• foo marks bar as started
• bar stops
• bees starts
• foo marks bees as started
• foo starts as all dependencies are satisfied (they aren't)

The code here refers to a "mutation source". The correct term is now Metadata Source

Currently these are referred to in both code and docs as both disks and drives.

The canonical term is disk. Any references to drives should be replaced with this term to reduce confusion.

The robot.zip must contain a bundle.toml as defined in the spec. If it does not, then it should fail to load.

All of the problems with this.

It would be very nice if log messages had a timestamp alongside them.

We should package this application for:

• Debian
• PyPI
  • Automate PyPI upload in CI

Implementation of #52

This may need to be broken down further.

The kit version in bundle.toml should be compared to astoria.toml.

The kit target should also be checked

It should be compared as defined in the spec

This should be done in astprocd