Setup initial schema:

tenants

id
name
jwt/auth key? - or user specific?
public key(s)

users

id
tenant_id
name
email
password

devices

id
tenant_id
identifier (name?) text, unique
current_version
target_version
last_comm
last_update?
channel
architecture/platform?
tags array<string>
metadata - from [nerves](https://github.com/nerves-project/nerves_runtime#nerves-system-and-firmware-metadata)

firmwares

id
tenant_id
signed?
in_use?
s3/gcs/identifier (start with s3, can be adapted)
(can extract metadata)
channel
architecture/platform

deployments

id
tenant_id
conditions/query
firmware_id
status
started_at
finished_at

Detail view of a deployment

Acceptance Criteria

• Logged in user should only be able to see deployments for their own tenant
• Show name, criteria and status of deployment
• Ability to toggle the status of a deployment between inactive and active

Technical Acceptance Criteria

• Change status to a boolean is_active field

Acceptance Criteria:

• Any time a password is created/updated, there is a sane password strength check in place to ensure terrible/short passwords aren't used.
• There's no upper limit on password length or complexity, and no characters are disallowed.

Acceptance Criteria:

• On the "Manage Devices" page there's an "Add" button that takes the user to a page where they can create a device.
• The user provides a device ID, Description, and Tags. (Text fields; Tags are interpreted as a comma separated list.)
• Device ID and Description are required (validation errors are shown and the device isn't created, if either are missing)
• Device ID is unique per tenant.

Acceptance Criteria

• On the device view, the user should be able to edit the tags list
• On save/update, the tag input should be split on commas and trimmed and then stored
• Tags is the only field that is editable

Acceptance Criteria:

• A logged in user can view the settings for the tenant they belong to
  • Not sure what those settings are yet

Current behavior:
When you upload a .fw file, the browser acts like it is working, but it's hard to tell how far it's gotten.

Expected behavior:
There's a progress bar or some feedback that lets you know that things are happening.

Steps to reproduce:
Go to the firmware tab, browse and then upload a .fw file.

Acceptance Criteria:

• When a user logs in, they're taken to the "logged in home page" (at / -- but it detects they're logged in so it doesn't show the login page).
• The page displays links:
  • Manage Devices
  • Manage Firmware
  • Manage Updates
  • Settings
• There's a logout link (for all logged in pages; do this via a Phoenix layout) and the user's name / email address is displayed in the same toolbar/area as the logout link.

References: https://github.com/nerves-hub/nerves_hub/blob/master/docs/client-server.md

The final step of the join process is to respond to the device with a list of the group channels that it should join.

Depends On: #79

Acceptance Criteria:

• When the user clicks on the Manage Devices link from #13, they're taken to a page that lists all of their devices in a table.
• The table has the following columns:
  • Device (shows device name/id)
  • Version
  • Update Pending (has a disabled checkbox)
  • Last Connect
  • Operations (shows Delete and Disable links)

Minimum requirements for the device to report to Nerves Hub:

• Firmware version - so that the server knows if new firmware needs to be sent down and can record that an upgrade completed.

Minimum needed to send back down to the device:

• Whether the device should update its firmware
• A URL to the new firmware

Nice to have notifications from the device:

• An error message if the firmware failed to download or apply - useful for debug

Other items to consider:

• If devices report more of their metadata (see https://github.com/nerves-project/nerves_runtime#nerves-system-and-firmware-metadata), the server can detect misconfigurations
• We've had requests for lightweight device management features like remote reboot, retrieving recent logs (like the most recent 16KB or so of log messages from RingLogger), and setting key/value pairs to modify the device configuration.
• Would be good to know when devices have not connected for some amount of time - this may not require anything from the protocol - just that the system tracks when a device is connected.

Acceptance Criteria:

• Any user can invite others to create an account with access to the given tenant.
• There is a page where users send these invitations, by simply providing an email address.
• When the form is submitted, an email is sent to the given email address, and an "invitation" record is created, associating the invitation to the tenant of the inviter.
• The invitation email includes a link to create an account. When the user clicks the link, the page shows a form with their email address (read only), optional name field, and a field to create a password (required).
• When the user submits the form, the new user is created. (If a blank password is supplied, a validation error is shown and the form is re-displayed.)

Hi.
So without any concern of how things get done, or what anyone else needs, I've looked the deployments from a birds eye view of how I'd like things to work. I figured I'd jot it down here. Feel free to ignore or dismiss as unworkable lol. It could be a good starting point or at least we can define our use cases by analyzing what this misses.

To me it seems like we are missing something in our current schemas. I'd add a Product and a DeviceGroup (names tbd).

So, a Tenant has multiple Products.

A Product has one or more DeviceGroups.

A DeviceGroup would hold the set of conditions that defined its membership (processor, memory, etc) and have one or more deployments, some active, some inactive, one Deployment which is "current".

A Deployment should have one firmware, possibly some conditions and zero or more devices that have reported running that firmware. I would like to keep this info static, so it could provide a history of what device got what firmware when.

So the paths a product would go on...

Case 1:
Identity DeviceGroup, its criteria is '*' and it has one active and current deployment. Whenever you release a new firwmare you would upload it to the DeviceGroup and mark it as current and active. It would create a new deployment with with that firmware and start notifying whatever Devices are logged in, adding them to the deployment once they have updated. The former current deployment would be marked as inactive and hidden but would still be there. So you'd have a history of what devices got what firmware when, which could be useful.

Case 2:
Still the identity DeviceGroup, but theres now just a happy path, ie all devices get version 1.2.0 except v0.9.0 which needs v1.0.1 first, etc. So theres an still a current deployment with firmware v1.2 but an extra active deployment exists that differentiates by v0.9.0. But you could differentiate on other things too. We could set it up where the order does matter for the non current and active deployments and if they all fail, apply the current one.

Case 3:
For a split in the Product firmware builds we would need to create a second DeviceGroup for the product. To indicate a hardware change most likely, but it could be anything, even a firmware version that could not be upgraded, anything that required a different build of the firmware. Devices logging in would be queried and placed in the current DeviceGroup by any number of criteria and receive the correct deployment. Each DeviceGroup would be on its own as far as receiving new deployments and persisting the devices that have upgraded.

Alpha, Beta, BleedingEdge etc.
It seems like all of the non production kind of labels would really want to be applied at the DeviceGroup level. If you were shipping one beta firmware for DeviceGroup RPI0 and one for DeviceGroup RPI0W you would want to be testing devices for each firmware. So there would be one beta tag, but which firmwware each device gets depends on what DeviceGroup its in. That would be adding an extra active deployment to each DeviceGroup using tags instead of firmware version as the qualifier. These you would want to make sure are applied first.

Anyway thanks for taking the time to read this! Its just an idea and I'm sure there are plenty of necessary use cases it doesn't encompass.

Right now it's impossible, so #38 will never return an upgrade.

References: https://github.com/nerves-hub/nerves_hub/blob/master/docs/client-server.md

Next, the server compares the running version of the firmware to the target version for that device on NervesHub. If those versions are different, then the channel queues a firmware update message to be sent as soon as the channel finishes the join process.

Depends On: #78

Acceptance Criteria:

• A home page exists with email/password fields.
• The user can log in via the homepage login form.

It's useful to know a high level status of the device (like red, yellow, green or failed, warnings, ok). The idea is that once you get enough devices, it's convenient to be able to see at a glance what's happening rather than clicking through to a device detail view.

It seems difficult to combine all interesting device status into one field, so here's a high level idea that has two columns:

• Overall status

  • Red/failed -> offline for > 1 day
  • Yellow/warning -> connected in the past day; an error has been submitted
  • Green/ok -> connected in the past day; no error logs

• Firmware update status

  • Red/failed -> firmware update needed, but errors or retries detected
  • Yellow/warning -> firmware out-of-date. An update will be requested on next connect
  • Green/ok - firmware up-to-date

The firmware update status field would replace the "update pending" column

Acceptance Criteria:

• There is an API endpoint at GET /firmware-update
• When any of the following conditions are met, the endpoint returns a 401 error:
  • The X-Client-Dn header is missing.
  • The X-Client-Dn header is not in the format CN=identifier.
  • The identifier in the X-Client-Dn header is not the case-sensitive identifier of a device in the devices table.
• The endpoint checks whether the device is eligible for a firmware update.
• A device is eligible for a firmware update if a deployment is found where the following is true:
  • The device (in the database) and firmware match in both architecture and platform
  • The device's current_version matches each of the version_requirements of the deployment. (Via Version.match?/3)
  • The device has all tags specified in the deployment.
• Integration tests
• The endpoint returns the following structure:

{
  "eligible_firmware_update": "http://example.com/path/to/firmware/1.0.1/file.fw"
}

(If not eligible for a firmware update, null is returned:)

{
  "eligible_firmware_update": null
}

Notes:

• This implementation does not allow for different tenants to have devices with the same identifier. This is an acknowledged major problem with this first version; we'll need to address it before using the product.

Currently the web app lists all devices per tenant. A tenant likely will have multiple products. It is expected that users will want to focus on one product at a time. Some way of filtering or grouping firmware, devices, and deployments on a per product basis would be good.

Some notes on products:

• The .fw files contain a meta-product tag that has the product name. This could be extracted for to categorize firmware, devices, and deployments.
• The current fwup scripts used with Nerves don't force the meta-product tag to match what's running on devices. This makes it possible to turn one product into another.
• Companies and individuals change product names. It may be useful to have two or more meta-product names that correspond to one "product" in the UI.

Instead of requesting all devices to update simultaneously, it's useful to roll out the updates. This has two benefits:

1. Should an update break a device or cause a support call, it gives time to pause the deployment.
2. It can reduce the pressure on any network bottlenecks since the .fw files are large.

When a firmware file is uploaded, we verify the firmware is valid and then check if it is signed, and if so, match the verify the signature with they key(s) on the tenant.

The way we determine a firmware is signed is no longer valid - update to use fwup's programmatic interface

https://github.com/smartrent/beamware/blob/123db77702a2fc9655ca962a9fd7c2093e34235b/config/config.exs#L14

We don't want just anyone signing up on nerves_hub yet.

Acceptance Criteria

• Only deployments from the user's tenant should be shown
• Show Name, criteria and status of the deployments
• Button to add a new deployment

When on the tenant settings, there should be a remove button or X icon to the side of each key, when clicked it should confirm you want to remove the key and then remove it

Accessed from the Deployment List

• Name
• Set criteria for selecting devices
• Select target firmware

It would be nice if the server could maintain an interaction history with each device. This doesn't have to be fancy. It could just contain things like "device connected at time x", "firmware update requested", "device's firmware version is now y". The idea is to have historical information about a device if a bug report is incomplete or there's an issue with firmware updates reverting.

(stub)

Acceptance Criteria:

• When the user clicks on a device name in the Manage Devices page, they're taken to a page that shows information about the device.
• The page displays a bunch of information about the device.

(For the purpose of eventually limiting the amount of disk space a single tenant's firmwares can take up.)

(Stub)

(stub)

There are a few ways of creating certificates for devices. This issue covers having a screen that lets users enter in a name or ID for a device and then having the server generate (with the help of nerves_hub_ca) a device certificate. The user would then download the certificate and keys and would install them on the device. I think that this is part of the device creation page, but if another part of the UI makes more sense, it could be there too. We will eventually want to allow users to create certificates for devices as a separate step to creating/registering the device with Nerves Hub. However, that separation could be done as part of another issue if it is involved.

Currently, firmware metadata is being parsed for a few key fields, and then the entire metadata string is being stored in a metadata string field on a firmware record.

The current issue, is that it is very common in the real world for the metadata field to be greater than the database imposed limit of 255 characters.

The proposed fix:

• A migration that removes the metadata string field, and adds the following relevant fields supplied by @fhunleth:

meta-product
meta-description
meta-version
meta-author
meta-platform
meta-architecture
meta-vcs-identifier
meta-misc
meta-uuid

• Refactor the firmware_controller and the firmware schema to handle these changes as well.

The device delete and disable buttons are on the main device listing screen at the moment. This is fine for now, but it would be nice to move them to either a device detail screen or have a sequence of "are you sure" screens. The idea is to prevent an accidental delete or disable since those should be rare operations in production and they have big consequences to devices.

Acceptance Criteria:

• The user can navigate to a registration page from the login page, and create an account. (When the account is created, an associated tenant is created.)
• The registration form has email and password fields.
• Registration can be disabled with a config setting.

Acceptance Criteria

• Logged in use should only see firmware from their own tenant
• Show the filename, timestamp, whether or not the firmware is signed and the metadata (extracted from the fwup file)
• Button to upload a new firmware

This issue tracks whether we'll use Bootstrap 3, 4 or Materialize for the initial web framework. While the UI should be straightforward, there is an expectation there will be enough client-side validation and and feedback that we might as well start with one.

Since we're requiring client-side certificates, offering a JIT registration capability seems really convenient. See https://aws.amazon.com/blogs/iot/just-in-time-registration-of-device-certificates-on-aws-iot/ for how AWS handles it.

The idea here is to expose an API for CI servers and other systems to interact with the nerves_hub and be able to do the same things that humans can do via the web UI.

Acceptance Criteria

• User can select a firmware file from their computer and upload
• When uploaded, firmware should be validated (fwup does this?), and metadata extracted
  • If the firmware is not valid, user should be redirected back to the upload page with a message stating it was not valid, and why (if we get that from fwup?)
  • If the firmware is valid, upload the firmware to S3, redirect to firmware list

Need additional detail for S3 upload

Acceptance Criteria:

• The user can click a link on the homepage to go to a Forgot Password form.
• The form simply contains an email address field.
• When the user submits the form, it shows a flash message saying "We've sent you an email with instructions to reset your password" -- even if the email address isn't associated with a user.
• If the email provided is associated with a user, a reset password token is created in the database (associated with that user) and an email is sent to that address.
• When the user clicks on the link, it sends them to a page that checks the validity of the token. If the token is invalid, they're redirected to the homepage. If the token is valid, they're shown a password field.
• When the user fills in the password field and hits "submit", their password is reset and they're taken back to the login page.
• Tokens should expire within 4 days (or more/less? up for discussion)

Create an S3 Firmware Upload module that can be configured to use instead of the Beamware.Firmware.Upload.File module that uploads firmware files to Amazon S3 and generates download links when requested. Should implement both upload_file and download_file like Beamware.Firmware.Upload.File

Acceptance Criteria

• A logged in user can see their email address
• A logged in user can change their email address
• A logged in user can change their password
  • Current password required to change password

References: https://github.com/nerves-hub/nerves_hub/blob/master/docs/client-server.md

The device joins device:#{device_identifier} with a payload of %{"version" => version}. The server first verifies that the device is joining its own channel, by comparing the id in the channel name to the id on the certificate.