Support uploading sketches over WebUSB about arduino HOT 17 OPEN

I think @PaintedBlck's confusion stems from the fact that there are many ways to program the application processor in an Arduino:

1. If the board uses a 32u4 or similar then the application processor is the only chip on the board and is responsible for running your sketch and communicating over the USB port. When the host sends the right signal over the USB port the chip puts itself in programming mode, receives the new firmware and burns it to flash.
2. If the board uses a 328P or similar then there is an additional chip between the application processor (running your sketch) and the USB port. On early Arduinos (and many other embedded devices) this is a USB-to-serial converter chip from a company like FTDI or Prolific. Modern Arduinos like the UNO use a 8u2 but the function is the same. The host drivers make it look like chip is a serial port with the application processor connected to it. The application processor never sees the USB traffic directly, only serial data on its TX/RX lines. By twiddling the serial port state the host can put the application processor into programming mode after which it will receive the new firmware over it's internally wired TX/RX pins and burn it to flash.
3. With either of these kinds of boards you can attach a "programmer" to a debug connector on the board. This is a completely separate interface specifically designed for programming and debugging (think JTAG). How the host communicates with the programmer varies. Sometimes it's a custom USB protocol. Sometimes it emulates a serial port.

What I was talking about in my earlier comments was adapting the firmware on the Arduino boards of type (1) so that they understand how to receive firmware from the USB interface that this library adds. Right now they only support receiving firmware on their first USB interface, which is a virtual serial port and inaccessible to the WebUSB API.

What @noopkat did was use a programmer as described in (3). I think the one they are using is one that speaks a custom USB protocol instead of presenting a virtual serial port because you can't connect to USB serial ports with WebUSB (the kernel driver claims the interface first).

That leaves the boards in configuration (2). To make these work you need to modify the firmware running on that chip between the USB port and the processor receiving and running your sketch. This modification would add an interface which is will not be claimed by the kernel USB serial port driver and so can be accessed by a page using WebUSB. Since on modern Arduino's that's just a 8u2 this is possible. If you look at an Arduino UNO you'll notice there are 2 3x2 jumper connectors on the board. One of those is the ISP (in-circuit programming) header for the application processor (what @noopkat used) and the other is the ISP header for the chip acting as USB-to-serial converter. Using this header and an appropriate programmer would allow you to change the firmware of that converter as much as necessary.

from arduino.

@PaintedBlck thanks for sharing this!

But I feel I need to clarify a few points about my talk and avrgirl:

1. I'm not directly uploading the program to the Arduino via the regular USB cable route. I'm using a programmer device sitting in between the Arduino and the computer. This device does not use serial and is fully compatible with WebUSB. The programmer is the AVR ISP mkII, which is plugged into the ICSP pin outs on the Arduino board. This bypasses the Arduino serial path and is not a traditional 'upload' using the existing Arduino bootloader. I'm using a modified version of avrgirl-ispmkii in order to do this.

2. The Chrome app you link is using serial to upload, which is not WebUSB. Therefore the repo would not be super relevant to this discussion.

There still isn't a straightforward way to plug an Arduino in via regular USB and upload a program to it because of needed adjustments to the Arduino bootloader as @reillyeon explained. I think I just made it look that way in my talk and I'm sorry for the confusion! I was just really excited to show some demos with an actual WebUSB supported device instead of 'simulated' WebUSB 😄

from arduino.

For instance, you could add the ability of being a HID device (like Leonardo) to regular UNO boards with this specific firmware: https://github.com/NicoHood/HID

from arduino.

@PsiOps board with dedicated USB to Serial chip like CH340, FTDI and etc will not be able to run WebUSB.

If you want flashing over WebUSB, check out https://github.com/devanlai/webdfu

from arduino.

How does the WebUSB process differs from other, JS-bound upload processes (like that of avrgirl-arduino)?
That one uses node-serial to enter/restart in bootloader mode (opening the serialport at 1200 baud) and then uses the avr protocol to flash the chip.

@reillyeon could you please help me by elaborating on why the "once a sketch with support for WebUSB is uploaded" prerequisite is there? Thank you!

from arduino.

The WebUSB library in this repo adds an additional interface on which WebUSB communication happens. A site using WebUSB will not have access to the serial interface on which the upload normally happens.

from arduino.

Thanks for the answer Reilly, I did my fair reading on the spec in the meantime. So this "uploading" would practically happen through some kind of custom code uploaded to the device, capable of "self-modification", based on incoming data?

from arduino.

No, the goal would be to upload a new image to the chip's normal flash memory through the bootloader. I have not yet done an investigation into what modifications to the bootloader will be necessary to make that possible.

from arduino.

Hi @reillyeon, are there any updates on this request?

from arduino.

I have found the following presentation of @noopkat : I'm afraid your browser has been talking to the robots again a gentle intro to WebUSB where she shows (27:30) that she can upload a sketch to an arduino. She also shows what tools she uses (29:47).

In short, she uses avr-girl variants. You can also find some inspiration on noopkat/avrgirl-chrome-app.

I hope this helps!

Grtz,
PaintedBlck

from arduino.

Thanks for the hint :)

from arduino.

Hm :( Are there any other plans to follow up on this @reillyeon?

from arduino.

@noopkat Good presentations must be shared with the world 😄

I'm not following the reasoning of @reillyeon or not understanding it (probably the last is the most credible one). So I'd like to have a better understanding of this.

I understand it to work as follows (referencing this Arduino Uno schematic): Arduino IDE (avrdude) transfers the hex/bin files through the USB to the Atmega8U2 who translates the communication protocol from the used protocol/voltage levels on the USB line (D-/D+) to serial (Rx/Tx, no CTS, no RTS, other voltage level). On the Atmega328P, the standard boot-loader listens on the Rx/Tx lines, takes the received file, and stores it in flash (above the boot-loader memory line).

Starting from that principle, if we know the protocol used between avrdude en the Atmega8U2, we're in business, since we can send and receive the same byte sequences from a javascript program to the USB port and hence to the default program running in the Atmega8U2.

To my understanding, the communication between avrdude and the boot-loader on the Atmega328p can be seen as a one-to-one dataflow where the Atmega8U2 is merely an underlaying protocol convertor, but not a content convertor.

If I understand @reillyeon, he is saying that the program in the Atmega8U2 or the boot-loader in the Atmega328p has to be adapted, and I don't understand why.

I would understand it, if it was the goal to flash a program to the Atmega328p without using a default boot-loader present in the chip (as Atmel Studio does it). I would also understand it if the Atmega328 were overclocked and running at a different speed than the Atmega8U2. But neither are the case.

So I feel kind of lost in my own reasoning...

My question is (trying to understand): is webusb more than a simple communications protocol in the sense that it requires specific hardware and communications from that hardware? I am testing webusb with an Arduino Uno and I see the device, and can get meta-information from it (struggling with the claiming for the moment).

from arduino.

@reillyeon Thank you for your clarification. Much appreciated!

from arduino.

@reillyeon thanks a lot for your analysis. Is there any progress on this functionality? I cannot seem to find a working implementation of this.

To make sure I understand: So modifying the bootloader would have no effect on devices with a separate chip for usb<->serial? Why is that exactly? I have a clone Arduino device that uses a ch340 chip for this, and there is no header for programming that on the board. Does that make it impossible to flash over WebUSB?

from arduino.

If you're looking for web-based flashing of devices over serial I recommend following @noopkat's work on porting noopkat/avrgirl-arduino to the Serial API.

from arduino.

current working branch of Serial API compatibility for avrgirl-arduino: https://github.com/noopkat/avrgirl-arduino/tree/chrome-serial

Currently Arduino Uno works, and Arduino Mega almost works. Any 32u4 based Arduino board will bring some complications that I'm still thinking through and will almost certainly require more major refactoring on my side. But if you'd like to follow progress, the branch mentioned above is where it's at.

from arduino.