If you're reading this and you're affected, please subscribe to this issue! A new firmware will be released after fixing this.

So far, about three users have reported that the left half of their keyboard sometimes stops working all of a sudden. (The LED display stays on.) This is already a significant number and I expect it to be higher. Usability-wise this is clearly unacceptable, making this the most pressing firmware issue. Luckily, all evidence suggests that this issue can be fixed on the firmware level.

According to my findings, the KL03 microcontroller of the left keyboard half sometimes halts the I2C communication. Unplugging the left half and replugging it (hence power-cycling it) temporarily solves the issue, proving that the left keyboard half is the culprit. Furthermore, hardware-resetting the KL03 (by shorting the GND and RESET pins) in specific also always fixed the issue for me, which proves that the KL03 is the culprit (and not the left-side LED driver IC).

My planned solution is to implement an I2C watchdog for the left side firmware. (An I2C watchdog for the right half is already in place, and it's critical for reliable communication.)

Interestingly, this issue never occurs on my current UHK, but I have a couple UHKs around, and I'll try to reproduce it to be able to implement and test a fix.

Currently, the Caps Lock LED of the UHK is always off because the USB API of KSDK doesn't expose the state of Caps Lock.

Please simply call LedDisplay_SetIcon(LedDisplayIcon_CapsLock, capsLockStateVariable) to set the LED.

According to my knowledge, keyboard LED state is sent in the set report request of endpoint 0, so I assumed that it resides in the report buffer of this code block. I dumped the value of the buffers of both clauses but none contained keyboard LED state.

• https://msdn.microsoft.com/en-us/library/windows/hardware/hh450799(v=vs.85).aspx
• https://msdn.microsoft.com/en-us/windows/hardware/gg463179.aspx

Make Agent parse the whole configuration. This is not supposed to be a fully-fledged implementation, and it should be simplified in the following manners:

• Apply the default keymap and ignore the rest. Consequently, the keymap switch actions will still be ignored.
• Don't parse macros. When testing, delete every macro in Agent and don't care if the parser fails when adding some. Macros are low priority, and I'd like to get some other issues implemented before macros.

The purpose of the macro engine is to index and play back macros.

Indexing is done while parsing the user configuration in non-dry-run mode. The offsets of macros within the user configuration should be put into a uint16 macroIndexes[1024] array.

When playing back a macro, the engine should look up its offset in the above array. Then it should play back the macro actions of the current macro one by one until reaching the last macro action of the macro at which point it should stop macro playback.

On the USB level, the macro engine should be hooked to UpdateActiveUsbReports() where it can make the UHK emit basic keystrokes, media keystrokes, system keystrokes and mouse actions.

The various macro action types should be fairly self-explanatory. In order to implement the macro delay action the FlexTimer (FTM) peripheral of the MCU will be used. I have yet to configure the FTM module which I'll do soon. Please let's leave the delay action as the last to implement.

Hitting Esc should terminate macro playback.

The layer switch locks on double press of layer switch key are a nice feature (clearer documention of it might be nice at some point), but I've found they also can be a little surprising. I would suggest that the lock should be activated only on a double quick press, but if the second press lasts more than ~.5s then assume the user was careless about initially pressing the key (or something went wrong with switch debouncing) and release the layer when the layer switch key is released.

Currently, I2C communication stalls when the I2C bus breaks between the I2C master (right keyboard half) and slaves (left keyboard half or add-on modules). This happens when the keyboard halves get split or an add-on module gets disconnected. I2C communication must always stay functional in such cases, and recover when connecting a slave.

The fix involves making I2cRead() and I2cWrite() more resilient. There is an alternative, more resilient implementation of these functions that can be activated by defining the ROBI_I2C macro. Unfortunately, this alternative implementation make communication unreliable (the blink-test-led.js script doesn't blink the test LED of the left keyboard half), so it should be fixed and used in place of the current implementation.

The current mouse interface is fairly advanced featuring horizontal + vertical scrolling. The problem is that it doesn't seem to be handled by BIOSes. Let's also expose a basic BIOS compatible mouse and fall back to it if needed the same way as for #70.

Read the voltage level via ADC, expose the measurement via USB, and set LED brightness accordingly.

OSX by default does inverted scrolling. (Since OSX Lion)

The Y and H scrolling feel wrong to me. I think if you have selected QWM, it should default to inverted scrolling on Mac.

Two ways to querry the OS setting for scroll

https://stackoverflow.com/questions/30767674/change-scroll-direction-on-mac-programatically-assign-to-key-combination-comman

https://stackoverflow.com/questions/12433018/how-to-check-scrolling-direction-of-osx-with-cocoa-apis

On Windows {7,8,10} consumer codes are being repeated (quickly toggled) when relevant keys are pressed. I encountered similar issues earlier and though I fixed them all, but apparently not. I'll look into how Windows sends HID reports, and adjust UpdateUsbReports() accordingly.

In order to make the firmware handle keymap switch actions, a number of changes must be made.

First and foremost, a staging area should be created for newly transferred configurations. This would involve a config_buffer_t StagingBuffer variable. Agent will transfer the new configuration to this variable, then the parser will validate the configuration of the staging area. Agent will then apply the configuration, at which point the firmware copies StagingBuffer to ConfigBuffer.

Admittedly, StagingBuffer takes a lot of RAM, but we can afford it, and this is the only way for the firmware to properly validate the new configuration without overwriting it. ConfigBuffer will be actively used during the operation of the UHK to switch keymaps, play back macros, and who knows for what else, so we cannot just overwrite it with a possibly invalid configuration.

Make sure to validate the keymap ids of switch keymap actions.

The active keymap must be remembered when a new configuration gets applied. The 3 letter abbreviation must be remembered because the index of the keymap can change in the new configuration. If the active keymap no longer exists in the new configuration then the default keymap must be activated.

I've been pondering about macros recently, and the best way to implement them. With other firmware I worked on, they were pretty darn simple, because they are compiled into the firmware itself. With the UHK, they're not, as they can come through the Agent. This presents some non-trivial problems:

• We can't reasonably pre-allocate memory for the macros, because that'd impose serious limits on what one can do. Not to mention that the limit may well be off, by a lot, and in a lot of cases, be terribly wasteful.
• We do not want to dynamically allocate memory, even if we could, because it would become very hard to predict how much memory we actually have to allocate from.

So, I thought: what if we didn't copy macros from EEPROM to RAM, like we do for the keymaps? What if we re-played macros from EEPROM directly? We'd still need a lookup table in RAM, that tells us which macro is where, so we would have to parse only a fraction of the config (or we could have a table of contents in EEPROM too, supplied by the Agent, to make this even easier on the firmware), but the macro would be replayed from EEPROM. While EEPROM access may be slower than RAM, in case of macros, I do not think that matters much - delays will have to be introduced anyway, otherwise playback would be faster than what the host is ready to handle.

So my proposal is that I'll extend the config parser to also parse macros, and store their location in a pre-allocated array (allowing up to, say, 64 macros). Then implement a macro-playback method on top of this.

What do you think?

55e4997 fixed the issue that caused the mouse pointer to stuck to the top left corner on OSX, but apparently, it also made Windows 7 Ultimate not install a default driver for the system keyboard interface, making every other USB interface useless, too.

The old and new system descriptors will have to be combined in some way to make both OSes happy.

I'd be useful to say a couple of words about clang-format in CONTRIBUTING.md, so that our awesome contributors would know what it is and how to use it from the command line and IDEs. I'm thinking about something short, and linking relevant resources.

When a dual-role key is held whose secondary role is layer switching, and another key gets pressed that has a keymap switch action associated to it (on the relevant layer according to the secondary role), the keymap doesn't get switched first, only when pressed the second time while the dual-role key is held.

The configuration has to be transferred between Agent and the UHK.

The writing of the configuration will happen as follows:

1. Agent sends the configuration to the UHK over USB in a series of 64 byte packets. The firmware saves these chunks in the RAM of the UHK.
2. Agent sends a "validate configuration" command to the UHK. The UHK validates the configuration by executing the configuration parser in dry run mode.
3. If the configuration parser succeeded then Agent sends a "save configuration" command to the UHK which will persist the validated binary configuration from its RAM to the EEPROM.

The reading of the configuration will happen as follows:

1. Agent reads the configuration from the UHK over USB in a series of 64 byte packets.
2. Agent validates the binary configuration and proceeds further if it's valid.

I'll implement the firmware part of the above process and write up some crude JavaScript scripts to demonstrate its use. I'll implement the "validate configuration" part as a dummy command because the configuration parser will only be available later.

Sometimes the factory keymap gets activated instead of the default keymap right after firmware updates.

Robi reported that according to his experience this always happened on Mac after firmware updates.

Double-tapping a layer switcher key locks the layer until its key is pressed again.

However, if a layer switcher key is tapped, then double-tapped, then it won't register. In this case an extra tap is needed, and then it works again.

Currently, KeyMatrix_Scan() is used to scan the keyboard matrices which is suboptimal because it utilizes a much needed busy loop, so we're wasting precious cycles.

As an alternative solution, I implemented KeyMatrix_ScanRow() and called it from a PIT timer interrupt. On Linux Mint for some reason this causes the mouse pointer to go really slow and jump hundreds of pixels from time to time. Weirdly, it works fine on Windows 7 and Android.

By commenting out INTERRUPT_KEY_SCANNER, one can test the my interrupt based implementation.

Ultimately, we should use the interrupt based implementation.

Currently, we have several blocks like the following across the codebase:

    i2c_master_transfer_t masterXfer;
    masterXfer.slaveAddress = I2C_ADDRESS_EEPROM;
    masterXfer.direction = kI2C_Write;
    masterXfer.subaddress = 0;
    masterXfer.subaddressSize = 0;
    masterXfer.data = GenericHidInBuffer+2;
    masterXfer.dataSize = 2;
    masterXfer.flags = kI2C_TransferDefaultFlag;
    I2C_MasterTransferBlocking(I2C_EEPROM_BUS_BASEADDR, &masterXfer);

These should be replaced by I2cRead() or I2cWrite() calls to minimize redundancy.

the contents of left/build/armgcc/CMakeLists.txt seem to be out of date.

could those be updated? i'm trying to add travis, but i need to use the armgcc build system instead of the kds system

It would be great to implement a means for users to sign the firmware to prevent unwanted and potentially malicious firmware flashes when plugging the keyboard into unknown hosts. This would also require a means of doing a hardware based reflash of the public key stored on the keyboard or of the entire firmware as it would be pretty bad to loose your private key and have a bricked keyboard. I don't see a means of hardware re-flash as a security bug because if someone malicious has that much physical access to your keyboard then it's no longer your keyboard anyways ;)

While this is mostly handled by "multiple security modes according to https://ultimatehackingkeyboard.com/blog/2015/02/26/securing-firmware-upgrades"

I can still see benefits to the signed firmware model, mainly mass production and the fact that unless you use the same password for the security mode in the keyboard as you do for a common key (maybe not the most secure), It seems likely to forget the password.

Probably as a side note this issue can probably also benefit the libreboot project:
https://libreboot.org/

I'll contact libreboot to see if they have any information on the matter but it may become an issue of me or someone creating a seperate repository, or adapting an existing one, for a firmware module that both and other projects can share. I know there are proprietary implementations of signed firmware on small firmware on devices such as kanguru USB drives so it should be possible but it will probably take quite a bit of time.

At this time though, this issue should probably be pretty low priority since the above security modes handle the issue pretty well and make signed firmware more of a "nice to have" than anything.

Playing around with a prototype, the blink-test-led.js script from the agent only blinks the right LED on master. After bisecting the problem, I narrowed the issue down to commit 1d2f35d. I tried looking into what the problem may be, but this stuff is closer to the hardware than what I'm capable of debugging on my own.

So I'm reporting it here, so the issue isn't forgotten.

The current UHK firmware doesn't seem to play nice with Windows for some reason.

Sometimes, it doesn't enumerate at all on Windows 7 Ultimate SP1 32-bit and Windows 8 Professional x64. (No other Windows versions have been tested yet.)

The configuration parser is akin to the configuration serializer but it:

1. is implemented in the firmware
2. only reads the configuration
3. interacts with the various parts of the firmware to actualize relevant data structures

It'd be practical to proceed in a bottom-top manner when implementing the parser. What I mean by that is that first the parser could only parse a single layer, then a keymap, then multiple keymaps, then the root of the configuration. This would allow to gradually extend it while having something working all the time.

And then we could iterate slots at the top level.

There are 4 layers per keymap. The base layer is active by default when no layer switcher key is pressed. There are also the Mod, Mouse, and Fn layers activated by the Mod, Mouse, and Fn keys respectively.

When multiple layer switcher keys are pressed the first one dominates. I mean the one that gets pressed before the others. This allows us to switch to the Mouse layer by holding the Mouse key pressed, then also hold the Mod key to accelerate the movement of the pointer according to the factory keymap.

The sleep key is only working on Linux. It's not working on any other OSes. The system keyboard USB report doesn't seem to be right. Fixing that should probably fix this issue.

Santiago's busy loop I2C recovery implementation works as intended, and it's a solid workaround, but it is a workaround. We should not have workarounds in the codebase in the long run, so eventually, this should be fixed in the KSDK codebase.

There was a mention of an "Adapting UHK Project to KBOOT.docx" file containing the relevant instructions.

When compliling the firmware not in debug mode but in production mode, the compiler optimizes the busy loop that is responsible for I2C recovery, making it not work. This should be fixed.

When checking the USB interfaces of the UHK in the device manager, one interface has an error message saying:

This device cannot start (Code 10).
Report Descriptor was not byte aligned.

The UHK should be able to detect reboot (possibly a USB stall state?) and then Usb*ReportEverSent should be set to false.

Right now, every LED of the LED display is enabled all the time, so it's time to make it do something useful.

You will have to manipulate the bytes of ledDriverStates[keyboardHalf].ledValues[ledId] which represent the brightness levels of the individual LEDs, including the per-key backlighting and the LEDs of the LED display. For the time being, let's just use 0 (disabled) 0xFF (enabled) values for the LEDs. We will make the brightness adjustable later.

ledDriverStates[LedDriverId_Left] contains 144 LEDs arranged into two matrices: matrix A and matrix B. Matrix A drives the per-key LEDs and matrix B drives the LEDs of the LED display.

The memory regions of matrix A and matrix B are interspersed according to page 11 of the 31FL3731 LED driver datasheet.

Also consider the schematic of the LED display to see how the individual LEDs of the LED display are mapped to the matrix.

The content of led_display.[ch] is totally obsolete, so you're welcome to completely ditch it and implement the following API:

• LedDisplay_SetText(char* text) expects a 3 character ASCII string and displays the characters on the LED display. Only capital letters, numbers, and space must be displayed. In order to see which segments to use for which character check out this image.
• LedDisplay_SetLayer(uint8_t layerId) sets the LED of the relevant layer. All layer LEDs are disabled for the base layer, and only the relevant layer LED is enabled for any other layer. See layer.h and please make an enum out of these values.
• LedDisplay_SetIcon(led_display_icon_t icon, bool isEnabled) enables/disables the icon of LedDisplayIcon_{CapsLock, Agent, AdaptiveMode} respectively.

Currently, the configuration parser is only able to parse layers. As the next step, it should parse keymaps (which are composed of 4 layers). Please see right/src/config/parse_keymap.c for the configuration parser and also the shared/src/config-serializer directory of Agent for the serialization format.

When using Agent, ctrl+u sends the current layer to the UHK. Similarly, ctrl+i sends the current keymap and ctrl+o sends the whole configuration. Press ctrl+shift+i in Agent to invoke developer tools, then click on the Console tab to see the USB packages that get sent to the UHK.

Upon finishing the implementation, the ParseLayer function call of usb_protocol_handler.c should be replaced with the newly created ParseKeymap function call.

Currently, the default keymap is a 2 dimensional array of key actions. Robi implemented it a while ago which was a step forward but our modular architecture and configuration structure calls for a different hierarchy.

A keymap should be composed of 4 layers. A layer should be composed of a set of modules. A module should be composed of an array of key actions.

The array should be declared as

uhk_key_t currentKeymap[LAYER_COUNT][SLOT_COUNT][MAX_KEY_COUNT_PER_MODULE]

and it shouldn't be const because it will be changed runtime.

@algernon It would be awesome if you could refactor the current code according to the above because the more we wait, the more refactoring we'll have to do later down the road.

Currently the firmware exposes a basic 6KRO keyboard USB interface. We'll expose an NKRO keyboard interface, but its compatibility will be limited (no BIOS support) so it'd probably be better to to leave the 6KRO keyboard USB interface instead of replacing it, and add an NKRO interface, too. Then the firmware would detect which interface is polled. If the NKRO interface is polled, then use that. Otherwise fall back to the 6KRO interface.

I observed that the mouse interface often stops working on Linux (and possibly on other OSes) after Agent communicates with the UHK. This usually happens after saving the configuration by pressing the "Save to keyboard" button of Agent, but it also happens rarely after any generic HID communication.

Oddly, sometimes I can reproduce this easily for any number of times in a row, but at other times I cannot reproduce this at all.

What happens is that UsbMouseCallback() is not called anymore. I tried to find the source of this (see 9481420) by adding counter variables at various relevant places, and it seems that even USB_DeviceKhciInterruptTokenDone() doesn't get triggered, which is a very low level USB function that indirectly calls UsbMouseCallback().

Then I fired up Wireshark and noticed that according to the captured USB packages the host stops polling the device after pressing the "Save the keyboard button.

Currently, Agent doesn't send USB packages completely synchronously to the UHK. The relevant Agent issue will be resolved soon. Hopefully, it'll fix this one. If not, a minimal test case will be written.

• USB HID Usage Tables document, Consumer page on page 75
• Useful info at https://github.com/NicoHood/HID/wiki

According to Jan:

the default function is NMI_b. You will need to disable NMI in the flashed image, otherwise the chip won't boot sometimes. Found out the hard way. From my notes: "NMI functionality is active by default. To disable it, I had to set the FOPT byte in the flash configuration section of the binary. This byte is then loaded on reset and the system is configured accordingly. Flash configuration can be found in the startup_SYSTEMNAME.S file. The previous-to-last byte is the FOPT register, and setting it to 0xFB instead of 0xFF (0xFF-0x04, to disable bit 2) disables NMI.
Layout of the flash configurations is described in "Flash Configuration Field Description" in the FTFA section of the RM."

This should be done for the firmware of every module.

Also gotta check whether KBOOT is affected.

Currently, BusPal partially works. For example, the get-property command works as expected:

$ blhost --usb 0x1d50,0x6121 --buspal i2c -d get-property 1
Inject pre-config command 'i2c'
[01 00 0c 00 c1 00 00 02 10 00 00 00 64 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00]
<03 00 0c 00 a0 00 00 02 00 00 00 00 c1 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00>
Successful generic response to command 'i2c'
  - took 0.000 seconds
Response status = 0 (0x0) Success.
Inject command 'get-property'
[01 00 0c 00 07 00 00 02 01 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00]
<03 00 0c 00 a7 00 00 02 00 00 00 00 00 00 01 4b 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00>
Successful response to command 'get-property(current-version)'
  - took 0.000 seconds
Response status = 0 (0x0) Success.
Response word 1 = 1258356736 (0x4b010000)
Current Version = K1.0.0

Unfortunately, the flash-image command aborts:

$ blhost --usb 0x1d50,0x6121 --buspal i2c -d flash-image uhk-left.srec 
Inject pre-config command 'i2c'
[01 00 0c 00 c1 00 00 02 10 00 00 00 64 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00]
<03 00 0c 00 a0 00 00 02 00 00 00 00 c1 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00>
Successful generic response to command 'i2c'
  - took 0.000 seconds
Response status = 0 (0x0) Success.
Inject command 'flash-image'
Wrote 192 bytes to address 0
[01 00 0c 00 04 01 00 02 00 00 00 00 c0 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00]
<03 00 0c 00 a0 00 0c 02 00 00 00 00 04 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00>
Successful generic response to command 'write-memory'
[02 00 20 00 00 06 00 20 e9 04 00 00 45 05 00 00 49 05 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00]
(1/3)16%[02 00 20 00 00 00 00 00 00 00 00 00 00 00 00 00 4d 05 00 00 00 00 00 00 00 00 00 00 51 05 00 00 55 05 00 00]
(1/3)33%[02 00 20 00 41 05 00 00 41 05 00 00 41 05 00 00 41 05 00 00 41 05 00 00 41 05 00 00 41 05 00 00 41 05 00 00]
(1/3)50%[02 00 20 00 59 05 00 00 41 05 00 00 5d 05 00 00 41 05 00 00 61 05 00 00 41 05 00 00 41 05 00 00 41 05 00 00]
(1/3)66%[02 00 20 00 41 05 00 00 41 05 00 00 41 05 00 00 41 05 00 00 41 05 00 00 41 05 00 00 41 05 00 00 41 05 00 00]
(1/3)83%[02 00 20 00 41 05 00 00 41 05 00 00 41 05 00 00 41 05 00 00 41 05 00 00 41 05 00 00 41 05 00 00 41 05 00 00]
(1/3)100% Completed!
<03 00 0c 00 a0 00 00 02 01 00 00 00 04 00 00 00 49 05 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00>
  - took 0.001 seconds
Response status = 1 (0x1) Failure.

Strictly speaking, BusPal may be flawless, but errata ID 806 is probably the culprit, so it should be worked around by BusPal. Please read more about the erratic behavior in a related NXP forum post.

I've created the the buspal-fix branch to easily reproduce the above issue. My commit of this branch merely sets the timeout of the left bootloader to the maximum 65 seconds and enumerates the right keyboard half as the BusPal proxy over USB.

The objective is to make the flash-image command work, so that we'll be able to update the firmware of the left keyboard and the add-on modules via the right keyboard half.

I2C messaging isn't very robust right now. Electromagnetic noise and the mounting/unmounting of I2C slaves can cause messages to be damaged. We should use CRC to check message integrity.

We should use the CRC functions of the KSDK because they're highly optimized and readily available. We will have to choose the most suitable CRC function for our needs.

Currently, when executing the "uhk-right debug blhost" or "uhk-right release blhost" run configuration to upgrade the firmware of the right keyboard half blhost-unix.sh gets executed which likely only works on Linux. It'd be nice to port this script Mac and create relevant build configurations by cloning the existing configurations, using the ported script for them, and renaming them to "uhk-right {debug,release} blhost mac".

This is not a critical issue but rather a good to have that would streamline firmware development on the Mac.

For some reason, when calling InitTestLed() before LedPwm_Init(), USB enumeration gets delayed very much.

First up, it should be tested whether this can be reproduced not only in release mode but in debug mode, too. I suspect that the compiler optimizes something it shouldn't.

Add a dummy macro parser to the configuration parser. As the name suggests, please to throw the actual macros, just parse them. This way, the parser will be able to deal with the whole configuration that Agent sends with macros included. Let's deal with the macro engine later as it's lower priority.