We need a specific logging level for logging in usage information.
We have to investigate how it can work with the standard logging messages.

Usage focuses on bindings usage. So just have to log binding information:

{
  "id": "...",
  "start": 0,
  "duration": 1,
  "cancel": false
}

POST /api/interacto/stats

• KeyDataImpl
• KeysDataImpl
• PointDataImpl
• PointsDataImpl
• TouchDataImpl
• TapDataImpl
• MultiTouchDataImpl
• ScrollDataImpl
• SrcTgtPointsDataImpl
• SrcTgtTouchDataImpl
• WidgetDataImpl

Even if the touch DnD must stops on several touches, in a specific case it goes on:
The first touch is on a shape, and the second on the containing canvas.

After an undo, a user can perform an new command. If this new command is equivalent to the redo one, this is not necessary to:

• create a history branch (tree-based history)
• clean the previous timeline (linear history)

Example: sudoku game. Undo a set value command and then perform the same than the undone one.

We can add various methods in SrcTgtPointsData.

• isVertical(pxTolerance) that returns true whether the line between the two points is relatively vertical. pxTolerance stands for the pixel tolerance for considering the line vertical. The methods isStable in Pan.ts will help.
• isHorizontal(pxTolerance) that returns true whether the line between the two points is relatively horizontal
• duration that computes the duration of the move (in milliseconds)
• velocity that computes the velocity of the move distance / duration
• write tests

These methods will be useful to refactor the pan, swipe, etc. interactions.

In some cases we do not what to execute the when all the time. But just at the beginning to check some preconditions.
We may support that by adding an optional parameter to the when routine:

when(fn, once?: boolean)

Since we can cumulate the when routines, we can write several when routines to cover different cases.

For example: with('KeyA') works when the q key is pressed with a French keyboard.
A workaround is to change the prototype of with to allow both code and key.

with(...keys: Array<string>, codes?: Array<string>)

Investigate the possible computation of Fitts' law value during the execution of a binding.
Also see Finger-Fitts' law.

May be integrated within the logging system.

Examples:

• how to add a timeout circle around the cursor for longtouch interactions
• how to add press/tap effects

Should effects be added to user interactions or to binders?

The goal of this class is just to implement by default several methods from Undoable.
This abstract class should extend CommandBase and implements Undoable and implements by default:

• getVisualSnapshot see #7 with undefined
• getUndoName with the name of the class (introspection)

The commands HistoryGoBack and HistoryGoForward are defined in the example-angular app.
We should move it in Interacto with the name UndoNTimes and RedoNTimes.

Get the history related to a given object, instead of having a unique global history

Use cases:

• drawing app: select an object. Its history is extracted from the global one to be then display
• text editor: from a given caret location, the history related to the position (more or less) is extracted from the global one to be then display

This is a specific case of selective undo as undoing the latest command of a given object may mean cherry-picking this command from the global one.

They are various corner scenarios / challenges:

Composite commands

Illustrative example with a drawing app: the user moves 3 shapes. One command that moves those 3 shapes is produced. The user selects one of those three shapes. Its history is displayed. The user undoes the move command => how to undoes only part of the initial command? (part of the solution: composite pattern)

Producing regional histories

How to extract from a global history a history specific to a given object? Part of the solution: an undoable command has a method that returns the associated object. By default returns undefined or [].

First example with the drawing app. The user draws several shapes and then selects one of them. The regional history asks the global one commands which associated object is the data object corresponding to the model of the selected shape.

Second example with a text editor. The user types some text: three successive commands "Supporting" "the" "regional undo".
Then the user selects the "the" text and put it in red. Finally, the user put the caret on the "the" word. The commands related to the given caret position form the regional undo.

Managing regional conflicts

Using the previous example of the text editor, the user undoes the command that produced the "the" word. What to do with the command that put this word in red? Part of the solution: identify command that creates objects, so that cancelling such commands removes the related command from the global history.

Second example using the text example. The user selects the whole text at put it in green. Then the user undoes the creation of the word "the". How to manage the green color command (composite command?)?

Redoing cherry-picked creation command

Using the previous example of the text editor. The user now wants to redo the creation of the word "the" (so that it includes the re-integration of the color command). How to access regional history of removed objects?

TypeScript 4.3 just released:
https://www.typescriptlang.org/docs/handbook/release-notes/typescript-4-3.html

• update TypeDoc npm install typedoc@latest (have to wait for an update of it)
• update eslint typescript npm install @typescript-eslint/eslint-plugin @typescript-eslint/parser
• npm install typescript
• Let mandatory the use of the new override keyword: "noImplicitOverride": true, in tsconfig.json
• Fix all the compiler errors by adding override to methods :s

Implement a new logging system that fits Interacto needs:

• log interaction, binding, command messages, errors
• compute fitts' law on demande
• compute/provide UI usage information (paths, etc.)
• send data to a back-end (so support privacy issues)
• support the bridge pattern to ease testability and customisation of loggers

Will remove typescript-logging dependency

• first
• then
• end
• cancel
• endOrCancel
• ifHadEffects
• ifHadNoEffect
• ifCannotExecute
• catch

https://research.yyoon.net/papers/chi15-bam-Aquamarine.pdf

• check the potential different with the code editor history (#40)
• implement the painting algorithm to be use on angular-example

We can add various methods in MultiTouchData.

• isVertical(pxTolerance) that returns true whether the line of each touch are relatively vertical in the same direction. pxTolerance stands for the pixel tolerance for considering the line vertical. Will use the related methods to add in SrcTgtPointsData
• isHorizontal(pxTolerance) that returns true whether the line of each touch is relatively horizontal in the same direction.

The goal of such a routine is to capture errors in bindings definition for specific processing.
Currently, bindings log errors (exceptions).

If I pause the DnD to let the spring appear, and then a re-move again, the spring is still here. After another pause the spring does not update.

Investigate the possibility for a command to produce a visual or textual snapshot of the undo/redo states.

The idea is to add a novel method in Undoable:

getVisualSnapshot(): string | SVGElement | undefined;

The default implementation of this method should return undefined.

Developers can override this method in their command to, for example, get some information about the impact of the undoable command (more than the mere method getUndoName that just returns the name of the undoable).

Scenario with example-angular: one mouse-over undo buttons (in the undo history panel) to have a thumbnail depicting the impact of the command. Example: a text diff for SetText.

So:

• Add this new method getVisualSnapshot in Undoable (with documentation)
• Implements it by default in `UndoableCommand

For image (to see later):
For canvas, see toDataURL:
https://stackoverflow.com/questions/6887183/how-to-take-screenshot-of-a-div-with-javascript

For other elements:
https://www.npmjs.com/package/html2canvas

Interacto binding must have unique ID to be identified in logging messages.
Those ID must be inferable (eg interaction name + command name) and not randomly generated at run time (which would prevent the clear identification of the bindings at run time).

Writing:

binder
  .toProduce(() => new AnonCmd(() => {
    ...
  }))
  .bind();

is verbose.
Maybe we can shortcut that, for examples :

binder
  .toProduce(() => {
    ...
  }))
  .bind();

where the arrow function is of type () => void so that is detects it is an AnonCmd to produce. Have to investigate ambiguities.

Or:

binder
  .toProduceAnon(() => {
    ...
  }))
  .bind();

Currently a mouse binding does not work using a touch screen.
We should investigate how one can write binders like that:

longTouchOrPressBinder(100)...
touchOrPressBinder()...
tapOrClicksBinder(3)...

https://kilthub.cmu.edu/articles/Supporting_Selective_Undo_in_a_Code_Editor/6470474/files/11899031.pdf

See his github repo for more details on the implementation. Exemple:
https://github.com/yyoon/azurite-eclipse/blob/develop/edu.cmu.scs.azurite/src/edu/cmu/scs/azurite/model/undo/SelectiveUndoEngine.java

In interaction data SrcTgtPointsData we want to provide six new computed properties:

• diffClientX and diffClientY
• diffPageX and diffPageY
• diffScreenX and diffScreenY

For example diffClientX computes tgt.clientX - src.clientX. ie the translation vector.

These new properties must be documented and tested.

Single instances are global variables/constants.
A correct design should rely on dependency injection.

To investigate:

• For Angular, we should create an Angular library that relies on the Angular DI. Create an Angular module.
• Create an Angular lib to demonstrate the ability to use the Angular DI within Interacto.
• For VueJS, look at their DI system.
• For React, look at their DI system.

The single instances / global variables to remove:

• CommandsRegistry
• UndoCollector
• observer in Bindings.ts

This may lead to the encapsulation all the routines located in Bindings.ts in a class injected in, for example, Angular components (similarly to other Angular Modules).

Developers should be able to name their Interacto bindings.
To do so, we can add a dedicated routine name.
To be used to identify bindings in the logging system

If the swipe is defined to use one touch, and the user swipe with two touches, the interaction should be cancelled.

Principle

This allows users to cherry-pick on command to undo in the history. Undoing such a picked command does not undo all the command performed after, i.e. it is not an n-time undo operation.

Selective undo history can be linear or tree-based.

Note that the behavior of the undo algorithm may depend on the semantics of the command.
Some commands of the same type, if executed successively, are cumulative. Example: a move command (drawing app) based on a translation vector (relative position).

Similarly, some commands replace. Examples: a move command based on a absolute position; changing the color of a shape.
With replacing commands, eg two change color commands on the same object (so regional history too), if the user undoes the first change color command, what to do with the next ones? And how to redo this command (and reinsert of related commands)?

Challenges

Conflicts/dependencies between commands

Example 1: drawing app. The user moves two times a shape. Then the user cherry-pick the first move command and undoes it. What happens to the second move command since it depends on the first one?

Touch Draglock is missing

Use the multi-touch interaction to create a predefined pinch binder

Currently the reciprocal DnD can cancel the ongoing interaction and command.
Using the painting history (#41), we may extend the reciprocal DnD to support prior move commands done on the selected shape

MultiTouchData should contain a method pinchFactor(pxTolerance: number): number | undefined

This method will compute the pinch ratio made by a two-touches interaction.
The principle of a pinch:

• the two touches must get closer
• the two touches must follow the same line but in opposite direction (to get closer)
• the ratio is computed as the distance between the end position of the touches / the distance between the starting position of the touches
• if the two touches do not get closer or do not follow the same line, the method will return undefined.

From research paper:
https://hal.archives-ouvertes.fr/hal-01185805/document

The goal is to implement this user interaction either as a novel user interaction class or a specific redefined binder.

See https://inria.hal.science/hal-01714269/file/pn1142-averyA_noCR.pdf

Use the multi-touch interaction to create a predefined pan binder