Giter Club home page Giter Club logo

canvastools-for-vott's Introduction

CanvasTools library for VoTT

CanvasTools is one of the UI modules used in the VoTT project. The library implements the following core features:

  • Region (box, point, polyline & polygon) selection & manipulation
  • Filters pipeline for underlaying canvas element
  • Toolbar for all available tools

Dependencies

  • CanvasTools heavily uses the Snap.Svg library. In the webpack-eged version it is bundled with CanvasTools into one ct.js file, including also styles.
  • Current version of the library depends on some features (e.g., masks-support in SVG) that are not fully cross-browser, but are targeting Electron (Chromium).

How to use

Install npm package

Install package from npm:

npm i vott-ct

The package structure:

dist/
    ct.d.ts -- bundled typings
    ct.dev.js -- webpack bundle for development (incl source map)
    ct.js -- webpack bundle for production ({tsc->commonjs, snapsvg, styles} -> umd)
    ct.js.map -- source map for ct.js
    ct.min.js -- webpack minimized bundle for production
    ct.min.js.map -- source map for ct.min.js
lib/
    css/
        canvastools.css
    icons/
        {*.png, *.svg} - collection of icons for toolbar and cursor
    js/
        ct.d.ts -- typings generated by tcs
        ct.js -- AMD module generated by tcs
        ct.js.map -- map file generated by tcs
        snapsvg-cjs.d.ts -- typings for the snapsvg-cjs package
        CanvasTools/
            {*.js, *.d.ts} -- compilied js and typings files

Add library to the app

  1. Add the ct.js file to your web-app (e.g., an Electron-based app).

    <script src="ct.js"></script>
    <!-- OR -->
    <script src="ct.min.js"></script>
  2. Copy toolbar icons from the src folder to your project.

Add Editor to the page

  1. Add container elements to host SVG elements for the toolbar and the editor.

    <div id="canvasToolsDiv">
        <div id="toolbarDiv"></div>
        <div id="selectionDiv">
            <div id="editorDiv"></div>
        </div>
    </div>
  2. Initiate the Editor-object from the CanvasTools.

    var editorContainer = document.getElementById("editorDiv");
    var toolbarContainer = document.getElementById("toolbarDiv");
    
    var editor = new CanvasTools.Editor(editorContainer).api;
    editor.addToolbar(toolbarContainer, CanvasTools.Editor.FullToolbarSet, "./images/icons/");

    The editor will auto-adjust to available space in provided container block. FullToolbarSet icons set is used by default and exposes all available tools. The RectToolbarSet set contains only box-creation tools. Correct the path to toolbar icons based on the structure of your project.

Add callbacks to the Editor

  1. Add a callback for the onSelectionEnd event to define what should happen when a new region is selected (created). Usually at the end of processing the new regionData you also want to add it to the screen with some tags applyed. Use the addRegion method for that.

    // Create some ID for regions
    let incrementalRegionID = 100;
    
    // Set callback for onSelectionEnd
    editor.onSelectionEnd = (regionData) => {
        let id = (incrementalRegionID++).toString();
        let tags = getTagsDescriptor();            
        editor.addRegion(id, regionData, tags);
    };        
    
    const Color = CanvasTools.Core.Colors.Color;
    const LABColor = CanvasTools.Core.Colors.LABColor;
    const Tag = CanvasTools.Core.Tag;
    const TagsDescriptor = CanvasTools.Core.TagsDescriptor;
    
    // Generate tags
    function getTagsDescriptor() {
        // Use the Color class to specify color
        let primaryTag = new Tag("Awesome", new Color("#c48de7"));
        // Use a string color to specify color
        let secondaryTag = new Tag("Yes", "#f94c48");
        // Use one of the color spaces classes (e.g., LABColor) to specify color
        let ternaryTag = new Tag("one", new Color(new LABColor(0.62, 0.50, -0.55)));
        return new TagsDescriptor(primaryTag, [secondaryTag, ternaryTag]);
    }
  2. Add a callback for the onRegionMove event to track region changes.

    editor.onRegionMove = (id, regionData) => {
        console.log(`Moved ${id}: {${regionData.x}, ${regionData.y}} x {${regionData.width}, ${regionData.height}}`);
    };

Update background

Once the background image for tagging task is loaded (or a video element is ready, or a canvas element is created), pass it to the editor as a new content source.

let imagePath = "./../images/background-forest-v.jpg";
let image = new Image();
image.addEventListener("load", (e) => {
    editor.addContentSource(e.target);
});
image.src = imagePath;

Examples

Samples on the usage of the library are located at samples folder. To view the sample CanvasTools UI module, load the index.html present in one of the samples folder. Eg. samples/editor/index.html folder

Changelog

Moved to a new file: CHANGELOG.md.

canvastools-for-vott's People

Contributors

code-vicar avatar fathima1994 avatar harshupadhyay-msft avatar hawestra avatar kichinsky avatar microsoft-github-policy-service[bot] avatar pritamdas9191 avatar samclee avatar soyfestivo avatar yashvkamat avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

canvastools-for-vott's Issues

Add accessibility checks and fixes

Review the code base from the accessibility point of view:

  • Fix accessibility issues.
  • Setup a process for accessibility checks in CI/CD.

Dragging issues when switching selection modes

  1. In some cases, dragging doesn't work after locking/unlocking and switching mode. [Exploration is required.]
  2. In the locked state template and copy modes allow dragging, but it should be disabled.

Add AI-driven box selection mode

Idea:

  1. use tagged frames to tune some pre-trained model,
  2. use that model to predict boxes on next frame based on the area of interest click

P.S. It differs from active learning, predicting boxes and labels for the whole frame, as the goal of new tools is to simplify only box selection.

Image manipulation at editing phase (zooming, rotation)

Zoom in/out .

  • Scaling engine for both the underlaying image (in canvas element) and the collection of regions.
  • UI for scaling (visual + shortcuts)

Rotation

  • Rotation engine for both the underlaying image (in canvas element) and the collection of regions.
  • UI for rotation (visual + shortcuts)

Check whether it is possible to achieve it with css transform or something similar.

Rewrite css-templates using string template tags for better readability

Current implementation with hard-coded conversions (see TagsElement and TagsComponent classes) containes hardcoded conversions like:

// TagsElement
{
   rule: `.${this.styleId} .primaryTagPointStyle`,
   style: `fill: ${tags.primary.colorAccent};`,
},

and

// TagsComponent
rule = this.styleSheet.insertRule(`${r.rule}{${r.style}}`, 0);

It seems like such code can be optimized with string template tags: https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Template_literals#Tagged_templates

Add "none" selection-mode

Add a new "selection" mode, for regions manipulation with all selectors disabled (something opposite to freezing regionsmanager).

Advanced toolbar for managing modes, pipelines, etc.

Implement an advanced version of Toolbar allowing editing, pipelines, etc.

  • Group similar actions under one toolbar slot (icon set)
  • Editing pipeline (of filters)

Optional:

  • Separators
  • Dragging toolbar

Questions to explore:

  1. Does it make sense to continue implementing it using SVG? (e.g. SVG viewport is a limitation).
  2. What if we will wrap toolbar into a react-component?

integration with angular

Hello,

Trying to import your module but keep getting type issues.
error TS2503: Cannot find namespace 'Snap'.

Any idea? Already tried to load in the @types/snapsvg. But doesn't work.

Thanks!
Cédric

New selection mode: polyline

Polyline selection.

  • New selection mode for polyline.
  • New manipulation mode for polyline.
  • UI to choose selection mode (visual + shortcuts). Add icon for toolbar and switching logic.
  • Changes to the sorting model (area based?) in RegionsManager

Adding a clear button to the toolbar

It would be nice if there could be a button on the toolbar for clearing all current regions from the canvas. In playing with the tool it is easy to make accidental mistakes that are hard to remove (making a very small rectangular selection).

Can you add a button that clears the regions?

Update cursor icons for CT

  1. SVG cursors are not supported in Edge, and fallback doesn't work in current implementations.
  2. The resize cursor is too similar to the move cursor.

Item selection on "delete" press

After user presses "delete" button the next element is selected (and deleted in second press) even if there was no active element before.

Expected behavior: if nothing is selected pressing "delete" does no effect.

Optimize redrawing of regions

Current implementation redraws all the regions to resort them on each update for RegionsManager. It seems like it takes too much time.

Draw the annotations differently dependent on their type

You probably have that on your sheet allready but I wanted to add this for completeness.
As for now there are 4 different kind of annotations (rect, template, point, polygon) however as they are drawn on the image they all look very much alike.

I think the following ideas are related to this

  • Draw a point as a point (not as a small rectangle), also a point annotation should not be resizable
  • Draw the single nodes of a polygon, not only the bounding box

Greetings!

Merge box selection modes

Merge box selection modes:

  • Refactor code to split selection modes
  • Basic box selection + two points mode (ctrl+click)
  • Template-based and copy-based
  • For copy-based mode add an option for a full copy including tags

Issue: #11

It seems it is natural when basic and two-points selection modes for boxes are merged without extra keys (ctrl). Example of implementation: https://leaflet.github.io/Leaflet.draw/docs/examples/full.html

For template-based mode -- we don't have yet any specific UI to setup template and it also seems natural to use a template already existing box. For the copy-mode

vott-ct v2.1.25 compile bug

There is an issue with the current version I cannot get it to compile right when imported into another project: Module not found: Can't resolve '../../Core/Point2D' in '.\node_modules\vott-ct\lib\js\CanvasTools\Region\Component':

image

New features to add for V2

Based on issues for the VoTT (in priority order)

  • Implement the basic Toolbar UI element for choosing selection mode.
  • Lasso/polygon selection. 1) New selection and manipulation modes. 2) Various options to implement the tool UI are possible. 3) UI to choose selection mode (visual + shortcuts), 4) Changes to the sorting model..
  • One point selection mode. 1) UI to selection mode, 2) Changes to the storing model.
  • Photo adjustments. 1) UI to select photo filters and manage pipeline (visual + shortcuts?). 2) New filters.
  • Template boxes. One-click boxes or points creation.
  • New coloring schemas. Generate inclusive pallets sets, taking into account contrast ratio with the image.
  • Zoom in/out . 1) Scaling engine for both the underlaying image (in canvas element) and the collection of regions. 2) UI for scaling (visual + shortcuts)
  • Implement an advanced version of Toolbar allowing edigitng, pipelines, etc.

Issues running the samples on Chrome

Hello, Amazing library!

I'm having issues visualising the background image, and the toolbar is not shown completely anymore using latest chrome.

Any idea?

Cédric

Screenshot 2019-05-10 at 16 47 00

"Point" rect selection is not triggered

When a rect is created by one click, so that it looks like a point -- attempting to select the region does not trigger selection.

  • Clicnking on anchor does not trigger selection
  • Outcome: element can't be easily removed from UI

Fix: Add selection trigger for the anchors.

Related: #15 (Auto-activating two points selection should reduce the number on occasionally created points).

Dragging elements has a small activation delay

Current implementation creates a feeling of slow UI response and draggable elements may be visually unattached from the pointer cursor.

[x] Replace drag/undrag methods from SnapSvg library with direct manipulation based on pointer events (DragComponent, AnchorComponent).

Support rich description export for AreaSelector

Update how AreaSelector shares data on selected region. E.g. consider wrapping rect as bounding box and add support other types as "meta".

E.g.

{
    boundingBox: { x1: 0, y1: 0, x2: 10, y2: 20 },
    meta: { 
       point: { x: 3, y: 5 }
    }
}

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.