martin-cox / eyespy Goto Github PK

Need some system that can perform concurrent requests for image analysis and then handle the processing of results for that image. Things to consider:

• How many concurrent requests will Clarifai allow us to make?
• How many concurrent requests should we be limited to for performances sake.
• How to handle instances where multiple requests are being made for the same image?

Each image should retain it's Clarfiai results on a per model basis so that we don't need to repeat requests for images.

This action is to block the image by replacing it with a black element.

This action will replace the original image with one that has a text caption added to it. Things to consider:

• Should we allow users to defien a pool of captions to randomly draw from for each image? Should that be on a global or individual ruleset basis?
• Can we re-use some of the Edit actions logic for this?

Some useful functions include:

getSourceUrlFromElement(element: JQuery): string => ...
validateUrl(url: String): string => ...

This action is to download the image to the users PC. Things to consider include:

• Where should images be saved? Maybe use Chrome's default download location and allow each user to specify a different location they want to download images to (on a global or individual ruleset level?)
• If the ruleset is poorly designed or too general we could be attempting to download lots of images. Should we warn the user before attempting to download a number of images above a threshold?

It would be useful to have information regarding image processing status, filter status, Clarifai requests etc. logged to the console. This information should only be logged when the user has entered debug mode.

Need to work out:

• Best way of implementing conditional logging functionality
• Best way of enabling/disabling debug mode. General users shouldn't be able to access this by accident.

It would be nice to be able to analyse an image against multiple models and handle the results as a single operation. Things to consider include:

• Clarifai only allows one model per request, so we will need to perform multiple requests for a single image
• Processing the results should only occur when all requests have been completed for a given image

The action will open a new tab that navigates to an image search engine pre-populated with some of the tags from the original image. Things to consider:

• Should we let users choose which search engine to use? For an initial implementation Google Images would be sufficient.

Users should be able to specify many rulesets. A ruleset consists of one or more rules and a trigger. Rulesets will analyse all images on a page, and for any image that meets all of the rulesets rules an action will be applied to that image.

Trigger
An inciting event (e.g. page load) that triggers re-evaluation of the ruleset.

Rule
A rule consists of a tag that corresponds to a tag returned by the Clarifai API after analysing a given image. For example, an image containing a cat playing with a ball might return the following tags: Cat, Kitten, Ball, Toy, Pet.

A rule also has an operator. These are standard conditional operators e.g. > (Greater than), <= (Lesser than or equal to).

Lastly, a rule has a probability, being expressed as a % value between 0-100.

Putting these three elements together makes up a single rule. An example of a rule is: Cat >= 80%

Action
An action to take against each image that meets all the rules. See the Actions for further information.

This action is to hide an image without any regards to page layout or user experience. The simplest way to do this would be to add display: none to the image DOM element.

There already exists a mechanism for adding the keys via the extension popup, and the keys are stored in Chrome's local storage, but the current implementation isn't ideal:

• Storing keys in local storage probably isn't very secure
• UI for entering the keys is WIP
• No indication is given to the user when the keys are invalid/no keys are provided
• Changing the keys will cancel any existing requests
• Changing the keys may invalidate queued requests

Users should be able to choose which Clarifai iamge recognition model they want to analyse an image against. Things to consider include:

• The list of available models needs to be kept up to date with Clarifai's list
• Should users be able to choose specific versions of models to use?
• What would be the best way to allow users to choose a model? Simplest implementation would be to have a dropdown in the popup that allows them to select which model to use, and have any future requests use that model, but should users be able to choose which model to use on a per-request basis?

Clarifai cannot process animated images, so animated .gif, .webm, .png will need to be prevented from being analysed.

This action replaces the original image with oen that has been edited. Things to consider:

• What library should we use for image editing. Ideally one that is small and performant as this action could be performed for many images at a time.
• What sort of manipulations should we allow users to define in this action? Basic things such as resize and tint are obvious, but what about adding complex shapes or adding complex effects?
• The UI for defining what manipulations to perform on images may be complex.

This action extends the Replace with... action but instead of the user choosing which image to use as the replacement, we should find a similar image online by using the Clarifai tags of the original image.

This action will allow users to replace the iamge with a user specified replacement. Things to consider:

• Should the replacement image be defined on a global or individual ruleset level?
• Replacing an image with a different size one could break a sites styling, should we attempt to handle this in any way?
• Users should be able to choose a replacement image either by "uploading" it from their local PC or via hotlinking it by URL.
• How to handle instances where the replacement image doesn't exist or got corrupted?
• If we are using a local image to replace with, it should be cached for performance.

See https://greenkeeper.io/

This is already partly implemented. Things to consider include:

• The tooltip should appear above the page and not mess with the page styling.
• What results would be useful to show in the tooltip and how should they be displayed?