Giter Club home page Giter Club logo

filmviz's Introduction

FilmViz

About the project

There are many film analysis on the internet, particularly of color. Some are manual and subjective, some analyze just a frame while others span the whole film, and many use their own private tools.

That's why we're building FilmViz: a web tool that helps you analyze your favorite films, compare them in a visual way and share your discoveries with others.

We want to provide a common tool for film analysis and comparison to the community.

FilmViz

Description

The final aim of this project is to build an online database of film analysis that allows users to visually compare different aspects of films, such as their main colors, their motion or rhythm through the number of cuts, the amount of dialogue and music, etc. Combining all this information will allow you to understand the whole picture.

The project is inspired by a number of independent analysis of different films that are available out there (such as this, this, this, this or this).

Objectives

The first logical step is to build a film analysis tool. The tool will run locally in the browser and will perform automatic color, motion and audio analysis, and will also allow the user to manually input additional metadata (such as types of camera shots, actors in the shots or a day/night classification). The tool will show visualizations for all the data gathered as well. Once the analysis is done, the user will be able to share it by uploading it to an online database.

The second part of the project would be to build an online film analysis database, to which users will upload the analysis performed locally with the tool. A website will display visualizations for all the film analysis uploaded, and will allow users to see comparisons between films.

Sample images

It's still a work in progress, but we can already get automatic analysis for color and motion.

Here you can see the color analysis of Sin City.

Uhm, let's see... where does the yellow character appear on screen? Yep, you guessed it.You can also add your own tags to measure any other aspect of the film you want, and the tool can even tell you things like the number and duration of cuts.

Sin City

Another example: The Goonies.

Here you can appreciate the rhythm of the film through the motion analysis.

The Goonies

Test it!

You can try an early prototype of the analysis tool at filmviz.kinoraw.net (kinoraw are kindly hosting us for now)

Tell us what you think

Follow us on Twitter (@filmviz) to get updates about the project. We're eager to hear what you think, so please don't hesitate to come talk to us.

Installation

Before you can run a development version of FilmViz you need to install the bower dependencies. Execute this command on a terminal:

bower install

(You need to install node.js first and run npm install bower -g to install bower)

That's it! Now you can run a static HTTP server and enjoy FilmViz:

python -m SimpleHTTPServer 8000  # Python 2
python -m http.server 8000  # Python 3

Then go to http://0.0.0.0:8000/. Please keep in mind that the development is still in early stages. A lot of things are broken or not implemented yet.

Tips for contributing

We follow the rules of JavaScript Code Style from Airbnb.

You'll fit better with us if you install jscs with npm install jscs -g an run jscs your-file-or-path before requesting a pull.

Every contribution is appreciated, but they will be better if they cause a good impression to us. Thanks!

Documentation

Findings

We run some tests to figure out an acceptable sample rate for the motion analysis, since we needed to find the right balance between accuracy and the time the analysis takes: getting a lot of samples per second would make the analysis unbearably slow, but getting too few would affect our ability to detect cuts in the video.

We performed a few motion analysis for the same one minute video with different sample rates. The results can be seen in the graph below (the x axis is time, while the y axis indicates the motion difference between two samples, ranging from 0 to 1). The video used for testing has two cuts, around second 6 and second 20.

Sample rates graph

  • For 2000 ms, the analysis took 8.143 s (13.6% of the video duration)
  • For 1000 ms, the analysis took 17.119 s (28.5% of the video duration)
  • For 500 ms, the analysis took 36.136 s (60,2% of the video duration)
  • For 250 ms, the analysis took 71.847 s (120% of the video duration)
  • For 100 ms, the analysis took 182.665 s (304% of the video duration)

We concluded that getting a frame every half second (500 ms) will be good enough for our application. This conclusion is however susceptible of being revisited in the future.

References

WebVTT

Color analysis

Motion analysis

Audio analysis

Metadata analysis

filmviz's People

Contributors

roboe avatar surreal6 avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar

filmviz's Issues

'use strict';

We should 'use strict'; in all our .js files.
When I reformatted that files to comply with our code style, I tried to make them strict, but project object was not found because it is declared inside a closure and strict forced it to be local.

One posible solution is manage to put project on an Angular Service or similar, to make it global to our Angular components.

Analyze video from online platforms (youtube)

It would be cool to allow the user to analyze a video from youtube, vimeo, etc... so he doesn't need to have the video file locally. For example being able to analyze all the music videos from a certain artist or album...

Interesting article about getting the video files from youtube based just on a video id/url: http://coding-everyday.blogspot.com.es/2013/03/how-to-grab-youtube-playback-video-files.html

Obviously this is an improvement for the final version of the project

Tag Editor

[X] manage the project tags for each user defined vtt
[X] generate buttons for each tag in the tag list
[X] add x button to each tag to remove the tag from the list
[X] add functionality to create data into the analysis when playing...
[ ] hide delete button for tags when playing
[ ] show label with vtt content from video track broadcast...

add progress bar for analysis

var progress = document.getElementById('progress');
progress.max = parseInt(video.duration);

progress.value = parseInt(video.currentTime);

Slugify the project name

Make a slug ('some-file-name') of the project name to be usable as filename.

We may use some RegExp for this.

IndexedDB for storing advanced data + screenshots

As I said, IndexedDB would be a nice improvement once the version with localstorage is ok (http://w3c.github.io/IndexedDB/). It works more like a database (hability to do searches, and have different storages and data structures, ....) nowadays its supported more widely than when I started working with it.

But the most interesting feature is the ability to store "blobs", in our case scenario will be the screenshot of each "capture". Example with ydn-db: http://dev.yathit.com/ydn-db/doc/usage/blob.html

I encourage to use a wrapper above indexedDB if you are not interested in diving how it really works underneath and deal with too much code. I developed with Ydn-db but there are much more, just find the one that looks better documented and mantained.

Count with my help with this issue, I lost interest in the technology but it's always nice not to loose knowledge.

Timeline

Full width timeline with video seeking capabilities.

Make color analysis work on Firefox

Running an analysis on Firefox give this exception:

IndexSizeError: Index or size is negative or greater than the allowed amount

On the contrary, after this first error, if you run the analysis another time, it works.

Maybe it has to be with the way Firefox load videos.

local app interface

  • load and save project files
    (project structure)
    zip -->
    project.json (with relative paths to all files:)
    vtt/
    color.vtt
    audio.vtt
    rythm.vtt
    ...
    metadata4.vtt
    img/
    thumb0001.jpg
    thumb0002.jpg
    ....
    thumb8653.jpg
  • view and edit individual VTT files
  • generate limited analysis (from TC to TC)
  • Edit individual CUEs

two simple bugs related to vtt protocol (3 digits miliseconds and tcOut)

It seems the track feature doesn't like to receive cues without tcOut...

Cues in vtt files should have always tcOut
we can generate the tcOut when file-saving, using next tcIn in each tcOut and for the last one, put in tcOut the total lenght from the video

Cue Timecodes should have miliseconds expressed with 3 digits:
00:00:00.0 WRONG
00:00:00.000 CORRECT

very useful link, a vtt validator
https://quuz.org/webvtt/

Tidy up README.md

Separate CONTRIBUTING.md and maybe move documentation to a Wiki or another file seem like good ideas to me.

Color sorting on frame visualization only works when video is paused

The selected sorting method is only applied to the current color sample when the video is playing, as soon as the next sample comes it appears unsorted.

If some user chooses sort by value and plays the video, s/he expects to see colors ordered in that way for new samples.

Metadata Editor

edit capabilities through vtt file...

next/prev buttons, edit and save buttons...

Offline capabilities for long processings

Needing internet connection while you are processing a long video could be an issue for some users, even needing connection at the start of the processing.

Since all is done in the front-end (in the browser), it will be easier to make the app work totally offline (alerting the user when connection goes of and on to upload the results). One way could be making a firefox/chrome plugin, or just download the code and launch the app in localhost... but both seems too much hassle.

Using the Cache Manifest feature and some tricks with js and localstorage the process would be transparent for the user, and much faster than the other options.

Just read this article and take a peek at the demo http://alistapart.com/article/application-cache-is-a-douchebag

Again, count with my help to develop this suggestion. I would start it after #4 is completed since localstorage would be used for permanent storage of html content, and erasing processing info would require a little more precaution.

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.