skykistler / smudgr Goto Github PK

Controls to utilize the new graph model and renderer, add algorithms, control parameters, etc

Changed the way img's pixels are set at the end of execute from

img.pixels = copy.pixels

to

img.setBufferedImage(copy.getBufferedImage());

and also changed the step of the parameter from a default of 0.01 to 0.001 for higher precision.

Make control for recording and outputting full-res/full-frame gif's with proper delay, pruned if need be to meet minimum delay requirements

The experimental pixel shift could do more than the current implementation.

• Shift in different directions
• Start shift at different parts of image, or rework how each interval is shifted
• Interval size based on univariate function
• The interval count and speed need to shift at percentages so when downsampled, the speed/interval count does not change.

Downsampled pixel shift

• The first interval needs to shift. Somehow the algorithm needs to be reworked so that there is not always a static first interval.

An effect I've wanted to work with for a while. Here's what I have so far for it.

This is what it looks like:

https://gyazo.com/90b9c71acbd3db59c7d3dd357b4801eb
https://gyazo.com/9daf00a5ebf6f39193d140a202735020

When MIDI system control messages are received for start, stop, and pause, the controller needs to act accordingly

• Elliptical bounds
• Font bounds
• Composite bounds (multiple bounds in one)

Any other interesting things we can think of?

Allow users to give custom names to various algorithms in the UI for easy recognition

Writing an algorithm that all in one can manipulate the following in separate parameters:

• hue
• saturation
• lightness
• color channel manipulations, etc.

I'm thinking that I will just write color helper functions for each of these operations, so that any other algorithm would be free to use them, but this algorithm would just tie them together for practical use.

I'm making a general issue to keep track of some ideas I don't want to lose before getting to work on pixel shifting.

• Pixel Propagation
• Implement my version of ChannelDrift but call it something different
• Implement energy function and use for seam carving and pixel sorting
• Create set of sub-algorithms that mimic marble paint algorithm
• zig zag stack (very useless name I know, but I can explain if you want. It's only here as a reminder)
• Add rotation to algorithms

Add the ability to listen to multiple midi channels at once

Control toggle for enabling/disabling all algorithms

Look into controllers that only use note on messages with 0-127 values for actual off/on signal

Parameters need to be tied to each piece of our algorithm pipeline, not just the algorithm itself. This may involve reworking the algorithm class/package structure

We need two types of animators, one can be beat synced like it is currently, and one should be arbitrarily stepped for the smooth animation of other parameters.

In cases like Gif work, video work, and sometimes even performance work, syncing to beat will always look choppy

Make an interleaved/non-interleaved indestructible raw format for use in audacity and similar programs. Ideally, each frame of a gif or video could be saved as raw, and brought back into the program where another gif is generated (and used as a source).

Interleaved/non-interleaved option and ARGB/RGB option. Consider other color formats.

Abstract the selection of an operation into a separate class. These will be run when an image is changed, in contrast to coordinate functions which are fun when the dimensions of an image or a bound is changed.

Coordinate function -> Selector function -> Operative algorithm( Univariate/other functions )

Sometimes this is fast, or being run in the studio, but in some situations it is too slow or impractical to do live. Thus, the goal is be able to optionally threshold an operation.

This will also open another dimension of complexity when building algorithms. For the foreseeable future, I think the above four components will be what a given algorithm pipeline is comprised of.

Re-iterate over a set of algorithms a certain amount of times

GIF's need to match the framerate of their source

We need to think of a solution for mixing images with the control of parameters.

We should be able to automate or trigger the switch or mix of sources

Currently, different channels are ignored and if multiple algorithms with the same name/type exist, the parameters will overlap

In a studio setting, performance indicators are helpful but not necessarily needed. When designing a live instrument, these performance differences can mean the world. Is there a way we can reflect bottlenecks in a smudge through the UI?

@skykistler

Working on an Elliptical Bound :

Branch: eee1e86

Notes:

• Inherited Bound class and changed containsPoint method.
• kept Bound's offsetX, offsetY, width, and height parameters. Made ellipse using those parameters. Not sure if this will always be the case
  for more complex boundings, but it will work here.
• To change bound, I used Algorithm's setBound method in EricMain

auto-hack any GPU to trick OpenGL into parallel processing

Test and expand our current functions, and devise new ones

What optimizations can we make to prevent too many images from being loaded at once, while maintaining live access to as many sources as possible?

The single interval scroll is cool but shifting in intervals is cooler. Maybe make a toggle? Could be unnecessary. Let me know what you think

Look through the radial coordinate function in relation to pixel shift

Improve support for relative knobs and offer support for absolute knobs

a) detect if absolute or relative knob being used
b) support different modes
c) allow manual change if unable to perfectly detect relative/absolute

Implement Processing video library or similar in order to smoothly grab frames at proper rate, independent from render loop

Adding directionality + different scaling options to pixel shifting: 5e25ab1

Notes:

• The reverse parameter should work on any given direction type
• "Binary" direction shift added. Switches back and forth in direction from shifting in positive direction
  and shifting in negative direction based on even/odd check. Case 2 of the switch statement:

        double shiftScale = 1;

        //  find type of shift

        switch(shiftType.getIntValue()) {

        case 1:
            shiftScale = currentInterval;
            break;
        case 2:
            shiftScale = (currentInterval % 2 == 0 ? -currentInterval : currentInterval);
            break;
        }

        //  reverse direction of shifting 

        if (reverse.getIntValue() == 1)
            shiftScale = -shiftScale;

Goals:

• Add reverse parameter
• Add at least 3 different direction types
• Add scaling option of some type

Completed So Far:

• 1/3 direction types added so far.
• Added reverse Parameter

Creating a static noise algorithm that uses the YUV color space , which was used in analog color TV's.

• Retains relative luma while randomizing the chroma within the pixels

Processing incoming audio signals to affect parameters

• Audio input
• Set frequency range
• Set easing function and sensitivity

Leave algorithm ideas here. When you start on an algorithm, make a separate ticket and reference it here.

Probably due to concurrency issues, midi messages can be missed or their effects are ignored

Need to be able to change univariate functions on the fly

SelectiveHueRotate branch

Given a function like luma or hue that maps onto the range from 0 to 1, the SelectiveHueRotate algorithm rotates the hue all pixels whose values(hue or luma for example) are in the sub-range from start to (start + length).

This is more of a selector, but that idea has to be developed.

Allow copying, deleting, rearranging, and changing the delay of frames in a gif

Marbeler needs to work with coordinate functions, interpolation functions, and differentiated from pixel shift

Figure out the best way to allocate source "sets" and switch between them individually, rather than have a bunch of sources in the same set to sift thru

Animator needs to work in the paradigm of quarter steps/half steps/full steps etc in order to sync to the controller BPM and be more predictable when running live

We should be able to have separate effect pipelines that can be mixed or switched, triggered, and automated

After integrating CEF, a basic HTML/JS interface needs to be created. Needs to be relatively clean but does not need to be fancy.

Needs to control:

• Sources
• Bounds
• Coordinate functions
• Algorithm sequencing
• MIDI mapping

Marbeler needs wavy-animation support

Basic shapes renderer that can be layered in, which can be extended to rendering fonts

Save algorithms, source locations, parameter presets, and midi mappings

Chromium Extendable Framework needs to be built and integrated into smudgr

Implement parallelism and integrate into existing algorithms

Render to window and specified monitor fullscreen, so window can be on laptop screen while fullscreen is on projector.