smashingboxes / fraqture Goto Github PK

An interactive digital art project built with Clojure and Arduino

Clojure 87.39% Arduino 2.40% C 7.31% Ruby 2.58% Shell 0.32%

fraqture's Introduction

fraqture

An interactive art framework that takes inputs from social media, maps them to the interactive parts of a drawing, and renders the art. The platform consists of a 50" TV display driven by Clojure's Quil library outlined with 540 DotStar LEDs driven by a Teensy 3.2.

Exhibitions

Come see fraqture live!

You can find Fraqture today in the 21C storefront window in downtown Durham, NC.

March 14-16th - New Orleans Entrepreneurs Week
Now - Downtown Durham

Usage

lein run <drawing name>

For examples of what each of the drawings does, check out the gifs in examples.md. Start by trying lein run cycle.

Hardware

The Fraqture Project uses two custom-build LED panels with 270 individually addressable LEDs each to extend our canvas out from the screen. We are using a Teensy 3.2 to receive commands over a serial connection and use them to refresh the panels. The sections below discusses how to connect and issue commands to the panels.

Reset Communications

The LED serial communication is a basic state machine. When a device connects to the panel driver, it is impossible to know whether it is in the default state or not. To reset the state machine of the panel driver, simply wait 2 seconds in between messages and the state will be reset.

LED Commands

All LED panel commands follow the same structure, where the payload is strictly packed and all numbers larger than 8 bits are packed little endian.

1 BYTE        0-7 BYTES
+------------+-----------------+
| COMMAND ID | COMMAND PAYLOAD |
+------------+-----------------+

New commands can easily be added using the terminal_attach(terminal_t *term, terminal_cmd_t *cmd) function found in arduino/strobe/terminal.c.

Refresh Pixels

This command will refresh the LED array. This allows you to do multiple paint operations without eating the cost of refreshing the array after each one.

Command ID: R

Clear Pixels

This command will clear the LED arrays. This command has no payload.

Command ID: S

Window Pixels

This command will paint a window of pixels.

Command ID: W

Payload (in order):

u8 row-start: the starting row for the window
u8 col-start: the starting column for the window
u8 row-end: the starting row for the window
u8 col-end: the ending row for the window
u8 r: the brightness of the red LED in the pixel
u8 g: the brightness of the green LED in the pixel
u8 b: the brightness of the blue LED in the pixel

Set Pixel

This command will set an individual pixel.

Command ID: S

Payload (in order):

u16 index: the index of the pixel. The top left is pixel 0, and then it reads like a book.
u8 r: the brightness of the red LED in the pixel
u8 g: the brightness of the green LED in the pixel
u8 b: the brightness of the blue LED in the pixel

Requirements

Install Java JRE or JDK from Oracle's website.
brew install leiningen (or install it for your OS using these instructions)
Imagesnap brew install imagesnap (or install it from here)

System Setup

The overall diagram of how the LED controller, LED panels, power supply, and Mac Mini all interact can be seen below:

Since LED data is a serial stream, it originates by the Mac Mini's program sending data to the LED controller, which then sends it out to the 540 LEDs using SPI. The individual wires that need to be connected can be found below.

For the panel connections, the diagram shows a closeup of the screw terminals found on the backs of the panels. The panels should be positioned so that the screw terminals are in the lower-left of each panel. A legend for these connectors is below:

Line - Wire to another component.
X - Internal panel connection; leave connected.
N/C - No connection.

LED Controller

Panel 1 (Top)

Panel 2 (Bottom)

Power Supply Unit (PSU)

fraqture's People

Contributors

Stargazers

Watchers

Forkers

brandonmathis

fraqture's Issues

Finish tweet streaming

Reorder the glitch drag exit criteria so that it will not draw a new image right before exiting.

Nine pixels in the bottom panel are always one frame behind

Improve tweetreader

We would like to make two improvements to the tweetreader image:

Instead of fading to the SB logo, it should draw in the photo attached to the text.
It runs a bit long. Exit criteria should be added.

Add LED animation to the tweetreader drawing.

When typing the message, use the bottom LEDs as a keyboard.
When scrambling the message, scramble the LEDs as well.

Create a universal Arduino interface

Right now, every sketch that uses an Arduino will attempt to connect to the Arduino during its setup phase. When we begin cycling these drawings, the second drawing will crash because the port will already be open. I'd like to make a serial interface that:

Asks the user what port they would like to connect to upon opening the program. You can use list-ports from our serial library to enumerate their options.
Opens the serial once during global set up, then puts the serial resource in the options hash being passed in to the sketches.
Update all sketches that use the serial port to use this new interface.

Clean up the Readme

Our readme has a couple of broken images and could be more descriptive and helpful.

Create a Night Cycle

Change the cycle drawing so that during the day it uses live camera feed + seeds content in the main glitch cycle, but at night it uses more subtle generative art.

Clip the drag drawing on the sides so that the 30 LED rows line up.

Add LED animation to swap.

Add a test suite

A simple test suite could run each of the drawing commands (eg lein run drag) and kill it after 30 seconds or so, just to make sure each one runs without error. It could also look for stack-traces.

Add exit function to tweetreader

The tweetreader function needs an exit condition. I'd like for it to be 1-2 minutes after the textify animation starts, but am open to suggestions.

Add protections in the C code

Right now, if a drawing writes outside of the 18 rows and 30 columns, it is possible to crash the application. Some bounding would be great.

Write a script that will use a camera to put a picture in /images

As a jumping off point for adding live content, we can first find some way to utilize our current infrastructure by having a module in Clojure that can take a picture and place it in the images folder or have Clojure call out into a script in some other language that does so.

Split image sources into logos and rasters

Some glitches work better on logos, and others on rasters. If we split the folders in two, we can replace stream/get-image with stream/get-logo and stream/get-raster.

Remove time at the end of the photo countdown

Improve the swap algorithm

We would like to make two minor changes:

Use colors from the image when filling in
Make the size of the squares being swapped roughly equal to the LED squares.

Minor improvements on shifting-grid

Three minor improvements for shifting grid:

Remove pink outlines
Slow it down
Add a delay to the end

Features:

includes a countdown
captures a live image
runs the image through one of the glitch algorithms
transitions between live camera & seeded content

Not all columns trickle at the same time.
Columns trickle in from the top using the LEDs

Create a pool of images and texts for display

We currently have a mechanism for grabbing images for display, lets also have a way to grab text. Ideally we want to have some weighting system too to show more recent images.