projects
The following are a few of the projects I have worked on in the past. Many of them are works in progress but I share them here in their current state.
windidi
Windidi allows a Nintendo DS to be used as a wireless wind instrument. Windidi utilizes the functionality of the DSMI library, to turn a homebrew capable DS into a wireless midi controller. By using Windidi with the DSMI server application, midi note messages are sent to your computer via wifi based on input through the touchscreen and microphone. More specifically, note velocities correspond to the input level of the mic, making it work like a wind instrument of sorts.
Download -- file includes source code
windidi.pd -- a pure data patch to use with windidi...select different synths by changing midi channels
Video of windidi in action, sorry about the quality
Example - created using windidi and tin whistle
DSMI - OSC implementation
I created a small and very basic implementation of the OSC (Open Sound Control) protocol, for use on the Nintendo DS. It allows for both sending and receiving OSC messages over wifi. Its addition to DSMI enables direct network communication via OSC without the need for an intermediary MIDI translation server. I also created an example application for the DS to utilize the OSC client/server capabilities of DSMI. The application turns a DS into a wireless OSC controller with a small kaos pad and 3 sliders.
Video of OSC example application in action.
The project is hosted at http://code.google.com/p/dsmi/. You can download the OSC example application for DS from there.
sound quack
Sound Quack is a desktop audio application for processing sounds and live performance, which I built for the Media Arts and Technology 240 course at UCSB. It is built using the JUCE C++ class libraries and FFTW for fft based phase vocoder effects. In its current state, Sound Quack is incomplete but allows for realtime fft based pitch and speed alteration, and convolution of sounds. It needs a lot of work until It will be useful as a performance tool.
You can download the source code here.
object tracker
This is an OpenCV based object tracker I built as part of a group project for a computer vision class at UCSB. It lets you select an area in a video stream to track, and then outlines that object and outputs a plot of the location of its center. It is meant to track objects of solid color.
Download source code herepocket ninja
This is a physics based platform game I started working on for the DS. It will feature physics based puzzles and innovative gameplay mechanics steming from awesome ninja powers. Currently, it is the begginings of a high level game engine running on top of the Box2D physics engine. I am planning on moving the project to android for the possibility of higher exposure.
Here is an early video of the engine running.
Oooo rhino crates!Download source here.
wiimote gesture recognition
I built and worked on a project using neural networks to learn simple gestures based on accelerometer input from a Nintendo Wii remote. It took the form of a pure data patch, using at first a ANN object I built for pd, which I couldn't quite get to work on windows at the time, so instead I used an existing ANN extension. The accelerometer readings were sent, at the time from a script running in GlovePIE on windows. The results were partially sucessful. I wasn't able to get full gesture recognition, just an initial recognition of static states of orientation. This is largly due to my learning model, which I didn't think all the way through. My first mistake was that the neural network was learning off of raw accelerometer data. Raw data can be quite noisy, making training a neural network extremely difficult. Ideally I should have reduced the data to a simple limited set of directional vectors, greatly reducing the space of learnable states. I like the idea, but clearly I need a better technique.
You can check out the patches here and also here.
beepintext
This is just a fun little thing I threw together my freshman year. It is a simple text editor that beeps a tone on keypress. The pitches are ordered across the keyboard so that you may type simple melodies, or discover melodies by playing with words. It allows for the selection of different basic scale types including chromatic, whole tone, major, minor, and harmonic minor. It also lets you playback whatever is written in a text file.
It is only available as an ancient Windows executable. Download it here. It may complain about missing library files, if it does download this. Copy that file to your \Windows\System32 directory, and run 'regsvr32.exe COMDLG32.OCX' as an administrator to register the library.
