final

enchanted forest

enchanted forest is an experiential environment for exploring audiovisual consonance and dissonance. you may make it a calming or harsh, dense or sparse, the controls through the forest are simple:

movement: WASD + right mouse

intensity: up arrow / down arrow

noise: T / G

to interact with a tree you must approach it. if it is dark you must find it. there is no map, there are no instructions.

milestone 2

milestone 1

audio-responsive point cloud renderer

For the milestone I built a point cloud renderer that responds to audiovisual input. It works by drawing a particle system on the vertices of a mesh, and then modifying the positions of particles based on ChucK input. My hope is to apply this to more complex meshes and eventually build a scene that is rendered as a large point cloud. The renderer is built in VFX graph, so I can modify it to add additional effects like trailing or gravity. Eventually I want these particles to be manipulated by a vectorfield in addition to manipulations driven by the VFX graph effect.

The goals for the milestone were as follows: import a mesh (the video below shows a houseplant), render the mesh as a point cloud, and have the mesh respond to audio input. The way the points respond to audio now is just proof of concept. I'm aiming to tailor the particle effects / motion to the audio composition I pair with it.

Here is the VFX graph for the point cloud effect:

milestone 0

idea 1

• import point cloud photogrammetry data
• let user explore scene in three dimensions
• sound integration:
  • option 1: FFT of audio input modulates a vectorfield that the point cloud follows. in the absence of sound the point cloud will eventually return to the original state.
  • option 2: have specific controls for instrumentation, instrumentation modulates swirling vector field parameters.

idea 2

• an embodied virtual instrument
• use 3d scan of face and set certain articulation points
• articulating these points drives different sound generators
• distortions of the facial geometry correspond to sonic changes

idea 3

• start with distance estimated 3d fractals
• user controls various generators / instruments
• modulate parameters of 3d fractal with generators