Part 1 – Art, Computers and Systems
Part 3 – Art, Computers and Systems
As digital technology began taking root in the mid-20th century, artists started incorporating various technical and conceptual systems into their practices. This reframing of art, in terms of systems, built the foundation for contemporary new media art practices, such as creative coding and generative art. As an artist using code as a creative medium, I depend on systems for the design and realization of my work.
In the first part of my series on art, computers, and systems, I explored the interplay between early computers and system theory. This combination had a lasting impact on the art world, moving practices towards the intersection of art and technology. Prominent exhibitions, such as Cybernetic Serendipity in 1968, showcased a variety of avant-garde artworks that embodied the values of earlier Modern art movements and creatively utilized new media.
Join me in exploring two of my artworks that exemplify what I term “expressive systems”: Metaworm, a computer-driven installation at Meow Wolf’s Omega Mart in Las Vegas, NV; and Waveform No. 2, a kinetic installation at Brown University in Providence, RI. In both cases, the system serves not just as a means to enact the artwork, but also as the final form driving its expression. These two installations serve as good examples of using systems and computation in art.
Metaworm
Preamble
Defining nature is as difficult as defining art. Its definition means different things to different people and cultures. In Western society, “nature” tends to encompass everything that is naturally occurring – the environment, Earth and the cosmos. Conversely, traditionally speaking, computers are viewed as human inventions, artificial objects. However, as I discussed in a previous article, the natural world is abound with computation – animal behavior, plant systems and even the evolution of the universe itself, can all be attributed to it. While it would be a mistake to directly compare generative technologies to organic life, it is difficult to overlook the similarities between machines that generate media from digital code and the organic machines, such as cells making complex proteins, that produce media from DNA code.
I have always been captivated by the interaction between natural and artificial systems. I set out to explore this interplay by creating Metaworm, a digital lifeform caught between two realms – the physical and the digital.
The Artwork
Metaworm can be described as a kinetic media sculpture. It is made up of fabric and digital projections, controlled by a custom-designed system of computerized fans driven by sensors. Situated at Meow Wolf’s permanent art installation in Las Vegas, Nevada, the artwork portrays a lifeform and explores the emergence of artificial nature through colorful abstractions and patterns in motion. Using a combination of fabric, interactivity, generative graphics and audio, the kinetic form transcends reality, as it ebbs and flows in a perpetual dance of light, sound and motion. In the presence of people, the artwork becomes erratic and eventually mushrooms into a climatic billow of chaotic flurry that feels alive.
Metaworm is a live projection-mapped media sculpture that uses video from a series of projectors, and applies generative graphics to a moving surface in motion. Typically, projection-mapped objects need to be static – unmoving. In this scenario, the video is mapped onto a moving fabric surface using an overhead depth camera that is responsible for reading the shape of the fabric and morphing the graphics to fit the shape at a rate of 60 frames per second.
Secondly, the camera detects when visitors approach. When people gather around the virtual lifeform – represented by projected abstract generative graphics and sound – it becomes evermore unsettled. Poised as a creature that is trapped, the entity struggles between digital and physical reality, becoming more abstract and fluid with each turn. Standing too close to the railing surrounding this mysterious creature is what ultimately causes the form to erupt into a dazzling array of colors and sounds.
The System
A large part of my design process typically consists of specifying various technical elements and how they communicate in order to create a seamless experience. Everything in this artwork is part of an “expressive system”. The motion, visuals and sounds are generated or modulated through live interaction between the audience and the air being driven underneath the fabric. Below are two diagrams that represent the high-level system, which is in turn composed of several subsystems. This includes: software; control; sound; sensing; projection and actuation.
Crafting new technology involves a great deal of prototyping. While experience and skill is crucial, I attribute constant testing and iteration as the most crucial factor in overcoming the inherent challenges and constraints of engineering art. Workarounds are common. I often need to relinquish aspirations for things like reality, physical laws, budgets, deadlines and acts of god. Metaworm is no exception. To bring it to life, I had to tackle many problems at each stage. I try to start with the most difficult problems – the
“unknowns” – and work down to minor details. Ultimately, the quality of my work is measured in a tally of resolved issues, well-defined specifications, tested features and the consideration of as many details as time will allow for. The magic lies in the accumulation of all these details, which enable the creative system to perform at a level that transcends its individual parts.
As an artist, I enjoy making tangible installations. Yet, a significant portion of my time is spent crafting software. Code is the lifeblood – the spirit – that makes projects come to life. With Metaworm, I designed, what we call, a unique tech stack to drive the artwork’s behavior.
By “tech stack”, I mean a set of technologies used to develop the system. Through this custom framework, I’ve created numerous applications that handle different aspects of the piece – e.g. the subsystems. For instance, to control the fans, there’s an application devoted to operating the microcontroller responsible for the relay-circuit; another for processing images from a depth-camera; and yet others for generating graphics, applying effects and adjusting projections on the fabric. An entirely separate application was developed just for controlling the dynamic audio that interacts with and evolves alongside the blowing fabric – it’s a beast!
I developed these systems using several computer languages–C, C# and C++. Ancillary scripts written in Bash, Python, Max, TouchDesigner, JavaScript, GLSL and HLSL enabled me to modify shaders, extend hardware features and automate bootup. Creative coding is often about persisting through many layers to get what you want. It takes some patience. Reflecting back, I see ways to reduce its complexity. I like to experiment and refine with each iteration, which is why you’ll find different versions of nearly everything in my work.
Despite its complexity and success, I must admit, when I first embarked on the journey of making the system – that is – Metaworm, I wasn’t sure if it would work. After all, live projection mapping technology is very nascent, and I didn’t have the tools to help me achieve my vision – I had to make my own. Yet, this is precisely where I thrive. For me, the process of invention is an integral part of my artistic drive. Through the experience of making – including the inevitable failure – I end up stumbling upon something that I find remarkable. Whether it’s a new piece of code, a tool, or a technique, I invariably discover innovations that can be applied in future works. In essence, my experience in making new media art revolves around building creative systems and making discoveries along the way that either inform or lead to new projects.
Waveform No. 2
Preamble
With various interests and backgrounds, I hold a unique perspective on the interplay between nature and culture. For me, the blending of technology, research and art presents an opportunity to explore ideas and find solutions that lead to stronger environmental connectivity.
Through my kinetic installations, I aim to merge cultural, technological and natural systems to create a unique medium of experience. I use data gathered from nature to comprehend the physical forces at play and express them through form, material, poetry and motion. These artistic expressions also draw a connection between the digital, physical, and environmental systems that we interact with daily.
I’m always in search of different ways of experiencing the world around us. By integrating the natural forces and phenomena into our built spaces we can dissolve the barriers that separate us from our natural environments and create a better empathic connection to the ecosystems we depend upon.
Waveform No. 2, a kinetic fabric sculpture, is one such piece of art. Part of a continuing series of works by the collective group Natura Machina, which I co-founded with my friend and artistic collaborator, Kuan Ju Wu in 2015, it probes at the relationship between the built environment and nature. Throughout the series, we contemplate the fusion of ecologically-engaged art with cultural sensibilities regarding nature and the built environment.
The Artwork
Using data derived from nearby ecosystems an assembly of light fabric extrusions gently rise and fall in response to the shifting environment. The structure enables users to observe how seemingly subtle changes can produce powerful results when brought into balance with one another. The result is a poetic dance that gives insight into our relationship with nature and how we might find unique and meaningful ways to integrate its living qualities into our built environment.
Taking inspiration from natural processes, Waveform No. 2 creates an awareness of the environment while remaining consistent with the surrounding architectural language. Commissioned and installed in the Cohen Gallery at Brown University, this site-specific artwork utilizes colors that reflect Spring in its creation. With light, voile fabric and minimal geometric shapes, it provides a visual contrast against the heavy concrete walls and floors while referencing the building’s modern architecture.
The artwork relies on a hidden bank of computer-controlled motors to drive the kinetic animation. Counterweights are used to balance the twelve forms, allowing them to rise and fall with ease. Each form is then lifted by a single monofilament line, while strategically gathered fabric on the ground stabilizes their ascent and prevents any spinning. Thanks to the extra fabric, these floating forms soar upwards of 4 meters (~12ft) and descend as low as 1 meter (~3ft).
The System
Creating this sculpture involved a testing and prototyping process similar to what was required for the Metaworm installation. Simulating the sculpture in a virtual domain played a key role in designing the system’s behavior and communicating its creative intentions. Using a combination of a detailed CAD (computer aided design) model and using creative coding, I was able to animate the sculpture’s waveform motion virtually. As the simulation code evolved, it eventually morphed into the software for the real-life installation, which used sensors to influence its movement. Embracing this process helped me to communicate the artwork more successfully and produce a working system that matched the intent.
The overarching goal of the artist collective Natura Machina is to engage in creative dialogue with natural systems and environments via media art installations. The computationally driven kinetic system we developed for Waveform No. 2 is part of an ongoing body of research that seeks to transform signals from nature into poetic installations that relate audiences to living ecosystems. Our group operates under the pretense that a cultural definition for nature can be reimagined through insightful empathic connections to real living systems. This is a topic we’ve explored in several other publications.
Reflections
Both Metaworm and Waveform No. 2 are examples of new media artworks that rely on computation and systems as inherent to their design. The art – I feel – is what emerges from the interplay of materials, electronics, code and signals. Each of these artworks is part of an ongoing process of invention and experimentation that encircles ideas about how natural systems can more meaningfully intersect our built spaces. As an artist who uses code as a creative medium, I am especially interested in the algorithms – natural and artificial – that drive such sculptures using data derived from outside forces or phenomena. The nature of computation and systems is a viable and interesting way to approach art in the 21st century.
In my upcoming article, “Part 3: Art, Computers, and Systems,” I will introduce several contemporary artists who are redefining the boundaries between art and technology – illustrating how their innovative and experimental approaches have been pushing new media art to new heights. By the end of this series, you will have a deeper insight into the influence of computation and systems on modern art, as well as find inspiration in the expanding field of contemporary new media arts practices.
Credits & Acknowledgements
The Metaworm artwork benefitted from contributions from several others, including the team at Meow Wolf, who originally commissioned the installation. Nancy Good is the artist who produced the mural that surrounds the installation, which is a special UV-reactive dark painting. Additional contributors include Benjamin Wright (Sound Creative Lead); Eric Davis (Technical Integrator); Jeremy Joachim (Software Developer); Jon Haas (Projection Designer); Matthew “Roz” Rosvold (Lighting Designer); Stormi Holt (Artist Assistant); Tellart (Concept Developer).
Waveform No. 2. is part of an ongoing series by the artist collective Natura Machina. The artwork was designed and produced in collaboration with Kuan Ju Wu and Fernando Diaz Smith and was commissioned by Brown University, Providence, RI, USA, in 2016.
