Motion to Art: The Intersection of Body Capture and Interactive Digital Media

ResoFlow is an interactive installation that captures participants' body motion through webcam-based pose estimation and transforms gestures into dynamic, fluid visualizations and responsive soundscapes. Real-time skeletal tracking is achieved via the PoseNet model in ml5.js, integrated within a p5.js creative coding framework. Joint coordinates drive a fluid particle system flowing through a Perlin noise – based vector field, generating colorful, decaying motion trails that reflect user movement. Simultaneously, audio parameters are mapped to wrist positions, modulating volume, playback rate, and stereo panning to create an ambient, interactive sound environment. Developed through iterative prototyping and refined in a gallery exhibition, ResoFlow emphasizes intuitive cause-and-effect interactions, requiring no traditional controllers. The installation explores themes of human-computer interaction and positions users as artists within a responsive digital canvas.

Accessibility Statement

Purchase College - State University of New York (PC) is committed to ensuring that people with disabilities have an opportunity equal to that of their nondisabled peers to participate in the College's programs, benefits, and services, including those delivered through electronic and information technology. If you encounter an access barrier with a specific item and have a remediation request, please contact lib.ir@purchase.edu.

Motion to Art: The Intersection of Body Capture and Interactive Digital Media

Journal Title

Keywords

Readers/Advisors

Journal Title

Term and Year

Publication Date

Book Title

Publication Volume

Publication Issue

Publication Begin

Publication End

Number of pages

Collections

Files

Research Projects

Organizational Units

Journal Issue

URI

Abstract

Citation

DOI

Description

Accessibility Statement

Embedded videos