Description

Vision Synth is an innovative and experimental program that transforms hand movements into musical expressions, creating a unique interactive experience. It leverages a vision-based neural network with pretrained weights, utilizing the YOLO hand detection model, to accurately detect hand positions and sizes through a webcam.

About the Prototype

Vision Synth uses a neural network, based on YOLO hand detection, to track hand movements through a webcam, translating them into musical parameters. This interaction allows users to control pitches, filters, and oscillators by simply moving their hands, supporting multiple musical scales for varied sound creation. The program is an experimental blend of motion and music, offering an interactive experience that turns hand gestures into a dynamic and immersive musical journey.