The ARpeggiator is a collaborative AR music-making application that works in Chrome on Android devices allowing multiple users to make music together in real-time by creating spherical notes in AR space.
The spherical notes created by users will be played in a loop while they are in the camera’s view. The notes will be played in the order they are created. The vertical position of the note determines the pitch, and the distance between the note and the camera determines the loudness. Tilting the device along the x-axis controls the low-pass filter frequency, while the z-axis controls the decay of notes.
When another user joins, the notes created by other users will be visible to the newly joined user and they will also see each other’s virtual phone in AR space. Since only the notes that are in the camera’s view will be played, each user can play a different set of notes even though they are in the same AR space. This can generate interesting polyphonic/polyrhythmic patterns when two or more users share each other’s audio by visiting the app link in a desktop Chrome browser or play together in the same physical space.
The performance will consist of two performers using the app in different areas but their performance will be shared online in real-time, so the audience will be able to see/hear the mixed output. And since anyone who has an Android device can use the app, the audience can join the performance as well.
The ARpeggiator was created with Node.js, Express, Three.js, WebXR, Socket.io, Pure Data, and Emscripten and it is open source on Github
Zack Lee is a graduate student at NYU’s ITP program with a focus on creative technologies such as AR/VR, UI/UX design, and web development.
He has over five years of software development experience focusing on mobile applications and open-source tool development. Previously Zack was a co-founder of a mobile music creation application startup called Iceblink Digital, Inc. which won numerous prizes and awards including the 3rd Prize in Creative Business Cup 2015. More recently, he built an open-source creative coding tool called Ofelia for multimedia application development that has been used by many companies and individuals for various projects. Currently, he is working on making an open-source software Purr Data run in a web browser through the Google Summer of Code program.
Zack continues to explore new ways to support and inspire creative minds including himself.
https://github.com/cuinjune/ar-peggiator
Back to
