Music Box VR
Role: 3D Modelling, VR Design, Music Production
Music Box is a virtual reality application made for the Oculus Go that I created, along with my partner Soren Thompson, for the final project of CS 29/89: AR/VR Design and Development at Dartmouth. It is a song-writing tool where users can interactively create sounds and melodies in an immersive atmosphere.
Inspiration
The inspiration of this project was the Google Chrome Music Lab where users can choose different instruments and play notes on a loop, creating melodies and rhythms. Both Soren and I wanted to create this experience in VR because music can be a very immersive form of art and we wanted to merge that with immersive tech like VR.
Chrome Melody Maker
Challenges
-
User experience - how do users interact with the song writer while utilizing the 3D space?
-
How might we visually represent sounds and notes?
-
How might we allow flexible song setups and allow users to manipulate complex controls easily?
-
How might we create a an effective and intuitive interface?
Modelling sound
Each instrument in Music Box is represented by a different note model, meant to be a visual representation of the instrument’s sound qualities. Being a musician myself, I created artistic 3D renderings of what I thought the three-dimensional embodiment of certain musical sounds could be.
From top to bottom: Tines, Electric Guitar, Piano, French Horn
Navigating the environment
In virtual space, a user is able to select notes by interacting with a three-dimensional grid in the shape of a cylinder surrounding the player. The grid axes (in order of width, height, depth) define the selected note’s placement in the song, the pitch of the note, and the instrument used to create the note, respectively. The sound of each instrument is represented by a different note model. A miniature copy of the song ‘barrel’ is located inside the writing area, so a player may see the entirety of their current song without stepping outside the main area.
The user is able to select the notes and make a music pattern in any way he/she wants. Hitting the back button on the Oculus Go controller pops up a menu through which the user can play/pause the song, and even increase/decrease the tempo. To change instruments, the user can hit the up/down buttons. This also changes the layer of notes as the layer for the selected instrument comes to the front. The player will hear a preview of each note when they select it and can hit it again to deselect it.
Design features
-
Note selection and feedback: hitting a note on the surrounding wall plays that particular note depending on the instrument selected and animates the popping up of the note.
-
Miniature preview: a mini version of the entire cylindrical music maker is placed at the bottom for the user to see what they are making.
-
Options menu: an intuitive user interface to start/stop the melody after the user has created a pattern of notes. The user can also change the tempo of the melody (ie. make it faster or slower).
The musical notes are in the key of C and a user can make any melody using those notes. To obtain the note sounds, I connected a standard MIDI keyboard to Ableton Live 9 and applied different effects to mimic specific instruments.
Showcasing
Music Box VR was presented at Technigala ‘19, which is a tech event at Dartmouth where many students publicly showcase their latest projects. All projects are interactive and all users experience the projects first-hand. Music Box VR was one of the most popular applications at the event, with many faculty, students and members of the tech industry complementing the immersive and artistic qualities of the project.
