Developer Diaries: Uniting musicians with AI and IBM Cloud Functions

Choirless is an AI-powered web-based serverless platform that lets choirs, bands, and any kind of performer rehearse and perform together without any of that latency or syncing up business that you have to put up with in video calling software. Each singer’s contributed videos are automatically synchronized and stitched together to create a song that sounds as if the musicians were all in the same room.

As part of last year’s Call for Code Global Challenge, we created Choirless and were runners-up in the 2020 Call for Code IBMer Challenge. To build such an ambitious application we utilized key IBM Cloud and AI solutions to bring our idea to life.

You can learn more about this solution and the relationship between music and AI at this crowdcast virtual event, on March 16 (or catch the replay). You can read more about this solution in their technical article, GRAMMY Debates with Watson: From the Lab to Music’s Biggest Night.

AI

Choirless aims to make it very easy for users to create songs made out of several parts (e.g. alto, tenor, soprano) and to organize choir, band and other members to provide renditions of a part. All of the contributed videos are stitched together into a video wall with no special equipment and without employing costly and time-consuming video editing software. To do so, Choirless uses machine learning to measure the spectral flux of each instrument or voice. The AI recognizes where the musicians change notes and then aligns the peaks and troughs so it sounds as if musicians had recorded the song together. The AI can keep layering new sounds, using the same method.

Cloud

The rendering of the different recorded pieces of audio requires a lot of computing power. Choirless is built on open-source technology, using Apache OpenWhisk, Apache CouchDB, Cloud Foundry, Docker, Python, Node.js and FFmpe. In the backend, all of these recordings are processed in parallel using IBM Cloud Functions, making processing faster and letting the app effortlessly scale with an increase in demand. Choirless runs on the IBM Cloud and uses the same kind of acoustic analysis that helps the US Masters golf tournament pull highlights.

The code for Choirless’ front-end website, back-end API and serverless rendering pipeline is available on GitHub for developers to contribute or embed into their own solutions.

The Choirless community grows

Beyond the pandemic, Choirless has the potential to allow for ongoing musical collaboration without borders.

As of January, users have recorded 2,740 individual parts forming 745 distinct performances. For almost a century, we’ve relied on the recorded music industry as the dominant player when it comes to music consumption.

The philosophy underpinning Choirless takes us to a new place where the idea of a single immutable version of a song is questioned, as is the idea of a ‘band’ or entity coming together and owning a piece of music. This can be likened to the emergence of open source software where the community cares more about creation and collaboration than they do around questions of ownership.

The open source Choirless project is now supported by the Linux Foundation.

Call for Code

Call for Code is the largest and most ambitious effort bringing together developers to take on pressing societal issues using open source technology. The need to apply the power of hybrid cloud, AI, and open source technology to address society’s most pressing issues is greater than ever. The employee challenge is a parallel Call for Code challenge that gives IBMers, who are not eligible to participate in the global challenge, a chance to get involved in Call for Code. More than fifteen thousand solutions have been created through Call for Code, using technology including Red Hat OpenShift, IBM Cloud, IBM Watson, IBM Blockchain, data from The Weather Company, and APIs from ecosystem partners.

Learn more

The capabilities for building industry-shifting applications are endless. Learn more about the code behind the Choirless and about going serverless with IBM Cloud Functions.

Developer diaries

Watch more IBM Developer Diaries.