Creating Music by Machine Learning

© 2020 by Radna Intellectual Ventures, LLC.

In Sweden, there’s something called the Allemansrätten, which literally translates to “Everyman’s Right.”

Allemansrätten gives everyone the right to access, trespass, and camp on any land within the immediate vicinity of a residential dwelling.

Imagine a stranger strolling onto your property in the U.S. or U.K., pitching a tent on your lawn, and proclaiming “Allemansrätten!”, as they roast marshmallows over an open fire.

In Sweden, this is legal and they’re allowed to stay for up to 24 hours.

Allemansrätten is a very Nordic notion of the social contract and it embodies the idea of Trust that is at the root of Swedish culture.

The idea of “public” vs. “private” just doesn’t exist to some extent in Sweden, and thus it is not surprising that Sweden has been a leader in both Music Piracy and Music Innovation.

The Age of the Artist

Music Piracy, with the launch of Napster, defined much of the 2000s and was catalyzed with the adoption of the MP3 technology and the launch of Apple’s iPod in 2001.

Source: Getty Images; Swedish DJ Avicii – the late Tim Bergling – setting precedence for the upcoming “Age of the Artist” of the 2020s.

Even with Napster’s shut down by the Recording Industry Association of America (RIAA) in July 2001, the flood gates were opened and nothing could stop Music Piracy, which proliferated in various forms including Sweden’s own Pirate Bay and uTorrent.

A startling statistic: CD Album Sales topped USD 13.4 Billion in 2000 but by 2014, eroded to just 1.9 Billion, a staggering collapse of 86 percent.

The 2010s brought with it the exponential growth of Music Streaming through services like Apple iTunes, Amazon Music, and Spotify.

However, amidst the sea of change in the Music Distribution industry, the underlying relationship between Artists and Labels remains relatively unchanged.

This brings us to the “Age of the Artist”.

The Age of the Artist represents a structural change in the music industry at the “Top of the Funnel” and will fundamentally shift the relationship between Labels and Artists.

1. Alternative Revenue Models: creative deal structures more aligned with Artists than the traditional model of signing artist rights away to a label into perpetuity.
2. Global Access to Audience: according to legendary VC investor Mary Meeker, 60% of the world’s population is connected to the Internet, and with it access to YouTube, SoundCloud, and a plethora of distribution platforms, allowing any independent artist to share their content and connect with a global audience.
3. Labels as a Service (“LaaS”): similar to the idea of SaaS (“software as a service”) – the subscription-based business model that has disrupted the software industry – I believe LaaS will gain momentum, with Labels becoming more commoditized.

With the rapid expansion of creation, collaboration, production, marketing, and distribution tools, the effect is a shift of bargaining power from Labels to Artists.

So the Age of the Artist will be defined by a paradigm shift in companies establishing relationships with artists and songwriters during the creation process, setting up the stage for the new label of the future.

Spotify – one of the key disrupters in the $22 Billion Recorded Music industry – agrees with this thesis which we will highlight through their Corporate Development strategy below.

But before we do, let’s do a quick primer on Product Development and specifically the development of products whose core tech is based on Machine Learning.

Product Development in the Age of Machine Learning

One of my first roles after graduating from Cornell University and U.C. Berkeley with degrees in Computer Science was in Product Management at a financial services firm.

Back in the early 2010s, “Data Science” and “AI / ML” were still relatively new words in the financial industry, yet our group leverage “Big Data” and utilized Machine Learning algorithms to estimate movements in certain financial markets with remarkable precision.

The idea of an “AI Product Manager” simply didn’t exist back then, yet that was what our group was doing on a daily basis, which allowed us to generate record profits among our competitors in the industry.

Fast forward a decade and the idea of AI Product Management or APM is a clearly defined role in Tech, with the largest tech companies like Amazon, Facebook, and Google even having dedicated APM teams.

Source: Radna Intellectual Ventures; Traditional Product Development using Wireframes.

What is the role of an AI Product Manager?

The AI Product Manager is principally responsible for using AI, Deep Learning, and Machine Learning to enhance, improve, create, and shape products.

Artificial Intelligence has far-reaching implications for every conceivable industry according to a seminal report by McKinsey and Company, which identified over 400 business use cases of AI for companies today.

Another recent survey of global business leaders and corporate CEOs indicated that over 70% have started AI initiatives. With the proliferation of AI into business (e.g., Google Search, Amazon Product Recommendations, Tesla Autopilot), Machine Learning is the common thread among all these novel products.

Andrew Ng – a Stanford Professor and one of the leading authorities in Machine Learning – developed the concept of the “Virtuous Cycle of AI” – an idea central to the development of Machine Learning products.

Source: Andrew Ng; Landing AI’s Playbook featuring the Virtuous Cycle of AI.

Andrew’s insight that “building a better AI product requires more users producing more data” has profound implications for the Product Development Life Cycle.

Why does this matter?

Well, products fueled by data and machine learning can be remarkably powerful tools to discover and solve customer problems, enabling a company to strengthen its business moat amidst an increasingly competitive environment with lower barriers of entry.

As the diagram below shows, Product Development can be broken into four components:

Source: Radna Intellectual Ventures; Product Development in the Age of Machine Learning.

From the perspective of AI / ML products, here is what these four components really mean.

1. User Experience: this is where user problems are discovered and validated. As Marc Andreessen of the Venture Capital firm A16Z, notes in “It’s Time to Build”, innovation is sorely missing in Tech. UX / UI – beyond providing an attractive and intuitive user interface – is about discovering what the user really wants and validating the problem, which often leads to product innovation.
2. Technical Development: this covers both software or hardware development. Especially relevant for software is the Agile Software Development methodology, which is particularly well-suited for AI / ML product development.
3. Data Science: although Data Science is a term that is often associated with buzzwords like “Cloud Computing” and “Big Data”, it has been a concrete field used by the best innovators for decades. Leveraging Machine Learning Algorithms and Big Data, one can truly achieve the extraordinary – see Jim Simmons of Renaissance Technologies– the most successful investor in history (in fact, twice as good as the legendary Warren Buffet as measured by returns).
4. Product Management: a Product Manager is akin to the Conductor of an Orchestra. Product Management is an iterative process of gathering user data to build better products, which generates more novel data (see Virtuous Cycle above). An exceptional Product Manager, especially in AI / ML, is someone with a balance of technical expertise + business acumen + high customer empathy.

Given the above, another valuable insight from Andrew Ng is that “Training Sets are the new Wireframes in product management” – a deep observation which you can read more about in Why AI is the New Electricity.

With an understanding of Product Development in the Age of Machine Learning, let’s exercise our knowledge through a Case Study for one of the most remarkable Machine Learning companies that exist today: Spotify.

Case Study: Spotify

Source: Getty Images; Spotify’s Founder and CEO, Daniel Ek.

I’d argue Spotify’s history is much more remarkable than Facebook’s story.

However, largely as a function of its Swedish origin, Spotify has been less glamorized than Facebook, as epitomized in the popular American film “The Social Network”.

What were the humble beginnings of Spotify?

It began in the mind of a music-loving young teenager and prodigy, Daniel Ek.

By the age of nine, Daniel taught himself how to code C++ – one of the more challenging programming languages to master – and wrote 40 pages of code, constructing a cursor to move around the screen.

Daniel continued to hone his coding skills and by age 14 in 1997, Daniel started building websites for other Swedish entrepreneurs, charging a rate of SEK 10,000 per website.

With this first successful business venture, Daniel caught the entrepreneurial bug, planting the seed for the conception and launch of Spotify nine years later in 2006.

If you don’t know or use Spotify, I’d urge you to stop here.

As a Zen practitioner, I jokingly compare the Spotify music experience to the journey of discovering Zen, as they are both experiential in nature – we can talk about both, but you must experience them for yourselves in order to truly understand.

Okay, let’s move on to the secret sauce behind Spotify, which relies on Machine Learning.

Machine Learning at Spotify

Source: Radna Intellectual Ventures; Spotify Paved Road Machine Learning Infrastructure.

Although Spotify has over 50 million songs, it has a staggering 3 Billion Playlists or 60 times more playlists than songs.

How do we wrap our heads around this?

From a Machine Learning perspective, the answer should be clear if you have read the prequel to this article: “The VC’s Guide to Machine Learning”.

If you haven’t, don’t panic.

Spotify considers each Playlist as a state space consisting of individual states representing a single song. By leveraging reinforcement learning techniques and user data, Spotify is able to generate novel “Journeys” / Playlists through the vast 50-million dimensional vector space of songs.

What does that mean in plain English?

Simply stated, Spotify’s content recommendation and playlist generation rely heavily on state-of-the-art machine learning algorithms to generate remarkably impactful music recommendations for its 286 Million Monthly Active Users (“MAUs”).

Of note, the development of Machine Learning at Spotify has closely tracked academic research into content recommenders and neural networks and from the early days of Collaborative Filtering in 2006 to the latest in Generative Adversarial Networks (GANs), with coherent music generation possible through LTSM (Long Short-Term Memory)

You can read more about the technical details of how Spotify’s “Paved Road” machine learning infrastructure is implemented through a collection of open-source software platforms like Google Cloud, TensorFlow and Kubelflow.

Let’s do a quick comparison with Netflix – with a market cap of 200 Billion is about 5 times larger than that of Spotify – whose content recommendation system is also a byproduct of Machine Learning.

According a recent article by Spotify’s VP of Engineering: “Netflix has about 158 million users, and at Spotify, we have 248 million. On top of that Netflix has around 5,800 movies and shows across all territories, while Spotify has more than 50 million music tracks and 500,000 podcast titles.”

From a computational complexity perspective, Spotify’s real-time complexity requirements are an order of magnitude larger than that of NetFlix and quickly growing – see the observation above about Spotify’s Playlist generation.

With its Paved Road approach, Spotify has indeed built one of Tech’s most robust and efficient Machine Learning infrastructure.

Let’s now shift gears from Product to Business.

Corporate Development at Spotify

Corporate Development is an ambiguous industry term, but generally for larger companies, it relates to activities like acquiring other companies (M&A), securing corporate financing, divesting of assets or divisions, and management of the intellectual property (IP).

Among the Tech Decacorns (companies with a valuation greater than $10 Billion), Spotify has perhaps one of the most active Corporate Development teams in the industry, with a large part of that activity focused on acquiring startups with novel music tech.

In a similar vein to what we did in “The VC’s Guide to Machine Learning”, we have mapped Spotify’s startup acquisitions with investable themes in the AudioTech industry.

Source: Radna Intellectual Ventures; Investable Themes in AudioTech: Connecting Spotify’s Strategic Vision with their Acquisition Strategy.

Although music continues to be their core offering, Spotify’s strategic vision is to expand into Audio in general like Podcasting – a thesis reinforced by their numerous acquisitions of Podcasting startups like Anchor, Parcast and Gimlet.

From a thematic perspective, I expect Spotify’s leadership – principally driven by Daniel Ek and his Lieutenant Gustav Soderstrom – to shift more investment capital to Content Distribution / Licensing / Marketing and “Exploration”.

I define Exploration broadly as tools and services that enable music (and audio in general) listeners to (1) discover new content and (2) experience audio content in a novel manner.

Novel in what way?

With the coming of age of Virtual Reality and Augmented Reality (which is a more challenging problem from a technical perspective), I can see virtual concerts and live “jam sessions” being an active area of investment for Spotify this decade.

Returning now to Allemansrätten where we started.

Final Thoughts

As you may recall Allemansrätten or “Everyman’s Right” is a core tenet of Swedish culture and I believe it is also a core tenet of Decentralization during this Roaring 2020s.

Decentralization – with Blockchain as the enabling technology – will play a key role during this Fourth Industrial Revolution, and unless you get on board the “Allemansrätten train”, you will very likely be missing out on the next Trillion dollar opportunity in Tech.

So if you enjoy the scenic train view, let’s continue Creating Music by Machine Learning.

A good composer does not imitate; he steals.

—Igor Stravinsky

By Andrew Vo, CFA

Investor at Radna Intellectual Ventures

About Radna Intellectual Ventures

Radna Intellectual Ventures is a venture startup studio with a mission to enable the next generation of Deep Technologies or Deep Tech companies, with a focus on Machine Learning, Natural Language Processing, and Blockchain technologies.

Our General Partner, Andrew Vo, spent a decade in finance, working at some of the largest investment managers like J.P. Morgan, before pursuing the entrepreneurial path. Andrew serves as an Advisor for startup companies focusing on the FinTech and DeepTech sectors. Andrew is a CFA Charterholder and holds a Masters of Science in Computer Engineering from Cornell University, and a Bachelors of Science in Electrical Engineering and Computer Science from the University of California, Berkeley.