MusicMatch: Using Spotify API and Machine Learning for Audio Feature Visualizations and Song Recommendations
Andrew Nickerson, Catherine Huang | December 13, 2021
With over 380 million monthly active users, Spotify is the world’s largest music and media streaming service provider. Many of us know Spotify as the go-to platform on which to unwind during study breaks, share playlists with friends, discover new music, and—for the ambitious among us—land among the top 0.1% of active listen- ers for our top artists. From an engineering and technology standpoint, however, how does Spotify provide the aforementioned features and experiences for its users? Specifically, how does Spotify use technology to capture an impression of each user’s music taste, and how does it generate song recommendations based on that impression? Fortunately, Spotify’s open-source Web API (Applica- tion Programming Interface) (https://developer.spotify.com/) helps us get one step closer towards exploring these questions. This API allows any programmer to perform functionalities similar to what Spotify’s web and mobile applications perform every instant. One can explore metadata about songs, playlists, users, or artists (for example, a user’s top tracks, an artist’s top albums, or a song’s audio features) with the API. Given that most other companies do not make the technology behind their products open to the public, we were eager to explore Spotify’s Web API to get a sense of what lay behind the scenes of a service whose underpinnings are at the intersection of music and technology. In this project, we set out to 1) visualize the audio features of any playlist by any user, as well as 2) generate song recommendations for that user given the playlist.
There are several ways to use the API, for example, either through a node.js web server or through Spotipy, a Python wrapper for the API. We chose the latter, since 1) it allowed us to focus directly on the API’s features rather than setting up a complicated web environment, and 2) Python is generally a more succinct and versatile langauge to work with than JavaScript. To call Spotipy, we worked on Jupyter Notebooks hosted on DeepNote, a cloud-based notebook editing and execution environment. In order to visualize audio features and generate song recommendations, we used the Web API to first fetch metadata about a user’s playlists and the songs in each playlist (given a username), and then we used various machine learning and data visualization techniques to handle the rest. Regarding the latter, Spotify API already supports generating song recommendations given a set of audio features, but we intentionally wanted to explore the feature representation, clustering, and machine learning behind creating such a recommendation algorithms. Our algorithm combines information from web sources (as initial inspiration) with our existing programming, machine learning, and musical knowledge.
In this paper, we first discuss the Spotify Web API and its features in detail, as well as how we used the API. We then present our code, and this includes a walkthrough of all three Jupyter Notebooks we worked on, as well as a conceptual overview of our recommendation algorithm and of the clustering and dimensionality reduction techniques we use to generate two-dimensional visualizations of which songs related to each other and which are not. In the section after that, we discuss how our work both draws from and connects to topics covered in GENED 1080. Lastly, we will analyze and reflect upon this project and the process of working on it.