Music Recommender System — Part 6
Deploy ML Based Recommender System into Production
Recap
In the previous article, we created a few machine learning models to find out the best recommendations based on user Spotify favorite music or playlist.
This article documents how we can put it all together for public use. We don’t want our ML models to die in a Jupyter Notebook. We want our models to be integrated into a product/application and available to any user. The complete deployment process involves three major steps.
- API which can easily access the machine learning models.
- Front-end application which will allow the users to access the predictions.
- Cloud/Server where we will deploy our application.
Creating API to access the ML model predictions
In order to access the results from the machine learning models through web app, first we cleaned our code and wrote clean functions for each task/result that we want to show in the app. We decided to have two options to take input from users in the web app. The first one is to take any Spotify playlist and the second one is to take user favorites, this option requires the user to autorize our web app using his Spotify credentials. For both these options, we collect the list of the songs and extract the audio features for each song. Then we use these audio features as test data for our alredy trained machine learning model to predict the cluster that the user belongs to. After finding the cluster, we extract the top playlists that are either similar or dissimilar to the user playlist/favorites. Then, we show the top songs from these top playlists as our recommendations. The details of complete modeling and prediction is explained in our previous article.
We created a SpotifyRecommender Python class which has all these tasks mentioned above as separate function. We used this class as the API to our frontend application. The frontend application will pass the trained machine learning model and either the playlist url or user name as inputs to the API and it will receive the top recommended songs as output. We added a few more functions to this Python class to create the necessary plots. The example plots are here:
Frontend Application
Now we have clean, modularized code and we need an easy way for any user to interact with our machine learning model. There are many frameworks to create a website, but we want to spend as less time as possible on this part because it’s not mandatory for our project. So, we decided to use Streamlit Python library. It allows to create web applications in Python without having to know anything about creating websites or complex web frameworks. They also have great documentation, you can check it out here. Now, Let’s see how to create a simple web app for our recommender system.
The first step to create any website is to decide the layout and it’s features. After watching a few Streamlit app examples, we decided to have sidebar which will have some buttons to take the user to different pages. We also decided to have pages which include information about the dataset that we used, user inputs for machine learning model, our recommendations, conclusions and also links to all these blog posts.
After deciding the layout and content in each page, the next step is to install the required Streamlit library for frontend. We already installed this as part of creating development environment using requirements.txt file. If it’s not, you can install using pip as shown below:
pip install streamlitNext, we created a file app.py in streamlit folder and wrote all the frontend related code in this file. In this article, I show the minimum code and important features/improvements that we included in our app. If you want to see the complete code, it’s there in the GitHub repo here. We also defined some styling for the buttons using style.css file as shown below:
body {
color: rgb(45, 49, 45);
background-color: #22222218;
}
.stButton>button {
width: 100%;
display: inline-flex;
align-items: center;
justify-content: center;
background-color: rgb(255, 255, 255);
color: rgb(38, 39, 48);
padding: .25rem .75rem;
position: relative;
text-decoration: none;
border-radius: 4px;
border-width: 1px;
border-style: solid;
border-color: aquamarine;
border-image: initial;
}
.stButton>button:hover {
border-color: rgb(246, 51, 102);
color: rgb(246, 51, 102);
}
.stButton>button:active{
box-shadow: none;
background-color: rgb(246, 51, 102);
color: white;
}From the documentation, we understand that Streamlit reruns the script from top to bottom every time we interact with the app and each rerun takes place in a blank slate. So, no variables defined in the code are shared between runs. We found out that it provides an option called Session State, which helps to share variables between reruns, for each user session. In addition to the ability to store and persist state, Streamlit also exposes the ability to manipulate state variables using Callbacks from button clocks or input changes. We used this option to show the correct page information based on the page selected. The sample code is shown below:
import streamlit as st# Sidebar
def spr_sidebar():
with st.sidebar:
st.info('**Spotify Playlist Recommender**')
home_button = st.button("Home")
blog_button = st.button('Blog Posts')
if home_button:
st.session_state.app_mode = 'home'
if blog_button:
st.session_state.app_mode = 'blog'def main():
spr_sidebar()
if st.session_state.app_mode == 'home':
home_page()
if st.session_state.app_mode == 'blog':
blog_page()
spr_footer()def spr_footer():
st.markdown('---')
st.markdown('© Copyright 2021 - Spotify Playlist Recommender')
As part of our web app, we wanted to include the Spotify web player in the page. For this, we used Streamlit Component API to load Spotify playlist URL in an iframe as shown below:
import streamlit as st
import streamlit.components.v1 as componentsplaylist_uri = '37i9dQZF1DX0kbJZpiYdZl'
uri_link = 'https://open.spotify.com/embed/playlist/' + playlist_uri
components.iframe(uri_link, height=300)
In order to optimize the performation to predict the recommendations for different user inputs, we loaded the machine learning model using the Streamlit caching mechanism which allows the app to stay performant even when loading data from the web, manipulating large datasets, or performing expensive computations. This is done with the @st.cachedecorator and it is global to all users.
import streamlit as st@st.cache(suppress_st_warning=True)
def load_spr_ml_model():
st.session_state.ml_model = SPR_ML_Model()
The last update that I want to mention is about SPOTIPY_CLIENT_ID and SPOTIPY_CLIENT_SECRET that we defined in Part 2. We don’t want these to be directly visible to the public, so we created “.streamlit” folder, added these variables to “secrets.toml” file as shown below and added this file to “.gitignore” so that it will not be pushed to GitHub.
# Spotify Credentials
SPOTIPY_CLIENT_ID = 'Replace with your id'
SPOTIPY_CLIENT_SECRET = 'Replace with your secret'
# Needed for user authorization
SPOTIPY_REDIRECT_URI = 'http://localhost:8080'In our web app, these variables are considered as environment variable and the spotipy API calls worked fine. If we need to access them in Python code, we can use SPOTIPY_CLIENT_ID as st.secrets[“SPOTIPY_CLIENT_ID”].
Once we finish writing code for all pages, we tested thoroughly in our development environment and we pushed all the code to GitHub repository.
Cloud/Server
Up to this point, all of our work has been on our development workspace. We have a front-end web app running in our workspace that allows us to access our machine learning model predictions. We want to extend and everyone to have the access to this web app! We can containerize our application using Docker and deploy to any of the cloud providers such as AWS, Google Cloud, and Microsoft Azure. But that costs some credits/money to run the Docker image all the day. For this project, we wanted to try Streamlit Cloud which provides a platform to deploy and share unlimited public apps. The deployment process is straight forward.
We login on Streamlit Cloud and allow it to access GitHub repositories. Once we give the repo access, click on New app with existing repo and fill the details as shown in below picture with the repository details and Main file path.
Then in the “Advanced settings…”, we entered the information from “secrets.toml” file mentioned above. To know the usage of Streamlit Secrets in detail, check this blog. Once we press the Deploy button, Streamlit launched an instance and clone all the code from GitHub and ran the file mentioned in “Main file path”. Finally, our web app is ready with public URL that anyone can access. Below is the demo video of our final web application.
If you enjoy reading my articles and want to support me, please consider signing up to become a Medium member. It’s $5 a month and gives you unlimited access to stories on Medium. Please signup using my link to support me: https://nsanka.medium.com/membership.
