Photo To Playlist
Table of Contents
1. Project Overview
2. App Demo
3. Tools Used & Workflow
4. Data Collection
5. Modeling: NLP Processing
6. Lyrics Based Recommendation
7. Github Repo - Link
1. Project Overview
“Emotion Based music recommendation with facial detection.”
Designed an app that will take in an input image and output a recommended playlist based on emotions of the users.
The 3 goals for this project are:
- Classify Emotions in a Given Picture Real Time
- Use emotions to generate and recommend playlist for the user.
- Create an app that would generate one unique playlist for each picture.
2. App Preview:
Demo for the app Available at:
Link to App Demo
3. Tools Used and Workflow:
Python, Sklearn, TSNE, Word2Vec, Flask, S3, Amazon Rek, EC2, MongoDB, Pandas, Matplotlib and Seaborn, Jupyter Notebook, HTML, CSS.
3. Project Timeline:
—
4. Data Collection
Raw JSON data was collected from AWS Rekognition interface for each picture and saved to MongoDB.
Lyrics Data was scraped from AZLyrics.com and song features were scraped from Spotify. A collection of 350K songs with features were inserted into a MongoDB.
—
5. NLP Processing (Lyrics)
Phase 1: Used GloVe Corpus (Global Vectors for Word Representation - Stanford NLP)
Stanford University NLP Department put the corpus together by extracting word context from 28B tweets Corpus consisted of 1.2M words in 100 Dimensional Vector Space.
I Collected vectors for each word in each song from the 1.2M word corpus. Then, averaged the vectors for all the words in 1 song. As a result, was able to produce the following plot for which each vector on this TSNE plot represents a song.
6. Lyrics Based Recommendation
For each song in a 100D vector space. Used Cosine Distance to calculate the distance between the emotion and the song. Below is the list of emotions used for this project.
Emotion list: Happy, Sad, Angry, Confused, Stunned, Calm, Annoyed
For Example, based on the lyrics one song has a distance associated with each emotion. Lower the distance the better. Using this tactic, I was able to leverage gradiance of emotions to recommend a playlist. As a result, I am able to produce a unique playlist for each picture.
Contact:
Thank you for visiting my blog, If you have any questions free to contact me at smeet.vikani@gmail.com