A survey of music similarity and recommendation from music context data

Authors : Peter Knees , Markus Schedl Authors Info & Claims

Article No.: 2, Pages 1 - 21 Published : 27 December 2013 Publication History 88 citation 5,839 Downloads Total Citations 88 Total Downloads 5,839 Last 12 Months 569 Last 6 weeks 50 Get Citation Alerts

New Citation Alert added!

This alert has been successfully added and will be sent to: You will be notified whenever a record that you have chosen has been cited.

To manage your alert preferences, click on the button below. Manage my Alerts

New Citation Alert!

Abstract

In this survey article, we give an overview of methods for music similarity estimation and music recommendation based on music context data. Unlike approaches that rely on music content and have been researched for almost two decades, music-context-based (or contextual) approaches to music retrieval are a quite recent field of research within music information retrieval (MIR). Contextual data refers to all music-relevant information that is not included in the audio signal itself. In this article, we focus on contextual aspects of music primarily accessible through web technology. We discuss different sources of context-based data for individual music pieces and for music artists. We summarize various approaches for constructing similarity measures based on the collaborative or cultural knowledge incorporated into these data sources. In particular, we identify and review three main types of context-based similarity approaches: text-retrieval-based approaches (relying on web-texts, tags, or lyrics), co-occurrence-based approaches (relying on playlists, page counts, microblogs, or peer-to-peer-networks), and approaches based on user ratings or listening habits. This article elaborates the characteristics of the presented context-based measures and discusses their strengths as well as their weaknesses.

References

Aizenberg, N., Koren, Y., and Somekh, O. 2012. Build your own music recommender by modeling internet radio streams. In Proceedings of the 21st International Conference on World Wide Web. ACM, New York, 1--10.

Aucouturier, J.-J. and Pachet, F. 2002. Scaling up music playlist generation. In Proceedings of the IEEE International Conference on Multimedia and Expo (ICME'02). 105--108.

Aucouturier, J.-J., Pachet, F., Roy, P., and Beurivé, A. 2007. Signal + Context = Better Classification. In Proceedings of the 8th International Conference on Music Information Retrieval (ISMIR'07).

Aucouturier, J.-J., Pachet, F., and Sandler, M. 2005. “The Way It Sounds”: Timbre models for analysis and retrieval of music signals. IEEE Trans. Multimed. 7, 6, 1028--1035.

Baccigalupo, C., Plaza, E., and Donaldson, J. 2008. Uncovering affinity of artists to multiple genres from social behaviour data. In Proceedings of the 9th International Conference on Music Information Retrieval (ISMIR'08).

Barrington, L., Turnbull, D., Yazdani, M., and Lanckriet, G. 2009. Combining audio content and social context for semantic music discovery. In Proceedings of the 32nd ACM SIGIR.

Baumann, S. and Hummel, O. 2003. Using cultural metadata for artist recommendation. In Proceedings of the 3rd International Conference on Web Delivering of Music (WEDELMUSIC'03).

Bell, R. M. and Koren, Y. 2007. Lessons from the Netflix prize challenge. SIGKDD Explorat. 9, 2, 75--79.

Berenzweig, A., Logan, B., Ellis, D. P., and Whitman, B. 2003. A large-scale evaluation of acoustic and subjective music similarity measures. In Proceedings of the 4th International Conference on Music Information Retrieval (ISMIR'03).

Brill, E. 1992. A simple rule-based part of speech tagger. In Proceedings of the 3rd Conference on Applied Natural Language Processing. 152--155.

Brochu, E., de Freitas, N., and Bao, K. 2003. The sound of an album cover: Probabilistic multimedia and IR. In Proceedings of the 9th International Workshop on Artificial Intelligence and Statistics.

Cano, P. and Koppenberger, M. 2004. The emergence of complex network patterns in music artist networks. In Proceedings of the 5th International Symposium on Music Information Retrieval (ISMIR'04). 466--469.

Casey, M. A., Veltkamp, R., Goto, M., Leman, M., Rhodes, C., and Slaney, M. 2008. Content-based music information retrieval: Current directions and future challenges. Proc. IEEE 96, 668--696.

Celma, O. 2008. Music recommendation and discovery in the long tail. Ph.D. thesis, Universitat Pompeu Fabra, Barcelona, Spain.

Celma, O., Cano, P., and Herrera, P. 2006. SearchSounds: An audio crawler focused on weblogs. In Proceedings of the 7th International Conference on Music Information Retrieval (ISMIR'06).

Celma, O. and Lamere, P. 2007. ISMIR 2007 tutorial: Music recommendation. http://www.dtic.upf.edu/˜ocelma/Music RecommendationTutorial-ISMIR2007/ (last accessed: April 2013).

Charniak, E. 1997. Statistical techniques for natural language parsing. AI Magazine 18, 33--44.

Chen, S., Moore, J., Turnbull, D., and Joachims, T. 2012. Playlist prediction via metric embedding. In Proceedings of the 18th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining. ACM, 714--722.

Cohen, W. W. and Fan, W. 2000. Web-collaborative filtering: Recommending music by crawling the web. Computer Netw. 33, 1--6, 685--698.

Deerwester, S., Dumais, S. T., Furnas, G. W., Landauer, T. K., and Harshman, R. 1990. Indexing by latent semantic analysis. J. ASIS 41, 391--407.

Dror, G., Koenigstein, N., and Koren, Y. 2011a. Yahoo! music recommendations: Modeling music ratings with temporal dynamics and item taxonomy. In Proceedings of the 5th ACM Conference on Recommender Systems. ACM. 165--172.

Dror, G., Koenigstein, N., Koren, Y., and Weimer, M. 2011b. The Yahoo! music dataset and KDD-Cup'11. J. Mach. Learn. Res. 8, 3--18.

Eck, D., Lamere, P., Bertin-Mahieux, T., and Green, S. 2008. Automatic generation of social tags for music recommendation. In Advances in Neural Information Processing Systems 20 (NIPS'07). MIT Press.

Ellis, D. P., Whitman, B., Berenzweig, A., and Lawrence, S. 2002. The quest for ground truth in musical artist similarity. In Proceedings of 3rd International Conference on Music Information Retrieval (ISMIR'02).

Fields, B., Casey, M., Jacobson, K., and Sandler, M. 2008. Do you sound like your friends? Exploring artist similarity via artist social network relationships and audio signal processing. In Proceedings of the International Computer Music Conference (ICMC'08).

Geleijnse, G., Schedl, M., and Knees, P. 2007. The quest for ground truth in musical artist tagging in the social web era. In Proceedings of the 8th International Conference on Music Information Retrieval (ISMIR'07).

Hofmann, T. 1999. Probabilistic latent semantic analysis. In Proceedings of Uncertainty in Artificial Intelligence (UAI).

Hu, X., Downie, J. S., and Ehmann, A. F. 2009. Lyric text mining in music mood classification. In Proceedings of the 10th International Society for Music Information Retrieval Conference (ISMIR'09).

Hu, X., Downie, J. S., West, K., and Ehmann, A. 2005. Mining music reviews: Promising preliminary results. In Proceedings of the 6th International Conference on Music Information Retrieval (ISMIR'05).

Jacobson, K., Fields, B., and Sandler, M. 2008. Using audio analysis and network structure to identify communities in on-line social networks of artists. In Proceedings of the 9th International Conference on Music Information Retrieval (ISMIR'08).

Kim, J. H., Tomasik, B., and Turnbull, D. 2009. Using artist similarity to propagate semantic information. In Proceedings of the 10th International Society for Music Information Retrieval Conference (ISMIR'09).

Kleedorfer, F., Knees, P., and Pohle, T. 2008. Oh Oh Oh Whoah! Towards automatic topic detection in song lyrics. In Proceedings of the 9th International Conference on Music Information Retrieval (ISMIR'08). 287--292.

Knees, P., Pampalk, E., and Widmer, G. 2004. Artist classification with web-based data. In Proceedings of the 5th International Symposium on Music Information Retrieval (ISMIR'04). 517--524.

Knees, P., Pohle, T., Schedl, M., and Widmer, G. 2006a. Combining audio-based similarity with web-based data to accelerate automatic music playlist generation. In Proceedings of the 8th ACM SIGMM International Workshop on Multimedia Information Retrieval (MIR'06) (Santa Barbara, CA).

Knees, P., Pohle, T., Schedl, M., and Widmer, G. 2007. A music search engine built upon audio-based and web-based similarity measures. In Proceedings of the 30th Annual International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR'07).

Knees, P., Schedl, M., and Pohle, T. 2008. A deeper look into web-based classification of music artists. In Proceedings of 2nd Workshop on Learning the Semantics of Audio Signals (LSAS'08).

Knees, P., Schedl, M., Pohle, T., and Widmer, G. 2006b. An innovative three-dimensional user interface for exploring music collections enriched with meta-information from the web. In Proceedings of the 14th ACM International Conference on Multimedia (MM'06).

Laurier, C., Grivolla, J., and Herrera, P. 2008. Multimodal music mood classification using audio and lyrics. In Proceedings of the International Conference on Machine Learning and Applications.

Law, E., von Ahn, L., Dannenberg, R., and Crawford, M. 2007. Tagatune: A game for music and sound annotation. In Proceedings of the 8th International Conference on Music Information Retrieval (ISMIR'07).

Levy, M. and Sandler, M. 2007. A semantic space for music derived from social tags. In Proceedings of the 8th International Conference on Music Information Retrieval (ISMIR'07).

L&iamcr;beks, J. and Turnbull, D. 2011. You can judge an artist by an album cover: Using images for music annotation. IEEE MultiMedia, 18, 4, 30--37.

Linden, G., Smith, B., and York, J. 2003. Amazon.com recommendations: Item-to-item collaborative filtering. IEEE Internet Comput. 4, 1.

Logan, B., Ellis, D. P., and Berenzweig, A. 2003. Toward evaluation techniques for music similarity. In Proceedings of the 26th Annual International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR'03): Workshop on the Evaluation of Music Information Retrieval Systems.

Logan, B., Kositsky, A., and Moreno, P. 2004. Semantic analysis of song lyrics. In Proceedings of the IEEE International Conference on Multimedia and Expo (ICME'04).

MacQueen, J. 1967. Some methods for classification and analysis of multivariate observations. In Proceedings of the 5th Berkeley Symposium on Mathematical Statistics and Probability, L. M. L. Cam and J. Neyman, Eds., Statistics Series, vol. I, University of California Press, Berkeley and Los Angeles, CA, 281--297.

Mahedero, J. P. G., Martínez, A., Cano, P., Koppenberger, M., and Gouyon, F. 2005. Natural language processing of lyrics. In Proceedings of the 13th ACM International Conference on Multimedia (MM'05). 475--478.

Mandel, M. I. and Ellis, D. P. 2007. A web-based game for collecting music metadata. In Proceedings of the 8th International Conference on Music Information Retrieval (ISMIR'07).

Mayer, R., Neumayer, R., and Rauber, A. 2008. Rhyme and style features for musical genre classification by song lyrics. In Proceedings of the 9th International Conference on Music Information Retrieval (ISMIR'08).

McFee, B. and Lanckriet, G. 2009. Heterogeneous embedding for subjective artist similarity. In Proceedings of the 10th International Society for Music Information Retrieval Conference (ISMIR'09).

Mesnage, C. S., Rafiq, A., Dixon, S., and Brixtel, R. P. 2011. Music discovery with social networks. In Proceedings of the 2nd Workshop on Music Recommendation and Discovery (WOMRAD'11). 7--12.

Nanopoulos, A., Rafailidis, D., Symeonidis, P., and Manolopoulos, Y. 2010. Musicbox: Personalized music recommendation based on cubic analysis of social tags. IEEE Trans. Audio, Speech, Lang. Process. 18, 2, 407--412.

Pachet, F., Westerman, G., and Laigre, D. 2001. Musical data mining for electronic music distribution. In Proceedings of the 1st International Conference on Web Delivering of Music (WEDELMUSIC'01).

Pampalk, E., Flexer, A., and Widmer, G. 2005. Hierarchical organization and description of music collections at the artist level. In Proceedings of the 9th European Conference on Research and Advanced Technology for Digital Libraries (ECDL'05).

Pampalk, E. and Goto, M. 2007. MusicSun: A new approach to artist recommendation. In Proceedings of the 8th International Conference on Music Information Retrieval (ISMIR'07).

Pohle, T., Knees, P., Schedl, M., Pampalk, E., and Widmer, G. 2007. “Reinventing the Wheel”: A novel approach to music player interfaces. IEEE Trans. Multimed. 9, 567--575.

Pohle, T., Knees, P., Schedl, M., and Widmer, G. 2007. Building an interactive next-generation artist recommender based on automatically derived high-level concepts. In Proceedings of the 5th International Workshop on Content-Based Multimedia Indexing (CBMI'07).

Sarwar, B., Karypis, G., Konstan, J., and Reidl, J. 2001. Item-based collaborative filtering recommendation algorithms. In Proceedings of 10th International Conference on World Wide Web (WWW'01). 285--295.

Schedl, M. 2008. Automatically extracting, analyzing, and visualizing information on music artists from the world wide web. Ph.D. thesis, Johannes Kepler University, Linz, Austria.

Schedl, M. and Hauger, D. 2012. Mining microblogs to infer music artist similarity and cultural listening patterns. In Proceedings of the 21st International World Wide Web Conference (WWW'12): 4th International Workshop on Advances in Music Information Research (AdMIRe'12).

Schedl, M., Hauger, D., and Schnitzer, D. 2012. A model for serendipitous music retrieval. In Proceedings of the 16th International Conference on Intelligent User Interfaces (IUI'12): 2nd International Workshop on Context-awareness in Retrieval and Recommendation (CaRR'12).

Schedl, M. and Knees, P. 2009. Context-based music similarity estimation. In Proceedings of the 3rd International Workshop on Learning the Semantics of Audio Signals (LSAS'09).

Schedl, M. and Knees, P. 2011. Personalization in multimodal music retrieval. In Proceedings of the 9th Workshop on Adaptive Multimedia Retrieval (AMR'11).

Schedl, M., Knees, P., Pohle, T., and Widmer, G. 2006. Towards automatic retrieval of album covers. In Proceedings of the 28th European Conference on Information Retrieval (ECIR'06).

Schedl, M., Knees, P., and Widmer, G. 2005. A web-based approach to assessing artist similarity using co-occurrences. In Proceedings of the 4th International Workshop on Content-Based Multimedia Indexing (CBMI'05).

Schedl, M., Pohle, T., Knees, P., and Widmer, G. 2011. Exploring the music similarity space on the web. ACM Trans. Info. Syst. 29, 3.

Serra, X. 2012. Data gathering for a culture specific approach in MIR. In Proceedings of the 21st International World Wide Web Conference (WWW'12): 4th International Workshop on Advances in Music Information Research (AdMIRe'12).

Shavitt, Y. and Weinsberg, U. 2009. Songs clustering using peer-to-peer co-occurrences. In Proceedings of the IEEE International Symposium on Multimedia (ISM'09): International Workshop on Advances in Music Information Research (AdMIRe'09).

Shen, J., Meng, W., Yan, S., Pang, H., and Hua, X. 2010. Effective music tagging through advanced statistical modeling. In Proceedings of the 33rd International ACM SIGIR Conference on Research and Development in Information Retrieval. 635--642.

Slaney, M. 2011. Web-scale multimedia analysis: Does content matter? IEEE MultiMedia 18, 2, 12--15.

Slaney, M. and White, W. 2007. Similarity based on rating data. In Proceedings of the 8th International Conference on Music Information Retrieval (ISMIR'07).

Sordo, M., Laurier, C., and Celma, O. 2007. Annotating music collections: How content-based similarity helps to propagate labels. In Proceedings of the 8th International Conference on Music Information Retrieval (ISMIR'07). 531--534.

Turnbull, D., Barrington, L., and Lanckriet, G. 2008. Five approaches to collecting tags for music. In Proceedings of the 9th International Conference on Music Information Retrieval (ISMIR'08).

Turnbull, D., Liu, R., Barrington, L., and Lanckriet, G. 2007. A game-based approach for collecting semantic annotations of music. In Proceedings of the 8th International Conference on Music Information Retrieval (ISMIR'07).

Wang, X., Rosenblum, D., and Wang, Y. 2012. Context-aware mobile music recommendation for daily activities. In Proceedings of the 20th ACM International Conference on Multimedia.

Whitman, B. and Lawrence, S. 2002. Inferring descriptions and similarity for music from community metadata. In Proceedings of the 2002 International Computer Music Conference (ICMC'02). 591--598.

Yang, D., Chen, T., Zhang, W., Lu, Q., and Yu, Y. 2012. Local implicit feedback mining for music recommendation. In Proceedings of the 6th ACM Conference on Recommender Systems. 91--98.

Zadel, M. and Fujinaga, I. 2004. Web services for music information retrieval. In Proceedings of the 5th International Symposium on Music Information Retrieval (ISMIR'04).

Zangerle, E., Gassler, W., and Specht, G. 2012. Exploiting Twitter's collective knowledge for music recommendations. In Proceedings of the 21st International World Wide Web Conference (WWW'12): Making Sense of Microposts (#MSM'12). 14--17.

Zhang, B., Xiang, Q., Lu, H., Shen, J., and Wang, Y. 2009. Comprehensive query-dependent fusion using regression-on-folksonomies: A case study of multimodal music search. In Proceedings of the 17th ACM International Conference on Multimedia. 213--222.

Zhao, Z., Wang, X., Xiang, Q., Sarroff, A., Li, Z., and Wang, Y. 2010. Large-scale music tag recommendation with explicit multiple attributes. In Proceedings of the 18th ACM International Conference on Multimedia. 401--410.