BRAIN-COMPUTER MUSIC INTERFACING – CURRENT APPROACHES AND PROSPECTS
Keywords:
brain-computer music interfacing (BCMI), electroencephalogram (EEG), music performance, musical interactionAbstract
Brain-computer music interfacing (BCMI) is a field of research addressing the idea that electrical oscillations within the brain can be used to generate or manipulate music, or support a musical activity. This is achieved by transmitting brainwave activity expressed as electrical frequencies using electroencephalogram (EEG) electrodes placed upon the scalp to a computer which maps or translates this input to audible output with musical structures or rules. The concept of using this rhythm rich EEG signal for musical applications has led to the emergence of new types of musical instruments, interactions, performances and experiences which have captured the imaginations of many artists and technologists.
Downloads
References
Adrian, Edgar, and Bryan Matthews (1934). The Berger rhythm: Potential changes from the occipital lobes in man. Brain 57 (1), pp. 355–385
Berger, Hans (1929). Über das Elektrenkephalogramm des Menschen. Archiv für Psychiatrie und Nervenkrankheiten 87 (1), S. 527–570
Buzsáki, György (2006). Rhythms of the Brain. New York: Oxford University Press
Cherninskyi, Andrii (2015). The samples of main types of artifacts in human EEG. Wikimedia European Science Photo Competition 2015. https://commons.wikimedia.org/wiki/File:Human_EEG_artefacts.png (accessed November 9, 2018)
Christopher, Kameron R., Ajay Kapur, Dale A. Carnegie, and Gina M. Grimshaw (2014). A history of emerging paradigms in EEG for music. Music Technology Meets Philosophy: From Digital Echos to Virtual Ethos. Proceedings of the 40th International Computer Music Conference joint with the 11th Sound and Music Computing Conference, 2. Edited by Anastasia Georgaki and Georgios Kouroupetroglou. Athens: National and Kapodistrian University of Athens, pp. 1142–1148
Clayton, Martin, Rebecca Sager, and Udo Will (2005). In time with the music: The concept of entrainment and its significance for ethnomusicology. European Meetings in Ethnomusicology 11 (Esem CounterPoint 1), pp. 1–82
Cross, Ian (2005). Music and meaning, ambiguity and evolution. Musical Communication. Edited by Dorothy Miell, Raymond MacDonald, and David J. Hargreaves. Oxford: Oxford University Press, pp. 27–44
Eaton, Joel, and Eduardo Reck Miranda (2016). The hybrid BCMI – integrating brainwave detection methods for extended control in musical performance systems. Music, Mind, and Embodiment. Proceedings of 11th International Symposium on Computer Music Multidisciplinary Research (CMMR 2015). Edited by Richard Kronland-Martinet, Mitsuko Aramaki, and Sølvi Ystad. Part of the Lecture Notes in Computer Science. Volume 9617. Switzerland: Springer, pp. 132–145
Emotiv. Webpage. www.emotiv.com (accessed November 10, 2017)
Giacomo, Novembre, and Peter E. Keller (2014). A conceptual review on action-perception coupling in the musicians’ brain: What is it good for? Frontiers in Human Neuroscience 8, pp. 1–11
Gill, Satinder P. (2012). Rhythmic synchrony and mediated interaction: Towards a framework of rhythm in embodied interaction. AI&Society 27 (1), pp. 111–128
Gruzelier, John (2011). Enhancing imaginative expression in the performing arts with EEG-neurofeedback. Musical Imaginations: Multidisciplinary Perspectives on Creativity, Performance and Perception. Edited by David Hargreaves, Dorothy Miell, and Raymond MacDonald. 1st edition. Oxford: Oxford University Press, pp. 313–331
Gürkök, Hayrettin, and Antinus Nijholt (2013). Affective brain-computer interfaces for arts. Affective Computing and Intelligent Interaction (ACII 2013). Edited by Antinus Nijholt, Sidney K. D’Mello, and Maja Pantic. Switzerland: IEEE, pp. 827–831
Hargreaves, David, Dorothy Miell, and Raymond MacDonald (eds, 2011). Musical Imaginations: Multidisciplinary Perspectives on Creativity, Performance and Perception. 1st edition. Oxford: Oxford University Press
Haselsteiner, Ernst, and Gert Pfurtscheller (2000). Using time-dependent neural networks for EEG classification. IEEE Transactions on Rehabilitation Engineering 8 (4), pp. 457–463
Haumann, Niels T. (2015). An introduction to cognitive musicology: Historical-scientific presuppositions in the psychology of music. Danish Musicology Online, Special Edition, pp. 11–45. http://www.danishmusicologyonline.dk/arkiv/arkiv_dmo/dmo_saernummer_2015/dmo_saernummer_2015_musik_hjerneforskning_01.pdf (accessed November 10, 2017)
Hermann, Thomas (2008). Taxonomy and definitions for sonification and auditory display. Proceedings of the 14th International Conference on Auditory Display (ICAD 2008). Edited by Olivier Houix. Paris: IRCAM, pp. 1–8
Hinterberger, Thilo (2011). The sensorium: A multimodal neurofeedback environment. Advances in Human-Computer Interaction. Volume 2011, pp. 1–10
Hjorth, Bo (1970). EEG analysis based on time series properties. Electroencephalography and Clinical Neurophysiology 29, pp. 306–310
Hondrou, Charline, and George Caridakis (2012). Affective, natural interaction using EEG: Sensors, application and future directions. Artificial Intelligence: Theories and Applications. 7th Hellenic Conference on Artificial Intelligence: Proceedings (SETN 2012). Edited by Ilias Maglogiannis, Vassilis Plagianakos, and Ioannis Vlahavas. Part of the Lecture Notes in Computer Science. Volume 7297. Lamia: Springer, pp. 331–338
Hunt, Andy, Ross Kirk, and Marcelo M. Wanderley (2000). Towards a model for instrumental mapping in expert musical interaction. Proceedings of the 2000 International Computer Music Conference (ICMC 2000). Edited by Ioannis Zannos. Berlin: International Computer Music Association, pp. 209–212
iMotions. Webpage. www.imotions.com (accessed November 10, 2017)
Jasper, Heinrich (1958). The ten twenty electrode system of the international federation. Electroencephalography and Clinical Neurophysiology 10, pp. 371–375
Juslin, Patrik N. (2005). From mimesis to catharsis: Expression, perception, and induction of emotion in music. Musical Communication. Edited by Dorothy Miell, Raymond MacDonald, and David J. Hargreaves. Oxford: Oxford University Press, pp. 85–117
Keller, Peter E., Giacomo Novembre, and Michael J. Hove (2014). Rhythm in joint action: Psychological and neurophysiological mechanisms for real-time interpersonal coordination. Philosophical Transactions of the Royal Society B: Biological Sciences 369 (1658), pp. 1–12
Large, Edward W. (2008). Resonating to musical rhythm: Theory and experiment. Psychology of Time. Edited by Simon Grondin. United Kingdom et al.: Emerald Group Publishing Limited, pp. 189–231
Laroche, Julien, and Ilan Kaddouch (2015). Spontaneous preferences and core tastes: Embodied musical personality and dynamics of interactions in a pedagogical method of improvisation. Frontiers in Psychology 6, 522. https://www.ncbi.nlm.nih.gov/pmc/articles/PMC4439548/ (accessed November 10, 2017)
Lavy, Matthew M. (2001). Emotion and the Experience of Listening to Music: A Framework for Empirical Research. Dissertation. Cambridge: University of Cambridge. http://www.hugoribeiro.com.br/biblioteca-digital/Lavy-Emotion_Experience_of_Listening_Music.pdf (accessed November 10, 2017)
Lee, Hyekyung, and Seungjin Choi (2003). PCA+HMM+SVM for EEG pattern classification. Proceedings of the Seventh International Symposium on Signal Processing and Its Applications (ISSPA 2003). Volume 1. Edited by Suvisoft Oy Ltd. Paris: IEEE, pp. 541–544
Leman, Marc (2008). Systematic musicology at the crossroads of modern music research. Systematic and Comparative Musicology: Concepts, Methods, Findings. Hamburger Jahrbuch für Musikwissenschaft, 24. Edited by Albrecht Schneider. Frankfurt am Main: Peter Lang, pp. 89–115
Leman, Marc (2010). Some reflections on systematic musicology as proactive science. Concepts, Experiments, and Fieldwork: Studies in Systematic Musicology and Ethnomusicology. Edited by Rolf Bader, Christiane Neuhaus, and Ulrich Morgenstern. Frankfurt am Main, New York: Peter Lang, pp. 21–34
Leslie, Grace, and Tim Mullen (2011). MoodMixer: EEG-based collaborative sonification. Proceedings of the International Conference on New Interfaces for Musical Expression (NIME 2011). Edited by Alexander Jensenius Refsum, Anders Tveit, Inge Rolf Godøy, and Daniel Overholt. Oslo: University of Oslo, pp. 296–299
Levicán, Constanza, Andrés Aparicio, Vernon Belaunde, and Rodrigo F. Cádiz (2017). Insight2OSC: Using the brain and the body as a musical instrument with the emotiv insight. New Interface for Musical Expression. Proceedings (NIME 2017). Edited by Cumhur Erkut. Copenhagen: Aalborg University Copenhagen, pp. 287–290
Lopata, Joel A. (2014). Creativity as a Mental State: An EEG Study of Musical Improvisation. A thesis submitted in partial fulfillment of the requirements for the degree in Doctor of Philosophy. London, Ontario, Canada: The School of Graduate and Postdoctoral Studies, The University of Western Ontario. http://citeseerx.ist.psu.edu/viewdoc/download;jsessionid=134A904AF2D86B3C2C2F09B8D97E143B?doi=10.1.1.670.9838&rep=rep1&type=pdf (accessed November 10, 2017)
Lotte, Fabien, Marco Congedo, Anatole Lécuyer, Fabrice Lamarche, and Bruno Arnaldi (2007). A review of classification algorithms for EEG-based brain-computer interfaces. Journal of Neural Engineering 4 (2), pp. 1–24
Lotte, Fabien (2014). A tutorial on EEG signal processing techniques for mental state recognition in brain-computer interfaces. Guide to Brain-Computer Music Interfacing. Edited by Eduardo Reck Miranda and Julien Castet. London: Springer, pp. 133–161
Malmivuo, Plonsey, Jaakko Malmivuo, and Robert Plonsey (1995). Bioelectromagnetism: Principles and Applications of Bioelectric and Biomagnetic Fields. New York: Oxford University Press
Maskeliunas, Rytis, Robertas Damasevicius, Ignas Martisius, and Mindaugas Vasiljevas (2016). Consumer grade EEG devices: Are they usable for control tasks? PeerJ 4:e1746, pp. 1–27. https://doi.org/10.7717/peerj.1746 (accessed November 10, 2017)
McCreadie, Karl A., Damien H. Coyle, and Girijesh Prasad (2013). Sensorimotor learning with stereo auditory feedback for a brain- computer interface. Medical, Biological Engineering, Computing 51 (3), pp. 285–293
McGuiness, Andy, and Katie Overy (2011). Music, consciousness, and the brain: Music as shared experience of an embodied present. Music and Consciousness. Philosophical, Psychological, and Cultural Perspectives. Edited by David Clarke and Eric Clarke. Oxford: Oxford University Press, pp. 245–262
MindMedia. Webpage. www.mindmedia.com (accessed November 10, 2017)
Miranda, Eduardo Reck (2006a). Brain-computer interface for composition and performance. International Journal on Disability and Human Development 5 (2), pp. 61–67
Miranda, Eduardo Reck (2006b). Brain-computer music interface for generative music. Proceedings of the 6th International Conference on Disability, Virtual Reality and Associated Technologies (ICDVRAT 2006). Edited by Paul Sharkey, Tony Brooks, and Sue Cobb. United Kingdom: University of Reading, pp. 295–302
Miranda, Eduardo Reck, and Joel Eaton (2014). On mapping EEG information into music. Guide to Brain-Computer Music Interfacing. Edited by Eduardo Reck Miranda and Julien Castet. London: Springer, pp. 221–254
Miranda, Eduardo Reck, and Julien Castet (eds, 2014). Guide to Brain-Computer Music Interfacing. London: Springer
Molnar-Szakacs, Istvan, Vanya G. Assuied, and Katie Overy (2011). Shared affective motion experience (SAME) and creative, interactive music therapy. Musical Imaginations: Multidisciplinary Perspectives on Creativity, Performance and Perception. Edited by David Hargreaves, Dorothy Miell, and Raymond MacDonald. 1st edition. Oxford: Oxford University Press, pp. 313–331
Muse. Webpage. www.choosemuse.com (accessed November 10, 2017)
Neuroelectrics [Products / Enobio]. Webpage. https://www.neuroelectrics.com/products/enobio/ (accessed November 10, 2017)
Neurosky. Webpage. www.neurosky.com (accessed November 10, 2017)
Niedermeyer, Ernst, and Fernando Lopes da Silva (2005). Electroencephalography: Basic Principles, Clinical Applications, and Related Fields. Philadelphia et al.: Lippincott Williams, Wilkins
Nunez, Paul L., and Ramesh Srinivasan (2006). Electric Fields of the Brain: The Neurophysics of EEG. Oxford: Oxford University Press
Overy, Katie, and Istvan Molnar-Szakacs (2009). Being together in time: Musical experience and the mirror neuron system. Music Perception: An Interdisciplinary Journal 26 (5), pp. 489–504
Palaniappan, Ramaswamy (2014). Electroencephalogram-based brain-computer interface: An introduction. Guide to Brain-Computer Music Interfacing. Edited by Eduardo Reck Miranda and Julien Castet. London: Springer, pp. 29–41
Rakotomamonjy, Alain, Vincent Guigue, G. Mallet, and Vania Alvarado (2005). Ensemble of SVMs for improving brain computer interface p300 speller performances. Artificial Neural Networks: Biological Inspirations (ICANN 2005). Proceedings. Edited by Wlodzislaw Duch, Erkki Oja, and Slawomir Zadrozny. Part 1. Part of the Lecture Notes in Computer Science. Volume 3696. Berlin, Heidelberg: Springer, pp. 45–50
Ramirez, Rafael, and Andris Zacharias Vamvakousis (2012). Detecting emotion from EEG signals using the emotive epoc device. International Conference on Brain Informatics (BI 2012). Proceedings. Edited by Fabio Massimo Zanzotto, Shusaky Tsumoto, Niels Taatgen, and Yiyu Yao. Part of the Lecture Notes in Computer Science. Volume 7670. Macau: Springer, pp. 175–184
Reidsma, Dennis, Mustafa Radha, and Anton Nijholt (2014). Mediated interactions and musical expression – a survey. Digital Da Vinci. Edited by Newtoon Lee. New York: Springer, pp. 79–98
Rocchesso, Davide (2011). Explorations in Sonic Interaction Design. Berlin: Logos Verlag
Rosenboom, David (1999). Extended musical interface with the human nervous system: Assessment and prospectus. Leonardo 32 (4), p. 257
Rosenboom, David (2014). Active imaginative listening – a neuromusical critique. Frontiers in Neuroscience 8: 251. https://www.frontiersin.org/articles/10.3389/fnins.2014.00251/full (accessed November 10, 2017)
Sanei, Saeid, and Jonathon A. Chambers (2007). EEG Signal Processing. Hoboken, NJ: John Wiley, Sons
Tan, Desney S., and Anton Nijholt (eds, 2010). Brain-Computer Interfaces: Applying Our Minds to Human-Computer Interaction. London: Springer
Tatum, William O., Aatif M. Husain, Selim R. Benbadis, and Peter W. Kaplan (2007). Handbook of EEG Interpretation. New York: Demos Medical Publishing
Teplan, Michal (2002). Fundamentals of EEG measurement. Measurement Science Review 2 (2), pp. 1–11
Väljamäe, Aleksander, Sebastián Mealla, Mathieu Bosi, and Sergi Jordà (2011). Listening to your brain: Implicit interaction in collaborative music performances. Proceedings of the International Conference on New Interfaces for Musical Expression (NIME 2011). Edited by Alexander Jensenius Refsum, Anders Tveit, Inge Rolf Godøy, and Daniel Overholt. Oslo: University of Oslo, pp. 1–6
Vidal, Jacques J. (1973). Toward direct brain-computer communication. Annual Review of Biophysics and Bioengineering 2, pp. 157–180
Volpe, Gualtiero, Alessandro D’Ausilio, Leonardo Badino, Antonio Camurri, and Luciano Fadiga (2016). Measuring social interaction in music ensembles. Philosophical Transactions of the Royal Society B: Biological Sciences 371, pp. 1–8
Wadeson, Amy, Antinus Nijholt, and Chang S. Nam (2015). Artistic brain-computer interfaces: State-of-the-art of control mechanisms. Brain-Computer Interfaces 2 (2–3), pp. 70–75
Wu, Dan, Chaoyi Li, Yu Yin, Changzheng Zhou, and Dezhong Yao (2010). Music composition from the brain signal: Representing the mental state by music. Computational Intelligence and Neuroscience. Volume 2010, pp. 1–6
Zhuang, Tianbao, Hong Zhao, and Zheng Tang (2009). A study of brainwave entrainment based on EEG brain dynamics. Computer and Information Science 2 (2), pp. 80–86
