Vitória - ES - Brazil

Transcrição

1 Anais do V Workshop de Música Ubíqua Proceedings of V Workshop on Ubiquitous Music Vitória - ES - Brazil Das Artes Digitais à Música Ubíqua From Digital Arts to Ubiquitous Music 29 de Outubro a 1 de Novembro October 29th - November 1st

2 Anais do 5 o Workshop de Música Ubíqua Proceedings of 5 th Workshop on Ubiquitous Music 29 de Outubro a 1 de Novembro / 2014 October 29th - November 1st / 2014 Vitória - ES - Brazil Os artigos publicados nestes Anais foram editorados a partir dos originais finais entregues pelos autores, sem edições ou correções feitas pelo comitê técnico. Editoração e arte / Publishing and art Flávio Luiz Schiavoni

3 Organização Apoio

4 Organização / Organization Coordenadores / Organizing Committee Geral / General Chair Leandro L. Costalonga, Universidade Federal do Espírito Santo (Nescom) Artigos / Papers Chair Maria Helena de Lima, Universidade Federal do Rio Grande do Sul (CAp) Damián Keller, Universidade Federal do Acre (NAP) Atividades artísticas / Artistic Session Chair Marcus Vinícius Marvila das Neves, Universidade Federal do Espírito Santo (Nescom) Divulgação / Public Liaison Chair Flávio Schiavoni, Universidade Federal de São João Del Rei (UFSJ) Comitê de Programa / Program Committee Almerinda da Silva Lopes, UFES, Artes visuai Andrew R. Brown, Griffith University Damián Keller, Universidade Federal do Acre (NAP) Daniel Spikol, Malmö University Evandro M. Miletto, IFRS, Porto Alegre Flavio Schiavoni, IME USP Georg Essl, University of Michigan Ian Oakley, UNIST, South Korea José (Tuti) Fornari, UNICAMP Joseph Timoney, NUIM Juan P. Bello, New York University Leandro Costalonga, UFES Liliana Mónica Vermes, UFES Luciano Vargas Flores, UFRGS (LCM, ENDEEPER) Marcelo Johann, UFRGS Marcelo Milrad, Linnaeus University Marcelo Queiroz, IME - USP Marcelo Soares Pimenta, UFRGS (LCM) Maria Helena de Lima, UFRGS (CAp) MarthaPaz, UFRGS Mônica Estrázulas, UFRGS Nuno Otero, Linnaeus University Patrick McGlynn, NUIM - National University of Ireland, Maynooth Reginaldo Braga, UFRGS Rodolfo Coelho de Souza, USP-RP Rodrigo Cicchelli Velloso, UFRJ Rogério Costa, USP (ECA) Silvio Ferraz, USP (ECA) Victor Lazzarini, NUIM - National University of Ireland, Maynooth

5 Prologue to the Proceedings of the V Workshop on Ubiquitous Music (V UbiMus) Damián Keller, Maria Helena de Lima, Flávio Schiavoni (Editors) Ubiquitous Music Group (Grupo de Música Ubíqua] October 2014 Over the last five years, the Ubiquitous Music Workshop has grown from an informal gathering on ongoing projects and ideas, to a full blown event yielding key references for the area. The initial reports on programming platforms for mobile environments (Lazzarini et al. 2013], on educational initiatives based on ubimus research [Lima et al. 2012], on design patterns that shaped recent developments in musical software classifications [Flores et al. 2014), and on everyday creativity research in music making [Pinheiro da Silva et al. 2014], they were all initially discussed in ubimus workshops. The Fifth Workshop on Ubiquitous Music took place at the Federal University of Espírito Santo (UFES), from October 31 to November 3. Researchers dealing with digital arts and ubiquitous music shared proposals, initial results and complete research projects. The V UbiMus featured contributions from Australia, Ireland, Sweden and Italy. These proceedings feature reports in Portuguese and English, encompassing full papers, posters and summaries of artistic works. There were six full paper proposals accepted. Andrew Brown presented results from his group's ongoing research on meaningful engagement. This work has close ties to the dialogical approach to education that has been championed by Lima et al. (2014). Keller's (2014) paper identifies three methodological approaches to creativity centered design: the computational approach, the dialogical perspective and the ecologically grounded framework. The text analyzes how these three methods relate to a current definition of the ubiquitous music field and proposes two new theoretical tools to study design qualities from a creativity centered perspective: volatility and rivalry. Timoney et al. (2014) describe the EU Beathealth project as an initiative to create an intelligent technical architecture capable of delivering embodied, flexible, and efficient rhythmical stimulation adapted to the individuals motor performance and skills for the purpose of enhancing or recovering movement activity. Lazzarini et al. (2014) focus on the prototyping stage of the design cycle of ubiquitous music ecosystems. The paper presents three case studies of prototype deployments for creative musical activities. Farias's et al. (2014) report on their findings regarding the use of the time tagging metaphor for musical creativity endeavors. The authors developed a new ubiquitous music prototype and carried out experimental work to investigate the relationships between the technological support strategies and their creative yield, involving assessments of creative products produced with Audacity, a mixdroid first generation prototype and a mixdroid second generation prototype. Villena (2014) aims to observe the potential contributions to musical composition of two research areas: ubiquitous music and soundscape studies. The paper draws on Brazilian composers works, establishing links between the traditional soundscape composition approach and the latest advances in experimental art. The remaining presentations consisted of artistic works and posters. Schiavoni s and Costalonga s (2014) text serves as a proposal for computer science research to delve into ubimus issues, stimulating discussions at the borderland between ubiquitous computing and music practice. On a complementary vein, Santos et al. (2014) describe the use of ubiquitous computing to promote learning activities in the context of an elementary school music class. Delgado et al. (2014) present a first prototype supporting collaborative musical activities using location aware mobile technology based on Near Field Communication (NFC) and Multi Agent Systems (MAS). Gobira et al. (2014) present the development of an experimental interactive installation that uses a 3D motion sensor to

6 capture body movement of a person balancing on a slackline tape. D Amato s (2014) Progressive Disclosure is a short piece situated in an imaginary landscape where an unknown machine is progressively disclosed to reveal its inner functions. And McGlynn s (2014) DroneUnknown aims to explore the ambiguous interaction space of self modifying instruments. The performer navigates through the material, discovering it while it is shared with the audience. Full paper reports Brown et al. (2014) report on their experiences using ubiquitous computing devices to introduce music based creative activities into an Australian school. The use of music applications on mobile computers (ipads) made the proposed activities accessible to students with a limited range of prior musical background. The activities were designed to be meaningful and contribute toward personal resilience in the students. Brown and coauthors describe the approach to meeting the objectives of the study and discuss their results. The paper includes an overview of the ongoing project on music education including its aims, objectives and utilisation of mobile technologies and software with generative and networkable capabilities. Two theoretical frameworks inform the research design; the meaningful engagement matrix and personal resilience. These frameworks inform the activity planning. The report focuses on the activities undertaken and shares results from questionnaires, interviews, musical outcomes and observation. Farias's et al. (2014) paper focuses on mixing as the object of study of creativity centered interaction design. The authors applied the time tagging metaphor to develop a new ubiquitous music prototype and carried out experimental work to investigate the relationships between the technological support strategies and their creative yield. A musician produced 30 sound mixes using different tools and similar sound resources in the same location. From that output, three creative products each of approximately 3 minutes were chosen. In the first creative session the sound editor Audacity was used. The second session was done with the ubiquitous music system mixdroid 1.0 or first generation (1G). The third session involved the use of a new prototype mixdroid 2.0 or second generation (2G). The time invested on each mix was: 97 minutes with Audacity; 6:30 minutes using mixdroid 1G; and 3:30 minutes using mixdroid 2G. 24 subjects evaluated the three products through the Creative Product Profile (CrePP NAP) protocol. Results indicated very similar profiles for the mixdroid 1G and 2G products. On a scale of 2 to +2, differences weren't larger than 17 cents. Scores for the descriptors 'relaxing' and 'pleasant' were 0.96 and 1.42 points higher for the Audacity made product, but variations among scores were also high. Originality and expressiveness were slightly higher for Audacity 21 and 42 cents respectively. In contrast, the relevance factor of the mixdroid 2G product was 25 cents higher than the score given to the Audacity product. This study indicates that the application of the time tagging metaphor boosts the efficiency of the creative activity, but that boost does not extend to the creativity profile of the products. Timoney et al. (2014) describe the EU Beathealth project as an initiative to create an intelligent technical architecture capable of delivering embodied, flexible, and efficient rhythmical stimulation adapted to individuals motor performance and skills for the purpose of enhancing or recovering muscle movement. The text explains how the project embodies the principles of Ubiquitious Music and how it draws on many aspects of this field. The Beathealth collaborative research project is essentially about using the power of beats and rhythm in new technology applications to help us to achieve better health. For those with declining physical health such as Parkinson s, the project strives to create tools to assist patients in therapy and quicken their rate of improvement. The proposed technology uses data gleaned from regular, repeated bodily movement employing real

7 time sensors. This drives a synchronous adaptation with the music thereby reinforcing the rhythm of the activity at a neurological level. Among the expected results, the authors envisage to increase the harmony of body and mind activity, hopefully boosting the beneficial impacts of physical activity. Villena (2014) aims to observe the possible contributions to musical composition from two areas of research: ubiquitous music and the work done in soundscape composition. Although these two areas present important points in common, especially when considering their theoretical foundations, the author argues that the first is centered on computing, while the second deals with acoustic ecology. The conceptual boundaries of the two areas are established through a discussion of artists working in Brazil encompassing their methodologies, their conceptions and the mutual influence within the area of music composition. Keller's (2014) paper identifies three methodological approaches to creativity centered design: the computational approach, the dialogical perspective and the ecologically grounded framework. The author analyzes how these three methods relate to a current definition of the ubiquitous music field. Social interaction is one of the factors to be accounted for in ubimus experimental studies. Hence, he proposes the label social resources for the shared knowledge available within a community of practice. Five aspects of creativity centered design that have targeted social resources are identified. Material resources are factors to be considered for the design of ubimus ecosystems, so two new design qualities are proposed as variables for experimental studies: volatility and rivalry. This discussion is framed by a split between creative products and creative resources which points to three observables: material resources, material products and material by products, including creative waste. The discussion concludes with a summary of the main arguments of the paper, pointing to applications of these concepts in experimental design studies. Lazzarini et al. (2014) focus on the prototyping stage of the design cycle of ubiquitous music (ubimus) ecosystems. The paper presents three case studies of prototype deployments for creative musical activities. The first case exemplifies a ubimus system for synchronous musical interaction using a hybrid Java JavaScript development platform, mow3s ecolab. The second case makes use of the HTML5 Web Audio library to implement a loop based sequencer. The third prototype an HTML controlled sine wave oscillator provides an example of using the Chromium open source sand boxing technology Portable Native Client (PNaCl) platform for audio programming on the web. The Csound PNaCl environment provides programming tools for ubiquitous audio applications that go beyond the HTML5 Web Audio framework. This new approach demanded porting the Csound language and audio engine to the PNaCl web technology. The limitations and advantages of the three approaches proposed the hybrid Java JavaScript environment, the HTML5 audio library and the Csound PNaCl infrastructure are discussed in the context of rapid prototyping of ubimus ecosystems. Posters and artistic presentations Schiavoni and Costalonga (2014) state that ubimus concepts and motivations as defined by Keller et al. (2009) include merging sound sources and musical support technology with environmental resources. Previous research and efforts from the Ubiquitous Music Group included several discussion involving Collective Music Creation [Ferraz and Keller 2014], Interaction Aesthetics [Keller et al. 2014], Creativity centered Software Design [Lima et al. 2012], pointing to open issues in current musical practices [Keller et al. 2011] and relevant aspects of musical and extra musical dimensions. Beyond the musical and social discussion in ubimus, the authors suggest that computer scientists can also take part in this research field contributing to the growth of ubimus initiatives.

8 Thus, the text serves as a proposal for Computer Science research to delve into ubimus issues, stimulating discussions in the interdisciplinary borderland between ubiquitous computing and music. Delgado et al. (2014) developed a prototype based on Near Field Communication (NFC) and Multi Agent Systems (MAS). NFC is a wireless technology of short range communication that enable users to connect the physical world with the virtual [Want 2011]. The prototype uses a Multi Agent System (MAS) framework and is based on previous efforts on mobile collaboration [Gil et al. 2014]. This framework features the instantiation of a distributed system that allows mobile devices to perform individual tasks through its agents and also to perform common tasks such as communication among its agents/devices [Wooldridge 2002]. The authors present the first prototype of a framework that supports collaborative musical activities using location aware mobile technology. To explore the design potential, they propose a series of workshops with ubimus practitioners to elicit preferences and requirements. The proposal is part of a long term project exploring how mobile technologies can enable the emergence of ubiquitous musical activities. McGlynn (2014) presents an experimental performance tool with random characteristics. Previous computer music approaches have pointed out a number of problems that arise while designing 'intelligent' digital musical instruments: the performance tools modify their behaviour in response to user input patterns. This concept may appear to be intelligent and helpful but in practice such designs can hinder the formation of meaningful performer instrument relationships. Thus, Cook (2001) states that "Smart instruments are often not smart". DroneUnknown aims to explore this ambiguous interaction space by capitalizing on the unpredictability of self modifying instruments, rather than trying to restrain it. The program runs on the Oscar multi touch performance platform and draws random source material from a bank of samples at initialization time. The player must navigate through the material during the live performance and discover it while it is shared with the audience. This leads to a number of possible approaches, ranging from the tentative exploration of the program states to more aggressive journeys. A discussion of the gestural tools available for performance of ubiquitous music concludes with a live demonstration of DroneUnknown. D Amato s (2014) Progressive Disclosure is a short piece situated in an imaginary landscape where an unknown machine is progressively disclosed. Long slow sound objects and impulsive sounds are merged and overlapped to develop an imaginary panorama. Both synthesized and acoustically derived sounds are used. The piece proposes a reflection on how to approach object properties or qualities and their functions. Santos et al. (2014) describe the use of ubiquitous computing in an elementary school music class, to promote learning of rhythmic concepts. Starting from bodily contacts already known to the students, such as clapping, the classroom is presented as a collective musical tool. The school environment is turned into a sound lab where participants interact. This research proposes a exploratory method. Data collection is conducted through questionnaires, focus groups and direct observations. Gobira et al. (2014) present an experimental interactive installation that uses 3D motion sensors to capture body movement to create graphics and sounds. The proposal seeks to merge the fields of music, video art, technology, gaming and performance as an audiovisual product. The proposed installation suggests a connection between body and machine. Sounds, designed from various sources, are triggered by voluntary and involuntary processes while maintaining the balance on a slackline tape. The limb movements feet, hands, and head are translated to data for a generative audio source developed in Open Frameworks, Pure Data. The body movement controls the

9 variations in resonance frequency synthesizing five independent sound waves. Through the translation of body effort to audiovisual stimuli and the combination of postural control with external audiovisual stimuli, the authors expect to create an immersive notion of interface control. Summing up, as in previous workshops we see a healthy mix of technical, conceptual and applied proposals. This edition features a growing presence of artistic projects, providing a chance for a hands on experience of ubiquitous music in the making. Interestingly, two major approaches to creativity are represented [Keller 2013]: the algorithmic view and the ecocognitive creative practices. It is still a mystery whether these two approaches to musical creativity will furnish viable alternatives to the predominant acoustic instrumental paradigm. While gathering experience through exploratory and participatory methods, the Ubiquitous Music Group may provide a space for community endeavors that are currently lacking in the compartmentalized disciplinary venues. Echoing the sounds of what may lie ahead, we are tempted to say, join us, resistance is futile!. References Brown, A., Stewart, D., Hansen, A. & Stewart, A. (2014). Making meaningful musical experiences accessible using the ipad. In D. Keller, M. H. Lima & F. Schiavoni (ed.), Proceedings of the V Workshop on Ubiquitous Music (V UbiMus). Vitória, ES: Ubiquitous Music Group. Retrieved from D'Amato, A. (2014). Progressive Disclosure [Ubiquitous Music Artwork]. In D. Keller, M. H. Lima & F. Schiavoni (ed.), Proceedings of the V Workshop on Ubiquitous Music (V UbiMus). Vitória, ES: Ubiquitous Music Group. Retrieved from Farias, F. M., Keller, D., Pinheiro Da Silva, F., Pimenta, M. S., Lazzarini, V., Lima, M. H., Costalonga, L. & Johann, M. (2014). Suporte para a criatividade musical cotidiana: mixdroid segunda geração. In D. Keller, M. H. Lima & F. Schiavoni (ed.), Proceedings of the V Workshop on Ubiquitous Music (V UbiMus). Vitória, ES: Ubiquitous Music Group. Retrieved from Gil de la Iglesia D, et al. (Forthcoming) (2014). A Self Adaptive Multi Agent System Approach for Collaborative Mobile Learning. Under submission process in Transactions on Learning Technologies. Gobira, P., Prota, R. & Ítalo Travenzoli (2014). Balance: Um estudo sobre a tradução digital do corpo em equilíbrio [Ubiquitous Music Artwork]. In D. Keller, M. H. Lima & F. Schiavoni (ed.), Proceedings of the V Workshop on Ubiquitous Music (V UbiMus). Vitória, ES: Ubiquitous Music Group. Retrieved from Keller, D. (2013). A mão na massa da criatividade musical (prólogo) / La mano en la masa de la creatividad musical (prólogo) / Musical creativity (prologue). In D. Keller, D. Quaranta & R. Sigal (ed.), Sonic Ideas, Vol. Criatividade Musical / Creatividad Musical. México, DF: CMMAS. Keller, D. (2014). Characterizing Resources in Ubimus Research: Volatility and Rivalry. In D. Keller, M. H. Lima & F. Schiavoni (ed.), Proceedings of the V Workshop on Ubiquitous Music (V UbiMus). Vitória, ES: Ubiquitous Music Group. Retrieved from

10 Keller, D., Barros, A. E. B., Farias, F. M., Nascimento, R. V., Pimenta, M. S., Flores, L. V., Miletto, E. M., Radanovitsck, E. A. A., Serafini, R. O. & Barraza, J. F. (2009). Ubiquitous music: concept and background (Música ubíqua: conceito e motivação). In Proceedings of the National Association of Music Research and Post Graduation Congress ANPPOM (Anais do Congresso da Associação Nacional de Pesquisa e Pós Graduação em Música ANPPOM) (pp ). Goiânia, GO: ANPPOM. Keller, D., Flores, L. V., Pimenta, M. S., Capasso, A. & Tinajero, P. (2011). Convergent trends toward ubiquitous music. Journal of New Music Research 40 (3), (Doi: / ) Lima, M. H., Brandão, R., Keller, D., Pezzi, R., Pimenta, M., Lazzarini, V., Costalonga, L., Depaoli, F. & Kuhn, C. (2014). Música Ubíqua no Colégio de Aplicação da UFRGS e Centro de Tecnologia Acadêmica e Ciência Cidadã Jr: Transversalidades em pesquisa em ensino [Poster]. In D. Keller, M. H. Lima & F. Schiavoni (ed.), Proceedings of the V Workshop on Ubiquitous Music (V UbiMus). Vitória, ES: Ubiquitous Music Group. Lima, M. H., Keller, D., Pimenta, M. S., Lazzarini, V. & Miletto, E. M. (2012). Creativity centred design for ubiquitous musical activities: Two case studies. Journal of Music, Technology and Education 5 (2), (Doi: /jmte _1.) McGlynn, P. (2014). DroneUnknown: An experiment in embracing unpredictability in live electronic performance [Ubiquitous Music Artwork]. In D. Keller, M. H. Lima & F. Schiavoni (ed.), Proceedings of the V Workshop on Ubiquitous Music (V UbiMus). Vitória, ES: Ubiquitous Music Group. Retrieved from Real Delgado, Y., de la Iglesia, D. G. & Otero, N. (2014). Exploring the potential of mobile technology for creating music collaboratively [Poster]. In D. Keller, M. H. Lima & F. Schiavoni (ed.), Proceedings of the V Workshop on Ubiquitous Music (V UbiMus). Vitória, ES: Ubiquitous Music Group. Retrieved from Santos, T., Filippo, D. & Pimentel, M. (2014). Computação Ubíqua e a interação corporal na aprendizagem de execução rítmica [Poster]. In D. Keller, M. H. Lima & F. Schiavoni (ed.), Proceedings of the V Workshop on Ubiquitous Music (V UbiMus). Vitória, ES: Ubiquitous Music Group. Retrieved from Schiavoni, F. & Costalonga, L. (2014). Ubiquitous computing meets ubiquitous music [Poster]. In D. Keller, M. H. Lima & F. Schiavoni (ed.), Proceedings of the V Workshop on Ubiquitous Music (V UbiMus). Vitória, ES: Ubiquitous Music Group. Retrieved from Timoney, J., Lazzarini, V., Ward, T., Villing, R., Conway, E. & Czesak, D. (2014). The Beathealth project: Synchronizing movement and music. In D. Keller, M. H. Lima & F. Schiavoni (ed.), Proceedings of the V Workshop on Ubiquitous Music (V UbiMus). Vitória, ES: Ubiquitous Music Group. Retrieved from Villena, M. (2014). Música ubiqua e paisagens sonoras. Possíveis contribuições. In D. Keller, M. H. Lima & F. Schiavoni (ed.), Proceedings of the V Workshop on Ubiquitous Music (V UbiMus). Vitória, ES: Ubiquitous Music Group. Retrieved from

11 Want, R. (2011). Near Field Communication. IEEE Pervasive Computing. vol. 10, n. 3 pp. 4 7, July September. Wooldridge M. (2009). An Introduction to MultiAgent Systems (2nd ed.). John Wiley & Sons. ISBN 13:

12 Sumário Música ubiqua e paisagens sonoras. Possíveis contribuições Exploring the potential of mobile technology for creating music collaboratively. 15 DroneUnknown: An experiment in embracing unpredictability in live electronic performance Suporte para a Criatividade Musical Cotidiana: Mixdroid Segunda Geração Making meaningful musical experiences accessible using the ipad Música Ubíqua no Colégio de Aplicação da UFRGS e Centro de Tecnologia Acadêmica e Ciência Cidadã Jr: transversalidades em pesquisa em ensino The Beathealth Project: Synchronising Movement and Music Characterizing Resources in Ubimus Research: Volatility and Rivalry Prototyping of Ubiquitous Music Ecosystems Ubiquitous Computing meets Ubiquitous Music Progressive Disclosure Computação Ubíqua e a interação corporal na aprendizagem de execução rítmica 85 Balance: um estudo sobre a tradução digital do corpo em equilíbrio

13 From Digital Arts to Ubiquitous Music V Ubimus Música ubiqua e paisagens sonoras. Possíveis contribuições. Marcelo Ricardo Villena (UNILA/UFMG) Resumo O presente texto visa observar as possíveis contribuições, na composição musical, entre duas áreas de pesquisa: a música ubiqua e trabalhos realizados sob o conceito de paisagem sonora. Embora estas duas áreas apresentem importantes pontos em comum, sobretudo se considerarmos alguns dos fundamentos teóricos empregados, têm focos de pesquisa diferenciados, o primeiro centrado na computação, o segundo na ecologia acústica. O artigo aborda numa primeira parte a delimitação conceitual das duas áreas para posteriormente discutir escritos de autores que atuam no Brasil com o intuito de analisar suas metodologias, suas concepções e as possíveis contribuições mútuas no âmbito da composição. Palavras-chave: composição musical, computação pervasiva, ecologia acústica. Introdução A área da composição musical passa por um momento de redefinições. As metodologias de composição tradicional, ancoradas num processo criativo que dispensa a priori a colaboração entre compositor e intérprete, na fixação de símbolos num papel pautado ou no uso exclusivo do palco italiano 1 como ambiente performático, parecem ser diariamente contestadas. Observam-se atualmente propostas em que o compositor sai do conforto do seu escritório, vivencia com seus sentidos o entorno, testa metodologias de criação coletiva e emprega tecnologias que prescindem da escrita em pauta musical. Entre essas diversas formas de criação, questionadoras dos procedimentos da tradição, podemos mencionar a música ubíqua e a composição a partir do estudo de paisagens sonoras (soundscape). Estas dois conceitos englobam interesses que vão além da área composição musical. As paisagens sonoras são objeto de estudo, por exemplo, da antropologia, sociologia, urbanismo, ecologia, biologia, engenharia, acústica, artes visuais, educação, entre outras. De fato, o conceito, criado por Murray Schafer em fins dos anos 60, foi concebido como um campo de pesquisa multidisciplinar de uma área maior denominada ecologia acústica e que tinha por objetivo, em definitiva, zelar pela qualidade dos ecossistemas no aspecto sonoro. A intenção de Schafer era criar equipes integradas por profissionais de diversas disciplinas que contribuíssem com seu conhecimento ao desenvolvimento de pesquisas de caráter coletivo. 1 Palco italiano é aquele em que os intérpretes se situam à frente do público, como numa sala de cinema. 1 Vitória - ES - Brazil

14 V Ubimus From Digital Arts to Ubiquitous Music A música ubíqua, por outro lado, é um conceito derivado da computação ubíqua, isto é, da área da computação que trata de disposítivos móveis, redes e ferramentas acessíveis em diferentes objetos e locais, como por exemplo um smartphone que acessa uma rede de internet num aeroporto. A música ubiqua estuda, principalmente, as possibilidades que estas ferramentas oferecem à prática musical em diferentes âmbitos de estudo: composição, educação, práticas interpretativas etc. Desta maneira, compreendemos que os conceitos abordam questões diferentes: ora com foco no estudo do meio ambiente, ora na tecnologia. Nosso interesse principal é ver quais as diferenças existentes entre as duas áreas, para finalmente compreender como suas pesquisas, que ocorrem de forma simultânea, podem se retroalimentar, sem cair na tentação de uma área querer se sobrepor à outra. Paisagem sonora Antes que nada, devemos esclarecer que o termo paisagem sonora, que pode ser definido como todas as sonoridades presentes no entorno acessíveis à percepção humana, talvez seja novo como conceito, mas é um fenômeno que sempre esteve em discussão. 2 Bernie Krause, por exemplo, argui (a partir de experiências de escuta de geofonia com grupos indígenas) que a música muito provavelmente se originou na escuta do som ambiental (KRAUSE, 2013, p ). O homem primitivo, ao perceber os sons do entorno, teria buscado imitá-los e organizá-los a fim de inseri-los em seus ritos mítico-mágicos. No entanto, a pesar de podermos observar alusões a fenômenos sonoros ambientais em quase todos os repertórios humanos de todas os períodos e regiões do mundo, através de procedimentos miméticos incorporados a um discurso musical de caráter predominantemente abstrato, o emprego do som ambiental em si, como material compositivo, ganha papel solista a partir das experiências da musique concrète (mediado pelas tecnologias de gravação e reprodução) e dos trabalhos de John Cage, em que o som ambiental ingressa inclusive sem mediação tecnológica, como fator de indeterminação. Por exemplo, na emblemática peça Tacet 4 33, que consiste basicamente de um período de silêncio que permite o som ambiental se manifestar sem interferências. A questão principal introduzida por Cage, além do fator de indeterminação, era a possibilidade de compreender o som ambiental como objeto de fruição estética: Em qualquer lugar que estejamos, aquilo que ouvimos maioritariamente é ruído. Quando o ignoramos, ele nos incomoda. Quando o ouvimos, nos parece fascinante. 3 Esta forma de relacionamento com o som do meio ambiente, como objeto artístico pronto, acabado, pode ser observada, no entanto, em outros períodos e autores, por exemplo: um texto taoísta antigo e o ciclo de peças Presque Rien de Luc Ferrari. 2 Há inúmeras descrições de paisagens sonoras anteriores a Schafer tanto nas artes literárias como na própria discussão teórica e estética da música. 3 Tradução minha. No original: Whenever we are, what we hear is mostly noise. When we ignore it, it disturbs us. When you listen to it, we find it fascinating. (CAGE, 1973, p. 3). Vitória - ES - Brazil 2

15 From Digital Arts to Ubiquitous Music V Ubimus Um poema do pensador taoísta Zhuangzi (369?-286? a.c.), apresenta uma oposição entre a música dos homens, compreendido como um fenômeno sonoro limitado e a música da Terra, de caráter ilimitado. No fim do texto, o autor menciona a experiência de um músico célebre (chamado Zhao) que compreende que quando toca um som com seu instrumento acaba negligenciando todos os outros sons do mundo. (ROTHENBERG & ULVAEUS, 2001). Já Luc Ferrari, em Presque Rien opta (em rebeldia às diretrizes de Pierre Schaeffer sobre a musique concrète) gravar o som de um entorno sonoro em particular e apresentá-lo ao público com o mínimo possível de alterações decorrentes de edição e processamento. Poderiamos associar estas propostas em que os sons que o mundo oferece são apresentados quase sem interferência como anti-música? Preferimos argumentar que não. O que está em jogo, em realidade, é compreender a possibilidade de percebermos o som ambiental como um evento passível de ser usado como material musical. São propostas que deixam tácito um convite a se imaginar novas concepções musicais fundamentadas em um diálogo íntimo e concreto (real) com o som do entorno em sua manifestação espontânea, em vez de negá-lo através do isolamento em salas acusticamente fechadas. A partir das experiências de Cage, Murray Schafer apontou um caminho diverso, a sua preocupação não era simplesmente estética senão social e ambiental. Aludindo tacitamente à arte conceitual, a criação do termo implicava imaginar o mundo como uma grande obra de arte sonora, na qual todo ser humano seria partícipe. Cuidar dessa composição coletiva seria tarefa de diversos segmentos da sociedade, e para tal, montou uma equipe de pesquisa na qual o compositor contribuiria com sua sensibilidade : Embora tivéssemos muitos músicos em nossos cursos sobre soundscape, eu sabia desde o início que nós não estávamos treinando compositores, mas tentando definir uma nova profissão que ainda não existia e mesmo hoje não existe na medida do desejável. Eu imaginava um especialista em som que combinasse habilidades técnicas e preocupações sociais com a sensibilidade estética de um compositor. 4 (R. Murray SCHAFER, 1993). O trabalho dos compositores da sua equipe na Simon Fraser University, no entanto, ganhou uma relevância inesperada, criando o que Barry Truax passou a denominar soundscape composition (TRUAX, 2002), um gênero de composição eletroacústica guiado por uma intencionalidade referencial, isto é, compor com materiais que façam alusão a sonoridades ambientais, mas, principalmente, atento às relações presentes no meio ambiente estudado. Surpreendentemente, quase quatro décadas após o surgimento deste estilo, há compositores que declaram fazer soundscape 4 No original: While we had many musicians in our soundscape courses, I knew from the beginning that we were not training composers but were trying to define a new profession that did not yet exist and even today does not exist to the extent desirable. I imagined a sound specialist combining technical skills and social concerns with the aesthetic sensitivity of a composer. 3 Vitória - ES - Brazil

16 V Ubimus From Digital Arts to Ubiquitous Music composition sem considerar o meio ambiente de forma global, tomando uma sonoridade isolada do contexto em uma poética mais próxima à música espectral. 5 A partir da soundscape composition surgiram diversos trabalhos baseados no som ambiental e teorias vinculadas: ecomusic, ecocentric music, eco-composition, ecostructuralism, environmental performance works e ecoacoustic. (GUILMURRAY, 2014). O âmbito destes trabalhos, em geral, é a música computacional, variando entre uma relação íntima entre a percepção e estudo ambiental em si como processo de composição (ecoacoustic) e o uso de teorias ecológicas no ambiente virtual como fundamento para a elaboração de modelos compositivos, com menos preocupação, talvez, em revelar a paisagem em si (eco-composition). Finalmente, devemos considerar o estudo de paisagens sonoras como fundamento para composição de música instrumental. Além dos trabalhos do próprio Schafer (a ópera situacional 6 The Princess of the Stars), 7 há experiências de compositores do núcleo original da soundscape composition combinando eletrônica e instrumentos acústicos (Phantasy for Horns, de Hildegard Westerkampf) e trabalhos do compositor alemão Peter Ablinger, como seu ciclo de Regenstücke, para diversos instrumentos sólo imitando sons de chuva. 8 No Brasil, Ulises Ferretti e Marcelo Villena transitam também nesse âmbito, 9 desenvolvendo metodologias específicas que, evidentemente, não se relacionam de forma direta com a música ubíqua, mas que podem, porventura, colher suas contribuições para enriquecer sua poética. Música ubíqua Se o termo paisagem sonora se origina na escuta ambiental, inicialmente em procura de prazer estético e um fator de indeterminação (Cage), definindo posteriormente seu campo de estudo no âmbito da ecologia acústica (Schafer), para finalmente ser usado como fundamento estético de um gênero específico de música 5 Os princípios estéticos da soundscape composition, que podem ser encontrados no site da SFU a partir de conclusões de Truax, são claros neste sentido: 1) Os sons são trabalhados de maneira que o ouvinte possa reconhecer a origem dos materiais. 2) O conhecimento do ouvinte sobre o ambiente e seu contexto psicológico é invocado. 3) O conhecimento do compositor sobre o ambiente e seu contexto psicológico influencia a forma da composição em todos os níveis. 4) O trabalho aumenta nosso entendimento do mundo e sua influência se estende para nossos hábitos perceptivos do dia-a-dia. (TRUAX, 2002). Tradução livre nossa sobre o texto colocado no site: < 6 Composta para ser interpretada num local específico. 7 Informações adicionais podem ser encontradas no site da ópera: < (R. Murray. SCHAFER, 2014). 8 Informações adicionais sobre estas peças de Ablinger podem ser obtidas no seu web site: < (ABLINGER). Há outros exemplos que poderíamos citar, mas que não se vinculam especificamente a um trabalho contínuo na temática ambiental como é o caso destes autores. 9 (FERRETTI, 2006) e (VILLENA, 2013). Ambos autores trabalharam seus portfólios de mestrado integralmente dentro da ideia da escuta de paisagens sonoras como fundamento para a composição com instrumentos acústicos. Já no doutorado, Ferretti opta por apresentar alguns trabalhos com instrumentos acústicos (peças de palco), alguns eletrônicos (instalações) e outros mistos (instalações com performance). Os links para estes trabalhos estão nas referências bibliográficas. Vitória - ES - Brazil 4

17 From Digital Arts to Ubiquitous Music V Ubimus eletroacústica (a soundscape composition), a música ubiqua toma emprestado seu termo da computação ubiqua, tendo, portanto, um campo de estudo diverso. Este campo não se reduz, evidentemente, ao estudo exclusivo de ferramentas tecnológicas, senão que abrange as interações entre pessoas e dispositivos, e entre estes meios tecnológicos e o meio ambiente. Ao observar a literatura sobre a música ubíqua podemos facilmente perceber que o foco central das atenções é a capacidade que a tecnologia tem de gerar novas relações entre os usuários dos dispositivos e a sua vivência musical, o compartilhamento de dados, ou inclusive à forma em que a tecnologia possibilita novas formas de percepção ambiental. Porém, neste caso estamos permanentemente diante de uma percepção mediada, não direta. Em todos os textos observados torna-se presente a discussão sobre qual dispositivo está sendo usado, qual a ferramenta. Os estudos de paisagens sonoras, em contrapartida, 10 destacam mais a percepção sensorial, a confluência de informações recebidas pelos diferentes sentidos, destacando a vivência corporal no meio ambiente. Esta divergência de focos de pesquisa acabam derivando em trabalhos de caráter diferenciado, seja no âmbito da composição, nas práticas interpretativas ou na educação musical. Tendo em mente esta diferença, voltemos a definição de música ubíqua entendedo-a como uma terminologia derivada da computação. Regina Borges de Araújo nos informa: A idéia básica da computação ubíqua é que a computação move-se para fora das estações de trabalho e computadores pessoais e torna-se pervasiva em nossa vida cotidiana. Marc Weiser, considerado o pai da computação ubíqua, vislumbrou há uma década atrás que, no futuro, computadores habitariam os mais triviais objetos: etiquetas de roupas, xícaras de café, interruptores de luz, canetas, etc, de forma invisível para o usuário. (de ARAUJO, 2003) O conceito, aplicado à música, abre espaço para a criação de inúmeras situações inusitadas, como o caso da peça Pandora do compositor Sérgio Freire, em que uma caixa clara é manipulada à distância por meio de comandos similares aos de um video-game (o Lightning II, um controlador MIDI). A situação performática é recebida pelo público como um truque de mágica, pelo fato do som ser feito prescindindo do contato físico entre performer e instrumento. 11 A música ubíqua fomenta também o trabalho compositivo colaborativo e/ou à distância, com os criadores interagindo por meio da rede de computadores, trocando informações (dados), compartilhando o uso de ferramentas, alterando, em definitiva a ideia consagrada do compositor solitário que escreve notas em uma pauta para posteriormente repassar o resultado a um intérprete que não precisa, a princípio, fazer nenhuma contribuição para a produção desse texto musical. Estes dispositivos, por outro lado, fomentam a inclusão na prática compositiva de pessoas que não conhecem a escrita convencional, mas que em muitos casos, tem ótimo ouvido e sensibilidade, além 10 Além dos autores mencionados no corpo de texto, sugerimos observar os trabalhos de (BARRIOS & RODRÍGUEZ, 2005) e (ATIENZA, 2008). Ver: referências bilbiográficas. 11 Para mais informações sobre a obra, consultar o artigo Pandora: uma caixa-clara tocada à distância (FREIRE, 2007), disponível em: < 5 Vitória - ES - Brazil

18 V Ubimus From Digital Arts to Ubiquitous Music de conhecimentos de outras áreas importantes para o desenvolvimento do processo criativo, como por exemplo acústica e programação de computadores. Um exemplo desse tipo de investigação é o artigo MDF: Proposta Preliminar do Modelo Dentro-Fora de Criação Coletiva, em que Ferraz e Keller propõem um modelo para se refletir quanto aos processos sociais, pessoais e materiais que ocorrem durante a criação musical coletiva (FERRAZ & KELLER, 2012).O objetivo do texto é elaborar critérios de análise dos coletivos de criação em aspectos materiais, humanos e procidimentais. Através dos termos in-group e out-group, os autores estabelecem uma categoria binária para classificar resultados obtidos em pesquisas nesses 3 âmbitos. No aspecto humano avaliam a capacidade de interação entre indivíduos, determinada, de certa forma, pela sua bagagem cultural, por suas experiências anteriores e pelo seu conhecimento musical: [...] a noção de in-group (para dentro) seria equivalente à força de aglutinação que homogeniza os campos epistêmicos, e a noção de out-group corresponderia à força oposta que leva os componentes do grupo para a divergência e eventualmente à desagregação. (FERRAZ & KELLER, 2012) No aspecto material, a dicotomia é estabelecida entre recursos renováveis (aqueles que podem ser utilizados mais de uma vez sem perder a capacidade criativa) e não-renováveis e entre rivais e não-rivais (este último, quando os recursos oferecem a possibilidade de compartilhamento entre os usuários). Já no aspecto procedimental, a dicotomia in-group/out-group é definida pela discriminação entre lixo criativo (aqueles materiais musicais descartados não aceitos pelo grupo) e produto (o material que é consenso coletivo). Vemos através do exemplo deste artigo uma das questões problematizadas pela música ubíqua (pouco explorada por autores que declaram trabalhar a partir do conceito de soundscape): 12 a procura de um corpo teórico sobre os métodos de criação coletiva, de maneira a estabelecer ferramentas de análise da interação entre grupos, um aporte de interesse para outras áreas de pesquisa, inclusive para propostas além da música eletrônica ou computacional. Teoria Ecológica da percepção em trabalhos compositivos de paisagens sonoras e música ubíqua Quando observamos textos de música ubíqua e paisagem sonora salta à vista o uso comum da Teoria Ecológica da Percepção, como fundamento, sobretudo, para a criação musical. O ponto inicial desta teoria pode ser encontrado no livro The Perception of Visual World (GIBSON, 1950) em que James Gibson nos apresenta a genealogia da Ground Theory (teoria do solo, do chão, da terra): a partir de experimentos em aviação militar, na segunda guerra mundial, foi constatado que os 12 Há alguns casos de parceria no Brasil, como a do compositor Ulises Ferretti e a artista plástica Claudia Paim, que embora pertençam a áreas diferentes concebem e compõem juntos as obras tanto no aspecto sonoro quanto visual. Mas de qualquer maneira não apresentam nenhuma análise minuciosa sobre como essa criação compartilhada aconteceu. Vitória - ES - Brazil 6

19 From Digital Arts to Ubiquitous Music V Ubimus pilotos, mesmo no ar, tinham sempre o solo como referência última para sua orientação. Gibson, então, partiu de esta constatação experimental, combinando-a a conceitos próprios da Teoria da Gestaldt para definir esta teoria que considera a percepção, em última instância, um processo relacional direto entre homem (ou outro ser vivo qualquer) e meio ambiente. Um meio ambiente que é decodificado e mapeado (escaneado) a partir das informações colhidas pelos sentidos. Eric Clarke nos apresenta os fatores essenciais a este processo: 1) a relação entre percepção e ação; 2) a adaptação e 3) o aprendizado perceptual. (Clarke, 2005). A relação entre percepção e ação se refere à relação existente entre cada ação executada por um corpo e uma resposta ambiental, desde a diferença entre tocar um objeto inanimado e um animal (o último provavelmente se movimentará), ou às diferentes formas de percepção que podem ser acessadas dependendo da disposição de certas partes do corpo. A adaptação é entendida como o processo que possibilita compreender o mundo para questões de sobrevivencia e que está relacionada a experimentação de estímulos e à memória da resposta ambiental. Já o aprendizado perceptual implica não só questões da relação direta do corpo com o ambiente, mas também aspectos culturais. Os processos de representação mental, por exemplo, são colocados por Clarke (2005, p. 11) nesta última categoria. Mas, evidentemente, estes 3 itens não são compartimentos separados, senão que atuam simultaneamente no processo de sintonizar o corpo com o mundo. 13 Sem querer entrar em detalhes sobre esta teoria, algo que ultrapassa, evidentemente, o objetivo deste breve texto, tentaremos compreender como ela é usada por alguns compositores residentes no Brasil na atualidade. André Luiz de Gonçalves de Oliveira, em seu texto Paisagem sonora como obra híbrida (OLIVEIRA, 2011), procura sua aplicação para problemáticas de espacialização multicanal, compreendendo que a disposição do público na relação convencional de palco italiano estaria divorciada da percepção ambiental. Além do que, o modelo de percepção gibsioniano é tomado como um fundamento estético para interatividade entre espectador e obra. O problema central para o autor, portanto, é encontrar metodologias de composição que permitam produzir uma experiência estética similar à do corpo inserido no meio ambiente. Outro compositor vinculado ao estudo de paisagens sonoras que emprega as teorias de Gibson e Clarke é Ulises Ferretti. Na sua tese Entornos Sonoros. Sonoridades e ordenamentos (FERRETTI, 2011) menciona a Teoria Ecológica em vínculo estreito com sua forma de percepção do meio ambiente ao trabalhar na composição de peças instrumentais e instalações sonoras: [...] essas práticas de escuta [baseadas na teoria de Gibson] estão influenciadas por características do som e pela maneira que se o ouve. Nelas, influem aspectos como a forma esférica em todas as direções de propagar-se o som, a particularidade do sistema auditivo de captar o som proveniente tanto de cima, dos lados e de baixo, e a capacidade auditiva de focar a atenção de maneira diversa. Muitas particularidades que nascem dessas maneiras de escuta têm sido utilizadas nos processos compositivos deste trabalho. Várias delas como as diferenças entre escuta focada, periférica e outras interações adiante expostas diferenciam propostas como 13 O interessante da teoria de Gibson é que ela não dá conta só da percepção humana, mas também da percepção dos animais. 7 Vitória - ES - Brazil

20 V Ubimus From Digital Arts to Ubiquitous Music Duplo Coro [uma instalação sonora] e Canon Tipológico [uma peça instrumental]. (FERRETTI, 2011) Ferretti parece procurar nas suas composições a mesma relação entre experiência ambiental e obra, destacada por Gonçalves, com a diferença, talvez, de dar maior ênfase à fase inicial do processo, à percepção ambiental como motivação compositiva, descrevendo em seus textos a recepção dos fenômenos sonoros ambientais minuciosamente. Parece empregar a teoria de Gibson para melhor mergulhar nos entornos e guardá-los na memória. Na concepção da obra, assimesmo,a inserção do corpo no ambiente de difusão é uma problemática central, considerando inclusive a questão das diferenças de perspectiva na escuta do meio ambiente. E é neste ponto que Ferretti reconhece a utilidade das ferramentas tecnológicas e, por extensão, das pesquisas em música ubíqua. Na música ubíqua, por outro lado, os trabalhos compositivos não destacam, aparentemente, a centralidade da percepção do corpo no ambiente de estudo. Não fazem esta referência direta à paisagem sonora em si. A Teoria Ecológica é empregada dentro da ideia de criação de modelos compositivos (modelos ecológicos) através de ferramentas computacionais. Eric Clarke nos dá indícios para compreender estes procedimentos quando trata sobre os modelos conexionistas em computação: O modelo conexionista, que foi amplamente discutido na psicologia e nas ciências computacionais [...] se diferencia em si mesmo da tradional Inteligência Artificial [AI no original em ingles] por afirmar que o processo perceptual e cognitivo pode ser modelado como a forma [property] de distribuição da totalidade de um sistema, não de uma parte em particular da qual processa qualquer conhecimento em si, ou do funcionamento de papéis especificos operando em direcionamentos específicos que contém representações de conhecimentos delimitados (uma crua caracterização da AI). Um modelo conexionista típico consiste de uma rede de nós, interligados por conexões que podem tomar valores variáveis representando sua extensão (ou peso). Uma camada de unidades de entrada de informação [inputs] é conectada a uma camada de unidades de saída [outputs], com um número variável de camadas ocultas (usualmente não mais que duas ou três). Quando as unidades de saída são estimuladas, um padrão de ativação corre através da rede, dependendo dito padrão da estrutura das conexões e os pesos assinados a eles, que convergem em direção a um número de unidades de saída. Convencionalmente, a rede é inicialmente ativada com valores aleatórios [random] em relação ao peso das conexões, de maneira que a primeira ativação resulta de um comportamento aleatório do sistema como um todo. 14 (Clarke, 2005). 14 No original: Connectionist modeling, which was widely discussed in psychology and computer science [ ] differentiates itself from traditional Artificial Intelligence (AI) by claiming that perceptual and cognitive processes can be modeled as the distributed property of a whole system, no particular part of which possesses any knowledge at all, rather than as the functioning of explicit rules operating on fixed storage addresses which contain representations or knowledge stores (a crude characterization of AI). A connectionist model typically consists of a network of nodes, interlinked with connections that can take variable values representing their strength (or weight). A layer of input units is connected to a layer of output units, with a variable number of hidden layers (usually no more than about two or three) in between. When input units are stimulated, a pattern of activation spreads through the network, the pattern depending on the structure of the connections and the weights assigned to them, and converging on a number of output units. Typically, the network is initially set up with random values assigned to the connection weights, so that the first activation results in random behavior of the system as a whole. Vitória - ES - Brazil 8

21 From Digital Arts to Ubiquitous Music V Ubimus A rede de conexões de um sistema computacional, desta maneira, pode ser programado para funcionar de maneira similar a um ecossistema, observado como uma estrutura, com suas relações entre partes componentes. Pode-se programar respostas a determinadas ações ou estimulá-las por uma simples recorrência maior do estimulo (no caso: informação computacional). Estes tipos de procedimentos computacionais parecem ser a base para a construção de modelos compositivos empregados preferencialmente no campo de música ubiqua, sob o termo ecocomposição. Uma pergunta que pode ser feita é em que medida essa forma de proceder não é uma continuação da línea de pensamento da música estocástica, inclusive se considerarmos que o modelo pode ser empregado sem fins de referencialidade extramusical. A procura por modelos leva em si um grau de abstração que prescinde do caso particular. Vejamos em outro âmbito: se procuramos um modelo de análise estrutural de mitos indígenas, por exemplo, estamos tentando dar conta não das diferenças entre cada mito, senão das generalidades que permitem estabelecer vínculos entre as diferentes narrativas. Os trabalhos dos paisagistas, ao contrário, tendem a tratar das particularidades locais, como veremos a seguir. Cartografias sonoras Em sua dissertação de mestrado, Cartografias sonoras: um estudo sobre a produção de lugares a partir das práticas sonoras contemporâneas (NAKAHODO, 2014), a artista sonora Lilian Nakahodo procura a construção de um corpo teórico sobre trabalhos vinculados às paisagens sonoras a partir de uma ótica da geografia e das ciências sociais. Esclarece desde o início que a proposta se origina numa postura humanista, focado em aspectos subjetivos da percepção do que denomina lugares. [...] neste mundo urbano contemporâneo, atravessamos um período de uniformidade na intermediação dos relacionamentos cotidianos; há que se ter bandas cada vez mais largas para as conexões, voos cada vez mais numerosos, shoppings cada vez maiores, mais... mais... em tempos cada vez menores. Neste contexto, vive-se mais pela tela de um computador, pelos fones plugados em um estéreo pessoal e enviando mensagens via whatsapp. Essa realidade marcada pela velocidade e supostos encurtamentos de distâncias é, aparentemente, um reflexo das transformações da sociedade que cria esses espaços que se pode denominar como não lugares. (NAKAHODO, 2014). A postura adotada não deixa lugar a dúvidas. A pesar de ser um trabalho teórico em procura de subsídios para um trabalho compositivo pessoal permeado integralmente pelo uso de recursos tecnológicos (gravadores portáteis, GPS, softwares multitrack e sistemas de espacialização) a autora apresenta um discurso crítico, em que o uso indiscriminado destes dispositivos, sem um objetivo humanista pode se tornar um empecilho à construção de relações de intimidade com o entorno. Os termos chave do trabalho, lugares e não lugares (emprestados do geógrafo Yi-fu Tang e do antropólogo Marc Augé), são propostos como metodologia para a prática de soundwalk, a caminhada de escuta de paisagens sonoras. A construção de lugares que Nakahodo propõe, em oposição aos não lugares, impessoais, fugazes, uniformes, é processada através do contato físico do compositor com o ambiente, na vivência íntima com o local de estudo, na construção de laços afetivos e na descoberta (tal qual Ferretti menciona) de diferentes perspectivas de 9 Vitória - ES - Brazil

22 V Ubimus From Digital Arts to Ubiquitous Music escuta. No entanto, Nakahodo parece ir além, quando comenta seus procedimentos de escuta pré-composicional, num território lindante com o perspectivismo ameríndio: 15 Passo a gravar as caminhadas com o intuito de fazer uma peça grilesca, um dos temas preferidos dos paisagistas sonoros, admito. Mas queria que meus grilos tivessem um tratamento diferente, então pus a gravá-los de todos os ângulos que me foi possível captar com meu gravador digital portátil, até imaginar que poderia ser, de fato, um deles. (NAKAHODO, 2014). A distância entre esta forma de proceder e a procura de modelos composicionais da eco-composição é considerável. A tecnologia é empregada dentro de uma metodologia muito diversa, que oferece, em definitiva, um objeto artístico que visa a relação com um entorno específico. Análise comparartiva de duas peças Nesta última seção abordaremos algumas questões da poiética compositiva de duas peças: touch n go/toco y me voy ( ), de Damián Keller e Urbana A2 (2010), instalação do compositor Ulises Ferretti e a artista plástica Cláudia Paim. A intenção é observar as formas de pensamento por trás de uma produção da música ubíqua e uma composição motivada por um entorno sonoro que é deslocado para um local específico. Para começar, Keller nos informa em seu artigo Compositional Process from an ecologycal perspective (2000), que touch n go/toco y me voy 16 pode ser apresentada em três diferentes formatos: 1) como tape music, em sistema de espacialização de oito caixas de som; 2) como peça estéreo com hipertext markup language (HTML); e 3) como uma performance ao vivo com um ator bilíngue ou dois atores + sistema 8 canais. Já Urbana A2, de Ferretti e Paim, é uma instalação sonora e visual, concebida em função de um local e um ritual: para ser apresentada na chegada do público a um concerto da Orquestra de Câmara do Theatro São Pedro (Porto Alegre), aproveitando as características arquitetônicas do espaço para projetar som e imagem, e o contexto social específico. Em vez de dois atores declamando um texto, neste caso contamos com dois músicos realizando uma performance. Outra diferença marcante é a preocupação de Keller de oferecer a obra em formato doméstico : um CD em que a peça (feita de seções independentes, mas relacionadas na temática) pode ser ouvida de maneira similar às opções de leitura Rayuela, de Julio Córtazar, na sequencia disposta pelo autor ou de forma randômica. Urbana A2, ao contrário é uma peça para ser degustada na circunstância e local singulares em que fora concebida. O registro em áudio e vídeo são meramente ilustrativos da experiência que implica, em definitiva, a discusão da cidade num ritual social. É arte efêmera, difícil de ser reapresentada, que se vivencia e se guarda na 15 O antropólogo Viveiros de Castro emprega esse termo para se referir ao processo xamânico pelo qual diversos povos brasileiros tradicionais referem sua procura de uma percepção afinada com a percepção dos animais (ou dos espíritos-animais). (VIVEIROS DE CASTRO, 2004). 16 A peça pode ser ouvida on-line no endereço: < (KELLER, 2014) Vitória - ES - Brazil 10

23 From Digital Arts to Ubiquitous Music V Ubimus memória, como uma paisagem sonora que nos pegou de surpresa sem o gravador portátil. 17 Toco y me voy, por outro lado contém partes que foram estruturadas a partir de um texto literário o conto de Jorge Luis Borges El jardín de los senderos que se bifurcan. Tanto macro-estruturalmente, como obra aberta no modelo cortazariano, quanto no interior de algumas partes, baseadas num conto, a obra é concebida em diálogo com a literatura e talvez (sugerimos isto após a experiência de escuta) com a estética da peça radiofônica. Não há uma estrutura que possa ser relacionada com a rememoração de um meio ambiente de características únicas, mas um fluxo de grande variedade de informações ambientais e culturais, que convivem numa concepção relacionada à Perspectiva Espacial Variável, se empregarmos a terminologia de Truax (TRUAX, 2002) ao referir o tipo de soundscape composition mais esquizofônica. O conceito schaferiano de esquizofonia 18 (isto é, a colocação das sonoridades de uma paisagem sonora em outro contexto) também pode ser invocado na instalação de Ferreti e Paim. Amostras de áudio captadas no Túnel Conceição (Porto Alegre) e filmagens desse e outros entornos da cidade de Porto Alegre são projetados no hall de entrada e na sala de concertos do teatro. Porém, som e imagem tratam do mesmo assunto: o trânsito na cidade. Além do quê, esses registros do entorno do túnel são inseridos numa experiência que reforça a vivência do ritual que está acontecendo no local. A ideia da performance é refletir sobre o processo de se desligar da correria do dia-a-dia para participar de um ritual estético (um concerto de música clássica). As pessoas que chegavam eram recebidas por um violinista que, vestido de terno e gravata, ficava do lado de fora do teatro tocando músicas incompletas. Ao ingressar no hall, deparavam-se com um flautista, vestido com ropa informal, também tocando trechos de música. Junto a isso, na fachada do teatro foram projetadas imagens de vídeo com cenas da cidade, enquanto que todo o hall estava cheio de sons da paisagem sonora do Tunel Conceição. Finalmente, ao ingressar na sala de concertos, os músicos da orquestra já se encontravam ensaiando partes do concerto sob o som das paisagens e as imagens de vídeo projetadas no telão. A ideia da performance-instalação foi deslocar as situações, colocando o instrumento mais associado à música erudita (vestido de gala) na rua, no suposto âmbito da música popular e os instrumentos mais próximos ao repertório popular (flauta doce e transversal) vestido informalmente no hall do teatro (por sinal, o mais representativo da tradição oitocentista da cidade). Foi um deslocamento de funções sociais pensado como forma de gerar uma transição entre os dois ambientes radicalmente opostos e (por que não?) um certo grau de confusão perceptual. Esta breve análise dos dois trabalhos serve para reafirmar as características distintivas das propostas: uma centrada nas possibilidades compositivas que os diversos materiais ambientais e culturais podem oferecer para uma peça de fundo político, a outra, vinculada de forma estreita a vivência de dois lugares (como diria Nakahodo), inventando um dispositivo performático vivo (pessoas tocando instrumentos acústicos) que gera uma inversão de valores sociais, para que o público, em última instância, 17 Ou mesmo se tivéssemos o gravador. A experiência de escuta da paisagem ao vivo é única e irrepetível por meios eletrônicos. 18 (R. Murray SCHAFER, 1992) 11 Vitória - ES - Brazil

24 V Ubimus From Digital Arts to Ubiquitous Music reflita sobre as questões sociais que estão implicadas na sua vivência desse lugar e esse ritual. A modo de conclusão início Esta discussão não tem conclusão: é o início de um diálogo possível entre duas áreas de pesquisa de características diferentes mas que podem se retroalimentar. Os autores que trabalham sob o conceito de paisagem sonora pretendem continuar, aparentemente, a basear suas criações pela intencionalidade evocativa dos entornos, pretendem trazer à tona, de diferentes maneiras, a relação do ser humano com o meio ambiente. Os autores que tratam da computação ubíqua aplicada a criação musical dedicam-se a mostrar como a tecnologia abre novas portas à metodologia de composição musical e instigam novas interações sociais. Pretender englobar as pesquisas de uma área no âmbito da outra não contribui ao desenvolvimento do conhecimento, é uma tentativa estéril de esmagar as divergências: a tentativa de compor fundamentado antes que nada na observação atenta dos fenômenos espontâneos do mundo (um processo de captura de algo externo ) ou a dedução de modelos para uma forma de composição de caráter talvez mais interno. O foco no estudo do meio ambiente dos paisagistas pode alimentar a descoberta de modelos compositivos para os ubiquos e a pesquisa tecnológica pode trazer novas ferramentas para transmitir as sensações ambientais, por exemplo (como aponta Ferretti), 19 para emular as diferentes perspectivas de percepção ambiental numa instalação sonora. Reconhecer as diferenças talvez seja o caminho para crecermos juntos. Post-script Após a conclusão do presente artigo o compositor Damián Keller ofereceu novas leituras para complementar o texto e que colocam em questionamento algumas afirmações do presente artigo. A partir do artigo Composing with Soundscapes: an Approach Based on Raw Data Reinterpretation (GOMES et al., 2014) tomamos novos conhecimento de preocupações da música ubiqua em relação a questões relacionadas intimamente aos fundamentos estéticos da soundscape composition: a preocupação por trabalhar a composição com base em dados históricos, etnográficos e geográficos, algo comum, de qualquer maneira em trabalhos de paisagistas sonoros de outras vertentes. O problema, no entanto, é saber de que forma esses dados são usados no trabalho compositivo. O artigo menciona, por outro lado, ferramentas tecnológicas para a coleta de dados geográficos, de grande utilidade para o trabalho dos compositores, desenvolvidas no projeto URB. 20 Outro aspecto que deve-se destacar é a relação da eco-composição com outras vertentes ecológicas, como o ecoestruturalismo, que trabalha a partir de padrões da análise de entornos naturais que revelam [em definitiva] estruturas [intrínsecas] nos próprios materiais. (GOMES et al., 2014)); assim como a forte relação da ecocomposição com as propostas de Agostino Di Scipio, Mathew Burtner e David Monacchi. Estes dois últimos (com os quais já tinhamos certa familiaridade anteriormente), coincidem no uso do termo ecoacoustic para definir seu trabalho 19 Observação do compositor em conversa via vídeo conferência Vitória - ES - Brazil 12

25 From Digital Arts to Ubiquitous Music V Ubimus criativo, manifestando um engajamento explícito no ativismo ambiental, principalmente na escolha de ecossistemas em crise como locais de pesquisa: a Floresta Amazônica e as regiões geladas do Ártico. O trabalho de Monacchi, sobretudo, consegue evocar de forma muito convincente a sensação de escuta inserido na floresta tropical, empregando os drones eletrônicos para reforçar os ritmos e a espectromorfologia presente no espaço estudado. 21 Finalmente, as leituras revelaram o uso de conceitos da eco-composição em trabalhos com meios acústicos do compositor Rick Nance, como presente em sua tese Compositional explorations of plastic sounds (NANCE, 2014). Isto é, os fundamentos teóricos e os procedimentos da eco-composição podem perfeitamente serem aplicados a propostas que fogem do campo da música ubiqua, se entendemos esta última como uma extensão da computação ubiqua. Referências bibliográficas ABLINGER, P. Peter Abliger web site. ATIENZA, R. (2008). Identidad sonora urbana. Les 4èmes Jounées Europénnes de la Recherche Architecturale et Urbaine EURAU'08 : Paysage Culturel, 1, BARRIOS, I., & RODRÍGUEZ, J. D. G. (2005). Calidad acústica urbana: influencia de las interacciones audiovisuales en la valoración del ambiente sonoro. Medio Ambiente y Comportamiento Humano, 8, CAGE, J. (1973). Silence. Lectures and Wirtting by John Cage. (1º ed.). Hannover: Wesleylan University Press. Clarke, E. (2005). Ways of listenings. An ecological approach of musical meaning. (First ed. Vol. 1). New York: Oxford University Press New York. de ARAUJO, R. B. (2003). Computação ubíqua: princípios tecnologias e desafios. Paper presented at the XXI Simpósio Brasileiro de Redes de Computadores, Natal. FERRAZ, S., & KELLER, D. (2012). Preliminary proposal of the MDF model of collective creation (MDF: Proposta preliminar do modelo dentro-fora de criação coletiva). Paper presented at the Proceedings of the III Ubiquitous Music Workshop (III UbiMus). São Paulo. FERRETTI, U. (2006). Entorno sonoro del cotidiano. Cinco piezas instrumentales. (Mestrado Dissertação), Universidade Federal do Rio Grande do Sul, Porto Alegre. Retrieved from e=1 FERRETTI, U. (2011). Entornos sonoros. Sonoridades e ordenamentos. (Doctorado Tesis), Universidade Federal do Rio Grande do Sul, Porto Alegre. Retrieved from ce=1 FREIRE, S. (2007). Pandora: uma caixa clara tocada a distância. Paper presented at the Simpósio Brasileiro de Computação Musical, São Paulo. 21 Audios disponíveis no web site do autor: < (MONACCHI, 2014) 13 Vitória - ES - Brazil

26 V Ubimus From Digital Arts to Ubiquitous Music GIBSON, J. (1950). The perception of the visual world (H. Mifflin Ed.). Massachussetts: The Riverside Press. GOMES, J. A., de Pinho, N. P., LOPES, F., COSTA, G., DIAS, R., & BARBOSA, Á. (2014). Composing with Soundscapes: an Approach Based on Raw Data Reinterpretation - xcoax2014-gomes.pdf. Paper presented at the Second conference on Computation, Communication, Aesthetics and X, Porto. GUILMURRAY, J. (2014). ECOACOUSTICS: Ecology and Environmentalism in Contemporary Music and Sound Art. mentalism_in_contemporary_music_and_sound_art KELLER, D. (Producer). (2014). Touch n Go. [Music] Retrieved from KRAUSE, B. (2013). A grande orquestra da natureza. Descobrindo a origem da música no mundo selvagem. (I. W. Kuck, Trans. 1º ed. Vol. 1). Rio de Janeiro: Jorge Zahar Editor. MONACCHI, D. (2014). David Monacchi - Sound design. from NAKAHODO, L. (2014). CARTOGRAFIAS SONORAS: Um estudo sobre a produção de lugares a partir de práticas sonoras contemporâneas. (Mestrado), Universidade Federal do Paraná, Curitiba. NANCE, R. (2014). Plastic Music and Aural Models. OLIVEIRA, A. L. G. d. (2011). Paisagem Sonora como obra híbrida: espaço e tempo na produção imagética e sonora. Semeiosis. ROTHENBERG, D., & ULVAEUS, M. (2001). The book of music and nature (D. Rothenberg & M. Ulvaeus Eds. 2º ed. Vol. 1). Middletown: Wesleylan University Press. SCHAFER, R. M. (1992). O ouvido pensante (Primeira ed.). São Paulo: Fundação Editora da UNESP. SCHAFER, R. M. (1993). Voices of Tyranny (J. Donelson Ed. Second ed.). Ontario: Arcana Editions. SCHAFER, R. M. (2014). Princess of the Stars. from TRUAX, B. (2002). Genres and Techniques of Soundscape Composition as developed at Simon Fraser University. Retrieved 21/04/2014, 2014, from TRUAX, B. (2014). Soundscape Composition. from VILLENA, M. (2013). Paisagens sonoras instrumentais. Um processo compositivo através da mímesis de sonoridades ambientais. (Mestrado Dissertação), Universidade Federal do Paraná, Curitiba. Retrieved from A3o-Marcelo-Ricardo-Villena-2013.pdf VIVEIROS DE CASTRO, E. (2004). Pespectivismo e naturalismo. O que nos faz pensar?, 18, Vitória - ES - Brazil 14

27 From Digital Arts to Ubiquitous Music V Ubimus Exploring the potential of mobile technology for creating music collaboratively Yeray Real-Delgado, Didac Gil de la Iglesia, Nuno Otero Department of Media Technology, Linnaeus University, Sweden yerayrd@student.lnu.se, didac.gil-de-la-iglesia@lnu.se, nuno.otero@lnu.se Abstract. We will present the first prototype of a framework that supports collaborative music creation activities using short distance-location aware mobile technology. In order to explore the corresponding design space we are planning to run a series of workshops with practitioners to elicit knowledge, find likes and dislikes. Such activities will frame the creation of new features. This is part of a long-term goal to explore how mobile technologies can enable the emergence of ubiquitous music activities. 1. Framing the idea and creating the first prototype According to Keller et al. (2011), recent digital technology developments and artistic explorations put the creation of sonic products beyond the traditional frames of learning to play musical instruments and accompanying social practices. Keller et al. (2011) see interactive installations, performance art, eco-composition, co-operative composition, mobile music and network music as instantiations of what they call ubiquitous music. Ubiquitous music research intends to investigate the social practices involved in these activities and create new ensembles of artifacts to support them. We believe that this conceptual framework proposed by Keller et al. (2011) is useful for researchers and practitioners alike in order to give full account of the current and emerging music practices. Considering the domain, collaborative music creation using mobile technology within the ubiquitous music research framing, our research focuses on the following research questions: How can relative-position aware mobile technology support collaborative music creation activities, such as mixing and sequencing sounds? Which interactions occur when the proposed system is deployed and the users are able to edit, mix and sequence sounds? During the last months, we have implemented a number of mobile application prototypes with the aim of studying the potential benefits that mobile devices can bring in the field of ubiquitous music. The different prototype versions are the results of iterative software development process that has included co-design phases and users' studies. Currently, the latest version of the mobile application, which will be presented during the event, incorporates interaction choices that reflect the results from our previous studies. Some examples of interaction choices include visualization elements to provide a user's friendly experience, software mechanisms to support collaboration in 15 Vitória - ES - Brazil

28 V Ubimus From Digital Arts to Ubiquitous Music a transparent manner and novel usages of emerging communication technologies to provide location context awareness. More concretely, the current mobile prototype is based on the following technologies; Near Field Communication (NFC) and Multi-Agent System (MAS). NFC is a wireless technology of short-range-communication that enables its user to connect the physical world with the virtual (Want, 2011). This technology is being applied to provide location awareness to the application, and to allow rich interactions based on the specific placement of the mobile devices. In order to support real-time collaboration through the mobile devices, the prototype uses a Multi-Agent System (aka, MAS) framework (Wooldridge, 2002) and recent developments based on previous efforts on mobile collaboration (Gil et al., 2014). The solution allows continuous communication between the mobile devices for coordination and collaboration, which could enable the users to perform individual tasks (e.g., recording audio samples to be used in a music composition, the selection of an instrument to be emulated via the mobile device and personal configuration parameters, such as the device volume) as well as to perform collaborative tasks between the participants in the activity (e.g., defining, for each mobile device, its initial and ending time for playing a sound within the music composition, and providing a tangible interface for discussion the structure of the music composition). 2. Future steps At this current stage, we have run three user studies with novices in the field of music composition. Through these studies, we have been able to identify a number of features that the combination of connected mobile devices and NFC technologies can provide for music creation and to enrich the learning discussions in such area. We are planning to run workshops with music experts/practitioners with different levels of formal music education in order to explore what features these users judge to be more useful and engaging from an their perspective. We expect that an efficient and engaging system will not only be more readily adopted but will also promote creative outcomes. Our initial results support this statement, and we believe that our current and future efforts will provide stronger evidences of the benefits of our proposed solution. References Gil de la Iglesia D, et al. (Forthcoming) (2014). A Self-Adaptive Multi-Agent System Approach for Collaborative Mobile Learning. Under submission process in Transactions on Learning Technologies. Keller et al. Convergent trends toward ubiquitous music. Journal of New Music Research (2011) vol. 40 (3) pp Want, R. (2011). Near Field Communication. IEEE Pervasive Computing. vol. 10, n. 3 pp. 4-7, July-September. Wooldridge M. (2009). An Introduction to MultiAgent Systems (2nd ed.). John Wiley & Sons. ISBN-13: Vitória - ES - Brazil 16

29 From Digital Arts to Ubiquitous Music V Ubimus DroneUnknown: An experiment in embracing unpredictability in live electronic performance Patrick McGlynn National University of Ireland, Maynooth patrick.j.mcglynn@nuim.ie Abstract. DroneUnknown aims to explore this ambiguous space by capitalising on the unpredictability of self modifying instruments, rather than trying to restrain it. The program runs on intunative (the author's own multitouch performance platform formerly known as Oscar) and draws random source material from a bank of samples at initialisation time. The player must navigate through the material live during performance and discover it as the audience does. This leads to a man machine duel with a number of possible approaches, ranging from a tentative exploration of the programs state to much more aggressive journeys. The presentation will feature a discussion of the gestural tools available to the performer and conclude with a live demonstration of DroneUnknown. Further information on the performance software can be found at intunative.com. 17 Vitória - ES - Brazil

30 V Ubimus From Digital Arts to Ubiquitous Music V Workshop em Música Ubíqua (V UbiMus), Vitória, ES, 29 de outubro a 3 de novembro de 2014 Suporte para a Criatividade Musical Cotidiana: mixdroid Segunda Geração Flávio Miranda de Farias 1,2,3, Damián Keller 1, Floriano Pinheiro da Silva 1, Marcelo Soares Pimenta 2, Victor Lazzarini 3, Maria Helena de Lima 2, Leandro Costalonga 4, Marcelo Johann 2 1 NAP, Universidade Federal do Acre (UFAC) e Instituto Federal de Ciência e Tecnologia do Acre (IFAC) Rio Branco, AC BR 2 Instituto de Informática e Colégio de Aplicação Universidade Federal do Rio Grande do Sul (UFRGS) Porto Alegre, RS BR 3 National University of Ireland, Maynooth 4 Nescom, Universidade Federal de Espírito Santo, São Mateus, ES BR Abstract. fmflavio@gmail.com, dkeller@ccrma.stanford.edu This paper focuses on mixing as the object of study of creativity-centered interaction design. We applied the time tagging metaphor to develop a new ubiquitous music prototype and carried out experimental work to investigate the relationships between the technological support strategies and their creative yield. A musician produced 30 sound mixes using different tools and similar sound resources in the same location. From that output, three creative products each of approximately 3 minutes were chosen. For the first session he used the sound editor Audacity. The second session was done with the ubiquitous music system mixdroid 1.0 or first generation (1G). The third session involved the use of a new prototype mixdroid 2.0 or second generation (2G). The time invested on each mix was: 97 minutes using Audacity; 6:30 minutes using mixdroid 1G; and 3:30 minutes using mixdroid 2G. 24 subjects evaluated the three products through the Creative Product Profile (CrePP-NAP) protocol. Results indicated very similar profiles for the mixdroid 1G and 2G products. On a scale of -2 to +2, differences weren't larger than 17 cents. Scores for the descriptors 'relaxing' and 'pleasant' were 0.96 and 1.42 points higher for the Audacity-made product, but variations among scores were also high. Originality and expressiveness were slightly higher for Audacity 21 and 42 cents respectively. In contrast, the relevance factor of the mixdroid 2G product was 25 cents higher than the score given to the Audacity product. We conclude that the application of the time tagging metaphor boosts the efficiency of the creative activity, but that boost does not extend to the creativity profile of the products. Resumo. Adotamos a atividade de mixagem como objeto de estudo do suporte tecnológico necessário para atividades criativas musicais. Mais especificamente, aplicamos a metáfora de marcação temporal [Keller et al. 2010] ou time tagging como forma de utilizar as pistas sonoras existentes no ambiente para determinar os tempos de ataque dos eventos sonoros durante a mixagem. Realizamos um estudo experimental de caráter exploratório com o objetivo de investigar a relação entre a infraestrutura de suporte e os resultados criativos. Um músico executou trinta mixagens utilizando os mesmos recursos materiais (amostras sonoras e local de realização) em três condições diferentes. Na primeira sessão foi usado um editor de Vitória - ES - Brazil 18

31 From Digital Arts to Ubiquitous Music V Ubimus V Workshop em Música Ubíqua (V UbiMus), Vitória, ES, 29 de outubro a 3 de novembro de 2014 áudio para dispositivos estacionários: Audacity. Na segunda foi usada a ferramenta musical ubíqua mixdroid 1.0 [Radanovitsck et al. 2011]. Para a terceira sessão foi implementado um novo protótipo embasado na metáfora de interação marcação temporal: mixdroid 2G ou segunda geração. Desses resultados foram escolhidos três produtos criativos de aproximadamente 3 minutos de duração. O tempo de realização de cada uma das mixagens foi: 97 minutos Audacity; 6,5 minutos mixdroid 1G; 3,5 minutos mixdroid 2G. 24 sujeitos leigos avaliaram as mixagens utilizando a ferramenta CrePP-NAP de aferição do perfil do produto criativo. Os resultados indicam que os produtos criativos obtidos com mixdroid 1G e 2G são similares. Não observamos diferenças maiores do que 17 centésimos numa escala de -2 a +2. Já a aferição dos produtos criativos realizados com o editor Audacity apontou diferenças nos descritores relaxante e agradável, ficando entre 1,42 e 0,96 pontos acima dos escores dados aos produtos feitos com mixdroid 1G e 2G. No entanto, a variabilidade das respostas também foi alta. Os itens originalidade e expressividade foram levemente superiores nas avaliações do produto feito com Audacity (21 e 42 centésimos respectivamente). Mas no fator relevância, o produto obtido com mixdroid 2G teve uma média de 25 centésimos acima da média dada à mixagem realizada com Audacity. Concluímos que a aplicação da metáfora de marcação temporal aumenta significativamente a eficiência da atividade criativa, porém os produtos não são necessariamente mais criativos que os produtos resultantes de estratégias de suporte assíncrono. 1. Introdução Na pesquisa em música ubíqua, abordamos o problema do suporte à criatividade aplicando três estratégias experimentais: (1) estudos das atividades prévias ao produto criativo, (2) estudos das atividades realizadas durante a geração do produto, (3) estudos de aferição dos resultados obtidos [Keller et al. 2013a]. A primeira categoria abrange os estudos de design de suporte tecnológico para atividades criativas [Lima et al. 2012; Pimenta et al. 2012]. O foco desse tipo de pesquisa é entender as implicações das decisões de design e as demandas e o impacto nos recursos materiais e sociais utilizados durante o processo criativo. A segunda categoria é ativamente desenvolvida na área de interação humano-computador e envolve a observação das ações dos participantes durante atividades criativas, com ênfase nos aspectos funcionais e utilitários do suporte à interação [Keller et al. 2010; Keller et al. 2013b; Radanovitsck et al. 2011; Pinheiro et al. 2012; Pinheiro et al. 2013; Pimenta et al. 2013]. A terceira categoria foca a observação de aspectos da criatividade através da aferição dos produtos criativos. Neste artigo relatamos os resultados da utilização de produtos criativos para comparar o perfil de suporte à criatividade cotidiana, utilizando como estudo de caso a atividade de mixagem. O problema enfocado neste trabalho pode ser separado em dois aspectos. Primeiro temos a questão da aferição do suporte a atividades criativas. Nas atividades assíncronas o fator temporal relacionado à eficiência do suporte não é o principal determinante da qualidade da interação [Miletto et al. 2011]. Já nas atividades síncronas o fator temporal deveria ter um grande impacto na qualidade da interação. Por esse motivo, quando a atividade criativa é adotada como objeto de estudo, os mecanismos síncronos e assíncronos não são comparáveis. Para viabilizar a comparação entre sistemas de suporte síncronos e assíncronos, aferimos os produtos criativos resultantes das atividades em lugar de focar a observação das atividades em si. A segunda questão é se os sistemas assíncronos permitem obter resultados qualitativamente melhores do que 19 Vitória - ES - Brazil

32 V Ubimus From Digital Arts to Ubiquitous Music V Workshop em Música Ubíqua (V UbiMus), Vitória, ES, 29 de outubro a 3 de novembro de 2014 os sistemas síncronos. Se a resposta for afirmativa, a quantidade de tempo investida na atividade deveria ter impacto no perfil dos produtos criativos. Os dois problemas conceituais levantados a dificuldade de comparar atividades criativas síncronas e assíncronas e a relação entre o tempo investido no ciclo criativo e o perfil dos produtos criativos resultantes indicam a necessidade de separar a metodologia em duas partes. Inicialmente precisamos coletar dados sobre o tempo investido na atividade, mantendo padronizados a duração dos produtos e as condições experimentais em múltiplas sessões. Seguidamente, podemos aferir os produtos obtidos. Os resultados dessa aferição nos permitem avaliar de forma indireta o impacto do sistema de suporte e a relação entre o tempo investido e o perfil dos produtos criativos. O texto a seguir está divido em três seções. Na primeira apresentamos as características principais das três ferramentas utilizadas no estudo experimental. Discutimos a motivação para o desenvolvimento de um novo aplicativo de suporte para atividades criativas em contexto ubíquo e fornecemos exemplos sucintos de uso em atividades síncronas e assíncronas. Na segunda seção descrevemos os procedimentos de geração e escolha de material sonoro utilizado no experimento descrito na terceira seção. Seguidamente descrevemos o protocolo aplicado para aferir o perfil dos três produtos criativos e apresentamos os resultados obtidos em duas sessões experimentais envolvendo 24 sujeitos leigos. Com base nesses resultados, analisamos as implicações dos perfis dos produtos criativos e indicamos as limitações e as perspectivas abertas pela comparação entre atividades criativas síncronas e assíncronas. 2. A marcação temporal em mixdroid 1G e mixdroid 2G Com o intuito de viabilizar as atividades criativas em contexto ubíquo, Keller e coautores (2010) sugeriram o desenvolvimento de metáforas de interação baseadas no mecanismo cognitivo de ancoragem. Como prova de conceito foi desenvolvida a primeira geração de protótipos mixdroid [Radanovitsck et al. 2011] no sistema operacional aberto Android para dispositivos portáteis. O protótipo mixdroid 1.0 (ou 1G ou clássico) permite combinar sons em tempo real através de um teclado virtual com nove botões acionados pelo toque na tela sensível. A atividade de mixagem está baseada no disparo de sons através de botões e no registro dos tempos de acionamento. Dado que o controle se limita a um único parâmetro (o tempo), as habilidades exigidas estão muito aquém das aplicadas na execução de um instrumento acústico, não dependem de um sistema simbólico a ser aprendido, e podem ser aprimoradas em função das características do material sonoro utilizado. Esse mecanismo permite a execução rápida de até nove sons, dependendo exclusivamente da pré-configuração da matriz de sons que é construída durante a atividade de seleção, através do carregamento de cada amostra individualmente para cada botão da interface. Devido à adoção do formato de áudio estéreo, o resultado de uma sessão pode ser reutilizado como amostra dentro de uma nova sessão, de forma similar ao processo de overdubbing usado nos sistemas analógicos de gravação (figuras 1 e 2). Vitória - ES - Brazil 20

33 From Digital Arts to Ubiquitous Music V Ubimus V Workshop em Música Ubíqua (V UbiMus), Vitória, ES, 29 de outubro a 3 de novembro de 2014 Figuras 1 e 2. mixdroid 1G [Radanovitsck et al. 2011]. Figura 3 e 4. mixdroid 2G tela de mixagem. O protótipo mixdroid 2G versão 2.0 beta amplia as possibilidades de aplicação da metáfora de marcação temporal introduzindo novas funcionalidades na leitura e gravação dos dados sonoros (figuras 3 e 4). Para o desenvolvimento nativo no sistema operacional Android foi utilizada a linguagem de programação Java. A configuração de layout da interface e o sistema de marcação temporal foram implementados em XML (Extensible Markup Language XML 2014). A plataforma de desenvolvimento (IDE) e o kit de desenvolvimento (SDK) Android foram Eclipse para Windows e Android (API 19), respectivamente. A primeira geração de protótipo mixdroid foi desenvolvida quando o sistema operacional não tinha suporte para manipulação de arquivos ou para manipulação de áudio em tempo real. A análise de múltiplos estudos de caso e a coleta de informações com usuários de mixdroid 1G indicaram a necessidade de atualizar o código-base fornecendo uma nova versão que incorporasse os avanços do sistema operacional no suporte à portabilidade dos arquivos, a ampliação da documentação de desenvolvimento facilitando a atualização, e a aplicação estrita da estrutura hierárquica orientada a objetos [Keller et al. 2013b; Pinheiro et al. 2013; Pimenta et al. 2013]. Porém, levando em conta a necessidade de manter a compatibilidade retroativa, somente foram adicionadas bibliotecas dentro do perfil de requisitos da versão Android 1.6 (API 4). 21 Vitória - ES - Brazil

34 V Ubimus From Digital Arts to Ubiquitous Music V Workshop em Música Ubíqua (V UbiMus), Vitória, ES, 29 de outubro a 3 de novembro de 2014 A tabela 1 fornece um quadro das funcionalidades da segunda geração em comparação com a primeira geração de protótipos. Entre os vários módulos e funções desenvolvidos com o objetivo de aumentar a usabilidade, tem destaque a licença de leitura e gravação primária no sistema de armazenamento externo do aparelho (SD-card). Nesse dispositivo são criadas duas pastas ao iniciar sessão: (1) MixdroidSongs, onde deverão ser adicionados os arquivos que serão listados e executados pelo reprodutor de áudio; e (2) MixdroidRecords que é responsável pelo armazenamento dos arquivos de mixagen (ver figuras 4 e 5). Ambas pastas podem ser posteriormente alteradas pelo usuário acessando a tela de configurações. Tabela 1. Tabela expositiva de características da primeira e da segunda geração de mixdroid. Características mixdroid 1G mixdroid 2G Formatos de leitura wav wav, mp3, ogg, entre outros Formatos de armazenamento Banco de dados interno XML, WAV, MP3 Quantidade de arquivos manipulados Limitado pela capacidade de memória No máximo 9 por sessão simultaneamente RAM do dispositivo Versão Android/API retro compatibilidade Android 1.6/API 4 Android 1.6/API 4 Gerenciamento de diretórios de leitura não sim e/ou gravação Captura de som via microfone não sim Exportação de mixagens não sim Carregamento automático por seleção Seleção de arquivos de áudio Individual para cada música de pastas Visualização do histórico de gravação sim (em forma de animação) sim (no formato XML) Desinstalação limpa sim não Necessita de pré-instalação de Instalação Direta e sem pré-requisitos software de terceiros Uma das ferramentas de código aberto mais utilizadas atualmente no trabalho de edição e mixagem de áudio é o editor para dispositivos estacionários Audacity [Mazzoni e Dannenberg 2000]. A interface para o trabalho de mixagem adota a metáfora da fita, onde as amostras de áudio são visualizadas em trilhas, fornecendo suporte visual para as operações de posicionamento dos eventos no eixo temporal. Essa metáfora de interação é útil em dispositivos com tela ampla e boa disponibilidade de CPU, no entanto apresenta limitações em dispositivos com tela pequena ou com recursos limitados, já que a maioria das operações de áudio são acompanhadas por atualizações na representação visual dos dados sonoros. Levando em conta esse perfil, o ambiente natural de uso dos editores que adotam esse tipo de metáfora é o ambiente de estúdio. Durantes pesquisas comparativas foi encontrada uma ferramenta livre para navegadores de Internet chamada FreeSounds (2014) 1. Hospedada em site de nome semelhante, ela oferece uma grande gama de opções e recursos semelhantes ao do mixdroid 2G, porém em nível mais avançado. Devido a que o foco do FreeSounds não é a mixagem propriamente dita, acaba comprometendo o objetivo principal do projeto, que é ter uma interface simples voltada para usuários leigos que utilize poucos recursos, com suporte offline e centrada na portabilidade. Esses itens vêm sendo indicados como requisitos básicos das ferramentas musicais ubíquas [Keller et al. 2011a; Pimenta et al. 2012]. Estudos futuros poderão estabelecer quais itens são mais relevantes para o suporte à criatividade musical cotidiana, e se é viável incorporar ferramentas que dependam da conectividade à rede, como é o caso do FreeSounds. 1 Agradecemos ao revisor anônimo do V UbiMus por ter apontado as similaridades entre mixdroid e FreeSounds. Vitória - ES - Brazil 22

35 From Digital Arts to Ubiquitous Music V Ubimus V Workshop em Música Ubíqua (V UbiMus), Vitória, ES, 29 de outubro a 3 de novembro de Procedimento de geração de produtos criativos: minicomps Tendo apresentado as principais características dos ambientes de suporte, nesta seção descrevemos o método utilizado para a geração dos produtos criativos, e fornecemos dados sobres as amostras sonoras utilizadas, as ferramentas de suporte para mixagem e os resultados sonoros obtidos. Adotamos os mesmos procedimentos utilizados em estudos anteriores: as minicomps ou mini-composições [Keller et al. 2011b]. As minicomps propõem a realização de um ciclo criativo completo em uma única sessão, permitindo a quantificação do tempo de atividade criativa Amostras sonoras Os materiais sonoros usados nas mini-composições foram gravados em formato estéreo, com taxa de amostragem de 44.1 khz e resolução de 16 bits, utilizando um gravador digital portátil profissional e um microfone direcional cardioide de tipo condensador. A edição e segmentação foram realizadas no editor Audacity. Do material coletado foram selecionadas nove amostras sonoras gravadas em três ambientes diferentes, abrangendo sons urbanos, sons de animais e sons domésticos. Descrições detalhadas desse material estão disponíveis em [Keller et al. 2013b; Pinheiro et al. 2012, 2013] Tabela 2. Amostras sonoras. Amostras Formato Tamanho Tipo de amostra Carro 01 Som Wave 796 KB sons urbanos Carro 02 Som Wave 312 KB sons urbanos Carro 03 Som Wave 326 KB sons urbanos Carro 04 Som Wave 649 KB sons urbanos Cozinha 01 Som Wave KB sons domésticos Cozinha 02 Som Wave KB som domésticos Cozinha 03 Som Wave KB som domésticos Rã 01 Som Wave KB sons de animais Rã 02 Som Wave 643 KB sons de animais 3.2. Ferramentas Os sistemas utilizados foram: Audacity versão 2.3 para Windows, rodando em computador portátil, com mouse óptico e teclado QWERTY padrão; mixdroid versão 1.0 rodando em tablet nacional Coby de 7 polegadas com sistema operacional Android 2.2 (figura 2); mixdroid versão 2.0 rodando em tablet nacional Coby de 7 polegadas com sistema operacional Android 2.2 (figura 4) Procedimentos Para a aferição dos produtos criativos, um músico com experiência no uso das três ferramentas produziu utilizando as amostras listadas na seção anterior trinta mixagens de aproximadamente 3 minutos de duração. Das mixagens aprovadas pelo músico como resultados satisfatórios, foram escolhidas três mixagens correspondentes às médias do tempo de execução com cada ferramenta. 23 Vitória - ES - Brazil

36 V Ubimus From Digital Arts to Ubiquitous Music V Workshop em Música Ubíqua (V UbiMus), Vitória, ES, 29 de outubro a 3 de novembro de 2014 Figura 5. Músico criando uma minicomp no mixdroid 2G Resultados das minicomps Como é possível observar na figura 4, o tempo de produção das mixagens por um usuário profissional foi de aproximadamente uma hora e meia com Audacity, em contraste com a média de três minutos e meio com mixdroid 2G, e de seis minutos e meio com mixdroid 1G. Esses resultados são consequentes com as características dos sistemas de suporte descritas na primeira seção deste artigo. Audacity fornece suporte para atividades assíncronas e mixdroid dá suporte para atividades síncronas. Portanto o tempo de realização das mixagens é levemente superior ao tempo total do produto sonoro, de 150% a 200% no caso da ferramenta mixdroid. Já o tempo de realização com o sistema de suporte assíncrono Audacity supera em mais de 30 vezes o tempo do produto sonoro. Dada essa diferença no investimento temporal na atividade, espera-se que os resultados obtidos de forma assíncrona sejam muito superiores aos resultados da atividade síncrona. Uma forma de verificar se essa hipótese é correta envolve a aferição dos produtos criativos obtidos com cada uma das ferramentas. O objetivo do estudo descrito na terceira seção deste artigo é determinar se o investimento temporal na atividade criativa pode ser correlacionado com o perfil dos produtos obtidos. 4. Aferição dos produtos criativos Nesta seção descrevemos os procedimentos utilizados para aferir os produtos criativos obtidos através do protocolo minicomps. O objetivo do experimento é comparar os descritores vinculados ao nível de criatividade de cada produto. Os dados obtidos nesta fase da pesquisa servirão para determinar se as estratégias de suporte aplicadas no design de interação das três ferramentas adotadas têm impacto nos resultados sonoros produzidos por um usuário experiente. Vitória - ES - Brazil 24

37 From Digital Arts to Ubiquitous Music V Ubimus V Workshop em Música Ubíqua (V UbiMus), Vitória, ES, 29 de outubro a 3 de novembro de 2014 Figura 6. Gráfico comparativo do tempo de produção de mixagens em três ambientes de suporte: Audacity, mixdroid 1G e mixdroid 2G Localização das sessões de aferição dos produtos criativos Todo o processo experimental foi realizado no Instituto Federal do Acre (IFAC), Campus Rio Branco. As duas sessões experimentais aconteceram dentro das salas de aula do curso de ensino médio integrado ao técnico em informática (sessão 1, sala 104) e dos cursos técnicos de informática (sessão 2, sala 107). As salas têm aproximadamente 15 metros de largura por 20 metros de comprimento, sistema de climatização modelo Split, e carecem de tratamento acústico Perfil dos sujeitos A aferição dos produtos criativos contou com a participação de 24 sujeitos com idades entre 17 e 55 anos; escolaridade média = 11 a 10 anos; e estudo musical entre 0 e 10 anos. Todos os sujeitos tiveram alguma experiência prévia em uso de tecnologia. 90% dos sujeitos fez uso de telefone celular durante os últimos 5 anos. Vinte e três dos vinte quatro sujeitos tiveram experiências com ferramentas multimídia (como YouTube e MediaPlayer). Dois sujeitos afirmaram possuir conhecimento de linguagens de programação e de tecnologias desenvolvidas para fins musicais, incluindo o editor Audacity Procedimentos de aferição Para aferir o perfil criativo das três mixagens escolhidas, foi utilizado o protocolo Creative Product Profile ou Perfil do Produto Criativo (CrePP-NAP v.04). Ao longo de múltiplos estudos preliminares foram ajustados: o número de fatores, a escala de aferição, e o tipo de dados pessoais coletados durante a sessão. Na sua versão 0,04 o CrePP consiste em um formulário eletrônico que avalia o produto através de cinco descritores bem feito, original, expressivo, relaxante, agradável e inclui um campo para observações por parte dos sujeitos [Barbosa et al. 2010; Keller et al. 2011b]. A escala de aferição é de -2 a +2. Para fins de aplicação, as perguntas foram impressas em folhas de papel e o questionário foi apresentado a todos os sujeitos de forma simultânea. A atividade de aferição foi dividida em duas partes. Na primeira parte foi apresentado um questionário sobre a experiência do sujeito com tecnologia e sobre seus conhecimentos musicais. As mixagens foram tocadas para o grupo de alunos uma única vez por sala, na seguinte sequência: produto do Audacity, depois produto do mixdroid 1G e por último produto do mixdroid 2G. Ao fim de cada execução, os participantes 25 Vitória - ES - Brazil

38 V Ubimus From Digital Arts to Ubiquitous Music V Workshop em Música Ubíqua (V UbiMus), Vitória, ES, 29 de outubro a 3 de novembro de 2014 preencheram o formulário. As aferições realizadas totalizaram 72 para os três produtos criativos. Figura 7. Sujeitos preenchendo formulário de avaliação das mini composições feitas com Audacity, mixdroid 1G e 2G Resultados Os resultados indicam que os produtos criativos obtidos com mixdroid 1G e 2G têm um perfil similar. Não observamos diferenças maiores do que 17 centésimos numa escala de -2 a +2. Já a aferição dos produtos criativos realizados com o editor Audacity apontou diferenças nos descritores relaxante e agradável, ficando entre 1,42 e 0,96 pontos acima dos escores dados aos produtos feitos com mixdroid 1G e 2G. No entanto, a variabilidade das respostas também foi alta. Os itens originalidade e expressividade foram levemente superiores nas avaliações do produto feito com Audacity (21 e 42 centésimos respectivamente). Mas no fator qualidade (descritor: bem feito), o produto obtido com mixdroid 2G teve uma média de 25 centésimos acima da média dada à mixagem realizada com Audacity. Figura 8. Comparação entre os perfis dos três produtos. Numa análise mais apurada, o perfil do produto criativo obtido com Audacity mostra resultados similares para os fatores relevância (se é bom), originalidade e prazer. Já os fatores vinculados a relaxamento e agradabilidade ficaram abaixo dos outros escores. Vitória - ES - Brazil 26

39 From Digital Arts to Ubiquitous Music V Ubimus V Workshop em Música Ubíqua (V UbiMus), Vitória, ES, 29 de outubro a 3 de novembro de 2014 Figura 9. Perfil do produto obtido com Audacity. Em contraste, os descritores relaxante e agradável receberam escores negativos nas duas mixagens feitas com mixdroid 1G e 2G. Por outra parte, a variabilidade nas respostas para esses dois fatores foi maior em todos os casos exceto no descritor agradável para a mixagem feita com mixdroid 1G. Figura 10. Perfil do produto obtido com mixdroid 1G. É interessante observar que o fator relevância teve o melhor resultado para o produto feito com mixdroid 2G, seguido pelo escore dado à mixagem feita com mixdroid 1G. No entanto, essa tendência não foi acompanhada pelo escore dado ao fator originalidade. Os resultados para os produtos feitos com mixdroid 1G e 2G foram menores para os fatores originalidade e expressividade. Observando os dados em conjunto, as mixagens feitas com mixdroid 1G e 2G apresentam praticamente o mesmo perfil. Os fatores seguem a mesma ordem de maior a menor: bom, original, expressivo, agradável, relaxante. Essa tendência contrasta com a variação pequena entre as médias do perfil do produto feito com Audacity. Figura 11. Perfil do produto obtido com mixdroid 2G. 5. Discussão dos resultados e considerações finais Os resultados obtidos confirmam parte das hipóteses formuladas a partir de observações em estudos preliminares. (1) A aferição de produtos criativos fornece resultados 27 Vitória - ES - Brazil

40 V Ubimus From Digital Arts to Ubiquitous Music V Workshop em Música Ubíqua (V UbiMus), Vitória, ES, 29 de outubro a 3 de novembro de 2014 consistentes para as duas ferramentas de suporte a atividades síncronas e gera um perfil contrastante para a ferramenta de suporte a atividades assíncronas. Esse resultado indica que a metodologia proposta é viável. (2) Apesar do alto investimento temporal exigido pela ferramenta de mixagem assíncrona, a diferença no perfil de criatividade dos produtos obtidos com as ferramentas síncronas não indicou quedas generalizadas nos escores. Surpreendentemente, os resultados no fator relevância foram inversamente proporcionais ao tempo investido na mixagem. Porém essa tendência não foi acompanhada pelo escores dados à originalidade. Portanto podemos concluir que o suporte síncrono favorece a relevância do produto mas não tem o mesmo impacto na originalidade. (3) Se todos os fatores de criatividade tivessem mostrado aumentos para os produtos feitos com ferramentas síncronas, poderíamos concluir que esse tipo de suporte não só é mais eficiente mas ele também fomenta a criatividade. Os resultados não são uniformes para todos os fatores. Houve aumento na relevância dos produtos, e queda nos descritores agradável e relaxante. Já as diferenças nos escores de originalidade e expressividade foram relativamente pequenas. Entre as limitações do estudo, apontamos a possibilidade de que a ordem de aplicação do CrePP-NAP tenha tido impacto nas aferições. Em experimentos futuros estabeleceremos uma matriz de aplicação que elimine o possível efeito da ordem de apresentação das minicomps. Outra limitação é mudança entre o ambiente de realização da atividade criativa e o ambiente de aferição do produto criativo. Qualificando os resultados como preliminares, podemos afirmar que o suporte síncrono fomenta a geração de produtos com perfil diferente do suporte assíncrono porém não necessariamente mais ou menos criativo. Portanto concluímos que a metáfora de marcação temporal aumenta significativamente a eficiência da atividade criativa e tem impacto no perfil dos produtos criativos. Esse perfil é diferente dos produtos gerados a partir do suporte assíncrono porém não é necessariamente mais ou menos criativo. A utilização do método de aferição proposto neste trabalho permite a comparação do impacto de diversas estratégias de suporte, ampliando o leque de técnicas disponíveis para o design de interação centrado em criatividade. Referências FreeSound, Disponível em: < Acesso em: 26 de agosto de 2014 às 20:30 hs. Keller, D., Barreiro, D. L., Queiroz, M. & Pimenta, M. S. (2010). Anchoring in ubiquitous musical activities. In Proceedings of the International Computer Music Conference (pp ). Ann Arbor, MI: MPublishing, University of Michigan Library. Keller, D., Ferreira da Silva, E., Pinheiro da Silva, F., Lima, M. H., Pimenta, M. S. & Lazzarini, V. (2013). Everyday musical creativity: An exploratory study with vocal percussion (Criatividade musicalccotidiana: Um estudo exploratório com sons vocais percussivos). In Anais do Congresso da Associação Nacional de Pesquisa e Pós-Graduação em Música - ANPPOM. Natal, RN: ANPPOM. Vitória - ES - Brazil 28

41 From Digital Arts to Ubiquitous Music V Ubimus V Workshop em Música Ubíqua (V UbiMus), Vitória, ES, 29 de outubro a 3 de novembro de 2014 Keller, D., Flores, L. V., Pimenta, M. S., Capasso, A. & Tinajero, P. (2011). Convergent Trends Toward Ubiquitous Music. Journal of New Music Research 40 (3), (Doi: / ) Keller, D., Lima, M. H., Pimenta, M. S. & Queiroz, M. (2011). Assessing musical creativity: material, procedural and contextual dimensions. In Anais do Congresso da Associação Nacional de Pesquisa e Pós-Graduação em Música - ANPPOM (pp ). Uberlândia, MG: ANPPOM. Keller, D., Pimenta, M. S. & Lazzarini, V. (2013). Os Ingredientes da Criatividade em Música Ubíqua. In D. Keller, D. Quaranta & R. Sigal (eds.), Sonic Ideas, Vol. Criatividade Musical / Creatividad Musical. México, DF: CMMAS. Keller, D., Pinheiro da Silva, F., Ferreira da Silva, E., Lazzarini, V. & Pimenta, M. S. (2013). Design oportunista de sistemas musicais ubíquos: O impacto do fator de ancoragem no suporte à criatividade. In E. Ferneda, G. Cabral & D. Keller (eds.), Proceedings of the XIV Brazilian Symposium on Computer Music (SBCM 2013). Brasília, DF: SBC. Keller, D., Pinheiro da Silva, F., Giorni, B., Pimenta, M. S. & Queiroz, M. (2011). Marcação espacial: estudo exploratório. In Proceedings of the 13th Brazilian Symposium on Computer Music. Vitória, ES: SBC. Mazzoni, D. & Dannenberg, R. (2000). Audacity [Editor de Áudio]. Pittsburgh, PA: Carnegie Mellon University. Pimenta, M. S., Flores, L. V., Radanovitsck, E. A. A., Keller, D. & Lazzarini, V. (2013). Aplicando a Metáfora de Marcação Temporal para Atividades Criativas com mixdroid. In D. Keller, D. Quaranta & R. Sigal (eds.), Sonic Ideas, Vol. Criatividade Musical / Creatividad Musical. México, DF: CMMAS. Pimenta, M. S., Miletto, E. M., Keller, D. & Flores, L. V. (2012). Technological support for online communities focusing on music creation: Adopting collaboration, flexibility and multiculturality from Brazilian creativity styles. In N. A. Azab (ed.), Cases on Web 2.0 in Developing Countries: Studies on Implementation, Application and Use. Vancouver, BC: IGI Global Press. (ISBN: ) Pinheiro da Silva, F., Keller, D., Ferreira da Silva, E., Pimenta, M. S. & Lazzarini, V. (2013). Criatividade musical cotidiana: estudo exploratório de atividades musicais ubíquas. Música Hodie 13, Pinheiro da Silva, F., Pimenta, M. S., Lazzarini, V. & Keller, D. (2012). A marcação temporal no seu nicho: Engajamento, explorabilidade e atenção criativa. In Proceedings of the III Ubiquitous Music Workshop (III UbiMus). São Paulo, SP: Ubiquitous Music Group. Radanovitsck, E. A. A., Keller, D., Flores, L. V., Pimenta, M. S. & Queiroz, M. (2011). mixdroid: Marcação temporal para atividades criativas. In Proceedings of the XIII 29 Vitória - ES - Brazil

42 V Ubimus From Digital Arts to Ubiquitous Music Making meaningful musical experiences accessible using the ipad Andrew R. Brown 1, Donald Stewart 2, Amber Hansen 1, Alanna Stewart 2 1 Queensland Conservatorium, Griffith University, Brisbane, Austrália 2 School of Medicine, Griffith University, Brisbane, Australia {andrew.r.brown, donald.stewart, a.hansen, Abstract. In this paper we report on our experiences using ubiquitous computing devices to introduce music-based creative activities into an Australian school. The use of music applications on mobile tablet computers (ipads) made these activities accessible to students with a limited range of prior musical background and in a general purpose classroom setting. The activities were designed to be meaningful and contribute toward personal resilience in the students. We describe the approach to meeting these objectives and discuss results of the project. The paper includes an overview of the ongoing project including its aims, objectives and utilisation of mobile technologies and software with generative and networkable capabilities. Two theoretical frameworks inform the research design; the meaningful engagement matrix and personal resilience. We describe these frameworks and how they inform the activity planning. We report on the activities undertaken to date and share results from questionnaires, interviews, musical outcomes, and observation. 1. Introduction This project builds on the authors previous work with network music jamming systems (Brown and Dillon 2007) and youth resilience (Stewart et al. 2004, Stewart 2014). These research threads have come together in this project. Taking advantage of the ubiquitous nature of mobile computing devices (in particular of Apple s ipad), the project aims to provide school students who have no particular background in music, with access to the creative and well-being benefits of collaborative and personally expressive music making. This project takes a step forward from our previous network jamming research by using Apple s GarageBand software on the ipad rather than the our own jam2jam software on laptop and desktop computers. jam2jam was specifically written for our previous research on how technologies afford meaningful engagement with music. It was used in this capacity between 2002 and The main software features of jam2jam that support accessibility and engagement are 1) the use of generative music processes to enable participation by inexperienced musicians, 2) the ability for systems to be synchronized over a network facilitating coordination amongst users, either locally or at a distance, and 3) the ability to record music making activities Vitória - ES - Brazil 30

43 From Digital Arts to Ubiquitous Music V Ubimus and export these for sharing. These features are now present in GarageBand for ipad (and an increasing number of other commercial software and hardware combinations). In our previous examination of developing resilience in school contexts, positive contributing factors included students developing a sense of autonomy and feelings of connectedness with peers and adults. We suggest that the scaffolding effect of generative music process can assist in promoting a sense of creative autonomy in inexperienced musicians and that the collaborative aspects of group music making can strengthen feelings of connectedness amongst peers. An aim of this project is to show how the principles of education and health-promotion developed in our previous research can transfer to the use of ubiquitous computing systems. 1.1 Brief description of the project This project focuses on building and supporting young people s engagement and connectedness with their creative selves and to help build resilience through musical collaboration and success. Working with a school of Indigenous Australian students (the Murri school) based in Brisbane, Australia we have provided opportunities for musical expression using music technology through the school curriculum. The project engages Indigenous Australian students using a digital audio production system that allows them to use their personal, social and cultural identities to make meaningful creative endeavors. The project trials newly emerging technology using ipads with GarageBand software to explore the development of self-confidence and self-esteem. The approach involves the trialing of weekly music-based activities is several classes over two terms (20 weeks). The activities are designed to offer opportunities for students to achieve creative educational goals, to engage them in expressive musicmaking, to develop self-esteem and to develop creative collaborations with peers. The project aims to provide evidence of a positive model for engaging school students in an interactive music-based education program and for building confidence and resilience. Objectives of the project include, to: Trial and evaluate new generative music technology to explore improvements in engagement and connectedness between students and the education system. Build resilience and raise the levels of educational achievement and aspirations of Indigenous students. Identify positive models of music education and health promotion. Use music technologies to build a sense of belonging and connectedness within the school environment that is protective of mental and emotional wellbeing. 2. Accessibility via mobile technologies A catalyst for this project is the availability of appropriate computing software and hardware for music making. Apple s ipad and GarageBand software have features that make the activities of this project much more accessible than they have previously been. The ipad s small size and battery life make it easy for students to handle and easy for schools to accommodate. The GarageBand software utilizes smart instruments and Apple loops that simplify music production. The smart instruments provide a constrained performance environment that minimizes mistakes and can be used in 31 Vitória - ES - Brazil

44 V Ubimus From Digital Arts to Ubiquitous Music music education in a similar way that restricted acoustic instruments (such as small xylophones) have been in the past. The music clips (Apple loops) allow for a constructor-set approach to music making where students can combine these building blocks without needing (yet) the facility to make the clips from scratch. The ipads and GarageBand combination support collaboration by allowing students, each with an ipad, to synchronize their music making over a local network. This activity, which we have previously called network jamming, facilitates groups of students to perform together. Finally, the ability of the software to record the music they compose and export files for later review and distribution, means that student s work can be available for reflection and/or sharing with the wider community. 3. Meaningful Engagement The theory of meaningful engagement was developed by Andrew R. Brown and Steve Dillon (2012) and has underscored the development of network jamming research more broadly. It involves two dimensions. Musical engagement includes various creative behaviors, or ways of being involved in music. The modes of engagement outlined in the theory cover a range of interactions from listening and appreciating, to creating, performing and leading. The theory suggests that meaning can arise from engagements with music in three contexts; personal, social and cultural. That is, music can be personally satisfying, it can lead to positive social relationships, and it can provide a sense of cultural or community identity. Below is a summary of the modes of engagement and context for meaning. Modes of Creative Engagement Appreciating paying careful attention to creative works and their representations Evaluating judging aesthetic value and cultural appropriateness Directing leading creative making activities Exploring searching through artistic possibilities Embodying being engrossed in fluent creative expression Contexts of Creative Meaning Personal intrinsically enjoying the activity Social developing relationships with others Cultural feeling that actions are valued by the community The two aspects of meaningful engagement can be depicted as the axes of a matrix, as shown in figure 1. Vitória - ES - Brazil 32

45 From Digital Arts to Ubiquitous Music V Ubimus Figure 1. The Meaningful Engagement Matrix with exemplar musical activities The meaningful engagement matrix (MEM) is a framework for describing creative experiences and evaluating creative resources, plans or activities. This can be, for example, assessing a community or educational workshop, reviewing the comprehensiveness of an arts curriculum or lesson plan, evaluating the affordances of a software application for creating media content. While this matrix was developed for musical activities it can be applied to other pursuits, especially in the Arts. Artistic experiences become meaningful when they resonate with us and are satisfying. The meaningful engagement matrix has been developed to assist inquiry into our creative activities and relationships. A full creative life, the theory suggests, involves experiences across all cells of the matrix. Therefore, this framework can be useful when auditing the range of experiences afforded by any particular activity, program or resource, or across a set/series of these. It is in the assessment of the wholeof-program view of this project that the MEM provides its greatest utility. 4. Resilience Resiliency refers to the capacities within a person that promote positive outcomes such as mental health and well-being, and provide protection from factors that might otherwise place that person at increased development, social and/or health risk (Rowe & Stewart, 2009; Fraser, 1997). Factors that contribute to resilience include personal coping skills and strategies for dealing with adversity such as problem-solving, cognitive and emotional skills, communication skills and help-seeking behaviors (Fraser, 1997). This project builds on previous work that indicates that creative activities can improve resilience. There is an abundance of research that highlights the importance of the social environment, or social relationships for fostering resilience (Maggi et al., 2005; Rowe & Stewart, 2009; Lee & Stewart 2013). Social cohesion or connectedness refers to broader features of communities and populations and is characterized by strong social bonds with high levels of interpersonal trust and norms of reciprocity, otherwise known as social capital (Siddiqui, et al., 2007). This network of rich social relationships and strong connections promote a sense of belonging and community connectedness which, in turn, impacts on an individual s mental health and overall well-being (AIHW, 2009). Social capital, spirituality, family support and a strong sense of cultural identity are key protective factors for Indigenous people (and children) (Malin, 2003). 33 Vitória - ES - Brazil

46 V Ubimus From Digital Arts to Ubiquitous Music Schools that aim to strengthen their capacity as healthy settings for living, learning, working and playing, and are underpinned by inclusive participatory approaches to decision-making and action, can help to build resilience (Rowe & Stewart, 2009). Connectedness in the school setting has been shown to be a protective factor of adolescent health risk behaviors related to emotional health, violence, substance use and sexuality. Creative activities, especially collaborative ones such as music making, share many of the characteristics that have been shown to promote resilience. This project seeks to take advantage of these connections. 5. Collaboration and Sustainability With relevant support from the Murri school community, the project offered the opportunity to develop a creative and sustainable program for young people, in this case young Indigenous Australians, to engage in collaborative music making activities using interactive music technologies. The reason that music technology is appropriate for the project was because of its familiarity to young people and also because of our expertise in the use of generative systems in collaborative music making. A number of creative projects use music jamming as a means of improving creativity, social justice and wellbeing, hence there are many collaborations with communities that are sometimes marginalized from mainstream society (Adkins et al. 2012). The GarageBand software for the ipad supports collaborative audio production through local synchronization via Bluetooth and through file and audio material export and import. When used as a musical instrument and compositional platform this software enables students to build on basic skills of exploration and improvisation and encourages engagement. These technologies are also easy for staff to learn and use and this, it is hoped will increase the likelihood that the network jamming activities will continue in the school beyond the life of this project. A number of strategies were used to facilitate the sustainability of the activities. These include: Involvement of school administration and teaching staff in the planning and execution of the activities. Integration of the music activities into the broader curriculum. Sharing of the musical outcomes amongst the school community. Regular reporting on progress with the school administration. Provision to leave the equipment used for the project with the school. 6. Case Study - ipads and Music at the Murri School The goal of the project was to examine how music technology can work to improve Indigenous health and wellbeing by creating a sustainable program for Indigenous youth to engage in collaborative music making activities using interactive music technology. Vitória - ES - Brazil 34

47 From Digital Arts to Ubiquitous Music V Ubimus Figure 2. Images from the project school The project integrated music activities using the ipad into the normal school curriculum and involved relevant teachers. It used standard classroom procedures and resources but the project provided a facilitator proficient in the technologies and familiar with theories and objectives of the project. The project involved a weekly session with each class facilitated by a member of the project team and the class teacher. Prior to commencing, approval was gained from Griffith University s Human Research Ethics Committee to conduct the research and teachers and students were provided with information about the project and teachers were consulted about how the music-based activities might integrate with existing curriculum objectives. Many teachers chose to incorporate creative writing tasks as the basis for song writing and rapping. The project used a whole-school approach and classes were chosen from across the full age range of the school for participation. Students and teachers were not screened for musical background nor on any measure of resilience as we were keen to investigate the versatility and flexibility of this approach across the school community. After consultation with staff, three grade levels were selected to participate in the project. The year levels and project summaries for these classes are summarized in the following table. Year Level Approx. Age Activity Objective 2/3 7/8 Students to write and record a short 4-line rap about the good qualities they see in themselves 4 9 Students to record a creative interpretation of their sonic personal profiles utilizing sounds and music to express their personalities Students to write and record a sonic poem using text and music describing themselves and their hopes, expectations and dreams. Table 1. Participating Groups and Activities 6.1 Designing music-based activities Prior to facilitating the intervention with the students at the Murri school, a series of generic activities were designed in order to facilitate creative participation in a way that 35 Vito ria - ES - Brazil

48 V Ubimus From Digital Arts to Ubiquitous Music adheres to the philosophy interwoven in the aforementioned MEM framework. The key objectives of the music-based activities designed for this project were to: 1) enable the students to engage in diverse music making opportunities that utilize music technology in a meaningful capacity; 2) to enable participants the opportunity to engage in creative experiences that assist in positively strengthening their sense of well being and resilience. The activities designed for each year level were collaboratively developed by the researchers and participating class teachers, keeping the MEM in mind throughout this process. Each teacher chose to utilize an age/ability appropriate literacy basis for their class project in order to facilitate the opportunity for students to individually and collectively express themselves and their interests in a personal and creative manner. The objective for Term 1 was to enable students of each participating group to develop and record their own composition using GarageBand on the ipads. The timeline below outlines the context of each weekly session dedicated to the project, allowing for students of each group to spend time experimenting, jamming, practicing playing and recording instruments and external audio, and for recording the final product. The objective for Term 2 was for students to develop and refine their work into a form ready for a signature event a public performance at the school assembly. Table 2 lists the mode and context of activities designed to achieve the key objectives of this project. Each cell corresponds to specific mode and context combination within the MEM. ATTEND Listening / Observing EVALUATE Reflecting / Analyzing EXPLORE Experimenting / Improvising Conscious DIRECT Decision Making / Instructing EMBODY Playing / Performing Establishing habits PERSONAL (Of the self) Objective Independently listen, read and observe in order to become aware of relevant knowledge Independently reflect and analyze personal practice as a means of facilitating continued learning. Independently explore and experiment with relevant artefacts and processes. Engage in technical activities that lead to creating a musical artefact. Independent practice / playing Activity Introduction to Network Jamming Demonstration of available interactive music hardware and software. Record/journal learning and practical experiences. Music analysis to enable the development of aural skills. Independently explore and experiment with sounds and functions of Network Jamming devices. Building knowledge. Setting up a Jam session. Composing a song. Guided and Independent Play / Practice of Network Jamming devises and processes to building skills. SOCIAL (Collaborative) Objective Share work and progress with peers. Reflect upon learning and practical experiences with peers as part of group discussions. Extend learning through collaborative experimentation. Take on a leadership role within a group activity. Rehearse and record with a group. Vitória - ES - Brazil 36

49 From Digital Arts to Ubiquitous Music V Ubimus Activity Workshop presentations of individuals and collaborative engagement and progress with Network Jamming. Group Discussion. Engage in a group (Networked) Jam session. Lead and Conduct a Jam Session. Group Composition. Time to play/practice with Network Jamming devices and processes collaboratively. CULTURAL (Connection with external) Objective Observe relevant activity as performed in a public context. Extend and connect reflective practice to include a wider cultural participation and dialogue. Examine / research relevant practice in a wider cultural context. Support and promote a musical artefact for public distribution. Participate in a group public performance. Activity Attending / Observing a performance that utilizes Network Jamming as a key composition / performance process. Develop a creative project for public presentation. Create a Blog /website as a reference for music work. Investigating Network Jamming in diverse cultural contexts. Explore other commercial music apps. Create and Promote a CD/DVD showcasing creative progress. Perform a group Jam or composition to an audience. Table 2. Music-based activities across the Meaningful Engagement Matrix. 6.2 Measuring resilience and engagement Evaluation of this project relied on a mixed methods research design combining quantitative and qualitative methods of data collection, analysis and inference in order to investigate both the processes developed through the life of the project as well as the impact of the project over time. Students were asked to complete a modified version of a pre-existing resilience questionnaire that has high levels of reliability and validity (e.g., Healthy Kids Survey - California Dept of Education, 2004). Key informant interviews with staff were conducted and subject to an ongoing thematic analysis. An introductory school consultation session was attended by 9 staff members at the outset of the project. All were supportive and identified ways that they could integrate the project into their curriculum. Due to timetabling constraints only three of these staff and their classes are participating in the project. Thirty four students participated in the project across three grade levels: Years 2/3 (14 students); Year 4 (12 students); and Year 8 (8 students in the English stream). Activities included developing a Rap, recording a personal sonic profile and writing and recording a bio-poem. Observations of class sessions were recorded in a journal by a member of the research team. In addition, files of work completed on the ipad were regularly saved allowing for analysis of the steps taken during the creative process. 37 Vitória - ES - Brazil

50 V Ubimus From Digital Arts to Ubiquitous Music 7. Survey results summary The first stage of data collection provided a baseline and descriptive statistics show some differences emerging between the younger students in Grade 2/3 and 4 and their fellow students in Grade 8. We have not completed tests of statistical significance as the sample is small. We provide, below, a selection of the results and findings. This summary begins with some data from the first resilience survey to give a sense of the student s attitudes and expectations from the project. Over 75% of the total student sample thought that being involved in the project would be fun and most (younger students) were excited at the prospect. The creative levels and aspirations of the students were uniformly high and almost all indicated that they enjoyed going to music performances. However, compared to the grade 2/3 and 4 students who relished the creative opportunities of the project, a substantially lower percentage of the Grade 8 sample felt confident and supportive of the activity and their creative role. With regard to their confidence with and support structure for creative activities: Over 85% of all students like making things that are creative and different. Students felt variously confident with their own creative ability and ideas. (71% of Grade 2/3, over 90% of Grade 4 students, Grade 8 = 63%). Most students have family/elders that they can go to for help (Grade 2/3=79%, Grade 4 = 90%, Grade 8 = 75%) The students attitudes toward peer collaboration varied between the younger and older students. The following data reflect these attitudes to working with classmates: Students like to share their creative ideas with their classmates (Grade 2/3 = 78%, Grade 4 = 90%, Grade 8 = 37%). Students enjoy hearing about their classmates creative ideas (Grade 2/3 82%, Grade 4 = %85, Grade 8 = 63%). Students thought that being a part of the project would help them have more friends (Grade 2/3 = 75%, Grade 4 = 75%, Grade 8 = 12%). As with attitudes to collaboration, the students sense of self-confidence in public music making also reduced with age. In relation to producing a performance or recorded outcome: Students thought that they could put together a performance or recording that would be enjoyed by others (Grade 2/3 = 86%, Grade 4 = 66%, Grade 8 = 12%). Students felt that people would come to watch their performance or record launch (Grade 2/3 = 90%, Grade 4 = 75%, Grade 8 = 25%). A clear trend in this data is the difference in reported self-confidence, in music at least, between the younger (7-9 year old) and older (13 year old) students. This is consistent with much more extensive research that shows a dip is self-confidence in adolescents (Orenstein 1994). As a result of this, and supported by informal feedback from the grade 8 teacher, we adopted a different strategy for the older group. Activities for this class focused more on personal meaning than on social or cultural meaning, and we tried to Vitória - ES - Brazil 38

51 From Digital Arts to Ubiquitous Music V Ubimus minimize potentially embarrassing public presentations of the music. As well, work for older students has a greater individual focus whereas activities for younger students are heavily biased toward group work and include class and public presentation of outcomes in the form of recorded media and live performance. What is interesting to note, is that the accessibility features of the music technologies employed are equally applicable for both groups and approaches. A comparison of results from participants in both baseline and follow up surveys shows that in relation to project participation 75% of Grade 4; 72.7% of Grade 2 and 66.7% of Grade 8 thought that being involved in the project was fun all, or most of the time. At the same time, however, participating in the project was also considered stressful by some at least, with 27.3% of Grade 2/3 feeling worried about taking part in the project all or most of the time (Grade 4=75%; Grade 8=33.3%). 8. Qualitative results summary Qualitative data collected included interviews conducted with teachers and notes maintained by research team members. 8.1 Pre-intervention results Staff members recorded their initial plans for implementing the project within their classrooms for Term 1 and Term 2, Eight out of the nine staff members participated in this component of the staff session. Participant responses to what they hoped to achieve by being involved in the project include: For the students to record stories created for English unit. The story can be edited and compiled onto a CD. Hopefully children will gain confidence in speaking and sharing their stories/ideas. I would like to see students engage with ipad technology to enhance and extend learning already happening in subjects. Improve teacher and student confidence and participation with technology; having children work together cooperatively; tap into children s different learning styles i.e. rap songs to learn spellings; student enjoyment. To use the jamming as a learning/teaching tool in classroom to integrate curriculum to make learning fun. To learn myself and get children involved in expressing themselves orally and musically. To record for a performance, to make learning fun and for students to use an ipad. Enhancement of student work (oral and written) familiarity with technology. Increase ipad literacy, learn with students how to use this tool for work. The research team utilized the Meaningful Engagement Matrix to record the frequency and intensity of meaningful engagements they observe in students participating in the project. Video footage and photography were also being used to provide further documentation of project implementation activities, and for review and analysis. 39 Vitória - ES - Brazil

52 V Ubimus From Digital Arts to Ubiquitous Music 8.2 Post-intervention results Classroom management In terms of general process, the participating classroom teachers had differing opinions regarding how manageable it is to have a class of students work with the ipads for engaging in learning and collaborative work. Two of the teachers felt that this was a manageable task, whereas one of the teachers (Grade 4) felt that this process of learning would work best in smaller groups as children may have difficulty listening to instructions and paying attention in a larger group. Some of the challenges in participating in the project include student s inability to share ipads they preferred to work on their own. Another challenge lay in having a consistent and clear idea of the long-term goal and clarifying goals for students to be achieved at the end of each session. One teacher felt that at times it seemed that the students were all over the place. This was due to the students showcasing their ability to jam on the ipads using different musical instruments available on GarageBand. Jamming with colleagues allows for creative expression that relies on self-expression. The grade 4 teacher felt that not being present regularly, and not understanding how to use the ipads and remembering it were challenges. Also, keeping all the students on task when the whole class was involved was a challenge. She felt that keeping the ipad project in a small group environment might assist in overcoming some of these challenges. However, in terms of how satisfied the teachers were with the way the project had been implemented in their class, there was general consensus that they felt that the project went well and that the students looked forward to the sessions on the ipad. Student engagement The Grade 2 teacher was really impressed that his more challenging students, who rarely engaged in classroom activities, were able to participate confidently in the project. Those that had difficulty with directing their attention to one specific task for a period of time were able to participate in the ipad sessions for the course of the weekly schedule. The Grade 8 teacher felt that he under-estimated the students reluctance to share their work. He felt that his lack of knowledge of technology/ipads required increased reliance on the research assistants. He acknowledged that the students had a product at the end of the project, but considered that the ipads could have been better used. Teacher engagement The Grade 4 teacher felt that there were components of the program that she liked and some parts of the program she did not find helpful in making the project run smoothly. Teacher ownership is a critical success factor for the sustainability of the project. She felt that because she wasn t there most of the time for the weekly ipad sessions, she found it difficult to gauge how effective the implementation was going. She indicated that there were times when it was confusing what the object of the lesson was. This Vitória - ES - Brazil 40

53 From Digital Arts to Ubiquitous Music V Ubimus reinforced the importance of working with the teachers to develop an action plan for their students and take a leadership role in achieving their goals and objectives. The participating teachers relied on the research assistants to set weekly plans for the students, offering limited guidance and support. Behaviour management was a challenge each week for the research assistants. Often teacher aides were the only other adults present to provide additional supervision for the children and at times sessions were taken up with disciplining students. The signature event The grade 4 teacher stated that he enjoyed watching those children who performed on assembly and that the end of project performance sounded good. He stated that some of the students are normally really shy and would never get up on their own. But, because they were in a group and focusing on the ipad they coped. All teachers stated that they were happy with what their class had achieved by participating in the project. The Grade 8 teacher stated that hopefully they will have greater confidence to use technology in relation to English. 9. Findings All teachers considered that their involvement in the project has made a difference to the way they have looked at teaching. The grade 4 teacher stated that it gave her another avenue through which to teach. Technology is the focus of our learning now, she said. The grade 8 teacher stated that it has highlighted a need to use technology in the class. Students have access to it outside of school they use it all the time it is a tool he feels he needs to tap into for learning. All participating teachers have plans to continue to use this form of learning for future teaching. The teachers felt it is beneficial to have a structure around using the ipads. To start with structure was thought to be important i.e., weekly plan/within a subject such as English. The grade 8 teacher felt that freedom to be creative can flow on from this. All teachers felt that the project had had a positive impact on the students. The Grade 8 teacher stated that the students looked forward to Friday sessions. He stated that although they were shy, he believes that they were secretly proud of what they did. The Grade 4 teacher said that they loved it and looked forward to it. She also said that she could use the ipads as a reward for good behavior. All teachers stated that they would recommend using the ipads as an approach to learning to other teachers. The Grade 2 and 8 teachers felt confident in sharing this approach to learning with colleagues. All teachers felt that they would have liked more professional development on using the ipads Lessons learnt This project aimed to examine how music technology can work to improve student health and wellbeing. The project aimed to offer the opportunity to develop a creative and sustainable program for young Indigenous Australians to engage in collaborative music making activities using interactive music technologies. The following lessons have been learned from this pilot project: 41 Vitória - ES - Brazil

54 V Ubimus From Digital Arts to Ubiquitous Music An in-class project of this nature requires relevant support from the whole Murri school (Indigenous) community. A planned period of in-service training and support with the teachers would help to ensure that the project is introduced with confidence and becomes sustainable beyond the life of the project. Small group work with all students accessing the technology would ensure better student engagement. A clear link between curriculum frameworks and the use of ipad technology would help to engage the project within the School s learning framework. Indigenous students enjoy and engage with advanced technology within the classroom and develop meaningful, creative compositions. The Meaningful Engagement Matrix provides a strong theoretical framework for a School-based creative project. Additional research is needed to confirm the reliability and validity of the questionnaire with consideration given to a range of instrument structures to allow for widely varying age/developmental conditions. This project provides a constructive and stimulating experience for many young people who find group work difficult and have communication difficulties. Public performance of creative, music-based projects provide important opportunities to enhance self-esteem and promote creative partnerships. 10. Conclusion In this paper we have described our use of mobile technologies and software to make music-based activities accessible to young people in a way that promotes meaningful engagement and resilience. The project is based in the Murri school in Brisbane, Australia that is dedicated to the education of Indigenous Australians. The project involved weekly activities with three classes from that school over 20 weeks with students ranging from ages The design of project activities was informed by theories of meaningful engagement and resilience, but were guided by the advice of class teachers and student survey responses to ensure appropriateness to the local context. Data indicate that staff and students are enthusiastic about using the tablet computers and music apps, and that their ease of use is making previously unimagined music production activities accessible. Consistent with other studies, our data show a dip in the creative self confidence of students in their early teens (compared to younger students). This has been accommodated for by shifting the emphasis for those students toward individual and personal expression and away from collaborative and public activities. The portability of the ipad hardware has assisted with the integration of the devices into the school environment, and their multi-purpose nature makes for fluid shifts between music and other curricular tasks (such as creative writing). The GarageBand software has facilitated rich music production outcomes, although the Vitória - ES - Brazil 42

55 From Digital Arts to Ubiquitous Music V Ubimus devices alone provided limited audio recording and playback quality. We plan to address this in the next stage of the project through more extensive use of external microphones and headphones. Indications are that the students can be keenly engaged in network jamming activities but require an ongoing facilitator support to maximize creative outcomes. The features of the music-based activities with ubiquitous technologies align well with characteristics that promote resilience, including personal autonomy and connectedness with peers and adults, and we remain optimistic that evidence of a positive effect on student resilience from the project can be achieved. References Adkins, B., Bartleet, B.-L., Brown, A. R., Foster, A., Hirche, K., Procopis, B., Ruthmann, A., & Sunderland, N. (2012) Music as a tool for social transformation: A dedication to the life and work of Steve Dillon (20 March April 2012). International Journal of Community Music, 5(2), AIHW (Australian Institute of Health and Welfare). (2009) A Picture of Australia s Children: Health and Wellbeing of Indigenous Children. Canberra: AIHW. Brown, A. R., & Dillon, S. (2007) Networked Improvisational Musical Environments: Learning through online collaborative music making. In: J. Finney & P. Burnard (Eds.), Music Education with Digital Technology, pp London: Continuum. Brown, A. R., & Dillon, S. (2012) Meaningful Engagement with music composition. In: D. Collins (Ed.), The Act of Musical Composition: Studies in the creative process, pp Surrey, UK: Ashgate. Fraser, M.W. (1997) Risk and Resilience in Childhood. USA: NASW Press. Lee, P. C, Stewart, D., (2013) Does a socio-ecological school model promote resilience in primary schools? Journal of School Health. 83: Maggi, S., Irwin, L. G., Siddiqi, A., Poureslami, I., Hertzman, E., & Hertzman, C. (2005) Knowledge Network for Early Child Development. British Columbia: World Health Organisation. Malin, M. (2003) Is Schooling Good for Aboriginal Children s Health? Northern Territory University: The Cooperative Research Centre for Aboriginal and Tropical Health. Orenstein, P. (1994) Schoolgirls: Young women, self esteem, and the confidence gap. Anchor Press. Rowe, F., & Stewart, D. (2009) Promoting Connectedness through Whole-School Approaches: A Qualitative Study. Health Education, 109: 5, Siddiqi, A., Irwin, L. G., & Hertzman, C. (2007) Total Environment Assessment Model for Early Child Development. Retrieved Vitória - ES - Brazil

56 V Ubimus From Digital Arts to Ubiquitous Music Stewart, D., Sun, J., Patterson, C., Lemerle, K., & Hardie, M. (2004) Promoting and building resilience in primary school communities: evidence from a comprehensive health promoting school approach. International Journal of Mental Health Promotion, 6(3), Stewart, D. (2014) Resilience: an entry point for African health promoting schools? Health Education, 114: 3, Vitória - ES - Brazil 44

57 From Digital Arts to Ubiquitous Music V Ubimus Progressive Disclosure category: artistic demonstrations Progressive Disclosure is a short piece where in an imaginary landscape an unknown machine is progressively disclosed and explained in order to revel its inner functions. The piece is an reflection on concepts of approach modalities and comprehension of properties or qualities, that an object possesses and its functions. Long-slow sound objects and impulsive sounds build up the piece. These elements are merged and extensively overlapped in order to develop an imaginary panorama with basic elements of a music vocabulary. Synthesized and acoustically derived sounds are both used, but the focus here is mainly on the description of a progressively closer observation of a visionary machine. Dropbox link to the audio file (both HQ wav and mp3): 45 Vitória - ES - Brazil

58 V Ubimus From Digital Arts to Ubiquitous Music The Beathealth Project: Synchronising Movement and Music Joseph Timoney 1, Tomas Ward 2, Rudi Villing 2, Victor Lazzarini 3, Eoghan Conway 2, and Dawid Czesak 2 Departments of 1 Computer Science, 2 Electronic Engineering, and 3 Music NUI Maynooth, Maynooth Co. Kildare, Ireland. jtimoney@cs.nuim.ie, victor.lazzarini@nuim.ie Abstract. This paper will describe the new EU Beathealth project 1 : an initiative to create an intelligent technical architecture capable of delivering embodied, flexible, and efficient rhythmical stimulation adapted to individuals motor performance and skills for the purpose of enhancing/recovering movement activity. Additionally, it will explain how it can exemplify the principles of Ubiquitious Music and how knowledge from this field can suggest creativity-driven social enhancements. 1. Introduction In recent times scientists have begun to seriously investigate how using rhythm and music can be harnessed as a drug-free way of simulating health (Pollack, 2014). Music works on our autonomic nervous system, thus stimulating our sensations of wellbeing at a subconscious level (Ellis and Thayer, 2010). This has naturally led behavioural scientists to posit that this could be a source of inspiration for a whole new set of therapeutic tools. Innovations in mobile technology in the last 10 years offer a very promising means by which such therapies can be delivered whenever the user or patient is free to practice them. The collaborative research project BeatHealth aims to be at the forefront of these technological developments (Beathealth, 2014). The objective of the project is to create a new method for improving health and wellness based on rhythmic stimulation. To achieve this requires an age-friendly, portable system that has the capability to invigorate the user through musical playlists and then simultaneously record their movements (i.e., during walking or running) and physiological activity via advanced sensors. These sensors must be tailored to the individual s motor performance and physiological response. Additionally, as the kinematic data and stimulation parameters are collected on the fly they are to be recorded via a dedicated e-health service network-based application for storage on a cloud service. This will facilitate the visualization of information on movement performance for the individual themselves and for sharing among family members, doctors and coaches. Such access to this information will empower the user to become aware of her/his motor condition, whether healthy or deficient, and encourage them to adopt a more active lifestyle to either enhance their performance or compensate for a motor disorder they might have. 1 Vitória - ES - Brazil 46

59 From Digital Arts to Ubiquitous Music V Ubimus An essential component to this application is the delivery of the music used to stimulation the kinematic activity. However, it is not simply a playback mechanism, but instead takes a significant role in the process. The belief is that by encouraging an entrainment, or synchronization, between the music and the movement then the maximum benefits should be obtained. This can be realized at both a coarse and fine degree, by choosing music whose tempo is simply close to the rhythm of movement, or even further by using audio processing techniques to dynamically adapt the beat pattern of the music to exactly match it. While not specifically being a music making application, the integration between music and computing in the BeatHealth project means that it is related to a branch of the research field of Ubiquitous Music Systems. According to (Pimenta et al, 2004) these systems should support mobility, social interaction, device independence, and context awareness. Certainly, from a first glance, it would seem that BeatHealth would satisfy these criteria. Additionally, in establishing such a connection ideas from Ubiquitous Music systems may inspire tangential developments. The remainder of the paper aims to investigate this more fully. Firstly, some detail on the conceptual framework behind BeatHealth will be given, followed an outline of the technological architecture. These will be covered in Sections 2 and 3 respectively. Section 4 will set out the characteristics of Ubiquitous Music System, while Section 5 will discuss the relationship between BeatHealth and these systems. Section 6 will provide some conclusions and future work. 2. The Theory and Science of BeatHealth Appreciation of musical rhythms is an important feature of human culture. A key feature of rhythm is an underlying regular beat: a perceived pulse that marks equally spaced points in time (Cooper and Mayer, 1960), (Lerdahl and Jackendoff, 1983). Humans are unique in their ability to couple movement to external rhythms. Beat perception can feel automatic and the majority of the adult population can easily achieve this (Drake, Penel, and Bigand, 2000); the ability to engage in dancing being an obvious example. It occurs without musical training and can even be seen in young children. Neuroimaging has confirmed activity in motor areas of the brain during the production and perception of rhythm (Schubotz, Friederici, and von Cramon, 2000), (Danielsen et al, 2014). Thus, moving to the beat of an external auditory stimulus is sustained by a dedicated neuronal circuitry including subcortical areas, such as the basal ganglia and the cerebellum, and cortical regions (e.g., temporal cortex, prefrontal areas, and the Supplementary Motor Area) (Repp and Keller, 2008), (Zatorre, Chen, and Penhune, 2007). The basal ganglia particularly show a specific response to the beat during rhythm perception, regardless of musical training or how the beat is indicated. A natural extension of these findings to applied research is to exploit rhythm as a way to enhance movement performance. Rhythm, by its tendency to recruit regions of the brain involved in motor control and timing (Zatorre, Chen, and Penhune, 2007), (Grahn and Brett, 2007), and by fostering synchronized movement, is ideally suited for modifying and improving movement performance (e.g., increasing movement speed or frequency or reducing variability). It is worth noting that the basal ganglia mentioned above are compromised in people suffering from motor disorders, for example Parkinson's disease, and patient studies have shown that they exhibit deficits in timing tasks (O'Boyle, Freeman, and Cody, 1996). However, rhythmic signals with a strong external 47 Vitória - ES - Brazil

60 V Ubimus From Digital Arts to Ubiquitous Music beat have been observed to ameliorate gait problems in persons with Parkinson's disease (Nombela et al., 2014). 2.1 Entrainment and Self-Entrainment This link between an external rhythm and the human body s movement response is a phenomenon known as entrainment (Clayton, Sager, and Will, 2004). This theory describes the synchronicity of two or more independent rhythmic processes. Among its many applications entrainment also appears as a topic in music research and is best illustrated in its use in the study of musical meter. An element of Meter is the beat : this is a perceived emphasis of certain events or pulses within it that are equally spaced in time. (Trost et al., 2014). A current model under study by music psychologists is the Dynamic Attending Theory (DAT) that focuses on the role of metrical structure as an active listening strategy (Bolger, Trost, and Schön, 2013). Essentially, rather than assuming that the perception of time and meter are solely determined to be within the musical cues transmitted from performer to listener, this model proposes that rhythmic processes endogenous to the listener entrain to cues in the musical sound. This entrainment model appears to better reflect the cognitive processes than others (Bolger, Trost, and Schön, 2013). It has also been suggested that the entrainment concept can be used to study of proto-musical behavior in infants (Bolger, Trost, and Schön, 2013). Not all entrainment involves an external stimulus, either environmental or interpersonal. 'Self-entrainment' describes the case where two or more of the body's oscillatory systems, such as respiration and heart rhythm patterns, become synchronized (Phillips-Silver, Aktipis, and Bryant, 2010). It is the rhythmic responsiveness to selfgenerated rhythmic signals. A simple block diagram of the process involved is shown in Figure 1 (Phillips-Silver, Aktipis, and Bryant, 2010). In the figure the feedback from the output to the rhythmic input of the entrainment system is the source of the selfentrainment. Self Entrainment Rhythmic input Entrainable System Rhythmic output Figure 1 Block Diagram illustrating the process of Self-Entrainment (Phillips- Silver, Aktipis, and Bryant, 2010) It has been considered that complex-bodied humans and animals typically exhibit selfentrainment in their physical activity, that is, a gesture by one part of the body tends to entrain gestures by other parts of the body (Clayton, Sager, and Will, 2004). For example, arm movements in walking could, in principle, be totally independent from leg movements, but in fact they are not. It 'feels' much easier, is more harmonious, and less strenuous if the arms lock into the leg movements. A similar effect is reported for the locking of step and inhalation cycles in jogging (Clayton, Sager, and Will, 2004). The degree and kind of self-entrainment exhibited depends on the individual and the task being carried out. Vitória - ES - Brazil 48

61 From Digital Arts to Ubiquitous Music V Ubimus Entrainment and Health As mentioned above, the concept of Entrainment is readily applicable to the human body and its response to external stimuli. Relevant medical research has considered the behavior of endogenous physiological rhythms in humans (such as the variation of body temperature over the 24-hour cycle), and how the study of those rhythms might be further developed as a tool in the diagnosis of pathological states. The hope is that this could lead to the development of new treatments. Other research investigations are considering the field of music therapy and determining a link between entrainment and socialization. However, the relationship between entrainment, the stability of biological rhythms and health is still not well understood. There are examples of where relatively stable and entrained biological rhythms are associated with good health. A good example is the enhanced stability of the heart rate afforded by a pacemaker. Conversely asynchrony and instability of rhythmic processes can be associated with pathologies (Clayton, Sager, and Will, 2004). However, entrainment does not necessarily imply stability of biological rhythms, and stability on its own is not necessarily associated with good health. The behavior of Brain waves is a case in point: stable brain waves may indicate a condition such as epilepsy, while unstable waves can indicate a healthy state (Clayton, Sager, and Will, 2004). A certain amount of flexibility and dynamic equilibrium is more likely to be associated with health in many systems, as is a degree of "noise", or random variation in normal physiological rhythms (Clayton, Sager, and Will, 2004). According to (Phillips-Silver, Aktipis, and Bryant, 2010) the capacity to exhibit the simplest form of entrainment emerges when three critical building blocks are in place: (1) the ability to detect rhythmic signals in the environment; (2) the ability to produce rhythmic signals (including rhythmic signals that are byproducts of other functions, such as locomotion or feeding behavior); and (3) the ability to integrate sensory information and motor production that enables adjustment of motor output based on rhythmic input. Observing these three criteria can indicate whether entrainment is being manifested in a healthy or less healthy, or pathological, manner. If the healthy functioning of a system requires a certain degree of entrainment, then either a lack of entrainment, a weakening or even an excessive strengthening of entrainment can be associated with a change to a pathological state (Phillips-Silver, Aktipis, and Bryant, 2010). 2.3 Stimulating health through entrainment The fundamental idea is that by stimulating an entrainment between auditory rhythmical cues and spontaneous or deliberate movement, it boosts individual performance and leads to enhancements in health and wellness. For healthy people, this means that they should synchronize their movement with the beat of an external music source when dancing or when performing physical and sport activities such as running or cycling. This should lead to measurable improvements in their gait kinematics, for example increased velocity and cadence (Wittwer, Webster, and Hill, 2013), and produce (i) better coupling between breathing and running, (ii) a reduction of energy expenditure, and (iii) a general increase in endurance and (iv) a desire to run (Hoffmann, Torregrosa, and Bardy, 2012). Additionally, entrainment has a role in a therapeutic context where 49 Vitória - ES - Brazil

62 V Ubimus From Digital Arts to Ubiquitous Music movement is constrained by a motor disease. One study reported how it been integrated into a rehabilitation therapy in patients with motor disorders (Wittwer, Webster, and Hill, 2013). The idea is to use external rhythmical cues to help patients regularize their gait. The patient is asked to match her/his walking speed to a regular stimulation in the form of a repeated isochronous sound (metronome) (Nombela et al., 2013). 2.3 Related Consumer Technologies To date only a few consumer applications and technologies that exploit rhythm for enhancing movement have been introduced. These have been designed for healthy people that are trying to improve their exercise regime. Yamaha released BODiBeat in 2007 (Yamaha, 2007) followed by Philips Activa (Philips, 2010) in Applications are now appearing for mobile devices. No similar commercial products are available for people with movement disorders. This technology is in its infancy however. There is a lack of sophistication in means of achieving and maintaining synchronization between the music and movement. Furthermore, there is a need for more scientific insight into how best to capture and analyze the relevant physiological signals and to relate them to the auditory cues. This is the motivation for BeatHealth. Its objective is to realize an intelligent technological architecture that can deliver flexible and efficient rhythmical stimulation that can be adapted to any individual's skills, whether the individual is healthy or not, which will enhance and monitor features of their movement performance. The next section will explain the organization of the Beathealth project. 3. Beathealth Organisation The fact that there are gaps in both the current science and technology meant that the Beathealth project needed to be a highly multidisciplinary endeavor, requiring input from physiology scientists, medical consultants, music technology researchers, and software engineers. The BeatHealth project was designed for healthy citizens of various ages that engage in physical activity and for patients with the movement disorder of Parkinson's disease. Three primary challenges were identified for the project: (i) fundamental research aimed at improving information parameters for maximizing the beneficial effects of rhythmic stimulation on movement kinematics and physiology, (ii) technological development to a achieve state-of-the-art implementation platform to deliver the rhythmical stimulation that has attributes of portability, flexibility and reliability, (iii) the creation of a new IT service in the form of a network-based application for collecting on the fly kinematic data and sharing them with online with others such as medical doctors, family, and trainers. The process of facing these challenges can be illustrated in the three interconnected areas shown in figure 2 below: Rhythmic stimulation Mobile App Cloud Service Vitória - ES - Brazil 50

63 From Digital Arts to Ubiquitous Music V Ubimus Figure 2 Three areas of Beathealth and their interconnections Rhythmic stimulation is about the boosting of motor performance. It is a fundamental scientific research component of the project. This aims to improve our knowledge of the auditory stimulation parameters that are best suited for entraining movement. This will be investigated for both healthy individuals and patients with motor disorders. For patients with motor disorders it will investigate how to produce more effective novel therapies using such stimulation parameters. It is of particular interest to find rehabilitation strategies that can create long term benefits that will extend beyond the clinic. For the audio stimuli, attention will be devoted to understanding which type of stimulus (i.e. existing music or artificially generated signals) best fits the particular individual preferences and functionalities in relation to the motivational effort. Possibly, the use of automated composition tools may also help for certain tasks. The Mobile Application for the Beathealth system is a redevelopment of and builds on the ideas of D-Jogger (Moens, Van Norden, and Leman, 2010), which was previously developed by one group in the project consortium. The structure is that a sensor or sensors detect bodily movement and complimentary physiological responses, and these sensor responses are transmitted to a mobile device that is carried by the user. Processing of the sensor signals is required to smooth out noisy fluctuations if the user is engaged in vigorous activity. Special algorithms are required in the case of multiple sensors to fuse the signals together into a single waveform that is used to synchronize the auditory stimulation with the rhythm of the activity in an optimal manner. Self Entrainment of the user Sensor input Mobile App Audio output Figure 3 The user process of Self-Entrainment using the mobile app The mobile application will contain the playlist of audio stimuli. It will either reside on the device itself or be streamed over the network. A music synchronization (Moens, Van Norden, and Leman, 2010), algorithm will be responsible for the alignment of the tempo of the audio stimuli with the movement. Thus, the user will exhibit self-entrain their movements with the audio, which will in turn effect the periodicities of the sensor input. Figure 3 shows a block diagram of the process and it is obvious that it matches the feedback system of Figure 1. Lastly, the mobile application will be available to run on a cost-effective smartphone platform. 51 Vitória - ES - Brazil

64 V Ubimus From Digital Arts to Ubiquitous Music The Cloud Service is a network-based application for the visualizing and sharing of information on the movement performance collected via the application. This will be sent on-the-fly over the internet and made available on a dedicated e-health platform. The user will be able to create and maintain a profile facilitating ongoing regular assessment and monitoring of physical fitness and wellbeing. The user s health consultants can also access this information for assessment. Examples of current commercial services are Apple s HealthKit (Apple, 2014) and Microsoft s HealthVault (Microsoft, 2014). 3.1 Beathealth Evaluation The BeatHealth application will be validated continuously throughout the project with healthy users and in patients with motor disorders. Close to the end of the project a full evaluation procedure will be carried out. Indicators of change in performance along with changes in health status and the motivation to perform physical activity will be recorded. Alongside this, measurement of the quality of the actual software product using metrics to assess attributes such as usability and efficiency will be done and subject to analysis. 4. Ubiquitous Computing and Music Following mainframe and personal computing, ubiquitous computing is considered to be the third wave of computing technology (Moloney, 2011). It is also known as Pervasive computing. The underlying idea is that as technology improves devices become smaller but with increasing power such that they can be imperceptibly embedded in the general environment, thus delivering ubiquitous access to a computing environment (Moloney, 2011). Its benefit is that it simplifies people s lives with technology that facilitates that uses sensors to understand what they are doing in the world and then self-adapts to respond to their users needs. The five key components of ubiquitous computing systems have been determined as being (Kurkovsky, 2007): (1) Embedded and Mobile Devices (2) Wireless Communications (3) Mobility (4) Distributed Systems and (5) Context Awareness and Invisibility. Ubiquitous computing integrates a broad range of research topics, which includes, but is not limited to, distributed computing, mobile computing, location computing, mobile networking, context-aware computing, sensor networks, human-computer interaction, and artificial intelligence. The initial incarnation of ubiquitous computing was in the form of "tabs", "pads", and "boards" (Weiser, 1991) built at Xerox PARC from However, it has come through a revolution with the advent of the mobile smartphone. It facilitates a Ubiquitous computing that is invisible, everywhere computing that does not live on a personal device of any sort, but is in the woodwork everywhere (Weiser, 1991). The mobile phone has now become a true manifestation of the pervasive service and is much easier for the majority of users to conceptualize and interact with (Roussos, Marsh, and Maglavera, 2005). Ubiquitous music is a research area that is a subset of ubiquitous computing and features mobile and networked music, eco-composition and cooperative composition. A ubiquitous computing music system can be defined as a musical computing environment that supports multiple users, devices, sound sources and activities in an integrated way. The technology allows for mobility, social interaction, device Vitória - ES - Brazil 52

65 From Digital Arts to Ubiquitous Music V Ubimus independence, and context awareness (Pimenta et al., 2009). However, a Ubiquitous music systems places strong demands on the computing interface. A good example is the use of mobile devices. Depending on the desired activity, there may be needs beyond the screen interface requiring context awareness mechanisms and locationspecific configuration of parameters that necessitate sensor or actuator capabilities to the system. However, the benefit of the Ubiquitous computing platform for music is that it may empower both non-musicians as well as musicians to express themselves though the medium of music in a collective, open-ended manner. 5. Relationship of Beathealth to Ubiquitous Music and Computing In its architecture the Beathealth application certainly references all the components of Ubiquitous computing given in Section 4. It facilitates external sensors for gathering physiological and kinematic information. It runs on a smartphone. Data gathered is stored on a cloud service. Ideally, the audio tracks will come from a streaming service. Lastly, it reacts to the user movement by adapting the audio in terms of its beat pattern to fit with the rhythm of the movement. With respect to the definition of ubiquitous computing and music the Beathealth application, with some adjustment, can align itself with the concepts promulgated by the practitioners in this field. It facilitates an alignment between movement and music so strictly speaking it is not an instrument of musical expression or composition. However, the alignment it does facilitate embodies a profound interaction between the human user and the computing system playing the audio: the rhythmic time-scale of the audio adapts to the movement of the user. Thus, the user is engaging physically and mentally with the music in a dynamic feedback system, as in the self-entrainment system of Figure 1. Additionally, the Beathealth application is not necessarily constrained to use standard commercial audio tracks as mentioned in Section 3. Actually, it has the flexibility of allowing the use of artificially generated test signals that can be applied in the scientific study of movement. An example could be Amplitude Modulated sounds (Joris, Schreiner, and Rees, 2004). Moreover, if desired, the commercial audio can be extended or replaced to incorporate other compositionally inspired sounds that a user may desire or even require. This means that the Beathealth application can be brought beyond its original intention as an exercise app or a novel therapeutic tool for patients with motor disorders. This can lead to more creative approaches to auralizing all the potential kinematic features derived within the complete Beathealth framework where specific apps are just particular manifestations of what it can be configured to achieve. If it is configured to support multiple users, dealing with user-selectable audio streams that can be modified by the users activities, it can therefore become a compositional tool within a suitable environment that stimulates kinematic activity. The activity and setting then define what the Beathealth application can be. Thus, with realtime modification of multiple-user defined audio performed as a collective activity it can transform the activity into a social and artistic experience. This interacts on many levels, harnessing the intellectual and the emotional along with their physical selves. The impact this may have on the sense of wellness could be more profound than just a kinematic motivator alone. 53 Vitória - ES - Brazil

66 V Ubimus From Digital Arts to Ubiquitous Music 6. Conclusion This paper has discussed the scientific background to the Beathealth project, explaining its origins from the theories of entrainment, and particularly self-entrainment. It then explained the organization and components of the Beathealth application itself. The features of Ubiquitous computing systems were discussed that were followed by the specifics of such systems when designed for music. Finally, it described how the Beathealth application fits within these definitions but also suggested how it can be brought beyond its original health and therapeutic contexts to a vision where it can embody social interaction among multiple users where it could stimulates a musical creativity fused with kinematics that could enhance the sense of wellness it can deliver to collectives of users. 7. Acknowledgement The Beathealth- Health and Wellness on the Beat project (no: ) has received research funding from the European Union under the FP7 program ( ). The work in this paper reflects only the authors views and that the European Union is not liable for any use that may be made of the information contained therein. 8. References Pollack, S., Scientists investigate health benefits of music, rhythm and movement, The Irish Times. Jan. 14, Ellis, R., and Thayer, J.F., Music and Autonomic Nervous System (Dys)function, Music Perception Apr 2010; 27(4): BeatHealth: Health and Wellness on the Beat, 2014, Pimenta, M., Flores, L.V., Capasso, A., Tinajero, P., and Keller, D., Ubiquitous Music: Concepts and Metaphors, in Proc. of the XII Brazilian Symposium on Computer Music, Recife, 2009, pp Cooper, G.W., and Meyer, L.B., The rhythmic structure of music, University of Chicago press, Lerdahl, F., and Jackendoff, R., A generative theory of tonal music. Cambridge, MA: MIT Press, Drake, C., Penel, A., and Bigand, E., Tapping in time with mechanically and expressively performed music, Music Perception, 18(1), 2000, pp Schubotz, R., Friederici, A.D., and von Cramon, Y., Time perception and motor timing: a common cortical and subcortical basis revealed by fmri, Neuroimage, 11, 2000, pp Vitória - ES - Brazil 54

67 From Digital Arts to Ubiquitous Music V Ubimus Danielsen, A., Otnæss, M.K., Jensen, J., Williams, S.C.R., and Østberg, B.C., Investigating repetition and change in musical rhythm by functional MRI, Neuroscience, 275(5), Sept. 2014, pp Repp, B.H. and Keller, P., Sensorimotor synchronization with adaptively timed sequences, Human Movement Science, 27, 2008, pp Zatorre, R. J., Chen, J. L., and Penhune, V. B., When the brain plays music: auditory motor interactions in music perception and production, Nature Reviews Neuroscience 8, 2007, pp Grahn and J. Brett, M., Rhythm and beat perception in motor areas of the brain, Journal of Cognitive Neuroscience, 19(5), 2007, pp O'Boyle, D. J., Freeman, J.S., and Cody, F.W., The accuracy and precision of timing of self-paced, repetitive movements in subjects with Parkinson's disease, Brain, 119, 1996, pp Nombela, C., Hughes, L.E., Owen, A. M., and Grahn, J.A., Into the groove: Can rhythm influence Parkinson's Disease?, Neuroscience & Biobehavioral Reviews, 37(10), 2013, pp Clayton, M., Sager, R.; and Will, U., In time with the music: The concept of entrainment and its significance for ethnomusicology, ESEM Counterpoint, 1, 2004, pp Trost, W., Frühholz, S., Schön, D., Labbé, C., Pichon, S., Grandjean, D., and Vuilleumier, P., Getting the beat: Entrainment of brain activity by musical rhythm and pleasantness, NeuroImage, 103, 2014, pp Bolger, D., Trost, W., and Schön, D., Rhythm implicitly affects temporal orienting of attention across modalities, Acta Psychologica, 142, 2013, pp Phillips-Silver, J., Aktipis, A., and Bryant, G., The ecology of entrainment: Foundations of coordinated rhythmic movement, Music Perception, 28 (1), 2010, pp Wittwer, J.E., Webster, K. E., and Hill, K., Music and metronome cues produce different effects on gait spatiotemporal measures but not gait variability in healthy older adults, Gait Posture. 37, 2013, pp Hoffmann, D., Torregrosa, G., and Bardy, B.G., Sound stabilizes locomotor-respiratory coupling and reduces energy cost, PLoS ONE, 7(9), e45206, Yamaha Corp. BodiBeat, 2007, ( Philips, Activa, 2010, ( Moens, B., Van Norden, L., and Leman, M., D-Jogger: syncing music with walking, in Proceedings of the 2010 Sound and music computing conference, Barcelona, Spain. Apple HealthKit, 2014, ( Microsoft HealthVault, 2014, ( Moloney, M., Into the future ubiquitous computing is here to stay, Dublin Institute of Technology Paper, Vitória - ES - Brazil

68 V Ubimus From Digital Arts to Ubiquitous Music Kurkovsky, S., Pervasive computing: Past, present and future, in Proc. of the 5th International Conference on Information and Communications Technology (ICICT 2007), Dec. 2007, Cairo, Egypt, pp Weiser, M., The Computer for the 21st Century, Scientific American magazine, 265(3), Sept Roussos, G., Marsh, A.J., and Maglavera, S., Enabling pervasive computing with smartphones, IEEE Journal of Pervasive Computing, 4(2), April 2005, pp Joris, P.X., Schreiner, C.E., and Rees, A., Neural Processing of Amplitude-Modulated Sounds, Physiological Review, 84(2), Apr. 2004, pp Vitória - ES - Brazil 56

69 From Digital Arts to Ubiquitous Music V Ubimus Characterizing resources in ubimus 1 research: Volatility and rivalry Damián Keller Núcleo Amazônico de Pesquisa Musical (NAP), Universidade Federal do Acre, Rio Branco, AC, Brasil - Grupo de Música Ubíqua dkeller@ccrma.stanford.edu Abstract. In this paper I identify three methodological approaches to creativitycentered design: the computational approach, the dialogical perspective and the ecologically grounded framework. And I analyze how these three methods relate to a current definition of the ubiquitous music field (ubimus). Social interaction is one of the factors to be accounted for in ubimus experimental studies. I propose the label social resources for the shared knowledge available within a community of practice. I identify five aspects of creativity-centered design that have targeted social resources. Then I discuss material resources as factors to be considered for the design of ubimus ecosystems and present two new design qualities as variables for experimental studies: volatility and rivalry. This discussion is framed by a split between creative products and creative resources which points to three observables: material resources, material products and material by-products, including creative waste. I conclude with a summary of the main proposals of the paper and point to applications of these concepts in experimental design studies. Resumo. Neste artigo identifico três linhas metodológicas em design criativo o enfoque computacional, a perspectiva dialógica e o método cognitivo-ecológico e analiso como essas linhas se relacionam com uma definição recente do campo de pesquisa em música ubíqua (ubimus). A interação social é um dos fatores que devem ser considerados nos estudos experimentais em ubimus. Proponho o conceito de recursos sociais para o conhecimento compartilhado dentro da comunidade de prática e identifico cinco aspectos do design criativo que tratam dos recursos sociais. Seguidamente discuto os recursos materiais como fatores para o design de sistemas musicais ubíquos, sugerindo duas qualidades de design como variáveis para estudos experimentais: a volatilidade e a rivalidade. A proposta tem como contexto a separação entre os produtos criativos e os recursos criativos, apontando para três tipos de fatores observáveis: recursos materiais, produtos criativos e produtos materiais não intencionais, incluindo o lixo criativo. O texto finaliza resumindo as propostas conceituais e indicando aplicações desses conceitos nos estudos experimentais de design. 1. Ubimus methodological proposals Since 2007, our group has been engaged in a multidisciplinary effort to investigate the creative potential of converging forms of social interaction, mobile and distributed technologies and materially grounded artistic practices. We have proposed the adoption of the term 'ubiquitous music' (ubimus) to define practices that empower participants of musical experiences through socially oriented, creativity-enhancing tools [Keller et al. 2011a]. Ubiquitous music is defined as a research field that deals with distributed systems of human agents and material resources that afford musical activities through sustainable creativity support tools. This consensual definition, established through collaborative work within our community of practice, summarizes the research efforts of three distinct but complementary methodological approaches to the study of ubimus 1 Ubimus: short-hand for ubiquitous music, a research field proposed in [Keller et al. 2011a]. 57 Vitória - ES - Brazil

70 V Ubimus From Digital Arts to Ubiquitous Music phenomena: (1) the computational perspective, (2) the dialogical view, and (3) the ecologically grounded framework Information technology creative practices: proposals for ubimus insfrastructure The computationally oriented perspective on ubimus research has contributed to the material resources and the creativity support components of the above definition [Pimenta et al. 2012]. This line of investigation attempts to expand what is currently known about musical interaction, focusing on human aspects of Information Technology Creative Practices [Mitchell et al. 2003]. Whether involving computing devices or not, musical interaction is defined as interaction that produces creative sonic products through a variety of musical activities. Seen from this light, ubiquitous music comprises sound oriented activities supported by ubiquitous computing (or ubicomp) concepts and technology [Weiser 1991]. Material resources and tools are the various kinds of stationary and portable computing devices integrated into ubimus ecosystems [Flores et al. 2010; Lazzarini et al. 2012]. Distributed systems of human agents and material resources generally involve interactive computing processes and synchronous or asynchronous exchanges of data. Complementarily, musical interfaces comprise the material and the virtual resources that support musical experiences in real-world contexts. Therefore, experimental work from the computationally oriented perspective strives to capture human-computer interactions that occur during actual music making, independently of the type of interfaces employed, the locations of the participants and the temporal distribution of the interactions. Given the multiplicity of factors involved in music making, it comes as no surprise that ubiquitous music systems place high demands on the design of the support infrastructure. These requirements are hard to satisfy if the relationships among the components of the systems are not taken into account. Depending on the context, devices may provide sensor or actuator capabilities encompassing both stationary and mobile components. Synchronous activities place high pressure on the computational resources, especially when synchronous rendering of audio is involved [Lazzarini et al. 2012]. In the context of mobile, external group activities, both reliable connectivity and the ability to handle fairly large amounts of data may be necessary. When engaged in musical activities with portable devices, participants may need access to the state of the system regardless of the location where the action takes place [Pinheiro da Silva et al. 2013b; Keller et al. 2013]. Distributed asynchronous activities require consistent data representations for simultaneous or intermittent access by multiple users [Miletto et al. 2011; Scheeren et al. 2013; Testa et al. 2013]. While in this scenario time synchronization support may be forfeited, ensuring persistent data mechanisms across all the network components is a minimal requirement. The multiplicity of use scenarios and contexts proposed for ubimus activities [Miletto et al. 2011; Keller et al. 2011a; Pimenta et al. 2012] relegates the case of the collocated, synchronous performance of digital musical instruments to the exception rather than the ideal model on which to base all design decisions. The results of seven years of ubimus research indicate that a ubiquitous music ecosystem can hardly be considered a musical instrument or a passive object to be played by a musician. A more appropriate metaphor encompasses agents in a dynamical system adapting to the local environment and to remotely accessible resources while carrying out musical activities [Keller et al. 2011a; Lazzarini et al. 2012] The dialogical approach to ubiquitous musical phenomena: the local context Vitória - ES - Brazil 58

71 From Digital Arts to Ubiquitous Music V Ubimus The focus on human agents and the centrality of sustainability issues suggested by the proposed definition of ubiquitous music research are grounded on two current approaches to educational practices: the dialogical perspective pioneered by Paulo Freire (1999) and the free circulation of know-how and material resources proposed by the open educational resources initiative [Lima et al. 2012, 2014]. This research agenda is based on a participatory, community-based, subject-centered view of education [Lima and Beyer 2010], targeting both formal and informal educational settings. Paulo Freire s (1999) educational philosophy pushes the teacher s role beyond a mere conduit for technical-theoretical information and encourages active protagonism by the stakeholders of educational activities. Freire's dialogical conception sharply contrasts with views that see creativity as a purely mental, individual process. Through hands-on activity and social interaction among peers, students are stimulated to evaluate their work. Given the relevance of the local referents, participants are encouraged to reflect about their own processes and products during musical activities. While keeping tabs on the local reality, they develop a critical view on their products and creative processes. Through iterative cycles of exchanges, dialogical methods foster individual and collective reflections. Converging trends in creative practice research, educational research and music education point to the local context as a key factor in shaping creativity in educational settings [Burnard 2007; Keller 2000; Loi and Dillon 2006; Keller et al. 2010]. Loi and Dillon (2006) propose that adaptive educational environments can be designed as creative spaces that foster interaction through situational and social dynamics. Technology becomes a key resource in this type of educational environments. Burnard (2007) applies this framework within the music domain by placing creativity and technology as the two central forces enabling innovative educational practices. She cites the use of online and collaborative technology as enablers for creativity in educational settings, proposing practice, participation and collaborative networking as objectives of music education research. These situated, socially informed approaches stand in stark contrast to the standard educational views on musical creativity. While standard models were concerned with activities that (in theory) could be carried out without the need for social interaction or place-specific experience, such as problem-solving and thinking [Webster 2003], situated approaches bring socially acquired musical experience to the forefront of the research agenda. Thus, they highlight two aspects that need to be considered in creativity-centered design: the place factor and the mutual processes of adaptation that emerge through social interactions. Both aspects can be handled by methods proposed in the context of ecologically grounded creative practices Ecologically grounded creative practices Western art practices have usually focused on what to do with musical materials, rather than what to do to empower people as creative musicians. Arguably, music creation can only be carried out by well-prepared, creative individuals who are versed in the secrets of Euterpe [Euterpe Εὺτέρπη 'well delighting' from Indo-European 'ei', 'to go' and 'terp-', 'to satisfy oneself']. Special stress has been placed on the concept of the individual activity done for self-fulfilling purposes. The view on musical creativity as an individual activity has also been adopted by technologically based musical practice. As a result, the constraints formerly imposed by acoustic instrumental writing such as working indoors and the exclusion of the audience as active participant in the creative process were inherited by mainstream computer music practices (see Wishart 2009 for an example of this perspective). Sonic art remains an activity carried out in the isolation 59 Vitória - ES - Brazil

72 V Ubimus From Digital Arts to Ubiquitous Music of the studio. This gap between the organizational systems applied on the musical material and the context where the material resources are gathered enforces creative techniques based on the objectification of sound. The studio as a compositional environment follows the model of the physics or the biology lab. Sounds are isolated and dissected according to well-established protocols, giving the composer total control over his creative product. The studio-centered working methods enforce the idea that the creative process consists of abstract relationships among sound objects masterfully executed by a well-trained musician. In the late 1990s, the application of embedded-embodied theories on cognition [Gibson 1979] laid out a path to an alternative view of musical creativity. Windsor (1995) and Keller (1999a; 2000) provided the initial coverage of the embedded-embodied approach to music making and music perception literature. Through an acute and highly critical essay, Windsor (1995) brought several ecological concepts into the realm of musical analysis. His proposal although tuned to the demands of studio-centered electroacoustic practice highlighted the close affinity between sonic art practices and ecologically oriented theoretical efforts. He attempted to establish a bridge between the concept of affordance and the triadic representational model proposed by Peirce (1991), arguing for a sign-oriented reinterpretation of affordances. Working independently from a complementary perspective, Keller and Truax (1998) proposed a Gibsonean approach to music making. Ecologically grounded synthesis techniques were presented as a proof of concept of the applicability of the embedded-embodied view on cognition within the context of creative music making. Two ecologically grounded works featured examples of natural synthetic textures and everyday sonic events:... soretes de punta. (Keller 1998; see [Basanta 2010] for a thorough analysis of this piece) and touch'n'go [Keller 2000]. After Windsor's and Keller's initial proposals, several artists embraced embeddedembodied cognition as a conceptual and methodological basis for their creative practice. Matthew Burtner (2005; 2011) realized a number of compositional experiences involving field recordings and interactive techniques. As a reference to early perceptual research [Vanderveer 1979], he labelled his work 'ecoacoustics.' Agostino Di Scipio (2002) expanded the palette of synthesis techniques by applying iterated functions to produce natural textures. His compositional work Audible Ecosystemics [Di Scipio 2008] featured the use of space as a key parameter for real-time creative practices. Natasha Barrett (2000) and Tim Opie proposed techniques for gathering acoustic field data produced by animals and physical agents [Opie and Brown 2006]. Barrett's compositional work included the use and implementation of spatialization techniques based on ambisonics. Davis (2008) and Basanta (2010) adopted ecologically oriented approaches to increase the participatory appeal of their sonic installations. And Nance (2007) and Lockhart introduced ecologically grounded practices into the realm of instrumental composition [Lockhart and Keller 2006]. A common denominator of ecologically grounded creative practices is the close integration of sound processes shaped after natural phenomena with perceptual and /or social factors wrought by everyday experience. The ecocompositional paradigm that has emerged from the multiple creative projects realized since 1997 encompasses two strategies: (1) the construction of a theoretical framework for creative practices supported by embedded-embodied cognitive mechanisms [Keller 2000; Keller and Capasso 2006; Keller 2012]; and (2) the concurrent development of design techniques coherent with this theoretical scaffolding, featuring participation and emergence as the two central creative driving forces [Keller et al. 2011a]. Soundscape composition brought real-world context into the musical work. Ecocomposition sought to place Vitória - ES - Brazil 60

73 From Digital Arts to Ubiquitous Music V Ubimus music creativity into real-world contexts. During the last decade, two strategies were developed for this purpose. On the one hand, music making involved reenacting experiences in their original geographical milieu [Keller 2004]. On the other, musical works were co-composed with the public [Keller 2000; Keller et al. 2011a]. Thus, ecocomposition took the act of creation out of the realm of the studio. Techniques such as accumulation and enactive social interaction helped to lower the usability requirements of musical systems, bringing the audience into the creative act [Keller et al. 2002]. 2. Social factors: communities of practice One of the objectives of ubiquitous music research is to gather insights on the relationships between the subjects profiles and the strategies they use to handle the creative tasks. Subjects may choose to approach the creative activity by applying previously learned strategies. Sometimes, this background knowledge may not be applicable to technologically enhanced environments. So ubiquitous music experiments have adopted a parsimonious method for increasing tool access without hindering reuse of previous knowledge [Lima et al. 2012; Keller et al. 2013]. Tools are presented as opportunities for interaction, but they are not given as requirements until a series of preliminary planning studies has been completed. Depending on their specific profile and their previous experience, some subjects take advantage of computationally based support while others limit their actions to simple forms of sonic manipulation. Again, this aspect of the procedural dimension is treated as a variable to be observed instead of being a predetermined condition. Community-based methods are at the center of ubiquitous music practice [Pimenta et al. 2012]. The free access to know-how and the fast circulation of resources within social groups with common objectives foster the emergence of a phenomenon quite relevant to ubiquitous music research: the communities of practice [Wenger 2010]. A community of practice is a social system that arises out of learning and exchange processes. This type of community unfolds through practice, not prescription [Wenger 2010:192], so it can be seen as an extension of the dialogical perspective [Freire 1999; Lima et al. 2012]. Take as an example open-source communities. Communities that are nimble and flexible consisting of volunteer developers who make contributions either individually or as part of temporary teams with shared governance foster imagination, engagement and consensus [Pimenta et al. 2012]. Brown and Dillon (2007) and Bryan-Kinns (2004) network music experiments suggest that these characteristics afford increased levels of participation in musical activities. Therefore, communities of practice should constitute a fertile context for creativity-centered design. Summing up how social factors have impacted ubimus research, this section has focused on the use of social resources at several levels: (a) ubimus planning studies have provided insights on the relationships between the subjects profiles and the strategies they use to handle the creative tasks [Lima et al. 2012]; (b) community exchanges of material and social resources have been used to support learning activities; (c) communities of practice were employed as the social grounding for creativity-centered design activities; (d) social interactions were used as tools for design assessment and critical evaluation; (e) socially shared resources have served as a factor for growth and consolidation of a community of practice engaged in ubiquitous music research [Pimenta et al. 2012]. By fostering social exchanges among music practitioners, the activity of prototyping creative products has been incorporated into creativity-centered design. Design activities involving negotiation among artistic, computational and educational perspectives have helped to adjust the objectives and methods of the ubimus research agenda. And at the longest time span, the formation of a ubiquitous 61 Vitória - ES - Brazil

74 V Ubimus From Digital Arts to Ubiquitous Music music community of practice encompassing both novice practitioners and experienced designers has encouraged the circulation of material and social resources feeding the community s sustainable growth. 3. Material factors: volatility and rivalry Keller and coauthors (2011b) define the material dimension as the collection of resources available to the participants of a creative activity. In the case of ubiquitous music systems, the material dimension encompasses the sound sources and the tools used to generate creative musical products and the material results of the musical activity. Music creativity models that emphasize the material dimension provide the most direct window to experimental observation. Two of the three interrelated stages suggested by Dingwall (2008) the generation stage and the development stage can easily be assessed by measuring the quantity of the material produced. The stage putting the pieces together may involve selection, grouping and disposal of material resources; therefore both objective and subjective assessments may be necessary. Objective assessment demands measurements of the resource yield and the resource consumption as a function of time [Ferraz and Keller 2012; Keller et al. 2011c]. Bennett s (1976) model suggests that musical creative processes start from a single germinal idea. Collins (2005) also adopts this view but allows for several musical ideas (he calls them themes or motifs) at the initial stage. Contrastingly, Hickey (2003), Burnard and Younker (2004), Chen (2006) and Dingwall (2008) models suggest that exploratory activities precede the selection of materials. The methodological difficulty resides in the task choice for creativity assessment experiments. The underlying hypothesis is as suggested by Hickey, Burnard and Younker, Chen and Dingwall models that both restricting and providing access to materials are part of the compositional process. Therefore, by selecting materials or tools the experimenter is taking the place of the composer and the resulting data cannot be used to determine whether the creative musical activity begins by exploratory actions or by a well-defined procedural plan with an explicit material objective. When the musical materials are given by the experimenter, it is not possible to draw conclusions regarding how the material resources are collected. This methodological problem is called early domain restriction [Keller et al. 2011b]. Focusing on creative music making as an activity [Barreiro and Keller 2010] has several implications on the study of material resources. Ubiquitous music phenomena involve both the locally available objects and the remote materials accessible through technological infrastructure. Therefore, we need to consider at least two types of resources: 1. the resources present on site, defined in the creativity literature as the place factor (i.e., collocated resources), and 2. the materials accessed through creativity support tools [Shneiderman 2007] which may or may not be collocated (i.e., distributed resources). Iannis Xenakis (1971/1992) suggested that creative musical activities may occur in-time or out-of-time. This idea has been adopted by the human-computer interaction literature under the labels of synchronous and asynchronous activities [Miletto et al. 2011]. Applying this notion to material resources introduces a new target for experimental work. Some materials may only become available during the creative activity and cannot be recycled for future use. Other resources may be repeatedly used in the context of asynchronous creative work. An example of the former case are the improvisatory performances based on network infrastructure. Each participant's action depends on the sonic cues provided synchronously by the other participants. These sonic cues are only available in-time, therefore they can be classified as volatile material resources. Other resources can be incorporated in the context of iterative cycles of creative activity. A good example is provided by the concept of musical prototype Vitória - ES - Brazil 62

75 From Digital Arts to Ubiquitous Music V Ubimus [Miletto et al. 2011]. A musical prototype is a data structure that supports actions by multiple users through a network infrastructure. A single creative product is shared by the participants collaborating throughout the creative cycle. Participants access the musical prototype remotely and cooperate by doing direct modifications and by providing comments on their actions and on their partners' actions. Creative decisions are the result of a cumulative process of material exchanges that can last from a few hours to several months. Hence, we can say that a musical prototype is a non-volatile material resource. Recent theoretical proposals on creativity generally label the results of creative activity as 'products' [Kozbelt et al. 2010]. If we take into account the ongoing mutual adaptations among agents and objects during creative activities [Keller and Capasso 2006], a functionally oriented description of the material resources becomes necessary. Material results of creative activity may be either resources or products depending on their role within the context of the activity. For example, the sounds collected in San Francisco's Bart transportation system (metro or subway) served as material resources for the creative product Metrophonie [Keller 2005]. The same collection of sounds were expanded through ecological modeling techniques [Keller and Berger 2001; Keller and Truax 1998] to be employed as material resources within the multimedia installation The Urban Corridor [Capasso et al. 2001]. In The Urban Corridor, the action of the participants shape the organization of the sonic matter [Keller 2012; Keller et al. 2002]. Every instance of the piece produces a personalized creative product that is different each time the installation is visited. In this case, instead of being delivered as a single creative product, the sound sources of The Urban Corridor are available as material resources for the creative actions exerted by the audience. Hence, while the sound sources and creative products can be clearly separated in Metrophonie, this separation is not possible in The Urban Corridor. In the latter, sound sources remain as material resources and the creative product is equated to the emergent qualities of the interaction among multiple agents within the ubiquitous music ecosystem. A group of perspectives that has direct application in ubiquitous music research comprises the psycho-economic theories of general creativity [Rubenson and Runco 1992, 1995; Sternberg and Lubart 1991]. The underlying assumption of this group of theories is that creative activity both demands and produces resources. Economically oriented approaches provide opportunities for observation and quantification of variables that are hard to assess within other creativity paradigms (for a comparison among creative theories see Kozbelt et al. 2010). Given that available resources for creative activity are finite, they may be quantified. By observing the flux of consumption and production of resources, quantitative predictions may be linked to specific environmental conditions. The effectiveness of the creative strategy can be assessed by comparing the use of resources with the creative yield. The type of creative outcomes could be predicted by identifying what resources are available and how they are used throughout the creative cycle. And the relationship between resource consumption and creative waste can be used to assess the sustainability of the creative ecosystem under observation. Consequently, creative potentials and creative performance become linked to specific variables that can be studied through empirical work. Observable resources become the focus of the experiments, opening a window to quantitative comparisons among different strategies for support of creative activities. From an economy-oriented perspective, material resources may be rival or non-rival. Rival resources lose value when shared. Non-rival resources can be widely distributed without losing value. Information is a good example of a non-rival resource. Information can be freely shared without any impact on its social value. Contrastingly, if 63 Vitória - ES - Brazil

76 V Ubimus From Digital Arts to Ubiquitous Music a food stock is partitioned within a community its value is reduced proportionally to its depletion rate. An empty food stock has no social value. There are some interesting observations to be gathered through the application of the quality of rivalry in creativity-centered design. Resources for creative activities can be characterized by their level of relevance and originality [Weisberg 1983]. In the context of group activities, these two factors constitute opposite forces [Ferraz and Keller 2012]. Creative resources that are unique and have not been shared among group members keep their creative potential and have a high level of originality. Through sharing, original resources lose their creative potential while they gain acceptance among group members. The most relevant resources are the ones most widely distributed with the highest social acceptance. Therefore since creative rival (c-rival) resources lose value through social acceptance, they can negatively impact originality. On the other hand, creative non-rival (c-non-rival) resources can be freely distributed without affecting originality. Given that c-non-rival resources can be widely shared, they can attain higher levels of relevance than the c-rival resources. Sound samples can be classified as creative rival resources. The novelty of the creative products that use samples decrease proportionally to the number of copies of the original sound. Deterministic synthesis models generate the same sound for the same set of parameters, so they can also be classified as c-rival resources. Given that physical objects produce different sonic results each time they are excited, the events they produce can be classified as c-non-rival resources. On a similar vein, an stochastic synthesis algorithm can render multiple events without producing repeated instances [Keller and Truax 1998]. Timbre-based musical practices such as the use of distorted guitar sounds are also examples c-non-rival resources 2. An example of a creative application of resource degradation is provided by [Fenerich et al. 2013]. The authors used an iterative network transmission process to emulate the sonic feedback mechanism proposed by Alvin Lucier (1969) in his piece I am sitting in a room... [Lucier and Simon 2012]. In Fenerich's and coauthors' piece the disruptive noises of the network transmission furnish new material as each copy of the sound is sent through the network. The sonic output is the result of multiple degraded copies of the original sound. 4. Summary and implications for creativity-centered design Taking as a point of departure the current definition of ubiquitous music a research field that deals with distributed systems of human agents and material resources that afford musical activities through sustainable creativity support tools I proposed the use of two design qualities in creativity-centered experimental work: volatility and rivalry. Ubiquitous music experiments need to assess their resource usage through observations of creative products and material resources. While some creative techniques provide a high product yield, other methods tend to produce high levels of creative waste. Therefore, creative waste assessments may furnish a window to the resource flow mechanics of ubiquitous music ecosystems. From a resource-flow perspective, the volatility of the material resources employed is a design quality that can be applied to gauge the level of support for asynchronous activities. Persistent resources, such as network-shared musical data allied to consistent metaphors for interaction, may prove useful to support creative activities across multiple devices, involving access by multiple stakeholders. Ubimus research carried out during the last seven years suggests that the resources' volatility should be taken into account when designing ubimus ecosystems. Creative rival resources do not add value to the creative product when 2 This example was provided by an anonymous reviewer. Vitória - ES - Brazil 64

77 From Digital Arts to Ubiquitous Music V Ubimus shared. Therefore, distribution of copies of creative rival resources among group members should be reduced to a minimum. This limitation does not apply to the case of creative non-rival resources, (e.g. synthesis techniques that generate new material for each iteration [Keller and Truax 1998]). These resources can be shared without imposing a steep reduction on the originality of the stakeholders' creative products. 5. References Bennett, S. (1976). The process of musical creation: Interview with eight composers. Journal of Research in Music Education 24, Brown, A. R. & Dillon, S. C. (2007). Networked improvisational musical environments: learning through online collaborative music making. In J. Finney & P. Burnard (eds.), Teaching Music in the Digital Age (pp ). Continuum International Publishing Group. Barreiro, D. L. & Keller, D. (2010). Composing with sonic models: fundamentals and electroacoustic applications (Composição com modelos sonoros: fundamentos e aplicações eletroacústicas). In D. Keller & R. Budasz (eds.), Criação Musical e Tecnologias: Teoria e Prática Interdisciplinar. Goiânia, GO: Editora ANPPOM. Barrett, N. (2000). A compositional methodology based on data extracted from natural phenomena. In Proceedings of the International Computer Music Conference (ICMC 2000). Ann Arbor, MI: MPublishing, University of Michigan Library. Basanta, A. (2010). Syntax as Sign: The use of ecological models within a semiotic approach to electroacoustic composition. Organised Sound 15, (Doi: /S ) Bryan-Kinns, N. (2004). Daisyphone: the design and impact of a novel environment for remote group music improvisation. In Proceedings of the 5th Conference on Designing Interactive Systems: Processes, Practices, Methods and Techniques (pp ). New York, NY: ACM. (ISBN: ) Burnard, P. & Younker, B. A. (2004). Problem-solving and creativity: insights from students individual composing pathways. International Journal of Music Education 22, Burnard, P. (2007). Reframing creativity and technology: promoting pedagogic change in music education. Journal of Music Technology and Education 1(1), (Doi: /jmte /1.) Burtner, M. (2005). Ecoacoustic and shamanic technologies for multimedia composition and performance. Organised Sound 10 (1), (Doi: /S ) Burtner, M. (2011). EcoSono: Adventures in interactive ecoacoustics in the world. Organised Sound 16 (3), (Doi: /S ) Chen, C. W. (2006). The creative process of computer-assisted composition and multimedia composition: Visual images and music. Doctor of Philosophy Thesis. Melbourne: Royal Melbourne Institute of Technology. Collins, D. (2005). A synthesis process model of creative thinking in music composition. Psychology of Music 33 (2), (Doi: / ) Davis, T. (2008). Cross-Pollination: Towards an aesthetics of the real. In Proceedings of the International Computer Music Conference (ICMC 2008). Ann Arbor, MI: MPublishing, University of Michigan Library. Dingwall, C. (2008). Rational and intuitive approaches to music composition: The impact of individual differences in thinking/learning styles on compositional processes. Bachelor of Music Dissertation. Sidney: University of Sydney. Di Scipio, A. (2008). Émergence du son, son d'emergence: Essai d'épistémologie expérimentale par un compositeur. Intellectica 48-49, Vitória - ES - Brazil

78 V Ubimus From Digital Arts to Ubiquitous Music Di Scipio, A. (2002). The Synthesis of Environmental Sound Textures by Iterated Nonlinear Functions, and its Ecological Relevance to Perceptual Modeling. Journal of New Music Research 31 (2), (Doi: /jnmr ) Eaglestone, B., Ford, N., Brown, G. J. & Moore, A. (2007). Information systems and creativity: an empirical study. Journal of Documentation 63 (4), (Doi: / ) Fenerich, A., Obici, G. & Schiavoni, F. (2013). Marulho TransOceânico: Performance musical entre dois continentes. In E. Ferneda, G. Cabral & D. Keller (eds.), Proceedings of the XIV Brazilian Symposium on Computer Music (SBCM 2013). Brasília, DF: SBC. Ferraz, S. & Keller, D. (2012). Preliminary proposal of the MDF model of collective creation (MDF: Proposta preliminar do modelo dentro-fora de criação coletiva). In Proceedings of the III Ubiquitous Music Workshop (III UbiMus). São Paulo, SP: Ubiquitous Music Group. Flores, L. V., Pimenta, M. S., Miranda, E. R., Radanovitsck, E. A. A. & Keller, D. (2010). Patterns for the design of musical interaction with everyday mobile devices. In Proceedings of the IX Symposium on Human Factors in Computing Systems (pp ). Belo Horizonte, MG: SBC. Freire, P. (1999). Pedagogy of Hope (Pedagogia da Esperança: Um Reencontro com a Pedagogia do Oprimido). Rio de Janeiro, RJ: Paz e Terra. Gibson, J. J. (1979). The ecological approach to visual perception. Boston: Houghton Mifflin. (ISBN: ) Hickey, M. M. (2003). Creative thinking in the context of music composition. In M. M. Hickey (ed.), Why and How to Teach Music Composition: A New Horizon for Music Education (pp ). Reston, VA: MENC, The National Association for Music Education. Hutchins, E. (2010). Cognitive ecology. Topics in Cognitive Science 2 (4), (Doi: /j x.) Keller, D. (1998). "... soretes de punta.". In, Vol. Harague II [Compact Disc]. New Westminster, BC: earsay productions. Keller, D. (2000). Compositional processes from an ecological perspective. Leonardo Music Journal, (Doi: / ) Keller, D. (2004). Paititi: A Multimodal Journey to El Dorado. Doctor in Musical Arts Thesis, Stanford, CA: Stanford University. Keller, D. (2012). Sonic ecologies. In A. R. Brown (ed.), Sound Musicianship: Understanding the Crafts of Music (pp ). Newcastle upon Tyne, UK: Cambridge Scholars Publishing. (ISBN: ) Keller, D., Barreiro, D. L., Queiroz, M. & Pimenta, M. S. (2010). Anchoring in ubiquitous musical activities. In Proceedings of the International Computer Music Conference (pp ). Ann Arbor, MI: MPublishing. Keller, D. & Capasso, A. (2006). New concepts and techniques in eco-composition. Organised Sound 11 (1), (Doi: /S ) Keller, D., Flores, L. V., Pimenta, M. S., Capasso, A. & Tinajero, P. (2011a). Convergent trends toward ubiquitous music. Journal of New Music Research 40 (3), (Doi: / ) Keller, D., Lima, M. H., Pimenta, M. S. & Queiroz, M. (2011b). Assessing musical creativity: material, procedural and contextual dimensions. In Proceedings of the National Association of Music Research and Post-Graduation Congress - ANPPOM (pp ). Uberlândia, MG: ANPPOM. Keller, D., Otero, N., Pimenta, M. S., Lima, M. H., Johann, M., Costalonga, L. & Lazzarini, V. (2014). Relational properties in interaction aesthetics: The ubiquitous music turn. In Proceedings of the Electronic Visualisation and the Arts Conference (EVA-London 2014). London: Computer Arts Society Specialist Group. Vitória - ES - Brazil 66

79 From Digital Arts to Ubiquitous Music V Ubimus Keller, D. & Truax, B. (1998). Ecologically based granular synthesis. In Proceedings of the International Computer Music Conference (pp ). Ann Arbor, MI: MPublishing, University of Michigan Library. Kozbelt, A., Beghetto, R. A. & Runco, M. A. (2010). Theories of Creativity. In J. C. Kaufman & R. J. Sternberg (ed.),, Vol. The Cambridge Handbook of Creativity. Cambridge, UK: Cambridge University Press. (ISBN: ) Lazzarini, V., Yi, S., Timoney, J., Keller, D. & Pimenta, M. S. (2012). The Mobile Csound Platform. In Proceedings of the International Computer Music Conference (pp ). Ann Arbor, MI: MPublishing, University of Michigan Library. Lima, M. H. & Beyer, E. (2010). An experience in musical education and new technologies in school context with Brazilian young people: Reflections and perspectives. In Music Education Policy and Implementation: Culture and Technology. Proceedings of the 15th International Seminar of the Policy Commission on Culture, Education and Media, Kaifeng: ISME. Lima, M. H., Keller, D., Pimenta, M. S., Lazzarini, V. & Miletto, E. M. (2012). Creativitycentred design for ubiquitous musical activities: Two case studies. Journal of Music, Technology and Education 5 (2), (Doi: /jmte _1.) Lockhart, A. & Keller, D. (2006). Exploring cognitive process through music composition. In Proceedings International Computer Music Conference (ICMC 2006) (pp. 9-12). Ann Arbor, MI: MPublishing, University of Michigan Library. Loi, D. & Dillon, P. (2006). Adaptive educational environments as creative spaces. Cambridge Journal of Education 36 (3), (Doi: / ) Lucier, A. & Simon, D. (2012). Chambers: Scores by Alvin Lucier. Middletown, CT: Wesleyan University Press. Miletto, E. M., Pimenta, M. S., Bouchet, F., Sansonnet, J.-P. & Keller, D. (2011). Principles for music creation by novices in networked music environments. Journal of New Music Research 40 (3), (Doi: / ) Miller, S. (2005). Audible-Mobiles: An application of eco-systemic programming in Kyma. In N. Zahler (ed.), Proceedings of Spark: Festival of Electronic Music and Art (pp ). Minneapolis, MN: University of Minnesota. Mitchell, W. J., Inouye, A. S. & Blumenthal, M. S. (2003). Beyond Productivity: Information Technology, Innovation, and Creativity. Washington, DC: The National Academies Press. Nance, R. W. (2007). Compositional Explorations of Plastic Sound. Doctoral Thesis in Music, De Montfort University. Odena, O. (2012). Musical Creativity: Insights from Music Education Research. Ashgate Publishing Company. (ISBN: ) Peirce, C. S. (1991). Peirce on Signs: Writings on Semiotic[s]. J. Hoopes (ed.). Chapel Hill, NC: University of North Carolina Press. (ISBN: ) Pimenta, M. S., Miletto, E. M., Keller, D. & Flores, L. V. (2012). Technological support for online communities focusing on music creation: Adopting collaboration, flexibility and multiculturality from Brazilian creativity styles. In N. A. Azab (ed.), Cases on Web 2.0 in Developing Countries: Studies on Implementation, Application and Use. Vancouver, BC: IGI Global Press. (ISBN: ) Pinheiro da Silva, F., Keller, D., Ferreira da Silva, E., Lazzarini, V. & Pimenta, M. S. (2013a). Creativity in everyday settings: The impact of anchoring (Criatividade em ambientes cotidianos: o impacto do fator de ancoragem). In Proceedings of the IV Ubiquitous Music Workshop (IV UbiMus). Porto Alegre, RS: Ubiquitous Music Group. Pinheiro da Silva, F., Keller, D., Ferreira da Silva, E., Pimenta, M. S. & Lazzarini, V. (2013b). Everyday musical creativity: exploratory study of ubiquitous musical activities (Criatividade musical cotidiana: estudo exploratório de atividades musicais ubíquas). Música Hodie 13, Vitória - ES - Brazil

80 V Ubimus From Digital Arts to Ubiquitous Music Rubenson, D. L. & Runco, M. A. (1995). The psychoeconomic view of creative work in groups and organizations. Creativity and Innovation Management 4 (4), (Doi: /j tb00228.x.) Rubenson, D. L. & Runco, M. A. (1992). The psychoeconomic approach to creativity. New Ideas in Psychology 10 (2), (Doi: Schaeffer, P. (1977). Traité des Objets Musicaux: Essai Interdisciplines. Paris: Éditions du Seuil. (ISBN: ) Schafer, R. M. (1977). The Tuning of the World. New York, NY: Knopf. Scheeren, F. M., Pimenta, M. S., Keller, D. & Lazzarini, V. (2013). Coupling social network services and support for online communities in codes environment. In A. L. Koerich & G. Tzanetakis (eds.), Proceedings of the 14th International Society for Music Information Retrieval Conference (ISMIR 2013) (pp ). Curitiba, PR: ISMIR. Shneiderman, B. (2007). Creativity support tools: accelerating discovery and innovation. Communications of the ACM 50 (12), (Doi: / ) Sternberg, R. & Lubart, T. (1991). An Investment Theory of Creativity and Its Development. Human Development 34 (1), Truax, B. (2002). Genres and techniques of soundscape composition as developed at Simon Fraser University. Organised Sound 7 (1), (Doi: /S ) Webster, P. (2003). Asking music students to reflect on their creative work: Encouraging the revision process. In Yip, L. C. R., Leung, C. C. & Lau, W. T. (Eds.), Curriculum Innovation in Music. Hong Kong: The Hong Kong Institute of Education, Weisberg, R. W. (1993). Creativity: Beyond the Myth of Genius. New York, NY: W. H. Freeman. (ISBN: ) Wen-Chung, C. (1966). Open rather than bounded. Perspectives of New Music 5 (1), 1-6. Wenger, E. (2010). Communities of practice and social learning systems: the career of a concept. In C. Blackmore (Ed.), Social Learning Systems and Communities of Practice. London: Springer Verlag and the Open University. Weiser, M. (1991). The Computer for the Twenty-First Century, Scientific American 265(3), Windsor, W. L. (1995). A Perceptual Approach to the Description and Analysis of Acousmatic Music. Doctoral Thesis in Music, London: City University. Wishart, T. (2009). Computer music: Some reflections. In R. T. Dean (ed.), The Oxford Handbook of Computer Music (pp ). New York, NY: Oxford University Press. (ISBN: ) Xenakis, I. (1971/1992). Formalized Music: Thought and Mathematics in Composition. Hillsdale, NY: Pendragon Press. (ISBN: ) Zawacki, L. & Johann, M. (2012). A prospective analysis of analog audio recording with web servers. In Proceedings of the III Ubiquitous Music Workshop (III UbiMus). Vitória - ES - Brazil 68

81 From Digital Arts to Ubiquitous Music V Ubimus Prototyping of Ubiquitous Music Ecosystems Victor Lazzarini 1, Damián Keller 2, Carlos Kuhn 3, Marcelo Pimenta 3, Joseph Timoney 1 1 Sound and Music Research Group National University of Ireland, Maynooth Co. Kildare Ireland 2 Amazon Center for Music Research - NAP Universidade Federal do Acre - Federal University of Acre 3 Computer Science Department Universidade Federal do Rio Grande do Sul {victor.lazzarini,joseph.timoney}@nuim.ie, dkeller@ccrma.stanford.edu, {mpimenta,ckuhn}@inf.ufrgs.br Abstract. This paper focuses the prototyping stage of the design cycle of ubiquitous music (ubimus) ecosystems. We present three case studies of prototype deployments for creative musical activities. The first case exemplifies a ubimus system for synchronous musical interaction using a hybrid Java-JavaScript development platform, mow3s-ecolab. The second case study makes use of the HTML5 Web Audio library to implement a loop-based sequencer. The third prototype - an HTML-controlled sine-wave oscillator - provides an example of using the Chromium open-source sand-boxing technology Portable Native Client (PNaCl) platform for audio programming on the web. This new approach involved porting the Csound language and audio engine to the PNaCl web technology. The Csound PNaCl environment provides programming tools for ubiquitous audio applications that go beyond the HTML5 Web Audio framework. The limitations and advantages of the three approaches proposed - the hybrid Java/- JavaScript environment, the HTML5 audio library and the Csound PNaCl infrastructure - are discussed in the context of rapid prototyping of ubimus ecosystems. 1. Introduction Creativity-centered design of ubiquitous musical systems involves at least four developmental stages: defining strategies, planning, prototyping and assessment. This paper focuses on the third stage of the design cycle, prototyping. The first section shows related works in the field and the second places the activity of prototyping within the context of ubimus design. Then we present a case study focusing on the deployment of a ubimus system for synchronous musical interaction using a hybrid Java-JavaScript development platform based on browser technology. The second case involves the use of Web Audio in HTML5 to implement a loop-based sequencer. And the third case features a simple example of an HTML-controlled sine-wave oscillator using the Csound PNaCl programming environment. The final section provides a summary of the observations gathered 69 Vitória - ES - Brazil

82 V Ubimus From Digital Arts to Ubiquitous Music during the design of these three prototypes and discusses the limitations and advantages of each approach. 2. Related work In recent years, there has been some research (and commercial) work aiming to provide support for development of audio applications for mobile platforms like MobileSTK [Essl and Rohs 2006], based on STK and released in 2006, with support for Symbiam and Windows CE devices. This platform was also ported do ios in 2010 [Bryan et al. 2010] and incorporated in a toolkit called MOMU. Also from Essl [Essl 2010], we have Urmus, a LUA framework that is a multi-layered environment intended to support interface design, interaction design, interactive music performance and live patching on multi-touch mobile devices. Control [Roberts 2011] is an application that allows users to define custom graphic interfaces for MIDI and OSC. The interfaces are defined using web standards like HTML,CSS and Javascript. Roberts also is one of the creators of Gibber [Roberts et al. 2013], a language for live-coding in the browser. Gibber also has a 2D drawing API and event handlers for touch, mouse, and keyboard events, enabling fast prototyping. Since Gibber is centralized on a server, users can create collaborative programming sessions and publish compositions and instruments. 3. Designing ubiquitous music systems Defining design strategies for ubiquitous music encompasses two areas of expertise: interaction and signal processing. The Ubiquitous Music Group (g-ubimus) has been investigating the musical applications of methods based on human-computer interaction and ubiquitous computing techniques. Metaphors for interaction provide abstractions that encapsulate solutions applicable to a variety of activities without making unnecessary technical assumptions [Pimenta et al. 2012]. Thus, interaction metaphors materialize general ergonomic principles to fulfil the human and the technological demands of the activity [Keller et al. 2010] [Pimenta et al. 2012]. On a similar vein, recurring technological solutions can be grouped as interactions patterns [Flores et al. 2010]. These patterns are particularly useful when developers face the task of finding suitable strategies to deal with specific interface implementation issues. So far, our group s research has unveiled four musical interaction patterns: natural interaction, event sequencing, process control and mixing [Flores et al. 2012]. Each of these patterns tackles a specific interaction problem. Natural interaction deals with forms of musical interaction that are closely related to handling everyday objects. Event sequencing lets the user manipulate temporal information by freeing the musical events from their original time-line. Process control provides high-level abstractions of multiple parametric configurations, letting the user control complex processes by using simple actions. Mixing can be seen as the counterpart of event sequencing for synchronous interaction. Musical data - including control sequences and sound samples - is organized by user actions that occur in-time. Furthermore, technologically based musical environments also demand tailoring support for sound rendering. Signal processing techniques for creative musical activities have to be developed according to the characteristics of the tasks involved in the creative cycle, the computational resources provided by the support infrastructure and the profile of the target users. Ubiquitous musical activities may involve mobility, connectivity and coordination among heterogeneous devices with scarce computational resources. Thus, carefully chosen soft- Vitória - ES - Brazil 70

83 From Digital Arts to Ubiquitous Music V Ubimus ware design strategies are a prerequisite to tackle signal processing support in ubiquitous contexts [Lazzarini et al. 2012] [Lazzarini et al. 2014]. Ubiquitous-music planning studies involve early assessment of target population expectations and identification of opportunities for creativity support. Through a ubimus planning study, Lima and coauthors (2012)[Lima et al. 2012] found sharply differing expectations on technological usage by musicians and musically naive subjects in educational contexts. Based on these results, they proposed a simple rule of thumb: users like what comes closer to reenacting their previous musical experiences. Non-technical approaches, such as those proposed by traditional soundscape activities [Schafer 1977], may not be suited for introducing non-musicians to sonic composition. Naive subjects may respond better to technologically oriented approaches, as those found in ecologically grounded creative practices [Keller et al. 2014]. If the rule of thumb previously stated holds true, musically naive participants welcome easiness of use and naturality while musicians tend to prefer interfaces that foster behaviors based on acoustic-instrumental metaphors and common-practice music notation. Therefore, design of creatively oriented technologies needs to fulfil different demands depending on the intended user base. Technological support for pervasive musical activities increases the difficulty of the design task on two fronts. Ubimus systems may enhance the users creative potential by providing access to previously unavailable material and social resources. But a more intensive usage of resources can introduce unintended complexities, narrowing the access to a small user base. Thus, one challenge faced by ubimus designers is to provide intuitive tools for complex creative tasks. Furthermore, custom-made, special purpose hardware interfaces - such as those proposed by tangible user interface design approaches - may fill the requirements of transparency and naturality reducing the cognitive load of complex tasks. But they do not guarantee wide accessibility. In this case, the catch lies in the financial toll. Special purpose systems are difficult to distribute and maintain. As a consequence, the user base is narrowed by the increased costs of the hardware. Previous research indicated that another important difficulty faced by the designers of ubiquitous music tools is the slowness of the validation cycle [Keller et al. 2011a]. Because complete integrated systems are hard to design and test, tools usually deal with isolated aspects of musical activity. Musicians usage of the tools may not correspond to the intended design and integration of multiple elements may give rise to unforeseen problems. As a partial solution to these hurdles, the Ubiquitous Music Group has suggested the inclusion of music making within the development cycle. This integration of music making and software development is based on a broad approach to usability [Hornbaek 2006]. Fine-grained technical decisions are done after the usability requirements of the system have been well established through actual usage. So rapid deployment is prioritized over testing on a wide user base. Given the lack of standard support for audio and musical data formats, initial development of audio applications for mobile platforms was feasible but complex and unintuitive [Keller et al. 2010] [Radanovitsck et al. 2011]. Recent advances have paved the way to wider distribution of tools within the computer music community [Brinkmann 2012] [Lazzarini et al. 2012] [Lazzarini et al. 2014]. Within an iterative approach to design - involving creative musical activities and usability assessments - we have developed rapid prototyping techniques tailored for ubiquitous music contexts. Since our research targets both interaction and signal processing, flaws that 71 Vitória - ES - Brazil

84 V Ubimus From Digital Arts to Ubiquitous Music arise from coordination among these two processes can be identified early within the design cycle. Furthermore, full-blown creative musical activities uncover opportunities for creative exploration of the software AND of the local resources [Keller et al. 2013]. The prototypes reported in the second part of this paper provide examples of the advantages and limitations of an experimentally grounded approach to the development of ubiquitous music ecosystems. The last stage of the ubimus design cycle involves the assessment of creative processes and products, targeting the expansion of the creative potentials and the sustainable usage of resources. Although creativity assessment is an active area of research within psychology. [Amabile 1996] assessment of creative outcomes is still a taboo topic among music practitioners. From a product-centered perspective [Boulez 1986], creativity assessment would be equivalent to the measurement of musical value. This approach makes two assumptions. First, the objective of musical activity is to obtain a product that can be labelled as an expression of eminent creativity. Second, the value of the musical product lies in its material constituents (the sounds or their symbolic representation, i.e., scores or recordings of performances). In this case, standards are defined by the adopted compositional technique. Given a technique-centered metric, deviations are seen as spurious, less valuable manifestations. Another problem of the product-centered approach is the overrated reliance on domain-specific expert judgement. When asked to evaluate musical products - as it is done using Amabile s (1996) Consensual Assessment Technique - experts apply socially accepted views on creativity. These views are the result of several years of musical training and experience with eminent forms of creativity. Given the different requirements of professional and non-professional participants [Lima et al. 2012], this bias may render their assessment less useful to everyday-creativity manifestations. To avoid these pitfalls, ubiquitous music projects rely on a mix of assessment techniques [Keller et al. 2011b] engaging a small numbers of expert and untrained subjects in different musical activities in a variety of environmental conditions. This is usually described as triangulation within the behavioral research literature. This approach does not make assumptions regarding the compositional techniques, giving the same weight to musicians and lay-people s feedback. Data is extracted from the emerging relationships among the user profiles, the activities, the environmental conditions and the support infrastructure. 4. Prototyping platforms for ubiquitous music During the initial phase of ubiquitous music research ( ), the need of a short development cycle for ubimus infrastructure was faced with multiple obstacles. On one hand, web deployment featured little or no support for audio prototyping. Java and Adobe Flash were the two languages that provided more extensive resources for audio applications [Keller et al. 2011a] [Miletto et al. 2011]. While Java was supported on several mobile platforms, such as JavaME and Android, Adobe Flash was not always available on mobile devices. Hybrid approaches to ubimus system development were introduced involving the use of Javascript and Java-based synthesis engines [Keller et al. 2011b]. An example of this proposal is the ubimus prototyping environment mow3s-ecolab. We describe a case example of the use of mow3s-ecolab in ubiquitous musical activities: the Harpix 1.0 study. More recently, the development of HTML led to the introduction of audio-oriented web tools. HTML5 features Web Audio and Web Midi JavaScript-based technologies Vitória - ES - Brazil 72

85 From Digital Arts to Ubiquitous Music V Ubimus intended as standards for web deployment. Through the implementation of a ubiquitous music application, we explore some of the potentials and constraints of the Web Audio platform. We describe the development of the LCM Sequencer HTML5, a prototype for the creation of loop-based musical patterns. The design of this ubimus prototype illuminates aspects of the interaction demands of the sequencing activity and highlights the need for accurate timing support for synchronous usage. As extensively discussed in a recent survey by Wyse and Subramanian (2013)[Wyse and Subramanian 2013], the web browser is now a viable platform for the deployment of music computing applications. Three technologies are dominant in audio development for world-wide web applications: Java, Adobe Flash, and HTML5 Web Audio. Applications based on Java can be rendered either as Applets or via Java Web Start. Adobe Flash has grown in support by multiple browser vendors across various operating systems. Flash applications can be deployed as browser plug-ins, as well as through Adobe Air6. The HTML5 Web Audio framework for Javascript is the newest of these three technologies. Unlike the others, it is a proposed standard that is designed to be implemented by the browser vendors. Web Audio is today possibly the most popular toolkit for audio development on the web. However it has a number of limitations. Firstly, its set of audio operations is somewhat limited. Its functionality can be extended by Javascript code, which still pays a significant performance penalty if compared to natively-compiled C/C++ code. Although Javascript engines are constantly improving in speed and efficiency, running audio code entirely in Javascript is a processor intensive task on modern systems. However, the worst limitation is that the ScriptProcessorNode which is used to extend the API runs on the main thread. This can result in dropouts when another process on the main thread, for instance the user interface, interrupts or blocks the audio processing. This severely limits what is possible to do with Web Audio in practical terms to the built-in processing nodes. Consequently, we need to look for a technology that allows native applications to do audio processing beyond what it is possible with Javascript and Web Audio. An alternative is provided by the Native Clients (NaCl) platform. The next three sections present examples of the three approaches just discussed: a prototype using a hybrid Java-JavaScript support system - mow3s-ecolab; an HTML5- based prototype using the Web Audio library; and a sine-wave oscillator exemplifying the usage of the Csound PNaCl programming environment. Each example highlights key requirements of the support for creativity-centered ubiquitous music design involving both musical interaction and audio processing capabilities. 5. Case study: Harpix Interaction patterns and metaphors The first prototype - Harpix exemplifies the use of the spatial tagging [Keller et al. 2011b]. Spatial tagging is defined as an interaction metaphor that makes use of virtual or material visual cues - anchors - to support creative activity (fig. 1). Anchors provide a bridge between material and cognitive resources, enhancing the creative potential. This approach to the design of ubiquitous music systems has found support in multiple experiments with musicians and non-musicians applying a closely related interaction metaphor: time tagging [Keller et al. 2010] [Keller et al. 2013] 73 Vitória - ES - Brazil

86 V Ubimus From Digital Arts to Ubiquitous Music [Pinheiro da Silva et al. 2012] [Pinheiro da Silva et al. 2013] [Radanovitsck et al. 2011] [Pimenta et al. 2012]. Alternatively, Harpix 1.0 can be described as an instantiation of the natural interaction pattern [Flores et al. 2012]. The visual elements of the interface - or anchors - can be manipulated directly, establishing a straightforward relationship between user actions and sound events. This section summarizes the results of an experimental study reported in [Keller et al. 2011b] Materials and procedures MOW3S is a set of tools for multiplatform interface design specifically targeted for web usage. Given the adoption of the standard HTML syntax, MOW3S can be combined with any tool implemented in Javascript. User actions are tracked to generate control data formatted as standard MIDI events which are used to drive the synthesis engine Ecolab. Ecolab is a wavetable synthesis engine implemented in Java. It features support for network connections through standard IP sockets. By adopting DLS and General MIDI standards consistent sonic renditions can be achieved without the need for real-time streaming of audio. Thus, Ecolab can be used as a multiplatform back-end for desktop systems with low computational resources. Using the mow3s-ecolab environment, Keller and coauthors [Keller et al. 2011b] implemented a prototype based on the spatial tagging metaphor: Harpix 1.0. The Harpix architecture comprises three layers. On the first layer, user interaction is done through text input, mouse position tracking and mouse-wheel movement tracking. The second layer features spatial anchors represented by multiple draggable rectangles distributed on the browser pane. This layer provides synthesis parameter mappings linked to the positions of the anchors on the horizontal and vertical axes. The third layer deals with data routing and sound synthesis. Figure 1. The spatial tagging metaphor in Harpix 1.0. Three subjects realized 37 interaction essays, comprising multiple conditions (see table 1). The experimenters applied the CSI-NAP v.01 protocol to assess the level of support of the Harpix system for creative musical activities (in a range of 0-10), focusing on six creativity support factors: productivity, expressiveness, explorability, concentration, enjoyment and collaboration. Enjoyment was high during creative (9.5 ± 1.08) and Vitória - ES - Brazil 74

87 From Digital Arts to Ubiquitous Music V Ubimus exploratory activities (8.42 ± 1.78). Expressiveness was also highly rated in creative activities (9.10±0.99). On the other hand, collaboration was poorly judged in all conditions (5.95 ± 2.84). Activity/Participants solo duo trio i creative sessions exploratory sessions imitative sessions i Table 1. Matrix of experimental conditions. N=3,i=37 productivity expressiveness explorability concentration enjoyment collaboration mean std. dev Table 2. Matrix of experimental conditions. 6. Case study 2: LCM Sequencer HTML Interaction patterns and metaphors The second prototype provides an example of the application of the event sequencing interaction pattern (figure 2). Multiple loop-based sequences are controlled through a two-tier grid interaction metaphor. On the first tier - selected by clicking the pattern option - each line is assigned a timbre. Columns provide a visual representation of the temporal distribution of the sound events. Color cells indicate events and black cells stand for pauses. Sequence playback is controlled through three GUI elements: the start button, the stop button and the tempo, set as beats-per-minute units. Sound events are rendered through callbacks to the Web Audio synthesis engine. The second tier - available by choosing the song option - provides a preset mechanism. A drag-and-drop mechanism supports direct manipulation of the presets orderings. Up to five presets can be sequenced using the two preset-cell rows Discussion of results One of the caveats encountered during the preliminary design cycle was the imprecision of the Javascript timer. To circumvent this limitation we resorted to the use of the settimeout method, implementing a pooling system with higher resolution for event scheduling. This worked well for stationary platforms but presents occasional problems when running on the Android operating system. The Web Audio timer was accurate at speeds close to 500 BPM. Audio clicks were observed at higher speeds. Through informal usage testing, we observed that the drag-and-drop preset feature provides an effective shortcut for quick comparisons among multiple complex sequences. This is particularly advantageous when compared to sequencers that are operated through buttons. Rather than manipulating numeric values, the user has direct access to the reordering operations. Given that the temporal order of the sequences is correlated to the spatial order of the GUI elements, the anchoring cognitive mechanism furnishes grounding for this interaction metaphor [Keller et al. 2010]. 75 Vitória - ES - Brazil

88 V Ubimus From Digital Arts to Ubiquitous Music Figure 2. Two-tier grid interaction metaphor for event sequencing: LCM Sequencer HML5. 7. Case study 3: sine-wave oscillator in Csound PNaCl 7.1. Interaction patterns and metaphors The sine-wave oscillator demonstrates a minimal application of the functionality provided by the Csound PNaCl module. Given the didactic objective of the code, we focused on the use of one controller - represented by an HTML element - and one synthesis parameter - the oscillator s frequency. This is one of the simplest uses of the process control interaction pattern Materials and procedures The Native Clients (NaCl) platform 1 allows the use of C and C++ code to create components that are accessible to client-side Javascript, and run natively inside the browser. NaCl is described as a sandboxing technology, as it provides a safe environment for code to be executed, in an OS-independent manner [Yee et al. 2009] [Sehr et al. 2010]. The Portable NaCl [Donovan et al. 2010] toolchain, used to implement Csound in this case study, is completely independent of any existing architecture, and thus it is available for a variety of systems. However, PNaCl is currently only currently supported by the Chrome and Chromium browsers (under most operating systems, the ios and Android versions do not yet support it). Since version 31, Chrome enables PNaCl by default, allowing applications created with that technology to work completely out-ofthe-box. PNaCl modules can be served from anywhere in the open web. 1 Vitória - ES - Brazil 76

89 From Digital Arts to Ubiquitous Music V Ubimus The port of the Csound language to the PNaCl platform is complete, apart from its plugin opcodes, which are not available due to non-existence of dynamic loading here. It allows for realtime audio input and output, and it contains a complete Javascript interface that is used to control it. MIDI can be used in the form of MIDI files or through the Javascript implementation (WebMIDI) Prototype Example The following script demonstrates a minimal application using the functionality provided by the Csound PNaCl module 2. It consists of the implementation of the moduledidload() callback, where the Csound engine is started (with csound.play() and a simple Csound code is compiled with csound.compileorc(). This will produce a sine wave whose frequency can be controlled by changing the value of the HTML element with id freq: function moduledidload() { csound.play(); csound.compileorc( "schedule 1,0,-1 \n" + "instr 1 \n" + "kfr chnget \"freq\" \n" + "a1 oscili 0.5,kfr \n" + "outs a1,a1 \n" + "endin"); SetFreq(); } function attachlisteners() { document.getelementbyid("freq"). addeventlistener("change",setfreq); } function SetFreq(){ var val = document.getelementbyid("freq").value; csound.setchannel("freq", val); } 8. Discussion of results To test the application of the spatial tagging metaphor, our team implemented a prototype based on Java and Javascript browser technology to support creative musical activities: Harpix 1.0. Harpix was used in an experiment encompassing three types of musical activities by three subjects. The assessment of creativity support indicated a high performance in the creative and exploratory activities, with particular emphasis on two factors: enjoyment and expressiveness. However, the collaboration and explorability factors were not evaluated positively. Imitative activities also yielded low scores Vitória - ES - Brazil

90 V Ubimus From Digital Arts to Ubiquitous Music A second prototype used the HTML5 Web Audio library to support the application of the event sequencing interaction pattern. A two-tier interaction metaphor was developed for synchronous manipulation of complex-sequence orderings. The presetcells drag-and-drop mechanism showed good potential during preliminary testing, hinting at a common ground for this interaction metaphor, time tagging and spatial tagging [Keller et al. 2010] [Keller et al. 2011b]. The Javascript timer did not perform well. The Web Audio timer performed better, but usage at high metronome speeds produced clicks. The third prototype explored the facilities provided by the Csound PNaCl environment. We introduced the use of PNaCl for audio programming through a port the Csound language and audio engine. As one of the simplest uses of the process control interaction pattern, the prototype sine-wave HTML-controlled oscillator provided an opportunity to demonstrate the advantages and limitations of this new open-source sand-boxing technology developed as part of the Chromium project. The fully functional implementation of the Csound PNaCl environment features a mid-latency callback mechanism (ca ms, 512 frames at 44.1 or 48 KHz sampling rate) with uniform performance across various platforms. The Audio API design is very straightforward, but it only supports stereo output at one of the two sampling rates just mentioned. The technologies employed in the development the three prototypes reported in this paper showed different types of limitations for audio programming and interaction support. On one hand, browser-based prototyping, as introduced by the mow3s-ecolab environment, provides a flexible way to deploy and test interaction metaphors. Standard libraries, such as the HTML5 Web Audio and Web Midi, have good potential for wide adoption but currently present design problems that limit their usage in synchronous activities and audio programming tasks. At this point, they are better suited for asynchronous support. We implemented a new set of technologies for audio programming for web applications. The Csound PNaCl environment features a relatively low-latency performance and incorporates the know-how developed over 30 years of Csound usage, providing a path for the development of ubiquitous music ecosystems that goes beyond the HTML5 Web Audio framework. References Amabile, T. (1996). Creativity in Context. Boulder, CO: Westview Press. Boulez, P. (1986). Orientations: Collected Writings. London, UK: Faber and Faber. Brinkmann, P. (2012). Making Musical Apps: Using the Libpd Sound Engine. O Reilly & Associates Incorporated. Bryan, N. J., Herrera, J., Oh, J., and Wang, G. (2010). Momu: A mobile music toolkit. In Proceedings of NIME Donovan, A., Muth, R., Chen, B., and Sehr, D. (2010). PNaCl: Portable Native Client Executables. Google White Paper. Essl, G. (2010). Urmus: an environment for mobile instrument design and performance. In Proceedings of ICMC Essl, G. and Rohs, M. (2006). Mobile stk for symbian os. In Proceedings of ICMC Vitória - ES - Brazil 78

91 From Digital Arts to Ubiquitous Music V Ubimus Flores, L., Miletto, E., Pimenta, M., Miranda, E., and Keller, D. (2010). Musical interaction patterns: Communicating computer music knowledge in a multidisciplinary project. In Proceedings of the 28th ACM International Conference on Design of Communication, SIGDOC 10, pages , New York, NY, USA. ACM. Flores, L. V., Pimenta, M. S., and Keller, D. (2012). Patterns of musical interaction with computing devices. In Proceedings of the III Ubiquitous Music Workshop (III UbiMus), São Paulo, SP, Brazil. Ubiquitous Music Group (g-ubimus), São Paulo, SP, Brazil: Ubiquitous Music Group. Hornbaek, K. (2006). Current practice in measuring usability: Challenges to usability studies and research. Internation Journal of Human-Computer Studies, 64(2): Keller, D., Barreiro, D. L., Queiroz, M., and Pimenta, M. S. (2010). Anchoring in ubiquitous musical activities. In Proceedings of the International Computer Music Conference, pages , Ann Arbor, MI: MPublishing, University of Michigan Library. Ann Arbor, MI: MPublishing, University of Michigan Library. Keller, D., Ferreira da Silva, E., Pinheiro da Silva, F., Lima, M. H., Pimenta, M. S., and Lazzarini, V. (2013). Everyday musical creativity: An exploratory study with vocal percussion (criatividade musicalccotidiana: Um estudo exploratório com sons vocais percussivos). In Proceedings of the National Association of Music Research and Post- Graduation Congress - ANPPOM (Anais do Congresso da Associação Nacional de Pesquisa e Pós-Graduação em Música - ANPPOM). Natal, RN: ANPPOM. Keller, D., Flores, L. V., Pimenta, M. S., Capasso, A., and Tinajero, P. (2011a). Convergent trends toward ubiquitous music. Journal of New Music Research, 40(3): Keller, D., Lima, M. H., Pimenta, M. S., and Queiroz, M. (2011b). Assessing musical creativity: material, procedural and contextual dimensions. In Proceedings of the National Association of Music Research and Post-Graduation Congress - ANPPOM (Anais do Congresso da Associação Nacional de Pesquisa e Pós-Graduação em Música - ANPPOM), pages , Uberlândia, MG: ANPPOM. National Association of Music Research and Post-Graduation (ANPPOM), Uberlândia, MG: ANPPOM. Keller, D., Otero, N., Pimenta, M. S., Lima, M. H., Johann, M., Costalonga, L., and Lazzarini, V. (2014). Relational properties in interaction aesthetics: The ubiquitous music turn. In Proceedings of the Electronic Visualisation and the Arts Conference (EVA-London 2014). London: Computer Arts Society Specialist Group. Lazzarini, V., Costello, E., Yi, S., and Fitch, J. (2014). Csound on the web. In Proceedings of the Linux Audio Conference (LAC2014). Lazzarini, V., Yi, S., Timoney, J., Keller, D., and Pimenta, M. S. (2012). The mobile csound platform. In Proceedings of the International Computer Music Conference, pages , Ljubljana. ICMA, Ann Arbor, MI: MPublishing, University of Michigan Library. Lima, M. H., Keller, D., Pimenta, M. S., Lazzarini, V., and Miletto, E. M. (2012). Creativity-centred design for ubiquitous musical activities: Two case studies. Journal of Music, Technology and Education, 5(2): Vitória - ES - Brazil

92 V Ubimus From Digital Arts to Ubiquitous Music Miletto, E. M., Pimenta, M. S., Bouchet, F., Sansonnet, J.-P., and Keller, D. (2011). Principles for music creation by novices in networked music environments. Journal of New Music Research, 40(3): Pimenta, M. S., Miletto, E. M., Keller, D., and Flores, L. V. (2012). Technological support for online communities focusing on music creation: Adopting collaboration, flexibility and multiculturality from Brazilian creativity styles, volume Cases on Web 2.0 in Developing Countries: Studies on Implementation, Application and Use, chapter 11. Vancouver, BC: IGI Global Press. Pinheiro da Silva, F., Keller, D., Ferreira da Silva, E., Pimenta, M. S., and Lazzarini, V. (2013). Everyday musical creativity: exploratory study of ubiquitous musical activities (criatividade musical cotidiana: estudo exploratório de atividades musicais ubíquas). Música Hodie, 13: Pinheiro da Silva, F., Pimenta, M. S., Lazzarini, V., and Keller, D. (2012). Time tagging in its niche: Engagement, explorability and creative attention (a marcação temporal no seu nicho: Engajamento, explorabilidade e atenção criativa). In Proceedings of the III Ubiquitous Music Workshop (III UbiMus), São Paulo, SP, Brazil. Ubiquitous Music Group (g-ubimus), São Paulo, SP: Ubiquitous Music Group. Radanovitsck, E. A. A., Keller, D., Flores, L. V., Pimenta, M. S., and Queiroz, M. (2011). mixdroid: Time tagging for creative activities (mixdroid: Marcação temporal para atividades criativas). In Proceedings of the XIII Brazilian Symposium on Computer Music (SBCM), Vitória, ES: SBC. Vitória, ES: SBC. Roberts, C. (2011). Control: Software for End-User Interface Programming and Interactive Performance. In Proceedings of the ICMC 2011, Huddersfield, UK. Roberts, C., Wakefield, G., and Wright, M. (2013). The Web Browser As Synthesizer And Interface. In Proceedings of the International Conference on New Interfaces for Musical Expression. Schafer, R. M. (1977). The Tuning of the World. New York, NY: Knopf. Sehr, D., Muth, R., Biffe, C., Khimenko, V., Pasko, E., Schimpf, K., Yee, B., and Chen, B. (2010). Adapting Software Fault Isolation to Contemporary CPU Architectures. In 19th USENIX Security Symposium. Wyse, L. and Subramanian, S. (2013). The Viability of the Web Browser as a Computer Music Platform. Computer Music Journal, 37(4): Yee, B., Sehr, D., Dardyk, G., Chen, J. B., Muth, R., Ormandy, T., Okasaka, S., Narula, N., and Fullagar, N. (2009). Native Client: A Sandbox for Portable, Untrusted x86 Native Code. In 2009 IEEE Symposium on Security and Privacy. Vitória - ES - Brazil 80

93 From Digital Arts to Ubiquitous Music V Ubimus Ubiquitous Computing meets Ubiquitous Music Flávio L. Schiavoni 1, Leandro Costalonga 2 1 Deparamento de Computação Universidade Federal de São João Del Rei (UFSJ) Av. Visconde do Rio Preto, s/nº, CEP , São João Del Rei MG Brazil 2 CEUNES - Universidade Federal do Espírito Santo (UFES), Rodovia BR 101 Norte, Km. 60, Bairro Litorâneo, CEP , São Mateus ES fls@ufsj.edu.br, leandro.costalong@ufes.br 1. Extended Abstract Ubiquitous computing (Ubicomp) is computing everywhere, anywhere [Langheinrich 2001] anytime [Coroama et al. 2004] and also computing in anything and everything [Greenfield 2006]. Is is also called Invisible computing [Borriello 2008], Pervasive computing [Satyanarayanan 2001], Everyday computing [Abowd and Mynatt 2000] among others. Despite the different names, Ubicomp is a way to see computers where several devices typically have to work together to perform a particular task creating smart environments [Coroama et al. 2004] or intelligent environments [Brumitt et al. 2000]. Nowadays (2014) music devices are ubiquitous in daily life. Increasingly, we are seeing computational systems incorporating sensors such as microphones and headphones outputs [Bellotti and Sellen 1993] and transforming several daily devices into ubiquitous music devices. The popularity of Ubicomp with the evolution of musical devices guided this concept to arts in a field called Ubiquitous Music (Ubimus). Ubimus concepts and motivations, defined by Keller [Keller et al. 2009], includes to merge sound sources and music interfaces with the environment in a ubiquitous form. Previous research and efforts from the Ubiquitous Music Group included several discussion involving Collective creation [Ferraz and Keller 2014], Interaction aesthetics [Keller et al. 2014], Methodology for creativity centered software design [Lima et al. 2012], Open issues in current musical practices [Keller et al. 2011] and other relevant aspects of social and musical dimensions. Beyond the musical and social discussion in Ubimus, we believe that computer scientists can also take part of this research field once we can find a clear way to contribute with Ubimus researches. Regarding the technological point of view, Ubimus, like Ubicomp, is not a particular research field in Computer Science. Ubimus merge Ubicomp research field with fields defined in Sound and Music Computing and / or Computer Music. Computer Music and Sound and Music Computing involves several subjects on Computer Science field namely: Music Information Retrieval (MIR), Sonic Interaction Design, Mobile Music Computing, Live Coding, Networked Music Performance, Human Computer musical interaction, New Interface for Music Expression (NIME), Digital Audio Effects, Languages for Computer Music and more 81 Vitória - ES - Brazil

94 V Ubimus From Digital Arts to Ubiquitous Music The possibility of doing all these Computer Music activities on mobile devices for computing (like mobile phones, tablets and netbooks) [Lazzarini and Yi 2012] can emerge as the Ubimus research field to computer scientists. To try to map a computer science research field in Ubimus is not an easy task. It is possible to fall into a gap between technological possibilities and our ability to put them to good useful. For this reason, maybe it is completely useless to develop Ubimus hardware or software without some partnership with musicians, composers and artists, without focusing that our main goal is not to develop technology to technology but to music. The partnership can help a scientist to find a human need, a need of expression, a impossible musical creation that only this kind of devices can bring to real. Most part of Ubimus issues should not be solved in technological field but in social fields. Nonetheless, without the support of computer guys is not easy to social scientists to break the barriers of technology and create new concepts using technology. Keller [Keller et al. 2010] describes Ubimus as An Uncharted Territory. Music researchers are doing their work. Maybe it is time to start exploring this territory quirks in computer science field. References Abowd, G. D. and Mynatt, E. D. (2000). Charting past, present, and future research in ubiquitous computing. ACM Trans. Comput. Hum. Interact., 7(1): Bellotti, V. and Sellen, A. (1993). Design for privacy in ubiquitous computing environments. In Proceedings of the Third Conference on European Conference on Computer Supported Cooperative Work, ECSCW 93, pages 77 92, Norwell, MA, USA. Kluwer Academic Publishers. Borriello, G. (2008). Invisible computing: automatically using the many bits of data we create. Philosophical Transactions of the Royal Society A: Mathematical, Physical and Engineering Sciences, 366(1881): Brumitt, B., Krumm, J., Meyers, B., and Shafer, S. (2000). Ubiquitous computing and the role of geometry. Personal Communications, IEEE, 7(5): Coroama, V., Bohn, J., and Mattern, F. (2004). Living in a smart environment implications for the coming ubiquitous information society. In Systems, Man and Cybernetics, 2004 IEEE International Conference on, volume 6, pages vol.6. Ferraz, S. and Keller, D. (2014). Mdf: Proposta preliminar do modelo dentro fora de criação coletiva. Cadernos de Informática, 8(2): Greenfield, A. (2006). Everyware: The Dawning Age of Ubiquitous Computing. Peachpit Press, Berkeley, CA, USA. Keller, D., Barreiro, D. L., Queiroz, M., and Pimenta, M. S. (2010). Anchoring in ubiquitous musical activities. In Proceedings of the International Computer Music Conference, pages Ann Arbor, MI: MPublishing. Keller, D., Barros, A., Farias, F., Nascimento, R., Pimenta, M., Flores, L., Miletto, E., Radanovitsck, E., Serafini, R., and Barraza, J. (2009). Música ubíqua: conceito e Vitória - ES - Brazil 82

95 From Digital Arts to Ubiquitous Music V Ubimus motivação. Proceedings of the ANPPOM, Curitiba, PR: Associação Nacional de Pesquisa e Pós Graduação em Música, pages Keller, D., Flores, L. V., Pimenta, M. S., Capasso, A., and Tinajero, P. (2011). Convergent trends toward ubiquitous music. Journal of New Music Research, 40(3): Keller, D., Otero, N., Lazzarini, V., Pimenta, M. S., do Sul, R. G., de Lima, M. H., Johann, M., and Costalonga, L. (2014). Relational properties in interaction aesthetics: The ubiquitous music turn. Langheinrich, M. (2001). Privacy by design principles of privacy aware ubiquitous systems. In Proceedings of the 3rd International Conference on Ubiquitous Computing, UbiComp 01, pages , London, UK, UK. Springer Verlag. Lazzarini, V. and Yi, S. (2012). Csound for android. In Proceedings of Linux Audio Conference, Stanford CA USA. LAD. Lima, M. H., Keller, D., Pimenta, M. S., Lazzarini, V., and Miletto, E. M. (2012). Creativity centred design for ubiquitous musical activities: Two case studies. pages Satyanarayanan, M. (2001). Pervasive computing: vision and challenges. Personal Communications, IEEE, 8(4): Vitória - ES - Brazil

96 V Ubimus From Digital Arts to Ubiquitous Music Progressive Disclosure Antonio D'Amato Conservatorio Statale di Musica di Avellino IT antonio089@yahoo.it Abstract. Progressive Disclosure is a short piece where in an imaginary landscape an unknown machine is progressively disclosed and explained in order to revel its inner functions. The piece is an reflection on concepts of approach modalities and comprehension of properties or qualities, that an object possesses and its functions. Long slow sound objects and impulsive sounds build up the piece. These elements are merged and extensively overlapped in order to develop an imaginary panorama with basic elements of a music vocabulary. Synthesized and acoustically derived sounds are both used, but the focus here is mainly on the description of a progressively closer observation of a visionary machine. link to the audio file (both HQ wav and mp3): 4moBAfHNia. Vitória - ES - Brazil 84

97 From Digital Arts to Ubiquitous Music V Ubimus Computação Ubíqua e a interação corporal na aprendizagem de execução rítmica Thiago Marcondes Santos 1, Denise Filippo 2, Mariano Pimentel 3 1,3 PPGI-Programa de Pós Graduação em Informática Universidade Federal do Estado do Rio de Janeiro (UniRio) Av. Pasteur 458, Térreo, Urca, , Rio de Janeiro, RJ, Brasil. 2 Escola Superior de Desenho Industrial (Esdi) - Universidade do Estado do Rio de Janeiro (Uerj) R. Evaristo da Veiga, 95, Centro, , Rio de Janeiro, RJ, Brasil thiago.marcondes@uniriotec.br,dfilippo@esdi.uerj.br, pimentel@unriotec.br Abstract.This article describes an investigation in the use of ubiquitous computing in the context of a primary school classroom with the aim of promoting the learning and experiencing of rhythmic concepts. The school environment was transformed into a sound laboratory, where students participated in different types of interactions. Building upon physical contacts that can be easily executed and with which students were already familiar, such as clapping hands, the classroom was presented as a collective musical instrument, which lessened the technical barriers necessary to musical execution. 1. Introdução A aprendizagem técnica para executar os instrumentos musicais tradicionais é um processo árduo e prolongado e, por isto, quando o estudante não tem a habilidade necessária para produzir notas ou elementos que estão presentes no discurso musical, ele se concentra mais em como gerar o som e menos em como os sons se combinam para construir o discurso. Educadores como Dalcroze (1921) defendiam o uso de diferentes formas de se aprender música, como o uso da percussão corporal e dos gestos, para diminuir as dificuldades dos estudantes. Neste trabalho foi investigada uma proposta para as dificuldades técnicas/instrumentais dos estudantes: a utilização de pisos sonoros em sala de aula com o intuito de facilitar o acesso aos sons de diferentes instrumentos gerenciados no computador. 2.O dispositivo O dispositivo proposto é composto de um software (SoundPlant), um hardware (computador e interface Makey Makey) e um piso de placas EVA. A produção do som ocorre quando os 2 estudantes, cada um em pé em cada placa, encostam um no corpo do outro. O som produzido é associado às placas através de sua configuração no software. 3. Estudo de caso Foi realizado um estudo de caso exploratório com estudantes do 5º ano do ensino fundamental. Os dados coletados para este estudo de caso foram questionário, grupo focal e observação direta. A atividade foi gravada por 2 câmeras. Seis estudantes foram convidados pelo professor de música da turma para participar da atividade: 1 aula de música com 40 minutos de duração e com o apoio do dispositivo proposto. A aula teve 85 Vitória - ES - Brazil

98 V Ubimus From Digital Arts to Ubiquitous Music diferentes etapas: abordagem conceitual sobre pulso e suas divisões; escuta de música para perceber diferentes sons e ritmos da bateria em uma canção sugerida pelo professor; execução dos ritmos com palmas dos estudantes; apresentação e exploração do ASU; execução dos ritmos anteriormente analisados e improvisações com o ASU. Na etapa apresentação e exploração do ASU, 6 estudantes foram organizados em 3 duplas de forma a obter os 3 sons de bateria associados via SoundPlant às placas EVA.Os estudantes também experimentaram tocar as mãos nos braços e pernas dos companheiros para produzir sons e se divertirem nesse processo.em seguida, na etapa de execução dos ritmos, as mesmas 3 linhas rítmicas da canção estudada, que haviam sido praticadas anteriormente apenas com suas palmas individuais,foram então executadas por meio do dispositivo. Neste processo, os estudantes puderam executar e ouvir a música estudada com uma bateria sendo tocada a 12 mãos.em seguida, foi solicitado aos estudantes que criassem livremente ritmos e interagissem entre si. 4. Resultados Os dados do questionário indicaram que todos os 6 estudantes aprendem música unicamente através da escola pública e nenhum deles tinham instrumentos musicais em casa. Também indicaram ausência de locais, parentes ou amigos que oferecessem a iniciação musical fora da escola. O questionário também indicou que a atividade foi considerada muito agradável por todos os estudantes. Todos também responderam no questionário que tinham interesse em fazer outra aula com o dispositivo ASU. A atividade foi percebida como algo lúdico e de fácil acesso e utilização. Dois alunos, um de 14 e outro de 11anos, respondiam as perguntas mais rapidamente, mas todos os 6 alunos entenderam e souberam operar o ASU com facilidade e rapidez.dois estudantes que tinham que executar uma linha rítmica mais rápida tiveram dificuldade de inserir sua parte junto aos ritmos dos demais colegas. Contudo, através da colaboração e de muita comunicação eles foram melhorando seu ritmo. 5. Conclusão Esse trabalho apresentou uma proposta de atividade de educação musical baseada num dispositivo que mostra como o uso de novas tecnologias computacionais possibilita o ensino da música por meio de artefatos que não demandam o prolongado aprendizado técnico dos instrumentos musicais tradicionais. O estudo de caso realizado mostra indícios de que o dispositivo pode ser usado como uma alternativa de instrumento não excludente aos instrumentos musicais tradicionais na aula de música. Referências Dalcroze, E.J. (1921) Rhythm music and Education.G.P.Putnam s Sons New York. Volpe, G., Varni, G., Mazzarino, B., Addessi, Anna.(2012)BeSound: Embodied Reflexion for Music Education in Childhood. IDC 2012 SHORT PAPERS 12th-15th June, Bremen, Germany Weiser, M. (1991)The computer for the twenty-first century. Scientific American, 65(3): Zhou,Y., Percival,G, Wang,X., Wang., Zhao,S. (2011) - MOGCLASS: Evaluation of a Collaborative System of Mobile Devices for Classroom Music Education of Young ChildrenSchool of Computing (SoC), National University of Singapore CHI-2011 Vitória - ES - Brazil 86

99 From Digital Arts to Ubiquitous Music V Ubimus Balance: um estudo sobre a tradução digital do corpo em equilíbrio Pablo Gobira Escola Guignard Universidade do Estado de Minas Gerais (UEMG) Belo Horizonte MG Brasil, pablo.o.gobira@gmail.com Raphael Prota Escola Guignard Universidade do Estado de Minas Gerais (UEMG) Belo Horizonte MG Brasil, raphael.eu@gmail.com Ítalo Travenzoli Escola de Belas Artes Universidade Federal de Minas Gerais (UFMG) Belo Horizonte MG Brasil, italotravenzoli@gmail.com Palavras-chave: interface, som e imagem, arte, corpo-mente, equilíbrio Resumo Este trabalho apresenta o desenvolvimento de uma instalação interativa experimental que utiliza um sensor de movimento 3D para captar o movimento corporal de uma pessoa se equilibrando em uma fita de slackline para criar gráficos e sons a serem projetados no espaço expositivo. Essa instalação busca mesclar os campos da música, video arte, tecnologia, jogo e performance em um dispositivo audiovisual que utiliza a dinâmica de manutenção do equilíbrio corporal como condição da composição gráfica e sonora. A proposta de ubiquidade está presente na inserção de sons e imagens projetados por fontes supostamente diversas do local onde o interator/performer se encontra para interação, acionados pela combinação dos processos voluntários e involuntários de manutenção do equilíbrio corporal como dados para as manifestações sonoras e visuais. A proposta da instalação é uma conexão extrema entre corpo e máquina através da dispersão provocada pela necessidade de equilíbrio do interator, provocando uma imersão no mundo da instalação que se pretende o mundo do experimento sobre o corpo-mente. Afirmamos, enquanto experimento artístico, que é nesse mundo que a produção do som está ubiquamente inserida. Especificamente sobre o áudio, os movimentos dos membros pés, mãos e cabeça são traduzidos, através de um código gerativo desenvolvido no openframeworks e PureData, em variações na frequência e filtros de ressonância de cinco ondas sonoras independentes criando uma atmosfera que oscila junto ao corpo como uma representação do equilíbrio. As respostas sonoras acompanham alterações de tamanho e subdivisões nas camadas sobrepostas de uma mandala. Pela tradução do esforço na busca do equilíbrio em estímulos audiovisuais e pela combinação dos fluxos sensoriais do controle postural com os estímulos audiovisuais externos, espera-se criar uma situação de imersão multitarefa que problematize a noção de controle da interface e revele novas percepções sobre a interação entre corpo e mente. Links Teste interface Teste interface Vitória - ES - Brazil

100 Vitória - ES - Brazil