emoTouch Web – A Web-Based System for Continuous Real Time Studies With Smartphones, Tablets, and Computers

Music and its performance are dynamic phenomena that evolve over time. Therefore, research on musical processes needs suitable methods and tools that make these processes observable continuously in real time, rather than just collecting summarizing data at the end of a test session. The necessity for continuous response research approaches and tools has been stressed in music psychology in recent years (Kopiez, 2005; Kopiez et al., 2011; Schubert, 2010) and is basically similar to the situation in other research fields on time-bound phenomena, such as theatre, dance, film, commercials, lectures, speeches, or sports performances. Since the 1980s, several continuous response interfaces have been developed (see Kopiez et al., 2011). In 2013, our team at Osnabrück University published emoTouch for Apple iPad, the first continuous response tool on a standalone multitouch device (Louven & Scholle, 2015).

However, all these interfaces had a raving disadvantage since they required special hardware (sliders, knobs, PCs with mice, PDAs, tablets with specific apps, etc.) that a researcher first had to provide to all participants before a real-time study could be conducted. This imposes a significant limitation for various reasons, e.g. it is hardly possible to provide costly iPads or wired hardware sliders to hundreds of listeners in a live concert setting.

The new Web-Based Concept

The new emoTouch Web continuous response research system implements a completely new concept that solves these problems fundamentally (Louven et al., 2022).

The whole system is based on web and network technologies with an emoTouch server as central element that hosts the database and manages the information flow (Figure 1). Studies are designed, controlled und monitored in the Researcher Interface (RI), a web application that runs on any kind of modern device and that stores its data directly on the server. The RI also creates a specific QR Code with an URL that just has to be accessed in order to participate in a study. Whenever this URL is called, the server deploys the study to the calling device where it runs within the web browser (‘Participant Interface’, PI). The PI is completely hardware independent and works on any modern smartphone, tablet, laptop and desktop computer with an internet connection. The data that is generated by the participant’s action (slider movements etc.) is continuously recorded with their time stamps and directly stored on the emoTouch server.

Of course, emoTouch Web can be used to conduct continuous real-time studies in laboratory settings with desktop computers or to realize continuous response online studies across the internet. But the system particularly provides completely new possibilities for research at live events with potentially hundreds of participants: Subjects can easily participate in a study by using the smartphones they carry in their pockets anyway ('Bring-Your-Own-Device', BYOD), simply by scanning an emoTouch QR Code that is, e.g. printed out in the program booklet.

Research With emoTouch Web

emoTouch Web provides all necessary tools to design, conduct and analyse continuous real-time studies in laboratory, online and live settings.

The screen layout that will show up as PI on the participants’ devices is designed in the RI’s study editor (Figure 2). It offers numerous, fully customizable interface elements like one-dimensional horizontal and vertical sliders, 2D rating areas, category scales, checkboxes, buttons, images and text elements.

The number of interface elements on a screen layout is unlimited. Multiple scales, rating areas and buttons can even be used simultaneously on a multitouch device. All layouts will dynamically adapt to the different screen sizes and display ratios of the various participants’ devices. Beside of graphical elements, layouts can also contain audio and video files which will play directly from the participant’s device, which is especially useful for stimuli in laboratory settings or online studies. Since emoTouch Web studies are not limited to just one layout, but can consist of several ‘parts’ with different layouts, it is also possible to implement quite complex study workflows that require multiple, changing interfaces.

While a study is executed, the course and data flow can be controlled, monitored and observed in real time from the ‘realisation’ section of the Researcher Interface. For example, the researcher can actively trigger layout changes for all participating devices simultaneously if an appropriate layout needs to be shown in a changing live situation.

All data that is generated from the participating devices is stored immediately on the emoTouch server. The RI provides built-in coordinated tools for the graphical and numerical review, evaluation and analysis of this data in longitudinal and cross-section, e.g. timeline graphs with mean and STD, heat plots, descriptive statistics, or the calculation of interrater reliabilities, autocorrelations and Granger causalities. This also allows to review some aspects of the participants’ behaviour that are essential for assessing data quality in live situations, since subjects use their own devices voluntarily and autonomously and could, for example, briefly switch to a different application or browser tab during a study.

If the built-in tools don’t match the needs, the analysis can be extended by any other Python and JavaScript evaluation scripts. In addition, all data can also be exported in a variety of formats for further processing and analyses with different software tools.

emoTouch Web may be used free of charge for scientific purposes. See detailed information on https://www.emotouch.de