Accepted Entries - ECogS Seminars 2021-2022
List of Accepted Entries
Please, click the name for full author information and abstract
Cognitive Probatonics: what cognitive ethnography can teach us about problem-solving
Narrative self and habits
Social connection in a digital world: Phubbing as extended mind-wandering
The use of multimodal resources for joint meaning-making in conversational repair sequences
Growing a social brain
Exploring the association between mental health and interpersonal coordination.
2nd person approach to studying cooperative social interactions
'I interact therefore I am': Human becoming in and through social interaction
Music and Wellbeing in Ritual contexts: pushing the embodied cognitive science of social dynamics beyond WEIRD research
Hallucination Machine and Computational Neurophenomenology
Participatory Sense-Making and Sociomorphing: The Role of Embodied Artificial Systems in Social Interaction
Drumming humans obey similar collective dynamics as swarming and flocking animals
The role of emotions in binding us
A generalized framework for modelling human joint actions using dynamical perceptual motor primitives
Social Motion - On social prediction and decision making
Artificial empathy: an active inference approach to affective computing
The influence of sensory deficits and social cognition on spatial perspective-taking.
Stage Presence in Dance: A Cognitive Ecological Ethnographic Approach
Coupled – for better or worse? Exploring how coordination dynamics shape collective performance.
Objectification, shame, and the gaze of digital technology
- Sune Vork Steffensen
Abstract: In this talk I argue for the need of developing a theory of cognitive particulars (“probatonics”), that is, a radical idiographic approach to complement the nomothetic approach prevalent in cognitive science (Steffensen, 2016). The study of cognitive particulars is suited for generating an insight into the micro-ecological contingencies that shape actual behaviour, in line with Turvey’s observation that “for something to happen, […] circumstances must be added to laws” (Turvey, 1992:177). My starting point is broadly ecological/systemic, and I illustrate the approach with two examples from problem-solving psychology. The cases contribute to an understanding of affordances as “thick relations”, where micro-fluctuations reveal problem-solving affordances.
- Katsunori Miyahara
Abstract: I will present my work in progress on the hypothesis that narrative selfhood consists of habits. My case for this “Narrative Self as Habits hypothesis” draws on parallels between the ways in which habits and narrative selfhood shape human life. Both narrative selfhood and habits serve as principles of actions, are formed through repeated engagement with a practice, and inform perception. I propose to see this as indicating that narrative selfhood is constituted by a collection of habits.
- Jelle Bruineberg and Regina Fabri
Abstract: The omnipresence of smartphones has transformative effects on our individual and social lives. Paradoxically, the perpetual possibility of digital social connection appears to impede direct, attentive social interaction. One example of this phenomenon is phubbing: engaging absent-mindedly and against one’s intentions with one’s smartphone at the expense of engaging with others in the occurrent situational context (Aagaard, 2020; Chotpitayasunondh & Douglas, 2016). While recent empirical research has started to investigate the cognitive and affective characteristics of phubbing (e.g. Chotpitayasunondh & Douglas, 2018; Vanden Abeele et al., 2019), a conceptual framework for understanding the descriptive and normative dimensions of phubbing is still missing. To close this gap, we bring together research on the extended mind (Menary, 2010; Sutton, 2010) and mind-wandering (Irving & Thompson, 2018; Seli et al., 2018). In a first step, we propose to conceptualize phubbing as a form of extended mind-wandering that has the following characteristics: 1.) it is extended (characterised by a perceptual coupling relation between the agent and their smartphone); 2.) it is task-unrelated (unrelated to the task of social interaction); 3.) it is unguided (the focus of attention shifts and drifts spontaneously). In a second step, we will apply this framework to a comparison of the costs and benefits of extended and non-extended mind-wandering. In particular, while phubbing and non-extended forms of mind-wandering share “costs” in the sense that little attention is paid to direct social interaction, the benefits of phubbing and non-extended mind-wandering are not necessarily comparable. We will analyse both phubbing and non-extended mind-wandering along the following dimensions: affective valence, context-sensitivity, and meta-awareness. This functional comparison allows us to link the characterizations of phubbing with normative debates about how digital technologies transform our individual and social lives (Turkle, 2016).
- Marlou Rasenberg
Abstract: If interactional trouble arises in a social interaction, people coordinate their actions in an effort to restore intersubjective understanding. To this end, other-initiated repair can be employed; an addressee signals having trouble hearing or understanding a message (the repair initiation), thereby inviting the sender of that message to repair it (the repair solution). In this talk I will present a study in which we investigated the multimodal nature of such repair sequences. We quantified the use of speech and co-speech gestures in repair initiations and solutions, and analysed how people share the “multimodal workload”. Qualitative analyses revealed that particular types of gestures are used for specific purposes that are at the core of other-initiated repair as an interactional practice. Together, these findings show how both speech and gesture are coordinated to negotiate and achieve intersubjectivity in social interaction.
- Shir Atzil
Abstract: It has long been assumed that social animals, such as humans, are born with a brain system that has evolved to support social affiliation. However, the evidence does not necessarily support this assumption. Alternatively, social animals can be defined as those who cannot survive alone and rely on members from their group to regulate their ongoing physiology (or allostasis). The rather simple evolutionary constraint of social dependency for survival can be sufficient to make the social environment vitally salient, and to provide the ultimate driving force for socially crafted brain development and learning. In this talk, I will discuss some neuro-behavioral mechanisms of early mother-infant bonding, and specify a set of hypotheses on the mechanisms of social development. I will suggest that profound human characteristics, including but not limited to sociality, are acquired at an early age, while social regulation provide key wiring instructions that shape bonding and development.
- Cathy Macpherson
Abstract: When viewed from the perspective of an embodied-embedded approach to psychological activity, interpersonal coordination is seen to play an essential role in shaping social interaction. Yet individuals diagnosed with a range of mental health disorders show deficits in this foundational aspect of everyday social exchange. Contemporary theories of mental health posit the distribution of symptomology across a continuum, however the majority of the relevant work concerning interpersonal synchrony has been conducted in clinical samples. To extend this literature, across two experiments we examined the influence of subclinical variation in mental health traits on the real-time dynamics underpinning interpersonal coordination. In Study 1, we used a dyadic pendulum swinging task, and found that increasing levels of mental health symptoms were predictive of disruption to the patterns of coordination dynamics that characterise effective social exchange. In Study 2, we utilised virtual reality (VR) to examine whether variation in interpersonal coordination associated with mental health has a basis in patterns of social attention. Participants performed a coordination task with a virtual avatar, whilst the attentional patterns of the avatar were manipulated such that she appeared to either engage or avoid the gaze of the participant. The results revealed that both symptoms of social anxiety and the actions of the avatar were related to variation in coordination patterns. Taken together, these findings strengthen support for the claim that disruption to interpersonal coordination may act as an embodied-embedded marker of variation in mental health.
- Artur Czeszumski
Abstract: The embodied-embedded-enactive-extended (4E) approach to study cognition suggests that interaction with the world is a crucial component of our cognitive processes. Most of our time, we interact with other people. Therefore, studying cognition without interaction is incomplete. Until recently, social neuroscience has only focused on studying isolated human and animal brains, leaving interaction unexplored. To fill this gap, we studied interacting participants, focusing on both intra- and inter-brain (hyperscanning) neural activity. In the first study, we invited dyads to perform a visual task in both a cooperative and a competitive context while we measured EEG. We found that mid-frontal activity around 200-300 ms after receiving monetary rewards was sensitive to social context and differed between cooperative and competitive situations. In the second study, we asked participants to coordinate their movements with each other and with a robotic partner. We found significantly stronger EEG amplitudes at frontocentral electrodes when people interacted with a robotic partner. Lastly, we performed a comprehensive literature review and the first meta-analysis in the emerging field of hyperscanning that validated it as a method to study social interaction. Taken together, our results showed that adding a second participant (human or AI/robotic) fostered our understanding of human cognition. We learned that the activity at frontocentral electrodes is sensitive to social context and type of partner (human or robotic). In both studies, the participants’ interaction was required to show these novel neural processes involved in action monitoring. Similarly, studying inter-brain neural activity allows for the exploration of new aspects of cognition. Many cognitive functions involved in successful social interactions are accompanied by neural synchrony between brains, suggesting the extended form of our cognition.
- Dimitris Bolis
Abstract: In this talk we will attempt to motivate a systematic shift of focus from being to becoming; in fact, becoming-with. More concretely, leaning on the dialectical method, cultural-historical theory and recent developments of social computational neuroscience, we will discuss human becoming as dialectical attunement, arguing that a multiscale analysis of social interaction might allow us to scientifically reconsider the self, beyond the individual, where it really emerges, unfolds and manifests itself — in social relationships. In this light, we will describe the dialectical misattunement hypothesis, which views psychopathology as a dynamic interpersonal mismatch, rather than a misfunction of single brains. Then, we will consider an operationalization of these hypotheses through ‘collective psychophysiology’, an empirical framework which aims at measuring and analyzing the multiscale dynamics of social interaction. Deploying this framework, we will empirically demonstrate that real-time dynamics of social interaction do matter in shaping communication, behavior and cognition ‒ even beyond awareness. Here, we will discuss how our results provide an empirical validation to the second-person perspective. Subsequently taking our research out of the lab, we will show that it is primarily the mismatch of autistic traits – not traits per se – which predicts core aspects of interpersonal attunement in real-life social relationships, offering evidence in favor of the intersubjective approach to autism. Concluding, an approach to human becoming in and through social interaction, we will claim, encourages a social change pertinent to various fields of human research and practice, ranging from psychiatry and pedagogy to ethics and artificial intelligence.
- Juan M. Loaiza
Abstract: I present an approach to the understanding of the relationships between music, health, and wellbeing that explicitly tackles the problem of WEIRD over-representation in cognitive sciences. This is done not only in terms of the need to increase cross-cultural samples but also in challenging the underlying theoretical assumptions at play. With the use of ethnographic examples from traditional music of the african diaspora in Latin America, I identify a unique starting point around the notion of music-as-health-establishing, and an opportunity for interdisciplinary convergence: the marriage between holistic, ethnographic conceptualisations of music’s socio-functional connection with wellbeing and health, and embodied approaches to cognition that emphasise the social situatedness and irreducibility of the mind. The proposal builds on the hypothesis about mind and behaviour as shaped by the continuous coordination between body, brain, physical and social environment across multiple timescales. I interpret music-as-health-establishing through the lens of what I call multiscalar coordination and discuss how this view is best complemented and amplified through an anthropological understanding of music, healing and wellbeing as processes intrinsic to the maintenance of human, material and intangible relationships.
- Keisuke Suzuki
Abstract: Recent developments in deep learning models have significantly scaled up the output of computational models to a quality that we can directly experience with e.g. XR technology. Here, I present the Hallucination Machine, a novel platform to simulate hallucinatory visual phenomenology of hallucinations generated by a deep neural network. Considering the experimental results from Hallucination Machine, I will discuss a possible computational mechanism underlying the distinctive visual alterations specifically under psychedelic substances. Computational models have been used mainly to simulate objective performance of perception and cognition. In contrast, in our approach, which we call 'computational neurophenomenology', computational models are employed to simulate conscious phenomenology. Such an approach will become more powerful in the future as both machine learning models and XR technologies.
- Robin Zebrowski and Eli B. McGraw
Abstract: Seibt, Vestergaard, and Damholdt (2020) argued that social robotics/human robot interaction has a “description problem” insofar as it lacks a multidisciplinary set of terminology for describing apparently social interactions between (at least) people and social robots. They point out that some capacities in these interactions can literally, rather than figuratively, be ascribed to robots, but that our current ontologies for making sense of these interactions fail us. While Seibt’s work here focuses on social robotics in particular, the ontology she lays out may be applicable outside of just social robotics work, and might offer important insights into strong AI work as it relates to robotics more broadly (Seibt, 2017). Participatory Sense-Making (PSM) (De Jaegher and Di Paolo 2007) offers a new level of enactive description with regard to interaction among multiple autonomous agents. As we have argued elsewhere (anonymized, in press) PSM is a missing and valuable variable in robotic systems as they relate to AGI (in service to something like artificial minds, as opposed to just social robotics). In attempting to apply PSM to robotic systems, it appears mysterious how and when we might attribute certain capacities, such as sentience or autonomy, to artificial systems. However, here, we propose that Seibt’s notion of sociomorphing offers appropriate and useful gradations in relation to what might count as autonomous and (perhaps eventually) sentient robotic systems. In fact, these perspectival gradations offer clear theoretical distinctions that can be repurposed or reconciled with robotic approaches to strong AI in ways that might capture the interesting but currently-overlooked level of description that PSM offers. We will argue that attempts to include PSM-levels of analysis in AI work could be radically served by Seibt’s notion of sociomorphing, which is important for evaluating social interactions including robots because it helps capture the asymmetry of such social interactions, previously hinted at but never explored or explained in earlier work in social cognition (De Jaegher, Di Paolo, and Gallagher 2010).
De Jaegher, Hanne, and Ezequiel Di Paolo.  Participatory sense-making, Phenomenology and the cognitive sciences, 6(4): 485-507.
De Jaegher, Hanne, Ezequiel Di Paolo, and Shaun Gallagher.  Can social interaction constitute social cognition?, Trends in cognitive sciences 14(10): 441-447.
Seibt, J. (2017). Towards an ontology of simulated social interaction: varieties of the “As If” for robots and humans. In Sociality and normativity for robots, Halki and Seibt (Eds.) (pp. 11-39). Springer, Cham.
Seibt, J., Vestergaard, C. and Damholdt, M. (2020). Sociomorphing, not anthropomorphizing: towards a typology of experienced sociality. In Culturally Sustainable Social Robotics- Proceedings of Robophilosophy 2020, Nørskov et al. (Eds.) (pp. 51-67). IOS Press.
- Dobromir Dotov
Abstract: Humans are social animals who engage in a variety of collective activities requiring coordinated action. Among these, music is a defining and ancient aspect of human sociality. Despite the importance and ubiquity of group action, social interaction has largely been studied in dyadic context: two persons communicating or performing a task. The presence of multiple agents in the same task space creates different constraints and possibilities. In swarming and flocking animals groups arise from short-range interactions. Such interactions lead synchronized individual actions to merge into coherent aggregate behaviour which serves as stabilizing feedback to individuals. We investigated whether such collective dynamics also play a role in human circle drumming. The evidence from the variability, speeding up, autocorrelations, and cross-correlations was that performance stability increased as group size increased. A model of joint action based on mutual prediction might struggle with the observed stability of larger groups because predicting a larger number of partners is challenging and creates opportunity to compound uncertainty. We propose a hybrid Kuramoto model for group synchronization with discrete coupling. It matches the results qualitatively and is consistent with models of collective animal behaviour. We then discuss the implications of this study for notions of collective intentionality in humans and other animals.
- Mariana Itatí Branca
Abstract: Social interaction shapes our brains, behaviors, the ways we inhabit and feel the world. In the last 20 years, the hyperscanning paradigm provided new perspectives for studying dynamic brain-to-brain real-time relations, revealing an enhanced inter-brain neural synchronization related to subjective experiences of social closeness. At the same time, emotions are widely recognized to be an essential factor in social interaction. Yet, their role in enhancing or interfering with inter-brain neural synchronizations is unclear and has not been considered enough. Intriguingly, at an individual level, rapid detection, integration, and evaluation of emotional expressions have been associated with the synchronization of neural oscillations within a specific frequency, which highlights their potential relevance for brain hyperscanning studies. In this presentation, I will be reviewing the role of affectivity and emotion in interpersonal synchronization, analyzing potential relevant factors involved, such as interception.
- Gaurav Patil and Patrick Nalepka
Abstract: Many everyday tasks involve coordinated actions between individuals, without requiring explicit communication or a priori planning, to effectively meet shared goals. An essential issue in understanding these interpersonal human behaviors is determining how individuals spontaneously coordinate their actions to achieve a common task goal. One way to explicate this behavior in various task contexts is by modelling the individuals’ actions using respective task dynamical models. The modeling of movement dynamics in these tasks stems from the fundamental understanding of human movements which can either be discrete or rhythmic and the same differential equations can generate both movement types. Importantly, by combining the movement dynamics with task specific action decision models results in dynamical models of joint action during these task contexts. These dynamical models, termed as dynamical perceptual motor primitives, can be used to model human behavior in various competitive and cooperative tasks. In that regard, this talk will present a framework for modeling human actions during air hockey, object pick-and-place, and cooperative herding tasks using dynamical perceptual motor primitives. Furthermore, implications of applying artificial intelligence techniques to model human action decisions for understanding the perceptual processes that underlie human action decisions will be discussed.
- Yuval Hart
Abstract: At the basis of our social interactions lies a cohort of subtle movements that signal invitations, affects, and intentions. While social interactions are highly complex, in both cooperative and competitive scenarios people recognize and act upon motoric cues in surprising accuracy and speed. In this talk, I will present a study that asks what are the informational cues that underlie human quick and accurate inference of the other's motion onset. In the experiment, Blockers need to react quickly to Attackers' motion onset and reach the attacker's target as soon as they can. We used concepts from criticality theory to show that people's motoric decision making is akin to a critical transition and thus is accompanied by early warning signals. The early warning signals - a sharp rise in motion's autocorrelation at lag -1 and a sharp rise in the autocorrelation decay time - occur ~130 ms before motion onset and strongly correlate with Attackers' motion onset and with Blockers' response times. The divergence of motion's variance near the transition point indicates that the transition is of a fold-bifurcation type. Lastly, we simulated the motion of the fold-bifurcation events to show that people react faster to motion driven by fold-bifurcation dynamics than to its uncorrelated counterpart. Together, our findings suggest people can recognize motoric decision making by inferring its early warning signals, allowing for a fast and accurate response.
- Mark Miller
Abstract: Affective computing is an exciting new research program aimed at developing computer and robotic systems capable of recognizing and responding intelligently to human emotions (Demekas et al. 2020). So far, the program has focused on refining forms of passive pattern recognition - deep-learning trained on emotional image data. While many of these systems have bodies they are not meaningfully embodied - their bodies play no role in the computational work of understanding emotions. In this paper, we propose a more embodied starting place for the affective computing program: the active inference framework (AIF). According to the AIF, the embodied brain's dynamics can be functionally expressed as an internal (generative) model making inferences about sensory information. Perception and action are both explainable as attempts to reconcile the discrepancies between the generative model and these signals - either by updating the model (perception) or by changing the world (action) (Clark 2016). According to this framework, to understand what someone is feeling is to have a generative model that is attuned to the other in ways where useful predictions can be made about the underlying causes (e.g., feeling happy) of the sensory evidence (e.g., smiling). This feat is accomplished not only by passively perceiving the other’s emotional expressions, but by actively probing, eliciting and confirming one’s predictions as a form of embodied epistemic foraging (Parr & Friston 2017). We will argue that the AIF offers researchers from neighboring fields interested in affective computing a shared computational currency capable of both modelling the complex dynamics underlying human emotional understanding and interaction and providing a solid basis for simulating those dynamics in computer systems (Linson et al. 2018). The outcome, we suggest, are artifacts capable of constructing for themselves useful emotional lexicons (a major challenge in affective computing) by actively engaging, sampling and updating predictions about emotional responses against their user’s behaviour in real time (Friston & Frith, 2015). We will conclude by discussing the possibility that uncertainty-minimizing artifacts and their human users may converge overtime upon a shared generative model of the world (see Friston et al. 2020), which would in turn allow for new forms of embodied collaboration and value alignment to emerge.
- Malika Auvray
Abstract: Different spatial perspectives can be taken on tactile stimuli displayed on the body surface. These stimuli can be mentally projected out of the body surface, as if they were coming from external objects or thought as being localized on the body. These possible projections answer different requirements. Indeed, on the one hand, integrating different stimuli, across sensory modalities, from a self-centred perspective is crucial for the unity of the self. On the other hand, understanding external space and communicating spatial knowledge with others requires adopting other-centred perspectives. How do we juggle these two requirements? In this talk I will review the set of studies we conducted investigating how people differ in the spatial perspectives they naturally adopt. In addition, the graphesthesia task we developed allows investigating people’s ability to flexibly change spatial perspectives, a measure that is often neglected by other perspective-taking tasks. In particular, I will highlight how perspective-taking varies as a function of sensory parameters (in particular visual and proprioceptive impairments) and social cognition (such as social intelligence and anxiety).
- Sarah Pini
Abstract: The concept of presence in Western culture informs a cluster of different connotations, encompassing metaphysical, existential, psychological, cognitive, and performative dimensions (Heidegger, 1996 ; Merleau-Ponty, 2012 ; Derrida, 1997 ; Clark, 1997; Noë, 2012). Here I focus on the concept of stage presence in different dance and performance practices. The classic model of stage presence broadly relates to the performer’s individual quality to enchant audience’s attention, and by focusing primarily on the agency of the skilled performer, it neglects audience’s participation (Sherman, 2016). Scholars who adopted an enactive and phenomenological perspective (Pini 2021; Pini 2019; Sherman, 2016; Zarrilli, 2009, 2012; Macneill, 2014) have tackled this classic view, revealing and accounting for the complexities of such encounter. Through a cognitive ecological and ethnographic approach (Hutchins, 1995, 2010), I investigate variations of presence in different dance practices and choreographic contexts: the case of the Ballet National de Marseille and the re-creation of Emio Greco’s piece Passione (Pini and Sutton, 2021), improvising together and interkinaesthetic agency in Contact Improvisation (Deans & Pini, 2022; Pini, McIlwain & Sutton, 2016); environmental attunement and ecological agency in Body Weather, a radical movement ideology informing the short dance film AURA NOX ANIMA by Australian visual artist Lux Eterna (Pini, 2022; Pini & Deans 2021). By exploring how dancers articulate their lived experience of presence, and how different performance ecologies shape different enactments of presence, I suggest adopting an ecological notion of stage presence in dance and performance.
- Lynden Miles
Abstract: More than ever before, teamwork is critical to achieving fundamental societal goals. Although the subject of debate, psychologists have typically considered collective performance as a product of individual contributions. While profitable, here we suggest that this approach has neglected a key ingredient of social exchange – interpersonal coordination. In group settings, behaviour is typically interdependent whereby each individual’s actions are reciprocally coupled to those of their co-actors, demanding some degree of coordination to ensure functional interpersonal outcomes. This raises the possibility that factors that govern how people coordinate their behaviour with others may also influence collective performance. The current research investigated this possibility by quantifying the relationship between interpersonal coordination and group productivity. Participants performed a joint object movement task while their behaviour was captured using high-resolution motion-tracking. We manipulated task-relevant affordances that varied the potential for coordination, along with the social context (i.e., competitive or cooperative) in which participants performed. Results indicated that these factors shaped between-person coupling, which in turn impacted the emergence of coordination, and ultimately group performance. Examination of the underlying dynamics revealed that key parameters that reside exclusively at the collective level of behaviour accounted for unique variance in productivity. These findings challenge traditional conceptualisations of social behaviour and underscore the utility of employing the group as a unit-of-analysis when considering the dynamics of teamwork.
- Tomokatsu Kono
- Christian G. Schütz
- Lucy Osler
Abstract: In 2007, Gary Wolf and Kevin Kelly founded the “Quantified Self” network - a network that promotes the monitoring of ‘the self’ through self-tracking, particularly by using wearable technology and apps. Many self-trackers focus on bodily health, bodily fitness, bodily measurements, and bodily feelings. Whether we subscribe to the Quantified Self network or not, many of us now use technology to monitor our step count, our weight, our calorie in-take, our moods, our periods of concentration, our work outputs, how far we have run, the number of minutes we have spent on our phones, and more. Often self-trackers provide us with information about our bodies that would not typically be (easily) available to us, for instance information about heart rate or eye-movement. As Smart et al. (2017, 268) put it: "One of the implications of the quantified self is that it provides a greater degree of awareness regarding one’s bodily states and processes". In this talk, I consider: (1) what kind of bodily awareness self-tracking apps promote and (2) whether this is a desirable kind of awareness.One of the key insights that phenomenology teaches us is that we experience our bodies in two ways. We can experience our bodies as objects in the world - as having mass, texture, colour. Typically, though, we do not turn our gaze upon our bodies but live through our bodies as a bodily subject. I argue that by placing our biological bodies under constant scrutiny, quantified self tech can encourage us to experience our bodies as objectified. This is particularly the case when our self-tracking devices prompt experiences of shame.