Next Article in Journal
Evolution of the Diagnosis and Treatment of Primary Hyperparathyroidism
Next Article in Special Issue
Mobile Solutions for Clinical Surveillance and Evaluation in Infancy—General Movement Apps
Previous Article in Journal
A TriNetX Registry Analysis of the Need for Second Procedures following Index Anterior and Posterior Urethroplasty
Previous Article in Special Issue
Mental Health and Health-Related Quality of Life in Austrian Adolescents with Chronic Physical Health Conditions: Results from the MHAT Study
 
 
Search for Articles:
Title / Keyword
Author / Affiliation / Email
Journal
Article Type
 
 
Advanced Search
 
Section
Special Issue
Volume
Issue
Number
Page
 
Journals
JCM
Volume 12
Issue 5
10.3390/jcm12052056
jcm-logo
Submit to this Journal Review for this Journal Propose a Special Issue
► Article Menu

Article Menu

share Share announcement Help format_quote Cite question_answer Discuss in SciProfiles thumb_up
...
Endorse textsms
...
Comment
first_page
settings
Order Article Reprints
Font Type:
Arial Georgia Verdana
Font Size:
Aa Aa Aa
Line Spacing:
Column Width:
Background:
Article

Does Being Ignored on WhatsApp Hurt? A Pilot Study on the Effect of a Newly Developed Ostracism Task for Adolescents

by
Delia Latina
1,2,*,
Andreas Goreis
3,
Polona Sajko
4 and
Oswald D. Kothgassner
3
1
Center for Lifespan Developmental Research, Örebro University, 702 81 Örebro, Sweden
2
Department of Child and Adolescents Psychiatry and Psychotherapy, University of Ulm, 89081 Ulm, Germany
3
Department of Child and Adolescent Psychiatry, Medical University of Vienna, 1090 Vienna, Austria
4
Klinik für Kinder- und Jungendpsychiatrie, Psychosomatik und Psychotherapie Christophsbab, 73035 Göp**en, Germany
*
Author to whom correspondence should be addressed.
J. Clin. Med. 2023, 12(5), 2056; https://doi.org/10.3390/jcm12052056
Submission received: 1 February 2023 / Revised: 28 February 2023 / Accepted: 2 March 2023 / Published: 6 March 2023
(This article belongs to the Special Issue Clinical Updates on Psychology in Children and Adolescents)
Browse Figures
Versions Notes

Abstract

:
(1) Background: Many studies have used a well-known social exclusion task, namely Cyberball, to assess the psychophysiological reactions to ostracism in laboratory settings. However, this task has been recently criticized for its lack of realism. Instant messaging communication platforms are currently central communication channels where adolescents conduct their social life. These should be considered when recreating the emotional experiences that fuel the development of negative emotions. To overcome this limitation, a new ostracism task, namely SOLO (Simulated On-Line Ostracism), recreating hostile interactions (i.e., exclusion and rejection) over WhatsApp was developed. The aim of this manuscript is to compare adolescents’ self-reported negative and positive affect, as well as physiological reactivity (i.e., heat rate, HR; heart rate variability, HRV) exhibited during SOLO to Cyberball. (2) Method: A total of 35 participants (Mage = 15.16; SD = 1.48; 24 females) took part in the study. The first group (n = 23; transdiagnostic group), recruited at an inpatient and outpatient unit of a clinic for children and adolescent psychiatry, psychotherapy, and psychosomatic therapy in Baden-Württemberg (Germany), reported clinical diagnoses linked with emotional dysregulation (e.g., self-injury and depression). The second group (n = 12; control group), recruited in the district of Bavaria and Baden-Württemberg, had no pre-existing clinical diagnoses. (3) Results: The transdiagnostic group showed higher HR (b = 4.62, p < 0.05) and lower HRV (b = 10.20, p < 0.01) in SOLO than in Cyberball. They also reported increased negative affect (interaction b = −0.5, p < 0.01) after SOLO but not after Cyberball. In the control group, no differences in either HR (p = 0.34) or HRV (p = 0.08) between tasks were found. In addition, no difference in negative affect after either task (p = 0.83) was found. (4) Conclusion: SOLO could be an ecologically valid alternative to Cyberball when assessing reactions to ostracism in adolescents with emotional dysregulation.

1. Introduction

The peer context, in general, and adverse peer experiences, in particular, seem to have strong implications on emotional development during adolescence [1]. In this phase of life, adolescents look at peers for a sense of inclusion, support, and security [2]. Thus, acceptance from peers represents a critical aspect of positive emotional development [3,4,5]. In addition, adolescence is well known for its higher sensitivity to peer evaluation and feedback [6,7]. In this scenario, experiences of ostracism in the form of social exclusion and peer rejection seem to influence the physical and psychological wellbeing of adolescents [8,9,10]. Overall, negative experiences, such as ostracism, can serve as fuel for the development of those negative emotions that are particularly difficult to regulate.
Although the experience of peers’ ostracism could be maladaptive for all adolescents, it could be particularly problematic for those that are more vulnerable. For example, a high percentage of adolescents who report depressive symptoms [11] or non-suicidal self-injury (hereafter only “self-injury”) [12] have been excluded or rejected by their peers. These groups of adolescents are characterized by a high inability to regulate their negative emotions in an adaptive manner [13,14]. Different models can help explain why these adolescents are particularly vulnerable to peers’ ostracism. On the one hand, the cognitive model of depression [15,16] postulated that adolescents who suffer from depression have automatic and uncontrollable negative thoughts about themselves (“I’m worthless”), their environment (“Nobody loves me”), as well as their future (“I will never have any success in life”). These negative schemas can be activated by stressful events or by a chain of events (e.g., ostracism), and they tend to skew the information processing system, which directs the attention of the adolescent toward negative stimuli. This translates a specific neutral experience into a negative interpretation, which tends to exacerbate depressive symptoms. On the other hand, the functional model of non-suicidal self-injury [17,18] or the experiential avoidance model of self-injury (EAM) [19] conceptualized self-injury as an emotional regulation strategy, which is used to regulate negative affect. Studies report how these negative emotions mostly occur during stressful interactions with peers and interactions that include several forms of ostracism [20,21], and how they tend to decrease right after the use of self-injury [17]. Overall, the current literature seems to point to peers’ ostracism as an emotionally upsetting experience that can have a major impact on adolescents’ psychological wellbeing, especially for those adolescents who have difficulties regulating negative emotions in an adaptive way.
To better understand how ostracism is linked to the emotional and, therefore, psychosocial wellbeing of adolescents, scholars have tried to replicate these socially upsetting experiences in laboratory settings. More specifically, they assessed the physiological stress effects of ostracism on different parameters of the autonomic nervous system (ANS), including heart rate (HR) and heart rate variability (HRV) [22,23,24]. Through the interplay between the sympathetic and parasympathetic systems, the ANS has the objective to respond to changes in environmental conditions in an adaptive manner and is therefore considered a crucial node in the relation between psyche and soma. According to the neurovisceral integration model [25,26], a dysregulation of the ANS is linked to emotional dysregulation. In this respect, several studies showed lower HRV in conditions of deficient emotional regulation, including depression [27,28], as well as difficulties in controlling impulsive behaviors (e.g., self-injury) when experiencing negative emotions in daily life [29].
To simulate ostracism that showed reliable effects in eliciting negative affect, Williams and colleagues [30] introduced the Cyberball task with the aim of assessing the psychophysiological reactions to social exclusion and peer rejection in social psychological experiments. Cyberball is a computerized virtual ball-tossing game played between the participant and other virtual players [30]. Specifically, the participant is invited to play an online ball-tossing game, which they believe to be playing with two other people currently sitting elsewhere. However, the other two participants are simply standardized computer players. The entire game is then divided into two sections: an “inclusion” section, during which all the players receive and toss the ball equally, and an “exclusion” section, where the other two members (computer players) toss the ball to each other, excluding the actual participant, and thus eliciting feelings of social exclusion. Cyberball sets itself apart from other paradigms tailored to study rejection [31,32,33] in that participants are not explicitly informed about their exclusion. This manipulation allows the study of the primary reaction to being ignored and excluded by peers.
To the best of our knowledge, since 2000, hundreds of studies have used the Cyberball task to assess the psychophysiological reactions to ostracism in social psychological experiments both in adults and adolescents [24,34,35]. These studies have reported higher negative affect and lower positive mood [23,36,37], as well as an increase in heart rate and altered cortisol levels [22,23] after exposure to the task. This also seems to be the case for adolescents reporting depression and non-suicidal self-injury [38,39]. Despite its broad use, this task has been recently criticized for its lack of realism [40] and, thus, ecological validity. For example, although Cyberball allows for the inclusion of pictures above each figurine to represent a participant, the abstract stimulus bears only minimal resemblance to the social interactions that people have daily. To confirm this critique, scholars reported that the social pain followed by exposure to Cyberball is less intense compared to that following a more personal form of social rejection [41,42]. To overcome this limitation, researchers have made use of virtual reality and developed a virtual counterpart of Cyberball. By creating a sense of “presence” within the situation, this new paradigm should be able to evoke reactions that are comparable to real-life situations. Some studies have reported preliminary results on the usefulness of this new virtual reality version of Cyberball [22,43,44]. Despite virtual reality making Cyberball approximate real-life settings, it still does not fully reflect real-life situations that people experience daily.
Since the introduction of smartphones, people belonging to “Generation Z” (born between 1995 and 2012) are the first ones to become immersed in technologies with the possibility of creating their own media content [45,46]. In the last few years, instant messaging communication platforms such as WhatsApp, Instagram, or Facebook messenger have become central means of communication and have helped to create and maintain relationships. Many people perceive these apps as major platforms on which to conduct their social life [47]. According to a recent national survey [48], WhatsApp represents the most used platform among German adolescents between 12 and 19 years of age, with more than nine out of ten using it more than once per day as the main communication service. Considering that platforms such as WhatsApp are a complementary and indispensable part of everyday adolescent life, scholars should take these new forms of interactions into account when recreating the emotional experiences in the interactions with peers that could fuel the development of negative emotions that are difficult to cope with and thus increase the likelihood of adolescents’ maladjustment.

The Present Study

The current study aims to assess a newly developed ostracism task, namely SOLO (simulated online ostracism). Due to the extensive use of instant messaging platforms such as WhatsApp as interaction and communication tools among adolescents [48], we hypothesized that SOLO would make adolescents likely experience negative affect, which will be assessed using self-reported emotional states as well as physiological reactivity (i.e., HR and HRV). This could be especially true for those adolescents who experience emotions as particularly strong and difficult to deal with, including adolescents with depression and lived experiences of self-injury. We compared the reactions to SOLO to the ones to Cyberball. Given the recent critics of the validity of Cyberball [40], we expected the self-reported emotional states as well as physiological reactivity followed by the exposure to Cyberball to be less intense compared to the ones following SOLO.

2. Materials and Methods

2.1. Participants

A total of 37 participants were recruited for this study. Due to a malfunctioning of the device used to collect data or to the dissociation of a participant, two participants were excluded from the final analyses. Thus, the analytical sample constituted a total of 35 participants (Mage = 15.16; SD = 1.48; 24 females). Of those, the majority (n = 23) reported different clinical diagnoses, all linked with an inability to cope with negative emotions, including at least five episodes of self-injury in the last year, depression, eating problems, and social phobia. The rest of the participants (n = 12) did not report any clinical diagnosis. For easier interpretability of the results, we created two different groups: a transdiagnostic group reporting emotional dysregulation, which included adolescents with a clinical diagnosis, and a control group, which included the rest of the adolescents. The participants belonging to the transdiagnostic group were patients recruited at the inpatient and outpatient unit of the clinic for children and adolescent psychiatry, psychotherapy, and psychosomatic therapy in Göp**en, Germany. The participants belonging to the control group were recruited in the German districts of Bavaria and Baden-Württemberg. To participate in the study, the adolescents needed to be between 12 and 18 years of age and needed to speak fluent German. Prior to the data collection, the study was approved by the Institutional Review Board of Ulm University, Ulm, Germany, as well as by the state medical association of Baden-Württemberg, Germany. Interested participants contacted the main author after reading the flyers promoting the study or after receiving an introduction to the study at the clinic where they were treated. Written informed consent was obtained from the participants as well as their legal guardians. Participants received a voucher for EUR 20 for participation.
Initially, we planned to recruit a balanced sample between the transdiagnostic group and the control group. However, due to entrance restrictions to the clinic where the laboratory was located, the recruitment of the control group was rendered unfeasible during the COVID-19 pandemic. While the study could be continued with inpatients (i.e., most of our transdiagnostic sample), the control group had to be limited to 12 adolescents. Nonetheless, we conducted a post hoc power analysis simulation. Our power calculation was based on a meta-analysis of 120 Cyberball studies that, on average, reported a large negative effect of the paradigm on psychological outcomes [34]. Therefore, we simulated the power to find a large effect in the chat or game condition using the R package powerlmm [49]. We based this simulation on the intra-class correlations between our four outcomes (i.e., heart rate, heart rate variability, negative and positive affect; ICC = 0.76) and our final sample size (n = 23 emotion dysregulation group, n = 12 control group). The simulation showed that our sample size still had 97% power to detect a large effect size of d = 0.8.

2.2. Measures

In order to elicit the negative emotions linked to rejection from peers, we used the well-established Cyberball task [30], as well as our new SOLO task. To ensure a lack of biases, the order in which the tasks were presented was randomized. Information about the real nature of the study and an explanation for the necessity of deception was given after the experiment in a debriefing session. An exemplary overview of the study design is provided in Figure 1.

2.2.1. Experimental Task Cyberball

Participants were made to believe that they were playing a virtual ball-tossing game with two other adolescents, currently patients of the clinic where the experiment took place. In reality, the other players did not exist, and all actions were pre-programmed [30,50]. To make the situation more realistic, we uploaded a picture of every participant below the participant’s player in the game. The pictures of the two fake participants were obtained online. All participants played two rounds of Cyberball for a total of around six minutes. In the first round (“Inclusion”; around 3 min), participants were included in the game; thus, the three players tossed the ball to each other. In the second round (“Exclusion”; around two minutes), the participant was excluded from the game, and only the two fake participants kept tossing the ball and playing with each other.

2.2.2. Experimental Task SOLO

Before initiating the task, participants were told either (1) that two other participants would be participating in the same study and would have appreciated some information from our current participant on what the study entailed; or (2) that two participants already took part in the study before our current participant, and that they wanted to share some information regarding the features of the study with them. Once given these premises, the research assistant provided the participant with a cell phone where a WhatsApp chat was already organized. The chat included only the current participant and the other two participants. In reality, the other two participants were impersonated by one research assistant. The entire task consisted of two sections for a total of around 10 min. In the first section (around 5 min), the participant was included in the conversation. All three participants were writing to each other, starting to get to know each other, and sharing some information regarding the project. In the second section, the two fake participants started to ostracize the actual participants, using name-calling and mobbing. At the end of the conversation, the two fake participants agreed to take the chat somewhere else without the actual participant. Except for the different answers provided by the adolescents, the chat was standardized for all the participants. This script was based on a previous study assessing the situations over instant messaging communication platforms that are experienced as stressful by adolescents with different clinical diagnoses, including depression and non-suicidal self-injury, as well as healthy adolescents [51].

2.2.3. Physiological Arousal

To assess the physiological arousal as reactions to SOLO and Cyberball, we used heart rate (HR) and heart rate variability (HRV). We recorded these indexes five minutes before and five minutes after the task. To record and collect data, we made use of an EcgMove 3 sensor (movisens GmbH, Karlsruhe, Germany) attached to the participant’s chest at the base of the sternum. We used an elastic chest belt with two integrated electrodes. All participants were sitting on a chair during the recordings. Data were visually inspected after every recording using the unisens viewer (version: 2.0). The DataAnalyzer® software (movisens GmbH, Karlsruhe, Germany) was used to convert the ECG signals into time-series HR (bpm) and the RMSSD (ms). Continuous five-minute segments were analyzed for both HR and HRV.

2.2.4. Negative and Positive Emotional Affect

Right before and right after each experimental task, participants filled out self-reported information regarding their current emotions. Specifically, we asked adolescents how much, on a scale from “very little” (0) to “very much” (4), they felt angry, tense, happy, and content. Using these items, we compiled a scale of negative affect (mean value between the angry and tense items) and a scale of positive affect (mean value between the happy and content items). Cronbach’s Alpha for the negative affect scale across the paradigms was 0.80, and for the positive affect scale, Cronbach’s Alpha was 0.91.

2.3. Statistical Analyses

All statistical analyses were conducted using R version 4.1.3. To analyze our hypotheses, we computed multilevel models with repeated measures data at level 1 (i.e., outcomes before, during, and after either paradigm) that were nested in participants. The task (SOLO or Cyberball) was modeled as a level 2 fixed effect, and the interaction task x time was tested to identify potential differences between tasks at time points.

3. Results

3.1. Descriptive Statistics

The majority of the adolescents belonging to the transdiagnostic group was constituted by females, while the control group was more homogeneous in terms of sex distribution. Slightly more than half of the transdiagnostic group reported a diagnosis of depression and/or self-injury. Significantly more adolescents belonging to the transdiagnostic group reported experiences of peer victimization, while significantly more adolescents belonging to the control group reported more experience of active aggression toward peers. More information regarding the participants can be found in Table 1.

3.2. Physiological Arousal

Adolescents in the transdiagnostic group reported higher HR during SOLO than during Cyberball (b = 4.62, p = 0.036, d = 0.35). In the control group, there was no difference in the effect of the task on HR (p = 0.335, see Figure 2). Similar to the HR results, the adolescents belonging to the transdiagnostic group showed lower HRV during the SOLO task than during Cyberball (b = 10.20, p = 0.008, d = −0.42). In the control group, no differences in HRV between SOLO and Cyberball were found (p = 0.078). See Figure 3 for a graphical representation.

3.3. Negative and Positive Affect

When assessed, negative affect was significantly higher after SOLO than Cyberball in the transdiagnostic group (interaction: b = −0.46, p = 0.003, d = 0.52). In the control group, there was no difference between the negative affective response to either task (p = 0.827). Only an effect of time, irrespective of the task, could be identified (b = 0.44, p = 0.178, d = 0.94; see Figure 4).
Contrary to negative affect, positive affect responses did not differ between SOLO and Cyberball when measured in the transdiagnostic group (interaction: p = 0.279) or the control group (p = 0.080). We only found an effect of time on both the transdiagnostic (b = −0.71, p < 0.001, d = 0.53) and the control group (b = −1.33, p < 0.001, d = 0.95). See Figure 5 for a depiction of the results.

4. Discussion

This is the first study that assesses the psychophysiological effect of a newly developed ostracism task in laboratory settings, namely SOLO (simulated online ostracism). We proposed SOLO as an alternative to the well-known but recently criticized Cyberball task [40]. Our study showed that the transdiagnostic group of adolescents with emotion dysregulation (e.g., adolescents with depression, non-suicidal self-injury) showed higher self-reported negative affect after SOLO compared to Cyberball. They also showed higher physiological arousal (i.e., higher HR and lower HRV) during SOLO compared to the traditional Cyberball. No differences in any psychophysiological variables and between tasks were found among community adolescents who had no pre-existing clinical diagnosis.
When asked about their emotions right after both tasks, adolescents belonging to the transdiagnostic group reported an increase in negative affect (i.e., anger and tension) after SOLO. This was not the case for the adolescents belonging to the control group, who did not report any difference in negative affect after the task. These results are in line with the studies reporting high negative affectivity in adolescents with specific psychosomatic disorders, including depression [52,53,54] and self-injury [55,56]. Studies showed that adolescents who report depressive symptoms and self-injury experience high emotional reactivity [57,58,59]. These two diagnoses were met by most of the adolescents in the group. Both the cognitive model of depression [15,16] as well as the various functional models of self-injury [17,18,19] highlight the inability of these adolescents to regulate their negative emotions in an adaptive manner and when exposed to social stressors, such as ostracism.
Contrary to the negative affect, we were not able to find a decrease in positive affect (i.e., happiness, content) in the transdiagnostic group after SOLO. Researchers have shown a link between depression and low positive affect among adolescents [60], as well as a link between low positive affect and exposure to stressful events in adolescents with depression [61]. In addition, patients with depression are particularly prone to maladaptive mood repair [62,63], a maladaptive regulatory response used to attenuate sadness. The connection between low positive affect, self-injury, and inability to regulate negative emotions has been confirmed in adolescents with lived experiences [64,65,66]. Although these findings appear in contradiction with the ones presented in our study, two possible explanations could help us justify our results. First, it is possible that SOLO was either not stressful enough or not long enough to provoke the desired strong effects on changes in positive affectivity. Second, adolescents reported their positive emotions after and not during the task. Considering that during the participation in the study, all patients were undergoing psychotherapy treatment, it can be the case that they have learned how to regulate their negative emotions in an adaptive manner. For example, they could have learned reappraisal and therefore acquired the capacity to reframe the meaning of a stressful situation to alter its emotional impact. Based on these speculations, future studies should replicate our results.
When looking at the adolescents belonging to the transdiagnostic group, our results showed that their self-reported negative affect in reaction to SOLO was also present at the physiological level. Specifically, these adolescents reported an increase in HR and a decrease in HRV during the task. Lower HRV has been reported in conditions of deficient emotional regulation, such as depression [67,68]. In line with the cognitive model of depression [15,16], adolescents suffering from this diagnosis are less capable of regulating their response to negative stimuli, which can then be reflected in an alteration of the HRV. The neurovisceral integration model posits that HRV is not only an index of healthy heart function but may serve as a link between emotional regulation and healthy physiological processes [69]. In addition, other studies showed that adolescents with low HRV report an inability to inhibit impulsive behaviors in response to daily negative emotions [29]. Considering that self-injury is often performed quickly in response to negative emotions and without prior planning [18,70], this behavior perfectly fits the definition of “impulsive”. Overall, these findings could explain why only the adolescents belonging to the transdiagnostic group had a stronger reaction to SOLO.
During Cyberball, the participants belonging to both groups showed no changes in either self-reported affect or physiological arousal. In addition, there seems to be a trend showing a decrease in HR and an increase in HRV from the baseline to the execution of the task. This seems to be in line with some studies, which did not find any reliable physiological or hormonal stress response during the task [22,23]. Together with the recent criticism of the validity of Cyberball [40], these findings indicate that maybe the rejection experienced during this task may not be strong enough to elicit a full-blown stress response. The limited social interactions that participants experience during traditional Cyberball might be too artificial, too experiment-like. Furthermore, it might even decrease the level of tension that participants had at the beginning of the task and due to the novelty of the setting. This is not the case for SOLO. The WhatsApp-like format used for this new task is more ecological. As WhatsApp has become an extremely popular medium for communication [71], adolescents use it in every setting in which they interact (i.e., at home, at school, with peers, and during their leisure time). By focusing on the aspect of social interactions, SOLO was able to create the illusion of real communication, making the experimental task ecologically valid and, therefore, more triggering.

Limitations

We should consider some limitations when discussing and interpreting the results of this study. First, due to the small sample size, as well as the differences in group sizes of the current study, the results of this study might not be generalized to the entire population. However, our results can be considered reliable when considering that the post hoc power analysis simulation showed that our sample size could detect a large effect size with 97% power. Second, the majority of the participants in the study were females. Adolescent girls are more sensitive to environmental stressors, such as ostracism and hostile interactions [72]. Thus, they may perceive these experiences as more stressful than their male counterpart. In addition, recent literature has reported valuable insights into the association between some of the diagnoses linked to emotional dysregulation (i.e., self-injury and depression) [73,74] and identifying outside the gender binary. Unfortunately, our measure does not allow us to assess adolescents who do not fit into the category of “male” or “female”. Future studies should replicate our findings, including adolescents who identify themselves as male or outside of the gender binary.

5. Conclusions

In the current study, adolescents with a diagnosis linked to emotional regulation problems, including depression and self-injury, responded more strongly to the newly developed SOLO task compared to the golden-standard Cyberball. Not only did they report more negative affect after the task, but they also showed stronger physiological activation (i.e., higher HR and lower HRV) during the task. Due to these findings, SOLO could be a valid and ecological alternative to Cyberball when assessing reactions to ostracism in adolescents with emotional dysregulation problems.

Author Contributions

Conceptualization, D.L.; methodology, A.G.; formal analysis, A.G.; writing—original draft preparation, D.L., A.G. and O.D.K.; supervision, O.D.K. and P.S.; funding acquisition, D.L. All authors have read and agreed to the published version of the manuscript.

Funding

This research was funded by Swedish Research Council (Vetenskapsrådet), grant number 2018-00621, and Universitätsklinikum ULM, project number L.SBN.0179.

Institutional Review Board Statement

The study was conducted in accordance with the Declaration of Helsinki, and approved by the Institutional Review Board of Ulm University, Ulm, Germany, as well as by the state medical association of Baden-Württemberg, Germany.

Informed Consent Statement

Informed consent was obtained from all subjects involved in the study.

Conflicts of Interest

The authors declare no conflict of interest.

References

  1. Herd, T.; Kim-Spoon, J. A systematic review of associations between adverse peer experiences and emotion regulation in adolescence. Clin. Child. Fam. Psychol. Rev. 2021, 24, 141–163. [Google Scholar] [CrossRef]
  2. Erdley, C.A.; Day, H.J. Friendship in childhood and adolescence. In The Psychology of Friendship; Oxford University Press: Oxford, UK, 2017; pp. 3–19. [Google Scholar]
  3. Kim, J.; Cicchetti, D. Longitudinal pathways linking child maltreatment, emotion regulation, peer relations, and psychopathology. J. Child Psychol. Psychiatry 2010, 51, 706–716. [Google Scholar] [CrossRef] [Green Version]
  4. King, K.M.; McLaughlin, K.A.; Silk, J.; Monahan, K.C. Peer effects on self-regulation in adolescence depend on the nature and quality of the peer interaction. Dev. Psychopathol. 2018, 30, 1389–1401. [Google Scholar] [CrossRef] [PubMed]
  5. O’Brien, S.F.; Bierman, K.L. Conceptions and perceived influence of peer groups- Interviews with preadolescents and adolescents. Child Dev. 1998, 59, 1360–1365. [Google Scholar] [CrossRef]
  6. Sebastian, C.L.; Tan, G.C.; Roiser, J.P.; Viding, E.; Dumontheil, I.; Blakemore, S.J. Developmental influences on the neural bases of responses to social rejection: Implications of social neuroscience for education. Neuroimage 2011, 57, 686–694. [Google Scholar] [CrossRef]
  7. Somerville, L.H. Special issue on the teenage brain: Sensitivity to social evaluation. Curr. Dir. Psychol. Sci. 2014, 22, 121–127. [Google Scholar] [CrossRef] [Green Version]
  8. Graham, S.; Bellmore, A.D. Peer victimization and mental health during early adolescence. Theory Pract. 2007, 46, 138–146. [Google Scholar] [CrossRef]
  9. Guyer, A.E.; Caouette, J.D.; Lee, C.C.; Ruiz, S.K. Will they like me? Adolescents’ emotional responses to peer evaluation. Int. J. Behav. Dev. 2014, 38, 155–163. [Google Scholar] [CrossRef] [PubMed] [Green Version]
  10. Rubin, K.H.; Bukowski, W.M.; Parker, J.G. Peer Interactions, Relationships, and Groups. 2006. Available online: https://www.researchgate.net/profile/Kenneth-Rubin-2/publication/228017617_Handbook_of_Child_Psychology/links/5a3a98d2a6fdcc3d07afc11b/Handbook-of-Child-Psychology.pdf (accessed on 10 January 2023).
  11. Adrian, M.; Jenness, J.L.; Kuehn, K.S.; Smith, M.R.; McLaughlin, K.A. Emotion regulation processes linking peer victimization to anxiety and depression symptoms in adolescence. Dev. Psychopathol. 2019, 31, 999–1009. [Google Scholar] [CrossRef]
  12. Cheek, S.M.; Reiter-Lavery, T.; Goldston, D.B. Social rejection, popularity, peer victimization, and self-injurious thoughts and behaviors among adolescents: A systematic review and meta-analysis. Clin. Psychol. Rev. 2020, 82, 101936. [Google Scholar] [CrossRef] [PubMed]
  13. Wolff, J.C.; Thompson, E.; Thomas, S.A.; Nesi, J.; Bettis, A.H.; Ransford, B.; Scopelliti, K.; Frazier, E.A.; Liu, R.T. Emotion dysregulation and non-suicidal self-injury: A systematic review and meta-analysis. Eur. Psychiatry 2019, 59, 25–36. [Google Scholar] [CrossRef] [PubMed]
  14. Young, K.S.; Sandman, C.F.; Craske, M.G. Positive and negative emotion regulation in adolescence: Links to anxiety and depression. Brain Sci. 2019, 9, 76. [Google Scholar] [CrossRef] [Green Version]
  15. Beck, A.T.; Rush, A.J.; Shaw, B.E.; Emery, G. Cognitive Therapy of Depression; Guilford Press: New York, NY, USA, 1979. [Google Scholar]
  16. Beck, A.T. The evolution of the cognitive model of depression and its neurobiological correlates. Am. J. Psychiatry 2008, 165, 969–977. [Google Scholar] [CrossRef]
  17. Klonsky, E.D. The functions of deliberate self-injury: A review of the evidence. Clin. Psychol. Rev. 2007, 27, 226–239. [Google Scholar] [CrossRef] [PubMed]
  18. Nock, M.K.; Prinstein, M.J. Contextual features and behavioral functions of self-mutilation among adolescents. J. Abnorm. Psychol. 2005, 114, 140–146. [Google Scholar] [CrossRef] [PubMed] [Green Version]
  19. Chapman, A.L.; Gratz, K.L.; Brown, M.Z. Solving the puzzle of deliberate self-harm: The experiential avoidance model. Behav. Res. Ther. 2006, 44, 371–394. [Google Scholar] [CrossRef] [PubMed]
  20. Esposito, C.; Bacchini, D.; Affuso, G. Adolescent non-suicidal self-injury and its relationships with school bullying and peer rejection. Psychiatry Res. 2019, 274, 1–6. [Google Scholar] [CrossRef] [PubMed]
  21. Lereya, S.T.; Winsper, C.; Heron, J.; Lewis, G.; Gunnell, D.; Fisher, H.L.; Wolke, D. Being bullied during childhood and the prospective pathways to self-harm in late adolescence. J. Am. Acad. Child Adolesc. Psychiatry 2013, 52, 608–618. [Google Scholar] [CrossRef] [Green Version]
  22. Kothgassner, O.D.; Goreis, A.; Glenk, L.M.; Kafka, J.X.; Beutl, L.; Kryspin-Exner, I.; Hlavacs, H.; Palme, R.; Felnhofer, A. Virtual and real-life ostracism and its impact on a subsequent acute stressor. Physiol. Behav. 2021, 228, 113205. [Google Scholar] [CrossRef]
  23. Iffland, B.; Sansen, L.M.; Catani, C.; Neuner, F. The trauma of peer abuse: Effects of relational peer victimization and social anxiety disorder on physiological and affective reactions to social rejection. Front. Psychiatry 2014, 5, 26. [Google Scholar] [CrossRef] [Green Version]
  24. Weik, U.; Kuepper, Y.; Hennig, J.; Deinzer, R. Effects of pre-experience of social exclusion on hypothalamus-pituitary-adrenal axis and catecholaminergic responsiveness to public speaking stress. PLoS ONE 2013, 8, e60433. [Google Scholar] [CrossRef]
  25. Thayer, J.F.; Lane, R.D. A model of neurovisceral integration in emotion regulation and dysregulation. J. Affect. Disord. 2000, 61, 201–216. [Google Scholar] [CrossRef] [PubMed] [Green Version]
  26. Thayer, J.F.; Lane, R.D. Claude Bernard and the heart–brain connection: Further elaboration of a model of neurovisceral integration. Neurosci. Biobehav. Rev. 2009, 33, 81–88. [Google Scholar] [CrossRef] [PubMed]
  27. de Jonge, P.; Rosmalen, J.G.; Kema, I.P.; Doornbos, B.; van Melle, J.P.; Pouwer, F.; Kupper, N. Psychophysiological biomarkers explaining the association between depression and prognosis in coronary artery patients: A critical review of the literature. Neurosci. Biobehav. Rev. 2010, 35, 84–90. [Google Scholar] [CrossRef] [PubMed] [Green Version]
  28. Taylor, C.B. Depression, heart rate related variables and cardiovascular disease. Int. J. Psychophysiol. 2010, 78, 80–88. [Google Scholar] [CrossRef]
  29. Williams, D.P.; Cash, C.; Rankin, C.; Bernardi, A.; Koenig, J.; Thayer, J.F. Resting heart rate variability predicts self-reported difficulties in emotion regulation: A focus on different facets of emotion regulation. Front. Psychol. 2015, 6, 261. [Google Scholar] [CrossRef] [Green Version]
  30. Williams, K.D.; Cheung, C.K.; Choi, W. Cyberostracism: Effects of being ignored over the Internet. J. Pers. Soc. Psychol. 2000, 79, 748. [Google Scholar] [CrossRef]
  31. Baumeister, R.F.; Twenge, J.M.; Nuss, C.K. Effects of social exclusion on cognitive processes: Anticipated aloneness reduces intelligent thought. J. Pers. Soc. Psychol. 2002, 83, 817. [Google Scholar] [CrossRef]
  32. Craighead, W.E.; Kimball, W.H.; Rehak, P.J. Mood changes, physiological responses, and self-statements during social rejection imagery. J. Consult. Clin. Psychol. 1979, 47, 385. [Google Scholar] [CrossRef]
  33. Nezlek, J.B.; Kowalski, R.M.; Leary, M.R.; Blevins, T.; Holgate, S. Personality moderators of reactions to interpersonal rejection: Depression and trait self-esteem. Pers. Soc. Psychol. Bull. 1997, 23, 1235–1244. [Google Scholar] [CrossRef]
  34. Kothgassner, O.D.; Felnhofer, A. Does virtual reality help to cut the Gordian knot between ecological validity and experimental control? Ann. Int. Commun. Assoc. 2020, 44, 210–218. [Google Scholar] [CrossRef]
  35. Davidson, C.A.; Willner, C.J.; van Noordt, S.J.R.; Banz, B.C.; Wu, J.; Kenney, J.G.; Johannesen, J.K.; Crowley, M.J. One-month stability of cyberball post-exclusion ostracism distress in adolescents. J. Psychopathol. Behav. Asses.s 2019, 41, 400–408. [Google Scholar] [CrossRef] [PubMed]
  36. Carter-Sowell, A.R.; Chen, Z.; Williams, K.D. Ostracism increases social susceptibility. Soc Influ 3: 143–153. clinical profile. Lancet Child Adolesc. Health 2008, 4, 322–330. [Google Scholar]
  37. Lustenberger, D.E.; Jagacinski, C.M. Exploring the effects of ostracism on performance and intrinsic motivation. Hum. Perform. 2010, 23, 283–304. [Google Scholar] [CrossRef]
  38. De Rubeis, J.; Sütterlin, S.; Lange, D.; Pawelzik, M.; van Randenborgh, A.; Victor, D.; Vögele, C. Attachment status affects heart rate responses to experimental ostracism in inpatients with depression. PLoS ONE 2016, 11, e0150375. [Google Scholar] [CrossRef]
  39. Groschwitz, R.C.; Plener, P.L.; Groen, G.; Bonenberger, M.; Abler, B. Differential neural processing of social exclusion in adolescents with non-suicidal self-injury: An fMRI study. Psychiatry Res. Neuroimaging 2016, 255, 43–49. [Google Scholar] [CrossRef] [Green Version]
  40. Parsons, T.D. Virtual reality for enhanced ecological validity and experimental control in the clinical, affective and social neurosciences. Front. Hum. Neurosci. 2015, 9, 660. [Google Scholar] [CrossRef] [Green Version]
  41. Eisenberger, N.I. Meta-analytic evidence for the role of the anterior cingulate cortex in social pain. Soc. Cogn. Affect. Neurosci. 2015, 10, 1–2. [Google Scholar] [CrossRef] [Green Version]
  42. Rotge, J.-Y.; Lemogne, C.; Hinfray, S.; Huguet, P.; Grynszpan, O.; Tartour, E.; George, N.; Fossati, P. A meta-analysis of the anterior cingulate contribution to social pain. Soc. Cogn. Affect. Neurosci. 2015, 10, 19–27. [Google Scholar] [CrossRef]
  43. Kothgassner, O.D.; Griesinger, M.; Kettner, K.; Wayan, K.; Völkl-Kernstock, S.; Hlavacs, H.; Beutl, L.; Felnhofer, A. Real-life prosocial behavior decreases after being socially excluded by avatars, not agents. Comput. Hum. Behav. 2017, 70, 261–269. [Google Scholar] [CrossRef]
  44. Kassner, M.P.; Wesselmann, E.D.; Law, A.T.; Williams, K.D. Virtually ostracized: Studying ostracism in immersive virtual environments. Cyberpsychol. Behav. Soc. Netw. 2012, 15, 399–403. [Google Scholar] [CrossRef] [PubMed] [Green Version]
  45. Bobkowski, P.S.; Shafer, A.; Ortiz, R.R. Sexual intensity of adolescents’ online self-presentations: Joint contribution of identity, media consumption, and extraversion. Comput. Hum. Behav. 2016, 58, 64–74. [Google Scholar] [CrossRef]
  46. Twenge, J.M.; Martin, G.N.; Campbell, W.K. Decreases in psychological well-being among American adolescents after 2012 and links to screen time during the rise of smartphone technology. Emotion 2018, 18, 765. [Google Scholar] [CrossRef]
  47. Aizenkot, D.; Kashy-Rosenbaum, G. Cyberbullying in WhatsApp classmates’ groups: Evaluation of an intervention program implemented in Israeli elementary and middle schools. New Media Soc. 2018, 20, 4709–4727. [Google Scholar] [CrossRef]
  48. Medienpädagogischen Forschungsverbund Südwest. Jugend, Information, Medien. 2021. Available online: https://www.mpfs.de/fileadmin/files/Studien/JIM/2021/JIM-Studie_2021_barrierefrei.pdf (accessed on 15 December 2022).
  49. Magnusson, K. Powerlmm: Power Analysis for Longitudinal Multilevel Models. 2018. Available online: https://github.com/rpsychologist/powerlmm (accessed on 10 January 2023).
  50. Eisenberger, N.I.; Lieberman, M.D.; Williams, K.D. Does rejection hurt? An fMRI study of social exclusion. Science 2003, 302, 290–292. [Google Scholar] [CrossRef] [PubMed]
  51. Latina, D.; Claréus, B.; Gayfer, B.; Sajco, P.; Lewis, S.P. “You leave the chat with a different feeling than when you came in.”: A Content Analysis about Negative Experiences Following Instant Messaging Among Adolescents with and without a History of Non-Suicidal Self-Injury. Digit. Psychol. 2022, 3, 32–39. [Google Scholar] [CrossRef]
  52. Lonigan, C.J.; Hooe, E.S.; David, C.F.; Kistner, J.A. Positive and negative affectivity in children: Confirmatory factor analysis of a two-factor model and its relation to symptoms of anxiety and depression. J. Consult. Clin. Psychol. 1999, 67, 374. [Google Scholar] [CrossRef]
  53. Miles, H.; MacLeod, A.K.; Pote, H. Retrospective and prospective cognitions in adolescents: Anxiety, depression, and positive and negative affect. J. Adolesc. 2004, 27, 691–701. [Google Scholar] [CrossRef]
  54. Vera-Villarroel, P.; Urzúa, A.; Jaime, D.; Contreras, D.; Zych, I.; Celis-Atenas, K.; Silva, J.R.; Lillo, S. Positive and Negative Affect Schedule (PANAS): Psychometric properties and discriminative capacity in several Chilean samples. Eval. Health Prof. 2019, 42, 473–497. [Google Scholar] [CrossRef]
  55. Koenig, J.; Klier, J.; Parzer, P.; Santangelo, P.; Resch, F.; Ebner-Priemer, U.; Kaess, M. High-frequency ecological momentary assessment of emotional and interpersonal states preceding and following self-injury in female adolescents. Eur. Child Adolesc. Psychiatry 2021, 30, 1299–1308. [Google Scholar] [CrossRef]
  56. Kranzler, A.; Fehling, K.B.; Lindqvist, J.; Brillante, J.; Yuan, F.; Gao, X.; Miller, A.L.; Selby, E.A. An ecological investigation of the emotional context surrounding nonsuicidal self-injurious thoughts and behaviors in adolescents and young adults. Suicide Life-Threat Behav. 2018, 48, 149–159. [Google Scholar] [CrossRef]
  57. Herres, J.; Ewing, E.S.K.; Kobak, R. Emotional reactivity to negative adult and peer events and the maintenance of adolescent depressive symptoms: A daily diary design. J. Abnorm. Child Psychol. 2016, 44, 471–481. [Google Scholar] [CrossRef] [PubMed] [Green Version]
  58. Nock, M.K.; Wedig, M.M.; Holmberg, E.B.; Hooley, J.M. The emotion reactivity scale: Development, evaluation, and relation to self-injurious thoughts and behaviors. Behav. Ther. 2008, 39, 107–116. [Google Scholar] [CrossRef] [PubMed]
  59. Santee, A.C.; Starr, L.R. Examining linear and nonlinear associations between negative emotional reactivity to daily events and depression among adolescents. Clin. Psychol. Sci. 2022, 10, 675–689. [Google Scholar] [CrossRef]
  60. Compas, B.E.; Connor-Smith, J.; Jaser, S.S. Temperament, stress reactivity, and co**: Implications for depression in childhood and adolescence. J. Clin. Child Adolesc. Psychol. 2004, 33, 21–31. [Google Scholar] [CrossRef]
  61. Kuhlman, K.R.; Chiang, J.J.; Bower, J.E.; Irwin, M.R.; Cole, S.W.; Dahl, R.E.; Almeida, D.M.; Fuligni, A.J. Persistent low positive affect and sleep disturbance across adolescence moderate link between stress and depressive symptoms in early adulthood. Res. Child Adolesc. Psychopathol. 2020, 48, 109–121. [Google Scholar] [CrossRef]
  62. Kovacs, M.; Lopez-Duran, N.L.; George, C.; Mayer, L.; Baji, L.; Kiss, E.; Vetro, A.; Kapornai, K. The development of mood repair response repertories: I. Age-related changes among 7-to 14-year-old depressed and control children and adolescents. J. Clin. Child Adolesc. Psychol. 2019, 48, 143–152. [Google Scholar] [CrossRef] [Green Version]
  63. Kovacs, M.; Yaroslavsky, I.; Rottenberg, J.; George, C.J.; Kiss, E.; Halas, K.; Dochnal, R.; Benák, I.; Baji, I.; Vetró, A.; et al. Maladaptive mood repair, atypical respiratory sinus arrhythmia, and risk of a recurrent major depressive episode among adolescents with prior major depression. Psychol. Med. 2016, 46, 2109–2119. [Google Scholar] [CrossRef] [Green Version]
  64. Andrewes, H.E.; Hulbert, C.; Cotton, S.M.; Betts, J.; Chanen, A.M. Ecological momentary assessment of nonsuicidal self-injury in youth with borderline personality disorder. Pers. Disord. Theory Res. Treat 2017, 8, 357. [Google Scholar] [CrossRef]
  65. Chen, W.L.; Chun, C.C. Association between emotion dysregulation and distinct groups of non-suicidal self-injury in Taiwanese female adolescents. Int. J. Environ Health Res. 2019, 16, 3361. [Google Scholar] [CrossRef] [PubMed] [Green Version]
  66. Muehlenkamp, J.J.; Engel, S.G.; Wadeson, A.; Crosby, R.D.; Wonderlich, S.A.; Simonich, H.; Mitchell, J.E. Emotional states preceding and following acts of non-suicidal self-injury in bulimia nervosa patients. Behav. Res. Ther. 2009, 47, 83–87. [Google Scholar] [CrossRef] [PubMed] [Green Version]
  67. Brown, L.; Karmakar, C.; Gray, R.; **dal, R.; Lim, T.; Bryant, C. Heart rate variability alterations in late life depression: A meta-analysis. J. Affect. Disord. 2018, 235, 456–466. [Google Scholar] [CrossRef]
  68. da Estrela, C.; McGrath, J.; Booij, L.; Gouin, J.P. Heart rate variability, sleep quality, and depression in the context of chronic stress. Ann. Behav. Med. 2021, 55, 155–164. [Google Scholar] [CrossRef] [PubMed]
  69. Thayer, J.F.; Ahs, F.; Fredrikson, M.; Sollers, J.J.; Wager, T.D. A meta-analysis of heart rate variability and neuroimaging studies: Implications for heart rate variability as a marker of stress and health. Neurosci. Biobehav. Rev. 2012, 36, 747–756. [Google Scholar] [CrossRef] [PubMed]
  70. Lockwood, J.; Townsend, E.; Allen, H.; Daley, D.; Sayal, K. What young people say about impulsivity in the short-term build up to self-harm: A qualitative study using card-sort tasks. PLoS ONE 2020, 15, e0244319. [Google Scholar] [CrossRef]
  71. Rosenfeld, A.; Sina, S.; Sarne, D.; Avidov, O.; Kraus, S. WhatsApp usage patterns and prediction of demographic characteristics without access to message content. Demographic Res. 2018, 39, 647–670. [Google Scholar] [CrossRef] [Green Version]
  72. Oldehinkel, A.J.; Bouma, E.M. Sensitivity to the depressogenic effect of stress and HPA-axis reactivity in adolescence: A review of gender differences. Neurosci. Biobehav. Rev. 2011, 35, 1757–1770. [Google Scholar] [CrossRef]
  73. Chew, D.; Tollit, M.A.; Poulakis, Z.; Zwickl, S.; Cheung, A.S.; Pang, K.C. Youths with a non-binary gender identity: A review of their sociodemographic and clinical profile. Lancet Child Adolesc. Health 2020, 4, 322–330. [Google Scholar] [CrossRef]
  74. Liu, R.T.; Sheehan, A.E.; Walsh, R.F.; Sanzari, C.M.; Cheek, S.M.; Hernandez, E.M. Prevalence and correlates of non-suicidal self-injury among lesbian, gay, bisexual, and transgender individuals: A systematic review and meta-analysis. Clin. Psychol. Rev. 2019, 74, 101783. [Google Scholar] [CrossRef]
Jcm 12 02056 g001 550
Figure 1. Example time plan of the experimental procedure.
Figure 1. Example time plan of the experimental procedure.
Jcm 12 02056 g001
Jcm 12 02056 g002 550
Figure 2. Mean HR of the transdiagnostic group (A) and the control group (B) over the two tasks. * p < 0.05, n.s. = non-significant.
Figure 2. Mean HR of the transdiagnostic group (A) and the control group (B) over the two tasks. * p < 0.05, n.s. = non-significant.
Jcm 12 02056 g002
Jcm 12 02056 g003 550
Figure 3. Mean HRV in the transdiagnostic group (A) and the control group (B) over the two tasks. * p < 0.01, n.s. = non-significant.
Figure 3. Mean HRV in the transdiagnostic group (A) and the control group (B) over the two tasks. * p < 0.01, n.s. = non-significant.
Jcm 12 02056 g003
Jcm 12 02056 g004 550
Figure 4. Mean negative affect of the transdiagnostic group (A) and the control group (B) over the two tasks. * p < 0.001, n.s. = non-significant.
Figure 4. Mean negative affect of the transdiagnostic group (A) and the control group (B) over the two tasks. * p < 0.001, n.s. = non-significant.
Jcm 12 02056 g004
Jcm 12 02056 g005 550
Figure 5. Mean positive affect of the transdiagnostic group (A) and the control group (B) over the two tasks. n.s. = non-significant.
Figure 5. Mean positive affect of the transdiagnostic group (A) and the control group (B) over the two tasks. n.s. = non-significant.
Jcm 12 02056 g005
Table
Table 1. Descriptive information of the sample.
Table 1. Descriptive information of the sample.
Emotion Dysregulation Group (n = 23)Control Group (n = 12)
Sex
  Femalesn = 17n = 5
  Malesn = 6n = 7
AgeMage = 14.74 (SD = 1.389)Mage = 16 (SD = 1.477)
Diagnosis
  Non-Suicidal Self-Injuryn = 13n = 0
  Depressionn = 12n = 0
  Social Phobian = 3n = 0
  Eating Disordersn = 2n = 0
  Trauman = 1n = 0
  Paranoid Schizophrenia n = 1n = 0
  Emotional disordersn = 1n = 0
  Multiple Diagnosesn = 10n = 0
Aggression towards PeersM = 1.04 (SD = 0.209) 11.43 (SD = 0.607) 1
Victimization from Peers M = 1.72 (SD = 0.816) 1M = 1.37 (SD = 0.168) 1
1 Likert scale ranging from 1 “It never happened” to 5 “It happened more than five times per week”.
Disclaimer/Publisher’s Note: The statements, opinions and data contained in all publications are solely those of the individual author(s) and contributor(s) and not of MDPI and/or the editor(s). MDPI and/or the editor(s) disclaim responsibility for any injury to people or property resulting from any ideas, methods, instructions or products referred to in the content.

Share and Cite

MDPI and ACS Style

Latina, D.; Goreis, A.; Sajko, P.; Kothgassner, O.D. Does Being Ignored on WhatsApp Hurt? A Pilot Study on the Effect of a Newly Developed Ostracism Task for Adolescents. J. Clin. Med. 2023, 12, 2056. https://doi.org/10.3390/jcm12052056

AMA Style

Latina D, Goreis A, Sajko P, Kothgassner OD. Does Being Ignored on WhatsApp Hurt? A Pilot Study on the Effect of a Newly Developed Ostracism Task for Adolescents. Journal of Clinical Medicine. 2023; 12(5):2056. https://doi.org/10.3390/jcm12052056

Chicago/Turabian Style

Latina, Delia, Andreas Goreis, Polona Sajko, and Oswald D. Kothgassner. 2023. "Does Being Ignored on WhatsApp Hurt? A Pilot Study on the Effect of a Newly Developed Ostracism Task for Adolescents" Journal of Clinical Medicine 12, no. 5: 2056. https://doi.org/10.3390/jcm12052056

Note that from the first issue of 2016, this journal uses article numbers instead of page numbers. See further details here.

Article Metrics

Back to TopTop