Research Paper

TWON report: Defining metrics for democratic online discourse

Our researcher Sjoerd Stolwijk recently published a deliverable, proposing a set of metrics to determine the deliberative quality of discussions on social media in general, and TWON in particular.

The report lists the key indicators of: (1) Exposure to political content, (2) Engagement with political content, (3) Contributing political content, (4) Diversity of exposure and (5) Quality of exposure.

It is then explained how and why this set of indicators differs from the typical list of deliberative indicators and proposes to view deliberation from a summative rather than an additive perspective. In this view, social media do not need to aim at perfect deliberation within one platform; rather, the goal is to contribute to deliberation at a societal scale via the platform. 

We propose that social media can contribute especially by offering an avenue for users (citizens, journalists and politicians alike) to be exposed to political debate, but also to engage and participate in that debate. In addition, social media can connect otherwise unconnected users and expose them to ideas they might otherwise have missed. Ideally, these ideas are substantiated with arguments and evidence.

Our researcher evaluates a large set of automatic classifiers to determine the degree to which social media comments meet several deliberative criteria, specifically whether comments are rational, interactive, diverse and civil.

Results show how more modern techniques like fine-tuned transformers and generative large language models have improved our ability to reproduce manual codings automatically, but also that results vary considerably between models.

We then integrate the aims of Chapter 3 with the results of Chapter 4 and translate them to the case of TWON to arrive at the metrics proposed in Chapter 2. It adds tests of the performance of different classifiers to determine whether a comment is political or not.

Finally, we take a look into the future, beyond what is currently feasible for TWON, to explore whether new techniques can help determine the deliberative quality of online social media debates to the more fine-grained level of specific claims and show some promising first results.

Download the Deliverable here.

A Call to Action: New Paper on Youth Online Mental Health

The World Health Organization (WHO) has put forward guiding principles on online mental health for young people. In a newly published study, TWON-researchers integrate the WHO framework into a European context. They emphasize the impact of digital platforms and online social networks (OSNs) on mental health. The COVID-19 pandemic has amplified both the usage of digital technologies and a mental health crisis that is affecting young people in particular. Traditional services are increasingly losing touch with the young target group. Substantial challenges arise from the need to provide accessible and flexible mental health support while ensuring adherence to evidence-based guidelines.

The authors stress the special character of the European context: Typically, most young Europeans are growing up as digital natives, given the broad accessibility of advanced technology. Yet, there are important regional differences concerning the approach to mental health and the influence of social stigma related to the concept. Taking into account socioeconomic discrepancies and varying degrees of media literacy across Europe, this stresses the need for culturally sensitive mental health support. So far, online mental health services are often provided by independent organizations. But like other health-related services and products, the sector is in need of a sound regulatory framework to ensure both the quality and accessibility of services.

This highlights the need to gain a better understanding of platform mechanisms and the effects of algorithmic hyper-personalization and, therefore, the importance of our joint project TWON. Based on their research, the authors developed five core recommendations:

1. The specific European context must be considered.

2. Quality and accessibility of services must be balanced carefully.

3. Involving young people and families, leveraging their knowledge while enhancing digital literacy, is key.

4. The rise of AI must be addressed, concerning both the potential of AI for developing mental health support and the risks of biases and misinformation.

5. A regulatory framework must be developed to ensure safety, effectiveness, and ethical standards.

If you want to learn more about the issue, you can access the full article here!