Enhancing social science research on cyberbullying through human machine collaboration | #childpredator | #kidsaftey | #childsaftey

Causal DAG construction

The generalized form (without causal implications) of the PGCM is the BN, a graphical model representing the joint probability distribution of random variables. A BN comprises a DAG and conditional probability tables (CPT)^36,37. Given a DAG, namely G, and a joint probability distribution P over a set of discrete variables $X=\left\{ X_1, \ldots , X_n\right\}$, we can say that G is modeling or representing P correctly if there is a one-to-one correspondence between the variables in X and G such that Eq. (1) is satisfied. Where ${p a}_i$ are the direct parent nodes of $x_i$ in G, and $P\left( x_i \mid {p a}_i\right)$ is the conditional probability distribution.

$$\begin{aligned} P\left( x_1, \ldots , x_n\right) =\prod _i P\left( x_i \mid {p a}_i\right) \end{aligned}$$

(1)

Conditional probabilities play a crucial role in establishing causality, as they allow us to compute the likelihood that one event will occur, given that another has already happened. However, interpreting a BN as carrying conditional dependence and Independence assumptions does not necessarily imply causality; a valid graph set can be constructed from independent variables with any ordering, not necessarily causal or chronological³⁸. As discussed above, determining causal relationships requires additional information or assumptions beyond the data, such as experimental manipulation or adjustment for confounding variables in observational studies³⁹.

What do we mean by “expert knowledge”?

In this work, we assume that the expert-defined DAG is known and its connections have a causal interpretation since it has been constructed based on insights gained through practical experience and literature study⁴⁰. The construction of this DAG is not a trivial task as causality is a loaded word with strong connotations (especially in the Social Sciences where it is sometimes interpreted as deterministic association). For the sake of consistency, in this section, we expand on previously open-access documents produced during the development of the RAYUELA project⁴⁰.

In the mentioned project, a team formed by psychologists, criminologists, social workers, and sociologists was given a seminar to introduce them to the relevant terminology related to causal inference³⁷, including concepts such as collider, confounder, and mediator, as well as their implications for causal connections (https://eventos.comillas.edu/101590/detail/si-la-correlacion-no-implica-causalidad-entonces-que.html). After the exposition, we asked the participants the following questions to help them to identify the main building blocks of the DAG:

(i)
Confounder: Can you think of any factor that might influence both your main independent variable and your outcome variable, even if it’s not part of your main hypothesis?
(ii)
Mediator: Is there a step or process through which your independent variable affects your outcome? What happens because of your independent variable that then leads to the outcome?
(iii)
Collider: Is there a variable that is influenced by both your independent variable and your outcome? Something that might be a result of both, rather than a cause?

Before that seminar, the team members had already assembled (and conducted some data gathering) information from different sources (see Table 1): Focus groups⁴⁰; Interviews⁴¹; Court sentences⁴²; and Literature review (this work, see the following paragraphs and section “Literature-informed causal connections”). Here, we summarize the main findings of our team (led by co-author M. Reneses) (see section “Literature-informed causal connections” for a more detailed account). Previous research⁴³ shows that in countries where young people spend less time online, they reported lower rates of CB. This trend includes CB victimization (e.g.,^44,45) as well as CB perpetration (e.g.,^46,47). Besides, victimization and perpetration are also related to less open and more avoidant communication with parents⁴⁸ and family conflicts⁴⁸. Aizencot⁴⁹ showed the connection between Internet and social media activity, online self-disclosure, and the education institution phase and CB victimization. Previous cyber victimization also increases the likelihood of later CB activities⁵⁰.

The prevalence of cyber-victimization is greater among LGBTQ students^51,52,53, as well as young migrants and ethnic minorities^54,55 , and girls⁵⁶. Although girls’ and boys’ time spent online is similar, their use is not: boys engage in more CB⁵⁷ and report less to an adult when it occurs. Boys are also more aggressive in their interactions⁵⁸. Shohoudi and colleagues⁵⁹ showed that, regardless of background, girls rated abusive behaviors more negatively. Regarding Age, most studies point out that older teenagers are more likely to be both victims and offenders of CB⁶⁰.

Once the information is collected and aggregated from those four sources, the mentioned seminar on causal thinking included hands-on work to clarify the differences between different concepts, in particular, colliders vs confounders, and the perils of controlling (or not) them—including also a discussion on Simpson’s effect.

Consensus-driven model building

Once we have exposed the formal semantics and assumptions of the model, the proposed methodology to build the causal DAG that brings together expert knowledge and data consists of the following four steps, we created Table 1 summarizing our integrated knowledge and as a guide for the first step in the following methodology (a graphical representation of these steps is shown in Fig. 1):

Table 1 Risk factors for cyberbullying (CB) victimization. FG focus groups (see our work in Ref.⁴⁰), IV interviews (see Ref.⁴¹), CS court sentences (see Ref.⁴²).

1.
Initial proposals: Structure learning algorithms and the experts build their first proposals without transmitting information between these actors. The experts’ proposals may be based on their experience, previous literature, or common sense. From this step, we obtain a series of potential causal DAGs. The experts are informed on relevant terminology related to Bayesian networks, including concepts such as collider, confounder, and mediator and their implications for causal connections.
2.
Consistency causal restrictions: Together with their proposals, the experts forbid a set of causal connections to the algorithms that will be included as initial conditions. Naturally, one must be completely confident when prohibiting these connections. These restrictions come from common sense or widely accepted knowledge in the literature. Finally, in those cases where the collected evidence is not conclusive or even absent, the potential connection is left without restriction.
3.
Suggested causal arrows: Conversely, the experts analyze the initial algorithms’ proposals (without restrictions) and consider whether any causal relationships found should be incorporated into their second proposal.
4.
Quantitative comparison and consensus: Once the new algorithmic proposals (with restrictions) and the second proposal of the experts have been obtained, a quantitative comparison is made to identify the model that best explains the available data. Finally, the best-performing models for the specified metrics are selected, and the implications of each proposal are discussed.

Fig. 1

Conceptual graphical representation of the steps involved in constructing a causal Directed Acyclic Graph (DAG). First, experts and structure learning algorithms propose models based on existing literature and field research. Next, experts impose certain hard restrictions—while remaining open to others—and revise their initial proposals based on the relationships identified by the algorithms. Finally, a quantitative comparison is performed, and consensus is reached, as there are no qualitative differences between semi-supervised automatic DAGs and refined DAGs; both lead to the same conclusions.

Metrics

Each causal DAG candidate is evaluated using the metrics described below to perform the last step of the proposed methodology, which is the quantitative comparison of structures.

Log-Likelihood (LL) Score: The LL score measures how well the DAG fits the observed data. It calculates the logarithm of the likelihood function, which represents the probability of the observed data (x) given the network structure and parameters ($\Theta$). Formally, it is expressed as $log(\mathcal {L}(\Theta \mid x))$. Higher LL scores indicate a better fit.

Bayesian Information Criterion (BIC): The BIC is a widely used metric that balances the goodness-of-fit and model complexity. The BIC score is calculated using the LL score and penalizing the number of parameters in the model to avoid overfitting⁶⁹. Formally, it is expressed as shown in Eq. (2) where k represents the number of parameters estimated, n is the number of data points, and $\widehat{L}$ is the maximized value of the likelihood function of the model. Lower BIC scores indicate a better trade-off between fit and complexity.

$$\begin{aligned} \textrm{BIC}=k \ln (n)-2 \ln (\widehat{L}) \end{aligned}$$

(2)

K2 score: The K2 score is a particular case of the Bayesian Dirichlet score and is commonly used in BN structure learning. It is based on the likelihood of the data given the network structure and parameters⁷⁰. It incorporates prior probabilities and can handle small sample sizes. It also penalizes model complexity, but to a lesser extent than BIC. Higher K2 scores indicate a better fit.

Correlation score: This score evaluates how well the Directed Acyclic Graph (DAG) captures correlations in the data using the d-separation property⁷¹. For each variable pair, a correlation test, typically chi-square, is conducted. We then check if these variables are d-connected in the DAG, leading to the calculation of a classification metric (e.g., F1 score) based on the correlation test as the true value and the d-connections as predicted values. Higher correlation scores reflect better alignment between the data correlations and the DAG.

The choice of metric is influenced by the application goals, model complexity, prior knowledge, and sample size. Importantly, the objective of creating the best DAG is to accurately represent hypothetical causal relationships rather than merely achieving predictive accuracy. Each metric has its pros and cons. The LL score may favor overly complex models, risking overfitting. The K2 score serves as a baseline but depends on specific hyperparameters. The BIC is helpful for finding a balance between model fit and complexity, following Occam’s razor. The correlation score focuses on testing conditional independence without assessing data fit.

For the LL score, we used a k-fold cross-validation method to compare candidate models, a common approach in machine learning to mitigate overfitting⁷². This technique involves dividing the dataset into k segments and training the model k times with different test sets. Notably, the LL score uniquely requires model parameters for evaluation, making it the only metric justifying k-fold cross-validation.

Performing causal tasks

Once we have selected a causal DAG with which we are satisfied and it has been trained with the available data, we can perform different analyses to interrogate and validate the model. For example, effect estimation (If we change A, how much will it cause B to change?), attribution (Why did an event occur?), counterfactual estimation (What would have changed if we had measured a value in A different from the observed value?), or prediction (What will we get as a result of a new data entry?)³⁷.

In this paper, we are interested in the task of effect estimation. For instance, estimate the effect of different interventions on the risk of suffering CB. To this end, we computed the Average Causal Effect (ACE)⁷³, also known as Average Treatment Effect (ATE), to answer the following question: How much does a certain target quantity differ under two different interventions? Using the do-calculus notation⁷⁴, the ACE can be written as in Equation (3). The $\operatorname {do}$ operator symbolizes an intervention and can be defined as in Equation (4), where S is the sufficient adjustment set, the set of variables in the DAG that block all the confounding paths from treatment T to outcome Y and meet the requirements of the back-door criterion⁷⁵. The key strength of this method is that it enables us to estimate the interventional probability distribution of the outcome from observational probability distributions. This method provides a single value representing the influence of specific interventions.

$$\begin{aligned} \text {ACE} = \text {ATE} = \mathbb {E}[Y \mid \operatorname {do}(T:=A)]-\mathbb {E}[Y \mid \operatorname {do}(T:=B)] \end{aligned}$$

(3)

$$\begin{aligned} P(Y \mid \operatorname {do}(T:=t))=\sum _{s \in S} P(Y \mid S=s, T=t) P(S=s) \end{aligned}$$

(4)

If the treatment variable T has more than two options, the ACE can be calculated for a pair of values of interest or the values that give the most extreme results. We expressed the ACE results as the percentage difference between treatment A and B (ACE $= P_A – P_B$) and, equivalently, as an Odds Ratio (OR) following Equation (5). Having obtained the results for each variable, we can rank them. Larger ACE values indicate a greater causal influence on the outcome.

$$\begin{aligned} \text {Odds Ratio } (OR) = \frac{P_A(1-P_B)}{P_B(1-P_A)} \end{aligned}$$

(5)

Ensemble causal analysis

One benefit of the LL and BIC metrics is that they can be interpreted probabilistically. In particular, given a model $M_i$, with parameters $\theta _i$ over data D:

$$P(D|M_i)=\exp \left( \text {Log-Likelihood}\right) .$$

Similarly, BIC arises from a Laplace approximation to the marginal likelihood (e.g., the evidence) of a model⁷⁶:

$$P(D \mid M_i) = \int P(D \mid \theta _i, M_i) \cdot P(\theta _i \mid M_i) \, d\theta _i$$

This integral is often intractable, but under certain assumptions (large sample size, regular priors, etc.), it can be approximated as:

$$\log P(D \mid M_i) \approx \log L_i – \frac{k_i}{2} \log n.$$

So, from (2)

$$P(D \mid M_i) \propto \exp \left( -\frac{1}{2} \cdot \text {BIC}_i\right) p(M_i).$$

If we do not have any preferred model, then all the prior probabilities for each model, $p(M_i)$, are equal. Using both definitions, in Table 5 we compute an Ensemble Average Causal Effect (EACE), in which we weigh ACE for each model according to its plausibility. Namely, for each model i,

$$\begin{aligned} w_{M_i}=\frac{P(D \mid M_i)}{\sum _{j}P(D \mid M_j)} \quad \Longrightarrow \quad \text {EACE}\equiv \sum _i w_{M_i}\text {ACE}_{M_i}. \end{aligned}$$

(6)

We think that borrowing the concept of ensemble prediction and adapting it to the estimation of the causal effect is an interesting addition to our methodology.

Ablation analysis

Ablation analysis is a geometrically-inspired methodology to evaluate the robustness and reliability of networks after link removal, mostly used in the context of deep learning⁷⁷, but it can be used to analyze BN structures, particularly when comparing expert-constructed and automatically learned DAGs against a ground truth. It involves systematically removing individual edges from the DAGs to assess their impact on key performance metrics. This approach reveals which edges are most influential to the model’s fit and accuracy, providing insights into potential overfitting, misspecified relationships, or the loss of critical causal links. By comparing these metrics before and after edge removal, discrepancies between the expert knowledge embedded in the DAG and the data-driven inferences of automated algorithms can be quantified, thereby validating or refining the network structure.

Here, for illustration, we will limit the analysis to variations after edge removal of LL and Structural Hamming Distance (SHD). These parameters measure how different two graphs are by counting the minimum number of edge insertions, deletions, or direction flips needed to transform one graph into the other. In our case, we do not have the ground truth, but as the final DAGs are relatively similar—they are indeed for the 2nd experts’ proposal and DAGS with NO TEARS with restrictions—we will use these ones as an effective ground truth in the analysis.

Data collection

This dataset was collected as part of a previous study⁷⁸ through a representative survey of children in Madrid (Spain) schools, freely available at the Zenodo repository (https://doi.org/10.1016/j.childyouth.2025.108285). Informed consent has been obtained from a parent and/or legal guardian. All methods were performed in accordance with the relevant guidelines and regulations.

The survey collected responses from 682 students aged 13 to 17, where 46.6% identified themselves as males, 45.2% as females, 3.1% as non-binary, and 5.1% preferred not to say. The data includes demographic information (Age, gender, sexual orientation, migratory background, and family communication), participants’ relationship with technology and the Internet, and 4 inquiries about Cyberbullying-related situations and 1 (question 13) related to physical isolation.

Table 2 provides the variable values and their percentage of occurrences (i.e., marginal probability). To analyze the data, we defined a binary variable called Cyberbullying Victimization Risk that takes the value 1 if the respondent has answered at least 2 out of the 5 mentioned questions. We use this threshold in 2 answers rather than in 1 to improve the specificity of the survey regarding CB Victimization.

The Universidad Pontificia Comillas Ethics Committee approved the data collection and experimental procedures. In addition, RAYUELA’s legal experts also took the necessary measures to ensure that data collection, storage, and disclosure comply with the European GDPR. In each session, the researchers and teachers explained the project, its main objective, and the data to be collected.

Table 2 Survey of Spanish minors dataset ($N=682$): the table shows the possible values of each variable and its marginal probability (i.e., the percentage of observations).

Collaboration with cyberbullying experts from RAYUELA

The expert knowledge comes from members of the RAYUELA project consortium. Work package 1 in that project was concerned with creating a knowledge base on the drivers of cybercrime in young people. For this purpose, this team conducted research that sought to understand the pathology and physiology of online behaviors, characterizing the victims and offenders of cybercrime and the modus operandi.

This team included members from Universidad Pontificia Comillas (Spain), University of Ghent (Belgium), University of Tartu (Estonia), University College Limburg (Belgium), Bratislava Policy Institute (Slovakia), Ellinogermaniki Agogi (Greece), Polícia Judiciária (Portugal), Valencian Local Police (Spain), Police Service of Northern Ireland (United Kingdom), Estonian Police and Border Guard Board (Estonia).

Regarding the crime of CB, this team conducted a total of 33 interviews (8 offenders, 12 victims, and 13 experts)⁴¹ and analyzed 46 court sentences⁴². As a result, the team acquired a profound understanding of the issue, which has been used on several occasions throughout the project and documented in the cited technical reports. To interact with the expert knowledge when constructing the causal DAGs, we held discussion sessions with some team members from Universidad Pontificia Comillas, who were the leaders of this work package.

Source link

——————————————————–

Click Here For The Original Source.