Cumulative, open, and accessible.

A living systematic review and meta-analysis for SEB skills

Tommaso Feraco

Background

Soft skills

“Noncognitive skills, a construct that nearly everyone can agree is important but no one can agree on how to measure.” (Harden, 2021).

Putting things together

What they are:

  • Qualities that are beneficial to the individual and their social partners

  • Expressed mostly in relevant situation

  • Distinct from measured intelligence

  • Potentially responsive to interventions

What they are NOT:

  • Personality traits

How to organize them

Despite the differences between these models, their key areas of overlap resemble the Big Five personality traits in terms of their social, emotional, and behavioral content.

  • Breadth

  • Facet-level constructs

  • Allow researchers to readily compare findings regarding skills and traits

From soft to SEB (social, emotional, and behavioral) skills

These represent the abilities to establish and maintain social relationships, regulate emotions, and effectively manage behaviors toward goals and learning and mainly fall into five areas (Soto, Napolitano, and Roberts 2021; Napolitano et al. 2021) that resemble the Big Five.

  • Self-management skills
  • Social engagement skills
  • Cooperation skills
  • Innovation skills
  • Emotional resilience skills

  • Conscientiousness

  • Extraversion

  • Agreeableness

  • Openness

  • Emotional stability

From soft to SEB (social, emotional, and behavioral) skills

Assessing SEB skills

This approach is designed to better align the measurement method with the construct being assessed.

Rationale

State of the things

New opportunities

  • We finally have a new integrative and systematic framework
  • This framework offers a new measurement tool that should better measure skills
  • This framework allows for a distinction between skills and traits

Old problems

  • It is hard to remain up-to-date with the literature

  • Meta-analysis and reviews are static, slow, and specific

HOW CAN WE MAKE THIS NEW LINE OF RESEARCH TRULY RELIABLE, ACCESSIBLE, AND SCIENTIFIC?

The idea

A living systematic review and meta-analysis: By creating an adaptable infrastructure for evidence integration, we hope to support more transparent, efficient, and rigorous scientific progress. This initiative lays the foundation not only for SEB research but also as a model for other emerging fields to follow—advancing toward a new era of living meta-synthesis in psychology.

Structures and aims

  • ‘Systematic’ review: Building a continuously updated and systematic organization of our knowledge about SEB skills, descriptively synthesizing and organizing all the studies conducted that focus on the SEB framework.

    • Measurement

    • Nomological validity

    • Development

    • Applied settings

    • ….

  • Meta-analysis: Develop a meta-analytical database of correlational studies that may facilitate the synthesis of evidence about the association between SEB skills, related constructs, and outcomes.

    • Classic correlational synthesis:

    • SEM synthesis using metaSEM:

  • Open materials: Provide an open-access, regularly updated database of all relevant publications, including metadata on study design, samples, measures, and data availability

Method

see preregistration

Search strategy (WOS + Scopus)

  • Seminal theoretical papers

  • Measurement

    • An Integrative Framework for Conceptualizing and Assessing Social, Emotional, and Behavioral Skills: The BESSI (Soto et al. 2022)

    • The Behavioral, Emotional, and Social Skills Inventory (BESSI): Psychometric Properties of a German-Language Adaptation, Temporal Stabilities of the Skills, and Associations with Personality and Intelligence (Lechner et al. 2022)

    • The Italian Behavioral, Emotional, and Social Skills Inventory (BESSI-I) (Feraco et al. 2024)

    • The Behavioral, Emotional, and Social Skills Inventory: A Spanish Adaptation and Further Validation in Adult Population (Postigo et al. 2024)

  • Short measurement versions

    • Assessing Social, Emotional, and Behavioral Skills in Just a Few Minutes: 96-, 45-, and 20-Item Short Forms of the BESSI (Sewell et al. 2024)

Eligibility criteria - review

  • Review eligibility: Citations are included in the review synthesis if:

    • The main topic of the paper is SEB skills.

    • The paper provides substantial theoretical and quantitative information about SEB skills.

  • Exclusion from the review. Citations are completely excluded from the review synthesis if:

    • They were not focusing on or adopting the SEB skill framework or emerging frameworks based on it. For example, a study focusing on the development of the Big Five should be excluded although it cites the reference papers. However, if it focuses on the development of the Big Five but also collects side data using the BESSI, it will be included in the meta-analysis and -if it also interprets and discuss SEB results- in the review;

    • They were published as books;

    • They were published in languages that were not comprehensible for the authors;

    • Referer to the SEB framework, but focus on different frameworks, such as the OECD or CASEL.

Eligibility criteria - meta-analysis

  • Meta analysis eligibility. Citations are included in the meta-analysis if:

    • The study assessed SEB skills using a validated skill-based assessment referencing to the SEB framework;

    • The study referred to original data that are not already reported in other included studies;

    • The population study was either the general population or a clinical population with mild/moderate mental health issues.

  • Exclusion from the meta analysis. Citations are excluded if:

    • It is a qualitative study;

    • The study does not measure participants’ SEB skills;

    • SEB skills were measured with a non-validated measure or with a measure that does follow the SEB framework;

    • The study does not report correlations at baseline or the data could not be reduced to correlations nor obtained by the authors;

    • It was a study whose data were previously used in a paper already included in the meta-analysis;

    • The study focused on severe mental health issues (e.g., schizophrenia, bipolar disorder);

    • They were published in languages that were not comprehensible for the authors.

Outcomes

This review is not limited to any outcome, but all possible outcomes will be coded and included. Outcomes will be treated in the following way:

  • Broad category: Each outcome is assigned to the a broad category. Examples might be skills, mental health, school, demographic variables, job, personality traits…

  • Second-level category: Subsequently, within each broad category, each outcome is assigned to a more specific category. For instance, the broad ‘traits’ category may include the big five, hexaco, character strengths. School may include academic achievement, learning factors.

  • Specific construct: Finally, each construct is labeled with its own construct label (e.g., extraversion, self-management).

Results

A shiny app!

Open questions

  • Quality assessment?

  • How do we collaborate without using word files?

  • Preregistration

    • Databases

    • String

    • Criteria for review and meta-analysis

  • Publication

    • Who could accept this and in which format?

    • Why should we publish it?

  • Long-term feasibility

    • Can I (we) keep updating it forever?

    • Can we involve the community? How?

Next steps

  • Involve the BESSI team

  • Finalize the preregistration

  • Finalize the app

    • Possibly find a way to add modules for future development

      • Longitudinal / intervention data

      • Non-correlational designs

References

European Commission. 2016. “A New Skills Agenda for Europe.” https://eur-lex.europa.eu/legal-content/EN/TXT/?uri=CELEX%3A52016DC0381.
Feraco, Tommaso, Nicole Casali, Gerardo Pellegrino, Christopher J. Soto, Christopher M. Napolitano, Barbara Carretti, and Chiara Meneghetti. 2024. “The Italian Behavioral, Emotional, and Social Skills Inventory (BESSI-i).” JOURNAL OF PERSONALITY ASSESSMENT 106 (6): 750–64. https://doi.org/10.1080/00223891.2024.2335912.
Feraco, Tommaso, Dario Resnati, Davide Fregonese, Andrea Spoto, and Chiara Meneghetti. 2022. “Soft Skills and Extracurricular Activities Sustain Motivation and Self-Regulated Learning at School.” The Journal of Experimental Education 90 (3): 550–69. https://doi.org/10.1080/00220973.2021.1873090.
Heckman, James J., and Tim Kautz. 2012. “Hard Evidence on Soft Skills.” Labour Economics, European Association of Labour Economists 23rd annual conference, Paphos, Cyprus, 22-24th September 2011, 19 (4): 451–64. https://doi.org/10.1016/j.labeco.2012.05.014.
Lechner, Clemens M., Thomas Knopf, Christopher M. Napolitano, Beatrice Rammstedt, Brent W. Roberts, Christopher J. Soto, and Marion Spengler. 2022. “The Behavioral, Emotional, and Social Skills Inventory (BESSI): Psychometric Properties of a German-Language Adaptation, Temporal Stabilities of the Skills, and Associations with Personality and Intelligence.” JOURNAL OF INTELLIGENCE 10 (3). https://doi.org/10.3390/jintelligence10030063.
Ministry of Education, University and Research, MIUR. 2018. “Linee guida dei percorsi per le competenze trasversali e per l’orientamento - Linee guida dei percorsi per le competenze trasversali e per l’orientamento.” https://www.miur.gov.it/web/guest/-/linee-guida-dei-percorsi-per-le-competenze-trasversali-e-per-l-orientamento.
Napolitano, Christopher M., Madison N. Sewell, Hee J. Yoon, Christopher J. Soto, and Brent. W. Roberts. 2021. “Social, Emotional, and Behavioral Skills: An Integrative Model of the Skills Associated with Success During Adolescence and Across the Life Span.” Frontiers in Education 6. https://www.frontiersin.org/article/10.3389/feduc.2021.679561.
OECD. 2015. “Skills for Social Progress: The Power of Social and Emotional Skills.” https://www.oecd.org/education/skills-for-social-progress-9789264226159-en.htm.
———. 2017. PISA 2015 Results (Volume V): Collaborative Problem Solving. Paris: Organisation for Economic Co-operation; Development. https://www.oecd-ilibrary.org/education/pisa-2015-results-volume-v_9789264285521-en.
———. 2021. OECD Skills Outlook 2021: Learning for Life. OECD Skills Outlook. OECD. https://doi.org/10.1787/0ae365b4-en.
Postigo, Alvaro, Covadonga Gonzalez-Nuevo, Jaime Garcia-Fernandez, Eduardo Garcia-Cueto, Christopher J. Soto, Christopher M. Napolitano, Brent W. Roberts, and Marcelino Cuesta. 2024. “The Behavioral, Emotional, and Social Skills Inventory: A Spanish Adaptation and Further Validation in Adult Population.” ASSESSMENT 31 (7): 1525–47. https://doi.org/10.1177/10731911231225197.
Robles, Marcel M. 2012. “Executive Perceptions of the Top 10 Soft Skills Needed in Today’s Workplace.” Business Communication Quarterly 75 (4): 453–65. https://doi.org/10.1177/1080569912460400.
Sewell, Madison N., Hee J. Yoon, Clemens M. Lechner, Christopher M. Napolitano, Beatrice Rammstedt, Brent W. Roberts, and Christopher J. Soto. 2024. “Assessing Social, Emotional, and Behavioral Skills in Just a Few Minutes: 96-, 45-, and 20-Item Short Forms of the BESSI.” ASSESSMENT. https://doi.org/10.1177/10731911241256434.
Soto, Christopher J., Christopher M. Napolitano, and Brent W. Roberts. 2021. “Taking Skills Seriously: Toward an Integrative Model and Agenda for Social, Emotional, and Behavioral Skills.” Current Directions in Psychological Science 30 (1): 26–33. https://doi.org/10.1177/0963721420978613.
Soto, Christopher J., Christopher M. Napolitano, Madison N. Sewell, Hee J. Yoon, and Brent W. Roberts. 2022. “An Integrative Framework for Conceptualizing and Assessing Social, Emotional, and Behavioral Skills: The BESSI.” JOURNAL OF PERSONALITY AND SOCIAL PSYCHOLOGY 123 (1): 192–222. https://doi.org/10.1037/pspp0000401.
“What Is the CASEL Framework?” n.d. https://casel.org/fundamentals-of-sel/what-is-the-casel-framework/.
World Economic Forum. 2016. “New Vision for Education: Fostering Social and Emotional Learning Through Technology.” https://www.weforum.org/reports/new-vision-for-education-fostering-social-and-emotional-learning-through-technology/.

A living systematic review and meta-analysis for SEB skills