Research Group:
Research on Assessment and Intervention to Support Equity (RAISE) Lab
Despite decades of research, rigorously designed randomized controlled trials, carefully conducted systematic meta-analyses, and national clearinghouses of effective practice (e.g., What Works Clearinghouse), there remains a substantial gap in research that informs and improves educational practice. Most of the relatively recent advances in how research is applied in the educational context, including implementation science, focus on resource intensive (e.g., research-practice partnerships) and time intensive (e.g., technical assistance) processes that when removed often fail to generalize and maintain the initial effectiveness. This focus on should (i.e., research evidence in support of a practice) and how (i.e., implementation science, measurement of fidelity) often ignores the could (i.e., will this work at my school?). Although an intervention or assessment may be considered “research-based”, there will always remain barriers to the generalization of practices to a specific school or population without adaptive tools to localize implementation conditions. Without feasible and applied decision-making tools, educators may have access to research-based interventions and assessments but consistently lack the capacity and knowledge to make an informed decision on how to best use evidence-based methods with direct impacts on youth outcomes. It is critical to understand the facilitators and inhibitors to use of evidence within decision-making—this is an essential link from what works to how it works and for whom.
My research program seeks to promote equitable outcomes through improving educational decision-making by understanding the influence of external policies and contextual drivers and developing novel solutions to improve the selection and use of preventative school mental health assessments and interventions. The research program consists of three distinct, yet conceptually related lines of inquiry:
(1) An examination of external policies such as high-stakes test-based accountability on student and teacher emotional wellbeing as well as instructional practices;
(2) The development of effective and efficient decision-making tools including universal screening assessments and progress monitoring measures;
(3) The creation of evidence-based decision-making models including teacher trainings in assessment use and data interpretation, multi-informant decision-making algorithms, and discrete event simulations.
Students in the RAISE lab will have multiple opportunities for research experiences across lines, as well as mentorship, grant writing, and publication activities. Interested students are encouraged to review the RAISE lab manual that describes my mentorship style, areas of development, and yearly expectations. In addition, students will have opportunities to participate within my various service commitments including reviewing manuscripts for Journal of School Psychology and School Psychology Review, and advocacy and government relations for the FASP Government and Professional Relations committee.
Ongoing Research Projects:
*=student or post-doc co-author, **=invited commentary
1. *Mennes, H., von der Embse, N., Kim, E., *Sundar, R., *Hines, D., & *Welliver, M. (accepted with revision). Are "well" teachers "better" teachers?: The relationship between first year teacher emotion and use of evidence-based instructional strategies. School Psychology.
2. Ruan, L., Pendergast, L., Liao, P., Jones, P., von der Embse, N., Innamorati, M., & Balsamo, M. (in press). Measuring Depression in Young Adults: Preliminary Development of an English Version of the Teate Depression Inventory. International Journal of Environmental Research and Public Health.
3. Kim, E. & von der Embse, N. (in press). Brief Report: Polynomial Regression Mixture Modeling for Heterogeneous Effects of Informant Congruence. Journal of Experimental Education.
4. von der Embse, N. & De Los Reyes, A. (in press). Advancing equity in access to school mental health through multiple informant decision-making. Journal of School Psychology.
5. von der Embse, N., Kim, E., *Ross, D., Kilgus, S., & *Koza, T. (2023). Multi-informant assessment of internalizing concerns: Rater concordance and implications for decision-making. Journal of Psychopathology and Behavioral Assessment, 45, 234-346. Doi: 10.1007/s10862-023-10026-y.
6. De Los Reyes, A., Epkins, C.C., Asmundson, G.J.G., Augenstein, T.M., Becker, K.D., Becker, S.P., … von der Embse, N., …Youngstrom, E.A. (2023). Editorial statement about JCCAP’s 2023 special issue on informant discrepancies in youth mental health assessments: Observations, guidelines, and future directions grounded in 60 years of research. Journal of Clinical Child and Adolescent Psychology, 52(1).
7. *Drymond, M., *Sanchez, A., von der Embse, N., *Francis, G., *Ross, D., & *Khallaghi, S. (2023). Universal screening in early childhood populations: A systematic review. Perspectives on Early Childhood Psychology and Education: Vol. 7: Iss. 1, Article 6.
8. Kilgus, S. P., Eklund, K., von der Embse, N. P., *Zahn, M., *Meyer, L., *Peet, C., & *Willenbrink, J. (2022). Educator training to support the collection and use of Intervention Selection Profile-Function data: Two randomized controlled trials. Journal of School Psychology.
9. Eklund, K., Kilgus, S., *Zahn, M., von der Embse, N., *Willenbrink, J., *Davis, E., & *Twombly, T. (2022). Intervention Selection Profile-Function: An examination of decisional accuracy relative to traditional FBA data. School Psychology. Doi: 10.1037/spq0000524
10. *Ross, D., von der Embse, N., *Andrews, J., Headley, M., & *Mierzwa, C. (2022). A systematic review of threat assessment trainings in K-12 schools. Journal of School Violence, 21, 444-458. Doi: 10.1080/15388220.2022.2108434
11. von der Embse, N.P., Kilgus, S.P., Eklund, K., *Zahn, M., *Peet, C., & *Durango, S. (2022). Promoting effective decision-making: Training educators to collect and use social-emotional skill assessment data. School Psychology Review, 51(5), 574-588. doi: 10.1080/2372966X.2020.1827680
12. Putwain, D.W., Loderer, K., von der Embse, N.P., Suldo, S. & *Daumiller, M. (2021). Test anxiety, anxiety disorders, and school-related wellbeing: Manifestations of the same or different constructs? Journal of School Psychology, 88, 47-67. doi.org/10.1016/j.jsp.2021.08.001.
13. *Jenkins, A., *Thoman, S., *Wang, Y., von der Embse N.P., Kilgus, S., & *Reynolds, F. (2021). Identifying type and stability of emotional and behavioral risk. Psychology in the Schools, 58, 225-2237. doi.org/10.1002/pits.22587.
14. von der Embse, N.P., Putwain, D. & *Francis, G. (2021). Interpretation and Use of the Multidimensional Test Anxiety Scale. School Psychology, 36(2), 86–96. doi.org/10.1037/spq0000427
15. Canivez, G., von der Embse, N.P., & McGill R. (2021). Construct validity of the BASC-3 Teacher Rating Scales: Independent hierarchical exploratory factor analyses with the standardization sample. School Psychology, 36(4), 235–254. doi.org/10.1037/spq0000444
16. Kim, E. & von der Embse, N.P. (2021). Combination of variable-centered and person-centered approaches to multi-informant data: Trifactor mixture model. Educational and Psychological Measurement, 81 (4), 728-755. doi: 10.1177/0013164420973722.
17. von der Embse, N.P., Kim, E., *Jenkins, A., *Sanchez, A., Kilgus, S., & Eklund, K. (2021). Profiles of rater dis/agreement within universal screening in predicting distal outcomes. Journal of Psychopathology and Behavioral Assessment, 43(3), 632-645. doi: 10.1007/s10862-021-09869-0
18. Putwain, D.W., *Gallard, D., *Beaumont, J., Loderer, K. & von der Embse, N.P. (2021). Does test anxiety predispose poor school-related wellbeing and enhanced risk of emotional disorders? Cognitive Research and Therapy, 45(6), 1150-1162. doi: 10.1007/s10608-021-10211-x
19. von der Embse, N.P., *Jenkins, A., Christensen, K., Kilgus, S., *Mishra, M., & *Chin, B. (2021). Evaluating the cost of prevention programming and universal screening with discrete event simulation. Administration and Policy in Mental Health, 48, 962–973. doi: 10.1007/s10488-021-01108-8
20. *Sanchez, A., *Latimer, J., *Scarimbolo, K., von der Embse, N. P., Suldo, S., & *Salvatore, C., (2021). Youth Mental Health First Aid (Y-MHFA) trainings for Educators: A Systematic Review. School Mental Health, 13 (1), 1-12. doi: 10.1007/s12310-020-09393-8
21. von der Embse, N.P., & *Mankin, A. (2021). Changes in teacher stress and wellbeing throughout the academic year. Journal of Applied School Psychology, 37(2), 165-184. doi: 10.1080/15377903.2020.1804031
22. Kilgus, S.P., Eklund, K., von der Embse, N., *Weist, M., *Barber, A., *Kaul, M., & *Dodge, S. (2021). Structural validity and reliability of Social, Academic, and Emotional Behavior Risk Screener – Student Rating Scale (mySAEBRS) scores: A replication study. Assessment for Effective Intervention, 46 (4), 259-269. doi: 10.1177/1534508420909527.
23. von der Embse, N.P., *Jenkins, A., *West, G., Eklund, K., Kilgus, S. P., & *Morgan, M. (2021). Comparing teacher and student report of behavioral risk in predicting elementary student math outcomes. Assessment for Effective Intervention, 46 (4), 304-309. doi: 10.1177/1534508419885016.
My research program seeks to promote equitable outcomes through improving educational decision-making by understanding the influence of external policies and contextual drivers and developing novel solutions to improve the selection and use of preventative school mental health assessments and interventions. The research program consists of three distinct, yet conceptually related lines of inquiry:
(1) An examination of external policies such as high-stakes test-based accountability on student and teacher emotional wellbeing as well as instructional practices;
(2) The development of effective and efficient decision-making tools including universal screening assessments and progress monitoring measures;
(3) The creation of evidence-based decision-making models including teacher trainings in assessment use and data interpretation, multi-informant decision-making algorithms, and discrete event simulations.
Students in the RAISE lab will have multiple opportunities for research experiences across lines, as well as mentorship, grant writing, and publication activities. Interested students are encouraged to review the RAISE lab manual that describes my mentorship style, areas of development, and yearly expectations. In addition, students will have opportunities to participate within my various service commitments including reviewing manuscripts for Journal of School Psychology and School Psychology Review, and advocacy and government relations for the FASP Government and Professional Relations committee.
Ongoing Research Projects:
- Measurement Grant from the Institute for Education Sciences entitled, “Improving the Delivery of Tiered Intervention Through the Development of Multi-Informant Rating Systems” (Year 3 of 5)
- Grant from the Substance Abuse and Mental Health Service Agency entitled, “Project Florida SMARTS: School Mental health Assessment, Response, and Training for Suicide prevention” (Year 2 of 6)
- Grant from the US Department of Justice, “Project SAFETY-2: School Assessment for Evaluation of Threat to Youth” (Year 2 of 4)
- Grant from FL Department of Education, “Development of a statewide resilience assessment” (ongoing)
- Grant from FL Department of Education, “Development of a mental health awareness training for educators” (Year 1 of 5)
- Grant from FL Department of Education, “Development of a resilience coaching training” (Year 1 of 2)
- Development of universal screening tools in early childhood populations in collaboration with Pyramid Model Center at USF
- Development of cost-effectiveness and implementation simulation tools with colleagues in Public Health and Computer Science
- International collaboration opportunities with colleagues in Germany and England on mental health screening, test anxiety, and educator emotional wellbeing
*=student or post-doc co-author, **=invited commentary
1. *Mennes, H., von der Embse, N., Kim, E., *Sundar, R., *Hines, D., & *Welliver, M. (accepted with revision). Are "well" teachers "better" teachers?: The relationship between first year teacher emotion and use of evidence-based instructional strategies. School Psychology.
2. Ruan, L., Pendergast, L., Liao, P., Jones, P., von der Embse, N., Innamorati, M., & Balsamo, M. (in press). Measuring Depression in Young Adults: Preliminary Development of an English Version of the Teate Depression Inventory. International Journal of Environmental Research and Public Health.
3. Kim, E. & von der Embse, N. (in press). Brief Report: Polynomial Regression Mixture Modeling for Heterogeneous Effects of Informant Congruence. Journal of Experimental Education.
4. von der Embse, N. & De Los Reyes, A. (in press). Advancing equity in access to school mental health through multiple informant decision-making. Journal of School Psychology.
5. von der Embse, N., Kim, E., *Ross, D., Kilgus, S., & *Koza, T. (2023). Multi-informant assessment of internalizing concerns: Rater concordance and implications for decision-making. Journal of Psychopathology and Behavioral Assessment, 45, 234-346. Doi: 10.1007/s10862-023-10026-y.
6. De Los Reyes, A., Epkins, C.C., Asmundson, G.J.G., Augenstein, T.M., Becker, K.D., Becker, S.P., … von der Embse, N., …Youngstrom, E.A. (2023). Editorial statement about JCCAP’s 2023 special issue on informant discrepancies in youth mental health assessments: Observations, guidelines, and future directions grounded in 60 years of research. Journal of Clinical Child and Adolescent Psychology, 52(1).
7. *Drymond, M., *Sanchez, A., von der Embse, N., *Francis, G., *Ross, D., & *Khallaghi, S. (2023). Universal screening in early childhood populations: A systematic review. Perspectives on Early Childhood Psychology and Education: Vol. 7: Iss. 1, Article 6.
8. Kilgus, S. P., Eklund, K., von der Embse, N. P., *Zahn, M., *Meyer, L., *Peet, C., & *Willenbrink, J. (2022). Educator training to support the collection and use of Intervention Selection Profile-Function data: Two randomized controlled trials. Journal of School Psychology.
9. Eklund, K., Kilgus, S., *Zahn, M., von der Embse, N., *Willenbrink, J., *Davis, E., & *Twombly, T. (2022). Intervention Selection Profile-Function: An examination of decisional accuracy relative to traditional FBA data. School Psychology. Doi: 10.1037/spq0000524
10. *Ross, D., von der Embse, N., *Andrews, J., Headley, M., & *Mierzwa, C. (2022). A systematic review of threat assessment trainings in K-12 schools. Journal of School Violence, 21, 444-458. Doi: 10.1080/15388220.2022.2108434
11. von der Embse, N.P., Kilgus, S.P., Eklund, K., *Zahn, M., *Peet, C., & *Durango, S. (2022). Promoting effective decision-making: Training educators to collect and use social-emotional skill assessment data. School Psychology Review, 51(5), 574-588. doi: 10.1080/2372966X.2020.1827680
12. Putwain, D.W., Loderer, K., von der Embse, N.P., Suldo, S. & *Daumiller, M. (2021). Test anxiety, anxiety disorders, and school-related wellbeing: Manifestations of the same or different constructs? Journal of School Psychology, 88, 47-67. doi.org/10.1016/j.jsp.2021.08.001.
13. *Jenkins, A., *Thoman, S., *Wang, Y., von der Embse N.P., Kilgus, S., & *Reynolds, F. (2021). Identifying type and stability of emotional and behavioral risk. Psychology in the Schools, 58, 225-2237. doi.org/10.1002/pits.22587.
14. von der Embse, N.P., Putwain, D. & *Francis, G. (2021). Interpretation and Use of the Multidimensional Test Anxiety Scale. School Psychology, 36(2), 86–96. doi.org/10.1037/spq0000427
15. Canivez, G., von der Embse, N.P., & McGill R. (2021). Construct validity of the BASC-3 Teacher Rating Scales: Independent hierarchical exploratory factor analyses with the standardization sample. School Psychology, 36(4), 235–254. doi.org/10.1037/spq0000444
16. Kim, E. & von der Embse, N.P. (2021). Combination of variable-centered and person-centered approaches to multi-informant data: Trifactor mixture model. Educational and Psychological Measurement, 81 (4), 728-755. doi: 10.1177/0013164420973722.
17. von der Embse, N.P., Kim, E., *Jenkins, A., *Sanchez, A., Kilgus, S., & Eklund, K. (2021). Profiles of rater dis/agreement within universal screening in predicting distal outcomes. Journal of Psychopathology and Behavioral Assessment, 43(3), 632-645. doi: 10.1007/s10862-021-09869-0
18. Putwain, D.W., *Gallard, D., *Beaumont, J., Loderer, K. & von der Embse, N.P. (2021). Does test anxiety predispose poor school-related wellbeing and enhanced risk of emotional disorders? Cognitive Research and Therapy, 45(6), 1150-1162. doi: 10.1007/s10608-021-10211-x
19. von der Embse, N.P., *Jenkins, A., Christensen, K., Kilgus, S., *Mishra, M., & *Chin, B. (2021). Evaluating the cost of prevention programming and universal screening with discrete event simulation. Administration and Policy in Mental Health, 48, 962–973. doi: 10.1007/s10488-021-01108-8
20. *Sanchez, A., *Latimer, J., *Scarimbolo, K., von der Embse, N. P., Suldo, S., & *Salvatore, C., (2021). Youth Mental Health First Aid (Y-MHFA) trainings for Educators: A Systematic Review. School Mental Health, 13 (1), 1-12. doi: 10.1007/s12310-020-09393-8
21. von der Embse, N.P., & *Mankin, A. (2021). Changes in teacher stress and wellbeing throughout the academic year. Journal of Applied School Psychology, 37(2), 165-184. doi: 10.1080/15377903.2020.1804031
22. Kilgus, S.P., Eklund, K., von der Embse, N., *Weist, M., *Barber, A., *Kaul, M., & *Dodge, S. (2021). Structural validity and reliability of Social, Academic, and Emotional Behavior Risk Screener – Student Rating Scale (mySAEBRS) scores: A replication study. Assessment for Effective Intervention, 46 (4), 259-269. doi: 10.1177/1534508420909527.
23. von der Embse, N.P., *Jenkins, A., *West, G., Eklund, K., Kilgus, S. P., & *Morgan, M. (2021). Comparing teacher and student report of behavioral risk in predicting elementary student math outcomes. Assessment for Effective Intervention, 46 (4), 304-309. doi: 10.1177/1534508419885016.