Pragmatic Fidelity Measurement in Youth Service Settings

Lu Wang, Samantha J. Stoll, Christopher J. Eddy, Sarah Hurley, Jocelyn Sisson, Nicholas Thompson, Jacquelyn N. Raftery-Helmer, J. Stuart Ablon, and Alisha R. Pollastri

Developing brief, easy-to-use, and reliable tools to measure how well providers deliver evidence-based treatments in community clinical settings is critical to ensure the benefits of these treatments. However, reliable tools are often too time-consuming and not feasible to use in community settings because they require independent observers to receive intensive training on a coding system and to observe live or recorded treatment sessions for reliable and accurate evaluation.

This paper describes steps Think:Kids took to develop a more practical measure of how well providers deliver one evidence-based treatment, Collaborative Problem Solving (CPS), based on a previously validated measure, to explore whether the quality of the measure can be maintained while reducing the need for training independent observers and the need for recording treatment sessions. This work contributes to the growing efforts of developing more pragmatic fidelity measures and introduces a new tool, the CPS Practice Integrity Form (CPS-PIF), as a promising measure for community-based clinical settings using CPS.

The CPS Practice Integrity Form (CPS-PIF) allows us to measure whether CPS learners are doing CPS the way it is intended to be done. That is important when we are implementing in service settings (to see which learners may need more support) and when we are doing studies on CPS outcomes. The CPS-PIF is a relatively simple tool that can be completed by CPS experts from Think:Kids or by CPS Certified staff in organizations implementing the approach.