When an agency is preparing to implement the Collaborative Problem Solving approach, their leadership teams frequently ask Think:Kids for advice about collecting data. This is an important question, as agencies are being increasingly held accountable for tracking positive outcomes related to new interventions.
If you are an agency using (or thinking about using) CPS, we encourage you to think early and often about program evaluation! Below are three pieces of advice that we find ourselves giving most often:
- Find a comparison group if possible. Sure, you can compare this year’s data (with CPS) to last year’s data (without CPS). However, some will wonder whether there are other time-related changes that might account for your improved results. The best way to show that CPS is resulting in improved outcomes compared to the conventional treatment is to do a direct comparison. You may not be able to randomize children to receive or not receive CPS, however, consider a quasi-experimental design: If your agency has two campuses, you could start the intervention on one campus prior to the other, and track outcomes on both. If you only have one campus, you might collaborate with a local agency that is very similar to yours in order to compare your data with their de-identified comparison data.
- Think longitudinally. Two time points are better than one, and three are often better than two. Most importantly, be sure to collect data prior to starting the CPS intervention, not just afterward!
- Spend time planning what variables are best to measure at your agency. On our website, you will find a list of measures that Think:Kids recommends using to assess outcomes, based on the domain you wish to measure (symptoms, skills, etc.), your setting (inpatient, residential, school, etc.), and from whom you will be collecting data (child, parent, teacher, etc.). This list includes measures that have been used previously to measure change related to use of CPS as well as other measures assessing domains that we feel are relevant. Two such measures, developed at Think:Kids, are also available on this webpage: a research version of the Thinking Skills Inventory (TSI-RS), and a parent-report measure designed to track change over time in family and parenting variables proximal to CPS (TK-COT).
Do you have additional questions about evaluating outcomes related to CPS? If so, contact the Director of Research and Evaluation at Think:Kids.
