The Problem-Solving Analysis Protocol (P-SAP) is a written problem-solving protocol for assessing problem solving skills that can be easily integrated into the normal activities of a class (Steinke & Fitch, 2003). We created the P-SAP based on the work of researchers on cognitive outcomes of service-learning (Eyler & Giles, 1999) and the reflective judgment framework of intellectual development (King & Kitchener, 1994). The P-SAP presents a real-world issue to the student that is directly relevant to the application of material the student is learning in the course, and by simply changing the issue the protocol can be used in a wide variety of classes. Students answer a series of questions about the causes, consequences and solutions for a problem that arises from the issue. The problem-solving protocol can be used in class as a graded assignment or exam question or as a class exercise to start discussion. The P-SAP is especially designed to help measure cognitive skills developed in academic experiential learning activities such as service-learning but can be adapted for other uses. The P-SAP has demonstrated good inter-rater reliability and construct validity with intellectual development and cognitive learning measures.
The P-SAP allows two different uses for assessment purposes. First, whether the protocol is used as a graded assignment or not, faculty in the discipline can score a sample of protocols for students’ comprehension and application of content knowledge. Many departmental program assessment plans include outcomes about students’ ability to apply knowledge but faculty members have difficulty identifying how to assess application.
Second, assessment teams can also code the same protocol for more general problem-solving skills as related to other intellectual skills such as critical thinking, knowledge transfer and perspective taking. This second application can be scored using the P-SAP rubrics. The P-SAP rubrics measure locus and complexity of reasoning in relationship to problem-solving. If you are using the P-SAP for outcomes assessment you will probably want to use the global coding rubrics which are easier to score. If you are using the P-SAP for research, you may either want to use the global rubrics or use the full set of rubrics for scoring. The full set of P-SAP rubrics provides scoring criteria for the two dimensions (locus/source and complexity) separately for each of the four questions in the protocol (questions about problem, cause, solution and analysis of solution). See examples of low, medium, and high coding for locus and complexity using the global coding rubric with problems from educational psychology and child development courses.