JoVE Logo
Faculty Resource Center

Sign In





Representative Results






Problem-Solving Before Instruction (PS-I): A Protocol for Assessment and Intervention in Students with Different Abilities

Published: September 11th, 2021



1Department of Psychology, Oviedo University
* These authors contributed equally

This protocol guides researchers and educators through implementation of the Problem-Solving before Instruction approach (PS-I) in an undergraduate statistics class. It also describes an embedded experimental evaluation of this implementation, where the efficacy of PS-I is measured in terms of learning and motivation in students with different cognitive and affective predispositions.

Nowadays, how to encourage students' reflective thinking is one of the main concerns for teachers at various educational levels. Many students have difficulties when facing tasks that involve high levels of reflection, such as on STEM (Science, Technology, Engineering and Mathematics) courses. Many also have deep-rooted anxiety and demotivation towards such courses. In order to overcome these cognitive and affective challenges, researchers have suggested the use of "Problem-Solving before Instruction" (PS-I) approaches. PS-I consists of giving students the opportunity to generate individual solutions to problems that are later solved in class. These solutions are compared with the canonical solution in the following phase of instruction, together with the presentation of the lesson content. It has been suggested that with this approach students can increase their conceptual understanding, transfer their learning to different tasks and contexts, become more aware of the gaps in their knowledge, and generate a personal construct of previous knowledge that can help maintain their motivation. Despite the advantages, this approach has been criticized, as students might spend a lot of time on aimless trial and error during the initial phase of solution generation or they may even feel frustrated in this process, which might be detrimental to future learning. More importantly, there is little research about how pre-existing student characteristics can help them to benefit (or not) from this approach. The aim of the current study is to present the design and implementation of the PS-I approach applied to statistics learning in undergraduate students, as well as a methodological approach used to evaluate its efficacy considering students' pre-existing differences.

One of the questions that teachers are most concerned about currently is how to stimulate students' reflection. This concern is common in courses of a mathematical nature, such as STEM courses (Science, Technology, Engineering and Mathematics), in which the abstraction of many concepts requires a high degree of reflection, yet many students report approaching these courses purely through memory-based methods1. In addition, students often show superficial learning of the concepts1,2,3. The difficulties that students experience applying reflection an....

Log in or to access full content. Learn more about your institution’s access to JoVE content here

This protocol follows the Helsinki Declaration of Ethical Principles for Research with Humans, but applies these principles to the added difficulties of integrating research within real-life settings in education32. Specifically, neither the assignment of learning conditions nor the decision to participate can have consequences for students' learning opportunities. In addition, confidentiality and the anonymity of students is maintained even when it is the teachers who are in charge of the eva.......

Log in or to access full content. Learn more about your institution’s access to JoVE content here

This protocol was satisfactorily implemented in a previous study23, with the exception of the measures of students' predispositions in terms of their sense of competence, mastery approach goals, metacognition, and divergent thinking.

To address these predispositions, this protocol includes measures that have been previously validated and that have shown high levels of reliability (Table 1).

Typical solutions generated by .......

Log in or to access full content. Learn more about your institution’s access to JoVE content here

The aim of this protocol is to guide researchers and educators in the implementation and evaluation of the PS-I approach in real classroom contexts. According to some previous experiences, PS-I can help promote deep learning and motivation in students19,21,24, but there is a need for more research about its efficacy in students with different abilities and motivational predispositions14,

Log in or to access full content. Learn more about your institution’s access to JoVE content here

This work was supported by a project of the Principality of Asturias (FC-GRUPIN-IDI/2018/000199) and a predoctoral grant from the Ministry of Education, Culture, and Sports of Spain (FPU16/05802). We would like to thank Stephanie Jun for her help editing the English in the learning materials.


Log in or to access full content. Learn more about your institution’s access to JoVE content here

Name Company Catalog Number Comments
SPSS Program International Business Machines Corporation (IBM) Other programs for general data analysis might be used instead
PROCESS program Andrew F. Hayes (Ohio State University) Freely accesible at: Other programs for mediation, moderation, or conditional process analyses might be used instead
Cognitive Competence Scale in the Survey of Attitudes towards Statistics (SATS-28) Candace Schau (Arizona State University) In case it is used, request should be requested from the author, who whold the copyright
Mastery Approach Scale in the Achievement Goal Questionnaire-Revised Andrew J. Elliot (University of Rochester) In case it is used, request should be requested from the author
Regulation of Cognition Scale of the Metacognitive Awareness Inventory Gregory Schraw (University of Nevada Las Vegas) In case it is used, request should be requested from the creator

  1. Silver, E. A., Kenney, P. A. Results from the seventh mathematics assessment of the National Assessment of Educational Progress. Council of Teachers of Mathematics. , (2000).
  2. OECD. Results (Volume I): Excellence and Equity in Education. PISA, OECD. , (2016).
  3. Mallart Solaz, A. . La resolución de problemas en la prueba de Matemáticas de acceso a la universidad: procesos y errores. , (2014).
  4. García, T., Rodríguez, C., Betts, L., Areces, D., González-Castro, P. How affective-motivational variables and approaches to learning predict mathematics achievement in upper elementary levels. Learning and Individual Differences. 49, 25-31 (2016).
  5. Lai, Y., Zhu, X., Chen, Y., Li, Y. Effects of mathematics anxiety and mathematical metacognition on word problem solving in children with and without mathematical learning difficulties. PloS one. 10 (6), 0130570 (2015).
  6. Ma, X., Xu, J. The causal ordering of mathematics anxiety and mathematics achievement: a longitudinal panel analysis. Journal of Adolescence. 27 (2), 165-179 (2004).
  7. Kapur, M. Productive Failure in Learning Math. Cognitive science. 38 (5), 1008-1022 (2014).
  8. Kirschner, P. A., Sweller, J., Clark, R. E. Why Minimal Guidance During Instruction Does Not Work: An Analysis of the Failure of Constructivist, Discovery, Problem-Based, Experiential, and Inquiry-Based Teaching. Educational Psychologist. 41 (2), 75-86 (2006).
  9. Stockard, J., Wood, T. W., Coughlin, C., Khoury, C. R. The Effectiveness of Direct Instruction Curricula: A Meta-Analysis of a Half Century of Research. Review of educational research. 88 (4), 479-507 (2018).
  10. Clark, R., Kirschner, P. A., Sweller, J. Putting students on the path to learning: The case for fully guided instruction. American Educator. , (2012).
  11. Schwartz, D. L., Martin, T. Inventing to prepare for future learning: The hidden efficiency of encouraging original student production in statistics instruction. Cognition and instruction. 22 (2), 129-184 (2004).
  12. Loibl, K., Rummel, N. The impact of guidance during problem-solving prior to instruction on students' inventions and learning outcomes. Instructional Science. 42 (3), 305-326 (2014).
  13. Kapur, M., Bielaczyc, K. Designing for Productive Failure. Journal of the Learning Sciences. 21 (1), 45-83 (2012).
  14. Glogger-Frey, I., Fleischer, C., Grueny, L., Kappich, J., Renkl, A. Inventing a solution and studying a worked solution prepare differently for learning from direct instruction. Learning and Instruction. 39, 72-87 (2015).
  15. Glogger-Frey, I., Gaus, K., Renkl, A. Learning from direct instruction: Best prepared by several self-regulated or guided invention activities. Learning and Instruction. 51, 26-35 (2017).
  16. Likourezos, V., Kalyuga, S. Instruction-first and problem-solving-first approaches: alternative pathways to learning complex tasks. Instructional Science. 45 (2), 195-219 (2017).
  17. Lamnina, M., Chase, C. C. Developing a thirst for knowledge: How uncertainty in the classroom influences curiosity, affect, learning, and transfer. Contemporary educational psychology. 59, 101785 (2019).
  18. Loibl, K., Rummel, N. Knowing what you don't know makes failure productive. Learning and Instruction. 34, 74-85 (2014).
  19. Weaver, J. P., Chastain, R. J., DeCaro, D. A., DeCaro, M. S. Reverse the routine: Problem solving before instruction improves conceptual knowledge in undergraduate physics. Contemporary educational psychology. 52, 36-47 (2018).
  20. Loibl, K., Roll, I., Rummel, N. Towards a Theory of When and How Problem Solving Followed by Instruction Supports Learning. Educational psychology review. 29 (4), 693-715 (2017).
  21. Darabi, A., Arrington, T. L., Sayilir, E. Learning from failure: a meta-analysis of the empirical studies. Etr&D-Educational Technology Research and Development. 66 (5), 1101-1118 (2018).
  22. Chen, O. H., Kalyuga, S. Exploring factors influencing the effectiveness of explicit instruction first and problem-solving first approaches. European Journal of Psychology of Education. , (2019).
  23. González-Cabañes, E., García, T., Rodríguez, C., Cuesta, M., Núñez, J. C. Learning and Emotional Outcomes after the Application of Invention Activities in a Sample of University Students. Sustainability. 12 (18), 7306 (2020).
  24. Schwartz, D. L., Chase, C. C., Oppezzo, M. A., Chin, D. B. Practicing Versus Inventing With Contrasting Cases: The Effects of Telling First on Learning and Transfer. Journal of educational psychology. 103 (4), 759-775 (2011).
  25. Chase, C. C., Klahr, D. Invention Versus Direct Instruction: For Some Content, It's a Tie. Journal of Science Education and Technology. 26 (6), 582-596 (2017).
  26. Newman, P. M., DeCaro, M. S. Learning by exploring: How much guidance is optimal. Learning and Instruction. 62, 49-63 (2019).
  27. Belenky, D. M., Nokes-Malach, T. J. Motivation and Transfer: The Role of Mastery-Approach Goals in Preparation for Future Learning. Journal of the Learning Sciences. 21 (3), 399-432 (2012).
  28. Bergold, S., Steinmayr, R. The relation over time between achievement motivation and intelligence in young elementary school children: A latent cross-lagged analysis. Contemporary educational psychology. 46, 228-240 (2016).
  29. Mazziotti, C., Rummel, N., Deiglmayr, A., Loibl, K. Probing boundary conditions of Productive Failure and analyzing the role of young students' collaboration. NPJ science of learning. 4, 2 (2019).
  30. Stiglitz, J. E. Las limitaciones del PIB. Investigacion y ciencia. (529), 26-33 (2020).
  31. Holmes, N. G., Day, J., Park, A. H., Bonn, D., Roll, I. Making the failure more productive: scaffolding the invention process to improve inquiry behaviors and outcomes in invention activities. Instructional Science. 42 (4), 523-538 (2014).
  32. Herreras, E. B. La docencia a través de la investigación-acción. Revista Iberoamericana de Educación. 35 (1), 1-9 (2004).
  33. Schau, C., Stevens, J., Dauphinee, T. L., Delvecchio, A. The development and validation of the survey of attitudes toward statistics. Educational and Psychological Measurement. 55 (5), 868-875 (1995).
  34. Elliot, A. J., Murayama, K. On the measurement of achievement goals: Critique, illustration, and application. Journal of educational psychology. 100 (3), 613-628 (2008).
  35. Schraw, G., Dennison, R. S. Assessing metacogntive awareness. Contemporary educational psychology. 19 (4), 460-475 (1994).
  36. Guilford, J. P. . The nature of human intelligence. , (1967).
  37. Zmigrod, L., Rentfrow, P. J., Zmigrod, S., Robbins, T. W. Cognitive flexibility and religious disbelief. Psychological Research-Psychologische Forschung. 83 (8), 1749-1759 (2019).
  38. Wilson, S. Divergent thinking in the grasslands: thinking about object function in the context of a grassland survival scenario elicits more alternate uses than control scenarios. Journal of Cognitive Psychology. 28 (5), 618-630 (2016).
  39. Autin, F., Croizet, J. -. C. Improving working memory efficiency by reframing metacognitive interpretation of task difficulty. Journal of experimental psychology: General. 141 (4), 610 (2012).
  40. Pekrun, R., Vogl, E., Muis, K. R., Sinatra, G. M. Measuring emotions during epistemic activities: the Epistemically-Related Emotion Scales. Cognition and Emotion. 31 (6), 1268-1276 (2017).
  41. Pallant, J. Statistical techniques to compare groups. SPSS survival manual. , 211 (2013).
  42. Pallant, J. Statistical techniques to explore relationships among variables. SPSS survival manual. , 125-149 (2013).
  43. Hayes, A. F. . Introduction to mediation, moderation, and conditional process analysis: A regression-based approach. , (2017).
  44. Kapur, M. Productive failure in learning the concept of variance. Instructional Science. 40 (4), 651-672 (2012).
  45. Nolan, M. M., Beran, T., Hecker, K. G. Surveys Assessing Students' Attitudes Toward Statistics: A Systematic Review of Validity and Reliability. Statistics Education Research Journal. 11 (2), (2012).
  46. Schraw, G., Dennison, R. S. Assessing metacognitive awareness. Contemporary educational psychology. 19 (4), 460-475 (1994).
  47. Dumas, D., Dunbar, K. N. Understanding Fluency and Originality: A latent variable perspective. Thinking Skills and Creativity. 14, 56-67 (2014).
  48. Roberts, R., et al. An fMRI investigation of the relationship between future imagination and cognitive flexibility. Neuropsychologia. 95, 156-172 (2017).
  49. Chamorro-Premuzic, T. Creativity versus conscientiousness: Which is a better predictor of student performance. Applied Cognitive Psychology: The Official Journal of the Society for Applied Research in Memory and Cognition. 20 (4), 521-531 (2006).
  50. Kapur, M. Examining productive failure, productive success, unproductive failure, and unproductive success in learning. Educational Psychologist. 51 (2), 289-299 (2016).

This article has been published

Video Coming Soon

JoVE Logo


Terms of Use





Copyright © 2024 MyJoVE Corporation. All rights reserved