The development of a national e-test on science competence for the third school level: An assessment to support learning

Authors

  • Katrin Vaino
  • Triin Rosin
  • Ülle Liiber
  • Regina Soobard
  • Moonika Teppo
  • Ana Valdmann
  • Elle Reisenbuk
  • Miia Rannikmäe

DOI:

https://doi.org/10.12697/eha.2024.12.1.05

Keywords:

science competence, diagnostic e-test, feedback, design-based research, context-based

Abstract

Selle disainipõhise uuringu eesmärk oli välja töötada e-tasemetöö, millega saaks hinnata riiklikus õppekavas määratletud III kooliastme lõpuks omandatud loodusteadusliku pädevuse taset. Ühtlasi pidi tasemetöö andma kirjeldavat tagasisidet nii õpilasele, õpetajale, lapsevanemale kui ka haridusüldsusele. Protsessi komplekssusest tingituna keskendutakse selles artiklis kitsamalt tasemetöö disainimise protsessile ja saadud lõpptulemusele ning põhjendatakse tehtud disainiotsuseid. Uurijate ja praktikute vahelises koostöös (2018–2022) jõuti uudse lahenduse – kontekstipõhise tasemetööni, mida saab kasutada üle-eestiliseks põhiuuringuks. Õpilaste saavutatud loodusteadusliku pädevuse taset kirjeldatakse neljal tasemel üheksa tunnuse abil, mis on ülevaatlikkuse nimel rühmitatud omakorda neljaks: loodusteaduslikud teadmised, uurimuslikud oskused, probleemi lahendamise ja otsuse tegemise oskused ning kommunikatsioonioskused. Tasemetöö koosneb neljast alatestist ning kokku 35 ülesandest.

Summary

Downloads

Download data is not yet available.

References

Aksen, M., Jürimäe, M., Nõmmela, K., Saarsen, K., Sillak, S., Eskor, J., Vool, E., & Urmann, H. (2018). Eesti üldhariduskoolides kasutatavad hindamissüsteemid. Tartu Ülikool. https://www.hm.ee/sites/default/files/uuringud/hindamine_lopparuanne_15.okt_loplik.pdf.

Anderson, L. W., & Krathwohl, D. R. (toim) (2001). A Taxonomy for Learning, Teaching, and Assessing: A Revision of Bloom’s Taxonomy of Educational Objectives. New York: Addison Wesley Longman, Inc.

Anderson, T., & Shattuck, J. (2012). Design-based research: A decade of progress in education research? Educational Researcher, 41(1), 16–25. https://doi.org/10.3102/0013189X11428813

Assessment Reform Group (1999). Assessment for Learning: Beyond the Black Box. Cambridge: University of Cambridge School of Education.

Baker, P., & Schmude, M. (2022). Structure of the observed learning outcomes (SOLO) model: A mixed-method systematic review of research in mathematics education. Eurasia Journal of Mathematics, Science and Technology Education, 18(6), 1–17. https://doi.org/10.29333/ejmste/12087

Barab, S., & Squire, K. (2004). Design-based research: Putting a stake in the ground. The Journal of the Learning Sciences, 13(1), 1–14. https://doi.org/10.1207/s15327809jls1301_1

Biggs, J. B., & Collis, K. F. (2014). Evaluating the Quality of Learning: The SOLO Taxonomy (Structure of the Observed Learning Outcome). Academic Press.

Bolhuis, S. (2003). Towards process-oriented teaching for self-directed lifelong learning: A multidimensional perspective. Learning and Instruction, 13(3), 327–347. https://doi.org/10.1016/S0959-4752(02)00008-7

Bond, T. (2015). Applying the Rasch Model: Fundamental Measurement in the Human Sciences (3rd ed.). Routledge.

Boyle, A., & Hutchison, D. (2009). Sophisticated tasks in e-assessment: What are they and what are their benefits?. Assessment & Evaluation in Higher Education, 34(3), 305–319. https://doi.org/10.1080/02602930801956034

Champagne, A. B. (2013). Content to be assessed across the history of the national assessment of educational progress. D. Corrigan, R. Gunstone, & A. Jones (Eds.), Valuing Assessment in Science Education: Pedagogy, Curriculum, Policy. (pp. 119–151). Dordrecht: Springer Netherlands. https://doi.org/10.1007/978-94-007-6668-6_7

Choi, K., Lee, H., Shin, N., Kim, S.-W., & Krajcik, J. (2011). Re-conceptualization of scientific literacy in South Korea for the 21st century. Journal of Research in Science Teaching, 48(6), 670–697. https://doi.org/10.1002/tea.20424

Cobb, P., diSessa, A., Lehrer, R., & Schauble, L. (2003). Design experiments in educational research. Educational Researcher, 32(1), 9–13. https://doi.org/10.3102/0013189X032001009

Csapó, B., & Molnár, G. (2019). Online diagnostic assessment in support of personalized teaching and learning: The eDia system. Frontiers in Psychology, 10 (1522), 1–14. https://doi.org/10.3389/fpsyg.2019.01522

Daniels, H. (2007). Pedagogy. In H. Daniels, M. Cole, & J. V. Wertsch (Eds.), The Cambridge companion to Vygotsky. (pp 307–331). Cambridge University Press. https://doi.org/10.1017/CCOL0521831040.013

Fensham, P. J., & Rennie, L. J. (2013). Towards an authentically assessed science curriculum. D. Corrigan, R. Gunstone, & A. Jones (Eds.), Valuing Assessment in Science Education: Pedagogy, Curriculum, Policy (pp. 69–100). Dordrecht: Springer. https://doi.org/10.1007/978-94-007-6668-6_5

Finn, B. (2015). Measuring motivation in low-stakes assessments. ETS Research Report Series, 2015(2), 1–17. https://doi.org/10.1002/ets2.12067

Gaffney, T. W., Cudeck, R., Ferrer, E., & Widaman, K. F. (2010). On the factor structure of standardized educational achievement tests. Journal of Applied Measurement, 11(4), 384.

Gweon, H., & Schonlau, M. (2022). Automated classification for open-ended questions with BERT. arXiv preprint arXiv:2209.06178.

Haridus- ja Noorteamet (Harno) (2023a). Tasemetööd. https://harno.ee/tasemetood.

Haridus- ja Noorteamet (Harno) (2023b). III kooliastme testi materjalid 2022: Loodusteadusliku kirjaoskuse komponendid ja tasemed koos näidisülesannetega III kooliastmes. https://projektid.edu.ee/pages/viewpage.action?pageId=132157214.

Haridus- ja Teadusministeerium (2014). Eesti elukestva õppe strateegia 2020. https://www.haridusfoorum.ee/images/haridusstrateegia/Eesti_elukestva_oppe_strateegia_loplik.pdf.

Harvey, N. D. (2021). A Simple guide to inter-rater, intra-rater and test-retest reliability for animal behaviour studies. https://doi.org/10.31219/osf.io/8stpy

Hattie, J., & Timperley, H. (2007). The power of feedback. Review of Educational Research, 77(1), 81–112. https://doi.org/10.3102/003465430298487

Henno, I., Kollo, L., & Mikser, R. (2017). Eesti loodusainete õpetajate uskumused, õpetamispraktika ja enesetõhusus TALIS 2008 ja 2013 uuringu alusel. Eesti Haridusteaduste Ajakiri, 5(1), 268–296. https://doi.org/10.12697/eha.2017.5.1.09

Hsieh, H.-F., & Shannon, S. E. (2005). Three approaches to qualitative content analysis. Qualitative Health Research, 15(9), 1277–1288. https://doi.org/10.1177/1049732305276687

Höttecke, D. & Allchin, D. (2020). Reconceptualizing nature-of-science education in the age of social media. Science Education, 104(4), 641–666. https://doi.org/10.1002/sce.21575

Jonassen, D.H. (2011). Learning to Solve Problems: A Handbook for Designing Problem-Solving Learning Environments. New York: Routledge.

Kline, P. (1986). A Handbook of Test Construction: Introduction to Psychometric Design, Methuen, London.

Loogma, K., Erss, M., Ümarik, M., & Aasa, M. (2020). Õpetaja professionalismi võimalikud tulevikustsenaariumid aastaks 2035. Eesti Haridusteaduste Ajakiri, 8(1), 180–212. https://doi.org/10.12697/eha.2020.8.1.08

OECD (2018). The future of education and skills: Education 2030. Paris: OECD Publishing.

OECD (2019). TALIS 2018 Results (Volume I): Teachers and school leaders as lifelong learners, TALIS, OECD Publishing. https://doi.org/10.1787/1d0bc92a-en

OECD (2020). PISA 2024 strategic vision and direction for science (Issue March). https://www.oecd.org/pisa/publications/PISA-2024-Science-Strategic-Vision-Proposal.pdf.

OECD (2021). The assessment frameworks for cycle 2 of the programme for the international assessment of adult competencies, OECD Skills Studies, OECD Publishing, Paris.

Osborne, J., Pimentel, D., Alberts, B., Allchin, D., Barzilai, S., Bergstrom, C., Coffey, J., Donovan, B., Kivinen, K., Kozyreva. A., & Wineburg, S. (2022). Science Education in an Age of Misinformation. Stanford University, Stanford, CA.

Pedaste, M., Baucal, A., & Reisenbuk, E. (2021). Towards a science inquiry test in primary education: Development of items and scales. International Journal of STEM Education, 8, 1–19. https://doi.org/10.1186/s40594-021-00278-z

Pedaste, M., Brikker, M., Rannikmäe, M., Soobard, R., Mäeots, M., & Reiska, P. (2017). Loodusvaldkonna õpitulemuste hindamine. Raport, Tartu.

Pereira, T., Amaral, A., & Mendes, I. (2022). A Competency definition based on the knowledge, skills, and human dispositions constructs. In International Conference on Internet of Everything (pp. 29–38). Cham: Springer Nature Switzerland. https://doi.org/10.1007/978-3-031-25222-8_3

Põhikooli riiklik õppekava (2023). https://www.riigiteataja.ee/akt/108032023005.

Põhikooli riiklik õppekava (2023). Lisa 4. https://www.riigiteataja.ee/aktilisa/1080/3202/3005/18m_pohi_lisa4.pdf#.

Rannikmäe, M., Vaino, K., Teppo, M., Soobard, R., Rosin, T., & Must, O. (2018). Loodusvaldkonna õpitulemuste hindamine III kooliastmes. Raport. Tartu.

Rannikmäe, M., Soobard, R., Vaino, K., Rosin, T., Teppo, M., & Must, O. (2019). Loodusvaldkonna õpitulemuste hindamine III–IV kooliastmes. Raport. Tartu.

Rannikmäe, M., Soobard, R., & Vaino, K. (2020). Loodusvaldkonna õpitulemuste e-hindamine põhikooli kolmandas astmes ja gümnaasiumis. Kontseptsioon. Tartu.

Rannikmäe, M., Soobard, R., Vaino, K., & Rosin, T. (2021a). Loodusvaldkonna õpitulemuste e-hindamine põhikooli kolmandas astmes ja gümnaasiumis. Kontseptsioon. Tartu.

Rannikmäe, M., Soobard, R., Vaino, K., Teppo, M., Valdmann, A., & Rosin, T. (2021b). Loodusvaldkonna õpitulemuste hindamine. Raport. Tartu.

Rannikmäe, M., Vaino, K., Soobard, R., Teppo, M., & Reisenbuk, E. (2023). Lühikokkuvõte 2022/2023. õppeaasta loodusainete III kooliastme katselise tasemetöö tulemustest.

Reeves, T. (2006). Design research from a technology perspective. In J. van den Akker, K. Gravemeijer, S. McKenney, & N. Nieveen (Eds.), Educational design research (pp. 52–66). London: Routledge.

Rosin, T., Vaino, K., Soobard, R., & Rannikmäe, M. (esitatud). Understanding science teachers’ beliefs about the teaching and assessment of scientific competences: Explaining the reasons for implementing or not implementing science e-test feedback.

Saks, K., & Leijen, Ä. (2015). Kognitiivsete ja metakognitiivsete õpistrateegiate toetamine tehnoloogiaga tõhustatud keeleõppes. Eesti Haridusteaduste Ajakiri, 3(2), 130–155. https://doi.org/10.12697/eha.2015.3.2.05

Schut, S., Heeneman, S., Bierer, B., Driessen, E., van Tartwijk, J., & van Der Vleuten, C. (2020). Between trust and control: Teachers’ assessment conceptualisations within programmatic assessment. Medical Education, 54(6), 528–537. https://doi.org/10.1111/medu.14075

Sjöstrom, J., & Talanquer, V. (2014). Humanizing chemistry education: From simple contextualization to multifaceted problematization. Journal of Chemical Education, 91(8), 1125–1131. https://doi.org/10.1021/ed5000718

Zimmerman, B. J. (1989). A social cognitive view of self-regulated academic learning. Journal of Educational Psychology, 81(3), 329–339. https://doi.org/10.1037/0022-0663.81.3.329

Zhang, L., Huang, Y., Yang, X., Yu, S., & Zhuang, F. (2022). An automatic short-answer grading model for semi-open-ended questions. Interactive Learning Environments, 30(1), 177–190. https://doi.org/10.1080/10494820.2019.1648300

Zheng, L. (2015). A systematic literature review of design-based research from 2004 to 2013. Journal of Computers in Education, 2, 399–420. https://doi.org/10.1007/s40692-015-0036-z

Thier, M., & Daviss, B. (2002). The New Science Literacy: Using Language Skills to Help Students Learn Science. Portsmouth, NH: Heinemann.

Tinoca, L., Piedade, J., Santos, S., Pedro, A., & Gomes, S. (2022). Design-based research in the educational field: A systematic literature review. Education Sciences, 12(6), 410. https://doi.org/10.3390/educsci12060410

Tire, G., Puksand, H., Lepmann, T., Henno, I., Lindemann, K., Täht, K., Lorenz, B., & Silm, E. (2019). PISA 2018 Eesti tulemused: Eesti 15-aastaste õpilaste teadmised ja oskused funktsionaalses lugemises, matemaatikas ja loodusteadustes. https://www.innove.ee/uuringud/pisa-uuring/pisa-2018/.

The Design-Based Research Collective. (2003). Design-based research: An emerging paradigm for educational inquiry. Educational Researcher, 32(1), 5–8. https://doi.org/10.3102/0013189X032001005

Toomaneejinda, A. (2017). Zone of proximal development, dynamic assessment and learner empowerment. LEARN Journal: Language Education and Acquisition Research Network, 10(1), 176–185.

Valdmann, A., Holbrook, J., & Rannikmäe, M. (2012). Evaluating the teaching impact of a prior context-based, professional development programme. Science Education International, 23(2), 166–185.

Van den Akker, J., Gravemeijer, K., McKenney, S., & Nieveen, N. (2006). Educational Design Research. Routledge.

Vitello, S., & Williamson, J. (2017). Internal versus external assessment in vocational qualifications: A commentary on the government’s reforms in England. London Review of Education, 15(3), 536–548. https://doi.org/10.18546/LRE.15.3.14

Vygotsky, L. S., & Cole, M. (1978). Mind in Society: Development of Higher Psychological Processes. Harvard University Press.

Wainer, H., Sheehan, K. M., & Wang, X. (2000). Some paths towards making Praxis scores more useful. Journal of Educational Measurement, 37, 113–140. https://doi.org/10.1111/j.1745-3984.2000.tb01079.x

Wood, D., Bruner, J., & Ross, G. (1976). The role of tutoring in problem-solving. Journal of Child Psychology and Psychiatry and Allied Disciplines, 17, 89–100. https://doi.org/10.1111/j.1469-7610.1976.tb00381.x

Published

2024-04-25

How to Cite

Vaino, K., Rosin, T., Liiber, Ülle, Soobard, R., Teppo, M., Valdmann, A., Reisenbuk, E., & Rannikmäe, M. (2024). The development of a national e-test on science competence for the third school level: An assessment to support learning. Eesti Haridusteaduste Ajakiri. Estonian Journal of Education, 12(1), 88–120. https://doi.org/10.12697/eha.2024.12.1.05

Issue

Section

Articles