The development of a national e-test on science competence for the third school level: An assessment to support learning
DOI:
https://doi.org/10.12697/eha.2024.12.1.05Keywords:
science competence, diagnostic e-test, feedback, design-based research, context-basedAbstract
Selle disainipõhise uuringu eesmärk oli välja töötada e-tasemetöö, millega saaks hinnata riiklikus õppekavas määratletud III kooliastme lõpuks omandatud loodusteadusliku pädevuse taset. Ühtlasi pidi tasemetöö andma kirjeldavat tagasisidet nii õpilasele, õpetajale, lapsevanemale kui ka haridusüldsusele. Protsessi komplekssusest tingituna keskendutakse selles artiklis kitsamalt tasemetöö disainimise protsessile ja saadud lõpptulemusele ning põhjendatakse tehtud disainiotsuseid. Uurijate ja praktikute vahelises koostöös (2018–2022) jõuti uudse lahenduse – kontekstipõhise tasemetööni, mida saab kasutada üle-eestiliseks põhiuuringuks. Õpilaste saavutatud loodusteadusliku pädevuse taset kirjeldatakse neljal tasemel üheksa tunnuse abil, mis on ülevaatlikkuse nimel rühmitatud omakorda neljaks: loodusteaduslikud teadmised, uurimuslikud oskused, probleemi lahendamise ja otsuse tegemise oskused ning kommunikatsioonioskused. Tasemetöö koosneb neljast alatestist ning kokku 35 ülesandest.
Downloads
References
Aksen, M., Jürimäe, M., Nõmmela, K., Saarsen, K., Sillak, S., Eskor, J., Vool, E., & Urmann, H. (2018). Eesti üldhariduskoolides kasutatavad hindamissüsteemid. Tartu Ülikool. https://www.hm.ee/sites/default/files/uuringud/hindamine_lopparuanne_15.okt_loplik.pdf.
Anderson, L. W., & Krathwohl, D. R. (toim) (2001). A Taxonomy for Learning, Teaching, and Assessing: A Revision of Bloom’s Taxonomy of Educational Objectives. New York: Addison Wesley Longman, Inc.
Anderson, T., & Shattuck, J. (2012). Design-based research: A decade of progress in education research? Educational Researcher, 41(1), 16–25. https://doi.org/10.3102/0013189X11428813
Assessment Reform Group (1999). Assessment for Learning: Beyond the Black Box. Cambridge: University of Cambridge School of Education.
Baker, P., & Schmude, M. (2022). Structure of the observed learning outcomes (SOLO) model: A mixed-method systematic review of research in mathematics education. Eurasia Journal of Mathematics, Science and Technology Education, 18(6), 1–17. https://doi.org/10.29333/ejmste/12087
Barab, S., & Squire, K. (2004). Design-based research: Putting a stake in the ground. The Journal of the Learning Sciences, 13(1), 1–14. https://doi.org/10.1207/s15327809jls1301_1
Biggs, J. B., & Collis, K. F. (2014). Evaluating the Quality of Learning: The SOLO Taxonomy (Structure of the Observed Learning Outcome). Academic Press.
Bolhuis, S. (2003). Towards process-oriented teaching for self-directed lifelong learning: A multidimensional perspective. Learning and Instruction, 13(3), 327–347. https://doi.org/10.1016/S0959-4752(02)00008-7
Bond, T. (2015). Applying the Rasch Model: Fundamental Measurement in the Human Sciences (3rd ed.). Routledge.
Boyle, A., & Hutchison, D. (2009). Sophisticated tasks in e-assessment: What are they and what are their benefits?. Assessment & Evaluation in Higher Education, 34(3), 305–319. https://doi.org/10.1080/02602930801956034
Champagne, A. B. (2013). Content to be assessed across the history of the national assessment of educational progress. D. Corrigan, R. Gunstone, & A. Jones (Eds.), Valuing Assessment in Science Education: Pedagogy, Curriculum, Policy. (pp. 119–151). Dordrecht: Springer Netherlands. https://doi.org/10.1007/978-94-007-6668-6_7
Choi, K., Lee, H., Shin, N., Kim, S.-W., & Krajcik, J. (2011). Re-conceptualization of scientific literacy in South Korea for the 21st century. Journal of Research in Science Teaching, 48(6), 670–697. https://doi.org/10.1002/tea.20424
Cobb, P., diSessa, A., Lehrer, R., & Schauble, L. (2003). Design experiments in educational research. Educational Researcher, 32(1), 9–13. https://doi.org/10.3102/0013189X032001009
Csapó, B., & Molnár, G. (2019). Online diagnostic assessment in support of personalized teaching and learning: The eDia system. Frontiers in Psychology, 10 (1522), 1–14. https://doi.org/10.3389/fpsyg.2019.01522
Daniels, H. (2007). Pedagogy. In H. Daniels, M. Cole, & J. V. Wertsch (Eds.), The Cambridge companion to Vygotsky. (pp 307–331). Cambridge University Press. https://doi.org/10.1017/CCOL0521831040.013
Fensham, P. J., & Rennie, L. J. (2013). Towards an authentically assessed science curriculum. D. Corrigan, R. Gunstone, & A. Jones (Eds.), Valuing Assessment in Science Education: Pedagogy, Curriculum, Policy (pp. 69–100). Dordrecht: Springer. https://doi.org/10.1007/978-94-007-6668-6_5
Finn, B. (2015). Measuring motivation in low-stakes assessments. ETS Research Report Series, 2015(2), 1–17. https://doi.org/10.1002/ets2.12067
Gaffney, T. W., Cudeck, R., Ferrer, E., & Widaman, K. F. (2010). On the factor structure of standardized educational achievement tests. Journal of Applied Measurement, 11(4), 384.
Gweon, H., & Schonlau, M. (2022). Automated classification for open-ended questions with BERT. arXiv preprint arXiv:2209.06178.
Haridus- ja Noorteamet (Harno) (2023a). Tasemetööd. https://harno.ee/tasemetood.
Haridus- ja Noorteamet (Harno) (2023b). III kooliastme testi materjalid 2022: Loodusteadusliku kirjaoskuse komponendid ja tasemed koos näidisülesannetega III kooliastmes. https://projektid.edu.ee/pages/viewpage.action?pageId=132157214.
Haridus- ja Teadusministeerium (2014). Eesti elukestva õppe strateegia 2020. https://www.haridusfoorum.ee/images/haridusstrateegia/Eesti_elukestva_oppe_strateegia_loplik.pdf.
Harvey, N. D. (2021). A Simple guide to inter-rater, intra-rater and test-retest reliability for animal behaviour studies. https://doi.org/10.31219/osf.io/8stpy
Hattie, J., & Timperley, H. (2007). The power of feedback. Review of Educational Research, 77(1), 81–112. https://doi.org/10.3102/003465430298487
Henno, I., Kollo, L., & Mikser, R. (2017). Eesti loodusainete õpetajate uskumused, õpetamispraktika ja enesetõhusus TALIS 2008 ja 2013 uuringu alusel. Eesti Haridusteaduste Ajakiri, 5(1), 268–296. https://doi.org/10.12697/eha.2017.5.1.09
Hsieh, H.-F., & Shannon, S. E. (2005). Three approaches to qualitative content analysis. Qualitative Health Research, 15(9), 1277–1288. https://doi.org/10.1177/1049732305276687
Höttecke, D. & Allchin, D. (2020). Reconceptualizing nature-of-science education in the age of social media. Science Education, 104(4), 641–666. https://doi.org/10.1002/sce.21575
Jonassen, D.H. (2011). Learning to Solve Problems: A Handbook for Designing Problem-Solving Learning Environments. New York: Routledge.
Kline, P. (1986). A Handbook of Test Construction: Introduction to Psychometric Design, Methuen, London.
Loogma, K., Erss, M., Ümarik, M., & Aasa, M. (2020). Õpetaja professionalismi võimalikud tulevikustsenaariumid aastaks 2035. Eesti Haridusteaduste Ajakiri, 8(1), 180–212. https://doi.org/10.12697/eha.2020.8.1.08
OECD (2018). The future of education and skills: Education 2030. Paris: OECD Publishing.
OECD (2019). TALIS 2018 Results (Volume I): Teachers and school leaders as lifelong learners, TALIS, OECD Publishing. https://doi.org/10.1787/1d0bc92a-en
OECD (2020). PISA 2024 strategic vision and direction for science (Issue March). https://www.oecd.org/pisa/publications/PISA-2024-Science-Strategic-Vision-Proposal.pdf.
OECD (2021). The assessment frameworks for cycle 2 of the programme for the international assessment of adult competencies, OECD Skills Studies, OECD Publishing, Paris.
Osborne, J., Pimentel, D., Alberts, B., Allchin, D., Barzilai, S., Bergstrom, C., Coffey, J., Donovan, B., Kivinen, K., Kozyreva. A., & Wineburg, S. (2022). Science Education in an Age of Misinformation. Stanford University, Stanford, CA.
Pedaste, M., Baucal, A., & Reisenbuk, E. (2021). Towards a science inquiry test in primary education: Development of items and scales. International Journal of STEM Education, 8, 1–19. https://doi.org/10.1186/s40594-021-00278-z
Pedaste, M., Brikker, M., Rannikmäe, M., Soobard, R., Mäeots, M., & Reiska, P. (2017). Loodusvaldkonna õpitulemuste hindamine. Raport, Tartu.
Pereira, T., Amaral, A., & Mendes, I. (2022). A Competency definition based on the knowledge, skills, and human dispositions constructs. In International Conference on Internet of Everything (pp. 29–38). Cham: Springer Nature Switzerland. https://doi.org/10.1007/978-3-031-25222-8_3
Põhikooli riiklik õppekava (2023). https://www.riigiteataja.ee/akt/108032023005.
Põhikooli riiklik õppekava (2023). Lisa 4. https://www.riigiteataja.ee/aktilisa/1080/3202/3005/18m_pohi_lisa4.pdf#.
Rannikmäe, M., Vaino, K., Teppo, M., Soobard, R., Rosin, T., & Must, O. (2018). Loodusvaldkonna õpitulemuste hindamine III kooliastmes. Raport. Tartu.
Rannikmäe, M., Soobard, R., Vaino, K., Rosin, T., Teppo, M., & Must, O. (2019). Loodusvaldkonna õpitulemuste hindamine III–IV kooliastmes. Raport. Tartu.
Rannikmäe, M., Soobard, R., & Vaino, K. (2020). Loodusvaldkonna õpitulemuste e-hindamine põhikooli kolmandas astmes ja gümnaasiumis. Kontseptsioon. Tartu.
Rannikmäe, M., Soobard, R., Vaino, K., & Rosin, T. (2021a). Loodusvaldkonna õpitulemuste e-hindamine põhikooli kolmandas astmes ja gümnaasiumis. Kontseptsioon. Tartu.
Rannikmäe, M., Soobard, R., Vaino, K., Teppo, M., Valdmann, A., & Rosin, T. (2021b). Loodusvaldkonna õpitulemuste hindamine. Raport. Tartu.
Rannikmäe, M., Vaino, K., Soobard, R., Teppo, M., & Reisenbuk, E. (2023). Lühikokkuvõte 2022/2023. õppeaasta loodusainete III kooliastme katselise tasemetöö tulemustest.
Reeves, T. (2006). Design research from a technology perspective. In J. van den Akker, K. Gravemeijer, S. McKenney, & N. Nieveen (Eds.), Educational design research (pp. 52–66). London: Routledge.
Rosin, T., Vaino, K., Soobard, R., & Rannikmäe, M. (esitatud). Understanding science teachers’ beliefs about the teaching and assessment of scientific competences: Explaining the reasons for implementing or not implementing science e-test feedback.
Saks, K., & Leijen, Ä. (2015). Kognitiivsete ja metakognitiivsete õpistrateegiate toetamine tehnoloogiaga tõhustatud keeleõppes. Eesti Haridusteaduste Ajakiri, 3(2), 130–155. https://doi.org/10.12697/eha.2015.3.2.05
Schut, S., Heeneman, S., Bierer, B., Driessen, E., van Tartwijk, J., & van Der Vleuten, C. (2020). Between trust and control: Teachers’ assessment conceptualisations within programmatic assessment. Medical Education, 54(6), 528–537. https://doi.org/10.1111/medu.14075
Sjöstrom, J., & Talanquer, V. (2014). Humanizing chemistry education: From simple contextualization to multifaceted problematization. Journal of Chemical Education, 91(8), 1125–1131. https://doi.org/10.1021/ed5000718
Zimmerman, B. J. (1989). A social cognitive view of self-regulated academic learning. Journal of Educational Psychology, 81(3), 329–339. https://doi.org/10.1037/0022-0663.81.3.329
Zhang, L., Huang, Y., Yang, X., Yu, S., & Zhuang, F. (2022). An automatic short-answer grading model for semi-open-ended questions. Interactive Learning Environments, 30(1), 177–190. https://doi.org/10.1080/10494820.2019.1648300
Zheng, L. (2015). A systematic literature review of design-based research from 2004 to 2013. Journal of Computers in Education, 2, 399–420. https://doi.org/10.1007/s40692-015-0036-z
Thier, M., & Daviss, B. (2002). The New Science Literacy: Using Language Skills to Help Students Learn Science. Portsmouth, NH: Heinemann.
Tinoca, L., Piedade, J., Santos, S., Pedro, A., & Gomes, S. (2022). Design-based research in the educational field: A systematic literature review. Education Sciences, 12(6), 410. https://doi.org/10.3390/educsci12060410
Tire, G., Puksand, H., Lepmann, T., Henno, I., Lindemann, K., Täht, K., Lorenz, B., & Silm, E. (2019). PISA 2018 Eesti tulemused: Eesti 15-aastaste õpilaste teadmised ja oskused funktsionaalses lugemises, matemaatikas ja loodusteadustes. https://www.innove.ee/uuringud/pisa-uuring/pisa-2018/.
The Design-Based Research Collective. (2003). Design-based research: An emerging paradigm for educational inquiry. Educational Researcher, 32(1), 5–8. https://doi.org/10.3102/0013189X032001005
Toomaneejinda, A. (2017). Zone of proximal development, dynamic assessment and learner empowerment. LEARN Journal: Language Education and Acquisition Research Network, 10(1), 176–185.
Valdmann, A., Holbrook, J., & Rannikmäe, M. (2012). Evaluating the teaching impact of a prior context-based, professional development programme. Science Education International, 23(2), 166–185.
Van den Akker, J., Gravemeijer, K., McKenney, S., & Nieveen, N. (2006). Educational Design Research. Routledge.
Vitello, S., & Williamson, J. (2017). Internal versus external assessment in vocational qualifications: A commentary on the government’s reforms in England. London Review of Education, 15(3), 536–548. https://doi.org/10.18546/LRE.15.3.14
Vygotsky, L. S., & Cole, M. (1978). Mind in Society: Development of Higher Psychological Processes. Harvard University Press.
Wainer, H., Sheehan, K. M., & Wang, X. (2000). Some paths towards making Praxis scores more useful. Journal of Educational Measurement, 37, 113–140. https://doi.org/10.1111/j.1745-3984.2000.tb01079.x
Wood, D., Bruner, J., & Ross, G. (1976). The role of tutoring in problem-solving. Journal of Child Psychology and Psychiatry and Allied Disciplines, 17, 89–100. https://doi.org/10.1111/j.1469-7610.1976.tb00381.x
Downloads
Published
How to Cite
Issue
Section
License
The authors who publish in Eesti Haridusteaduste Ajakiri. Estonian Journal of Education agree to the following terms:
- This journal provides immediate open access to its content. All the articles are licensed under a Creative Commons Attribution-NonCommercial-NoDerivs 3.0 License.
- Authors grant the journal right of (1) first publication and distribution of the article, (2) making it available to public, (3) public presentation.
- Authors have the right to enter into separate contractual arrangements for posting the article to an institutional repository or publish it in a book with an acknowledgement of its initial publication in this journal.
- Authors are permitted to post citations from their work online (e.g. on their website) with an acknowledgement of its initial publication in this journal (see Open Access).
