Automated summary evaluation with inbuilt rubric method: An alternative to constructed responses and multiple-choice tests assessments

Martínez-Huertas, José Ángel, Jastrzebska, Olga, Olmos, Ricardo y León, Jose A. . (2019) Automated summary evaluation with inbuilt rubric method: An alternative to constructed responses and multiple-choice tests assessments. Assessment & Evaluation in Higher Education

Ficheros (Some files may be inaccessible until you login with your old.e-spacio credentials)
Nombre Descripción Tipo MIME Size
Martinez-Huertas_JA_AutomatedSum.pdf Martinez-Huertas_JA_AutomatedSum.pdf application/pdf 313.91KB

Título Automated summary evaluation with inbuilt rubric method: An alternative to constructed responses and multiple-choice tests assessments
Autor(es) Martínez-Huertas, José Ángel
Jastrzebska, Olga
Olmos, Ricardo
León, Jose A.
Materia(s) Psicología
Abstract Automated Summary Evaluation is proposed as an alternative to rubrics and multiple-choice tests in knowledge assessment. Inbuilt rubric is a recent Latent Semantic Analysis (LSA) method that implements rubrics in an artificially-generated semantic space. It was compared with classical LSA’s cosine-based methods assessing knowledge in a within-subjects design regarding two validation sources: a comparison with the results of rubric scores and multiple-choice tests, and the sensitivity of predicting the academic level of the test-taker. Results showed a higher reliability for inbuilt rubric (from Pearson correlation coefficient .81 to .49) over the classical LSA method (from .61 to .34) and a higher sensitivity using binary logistic regressions and effect sizes to predict academic level. It is concluded that inbuilt rubric has a qualitatively higher reliability and validity than classical LSA methods in a way that is complementary to models based on semantic networks. Thus, it is concluded that new Automated Summary Evaluation approaches such as the inbuilt rubric method can be practical in terms of reliability and efficiency, and, thus, they can offer an affordable and valuable form of knowledge assessment in different educational levels.
Palabras clave Automated Summary Evalution
Inbuilt Rubric
rubrics
summaries
multiple-choice tests
Editor(es) Taylor and Francis Group
Fecha 2019-02-09
Formato application/pdf
Identificador bibliuned:DptoMCC-FPSI-Articulos-Jamartinez-0007
http://e-spacio.uned.es/fez/view/bibliuned:DptoMCC-FPSI-Articulos-Jamartinez-0007
DOI - identifier https://doi.org/10.1080/02602938.2019.1570079
ISSN - identifier 0260-2938 eISSN 1469-297X
Nombre de la revista Assessment & Evaluation in Higher Education
Número de Volumen 44
Número de Issue 7
Página inicial 1029
Página final 1041
Publicado en la Revista Assessment & Evaluation in Higher Education
Idioma eng
Versión de la publicación acceptedVersion
Tipo de recurso Article
Derechos de acceso y licencia http://creativecommons.org/licenses/by-nc-nd/4.0
info:eu-repo/semantics/openAccess
Tipo de acceso Acceso abierto
Notas adicionales La versión registrada de este artículo, publicado por primera vez en Assessment & Evaluation in Higher Education (2019) 44-7, p. 1029-1041, está disponible en línea en el sitio web del editor: Taylor and Francis Group; https://doi.org/10.1080/02602938.2019.1570079
Notas adicionales The registered version of this article, first published in Assessment & Evaluation in Higher Education (2019) 44-7, p. 1029-1041, is available online at the publisher's website: Taylor and Francis Group; https://doi.org/10.1080/02602938.2019.1570079

 
Versiones
Versión Tipo de filtro
Contador de citas: Google Scholar Search Google Scholar
Estadísticas de acceso: 42 Visitas, 15 Descargas  -  Estadísticas en detalle
Creado: Thu, 29 Feb 2024, 20:58:44 CET