Development of Digital-based Program Evaluation Model (McIPP): A Study on Elementary School Indonesian Language Learning

Abstract

Introduction: Traditional models of program evaluations are less responsive to learning in the 21st century, particularly with technology incorporation and precision in data. Hence, the construction of the digital-based evaluation model (McIPP) plays an important role in the improvement of Indonesian language skill learning at the Elementary School level.

Objectives: The goal of this study was to evaluate the content validity of the digital McIPP Evaluation Model through the Aiken's V method. Additionally, the study aimed to assess the construct validity of the model using Confirmatory Factor Analysis (CFA).

Methods: This research is a type of development study aimed at creating a valid and practical digital-based McIPP program evaluation model for use in Indonesian language instruction at the elementary school level. The study follows an adapted version of the Borg & Gall development framework, consisting of four primary phases: a literature review and needs assessment, initial model and indicator design, expert evaluation, and empirical validation of the model’s constructs. Participants in this research included experts in educational evaluation, instructional technology, and Indonesian language for content validation, along with principals, teachers, and students for construct validation. The study involved 15 school principals, 86 teachers, and 508 students from 15 elementary schools representing high, medium, and low quality categories in Pasuruan City and Southeast Sulawesi Province. Data were gathered through the McIPP evaluation questionnaire. Content validity was evaluated using Aiken’s V, with a threshold of 0.70 or above, while construct validity was assessed through Confirmatory Factor Analysis (CFA), utilizing standard loading factor criteria to evaluate the model's fit.

Results: The instrument used in this study demonstrates strong content validity, as indicated by Aiken’s V scores  0.08 across all items, with values ranging between 0 and 1.00 confirming the validity of the content in the digital-based evaluation model. Additionally, results from the Confirmatory Factor Analysis (CFA) indicate that the McIPP digital evaluation model aligns well with the empirical data. This is supported by model fit indices that meet established criteria, including Chi-Square/df (1.381), RMSEA (0.058), SRMR (0.072), and other key indicators such as NFI, CFI, TLI, and GFI, all of which score above 0.90.

Conclusions: In conclusion, this study confirms that the digital-based McIPP evaluation model satisfies the validity criteria for both content and construct. As a result, it is considered suitable for implementation in the context of contemporary, responsive digital education evaluation.