SBERT-based Deep Learning model for mapping of PEOs and POs with Justification Rubrics

Abstract

Introduction: Aligning Program Educational Objectives (PEOs) with Program Outcomes (POs) is a key step in developing a meaningful engineering curriculum. It ensures that what students learn is in line with both academic goals and industry needs. However, doing this manually can be time-consuming and biased.

Objectives: To support curriculum designers in developing more cogent and industry-relevant engineering education programs by developing an automated, objective, and efficient system that uses natural language processing to assess and align Program Educational Objectives (PEOs) with Program Outcomes (POs).

Methods: This study explores a more efficient and objective method using Natural Language Processing (NLP), specifically the Sentence-BERT (SBERT) model, to compare the meanings of PEOs and POs. We used real data from the Civil Engineering Department at B.S. Abdur Rahman Crescent Institute of Science and Technology. Since the dataset was limited we applied text augmentation techniques like synonym replacement, random insertion or deletion, and shuffling to create a more robust dataset. Cosine similarity was used to measure how closely each PEO aligns with the POs, and the results were categorized into High, Medium, Low, or No Similarity based on expert-defined thresholds.

Results: The results show that this approach is effective in identifying meaningful connections between PEOs and POs. It offers a helpful tool for curriculum designers and academic reviewers who want a clearer, more consistent way to evaluate and improve educational programs.

Conclusions: This study provides a way to connect Program Educational Objectives (PEOs) to Program Outcomes (POs) using the SBERT model.  Using text augmentation approaches and fine-tuning SBERT, we successfully categorized the similarity scores into four groups: High, Medium, Low, and No Similarity.  Implementing a rubric-based evaluation adds a new level of understanding to the model's judgments, enabling more informed and logical instructional planning.  Future studies can concentrate on improving text augmentation methods and investigating alternative transformer-based models to BERT in order to further improve the mapping process.