Mobile Learning Platform focused on Learning Monitoring and Customization: Usability Evaluation Based on a Laboratory Study

The learning customization and monitoring are considered key aspects of the teaching-learning processes. Some works have proposed mobile learning systems that provide teachers and students learning monitoring and personalization services. One of the main requirements of these kinds of systems in terms of software quality is usability; however, few works have addressed the usability issues using laboratory studies with users in real domains. In this work, we present a usability evaluation of the learning monitoring and personalization services of a mobile learning platform based on a laboratory study in which nine teachers and ten students participated. In our usability evaluation, the aspects evaluated were effectiveness, efficiency, and level of user satisfaction as proposed by the ISO/IEC 25000 family of standards. The results show that the teachers presented effectiveness, efficiency, and satisfaction considered satisfactory, while the students presented effectiveness and satisfaction classified as satisfactory and acceptable efficiency. The usability evaluation described in this work can serve as a reference for developers seeking to improve learning monitoring and personalization services development.

1. UNESCO, Personalized Learning, 2017. Available at: https://unesdoc.unesco.org/ark:/48223/pf0000250057, accessed: Nov. 21, 2021.

2. Y. Hernández-Velázquez, C. Mezura-Godoy, V.Y. Rosales-Morales. M-Learning and Student-Centered Design: A Systematic Review of the Literature. Advances in Intelligent Systems and Computing, vol. 1297, 2020, pp. 349-363.

3. M. Abech, C.A. da Costa et al. A Model for Learning Objects Adaptation in Light of Mobile and Context-Aware Computing. Personal and Ubiquitous Computing, vol. 20, no. 2, 2016, pp. 167-184.

4. S.S. Oyelere, J. Suhonen et al. Design, development, and evaluation of a mobile learning application for computing education. Education and Information Technologies, vol. 23, no. 1, 2018, pp. 467-495.

5. S. Benhamdi, A. Babouri, and R. Chiky. Personalized recommender system for e-Learning environment. Education and Information Technologies, vol. 22, no. 4, 2017, pp. 1455-1477.

6. H. Xie, D. Zou et al. Personalized word learning for university students: a profile-based method for e-learning systems. Journal of Computing in Higher Education, vol. 31, no. 2, 2019, pp. 273-289.

7. D. Cáliz, J. Gomez et al. Evaluation of a usability testing guide for mobile applications focused on people with down syndrome (USATESTDOWN). Lecture Notes in Computer Science, vol. 10069, 2016, pp. 497-502.

8. S. S. Rani, S. Krishnanunni. Educational App for Android-Specific Users—EA-ASU. Advances in Intelligent Systems and Computing, vol. 1097, 2020, pp. 325-335.

9. H. Imran, M. Belghis-Zadeh. PLORS: a personalized learning object recommender system. Vietnam Journal of Computer Science, vol. 3, no. 1, 2016, pp. 3-13.

10. S. Saryar, S.V. Kolekar et al. Mobile learning recommender system based on learning styles. Advances in Intelligent Systems and Computing, vol. 900, 2019, pp. 299-312.

11. K. Meenakshi, R. Sunder et al. An intelligent smart tutor system based on emotion analysis and recommendation engine. In Proc. of the 2017 International Conference on IoT and Application (ICIOT), 2017, pp. 1-4.

12. L. G. Martínez, S. Marrufo et al. Using a Mobile Platform for Teaching and Learning Object Oriented Programming. IEEE Latin America Transactions, vol. 16, no. 6, 2018, pp. 1825-1830.

13. Y. Li, L. Wang. Using iPad-based mobile learning to teach creative engineering within a problem-based learning pedagogy. Education and Information Technologies, vol. 23, no. 1, 2018, pp. 555-568.

14. A. Sharma. A proposed e-learning system facilitating recommendation using content tagging and student learning styles. In Proc. of the 2017 5th National Conference on E-Learning &ELearning Technologies (ELELTECH), 2017, pp. 1-6.

15. R.A.W. Tortorella and S. Graf. Considering learning styles and context-awareness for mobile adaptive learning. Education and Information Technologies, vol. 22, no. 1, 2017, pp. 297-315.

16. T. Jagušt, I. Botički. Mobile learning system for enabling collaborative and adaptive pedagogies with modular digital learning contents. Journal of Computers in Education, vol. 6, 2019, pp. 335-362.

17. C. B. Yao. Constructing a User-Friendly and Smart Ubiquitous Personalized Learning Environment by Using a Context-Aware Mechanism. IEEE Transactions on Learning Technologies, vol. 10, no. 1, 2017, pp. 104-114.

18. C. Huerta-Guerrero, E. Lopez-Dominguez et al. Kaanbal: A Mobile Learning Platform Focused on Monitoring and Customization of Learning. International Journal of Emerging Technologies in Learning, vol. 16, no. 1, 2020, pp. 18-43.

19. ISO/IEC 25010:2011 Systems and software engineering — Systems and software Quality Requirements and Evaluation (SQuaRE) — System and software quality models, 2011.

20. ISO/IEC 25022:2016 Systems and software engineering — Systems and software quality requirements and evaluation (SQuaRE) — Measurement of quality in use, 2016.

21. A. A. Arain, Z. Hussain et al. Evaluating usability of M-learning application in the context of higher education institute. Lecture Notes in Computer Science, vol. 9753, 2016, pp. 259-268.

22. B. A. Kumar, P. Mohite. Usability Study of Mobile Learning Application in Higher Education Context: An Example from Fiji National University. In Mobile Learning in Higher Education in the Asia-Pacific Region, Springer Singapore, 2017, pp. 607-622.

23. A. Pensabe-Rodriguez, E. Lopez-Dominguez et al. Context-aware mobile learning system: Usability assessment based on a field study. Telematics and Informatics, vol. 48, issue C, 2020, article no. 101346.

24. D. Hariyanto, M. B. Triyono, and T. Köhler. Usability evaluation of personalized adaptive e-learning system using USE questionnaire. Knowledge Management & E-Learning: An International Journal, vol. 12, no. 1, 2020, pp. 85-105.

25. B. A. Kumar and P. Mohite. Usability guideline for mobile learning apps: An empirical study. International Journal of Mobile Learning and Organisation, vol. 10, no. 4, 2016, pp. 223-237.

26. S. Yağmur and M. P. Çakır. Usability evaluation of a dynamic geometry software mobile interface through eye tracking. Lecture Notes in Computer Science, vol. 9753, 2016, pp. 391-402.

27. M. Asghar, I.S. Bajwa et al. A Genetic Algorithm-Based Support Vector Machine Approach for Intelligent Usability Assessment of m-Learning Applications. Mobile Information Systems, vol. 2022, 2022, Article ID 1609757, 20 p.

28. B. A. Kumar, P. Mohite. Usability of mobile learning applications: a systematic literature review. Journal of Computers in Education, vol. 5, no. 3, 2018, pp. 1-17.

29. T. S. Tullis and J. N. Stetson. A Comparison of Questionnaires for Assessing Website Usability. In Proc. of the UPA 2004 Conference, 2004, pp. 1-12.

30. N. Jacob. Usability Heuristics for User Interface Design. URL: https://www.nngroup.com/articles/ten-usability-heuristics/, accessed: Nov. 21, 2021.

31. C. M. Alonso, D. J. Domingo, and P. Honey. Los estilos de aprendizaje, Procedimientos de diagnóstico y mejora. Séptima edición, Bilbao: Editorial Mensajero, 2007 (in Spanish).

32. K. L. Chin, J.P., Diehl, V.A., Norman. Questionnaire for User Interface Satisfaction. Development of an Instrument Measuring User Satisfaction of the Human-Computer Interface. In Proc. of the SIGCHI Conference on Human Factors in Computing Systems, 1988, pp. 213-218.

33. ISO, Ergonomics of human-system interaction - Part 11: Usability: Definitions and concepts. Iso/Np 9241-11, 2018.

34. E.N. Chujkova, A.R. Aidinyan, O.L. Tsvetkova. Adaptation Algorithm for Application Menus. Programming and Computer Software, vol. 46, no. 6, 2020, pp. 397–405 / Е.Н. Чуйкова, А.Р. Айдинян, О.Л. Цветкова. Алгоритм адаптивного изменения меню программного приложения. Программирование, том 46, no. 6, 2020 г., стр. 30-40.