Measuring student perceptions of AI-assisted academic communication: A questionnaire development study

Gulnara R. Ibraeva 1 * , Olga V. Sergeeva 2, Marina R. Zheltukhina 3, Yanina V. Gribova 4, Kirena G. Kelina 4, Natalia L. Sokolova 5
More Detail
1 Kazan State Power Engineering University, Kazan, RUSSIA
2 Department of English Philology, Kuban State University, Krasnodar, RUSSIA
3 Scientific and Educational Center “Person in Communication”, Pyatigorsk State University, Pyatigorsk, RUSSIA
4 Sechenov First Moscow State Medical University, Moscow, RUSSIA
5 Institute of Foreign Languages, Peoples’ Friendship University of Russia (RUDN University), Moscow, RUSSIA
* Corresponding Author
Online Journal of Communication and Media Technologies, Volume 16, Issue 2, Article No: e202627. https://doi.org/10.30935/ojcmt/18492
OPEN ACCESS   47 Views   28 Downloads   Published online: 29 Apr 2026
Download Full Text (PDF)
This article belongs to the special issue "Interdisciplinary Perspectives on Communication, Education, and Ethics in the Digital Age"

ABSTRACT

The aim of this study is to develop a valid and reliable scale evaluating students’ views on artificial intelligence (AI) supported academic communication. Knowing how students view these tools is crucial considering AI’s increasing presence in the classroom. We applied a thorough approach comprising content validation, pilot research and factor analysis in addition to literature review. Initially, 40 items were created and reduced to 37 items after expert evaluation. As a result of the analysis of the data obtained from 580 participants, it was determined that the scale showed a two-factor structure as “positive dimension” and “negative dimension”. Factor analyses both exploratory and confirmatory helped to establish the scale’s construct validity. The scale’s internal consistency reliability came out to be really strong. Based on gender and age, Bayesian statistical studies revealed no appreciable variation in students’ opinions of academic communication supported by AI. The developed scale offers academics and teachers a consistent instrument to evaluate students’ perception of AI technologies. This scale will help to shape plans for better integration of AI into learning environments.
Keywords: artificial intelligence, academic communication, scale development, student perceptions, educational technologies, Bayesian analysis

CITATION

Ibraeva, G. R., Sergeeva, O. V., Zheltukhina, M. R., Gribova, Y. V., Kelina, K. G., & Sokolova, N. L. (2026). Measuring student perceptions of AI-assisted academic communication: A questionnaire development study. Online Journal of Communication and Media Technologies, 16(2), e202627. https://doi.org/10.30935/ojcmt/18492

REFERENCES

  • Adiguzel, T., Kaya, M. H., & Cansu, F. K. (2023). Revolutionizing education with AI: Exploring the transformative potential of ChatGPT. Contemporary Educational Technology, 15(3), Article ep429. https://doi.org/10.30935/cedtech/13152
  • Avsheniuk, N., Lutsenko, O., Svyrydiuk, T., & Seminikhyna, N. (2024). Empowering language learners’ critical thinking: Evaluating ChatGPT’s role in English course implementation. Arab World English Journal, 1(1), 210-224. https://doi.org/10.24093/awej/chatgpt.14
  • Brown, T. A. (2015). Confirmatory factor analysis for applied research. Guilford Publications.
  • Browne, M. W., & Cudeck, R., (1992). Alternative ways of assessing model fit. Sociological Methods & Research, 21(2), 230-258. https://journals.sagepub.com/doi/10.1177/0049124192021002005
  • Celik, I., Dindar, M., Muukkonen, H., & Järvelä, S. (2022). The promises and challenges of artificial intelligence for teachers: A systematic review of research. TechTrends, 66(4), 616-630. https://doi.org/10.1007/s11528-022-00715-y
  • Choi, S., Jang, Y., & Kim, H. (2024). Exploring factors influencing students’ intention to use intelligent personal assistants for learning. Interactive Learning Environments, 32(8), 4049-4062. https://doi.org/10.1080/10494820.2023.2194927
  • Chugai, O., & Havrylenko, K. (2024). ChatGPT: Attitudes and experiences of technical university students in Ukraine. Information Technologies and Learning Tools, 101(3), 15-27. https://doi.org/10.33407/itlt.v101i3.5559
  • Clark, L. A., & Watson, D. (1995). Constructing validity: Basic issues in objective scale development. Psychological Assessment, 7(3), 309-319. https://doi.org/10.1037/1040-3590.7.3.309
  • Cooper, C., Booth, A., Varley-Campbell, J., Britten, N., & Garside, R. (2018). Defining the process to literature searching in systematic reviews: A literature review of guidance and supporting studies. BMC Medical Research Methodology, 18, Article 85. https://doi.org/10.1186/s12874-018-0545-3
  • Costello, A. B., & Osborne, J. (2005). Best practices in exploratory factor analysis: Four recommendations for getting the most from your analysis. Practical Assessment, Research, and Evaluation 10(1), Article 7. https://doi.org/10.7275/jyj1-4868
  • Cronbach, L. J., & Meehl, P. E. (1955). Construct validity in psychological tests. Psychological Bulletin, 52(4), Article 281. https://doi.org/10.1037/h0040957
  • Currie, G. M. (2023). Academic integrity and artificial intelligence: Is ChatGPT hype, hero or heresy? Seminars in Nuclear Medicine, 53(5), 719-730. https://doi.org/10.1053/j.semnuclmed.2023.04.008
  • DeVellis, R. F. (2016). Scale development: Theory and applications. SAGE.
  • DiStefano, C., & Motl, R. W. (2006). Further investigating method effects associated with negatively worded items on self-report surveys. Structural Equation Modeling: A Multidisciplinary Journal, 13(3), 440-464. https://doi.org/10.1207/s15328007sem1303_6
  • Essel, H. B., Vlachopoulos, D., Tachie-Menson, A., Johnson, E. E., & Baah, P. K. (2022). The impact of a virtual teaching assistant (chatbot) on students’ learning in Ghanaian higher education. International Journal of Educational Technology in Higher Education, 19, Article 57. https://doi.org/10.1186/s41239-022-00362-6
  • Fabrigar, L. R., Wegener, D. T., MacCallum, R. C., & Strahan, E. J. (1999). Evaluating the use of exploratory factor analysis in psychological research. Psychological Methods, 4(3), Article 272. https://doi.org/10.1037/1082-989X.4.3.272
  • Fornell, C., & Larcker, D. F. (1981). Evaluating structural equation models with unobservable variables and measurement error. Journal of Marketing Research, 18(1), 39-50. https://doi.org/10.2307/3151312
  • García-Martínez, I., Fernández-Batanero, J. M., Fernández-Cerero, J., & León, S. P. (2023). Analysing the impact of artificial intelligence and computational sciences on student performance: Systematic review and meta-analysis. Journal of New Approaches in Educational Research, 12(1), 171-197. https://doi.org/10.7821/naer.2023.1.1240
  • Gie Yong, A., & Pearce, S. (2013). A beginner’s guide to factor analysis: Focusing on exploratory factor analysis. Tutorials in Quantitative Methods for Psychology, 9(2), 79-94. https://doi.org/10.20982/tqmp.09.2.p079
  • Gudmundsson, E. (2009). Guidelines for translating and adapting psychological instruments. Nordic Psychology, 61(2), 29-45. https://doi.org/10.1027/1901-2276.61.2.29
  • Henseler, J., Ringle, C. M., & Sarstedt, M. (2015). A new criterion for assessing discriminant validity in variance-based structural equation modeling. Journal of The Academy of Marketing Science, 43(1), 115-135. https://doi.org/10.1007/s11747-014-0403-8
  • Hinkin, T. R. (1998). A brief tutorial on the development of measures for use in survey questionnaires. Organizational Research Methods, 1(1), 104-121. https://doi.org/10.1177/109442819800100106
  • Holmes, W., Bialik, M., & Fadel, C. (2019). Artificial intelligence in education promises and implications for teaching and learning. Center for Curriculum Redesign.
  • Holstein, K., McLaren, B. M., & Aleven, V. (2019). Co-designing a real-time classroom orchestration tool to support teacher-AI complementarity. Journal of Learning Analytics, 6(2), 27-52. https://doi.org/10.18608/jla.2019.62.3
  • Hu, Y. H. (2022). Effects and acceptance of precision education in an AI-supported smart learning environment. Education and Information Technologies, 27(2), 2013-2037. https://doi.org/10.1007/s10639-021-10664-3
  • Hu, L. T., & Bentler, P. M. (1999). Cutoff criteria for fit indexes in covariance structure analysis: Conventional criteria versus new alternatives. Structural Equation Modeling: A Multidisciplinary Journal, 6(1), 1-55. https://doi.org/10.1080/10705519909540118
  • Huang, W., Hew, K. F., & Fryer, L. K. (2022). Chatbots for language learning—Are they really useful? A systematic review of chatbot-supported language learning. Journal of Computer Assisted Learning, 38(1), 237-257. https://doi.org/10.1111/jcal.12610
  • Hwang, S. (2022). Examining the effects of artificial intelligence on elementary students’ mathematics achievement: A meta-analysis. Sustainability, 14(20), Article 13185. https://doi.org/10.3390/su142013185
  • Jeffreys, H. (1998). The theory of probability. Oxford.
  • Johanson, G. A., & Brooks, G. P. (2010). Initial scale development: Sample size for pilot studies. Educational and Psychological Measurement, 70(3), 394-400. https://doi.org/10.1177/0013164409355692
  • Kim, J., Lee, H., & Cho, Y. H. (2022). Learning design to support student-AI collaboration: Perspectives of leading teachers for AI in education. Education and Information Technologies, 27(5), 6069-6104. https://doi.org/10.1007/s10639-021-10831-6
  • Kim, J., Merrill, K., Xu, K., & Sellnow, D. D. (2020). My teacher is a machine: Understanding students’ perceptions of AI teaching assistants in online education. International Journal of Human-Computer Interaction, 36(20), 1902-1911. https://doi.org/10.1080/10447318.2020.1801227
  • Kim, M. K., Kim, N. J., & Heidari, A. (2022). Learner experience in artificial intelligence-scaffolded argumentation. Assessment and Evaluation in Higher Education, 47(8), 1301-1316. https://doi.org/10.1080/02602938.2022.2042792
  • Kim, T., & Song, H. (2023). Communicating the limitations of AI: The effect of message framing and ownership on trust in artificial intelligence. International Journal of Human-Computer Interaction, 39(4), 790-800. https://doi.org/10.1080/10447318.2022.2049134
  • Kline, R. B. (2016). Principles and practice of structural equation modeling. Guilford Publications.
  • Koć-Januchta, M. M., Schönborn, K. J., Tibell, L. A. E., Chaudhri, V. K., & Heller, H. C. (2020). Engaging with biology by asking questions: Investigating students’ interaction and learning with an artificial intelligence-enriched textbook. Journal of Educational Computing Research, 58(6), 1190-1224. https://doi.org/10.1177/0735633120921581
  • Lee, D., & Yeo, S. (2022). Developing an AI-based chatbot for practicing responsive teaching in mathematics. Computers and Education, 191, Article 104646. https://doi.org/10.1016/j.compedu.2022.104646
  • Li, C. H. (2016). The performance of ML, DWLS, and ULS estimation with robust corrections in structural equation models with ordinal variables. Psychological Methods, 21(3), 369-387. https://psycnet.apa.org/doi/10.1037/met0000093
  • Liu, Y., Park, J., & McMinn, S. (2024). Using generative artificial intelligence/ChatGPT for academic communication: Students’ perspectives. International Journal of Applied Linguistics, 34(4), 1437-1461. https://doi.org/10.1111/ijal.12574
  • Ma, D., Akram, H., & Chen, I.-H. (2024). Artificial intelligence in higher education: A cross-cultural examination of students’ behavioral intentions and attitudes. International Review of Research in Open and Distributed Learning, 25(3). https://doi.org/10.19173/irrodl.v25i3.7703
  • Ma, S., & Lei, L. (2024). The factors influencing teacher education students’ willingness to adopt artificial intelligence technology for information-based teaching. Asia Pacific Journal of Education, 44(1), 94-111. https://doi.org/10.1080/02188791.2024.2305155
  • MacKenzie, S. B., Podsakoff, P. M., & Podsakoff, N. P. (2011). Construct measurement and validation procedures in mis and behavioral research: Integrating new and existing techniques. MIS Quarterly, 35(2), Article 293. https://doi.org/10.2307/23044045
  • Maghsudi, S., Lan, A., Xu, J., & Van Der Schaar, M. (2021). Personalized education in the artificial intelligence Era: What to expect next. IEEE Signal Processing Magazine, 38(3), 37-50. https://doi.org/10.1109/MSP.2021.3055032
  • Mellinger, C. D., & Hanson, T. A. (2020). Methodological considerations for survey research: Validity, reliability, and quantitative analysis. Linguistica Antverpiensia, New Series: Themes in Translation Studies, 19(19), 172-190. https://doi.org/10.52034/lanstts.v19i0.549
  • Mijwil, M., Hiran, K. K., Doshi, R., Dadhich, M., Al-Mistarehi, A.-H., & Bala, I. (2023). ChatGPT and the future of academic integrity in the artificial intelligence Era: A new frontier. Al-Salam Journal for Engineering and Technology, 2(2), 116-127. https://doi.org/10.55145/ajest.2023.02.02.015
  • Ouyang, F., Zheng, L., & Jiao, P. (2022). Artificial intelligence in online higher education: A systematic review of empirical research from 2011 to 2020. Education and Information Technologies, 27(6), 7893-7925. https://doi.org/10.1007/s10639-022-10925-9
  • Palasundram, K., Sharef, N. M., Nasharuddin, N. A., Kasmiran, K. A., & Azman, A. (2019). Sequence to sequence model performance for education chatbot. International Journal of Emerging Technologies in Learning, 14(24), 56-68. https://doi.org/10.3991/ijet.v14i24.12187
  • Polit, D. F., & Beck, C. T. (2006). The content validity index: Are you sure you know what’s being reported? Critique and recommendations. Research in Nursing and Health, 29(5), 489-497. https://doi.org/10.1002/nur.20147
  • Putri Erito, S. N. (2023). Exploring ESP students’ perception toward the potential of artificial intelligence to promote students’ self-efficacy in English writing skill. Journal of English Language Learning, 7(2), 459-466. https://doi.org/10.31949/jell.v7i2.7598
  • Sánchez-Reina, J. R., Theophilou, E., Hernández-Leo, D., & Ognibene, D. (2024). Exploring undergraduates’ attitudes towards ChatGPT. Is AI resistance constraining the acceptance of chatbot technology? In G. Casalino, R. Di Fuccio, G. Fulantelli, P. Raviolo, P. C. Rivoltella, D. Taibi, & G. A. Toto (Eds.), Higher education learning methodologies and technologies online. HELMeTO 2023. Communications in Computer and Information Science, vol 2076 (pp. 383-397). Springer. https://doi.org/10.1007/978-3-031-67351-1_26
  • Sayed, W. S., Noeman, A. M., Abdellatif, A., Abdelrazek, M., Badawy, M. G., Hamed, A., & El-Tantawy, S. (2023). AI-based adaptive personalized content presentation and exercises navigation for an effective and engaging e-learning platform. Multimedia Tools and Applications, 82(3), 3303-3333. https://doi.org/10.1007/s11042-022-13076-8
  • Selwyn, N. (2019). Should robots replace teachers?: AI and the future of education. John Wiley & Sons.
  • Seo, K., Tang, J., Roll, I., Fels, S., & Yoon, D. (2021). The impact of artificial intelligence on learner-instructor interaction in online learning. International Journal of Educational Technology in Higher Education, 18, Article 54. https://doi.org/10.1186/s41239-021-00292-9
  • Shalevska, E., & Kostadinovska-Stojchevska, B. (2024). Ethics in times of advanced AI: Investigating students’ attitudes towards ChatGPT and academic integrity. Teacher, 27, 72-78. https://doi.org/10.20544/teacher.27.08
  • Shoaib, M., Sayed, N., Singh, J., Shafi, J., Khan, S., & Ali, F. (2024). AI student success predictor: Enhancing personalized learning in campus management systems. Computers in Human Behavior, 158, Article 108301. https://doi.org/10.1016/j.chb.2024.108301
  • Taylor, R., Fakhimi, M., Ioannou, A., & Spanaki, K. (2024). Personalized learning in education: A machine learning and simulation approach. Benchmarking: An International Journal, 32(7), 2662-2689. https://doi.org/10.1108/BIJ-06-2023-0380
  • Tossell, C. C., Tenhundfeld, N. L., Momen, A., Cooley, K., & De Visser, E. J. (2024). Student perceptions of ChatGPT use in a college essay assignment: Implications for learning, grading, and trust in artificial intelligence. IEEE Transactions on Learning Technologies, 17, 1069-1081. https://doi.org/10.1109/TLT.2024.3355015
  • Utami, S. P. T., Andayani, Winarni, R., & Sumarwati. (2023). Utilization of artificial intelligence technology in an academic writing class: How do Indonesian students perceive? Contemporary Educational Technology, 15(4), Article ep450. https://doi.org/10.30935/cedtech/13419
  • Vallis, C., Wilson, S., Gozman, D., & Buchanan, J. (2024). Student perceptions of AI-generated avatars in teaching business ethics: We might not be impressed. Postdigital Science and Education, 6(2), 537-555. https://doi.org/10.1007/s42438-023-00407-7
  • van Teijlingen, E., & Hundley, V. (2001). The importance of pilot studies. Social Research Update, 35, 1-4. https://sru.soc.surrey.ac.uk/SRU35.PDF
  • Wagenmakers, E. J., Marsman, M., Jamil, T., Ly, A., Verhagen, J., Love, J., Selker, R., Gronau, Q. F., Šmíra, M., Epskamp, S., Matzke, D., Rouder, J. N., & Morey, R. D. (2018). Bayesian inference for psychology. Part I: Theoretical advantages and practical ramifications. Psychonomic Bulletin and Review, 25(1), 35-57. https://doi.org/10.3758/s13423-017-1343-3
  • Wang, X., Pang, H., Wallace, M. P., Wang, Q., & Chen, W. (2024). Learners’ perceived AI presences in AI-supported language learning: A study of AI as a humanized agent from community of inquiry. Computer Assisted Language Learning, 37(4), 814-840. https://doi.org/10.1080/09588221.2022.2056203
  • Williams, B., Brown, T., & Onsman, A. (2010). Exploratory factor analysis: A five-step guide for novices. Australasian Journal of Paramedicine, 8(3). https://doi.org/10.33151/ajp.8.3.93
  • Williamson, B., Eynon, R., & Potter, J. (2020). Pandemic politics, pedagogies and practices: Digital technologies and distance education during the coronavirus emergency. Learning, Media and Technology, 45(2), 107-114. https://doi.org/10.1080/17439884.2020.1761641
  • Yusoff, M. S. B. (2019). ABC of content validation and content validity index calculation. Education in Medicine Journal, 11(2), 49-54. https://doi.org/10.21315/eimj2019.11.2.6
  • Zhou, C., Zhang, X., & Chan, J. (2024). Unveiling students’ experiences and perceptions of artificial intelligence usage in higher education. Journal of University Teaching and Learning Practice, 21(6). https://doi.org/10.53761/xzjprb23