Cybersecurity and Privacy Risks of AI Assistants in Academic Institutions: A Risk Classification and Governance Framework
Keywords:
AI assistants, large language models, cybersecurity, privacy, higher education, prompt injectionAbstract
Large language model-based AI assistants are rapidly becoming part of academic work, supporting students, teachers, researchers, and administrators in writing, summarisation, translation, coding, and information retrieval. However, their adoption in academic institutions creates cybersecurity and privacy risks that extend beyond the commonly discussed problem of academic integrity. Using a qualitative literature review and conceptual synthesis of recent research on generative AI in higher education, LLM security, privacy protection, and institutional AI governance, this article classifies the main risks of AI assistants into four groups: data-related risks, model-related risks, application and integration risks, and human and governance risks. It shows that universities process sensitive categories of information — student records, grades, unpublished research, examination materials, administrative documents — which makes uncontrolled AI use a significant institutional risk. Special attention is given to prompt injection, sensitive data disclosure, insecure integrations, overreliance on AI outputs, and policy fragmentation. The article proposes an Academic AI Security and Privacy Governance Framework based on six layers: governance and policy; data classification and privacy; secure AI architecture; prompt and output security; human oversight and AI literacy; and monitoring, audit, and continuous improvement. Because each layer translates external legal and regulatory requirements — data-protection obligations, accountability, auditability — into institutional rules and technical controls, the framework is offered as a contribution to the governance of digital technology in education as much as to information security practice. The study concludes that AI assistants should be treated as security- and privacy-sensitive socio-technical systems requiring coordinated institutional governance.
Downloads
References
An, Y., Yu, J. H., & James, S. (2025). Investigating the higher education institutions' guidelines and policies regarding the use of generative AI in teaching, learning, research, and administration. International Journal of Educational Technology in Higher Education, 22. https://doi.org/10.1186/s41239-025-00507-3
Autio, C., Schwartz, R., Dunietz, J., Jain, S., Stanley, M., Tabassi, E., Hall, P., & Roberts, K. (2024). Artificial intelligence risk management framework: Generative artificial intelligence profile (NIST AI 600-1). National Institute of Standards and Technology. https://doi.org/10.6028/NIST.AI.600-1
Bittle, K., & El-Gayar, O. (2025). Generative AI and academic integrity in higher education: A systematic review and research agenda. Information, 16(4), Article 296. https://doi.org/10.3390/info16040296
Borodiyenko, O., Drach, I., Bazeliuk, N., Petroye, O., Reheilo, I., Bazeliuk, O., & Slobodianiuk, O. (2025). Opportunities and risks of using AI-based applications in research: The case of Ukrainian universities. Information Technologies and Learning Tools, 105(1), 125–143. https://doi.org/10.33407/itlt.v105i1.5794
Chang, V., Ansari, Y., & Arami, M. (2024). ChatGPT in higher education: A risk management approach to academic integrity, critical thinking, and workforce readiness. In Proceedings of the 6th International Conference on Finance, Economics, Management and IT Business. https://doi.org/10.5220/0012764100003717
Dahabiyeh, L., Taha, N., Thneibat, M., & Bhat, M. A. (2026). Privacy awareness in generative AI: The case of ChatGPT. Interactive Technology and Smart Education, 23(1), 25–48. https://doi.org/10.1108/ITSE-01-2025-0009
Das, B. C., Amini, M. H., & Wu, Y. (2025). Security and privacy challenges of large language models: A survey. ACM Computing Surveys, 57(6), Article 152, 1–39. https://doi.org/10.1145/3712001
European Commission, Directorate-General for Research and Innovation. (2026). Living guidelines on the responsible use of generative AI in research (3rd ed.). European Commission. https://research-and-innovation.ec.europa.eu/news/all-research-and-innovation-news/updated-era-living-guidelines-responsible-use-generative-ai-research-2026-05-08_en
European Parliament & Council of the European Union. (2024). Regulation (EU) 2024/1689 of 13 June 2024 laying down harmonised rules on artificial intelligence (Artificial Intelligence Act). Official Journal of the European Union, L, 2024/1689. https://eur-lex.europa.eu/eli/reg/2024/1689/oj
European Data Protection Board. (2024). Opinion 28/2024 on certain data protection aspects related to the processing of personal data in the context of AI models. https://www.edpb.europa.eu/our-work-tools/our-documents/opinion-board-art-64/opinion-282024-certain-data-protection-aspects_en
European Data Protection Board. (2025). AI privacy risks & mitigations: Large language models (LLMs). https://www.edpb.europa.eu/our-work-tools/our-documents/support-pool-experts-projects/ai-privacy-risks-mitigations-large_en
Miao, F., & Cukurova, M. (2024). AI competency framework for teachers. UNESCO. https://www.unesco.org/en/articles/ai-competency-framework-teachers
Miao, F., & Holmes, W. (2023). Guidance for generative AI in education and research. UNESCO. https://www.unesco.org/en/articles/guidance-generative-ai-education-and-research
Miranda, M., Ruzzetti, E. S., Santilli, A., Zanzotto, F. M., Bratières, S., & Rodolà, E. (2025). Preserving privacy in large language models: A survey on current threats and solutions. Transactions on Machine Learning Research. https://arxiv.org/abs/2408.05212
Moorhouse, B. L., Yeo, M. A., & Wan, Y. (2023). Generative AI tools and assessment: Guidelines of the world's top-ranking universities. Computers and Education Open, 5, Article 100151. https://doi.org/10.1016/j.caeo.2023.100151
Muliarevych, O. (2024). Enhancing system security: LLM-driven defense against prompt injection vulnerabilities. In 2024 IEEE 17th International Conference on Advanced Trends in Radioelectronics, Telecommunications and Computer Engineering (TCSET) (pp. 420–423). IEEE. https://doi.org/10.1109/TCSET64720.2024.10755823
National Cyber Security Centre. (2025a). Impact of AI on cyber threat from now to 2027. https://www.ncsc.gov.uk/report/impact-ai-cyber-threat-now-2027
National Cyber Security Centre. (2025b). Prompt injection is not SQL injection: It may be worse. https://www.ncsc.gov.uk/blog-post/prompt-injection-is-not-sql-injection
Neel, S., & Chang, P. (2024). Privacy issues in large language models: A survey. arXiv. https://doi.org/10.48550/arXiv.2312.06717
Novelli, C., Casolari, F., Hacker, P., Spedicato, G., & Floridi, L. (2024). Generative AI in EU law: Liability, privacy, intellectual property, and cybersecurity. Computer Law & Security Review, 55, Article 106066. https://doi.org/10.1016/j.clsr.2024.106066
OWASP Foundation. (2025). OWASP Top 10 for LLM and generative AI applications 2025. https://genai.owasp.org/llm-top-10/
Pikhart, M., & Al-Obaydi, L. H. (2025). Reporting the potential risk of using AI in higher education: Subjective perspectives of educators. Computers in Human Behavior Reports, 18, Article 100693. https://doi.org/10.1016/j.chbr.2025.100693
Temper, M., Tjoa, S., & David, L. (2025). Higher Education Act for AI (HEAT-AI): A framework to regulate the usage of AI in higher education institutions. Frontiers in Education, 10, Article 1505370. https://doi.org/10.3389/feduc.2025.1505370
Wilson, T. D. (2025). The development of policies on generative artificial intelligence in UK universities. IFLA Journal, 51(3), 722–734. https://doi.org/10.1177/03400352251333796
Wu, C., Zhang, H., & Carroll, J. M. (2024). AI governance in higher education: Case studies of guidance at Big Ten universities. Future Internet, 16(10), Article 354. https://doi.org/10.3390/fi16100354
Xu, X. S., Liu, J., Zheng, R., Lei, V. N.-L., & An, Q. (2025). Learners' perception of data privacy when using AI language models: Reflective diary analysis of undergraduates in China. Acta Psychologica, 260, Article 105491. https://doi.org/10.1016/j.actpsy.2025.105491
Xue, Y., Chinapah, V., & Zhu, C. (2025). A comparative analysis of AI privacy concerns in higher education: News coverage in China and Western countries. Education Sciences, 15(6), Article 650. https://doi.org/10.3390/educsci15060650
Yao, Y., Duan, J., Xu, K., Cai, Y., Sun, Z., & Zhang, Y. (2024). A survey on large language model security and privacy: The good, the bad, and the ugly. High-Confidence Computing, 4(2), Article 100211. https://doi.org/10.1016/j.hcc.2024.100211
Downloads
Published
Issue
Section
License
Copyright (c) 2026 Oleksandr Muliarevych (Author)
This work is licensed under a Creative Commons Attribution 4.0 International License.
All papers are published under the terms of the Creative Commons Attribution 4.0 International License (CC BY 4.0).
Authors retain copyright and grant ESS Press the right of first publication.
This license permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are properly credited.
