Uni-Mate: A Retrieval-Augmented Generation System to Provide High School Students with Accurate Academic Guidance

This paper introduces the development and evaluation of a Retrieval-Augmented Generation (RAG) system designed to assist prospective students in navigating university options. The system provides accurate academic guidance by retrieving and synthesizing information on undergraduate and single-cycle master’s degree programs, as well as library resources, from the University of Trento and the University of Verona. The RAG pipeline utilizes a streamlined toolchain, incorporating a Markdown parser for efficient data handling and the Llama3-8b-8192 Large Language Model (LLM) for query processing. The system’s performance was assessed through both automated evaluation, using the Llama3-70b LLM as a reference, and blinded human evaluation. The results demonstrate the system’s potential for providing relevant and accurate information to students. The evaluation also highlighted areas for further development, including enhanced retrieval mechanisms and expanded LLM testing. Future work aims to broaden the system’s scope to include more degree levels and universities, ultimately creating a comprehensive platform to support students in their academic decision-making journey.