Pengembangan Chatbot Perpustakaan Berbasis Retrieval-Augmented Generation (RAG) pada Telegram

Elrica Meriana Isabel Tutu; Benny Pinontoan; Edwin Tenda; Christian A. J. Soewoeh; Mahardika I. Takaendengan; Stephano C. W. Ngangi

doi:10.54895/intech.v7i1.3435

Authors

Elrica Meriana Isabel Tutu Universitas Sam Ratulangi Author
Benny Pinontoan Universitas Sam Ratulangi Author
Edwin Tenda Universitas Sam Ratulangi Author
Christian A. J. Soewoeh Universitas Sam Ratulangi Author
Mahardika I. Takaendengan Universitas Sam Ratulangi Author
Stephano C. W. Ngangi Universitas Sam Ratulangi Author

DOI:

https://doi.org/10.54895/intech.v7i1.3435

Keywords:

Chatbot, Retrieval-Augmented Generation (RAG), Large Language Model (LLM), Telegram Bot, Library

Abstract

The library of Sam Ratulangi University (UNSRAT) has a large collection of books and academic resources; however, users still experience difficulties in obtaining book availability information quickly and efficiently. This study aims to develop and evaluate a Telegram-based chatbot using a Retrieval-Augmented Generation (RAG) approach integrated with the GPT-4.1-mini Large Language Model (LLM). The system was developed using the Waterfall method and implemented through the n8n workflow automation platform by integrating Telegram Bot, MySQL, and Pinecone as a vector database. The chatbot applies a Text-to-SQL RAG mechanism, where user questions are converted into embeddings, matched with database context, and transformed into SQL queries limited to SELECT operations. System evaluation was conducted using Black Box Testing, User Acceptance Testing (UAT), and RAG evaluation metrics consisting of Answer Relevancy and Faithfulness. The results show that the chatbot successfully performs its main functions and achieved a UAT score of 82.3%, while Answer Relevancy and Faithfulness obtained scores of 100%. The developed system is capable of providing relevant and interactive information regarding library collections.

Downloads

Download data is not yet available.

References

Fardi C. F. P., Anwar R. K., Amar S. C. D., and Rukmana E. N., “Perkembangan Pelayanan Perpustakaan Perguruan Tinggi,” Jurnal Perpustakaan dan Kearsipan, vol. 4, no. 2, pp. 60–72, 2024, doi: https://doi.org/10.24821/jap.v4i2.9610.

Lubis A. T. U., Harahap N. S., Agustian S., Irsyad M., and Afrianty I., “Question Answering System on Telegram Chatbot Using Large Language Models (LLM) and Langchain (Case Study: Health Law),” MALCOM: Indonesian Journal ofMachine Learningand Computer Science, vol. 4, no. 3, pp. 955–964, Jul. 2024, doi: https://doi.org/10.57152/malcom.v4i3.1378.

Gao Y. et al., “Retrieval-Augmented Generation for Large Language Models: A Survey,” pp. 1–21, Mar. 2024.

Bang Y. et al., “AMultitask, Multilingual, Multimodal Evaluation of ChatGPT on Reasoning, Hallucination, and Interactivity,” Centre for Artificial Intelligence Research (CAiRE), Nov. 2023.

Brown T. B. et al., “Language Models are Few-Shot Learners,” pp. 1877–1901, Dec. 2020, Accessed: Sep. 07, 2025. [Online]. Available: https://dl.acm.org/doi/abs/10.5555/3495724.3495883

Tohir H., Merlina N., and Haris M., “Utilizing Retrieval-Augmented Generation In Large Language Models To Enhance Indonesian Language NPL,” JTIK (Jurnal Ilmu Pengetahuan Dan Teknologi Komputer), vol. 10, no. 2, pp. 352–360, 2024, doi: https://doi.org/10.33480/jitk.v10i2.5916.

Lewis P. et al., “Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks,” https://arxiv.org/abs/2005.11401, 2021.

Samudra G., Zy A. T., and Ermanto, “Implementasi Retrieval Augmented Generation dalam Perancangan Chatbot Kesehatan Pencernaan,” JSAI: Journal Scientific and Applied Informatics, vol. 8, no. 1, pp. 181–188, 2025.

Ghozi W., Rahmawan E., and Setiono O., “Design of a Telegram Chatbot to Control Internet Connection on Computer Laboratory,” Jurnal Mantik, vol. 6, no. 3, pp. 3591–3599, 2022.

Pujiono I., Agtyaputra I.M., and Ruldeviyani Y., “Implementing Retrieval-Augmented Generation and Vector Databases for Chatbots in Public Services Agencies Context,” Jurnal Ilmu Pengetahuan dan Teknologi Komputer, vol. 10, no. 1, pp. 216–223, 2024, doi: https://doi.org/10.33480/jitk.v10i1.5572.

Ramahdani A., Yantoro M. D., Akmal M. F., Mahfud M., and Fauzi, “Chatbot Otomatis dengan N8N dan AI untuk Analisis Data dan Pelaporan Hasil,” Jurnal Riset Teknik Komputer, vol. 2, no. 2, pp. 18–23, 2025, doi: https://doi.org/10.69714/x1p94182.

Hidayat L. R., Wijaya G. P. S., and Dwiyansaputra R., “Optimalisasi Layanan Sistem Informasi Mahasiswa dengan Integrasi Telegram : Chatbot Retrieval-Augmented-Generation berbasis Large Language Model,” Jurnal Teknologi Informasi, Komputer dan Aplikasinya (JTIKA), vol. 7, Mar. 2025, [Online]. Available: http://jtika.if.unram.ac.id/index.php/JTIKA/

Jailani A. and Yaqin M.A., “Pengujian Aplikasi Sistem Informasi Akademik menggunakan Metode Blackbox dengan Teknik Boundary Value Analysis,” Journal Automation Computer Information System, vol. 4, no. 2, pp. 60–66, 2024, doi: https://doi.org/10.47134/jacis.v4i2.78.

Aliyah, Hartono N., and Muin A. A., “Penggunaan User Acceptance Testing(UAT) Pada Pengujian Sistem Informasi Pengelolaan Keuangan Dan Inventaris Barang,” Switch: Jurnal Sains dan Teknologi Informasi, vol. 3, no. 1, pp. 84–100, 2025, doi: https://doi.org/10.62951/switch.v3i1.330.