Perbandingan Evaluasi Kernel Support Vector Machine dalam Analisis Sentimen Chatbot AI pada Ulasan Google Play Store

Authors

  • Celine Mutiara Putri Universitas Islam Negeri Sultan Syarif Kasim Riau
  • M. Afdal Universitas Islam Negeri Sultan Syarif Kasim Riau
  • Rice Novita Universitas Islam Negeri Sultan Syarif Kasim Riau
  • Mustakim Mustakim Universitas Islam Negeri Sultan Syarif Kasim Riau

DOI:

https://doi.org/10.32493/jtsi.v7i3.41354

Keywords:

ChatGPT, Perplexity AI, Microsoft Bing Chat, Sentiment analysis, Support Vector Machine, Kernel, Hyperparameter

Abstract

AI (Artificial Intelligence) is becoming very important these days due to its ability as a personal assistant to increase efficiency, automate routine tasks, and speed up manual processes. AI chatbot are one of the practical applications of AI in language understanding, have various benefits and drawbacks that cause various comments from users in the review column on the Google Play Store. This research discusses sentiment analysis of AI chatbot application reviews using four SVM kernels. Labeling uses InSet Lexicon and hyperparameters to produce the best parameters. The purpose of the research is to find out how users respond to interactions with ChatGPT, Perplexity AI, and Bing Chat and prove whether the kernel in SVM can increase the accuracy value. The percentage division between test data and training data is 70:30, 80:20, and 90:10, data labeling using 2 sentiment classes and 3 sentiment classes, and using and not using the SMOTE Oversampling technique. The experimental results obtained the highest accuracy using SVM kernel Linear scenario 90:10 with an accuracy value of 92.68%.

References

Ananda, D., & Suryono, R. R. (2024). Analisis Sentimen Publik Terhadap Pengungsi Rohingya di Indonesia dengan Metode Support Vector Machine dan Naï ve Bayes. Jurnal Media Informatika Budidarma, 8(2), 748–757.

Auliaddina, S., & Arifin, T. (2024). Use of Augmentation Data and Hyperparameter Tuning in Batik Type Classification using the CNN Model. Sistemasi: Jurnal Sistem Informasi, 13(1), 114–128.

Caramiaux, B., & Fdili Alaoui, S. (2022). “Explorers of Unknown Planets”: Practices and Politics of Artificial Intelligence in Visual Arts. Proc. ACM Hum.-Comput. Interact., 6(CSCW2). https://doi.org/10.1145/3555578

Ernawati, S., & Wati, R. (2024). Evaluasi Performa Kernel SVM dalam Analisis Sentimen Review Aplikasi ChatGPT Menggunakan Hyperparameter dan VADER Lexicon. Jurnal Buana Informatika, 15(1), 40–49.

Gezici, B., Bölücü, N., Tarhan, A., & Can, B. (2019). Neural Sentiment Analysis of User Reviews to Predict User Ratings. 2019 4th International Conference on Computer Science and Engineering (UBMK), 629–634. https://doi.org/10.1109/UBMK.2019.8907234

Iorliam, A., & Ingio, J. A. (2024). A Comparative Analysis of Generative Artificial Intelligence Tools for Natural Language Processing. Journal of Computing Theories and Applications, 1(3), 311–325. https://doi.org/10.62411/jcta.9447

Jarrahi, M. H. (2018). Artificial intelligence and the future of work: Human-AI symbiosis in organizational decision making. Business Horizons, 61(4), 577–586. https://doi.org/https://doi.org/10.1016/j.bushor.2018.03.007

Jiang, Y., Li, X., Luo, H., Yin, S., & Kaynak, O. (2022). Quo vadis artificial intelligence? Discover Artificial Intelligence, 2(1), 4. https://doi.org/10.1007/s44163-022-00022-8

Kavak, A. C. (2023). ChatGPT, Google Bard, Microsoft Bing, Claude, and Perplexity: Which is the Right AI Tool? In Zeo.org. https://zeo.org/resources/blog/chatgpt-google-bard-microsoft-bing-claude-and-perplexity-which-is-the-right-ai-to

Kumar, P., & Prusty, M. (2020). Outlier-SMOTE: A refined oversampling technique for improved detection of COVID-19. Intelligence-Based Medicine, 3–4, 100023. https://doi.org/10.1016/j.ibmed.2020.100023

Liu, B. (2022). Sentiment analysis and opinion mining. Springer Nature.

Nanda, M. A., Seminar, K. B., Nandika, D., & Maddu, A. (2018). A comparison study of kernel functions in the support vector machine and its application for termite detection. Information, 9(1), 5.

Nurkholis, A., Alita, D., & Munandar, A. (2022). Comparison of Kernel Support Vector Machine Multi-Class in PPKM Sentiment Analysis on Twitter. Jurnal RESTI (Rekayasa Sistem Dan Teknologi Informasi), 6(2 SE-Information Systems Engineering Articles). https://doi.org/10.29207/resti.v6i2.3906

Permata Aulia, T. M., Arifin, N., & Mayasari, R. (2021). Perbandingan Kernel Support Vector Machine (Svm) Dalam Penerapan Analisis Sentimen Vaksinisasi Covid-19. SINTECH (Science and Information Technology) Journal, 4(2), 139–145. https://doi.org/10.31598/sintechjournal.v4i2.762

Prastyo, P. H., Ardiyanto, I., & Hidayat, R. (2020a). Indonesian Sentiment Analysis: An Experimental Study of Four Kernel Functions on SVM Algorithm with TF-IDF. 2020 International Conference on Data Analytics for Business and Industry: Way Towards a Sustainable Economy, ICDABI 2020. https://doi.org/10.1109/ICDABI51230.2020.9325685

Prastyo, P. H., Ardiyanto, I., & Hidayat, R. (2020b). Indonesian Sentiment Analysis: An Experimental Study of Four Kernel Functions on SVM Algorithm with TF-IDF. 2020 International Conference on Data Analytics for Business and Industry: Way Towards a Sustainable Economy (ICDABI), 1–6. https://doi.org/10.1109/ICDABI51230.2020.9325685

Rabbani, S., Safitri, D., Rahmadhani, N., & Anam, M. K. (2023). Perbandingan Evaluasi Kernel SVM untuk Klasifikasi Sentimen dalam Analisis Kenaikan Harga BBM: Comparative Evaluation of SVM Kernels for Sentiment Classification in Fuel Price Increase Analysis. MALCOM: Indonesian Journal of Machine Learning and Computer Science, 3(2), 153–160.

Rianti, D. L., Umaidah, Y., & Voutama, A. (2021). Tren Marketplace Berdasarkan Klasifikasi Ulasan Pelanggan Menggunakan Perbandingan Kernel Support Vector Machine. STRING (Satuan Tulisan Riset Dan Inovasi Teknologi), 6(1), 98–105. https://doi.org/10.30998/string.v6i1.9993

Ribas, J. (2023). Building the New Bing. Bing Blogs. https://blogs.bing.com/search-quality-insights/february-2023/Building-the-New-Bing/

Rudd, J. M. (2018). Application of support vector machine modeling and graph theory metrics for disease classification. Model Assisted Statistics and Applications, 13(4), 341–349.

Rudolph, J., Tan, S., & Tan, S. (2023). War of the chatbot: Bard, Bing Chat, ChatGPT, Ernie and beyond. The new AI rush and its impact on higher education. Journal of Apllied Learning & Teaching, 6(1), 1–9. https://doi.org/https://doi.org/10.37074/jalt.2023.6.1.23 Abstract

Saeidnia, H. R., Hosseini, E., Abdoli, S., & Ausloos, M. (2024). Unleashing the power of AI: a systematic review of cutting-edge techniques in AI-enhanced scientometrics, webometrics and bibliometrics. Library Hi Tech, ahead-of-p(ahead-of-print). https://doi.org/10.1108/LHT-10-2023-0514

Sari, P. K., & Suryono, R. R. (2024). Komparasi Algoritma Support Vector Machine dan Random Forest untuk Analisis Sentimen Metaverse. Jurnal Mnemonic, 7(1), 31–39.

Sengkey, D. F., Jacobus, A., & Manoppo, F. J. (2020). Effects of kernels and the proportion of training data on the accuracy of svm sentiment analysis in lecturer evaluation. IAES International Journal of Artificial Intelligence, 9(4), 734–743. https://doi.org/10.11591/ijai.v9.i4.pp734-743

Sullivan, M. (2023). Is Perplexity AI showing us the future of search? https://www.fastcompany.com/90883562/is-perplexity-ai-showing-us-the-future-of-search

Tuttle, J. F., Blackburn, L. D., & Powell, K. M. (2020). On-line classification of coal combustion quality using nonlinear SVM for improved neural network NOx emission rate prediction. Computers & Chemical Engineering, 141, 106990.

Wahyu Handani, S., Intan Surya Saputra, D., Hasirun, Mega Arino, R., & Fiza Asyrofi Ramadhan, G. (2019). Sentiment analysis for go-jek on google play store. Journal of Physics: Conference Series, 1196(1). https://doi.org/10.1088/1742-6596/1196/1/012032

Wu, T., He, S., Liu, J., Sun, S., Liu, K., Han, Q.-L., & Tang, Y. (2023). A Brief Overview of ChatGPT: The History, Status Quo and Potential Future Development. IEEE/CAA Journal of Automatica Sinica, 10(5), 1122–1136. https://doi.org/10.1109/JAS.2023.123618

Xu, Y., Liu, X., Cao, X., Huang, C., Liu, E., Qian, S., Liu, X., Wu, Y., Dong, F., Qiu, C.-W., Qiu, J., Hua, K., Su, W., Wu, J., Xu, H., Han, Y., Fu, C., Yin, Z., Liu, M., … Zhang, J. (2021). Artificial intelligence: A powerful paradigm for scientific research. The Innovation, 2(4). https://doi.org/10.1016/j.xinn.2021.100179

Zirar, A., Ali, S. I., & Islam, N. (2023). Worker and workplace Artificial Intelligence (AI) coexistence: Emerging themes and research agenda. Technovation, 124, 102747. https://doi.org/https://doi.org/10.1016/j.technovation.2023.102747

Jurnal Teknologi Sistem Informasi dan Aplikasi Vol. 7 No. 3 Juli 2024

Downloads

Published

2024-07-31

How to Cite

Putri, C. M., Afdal, M., Novita, R., & Mustakim, M. (2024). Perbandingan Evaluasi Kernel Support Vector Machine dalam Analisis Sentimen Chatbot AI pada Ulasan Google Play Store. Jurnal Teknologi Sistem Informasi Dan Aplikasi, 7(3), 1236–1245. https://doi.org/10.32493/jtsi.v7i3.41354

Issue

Vol. 7 No. 3 (2024): Jurnal Teknologi Sistem Informasi dan Aplikasi

Section

Article

License

Copyright (c) 2024 Celine Mutiara Putri, M. Afdal, Rice Novita, Mustakim Mustakim

Creative Commons License

This work is licensed under a Creative Commons Attribution-NonCommercial 4.0 International License.

Authors who publish with this journal agree to the following terms:

  1. Authors retain copyright and grant the journal right of first publication with the work simultaneously licensed under a Creative Commons Attribution License that allows others to share the work with an acknowledgement of the work's authorship and initial publication in this journal.
  2. Authors are able to enter into separate, additional contractual arrangements for the non-exclusive distribution of the journal's published version of the work (e.g., post it to an institutional repository or publish it in a book), with an acknowledgement of its initial publication in this journal.
  3. Authors are permitted and encouraged to post their work online (e.g., in institutional repositories or on their website) prior to and during the submission process, as it can lead to productive exchanges, as well as earlier and greater citation of published work (See The Effect of Open Access).

Jurnal Teknologi Sistem Informasi dan Aplikasi have CC BY-NC or an equivalent license as the optimal license for the publication, distribution, use, and reuse of scholarly work.

In developing strategy and setting priorities, Jurnal Teknologi Sistem Informasi dan Aplikasi recognize that free access is better than priced access, libre access is better than free access, and libre under CC BY-NC or the equivalent is better than libre under more restrictive open licenses. We should achieve what we can when we can. We should not delay achieving free in order to achieve libre, and we should not stop with free when we can achieve libre.

This work is licensed under a Creative Commons Attribution-NonCommercial 4.0 International (CC BY-NC 4.0) License

YOU ARE FREE TO:

  • Adapt - remix, transform, and build upon the material for any purpose, even commercially.
  • The licensor cannot revoke these freedoms as long as you follow the license terms