International Journal of Scientific & Technology Research

Home About Us Scope Editorial Board Blog/Latest News Contact Us
10th percentile
Powered by  Scopus
Scopus coverage:
Nov 2018 to May 2020


IJSTR >> Volume 5 - Issue 6, June 2016 Edition

International Journal of Scientific & Technology Research  
International Journal of Scientific & Technology Research

Website: http://www.ijstr.org

ISSN 2277-8616

Using Semantic Similarity In Automated Call Quality Evaluator For Call Centers

[Full Text]



Ria A. Sagum, MCS



Automatic Transcription, Speaker Diarization, Text Similarity checking



Conversation between the agent and client are being evaluated manually by a quality assurance officer (QA). This job is only one of the responsibilities being done by a QA and particularly eat ups a lot of time for them which lead to late evaluation results that may cause untimely response of the company to concerns raised by their clients. This research developed an application software that automates and evaluates the quality assurance in business process outsourcing companies or customer service management implementing sentence similarity. The developed system includes two modules: speaker diarization, which includes transcription and question and answer extraction, and similarity checker, which checks the similarity between the extracted answer and the answer of the call center agent to a question. The system was evaluated for Correctness of the extracted answers, and accurateness of the evaluation for a particular call. Audio conversations were tested for the accuracy of the transcription module which has an accuracy of 27.96%. The Precision, Recall and F-measure of the extracted answer was tested as 78.03%, 96.26% and 86.19% respectively. The Accuracy of the system in evaluating a call is 70%.



[1] Talkdesk. Quality Assurance Best Practices in the Call Center. Internet: https://www.talkdesk.com/blog/call-center-management/quality-assurance-best-practices-in-the-call-center/, 2013 [March 2016].

[2] Genesys. Quality Management: Improve Call Center Quality Assurance and Agent Performance by Completely Understanding Every Conversation. Internet: http://www.genesys.com/platform-services/workforce-optimization/quality-management, 2015 [March 2016].

[3] Linton, I. Demand Media, About Call Center Quality Assurance Programs, Hearst Newspaper, 2015. Internet: http://smallbusiness.chron.com/call-center-quality-assurance-programs-39627.html, 2015 [March 2016].

[4] Hwabgbo, H. Mining Customer Insights with Speech-to-Text Technology. Internet: http://usblogs.pwc.com/emerging-technology/mining-customer-insights-with-speech-to-text-technology/, 2014 [March 2016].

[5] Vreede, S. V. What is Speech Analytics? Call Center and Business Applications. Internet: http://www.smartcustomerservice.com/Articles/What-Is/What-is-Speech-Analytics-Call-Center-and-Business-Applications-98113.aspx, 2014 [March 2016].

[6] Stepanov, E. et.al., “Automatic Summarization of Call-Center Conversations”

[7] Pallota, V. et.al., “Interaction Mining: The new frontier of Call Center Analytics”

[8] Zweig, G., “Automated Quality Monitoring for Call Centers Using Speech and NLP Technologies”, Proceedings of the Human Language Technology Conference of the NAACL, Companion Volume, pages 292–295, June 2006.

[9] "FFmpeg License and Legal Considerations". ffmpeg.org. Retrieved 2012-01-04.

[10] P. Lamere, P. Kwok, W. Walker, E. Gouvea, R. Singh, B. Raj, P. Wolf, "Design of the CMU Sphinx-4 Decoder," cmusphinx.sourceforge.net, 2003 (USA).

[11] S. Meignier, T. Merlin, “LIUM SpkDiarization: An Open Source Toolkit For Diarization,” in Proc. CMU SPUD Workshop, March 2010, Dallas (Texas, USA).

[12] George A. Miller, “WordNet: A Lexical Database for English”, Communications of the ACM Vol. 38, No. 11: 39-41, 1995.

[13] A. Das and S. Bandyopadhyay, “SentiWordNet: Dr Sentiment Knows Everything!”,ACL/HLT Demo Session, Pages 50-55, June2011.

[14] Sourceforge, “Calculate Semantic Similarity”, Internet: https://sourceforge.net/projects/semantics/, August 23, 2012 [March 2016].

[15] E-Learning Tees. “The Overall Similarity Index”, Internet: https://eat.scm.tees.ac.uk/bb8content/resources/recipes/interpretTurnitin.pdf, August 2007 [March 2016].