Paper presented at the DART 2011 workshop in Palermo. The paper introduces a new type of call center analytics based on interaction mining. It shows how advanced metrics and KPIs for call center quality management can be implemented through interAnalytics NLP technology.

1. Interaction Mining: the new frontier of Call Center Analytics Vincenzo Pallotta Rodolfo Delmonte LammertVrieling David Walker © 2011 interAnalytics 1

2. Outline Call Center Analytics Automatic Argumentative Analysis for Interaction Mining Experiments with Call Center Data Conclusions © 2011 interAnalytics 2

3. Call Center Analytics © 2011 interAnalytics 3

4. Call Center Analytics Call centers data represent a valuable asset for companies, but it is often underexploited for business purposesbecause: it is highly dependent on quality of speech recognition technology it is mostly based on text-based content analysis. Interaction Mining as a viable alternative: more robust tailored for the conversational domain slanted towards pragmatic and discourse analysis © 2011 interAnalytics 4

5. Mainstream Call Center Analytics © 2011 interAnalytics 5 Does not unveil real insights about customer satisfaction

6. Call Center Analytics: metrics and KPIs Agent Performance Statistics: Average Speed of Answer, Average Hold Time, Call Abandonment Rate, Attained Service Level, and Average Talk Time. Quantitative measurements that can be obtained directly through ACD (Automatic Call Distribution), Switch Output and Network Usage Data. Peripheral Performance Data: Cost Per Call, First-Call Resolution Rate, Customer Satisfaction, Account Retention, Staff Turnover, Actual vs. Budgeted Costs, and Employee Loyalty. Quantitative, with the exception of Customer Satisfaction that is usually obtained through Customer Surveys. Performance Observation: Call Quality, Accuracyand Efficiency, Adherence to Script, Communication Etiquette, and Corporate Image Exemplification. Qualitativemetrics based on analysis of recorded calls and session monitoring by a supervisor. © 2011 interAnalytics 6

7. Four objectives Identify Customer Oriented Behaviors, which are highly correlated to positive customer ratings (Rafaeli et al. 2007); Identify Root Cause of Problems by looking at controversial topics and how agents are able to deal with them; Identify customers who need particular attention based on history of problematic interactions; Learn best practices in dealing with customers by identifying agents able to carry cooperative conversations. © 2011 interAnalytics 7

8. Argumentative Analysis for Interaction Mining © 2011 interAnalytics 8

9. Argumentative Structure of Conversations REJECT(alternative) 1708.89 John: not before having one come here and have some people try it out . {s^arp^co} 61b.62a PROVIDE(justification) 1714.09 B: because there's no point in doing that if it's not going to be any better . {s} 61b+ PROVIDE(justification) 1714.09 John: because there's no point in doing that if it's not going to be any better . {s} 61b+ ACCEPT(justification) 1712.69 David: okay . {s^bk} 62b PROPOSE(alternative) 1716.85 John: so why don't we get one of these with the crown with a different headset ? {qw^cs} 63a PROVIDE(justification) 1722.4 John: and - and see if that works . {s^cs} 63a+.64a 1723.53 Mark: and see if it's preferable and if it is then we'll get more . {s^cs^2} 64b 1725.47 Mark: comfort . {s} ACCEPT(alternative) 1721.56 David: yeah . {s^bk} 63b 1726.05 Lucy: yeah . {b} 1727.34 John: yeah . {b} DISCUSS(issue) <- PROPOSE(alternative) 1702.95 David: so - so my question is should we go ahead and get na- - nine identical head mounted crown mikes ? {qy} 61a Why was David’s proposal on microphones rejected? © 2011 interAnalytics 9

10. Automatic Argumentative Analysis Based on the GETARUNS system1. Clauses in Turns are labelled with Primitive Discourse Relations: statement, narration, adverse, result, cause, motivation, explanation, question, hypothesis, elaboration, permission, inception, circumstance, obligation, evaluation, agreement, contrast, evidence, hypoth, setting, prohibition. And then Turns are labelled with Argumentative labels: ACCEPT, REJECT/DISAGREE, PROPOSE/SUGGEST, EXPLAIN/JUSTIFY, REQUEST EXPLANATION/JUSTIFICATION. 1 DelmonteR., Bistrot A., Pallotta V.,Deep Linguistic Processing with GETARUNS for spoken dialogue Understanding. Proceedings LREC 2010 (P31 Dialogue Corpora). © 2011 interAnalytics 10

11. Evaluation ICSI corpus of meetings (Janin et al., 2003) Precision: 81.26% Recall: 97.53% Delmonte R., Bistrot A., Pallotta V.,Deep Linguistic Processing with GETARUNS for spoken dialogue Understanding. Proceedings LREC 2010 (P31 Dialogue Corpora). © 2011 interAnalytics 11

12. Experiments with Call Center data © 2011 interAnalytics 12

13. Rationale: implement the four objectives Identify Customer Oriented Behaviors, Identify Root Cause of Problems Identify customers who need particular attention Learn best practices in dealing with customers © 2011 interAnalytics 13

14. The Data Corpus of 213 manually transcribed conversations of a help desk call center in the banking domain. Average of 66turns per conversation. Average of 1.6 calls per agent. Collected for a study aimed at identifying customer oriented behaviors that could favor satisfactory interaction with customers (Rafaeli et al. 2007). © 2011 interAnalytics 14

15. Identify Customer Oriented Behaviors Based on the work of Rafaeli et al. 2006. Customer Oriented Behaviors anticipating customers requests 22,45% educating the customer 16,91% offering emotional support 21,57% offering explanations / justifications 28,57% personalization of information 10,50% © 2011 interAnalytics 15

16. Significant correlation with argumentative labels © 2011 interAnalytics 16

17. Identify Root Cause of Problems Cooperativeness score a measure obtained by averaging the score obtained by mapping argumentative labels of each turn in the conversation into a [-5 +5] scale. Sentiment Analysis module. © 2011 interAnalytics 17

18. Top 20 Controversial Topics with average cooperativeness scores and sentiment © 2011 interAnalytics 18

19. Cooperativeness of speakers on top discussed topics © 2011 interAnalytics 19

20. Identify problematic customers © 2011 interAnalytics 20

21. Select a specific customer © 2011 interAnalytics 21

22. Visualize a selected call © 2011 interAnalytics 22

23. Conclusions © 2011 interAnalytics 23

24. Conclusions New Generation Call Center Analytics requires Interaction Mining Call Center Qualitative metrics and KPIs can be only implemented with a full understanding of the customer interaction dynamics Argumentation is pervasive in conversations. In order to recognize argumentative acts, advanced Natural Language Understanding is necessary. Future work: Scalability: need to process millions of call per day! Multi-language: call centers all over the world. © 2011 interAnalytics 24

25. The Team © 2011 interAnalytics 25

26. …find us at www.interanalytics.ch