As part of Couchbase[SF]2013, Ido Shilon, R&D Group Leader at LivePerson, discusses LivePerson's project to re-architect their LiveEngage platform backend, with a focus on LivePerson's decision to use NoSQL technologies, the challenges encountered, and the lessons learned. Video: http://www.youtube.com/watch?v=rYKWFmJEHX0

1. I get by with a little help
from my friends
Ido Shilon | 7/22/2013

2. Today’s programme
Who is LivePerson?
What was our journey?
Why was Couchbase chosen?
How did we build the new platform?
What can you take with you?

3. @idoshilon
{
name: "Ido Shilon",
age: 37,
kids: [
"illy"
],
wife: "Oshrat",
Title: "Group Manager @ LivePerson (Heading the
platform group)",
Lived_Worked_At: [
"Silicon Wadi (Israel)",
"Silicon Alley (NYC)",
"Silicon Valley (Bay Area)"
]
}

4. LivePerson is…
Mission
Customers
Creating Meaningful
Customer Connections
8,500
customers
Technology
SaaS pioneer since 1998

5. Optimize Customer Acquisition & Reduce Bounce Rate
Live engagement for
Rich multimedia to
lingering customer
drive sales closure
Recommended use case

6. Technology @ LivePerson
Application Stack
JVM heavy - Java & Scala
Private cloud based on openstack
Linux on commodity servers

7. Data @ LP
13
VOLUME
TB
per
month
20
M
Engagements
per
month
1.8
B
Visits
per
month

8. Data stack
MONITORING
CHAT/VOICE
system
APACHE KAFKA
Batch track
Real-Time track
PERPETUAL STORE
COMPLEX EVENT
PROCESSING
STORM
ANALYTICAL DB
RT REPOSITORY
BUSINESS INTELLIGENCE
Cassandra
LiveEngage
DASHBOARD

9. The use case
Web agent console
Enables your agents to interact with
website visitors
Improve agent efficiency
Reduce chat time

10. The story - once upon a time
Agents console
(Java app)
Web Tier
Visitor’s
Events
Visitors

11. And then the story continues
Web Agent
???
Data center 1
Kafka & Strom
(Event bus)
Data center 2

12. Possible solutions we considered

13. Why did we pick Couchbase
Cassandra
Always on
Linear scale
Searchable
Document store
Key Value
High throughput (R/W)
XDCR

14. Architecture
REST API
Application server
Tomcat
Couchbase Java SDK
cluster
cluster
XDCR
M/R views
M/R views
Storm Topology
Storm Topology
Couchbase Java SDK
Couchbase Java SDK

15. Data flow
Agent
Visitor
(5) Get
visitors List
Every 3 sec
(7) Return
relevant
visitors
(1) Visitor
browsing
Visitor
Monitoring
Service
Visitor Feed
API
(2)
Visitor
events
Kafka
(6) Return
relevant
visitors
Couchbase
(4) Write
event to visitor
document
Visitor Feed Storm Topology
(3) Analyze relevant
events

16. Data modeling - doc
Visitor type doc
Document = Active visitor
Contain session level attributes
Multi tenant bucket
Views
Used for secondary indexes

17. Document structure
{
"accountId": "64302875",
"id": 121640710013,
"rtSessionId": "643028754295878498",
"eventSequence": 5104,
"ipAddress": {
"fieldValue": "194.39.63.10",
"seq": 1
},
Multi tenant DB
"browser": {
"fieldValue": "Chrome 27.0.1453.116",
"seq": 1
},
Basic visitor
"state": {
information
"fieldValue": "LEFT_SITE",
"seq": 5104
}
......................................
Sequence use
due to Kafka
}

18. Cross data center replication
Using Bi Directional replication (A/A)
Tips :
Key space is the same across the two
clusters (avoid conflicts)
Impact on the sizing

19. Couchbase in production

20. What did we learn till now ?
Test in production
Use the Couchbase sizing guidelines
RAM and SSD are key factors in
scalability

21. Data stack now with Couchbase
MONITORING
CHAT/VOICE
system
APACHE KAFKA
Batch track
Real-Time track
PERPETUAL STORE
COMPLEX EVENT
PROCESSING
STORM
ANALYTICAL DB
RT REPOSITORY
BUSINESS INTELLIGENCE
Cassandra
LiveEngage
DASHBOARD

22. Couchbase in LP - Strategic choice
Visitor Session state
Cross Session state
Fast RW persistent store
Generic caching layer (Memcached
style)

23. Summary
Who we are
Our journey
The problem we have encountered
We are not alone - Couchbase helped us

24. Thank You