The document discusses Dell's approach to driving customer experience through testing, research, and analytics. It outlines Dell's tenets which include being customer obsessed, acting as subject matter experts, and delivering advanced analytics in a consumable manner. Several case studies are presented showing how iterative testing of features and changes to the Dell website helped improve key metrics like conversion rates and revenue. The importance of continuous testing, data collection, and keeping changes simple for customers is emphasized.

1. Driving Customer Experience
Through Testing, Research &
Analytics
Vab Dwivedi
Dell | Director, Testing & Research
@vabdwivedi
linkedin.com/in/vabdwivedi

2. 2 of Y
What’s our approach?

3. 3 of Y
Our tenets
W1
A2
D3
G4
S5
B6

4. 4 of Y
Our tenets
We Are Customer Obsessed1
A2
D3
G4
S5
B6

5. 5 of Y
Our tenets
W1
Act as Subject Matter Experts2
D3
G4
S5
B6

6. 6 of Y
Our tenets
W1
A2
Deliver Advanced Analytics In A Consumable Manner3
G4
S5
B6

7. 7 of Y
Our tenets
W1
A2
D3
Guide Decision Making Through Insights And Experimentation4
S5
B6

8. 8 of Y
Our tenets
W1
A2
D3
G4
Serve As A Channel of Knowledge5
B6

9. 9 of Y
Our tenets
W1
A2
D3
G4
S5
Be Agile6

10. 10 of Y
Our tenets
W1
A2
D3
G4
S5
B6

11. 11 of Y
Holding ourselves accountable
In1
D2
D3
M4
E5
D6

12. 12 of Y
Holding ourselves accountable
Influencing Agenda/Roadmap1
D2
D3
M4
E5
D6

13. 13 of Y
Holding ourselves accountable
In1
Delivering Actionable Insights2
D3
M4
E5
D6

14. 14 of Y
Holding ourselves accountable
In1
D2
Delighting Our Customers3
M4
E5
D6

15. 15 of Y
Holding ourselves accountable
In1
D2
D3
Managing Stakeholders4
E5
D6

16. 16 of Y
Holding ourselves accountable
In1
D2
D3
M4
Executing With Operational Excellence5
D6

17. 17 of Y
Holding ourselves accountable
In1
D2
D3
M4
E5
Driving An Innovative Vision6

18. 18 of Y
Holding ourselves accountable
In1
D2
D3
M4
E5
D6

19. Setting our team up for
success

20. 20 of Y

21. 21 of Y

22. 22 of Y

23. 23 of Y

24. 24 of Y
That was ‘how’…let’s
look at some projects…

25. Dell.com Home Page

26. 26 of Y

27. 27 of Y

28. 28 of Y

29. 29 of Y

30. 30 of Y
Iterative testing over 18 months…

31. 31 of Y
Iterative testing over 18 months…

32. 32 of Y
We measured EVERYTHING!

33. 33 of Y
We measured EVERYTHING!

34. 34 of Y
We measured EVERYTHING!

35. 35 of Y
We measured EVERYTHING!

36. 36 of Y
We measured EVERYTHING!

37. 37 of Y
We measured EVERYTHING!

38. 38 of Y
…And then some more…

39. 39 of Y
…And then some more…

40. 40 of Y
…And then some more…

41. 41 of Y
…And then some more…

42. 42 of Y
…And then some more…

43. 43 of Y
…Look into device type performance…

44. 44 of Y
…Look into device type performance…

45. 45 of Y
…Look into device type performance…

46. 46 of Y
…Look into device type performance…

47. Drive Big Change:
A new Dell.com

48. 48 of Y

49. 49 of Y

50. 50 of Y
Not just a new visual identity…

51. 51 of Y
Not just a new visual identity…
The new Dell.com was new in every way….platform,
architecture, back end services

52. 52 of Y
Not just a new visual identity…
The new Dell.com was new in every way….platform,
architecture, back end services
Before going live, test it to ensure a win for customers
and the business

53. 53 of Y
Not just a new visual identity…
The new Dell.com was new in every way….platform,
architecture, back end services
Before going live, test it to ensure a win for customers
and the business
Expecting a quick win….

54. 54 of Y
Not just a new visual identity…
The new Dell.com was new in every way….platform,
architecture, back end services
Before going live, test it to ensure a win for customers
and the business
Expecting a quick win….
… -33% revenue/visit

55. 55 of Y
OK, no big deal…we know what to do

56. 56 of Y
OK, no big deal…we know what to do
Let’s introduce UI enhancements

57. 57 of Y
OK, no big deal…we know what to do
Let’s introduce UI enhancements
24 of them…through various parts in the site

58. 58 of Y
OK, no big deal…we know what to do
Let’s introduce UI enhancements
24 of them…through various parts in the site
Recipe C will be our super recipe that will bridge the
gap when we re-run the test

59. 59 of Y
OK, no big deal…we know what to do
Let’s introduce UI enhancements
24 of them…through various parts in the site
Recipe C will be our super recipe that will bridge the
gap when we re-run the test
… -45% revenue/visit

60. 60 of Y
Maybe its not the UI…

61. 61 of Y
Maybe its not the UI…
It is a new platform…

62. 62 of Y
Maybe its not the UI…
It is a new platform…
We need re-run and this time look at the metrics that
can point to any issues with the platform

63. 63 of Y
Performance metrics

64. 64 of Y
Performance metrics

65. 65 of Y
Performance metrics

66. 66 of Y
Performance metrics

67. 67 of Y
Performance metrics

68. 68 of Y
Performance metrics

69. 69 of Y
Performance metrics

70. 70 of Y
Well that explains it…

71. 71 of Y
Well that explains it…
Now we can work with engineering and provide
guidance on where the issues are

72. 72 of Y
Well that explains it…
Now we can work with engineering and provide
guidance on where the issues are
Emergency sprints scheduled to address live issues

73. 73 of Y
Well that explains it…
Now we can work with engineering and provide
guidance on where the issues are
Emergency sprints scheduled to address live issues
Re-test now….

74. 74 of Y
Well that explains it…
Now we can work with engineering and provide
guidance on where the issues are
Emergency sprints scheduled to address live issues
Re-test now….
… +3% revenue/visit!!!

75. Find something small that
can make an impact:
Continue shopping

76. 76 of Y
Should ‘Continue Shopping’ be smarter?

77. 77 of Y
Should ‘Continue Shopping’ be smarter?

78. 78 of Y
Should ‘Continue Shopping’ be smarter?

79. 79 of Y
Should ‘Continue Shopping’ be smarter?

80. 80 of Y
Did it work?

81. 81 of Y
Did it work?

82. 82 of Y
Are you sure?

83. 83 of Y
Are you sure?

84. 84 of Y
Are you sure?

85. 85 of Y
Are you sure?

86. 86 of Y
Now iterate, globalize and future proof

87. 87 of Y
Now iterate, globalize and future proof

88. 88 of Y
Now iterate, globalize and future proof

89. 89 of Y
Now iterate, globalize and future proof

90. 90 of Y
Now iterate, globalize and future proof

91. 91 of Y
Now iterate, globalize and future proof

92. 92 of Y
Now iterate, globalize and future proof

93. 93 of Y
Now iterate, globalize and future proof

94. Rapid fire: Everything doesn’t
have to be a ‘project’…

95. 95 of Y
Cycling through order codes
vs

96. 96 of Y
Cycling through order codes
vs
+9.1% Conversion
+11.9% Revenue/Visit
+91.6% Button Engagement
-1.49% Exit rates
+.6% Compare usage

97. 97 of Y
Adding payment methods to cart
vs

98. 98 of Y
Adding payment methods to cart
vs
+4.3% Conversion
+10.5% Revenue/Visit
+2.2% Payment Pg > Confirm Page

99. 99 of Y
Results sorting method
vs

100. 100 of Y
Results sorting method
vs
+6.7% Conversion
+15% Revenue/Visit
+51% Conversion for highest price
band
-64% Engagement with sort option

101. 101 of Y
Category page filter layout
vs

102. 102 of Y
Category page filter layout
vs
-7.5% Conversion
-6.4% Revenue/Visit
-40% Engagement
-17% Conversion for
engagers
-8% Next page

103. 103 of Y
Sequencing options for systems
Based on business rules
and availability
Hardware, Software,
Services, Misc.
vs

104. 104 of Y
Sequencing options for systems
Based on business rules
and availability
Hardware, Software,
Services, Misc.
vs
+13.8% Conversion
+15.5% Revenue/Visit
+12.7% Margin/Visit

105. 105 of Y
Clickable tabs and ‘Add to Cart’ CTA
vs

106. 106 of Y
Clickable tabs and ‘Add to Cart’ CTA
vs
+11% Conversion
+52% Engagement w/tabs & CTAs
+13% Movement through funnel
-2% Exit rate

107. 107 of Y
Sign in page titles
vs

108. 108 of Y
Sign in page titles
vs
+3% Conversion
-4% Guest checkout
+3.7% Movement through funnel
+1% Authentication

109. 109 of Y
Navigation title change
vs

110. 110 of Y
Navigation title change
-3.5% Conversion
-5.8% Revenue/Visit
-3% Engagement
No Change Movement to
browse
vs

111. If you only remember
three things from
today…

112. 112 of Y
Our daily mantra

113. 113 of Y
Our daily mantra
You want iterative testing & a LOT of data1
Winners and Losers are good2
Keep it simple for your customer3

114. 114 of Y

115. Driving Customer Experience
Through Testing, Research &
Analytics
Vab Dwivedi
Dell | Director, Testing & Research
@vabdwivedi
linkedin.com/in/vabdwivedi