Here's the latest Weekly Buzz. Links in the doc!  How to A/B Test Your Pricing (And Why It Might Be a Bad Idea) by HubSpot  Demystifying Multivariate Testing by Merritt Aho from Search Discovery  AB-Testing: A Trade-Off Between Profit and Social Responsibility? by Dennis Meisner Have an interesting piece of research or blog to share? Please reach out to Bithika Mehra, MBA, MSBA.

1. THE
WEEKLY
BUZZ
APRIL 30, 2021

2. How to A/B Test
Your Pricing (And
Why It Might Be a
Bad Idea)
SOURCE
Hubspot

3. A/B testing your pricing is not ideal but
if you must, follow these steps...
#1 Test different products/ plans in the
same category i.e. basic plan at $50/mo
for 1 user vs. professional plan at
$140/mo for 2 users - more than double.

4. #2 Figure out price points to test based
on compett./ cost/ value based pricing.
#3 Measure revenue not conversions
and choose the price that maximizes
revenue.
#4 Iteratively test new price points.

5. Alternative to A/B testing - survey
prospective customers and ask:
“At what point would <product> be too expensive?”
OR
“What is the maximum price you're willing to pay for
<product>?”
Check out the full post.

6. Demystifying
Multivariate
Testing
SOURCE
Search Discovery

7. A/B/n shortened to “multi-variant” is
not the same as Multivariate
Experiments (MVTs).
MVTs are factorial experiment designs
which focus on efficiently assessing
how multiple types of interventions
(headline, CTA, testimonials etc.)
influence a metric (conversion rate).

8. Each intervention is a factor, and each
factor has multiple variations called
levels.
Factor -> CTA; Levels -> Green, Blue
MVTs help determine main effects
(observed effect of single factor) and
interaction effects (observed effects of
combination of factors).

9. Sample size increases based on types
of effects measured - main to
secondary to tertiary.
MVTs not only determine which factors
and levels work best, but also their
performance dependencies which A/B
tests don't.
For analyzing MVTs, check out the post here.

10. AB-Testing: A
Trade-Off Between
Profit and Social
Responsibility?
SOURCE
Dennis Meisner

11. Getting started with AB-Testing could
not be simpler. Hence, it is more
important than ever to educate teams
about the potential side effects of their
changes.

12. Here’s how:
#1 Encourage experimenters to think
about equality and fairness when
designing tests.
#2 Focus on a broader set of metrics
beyond just conversion rates.
#3 Segment results to understand
impact on specific user groups.

13. #4 A technical solution: Atkinson Index
Commonly used by economists,
experimenters can measure the
inequality impact by calculating the
Atkinson Index for each treatment
group and comparing it to the index of
the control group.
For more on Atkinson Index, check out the post here.

14. Sponsored by