PayU is a leading payment services provider with presence in 16 growth markets across the world. Its mantra within IT is "fail early, fail often and never roll back," but this is a challenge in a global environment, with cross-located development and operations teams, multiple time zones, cultures, languages, and skill sets. To solve this challenge and provide transparency to development and production teams, PayU chose the AppDynamics Application Intelligence platform. Today AppDynamics gives PayU the ability to get immediate feedback of code changes regardless of the environment or the origin of change. The solution fits perfectly with the microservice architecture and has helped with DevOps adoption in all locations. Key takeaways: o Challenges faced in monitoring microservice-based applications in a globally dispersed operation o How AppDynamics provides a single pane of glass to monitor application changes o Best practices for utilizing AppDynamics in a DevOps culture For more information, go to: www.appdynamics.com

1. PayU's digital transformation:
Transparency from Dev to Prod, monitoring
microservices and DevOps
Guy Duncan
Chief Technical Officer
Markus Hraba
Infrastructure Architect

2. www.payu.com
PayU's Digital Transformation:
Transparency from Dev to Prod,
monitoring micro services and DevOps
Guy Duncan, Chief Technical Officer
Markus Hraba, Chief Infrastructure Architect

3. years of
experience
Operations in
high-growth
markets
200.000
More than
merchants
Locally trusted brand
for merchants and consumers
Tens
of Billions
of dollars
Part of
Naspers
$
14+
16
TPV
Payment Options
250+
Including: Credit Cards, Bank Transfers,
Cash Payments, e-Wallets
Regulated
in countries
where applicable

4. Classifieds Online
Comparison
Shopping
E-Payment Online ServicesMarket-placesEtail
Listed
E-commerce
Pay TV Print
Naspers is listed on London’s and Johannesburg’s stock
exchanges and has a valuation of over 60
billion USD
!
Classifie
ds
Global platform operator
Pay-TV
Internet
Print
DTH DTT Classifieds e-Tail Payments
FormatType
eCommerce Listed

5. PayU: Expert in High Growth Markets
Argentina ● Brazil ● Chile ● Colombia ● Czech Republic ● Hungary ● India ● Mexico ● Nigeria ● Panama ● Peru
Poland ● Romania ● Russia ● South Africa ● Turkey

6. Optimizing the value chain
Traditional value chain
Merchant
Issuers / Banks
Schemes
Gateway
Payment
Processing
Reporting&
Reconciliation
Card Schemes
Acquirers / Banks
PayU value chain
Risk management
$
Card settlement
Payment
Processing
Reporting&
Reconciliation
Risk management
Card Schemes Alternatives
(Bank Transfers,Cash
Payments)
Wallets
$
Wallet
settlement
Alternatives
settlement
$
$
Card settlement
LATAM CEE INDIA
Multiple AcquiringConnections
MEA

7. More than 200.000 companies around the world
use PayU to accept online payments

8. PayU Offering
Enabling merchants and businesses to monetize their operations over the internet in 16 high growth
countries across the globe.
Direct connections with 40+
local and international
acquirers and banks
Availability of 250+ unique payment
methods including local and international
cards, bank transfer, cash
payment and eWallet
Sales protection through locally
optimised fraud prevention
systems
Secure payment processing through
PCI DSS Level 1 certified payment
platform
Supporting tokenization for one-
click-payments, recurring payments
and instalments
Able to process local currencies
and both settle locally and
internationally

9. Alternative payment methods
Bank Transfers Cash Payments Cash on Delivery
Payment done in cash
upon the delivery of the
goods.
Payments done by
transferring money
from buyer’s bank
account to the
merchant’s virtual
account in PayU.
Payments completed
in cash at authorized
convenience/corner
stores and
bank branches.

10. PayU LATAM PayU CEE PayU MEA PayU India
Local &
International Credit
& Debit card
schemes
Local Alternatives
International Credit
& Debit card
schemes
Local Alternatives
International Credit
& Debit card
schemes
Local Alternatives
Local &
International Credit
& Debit card
schemes
Local Alternatives
Proprietary and
Local Wallet
Local Wallets
PayU Payment Methods – High Level
Full Service PSP Offering Full Service PSP Offering Full Service PSP Offering Full Service PSP Offering
Local Wallets

11. Enabled for Visa, MasterCard
and Amex
Offering Instalments
The credit risk is always assumed
by the bank
Approximately 60% of all retail
ecommerce transactions using credit
cards use instalments.
People use instalments even for purchases as
low as $10 USD.
Instalments go from
1 to up to 48 months.
In some countries the expense of these
instalments can be assumed by the
merchant.
48

12. Why can approval rates be higher?
• International processors do not have as much local knowledge on Fraud and therefore
might reject good transactions.
• Banks in general are more comfortable if their cards gets processed locally rather
than internationally as more fraud occurs internationally. Therefore less restriction on
local transactions.
• Not all consumers have international transactions approved for their credit cards.
Processing Locally
Visa, MasterCard and AMEX

13. Local fraud management
The use of local websites and
credit bureaus to validate buyer
information is key
Why is it important?
Local transactional information, as we process
for main airlines and retailers, we know the
trends of what's happening
Each market is different and
understanding the transaction
behavior in each market can help
detect fraud.
(Example of knowing the zones
of a city)
Local contact with the banks and acquirers can
help resolve doubts and work with the processor
to stop fraud quickly

14. Web Responsive Checkout
Company.com
Company.com

15. Integrations options with PayU
Web Checkout (Desktop & Mobile)
Web Checkout Plugin (Desktop & Mobile)
Direct API Integration
SDK (Software Development Kit) Integration
Virtual Terminal Integration
Standalone back office + Virtual Terminal
PayU offers several types of integrations for payment processing including:
1
2
3
4
5
6

16. Supported Plugins
The following standard shopping cart platform plugins are available for swift
and frictionless integration:
• Magento
• PrestaShop
• OpenCart
• Drupal Commerce
• OsCommerce
• VirtueMart
• WooCommerce
• Joomla
• X-Cart
• CS Cart
• WP E-commerce

17. Polish Single Platform
aka PSP

18. Complexity – 41 Tiers & 188 Nodes

19. Legacy Platform
• Polish Single Platform is
• working in active/passive multi site environment mode
• divided into several monolithic services
• refactoring legacy code step by step into (m)SoA
• bound to PCI-DSS standard
Some sections of the code are older than 10-15 years

20. Challenges
• Big platform – around 100 different components with complex asynchronous flows in
single active DC
• Many entry points to monitor for different types of customers (shops, banks, individual
users, external systems)
• Service degradation or downtime causes significant financial and reputational loss
• Complex compliance & legal regulations
• Monolith architecture has huge impact on scalability
• Bi-weekly release cycle
• Peak season before Christmas time

21. Legacy monitoring stack
• Splunk - log and event processing
• Zabbix - infrastructure monitoring
• dynaTrace - APM
• Selene - synthetic monitoring
• NOC - alerting
• Internal custom solutions

22. Why AppDynamics for PSP?
Overview Automate or die! Single pane of glass Collaboration
Global Synthetic monitoring Baseline alerting

23. AppDynamics adoption so far…
• AppDynamics rolled out to PROD & STAGE environments
• Automated On-premise setup
• Most important business transaction monitored
• External synthetic availability monitoring set up
• Alert automation with PagerDuty integration
• PROD deployment done within 10 weeks (on-premise, EUM, Analytics, SIM, Synthetic
Monitoring)

24. What’s next?
• Finalize APM migration – within the next 6 months
• Review business transactions with product teams and extend/enhance them
• Setting up AppDynamics on development/integration environments to benefit from
analyse capabilities even earlier in the development process
• Utilize analytics module to provide business reports and real-time dashboards
• Provide availability reports to customers
• Automate, automate, automate … (runbooks)

25. Global Wallet
aka Core Consumer Platform

26. Global Wallet – The team
• seven engineers – cross functional - following
the DevOps practices to ensure the quality of
the product is kept at the highest.
• A team of hands-on and coding architects
working with the team on strategic and day to
day requirements/tasks.
• Product team of PO & APO supported by the
Agile coach help the team to do the right stuff
at the right time.
By PierreSelim - Own work, CC BY-SA 3.0, https://commons.wikimedia.org/w/index.php?curid=17336884

27. Product Vision
• The PayU Global Wallet is a platform to offer financial services to fast
developing and highly regulated markets
• Services include Payment, Escrow, Consumer Credit and SMB Credit
Products
• Think Global, Go Local – one technology, many many markets
• Flexibility and Speed is key – we want to pursue new opportunities where
a One Size Fits all Solution does not work
• Our product needs to adapt to fast changing regulations and fast changing
competitive landscape

28. Product Vision – How?
• Global Wallet Core Services include Order, Payment, Marketplace, Ledgers and
many more APIs
• Every commit that goes through the CI/CD pipeline is deployed continuously to all
Delivery Centers
• All CI/CD for all services are controlled from Central MGT that is served out of
Europe – internal CICD PaaS
• Proprietary Encryption Vault technology takes care of card tokenization – it needs
to be deployed into PCI-certified Clouds additionally with the Global Wallet Core
Services

29. Global Wallet - PROD

30. Logical Overview
• Hybrid Cloud Rackspace UK
• Core = Global features (80%)
• Specific = Country features (20%)
• All managed by central MGT
platform

31. A Vertical Slice of Global Wallet…
• JVM Languages for Backend
• Single Page Front End Applications with Ember
• Container & Clustering - Docker Swarm, with Consul for
Discovery
• CI with Gitlab & Jenkins, CD with Jenkins & Ansible
• Fabio – canary testing
• HA Proxy - Blue / Green deployment

32. Team contract
API contract
1
2
Team principles3
Infrastructure principles4
5 Development principles
6 Questions
Global Wallet – DevOps HowTo
Global Wallet
DevOps HowTo

33. Team contract
Roles & Responsibilities Team definition of
done
Defined support process Defined communication channel

34. Why?
• Multiple contributing PayU entities in different geo-locations
• Different languages
• Different team cultures
AppDynamics:
• Unified language - Technology
• Common SaaS dashboard – single source of truth for product health

35. Team contract
API contract
1
2
Team principles3
Infrastructure principles4
5 Development principles
6 Questions
Global Wallet – DevOps HowTo
Global Wallet
DevOps HowTo

36. API contract
Defined protocol – https (https/2) Defined response
Versioning SLA – 99,99% uptime

37. Why?
• Predictability
• Prevent proprietary interface implementations
• Prevent artificial dependencies within PayU
AppDynamics:
• Real time architectural overview with service dependencies
• Real time service/endpoint performance

38. Team contract
API contract
1
2
Team principles3
Infrastructure principles4
5 Development principles
6 Questions
Global Wallet – DevOps HowTo
Global Wallet
DevOps How To

39. Team principles
The team owns the product Frequent zero downtime releases Docker ecosystem
All tasks are done by the team (Dev&Ops) Never stop learning! Embrace failure!

40. Why?
• Full product ownership – Product = Feature + Technology
• Keep high frequency of changes
• Innovation requires flexibility
AppDynamics:
• Significantly improved MTTD & MTTR for DevOps
• Wide variety of supported protocols / technologies

41. Team contract
API contract
1
2
Team principles3
Infrastructure
principles
4
5 Development principles
6 Questions
Global Wallet – DevOps HowTo
Global Wallet
DevOps How To

42. Infrastructure Principles
end to end CICD pipeline Deploy infrastructure as code
SDN No SPoF

43. Why?
• Vendor agnostic flexibility
• Support high frequency of changes
• Phoenix infrastructure
• Blue / Green deployments
AppDynamics:
• Tracking of infrastructure & application changes
• Easy to deploy agents

44. Team contract
API contract
1
2
Team principles3
Infrastructure principles4
5 Development
principles
6 Questions
Global Wallet – DevOps HowTo
Global Wallet
DevOps How To

45. Development Principles
Continuous deployment Roll forward – never roll back Micro Services
Internal ART <= 100ms Feature toggles

46. Why?
• Ensure high code quality
• Reduce review times
• Enable easy collaboration
AppDynamics:
• Method level performance analysis
• Code performance trending

47. Team contract
API contract
1
2
Team principles3
Infrastructure principles4
5 Development principles
6 Questions
Global Wallet
DevOps How To

48. Please give us your feedback—Session T13024
• Complete the online survey you'll receive via
email later today or via text at:
Text this number: 878787
Text this word: APPSPHERE
• Every time you submit a session survey, your
name will be entered in a random drawing.
We're giving away Amazon Echos
to 5 lucky winners!
• Thank you for your input
APPDYNAMICS CONFIDENTIAL AND PROPRIETARY 48
Win!

49. Thank you