Reference Test Harness for Algorithmic Trading Platforms Viktoria Leonchik, Alexey Sukhov, Eugene Ushakov, Iosif Itkin, Anna-Maria Lukina, Exactpro Systems 12 - 14 November 2015 Tools and Methods of Program Analysis in St. Petersburg

1. Quality Assurance & Related Software Development for Innovative Trading Systems
Tel: +7 495 640 2460, +1 415 830 38 49
www.exactpro.com

2. 2
• Introduction
• Regulators’ requirements
• Test Harness for Algorithmic trading system
• Minirobots
• Algorithmic Test Agents
• Technical Criteria
• Business Criteria
• Conclusion
Agenda
Quality Assurance & Related Software Development for Innovative Trading Systems Tel: +7 495 640 24 60 , +1 415 830 38 49
www.exactpro.com

3. 3
Exactpro Systems, LLC Quality Assurance & Related Software Development for Innovative Trading Systems Tel: +7 495 640 24 60 , +1 415 830 38 49
exactpro.com
Introduction
Flash Crash in 2010
The Knight Capital
runaway algorithm
disaster in 2012

4. SEC ESMA
Conducting testing: confirmation that core code
components operate as intended and do not produce
unintended consequences
Clearly delineated development and testing methodologies
Quality Assurance process should be separated from any
development work
Testing methodologies should include performance simulations /
back-testing and non-live testing within a trading venue
testing environment
Periodically evaluating test controls Ensure that tests are commensurate with the risks that this
strategy may pose to itself and to the fair and orderly
functioning of the markets operated by the trading venue
Data integrity, accuracy and workflow validation Periodically evaluating test controls
Recording of all testing protocols and results
Conducting all testing in a development environment
that is segregated from production
Investment firms should ensure that the production and testing
environments are kept segregated at all times
4
Exactpro Systems, LLC Quality Assurance & Related Software Development for Innovative Trading Systems Tel: +7 495 640 24 60 , +1 415 830 38 49
exactpro.com
Regulators’ requirements

5. 5
Exactpro Systems, LLC Quality Assurance & Related Software Development for Innovative Trading Systems Tel: +7 495 640 24 60 , +1 415 830 38 49
exactpro.com
Test Harness for Algorithmic trading system

6. 6
Exactpro Systems, LLC Quality Assurance & Related Software Development for Innovative Trading Systems Tel: +7 495 640 24 60 , +1 415 830 38 49
exactpro.com
Test Harness for Algorithmic trading system

7. 7
Exactpro Systems, LLC Quality Assurance & Related Software Development for Innovative Trading Systems Tel: +7 495 640 24 60 , +1 415 830 38 49
exactpro.com
Test Harness for Algorithmic trading system

8. 8
Exactpro Systems, LLC Quality Assurance & Related Software Development for Innovative Trading Systems Tel: +7 495 640 24 60 , +1 415 830 38 49
exactpro.com
Test Harness for Algorithmic trading system

9. 9
Exactpro Systems, LLC Quality Assurance & Related Software Development for Innovative Trading Systems Tel: +7 495 640 24 60 , +1 415 830 38 49
exactpro.com
Test Harness for Algorithmic trading system

10. 10
Exactpro Systems, LLC Quality Assurance & Related Software Development for Innovative Trading Systems Tel: +7 495 640 24 60 , +1 415 830 38 49
exactpro.com
Test Harness for Algorithmic trading system

11. 11
Exactpro Systems, LLC Quality Assurance & Related Software Development for Innovative Trading Systems Tel: +7 495 640 24 60 , +1 415 830 38 49
exactpro.com
Test Harness for Algorithmic trading system

12. 12
Exactpro Systems, LLC Quality Assurance & Related Software Development for Innovative Trading Systems Tel: +7 495 640 24 60 , +1 415 830 38 49
exactpro.com
Test Harness for Algorithmic trading system

13. 13
Exactpro Systems, LLC Quality Assurance & Related Software Development for Innovative Trading Systems Tel: +7 495 640 24 60 , +1 415 830 38 49
exactpro.com
Minirobots
Item Description
Testing Type Active Multi-Participants (applicable for testing at the confluence of functional and
non-functional testing)
Target SUT Trading Platforms and Market Data Delivery Systems
SUT Interface Back-end (typically connected to message gateways / APIs, and DBs); GUI Testing
Capabilities supported via plug-ins to other tools (e.g., Selenium)
SUT Interaction
Method
Message injection and capture to emulate multiple participants’ activity in electronic markets
(essential when there is a need to reproduce complex scenarios that can be created by trading
algorithms)
Protocols Extant plug-ins for Industry-standard (FIX and dialects, etc.) and proprietary protocols. New
plug-ins for additional protocols developed by request
Test Scripts Multi-threaded Java code specifying different liquidity profiles
Test Management,
Execution and
Reporting
Integrated (Web front-end), allows for multiple simultaneous heterogeneous connections,
concurrent emulation of multiple participants, detailed test reports. Optional Big Button
framework supported
Platform requirements Low footprint cross-platform application, MySQL or other RDBMS

14. 1. Arbitraging and Market Making Minirobots
2. Minirobots emulating ‘Slicing’ algorithms
3. Minirobots emulating ‘Synthetic’ algorithms
4. Exchange simulated orders
5. Aggressive buyer/seller (Market panic scenarios)
6. ‘Bandit’-algorithms
• ‘Layering’
• ‘Quote-stuffing’
• ‘Painting the tape’
• ‘Abusive liquidity detection’
14
Exactpro Systems, LLC Quality Assurance & Related Software Development for Innovative Trading Systems Tel: +7 495 640 24 60 , +1 415 830 38 49
exactpro.com
Algorithmic Test Agents

15. 1. Functional criteria
The smaller number of existing errors an algorithm contains, the lower the
expectations of potential losses and fines are.
2. Non-Functional criteria
• Performance
• Scalability
• Reliability
• Efficiency
• Recoverability
15
Exactpro Systems, LLC Quality Assurance & Related Software Development for Innovative Trading Systems Tel: +7 495 640 24 60 , +1 415 830 38 49
exactpro.com
Technical Criteria
Shsha

16. 1. Price
• execution
• price improvement
• spread capture
2. Cost
• explicit
• market impact
• adverse selection
3. Probability of execution
4. Implementation Shortfall (IS).
16
Exactpro Systems, LLC Quality Assurance & Related Software Development for Innovative Trading Systems Tel: +7 495 640 24 60 , +1 415 830 38 49
exactpro.com
Business Criteria

17. Thank you!
17
Exactpro Systems, LLC Quality Assurance & Related Software Development for Innovative Trading Systems Tel: +7 495 640 24 60 , +1 415 830 38 49
exactpro.com
Q&A