SlideShare ist ein Scribd-Unternehmen logo
1 von 39
Downloaden Sie, um offline zu lesen
Introduction to trader bots
with Python
Thomas Aglassinger
http://roskakori.at
@Taglassinger
https://github.com/roskakori/talks/tree/master/pygraz/traderbot
Overview
● Outline a simple trading bot
● Example usages for several Python modules
(most part of the standard library)
● (Probably) Python beginner friendly
Agenda
● Communication and configuration
● Trading
● Data analysis and debugging
● Testing
Limitations
● You won't get rich (but it's fun nevertheless)
● Code does not work anymore due API update
● Masterexchange is going to shut down soon
● Terms of service
Communication and Configuration
Overview
● Uses https://masterxchange.com/api.php
(defunct after 2015-11-15)
● Communicate using HTTPS and JSON
● Public queries available to anyone (e.g. current
bids)
● Private queries requiring a personal token
bound to you account (e.g. orders)
Query trades
● Query the last 500 trades for maidsafe coins:
https://masterxchange.com/api/v2/trades.php?currency=maid
● Result:
[{"tradeid":"31242","price":"0.00990000","amount":"0.151
10800","date":"1446399439","market":"msc_btc"},
{"tradeid":"31241","price":"0.00990000","amount":"0.099
89200","date":"1446319270","market":"msc_btc"},
{"tradeid":"31240","price":"0.00562223","amount":"0.037
79028","date":"1446309947","market":"msc_btc"}, ...]
Print the top 3 trades
import json
import requests
def _without_utf8_bom(text):
return text[3:] if text.startswith('xefxbbxbf') else text
query = requests.get(
'https://masterxchange.com/api/v2/trades.php',
headers={'User-Agent': 'demobot/0.1'},
params={'currency': 'maid'}
)
print('query.status_code =', query.status_code)
if query.status_code < 300:
query_text = _without_utf8_bom(query.text)
print('query_text = %r...' % query_text[:40])
trades = json.loads(query_text)
print('trades =', trades[:3])
Print the top 3 trades - result
query.status_code = 200
query_text = '[{"tradeid":"31246","price":"0.00002603"'...
trades = [{'market': 'maid_btc', 'date': '1446500342', 'amount':
'7000.00000000', 'price': '0.00002603', 'tradeid': '31246'}, {'market':
'maid_btc', 'date': '1446489311', 'amount': '22000.00000000',
'price': '0.00002655', 'tradeid': '31244'}, {'market': 'maid_btc', 'date':
'1446462486', 'amount': '1250.00000000', 'price': '0.00002655',
'tradeid': '31243'}]
Configuring the API key
● Private queries require an API key.
● Simple way to manage: configparser
● Store key in a config file
● Read it during startup
Example config file
[demobot]
api_key = ou1IurT4HQrFfN1ch...
Read the API key from the config
import configparser
config = configparser.ConfigParser()
config.read('demobot.cfg')
api_key = config.get('demobot', 'api_key')
Query your balances
https://masterxchange.com/api/v2/private/balance
s.php?APIkey=ou1IurT4H...
{"balances":{"total":
{"btc":"9.16311816","msc":"34.63724456","maid":"
43233.50000000"},"available":
{"btc":7.16311816,"msc":26.63724456,"maid":426
33.5}},"error_message":"","error_code":0}
Print your balances
query = requests.get(
'https://masterxchange.com/api/v2/private/balances.php' ,
headers={'User-Agent': 'demobot/0.1'},
params={'APIkey': api_key}
)
print('query.status_code =', query.status_code)
if query.status_code < 300:
query_text = _without_utf8_bom(query.text)
print('query_text = %r...' % query_text[:40])
balances_result = json.loads(query_text)
if balances_result['error_code'] == 0:
balances = balances_result['balances']
print('balances =', balances)
Print your balances - result
query.status_code = 200
query_text = '{"balances":{"total":
{"btc":0,"msc":0,"m'...
balances = {'total': {'xcpjenga': 0, 'colzoidy': 0,
'coltdu': 0, 'xcpopcu': 0, 'colgauto': 0, ...}}
Masterexchange error handling
1.Http status < 300?
2.Query result error_code = 0?
3.Process actual data in query result
Wrap error handling in Exceptions
class BotError(Exception):
pass
class HttpsConnection(object):
...
def query(self, function_name, payload=None, use_api_key=True):
function_url = 'https://masterxchange.com/api/v2/%s.php' % function_name
actual_payload = {} if payload is None else dict(payload)
if use_api_key:
actual_payload['APIkey'] = self._api_key
headers = {'User-Agent': 'demobot/0.1'}
r = requests.get(function_url, headers=headers, params=actual_payload)
if r.status_code >= 300:
raise BotError(
'cannot query %s with %s: HTTP error code=%d'
% (function_url, actual_payload, r.status_code))
result = json.loads(
r.text[3:] if r.text.startswith('xefxbbxbf') else r.text)
if use_api_key and (result['error_code'] != 0):
raise BotError(
'cannot query %s with %s: %s'
% (function_url, actual_payload, result['error_message']))
return result
Trading
Processing monetary values
● Use decimal instead of float
http://floating-point-gui.de/
● Python has a decimal module:
https://docs.python.org/3/library/decimal.html
● json and configparser only support float
→ convert after reading and before writing
● Bitcoin uses 8 digits after decimal separator
Use formatting for decimals
>>> from decimal import Decimal
>>> print(Decimal('0.00000001'))
1E-8
>>> print('%.8f' % Decimal('0.00000001'))
0.00000001
Modes of operation
● Advise: only suggest to sell or buy → user has to manually initiate
transactions
● Lower risk for “stupid” transactions
● Might miss opportunities due slow reaction time
● Helpful when trying out a hopefully improved trading algorithm
● Action: automatically sell and buy on market conditions deemed
favorable
● Can react quickly to changes
● Possibility for epic fail on buggy trading algorithms
● Recommendation: reduce risk (but also opportunities) by limiting
amount traded per transaction and hour, stop loss limits etc.
Basic bot loop
1.Update own balances
2.Update open orders on the market
3.Apply trading algorithm and decide next action
4.Possibly buy or sell
5.Wait some time
6.Repeat
Trading algorithms
On the long run, nothing really works
Some simple trading algorithms
● Spread between 2 different but interchangeable items;
e.g. Team Fortress 2's keys and earbuds:
http://icrontic.com/article/tf2-black-market-explained
● Delayed correlation between two items; e.g. stocks for
Coca Cola and Pepsi:
http://www.investopedia.com/university/guide-pairs-trading/pairs-trading-correlation.asp
● Wait for slips from sellers, buy “unusually” cheap items
and resell for “normal” price;
article about such a bot (violating terms of service):
http://diablo3story.blogspot.com.au/2014/07/a-diablo-3-story.html
Data analysis and Debugging
Tracking statistics
● Collect statistics in database
● To debug bot decisions
● To improve trading algorithm
● To monitor market conditions
Sqlite
● Robust and stable
● Efficient for single client use
● Easy to set up
● Included with Python:
https://docs.python.org/3/library/sqlite3.html
● Rather creative type system
● “Real” instead of “decimal”
● “int” for timestamp instead of “datetime” type
● Type anarchy concerning comparison
● http://www.sqlite.org/datatype3.html
Create a statistics database
def _create_database(self, database_path):
_log.info('connect to database %r', database_path)
result = sqlite3.connect(database_path)
with closing(result.cursor()) as cursor:
cursor.execute("""
create table if not exists balances (
action char(4) not null,
balance_time int not null,
btc real not null,
maid real not null,
price_per_maid real not null,
transferred int not null
)
""")
cursor.execute("""
create index if not exists
idx_balance_time on balances (balance_time)
""")
result.commit()
return result
Insert a statistics row
values_to_insert = (
action,
int(time.time()),
float(self.btc_balance),
float(self.maid_balance),
float(price_per_maid),
int(maid_transferred),
)
with closing(self._database.cursor()) as cursor:
cursor.execute("""
insert into balances (
action, balance_time, btc, maid, price_per_maid, transferred
)
values (?, ?, ?, ?, ?, ?)
""", values_to_insert)
self._database.commit()
Logging
● More detailed tracking of trading decisions than
database
● But no easy structured analysis
● Use logging Module
https://docs.python.org/3/library/logging.html
● Use RotatingFileHandler
https://docs.python.org/3/library/logging.handler
s.html#rotatingfilehandler
Example logging configuration 1/2
# Logging configuration as described in
# <https://docs.python.org/3/howto/logging-cookbook.html>.
[loggers]
keys=root,demobot
[handlers]
keys=console,file
[formatters]
keys=default
[logger_root]
level=DEBUG
handlers=console,file
[logger_demobot]
level=DEBUG
handlers=console,file
qualname=demobot
propagate=0
Example logging configuration 2/2
[handler_console]
class=StreamHandler
level=INFO
formatter=default
args=(sys.stderr,)
[handler_file]
class=RotatingFileHandler
level=DEBUG
formatter=default
args=('/tmp/demobot.log', mode='a', maxBytes=1000000, backupCount=5, encoding='utf-8')
[formatter_default]
format=%(asctime)s - %(name)s - %(levelname)s - %(message)s
datefmt=
Testing
Testing challenges
● Network communication is slow
● Many sites have limits on transactions per
second
● Testing actual orders requires money
Mock-up connections with scenarios
● Scenarios are simple text file containing
expected queries and JSON results
● The test case makes a decisions that results in
a query, parses the scenarios JSON result, and
makes the next decision and query
● Scenarios can be maintained by domain
experts
Example scenario file
# got maid, no open orders
# the bot should create a maid-order for 500maid and 0.10000001btc/maid
private/balances
{
"balances": {
"total":{"btc":0.9,"maid":500},
"available" {"btc":0,"maid":0}
},
"error_message":"", "error_code":0
}
private/openedOrders
{
"open_orders": [],
"error_message":"", "error_code":0
}
orderbook
[
{"market":"maid_btc","type":"sell","amount":"120.00000000","price":"0.20000000","date_created":"1401077847"},
{"market":"maid_btc","type":"buy","amount":"270.00000000","price":"0.10000000","date_created":"1418566454"}
]
private/createOrder
Scenario implementation
● Bot constructor gets a connection
● Class HttpConnection → query() returns JSON
from Masterexchange
● Class ScenarioConnection → query() checks
that function matches next line in scenario file
and if so returns next JSON from it
Audience feedback: try Gherkin!
https://pypi.python.org/pypi/gherkin3
Summary
Summary
● Python has many useful libraries to quickly
implement simple trading bots.
● You probably won't get rich.
● It's fun!

Weitere ähnliche Inhalte

Ähnlich wie Introduction to trader bots with Python

Docker Logging and analysing with Elastic Stack
Docker Logging and analysing with Elastic StackDocker Logging and analysing with Elastic Stack
Docker Logging and analysing with Elastic StackJakub Hajek
 
[245] presto 내부구조 파헤치기
[245] presto 내부구조 파헤치기[245] presto 내부구조 파헤치기
[245] presto 내부구조 파헤치기NAVER D2
 
Presto anatomy
Presto anatomyPresto anatomy
Presto anatomyDongmin Yu
 
Fun Teaching MongoDB New Tricks
Fun Teaching MongoDB New TricksFun Teaching MongoDB New Tricks
Fun Teaching MongoDB New TricksMongoDB
 
Altitude NY 2018: Leveraging Log Streaming to Build the Best Dashboards, Ever
Altitude NY 2018: Leveraging Log Streaming to Build the Best Dashboards, EverAltitude NY 2018: Leveraging Log Streaming to Build the Best Dashboards, Ever
Altitude NY 2018: Leveraging Log Streaming to Build the Best Dashboards, EverFastly
 
using Mithril.js + postgREST to build and consume API's
using Mithril.js + postgREST to build and consume API'susing Mithril.js + postgREST to build and consume API's
using Mithril.js + postgREST to build and consume API'sAntônio Roberto Silva
 
Web Scraping for Non Programmers
Web Scraping for Non ProgrammersWeb Scraping for Non Programmers
Web Scraping for Non Programmersitnig
 
10 ways to make your code rock
10 ways to make your code rock10 ways to make your code rock
10 ways to make your code rockmartincronje
 
Spring Transaction
Spring TransactionSpring Transaction
Spring Transactionpatinijava
 
What we learnt at carousell tw for golang gathering #31
What we learnt at carousell tw for golang gathering #31What we learnt at carousell tw for golang gathering #31
What we learnt at carousell tw for golang gathering #31Ronald Hsu
 
PVS-Studio vs Chromium. 3-rd Check
PVS-Studio vs Chromium. 3-rd CheckPVS-Studio vs Chromium. 3-rd Check
PVS-Studio vs Chromium. 3-rd CheckAndrey Karpov
 
Introduction to Structured Streaming
Introduction to Structured StreamingIntroduction to Structured Streaming
Introduction to Structured StreamingKnoldus Inc.
 
Headache from using mathematical software
Headache from using mathematical softwareHeadache from using mathematical software
Headache from using mathematical softwarePVS-Studio
 
PVS-Studio and Continuous Integration: TeamCity. Analysis of the Open RollerC...
PVS-Studio and Continuous Integration: TeamCity. Analysis of the Open RollerC...PVS-Studio and Continuous Integration: TeamCity. Analysis of the Open RollerC...
PVS-Studio and Continuous Integration: TeamCity. Analysis of the Open RollerC...Andrey Karpov
 
Finagle and Java Service Framework at Pinterest
Finagle and Java Service Framework at PinterestFinagle and Java Service Framework at Pinterest
Finagle and Java Service Framework at PinterestPavan Chitumalla
 
Altitude San Francisco 2018: Logging at the Edge
Altitude San Francisco 2018: Logging at the Edge Altitude San Francisco 2018: Logging at the Edge
Altitude San Francisco 2018: Logging at the Edge Fastly
 
Introduction to Django
Introduction to DjangoIntroduction to Django
Introduction to DjangoJames Casey
 
MongoDB World 2018: Tutorial - Free the DBA: Building Chat Bots to Triage, Mo...
MongoDB World 2018: Tutorial - Free the DBA: Building Chat Bots to Triage, Mo...MongoDB World 2018: Tutorial - Free the DBA: Building Chat Bots to Triage, Mo...
MongoDB World 2018: Tutorial - Free the DBA: Building Chat Bots to Triage, Mo...MongoDB
 

Ähnlich wie Introduction to trader bots with Python (20)

Docker Logging and analysing with Elastic Stack
Docker Logging and analysing with Elastic StackDocker Logging and analysing with Elastic Stack
Docker Logging and analysing with Elastic Stack
 
[245] presto 내부구조 파헤치기
[245] presto 내부구조 파헤치기[245] presto 내부구조 파헤치기
[245] presto 내부구조 파헤치기
 
Presto anatomy
Presto anatomyPresto anatomy
Presto anatomy
 
Crafting APIs
Crafting APIsCrafting APIs
Crafting APIs
 
Fun Teaching MongoDB New Tricks
Fun Teaching MongoDB New TricksFun Teaching MongoDB New Tricks
Fun Teaching MongoDB New Tricks
 
Altitude NY 2018: Leveraging Log Streaming to Build the Best Dashboards, Ever
Altitude NY 2018: Leveraging Log Streaming to Build the Best Dashboards, EverAltitude NY 2018: Leveraging Log Streaming to Build the Best Dashboards, Ever
Altitude NY 2018: Leveraging Log Streaming to Build the Best Dashboards, Ever
 
using Mithril.js + postgREST to build and consume API's
using Mithril.js + postgREST to build and consume API'susing Mithril.js + postgREST to build and consume API's
using Mithril.js + postgREST to build and consume API's
 
Web Scraping for Non Programmers
Web Scraping for Non ProgrammersWeb Scraping for Non Programmers
Web Scraping for Non Programmers
 
Cubes 1.0 Overview
Cubes 1.0 OverviewCubes 1.0 Overview
Cubes 1.0 Overview
 
10 ways to make your code rock
10 ways to make your code rock10 ways to make your code rock
10 ways to make your code rock
 
Spring Transaction
Spring TransactionSpring Transaction
Spring Transaction
 
What we learnt at carousell tw for golang gathering #31
What we learnt at carousell tw for golang gathering #31What we learnt at carousell tw for golang gathering #31
What we learnt at carousell tw for golang gathering #31
 
PVS-Studio vs Chromium. 3-rd Check
PVS-Studio vs Chromium. 3-rd CheckPVS-Studio vs Chromium. 3-rd Check
PVS-Studio vs Chromium. 3-rd Check
 
Introduction to Structured Streaming
Introduction to Structured StreamingIntroduction to Structured Streaming
Introduction to Structured Streaming
 
Headache from using mathematical software
Headache from using mathematical softwareHeadache from using mathematical software
Headache from using mathematical software
 
PVS-Studio and Continuous Integration: TeamCity. Analysis of the Open RollerC...
PVS-Studio and Continuous Integration: TeamCity. Analysis of the Open RollerC...PVS-Studio and Continuous Integration: TeamCity. Analysis of the Open RollerC...
PVS-Studio and Continuous Integration: TeamCity. Analysis of the Open RollerC...
 
Finagle and Java Service Framework at Pinterest
Finagle and Java Service Framework at PinterestFinagle and Java Service Framework at Pinterest
Finagle and Java Service Framework at Pinterest
 
Altitude San Francisco 2018: Logging at the Edge
Altitude San Francisco 2018: Logging at the Edge Altitude San Francisco 2018: Logging at the Edge
Altitude San Francisco 2018: Logging at the Edge
 
Introduction to Django
Introduction to DjangoIntroduction to Django
Introduction to Django
 
MongoDB World 2018: Tutorial - Free the DBA: Building Chat Bots to Triage, Mo...
MongoDB World 2018: Tutorial - Free the DBA: Building Chat Bots to Triage, Mo...MongoDB World 2018: Tutorial - Free the DBA: Building Chat Bots to Triage, Mo...
MongoDB World 2018: Tutorial - Free the DBA: Building Chat Bots to Triage, Mo...
 

Mehr von roskakori

Expanding skill sets - Broaden your perspective on design
Expanding skill sets - Broaden your perspective on designExpanding skill sets - Broaden your perspective on design
Expanding skill sets - Broaden your perspective on designroskakori
 
Django trifft Flutter
Django trifft FlutterDjango trifft Flutter
Django trifft Flutterroskakori
 
Multiple django applications on a single server with nginx
Multiple django applications on a single server with nginxMultiple django applications on a single server with nginx
Multiple django applications on a single server with nginxroskakori
 
Helpful pre commit hooks for Python and Django
Helpful pre commit hooks for Python and DjangoHelpful pre commit hooks for Python and Django
Helpful pre commit hooks for Python and Djangoroskakori
 
Startmeeting Interessengruppe NLP NLU Graz
Startmeeting Interessengruppe NLP NLU GrazStartmeeting Interessengruppe NLP NLU Graz
Startmeeting Interessengruppe NLP NLU Grazroskakori
 
Helpful logging with python
Helpful logging with pythonHelpful logging with python
Helpful logging with pythonroskakori
 
Helpful logging with Java
Helpful logging with JavaHelpful logging with Java
Helpful logging with Javaroskakori
 
Einführung in Kommunikation und Konfliktmanagement für Software-Entwickler
Einführung in Kommunikation und Konfliktmanagement für Software-EntwicklerEinführung in Kommunikation und Konfliktmanagement für Software-Entwickler
Einführung in Kommunikation und Konfliktmanagement für Software-Entwicklerroskakori
 
Analyzing natural language feedback using python
Analyzing natural language feedback using pythonAnalyzing natural language feedback using python
Analyzing natural language feedback using pythonroskakori
 
Microsoft SQL Server with Linux and Docker
Microsoft SQL Server with Linux and DockerMicrosoft SQL Server with Linux and Docker
Microsoft SQL Server with Linux and Dockerroskakori
 
Migration to Python 3 in Finance
Migration to Python 3 in FinanceMigration to Python 3 in Finance
Migration to Python 3 in Financeroskakori
 
Introduction to pygments
Introduction to pygmentsIntroduction to pygments
Introduction to pygmentsroskakori
 
Lösungsorientierte Fehlerbehandlung
Lösungsorientierte FehlerbehandlungLösungsorientierte Fehlerbehandlung
Lösungsorientierte Fehlerbehandlungroskakori
 
XML namespaces and XPath with Python
XML namespaces and XPath with PythonXML namespaces and XPath with Python
XML namespaces and XPath with Pythonroskakori
 
Erste-Hilfekasten für Unicode mit Python
Erste-Hilfekasten für Unicode mit PythonErste-Hilfekasten für Unicode mit Python
Erste-Hilfekasten für Unicode mit Pythonroskakori
 
Open source projects with python
Open source projects with pythonOpen source projects with python
Open source projects with pythonroskakori
 
Python builds mit ant
Python builds mit antPython builds mit ant
Python builds mit antroskakori
 
Kanban zur Abwicklung von Reporting-Anforderungen
Kanban zur Abwicklung von Reporting-AnforderungenKanban zur Abwicklung von Reporting-Anforderungen
Kanban zur Abwicklung von Reporting-Anforderungenroskakori
 

Mehr von roskakori (18)

Expanding skill sets - Broaden your perspective on design
Expanding skill sets - Broaden your perspective on designExpanding skill sets - Broaden your perspective on design
Expanding skill sets - Broaden your perspective on design
 
Django trifft Flutter
Django trifft FlutterDjango trifft Flutter
Django trifft Flutter
 
Multiple django applications on a single server with nginx
Multiple django applications on a single server with nginxMultiple django applications on a single server with nginx
Multiple django applications on a single server with nginx
 
Helpful pre commit hooks for Python and Django
Helpful pre commit hooks for Python and DjangoHelpful pre commit hooks for Python and Django
Helpful pre commit hooks for Python and Django
 
Startmeeting Interessengruppe NLP NLU Graz
Startmeeting Interessengruppe NLP NLU GrazStartmeeting Interessengruppe NLP NLU Graz
Startmeeting Interessengruppe NLP NLU Graz
 
Helpful logging with python
Helpful logging with pythonHelpful logging with python
Helpful logging with python
 
Helpful logging with Java
Helpful logging with JavaHelpful logging with Java
Helpful logging with Java
 
Einführung in Kommunikation und Konfliktmanagement für Software-Entwickler
Einführung in Kommunikation und Konfliktmanagement für Software-EntwicklerEinführung in Kommunikation und Konfliktmanagement für Software-Entwickler
Einführung in Kommunikation und Konfliktmanagement für Software-Entwickler
 
Analyzing natural language feedback using python
Analyzing natural language feedback using pythonAnalyzing natural language feedback using python
Analyzing natural language feedback using python
 
Microsoft SQL Server with Linux and Docker
Microsoft SQL Server with Linux and DockerMicrosoft SQL Server with Linux and Docker
Microsoft SQL Server with Linux and Docker
 
Migration to Python 3 in Finance
Migration to Python 3 in FinanceMigration to Python 3 in Finance
Migration to Python 3 in Finance
 
Introduction to pygments
Introduction to pygmentsIntroduction to pygments
Introduction to pygments
 
Lösungsorientierte Fehlerbehandlung
Lösungsorientierte FehlerbehandlungLösungsorientierte Fehlerbehandlung
Lösungsorientierte Fehlerbehandlung
 
XML namespaces and XPath with Python
XML namespaces and XPath with PythonXML namespaces and XPath with Python
XML namespaces and XPath with Python
 
Erste-Hilfekasten für Unicode mit Python
Erste-Hilfekasten für Unicode mit PythonErste-Hilfekasten für Unicode mit Python
Erste-Hilfekasten für Unicode mit Python
 
Open source projects with python
Open source projects with pythonOpen source projects with python
Open source projects with python
 
Python builds mit ant
Python builds mit antPython builds mit ant
Python builds mit ant
 
Kanban zur Abwicklung von Reporting-Anforderungen
Kanban zur Abwicklung von Reporting-AnforderungenKanban zur Abwicklung von Reporting-Anforderungen
Kanban zur Abwicklung von Reporting-Anforderungen
 

Kürzlich hochgeladen

A Year of the Servo Reboot: Where Are We Now?
A Year of the Servo Reboot: Where Are We Now?A Year of the Servo Reboot: Where Are We Now?
A Year of the Servo Reboot: Where Are We Now?Igalia
 
Boost Fertility New Invention Ups Success Rates.pdf
Boost Fertility New Invention Ups Success Rates.pdfBoost Fertility New Invention Ups Success Rates.pdf
Boost Fertility New Invention Ups Success Rates.pdfsudhanshuwaghmare1
 
Strategies for Unlocking Knowledge Management in Microsoft 365 in the Copilot...
Strategies for Unlocking Knowledge Management in Microsoft 365 in the Copilot...Strategies for Unlocking Knowledge Management in Microsoft 365 in the Copilot...
Strategies for Unlocking Knowledge Management in Microsoft 365 in the Copilot...Drew Madelung
 
Data Cloud, More than a CDP by Matt Robison
Data Cloud, More than a CDP by Matt RobisonData Cloud, More than a CDP by Matt Robison
Data Cloud, More than a CDP by Matt RobisonAnna Loughnan Colquhoun
 
Handwritten Text Recognition for manuscripts and early printed texts
Handwritten Text Recognition for manuscripts and early printed textsHandwritten Text Recognition for manuscripts and early printed texts
Handwritten Text Recognition for manuscripts and early printed textsMaria Levchenko
 
A Call to Action for Generative AI in 2024
A Call to Action for Generative AI in 2024A Call to Action for Generative AI in 2024
A Call to Action for Generative AI in 2024Results
 
The Codex of Business Writing Software for Real-World Solutions 2.pptx
The Codex of Business Writing Software for Real-World Solutions 2.pptxThe Codex of Business Writing Software for Real-World Solutions 2.pptx
The Codex of Business Writing Software for Real-World Solutions 2.pptxMalak Abu Hammad
 
Apidays Singapore 2024 - Building Digital Trust in a Digital Economy by Veron...
Apidays Singapore 2024 - Building Digital Trust in a Digital Economy by Veron...Apidays Singapore 2024 - Building Digital Trust in a Digital Economy by Veron...
Apidays Singapore 2024 - Building Digital Trust in a Digital Economy by Veron...apidays
 
Presentation on how to chat with PDF using ChatGPT code interpreter
Presentation on how to chat with PDF using ChatGPT code interpreterPresentation on how to chat with PDF using ChatGPT code interpreter
Presentation on how to chat with PDF using ChatGPT code interpreternaman860154
 
The 7 Things I Know About Cyber Security After 25 Years | April 2024
The 7 Things I Know About Cyber Security After 25 Years | April 2024The 7 Things I Know About Cyber Security After 25 Years | April 2024
The 7 Things I Know About Cyber Security After 25 Years | April 2024Rafal Los
 
Automating Google Workspace (GWS) & more with Apps Script
Automating Google Workspace (GWS) & more with Apps ScriptAutomating Google Workspace (GWS) & more with Apps Script
Automating Google Workspace (GWS) & more with Apps Scriptwesley chun
 
Artificial Intelligence: Facts and Myths
Artificial Intelligence: Facts and MythsArtificial Intelligence: Facts and Myths
Artificial Intelligence: Facts and MythsJoaquim Jorge
 
From Event to Action: Accelerate Your Decision Making with Real-Time Automation
From Event to Action: Accelerate Your Decision Making with Real-Time AutomationFrom Event to Action: Accelerate Your Decision Making with Real-Time Automation
From Event to Action: Accelerate Your Decision Making with Real-Time AutomationSafe Software
 
Raspberry Pi 5: Challenges and Solutions in Bringing up an OpenGL/Vulkan Driv...
Raspberry Pi 5: Challenges and Solutions in Bringing up an OpenGL/Vulkan Driv...Raspberry Pi 5: Challenges and Solutions in Bringing up an OpenGL/Vulkan Driv...
Raspberry Pi 5: Challenges and Solutions in Bringing up an OpenGL/Vulkan Driv...Igalia
 
08448380779 Call Girls In Greater Kailash - I Women Seeking Men
08448380779 Call Girls In Greater Kailash - I Women Seeking Men08448380779 Call Girls In Greater Kailash - I Women Seeking Men
08448380779 Call Girls In Greater Kailash - I Women Seeking MenDelhi Call girls
 
Slack Application Development 101 Slides
Slack Application Development 101 SlidesSlack Application Development 101 Slides
Slack Application Development 101 Slidespraypatel2
 
Understanding Discord NSFW Servers A Guide for Responsible Users.pdf
Understanding Discord NSFW Servers A Guide for Responsible Users.pdfUnderstanding Discord NSFW Servers A Guide for Responsible Users.pdf
Understanding Discord NSFW Servers A Guide for Responsible Users.pdfUK Journal
 
The Role of Taxonomy and Ontology in Semantic Layers - Heather Hedden.pdf
The Role of Taxonomy and Ontology in Semantic Layers - Heather Hedden.pdfThe Role of Taxonomy and Ontology in Semantic Layers - Heather Hedden.pdf
The Role of Taxonomy and Ontology in Semantic Layers - Heather Hedden.pdfEnterprise Knowledge
 
TrustArc Webinar - Stay Ahead of US State Data Privacy Law Developments
TrustArc Webinar - Stay Ahead of US State Data Privacy Law DevelopmentsTrustArc Webinar - Stay Ahead of US State Data Privacy Law Developments
TrustArc Webinar - Stay Ahead of US State Data Privacy Law DevelopmentsTrustArc
 
Mastering MySQL Database Architecture: Deep Dive into MySQL Shell and MySQL R...
Mastering MySQL Database Architecture: Deep Dive into MySQL Shell and MySQL R...Mastering MySQL Database Architecture: Deep Dive into MySQL Shell and MySQL R...
Mastering MySQL Database Architecture: Deep Dive into MySQL Shell and MySQL R...Miguel Araújo
 

Kürzlich hochgeladen (20)

A Year of the Servo Reboot: Where Are We Now?
A Year of the Servo Reboot: Where Are We Now?A Year of the Servo Reboot: Where Are We Now?
A Year of the Servo Reboot: Where Are We Now?
 
Boost Fertility New Invention Ups Success Rates.pdf
Boost Fertility New Invention Ups Success Rates.pdfBoost Fertility New Invention Ups Success Rates.pdf
Boost Fertility New Invention Ups Success Rates.pdf
 
Strategies for Unlocking Knowledge Management in Microsoft 365 in the Copilot...
Strategies for Unlocking Knowledge Management in Microsoft 365 in the Copilot...Strategies for Unlocking Knowledge Management in Microsoft 365 in the Copilot...
Strategies for Unlocking Knowledge Management in Microsoft 365 in the Copilot...
 
Data Cloud, More than a CDP by Matt Robison
Data Cloud, More than a CDP by Matt RobisonData Cloud, More than a CDP by Matt Robison
Data Cloud, More than a CDP by Matt Robison
 
Handwritten Text Recognition for manuscripts and early printed texts
Handwritten Text Recognition for manuscripts and early printed textsHandwritten Text Recognition for manuscripts and early printed texts
Handwritten Text Recognition for manuscripts and early printed texts
 
A Call to Action for Generative AI in 2024
A Call to Action for Generative AI in 2024A Call to Action for Generative AI in 2024
A Call to Action for Generative AI in 2024
 
The Codex of Business Writing Software for Real-World Solutions 2.pptx
The Codex of Business Writing Software for Real-World Solutions 2.pptxThe Codex of Business Writing Software for Real-World Solutions 2.pptx
The Codex of Business Writing Software for Real-World Solutions 2.pptx
 
Apidays Singapore 2024 - Building Digital Trust in a Digital Economy by Veron...
Apidays Singapore 2024 - Building Digital Trust in a Digital Economy by Veron...Apidays Singapore 2024 - Building Digital Trust in a Digital Economy by Veron...
Apidays Singapore 2024 - Building Digital Trust in a Digital Economy by Veron...
 
Presentation on how to chat with PDF using ChatGPT code interpreter
Presentation on how to chat with PDF using ChatGPT code interpreterPresentation on how to chat with PDF using ChatGPT code interpreter
Presentation on how to chat with PDF using ChatGPT code interpreter
 
The 7 Things I Know About Cyber Security After 25 Years | April 2024
The 7 Things I Know About Cyber Security After 25 Years | April 2024The 7 Things I Know About Cyber Security After 25 Years | April 2024
The 7 Things I Know About Cyber Security After 25 Years | April 2024
 
Automating Google Workspace (GWS) & more with Apps Script
Automating Google Workspace (GWS) & more with Apps ScriptAutomating Google Workspace (GWS) & more with Apps Script
Automating Google Workspace (GWS) & more with Apps Script
 
Artificial Intelligence: Facts and Myths
Artificial Intelligence: Facts and MythsArtificial Intelligence: Facts and Myths
Artificial Intelligence: Facts and Myths
 
From Event to Action: Accelerate Your Decision Making with Real-Time Automation
From Event to Action: Accelerate Your Decision Making with Real-Time AutomationFrom Event to Action: Accelerate Your Decision Making with Real-Time Automation
From Event to Action: Accelerate Your Decision Making with Real-Time Automation
 
Raspberry Pi 5: Challenges and Solutions in Bringing up an OpenGL/Vulkan Driv...
Raspberry Pi 5: Challenges and Solutions in Bringing up an OpenGL/Vulkan Driv...Raspberry Pi 5: Challenges and Solutions in Bringing up an OpenGL/Vulkan Driv...
Raspberry Pi 5: Challenges and Solutions in Bringing up an OpenGL/Vulkan Driv...
 
08448380779 Call Girls In Greater Kailash - I Women Seeking Men
08448380779 Call Girls In Greater Kailash - I Women Seeking Men08448380779 Call Girls In Greater Kailash - I Women Seeking Men
08448380779 Call Girls In Greater Kailash - I Women Seeking Men
 
Slack Application Development 101 Slides
Slack Application Development 101 SlidesSlack Application Development 101 Slides
Slack Application Development 101 Slides
 
Understanding Discord NSFW Servers A Guide for Responsible Users.pdf
Understanding Discord NSFW Servers A Guide for Responsible Users.pdfUnderstanding Discord NSFW Servers A Guide for Responsible Users.pdf
Understanding Discord NSFW Servers A Guide for Responsible Users.pdf
 
The Role of Taxonomy and Ontology in Semantic Layers - Heather Hedden.pdf
The Role of Taxonomy and Ontology in Semantic Layers - Heather Hedden.pdfThe Role of Taxonomy and Ontology in Semantic Layers - Heather Hedden.pdf
The Role of Taxonomy and Ontology in Semantic Layers - Heather Hedden.pdf
 
TrustArc Webinar - Stay Ahead of US State Data Privacy Law Developments
TrustArc Webinar - Stay Ahead of US State Data Privacy Law DevelopmentsTrustArc Webinar - Stay Ahead of US State Data Privacy Law Developments
TrustArc Webinar - Stay Ahead of US State Data Privacy Law Developments
 
Mastering MySQL Database Architecture: Deep Dive into MySQL Shell and MySQL R...
Mastering MySQL Database Architecture: Deep Dive into MySQL Shell and MySQL R...Mastering MySQL Database Architecture: Deep Dive into MySQL Shell and MySQL R...
Mastering MySQL Database Architecture: Deep Dive into MySQL Shell and MySQL R...
 

Introduction to trader bots with Python

  • 1. Introduction to trader bots with Python Thomas Aglassinger http://roskakori.at @Taglassinger https://github.com/roskakori/talks/tree/master/pygraz/traderbot
  • 2. Overview ● Outline a simple trading bot ● Example usages for several Python modules (most part of the standard library) ● (Probably) Python beginner friendly
  • 3. Agenda ● Communication and configuration ● Trading ● Data analysis and debugging ● Testing
  • 4. Limitations ● You won't get rich (but it's fun nevertheless) ● Code does not work anymore due API update ● Masterexchange is going to shut down soon ● Terms of service
  • 6. Overview ● Uses https://masterxchange.com/api.php (defunct after 2015-11-15) ● Communicate using HTTPS and JSON ● Public queries available to anyone (e.g. current bids) ● Private queries requiring a personal token bound to you account (e.g. orders)
  • 7. Query trades ● Query the last 500 trades for maidsafe coins: https://masterxchange.com/api/v2/trades.php?currency=maid ● Result: [{"tradeid":"31242","price":"0.00990000","amount":"0.151 10800","date":"1446399439","market":"msc_btc"}, {"tradeid":"31241","price":"0.00990000","amount":"0.099 89200","date":"1446319270","market":"msc_btc"}, {"tradeid":"31240","price":"0.00562223","amount":"0.037 79028","date":"1446309947","market":"msc_btc"}, ...]
  • 8. Print the top 3 trades import json import requests def _without_utf8_bom(text): return text[3:] if text.startswith('xefxbbxbf') else text query = requests.get( 'https://masterxchange.com/api/v2/trades.php', headers={'User-Agent': 'demobot/0.1'}, params={'currency': 'maid'} ) print('query.status_code =', query.status_code) if query.status_code < 300: query_text = _without_utf8_bom(query.text) print('query_text = %r...' % query_text[:40]) trades = json.loads(query_text) print('trades =', trades[:3])
  • 9. Print the top 3 trades - result query.status_code = 200 query_text = '[{"tradeid":"31246","price":"0.00002603"'... trades = [{'market': 'maid_btc', 'date': '1446500342', 'amount': '7000.00000000', 'price': '0.00002603', 'tradeid': '31246'}, {'market': 'maid_btc', 'date': '1446489311', 'amount': '22000.00000000', 'price': '0.00002655', 'tradeid': '31244'}, {'market': 'maid_btc', 'date': '1446462486', 'amount': '1250.00000000', 'price': '0.00002655', 'tradeid': '31243'}]
  • 10. Configuring the API key ● Private queries require an API key. ● Simple way to manage: configparser ● Store key in a config file ● Read it during startup
  • 11. Example config file [demobot] api_key = ou1IurT4HQrFfN1ch...
  • 12. Read the API key from the config import configparser config = configparser.ConfigParser() config.read('demobot.cfg') api_key = config.get('demobot', 'api_key')
  • 14. Print your balances query = requests.get( 'https://masterxchange.com/api/v2/private/balances.php' , headers={'User-Agent': 'demobot/0.1'}, params={'APIkey': api_key} ) print('query.status_code =', query.status_code) if query.status_code < 300: query_text = _without_utf8_bom(query.text) print('query_text = %r...' % query_text[:40]) balances_result = json.loads(query_text) if balances_result['error_code'] == 0: balances = balances_result['balances'] print('balances =', balances)
  • 15. Print your balances - result query.status_code = 200 query_text = '{"balances":{"total": {"btc":0,"msc":0,"m'... balances = {'total': {'xcpjenga': 0, 'colzoidy': 0, 'coltdu': 0, 'xcpopcu': 0, 'colgauto': 0, ...}}
  • 16. Masterexchange error handling 1.Http status < 300? 2.Query result error_code = 0? 3.Process actual data in query result
  • 17. Wrap error handling in Exceptions class BotError(Exception): pass class HttpsConnection(object): ... def query(self, function_name, payload=None, use_api_key=True): function_url = 'https://masterxchange.com/api/v2/%s.php' % function_name actual_payload = {} if payload is None else dict(payload) if use_api_key: actual_payload['APIkey'] = self._api_key headers = {'User-Agent': 'demobot/0.1'} r = requests.get(function_url, headers=headers, params=actual_payload) if r.status_code >= 300: raise BotError( 'cannot query %s with %s: HTTP error code=%d' % (function_url, actual_payload, r.status_code)) result = json.loads( r.text[3:] if r.text.startswith('xefxbbxbf') else r.text) if use_api_key and (result['error_code'] != 0): raise BotError( 'cannot query %s with %s: %s' % (function_url, actual_payload, result['error_message'])) return result
  • 19. Processing monetary values ● Use decimal instead of float http://floating-point-gui.de/ ● Python has a decimal module: https://docs.python.org/3/library/decimal.html ● json and configparser only support float → convert after reading and before writing ● Bitcoin uses 8 digits after decimal separator
  • 20. Use formatting for decimals >>> from decimal import Decimal >>> print(Decimal('0.00000001')) 1E-8 >>> print('%.8f' % Decimal('0.00000001')) 0.00000001
  • 21. Modes of operation ● Advise: only suggest to sell or buy → user has to manually initiate transactions ● Lower risk for “stupid” transactions ● Might miss opportunities due slow reaction time ● Helpful when trying out a hopefully improved trading algorithm ● Action: automatically sell and buy on market conditions deemed favorable ● Can react quickly to changes ● Possibility for epic fail on buggy trading algorithms ● Recommendation: reduce risk (but also opportunities) by limiting amount traded per transaction and hour, stop loss limits etc.
  • 22. Basic bot loop 1.Update own balances 2.Update open orders on the market 3.Apply trading algorithm and decide next action 4.Possibly buy or sell 5.Wait some time 6.Repeat
  • 23. Trading algorithms On the long run, nothing really works
  • 24. Some simple trading algorithms ● Spread between 2 different but interchangeable items; e.g. Team Fortress 2's keys and earbuds: http://icrontic.com/article/tf2-black-market-explained ● Delayed correlation between two items; e.g. stocks for Coca Cola and Pepsi: http://www.investopedia.com/university/guide-pairs-trading/pairs-trading-correlation.asp ● Wait for slips from sellers, buy “unusually” cheap items and resell for “normal” price; article about such a bot (violating terms of service): http://diablo3story.blogspot.com.au/2014/07/a-diablo-3-story.html
  • 25. Data analysis and Debugging
  • 26. Tracking statistics ● Collect statistics in database ● To debug bot decisions ● To improve trading algorithm ● To monitor market conditions
  • 27. Sqlite ● Robust and stable ● Efficient for single client use ● Easy to set up ● Included with Python: https://docs.python.org/3/library/sqlite3.html ● Rather creative type system ● “Real” instead of “decimal” ● “int” for timestamp instead of “datetime” type ● Type anarchy concerning comparison ● http://www.sqlite.org/datatype3.html
  • 28. Create a statistics database def _create_database(self, database_path): _log.info('connect to database %r', database_path) result = sqlite3.connect(database_path) with closing(result.cursor()) as cursor: cursor.execute(""" create table if not exists balances ( action char(4) not null, balance_time int not null, btc real not null, maid real not null, price_per_maid real not null, transferred int not null ) """) cursor.execute(""" create index if not exists idx_balance_time on balances (balance_time) """) result.commit() return result
  • 29. Insert a statistics row values_to_insert = ( action, int(time.time()), float(self.btc_balance), float(self.maid_balance), float(price_per_maid), int(maid_transferred), ) with closing(self._database.cursor()) as cursor: cursor.execute(""" insert into balances ( action, balance_time, btc, maid, price_per_maid, transferred ) values (?, ?, ?, ?, ?, ?) """, values_to_insert) self._database.commit()
  • 30. Logging ● More detailed tracking of trading decisions than database ● But no easy structured analysis ● Use logging Module https://docs.python.org/3/library/logging.html ● Use RotatingFileHandler https://docs.python.org/3/library/logging.handler s.html#rotatingfilehandler
  • 31. Example logging configuration 1/2 # Logging configuration as described in # <https://docs.python.org/3/howto/logging-cookbook.html>. [loggers] keys=root,demobot [handlers] keys=console,file [formatters] keys=default [logger_root] level=DEBUG handlers=console,file [logger_demobot] level=DEBUG handlers=console,file qualname=demobot propagate=0
  • 32. Example logging configuration 2/2 [handler_console] class=StreamHandler level=INFO formatter=default args=(sys.stderr,) [handler_file] class=RotatingFileHandler level=DEBUG formatter=default args=('/tmp/demobot.log', mode='a', maxBytes=1000000, backupCount=5, encoding='utf-8') [formatter_default] format=%(asctime)s - %(name)s - %(levelname)s - %(message)s datefmt=
  • 34. Testing challenges ● Network communication is slow ● Many sites have limits on transactions per second ● Testing actual orders requires money
  • 35. Mock-up connections with scenarios ● Scenarios are simple text file containing expected queries and JSON results ● The test case makes a decisions that results in a query, parses the scenarios JSON result, and makes the next decision and query ● Scenarios can be maintained by domain experts
  • 36. Example scenario file # got maid, no open orders # the bot should create a maid-order for 500maid and 0.10000001btc/maid private/balances { "balances": { "total":{"btc":0.9,"maid":500}, "available" {"btc":0,"maid":0} }, "error_message":"", "error_code":0 } private/openedOrders { "open_orders": [], "error_message":"", "error_code":0 } orderbook [ {"market":"maid_btc","type":"sell","amount":"120.00000000","price":"0.20000000","date_created":"1401077847"}, {"market":"maid_btc","type":"buy","amount":"270.00000000","price":"0.10000000","date_created":"1418566454"} ] private/createOrder
  • 37. Scenario implementation ● Bot constructor gets a connection ● Class HttpConnection → query() returns JSON from Masterexchange ● Class ScenarioConnection → query() checks that function matches next line in scenario file and if so returns next JSON from it Audience feedback: try Gherkin! https://pypi.python.org/pypi/gherkin3
  • 39. Summary ● Python has many useful libraries to quickly implement simple trading bots. ● You probably won't get rich. ● It's fun!