This talk covers the journey of running Python code as a blockchain (Ethereum) Smart Contract by implementing a modified Python runtime environment / Ethereum Virtual Machine software, and building a blockchain based home device network with the modified runtime and Smart Contract code written in Python.
본 발표는 파이썬 코드를 Ethereum 블록체인에서 구동될 수 있는 Smart Contract로 구동할 수 있도록 하는 삽질의 여정을 다룹니다. 보다 상세하게는, 상기의 목적에 맞도록 수정된 파이썬 런타임과 Ethereum Virtual Machine (EVM) 소프트웨어를 구현하고, 이를 사용하여 블록체인 기반의 가정 IoT 네트워크 위에서 파이썬 언어로 작성된 Smart Contract 코드를 작성하는 것에 관하여 다룹니다.
5. 1. Python is a universal language that can run on almost every computer.
SOME BASIC FACTS:
2. Blockchain platforms can also act as a computer, and be programmed like any other computer.
3. Python and some blockchain platforms share a similarly structured runtime architecture.
7. BLOCKCHAIN PLATFORMS AS A COMPUTER
Ethereum Virtual Machine (EVM)
Provides a Turing-complete execution environment
Provides a high level programming language
& easier code deployment support
Adjustable code execution fees (Gas)
Better blockchain extensibility for
application usage
8. CODE EXECUTION ON THE EVM
pragma solidity ^0.4.11;
contract hello
{
string world;
constructor() public
{
world = "Hello, World!";
}
function sayHello() public view returns (string)
{
return world;
}
}
Solidity Code (Smart Contract)
9. CODE EXECUTION ON THE EVM
pragma solidity ^0.4.11;
contract hello
{
string world;
constructor() public
{
world = "Hello, World!";
}
function sayHello() public view returns (string)
{
return world;
}
}
Solidity Code (Smart Contract)
Ethereum Blockchain
Compile & Deploy
10. CODE EXECUTION ON THE EVM
pragma solidity ^0 .4 .1 1 ;
contract hello
{
string world;
constructor() public
{
world = "Hello, World!";
}
function sayHello() public view returns (string)
{
return world;
}
}
Solidity Code (Smart Contract)
Ethereum Blockchain
Compile & Deploy
Application Code
Smart Contract
Interaction Library
Access Smart Contract
via Contract ABI
11. BLOCKCHAIN PLATFORMS AS A COMPUTER
Ethereum Virtual Machine (EVM)
Poor development environment support
Confusing language syntax
Inefficient Gas system
(The Extensibility Issue)
Language compatibility
Resource-intensive;
not suitable for embedded systems
12. BLOCKCHAIN PLATFORMS AS A COMPUTER
Ethereum Virtual Machine (EVM)
Poor development environment support
Confusing language syntax
Inefficient Gas system
(The Extensibility Issue)
Language compatibility
Resource-intensive;
not suitable for embedded systems
14. PYTHON TO THE RESCUE
Integrating Smart Contracts with existing
Python code and applications
Writing & Compiling Ethereum Smart Contracts
with Python
Modifying & Running the EVM
on embedded devices for IoT system development
15. PYTHON TO THE RESCUE
Integrating Smart Contracts with existing
Python code and applications
Writing & Compiling Ethereum Smart Contracts
with Python
Modifying & Running the EVM
on embedded devices for IoT system development
16. SMART CONTRACT INTEGRATION
‣ web3.py: A Python implementation of the web3.js library (originally written in JavaScript);
used to call Smart Contract code within local Python code
‣ Populus: A Python-based Smart Contract development environment
for Ethereum blockchain developers using Python
‣ Py-EVM: A Python implementation of the Ethereum Virtual Machine (EVM) itself;
will modify this to build a custom implementation of the EVM optimized for embedded systems
SOME PREREQUISITES:
‣ Trinity: An Ethereum blockchain client based on the Py-EVM project
& completely written in Python
18. SMART CONTRACT INTEGRATION
WRITING A “HELLO WORLD” SMART CONTRACT
& INTEGRATING SOLIDITY CODE WITH PYTHON
pragma solidity ^0.4.11;
contract hello
{
string world;
constructor() public
{
world = "Hello, World!";
}
function sayHello() public view returns (string)
{
return world;
}
}
Solidity Code (Smart Contract)
./contracts/hello.sol
19. SMART CONTRACT INTEGRATION
WRITING A “HELLO WORLD” SMART CONTRACT
& INTEGRATING SOLIDITY CODE WITH PYTHON
pragma solidity ^0.4.11;
contract hello
{
string world;
constructor() public
{
world = "Hello, World!";
}
function sayHello() public view returns (string)
{
return world;
}
}
Solidity Code (Smart Contract)
./contracts/hello.sol
Python Code
./tests/test_HelloWorld.py
def test_HelloWorld(chain):
hi, _ = chain.provider.get_or_deploy_contract(hello)
helloString = hi.call().sayHello()
assert helloString == “Hello, World!” #Test Only
20. def test_HelloWorld(chain):
hi, _ = chain.provider.get_or_deploy_contract(hello)
helloString = hi.call().sayHello()
assert helloString == “Hello, World!” #Test Only
SMART CONTRACT INTEGRATION
WRITING A “HELLO WORLD” SMART CONTRACT
& INTEGRATING SOLIDITY CODE WITH PYTHON
pragma solidity ^0.4.11;
contract hello
{
string world;
constructor() public
{
world = "Hello, World!";
}
function sayHello() public view returns (string)
{
return world;
}
}
Solidity Code (Smart Contract)
./contracts/hello.sol
Python Code
./tests/test_HelloWorld.py
1. chain argument: py.test fixture, provided by populus’ pytest plugin
- chain object provided by populus, creates & calls a temp blockchain (“tester”) on memory
2. Creates a web3 contract object instance that corresponds to the hello contract written in Solidity
- The hi object created here acts as a wrapper for Solidity code for use within Python
3. Call the sayHello() function
defined within the hello contract
4. (Used with pytest) Check whether the hello contract was executed successfully
21. SMART CONTRACT INTEGRATION
TESTING SMART CONTRACT INTEGRATION WITH PYTEST
# Create a new Populus Project
$ mkdir helloTest
$ cd helloTest
$ populus init
$ rm contracts/Greeter
$ rm tests/test_greeter.py
# Copy Solidity & Python source code
$ cp ~/sources/hello.sol contracts/hello.sol
$ cp ~/sources/test_HelloWorld.py tests/test_HelloWorld.py
# Compile & Deploy on a test blockchain
$ populus compile
$ populus deploy —chain tester hello
# Run pytest
$ py.test
25. PYTHON TO THE RESCUE
Integrating Smart Contracts with existing
Python code and applications
Writing & Compiling Ethereum Smart Contracts
with Python
Modifying & Running the EVM
on embedded devices for IoT system development
26. PRIVATE BLOCKCHAIN SETUP FOR IOT
Private Ethereum Blockchain
Controller Smart Contract
Django Controller Server
Web Browser Mobile App
Raspberry Pi Raspberry Pi
Raspberry Pi Raspberry Pi
Coffee Machine Curtains
Light Controllers A/C
27.
28. PRIVATE BLOCKCHAIN SETUP FOR IOT
Private Ethereum Blockchain
Controller Smart Contract
Django Controller Server
Web Browser Mobile App
Raspberry Pi Raspberry Pi
Raspberry Pi Raspberry Pi
Coffee Machine Curtains
Light Controllers A/C
?
29. EVM CODE EXECUTION OVERVIEW
THE ETHEREUM BLOCKCHAIN
Block N Header:
Previous Block Hash Nonce Timestamp Uncle Hash Beneficiary
Logs Bloom PoW Difficulty Extra Data Block Number
Gas Limit Used Gas Amount Mix Hash
State Root Transaction Root Receipt Root
Block N+1 Header:
Previous Block Hash Nonce Timestamp Uncle Hash Beneficiary
Logs Bloom PoW Difficulty Extra Data Block Number
Gas Limit Used Gas Amount Mix Hash
State Root Transaction Root Receipt Root
Receipt Merkle Tree
Transaction Merkle Tree
Receipt Merkle Tree
Transaction Merkle Tree
Nonce Code Hash Balance Storage Root
Code
Nonce Code Hash Balance Storage Root
Merkle-Patricia State Trie for block N Merkle-Patricia State Trie for block N+1
73 95
Uncle Block Headers List Uncle Block Headers List
30. THE ETHEREUM BLOCKCHAIN
Block N Header:
Previous Block Hash Nonce Timestamp Uncle Hash Beneficiary
Logs Bloom PoW Difficulty Extra Data Block Number
Gas Limit Used Gas Amount Mix Hash
State Root Transaction Root Receipt Root
Block N+1 Header:
Previous Block Hash Nonce Timestamp Uncle Hash Beneficiary
Logs Bloom PoW Difficulty Extra Data Block Number
Gas Limit Used Gas Amount Mix Hash
State Root Transaction Root Receipt Root
Receipt Merkle Tree
Transaction Merkle Tree
Receipt Merkle Tree
Transaction Merkle Tree
Nonce Code Hash Balance Storage Root
Code
Nonce Code Hash Balance Storage Root
Merkle-Patricia State Trie for block N Merkle-Patricia State Trie for block N+1
73 95
Uncle Block Headers List Uncle Block Headers List
EVM CODE EXECUTION OVERVIEW
31. THE ETHEREUM VIRTUAL MACHINE: AN OVERVIEW (MEMORY MANAGEMENT)
‣ Memory: A temporary storage area used when calling functions, or when during
regular memory operations
‣ Stack: No concepts of “registers” are provided; a virtual stack is used instead for operations
such as opcode parameters
‣ Storage: A permanent key-value storage mapping 256-bit words to 256-bit words;
memory space declared outside user-defined functions & within the contract context are stored here
Reference: https://github.com/comaeio/porosity/wiki/Ethereum-Internals
EVM CODE EXECUTION OVERVIEW
32. THE ETHEREUM VIRTUAL MACHINE: OPCODES & ASSEMBLY
‣ The EVM is (theoretically) language independent; however, due to its architectural limitations
the EVM cannot take full advantage of all features present in modern programming languages
& requires some unique language syntax (to access unique EVM features)
‣ The EVM implements a simple instruction set to make up EVM bytecode (to ensure platform / ISA
independence)
‣ The EVM can also be directly programmed at a low-level using an Assembly Language provided
by Solidity; Assembly can be used along with Solidity code with inline Assembly
EVM CODE EXECUTION OVERVIEW
33. THE ETHEREUM VIRTUAL MACHINE: OPCODES & ASSEMBLY
‣ The EVM is (theoretically) language independent; however, due to its architectural limitations
the EVM cannot take full advantage of all features present in modern programming languages
& requires some unique language syntax (to access unique EVM features)
‣ The EVM can also be directly programmed at a low-level using an Assembly Language provided
by Solidity; Assembly can be used along with Solidity code with inline Assembly
EVM CODE EXECUTION OVERVIEW
‣ The EVM implements a simple instruction set to make up EVM bytecode (to ensure platform / ISA
independence)
36. WILL IT BE POSSIBLE
TO DEVELOP A SMART CONTRACT
USING PYTHON?
37. SOME METHODS TO PORTING PYTHON TO THE EVM
2. ADDING EVM-SPECIFIC SYNTAX TO PYTHON AS A SUPERSET
OF THE PYTHON LANGUAGE
3. (OUR APPROACH) ALLOWING PYTHON CODE TO COMPILE DIRECTLY TO EVM BYTECODE
WITHOUT LARGELY MODIFYING OR ADDING PYTHON LANGUAGE SYNTAX
1. IMPLEMENTING THE PYTHON INTERPRETER ITSELF AS A SMART CONTRACT
IN SOLIDITY
38. SOME METHODS TO PORTING PYTHON TO THE EVM
1. IMPLEMENTING THE PYTHON INTERPRETER ITSELF AS A SMART CONTRACT
IN SOLIDITY
‣ However, as the EVM is a distributed execution environment, this method may consume
excessive compute power & Gas (which is very inefficient in terms of extensibility)
‣ Jython (the JVM implementation of Python) uses this method to run Python code on the JVM
(Reimplementing the Python language with Java, rather than C as used by standard CPython)
‣ Because this method abstracts the language compilation process through a
modular implementation, existing pure Python code is almost completely reusable
39. SOME METHODS TO PORTING PYTHON TO THE EVM
‣ The problem: adding new syntax to Python means backwards compatibility cannot be ensured
with existing Python code (Can Python with added custom syntax be still considered as Python?)
‣ Similar to how Objective-C added object-oriented programming capabilities with added syntax
while also maintaining full backwards compatibility with C
‣ The Ethereum Foundation chose a similar approach with a new language called Vyper,
which also targets the EVM (like Solidity) but with Pythonic syntax
2. ADDING EVM-SPECIFIC SYNTAX TO PYTHON AS A SUPERSET
OF THE PYTHON LANGUAGE
40. OUR SOLUTION: A FULL PYTHON PORT
BUILDING A PYTHON COMPILER THAT TARGETS THE EVM
pragma solidity ^0.4.11;
contract Wallet
{
mapping (address => uint) public balance;
function deposit()
{
balance[msg.sender] += 1;
}
function withdraw(uint value)
{
if(balance[msg.sender] < value) throw;
balance[msg.sender] -= 1;
if(!msg.sender.call.value(value)()) throw;
}
}
Solidity Code (Smart Contract)
41. OUR SOLUTION: A FULL PYTHON PORT
BUILDING A PYTHON COMPILER THAT TARGETS THE EVM
pragma solidity ^0.4.11;
contract Wallet
{
mapping (address => uint) public balance;
function deposit()
{
balance[msg.sender] += 1;
}
function withdraw(uint value)
{
if(balance[msg.sender] < value) throw;
balance[msg.sender] -= 1;
if(!msg.sender.call.value(value)()) throw;
}
}
Solidity Code (Smart Contract) Python Code (Smart Contract)
import ethereum as eth
class Wallet(eth.contract):
balance = eth.mapping(eth.address, int)
def deposit():
balance[eth.msg.sender] += 1
def withdraw(value: int):
if balance[eth.msg.sender] < value:
raise ValueError
balance[eth.msg.sender] -= 1
if not eth.msg.sender.call.value(value)():
raise ValueError
42. OUR SOLUTION: A FULL PYTHON PORT
BUILDING A PYTHON COMPILER THAT TARGETS THE EVM
Python Source Code
Bytecode Translation
Python Bytecode
(.pyc)
Python Virtual Machine
Library Modules
Code Execution
A standard CPython implementation of Python
43. OUR SOLUTION: A FULL PYTHON PORT
BUILDING A PYTHON COMPILER THAT TARGETS THE EVM
Python Source Code
Grammar
Python Bytecode
(.pyc)
Python Virtual Machine
Library Modules
Code Execution
A standard CPython implementation of Python
Lexer
Parser
Compiler
44. OUR SOLUTION: A FULL PYTHON PORT
BUILDING A PYTHON COMPILER THAT TARGETS THE EVM
Python Source Code
On-the-fly
EVM Bytecode Translation
EVM Bytecode
Modified EVM
Code Execution
Precompiled EVM bytecode
(Imported as Python Library)
Deploy
Our Python-EVM Implementation Structure
45. OUR SOLUTION: A FULL PYTHON PORT
BUILDING A PYTHON COMPILER THAT TARGETS THE EVM
Python Source Code
EVM Bytecode
Code Execution
Our Python-EVM Implementation Structure
Precompiled EVM bytecode
(Imported as Python Library)
Deploy
Grammar (CPython)
Modified Lexer
(Added EVM-specific tokens)
Modified Parser
(Generates Solidity-style structure)
Compiler with solc backend
Modified EVM
46. OUR SOLUTION: A FULL PYTHON PORT
BUILDING A PYTHON COMPILER THAT TARGETS THE EVM
pragma solidity ^0.4.11;
contract Wallet
{
mapping (address => uint) public balance;
function deposit()
{
balance[msg.sender] += 1;
}
function withdraw(uint value)
{
if(balance[msg.sender] < value) throw;
balance[msg.sender] -= 1;
if(!msg.sender.call.value(value)()) throw;
}
}
Solidity Code (Smart Contract) Python Code (Smart Contract)
import ethereum as eth
class Wallet (eth.contract):
balance = eth.mapping(eth.address, int)
def deposit():
balance[eth.msg.sender] += 1
def withdraw(value: int):
if balance[eth.msg.sender] < value:
raise ValueError
balance[eth.msg.sender] -= 1
if not eth.msg.sender.call.value(value)():
raise ValueError
1. ethereum module: A Python wrapper that allows
EVM-specific features to be called within
Python code
2. Wallet class: inherits the contract class
defined within the ethereum module
to define a Contract
3. eth.mapping: returns an access token
to the Mapping type value ;
eth.address: same as the address type defined
with Solidity (20 byte value)
4. Modified syntax to access
Mapping type variable
47. OUR SOLUTION: A FULL PYTHON PORT
DYNAMIC PYTHON-STYLE ETHEREUM VIRTUAL MACHINE
Python Source Code
EVM Bytecode
Code Execution
Our Python-EVM Implementation Structure
Precompiled EVM bytecode
(Imported as Python Library)
Grammar (CPython)
Modified Lexer
(Added EVM-specific tokens)
Modified Parser
(Generates Solidity-style structure)
Compiler with solc backend
Modified EVM
Deploy
48. ‣ …but I cannot disclose all of them in detail just yet
‣ Execution Logic Improvements: No simultaneous code execution on devices
marked as “embedded”; only codehash checks (integrity insurance) are performed on-device
‣ AOT Compiler: Allows ahead-of-time EVM bytecode compilation with embedded devices
OUR SOLUTION: A FULL PYTHON PORT
DYNAMIC PYTHON-STYLE ETHEREUM VIRTUAL MACHINE
51. BLOCKCHAIN PLATFORMS AS A COMPUTER
Ethereum Virtual Machine (EVM)
Poor development environment support
Confusing language syntax
Inefficient Gas system
(The Extensibility Issue)
Language compatibility
Resource-intensive;
not suitable for embedded systems