The document discusses using genetic algorithms and Java to develop trading algorithms that can find repeating patterns in market data and time the market. It describes collecting market data, generating trading signals from the data, defining possible trading decisions, using genetic algorithms to evolve candidate trading strategies by simulating trades and scoring strategies based on profits, and testing the best evolved strategies on out-of-sample market data. The results showed average profits of 3% per trade for the generated trading models.

1. Using Java & Genetic Algorithms to Beat the Market Matthew Ring JavaOne 2011 BOF Session 22382 [20111007 15:11CDT – MRing – added results pages]

2. 401K Problem?

3. I can't speak for you, but...

4. I know, I'll write software to time the market!

5. Give it the 'ole college try, son!

7. Cheap or free market data. Especially daily OHLCV data.

8. Generally Liquid.

9. Low transaction fees.

11. Free.

12. Huge number and variety of open source libraries.

15. Inside information.

16. Gut instinct.

17. Find repeating patterns in the market.

18. Manipulate other participants.

19. Better or faster information.

20. Faster execution.

22. Patterns? … Signals?

23. Technical Analysis!

24. TA-Lib : Technical Analysis Library – Open-source API for C/C++, Java , Perl, Python and 100% Managed .NET

26. Oscillators

27. Moving averages

28. Candlesticks

29. Trading decisions as reactions to these signals

30. Voodoo?

33. Goal is optimize for profitable trades, of course.

35. Simulated Annealing

36. Emergent Behavior (swarm, cellular automata)

37. Genetic Programming

38. Genetic Algorithms

39. Support Vector Machines

40. Genetic Algorithms Sources: http://www.slideshare.net/kancho/genetic-algorithm-by-example (Nobal Niraula)

41. Genetic Algorithms Sources: http://www.slideshare.net/Mathijsje/genetic-algorithms-7276974 (Mathijs van Meerkerk)

42. Some possibly familiar examples of GA...

44. Provides easy-to-follow examples & interface-based API.

45. Our Machine Market Data Operational Settings Trading Decisions ? Backtesting Feedback GA

47. Generate signals from market data with TA-LIB.

48. Generate choices for trading decisions.

49. React to the signals

50. Map reactions to choices

51. Test the choices made.

52. Generate Signals

54. How much to buy?

55. How long to hold?

56. Predicted low price (for buying)?

57. Predicted high price (for selling)?

58. Price at which I should sell prior to end of holding period (opportunistic sell)?

59. Generate Choices for Trading Decisions

60. Generate Choices for Trading Decisions

62. Choice values are generated from 'random' doubles, scaled to meet their various requirements.

64. Each Thresholds instance is initialized w/ a 'random', sorted array of doubles, scaled to span the range of the signal it is to consume.

65. This initialization step establishes both the number of bins and the bin boundaries. public int findBin(double testVal)

67. The bin index is the reaction.

68. Sort of like an A/D converter, if you pretend that the signal is analog, sampled daily.

69. A subclass provides a memory of its previous reaction, so it reacts to both the current signal level and the change from its previous reaction.

71. I have chosen 24 various TA signals.

72. I have defined 6 trading-related decisions: Buy?, How much?, Holding period? Low price guess? High price guess? Opportunistic sell price?

73. For each decision, for each signal, a Thresholds instance will react.

75. The mappings are established 'randomly'

76. The choice that gets the most bin reactions is selected.

77. Winner take-all voting.

78. Note that some mappings will effectively abstain from voting

79. Ties are broken based on iteration order.

81. Added combinational voting to each layer.

82. Some 'randomly' selected bins are ANDed with others, then 'randomly' mapped to choices for the given decision layer.

83. More opinions in the mix.

86. Sell X at T', where T' < T+3

87. Buy Y on T' with proceeds from sale of X

88. Cannot sell Y before T'+3

89. If you do, the Federal Reserve Board requires that your broker freeze your account for 90 days

91. Sell X later on T

92. If you do this more than 3x in a 5-day period, you will be classified as a Pattern Day Trader

94. Market will eat up high bids & low asks.

96. Iterate signal data while trying to create trades based on machine advice & market actuals.

97. Remember to skip first max 'lookback' number of days, as many signals involve moving averages.

98. Strongly bias 'buy' fills toward the high, 'sell' fills toward the low.

100. Sell is greedy – will try to sell at Max(predicted high price, chosen opportunistic sell price)

101. Any position where holdDays >= maxHoldDays is dumped at the end of the day at a price biased toward the low and the close.

103. This is where the GA comes into play.

104. The genome, a double[6850] serves as the settings, the 'randomness', for each candidate.

106. The GA will select the fittest candidates for mating, based on their scores.

107. The GA will recombine (crossover) and mutate (spontaneously change values) mated genomes.

111. I pre-screen the stocks used, for swing trading appropriateness. I don't want to wait while it runs through the whole Nasdaq.

112. I backtest & evolve on a subset of available data.

113. Then I try (backtest w/o evolution) the top evolved candidates against recent out-of-sample data.

115. Designing the machine to work with it is a bit of an art.

118. Scoring – I'm using the ending account balance (cash + approx val of open positions) as the score.

119. The chosen scoring formula has a critical impact on the out come. It provides the 'motivation' for the GA to 'improve'.

120. What happens when you try a score like numGoodTrades / numBadTrades?

121. Watchmaker only supports positive double score values.

122. The Trade PnL calculation includes transaction costs (my target broker is $2.50 a trade)

124. http://www.eoddata.com/

125. http://ta-lib.org/

126. http://www.slideshare.net/kancho/genetic-algorithm-by-example

127. http://www.slideshare.net/Mathijsje/genetic-algorithms-7276974

129. Each model starts w/ $2000.

130. Evolve & test models on 20080101 thru (Today - 45 trading days). This is the in-sample data.

131. Stagnation Condition: quit after 7 generations w/ no improvement.

132. Keep the best 2 evolved individuals for each stock, for a total of 8 reserved candidates.

133. Test them on out-of sample data of (Today - 44 trading days) thru Today.

134. Print out the trading results of the top 3 out-of-sample tests.

135. Results – Talkin' 'bout an evolution... Run 1 Run 2 Best candidate score Vs. the number of generations Yes, the score is in $, but since this is in-sample data, please take it with a grain of salt.

136. Results – Out-of-sample trades Run 1 Run 2

138. The graphs show that evolution is a choppy process, due in part to my use of aggressive crossover & mutation strategies as well my choice not to enable elitism (best members transcend their own generation).

140. So, the generated models appear to have predictive power.

141. The validity of these results depends on my belief that my backtesting routine is both reasonable & bug-free.

142. No guarantee that the evolved models will continue to work when facing new, out-of-sample data. I'm still just betting. But, maybe, with an edge.