5. 인공지능에 대한 다양한 관점
• 데이터
• 수학/통계
• 정보 검색
• 심리/인지과학
• 컴퓨터과학
• and so on
6. 튜링테스트
• 튜링 테스트(Turing test): 기
계가 인간과 얼마나 비슷하게
대화할 수 있는지를 기준으로
기계에 지능이 있는지를 판별
하고자 하는 테스트
• “컴퓨터로부터의 반응을 인간
과 구별할 수 없다면 컴퓨터는
생각(사고, thinking)할 수 있는
것” - 앨런 튜닝
8. Human-Like?
Definition: Software Agents in AI
An agent is a computer system that is situated in some
environment, and that is capable of autonomous action in
this environment in order to meet its design objectives
9. 대다수 인공지능 연구의 목적
Problem Solving
사람을 대신하거나 도와서 어떤 문제를 해결
14. 문제해결을 위해 필요한 것
• 경험: Explicit Memory ==> Data
• 규칙: Implicit Memory ==> Hand-coded Rules
• 사고방식: Implicit Memory ==> Model, Machine
Learning
15. 기계학습
• 학습: 직간접적 경험이나 훈련에 의해 지속적으로 지각하고, 인지하며, 변
화시키는 행동 변화
• 기계학습 정의
• 컴퓨터가 학습할 수 있도록 하는 알고리즘과 기술을 개발하는 분야
• 컴퓨터에게 배울 수 있는 능력, 즉 코드로 정의하지 않은 동작을 실행하
는 능력에 대한 연구 분야
• 기계 학습에서의 일반화는 훈련 이후 새롭게 들어온 데이터를 정확히
처리할 수 있는 능력
• 문제해결의 규칙/수식을 알고 기계에게 알려주는 방식은 기계학습이 아님
16. Three niches for machine learning
• Data mining: using historical data to improve decisions
• ex) medical records —> medical knowledge
• Software applications we can’t program by hand
• ex) autonomous driving, speech recognition
• Self customizing programs
• ex) Newsreader that learns user interests
• (출처: lecture slides for textbook Machine Learning, T. Mitchell,
MacGraw Hill, 1997)
17. Ms. Pac-Man 관련 기계학습방법
• 인공신경망(Artificial Neural Networks)
• 진화연산(Evolutionary Computation)
• 진화신경망(Evolved Artificial Neural Networks)
• 공통특징
• 소프트컴퓨팅
• 학습 결과 모델이 추출됨
• 사람의 수학/통계적 해석이 어려움(거의 불가능)
18. 인공신경망
• 신경망: 인간의 중추신경계. 감각을 인식하고 반응을 하기까지 정보
처리는 뉴런들 사이의 정보 이동을 통해 진행
• 인공신경망: 인공신경망은 시냅스의 결합으로 네트워크를 형성한 인
공 뉴런(노드)이 학습을 통해 시냅스의 결합 세기를 변화시켜, 문제
해결 능력을 가지는 모델
출처: http://study.zum.com/book/13893 출처: https://ko.wikipedia.org/wiki/인공신경망
19. 인공신경망 연구 예시
• 이미지 인식
• 글씨 인식
• 군집화(SOM: Self-
Organized Map)
• 딥러닝(Deep Learning)
http://www.cs.cmu.edu/~tom/faces.html
20. When to consider ANN
• Input is high-dimensional discrete or real-valued
• Output is discrete or real valued
• Output is a vector of values
• Possibly noisy data
• Form of target function is unknown
• Human readability of result is unimportant
• ex) Image classification, Financial prediction, Speech phoneme recognition
• (출처: lecture slides for textbook Machine Learning, T. Mitchell, McGraw Hill,
1997)
21. 진화연산
• EC: Evolutionary Computation
• 자연계의 진화에서 아이디어를 얻음(다윈주의)
• 환경에 적합한 유전자를 가진 종이 다음 세대에 자신의
유전자를 전달할 확률이 높음
• 환경에 대한 반응을 유전자로 정의하고 실제 환경에 얼마
나 적합한지를 평가하여 다음 세대에 반영
22. 유전자 알고리즘
• 유전자: 개체의 특징이 정의 (주로 배열)
• 적합도(Fitness): 유전자가 환경에 적합한 점수. 선택의
기준이 됨
• 선택(Selection): 다음 세대에 유전자를 전달할 개체 후보
선택
1 1 0 1 0 0 1 1
23. 유전자 알고리즘(계속)
• 교차/교배(Crossover): 두 부모 유전자의 특성을 조합하
여 새로운 유전자를 생성
• 돌연변이(Mutation): 교배없이 유전자가 스스로 변이
출처: http://www.j-kosacs.or.kr/journal/article.php?code=11856
24. 진화 연산 연구 예시
• Game Theory
• Genetic Programming
• Artificial Life
• Evolving the user interface
• https://blog.mozilla.org/
ux/2013/04/firefox-next-
evolving-the-user-
interface-using-genetic-
algorithms/
25. 진화신경망
• ENN: Evolved Neural Networks, Evolutionary Neural Networks, Evolving Neural
Networks
• 진화연산을 이용하여 신경망 학습
• 게임, 로봇 등 평가를 위해 시뮬레이션이 필요한 학습에 다양하게 이용
• HyperNEAT/NEAT
• Kenneth O. Stanley, et al. “A Hypercube-Based Encoding for Evolving Large-Scale
Neural Networks,” Artificial Life journal vol. 15, no. 2, pp. 185–212. Cambridge, 2009
• 신경망의 토폴로지 기반 진화연산 사용
• Hidden Layer의 구조와 연산 방법을 진화연산으로 결정
• C++/C# 라이브러리 제공(http://eplex.cs.ucf.edu/hyperNEATpage/)
28. A Hybrid Method of Dijkstra
Algorithm and Evolutionary Neural
Networks for Ms. Pac-Man Agent
Keunhyun Oh and Sung-Bae Cho
Republic of Korea
World Congress on Nature and Biologically Inspired Computing 2010
29. Background: Game AI
• Ideal test environments for artificial intelligent
• Happening random events
• Huge decision space
• Challenging task to find Good Strategy
30. Background: Ms. Pac-Man
• One of the most popular area in game AI
• Navigating Ms. Pac-Man without user control
• Accumulating points and avoiding attacking ghosts
31. Motivation: Previous works
• Two Methods for Ms. Pac-Man agent
• Human-defined rules
• GOOD: Good rules’ guarantying stable high scores
• BAD: Difficulty in covering every situation because of non-
deterministic ghosts
• Evolutionary Computation
• GOOD: Covering uncertain environment
• BAD: Difficulty in designing an architecture for getting good score
and consuming high costs such as time
32. Motivation: Hybrid Approach
• To help covering each method’s advantage and
covering weakness
• Basically using rules based on Dijkstra algorithm
• Using evolved artificial neural networks to deal with
unpredicted circumstances that cannot be
determined by defined rules
33. Ms. Pac-Man Competition
• Providing the best software controller for the game of Ms. Pac-Man
• Annual popular competition
• IEEE CIG
• IEEE CEC
• The world record (2010)
• Human player: 921,360
• Software agent: 17,102 (avg. of 10 games)
34. Ms. Pac-Man Competition(cont’)
• Game version
• the Microsoft Revenge
of Arcade port of the
Ms Pac-Man game
• WebPacman: http://
www.webpacman.com/
• Software toolkit: Java
and C#
35. Approach: Hand-Coded Rule
• Designed by human experts who
understand the game well
• If-then rules and graph search algorithms
• Tree search strategy(D. Robles and S. M.
Lucas, 2009)
• RAMP(A. Fitzgerald et al., 2009)
• Ice Pambush(R. T. H. Matsumoto and C.
Tokuyama, 2008)
• Advantage: High scoring controllers
• Disadvantage: Hard to consider every
situation
36. Approach: EC
• Methods using evolutionary computation
• Problem solving using genetic algorithm
• Evolved artificial neural networks and fuzzy systems that are often used
• Reinforcement learning using EC(I. Szita and A. Lorenz, 2007)
• Evolved neural networks(S. M. Lucas, 2005)
• M. Gallagher and A. Ryan, 2008
• Advantage: Ability to deal with unpredicted situations
• Disadvantage: Hard to get high scores
38. Sensing
• Catching the information on the current situation
• Screen capture
• Input the game state to the game agent
• using capture of a game interface
• Pixel extractor: Finding the color of each pixel
• Feature extractor
• Understanding coordinates of each components
• ex) Location of pills and ghosts
• Information extactor
• Producing information on the environment
• ex) Distance between Ms. Pac-Man and a ghost
42. Evolutionary Neural Networks
• To response to uncertain circumstances
• Using NEAT Algorithm(K. O. Stanely and R.
Miikkulainen, 2002)
43. Evolutionary Neural Networks
• Input(#: 24): Game state
• Output(#: 4): Direction of Ms. Pac-Man(Up, Down, Right, Left)
Component Parameter Type
The nearest ghost,
The nearest edible ghost,
The nearest pill,
The nearest power pill
Distance Float
Up Bool
Down Bool
Right Bool
Left Bool
Ms. Pac-Man
Up Bool
Down Bool
Right Bool
Left Bool
44. The pseudo-code for evolving
neural networksInput:
int MAX_POPULATION, int MAX_GENERATION,
int number_of_game
GENE[] PacMan::EANN{
int generation=0;
int number_of_population=0;
int number_of_game=0;
int score=0;
int avg_score=0;
NEAT::GENE[] POPULATION=new GENE(MAX_POPULATION);
NEAT::Parameters params=new ECParameters();
LoadParameters(params);
RandomPopulation(POPULATION);
for(int i=0;j<MAX_GENERATION;j++){
for(int j=0;i<MAX_POPULATION;i++){
PacManSimulator(
POPULATION[j],number_of_game
);
}
if(i<MAX_GENERATION-1){
/* Fitness sharing, Selection,
crossover,and mutation */
NEAT::Generation(POPULATION, params);
}else Sort_by_fitness(POPULATION);
}
return POPULATION[0]; // best gene
}
45. Experiment
• Simulating three agents 10 times
• 1st: Evolved neural networks
• 2nd: Dijkstra based rules
• 3rd: Hybrid method(Proposed method)
• Ms. Pac-Man Software Kit
• C# toolkit developed by J. Flensbank and G. Yannakakis
• Containing Ms. Pac-Man simulator(we use it)
• url: http://mspacmanai.codeplex.com/
46. Experiment Settings
• Threshold of Dijkstra based rules: 2
• Parameters for evolving neural networks
Parameter Value
Population 100
Generation 1000
The mutation rate for connection’s weight 0.2
The mutation rate to add and delete nodes 0.2
Elitism proportion 0.1
Selection proportion 0.8
49. The score of each controller
• Max and avg score: The hybrid method is outstand!
• Min score: Though designed rules are highest, the
gap among the proposed methods is tiny
50. Statistics of decisions
in the hybrid method
• The average number of decisions
• The Dijkstra-based rules
• Mainly used
• Guarantying hish scores
• The evolved neural network
• Controlling her in only special environments
• Covering weakness of human-designed rules
• One decision’s influencing the overall game
Method # of decisions % of decisions
The Dijkstra-based rules 71504.6 0.99860343
The evolved neural network 100.5 0.00139657
51. Summary
• Hybrid method for optimal Ms. Pac-Man agent
• Using hand-coded rules and the evolved neural network
• Human designed rule based on Dijkstra algorithm
• Guarantying best choice in some cases
• Not covering every circumstance —> using evolved neural network
• Evolutionary computation
• Making her go to the safety location in every environment
• Helping human designed rule more safety
• The method’s getting higher scores than them of rules EC separately
