Novi Sad AI is the first AI community in Serbia with goal of democratizing knowledge of AI. On our first event we talked about Belief networks, Deep learning and many more.
4. NOVI SAD APPLIED INTELLIGENCE
COMMUNITY
City.AI NOVI SAD shaping up around a community whose goal
are :
To help local actors develop efficiently the Serbian
branch on AI internationally
To work around applied AI challenges with the local &
global ecosystem actors
To democratize AI innovation and close the gap
between technology and society
To train and challenge the local community
5. LEVERAGING THE POTENTIAL OF AI IN 40+ CITIES
AFRICA
Accra - Lagos
ASIA
Bangalore - Bangkok - Beirut - Chiang Mai - Hanoi - Hong Kong -
Jakarta - Johor Bahru - Karachi - Lahore - Manila - Pune - Seoul -
Singapore - Taipei
AUSTRALASIA
Wellington
EUROPE
Amsterdam - Berlin - Bratislava - Bristol - Brussels - Bucharest -
Budapest - Cambridge - Cluj - Cologne - Copenhagen - Hamburg -
Iasi - Krakow - Kyiv - London - Madrid - Munich - Novi Sad - Oxford
- Paris - Sofia - Stockholm - Stuttgart - Tallinn - Tirana - Valencia -
Valletta - Vienna - Vilnius
NORTH AMERICA
Austin - LA - New York - San Diego - San Francisco
SOUTH AMERICA
Bogota - La Paz - Sao Paulo
7. Our team
NOVI SAD AI TEAM
Jovan Stojanovic
Ambassador of Novi Sad-AI
Marko Jocic
Co-Ambassador of Novi Sad-AI
Jovana Miletic
Operation manager of Novi Sad-AI
8. Dejan Vukobratovic
PhD Proffesor and researcher, FTN
on
Belief networks
Ivan Peric
Deep learning engineer
on
NLP in Fintech
LESSONS LEARNED BY
9. Outline of the Talk
âą Who are we?
â iCONIC Centre â Hotspot for massive communications and
information processing
âą Our Focus?
â Large-scale networks for information acquisition and
processing (5G)
âą Topic This Talk?
â Probabilistic Graphical Models and Belief Propagation
â Applications and Connections (to Deep Learning?)
10. Outline of the Talk
âą Who are we?
â iCONIC Centre â Hotspot for massive communications and
information processing
âą Our Focus?
â Large-scale networks for information acquisition and
processing (5G)
âą Topic This Talk?
â Probabilistic Graphical Models and Belief Propagation
â Applications and Connections (to Deep Learning?)
12. Outline of the Talk
âą Who are we?
â iCONIC Centre â Hotspot for massive communications and
information processing
âą Our Focus?
â Large-scale networks for information acquisition and
processing (5G)
âą Topic This Talk?
â Probabilistic Graphical Models and Belief Propagation
â Applications and Connections (to Deep Learning?)
15. https://shop.sodaq.com/en/nb-iot-shield-deluxe.html
3GPP Narrowband IoT (NB-IoT)
NarrowBand IOT
âą Standardized within 3GPP
Release 13 (Nov 2016)
âą Already in testing/
deployment at many mobile
operators
âą Low-Power WAN
âą Alternative to LoRa, SigFox
âą âNext big thingâ for mobile
operators
Early market solutions
16. 5G Focus No. 2: Distributed Information Processing in
Mobile Edge Computing (MEC)
17. Example: Massive Data Acquisition and Distributed
Information Processing in Smart Grids
IEEE Communications Magazine, Vol. 55, No. 10, October 2017. (http://ieeexplore.ieee.org/document/8067687/)
18. Outline of the Talk
âą Who are we?
â iCONIC Centre â Hotspot for massive communications and
information processing
âą Our Focus?
â Large-scale networks for information acquisition and
processing (5G)
âą Topic This Talk?
â Probabilistic Graphical Models and Belief Propagation
â Applications and Connections (to Deep Learning?)
19. Probabilistic Graphical Models
âą Model dependencies between random variables of a large-scale system
x1
x2
xN-1
xN
x4
x5
xN-3
xN-2
fs1
fs2
fsM
. . .
x3
21. Probabilistic Graphical Models
âą We observe (measure) subset of random variables of a large-scale system
x1
x2
xN-1
xN
x4
x5
xN-3
xN-2
fs1
fs2
fsM
. . .
x3
22. Belief Propagation Algorithm
âą Based on a new evidence, we infer values of all variables in the system
x1
x2
xN-1
xN
x4
x5
xN-3
xN-2
fs1
fs2
fsM
. . .
x3
23. Belief Propagation Algorithm
âą Based on a new evidence, we infer values of all variables in the system
x1
x2
xN-1
xN
x4
x5
xN-3
xN-2
fs1
fs2
fsM
. . .
x3
24. Belief Propagation Algorithm
âą After BP converges, we obtain new beliefs about all system variables
x1
x2
xN-1
xN
x4
x5
xN-3
xN-2
fs1
fs2
fsM
. . .
x3
p(xN) p(xN)
29. BP and Deep Learning
âą https://sinews.siam.org/Details-Page/deep-deep-trouble
âą Michael Elad (Technion) blog âDeep, deep trouble: Deep Learningâs
Impact on Image Processing, Mathematics and Humanity â
â âUnfortunately, all of these great empirical achievements were obtained
with hardly any theoretical understanding of the underlying paradigm.
Moreover, the optimization employed in the learning process is highly non-
convex and intractable from a theoretical viewpoint.â
â âShould we be happy about this trend? Well, if we are in the business of
solving practical problems, the answer must be positive. Right? Therefore, a
company seeking such a solution should be satisfied. But what about us
scientists?â
â âThis is clearly not the school of research we have been taught, and not the
kind of science we want to practice. Should we insist on our more rigorous
ways, even at the cost of falling behind in terms of output quality?â
31. Thinking Fast and Slow
âą Two cognitive mechanisms:
âą FAST: akin to DNN
â Instinctive
â Instantaneous
â Computationally fast shortcut
âą SLOW: akin to PGM and BP
â Thoughtful but slow
â Deep
â Natural
32. Dejan Vukobratovic
PhD professor and researcher
on
Belief networks
Ivan Peric
Deep learning engineer
on
NLP in Fintech
LESSONS LEARNED BY
33. Application of Deep Learning to NLP
task in commercial projects
IVAN PERIÄ, NOVI SAD 2018
34. AI/ML Engineer â Synechron Serbia,
Novi Sad
Who am I?
Teaching Assistant â Chair of
Informatics, Faculty of Technical
Sciences, University of Novi Sad
35. What is Deep Learning?
ï” Main goal â âCompletelyâ simulate human brain
ï” Mathematical models that approximate biological concepts, like neurons
ï” Very big networks of artificial neurons
ï” A lot of computations
ï” The result is a hierarchical feature set extractor, that can be used for classification,
transformation to other feature sets, etc.
ï” Known architectures
ï” Multilayer Perceptron Neural Networks - MLPs
ï” Convolutional Neural Networks â CNNs
ï” Recurrent Neural Networks - RNNs
36. Where are we now?
ï” State of the art deep artificial neural networks contain at most thousands of
neurons
ï” Biological human brain contains 15-30 billion neurons
37. Use Case - Automated Question Answering Engine
ï” Terabytes (or even petabytes) of documents
ï” Banking contracts
ï” Financial reports
ï” Financial news articles
ï” Client emails
ï” Answer questions that might have an answer in
these documents
ï” No predefined question list
38. Automated Question Answering Engine - Idea
[Question,
Answer,
Context]
Word
Embedding
Bidirectional
LSTM with
Attention
Mechanism
Answer
39. Word Embedding
ï” Words need to be coded as numbers to be a valid input to ANN
ï” One-Hot representation (high dimensionality, no semantics in word positions)
ï” Word embedding allows word representation in a dense vector space, and adds
contextual similarity to words (word2vec, GloVe)
40. Bidirectional LSTM (Long Short-Term Memory)
ï” Unidirectional LSTMs capture dependencies only in one direction
ï” Bidirectional LSTMs captures future and past dependencies and work very good in
tasks that can benefit from this fact
41. Attention mechanism in LSTM
ï” Attention mechanism lets network decide which
part of the input sequence it should focus on
ï” Focused part is probably more important for the
task the sequence model is being trained for than
the rest of the sequence
ï” In this case, decoder output depends on a
weighted combination of all the input states, not
just the last state
42. Bidirectional Attention Flow mechanism
ï” Machine Comprehension of textual data
Published as a conference paper at ICLR 2017
43. Constraints in Deep Learning application
to commercial projects
ï” Technical constraints in commercial use of
Deep Learning
ï” Lack of any kind of data
ï” Vast amounts of data, but unlabeled
ï” Data is not opened to the public
ï” Bad infrastructure for model training
44. Constraints in Deep Learning application
to commercial projects
ï” Functional constraints in commercial use of Deep Learning
ï” Lack of understanding of capabilities of Deep Learning
ï” Black box model
ï” Hard or impossible to explain any non-working cases to the client
ï” Hard or impossible to explain any working cases to the client
ï” Traditional Data Science approaches based on statistical models, as well as conventional
reasoning in AI and ML (searches, optimization algorithms, fuzzy logic, âŠ) still look more
acceptable to clients
ï” Usually easier to explain
ï” They can work very well without big amounts of data