2. 2
Objetivo
Apresentar tecnologias e meios de forma objetiva e simples,
enriquecendo a bagagem de conhecimento dos presentes.
Agregar conhecimento que permita capitalizar.
3. 3
Disclaimmer
Esta apresentação NÃO FOI cuidadosamente revisada e por
isso não esta livre de erros.
Toda critica será muito bem aceita ;)
As opiniões apresentadas durante a palestra são de minha
responsabilidade e não tem relação com as
empresas/Instituições nas quais trabalhei/trabalho ou tive
qualquer tipo de relacionamento.
8. 8
Big Data
“Data is a new Class of economic asset, like
currency or gold.” Davos, Suiça, 2012 – World
Economic Summit
9. 9
Intro
Tecnologia + Metodologia para → Capturar, Armazenar e Analisar
grandes volumes de dados
Volume, Velocidade e Variedade (+ veracidade e valor)
Finalidade: dar subsidios para tomadas de decisão
Temos muitos dados, pouca informação
10. 10
Intro
Análise de sentimento com dados da Web e sociais
Departamentos de marketing usam feeds do Twitter para
realizar análise de sentimento e determinar o que os usuários
estão falando sobre a empresa e seus produtos ou serviços,
especialmente após o lançamento de um novo produto ou
release.
O sentimento do cliente deve ser integrado aos dados de
perfil do cliente para derivar resultados significativos. O
feedback do cliente pode variar de acordo com seus
aspectos demográficos.
Fraudes em plano de saúde, abertura de contas
O case Abertura de conta
12. 12
METODOLOGIA
Identificação Coleta
Algoritmo
pré
determinado
Resposta
Identificação Coleta Algoritmo Resposta
VARIÁVEL
INCERTA
CRESCENTE
RÁPIDA
TERABYTES
? ? Se não responder
rápido, nem adianta
responder.
ISSO É BIGDATA!
IDENTIFICAÇÃO, PREDIÇÃO E RECOMENDAÇÃO
O que acontece
quando os
problemas variam
seus padrões em
condições incertas?
Resolução de problemas determinísticos.
FORECASTING
&
NOWCASTING
13. 13
ONDE,OQUÊ,QUANDO?
Dados Estruturados
Dados Não
Estruturados
Armaz.Distribuído
Machine Learning, Data Mining
Classificação: SVM,
Naive-Bayes, logistic regression,
K-nearest
Regressão: Linear (lasso,
ridge), polinomial, etc
Clustering: K-means,
Spectral clustering, etc
Redução Dimensional:
Isomap, kernel aprox, PCA,
singular value decomposition
Árvores de Decisão
Random Forests
Análise de
Associação: Apriori, FP-
growth
Modelo Escondido de
Markov
Não supervisionados Supervisionados
ContínuoCategórico
ETL¹/MR²
DADOS ANÁLISE
1. ETL: Extract, Transform and Load
2. MR: Map Reduce
Data WareHouse,
Sistemas Legados, Main
Frame
Dispositivos, SMS,
Redes Sociais, Áudio,
Vídeo, Geo, Logs
DATA VISUALIZATION - SEARCH
Copiado livremente da Apresentação de José Damico – FEMA – Set, 2015
15. 15
O Profissional
O profissional big data (Data Scientist) deve ser multidisciplinar
(matemática, estatística, modelagem, reconhecimento de padrões,
IA, DW, ETL, High Performance Computation, etc).
17. 17
What is?
DevOps is a software development method that stresses
collaboration between Software Developers (Development) and
Information Technology professionals (Operations).
DevOps brings together Development and Operations to
Improve collaborations and productivity
Automate InfraStructure/Workflows
Measure app performance / code testing
18. 18
Meu slide resumo
ALM - Rational Team Concert, Microsoft Team Foundation, Jira
Versionamento/SCM: Git, SVN, CVS
Jenkins, Hudson, LuntBuild – CI – Integração continua
BugZilla
Jmeter, FindBugs, PMP, CheckStyle, Estilo de codificação.
SOAP UI
Linux / Windows – Ambiente do Cliente (versões)
Instrumentalizar – criar ferramentas
Geradores – Trabalho repetitivo
19. 19
History of Development and Operations
SoftwareSoftware
AdminAdmin
In the beginning, Development and Operations were
typically performed by the same person.
As time went on, that changed, and Development and
Operations separated.
20. 20
Solution production: The Legacy Model
In this model, Development
and Operations are separate,
and do not interact until the
end of the process.
In this model, Development
and Operations are separate,
and do not interact until the
end of the process.
21. 21
Pre-DevOps processes
Operations First:
» Charter / Fund: 2 weeks to 3 months
» Acquire Infrastructure (CAPEX): 6 weeks to 6 months
» Integrate Infrastructure (OPEX): 4 weeks to 8 weeks
» Dry runs, validation, acceptance testing (OPEX): 2 weeks to 4 weeks
Development Second:
» Workstation SCM
» SCM Dev
» Dev Test
» Test Stage, Prod
Issues:
» Patch Level / Security
» Demand, Scale to Max Peak
» Release schedules per Quarter / Year
» Governance Process alignment
» Consistency, Speed, Reliability
A new project can take between 6 and 18 months
before release 1.0 is in production!
A new project can take between 6 and 18 months
before release 1.0 is in production!
22. 22
Issues with the Legacy Model
Your customers find major defects.
» Major defects take a long time to fix.
You cannot do anything until you have everything.
» Legacy model leads to unique infrastructure dependencies.
People do not talk to each other.
» This leads to broken processes and overbearing governance.
Even if you are “agile,” the delivery process is still broken.
» Going live still takes as long as ever.
Any problems lead to finger pointing.
» Development and Operations are not able to operate as one team.
23. 23 Introduction to DevOps: Overview
The Application Deployment Model Must Change
DevOps change is as much a culture change as it
is a technology change.
DevOps change is as much a culture change as it
is a technology change.
24. 24 Introduction to DevOps: Overview
Solution production: The DevOps Model – who is involved?
Business function owners,
developers, and operations all are
involved early on in the process.
Business function owners,
developers, and operations all are
involved early on in the process.
25. 25 Introduction to DevOps: Overview
Solution production: The DevOps Model – swimlane view
The DevOps software
delivery process
introduces
transparency and
culture change.
The DevOps software
delivery process
introduces
transparency and
culture change.
26. 26
Where will you typically see DevOps solutions?
DevOps Solutions are commonly found in:
» Web sites (mobile, desktop)
» Mobile apps using backends as a server component for information resources
» Mobile application front-end development
DevOps solution are not found as commonly in:
» Typical packaged / platform-based services (BI Systems, ERP systems)
» Back-end Systems (ETL Systems, Systems of Record data sources)
DevOps lends itself to situations requiring rapid
changes in response to functionality and scale.
DevOps lends itself to situations requiring rapid
changes in response to functionality and scale.
27. 27
Key DevOps attributes
Startups, quick initiatives, exploring / prototyping:
» More systems of engagement involved
» Mobile on the front-end
MBaaS middleware to API catalog
Back end APIs or “mashable” APIs are used
Common unit of deployment (Operations) allows for:
» Scalability(demand management)
» Movement
» Recovery
» Feature deployment
Scripting / automation is critical
28. 28 Introduction to DevOps: Overview
Post-DevOps implementation
Infrastructure is more consistent and predictable.
Build / deploy processes are typically done outside of the
bastion of enterprise.
DevelopmentDevelopment OperationsOperations
QAQA
Frequent
Builds
Frequent
Builds
Automated
Builds
Automated
Builds
Known Deploy UnitsKnown Deploy Units Known
Accounting
Known
Accounting
Fast Setup
& Teardown
Fast Setup
& Teardown
Automated
Monitoring
Automated
Monitoring
Consistent
Scripts
Consistent
Scripts
Peer Code ReviewsPeer Code Reviews
Automated
Tests
Automated
Tests
Provisioning is fast.
On-demand provisioning is both good and bad at the
same time, from a financial standpoint.
29. 29 Introduction to DevOps: Overview
Sample DevOps maturity model
Define release with business
objectives
Measure to customer value
Define release with business
objectives
Measure to customer value
Optimize applications
Use enterprise issue
resolution procedures
Optimize applications
Use enterprise issue
resolution procedures
Fully Achieved Initiative GoalsPartially Achieved
Centralize requirements
management
Measure to project metrics
Centralize requirements
management
Measure to project metrics
Document objectives locally
Manage department
resources
Document objectives locally
Manage department
resources
Manage lifecycle artifacts
Schedule SCM integrations
and automated builds
Test following construction
Manage lifecycle artifacts
Schedule SCM integrations
and automated builds
Test following construction
Plan and manage releases
Standardize deployments
Plan and manage releases
Standardize deployments
Monitor resources
consistently
Collaborate DevOps informally
Monitor resources
consistently
Collaborate DevOps informally
Plan and source strategically
Dashboard portfolio
measures
Plan and source strategically
Dashboard portfolio
measures
Link lifecycle information
Deliver and build with test
Centralize and automate test
management
Link lifecycle information
Deliver and build with test
Centralize and automate test
management
Monitor using business and
end user context
Centralize event notification
and incident resolution
Monitor using business and
end user context
Centralize event notification
and incident resolution
Automate problem isolation
and issue resolution
Optimize to customer KPIs
continuously
Automate problem isolation
and issue resolution
Optimize to customer KPIs
continuously
Improve continuously with
development intelligence
Test continuously
Improve continuously with
development intelligence
Test continuously
Manage environments through
automation
Provide self-service build,
provision, and deploy
Manage environments through
automation
Provide self-service build,
provision, and deploy
Manage data and virtualize
services for test
Deliver and integrate
continuously
Manage data and virtualize
services for test
Deliver and integrate
continuously
Standardize and automate
cross-enterprise
Automate patterns-based
provision and deploy
Standardize and automate
cross-enterprise
Automate patterns-based
provision and deploy
Plan departmental releases
and automate status
Automated deployment with
standard topologies
Plan departmental releases
and automate status
Automated deployment with
standard topologies
30. 30 Introduction to DevOps: Overview
Example IBM deployment tooling
Line of
Business
IBM UrbanCode Build
Rational Team Concert
Rational Quality Manager
Rational Test Workbench
Rational Test Virtualization Server
SmartCloud Application Performance Management
Rational Focal Point
Rational Requirements Composer
SmartCloud Orchestrator
IBM Pure Application System
IBM UrbanCode Deploy
IBM UrbanCode Release
31. 31
DevOps aligns Operations and Development, stressing collaboration between the two areas.
DevOps is a pattern, not necessarily a product.
DevOps is as much about processes, culture, and tools, as it is about a methodology.
Think on how you can impact your customer’s business by helping them:
» Accelerate
» Fail fast (and recover!)
» Transform the business
» Meet business goals and objectives
DevOps - Summary
36. 36
Intro
Profissionais não se preparam antes que o problema aconteca
SQL
Indices
Where
Limite
Select com FOR READ
Threads
Paralelismo
37. 37
Exemplo simples
SELECT * FROM TABELA;
SELECT * FROM TABELA FETCH FIRST 10 ROWS ONLY
FOR READ ONLY WITH UR; (where? where? where?)
FETCH FIRST 10 ROWS ONLY -> quer dizer que é pra trazer
somente as 10 primeiras linhas encontradas
FOR READ -> fala pro banco que esta trazendo dados
somente para LEITURA, de forma superficial, quando você
não usa isso, o banco deixa CURSORES posicionados na
posição dos registros que você trouxe, ja preparando pra um
update/delete – isso, consome recursos.
WITH UR -> quer dizer “with uncommited records”, ou seja, já
traz os dados mais atualizados do banco, que sofreram um
update, mesmo não tendo sofrido commit ainda.
46. 46
Hybrid Mobile App
Built with HTML, CSS and JavaScript and is contained in a
native wrapper so that it can be installed on a mobile device.
This allows web developers to build mobile apps without
having to learn the native programming languages (e.g.,
Objective-C, Swift, Java).
It also means that you can have one codebase for all the
different platforms.
Frameworks...
Attention Points:
Maybe you can loose some native functions that use
specific OS/Hardware resources.
Maybe the performance can be lower than the native
development.
47. 47
Hybrid PROS
1- Easy to learn and Develop
2- Fast to create a prototype
3- Easy to find workforce
4- Easy to create a beautiful app
5- No need to learn native technology
6- Its faster to develop than Native (supposing that you are
good in both technologies)
7- Write once! (keep in mind that there are differences from
Platforms)
48. 48
Hybrid CONS
1- Performance can be lower (supposing that you are expert in
Native and Hybrid)
2- You can loose native functions specific for a OS/Device
49. 49
When go Native or Hybrid*
IF ( NEED HIGH PERFORMANCE or
NEED A LOT NATIVE RESOURCES or
NEED A NATIVE RESOURCE NOT SUPPORTED) {
goNative();
} ELSE {
goHibrid();
}
* this is what I THINK. Please, research before start your project. Many fail because
there are no planning, just code write.
51. 51
Apache Cordova
Apache Cordova is a platform
for building native mobile
applications using HTML, CSS
and JavaScript.
http://cordova.apache.org/
●Amazon Fire OS
●Android
●BlackBerry 10
●Firefox OS
●iOS
●Ubuntu
●Windows Phone 8
●Windows
●Tizen
53. 53
IONIC
Free and open source, Ionic offers a
library of mobile-optimized HTML,
CSS and JS components, gestures,
and tools for building highly
interactive apps. Built with Sass and
optimized for AngularJS.
http://ionicframework.com
54. 54
Installation
● Install Git - https://git-scm.com/
● Install Node Package Manager (npm) - https://nodejs.org/
● Install Cordova/Ionic - npm install -g cordova ionic
● Install Android SDK - http://developer.android.com/sdk/index.html
Create Environment Variable ANDROID_HOME , Example:
ANDROID_HOME=/home/julianom/Android/Sdk
● Install Java JDK 7
Create Environment Variable JAVA_HOME , Example (Linux):
JAVA_HOME=/opt/java
● Its a good idea put at your Path:
$JAVA_HOME/bin and $ANDROID_HOME/tools, Example (Linux):
PATH=$JAVA_HOME/bin:$ANDROID_HOME/tools:$PATH
● Install Genymotion (to emulate Android) - https://www.genymotion.com/#!/
● Create a virtual device at Genymotion and test it. I recommend a not powerful
device, like a Galaxy S3.
● Install a good Editor, I recommend Brackets (Its free) - http://brackets.io/
●
Create a user at Git Hub - https://github.com/
Take note about the users that you will create, you will use along the course.
56. 56
Hands on IONIC
ionic platform list → list available platforms
Android
●ionic platform add android
●ionic emulate android
or
●ionic run android
IOs
●ionic platform add ios
●ionic emulate ios
or
●ionic run ios
57. 57
Testing
Android
● We recommend test with Genymotion, by running the command “ionic run android”. Also,
if you plug your Android phone at your computer, the run command should install it at your
device.
IOs
● You can test in the simulator running “ioinic emulate ios”.
● You can open your project in at XCode by going to the folder platforms/PLATFORM of
your project, and run this as a normal XCode project. Also, you can publish your app from
here.
● Any time that you change content inside www folder, run the command “cordova prepare
ios” in order to update the XCode project. Have in mind that this will overwrite any change
that you have done at XCode.
● More about XCode/Cordova:
http://cordova.apache.org/docs/en/3.3.0/guide_platforms_ios_index.md.html
58. 58
All text and image content in this document is licensed under the Creative Commons Attribution-Share Alike 3.0 License
(unless otherwise specified). Adobe, Google, Apple, Apache, and other are registered trademarks. Their respective logos and
icons are subject to international copyright laws.
Thank you …
… for your dedication and patience!