杨磊将分享他的部门如何开始使用 Elasticsearch 以及如何将其应用于中间件用例。

1. 1
Leon Yang
April 10th 2019, Senior Architect, Lenovo
ELK Practice in
Enterprise Integration

2. 2
Agenda Slide
目录
About Us1
Using the Elastic Stack in Enterprise Integration3
Future Projects with IoT4
Key Business Values from using Elastic2
5 Q&A

3. 3
About us
0
1
Integration Product DevOps
Lenovo integration / middleware products development.
A2A Integration
Lenovo business application / process integration
B2B Integration
Lenovo business partner / supply chain integration
Data Integration
Lenovo Mater Data integration
0
2
0
3
0
4
Leon Yang （杨磊）
Senior Architect, Lenovo.
关于我们

4. 4
Challenges Faced Before Using Elastic
Cost Efficiency Security
High
Performance
Indexing
• Limited Team Resources
(Finance and People)
• Large-Scale Log Monitoring
• Support Global Functions
• Upper management
concerns
• Complex to separate data
permission by data owner
and data usage
• MySQL cannot support
large scale data persistency
• Cassandra and HBase are
very complex to use
• High Cost of Operation

5. 5
Key Business Values of Using Elastic
• Cost Efficiency
‒ It took only two days to complete the Elastic Stack deployment and generate all the required reports in the
perfect style. With the Elastic Stack, our small team is can manage our middleware.
• Elastic Security Feature
‒ Elastic features, part of the commercial license, provides index level and index-field level security check to
easy separate data by ownership or usage
• High Performance Indexing
‒ Elasticsearch provides rapid indexing and the cluster nodes are easy to operate and maintain.
• Rich Plugins for Data Collection and Publishing
‒ Lenovo has 1000+ integration scenarios on data exchange or process collaboration between hundreds of
applications or services, on premises and cloud, including Lenovo internal and Lenovo external partners.

6. 6
Efficiently implement middleware
system exception monitoring and
log analytics for more than 100
cluster nodes with 0.5 billion
transaction logs per day.
Leon Yang, Senior Architect, Lenovo

7. 7
80
%Copy paste fonts. Choose the
only option to retain text.
• Supporting Lenovo integration system monitoring
• Supporting Lenovo API transaction log statistics & monitoring
API Service Statistics & Monitoring
100%
Successfully delivered
log statistics and
monitoring
ELK Practice in Enterprise Integration
1

8. 8
1 2
Collection Transform
3
Indexing
4
Dashboard
Filebeat Logstash Elasticsearch Kibana
Application Log Files
Using the Elastic Stack in Enterprise Integration
Service Transaction Statistics

9. 9
1 2
Collection Transform
3
Indexing
4
Dashboard
Logstash Elasticsearch Kibana
100+ cluster nodes
Filebeat
Business Application
Middleware
Middleware
Input: JDBC
Input: Kafka
Input: filebeat
Using the Elastic Stack in Enterprise Integration
Service Transaction Log Monitoring & Alarm

10. 10
Using the Elastic Stack in Enterprise Integration

11. 11
Using the Elastic Stack in Enterprise Integration

12. 12
Using the Elastic Stack in Enterprise Integration

13. 13
Using the Elastic Stack in Enterprise Integration

14. 14
80
%Copy paste fonts. Choose the
only option to retain text.
• Exploring Lenovo stream processing integration and
data publish
Stream Processing Integration
80%
Incremental data
integration and
analytics.
2
Using the Elastic Stack in Enterprise Integration

15. 15
LogstashFilebeat Stream Processing Cluster
Logstash Elasticsearch
Business
Application
A
Logstash
API GatewayData Warehouse
1
Data Collection
2
Stream Data Processing
3
Data Service Provision
Business
Application
B
4
Data Consuming
Restful
API
Zookeeper, Topic
Business
Application
C
Incremental data computing and integration
Using the Elastic Stack in Enterprise Integration

16. 16
80
%Copy paste fonts. Choose the
only option to retain text.
• Exploring IoT big data integration and analytics
• Part of the Lenovo Strategy
IoT Data Integration
50%
IoT data integration and
analytics 3
Using the Elastic Stack in Enterprise Integration

17. 17
Data Collection and Publish
Supporting various input and output plugins such as file,
jdbc, kafka, http, tcp and etc.
Data Analytics & Statistics
Providing rich visualization types for easily generating
report.
Data Indexing and Service Provision
With strong indexing engine and fantastic API accessing mode.
Device Health Check
High efficient log collection and indexing to support rapid status check
for devices
IoT Data Integration
Using the Elastic Stack in Enterprise Integration
IoT Elastic
Stack
New
Ideas

18. 18
Thank You
Q&A
leonyaa@sina.com