用户画像介绍

2. &
Olap
CRM

3. &
PM
Olap

4. Lambda & T+1
Spark Stack & ElasticSearch
Historical
Hbase
http://git.lianjia.com/bigdata/pr
ofiling

5. Spark streaming
Vs. Storm, DStream & Micro Batch
Window Operation
Kafka Direct API & At least once Semantic
DataFrame & Sql On Stream
Fault Tolerance
Streaming job track url http://10.10.35.14:8089

6. Olap
Elasticsearch HBase
KV / / Olap
schema free,
Elastic
In Memory, & BitMap (Doc Value),
LSM segments refresh / flush / merge
Sharding & Replication, HA,
Sql ( )
Groovy Script
( ), Rollover
Nested & Parent-Child
Rolling Upgrade & Index Alias
UI
http://10.10.35.14:5601/app/marvel
http://10.10.35.14:5601/app/kibana
http://10.10.35.14:5601/app/sense
http://10.10.35.14:9200/_plugin/sql/

7. Spam

8. ...
TF/IDF, or customized

9. DMP , retargeting
Deterministic & Probabilistic
( uuid dict )
3 3 dict
3 4 dict
3 5 dict

10. MLlib
(RDD / Dataset)
pileline
Vs. MPI

11. 1.
Features of Entity A Features of Entity B Crossed (High Level) Features 0 / 1
2. ( )
Categorical:
Binary:
Crossed: ( / )
( ) ip

12. Model Precision (thd - 0.9) Recall (thd - 0.9) FP (thd - 0.9)
LR (L2) 0.76 0.53 4
DT (Gini, MaxDepth: 5,
MaxBins: 32)
0.8725 0.8872 55
GBDT (MSE loss, Trees:
30, MaxDepth: 4)
0.8768 0.7960 13
: 5070, 1549 ...
IP
xgboost4J

13. CRM
Churn Prediction:
Will to Buy / Sell: ( ) & ( )
(ucid )
— — — —
( )
( )
( )
3 - 4 59%
3 - 5 52%
3 - 6 45%
3 - 7 41%

14. &
(
)

15. End