Practical data science project groups energy supplier accounts using ML and NLP
•
3 gefällt mir•969 views
20min talk given at PyData London 2014 A client in the energy sector wanted to create predictive behavioural models of business customers at the company level, but the CRM data was messy, often containing several sub-accounts for each business, without any grouping identifiers, and so aggregation was impossible. In this talk I describe a short project where we used text mining, a handful of unsupervised learning techniques and pragmatic use of human skill, to identify the true company level structures in the CRM data.
Empfohlen
Empfohlen
Weitere ähnliche Inhalte
Andere mochten auch
Andere mochten auch (13)
Ähnlich wie Practical data science project groups energy supplier accounts using ML and NLP
Ähnlich wie Practical data science project groups energy supplier accounts using ML and NLP (20)
Mehr von Jonathan Sedar
Mehr von Jonathan Sedar (8)
Kürzlich hochgeladen
Kürzlich hochgeladen (20)
Practical data science project groups energy supplier accounts using ML and NLP
- 1. ON I EDIT G NIN T LIGH A practical data science project
- 2. ● A CRM dataset (100k business accounts) belonging to a national energy supplier ● A knotty problem: multiple accounts per company, without any grouping ids ● How can we to find groups of accounts (larger company structures), using just the CRM data? ● Machine Learning (ML) and Natural Language Processing (NLP) tools and techniques in Python. ● Import: Scikit Learn and TextBlob (NLTK & Pattern)
- 3. Company ID Account name Contact name Premises address lines 1 - 4 Billing address lines 1 - 4 1 Bob’s Pizza Big Bob 5 High St, Wexford 5 High St, Wexford 1 Bob’s Pizza Big Bob Temple Bar, D2 5 High St, Wexford 1 Mike’s Kebabs Mad Mike 3 Upper St, Dublin 5 High St, Wexford 2 Mark’s Kebabs Mild Mark 8 Upper St, Dublin Main St, Waterford 3 Fred’s Falafel Fat Fred 9 Henry St, Cork 9 Henry st, cork 3 Fred Fallafell Freddie Bridges St, Galway Henrys St, Cork This crucial bit of info groups the separatelyrecorded accounts into companies… and was missing from the dataset … x100,000
- 4. Account holder Business name Premises address Billing address ... Cleaned, parsed, tokenised text strings NTLK.PunktTokeniser x100,000 sklearn.TfidfVectorizer SVD N-D MATRIX TF-IDF 2-D MATRIX sklearn.TruncatedSVD
- 5. 1. Suggest similar accounts to be grouped 2. Human validation & verification 3. Incorporate & propagate valid groupings sklearn.MiniBatchKMeans sklear.AffinityPropagation sklearn.RadiusNeighborsClassifier
- 6. ● A very quick turnaround from raw data to tagged companies to 93% accuracy ● ~40% of accounts found to belong to a company, ~3.5 accounts per company ● NLP toolkits and scikit-learn allowed rapid development and testing of solution ● Incorporated human identification at critical stages: no ML problem is an island