En este webinar repasamos - mediante una demostración con el mercado de Real Estate de Los Angeles como ejemplo - cada uno de los cinco pasos que la plataforma de CARTO sigue para una toma de decisiones eficaz basada en los datos. Watch it now at: https://go.carto.com/carto-pasos-dato-toma-decisiones-recorded

1. CARTO en 5 pasos:
del dato a la toma de decisiones
SIGUE A @CARTO EN TWITTER

2. Cuáles de las siguientes descripciones
describen mejor tu trabajo:
● Desarrollador de Aplicaciones (backend o frontend)
● Analista de Datos
● Educación / Investigación
● Comercial / Ingeniería de Ventas / Soporte

3. CARTO en 5 pasos
Facilitadores para hoy
Jaime Sánchez Arias Jorge Sanz
Customer Success Engineer Solutions Engineer & Support Manager

4. CARTO en 5 pasos
As an organization, we have defined 5
steps that, together, create a holistic
Location Intelligence approach.
Our goal is to empower organizations as
they traverse each of these 5 steps.
The Complete
Journey

5. CARTO en 5 pasos
The Complete
Journey
1. Data
2. Enrichment
3. Analysis
4. Solutions
5. Integration

6. The Journey - Data & Enrichment
Augment any data
with demographic
data from around
the globe with easeData
Observatory
Data Services
API
Routing &
Traffic
Geocoding
ETL
Develop robust ETL
processes and
update
mechanisms so
your data is always
enriched
Mastercard Human
Mobility
POI

7. The Journey - Analysis
Bring CARTO maps and data into
your data science workflows and
the Python data science
ecosystem to work with Pandas,
PySal,PyMC3, scikit-learn, etc.
CARTOFrames
Machine learning embedded in
CARTO as simple SQL calls for
clustering, outliers analysis, time
series predictions, and geospatial
weighted regression.
Crankshaft
Use the power of PostGIS and our APIs to
productionalize analysis workflows in your
CARTO platform.
PostGIS by
CARTO
SQL API Python
SDK

8. The Journey - Solutions
Builder Dashboard
CARTO.js Airship CARTO VL
Analysis tools for creating
lightweight spatial data science
dashboards and sharing insights
with your team
Develop and build custom spatial
data science applications with a full
suite of frontend libraries

9. The Journey - Integration
Using CARTO’s APIs and
SDKs, connect your
analysis into the places
that matter most for
you and your team

10. Let’s apply the journey to
a real world business
question!

11. How can we analyze and
understand real estate sales
in Los Angeles?

12. Pains
1. “Disconnected experiences to consume data - it is broken into
separate tools, teams DBs, excels.”
2. “Limited developer time in our team.”
3. “Current data science workflow doesn’t have a geo focus. and Spatial
modeling is cumbersome because I have to export results to XYZ
tool in order to visualize and test my model effectively.”
4. “Having trouble handling and visualizing big datasets.“

13. Outline the Process
1. Integrate spatial data of past home sales and property locations
in Los Angeles county
2. Enrich the data with a spatial context using a variety of relevant
resources (demographics, mastercard transactions, OSM)
3. Clean and analyze the data, and create a predictive model for
homes that have not sold
4. Present the results in a Location Intelligence solution for users
5. Integrate and deploy the model into current workflows for day
to day use

14. 1. Data

15. Integrate LA Housing Data
The Los Angeles County Assessor's office provides two different datasets
which we can use for this analysis:
● All Property Parcels in Los Angeles County for Record Year 2018
● All Property Sales from 2017 to Present

16. 2018 Parcel Data

17. 2018 Parcel Data
Import this data into CARTO
using the Sync via Import API
or the COPY command via the
SQL API for efficient streaming
of long CSV data into CARTO.

18. Past Sales Data

19. Past Sales Data
Import this data into CARTO
using the built in ArcGIS
Connector, and Sync the
results to keep them up to data
as they are added.

20. CREATE TABLE la_join AS
SELECT s.*,
p.zipcode as zipcode_p,
p.taxratearea_city,
p.ain as ain_p,
p.rollyear,
p.taxratearea,
p.assessorid,
p.propertylocation,
p.propertytype,
p.propertyusecode,
p.generalusetype,
p.specificusetype,
p.specificusedetail1,
p.specificusedetail2,
p.totbuildingdatalines,
p.yearbuilt as yearbuilt_p,
p.effectiveyearbuilt,
p.sqftmain,
p.bedrooms as bedrooms_p,
p.bathrooms as bathrooms_p,
p.units,
p.recordingdate,
p.landvalue,
p.landbaseyear,
p.improvementvalue,
p.impbaseyear,
p.the_geom as centroid
FROM sales_parcels s
LEFT JOIN assessor_parcels_data_2018 p ON s.ain::numeric = p.ain
Clean and join the data on unique
identifier using a SQL statement in Builder
or via SQL API.

21. 2. Enrichment

22. Integrate LA Housing Data
Next we want to add spatial context to our housing data to
understand more about the areas around:
● Demographics
● Mastercard (Scores and Merchants) (Nearest 5 Areas)
● Nearby Grocery Stores and Restaurants
● Proximity to Roads

23. Demographics
Add total population and median income from the US Census via the
Data Observatory in CARTOframes.

24. query_obs = '''
CREATE TABLE obs_la_half_mile AS
SELECT *,
OBS_GetMeasure(the_geom, 'us.census.acs.B01003001','predenominated','us.census.tiger.census_tract', '2011 -
2015') as total_pop_2011_2015,
OBS_GetMeasure(the_geom, 'us.census.acs.B19013001','predenominated','us.census.tiger.census_tract', '2011 -
2015') as median_income_2011_2015
FROM la_half_mile
'''
# Apply JOB query
batch_job_obs= batchSQLClient.create(query_obs)
# Check status of the Batch Job
status_obs = check_status(batch_job_obs)
# cartodbfy table when table is created
if status_obs == 'done':
cc.query("SELECT cdb_cartodbfytable('obs_la_half_mile')")

25. Mastercard
Find the merchants and sales/growth scores in the five nearest block
groups to the home via Mastercard Retail Location Insights data
and SQL API

26. (
SELECT AVG(sales_metro_score)
FROM (
SELECT sales_metro_score
FROM mc_blocks
ORDER BY la_eval_clean.the_geom <-> mc_blocks.the_geom
LIMIT 5
) a
) as sale_metro_score_knn,
(
SELECT AVG(growth_metro_score)
FROM (
SELECT growth_metro_score
FROM mc_blocks
ORDER BY la_eval_clean.the_geom <-> mc_blocks.the_geom
LIMIT 5
) a
) as growth_metro_score_knn

27. Grocery Stores/Restaurants
Find the number of grocery stores and restaurants using
OpenStreetMap Data and the SQL API.

28. (
SELECT count(restaurants_la.*)
FROM restaurants_la
WHERE ST_DWithin(
ST_Centroid(la_eval_clean.the_geom_webmercator),
restaurants_la.the_geom_webmercator,
1609 / cos(radians(ST_y(ST_Centroid(la_eval_clean.the_geom)))))
) as restaurants,
(
SELECT count(grocery_la.*)
FROM grocery_la
WHERE ST_DWithin(
ST_Centroid(la_eval_clean.the_geom_webmercator),
grocery_la.the_geom_webmercator,
1609 / cos(radians(ST_y(ST_Centroid(la_eval_clean.the_geom)))))
) as grocery_stores

29. Roads
See if a home is within one mile of a major highway or trunk highway
using the SQL API and major roads from OpenStreetMap.

30. (
SELECT CASE WHEN COUNT(la_roads.*) > 0 THEN 1 ELSE 0 END
FROM la_roads
WHERE ST_DWithin(
la_eval_clean.the_geom_webmercator,
la_roads.the_geom_webmercator,
1609 / cos(radians(ST_y(ST_Centroid(la_eval_clean.the_geom)))))
AND highway in ('motorway', 'trunk')
) as highways_in_1mile

31. 3. Analysis

32. Analysis
The analysis for this project followed the following steps:
● Moran’s I Clusters & Outliers (Exploratory Data Analysis)
● Neighbor Homes Analysis (Spatial Feature Engineering)
● Predictive Modeling & Hyperparameter Tuning (using XGBoost)

33. Moran’s I
Using Moran’s I to evaluate spatial clusters and outliers via the PySAL
package, we can see these groupings and visualize them in
CARTOframes.

34. import esda
moran = esda.Moran_Local(sfr_ps.saleprice, W, transformation = "r")
lag = libpysal.weights.lag_spatial(W, sfr_ps.saleprice)
data = sfr_ps.saleprice
sig = 1 * (moran.p_sim < 0.05)
HH = 1 * (sig * moran.q==1)
LL = 3 * (sig * moran.q==3)
LH = 2 * (sig * moran.q==2)
HL = 4 * (sig * moran.q==4)
spots = HH + LL + LH + HL
spots
spot_labels = [ '0 Non-Significant', 'HH - Hot Spot', 'LH - Donut', 'LL - Cold Spot', 'HL -
Diamond']
labels = [spot_labels[i] for i in spots]
moran_to_carto = sfr_ps.assign(cl=labels, p_sim = moran.p_sim, p_z_sim = moran.p_z_sim)
moran_to_carto.head(2)

35. CARTO en 5 pasos
Moran’s I

36. CARTO en 5 pasos
Neighbor Analysis
Evaluate the attributes of
neighbor properties using
k-nearest neighbor spatial
weights in PySAL to perform
spatial feature engineering.

37. CARTO en 5 pasos
W = libpysal.weights.KNN.from_dataframe(neighbors, k=10, ids='plot_id')
# Given a target, get the average value for a selected list of KPIs of its neighbors
def evalNeighbors(data, target, neighbors):
sample_target_neighbors = neighbors[target]
if len(sample_target_neighbors) == 0:
pass
else:
d = data[data['plot_id'].isin(sample_target_neighbors)].loc[:,['median_income_2011_2015',
'merchants_in_1m', 'total_pop_2011_2015', 'restaurants', 'grocery_stores', 'highways_in_1mile',
'sale_metro_score_knn', 'growth_metro_score_knn']].mean()
d['plot_id'] = target
return d
# Generate the full Dataframe with properties and neighbors data
def buildDataframe(targetlist, **args):
newdf = pd.DataFrame(columns = ['plot_id', 'median_income_2011_2015', 'merchants_in_1m',
'total_pop_2011_2015', 'restaurants', 'grocery_stores', 'highways_in_1mile', 'sale_metro_score_knn',
'growth_metro_score_knn'])
for i in targetlist:
try:
newdf = newdf.append(pd.DataFrame(evalNeighbors(neighbors, i, W.neighbors)).T)
except KeyError:
pass
return newdf

38. CARTO en 5 pasos

39. CARTO en 5 pasos

40. CARTO en 5 pasos
Predictive Modeling
Using XGBoost we can use this data to create a regression model to
predict housing prices and push that data back to CARTO using
CARTOframes, never leaving the notebook environment.

41. CARTO en 5 pasos

42. CARTO en 5 pasos
Predictive Modeling
After hyperparameter tuning the model, we can reduce the Mean
Average Error down to $58,179.78.

43. CARTO en 5 pasos
Feature Importance

44. 4. Solutions

45. CARTO en 5 pasos
Solutions
To present the data and predictive analysis, both on data from the
model that has a sales price and for homes that have not sold, we
can develop a location intelligence application to showcase these
results.

46. CARTO — Turn Location Data into Business Outcomes
Application Development
Using CARTO VL we can
quickly visualize our data
layers and add a user interface
with Airship, in less than 400
lines of code.

47. CARTO — Turn Location Data into Business Outcomes
Los-Angeles
Prediction
Explorer

48. 5. Integration

49. CARTO — Turn Location Data into Business Outcomes
Application Development
Deploy the model via a Python
based API and sync to CARTO
data via the Python SDK to
perform on the fly predictions
for specific properties.

50. CARTO — Turn Location Data into Business Outcomes
Los Angeles
Property Sales
Prediction
App

51. CARTO en 5 pasos
Other Use Cases
Predicting revenue from different physical retail locations
Identify clusters and groups of specific patterns to optimize
activities such as sales outreach or site selection
Classify property types or buying patterns in a city
Review spatial feature importance for site performance, and
modify models using different spatial

52. ¡Muchas gracias!
¿Preguntas?
Solicita una demo en CARTO.COM
Jaime Sánchez Arias
Customer Success Manager // jsanchez@carto.com
Jorge Sanz
Solutions Engineer & Support Manager // jsanz@carto.com