Automatic Forecasting using Prophet, Databricks, Delta Lake and MLflow
•
1 gefällt mir•831 views
As Atlassian continues to scale to more and more customers, the demand for our legendary support continues to grow. Atlassian needs to maintain balance between the staffing levels needed to service this increasing support ticket volume with the budgetary constraints needed to keep the business healthy – automated ticket volume forecasting is at the centre of this delicate balance
Empfohlen
Empfohlen
Weitere ähnliche Inhalte
Was ist angesagt?
Was ist angesagt? (20)
Ähnlich wie Automatic Forecasting using Prophet, Databricks, Delta Lake and MLflow
Ähnlich wie Automatic Forecasting using Prophet, Databricks, Delta Lake and MLflow (20)
Mehr von Databricks
Mehr von Databricks (20)
Kürzlich hochgeladen
Kürzlich hochgeladen (20)
Automatic Forecasting using Prophet, Databricks, Delta Lake and MLflow
- 2. PERRY STEPHENSON | SENIOR SOFTWARE ENGINEER | ATLASSIAN Automatic Forecasting Creating a robust, fault-tolerant, auditable and reproducible forecasting pipeline
- 3. Empowered End User Building pipelines with end-user tooling means I can Get S#!* Done™ all by myself Reproducibility Lots of people talk about it. I did something about it. I ❤ Databricks It makes my life easier and makes me look good in front of colleagues and managers Why am I speaking? Perry Stephenson ATLASSIAN
- 6. Forecasting a special case for Machine Learning because you need to retrain every time you make a CONTEXT
- 8. Holdout
- 9. Holdout
- 10. End-to-End Pipeline Create a robust, fault- tolerant, auditable and reproducible forecasting pipeline Backtesting Evaluate the whole pipeline using backtesting. Prioritise stability Ensuring Fresh Forecasts
- 11. MEME TRANSLATION Update Dataset Train Models Score Models Not Production Production Update Dataset Train Models Score Models Production
- 12. Support Ticket Forecasting Pipeline Scheduler (monthly) Create Modelling Dataset Update Modelling Dataset Train Forecast Models Score Forecast Models 1 2 3 4
- 14. Delta Lake Reproducible Training Data
- 15. Support Ticket Forecasting Pipeline Scheduling Create Modelling Dataset Update Modelling Dataset Train Forecast Models Score Forecast Models 1 2 3 4
- 16. Support Ticket Forecasting Pipeline Create Modelling Dataset create table zone_myteam.my_table using delta as select * from … … … 1
- 17. Update Modelling Dataset Support Ticket Forecasting Pipeline - Runs every month to update the dataset - Merges changes and retains history by leveraging Delta Lake 2
- 18. MORE INFO ABOUT THE PIPELINE DELTA LAKE THIS TALK >>>
- 19. FAKE NEW S
- 20. DELTA LAKE GIVES YOU VERSIONED TABLES FOR REPRODUCIBLE DATA SCIENCE
- 22. merge into zone_myteam.my_table as existing using my_latest_data as new on existing.date_name = new.date_name and existing.platform = new.platform and existing.customer_region = new.customer_region when matched then update set * when not matched then insert *
- 25. Update Modelling Dataset Support Ticket Forecasting Pipeline - Creates a new version every time it runs - Latest version is always the most accurate - Can recover any previous version of the training dataset
- 27. MLflow Tracking Reproducible Forecasting Models
- 28. Support Ticket Forecasting Pipeline Scheduling Create Modelling Dataset Update Modelling Dataset Train Forecast Models Score Forecast Models
- 29. Support Ticket Forecasting Pipeline Train Forecast Models - Written in R, using Facebook Prophet - Trains a model for every ticket grouping, stores model + metadata in MLflow - Takes an argument for “forecast_date” to allow backfilling, defaults to month end
- 30. MLflow THIS TALK >>> MORE INFO ABOUT THE PIPELINE
- 38. LOGGING TO MLFLOW mlflow_client_static <- mlflow_client() run_info <- mlflow_start_run(experiment_id = 611628) … … … mlflow_log_param(key = “forecast_date", value = forecast_date) mlflow_log_param(key = “platform", value = platform) … … … mlflow_log_artifact(path = 'prophet_model.rds', artifact_path = ‘model’) … … … mlflow_end_run(run_id = run_info$run_uuid, client = mlflow_client_static)
- 44. Score Forecast Models Support Ticket Forecasting Pipeline - Reads from MLflow (two passes to recover params), builds an execution plan - Scores each forecast, and prepares aggregates for consumption - Appends/overwrites results in our data lake, with history maintained using Delta - Includes MLflow links for every row in the forecast table
- 45. READING FROM MLFLOW required_forecasts <- mlflow_list_run_infos(experiment_id=611628) for (i in 1:nrow(required_forecasts)) { run_details <- mlflow_get_run(required_forecasts$run_uuid[i]) run_params <- run_details$params[[1]] … … … # not shown: unpack params and score model # not shown: weekly/monthly/quarterly aggregations # not shown: add MLFlow URL } # not shown: union and upload all forecasts at once
- 46. Score Forecast Models Support Ticket Forecasting Pipeline - Uploads all results to in-memory temporary table - Deletes all records from the final table with the same forecast_date - Merges changes in to forecast output table
- 47. Notebook Workflows Flexible and Reliable Execution
- 48. NOTEBOOK WORKFLOWS THIS TALK >>> MORE INFO ABOUT THE PIPELINE
- 55. WIDGETS (AKA NOTEBOOK ARGUMENTS)
- 58. Review
- 59. Scheduling Update Modelling Dataset Train Forecast Models Score Forecast Models 15th day of the month Delta table MLflow
- 61. ULTIMATE PRETTY GOOD REPRODUCIBILITY Forecast Table Forecast Row MLflow URL Model Binary Table Name Table Version Training Data
- 64. The Databricks platform supports the chaos during early development, and provides pathways to CONCLUSION