2019 WIA - Data-Driven Product Improvements
•Als PPTX, PDF herunterladen•
0 gefällt mir•101 views
2019 Women in Analytics Conference Amanda Gadrow womeninanalytics.com
Empfohlen
Empfohlen
Weitere ähnliche Inhalte
Ähnlich wie 2019 WIA - Data-Driven Product Improvements
Ähnlich wie 2019 WIA - Data-Driven Product Improvements (20)
Mehr von Women in Analytics Conference
Mehr von Women in Analytics Conference (10)
Kürzlich hochgeladen
Kürzlich hochgeladen (20)
2019 WIA - Data-Driven Product Improvements
- 1. Data-Driven Product Improvement Amanda Gadrow Director of Quality Assurance and Support RStudio amanda@rstudio.com Women in Analytics, March 2019 ajmcoqui ajmcoqui
- 2. Problem: Know Thy User
- 3. Problem: Know Thy User
- 4. Problem: Know Thy User
- 5. Problem: Know Thy User
- 6. Problem: Know Thy User
- 7. Problem: Know Thy User
- 8. Problem: Know Thy User
- 10. 57%* of the company is Engineering RStudio Strategy 56%* of Engineering is open source * As of December 2018
- 11. Customer experience data RStudio Community
- 12. Customer experience data RStudio Community GitHub Repositories
- 20. Required outcomes Reproducibility Scalability Measurability
- 21. Tools as Data Sources
- 22. Tools as Data Sources
- 23. Tools as Data Sources
- 25. Data Science Toolchain Exploration Deep analysis Modeling Visualization Communication
- 26. Data Science Toolchain Exploration Deep analysis Modeling Visualization Communication http://r4ds.had.co.nz/explore-intro.html
- 27. Exploration
- 28. Exploration
- 29. Exploration
- 30. Deep Analysis
- 31. Deep Analysis
- 32. Deep Analysis
- 33. Modeling
- 34. Visualization
- 35. Visualization
- 36. Visualization
- 37. Communication
- 38. Communication
- 39. Communication
- 40. Communication
- 41. Communication
- 42. Communication
- 43. Communication
- 44. Communication
- 45. Communication
- 46. Communication
- 53. Improvement - Documentation & Process
- 54. Improvement - Documentation & Process Documentation
- 55. Improvement - Documentation & Process Documentation Support
- 56. Improvement - Documentation & Process Documentation Support
- 57. Improvement - Documentation & Process Documentation Support
- 58. Improvement - Documentation & Process Documentation Support
- 59. Improvement - Documentation & Process Documentation Support Customer Success
- 60. Measure
- 61. Measure
- 62. Measure Offline license activation tickets Offline license application usage
- 63. Measure
- 64. Measure
- 65. Measure
- 66. Measure
- 67. Measure
- 68. Measure
- 69. Measure
- 70. How It Is Done
- 71. How It Is Done http://r4ds.had.co.nz/explore-intro.html
- 72. How It Is Done
- 73. How We Think about Data Science
- 74. How We Think about Data Science
- 75. Import & Tidy Import Tidy readr tidyr haven readxl googlesheets sparklyr odbc httr library(httr) library(jsonlite) response <- GET(zendesk_url, authenticate(user, pwd)) text <- content(req, as = "text") fromJSON(text) support_tickets <- response$tickets
- 76. Import & Tidy Import Tidy readr tidyr haven readxl googlesheets sparklyr odbc httr library(httr) library(jsonlite) response <- GET(zendesk_url, authenticate(user, pwd)) text <- content(req, as = "text") fromJSON(text) support_tickets <- response$tickets
- 77. Import & Tidy Import Tidy readr tidyr haven readxl googlesheets sparklyr odbc httr library(httr) library(jsonlite) response <- GET(zendesk_url, authenticate(user, pwd)) text <- content(req, as = "text") fromJSON(text) support_tickets <- response$tickets
- 78. Import & Tidy Import Tidy readr tidyr haven readxl googlesheets sparklyr odbc httr library(httr) library(jsonlite) response <- GET(zendesk_url, authenticate(user, pwd)) text <- content(req, as = "text") fromJSON(text) support_tickets <- response$tickets
- 79. Understand: Transform Transform dplyr stringr lubridate forcats purrr magrittr tidyverse library(dplyr) library(stringr) library(lubridate) # pull Product out from custom_fields prod_matrix <- lapply(support_tickets$custom_fields, function(x) x[x$id == xyz,"value"]) %>% unlist %>% matrix prod_df <- data.frame(product = prod_matrix[,1], stringsAsFactors = FALSE) support_tix <- cbind(support_tickets, prod_df)
- 81. Understand: Transform Transform dplyr stringr lubridate forcats purrr magrittr tidyverse library(dplyr) library(stringr) library(lubridate) # pull Product out from custom_fields prod_matrix <- lapply(support_tickets$custom_fields, function(x) x[x$id == xyz,"value"]) %>% unlist %>% matrix prod_df <- data.frame(product = prod_matrix[,1], stringsAsFactors = FALSE) support_tix <- cbind(support_tickets, prod_df)
- 82. Understand: Transform Transform dplyr stringr lubridate forcats purrr magrittr tidyverse library(dplyr) library(stringr) library(lubridate) # pull Product out from custom_fields prod_matrix <- lapply(support_tickets$custom_fields, function(x) x[x$id == xyz,"value"]) %>% unlist %>% matrix prod_df <- data.frame(product = prod_matrix[,1], stringsAsFactors = FALSE) support_tix <- cbind(support_tickets, prod_df)
- 83. Understand: Transform Transform dplyr stringr lubridate forcats purrr magrittr tidyverse # Status: convert chr to factor support_tix$status <- ordered(support_tix$status, levels = c("new", "open", "pending", "hold", "solved", "closed")) # Dates: convert to actual dates support_tix$created_month <- month(strptime(support_tix$created_at, format = "%Y-%m- %dT%H:%M:%SZ"), label = TRUE, abbr = FALSE)
- 84. Understand: Transform Transform dplyr stringr lubridate forcats purrr magrittr tidyverse # Status: convert chr to factor support_tix$status <- ordered(support_tix$status, levels = c("new", "open", "pending", "hold", "solved", "closed")) # Dates: convert to actual dates support_tix$created_month <- month(strptime(support_tix$created_at, format = "%Y-%m- %dT%H:%M:%SZ"), label = TRUE, abbr = FALSE)
- 85. Understand: Transform Transform dplyr stringr lubridate forcats purrr magrittr tidyverse # Status: convert chr to factor support_tix$status <- ordered(support_tix$status, levels = c("new", "open", "pending", "hold", "solved", "closed")) # Dates: convert to actual dates support_tix$created_month <- month(strptime(support_tix$created_at, format = "%Y-%m- %dT%H:%M:%SZ"), label = TRUE, abbr = FALSE)
- 86. Understand: Visualize Visualize shiny ggplot2 plotly dygraphs highcharter plumber library(shiny) library(ggvis) <Not shown: Filter data based on user input> <see shiny.rstudio.com for details> # Number of tickets created, by week created_count <- reactive({ cc <- support_tickets() %>% select(id, week) %>% group_by(week) %>% summarize(count = n()) %>% mutate(action = "created") }) created_count %>% ggvis(~week, ~count, stroke = ~action) %>% add_axis("x", title = "Week") %>% add_axis("y", title = "# Tickets") %>% add_legend("stroke")
- 87. Understand: Visualize Visualize shiny ggplot2 plotly dygraphs highcharter plumber library(shiny) library(ggvis) <Not shown: Filter data based on user input> <See shiny.rstudio.com for details> # Number of tickets created, by week created_count <- reactive({ cc <- support_tickets() %>% select(id, week) %>% group_by(week) %>% summarize(count = n()) %>% mutate(action = "created") }) created_count %>% ggvis(~week, ~count, stroke = ~action) %>% add_axis("x", title = "Week") %>% add_axis("y", title = "# Tickets") %>% add_legend("stroke")
- 88. Understand: Visualize Visualize shiny ggplot2 plotly dygraphs highcharter plumber library(shiny) library(ggvis) <Not shown: Filter data based on user input> <See shiny.rstudio.com for details> # Number of tickets created, by week created_count <- reactive({ cc <- support_tickets() %>% select(id, week) %>% group_by(week) %>% summarize(count = n()) %>% mutate(action = "created") }) created_count %>% ggvis(~week, ~count, stroke = ~action) %>% add_axis("x", title = "Week") %>% add_axis("y", title = "# Tickets") %>% add_legend("stroke")
- 89. Understand: Visualize Visualize shiny ggplot2 plotly dygraphs highcharter plumber library(shiny) library(ggvis) <Not shown: Filter data based on user input> <see shiny.rstudio.com for details> # Number of tickets created, by week created_count <- reactive({ cc <- support_tickets() %>% select(id, week) %>% group_by(week) %>% summarize(count = n()) %>% mutate(action = "created") }) created_count %>% ggvis(~week, ~count, stroke = ~action) %>% add_axis("x", title = "Week") %>% add_axis("y", title = "# Tickets") %>% add_legend("stroke")
- 92. Communicate
- 93. Communicate
- 94. Communicate
- 97. Recap
- 98. Recap
- 99. Recap
- 100. Recap
- 101. Recap
- 103. Thank you Amanda Gadrow Director of Quality Assurance and Support RStudio amanda@rstudio.com Women in Analytics, March 2019 ajmcoqui ajmcoqui Image Sources: ● RStudio for topic-specific images and screenshots ● All company logos are owned by the trademark holders (Zendesk, Salesforce, Marketo, wyday LimeLM, R) ● The black hole image is from Science News ● Clipart Max is the source of the rest of the graphics
Hinweis der Redaktion
- Good afternoon. I’m excited to be talking to you today about a topic that is near and dear to my heart. I *am* the Director of Quality Assurance and Support at RStudio, but I am also a software engineer turned data scientist, so I’m going to talk to you not only about *what* we did to drive product improvement through analysis, but also *why* we did it, and *how* you can use similar strategies to drive the kinds of improvement and growth that we have seen in the past couple of years. Today I’m going to tell you the story of how we at RStudio are doing this, and how you can use the same techniques to leverage the data you already have to effect meaningful change in your organization with the help of some powerful and easy-to-use tools.
- When I joined RStudio three years ago, we had a problem...
- ...no one knew very much about how our commercial products were being used, or where our customers may be struggling. We wanted this information for several reasons:
- so we could ensure that our customers had the best possible experience
- to foster customer loyalty by providing excellent products that are easy to use
- to educate users on best practices
- to make it easier to expand their usage
- and to promote the R language and community with the best tools we can build. To do all that, we needed to understand where people were getting hung up so we could remove those roadblocks. We were collecting data in the form of support tickets and Sales information, but we weren’t looking at that data in aggregate once we’d resolved the immediate problem, so we didn’t have a good idea of how best to improve our products to meet our goals based on what our customers were telling us as a whole.
- To give you a little background on our company, RStudio is a small software company based in Boston that builds tools around the R programming language. We have a mission to provide the most widely used open source and enterprise-ready professional software for the R statistical computing environment. These tools further the cause of equipping everyone, regardless of means, to participate in a global economy that increasingly rewards data literacy. RStudio open-source and enterprise-ready, professional software combines robust and reproducible data analysis with tools to effectively share data products.
- Our goal is to empower users to be productive with R, and to accomplish our goal, we invest a huge proportion of our resources into open-source language and tool development. 57% of the company is Engineering, and 56% of *that* is working on open-source assets. Our core mission is essentially “to make R better” so more people can get value out of their own data. Our open-source tools, including the R packages we maintain and the free version of the development environment, are accessible to everyone, and they create demand for our commercial products, which we then use to fund further improvements to the open-source tools. We are committed to promoting the inclusive, collaborative, and transparent community that surrounds the R language, so everyone can benefit from meaningful analysis in an increasingly data-driven world. And as we’ve been working toward this goal, we have learned quite a bit about our users and customers by using our own tools.
- When we started this project, we knew plenty about how our open-source products were being used, since our users were pleasantly vocal in our Community forum...
- ... and in issues they filed in our public GitHub repositories, and everyone in the company has free and open access to this feedback. But ironically, we knew far less about the user experience from the users who were actually paying us for the software, since their issues were obscured behind the third-party tools we were using to track them. These tools are fantastic for solving immediate problems, but less so for aggregating the data and learning how we might make the user experience better.
- We were collecting user feedback...
- ...storing it in several disparate data sources...
- ...and then letting it fall into a black hole.
- Except the data wasn’t really falling into a black hole, it was just being used inconsistently. Our support team reported trends as we found them, and our development teams were, and are, quite active with support tickets and issues that come in from the field through Sales. They react quickly when bugs are found, and the “team leads” keep an eye on trends in customer issues. But this was done ad hoc and manually, not in an organized and strategic way. We reacted well, but the truth was, nothing we did was repeatable, and we had no way of telling whether the improvements we implemented at the time were working.
- What we needed was a way to analyze our own data...
- ...such that it was reproducible, so we could run the same analysis at a later date and be confident that the results were consistent and relevant.
- scalable, so as the business grows, the analyses are able to handle larger amounts of data, and be easily adapted to changes in the business, expansion in the group of stakeholders, and the types of issues that were coming in.
- And we needed the outcome to be measurable. We needed to be able to tell whether a particular change in the product, the documentation, or the support process was having the intended effect. Until then, we didn’t have an organized way to get meaningful, actionable information from the data, the kind of information that would help us make business decisions, improve our products, and provide a better user experience for our customers. So we decided to take our own advice, and build a tool that would do exactly that.
- The data we have comes from multiple sources...
- including Zendesk for customer support ticket data, Salesforce for customer account data, Marketo for customer communication data, and LimeLM for license management. There was no good way to bring all these data together, so we built one ourselves...
- ... by importing the data into R using these tools’ APIs, their application programming interfaces.
- Once we got the data into R, we were able to do wonderful things with it by...
- ...putting it through our data science toolchain to explore, analyze, model, visualize, and communicate insights.
- This is how we view data science, as a continuous improvement loop where, after importing and tidying the data for analysis, you go through a cycle of transformation, visualization, and modeling until you have the business intelligence you need. Once you come to a good understanding of the data, you can then communicate it to the stakeholders, who can use it to make decisions. In our case, we are using it to make decisions about product improvement and user experience.
- Once the data are in R, it is relatively easy to build scripts and applications that allow you to run analyses and communicate the results. Interactive Shiny applications are great for discovering meaningful trends in the data. User controls allow people to explore the data based on the values of various parameters and filters, providing a powerful way to visualize specific scenarios, and potentially make decisions based on that data. This is just an example application using financial data, but when we used similar techniques on our own customer data, we found some very interesting things.
- For instance, when we explored the support ticket data via a Shiny application, we discovered that the bulk of our requests come in the mid-afternoon, Eastern Time....
- ...which drove our decision to hire our next two support engineers in the Mountain and Pacific time zones to increase our coverage and shorten response times for customers filing tickets late in the day.
- We also built applications to help us understand complicated data from multiple sources. In this app, we’re pulling data from our support ticketing system, our sales information database, and our licensing vendor to identify customers who exhibit risk factors...
- ...like lower than ideal Net Promoter Scores, underutilized licenses, and high-priority support requests. The app has a number of interactive elements that allow us to filter and transform the data...
- For instance, we can decide how low the Net Promoter Score should be for the threshold of risk, filter by account owner, and exclude accounts that have no active support tickets. Our customer success representatives use this app to identify accounts that may need attention soon, so they can proactively reach out to customers and focus on getting them successful as quickly as possible.
- In another report, my colleague Andrie de Vries took some of the same data and created a model to predict customer churn, based on the RFM method for analyzing customer value: https://connect.rstudioservices.com/connect/#/apps/169/access/116 RFM is an initialism for Recency, Frequency, and Money, and it’s a method of aggregating sales data into a single number that attempts to identify high-value customers. In this model, we’re looking at the confluence of Sales data on the X axis and Support ticket data on the Y axis to learn what types of customers tend to churn, so we can apply the model to incoming customer data and try to predict who is likely to not renew their licenses. Once identified, we proactively reach out to them to provide resources and shorten their time to value with our products. This model is instrumental to that effort, and he created it with entirely open-source tools.
- The importance of visualization cannot be overstated when the goal is to influence product decisions. I am fortunate enough to work at a company where the value of data, and effective techniques for making use of it, are well-understood, but it’s still much easier to convince someone to fix an issue when you can show them a pretty picture backed by meaningful data. This is a dashboard we built to explore trends in support ticket data...
- ...based on date, product, and various filters. This tab shows the sheer volume of tickets....
- ...but other tabs dive deeper into who are filing support tickets, why they’ve reported an issue and when they filed it. This information drives decisions based on what types of customers are asking questions, what products and features we should focus our resources on, and how we should staff the support organization to make sure response times are short.
- Once we have done some data analysis, we need a good way to communicate the results to others in the company, so we can make informed decisions about improvements to the product, and where to direct our business resources. We need to get this data into the hands of the people who can use it in meaningful ways. This work is for naught if the right people don’t get their eyes on it. So to that end, we created a variety of assets for distribution. This is a Shiny dashboard designed to visualize the results of our user survey after the 2019 RStudio conference. It automatically pulls the most recent data and presents it in an R Markdown document, which is deployed to a Connect server to which all of my colleagues have access. Everyone can easily log in and see the latest information...
- ...explore the data, and look for the trends that inform, and in some cases, drive, our business decisions.
- For example, this map from that same application tells us where our conference attendees traveled from, and we are actively using it to help us decide where to take our Roadshow this year so we can travel to our users to share resources and best practices most effectively. As you can see in this screen capture, it’s very easy to interact with the application to drill into the different areas of the map and find the best locations for us to sponsor events or visit meetups to promote the R community.
- These applications and reports are freely available to whomever at RStudio chooses to log into the server and view them. However, sometimes I want to be more proactive about disseminating the information, instead of waiting for the stakeholders to come to my dashboard.
- For this, I use RStudio Connect to schedule an update to an R Markdown document, and email it to my colleagues with the latest data.
- You can also create parameterized reports that are rendered based on user input....
- ...and you can even designate which data to process, based on who is viewing the report. This is a stock market example, but we have similar reports for our customer data, including one that identifies missed check-in calls or quiet accounts, filtered by the Customer Success representative, so they can each reach out to their customer accounts to find out how we can encourage their adoption of our products. We use this feature in conjunction with the Scheduler, to send different variations of the report to different users in a custom email message, so they see the information that is most meaningful to them.
- This is powerful, because it makes it easy to push targeted analyses directly to the people who are making the business and product decisions, and enables them to make informed choices about the future of the product, and the company. In our case, we use parameterized, scheduled reports to send support ticket data filtered by product to each of the appropriate development teams, so they see the information most relevant to them on a regular basis. Finally, in addition to keeping our teams informed about how the product is being used, we also use these reports and applications to identify trends in our customer data, and find areas where users are having the most trouble.
- We take that data and update the documentation if it turns out that our usage instructions are not clear to people
- ...or file issues with the development teams to drive product improvements if we think the product could handle certain scenarios better.
- We then follow up with the teams to help them prioritize these issues in the context of their other work, and in turn, they gain a better understanding of how the product is being used, which inspires them to think about how we might make our users’ experience as *painless*, productive, and enjoyable as possible. Some examples of improvements we have made based on the data analysis we have done with our customer data are...
- Allowing customers to change the authentication method
- ..customize the body of email messages sent from the server
- ...see better in-product feedback when they make common mistakes
- ... run new diagnostic scripts to help troubleshoot problems
- ...and use the products more effectively via a host of user interface improvements. All this was done in response to user feedback that we deliberately analyzed to find areas for improvements in the products.
- In addition to changes in the product, we have also...
- ...implemented many documentation improvements - clarifications, examples, and cross-references to other resources
- ...optimized our support processes. I already mentioned the impact on our hiring plan, and then we...
- ...created better templates for ticket responses to common issues
- ...implemented a more efficient ticket escalation procedure based on the type of issue being reported...
- ...and created an enhanced training program on Linux administration for our support engineers, once we found that a large number of our tickets require server environment diagnostic skills
- Finally, we introduced transparency into support tickets and license usage for the Customer Success representatives, for context as they nurture a continuing relationship with our customers and foster customer satisfaction and loyalty.
- To measure the effectiveness of our changes, we go back to our original analysis.
- Remember that it is reproducible, so we can rerun the reports easily to see if our enhancements produce the kind of reaction we wanted.
- In this example, we learned from our initial analysis that we were seeing a rise in the number of support tickets related to offline licensing. By default, our server products require an internet connection so they can call out to our license server to validate themselves. However, we also have a way to license our products in offline mode if your host machine does not have a connection to the internet. The process to activate an offline license is straightforward, but it used to require manual intervention by our support engineers. When we did the analysis and realized that offline licensing was causing grief for our customers, we decided to build a customer-facing application to allow users to execute the steps themselves. We created the app, linked to it in our documentation, and subsequently saw a drop in these requests. This top graph shows the number of requests we get for offline licensing, with the red line indicating when we released that customer-facing app. Note that the number of tickets did not drop to zero because people often ask for help before reading the documentation, but the longevity of the tickets plummeted because we now had a quick answer: just go to this app to activate your license in offline mode. The graph on the bottom shows the usage of the app, as measured by Google Analytics. We’re seeing pretty constant usage, so people are finding it, using it, and getting an instant activation instead of having to wait for us to assist them manually. This is a faster resolution for the customer, and in addition, our support engineers got some of their time back, and are now able to spend it on more complex questions from the field. Measuring the success of the changes we made based on the results of our analysis is important because we want to ensure that the investments we make materially improve the experience of our customers and users.
- To gauge success, we check a few things.
- First, we re-run the analysis and compare it to the previous results, to measure the before and after of a release of a custom tool or new feature.
- We look at the number of support tickets for the relevant feature to see if the volume goes down once we’ve fixed something.
- We also look at the longevity of tickets - for example, tickets can be closed faster if there’s good documentation explaining how features can be used, or if we can tell the customer that they just need to download the latest version of the software to get the feature they’re requesting
- We use Google Analytics to look at the usage information around our custom tools and our documentation to make sure that people are finding them
- We can perform sentiment analysis to understand users’ reaction to blog posts describing new features, or measure Twitter mentions of new features or documentation.
- And we can look at the Net Promoter Score given to us by the customer, along with other data collected by our Customer Success representatives in their regular interactions with customers.
- So, how did we actually build this system to produce data-driven product improvements?
- Remember our goals: we have data, and we want to gain insights from it so we can make informed business decisions. We need to gather and understand the data, then communicate the results of our analysis to the stakeholders so they can act on it. And the best way to do that is with code.
- Full disclosure: we love code. Code is repeatable, inspectable, and most importantly, reusable. With code, you can go in and look at how a problem is solved, and either adopt it, or figure out how to improve it yourself. To quote our President, Tareef Kawaf, “Code is communication, and communication is critical.” So we used code written in the R language to solve our problem, with the help of some excellent open-source packages, or libraries.
- For each of the steps in the data science workflow, there are a group of R packages that are particularly suited to the task at hand. These contain functions that make it easier to write efficient and effective code for the target operation. Let’s walk through some code to see how it works, using as an example a simplified version of our application that analyzes our customer support tickets. I know it is unusual to walk through code in a keynote, but I’d like to show you how straightforward it is to create dynamic and powerful applications that can change the way you approach business decisions. You don’t need to be a programmer to write code; you just need good tools and a strong community for support.
- As we walk through the code, we’ll keep track of where we are in this process in the lower left-hand corner.
- First, the import and tidy steps. We need to get the data into R, and there are a variety of packages available based on the source of the data, including text, databases, Spark, Excel, Google Sheets, 'SPSS', 'Stata' and 'SAS'.
- For this application, we are working with web APIs, so we’ll use a pair of packages designed to work with HTTP-based calls.
- We use a couple of functions from these packages to call the Zendesk API and parse the response.
- Then we pull out the object that contains the ticket data itself. The data coming back from the API is paged, so we use this same pattern to loop through all the pages to collect all the support ticket data into one object.
- Now that we have the raw data in R, we’re going to need to transform it into a shape we can use. As is the case with many data sources, Zendesk sends API responses in a format that isn’t terribly easy to work with right out of the box.
- We need to transform the data into a form that we *can* to work with: a dataframe with variables in the columns, observations in the rows, and values in the cells. This makes the data easy to analyze, model, and visualize in later steps of this process.
- With that goal in mind, we load the packages we’ll need, and start flattening the data by pulling nested variables up to the top level. The Product data, for instance, is stored in a custom field that we need to find and convert to another form.
- Then we bind the product data to the main support ticket data, and the result is a new dataframe with the base support ticket data, plus a new column for the Product. We repeat this process for all of the nested fields, and once we have everything we need in one, flat, dataframe, we can move on to the next step and clean up the data types.
- Again, the goal of transforming the data is to make it tidy and consistent. The data we get from the Zendesk API is a little messy, in that dates and factors are stored as strings, and the formats aren’t where we need them to be. We want to clean all this up so the data is easy to analyze and plot.
- We start by converting the Status field to a factor to make it easier to work with.
- Next, we convert dates to the format we need, and add new columns for pieces of the date we need for reporting. We repeat these steps for any other column in the data that needed to be transformed, until we are left with a tidy, well-organized dataset.
- Now that we have tidy data, we can start visualizing and exploring it. Here I’ll show you how few lines of code it takes to get meaningful visualizations out of clean and tidy data.
- Here I’m using the shiny package to build an interactive application, and ggvis to create the plots. There are lots of options for plotting data, and you can use whatever suits your needs, including ggplot2, plotly, dygraphs, highcharter, and dozens of others.
- I’m not showing it because I have limited space on this slide, but since Shiny applications are interactive, we start by gathering our user’s inputs and using them to filter or transform the data that will be visualized based on their selections. We can then summarize the ticket count per week so it can be plotted. We select the columns we want, group the data, count the entries, and add a label.
- Finally, we plot that object using the week as the X axis, the ticket count as the Y axis, and the action as the stroke in the line graph. We add axis labels and a legend, and with these few lines of code...
- ...we’ve created a meaningful visualization of the data. In this case, we see that, after the usual holiday-related lull in December, there were a high number of tickets filed in January and February of this year. This should prompt us to investigate the cause and develop mitigation strategies if necessary. Without this script, this trend would not have been easily visible, but now we have tangible information that we can act upon.
- The next step in the Data Science cycle is to develop models. We didn’t need to do this for our support ticket data application since it is designed to be *descriptive* rather than *predictive*, but as I mentioned earlier, my colleague Andrie created a model for predicting customer churn using packages like broom, recipes, and tidymodels.
- Once the analysis is in a state where we want to communicate the results, there are several ways to disseminate the information.
- In my previous examples, we used a Connect server to get the data into the hands of decision makers, because it makes collaboration and distribution so easy. However, RStudio Connect is built on open-source tools - Shiny, R Markdown, etc. - that can be shared in other ways, as well.
- You could also deploy applications and reports to a Shiny Server, which has an open-source version, as well as a commercial version.
- Or you can publish it to shinyapps.io, our publishing platform in the cloud, which has both free and paid plans, as well. This is a quick way to try out deploying Shiny apps, since it doesn’t require you to stand up your own server.
- And for static R Markdown documents, you can knit or export them into various formats and distribute them manually. Output formats include HTML, PDF, Microsoft Word, and presentations, which allow you to create slides in various formats like Slidy, Beamer, and PowerPoint. You can also build interactive Shiny dashboards, books, and websites with R Markdown if you are so inclined. But regardless of the output format, the analysis remains reproducible - you can rerun it and see the same results, or re-render it with updated data, and have confidence that the scope and logic of the analysis are unchanged. Or, you can easily make changes to the applications and reports to adapt to your organization’s needs, and scale the analysis based on the questions you want to answer, and the data you have available.
- So to review, we were able to drive product improvement with data analysis by...
- Collecting and storing actionable data from the customer
- Pulling it all into one place - in our case, we use R.
- Creating scripts to produce analyses that are reproducible, scalable, and measurable.
- Visualizing the results of the analysis, and communicating that information to our stakeholders in a way that makes it easy for them to make decisions based on that data. The result of this work is improved products, happier customers, and a reusable set of analytic scripts that not only help us identify usability issues, but can also confirm when they’ve actually been addressed by the changes we’ve made. We built all this with open-source R packages, and our success is rooted firmly in the amazing community of R users and contributors who create and nurture these tools. With this approach, it is not only possible, but I would argue *advisable*, to use the data you already have to effect meaningful change in your organization. It is easy to get started, straightforward to customize, and profoundly effective to the process of making business decisions. Doing data science with R is affordable, transparent, and impactful, and I hope I’ve inspired you to give it a try yourself. Code doesn’t have to be intimidating; simple scripts can have big impacts, and there is a strong and inclusive community here to help you get started.
- And it’s not just us who have found value this way. If you’d like to see how other people have built, visualized, and communicated analyses using their own data, I encourage you to visit the Shiny User Showcase page on our website for some great examples.
- My contact information is here if you have any questions, and please come find me during the rest of the conference; I love to talk to you further about this topic. Thank you very much.