Moving Data Science from an Event to A Program: Considerations in Creating Sustainable and Reusable Data Sources

•Als PPTX, PDF herunterladen•

1 gefällt mir•293 views

This document discusses how organizations are increasingly experiencing information crises due to their inability to effectively govern and trust enterprise data across silos. It argues that data governance needs to expand its scope to support both transactional data and business decisions by integrating data sources into a robust infrastructure and data hub. Implementing effective data governance early is important to allow data reuse, maximize value, and help organizations avoid repeating past mistakes of working in silos.

Technologie

Moving Data Science from an Event to
a Program
Wayne Applebaum, Ph. D.

What Gartner Sees
2
“… by 2017, 33 percent of Fortune 100
organizations will experience an
information crisis, due to their inability to
effectively value, govern and trust their
enterprise information.”
Gartner Press Release, February 27, 2014

How it should work
Business
processes and
Business
decisions
bracket a
robust
infrastructure of
data tools and
processes.Transactional Information/Other Data
Measures Analytics Tools
Business Decisions
Target data store
Load Quality Processes
Business Processes

How it usually works
Silo’s of
data that
are difficult
to put
together
"Those who don't know history are destined to repeat
it.”-Edmund Burke

Here’s a Data Scientist Viewpoint
• Identifying Data
Sources
• Data Correctness
• Data Quality
• Business
Involvement
• Multiple Sources
• Data Governance
• Flexibility
• Takes up 80% of
their time

Why the Problem is Getting Worse
• Use and value placed on data and is increasing
• More decisions are being made in the same
amount of time
• Answers aren’t in the silos-you need to cross the
silos to get them
• Business demand for information based decision
is not discussed in the popular media
6
Pressure and opportunity for data and analytics
is rising
Reuse is becoming a business necessity

Emergence of Business Decision Data
7
Data
Business
Decision
Data
MasterTransactional
While the basic rules of Data Governance remains the
same. the scope is expanding

Transactions Vs. Decisions
8
Transactions
Decisions
Process each transaction as quickly as possible
Consolidate Information to make the correct
decision as quickly as possible

The Data Governance-No Free
Lunch Rule
9
When it comes to integrating data sources
There is no free lunch
You have to understand the data and
context to be able to make decisions

10
Creating the Data Hub: Overview
Scope
Identifying Key Objects/Values
Creating the
Controlled
Vocabulary
Object
Mapping
Creating the
Canonical/Targ
et Model
Creating and Rules and Standards
Implementing
Data Retrieval
Creating the
User Interface
Developing
Load
Procedures
Architecture Decisions
Ingestion, Database, Data Governance. Retrieval

Where do we go from here?
11
• Implement Data Governance early
• Integrate Data Governance Across Silo’s
• Recognize that Data Governance doesn’t end with
Master Data
• Big Data represents a new challenges because the
meaning of a transaction is no longer defined on entry
• Create the governance and structures to support both
transactions and decisions
• Consider Data Hubs for cross silo integrations
Governance is essential for reuse and reuse is essential
to maximize value

Weitere ähnliche Inhalte

Was ist angesagt?

Reproducible Dashboards and other great things to do with Jupyter

Domino Data Lab

Andreas weigend

BigDataExpo

Notilyze SAS

BigDataExpo

Building Data Science Teams: A Moneyball Approach

joshwills

Domino and AWS: collaborative analytics and model governance at financial ser...

Domino Data Lab

Data quality management Basic

Khaled Mosharraf

Most of analytics modeling work today focuses on the production of single-purpose "artisanal" models for predictions. This approach to analytics is fragile with respect to model consistency, reorganization, and resource availability. This talk will argue that instead the focus of analytics modeling should be toward the production of analytics interchangeable parts, which can be combined in creative ways to produce a wide variety of analytics results. This "nuts and bolts" approach allows analytics groups to produce results in an agile way where the time between ask and answer is determined by the right combination of analytics, rather than the modeling.

Leveraged Analytics at Scale

Domino Data Lab

Data Science Salon: Quit Wasting Time – Case Studies in Production Machine Le...

Formulatedby

Giovanni Lanzani GoDataDriven

BigDataExpo

H2O World - What you need before doing predictive analysis - Keen.io

Sri Ambati

Data quality - The True Big Data Challenge

Stefan Kühn

Goal: explain the nature of the work of an analytics team to a manager, and enable people on those teams to explain what a data science team needs to a manager. It seems as if every organization wants to enable analytical-decision making and embed analytics into operational processes. What can you do with analytics? It looks like anything is possible. What can you really do? Probably a lot less than you expect. Why is this? Vendors promise easy-to-use analytics tools and services but they rarely deliver. The products may be easy but the work is still hard. Using analytics to solve problems depends on many factors beyond the math: people, processes, the skills of the analyst, the technology used, the data. Technology is the easy part. Figuring out what to do and how to do it is a lot harder. Despite this, fancy new tools get all the attention and budget. People and data are the truly hard parts. People, because many believe that data is absolute rather than relative, and that analytic models produce an answer rather than a range of answers with varying degrees of truth, accuracy and applicability. Data, because managing data for analytics is a nuanced, detail-oriented and seemingly dull task left to back-office IT. If your goal is to build a repeatable analytics capability rather than a one-off analytics project then you will need to address the parts that are rarely mentioned. This talk will explain some of the unseen and little-discussed aspects involved when building and deploying analytics.

Pay no attention to the man behind the curtain - the unseen work behind data ...

mark madsen

Generally speaking, big data and data science originated in the west and are coming to Europe with a bit of a delay. There is at least one exception though: The London-based music discovery website Last.fm is a data company at heart and has been doing large-scale data processing and analysis for years. It started using Hadoop in early 2006, for instance, making it one of the earliest adopters worldwide. When I left Last.fm to join Massive Media, the social media company behind Netlog.com and Twoo.com, I basically moved from a data science forerunner to a newcomer. Massive Media had at least as much data to play with and tremendous potential, but they were not doing much with it yet. The data science team had to be build from the ground up and every step had to be argued for and justified along the way. Having done this exercise of evaluating everything I learned at Last.fm and starting over completely with a clean slate at Massive Media, I developed a pretty clear perspective on how to find good data scientists, what they should be doing, what tools they should be using, and how to organize them to work together efficiently as team, which is precisely what I would like to share in this talk.

Back to Square One: Building a Data Science Team from Scratch

Klaas Bosteels

Building a Data Platform Strata SF 2019

mark madsen

Do you have data? Do you have users? Do they use that data to solve problems? Then you have a data architecture. Maybe your architecture is organic and accidental, or maybe it’s an accumulation of the latest practices and technologies you heard about on Stack Overflow. Spoiler: data architecture is about people and how they use data, not the latest pipeline framework or AI model. Data architecture is about enabling users to be productive, not adding the next “shiny object” and then blaming the users for using it wrong. What you design needs to focus on a different subject than either technology or data. Join Kevin Bogusch, Ecosystem Architect, as he talks with Mark Madsen, Fellow at the Technology Innovation Office, on the crucial elements you’re missing in a successful data architecture: people and process. Find out why Mark says, “don’t buy one problem to solve another problem.”

Data Architecture: OMG It’s Made of People

mark madsen

Valuing the data asset

Bala Iyer

Building a data lake involves more than installing Hadoop or putting data into AWS. The goal in most organizations is to build multi-use data infrastructure that is not subject to past constraints. This tutorial covers design assumptions, design principles, and how to approach the architecture and planning for multi-use data infrastructure in IT. Long: The goal in most organizations is to build multi-use data infrastructure that is not subject to past constraints. This session will discuss hidden design assumptions, review design principles to apply when building multi-use data infrastructure, and provide a reference architecture to use as you work to unify your analytics infrastructure. The focus in our market has been on acquiring technology, and that ignores the more important part: the larger IT landscape within which this technology lives and the data architecture that lies at its core. If one expects longevity from a platform then it should be a designed rather than accidental architecture. Architecture is more than just software. It starts from use and includes the data, technology, methods of building and maintaining, and organization of people. What are the design principles that lead to good design and a functional data architecture? What are the assumptions that limit older approaches? How can one integrate with, migrate from or modernize an existing data environment? How will this affect an organization's data management practices? This tutorial will help you answer these questions. Topics covered: * A brief history of data infrastructure and past design assumptions * Categories of data and data use in organizations * Data architecture * Functional architecture * Technology planning assumptions and guidance

Architecting a Data Platform For Enterprise Use (Strata NY 2018)

mark madsen

Join our #DataTalk on Thursdays at 5 p.m. ET. This week, we tweeted with Dr. Michael Wu, the Chief Scientist at Lithium, where he applies data-driven methodologies to investigate the complex dynamics of the social web. Michael works with big data and has developed many predictive and prescriptive social analytics with actionable insights. His R&D won him the recognition as a 2010 Influential Leader by CRM Magazine. You can see all tweets and resources here: http://www.experian.com/blogs/news/about/data-scientists/

What is a Data Scientist

Experian_US

Teams working on new business initiatives, whether for enhancing customer engagement, creating new value, or addressing compliance considerations, know that a successful strategy starts with the synchronization of operational and reporting data from across the organization into a centralized repository for use in advanced analytics and other projects. However, the range and complexity of data sources as well as the lack of specialized skills needed to extract data from critical legacy systems often causes inefficiencies and gaps in the data being used by the business. The first part of our webcast series on Foundation Strategies for Trust in Big Data provides insight into how Syncsort Connect with its design once, deploy anywhere approach supports a repeatable pattern for data integration by enabling enterprise architects and developers to ensure data from ALL enterprise data sources– from mainframe to cloud – is available in the downstream data lakes for use in these key business initiatives.

Foundational Strategies for Trust in Big Data Part 1: Getting Data to the Pla...

Precisely

Over the past eight or nine years, applying DevOps practices to various areas of technology within business has grown in popularity and produced demonstrable results. These principles are particularly fruitful when applied to a data analytics environment. Bob Eilbacher explains how to implement a strong DevOps practice for data analysis, starting with the necessary cultural changes that must be made at the executive level and ending with an overview of potential DevOps toolchains. Bob also outlines why DevOps and disruption management go hand in hand. Topics include: - The benefits of a DevOps approach, with an emphasis on improving quality and efficiency of data analytics - Why the push for a DevOps practice needs to come from the C-suite and how it can be integrated into all levels of business - An overview of the best tools for developers, data analysts, and everyone in between, based on the business’s existing data ecosystem - The challenges that come with transforming into an analytics-driven company and how to overcome them - Practical use cases from Caserta clients This presentation was originally given by Bob at the 2017 Strata Data Conference in New York City.

Creating a DevOps Practice for Analytics -- Strata Data, September 28, 2017

Caserta

Was ist angesagt? (20)

Reproducible Dashboards and other great things to do with Jupyter

Andreas weigend

Notilyze SAS

Building Data Science Teams: A Moneyball Approach

Domino and AWS: collaborative analytics and model governance at financial ser...

Data quality management Basic

Leveraged Analytics at Scale

Data Science Salon: Quit Wasting Time – Case Studies in Production Machine Le...

Giovanni Lanzani GoDataDriven

H2O World - What you need before doing predictive analysis - Keen.io

Data quality - The True Big Data Challenge

Pay no attention to the man behind the curtain - the unseen work behind data ...

Back to Square One: Building a Data Science Team from Scratch

Building a Data Platform Strata SF 2019

Data Architecture: OMG It’s Made of People

Valuing the data asset

Architecting a Data Platform For Enterprise Use (Strata NY 2018)

What is a Data Scientist

Foundational Strategies for Trust in Big Data Part 1: Getting Data to the Pla...

Creating a DevOps Practice for Analytics -- Strata Data, September 28, 2017

Ähnlich wie Moving Data Science from an Event to A Program: Considerations in Creating Sustainable and Reusable Data Sources

DC Salesforce1 Tour Data Governance Lunch Best Practices deck

Beth Fitzpatrick

How Data Integration and Governance Enables HR to Drive Value .pptx

Natasha Ramdial - Roopnarine

Organizations must realize what it means to utilize data quality management in support of business strategy. This webinar will demonstrate how chronic business challenges can often be attributed to the root problem of poor data quality. Showing how data quality should be engineered provides a useful framework in which to develop an effective approach. Establishing this framework allows organizations to more efficiently identify business and data problems caused by structural issues versus practice-oriented defects; giving them the skillset to prevent these problems from re-occurring. Learning Objectives: Understanding foundational data quality concepts based on the DAMA DMBOK Utilizing data quality engineering in support of business strategy Case Studies illustrating data quality success Data quality guiding principles & best practices Steps for improving data quality at your organization

Data-Ed Webinar: Data Quality Success Stories

DATAVERSITY

Federated data organizations in public sector face more challenges today than ever before. As discovered via research performed by North Highland Consulting, these are the top issues you are most likely experiencing: • Knowing what data is available to support programs and other business functions • Data is more difficult to access • Without insight into the lineage of data, it is risky to use as the basis for critical decisions • Analyzing data and extracting insights to influence outcomes is difficult at best The solution to solving these challenges lies in creating a holistic enterprise data governance program and enforcing the program with a full-featured enterprise data management platform. Kreig Fields, Principle, Public Sector Data and Analytics, from North Highland Consulting and Rob Karel, Vice President, Product Strategy and Product Marketing, MDM from Informatica will walk through a pragmatic, “How To” approach, full of useful information on how you can improve your agency’s data governance initiatives. Learn how to kick start your data governance intiatives and how an enterprise data management platform can help you: • Innovate and expose hidden opportunities • Break down data access barriers and ensure data is trusted • Provide actionable information at the speed of business

Is Your Agency Data Challenged?

DLT Solutions

Does this sound familiar? "Are you sure those numbers are right?" "Why are your numbers different than theirs?" We've all heard it and had that gut wrenching feeling of doubt that comes with uncertainty around the quality of the numbers. Stop the madness! Presented in Dunwoody on April 18 by industry leading expert Mary Levins who discusseses what it takes to successfully take control of your data using the Data Governance Framework. This framework is proven to improve the quality of your BI solutions. Mary is the founder of Sierra Creek Consulting

Stop the madness - Never doubt the quality of BI again using Data Governance

Mary Levins, PMP

Securing big data (july 2012)

Marc Vael

Teams working on new initiatives whether for customer engagement, advanced analytics, or regulatory and compliance requirements need a broad range of data sources for the highest quality and most trusted results. Yet the sheer volume of data delivered coupled with the range of data sources including those from external 3rd parties increasingly precludes trust, confidence, and even understanding of the data and how or whether it can be used to make effective data-driven business decisions. The second part of our webcast series on Foundation Strategies for Trust in Big Data provides insight into how Trillium Discovery for Big Data with its natively distributed execution for data profiling supports a foundation of data quality by enabling business analysts to gain rapid insight into data delivered to the data lake without technical expertise.

Foundational Strategies for Trust in Big Data Part 2: Understanding Your Data

Precisely

This was presented on October 24, 2018 at the ASUG EIM Conference. One of the many challenges presented by an acquisition and divestiture event is unifying disparate data and integrating systems together. If you are leading an integration, you may have more questions than answers on how to approach this event. Learn how to best leverage the momentum and budgets that accompany these activities to jump start good governance practices up front, as well as how to measure the return on investment, ensuring data and EIM professionals' ongoing success.

The Merger is Happening, Now What Do We Do?

DATUM LLC

Todays’ increasing emphasis on differentiation in the digital economy further complicates the data governance challenge. Learn about today’s common challenges and about the new adaptations that are required to support the digital era. Avoid the pitfalls and follow along on Johnson & Johnson’s journey to: - Establish and scale a best in class enterprise data governance program - Identify and focus on the most critical data and information to bolster incremental wins and garner executive support - Ensure readiness for automation with SAP MDG on HANA

Most Common Data Governance Challenges in the Digital Economy

Robyn Bollhorst

Cff data governance best practices

Beth Fitzpatrick

Data driven decision making

SHAHZAD M. SALEEM

Why data governance is the new buzz?

Aachen Data & AI Meetup

Explorasi Data untuk Peluang Bisnis dan Pengembangan Karir.pptx

windu19

Fate of the Chief Data Officer

Tamarah Usher

Hcd wp-2012-better dataleadstobetteranalytics

Health Care DataWorks

Successful stewardship Presentation

Certus Solutions

During the Data Summit Conference in New York, our CMO Ravi Shankar and BJ Fesq, Chief Data Officer at CIT Group, were discussing the modernization of data architectures with data virtualization. This presentation explores how data virtualization is being used to dramatically reduce data proliferation and ensure that all consumers are working with a single source of the truth. It also looks at how data virtualization can drive standardization, measure and improve data quality, abstract data consumers from data providers, expose data lineage, enable cross-company data integration, and serve as a common provisioning point from which to access all authoritative sources of data.

Increasing Agility Through Data Virtualization

Denodo

Data governance and data quality are often described as two sides of the same coin. Data governance provides a data framework relevant to business needs, and data quality provides visibility into the health of the data. If you only have a data governance tool, you’re missing half the picture. Trillium Discovery seamlessly integrates with Collibra for a complete, closed-loop data governance solution. Build your data quality rules in Collibra, and they are automatically passed to Trillium for data quality processing. The data quality results and metrics are then passed back to Collibra – allowing data stewards and business users to see the health of the data right within their Collibra dashboard. View this webinar on-demand to see how you can leverage this integration in your organization to readily build, apply, and execute business rules based on data governance policies within Collibra.

Building Rules for Data Governance

Precisely

In this new paper, I explore the organisational and cultural challenges of implementing information governance and data quality. I identify potential problems with the traditional centralised methods of data quality management, and offer alternative organistional models which can enable a more distributed and democratised approach to improving your organisations data. I also propose a simple four-step approach to delivering immediate business value from your data.

WHITE PAPER: Distributed Data Quality

Alan D. Duncan

Sgcp14dunlea

Justin Hayward

Ähnlich wie Moving Data Science from an Event to A Program: Considerations in Creating Sustainable and Reusable Data Sources (20)

DC Salesforce1 Tour Data Governance Lunch Best Practices deck

How Data Integration and Governance Enables HR to Drive Value .pptx

Data-Ed Webinar: Data Quality Success Stories

Is Your Agency Data Challenged?

Stop the madness - Never doubt the quality of BI again using Data Governance

Securing big data (july 2012)

Foundational Strategies for Trust in Big Data Part 2: Understanding Your Data

The Merger is Happening, Now What Do We Do?

Most Common Data Governance Challenges in the Digital Economy

Cff data governance best practices

Data driven decision making

Why data governance is the new buzz?

Explorasi Data untuk Peluang Bisnis dan Pengembangan Karir.pptx

Fate of the Chief Data Officer

Hcd wp-2012-better dataleadstobetteranalytics

Successful stewardship Presentation

Increasing Agility Through Data Virtualization

Building Rules for Data Governance

WHITE PAPER: Distributed Data Quality

Sgcp14dunlea

Mehr von Domino Data Lab

While business analysis rapidly grows more data-driven, the analyst community is slow to adapt the best practices of data science workflows. Many parallels exists between data science “top topics” (e.g. reproducibility) and business pain points, but these common needs are obscured by the different “languages” of these two communities. The opportunity cost is greatest in heavily regulated industries such as finance and insurance where documentation and compliance are paramount. In this talk, we will review our experience transitioning Capital One business analysts from legacy systems to open-source workflows by developing user-friendly tools. We incentivized business analysts to adopt the data science mindset by curating open-source tools and developing code packages which simplify workflows and eliminate pain points. Our internal R package, tidycf, reimagines cumbersome Excel cashflow statements as dataframes and uses RMarkdown templates and the RStudio IDE for an intuitive, user-friendly experience without the overhead of maintaining a custom GUI. We tackle challenges in documentation and communication while immersing new users in the R language. We will share best practices and lessons learned from our experience designing tools for non-technical end-users, standardizing workflows based on the RStudio IDE’s infrastructure, and evangelizing data science methods.

What's in your workflow? Bringing data science workflows to business analysis...

Domino Data Lab

In this talk, we’ll describe NoSQL (“not-only SQL”) and document-oriented databases and the value they provide for data science companies like Uptake. We will walk through the unique challenges such datastores pose for data science workflows. To make these challenges and lessons learned concrete, we’ll explore data science workflows through a discussion of the development efforts that led to “uptasticsearch”, an R package released by the Uptake Data Science team to reduce friction in interacting with a document store called Elasticsearch. The talk will conclude with a discussion of recent developments in NoSQL technologies and implications for data scientists.

The Proliferation of New Database Technologies and Implications for Data Scie...

Domino Data Lab

Since 2004, Illinois has collected demographic information about traffic stops conducted by police in an effort to identify racial bias. This data has been used by groups such as the ACLU and the Stanford Open Policing Project to identify key markers that infer racial bias in policing. We have applied exploratory data analysis to investigate whether systemic racial bias may appear and to what extent. This talk will walk the audience through the insights gleaned from the exploration of this data along with the challenges posed and ongoing questions raised.

Racial Bias in Policing: an analysis of Illinois traffic stops data

Domino Data Lab

Cars.com Inc. is a decision engine for car buyers and a growth engine for our partners. Data Science is the bread and butter of any decision engine and Cars is no different. In this talk, I will discuss how we quantify various parameters of a car and plan to make use of all the data in hand to put predictive models at various stages of a users’ automobile lifecycle. This talk will also cater to students looking to gain knowledge on how data science is utilized at scale while still following certain processes and leading the way for business and product partners.

Leveraging Data Science in the Automotive Industry

Domino Data Lab

Lake Michigan and outdoor recreation are enjoyable aspects of summers in Chicago, but it can come with risk of potential E. coli in Lake Michigan or West Nile Virus from mosquitos. This summer, the City of Chicago launched two new predictive analytics projects to forecasts the risks and to proactively limit these risks. Members of the research team, Gene Leynes and Nick Lucius discuss the projects and how they’re being used as part of city operations.

Summertime Analytics: Predicting E. coli and West Nile Virus

Domino Data Lab

Today, more than ever before, maps are being used to bring data to life. In this presentation I will demonstrate how geoviz can make data science more tangible by providing an interactive canvas for spatial data. Gregory Brunner will shows several examples of how maps are being used enhance how we communicate data and how this applies across all scales, including spatial, temporal, and size of data.

GeoViz: A Canvas for Data Science

Domino Data Lab

Doing your first Kaggle (Python for Big Data sets)

Domino Data Lab

How I Learned to Stop Worrying and Love Linked Data

Domino Data Lab

Although both disciplines are unique in their own ways, Software Engineering and Data Science make heavy use of programing languages to do their respective jobs. Data Science is a relatively new discipline and many of its practitioners have not previously been professional software engineers. There are a few techniques that Data Scientists can leverage from Software Engineering in order to make their tooling and environments, faster to design, more easily debugged and most importantly, clearer to read. This talk will be going over some practical tips that anyone can use to help better understand their code; give clarity around cloud environments, their uses and drawbacks and finally briefly touching on the Software Development Lifecycle.

Software Engineering for Data Scientists

Domino Data Lab

Within marketing research, big data is often described as being “census” data for the population that it represents. The devil is in the details and when we take a closer look we can see that this isn’t the case. There are many situations that are not captured within the population that big data purports to be a census of. Big data isn’t even a census of itself since it’s not uncommon for records to be excluded either by accident during the collection process or by design in the cleaning processor. Unfortunately, our industry is so enamored with the size of big data that some users of data are willing to trade off precision for tonnage. Fortunately, if the shortcomings of big data are understood and corrected it can accurately represent the population that it measures in the correct proportion to the universe. We will discuss a method that Nielsen has developed called “Common Homes” that is designed to identify and correct the shortcomings of big data sets that represent media consumption.

Making Big Data Smart

Domino Data Lab

Big Data analytics is well known to uncover hidden insights that gives an organization an edge over the competition. But data does not need to be big in order to be useful. Smaller companies and startups may lack the volume of data that qualifies as big data, yet the variety of data can still yield a trove of insights that helps in driving the business strategies of a company. Startups may also lack the resources to fund an additional, seemingly expensive development project. The key is in simplicity, start small, simple and architect for scalability and performance. But how do you start? In this presentation, we share our experience in building a cost effective, AWS serverless data analytics platform that became an invaluable tool for sales, marketing and operational efficiencies.Serverless architectures simplify development work where servers and software are managed by a third party cloud provider. Developers can focus on just building the data wrangling and data analysis logic where critical aspects like scalability and high availability are guaranteed by the cloud provider. Besides, serverless services offer the pay as you go model, where you pay only based on the amount of resources you use. This turns out to be another attractive aspect where costs can be managed based on the usage. In this presentation we will focus on techniques and best practices to build a big data analytics platform using AWS serverless services like Lambda, DynamoDB, S3, Kinesis, Athena, QuickSight and Amazon ML. We will highlight the strengths of each of these services and what role each plays in the data analytics pipeline. We compare and contrast these services with some of the other popularly used big data technologies like Hadoop, Spark and Kafka. We also demonstrate the usage of these services to build intelligent components that detect anomalies, yield recommendations, simulate chat bots and generate predictive analytics.

Building Data Analytics pipelines in the cloud using serverless technology

Domino Data Lab

The data science process seeks to transform and empower organizations by finding and exploiting market inefficiencies and potentially hidden opportunities, but this is often an expensive, tedious process. However, many steps can be automated to provide a streamlined experience for data scientists. Eduardo Arino de la Rubia explores the tools being created by the open source community to free data scientists from tedium, enabling them to work on the high-value aspects of insight creation and impact validation. The promise of the automated statistician is almost as old as statistics itself. From the creations of vast tables, which saved the labor of calculation, to modern tools which automatically mine datasets for correlations, there has been a considerable amount of advancement in this field. Eduardo compares and contrasts a number of open source tools, including TPOT and auto-sklearn for automated model generation and scikit-feature for feature generation and other aspects of the data science workflow, evaluates their results, and discusses their place in the modern data science workflow. Along the way, Eduardo outlines the pitfalls of automated data science and applications of the “no free lunch” theorem and dives into alternate approaches, such as end-to-end deep learning, which seek to leverage massive-scale computing and architectures to handle automatic generation of features and advanced models.

Leveraging Open Source Automated Data Science Tools

Domino Data Lab

The Role and Importance of Curiosity in Data Science

Domino Data Lab

by Jennifer Shin Senior Principal Data Scientist, Nielsen With more and more data being collected from consumers, finding a efficient solution to aligning data over time can become increasingly difficult and yet, even more necessary. Whether it's a change in the data collection process or an error in the system, working with big data requires tools that can account for real world complexities. This talk with introduce the benefits and complexities of implementing a 'fuzzy' solution using the Levenshtein algorithm. Attendees will walk away with a high level understanding of fuzzy matching algorithms and learn how it can be effectively applied to solve real word business problem.

Fuzzy Matching to the Rescue

Domino Data Lab

by Liangjie Hong Head of Data Science, Etsy Latent factor models and decision tree based models are widely used in tasks of prediction, ranking and recommendation. Latent factor models have the advantage of interpreting categorical features by a low-dimensional representation, while such an interpretation does not naturally fit numerical features. In contrast, decision tree based models enjoy the advantage of capturing the nonlinear interactions of numerical features, while their capability of handling categorical features is limited by the cardinality of those features. Since in real-world applications we usually have both abundant numerical features and categorical features with large cardinality (e.g. geolocations, IDs, tags etc.), we design a new model, called GB-CENT, which leverages latent factor embedding and tree components to achieve the merits of both while avoiding their demerits. With two real-world data sets, we demonstrate that GB-CENT can effectively (i.e. fast and accurately) achieve better accuracy than state-of-the-art matrix factorization, decision tree based models and their ensemble

How to Effectively Combine Numerical Features and Categorical Features

Domino Data Lab

Building Up Local Models of Customers

Domino Data Lab

by Gaurav Chakravorty Head of Strategy Development, qplum.co Benjamin Graham spoke in 1945 about his goal of making investing a science, a utility. This goal was so far elusive, primarily because human traders were, for the most part, better than machines. With Deep Learning, that is changing. I'll summarize the major changes that are happening in asset management industry. I'll talk about the specific goals that data-science based trading is trying to achieve. I'll talk about what AI has been able to accomplish already and what are the main challenges. Deep Learning has been very successful in social sciences and especially areas where there is a lot of data. Trading or investment management is similarly a social science problem where the complexities are human generated. I'll talk about why Deep Learning is the solution here.

Making Investing A Science

Domino Data Lab

Jonathan Roberts Chief Innovation Officer, Dotdash.com Companies make many decisions for many different reasons, some of which are sometimes based in data. As a data scientist, your research skills can get you to the correct answer. But the real value of a data science team comes when you can identify the correct question. Jon will talk about his four-year transition from research physicist to senior executive, how data science research helped drive the biggest change in the company’s history, and drove one of the biggest experiments ever undertaken on the internet.

How to Use Data Science to Affect Company Change

Domino Data Lab

Andrew Odewahn CTO, O'Reilly Media The Jupyter Notebook is a key part of every data scientist's tool chest, but it's often seen as just an incremental tool in a process, a scratchpad that is discarded once the real work is done. But, what if instead the notebook was itself the final product that communicated the results of a data science team's efforts in a lasting format? In this talk, I'll show how O'Reilly Media is using Jupyter to create a variety of media (books, articles, and new kinds of code+video+text hybrids) and cover some of the lessons learned.

Making Media with Jupyter

Domino Data Lab

by Noelle Sio Saldana Principal Data Scientist at Pivotal The success of a Data Science project is not simply the model fit or the accuracy of its predictions; it is whether those models are being leveraged to make smarter business decisions. Over the past few years, Pivotal’s Data Scientists have experimented with software development methods practiced and taught by their Pivotal Labs counterparts in engineering, design and product management. By reframing Data Science as building software and products instead of research, we found that we reaped similar benefits: shorter and more productive iterations, and clients who actually used the models that we built and skills we taught long after we left. In this talk, we discuss how we have successfully (and maybe not as successfully) borrowed principles from practices like Lean and Agile to Data Science. Topics include: Minimum Viable Product Models Build-Measure-Learn instead of a silver bullet Pair programming Scrums and retrospectives Practicing empathy instead of elitism

Lean Data Science

Domino Data Lab

Mehr von Domino Data Lab (20)

What's in your workflow? Bringing data science workflows to business analysis...

The Proliferation of New Database Technologies and Implications for Data Scie...

Racial Bias in Policing: an analysis of Illinois traffic stops data

Leveraging Data Science in the Automotive Industry

Summertime Analytics: Predicting E. coli and West Nile Virus

GeoViz: A Canvas for Data Science

Doing your first Kaggle (Python for Big Data sets)

How I Learned to Stop Worrying and Love Linked Data

Software Engineering for Data Scientists

Making Big Data Smart

Building Data Analytics pipelines in the cloud using serverless technology

Leveraging Open Source Automated Data Science Tools

The Role and Importance of Curiosity in Data Science

Fuzzy Matching to the Rescue

How to Effectively Combine Numerical Features and Categorical Features

Building Up Local Models of Customers

Making Investing A Science

How to Use Data Science to Affect Company Change

Making Media with Jupyter

Lean Data Science

Kürzlich hochgeladen

As privacy and data protection regulations evolve rapidly, organizations operating in multiple jurisdictions face mounting challenges to ensure compliance and safeguard customer data. With state-specific privacy laws coming up in multiple states this year, it is essential to understand what their unique data protection regulations will require clearly. How will data privacy evolve in the US in 2024? How to stay compliant? Our panellists will guide you through the intricacies of these states' specific data privacy laws, clarifying complex legal frameworks and compliance requirements. This webinar will review: - The essential aspects of each state's privacy landscape and the latest updates - Common compliance challenges faced by organizations operating in multiple states and best practices to achieve regulatory adherence - Valuable insights into potential changes to existing regulations and prepare your organization for the evolving landscape

TrustArc Webinar - Stay Ahead of US State Data Privacy Law Developments

TrustArc

Imagine a world where information flows as swiftly as thought itself, making decision-making as fluid as the data driving it. Every moment is critical, and the right tools can significantly boost your organization’s performance. The power of real-time data automation through FME can turn this vision into reality. Aimed at professionals eager to leverage real-time data for enhanced decision-making and efficiency, this webinar will cover the essentials of real-time data and its significance. We’ll explore: FME’s role in real-time event processing, from data intake and analysis to transformation and reporting An overview of leveraging streams vs. automations FME’s impact across various industries highlighted by real-life case studies Live demonstrations on setting up FME workflows for real-time data Practical advice on getting started, best practices, and tips for effective implementation Join us to enhance your skills in real-time data automation with FME, and take your operational capabilities to the next level.

From Event to Action: Accelerate Your Decision Making with Real-Time Automation

Safe Software

[2024]Digital Global Overview Report 2024 Meltwater.pdf

hans926745

GenCyber Cyber Security Day Presentation

Michael W. Hawkins

Real Time Object Detection Using Open CV

Khem

Three things you will take away from the session: • How to run an effective tenant-to-tenant migration • Best practices for before, during, and after migration • Tips for using migration as a springboard to prepare for Copilot in Microsoft 365 Main ideas: Migration Overview: The presentation covers the current reality of cross-tenant migrations, the triggers, phases, best practices, and benefits of a successful tenant migration Considerations: When considering a migration, it is important to consider the migration scope, performance, customization, flexibility, user-friendly interface, automation, monitoring, support, training, scalability, data integrity, data security, cost, and licensing structure Next Wave: The next wave of change includes the launch of Copilot, which requires businesses to be prepared for upcoming changes related to Copilot and the cloud, and to consolidate data and tighten governance ShareGate: ShareGate can help with pre-migration analysis, configurable migration tool, and automated, end-user driven collaborative governance

Strategize a Smooth Tenant-to-tenant Migration and Copilot Takeoff

sammart93

How to Troubleshoot Apps for the Modern Connected Worker

ThousandEyes

Data Cloud, More than a CDP by Matt Robison

Anna Loughnan Colquhoun

With more memory available, system performance of three Dell devices increased, which can translate to a better user experience Conclusion When your system has plenty of RAM to meet your needs, you can efficiently access the applications and data you need to finish projects and to-do lists without sacrificing time and focus. Our test results show that with more memory available, three Dell PCs delivered better performance and took less time to complete the Procyon Office Productivity benchmark. These advantages translate to users being able to complete workflows more quickly and multitask more easily. Whether you need the mobility of the Latitude 5440, the creative capabilities of the Precision 3470, or the high performance of the OptiPlex Tower Plus 7010, configuring your system with more RAM can help keep processes running smoothly, enabling you to do more without compromising performance.

Boost PC performance: How more available memory can improve productivity

Principled Technologies

How to Troubleshoot Apps for the Modern Connected Worker

ThousandEyes

Building Digital Trust in a Digital Economy Veronica Tan, Director - Cyber Security Agency of Singapore Apidays Singapore 2024: Connecting Customers, Business and Technology (April 17 & 18, 2024) ------ Check out our conferences at https://www.apidays.global/ Do you want to sponsor or talk at one of our conferences? https://apidays.typeform.com/to/ILJeAaV8 Learn more on APIscene, the global media made by the community for the community: https://www.apiscene.io Explore the API ecosystem with the API Landscape: https://apilandscape.apiscene.io/

Apidays Singapore 2024 - Building Digital Trust in a Digital Economy by Veron...

apidays

AWS Community Day CPH - Three problems of Terraform

Andrey Devyatkin

presentation ICT roal in 21st century education

jfdjdjcjdnsjd

2024: Domino Containers - The Next Step. News from the Domino Container commu...

Martijn de Jong

The 7 Things I Know About Cyber Security After 25 Years | April 2024

Rafal Los

Scaling API-first – The story of a global engineering organization Ian Reasor, Senior Computer Scientist - Adobe Radu Cotescu, Senior Computer Scientist - Adobe Apidays New York 2024: The API Economy in the AI Era (April 30 & May 1, 2024) ------ Check out our conferences at https://www.apidays.global/ Do you want to sponsor or talk at one of our conferences? https://apidays.typeform.com/to/ILJeAaV8 Learn more on APIscene, the global media made by the community for the community: https://www.apiscene.io Explore the API ecosystem with the API Landscape: https://apilandscape.apiscene.io/

Apidays New York 2024 - Scaling API-first by Ian Reasor and Radu Cotescu, Adobe

apidays

Join our latest Connector Corner webinar to discover how UiPath Integration Service revolutionizes API-centric automation in a 'Quote to Cash' process—and how that automation empowers businesses to accelerate revenue generation. A comprehensive demo will explore connecting systems, GenAI, and people, through powerful pre-built connectors designed to speed process cycle times. Speakers: James Dickson, Senior Software Engineer Charlie Greenberg, Host, Product Marketing Manager

Connector Corner: Accelerate revenue generation using UiPath API-centric busi...

DianaGray10

Discord is a free app offering voice, video, and text chat functionalities, primarily catering to the gaming community. It serves as a hub for users to create and join servers tailored to their interests. Discord’s ecosystem comprises servers, each functioning as a distinct online community with its own channels dedicated to specific topics or activities. Users can engage in text-based discussions, voice calls, or video chats within these channels. Understanding Discord Servers Discord servers are virtual spaces where users congregate to interact, share content, and build communities. Servers may revolve around gaming, hobbies, interests, or fandoms, providing a platform for like-minded individuals to connect. Communication Features Discord offers a range of communication tools, including text channels for messaging, voice channels for real-time audio conversations, and video channels for face-to-face interactions. These features facilitate seamless communication and collaboration. What Does NSFW Mean? The acronym NSFW stands for “Not Safe For Work,” indicating content that may be inappropriate for professional or public settings. NSFW Content NSFW content encompasses material that is sexually explicit, violent, or otherwise graphic in nature. It often includes nudity, profanity, or depictions of sensitive topics.

Understanding Discord NSFW Servers A Guide for Responsible Users.pdf

UK Journal

What are drone anti-jamming systems? The drone anti-jamming systems and anti-spoof technology protect against interference, jamming, and spoofing of the UAVs. To protect their security, countries are beginning to research drone anti-jamming systems, also known as drone strike weapons. The anti-jam and anti-spoof technology protects against interference, jamming and spoofing. A drone strike weapon is a drone attack weapon that can attack and destroy enemy drones. So what is so unique about this amazing system?

What Are The Drone Anti-jamming Systems Technology?

Antenna Manufacturer Coco

Advantages of Hiring UIUX Design Service Providers for Your Business

Pixlogix Infotech

Kürzlich hochgeladen (20)

TrustArc Webinar - Stay Ahead of US State Data Privacy Law Developments

From Event to Action: Accelerate Your Decision Making with Real-Time Automation

[2024]Digital Global Overview Report 2024 Meltwater.pdf

GenCyber Cyber Security Day Presentation

Real Time Object Detection Using Open CV

Strategize a Smooth Tenant-to-tenant Migration and Copilot Takeoff

How to Troubleshoot Apps for the Modern Connected Worker

Data Cloud, More than a CDP by Matt Robison

Boost PC performance: How more available memory can improve productivity

How to Troubleshoot Apps for the Modern Connected Worker

Apidays Singapore 2024 - Building Digital Trust in a Digital Economy by Veron...

AWS Community Day CPH - Three problems of Terraform

presentation ICT roal in 21st century education

2024: Domino Containers - The Next Step. News from the Domino Container commu...

The 7 Things I Know About Cyber Security After 25 Years | April 2024

Apidays New York 2024 - Scaling API-first by Ian Reasor and Radu Cotescu, Adobe

Connector Corner: Accelerate revenue generation using UiPath API-centric busi...

Understanding Discord NSFW Servers A Guide for Responsible Users.pdf

What Are The Drone Anti-jamming Systems Technology?

Advantages of Hiring UIUX Design Service Providers for Your Business

Moving Data Science from an Event to A Program: Considerations in Creating Sustainable and Reusable Data Sources

1. Moving Data Science from an Event to a Program Wayne Applebaum, Ph. D.

2. What Gartner Sees 2 “… by 2017, 33 percent of Fortune 100 organizations will experience an information crisis, due to their inability to effectively value, govern and trust their enterprise information.” Gartner Press Release, February 27, 2014

3. How it should work Business processes and Business decisions bracket a robust infrastructure of data tools and processes.Transactional Information/Other Data Measures Analytics Tools Business Decisions Target data store Load Quality Processes Business Processes

4. How it usually works Silo’s of data that are difficult to put together "Those who don't know history are destined to repeat it.”-Edmund Burke

5. Here’s a Data Scientist Viewpoint • Identifying Data Sources • Data Correctness • Data Quality • Business Involvement • Multiple Sources • Data Governance • Flexibility • Takes up 80% of their time

6. Why the Problem is Getting Worse • Use and value placed on data and is increasing • More decisions are being made in the same amount of time • Answers aren’t in the silos-you need to cross the silos to get them • Business demand for information based decision is not discussed in the popular media 6 Pressure and opportunity for data and analytics is rising Reuse is becoming a business necessity

7. Emergence of Business Decision Data 7 Data Business Decision Data MasterTransactional While the basic rules of Data Governance remains the same. the scope is expanding

8. Transactions Vs. Decisions 8 Transactions Decisions Process each transaction as quickly as possible Consolidate Information to make the correct decision as quickly as possible

9. The Data Governance-No Free Lunch Rule 9 When it comes to integrating data sources There is no free lunch You have to understand the data and context to be able to make decisions

10. 10 Creating the Data Hub: Overview Scope Identifying Key Objects/Values Creating the Controlled Vocabulary Object Mapping Creating the Canonical/Targ et Model Creating and Rules and Standards Implementing Data Retrieval Creating the User Interface Developing Load Procedures Architecture Decisions Ingestion, Database, Data Governance. Retrieval

11. Where do we go from here? 11 • Implement Data Governance early • Integrate Data Governance Across Silo’s • Recognize that Data Governance doesn’t end with Master Data • Big Data represents a new challenges because the meaning of a transaction is no longer defined on entry • Create the governance and structures to support both transactions and decisions • Consider Data Hubs for cross silo integrations Governance is essential for reuse and reuse is essential to maximize value

Moving Data Science from an Event to A Program: Considerations in Creating Sustainable and Reusable Data Sources

Empfohlen

Empfohlen

Weitere ähnliche Inhalte

Was ist angesagt?

Was ist angesagt? (20)

Ähnlich wie Moving Data Science from an Event to A Program: Considerations in Creating Sustainable and Reusable Data Sources

Ähnlich wie Moving Data Science from an Event to A Program: Considerations in Creating Sustainable and Reusable Data Sources (20)

Mehr von Domino Data Lab

Mehr von Domino Data Lab (20)

Kürzlich hochgeladen

Kürzlich hochgeladen (20)

Moving Data Science from an Event to A Program: Considerations in Creating Sustainable and Reusable Data Sources