1. DATA WAREHOUSING AND DATA MINING M.Mageshwari,Lecturer Lecturer,Department of CE M.S.P.V.L Polytechnic College

3. A producer wants to know…. Which are our lowest/highest margin customers ? Who are my customers and what products are they buying? Which customers are most likely to go to the competition ? What impact will new products/services have on revenue and margins? What product prom- -otions have the biggest impact on revenue? What is the most effective distribution channel?

19. Explorers, Farmers and Tourists Explorers: Seek out the unknown and previously unsuspected rewards hiding in the detailed data Farmers: Harvest information from known access paths Tourists: Browse information about Tourists

20. Application-Orientation vs. Subject-Orientation Application-Orientation Operational Database Loans Credit Card Trust Savings Subject-Orientation Data Warehouse Customer Vendor Product Activity

21. Functioning of Data warehousing Data Source Cleaning Transformation Data Warehouse New Update

29. OLAP User Result Result set Request SQL DATA WAREHOUSE OLAP SERVER FRONT END TOOL

33. Data Warehouse Architecture Data Warehouse Engine Optimized Loader Extraction Cleansing Analyze Query Metadata Repository Relational Databases Legacy Data Purchased Data ERP Systems

34. Architecture of data warehousing External data Data Acquisition Data Manager Warehouse data External data Data Dictionary Information Directiory Warehouse data Middleware Design Management Data Access

35. Architecture of

46. Different between data warehouse and data mart Data warehouse Data Mart Data mart is therefore useful for small organizations with very few departments data warehousing is suitable to support an entire corporate environment. If you listen to some vendors, you may be left thinking that building data warehouses is a waste of time. data mart vendor that tells you this are looking out for their own best interests. This supports the entire information requirement of an organization. This support the information requirement of a department in an organization This has large model, wider implementation, large data and more number of users. This has small data model, shorter implementation, less data and some users.

48. Data warehousing life cycle Design Enhance prototype Operate deploy

52. Different between Data warehouse and views Data warehouse Views Data warehouse is a permanent storage data. Views are created from warehouse data when needed and it is not permanent Data warehouse are multidimensional Views are relational Data warehouse can be indexed to maximize performance. Views cannot be indexed. Data warehouse provides specific support to a functionality Views cannot give specific support to a functionality. Data warehouse provide large amount of data. Views are created by extracting minimum data from data warehouse.

55. Data Mining (cont.)

58. Application Areas Industry Application Finance Credit Card Analysis Insurance Claims, Fraud Analysis Telecommunication Call record analysis Consumer goods promotion analysis Data Service providers Value added data Utilities Power usage analysis

63. Loading the Warehouse Cleaning the data before it is loaded

64. Data Integration Across Sources Trust Credit card Savings Loans Same data different name Different data Same name Data found here nowhere else Different keys same data

65. Data Transformation Example encoding unit field appl A - balance appl B - bal appl C - currbal appl D - balcurr appl A - pipeline - cm appl B - pipeline - in appl C - pipeline - feet appl D - pipeline - yds appl A - m,f appl B - 1,0 appl C - x,y appl D - male, female Data Warehouse

66. Structuring/Modeling Issues

67. Data Warehouse vs. Data Marts

68. From the Data Warehouse to Data Marts Departmentally Structured Individually Structured Data Warehouse Organizationally Structured Less More History Normalized Detailed Data Information

69. Data Warehouse and Data Marts OLAP Data Mart Lightly summarized Departmentally structured Organizationally structured Atomic Detailed Data Warehouse Data

72. Data Mart Centric Data Marts Data Sources Data Warehouse

73. True Warehouse Data Marts Data Sources Data Warehouse

74. II. On-Line Analytical Processing (OLAP) Making Decision Support Possible

80. Visualizing Neighbors is simpler

81. A Visual Operation: Pivot (Rotate) 10 47 30 12 Juice Cola Milk Cream NY LA SF 3/1 3/2 3/3 3/4 Date Month Region Product

82. “ Slicing and Dicing” Product Sales Channel Regions Retail Direct Special Household Telecomm Video Audio India Far East Europe The Telecomm Slice

87. OLAP Operations © Prentice Hall Single Cell Multiple Cells Slice Dice Roll Up Drill Down

88. Relational OLAP: 3 Tier DSS Store atomic data in industry standard RDBMS. Generate SQL execution plans in the ROLAP engine to obtain OLAP functionality. Obtain multi-dimensional reports from the DSS Client. Data Warehouse ROLAP Engine Decision Support Client Database Layer Application Logic Layer Presentation Layer

89. MD-OLAP: 2 Tier DSS MDDB Engine MDDB Engine Decision Support Client Database Layer Application Logic Layer Presentation Layer Store atomic data in a proprietary data structure (MDDB), pre-calculate as many outcomes as possible, obtain OLAP functionality via proprietary algorithms running against this data. Obtain multi-dimensional reports from the DSS Client.

90. MSPVL Polytechnic College Pavoorchatram