Web & Social Media Analytics Previous Year Question Paper.pdf
Unlocking the secrets to how essbase thinks e roske in sync10 oracle epm track
1. Unlocking the Secrets to
How Essbase Thinks
Edward Roske
eroske@interrel.com
BLOG: LookSmarter.blogspot.com
WEBSITE: www.interrel.com
TWITTER: ERoske
2. About interRel
• 2008 & 2009 Oracle Titan Award winner - EPM Solution of the
year
• 2008 Oracle EPM Excellence Award
• 2009 Oracle EPM/BI Innovation Award
• One of the fastest growing companies in the world
(Inc. Magazine, ‘08 & ‗09)
• Two of the four Hyperion Oracle ACE Directors in the world
• Oracle Gold Specialized Partner in Oracle EPM
• Focused exclusively on Oracle Hyperion EPM software
– Consulting
– Training
– Infrastructure and Installation
– Support
– Software sales
6. Bob Earle Invented ―Sparse and Dense‖
• How is data distributed?
SPARSE DENSE
Products Time Periods
X X X X
Accounts
X X X X X X
Locations
X X X X X
X X X X X
X X X X X
7. Block Structure
Var%
Var
Bud
Act
UnitsSold
Headcnt
Profit
Account
(dense)
Expense
OtherExp
Marketing
Salary
Rev
COGs
Sales
Jan Feb Mar Q1 Q2 Q3 Yr
Period
(dense)
8. Data Cells Within the Block
Actual -> Profit -> Jan
Cross-dimensional operator (means Actual BY Profit BY Jan)
9. Blocks
• An individual block is created for each combination of sparse
stored members
Cola->East
Cola->New York Cola->Florida Cola->Massachusetts
11. Storage
• PAG File
– Contains the data (the blocks with a header for each)
– Contains up to 2 Gb of data in each PAG file (32-bit)
– Can be 1,024 different files
– Can be compressed and fragmented
– Can be stored on multiple drive locations
• IND File
– Contains the list or pointers to the blocks (intersection of sparse dimensions)
– Contains up to 2 Gb of index in each IND file (32-bit)
– Can be 1,024 different files
– Can be fragmented
– Can be stored on multiple drive locations
12. Compression
• ―Simple‖ compression settings
– None
– zLib
– Index Value Pair
• Can‘t assign directly
• Good for really large blocks with really sparse data
• Following types use multiple compressions (one per block)
– Bitmap
• Good for non-repeating data
• Will use Bitmap or IVP
– RLE = Run Length Encoding
• Good for data with zeros and data that repeats (such as
budgeting)
• Will use RLE, Bitmap, or IVP
13. Dimension Order & Compression
• Dimension order affects compression
• First dense dimension determines your ―columns‖ in PAG file
• Compression is from left to right, top to bottom
• Move Time to first dense dimension and we get:
BUDGET Jan Feb Mar Apr May Jun
Sales 100 100 100 120 120 120
COGS 50 50 50 50 50 50
Margin 50 50 50 70 70 70
Exp. 30 30 30 30 30 30
Profit 20 20 20 40 40 40
• Notice repeating values
• Time should be dense then Measures for RLE compression
15. Calculation Process
Accounts Jan Feb Mar Qtr1
Sales 124.71 119.43 161.93 406.07
COGS 42.37 38.77 47.28 128.42
Margin 82.34 80.66 114.65 277.65
16. Dense Calculation
After calc of Time
Year
dimension
Qtr1
Data load from table
XXXX XXXXXXX Jan
After calc of
XX ###
Accounts
XX ### Feb dimension
XX ### Mar
Sales COGS Margin Profit
17. Sparse Calculation
East -> Cola Calculated
blocks
Upper-level
blocks
Input blocks
Level zero
blocks
Vermont -> Cola New York -> Cola
18. Calculation Order: All Dimensions
• First, Accounts
• Second, Time
• Third, remaining dense dimensions
• Fourth, remaining sparse dimensions
• Two Pass Calculation
20. Commit Blocks
• Using Uncommitted Access
– When Commit Level is reached, blocks write to hard drive
• Default is 3000 blocks
• Setting Commit Blocks to Zero
– Writes at completion of the entire transaction
– Will dramatically improve calculation time
– Will fragment your PAG file during a calculation
25. BSO Limitations
• Financial applications are more densely populated so BSO works great in
those instances
• BSO engine can handle sparse data but on a ―limited‖ scale
• Outline size limited
• Batch times required for loads and calcs
26. Aggregate Storage Option
• Remember all the concepts we just learned:
– Dense / Sparse
– Index / page files
X
– Cache settings
– Block storage
27. Aggregate Storage Databases
• Similar to a BSO database – outline, dimensions, hierarchies… BUT
• Different method for storing/calculating databases
• New storage kernel built to handle ASO databases
• Calculates 10-100x faster
• Stores up to 252 dimension combinations
28. Aggregate Storage Databases
• ASO addresses the following types of databases:
– Read-only*
• Write back to level zero available in 9.3.1
– ―Rack and stack‖
– Large dimensions
• New Types of databases are possible
– Customer analysis—data is analyzed from any dimension and there are potentially
millions of customers
– Procurement analysis—many products are tracked across many vendors
– Logistics analysis—near real-time updates of product shipments
– Market Basket analysis—products purchased along with other products
29. When to use ASO
• Database is sparse and has many dimensions, and/or the dimensions have
many levels of members
• Database is used primarily for read-only purposes, with few or no data
updates
• Calculation of the database is frequent, is based mainly on summarizing of the
data, and does not rely on calculation scripts
• Starting in Essbase 11x, ASO should be your default/starting idea
31. End User Perspective
• End users won‘t care whether their database is ASO or BSO
• The way users access ASO is the same as BSO
– Excel Add-in
– Financial Reporting, Web Analysis, etc.
– Smart View Add-in
• Repeat… no differences (just more data/dimensions)
32. ASO Benefits from IT Perspective
• Faster load and calc times provide
– Lower hardware costs
– Lower maintenance costs
– Higher availability
• Small disk footprints
• Efficient tuning for storage and query response
33. How does ASO work?
• Simple question with not so simple answer
• Asked greatest minds in the business how ASO works and got the same
resounding answer:
• ―It‘s a black box‖ or ―it's top secret and hard to understand‖
• There must be a better answer!
34. ASO is Designed For…
• More dimensions and members
• Less time required for batches
– Fast aggregation of sparse data sets
– Faster loads
– Incremental loads
• Reduction in database footprint
36. ASO Storage (ROLAP in disguise)
• ASO can also be said to be ―ROLAP with a super fancy index
scheme that rules.‖
• Big difference between ASO and BSO and ROLAP is the ASO
storage mechanism
• ROLAP stores data in a table and indexes a combination key
between the rows
• Essbase stores the concept of a cube of data in multiple
dimensions or rather multiple keys
37. ASO Storage (ROLAP in disguise)
• It‘s a multidimensional index
• ASO takes it a step further and indexes the indexes in a way for more rapid
aggregation of data
• Storage is no longer in "blocks" but in highly optimized aggregation nodes
• Visualize it as an asymmetric fractal Christmas tree flattened out and then
indexed again
38. How does ASO work
• Load Data only at level 0
• Create aggregate views
• Algorithm selects and stores ―most taxing‖ queries
• Dynamic queries at runtime and increased
speed by leveraging nearest stored view
39. ASO Concepts
• Concept of stored and dynamic hierarchies
– Stored hierarchies can only aggregate
– Dynamic hierarchies can utilize formulas and advanced unary operators
• Formulas on dimension members use MDX syntax
• Pre-aggregated views can be defined to help query performance
• Aggregated design wizard helps you create aggregation scripts
• Outline paging helps you page portions of the outline in and out
of memory to assist in performance
• You can convert BSO outlines to ASO outlines using a wizard
41. Directory Structure
• Directory structure differs in both content and purpose from BSO
• Tablespaces are utilized to store data and metadata
– Default – stores numeric data (.dat file)
– Log – records database activity
– Metadata – stores metadata information about the objects in the database
– Temp – temporary working space for the Essbase kernel
42. Tablespace Overview
• Defines the database storage in the form of file locations
• Each ASO application has 4 tablespaces
– Default – database values
– Temp – temporary work space
– Log – transaction log files
– Metadata – database data structure
• File location specifies a physical disk space for storing database
files
• Each tablespace may contain one or more file locations
– Can span multiple physical drives and/or logical volumes
44. Sizing Tablespaces
• During the data load and aggregation process, data is stored in both the
Temp and Default directories
• ASO will always build the full .dat file in the temp tablespace while the default
tablespace still has the production .dat
• Hence, for your maximum drive size you have to plan on AT LEAST 3x your
max bloated .dat size if you want to be "safe" (buffer to temp to default)
45. Compression Dimension ASO
• In old releases, the Accounts dimension enabled database
compression
• Beginning in 9.3, the Accounts dimension is the default
compression dimension BUT you can choose a different
compression dimension
• Essbase helps you choose a compression dimension by
estimating what the database size would be depending on
which dimension is tagged as compression
• Time is an excellent candidate for compression dimension,
especially if you have fiscal year as a separate dimension
47. Aggregating ASO Data
• For ASO databases, after data values are loaded into the level 0 cells of an
outline, the database requires no separate calculation step
• From any point in the database, users can retrieve and view values that are
aggregated for only the current retrieval
• ASO databases are smaller than block storage databases, enabling quick
retrieval of data values
• For even faster retrieval, you can precalculate data values and store the
precalculated results in aggregations
48. Aggregations – The Down Side
• Lengthy process
• Requires extra disk space
– Sometimes yes, but it is rare that default aggs are more than 40 percent the size
of the input data.
• You want to balance query time and storage space
49. Intelligent Aggregations for ASO
• You can define hard restrictions for a dimension
– Default (no restriction for primary hierarchy, no aggregation for alternate
hierarchies)
– Consider all levels
– Do not aggregate
– Consider top level only
• (you only query top level)
– Never aggregate to intermediate levels
• (you only query level zero or top dimension)
• Level based weighting – provide levels to consider
• Process
– ASO considers hints when creating aggregations
– Attempts to create the most useful aggregations based on
hints
50. Query Hints
• You can define ―soft restrictions‖ as a query hint
• Just select a representative member (any member)
• Essbase will take this into consideration when creating aggregation views
52. Design Considerations
• BSO
– Complex calculations and allocations
– Write back at upper levels from end users are required
• ASO
– Large analysis applications with many dimensions and members
– Rolling up and analyzing large volumes of data
• Both ASO and BSO
– Take advantage of the strengths of both database types
53. Consider Using Both ASO and BSO
BSO Budget
Partition
Product ASO
SKU
Analysis
54. Consider Using Both ASO and BSO –
version 11
Product ASO
SKU
Analysis
Partition
BSO Budget
55. ASO vs. BSO Recap
• What are the similarities between ASO and BSO?
– Building dimensions
– Loading data (level zero only for ASO)
– Retrieving data
• What are the differences?
– Many dimensions, many members
– No calc scripts
– Use MDX member formulas
– Aggregations for improved query performance
• Lots of improvements in System 9 and version 11
– Understand the limitations in early versions of Essbase ASO
– Don‘t miss the new features in 9.3.1 and 11x