Exadata is still oracle

1. Data Warehousing
Exadata is Still Oracle
Richard Burns
Teradata Corporation

2. Exadata is Still Oracle
Table of Contents
Executive Summary 2 Executive Summary
Introduction 3
At the Oracle Open World conference in September 2010, Oracle
Oracle Exadata Database Machine 3
Overview introduced the third version of its Exadata platform, mainly a
Exadata Storage Server 3 hardware upgrade sporting the latest generation Sun Intel®-based
Oracle Exadata Database Machine 6 servers. Since its introduction of Exadata in 2008, Oracle has
Summary of Oracle Exadata 8 blitzed the marketplace, extolling Exadata technology with great
Architecture
fanfare, but only referencing customers executing simple batch
Exadata for Data Warehousing – 8
A Critique reports. Exadata may help Oracle to address basic 1980’s report-
Exadata is NOT Intelligent Storage 9 ing problems, but it does little to handle the complex workloads
Exadata is NOT Shared Nothing 10 and analytics demanded from today’s active data warehouses.
Exadata does NOT Enable High 11
Concurrency In our view, fundamental database issues remain with Oracle
Exadata does NOT Support 12 Exadata. Exadata still relies on the same Oracle shared disk
Active Data Warehousing
architecture that was designed to optimize transaction processing
Exadata does NOT Provide 13
Superior Query Performance
performance but which is ill-suited to manage complex data
Exadata is Complex 15
warehouse tasks and active analytic workloads.
Conclusion 17
While Exadata improves Oracle’s I/O performance, Exadata
does not tackle Oracle’s underlying performance and scalability
problems with large-scale data warehousing that stem from
its shared disk architectural foundation. Analysis shows that
resource contention continues to limit Exadata I/O performance
despite its increased I/O bandwidth. Many query operations
remain unaffected by Exadata and are subject to the same
resource sharing constraints as previously. These exhibit the same
poor performance characteristics as they did before Exadata.
EB-6164 > 0311 > PAGE 2 OF 17

3. Exadata is Still Oracle
Despite Oracle’s claims that Exadata solves Exadata still relies on the same Oracle In the end, Exadata provides far less
its performance and scalability problems shared disk architecture that was designed improvement in data warehouse perform-
in data warehousing, a close examination to optimize transaction processing perform- ance than Oracle promises.
of the Exadata architecture reveals how ance but which is ill-suited to manage
little Oracle has really changed, how few of complex data warehouse tasks and active Oracle Exadata Database
Oracle’s classic data warehousing perform- analytic workloads. Machine Overview
ance issues are addressed by Exadata, and
In this paper, we review the latest members The Oracle Exadata Database Machine is
how complex Exadata really is.
of the Exadata product family, and conclude a complete, pre-configured Oracle RAC
In the end, Exadata delivers far less that the Oracle Exadata Database Machine, database system that combines Oracle
improvement in data warehouse perform- even with these upgrades, falls well short of RAC database servers with new Exadata
ance than Oracle promises. meeting critical needs of large- and medium- Storage Servers, all hosted on an Intel
scale data warehouse users. Xeon® hardware platform produced by
Introduction Oracle’s Sun division. The latest genera-
The fundamental cause of Oracle’s data tion offers two distinct Exadata products.
In September 2010, at the Oracle Open warehousing limitations, in our analysis, In the Oracle Exadata Database Machine
World conference, Oracle introduced the remains Oracle’s shared disk architecture. X2-2 database servers contain two Intel
third generation of its Exadata database Oracle’s shared disk approach, even in its Westmere six-core processors, while in the
platform. The new edition is primarily a Exadata incarnation, continues to be a Oracle Exadata Database Machine X2-8
hardware upgrade to the latest generation poor match for the requirements of large- each server contains eight Intel Nehalem
Sun Intel-based servers. Oracle Exadata scale data warehousing. Exadata may well eight-core processors. According to Oracle,
has been successful upgrading existing be the “world’s best OLTP database,” as the X2-2 replaces Exadata V2 and is aimed
Oracle operational databases and applica- Larry Ellison claims, but Oracle’s funda- at data warehousing while the X2-8 is a
tions, but Exadata’s success in data mental dependence on a shared resource new product intended for operational
warehousing and business intelligence design limits its ability to overcome its database consolidation.
uses, especially for large-scale business- shortcomings for the more data-intensive
critical applications, is still unproven requirements of data warehousing. The Exadata Storage Servers provide an
nearly three years after its introduction. alternate storage sub-system for Oracle
Despite Oracle’s claims that Exadata solves database systems; one that is designed to
Since its introduction, Oracle has blitzed the its performance and scalability problems improve I/O performance and scalability
marketplace, extolling Exadata technology in data warehousing, a close examination for Oracle databases. Oracle uses the same
with great fanfare, but only referencing of the Exadata architecture reveals how Exadata Storage Server configuration for
customers executing simple batch reports. little Oracle has really changed, how few of both the X2-2 and the X2-8 products.
Exadata may adequately address basic Oracle’s classic data warehousing perform-
1980’s reporting problems, but it does little ance issues are addressed by Exadata, how Exadata Storage Server
to handle the complex workloads and narrow is the class of business intelligence The Oracle Exadata Storage Server pro-
analytics demanded from today’s active queries that significantly benefits from vides special-purpose storage for Oracle
data warehouses. In our view, fundamental Exadata, and how complex and costly databases. It replaces SAN or NAS storage
database issues remain with Exadata. Exadata really is. systems from third-party vendors that
have been used to provide Oracle shared
EB-6164 > 0311 > PAGE 3 OF 17

4. Exadata is Still Oracle
Oracle Before Exadata Oracle Exadata
Oracle Oracle Oracle Oracle Oracle Oracle Oracle Oracle
DBMS DBMS DBMS DBMS DBMS DBMS DBMS DBMS
Fibre Channel
InfiniBand
Array Array
Controller Controller
Exadata Exadata Exadata
Software Software Software
Array Array Array
Disk Array Disk Array Disk Array Controller Controller Controller
Figure 1. Exadata Storage architecture.
storage in the past. Exadata Storage Exadata Storage Cells are based on stan- (15K RPM) 600GB SAS disks, called the
Servers, or cells, replace the disk arrays and dard Intel Xeon processors. The same high-performance option, and another with
controllers, while an InfiniBand® network Exadata Storage infrastructure is used for 12 slower (7200 RPM), high capacity 2TB
provides connectivity to the Oracle RAC both X2-2 and X2-8 products. Exadata drives, referred to as the high capacity option.
database servers. From the viewpoint of servers employ Intel’s latest generation
The 600GB SAS storage option provides
the Oracle database, Exadata storage is processor, code named Westmere. In X2-
up to 7.2TB of spinning storage per server
treated the same as the SAN or NAS based systems, Exadata cells are built on
and provides up to 1.8 GBps of data
storage subsystem it replaces. In other Sun x4270 M2 servers that contain dual
bandwidth to the Oracle database. The
words, from the perspective of the Oracle six-core Xeon L5640 processors running
2TB SAS storage option offers up to 24TB
database, Exadata is simply another at 2.26 GHz, with 24GB of memory per
of spinning storage and up to 1 GBps of
storage device. It is managed by Oracle’s cell and 12 internal disks for data storage
data bandwidth (See Figure 2.). Oracle
Automated Storage Manager in the same connected through a PCI-mounted disk
strongly recommends the 600GB disk
way as any other storage device in an array controller. Two disk configurations
storage option for users concerned about
Oracle database system (See Figure 1.). are available, one containing 12 high-speed
query performance and reliability.
Storage Storage User Data Data Each Exadata cell also contains 384GB of
Type Capacity Volume Bandwidth flash storage implemented via four PCI-e
600GB SAS 7.2TB 2TB 1.8 GBps cards that hold 96GB of flash storage each.
2TB SAS 24TB 7TB 1 GBps
Figure 2. Oracle Exadata Storage Server capacity.
EB-6164 > 0311 > PAGE 4 OF 17

5. Exadata is Still Oracle
The Exadata flash configuration is > Adding flash storage to improve The principal innovation of Exadata is
unchanged from the previous Exadata V2 random I/O throughput for some of to migrate some query processing steps
system. By eliminating the mechanical the data. to the Exadata cells. With Exadata, Oracle
delays inherent in rotating disk technology, query processing is split into two stages,
For many years, Oracle has advocated
Exadata Flash Storage supports higher running on separate groups of servers
configuring data warehouse I/O subsys-
random I/O throughput for a portion of the connected to one another by an Infiniband
tems to deliver high sustained data rates,
data. And it is primarily used as cache space network (See Figure 3.). In the first stage,
and scaling I/O bandwidth to maintain
for data aging out of Oracle memory-based the Exadata software retrieves data into
that rate as the system grows. Oracle’s best
buffer cache. Exadata cells, performs column projections
practices, reflected in its Oracle Optimized
and row restrictions based on the SELECT
The purpose of Exadata is to improve I/O Warehouse reference architectures, specify
list and the WHERE clause predicates of
performance for both OLTP and data I/O configurations designed to meet these
the query, decompresses data and reassem-
intensive business intelligence applications. objectives. Leading Oracle customers have
bles rows as needed, and returns filtered
Exadata achieves improved I/O perform- been deploying large data warehouse
row sets across the network to the Oracle
ance in four ways by: systems that achieve these goals within
database server or Real Application Cluster
> Employing a high-speed Infiniband Oracle’s shared disk architecture with very
(RAC), where the second stage occurs.
network between every Exadata cell large, high bandwidth, and quite expensive
In the second stage, the Oracle database
and each Oracle database server. storage subsystems. So supporting parallel
performs the remaining query operations,
I/O across a high bandwidth, scalable
> Expanding the bandwidth of the storage which may include sorting, group by,
storage network is not a new concept for
network as Exadata cells are added. aggregation, and analytic operations, and
Oracle. By delivering a hardware-based
> Filtering only data of interest to execut- returns the answer set to the requestor.
solution, Exadata simply builds the storage
ing queries in Exadata processors to
network into its architecture to guarantee The Oracle Exadata Storage Server is
reduce data volume before transmitting
adequate I/O performance. connected to the Oracle RAC system via
data from Exadata cells to the Oracle
dual Infiniband links, each running at
database servers.
40 Gbps. Multiple Exadata Storage Servers
can be linked together to provide a scalable
data access layer for the Oracle DBMS,
Exadata Server Oracle Server though since Exadata V2, this approach
appears to be discouraged by Oracle. An
Exadata
Software Exadata Storage Server can contain data
Oracle
DBMS for multiple Oracle databases, and individ-
InfiniBand Software ual Oracle databases can be deployed
across clusters of Exadata Storage Servers.
Exadata Layer Oracle DBMS Layer
Oracle Automated Storage Management
provides software mirroring for data
Figure 3. Exadata two-layer query architecture.
EB-6164 > 0311 > PAGE 5 OF 17

6. Exadata is Still Oracle
protection. The Exadata Storage Server Westmere six-core processors in both The Oracle Exadata Database Machine X2-2
comes with Oracle Enterprise Linux, the Oracle database and Exadata servers. delivers an Optimized Warehouse configu-
Oracle Exadata Storage software, and And it increases the memory on the Oracle ration designed to deliver a peak I/O
management software already installed. database nodes. No software changes are bandwidth slightly greater than the 260MB
Each Exadata Storage Server is a self- added, and the Exadata storage devices per second per processor core that Oracle
contained server, holding database data remain the same as in Exadata V2. Oracle advocates for best system performance.
and running Exadata Storage software. is targeting the X2-2 product at data
Each Oracle RAC database server comes
warehousing applications.
Exadata is currently available only for with Oracle Database 11gR2 Enterprise
Oracle Enterprise Linux installations. The Oracle Exadata Database Machine Edition pre-loaded and includes software
Oracle systems deploying Exadata Storage X2-2 is available in several configuration components that Oracle strongly recom-
Servers are required to run Oracle 11g, sizes. A full cabinet Oracle Exadata mends for large-scale data warehousing:
Release 2 (11gR2) or higher. Database Machine includes the following > Real Application Clusters
components packaged in a standard
Oracle Exadata Database > Partitioning
19-inch rack (42U):
Machine > Hybrid Columnar Compression
> Eight Oracle RAC database servers
The primary Exadata product however, is > High availability options
running Oracle 11gR2 software on Sun
a pre-configured, appliance-like database
x4170 M2 servers with dual Intel Xeon > Enterprise Manager Diagnostics and
machine that integrates Exadata Storage
X5670 quad-core processors running at Tuning Pack
Servers and Oracle RAC Database Servers
2.93 GHz and 96GB of memory
in a single system connected via a high- Each Oracle Exadata Database Machine
speed Infiniband communication fabric. > Fourteen Exadata cells configured with X2-2 full cabinet can hold up to 100TB of
In the latest generation, Oracle offers two 384GB of PCI mounted Flash Cache spinning disk with 600GB SAS disks, and
distinct Exadata products, the X2-2 and and either 12 x 600GB SAS or 12 x 2TB up to 336TB with 2TB SATA drives (See
the X2-8. SAS disks Figure 4.). Oracle claims that up to eight
> Three Sun Infiniband switches for cabinets can be connected to expand
Oracle Exadata Database Machine X2-2
scalable inter-processor communications system capacity before additional external
The Oracle Exadata Database Machine
> Ethernet network for client Infiniband switches must be added. Field
X2-2 is a hardware refresh of the previous
communications experience to date however, has rarely
Exadata V2 system. It replaces V2’s
spotted even a two-cabinet Oracle Exadata
Nehalem quad-core processors with
Database Machine devoted to a single
database. Multi-cabinet Exadata configura-
Storage Storage User Data Data
tions appear mainly for hosting multiple
Type Capacity Volume Bandwidth
databases on dedicated servers in a
600GB SAS 100TB 28TB 25 GBps
consolidated hardware cluster.
2TB SATA 336TB 100TB 14 GBps
Figure 4. Oracle Exadata Database Machine capacity.
EB-6164 > 0311 > PAGE 6 OF 17

7. Exadata is Still Oracle
Oracle Exadata Database Machine X2-8 and reliability problems that have con- power as a full cabinet X2-2 system. The
The Oracle Exadata Database Machine X2-8 fronted large RAC environments. Large X2-8 has the same Exadata storage as the
is a new Exadata product based on the Sun SMP servers still command a price pre- X2-2 product (See Figure 4.), and Oracle
x4800 server. In recent product offerings, mium, however, and the X2-8 is no claims that up to eight X2-8 cabinets can
Oracle has stressed the price-performance exception. The X2-8 hardware costs 50% be linked together. Like the X2-2, the X2-8
benefits of clusters of smaller servers like more per cabinet than the equivalently product comes with Oracle Database
the Exadata X2-2 configuration and powered X2-2 system. Software and 11gR2 preloaded.
previous Exadata generations. The X2-8 maintenance costs are also higher due to
Oracle’s target market for the Exadata
represents a return to Oracle configura- the database license requirements for the
X2-8 product is operational database
tions based on large processor count SMP higher core count on the X2-8 system.
consolidation, so is not expected to be
servers. It allows Oracle to offer more
The Oracle Exadata Database Machine widely used for data warehousing. As a
processing power and more memory on
X2-8 starts at a full cabinet configuration, result, our analysis will focus primarily
smaller RACs, thus reducing the scalability
containing two Sun x4800 servers, and is on the X2-2 product.
available in full cabinet increments. The
Hybrid Columnar Compression
X2-8 combines the Sun 4800 Oracle
On the software front, Oracle has added
database servers with the same Exadata
Hybrid Columnar Compression (HCC),
storage infrastructure as the X2-2 system.
an Oracle 11gR2 feature exclusively for
A full cabinet Exadata X2-8 system
Exadata systems. HCC works by taking
contains these components:
rows in adjacent data blocks, called a
> Two Oracle RAC database servers compression group, vertically partitioning
running Oracle 11gR2 software on Sun them by column, compressing each
x4800 servers (5U) with eight Intel column partition, and storing the resulting
Xeon X7560 eight-core processors compressed column partitions side-by-
running at 2.26 GHz and 1TB of side in one or more data blocks as needed.
memory
While Exadata SmartScan can operate on
> Fourteen Exadata cells configured with
compressed data, subsequent query
384GB of PCI mounted Flash Cache
processing requires decompression and
and either 12 x 600GB SAS or 12 x 2TB
reassembly of rows. This typically requires
SAS disks
multiple I/Os, perhaps as many as the
> Sun Infiniband switches for scalable number of blocks in a compression group,
inter-processor communications and significant processing power to
> Ethernet network for client communi- decompress requested columns and
cations reassemble column values into rows.
A full cabinet Exadata X2-8 system
HCC offers two compression modes – query
contains 128 cores and 2TB of memory,
and archive. Query mode is prescribed for
Figure 5. Oracle Exadata Database Machine. and has roughly the same processing
active data, with archive mode reserved
EB-6164 > 0311 > PAGE 7 OF 17

8. Exadata is Still Oracle
for dormant data. Archive mode yields a features. HCC also effectively disables row the use of Intel Westmere processors, and
higher compression ratio at much higher level locking within the compression unit demonstrates minimal added value from
cost to compress and decompress. by acquiring the same locking level on all the database software itself. With Exadata
rows in the compression unit as the most X2, Oracle continues to throw even more
Oracle claims that HCC in query mode
stringent lock request. Because of these hardware to address limitations of their
provides up to 10X compression with low
limitations, Oracle recommends HCC for database software. As the next section
impact on query performance. Like any
use with static data only. In other words, shows, the additional hardware does not
compression technique, user mileage will
the HCC feature is not applicable to solve the fundamental problem Oracle has
vary, but skepticism is certainly warranted.
actively updated data warehouses required with data intensive analytic workloads.
For real-world data warehouses it seems
for operational business intelligence.
reasonable to expect that fitting ten pounds
Exadata for Data
of data in a one-pound bag will be rare. Summary of Oracle Exadata
Warehousing – A Critique
What we have seen to date is roughly half Architecture
that compression at best. Compared to the previous Exadata With Exadata, Oracle claims to deliver
generation, in the X2-2 product Oracle superior performance for data warehouse
By shrinking data volume, HCC reduces workloads at a lower price, lower even
has modestly upgraded the hardware per
the I/O necessary to process data-intensive than inexpensive data warehouse appliances
cabinet, including more powerful Intel
queries. In many cases however, the cost of from Teradata Corporation or Netezza.
processors and more memory in the
decompression and row reassembly offsets According to Oracle, this is possible
database tier. Oracle has also introduced
the I/O savings, resulting in little, if any, because Exadata combines intelligent
a new “fat SMP” Exadata configuration,
performance benefit to either CPU time or data filtering and fast, scalable data
the X2-8, which appears to be targeted at
query response time. This may be part of movement with the sophisticated Oracle
operational database consolidation rather
the motivation for Oracle’s decision to database. The combination allows Oracle
than data warehousing.
limit availability of the HCC feature to to deliver “the world’s fastest database
Exadata systems, which has ample processor Oracle claims only 20 percent improve- machine,” according to Ellison.
power in the Exadata layer. Consequently, ment in disk I/O performance, the limiting
the main benefit of HCC is likely to be factor in scan performance. The latest However, a closer look at the Exadata
storage savings rather than improved generation includes no upgrade to the architecture leads to a very different
query performance. Smart Flash Storage system, so like conclusion. Our analysis indicates that:
previous Exadata systems, the flash device > Exadata does not put query intelligence
The Exadata HCC feature has other costs
throughput, although nominally higher closer to the storage than competitive
that limit its use. HCC can only be applied
than disk, is limited by the rate at which products such as the Teradata® Database.
at bulk load time. SQL inserts are not
the processors can ingest data to be > Exadata is not a shared nothing system
eligible for compression using HCC.
approximately the same rate as disk like the Teradata Database so it contin-
Updates to HCC compressed data cause
throughput, and thus offers no additional ues to wrestle with scalability issues
the compression unit to revert to row
throughput advantages. caused by resource contention in its
organization, either in uncompressed
shared resource environment.
format, or compressed using other, less As a result, the data warehouse perform-
aggressive Oracle compression options, ance improvements Oracle claims with > Exadata does not enable high con-
some of which are optional, extra cost Exadata X2-2 come almost entirely from currency due to its shared resource
EB-6164 > 0311 > PAGE 8 OF 17

9. Exadata is Still Oracle
architecture, a problem the addition
Oracle Teradata
of flash storage does little to help.
> Exadata does not support active data Oracle Server
warehousing because its key query RAC
performance optimizations are DBMS
ineffective when forced to handle ASM
concurrent updates.
> Exadata does not provide superior
query performance because it only Teradata Node
Exadata Server
addresses a small part of the query CPU CPU
CPU CPU
performance issues Oracle faces, and
that part shrinks as queries grow
Array
more complex. Controller Array
Controller
> Exadata is complex because it adds
multiple layers of query processing
requiring significant hardware
resources to attempt to work around
its architectural limitations for data- Figure 6. Intelligent Storage: Oracle versus Teradata Database.
intensive decision support workloads.
> Exadata is expensive to purchase and
complex analytics, standard reports, ad-hoc fact, because a Teradata node has more
operate.
queries, and continuous updates. storage network connections, it can deliver
The bottom line? The Oracle Exadata data to the processors more than two
The next sections examine Exadata’s
Database Machine is simply throwing times faster than Exadata does.
architecture to illustrate its capabilities
hardware – lots of hardware – at what is
and limitations. Unlike Teradata Database, which performs
fundamentally a software problem.
all query processing within the nodes,
Exadata is NOT Intelligent
Our analysis shows that the Teradata Data Exadata cells only perform initial column
Storage
Warehouse Appliance, which addresses the projections from the select list, and row
Under the covers, an Exadata cell is a
same data warehouse appliance market restrictions using the WHERE clause
standard Intel-based server, just like a
segment as Exadata, outperforms the Oracle predicates. Exadata can also perform some
Teradata node, and accesses data in the
Exadata Database Machine with less hard- fact table row restrictions based on joined
same way (See Figure 6.). Both read
ware at a lower price. Further, our analysis dimensions (for star schema joins). The
database data from a disk array into server
shows that Exadata cannot achieve the active Exadata cells transmit the resulting row
memory to process it. In Exadata, the disks
data warehouse capabilities of the Teradata sets to the Oracle database server, where all
and array controller are contained within
Active Enterprise Data Warehouse to enable other query operations are executed, just
the server, while for the Teradata platform,
enterprise data integration, fully accessible by as they have always been. This includes
the disk subsystem is in a separate enclo-
thousands of concurrent users performing operations such as aggregation, sorting,
sure – a simple packaging distinction. In
EB-6164 > 0311 > PAGE 9 OF 17

10. Exadata is Still Oracle
analytic operations, data transformations, shared disk into a shared-nothing archi- policy. So in fact, not only can each query
updates, temporary table creation and tecture – one that minimizes the number process read from all disks, under SAME
processing, locking and concurrency control, of shared system resources to reduce time data allocation, all query processes will
as well as row and column restrictions too spent waiting for other processes to finish read from all disks.
complex for “SmartScan” processing in using them. While superficially Exadata
Oracle advocates the SAME allocation policy
Exadata. Even simple reporting queries cells run independently of each other –
for data warehousing because it believes
usually require other query operations just as disk arrays never interact with one
that in its shared disk environment this
such as aggregation, and all of these are another – every Oracle database process
policy optimizes access performance
performed in the database server, not the still must have access to all database
across diverse access patterns to different
Exadata server. data. Remember, to the Oracle database,
tables. While it’s possible to control data
Exadata is simply another storage device.
In an Exadata environment, the storage allocation manually, as the number of
path between disk and database server may Distribution of data on Exadata storage is tables grows, the complexity of specifying
well be shorter than in a classic large-scale managed by Oracle’s Automatic Storage data placement manually becomes quickly
Oracle SAN environment with multiple Manager (ASM). By default, ASM stripes unmanageable. For these reasons, Oracle
layers of storage switches. On the other each Oracle data partition across all users are likely to rely on ASM’s automated
hand, while Exadata offloads I/O opera- available disks on every Exadata cell (See data allocation strategy.
tions and initial filtering from the database Figure 7.). This produces thin stripes on
By contrast, Teradata Database assigns
server, compared to Teradata Database, it all disks for every partition. Oracle calls
every data partition to a distinct set of
introduces latency that lengthens query this allocation policy Stripe and Mirror
disks, and each data partition is owned
response time even for the simplest queries, Everywhere, or SAME, and ASM automati-
by a separate database process. At query
because it requires data to be transmitted cally implements the SAME data allocation
between the Exadata cell and the Oracle
database server during query execution.
Oracle Teradata
To put it simply, Teradata Database puts
Worker 1
Worker 2
Worker 3
Worker 4
all query intelligence as close to the data as
AMP 1
AMP 2
AMP 3
AMP 4
Exadata’s initial data filtering operations.
In addition, those query operations perfor-
med in the Oracle database server are now
further away from the data than the compa-
rable operations in Teradata Database. Exadata Exadata Exadata
1
Exadata is NOT Shared Nothing 2
3
Exadata improves parallel I/O performance 4
and speeds up data retrieval compared to 1 2 3 4
previous Oracle versions, but Exadata does
Figure 7. Architecture: Oracle shared disk versus Teradata shared nothing.
not magically transform Oracle from a
EB-6164 > 0311 > PAGE 10 OF 17

11. Exadata is Still Oracle
execution time, each Teradata process, Exadata does NOT Enable parallel slaves belonging to the second
called an Access Module Processor (AMP), High Concurrency query read data, they will also access all
reads data from its own disks into its own Increasing the number of users accessing the same disks. In the best case, only three
memory, without contending with other the data warehouse requires the ability to of them can begin their I/O operations
AMPs for resources. handle an expanding volume of concurrent before some thread encounters an I/O
queries and to service the rapidly growing Wait condition (See Figure 8.).
In Oracle, as a consequence of SAME policy,
I/O demand this entails. The enterprise-
every parallel slave or worker running part Additional I/O requests from the parallel
class SAS disks used by Exadata, rotating at
of a parallel query will request data from all query slaves of other concurrent queries
15K RPM, are capable of delivering data to
Exadata cells. This means that even within a will be forced to queue, waiting for the
requestors at about 120 MBps. To maximize
single parallel query, the individual query I/O requests from queries 1-3 to complete.
I/O throughput, Exadata reads data off
worker processes may all request data from While these I/O requests are queued,
disk in large chunks. Exadata defaults to
the same disks concurrently. This produces forward progress on the requesting queries
4MB data blocks. At a 4MB block size,
potential contention on each disk for disk is stalled, and processors for both the
30 concurrent I/Os saturate a drive, even
head location and I/O bandwidth. So even Exadata servers and the database servers
without allowances for seek time.
within a single parallel query, I/O resource may sit idle for longer periods, waiting for
contention is likely. Assuming an optimal data allocation using I/O. Higher query concurrency exacerbates
the SAME policy described above, it takes the problem, yielding longer and longer I/O
So while Exadata certainly increases the I/O
very few concurrent parallel queries to queues and less efficient processor utiliza-
parallelism of Oracle, its shared disk arch-
fully consume the data bandwidth avail- tion, affecting both throughput and query
itectural foundation remains unchanged.
able in an Exadata system. Consider four response time. In a large enterprise data
Exadata does nothing to reduce or eliminate
concurrent queries running eight-way warehouse environment, more than 1,000
the structural contention for shared
parallel. The eight parallel slave processes concurrent queries may be active. Despite
resources that fundamentally limits the
of the first query will read concurrently the large number of processors in both the
scalability – of data, users, and workload –
from all the Exadata disks. When the eight Exadata cells and the database servers, an
of Oracle data warehouses.
DB DB
ASM ASM
Parallel
Query 1 Exadata
Disk
Parallel
I/O Queue
Query 2
Exadata Exadata Exadata Parallel
Query 3
Parallel
Query 4
...
Figure 8. Concurrent I/O to an Exadata disk.
EB-6164 > 0311 > PAGE 11 OF 17

12. Exadata is Still Oracle
Oracle Exadata system will be quickly over- hit ratio. In addition, data accessed via a Oracle maintains multiple versions of
whelmed by workload of such magnitude. table scan, the preferred access method updated data blocks, each representing its
in Exadata, occupies a separate memory data contents at a different point as updates
Caching of database pages only provides
area outside the buffer cache, so as not to to the block occur. Oracle distinguishes
modest mitigation of the concurrent I/O
flush the cache too rapidly. Thus, it’s not different versions of a data block using a
bottleneck intrinsic to Exadata systems for
available for storing in Flash Cache. Also, System Change Number (SCN), which is
two principal reasons. First, the memory
updates to cached data require that the an Oracle system-wide value that incre-
allocated to buffer cache is such a small
Flash Cache be refreshed from disk. This ments with every transaction. An SCN
fraction of the storage volume, typically
undercuts any concurrency benefits in the represents the state of the database at a
1/5th of one percent. A basic sales report-
continuous update environment common particular point. Data blocks contain the
ing application, for example, needs to
for active data warehouses. In other words, SCN of the transaction that last updated
service numerous concurrent requests
Oracle Flash Cache benefits OLTP applica- them. Queries are assured of obtaining a
from managers of different stores, for
tions, for which it was designed, but has consistent view of the data by only access-
distinct sales data that would generate
limited benefit for data warehousing. ing versions of data blocks with SCNs
rapid buffer flushes. Second, in a large-
equal to or less than the current SCN at
scale enterprise data warehouse covering Exadata does NOT Support
the time the query begins (See Figure 9.).
a number of subject areas, highly diverse Active Data Warehousing
queries yielding unpredictable access Enterprise data warehouses that support The software algorithms that maintain this
patterns predominate, limiting the benefits operational business intelligence applica- logically consistent view of data for every
of caching. Even with caching, the Exadata tions demand intraday updates. The latency query run in the Oracle database layer, not
architecture suffers from a concurrent between events and the ability of the in the Exadata layer. Since it relies on
I/O bottleneck that limits its ability to business to respond intelligently to them shared database structures, such as Oracle
adequately support even modest concur- is continually shrinking. Even single rollback segments, the database buffer
rent query levels. application data marts and special purpose cache, and global lock data structures, for
analytic data warehouses are increasingly access to previous versions of data blocks
Oracle Flash Cache and Concurrency
requiring online updates to respond rapidly (with lower SCNs), this logic can only be
Oracle Flash Cache, added in Exadata V2,
to recent events. Exadata’s performance run from a software layer that has a global
partially mitigates, but does not solve
benefits are seriously compromised in an view of the Oracle instance.
Oracle’s concurrency problem for data
online or active update environment.
warehouse queries. Flash Cache provides a Exadata filtering occurs prior to SCN
second level cache to hold data flushed out To ensure data integrity, databases must checking. For tables or partitions being
of Oracle’s main-memory-resident Buffer give each query a consistent view of the actively updated, Exadata simply does not
Cache. While the Flash Cache is larger data it needs. Typically this is accomplished know whether the version of each data
than the Buffer Cache, it is still smaller by ensuring that a query only sees the state block it reads is the correct version for
than the active data in a busy enterprise or contents of the database at the time it the query requesting it, so the Exadata run-
data warehouse. While data can be pinned begins execution. Oracle uses a multi- time system will disable SmartScan filtering
in Flash Cache, diverse access patterns in a version concurrency control (MVCC) in these cases. Exadata simply passes
large-scale data warehouse lower the cache mechanism to manage a logically consis- unfiltered blocks to the database server,
tent view of data for every query.
EB-6164 > 0311 > PAGE 12 OF 17

13. Exadata is Still Oracle
Select… DB DB
(SCN 12033)
ASM ASM
12033 Data Blocks Rollback
12029 Segment
12035 12033
12033
12035 12033 Exadata Exadata Exadata
12029
12033
Scan Path
...
Figure 9. Oracle transaction processing with Exadata.
which performs the version check to specification, which may drastically reduce aggregated results all happens in the Oracle
determine whether or not it has the right query data volume, is not useful for tables database layer (See Figure 10.). The overall
version of the block before applying column or partitions being actively updated. For performance of this query depends on the
projections and WHERE clause restrictions. active data warehouse or operational efficiency of both the operations executed
In some cases, Oracle may also need to do business intelligence environments, these by Exadata and the operations performed
additional I/O to retrieve the correct version are precisely the tables and partitions you in the Oracle database layer.
of the data block from the Oracle Rollback want to benefit fully from all performance
As queries become more complex, Exadata
Segment. In these increasingly common enhancement that Exadata provides.
continues to perform the needed filtering,
active update cases, the benefits of Exadata
Exadata does NOT Provide but even more of the query operations, and
SmartScan are eliminated.
Superior Query Performance a greater share of the resources consumed,
Improving I/O parallelism may yield large Query performance involves all SQL take place in the Oracle database layer.
improvement in I/O performance. How- operations, from scanning data, to sorting A more complex query, for example,
ever, filtering, which can easily offer a 100 and grouping, to complex OLAP opera- may need to perform one or more non-
times or greater reduction in data volume, tors. Any query, even a relatively simple partition-wise joins and complex OLAP
has a much larger effect on query per- one, involves several operations unaffected operations, all of which will happen in the
formance. While Exadata still performs by Exadata. For example, a single table database layer (See Figure 11.). In general,
parallel I/O for the query, the largest reporting query will perform column and as the complexity of queries grows, the
benefit provided by Exadata, early filtering row filtering in Exadata, but grouping, work performed by Exadata shrinks as a
of columns and rows meeting the query local aggregation, and global merging of proportion of the total work of the query.
EB-6164 > 0311 > PAGE 13 OF 17

14. Exadata is Still Oracle
Thus, Exadata has the biggest performance
impact on the simplest data warehouses, Query: Show Total Sales by Store for YTD YTD
Query: Show Total Sales by Store for
such as single application data marts with Oracle Server
few tables arranged in a simple star schema. 7 1. Query coordinator dispatches parallel
Oracle Exadata query plan.
Conversely, Exadata exhibits a significantly 1 DBMS 6 2. Each parallel slave requests exadata
diminishing benefit with more complex Software to read sales partitions from disk.
2 5 3. Exadata filters columns and rows.
data warehouse environments, those with
> Store, date, amount
complex multi-subject schemas, running > For current year
complex queries and ad-hoc data explo- 4. Exadata sends matching tuples to
parallel slaves.
ration in a mixed workload environment Exadata Server
5. Parallel slaves perform local group
that includes online updating and opera- by and sum.
Exadata
tional business intelligence applications. In 3 4 6. Parallel slaves send sums by store
Software
to query coordinator.
other words, as query and data complexity
7. Query coordinator performs global
grow, the benefits of Exadata diminish. merge and returns result set to requester.
The standard performance difficulties that
Oracle data warehouses typically encounter
as the scale of data, users, and workload Figure 10. Simple query processing in Exadata.
increases, none of which is addressed by
Exadata, grow to dominate the perform-
ance profile of the data warehouse (See Query: Show Sales by Store by
Figure 12.). The inconsistent performance Customer Age, Sex, Income
patterns that plague Oracle data warehouses
Oracle Server
remain. For large-scale EDW workloads, 9 1. Query coordinator dispatches parallel
8
the work performed post-scan, and there- Oracle Exadata query plan.
1 DBMS 7
fore out of Exadata’s scope, is likely to 2. Each parallel slave requests exadata
Software to read sales and customer data from disk.
overwhelm any performance contribution 2 5 6 3. Exadata filters columns and rows.
provided by Exadata. > Store, date, amount for current year
> Customer demographics
4. Exadata sends matching tuples to
In summary, Exadata benefits appear to be
Exadata Server parallel slaves.
limited to a small fraction of the work a 5. Parallel slaves redistribute customer
medium- to large-scale data warehouse data for join.
Exadata
3 4 6. Parallel slaves perform join.
needs to be able to perform both efficiently Software
7. Parallel slaves perform local group
and scalably. As a whole, data warehousing by and sum.
spans a wide variety of query characteristics 8. Parallel slaves send sums by store to
query coordinator.
that can be categorized into five stages of
9. Query coordinator performs global
data warehouse maturity. As data warehous- merge and returns result set to requester.
ing practice matures over time, higher stages,
involving more complex business questions Figure 11. More complex query processing in Exadata.
EB-6164 > 0311 > PAGE 14 OF 17

15. Exadata is Still Oracle
and more demanding technical require-
ments, play an ever larger role in the mix of
100
work performed on the data warehouse.
Resource Allocation (%)
Compared to pre-Exadata Oracle technol-
75
ogy, Exadata offers the largest impact on
Oracle performance for entry-level data Oracle DBMS
marts with simple schemas and few users. 50
Exadata I/O improvements are seriously
Exadata Cells
compromised at higher levels of query
concurrency, however, and virtually non- 25
existent at concurrency levels common in
large-scale EDWs. The early data reduction
provided by Exadata-level filtering of rows Low High
Query Complexity
and columns is turned off in the face of
concurrent updates to the tables or parti- Figure 12. Exadata impact on query performance.
tions being queried, a basic requirement of
operational intelligence applications, such
as customer service or impact analysis of
excellent performance results at every In addition, there is nothing in Exadata to
marketing promotions. Users should expect
stage of data warehouse maturity. address the complex challenges of using
that for most of the work performed in
parallelism in an Oracle environment. Now
mature data warehouses, Exadata will show Exadata is Complex
we add to that the challenges of balancing
no better than a neutral performance Exadata achieves its limited benefits at the
workload and resource utilization across
impact (See Figure 13.). high cost of increased architectural
a separate architectural layer. Plus new
complexity. A single cabinet Oracle
Unlike Exadata, Teradata’s unified query analysis and tuning will be needed to
Exadata Database Machine X2-2 contains
execution supports very high levels of determine appropriate I/O block sizes, I/O
an eight-way RAC system running eight
parallelism for all query operations; its workload types, and other factors, to use
independent instances of Oracle 11gR2
shared nothing model has allowed Teradata the Exadata storage layer effectively and
that must all share data. All of the chal-
to demonstrate excellent scalability charac- efficiently. Clearly, Oracle does not gain
lenges of managing workloads and shared
teristics across multiple dimensions – the ease of management, virtualization of
data access in a RAC data warehousing
data, users, and workload – in the same computing resources, and linear scalability
environment remain. There are very few
database. And its query execution architec- that has been, and continues to be, the
Oracle customers using eight-way RAC
ture is not constrained in the face of active hallmark of Teradata systems. In fact, they
environments for data warehousing. And
updates. These characteristics are among have created a system that is even more
with two cabinets of Oracle’s Database
many that have enabled Teradata to claim difficult for DBAs to manage.
Machine X2-2, that jumps to a 16-way
numerous customers who are achieving
RAC system.
EB-6164 > 0311 > PAGE 15 OF 17

16. Exadata is Still Oracle
Exadata’s design involves a two-layer query Requiring far more hardware resources to hardware at what is fundamentally a data-
execution engine that requires substantial support a two-tiered query engine, Exadata base software problem. At its core, Oracle
processing power. To achieve modest is inherently more complex. Balancing work is a shared disk system and has always
improvements in peak scan performance across the two layers adds new challenges to struggled to achieve consistently high
and user data capacity, the Oracle Exadata the already extensive DBA tuning workload. levels of performance for data intensive
Database Machine requires far more It has more components to maintain. More analytic databases. Exadata offers some
computing resources than Teradata components mean higher failure rates. Both improvement in I/O performance com-
Database requires. imply higher DBA labor requirements. The pared to earlier Oracle versions, but does
large hardware requirement of the Exadata not fundamentally alter this architectural
The Teradata Data Warehouse Appliance,
implementation is also more expensive to constraint. As a result, despite a large
on the other hand, manages significantly
acquire and has much higher power and hardware commitment, Exadata is likely
higher levels of parallelism and balanced
cooling requirements, so it’s also more to suffer similar performance limitations
resource utilization that deliver much
expensive to operate. in large-scale data warehouse applications
more performance per server than the
to pre-Exadata versions of Oracle.
comparable Exadata appliance product. Such large disparity in hardware resources
suggests that Oracle is simply throwing
ACTIVATING
MAKE it happen!
OPERATIONALIZING
WHAT IS happening?
Workload Complexity
Exadata Teradata
PREDICTING Poor Excellent
WHAT WILL
happen?
Exadata Teradata
Poor Excellent
ANALYZING
WHY
did it happen? Exadata Teradata
REPORTING Poor Excellent
WHAT
happened?
Exadata Teradata
Fair Very Good
Exadata Teradata
Very Good Very Good
Data Sophistication
Figure 13. Exadata performance impact on the five stages of data warehouse maturity.
EB-6164 > 0311 > PAGE 16 OF 17

17. Exadata is Still Oracle
Teradata.com
Conclusion Exadata Database Machine packages this same resource sharing constraints, and so
into a pre-configured system to avoid the continue to exhibit the same performance
Exadata technology and the Oracle
poor user configurations that commonly characteristics as before Exadata. As data
Exadata Database Machine address a
lead to Oracle performance problems. warehouse systems grow in scale and
serious problem that Oracle faces with
complexity, the part of the performance
data warehouse processing. Data ware- As a result, Exadata-based systems offer
problem addressed by Exadata shrinks in
houses are much larger and far more data higher data throughput than most previ-
size and importance.
intensive than the standard transaction ous Oracle versions achieved. While in this
processing applications for which Oracle sense Exadata is a better Oracle, it is far Whatever performance improvements
has a well-deserved reputation. Data from the groundbreaking innovation that that Exadata achieves come at the cost of
warehouse systems must be able to service Oracle claims. Exadata does not tackle increased software complexity due to the
requests for large data volumes efficiently Oracle’s underlying performance and introduction of a two-level query architec-
and be able to scale effortlessly to accom- scalability problems with large-scale data ture and significant additional hardware.
modate expanding workloads – tasks that warehousing that stem from its shared Exadata throws a lot of hardware at what
have historically exposed the limits of disk architectural foundation. Analysis is essentially a database software limitation
Oracle’s shared disk architecture. With shows that resource contention continues handling data intensive analytic work-
Exadata, Oracle combines higher band- to limit Exadata I/O performance despite loads. The resulting configuration is more
width, more scalable networks with its increased I/O bandwidth and paral- costly to acquire, more expensive to
smarter software that allows early filtering lelism. Many query operations are operate. In the final analysis, Exadata
to reduce query data volume. The Oracle untouched by Exadata, are subject to the delivers far less than promised.
Raising Intelligence is a trademark, and Teradata and the Teradata logo are registered trademarks of Teradata Corporation and/or its affiliates in the U.S. and
worldwide. Intel and Xeon are registered trademarks of Intel Corporation. InfiniBand is a registered trademark and a service mark of the InfiniBand Trade Associa-
tion. Teradata continually enhances products as new technologies and components become available. Teradata, therefore, reserves the right to change
specifications without prior notice. All features, functions, and operations described herein may not be marketed in all parts of the world. Consult your Teradata
representative or Teradata.com for more information.
Copyright © 2010-2011 by Teradata Corporation All Rights Reserved. Produced in U.S.A.
EB-6164 > 0311 > PAGE 17 OF 17