More Related Content Similar to Planning Very Large Scale Document Repositories with High Availability in SharePoint 2013 (20) More from WinWire Technologies Inc (20) Planning Very Large Scale Document Repositories with High Availability in SharePoint 20131. Planning Very Large Scale Document
Repositories with High Availability (HA) in
SharePoint Server 2013
Wired2Win Webinar Series
WinWire Technologies Inc.
2350 Mission College Boulevard,
Suite 480, Santa Clara, California 95054
Phone/Fax: +1 (888) 849 7339
Email : info@winwire.com
WinWire Technologies, Inc. Confidential
© 2010 WinWire Technologies
@WinWire
3. Who We Are
WinWire Technologies is a
specialized IT solutions
company focused on helping
business and technology
leaders achieve an “on-themove” business environment
leveraging pre-built
collaborative and analytic
solution accelerators and
cross-platform mobile
technologies.
WinWire Technologies, Inc. Confidential
© 2010 WinWire Technologies
3
4. This is the story of Bob.
WinWire Technologies, Inc. Confidential
© 2010 WinWire Technologies
4
5. Why is Bob sad?
WinWire Technologies, Inc. Confidential
© 2010 WinWire Technologies
5
7. Things were pretty good.
WinWire Technologies, Inc. Confidential
© 2010 WinWire Technologies
7
8. So what’s the problem?
WinWire Technologies, Inc. Confidential
© 2010 WinWire Technologies
8
9. He needs to provide the following functionality to his organization.
Very Large Scale
Document
Repositories
Lowest possible
maintenance and
operations costs
WinWire Technologies, Inc. Confidential
Highest possible
availability
Predictable
“Elastic”
architecture that
can grow with the
deployment
© 2010 WinWire Technologies
9
10. He’s got many folks to please,
including IT Support, legal and
compliance.
WinWire Technologies, Inc. Confidential
© 2010 WinWire Technologies
10
11. Users are concerned.
They don’t want it to be hard.
WinWire Technologies, Inc. Confidential
© 2010 WinWire Technologies
11
12. SharePoint seems to provide the
right capabilities!
WinWire Technologies, Inc. Confidential
© 2010 WinWire Technologies
12
14. Bob isn’t sure how to proceed.
WinWire Technologies, Inc. Confidential
© 2010 WinWire Technologies
14
15. Let’s help Bob with
Planning Very Large Scale
Document Repositories with
High Availability (HA) in
SharePoint
WinWire Technologies, Inc. Confidential
© 2010 WinWire Technologies
15
18. Enterprise Content Management
Create
Control
Create and organize
content easily
WinWire Technologies, Inc. Confidential
Manage content policy,
information architecture
and taxonomy
© 2010 WinWire Technologies
18
Protect
Reduce risk and manage
compliance with
centralized tools
19. SharePoint 2013 Designing Your Farm
•
Motivation
• End-user experience is what really
matters.
• Tail end of performance is sticky.
•
Basic Idea
• Categorize applications into SLA
buckets.
• Homogenize front-end, backend,
and data layers.
• Prevent proliferation of app server
deployments.
WinWire Technologies, Inc. Confidential
© 2010 WinWire Technologies
19
20. Designing for better SLA
Latency
Request
Management
Front-End
Resource
Utilization
Desired
Behavior
Very low
< 5 msec
Very High
Medium
Consistent
Latency
Low
<500 msec
Distributed
Cache
Throughput
Medium
Low-Medium
Consistent
Latency
Low
<500 msec
Medium
Low – High
Fairly
Consistent
Latency
Batch Processing Layer
High
> 1 min
High
High-Very
High
High
Throughput
Database Layer
Very Low
< 5 msec
Very High
Low-Medium
Consistent
Latency
Specialized
Workload
WinWire Technologies, Inc. Confidential
Search
© 2010 WinWire Technologies
20
21. Evolution of topology
Schematic Diagram…
Machines could be virtual or physical.
Topologies are rough estimates.
<100 Users
< 10KUsers
1,000 Users
Routing &
Caching
Front-End
More & More Users
Search
Specialized
More Usage
Small Resources
Very Content
MoreLight And Simple
Workload
Large Content
Fault Tolerance
< 10M Items
Query
Batch
Processing
Multiple Index Partitions
Advanced Routing
Crawl
Database
WinWire Technologies, Inc. Confidential
More Workloads
Simple Workload
Evaluation
More Workloads
© 2010 WinWire Technologies
21
22. What defines your SharePoint?
WORKLOAD
Facts: purpose, services, # of users,
concurrency, rps, location
Tuning options: satellite farms, disable
services, block clients …
Facts: # of servers, hw spec, roles, network
Tuning Options: add server, scale-up, tune
settings …
Facts: #of site collections, DBs, data size
Tuning Options: split site collections,
balance content…
WinWire Technologies, Inc. Confidential
© 2010 WinWire Technologies
22
23. Microsoft’s 2013 Dogfood Farm
WORKLOAD
HARDWARE
15,000 users
Role
HW Spec
8,800 active users per week
Routing & Caching
VM, 4 cores, 14GB RAM
1.7M requests per day
Front End
VM, 4 cores, 14GB RAM
Collab, Social, Doc Mgmt, Project
Batch Processing
VM, 4 cores, 14GB RAM
Database
8 cores, 64 GB RAM
204,106 profiles in UPA
1 Web App
DATASET
SLAs (During Peak Hours)
1.3 TB total dataset size
Availability
1,001,141 documents
99.9%
10 content DBs
Largest content DB - 290 GB
Role
Avg CPU/ Mem
Routing & Caching
12% / 8 GB
511 ms
Front End
45% / 11GB
Batch Processing
80% / 12 GB
Database
11% / 46 GB
Active Users RPS
(hr)
8,297 Site collections
2,500
Largest site collection - 275 GB
WinWire Technologies, Inc. Confidential
Latency
(99th ntile)
© 2010 WinWire Technologies
23
50
25. Components – Scaling Cheat sheet
Component
Search administration
Crawler
Content processing (CPC)
Analytics processing (APC)
Index
Query processing (QPC)
CPU
3
3
Network
4
5
Disk
2
Memory
1
1.
Admin: Memory footprint increase with topology size
4.
2.
Crawler: Crawled documents are temporarily stored to
disk
APC: Network bandwidth increase with scale out, but
only in-between APC nodes
5.
QPC: Network IOPS increases linearly with number of
index partitions and QPS
3.
CPC and APC are scheduled with “below normal”
priority
WinWire Technologies, Inc. Confidential
© 2010 WinWire Technologies
25
26. Sample Worksheet
Current Size
Recommended
storage
Notes
Configuration
2 GB
The configuration database grows very slowly. Approximate growth is 40 MB per 50,000 Site
Collections
Central Administration
Content
1 GB
Very slow growth. Not expected to grow beyond 1 GB.
Content (one or more)
???
???
???
???
•
•
•
•
Functional Area (PROD)
Portal and Self Service
Legacy Data to be migrated. When migration of legacy data is performed, database sizing and
Site Collection design has to take place.
Cross Functional Area Collaboration
Usage and Health Data
10 GB
Dependent on logging level selected
Subscription Settings Service
5 GB
Dependent on number of alerts created by users
Search Administration
10 GB
Search Administration is typically small
Crawl (one or more)
50 GB
Crawl = 0.046 x (sum of content databases)
Property (one or more)
15 GB
Property = 0.015 x (sum of content databases)
Profile
10 GB
Approximately 1 MB per user profile
Synchronization
5 GB
Approximately 650 kb per user profile
Social Tagging
2 GB
Approximately 0.009 MB per tag
Staging
5 GB
Web Analytics Service
Reporting
5 GB
Web Analytics Service
Secure Store
1 GB
Word Automation Services
1 GB
Managed Metadata
10 GB
The size of the database is affected by the number of content types and keywords.
Total
WinWire Technologies, Inc. Confidential ??? © 2010 WinWire Technologies
26
28. Results from SharePoint 2013 Deployments
• 50% faster server response
• 4 X faster profile sync.
•
10 X faster w/ Active Directory import
• 40% more efficient use of
bandwidth
• 80% less file i/o related SQL IOPS
• More scalable SharePoint farms
WinWire Technologies, Inc. Confidential
© 2010 WinWire Technologies
28
29. SharePoint Farms Scale More than Ever
Product
does have
its limits
WinWire Technologies, Inc. Confidential
© 2010 WinWire Technologies
29
Limits are
for each
Dimension
Each
dimension
needs to
stay within
limits
30. SharePoint 2013 Performance Benchmarks
SharePoint 2010
benchmark on
SharePoint 2013
SharePoint 2010
SharePoint 2013
4 front-end VMs w/ 4 CPUs, 12 GB RAM
Green Zone: < 60% CPU on FEs, < 0.5 sec server response time
1 physical SQL Server w/ 8 CPUs, 32 GB RAM
Max Zone: 90+% CPU on FEs, < 1 sec server response time
WinWire Technologies, Inc. Confidential
© 2010 WinWire Technologies
30
31. SharePoint 2013 Scale Divisional Benchmark
Collaboration + social + communities Green Zone: < 60% CPU on FEs, < 0.5 sec server response time
+ meta-data navigation
Max Zone: 90+% CPU on FEs, < 1 sec server response time
WinWire Technologies, Inc. Confidential
© 2010 WinWire Technologies
31
33. SharePoint Security Model: Conceptual
Identity
Authentication
Authorization
Identities
DS
Group
Authentication
Method
Authentication
Provider
Permissions
F
User
STS
Web App
TOKEN
Permission
Level
Role
Assignment
Site
USER POLICY
Token
© 2010 WinWire Technologies
R
ANONYMOUS
ACCESS & POLICY
User
Groups
Roles
Claims
WinWire Technologies, Inc. Confidential
D
C
DS
User
Policies
33
Security Scope
Group
34. End User Security Recommendations - Inheritance
Site
Role Assignment
• Roles assignments are inherited by child objects
• Subsite, list, library, folder, item or document
Library
• Best Practice
• Use inheritance wherever possible
Role Assignment
W
WinWire Technologies, Inc. Confidential
© 2010 WinWire Technologies
34
35. End User Security Recommendations - Inheritance
1
Web Object
Site
Scope 1
1
Document Library Object
SPGroup Object FullGP
+ FullControlGP (Full Control)
Role Assignment
Folder Object
Item 1 Object
+ User 4
+ ReaderGP (Reader)
1
Item 2 Object
+ User 3
+ ContributorGP (Contributor)
1
1
SPGroup Object ContributeGP
Scope 2
Library
+ User 1
User 5 (Reader)
+ User 2
+ User 2 (Limited Access)
+ User 1 (Limited Access)
Item 3 Object
SPGroup Object ReaderGP
+ User 5
1
Scope 3
+ AD Group X
User 1 (Contributor)
Scope 4
User 2 (Contributor)
Role Assignment
Scope 5
W
AD Group X (Reader)
User 3 (Contributor)
User 4 (Full Control)
● Roles assignments are inherited by child objects
– Subsite, list, library, folder, item or document
WinWire Technologies, Inc. Confidential
© 2010 WinWire Technologies
35
36. End User Security Recommendations - Fine-grained Permissions
Site
Role Assignment
Library
X
Role Assignment
W
You can break inheritance
– Assign explicit permissions
– Dynamic security changing code redesign
– Recommendation is to modify scope architecture so
that scope membership does not cause ACL
recalculation at the parent document library and
Web. Only possible using custom code.
WinWire Technologies, Inc. Confidential
© 2010 WinWire Technologies
36
37. End User Security Recommendations - Fine-grained Permissions
Scope 1
Site
1
Web Object
Role Assignment
Folder Object
Library
1
Document Library Object
Item 1 Object
User 2 (Reader)
User 3 (Full Control)
User 6 (Contributor)
+ AccessGP1 (Limited Access)
2
Scope 2
3
User 5 (Reader)
+ User 1 (Limited Access)
X
Item 2 Object
+ User 2 (Limited Access)
4
SPGroup Object AccessGP1
Scope 3
User 1 (Contributor)
Item 3 Object
5
+ User 1
Scope 4
User 2 (Contributor)
Role Assignment
Scope 5
AD Group X (Reader)
User 3 (Contributor)
User 4 (Full Control)
W
WinWire Technologies, Inc. Confidential
© 2010 WinWire Technologies
+ User 5
37
+ User 2
+ AD Group X
+ User 3
+ User 4
39. Production Farm Disaster Recovery Architecture
WinWire Technologies, Inc. Confidential
© 2010 WinWire Technologies
39
40. Disaster Recovery Options
Standby
Type
Definition
Description
Pros
Cons
Hot Standby
A second data
center that can
provide availability
within seconds or
minutes.
Multiple data centers
Relatively fast to recover
Can be quite expensive
to configure and
maintain
Warm
Standby
A second data
center that can
provide availability
within minutes or
hours
Virtual server images
shipped to local and
regional disaster
recovery farms
Relatively inexpensive to
recover, because a virtual
server farm can require little
configuration upon recovery
Can be very expensive
and time consuming to
maintain
Cold
Standby
A second data
center that can
provide availability
within hours or days
Backups are shipped to
support bare metal
recovery to local and
regional offsite storage
on a regular basis, and
contracts are in place
for emergency server
rentals in another region
Often the cheapest option to
maintain, operationally
The slowest option to
recover
WinWire Technologies, Inc. Confidential
Content and services
through only one data
center
© 2010 WinWire Technologies
40
Often an expensive option
to recover, because it
requires that virtual servers
be configured correctly after
a disaster has occurred
42. High availability (HA) with RBS
Effects of RBS on SQL Server Database Size
Database size (100 GB)
Database data file size
(100 GB)
Database transaction log
file size (100 GB)
Size of RBS externalized
data
Database size (1 TB)
Database data file size (1
TB)
Database transaction log
file size (1 TB)
Size of RBS externalized
data
Size (GB)
Without
With
RBS
RBS
217.2
7.0
Reduction
96.8%
106.9
3.2
97.0%
111.6
3.8
96.6%
-2,292
96.2
26
-98.9%
1,120
6.5
99.4%
1,173
20
98.3%
--
1,115
--
WinWire Technologies, Inc. Confidential
© 2010 WinWire Technologies
42
43. Q&A
Next Webinar
Topic: Line of Business (LOB) + Enterprise Search
When: December 05, 2013
Time: 8AM PST
Presented by: Bill Baer and Cem Aykan - Microsoft
WinWire Technologies, Inc. Confidential
WinWire Technologies, Inc. Confidential
© 2010 WinWire Technologies
44. Thank You
Reach out to us at marketing@winwire.com
Visit us: www.winwire.com
http://www.winwire.com/winwire-blog/
www.twitter.com/winwire
WinWire Technologies, Inc. Confidential
WinWire Technologies, Inc. Confidential
© 2010 WinWire Technologies