Abdul Kader Baba- Managing Cybersecurity Risks and Compliance Requirements i...
Building a SharePoint Platform that Scales
1. WELCOME
Building A SharePoint
SharePoint Saturday Dayton
Dayton, Ohio
6/30/2012
Platform That Scales
Scott Hoag and Dan Usher
2. Who are we?
Scott Hoag
• Infrastructure Consultant at Applied
Information Sciences
• Jack of All Trades, Master of Some
• With over 8 years of experience, Scott has
been utilizing Microsoft based content
management solutions from MCMS 2002 to
SharePoint 2010 today
• Enjoys discussions about user adoption,
search, and world peace
• Twitter: @ciphertxt
3. Who are we?
Dan Usher
• SharePoint Engineer at Booz Allen Hamilton
• 7 years of experience with SharePoint going
back to adventures with STS 2001 and SPS
2003 to the present…
• Enjoys discussions about Claims AuthZ,
SmartCard AuthN, Atomic Molecular Optics
& the Walking Dead
• Follows the SharePoint Credo - ADIDAS
All Day I Dream About SharePoint
• Twitter: @usher
6. Monologue
• Determine your business case
• Determine your scope
• Determine your metrics and success criteria
• Determine your boundaries
7. What could go wrong?
• Logical Architectures skipped…
• Site Collections gone viral…
• Site Permissions are confusing…
• Where’d my admin access go….
• Information can’t be found…
• Search isn’t working right…
• System seems to be dragging…
Was this system intended for users?
8. System Visioning…
Content Management
Enterprise Search
Collaboration
Business Processes
Portals Findability
Extranets Business Intelligence
Workflow
Process Reporting
Forms Dashboards
9. What’s a vision look like?
• Business Goals
– What’s the context of your use for SharePoint?
– What are you trying to accomplish with
SharePoint?
• Business Structure
– Do you have a defined organizational hierarchy?
– Do you know your key organizational processes?
• System Functionality
– Do you need to be able to aggregate and roll up
data to dashboards?
10. Vision components (cont’d)
• System Functional (con’t)
– Are Workflow or Business Process Management
tools a need?
– Are forms and their lifecycles important to you?
– Integrations and interoperability with external
systems
• SharePoint as a Service
– Small order of document library
– Large order of Business Intelligence and Excel
Services
– Medium order of collaboration
11. Stepping into Contextual Thinking…
• Considerations, Tradeoffs and
Compromises to meet the Context
• Assessing the context…
…every situation has its circumstances
13. Logical Architecture and Users
• Defines the underlying information lifecycle
management framework
• Impacts usability of information and
experience
• Defines process flow
14. Logical Architecture
• What defines a logical architecture?
• Why is a logical architecture important?
• How can you really make use of a logical
architecture?
• What does a logical architecture consist of
and look like?
15. What technically makes up a logical architecture?
• Web Zones (Intranet,• Web Applications
Extranet, Internet, – Service Apps
etc.) and Policies – Subscription Groups
• Different – My Sites
Authentication – Team Sites
Models – Publishing Sites
• Content Databases • Site Collections
• Application Pools
18. How is your logical architecture affected by your requirements?
• Extranet / Public Facing Website
• Permissions models
• Authentication Schemes
• Interoperability with other applications
– Office Client, WPF, WCF, BCS, Silverlight…
• How you anticipate your users interacting
with your platform
19. What is a taxonomy?
• Taxonomy is the science (and art) of
classifying a broad range of things.
Originally used to classify plants and
animals – phylum, genus, species, etc. –
taxonomy is now applied to everything
from product inventory to web sites.
Reference: http://bit.ly/sps-ref-tax
20. SharePoint’s Site Taxonomy
• SharePoint Farms
– Nesting Paths / Excluding Paths
• Reflection of the Organization
• Requires out of the box thinking
• Federated Search
• Shared Services across Farms
• Multitenancy
21. SharePoint’s Site Taxonomy
Farm
Web Application Web Application
Site
Site Collection Site Collection
Collection
Site Site Site Site Site Site Site Site Site
22. What’s that look like?
SharePoint Monkey
(Farm)
sharepointmonkey.org
(Web App)
Work Sites
Root Site (SC)
(Excluded Path)
/
/work/
Personal Sites
Blog (Site) Search Center (Site) Development (SC) SharePoint (SC) Networking (SC)
(Excluded Path)
/blog /search /work/development /work/sharepoint /work/networking
/personal
Program Management
Dan’s MySite (SC) Laura’s MySite (SC) Todd’s MySite (SC)
(Site)
/personal/dan /personal/laura /personal/todd
/work/sharepoint/pm
23. But do I really need a taxonomy?
• Why not just deposit everything in a single
document library?
• Won’t document sets solve that problem?
• Search will solve all my problems…
– What document is authoritative?
– The document isn’t surfaced…
• End User Impact
24. Taxonomy Limitations
• Cross Site Collection Navigation
– 2007 Solution - CodeProject.com – Shared
Navigation Provider
– 2010 Solution - Roll your own solution…
• Content Types & Site Columns
– Managed Metadata Service
– Content Hubs and number of associations
• Workflows
• SharePoint Security Groups and Scopes
available through a Site Collection
25. What about permissions?
• Well it depends on the site taxonomy…
• Inheritance and Breaking it…
– …and re-inheriting it
• Web Application Policy
– Overriding all your other policies…
• Defined in a Governance Plan hopefully?
– SharePoint Groups
– AD / LDAP Groups
– Single Users
– Claims
26. Taxonomy & Logical Architecture – What’s the Bridge?
• Site collections and their web applications
– Site Collections root container for sites
– Web Applications tied to content databases
and infrastructure
• The design goals for site collections in the
model are to satisfy requirements for URL
design and to create logical divisions of
content.*
Reference: http://bit.ly/sps-ref-sc
27. Is there a secret to success?
Gather Complete
Stakeholder Planning for Determine Prototype Validate User Implement
Functional Information and Design Micro- Acceptance of Macro-
and Process Flow and Taxonomy Taxonomy Taxonomy Taxonomy
Requirements Aggregation
28. Project Plans
• How does a project plan fit into
logical architectures and taxonomies?
• Or rather…
• How does a logical architecture and
taxonomy fit into a project plan…
30. Course Corrections…
• Never too late to pin down a way ahead
• SharePoint’s flexibility provides for change
– Third party management and migration tools
– SharePoint Admin Toolkit
– Using the tools that are a part of the platform
(Features/Web Parts/WSPs)
• Migrate to a clean environment
– Utilize a proxy for access
31. Technical Requirement Considerations
• What’s the vision again? What will the
platform provide for?
– Heavy Collaboration?
– WCM Publishing?
– Workflow Application Development Platform?
• How much content will there be?
• How will it be accessed?
• What will be the level of usage?
• Are we dealing with a cross domain solution?
• Do we have SLAs to meet?
32. What are your limitations technically?
• Surrounding Infrastructure
– AD
– Network Capacity
– Boundaries
• Offline Capability
• System Memory
• IIS
– Number of Web Applications
– Number of Identity Pools
• Number of sites / site collections
33. Technical limitations (cont’d)
• DNS
• Authentication Methods
• PKI / SSL / Wildcard Certificates
• Network Interfaces / IP Addresses
• Storage
It all ends up impacting the end user experience…
34. Recovery Limitations
High Availability and Disaster Potential Potential
Automatic Readable
Recovery Data Loss Recovery
Failover Secondaries
(RPO) Time (RTO)
SQL Server Solution
AlwaysOn Availability Group - synchronous- Zero Seconds Yes 0-2
commit
AlwaysOn Availability Group - asynchronous- Seconds Minutes No 0-4
commit
AlwaysOn Failover Cluster Instance NA Seconds Yes NA
-to-minutes
Database Mirroring - High-safety (sync + witness) Zero Seconds Yes NA
Database Mirroring - High-performance (async) Seconds Minutes No NA
Log Shipping Minutes Minutes No Not during
-to-hours a restore
Backup, Copy, Restore Hours Hours No Not during
-to-days a restore
Source: Michael T. Noel - http://slidesha.re/MIjJM7
35. Conclusion I
• Each SharePoint implementation project
requires that you examine the contextual
considerations of the environment and
define a vision.
• Defining such a vision will provide goals to
work toward, to make your implementation
both successful and effective to end users.
36. Conclusion II
• Your requirements drive your taxonomy
and logical architecture...
• Which in turn drive your hardware
requirements...
• If you don't know what you're going to use
SharePoint for, start off small and scale your
farm up as you go...
• Crawl… Walk… Run…
37. Conclusion III
• What you start with on Day One isn’t what
you’re going to end up with in…
– Six months…
– A year…
– Day 472…
Remain Flexible!!!
38. Conclusion IV
User adoption in and of itself will cause your
environment to change…
…adapt to the context as it changes.
39. Come Read Our Blogs!
It’s all true… really… • Follow us…
http://psconfig.com – twitter.com/ciphertxt
http://www.spdan.com – twitter.com/usher
40. SharePoint Saturday Dayton has been made possible because of
generous sponsorship from the following friends…
41. What’s a good reference?
• Planning for Software Boundaries
– Software Boundaries and Limits -
http://technet.microsoft.com/en-
us/library/cc262787.aspx
– Capacity Management and Sizing -
http://technet.microsoft.com/en-
us/library/cc261700
Hinweis der Redaktion
SCOTT
DAN
DAN
So a lot of the time we’re working on our design and reducing our requirements into fashionable
MSTechEdShane Young and Todd Klindt held a session called “What could go wrong?”So what could go wrong with logical architectures and taxonomies?What could go wrong if you skip your logical architecture or your planning to make your system effective for your end user?Should the logical architecture and design planning be skipped because you have a vital requirement for SharePoint to be deployed immediately, your system will begin to see issues and degradation unfortunately at both a technical and information worker level.Typically the first thing to go is any sensibility of governance as to how sites are provisioned, created and permissions applied to them. Sometimes what I like to call the out of the box model is used where everything is in a single site collection and security groups become fairly painful and troublesome.Next the issue of those security groups and permissions become pain points as users learn about inheritance – something I’m sure that some of you have witnessed first hand.Then there’s the idea that the information, while readily available, cannot be found very easily by the end user – something perhaps to think about while constructing the underlying architecture of the system.Search isn’t working properly – the shared service provider or providers as the case may be isn’t setup to scope the appropriate web application(s) and users search results begin to show them data that wasn’t necessarily intended for them.And my personal favorite outcome of what could go wrong – the system begins to drag because it wasn’t planned with the purpose in mind. One of my previous clients intended to put terabytes upon terabytes of data into their SharePoint system to house their ISOs, executables and everything else that their help desk could want – needless to say after changing the allowed upload file types and upload sizes on the 2007 based system on an underlying 2003 infrastructure it became painfully aware that their content databases were bloated. Search wasn’t too happy either indexing the first few hundred gigabytes of data.
I’m sure that everyone here has seen the pinwheel of SharePoint… this is my crafty word art rendition.So as with any SharePoint implementation, figuring out the vision behind the system is fairly important. Some systems are simply like my former client, a document workspace. For other clients I’ve worked with, they’re more interested in the centralized point for collaboration to run their workflows and view KPIs of their data – relying on the collaboration capabilities of a centralized workspace and document management to handle their documents as they press through the system.When scoping for Enterprise Search, it’s a capability that sometimes is forgotten about and requires a little planning as well on the infrastructure side to make sure that there’s space for both query indices to be pushed forward as well as to ensure that the documents being placed in the system are understood size wise – Search stops reading your documents at 16 MB and requires a registry fix to direct your crawler to parse more.So thinking forward, during your requirements gathering and analysis of your client’s system, start thinking through what the considerations for a System Vision is… will it be a highly collaborative system? Or will it be full of applications and workflows leveraging SharePoint lists and document libraries? Is the intent to be more involved with content deployment and the WCM side of SharePoint for an internet facing site that will require additional security controls? Is SharePoint just being used to replace a static information portal that has WCM capability? Is it being looked at to replace Shared Drives?So remember, what are you trying to meet?
What works best for your organization?If you look at the SharePoint wheel there are six areas that they attempt to accomplish, are you trying to bite them all off at the same time or are you working to separate them out and accomplish them one at a time.Consider the context - What may work wonderfully for a small organization – task management for instance with users in groups, rolling up data to a top level site, may not work so well when you span it out across the enterprise. Think about investigating Project Server or something along these lines to handle a true task management system.
SharePoint as a Service – is this what you’re looking for? The McDonald’s like approach, I’d like a site collection with a wiki, and a web appication… similar to Microsoft BPOS, or 1and1 Hosting, or SharePointHosting.com
Client that wants office integration, but also wants to use ADFS, but also wants to use an LDAP – there will be compromises and tradeoffs that have to be assessed.So what are we going to talk about? You might say, “You’re skipping to the chase” and in some respects you’re right, but I also like to catch a vision as to what we’re working toward before actually trudging out.So stepping into the design side of things, remembering the context of your environment is the key thing. During your initial assessment and design phase of a project, you’ll make considerations, define tradeoffs, perform analysis of the user population and functional community that you’re working with and probably end up with a list of compromises that need to be made that need to be made with your stakeholders during a decision briefing.For instance, are you hosting everything locally at a single site, or are you using a distributed architecture, or are you merely buying the service of SharePoint from a hosting provideWhat capabilities are sought after?What are the environment limitations?Are you building into the cloud?
So you’re pulling together your entire vision, you’re gathering your requirements, the scope is growing and growing… and you feel like you’ve got something like this.Do you feel like you’re trying to chomp off more than you can deliver? Take a step back.Build out small and scale.On a side note, a friend of mine thought that I actually made this80/20 rule – 80% of users should be using 20% of the functionality. Don’t worry about the 20% that want to use 80%!
This ends up impacting the end user navigation and a slew of other things.Incorporating how your end user thinks will define that taxonomy which will drive the logical architectureWhat defines a logical architecture?Why is a logical architecture important?How can you really make use of a logical architecture?What does a logical architecture consist of and look like?What is a logical architecture?The purpose of investing in logical architectureis to tackle some of the complex and interrelated design questions that are most common when standing up your SharePoint environment.Hopefully, it will serve its purpose in that once you’ve got your logical architecture defined, you’ll be able to better facilitate with other engineers and developers. Additionally, as you build out your logical architecture and environment you’ll be able to more easily see what interfaces will need to be connected to. Furthermore, for those of you that live in a cube farm while you’re on client site, it’s always nice decorate with a large print off from a plotter to hang on your wall.Why is a logical architecture important?You’ve invested in the design, you know the logical data flow and integration between systems and you can now more easily map this to your taxonomy and to your physical hardware. In fact, you can better plan your hardware requirements based off of a logical architecture once you’ve got it on paper rather than blindly ordering hardware that looks really cool on the Dell web site.How can you really make use of a logical architecture? Does it need to be updated?Well, I think that question is somewhat rhetorical at this point, it’s an important artifact that will help you to better justify your scalability needs as well as resource needs when integration tasks come down the line.What does a logical architecture look like?Next slide
What is a logical architecture?The purpose of investing in logical architectureis to tackle some of the complex and interrelated design questions that are most common when standing up your SharePoint environment.Hopefully, it will serve its purpose in that once you’ve got your logical architecture defined, you’ll be able to better facilitate with other engineers and developers. Additionally, as you build out your logical architecture and environment you’ll be able to more easily see what interfaces will need to be connected to. Furthermore, for those of you that live in a cube farm while you’re on client site, it’s always nice decorate with a large print off from a plotter to hang on your wall.Why is a logical architecture important?You’ve invested in the design, you know the logical data flow and integration between systems and you can now more easily map this to your taxonomy and to your physical hardware. In fact, you can better plan your hardware requirements based off of a logical architecture once you’ve got it on paper rather than blindly ordering hardware that looks really cool on the Dell web site.How can you really make use of a logical architecture? Does it need to be updated?Well, I think that question is somewhat rhetorical at this point, it’s an important artifact that will help you to better justify your scalability needs as well as resource needs when integration tasks come down the line.What does a logical architecture look like?Next slide
It provides a logical flow to some extent of how a user enters a SharePoint instance, all the way to what site they’re accessing to the content database that is serving up the data.Web Zones – Default, Intranet, Internet, Extranet, CustomZone Polices – Deny All, Allow All, Specific groups – this is how you can blend CALs, deny all except a group of users on a particular web application or site.Authentication Providers – Integrated Windows Authentication with NTLM or Kerberos? Forms Based through an LDAP, a SQL server? A custom membership provider?Application Pools – Are you using domain accounts, local accounts to host your SharePoint instance? This is what the web application runs as. If it doesn’t have access to talk to a domain controller if you’re using domain accounts, your SharePoint instance will essentially be in the dark.Web Applications – The true way that a user really interfaces with SharePointMySites, your Collaborative sites, your Secure Content Authoring and Publishing for web content managementSite Collections – The bread and butter of SharePoint taxonomy
Available from MicrosoftYou’ll noticed that the key difference is the SSP which provides for MySites, Search, Personalization and the WCM publishing aspect of things.
When talking about taxonomy with clients, I typically talk to them about how to group things together to make best use of their site structure – organizations like your human resources, accounting, facilities, operations, etc… and also by the programs, projects, cross functional groups, and so on and so forth. More often than note we find a need for developing a taxonomy that provides for initiatives to live outside of the taxonomy of a site collection.Taxonomy of dataTaxonomy or organizationTaxonomy of functional groupsTaxonomy is can be defined as a classification schemes of like things, defining a structure of knowledge and information, providing a common language for users to better navigate.So remember, while the taxonomy of a system may be something that is planned by a technical resource, we’re working to make it available for our end users to be able to find their data most easily.Remember that we’re humans, most of the time dealing with folks that aren’t always as technical as we are. Structuring a taxonomy to be as user friendly as possible is key, especially if the user base hasn’t gone through some semblance of training is a must.
So what’s a site taxonomy?Well, a site taxonomy is composed of web applications – you may just have one such as wsshacker.net.Or you may have several, where you’ve got names such as operations.wsshacker.net, teams.wsshacker.net, my.wsshacker.net where you’ve split our content and taxonomy, and potentially functionality through the use of CNAMES in DNS. This is pretty easy to do but requires that you do a little more coordination with your DNS management team or with your service provider.Another option you might choose is the way of wsshacker.net/operations, wsshacker.net/teams, wsshacker.net/myAs you can see, all of these are completely fine, each has there benefit, each as their limitation. If you’re going to scale out extremely large where you need to partition things out, I’d recommend going the way of defined CNAMES with separated out web applications. If you’re probably going to maintain a small organization’s system with limited taxonomy changes, I’d recommend going the way of a single URL with nested site collections (the draw back being that you must create them manually).
So what’s a site taxonomy?Well, a site taxonomy is composed of web applications – you may just have one such as wsshacker.net.Or you may have several, where you’ve got names such as operations.wsshacker.net, teams.wsshacker.net, my.wsshacker.net where you’ve split our content and taxonomy, and potentially functionality through the use of CNAMES in DNS. This is pretty easy to do but requires that you do a little more coordination with your DNS management team or with your service provider.Another option you might choose is the way of wsshacker.net/operations, wsshacker.net/teams, wsshacker.net/myAs you can see, all of these are completely fine, each has there benefit, each as their limitation. If you’re going to scale out extremely large where you need to partition things out, I’d recommend going the way of defined CNAMES with separated out web applications. If you’re probably going to maintain a small organization’s system with limited taxonomy changes, I’d recommend going the way of a single URL with nested site collections (the draw back being that you must create them manually).
Web ApplicationsSite CollectionsSitesNested Site CollectionsExcluded Paths.net 2 assists greatly as it will actually pick up virtual folders and exclude them from the path use if they’re there – something that you couldn’t do with SharePoint Portal Server 2003, at least not easily.
1 – Content types and site columns, there are boundaries that you probably don’t want to exceed.2 – System constraints and performance degradation as your content databases become enormously large. Admin DR / Search / End User ViewsWhy not just search for everything?First – Again, constraints with the number of files that reside in the document library.Second – File permissions become even more of a hassle, consider if you have a fairly large organizationThird – Authoritative data, what’s the correct copy. SharePoint unfortunately still maintains separate copies of files – no single instance, you’re increasing your storage requirements.
What about permissions? How do they affect taxonomy?Inheritance of permissions is a big reason to keep users all within a single site collectionIt’s also a reason to break things out – you’ll end up with 50 user groups that are all distributed throughoutBreaking inheritance, copies the site permissions that are existing already – if you have a huge user base, this could be an issue when you’re cleaning folks outWe had an issue with Inheritance and breaking it, primarily with Project Server
What’s that really mean? Basically the logical architecture is the model to define your SharePoint architecture, what it looks like and how it operates.
From the overarching stakeholders, gather what they are looking to do, whether it’s organizationally based, functionally, application or process basedAggregate the capabilities and begin specing out how they match to SharePoint sites and site collectionsDetermine the overarching taxonomy of sites and permissions – use Mark Miller’s Mind Map site collection builder as a model available from End User SharePointPrototype the taxonomy on a microcosm scale to ensure that it will work and that the security permissions that are in place will be usableValidate the taxonomy in a virtual environment perhaps, showing them the navigation, the overarching flow of the taxonomyImplement the taxonomy on a macrocosm level of the web application(s) and begin end user deploymentsHighly recommend Mark Miller’s tool to allow for site collection deployment buildouts.
Scope Creep… your taxonomy might start off looking like this… and end up looking like this and require you to go out and buy a migration tool or put your administrators feet to the fire.Logical Architecture and Taxonomy should begin near the beginning of the project. Without a logical architecture in place, it’s difficult to create the different mappings that determine your taxonomy and furthermore your integration into other systems.You’ll end up with something where you’ve got no rhyme or reason to your taxonomy, publishing sites mixed with team sites, mixed with site collections and random templates used across the board.While the Microsoft project plan doesn’t explicitly say Logical Architecture, it speaks to many facets of it.
There is nothing that states to build a logical architecture, but there is significant planning – all of which would get rolled into your logical architecture.While this project plan isn’t “everything” that you’re going to do – for small implementations you might not have quite as much, but for larger organizations you might go above and beyond this.
What will the system be doing?Dependent on it’s purpose determines what the roles installed may be.How big will the system be?Will you have lots of flat site collections or several deep site collectionsHow will it be accessed?What’s the authentication method? Are you on an ID island?Are we dealing with a cross domain solution?Do we need federation? Can there be a trust in place?What will the usage levels be?Will you have lots of users going through and performing searches constantly? If so, you might determine that your logical architecture and roles are slightly different.SQL mirroring or clustering?More something to note in your logical architecture to assist your implementation engineer.
The surrounding infrastructure will impact your logical architecture and taxonomy significantly. If you have a system that has a gigabit backbone switch for the SharePoint enclave itself, but you’re working in a datacenter that still uses CAT-3, you’re going to be limited.Do you have a network boundary surrounding your system? Is there a Bluecoat proxy? A load balancer available? Split DNS available?Offline - Likewise, if you’re working for a commercial company that wants to be able to take data and replicate it, this will affect your taxonomy – perhaps you’ve got locked down information sites in your taxonomy because of it with a few sites that you take offline with Groove, Infonic, Outlook or Colligo.System Memory – if you’re running x64, then you should be able do quite a bit with your SharePoint system – there are limits to how much memory you can allocate to an individual worker process.Number of sites and site collections – rule of thumb is not to go greater than 50,000 site collections within a web application, and no greater than 50,000 sites below each site collection - ~250k sites. If you see your implementation getting larger than this, you’re going to need to branch out your sites and partition them separately.DNS – Do you have management? Can you create CNAMES / A RECORDS? Are you able to create MX records?Authentication Methods – Are you forced to use a third party LDAP that has groups available? Are you utilizing something for external users?PKI / SSL / Wildcard – Do you have certificate services available? Have you defined how you will utilize SSL?Network Interfaces – Are you able to do network load balancing with multicasting? Unicasting? Do you have a limited number of IP addresses?Storage – It’s not really a logical architecture consideration, except that you probably need to consider that if you don’t have a large storage array for your environment, you may not be able to scale your system up, thereby redefining what your system is actually used for.
Courtesy of Michael T. Noel @michaeltnoelcco.com
User Experience and process drives taxonomyTaxonomy drives the logical architecture and partitioningIn turn drives your hardware requirements
