Measures of Dispersion and Variability: Range, QD, AD and SD
Practical MS SQL Introduction
1. www.formatech.com.lb
Microsoft SQL Server
Design, Performance and Reporting
Version 1.4 - 08
.
By Naji El Kotob.
Certified Trainer
Information Technology Consultant
_______________________________
naji@dotnetheroes.com
2. Database
◦ Definitions and Fundamentals
◦ Relational Database Objects
◦ B-Tree
Microsoft SQL Server
◦ Design and Architecture
Database Design
◦ Relationships
◦ Database Normalization
SQL – Structured Query Language
◦ DML
◦ JOINS
Inner, Outer, Full, Cross and self
Optimization and Performance Tips
Microsoft SQL Server Development Tools
◦ Business Intelligence Development Studio
Resources and References
Outlines
By Naji El Kotob {naji@dotnetheroes.com}
4. An organized collection of data.
Database?
By Naji El Kotob {naji@dotnetheroes.com}
5. DEMO
◦ [Real Case Studies]
Database?
By Naji El Kotob {naji@dotnetheroes.com}
6. There are two main types of database;
flat-file and relational.
Database Types
By Naji El Kotob {naji@dotnetheroes.com}
7. RDBMS?
By Naji El Kotob {naji@dotnetheroes.com}
8. The "relation" comes from the fact that
the tables can be linked to each other.
Relational
By Naji El Kotob {naji@dotnetheroes.com}
9. DBMS is the software that controls the
organization, storage, retrieval, security
and integrity of data in a database.
DBMS [Database Management System]
By Naji El Kotob {naji@dotnetheroes.com}
10. A type of database management
system (DBMS) that stores data in the
form of related tables.
Source: http://www.webopedia.com
RDBMS
By Naji El Kotob {naji@dotnetheroes.com}
11. One major advantage of the relational
model is that, if a database is designed
efficiently, there should be NO duplication
of any data; helping to maintain database
integrity.
Customers
Orders
Relational Model
By Naji El Kotob {naji@dotnetheroes.com}
13. Database objects are tables, indexes,
views, triggers, procedures…
Database Objects
By Naji El Kotob {naji@dotnetheroes.com}
14. Tables are the basic component of a
relational database (RDB). It has a
number of rows and columns, similar to a
spreadsheet.
Database Tables
By Naji El Kotob {naji@dotnetheroes.com}
15. A database index is a data structure that
improves the speed of operations in a
table.
Indexes can be created using one or more
columns, providing the basis for both
rapid random lookups and efficient
ordering of access to records.
Database Index
By Naji El Kotob {naji@dotnetheroes.com}
25. Primary Filegroup
Data Log
.mdf .ldf
MSSQL: Design and Architecture
By Naji El Kotob {naji@dotnetheroes.com}
26. Primary Filegroup
Data
.mdf
New Filegroup
Data Log
Data
.ndf .ldf
.ndf
MSSQL: Design and Architecture (Cont’d)
By Naji El Kotob {naji@dotnetheroes.com}
27. SQL Server uses a write-ahead log
(WAL), which guarantees that no data
modifications are written to disk before
the associated log record is written to
disk.
◦ Source: http://msdn.microsoft.com/en-us/library/ms186259(SQL.90).aspx
Log Files: WAL
By Naji El Kotob {naji@dotnetheroes.com}
28. 1 Data modification is
sent by application Modification is recorded
3 in transaction log on disk
Buffer
Cache
Disk
Disk
Data pages are located in,
2 or read into, buffer cache
and modified Checkpoint writes
4 committed transactions
to database
Log Files: Transaction Logging
Image source: http://www.vovisoft.com/SQLServer2000/images/HowTransactionLogWorks.gif (Nov. 2008)
By Naji El Kotob {naji@dotnetheroes.com}
30. Relationships
By Naji El Kotob {naji@dotnetheroes.com}
31. Unique Key:
◦ It creates non-clustered index by default
◦ It allows only one null value
◦ Multiple Key(s) per table
Primary Key:
◦ It creates clustered index by default
◦ It doesn't allow nulls
◦ One key per table
Foreign Key
◦ A key used in one table to represent the value of a
primary key in a related table. While primary keys must
contain unique values, foreign keys may have duplicates.
Tables: Keys
By Naji El Kotob {naji@dotnetheroes.com}
33. One record in Table-1 has one and only
one matching record in Table-2.
Relationships: One-to-One
By Naji El Kotob {naji@dotnetheroes.com}
34. One record in Table-1 can have multiple
matching records in Tabl-2, but only one
in Table-1.
Relationships: One-to-Many
By Naji El Kotob {naji@dotnetheroes.com}
35. Each record in Table-1 and Table-2 can
have multiple matching records in either
table.
Relationships: Many-to-Many
By Naji El Kotob {naji@dotnetheroes.com}
37. Normalization
By Naji El Kotob {naji@dotnetheroes.com}
38. Normalization is the process of efficiently
organizing data in a database.
◦ Eliminating redundant data (e.g. storing the same
data in more than one table)
◦ Ensuring data dependencies make sense (only
storing related data in a table).
What is Normalization?
By Naji El Kotob {naji@dotnetheroes.com}
39. Project Project Employee Employee Department
ID Name ID Name
123 Payroll 5 Naji IT
8 Dani Finance
134 CRM 7 Sandra IT
5 Naji Marketing
2 Nada HR
Repeating groups of
similar data
ZERO Normal Form
By Naji El Kotob {naji@dotnetheroes.com}
40. Project Project Employee Employee Departmen
ID Name ID Name t
123 Payroll 5 Naji IT
8 Dani Finance
134 CRM 7 Sandra IT
5 Naji Marketing
2 Nada HR
NF1: Project Project Employee Employee Departme
A row of data cannot ID Name ID Name nt
contain repeating 123 Payroll 5 Naji IT
groups of similar data
(atomicity) 123 Payroll 8 Dani Finance
134 CRM 7 Sandra IT
Each row of data must
have a unique identifier 134 CRM 5 Naji IT
(or Primary Key). 134 CRM 2 Nada HR
First Normal Form
By Naji El Kotob {naji@dotnetheroes.com}
41. Project Project Employee Employee Department
ID Name ID Name
123 Payroll 5 Naji IT
123 Payroll 8 Dani Finance
134 CRM 7 Sandra IT
134 CRM 5 Naji IT
134 CRM 2 Nada HR
NF2: Project Project
No Partial ID Name
Dependencies on a 123 Payroll Employee Employee Department
Concatenated Key
134 CRM ID Name
5 Naji IT
8 Dani Finance
7 Sandra IT
2 Nada HR
Second Normal Form
By Naji El Kotob {naji@dotnetheroes.com}
42. Project Project Employee Employee Department Department column:
ID Name ID Name Rely on non-key attributes
123 Payroll 5 Naji IT
Can it exist independent
134 CRM
8 Dani Finance of the Employee ID
column?
7 Sandra IT
2 Nada HR
Project Project
ID Name
Employee Employee
123 Payroll ID Name
134 CRM 5 Naji
8 Dani
NF3: 7 Sandra Department Department
It contains no transitive ID Name
2 Nada
dependencies 1 IT
(No Dependencies on
Non-Key Attributes). 2 Finance
3 HR
Third Normal Form
By Naji El Kotob {naji@dotnetheroes.com}
46. SQL is a standardized query language for
accessing/manipulating data and schema
in relational database management
systems, such as Microsoft SQL Server,
Oracle, etc.
SQL
Data/Feedback
SQL (Structured Query Language)
By Naji El Kotob {naji@dotnetheroes.com}
48. Northwind Sample
Database
SELECT * FROM INFORMATION_SCHEMA.TABLES
WHERE TABLE_TYPE = 'BASE TABLE‘
Customers
Shippers
Suppliers
Orders
Products
Order Details
CustomerCustomerDemo
CustomerDemographics
Region
Territories
EmployeeTerritories
sysdiagrams
Employees
Categories Source:
http://www.microsoft.com/downloads/details.aspx?Fam
ilyID=06616212-0356-46A0-8DA2-
EEBC53A68034&displaylang=en
By Naji El Kotob {naji@dotnetheroes.com}
49. The SQL SELECT statement is the SQL
command that retrieves data from an SQL
database. This operation is also known as
a query and is the key to the use of SQL
for analysis and reporting.
SQL: Select Statement
By Naji El Kotob {naji@dotnetheroes.com}
50. Let’s experience it ;)
SQL: Select Statement
By Naji El Kotob {naji@dotnetheroes.com}
51. SELECT * FROM Customers
SQL: Select Statement
By Naji El Kotob {naji@dotnetheroes.com}
52. SELECT ContactName + ' (' + CompanyName + ')'
FROM Customers
SQL: Select Statement (Concat)
By Naji El Kotob {naji@dotnetheroes.com}
53. SELECT SUBSTRING(CustomerID, 1, 3) + ' - ' +
CompanyName FROM Customers
SQL: Select Statement (Substring)
By Naji El Kotob {naji@dotnetheroes.com}
54. SELECT DISTINCT Country
FROM Customers
SQL: Select Statement (DISTINCT)
By Naji El Kotob {naji@dotnetheroes.com}
55. SELECT DISTINCT Country
FROM Customers
Order by Country ASC -- or DESC
SQL: Select Statement (ORDER BY)
By Naji El Kotob {naji@dotnetheroes.com}
56. SELECT OrderID, ProductID, (UnitPrice *
Quantity * (1 - Discount)) AS SubTotal
FROM [Order Details]
SQL: Select Statement (Expression)
By Naji El Kotob {naji@dotnetheroes.com}
57. SELECT ORDERID,
SUM((UnitPrice * Quantity * (1 - Discount))) AS
SubTotal
FROM [Order Details]
GROUP BY ORDERID
SQL: Select Statement
(Aggregate Functions)
By Naji El Kotob {naji@dotnetheroes.com}
58. SELECT o.CustomerID,
(SELECT SUM((UnitPrice * Quantity * (1 - Discount))) AS
SubTotal
FROM [Order Details]
WHERE OrderID = o.OrderID)
FROM Orders o
SQL: Select Statement (Aggregate
Functions + Subquery)
By Naji El Kotob {naji@dotnetheroes.com}
59. SELECT ORDERID,
ROUND(SUM((UnitPrice * Quantity * (1 - Discount))), 2) AS
SubTotal
FROM [Order Details]
GROUP BY ORDERID
HAVING SUM(UnitPrice * Quantity * (1 - Discount)) > 500
SQL: Select Statement (Aggregate
Functions + Filtration)
By Naji El Kotob {naji@dotnetheroes.com}
61. ◦ = Equal
◦ <> Not Equal
◦ > Greater Than
◦ < Less Than
◦ >= Greater Than Or Equal
◦ <= Less Than Or Equal
◦ _______________________
◦ TOP # [PERCENT]
◦ BETWEEN
◦ LIKE %, _, [], [^]
◦ IN
◦ IS NULL / IS NOT NULL
◦ NOT, AND, OR
◦ …
SQL: Select Statement: WHERE
By Naji El Kotob {naji@dotnetheroes.com}
66. The SQL join command is used to join
two or more tables.
SQL: Select Statement: JOIN
By Naji El Kotob {naji@dotnetheroes.com}
67. JOIN match in both tables
LEFT JOIN match in the right table
RIGHT JOIN match in the left table
FULL JOIN match in one of the tables
CROSS JOIN Cartesian Product - combines
two tables by generating all possible
SQL: Select Statement: JOINS
By Naji El Kotob {naji@dotnetheroes.com}
68. What’s the name of the top 5 customers
with their order value and order date?
INNER JOIN
By Naji El Kotob {naji@dotnetheroes.com}
69. SELECT TOP 5 C.CustomerID, C.CompanyName, O.OrderDate,
ROUND(SUM((OD.UnitPrice * OD.Quantity * (1 - OD.Discount))), 2)
AS SubTotal
FROM Customers C INNER JOIN
Orders O ON C.CustomerID = O.CustomerID INNER
JOIN
[Order Details] OD ON O.OrderID = OD.OrderID
GROUP BY C.CustomerID, C.CompanyName, O.OrderDate
ORDER BY SubTotal DESC
INNER JOIN
By Naji El Kotob {naji@dotnetheroes.com}
70. Generate a list of customers with the
number of orders?
Generate a list of all the customers with
the number orders?
Generate a list of employees and their
managers?
JOINS: DEMO
By Naji El Kotob {naji@dotnetheroes.com}
71. SELECT Customers.CompanyName,
COUNT(Orders.OrderID) AS [Number of Orders]
FROM Customers INNER JOIN
Orders ON Customers.CustomerID =
Orders.CustomerID
GROUP BY Customers.CompanyName
ORDER BY Customers.CompanyName
INNER JOIN
By Naji El Kotob {naji@dotnetheroes.com}
72. SELECT Customers.CompanyName,
COUNT(Orders.OrderID) AS [Number of Orders]
FROM Customers LEFT JOIN
Orders ON Customers.CustomerID =
Orders.CustomerID
GROUP BY Customers.CompanyName
ORDER BY [Number of Orders] DESC
LEFT JOIN
By Naji El Kotob {naji@dotnetheroes.com}
73. SELECT e.FirstName + ' ' + e.LastName AS 'Employee
Name', m.FirstName + ' ' + m.LastName AS 'Manager
Name'
FROM Employees AS e LEFT OUTER JOIN Employees AS m
ON e.ReportsTo = m.EmployeeID
SELF JOIN
By Naji El Kotob {naji@dotnetheroes.com}
74. SELECT Products.ProductName,
Suppliers.CompanyName
FROM Products CROSS JOIN Suppliers
CROSS JOIN
By Naji El Kotob {naji@dotnetheroes.com}
The flat-file style of database are ideal for small amounts of data that needs to be human readable or edited by hand…The typical flat-file database is split up using a common delimiter. http://websiteowner.info/articles/cgi/databasetypes.aspThe relational databases such as MySQL, Microsoft SQL Server and Oracle, have a much more logical structure in the way that it stores data. Tables can be used to represent real world objects, with each field acting like an attribute. For example, a table called books could have the columns title, author and ISBN, which describe the details of each book where each row in the table is a new book.A data file that is not related to or does not contain any linkages to another file. It is generally used for stand-alone lists. When files must be related (customers to orders, vendors to purchases, etc.), a relational database manager is used, not a flat file manager. Flat files can be related, but only if the applications are programmed to do so. http://www.answers.com/topic/flat-fileLike a flat file, an XML file can contain many different types of data. There are many possible ways to represent the information in a flat file using XML. For example, each field and each record could be an XML element. One advantage of using XML would be that each field is named. A disadvantage is that the file would be larger. A file containing XML is not generally called a flat file, even though it satisfies the definition. It usually is called an XML file.
Relational databases do not link records together physically, but the design of the records must provide a common field, such as account number, to allow for matching. Often, the fields used for matching are indexed in order to speed up the process. http://www.answers.com/topic/dbms-technologyData SecurityData IntegrityInteractive Query
Database integrity ensures that data entered into the database is accurate, valid, and consistent. Any applicable integrity constraints and data validation rules must be satisfied before permitting a change to the database.
tables, indexes, views, defaults, triggers, rules constraints and procedures
Each database consists of at least two files: one is a primary datafile (by default, with the .mdf extension), the other is log file(by default, with the .ldf extension).
RAID 0 (striped disks) distributes data across several disks in a way that gives improved speed and full capacity, but all data on all disks will be lost if any one disk fails. RAID 5 (striped disks with parity) combines three or more disks in a way that protects data against loss of any one disk; the storage capacity of the array is reduced by one disk.
How do I recover data from SQL Server's log files?There are several tools available for reviewing SQL Server log files, and restoring data that was accidentally lost or corrupted, say, after your last backup. Three such products are: Apex SQL Log Log Explorer SQL Log Rescue http://sqlserver2000.databases.aspfaq.com/how-do-i-recover-data-from-sql-server-s-log-files.html
Tip: Place log on separate drive for performanceIf the log contains multiple physical log files, the logical log will move through all the physical log files before it wraps back to the start of the first physical log file. http://msdn.microsoft.com/en-us/library/ms179355(SQL.90).aspxhttp://jmkehayias.blogspot.com/2008/11/database-transaction-log-part-2-how.htmlDBCC LOGINFO -- Database Consistency Checker (DBCC)
The database community has developed a series of guidelines for ensuring that databases are normalized. These are referred to as normal forms and are numbered from one (the lowest form of normalization, referred to as first normal form or 1NF) through five (fifth normal form or 5NF). In practical applications, you'll often see 1NF, 2NF, and 3NF along with the occasional 4NF. Fifth normal form is very rarely seen and won't be discussed in this article. http://databases.about.com/od/specificproducts/a/normalization.htm
The database community has developed a series of guidelines for ensuring that databases are normalized. These are referred to as normal forms and are numbered from one (the lowest form of normalization, referred to as first normal form or 1NF) through five (fifth normal form or 5NF). In practical applications, you'll often see 1NF, 2NF, and 3NF along with the occasional 4NF. Fifth normal form is very rarely seen and won't be discussed in this article. http://databases.about.com/od/specificproducts/a/normalization.htm
Next we test each table for partial dependencies on a concatenated key. This means that for a table that has a concatenated primary key, each column in the table that is not part of the primary key must depend upon the entire concatenated key for its existence. If any column only depends upon one part of the concatenated key, then we say that the entire table has failed Second Normal Form and we must create another table to rectify the failure. http://www.phlonx.com/resources/nf3/#intro
At last, we return to the problem of the repeating Department information. As our database now stands, if an employee works in more than one project then we have to input all of that employee's departmentinformation again. This is because there are columns in the Employees table that rely on "non-key attributes".
In 1986, ANSI approved a rudimentary version of SQL as the official standard, but most versions of SQL since then have included many extensions to the ANSI standard. In 1991, ANSI updated the standard. The new standard is known as SAG SQL. http://www.webopedia.com/TERM/S/SQL.htmlAmerican National Standards Institute. This organization is responsible for approving US standards in many areas, including computers and communications. Standards approved by this organization are often called ANSI standards (eg, ANSI C is the version of the C language approved by ANSI). www.incredible.co.za/services/glossary/glossary.asp
SELECT <columnlist>FROM <tablename>* = All Columns
SELECTOrd.OrderID,Ord.OrderDate,(SELECTMAX(OrdDet.UnitPrice)FROM Northwind.dbo.[Order Details] ASOrdDetWHEREOrd.OrderID=OrdDet.OrderID)ASMaxUnitPriceFROMNorthwind.dbo.OrdersASOrdA subquery is a SELECT query that returns a single value and is nested inside a SELECT, INSERT, UPDATE, or DELETE statement, or inside another subquery. A subquery can be used anywhere an expression is allowed.
Round, Ceiling, Floor, … Mathematical Functions
BETWEEN Between an inclusive range LIKE Search for a pattern % _ [^]IN If you know the exact value you want to return for at least one of the columns select * from orders where CustomerID like '[^vtbsp]%'select * from Orders WHERE OrderDate BETWEEN '1996-08-01' AND '1996-09-02'
At times you may need to join two or more tables to retrieve the information you need from your database.