The Sky's the Limit: Running MySQL on Standard Servers and in the Cloud

The Sky’s the Limit
Hints and Best Practices to Integrate and Run MySQL on Standard Servers and in the
Cloud

Ivan Zoratti
V1212.01
Wednesday, 5 December 12

Who is Ivan

?

SkySQL
•#1 provider of MySQL-based
solutions, for software and
services
•Backed by Monty Program AB
and 3rd parties active in the
MySQL ecosystem
•Committed to provide the best
solutions around MySQL for the
Community and for the
Enterprise


Agenda

• Understanding the Cloud Landscape

• Anatomy of a MySQL Database in the Cloud

• MySQL in AWS - Tips and Tricks

PAGE 4

Understanding the Cloud Landscape

The Landscape
Or maybe an aerial view...

PAGE 6

The Landscape

Public Clouds

PAGE 6

The Landscape

Public Clouds Hybrid Clouds

PAGE 6

The Landscape

Public Clouds Hybrid Clouds Private Clouds

PAGE 6

The Landscape

IaaS
Infrastructure as a Service


PAGE 6

The Landscape

PaaS
Platform as a Service

IaaS


PAGE 6

The Landscape

SaaS
Software as a Service

PaaS

IaaS


PAGE 6

The Landscape

SaaS
Level of Abstraction

PaaS

IaaS


PAGE 6

The Landscape
Control / Governance

SaaS

PaaS

IaaS


PAGE 6

The Landscape
Control / Governance

SaaS

Flexibility of Purpose

PaaS

IaaS


PAGE 6

The Landscape

PAGE 7

The Cloud Promise

• Agility • Reduced Costs

• Compatibility • Reliability

• Elasticity • Security
• Low Maintenance • Systems Independence
• Multi-tenancy • Virtualization

PAGE 8

The Cloud Promise

• Agility
F AC T!• Reduced Costs

• Compatibility
lo
•ud ]
Reliability
he C on al (*)
• Elasticity T
t[ Re •
la i
tSecurity !
nd ly
sN o ie
Fr Systems Independence
I
• Low Maintenance atabas e- •
D
• Multi-tenancy et...
no ty
• Virtualization
Or
PAGE 8

Challenges for a MySQL DBA
in the next 3 years
• Integration with other databases - primarily NoSQL
• Adopt the right technology to make MySQL and the Cloud work
together
• Integration with provisioning, administration and monitoring tools
• Increase/provide better security for the database and for the users’
data
• Provide elasticity and reduce TCO

PAGE 9

Anatomy of a MySQL Database
in the Cloud

A Mix & Match
• Deployment

• Internal / External Communication

• High Availability
• Scalability
• Daily Operations

• Special (non-InnoDB and non-Replication) features

PAGE 11

Deployment

PAGE 12

Deployment
Amazon RDS Rackspace

•One-click
•Limited oﬀering (max 4GB RAM)
•Master/Slave replicas (up to 5 replicas)
•You can get up to 30GB RAM and 1.2TB/server
•Choice of instances, IOPs, storage etc.

Amazon EC2 Google SQL

•Deployment procedure needed
•Up to 16GB RAM and 100GB storage
•Full control on technology and conﬁguration
•Can go on SSDs, High
I/O etc.

PAGE 13

Communication & Networking

PAGE 14

Communication & Networking

•AWS vs Rackspace
•550MB/min (73Mbps) •No control on Networking
160MB/min (21Mbps)
•http://bit.ly/TLrILl


•EC2 Placement Groups - 10Gbps connectivity •No control on Networking

PAGE 15

Highly Availability

PAGE 16

Highly Availability

•Internal NAS replication
•SAN Replication
•MySQL Replication
•Multi-region replication


•BYOR (Bring Your Own Replication) •Internal sync/async replication (not MySQL
Replication)
•Multi-region replication

PAGE 17

Scalability

PAGE 18

Scalability

•From Micro to XLarge instances (64GB RAM)
•No Replication or other solutions
•Dedicated Instances
•Up to 5 Read Replicas

•From Micro to XLarge instances •No Replication
•Hi I/O with SSDs and placement groups •Used for Google applications
•Dedicated Instances •Limited to 16MB for external applications
•Unlimited Replicas

PAGE 19

Daily Operations

PAGE 20

Other Features

PAGE 21

Cost Factors

PAGE 22

MySQL in AWS - Tips & Tricks

Some Tips for...

• New instances

• Tagging

• Networking
• Security
• Storage

• Backup & Restore

PAGE 24

Hidden Gems
AWS API
# Create the actual EC2 instance from the appropriate AMI using the key pair, security group, IAM role
# and user data script we previously created for this deployment

resource_id=`ec2-run-instances $ami -k $keypair -g $group -f $udscript -t $size -b /dev/sdf=:$ebssize:false -p $role
$extras 2> error | awk '$1 == "INSTANCE" { print $2 }'`

if [ "$resource_id" = "" ]; then
./logerror $session_id "Create Instance" `cat error`
(echo "Create instance failed" ; cat error) > error.$SystemId
exit 1
fi

./add_inventory "Machine Instance Node $NodeNo" $resource_id
if [ "$demo" = "1" ] ; then
./db.sh "insert into DemoResources values ('$SystemId', 'EC2 Instance', '$resource_id');"
fi

echo $resource_id >> resources

# Add tags to the instances so that they can locate each other and synchronise during the boot phase
ec2-create-tags $resource_id --tag SystemID=$SystemId --tag NodeNo=$NodeNo
--tag EIP=$EIP --tag IAM=SDS$SystemId
--tag Name="SkySQL Data Suite $SystemId Node $NodeNo"
--tag SystemStatus=pending

# Find all the EBS volumes associated with an instance and tag those so that destruction can take place
# cleanly
cnt=0
devices=`ec2-describe-instances $resource_id | grep BLOCKDEVICE | awk '{ print $3 }'`
...

PAGE 25

Hidden Gems
Userdata.sh Variable definition - they will
be used later when other
scripts are called
#!/bin/sh
export SystemId=12345
export keypair=SDS12345
export role=SDS12345
export security=SDS12345
export EIP=107.21.101.100
export DiskSize=5
export NodeNo=3
export NumberNodes=3
export InitialPassword=MyPassword
export EmailAddress="skysql.test@gmail.com"
echo Start boot script execution >> /var/log/SDS.boot.log
cat > /etc/my.cnf << MY_CNF_EOF On-the-fly configuration
files can be created
... Specific my.cnf ...

MY_CNF_EOF
if [ -d /usr/local/skysql/init.boot ] ; then
for script in /usr/local/skysql/init.boot/* ; do
echo Run: $script >> /var/log/SDS.boot.log 2>&1
sh $script >> /var/log/SDS.boot.log 2>&1
done
fi Predefined scripts are
if [ -d /usr/local/skysql/init.boot.3 ] ; then
for script in /usr/local/skysql/init.boot.3/* ; do present in the AMI
echo Run: $script >> /var/log/SDS.boot.log 2>&1
sh $script >> /var/log/SDS.boot.log 2>&1
done
fi

echo Completed boot script execution >> /var/log/SDS.boot.log

PAGE 26

Hidden Gems
Userdata.sh Work with IAMs if you
want to interact with
other instances

Use custom AMIs with
prepared scripts and
Paste your boot preloaded software for
commands here more complex
operations

PAGE 27

Creating New Instances hi1.4xlarge
(8 cores, 60GB RAM) provides
10GBit networking and
placement groups

Heavy I/O ops can benefit
from the use of EBS
optimised instances
Best fit for a non-sync replication
cluster is m2.4xlarge
(8 cores, 68GB RAM)

Use VPCs to create a VP DB
Cluster or if you have already
a VPC for your app

PAGE 28

Using AMIs

PAGE 29

Tagging

Volume tagging is also
helpful to match the
instances

Name, Node etc. are
helpful in scripts

PAGE 30

Networking

Elastic IP replaces
public IP address

It reflects the EIP

Internal IP - it can be
used for intra-instance
communication

PAGE 31

Security
Groups

PAGE 32

Security
Identity and Access Management

PAGE 33

Storage

Use Separate System
and Data storage... or
even file per tables

PAGE 34

Backup & Restore Even better:
•Attach a EBS Volume
•Direct the backup to the new volume
innobackupex •Stream to S3
--user=user •...
--password=password •Keep the last backup for emergencies
--stream=tar ./ 2>

tmp-file | S3Control.sh stream bucket backup-name

#!/bin/sh
SkySQLLIB=/usr/local/skysql/skysql_aws/
if [ ! -n "$EC2_HOME" ] ; then
export EC2_HOME=/opt/aws/apitools/ec2
fi
LIBDIR="${EC2_HOME}/lib"

for jar in "${LIBDIR}"/*.jar ; do
CP="${CP}:${jar}"
done

LIBDIR="${EC2_HOME}/lib"
for jar in "${LIBDIR}"/*.jar ; do
CP="${CP}:${jar}"
done
CP=.:${SkySQLLIB}SkySQLAWS.jar:${SkySQLLIB}aws-java-sdk-1.3.16.jar:$CP
java -classpath $CP S3Control $*

PAGE 35

Why RDS?

• One-Click solution, good feature set

• Peace of mind backup and disaster recovery

• Great if you have “few” DB Servers

PAGE 36

Why MySQL/MariaDB on EC2?
• You can choose among different technologies
• MySQL, MariaDB, Percona
• MySQL Replication vs Galera, MySQL Cluster, TokuDB, InfiniDB etc.

• You want to use High I/O instances and SSDs
• You want to control the deployment with tuned configuration
params

• You want to save between 10% and 25%
PAGE 37

Why MySQL on Rackspace?

• Fanatical about support!

• Other systems running on Rackspace - Rackspace hosting

• More control of your instances

PAGE 38

Why Google Cloud SQL?

• You develop and use applications with Google AppEngine

PAGE 39

For More Information...
• Try the Configurator and How-To Video: http://
cloud.skysql.com

• The Admin Console: http://www.skysql.com/
downloads/community/skysql-console

• Forums: https://www.skysql.com/forums/
skysql-cloud-data-suite

• Web: https://www.skysql.com/cloud

• Bugs: http://bugs.skysql.com

• Try it! http://config.skysql.com
• Free SkySQL account available for testing
• Test on your account

PAGE 40

Thank You!

ivan@skysql.com
www.skysql.com izoratti.blogspot.com
www.slideshare.net/izoratti


The Sky's the Limit: Running MySQL on Standard Servers and in the Cloud

Empfohlen

Empfohlen

Weitere ähnliche Inhalte

Andere mochten auch

Andere mochten auch (6)

Ähnlich wie The Sky's the Limit: Running MySQL on Standard Servers and in the Cloud

Ähnlich wie The Sky's the Limit: Running MySQL on Standard Servers and in the Cloud (20)

Mehr von Ivan Zoratti

Mehr von Ivan Zoratti (20)

Kürzlich hochgeladen

Kürzlich hochgeladen (20)

The Sky's the Limit: Running MySQL on Standard Servers and in the Cloud