Presentation on how to chat with PDF using ChatGPT code interpreter
Performence tuning
1. - MySQL has overwritten your parameter and given the file an extension, as
Verified by the runtime value.
- We can easily check this by looking at the server uptime and the server error
log.
--- Do I need to commit after DML?
By default, auto_commit=1, which means it will command after each
Statement.
--- How to list tables from another database?
2. MySQL> show tables from information_schema;
-- How to retrieve the DDL for a table?
MySQL> show create table database.tablename;
- How to retrieve list of indexes of a table?
MySQL> show index from database.tablename;
- How to create an index?
MySQL> create index employees_pk on employees (employee_id);
-- How to monitor mysql status?
3. Mysql> show status; (It gives Total 312 Rows selected)
Mysql> show status like '%conn%';
--- How to monitor thread status? (Similar to v$process&v$session)
Mysql> show processlist;
Mysql> show full processlist;
-- How to kill a process (based on ID retrieved from “show processlist”)?
Mysql> kill 25;
-- How to “spool” the output to a file?
Mysql> tee output.txt
4. Logging to file 'output.txt'
Mysql> select * from database.tablename;
Mysql>notee
Outfile disabled.
-- How to execute OS command inside mysql command prompt?
Mysql> system date
Wed Jul 28 22:50:01 SGT 2010
Mysql> ! date
Wed Jul 28 22:50:04 SGT 2010
- How to run a SQL file inside mysql command prompt?
Mysql> source test.sql
Mysql> . test.sql
--How to cancel a partial finished SQL statement?
Mysql> select
-> c
Mysql>
- How to retrieve your session status?
Mysql> status
5. -- How to dump data into a text file?
Mysql> select * from sample1.emp into outfile '/tmp/emp.txt';
-- How to load data from text file into table?
Mysql> load data infile '/tmp/emp.txt' into table sample2.emp;
6. -- How to list global variables? (It returns 317 Rows)
Mysql> show global variables;
-- How to list session variables?
Mysql> show session variables; (It returns 329 Rows)
--- How to retrieve on 1 row (or n rows) from a table?
Mysql> select * from sample1.emp limit 1;
--- How to turn on query log and slow query log?
[root@hostname ~]# cat /etc/my.cnf
[mysqld]
datadir=/var/lib/mysql
8. --- Mysql> set global slow_query_log=on;
--Mysql> show variables like 'slow_query_log_file';
-- Mysql> show variables like 'long_query_time'
---Mysql> set global long_query_time=3;
-- Mysql> select sleep (10) from mysql.db limit 1;
9. --- Permanent settings in my.ini (my.cnf in Unix/Linux)
# The MySQL server
[mysqld]
slow-query-log = 1
slow_query_log_file = C:mysql-advanced-5.5.13-win32dataDonghua1-
slow.log
long_query_time = 3
--- The "mysqld is alive" message tells you that your MySQL server is running
Ok. If your MySQL server is not running, you will get a "connect ... failed"
Message.
- To know what else you can do with "mysqladmin", you should run the "-?"
[root@INVIRH54DB3 ~]# mysqladmin -?
Process list:
10. MySQL–Performance Features
Some of the features provided by MySQL that help to make it a high performing and
fast responsive server
Flexibility to choose the most appropriate Storage Engine as per performance
requirements. A table’s Storage Engine can be changed later also, using the ALTER
TABLE syntax.
The feature EXPLAIN PLAN can be used to find out the actual path used by a query to
fetch data from tables, so that query optimization can be performed.
Flexibility to choose the most appropriate data type as per performance and storage
requirements. For example, a number type of data can be represented by INT, TINYINT,
SMALLINT, MEDIUMINT, BIGINT, FLOAT or DOUBLE, depending on its characteristics
and the range of values it can have, and consequently, the storage requirement for
each is also different (1byte, 2bytes, 3bytes, 4bytes etc)
Table maintenance utilities like mysqlcheck can be used to perform activities like
checking, repairing, analyzing and optimizing tables to help improve performance.
There are commands available too, like Analyze table, or Optimize table, that can be
used to update key value distribution or reclaim unused space.
For string data types, it is also possible to index on a column prefix rather than the
entire width of it, which means that it is possible to create an index on a specified
width of a column. For example, if a name column is of 255 characters, and using a
query, we find that the first 10 characters of each row are sufficient to obtain distinct
values for most of them, then an index can be created using only the first 10 characters
11. of each row. This will not only allow more values to be cached in memory due to its
small size, but also can improve index performance dramatically.
The Leftmost Index prefixing feature can be used to avoid unnecessary indexes on
tables. For example, if a composite index is created on columns A and B of a table (in
the same sequence), then if a query requires an index on column A, it can use the same
composite index without the need for a separate index. However, if the sequence had
been different in composite index, or had the other query needed an index on B, then
we would require a separate index.
MySQL also provides Engine specific optimizations. For example, the following
optimizations are possible on MyISAM tables.
Tables can be defined to have fixed-length or dynamic-length row format. Fixed-length
allows data to be stored at a multiple of the row size resulting in faster access, whereas
dynamic-row columns occupy smaller space, but can cause fragmentation.
Read-only tables can be compressed to a very small size using myisampack utility. The
compression is done in an optimized form that allows quick retrievals.
It is also possible to split a Dynamic-row table into two separate tables (one fixed-
length and the other dynamic length) keeping the Primary Key as common to both, so
as to gain the advantages of both types. This is usually considered when most queries
access the fixed length columns of a dynamic row table.
To distribute disk activity, symlinking can be used to move MyISAM tables to different
disks.
The MySQL server commands like STATUS can be used to obtain a snapshot report of
current server status, showing its complete details, like current load, slow queries,
open tables etc.
The server parameters that define various cache and buffer sizes can also be tuned as
per performance requirements
12. Performing Administrative Tasks: -----
(1) Database Management :
Alter Database :
ALTER DATABASE DATABASE1 DEFAULT CHARACTER SET utf8 DEFAULT COLLATE
UTF8_BIN;
(2) Security
(i) Create SSL User
CREATE USER USER1 IDENTIFIED BY 'qwerty';
GRANT USAGE ON *.* TO USER1 REQUIRE SSL;
(ii) Create user
CREATE USER USER1 IDENTIFIED BY 'qwerty';
(iii) Create User with privileges
CREATE USER USER1;
GRANT ALL PRIVILEGES ON *.* TO USER1;
SET PASSWORD FOR USER1 = Password('test');
FLUSH PRIVILEGES;
(iv) Create X509 user
CREATE USER USER1 IDENTIFIED BY 'qwerty';
GRANT USAGE ON *.* TO USER1 REQUIRE X509;
(v) Grant
GRANT RELOAD, PROCESS ON *.* TO USER1@localhost;
(vi) Revoke
REVOKE ALTER, UPDATE ON *.* FROM USER1@localhost;
EVENTS
(i) Alter Event
ALTER EVENT EVENT1 ON SCHEDULE EVERY 12 HOUR STARTS CURRENT_TIMESTAMP +
'2006-02-10 23:59:00' ENABLE;
(ii) Create Event
CREATE EVENT IF NOT EXISTS EVENT1 ON SCHEDULE AT '2008-02-10 23:59:00' ON
COMPLETION NOT PRESERVE DISABLE COMMENT 'comment this event' DO
(iii) Create Event with interval
13. CREATE EVENT IF NOT EXISTS EVENT1 ON SCHEDULE EVERY 1 DAY STARTS '2006-02-10
23:59:00' ON COMPLETION PRESERVE ENABLE COMMENT 'comment this event' DO
(iv) Drop Event
DROP EVENT IF EXISTS EVENT1;
Enable slow query log in mysql
When it comes to optimizing and tuning MySQL the most important aspect is to
identify the inefficient/slow queries.
How we can find the queries which are taking long time to execute so we can
optimize/improve them to improve the overall performance.
MySQL helps us with its built in support for logging slow queries.Activating the slow
query logging:
We need check if slow query logging is already enabled or not , it can be checked as
below :
mysqladminvar |greplog_slow_queries
| log_slow_queries | OFF
If it’s already set to ON then you are set, if its set to OFF like above then you will need
to enable slow query logging.
The MySQL variable long_query_time (default 1) defines what is considered as a slow
query. In the default case, any query that takes more than 1 second will be considered
a slow query.
Now to enable the slow query logging we will need following entries in
the /etc/my.cnfmysql configuration file.
[mysqld]
long_query_time = 1
log-slow-queries = /var/log/mysql/mysql-slow.log
You can define the path for logging according to your requirements. Also the log query
time which is by default 1 sec can be adjusted according to your needs.
14. Once you have done the configuration, restart MySQL service to load the new
configurations.
Once slow query logging is enabled we can check the log file for each slow query that
was executed by the server.
Different details are logged to help you understand how the query was executed:
Time: the time it took to execute the query
Lock: how long was a lock required
Rows: how many rows were investigated by the query
Host: this is the actual host that launched/initiated the query
Query: The actual MySQL query.
This information will help us to see what queries need to be optimized.
Calculate the size of innodb_buffer_pool_size and key_buffer_size
As a DBA sometime you will be working on Performance
Optimization/Tuning/Configuration Optimization.
You must be working to tune RAM of existing/New server. Keep in mind below points.
To working with transactional database you need to configureinnodb_buffer_pool_size.
It will cache Indexes as well as data.
Use below query to check the size:
SELECT CONCAT (ROUND (KBS/POWER (1024, IF (PowerOf1024<0, 0, IF (PowerOf1024>
3, 0, PowerOf1024)))+0.49999),SUBSTR('KMG',IF(PowerOf1024<0,0, IF(PowerOf1024>3,
0, PowerOf1024)) +1, 1)) recommended_innodb_buffer_pool_size FROM
(SELECT SUM (data_length+index_length) KBS FROM information_schema.tables
WHERE engine='InnoDB') A, (SELECT 2 PowerOf1024) B;
If you are working with MyISAM engine, you can calculate size of key_buffer_sizeusing
the below query. It will provide you approximate size of key_buffer_size. It cache only
MyISAM Indexes.
SELECT CONCAT (ROUND (KBS/POWER (1024, IF (PowerOf1024<0, 0, IF (PowerOf1024>
15. 3, 0, PowerOf1024)))+0.49999),SUBSTR('KMG',IF(PowerOf1024<0,0, IF(PowerOf1024>3,
0, PowerOf1024)) +1, 1)) recommended_key_buffer_size FROM
(SELECT LEAST(POWER(2,32),KBS1)KBS1 FROM information_schema.tables
WHERE engine='MyISAM' AND table_schema NOT IN
(‘information_schema’,’mysql’))AA)A, (SELECT 2 PowerOf1024) B;
-How to view Table engines using Information_schema for a table.
MySQL>SELECT table_name, table_type, engine FROM information_schema.tables
WHERE table_schema'mysql' ORDER BY table_name DESC;
How to monitor performance of MySQL Server
Following are the command which we can use for session or several level performance
for MySQL Server.
SHOW GLOBAL STATUS ----------------- Show global server status.
SHOW LOCAL STATUS -------------------- This is used for session level server status.
Have to check following values know how server works.
Aborted_Clients: Usually no need to worry for this because many programs application
don’t close connection properly.
Aborted_Connects : This means authentication failure.Network timeout or any other
error.If the value is high than its possible that someone tries to break the password or
something.
Comm_XXX : This can be used to check server load that which statement are running
most on server .
Temprary Objects :
Created_tmp_tables : Temporary tables can often be avoided by query optimization.
16. Created_tmp_disk_tables : Not enough memory is allocated need to increase
tmp_table_size and max_heap _table_size
Handler XXX:
Handler_readkey ,Handler_read_next --------- Indexes are used or not by the queries.
Handler_read_rnd,Handler_read_rnd_next ---------------- Full table scans are done or
not.
Key Cache Info :
Key_blocks_used / Key_blocks_unused : This will show how much key_buffer is used
key_blocks_used should be key_blocks_ever_used or not .This will help us that how
much Key_buffer should be set.
Key_read_requests,Key_reads,Key_write _requests,Key_writes : THIs will show how
much good is key buffer usage.
Connections and Tables :
Max_used_connections : if this is > = max_connections than you need to increase
max_connections size.
Open_files : This should not be run of limit.
Open_tables : This will show how table cache is used.
Opened_tables : We can adjust-table-cache variable for this if its value is high or as per
our requirement.Before that we have to make sure that open-file-limit should be large
enough.
Query Cache Status :
Qcache_hits : This will show frequently query is used from query cache.
Qcache_inserts : How much queries are stored in query cache.
Qcache_free_memory : Free /Unused memory in query cache.Often query cache can
use limited memory because of invalidation.
17. Qcache_lowmem_prunes : Not enough memory or too fragmented.
Select :
Select_full_join : Joins without indexes.This very bad and dangerous for query
performance.
Select_range : This will show range queries.
Select_range_check : Usually queries worth looking to optimize because queries are
not using indexes.
Select_scan : Full Table Scan.Small or Large..This is also dangerous.
Sorting :
Sort_merge_passes : If this high than should increase sort_buffer_size.
Sort_range : THIs shows sorting of ranges.
Sort_scan :This shows sorting full table scans.
Table Locks :
Table_locks_immediate : This shows table locks granted without waiting.
Table_locks_waited : This shows table locks which had to be waited.Long wait on table
lock is bad for server performance.
Threads :
Threads_cached : This shows how many threads are cached.
Threads_connected : This shows how many thread are connected.
Threads-created : This shows how much threads are missed to cache.If this is high than
should increase thread_ cache
18. Threads_running : This shows how many threads are running currently.
--Find the Column using Information_schema for a table .
mysql>select table_schema "Data Base Name",table_name,column_name from
information_schema.columns where column_name LIKE 'dl;
--- Find the Table in which database it has.
mysql>select table_schema "Data Base Name",table_name from
information_schema.tables where table_name='plugin';
We can use a lot of things using information_schema.In the above I gave three
examples about information_schema.I think it will help you more.
-- Improve local and remote access security:
Disable the use of LOAD DATA LOCAL INFILE command, which will help to prevent
against unauthorized reading from local files. This matters especially when new SQL
Injection vulnerabilities in PHP applications are found. This can be set to 1 temporarily
for a local admin to import a csv file into the database and then turned off again as
well. The mysqld service will need to be restarted after each change.
For that purpose, the following parameter should be added in the [mysqld] section in
/etc/my.cnf:
Set-variable=local-infile=0
The first change applies to the 3306/tcp port, on which MySQL listens by default.
Because, according to the initial assumptions, the database will be used only by locally
installed PHP applications, we can freely disable listening on that port. This will limit
possibilities of attacking the MySQL database by direct TCP/IP connections from other
hosts. Local communication will be still possible throw the mysql.sock socket.
In order to disable listening on the mentioned port, the following parameter should be
added to the [mysqld] section of /etc/my.cnf:
skip-networking
19. SSH Tunneling can be used for remote backup scripts which require access to the
machine.