The document describes a toolkit for recovering data from MySQL databases using InnoDB tables. It provides an overview of the tools in the toolkit, including stream_parser to parse raw data files into InnoDB pages, sys_parser to recover table structures from those pages, and c_parser and recover_dictionary tools to extract and load table records and reconstruct the InnoDB data dictionary. It demonstrates dropping a sample table and then using the toolkit to recover the table structure and records from the raw data files.
6. Table in File System
# ls -la /var/lib/mysql/sakila/actor.*
-rw-rw---- 1 mysql mysql actor.frm
-rw-rw---- 1 mysql mysql actor.ibd
# ls -la /var/lib/mysql/
-rw-rw----. 1 mysql mysql ibdata1
-rw-rw----. 1 mysql mysql ib_logfile0
-rw-rw----. 1 mysql mysql ib_logfile1
7. PRIMARY index - B+ Tree Structure
1 100
page 2 page 3
1 50
page 4 page 5
100 150
page 6 page 7
actor_id 1 2 … 49
first_name A C E
last_name K D F
last_updat
e
X Y … Z
actor_id 50 51 … 99
first_name G I K
last_name H J L
last_updat
e
X Y … Z
Page 3Page 2
Page 1
Page 4 Page 5
8. Record format
A row: (10, ‘Kevin’, ‘Spacey’, ‘2015-04-08 21:47:36’)
8
Actually stored as: (10, TRX_ID, PTR_ID, ‘Kevin’, ‘Spacey’, ‘2015-04-08 21:47:36’)
…. next
Extra 5 bytes:
0x00 00 00 0A
Fields
... ... Kevin 0x5525A1FF
A bit per NULL-able field
7 Spacey
Lengths
9. InnoDB Dictionary
mysql> SELECT * FROM SYS_TABLES WHERE NAME = 'sakila/actor'G
*************************** 1. row ***************************
NAME: sakila/actor
ID: 2642
N_COLS: 4
TYPE: 41
MIX_ID: 0
MIX_LEN: 80
CLUSTER_NAME:
SPACE: 2337
1 row in set (0.00 sec)
SYS_TABLES
16. c_parser Result
# head -5 dumps/default/actor
-- Page id: 3, Format: COMPACT, Records list: Valid, Expected records: (200 200)
000000000740 C0000001630110 actor 1 "PENELOPE" "GUINESS" "2006-02-15
04:34:33"
000000000740 C000000163011A actor 2 "NICK" "WAHLBERG" "2006-02-15 04:34:33"
000000000740 C0000001630124 actor 3 "ED" "CHASE" "2006-02-15 04:34:33"
000000000740 C000000163012E actor 4 "JENNIFER" "DAVIS" "2006-02-15 04:34:33"
# cat dumps/default/actor.sql
SET FOREIGN_KEY_CHECKS=0;
LOAD DATA LOCAL INFILE '/root/undrop-for-innodb/dumps/default/actor'
REPLACE INTO TABLE `actor` FIELDS TERMINATED BY 't' OPTIONALLY
ENCLOSED BY '"' LINES STARTING BY 'actort' (`actor_id`, `first_name`,
`last_name`, `last_update`);
17. Introducing sys_parser
# yum install mysql-community-devel.x86_64
# make sys_parser
/usr/bin/mysql_config
cc `mysql_config --cflags` `mysql_config --libs` -o sys_parser sys_parser.c
# ./sys_parser sakila/actor
CREATE TABLE `actor`(
`actor_id` SMALLINT UNSIGNED NOT NULL,
`first_name` VARCHAR(45) CHARACTER SET 'utf8' COLLATE
'utf8_general_ci' NOT NULL,
`last_name` VARCHAR(45) CHARACTER SET 'utf8' COLLATE
'utf8_general_ci' NOT NULL,
`last_update` TIMESTAMP NOT NULL,
PRIMARY KEY (`actor_id`)
) ENGINE=InnoDB;
18. Recover InnoDB Dictionary
# ./stream_parser -f /var/lib/mysql/ibdata1
Opening file: /var/lib/mysql/ibdata1
Size to process: 12582912 (12.000 MiB)
All workers finished in 0 sec
# ./recover_dictionary.sh
Generating dictionary tables dumps... OK
Creating test database ... OK
Creating dictionary tables in database test:
SYS_TABLES ... OK
SYS_COLUMNS ... OK
SYS_INDEXES ... OK
SYS_FIELDS ... OK
All OK
Loading dictionary tables data:
SYS_TABLES ... 46 recs OK
SYS_COLUMNS ... 304 recs OK
SYS_INDEXES ... 96 recs OK
SYS_FIELDS ... 118 recs OK
All OK
19. Getting index_id from Dictionary
mysql> SELECT SYS_TABLES.NAME, SYS_INDEXES.NAME,
SYS_INDEXES.ID
FROM SYS_TABLES
LEFT JOIN SYS_INDEXES
ON SYS_TABLES.ID = SYS_INDEXES.TABLE_ID
WHERE SYS_TABLES.NAME = 'sakila/actor';
+--------------+---------------------+------+
| NAME | NAME | ID |
+--------------+---------------------+------+
| sakila/actor | PRIMARY | 22 |
| sakila/actor | idx_actor_last_name | 23 |
+--------------+---------------------+------+
2 rows in set (0.00 sec)