1berry/MariaDB-server
1
0
Fork 0
Code Issues Packages Projects Releases Wiki Activity

MDEV-12253: Buffer pool blocks are accessed after they have been freed

Browse Source 
Problem was that bpage was referenced after it was already freed
from LRU. Fixed by adding a new variable encrypted that is
passed down to buf_page_check_corrupt() and used in
buf_page_get_gen() to stop processing page read.

This patch should also address following test failures and
bugs:

MDEV-12419: IMPORT should not look up tablespace in
PageConverter::validate(). This is now removed.

MDEV-10099: encryption.innodb_onlinealter_encryption fails
sporadically in buildbot

MDEV-11420: encryption.innodb_encryption-page-compression
failed in buildbot

MDEV-11222: encryption.encrypt_and_grep failed in buildbot on P8

Removed dict_table_t::is_encrypted and dict_table_t::ibd_file_missing
and replaced these with dict_table_t::file_unreadable. Table
ibd file is missing if fil_get_space(space_id) returns NULL
and encrypted if not. Removed dict_table_t::is_corrupted field.

Ported FilSpace class from 10.2 and using that on buf_page_check_corrupt(),
buf_page_decrypt_after_read(), buf_page_encrypt_before_write(),
buf_dblwr_process(), buf_read_page(), dict_stats_save_defrag_stats().

Added test cases when enrypted page could be read while doing
redo log crash recovery. Also added test case for row compressed
blobs.

btr_cur_open_at_index_side_func(),
btr_cur_open_at_rnd_pos_func(): Avoid referencing block that is
NULL.

buf_page_get_zip(): Issue error if page read fails.

buf_page_get_gen(): Use dberr_t for error detection and
do not reference bpage after we hare freed it.

buf_mark_space_corrupt(): remove bpage from LRU also when
it is encrypted.

buf_page_check_corrupt(): @return DB_SUCCESS if page has
been read and is not corrupted,
DB_PAGE_CORRUPTED if page based on checksum check is corrupted,
DB_DECRYPTION_FAILED if page post encryption checksum matches but
after decryption normal page checksum does not match. In read
case only DB_SUCCESS is possible.

buf_page_io_complete(): use dberr_t for error handling.

buf_flush_write_block_low(),
buf_read_ahead_random(),
buf_read_page_async(),
buf_read_ahead_linear(),
buf_read_ibuf_merge_pages(),
buf_read_recv_pages(),
fil_aio_wait():
        Issue error if page read fails.

btr_pcur_move_to_next_page(): Do not reference page if it is
NULL.

Introduced dict_table_t::is_readable() and dict_index_t::is_readable()
that will return true if tablespace exists and pages read from
tablespace are not corrupted or page decryption failed.
Removed buf_page_t::key_version. After page decryption the
key version is not removed from page frame. For unencrypted
pages, old key_version is removed at buf_page_encrypt_before_write()

dict_stats_update_transient_for_index(),
dict_stats_update_transient()
        Do not continue if table decryption failed or table
        is corrupted.

dict0stats.cc: Introduced a dict_stats_report_error function
to avoid code duplication.

fil_parse_write_crypt_data():
        Check that key read from redo log entry is found from
        encryption plugin and if it is not, refuse to start.

PageConverter::validate(): Removed access to fil_space_t as
tablespace is not available during import.

Fixed error code on innodb.innodb test.

Merged test cased innodb-bad-key-change5 and innodb-bad-key-shutdown
to innodb-bad-key-change2.  Removed innodb-bad-key-change5 test.
Decreased unnecessary complexity on some long lasting tests.

Removed fil_inc_pending_ops(), fil_decr_pending_ops(),
fil_get_first_space(), fil_get_next_space(),
fil_get_first_space_safe(), fil_get_next_space_safe()
functions.

fil_space_verify_crypt_checksum(): Fixed bug found using ASAN
where FIL_PAGE_END_LSN_OLD_CHECKSUM field was incorrectly
accessed from row compressed tables. Fixed out of page frame
bug for row compressed tables in
fil_space_verify_crypt_checksum() found using ASAN. Incorrect
function was called for compressed table.

Added new tests for discard, rename table and drop (we should allow them
even when page decryption fails). Alter table rename is not allowed.
Added test for restart with innodb-force-recovery=1 when page read on
redo-recovery cant be decrypted. Added test for corrupted table where
both page data and FIL_PAGE_FILE_FLUSH_LSN_OR_KEY_VERSION is corrupted.

Adjusted the test case innodb_bug14147491 so that it does not anymore
expect crash. Instead table is just mostly not usable.

fil0fil.h: fil_space_acquire_low is not visible function
and fil_space_acquire and fil_space_acquire_silent are
inline functions. FilSpace class uses fil_space_acquire_low
directly.

recv_apply_hashed_log_recs() does not return anything.
This commit is contained in:
Jan Lindström 2017-04-26 15:19:16 +03:00
parent 97e0c260dc
commit 765a43605a
125 changed files with 3135 additions and 3229 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

8
extra/innochecksum.cc
View File

@ -583,13 +583,17 @@ void defrag_analysis(unsigned long long id, const per_index_stats& index)
}
}
}
if (!n_leaf_pages) {
n_leaf_pages=1;
}
printf("count = " ULINTPF " free = " ULINTPF "\n",
index.count, index.free_pages);
if (n_leaf_pages)
if (n_leaf_pages) {
printf("%llu\t\t%llu\t\t"
ULINTPF "\t\t%lu\t\t" ULINTPF "\t\t%.2f\t" ULINTPF "\n",
id, index.leaf_pages, n_leaf_pages, n_merge, n_pages,
1.0 - (double)n_pages / (double)n_leaf_pages, index.max_data_size);
1.0 - (double)n_pages / (double)n_leaf_pages, index.max_data_size);
}
}
void print_leaf_stats()

2
mysql-test/std_data/keys.txt
View File

@ -9,4 +9,4 @@
5;966050D7777350B6FD5CCB3E5F648DA45C63BEFB6DEDDFA13443F156B7D35C84
6;B5EA210C8C09EF20DB95EC584714A89F # and yet another
10;770A8A65DA156D24EE2A093277530143

3
mysql-test/std_data/keys2.txt
View File

@ -1,7 +1,8 @@
1;593E580927F47AB530D3B1237CDEF6D6
1;770A8A65DA156D24EE2A093277530142
2;352E42F1B9DB5CB915C3262FE745520A
3;CFE065600F5EB57481075C65180C3F8A
4;205379930183490D3BECA139BDF4DB5B
5;E2D944D5D837A1DCB22FF7FD397892EE
6;BAFE99B0BB87F2CD33A6AF26A11F6BD1
19;678D6B0063824BACCE33224B385104B35F30FF5749F0EBC030A0955DBC7FAC34
20;BAFE99B0BB87F2CD33A6AF26A11F6BD1

2
mysql-test/std_data/keys3.txt
View File

@ -1,4 +1,4 @@
1;593E580927F47AB530D3B1237CDEF6D6
1;770A8A65DA156D24EE2A093277530142
2;E4B00A45BF775B4E07D634EC5CA5912B
3;6E35ACB162B29D1FB9E178021DAF16ED
4;971A664A88EE0022D408E40BFAB17E79

39
mysql-test/suite/encryption/r/encrypt_and_grep.result
View File

@ -1,10 +1,15 @@
SET GLOBAL innodb_file_per_table = ON;
SET GLOBAL innodb_file_format = `Barracuda`;
create table t1 (a varchar(255)) engine=innodb encrypted=yes;
create table t2 (a varchar(255)) engine=innodb;
show warnings;
Level Code Message
create table t3 (a varchar(255)) engine=innodb encrypted=no;
insert t1 values (repeat('foobar', 42));
insert t2 values (repeat('temp', 42));
insert t3 values (repeat('dummy', 42));
insert t1 values (repeat('foobarsecret', 22));
Warnings:
Warning 1265 Data truncated for column 'a' at row 1
insert t2 values (repeat('tempsecret', 22));
insert t3 values (repeat('dummysecret', 22));
# Wait max 10 min for key encryption threads to encrypt all spaces
SELECT NAME FROM INFORMATION_SCHEMA.INNODB_TABLESPACES_ENCRYPTION WHERE MIN_KEY_VERSION = 0;
NAME
@ -17,15 +22,14 @@ test/t1
test/t2
./ibdata1
# t1 yes on expecting NOT FOUND
NOT FOUND /foobar/ in t1.ibd
NOT FOUND /foobarsecret/ in t1.ibd
# t2 ... on expecting NOT FOUND
NOT FOUND /temp/ in t2.ibd
NOT FOUND /tempsecret/ in t2.ibd
# t3 no on expecting FOUND
FOUND /dummy/ in t3.ibd
FOUND /dummysecret/ in t3.ibd
# ibdata1 expecting NOT FOUND
NOT FOUND /foobar/ in ibdata1
NOT FOUND /foobarsecret/ in ibdata1
# Now turn off encryption and wait for threads to decrypt everything
SET GLOBAL innodb_encryption_threads = 1;
SET GLOBAL innodb_encrypt_tables = off;
# Wait max 10 min for key encryption threads to decrypt all spaces
SELECT NAME FROM INFORMATION_SCHEMA.INNODB_TABLESPACES_ENCRYPTION WHERE MIN_KEY_VERSION = 0;
@ -39,15 +43,14 @@ SELECT NAME FROM INFORMATION_SCHEMA.INNODB_TABLESPACES_ENCRYPTION WHERE MIN_KEY_
NAME
test/t1
# t1 yes on expecting NOT FOUND
NOT FOUND /foobar/ in t1.ibd
# t2 ... on expecting FOUND
NOT FOUND /temp/ in t2.ibd
NOT FOUND /foobarsecret/ in t1.ibd
# t2 ... default expecting FOUND
FOUND /tempsecret/ in t2.ibd
# t3 no on expecting FOUND
FOUND /dummy/ in t3.ibd
FOUND /dummysecret/ in t3.ibd
# ibdata1 expecting NOT FOUND
NOT FOUND /foobar/ in ibdata1
NOT FOUND /foobarsecret/ in ibdata1
# Now turn on encryption and wait for threads to encrypt all spaces
SET GLOBAL innodb_encryption_threads = 4;
SET GLOBAL innodb_encrypt_tables = on;
# Wait max 10 min for key encryption threads to encrypt all spaces
SELECT NAME FROM INFORMATION_SCHEMA.INNODB_TABLESPACES_ENCRYPTION WHERE MIN_KEY_VERSION = 0;
@ -61,11 +64,11 @@ test/t1
test/t2
./ibdata1
# t1 yes on expecting NOT FOUND
NOT FOUND /foobar/ in t1.ibd
NOT FOUND /foobarsecret/ in t1.ibd
# t2 ... on expecting NOT FOUND
NOT FOUND /temp/ in t2.ibd
NOT FOUND /tempsecret/ in t2.ibd
# t3 no on expecting FOUND
FOUND /dummy/ in t3.ibd
FOUND /dummysecret/ in t3.ibd
# ibdata1 expecting NOT FOUND
NOT FOUND /foobar/ in ibdata1
NOT FOUND /foobarsecret/ in ibdata1
drop table t1, t2, t3;

65
mysql-test/suite/encryption/r/innodb-bad-key-change.result
View File

@ -1,13 +1,7 @@
call mtr.add_suppression("Plugin 'file_key_management' init function returned error");
call mtr.add_suppression("InnoDB: Database page corruption on disk or a failed.*");
call mtr.add_suppression("Plugin 'file_key_management' registration.*failed");
call mtr.add_suppression("InnoDB: Block in space_id .* in file test/.* encrypted");
call mtr.add_suppression("InnoDB: However key management plugin or used key_version .* is not found or used encryption algorithm or method does not match.");
call mtr.add_suppression("InnoDB: Marking tablespace as missing. You may drop this table or install correct key management plugin and key file.");
call mtr.add_suppression(".*InnoDB: Cannot open table test/.* from the internal data dictionary of InnoDB though the .frm file for the table exists. See .* for how you can resolve the problem.");
call mtr.add_suppression("InnoDB: .ibd file is missing for table test/.*");
call mtr.add_suppression("InnoDB: The page .*");
call mtr.add_suppression("InnoDB: Cannot calculate statistics for table .* ");
call mtr.add_suppression("Plugin 'file_key_management' .*");
call mtr.add_suppression("mysqld: File .*");
call mtr.add_suppression("InnoDB: Tablespace id .* is encrypted but encryption service or used key_id .* is not available. Can't continue opening tablespace.");
# Start server with keys2.txt
SET GLOBAL innodb_file_format = `Barracuda`;
@ -37,17 +31,13 @@ ERROR HY000: Got error 192 'Table encrypted but decryption failed. This could be
SHOW WARNINGS;
Level Code Message
Warning 192 Table test/t1 in tablespace is encrypted but encryption service or used key_id is not available. Can't continue reading table.
Warning 192 Table test/t1 is encrypted but encryption service or used key_id is not available. Can't continue reading table.
Warning 192 Table "test"."t1" in file ./test/t1.ibd is encrypted but encryption service or used key_id is not available. Can't continue reading table.
Error 1296 Got error 192 'Table encrypted but decryption failed. This could be because correct encryption management plugin is not loaded, used encryption key is not available or encryption method does not match.' from InnoDB
DROP TABLE t1;
Warnings:
Warning 192 Table in tablespace encrypted.However key management plugin or used key_id 1 is not found or used encryption algorithm or method does not match. Can't continue opening the table.
Warning 192 Table in tablespace encrypted.However key management plugin or used key_id 1 is not found or used encryption algorithm or method does not match. Can't continue opening the table.
SHOW WARNINGS;
Level Code Message
Warning 192 Table in tablespace encrypted.However key management plugin or used key_id 1 is not found or used encryption algorithm or method does not match. Can't continue opening the table.
Warning 192 Table in tablespace encrypted.However key management plugin or used key_id 1 is not found or used encryption algorithm or method does not match. Can't continue opening the table.
# Start server with keys.txt
# Start server with keys3.txt
SET GLOBAL innodb_default_encryption_key_id=5;
CREATE TABLE t2 (c VARCHAR(8), id int not null primary key, b int, key(b)) ENGINE=InnoDB ENCRYPTED=YES;
INSERT INTO t2 VALUES ('foobar',1,2);
@ -57,69 +47,60 @@ ERROR HY000: Got error 192 'Table encrypted but decryption failed. This could be
SHOW WARNINGS;
Level Code Message
Warning 192 Table test/t2 in tablespace is encrypted but encryption service or used key_id is not available. Can't continue reading table.
Warning 192 Table test/t2 is encrypted but encryption service or used key_id is not available. Can't continue reading table.
Warning 192 Table "test"."t2" in file ./test/t2.ibd is encrypted but encryption service or used key_id is not available. Can't continue reading table.
Error 1296 Got error 192 'Table encrypted but decryption failed. This could be because correct encryption management plugin is not loaded, used encryption key is not available or encryption method does not match.' from InnoDB
SELECT * FROM t2 where id = 1;
ERROR HY000: Got error 192 'Table encrypted but decryption failed. This could be because correct encryption management plugin is not loaded, used encryption key is not available or encryption method does not match.' from InnoDB
SHOW WARNINGS;
Level Code Message
Warning 1812 Tablespace is missing for table 'test/t2'
Warning 192 Table test/t2 is encrypted but encryption service or used key_id is not available. Can't continue reading table.
Warning 192 Table "test"."t2" in file ./test/t2.ibd is encrypted but encryption service or used key_id is not available. Can't continue reading table.
Error 1296 Got error 192 'Table encrypted but decryption failed. This could be because correct encryption management plugin is not loaded, used encryption key is not available or encryption method does not match.' from InnoDB
SELECT * FROM t2 where b = 1;
ERROR HY000: Got error 192 'Table encrypted but decryption failed. This could be because correct encryption management plugin is not loaded, used encryption key is not available or encryption method does not match.' from InnoDB
SHOW WARNINGS;
Level Code Message
Warning 1812 Tablespace is missing for table 'test/t2'
Warning 192 Table test/t2 is encrypted but encryption service or used key_id is not available. Can't continue reading table.
Warning 192 Table "test"."t2" in file ./test/t2.ibd is encrypted but encryption service or used key_id is not available. Can't continue reading table.
Error 1296 Got error 192 'Table encrypted but decryption failed. This could be because correct encryption management plugin is not loaded, used encryption key is not available or encryption method does not match.' from InnoDB
INSERT INTO t2 VALUES ('tmp',3,3);
ERROR HY000: Got error 192 'Table encrypted but decryption failed. This could be because correct encryption management plugin is not loaded, used encryption key is not available or encryption method does not match.' from InnoDB
SHOW WARNINGS;
Level Code Message
Warning 1812 Tablespace is missing for table 'test/t2'
Warning 192 Table test/t2 is encrypted but encryption service or used key_id is not available. Can't continue reading table.
Warning 192 Table "test"."t2" in file ./test/t2.ibd is encrypted but encryption service or used key_id is not available. Can't continue reading table.
Error 1296 Got error 192 'Table encrypted but decryption failed. This could be because correct encryption management plugin is not loaded, used encryption key is not available or encryption method does not match.' from InnoDB
DELETE FROM t2 where b = 3;
ERROR HY000: Got error 192 'Table encrypted but decryption failed. This could be because correct encryption management plugin is not loaded, used encryption key is not available or encryption method does not match.' from InnoDB
SHOW WARNINGS;
Level Code Message
Warning 1812 Tablespace is missing for table 'test/t2'
Warning 192 Table test/t2 is encrypted but encryption service or used key_id is not available. Can't continue reading table.
Warning 192 Table "test"."t2" in file ./test/t2.ibd is encrypted but encryption service or used key_id is not available. Can't continue reading table.
Error 1296 Got error 192 'Table encrypted but decryption failed. This could be because correct encryption management plugin is not loaded, used encryption key is not available or encryption method does not match.' from InnoDB
DELETE FROM t2 where id = 3;
ERROR HY000: Got error 192 'Table encrypted but decryption failed. This could be because correct encryption management plugin is not loaded, used encryption key is not available or encryption method does not match.' from InnoDB
SHOW WARNINGS;
Level Code Message
Warning 1812 Tablespace is missing for table 'test/t2'
Warning 192 Table test/t2 is encrypted but encryption service or used key_id is not available. Can't continue reading table.
Warning 192 Table "test"."t2" in file ./test/t2.ibd is encrypted but encryption service or used key_id is not available. Can't continue reading table.
Error 1296 Got error 192 'Table encrypted but decryption failed. This could be because correct encryption management plugin is not loaded, used encryption key is not available or encryption method does not match.' from InnoDB
UPDATE t2 set b = b +1;
ERROR HY000: Got error 192 'Table encrypted but decryption failed. This could be because correct encryption management plugin is not loaded, used encryption key is not available or encryption method does not match.' from InnoDB
SHOW WARNINGS;
Level Code Message
Warning 1812 Tablespace is missing for table 'test/t2'
Warning 192 Table test/t2 is encrypted but encryption service or used key_id is not available. Can't continue reading table.
Warning 192 Table "test"."t2" in file ./test/t2.ibd is encrypted but encryption service or used key_id is not available. Can't continue reading table.
Error 1296 Got error 192 'Table encrypted but decryption failed. This could be because correct encryption management plugin is not loaded, used encryption key is not available or encryption method does not match.' from InnoDB
OPTIMIZE TABLE t2;
Table Op Msg_type Msg_text
test.t2 optimize Warning Tablespace is missing for table 'test/t2'
test.t2 optimize Warning Table test/t2 is encrypted but encryption service or used key_id is not available. Can't continue reading table.
test.t2 optimize Warning Table "test"."t2" in file ./test/t2.ibd is encrypted but encryption service or used key_id is not available. Can't continue reading table.
test.t2 optimize Error Got error 192 'Table encrypted but decryption failed. This could be because correct encryption management plugin is not loaded, used encryption key is not available or encryption method does not match.' from InnoDB
test.t2 optimize error Corrupt
SHOW WARNINGS;
Level Code Message
ALTER TABLE t2 ADD COLUMN c INT;
ALTER TABLE t2 ADD COLUMN d INT;
ERROR HY000: Got error 192 'Table encrypted but decryption failed. This could be because correct encryption management plugin is not loaded, used encryption key is not available or encryption method does not match.' from InnoDB
SHOW WARNINGS;
Level Code Message
Warning 1812 Tablespace is missing for table 'test/t2'
Warning 192 Table test/t2 is encrypted but encryption service or used key_id is not available. Can't continue reading table.
Warning 192 Table "test"."t2" in file ./test/t2.ibd is encrypted but encryption service or used key_id is not available. Can't continue reading table.
Error 1296 Got error 192 'Table encrypted but decryption failed. This could be because correct encryption management plugin is not loaded, used encryption key is not available or encryption method does not match.' from InnoDB
ANALYZE TABLE t2;
Table Op Msg_type Msg_text
test.t2 analyze Warning Tablespace is missing for table 'test/t2'
test.t2 analyze Warning Table test/t2 is encrypted but encryption service or used key_id is not available. Can't continue reading table.
test.t2 analyze Warning Table "test"."t2" in file ./test/t2.ibd is encrypted but encryption service or used key_id is not available. Can't continue reading table.
test.t2 analyze Error Got error 192 'Table encrypted but decryption failed. This could be because correct encryption management plugin is not loaded, used encryption key is not available or encryption method does not match.' from InnoDB
test.t2 analyze error Corrupt
SHOW WARNINGS;
@ -128,16 +109,8 @@ TRUNCATE TABLE t2;
ERROR HY000: Got error 192 'Table encrypted but decryption failed. This could be because correct encryption management plugin is not loaded, used encryption key is not available or encryption method does not match.' from InnoDB
SHOW WARNINGS;
Level Code Message
Warning 1812 Tablespace is missing for table 'test/t2'
Warning 192 Table test/t2 is encrypted but encryption service or used key_id is not available. Can't continue reading table.
Warning 192 Table "test"."t2" in file ./test/t2.ibd is encrypted but encryption service or used key_id is not available. Can't continue reading table.
Error 1296 Got error 192 'Table encrypted but decryption failed. This could be because correct encryption management plugin is not loaded, used encryption key is not available or encryption method does not match.' from InnoDB
DROP TABLE t2;
ERROR HY000: Got error 192 'Table encrypted but decryption failed. This could be because correct encryption management plugin is not loaded, used encryption key is not available or encryption method does not match.' from InnoDB
SHOW WARNINGS;
Level Code Message
Error 1296 Got error 192 'Table encrypted but decryption failed. This could be because correct encryption management plugin is not loaded, used encryption key is not available or encryption method does not match.' from InnoDB
# Restart server with keys.txt
DROP TABLE t2;
SHOW WARNINGS;
Level Code Message
# Start server with keys2.txt

69
mysql-test/suite/encryption/r/innodb-bad-key-change2.result
View File

@ -1,26 +1,59 @@
call mtr.add_suppression("InnoDB: Block in space_id .* in file test/.* encrypted");
call mtr.add_suppression("InnoDB: However key management plugin or used key_version .* is not found or used encryption algorithm or method does not match.");
call mtr.add_suppression("InnoDB: Marking tablespace as missing. You may drop this table or install correct key management plugin and key file.");
call mtr.add_suppression(".*InnoDB: Cannot open table test/.* from the internal data dictionary of InnoDB though the .frm file for the table exists. See .* for how you can resolve the problem.");
call mtr.add_suppression("InnoDB: .ibd file is missing for table test/.*");
call mtr.add_suppression("Couldn't load plugins from 'file_key_management*");
CREATE TABLE t1 (pk INT PRIMARY KEY, f VARCHAR(8)) ENGINE=InnoDB ENCRYPTION_KEY_ID=4;
call mtr.add_suppression("InnoDB: The page \[page id: space=[0-9]+, page number=[0-9]+\] in file test/t1new cannot be decrypted.");
call mtr.add_suppression("InnoDB: The page \[page id: space=[0-9]+, page number=[0-9]+\] in file test/t1 cannot be decrypted.");
call mtr.add_suppression("Couldn't load plugins from 'file_key_management.*");
call mtr.add_suppression("InnoDB: Table \'\"test\".\"t1\"\' tablespace is set as discarded.");
SET GLOBAL innodb_file_format = `Barracuda`;
SET GLOBAL innodb_file_per_table = ON;
CREATE TABLE t1 (pk INT PRIMARY KEY, f VARCHAR(8)) ENGINE=InnoDB ENCRYPTED=YES ENCRYPTION_KEY_ID=4;
INSERT INTO t1 VALUES (1,'foo'),(2,'bar');
select * from t1;
SELECT * FROM t1;
ERROR HY000: Got error 192 'Table encrypted but decryption failed. This could be because correct encryption management plugin is not loaded, used encryption key is not available or encryption method does not match.' from InnoDB
show warnings;
SHOW WARNINGS;
Level Code Message
Warning 192 Table test/t1 is encrypted but encryption service or used key_id is not available. Can't continue reading table.
Warning 192 Table test/t1 is encrypted but encryption service or used key_id is not available. Can't continue reading table.
Warning 192 Table test/t1 in tablespace is encrypted but encryption service or used key_id is not available. Can't continue reading table.
Warning 192 Table "test"."t1" in file ./test/t1.ibd is encrypted but encryption service or used key_id is not available. Can't continue reading table.
Error 1296 Got error 192 'Table encrypted but decryption failed. This could be because correct encryption management plugin is not loaded, used encryption key is not available or encryption method does not match.' from InnoDB
alter table t1 discard tablespace;
ALTER TABLE t1 engine=InnoDB;
ERROR HY000: Got error 192 'Table encrypted but decryption failed. This could be because correct encryption management plugin is not loaded, used encryption key is not available or encryption method does not match.' from InnoDB
show warnings;
SHOW WARNINGS;
Level Code Message
Warning 192 Table "test"."t1" in file ./test/t1.ibd is encrypted but encryption service or used key_id is not available. Can't continue reading table.
Error 1296 Got error 192 'Table encrypted but decryption failed. This could be because correct encryption management plugin is not loaded, used encryption key is not available or encryption method does not match.' from InnoDB
alter table t1 engine=InnoDB;
OPTIMIZE TABLE t1;
Table Op Msg_type Msg_text
test.t1 optimize Warning Table "test"."t1" in file ./test/t1.ibd is encrypted but encryption service or used key_id is not available. Can't continue reading table.
test.t1 optimize Error Got error 192 'Table encrypted but decryption failed. This could be because correct encryption management plugin is not loaded, used encryption key is not available or encryption method does not match.' from InnoDB
test.t1 optimize error Corrupt
SHOW WARNINGS;
Level Code Message
CHECK TABLE t1;
Table Op Msg_type Msg_text
test.t1 check Warning Table "test"."t1" in file ./test/t1.ibd is encrypted but encryption service or used key_id is not available. Can't continue reading table.
test.t1 check Error Got error 192 'Table encrypted but decryption failed. This could be because correct encryption management plugin is not loaded, used encryption key is not available or encryption method does not match.' from InnoDB
test.t1 check error Corrupt
SHOW WARNINGS;
Level Code Message
FLUSH TABLES t1 FOR EXPORT;
backup: t1
UNLOCK TABLES;
ALTER TABLE t1 DISCARD TABLESPACE;
Warnings:
Warning 192 Table test/t1 in tablespace is encrypted but encryption service or used key_id is not available. Can't continue reading table.
Warning 1812 Tablespace is missing for table 't1'
restore: t1 .ibd and .cfg files
ALTER TABLE t1 IMPORT TABLESPACE;
Warnings:
Warning 1814 Tablespace has been discarded for table 't1'
SHOW CREATE TABLE t1;
Table Create Table
t1 CREATE TABLE `t1` (
`pk` int(11) NOT NULL,
`f` varchar(8) DEFAULT NULL,
PRIMARY KEY (`pk`)
) ENGINE=InnoDB DEFAULT CHARSET=latin1 `ENCRYPTED`=YES `ENCRYPTION_KEY_ID`=4
RENAME TABLE t1 TO t1new;
ALTER TABLE t1new RENAME TO t2new;
ERROR HY000: Got error 192 'Table encrypted but decryption failed. This could be because correct encryption management plugin is not loaded, used encryption key is not available or encryption method does not match.' from InnoDB
show warnings;
Level Code Message
Warning 192 Table test/t1 is encrypted but encryption service or used key_id is not available. Can't continue reading table.
Error 1296 Got error 192 'Table encrypted but decryption failed. This could be because correct encryption management plugin is not loaded, used encryption key is not available or encryption method does not match.' from InnoDB
DROP TABLE t1new;
DROP TABLE t1, t1new;
ERROR 42S02: Unknown table 'test.t1,test.t1new'

24
mysql-test/suite/encryption/r/innodb-bad-key-change4.result
View File

@ -1,19 +1,23 @@
call mtr.add_suppression("InnoDB: Block in space_id .* in file test/.* encrypted");
call mtr.add_suppression("InnoDB: However key management plugin or used key_version .* is not found or used encryption algorithm or method does not match.");
call mtr.add_suppression("InnoDB: Marking tablespace as missing. You may drop this table or install correct key management plugin and key file.");
call mtr.add_suppression(".*InnoDB: Cannot open table test/.* from the internal data dictionary of InnoDB though the .frm file for the table exists. See .* for how you can resolve the problem.");
call mtr.add_suppression("InnoDB: The page \[page id: space=[0-9]+, page number=[0-9]+\] in file test/t1 cannot be decrypted.");
call mtr.add_suppression("InnoDB: Cannot open table .*");
call mtr.add_suppression("InnoDB: .ibd file is missing for table test/.*");
call mtr.add_suppression("Couldn't load plugins from 'file_key_management*");
call mtr.add_suppression("Couldn't load plugins from 'file_key_management.*");
SET GLOBAL innodb_file_format = `Barracuda`;
SET GLOBAL innodb_file_per_table = ON;
CREATE TABLE t1 (pk INT PRIMARY KEY, f VARCHAR(8)) ENGINE=InnoDB ENCRYPTION_KEY_ID=4;
CREATE TABLE t1 (pk INT PRIMARY KEY, f VARCHAR(8)) ENGINE=InnoDB ENCRYPTED=YES ENCRYPTION_KEY_ID=4;
INSERT INTO t1 VALUES (1,'foo'),(2,'bar');
SET GLOBAL innodb_file_format = `Barracuda`;
SET GLOBAL innodb_file_per_table = ON;
OPTIMIZE TABLE t1;
Table Op Msg_type Msg_text
test.t1 optimize Warning Table test/t1 in tablespace is encrypted but encryption service or used key_id is not available. Can't continue reading table.
test.t1 optimize Warning Table "test"."t1" in file ./test/t1.ibd is encrypted but encryption service or used key_id is not available. Can't continue reading table.
test.t1 optimize Error Got error 192 'Table encrypted but decryption failed. This could be because correct encryption management plugin is not loaded, used encryption key is not available or encryption method does not match.' from InnoDB
test.t1 optimize error Corrupt
SHOW WARNINGS;
Level Code Message
CHECK TABLE t1;
Table Op Msg_type Msg_text
test.t1 check Warning Table test/t1 in tablespace 4 is encrypted but encryption service or used key_id is not available. Can't continue reading table.
test.t1 check Warning Table test/t1 is encrypted but encryption service or used key_id is not available. Can't continue checking table.
test.t1 check Warning Table "test"."t1" in file ./test/t1.ibd is encrypted but encryption service or used key_id is not available. Can't continue reading table.
test.t1 check Error Got error 192 'Table encrypted but decryption failed. This could be because correct encryption management plugin is not loaded, used encryption key is not available or encryption method does not match.' from InnoDB
test.t1 check error Corrupt
SHOW WARNINGS;
Level Code Message

28
mysql-test/suite/encryption/r/innodb-bad-key-change5.result
View File

@ -1,28 +0,0 @@
call mtr.add_suppression("InnoDB: Block in space_id .* in file test/.* encrypted");
call mtr.add_suppression("InnoDB: However key management plugin or used key_version .* is not found or used encryption algorithm or method does not match.");
call mtr.add_suppression("InnoDB: Marking tablespace as missing. You may drop this table or install correct key management plugin and key file.");
call mtr.add_suppression(".*InnoDB: Cannot open table test/.* from the internal data dictionary of InnoDB though the .frm file for the table exists. See .* for how you can resolve the problem.");
call mtr.add_suppression("InnoDB: .ibd file is missing for table test/.*");
call mtr.add_suppression("Couldn't load plugins from 'file_key_management*");
SET GLOBAL innodb_file_format = `Barracuda`;
SET GLOBAL innodb_file_per_table = ON;
CREATE TABLE t1 (pk INT PRIMARY KEY, f VARCHAR(8)) ENGINE=InnoDB ENCRYPTION_KEY_ID=4;
INSERT INTO t1 VALUES (1,'foo'),(2,'bar');
SET GLOBAL innodb_file_format = `Barracuda`;
SET GLOBAL innodb_file_per_table = ON;
OPTIMIZE TABLE t1;
Table Op Msg_type Msg_text
test.t1 optimize Warning Table test/t1 is encrypted but encryption service or used key_id is not available. Can't continue checking table.
test.t1 optimize Warning InnoDB: Cannot defragment table test/t1: returned error code 192
test.t1 optimize note Table does not support optimize, doing recreate + analyze instead
test.t1 optimize error Got error 192 'Table encrypted but decryption failed. This could be because correct encryption management plugin is not loaded, used encryption key is not available or encryption method does not match.' from InnoDB
test.t1 optimize status Operation failed
Warnings:
Warning 192 Table test/t1 is encrypted but encryption service or used key_id is not available. Can't continue reading table.
Error 1296 Got error 192 'Table encrypted but decryption failed. This could be because correct encryption management plugin is not loaded, used encryption key is not available or encryption method does not match.' from InnoDB
SHOW WARNINGS;
Level Code Message
Warning 192 Table test/t1 is encrypted but encryption service or used key_id is not available. Can't continue reading table.
Error 1296 Got error 192 'Table encrypted but decryption failed. This could be because correct encryption management plugin is not loaded, used encryption key is not available or encryption method does not match.' from InnoDB
DROP TABLE t1;

20
mysql-test/suite/encryption/r/innodb-bad-key-shutdown.result
View File

@ -1,20 +0,0 @@
call mtr.add_suppression("InnoDB: Block in space_id .* in file test/.* encrypted");
call mtr.add_suppression("InnoDB: However key management plugin or used key_version .* is not found or used encryption algorithm or method does not match.");
call mtr.add_suppression("InnoDB: Marking tablespace as missing. You may drop this table or install correct key management plugin and key file.");
call mtr.add_suppression(".*InnoDB: Cannot open table test/.* from the internal data dictionary of InnoDB though the .frm file for the table exists. See .* for how you can resolve the problem.");
call mtr.add_suppression("InnoDB: .ibd file is missing for table test/.*");
call mtr.add_suppression("Couldn't load plugins from 'file_key_management*");
#
# Restart the server with key 4 in the key file
#
CREATE TABLE t1 (i INT, KEY(i)) ENGINE=InnoDB ENCRYPTED=YES ENCRYPTION_KEY_ID=4;
INSERT INTO t1 VALUES (1);
#
# Restart the server with a different value for key 4 in the key file
#
SELECT * FROM t1;
ERROR HY000: Got error 192 'Table encrypted but decryption failed. This could be because correct encryption management plugin is not loaded, used encryption key is not available or encryption method does not match.' from InnoDB
SELECT * FROM t1;
i
1
DROP TABLE t1;

24
mysql-test/suite/encryption/r/innodb-compressed-blob.result Normal file
View File

@ -0,0 +1,24 @@
call mtr.add_suppression("InnoDB: However key management plugin or used key_version .*");
call mtr.add_suppression("InnoDB: The page \[page id: space=[0-9]+, page number=[0-9]+\] in file test/t1 cannot be decrypted.");
call mtr.add_suppression("InnoDB: The page \[page id: space=[0-9]+, page number=[0-9]+\] in file test/t2 cannot be decrypted.");
call mtr.add_suppression("InnoDB: The page \[page id: space=[0-9]+, page number=[0-9]+\] in file test/t3 cannot be decrypted.");
# Restart mysqld --file-key-management-filename=keys2.txt
SET GLOBAL innodb_file_format = `Barracuda`;
SET GLOBAL innodb_file_per_table = ON;
set GLOBAL innodb_default_encryption_key_id=4;
create table t1(a int not null primary key, b blob, index(b(10))) engine=innodb row_format=compressed;
create table t2(a int not null primary key, b blob, index(b(10))) engine=innodb row_format=compressed encrypted=yes;
create table t3(a int not null primary key, b blob, index(b(10))) engine=innodb row_format=compressed encrypted=no;
insert into t1 values (1, repeat('secret',6000));
insert into t2 values (1, repeat('secret',6000));
insert into t3 values (1, repeat('secret',6000));
# Restart mysqld --file-key-management-filename=keys3.txt
select count(*) from t1 FORCE INDEX (b) where b like 'secret%';
ERROR HY000: Got error 192 'Table encrypted but decryption failed. This could be because correct encryption management plugin is not loaded, used encryption key is not available or encryption method does not match.' from InnoDB
select count(*) from t2 FORCE INDEX (b) where b like 'secret%';
ERROR HY000: Got error 192 'Table encrypted but decryption failed. This could be because correct encryption management plugin is not loaded, used encryption key is not available or encryption method does not match.' from InnoDB
select count(*) from t3 FORCE INDEX (b) where b like 'secret%';
count(*)
1
# Restart mysqld --file-key-management-filename=keys2.txt
drop table t1,t2,t3;

10
mysql-test/suite/encryption/r/innodb-encryption-disable.result
View File

@ -1,12 +1,6 @@
SET GLOBAL innodb_file_format = `Barracuda`;
SET GLOBAL innodb_file_per_table = ON;
call mtr.add_suppression("InnoDB: Block in space_id .* in file test/.* encrypted");
call mtr.add_suppression("InnoDB: However key management plugin or used key_version .* is not found or used encryption algorithm or method does not match.");
call mtr.add_suppression("InnoDB: Marking tablespace as missing. You may drop this table or install correct key management plugin and key file.");
call mtr.add_suppression(".*InnoDB: Cannot open table test/.* from the internal data dictionary of InnoDB though the .frm file for the table exists. See .* for how you can resolve the problem.");
call mtr.add_suppression("InnoDB: .ibd file is missing for table test/.*");
call mtr.add_suppression("InnoDB: The page \[page id: space=[0-9]+, page number=[0-9]+\] in file test/t1 cannot be decrypted.");
call mtr.add_suppression("InnoDB: The page \[page id: space=[0-9]+, page number=[0-9]+\] in file test/t5 cannot be decrypted.");
call mtr.add_suppression("Couldn't load plugins from 'file_key_management*");
call mtr.add_suppression("InnoDB: Tablespace id.* is encrypted but encryption service or used key_id .* is not available. Can't continue opening tablespace.");
create table t5 (
`intcol1` int(32) DEFAULT NULL,
`intcol2` int(32) DEFAULT NULL,

29
mysql-test/suite/encryption/r/innodb-force-corrupt.result Normal file
View File

@ -0,0 +1,29 @@
CALL mtr.add_suppression("InnoDB: Database page corruption on disk or a failed .*");
CALL mtr.add_suppression("InnoDB: Corruption: Block in space_id .*");
CALL mtr.add_suppression("InnoDB: However key management plugin or used key_version .*");
call mtr.add_suppression("InnoDB: The page \[page id: space=[0-9]+, page number=[0-9]+\] in file test/t1 cannot be decrypted.");
call mtr.add_suppression("InnoDB: The page \[page id: space=[0-9]+, page number=[0-9]+\] in file test/t2 cannot be decrypted.");
call mtr.add_suppression("InnoDB: The page \[page id: space=[0-9]+, page number=[0-9]+\] in file test/t3 cannot be decrypted.");
SET GLOBAL innodb_file_format = `Barracuda`;
SET GLOBAL innodb_file_per_table = ON;
set global innodb_compression_algorithm = 1;
# Create and populate tables to be corrupted
CREATE TABLE t1 (a INT AUTO_INCREMENT PRIMARY KEY, b TEXT,c char(200)) ENGINE=InnoDB encrypted=yes;
CREATE TABLE t2 (a INT AUTO_INCREMENT PRIMARY KEY, b TEXT,c char(200)) ENGINE=InnoDB row_format=compressed encrypted=yes;
CREATE TABLE t3 (a INT AUTO_INCREMENT PRIMARY KEY, b TEXT, c char(200)) ENGINE=InnoDB page_compressed=yes encrypted=yes;
BEGIN;
INSERT INTO t1 (b,c) VALUES ('corrupt me','secret');
INSERT INTO t1 (b,c) VALUES ('corrupt me','moresecretmoresecret');
INSERT INTO t2 select * from t1;
INSERT INTO t3 select * from t1;
COMMIT;
# Backup tables before corrupting
# Corrupt tables
SELECT * FROM t1;
ERROR HY000: Got error 192 'Table encrypted but decryption failed. This could be because correct encryption management plugin is not loaded, used encryption key is not available or encryption method does not match.' from InnoDB
SELECT * FROM t2;
ERROR HY000: Got error 192 'Table encrypted but decryption failed. This could be because correct encryption management plugin is not loaded, used encryption key is not available or encryption method does not match.' from InnoDB
SELECT * FROM t3;
ERROR HY000: Got error 192 'Table encrypted but decryption failed. This could be because correct encryption management plugin is not loaded, used encryption key is not available or encryption method does not match.' from InnoDB
# Restore the original tables
DROP TABLE t1,t2,t3;

6
mysql-test/suite/encryption/r/innodb-missing-key.result
View File

@ -1,6 +1,6 @@
call mtr.add_suppression("InnoDB: Block in space_id .* in file test/.* encrypted");
call mtr.add_suppression("InnoDB: However key management plugin or used key_version .* is not found or used encryption algorithm or method does not match.");
call mtr.add_suppression("InnoDB: Marking tablespace as missing. You may drop this table or install correct key management plugin and key file.");
call mtr.add_suppression("InnoDB: The page \[page id: space=[0-9]+, page number=[0-9]+\] in file test/t1 cannot be decrypted.");
call mtr.add_suppression("InnoDB: The page \[page id: space=[0-9]+, page number=[0-9]+\] in file test/t2 cannot be decrypted.");
call mtr.add_suppression("InnoDB: The page \[page id: space=[0-9]+, page number=[0-9]+\] in file test/t3 cannot be decrypted.");
# Start server with keys2.txt
CREATE TABLE t1(a int not null primary key auto_increment, b varchar(128)) engine=innodb ENCRYPTED=YES ENCRYPTION_KEY_ID=19;

117
mysql-test/suite/encryption/r/innodb-page_encryption_compression.result
View File

@ -38,75 +38,14 @@ set current_num = current_num + 1;
end while;
end//
commit;
set autocommit=0;
call innodb_insert_proc(5000);
commit;
set autocommit=1;
begin;
call innodb_insert_proc(2000);
insert into innodb_compact select * from innodb_normal;
insert into innodb_dynamic select * from innodb_normal;
update innodb_normal set c1 = c1 + 1;
update innodb_compact set c1 = c1 + 1;
update innodb_dynamic set c1 = c1 + 1;
select count(*) from innodb_normal;
count(*)
5000
select count(*) from innodb_compact where c1 < 1500000;
count(*)
5000
select count(*) from innodb_dynamic where c1 < 1500000;
count(*)
5000
select count(*) from innodb_compact t1, innodb_normal t2 where
t1.c1 = t2.c1 and t1.b = t2.b;
count(*)
5000
select count(*) from innodb_dynamic t1, innodb_normal t2 where
t1.c1 = t2.c1 and t1.b = t2.b;
count(*)
5000
SELECT variable_value >= 0 FROM information_schema.global_status WHERE variable_name = 'innodb_num_pages_encrypted';
variable_value >= 0
1
SELECT variable_value >= 0 FROM information_schema.global_status WHERE variable_name = 'innodb_num_pages_decrypted';
variable_value >= 0
1
SELECT variable_value >= 0 FROM information_schema.global_status WHERE variable_name = 'innodb_num_pages_compressed';
variable_value >= 0
SELECT variable_value >= 0 FROM information_schema.global_status WHERE variable_name = 'innodb_num_pages_decompressed';
variable_value >= 0
commit;
SET GLOBAL innodb_file_format = `Barracuda`;
SET GLOBAL innodb_file_per_table = ON;
set global innodb_compression_algorithm = 1;
update innodb_normal set c1 = c1 + 1;
update innodb_compact set c1 = c1 + 1;
update innodb_dynamic set c1 = c1 + 1;
select count(*) from innodb_normal;
count(*)
5000
select count(*) from innodb_compact where c1 < 1500000;
count(*)
5000
select count(*) from innodb_dynamic where c1 < 1500000;
count(*)
5000
select count(*) from innodb_compact t1, innodb_normal t2 where
t1.c1 = t2.c1 and t1.b = t2.b;
count(*)
5000
select count(*) from innodb_dynamic t1, innodb_normal t2 where
t1.c1 = t2.c1 and t1.b = t2.b;
count(*)
5000
SELECT variable_value >= 0 FROM information_schema.global_status WHERE variable_name = 'innodb_num_pages_encrypted';
variable_value >= 0
1
SELECT variable_value >= 0 FROM information_schema.global_status WHERE variable_name = 'innodb_num_pages_decrypted';
variable_value >= 0
1
SELECT variable_value >= 0 FROM information_schema.global_status WHERE variable_name = 'innodb_num_pages_compressed';
variable_value >= 0
SELECT variable_value >= 0 FROM information_schema.global_status WHERE variable_name = 'innodb_num_pages_decompressed';
variable_value >= 0
alter table innodb_normal engine=innodb page_compressed=DEFAULT;
show create table innodb_normal;
Table Create Table
@ -128,56 +67,6 @@ innodb_dynamic CREATE TABLE `innodb_dynamic` (
`c1` bigint(20) NOT NULL,
`b` char(200) DEFAULT NULL
) ENGINE=InnoDB DEFAULT CHARSET=latin1 ROW_FORMAT=DYNAMIC
SET GLOBAL innodb_file_format = `Barracuda`;
SET GLOBAL innodb_file_per_table = ON;
show create table innodb_normal;
Table Create Table
innodb_normal CREATE TABLE `innodb_normal` (
`c1` bigint(20) NOT NULL,
`b` char(200) DEFAULT NULL
) ENGINE=InnoDB DEFAULT CHARSET=latin1
show create table innodb_compact;
Table Create Table
innodb_compact CREATE TABLE `innodb_compact` (
`c1` bigint(20) NOT NULL,
`b` char(200) DEFAULT NULL
) ENGINE=InnoDB DEFAULT CHARSET=latin1 ROW_FORMAT=COMPACT
show create table innodb_dynamic;
Table Create Table
innodb_dynamic CREATE TABLE `innodb_dynamic` (
`c1` bigint(20) NOT NULL,
`b` char(200) DEFAULT NULL
) ENGINE=InnoDB DEFAULT CHARSET=latin1 ROW_FORMAT=DYNAMIC
update innodb_normal set c1 = c1 + 1;
update innodb_compact set c1 = c1 + 1;
update innodb_dynamic set c1 = c1 + 1;
select count(*) from innodb_normal;
count(*)
5000
select count(*) from innodb_compact where c1 < 1500000;
count(*)
5000
select count(*) from innodb_dynamic where c1 < 1500000;
count(*)
5000
select count(*) from innodb_compact t1, innodb_normal t2 where
t1.c1 = t2.c1 and t1.b = t2.b;
count(*)
5000
select count(*) from innodb_dynamic t1, innodb_normal t2 where
t1.c1 = t2.c1 and t1.b = t2.b;
count(*)
5000
SELECT variable_value = 0 FROM information_schema.global_status WHERE variable_name = 'innodb_num_pages_encrypted';
variable_value = 0
1
SELECT variable_value = 0 FROM information_schema.global_status WHERE variable_name = 'innodb_num_pages_decrypted';
variable_value = 0
1
SELECT variable_value = 0 FROM information_schema.global_status WHERE variable_name = 'innodb_num_pages_compressed';
variable_value = 0
SELECT variable_value = 0 FROM information_schema.global_status WHERE variable_name = 'innodb_num_pages_decompressed';
variable_value = 0
drop procedure innodb_insert_proc;
drop table innodb_normal;
drop table innodb_compact;

143
mysql-test/suite/encryption/r/innodb-page_encryption_log_encryption.result
View File

@ -1,6 +1,3 @@
call mtr.add_suppression("KeyID 0 not found or with error. Check the key and the log*");
call mtr.add_suppression("Disabling redo log encryp*");
call mtr.add_suppression("InnoDB: Redo log crypto: Can't initialize to key version*");
SET GLOBAL innodb_file_format = `Barracuda`;
SET GLOBAL innodb_file_per_table = ON;
create table innodb_normal(c1 bigint not null, b char(200)) engine=innodb;
@ -52,49 +49,15 @@ set current_num = current_num + 1;
end while;
end//
commit;
set autocommit=0;
begin;
call innodb_insert_proc(2000);
commit;
set autocommit=1;
insert into innodb_compact select * from innodb_normal;
insert into innodb_compressed select * from innodb_normal;
insert into innodb_dynamic select * from innodb_normal;
insert into innodb_redundant select * from innodb_normal;
update innodb_normal set c1 = c1 +1;
update innodb_compact set c1 = c1 + 1;
update innodb_compressed set c1 = c1 + 1;
update innodb_dynamic set c1 = c1 + 1;
update innodb_redundant set c1 = c1 + 1;
select count(*) from innodb_compact where c1 < 1500000;
count(*)
2000
select count(*) from innodb_compressed where c1 < 1500000;
count(*)
2000
select count(*) from innodb_dynamic where c1 < 1500000;
count(*)
2000
select count(*) from innodb_redundant where c1 < 1500000;
count(*)
2000
select count(*) from innodb_compact t1, innodb_normal t2 where
t1.c1 = t2.c1 and t1.b = t2.b;
count(*)
2000
select count(*) from innodb_dynamic t1, innodb_normal t2 where
t1.c1 = t2.c1 and t1.b = t2.b;
count(*)
2000
select count(*) from innodb_compressed t1, innodb_normal t2 where
t1.c1 = t2.c1 and t1.b = t2.b;
count(*)
2000
select count(*) from innodb_redundant t1, innodb_normal t2 where
t1.c1 = t2.c1 and t1.b = t2.b;
count(*)
2000
SELECT variable_value >= 0 FROM information_schema.global_status WHERE variable_name = 'innodb_num_pages_encrypted';
variable_value >= 0
commit;
SELECT variable_value > 0 FROM information_schema.global_status WHERE variable_name = 'innodb_num_pages_encrypted';
variable_value > 0
1
SELECT variable_value >= 0 FROM information_schema.global_status WHERE variable_name = 'innodb_num_pages_decrypted';
variable_value >= 0
@ -106,40 +69,13 @@ update innodb_compact set c1 = c1 + 1;
update innodb_compressed set c1 = c1 + 1;
update innodb_dynamic set c1 = c1 + 1;
update innodb_redundant set c1 = c1 + 1;
select count(*) from innodb_compact where c1 < 1500000;
count(*)
2000
select count(*) from innodb_compressed where c1 < 1500000;
count(*)
2000
select count(*) from innodb_dynamic where c1 < 1500000;
count(*)
2000
select count(*) from innodb_redundant where c1 < 1500000;
count(*)
2000
select count(*) from innodb_compact t1, innodb_normal t2 where
t1.c1 = t2.c1 and t1.b = t2.b;
count(*)
2000
select count(*) from innodb_dynamic t1, innodb_normal t2 where
t1.c1 = t2.c1 and t1.b = t2.b;
count(*)
2000
select count(*) from innodb_compressed t1, innodb_normal t2 where
t1.c1 = t2.c1 and t1.b = t2.b;
count(*)
2000
select count(*) from innodb_redundant t1, innodb_normal t2 where
t1.c1 = t2.c1 and t1.b = t2.b;
count(*)
2000
SELECT variable_value >= 0 FROM information_schema.global_status WHERE variable_name = 'innodb_num_pages_encrypted';
variable_value >= 0
SELECT variable_value > 0 FROM information_schema.global_status WHERE variable_name = 'innodb_num_pages_encrypted';
variable_value > 0
1
SELECT variable_value >= 0 FROM information_schema.global_status WHERE variable_name = 'innodb_num_pages_decrypted';
variable_value >= 0
SELECT variable_value > 0 FROM information_schema.global_status WHERE variable_name = 'innodb_num_pages_decrypted';
variable_value > 0
1
SET GLOBAL innodb_encrypt_tables=OFF;
alter table innodb_compact engine=innodb encrypted=DEFAULT encryption_key_id=DEFAULT;
show create table innodb_compact;
Table Create Table
@ -168,65 +104,6 @@ innodb_redundant CREATE TABLE `innodb_redundant` (
`c1` bigint(20) NOT NULL,
`b` char(200) DEFAULT NULL
) ENGINE=InnoDB DEFAULT CHARSET=latin1 ROW_FORMAT=REDUNDANT
SET GLOBAL innodb_file_format = `Barracuda`;
SET GLOBAL innodb_file_per_table = ON;
show create table innodb_compact;
Table Create Table
innodb_compact CREATE TABLE `innodb_compact` (
`c1` bigint(20) NOT NULL,
`b` char(200) DEFAULT NULL
) ENGINE=InnoDB DEFAULT CHARSET=latin1 ROW_FORMAT=COMPACT
show create table innodb_compressed;
Table Create Table
innodb_compressed CREATE TABLE `innodb_compressed` (
`c1` bigint(20) NOT NULL,
`b` char(200) DEFAULT NULL
) ENGINE=InnoDB DEFAULT CHARSET=latin1 ROW_FORMAT=COMPRESSED
show create table innodb_dynamic;
Table Create Table
innodb_dynamic CREATE TABLE `innodb_dynamic` (
`c1` bigint(20) NOT NULL,
`b` char(200) DEFAULT NULL
) ENGINE=InnoDB DEFAULT CHARSET=latin1 ROW_FORMAT=DYNAMIC
show create table innodb_redundant;
Table Create Table
innodb_redundant CREATE TABLE `innodb_redundant` (
`c1` bigint(20) NOT NULL,
`b` char(200) DEFAULT NULL
) ENGINE=InnoDB DEFAULT CHARSET=latin1 ROW_FORMAT=REDUNDANT
update innodb_normal set c1 = c1 +1;
update innodb_compact set c1 = c1 + 1;
update innodb_compressed set c1 = c1 + 1;
update innodb_dynamic set c1 = c1 + 1;
update innodb_redundant set c1 = c1 + 1;
select count(*) from innodb_compact where c1 < 1500000;
count(*)
2000
select count(*) from innodb_compressed where c1 < 1500000;
count(*)
2000
select count(*) from innodb_dynamic where c1 < 1500000;
count(*)
2000
select count(*) from innodb_redundant where c1 < 1500000;
count(*)
2000
select count(*) from innodb_compact t1, innodb_normal t2 where
t1.c1 = t2.c1 and t1.b = t2.b;
count(*)
2000
select count(*) from innodb_dynamic t1, innodb_normal t2 where
t1.c1 = t2.c1 and t1.b = t2.b;
count(*)
2000
select count(*) from innodb_compressed t1, innodb_normal t2 where
t1.c1 = t2.c1 and t1.b = t2.b;
count(*)
2000
select count(*) from innodb_redundant t1, innodb_normal t2 where
t1.c1 = t2.c1 and t1.b = t2.b;
count(*)
2000
SELECT variable_value >= 0 FROM information_schema.global_status WHERE variable_name = 'innodb_num_pages_encrypted';
variable_value >= 0
1
@ -239,6 +116,8 @@ drop table innodb_compact;
drop table innodb_compressed;
drop table innodb_dynamic;
drop table innodb_redundant;
SET GLOBAL innodb_file_format = `Barracuda`;
SET GLOBAL innodb_file_per_table = ON;
CREATE TABLE t1 (pk INT PRIMARY KEY) ENGINE=InnoDB ENCRYPTION_KEY_ID=2 ENCRYPTED=YES;
INSERT INTO t1 VALUES (1),(2);
# Restarting server...

45
mysql-test/suite/encryption/r/innodb-redo-badkey.result Normal file
View File

@ -0,0 +1,45 @@
call mtr.add_suppression("InnoDB: Block in space_id .* in file .* encrypted.");
call mtr.add_suppression("Plugin 'file_key_management' .*");
call mtr.add_suppression("Plugin 'InnoDB' init function returned error.");
call mtr.add_suppression("Plugin 'InnoDB' registration as a STORAGE ENGINE failed.");
call mtr.add_suppression("InnoDB: Read operation failed for tablespace .*");
call mtr.add_suppression("InnoDB: Database page corruption on disk or a failed.*");
call mtr.add_suppression("InnoDB: Recovery read page .*");
call mtr.add_suppression("InnoDB: The page \[page id: space=[0-9]+, page number=[0-9]+\] in file test/t2 cannot be decrypted.");
call mtr.add_suppression("InnoDB: The page \[page id: space=[0-9]+, page number=[0-9]+\] in file test/t3 cannot be decrypted.");
call mtr.add_suppression("InnoDB: The page \[page id: space=[0-9]+, page number=[0-9]+\] in file test/t4 cannot be decrypted.");
call mtr.add_suppression("InnoDB: The page \[page id: space=[0-9]+, page number=[0-9]+\] in file test/t1 cannot be decrypted.");
call mtr.add_suppression("InnoDB: Missing MLOG_FILE_NAME or MLOG_FILE_DELETE .*");
call mtr.add_suppression("InnoDB: Plugin initialization aborted .*");
call mtr.add_suppression("InnoDB: ############### CORRUPT LOG RECORD FOUND ##################");
# Restart mysqld --file-key-management-filename=keys2.txt
# Wait max 10 min for key encryption threads to encrypt all spaces
SET GLOBAL innodb_file_format = `Barracuda`;
SET GLOBAL innodb_file_per_table = ON;
create table t1(a int not null primary key auto_increment, c char(250), b blob, index(b(10))) engine=innodb row_format=compressed encrypted=yes encryption_key_id=4;
create table t2(a int not null primary key auto_increment, c char(250), b blob, index(b(10))) engine=innodb row_format=compressed;
create table t3(a int not null primary key auto_increment, c char(250), b blob, index(b(10))) engine=innodb encrypted=yes encryption_key_id=4;
create table t4(a int not null primary key auto_increment, c char(250), b blob, index(b(10))) engine=innodb;
begin;
insert into t2 select * from t1;
insert into t3 select * from t1;
insert into t4 select * from t1;
commit;
SET GLOBAL innodb_flush_log_at_trx_commit=1;
begin;
update t1 set c = repeat('secret3', 20);
update t2 set c = repeat('secret4', 20);
update t3 set c = repeat('secret4', 20);
update t4 set c = repeat('secret4', 20);
insert into t1 (c,b) values (repeat('secret5',20), repeat('secret6',6000));
insert into t2 (c,b) values (repeat('secret7',20), repeat('secret8',6000));
insert into t3 (c,b) values (repeat('secret9',20), repeat('secre10',6000));
insert into t4 (c,b) values (repeat('secre11',20), repeat('secre12',6000));
COMMIT;
# Kill the server
# restart
# Kill the server
# Restart mysqld --innodb-force-recovery=1
# Kill the server
# Restart mysqld --file-key-management-filename=keys2.txt
drop table t1, t2,t3,t4;

37
mysql-test/suite/encryption/r/innodb-redo-nokeys.result Normal file
View File

@ -0,0 +1,37 @@
call mtr.add_suppression("InnoDB: Block in space_id .*");
call mtr.add_suppression("mysqld: File .*");
call mtr.add_suppression("Plugin 'file_key_management' .*");
call mtr.add_suppression("InnoDB: cannot enable encryption, encryption plugin is not available");
call mtr.add_suppression("Plugin 'InnoDB' init function returned error.");
call mtr.add_suppression("Plugin 'InnoDB' registration as a STORAGE ENGINE failed.");
call mtr.add_suppression("InnoDB: The page \[page id: space=[0-9]+, page number=[0-9]+\] in file test/t2 cannot be decrypted.");
call mtr.add_suppression("InnoDB: The page \[page id: space=[0-9]+, page number=[0-9]+\] in file test/t3 cannot be decrypted.");
call mtr.add_suppression("InnoDB: The page \[page id: space=[0-9]+, page number=[0-9]+\] in file test/t4 cannot be decrypted.");
call mtr.add_suppression("InnoDB: The page \[page id: space=[0-9]+, page number=[0-9]+\] in file test/t1 cannot be decrypted.");
# Restart mysqld --file-key-management-filename=keys2.txt
SET GLOBAL innodb_file_format = `Barracuda`;
SET GLOBAL innodb_file_per_table = ON;
create table t1(a int not null primary key auto_increment, c char(200), b blob, index(b(10))) engine=innodb row_format=compressed encrypted=yes encryption_key_id=20;
create table t2(a int not null primary key auto_increment, c char(200), b blob, index(b(10))) engine=innodb row_format=compressed;
create table t3(a int not null primary key auto_increment, c char(200), b blob, index(b(10))) engine=innodb encrypted=yes encryption_key_id=20;
create table t4(a int not null primary key auto_increment, c char(200), b blob, index(b(10))) engine=innodb;
begin;
insert into t2 select * from t1;
insert into t3 select * from t1;
insert into t4 select * from t1;
commit;
SET GLOBAL innodb_flush_log_at_trx_commit=1;
begin;
update t1 set c = repeat('secret3', 20);
update t2 set c = repeat('secret4', 20);
update t3 set c = repeat('secret4', 20);
update t4 set c = repeat('secret4', 20);
insert into t1 (c,b) values (repeat('secret5',20), repeat('secret6',6000));
insert into t2 (c,b) values (repeat('secret7',20), repeat('secret8',6000));
insert into t3 (c,b) values (repeat('secret9',20), repeat('secre10',6000));
insert into t4 (c,b) values (repeat('secre11',20), repeat('secre12',6000));
COMMIT;
# Kill the server
# restart
# Restart mysqld --file-key-management-filename=keys2.txt
drop table t1, t2,t3,t4;

118
mysql-test/suite/encryption/r/innodb_encryption-page-compression.result
View File

@ -97,13 +97,8 @@ set current_num = current_num + 1;
end while;
end//
commit;
set autocommit=0;
call innodb_insert_proc(5000);
commit;
set autocommit=1;
select count(*) from innodb_normal;
count(*)
5000
begin;
call innodb_insert_proc(2000);
insert into innodb_page_compressed1 select * from innodb_normal;
insert into innodb_page_compressed2 select * from innodb_normal;
insert into innodb_page_compressed3 select * from innodb_normal;
@ -116,120 +111,45 @@ insert into innodb_page_compressed9 select * from innodb_normal;
commit;
select count(*) from innodb_page_compressed1 where c1 < 500000;
count(*)
5000
2000
select count(*) from innodb_page_compressed2 where c1 < 500000;
count(*)
5000
2000
select count(*) from innodb_page_compressed3 where c1 < 500000;
count(*)
5000
2000
select count(*) from innodb_page_compressed4 where c1 < 500000;
count(*)
5000
2000
select count(*) from innodb_page_compressed5 where c1 < 500000;
count(*)
5000
2000
select count(*) from innodb_page_compressed6 where c1 < 500000;
count(*)
5000
2000
select count(*) from innodb_page_compressed7 where c1 < 500000;
count(*)
5000
2000
select count(*) from innodb_page_compressed8 where c1 < 500000;
count(*)
5000
2000
select count(*) from innodb_page_compressed9 where c1 < 500000;
count(*)
5000
SELECT variable_value >= 0 FROM information_schema.global_status WHERE variable_name = 'innodb_num_pages_encrypted';
variable_value >= 0
2000
SELECT variable_value > 0 FROM information_schema.global_status WHERE variable_name = 'innodb_num_pages_encrypted';
variable_value > 0
1
SELECT variable_value >= 0 FROM information_schema.global_status WHERE variable_name = 'innodb_num_pages_decrypted';
variable_value >= 0
1
SELECT variable_value >= 0 FROM information_schema.global_status WHERE variable_name = 'innodb_num_pages_page_compressed';
variable_value >= 0
1
SELECT variable_value >= 0 FROM information_schema.global_status WHERE variable_name = 'innodb_num_pages_page_decompressed';
variable_value >= 0
1
select count(*) from innodb_page_compressed1 where c1 < 500000;
count(*)
5000
select count(*) from innodb_page_compressed2 where c1 < 500000;
count(*)
5000
select count(*) from innodb_page_compressed3 where c1 < 500000;
count(*)
5000
select count(*) from innodb_page_compressed4 where c1 < 500000;
count(*)
5000
select count(*) from innodb_page_compressed5 where c1 < 500000;
count(*)
5000
select count(*) from innodb_page_compressed6 where c1 < 500000;
count(*)
5000
select count(*) from innodb_page_compressed7 where c1 < 500000;
count(*)
5000
select count(*) from innodb_page_compressed8 where c1 < 500000;
count(*)
5000
select count(*) from innodb_page_compressed9 where c1 < 500000;
count(*)
5000
update innodb_page_compressed1 set c1 = c1 + 1;
update innodb_page_compressed2 set c1 = c1 + 1;
update innodb_page_compressed3 set c1 = c1 + 1;
update innodb_page_compressed4 set c1 = c1 + 1;
update innodb_page_compressed5 set c1 = c1 + 1;
update innodb_page_compressed6 set c1 = c1 + 1;
update innodb_page_compressed7 set c1 = c1 + 1;
update innodb_page_compressed8 set c1 = c1 + 1;
update innodb_page_compressed9 set c1 = c1 + 1;
SELECT variable_value >= 0 FROM information_schema.global_status WHERE variable_name = 'innodb_num_pages_encrypted';
variable_value >= 0
1
SELECT variable_value >= 0 FROM information_schema.global_status WHERE variable_name = 'innodb_num_pages_decrypted';
variable_value >= 0
1
SELECT variable_value >= 0 FROM information_schema.global_status WHERE variable_name = 'innodb_num_pages_page_compressed';
variable_value >= 0
SELECT variable_value > 0 FROM information_schema.global_status WHERE variable_name = 'innodb_num_pages_page_compressed';
variable_value > 0
1
SELECT variable_value >= 0 FROM information_schema.global_status WHERE variable_name = 'innodb_num_pages_page_decompressed';
variable_value >= 0
1
SET GLOBAL innodb_encryption_threads = 4;
SET GLOBAL innodb_encrypt_tables = off;
select count(*) from innodb_page_compressed1 where c1 < 500000;
count(*)
5000
select count(*) from innodb_page_compressed2 where c1 < 500000;
count(*)
5000
select count(*) from innodb_page_compressed3 where c1 < 500000;
count(*)
5000
select count(*) from innodb_page_compressed4 where c1 < 500000;
count(*)
5000
select count(*) from innodb_page_compressed5 where c1 < 500000;
count(*)
5000
select count(*) from innodb_page_compressed6 where c1 < 500000;
count(*)
5000
select count(*) from innodb_page_compressed7 where c1 < 500000;
count(*)
5000
select count(*) from innodb_page_compressed8 where c1 < 500000;
count(*)
5000
select count(*) from innodb_page_compressed9 where c1 < 500000;
count(*)
5000
update innodb_page_compressed1 set c1 = c1 + 1;
update innodb_page_compressed2 set c1 = c1 + 1;
update innodb_page_compressed3 set c1 = c1 + 1;
@ -242,11 +162,11 @@ update innodb_page_compressed9 set c1 = c1 + 1;
SELECT variable_value >= 0 FROM information_schema.global_status WHERE variable_name = 'innodb_num_pages_encrypted';
variable_value >= 0
1
SELECT variable_value >= 0 FROM information_schema.global_status WHERE variable_name = 'innodb_num_pages_decrypted';
variable_value >= 0
SELECT variable_value > 0 FROM information_schema.global_status WHERE variable_name = 'innodb_num_pages_decrypted';
variable_value > 0
1
SELECT variable_value >= 0 FROM information_schema.global_status WHERE variable_name = 'innodb_num_pages_page_compressed';
variable_value >= 0
SELECT variable_value > 0 FROM information_schema.global_status WHERE variable_name = 'innodb_num_pages_page_compressed';
variable_value > 0
1
SELECT variable_value >= 0 FROM information_schema.global_status WHERE variable_name = 'innodb_num_pages_page_decompressed';
variable_value >= 0

8
mysql-test/suite/encryption/r/innodb_onlinealter_encryption.result
View File

@ -12,8 +12,8 @@ begin
declare current_num int;
set current_num = 0;
while current_num < repeat_count do
insert into t1 values (current_num,repeat('foobar',42));
insert into t2 values (current_num,repeat('temp', 42));
insert into t1 values (current_num,repeat('foobar',12));
insert into t2 values (current_num,repeat('tempsecret', 12));
insert into t3 values (current_num,repeat('barfoo',42));
insert into t4 values (current_num,repeat('repeat',42));
insert into t5 values (current_num,substring('A BC DEF GHIJ KLM NOPQRS TUV WXYZ 012 3456789', rand()*36+1, 100), repeat('author new',22));
@ -31,7 +31,7 @@ set autocommit=1;
# t1 yes on expecting NOT FOUND
NOT FOUND /foobar/ in t1.ibd
# t2 ... on expecting NOT FOUND
NOT FOUND /temp/ in t2.ibd
NOT FOUND /tempsecret/ in t2.ibd
# t3 ... on expecting NOT FOUND
NOT FOUND /barfoo/ in t3.ibd
# t4 ... on expecting NOT FOUND
@ -126,7 +126,7 @@ t7 CREATE TABLE `t7` (
# t1 yes on expecting NOT FOUND
NOT FOUND /foobar/ in t1.ibd
# t2 ... on expecting NOT FOUND
NOT FOUND /temp/ in t2.ibd
NOT FOUND /tempsecret/ in t2.ibd
# t3 ... on expecting NOT FOUND
NOT FOUND /barfoo/ in t3.ibd
# t4 ... on expecting NOT FOUND

2
mysql-test/suite/encryption/t/encrypt_and_grep.opt
View File

@ -1,7 +1,7 @@
--innodb-encrypt-tables=ON
--innodb-encrypt-log=ON
--innodb-encryption-rotate-key-age=15
--innodb-encryption-threads=1
--innodb-encryption-threads=4
--innodb-tablespaces-encryption

43
mysql-test/suite/encryption/t/encrypt_and_grep.test
View File

@ -13,17 +13,20 @@
--let t2_IBD = $MYSQLD_DATADIR/test/t2.ibd
--let t3_IBD = $MYSQLD_DATADIR/test/t3.ibd
--let SEARCH_RANGE = 10000000
--let SEARCH_PATTERN=foobar
--disable_warnings
SET GLOBAL innodb_file_per_table = ON;
SET GLOBAL innodb_file_format = `Barracuda`;
--enable_warnings
create table t1 (a varchar(255)) engine=innodb encrypted=yes;
create table t2 (a varchar(255)) engine=innodb;
show warnings;
create table t3 (a varchar(255)) engine=innodb encrypted=no;
insert t1 values (repeat('foobar', 42));
insert t2 values (repeat('temp', 42));
insert t3 values (repeat('dummy', 42));
insert t1 values (repeat('foobarsecret', 22));
insert t2 values (repeat('tempsecret', 22));
insert t3 values (repeat('dummysecret', 22));
--echo # Wait max 10 min for key encryption threads to encrypt all spaces
--let $wait_timeout= 600
@ -35,19 +38,19 @@ SELECT NAME FROM INFORMATION_SCHEMA.INNODB_TABLESPACES_ENCRYPTION WHERE MIN_KEY_
--source include/shutdown_mysqld.inc
--let SEARCH_PATTERN=foobar
--let SEARCH_PATTERN=foobarsecret
--echo # t1 yes on expecting NOT FOUND
-- let SEARCH_FILE=$t1_IBD
-- source include/search_pattern_in_file.inc
--let SEARCH_PATTERN=temp
--let SEARCH_PATTERN=tempsecret
--echo # t2 ... on expecting NOT FOUND
-- let SEARCH_FILE=$t2_IBD
-- source include/search_pattern_in_file.inc
--let SEARCH_PATTERN=dummy
--let SEARCH_PATTERN=dummysecret
--echo # t3 no on expecting FOUND
-- let SEARCH_FILE=$t3_IBD
-- source include/search_pattern_in_file.inc
--let SEARCH_PATTERN=foobar
--let SEARCH_PATTERN=foobarsecret
--echo # ibdata1 expecting NOT FOUND
-- let SEARCH_FILE=$ib1_IBD
-- source include/search_pattern_in_file.inc
@ -55,12 +58,12 @@ SELECT NAME FROM INFORMATION_SCHEMA.INNODB_TABLESPACES_ENCRYPTION WHERE MIN_KEY_
-- source include/start_mysqld.inc
--echo # Now turn off encryption and wait for threads to decrypt everything
SET GLOBAL innodb_encryption_threads = 1;
SET GLOBAL innodb_encrypt_tables = off;
--echo # Wait max 10 min for key encryption threads to decrypt all spaces
--let $wait_timeout= 600
--let $wait_condition=SELECT COUNT(*) = 5 FROM INFORMATION_SCHEMA.INNODB_TABLESPACES_ENCRYPTION WHERE MIN_KEY_VERSION = 0 AND KEY_ROTATION_PAGE_NUMBER IS NULL;
--let $wait_condition=SELECT COUNT(*) = 5 FROM INFORMATION_SCHEMA.INNODB_TABLESPACES_ENCRYPTION WHERE MIN_KEY_VERSION = 0 AND CURRENT_KEY_VERSION = 0;
--source include/wait_condition.inc
SELECT NAME FROM INFORMATION_SCHEMA.INNODB_TABLESPACES_ENCRYPTION WHERE MIN_KEY_VERSION = 0;
@ -68,27 +71,27 @@ SELECT NAME FROM INFORMATION_SCHEMA.INNODB_TABLESPACES_ENCRYPTION WHERE MIN_KEY_
--source include/shutdown_mysqld.inc
--let SEARCH_PATTERN=foobar
--let SEARCH_PATTERN=foobarsecret
--echo # t1 yes on expecting NOT FOUND
-- let SEARCH_FILE=$t1_IBD
-- source include/search_pattern_in_file.inc
--let SEARCH_PATTERN=temp
--echo # t2 ... on expecting FOUND
--let SEARCH_PATTERN=tempsecret
--echo # t2 ... default expecting FOUND
-- let SEARCH_FILE=$t2_IBD
-- source include/search_pattern_in_file.inc
--let SEARCH_PATTERN=dummy
--let SEARCH_PATTERN=dummysecret
--echo # t3 no on expecting FOUND
-- let SEARCH_FILE=$t3_IBD
-- source include/search_pattern_in_file.inc
--let SEARCH_PATTERN=foobar
--let SEARCH_PATTERN=foobarsecret
--echo # ibdata1 expecting NOT FOUND
-- let SEARCH_FILE=$ib1_IBD
-- source include/search_pattern_in_file.inc
-- source include/start_mysqld.inc
--echo # Now turn on encryption and wait for threads to encrypt all spaces
SET GLOBAL innodb_encryption_threads = 4;
SET GLOBAL innodb_encrypt_tables = on;
--echo # Wait max 10 min for key encryption threads to encrypt all spaces
@ -101,19 +104,19 @@ SELECT NAME FROM INFORMATION_SCHEMA.INNODB_TABLESPACES_ENCRYPTION WHERE MIN_KEY_
--source include/shutdown_mysqld.inc
--let SEARCH_PATTERN=foobar
--let SEARCH_PATTERN=foobarsecret
--echo # t1 yes on expecting NOT FOUND
-- let SEARCH_FILE=$t1_IBD
-- source include/search_pattern_in_file.inc
--let SEARCH_PATTERN=temp
--let SEARCH_PATTERN=tempsecret
--echo # t2 ... on expecting NOT FOUND
-- let SEARCH_FILE=$t2_IBD
-- source include/search_pattern_in_file.inc
--let SEARCH_PATTERN=dummy
--let SEARCH_PATTERN=dummysecret
--echo # t3 no on expecting FOUND
-- let SEARCH_FILE=$t3_IBD
-- source include/search_pattern_in_file.inc
--let SEARCH_PATTERN=foobar
--let SEARCH_PATTERN=foobarsecret
--echo # ibdata1 expecting NOT FOUND
-- let SEARCH_FILE=$ib1_IBD
-- source include/search_pattern_in_file.inc

70
mysql-test/suite/encryption/t/innodb-bad-key-change.test
View File

@ -2,38 +2,26 @@
-- source include/have_file_key_management_plugin.inc
# embedded does not support restart
-- source include/not_embedded.inc
-- source include/not_valgrind.inc
# Avoid CrashReporter popup on Mac
-- source include/not_crashrep.inc
-- source include/not_windows.inc
#
# MDEV-8588: Assertion failure in file ha_innodb.cc line 21140 if at least one encrypted
# table exists and encryption service is not available.
#
call mtr.add_suppression("Plugin 'file_key_management' init function returned error");
call mtr.add_suppression("InnoDB: Database page corruption on disk or a failed.*");
call mtr.add_suppression("Plugin 'file_key_management' registration.*failed");
call mtr.add_suppression("InnoDB: Block in space_id .* in file test/.* encrypted");
call mtr.add_suppression("InnoDB: However key management plugin or used key_version .* is not found or used encryption algorithm or method does not match.");
call mtr.add_suppression("InnoDB: Marking tablespace as missing. You may drop this table or install correct key management plugin and key file.");
call mtr.add_suppression(".*InnoDB: Cannot open table test/.* from the internal data dictionary of InnoDB though the .frm file for the table exists. See .* for how you can resolve the problem.");
call mtr.add_suppression("InnoDB: .ibd file is missing for table test/.*");
call mtr.add_suppression("InnoDB: The page .*");
call mtr.add_suppression("InnoDB: Cannot calculate statistics for table .* ");
call mtr.add_suppression("Plugin 'file_key_management' .*");
call mtr.add_suppression("mysqld: File .*");
call mtr.add_suppression("InnoDB: Tablespace id .* is encrypted but encryption service or used key_id .* is not available. Can't continue opening tablespace.");
--echo
--echo # Start server with keys2.txt
-- let $restart_parameters=--file-key-management-filename=$MYSQL_TEST_DIR/std_data/keys2.txt
-- source include/restart_mysqld.inc
--disable_query_log
let $innodb_file_format_orig = `SELECT @@innodb_file_format`;
let $innodb_file_per_table_orig = `SELECT @@innodb_file_per_table`;
--enable_query_log
--disable_warnings
SET GLOBAL innodb_file_format = `Barracuda`;
SET GLOBAL innodb_file_per_table = ON;
--enable_warnings
CREATE TABLE t1 (c VARCHAR(8)) ENGINE=InnoDB ENCRYPTED=YES ENCRYPTION_KEY_ID=2;
INSERT INTO t1 VALUES ('foobar');
@ -54,7 +42,7 @@ SELECT * FROM t1;
--error ER_GET_ERRMSG
SELECT * FROM t1;
--replace_regex /tablespace [0-9]*/tablespace /
--replace_regex /tablespace [0-9]*/tablespace / /key_id [0-9]*/key_id /
SHOW WARNINGS;
-- let $restart_parameters=--file-key-management-filename=$MYSQL_TEST_DIR/std_data/keysbad3.txt
@ -62,7 +50,7 @@ SHOW WARNINGS;
--replace_regex /tablespace [0-9]*/tablespace /
DROP TABLE t1;
--replace_regex /tablespace [0-9]*/tablespace /
--replace_regex /tablespace [0-9]*/tablespace / /key_id [0-9]*/key_id /
SHOW WARNINGS;
#
@ -70,10 +58,11 @@ SHOW WARNINGS;
# line 2856 on querying a table using wrong default encryption key
#
--echo # Start server with keys.txt
-- let $restart_parameters=--file-key-management-filename=$MYSQL_TEST_DIR/std_data/keys.txt
--echo # Start server with keys3.txt
-- let $restart_parameters=--file-key-management-filename=$MYSQL_TEST_DIR/std_data/keys3.txt
-- source include/restart_mysqld.inc
SET GLOBAL innodb_default_encryption_key_id=5;
CREATE TABLE t2 (c VARCHAR(8), id int not null primary key, b int, key(b)) ENGINE=InnoDB ENCRYPTED=YES;
INSERT INTO t2 VALUES ('foobar',1,2);
@ -84,56 +73,51 @@ INSERT INTO t2 VALUES ('foobar',1,2);
--error ER_GET_ERRMSG
SELECT * FROM t2;
--replace_regex /tablespace [0-9]*/tablespace /
--replace_regex /tablespace [0-9]*/tablespace / /key_id [0-9]*/key_id /
SHOW WARNINGS;
--error ER_GET_ERRMSG
SELECT * FROM t2 where id = 1;
--replace_regex /tablespace [0-9]*/tablespace /
--replace_regex /tablespace [0-9]*/tablespace / /key_id [0-9]*/key_id /
SHOW WARNINGS;
--error ER_GET_ERRMSG
SELECT * FROM t2 where b = 1;
--replace_regex /tablespace [0-9]*/tablespace /
--replace_regex /tablespace [0-9]*/tablespace / /key_id [0-9]*/key_id /
SHOW WARNINGS;
--replace_regex /tablespace [0-9]*/tablespace /
--error ER_GET_ERRMSG
INSERT INTO t2 VALUES ('tmp',3,3);
--replace_regex /tablespace [0-9]*/tablespace /
--replace_regex /tablespace [0-9]*/tablespace / /key_id [0-9]*/key_id /
SHOW WARNINGS;
--error ER_GET_ERRMSG
DELETE FROM t2 where b = 3;
--replace_regex /tablespace [0-9]*/tablespace /
--replace_regex /tablespace [0-9]*/tablespace / /key_id [0-9]*/key_id /
SHOW WARNINGS;
--error ER_GET_ERRMSG
DELETE FROM t2 where id = 3;
--replace_regex /tablespace [0-9]*/tablespace /
--replace_regex /tablespace [0-9]*/tablespace / /key_id [0-9]*/key_id /
SHOW WARNINGS;
--error ER_GET_ERRMSG
UPDATE t2 set b = b +1;
--replace_regex /tablespace [0-9]*/tablespace /
--replace_regex /tablespace [0-9]*/tablespace / /key_id [0-9]*/key_id /
SHOW WARNINGS;
OPTIMIZE TABLE t2;
--replace_regex /tablespace [0-9]*/tablespace /
--replace_regex /tablespace [0-9]*/tablespace / /key_id [0-9]*/key_id /
SHOW WARNINGS;
--error ER_GET_ERRMSG
ALTER TABLE t2 ADD COLUMN c INT;
--replace_regex /tablespace [0-9]*/tablespace /
ALTER TABLE t2 ADD COLUMN d INT;
--replace_regex /tablespace [0-9]*/tablespace / /key_id [0-9]*/key_id /
SHOW WARNINGS;
ANALYZE TABLE t2;
--replace_regex /tablespace [0-9]*/tablespace /
--replace_regex /tablespace [0-9]*/tablespace / /key_id [0-9]*/key_id /
SHOW WARNINGS;
--error ER_GET_ERRMSG
TRUNCATE TABLE t2;
--replace_regex /tablespace [0-9]*/tablespace /
--replace_regex /tablespace [0-9]*/tablespace / /key_id [0-9]*/key_id /
SHOW WARNINGS;
--error ER_GET_ERRMSG
DROP TABLE t2;
--replace_regex /tablespace [0-9]*/tablespace /
SHOW WARNINGS;
--echo
--echo # Restart server with keys.txt
-- let $restart_parameters=--file-key-management-filename=$MYSQL_TEST_DIR/std_data/keys.txt
--echo # Start server with keys2.txt
-- let $restart_parameters=--file-key-management-filename=$MYSQL_TEST_DIR/std_data/keys2.txt
-- source include/restart_mysqld.inc
DROP TABLE t2;
--replace_regex /tablespace [0-9]*/tablespace /
SHOW WARNINGS;

126
mysql-test/suite/encryption/t/innodb-bad-key-change2.test
View File

@ -1,59 +1,103 @@
--source include/have_innodb.inc
# embedded does not support restart
-- source include/not_embedded.inc
-- source include/not_valgrind.inc
# Avoid CrashReporter popup on Mac
-- source include/not_crashrep.inc
-- source filekeys_plugin_exists.inc
#
# MDEV-8750: Server crashes in page_cur_is_after_last on altering table using a wrong encryption key
# MDEV-8769: Server crash at file btr0btr.ic line 122 when defragmenting encrypted table using incorrect keys
# MDEV-8768: Server crash at file btr0btr.ic line 122 when checking encrypted table using incorrect keys
# MDEV-8727: Server/InnoDB hangs on shutdown after trying to read an encrypted table with a wrong key
#
call mtr.add_suppression("InnoDB: Block in space_id .* in file test/.* encrypted");
call mtr.add_suppression("InnoDB: However key management plugin or used key_version .* is not found or used encryption algorithm or method does not match.");
call mtr.add_suppression("InnoDB: Marking tablespace as missing. You may drop this table or install correct key management plugin and key file.");
call mtr.add_suppression(".*InnoDB: Cannot open table test/.* from the internal data dictionary of InnoDB though the .frm file for the table exists. See .* for how you can resolve the problem.");
call mtr.add_suppression("InnoDB: .ibd file is missing for table test/.*");
call mtr.add_suppression("InnoDB: The page \[page id: space=[0-9]+, page number=[0-9]+\] in file test/t1new cannot be decrypted.");
call mtr.add_suppression("InnoDB: The page \[page id: space=[0-9]+, page number=[0-9]+\] in file test/t1 cannot be decrypted.");
# Suppression for builds where file_key_management plugin is linked statically
call mtr.add_suppression("Couldn't load plugins from 'file_key_management*");
call mtr.add_suppression("Couldn't load plugins from 'file_key_management.*");
call mtr.add_suppression("InnoDB: Table \'\"test\".\"t1\"\' tablespace is set as discarded.");
--exec echo "wait" > $MYSQLTEST_VARDIR/tmp/mysqld.1.expect
--shutdown_server
--source include/wait_until_disconnected.inc
--let $restart_parameters=--plugin-load-add=file_key_management.so --file-key-management --file-key-management-filename=$MYSQL_TEST_DIR/std_data/keys2.txt
--source include/restart_mysqld.inc
--write_file $MYSQLTEST_VARDIR/keys1.txt
1;770A8A65DA156D24EE2A093277530142
4;770A8A65DA156D24EE2A093277530143
EOF
--disable_warnings
SET GLOBAL innodb_file_format = `Barracuda`;
SET GLOBAL innodb_file_per_table = ON;
--enable_warnings
--exec echo "restart:--innodb-encrypt-tables --innodb-stats-persistent --plugin-load-add=file_key_management.so --file-key-management --file-key-management-filename=$MYSQLTEST_VARDIR/keys1.txt" > $MYSQLTEST_VARDIR/tmp/mysqld.1.expect
--enable_reconnect
--source include/wait_until_connected_again.inc
CREATE TABLE t1 (pk INT PRIMARY KEY, f VARCHAR(8)) ENGINE=InnoDB ENCRYPTION_KEY_ID=4;
CREATE TABLE t1 (pk INT PRIMARY KEY, f VARCHAR(8)) ENGINE=InnoDB ENCRYPTED=YES ENCRYPTION_KEY_ID=4;
INSERT INTO t1 VALUES (1,'foo'),(2,'bar');
--exec echo "wait" > $MYSQLTEST_VARDIR/tmp/mysqld.1.expect
--shutdown_server
--source include/wait_until_disconnected.inc
--let $restart_parameters=--plugin-load-add=file_key_management.so --file-key-management --file-key-management-filename=$MYSQL_TEST_DIR/std_data/keys3.txt
--source include/restart_mysqld.inc
--write_file $MYSQLTEST_VARDIR/keys2.txt
1;770A8A65DA156D24EE2A093277530142
4;770A8A65DA156D24EE2A093277530144
--error ER_GET_ERRMSG
SELECT * FROM t1;
--replace_regex /tablespace [0-9]*/tablespace / /key_id [0-9]*/key_id /
SHOW WARNINGS;
--error ER_GET_ERRMSG
ALTER TABLE t1 engine=InnoDB;
--replace_regex /tablespace [0-9]*/tablespace / /key_id [0-9]*/key_id /
SHOW WARNINGS;
OPTIMIZE TABLE t1;
--replace_regex /tablespace [0-9]*/tablespace / /key_id [0-9]*/key_id /
SHOW WARNINGS;
CHECK TABLE t1;
--replace_regex /tablespace [0-9]*/tablespace / /key_id [0-9]*/key_id /
SHOW WARNINGS;
--let $restart_parameters=--plugin-load-add=file_key_management.so --file-key-management --file-key-management-filename=$MYSQL_TEST_DIR/std_data/keys2.txt
--source include/restart_mysqld.inc
let MYSQLD_DATADIR =`SELECT @@datadir`;
let MYSQLD_TMPDIR = `SELECT @@tmpdir`;
FLUSH TABLES t1 FOR EXPORT;
perl;
do "$ENV{MTR_SUITE_DIR}/include/innodb-util.pl";
ib_backup_tablespaces("test", "t1");
EOF
UNLOCK TABLES;
--let $restart_parameters=--plugin-load-add=file_key_management.so --file-key-management --file-key-management-filename=$MYSQL_TEST_DIR/std_data/keys3.txt
--source include/restart_mysqld.inc
# Discard should pass even with incorrect keys
--replace_regex /tablespace [0-9]*/tablespace / /key_id [0-9]*/key_id /
ALTER TABLE t1 DISCARD TABLESPACE;
perl;
do "$ENV{MTR_SUITE_DIR}/include/innodb-util.pl";
ib_discard_tablespaces("test", "t1");
ib_restore_tablespaces("test", "t1");
EOF
--exec echo "restart:--innodb-encrypt-tables --innodb-stats-persistent --plugin-load-add=file_key_management.so --file-key-management --file-key-management-filename=$MYSQLTEST_VARDIR/keys2.txt" > $MYSQLTEST_VARDIR/tmp/mysqld.1.expect
--enable_reconnect
--source include/wait_until_connected_again.inc
--let $restart_parameters=--plugin-load-add=file_key_management.so --file-key-management --file-key-management-filename=$MYSQL_TEST_DIR/std_data/keys2.txt
--source include/restart_mysqld.inc
--error ER_GET_ERRMSG
select * from t1;
show warnings;
--error ER_GET_ERRMSG
alter table t1 discard tablespace;
show warnings;
--error ER_GET_ERRMSG
alter table t1 engine=InnoDB;
show warnings;
ALTER TABLE t1 IMPORT TABLESPACE;
SHOW CREATE TABLE t1;
--remove_file $MYSQLTEST_VARDIR/keys1.txt
--remove_file $MYSQLTEST_VARDIR/keys2.txt
--let $restart_parameters=--plugin-load-add=file_key_management.so --file-key-management --file-key-management-filename=$MYSQL_TEST_DIR/std_data/keys3.txt
--source include/restart_mysqld.inc
# Rename table should pass even with incorrect keys
RENAME TABLE t1 TO t1new;
# Alter table rename is not allowed with incorrect keys
--replace_regex /tablespace [0-9]*/tablespace / /key_id [0-9]*/key_id /
--error ER_GET_ERRMSG
ALTER TABLE t1new RENAME TO t2new;
# Drop should pass even with incorrect keys
--replace_regex /tablespace [0-9]*/tablespace / /key_id [0-9]*/key_id /
DROP TABLE t1new;
#
# Reset environment
#
--let $restart_parameters=
--source include/restart_mysqld.inc
--error ER_BAD_TABLE_ERROR
DROP TABLE t1, t1new;

0
mysql-test/suite/encryption/t/innodb-bad-key-change5.opt → mysql-test/suite/encryption/t/innodb-bad-key-change4.opt
View File

88
mysql-test/suite/encryption/t/innodb-bad-key-change4.test
View File

@ -1,89 +1,43 @@
--source include/have_innodb.inc
# embedded does not support restart
-- source include/not_embedded.inc
-- source include/not_valgrind.inc
# Avoid CrashReporter popup on Mac
-- source include/not_crashrep.inc
-- source filekeys_plugin_exists.inc
#
# MDEV-8769: Server crash at file btr0btr.ic line 122 when defragmenting encrypted table using incorrect keys
# MDEV-8768: Server crash at file btr0btr.ic line 122 when checking encrypted table using incorrect keys
#
call mtr.add_suppression("InnoDB: Block in space_id .* in file test/.* encrypted");
call mtr.add_suppression("InnoDB: However key management plugin or used key_version .* is not found or used encryption algorithm or method does not match.");
call mtr.add_suppression("InnoDB: Marking tablespace as missing. You may drop this table or install correct key management plugin and key file.");
call mtr.add_suppression(".*InnoDB: Cannot open table test/.* from the internal data dictionary of InnoDB though the .frm file for the table exists. See .* for how you can resolve the problem.");
call mtr.add_suppression("InnoDB: The page \[page id: space=[0-9]+, page number=[0-9]+\] in file test/t1 cannot be decrypted.");
call mtr.add_suppression("InnoDB: Cannot open table .*");
call mtr.add_suppression("InnoDB: .ibd file is missing for table test/.*");
# Suppression for builds where file_key_management plugin is linked statically
call mtr.add_suppression("Couldn't load plugins from 'file_key_management*");
call mtr.add_suppression("Couldn't load plugins from 'file_key_management.*");
--exec echo "wait" > $MYSQLTEST_VARDIR/tmp/mysqld.1.expect
--shutdown_server
--source include/wait_until_disconnected.inc
--write_file $MYSQLTEST_VARDIR/keys1.txt
1;770A8A65DA156D24EE2A093277530142
4;770A8A65DA156D24EE2A093277530143
EOF
--exec echo "restart:--innodb-encrypt-tables --innodb-stats-persistent --plugin-load-add=file_key_management.so --file-key-management --file-key-management-filename=$MYSQLTEST_VARDIR/keys1.txt" > $MYSQLTEST_VARDIR/tmp/mysqld.1.expect
--enable_reconnect
--source include/wait_until_connected_again.inc
--let $MYSQLD_TMPDIR = `SELECT @@tmpdir`
--let $MYSQLD_DATADIR = `SELECT @@datadir`
--disable_query_log
let $innodb_file_format_orig = `SELECT @@innodb_file_format`;
let $innodb_file_per_table_orig = `SELECT @@innodb_file_per_table`;
--enable_query_log
--let $restart_parameters=--plugin-load-add=file_key_management.so --file-key-management --file-key-management-filename=$MYSQL_TEST_DIR/std_data/keys2.txt
--source include/restart_mysqld.inc
--disable_warnings
SET GLOBAL innodb_file_format = `Barracuda`;
SET GLOBAL innodb_file_per_table = ON;
--enable_warnings
CREATE TABLE t1 (pk INT PRIMARY KEY, f VARCHAR(8)) ENGINE=InnoDB ENCRYPTION_KEY_ID=4;
CREATE TABLE t1 (pk INT PRIMARY KEY, f VARCHAR(8)) ENGINE=InnoDB ENCRYPTED=YES ENCRYPTION_KEY_ID=4;
INSERT INTO t1 VALUES (1,'foo'),(2,'bar');
--exec echo "wait" > $MYSQLTEST_VARDIR/tmp/mysqld.1.expect
--shutdown_server
--source include/wait_until_disconnected.inc
--let $restart_parameters=--plugin-load-add=file_key_management.so --file-key-management --file-key-management-filename=$MYSQL_TEST_DIR/std_data/keys3.txt
--source include/restart_mysqld.inc
--write_file $MYSQLTEST_VARDIR/keys2.txt
1;770A8A65DA156D24EE2A093277530142
4;770A8A65DA156D24EE2A093277530144
EOF
--exec echo "restart:--innodb-encrypt-tables --innodb-stats-persistent --plugin-load-add=file_key_management.so --file-key-management --file-key-management-filename=$MYSQLTEST_VARDIR/keys2.txt" > $MYSQLTEST_VARDIR/tmp/mysqld.1.expect
--enable_reconnect
--source include/wait_until_connected_again.inc
SET GLOBAL innodb_file_format = `Barracuda`;
SET GLOBAL innodb_file_per_table = ON;
CHECK TABLE t1;
--replace_regex /tablespace [0-9]*/tablespace / /key_id [0-9]*/key_id /
OPTIMIZE TABLE t1;
--replace_regex /tablespace [0-9]*/tablespace / /key_id [0-9]*/key_id /
SHOW WARNINGS;
--remove_file $MYSQLTEST_VARDIR/keys1.txt
--remove_file $MYSQLTEST_VARDIR/keys2.txt
--replace_regex /tablespace [0-9]*/tablespace / /key_id [0-9]*/key_id /
CHECK TABLE t1;
--replace_regex /tablespace [0-9]*/tablespace / /key_id [0-9]*/key_id /
SHOW WARNINGS;
--exec echo "wait" > $MYSQLTEST_VARDIR/tmp/mysqld.1.expect
--shutdown_server
--source include/wait_until_disconnected.inc
--write_file $MYSQLTEST_VARDIR/keys1.txt
1;770A8A65DA156D24EE2A093277530142
4;770A8A65DA156D24EE2A093277530143
EOF
--exec echo "restart:--innodb-encrypt-tables --innodb-stats-persistent --plugin-load-add=file_key_management.so --file-key-management --file-key-management-filename=$MYSQLTEST_VARDIR/keys1.txt" > $MYSQLTEST_VARDIR/tmp/mysqld.1.expect
--enable_reconnect
--source include/wait_until_connected_again.inc
--let $restart_parameters=--plugin-load-add=file_key_management.so --file-key-management --file-key-management-filename=$MYSQL_TEST_DIR/std_data/keys2.txt
--source include/restart_mysqld.inc
DROP TABLE t1;
--remove_file $MYSQLTEST_VARDIR/keys1.txt
# reset system
--disable_query_log
EVAL SET GLOBAL innodb_file_per_table = $innodb_file_per_table_orig;
EVAL SET GLOBAL innodb_file_format = $innodb_file_format_orig;
--enable_query_log

88
mysql-test/suite/encryption/t/innodb-bad-key-change5.test
View File

@ -1,88 +0,0 @@
--source include/have_innodb.inc
# embedded does not support restart
-- source include/not_embedded.inc
-- source include/not_valgrind.inc
# Avoid CrashReporter popup on Mac
-- source include/not_crashrep.inc
-- source filekeys_plugin_exists.inc
#
# MDEV-8769: Server crash at file btr0btr.ic line 122 when defragmenting encrypted table using incorrect keys
#
call mtr.add_suppression("InnoDB: Block in space_id .* in file test/.* encrypted");
call mtr.add_suppression("InnoDB: However key management plugin or used key_version .* is not found or used encryption algorithm or method does not match.");
call mtr.add_suppression("InnoDB: Marking tablespace as missing. You may drop this table or install correct key management plugin and key file.");
call mtr.add_suppression(".*InnoDB: Cannot open table test/.* from the internal data dictionary of InnoDB though the .frm file for the table exists. See .* for how you can resolve the problem.");
call mtr.add_suppression("InnoDB: .ibd file is missing for table test/.*");
# Suppression for builds where file_key_management plugin is linked statically
call mtr.add_suppression("Couldn't load plugins from 'file_key_management*");
--exec echo "wait" > $MYSQLTEST_VARDIR/tmp/mysqld.1.expect
--shutdown_server
--source include/wait_until_disconnected.inc
--write_file $MYSQLTEST_VARDIR/keys1.txt
1;770A8A65DA156D24EE2A093277530142
4;770A8A65DA156D24EE2A093277530143
EOF
--exec echo "restart:--innodb-encrypt-tables --innodb-defragment=1 --innodb-stats-persistent --plugin-load-add=file_key_management.so --file-key-management --file-key-management-filename=$MYSQLTEST_VARDIR/keys1.txt" > $MYSQLTEST_VARDIR/tmp/mysqld.1.expect
--enable_reconnect
--source include/wait_until_connected_again.inc
--let $MYSQLD_TMPDIR = `SELECT @@tmpdir`
--let $MYSQLD_DATADIR = `SELECT @@datadir`
--disable_query_log
let $innodb_file_format_orig = `SELECT @@innodb_file_format`;
let $innodb_file_per_table_orig = `SELECT @@innodb_file_per_table`;
--enable_query_log
SET GLOBAL innodb_file_format = `Barracuda`;
SET GLOBAL innodb_file_per_table = ON;
CREATE TABLE t1 (pk INT PRIMARY KEY, f VARCHAR(8)) ENGINE=InnoDB ENCRYPTION_KEY_ID=4;
INSERT INTO t1 VALUES (1,'foo'),(2,'bar');
--exec echo "wait" > $MYSQLTEST_VARDIR/tmp/mysqld.1.expect
--shutdown_server
--source include/wait_until_disconnected.inc
--write_file $MYSQLTEST_VARDIR/keys2.txt
1;770A8A65DA156D24EE2A093277530142
4;770A8A65DA156D24EE2A093277530144
EOF
--exec echo "restart:--innodb-encrypt-tables --innodb-defragment=1 --innodb-stats-persistent --plugin-load-add=file_key_management.so --file-key-management --file-key-management-filename=$MYSQLTEST_VARDIR/keys2.txt" > $MYSQLTEST_VARDIR/tmp/mysqld.1.expect
--enable_reconnect
--source include/wait_until_connected_again.inc
SET GLOBAL innodb_file_format = `Barracuda`;
SET GLOBAL innodb_file_per_table = ON;
OPTIMIZE TABLE t1;
SHOW WARNINGS;
--remove_file $MYSQLTEST_VARDIR/keys1.txt
--remove_file $MYSQLTEST_VARDIR/keys2.txt
--exec echo "wait" > $MYSQLTEST_VARDIR/tmp/mysqld.1.expect
--shutdown_server
--source include/wait_until_disconnected.inc
--write_file $MYSQLTEST_VARDIR/keys1.txt
1;770A8A65DA156D24EE2A093277530142
4;770A8A65DA156D24EE2A093277530143
EOF
--exec echo "restart:--innodb-encrypt-tables --innodb-defragment=1 --innodb-stats-persistent --plugin-load-add=file_key_management.so --file-key-management --file-key-management-filename=$MYSQLTEST_VARDIR/keys1.txt" > $MYSQLTEST_VARDIR/tmp/mysqld.1.expect
--enable_reconnect
--source include/wait_until_connected_again.inc
DROP TABLE t1;
--remove_file $MYSQLTEST_VARDIR/keys1.txt
# reset system
--disable_query_log
EVAL SET GLOBAL innodb_file_per_table = $innodb_file_per_table_orig;
EVAL SET GLOBAL innodb_file_format = $innodb_file_format_orig;
--enable_query_log

68
mysql-test/suite/encryption/t/innodb-bad-key-shutdown.test
View File

@ -1,68 +0,0 @@
#
# MDEV-8727: Server/InnoDB hangs on shutdown after trying to read an encrypted table with a wrong key
#
call mtr.add_suppression("InnoDB: Block in space_id .* in file test/.* encrypted");
call mtr.add_suppression("InnoDB: However key management plugin or used key_version .* is not found or used encryption algorithm or method does not match.");
call mtr.add_suppression("InnoDB: Marking tablespace as missing. You may drop this table or install correct key management plugin and key file.");
call mtr.add_suppression(".*InnoDB: Cannot open table test/.* from the internal data dictionary of InnoDB though the .frm file for the table exists. See .* for how you can resolve the problem.");
call mtr.add_suppression("InnoDB: .ibd file is missing for table test/.*");
# Suppression for builds where file_key_management plugin is linked statically
call mtr.add_suppression("Couldn't load plugins from 'file_key_management*");
--echo #
--echo # Restart the server with key 4 in the key file
--echo #
--source include/have_innodb.inc
--source include/not_embedded.inc
--exec echo "wait" > $MYSQLTEST_VARDIR/tmp/mysqld.1.expect
--shutdown_server
--source include/wait_until_disconnected.inc
--write_file $MYSQLTEST_VARDIR/keys1.txt
1;770A8A65DA156D24EE2A093277530142
4;18420B5CBA31CCDFFE9716E91EB61374D05914F3ADE23E03
EOF
--exec echo "restart:--plugin-load-add=file_key_management.so --file-key-management --file-key-management-filename=$MYSQLTEST_VARDIR/keys1.txt" > $MYSQLTEST_VARDIR/tmp/mysqld.1.expect
--enable_reconnect
--source include/wait_until_connected_again.inc
CREATE TABLE t1 (i INT, KEY(i)) ENGINE=InnoDB ENCRYPTED=YES ENCRYPTION_KEY_ID=4;
INSERT INTO t1 VALUES (1);
--echo #
--echo # Restart the server with a different value for key 4 in the key file
--echo #
--exec echo "wait" > $MYSQLTEST_VARDIR/tmp/mysqld.1.expect
--shutdown_server
--source include/wait_until_disconnected.inc
--write_file $MYSQLTEST_VARDIR/keys2.txt
1;770A8A65DA156D24EE2A093277530142
4;22222222222222222222222222222222
EOF
--exec echo "restart:--plugin-load-add=file_key_management.so --file-key-management --file-key-management-filename=$MYSQLTEST_VARDIR/keys2.txt" > $MYSQLTEST_VARDIR/tmp/mysqld.1.expect
--enable_reconnect
--source include/wait_until_connected_again.inc
--error 1296
SELECT * FROM t1;
--exec echo "wait" > $MYSQLTEST_VARDIR/tmp/mysqld.1.expect
--shutdown_server
--source include/wait_until_disconnected.inc
--exec echo "restart:--plugin-load-add=file_key_management.so --file-key-management --file-key-management-filename=$MYSQLTEST_VARDIR/keys1.txt" > $MYSQLTEST_VARDIR/tmp/mysqld.1.expect
--enable_reconnect
--source include/wait_until_connected_again.inc
SELECT * FROM t1;
DROP TABLE t1;
--remove_file $MYSQLTEST_VARDIR/keys2.txt
--remove_file $MYSQLTEST_VARDIR/keys1.txt

4
mysql-test/suite/encryption/t/innodb-compressed-blob.opt Normal file
View File

@ -0,0 +1,4 @@
--innodb-tablespaces-encryption
--innodb-encrypt-tables=on
--innodb-encryption-threads=2
--max_allowed_packet=64K

45
mysql-test/suite/encryption/t/innodb-compressed-blob.test Normal file
View File

@ -0,0 +1,45 @@
-- source include/have_innodb.inc
-- source include/have_file_key_management_plugin.inc
# embedded does not support restart
-- source include/not_embedded.inc
call mtr.add_suppression("InnoDB: However key management plugin or used key_version .*");
call mtr.add_suppression("InnoDB: The page \[page id: space=[0-9]+, page number=[0-9]+\] in file test/t1 cannot be decrypted.");
call mtr.add_suppression("InnoDB: The page \[page id: space=[0-9]+, page number=[0-9]+\] in file test/t2 cannot be decrypted.");
call mtr.add_suppression("InnoDB: The page \[page id: space=[0-9]+, page number=[0-9]+\] in file test/t3 cannot be decrypted.");
--echo # Restart mysqld --file-key-management-filename=keys2.txt
-- let $restart_parameters=--file-key-management-filename=$MYSQL_TEST_DIR/std_data/keys2.txt
-- source include/restart_mysqld.inc
--disable_warnings
SET GLOBAL innodb_file_format = `Barracuda`;
SET GLOBAL innodb_file_per_table = ON;
--enable_warnings
set GLOBAL innodb_default_encryption_key_id=4;
create table t1(a int not null primary key, b blob, index(b(10))) engine=innodb row_format=compressed;
create table t2(a int not null primary key, b blob, index(b(10))) engine=innodb row_format=compressed encrypted=yes;
create table t3(a int not null primary key, b blob, index(b(10))) engine=innodb row_format=compressed encrypted=no;
insert into t1 values (1, repeat('secret',6000));
insert into t2 values (1, repeat('secret',6000));
insert into t3 values (1, repeat('secret',6000));
--echo # Restart mysqld --file-key-management-filename=keys3.txt
-- let $restart_parameters=--file-key-management-filename=$MYSQL_TEST_DIR/std_data/keys3.txt
-- source include/restart_mysqld.inc
--error 1296
select count(*) from t1 FORCE INDEX (b) where b like 'secret%';
--error 1296
select count(*) from t2 FORCE INDEX (b) where b like 'secret%';
select count(*) from t3 FORCE INDEX (b) where b like 'secret%';
--echo # Restart mysqld --file-key-management-filename=keys2.txt
-- let $restart_parameters=--file-key-management-filename=$MYSQL_TEST_DIR/std_data/keys2.txt
-- source include/restart_mysqld.inc
drop table t1,t2,t3;

60
mysql-test/suite/encryption/t/innodb-encryption-disable.test
View File

@ -1,43 +1,20 @@
-- source include/have_innodb.inc
# embedded does not support restart
-- source include/not_embedded.inc
-- source include/not_valgrind.inc
# Avoid CrashReporter popup on Mac
-- source include/not_crashrep.inc
-- source filekeys_plugin_exists.inc
--disable_query_log
let $innodb_file_format_orig = `SELECT @@innodb_file_format`;
let $innodb_file_per_table_orig = `SELECT @@innodb_file_per_table`;
--enable_query_log
SET GLOBAL innodb_file_format = `Barracuda`;
SET GLOBAL innodb_file_per_table = ON;
#
# MDEV-9559: Server without encryption configs crashes if selecting from an implicitly encrypted table
#
call mtr.add_suppression("InnoDB: Block in space_id .* in file test/.* encrypted");
call mtr.add_suppression("InnoDB: However key management plugin or used key_version .* is not found or used encryption algorithm or method does not match.");
call mtr.add_suppression("InnoDB: Marking tablespace as missing. You may drop this table or install correct key management plugin and key file.");
call mtr.add_suppression(".*InnoDB: Cannot open table test/.* from the internal data dictionary of InnoDB though the .frm file for the table exists. See .* for how you can resolve the problem.");
call mtr.add_suppression("InnoDB: .ibd file is missing for table test/.*");
call mtr.add_suppression("InnoDB: The page \[page id: space=[0-9]+, page number=[0-9]+\] in file test/t1 cannot be decrypted.");
call mtr.add_suppression("InnoDB: The page \[page id: space=[0-9]+, page number=[0-9]+\] in file test/t5 cannot be decrypted.");
# Suppression for builds where file_key_management plugin is linked statically
call mtr.add_suppression("Couldn't load plugins from 'file_key_management*");
call mtr.add_suppression("InnoDB: Tablespace id.* is encrypted but encryption service or used key_id .* is not available. Can't continue opening tablespace.");
--exec echo "wait" > $MYSQLTEST_VARDIR/tmp/mysqld.1.expect
--shutdown_server
--source include/wait_until_disconnected.inc
--write_file $MYSQLTEST_VARDIR/keys1.txt
1;770A8A65DA156D24EE2A093277530142
4;770A8A65DA156D24EE2A093277530143
EOF
--exec echo "restart:--innodb-encrypt-tables --plugin-load-add=file_key_management.so --file-key-management --file-key-management-filename=$MYSQLTEST_VARDIR/keys1.txt" > $MYSQLTEST_VARDIR/tmp/mysqld.1.expect
--enable_reconnect
--source include/wait_until_connected_again.inc
--let $restart_parameters=--innodb-encrypt-tables=ON --plugin-load-add=file_key_management.so --file-key-management --file-key-management-filename=$MYSQL_TEST_DIR/std_data/keys2.txt
--source include/restart_mysqld.inc
create table t5 (
`intcol1` int(32) DEFAULT NULL,
@ -60,33 +37,16 @@ CREATE TABLE `t1` (
insert into t1 values (1,2,'maria','db','encryption');
alter table t1 encrypted='yes' `encryption_key_id`=1;
--exec echo "wait" > $MYSQLTEST_VARDIR/tmp/mysqld.1.expect
--shutdown_server
--source include/wait_until_disconnected.inc
--exec echo "restart:--innodb-encrypt-tables=OFF" > $MYSQLTEST_VARDIR/tmp/mysqld.1.expect
--enable_reconnect
--source include/wait_until_connected_again.inc
--let $restart_parameters=--innodb-encrypt-tables=OFF
--source include/restart_mysqld.inc
--error 1296
select * from t1;
--error 1296
select * from t5;
--exec echo "wait" > $MYSQLTEST_VARDIR/tmp/mysqld.1.expect
--shutdown_server
--source include/wait_until_disconnected.inc
--exec echo "restart:--innodb-encrypt-tables --plugin-load-add=file_key_management.so --file-key-management --file-key-management-filename=$MYSQLTEST_VARDIR/keys1.txt" > $MYSQLTEST_VARDIR/tmp/mysqld.1.expect
--enable_reconnect
--source include/wait_until_connected_again.inc
--let $restart_parameters=--innodb-encrypt-tables=ON --plugin-load-add=file_key_management.so --file-key-management --file-key-management-filename=$MYSQL_TEST_DIR/std_data/keys2.txt
--source include/restart_mysqld.inc
drop table t1;
drop table t5;
--disable_query_log
EVAL SET GLOBAL innodb_file_per_table = $innodb_file_per_table_orig;
EVAL SET GLOBAL innodb_file_format = $innodb_file_format_orig;
--enable_query_log
--remove_file $MYSQLTEST_VARDIR/keys1.txt

92
mysql-test/suite/encryption/t/innodb-force-corrupt.test Normal file
View File

@ -0,0 +1,92 @@
#
# MDEV-11759: Encryption code in MariaDB 10.1/10.2 causes compatibility problems
#
-- source include/have_innodb.inc
-- source include/have_file_key_management_plugin.inc
# Don't test under embedded
-- source include/not_embedded.inc
CALL mtr.add_suppression("InnoDB: Database page corruption on disk or a failed .*");
CALL mtr.add_suppression("InnoDB: Corruption: Block in space_id .*");
CALL mtr.add_suppression("InnoDB: However key management plugin or used key_version .*");
call mtr.add_suppression("InnoDB: The page \[page id: space=[0-9]+, page number=[0-9]+\] in file test/t1 cannot be decrypted.");
call mtr.add_suppression("InnoDB: The page \[page id: space=[0-9]+, page number=[0-9]+\] in file test/t2 cannot be decrypted.");
call mtr.add_suppression("InnoDB: The page \[page id: space=[0-9]+, page number=[0-9]+\] in file test/t3 cannot be decrypted.");
--disable_warnings
SET GLOBAL innodb_file_format = `Barracuda`;
SET GLOBAL innodb_file_per_table = ON;
set global innodb_compression_algorithm = 1;
--enable_warnings
--echo # Create and populate tables to be corrupted
CREATE TABLE t1 (a INT AUTO_INCREMENT PRIMARY KEY, b TEXT,c char(200)) ENGINE=InnoDB encrypted=yes;
CREATE TABLE t2 (a INT AUTO_INCREMENT PRIMARY KEY, b TEXT,c char(200)) ENGINE=InnoDB row_format=compressed encrypted=yes;
CREATE TABLE t3 (a INT AUTO_INCREMENT PRIMARY KEY, b TEXT, c char(200)) ENGINE=InnoDB page_compressed=yes encrypted=yes;
BEGIN;
INSERT INTO t1 (b,c) VALUES ('corrupt me','secret');
--disable_query_log
--let $i = 100
while ($i)
{
INSERT INTO t1 (b,c) VALUES (REPEAT('abcabcabc', 100),'secretsecret');
dec $i;
}
--enable_query_log
INSERT INTO t1 (b,c) VALUES ('corrupt me','moresecretmoresecret');
INSERT INTO t2 select * from t1;
INSERT INTO t3 select * from t1;
COMMIT;
let INNODB_PAGE_SIZE=`select @@innodb_page_size`;
let MYSQLD_DATADIR=`select @@datadir`;
--source include/shutdown_mysqld.inc
--echo # Backup tables before corrupting
--copy_file $MYSQLD_DATADIR/test/t1.ibd $MYSQLD_DATADIR/test/t1.ibd.backup
--copy_file $MYSQLD_DATADIR/test/t2.ibd $MYSQLD_DATADIR/test/t2.ibd.backup
--copy_file $MYSQLD_DATADIR/test/t3.ibd $MYSQLD_DATADIR/test/t3.ibd.backup
--echo # Corrupt tables
perl;
open(FILE, "+<", "$ENV{MYSQLD_DATADIR}/test/t1.ibd") or die "open";
binmode FILE;
seek(FILE, $ENV{'INNODB_PAGE_SIZE'} * 3 + 26, SEEK_SET) or die "seek";
print FILE pack("H*", "c00lcafedeadb017");
close FILE or die "close";
open(FILE, "+<", "$ENV{MYSQLD_DATADIR}/test/t2.ibd") or die "open";
binmode FILE;
seek(FILE, $ENV{'INNODB_PAGE_SIZE'} * 3 + 26, SEEK_SET) or die "seek";
print FILE pack("H*", "c00lcafedeadb017");
close FILE or die "close";
open(FILE, "+<", "$ENV{MYSQLD_DATADIR}/test/t3.ibd") or die "open";
binmode FILE;
seek(FILE, $ENV{'INNODB_PAGE_SIZE'} * 3 + 26, SEEK_SET) or die "seek";
print FILE pack("H*", "c00lcafedeadb017");
close FILE or die "close";
EOF
--source include/start_mysqld.inc
--error ER_GET_ERRMSG
SELECT * FROM t1;
--error ER_GET_ERRMSG
SELECT * FROM t2;
--error ER_GET_ERRMSG
SELECT * FROM t3;
--source include/shutdown_mysqld.inc
--echo # Restore the original tables
--move_file $MYSQLD_DATADIR/test/t1.ibd.backup $MYSQLD_DATADIR/test/t1.ibd
--move_file $MYSQLD_DATADIR/test/t2.ibd.backup $MYSQLD_DATADIR/test/t2.ibd
--move_file $MYSQLD_DATADIR/test/t3.ibd.backup $MYSQLD_DATADIR/test/t3.ibd
--source include/start_mysqld.inc
DROP TABLE t1,t2,t3;

10
mysql-test/suite/encryption/t/innodb-missing-key.test
View File

@ -2,16 +2,14 @@
-- source include/have_file_key_management_plugin.inc
# embedded does not support restart
-- source include/not_embedded.inc
-- source include/not_valgrind.inc
# Avoid CrashReporter popup on Mac
-- source include/not_crashrep.inc
#
# MDEV-11004: Unable to start (Segfault or os error 2) when encryption key missing
#
call mtr.add_suppression("InnoDB: Block in space_id .* in file test/.* encrypted");
call mtr.add_suppression("InnoDB: However key management plugin or used key_version .* is not found or used encryption algorithm or method does not match.");
call mtr.add_suppression("InnoDB: Marking tablespace as missing. You may drop this table or install correct key management plugin and key file.");
call mtr.add_suppression("InnoDB: The page \[page id: space=[0-9]+, page number=[0-9]+\] in file test/t1 cannot be decrypted.");
call mtr.add_suppression("InnoDB: The page \[page id: space=[0-9]+, page number=[0-9]+\] in file test/t2 cannot be decrypted.");
call mtr.add_suppression("InnoDB: The page \[page id: space=[0-9]+, page number=[0-9]+\] in file test/t3 cannot be decrypted.");
--echo
--echo # Start server with keys2.txt

79
mysql-test/suite/encryption/t/innodb-page_encryption_compression.test
View File

@ -1,21 +1,19 @@
-- source include/have_innodb.inc
-- source include/not_embedded.inc
-- source include/have_file_key_management_plugin.inc
-- source include/big_test.inc
# Test heavy not tested on valgrind
-- source include/not_valgrind.inc
-- source include/not_embedded.inc
--disable_query_log
let $innodb_compression_algorithm_orig=`SELECT @@innodb_compression_algorithm`;
let $innodb_file_format_orig = `SELECT @@innodb_file_format`;
let $innodb_file_per_table_orig = `SELECT @@innodb_file_per_table`;
let $encryption = `SELECT @@innodb_encrypt_tables`;
--enable_query_log
--disable_warnings
SET GLOBAL innodb_file_format = `Barracuda`;
SET GLOBAL innodb_file_per_table = ON;
# zlib
set global innodb_compression_algorithm = 1;
--enable_warnings
create table innodb_normal(c1 bigint not null, b char(200)) engine=innodb page_compressed=1;
show warnings;
@ -41,52 +39,24 @@ end//
delimiter ;//
commit;
set autocommit=0;
call innodb_insert_proc(5000);
commit;
set autocommit=1;
begin;
call innodb_insert_proc(2000);
insert into innodb_compact select * from innodb_normal;
insert into innodb_dynamic select * from innodb_normal;
commit;
update innodb_normal set c1 = c1 + 1;
update innodb_compact set c1 = c1 + 1;
update innodb_dynamic set c1 = c1 + 1;
select count(*) from innodb_normal;
select count(*) from innodb_compact where c1 < 1500000;
select count(*) from innodb_dynamic where c1 < 1500000;
select count(*) from innodb_compact t1, innodb_normal t2 where
t1.c1 = t2.c1 and t1.b = t2.b;
select count(*) from innodb_dynamic t1, innodb_normal t2 where
t1.c1 = t2.c1 and t1.b = t2.b;
SELECT variable_value >= 0 FROM information_schema.global_status WHERE variable_name = 'innodb_num_pages_encrypted';
SELECT variable_value >= 0 FROM information_schema.global_status WHERE variable_name = 'innodb_num_pages_decrypted';
SELECT variable_value >= 0 FROM information_schema.global_status WHERE variable_name = 'innodb_num_pages_compressed';
SELECT variable_value >= 0 FROM information_schema.global_status WHERE variable_name = 'innodb_num_pages_decompressed';
let $wait_condition= select variable_value > 0 from information_schema.global_status where variable_name = 'INNODB_NUM_PAGES_PAGE_COMPRESSED';
--source include/wait_condition.inc
--let $restart_parameters=--innodb-encrypt-tables=OFF
--source include/restart_mysqld.inc
--disable_warnings
SET GLOBAL innodb_file_format = `Barracuda`;
SET GLOBAL innodb_file_per_table = ON;
# zlib
set global innodb_compression_algorithm = 1;
update innodb_normal set c1 = c1 + 1;
update innodb_compact set c1 = c1 + 1;
update innodb_dynamic set c1 = c1 + 1;
select count(*) from innodb_normal;
select count(*) from innodb_compact where c1 < 1500000;
select count(*) from innodb_dynamic where c1 < 1500000;
select count(*) from innodb_compact t1, innodb_normal t2 where
t1.c1 = t2.c1 and t1.b = t2.b;
select count(*) from innodb_dynamic t1, innodb_normal t2 where
t1.c1 = t2.c1 and t1.b = t2.b;
SELECT variable_value >= 0 FROM information_schema.global_status WHERE variable_name = 'innodb_num_pages_encrypted';
SELECT variable_value >= 0 FROM information_schema.global_status WHERE variable_name = 'innodb_num_pages_decrypted';
SELECT variable_value >= 0 FROM information_schema.global_status WHERE variable_name = 'innodb_num_pages_compressed';
SELECT variable_value >= 0 FROM information_schema.global_status WHERE variable_name = 'innodb_num_pages_decompressed';
--enable_warnings
alter table innodb_normal engine=innodb page_compressed=DEFAULT;
show create table innodb_normal;
@ -95,30 +65,8 @@ show create table innodb_compact;
alter table innodb_dynamic engine=innodb encrypted=DEFAULT encryption_key_id=DEFAULT page_compressed=DEFAULT;
show create table innodb_dynamic;
--source include/restart_mysqld.inc
SET GLOBAL innodb_file_format = `Barracuda`;
SET GLOBAL innodb_file_per_table = ON;
show create table innodb_normal;
show create table innodb_compact;
show create table innodb_dynamic;
update innodb_normal set c1 = c1 + 1;
update innodb_compact set c1 = c1 + 1;
update innodb_dynamic set c1 = c1 + 1;
select count(*) from innodb_normal;
select count(*) from innodb_compact where c1 < 1500000;
select count(*) from innodb_dynamic where c1 < 1500000;
select count(*) from innodb_compact t1, innodb_normal t2 where
t1.c1 = t2.c1 and t1.b = t2.b;
select count(*) from innodb_dynamic t1, innodb_normal t2 where
t1.c1 = t2.c1 and t1.b = t2.b;
SELECT variable_value = 0 FROM information_schema.global_status WHERE variable_name = 'innodb_num_pages_encrypted';
SELECT variable_value = 0 FROM information_schema.global_status WHERE variable_name = 'innodb_num_pages_decrypted';
SELECT variable_value = 0 FROM information_schema.global_status WHERE variable_name = 'innodb_num_pages_compressed';
SELECT variable_value = 0 FROM information_schema.global_status WHERE variable_name = 'innodb_num_pages_decompressed';
let $wait_condition= select variable_value > 0 from information_schema.global_status where variable_name = 'INNODB_NUM_PAGES_PAGE_DECOMPRESSED';
--source include/wait_condition.inc
drop procedure innodb_insert_proc;
drop table innodb_normal;
@ -130,5 +78,4 @@ drop table innodb_dynamic;
EVAL SET GLOBAL innodb_compression_algorithm = $innodb_compression_algorithm_orig;
EVAL SET GLOBAL innodb_file_per_table = $innodb_file_per_table_orig;
EVAL SET GLOBAL innodb_file_format = $innodb_file_format_orig;
EVAL SET GLOBAL innodb_encrypt_tables = $encryption;
--enable_query_log

101
mysql-test/suite/encryption/t/innodb-page_encryption_log_encryption.test
View File

@ -2,17 +2,14 @@
-- source include/not_embedded.inc
-- source include/have_file_key_management_plugin.inc
--disable_query_log
let $innodb_file_format_orig = `SELECT @@innodb_file_format`;
let $innodb_file_per_table_orig = `SELECT @@innodb_file_per_table`;
--enable_query_log
call mtr.add_suppression("KeyID 0 not found or with error. Check the key and the log*");
call mtr.add_suppression("Disabling redo log encryp*");
call mtr.add_suppression("InnoDB: Redo log crypto: Can't initialize to key version*");
#call mtr.add_suppression("KeyID 0 not found or with error. Check the key and the log.*");
#call mtr.add_suppression("Disabling redo log encryp.*");
#call mtr.add_suppression("InnoDB: Redo log crypto: Can't initialize to key version.*");
--disable_warnings
SET GLOBAL innodb_file_format = `Barracuda`;
SET GLOBAL innodb_file_per_table = ON;
--enable_warnings
create table innodb_normal(c1 bigint not null, b char(200)) engine=innodb;
show warnings;
@ -43,64 +40,41 @@ end//
delimiter ;//
commit;
set autocommit=0;
begin;
call innodb_insert_proc(2000);
commit;
set autocommit=1;
insert into innodb_compact select * from innodb_normal;
insert into innodb_compressed select * from innodb_normal;
insert into innodb_dynamic select * from innodb_normal;
insert into innodb_redundant select * from innodb_normal;
commit;
update innodb_normal set c1 = c1 +1;
update innodb_compact set c1 = c1 + 1;
update innodb_compressed set c1 = c1 + 1;
update innodb_dynamic set c1 = c1 + 1;
update innodb_redundant set c1 = c1 + 1;
select count(*) from innodb_compact where c1 < 1500000;
select count(*) from innodb_compressed where c1 < 1500000;
select count(*) from innodb_dynamic where c1 < 1500000;
select count(*) from innodb_redundant where c1 < 1500000;
select count(*) from innodb_compact t1, innodb_normal t2 where
t1.c1 = t2.c1 and t1.b = t2.b;
select count(*) from innodb_dynamic t1, innodb_normal t2 where
t1.c1 = t2.c1 and t1.b = t2.b;
select count(*) from innodb_compressed t1, innodb_normal t2 where
t1.c1 = t2.c1 and t1.b = t2.b;
select count(*) from innodb_redundant t1, innodb_normal t2 where
t1.c1 = t2.c1 and t1.b = t2.b;
let $wait_condition= select variable_value > 0 from information_schema.global_status where variable_name = 'INNODB_NUM_PAGES_ENCRYPTED';
--source include/wait_condition.inc
# Note there that these variables are updated only when real I/O is done, thus they are not reliable
SELECT variable_value >= 0 FROM information_schema.global_status WHERE variable_name = 'innodb_num_pages_encrypted';
SELECT variable_value > 0 FROM information_schema.global_status WHERE variable_name = 'innodb_num_pages_encrypted';
SELECT variable_value >= 0 FROM information_schema.global_status WHERE variable_name = 'innodb_num_pages_decrypted';
--source include/restart_mysqld.inc
--disable_warnings
SET GLOBAL innodb_file_format = `Barracuda`;
SET GLOBAL innodb_file_per_table = ON;
--enable_warnings
update innodb_normal set c1 = c1 +1;
update innodb_compact set c1 = c1 + 1;
update innodb_compressed set c1 = c1 + 1;
update innodb_dynamic set c1 = c1 + 1;
update innodb_redundant set c1 = c1 + 1;
select count(*) from innodb_compact where c1 < 1500000;
select count(*) from innodb_compressed where c1 < 1500000;
select count(*) from innodb_dynamic where c1 < 1500000;
select count(*) from innodb_redundant where c1 < 1500000;
select count(*) from innodb_compact t1, innodb_normal t2 where
t1.c1 = t2.c1 and t1.b = t2.b;
select count(*) from innodb_dynamic t1, innodb_normal t2 where
t1.c1 = t2.c1 and t1.b = t2.b;
select count(*) from innodb_compressed t1, innodb_normal t2 where
t1.c1 = t2.c1 and t1.b = t2.b;
select count(*) from innodb_redundant t1, innodb_normal t2 where
t1.c1 = t2.c1 and t1.b = t2.b;
SELECT variable_value >= 0 FROM information_schema.global_status WHERE variable_name = 'innodb_num_pages_encrypted';
SELECT variable_value >= 0 FROM information_schema.global_status WHERE variable_name = 'innodb_num_pages_decrypted';
let $wait_condition= select variable_value > 0 from information_schema.global_status where variable_name = 'INNODB_NUM_PAGES_ENCRYPTED';
--source include/wait_condition.inc
SELECT variable_value > 0 FROM information_schema.global_status WHERE variable_name = 'innodb_num_pages_encrypted';
SELECT variable_value > 0 FROM information_schema.global_status WHERE variable_name = 'innodb_num_pages_decrypted';
SET GLOBAL innodb_encrypt_tables=OFF;
alter table innodb_compact engine=innodb encrypted=DEFAULT encryption_key_id=DEFAULT;
show create table innodb_compact;
alter table innodb_compressed engine=innodb encrypted=DEFAULT encryption_key_id=DEFAULT;
@ -110,33 +84,8 @@ show create table innodb_dynamic;
alter table innodb_redundant engine=innodb encrypted=DEFAULT encryption_key_id=DEFAULT;
show create table innodb_redundant;
--source include/restart_mysqld.inc
SET GLOBAL innodb_file_format = `Barracuda`;
SET GLOBAL innodb_file_per_table = ON;
show create table innodb_compact;
show create table innodb_compressed;
show create table innodb_dynamic;
show create table innodb_redundant;
update innodb_normal set c1 = c1 +1;
update innodb_compact set c1 = c1 + 1;
update innodb_compressed set c1 = c1 + 1;
update innodb_dynamic set c1 = c1 + 1;
update innodb_redundant set c1 = c1 + 1;
select count(*) from innodb_compact where c1 < 1500000;
select count(*) from innodb_compressed where c1 < 1500000;
select count(*) from innodb_dynamic where c1 < 1500000;
select count(*) from innodb_redundant where c1 < 1500000;
select count(*) from innodb_compact t1, innodb_normal t2 where
t1.c1 = t2.c1 and t1.b = t2.b;
select count(*) from innodb_dynamic t1, innodb_normal t2 where
t1.c1 = t2.c1 and t1.b = t2.b;
select count(*) from innodb_compressed t1, innodb_normal t2 where
t1.c1 = t2.c1 and t1.b = t2.b;
select count(*) from innodb_redundant t1, innodb_normal t2 where
t1.c1 = t2.c1 and t1.b = t2.b;
let $wait_condition= select variable_value > 0 from information_schema.global_status where variable_name = 'INNODB_NUM_PAGES_DECRYPTED';
--source include/wait_condition.inc
SELECT variable_value >= 0 FROM information_schema.global_status WHERE variable_name = 'innodb_num_pages_encrypted';
SELECT variable_value >= 0 FROM information_schema.global_status WHERE variable_name = 'innodb_num_pages_decrypted';
@ -147,10 +96,12 @@ drop table innodb_compact;
drop table innodb_compressed;
drop table innodb_dynamic;
drop table innodb_redundant;
#
# MDEV-8143: InnoDB: Database page corruption on disk or a failed file read
#
SET GLOBAL innodb_file_format = `Barracuda`;
SET GLOBAL innodb_file_per_table = ON;
CREATE TABLE t1 (pk INT PRIMARY KEY) ENGINE=InnoDB ENCRYPTION_KEY_ID=2 ENCRYPTED=YES;
INSERT INTO t1 VALUES (1),(2);
@ -160,9 +111,3 @@ INSERT INTO t1 VALUES (1),(2);
SELECT * FROM t1;
DROP TABLE t1;
# reset system
--disable_query_log
EVAL SET GLOBAL innodb_file_per_table = $innodb_file_per_table_orig;
EVAL SET GLOBAL innodb_file_format = $innodb_file_format_orig;
--enable_query_log

5
mysql-test/suite/encryption/t/innodb-redo-badkey.opt Normal file
View File

@ -0,0 +1,5 @@
--innodb-change-buffering=all
--innodb-encrypt-tables=on
--innodb-tablespaces-encryption
--innodb-encryption-threads=2
--innodb-default-encryption-key-id=4

100
mysql-test/suite/encryption/t/innodb-redo-badkey.test Normal file
View File

@ -0,0 +1,100 @@
-- source include/have_innodb.inc
-- source include/have_file_key_management_plugin.inc
# embedded does not support restart
-- source include/not_embedded.inc
call mtr.add_suppression("InnoDB: Block in space_id .* in file .* encrypted.");
call mtr.add_suppression("Plugin 'file_key_management' .*");
call mtr.add_suppression("Plugin 'InnoDB' init function returned error.");
call mtr.add_suppression("Plugin 'InnoDB' registration as a STORAGE ENGINE failed.");
call mtr.add_suppression("InnoDB: Read operation failed for tablespace .*");
call mtr.add_suppression("InnoDB: Database page corruption on disk or a failed.*");
call mtr.add_suppression("InnoDB: Recovery read page .*");
call mtr.add_suppression("InnoDB: The page \[page id: space=[0-9]+, page number=[0-9]+\] in file test/t2 cannot be decrypted.");
call mtr.add_suppression("InnoDB: The page \[page id: space=[0-9]+, page number=[0-9]+\] in file test/t3 cannot be decrypted.");
call mtr.add_suppression("InnoDB: The page \[page id: space=[0-9]+, page number=[0-9]+\] in file test/t4 cannot be decrypted.");
call mtr.add_suppression("InnoDB: The page \[page id: space=[0-9]+, page number=[0-9]+\] in file test/t1 cannot be decrypted.");
call mtr.add_suppression("InnoDB: Missing MLOG_FILE_NAME or MLOG_FILE_DELETE .*");
call mtr.add_suppression("InnoDB: Plugin initialization aborted .*");
call mtr.add_suppression("InnoDB: ############### CORRUPT LOG RECORD FOUND ##################");
--echo # Restart mysqld --file-key-management-filename=keys2.txt
-- let $restart_parameters=--file-key-management-filename=$MYSQL_TEST_DIR/std_data/keys2.txt
-- source include/restart_mysqld.inc
--echo # Wait max 10 min for key encryption threads to encrypt all spaces
--let $wait_timeout= 600
--let $wait_condition=SELECT COUNT(*) = 0 FROM INFORMATION_SCHEMA.INNODB_TABLESPACES_ENCRYPTION WHERE MIN_KEY_VERSION = 0
--source include/wait_condition.inc
--disable_warnings
SET GLOBAL innodb_file_format = `Barracuda`;
SET GLOBAL innodb_file_per_table = ON;
--enable_warnings
create table t1(a int not null primary key auto_increment, c char(250), b blob, index(b(10))) engine=innodb row_format=compressed encrypted=yes encryption_key_id=4;
create table t2(a int not null primary key auto_increment, c char(250), b blob, index(b(10))) engine=innodb row_format=compressed;
create table t3(a int not null primary key auto_increment, c char(250), b blob, index(b(10))) engine=innodb encrypted=yes encryption_key_id=4;
create table t4(a int not null primary key auto_increment, c char(250), b blob, index(b(10))) engine=innodb;
begin;
--disable_query_log
--let $i = 20
begin;
while ($i)
{
insert into t1(c,b) values (repeat('secret1',20), repeat('secret2',6000));
dec $i;
}
--enable_query_log
insert into t2 select * from t1;
insert into t3 select * from t1;
insert into t4 select * from t1;
commit;
--source ../../suite/innodb/include/no_checkpoint_start.inc
#
# We test redo log page read at recv_read_page using
# incorrect keys from std_data/keys.txt. If checkpoint
# happens we will skip this test. If no checkpoint
# happens, InnoDB refuses to start as used
# encryption key is incorrect.
#
SET GLOBAL innodb_flush_log_at_trx_commit=1;
begin;
update t1 set c = repeat('secret3', 20);
update t2 set c = repeat('secret4', 20);
update t3 set c = repeat('secret4', 20);
update t4 set c = repeat('secret4', 20);
insert into t1 (c,b) values (repeat('secret5',20), repeat('secret6',6000));
insert into t2 (c,b) values (repeat('secret7',20), repeat('secret8',6000));
insert into t3 (c,b) values (repeat('secret9',20), repeat('secre10',6000));
insert into t4 (c,b) values (repeat('secre11',20), repeat('secre12',6000));
COMMIT;
let $cleanup= drop table t1,t2,t3,t4;
--let CLEANUP_IF_CHECKPOINT= $cleanup;
--source ../../suite/innodb/include/no_checkpoint_end.inc
--echo # restart
--error 1
-- source include/start_mysqld.inc
--source include/kill_mysqld.inc
#
# Now test with innodb-force-recovery=1 i.e. ignore corrupt pages
#
--echo # Restart mysqld --innodb-force-recovery=1
-- let $restart_parameters=--innodb-force-recovery=1
--error 1
-- source include/start_mysqld.inc
--source include/kill_mysqld.inc
--echo # Restart mysqld --file-key-management-filename=keys2.txt
-- let $restart_parameters=--file-key-management-filename=$MYSQL_TEST_DIR/std_data/keys2.txt
-- source include/start_mysqld.inc
drop table t1, t2,t3,t4;

3
mysql-test/suite/encryption/t/innodb-redo-nokeys.opt Normal file
View File

@ -0,0 +1,3 @@
--innodb-change-buffering=none
--innodb-encrypt-tables=on
--innodb-default-encryption-key-id=20

80
mysql-test/suite/encryption/t/innodb-redo-nokeys.test Normal file
View File

@ -0,0 +1,80 @@
-- source include/have_innodb.inc
-- source include/have_file_key_management_plugin.inc
# embedded does not support restart
-- source include/not_embedded.inc
call mtr.add_suppression("InnoDB: Block in space_id .*");
call mtr.add_suppression("mysqld: File .*");
call mtr.add_suppression("Plugin 'file_key_management' .*");
call mtr.add_suppression("InnoDB: cannot enable encryption, encryption plugin is not available");
call mtr.add_suppression("Plugin 'InnoDB' init function returned error.");
call mtr.add_suppression("Plugin 'InnoDB' registration as a STORAGE ENGINE failed.");
call mtr.add_suppression("InnoDB: The page \[page id: space=[0-9]+, page number=[0-9]+\] in file test/t2 cannot be decrypted.");
call mtr.add_suppression("InnoDB: The page \[page id: space=[0-9]+, page number=[0-9]+\] in file test/t3 cannot be decrypted.");
call mtr.add_suppression("InnoDB: The page \[page id: space=[0-9]+, page number=[0-9]+\] in file test/t4 cannot be decrypted.");
call mtr.add_suppression("InnoDB: The page \[page id: space=[0-9]+, page number=[0-9]+\] in file test/t1 cannot be decrypted.");
--echo # Restart mysqld --file-key-management-filename=keys2.txt
-- let $restart_parameters=--file-key-management-filename=$MYSQL_TEST_DIR/std_data/keys2.txt
-- source include/restart_mysqld.inc
--disable_warnings
SET GLOBAL innodb_file_format = `Barracuda`;
SET GLOBAL innodb_file_per_table = ON;
--enable_warnings
create table t1(a int not null primary key auto_increment, c char(200), b blob, index(b(10))) engine=innodb row_format=compressed encrypted=yes encryption_key_id=20;
create table t2(a int not null primary key auto_increment, c char(200), b blob, index(b(10))) engine=innodb row_format=compressed;
create table t3(a int not null primary key auto_increment, c char(200), b blob, index(b(10))) engine=innodb encrypted=yes encryption_key_id=20;
create table t4(a int not null primary key auto_increment, c char(200), b blob, index(b(10))) engine=innodb;
begin;
--disable_query_log
--let $i = 20
begin;
while ($i)
{
insert into t1(c,b) values (repeat('secret1',20), repeat('secret2',6000));
dec $i;
}
--enable_query_log
insert into t2 select * from t1;
insert into t3 select * from t1;
insert into t4 select * from t1;
commit;
--source ../../suite/innodb/include/no_checkpoint_start.inc
#
# We test redo log page read at recv_read_page using
# keys that are not in std_data/keys.txt. If checkpoint
# happens we will skip this test. If no checkpoint
# happens, InnoDB refuses to start as used
# encryption key is not found.
#
SET GLOBAL innodb_flush_log_at_trx_commit=1;
begin;
update t1 set c = repeat('secret3', 20);
update t2 set c = repeat('secret4', 20);
update t3 set c = repeat('secret4', 20);
update t4 set c = repeat('secret4', 20);
insert into t1 (c,b) values (repeat('secret5',20), repeat('secret6',6000));
insert into t2 (c,b) values (repeat('secret7',20), repeat('secret8',6000));
insert into t3 (c,b) values (repeat('secret9',20), repeat('secre10',6000));
insert into t4 (c,b) values (repeat('secre11',20), repeat('secre12',6000));
COMMIT;
let $cleanup= drop table t1,t2,t3,t4;
--let CLEANUP_IF_CHECKPOINT= $cleanup;
--source ../../suite/innodb/include/no_checkpoint_end.inc
--echo # restart
-- source include/start_mysqld.inc
#
# In above server does start but InnoDB refuses to start
# thus we need to restart server with correct key file
#
--echo # Restart mysqld --file-key-management-filename=keys2.txt
-- let $restart_parameters=--file-key-management-filename=$MYSQL_TEST_DIR/std_data/keys2.txt
-- source include/restart_mysqld.inc
drop table t1, t2,t3,t4;

61
mysql-test/suite/encryption/t/innodb_encryption-page-compression.test
View File

@ -1,9 +1,6 @@
-- source include/have_innodb.inc
-- source include/have_example_key_management_plugin.inc
# embedded does not support restart
-- source include/not_embedded.inc
--disable_query_log
let $innodb_compression_algorithm_orig=`SELECT @@innodb_compression_algorithm`;
let $innodb_file_format_orig = `SELECT @@innodb_file_format`;
@ -12,10 +9,12 @@ let $innodb_encrypt_tables_orig = `SELECT @@innodb_encrypt_tables`;
let $innodb_encryption_threads_orig = `SELECT @@innodb_encryption_threads`;
--enable_query_log
--disable_warnings
SET GLOBAL innodb_file_format = `Barracuda`;
SET GLOBAL innodb_file_per_table = ON;
SET GLOBAL innodb_encryption_threads = 4;
SET GLOBAL innodb_encrypt_tables = on;
--enable_warnings
# zlib
set global innodb_compression_algorithm = 1;
@ -62,11 +61,8 @@ end//
delimiter ;//
commit;
set autocommit=0;
call innodb_insert_proc(5000);
commit;
set autocommit=1;
select count(*) from innodb_normal;
begin;
call innodb_insert_proc(2000);
insert into innodb_page_compressed1 select * from innodb_normal;
insert into innodb_page_compressed2 select * from innodb_normal;
insert into innodb_page_compressed3 select * from innodb_normal;
@ -77,23 +73,6 @@ insert into innodb_page_compressed7 select * from innodb_normal;
insert into innodb_page_compressed8 select * from innodb_normal;
insert into innodb_page_compressed9 select * from innodb_normal;
commit;
select count(*) from innodb_page_compressed1 where c1 < 500000;
select count(*) from innodb_page_compressed2 where c1 < 500000;
select count(*) from innodb_page_compressed3 where c1 < 500000;
select count(*) from innodb_page_compressed4 where c1 < 500000;
select count(*) from innodb_page_compressed5 where c1 < 500000;
select count(*) from innodb_page_compressed6 where c1 < 500000;
select count(*) from innodb_page_compressed7 where c1 < 500000;
select count(*) from innodb_page_compressed8 where c1 < 500000;
select count(*) from innodb_page_compressed9 where c1 < 500000;
SELECT variable_value >= 0 FROM information_schema.global_status WHERE variable_name = 'innodb_num_pages_encrypted';
SELECT variable_value >= 0 FROM information_schema.global_status WHERE variable_name = 'innodb_num_pages_decrypted';
SELECT variable_value >= 0 FROM information_schema.global_status WHERE variable_name = 'innodb_num_pages_page_compressed';
SELECT variable_value >= 0 FROM information_schema.global_status WHERE variable_name = 'innodb_num_pages_page_decompressed';
--source include/restart_mysqld.inc
select count(*) from innodb_page_compressed1 where c1 < 500000;
select count(*) from innodb_page_compressed2 where c1 < 500000;
@ -104,33 +83,18 @@ select count(*) from innodb_page_compressed6 where c1 < 500000;
select count(*) from innodb_page_compressed7 where c1 < 500000;
select count(*) from innodb_page_compressed8 where c1 < 500000;
select count(*) from innodb_page_compressed9 where c1 < 500000;
update innodb_page_compressed1 set c1 = c1 + 1;
update innodb_page_compressed2 set c1 = c1 + 1;
update innodb_page_compressed3 set c1 = c1 + 1;
update innodb_page_compressed4 set c1 = c1 + 1;
update innodb_page_compressed5 set c1 = c1 + 1;
update innodb_page_compressed6 set c1 = c1 + 1;
update innodb_page_compressed7 set c1 = c1 + 1;
update innodb_page_compressed8 set c1 = c1 + 1;
update innodb_page_compressed9 set c1 = c1 + 1;
SELECT variable_value >= 0 FROM information_schema.global_status WHERE variable_name = 'innodb_num_pages_encrypted';
let $wait_condition= select variable_value > 0 from information_schema.global_status where variable_name = 'INNODB_NUM_PAGES_PAGE_COMPRESSED';
--source include/wait_condition.inc
SELECT variable_value > 0 FROM information_schema.global_status WHERE variable_name = 'innodb_num_pages_encrypted';
SELECT variable_value >= 0 FROM information_schema.global_status WHERE variable_name = 'innodb_num_pages_decrypted';
SELECT variable_value >= 0 FROM information_schema.global_status WHERE variable_name = 'innodb_num_pages_page_compressed';
SELECT variable_value > 0 FROM information_schema.global_status WHERE variable_name = 'innodb_num_pages_page_compressed';
SELECT variable_value >= 0 FROM information_schema.global_status WHERE variable_name = 'innodb_num_pages_page_decompressed';
SET GLOBAL innodb_encryption_threads = 4;
SET GLOBAL innodb_encrypt_tables = off;
select count(*) from innodb_page_compressed1 where c1 < 500000;
select count(*) from innodb_page_compressed2 where c1 < 500000;
select count(*) from innodb_page_compressed3 where c1 < 500000;
select count(*) from innodb_page_compressed4 where c1 < 500000;
select count(*) from innodb_page_compressed5 where c1 < 500000;
select count(*) from innodb_page_compressed6 where c1 < 500000;
select count(*) from innodb_page_compressed7 where c1 < 500000;
select count(*) from innodb_page_compressed8 where c1 < 500000;
select count(*) from innodb_page_compressed9 where c1 < 500000;
update innodb_page_compressed1 set c1 = c1 + 1;
update innodb_page_compressed2 set c1 = c1 + 1;
update innodb_page_compressed3 set c1 = c1 + 1;
@ -141,9 +105,12 @@ update innodb_page_compressed7 set c1 = c1 + 1;
update innodb_page_compressed8 set c1 = c1 + 1;
update innodb_page_compressed9 set c1 = c1 + 1;
let $wait_condition= select variable_value > 0 from information_schema.global_status where variable_name = 'INNODB_NUM_PAGES_DECRYPTED';
--source include/wait_condition.inc
SELECT variable_value >= 0 FROM information_schema.global_status WHERE variable_name = 'innodb_num_pages_encrypted';
SELECT variable_value >= 0 FROM information_schema.global_status WHERE variable_name = 'innodb_num_pages_decrypted';
SELECT variable_value >= 0 FROM information_schema.global_status WHERE variable_name = 'innodb_num_pages_page_compressed';
SELECT variable_value > 0 FROM information_schema.global_status WHERE variable_name = 'innodb_num_pages_decrypted';
SELECT variable_value > 0 FROM information_schema.global_status WHERE variable_name = 'innodb_num_pages_page_compressed';
SELECT variable_value >= 0 FROM information_schema.global_status WHERE variable_name = 'innodb_num_pages_page_decompressed';
drop procedure innodb_insert_proc;

3
mysql-test/suite/encryption/t/innodb_onlinealter_encryption.opt
View File

@ -3,6 +3,3 @@
--innodb-encryption-rotate-key-age=15
--innodb-encryption-threads=4
--innodb-tablespaces-encryption
--innodb-max-dirty-pages-pct=0.001

10
mysql-test/suite/encryption/t/innodb_onlinealter_encryption.test
View File

@ -22,8 +22,8 @@ begin
declare current_num int;
set current_num = 0;
while current_num < repeat_count do
insert into t1 values (current_num,repeat('foobar',42));
insert into t2 values (current_num,repeat('temp', 42));
insert into t1 values (current_num,repeat('foobar',12));
insert into t2 values (current_num,repeat('tempsecret', 12));
insert into t3 values (current_num,repeat('barfoo',42));
insert into t4 values (current_num,repeat('repeat',42));
insert into t5 values (current_num,substring('A BC DEF GHIJ KLM NOPQRS TUV WXYZ 012 3456789', rand()*36+1, 100), repeat('author new',22));
@ -47,14 +47,13 @@ set autocommit=1;
--let $MYSQLD_DATADIR=`select @@datadir`
--source include/shutdown_mysqld.inc
--source include/wait_until_disconnected.inc
--let SEARCH_RANGE = 10000000
--let SEARCH_PATTERN=foobar
--echo # t1 yes on expecting NOT FOUND
-- let SEARCH_FILE=$MYSQLD_DATADIR/test/t1.ibd
-- source include/search_pattern_in_file.inc
--let SEARCH_PATTERN=temp
--let SEARCH_PATTERN=tempsecret
--echo # t2 ... on expecting NOT FOUND
-- let SEARCH_FILE=$MYSQLD_DATADIR/test/t2.ibd
-- source include/search_pattern_in_file.inc
@ -108,13 +107,12 @@ SHOW CREATE TABLE t6;
SHOW CREATE TABLE t7;
--source include/shutdown_mysqld.inc
--source include/wait_until_disconnected.inc
--let SEARCH_PATTERN=foobar
--echo # t1 yes on expecting NOT FOUND
-- let SEARCH_FILE=$MYSQLD_DATADIR/test/t1.ibd
-- source include/search_pattern_in_file.inc
--let SEARCH_PATTERN=temp
--let SEARCH_PATTERN=tempsecret
--echo # t2 ... on expecting NOT FOUND
-- let SEARCH_FILE=$MYSQLD_DATADIR/test/t2.ibd
-- source include/search_pattern_in_file.inc

36
mysql-test/suite/innodb/r/innodb_bug14147491.result
View File

@ -1,18 +1,8 @@
call mtr.add_suppression("InnoDB: Database page corruption on disk or a failed.*");
call mtr.add_suppression("InnoDB: Corruption: Block in space_id .* in file .* corrupted.");
call mtr.add_suppression("InnoDB: Based on page type .*");
CALL mtr.add_suppression("InnoDB: Error: Unable to read tablespace .* page no .* into the buffer pool after 100 attempts");
CALL mtr.add_suppression("InnoDB: Database page corruption on disk or a failed");
CALL mtr.add_suppression("InnoDB: Space .* file test/t1 read of page .*");
CALL mtr.add_suppression("InnoDB: You may have to recover from a backup.");
CALL mtr.add_suppression("InnoDB: It is also possible that your operatingsystem has corrupted its own file cache.");
CALL mtr.add_suppression("InnoDB: and rebooting your computer removes the error.");
CALL mtr.add_suppression("InnoDB: If the corrupt page is an index page you can also try to");
CALL mtr.add_suppression("InnoDB: fix the corruption by dumping, dropping, and reimporting");
CALL mtr.add_suppression("InnoDB: the corrupt table. You can use CHECK");
CALL mtr.add_suppression("InnoDB: TABLE to scan your table for corruption.");
CALL mtr.add_suppression("InnoDB: See also .* about forcing recovery.");
call mtr.add_suppression("InnoDB: Table \"test\".\"t1\" is corrupted. Please drop the table and recreate.");
call mtr.add_suppression("InnoDB: Cannot open table test/t1 from the internal data dictionary of InnoDB though the .frm file for the table exists. See http://dev.mysql.com/doc/refman/5.6/en/innodb-troubleshooting.html for how you can resolve the problem.");
call mtr.add_suppression("InnoDB: Database page corruption on disk or a failed file read of tablespace test/t1 page \[page id: space=[0-9]+, page number=[0-9]+\]. You may have to recover from a backup.");
# Create and populate the table to be corrupted
set global innodb_file_per_table=ON;
CREATE TABLE t1 (a INT AUTO_INCREMENT PRIMARY KEY, b TEXT) ENGINE=InnoDB;
INSERT INTO t1 (b) VALUES ('corrupt me');
INSERT INTO t1 (b) VALUES ('corrupt me');
@ -20,13 +10,17 @@ INSERT INTO t1 (b) VALUES ('corrupt me');
# Corrupt the table
Munged a string.
Munged a string.
SET DEBUG_DBUG = '+d,innodb_page_corruption_retries';
# Write file to make mysql-test-run.pl expect the "crash", but don't
# start it until it's told to
# The below SELECT query will crash the server because some pages
# on the disk are corrupted
# The below SELECT query will return that table is not
# in engine because table is corrupted
SELECT * FROM t1;
ERROR HY000: Lost connection to MySQL server during query
# Restore the original t1.ibd
ERROR 42S02: Table 'test.t1' doesn't exist in engine
INSERT INTO t1(b) VALUES('abcdef');
ERROR 42S02: Table 'test.t1' doesn't exist in engine
UPDATE t1 set b = 'deadbeef' where a = 1;
ERROR 42S02: Table 'test.t1' doesn't exist in engine
CHECK TABLE t1;
Table Op Msg_type Msg_text
test.t1 check Error Table 'test.t1' doesn't exist in engine
test.t1 check status Operation failed
# Cleanup
DROP TABLE t1;

65
mysql-test/suite/innodb/t/innodb_bug14147491.test
View File

@ -4,37 +4,21 @@
-- source include/not_encrypted.inc
call mtr.add_suppression("InnoDB: Database page corruption on disk or a failed.*");
call mtr.add_suppression("InnoDB: Corruption: Block in space_id .* in file .* corrupted.");
call mtr.add_suppression("InnoDB: Based on page type .*");
call mtr.add_suppression("InnoDB: Table \"test\".\"t1\" is corrupted. Please drop the table and recreate.");
call mtr.add_suppression("InnoDB: Cannot open table test/t1 from the internal data dictionary of InnoDB though the .frm file for the table exists. See http://dev.mysql.com/doc/refman/5.6/en/innodb-troubleshooting.html for how you can resolve the problem.");
call mtr.add_suppression("InnoDB: Database page corruption on disk or a failed file read of tablespace test/t1 page \[page id: space=[0-9]+, page number=[0-9]+\]. You may have to recover from a backup.");
# Don't test under valgrind, memory leaks will occur
source include/not_valgrind.inc;
# Avoid CrashReporter popup on Mac
source include/not_crashrep.inc;
# Don't test under embedded
source include/not_embedded.inc;
# Require InnoDB
source include/have_innodb.inc;
# Require Debug for SET DEBUG
source include/have_debug.inc;
# Test could open crash reporter on Windows
# if compiler set up
source include/not_windows.inc;
CALL mtr.add_suppression("InnoDB: Error: Unable to read tablespace .* page no .* into the buffer pool after 100 attempts");
CALL mtr.add_suppression("InnoDB: Database page corruption on disk or a failed");
CALL mtr.add_suppression("InnoDB: Space .* file test/t1 read of page .*");
CALL mtr.add_suppression("InnoDB: You may have to recover from a backup.");
CALL mtr.add_suppression("InnoDB: It is also possible that your operatingsystem has corrupted its own file cache.");
CALL mtr.add_suppression("InnoDB: and rebooting your computer removes the error.");
CALL mtr.add_suppression("InnoDB: If the corrupt page is an index page you can also try to");
CALL mtr.add_suppression("InnoDB: fix the corruption by dumping, dropping, and reimporting");
CALL mtr.add_suppression("InnoDB: the corrupt table. You can use CHECK");
CALL mtr.add_suppression("InnoDB: TABLE to scan your table for corruption.");
CALL mtr.add_suppression("InnoDB: See also .* about forcing recovery.");
--echo # Create and populate the table to be corrupted
--disable_warnings
set global innodb_file_per_table=ON;
--enable_warnings
CREATE TABLE t1 (a INT AUTO_INCREMENT PRIMARY KEY, b TEXT) ENGINE=InnoDB;
INSERT INTO t1 (b) VALUES ('corrupt me');
--disable_query_log
@ -49,11 +33,10 @@ INSERT INTO t1 (b) VALUES ('corrupt me');
let $MYSQLD_DATADIR=`select @@datadir`;
let t1_IBD = $MYSQLD_DATADIR/test/t1.ibd;
--exec echo "wait" > $MYSQLTEST_VARDIR/tmp/mysqld.1.expect
let INNODB_PAGE_SIZE=`select @@innodb_page_size`;
let MYSQLD_DATADIR=`select @@datadir`;
--source include/shutdown_mysqld.inc
--source include/wait_until_disconnected.inc
--echo # Backup the t1.ibd before corrupting
--copy_file $t1_IBD $MYSQLD_DATADIR/test/t1.ibd.backup
@ -87,30 +70,18 @@ EOF
--source include/start_mysqld.inc
SET DEBUG_DBUG = '+d,innodb_page_corruption_retries';
--echo # Write file to make mysql-test-run.pl expect the "crash", but don't
--echo # start it until it's told to
--exec echo "wait" > $MYSQLTEST_VARDIR/tmp/mysqld.1.expect
--echo # The below SELECT query will crash the server because some pages
--echo # on the disk are corrupted
--error 2013
--echo # The below SELECT query will return that table is not
--echo # in engine because table is corrupted
--error 1932
SELECT * FROM t1;
# The below mtr command --remove_file fails randomly on windows with
# error number 13 which is permission denied on nix systems. We sleep
# 1 second hoping that any process holding lock on t1.ibd is released.
SLEEP 1;
--error 1932
INSERT INTO t1(b) VALUES('abcdef');
--echo # Restore the original t1.ibd
--remove_file $MYSQLD_DATADIR/test/t1.ibd
--move_file $MYSQLD_DATADIR/test/t1.ibd.backup $MYSQLD_DATADIR/test/t1.ibd
--error 1932
UPDATE t1 set b = 'deadbeef' where a = 1;
--source include/start_mysqld.inc
# Note SET DEBUG = '-d,innodb_page_corruption_retries' is not required
# because the session information is lost after server restart
CHECK TABLE t1;
--echo # Cleanup
DROP TABLE t1;

6
storage/innobase/api/api0api.cc
View File

@ -247,7 +247,7 @@ ib_open_table_by_id(
table = dict_table_open_on_id(table_id, TRUE, DICT_TABLE_OP_NORMAL);
if (table != NULL && table->ibd_file_missing) {
if (table != NULL && table->file_unreadable) {
table = NULL;
}
@ -272,7 +272,7 @@ ib_open_table_by_name(
table = dict_table_open_on_name(name, FALSE, FALSE,
DICT_ERR_IGNORE_NONE);
if (table != NULL && table->ibd_file_missing) {
if (table != NULL && table->file_unreadable) {
table = NULL;
}
@ -292,7 +292,7 @@ ib_lookup_table_by_name(
table = dict_table_get_low(name);
if (table != NULL && table->ibd_file_missing) {
if (table != NULL && table->file_unreadable) {
table = NULL;
}

8
storage/innobase/btr/btr0btr.cc
View File

@ -2,7 +2,7 @@
Copyright (c) 1994, 2016, Oracle and/or its affiliates. All Rights Reserved.
Copyright (c) 2012, Facebook Inc.
Copyright (c) 2014, 2016, MariaDB Corporation
Copyright (c) 2014, 2017, MariaDB Corporation
This program is free software; you can redistribute it and/or modify it under
the terms of the GNU General Public License as published by the Free Software
@ -738,8 +738,7 @@ btr_root_block_get(
if (!block) {
if (index && index->table) {
index->table->is_encrypted = TRUE;
index->table->corrupted = FALSE;
index->table->file_unreadable = true;
ib_push_warning(index->table->thd, DB_DECRYPTION_FAILED,
"Table %s in tablespace %lu is encrypted but encryption service or"
@ -752,6 +751,7 @@ btr_root_block_get(
}
btr_assert_not_corrupted(block, index);
#ifdef UNIV_BTR_DEBUG
if (!dict_index_is_ibuf(index)) {
const page_t* root = buf_block_get_frame(block);
@ -5175,7 +5175,7 @@ btr_validate_index(
page_t* root = btr_root_get(index, &mtr);
if (root == NULL && index->table->is_encrypted) {
if (root == NULL && index->table->file_unreadable) {
err = DB_DECRYPTION_FAILED;
mtr_commit(&mtr);
return err;

35
storage/innobase/btr/btr0cur.cc
View File

@ -3,7 +3,7 @@
Copyright (c) 1994, 2016, Oracle and/or its affiliates. All Rights Reserved.
Copyright (c) 2008, Google Inc.
Copyright (c) 2012, Facebook Inc.
Copyright (c) 2015, MariaDB Corporation.
Copyright (c) 2015, 2017, MariaDB Corporation.
Portions of this file contain modifications contributed and copyrighted by
Google, Inc. Those modifications are gratefully acknowledged and are described
@ -613,7 +613,11 @@ retry_page_get:
space, zip_size, page_no, rw_latch, guess, buf_mode,
file, line, mtr, &err);
/* Note that block==NULL signifies either an error or change
buffering. */
if (err != DB_SUCCESS) {
ut_ad(block == NULL);
if (err == DB_DECRYPTION_FAILED) {
ib_push_warning((void *)NULL,
DB_DECRYPTION_FAILED,
@ -621,7 +625,7 @@ retry_page_get:
" used key_id is not available. "
" Can't continue reading table.",
index->table->name);
index->table->is_encrypted = true;
index->table->file_unreadable = true;
}
goto func_exit;
@ -872,7 +876,7 @@ btr_cur_open_at_index_side_func(
ulint offsets_[REC_OFFS_NORMAL_SIZE];
ulint* offsets = offsets_;
dberr_t err = DB_SUCCESS;
rec_offs_init(offsets_);
estimate = latch_mode & BTR_ESTIMATE;
@ -916,6 +920,9 @@ btr_cur_open_at_index_side_func(
block = buf_page_get_gen(space, zip_size, page_no,
RW_NO_LATCH, NULL, BUF_GET,
file, line, mtr, &err);
ut_ad((block != NULL) == (err == DB_SUCCESS));
if (err != DB_SUCCESS) {
if (err == DB_DECRYPTION_FAILED) {
ib_push_warning((void *)NULL,
@ -924,7 +931,7 @@ btr_cur_open_at_index_side_func(
" used key_id is not available. "
" Can't continue reading table.",
index->table->name);
index->table->is_encrypted = true;
index->table->file_unreadable = true;
}
goto exit_loop;
@ -1073,6 +1080,8 @@ btr_cur_open_at_rnd_pos_func(
RW_NO_LATCH, NULL, BUF_GET,
file, line, mtr, &err);
ut_ad((block != NULL) == (err == DB_SUCCESS));
if (err != DB_SUCCESS) {
if (err == DB_DECRYPTION_FAILED) {
ib_push_warning((void *)NULL,
@ -1081,8 +1090,9 @@ btr_cur_open_at_rnd_pos_func(
" used key_id is not available. "
" Can't continue reading table.",
index->table->name);
index->table->is_encrypted = true;
index->table->file_unreadable = true;
}
goto exit_loop;
}
@ -3667,6 +3677,8 @@ btr_estimate_n_rows_in_range_on_level(
NULL, BUF_GET_POSSIBLY_FREED,
__FILE__, __LINE__, &mtr, &err);
ut_ad((block != NULL) == (err == DB_SUCCESS));
if (err != DB_SUCCESS) {
if (err == DB_DECRYPTION_FAILED) {
ib_push_warning((void *)NULL,
@ -3675,14 +3687,13 @@ btr_estimate_n_rows_in_range_on_level(
" used key_id is not available. "
" Can't continue reading table.",
index->table->name);
index->table->is_encrypted = true;
index->table->file_unreadable = true;
}
mtr_commit(&mtr);
goto inexact;
}
page = buf_block_get_frame(block);
/* It is possible that the tree has been reorganized in the
@ -3801,6 +3812,10 @@ btr_estimate_n_rows_in_range(
mtr_commit(&mtr);
if (index->table->file_unreadable) {
return (0);
}
mtr_start_trx(&mtr, trx);
cursor.path_arr = path2;
@ -4094,6 +4109,11 @@ btr_estimate_number_of_different_key_vals(
because otherwise our algorithm would give a wrong estimate
for an index where there is just one key value. */
if (index->table->file_unreadable) {
mtr_commit(&mtr);
goto exit_loop;
}
page = btr_cur_get_page(&cursor);
rec = page_rec_get_next(page_get_infimum_rec(page));
@ -4180,6 +4200,7 @@ btr_estimate_number_of_different_key_vals(
mtr_commit(&mtr);
}
exit_loop:
/* If we saw k borders between different key values on
n_sample_pages leaf pages, we can estimate how many
there will be in index->stat_n_leaf_pages */

2
storage/innobase/btr/btr0defragment.cc
View File

@ -227,7 +227,7 @@ btr_defragment_add_index(
page = buf_block_get_frame(block);
}
if (page == NULL && index->table->is_encrypted) {
if (page == NULL && index->table->file_unreadable) {
mtr_commit(&mtr);
*err = DB_DECRYPTION_FAILED;
return NULL;

10
storage/innobase/btr/btr0pcur.cc
View File

@ -416,6 +416,11 @@ btr_pcur_move_to_next_page(
cursor->old_stored = BTR_PCUR_OLD_NOT_STORED;
page = btr_pcur_get_page(cursor);
if (UNIV_UNLIKELY(!page)) {
return;
}
next_page_no = btr_page_get_next(page, mtr);
space = buf_block_get_space(btr_pcur_get_block(cursor));
zip_size = buf_block_get_zip_size(btr_pcur_get_block(cursor));
@ -425,6 +430,11 @@ btr_pcur_move_to_next_page(
next_block = btr_block_get(space, zip_size, next_page_no,
cursor->latch_mode,
btr_pcur_get_btr_cur(cursor)->index, mtr);
if (UNIV_UNLIKELY(!next_block)) {
return;
}
next_page = buf_block_get_frame(next_block);
#ifdef UNIV_BTR_DEBUG
ut_a(page_is_comp(next_page) == page_is_comp(page));

3
storage/innobase/btr/btr0scrub.cc
View File

@ -139,6 +139,7 @@ btr_scrub_lock_dict_func(ulint space_id, bool lock_to_close_table,
} else {
return false;
}
os_thread_sleep(250000);
time_t now = time(0);
@ -577,7 +578,7 @@ btr_scrub_table_needs_scrubbing(
return false;
}
if (table->corrupted) {
if (!table->is_readable()) {
return false;
}

447
storage/innobase/buf/buf0buf.cc
View File

@ -1114,7 +1114,6 @@ buf_block_init(
block->page.state = BUF_BLOCK_NOT_USED;
block->page.buf_fix_count = 0;
block->page.io_fix = BUF_IO_NONE;
block->page.key_version = 0;
block->page.encrypted = false;
block->page.real_size = 0;
block->page.write_size = 0;
@ -2358,7 +2357,18 @@ lookup:
/* Page not in buf_pool: needs to be read from file */
ut_ad(!hash_lock);
buf_read_page(space, zip_size, offset, NULL);
dberr_t err = buf_read_page(space, zip_size, offset);
if (err != DB_SUCCESS) {
ib_logf(IB_LOG_LEVEL_ERROR,
"Reading compressed page " ULINTPF
":" ULINTPF
" failed with error: %s.",
space, offset, ut_strerr(err));
goto err_exit;
}
#if defined UNIV_DEBUG || defined UNIV_BUF_DEBUG
ut_a(++buf_dbg_counter % 5771 || buf_validate());
@ -2916,7 +2926,6 @@ loop:
}
if (block == NULL) {
buf_page_t* bpage=NULL;
/* Page not in buf_pool: needs to be read from file */
@ -2950,7 +2959,18 @@ loop:
return(NULL);
}
if (buf_read_page(space, zip_size, offset, &bpage)) {
/* Call path is buf_read_page() -> buf_read_page_low()
(_fil_io()) -> buf_page_io_complete() ->
buf_decrypt_after_read() here fil_space_t* is used
and we decrypt -> buf_page_check_corrupt() where
page checksums are compared. Decryption/decompression
is handled lower level, error handling is handled on lower
level, here we need only to know is page really corrupted
or encrypted page with correct checksum. */
dberr_t local_err = buf_read_page(space, zip_size, offset);
if (local_err == DB_SUCCESS) {
buf_read_ahead_random(space, zip_size, offset,
ibuf_inside(mtr));
@ -2958,87 +2978,47 @@ loop:
} else if (retries < BUF_PAGE_READ_MAX_RETRIES) {
++retries;
bool corrupted = false;
if (bpage) {
corrupted = buf_page_check_corrupt(bpage);
}
/* Do not try again for encrypted pages */
if (corrupted && bpage->encrypted) {
ib_mutex_t* pmutex = buf_page_get_mutex(bpage);
buf_pool = buf_pool_from_bpage(bpage);
buf_pool_mutex_enter(buf_pool);
mutex_enter(pmutex);
ut_ad(buf_pool->n_pend_reads > 0);
os_atomic_decrement_ulint(&buf_pool->n_pend_reads, 1);
buf_page_set_io_fix(bpage, BUF_IO_NONE);
mutex_exit(pmutex);
buf_LRU_free_page(bpage, true);
buf_pool_mutex_exit(buf_pool);
rw_lock_x_unlock_gen(&((buf_block_t*) bpage)->lock,
BUF_IO_READ);
if (err) {
*err = DB_DECRYPTION_FAILED;
}
return (NULL);
}
DBUG_EXECUTE_IF(
"innodb_page_corruption_retries",
retries = BUF_PAGE_READ_MAX_RETRIES;
);
} else {
bool corrupted = false;
if (bpage) {
corrupted = buf_page_check_corrupt(bpage);
if (err) {
*err = local_err;
}
if (corrupted && !bpage->encrypted) {
ib_logf(IB_LOG_LEVEL_ERROR, "Unable"
" to read tablespace %lu page no"
" %lu into the buffer pool after"
" %lu attempts\n"
"InnoDB: The most probable cause"
" of this error may be that the"
" table has been corrupted.\n"
"InnoDB: You can try to fix this"
" problem by using"
" innodb_force_recovery.\n"
"InnoDB: Please see reference manual"
" for more details.\n"
"InnoDB: Aborting...\n",
space, offset,
BUF_PAGE_READ_MAX_RETRIES);
ut_error;
} else {
ib_mutex_t* pmutex = buf_page_get_mutex(bpage);
buf_pool = buf_pool_from_bpage(bpage);
buf_pool_mutex_enter(buf_pool);
mutex_enter(pmutex);
ut_ad(buf_pool->n_pend_reads > 0);
os_atomic_decrement_ulint(&buf_pool->n_pend_reads, 1);
buf_page_set_io_fix(bpage, BUF_IO_NONE);
mutex_exit(pmutex);
buf_LRU_free_page(bpage, true);
buf_pool_mutex_exit(buf_pool);
rw_lock_x_unlock_gen(&((buf_block_t*) bpage)->lock,
BUF_IO_READ);
if (err) {
*err = DB_DECRYPTION_FAILED;
}
/* Pages whose encryption key is unavailable or used
key, encryption algorithm or encryption method is
incorrect are marked as encrypted in
buf_page_check_corrupt(). Unencrypted page could be
corrupted in a way where the key_id field is
nonzero. There is no checksum on field
FIL_PAGE_FILE_FLUSH_LSN_OR_KEY_VERSION. */
if (local_err == DB_DECRYPTION_FAILED) {
return (NULL);
}
/* Try to set table as corrupted instead of
asserting. */
if (space > TRX_SYS_SPACE &&
dict_set_corrupted_by_space(space)) {
return (NULL);
}
ib_logf(IB_LOG_LEVEL_FATAL, "Unable"
" to read tablespace %lu page no"
" %lu into the buffer pool after"
" %lu attempts"
" The most probable cause"
" of this error may be that the"
" table has been corrupted."
" You can try to fix this"
" problem by using"
" innodb_force_recovery."
" Please see " REFMAN " for more"
" details. Aborting...",
space, offset,
BUF_PAGE_READ_MAX_RETRIES);
}
#if defined UNIV_DEBUG || defined UNIV_BUF_DEBUG
@ -3773,7 +3753,6 @@ buf_page_init_low(
bpage->newest_modification = 0;
bpage->oldest_modification = 0;
bpage->write_size = 0;
bpage->key_version = 0;
bpage->encrypted = false;
bpage->real_size = 0;
bpage->slot = NULL;
@ -4414,55 +4393,50 @@ buf_page_monitor(
/********************************************************************//**
Mark a table with the specified space pointed by bpage->space corrupted.
Also remove the bpage from LRU list.
@return TRUE if successful */
@param[in,out] bpage Block */
static
ibool
void
buf_mark_space_corrupt(
/*===================*/
buf_page_t* bpage) /*!< in: pointer to the block in question */
buf_page_t* bpage)
{
buf_pool_t* buf_pool = buf_pool_from_bpage(bpage);
const ibool uncompressed = (buf_page_get_state(bpage)
== BUF_BLOCK_FILE_PAGE);
ulint space = bpage->space;
ibool ret = TRUE;
if (!bpage->encrypted) {
/* First unfix and release lock on the bpage */
buf_pool_mutex_enter(buf_pool);
mutex_enter(buf_page_get_mutex(bpage));
ut_ad(buf_page_get_io_fix(bpage) == BUF_IO_READ);
ut_ad(bpage->buf_fix_count == 0);
/* First unfix and release lock on the bpage */
buf_pool_mutex_enter(buf_pool);
mutex_enter(buf_page_get_mutex(bpage));
ut_ad(buf_page_get_io_fix(bpage) == BUF_IO_READ);
ut_ad(bpage->buf_fix_count == 0);
/* Set BUF_IO_NONE before we remove the block from LRU list */
buf_page_set_io_fix(bpage, BUF_IO_NONE);
/* Set BUF_IO_NONE before we remove the block from LRU list */
buf_page_set_io_fix(bpage, BUF_IO_NONE);
if (uncompressed) {
rw_lock_x_unlock_gen(
&((buf_block_t*) bpage)->lock,
BUF_IO_READ);
}
mutex_exit(buf_page_get_mutex(bpage));
if (uncompressed) {
rw_lock_x_unlock_gen(
&((buf_block_t*) bpage)->lock,
BUF_IO_READ);
}
/* Find the table with specified space id, and mark it corrupted */
if (dict_set_corrupted_by_space(space)) {
if (!bpage->encrypted) {
buf_LRU_free_one_page(bpage);
}
mutex_exit(buf_page_get_mutex(bpage));
/* If block is not encrypted find the table with specified
space id, and mark it corrupted. Encrypted tables
are marked unusable later e.g. in ::open(). */
if (!bpage->encrypted) {
dict_set_corrupted_by_space(space);
} else {
ret = FALSE;
dict_set_encrypted_by_space(space);
}
if (!bpage->encrypted) {
ut_ad(buf_pool->n_pend_reads > 0);
buf_pool->n_pend_reads--;
/* After this point bpage can't be referenced. */
buf_LRU_free_one_page(bpage);
buf_pool_mutex_exit(buf_pool);
}
ut_ad(buf_pool->n_pend_reads > 0);
buf_pool->n_pend_reads--;
return(ret);
buf_pool_mutex_exit(buf_pool);
}
/********************************************************************//**
@ -4470,24 +4444,29 @@ Check if page is maybe compressed, encrypted or both when we encounter
corrupted page. Note that we can't be 100% sure if page is corrupted
or decrypt/decompress just failed.
@param[in,out] bpage Page
@return true if page corrupted, false if not */
UNIV_INTERN
bool
buf_page_check_corrupt(
buf_page_t* bpage)
@return DB_SUCCESS if page has been read and is not corrupted,
@retval DB_PAGE_CORRUPTED if page based on checksum check is corrupted,
@retval DB_DECRYPTION_FAILED if page post encryption checksum matches but
after decryption normal page checksum does not match.
@retval DB_TABLESPACE_DELETED if accessed tablespace is not found */
static
dberr_t
buf_page_check_corrupt(buf_page_t* bpage)
{
ulint zip_size = buf_page_get_zip_size(bpage);
byte* dst_frame = (zip_size) ? bpage->zip.data :
((buf_block_t*) bpage)->frame;
ulint space_id = bpage->space;
fil_space_t* space = fil_space_acquire_silent(space_id);
FilSpace space(bpage->space, true);
bool still_encrypted = false;
dberr_t err = DB_SUCCESS;
bool corrupted = false;
ulint page_type = mach_read_from_2(dst_frame + FIL_PAGE_TYPE);
fil_space_crypt_t* crypt_data = NULL;
ut_ad(space);
crypt_data = space->crypt_data;
if (!space()) {
return(DB_TABLESPACE_DELETED);
}
crypt_data = space()->crypt_data;
/* In buf_decrypt_after_read we have either decrypted the page if
page post encryption checksum matches and used key_id is found
@ -4499,72 +4478,75 @@ buf_page_check_corrupt(
crypt_data->type != CRYPT_SCHEME_UNENCRYPTED &&
!bpage->encrypted &&
fil_space_verify_crypt_checksum(dst_frame, zip_size,
space, bpage->offset));
space(), bpage->offset));
if (!still_encrypted) {
/* If traditional checksums match, we assume that page is
not anymore encrypted. */
corrupted = buf_page_is_corrupted(true, dst_frame, zip_size, space);
corrupted = buf_page_is_corrupted(true, dst_frame, zip_size, space());
if (!corrupted) {
bpage->encrypted = false;
} else {
err = DB_PAGE_CORRUPTED;
}
}
/* Pages that we think are unencrypted but do not match the checksum
checks could be corrupted or encrypted or both. */
if (corrupted && !bpage->encrypted) {
ib_logf(IB_LOG_LEVEL_ERROR,
"%s: Block in space_id " ULINTPF " in file %s corrupted.",
page_type == FIL_PAGE_PAGE_COMPRESSED_ENCRYPTED ? "Maybe corruption" : "Corruption",
space_id, (space && space->name) ? space->name : "NULL");
ib_logf(IB_LOG_LEVEL_ERROR,
"Based on page type %s (" ULINTPF ")",
fil_get_page_type_name(page_type), page_type);
/* An error will be reported by
buf_page_io_complete(). */
} else if (still_encrypted || (bpage->encrypted && corrupted)) {
bpage->encrypted = true;
corrupted = true;
err = DB_DECRYPTION_FAILED;
ib_logf(IB_LOG_LEVEL_ERROR,
"Block in space_id " ULINTPF " in file %s encrypted.",
space_id, (space && space->name) ? space->name : "NULL");
ib_logf(IB_LOG_LEVEL_ERROR,
"However key management plugin or used key_version %u is not found or"
" used encryption algorithm or method does not match.",
bpage->key_version);
if (space_id > TRX_SYS_SPACE) {
ib_logf(IB_LOG_LEVEL_ERROR,
"The page [page id: space=%u"
", page number=%u]"
" in file %s cannot be decrypted.",
bpage->space, bpage->offset,
space()->name);
ib_logf(IB_LOG_LEVEL_INFO,
"However key management plugin or used key_version " ULINTPF
" is not found or"
" used encryption algorithm or method does not match.",
mach_read_from_4(dst_frame+FIL_PAGE_FILE_FLUSH_LSN_OR_KEY_VERSION));
if (bpage->space > TRX_SYS_SPACE) {
ib_logf(IB_LOG_LEVEL_INFO,
"Marking tablespace as missing. You may drop this table or"
" install correct key management plugin and key file.");
}
}
if (space) {
fil_space_release(space);
}
return corrupted;
return (err);
}
/********************************************************************//**
Completes an asynchronous read or write request of a file page to or from
the buffer pool.
@return true if successful */
@param[in,out] bpage Page to complete
@param[in] evict whether or not to evict the page
from LRU list.
@return DB_SUCCESS if page has been read and is not corrupted,
DB_PAGE_CORRUPTED if page based on checksum check is corrupted,
DB_DECRYPTION_FAILED if page post encryption checksum matches but
after decryption normal page checksum does not match.
in write only DB_SUCCESS is possible. */
UNIV_INTERN
bool
dberr_t
buf_page_io_complete(
/*=================*/
buf_page_t* bpage, /*!< in: pointer to the block in question */
bool evict) /*!< in: whether or not to evict the page
from LRU list. */
buf_page_t* bpage,
bool evict)
{
enum buf_io_fix io_type;
buf_pool_t* buf_pool = buf_pool_from_bpage(bpage);
const ibool uncompressed = (buf_page_get_state(bpage)
== BUF_BLOCK_FILE_PAGE);
fil_space_t* space = NULL;
byte* frame = NULL;
bool corrupted = false;
byte* frame = NULL;
dberr_t err = DB_SUCCESS;
ut_a(buf_page_in_file(bpage));
@ -4578,8 +4560,9 @@ buf_page_io_complete(
ut_ad(io_type == BUF_IO_READ || io_type == BUF_IO_WRITE);
if (io_type == BUF_IO_READ) {
ulint read_page_no;
ulint read_space_id;
ulint read_page_no = 0;
ulint read_space_id = 0;
uint key_version = 0;
buf_page_decrypt_after_read(bpage);
@ -4591,20 +4574,23 @@ buf_page_io_complete(
if (buf_page_get_zip_size(bpage)) {
frame = bpage->zip.data;
buf_pool->n_pend_unzip++;
os_atomic_increment_ulint(&buf_pool->n_pend_unzip, 1);
if (uncompressed
&& !buf_zip_decompress((buf_block_t*) bpage,
FALSE)) {
buf_pool->n_pend_unzip--;
os_atomic_decrement_ulint(&buf_pool->n_pend_unzip, 1);
ib_logf(IB_LOG_LEVEL_INFO,
"Page %u in tablespace %u zip_decompress failure.",
bpage->offset, bpage->space);
err = DB_PAGE_CORRUPTED;
goto database_corrupted;
}
buf_pool->n_pend_unzip--;
os_atomic_decrement_ulint(&buf_pool->n_pend_unzip, 1);
} else {
ut_a(uncompressed);
frame = ((buf_block_t*) bpage)->frame;
@ -4616,6 +4602,8 @@ buf_page_io_complete(
read_page_no = mach_read_from_4(frame + FIL_PAGE_OFFSET);
read_space_id = mach_read_from_4(
frame + FIL_PAGE_ARCH_LOG_NO_OR_SPACE_ID);
key_version = mach_read_from_4(
frame + FIL_PAGE_FILE_FLUSH_LSN_OR_KEY_VERSION);
if (bpage->space == TRX_SYS_SPACE
&& buf_dblwr_page_inside(bpage->offset)) {
@ -4636,103 +4624,79 @@ buf_page_io_complete(
page may contain garbage in MySQL < 4.1.1,
which only supported bpage->space == 0. */
ut_print_timestamp(stderr);
fprintf(stderr,
" InnoDB: Error: space id and page n:o"
" stored in the page\n"
"InnoDB: read in are " ULINTPF ":" ULINTPF ","
" should be %u:%u!\n",
ib_logf(IB_LOG_LEVEL_ERROR,
"Space id and page n:o"
" stored in the page"
" read in are " ULINTPF ":" ULINTPF ","
" should be %u:%u!",
read_space_id,
read_page_no,
bpage->space,
bpage->offset);
}
corrupted = buf_page_check_corrupt(bpage);
err = buf_page_check_corrupt(bpage);
database_corrupted:
if (corrupted) {
if (err != DB_SUCCESS) {
/* Not a real corruption if it was triggered by
error injection */
DBUG_EXECUTE_IF("buf_page_is_corrupt_failure",
if (bpage->space > TRX_SYS_SPACE
&& buf_mark_space_corrupt(bpage)) {
if (bpage->space > TRX_SYS_SPACE) {
buf_mark_space_corrupt(bpage);
ib_logf(IB_LOG_LEVEL_INFO,
"Simulated page corruption");
return(true);
return(err);
}
err = DB_SUCCESS;
goto page_not_corrupt;
);
if (!bpage->encrypted) {
if (err == DB_PAGE_CORRUPTED) {
fil_system_enter();
space = fil_space_get_by_id(bpage->space);
fil_system_exit();
ib_logf(IB_LOG_LEVEL_ERROR,
"Database page corruption on disk"
" or a failed");
ib_logf(IB_LOG_LEVEL_ERROR,
"Space %u file %s read of page %u.",
bpage->space,
space->name ? space->name : "NULL",
bpage->offset);
ib_logf(IB_LOG_LEVEL_ERROR,
"You may have to recover"
" from a backup.");
" or a failed file read of tablespace %s"
" page [page id: space=%u"
", page number=%u]"
". You may have to recover from "
"a backup.",
space->name,
bpage->space, bpage->offset);
fil_system_exit();
buf_page_print(frame, buf_page_get_zip_size(bpage),
BUF_PAGE_PRINT_NO_CRASH);
ib_logf(IB_LOG_LEVEL_ERROR,
"It is also possible that your operating"
"system has corrupted its own file cache.");
ib_logf(IB_LOG_LEVEL_ERROR,
"and rebooting your computer removes the error.");
ib_logf(IB_LOG_LEVEL_ERROR,
"If the corrupt page is an index page you can also try to");
ib_logf(IB_LOG_LEVEL_ERROR,
"fix the corruption by dumping, dropping, and reimporting");
ib_logf(IB_LOG_LEVEL_ERROR,
"the corrupt table. You can use CHECK");
ib_logf(IB_LOG_LEVEL_ERROR,
"TABLE to scan your table for corruption.");
ib_logf(IB_LOG_LEVEL_ERROR,
"See also "
REFMAN "forcing-innodb-recovery.html"
" about forcing recovery.");
ib_logf(IB_LOG_LEVEL_INFO,
"It is also possible that your"
" operating system has corrupted"
" its own file cache and rebooting"
" your computer removes the error."
" If the corrupt page is an index page."
" You can also try to fix the"
" corruption by dumping, dropping,"
" and reimporting the corrupt table."
" You can use CHECK TABLE to scan"
" your table for corruption. "
"Please refer to " REFMAN "forcing-innodb-recovery.html"
" for information about forcing recovery.");
}
if (srv_force_recovery < SRV_FORCE_IGNORE_CORRUPT) {
/* If page space id is larger than TRX_SYS_SPACE
(0), we will attempt to mark the corresponding
table as corrupted instead of crashing server */
if (bpage->space > TRX_SYS_SPACE
&& buf_mark_space_corrupt(bpage)) {
return(false);
if (bpage->space > TRX_SYS_SPACE) {
buf_mark_space_corrupt(bpage);
return(err);
} else {
if (!bpage->encrypted) {
ib_logf(IB_LOG_LEVEL_ERROR,
"Ending processing because of a corrupt database page.");
ut_error;
}
ib_push_warning(innobase_get_trx(), DB_DECRYPTION_FAILED,
"Table in tablespace %lu encrypted."
"However key management plugin or used key_id %lu is not found or"
" used encryption algorithm or method does not match."
" Can't continue opening the table.",
bpage->space, bpage->key_version);
if (bpage->encrypted && bpage->space > TRX_SYS_SPACE) {
buf_mark_space_corrupt(bpage);
} else {
ut_error;
}
return(false);
ib_logf(IB_LOG_LEVEL_FATAL,
"Ending processing because of a corrupt database page.");
}
}
}
@ -4752,11 +4716,11 @@ database_corrupted:
if (bpage && bpage->encrypted) {
ib_logf(IB_LOG_LEVEL_WARN,
"Table in tablespace %lu encrypted."
"Table in tablespace " ULINTPF " encrypted."
"However key management plugin or used key_version %u is not found or"
" used encryption algorithm or method does not match."
" Can't continue opening the table.\n",
(ulint)bpage->space, bpage->key_version);
" Can't continue opening the table.",
read_space_id, key_version);
} else {
ibuf_merge_or_delete_for_page(
@ -4861,7 +4825,7 @@ database_corrupted:
buf_pool_mutex_exit(buf_pool);
return(true);
return(err);
}
/*********************************************************************//**
@ -4886,7 +4850,7 @@ buf_all_freed_instance(
const buf_block_t* block = buf_chunk_not_freed(chunk);
if (UNIV_LIKELY_NULL(block) && block->page.key_version == 0) {
if (UNIV_LIKELY_NULL(block)) {
fil_space_t* space = fil_space_get(block->page.space);
ib_logf(IB_LOG_LEVEL_ERROR,
"Page %u %u still fixed or dirty.",
@ -6102,32 +6066,29 @@ buf_page_encrypt_before_write(
switch (bpage->offset) {
case 0:
/* Page 0 of a tablespace is not encrypted/compressed */
ut_ad(bpage->key_version == 0);
return src_frame;
case TRX_SYS_PAGE_NO:
if (bpage->space == TRX_SYS_SPACE) {
/* don't encrypt/compress page as it contains
address to dblwr buffer */
bpage->key_version = 0;
return src_frame;
}
}
fil_space_crypt_t* crypt_data = space->crypt_data;
const bool encrypted = crypt_data
&& !crypt_data->not_encrypted()
&& crypt_data->type != CRYPT_SCHEME_UNENCRYPTED
&& (!crypt_data->is_default_encryption()
|| srv_encrypt_tables);
if (!encrypted) {
bpage->key_version = 0;
}
bool page_compressed = FSP_FLAGS_HAS_PAGE_COMPRESSION(space->flags);
if (!encrypted && !page_compressed) {
/* No need to encrypt or page compress the page */
/* No need to encrypt or page compress the page.
Clear key-version & crypt-checksum. */
memset(src_frame + FIL_PAGE_FILE_FLUSH_LSN_OR_KEY_VERSION, 0, 8);
return src_frame;
}
@ -6149,9 +6110,6 @@ buf_page_encrypt_before_write(
src_frame,
dst_frame);
ulint key_version = mach_read_from_4(dst_frame + FIL_PAGE_FILE_FLUSH_LSN_OR_KEY_VERSION);
ut_ad(key_version == 0 || key_version >= bpage->key_version);
bpage->key_version = key_version;
bpage->real_size = page_size;
slot->out_buf = dst_frame = tmp;
@ -6220,19 +6178,17 @@ buf_page_decrypt_after_read(
buf_pool_t* buf_pool = buf_pool_from_bpage(bpage);
bool success = true;
bpage->key_version = key_version;
if (bpage->offset == 0) {
/* File header pages are not encrypted/compressed */
return (true);
}
fil_space_t* space = fil_space_acquire(bpage->space, true);
FilSpace space(bpage->space, false, true);
/* Page is encrypted if encryption information is found from
tablespace and page contains used key_version. This is true
also for pages first compressed and then encrypted. */
if (!space || !space->crypt_data) {
if (!space() || !space()->crypt_data) {
key_version = 0;
}
@ -6266,6 +6222,14 @@ buf_page_decrypt_after_read(
decrypt. */
if (!fil_space_verify_crypt_checksum(dst_frame,
zip_size, NULL, bpage->offset)) {
/* Mark page encrypted in case it should
be. */
if (key_version && space()->crypt_data &&
space()->crypt_data->type != CRYPT_SCHEME_UNENCRYPTED) {
bpage->encrypted=true;
}
return (false);
}
@ -6277,7 +6241,7 @@ buf_page_decrypt_after_read(
#endif
/* decrypt using crypt_buf to dst_frame */
byte* res = fil_space_decrypt(space,
byte* res = fil_space_decrypt(space(),
slot->crypt_buf,
dst_frame,
&bpage->encrypted);
@ -6313,8 +6277,5 @@ buf_page_decrypt_after_read(
}
}
if (space != NULL) {
fil_space_release(space);
}
return (success);
}

9
storage/innobase/buf/buf0dblwr.cc
View File

@ -493,7 +493,9 @@ buf_dblwr_process()
page_no = mach_read_from_4(page + FIL_PAGE_OFFSET);
space_id = mach_read_from_4(page + FIL_PAGE_SPACE_ID);
if (!fil_tablespace_exists_in_mem(space_id)) {
FilSpace space(space_id, true);
if (!space()) {
/* Maybe we have dropped the single-table tablespace
and this page once belonged to it: do nothing */
continue;
@ -508,8 +510,7 @@ buf_dblwr_process()
continue;
}
fil_space_t* space = fil_space_found_by_id(space_id);
ulint zip_size = fil_space_get_zip_size(space_id);
ulint zip_size = fsp_flags_get_zip_size(space()->flags);
ut_ad(!buf_page_is_zeroes(page, zip_size));
/* Read in the actual page from the file */
@ -545,7 +546,7 @@ buf_dblwr_process()
if (fil_space_verify_crypt_checksum(
read_buf, zip_size, NULL, page_no)
|| !buf_page_is_corrupted(
true, read_buf, zip_size, space)) {
true, read_buf, zip_size, space())) {
/* The page is good; there is no need
to consult the doublewrite buffer. */
continue;

5
storage/innobase/buf/buf0flu.cc
View File

@ -958,7 +958,12 @@ buf_flush_write_block_low(
/* true means we want to evict this page from the
LRU list as well. */
#ifdef UNIV_DEBUG
dberr_t err =
#endif
buf_page_io_complete(bpage, true);
ut_ad(err == DB_SUCCESS);
}
fil_space_release(space);

263
storage/innobase/buf/buf0rea.cc
View File

@ -1,7 +1,7 @@
/*****************************************************************************
Copyright (c) 1995, 2013, Oracle and/or its affiliates. All Rights Reserved.
Copyright (c) 2015. MariaDB Corporation.
Copyright (c) 2015, 2017, MariaDB Corporation. All Rights Reserved.
This program is free software; you can redistribute it and/or modify it under
the terms of the GNU General Public License as published by the Free Software
@ -100,33 +100,43 @@ flag is cleared and the x-lock released by an i/o-handler thread.
in buf_pool, or if the page is in the doublewrite buffer blocks in
which case it is never read into the pool, or if the tablespace does
not exist or is being dropped
@param[out] err DB_SUCCESS, DB_TABLESPACE_DELETED if we are
trying to read from a non-existent tablespace, or a
tablespace which is just now being dropped,
DB_PAGE_CORRUPTED if page based on checksum
check is corrupted, or DB_DECRYPTION_FAILED
if page post encryption checksum matches but
after decryption normal page checksum does not match.
@param[in] sync true if synchronous aio is desired
@param[in] mode BUF_READ_IBUF_PAGES_ONLY, ...,
ORed to OS_AIO_SIMULATED_WAKE_LATER (see below
at read-ahead functions)
@param[in] space space id
@param[in] zip_size compressed page size, or 0
@param[in] unzip TRUE=request uncompressed page
@param[in] tablespace_version if the space memory object has
this timestamp different from what we are giving here,
treat the tablespace as dropped; this is a timestamp we
use to stop dangling page reads from a tablespace
which we have DISCARDed + IMPORTed back
@param[in] offset page number
@return 1 if read request is issued. 0 if it is not */
static
ulint
buf_read_page_low(
/*==============*/
dberr_t* err, /*!< out: DB_SUCCESS or DB_TABLESPACE_DELETED if we are
trying to read from a non-existent tablespace, or a
tablespace which is just now being dropped */
bool sync, /*!< in: true if synchronous aio is desired */
ulint mode, /*!< in: BUF_READ_IBUF_PAGES_ONLY, ...,
ORed to OS_AIO_SIMULATED_WAKE_LATER (see below
at read-ahead functions) */
ulint space, /*!< in: space id */
ulint zip_size,/*!< in: compressed page size, or 0 */
ibool unzip, /*!< in: TRUE=request uncompressed page */
ib_int64_t tablespace_version, /*!< in: if the space memory object has
this timestamp different from what we are giving here,
treat the tablespace as dropped; this is a timestamp we
use to stop dangling page reads from a tablespace
which we have DISCARDed + IMPORTed back */
ulint offset, /*!< in: page number */
buf_page_t** rbpage) /*!< out: page */
dberr_t* err,
bool sync,
ulint mode,
ulint space,
ulint zip_size,
ibool unzip,
ib_int64_t tablespace_version,
ulint offset)
{
buf_page_t* bpage;
ulint wake_later;
ibool ignore_nonexistent_pages;
*err = DB_SUCCESS;
wake_later = mode & OS_AIO_SIMULATED_WAKE_LATER;
@ -215,18 +225,13 @@ buf_read_page_low(
if (sync) {
/* The i/o is already completed when we arrive from
fil_read */
if (!buf_page_io_complete(bpage)) {
if (rbpage) {
*rbpage = bpage;
}
*err = buf_page_io_complete(bpage);
if (*err != DB_SUCCESS) {
return(0);
}
}
if (rbpage) {
*rbpage = bpage;
}
return(1);
}
@ -262,7 +267,7 @@ buf_read_ahead_random(
ulint ibuf_mode;
ulint count;
ulint low, high;
dberr_t err;
dberr_t err = DB_SUCCESS;
ulint i;
const ulint buf_read_ahead_random_area
= BUF_READ_AHEAD_AREA(buf_pool);
@ -357,16 +362,28 @@ read_ahead:
&err, false,
ibuf_mode | OS_AIO_SIMULATED_WAKE_LATER,
space, zip_size, FALSE,
tablespace_version, i, NULL);
if (err == DB_TABLESPACE_DELETED) {
ut_print_timestamp(stderr);
fprintf(stderr,
" InnoDB: Warning: in random"
" readahead trying to access\n"
"InnoDB: tablespace %lu page %lu,\n"
"InnoDB: but the tablespace does not"
" exist or is just being dropped.\n",
(ulong) space, (ulong) i);
tablespace_version, i);
switch(err) {
case DB_SUCCESS:
break;
case DB_TABLESPACE_DELETED:
ib_logf(IB_LOG_LEVEL_WARN,
"In random"
" readahead trying to access"
" tablespace " ULINTPF " page " ULINTPF
" but the tablespace does not"
" exist or is just being dropped.",
space, i);
break;
case DB_DECRYPTION_FAILED:
ib_logf(IB_LOG_LEVEL_ERROR,
"Random readahead failed to decrypt page "
ULINTPF "." ULINTPF " .",
space, i);
break;
default:
ut_error;
}
}
}
@ -400,43 +417,57 @@ High-level function which reads a page asynchronously from a file to the
buffer buf_pool if it is not already there. Sets the io_fix flag and sets
an exclusive lock on the buffer frame. The flag is cleared and the x-lock
released by the i/o-handler thread.
@return TRUE if page has been read in, FALSE in case of failure */
@param[in] space_id space_id
@param[in] zip_size compressed page size in bytes, or 0
@param[in] offset page number
@return DB_SUCCESS if page has been read and is not corrupted,
@retval DB_PAGE_CORRUPTED if page based on checksum check is corrupted,
@retval DB_DECRYPTION_FAILED if page post encryption checksum matches but
after decryption normal page checksum does not match.
@retval DB_TABLESPACE_DELETED if tablespace .ibd file is missing */
UNIV_INTERN
ibool
dberr_t
buf_read_page(
/*==========*/
ulint space, /*!< in: space id */
ulint zip_size,/*!< in: compressed page size in bytes, or 0 */
ulint offset, /*!< in: page number */
buf_page_t** bpage) /*!< out: page */
ulint space_id,
ulint zip_size,
ulint offset)
{
ib_int64_t tablespace_version;
ulint count;
dberr_t err;
dberr_t err = DB_SUCCESS;
tablespace_version = fil_space_get_version(space);
tablespace_version = fil_space_get_version(space_id);
/* We do the i/o in the synchronous aio mode to save thread
switches: hence TRUE */
FilSpace space(space_id, true);
count = buf_read_page_low(&err, true, BUF_READ_ANY_PAGE, space,
if (space()) {
/* We do the i/o in the synchronous aio mode to save thread
switches: hence TRUE */
count = buf_read_page_low(&err, true, BUF_READ_ANY_PAGE, space_id,
zip_size, FALSE,
tablespace_version, offset, bpage);
srv_stats.buf_pool_reads.add(count);
if (err == DB_TABLESPACE_DELETED) {
ut_print_timestamp(stderr);
fprintf(stderr,
" InnoDB: Error: trying to access"
" tablespace %lu page no. %lu,\n"
"InnoDB: but the tablespace does not exist"
" or is just being dropped.\n",
(ulong) space, (ulong) offset);
tablespace_version, offset);
srv_stats.buf_pool_reads.add(count);
}
/* Page corruption and decryption failures are already reported
in above function. */
if (!space() || err == DB_TABLESPACE_DELETED) {
err = DB_TABLESPACE_DELETED;
ib_logf(IB_LOG_LEVEL_ERROR,
"Trying to access"
" page [space=" ULINTPF ": page=" ULINTPF
"] but the tablespace does not exist"
" or is just being dropped.",
space_id, offset);
}
/* Increment number of I/O operations used for LRU policy. */
buf_LRU_stat_inc_io();
return(count > 0);
return(err);
}
/********************************************************************//**
@ -444,23 +475,23 @@ High-level function which reads a page asynchronously from a file to the
buffer buf_pool if it is not already there. Sets the io_fix flag and sets
an exclusive lock on the buffer frame. The flag is cleared and the x-lock
released by the i/o-handler thread.
@return TRUE if page has been read in, FALSE in case of failure */
@param[in] space Tablespace id
@param[in] offset Page no */
UNIV_INTERN
ibool
void
buf_read_page_async(
/*================*/
ulint space, /*!< in: space id */
ulint offset) /*!< in: page number */
ulint space,
ulint offset)
{
ulint zip_size;
ib_int64_t tablespace_version;
ulint count;
dberr_t err;
dberr_t err = DB_SUCCESS;
zip_size = fil_space_get_zip_size(space);
if (zip_size == ULINT_UNDEFINED) {
return(FALSE);
return;
}
tablespace_version = fil_space_get_version(space);
@ -469,7 +500,29 @@ buf_read_page_async(
| OS_AIO_SIMULATED_WAKE_LATER
| BUF_READ_IGNORE_NONEXISTENT_PAGES,
space, zip_size, FALSE,
tablespace_version, offset, NULL);
tablespace_version, offset);
switch(err) {
case DB_SUCCESS:
break;
case DB_TABLESPACE_DELETED:
ib_logf(IB_LOG_LEVEL_ERROR,
"In async page read "
"trying to access "
"page " ULINTPF ":" ULINTPF
" in nonexisting or being-dropped tablespace",
space, offset);
break;
case DB_DECRYPTION_FAILED:
ib_logf(IB_LOG_LEVEL_ERROR,
"Async page read failed to decrypt page "
ULINTPF ":" ULINTPF ".",
space, offset);
break;
default:
ut_error;
}
srv_stats.buf_pool_reads.add(count);
/* We do not increment number of I/O operations used for LRU policy
@ -478,8 +531,6 @@ buf_read_page_async(
buffer pool. Since this function is called from buffer pool load
these IOs are deliberate and are not part of normal workload we can
ignore these in our heuristics. */
return(count > 0);
}
/********************************************************************//**
@ -528,7 +579,7 @@ buf_read_ahead_linear(
ulint fail_count;
ulint ibuf_mode;
ulint low, high;
dberr_t err;
dberr_t err = DB_SUCCESS;
ulint i;
const ulint buf_read_ahead_linear_area
= BUF_READ_AHEAD_AREA(buf_pool);
@ -725,19 +776,32 @@ buf_read_ahead_linear(
aio mode: hence FALSE as the first parameter */
if (!ibuf_bitmap_page(zip_size, i)) {
count += buf_read_page_low(
&err, false,
ibuf_mode,
space, zip_size, FALSE, tablespace_version, i, NULL);
if (err == DB_TABLESPACE_DELETED) {
ut_print_timestamp(stderr);
fprintf(stderr,
" InnoDB: Warning: in"
" linear readahead trying to access\n"
"InnoDB: tablespace %lu page %lu,\n"
"InnoDB: but the tablespace does not"
" exist or is just being dropped.\n",
(ulong) space, (ulong) i);
space, zip_size, FALSE, tablespace_version, i);
switch(err) {
case DB_SUCCESS:
break;
case DB_TABLESPACE_DELETED:
ib_logf(IB_LOG_LEVEL_WARN,
"In linear "
" readahead trying to access "
" tablespace " ULINTPF ":" ULINTPF
" but the tablespace does not"
" exist or is just being dropped.",
space, i);
break;
case DB_DECRYPTION_FAILED:
ib_logf(IB_LOG_LEVEL_ERROR,
"Linear readahead failed to decrypt page "
ULINTPF ":" ULINTPF ".",
space, i);
break;
default:
ut_error;
}
}
}
@ -799,9 +863,9 @@ buf_read_ibuf_merge_pages(
#endif
for (i = 0; i < n_stored; i++) {
dberr_t err;
buf_pool_t* buf_pool;
ulint zip_size = fil_space_get_zip_size(space_ids[i]);
dberr_t err = DB_SUCCESS;
buf_pool = buf_pool_get(space_ids[i], page_nos[i]);
@ -811,16 +875,19 @@ buf_read_ibuf_merge_pages(
}
if (UNIV_UNLIKELY(zip_size == ULINT_UNDEFINED)) {
goto tablespace_deleted;
}
buf_read_page_low(&err, sync && (i + 1 == n_stored),
BUF_READ_ANY_PAGE, space_ids[i],
zip_size, TRUE, space_versions[i],
page_nos[i], NULL);
page_nos[i]);
switch(err) {
case DB_SUCCESS:
break;
case DB_TABLESPACE_DELETED:
if (UNIV_UNLIKELY(err == DB_TABLESPACE_DELETED)) {
tablespace_deleted:
/* We have deleted or are deleting the single-table
tablespace: remove the entries for that page */
@ -828,6 +895,15 @@ tablespace_deleted:
ibuf_merge_or_delete_for_page(NULL, space_ids[i],
page_nos[i],
zip_size, FALSE);
break;
case DB_DECRYPTION_FAILED:
ib_logf(IB_LOG_LEVEL_ERROR,
"Failed to decrypt insert buffer page "
ULINTPF ":" ULINTPF ".",
space_ids[i], page_nos[i]);
break;
default:
ut_error;
}
}
@ -865,7 +941,7 @@ buf_read_recv_pages(
{
ib_int64_t tablespace_version;
ulint count;
dberr_t err;
dberr_t err = DB_SUCCESS;
ulint i;
zip_size = fil_space_get_zip_size(space);
@ -909,12 +985,19 @@ buf_read_recv_pages(
if ((i + 1 == n_stored) && sync) {
buf_read_page_low(&err, true, BUF_READ_ANY_PAGE, space,
zip_size, TRUE, tablespace_version,
page_nos[i], NULL);
page_nos[i]);
} else {
buf_read_page_low(&err, false, BUF_READ_ANY_PAGE
| OS_AIO_SIMULATED_WAKE_LATER,
space, zip_size, TRUE,
tablespace_version, page_nos[i], NULL);
tablespace_version, page_nos[i]);
}
if (err == DB_DECRYPTION_FAILED) {
ib_logf(IB_LOG_LEVEL_ERROR,
"Recovery failed to decrypt read page "
ULINTPF ":" ULINTPF ".",
space, page_nos[i]);
}
}

2
storage/innobase/dict/dict0crea.cc
View File

@ -690,7 +690,7 @@ dict_create_index_tree_step(
dberr_t err = DB_SUCCESS;
ulint zip_size = dict_table_zip_size(index->table);
if (node->index->table->ibd_file_missing
if (node->index->table->file_unreadable
|| dict_table_is_discarded(node->index->table)) {
node->page_no = FIL_NULL;

68
storage/innobase/dict/dict0dict.cc
View File

@ -2,7 +2,7 @@
Copyright (c) 1996, 2016, Oracle and/or its affiliates. All Rights Reserved.
Copyright (c) 2012, Facebook Inc.
Copyright (c) 2013, 2015, MariaDB Corporation.
Copyright (c) 2013, 2017, MariaDB Corporation.
This program is free software; you can redistribute it and/or modify it under
the terms of the GNU General Public License as published by the Free Software
@ -1185,14 +1185,31 @@ dict_table_open_on_name(
if (table != NULL) {
/* If table is encrypted return table */
/* If table is encrypted or corrupted */
if (ignore_err == DICT_ERR_IGNORE_NONE
&& table->is_encrypted) {
&& !table->is_readable()) {
/* Make life easy for drop table. */
if (table->can_be_evicted) {
dict_table_move_from_lru_to_non_lru(table);
}
if (table->corrupted) {
if (!dict_locked) {
mutex_exit(&dict_sys->mutex);
}
char buf[MAX_FULL_NAME_LEN];
ut_format_name(table->name, TRUE, buf, sizeof(buf));
ib_logf(IB_LOG_LEVEL_ERROR,
"Table %s is corrupted. Please "
"drop the table and recreate.",
buf);
return(NULL);
}
if (table->can_be_evicted) {
dict_move_to_mru(table);
}
@ -1205,28 +1222,6 @@ dict_table_open_on_name(
return (table);
}
/* If table is corrupted, return NULL */
else if (ignore_err == DICT_ERR_IGNORE_NONE
&& table->corrupted) {
/* Make life easy for drop table. */
if (table->can_be_evicted) {
dict_table_move_from_lru_to_non_lru(table);
}
if (!dict_locked) {
mutex_exit(&dict_sys->mutex);
}
ut_print_timestamp(stderr);
fprintf(stderr, " InnoDB: table ");
ut_print_name(stderr, NULL, TRUE, table->name);
fprintf(stderr, "is corrupted. Please drop the table "
"and recreate\n");
return(NULL);
}
if (table->can_be_evicted) {
dict_move_to_mru(table);
@ -6183,11 +6178,29 @@ dict_set_corrupted_by_space(
/* mark the table->corrupted bit only, since the caller
could be too deep in the stack for SYS_INDEXES update */
table->corrupted = TRUE;
table->corrupted = true;
table->file_unreadable = true;
return(TRUE);
}
/** Flags a table with specified space_id encrypted in the data dictionary
cache
@param[in] space_id Tablespace id */
UNIV_INTERN
void
dict_set_encrypted_by_space(ulint space_id)
{
dict_table_t* table;
table = dict_find_table_by_space(space_id);
if (table) {
table->file_unreadable = true;
}
}
/**********************************************************************//**
Flags an index corrupted both in the data dictionary cache
and in the SYS_INDEXES */
@ -6645,7 +6658,8 @@ dict_table_schema_check(
}
}
if (table->ibd_file_missing) {
if (!table->is_readable() &&
fil_space_get(table->space) == NULL) {
/* missing tablespace */
ut_snprintf(errstr, errstr_sz,

23
storage/innobase/dict/dict0load.cc
View File

@ -1963,7 +1963,7 @@ dict_load_indexes(
dict_mem_index_free(index);
goto func_exit;
} else if (index->page == FIL_NULL
&& !table->ibd_file_missing
&& !table->file_unreadable
&& (!(index->type & DICT_FTS))) {
fprintf(stderr,
@ -2191,8 +2191,7 @@ err_len:
ut_ad(len == 8); /* this was checked earlier */
(*table)->id = mach_read_from_8(field);
(*table)->ibd_file_missing = FALSE;
(*table)->file_unreadable = false;
return(NULL);
}
@ -2379,7 +2378,7 @@ err_exit:
"Table '%s' tablespace is set as discarded.",
table_name);
table->ibd_file_missing = TRUE;
table->file_unreadable = true;
} else if (!fil_space_for_table_exists_in_mem(
table->space, name, false, true, heap,
@ -2387,7 +2386,7 @@ err_exit:
if (DICT_TF2_FLAG_IS_SET(table, DICT_TF2_TEMPORARY)) {
/* Do not bother to retry opening temporary tables. */
table->ibd_file_missing = TRUE;
table->file_unreadable = true;
} else {
if (!(ignore_err & DICT_ERR_IGNORE_RECOVER_LOCK)) {
@ -2422,8 +2421,9 @@ err_exit:
/* We failed to find a sensible
tablespace file */
table->ibd_file_missing = TRUE;
table->file_unreadable = true;
}
if (filepath) {
mem_free(filepath);
}
@ -2447,9 +2447,10 @@ err_exit:
were not allowed while the table is being locked by a transaction. */
dict_err_ignore_t index_load_err =
!(ignore_err & DICT_ERR_IGNORE_RECOVER_LOCK)
&& table->ibd_file_missing
&& table->file_unreadable
? DICT_ERR_IGNORE_ALL
: ignore_err;
err = dict_load_indexes(table, heap, index_load_err);
if (err == DB_INDEX_CORRUPT) {
@ -2484,7 +2485,7 @@ err_exit:
of the error condition, since the user may want to dump data from the
clustered index. However we load the foreign key information only if
all indexes were loaded. */
if (!cached || table->ibd_file_missing) {
if (!cached || table->file_unreadable) {
/* Don't attempt to load the indexes from disk. */
} else if (err == DB_SUCCESS) {
err = dict_load_foreigns(table->name, NULL, true, true,
@ -2517,12 +2518,12 @@ err_exit:
table = NULL;
} else if (dict_index_is_corrupted(index)
&& !table->ibd_file_missing) {
&& !table->file_unreadable) {
/* It is possible we force to load a corrupted
clustered index if srv_load_corrupted is set.
Mark the table as corrupted in this case */
table->corrupted = TRUE;
table->corrupted = true;
}
}
@ -2531,7 +2532,7 @@ func_exit:
ut_ad(!table
|| ignore_err != DICT_ERR_IGNORE_NONE
|| table->ibd_file_missing
|| table->file_unreadable
|| !table->corrupted);
if (table && table->fts) {

1
storage/innobase/dict/dict0mem.cc
View File

@ -2,6 +2,7 @@
Copyright (c) 1996, 2016, Oracle and/or its affiliates. All Rights Reserved.
Copyright (c) 2012, Facebook Inc.
Copyright (c) 2013, 2017, MariaDB Corporation.
This program is free software; you can redistribute it and/or modify it under
the terms of the GNU General Public License as published by the Free Software

100
storage/innobase/dict/dict0stats.cc
View File

@ -1,6 +1,7 @@
/*****************************************************************************
Copyright (c) 2009, 2016, Oracle and/or its affiliates. All Rights Reserved.
Copyright (c) 2015, 2017, MariaDB Corporation.
This program is free software; you can redistribute it and/or modify it under
the terms of the GNU General Public License as published by the Free Software
@ -920,7 +921,11 @@ dict_stats_update_transient_for_index(
index->stat_n_leaf_pages = size;
btr_estimate_number_of_different_key_vals(index);
/* Do not continue if table decryption has failed or
table is already marked as corrupted. */
if (index->is_readable()) {
btr_estimate_number_of_different_key_vals(index);
}
}
}
@ -974,8 +979,9 @@ dict_stats_update_transient(
continue;
}
/* Do not continue if table decryption has failed. */
if (index->table->is_encrypted) {
/* Do not continue if table decryption has failed or
table is already marked as corrupted. */
if (!index->is_readable()) {
break;
}
@ -2437,6 +2443,62 @@ dict_stats_save_index_stat(
return(ret);
}
/** Report error if statistic update for a table failed because
.ibd file is missing, table decryption failed or table is corrupted.
@param[in,out] table Table
@param[in] defragment true if statistics is for defragment
@return DB_DECRYPTION_FAILED, DB_TABLESPACE_DELETED or DB_CORRUPTION
@retval DB_DECRYPTION_FAILED if decryption of the table failed
@retval DB_TABLESPACE_DELETED if .ibd file is missing
@retval DB_CORRUPTION if table is marked as corrupted */
static
dberr_t
dict_stats_report_error(
dict_table_t* table,
bool defragment = false)
{
char buf[3 * NAME_LEN];
dberr_t err;
innobase_format_name(buf, sizeof buf,
table->name,
true);
FilSpace space(table->space);
if (space()) {
if (table->corrupted) {
ib_logf(IB_LOG_LEVEL_INFO,
"Cannot save%s statistics because "
" table %s in file %s is corrupted.",
defragment ? " defragment" : " ",
buf, space()->chain.start->name);
err = DB_CORRUPTION;
} else {
ib_logf(IB_LOG_LEVEL_INFO,
"Cannot save%s statistics because "
" table %s in file %s can't be decrypted.",
defragment ? " defragment" : " ",
buf, space()->chain.start->name);
err = DB_DECRYPTION_FAILED;
}
} else {
ib_logf(IB_LOG_LEVEL_INFO,
"Cannot save%s statistics for "
" table %s because .ibd file is missing."
" For help, please "
"refer to " REFMAN "innodb-troubleshooting.html.",
defragment ? " defragment" : " ",
buf);
err = DB_TABLESPACE_DELETED;
}
dict_stats_empty_table(table, defragment);
return (err);
}
/** Save the table's statistics into the persistent statistics storage.
@param[in] table_orig table whose stats to save
@param[in] only_for_index if this is non-NULL, then stats for indexes
@ -2457,6 +2519,11 @@ dict_stats_save(
char db_utf8[MAX_DB_UTF8_LEN];
char table_utf8[MAX_TABLE_UTF8_LEN];
if (table_orig->is_readable()) {
} else {
return (dict_stats_report_error(table_orig));
}
table = dict_stats_snapshot_create(table_orig);
dict_fs2utf8(table->name, db_utf8, sizeof(db_utf8),
@ -3192,15 +3259,8 @@ dict_stats_update(
ut_ad(!mutex_own(&dict_sys->mutex));
if (table->ibd_file_missing) {
ut_print_timestamp(stderr);
fprintf(stderr,
" InnoDB: cannot calculate statistics for table %s "
"because the .ibd file is missing. For help, please "
"refer to " REFMAN "innodb-troubleshooting.html\n",
ut_format_name(table->name, TRUE, buf, sizeof(buf)));
dict_stats_empty_table(table, true);
return(DB_TABLESPACE_DELETED);
if (!table->is_readable()) {
return (dict_stats_report_error(table));
} else if (srv_force_recovery >= SRV_FORCE_NO_IBUF_MERGE) {
/* If we have set a high innodb_force_recovery level, do
not calculate statistics, as a badly corrupted index can
@ -3946,19 +4006,9 @@ dict_stats_save_defrag_stats(
{
dberr_t ret;
if (index->table->ibd_file_missing) {
ut_print_timestamp(stderr);
fprintf(stderr,
" InnoDB: Cannot save defragment stats because "
".ibd file is missing.\n");
return (DB_TABLESPACE_DELETED);
}
if (dict_index_is_corrupted(index)) {
ut_print_timestamp(stderr);
fprintf(stderr,
" InnoDB: Cannot save defragment stats because "
"index is corrupted.\n");
return(DB_CORRUPTION);
if (index->is_readable()) {
} else {
return (dict_stats_report_error(index->table, true));
}
if (dict_index_is_univ(index)) {

100
storage/innobase/fil/fil0crypt.cc
View File

@ -479,11 +479,12 @@ Parse a MLOG_FILE_WRITE_CRYPT_DATA log entry
@param[in] block buffer block
@return position on log buffer */
UNIV_INTERN
const byte*
byte*
fil_parse_write_crypt_data(
const byte* ptr,
byte* ptr,
const byte* end_ptr,
const buf_block_t* block)
const buf_block_t* block,
dberr_t* err)
{
/* check that redo log entry is complete */
uint entry_size =
@ -495,6 +496,8 @@ fil_parse_write_crypt_data(
4 + // size of key_id
1; // fil_encryption_t
*err = DB_SUCCESS;
if (ptr + entry_size > end_ptr) {
return NULL;
}
@ -541,6 +544,11 @@ fil_parse_write_crypt_data(
fil_space_release(space);
}
/* Check is used key found from encryption plugin */
if (crypt_data->should_encrypt() && !crypt_data->is_key_found()) {
*err = DB_DECRYPTION_FAILED;
}
return ptr;
}
@ -710,10 +718,12 @@ fil_space_encrypt(
}
bool corrupted = buf_page_is_corrupted(true, tmp_mem, zip_size, space);
memcpy(tmp_mem+FIL_PAGE_FILE_FLUSH_LSN_OR_KEY_VERSION, src+FIL_PAGE_FILE_FLUSH_LSN_OR_KEY_VERSION, 8);
bool different = memcmp(src, tmp_mem, size);
if (!ok || corrupted || corrupted1 || err != DB_SUCCESS || different) {
fprintf(stderr, "ok %d corrupted %d corrupted1 %d err %d different %d\n", ok , corrupted, corrupted1, err, different);
fprintf(stderr, "ok %d corrupted %d corrupted1 %d err %d different %d\n",
ok , corrupted, corrupted1, err, different);
fprintf(stderr, "src_frame\n");
buf_page_print(src_frame, zip_size, BUF_PAGE_PRINT_NO_CRASH);
fprintf(stderr, "encrypted_frame\n");
@ -768,25 +778,6 @@ fil_space_decrypt(
return false;
}
if (crypt_data == NULL) {
if (!(space == 0 && offset == 0) && key_version != 0) {
/* FIL_PAGE_FILE_FLUSH_LSN field i.e.
FIL_PAGE_FILE_FLUSH_LSN_OR_KEY_VERSION
should be only defined for the
first page in a system tablespace
data file (ibdata*, not *.ibd), if not
clear it. */
DBUG_PRINT("ib_crypt",
("Page on space %lu offset %lu has key_version %u"
" when it shoud be undefined.",
space, offset, key_version));
mach_write_to_4(src_frame + FIL_PAGE_FILE_FLUSH_LSN_OR_KEY_VERSION, 0);
}
return false;
}
ut_a(crypt_data != NULL && crypt_data->is_encrypted());
/* read space & lsn */
@ -838,9 +829,6 @@ fil_space_decrypt(
memcpy(tmp_frame + page_size - FIL_PAGE_DATA_END,
src_frame + page_size - FIL_PAGE_DATA_END,
FIL_PAGE_DATA_END);
// clear key-version & crypt-checksum from dst
memset(tmp_frame + FIL_PAGE_FILE_FLUSH_LSN_OR_KEY_VERSION, 0, 8);
}
srv_stats.pages_decrypted.inc();
@ -962,11 +950,12 @@ fil_space_verify_crypt_checksum(
/* If page is not encrypted, return false */
if (key_version == 0) {
return false;
return(false);
}
srv_checksum_algorithm_t algorithm =
static_cast<srv_checksum_algorithm_t>(srv_checksum_algorithm);
/* If no checksum is used, can't continue checking. */
if (algorithm == SRV_CHECKSUM_ALGORITHM_NONE) {
return(true);
@ -990,28 +979,31 @@ fil_space_verify_crypt_checksum(
return (true);
}
/* Compressed pages use different checksum method. We first store
the post encryption checksum on checksum location and after function
restore the original. */
ib_uint32_t cchecksum1 = 0;
ib_uint32_t cchecksum2 = 0;
/* Calculate checksums */
if (zip_size) {
ib_uint32_t old = static_cast<ib_uint32_t>(mach_read_from_4(
page + FIL_PAGE_SPACE_OR_CHKSUM));
cchecksum1 = page_zip_calc_checksum(
page, zip_size, SRV_CHECKSUM_ALGORITHM_CRC32);
mach_write_to_4(page + FIL_PAGE_SPACE_OR_CHKSUM, checksum);
if(cchecksum1 != checksum) {
cchecksum2 = page_zip_calc_checksum(
page, zip_size,
SRV_CHECKSUM_ALGORITHM_INNODB);
}
} else {
cchecksum1 = buf_calc_page_crc32(page);
bool valid = page_zip_verify_checksum(page, zip_size);
mach_write_to_4(page + FIL_PAGE_SPACE_OR_CHKSUM, old);
return (valid);
if (cchecksum1 != checksum) {
cchecksum2 = (ib_uint32_t) buf_calc_page_new_checksum(
page);
}
}
/* If stored checksum matches one of the calculated checksums
page is not corrupted. */
ib_uint32_t cchecksum1 = buf_calc_page_crc32(page);
ib_uint32_t cchecksum2 = (ib_uint32_t) buf_calc_page_new_checksum(
page);
bool encrypted = (checksum == cchecksum1 || checksum == cchecksum2
|| checksum == BUF_NO_CHECKSUM_MAGIC);
@ -1042,13 +1034,19 @@ fil_space_verify_crypt_checksum(
ulint checksum1 = mach_read_from_4(
page + FIL_PAGE_SPACE_OR_CHKSUM);
ulint checksum2 = mach_read_from_4(
page + UNIV_PAGE_SIZE - FIL_PAGE_END_LSN_OLD_CHKSUM);
ulint checksum2 = checksum1;
bool valid;
bool valid = (buf_page_is_checksum_valid_crc32(page,checksum1,checksum2)
if (zip_size) {
valid = (checksum1 == cchecksum1);
} else {
checksum2 = mach_read_from_4(
page + UNIV_PAGE_SIZE - FIL_PAGE_END_LSN_OLD_CHKSUM);
valid = (buf_page_is_checksum_valid_crc32(page,checksum1,checksum2)
|| buf_page_is_checksum_valid_none(page,checksum1,checksum2)
|| buf_page_is_checksum_valid_innodb(page,checksum1, checksum2));
}
if (encrypted && valid) {
/* If page is encrypted and traditional checksums match,
@ -1964,11 +1962,11 @@ fil_crypt_rotate_page(
bool modified = false;
int needs_scrubbing = BTR_SCRUB_SKIP_PAGE;
lsn_t block_lsn = block->page.newest_modification;
uint kv = block->page.key_version;
byte* frame = buf_block_get_frame(block);
uint kv = mach_read_from_4(frame+FIL_PAGE_FILE_FLUSH_LSN_OR_KEY_VERSION);
/* check if tablespace is closing after reading page */
if (space->is_stopping()) {
byte* frame = buf_block_get_frame(block);
if (!space->is_stopping()) {
if (kv == 0 &&
fil_crypt_is_page_uninitialized(frame, zip_size)) {
@ -1990,9 +1988,6 @@ fil_crypt_rotate_page(
FIL_PAGE_ARCH_LOG_NO_OR_SPACE_ID,
space_id, MLOG_4BYTES, &mtr);
/* update block */
block->page.key_version = key_state->key_version;
/* statistics */
state->crypt_stat.pages_modified++;
} else {
@ -2083,6 +2078,11 @@ fil_crypt_rotate_page(
state->end_lsn = block_lsn;
}
}
} else {
/* If block read failed mtr memo and log should be empty. */
ut_ad(dyn_array_get_data_size(&mtr.memo) == 0);
ut_ad(dyn_array_get_data_size(&mtr.log) == 0);
mtr_commit(&mtr);
}
if (sleeptime_ms) {

237
storage/innobase/fil/fil0fil.cc
View File

@ -327,7 +327,9 @@ fil_write(
}
/*******************************************************************//**
Returns the table space by a given id, NULL if not found. */
Returns the table space by a given id, NULL if not found.
It is unsafe to dereference the returned pointer. It is fine to check
for NULL. */
fil_space_t*
fil_space_get_by_id(
/*================*/
@ -345,26 +347,6 @@ fil_space_get_by_id(
return(space);
}
/*******************************************************************//**
Returns the table space by a given id, NULL if not found. */
fil_space_t*
fil_space_found_by_id(
/*==================*/
ulint id) /*!< in: space id */
{
fil_space_t* space = NULL;
mutex_enter(&fil_system->mutex);
space = fil_space_get_by_id(id);
/* Not found if space is being deleted */
if (space && space->stop_new_ops) {
space = NULL;
}
mutex_exit(&fil_system->mutex);
return space;
}
/****************************************************************//**
Get space id from fil node */
ulint
@ -2482,74 +2464,6 @@ fil_read_first_page(
/*================ SINGLE-TABLE TABLESPACES ==========================*/
#ifndef UNIV_HOTBACKUP
/*******************************************************************//**
Increments the count of pending operation, if space is not being deleted.
@return TRUE if being deleted, and operation should be skipped */
UNIV_INTERN
ibool
fil_inc_pending_ops(
/*================*/
ulint id, /*!< in: space id */
ibool print_err) /*!< in: need to print error or not */
{
fil_space_t* space;
mutex_enter(&fil_system->mutex);
space = fil_space_get_by_id(id);
if (space == NULL) {
if (print_err) {
fprintf(stderr,
"InnoDB: Error: trying to do an operation on a"
" dropped tablespace %lu\n",
(ulong) id);
}
}
if (space == NULL || space->stop_new_ops) {
mutex_exit(&fil_system->mutex);
return(TRUE);
}
space->n_pending_ops++;
mutex_exit(&fil_system->mutex);
return(FALSE);
}
/*******************************************************************//**
Decrements the count of pending operations. */
UNIV_INTERN
void
fil_decr_pending_ops(
/*=================*/
ulint id) /*!< in: space id */
{
fil_space_t* space;
mutex_enter(&fil_system->mutex);
space = fil_space_get_by_id(id);
if (space == NULL) {
fprintf(stderr,
"InnoDB: Error: decrementing pending operation"
" of a dropped tablespace %lu\n",
(ulong) id);
}
if (space != NULL) {
space->n_pending_ops--;
}
mutex_exit(&fil_system->mutex);
}
#endif /* !UNIV_HOTBACKUP */
/********************************************************//**
Creates the database directory for a table if it does not exist yet. */
static
@ -2936,7 +2850,7 @@ fil_check_pending_operations(
fil_space_t* sp = fil_space_get_by_id(id);
if (sp) {
sp->stop_new_ops = TRUE;
sp->stop_new_ops = true;
/* space could be freed by other threads as soon
as n_pending_ops reaches 0, thus increment pending
ops here. */
@ -5353,7 +5267,7 @@ fil_tablespace_deleted_or_being_deleted_in_mem(
space = fil_space_get_by_id(id);
if (space == NULL || space->stop_new_ops) {
if (space == NULL || space->is_stopping()) {
mutex_exit(&fil_system->mutex);
return(TRUE);
@ -6105,15 +6019,19 @@ fil_io(
/* If we are deleting a tablespace we don't allow async read operations
on that. However, we do allow write and sync read operations */
if (space == 0
|| (type == OS_FILE_READ && !sync && space->stop_new_ops)) {
|| (type == OS_FILE_READ
&& !sync
&& space->stop_new_ops)) {
mutex_exit(&fil_system->mutex);
ib_logf(IB_LOG_LEVEL_ERROR,
"Trying to do i/o to a tablespace which does "
"not exist. i/o type %lu, space id %lu, "
"page no. %lu, i/o length %lu bytes",
(ulong) type, (ulong) space_id, (ulong) block_offset,
(ulong) len);
"not exist. i/o type " ULINTPF
", space id " ULINTPF " , "
"page no. " ULINTPF
", i/o length " ULINTPF " bytes",
type, space_id, block_offset,
len);
return(DB_TABLESPACE_DELETED);
}
@ -6295,6 +6213,7 @@ fil_aio_wait(
mutex_enter(&fil_system->mutex);
fil_node_complete_io(fil_node, fil_system, type);
ulint purpose = fil_node->space->purpose;
mutex_exit(&fil_system->mutex);
@ -6306,9 +6225,27 @@ fil_aio_wait(
deadlocks in the i/o system. We keep tablespace 0 data files always
open, and use a special i/o thread to serve insert buffer requests. */
if (fil_node->space->purpose == FIL_TABLESPACE) {
if (purpose == FIL_TABLESPACE) {
srv_set_io_thread_op_info(segment, "complete io for buf page");
buf_page_io_complete(static_cast<buf_page_t*>(message));
buf_page_t* bpage = static_cast<buf_page_t*>(message);
ulint offset = bpage->offset;
dberr_t err = buf_page_io_complete(bpage);
if (err != DB_SUCCESS) {
ut_ad(type == OS_FILE_READ);
/* In crash recovery set log corruption on
and produce only an error to fail InnoDB startup. */
if (recv_recovery_is_on() && !srv_force_recovery) {
recv_sys->found_corrupt_log = true;
}
ib_logf(IB_LOG_LEVEL_ERROR,
"Read operation failed for tablespace %s"
" offset " ULINTPF " with error %s",
fil_node->name,
offset,
ut_strerr(err));
}
} else {
srv_set_io_thread_op_info(segment, "complete io for log");
log_io_complete(static_cast<log_group_t*>(message));
@ -6329,7 +6266,7 @@ fil_flush(
mutex_enter(&fil_system->mutex);
if (fil_space_t* space = fil_space_get_by_id(space_id)) {
if (!space->is_stopping()) {
if (!space->stop_new_ops) {
fil_flush_low(space);
}
}
@ -7219,76 +7156,6 @@ fil_mtr_rename_log(
return(err);
}
/******************************************************************
Get id of first tablespace or ULINT_UNDEFINED if none */
UNIV_INTERN
ulint
fil_get_first_space()
/*=================*/
{
ulint out_id = ULINT_UNDEFINED;
fil_space_t* space;
mutex_enter(&fil_system->mutex);
space = UT_LIST_GET_FIRST(fil_system->space_list);
if (space != NULL) {
do
{
if (!space->stop_new_ops) {
out_id = space->id;
break;
}
space = UT_LIST_GET_NEXT(space_list, space);
} while (space != NULL);
}
mutex_exit(&fil_system->mutex);
return out_id;
}
/******************************************************************
Get id of next tablespace or ULINT_UNDEFINED if none */
UNIV_INTERN
ulint
fil_get_next_space(
/*===============*/
ulint id) /*!< in: previous space id */
{
bool found;
fil_space_t* space;
ulint out_id = ULINT_UNDEFINED;
mutex_enter(&fil_system->mutex);
space = fil_space_get_by_id(id);
if (space == NULL) {
/* we didn't find it...search for space with space->id > id */
found = false;
space = UT_LIST_GET_FIRST(fil_system->space_list);
} else {
/* we found it, take next available space */
found = true;
}
while ((space = UT_LIST_GET_NEXT(space_list, space)) != NULL) {
if (!found && space->id <= id)
continue;
if (!space->stop_new_ops && UT_LIST_GET_LEN(space->chain) > 0) {
/* inc reference to prevent drop */
out_id = space->id;
break;
}
}
mutex_exit(&fil_system->mutex);
return out_id;
}
/** Acquire a tablespace when it could be dropped concurrently.
Used by background threads that do not necessarily hold proper locks
for concurrency control.
@ -7298,9 +7165,9 @@ for concurrency control.
(possibly executing TRUNCATE)
@return the tablespace
@retval NULL if missing or being deleted or truncated */
inline
UNIV_INTERN
fil_space_t*
fil_space_acquire_low(ulint id, bool silent, bool for_io = false)
fil_space_acquire_low(ulint id, bool silent, bool for_io)
{
fil_space_t* space;
@ -7324,32 +7191,6 @@ fil_space_acquire_low(ulint id, bool silent, bool for_io = false)
return(space);
}
/** Acquire a tablespace when it could be dropped concurrently.
Used by background threads that do not necessarily hold proper locks
for concurrency control.
@param[in] id tablespace ID
@param[in] for_io whether to look up the tablespace while performing I/O
(possibly executing TRUNCATE)
@return the tablespace
@retval NULL if missing or being deleted or truncated */
fil_space_t*
fil_space_acquire(ulint id, bool for_io)
{
return(fil_space_acquire_low(id, false, for_io));
}
/** Acquire a tablespace that may not exist.
Used by background threads that do not necessarily hold proper locks
for concurrency control.
@param[in] id tablespace ID
@return the tablespace
@retval NULL if missing or being deleted */
fil_space_t*
fil_space_acquire_silent(ulint id)
{
return(fil_space_acquire_low(id, true));
}
/** Release a tablespace acquired with fil_space_acquire().
@param[in,out] space tablespace to release */
void
@ -7478,7 +7319,7 @@ fil_space_keyrotate_next(
space->purpose == FIL_TABLESPACE. */
while (space != NULL
&& (UT_LIST_GET_LEN(space->chain) == 0
|| space->stop_new_ops)) {
|| space->is_stopping())) {
old = space;
space = UT_LIST_GET_NEXT(rotation_list, space);

140
storage/innobase/handler/ha_innodb.cc
View File

@ -5675,20 +5675,23 @@ ha_innobase::innobase_initialize_autoinc()
break;
}
case DB_RECORD_NOT_FOUND:
ut_print_timestamp(stderr);
fprintf(stderr, " InnoDB: MySQL and InnoDB data "
"dictionaries are out of sync.\n"
"InnoDB: Unable to find the AUTOINC column "
"%s in the InnoDB table %s.\n"
"InnoDB: We set the next AUTOINC column "
"value to 0,\n"
"InnoDB: in effect disabling the AUTOINC "
"next value generation.\n"
"InnoDB: You can either set the next "
"AUTOINC value explicitly using ALTER TABLE\n"
"InnoDB: or fix the data dictionary by "
"recreating the table.\n",
col_name, index->table->name);
char buf[MAX_FULL_NAME_LEN];
ut_format_name(index->table->name, TRUE, buf, sizeof(buf));
ib_logf(IB_LOG_LEVEL_ERROR,
"MySQL and InnoDB data "
"dictionaries are out of sync."
" Unable to find the AUTOINC column "
" %s in the InnoDB table %s."
" We set the next AUTOINC column "
"value to 0"
" in effect disabling the AUTOINC "
"next value generation."
" You can either set the next "
"AUTOINC value explicitly using ALTER TABLE "
" or fix the data dictionary by "
"recreating the table.",
col_name, buf);
/* This will disable the AUTOINC generation. */
auto_inc = 0;
@ -5780,16 +5783,17 @@ ha_innobase::open(
&& (table->s->fields
!= dict_table_get_n_user_cols(ib_table) - 1)))) {
ib_logf(IB_LOG_LEVEL_WARN,
"table %s contains %lu user defined columns "
"in InnoDB, but %lu columns in MySQL. Please "
"table %s contains " ULINTPF " user defined columns "
"in InnoDB, but %u columns in MySQL. Please "
"check INFORMATION_SCHEMA.INNODB_SYS_COLUMNS and "
REFMAN "innodb-troubleshooting.html "
"for how to resolve it",
norm_name, (ulong) dict_table_get_n_user_cols(ib_table),
(ulong) table->s->fields);
norm_name, dict_table_get_n_user_cols(ib_table),
table->s->fields);
/* Mark this table as corrupted, so the drop table
or force recovery can still use it, but not others. */
ib_table->file_unreadable = true;
ib_table->corrupted = true;
dict_table_close(ib_table, FALSE, FALSE);
ib_table = NULL;
@ -5890,7 +5894,7 @@ table_opened:
ib_table->thd = (void*)thd;
/* No point to init any statistics if tablespace is still encrypted. */
if (!ib_table->is_encrypted) {
if (ib_table->is_readable()) {
dict_stats_init(ib_table);
} else {
ib_table->stat_initialized = 1;
@ -5898,7 +5902,9 @@ table_opened:
MONITOR_INC(MONITOR_TABLE_OPEN);
bool no_tablespace;
bool no_tablespace = false;
bool encrypted = false;
FilSpace space;
if (dict_table_is_discarded(ib_table)) {
@ -5913,21 +5919,28 @@ table_opened:
no_tablespace = false;
} else if (ib_table->ibd_file_missing) {
} else if (!ib_table->is_readable()) {
space = fil_space_acquire_silent(ib_table->space);
ib_senderrf(
thd, IB_LOG_LEVEL_WARN,
ER_TABLESPACE_MISSING, norm_name);
if (space()) {
if (space()->crypt_data && space()->crypt_data->is_encrypted()) {
/* This means that tablespace was found but we could not
decrypt encrypted page. */
no_tablespace = true;
encrypted = true;
} else {
no_tablespace = true;
}
} else {
ib_senderrf(
thd, IB_LOG_LEVEL_WARN,
ER_TABLESPACE_MISSING, norm_name);
/* This means we have no idea what happened to the tablespace
file, best to play it safe. */
/* This means we have no idea what happened to the tablespace
file, best to play it safe. */
no_tablespace = true;
} else if (ib_table->is_encrypted) {
/* This means that tablespace was found but we could not
decrypt encrypted page. */
no_tablespace = true;
ib_table->ibd_file_missing = true;
no_tablespace = true;
}
} else {
no_tablespace = false;
}
@ -5940,27 +5953,34 @@ table_opened:
/* If table has no talespace but it has crypt data, check
is tablespace made unaccessible because encryption service
or used key_id is not available. */
if (ib_table) {
fil_space_crypt_t* crypt_data = ib_table->crypt_data;
if (encrypted) {
bool warning_pushed = false;
if (crypt_data && crypt_data->should_encrypt()) {
char buf[MAX_FULL_NAME_LEN];
ut_format_name(ib_table->name, TRUE, buf, sizeof(buf));
if (!encryption_key_id_exists(crypt_data->key_id)) {
push_warning_printf(thd, Sql_condition::WARN_LEVEL_WARN,
HA_ERR_DECRYPTION_FAILED,
"Table %s is encrypted but encryption service or"
" used key_id %u is not available. "
" Can't continue reading table.",
ib_table->name, crypt_data->key_id);
ret_err = HA_ERR_DECRYPTION_FAILED;
}
} else if (ib_table->is_encrypted) {
if (!encryption_key_id_exists(space()->crypt_data->key_id)) {
push_warning_printf(thd, Sql_condition::WARN_LEVEL_WARN,
HA_ERR_DECRYPTION_FAILED,
"Table %s is encrypted but encryption service or"
"Table %s in file %s is encrypted but encryption service or"
" used key_id %u is not available. "
" Can't continue reading table.",
buf, space()->chain.start->name,
space()->crypt_data->key_id);
ret_err = HA_ERR_DECRYPTION_FAILED;
warning_pushed = true;
}
/* If table is marked as encrypted then we push
warning if it has not been already done as used
key_id might be found but it is incorrect. */
if (!warning_pushed) {
push_warning_printf(thd, Sql_condition::WARN_LEVEL_WARN,
HA_ERR_DECRYPTION_FAILED,
"Table %s in file %s is encrypted but encryption service or"
" used key_id is not available. "
" Can't continue reading table.",
ib_table->name);
buf, space()->chain.start->name);
ret_err = HA_ERR_DECRYPTION_FAILED;
}
}
@ -6096,7 +6116,7 @@ table_opened:
/* Only if the table has an AUTOINC column. */
if (prebuilt->table != NULL
&& !prebuilt->table->ibd_file_missing
&& prebuilt->table->is_readable()
&& table->found_next_number_field != NULL) {
dict_table_autoinc_lock(prebuilt->table);
@ -9595,6 +9615,21 @@ ha_innobase::general_fetch(
ut_a(prebuilt->trx == thd_to_trx(user_thd));
if (prebuilt->table->is_readable()) {
} else {
if (prebuilt->table->corrupted) {
DBUG_RETURN(HA_ERR_CRASHED);
} else {
FilSpace space(prebuilt->table->space, true);
if (space()) {
DBUG_RETURN(HA_ERR_DECRYPTION_FAILED);
} else {
DBUG_RETURN(HA_ERR_NO_SUCH_TABLE);
}
}
}
innobase_srv_conc_enter_innodb(prebuilt->trx);
ret = row_search_for_mysql(
@ -12337,7 +12372,7 @@ ha_innobase::discard_or_import_tablespace(
user may want to set the DISCARD flag in order to IMPORT
a new tablespace. */
if (dict_table->ibd_file_missing) {
if (!dict_table->is_readable()) {
ib_senderrf(
prebuilt->trx->mysql_thd,
IB_LOG_LEVEL_WARN, ER_TABLESPACE_MISSING,
@ -12347,7 +12382,7 @@ ha_innobase::discard_or_import_tablespace(
err = row_discard_tablespace_for_mysql(
dict_table->name, prebuilt->trx);
} else if (!dict_table->ibd_file_missing) {
} else if (dict_table->is_readable()) {
/* Commit the transaction in order to
release the table lock. */
trx_commit_for_mysql(prebuilt->trx);
@ -13908,7 +13943,8 @@ ha_innobase::check(
DBUG_RETURN(HA_ADMIN_CORRUPT);
} else if (prebuilt->table->ibd_file_missing) {
} else if (prebuilt->table->file_unreadable &&
fil_space_get(prebuilt->table->space) == NULL) {
ib_senderrf(
thd, IB_LOG_LEVEL_ERROR,
@ -15032,7 +15068,7 @@ ha_innobase::transactional_table_lock(
ER_TABLESPACE_DISCARDED,
table->s->table_name.str);
} else if (prebuilt->table->ibd_file_missing) {
} else if (!prebuilt->table->is_readable()) {
ib_senderrf(
thd, IB_LOG_LEVEL_ERROR,

64
storage/innobase/handler/handler0alter.cc
View File

@ -1,7 +1,7 @@
/*****************************************************************************
Copyright (c) 2005, 2016, Oracle and/or its affiliates
Copyright (c) 2013, 2016, MariaDB Corporation. All Rights Reserved.
Copyright (c) 2013, 2017, MariaDB Corporation. All Rights Reserved.
This program is free software; you can redistribute it and/or modify it under
the terms of the GNU General Public License as published by the Free Software
@ -3129,7 +3129,7 @@ prepare_inplace_alter_table_dict(
clustered index of the old table, later. */
if (new_clustered
|| !ctx->online
|| user_table->ibd_file_missing
|| !user_table->is_readable()
|| dict_table_is_discarded(user_table)) {
/* No need to allocate a modification log. */
ut_ad(!ctx->add_index[a]->online_log);
@ -3540,6 +3540,43 @@ ha_innobase::prepare_inplace_alter_table(
goto func_exit;
}
indexed_table = prebuilt->table;
if (indexed_table->is_readable()) {
} else {
if (indexed_table->corrupted) {
/* Handled below */
} else {
FilSpace space(indexed_table->space, true);
if (space()) {
String str;
const char* engine= table_type();
char buf[MAX_FULL_NAME_LEN];
ut_format_name(indexed_table->name, TRUE, buf, sizeof(buf));
push_warning_printf(user_thd, Sql_condition::WARN_LEVEL_WARN,
HA_ERR_DECRYPTION_FAILED,
"Table %s in file %s is encrypted but encryption service or"
" used key_id is not available. "
" Can't continue reading table.",
buf, space()->chain.start->name);
my_error(ER_GET_ERRMSG, MYF(0), HA_ERR_DECRYPTION_FAILED, str.c_ptr(), engine);
DBUG_RETURN(true);
}
}
}
if (indexed_table->corrupted
|| dict_table_get_first_index(indexed_table) == NULL
|| dict_index_is_corrupted(
dict_table_get_first_index(indexed_table))) {
/* The clustered index is corrupted. */
my_error(ER_CHECK_NO_SUCH_TABLE, MYF(0));
DBUG_RETURN(true);
}
if (ha_alter_info->handler_flags
& Alter_inplace_info::CHANGE_CREATE_OPTION) {
/* Check engine specific table options */
@ -4192,7 +4229,7 @@ ok_exit:
DBUG_ASSERT(ctx->trx);
DBUG_ASSERT(ctx->prebuilt == prebuilt);
if (prebuilt->table->ibd_file_missing
if (prebuilt->table->file_unreadable
|| dict_table_is_discarded(prebuilt->table)) {
goto all_done;
}
@ -5214,7 +5251,7 @@ commit_try_rebuild(
/* The new table must inherit the flag from the
"parent" table. */
if (dict_table_is_discarded(user_table)) {
rebuilt_table->ibd_file_missing = true;
rebuilt_table->file_unreadable = true;
rebuilt_table->flags2 |= DICT_TF2_DISCARDED;
}
@ -5744,9 +5781,9 @@ ha_innobase::commit_inplace_alter_table(
if (ha_alter_info->group_commit_ctx) {
ctx_array = ha_alter_info->group_commit_ctx;
} else {
ctx_single[0] = ctx0;
ctx_single[1] = NULL;
ctx_array = ctx_single;
ctx_single[0] = ctx0;
ctx_single[1] = NULL;
ctx_array = ctx_single;
}
DBUG_ASSERT(ctx0 == ctx_array[0]);
@ -5775,6 +5812,19 @@ ha_innobase::commit_inplace_alter_table(
= static_cast<ha_innobase_inplace_ctx*>(*pctx);
DBUG_ASSERT(ctx->prebuilt->trx == prebuilt->trx);
/* If decryption failed for old table or new table
fail here. */
if ((ctx->old_table->file_unreadable &&
fil_space_get(ctx->old_table->space) != NULL)||
(ctx->new_table->file_unreadable &&
fil_space_get(ctx->new_table->space) != NULL)) {
String str;
const char* engine= table_type();
get_error_message(HA_ERR_DECRYPTION_FAILED, &str);
my_error(ER_GET_ERRMSG, MYF(0), HA_ERR_DECRYPTION_FAILED, str.c_ptr(), engine);
DBUG_RETURN(true);
}
/* Exclusively lock the table, to ensure that no other
transaction is holding locks on the table while we
change the table definition. The MySQL meta-data lock

2
storage/innobase/include/btr0btr.ic
View File

@ -61,7 +61,7 @@ btr_block_get_func(
if (err == DB_DECRYPTION_FAILED) {
if (index && index->table) {
index->table->is_encrypted = true;
index->table->file_unreadable = true;
}
}

29
storage/innobase/include/buf0buf.h
View File

@ -693,18 +693,6 @@ buf_page_is_corrupted(
ulint zip_size,
const fil_space_t* space)
MY_ATTRIBUTE((warn_unused_result));
/********************************************************************//**
Check if page is maybe compressed, encrypted or both when we encounter
corrupted page. Note that we can't be 100% sure if page is corrupted
or decrypt/decompress just failed.
@param[in] bpage Page
@return true if page corrupted, false if not */
bool
buf_page_check_corrupt(
buf_page_t* bpage) /*!< in/out: buffer page read from disk */
MY_ATTRIBUTE(( warn_unused_result));
/********************************************************************//**
Checks if a page is all zeroes.
@return TRUE if the page is all zeroes */
@ -1255,14 +1243,18 @@ buf_page_init_for_read(
/********************************************************************//**
Completes an asynchronous read or write request of a file page to or from
the buffer pool.
@return true if successful */
@param[in,out] bpage pointer to the block in question
@param[in] evict true if page should be evicted from LRU
@return DB_SUCCESS if page has been read and is not corrupted,
DB_PAGE_CORRUPTED if page based on checksum check is corrupted,
DB_DECRYPTION_FAILED if page post encryption checksum matches but
after decryption normal page checksum does not match.*/
UNIV_INTERN
bool
dberr_t
buf_page_io_complete(
/*=================*/
buf_page_t* bpage, /*!< in: pointer to the block in question */
bool evict = false);/*!< in: whether or not to evict
the page from LRU list. */
buf_page_t* bpage,
bool evict = false);
/********************************************************************//**
Calculates a folded value of a file page address to use in the page hash
table.
@ -1644,7 +1636,6 @@ struct buf_page_t{
if written again we check is TRIM
operation needed. */
unsigned key_version; /*!< key version for this block */
bool encrypted; /*!< page is still encrypted */
ulint real_size; /*!< Real size of the page

34
storage/innobase/include/buf0rea.h
View File

@ -1,7 +1,7 @@
/*****************************************************************************
Copyright (c) 1995, 2013, Oracle and/or its affiliates. All Rights Reserved.
Copyright (c) 2015, MariaDB Corporation.
Copyright (c) 2015, 2017, MariaDB Corporation.
This program is free software; you can redistribute it and/or modify it under
the terms of the GNU General Public License as published by the Free Software
@ -35,27 +35,35 @@ High-level function which reads a page asynchronously from a file to the
buffer buf_pool if it is not already there. Sets the io_fix flag and sets
an exclusive lock on the buffer frame. The flag is cleared and the x-lock
released by the i/o-handler thread.
@return TRUE if page has been read in, FALSE in case of failure */
@param[in] space space_id
@param[in] zip_size compressed page size in bytes, or 0
@param[in] offset page number
@return DB_SUCCESS if page has been read and is not corrupted,
@retval DB_PAGE_CORRUPTED if page based on checksum check is corrupted,
@retval DB_DECRYPTION_FAILED if page post encryption checksum matches but
after decryption normal page checksum does not match.
@retval DB_TABLESPACE_DELETED if tablespace .ibd file is missing */
UNIV_INTERN
ibool
dberr_t
buf_read_page(
/*==========*/
ulint space, /*!< in: space id */
ulint zip_size,/*!< in: compressed page size in bytes, or 0 */
ulint offset, /*!< in: page number */
buf_page_t** bpage);/*!< out: page */
ulint space,
ulint zip_size,
ulint offset);
/********************************************************************//**
High-level function which reads a page asynchronously from a file to the
buffer buf_pool if it is not already there. Sets the io_fix flag and sets
an exclusive lock on the buffer frame. The flag is cleared and the x-lock
released by the i/o-handler thread.
@return TRUE if page has been read in, FALSE in case of failure */
@param[in] space Tablespace id
@param[in] offset Page number */
UNIV_INTERN
ibool
void
buf_read_page_async(
/*================*/
ulint space, /*!< in: space id */
ulint offset);/*!< in: page number */
ulint space,
ulint offset);
/********************************************************************//**
Applies a random read-ahead in buf_pool if there are at least a threshold
value of accessed pages from the random read-ahead area. Does not read any

2
storage/innobase/include/db0err.h
View File

@ -138,6 +138,8 @@ enum dberr_t {
of missing key management plugin,
or missing or incorrect key or
incorret AES method or algorithm. */
DB_PAGE_CORRUPTED, /* Page read from tablespace is
corrupted. */
/* The following are partial failure codes */
DB_FAIL = 1000,
DB_OVERFLOW,

9
storage/innobase/include/dict0dict.h
View File

@ -1822,6 +1822,15 @@ dict_set_corrupted_by_space(
/*========================*/
ulint space_id); /*!< in: space ID */
/**********************************************************************//**
Flags a table with specified space_id encrypted in the data dictionary
cache
@param[in] space_id Tablespace id */
UNIV_INTERN
void
dict_set_encrypted_by_space(
ulint space_id);
/********************************************************************//**
Validate the table flags.
@return true if valid. */

2
storage/innobase/include/dict0dict.ic
View File

@ -1564,7 +1564,7 @@ dict_index_is_corrupted(
ut_ad(index->magic_n == DICT_INDEX_MAGIC_N);
return((index->type & DICT_CORRUPT)
|| (index->table && index->table->corrupted));
|| (index->table && index->table->corrupted));
}
/********************************************************************//**

33
storage/innobase/include/dict0mem.h
View File

@ -2,7 +2,7 @@
Copyright (c) 1996, 2016, Oracle and/or its affiliates. All Rights Reserved.
Copyright (c) 2012, Facebook Inc.
Copyright (c) 2013, 2016, MariaDB Corporation.
Copyright (c) 2013, 2017, MariaDB Corporation.
This program is free software; you can redistribute it and/or modify it under
the terms of the GNU General Public License as published by the Free Software
@ -779,6 +779,9 @@ struct dict_index_t{
to first_blob_page_no; protected by
blobs_mutex; @see btr_blob_dbg_t */
#endif /* UNIV_BLOB_DEBUG */
bool is_readable() const;
#ifdef UNIV_DEBUG
ulint magic_n;/*!< magic number */
/** Value of dict_index_t::magic_n */
@ -1046,9 +1049,10 @@ struct dict_table_t{
table is placed */
unsigned flags:DICT_TF_BITS; /*!< DICT_TF_... */
unsigned flags2:DICT_TF2_BITS; /*!< DICT_TF2_... */
unsigned ibd_file_missing:1;
/*!< TRUE if this is in a single-table
tablespace and the .ibd file is missing; then
unsigned file_unreadable:1;
/*!< true if this is in a single-table
tablespace and the .ibd file is missing or
page decryption failed and page is corrupted; then
we must return in ha_innodb.cc an error if the
user tries to query such an orphaned table */
unsigned cached:1;/*!< TRUE if the table object has been added
@ -1349,7 +1353,16 @@ struct dict_table_t{
locks; /*!< list of locks on the table; protected
by lock_sys->mutex */
#endif /* !UNIV_HOTBACKUP */
ibool is_encrypted;
/* Returns true if this is a single-table tablespace
and the .ibd file is missing or page decryption failed
and/or page is corrupted.
@return true if table is readable
@retval false if table is not readable */
inline bool is_readable() const
{
return(UNIV_LIKELY(!file_unreadable));
}
#ifdef UNIV_DEBUG
ulint magic_n;/*!< magic number */
@ -1358,6 +1371,16 @@ struct dict_table_t{
#endif /* UNIV_DEBUG */
};
/* Returns true if this is a single-table tablespace
and the .ibd file is missing or page decryption failed
and/or page is corrupted.
@return true if table is readable
@retval false if table is not readable */
inline bool dict_index_t::is_readable() const
{
return(UNIV_LIKELY(!table->file_unreadable));
}
/** A function object to add the foreign key constraint to the referenced set
of the referenced table, if it exists in the dictionary cache. */
struct dict_foreign_add_to_referenced_table {

13
storage/innobase/include/fil0crypt.h
View File

@ -117,8 +117,7 @@ struct fil_space_crypt_t : st_encryption_scheme
min_key_version(new_min_key_version),
page0_offset(0),
encryption(new_encryption),
mutex(),
key_found(new_min_key_version),
key_found(0),
rotate_state()
{
key_id = new_key_id;
@ -136,6 +135,8 @@ struct fil_space_crypt_t : st_encryption_scheme
type = CRYPT_SCHEME_1;
min_key_version = key_get_latest_version();
}
key_found = min_key_version;
}
/** Destructor */
@ -298,13 +299,15 @@ Parse a MLOG_FILE_WRITE_CRYPT_DATA log entry
@param[in] ptr Log entry start
@param[in] end_ptr Log entry end
@param[in] block buffer block
@param[out] err DB_SUCCESS or DB_DECRYPTION_FAILED
@return position on log buffer */
UNIV_INTERN
const byte*
byte*
fil_parse_write_crypt_data(
const byte* ptr,
byte* ptr,
const byte* end_ptr,
const buf_block_t* block)
const buf_block_t* block,
dberr_t* err)
MY_ATTRIBUTE((warn_unused_result));
/******************************************************************

139
storage/innobase/include/fil0fil.h
View File

@ -645,23 +645,45 @@ fil_write_flushed_lsn_to_data_files(
Used by background threads that do not necessarily hold proper locks
for concurrency control.
@param[in] id tablespace ID
@param[in] silent whether to silently ignore missing tablespaces
@param[in] for_io whether to look up the tablespace while performing I/O
(possibly executing TRUNCATE)
@return the tablespace
@retval NULL if missing or being deleted or truncated */
UNIV_INTERN
fil_space_t*
fil_space_acquire_low(ulint id, bool silent, bool for_io = false)
MY_ATTRIBUTE((warn_unused_result));
/** Acquire a tablespace when it could be dropped concurrently.
Used by background threads that do not necessarily hold proper locks
for concurrency control.
@param[in] id tablespace ID
@param[in] for_io whether to look up the tablespace while performing I/O
(possibly executing TRUNCATE)
@return the tablespace
@retval NULL if missing or being deleted or truncated */
inline
fil_space_t*
fil_space_acquire(ulint id, bool for_io = false)
MY_ATTRIBUTE((warn_unused_result));
{
return (fil_space_acquire_low(id, false, for_io));
}
/** Acquire a tablespace that may not exist.
Used by background threads that do not necessarily hold proper locks
for concurrency control.
@param[in] id tablespace ID
@param[in] for_io whether to look up the tablespace while performing I/O
(possibly executing TRUNCATE)
@return the tablespace
@retval NULL if missing or being deleted */
inline
fil_space_t*
fil_space_acquire_silent(ulint id)
MY_ATTRIBUTE((warn_unused_result));
fil_space_acquire_silent(ulint id, bool for_io = false)
{
return (fil_space_acquire_low(id, true, for_io));
}
/** Release a tablespace acquired with fil_space_acquire().
@param[in,out] space tablespace to release */
@ -694,6 +716,64 @@ fil_space_keyrotate_next(
fil_space_t* prev_space)
MY_ATTRIBUTE((warn_unused_result));
/** Wrapper with reference-counting for a fil_space_t. */
class FilSpace
{
public:
/** Default constructor: Use this when reference counting
is done outside this wrapper. */
FilSpace() : m_space(NULL) {}
/** Constructor: Look up the tablespace and increment the
reference count if found.
@param[in] space_id tablespace ID
@param[in] silent whether not print any errors
@param[in] for_io whether to look up the tablespace
while performing I/O
(possibly executing TRUNCATE) */
explicit FilSpace(ulint space_id, bool silent = false, bool for_io = false)
: m_space(fil_space_acquire_low(space_id, silent, for_io)) {}
/** Assignment operator: This assumes that fil_space_acquire()
has already been done for the fil_space_t. The caller must
assign NULL if it calls fil_space_release().
@param[in] space tablespace to assign */
class FilSpace& operator=(fil_space_t* space)
{
/* fil_space_acquire() must have been invoked. */
ut_ad(space == NULL || space->n_pending_ops > 0);
m_space = space;
return(*this);
}
/** Destructor - Decrement the reference count if a fil_space_t
is still assigned. */
~FilSpace()
{
if (m_space != NULL) {
fil_space_release(m_space);
}
}
/** Implicit type conversion
@return the wrapped object */
operator const fil_space_t*() const
{
return(m_space);
}
/** Explicit type conversion
@return the wrapped object */
const fil_space_t* operator()() const
{
return(m_space);
}
private:
/** The wrapped pointer */
fil_space_t* m_space;
};
/*******************************************************************//**
Reads the flushed lsn, arch no, and tablespace flag fields from a data
file at database startup.
@ -722,22 +802,6 @@ fil_read_first_page(
fil_space_crypt_t** crypt_data) /*!< out: crypt data */
__attribute__((warn_unused_result));
/*******************************************************************//**
Increments the count of pending operation, if space is not being deleted.
@return TRUE if being deleted, and operation should be skipped */
UNIV_INTERN
ibool
fil_inc_pending_ops(
/*================*/
ulint id, /*!< in: space id */
ibool print_err); /*!< in: need to print error or not */
/*******************************************************************//**
Decrements the count of pending operations. */
UNIV_INTERN
void
fil_decr_pending_ops(
/*=================*/
ulint id); /*!< in: space id */
#endif /* !UNIV_HOTBACKUP */
/*******************************************************************//**
Parses the body of a log record written about an .ibd file operation. That is,
@ -1382,13 +1446,6 @@ fil_space_get(
#define fil_system_exit() mutex_exit(&fil_system->mutex)
#ifndef UNIV_INNOCHECKSUM
/*******************************************************************//**
Returns the table space by a given id, NULL if not found. */
fil_space_t*
fil_space_found_by_id(
/*==================*/
ulint id); /*!< in: space id */
/*******************************************************************//**
Returns the table space by a given id, NULL if not found. */
fil_space_t*
@ -1396,36 +1453,6 @@ fil_space_get_by_id(
/*================*/
ulint id); /*!< in: space id */
/******************************************************************
Get id of first tablespace or ULINT_UNDEFINED if none */
UNIV_INTERN
ulint
fil_get_first_space();
/*=================*/
/******************************************************************
Get id of next tablespace or ULINT_UNDEFINED if none */
UNIV_INTERN
ulint
fil_get_next_space(
/*===============*/
ulint id); /*!< in: space id */
/******************************************************************
Get id of first tablespace that has node or ULINT_UNDEFINED if none */
UNIV_INTERN
ulint
fil_get_first_space_safe();
/*======================*/
/******************************************************************
Get id of next tablespace that has node or ULINT_UNDEFINED if none */
UNIV_INTERN
ulint
fil_get_next_space_safe(
/*====================*/
ulint id); /*!< in: previous space id */
/** Determine the block size of the data file.
@param[in] space tablespace
@param[in] offset page number

17
storage/innobase/log/log0recv.cc
View File

@ -1297,7 +1297,12 @@ recv_parse_or_apply_log_rec_body(
}
break;
case MLOG_FILE_WRITE_CRYPT_DATA:
ptr = const_cast<byte*>(fil_parse_write_crypt_data(ptr, end_ptr, block));
dberr_t err;
ptr = const_cast<byte*>(fil_parse_write_crypt_data(ptr, end_ptr, block, &err));
if (err != DB_SUCCESS) {
recv_sys->found_corrupt_log = TRUE;
}
break;
default:
ptr = NULL;
@ -1769,6 +1774,7 @@ recv_read_in_area(
/** Apply the hash table of stored log records to persistent data pages.
@param[in] last_batch whether the change buffer merge will be
performed as part of the operation */
UNIV_INTERN
void
recv_apply_hashed_log_recs(bool last_batch)
@ -1780,6 +1786,11 @@ recv_apply_hashed_log_recs(bool last_batch)
break;
}
if (recv_sys->found_corrupt_log) {
mutex_exit(&recv_sys->mutex);
return;
}
mutex_exit(&recv_sys->mutex);
os_thread_sleep(500000);
}
@ -1844,6 +1855,10 @@ recv_apply_hashed_log_recs(bool last_batch)
mutex_exit(&(recv_sys->mutex));
if (recv_sys->found_corrupt_log) {
return;
}
os_thread_sleep(500000);
mutex_enter(&(recv_sys->mutex));

11
storage/innobase/row/row0import.cc
View File

@ -2029,15 +2029,12 @@ PageConverter::validate(
buf_block_t* block) UNIV_NOTHROW
{
buf_frame_t* page = get_frame(block);
ulint space_id = mach_read_from_4(
page + FIL_PAGE_ARCH_LOG_NO_OR_SPACE_ID);
fil_space_t* space = fil_space_found_by_id(space_id);
/* Check that the page number corresponds to the offset in
the file. Flag as corrupt if it doesn't. Disable the check
for LSN in buf_page_is_corrupted() */
if (buf_page_is_corrupted(false, page, get_zip_size(), space)
if (buf_page_is_corrupted(false, page, get_zip_size(), NULL)
|| (page_get_page_no(page) != offset / m_page_size
&& page_get_page_no(page) != 0)) {
@ -2200,7 +2197,7 @@ row_import_discard_changes(
index->space = FIL_NULL;
}
table->ibd_file_missing = TRUE;
table->file_unreadable = true;
fil_close_tablespace(trx, table->space);
}
@ -3446,7 +3443,7 @@ row_import_for_mysql(
ut_a(table->space);
ut_ad(prebuilt->trx);
ut_a(table->ibd_file_missing);
ut_a(table->file_unreadable);
trx_start_if_not_started(prebuilt->trx);
@ -3751,7 +3748,7 @@ row_import_for_mysql(
return(row_import_error(prebuilt, trx, err));
}
table->ibd_file_missing = false;
table->file_unreadable = false;
table->flags2 &= ~DICT_TF2_DISCARDED;
if (autoinc != 0) {

7
storage/innobase/row/row0ins.cc
View File

@ -1512,7 +1512,7 @@ run_again:
}
if (check_table == NULL
|| check_table->ibd_file_missing
|| check_table->file_unreadable
|| check_index == NULL) {
if (!srv_read_only_mode && check_ref) {
@ -2375,8 +2375,7 @@ row_ins_clust_index_entry_low(
&cursor, 0, __FILE__, __LINE__, &mtr);
if (err != DB_SUCCESS) {
index->table->is_encrypted = true;
index->table->ibd_file_missing = true;
index->table->file_unreadable = true;
mtr_commit(&mtr);
goto func_exit;
}
@ -2698,7 +2697,7 @@ row_ins_sec_index_entry_low(
" used key_id is not available. "
" Can't continue reading table.",
index->table->name);
index->table->is_encrypted = true;
index->table->file_unreadable = true;
}
goto func_exit;
}

12
storage/innobase/row/row0merge.cc
View File

@ -1614,7 +1614,8 @@ row_merge_read_clustered_index(
page_cur_t* cur = btr_pcur_get_page_cur(&pcur);
/* Do not continue if table pages are still encrypted */
if (old_table->is_encrypted || new_table->is_encrypted) {
if (!old_table->is_readable() ||
!new_table->is_readable()) {
err = DB_DECRYPTION_FAILED;
trx->error_key_num = 0;
goto func_exit;
@ -3693,7 +3694,7 @@ row_merge_rename_tables_dict(
table is in a non-system tablespace where space > 0. */
if (err == DB_SUCCESS
&& old_table->space != TRX_SYS_SPACE
&& !old_table->ibd_file_missing) {
&& fil_space_get(old_table->space) != NULL) {
/* Make pathname to update SYS_DATAFILES. */
char* tmp_path = row_make_new_pathname(old_table, tmp_name);
@ -4003,6 +4004,7 @@ row_merge_build_indexes(
os_mem_alloc_large(&block_size));
if (crypt_block == NULL) {
fil_space_release(space);
DBUG_RETURN(DB_OUT_OF_MEMORY);
}
} else {
@ -4068,13 +4070,15 @@ row_merge_build_indexes(
pct_cost = COST_READ_CLUSTERED_INDEX * 100 / (total_static_cost + total_dynamic_cost);
/* Do not continue if we can't encrypt table pages */
if (old_table->is_encrypted || new_table->is_encrypted) {
if (!old_table->is_readable() ||
!new_table->is_readable()) {
error = DB_DECRYPTION_FAILED;
ib_push_warning(trx->mysql_thd, DB_DECRYPTION_FAILED,
"Table %s is encrypted but encryption service or"
" used key_id is not available. "
" Can't continue reading table.",
old_table->is_encrypted ? old_table->name : new_table->name);
!old_table->is_readable() ? old_table->name :
new_table->name);
goto func_exit;
}

141
storage/innobase/row/row0mysql.cc
View File

@ -1,6 +1,7 @@
/*****************************************************************************
Copyright (c) 2000, 2016, Oracle and/or its affiliates. All Rights Reserved.
Copyright (c) 2015, 2017, MariaDB Corporation.
This program is free software; you can redistribute it and/or modify it under
the terms of the GNU General Public License as published by the Free Software
@ -663,6 +664,7 @@ handle_new_error:
abort();
case DB_CORRUPTION:
case DB_PAGE_CORRUPTED:
fputs("InnoDB: We detected index corruption"
" in an InnoDB type table.\n"
"InnoDB: You have to dump + drop + reimport"
@ -1279,6 +1281,66 @@ run_again:
return(err);
}
/*********************************************************************//**
Determine is tablespace encrypted but decryption failed, is table corrupted
or is tablespace .ibd file missing.
@param[in] table Table
@param[in] trx Transaction
@param[in] push_warning true if we should push warning to user
@return DB_DECRYPTION_FAILED table is encrypted but decryption failed
DB_CORRUPTION table is corrupted
DB_TABLESPACE_NOT_FOUND tablespace .ibd file not found */
static
dberr_t
row_mysql_get_table_status(
const dict_table_t* table,
trx_t* trx,
bool push_warning = true)
{
dberr_t err = DB_SUCCESS;
if (fil_space_t* space = fil_space_acquire_silent(table->space)) {
if (space->crypt_data && space->crypt_data->is_encrypted()) {
// maybe we cannot access the table due to failing
// to decrypt
if (push_warning) {
ib_push_warning(trx, DB_DECRYPTION_FAILED,
"Table %s in tablespace %lu encrypted."
"However key management plugin or used key_id is not found or"
" used encryption algorithm or method does not match.",
table->name, table->space);
}
err = DB_DECRYPTION_FAILED;
} else {
if (push_warning) {
ib_push_warning(trx, DB_CORRUPTION,
"Table %s in tablespace %lu corrupted.",
table->name, table->space);
}
err = DB_CORRUPTION;
}
fil_space_release(space);
} else {
ib_logf(IB_LOG_LEVEL_ERROR,
"InnoDB: MySQL is trying to use a table handle"
" but the .ibd file for"
" table %s does not exist."
" Have you deleted the .ibd file"
" from the database directory under"
" the MySQL datadir, or have you"
" used DISCARD TABLESPACE?"
" Look from " REFMAN "innodb-troubleshooting.html"
" how you can resolve the problem.",
table->name);
err = DB_TABLESPACE_NOT_FOUND;
}
return (err);
}
/*********************************************************************//**
Does an insert for MySQL.
@return error code or DB_SUCCESS */
@ -1308,20 +1370,8 @@ row_insert_for_mysql(
return(DB_TABLESPACE_DELETED);
} else if (prebuilt->table->ibd_file_missing) {
ib_logf(IB_LOG_LEVEL_ERROR,
".ibd file is missing for table %s",
prebuilt->table->name);
return(DB_TABLESPACE_NOT_FOUND);
} else if (prebuilt->table->is_encrypted) {
ib_push_warning(trx, DB_DECRYPTION_FAILED,
"Table %s in tablespace %lu encrypted."
"However key management plugin or used key_id is not found or"
" used encryption algorithm or method does not match.",
prebuilt->table->name, prebuilt->table->space);
return(DB_DECRYPTION_FAILED);
} else if (!prebuilt->table->is_readable()) {
return (row_mysql_get_table_status(prebuilt->table, trx, true));
} else if (prebuilt->magic_n != ROW_PREBUILT_ALLOCATED) {
fprintf(stderr,
"InnoDB: Error: trying to free a corrupt\n"
@ -1707,28 +1757,8 @@ row_update_for_mysql(
ut_ad(trx != NULL);
UT_NOT_USED(mysql_rec);
if (prebuilt->table->ibd_file_missing) {
ut_print_timestamp(stderr);
fprintf(stderr, " InnoDB: Error:\n"
"InnoDB: MySQL is trying to use a table handle"
" but the .ibd file for\n"
"InnoDB: table %s does not exist.\n"
"InnoDB: Have you deleted the .ibd file"
" from the database directory under\n"
"InnoDB: the MySQL datadir, or have you"
" used DISCARD TABLESPACE?\n"
"InnoDB: Look from\n"
"InnoDB: " REFMAN "innodb-troubleshooting.html\n"
"InnoDB: how you can resolve the problem.\n",
prebuilt->table->name);
return(DB_ERROR);
} else if (prebuilt->table->is_encrypted) {
ib_push_warning(trx, DB_DECRYPTION_FAILED,
"Table %s in tablespace %lu encrypted."
"However key management plugin or used key_id is not found or"
" used encryption algorithm or method does not match.",
prebuilt->table->name, prebuilt->table->space);
return (DB_TABLE_NOT_FOUND);
if (!table->is_readable()) {
return (row_mysql_get_table_status(table, trx, true));
}
if (UNIV_UNLIKELY(prebuilt->magic_n != ROW_PREBUILT_ALLOCATED)) {
@ -3107,7 +3137,7 @@ row_discard_tablespace(
/* All persistent operations successful, update the
data dictionary memory cache. */
table->ibd_file_missing = TRUE;
table->file_unreadable = true;
table->flags2 |= DICT_TF2_DISCARDED;
@ -3164,8 +3194,6 @@ row_discard_tablespace_for_mysql(
if (table == 0) {
err = DB_TABLE_NOT_FOUND;
} else if (table->is_encrypted) {
err = DB_DECRYPTION_FAILED;
} else if (table->space == TRX_SYS_SPACE) {
char table_name[MAX_FULL_NAME_LEN + 1];
@ -3380,10 +3408,8 @@ row_truncate_table_for_mysql(
if (dict_table_is_discarded(table)) {
return(DB_TABLESPACE_DELETED);
} else if (table->is_encrypted) {
return(DB_DECRYPTION_FAILED);
} else if (table->ibd_file_missing) {
return(DB_TABLESPACE_NOT_FOUND);
} else if (!table->is_readable()) {
return (row_mysql_get_table_status(table, trx, true));
}
trx_start_for_ddl(trx, TRX_DICT_OP_TABLE);
@ -3549,7 +3575,7 @@ row_truncate_table_for_mysql(
"create a new tablespace",
table->name);
table->ibd_file_missing = 1;
table->file_unreadable = true;
err = DB_ERROR;
goto funct_exit;
}
@ -3967,19 +3993,6 @@ row_drop_table_for_mysql(
goto funct_exit;
}
/* If table is encrypted and table page encryption failed
return error. */
if (table->is_encrypted) {
if (table->can_be_evicted) {
dict_table_move_from_lru_to_non_lru(table);
}
dict_table_close(table, TRUE, FALSE);
err = DB_DECRYPTION_FAILED;
goto funct_exit;
}
/* Turn on this drop bit before we could release the dictionary
latch */
table->to_be_dropped = true;
@ -4351,7 +4364,7 @@ row_drop_table_for_mysql(
from table->heap, which will be freed by
dict_table_remove_from_cache(table) below. */
space_id = table->space;
ibd_file_missing = table->ibd_file_missing;
ibd_file_missing = table->file_unreadable;
table_flags = table->flags;
is_temp = DICT_TF2_FLAG_IS_SET(table, DICT_TF2_TEMPORARY);
@ -4362,7 +4375,8 @@ row_drop_table_for_mysql(
have a temp flag but not know the temp path */
ut_a(table->dir_path_of_temp_table == NULL || is_temp);
if (dict_table_is_discarded(table)
|| table->ibd_file_missing) {
|| (!table->is_readable()
&& fil_space_get(table->space) == NULL)) {
/* Do not attempt to drop known-to-be-missing
tablespaces. */
space_id = 0;
@ -4773,7 +4787,9 @@ loop:
"'%s.frm' was lost.", table->name);
}
if (table->ibd_file_missing) {
if (!table->is_readable()
&& fil_space_get(table->space) == NULL) {
ib_logf(IB_LOG_LEVEL_WARN,
"Missing %s.ibd file for table %s.",
table->name, table->name);
@ -5047,7 +5063,8 @@ row_rename_table_for_mysql(
stderr);
goto funct_exit;
} else if (table->ibd_file_missing
} else if (!table->is_readable()
&& fil_space_get(table->space) == NULL
&& !dict_table_is_discarded(table)) {
err = DB_TABLE_NOT_FOUND;
@ -5117,7 +5134,7 @@ row_rename_table_for_mysql(
which have space IDs > 0. */
if (err == DB_SUCCESS
&& table->space != TRX_SYS_SPACE
&& !table->ibd_file_missing) {
&& table->is_readable()) {
/* Make a new pathname to update SYS_DATAFILES. */
char* new_path = row_make_new_pathname(table, new_name);

2
storage/innobase/row/row0purge.cc
View File

@ -777,7 +777,7 @@ row_purge_parse_undo_rec(
goto err_exit;
}
if (node->table->ibd_file_missing) {
if (node->table->file_unreadable) {
/* We skip purge of missing .ibd files */
dict_table_close(node->table, FALSE, FALSE);

55
storage/innobase/row/row0sel.cc
View File

@ -2,7 +2,7 @@
Copyright (c) 1997, 2016, Oracle and/or its affiliates. All Rights Reserved.
Copyright (c) 2008, Google Inc.
Copyright (c) 2015, MariaDB Corporation.
Copyright (c) 2015, 2017, MariaDB Corporation.
Portions of this file contain modifications contributed and copyrighted by
Google, Inc. Those modifications are gratefully acknowledged and are described
@ -3714,13 +3714,12 @@ row_search_for_mysql(
return(DB_TABLESPACE_DELETED);
} else if (prebuilt->table->ibd_file_missing) {
return(DB_TABLESPACE_NOT_FOUND);
} else if (prebuilt->table->is_encrypted) {
return(DB_DECRYPTION_FAILED);
} else if (!prebuilt->table->is_readable()) {
if (fil_space_get(prebuilt->table->space) == NULL) {
return(DB_TABLESPACE_NOT_FOUND);
} else {
return(DB_DECRYPTION_FAILED);
}
} else if (!prebuilt->index_usable) {
return(DB_MISSING_HISTORY);
@ -4200,7 +4199,7 @@ wait_table_again:
" used key_id is not available. "
" Can't continue reading table.",
prebuilt->table->name);
index->table->is_encrypted = true;
index->table->file_unreadable = true;
}
rec = NULL;
goto lock_wait_or_error;
@ -4220,7 +4219,7 @@ rec_loop:
rec = btr_pcur_get_rec(pcur);
if (!rec) {
if (!index->table->is_readable()) {
err = DB_DECRYPTION_FAILED;
goto lock_wait_or_error;
}
@ -4314,13 +4313,15 @@ wrong_offs:
(void*) rec, (ulong)
btr_cur_get_block(btr_pcur_get_btr_cur(pcur))
->page.buf_fix_count);
fprintf(stderr,
"InnoDB: Index corruption: rec offs %lu"
" next offs %lu, page no %lu,\n"
ib_logf(IB_LOG_LEVEL_ERROR,
"Index corruption: rec offs " ULINTPF
" next offs " ULINTPF
", page no " ULINTPF " ,"
"InnoDB: ",
(ulong) page_offset(rec),
(ulong) next_offs,
(ulong) page_get_page_no(page_align(rec)));
page_offset(rec),
next_offs,
page_get_page_no(page_align(rec)));
dict_index_name_print(stderr, trx, index);
fputs(". Run CHECK TABLE. You may need to\n"
"InnoDB: restore from a backup, or"
@ -4362,16 +4363,18 @@ wrong_offs:
if (UNIV_UNLIKELY(srv_force_recovery > 0)) {
if (!rec_validate(rec, offsets)
|| !btr_index_rec_validate(rec, index, FALSE)) {
fprintf(stderr,
"InnoDB: Index corruption: rec offs %lu"
" next offs %lu, page no %lu,\n"
"InnoDB: ",
(ulong) page_offset(rec),
(ulong) next_offs,
(ulong) page_get_page_no(page_align(rec)));
dict_index_name_print(stderr, trx, index);
fputs(". We try to skip the record.\n",
stderr);
char buf[MAX_FULL_NAME_LEN];
ut_format_name(index->table->name, FALSE, buf, sizeof(buf));
ib_logf(IB_LOG_LEVEL_ERROR,
"Index %s corrupted: rec offs " ULINTPF
" next offs " ULINTPF
", page no " ULINTPF " ."
" We try to skip the record.",
buf,
page_offset(rec),
next_offs,
page_get_page_no(page_align(rec)));
goto next_rec;
}

2
storage/innobase/row/row0uins.cc
View File

@ -320,7 +320,7 @@ row_undo_ins_parse_undo_rec(
/* Skip the UNDO if we can't find the table or the .ibd file. */
if (UNIV_UNLIKELY(node->table == NULL)) {
} else if (UNIV_UNLIKELY(node->table->ibd_file_missing)) {
} else if (UNIV_UNLIKELY(node->table->file_unreadable)) {
close_table:
dict_table_close(node->table, dict_locked, FALSE);
node->table = NULL;

2
storage/innobase/row/row0umod.cc
View File

@ -1098,7 +1098,7 @@ row_undo_mod_parse_undo_rec(
return;
}
if (node->table->ibd_file_missing) {
if (node->table->file_unreadable) {
dict_table_close(node->table, dict_locked, FALSE);
/* We skip undo operations to missing .ibd files */

5
storage/innobase/srv/srv0start.cc
View File

@ -2494,6 +2494,11 @@ files_checked:
recv_group_scan_log_recs(). */
recv_apply_hashed_log_recs(true);
if (recv_sys->found_corrupt_log) {
return (DB_CORRUPTION);
}
DBUG_PRINT("ib_log", ("apply completed"));
}

2
storage/innobase/trx/trx0trx.cc
View File

@ -483,7 +483,7 @@ trx_resurrect_table_locks(
i != tables.end(); i++) {
if (dict_table_t* table = dict_table_open_on_id(
*i, FALSE, DICT_TABLE_OP_LOAD_TABLESPACE)) {
if (table->ibd_file_missing
if (table->file_unreadable
|| dict_table_is_temporary(table)) {
mutex_enter(&dict_sys->mutex);
dict_table_close(table, TRUE, FALSE);

2
storage/innobase/ut/ut0ut.cc
View File

@ -849,6 +849,8 @@ ut_strerr(
return("BLOB record length is greater than 10%% of redo log");
case DB_DECRYPTION_FAILED:
return("Table is encrypted but decrypt failed.");
case DB_PAGE_CORRUPTED:
return("Page read from tablespace is corrupted.");
/* do not add default: in order to produce a warning if new code
is added to the enum but not added here */

6
storage/xtradb/api/api0api.cc
View File

@ -247,7 +247,7 @@ ib_open_table_by_id(
table = dict_table_open_on_id(table_id, TRUE, DICT_TABLE_OP_NORMAL);
if (table != NULL && table->ibd_file_missing) {
if (table != NULL && table->file_unreadable) {
table = NULL;
}
@ -272,7 +272,7 @@ ib_open_table_by_name(
table = dict_table_open_on_name(name, FALSE, FALSE,
DICT_ERR_IGNORE_NONE);
if (table != NULL && table->ibd_file_missing) {
if (table != NULL && table->file_unreadable) {
table = NULL;
}
@ -292,7 +292,7 @@ ib_lookup_table_by_name(
table = dict_table_get_low(name);
if (table != NULL && table->ibd_file_missing) {
if (table != NULL && table->file_unreadable) {
table = NULL;
}

8
storage/xtradb/btr/btr0btr.cc
View File

@ -2,7 +2,7 @@
Copyright (c) 1994, 2016, Oracle and/or its affiliates. All Rights Reserved.
Copyright (c) 2012, Facebook Inc.
Copyright (c) 2014, 2016, MariaDB Corporation
Copyright (c) 2014, 2017, MariaDB Corporation
This program is free software; you can redistribute it and/or modify it under
the terms of the GNU General Public License as published by the Free Software
@ -744,8 +744,7 @@ btr_root_block_get(
if (!block) {
if (index && index->table) {
index->table->is_encrypted = TRUE;
index->table->corrupted = FALSE;
index->table->file_unreadable = true;
ib_push_warning(index->table->thd, DB_DECRYPTION_FAILED,
"Table %s in tablespace %lu is encrypted but encryption service or"
@ -760,6 +759,7 @@ btr_root_block_get(
SRV_CORRUPT_TABLE_CHECK(block, return(0););
btr_assert_not_corrupted(block, index);
#ifdef UNIV_BTR_DEBUG
if (!dict_index_is_ibuf(index)) {
const page_t* root = buf_block_get_frame(block);
@ -5211,7 +5211,7 @@ btr_validate_index(
page_t* root = btr_root_get(index, &mtr);
if (root == NULL && index->table->is_encrypted) {
if (root == NULL && index->table->file_unreadable) {
err = DB_DECRYPTION_FAILED;
mtr_commit(&mtr);
return err;

33
storage/xtradb/btr/btr0cur.cc
View File

@ -3,7 +3,7 @@
Copyright (c) 1994, 2016, Oracle and/or its affiliates. All Rights Reserved.
Copyright (c) 2008, Google Inc.
Copyright (c) 2012, Facebook Inc.
Copyright (c) 2015, MariaDB Corporation.
Copyright (c) 2015, 2017, MariaDB Corporation.
Portions of this file contain modifications contributed and copyrighted by
Google, Inc. Those modifications are gratefully acknowledged and are described
@ -658,7 +658,10 @@ retry_page_get:
space, zip_size, page_no, rw_latch, guess, buf_mode,
file, line, mtr, &err);
/* Note that block==NULL signifies either an error or change
buffering. */
if (err != DB_SUCCESS) {
ut_ad(block == NULL);
if (err == DB_DECRYPTION_FAILED) {
ib_push_warning((void *)NULL,
DB_DECRYPTION_FAILED,
@ -666,7 +669,7 @@ retry_page_get:
" used key_id is not available. "
" Can't continue reading table.",
index->table->name);
index->table->is_encrypted = true;
index->table->file_unreadable = true;
}
goto func_exit;
@ -944,7 +947,7 @@ btr_cur_open_at_index_side_func(
ulint offsets_[REC_OFFS_NORMAL_SIZE];
ulint* offsets = offsets_;
dberr_t err = DB_SUCCESS;
rec_offs_init(offsets_);
estimate = latch_mode & BTR_ESTIMATE;
@ -988,6 +991,9 @@ btr_cur_open_at_index_side_func(
block = buf_page_get_gen(space, zip_size, page_no,
RW_NO_LATCH, NULL, BUF_GET,
file, line, mtr, &err);
ut_ad((block != NULL) == (err == DB_SUCCESS));
if (err != DB_SUCCESS) {
if (err == DB_DECRYPTION_FAILED) {
ib_push_warning((void *)NULL,
@ -996,7 +1002,7 @@ btr_cur_open_at_index_side_func(
" used key_id is not available. "
" Can't continue reading table.",
index->table->name);
index->table->is_encrypted = true;
index->table->file_unreadable = true;
}
goto exit_loop;
@ -1161,6 +1167,8 @@ btr_cur_open_at_rnd_pos_func(
RW_NO_LATCH, NULL, BUF_GET,
file, line, mtr, &err);
ut_ad((block != NULL) == (err == DB_SUCCESS));
if (err != DB_SUCCESS) {
if (err == DB_DECRYPTION_FAILED) {
ib_push_warning((void *)NULL,
@ -1169,8 +1177,9 @@ btr_cur_open_at_rnd_pos_func(
" used key_id is not available. "
" Can't continue reading table.",
index->table->name);
index->table->is_encrypted = true;
index->table->file_unreadable = true;
}
goto exit_loop;
}
@ -3853,6 +3862,8 @@ btr_estimate_n_rows_in_range_on_level(
NULL, BUF_GET_POSSIBLY_FREED,
__FILE__, __LINE__, &mtr, &err);
ut_ad((block != NULL) == (err == DB_SUCCESS));
if (err != DB_SUCCESS) {
if (err == DB_DECRYPTION_FAILED) {
ib_push_warning((void *)NULL,
@ -3861,14 +3872,13 @@ btr_estimate_n_rows_in_range_on_level(
" used key_id is not available. "
" Can't continue reading table.",
index->table->name);
index->table->is_encrypted = true;
index->table->file_unreadable = true;
}
mtr_commit(&mtr);
goto inexact;
}
page = buf_block_get_frame(block);
/* It is possible that the tree has been reorganized in the
@ -4010,6 +4020,10 @@ btr_estimate_n_rows_in_range_low(
mtr_commit(&mtr);
if (index->table->file_unreadable) {
return (0);
}
mtr_start_trx(&mtr, trx);
#ifdef UNIV_DEBUG
@ -4360,6 +4374,11 @@ btr_estimate_number_of_different_key_vals(
because otherwise our algorithm would give a wrong estimate
for an index where there is just one key value. */
if (index->table->file_unreadable) {
mtr_commit(&mtr);
goto exit_loop;
}
page = btr_cur_get_page(&cursor);
SRV_CORRUPT_TABLE_CHECK(page, goto exit_loop;);

2
storage/xtradb/btr/btr0defragment.cc
View File

@ -227,7 +227,7 @@ btr_defragment_add_index(
page = buf_block_get_frame(block);
}
if (page == NULL && index->table->is_encrypted) {
if (page == NULL && index->table->file_unreadable) {
mtr_commit(&mtr);
*err = DB_DECRYPTION_FAILED;
return NULL;

10
storage/xtradb/btr/btr0pcur.cc
View File

@ -420,6 +420,11 @@ btr_pcur_move_to_next_page(
cursor->old_stored = BTR_PCUR_OLD_NOT_STORED;
page = btr_pcur_get_page(cursor);
if (UNIV_UNLIKELY(!page)) {
return;
}
next_page_no = btr_page_get_next(page, mtr);
space = buf_block_get_space(btr_pcur_get_block(cursor));
zip_size = buf_block_get_zip_size(btr_pcur_get_block(cursor));
@ -429,6 +434,11 @@ btr_pcur_move_to_next_page(
next_block = btr_block_get(space, zip_size, next_page_no,
cursor->latch_mode,
btr_pcur_get_btr_cur(cursor)->index, mtr);
if (UNIV_UNLIKELY(!next_block)) {
return;
}
next_page = buf_block_get_frame(next_block);
SRV_CORRUPT_TABLE_CHECK(next_page,

3
storage/xtradb/btr/btr0scrub.cc
View File

@ -139,6 +139,7 @@ btr_scrub_lock_dict_func(ulint space_id, bool lock_to_close_table,
} else {
return false;
}
os_thread_sleep(250000);
time_t now = time(0);
@ -577,7 +578,7 @@ btr_scrub_table_needs_scrubbing(
return false;
}
if (table->corrupted) {
if (!table->is_readable()) {
return false;
}

461
storage/xtradb/buf/buf0buf.cc
View File

@ -1181,7 +1181,6 @@ buf_block_init(
block->page.buf_fix_count = 0;
block->page.io_fix = BUF_IO_NONE;
block->page.encrypted = false;
block->page.key_version = 0;
block->page.real_size = 0;
block->page.write_size = 0;
block->modify_clock = 0;
@ -2351,7 +2350,17 @@ lookup:
/* Page not in buf_pool: needs to be read from file */
ut_ad(!hash_lock);
buf_read_page(space, zip_size, offset, trx, NULL);
dberr_t err = buf_read_page(space, zip_size, offset, trx);
if (err != DB_SUCCESS) {
ib_logf(IB_LOG_LEVEL_ERROR,
"Reading compressed page " ULINTPF
":" ULINTPF
" failed with error: %s.",
space, offset, ut_strerr(err));
goto err_exit;
}
#if defined UNIV_DEBUG || defined UNIV_BUF_DEBUG
ut_a(++buf_dbg_counter % 5771 || buf_validate());
@ -2957,7 +2966,6 @@ loop:
}
if (block == NULL) {
buf_page_t* bpage=NULL;
/* Page not in buf_pool: needs to be read from file */
@ -2993,7 +3001,18 @@ loop:
return(NULL);
}
if (buf_read_page(space, zip_size, offset, trx, &bpage)) {
/* Call path is buf_read_page() -> buf_read_page_low()
(_fil_io()) -> buf_page_io_complete() ->
buf_decrypt_after_read() here fil_space_t* is used
and we decrypt -> buf_page_check_corrupt() where
page checksums are compared. Decryption/decompression
is handled lower level, error handling is handled on lower
level, here we need only to know is page really corrupted
or encrypted page with correct checksum. */
dberr_t local_err = buf_read_page(space, zip_size, offset, trx);
if (local_err == DB_SUCCESS) {
buf_read_ahead_random(space, zip_size, offset,
ibuf_inside(mtr), trx);
@ -3001,89 +3020,47 @@ loop:
} else if (retries < BUF_PAGE_READ_MAX_RETRIES) {
++retries;
bool corrupted = false;
if (bpage) {
corrupted = buf_page_check_corrupt(bpage);
}
/* Do not try again for encrypted pages */
if (corrupted && bpage->encrypted) {
ib_mutex_t* pmutex = buf_page_get_mutex(bpage);
mutex_enter(&buf_pool->LRU_list_mutex);
mutex_enter(pmutex);
ut_ad(buf_pool->n_pend_reads > 0);
os_atomic_decrement_ulint(&buf_pool->n_pend_reads, 1);
buf_page_set_io_fix(bpage, BUF_IO_NONE);
if (!buf_LRU_free_page(bpage, true)) {
mutex_exit(&buf_pool->LRU_list_mutex);
}
mutex_exit(pmutex);
rw_lock_x_unlock_gen(&((buf_block_t*) bpage)->lock,
BUF_IO_READ);
if (err) {
*err = DB_DECRYPTION_FAILED;
}
return (NULL);
}
DBUG_EXECUTE_IF(
"innodb_page_corruption_retries",
retries = BUF_PAGE_READ_MAX_RETRIES;
);
} else {
bool corrupted = false;
if (bpage) {
corrupted = buf_page_check_corrupt(bpage);
if (err) {
*err = local_err;
}
if (corrupted && !bpage->encrypted) {
ib_logf(IB_LOG_LEVEL_ERROR, "Unable"
" to read tablespace %lu page no"
" %lu into the buffer pool after"
" %lu attempts\n"
"InnoDB: The most probable cause"
" of this error may be that the"
" table has been corrupted.\n"
"InnoDB: You can try to fix this"
" problem by using"
" innodb_force_recovery.\n"
"InnoDB: Please see reference manual"
" for more details.\n"
"InnoDB: Aborting...\n",
space, offset,
BUF_PAGE_READ_MAX_RETRIES);
ut_error;
} else {
ib_mutex_t* pmutex = buf_page_get_mutex(bpage);
mutex_enter(&buf_pool->LRU_list_mutex);
mutex_enter(pmutex);
ut_ad(buf_pool->n_pend_reads > 0);
os_atomic_decrement_ulint(&buf_pool->n_pend_reads, 1);
buf_page_set_io_fix(bpage, BUF_IO_NONE);
if (!buf_LRU_free_page(bpage, true)) {
mutex_exit(&buf_pool->LRU_list_mutex);
}
mutex_exit(pmutex);
rw_lock_x_unlock_gen(&((buf_block_t*) bpage)->lock,
BUF_IO_READ);
if (err) {
*err = DB_DECRYPTION_FAILED;
}
/* Pages whose encryption key is unavailable or used
key, encryption algorithm or encryption method is
incorrect are marked as encrypted in
buf_page_check_corrupt(). Unencrypted page could be
corrupted in a way where the key_id field is
nonzero. There is no checksum on field
FIL_PAGE_FILE_FLUSH_LSN_OR_KEY_VERSION. */
if (local_err == DB_DECRYPTION_FAILED) {
return (NULL);
}
/* Try to set table as corrupted instead of
asserting. */
if (space > TRX_SYS_SPACE &&
dict_set_corrupted_by_space(space)) {
return (NULL);
}
ib_logf(IB_LOG_LEVEL_FATAL, "Unable"
" to read tablespace %lu page no"
" %lu into the buffer pool after"
" %lu attempts"
" The most probable cause"
" of this error may be that the"
" table has been corrupted."
" You can try to fix this"
" problem by using"
" innodb_force_recovery."
" Please see " REFMAN " for more"
" details. Aborting...",
space, offset,
BUF_PAGE_READ_MAX_RETRIES);
}
#if defined UNIV_DEBUG || defined UNIV_BUF_DEBUG
@ -3856,7 +3833,6 @@ buf_page_init_low(
bpage->oldest_modification = 0;
bpage->write_size = 0;
bpage->encrypted = false;
bpage->key_version = 0;
bpage->real_size = 0;
HASH_INVALIDATE(bpage, hash);
@ -4499,61 +4475,55 @@ buf_page_monitor(
/********************************************************************//**
Mark a table with the specified space pointed by bpage->space corrupted.
Also remove the bpage from LRU list.
@return TRUE if successful */
@param[in,out] bpage Block */
static
ibool
void
buf_mark_space_corrupt(
/*===================*/
buf_page_t* bpage) /*!< in: pointer to the block in question */
buf_page_t* bpage)
{
buf_pool_t* buf_pool = buf_pool_from_bpage(bpage);
const ibool uncompressed = (buf_page_get_state(bpage)
== BUF_BLOCK_FILE_PAGE);
ulint space = bpage->space;
ibool ret = TRUE;
const ulint fold = buf_page_address_fold(bpage->space,
bpage->offset);
prio_rw_lock_t* hash_lock = buf_page_hash_lock_get(buf_pool, fold);
ib_mutex_t* block_mutex = buf_page_get_mutex(bpage);
/* First unfix and release lock on the bpage */
ut_ad(!mutex_own(&buf_pool->LRU_list_mutex));
mutex_enter(&buf_pool->LRU_list_mutex);
rw_lock_x_lock(hash_lock);
mutex_enter(block_mutex);
ut_ad(buf_page_get_io_fix(bpage) == BUF_IO_READ);
ut_ad(bpage->buf_fix_count == 0);
/* Set BUF_IO_NONE before we remove the block from LRU list */
buf_page_set_io_fix(bpage, BUF_IO_NONE);
if (uncompressed) {
rw_lock_x_unlock_gen(
&((buf_block_t*) bpage)->lock,
BUF_IO_READ);
}
/* If block is not encrypted find the table with specified
space id, and mark it corrupted. Encrypted tables
are marked unusable later e.g. in ::open(). */
if (!bpage->encrypted) {
mutex_enter(&buf_pool->LRU_list_mutex);
rw_lock_x_lock(hash_lock);
mutex_enter(buf_page_get_mutex(bpage));
ut_ad(buf_page_get_io_fix(bpage) == BUF_IO_READ);
ut_ad(bpage->buf_fix_count == 0);
/* Set BUF_IO_NONE before we remove the block from LRU list */
buf_page_set_io_fix(bpage, BUF_IO_NONE);
if (uncompressed) {
rw_lock_x_unlock_gen(
&((buf_block_t*) bpage)->lock,
BUF_IO_READ);
}
}
/* Find the table with specified space id, and mark it corrupted */
if (dict_set_corrupted_by_space(space)) {
if (!bpage->encrypted) {
buf_LRU_free_one_page(bpage);
}
dict_set_corrupted_by_space(space);
} else {
if (!bpage->encrypted) {
mutex_exit(buf_page_get_mutex(bpage));
}
ret = FALSE;
dict_set_encrypted_by_space(space);
}
if(!bpage->encrypted) {
mutex_exit(&buf_pool->LRU_list_mutex);
ut_ad(buf_pool->n_pend_reads > 0);
os_atomic_decrement_ulint(&buf_pool->n_pend_reads, 1);
}
/* After this point bpage can't be referenced. This
function will release the hash_lock acquired above. */
buf_LRU_free_one_page(bpage);
return(ret);
ut_ad(buf_pool->n_pend_reads > 0);
os_atomic_decrement_ulint(&buf_pool->n_pend_reads, 1);
mutex_exit(&buf_pool->LRU_list_mutex);
}
/********************************************************************//**
@ -4561,24 +4531,29 @@ Check if page is maybe compressed, encrypted or both when we encounter
corrupted page. Note that we can't be 100% sure if page is corrupted
or decrypt/decompress just failed.
@param[in,out] bpage Page
@return true if page corrupted, false if not */
UNIV_INTERN
bool
buf_page_check_corrupt(
buf_page_t* bpage)
@return DB_SUCCESS if page has been read and is not corrupted,
@retval DB_PAGE_CORRUPTED if page based on checksum check is corrupted,
@retval DB_DECRYPTION_FAILED if page post encryption checksum matches but
after decryption normal page checksum does not match.
@retval DB_TABLESPACE_DELETED if accessed tablespace is not found */
static
dberr_t
buf_page_check_corrupt(buf_page_t* bpage)
{
ulint zip_size = buf_page_get_zip_size(bpage);
byte* dst_frame = (zip_size) ? bpage->zip.data :
((buf_block_t*) bpage)->frame;
ulint space_id = bpage->space;
fil_space_t* space = fil_space_acquire_silent(space_id);
FilSpace space(bpage->space, true);
bool still_encrypted = false;
dberr_t err = DB_SUCCESS;
bool corrupted = false;
ulint page_type = mach_read_from_2(dst_frame + FIL_PAGE_TYPE);
fil_space_crypt_t* crypt_data = NULL;
ut_ad(space);
crypt_data = space->crypt_data;
if (!space()) {
return(DB_TABLESPACE_DELETED);
}
crypt_data = space()->crypt_data;
/* In buf_decrypt_after_read we have either decrypted the page if
page post encryption checksum matches and used key_id is found
@ -4590,62 +4565,65 @@ buf_page_check_corrupt(
crypt_data->type != CRYPT_SCHEME_UNENCRYPTED &&
!bpage->encrypted &&
fil_space_verify_crypt_checksum(dst_frame, zip_size,
space, bpage->offset));
space(), bpage->offset));
if (!still_encrypted) {
/* If traditional checksums match, we assume that page is
not anymore encrypted. */
corrupted = buf_page_is_corrupted(true, dst_frame, zip_size, space);
corrupted = buf_page_is_corrupted(true, dst_frame, zip_size, space());
if (!corrupted) {
bpage->encrypted = false;
} else {
err = DB_PAGE_CORRUPTED;
}
}
/* Pages that we think are unencrypted but do not match the checksum
checks could be corrupted or encrypted or both. */
if (corrupted && !bpage->encrypted) {
ib_logf(IB_LOG_LEVEL_ERROR,
"%s: Block in space_id " ULINTPF " in file %s corrupted.",
page_type == FIL_PAGE_PAGE_COMPRESSED_ENCRYPTED ? "Maybe corruption" : "Corruption",
space_id, (space && space->name) ? space->name : "NULL");
ib_logf(IB_LOG_LEVEL_ERROR,
"Based on page type %s (" ULINTPF ")",
fil_get_page_type_name(page_type), page_type);
/* An error will be reported by
buf_page_io_complete(). */
} else if (still_encrypted || (bpage->encrypted && corrupted)) {
bpage->encrypted = true;
corrupted = true;
err = DB_DECRYPTION_FAILED;
ib_logf(IB_LOG_LEVEL_ERROR,
"Block in space_id " ULINTPF " in file %s encrypted.",
space_id, (space && space->name) ? space->name : "NULL");
ib_logf(IB_LOG_LEVEL_ERROR,
"However key management plugin or used key_version %u is not found or"
" used encryption algorithm or method does not match.",
bpage->key_version);
if (space_id > TRX_SYS_SPACE) {
ib_logf(IB_LOG_LEVEL_ERROR,
"The page [page id: space=%u"
", page number=%u]"
" in file %s cannot be decrypted.",
bpage->space, bpage->offset,
space()->name);
ib_logf(IB_LOG_LEVEL_INFO,
"However key management plugin or used key_version " ULINTPF
" is not found or"
" used encryption algorithm or method does not match.",
mach_read_from_4(dst_frame+FIL_PAGE_FILE_FLUSH_LSN_OR_KEY_VERSION));
if (bpage->space > TRX_SYS_SPACE) {
ib_logf(IB_LOG_LEVEL_INFO,
"Marking tablespace as missing. You may drop this table or"
" install correct key management plugin and key file.");
}
}
if (space) {
fil_space_release(space);
}
return corrupted;
return (err);
}
/********************************************************************//**
Completes an asynchronous read or write request of a file page to or from
the buffer pool.
@return true if successful */
@param[in,out] bpage Page to complete
@return DB_SUCCESS if page has been read and is not corrupted,
DB_PAGE_CORRUPTED if page based on checksum check is corrupted,
DB_DECRYPTION_FAILED if page post encryption checksum matches but
after decryption normal page checksum does not match.
in write only DB_SUCCESS is possible. */
UNIV_INTERN
bool
dberr_t
buf_page_io_complete(
/*=================*/
buf_page_t* bpage) /*!< in: pointer to the block in question */
buf_page_t* bpage)
{
enum buf_io_fix io_type;
buf_pool_t* buf_pool = buf_pool_from_bpage(bpage);
@ -4653,8 +4631,8 @@ buf_page_io_complete(
== BUF_BLOCK_FILE_PAGE);
bool have_LRU_mutex = false;
fil_space_t* space = NULL;
byte* frame = NULL;
bool corrupted = false;
byte* frame = NULL;
dberr_t err = DB_SUCCESS;
ut_a(buf_page_in_file(bpage));
@ -4668,8 +4646,9 @@ buf_page_io_complete(
ut_ad(io_type == BUF_IO_READ || io_type == BUF_IO_WRITE);
if (io_type == BUF_IO_READ) {
ulint read_page_no;
ulint read_space_id;
ulint read_page_no = 0;
ulint read_space_id = 0;
uint key_version = 0;
buf_page_decrypt_after_read(bpage);
@ -4693,10 +4672,11 @@ buf_page_io_complete(
"Page %u in tablespace %u zip_decompress failure.",
bpage->offset, bpage->space);
corrupted = true;
err = DB_PAGE_CORRUPTED;
goto database_corrupted;
}
os_atomic_decrement_ulint(&buf_pool->n_pend_unzip, 1);
} else {
ut_a(uncompressed);
@ -4709,6 +4689,8 @@ buf_page_io_complete(
read_page_no = mach_read_from_4(frame + FIL_PAGE_OFFSET);
read_space_id = mach_read_from_4(
frame + FIL_PAGE_ARCH_LOG_NO_OR_SPACE_ID);
key_version = mach_read_from_4(
frame + FIL_PAGE_FILE_FLUSH_LSN_OR_KEY_VERSION);
if (bpage->space == TRX_SYS_SPACE
&& buf_dblwr_page_inside(bpage->offset)) {
@ -4729,12 +4711,11 @@ buf_page_io_complete(
page may contain garbage in MySQL < 4.1.1,
which only supported bpage->space == 0. */
ut_print_timestamp(stderr);
fprintf(stderr,
" InnoDB: Error: space id and page n:o"
" stored in the page\n"
"InnoDB: read in are " ULINTPF ":" ULINTPF ","
" should be %u:%u!\n",
ib_logf(IB_LOG_LEVEL_ERROR,
"Space id and page n:o"
" stored in the page"
" read in are " ULINTPF ":" ULINTPF ","
" should be %u:%u!",
read_space_id,
read_page_no,
bpage->space,
@ -4743,81 +4724,68 @@ buf_page_io_complete(
if (UNIV_LIKELY(!bpage->is_corrupt ||
!srv_pass_corrupt_table)) {
corrupted = buf_page_check_corrupt(bpage);
err = buf_page_check_corrupt(bpage);
}
database_corrupted:
if (corrupted) {
if (err != DB_SUCCESS) {
/* Not a real corruption if it was triggered by
error injection */
DBUG_EXECUTE_IF("buf_page_is_corrupt_failure",
if (bpage->space > TRX_SYS_SPACE
&& buf_mark_space_corrupt(bpage)) {
if (bpage->space > TRX_SYS_SPACE) {
buf_mark_space_corrupt(bpage);
ib_logf(IB_LOG_LEVEL_INFO,
"Simulated page corruption");
return(true);
return(err);
}
err = DB_SUCCESS;
goto page_not_corrupt;
);
if (!bpage->encrypted) {
if (err == DB_PAGE_CORRUPTED) {
fil_system_enter();
space = fil_space_get_by_id(bpage->space);
fil_system_exit();
ib_logf(IB_LOG_LEVEL_ERROR,
"Database page corruption on disk"
" or a failed");
ib_logf(IB_LOG_LEVEL_ERROR,
"Space %u file %s read of page %u.",
bpage->space,
space->name ? space->name : "NULL",
bpage->offset);
ib_logf(IB_LOG_LEVEL_ERROR,
"You may have to recover"
" from a backup.");
" or a failed file read of tablespace %s"
" page [page id: space=%u"
", page number=%u]"
". You may have to recover from "
"a backup.",
space->name,
bpage->space, bpage->offset);
fil_system_exit();
buf_page_print(frame, buf_page_get_zip_size(bpage),
BUF_PAGE_PRINT_NO_CRASH);
ib_logf(IB_LOG_LEVEL_ERROR,
"It is also possible that your operating"
"system has corrupted its own file cache.");
ib_logf(IB_LOG_LEVEL_ERROR,
"and rebooting your computer removes the error.");
ib_logf(IB_LOG_LEVEL_ERROR,
"If the corrupt page is an index page you can also try to");
ib_logf(IB_LOG_LEVEL_ERROR,
"fix the corruption by dumping, dropping, and reimporting");
ib_logf(IB_LOG_LEVEL_ERROR,
"the corrupt table. You can use CHECK");
ib_logf(IB_LOG_LEVEL_ERROR,
"TABLE to scan your table for corruption.");
ib_logf(IB_LOG_LEVEL_ERROR,
"See also "
REFMAN "forcing-innodb-recovery.html"
" about forcing recovery.");
ib_logf(IB_LOG_LEVEL_INFO,
"It is also possible that your"
" operating system has corrupted"
" its own file cache and rebooting"
" your computer removes the error."
" If the corrupt page is an index page."
" You can also try to fix the"
" corruption by dumping, dropping,"
" and reimporting the corrupt table."
" You can use CHECK TABLE to scan"
" your table for corruption. "
"Please refer to " REFMAN "forcing-innodb-recovery.html"
" for information about forcing recovery.");
}
if (srv_pass_corrupt_table && bpage->space != 0
&& bpage->space < SRV_LOG_SPACE_FIRST_ID) {
trx_t* trx;
fprintf(stderr,
"InnoDB: space %u will be treated as corrupt.\n",
ib_logf(IB_LOG_LEVEL_INFO,
"InnoDB: space %u will be treated as corrupt.",
bpage->space);
fil_space_set_corrupt(bpage->space);
trx = innobase_get_trx();
if (trx && trx->dict_operation_lock_mode == RW_X_LATCH) {
dict_table_set_corrupt_by_space(bpage->space, FALSE);
} else {
dict_table_set_corrupt_by_space(bpage->space, TRUE);
}
dict_set_corrupted_by_space(bpage->space);
bpage->is_corrupt = TRUE;
}
@ -4825,31 +4793,12 @@ database_corrupted:
/* If page space id is larger than TRX_SYS_SPACE
(0), we will attempt to mark the corresponding
table as corrupted instead of crashing server */
if (bpage->space > TRX_SYS_SPACE
&& buf_mark_space_corrupt(bpage)) {
return(false);
if (bpage->space > TRX_SYS_SPACE) {
buf_mark_space_corrupt(bpage);
return(err);
} else {
if (!bpage->encrypted) {
ib_logf(IB_LOG_LEVEL_ERROR,
"Ending processing because of a corrupt database page.");
ut_error;
}
ib_push_warning(innobase_get_trx(), DB_DECRYPTION_FAILED,
"Table in tablespace %lu encrypted."
"However key management plugin or used key_id %lu is not found or"
" used encryption algorithm or method does not match."
" Can't continue opening the table.",
bpage->space, bpage->key_version);
if (bpage->encrypted && bpage->space > TRX_SYS_SPACE) {
buf_mark_space_corrupt(bpage);
} else {
ut_error;
}
return(false);
ib_logf(IB_LOG_LEVEL_FATAL,
"Ending processing because of a corrupt database page.");
}
}
}
@ -4869,11 +4818,12 @@ database_corrupted:
if (bpage && bpage->encrypted) {
ib_logf(IB_LOG_LEVEL_WARN,
"Table in tablespace %lu encrypted."
"However key management plugin or used key_version %u is not found or"
"Table in tablespace " ULINTPF " encrypted."
"However key management plugin or used "
" key_version %u is not found or"
" used encryption algorithm or method does not match."
" Can't continue opening the table.\n",
(ulint)bpage->space, bpage->key_version);
" Can't continue opening the table.",
read_space_id, key_version);
} else {
ibuf_merge_or_delete_for_page(
@ -4995,7 +4945,7 @@ retry_mutex:
mutex_exit(block_mutex);
return(true);
return(err);
}
/*********************************************************************//**
@ -5023,7 +4973,7 @@ buf_all_freed_instance(
mutex_exit(&buf_pool->LRU_list_mutex);
if (UNIV_LIKELY_NULL(block) && block->page.key_version == 0) {
if (UNIV_LIKELY_NULL(block)) {
fil_space_t* space = fil_space_get(block->page.space);
ib_logf(IB_LOG_LEVEL_ERROR,
"Page %u %u still fixed or dirty.",
@ -6263,32 +6213,29 @@ buf_page_encrypt_before_write(
switch (bpage->offset) {
case 0:
/* Page 0 of a tablespace is not encrypted/compressed */
ut_ad(bpage->key_version == 0);
return src_frame;
case TRX_SYS_PAGE_NO:
if (bpage->space == TRX_SYS_SPACE) {
/* don't encrypt/compress page as it contains
address to dblwr buffer */
bpage->key_version = 0;
return src_frame;
}
}
fil_space_crypt_t* crypt_data = space->crypt_data;
const bool encrypted = crypt_data
&& !crypt_data->not_encrypted()
&& crypt_data->type != CRYPT_SCHEME_UNENCRYPTED
&& (!crypt_data->is_default_encryption()
|| srv_encrypt_tables);
if (!encrypted) {
bpage->key_version = 0;
}
bool page_compressed = FSP_FLAGS_HAS_PAGE_COMPRESSION(space->flags);
if (!encrypted && !page_compressed) {
/* No need to encrypt or page compress the page */
/* No need to encrypt or page compress the page.
Clear key-version & crypt-checksum. */
memset(src_frame + FIL_PAGE_FILE_FLUSH_LSN_OR_KEY_VERSION, 0, 8);
return src_frame;
}
@ -6310,9 +6257,6 @@ buf_page_encrypt_before_write(
src_frame,
dst_frame);
ulint key_version = mach_read_from_4(dst_frame + FIL_PAGE_FILE_FLUSH_LSN_OR_KEY_VERSION);
ut_ad(key_version == 0 || key_version >= bpage->key_version);
bpage->key_version = key_version;
bpage->real_size = page_size;
slot->out_buf = dst_frame = tmp;
@ -6381,19 +6325,17 @@ buf_page_decrypt_after_read(
buf_pool_t* buf_pool = buf_pool_from_bpage(bpage);
bool success = true;
bpage->key_version = key_version;
if (bpage->offset == 0) {
/* File header pages are not encrypted/compressed */
return (true);
}
fil_space_t* space = fil_space_acquire(bpage->space, true);
FilSpace space(bpage->space, false, true);
/* Page is encrypted if encryption information is found from
tablespace and page contains used key_version. This is true
also for pages first compressed and then encrypted. */
if (!space || !space->crypt_data) {
if (!space() || !space()->crypt_data) {
key_version = 0;
}
@ -6427,6 +6369,14 @@ buf_page_decrypt_after_read(
decrypt. */
if (!fil_space_verify_crypt_checksum(dst_frame,
zip_size, NULL, bpage->offset)) {
/* Mark page encrypted in case it should
be. */
if (key_version && space()->crypt_data &&
space()->crypt_data->type != CRYPT_SCHEME_UNENCRYPTED) {
bpage->encrypted = true;
}
return (false);
}
@ -6438,7 +6388,7 @@ buf_page_decrypt_after_read(
#endif
/* decrypt using crypt_buf to dst_frame */
byte* res = fil_space_decrypt(space,
byte* res = fil_space_decrypt(space(),
slot->crypt_buf,
dst_frame,
&bpage->encrypted);
@ -6474,8 +6424,5 @@ buf_page_decrypt_after_read(
}
}
if (space != NULL) {
fil_space_release(space);
}
return (success);
}

9
storage/xtradb/buf/buf0dblwr.cc
View File

@ -493,7 +493,9 @@ buf_dblwr_process()
page_no = mach_read_from_4(page + FIL_PAGE_OFFSET);
space_id = mach_read_from_4(page + FIL_PAGE_SPACE_ID);
if (!fil_tablespace_exists_in_mem(space_id)) {
FilSpace space(space_id, true);
if (!space()) {
/* Maybe we have dropped the single-table tablespace
and this page once belonged to it: do nothing */
continue;
@ -508,8 +510,7 @@ buf_dblwr_process()
continue;
}
fil_space_t* space = fil_space_found_by_id(space_id);
ulint zip_size = fil_space_get_zip_size(space_id);
ulint zip_size = fsp_flags_get_zip_size(space()->flags);
ut_ad(!buf_page_is_zeroes(page, zip_size));
/* Read in the actual page from the file */
@ -545,7 +546,7 @@ buf_dblwr_process()
if (fil_space_verify_crypt_checksum(
read_buf, zip_size, NULL, page_no)
|| !buf_page_is_corrupted(
true, read_buf, zip_size, space)) {
true, read_buf, zip_size, space())) {
/* The page is good; there is no need
to consult the doublewrite buffer. */
continue;

6
storage/xtradb/buf/buf0flu.cc
View File

@ -994,7 +994,13 @@ buf_flush_write_block_low(
if (sync) {
ut_ad(flush_type == BUF_FLUSH_SINGLE_PAGE);
fil_flush(space);
#ifdef UNIV_DEBUG
dberr_t err =
#endif
buf_page_io_complete(bpage);
ut_ad(err == DB_SUCCESS);
}
fil_space_release(space);

278
storage/xtradb/buf/buf0rea.cc
View File

@ -1,7 +1,7 @@
/*****************************************************************************
Copyright (c) 1995, 2013, Oracle and/or its affiliates. All Rights Reserved.
Copyright (c) 2013, 2014, MariaDB Corporation. All Rights Reserved.
Copyright (c) 2013, 2017, MariaDB Corporation. All Rights Reserved.
This program is free software; you can redistribute it and/or modify it under
the terms of the GNU General Public License as published by the Free Software
@ -103,36 +103,46 @@ flag is cleared and the x-lock released by an i/o-handler thread.
in buf_pool, or if the page is in the doublewrite buffer blocks in
which case it is never read into the pool, or if the tablespace does
not exist or is being dropped
@return 1 if read request is issued. 0 if it is not */
UNIV_INTERN
ulint
buf_read_page_low(
/*==============*/
dberr_t* err, /*!< out: DB_SUCCESS or DB_TABLESPACE_DELETED if we are
trying to read from a non-existent tablespace, or a
tablespace which is just now being dropped */
bool sync, /*!< in: true if synchronous aio is desired */
ulint mode, /*!< in: BUF_READ_IBUF_PAGES_ONLY, ...,
ORed to OS_AIO_SIMULATED_WAKE_LATER (see below
at read-ahead functions) */
ulint space, /*!< in: space id */
ulint zip_size,/*!< in: compressed page size, or 0 */
ibool unzip, /*!< in: TRUE=request uncompressed page */
ib_int64_t tablespace_version, /*!< in: if the space memory object has
@param[out] err DB_SUCCESS, DB_TABLESPACE_DELETED if we are
trying to read from a non-existent tablespace, or a
tablespace which is just now being dropped,
DB_PAGE_CORRUPTED if page based on checksum
check is corrupted, or DB_DECRYPTION_FAILED
if page post encryption checksum matches but
after decryption normal page checksum does not match.
@param[in] sync true if synchronous aio is desired
@param[in] mode BUF_READ_IBUF_PAGES_ONLY, ...,
ORed to OS_AIO_SIMULATED_WAKE_LATER (see below
at read-ahead functions)
@param[in] space space id
@param[in] zip_size compressed page size, or 0
@param[in] unzip TRUE=request uncompressed page
@param[in] tablespace_version if the space memory object has
this timestamp different from what we are giving here,
treat the tablespace as dropped; this is a timestamp we
use to stop dangling page reads from a tablespace
which we have DISCARDed + IMPORTed back */
ulint offset, /*!< in: page number */
trx_t* trx, /*!< in: trx */
buf_page_t** rbpage) /*!< out: page */
which we have DISCARDed + IMPORTed back
@param[in] offset page number
@param[in] trx transaction
@return 1 if read request is issued. 0 if it is not */
static
ulint
buf_read_page_low(
dberr_t* err,
bool sync,
ulint mode,
ulint space,
ulint zip_size,
ibool unzip,
ib_int64_t tablespace_version,
ulint offset,
trx_t* trx = NULL)
{
buf_page_t* bpage;
ulint wake_later;
ibool ignore_nonexistent_pages;
*err = DB_SUCCESS;
wake_later = mode & OS_AIO_SIMULATED_WAKE_LATER;
mode = mode & ~OS_AIO_SIMULATED_WAKE_LATER;
@ -259,18 +269,13 @@ not_to_recover:
if (sync) {
/* The i/o is already completed when we arrive from
fil_read */
if (!buf_page_io_complete(bpage)) {
if (rbpage) {
*rbpage = bpage;
}
*err = buf_page_io_complete(bpage);
if (*err != DB_SUCCESS) {
return(0);
}
}
if (rbpage) {
*rbpage = bpage;
}
return(1);
}
@ -307,7 +312,7 @@ buf_read_ahead_random(
ulint ibuf_mode;
ulint count;
ulint low, high;
dberr_t err;
dberr_t err = DB_SUCCESS;
ulint i;
const ulint buf_read_ahead_random_area
= BUF_READ_AHEAD_AREA(buf_pool);
@ -402,20 +407,33 @@ read_ahead:
mode: hence FALSE as the first parameter */
if (!ibuf_bitmap_page(zip_size, i)) {
count += buf_read_page_low(
&err, false,
ibuf_mode | OS_AIO_SIMULATED_WAKE_LATER,
space, zip_size, FALSE,
tablespace_version, i, trx, NULL);
if (err == DB_TABLESPACE_DELETED) {
ut_print_timestamp(stderr);
fprintf(stderr,
" InnoDB: Warning: in random"
" readahead trying to access\n"
"InnoDB: tablespace %lu page %lu,\n"
"InnoDB: but the tablespace does not"
" exist or is just being dropped.\n",
(ulong) space, (ulong) i);
tablespace_version, i, trx);
switch(err) {
case DB_SUCCESS:
break;
case DB_TABLESPACE_DELETED:
ib_logf(IB_LOG_LEVEL_WARN,
"In random"
" readahead trying to access"
" tablespace " ULINTPF ":" ULINTPF
" but the tablespace does not"
" exist or is just being dropped.",
space, i);
break;
case DB_DECRYPTION_FAILED:
ib_logf(IB_LOG_LEVEL_ERROR,
"Random readahead failed to decrypt page "
ULINTPF ":" ULINTPF ".",
i, space);
break;
default:
ut_error;
}
}
}
@ -449,44 +467,60 @@ High-level function which reads a page asynchronously from a file to the
buffer buf_pool if it is not already there. Sets the io_fix flag and sets
an exclusive lock on the buffer frame. The flag is cleared and the x-lock
released by the i/o-handler thread.
@return TRUE if page has been read in, FALSE in case of failure */
@param[in] space_id space_id
@param[in] zip_size compressed page size in bytes, or 0
@param[in] offset page number
@param[in] trx transaction
@param[out] encrypted true if page encrypted
@return DB_SUCCESS if page has been read and is not corrupted,
@retval DB_PAGE_CORRUPTED if page based on checksum check is corrupted,
@retval DB_DECRYPTION_FAILED if page post encryption checksum matches but
after decryption normal page checksum does not match.
@retval DB_TABLESPACE_DELETED if tablespace .ibd file is missing */
UNIV_INTERN
ibool
dberr_t
buf_read_page(
/*==========*/
ulint space, /*!< in: space id */
ulint zip_size, /*!< in: compressed page size in bytes, or 0 */
ulint offset, /*!< in: page number */
trx_t* trx, /*!< in: trx */
buf_page_t** bpage) /*!< out: page */
ulint space_id,
ulint zip_size,
ulint offset,
trx_t* trx)
{
ib_int64_t tablespace_version;
ulint count;
dberr_t err;
dberr_t err = DB_SUCCESS;
tablespace_version = fil_space_get_version(space);
tablespace_version = fil_space_get_version(space_id);
/* We do the i/o in the synchronous aio mode to save thread
switches: hence TRUE */
FilSpace space(space_id, true);
count = buf_read_page_low(&err, true, BUF_READ_ANY_PAGE, space,
if (space()) {
/* We do the i/o in the synchronous aio mode to save thread
switches: hence TRUE */
count = buf_read_page_low(&err, true, BUF_READ_ANY_PAGE, space_id,
zip_size, FALSE,
tablespace_version, offset, trx, bpage);
srv_stats.buf_pool_reads.add(count);
if (err == DB_TABLESPACE_DELETED) {
ut_print_timestamp(stderr);
fprintf(stderr,
" InnoDB: Error: trying to access"
" tablespace %lu page no. %lu,\n"
"InnoDB: but the tablespace does not exist"
" or is just being dropped.\n",
(ulong) space, (ulong) offset);
tablespace_version, offset, trx);
srv_stats.buf_pool_reads.add(count);
}
/* Page corruption and decryption failures are already reported
in above function. */
if (!space() || err == DB_TABLESPACE_DELETED) {
err = DB_TABLESPACE_DELETED;
ib_logf(IB_LOG_LEVEL_ERROR,
"Trying to access"
" tablespace [space=" ULINTPF ": page=" ULINTPF
"] but the tablespace does not exist"
" or is just being dropped.",
space_id, offset);
}
/* Increment number of I/O operations used for LRU policy. */
buf_LRU_stat_inc_io();
return(count > 0);
return(err);
}
/********************************************************************//**
@ -494,23 +528,23 @@ High-level function which reads a page asynchronously from a file to the
buffer buf_pool if it is not already there. Sets the io_fix flag and sets
an exclusive lock on the buffer frame. The flag is cleared and the x-lock
released by the i/o-handler thread.
@return TRUE if page has been read in, FALSE in case of failure */
@param[in] space Tablespace id
@param[in] offset Page no */
UNIV_INTERN
ibool
void
buf_read_page_async(
/*================*/
ulint space, /*!< in: space id */
ulint offset) /*!< in: page number */
ulint space,
ulint offset)
{
ulint zip_size;
ib_int64_t tablespace_version;
ulint count;
dberr_t err;
dberr_t err = DB_SUCCESS;
zip_size = fil_space_get_zip_size(space);
if (zip_size == ULINT_UNDEFINED) {
return(FALSE);
return;
}
tablespace_version = fil_space_get_version(space);
@ -519,7 +553,30 @@ buf_read_page_async(
| OS_AIO_SIMULATED_WAKE_LATER
| BUF_READ_IGNORE_NONEXISTENT_PAGES,
space, zip_size, FALSE,
tablespace_version, offset, NULL,NULL);
tablespace_version, offset);
switch(err) {
case DB_SUCCESS:
break;
case DB_TABLESPACE_DELETED:
ib_logf(IB_LOG_LEVEL_ERROR,
"In async page read "
"trying to access "
"page " ULINTPF ":" ULINTPF
" in nonexisting or being-dropped tablespace",
space, offset);
break;
case DB_DECRYPTION_FAILED:
ib_logf(IB_LOG_LEVEL_ERROR,
"Async page read failed to decrypt page "
ULINTPF ":" ULINTPF ".",
space, offset);
break;
default:
ut_error;
}
srv_stats.buf_pool_reads.add(count);
/* We do not increment number of I/O operations used for LRU policy
@ -528,8 +585,6 @@ buf_read_page_async(
buffer pool. Since this function is called from buffer pool load
these IOs are deliberate and are not part of normal workload we can
ignore these in our heuristics. */
return(count > 0);
}
/********************************************************************//**
@ -580,7 +635,7 @@ buf_read_ahead_linear(
ulint fail_count;
ulint ibuf_mode;
ulint low, high;
dberr_t err;
dberr_t err = DB_SUCCESS;
ulint i;
const ulint buf_read_ahead_linear_area
= BUF_READ_AHEAD_AREA(buf_pool);
@ -784,19 +839,34 @@ buf_read_ahead_linear(
aio mode: hence FALSE as the first parameter */
if (!ibuf_bitmap_page(zip_size, i)) {
count += buf_read_page_low(
&err, false,
ibuf_mode,
space, zip_size, FALSE, tablespace_version, i, trx, NULL);
if (err == DB_TABLESPACE_DELETED) {
ut_print_timestamp(stderr);
fprintf(stderr,
" InnoDB: Warning: in"
" linear readahead trying to access\n"
"InnoDB: tablespace %lu page %lu,\n"
"InnoDB: but the tablespace does not"
" exist or is just being dropped.\n",
(ulong) space, (ulong) i);
space, zip_size, FALSE, tablespace_version,
i, trx);
switch(err) {
case DB_SUCCESS:
break;
case DB_TABLESPACE_DELETED:
ib_logf(IB_LOG_LEVEL_WARN,
"In linear"
" readahead trying to access"
" tablespace " ULINTPF ":" ULINTPF
" but the tablespace does not"
" exist or is just being dropped.",
space, i);
break;
case DB_DECRYPTION_FAILED:
ib_logf(IB_LOG_LEVEL_ERROR,
"Linear readahead failed to decrypt page "
ULINTPF ":" ULINTPF ".",
i, space);
break;
default:
ut_error;
}
}
}
@ -858,9 +928,9 @@ buf_read_ibuf_merge_pages(
#endif
for (i = 0; i < n_stored; i++) {
dberr_t err;
buf_pool_t* buf_pool;
ulint zip_size = fil_space_get_zip_size(space_ids[i]);
dberr_t err = DB_SUCCESS;
buf_pool = buf_pool_get(space_ids[i], page_nos[i]);
@ -870,16 +940,19 @@ buf_read_ibuf_merge_pages(
}
if (UNIV_UNLIKELY(zip_size == ULINT_UNDEFINED)) {
goto tablespace_deleted;
}
buf_read_page_low(&err, sync && (i + 1 == n_stored),
BUF_READ_ANY_PAGE, space_ids[i],
zip_size, TRUE, space_versions[i],
page_nos[i], NULL, NULL);
page_nos[i], NULL);
switch(err) {
case DB_SUCCESS:
break;
case DB_TABLESPACE_DELETED:
if (UNIV_UNLIKELY(err == DB_TABLESPACE_DELETED)) {
tablespace_deleted:
/* We have deleted or are deleting the single-table
tablespace: remove the entries for that page */
@ -887,6 +960,15 @@ tablespace_deleted:
ibuf_merge_or_delete_for_page(NULL, space_ids[i],
page_nos[i],
zip_size, FALSE);
break;
case DB_DECRYPTION_FAILED:
ib_logf(IB_LOG_LEVEL_ERROR,
"Failed to decrypt insert buffer page "
ULINTPF ":" ULINTPF ".",
space_ids[i], page_nos[i]);
break;
default:
ut_error;
}
}
@ -924,7 +1006,7 @@ buf_read_recv_pages(
{
ib_int64_t tablespace_version;
ulint count;
dberr_t err;
dberr_t err = DB_SUCCESS;
ulint i;
zip_size = fil_space_get_zip_size(space);
@ -1013,12 +1095,20 @@ not_to_recover:
if ((i + 1 == n_stored) && sync) {
buf_read_page_low(&err, true, BUF_READ_ANY_PAGE, space,
zip_size, TRUE, tablespace_version,
page_nos[i], NULL, NULL);
page_nos[i], NULL);
} else {
buf_read_page_low(&err, false, BUF_READ_ANY_PAGE
| OS_AIO_SIMULATED_WAKE_LATER,
space, zip_size, TRUE,
tablespace_version, page_nos[i], NULL, NULL);
tablespace_version, page_nos[i],
NULL);
}
if (err == DB_DECRYPTION_FAILED) {
ib_logf(IB_LOG_LEVEL_ERROR,
"Recovery failed to decrypt read page "
ULINTPF ":" ULINTPF ".",
space, page_nos[i]);
}
}

2
storage/xtradb/dict/dict0crea.cc
View File

@ -692,7 +692,7 @@ dict_create_index_tree_step(
dberr_t err = DB_SUCCESS;
ulint zip_size = dict_table_zip_size(index->table);
if (node->index->table->ibd_file_missing
if (node->index->table->file_unreadable
|| dict_table_is_discarded(node->index->table)) {
node->page_no = FIL_NULL;

103
storage/xtradb/dict/dict0dict.cc
View File

@ -1185,14 +1185,31 @@ dict_table_open_on_name(
if (table != NULL) {
/* If table is encrypted return table */
/* If table is encrypted or corrupted */
if (ignore_err == DICT_ERR_IGNORE_NONE
&& table->is_encrypted) {
&& !table->is_readable()) {
/* Make life easy for drop table. */
if (table->can_be_evicted) {
dict_table_move_from_lru_to_non_lru(table);
}
if (table->corrupted) {
if (!dict_locked) {
mutex_exit(&dict_sys->mutex);
}
char buf[MAX_FULL_NAME_LEN];
ut_format_name(table->name, TRUE, buf, sizeof(buf));
ib_logf(IB_LOG_LEVEL_ERROR,
"Table %s is corrupted. Please "
"drop the table and recreate.",
buf);
return(NULL);
}
if (table->can_be_evicted) {
dict_move_to_mru(table);
}
@ -1205,28 +1222,6 @@ dict_table_open_on_name(
return (table);
}
/* If table is corrupted, return NULL */
else if (ignore_err == DICT_ERR_IGNORE_NONE
&& table->corrupted) {
/* Make life easy for drop table. */
if (table->can_be_evicted) {
dict_table_move_from_lru_to_non_lru(table);
}
if (!dict_locked) {
mutex_exit(&dict_sys->mutex);
}
ut_print_timestamp(stderr);
fprintf(stderr, " InnoDB: table ");
ut_print_name(stderr, NULL, TRUE, table->name);
fprintf(stderr, "is corrupted. Please drop the table "
"and recreate\n");
return(NULL);
}
if (table->can_be_evicted) {
dict_move_to_mru(table);
@ -6183,11 +6178,29 @@ dict_set_corrupted_by_space(
/* mark the table->corrupted bit only, since the caller
could be too deep in the stack for SYS_INDEXES update */
table->corrupted = TRUE;
table->corrupted = true;
table->file_unreadable = true;
return(TRUE);
}
/** Flags a table with specified space_id encrypted in the data dictionary
cache
@param[in] space_id Tablespace id */
UNIV_INTERN
void
dict_set_encrypted_by_space(ulint space_id)
{
dict_table_t* table;
table = dict_find_table_by_space(space_id);
if (table) {
table->file_unreadable = true;
}
}
/**********************************************************************//**
Flags an index corrupted both in the data dictionary cache
and in the SYS_INDEXES */
@ -6325,43 +6338,6 @@ dict_set_corrupted_index_cache_only(
index->type |= DICT_CORRUPT;
}
/*************************************************************************
set is_corrupt flag by space_id*/
void
dict_table_set_corrupt_by_space(
/*============================*/
ulint space_id,
ibool need_mutex)
{
dict_table_t* table;
ibool found = FALSE;
ut_a(space_id != 0 && space_id < SRV_LOG_SPACE_FIRST_ID);
if (need_mutex)
mutex_enter(&(dict_sys->mutex));
table = UT_LIST_GET_FIRST(dict_sys->table_LRU);
while (table) {
if (table->space == space_id) {
table->is_corrupt = TRUE;
found = TRUE;
}
table = UT_LIST_GET_NEXT(table_LRU, table);
}
if (need_mutex)
mutex_exit(&(dict_sys->mutex));
if (!found) {
fprintf(stderr, "InnoDB: space to be marked as "
"crashed was not found for id " ULINTPF ".\n",
space_id);
}
}
#endif /* !UNIV_HOTBACKUP */
/**********************************************************************//**
@ -6683,7 +6659,8 @@ dict_table_schema_check(
}
}
if (table->ibd_file_missing) {
if (!table->is_readable() &&
fil_space_get(table->space) == NULL) {
/* missing tablespace */
ut_snprintf(errstr, errstr_sz,

22
storage/xtradb/dict/dict0load.cc
View File

@ -1964,7 +1964,7 @@ dict_load_indexes(
dict_mem_index_free(index);
goto func_exit;
} else if (index->page == FIL_NULL
&& !table->ibd_file_missing
&& !table->file_unreadable
&& (!(index->type & DICT_FTS))) {
fprintf(stderr,
@ -2193,7 +2193,7 @@ err_len:
(*table)->id = mach_read_from_8(field);
(*table)->ibd_file_missing = FALSE;
(*table)->file_unreadable = false;
return(NULL);
}
@ -2380,7 +2380,7 @@ err_exit:
"Table '%s' tablespace is set as discarded.",
table_name);
table->ibd_file_missing = TRUE;
table->file_unreadable = true;
} else if (!fil_space_for_table_exists_in_mem(
table->space, name, false, true, heap,
@ -2388,7 +2388,7 @@ err_exit:
if (DICT_TF2_FLAG_IS_SET(table, DICT_TF2_TEMPORARY)) {
/* Do not bother to retry opening temporary tables. */
table->ibd_file_missing = TRUE;
table->file_unreadable = true;
} else {
if (!(ignore_err & DICT_ERR_IGNORE_RECOVER_LOCK)) {
@ -2423,8 +2423,9 @@ err_exit:
/* We failed to find a sensible
tablespace file */
table->ibd_file_missing = TRUE;
table->file_unreadable = true;
}
if (filepath) {
mem_free(filepath);
}
@ -2448,9 +2449,10 @@ err_exit:
were not allowed while the table is being locked by a transaction. */
dict_err_ignore_t index_load_err =
!(ignore_err & DICT_ERR_IGNORE_RECOVER_LOCK)
&& table->ibd_file_missing
&& table->file_unreadable
? DICT_ERR_IGNORE_ALL
: ignore_err;
err = dict_load_indexes(table, heap, index_load_err);
if (err == DB_INDEX_CORRUPT) {
@ -2485,7 +2487,7 @@ err_exit:
of the error condition, since the user may want to dump data from the
clustered index. However we load the foreign key information only if
all indexes were loaded. */
if (!cached || table->ibd_file_missing) {
if (!cached || table->file_unreadable) {
/* Don't attempt to load the indexes from disk. */
} else if (err == DB_SUCCESS) {
err = dict_load_foreigns(table->name, NULL, true, true,
@ -2518,12 +2520,12 @@ err_exit:
table = NULL;
} else if (dict_index_is_corrupted(index)
&& !table->ibd_file_missing) {
&& !table->file_unreadable) {
/* It is possible we force to load a corrupted
clustered index if srv_load_corrupted is set.
Mark the table as corrupted in this case */
table->corrupted = TRUE;
table->corrupted = true;
}
}
@ -2532,7 +2534,7 @@ func_exit:
ut_ad(!table
|| ignore_err != DICT_ERR_IGNORE_NONE
|| table->ibd_file_missing
|| table->file_unreadable
|| !table->corrupted);
if (table && table->fts) {

3
storage/xtradb/dict/dict0mem.cc
View File

@ -2,6 +2,7 @@
Copyright (c) 1996, 2016, Oracle and/or its affiliates. All Rights Reserved.
Copyright (c) 2012, Facebook Inc.
Copyright (c) 2013, 2017, MariaDB Corporation.
This program is free software; you can redistribute it and/or modify it under
the terms of the GNU General Public License as published by the Free Software
@ -137,8 +138,6 @@ dict_mem_table_create(
table->fts = NULL;
}
table->is_corrupt = FALSE;
#endif /* !UNIV_HOTBACKUP */
new(&table->foreign_set) dict_foreign_set();

100
storage/xtradb/dict/dict0stats.cc
View File

@ -1,6 +1,7 @@
/*****************************************************************************
Copyright (c) 2009, 2016, Oracle and/or its affiliates. All Rights Reserved.
Copyright (c) 2015, 2017, MariaDB Corporation.
This program is free software; you can redistribute it and/or modify it under
the terms of the GNU General Public License as published by the Free Software
@ -920,7 +921,11 @@ dict_stats_update_transient_for_index(
index->stat_n_leaf_pages = size;
btr_estimate_number_of_different_key_vals(index);
/* Do not continue if table decryption has failed or
table is already marked as corrupted. */
if (index->is_readable()) {
btr_estimate_number_of_different_key_vals(index);
}
}
}
@ -974,8 +979,9 @@ dict_stats_update_transient(
continue;
}
/* Do not continue if table decryption has failed. */
if (index->table->is_encrypted) {
/* Do not continue if table decryption has failed or
table is already marked as corrupted. */
if (!index->is_readable()) {
break;
}
@ -2437,6 +2443,61 @@ dict_stats_save_index_stat(
return(ret);
}
/** Report error if statistic update for a table failed because
.ibd file is missing, table decryption failed or table is corrupted.
@param[in,out] table Table
@param[in] defragment true if statistics is for defragment
@return DB_DECRYPTION_FAILED, DB_TABLESPACE_DELETED or DB_CORRUPTION
@retval DB_DECRYPTION_FAILED if decryption of the table failed
@retval DB_TABLESPACE_DELETED if .ibd file is missing
@retval DB_CORRUPTION if table is marked as corrupted */
static
dberr_t
dict_stats_report_error(
dict_table_t* table,
bool defragment = false)
{
char buf[3 * NAME_LEN];
dberr_t err;
innobase_format_name(buf, sizeof buf,
table->name,
true);
FilSpace space(table->space);
if (space()) {
if (table->corrupted) {
ib_logf(IB_LOG_LEVEL_INFO,
"Cannot save%s statistics because "
" table %s in file %s is corrupted.",
defragment ? " defragment" : " ",
buf, space()->chain.start->name);
err = DB_CORRUPTION;
} else {
ib_logf(IB_LOG_LEVEL_INFO,
"Cannot save%s statistics because "
" table %s in file %s can't be decrypted.",
defragment ? " defragment" : " ",
buf, space()->chain.start->name);
err = DB_DECRYPTION_FAILED;
}
} else {
ib_logf(IB_LOG_LEVEL_INFO,
"Cannot save%s statistics for "
" table %s because .ibd file is missing."
" For help, please "
"refer to " REFMAN "innodb-troubleshooting.html.",
defragment ? " defragment" : " ",
buf);
err = DB_TABLESPACE_DELETED;
}
dict_stats_empty_table(table, defragment);
return (err);
}
/** Save the table's statistics into the persistent statistics storage.
@param[in] table_orig table whose stats to save
@param[in] only_for_index if this is non-NULL, then stats for indexes
@ -2457,6 +2518,11 @@ dict_stats_save(
char db_utf8[MAX_DB_UTF8_LEN];
char table_utf8[MAX_TABLE_UTF8_LEN];
if (table_orig->is_readable()) {
} else {
return (dict_stats_report_error(table_orig));
}
table = dict_stats_snapshot_create(table_orig);
dict_fs2utf8(table->name, db_utf8, sizeof(db_utf8),
@ -3192,15 +3258,8 @@ dict_stats_update(
ut_ad(!mutex_own(&dict_sys->mutex));
if (table->ibd_file_missing) {
ut_print_timestamp(stderr);
fprintf(stderr,
" InnoDB: cannot calculate statistics for table %s "
"because the .ibd file is missing. For help, please "
"refer to " REFMAN "innodb-troubleshooting.html\n",
ut_format_name(table->name, TRUE, buf, sizeof(buf)));
dict_stats_empty_table(table, true);
return(DB_TABLESPACE_DELETED);
if (!table->is_readable()) {
return (dict_stats_report_error(table));
} else if (srv_force_recovery >= SRV_FORCE_NO_IBUF_MERGE) {
/* If we have set a high innodb_force_recovery level, do
not calculate statistics, as a badly corrupted index can
@ -3946,19 +4005,10 @@ dict_stats_save_defrag_stats(
{
dberr_t ret;
if (index->table->ibd_file_missing) {
ut_print_timestamp(stderr);
fprintf(stderr,
" InnoDB: Cannot save defragment stats because "
".ibd file is missing.\n");
return (DB_TABLESPACE_DELETED);
}
if (dict_index_is_corrupted(index)) {
ut_print_timestamp(stderr);
fprintf(stderr,
" InnoDB: Cannot save defragment stats because "
"index is corrupted.\n");
return(DB_CORRUPTION);
if (index->is_readable()) {
} else {
return (dict_stats_report_error(index->table, true));
}
if (dict_index_is_univ(index)) {

100
storage/xtradb/fil/fil0crypt.cc
View File

@ -479,11 +479,12 @@ Parse a MLOG_FILE_WRITE_CRYPT_DATA log entry
@param[in] block buffer block
@return position on log buffer */
UNIV_INTERN
const byte*
byte*
fil_parse_write_crypt_data(
const byte* ptr,
byte* ptr,
const byte* end_ptr,
const buf_block_t* block)
const buf_block_t* block,
dberr_t* err)
{
/* check that redo log entry is complete */
uint entry_size =
@ -495,6 +496,8 @@ fil_parse_write_crypt_data(
4 + // size of key_id
1; // fil_encryption_t
*err = DB_SUCCESS;
if (ptr + entry_size > end_ptr) {
return NULL;
}
@ -541,6 +544,11 @@ fil_parse_write_crypt_data(
fil_space_release(space);
}
/* Check is used key found from encryption plugin */
if (crypt_data->should_encrypt() && !crypt_data->is_key_found()) {
*err = DB_DECRYPTION_FAILED;
}
return ptr;
}
@ -710,10 +718,12 @@ fil_space_encrypt(
}
bool corrupted = buf_page_is_corrupted(true, tmp_mem, zip_size, space);
memcpy(tmp_mem+FIL_PAGE_FILE_FLUSH_LSN_OR_KEY_VERSION, src+FIL_PAGE_FILE_FLUSH_LSN_OR_KEY_VERSION, 8);
bool different = memcmp(src, tmp_mem, size);
if (!ok || corrupted || corrupted1 || err != DB_SUCCESS || different) {
fprintf(stderr, "ok %d corrupted %d corrupted1 %d err %d different %d\n", ok , corrupted, corrupted1, err, different);
fprintf(stderr, "ok %d corrupted %d corrupted1 %d err %d different %d\n",
ok , corrupted, corrupted1, err, different);
fprintf(stderr, "src_frame\n");
buf_page_print(src_frame, zip_size, BUF_PAGE_PRINT_NO_CRASH);
fprintf(stderr, "encrypted_frame\n");
@ -768,25 +778,6 @@ fil_space_decrypt(
return false;
}
if (crypt_data == NULL) {
if (!(space == 0 && offset == 0) && key_version != 0) {
/* FIL_PAGE_FILE_FLUSH_LSN field i.e.
FIL_PAGE_FILE_FLUSH_LSN_OR_KEY_VERSION
should be only defined for the
first page in a system tablespace
data file (ibdata*, not *.ibd), if not
clear it. */
DBUG_PRINT("ib_crypt",
("Page on space %lu offset %lu has key_version %u"
" when it shoud be undefined.",
space, offset, key_version));
mach_write_to_4(src_frame + FIL_PAGE_FILE_FLUSH_LSN_OR_KEY_VERSION, 0);
}
return false;
}
ut_a(crypt_data != NULL && crypt_data->is_encrypted());
/* read space & lsn */
@ -838,9 +829,6 @@ fil_space_decrypt(
memcpy(tmp_frame + page_size - FIL_PAGE_DATA_END,
src_frame + page_size - FIL_PAGE_DATA_END,
FIL_PAGE_DATA_END);
// clear key-version & crypt-checksum from dst
memset(tmp_frame + FIL_PAGE_FILE_FLUSH_LSN_OR_KEY_VERSION, 0, 8);
}
srv_stats.pages_decrypted.inc();
@ -962,11 +950,12 @@ fil_space_verify_crypt_checksum(
/* If page is not encrypted, return false */
if (key_version == 0) {
return false;
return(false);
}
srv_checksum_algorithm_t algorithm =
static_cast<srv_checksum_algorithm_t>(srv_checksum_algorithm);
/* If no checksum is used, can't continue checking. */
if (algorithm == SRV_CHECKSUM_ALGORITHM_NONE) {
return(true);
@ -990,28 +979,31 @@ fil_space_verify_crypt_checksum(
return (true);
}
/* Compressed pages use different checksum method. We first store
the post encryption checksum on checksum location and after function
restore the original. */
ib_uint32_t cchecksum1 = 0;
ib_uint32_t cchecksum2 = 0;
/* Calculate checksums */
if (zip_size) {
ib_uint32_t old = static_cast<ib_uint32_t>(mach_read_from_4(
page + FIL_PAGE_SPACE_OR_CHKSUM));
cchecksum1 = page_zip_calc_checksum(
page, zip_size, SRV_CHECKSUM_ALGORITHM_CRC32);
mach_write_to_4(page + FIL_PAGE_SPACE_OR_CHKSUM, checksum);
if(cchecksum1 != checksum) {
cchecksum2 = page_zip_calc_checksum(
page, zip_size,
SRV_CHECKSUM_ALGORITHM_INNODB);
}
} else {
cchecksum1 = buf_calc_page_crc32(page);
bool valid = page_zip_verify_checksum(page, zip_size);
mach_write_to_4(page + FIL_PAGE_SPACE_OR_CHKSUM, old);
return (valid);
if (cchecksum1 != checksum) {
cchecksum2 = (ib_uint32_t) buf_calc_page_new_checksum(
page);
}
}
/* If stored checksum matches one of the calculated checksums
page is not corrupted. */
ib_uint32_t cchecksum1 = buf_calc_page_crc32(page);
ib_uint32_t cchecksum2 = (ib_uint32_t) buf_calc_page_new_checksum(
page);
bool encrypted = (checksum == cchecksum1 || checksum == cchecksum2
|| checksum == BUF_NO_CHECKSUM_MAGIC);
@ -1042,13 +1034,19 @@ fil_space_verify_crypt_checksum(
ulint checksum1 = mach_read_from_4(
page + FIL_PAGE_SPACE_OR_CHKSUM);
ulint checksum2 = mach_read_from_4(
page + UNIV_PAGE_SIZE - FIL_PAGE_END_LSN_OLD_CHKSUM);
ulint checksum2 = checksum1;
bool valid;
bool valid = (buf_page_is_checksum_valid_crc32(page,checksum1,checksum2)
if (zip_size) {
valid = (checksum1 == cchecksum1);
} else {
checksum1 = mach_read_from_4(
page + UNIV_PAGE_SIZE - FIL_PAGE_END_LSN_OLD_CHKSUM);
valid = (buf_page_is_checksum_valid_crc32(page,checksum1,checksum2)
|| buf_page_is_checksum_valid_none(page,checksum1,checksum2)
|| buf_page_is_checksum_valid_innodb(page,checksum1, checksum2));
}
if (encrypted && valid) {
/* If page is encrypted and traditional checksums match,
@ -1964,11 +1962,11 @@ fil_crypt_rotate_page(
bool modified = false;
int needs_scrubbing = BTR_SCRUB_SKIP_PAGE;
lsn_t block_lsn = block->page.newest_modification;
uint kv = block->page.key_version;
byte* frame = buf_block_get_frame(block);
uint kv = mach_read_from_4(frame+FIL_PAGE_FILE_FLUSH_LSN_OR_KEY_VERSION);
/* check if tablespace is closing after reading page */
if (space->is_stopping()) {
byte* frame = buf_block_get_frame(block);
if (!space->is_stopping()) {
if (kv == 0 &&
fil_crypt_is_page_uninitialized(frame, zip_size)) {
@ -1990,9 +1988,6 @@ fil_crypt_rotate_page(
FIL_PAGE_ARCH_LOG_NO_OR_SPACE_ID,
space_id, MLOG_4BYTES, &mtr);
/* update block */
block->page.key_version = key_state->key_version;
/* statistics */
state->crypt_stat.pages_modified++;
} else {
@ -2083,6 +2078,11 @@ fil_crypt_rotate_page(
state->end_lsn = block_lsn;
}
}
} else {
/* If block read failed mtr memo and log should be empty. */
ut_ad(dyn_array_get_data_size(&mtr.memo) == 0);
ut_ad(dyn_array_get_data_size(&mtr.log) == 0);
mtr_commit(&mtr);
}
if (sleeptime_ms) {

Some files were not shown because too many files have changed in this diff Show More