起初发现了如下的现象:
mysql> show variables like ‘character%‘; +--------------------------+---------------------------------------+ | Variable_name | Value | +--------------------------+---------------------------------------+ | character_set_client | latin1 | | character_set_connection | latin1 | | character_set_database | latin1 | | character_set_filesystem | binary | | character_set_results | latin1 | | character_set_server | utf8mb4 | | character_set_system | utf8 | | character_sets_dir | /opt/mysql/server-5.6/share/charsets/ | +--------------------------+---------------------------------------+ mysql> show create table t4\G *************************** 1. row *************************** Table: t4 Create Table: CREATE TABLE `t4` ( `data` varchar(100) DEFAULT NULL ) ENGINE=InnoDB DEFAULT CHARSET=latin1 mysql> insert into t4 select ‘\U+1F600‘;
觉得很奇怪怎么latin1也支持emoji字符了呢?不是只有utf8mb4才支持吗? 于是在StackOverFlow上提问,一个网友的回答觉得有道理,回答如下:
I think you saved into and retrieved from the database a string of bytes that is interpreted by the terminal as an Unicode character. Check the output of SELECT LENGTH(data), CHAR_LENGTH(data) FROM t4 to see what‘s happening. They should return different values for multi-byte characters and the same value forlatin1. – axiac 19 hours ago
在加上无意中看到了一篇博客, 其中说道：
抛一个问题，latin1字符集的表，用户写入和读取汉字是否有问题？答案是只要合理设置，没有问题。假设SecureCRT为UTF8，character_set_client和表字符集均设置为latin1，参考第3节的分析，那么用户读取和写入数据的过程中，并不涉及字符集编码转换的问题，将UTF8的汉字字符转为二进制流写入database，提取出来后，secureCRT再将对应的二进制解码为对应的汉字，所以不影响用户的使用。
于是现在觉得上述现象很正常。
因为操作系统默认的字符集为utf8（LANG=en_US.UTF-8）, 而client、connection、database均为latin1, 于是这一路（从终端界面执行insert到保存数据到表中）都没有编码转换，直接传输的是utf8编码后的二进制流。
怎么验证上述结论呢？ 于是决定修改中间环节的字符集,看会发生什么?
mysql> set names gbk; mysql> show variables like ‘character%‘; +--------------------------+---------------------------------------+ | Variable_name | Value | +--------------------------+---------------------------------------+ | character_set_client | gbk | | character_set_connection | gbk | | character_set_database | latin1 | | character_set_filesystem | binary | | character_set_results | gbk | | character_set_server | utf8mb4 | | character_set_system | utf8 | | character_sets_dir | /opt/mysql/server-5.6/share/charsets/ | +--------------------------+---------------------------------------+ mysql> insert into t4 select ‘\U+1F600‘; ERROR 1366 (HY000): Incorrect string value: ‘\xF0\x9F\x98\x80‘ for column ‘data‘ at row 1
分析:
现在操作系统是utf8, client、connection是gbk， 字段是latin1， 因为一开始是utf8二进制流，且client和connection均为gbk，无需转码，故只在最后当保存到表字段中时需要由utf8转为latin1，由于latin1不能解码该utf8二进制流故导致了上述报错。
若将字符集不一致的情况再往前挪一步会怎样呢？ 如下所示：
mysql> set character_set_connection = latin1; mysql> show variables like ‘character%‘; +--------------------------+---------------------------------------+ | Variable_name | Value | +--------------------------+---------------------------------------+ | character_set_client | gbk | | character_set_connection | latin1 | | character_set_database | latin1 | | character_set_filesystem | binary | | character_set_results | gbk | | character_set_server | utf8mb4 | | character_set_system | utf8 | | character_sets_dir | /opt/mysql/server-5.6/share/charsets/ | +--------------------------+---------------------------------------+
现在client和connection就不一致了，就是说需要先将utf8-->gbk-->latin1, 那么现在能成功插入emoji字符吗?
mysql> insert into t4 select ‘\U+1F600‘;
可以插入，查询结果如下：
mysql> select data,hex(data) from t4; +------+-----------+ | data | hex(data) | +------+-----------+ | ?? | 3F3F | +------+-----------+
似乎在utf8-->gbk的过程中，将utf8编码后的二进制流(f0 9f 98 80)解码成了‘??’,而‘??’能被latin1成功解析。但如何通过java程序模拟上述的转换呢？
试了下面的代码 但未能成功再现。
即将utf8转为gbk时未能得到‘??’,而是‘馃榾’。不知何故？
那假如将上例中的client与connection交换一下位置呢，如下所示：
mysql> show variables like ‘character%‘; +--------------------------+---------------------------------------+ | Variable_name | Value | +--------------------------+---------------------------------------+ | character_set_client | latin1 | | character_set_connection | gbk | | character_set_database | latin1 | | character_set_filesystem | binary | | character_set_results | gbk | | character_set_server | utf8mb4 | | character_set_system | utf8 | | character_sets_dir | /opt/mysql/server-5.6/share/charsets/ | +--------------------------+---------------------------------------+
现在的转化流变成这样了：utf8-->latin1-->gbk-->latin1， 从之前的经验似乎可以预测进行第一步转化时就应该报错（Incorrect string value: ‘\xF0\x9F\x98\x80‘ for column ‘data‘ at row 1），但实际情况是：
mysql> insert into t4 select ‘\U+1F600‘; Query OK, 1 row affected (0.01 sec) mysql> select data,hex(data) from t4; +------+-----------+ | data | hex(data) | +------+-----------+ | ?? | 3F3F | | ???? | 3F3F3F3F | +------+-----------+
并未报错仍能成功插入， 似乎只要不是最后一步往表里插入记录就不会报错，但这次变成4个问号了。
这次用java程序模拟的结果如下所示：
注：
- 为什么用Cp1252表示latin1？
主要是参考了该表格：
Table 5.3 MySQL to Java Encoding Name Translations
时间： 2024-12-14 07:14:40