MySql 数据导出/导入一些Tips

最近接触MySql比较多,借此记录下一些知识点.

Tips:

1.数据导出为.sql文件时,会对单引号 ‘ 进行转义处理,最终INSERT 语句中的为 \‘

2.插入不重复数据(insert if not exist)时,可以用 INSERT IGNORE或者REPLACE

INSERT IGNORE:   插入重复则失败,但会忽略,继续执行下一条.

REPLACE:    插入重复会直接覆盖,没有则新增.

以上两条均会导致自增ID不连续问题,INSERT失败会自增,REPLACE是先删除后INSERT.所以自增ID都会不连续

ID不连续参考:http://blog.csdn.net/liyong199012/article/details/21516817

引用https://stackoverflow.com/questions/1361340/how-to-insert-if-not-exists-in-mysql

18th October 2007

To start: as of the latest MySQL, syntax presented in the title is not
possible. But there are several very easy ways to accomplish what is
expected using existing functionality.

There are 3 possible solutions: using INSERT IGNORE, REPLACE, or
INSERT … ON DUPLICATE KEY UPDATE.

Imagine we have a table:

CREATE TABLE `transcripts` (
`ensembl_transcript_id` varchar(20) NOT NULL,
`transcript_chrom_start` int(10) unsigned NOT NULL,
`transcript_chrom_end` int(10) unsigned NOT NULL,
PRIMARY KEY (`ensembl_transcript_id`)
) ENGINE=InnoDB DEFAULT CHARSET=latin1;

Now imagine that we have an automatic pipeline importing transcripts meta-data from Ensembl, and that due to various reasons the pipeline might be broken at any step of execution. Thus, we need to ensure two things: 1) repeated executions of the pipeline will not destroy our database, and 2) repeated executions will not die due to ‘duplicate primary key’ errors.

Method 1: using REPLACE

It’s very simple:

REPLACE INTO `transcripts`
SET `ensembl_transcript_id` = ‘ENSORGT00000000001′,
`transcript_chrom_start` = 12345,
`transcript_chrom_end` = 12678;

If the record exists, it will be overwritten; if it does not yet exist, it will be created. However, using this method isn’t efficient for our case: we do not need to overwrite existing records, it’s fine just to skip them.

Method 2: using INSERT IGNORE Also very simple:

INSERT IGNORE INTO `transcripts`
SET `ensembl_transcript_id` = ‘ENSORGT00000000001′,
`transcript_chrom_start` = 12345,
`transcript_chrom_end` = 12678;

Here, if the ‘ensembl_transcript_id’ is already present in the database, it will be silently skipped (ignored). (To be more precise, here’s a quote from MySQL reference manual: “If you use the IGNORE keyword, errors that occur while executing the INSERT statement are treated as warnings instead. For example, without IGNORE, a row that duplicates an existing UNIQUE index or PRIMARY KEY value in the table causes a duplicate-key error and the statement is aborted.”.) If the record doesn’t yet exist, it will be created.

This second method has several potential weaknesses, including non-abortion of the query in case any other problem occurs (see the manual). Thus it should be used if previously tested without the IGNORE keyword.

There is one more option: to use INSERT … ON DUPLICATE KEY UPDATE syntax, and in the UPDATE part just do nothing do some meaningless (empty) operation, like calculating 0+0 (Geoffray suggests doing the id=id assignment for the MySQL optimization engine to ignore this operation). Advantage of this method is that it only ignores duplicate key events, and still aborts on other errors.

As a final notice: this post was inspired by Xaprb. I’d also advise to consult his other post on writing flexible SQL queries.

时间: 2024-10-11 11:43:01

MySql 数据导出/导入一些Tips的相关文章

MySQL数据导出导入【转】

MySQL基础 关于MySQL数据导出导入的文章,目的有二: 1.备忘 2.供开发人员测试 工具 mysqlmysqldump 应用举例 导出 导出全库备份到本地的目录 mysqldump -u$USER -p$PASSWD -h127.0.0.1 -P3306 --routines --default-character-set=utf8 --lock-all-tables --add-drop-database -A > db.all.sql 导出指定库到本地的目录(例如mysql库) my

关于MySQL数据导出导入

工具 mysqlmysqldump 应用举例 导出 导出全库备份到本地的目录 mysqldump -u$USER -p$PASSWD -h127.0.0.1 -P3306 --routines --default-character-set=utf8 --lock-all-tables --add-drop-database -A > db.all.sql 导出指定库到本地的目录(例如mysql库) mysqldump -u$USER -p$PASSWD -h127.0.0.1 -P3306 -

mysql数据导出导入

1. 从mysql查询出数据导入 txt 文件 mysql -h 10.3.20.251 -u addata_r -P 3417 -pARreBOEhw9MijIEN_eP6BYKOxkTikUnl ad_data -e "select id, advertiser_id,date,cost,0 as cash_cost, 0 as reward_cost, 0 as return_goods_cost, income,balance, create_time, ad_count, 'show'

MySQL数据导出与导入

发一篇基础的,关于MySQL数据导出导入的文章,目的有二: 1.备忘 2.供开发人员测试 工具 mysql/source 导入mysqldump 导出 应用举例 导出 导出全库备份到本地的目录 mysqldump -u$USER -p$PASSWD -h127.0.0.1 -P3306 --routines --default-character-set=utf8 --lock-all-tables --add-drop-database -A > db.all.sql 导出指定库到本地的目录(

用命令从mysql中导出/导入表结构及数据

在命令行下mysql的数据导出有个很好用命令mysqldump,它的参数有一大把,可以这样查看:mysqldump最常用的:mysqldump -uroot -pmysql databasefoo table1 table2 > foo.sql这样就可以将数据库databasefoo的表table1,table2以sql形式导入foo.sql中,其中-uroot参数表示访问数据库的用户名是root,如果有密码还需要加上-p参数Eg: C:\Users\jack> mysqldump -uroo

用mapreduce实现将mysql数据导出到HDFS上

因为业务需要,需要将一批mysql数据导入到HBASE,现在先将数据从Mysql导出到HDFS. 版本:hadoop CDH4.5,Hbase-0.946 1.实体类 YqBean 是我的实体类,请根据自己需要修改,实体类需要 implements Writable, DBWritable. 2.MR实现 import java.io.IOException; import java.util.Iterator; import org.apache.hadoop.conf.Configurati

mysql数据导出权限问题

mysql数据导出的方法有非常多,比如mysqldump, mysql -e 'sql' > file, 这些都能够非常方便的导出数据,但是在使用普通用户导出数据的时候,出现了问题. 1 select * into outfile "file_path" from my_table 上面的语句也是mysql导出数据的一种方式,在使用普通用户运行语句时.出现了一下错误: 1 ERROR 1045 (28000): Access denied for user 'my_user'@'

Oracle如何实现创建数据库、备份数据库及数据导出导入的一条龙操作

Oracle如何实现创建数据库.备份数据库及数据导出导入的一条龙操作 Oracle中对数据对象和数据的管理,无疑都是使用PL/SQL Developer来进行管理,该工具也提供给我们很多方便.快捷的操作,使得我们不再为Oracle本身丑陋.难用的UI而抱怨.由于我们一般都是建建表.查查数据的操作居多,较少会考虑系统的整个Oracle的完整备份操作.但是在我们一些发布操作中,我们必须考虑如何把Oracle的对象.表数据导出到Sql脚本中,并且把创建Oracle表空间.创建Oracle数据库的操作也

[转]不同版本的SQL Server之间数据导出导入,降级还原等

鉴于大家经常遇到数据库不同版本之间的数据导出导入,降级还原等问题,Philo童鞋搜集了两则文章. 转载到此,希望对大家有用.对贡献者表示感谢! 不同版本的SQL Server之间数据导出导入的方法及性能比较 SQLServer数据库降级方法详解(百度经验)