postgresql遇到的性能问题

问题SQL

scwksmlcls.wk_cls_c
, scwklrgcls.wk_lrg_cls_nm
, scwkmdlcls.wk_mdl_cls_nm
, scwksmlcls.wk_sml_cls_nm
, scwksmlcls.wk_cls_rmk
FROM
screqrsnsws
INNER JOIN scwkclsreqrsnsws
ON scwkclsreqrsnsws.req_rsn_id = screqrsnsws.req_rsn_id
INNER JOIN scwksmlcls
ON scwksmlcls.wk_sml_cls_id = scwkclsreqrsnsws.wk_sml_cls_id
AND scwksmlcls.wk_sml_cls_id_ek IS NOT NULL
INNER JOIN scwkmdlcls
ON scwkmdlcls.wk_mdl_cls_id = scwksmlcls.wk_mdl_cls_id
AND scwkmdlcls.wk_lrg_cls_id = scwksmlcls.wk_lrg_cls_id
AND scwkmdlcls.wk_mdl_cls_id_ek IS NOT NULL
INNER JOIN scwklrgcls
ON scwklrgcls.wk_lrg_cls_id = scwkmdlcls.wk_lrg_cls_id
AND scwklrgcls.wk_lrg_cls_id_ek IS NOT NULL
WHERE
screqrsnsws.req_rsn_c = ‘996N‘
AND scwksmlcls.wk_cls_c = ‘112233‘
AND screqrsnsws.req_rsn_id_ek IS NOT NULL
ORDER BY
scwklrgcls.disp_odr
, scwkmdlcls.disp_odr
, scwksmlcls.disp_odr

-- 用到的表

select count(*) from screqrsnsws; --依赖理由 件数:58

select count(*) from scwkclsreqrsnsws; --依頼理由分類 件数:289142

select count(*) from scwksmlcls; -- 小分類 件数:289414

select count(*) from scwkmdlcls; -- 中分類 件数:285223

select count(*) from scwklrgcls; -- 大分類 件数:1962

表定义
create table score.screqrsnsws (
req_rsn_id integer not null
, req_rsn_id_ek integer
, req_rsn_c character varying(4) not null
, req_rsn_nm character varying(20) not null
, fix_assts_tagt_f character varying(1) not null
, pms_i_ymd timestamp without time zone default now() not null
, pms_i_usr character varying(32) default "current_user"() not null
, pms_i_class character varying(128)
, pms_u_ymd timestamp without time zone
, pms_u_usr character varying(32)
, pms_u_class character varying(128)
, primary key (req_rsn_id)
);

req_rsn_id、req_rsn_c、req_rsn_id_ek のindex

create table score.scwkclsreqrsnsws (
req_rsn_id integer not null
, wk_sml_cls_id integer not null
, drct_ind_h_k character varying(1) not null
, pms_i_ymd timestamp without time zone default now() not null
, pms_i_usr character varying(32) default "current_user"() not null
, pms_i_class character varying(128)
, pms_u_ymd timestamp without time zone
, pms_u_usr character varying(32)
, pms_u_class character varying(128)
, primary key (req_rsn_id,wk_sml_cls_id)
);

req_rsn_id,wk_sml_cls_idの複合index
req_rsn_id

create table score.scwksmlcls (
wk_sml_cls_id integer not null
, wk_sml_cls_id_ek integer
, work_lrg_cls_c character varying(2) not null
, wk_mdl_cls_c character varying(2) not null
, wk_sml_cls_c character varying(2) not null
, wk_cls_c character varying(6) not null
, wk_sml_cls_nm character varying(50) not null
, disp_odr integer
, wk_cls_rmk character varying(200)
, pg_dev_hndl_f character varying(1) not null
, fix_assts_tagt_f character varying(1) not null
, pms_i_ymd timestamp without time zone default now() not null
, pms_i_usr character varying(32) default "current_user"() not null
, pms_i_class character varying(128)
, pms_u_ymd timestamp without time zone
, pms_u_usr character varying(32)
, pms_u_class character varying(128)
, wk_dept_c character varying(4)
, wk_lrg_cls_id integer
, wk_mdl_cls_id integer
, primary key (wk_sml_cls_id)
);

wk_sml_cls_id,
work_lrg_cls_c,wk_mdl_cls_c,wk_sml_cls_c复合索引
wk_cls_c
wk_sml_cls_id_ek

create table score.scwkmdlcls (
wk_mdl_cls_id integer not null
, wk_mdl_cls_id_ek integer
, work_lrg_cls_c character varying(2) not null
, wk_mdl_cls_c character varying(2) not null
, wk_mdl_cls_nm character varying(50) not null
, disp_odr integer
, wk_lrg_cls_id integer
, wk_dept_c character varying(4)
, pms_i_ymd timestamp without time zone default now() not null
, pms_i_usr character varying(32) default "current_user"() not null
, pms_i_class character varying(128)
, pms_u_ymd timestamp without time zone
, pms_u_usr character varying(32)
, pms_u_class character varying(128)
, primary key (wk_mdl_cls_id)
);

wk_mdl_cls_id
work_lrg_cls_c,wk_mdl_cls_c
wk_mdl_cls_id_ek

create table score.scwklrgcls (
wk_lrg_cls_id integer not null
, wk_lrg_cls_id_ek integer
, work_lrg_cls_c character varying(2) not null
, wk_lrg_cls_nm character varyin g

通过SQL语句前加"EXPLAIN ANALYZE"执行SQL得到的执行计划

QUERY PLAN
Sort (cost=3589.88..3589.89 rows=1 width=96) (actual time=21816.121..21816.121 rows=1 loops=1)
Sort Key: scwklrgcls.disp_odr, scwkmdlcls.disp_odr, scwksmlcls.disp_odr
Sort Method: quicksort Memory: 25kB
-> Nested Loop (cost=557.47..3589.87 rows=1 width=96) (actual time=11548.596..21816.102 rows=1 loops=1)
-> Nested Loop (cost=557.47..3585.59 rows=1 width=80) (actual time=11548.550..21816.055 rows=1 loops=1)
Join Filter: (scwkclsreqrsnsws.req_rsn_id = screqrsnsws.req_rsn_id)
-> Nested Loop (cost=557.47..3585.31 rows=1 width=84) (actual time=1.472..21798.988 rows=4401 loops=1)
-> Hash Join (cost=557.47..1413.63 rows=1 width=84) (actual time=1.453..15.100 rows=4401 loops=1)
Hash Cond: ((scwksmlcls.wk_mdl_cls_id = scwkmdlcls.wk_mdl_cls_id) AND (scwksmlcls.wk_lrg_cls_id = scwkmdlcls.wk_lrg_cls_id))
-> Index Scan using i_scwksmlcls_ek on scwksmlcls (cost=0.00..823.09 rows=4408 width=55) (actual time=0.013..4.404 rows=4401 loops=1)
Index Cond: (wk_sml_cls_id_ek IS NOT NULL)
-> Hash (cost=540.67..540.67 rows=1120 width=37) (actual time=1.420..1.420 rows=1124 loops=1)
Buckets: 1024 Batches: 1 Memory Usage: 73kB
-> Index Scan using i_scwkmdlcls_ek on scwkmdlcls (cost=0.00..540.67 rows=1120 width=37) (actual time=0.047..1.051 rows=1124 loops=1)
Index Cond: (wk_mdl_cls_id_ek IS NOT NULL)
-> Index Scan using con_scwkclsreqrsnsws_pri on scwkclsreqrsnsws (cost=0.00..2171.67 rows=1 width=8) (actual time=1.140..4.948 rows=1 loops=4401)
Index Cond: (wk_sml_cls_id = scwksmlcls.wk_sml_cls_id)
-> Index Scan using i_screqrsnsws_alt2 on screqrsnsws (cost=0.00..0.27 rows=1 width=4) (actual time=0.003..0.003 rows=1 loops=4401)
Index Cond: ((req_rsn_c)::text = ‘996N‘::text)
Filter: (req_rsn_id_ek IS NOT NULL)
-> Index Scan using con_scwklrgcls_pri on scwklrgcls (cost=0.00..4.27 rows=1 width=28) (actual time=0.011..0.012 rows=1 loops=1)
Index Cond: (wk_lrg_cls_id = scwkmdlcls.wk_lrg_cls_id)
Filter: (wk_lrg_cls_id_ek IS NOT NULL)
Total runtime: 21816.279 ms

看了表的定义，索引，执行计划，认为小分类表和中分类表INNERJOIN的条件「AND scwkmdlcls.wk_lrg_cls_id = scwksmlcls.wk_lrg_cls_id」多余。

因为大中小分类表的主键是ID，用主键ID连接应该足够了。去掉此条件的执行时间会明显加快。

EXPLAIN ANALYZE SELECT
scwksmlcls.wk_cls_c
, scwklrgcls.wk_lrg_cls_nm
, scwkmdlcls.wk_mdl_cls_nm
, scwksmlcls.wk_sml_cls_nm
, scwksmlcls.wk_cls_rmk
FROM
screqrsnsws
INNER JOIN scwkclsreqrsnsws
ON scwkclsreqrsnsws.req_rsn_id = screqrsnsws.req_rsn_id
INNER JOIN scwksmlcls
ON scwksmlcls.wk_sml_cls_id = scwkclsreqrsnsws.wk_sml_cls_id
AND scwksmlcls.wk_sml_cls_id_ek IS NOT NULL
INNER JOIN scwkmdlcls
ON scwkmdlcls.wk_mdl_cls_id = scwksmlcls.wk_mdl_cls_id
-- AND scwkmdlcls.wk_lrg_cls_id = scwksmlcls.wk_lrg_cls_id
AND scwkmdlcls.wk_mdl_cls_id_ek IS NOT NULL
INNER JOIN scwklrgcls
ON scwklrgcls.wk_lrg_cls_id = scwkmdlcls.wk_lrg_cls_id
AND scwklrgcls.wk_lrg_cls_id_ek IS NOT NULL
WHERE
screqrsnsws.req_rsn_c = ‘996N‘
AND scwksmlcls.wk_cls_c = ‘112233‘
AND screqrsnsws.req_rsn_id_ek IS NOT NULL
ORDER BY
scwklrgcls.disp_odr
, scwkmdlcls.disp_odr
, scwksmlcls.disp_odr

QUERY PLAN
Sort (cost=5598.19..5598.20 rows=1 width=96) (actual time=3.270..3.270 rows=1 loops=1)
Sort Key: scwklrgcls.disp_odr, scwkmdlcls.disp_odr, scwksmlcls.disp_odr
Sort Method: quicksort Memory: 25kB
-> Nested Loop (cost=949.97..5598.18 rows=1 width=96) (actual time=3.254..3.260 rows=1 loops=1)
-> Nested Loop (cost=949.97..5597.80 rows=1 width=76) (actual time=3.245..3.249 rows=1 loops=1)
-> Hash Join (cost=949.97..5429.77 rows=77 width=47) (actual time=3.237..3.241 rows=1 loops=1)
Hash Cond: (scwkclsreqrsnsws.wk_sml_cls_id = scwksmlcls.wk_sml_cls_id)
-> Nested Loop (cost=71.78..4512.75 rows=5075 width=4) (actual time=0.038..0.041 rows=1 loops=1)
-> Seq Scan on screqrsnsws (cost=0.00..2.71 rows=1 width=4) (actual time=0.023..0.025 rows=1 loops=1)
Filter: ((req_rsn_id_ek IS NOT NULL) AND ((req_rsn_c)::text = ‘996N‘::text))
-> Bitmap Heap Scan on scwkclsreqrsnsws (cost=71.78..4443.07 rows=5357 width=8) (actual time=0.012..0.012 rows=1 loops=1)
Recheck Cond: (req_rsn_id = screqrsnsws.req_rsn_id)
-> Bitmap Index Scan on con_scwkclsreqrsnsws_pri (cost=0.00..70.44 rows=5357 width=0) (actual time=0.008..0.008 rows=1 loops=1)
Index Cond: (req_rsn_id = screqrsnsws.req_rsn_id)
-> Hash (cost=823.09..823.09 rows=4408 width=51) (actual time=3.186..3.186 rows=4401 loops=1)
Buckets: 1024 Batches: 1 Memory Usage: 313kB
-> Index Scan using i_scwksmlcls_ek on scwksmlcls (cost=0.00..823.09 rows=4408 width=51) (actual time=0.011..2.002 rows=4401 loops=1)
Index Cond: (wk_sml_cls_id_ek IS NOT NULL)
-> Index Scan using con_scwkmdlcls_pri on scwkmdlcls (cost=0.00..2.17 rows=1 width=37) (actual time=0.005..0.005 rows=1 loops=1)
Index Cond: (wk_mdl_cls_id = scwksmlcls.wk_mdl_cls_id)
Filter: (wk_mdl_cls_id_ek IS NOT NULL)
-> Index Scan using con_scwklrgcls_pri on scwklrgcls (cost=0.00..0.37 rows=1 width=28) (actual time=0.009..0.009 rows=1 loops=1)
Index Cond: (wk_lrg_cls_id = scwkmdlcls.wk_lrg_cls_id)
Filter: (wk_lrg_cls_id_ek IS NOT NULL)
Total runtime: 3.364 ms

postgresql和oracle的执行计划还真是不一样。

另外还发现有意思的事情，业务上“screqrsnsws.req_rsn_c = scwksmlcls.wk_dept_c”这个是成立的，加入这个条件后语句也能变快，原因是WHERE条件里的"screqrsnsws.req_rsn_c = ‘996N‘ "的条件，执行计划里会"scwksmlcls.wk_dept_c = ‘996N‘",内层表的选取数据变小。

EXPLAIN ANALYZE
SELECT
scwksmlcls.wk_cls_c
, scwklrgcls.wk_lrg_cls_nm
, scwkmdlcls.wk_mdl_cls_nm
, scwksmlcls.wk_sml_cls_nm
, scwksmlcls.wk_cls_rmk
FROM
screqrsnsws
INNER JOIN scwkclsreqrsnsws
ON scwkclsreqrsnsws.req_rsn_id = screqrsnsws.req_rsn_id
AND screqrsnsws.req_rsn_id_ek IS NOT NULL
INNER JOIN scwksmlcls
ON scwksmlcls.wk_sml_cls_id = scwkclsreqrsnsws.wk_sml_cls_id
AND screqrsnsws.req_rsn_c = scwksmlcls.wk_dept_c
AND scwksmlcls.wk_sml_cls_id_ek IS NOT NULL
INNER JOIN scwkmdlcls
ON scwkmdlcls.wk_mdl_cls_id = scwksmlcls.wk_mdl_cls_id
AND scwkmdlcls.wk_lrg_cls_id = scwksmlcls.wk_lrg_cls_id
AND scwkmdlcls.wk_mdl_cls_id_ek IS NOT NULL
INNER JOIN scwklrgcls
ON scwklrgcls.wk_lrg_cls_id = scwkmdlcls.wk_lrg_cls_id
AND scwklrgcls.wk_lrg_cls_id_ek IS NOT NULL
WHERE
screqrsnsws.req_rsn_c = ‘996N‘
AND screqrsnsws.req_rsn_id_ek IS NOT NULL
ORDER BY
scwklrgcls.disp_odr
, scwkmdlcls.disp_odr
, scwksmlcls.disp_odr

QUERY PLAN
Sort (cost=859.96..859.97 rows=1 width=96) (actual time=2.025..2.025 rows=1 loops=1)
Sort Key: scwklrgcls.disp_odr, scwkmdlcls.disp_odr, scwksmlcls.disp_odr
Sort Method: quicksort Memory: 25kB
-> Nested Loop (cost=0.00..859.95 rows=1 width=96) (actual time=1.050..2.013 rows=1 loops=1)
Join Filter: (scwksmlcls.wk_lrg_cls_id = scwkmdlcls.wk_lrg_cls_id)
-> Nested Loop (cost=0.00..855.66 rows=1 width=79) (actual time=1.041..2.003 rows=1 loops=1)
-> Nested Loop (cost=0.00..851.35 rows=1 width=87) (actual time=1.012..1.974 rows=1 loops=1)
-> Index Scan using con_screqrsnsws_pri on screqrsnsws (cost=0.00..8.68 rows=1 width=9) (actual time=0.015..0.030 rows=1 loops=1)
Filter: ((req_rsn_id_ek IS NOT NULL) AND (req_rsn_id_ek IS NOT NULL) AND ((req_rsn_c)::text = ‘996N‘::text))
-> Nested Loop (cost=0.00..842.67 rows=1 width=88) (actual time=0.995..1.942 rows=1 loops=1)
-> Index Scan using i_scwksmlcls_ek on scwksmlcls (cost=0.00..834.11 rows=2 width=60) (actual time=0.988..1.934 rows=1 loops=1)
Index Cond: (wk_sml_cls_id_ek IS NOT NULL)
Filter: ((wk_dept_c)::text = ‘996N‘::text)
-> Index Scan using con_scwklrgcls_pri on scwklrgcls (cost=0.00..4.27 rows=1 width=28) (actual time=0.004..0.005 rows=1 loops=1)
Index Cond: (wk_lrg_cls_id = scwksmlcls.wk_lrg_cls_id)
Filter: (wk_lrg_cls_id_ek IS NOT NULL)
-> Index Scan using con_scwkclsreqrsnsws_pri on scwkclsreqrsnsws (cost=0.00..4.29 rows=1 width=8) (actual time=0.028..0.028 rows=1 loops=1)
Index Cond: ((req_rsn_id = screqrsnsws.req_rsn_id) AND (wk_sml_cls_id = scwksmlcls.wk_sml_cls_id))
-> Index Scan using con_scwkmdlcls_pri on scwkmdlcls (cost=0.00..4.28 rows=1 width=37) (actual time=0.005..0.006 rows=1 loops=1)
Index Cond: (wk_mdl_cls_id = scwksmlcls.wk_mdl_cls_id)
Filter: (wk_mdl_cls_id_ek IS NOT NULL)
Total runtime: 2.143 ms

为了能大概看懂这执行计划，还特意搜索了一下"postgresql的explain命令详解"，那个挺有用。

时间： 2024-10-23 16:46:50

postgresql遇到的性能问题的相关文章

PostgreSQL hstore 列性能提升一例

PostgreSQL 支持hstore 来存放KEY->VALUE这类数据, 事实上也相似于ARRAY或者JSON类型.  要高效的使用这类数据,当然离不开高效的索引.我们今天就来看看两类不同的索引对于同一种检索请求的性能问题. 假如我们有这样一个原始表.基于str1字段有一个BTREE索引. t_girl=# \d status_check; Table "ytt.status_check" Column | Type | Modifiers --------+--------

Elasticsearch与Postgresql大数据检索性能对比

Elasticsearch与Postgresql数据检索性能对比与融合一般来说,影响数据库最大的性能问题有两个,一个是对数据库的读写操作,一个是数据库中的数据太大导致操作慢,对于前者我们可以适当借助缓存来减少一部分读操作,而针对一些复杂的报表分析和搜索可以交给hadoop和elasticsearch,对于写并发大,读也并发大,我们可以考虑分库分表,主从读写分离或者两者结合等方式来提高并发性和时效性,例如PG大并发写,大数据查看可以用elasticsearch与PG数据同步来读,可以启到很好的效果

PostgreSQL 参数调整(性能优化)

昨天分别在外网和无外网环境下安装PostgreSQL,有外网环境下安装的相当顺利.但是在无外网环境下就是两个不同的概念了,可谓十有八折.感兴趣的同学可以搭建一下. PostgreSQL安装完成后第一件事便是做相关测试,然后调整参数. /*CPU 查看CPU型号*/ cat /proc/cpuinfo | grep name | cut -f2 -d: | uniq -c /*查看物理CPU个数*/ cat /proc/cpuinfo | grep "physical id" | sor

[转帖]PostgreSQL 参数调整(性能优化)

https://www.cnblogs.com/VicLiu/p/11854730.html 知道一个 shared_pool 文章写的挺好的 还没仔细看 昨天分别在外网和无外网环境下安装PostgreSQL,有外网环境下安装的相当顺利.但是在无外网环境下就是两个不同的概念了,可谓十有八折.感兴趣的同学可以搭建一下. PostgreSQL安装完成后第一件事便是做相关测试,然后调整参数. /*CPU 查看CPU型号*/ cat /proc/cpuinfo | grep name | cut -f2

postgresql not in性能问题

postgresql的not in的执行计划,执行时间超出预期: 替代方案 1.not exists 2.左联接 原文地址:https://www.cnblogs.com/lilei2blog/p/9540295.html

CentOS7下安装并简单设置PostgreSQL笔记

为什么是PostgreSQL? 在.NET Core诞生之前,微软平台上最常见的开发组件便是.NET Framework + SQL Server了,但是现在.NET Core终于让跨平台部署成为了现实,这一模式还会常见吗?个人认为这一黄金搭档很可能会日渐势微了,因为未来很多的.NET应用将部署在Linux上,为了使用SQL Server,人们又部署一个Windows环境吗?想想都觉得不大可能,那么为Linux上的.NET Core选择一款合适的数据库就变得非常重要.其实也不难选,因为就两个选项

PostgreSQL概述

PostgreSQL概述 概要介绍: PostgreSQL是一个功能强大的开源数据库系统.经过长达15年以上的积极开发和不断改进,PostgreSQL已在可靠性.稳定性.数据一致性等获得了业内极高的声誉.目前PostgreSQL可以运行在所有主流操作系统上,包括Linux.Unix(AIX.BSD.HP-UX.SGI IRIX.Mac OS X.Solaris和Tru64)和Windows.PostgreSQL是完全的事务安全性数据库,完整地支持外键.联合.视图.触发器和存储过程(并支持多种语言

postgreSQL 学习资料

PostgreSQL 9.3性能优化培训系列课程 http://edu.51cto.com/course/course_id-2278.html

Advacned Puppet: Puppet Master性能调优

本文是Advanced Puppet系列的第一篇:Puppet master性能调优,谈一谈如何优化和提高C/S架构下master端的性能. 故事情节往往惊人地类似:你是一名使用Puppet管理线上业务的DevOps工程师,随着公司的业务发展,你所管理的集群规模日益扩大.终于某一天,你突然发现执行一次puppet agent -vt的时间长得不可接受,多台agent并发运行时竟然会有节点运行失败,往日从来没有考虑过Puppet的性能居然成为了瓶颈……首先要恭喜你,因为Puppet Master端