Common Scenarios to avoid with DataWarehousing

Database Design


Rule


Description


Value


Source


Problem Description


1


Excessive sorting and RID lookup operations   should be reduced with covered indexes.


Sys.dm_exec_sql_text

Sys.dm_exec_cached_plans


Large data warehouse can benefit from more   indexes. Indexes can be used to cover queries and avoid sorting. The cost of   index overhead is only paid when data is loaded.


2


Excessive fragmentation:

Average fragmentation_in_percent should be   <25%


>25%


sys.dm_db _index_physical_stats


Reducing index fragmentation through index   rebuilds can benefit big range scans, common in data warehouse and Reporting   scenarios.


3


Scans and ranges are common. Look for missing   indexes


>= 1


Perfmon object

SQL Server Access Methods

Sys.dm_db_missing_index_group_stats

Sys.dm_db_missing_index_groups

Sys.dm_db_missing_index_details


A missing index flushes the cache.


4


Unused Indexes should be avoided


If an index is NEVER used, it will not appear   in the DMV sys.dm_db_index_usage_stats


Index maintenance for unused indexes should be   avoided.

Resource issue: CPU


Rule


Description


Value


Source


Problem Description


1


Signal Waits


> 25%


Sys.dm_os_wait_stats


Time in runnable queue is pure CPU wait.


2


Avoid plan reuse


> 25%


Perfmon object

SQL Server Statistics


Data warehouse has fewer transactions than   OLTP, each with significantly bigger IO. Therefore, having the correct plan   is more important than reusing a plan. Unlike OLTP, data warehouse queries   are not identical.


3


Parallelism: Cxpacket waits


<10%


Sys.dm_os_wait_stats


Parallelism is desirable in data warehouse or reporting   workloads.

Resource issue: Memory


Rule


Description


Value


Source


Problem Description


1


Memory grants pending


>1


Perfmon object

SQL Server Memory Manager


Memory grant not available for query to run.  Check for

Sufficient memory and page life expectancy.


2


Page life expectancy


Drops by 50%


Perfmon object

SQL Server Buffer Manager


Page life expectancy is the average number of   seconds a data page stays in cache.    Low values could indicate a cache flush that is caused by a big read.

Look for possible missing index.

Resource issue: IO


Rule


Description


Value


Source


Problem Description


1


Average Disk sec/read


>20 ms


Perfmon object

Physical Disk


Reads should take 4-8ms without any IO   pressure.


2


Average Disk sec/write


>20 ms


Perfmon object

Physical Disk


Writes (sequential) can be as fast as 1 ms for   transaction log.


3


Big scans


>1


Perfmon object

SQL Server Access Methods


A missing index flushes the cache.


4


If Top 2 values for wait stats are any of the   following:

ASYNCH_IO_COMPLETION

IO_COMPLETION

LOGMGR

WRITELOG

PAGEIOLATCH_x


Top 2


Sys.dm_os_wait_stats


If top 2 wait_stats values include IO, there   is an IO bottleneck

Resource issue: Blocking


Rule


Description


Value


Source


Problem Description


1


Block percentage


>2%


Sys.dm_db_index_operational_stats


Frequency of blocks.


2


Block process report


30 sec


Sp_configure, profiler


Report of statements.


3


Average Row Lock Waits


>100ms


Sys.dm_db_index_operational_stats


Duration of blocks.


4


If Top 2 values for   wait stats are any of the following:

LCK_M_BU

LCK_M_IS

LCK_M_IU

LCK_M_IX

LCK_M_RIn_NL

LCK_M_RIn_S

LCK_M_RIn_U

LCK_M_RIn_X

LCK_M_RS_S

LCK_M_RS_U

LCK_M_RX_S

LCK_M_RX_U

LCK_M_RX_X

LCK_M_S

LCK_M_SCH_M

LCK_M_SCH_S

LCK_M_SIU

LCK_M_SIX

LCK_M_U

LCK_M_UIX

LCK_M_X


Top 2


Sys.dm_os_wait_stats


If top 2 wait_stats   values include IO, there is a blocking bottleneck.

Consider using row   versioning to minimize shared locking blocks.

Exactly the opposite of OLTP applications, reporting or relational data warehouse applications are characterized by small numbers of (different) big transactions. These are frequently SELECT intensive operations. The implications are significant for database design, resource usage, and system performance.

Reporting and data warehouse performance objectives are as follows:

  1. Data warehouse and relational data warehouse designs can have more indexes as the cost of index maintenance is paid only one time, during the batch update process.
  2. Plan reuse should generally be avoided. Plan reuse may result in picking up a plan that was good for some other query (with different data distribution), but may not be good for this query.  The time taken for plan generation of a large DataWarehouse query is not nearly as important as having the right plan.
  3. Sorts can and should be minimized with correct index usage.
  4. Missing index situations should be investigated and corrected.
  5. Large IOs such as range scans benefits from on disk contiguity. Index fragmentation should be frequently monitored and kept to a minimum with index rebuilds.
  6. Blocking is generally uncommon as most data warehouse transactions are read operations.
  7. Parallelism is generally desirable for data warehouse applications.
时间: 2024-10-18 05:53:29

Common Scenarios to avoid with DataWarehousing的相关文章

Common scenarios to avoid in OLTP

Database Design Rule Description Value Source Problem Description 1 High Frequency queries having a high number of   table joins. >4 Sys.dm_exec_sql_text Sys.dm_exec_cached_plans High frequency queries with lots of joins can be   too normalized for h

Url Rewrite 再说Url 重写

前几天看到园子里一篇关于 Url 重写的文章<获取ISAPI_Rewrite重写后的URL>, URL-Rewrite 这项技术早已不是一项新技术了,这个话题也已经被很多人讨论过多次.搜索一下URL-Rewrite可以找到很多URL-Rewrite方面的文章和组件,自己以前也多次接触过这个东东,也来说说吧. ScottGu 有一篇非常经典的 URL-Rewrite Blog Tip/Trick: Url Rewriting with ASP.NET http://weblogs.asp.net

Android - 抑制lint的Android XML的警告:tools:ignore

抑制lint的Android XML的警告:tools:ignore 本文地址:http://blog.csdn.net/caroline_wendy Android的XML经常会出现警告,对于一个良好的程序,应该认真对待所有的警告. 除非我们可以确认警告,才可以排除. 显示所有警告的方法:Analyze -> Inspect Code; 就可以检查出所有的警告: 抑制警告使用: tools:ignore. // 忽略全部 xmlns:tools="http://schemas.andro

Processing Images

https://developer.apple.com/library/content/documentation/GraphicsImaging/Conceptual/CoreImaging/ci_tasks/ci_tasks.html#//apple_ref/doc/uid/TP30001185-CH3-TPXREF101 Processing images means applying filters-an image filter is a piece of software that

c++ cast operator(类型转换操作符)

复制于:http://xiaochonganty.blog.163.com/blog/static/48527932008931104132748/ New Cast Operators Originally, the C++ standardization committee wanted to deprecate C-style casting, thereby enforcing the use of the new cast operators exclusively. However,

javascript中null和undefined的区别到底是什么?

8年前我开始学习js的时候,对我来说比较诡异的一个事情是undefined和null都代表空值.那么他们之间明确的不同点是什么呢?他们都能去定义空值,而且null == undefined的值也是TRUE. 大部分现代语言像Ruby,Python,或者Java都只有一个空值nil 或者null,  这是很明智的方法. 而js中,如果一个变量或者一个对象没有进行初始化,(编译器)就会返回一个undefined. 例如: let company; company; // => undefined l

Model-View-ViewModel (MVVM) Explained 转摘自:http://www.wintellect.com/blogs/jlikness/model-view-viewmodel-mvvm-explained

The purpose of this post is to provide an introduction to the Model-View-ViewModel (MVVM) pattern. While I've participated in lots of discussions online about MVVM, it occurred to me that beginners who are learning the pattern have very little to go

Using Sessions and Session Persistence---reference

Using Sessions and Session Persistence The following sections describe how to set up and use sessions and session persistence: Overview of HTTP Sessions Setting Up Session Management Configuring Session Persistence Using URL Rewriting Instead of Cook

Mojo Migration Guide

For Developers? > ?Design Documents? > ?Mojo? > ? Mojo Migration Guide 目录 1 Summary 2 How do I migrate my IPC to Mojo? 2.1 Don't panic 2.2 Read the Mojo documentation 2.3 Claim your message(s) 2.4 Convert your Chrome IPCs to Mojo 2.4.1 Convert th