ETL运行时出现 'ACTIVE_TRANSACTION' FULL 错误

看到ETL Step在进行 retry attemp,打开Log,看到错误信息:“The transaction log for database ‘xxxx‘ is full due to ‘ACTIVE_TRANSACTION‘.”

‘ACTIVE_TRANSACTION‘ full,说明 log 文件已经不能再容纳新的transaction。

Log 文件从逻辑行划分为多个VLF(Virtual Log Files),以VLF为resule unit,如果一个vlf 存在active transaction,那么vlf就不能被截断和重用。如果log文件中已经耗尽,没有可用的vlf,那么系统就不能处理新增的事务,并抛出 ‘ACTIVE_TRANSACTION‘ full 的错误。

It cannot reuse a virtual log file that contains transaction log records of a transaction that is still active.SQL Server will return a log_reuse_wait_desc value of ACTIVE_TRANSACTION if it runs out of virtual log because of that.

出现‘ACTIVE_TRANSACTION‘ full 错误的可能原因是Transaction 运行的时间太长,导致其他transaction虽然被commint,但是其占用的VLF仍然被标记为Active,不能被truncate和reuse,当log文件中没有可用的VLF,而SQL Server Engine又要处理新增的Transaction时,系统报错。此时,Log File的可用空间集合为0,Log File Size 达到最大值,不能增长。

这次ETL遇到的情况有所不同,DB的Recovery Mode 是 simple,log文件还能够持续增长,而Disk 还有很大的可用空间,很诡异。

可用的Scenario 如下:

  1. Issue a CHECKPOINT command to free up log space in the log file.
  2. Check the available log space with DBCC SQLPERF(‘logspace‘). If only a small percentage of your log file is actually been used, you can try a DBCC SHRINKFILE (LogFileName,1)  command. This can however possibly introduce corruption in your database.

引用文档《Factors That Can Delay Log Truncation

Log truncation frees space in the log file for reuse by the transaction log. Because the active part of the log cannot be truncated or removed by shrinking, truncation can be delayed when log records remain active for a long time.

You can discover what, if anything, is preventing log truncation by using the log_reuse_wait and log_reuse_wait_desc columns of the sys.databases catalog view.

引用文档《Log Reuse Waits Explained: ACTIVE_TRANSACTION

SQL Server will return a log_reuse_wait_desc value of ACTIVE_ TRANSACTION if it runs out of virtual log files because of an open transaction. Open transactions prevent virtual log file reuse, because the information in the log records for that transaction might be required to execute a rollback operation.

To prevent this log reuse wait type, make sure you design you transactions to be as short lived as possible and never require end user interaction while a transaction is open.

To resolve this wait, you have to commit or rollback all transactions. The safest strategy is to just wait until the transactions finish themselves. Well-designed transactions are usually short lived, but there are many reasons that can turn a normal transaction into a log running one. If you cannot afford to wait for an extra-long running transaction to finish, you might have to kill its session. However, that will cause that transaction to be rolled back. Keep this in mind when designing your application and try to keep all transactions as short as possible.

One common design mistake that can lead to very long running transactions is to require user interaction while the transaction is open. If the person that started the transaction went to lunch while the system is waiting for a response, this transaction can turn into a very-long-running transaction. During this time other transactions, if they are not blocked by this one, will eventually fill up the log and cause the log file to grow.

推荐阅读:

Log Reuse Waits Explained: CHECKPOINT

Log Reuse Waits Explained: LOG_BACKUP

Log Reuse Waits Explained: ACTIVE_BACKUP_OR_RESTORE

Log Reuse Waits Explained: ACTIVE_TRANSACTION

Log Reuse Waits Explained: DATABASE_MIRRORING

Log Reuse Waits Explained: REPLICATION

Log Reuse Waits Explained: DATABASE_SNAPSHOT_CREATION

Log Reuse Waits Explained: LOG_SCAN

ETL运行时出现 'ACTIVE_TRANSACTION' FULL 错误

时间: 2024-08-03 10:45:39

ETL运行时出现 'ACTIVE_TRANSACTION' FULL 错误的相关文章

BI - ETL运行时监控

对于任何事物而言,监督都是提高自身的有效手段,BI也是如此.从我个人的经验而言,BI的监督可以分为两类(欢迎拍砖讨论):运行时监督(Runtime Monitoring)与数据仓库健康状况的监督(DW Healthy Monitoring): 1. 运行时监督 所谓运行时监督是指监督数据从数据源到流到数据仓库的过程,通俗来讲就是监督ETL的执行过程.我相信绝大多数的BI系统都具有该功能,区别只在于实现监控的方式以及监控信息的详细程度 2. 数据仓库健康状况监督 所谓数据仓库的健康状况监督,其实就

解决Android Studio 3.x版本的安装时没有SDK,运行时出现SDK tools错误

好久没更新了,最近手机上的闹钟APP没一个好用的,所以想自己写个. 那Android开发环境搭起来,注意先装好jdk. 1.安装Android Studio google的Android开发网站已经有中文版本,不用跨越GFW就能访问:https://developer.android.google.cn/ 乍一看,感觉Android Studio的安装包小了不少,既然官方提供,下载来安装再说. 安装过程中组件选择时就两个,让我明白为啥安装包小了,但也多了另一个疑问:SDK去哪了?(因为之前2.x

ArcMap运行时出现Runtime Error错误的解决方案

运行ArcMap时弹出错误提示:"Microsoft Visual C++ Runtime Library. Runtime 1.开始->运行->regsvr32 "C:\Program Files\Common Files\Microsoft Shared\DAO\dao360.dll" 2.如果问题仍没有解决,将下面代码复制到文本编辑器中,并保存为RegisterArcgis.bat,双击运行 @echo Registering files... @for %

Jupyter运行时出现下面的错误:Unexpected error while saving file: arma/Untitled.ipynb [Errno 13] Permission denied:

装好jupyter 发现了notebook 项下面  只支持python2,,使用pip3安装,, 我们接下来点击 python3 创建notebook,报错,没有保存权限, 好,我们继续使用sudo运行 提示:running as root is  not recommended........... 给安装文件运行一下代码 sudo chmod 777 ~/.local/share/jupyter/ cd ~/.local/share/jupyter/ ls sudo chmod 777 r

操作可能会破坏运行时稳定性的解决办法

如果你用了ANTS Performance Profiler的话,并且选择了“Line-Level”就有可能出现下面问题了: System.Security.VerificationException: 操作可能会破坏运行时稳定性. 老外的解决方法:http://stackoverflow.com/questions/378895/operation-could-destabilize-the-runtimeThis happens regularily to me when using ANT

对抗静态分析——运行时修复dex

对抗静态分析——运行时修复dex 本文来源:i春秋社区-分享你的技术,为安全加点温度 零.写在前面 这个系列本来题目想写对抗反编译,可是想想对抗反编译的这个范围有点大,总结如下 灵魂作图 <ignore_js_op> 自己比较熟悉的是静态分析,所以就从这里入手吧,省的吹大了,挖了坑填不上那就不好了.当然也会写些动态分析的东西. 废话少说,开始吧,最近又把脱壳拿出来看了,就从这里开始吧 一.理论(总要解释一下) 壳是一个比较大的概念,发展的比较迅速.未来(或者现在?)加壳的主流会逐渐变为第四代壳

java运行时异常与一般异常有何异同?

Throwable 是所有 Java 程序中错误处理的父类 ,有两种资类: Error 和 Exception . Error :表示由 JVM 所侦测到的无法预期的错误,由于这是属于 JVM 层次的严重错误 ,导致 JVM 无法继续执行,因此,这是不可捕捉到的,无法采取任何恢复的操作,顶多只能显示错误信息. Exception :表示可恢复的例外,这是可捕捉到的. Java 提供了两类主要的异常 :runtime exception 和 checked exception . checked 

运行时异常和checked异常

运行时异常在运行期间才能被检查出来,一般运行期异常不需要处理.Checked异常在编译时就能确定,Checked异常需要自己处理. checked 异常也就是我们经常遇到的IO异常,以及SQL异常都是这种异常.对于这种异常,JAVA编译器强制要求我们必需对出现的这些异常进行catch.所以,面对这种异常不管我们是否愿意,只能自己去写一大堆catch块去处理可能的异常. runtime exception,也称运行时异常,我们可以不处理.当出现这样的异常时,总是由虚拟机接管.比如:我们从来没有人去

Exception、Error、运行时异常与一般异常有何异同

转自博客  https://blog.csdn.net/m0_37531231/article/details/79502778 一.开场白 对于程序运行过程中的可能出现异常情况,java语言使用一种称为异常处理的错误捕捉机制进行处理.相信大家对 try { }catch( ){} finally{} 这种结构非常熟悉,使用频率极高.既然经常使用它,而且也是面试常问知识点,我们就有必要去深入地了解一下.也谈不上深入,只是java语言的基本功.下面,开始吧! 二.异常分类 在java中,异常对象都