利用内存chunk充当数据buffer的vector的实现,和STL vector 有接口操作性能比较

问题描述:

1.vector是最常用到的容器,它其实是一种自动扩充的动态数组,底层实现通过两倍扩展,

所以再不能预知要存入元素多少时,难免要两倍扩展,这带来了拷贝所存对象的开销;

2.本文尝试利用memory chunk作为底层存储容器来避免动态扩展时copy 开销;

3.本实现屏蔽了STL  vector 一些接口比如erase,主要是为了减轻实现复杂性考虑;

4.它支持两边插入和删除操作,其接口功能完全等同于STL 的deque;

5.和STL vector 做了push_back, pop_back, at 操作性能比较,结果如下:

1)push_back 操作,性能改善了4-5倍;

2)at操作,基本上相当;

3)pop_back性能变慢了3-4倍;

4)  尽管pop_back性能变慢了,但是我的实现内存大小是可伸缩的;

程序代码:

#ifndef _ALVECTOR_H_
#define _ALVECTOR_H_

#include <vector>

#include "windows.h"

/*
* the class encapsulate dynamic array the same as STL vector from function
* it have different underlying storage with STL vector
* it apply to memory chunk tech linked the chunk by next point
* it have good insert performance because no copy burden when full storage
*
*/
template<class T>
class AlVector
{
public:
	/*
	* memory chunk(storage chunk) for interval using
	*
	*/
	enum
	{
		KSlotCount = 64
	};

	int m_totalCount;
	int m_count;

	struct Buffer
	{
		T        buf[KSlotCount];
		Buffer*  next;

		Buffer():next(0)
		{
			memset( buf, 0x00, sizeof(buf) );
		}
	};

	/*
	* Constructor
	*
	*/
	AlVector():m_totalCount(0), m_count(0),
		       m_chunkHead(0), m_chunkSize(0),
			   m_chunkCapacity(0)
	{
		InitChunkIndex();
		m_dataBuffer = &m_initBuffer;
		AssignChunkIndex( m_dataBuffer );
	}

	/*
	* Copy Constructor
	*
	*/
	AlVector( const AlVector& rhs )
	{
		size_t len = rhs.Size();
		for( size_t i = 0; i < len; i++ )
			this->PushBack( rhs[i] );
	}

	/*
	* Assignment operator overload
	*
	*/
	AlVector& operator = ( const AlVector& rhs )
	{
		if( this != &rhs )
		{
			Clear();
			size_t len = rhs.Size();
			for( size_t i = 0; i < len; i++ )
				this->PushBack( rhs[i] );

		}

		return *this;
	}

	/*
	* Destructor
	*
	*/
	~AlVector()
	{
		Clear();
	}

	/*
	*
	*
	*/
	T& operator [] ( size_t index )
	{
		return At( index );
	}

	/*
	*
	*
	*/
	const T& operator[] ( size_t index ) const
	{
		return At( index );
	}

	/*
	* Check whether or not exist element
	*
	*/
	bool IsEmpty()
	{
		return m_totalCount == 0;
	}

	/*
	* Retrieve the number of elements in container
	*
	*/
	size_t Size() const
	{
		return m_totalCount;
	}

	/*
	* Clear all object pushed and memory
	*
	*/
	void Clear()
	{
		while( m_dataBuffer != &m_initBuffer )
		{
			Buffer* buf = m_dataBuffer;
			m_dataBuffer = m_dataBuffer->next;
			delete buf;
		}

		delete [] m_chunkHead;
		m_chunkHead = 0;
		m_chunkSize = m_chunkCapacity = 0;
	}

	/*
	* Push element to container tail
	*
	*/
	void PushBack( const T& data )
	{
		*Push() = data;
	}

	/*
	* Pop element from container tail
	*
	*/
	void PopBack()
	{
		m_totalCount--;
		if( --m_count == 0 )
		{
			if( m_dataBuffer != &m_initBuffer )
			{
				Buffer* buf = m_dataBuffer;
				m_dataBuffer = m_dataBuffer->next;

				//important
				m_chunkHead[m_chunkSize] = 0;
				m_chunkSize--;

				delete buf;
				buf = 0;

				// reset the value of m_count
				m_count = KSlotCount;
			}
		}
	}

	/*
	* Retrieve the element of header in container
	*
	*/
	T& Front()
	{
		return m_chunkHead[0]->buf[0];
	}

	/*
	* Retrieve the element of header in container
	*
	*/
	const T& Front() const
	{
		return Front();
	}

	/*
	* Retrieve the element of tail in container
	*
	*/
	T& Back()
	{
		return Back()
	}

	/*
	* Retrieve the element of tail in container
	*
	*/
	const T& Back() const
	{
		assert( m_count > 0 );
		return m_dataBuffer[ m_count - 1 ];
	}

	/*
	* Retrieve the element by terms of position index
	*
	*/
	T& At( size_t index )
	{
		assert( index < m_totalCount );
		if( index < KSlotCount )
			return m_initBuffer.buf[index];

		return m_chunkHead[index / KSlotCount]->buf[index % KSlotCount];
	}

	/*
	* Retrieve the element by terms of position index
	*
	*/
	const T& At( size_t index ) const
	{
		assert( index < m_totalCount );
		if( index < KSlotCount )
			return m_initBuffer.buf[index];

		return m_chunkHead[index / KSlotCount]->buf[index % KSlotCount];
	}

protected:

	/*
	*
	*
	*/
	T* Push()
	{
		if( KSlotCount == m_count )
		{
			Buffer* buf = new Buffer;
			assert( buf );
			buf->next = m_dataBuffer;
			m_dataBuffer = buf;

			//important
			AssignChunkIndex( buf );

			//reset the value of value in single buffer
			m_count = 0;
		}

		m_totalCount++;
		return &m_dataBuffer->buf[m_count++];
	}

	/*
	* Init the array of index of buffer pointer
	*
	*/
	void InitChunkIndex()
	{
		m_chunkCapacity = KSlotCount;
		m_chunkHead = new Buffer*[m_chunkCapacity];
		memset( m_chunkHead, 0x00, sizeof(Buffer*)* m_chunkCapacity );
		m_chunkSize;
	}

	/*
	* Store currently buffer pointer to array of index
	*
	*/
	void AssignChunkIndex( Buffer* curBuffer )
	{
		if( m_chunkSize >= m_chunkCapacity )
		{
			m_chunkCapacity += 2;// important
			m_chunkCapacity <<= 1;
			Buffer** newHead = new Buffer*[ m_chunkCapacity ];
			memset( newHead, 0x00, sizeof(Buffer*)* m_chunkCapacity );

			if( m_chunkHead )
			{
				memcpy( newHead, m_chunkHead, sizeof(Buffer*) * m_chunkSize );
				delete [] m_chunkHead;
			}

			m_chunkHead = newHead;
		}

		m_chunkHead[m_chunkSize++] = curBuffer;
	}

protected:

	Buffer     m_initBuffer; // static storage for a small amount object operation
	Buffer*    m_dataBuffer; // currently the pointer of buffer of storage

	Buffer**    m_chunkHead;  // the array of index of buffer of storage allocated
	size_t      m_chunkSize;  // the length of array of index of buffer
	size_t      m_chunkCapacity; // the capacity of ...

};

/*
* Test STL vector
*
*/
void TestSTLVector()
{
	std::vector<int> stlVec;
	int len = 64 * 100000;

	printf("STL Vector test result : \n " );
	unsigned long start= GetTickCount();
	for( int i = 0; i < len; i++ )
	{
		stlVec.push_back( i );
	}
	unsigned long interval = GetTickCount() - start;
	printf( "insert operation consume time is %d  in STL vector\n", interval );

	start= GetTickCount();
	for( int i = 0; i < len; i++ )
	{
		assert( stlVec[i] == i );
	}
	interval = GetTickCount() - start;
	printf( "retrieve operation consume time is %d  in STL vector\n", interval );

    start= GetTickCount();
	for( int i = 0; i < len; i++ )
	{
		stlVec.pop_back();
	}
	interval = GetTickCount() - start;
	printf( "pop operation consume time is %d  in STL vector\n", interval );

	assert( stlVec.size() == 0 );

}

/*
* Test AlVector
*
*/
void TestVector()
{
	int len = 64 * 100000;
	AlVector<int> vecObj;

	printf("AlVector Vector test result : \n " );
	unsigned long start= GetTickCount();
	for( int i = 0; i < len; i++ )
	{
		vecObj.PushBack( i );
	}
	unsigned long interval = GetTickCount() - start;
	printf( "insert operation consume time is %d  in AlVector \n", interval );

	start= GetTickCount();
	for( int i = 0; i < len; i++ )
	{
		assert( vecObj[i] == i );
	}
	interval = GetTickCount() - start;
	printf( "retrieve operation consume time is %d  in AlVector\n", interval );

	start= GetTickCount();
	for( int i = 0; i < len; i++ )
	{
		vecObj.PopBack();
	}
	interval = GetTickCount() - start;
	printf( "pop operation consume time is %d  in AlVector\n", interval );

	assert( vecObj.Size() == 0 );

	vecObj.Clear();
}

/*
* Test interface
*
*/
void TestVectorSuite()
{
	TestVector();
	TestSTLVector();

}

#endif 

compile and run in visual studio 2005

test result as follows:

时间: 2024-10-13 09:38:00

利用内存chunk充当数据buffer的vector的实现,和STL vector 有接口操作性能比较的相关文章

利用内存映射文件在两个进程间共享数据 转

private hMapFile: THandle; MapFilePointer: Pointer; public { Public declarations } end; var Form1: TForm1; implementation {$R *.DFM} procedure TForm1.FormCreate(Sender: TObject); begin hMapFile := CreateFileMapping ( $FFFFFFFF, // 特殊内存映射句柄 nil, page_

Java学习笔记之[ 利用扫描仪Scanner进行数据输入 ]

/*********数据的输入********//**利用扫描仪Scanner进行数据输入 怎么使用扫描仪Scanner *1.放在类声明之前,引入扫描仪 import java.util.Scanner; *2.声明一个新的扫描仪(即向内存申请一个空间) Scanner in *3.赋值 in=new Scanner(System.in); Scanner in=new Scanner(System.in); *4.使用扫描仪 整形数据输入:in.nextInt()来接收 双精度小数输入:in

实现按行读取文件,把内容按照第三种内存模型打包数据传出,把行数通过函数参数传出。

/* 2 编写一个业务函数,实现按行读取文件.把内容按照第三种内存模型打包数据传出,把行数通过函数参数传出. 函数原型有两个,任意选择其一 要求1:请自己任意选择一个接口(函数),并实现功能:70分 要求2:编写测试用例.30分 要求3:自己编写内存释放函数 */ /********************************************************************** * 版权所有 (C)2015, Wu Yingqiang. * * 文件名称:ReadFi

简单了解:在内存中的数据

内存原理 开启电源,启动BIOS,CPU工作,调用内存,内存跟硬盘索要资源 当你点击一个文件的时候数据经过数据总线传达到CPU,CPU发送指令到内存,内存那里会跟硬盘沟通,问他有没有这个东西,他说有,你就会看到这个文件夹里面是什么东西. (Xee:RAM 是随机存取存储器,它的特点是易挥发性,即掉电失.--妈蛋,难怪一断电,我的东西没保存,就找不到了-- 既然内存是用来存放当前正在使用的(即执行中)的数据和程序,那么它是怎么工作的呢?我们平常所提到的计算机的内存指的是动态内存(即DRAM),动态

Lucene索引过程中的内存管理与数据存储

Lucene的索引过程分两个阶段,第一阶段把文档索引到内存中:第二阶段,即内存满了,就把内存中的数据刷新到硬盘上.          倒排索引信息在内存存储方式 Lucene有各种Field,比如StringField,TextField,IntField,FloatField,DoubleField-,Lucene在处理的过程中把各种Field都处理成相应的byte[],以最本质的方式来看待各种Field的内容,统一了数据的存储形式. 在写入内存阶段,第一步就是需要理清各个类之间的关系. 在索

ffmpeg 从内存中读取数据

http://blog.csdn.net/leixiaohua1020/article/details/12980423 ffmpeg一般情况下支持打开一个本地文件,例如"C:\test.avi" 或者是一个流媒体协议的URL,例如"rtmp://222.31.64.208/vod/test.flv" 其打开文件的函数是avformat_open_input(),直接将文件路径或者流媒体URL的字符串传递给该函数就可以了. 但其是否支持从内存中读取数据呢?这个问题困

linux 内存中cache和buffer解析

cache是从磁盘读数据到内存中的缓存,减少读取磁盘的次数, 大家知道的硬盘读取速度过慢. buffer是准备从内存写到硬盘的缓存,缓存中的数据会进行合并 同时,避免频繁操作硬盘. linux 内存中cache和buffer解析

Mysql InnDB 内存存储结构 -- Change Buffer

在InnoDB中,当对应的数据不存在与Buffer Pool中时,为了避免大量的随机磁盘I/O可能带来的性能瓶颈,InnoDB 在Buffer Pool 中划分出一部分内存,称为Change Buffer,由其负责缓存由DML操作引起的二级索引相关数据的变化.当对应的数据下次被读入Buffer Pool 中时,Change Buffer 中记录的变化信息会被合并到数据中.其结构如下图所示.简要的,使用Change Buffer有三个要点:a. DML操作, b. 涉及二级索引,c. 对应page

Struts2.5 利用Ajax将json数据传值到JSP

AJAX +JSON=>JSP AJAX AJAX 是一种在无需重新加载整个网页的情况下,能够更新部分网页的技术. 通过在后台与服务器进行少量数据交换,AJAX 可以使网页实现异步更新.这意味着可以在不重新加载整个网页的情况下,对网页的某部分进行更新. 传统的网页(不使用 AJAX)如果需要更新内容,必须重载整个网页页面. JSON JSON(JavaScript Object Notation, JS 对象标记) 是一种轻量级的数据交换格式.它基于 ECMAScript (w3c制定的js规范