Redis - HyperLogLogs

A HyperLogLog is a probabilistic data structure used in order to count unique things (technically this is referred to estimating the cardinality of a set). Usually counting unique items requires using an amount of memory proportional to the number of items you want to count, because you need to remember the elements you have already seen in the past in order to avoid counting them multiple times. However there is a set of algorithms that trade memory for precision: you end with an estimated measure with a standard error, in the case of the Redis implementation, which is less than 1%. The magic of this algorithm is that you no longer need to use an amount of memory proportional to the number of items counted, and instead can use a constant amount of memory! 12k bytes in the worst case, or a lot less if your HyperLogLog (We‘ll just call them HLL from now) has seen very few elements.

HLLs in Redis, while technically a different data structure, is encoded as a Redis string, so you can call GET to serialize a HLL, and SET to deserialize it back to the server.

Conceptually the HLL API is like using Sets to do the same task. You would SADD every observed element into a set, and would use SCARD to check the number of elements inside the set, which are unique since SADD will not re-add an existing element.

While you don‘t really add items into an HLL, because the data structure only contains a state that does not include actual elements, the API is the same:

  • Every time you see a new element, you add it to the count with PFADD.
  • Every time you want to retrieve the current approximation of the unique elements added with PFADD so far, you use the PFCOUNT.
127.0.0.1:6379> PFADD hll a b c d
(integer) 1
127.0.0.1:6379> PFCOUNT hll
(integer) 4

An example of use case for this data structure is counting unique queries performed by users in a search form every day.

时间: 2024-11-05 22:38:45

Redis - HyperLogLogs的相关文章

Redis学习笔记13Redis数据类型之(6)HyperLogLogs类型

1.1.1. pfadd 向指定的hyperloglog中添加一个或多个元素.hyperloglog中重复的元素将被合并成一个元素. 语法: PFADD key element [element ...] 参数: key: 键名,键值为一个hyperloglog对象. element:待添加的元素. 返回值: 1:hyperloglog的基数评估变化了. 0: 没有变化. 例子: redis.coe2coe.me:6379> pfadd myhpp green blue yellow (inte

redis应用之安装配置介绍

一.redis介绍: 1.redis定义: Redis是一个开源的使用ANSI C语言编写.支持网络.可基于内存亦可持久化的日志型.Key-Value数据库,并提供多种语言的API.从2010年3月15日起,Redis的开发工作由VMware主持.redis是一个key-value存储系统.和Memcached类似,它支持存储的value类型相对更多,包括string(字符串).list(链表).set(集合).zset(sorted set --有序集合)和hash(哈希类型).这些数据类型都

redis基本命令学习

看到大家都需要redis,而我工作却没怎么接触过,特来学习,现在先熟悉指令,我通过redis中文的 互动教程(interactive tutorial)来学习的. 1.redis是key-value存储的,放在内存中,并在磁盘持久化的数据结构存储系统,它可以用作数据库.缓存和消息中间件. 通过set key value来存储,通过get key来获取值 set key value:设置key的值,若存在则覆盖setnx key value:SET if Not eXists,若存在则不操作. M

玩转Redis之Window安装使用(干货)

距离上次定Gc.Db框架,好久没有更新博客了,今日没什么事,就打算就Redis写点东西. Redis是一个开源(BSD许可),内存存储的数据结构服务器,可用作数据库,高速缓存和消息队列代理.它支持字符串.哈希表.列表.集合.有序集合,位图,hyperloglogs等数据类型. 关于Redis,大家都不会陌生,网上关于Redis在Window和Linux系统安装教程也不少,但是我发现许多安装教程,有些过于简单,也不是很全面,故今天会从就Window下Redis临时服务.Redis默认服务安装.Re

asp.net core 使用 Redis 和 Protobuf

asp.net core 使用 Redis 和 Protobuf 前言 上篇博文介绍了怎么样在 asp.net core 中使用中间件,以及如何自定义中间件.项目中刚好也用到了Redis,所以本篇就介绍下怎么样在 asp.net core 中使用 Redis 进行资源缓存和Session缓存. 如果你觉得对你有帮助的话,不妨点个[推荐]. 目录 Redis 介绍 asp.net core Session 介绍 Redis & Session 实例讲解 Session的使用 使用 Protobuf

安装 redis

1: redis 是什么 Redis is an open source (BSD licensed), in-memory data structure store, used as database, cache and message broker. It supports data structures such as strings, hashes, lists, sets, sorted sets with range queries, bitmaps, hyperloglogs a

redis.conf配置

# Redis configuration file example # Redis示例配置文件 # Note on units: when memory size is needed, it is possible to specify # it in the usual form of 1k 5GB 4M and so forth: # 注意单位问题:当需要设置内存大小的时候,可以使用类似1k.5GB.4M这样的常见格式: # # 1k => 1000 bytes # 1kb => 102

Redis初探

转自菜鸟网络:http://www.runoob.com/redis/redis-data-types.html Redis 是一个开源(BSD许可)的,内存中的数据结构存储系统,它可以用作数据库.缓存和消息中间件. 它支持多种类型的数据结构,如 字符串(strings), 散列(hashes), 列表(lists), 集合(sets), 有序集合(sorted sets) 与范围查询, bitmaps, hyperloglogs 和 地理空间(geospatial) 索引半径查询. Redis

redis 数据持久化

转:redis 数据持久化 1.快照(snapshots) 缺省情况情况下,Redis把数据快照存放在磁盘上的二进制文件中,文件名为dump.rdb.你可以配置Redis的持久化策略,例如数据集中每N秒钟有超过M次更新,就将数据写入磁盘:或者你可以手工调用命令SAVE或BGSAVE. 数据保存的目录: 工作原理 Redis forks. 子进程开始将数据写到临时RDB文件中. 当子进程完成写RDB文件,用新文件替换老文件. 这种方式可以使Redis使用copy-on-write技术. 2.APP