Using Eredis, Redis With Erlang

http://no-fucking-idea.com/blog/2012/03/23/using-eredis-in-erlang/

Using Eredis, Redis With Erlang

MAR 23RD, 2012 | COMMENTS

Recently i decided to move my blog from tumblr.com to octopress engine because it is just easier for me to maintain it and it looks nicer. The old blog is under http://no-fucking-idea.tumblr.com. My first post on new blog is dedicated to using redis with erlang.

Eredis

Wooga created a really nice (performance driven) redis driver for erlang. You can get it here https://github.com/wooga/eredis. It is really easy and nice.

Initial sample

On project page you can find simple examples how to use eredis. Examples there are all you need (but i need something for front page of my new blog so i will rewrite them and explain them :) ).

First you need to start your eredis application

initialization

1

{ok, Client} = eredis:start_link().

Client is the “connection / state” we will be using with rest of queries.

To query things with redis we will use q method from eredis module which takes “Connection / Client” state and list with params. This api is very simple here are two most basic examples of get and set. GET:

get

1

{ok, <<"OK">>} = eredis:q(Client, ["SET", "name", "kuba"]).

and SET:

set

1

{ok, <<"kuba">>} = eredis:q(Client, ["GET", "name"]).

From my point of view this is ideal candidate to be turned into gen_server behavior. We will pass “Connection / Client” as state and also we will build some “key” serialization methods around it to make it more durable and make our life easy if we will decide to refactor it later on.

Free Api wrapper

First thing i saw during development using Erlang is that you get free api if you follow simple patterns and encapsulate things into gen_server’s and applications.

example_db.erl

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84

-module(example_db).
-behaviour(gen_server).

-author("[email protected]").
-define(Prefix, "example").

-export([start_link/0]).
-export([init/1, handle_call/3, handle_cast/2, terminate/2, handle_info/2, code_change/3, stop/1]).
-export([get_script/2, save_script/3]).

% public api

start_link() ->
  gen_server:start_link({local, ?MODULE}, ?MODULE, [], []).

init([]) ->
  {ok, Redis} = eredis:start_link(),
  {ok, Redis}.

stop(_Pid) ->
  stop().

stop() ->
    gen_server:cast(?MODULE, stop).

%% public client api

get_script(Api, Method) ->
  gen_server:call(?MODULE, {get_script, Api, Method}).

save_script(Api, Method, Script) ->
  gen_server:call(?MODULE, {save_script, Api, Method, Script}).

%% genserver handles

handle_call({get_script, Api, Method}, _From, Redis) ->
  Response = eredis:q(Redis, [ "GET", get_key(Api, Method) ]),
  {reply, Response, Redis};

handle_call({save_script, Api, Method, Script}, _From, Redis) ->
  Response = eredis:q(Redis, ["SET", get_key(Api, Method), Script]),
  {reply, Response, Redis};

handle_call(_Message, _From, Redis) ->
  {reply, error, Redis}.

handle_cast(_Message, Redis) -> {noreply, Redis}.
handle_info(_Message, Redis) -> {noreply, Redis}.
terminate(_Reason, _Redis) -> ok.
code_change(_OldVersion, Redis, _Extra) -> {ok, Redis}.

%% helper methods

get_key(Api, Method) ->
  generate_key([Api, Method]).

generate_key(KeysList) ->
  lists:foldl(fun(Key, Acc) -> Acc ++ ":" ++ Key end, ?Prefix, KeysList).

% tests

-ifdef(TEST).
-include_lib("eunit/include/eunit.hrl").
-endif.

-ifdef(TEST).

generate_key_test() ->
  Key = generate_key(["one", "two", "three"]),
  ?assertEqual("example:one:two:three", Key).

server_test_() ->
  {setup, fun() -> example_db:start_link() end,
   fun(_Pid) -> example_db:stop(_Pid) end,
   fun generate_example_db_tests/1}.
generate_example_db_tests(_Pid) ->
[
?_assertEqual({ok,<<"OK">>}, example_db:save_script("jakub", "oboza", <<"yo dwang">>) ),
?_assertEqual({ok,<<"yo dwang">>}, example_db:get_script("jakub", "oboza") )
].
-endif

Public Api

This code listing has two important parts first at top it starts at line 26. This is the public API which will be used by developer. This is this free api. Later on i will explain how to change redis to mongodb and probably to other db engines without doing changes in rest of our app. From my perspective this is awesome feature. In most cases when i had to make app scale problem of having code that was glues to one db engine was heavy.

eunit tests

At line 60. starts the declaration of tests, using rebar and eunit is very easy and it is always good to have test coverage in case of refactoring. I’m a fan of test driven development so i like to cover in tests first things that i will use or i think they might be error prone. Here is used “test generators” to just to show how to write tests for gen_server.

Rebar

Before i will explain more i need to say little about rebar. It is a command line tool that was developed by basho to help create app. it is by far the best thing i found learning erlang to help me do boring stuff and eliminate a lot of rage writing app.src files. To get rebar simply do (you can always go to https://github.com/basho/rebar to get most up to date informations about building it)

1
2
3

λ  git clone git://github.com/basho/rebar.git
λ  cd rebar
λ  ./bootstrap

I use my own set of zsh scripts so all i did to add it to my path was to edit .furby file in my home dir. I strongly suggest also adding it to $PATH just to make your life easier.

Back to example_db!

To create app using rebar you just need to

1
2
3
4
5
6

λ mkdir example_db
λ rebar create-app appid=example_db
==> example_db (create-app)
Writing src/example_db.app.src
Writing src/example_db_app.erl
Writing src/example_db_sup.erl

This command created src folder with scaffold of application OTP pattern and supervisorthats almost all we need :). Now you can compile it using rebar compile and run tests using rebar compile eunit in out app currently we will see

rebar compile eunit

1
2
3
4
5
6
7
8

λ rebar compile eunit
==> example_db (compile)
Compiled src/example_db_app.erl
Compiled src/example_db_sup.erl
==> example_db (eunit)
Compiled src/example_db_app.erl
Compiled src/example_db_sup.erl
  There were no tests to run.

Nothing to do because its empty. Lets add our db module. But before this we need to add dependencies for eredis module. Lets create rebar.config file and add it.

1
2
3
4
5
6
7
8
9
10
11
12

λ emacs rebar.config
λ cat rebar.config
%%-*- mode: erlang -*-

{erl_opts, []}.
{cover_enabled, true}.

{deps,
  [
    {eredis, ".*", {git, "https://github.com/wooga/eredis.git", "HEAD"}}
  ]
}.

Now just run rebar get-deps to get all dependencies downloaded. After adding our example_db.erl into src directory we can run rebar compile eunit to compile and run tests. We have added {cover_enabled, true} in rebar.conf so also test code coverage will be generated for us.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33

λ rebar compile eunit
==> eredis (compile)
==> example_db (compile)
Compiled src/example_db.erl
==> eredis (eunit)

=ERROR REPORT==== 28-Mar-2012::22:19:35 ===
** Generic server <0.263.0> terminating
** Last message in was {tcp,#Port<0.4516>,
                            <<"*3\r\n$7\r\nmessage\r\n$3\r\nfoo\r\n$2\r\n12\r\n">>}
** When Server state == {state,"127.0.0.1",6379,<<>>,100,#Port<0.4516>,
                               {pstate,undefined,undefined},
                               [<<"foo">>],
                               {#Ref<0.0.0.4058>,<0.180.0>},
                               {[{message,<<"foo">>,<<"11">>,<0.263.0>},
                                 {message,<<"foo">>,<<"10">>,<0.263.0>},
                                 {message,<<"foo">>,<<"9">>,<0.263.0>},
                                 {message,<<"foo">>,<<"8">>,<0.263.0>},
                                 {message,<<"foo">>,<<"7">>,<0.263.0>},
                                 {message,<<"foo">>,<<"6">>,<0.263.0>},
                                 {message,<<"foo">>,<<"5">>,<0.263.0>},
                                 {message,<<"foo">>,<<"4">>,<0.263.0>},
                                 {message,<<"foo">>,<<"3">>,<0.263.0>}],
                                [{message,<<"foo">>,<<"2">>,<0.263.0>}]},
                               10,exit,need_ack}
** Reason for termination ==
** max_queue_size
  All 53 tests passed.
Cover analysis: /private/tmp/example_db/deps/eredis/.eunit/index.html
==> example_db (eunit)
Compiled src/example_db.erl
  All 3 tests passed.
Cover analysis: /private/tmp/example_db/.eunit/index.html

All seems to be fine! lets create file called start.sh to test it out

1
2

λ cat start.sh
erl -pa ebin -pa deps/*/ebin

and make it executable with chmod +x start.sh

And lets rock ‘n’ roll

1
2
3
4
5
6
7
8
9
10

 λ ./start.sh
Erlang R15B (erts-5.9) [source] [64-bit] [smp:4:4] [async-threads:0] [hipe] [kernel-poll:false]

Eshell V5.9  (abort with ^G)
1> example_db:start_link().
{ok,<0.33.0>}
2> example_db:save_script("example", "script", "puts ‘2+2‘").
{ok,<<"OK">>}
3> example_db:get_script("example", "script").
{ok,<<"puts ‘2+2‘">>}

Have fun :) Hope it was useful. You can download code for this blog post here https://github.com/JakubOboza/example_db-code

Huh ^___^

that was my first post on new blog ;)

时间： 2024-10-21 03:20:31

Using Eredis, Redis With Erlang的相关文章

Erlang自研引擎与Redis-Mensia比较

数据库性能测试 windows 测试环境: 系统:win7 ,内存:16G , CPU:Intel(R) Core(TM) i7-3770 CPU @ 3.40GHz 3.40GHz ,p25单节点写(每条数据大小50Byte):速率(不间断写入):25000条/秒写入上限:1200w条 无锁读(单条数据50Byte):速率(不间断读取):35000条/秒上限:无 锁读(单条数据50Byte):速率(不间断读取):30000条/秒上限:无 写(单条数据200k):速率(不间断写入): 5000条

redis

Redis是一个开源的使用ANSI C语言编写.支持网络.可基于内存亦可持久化的日志型.Key-Value数据库,并提供多种语言的API.从2010年3月15日起,Redis的开发工作由VMware主持.从2013年5月开始,Redis的开发由Pivotal赞助. redis是一个key-value存储系统.和Memcached类似,它支持存储的value类型相对更多,包括string(字符串).list(链表).set(集合).zset(sorted set --有序集合)和hash(哈希类型

rabbitMQ、activeMQ、zeroMQ、Kafka、Redis 比较

Kafka作为时下最流行的开源消息系统,被广泛地应用在数据缓冲.异步通信.汇集日志.系统解耦等方面.相比较于RocketMQ等其他常见消息系统,Kafka在保障了大部分功能特性的同时,还提供了超一流的读写性能. 针对Kafka性能方面进行简单分析,相关数据请参考:https://segmentfault.com/a/1190000003985468,下面介绍一下Kafka的架构和涉及到的名词: Topic:用于划分Message的逻辑概念,一个Topic可以分布在多个Broker上. Parti

Python之路【第九篇】：Python操作 RabbitMQ、Redis、Memcache、SQLAlchemy

Python之路[第九篇]:Python操作 RabbitMQ.Redis.Memcache.SQLAlchemy Memcached Memcached 是一个高性能的分布式内存对象缓存系统,用于动态Web应用以减轻数据库负载.它通过在内存中缓存数据和对象来减少读取数据库的次数,从而提高动态.数据库驱动网站的速度.Memcached基于一个存储键/值对的hashmap.其守护进程(daemon )是用C写的,但是客户端可以用任何语言来编写,并通过memcached协议与守护进程通信. Memc

redis应用之安装配置介绍

一.redis介绍: 1.redis定义: Redis是一个开源的使用ANSI C语言编写.支持网络.可基于内存亦可持久化的日志型.Key-Value数据库,并提供多种语言的API.从2010年3月15日起,Redis的开发工作由VMware主持.redis是一个key-value存储系统.和Memcached类似,它支持存储的value类型相对更多,包括string(字符串).list(链表).set(集合).zset(sorted set --有序集合)和hash(哈希类型).这些数据类型都

windows下安装redis

1.redis简介redis是一个key-value存储系统.和Memcached类似,它支持存 储的value类型相对更多,包括string(字符串).list(链表).set(集合).zset(sorted set --有序集合)和hashs(哈希类型).这些数据类型都支持push/pop.add/remove及取交集并集和差集及更丰富的操作,而且这些操作都 是原子性的.在此基础上,redis支持各种不同方式的排序.与memcached一样,为了保证效率,数据都是缓存在内存中.区别的是red

项目中使用Redis的一些总结和体会

第一部分:为什么我的项目中要使用Redis 我知道有些地方没说到位,希望大神们提出来,我会吸取教训,大家共同进步! 注册时邮件激活的部分使用Redis 发送邮件时使用Redis的消息队列,减轻网站压力. 使用Lucene.Net在进行分词时使用Redis消息队列和多线程来避免界面卡死等性能问题. 请大家先思考一个问题:这个问题在大并发.高负载的网站中必须考虑!大家思考如何让速度更快. 三种方法:(1)数据库(2)页面静态化(3)Redis.Memcached 第二部分:Redis是什么 概述:r

linux 基础学习编译安装php+mysql+python3+memcached+radis+rabbitmq+libevent+tcl+gcc+erlang

CentOS 6.9 编译安装 Nginx+PHP+MYSQL 0x01 环境约束 源码位置 /home/[用户文件夹] 源码编译安装位置  /usr/local/[软件名] CentOS6.9 x86 - 64 最小化安装 配置好网络 用户 使用Xshell登录 0x02 软件下载 内网地址: http://192.168.20.252/src/php.tar.gz http://192.168.20.252/src/cmake.tar.gz http://192.168.20.252/src

使用redis减小数据库访问压力及提高系统性能

什么是redis redis是一个key-value存储系统.和Memcached类似,它支持存储的value类型相对更多,包括string(字符串).list(链表).set(集合).zset(sorted set --有序集合)和hash(哈希类型).这些数据类型都支持push/pop.add/remove及取交集并集和差集及更丰富的操作,而且这些操作都是原子性的.在此基础上,redis支持各种不同方式的排序.与memcached一样,为了保证效率,数据都是缓存在内存中.区别的是redis会