python 之Tornado

一、Tomado

Tornado 是 FriendFeed 使用的可扩展的非阻塞式 web 服务器及其相关工具的开源版本。这个 Web 框架看起来有些像web.py 或者 Google 的 webapp，不过为了能有效利用非阻塞式服务器环境，这个 Web 框架还包含了一些相关的有用工具 和优化。

Tornado 和现在的主流 Web 服务器框架（包括大多数 Python 的框架）有着明显的区别：它是非阻塞式服务器，而且速度相当快。得利于其 非阻塞的方式和对 epoll 的运用，Tornado 每秒可以处理数以千计的连接，这意味着对于实时 Web 服务来说，Tornado 是一个理想的 Web 框架。我们开发这个 Web 服务器的主要目的就是为了处理 FriendFeed 的实时功能 ——在 FriendFeed 的应用里每一个活动用户都会保持着一个服务器连接。（关于如何扩容 服务器，以处理数以千计的客户端的连接的问题，请参阅 C10K problem。）

安装：pip3 install Tomado

远吗安装：https://pypi.python.org/packages/source/t/tornado/tornado-4.3.tar.gz

二、快速上手

#!/usr/bin/env python
# -*- coding:utf-8 -*-

import tornado.ioloop
import tornado.web

class MainHandler(tornado.web.RequestHandler):
    def get(self):
        self.write("Hello, world")

application = tornado.web.Application([
    (r"/index", MainHandler),
])

if __name__ == "__main__":
    application.listen(8888)
    tornado.ioloop.IOLoop.instance().start()

第一步：执行脚本，监听8888端口

第二步：浏览器客户端访问/index---->http://127.0.0.1:8888/index

第三步：服务器接受请求，并交由对应的类处理请求

第四步：类接收到请求之后，根据请求方式（post/get/delete.....）的不同调用并执行相应的方法

第五步：方法返回值的字符串内容发到浏览器

三、路由系统

路由系统其实就是 url 和 类 的对应关系，这里不同于其他框架，其他很多框架均是 url 对应 函数，Tornado中每个url对应的是一个类。

#!/usr/bin/env python
# -*- coding:utf-8 -*-

import tornado.ioloop
import tornado.web

class MainHandler(tornado.web.RequestHandler):
    def get(self):
        self.write("Hello, world")

class StoryHandler(tornado.web.RequestHandler):
    def get(self, story_id):
        self.write("You requested the story " + story_id)

class BuyHandler(tornado.web.RequestHandler):
    def get(self):
        self.write("buy.wupeiqi.com/index")

application = tornado.web.Application([
    (r"/index", MainHandler),
    (r"/story/([0-9]+)", StoryHandler),
])

application.add_handlers(‘buy.wupeiqi.com$‘, [
    (r‘/index‘,BuyHandler),
])

if __name__ == "__main__":
    application.listen(80)
    tornado.ioloop.IOLoop.instance().start()

四、模板

Tornao中的模板语言和django中类似，模板引擎将模板文件载入内存，然后将数据嵌入其中，最终获取到一个完整的字符串，再将字符串返回给请求者。

Tornado 的模板支持“控制语句”和“表达语句”，控制语句是使用 {% 和 %} 包起来的 例如 {% if len(items) > 2 %}。表达语句是使用 {{ 和 }} 包起来的，例如 {{ items[0] }}。

控制语句和对应的 Python 语句的格式基本完全相同。我们支持 if、for、while 和 try，这些语句逻辑结束的位置需要用 {% end %} 做标记。还通过 extends 和 block 语句实现了模板继承。这些在 template 模块 的代码文档中有着详细的描述。

<!DOCTYPE html>
<html>
<head>
    <meta http-equiv="Content-Type" content="text/html; charset=UTF-8"/>
    <title>老男孩</title>
    <link href="{{static_url("css/common.css")}}" rel="stylesheet" />
    {% block CSS %}{% end %}
</head>
<body>

    <div class="pg-header">

    </div>

    {% block RenderBody %}{% end %}

    <script src="{{static_url("js/jquery-1.8.2.min.js")}}"></script>

    {% block JavaScript %}{% end %}
</body>
</html>

layout.html

{% extends ‘layout.html‘%}
{% block CSS %}
    <link href="{{static_url("css/index.css")}}" rel="stylesheet" />
{% end %}

{% block RenderBody %}
    <h1>Index</h1>

    <ul>
    {%  for item in li %}
        <li>{{item}}</li>
    {% end %}
    </ul>

{% end %}

{% block JavaScript %}

{% end %}

index.html

#!/usr/bin/env python
# -*- coding:utf-8 -*-

import tornado.ioloop
import tornado.web

class MainHandler(tornado.web.RequestHandler):
    def get(self):
        self.render(‘home/index.html‘)

settings = {
    ‘template_path‘: ‘template‘,
}

application = tornado.web.Application([
    (r"/index", MainHandler),
], **settings)

if __name__ == "__main__":
    application.listen(80)
    tornado.ioloop.IOLoop.instance().start()

Tornado默认提供的这些功能其实本质上就是 UIMethod 和 UIModule，我们也可以自定义从而实现类似于Django的simple_tag的功能：

1、定义

# uimethods.py

def tab(self):
    return ‘UIMethod‘

uimethods

#!/usr/bin/env python
# -*- coding:utf-8 -*-
from tornado.web import UIModule
from tornado import escape

class custom(UIModule):

    def render(self, *args, **kwargs):
        return escape.xhtml_escape(‘<h1>wupeiqi</h1>‘)
        #return escape.xhtml_escape(‘<h1>wupeiqi</h1>‘)

uimodules.py

uimodules

2、注册

#!/usr/bin/env python
# -*- coding:utf-8 -*-
#!/usr/bin/env python
# -*- coding:utf-8 -*-

import tornado.ioloop
import tornado.web
from tornado.escape import linkify
import uimodules as md
import uimethods as mt

class MainHandler(tornado.web.RequestHandler):
    def get(self):
        self.render(‘index.html‘)

settings = {
    ‘template_path‘: ‘template‘,
    ‘static_path‘: ‘static‘,
    ‘static_url_prefix‘: ‘/static/‘,
    ‘ui_methods‘: mt,
    ‘ui_modules‘: md,
}

application = tornado.web.Application([
    (r"/index", MainHandler),
], **settings)

if __name__ == "__main__":
    application.listen(8009)
    tornado.ioloop.IOLoop.instance().start()

main.py

3、使用

<!DOCTYPE html>
<html>
<head lang="en">
    <meta charset="UTF-8">
    <title></title>
    <link href="{{static_url("commons.css")}}" rel="stylesheet" />
</head>
<body>
    <h1>hello</h1>
    {% module custom(123) %}
    {{ tab() }}
</body>

index.html

五、实现功能

1、静态文件

对于静态文件，可以配置静态文件的目录和前段使用时的前缀，并且Tornaodo还支持静态文件缓存。

#!/usr/bin/env python
# -*- coding:utf-8 -*-

import tornado.ioloop
import tornado.web

class MainHandler(tornado.web.RequestHandler):
    def get(self):
        self.render(‘home/index.html‘)

settings = {
    ‘template_path‘: ‘template‘,
    ‘static_path‘: ‘static‘,
    ‘static_url_prefix‘: ‘/static/‘,
}

application = tornado.web.Application([
    (r"/index", MainHandler),
], **settings)

if __name__ == "__main__":
    application.listen(80)
    tornado.ioloop.IOLoop.instance().start()

main.py

<!DOCTYPE html>
<html>
<head lang="en">
    <meta charset="UTF-8">
    <title></title>
    <link href="{{static_url("commons.css")}}" rel="stylesheet" />
</head>
<body>
    <h1>hello</h1>
</body>
</html>

index.html

备注：静态文件缓存的实现

def get_content_version(cls, abspath):
        """Returns a version string for the resource at the given path.

        This class method may be overridden by subclasses.  The
        default implementation is a hash of the file‘s contents.

        .. versionadded:: 3.1
        """
        data = cls.get_content(abspath)
        hasher = hashlib.md5()
        if isinstance(data, bytes):
            hasher.update(data)
        else:
            for chunk in data:
                hasher.update(chunk)
        return hasher.hexdigest()

静态文件缓存源码

2、csrf

Tornado中的夸张请求伪造和Django中的相似，跨站伪造请求(Cross-site request forgery)

settings = {
    "xsrf_cookies": True,
}
application = tornado.web.Application([
    (r"/", MainHandler),
    (r"/login", LoginHandler),
], **settings)

配置

<form action="/new_message" method="post">
  {{ xsrf_form_html() }}
  <input type="text" name="message"/>
  <input type="submit" value="Post"/>
</form>

普通表单的使用

function getCookie(name) {
    var r = document.cookie.match("\\b" + name + "=([^;]*)\\b");
    return r ? r[1] : undefined;
}

jQuery.postJSON = function(url, args, callback) {
    args._xsrf = getCookie("_xsrf");
    $.ajax({url: url, data: $.param(args), dataType: "text", type: "POST",
        success: function(response) {
        callback(eval("(" + response + ")"));
    }});
};

Ajax使用

注：Ajax使用时，本质上就是去获取本地的cookie，携带cookie再来发送请求

3、cookie

Tornado中可以对cookie进行操作，并且还可以对cookie进行签名以放置伪造。

a、基本操作

class MainHandler(tornado.web.RequestHandler):
    def get(self):
        if not self.get_cookie("mycookie"):
            self.set_cookie("mycookie", "myvalue")
            self.write("Your cookie was not set yet!")
        else:
            self.write("Your cookie was set!")

Code

b、签名

Cookie 很容易被恶意的客户端伪造。加入你想在 cookie 中保存当前登陆用户的 id 之类的信息，你需要对 cookie 作签名以防止伪造。Tornado 通过 set_secure_cookie 和 get_secure_cookie 方法直接支持了这种功能。 要使用这些方法，你需要在创建应用时提供一个密钥，名字为 cookie_secret。 你可以把它作为一个关键词参数传入应用的设置中：

class MainHandler(tornado.web.RequestHandler):
    def get(self):
        if not self.get_secure_cookie("mycookie"):
            self.set_secure_cookie("mycookie", "myvalue")
            self.write("Your cookie was not set yet!")
        else:
            self.write("Your cookie was set!")

application = tornado.web.Application([
    (r"/", MainHandler),
], cookie_secret="61oETzKXQAGaYdkL5gEmGeJJFuYh7EQnp2XdTP1o/Vo=")

Code

def _create_signature_v1(secret, *parts):
    hash = hmac.new(utf8(secret), digestmod=hashlib.sha1)
    for part in parts:
        hash.update(utf8(part))
    return utf8(hash.hexdigest())

def _create_signature_v2(secret, s):
    hash = hmac.new(utf8(secret), digestmod=hashlib.sha256)
    hash.update(utf8(s))
    return utf8(hash.hexdigest())

内部算法

def create_signed_value(secret, name, value, version=None, clock=None,
                        key_version=None):
    if version is None:
        version = DEFAULT_SIGNED_VALUE_VERSION
    if clock is None:
        clock = time.time

    timestamp = utf8(str(int(clock())))
    value = base64.b64encode(utf8(value))
    if version == 1:
        signature = _create_signature_v1(secret, name, value, timestamp)
        value = b"|".join([value, timestamp, signature])
        return value
    elif version == 2:
        # The v2 format consists of a version number and a series of
        # length-prefixed fields "%d:%s", the last of which is a
        # signature, all separated by pipes.  All numbers are in
        # decimal format with no leading zeros.  The signature is an
        # HMAC-SHA256 of the whole string up to that point, including
        # the final pipe.
        #
        # The fields are:
        # - format version (i.e. 2; no length prefix)
        # - key version (integer, default is 0)
        # - timestamp (integer seconds since epoch)
        # - name (not encoded; assumed to be ~alphanumeric)
        # - value (base64-encoded)
        # - signature (hex-encoded; no length prefix)
        def format_field(s):
            return utf8("%d:" % len(s)) + utf8(s)
        to_sign = b"|".join([
            b"2",
            format_field(str(key_version or 0)),
            format_field(timestamp),
            format_field(name),
            format_field(value),
            b‘‘])

        if isinstance(secret, dict):
            assert key_version is not None, ‘Key version must be set when sign key dict is used‘
            assert version >= 2, ‘Version must be at least 2 for key version support‘
            secret = secret[key_version]

        signature = _create_signature_v2(secret, to_sign)
        return to_sign + signature
    else:
        raise ValueError("Unsupported version %d" % version)

内部算法-加密

def _decode_signed_value_v1(secret, name, value, max_age_days, clock):
    parts = utf8(value).split(b"|")
    if len(parts) != 3:
        return None
    signature = _create_signature_v1(secret, name, parts[0], parts[1])
    if not _time_independent_equals(parts[2], signature):
        gen_log.warning("Invalid cookie signature %r", value)
        return None
    timestamp = int(parts[1])
    if timestamp < clock() - max_age_days * 86400:
        gen_log.warning("Expired cookie %r", value)
        return None
    if timestamp > clock() + 31 * 86400:
        # _cookie_signature does not hash a delimiter between the
        # parts of the cookie, so an attacker could transfer trailing
        # digits from the payload to the timestamp without altering the
        # signature.  For backwards compatibility, sanity-check timestamp
        # here instead of modifying _cookie_signature.
        gen_log.warning("Cookie timestamp in future; possible tampering %r",
                        value)
        return None
    if parts[1].startswith(b"0"):
        gen_log.warning("Tampered cookie %r", value)
        return None
    try:
        return base64.b64decode(parts[0])
    except Exception:
        return None

def _decode_fields_v2(value):
    def _consume_field(s):
        length, _, rest = s.partition(b‘:‘)
        n = int(length)
        field_value = rest[:n]
        # In python 3, indexing bytes returns small integers; we must
        # use a slice to get a byte string as in python 2.
        if rest[n:n + 1] != b‘|‘:
            raise ValueError("malformed v2 signed value field")
        rest = rest[n + 1:]
        return field_value, rest

    rest = value[2:]  # remove version number
    key_version, rest = _consume_field(rest)
    timestamp, rest = _consume_field(rest)
    name_field, rest = _consume_field(rest)
    value_field, passed_sig = _consume_field(rest)
    return int(key_version), timestamp, name_field, value_field, passed_sig

def _decode_signed_value_v2(secret, name, value, max_age_days, clock):
    try:
        key_version, timestamp, name_field, value_field, passed_sig = _decode_fields_v2(value)
    except ValueError:
        return None
    signed_string = value[:-len(passed_sig)]

    if isinstance(secret, dict):
        try:
            secret = secret[key_version]
        except KeyError:
            return None

    expected_sig = _create_signature_v2(secret, signed_string)
    if not _time_independent_equals(passed_sig, expected_sig):
        return None
    if name_field != utf8(name):
        return None
    timestamp = int(timestamp)
    if timestamp < clock() - max_age_days * 86400:
        # The signature has expired.
        return None
    try:
        return base64.b64decode(value_field)
    except Exception:
        return None

def get_signature_key_version(value):
    value = utf8(value)
    version = _get_version(value)
    if version < 2:
        return None
    try:
        key_version, _, _, _, _ = _decode_fields_v2(value)
    except ValueError:
        return None

    return key_version

内部算法-解密

签名Cookie的本质是：

写cookie过程：

  • 将值进行base64加密
  • 对除值以外的内容进行签名，哈希算法（无法逆向解析）
  • 拼接 签名 + 加密值

读cookie过程：

  • 读取 签名 + 加密值
  • 对签名进行验证
  • base64解密，获取值内容

注：许多API验证机制和安全cookie的实现机制相同。

#!/usr/bin/env python
# -*- coding:utf-8 -*-

import tornado.ioloop
import tornado.web

class MainHandler(tornado.web.RequestHandler):

    def get(self):
        login_user = self.get_secure_cookie("login_user", None)
        if login_user:
            self.write(login_user)
        else:
            self.redirect(‘/login‘)

class LoginHandler(tornado.web.RequestHandler):
    def get(self):
        self.current_user()

        self.render(‘login.html‘, **{‘status‘: ‘‘})

    def post(self, *args, **kwargs):

        username = self.get_argument(‘name‘)
        password = self.get_argument(‘pwd‘)
        if username == ‘wupeiqi‘ and password == ‘123‘:
            self.set_secure_cookie(‘login_user‘, ‘武沛齐‘)
            self.redirect(‘/‘)
        else:
            self.render(‘login.html‘, **{‘status‘: ‘用户名或密码错误‘})

settings = {
    ‘template_path‘: ‘template‘,
    ‘static_path‘: ‘static‘,
    ‘static_url_prefix‘: ‘/static/‘,
    ‘cookie_secret‘: ‘aiuasdhflashjdfoiuashdfiuh‘
}

application = tornado.web.Application([
    (r"/index", MainHandler),
    (r"/login", LoginHandler),
], **settings)

if __name__ == "__main__":
    application.listen(8888)
    tornado.ioloop.IOLoop.instance().start()

Demo-基于cookie进行用户验证

#!/usr/bin/env python
# -*- coding:utf-8 -*-

import tornado.ioloop
import tornado.web

class BaseHandler(tornado.web.RequestHandler):

    def get_current_user(self):
        return self.get_secure_cookie("login_user")

class MainHandler(BaseHandler):

    @tornado.web.authenticated
    def get(self):
        login_user = self.current_user
        self.write(login_user)

class LoginHandler(tornado.web.RequestHandler):
    def get(self):
        self.current_user()

        self.render(‘login.html‘, **{‘status‘: ‘‘})

    def post(self, *args, **kwargs):

        username = self.get_argument(‘name‘)
        password = self.get_argument(‘pwd‘)
        if username == ‘wupeiqi‘ and password == ‘123‘:
            self.set_secure_cookie(‘login_user‘, ‘武沛齐‘)
            self.redirect(‘/‘)
        else:
            self.render(‘login.html‘, **{‘status‘: ‘用户名或密码错误‘})

settings = {
    ‘template_path‘: ‘template‘,
    ‘static_path‘: ‘static‘,
    ‘static_url_prefix‘: ‘/static/‘,
    ‘cookie_secret‘: ‘aiuasdhflashjdfoiuashdfiuh‘,
    ‘login_url‘: ‘/login‘
}

application = tornado.web.Application([
    (r"/index", MainHandler),
    (r"/login", LoginHandler),
], **settings)

if __name__ == "__main__":
    application.listen(8888)
    tornado.ioloop.IOLoop.instance().start()

Demo-Toando内部提供基于cookie进行用户验证

4、Ajax上传文件

<!DOCTYPE html>
<html>
<head lang="en">
    <meta charset="UTF-8">
    <title></title>
</head>
<body>
    <input type="file" id="img" />
    <input type="button" onclick="UploadFile();" />
    <script>
        function UploadFile(){
            var fileObj = document.getElementById("img").files[0];

            var form = new FormData();
            form.append("k1", "v1");
            form.append("fff", fileObj);

            var xhr = new XMLHttpRequest();
            xhr.open("post", ‘/index‘, true);
            xhr.send(form);
        }
    </script>
</body>
</html>

Html

#!/usr/bin/env python
# -*- coding:utf-8 -*-
#!/usr/bin/env python
# -*- coding:utf-8 -*-

import tornado.ioloop
import tornado.web

class MainHandler(tornado.web.RequestHandler):
    def get(self):

        self.render(‘index.html‘)

    def post(self, *args, **kwargs):
        file_metas = self.request.files["fff"]
        # print(file_metas)
        for meta in file_metas:
            file_name = meta[‘filename‘]
            with open(file_name,‘wb‘) as up:
                up.write(meta[‘body‘])

settings = {
    ‘template_path‘: ‘template‘,
}

application = tornado.web.Application([
    (r"/index", MainHandler),
], **settings)

if __name__ == "__main__":
    application.listen(8000)
    tornado.ioloop.IOLoop.instance().start()

Python

var fileObj = $("#img")[0].files[0];
var form = new FormData();
form.append("k1", "v1");
form.append("fff", fileObj);

$.ajax({
    type:‘POST‘,
    url: ‘/index‘,
    data: form,
    processData: false,  // tell jQuery not to process the data
    contentType: false,  // tell jQuery not to set contentType
    success: function(arg){
        console.log(arg);
    }
})

jQuery Ajax Upload

六、扩展功能

1、自定义Session

a.知识储备

#!/usr/bin/env python
# -*- coding:utf-8 -*-

class Foo(object):

    def __getitem__(self, key):
        print  ‘__getitem__‘,key

    def __setitem__(self, key, value):
        print ‘__setitem__‘,key,value

    def __delitem__(self, key):
        print ‘__delitem__‘,key

obj = Foo()
result = obj[‘k1‘]
#obj[‘k2‘] = ‘wupeiqi‘
#del obj[‘k1‘]

b.session实现机制

#!/usr/bin/env python
# -*- coding:utf-8 -*-

import tornado.ioloop
import tornado.web
from hashlib import sha1
import os, time

session_container = {}

create_session_id = lambda: sha1(‘%s%s‘ % (os.urandom(16), time.time())).hexdigest()

class Session(object):

    session_id = "__sessionId__"

    def __init__(self, request):
        session_value = request.get_cookie(Session.session_id)
        if not session_value:
            self._id = create_session_id()
        else:
            self._id = session_value
        request.set_cookie(Session.session_id, self._id)

    def __getitem__(self, key):
        return session_container[self._id][key]

    def __setitem__(self, key, value):
        if session_container.has_key(self._id):
            session_container[self._id][key] = value
        else:
            session_container[self._id] = {key: value}

    def __delitem__(self, key):
        del session_container[self._id][key]

class BaseHandler(tornado.web.RequestHandler):

    def initialize(self):
        # my_session[‘k1‘]访问 __getitem__ 方法
        self.my_session = Session(self)

class MainHandler(BaseHandler):

    def get(self):
        print self.my_session[‘c_user‘]
        print self.my_session[‘c_card‘]
        self.write(‘index‘)

class LoginHandler(BaseHandler):

    def get(self):
        self.render(‘login.html‘, **{‘status‘: ‘‘})

    def post(self, *args, **kwargs):

        username = self.get_argument(‘name‘)
        password = self.get_argument(‘pwd‘)
        if username == ‘wupeiqi‘ and password == ‘123‘:

            self.my_session[‘c_user‘] = ‘wupeiqi‘
            self.my_session[‘c_card‘] = ‘12312312309823012‘

            self.redirect(‘/index‘)
        else:
            self.render(‘login.html‘, **{‘status‘: ‘用户名或密码错误‘})

settings = {
    ‘template_path‘: ‘template‘,
    ‘static_path‘: ‘static‘,
    ‘static_url_prefix‘: ‘/static/‘,
    ‘cookie_secret‘: ‘aiuasdhflashjdfoiuashdfiuh‘,
    ‘login_url‘: ‘/login‘
}

application = tornado.web.Application([
    (r"/index", MainHandler),
    (r"/login", LoginHandler),
], **settings)

if __name__ == "__main__":
    application.listen(8888)
    tornado.ioloop.IOLoop.instance().start()

c. Session框架

#!/usr/bin/env python
#coding:utf-8

import sys
import math
from bisect import bisect

if sys.version_info >= (2, 5):
    import hashlib
    md5_constructor = hashlib.md5
else:
    import md5
    md5_constructor = md5.new

class HashRing(object):
    """一致性哈希"""

    def __init__(self,nodes):
        ‘‘‘初始化
        nodes : 初始化的节点，其中包含节点已经节点对应的权重
                默认每一个节点有32个虚拟节点
                对于权重，通过多创建虚拟节点来实现
                如：nodes = [
                        {‘host‘:‘127.0.0.1:8000‘,‘weight‘:1},
                        {‘host‘:‘127.0.0.1:8001‘,‘weight‘:2},
                        {‘host‘:‘127.0.0.1:8002‘,‘weight‘:1},
                    ]
        ‘‘‘

        self.ring = dict()
        self._sorted_keys = []

        self.total_weight = 0

        self.__generate_circle(nodes)

    def __generate_circle(self,nodes):
        for node_info in nodes:
            self.total_weight += node_info.get(‘weight‘,1)

        for node_info in nodes:
            weight = node_info.get(‘weight‘,1)
            node = node_info.get(‘host‘,None)

            virtual_node_count = math.floor((32*len(nodes)*weight) / self.total_weight)
            for i in xrange(0,int(virtual_node_count)):
                key = self.gen_key_thirty_two( ‘%s-%s‘ % (node, i) )
                if self._sorted_keys.__contains__(key):
                    raise Exception(‘该节点已经存在.‘)
                self.ring[key] = node
                self._sorted_keys.append(key)

    def add_node(self,node):
        ‘‘‘ 新建节点
        node : 要添加的节点，格式为：{‘host‘:‘127.0.0.1:8002‘,‘weight‘:1}，其中第一个元素表示节点，第二个元素表示该节点的权重。
        ‘‘‘
        node = node.get(‘host‘,None)
        if not node:
                raise Exception(‘节点的地址不能为空.‘)

        weight = node.get(‘weight‘,1)

        self.total_weight += weight
        nodes_count = len(self._sorted_keys) + 1

        virtual_node_count = math.floor((32 * nodes_count * weight) / self.total_weight)
        for i in xrange(0,int(virtual_node_count)):
            key = self.gen_key_thirty_two( ‘%s-%s‘ % (node, i) )
            if self._sorted_keys.__contains__(key):
                raise Exception(‘该节点已经存在.‘)
            self.ring[key] = node
            self._sorted_keys.append(key)

    def remove_node(self,node):
        ‘‘‘ 移除节点
        node : 要移除的节点 ‘127.0.0.1:8000‘
        ‘‘‘
        for key,value in self.ring.items():
            if value == node:
                del self.ring[key]
                self._sorted_keys.remove(key)

    def get_node(self,string_key):
        ‘‘‘获取 string_key 所在的节点‘‘‘
        pos = self.get_node_pos(string_key)
        if pos is None:
            return None
        return self.ring[ self._sorted_keys[pos]].split(‘:‘)

    def get_node_pos(self,string_key):
        ‘‘‘获取 string_key 所在的节点的索引‘‘‘
        if not self.ring:
            return None

        key = self.gen_key_thirty_two(string_key)
        nodes = self._sorted_keys
        pos = bisect(nodes, key)
        return pos

    def gen_key_thirty_two(self, key):

        m = md5_constructor()
        m.update(key)
        return long(m.hexdigest(), 16)

    def gen_key_sixteen(self,key):

        b_key = self.__hash_digest(key)
        return self.__hash_val(b_key, lambda x: x)

    def __hash_val(self, b_key, entry_fn):
        return (( b_key[entry_fn(3)] << 24)|(b_key[entry_fn(2)] << 16)|(b_key[entry_fn(1)] << 8)| b_key[entry_fn(0)] )

    def __hash_digest(self, key):
        m = md5_constructor()
        m.update(key)
        return map(ord, m.digest())

"""
nodes = [
    {‘host‘:‘127.0.0.1:8000‘,‘weight‘:1},
    {‘host‘:‘127.0.0.1:8001‘,‘weight‘:2},
    {‘host‘:‘127.0.0.1:8002‘,‘weight‘:1},
]

ring = HashRing(nodes)
result = ring.get_node(‘98708798709870987098709879087‘)
print result

"""

一致性哈希

from hashlib import sha1
import os, time

create_session_id = lambda: sha1(‘%s%s‘ % (os.urandom(16), time.time())).hexdigest()

class Session(object):

    session_id = "__sessionId__"

    def __init__(self, request):
        session_value = request.get_cookie(Session.session_id)
        if not session_value:
            self._id = create_session_id()
        else:
            self._id = session_value
        request.set_cookie(Session.session_id, self._id)

    def __getitem__(self, key):
        # 根据 self._id ，在一致性哈西中找到其对应的服务器IP
        # 找到相对应的redis服务器，如： r = redis.StrictRedis(host=‘localhost‘, port=6379, db=0)
        # 使用python redis api 链接
        # 获取数据，即：
        # return self._redis.hget(self._id, name)

    def __setitem__(self, key, value):
        # 根据 self._id ，在一致性哈西中找到其对应的服务器IP
        # 使用python redis api 链接
        # 设置session
        # self._redis.hset(self._id, name, value)

    def __delitem__(self, key):
        # 根据 self._id 找到相对应的redis服务器
        # 使用python redis api 链接
        # 删除，即：
        return self._redis.hdel(self._id, name)

Session

　　

2、自定义模型版定

模型绑定有两个主要功能：

  • 自动生成html表单
  • 用户输入验证

在之前学习的Django中为程序员提供了非常便捷的模型绑定功能，但是在Tornado中，一切需要自己动手！！！、

<!DOCTYPE html>
<html>
<head lang="en">
    <meta charset="UTF-8">
    <title></title>
    <link href="{{static_url("commons.css")}}" rel="stylesheet" />
</head>
<body>
    <h1>hello</h1>
    <form action="/index" method="post">

        <p>hostname: <input type="text" name="host" /> </p>
        <p>ip: <input type="text" name="ip" /> </p>
        <p>port: <input type="text" name="port" /> </p>
        <p>phone: <input type="text" name="phone" /> </p>
        <input type="submit" />
    </form>
</body>
</html>

html

#!/usr/bin/env python
# -*- coding:utf-8 -*-

import tornado.ioloop
import tornado.web
from hashlib import sha1
import os, time
import re

class MainForm(object):
    def __init__(self):
        self.host = "(.*)"
        self.ip = "^(25[0-5]|2[0-4]\d|[0-1]?\d?\d)(\.(25[0-5]|2[0-4]\d|[0-1]?\d?\d)){3}$"
        self.port = ‘(\d+)‘
        self.phone = ‘^1[3|4|5|8][0-9]\d{8}$‘

    def check_valid(self, request):
        form_dict = self.__dict__
        for key, regular in form_dict.items():
            post_value = request.get_argument(key)
            # 让提交的数据 和 定义的正则表达式进行匹配
            ret = re.match(regular, post_value)
            print key,ret,post_value

class MainHandler(tornado.web.RequestHandler):
    def get(self):
        self.render(‘index.html‘)
    def post(self, *args, **kwargs):
        obj = MainForm()
        result = obj.check_valid(self)
        self.write(‘ok‘)

settings = {
    ‘template_path‘: ‘template‘,
    ‘static_path‘: ‘static‘,
    ‘static_url_prefix‘: ‘/static/‘,
    ‘cookie_secret‘: ‘aiuasdhflashjdfoiuashdfiuh‘,
    ‘login_url‘: ‘/login‘
}

application = tornado.web.Application([
    (r"/index", MainHandler),
], **settings)

if __name__ == "__main__":
    application.listen(8888)
    tornado.ioloop.IOLoop.instance().start()

由于请求的验证时，需要考虑是否可以为空以及正则表达式的复用，所以：

#!/usr/bin/env python
# -*- coding:utf-8 -*-

import tornado.ioloop
import tornado.web
import re

class Field(object):

    def __init__(self, error_msg_dict, required):
        self.id_valid = False
        self.value = None
        self.error = None
        self.name = None
        self.error_msg = error_msg_dict
        self.required = required

    def match(self, name, value):
        self.name = name

        if not self.required:
            self.id_valid = True
            self.value = value
        else:
            if not value:
                if self.error_msg.get(‘required‘, None):
                    self.error = self.error_msg[‘required‘]
                else:
                    self.error = "%s is required" % name
            else:
                ret = re.match(self.REGULAR, value)
                if ret:
                    self.id_valid = True
                    self.value = ret.group()
                else:
                    if self.error_msg.get(‘valid‘, None):
                        self.error = self.error_msg[‘valid‘]
                    else:
                        self.error = "%s is invalid" % name

class IPField(Field):
    REGULAR = "^(25[0-5]|2[0-4]\d|[0-1]?\d?\d)(\.(25[0-5]|2[0-4]\d|[0-1]?\d?\d)){3}$"

    def __init__(self, error_msg_dict=None, required=True):

        error_msg = {}  # {‘required‘: ‘IP不能为空‘, ‘valid‘: ‘IP格式错误‘}
        if error_msg_dict:
            error_msg.update(error_msg_dict)

        super(IPField, self).__init__(error_msg_dict=error_msg, required=required)

class IntegerField(Field):
    REGULAR = "^\d+$"

    def __init__(self, error_msg_dict=None, required=True):
        error_msg = {‘required‘: ‘数字不能为空‘, ‘valid‘: ‘数字格式错误‘}
        if error_msg_dict:
            error_msg.update(error_msg_dict)

        super(IntegerField, self).__init__(error_msg_dict=error_msg, required=required)

class CheckBoxField(Field):

    def __init__(self, error_msg_dict=None, required=True):
        error_msg = {}  # {‘required‘: ‘IP不能为空‘, ‘valid‘: ‘IP格式错误‘}
        if error_msg_dict:
            error_msg.update(error_msg_dict)

        super(CheckBoxField, self).__init__(error_msg_dict=error_msg, required=required)

    def match(self, name, value):
        self.name = name

        if not self.required:
            self.id_valid = True
            self.value = value
        else:
            if not value:
                if self.error_msg.get(‘required‘, None):
                    self.error = self.error_msg[‘required‘]
                else:
                    self.error = "%s is required" % name
            else:
                if isinstance(name, list):
                    self.id_valid = True
                    self.value = value
                else:
                    if self.error_msg.get(‘valid‘, None):
                        self.error = self.error_msg[‘valid‘]
                    else:
                        self.error = "%s is invalid" % name

class FileField(Field):
    REGULAR = "^(\w+\.pdf)|(\w+\.mp3)|(\w+\.py)$"

    def __init__(self, error_msg_dict=None, required=True):
        error_msg = {}  # {‘required‘: ‘数字不能为空‘, ‘valid‘: ‘数字格式错误‘}
        if error_msg_dict:
            error_msg.update(error_msg_dict)

        super(FileField, self).__init__(error_msg_dict=error_msg, required=required)

    def match(self, name, value):
        self.name = name
        self.value = []
        if not self.required:
            self.id_valid = True
            self.value = value
        else:
            if not value:
                if self.error_msg.get(‘required‘, None):
                    self.error = self.error_msg[‘required‘]
                else:
                    self.error = "%s is required" % name
            else:
                m = re.compile(self.REGULAR)
                if isinstance(value, list):
                    for file_name in value:
                        r = m.match(file_name)
                        if r:
                            self.value.append(r.group())
                            self.id_valid = True
                        else:
                            self.id_valid = False
                            if self.error_msg.get(‘valid‘, None):
                                self.error = self.error_msg[‘valid‘]
                            else:
                                self.error = "%s is invalid" % name
                            break
                else:
                    if self.error_msg.get(‘valid‘, None):
                        self.error = self.error_msg[‘valid‘]
                    else:
                        self.error = "%s is invalid" % name

    def save(self, request, upload_path=""):

        file_metas = request.files[self.name]
        for meta in file_metas:
            file_name = meta[‘filename‘]
            with open(file_name,‘wb‘) as up:
                up.write(meta[‘body‘])

class Form(object):

    def __init__(self):
        self.value_dict = {}
        self.error_dict = {}
        self.valid_status = True

    def validate(self, request, depth=10, pre_key=""):

        self.initialize()
        self.__valid(self, request, depth, pre_key)

    def initialize(self):
        pass

    def __valid(self, form_obj, request, depth, pre_key):
        """
        验证用户表单请求的数据
        :param form_obj: Form对象（Form派生类的对象）
        :param request: Http请求上下文（用于从请求中获取用户提交的值）
        :param depth: 对Form内容的深度的支持
        :param pre_key: Html中name属性值的前缀（多层Form时，内部递归时设置，无需理会）
        :return: 是否验证通过，True：验证成功；False：验证失败
        """

        depth -= 1
        if depth < 0:
            return None
        form_field_dict = form_obj.__dict__
        for key, field_obj in form_field_dict.items():
            print key,field_obj
            if isinstance(field_obj, Form) or isinstance(field_obj, Field):
                if isinstance(field_obj, Form):
                    # 获取以key开头的所有的值，以参数的形式传至
                    self.__valid(field_obj, request, depth, key)
                    continue
                if pre_key:
                    key = "%s.%s" % (pre_key, key)

                if isinstance(field_obj, CheckBoxField):
                    post_value = request.get_arguments(key, None)
                elif isinstance(field_obj, FileField):
                    post_value = []
                    file_list = request.request.files.get(key, None)
                    for file_item in file_list:
                        post_value.append(file_item[‘filename‘])
                else:
                    post_value = request.get_argument(key, None)

                print post_value
                # 让提交的数据 和 定义的正则表达式进行匹配
                field_obj.match(key, post_value)
                if field_obj.id_valid:
                    self.value_dict[key] = field_obj.value
                else:
                    self.error_dict[key] = field_obj.error
                    self.valid_status = False

class ListForm(object):
    def __init__(self, form_type):
        self.form_type = form_type
        self.valid_status = True
        self.value_dict = {}
        self.error_dict = {}

    def validate(self, request):
        name_list = request.request.arguments.keys() + request.request.files.keys()
        index = 0
        flag = False
        while True:
            pre_key = "[%d]" % index
            for name in name_list:
                if name.startswith(pre_key):
                    flag = True
                    break
            if flag:
                form_obj = self.form_type()
                form_obj.validate(request, depth=10, pre_key="[%d]" % index)
                if form_obj.valid_status:
                    self.value_dict[index] = form_obj.value_dict
                else:
                    self.error_dict[index] = form_obj.error_dict
                    self.valid_status = False
            else:
                break

            index += 1
            flag = False

class MainForm(Form):

    def __init__(self):
        # self.ip = IPField(required=True)
        # self.port = IntegerField(required=True)
        # self.new_ip = IPField(required=True)
        # self.second = SecondForm()
        self.fff = FileField(required=True)
        super(MainForm, self).__init__()

#
# class SecondForm(Form):
#
#     def __init__(self):
#         self.ip = IPField(required=True)
#         self.new_ip = IPField(required=True)
#
#         super(SecondForm, self).__init__()

class MainHandler(tornado.web.RequestHandler):
    def get(self):
        self.render(‘index.html‘)
    def post(self, *args, **kwargs):
        # for i in  dir(self.request):
        #     print i
        # print self.request.arguments
        # print self.request.files
        # print self.request.query
        # name_list = self.request.arguments.keys() + self.request.files.keys()
        # print name_list

        # list_form = ListForm(MainForm)
        # list_form.validate(self)
        #
        # print list_form.valid_status
        # print list_form.value_dict
        # print list_form.error_dict

        # obj = MainForm()
        # obj.validate(self)
        #
        # print "验证结果：", obj.valid_status
        # print "符合验证结果：", obj.value_dict
        # print "错误信息:"
        # for key, item in obj.error_dict.items():
        #     print key,item
        # print self.get_arguments(‘favor‘),type(self.get_arguments(‘favor‘))
        # print self.get_argument(‘favor‘),type(self.get_argument(‘favor‘))
        # print type(self.get_argument(‘fff‘)),self.get_argument(‘fff‘)
        # print self.request.files
        # obj = MainForm()
        # obj.validate(self)
        # print obj.valid_status
        # print obj.value_dict
        # print obj.error_dict
        # print self.request,type(self.request)
        # obj.fff.save(self.request)
        # from tornado.httputil import HTTPServerRequest
        # name_list = self.request.arguments.keys() + self.request.files.keys()
        # print name_list
        # print self.request.files,type(self.request.files)
        # print len(self.request.files.get(‘fff‘))

        # obj = MainForm()
        # obj.validate(self)
        # print obj.valid_status
        # print obj.value_dict
        # print obj.error_dict
        # obj.fff.save(self.request)
        self.write(‘ok‘)

settings = {
    ‘template_path‘: ‘template‘,
    ‘static_path‘: ‘static‘,
    ‘static_url_prefix‘: ‘/static/‘,
    ‘cookie_secret‘: ‘aiuasdhflashjdfoiuashdfiuh‘,
    ‘login_url‘: ‘/login‘
}

application = tornado.web.Application([
    (r"/index", MainHandler),
], **settings)

if __name__ == "__main__":
    application.listen(8888)
    tornado.ioloop.IOLoop.instance().start()

Form验证框架

实现登录：

view.py

import tornado.ioloop
import tornado.web

# 视图
class MainHandler(tornado.web.RequestHandler):
    def get(self):
        #d={"name":"yuan","l":[123,345,666],"info":{"name":"egon","age":"123"}}
        self.render("templates/index.html",**d)

    def post(self, *args, **kwargs):
        pass

class LoginHandler(tornado.web.RequestHandler):
    def get(self, *args, **kwargs):
        self.render("templates/login.html")

    def post(self, *args, **kwargs):
        user=self.get_argument("user")
        pwd=self.get_argument("pwd")
        if user=="alex" and pwd=="123":
            self.redirect("/index")
            print("123ok")
        else:
            self.render("templates/login.html")

login.html

<!DOCTYPE html>
<html lang="en">
<head>
    <meta charset="UTF-8">
    <title>Title</title>
</head>
<body>

<h4>登录页面</h4>

<form action="" method="post">
    <input type="text" name="user">
    <input type="password" name="pwd">
    <input type="submit">
</form>

</body>
</html>

原文地址：https://www.cnblogs.com/mengqingjian/p/8525317.html

时间： 2024-10-11 18:19:11

python 之Tornado的相关文章

使用python的tornado配合html页面示例

背景:java写的非标加密算法,测试时执行java工程进行解密测试,很不方便. 目的:想写个web页面,使得任何测试人员都可以在输入加密串时得到解密后字段,方便日志查询及字段核对.(额,算法部分就不写了.) 步骤:由于内部使用的是非标Base64加密算法,所以我该次操作分为3个部分,以下具体介绍并附带python代码 1 将java的非标加密算法转为python 2 使用python的tornado调用该算法 3 编写html页面,简单明了 一 java非标加密算法转为python  dncry

记一个python+sqlalchemy+tornado的一个高并发下，产生重复记录的bug

场景:在用户通过支付通道支付完成返回时,发现我收到的处理数据记录中有两条同样的数据记录, 也就是同一笔钱,我数据库中记为了两条一样的记录. tornado端代码 from tornado import gen from tornado.concurrent import run_on_executor class processNetPay(BaseHandler): '''处理指定订单,指定支付请求,返回处理结果 ' 返回包含订单信息与用户信息体 ''' @tornado.web.asynch

RHEL6安装python包tornado

RHEL6安装python包tornado tornado是使用Python开发的全栈式(full-stack)Web框架和异步网络库,最早由Friendfeed开发.通过使用非阻塞IO,Tornado可以处理数以万计的开放连接,是long polling.WebSockets和其他需要为用户维护长连接应用的理想选择. tornado 跟其他主流的Web服务器框架(主要是Python框架)不同是采用epoll非阻塞IO,响应快速,可处理数千并发连接,特别适用用于实时的Web服务.Tornado

Centos搭建Python+Nginx+Tornado+Mysql环境

下面我将介绍在UNIX(LINUX.MAC下的配置方法其本一致)下如何快速搭建其运行环境: 1.安装Python wget http://www.python.org/ftp/python/2.7.5/Python-2.7.5.tgz tar xvfz Python-2.7.5.tgzcd Python-2.7.5 ./configure make sudo make install 2.安装Python的包管理工具setuptools.pip和打包工具distribute wget http:

[转载]知乎技术方案初探——基于Python的Tornado框架

原文出处:http://nonfu.me/p/5935.html 知乎的整个网站架构图如下: 知乎技术方案 知乎是国内很少的使用Python开发的一个网站,也很多值得我们学习的地方,从知乎让我们也可以了解到一些新的WEB技术. 一.Python框架 知乎目前使用的是Tornado 框架.Tornado 全称Tornado Web Server,是一个用Python 语言写成的Web 服务器兼Web 应用框架,由 FriendFeed 公司在自己的网站FriendFeed 中使用,被faceboo

python SSTI tornado render模板注入

原理tornado render是python中的一个渲染函数,也就是一种模板,通过调用的参数不同,生成不同的网页,如果用户对render内容可控,不仅可以注入XSS代码,而且还可以通过{{}}进行传递变量和执行简单的表达式.简单的理解例子如下:------------------------------------------------------------------------------------import tornado.ioloopimport tornado.web cla

Python之tornado框架实现翻页功能

1.结果如图所示,这里将html页面与网站的请求处理放在不同地方了 start.py代码 import tornado.ioloop import tornado.web from controllers import home settings = { 'static_path':'statics',#静态文件配置,需要特殊处理 'static_url_prefix':'/sss/',#标记文件开始的名字 } #路由映射,根据不同url对应到不同的类里面 application = torna

python 安装 tornado 、pycrypto、

安装MinGW之后 出现..

异步非阻塞IO的Python Web框架--Tornado

Tornado的全称是Torado Web Server,从名字上就可知它可用作Web服务器,但同时它也是一个Python Web的开发框架.最初是在FriendFeed公司的网站上使用,FaceBook收购之后便进行了开源. 作为Web框架,是一个轻量级的Web框架,类似于另一个Python web 框架Web.py,其拥有异步非阻塞IO的处理方式. 作为Web服务器,Tornado有较为出色的抗负载能力,官方用nginx反向代理的方式部署Tornado和其它Python web应用框架进行对