java如何实现python的urllib.quote(str,safe='/')

最近需要将一些python代码转成java,遇到url编码

urllib.quote(str,safe=‘/‘)

但java中URLEncoder.encode(arg, Constant.UTF_8)会将‘/‘转成%2F

网上查了一下 java没见到类似的safe方式,只好自己实现一个类

package com.ppc.spider.fc.util;
import java.io.ByteArrayOutputStream;
import java.io.BufferedWriter;
import java.io.OutputStreamWriter;
import java.io.IOException;
import java.io.UnsupportedEncodingException;
import java.net.URLDecoder;
import java.io.CharArrayWriter;
import java.nio.charset.Charset;
import java.nio.charset.IllegalCharsetNameException;
import java.nio.charset.UnsupportedCharsetException ;
import java.util.BitSet;
import java.security.AccessController;
import sun.security.action.GetPropertyAction;

public class UrlSafeEncoder {

    static BitSet dontNeedEncoding;
    static final int caseDiff = (‘a‘ - ‘A‘);
    static String dfltEncName = null;

    static {

        /* The list of characters that are not encoded has been
         * determined as follows:
         *
         * RFC 2396 states:
         * -----
         * Data characters that are allowed in a URI but do not have a
         * reserved purpose are called unreserved.  These include upper
         * and lower case letters, decimal digits, and a limited set of
         * punctuation marks and symbols.
         *
         * unreserved  = alphanum | mark
         *
         * mark        = "-" | "_" | "." | "!" | "~" | "*" | "‘" | "(" | ")"
         *
         * Unreserved characters can be escaped without changing the
         * semantics of the URI, but this should not be done unless the
         * URI is being used in a context that does not allow the
         * unescaped character to appear.
         * -----
         *
         * It appears that both Netscape and Internet Explorer escape
         * all special characters from this list with the exception
         * of "-", "_", ".", "*". While it is not clear why they are
         * escaping the other characters, perhaps it is safest to
         * assume that there might be contexts in which the others
         * are unsafe if not escaped. Therefore, we will use the same
         * list. It is also noteworthy that this is consistent with
         * O‘Reilly‘s "HTML: The Definitive Guide" (page 164).
         *
         * As a last note, Intenet Explorer does not encode the "@"
         * character which is clearly not unreserved according to the
         * RFC. We are being consistent with the RFC in this matter,
         * as is Netscape.
         *
         */

        dontNeedEncoding = new BitSet(256);
        int i;
        for (i = ‘a‘; i <= ‘z‘; i++) {
            dontNeedEncoding.set(i);
        }
        for (i = ‘A‘; i <= ‘Z‘; i++) {
            dontNeedEncoding.set(i);
        }
        for (i = ‘0‘; i <= ‘9‘; i++) {
            dontNeedEncoding.set(i);
        }
        dontNeedEncoding.set(‘ ‘); /* encoding a space to a + is done
                                    * in the encode() method */
        dontNeedEncoding.set(‘-‘);
        dontNeedEncoding.set(‘_‘);
        dontNeedEncoding.set(‘.‘);
        dontNeedEncoding.set(‘*‘);

        dfltEncName = AccessController.doPrivileged(
            new GetPropertyAction("file.encoding")
        );
    }

    /**
     * You can‘t call the constructor.
     */
    private UrlSafeEncoder() { }
    /**
     * Translates a string into {@code application/x-www-form-urlencoded}
     * format using a specific encoding scheme. This method uses the
     * supplied encoding scheme to obtain the bytes for unsafe
     * characters.
     * <p>
     * <em><strong>Note:</strong> The <a href=
     * "http://www.w3.org/TR/html40/appendix/notes.html#non-ascii-chars">
     * World Wide Web Consortium Recommendation</a> states that
     * UTF-8 should be used. Not doing so may introduce
     * incompatibilities.</em>
     *
     * @param   s   {@code String} to be translated.
     * @param   enc   The name of a supported
     *    <a href="../lang/package-summary.html#charenc">character
     *    encoding</a>.
     * @return  the translated {@code String}.
     * @exception  UnsupportedEncodingException
     *             If the named encoding is not supported
     * @see URLDecoder#decode(java.lang.String, java.lang.String)
     * @since 1.4
     */
    public static String encode(String s, String enc,char safe)
        throws UnsupportedEncodingException {
        dontNeedEncoding.set(safe);
        boolean needToChange = false;
        StringBuffer out = new StringBuffer(s.length());
        Charset charset;
        CharArrayWriter charArrayWriter = new CharArrayWriter();

        if (enc == null)
            throw new NullPointerException("charsetName");

        try {
            charset = Charset.forName(enc);
        } catch (IllegalCharsetNameException e) {
            throw new UnsupportedEncodingException(enc);
        } catch (UnsupportedCharsetException e) {
            throw new UnsupportedEncodingException(enc);
        }

        for (int i = 0; i < s.length();) {
            int c = (int) s.charAt(i);
            //System.out.println("Examining character: " + c);
            if (dontNeedEncoding.get(c)) {
                if (c == ‘ ‘) {
                    c = ‘+‘;
                    needToChange = true;
                }
                //System.out.println("Storing: " + c);
                out.append((char)c);
                i++;
            } else {
                // convert to external encoding before hex conversion
                do {
                    charArrayWriter.write(c);
                    /*
                     * If this character represents the start of a Unicode
                     * surrogate pair, then pass in two characters. It‘s not
                     * clear what should be done if a bytes reserved in the
                     * surrogate pairs range occurs outside of a legal
                     * surrogate pair. For now, just treat it as if it were
                     * any other character.
                     */
                    if (c >= 0xD800 && c <= 0xDBFF) {
                        /*
                          System.out.println(Integer.toHexString(c)
                          + " is high surrogate");
                        */
                        if ( (i+1) < s.length()) {
                            int d = (int) s.charAt(i+1);
                            /*
                              System.out.println("\tExamining "
                              + Integer.toHexString(d));
                            */
                            if (d >= 0xDC00 && d <= 0xDFFF) {
                                /*
                                  System.out.println("\t"
                                  + Integer.toHexString(d)
                                  + " is low surrogate");
                                */
                                charArrayWriter.write(d);
                                i++;
                            }
                        }
                    }
                    i++;
                } while (i < s.length() && !dontNeedEncoding.get((c = (int) s.charAt(i))));

                charArrayWriter.flush();
                String str = new String(charArrayWriter.toCharArray());
                byte[] ba = str.getBytes(charset);
                for (int j = 0; j < ba.length; j++) {
                    out.append(‘%‘);
                    char ch = Character.forDigit((ba[j] >> 4) & 0xF, 16);
                    // converting to use uppercase letter as part of
                    // the hex value if ch is a letter.
                    if (Character.isLetter(ch)) {
                        ch -= caseDiff;
                    }
                    out.append(ch);
                    ch = Character.forDigit(ba[j] & 0xF, 16);
                    if (Character.isLetter(ch)) {
                        ch -= caseDiff;
                    }
                    out.append(ch);
                }
                charArrayWriter.reset();
                needToChange = true;
            }
        }

        return (needToChange? out.toString() : s);
    }
}

验证下 基本ok

java如何实现python的urllib.quote(str,safe='/')

原文地址:https://www.cnblogs.com/davidwang456/p/9476384.html

时间: 2024-11-10 13:52:38

java如何实现python的urllib.quote(str,safe='/')的相关文章

python爬虫Urllib实战

Urllib基础 urllib.request.urlretrieve(url,filenname) 直接将网页下载到本地 import urllib.request >>> urllib.request.urlretrieve("http://www.hellobi.com",filename="D:\/1.html") ('D:\\/1.html', <http.client.HTTPMessage object at 0x0000000

【Python】Python的urllib、urllib2模块调用“百度翻译”API进行批量自动翻译

1.问题描述 在文本数据处理时,经常回出现文本中各种语言的混杂情况,包括:英文.日语.俄语.法语等,需要将不同语种的语言批量翻译成中文进行处理.可以通过Python直接调用百度提供的翻译API进行批量的翻译. 百度翻译API详细文档见:百度翻译API文档 2.问题解决 开发环境:Linux 将文本中的中文和非中文进行分离,对非中文的部分进行翻译. Python的代码如下:translate.py #!/usr/bin/python #-*- coding:utf-8 -*- import sys

001 发大招了 神器的效率工具--Java代码转python代码

今天发现一个好玩的工具:可以直接将java转成python? 1. 安装工具(windows 环境下面)? 先下载antlr:? 下载链接如下: http://www.antlr3.org/download/antlr-3.1.3.tar.gz? 或者到百度云下载: 百度云链接:http://pan.baidu.com/s/1gdgXUM3 密码:2qrx? ? ? 下载成功并解压后,进入\antlr-3.1.3\runtime目录,输入CMD,在CMD中输入如下指令: python setup

学习Python的urllib模块

 urllib 模块作为Python 3 处理 URL 的组件集合,如果你有 Python 2 的知识,那么你就会注意到 Python 2 中有 urllib 和 urllib2 两个版本的模块,这些现在都是 Python 3 的 urllib 包的一部分,具体如何来体现它们之间的关系 Python 3 的 urllib 模块是一堆可以处理 URL 的组件集合.如果你有 Python 2 的知识,那么你就会注意到 Python 2 中有 urllib 和 urllib2 两个版本的模块.这些现在

定义一个方法get_page(url),url参数是需要获取网页内容的网址,返回网页的内容。提示(可以了解python的urllib模块)

1 定义一个方法get_page(url),url参数是需要获取网页内容的网址,返回网页的内容.提示(可以了解python的urllib模块) 2 import urllib.request 3 4 def get_page(url): 5 response = urllib.request.urlopen(url) 6 html = response.read() 7 return html 8 9 print(get_page(url='https://www.baidu,com'))

Python错误:TypeError:&#39;str&#39; does not support the buffer interface

在socket套接字模块进行send和recv方法时出现这种问题,是因为Python3.x和Python2.x版本变化,In python 3, bytes strings and unicodestrings are now two different types. 相互之间需要进行转换decode()和encode(). send()需要的参数为bytes类型,因此需要对str进行encode() recv()返回的是bytes类型,因此我们需要对返回的bytes进行decode()转换为s

Python库urllib与urllib2有哪些区别

分享下Python库urllib与urllib2用法区别,初学python的同学常有此困惑,今天一揭谜底. 学习Python,一直不明白urllib和urllib2的区别,以为2是1的升级版.今天看到老外写的一篇<Python: difference between urllib and urllib2>才明白其中的区别You might be intrigued by the existence of two separate URL modules in Python -urllib an

在java中调用python方法

1.http://sourceforge.net/projects/jython/下载jython包,把其中的jython.jar添加到工程目录 示例: 1.摘自:http://blog.csdn.net/anbo724/article/details/6608632 1.在java类中直接执行python语句 import javax.script.*; import org.python.util.PythonInterpreter; import java.io.*; import sta

为什么用 Java:一个 Python 程序员告诉你

这篇文章专门给程序员写的,普通读者慎入.原作者:Kevin Sookocheff 译者:Celia Zhen,原文点击文末链接. 每当我告诉别人我一直在用Java工作时,大家的反应都是: “纳尼!Java?为啥是Java?” 说实话,本人刚开始的时候也是同样的反应.但是由于Java的类型安全,执行性能和坚如磐石的工具,我渐渐地开始欣赏Java.同时我注意到,现在的Java已今非昔比——它在过去的10年间稳健地改善着. 缘何是Java? 假 设每天都用Java的想法还没有让君恶心到食不下咽,我在此