importjava.io.File;
import com.amazonaws.AmazonClientException;
import com.amazonaws.auth.profile.ProfileCredentialsProvider;
import com.amazonaws.services.s3.transfer.TransferManager;
import com.amazonaws.services.s3.transfer.Upload;
public class UploadObjectMultipartUploadUsingHighLevelAPI {
public static void main(String[] args) throws Exception {
String existingBucketName = "*** Provide existing bucket name ***";
String keyName = "*** Provide object key ***";
String filePath = "*** Path to and name of the file to upload ***";
TransferManager tm = new TransferManager(new ProfileCredentialsProvider());
System.out.println("Hello");
// TransferManager processes all transfers asynchronously,
// so this call will return immediately.
Upload upload = tm.upload(
existingBucketName, keyName, new File(filePath));
System.out.println("Hello2");
try {
// Or you can block and wait for the upload to finish
upload.waitForCompletion();
System.out.println("Upload complete.");
} catch (AmazonClientException amazonClientException) {
System.out.println("Unable to upload file, upload was aborted.");
amazonClientException.printStackTrace();
}
}
}
问题描述：使用s3 java sdk 的如上代码分片文件上传，报‘SignatureDoesNotMatch’ 异常如下
Unable to abort multipart upload, you may need to manually remove uploaded parts: null (Service: Amazon S3; Status Code: 403; Error Code: SignatureDoesNotMatch; Request ID: tx000000000000000003ddf-005853b76e-1a56f9b-default)
com.amazonaws.services.s3.model.AmazonS3Exception: null (Service: Amazon S3; Status Code: 403; Error Code: SignatureDoesNotMatch; Request ID: tx000000000000000003ddf-005853b76e-1a56f9b-default), S3 Extended Request ID: 1a56f9b-default-default
定位过程：
下载s3 java sdk 源码，并配置好 log4j, 打印aws debug日志；
# For JBoss: Avoid to setup Log4J outside $JBOSS_HOME/server/default/deploy/log4j.xml! # For all other servers: Comment out the Log4J listener in web.xml to activate Log4J. #log4j.rootLogger=INFO, stdout, logfile log4j.rootLogger=INFO,stdout,logfile log4j.logger.com.amazonaws = DEBUG log4j.appender.stdout=org.apache.log4j.ConsoleAppender log4j.appender.stdout.layout=org.apache.log4j.PatternLayout log4j.appender.stdout.layout.ConversionPattern=%d %p [%c] - <%m>%n
RGW配置文件/etc/ceph/ceph.conf 中的日志级别调整为20，重启RGW进程
debug rgw = 20
执行分片上传代码，分别截获客户端和服务端的请求日志。 如下2个图片可以发现uploadId客户端和服务端不一致，导致服务端进行签名校验失败。
那么问题来了，到底是RGW服务端，还是java s3 sdk实现有bug呢？ 通过查看java 代码和aws文档描述，我认为是RGW签名实现存在bug。 RGW 应该在
获取从客户端url中的参数中uploadId值（已经被URLEncode）后，先decode，再根据aws文档描述的步骤进行encode（~特殊字符不encode），得到符合规则的参数字符串。
http://docs.aws.amazon.com/general/latest/gr/sigv4-create-canonical-request.html
规避方案：
因为修改RGW的代码周期比较长，暂时先修改好修改的java代码，编译一个新的jar包。 如下是修改方法，不按照aws的规则来，对参数值只执行URLencode。
最终还是要提交一个patch到rgw git库。
https://github.com/BodihTao/aws-sdk-java/blob/master/aws-java-sdk-core/src/main/java/com/amazonaws/auth/AbstractAWSSigner.java
为了方便大家，我上传修改后编译好的class文件，把这2个文件用winrar 放到aws-java-sdk-1.11.69.jar包中com/amazonaws/auth目录下 替换原来文件即可得到新的jar包。
https://github.com/BodihTao/aws-sdk-java/raw/master/aws-java-sdk/AbstractAWSSigner%241.class
https://github.com/BodihTao/aws-sdk-java/raw/master/aws-java-sdk/AbstractAWSSigner.class