Loading

Thursday, July 29, 2010

S3 Bucket Policy to Restrict Access by Referrer, Yet Allow Direct Access to File(s)

Recently Amazon rolled out S3 Bucket Policies (see Access Policy Language) to more finely control access to S3 buckets or resources in buckets, than with just ACL's alone.  This was very timely as I had a need arise to use a bucket policy just after it came out.  Basically I needed to block access of a single file, let's call it xyz.htm, from certain referrers, yet allow all others.  After a little research and some trial-and-error I was able to define a policy which did just this:
{
"Version":"2008-10-17",
"Id":"mydomain-widgettest",
"Statement":[{
"Sid":"1",
"Effect":"Deny",
"Principal":{
"AWS":"*"
},
"Action":"s3:GetObject",
"Resource":"arn:aws:s3:::widgettest.mydomain.com/xyz.htm",
"Condition":{
"StringLike":{
"aws:Referer":[
" http://blockedreferer1.com/*",
" http://blockedreferer2.net/*",
]}}},
{
"Sid":"2",
"Effect":"Allow",
"Principal":{
"AWS":"*"
},
"Action":"s3:GetObject",
"Resource":"arn:aws:s3:::widgettest.mydomain.com/xyz.htm",
"Condition":{
"StringLike":{
"aws:Referer":[
"*",
" http://widgettest.mydomain.com/*"
]}}}]}
However, this had the undesired effect of blocking direct access to the file, i.e. http://widgettest.mydomain.com/xyz.htm, where there is no referrer, or the referrer is null.  This one took me a little longer to figure out, and a key piece of it was found in the Amazon developer forums.  I was then able to write a bucket policy which behaves as desired:
{
"Version":"2008-10-17",
"Id":"mydomain-widgettest",
"Statement":[{
"Sid":"1- Allow direct access to xyz.htm - i.e. no referrer.",
"Effect":"Allow",
"Principal":{
"AWS":"*"
},
"Action":"s3:GetObject",
"Resource":"arn:aws:s3:::widgettest.mydomain.com/xyz.htm",
"Condition":{
"Null":{
"aws:Referer":true
}}},
{
"Sid":"2- Allow all referrers to xyz.htm except those listed.",
"Effect":"Allow",
"Principal":{
"AWS":"*"
},
"Action":"s3:GetObject",
"Resource":"arn:aws:s3:::widgettest.mydomain.com/xyz.htm",
"Condition":{
"StringNotLike":{
"aws:Referer":[
" http://blockedreferer1.com/*",
" http://blockedreferer2.net/*"
]}}}]}
This policy effectively allows direct access to xyz.htm (null or "no" referrer), and allows access to all referrers except those explicitly listed in the Sid:2 section.  One important note is that "public" read access must not be set in the ACL for this file as it will allow anyone access, effectively bypassing this policy.

NOTES:
  • Amazon S3 bucket policies use JSON.  If you aren't familiar with JSON as I wasn't you can read more here.
  • I found a handy JSON Formatter and Validator - to do just that. . .
  • Since Amazon doesn't provide an easy method for us non-programmers to apply bucket policies I found CloudBerry S3 Bucket Explorer Pro essential and simple to use to apply bucket policies.
  • Sometimes as I applied a policy to test I would receive the message "invalid aspen elements," which basically mean something is wrong, usually one of the required elements was either missing or incorrect, and, interestingly no results were found using Google.
See Also:

2 comments:

  1. This is a great example of useing Amazon s3 to restrict by referer. I struggled with this myself until I found this post.

    ReplyDelete
  2. Great post mate. You are right there arent any places on the net explaining any of this. Ive been scratching my head so much with it!

    One example of why im interested is I think you might have the solution to fixing Google Instant Preview working with missing images.

    I did try your code however using Cloudberry Pro but its stating the policy is invalid and the only thing I can narrow it down I recogn is:

    "Null":{
    "aws:Referer":true
    }}},

    As you cannot state anything like that in the builder built into Cloudberry.

    ReplyDelete