Since I have been using AWS for quite some time (since 2008) I have witnessed benefited from many of the innovations AWS has rolled out over the years. In this post I want to discuss a couple of specific values with VPC Endpoints (for S3 specifically – technically, Gateway endpoints for Amazon S3), including leveraging […]
Category: S3
7-zip (7zip) for Windows (7, 2008, etc.) Rocks!
I’ve been using 7-zip for a long time now. It quickly became my favorite zip/unzip utility. It’s quick, easy to use, opensource, and best of all, FREE. A few months ago I worked on a project to gather and zip log files from numerous S3 buckets and archive them in another location. Initially I specified the […]
S3 Bucket Policy to Restrict Access by Referrer, Yet Allow Direct Access to File(s)
Recently Amazon rolled out S3 Bucket Policies (see Access Policy Language) to more finely control access to S3 buckets or resources in buckets, than with just ACL’s alone. This was very timely as I had a need arise to use a bucket policy just after it came out. Basically I needed to block access of […]
Get Yesterday’s date in MS DOS Batch file
A while back while I was trying to figure out the best way to gather some log files from Amazon S3 buckets and some web servers I run. These resources are currently generating around 10-15GB of uncompressed log files daily. Besides being fairly large in size the S3 (and CloudFront) log files are numerous. Any […]
Amazon S3 Command Line Utilities for Windows
I’ve searched high and low for a good all around command line utility to interact with Amazon S3 buckets from Windows. While I’m still searching for just the right utility for me here are a few which I use from time-to-time. Why use more than one, you ask? Well, since I haven’t found just […]
Web log analysis and statistics for Amazon S3 with S3STAT
I’ve been using Amazon Web Services for several months now. Like anything else I need to know what’s going on with my services – what’s being downloaded, how often, from where, etc. In the middle of last month I finally found a service which bridges the gap allowing a good view into what’s going on […]
Analyzing Amazon S3 Logs with AWStats
The Amazon Simple Storage Service (Amazon S3) provides virtually limitless storage accessible over the Internet. Along with this functionality, however, comes the need to understand how the data stored on Amazon S3 is being used. Amazon S3 supports logging each request made to a given Amazon S3 bucket to a text file. Out of the […]
Glossary of Amazon EC2 terms
Amazon machine image (AMI)An Amazon Machine Image (AMI) is an encrypted machine image stored in Amazon S3. It contains all the information necessary to boot instances of your software. Amazon EBSA type of storage that enables you to create volumes that can be mounted as devices by Amazon EC2 instances. Amazon EBS volumes behave like […]
Amazon Elastic Compute Cloud (EC2) Command Line Tools Reference
The Amazon Elastic Compute Cloud Command Line Tools Reference Guide provides the syntax, a description, options, and usage examples for each command line tool. This section describes who should read this guide, how the guide is organized, and other resources related to Amazon Elastic Compute Cloud. The Amazon Elastic Compute Cloud is occasionally referred to […]
How to run Bucket Commander: A command line interface for Amazon S3
Bucket commander is a command line tool for Amazon S3. Bucket Commander needs a configuration file, which can be created using Bucket Explorer’s UI. Bucket Commander takes three arguments. ” -action ” , ” -authenticate ” and ” -emailprofile ” “ -emailprofile” is optional argument , you need to specify it only when you have […]