Also available at

Also available at my website and on Twitter @toshafanasiev

Tuesday, 4 May 2010

Amazon S3 url gotcha

I have recently been tasked with producing a 'one click' operation to provision a Windows server on EC2 to host an SQL Server database, a web application and an FTP server ( I won't go into the details of my solution as they warrant a post of their own ).

In order to get a production ready server online and secure from scratch I've had to
  • provision storage with S3
  • configure Elastic Block Storage ( EBS ) disks
  • upload and register a previously bundled Amazon Machine Image ( AMI )
  • configure Security Groups ( AWS firewalls )
  • manage key pairs
  • allocate Elastic IP addresses
all in all making it a really engaging introduction to the Amazon Web Services API.

One interesting featurette of the S3 service that particularly stood out was the impact of AWS Region on access to objects stored under S3.

Under the default, US - Standard ( East Coast ) Region objects can be accessed by one of two methods - the  subdomain way and the path way; the urls http://<bucket-name><key-name> ( subdomain ) and<bucket-name>/<key-name> ( path ) are both valid.

In order to reduce latency you could choose to deploy in one of the other regions ( US - Northern California, EU - Ireland or the newly added Asia Pacific - Singapore ) as the geographical distribution of your user base dictates. If you do, note that the path form of S3 object urls is not valid ( at time of writing 2006-03-01 was the latest version, see for details ) and the subdomain version must be used instead.

This poses no problem so long as you have control over the source of urls or your code can be made to follow 301 redirects but a problem I ran into was that without modification neither the official Java sdk ( ) nor .NET sdk ( ) can register AMIs from bundled data stored in S3 buckets in any region other than US - Standard. This is due to the fact that '' is prepended to any string you give them that does not already start that way, and for all regions but US - Standard the path form that is insisted upon results in a 301.

As it happens the user base of the product I am working on is never expected to grow beyond quite a small number and so I can afford the slight hit in latency terms and simply deploy to the default region.

I've written this post in case anyone else encounters the same problem and can either save themselves some trawling or let me know a simple way around this for future reference.


  1. You won't believe me, but I was planning to write a blog very similar to the one you have posted here. Great work!

    AWS Training In Chennai

    Hadoop Training In Chennai

    1. Great Article Cloud Computing Projects

      Networking Projects

      Final Year Projects for CSE

      JavaScript Training in Chennai

      JavaScript Training in Chennai

      The Angular Training covers a wide range of topics including Components, Angular Directives, Angular Services, Pipes, security fundamentals, Routing, and Angular programmability. The new Angular TRaining will lay the foundation you need to specialise in Single Page Application developer. Angular Training

  2. Thank you for the informative post about Security challenges in AWS , Found it useful . cloud migration services have now become secured and with no-risk

    Cloud Migration services

    Aws Cloud Migration services

    Azure Cloud Migration services

  3. I have gone through your post and I found it very helpfull. Looking forward to see more post from you.

    Vmware Cloud Migration services

    Database Migration services

  4. I am really impressed with the way of writing of this blog. The author has shared the info in a crisp and short way.

    Lia Infraservices

  5. An issue arises when there is no clear ownership of processes that move products within a distribution channel. visit site

  6. Enumerated as follows are a few tips to save the day! Oven Repair Services in Los Angeles

  7. For an overall idea, when do you get low scores for your blog? deep web sites