January 25, 2012

Accessing Windows Azure REST APIs with cURL

Tonight I was playing with cURL on my Mac wondering how easy would it be to develop few scripts to manage Windows Azure applications from non-Windows machine. As it turns out getting access to Windows Azure REST APIs was quite simple. Here are the steps I had to go though in order to be able to receive valid response from the APIs:


Set up Windows Azure management certificate from your Mac machine

The first thing I had to do is to create a self signed certificate that I can use to do the Service Management. Creating the cert with openssl (which is available on Mac) is quite simple - just type:


openssl req -x509 -nodes -days 365 -newkey rsa:2048 -keyout azure-cert.pem -out azure-cert.pem


During the creation openssl will ask you for all the necessary information like country name, organization name etc. and at the end will generate .pem file that contains the public and the private key.

In order to upload the certificate to your Windows Azure subscription using the Management Portal though you need to have the certificate in PKCS12 (or .pfx) format. Here is the openssl command that will do the work:


openssl pkcs12 -export -out azure-cert.pfx -in azure-cert.pem -name "My Self Signed Cert"


Now that you have the PKCS12 file you can go ahead and upload this to your Management Certificates using the portal.


Update: By writing this in the middle of the night I totally messed up what you need to do. PKCS12 you need if you want to enable SSL for your service. For management you only need the public key that you can export in .CER file. Here is the command that you use for this:


openssl x509 -outform der -in azure-cert.pem -out azure-cert.cer


Now you can upload the .CER to the Management Certificates section using the portal.


Windows Azure Management Certificates - Management Portal Screenshot


The initial set-up is done!


Using cURL to Access Windows Azure REST APIs

Now that you have the cert created and uploaded to Windows Azure you can easily play with the REST APIs. For example if you want to list all your existing hosted services you can use the List Hosted Services API as follows:


curl -E [cert-file] -H "x-ms-version: 2011-10-01" "https://management.core.windows.net/[subscr-id]/services/hostedservices"

where:

  • cert-file is the path to the .pem file containing the certificate
  • subscr-id is your Windows Azure subscription ID

Don't forget to specify the version header (the -H flag for cURL) else Windows Azure will return an error. As a result of the call above you will receive XML response with list of all the hosted services in your Windows Azure subscriptioin.


You can access any of the REST APIs by manually constructing the request and the URL as described in the Windows Azure Service Management REST API Reference.


I didn't get to any of my planned scripts but I can explore the APIs easily cURL.


September 11, 2011

Demystifying physicalDirectory or How to Configure the Site Entry in the Service Definition File for Windows Azure

If you played a bit more with the sites configuration in Windows Azure you may have discovered some inconsistent behavior between what Visual Studio does and what the cspack.exe command line does when it relates to physicalDirecroty attribute. I certainly did! Here is the problem I encountered while trying to deploy PHP on Windows Azure.

 

Project Folder Structure

I was following the instructions on Installing PHP on Windows Azure leveraging Full IIS Support but decided to leverage the help of Visual Studio instead building the package by hand. Not a good idea for this particular scenario :( After creating my cloud solution in Visual Studio I ended up with the following folder structure:

+ PHPonAzureSol

     + PHPonAzure

          …

          - ServiceConfiguration.cscfg

          - ServiceDefinition.csdef

     + PHPRole

          …

          + bin

               - install-php.cmd

               - install-php-azure.cmd

          + PHP-Azure

               - php-azure.dll

          + Sites

               + PHP

                    - index.php

          + WebPI-cmd

               …

 

Where:

  • PHPonAzureSol was my VS solution folder
  • PHPonAzure was my VS project folder containing the CSDEF and CSCFG files
  • and PHPRole was my VS project folder containing the code for my web role

The PHPRole folder contained the WebPI command line tool needed to install PHP in the cloud stored in the WebPI-cmd subfolder; the PHP extensions for Azure in the PHP-Azure subfolder; the installation scripts in the bin subfolder; and most importantly my PHP pages in Sites\PHP subfolder (in this case I had simple index.php page containing phpinfo()).

 

Configuring Site Entry in the CSCFG File

Of course my goal was to configure the site to point to the folder where my PHP files were stored. In this particular case this was the PHPonAzureSol\PHPRole\Sites\PHP folder if you follow the structure above. This is simply done by adding the physicalDirectory attribute to the Site tag in CSDEF. Here is how my Site tag looked like:

 

<Site name="Web" physicalDirectory="..\PHPRole\Sites\PHP">
     <Bindings>
          <Binding name="Endpoint1" endpointName="Endpoint1" />
     </Bindings>
</Site>

 

 

My expectation was that with this setting in CSDEF IIS will be configured to point to the content that comes from the physicalDirectory folder. Hence if I type the URL of my Windows Azure hosted service I should be able to see the index.php page (i.e. http://[my-hosted-service].cloudapp.net should point to your PHP code).

 

Visual Studio handling of physicalDirectory attribute

Of course when I used Visual Studio to pack and deploy my Web Role I was unpleasantly surprised. It seems Visual Studio ignores the physicalDirectory attribute from your CSDEF file, and points the site to your Web Role’s approot folder (or the content from PHPRole folder if you follow the structure above). Thus if I wanted to access my PHP page I had to type the following URL:

 

http://[my-hosted-service].cloudapp.net/Sites/PHP/index.php

 

Not exactly what I wanted :(

The reason for this is that Visual Studio calls cspack.exe with additional options (either /sitePhysicalDirectories or /sites) that overwrite the physicalDirectory attribute from CSDEF. As of now I am not aware of a way to change this behavior in VS.

Update (9-12-2011): As it seems VS ignores the physicalDirectory attribute ONLY if your web site is called Web (i.e. name="Web" as in the example above). If you rename the site to something else (name="PHPWeb" for example) you will end up with the expected behavior described below. Unfortunately name="Web" is the default setting, and this may result in unexpected behavior for your application.

 

cspack.exe handling of physicalDirectory attribute

Solution to the problem is to call cspack.exe from the command line (without the above mentioned options of course:)).

There are few gotchas about how you call cspack.exe using the folder structure that Visual Studio creates. After few trial-and-errors where I received several of those errors:

 

Error: Could not find a part of the path '[some-path-here]'.

 

I figured out that you should call cspack.exe from the solution folder (PHPonAzureSol in the above structure). Once I did this everything worked fine and I was able to access my index.php by just typing my hosted service’s URL.

 

How physicalDirectory attribute works?

For those of you interested how the physicalDirectory attribute works here is a simple explanation.

MSDN documentation for How to Configure a Web Role for Multiple Web Sites points out that physicalDirectory attribute value is relative to the location of the Service Configuration (CSCFG) file. This is true in the majority of the cases however I think the following two clarifications are necessary:

  1. Because the attribute is present in the Service Definition (CSDEF) file the correct statement is that physicalDirectory attribute value is relative to the location of the Service Definition (CSDEF) file instead. Of course if you use Visual Studio to build your folder structure you can always assume that the Service Configuration (CSCFG) and the Service Definition (CSDEF) files are placed in the same folder. If you build your project manually you should be careful how you set the physicalDirectory attribute. This is of course important if you want to use relative paths in the attribute.
  2. This one I think is much more important than the first one, and it states that you can use absolute paths in the physicalDirectory attribute. The physicalDirectory attribute can contain any valid absolute path on the machine where you build the package. This means that you can point cspack.exe to include any random folder from your local machine as your site’s root.

Here is how this works.

What cspack.exe does is to take the content of the folder configured in physicalDirectory attribute and copy it under [role]/sitesroot/[num] folder in the package. Here is how my package structure looked like (follow the path in the address line):

 

image

 

During deployment IIS on the cloud VM is configured to point the site to sitesroot\[num] folder, and serve the content from there. Here is how it is deployed in the cloud:

 

image

 

And here is the IIS configuration for this cloud VM:

 

image

August 30, 2011

Microsoft Windows Azure Development Cookbook Review

Recently I got my hands on the Microsoft Windows Azure Development Cookbook book written by Neil Mackenzie (@mknz), and honestly I was impress with the quality of the examples he gives into it. As a disclaimer I have to say that the book was sent to me by the publisher however there are no incentives for me to write this review. Once again I think the book provides very good hands on examples for the complete set of services that Windows Azure offers – starting with Windows Azure Storage, going through the Hosted Services and ending with SQL Azure and AppFabric.

The value I saw in the book is that 1.) it explains each of the services with real examples, and walks you step by step through what is happening, and 2.) is written from the customer point of view (Neil is a MVP, who is dealing with Windows Azure since it was announced at PDC08). I personally found the following three chapters very valuable even for me:

  • Using the Shared Access  Signature for a container blob
  • Using the retry policies with blob operations
  • Autoscaling with the Windows Azure Service Management REST API

I am sure that I will refer to those three chapters very often for my own work on Windows Azure.

As a suggestion I would like to say to Neil that in the next edition I would be happy to see a section dedicated to the troubleshooting, debugging and profiling services on Windows Azure. From my own experience I can say that the majority of question I’ve seen have been in this area, and his experience with the platform as well as his independent view will allow him to provide really good troubleshooting tips and helpful workarounds.

I know Neil remotely, and we have interacted couple of time on MSDN forums or via Twitter. I can say that he is very knowledgeable about the topic, and you can learn a lot from the book and his personal blog.

 

I hope you will find the book a good resource, and kudos to Neil for writing it!