PyAWS - Adding Request Authentication
From August 2009 Amazon requires you to add an authentication signature to every request to their Product Advertising API (ECS). I have been using the wonderful PyAWS library for a few years so took upon myself to add this feature.
(This post was ammended on 30/07/2009 in response to a comment from Nick Fishman, the script now uses GMT time, which is was not doing before. Thanks Nick.)
Here is a summary of the request authentication requirements from the Amazon documentation:
- Sort the UTF-8 query string components by parameter name with natural byte ordering.
- URL encode the parameter name and values
- Separate the encoded parameter names from their encoded values with the equals sign (duh)
- Separate the name-value pairs with an ampersand
- Create the string to sign according to the following pseudo-grammar (StringToSign = HTTPVerb + "n" + ValueOfHostHeaderInLowercase + "n" + HTTPRequestURI + "n" + CanonicalizedQueryString
- Calculate an RFC 2104-compliant HMAC with the string you just created, your Secret Access Key as the key, and SHA256 as the hash algorithm.
- Calculate an RFC 2104-compliant HMAC with the string you just created, your Secret Access Key as the key, and SHA256 as the hash algorithm.
- Use the resulting value as the value of the Signature request parameter.
You'll need to login to your amazon aws account to find out your secret key.
The main funciton to do this looks like this:
def buildRequest(argv):
"""Build the REST request URL from argv,
all key, value pairs in argv are quoted."""
url = "https://" + __supportedLocales[getLocale()] + "/onca/xml?"
argv['Service'] = 'AWSECommerceService' #add Service to url param to argv so it canbe sorted
argv['Timestamp'] = strftime("%Y-%m-%dT%H:%M:%SZ", gmtime()) # add GMT Timestamp url param to argv, this is required when using a signature
sortedArgv = argv.items()
sortedArgv.sort() #args must be sorted so both you and amazon generate the same hashed signature
args = '&'.join(['%s=%s' % (k,urllib.quote(str(v))) for (k,v) in sortedArgv if v])
paramsToEncode = '&'.join(['%s=%s' % (k,urllib.quote(str(v))) for (k,v) in sortedArgv if v])
stringToSign = "GET"+"\n"+__supportedLocales[getLocale()]+"\n"+"/onca/xml"+"\n"+paramsToEncode.encode('utf-8')
signature = urllib.quote(base64.b64encode(hmac.new(getSecretKey(), stringToSign, hashlib.sha256).digest()))
return url + '&'.join(['%s=%s' % (k,urllib.quote(str(v))) for (k,v) in sortedArgv if v])+"&Signature="+signature
A simple use of the updated PyAWS library follows:
import ecs
ecs.setLicenseKey('yourlicencekey')
ecs.setSecretKey('yoursecretkey')
ecs.setLocale('us')
ecssearch = ecs.ItemSearch('', SearchIndex='Books',Availability='Available', Condition='All',MerchantId='All',BrowseNode='3839',ResponseGroup='Small',AssociateTag='boon-21')
for i in range (1,20):
print ecssearch[i].Title
Comments
Nick Fishman commented, on July 29, 2009 at 7:52 a.m.:
Thanks for the investigation and the code! I actually had to change the following line:
argv['Timestamp'] = strftime("%Y-%m-%dT%H:%M:%SZ") # add Timestamp url param to argv, this is required when using a signature
to
argv['Timestamp'] = strftime("%Y-%m-%dT%H:%M:%SZ", gmtime()) # add Timestamp url param to argv, this is required when using a signature
because Amazon was complaining about the request expiring. Presumably, this is because my computer is set in Eastern time, and strftime() reports localized time rather than the GMT that Amazon expects.
I also added
from time import gmtime
Adam commented, on July 30, 2009 at 10:24 a.m.:
Hi Nick, thanks for spotting that, I am on GMT (actually British summer time which is GMT+1) and Amazon was giving me no errors. I have updated the script and the blog post.
Liz commented, on July 30, 2009 at 3:43 p.m.:
Hello!
Thanks a lot - it was extremely helpful :)!
1) I'd like to add some minor change - the parameters must be encoded accordingly to RFC 3986 (http://tools.ietf.org/html/rfc3986)
as stated in here: http://docs.amazonwebservices.com/AWS...
So, i changed urllib.quote() to this function:
----------------------------------------------------------------------------------
rfc_safe = "-._~"
# encodes string accordingly to RFC 3986 (do not encode unreserved characters)
def strToRFC(s):
return urllib.quote(s, safe = rfc_safe)
----------------------------------------------------------------------------------
2) Amazon testing tool:
(helps you build url and signature according to your credentials):
http://associates-amazon.s3.amazonaws...
The tool is not encoding properly, though.. for example
string ZEN="Q:/?#[]@!$'()*+,;Q"
amazon testing tool => ZEN=Q%3A%2F%3F%23%5B%5D%40!%24'()*%20%2C%3BQ
but should be => ZEN=Q%3A%2F%3F%23%5B%5D%40%21%24%27%28%29%2A%2B%2C%3BQ
(characters "!'()*+" weren't encoded in Amazon tool)
Hope this helps :)
Rob and Steve commented, on July 31, 2009 at 5:04 p.m.:
Just to add, the occurrences of
for (k,v) in sortedArgv if v
should really be
for (k,v) in sortedArgv if v is not None
because *without* this tweak, values of 0 are omitted from the request, which means that removing an item from a cart by setting its quantity to 0 would fail.
Hope this helps!
JKnecht commented, on August 2, 2009 at 6:53 a.m.:
Thank you very much for sharing your code. It was very helpful and saved me a lot of time.
Robin commented, on August 22, 2009 at 5:08 p.m.:
Adam, many thanks for your code. My access to AWS is functional once again. Brilliant!!
Robin
David commented, on August 25, 2009 at 4:29 p.m.:
Thank you for this code! I am having one small problem though, and I believe it has to do with hashlib as i'm still using Python2.4
Error Type: AttributeError
Error Value: 'builtin_function_or_method' object has no attribute 'new'
Any help would be very much appreciated!
Jim commented, on October 1, 2009 at 6:14 p.m.:
I was dusting off some ecs code and needed this fix. Thanks!
http://www.pressthered.com
dandu commented, on December 6, 2009 at 10:11 p.m.:
maybe you could update your patch to the latest pyaws version (0.3.0), that'd be helpful :)
Adam commented, on December 22, 2009 at 10:34 a.m.:
Wow, I didn't even know pyAWS 0.3.0 existed, although looking through it I don't it would be that hard to incorporate the authentication. I will post up the newer version when I get some free time.
Jacobo de Vera commented, on February 26, 2010 at 5:46 p.m.:
You can get version 0.3.0 from here:
http://trac2.assembla.com/pyaws
And then replace the ecs.py with this one:
http://gist.github.com/315868
So that signed requests work with 0.3.0
I have also sent the patch to the module author.
Adam commented, on March 9, 2010 at 12:45 p.m.:
Cool, nice work Jacobo!
Post a comment