BotoServerError - 400 Bad Request - Rate Exceeded
This is for anyone out there using python-boto with Amazon CloudSearch, who is getting a throttle limit and googling around for an answer. You’re probably getting an error like this:
<ErrorResponse xmlns=“http://cloudsearch.amazonaws.com/doc/2011-02-01/”>
<Error>
<Type>Sender</Type>
<Code>Throttling</Code>
<Message>Rate exceeded</Message>
</Error>
<RequestId>c753f49e-26b9-11e3-a879-0d3b8f7ade79</RequestId>
</ErrorResponse>
Here’s what’s happening: Although there aren’t any limits on the volume of searches you can make with CloudSearch, there are throttling limits on the preliminary calls to make to get the document & search service.
This can be solved by caching the search service object. Here’s an example with Django’s caching:
import boto
django_cache
cs_conn = boto.connect_cloudsearch(
aws_access_key_id=YOUR_KEY_ID,
aws_secret_access_key=YOUR_ACCESS_KEY,
)
search_kwargs = {}
search_kwargs['q'] = "Stuff to find"
try:
search_service = django_cache.get("cloudsearch_search_service")
# Need to try to execute something for it to fail
results = search_service.search(**search_kwargs)
except:
domain = cs_conn.lookup("domain-name")
document_service = domain.get_document_service()
search_service = document_service.get_search_service()
django_cache.set("cloudsearch_search_service", search_service, 300)
results = search_service.search(**search_kwargs)
print results.doc
Written on November 22, 2013