The Other 10 Percent

A new release of boto, the Python library for Amazon Web Services is now available at:

• http://code.google.com/p/boto/

Here are some highlights of this release:

• Boto now supports Amazon’s Mechanical Turk service. Many thanks to Ansel Halliburton for contributing this code.
• Boto now supports the 2007-01-19 version of the EC2 API
• SQS Queues now have methods to save and restore messages to/from S3
• The S3 interface now allows you to iterate over all keys in a bucket, transparently handling the paged results returned by S3
• The HTTPS connection has been parameterized within Boto to permit the use of the M2Crypto? library which actually verifies the certificate of the server. Many thanks to Jon Colverson for contributing this code as well as the contrib/m2helpers.py module.
• The new SQS methods GetQueueAttributes and SetQueueAttributes are now supported. This allows you to query a queue for the approximate number of messages in the queue.
• The new SQS method ChangeMessageVisibility is now supported. This feature is most easily accessed via the Message.change_visibility method.
• The S3 library now fully supports the delimiter option.
• The S3 library now allows a callback to be passed to the methods that get/put content to S3. This enables feedback during large file transfers. See this for more info.
• Improved error reporting by including the XML response body as part of the exceptions
• Cleaned up and improved the sending/receiving of files to/from S3, including addition of Expect header in PUT’s to address issues with files > 2.1GB in size. See this for more info.
• Many smaller interface changes suggested by the boto community (see this thread ) for details
• Improved docstrings for classes and methods. Still not perfect but better!

Mitch

I recently wrote an article for AWS that describes an architecture and sample implementation of scalable web services using S3, SQS, and EC2. You can find it here
The code in the article is based on my boto library, of course.

Mitch

A new version of boto, the Python library for Amazon Web Services, is available at:

http://code.google.com/p/boto/

Boto provides a Python interface to the following Amazon Web Services; Simple Storage Service (S3), Simple Queue Service (SQS), and Elastic Compute Cloud (EC2). This release provides a number of substantial improvements over the previous release including:

* Support for enabling/disabling logging on S3 buckets, including setting the ACL on target buckets.
* Support for generating query-string URL’s for S3 resources.

The release files, tutorials, and subversion repository are accessible from the project home page link above. Issues may also be entered from the project home page.

Mitch

A new version of boto, the Python library for Amazon Web Services, is available at:

http://code.google.com/p/boto/

Boto provides a Python interface to the following Amazon Web Services; Simple Storage Service (S3), Simple Queue Service (SQS), and Elastic Compute Cloud (EC2). This release provides a number of substantial improvements over the previous release including:

* Support for EC2 via the new Query API
* Automatic and transparent base64 encoding of SQS messages
* SQS and EC2 exceptions now print out the XML response body
* A simple command line utility for manipulating queues

In addition the package structure has changed considerably to allow the library to support multiple Amazon services more elegantly. This change in package structure may cause some existing code’s import statements to be modified slightly.

The release files, tutorials, and subversion repository are accessible from the project home page link above. Issues may also be entered from the project home page.

Mitch

A new release of BitBucket is available. This release adds one small new feature (an optional fetch_keys parameter on the get_bucket method) and also fixes a couple of small bugs. In addition, this release is intended to address an apparent inconsistency with version 0.3e of the code. You can access the new version at:

• http://code.google.com/p/bitbucket-s3/
  

Mitch

This is a maintenance release. It fixes no actual code bugs but does fix some whitespace formatting errors that crept into the code when I lost my .emacs file while switching computers. You can find the new release on the download page of the BitBucket Trac system located here:

• http://garnaat.org:9029/bitbucket/wiki/BitBucketDownload

ACM Queue has a great interview with Vogels. There’s quite a bit of information about Amazon’s move to services and the impact that has had on scalability, operations, etc. Also, particularly interesting to me is the discussion about developers at Amazon. It’s clear that Amazon places a lot of importance on having really good people, providing them with good tools and then letting them be creative. The best part was the description of how developer’s are given end to end responsibilities for the services they create including day to day operations. Very cool.

• Amazon CTO Interview

From TechCrunch comes an interesting article about WiFi being build into Flash memory cards. That would be very cool because you could essentially “upgrade” any existing digital camera to allow WiFi upload of pictures. The article only mentions SD cards but I hope CompactFlash is possible as well.

Eye-Fi to Combine Wifi, Flash Memory: ”

The one thing I wish my digital camera had was wifi so that I could auto-upload photos without the extra hassle of connecting the camera or memory card to a computer. There are cameras being released now that have this capability. However, the vast majority of cameras sold, plus the 100 million digital cameras already out there, don’t have any wireless capabilities.

Eye-Fi is tackling the problem from a different perspective. They’re building wifi directly into the flash storage. Their first product will be a 1GB SD card with built in Wifi. For about the same price as a 1 GB flash card sells for today - $100. You’ll be able to upload photos, or whatever, directly from your device to a computer using the built in storage wifi capabilities.

There’s lots more buzz on this. See Robert Scoble and Scott Beale, who are just as excited as I am about Eye-Fi. No guidance on when this will be available, but they have working prototypes. If Eye-Fi owns the intellectual property around this, look for them to license the technology to flash memory producers. I will buy this the second it becomes available.

Tags: flash, wifi, digitalcameras, techcrunch, web2.0, web_2.0

”

(Via TechCrunch.)

I love these stories about how real-world, high-traffic web sites are actually configured. This one comes from the O’Reilly Radar blog:
Database War Stories #5: craigslist: “By tim

Eric Scheide of craigslist offered me a stream of consciousness summary of the craigslist database setup. At a conference last year, Craig showed a slide (which helped inspire my postings about asymmetric competition [1, 2, 3]) that listed the number of employees at the top ten web sites. Most of them have thousands of employees. Some have tens of thousands. Craigslist, at #7 on the list, has 19.

Eric’s email has that embattled ‘news from the front’ feel that you might expect from a site handling that much traffic with only 19 employees!

“

(Via O’Reilly Radar.)

PC Magazine has a fairly detailed review of the new SketchUp beta, the latest in a steady stream of interesting offerings from the giant sucking sound in Mountain View known as Google. According to their latest quarterly report they hired over 1100 people this quarter. Apparently there have been reports of pony-tailed developers being sucked right out of their Birkenstocks if they even walk too close to the Google campus. Some sort of tractor beam or something. But, I digress…

• Review of SketchUp