Initial implementation of the Galaxy Web API. Disabled unless

'enable_api = True' in config file.  You should not enable the API on
production sites as this code is brand new and may contain serious bugs and
security flaws!  Implemented:

* Display libraries
* Display library info
* Display library contents
* Display library content info
* Create library folders
* Upload datasets to a library from a server directory or with a path paste.
* Basic example scripts in scripts/api/

Framework changes that were made to support this:

* API Key interface in User Preferences.
* New api_keys database table for storing users' API Keys.
* New API-specific route mapper in webapp.
* API controllers in galaxy.web.api
* Return handling in reused library_common methods.
* expose_api decorator for API controller methods validates key and ensures
  valid JSON format.
* UniverseWebTransaction renamed to GalaxyWebTransaction and subclassed for
  GalaxyWebUITransaction and GalaxyWebAPITransaction.

Things that need to be done next:

* Documentation!
* Refactor reused code from library_common and other controllers into an
  even-more-generic location and format.  The main changes are that the Web UI
  returns redirects and rendered templates, whereas the API returns various
  HTTP status codes and JSON.
* Implement more functionality.
* The request and response format should be considered alpha and are subject to
  change.  They will be standardized as the API matures.

Hints to get started can be found in scripts/api/README

scripts/api/README

@@ -0,0 +1,106 @@
+This is not documentation.  These are hints and examples to get you started
+until the documentation is written.
+
+Set these options in universe_wsgi.ini and start the server:
+
+enable_api = True
+admin_users = you@example.org
+library_import_dir = /path/to/some/directory
+
+In the directory you specified for 'library_import_dir', create some
+subdirectories, and put (or symlink) files to import into Galaxy into those
+subdirectories.
+
+In Galaxy, create an account that matches the address you put in 'admin_users',
+then browse to that user's preferences and generate a new API Key.  Copy the
+key to your clipboard.  Create a new library (doing this via the API is not yet
+implemented).  Then take your API Key and use the scripts in scripts/api/ to do
+things:
+
+% ./display.py my_key http://localhost:4096/api/libraries
+Collection Members
+------------------
+/api/libraries/f3f73e481f432006
+  name: api_test
+  id: f3f73e481f432006
+
+% ./display.py my_key http://localhost:4096/api/libraries/f3f73e481f432006
+Member Information
+------------------
+synopsys: None
+contents_url: /api/libraries/f3f73e481f432006/contents
+description: API Test Library
+name: api_test
+
+% ./display.py my_key http://localhost:4096/api/libraries/f3f73e481f432006/contents 
+Collection Members
+------------------
+/api/libraries/f3f73e481f432006/contents/28202595c0d2591f61ddda595d2c3670
+  name: /
+  type: folder
+  id: 28202595c0d2591f61ddda595d2c3670
+
+% ./library_create_folder.py my_key http://localhost:4096/api/libraries/f3f73e481f432006/contents 28202595c0d2591f61ddda595d2c3670 api_test_folder1 'API Test Folder 1'
+Response
+--------
+/api/libraries/f3f73e481f432006/contents/28202595c0d2591fa4f9089d2303fd89
+  name: api_test_folder1
+  id: 28202595c0d2591fa4f9089d2303fd89
+
+% ./library_upload_from_import_dir.py my_key http://localhost:4096/api/libraries/f3f73e481f432006/contents 28202595c0d2591fa4f9089d2303fd89 bed bed hg19
+Response
+--------
+/api/libraries/f3f73e481f432006/contents/e9ef7fdb2db87d7b
+  name: 2.bed
+  id: e9ef7fdb2db87d7b
+/api/libraries/f3f73e481f432006/contents/3b7f6a31f80a5018
+  name: 3.bed
+  id: 3b7f6a31f80a5018
+
+% ./display.py my_key http://localhost:4096/api/libraries/f3f73e481f432006/contents 
+Collection Members
+------------------
+/api/libraries/f3f73e481f432006/contents/28202595c0d2591f61ddda595d2c3670
+  name: / 
+  type: folder
+  id: 28202595c0d2591f61ddda595d2c3670
+/api/libraries/f3f73e481f432006/contents/28202595c0d2591fa4f9089d2303fd89
+  name: /api_test_folder1
+  type: folder
+  id: 28202595c0d2591fa4f9089d2303fd89
+/api/libraries/f3f73e481f432006/contents/e9ef7fdb2db87d7b
+  name: /api_test_folder1/2.bed
+  type: file
+  id: e9ef7fdb2db87d7b
+/api/libraries/f3f73e481f432006/contents/3b7f6a31f80a5018
+  name: /api_test_folder1/3.bed
+  type: file
+  id: 3b7f6a31f80a5018
+
+% ./display.py my_key http://localhost:4096/api/libraries/f3f73e481f432006/contents/e9ef7fdb2db87d7b
+Member Information
+------------------
+misc_blurb: 68 regions
+metadata_endCol: 3
+data_type: bed
+metadata_columns: 6
+metadata_nameCol: 4
+uploaded_by: nate@...
+metadata_strandCol: 6
+name: 2.bed
+genome_build: hg19
+metadata_comment_lines: None
+metadata_startCol: 2
+metadata_chromCol: 1
+file_size: 4272
+metadata_data_lines: 68
+message:
+metadata_dbkey: hg19
+misc_info: uploaded bed file
+date_uploaded: 2010-06-22T17:01:51.266119
+metadata_column_types: str, int, int, str, int, str
+
+Other parameters are valid when uploading, they are the same parameters as are
+used in the web form, like 'link_data_only' and etc.
+
+The request and response format should be considered alpha and are subject to change.

scripts/api/common.py

@@ -0,0 +1,94 @@
+import os, sys, urllib, urllib2
+
+new_path = [ os.path.join( os.path.dirname( __file__ ), '..', '..', 'lib' ) ]
+new_path.extend( sys.path[1:] )
+sys.path = new_path
+
+from galaxy import eggs
+import pkg_resources
+
+pkg_resources.require( "simplejson" )
+import simplejson
+
+def make_url( api_key, url, args=None ):
+    # Adds the API Key to the URL if it's not already there.
+    if args is None:
+        args = []
+    argsep = '&'
+    if '?' not in url:
+        argsep = '?'
+    if '?key=' not in url and '&key=' not in url:
+        args.insert( 0, ( 'key', api_key ) )
+    return url + argsep + '&'.join( [ '='.join( t ) for t in args ] )
+
+def get( api_key, url ):
+    # Do the actual GET.
+    url = make_url( api_key, url )
+    return simplejson.loads( urllib2.urlopen( url ).read() )
+
+def post( api_key, url, data ):
+    # Do the actual POST.
+    url = make_url( api_key, url )
+    req = urllib2.Request( url, headers = { 'Content-Type': 'application/json' }, data = simplejson.dumps( data ) )
+    return simplejson.loads( urllib2.urlopen( req ).read() )
+
+def display( api_key, url ):
+    # Sends an API GET request and acts as a generic formatter for the JSON response.
+    try:
+        r = get( api_key, url )
+    except urllib2.HTTPError, e:
+        print e
+        print e.read( 1024 ) # Only return the first 1K of errors.
+        sys.exit( 1 )
+    if type( r ) == unicode:
+        print 'error: %s' % r
+        return None
+    elif type( r ) == list:
+        # Response is a collection as defined in the REST style.
+        print 'Collection Members'
+        print '------------------'
+        for i in r:
+            # All collection members should have a name and url in the response.
+            print i.pop( 'url' )
+            print '  name: %s' % i.pop( 'name' )
+            for k, v in i.items():
+                print '  %s: %s' % ( k, v )
+        print ''
+        print '%d elements in collection' % len( r )
+    elif type( r ) == dict:
+        # Response is an element as defined in the REST style.
+        print 'Member Information'
+        print '------------------'
+        for k, v in r.items():
+            print '%s: %s' % ( k, v )
+    else:
+        print 'response is unknown type: %s' % type( r )
+
+def submit( api_key, url, data ):
+    # Sends an API POST request and acts as a generic formatter for the JSON response.
+    # 'data' will become the JSON payload read by Galaxy.
+    try:
+        r = post( api_key, url, data )
+    except urllib2.HTTPError, e:
+        print e
+        print e.read( 1024 )
+        sys.exit( 1 )
+    print 'Response'
+    print '--------'
+    if type( r ) == list:
+        # Currently the only implemented responses are lists of dicts, because
+        # submission creates some number of collection elements.
+        for i in r:
+            if type( i ) == dict:
+                if 'url' in i:
+                    print i.pop( 'url' )
+                else:
+                    print '----'
+                if 'name' in i:
+                    print '  name: %s' % i.pop( 'name' )
+                for k, v in i.items():
+                    print '  %s: %s' % ( k, v )
+            else:
+                print i
+    else:
+        print r

scripts/api/display.py

@@ -0,0 +1,11 @@
+#!/usr/bin/python
+
+import os, sys
+sys.path.insert( 0, os.path.dirname( __file__ ) )
+from common import display
+
+try:
+    display( *sys.argv[1:3] )
+except TypeError:
+    print 'usage: %s key url' % os.path.basename( sys.argv[0] )
+    sys.exit( 1 )

scripts/api/library_create_folder.py

@@ -0,0 +1,20 @@
+#!/usr/bin/python
+
+import os, sys
+sys.path.insert( 0, os.path.dirname( __file__ ) )
+from common import submit
+
+try:
+    data = {}
+    data[ 'folder_id' ] = sys.argv[3]
+    data[ 'name' ] = sys.argv[4]
+    data[ 'create_type' ] = 'folder'
+except IndexError:
+    print 'usage: %s key url folder_id name [description]' % os.path.basename( sys.argv[0] )
+    sys.exit( 1 )
+try:
+    data[ 'description' ] = sys.argv[5]
+except IndexError:
+    data[ 'description' ] = ''
+
+submit( sys.argv[1], sys.argv[2], data )

scripts/api/library_upload_from_import_dir.py

@@ -0,0 +1,19 @@
+#!/usr/bin/python
+
+import os, sys
+sys.path.insert( 0, os.path.dirname( __file__ ) )
+from common import submit
+
+try:
+    data = {}
+    data[ 'folder_id' ] = sys.argv[3]
+    data[ 'file_type' ] = sys.argv[4]
+    data[ 'server_dir' ] = sys.argv[5]
+    data[ 'dbkey' ] = sys.argv[6]
+    data[ 'upload_option' ] = 'upload_directory'
+    data[ 'create_type' ] = 'file'
+except IndexError:
+    print 'usage: %s key url folder_id file_type server_dir dbkey' % os.path.basename( sys.argv[0] )
+    sys.exit( 1 )
+
+submit( sys.argv[1], sys.argv[2], data )