* Update: Modified and slightly revised version of this post is available on the LabKey Blog in two parts. See Part1 and Part2
As you may or may not know, the LabKey Server has a whole suite of client APIs that can be used to programmatically access data inside your LabKey Server. Like any other open source project, some of these APIs have seen a lot more development than others (see the functionality available in the Javascript API, which is one of the first we developed; compared to our newest API, Python which was written last month by Elizabeth in her spare time).
Periodically, I will need to access data in a LabKey Server in an unsupported language(for example NODE.js or BASH). In these cases, one of the first hurdles that you need to get over is simply authenticating to the server. In this entry, I am going to provide some sample code for how to authenticate to the LabKey Server in a variety of languages (shell script, python and node.js).
How Does Authentication in the LabKey Server work?
LabKey Server uses form-based authentication, by default, for all user-agents (ie browsers). However, it will correctly accept HTTP basic authentication headers if presented. I recommend using Basic Auth when programmatically accessing the server.
Please note that when using either form-based authentication or basic authentication, your credentials are passed in clear text, thus I strongly recommend implementing SSL on your LabKey Server.
Here is a very simplistic example of how Basic Auth works:
- You attempt to connect to a web page which requires authentication
- The server will respond with an
Authorization Required error message (HTTP response code = 401).
- In addition, the server will add a new HTTP header to the response. It will be similar to
WWW-Authenticate: Basic realm="Default"
- If you are using a browser, the browser will then pop-up a dialog box asking you to enter a username and password to gain access.
This is how the LabKey Server works for all the APIs, which are designed to be accessed programmatically.
For all other pages(where the LabKey Server assumes you will accessing via a browser) the LabKey Server will not behave in the way described above but will behave completely different. Here is an example:
- You attempt to connect to a web page which requires authentication
- The server will respond with an
Moved Temporarily error message (HTTP response code = 302).
- The server does not include the
WWW-Authenticate:... header in the response headers
- If you are using a browser, the browser be redirected to the LabKey Server login page.
- If you are not using a browser, but accessing the server programmatically you simply get this error message
In general, the actions which respond with the WWW-Authenticate:... HTTP Header are
- All API requests (any page whose names ends with .api)
All other pages do not respond with the WWW-Authenticate:... HTTP Header, but redirect you to the LabKey Server login page. This includes actions such as
- Most Admin actions (creating new folders, viewing the Audit Log)
- Submitting MS2 searches to the server
- Viewing and managing Wikis
- Viewing or posting messages to a Message board or Issue tracker.
This is an important point. When accessing the LabKey Server programmatically tools (wget, curl, etc) or programming libraries which handle Basic Auth authentication will work properly when accessing an API, but will not work when accessing any other page on the server.
A very easy way to determine the authentication behavior of page is to make an unauthenticated connection to the web page and review the response HTTP Headers. You can do this using python
import urllib2
try:
f = urllib2.urlopen('http://my.labkey.server/labkey/your/page')
print f.info()
except urllib2.HTTPError, e:
print e.info()\
or using wget
wget --server-response http://my.labkey.server/labkey/your/page
Let me use an example to illustrate the behavior I explained above. Lets use wget to view a secured wiki page on the server
wget --server-response --max-redirects=0 https://www.labkey.org/wiki/Home/page.view?name=secureWiki
the response is
HTTP request sent, awaiting response...
HTTP/1.1 302 Moved Temporarily
Server: Apache-Coyote/1.1
Expires: Thu, 01 Jan 1970 00:00:00 GMT
Cache-Control: no-cache
Location: /login/Internal/login.view?returnUrl=%wiki%2FHome%page.view%3Fname=secureWiki
Content-Length: 156
Date: Wed, 24 Aug 2011 01:01:19 GMT
Connection: keep-alive
Location: /login/Internal/login.view?returnUrl=%wiki%2FHome%page.view%3Fname=secureWiki [following]
You can see that server:
- Does not respond with a
WWW-Authenticate:... HTTP Header, but
- Redirects the browser to the login page
Now lets look at an attempt to use the getQuery API (which you can use to query data in the LabKey Server).
wget --server-response --max-redirects=0 https://www.labkey.org/query/Internal/getQuery.api
the response is
HTTP request sent, awaiting response...
HTTP/1.1 401 Unauthorized
Server: Apache-Coyote/1.1
WWW-Authenticate: Basic realm="LabKey Server and CPAS Distribution and Support"
Content-Type: text/html
Content-Length: 2871
Vary: Accept-Encoding
Date: Wed, 24 Aug 2011 01:05:35 GMT
Connection: keep-alive
In this example, you can see that the server
- Responded with a HTTP response code of 401
- Added the
WWW-Authenticate: Basic realm="LabKey Server and CPAS Distribution and Support" header which you can then programmatically use to authenticate using Basic Auth.
Now that I have shown you how the LabKey Server will respond to authentication requests, I can provide some examples of how to authenticate to the LabKey Server using various programming languages. Before we get to the examples, here are few links to documentation for programmatically accessing a LabKey Server and using the LabKey APIs
In the examples below, you will see two URLs used; one is meant to represent viewing a secured wiki page and the other is for reading the contents of a list. The URLs are:
Authenticating to LabKey Server using Python
LabKey currently ships a client PYTHON client API. The PYTHON client api currently supports (as of 01/01/2012):
- reading data inside LabKey Server (selectRows)
- inserting new data inside LabKey Server (insertRows)
- deleting existing data inside LabKey Server (deleteRows)
- updating data inside LabKey Server (updateRows)
- executing SQL statements (executeSql)
- updating a wiki page
- posting a message to a message board
The client API will handle the authentication for you. If there are other tasks you need to perform, you can use the example code below to authenticate to the server
Authenticate to an API page
In this example, I will use the urllib2 library. I will load my creadentials into the a password manager and then let the OpenerDirector object handle passing the Basic Auth headers to the server when requested.
import urllib2
myemail = 'email'
mypassword = 'password'
# Create a password manager
passmanager = urllib2.HTTPPasswordMgrWithDefaultRealm()
# Add login info to the password manager
passmanager.add_password(None, mymachine, myemail, mypassword)
# Create the AuthHandler
authhandler = urllib2.HTTPBasicAuthHandler(passmanager)
# Create opener
opener = urllib2.build_opener(authhandler)
# Build the URL for querying LabKey Server (Replace SCHEMA, QUERY with the schema name and query name of your choice)
myurl = ' https://your.labkey.server/labkey/query/Home/getQuery.api?schemaName=lists&query.queryName=listName'
# Get authenticated to send URL requests
opener =_create_opener()
# Use the opener to fetch a URL request
myrequest = urllib2.Request(myurl)
try:
response = opener.open(myrequest)
print response.read()
except urllib2.HTTPError, e:
print e.code
print e.read()
Authenticate to an non API page
In this example, I will load my creadentials into the a password manager and in addition add the Basic Auth Header to all requests using the OpenerDirector object.
import urllib2
myemail = 'email'
mypassword = 'password'
# Create a password manager
passmanager = urllib2.HTTPPasswordMgrWithDefaultRealm()
# Add login info to the password manager
passmanager.add_password(None, mymachine, myemail, mypassword)
# Create the AuthHandler
authhandler = urllib2.HTTPBasicAuthHandler(passmanager)
# Create the Basic Authentication Header
authHeader = base64.encodestring("%s:%s" % (myemail, mypassword))[:-1]
authHeader = "Basic %s" % authHeader
# Create opener
opener = urllib2.build_opener(authhandler)
# Build the URL for viewing a secured wiki on LabKey Server
myurl = 'https://your.labkey.server/labkey/wiki/Home/page.view?name=securedWiki'
# Get authenticated to send URL requests
opener =_create_opener()
# Use the opener to fetch a URL request
myrequest = urllib2.Request(myurl)
try:
response = opener.open(myrequest)
print response.read()
except urllib2.HTTPError, e:
print e.code
print e.read()
NOTE: If you want to use only one opener for both API and non-API pages, you simply can use the 2nd example for accessing all pages.
Authenicating to LabKey Server with Node.js
There is currently not a client API for the Node.js language yet. (If you are using Node.js to work with data in your LabKey Server and want to assist in writing an client API, please post a message to the Developer Community Forum as we would love to work with you.)
As we saw above, with Python, there are two ways to authenticate. For API pages, the OpenerDirector object handles chaining of requests together and will properly resubmit a request if it receives a 401 HTTP response code with theWWW Authenicate: header. There is not something similar for NODE.js and thus there is only one way to authenticate to the server.
NOTE: Full disclosure, I am a Node.js novice, there are probably many better ways to do this. If you know a much better, simpler way, please post it in the comments and I will update this example.
https = require("https");
var email = 'myemail';
var password = 'mypassword';
var auth = 'Basic ' + new Buffer(email + ':' + password).toString('base64');
var authHeaders = {
Authorization: auth
};
var addUserURL = {
host: 'your.labkey.server',
port: 443,
path: ' /labkey/wiki/Home/page.view?name=securedWiki',
method: 'GET',
headers: authHeaders
};
var request = https.request(addUserURL, function(response) {
response.setEncoding('utf8');
responseData = '';
response.on('data', function (chunk) {
responseData += chunk;
});
response.on('error', function(e){
console.log("An error occurred during the request.")
console.log('HTTP Response Code: ' + response.statusCode);
console.log('HTTP Response Headers: ' +r esponse.headers);
return;
});
response.on('end', function(){
console.log("Response Output: " + responseData)
console.log('HTTP Response Code: ' + response.statusCode);
console.log('HTTP Response Headers: ' + response.headers);
});
});
request.end();
Authenticating to the LabKey Server using wget
Now, if you do not want to bother with these programming languages and just want to grab a page quickly or download some data and parse it with sed, you can just use wget. (To be completely honest, I use wget all the time for testing and/or grabbing a single page).
Authenticating to an API page.
This case is pretty simple one liner
wget --server-response --http-user=myEmail --http-password=myPassword \
"https://your.labkey.server/labkey/query/Home/getQuery.api?schemaName=lists&query.queryName=listName"
Authenticating to a non-API page.
For this case, you will need to make two wget requests. The first to login and the 2nd to access the secured page.
wget --server-response --save-cookies cookies.cpas --keep-session-cookies \
https://your.labkey.server/Login/login.post --post-data "email=myEmail&password=myPassword"
wget --load-cookies cookies.cpas https://your.labkey.server/labkey/wiki/Home/page.view?name=securedWiki
Now that you have gotten past the authentication, you can start doing all sorts of fun things with your data.
There are a few other languages that I have used in the past, namely perl, that I should have included in these examples. I will try and get to those in a later post(s). (If you are using perl, see the Perl Client API for an example)
I hope this is helpful and understandable. If you have any questions, please post a comment or post a message to the Developer Community Forum
