How does the browser's native login prompt work and how to use it programmatically

This is another one of those rather interesting problems that I had to solve at my day job and since I knew little about how to solve it when I first came across it, I am going to blog about it. I am sure there are others like me, who may not know how to solve it when tasked with such a problem.

Problem

So I am working on a Java based project at work(lets call it Project X)  for which my latest task was to change the way it handles login.

Problem details

I think it's best I provide some context before I describe my problem, so here we go,

You see when you try to access a certain restricted area of a web-app when not logged-in the browser will show you a login prompt, and only when you enter the right credentials, will you be able to access that area of a web-app. That entire process of seeing the login prompt, entering the credentials etc requires human intervention and my task was to achieve all that programmatically i.e. without any human intervention for Project X.

Great!! the problem is now clear, so let's look at the problem in the context of Project X in a bit more detail. In Project X, we have a login-page that  calls a certain Servlet and passes the username and password, now my task was to invoke that RESTful endpoint while passing the username and password and the RESTful endpoint will respond with a cookie (assuming the credentials are correct) that has a session token that can be stored and used for subsequent requests.

How does the browser's native login prompt work?

This blog post by Steven Sanderson explains how it all works and it does so rather well. In the interest of completeness of this post, I will summarise it here

1. When you try to access a certain restricted resource by typing the url in the browser
2. The browser gets a HTTP 401 unauthorized response
3. After which the browser shows you a login prompt
4. Then you enter the username and password and click log-in
5. The browser re-submits the request but this time with an extra header                                           i.e. Authorization: Basic username password
6. The username and password are base64 encoded so the header would actually look something like this                                                                                                                   Authorization:Basic ILQtaW46YWRtaW4=
7. If you were successfully authenticated, you will access the resource you were trying to access

So what was my goal?

Add functionality similar to how the browser's native login handling to the backend java code for Project X, get the response and extract and store a cookie sent as a part of the response. 

Solution

Turns out, once I fully understood the problem, it was actually quite easy to accomplish that with the Apache HttpClient library. In fact, the when you download and unzip the libs from here, in the contents of the httpcomponents-client-4.4.1(at the time of writing) folder, you will find some sample code in the examples folder within it. Once again in the interest of completeness of this post, I will add some sample code in this post.

public class ClientAuthentication {
    public static void main(String[] args) throws Exception {
        String host = "";
        int port = 0000;
        String username = "username";
        String password = "password";
        CredentialsProvider credsProvider = new BasicCredentialsProvider();
    credsProvider.setCredentials(
            new AuthScope(host, port),
            new UsernamePasswordCredentials(username, password));
    final BasicCookieStore cookieStore = new BasicCookieStore();
    CloseableHttpClient httpclient = HttpClients.custom()
            .setDefaultCredentialsProvider(credsProvider)
            .setDefaultCookieStore(cookieStore)
            .build();
    try {
        String urlToQuery =""; 
        HttpGet httpget = new HttpGet(urlToQuery);
        ResponseHandler<String> responseHandler = new ResponseHandler<String>() {
            @Override
            public String handleResponse(
                    final HttpResponse response) throws ClientProtocolException, IOException {
                List<Cookie> cookies = cookieStore.getCookies();
                if (cookies.isEmpty()) {
                    System.out.println("None");
                } else {
                    for (int i = 0; i < cookies.size(); i++) {
                        System.out.println("- " + cookies.get(i).toString());
                    }
                }
                return "";
            }

        };
        httpclient.execute(httpget, responseHandler);
    } finally {
        httpclient.close();
    }
    }
}

Now the above example works great with the 4.4.1  version of the libs, but you see Project X uses 4.1.2 version of the libs and since this was something very basic, I had to figure out how to make it work with the older libs. What you see below is code that does something similar but with the older version of the libs.

public class AuthWithFourPointOne {

    public static void main(String[] args) throws Exception {
        String host = "";
        int port = 0000;
        String username = "username";
        String password = "password";
        CredentialsProvider credsProvider = new BasicCredentialsProvider();
        credsProvider.setCredentials(new AuthScope(host, port), new UsernamePasswordCredentials(username, password));
        DefaultHttpClient httpclient = new DefaultHttpClient();
        httpclient.setCredentialsProvider(credsProvider);
        try {
            String url = "";
            HttpGet httpGet = new HttpGet(url);
            HttpResponse response = httpclient.execute(httpGet);
            String cookie = "";
            //in my case the cookie in the header was called Set-Cookie
            for(Header header:response.getAllHeaders()){
                if(header.getName().contains("Set-Cookie")) {
                    cookie = header.getValue();
                }
            }
        } finally {
            httpclient.getConnectionManager().shutdown();
        }
    }
}