How do I login to a system through an HTTP request?

Question

How do I login to a system through an HTTP request?

Navigation

#1 by (3 votes)
#2 by (2 votes)

3

I am performing an HTTPS get request for the next address , my initial intention is to receive the html data from the page. I followed the Mkyong tutorial, but I get the answer 302, I do not know what I'm doing wrong, I've added the necessary headers, but I'm still not getting a positive result. Here is the code:

public class HttpUrlConnectionUFAC {

    private List<String> cookies;
    private HttpsURLConnection conn;

    private final String USER_AGENT = "Mozilla/5.0";

    public static void main(String[] args) {
        String url = "https://portal.ufac.br/aluno/login/";

        HttpUrlConnectionUFAC http = new HttpUrlConnectionUFAC();

        // make sure cookies is turn on
        CookieHandler.setDefault(new CookieManager());

        // 1. Send a "GET" request, so that you can extract the form's data.
        String page = null;
        try {
            page = http.getPageContent(url);
        } catch (Exception e) {
            e.printStackTrace();
        }
        System.out.println("Resposta:\n"+page);
    }

    private String getPageContent(String url) throws Exception {

        URL obj = new URL(url);
        conn = (HttpsURLConnection) obj.openConnection();

        // default is GET
        conn.setRequestMethod("GET");

        // Acts like a browser
        conn.setUseCaches(false);
        conn.setRequestProperty("Remote Address",
                "200.129.173.7:443");
        conn.setRequestProperty("Accept",
                "text/html,application/xhtml+xml,application/xml;q=0.9,image/webp,*/*;q=0.8");
        conn.setRequestProperty("Accept-Encoding",
                "gzip, deflate, sdch");
        conn.setRequestProperty("Accept-Language", "pt-BR,pt;q=0.8,en-US;q=0.6,en;q=0.4");
        conn.setRequestProperty("Cache-Control", "max-age=0");
        conn.setRequestProperty("Connection", "keep-alive");
        conn.setRequestProperty("Host", "portal.ufac.br");
        conn.setRequestProperty("User-Agent", USER_AGENT);
        conn.setRequestProperty("Referer", "https://portal.ufac.br/aluno/login.action?error=");

        if (cookies != null) {
            for (String cookie : this.cookies) {
                conn.addRequestProperty("Cookie", cookie.split(";", 1)[0]);
            }
        }
        int responseCode = conn.getResponseCode();
        System.out.println("\nSending 'GET' request to URL : " + url);
        System.out.println("Response Code : " + responseCode);

        BufferedReader in = 
                new BufferedReader(new InputStreamReader(conn.getInputStream()));
        String inputLine;
        StringBuffer response = new StringBuffer();

        while ((inputLine = in.readLine()) != null) {
            response.append(inputLine);
        }
        in.close();

        // Get the response cookies
        setCookies(conn.getHeaderFields().get("Set-Cookie"));

        return response.toString();

    }



    public List<String> getCookies() {
        return cookies;
    }

    public void setCookies(List<String> cookies) {
        this.cookies = cookies;
    }
}

Output:

Sending 'GET' request to URL : https://portal.ufac.br/aluno/login/
Response Code : 302
Resposta:
‹

What am I doing wrong? How can I correctly perform this request?

java http-request

asked by anonymous 15.06.2015 / 00:18

2 answers

2

You can use Jsoup . To get the html code of a page just:

Document document =  Jsoup.connect("http://pt.stackoverflow.com").get();
System.out.println(document.html()); // html da página

In your case, it looks like you're wanting more than the html code. Since the user must be authenticated, there should probably be a cookie that holds the session for the next requests, you can get it like this:

Connection.Response response = Jsoup.connect("https://portal.ufac.br/aluno/")
          .data("j_username", "joaoDaSilva", "j_password", "joao1234")
          .method(Connection.Method.POST)
          .execute();

String theHtml = response.parse().html(); // html
Map<String, String> theCookies = response.cookies(); // obtém os cookies

And in the next requisitions:

Document randomPage = Jsoup.connect("https://portal.ufac/foo")
         .cookies(theCookies)
         .get();

System.out.println(randomPage.html()); // html da página.

If you need something more complete, an alternative is HtmlUnit . In this answer there is a minimal explanation and an example of how to access, fill out and submit a login form on a webpage .

15.06.2015 / 01:10

CMD, Console, MS-DOS and related terms Capture clipboard in Windows with JAVA

score 3 · Accepted Answer

Problem solved! I used the HtmlUnit tool suggested by @ re22 and I was able to retrieve site information.

First I created a WebClient object that simulates the Chrome browser, then I created a CookieManager to manage the session data, so that I can make multiple requests after the site authentication.

final WebClient webClient = new WebClient(BrowserVersion.CHROME);
CookieManager cookieMan = new CookieManager();
cookieMan = webClient.getCookieManager();
cookieMan.setCookiesEnabled(true);

I used these two methods to disable warning messages related to the html documents received when making requests:

webClient.getOptions().setJavaScriptEnabled(false);
webClient.getOptions().setCssEnabled(false);

In this section I capture the login page, its forms, in the case only 1, concatenated them in a single html form, where I add the information to log in the j_password and j_username :

    pagina = webClient.getPage("https://portal.ufac.br/aluno/login.action");

    List<HtmlForm> formularios = pagina.getForms();
    HtmlForm formulario = null;

    for (HtmlForm htmlForm : formularios) {
        formulario = htmlForm;
    }
    HtmlTextInput usuario = formulario.getInputByName("j_username");
    HtmlPasswordInput senha = formulario.getInputByName("j_password");              
    usuario.setValueAttribute("******");
    senha.setValueAttribute("******");

Finally I create a response html page simulating a click on a button, then using that html I make a web request to get the session data, being authenticated is stored by the CookieManager two sessions, otherwise a single session will be stored. Then I made a request to retrieve the contents of the user's profile page after authentication to the site.

final HtmlPage paginaResposta = (HtmlPage) formulario.getInputByValue("Entrar").click();
paginaResposta.getWebResponse();
String result = webClient.getPage("https://portal.ufac.br/aluno/aluno/perfil/perfil.action").getWebResponse().getContentAsString();

Here is the complete implementation:

    //Cria o cliente
    final WebClient webClient = new WebClient(BrowserVersion.CHROME);
    //O CookieManager vai gerenciar os dados da sessão
    CookieManager cookieMan = new CookieManager();
    cookieMan = webClient.getCookieManager();
    cookieMan.setCookiesEnabled(true);

    java.util.logging.Logger.getLogger("com.gargoylesoftware.htmlunit").setLevel(java.util.logging.Level.OFF);
    java.util.logging.Logger.getLogger("org.apache.http").setLevel(java.util.logging.Level.OFF);

    HtmlPage pagina;
    try {
        pagina = webClient.getPage("https://portal.ufac.br/aluno/login.action");


        List<HtmlForm> formularios = pagina.getForms();
        HtmlForm formulario = null;

        for (HtmlForm htmlForm : formularios) {
            formulario = htmlForm;
        }

        HtmlTextInput usuario = formulario.getInputByName("j_username");
        HtmlPasswordInput senha = formulario.getInputByName("j_password");              
        usuario.setValueAttribute("******");
        senha.setValueAttribute("******");

        final HtmlPage paginaResposta = (HtmlPage) formulario.getInputByValue("Entrar").click();
        paginaResposta.getWebResponse();

        //Navegando para a página de perfil do usuário
        String result = webClient.getPage("https://portal.ufac.br/aluno/aluno/perfil/perfil.action").getWebResponse().getContentAsString();
        System.out.println("RESULT:\n "+ result); 
    } catch (FailingHttpStatusCodeException | IOException e) {
        // TODO Auto-generated catch block
        e.printStackTrace();
    }
    System.out.println(cookieMan.getCookies());