Read data from a url

Question

Read data from a url

Navigation

#1 by (1 votes)

0

I would like to get data from a url address

Searching found this answer

Implemented for my reality

var http = require('http');

            var options = {
                host: 'http://192.168.1.6',
                path: '/sistema.cgi?lermc=0,33:80'
            }
            var request = http.request(options, function (res) {
                var data = '';
                res.on('data', function (chunk) {
                    data += chunk;
                });
                res.on('end', function () {
                    console.log(data);

                });
            });
            request.on('error', function (e) {
                console.log(e.message);
            });
            request.end();

I've adapted to the ip that I have that returns a vector

[124.15,1409.51,174988.56,95284.50,0.54,0.00, ....]

Just this, no formatting, no header, no nothing. If it looks in the source code of the page only the vector appears.

This is an internal network address.

Your google display shows

<HTML><HEAD><meta http-equiv="content-type" content="text/html;charset=utf-8">
<TITLE>301 Moved</TITLE></HEAD><BODY>
<H1>301 Moved</H1>
The document has moved
<A HREF="http://www.google.com/">here</A>.
</BODY></HTML>

But mine shows the message:

Parse error

url node.js request

asked by anonymous 20.08.2018 / 16:18

1 answer

Return a JavaScript variable and PHP content - AJAX Resize two divs at once with jquery-ui

score 1 · Accepted Answer

The first thing to do to recreate a HTTP request within Node.js is to identify the information that is sent in such a request.

I'll suggest a step by step using URL https://pt.stackoverflow.com/ as an example. For this you will need Chrome , Postman and the got of NPM :

Open an empty tab of Chrome , right-click, and select Inspecionar . A new window with DevTools will be opened (referencing the new tab that was previously opened);

Access the destination address;

Identify the request on the Network tab of the DevTools ;

Right-click the requisition and select the 'Copy > Copy as cURL (cmd);

Inthecaseofmyrequesttheresultwas:

curl"https://pt.stackoverflow.com/" -H "Connection: keep-alive" -H "Upgrade-Insecure-Requests: 1" -H "User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/68.0.3440.106 Safari/537.36" -H "Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/webp,image/apng,*/*;q=0.8" -H "Accept-Encoding: gzip, deflate, br" -H "Accept-Language: pt-BR,pt;q=0.9,en-US;q=0.8,en;q=0.7" --compressed

In the Postman click the Import option located in the upper left corner and go to the Paste Raw Text tab. Paste the feed of cURL that was obtained in DevTools of Chrome ;

When importing, the request with all the headers ( Headers ) and necessary information will be mounted;

Click SEND and check the result. If the status is 200 the request was executed successfully. You can also check the response content on the Body tab;

Youcanuse Postman to generate the Node code for you or continue with the options from step 8. To generate the code you can click the code option on the right side of the window and select the language you want;

After confirming all request data, the code for execution installs the module got :

npm install got

The code to run in Node.js with the duly completed parameters will be as follows:

(async () => {
  const { URLSearchParams } = require('url');
  const got = require('got');

  try {
    const params = new URLSearchParams({});
    const { body } = await got('https://pt.stackoverflow.com?${params}', {
      headers: {
        'Connection': 'keep-alive',
        'Upgrade-Insecure-Requests': '1',
        'User-Agent': 'Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/68.0.3440.106 Safari/537.36',
        'Accept': 'text/html,application/xhtml+xml,application/xml;q=0.9,image/webp,image/apng,*/*;q=0.8',
        'Accept-Encoding': 'gzip, deflate, br',
        'Accept-Language': 'pt-BR,pt;q=0.9,en-US;q=0.8,en;q=0.7'
      }
    });

    console.log(body)
  } catch (err) {
    console.log(err.message)
  }
})();

With the cURL information provided by your question request:

curl "http://147.1.31.61/sistema.cgi?lermc=0,33:80" -H "Connection: keep-alive" -H "Cache-Control: max-age=0" -H "Upgrade-Insecure-Requests: 1" -H "User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/68.0.3440.106 Safari/537.36" -H "Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/webp,image/apng,/;‌q=0.8" -H "Accept-Encoding: gzip, deflate" -H "Accept-Language: pt-BR,pt;q=0.9,en-US;q=0.8,en;q=0.7" --compressed

We will have the following execution code:

(async () => {
  const { URLSearchParams } = require('url');
  const got = require('got');

  try {
    const params = new URLSearchParams({ 'lermc': '0,33:80' });
    const { body } = await got('http://147.1.31.61/sistema.cgi?${params}', {
      headers: {
        'Connection': 'keep-alive',
        'Cache-Control': 'max-age=0',
        'Upgrade-Insecure-Requests': '1',
        'User-Agent': 'Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/68.0.3440.106 Safari/537.36',
        'Accept': 'text/html,application/xhtml+xml,application/xml;q=0.9,image/webp,image/apng,/;‌q=0.8',
        'Accept-Encoding': 'gzip, deflate',
        'Accept-Language': 'pt-BR,pt;q=0.9,en-US;q=0.8,en;q=0.7'
      }
    });

    console.log(body)
  } catch (err) {
    console.log(err.message)
  }
})();

Of course we should also look at some things, such as status of return of service and type of response, but in essence with the above information it is possible to carry out the request.