Ways to use cURL with PHP?

Question

Ways to use cURL with PHP?

Navigation

#1 by (2 votes)
#2 by (1 votes)

1

I'm studying cURL to continue my application , I reviewed my code and monitored the network Twitter and got the following HTTP Headers :

curl "https://twitter.com/" 

-H "accept-encoding: gzip, deflate, br" 
-H "accept-language: pt-BR,pt;q=0.8,en-US;q=0.6,en;q=0.4" 
-H "upgrade-insecure-requests: 1" 
-H "user-agent: Mozilla/5.0 (Windows NT 6.1; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/59.0.3071.115 Safari/537.36" 
-H "accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/webp,image/apng,*/*;q=0.8" 
-H "cache-control: max-age=0" 
-H "authority: twitter.com" 
-H "cookie: (RETIRADO POR SEGURANÇA!) --compressed

My code is very simple, but the question has arisen, is there a need to make the same calls?

Down my simpleton code:

$cookies = [];

$url_twitter = 'https://twitter.com';

$twitter_cookies = curl_init();
curl_setopt_array($twitter_cookies, [
        CURLOPT_URL             => $url_twitter,
        CURLOPT_RETURNTRANSFER  => 1,
        CURLOPT_SSL_VERIFYPEER  => 0,
        CURLOPT_SSL_VERIFYHOST  => 0,
        CURLOPT_COOKIEJAR               => ROOT . 'system' . SEPARATOR . 'cookies' . SEPARATOR . $TwitterUser . '.txt',
        CURLOPT_HEADERFUNCTION  => function($twitter_cookies, $header) use (&$cookies) {
            if (stripos($header, 'Set-Cookie:') === 0) {
                if (preg_match('/Set-Cookie:\s?(.*?);/i', $header, $matches)) {
                    $cookies[] = $matches[1];
                }
            }

            return strlen($header);
        }
    ]
);

As you can see, I did not implement half of HEADERS requests that Twitter showed by copying the cURL quoted at the beginning of the topic.

I do not know if it adds automatically, I understand the basics of HTTPS , but do I need to make these calls or is it automatically added?

"accept-encoding: gzip, deflate, br"

"accept-language: pt-BR,pt;q=0.8,en-US;q=0.6,en;q=0.4"

"upgrade-insecure-requests: 1"

"user-agent: Mozilla/5.0 (Windows NT 6.1; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/59.0.3071.115 Safari/537.36"

"accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/webp,image/apng,*/*;q=0.8"

"cache-control: max-age=0"

php twitter curl

asked by anonymous 14.07.2017 / 21:36

2 answers

2

Generally, HTTP requests are performed via the browser, which naturally provides several HTTP Headers within the request.

The use of the word need is complicated, as it depends more on the side of the application being requested than on what you are sending. It is necessary to study the application and understand the types of parameters it accepts.

Let's say you can send any information inside the curl header, such as teste=123 , and the application's return may vary, as an exception to additional arguments or even the application may ignore this.

14.07.2017 / 21:53

Asynchronous angular responses 4 Error posting through Delphi

score 1 · Accepted Answer

Headers are used so that both parties can understand each other in a general way, they define what the client supports or expects as a result, among other options, some for security.

The first accept-encoding tells what type of client compression it supports. If you use a cURL that does not support this the system will break, it will pop up a bunch of "strange characters". CURL can set these headers automatically, informing what it supports, as long as you set CURLOPT_ACCEPT_ENCODING (before it was CURLOPT_ENCODING ) to "" (empty string), if set to null (the default) it will not use no type of compression.

accept-language and upgrade-insecure-requests are self explanatory, the first can be useful if the website returns information based on the language, the second indicates that the client prefers to be redirected to HTTPS if it is in HTTP.

user-agent extremely important to cURL, so it can pass through any browser, including mobile browsers . The cURL has up to CURLOPT_USERAGENT so you can set the desired value, or you can do manually, what I usually do.

The accept and cache-control sets so that it does not use a cached version, but in cURL this is not enough. You should also use CURLOPT_FRESH_CONNECT and CURLOPT_FORBID_REUSE depending on the cases to actually create a new request.

The authority: (actually must be :authority ) is a " HTTP / 2 Pseudo-Header , equivalent to HOST , is recommended to be set, but cURL does this automatically, most of the time . But, it is extremely necessary to manually define if it will connect directly to an IP, to avoid DNS-Lookup .

cookie: are cookies can be used in a variety of ways, from CURLOPT_COOKIEFILE or CURLOPT_COOKIE or manually. There is also the option to use CURLOPT_COOKIESESSION to skip session cookies and the like.

Several other headers can exist (such as referer , pragma , origin , dnt plus things like content-type and content-length that are set automatically when needed).

But it's still possible to create a custom header (usually started in x- ), in case Twitter itself uses some as x-csrf-token .

Any header can be defined in cURL using -H or CURLOPT_HTTPHEADER , without needing to use any other function for that.