I can not get the input value "csrfmiddlewaretoken"

Navigation
1

I can not get the value of the input "csrfmiddlewaretoken" from this link, could anyone help me? "The script would basically have to visit the page that is in the code and return the value of the input whose name is" csrfmiddlewaretoken "in the console 

import requests as r
from bs4 import BeautifulSoup as bs
import time
import os
clear = lambda: os.system('cls')



req = r.get('https://www.udemy.com/join/login-popup/?ref=&display_type=popup&locale=pt_BR&response_type=json&next=https%3A%2F%2Fwww.udemy.com%2Fmobile%2Fipad%2F&xref=')

from bs4 import BeautifulSoup as bs

soup = bs(req.text, 'html.parser')
inp = soup.find('input', {'name': 'csrfmiddlewaretoken'})
val_inp = inp.get('value')

Error: 

Traceback (most recent call last):
  File "gg.py", line 18, in <module>
    val_inp = inp.get('value')
AttributeError: 'NoneType' object has no attribute 'get'

On page: 

<input type='hidden' name='csrfmiddlewaretoken' value='VyHNALqmdrUIRP72n3Yj8O8lsN0vfQwtsmzbCYbuTNDbtLR1sbg2Xk368y7U8M7d' />

The value changes every request / f5 / refresh

    
asked by anonymous 18.05.2018 / 17:13

1 answer

1

If you parse the request's redo in the soup variable (with the print(repr(soup)) command) you will see that the site is returning the message:

  

Access to this page has been denied because we believe you are using automation tools to browse the website.

That is, the site detects that the request was made by an automated tool and returns content different than expected.

A possible solution to this case is to inform the header User-Agent with the same value as a regular browser.

Follow the modified code using the user agent of the FireFox browser: 

import requests as r
from bs4 import BeautifulSoup as bs
import time
import os
clear = lambda: os.system('cls')


# AQUI => informar o User-Agent'
req = r.get(
         'https://www.udemy.com/join/login-popup/?ref=&display_type=popup&locale=pt_BR&response_type=json&next=https%3A%2F%2Fwww.udemy.com%2Fmobile%2Fipad%2F&xref=', 
          headers={'User-Agent':'Mozilla/5.0 (Windows NT 6.3; Win64; x64; rv:59.0) Gecko/20100101 Firefox/59.0'}
)

from bs4 import BeautifulSoup as bs

soup = bs(req.text, 'html.parser')
inp = soup.find('input', {'name': 'csrfmiddlewaretoken'})
val_inp = inp.get('value')
print("Valor = {}".format(val_inp))

After running, the program prints: 

Valor = tnRToqAGFLH7gsc6TEbGdv30DABebPYFcNf9WnZvsv6xoLmum3hhi3Y3ZUcnVn1a
    
18.05.2018 / 17:34
JavaFX - locked screens submit button does not show it was clicked