Fill in JS Form - Web Scraping in Python - Selenium and PhantomPS

0

Friends.

I am developing a code to access the Anbima site, fill in the fields and download the txt that is generated.

I've been looking for a solution to this problem for a few days now. So far, I've found that the information session is an iframe generated from another page . The code I've developed so far is this:

from selenium import webdriver

#Variables

url = 'http://www.anbima.com.br/reune/reune.asp'
path_phantom = 'C:\Users\TBMEPYG\AppData\Local\Continuum\Anaconda3\Lib\site-packages\phantomjs-2.1.1-windows\bin\phantomjs.exe'

#Processing

driver = webdriver.PhantomJS(executable_path= path_phantom)

data = driver.find_element_by_name('Dt_Ref')
data.clear()
data.send_keys('21/08/2017')
driver.quit()

Note that the idea and fill in the first date form, however I get the following error, stating that the element was not found.

Traceback (most recent call last):
  File "C:\Users\TBMEPYG\Desktop\beta_anbima.py", line 16, in <module>
    data = driver.find_element_by_name('Dt_Ref')
  File "C:\Users\TBMEPYG\AppData\Local\Continuum\Anaconda3\lib\site-packages\selenium\webdriver\remote\webdriver.py", line 426, in find_element_by_name
    return self.find_element(by=By.NAME, value=name)
  File "C:\Users\TBMEPYG\AppData\Local\Continuum\Anaconda3\lib\site-packages\selenium\webdriver\remote\webdriver.py", line 832, in find_element
    'value': value})['value']
  File "C:\Users\TBMEPYG\AppData\Local\Continuum\Anaconda3\lib\site-packages\selenium\webdriver\remote\webdriver.py", line 297, in execute
    self.error_handler.check_response(response)
  File "C:\Users\TBMEPYG\AppData\Local\Continuum\Anaconda3\lib\site-packages\selenium\webdriver\remote\errorhandler.py", line 194, in check_response
    raise exception_class(message, screen, stacktrace)
selenium.common.exceptions.NoSuchElementException: Message: {"errorMessage":"Unable to find element with name 'Dt_Ref'","request":{"headers":{"Accept":"application/json","Accept-Encoding":"identity","Connection":"close","Content-Length":"89","Content-Type":"application/json;charset=UTF-8","Host":"127.0.0.1:62051","User-Agent":"Python http auth"},"httpVersion":"1.1","method":"POST","post":"{\"using\": \"name\", \"value\": \"Dt_Ref\", \"sessionId\": \"855174e0-8e52-11e7-84d4-0792b107ed82\"}","url":"/element","urlParsed":{"anchor":"","query":"","file":"element","directory":"/","path":"/element","relative":"/element","port":"","host":"","password":"","user":"","userInfo":"","authority":"","protocol":"","source":"/element","queryKey":{},"chunks":["element"]},"urlOriginal":"/session/855174e0-8e52-11e7-84d4-0792b107ed82/element"}}
Screenshot: available via screen

Any idea of what might be happening?

Estre project is being developed in my work, which has high restrictions with external access. I would not have the flexibility to change the webdriver for Chrome or Firefox.

I'm waiting for you.

Edit 2:

I found an error in the code, I was not giving the get command. By making this adjustment, a new error arises. Follow the new code

from selenium import webdriver


url = 'http://www.anbima.com.br/reune/reune.asp'
path_phantom = 'C:\Users\TBMEPYG\AppData\Local\Continuum\Anaconda3\Lib\site-packages\phantomjs-2.1.1-windows\bin\phantomjs.exe'

#Processing

driver = webdriver.PhantomJS(executable_path= path_phantom)
driver.get(url)

#Encontrando o nome do field

name = driver.find_element_by_name("Dt_Ref")

driver.quit()

And the error found now is different:

Traceback (most recent call last):
  File "C:\Users\TBMEPYG\Desktop\beta_anbima.py", line 18, in <module>
    name = driver.find_element_by_name("Dt_Ref")
  File "C:\Users\TBMEPYG\AppData\Local\Continuum\Anaconda3\lib\site-packages\selenium\webdriver\remote\webdriver.py", line 426, in find_element_by_name
    return self.find_element(by=By.NAME, value=name)
  File "C:\Users\TBMEPYG\AppData\Local\Continuum\Anaconda3\lib\site-packages\selenium\webdriver\remote\webdriver.py", line 832, in find_element
    'value': value})['value']
  File "C:\Users\TBMEPYG\AppData\Local\Continuum\Anaconda3\lib\site-packages\selenium\webdriver\remote\webdriver.py", line 297, in execute
    self.error_handler.check_response(response)
  File "C:\Users\TBMEPYG\AppData\Local\Continuum\Anaconda3\lib\site-packages\selenium\webdriver\remote\errorhandler.py", line 194, in check_response
    raise exception_class(message, screen, stacktrace)
selenium.common.exceptions.WebDriverException: Message: {"request":{"headers":{"Accept":"application/json","Accept-Encoding":"identity","Connection":"close","Content-Length":"89","Content-Type":"application/json;charset=UTF-8","Host":"127.0.0.1:53945","User-Agent":"Python http auth"},"httpVersion":"1.1","method":"POST","post":"{\"using\": \"name\", \"value\": \"Dt_Ref\", \"sessionId\": \"89ca1270-8e57-11e7-a14f-5d3312ce36ce\"}","url":"/element","urlParsed":{"anchor":"","query":"","file":"element","directory":"/","path":"/element","relative":"/element","port":"","host":"","password":"","user":"","userInfo":"","authority":"","protocol":"","source":"/element","queryKey":{},"chunks":["element"]},"urlOriginal":"/session/89ca1270-8e57-11e7-a14f-5d3312ce36ce/element"}}
Screenshot: available via screen
    
asked by anonymous 31.08.2017 / 15:48

1 answer

0

Check the name of the element you want to access. The Dt_Ref element does not exist according to the error:

 data = driver.find_element_by_name('Dt_Ref')

"errorMessage":"Unable to find element with name 'Dt_Ref'",
    
31.08.2017 / 15:53