Scraping data from a site with dynamic tables filtering [closed]

-1

The platform of search of the programs conceptualized in Capes has a dynamic filtering for the own query. I would like to know how I collect the data from an output using Python. Why, using just the bs4 libs and requests I can not get output of these data (I believe that just because it is a dynamic query). On account of JS and Ajax.

Reference site CAPES programs

In Python, how could you wipe the data from this query with a specific filtering?

EDIT: Well, apparently, I was able to learn a bit more about the Selenium lib and made some progress. My code so far:

from selenium import webdriver
from selenium.webdriver.support.ui import Select

chrome_path=r"C:\Users\USERNAME\Desktop\chromedriver_win32\chromedriver.exe"
driver=webdriver.Chrome(chrome_path)
driver.get("https://sucupira.capes.gov.br/sucupira/public/consultas/coleta/programa/listaPrograma.jsf")
driver.find_element_by_id("""form:checkAreaAv""").click()
driver.find_element_by_xpath("""//*[@id="form:autocompleteAreaAv:input"]""").send_keys("CIÊNCIA DA COMPUTAÇÃO")
Select(driver.find_element_by_xpath("""//*[@id="form:autocompleteAreaAv:listbox"]""")).select_by_value(2)
f.find_element_by_xpath("""//*[@id="form:consultar"]""").click()
results=f.find_elements_by_class_name("resultados")

However, I'm having a hard time selecting an item from a drop-down menu in the Evaluation Area text itself .

    
asked by anonymous 06.06.2017 / 23:37

1 answer

0

Maybe you could use Selenium to emulate the usage and capture the data. If the amount of data to be captured is not very large, Selenium can certainly help you.

    
07.06.2017 / 16:17