How to push the button of a site in python

-2

I am doing a webscrapping of the site of the cipher club and taking the most accessed songs of a genre.

The code works normal but the way the page is made only the first 100 songs appear, to show the rest I have to physically click the "Show More" button of the site. Pressing the button does not change anything in the url that I can use in the program to access more pages.

<button id="js-top_more" class="btn btn-full g-sb">mostrar mais</button>

And after that you do not have anything else related to the button.

Is there any way around this?

    
asked by anonymous 18.08.2018 / 18:07

1 answer

0

A very intuitive library to control a browser through Python (from accessing urls to specific page elements and clicking on buttons) is Selenium .

To install it for Python 2. *:

pip install selenium

And for Python 3. *:

pip3 install selenium

After installation for Python 3. *, you can do the following:

from selenium import webdriver
from selenium.webdriver.common.by import By

driver = webdriver.Firefox()
driver.get("https://www.cifraclub.com.br/mais-acessadas/")

btn_more = driver.find_element(By.ID, 'js-top_more')
btn_more.click()

Clicking the button in question will display 20 new songs with each click, and will keep the same ID (which may be quite handy for you), then look at how many songs you want to load.

Another tip I can give you is to use WebDriverWait , which causes the driver to wait for an element to appear based on a delay time and expected_conditions , which will provide conditions for WebdriverWait, which may be key between one click and another on the button.

from selenium.webdriver.support import expected_conditions as EC
from selenium.webdriver.support.ui import WebDriverWait
from selenium import webdriver

driver = webdriver.Firefox()
driver.get("https://www.cifraclub.com.br/mais-acessadas/")

while(sua_condicao):
    btn_more = WebDriverWait(driver, 5).until(EC.presence_of_element_located((By.ID, 'js-top_more')))
    btn_more.click()
    
20.08.2018 / 17:59