Questions tagged as 'web-scraping'

Tagged questions
1
answer

Simple_html_dom what is the difference between the two URL's?

$URL1 = "http://ladbrokes.365dm.com/greyhounds/profile/dog/oor-millie/3334094"; $URL2 = "http://ladbrokes.365dm.com/greyhounds/profile/dog/nickoff-bingo/3357750"; Url2 works and can extract data, Url1 does not. <?php include "simple_htm...
asked by 07.09.2018 / 16:54
1
answer

Simple Html does not return anything?

Code: require_once("simple_html_dom/simple_html_dom.php"); html = new simple_html_dom(); $html->load_file('http://www.g1.globo.com/'); echo $html; Errors:    Notice: file_get_contents (): send of 16 bytes failed with errno = 10053...
asked by 27.05.2017 / 23:22
1
answer

Scraping data from a site with dynamic tables filtering [closed]

The platform of search of the programs conceptualized in Capes has a dynamic filtering for the own query. I would like to know how I collect the data from an output using Python. Why, using just the bs4 libs and requests I can not get output of...
asked by 06.06.2017 / 23:37
1
answer

Crawler for site scanning [closed]

Talk to me, all right? I'd like to create a Crawler to scan the day on some specific websites and bring me on a spreadsheet or something like the home materials of these sites. In case I would like to do a scan on news portals. I am a laym...
asked by 31.07.2017 / 14:20
1
answer

Problem click button python selenium

I'm having trouble clicking a button, so try gives the ElementNotVisibleException error: Message: element not interactable pdf = driver.find_element_by_xpath('//*[@class="btn btn-default btn-segunda-via-aberta ng-scope"]') pdf.click()...
asked by 17.10.2018 / 20:37
1
answer

How to push the button of a site in python

I am doing a webscrapping of the site of the cipher club and taking the most accessed songs of a genre. The code works normal but the way the page is made only the first 100 songs appear, to show the rest I have to physically click the "Show...
asked by 18.08.2018 / 18:07
1
answer

Web Scraping with python

Good evening. I want to make a simple algorithm to take data from a website ( link ). I've already done a part of the library code: from urllib.request import urlopen from bs4 import BeautifulSoup html = urlopen("http://www.riooilgas.com.br/?_...
asked by 30.08.2018 / 05:48
1
answer

Write file in excel within the Scrapy project [closed]

I have a Spider that takes the xlsx links, in the Request I call the files and saved in: def save_file(self, response): f = open("teste.xls", "wb") f = write(response.body) f.close() But it returns the error: f = write(resp...
asked by 09.05.2018 / 17:47
1
answer

BeautifulSoup find method returns empty [closed]

I did a web scraping program in Python to check the price of Bitcoin on some exchanges , however the field I'm trying to download is left blank. In case, I'm trying to get the data is between the tags <div class="info"> and...
asked by 18.12.2017 / 15:14