Questions tagged as 'web-scraping'

Tagged questions
0
answers

What is the best way to scratch the Datasus site in Python?

The link is this: link I'm trying to send a POST through the requests with a dictionary containing the categories I want, but then the URL is still static. Do you think Selenium would be more suitable for this? Has anyone done anything l...
asked by 15.12.2017 / 17:07
0
answers

phpQuery Web Scraping Event

I'm trying to get information from the php site using phpQuery, but I'm still learning how to use it. The information I want to get appears in select but only after clicking it. Without clicking it returns only a <option value=""...
asked by 11.10.2017 / 16:07
1
answer

Configure webdriver Firefox in Selenium

I'm using selenium (Python) to fetch some data from a website, at some point I access a link that downloads a file. How do I configure webdriver (Firefox) so that it automatically accepts the download, without which I need to click download?   ...
asked by 05.04.2017 / 17:14
0
answers

Converting Web Scraper from Python 2.7 to 3.5

Good afternoon everyone! It is as follows: I found a script in Python 2.7 but I have version 3.6. As I am new in this scope, I wanted to work manually to convert this script. Here is the code below: ### Modified for naming pics by tag impor...
asked by 28.03.2017 / 21:18
0
answers

Automate site navigation for software testing activities

I'm in a web systems development project, accessed by the browser. We are constantly making changes in the operation of the processes and at each specific period we perform a test on the system's functionalities (registers, validations, buttons,...
asked by 02.08.2016 / 16:35
2
answers

Web Scraping how to insert result within img src =

I'm doing a web scraping of a website, but I'd like the returned images to already come to me within <img src= but I'm not succeeding // Find all images foreach($html->find('img') as $element) echo $element->src . '&...
asked by 22.02.2017 / 04:13
2
answers

How to convert CSV to XLSX with python?

How do I convert a .csv file generated by python to .xlsx ? I have two problems: One of them is that I could not figure out how to do this conversion The second is that even if you pass the crawl <nome> -o <nom...
asked by 20.09.2018 / 06:35
1
answer

Web Scraping - convert HTML table into python dict

I'm trying to turn an HTML table into dict @ python, I came across some problems and I ask for help from you. Follow it as far as I could get ... def impl12(url='http://www.geonames.org/countries/', tmout=2): import requests from bs...
asked by 01.08.2017 / 16:59
2
answers

Concatenating multiple lists with Python

Good afternoon! I have a problem and need help, I'm working with 3 distinct lists that should be added to a dictionary, but so I can capture all the values without one overwriting the other, I need to concatenate those 3 lists. I know that us...
asked by 08.08.2018 / 20:14
1
answer

Scraping data using Robobrowser

I'm trying to scrape a form, to insert an attachment and send, using Robobrowser. To open the page I do: browser.open('url') To get the form I do: form = browser.get_form(id='id_form') To insert the data into the form I do: form['...
asked by 12.12.2018 / 16:40