Questions tagged as 'web-scraping'

Tagged questions
1
answer

How do I display variables on a django page?

I am new to django and am taking some information from a web page using lxml. I would like to know how I can display the values on my web page. import requests from lxml import html from django.shortcuts import render def list_data(request):...
asked by 18.04.2018 / 23:09
1
answer

How to click on a checkbox when another element obscure it?

I am writing a code to automate some processes in the SIAFI site, I could not make python click on a checkbox, other than importing the pynput package and using the mouse position with the coordinates (x, y): from pynput.mouse import Button, C...
asked by 20.03.2018 / 15:22
0
answers

Scrapy Multiples Requests returning the same information

Hello, I'm trying to extract data, but strangely the return is coming by 2 in 2, ie if I make 6 different requests it returns 6 results but only 3 of them are destined. In the example below the response even though a different request (ncm...
asked by 31.01.2018 / 22:57
0
answers

POST scraper error Itaú

I'm doing a JAVA Scraper for Itaú's mobile site and I have the error below: Asperthecode,I'mgettingtheGETtogetasessionandthenthePOSTwiththerequiredparametersandheaders(thesameonesdoneinPython henriquebastos ). I believe this is the default r...
asked by 22.01.2018 / 23:32
1
answer

Fill in JS Form - Web Scraping in Python - Selenium and PhantomPS

Friends. I am developing a code to access the Anbima site, fill in the fields and download the txt that is generated. I've been looking for a solution to this problem for a few days now. So far, I've found that the information session is...
asked by 31.08.2017 / 15:48
1
answer

Python: select checkbox in an orderly manner selenium web scraping

I have a list containing hundreds of data in the format [ '5008489', 'Órgão: MPF', 'PROCEDIMENTO DO JUIZADO ESPECIAL', 'CPF', <selenium.webdriver.remote.webelement.WebElement (session="8834847081a4be257906cce85807f88...
asked by 31.08.2017 / 21:18
1
answer

How to capture the td of a web page using selenium vba?

The html code looks like this: <table> <tr> <td width="01%" class="tex3b"><img height="14" src="/imagens/tm_bullet.gif" width="6"></td> <td width="20%" class="tex3b">Órgão</td>...
asked by 29.12.2016 / 14:58
0
answers

Crawler for Woocommerce

Good afternoon friends. I am developing a crawler in php that will do scrapping of some urls that I will inform. I'm trying to make it pull values from a dynamic url, but I'm not getting it. Someone could help me. <?php $page_title...
asked by 09.03.2016 / 21:37
1
answer

Macro to access website with login

I run a daily routine to access the Serasa site and make the CNPJ query. I need to develop a macro to access this Serasa site, log in and then the query, and then throw the information into Excel. Detail: I have already developed the macro to ac...
asked by 26.12.2014 / 12:09
1
answer

I want to scrape a page, but I can not get a text that has "& nbsp" in it

Well, the title says it all, I want to get the price of a product from link , which looks like this: / p> <span class="nm-price-value" itemprop="price">R$&nbsp;367,60</span> Using this code: import bs4 import requests def...
asked by 21.10.2018 / 17:13