Questions tagged as 'scraping'

Tagged questions
1
answer

How to collect text when it has no reference HTML class - Python Crawler

I have the following situation: Iwanttocollect"Crawler Text" below, how do I navigate there without a class or id? <td>Texto para crawler</td>     
asked by 15.04.2017 / 00:09
1
answer

How to do Scrapping a page that has a javascript's using python?

I need to scrape a page, however in the page entry has a button (apparently a Javascript) that gives access to the entire content of the page itself. Using the traditional libs (urllib2, requests, BeatifulSoap) I can not "pull" the content I nee...
asked by 09.03.2017 / 15:46
1
answer

Discover link in python

I am doing a web scrapping in python and sometimes I come across some links and / or buttons that are not with the actual URL of the url so that you will be redirecting if you click. In this case, if I click, it downloads a PDF file, but I j...
asked by 27.09.2016 / 21:44
0
answers

Incomplete tweets in search of searchTwitter method

Hello, the Stackoverflow BR community, I'm playing with the twitteR library of the programming language R , and I realize that in the searches I've performed, the texts are cut , example: "RT @ fulaniinn2070 I do not believe that after...
asked by 14.09.2018 / 23:57
0
answers

how to capture specific tokens in python requests

I need to catch specific cookies from a python code and pass them on afterwards, the problem is, if I set the cookies manually they work for a while but after a few days they expire. I can get a recursive token with the cookie = requests.cookies...
asked by 09.08.2018 / 15:58
1
answer

Web Crawler with Django view.py

I am doing a simple web crawler, using django 2.0, I want to capture only the "title" class of the news and then render "return render" to a simple html, below my view.py. At the moment I'm using the "Return HttpRensonse". how can I get the data...
asked by 31.05.2018 / 03:08
1
answer

Web Scraping - convert HTML table into python dict

I'm trying to turn an HTML table into dict @ python, I came across some problems and I ask for help from you. Follow it as far as I could get ... def impl12(url='http://www.geonames.org/countries/', tmout=2): import requests from bs...
asked by 01.08.2017 / 16:59
1
answer

Pass a URL list to Scrapy function

I have a Python API that gets two arguments (URL and a user-defined word) and provides in the JSON file how many times the specified word appears in the URL. However, I would like to pass a list of URLs. I would also like to make the request...
asked by 28.09.2018 / 16:33
1
answer

Scraping in Python. Assemble an insert [closed]

I would like to extract this table using perhaps Scraping in Python: link I would like to know if it is possible to extract this table so that the sql of insert is mounted as below: INSERT INTO TABELA (CAMPO1, CAMPO2) VALUES (1007, 'Co...
asked by 27.04.2017 / 22:53
1
answer

Data scraping with jsoup and saving in txt

Hello, how are you guys? I'm trying to learn how to scrape data on my account, and since my English does not help, I'm turning 30. It's basically the following. While executing my code, he lists the athletes of the International Judo Federation,...
asked by 29.10.2018 / 19:09