Questions tagged as 'web-crawler'

Tagged questions
1
answer

Web Crawler with Django view.py

I am doing a simple web crawler, using django 2.0, I want to capture only the "title" class of the news and then render "return render" to a simple html, below my view.py. At the moment I'm using the "Return HttpRensonse". how can I get the data...
asked by 31.05.2018 / 03:08
0
answers

Click for interactive online chart

I need to make the selection of the information on the left side of the chart according to my need, but I can not reference the information. I did not put any code because I tried it in many ways (according to what I learned from my other questi...
asked by 10.04.2018 / 15:07
0
answers

Automate site navigation for software testing activities

I'm in a web systems development project, accessed by the browser. We are constantly making changes in the operation of the processes and at each specific period we perform a test on the system's functionalities (registers, validations, buttons,...
asked by 02.08.2016 / 16:35
0
answers

Read all posts and comments from a facebook user?

Dear friends, I'm in need of a help. I need to make a crawler that read all the posts with mpremissão from friends and their comments using php. I am using docs-php to do a test with my posts. I read the first page and second, but I can no...
asked by 30.11.2015 / 10:50
3
answers

Get string inside tag a without attributes [duplicate]

I'm using Sun in PHP to get the link from a tag, where through "getAttribute" I can get such a link by the href attribute. Crawler script: <?php //carregamento da url $dom = new DOMDocument(); libxml_use_internal_errors(true); $dom->...
asked by 17.08.2017 / 21:50
1
answer

Tweets Crawler

I'm using the tweeter API provided by python to find certain tweets. The problem is that I want to see the tweets received by the person and not the tweets sent by the person , but I'm not having success, I only view the tweets of the page i...
asked by 22.10.2018 / 19:35
2
answers

simple dom php 404 error

I have the following code: <?php include './simple_html_dom.php'; //Este link existe $teste = new simple_html_dom("http://www.btolinux.com.br/"); echo $teste->original_size."<br>"; if($teste->original_size !== 0){ $teste-&...
asked by 19.05.2014 / 00:15
1
answer

Scrapy for login

I got this code from the internet and I changed a little, to log in to the cpfl website, but when I use the command scrapt crawl myproject nothing happens and the command scrapy runspider items.py gives the error error:    No el...
asked by 08.08.2018 / 15:44
1
answer

Can I automatically run a Python program outside my computer?

Good morning, I've done a Python script that searches the various websites for the occurrence of some words and stores them in SQL database. I would like to track the occurrence of these words over time, but I do not want to be left clicking to...
asked by 06.06.2017 / 10:32
1
answer

How do I integrate my Django project with Scrapy? [closed]

I want to develop a simple project using Django where I will create a web page and this page will capture data from other pages. The problem is that I can not integrate Scrapy with Django .     
asked by 26.04.2017 / 14:28