"missing scheme" error using Scrapy

Navigation
0

When I run my spider, the scrapy returns the following error:

  

ValueError: Missing scheme in request url h 

import scrapy

class QuotesSpider(scrapy.Spider):
    name = "Mineracao"

    def start_requests(self):

        link = "http://www.jornalpanorama.com.br/site/data-policia.php?page="
        y=1
        for x in range(240):
            urls=link+str(y)
            y=y+1
            print urls  

        for url in start_urls:
            yield scrapy.Request(url=url, callback=self.parse)

    def parse(self, response):

        url = "http://www.jornalpanorama.com.br/site/"
        for x in response.xpath("//*[contains(@class, 'listar-noticias-titulo')]/a/@href").extract():
            print url + x
    
asked by anonymous 06.05.2017 / 21:04

1 answer

0

Assuming you want to visit all pages (from 1 to 240), you probably wanted to do: 

def start_requests(self):
    link = "http://www.jornalpanorama.com.br/site/data-policia.php?page="
    for x in range(1, 240):
        yield scrapy.Request(url=link + str(x), callback=self.parse)

If you really want to start from page 1 and skip the even pages (as in your code), you can use range(1, 240, 2) instead of range(240) .

    
01.11.2017 / 15:09
How to consume data online in an Adapter Validation of form and localstorage