Create a new function with scrapy

Question

Create a new function with scrapy

Navigation

#1 by (2 votes)

0

I'm starting to learn scrapy and created the following function:

import scrapy


class ModelSpider(scrapy.Spider):
    name = "model"
    start_urls = [
        'http://www.icarros.com/'
    ]

def parse(self, response):
    with open('brands.csv', 'r') as all_brands:
        for brand in all_brands:
            brand = brand.replace("\n", "")
            url = 'http://www.icarros.com/'+brand
            yield scrapy.Request(url, self.success_connect)

def success_connect(self, response):
    self.log('Entrei')

But the following error appears:

AttributeError: 'ModelSpider' object has no attribute 'success_connect'

python python-3.x scrapy

asked by anonymous 06.09.2016 / 19:03

1 answer

In android which class corresponds to class Image of javaFX What error with my script? [closed]

score 2 · Accepted Answer

You have a problem with the indentation of your file. Both functions are out of class (as Python does not have { } , it's the indentation that defines the blocks of code). The carros_spider.py working file looks like this:

import scrapy


class ModelSpider(scrapy.Spider):
    name = "model"
    start_urls = [
        'http://www.icarros.com/'
    ]

    def parse(self, response):
        for brand in ['ford', 'toyota']:
            brand = brand.replace("\n", "")
            url = 'http://www.icarros.com/'+brand
            yield scrapy.Request(url, self.success_connect)

    def success_connect(self, response):
        self.log('Entrei')

To run:

scrapy runspider carros_spider.py

View the output of this command.