Scraping parameters from a post method, with scrapy in python!

Navigation
1

I need to collect information from a site using spiders within ScraPy in Python, however the site is a post method and I am learning the language while developing the project. I found a template for post but I'm not able to run it correctly. The code I have is this: 

scrapy.FormRequest(
    url='http://www.camex.gov.br/resolucoes-camex/resolucoes',
    formdata={
        'filter[search]': '',
        'filter[res]': '',
        'filter[ano]': '',
        'limit': paginas,
        'limitstart': quantidadeDeRegistros,
        'task': '',
        'boxchecked': 0,
        'filter_order': '',
        'filter_order_Dir': '',
        '46598c34d1ab5af3b00e8d84a4281fbc': 1,
        'list[fullordering]': 'null ASC'
    },
    callback=self.parsePagina
)

Is it correct or is there a better way to do it?

    
asked by anonymous 07.05.2018 / 13:34

1 answer

1

I ran some tests, this problem happens because the page you're trying to submit the form has two form elements. The scrapy is sending the pro request first, but it should be the second.

Tofixthisandyourspiderwork,youshouldaddtheformnameattributeinthemethodcall:

yieldFormRequest.from_response(response,url='http://www.camex.gov.br/resolucoes-camex/resolucoes',formname="adminForm", # nome do form que você deseja enviar a request
    formdata={
            'filter[search]': '', #codigo omitido
    
07.05.2018 / 16:52
Inverted Scroll, start at bottom [duplicate] Error in scan (file = file, what = what ...)