In scrapy, I am getting the error exceptions.AttributeError: 'unicode' object has no attribute 'dont_filter'
. After searching around, I found this answer (which made sense as it was the only bit of code I modified before getting the error) according to which I modified my code. I changed start_request
to yield values in the list instead of retruning it whole but I'm still getting it. Any ideas?
def start_requests(self):connection = pymongo.Connection(settings['MONGODB_SERVER'],settings['MONGODB_PORT'])db = connection[settings['MONGODB_DB']]collection = db[settings['MONGODB_COLLECTION']]for el in [i['url'] for i in collection.find({}, {'_id':0, 'url':1})]:yield el
I have checked the other parts of the code to affirm that everything else is fine.
Traceback:
[-] Unhandled ErrorTraceback (most recent call last):File "/home/myName/scrapy-test/venv/local/lib/python2.7/site-packages/scrapy/crawler.py", line 93, in startself.start_reactor()File "/home/myName/scrapy-test/venv/local/lib/python2.7/site-packages/scrapy/crawler.py", line 130, in start_reactorreactor.run(installSignalHandlers=False) # blocking callFile "/home/myName/scrapy-test/venv/local/lib/python2.7/site-packages/twisted/internet/base.py", line 1192, in runself.mainLoop()File "/home/myName/scrapy-test/venv/local/lib/python2.7/site-packages/twisted/internet/base.py", line 1201, in mainLoopself.runUntilCurrent()--- <exception caught here> ---File "/home/myName/scrapy-test/venv/local/lib/python2.7/site-packages/twisted/internet/base.py", line 824, in runUntilCurrentcall.func(*call.args, **call.kw)File "/home/myName/scrapy-test/venv/local/lib/python2.7/site-packages/scrapy/utils/reactor.py", line 41, in __call__return self._func(*self._a, **self._kw)File "/home/myName/scrapy-test/venv/local/lib/python2.7/site-packages/scrapy/core/engine.py", line 120, in _next_requestself.crawl(request, spider)File "/home/myName/scrapy-test/venv/local/lib/python2.7/site-packages/scrapy/core/engine.py", line 176, in crawlself.schedule(request, spider)File "/home/myName/scrapy-test/venv/local/lib/python2.7/site-packages/scrapy/core/engine.py", line 182, in schedulereturn self.slot.scheduler.enqueue_request(request)File "/home/myName/scrapy-test/venv/local/lib/python2.7/site-packages/scrapy/core/scheduler.py", line 48, in enqueue_requestif not request.dont_filter and self.df.request_seen(request):exceptions.AttributeError: 'unicode' object has no attribute 'dont_filter'