栏目分类:
子分类:
返回
名师互学网用户登录
快速导航关闭
当前搜索
当前分类
子分类
实用工具
热门搜索
名师互学网 > IT > 软件开发 > 后端开发 > Python

【亲测有效】[twisted] CRITICAL: Unhandled error in Deferred 解决办法

Python 更新时间: 发布时间: IT归档 最新发布 模块sitemap 名妆网 法律咨询 聚返吧 英语巴士网 伯小乐 网商动力

【亲测有效】[twisted] CRITICAL: Unhandled error in Deferred 解决办法

[twisted] CRITICAL: Unhandled error in Deferred 解决办法【亲测有效】
SO答案:https://stackoverflow.com/questions/45314514/scrapy-importerror-no-module-named-pipelines

先说原因:
ITEM_PIPELINES = {
‘tutorial.pipelines.jsonpipeline’: 800, // 错误
‘tutorial.pipelines.jsonpipeline.jsonpipeline’: 800, // 正确,原因是scrapy的目录结构有特殊约定
}
这里把pipiline管道放到pipelines包去了,导致scrapy需要重复一次类名。
----------------
默认pipelines.py包含所有的管道,如下:
ITEM_PIPELINES = {
‘tutorial.pipelines.jsonpipeline1’: 800,
‘tutorial.pipelines.jsonpipeline2’: 801,
}

Unhandled error in Deferred:
2022-04-30 23:19:28 [twisted] CRITICAL: Unhandled error in Deferred:

Traceback (most recent call last):
  File "C:ProgramDataMiniconda3envspython383libsite-packagesscrapycrawler.py", line 206, in crawl
    return self._crawl(crawler, *args, **kwargs)
  File "C:ProgramDataMiniconda3envspython383libsite-packagesscrapycrawler.py", line 210, in _crawl
    d = crawler.crawl(*args, **kwargs)
  File "C:ProgramDataMiniconda3envspython383libsite-packagestwistedinternetdefer.py", line 1905, in unwindGenerator
    return _cancellableInlineCallbacks(gen)
  File "C:ProgramDataMiniconda3envspython383libsite-packagestwistedinternetdefer.py", line 1815, in _cancellableInlineCallbacks
    _inlineCallbacks(None, gen, status)
---  ---
  File "C:ProgramDataMiniconda3envspython383libsite-packagestwistedinternetdefer.py", line 1660, in _inlineCallbacks
    result = current_context.run(gen.send, result)
  File "C:ProgramDataMiniconda3envspython383libsite-packagesscrapycrawler.py", line 102, in crawl
    self.engine = self._create_engine()
  File "C:ProgramDataMiniconda3envspython383libsite-packagesscrapycrawler.py", line 116, in _create_engine
    return ExecutionEngine(self, lambda _: self.stop())
  File "C:ProgramDataMiniconda3envspython383libsite-packagesscrapycoreengine.py", line 84, in __init__
    self.scraper = Scraper(crawler)
  File "C:ProgramDataMiniconda3envspython383libsite-packagesscrapycorescraper.py", line 75, in __init__
    self.itemproc = itemproc_cls.from_crawler(crawler)
  File "C:ProgramDataMiniconda3envspython383libsite-packagesscrapymiddleware.py", line 59, in from_crawler
    return cls.from_settings(crawler.settings, crawler)
  File "C:ProgramDataMiniconda3envspython383libsite-packagesscrapymiddleware.py", line 40, in from_settings
    mwcls = load_object(clspath)
  File "C:ProgramDataMiniconda3envspython383libsite-packagesscrapyutilsmisc.py", line 66, in load_object
    raise NameError(f"Module '{module}' doesn't define any object named '{name}'")
builtins.NameError: Module 'tutorial.pipelines' doesn't define any object named 'jsonpipeline'

2022-04-30 23:19:28 [twisted] CRITICAL: 
Traceback (most recent call last):
  File "C:ProgramDataMiniconda3envspython383libsite-packagesscrapyutilsmisc.py", line 64, in load_object
    obj = getattr(mod, name)
AttributeError: module 'tutorial.pipelines' has no attribute 'jsonpipeline'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "C:ProgramDataMiniconda3envspython383libsite-packagestwistedinternetdefer.py", line 1660, in _inlineCallbacks
    result = current_context.run(gen.send, result)
  File "C:ProgramDataMiniconda3envspython383libsite-packagesscrapycrawler.py", line 102, in crawl
    self.engine = self._create_engine()
  File "C:ProgramDataMiniconda3envspython383libsite-packagesscrapycrawler.py", line 116, in _create_engine
    return ExecutionEngine(self, lambda _: self.stop())
  File "C:ProgramDataMiniconda3envspython383libsite-packagesscrapycoreengine.py", line 84, in __init__
    self.scraper = Scraper(crawler)
  File "C:ProgramDataMiniconda3envspython383libsite-packagesscrapycorescraper.py", line 75, in __init__
    self.itemproc = itemproc_cls.from_crawler(crawler)
  File "C:ProgramDataMiniconda3envspython383libsite-packagesscrapymiddleware.py", line 59, in from_crawler
    return cls.from_settings(crawler.settings, crawler)
  File "C:ProgramDataMiniconda3envspython383libsite-packagesscrapymiddleware.py", line 40, in from_settings
    mwcls = load_object(clspath)
  File "C:ProgramDataMiniconda3envspython383libsite-packagesscrapyutilsmisc.py", line 66, in load_object
    raise NameError(f"Module '{module}' doesn't define any object named '{name}'")
NameError: Module 'tutorial.pipelines' doesn't define any object named 'jsonpipeline'

Process finished with exit code 1
转载请注明:文章转载自 www.mshxw.com
本文地址:https://www.mshxw.com/it/846306.html
我们一直用心在做
关于我们 文章归档 网站地图 联系我们

版权所有 (c)2021-2022 MSHXW.COM

ICP备案号:晋ICP备2021003244-6号