Giter Club home page Giter Club logo

topic_and_user_profile_analysis_system's Introduction

Topic and user profile analysis system

介绍

微博的话题、博文及用户画像分析系统。

功能:

  • 微博话题博文搜索：根据用户输入关键字搜索出热度最高的50页近1000条博文

  • 微博话题分析：用户可以将相关话题加入分析任务队列中，后端会异步执行分析任务(基于celery)

    • 话题任务列表：当前系统中的所有任务，包括分析中、完成和异常的任务。

    • 话题最热博文：展示出该话题下热度最高的十条博文。

    • 话题热度：展示该话题近一天、一个月、三个月的关注热度。

    • 话题词云：根据话题中爬取到的所有博文内容分析出话题关联的词语。

    • 话题简略传播关系图：用有向图展示出该话题的传播关系。

    avatar

  • 博文分析：对话题中热度前十的博文进行详细分析。

    • 博文详情：展示博文的文本内容和发博用户的粗略信息。

    • 博文关键转发节点：列出在博文传播过程中起重要扩散作用的转发节点，通常是转发后产生相对大量关注的节点。

    • 博文传播树：博文的转发传播树。

    • 博文评论词云：博文评论下关联的词语。

    • 博文主题分析：博文下评论主题及其占比。

    • 博文热点转发：显示关注较高的热点转发内容。

    avatar

  • 用户画像：根据用户在不同话题下微博文本的聚类主题为用户打标签，逐步完善用户画像。

    artvar

设计

  • 概要设计：前端页面低保真原型设计

  • 详细设计：

    • 架构设计：项目整体采用BS架构实现前后端分离。前端具体使用vue框架，参照[前端页面低保真原型设计来实现，后端使用python的[fastapi](FastAPI (tiangolo.com))框架搭建，结合分布式异步任务处理框架[celery](使用Redis - Celery 中文手册 (celerycn.io))实现任务发布和任务执行的解耦，使得异步、快速处理话题分析任务成为了可能。

    • 涉及算法：

      • 基于MGP的博文短文本聚类分析算法gsdmmCluster,用于对某一博文的评论进行聚类分析。

      • LDA，用于对话题内舆论场进行聚类分析。

      • Single_Pass，用于对话题舆论场进行聚类分析。

    • 路由、数据库设计文档：在\doc\desgin文件中

配置

  • 后端配置：

    • 在文件\code\back_end\config\config_class中修改如下配置：

      class AppConfig(BaseSettings):
    """
    fastapi_app启动的相关配置
    """
    HOST: str = '127.0.0.1'
    PORT: int = 81


class WeiBoConfig(BaseSettings):
"""weibo 爬虫api的相关配置"""
    BASEPATH: str = 'http://127.0.0.1:8000'

    • 在\code\back_end\celery_task\config\task_config_class.py文件中修改celery相关配置：

      from pydantic import BaseSettings

class CeleryConfig(BaseSettings):
    """
    celery 启动的相关配置
    """
    BROKER = 'redis://localhost:6379/0'
    BACKEND = 'redis://localhost:6379/1'


class MongoConfig(BaseSettings):
  """
  Mongo的相关配置
  """
  HOST: str = '127.0.0.1'
  PORT: int = 27017
  DB_NAME: str = 'test'

  # 话题任务数据库名称
  TASK: str = 'tag_task'
  BLOG: str = 'blog'
  CHARACTER: str = 'character_category'
  EVOLVE: str = 'tag_evolve'
  HOT: str = 'tag_hot'
  INTRODUCE: str = 'tag_introduce'
  RELATION: str = 'tag_relation_graph'
  RETWEET: str = 'tag_weibo_task'
  CLOUD: str = 'tag_word_cloud'
  USER: str = 'tag_user'

  # 评论任务数据库名称
  COMMENT_TASK = 'comment_task'
  COMMENT_REPOSTS = 'comment_reposts'
  COMMENT_CLOUD = 'comment_cloud'
  COMMENT_CLUSTER = 'comment_cluster'
  COMMENT_NODE = 'comment_node'
  COMMENT_TENDENCY = 'comment_tendency'
  COMMENT_TOPIC = 'comment_topic'
  COMMENT_TREE = 'comment_tree'


class ElasticSearchConfig(BaseSettings):
  """
  ES配置
  """
  ES_HOST = '127.0.0.1:9200'
  ES_SEARCH_INDEX = 'weibo'
  ES_TIMEOUT = 60
  LANG_TYPE = ['zh', 'en']

  • 微博爬虫微服务配置:

    请参照\code\weibo_crawler\README.md文件进行配置

启动

  • 后端

    • 安装Redis数据库

    • 依据项目中requestments.txt文件，安装好后端需要的所有依赖库

      强烈建议参照doc\话题分析系统后端部署.docx文件构建虚拟运行环境。

    • 运行\code\back_end\main.py启动fastapi的app

    • 启动Redis数据库

    • 进入\code\back_end文件夹，运行celery -A celery_task.celeryapp worker --loglevel=info -P threads命令启动celery

  • 前端

    • 进入\code\back_end文件夹，按照README.md文件安装依赖。

    • 执行npm install命令安装vue项目。

    • 执行npm run server运行前端。

  • 微博爬虫

    进入\code\weibo_crawler文件夹，配置好后，运行weibo_curl_api.py文件

topic_and_user_profile_analysis_system's People

Contributors

faker-lz avatar

Stargazers

avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar

Watchers

avatar avatar avatar

Forkers

dgo2dance deckcode cynoglossum lynnfox singer826 a101269 zgcanfly jiangjun-liang rianley jimersylee chenhaofan1118 learnmat huazai2333 windstarry szy833 djiango-up tsistelymo daweitech icarus-alpha davidleexxxx rk0123 butterflyandlotus aragornjia

topic_and_user_profile_analysis_system's Issues

后端包

后端两个包好像已经用不了了 大佬快看看psyco==1.6 scikit_learn==1.2.0

psyco无法安装

大佬，安装不了psyco呀，pip提示psyco已经不存在了，到psyco官网下载后又不支持win64位系统，请问你是怎么安装的啊？

全部安好运行后，在网页点击搜索报错 

    大佬，当在前端网页点击搜索按钮时，爬虫报错

Traceback (most recent call last):
File "C:\Users\hl199\Topic_and_user_profile_analysis_system-master\code\weibo_crawler\selector_parser\search_weibo_parser.py", line 22, in parse_page
weibo_list = self._get_all_weibo()
File "C:\Users\hl199\Topic_and_user_profile_analysis_system-master\code\weibo_crawler\selector_parser\search_weibo_parser.py", line 33, in _get_all_weibo
for weibo in self._parse_weibo():
File "C:\Users\hl199\Topic_and_user_profile_analysis_system-master\code\weibo_crawler\selector_parser\search_weibo_parser.py", line 55, in _parse_weibo
weibo['weibo_id'] = sel.xpath('(.//p[@Class="from"])[last()]/a[1]/@href')[0].split('/')[-1].split('?')[0]
IndexError: list index out of range
似乎是微博没有返回任何东西，是我爬虫配置出错了吗？
同时main.py报错pymongo.errors.ServerSelectionTimeoutError: 127.0.0.1:27017: [WinError 10061] 由于目标计算机积极拒绝，无法连接。
请问该怎么解决？

爬不出来内容

ERROR: Exception in ASGI application
Traceback (most recent call last):
File "D:\anaconda\envs\333\lib\site-packages\uvicorn\protocols\http\h11_impl.py", line 369, in run_asgi
result = await app(self.scope, self.receive, self.send)
File "D:\anaconda\envs\333\lib\site-packages\uvicorn\middleware\proxy_headers.py", line 59, in call
return await self.app(scope, receive, send)
File "D:\anaconda\envs\333\lib\site-packages\fastapi\applications.py", line 270, in call
await super().call(scope, receive, send)
File "D:\anaconda\envs\333\lib\site-packages\starlette\applications.py", line 124, in call
await self.middleware_stack(scope, receive, send)
File "D:\anaconda\envs\333\lib\site-packages\starlette\middleware\errors.py", line 184, in call
raise exc
File "D:\anaconda\envs\333\lib\site-packages\starlette\middleware\errors.py", line 162, in call
await self.app(scope, receive, _send)
File "D:\anaconda\envs\333\lib\site-packages\starlette\middleware\cors.py", line 92, in call
await self.simple_response(scope, receive, send, request_headers=headers)
File "D:\anaconda\envs\333\lib\site-packages\starlette\middleware\cors.py", line 147, in simple_response
await self.app(scope, receive, send)
File "D:\anaconda\envs\333\lib\site-packages\starlette\middleware\exceptions.py", line 79, in call
raise exc
File "D:\anaconda\envs\333\lib\site-packages\starlette\middleware\exceptions.py", line 68, in call
await self.app(scope, receive, sender)
File "D:\anaconda\envs\333\lib\site-packages\fastapi\middleware\asyncexitstack.py", line 21, in call
raise e
File "D:\anaconda\envs\333\lib\site-packages\fastapi\middleware\asyncexitstack.py", line 18, in call
await self.app(scope, receive, send)
File "D:\anaconda\envs\333\lib\site-packages\starlette\routing.py", line 706, in call
await route.handle(scope, receive, send)
File "D:\anaconda\envs\333\lib\site-packages\starlette\routing.py", line 276, in handle
await self.app(scope, receive, send)
File "D:\anaconda\envs\333\lib\site-packages\starlette\routing.py", line 66, in app
response = await func(request)
File "D:\anaconda\envs\333\lib\site-packages\fastapi\routing.py", line 236, in app
dependant=dependant, values=values, is_coroutine=is_coroutine
File "D:\anaconda\envs\333\lib\site-packages\fastapi\routing.py", line 161, in run_endpoint_function
return await dependant.call(**values)
File "D:\z\code\back_end\controller\tag_controller.py", line 64, in hot
data = await update_hot_data(tag_task_id=tag_task_id, mongo_db=mongo_db)
File "D:\z\code\back_end\service\tag_hot_extract.py", line 23, in update_hot_data
tag = hot_find['tag']
KeyError: 'tag'
{"levelname": "ERROR", "process": 21920, "filename": "h11_impl.py", "funcName": "run_asgi", "lineno": 372, "time": "2023-08-12 14:39:50,092", "message": "Exception in ASGI application\nTraceback (most recent call last):\n File "D:\anaconda\envs\333\lib\site-packages\uvicorn\protocols\http\h11_impl.py", line 369, in run_asgi\n result = await app(self.scope, self.receive, self.send)\n File "D:\anaconda\envs\333\lib\site-packages\uvicorn\middleware\proxy_headers.py", line 59, in call\n return await self.app(scope, receive, send)\n File "D:\anaconda\envs\333\lib\site-packages\fastapi\applications.py", line 270, in call\n await super().call(scope, receive, send)\n File "D:\anaconda\envs\333\lib\site-packages\starlette\applications.py", line 124, in call\n await self.middleware_stack(scope, receive, send)\n File "D:\anaconda\envs\333\lib\site-packages\starlette\middleware\errors.py", line 184, in call\n raise exc\n File "D:\anaconda\envs\333\lib\site-packages\starlette\middleware\errors.py", line 162, in call\n await self.app(scope, receive, _send)\n File "D:\anaconda\envs\333\lib\site-packages\starlette\middleware\cors.py", line 92, in call\n await self.simple_response(scope, receive, send, request_headers=headers)\n File "D:\anaconda\envs\333\lib\site-packages\starlette\middleware\cors.py", line 147, in simple_response\n await self.app(scope, receive, send)\n File "D:\anaconda\envs\333\lib\site-packages\starlette\middleware\exceptions.py", line 79, in call\n raise exc\n File "D:\anaconda\envs\333\lib\site-packages\starlette\middleware\exceptions.py", line 68, in call\n await self.app(scope, receive, sender)\n File "D:\anaconda\envs\333\lib\site-packages\fastapi\middleware\asyncexitstack.py", line 21, in call\n raise e\n File "D:\anaconda\envs\333\lib\site-packages\fastapi\middleware\asyncexitstack.py", line 18, in call\n await self.app(scope, receive, send)\n File "D:\anaconda\envs\333\lib\site-packages\starlette\routing.py", line 706, in call\n await route.handle(scope, receive, send)\n File "D:\anaconda\envs\333\lib\site-packages\starlette\routing.py", line 276, in handle\n await self.app(scope, receive, send)\n File "D:\anaconda\envs\333\lib\site-packages\starlette\routing.py", line 66, in app\n response = await func(request)\n File "D:\anaconda\envs\333\lib\site-packages\fastapi\routing.py", line 236, in app\n dependant=dependant, values=values, is_coroutine=is_coroutine\n File "D:\anaconda\envs\333\lib\site-packages\fastapi\routing.py", line 161, in run_endpoint_function\n return await dependant.call(**values)\n File "D:\z\code\back_end\controller\tag_controller.py", line 64, in hot\n data = await update_hot_data(tag_task_id=tag_task_id, mongo_db=mongo_db)\n File "D:\z\code\back_end\service\tag_hot_extract.py", line 23, in update_hot_data\n tag = hot_find['tag']\nKeyError: 'tag'"}
ERROR: Exception in ASGI application
Traceback (most recent call last):
File "D:\anaconda\envs\333\lib\site-packages\uvicorn\protocols\http\h11_impl.py", line 369, in run_asgi
result = await app(self.scope, self.receive, self.send)
File "D:\anaconda\envs\333\lib\site-packages\uvicorn\middleware\proxy_headers.py", line 59, in call
return await self.app(scope, receive, send)
File "D:\anaconda\envs\333\lib\site-packages\fastapi\applications.py", line 270, in call
await super().call(scope, receive, send)
File "D:\anaconda\envs\333\lib\site-packages\starlette\applications.py", line 124, in call
await self.middleware_stack(scope, receive, send)
File "D:\anaconda\envs\333\lib\site-packages\starlette\middleware\errors.py", line 184, in call
raise exc
File "D:\anaconda\envs\333\lib\site-packages\starlette\middleware\errors.py", line 162, in call
await self.app(scope, receive, _send)
File "D:\anaconda\envs\333\lib\site-packages\starlette\middleware\cors.py", line 92, in call
await self.simple_response(scope, receive, send, request_headers=headers)
File "D:\anaconda\envs\333\lib\site-packages\starlette\middleware\cors.py", line 147, in simple_response
await self.app(scope, receive, send)
File "D:\anaconda\envs\333\lib\site-packages\starlette\middleware\exceptions.py", line 79, in call
raise exc
File "D:\anaconda\envs\333\lib\site-packages\starlette\middleware\exceptions.py", line 68, in call
await self.app(scope, receive, sender)
File "D:\anaconda\envs\333\lib\site-packages\fastapi\middleware\asyncexitstack.py", line 21, in call
raise e
File "D:\anaconda\envs\333\lib\site-packages\fastapi\middleware\asyncexitstack.py", line 18, in call
await self.app(scope, receive, send)
File "D:\anaconda\envs\333\lib\site-packages\starlette\routing.py", line 706, in call
await route.handle(scope, receive, send)
File "D:\anaconda\envs\333\lib\site-packages\starlette\routing.py", line 276, in handle
await self.app(scope, receive, send)
File "D:\anaconda\envs\333\lib\site-packages\starlette\routing.py", line 66, in app
response = await func(request)
File "D:\anaconda\envs\333\lib\site-packages\fastapi\routing.py", line 236, in app
dependant=dependant, values=values, is_coroutine=is_coroutine
File "D:\anaconda\envs\333\lib\site-packages\fastapi\routing.py", line 161, in run_endpoint_function
return await dependant.call(**values)
File "D:\z\code\back_end\controller\tag_controller.py", line 78, in relation_graph
relation_graph_result = await get_relation_graph(tag_task_id, mongo_db)
File "D:\z\code\back_end\service\tag_index_service.py", line 79, in get_relation_graph
for node in relation_graph_result['nodes_list']:
KeyError: 'nodes_list'
{"levelname": "ERROR", "process": 21920, "filename": "h11_impl.py", "funcName": "run_asgi", "lineno": 372, "time": "2023-08-12 14:39:50,105", "message": "Exception in ASGI application\nTraceback (most recent call last):\n File "D:\anaconda\envs\333\lib\site-packages\uvicorn\protocols\http\h11_impl.py", line 369, in run_asgi\n result = await app(self.scope, self.receive, self.send)\n File "D:\anaconda\envs\333\lib\site-packages\uvicorn\middleware\proxy_headers.py", line 59, in call\n return await self.app(scope, receive, send)\n File "D:\anaconda\envs\333\lib\site-packages\fastapi\applications.py", line 270, in call\n await super().call(scope, receive, send)\n File "D:\anaconda\envs\333\lib\site-packages\starlette\applications.py", line 124, in call\n await self.middleware_stack(scope, receive, send)\n File "D:\anaconda\envs\333\lib\site-packages\starlette\middleware\errors.py", line 184, in call\n raise exc\n File "D:\anaconda\envs\333\lib\site-packages\starlette\middleware\errors.py", line 162, in call\n await self.app(scope, receive, _send)\n File "D:\anaconda\envs\333\lib\site-packages\starlette\middleware\cors.py", line 92, in call\n await self.simple_response(scope, receive, send, request_headers=headers)\n File "D:\anaconda\envs\333\lib\site-packages\starlette\middleware\cors.py", line 147, in simple_response\n await self.app(scope, receive, send)\n File "D:\anaconda\envs\333\lib\site-packages\starlette\middleware\exceptions.py", line 79, in call\n raise exc\n File "D:\anaconda\envs\333\lib\site-packages\starlette\middleware\exceptions.py", line 68, in call\n await self.app(scope, receive, sender)\n File "D:\anaconda\envs\333\lib\site-packages\fastapi\middleware\asyncexitstack.py", line 21, in call\n raise e\n File "D:\anaconda\envs\333\lib\site-packages\fastapi\middleware\asyncexitstack.py", line 18, in call\n await self.app(scope, receive, send)\n File "D:\anaconda\envs\333\lib\site-packages\starlette\routing.py", line 706, in call\n await route.handle(scope, receive, send)\n File "D:\anaconda\envs\333\lib\site-packages\starlette\routing.py", line 276, in handle\n await self.app(scope, receive, send)\n File "D:\anaconda\envs\333\lib\site-packages\starlette\routing.py", line 66, in app\n response = await func(request)\n File "D:\anaconda\envs\333\lib\site-packages\fastapi\routing.py", line 236, in app\n dependant=dependant, values=values, is_coroutine=is_coroutine\n File "D:\anaconda\envs\333\lib\site-packages\fastapi\routing.py", line 161, in run_endpoint_function\n return await dependant.call(**values)\n File "D:\z\code\back_end\controller\tag_controller.py", line 78, in relation_graph\n relation_graph_result = await get_relation_graph(tag_task_id, mongo_db)\n File "D:\z\code\back_end\service\tag_index_service.py", line 79, in get_relation_graph\n for node in relation_graph_result['nodes_list']:\nKeyError: 'nodes_list'"}
INFO: 127.0.0.1:55635 - "GET /api/tag/relation_graph?tag_task_id=fa4817ff1e563273e3049e8169dda917 HTTP/1.1" 500 Internal Server Error

前端问题

我对前端不是很懂 大佬这个项目是vue2还是vue3啊

求数据库文件

大佬可以分享项目的数据库嘛，爬虫爬不了数据库没有数据

关于node.js

我想问一下 大佬的node.js是什么版本啊

前端搜索出错

WARNING:root:2023-04-07 15:51:31.822836 occur a exception IndexError:
('list index out of range',)

Traceback (most recent call last):
File "C:\Users\hl199\Topic_and_user_profile_analysis_system-master\code\weibo_crawler\selector_parser\search_weibo_parser.py", line 22, in parse_page
weibo_list = self._get_all_weibo()
File "C:\Users\hl199\Topic_and_user_profile_analysis_system-master\code\weibo_crawler\selector_parser\search_weibo_parser.py", line 33, in _get_all_weibo
for weibo in self._parse_weibo():
File "C:\Users\hl199\Topic_and_user_profile_analysis_system-master\code\weibo_crawler\selector_parser\search_weibo_parser.py", line 55, in _parse_weibo
weibo['weibo_id'] = sel.xpath('(.//p[@Class="from"])[last()]/a[1]/@href')[0].split('/')[-1].split('?')[0]
IndexError: list index out of range

WARNING:root:2023-04-07 15:52:25.597015 occur a exception IndexError:
('list index out of range',)
当在前端网页进行搜索时，爬虫页面显示如上所示。

INFO: 127.0.0.1:2065 - "GET /api/tag/tag_list HTTP/1.1" 500 Internal Server Error
ERROR: Exception in ASGI application
Traceback (most recent call last):
File "D:\anconda\envs\wcjgraduation\lib\site-packages\uvicorn\protocols\http\h11_impl.py", line 369, in run_asgi
result = await app(self.scope, self.receive, self.send)
File "D:\anconda\envs\wcjgraduation\lib\site-packages\uvicorn\middleware\proxy_headers.py", line 59, in call
return await self.app(scope, receive, send)
File "D:\anconda\envs\wcjgraduation\lib\site-packages\fastapi\applications.py", line 270, in call
await super().call(scope, receive, send)
File "D:\anconda\envs\wcjgraduation\lib\site-packages\starlette\applications.py", line 124, in call
await self.middleware_stack(scope, receive, send)
File "D:\anconda\envs\wcjgraduation\lib\site-packages\starlette\middleware\errors.py", line 184, in call
raise exc
File "D:\anconda\envs\wcjgraduation\lib\site-packages\starlette\middleware\errors.py", line 162, in call
await self.app(scope, receive, _send)
File "D:\anconda\envs\wcjgraduation\lib\site-packages\starlette\middleware\cors.py", line 92, in call
await self.simple_response(scope, receive, send, request_headers=headers)
File "D:\anconda\envs\wcjgraduation\lib\site-packages\starlette\middleware\cors.py", line 147, in simple_response
await self.app(scope, receive, send)
File "D:\anconda\envs\wcjgraduation\lib\site-packages\starlette\middleware\exceptions.py", line 79, in call
raise exc
File "D:\anconda\envs\wcjgraduation\lib\site-packages\starlette\middleware\exceptions.py", line 68, in call
await self.app(scope, receive, sender)
File "D:\anconda\envs\wcjgraduation\lib\site-packages\fastapi\middleware\asyncexitstack.py", line 21, in call
raise e
File "D:\anconda\envs\wcjgraduation\lib\site-packages\fastapi\middleware\asyncexitstack.py", line 18, in call
await self.app(scope, receive, send)
File "D:\anconda\envs\wcjgraduation\lib\site-packages\starlette\routing.py", line 706, in call
await route.handle(scope, receive, send)
File "D:\anconda\envs\wcjgraduation\lib\site-packages\starlette\routing.py", line 276, in handle
await self.app(scope, receive, send)
File "D:\anconda\envs\wcjgraduation\lib\site-packages\starlette\routing.py", line 66, in app
response = await func(request)
File "D:\anconda\envs\wcjgraduation\lib\site-packages\fastapi\routing.py", line 235, in app
raw_response = await run_endpoint_function(
File "D:\anconda\envs\wcjgraduation\lib\site-packages\fastapi\routing.py", line 161, in run_endpoint_function
return await dependant.call(**values)
File "C:\Users\hl199\Topic_and_user_profile_analysis_system-master\code\back_end\controller\tag_controller.py", line 33, in tag_list_get
tag_list = await get_tag_task_list(mongo_db)
File "C:\Users\hl199\Topic_and_user_profile_analysis_system-master\code\back_end\service\tag_index_service.py", line 18, in get_tag_task_list
for tag in await tag_list_find.to_list(length=100):
File "D:\anconda\envs\wcjgraduation\lib\site-packages\motor\core.py", line 1417, in _to_list
result = get_more_result.result()
File "D:\anconda\envs\wcjgraduation\lib\concurrent\futures\thread.py", line 57, in run
result = self.fn(*self.args, **self.kwargs)
File "D:\anconda\envs\wcjgraduation\lib\site-packages\pymongo\cursor.py", line 1130, in _refresh
self.__session = self.__collection.database.client._ensure_session()
File "D:\anconda\envs\wcjgraduation\lib\site-packages\pymongo\mongo_client.py", line 1935, in _ensure_session
return self.__start_session(True, causal_consistency=False)
File "D:\anconda\envs\wcjgraduation\lib\site-packages\pymongo\mongo_client.py", line 1883, in __start_session
server_session = self._get_server_session()
File "D:\anconda\envs\wcjgraduation\lib\site-packages\pymongo\mongo_client.py", line 1921, in _get_server_session
return self._topology.get_server_session()
File "D:\anconda\envs\wcjgraduation\lib\site-packages\pymongo\topology.py", line 520, in get_server_session
session_timeout = self._check_session_support()
File "D:\anconda\envs\wcjgraduation\lib\site-packages\pymongo\topology.py", line 499, in _check_session_support
self._select_servers_loop(
File "D:\anconda\envs\wcjgraduation\lib\site-packages\pymongo\topology.py", line 218, in _select_servers_loop
raise ServerSelectionTimeoutError(
pymongo.errors.ServerSelectionTimeoutError: 127.0.0.1:27017: [WinError 10061] 由于目标计算机积极拒绝，无法连接。, Timeout: 30s, Topology Description: <TopologyDescription id: 64302e32923055799f72a3ec, topology_type: Single, servers: [<ServerDescription ('127.0.0.1', 27017) server_type: Unknown, rtt: None, error=AutoReconnect('127.0.0.1:27017: [WinError 10061] 由于目标计算机积极拒绝，无法连接。')>]>
{"levelname": "ERROR", "process": 16900, "filename": "h11_impl.py", "funcName": "run_asgi", "lineno": 372, "time": "2023-04-07 15:53:04,216", "message": "Exception in ASGI application\nTraceback (most recent call last):\n File "D:\anconda\envs\wcjgraduation\lib\site-packages\uvicorn\protocols\http\h11_impl.py", line 369, in run_asgi\n result = await app(self.scope, self.receive, self.send)\n File "D:\anconda\envs\wcjgraduation\lib\site-packages\uvicorn\middleware\proxy_headers.py", line 59, in call\n return await self.app(scope, receive, send)\n File "D:\anconda\envs\wcjgraduation\lib\site-packages\fastapi\applications.py", line 270, in call\n await super().call(scope, receive, send)\n File "D:\anconda\envs\wcjgraduation\lib\site-packages\starlette\applications.py", line 124, in call\n await self.middleware_stack(scope, receive, send)\n File "D:\anconda\envs\wcjgraduation\lib\site-packages\starlette\middleware\errors.py", line 184, in call\n raise exc\n File "D:\anconda\envs\wcjgraduation\lib\site-packages\starlette\middleware\errors.py", line 162, in call\n await self.app(scope, receive, _send)\n File "D:\anconda\envs\wcjgraduation\lib\site-packages\starlette\middleware\cors.py", line 92, in call\n await self.simple_response(scope, receive, send, request_headers=headers)\n File "D:\anconda\envs\wcjgraduation\lib\site-packages\starlette\middleware\cors.py", line 147, in simple_response\n await self.app(scope, receive, send)\n File "D:\anconda\envs\wcjgraduation\lib\site-packages\starlette\middleware\exceptions.py", line 79, in call\n raise exc\n File "D:\anconda\envs\wcjgraduation\lib\site-packages\starlette\middleware\exceptions.py", line 68, in call\n await self.app(scope, receive, sender)\n File "D:\anconda\envs\wcjgraduation\lib\site-packages\fastapi\middleware\asyncexitstack.py", line 21, in call\n raise e\n File "D:\anconda\envs\wcjgraduation\lib\site-packages\fastapi\middleware\asyncexitstack.py", line 18, in call\n await self.app(scope, receive, send)\n File "D:\anconda\envs\wcjgraduation\lib\site-packages\starlette\routing.py", line 706, in call\n await route.handle(scope, receive, send)\n File "D:\anconda\envs\wcjgraduation\lib\site-packages\starlette\routing.py", line 276, in handle\n await self.app(scope, receive, send)\n File "D:\anconda\envs\wcjgraduation\lib\site-packages\starlette\routing.py", line 66, in app\n response = await func(request)\n File "D:\anconda\envs\wcjgraduation\lib\site-packages\fastapi\routing.py", line 235, in app\n raw_response = await run_endpoint_function(\n File "D:\anconda\envs\wcjgraduation\lib\site-packages\fastapi\routing.py", line 161, in run_endpoint_function\n return await dependant.call(**values)\n File "C:\Users\hl199\Topic_and_user_profile_analysis_system-master\code\back_end\controller\tag_controller.py", line 33, in tag_list_get\n tag_list = await get_tag_task_list(mongo_db)\n File "C:\Users\hl199\Topic_and_user_profile_analysis_system-master\code\back_end\service\tag_index_service.py", line 18, in get_tag_task_list\n for tag in await tag_list_find.to_list(length=100):\n File "D:\anconda\envs\wcjgraduation\lib\site-packages\motor\core.py", line 1417, in _to_list\n result = get_more_result.result()\n File "D:\anconda\envs\wcjgraduation\lib\concurrent\futures\thread.py", line 57, in run\n result = self.fn(*self.args, **self.kwargs)\n File "D:\anconda\envs\wcjgraduation\lib\site-packages\pymongo\cursor.py", line 1130, in _refresh\n self.__session = self.__collection.database.client._ensure_session()\n File "D:\anconda\envs\wcjgraduation\lib\site-packages\pymongo\mongo_client.py", line 1935, in _ensure_session\n return self.__start_session(True, causal_consistency=False)\n File "D:\anconda\envs\wcjgraduation\lib\site-packages\pymongo\mongo_client.py", line 1883, in __start_session\n server_session = self._get_server_session()\n File "D:\anconda\envs\wcjgraduation\lib\site-packages\pymongo\mongo_client.py", line 1921, in _get_server_session\n return self._topology.get_server_session()\n File "D:\anconda\envs\wcjgraduation\lib\site-packages\pymongo\topology.py", line 520, in get_server_session\n session_timeout = self._check_session_support()\n File "D:\anconda\envs\wcjgraduation\lib\site-packages\pymongo\topology.py", line 499, in _check_session_support\n self._select_servers_loop(\n File "D:\anconda\envs\wcjgraduation\lib\site-packages\pymongo\topology.py", line 218, in _select_servers_loop\n raise ServerSelectionTimeoutError(\npymongo.errors.ServerSelectionTimeoutError: 127.0.0.1:27017: [WinError 10061] 由于目标计算机积极拒绝，无法连接。, Timeout: 30s, Topology Description: <TopologyDescription id: 64302e32923055799f72a3ec, topology_type: Single, servers: [<ServerDescription ('127.0.0.1', 27017) server_type: Unknown, rtt: None, error=AutoReconnect('127.0.0.1:27017: [WinError 10061] 由于目标计算机积极拒绝，无法连接。')>]>"}
main.py
页面报错如上所示。

不知道如何操作....

话题分析过程出错：KeyError: 'hot_count'

数据爬取正常，在分析的过程中，报错如下：

[2023-02-28 20:54:29,371: ERROR/MainProcess] Task celery_task.worker.task_schedule[f252c77a-183e-4b50-ac0e-3f138b6b8d31] raised unexpected: KeyError('hot_count')
Traceback (most recent call last):
File "G:\Anaconda\envs\zs\lib\site-packages\celery\app\trace.py", line 240, in trace_task
R = retval = fun(*args, **kwargs)
File "G:\Anaconda\envs\zs\lib\site-packages\celery\app\trace.py", line 438, in protected_call
return self.run(*args, **kwargs)
File "G:\code\Topic_and_user_profile_analysis_system-master\code\back_end\celery_task\worker.py", line 35, in task_schedule
weibo_data, weibo_post_list, user_id_list = spider(tag, tag_task_id)
File "G:\code\Topic_and_user_profile_analysis_system-master\code\back_end\celery_task\tag_task\tag_spider_task.py", line 52, in spider
result_data_list.sort(key=lambda x: int(x['hot_count']), reverse=True)
File "G:\code\Topic_and_user_profile_analysis_system-master\code\back_end\celery_task\tag_task\tag_spider_task.py", line 52, in
result_data_list.sort(key=lambda x: int(x['hot_count']), reverse=True)
KeyError: 'hot_count'
[2023-02-28 20:54:29,373: ERROR/MainProcess] Task celery_task.worker.task_schedule[e7d16c18-b9ec-40bd-97d3-b573fad0f27c] raised unexpected: KeyError('hot_count')
Traceback (most recent call last):
File "G:\Anaconda\envs\zs\lib\site-packages\celery\app\trace.py", line 240, in trace_task
R = retval = fun(*args, **kwargs)
File "G:\Anaconda\envs\zs\lib\site-packages\celery\app\trace.py", line 438, in protected_call
return self.run(*args, **kwargs)
File "G:\code\Topic_and_user_profile_analysis_system-master\code\back_end\celery_task\worker.py", line 35, in task_schedule
weibo_data, weibo_post_list, user_id_list = spider(tag, tag_task_id)
File "G:\code\Topic_and_user_profile_analysis_system-master\code\back_end\celery_task\tag_task\tag_spider_task.py", line 52, in spider
result_data_list.sort(key=lambda x: int(x['hot_count']), reverse=True)
File "G:\code\Topic_and_user_profile_analysis_system-master\code\back_end\celery_task\tag_task\tag_spider_task.py", line 52, in
result_data_list.sort(key=lambda x: int(x['hot_count']), reverse=True)
KeyError: 'hot_count'

后端启动错误AttributeError: type object 'MongoClient' has no attribute 'options'

Traceback (most recent call last):
File "D:/Topic_and_user_profile_analysis_system-master/code/back_end/main.py", line 9, in
import app
File "D:\Topic_and_user_profile_analysis_system-master\code\back_end\app.py", line 8, in
from controller import api_router
File "D:\Topic_and_user_profile_analysis_system-master\code\back_end\controller_init_.py", line 4, in
from .api_router import api_router
File "D:\Topic_and_user_profile_analysis_system-master\code\back_end\controller\api_router.py", line 3, in
from . import(
File "D:\Topic_and_user_profile_analysis_system-master\code\back_end\controller\tag_test_controller.py", line 2, in
from motor.motor_asyncio import AsyncIOMotorDatabase
File "D:\graduation\venv\lib\site-packages\motor_init_.py", line 35, in
from .motor_tornado import * # noqa: F403
File "D:\graduation\venv\lib\site-packages\motor\motor_tornado.py", line 41, in
MotorClient = create_motor_class(core.AgnosticClient)
File "D:\graduation\venv\lib\site-packages\motor\motor_tornado.py", line 38, in create_motor_class
return create_class_with_framework(cls, tornado_framework, "motor.motor_tornado")
File "D:\graduation\venv\lib\site-packages\motor\metaprogramming.py", line 288, in create_class_with_framework
new_class_attr = attr.create_attribute(new_class, name)
File "D:\graduation\venv\lib\site-packages\motor\metaprogramming.py", line 204, in create_attribute
doc = getattr(cls.delegate_class, attr_name).doc
AttributeError: type object 'MongoClient' has no attribute 'options'

不知道啥原因，请求大佬帮忙万分感谢

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.