Giter Club home page Giter Club logo

web_kg's Introduction

开源web知识图谱项目

  • 爬取百度百科中文页面
  • 解析三元组和网页内容
  • 构建中文知识图谱
  • 构建百科bot（构建中）
update 20200720

Windows上的部署参考如何在Windows上部署，感谢LMY-nlp0701！

update 20191121
  • 迁移代码到爬虫框架scrapy
  • 优化了抽取部分代码
  • 数据持久化迁移到mongodb
  • 修复chatbot失效问题
  • 开放neo4j后台界面，可以查看知识图谱成型效果
Tips
  • 如果是项目问题，请提issue。
  • 如果涉及到不方便公开的，请发邮件。
  • ChatBot请访问链接
  • 成型的百科知识图谱访问链接，用户名：neo4j,密码：123。效果如下：

环境

  • python 3.6
  • re:url正则匹配
  • scrapy:网页爬虫和网页解析
  • neo4j:知识图谱图数据库,安装可以参考链接
  • pip install neo4j-driver：neo4j python驱动
  • pip install pymongodb：mongodb的python支持
  • mongodb数据库：安装参考链接

代码执行：

cd WEB_KG/baike
scrapy crawl baike

执行界面(按ctrl+c停止)：

知识图谱效果图

mongodb存储的网页内容

mongodb存储的三元组

neo4j后台界面

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.