gerapy / gerapyautoextractor Goto Github PK

View Code? Open in Web Editor NEW

312.0 312.0 75.0 724 KB

Auto Extractor Module

Home Page: https://pypi.org/project/gerapy-auto-extractor/

License: Apache License 2.0

Python 100.00%

gerapyautoextractor's Introduction

Gerapy

Distributed Crawler Management Framework Based on Scrapy, Scrapyd, Scrapyd-Client, Scrapyd-API, Django and Vue.js.

Documentation

Documentation is available online at https://docs.gerapy.com/ and https://github.com/Gerapy/Docs.

Support

Gerapy is developed based on Python 3.x. Python 2.x may be supported later.

Usage

Install Gerapy by pip:

pip3 install gerapy

After the installation, you need to do these things below to run Gerapy server:

If you have installed Gerapy successfully, you can use command gerapy. If not, check the installation.

First use this command to initialize the workspace:

gerapy init

Now you will get a folder named gerapy. Also you can specify the name of your workspace by this command:

gerapy init <workspace>

Then cd to this folder, and run this command to initialize the Database:

cd gerapy
gerapy migrate

Next you need to create a superuser by this command:

gerapy createsuperuser

Then you can runserver by this command:

gerapy runserver

Then you can visit http://localhost:8000 to enjoy it. Also you can vist http://localhost:8000/admin to get the admin management backend.

If you want to run Gerapy in public, just run like this:

gerapy runserver 0.0.0.0:8000

Then it will run with public host and port 8000.

In Gerapy, You can create a configurable project and then configure and generate code of Scrapy automatically. But this module is unstable, we're trying to refine it.

Also you can drag your Scrapy Project to projects folder. Then refresh web, it will appear in the Project Index Page and comes to un-configurable, but you can edit this project through the web page.

As for deployment, you can move to Deploy Page. Firstly you need to build your project and add client in the Client Index Page, then you can deploy the project just by clicking button.

After the deployment, you can manage the job in Monitor Page.

Docker

Just run this command:

docker-compose up

Then it will run at port 8000. You can use the temp admin account (username: admin, password: admin) to login. And please change the password later for safety.

Command Usage:

docker run -d -v <workspace>:/home/gerapy -p <public_port>:<container_port> germey/gerapy

Please specify your workspace to mount Gerapy workspace by -v <workspace>:/app/gerapy and specify server port by -p <public_port>:<container_port>.

If you run Gerapy by Docker, you can visit Gerapy website such as http://localhost:8000 and enjoy it, no need to do other initialzation things.

TodoList

Add Visual Configuration of Spider with Previewing Website
Add Scrapyd Auth Management
Add Gerapy Auth Management
Add Timed Task Scheduler
Add Visual Configuration of Scrapy
Add Intelligent Analysis of Web Page

Communication

If you have any questions or ideas, you can send Issues or Pull Requests, your suggestions are really import for us, thanks for your contirbution.

gerapyautoextractor's People

Contributors

Stargazers

Watchers

gerapyautoextractor's Issues

报错了AttributeError: 'backports.zoneinfo.ZoneInfo' object has no attribute 'localize'

path  = "logs/生成过内容的记录表单.csv" # 文件保存路径
WebSite = "sohu.com" # 网站主页链接
Url = "https://www.sohu.com/a/419892530_603537" # 具体内容链接
Title = extract_detail(requests.get(Url).text)["title"] # 文章的标题

报错了 AttributeError: 'backports.zoneinfo.ZoneInfo' object has no attribute 'localize'

计算titlle 相似性

这里是不是写错了呀？
交集除以并集计算相似性？

How to distinguish whether the page is a list page or a detail page?

函数preprocess4content_extractor的bug

函数preprocess4content_extractor中的
for child in children(element):
只是遍历了子，而不是遍历所有节点，是否应该改为
for descendant in element.iterdescendants():

Bug of Gerapy Auto Extractor 爬取论坛帖子时候出错

爬取的链接是：https://www.19lou.com/forum-269-1.html
使用extract_list获取到的数据是：
[
{
"title": "19楼帮帮团维权月来啦！7月维权主题汽车类",
"url": "http://www.19lou.com/forum-79-thread-42261592790646553-1-1.html"
},
{
"title": "19楼帮帮团来咯，求助维权攻略请收下！",
"url": "http://www.19lou.com/forum-79-thread-82281589267909116-1-1.html"
},
{
"title": "【19楼帮帮团】每日诈骗连载！少点套路，多点幸福",
"url": "http://www.19lou.com/forum-79-thread-82681592968362354-1-1.html"
},
{
"title": "杭州人杭州事，你要知道的都在19楼",
"url": "http://www.19lou.com/forum-269-thread-63421567731405299-1-1.html"
},
{
"title": "楼外楼：杭州事【总版规】（本版不支持一切形式广告）",
"url": "http://www.19lou.com/forum-269-thread-31532348-1-1.html"
}
]

使用extract_detail获取的是：
{
"title": "",
"datetime": "2020-07-12 00:55:56",
"content": "浙公网安备 33010002000029号"
}

没有一个是想要的数据，想要的是帖子标题加链接加帖子正文。

max() arg is an empty sequence

Describe the bug
使用is_list函数判断内容是否属于列表页面,得到true后, 调用extract_list函数,发生了错误:
ValueError: max() arg is an empty sequence

To Reproduce

Content of your page html: https://www.qukuaiwang.com.cn/Index/ku_detail/kid/4812.html
Contnet of Your code:

import requests
from gerapy_auto_extractor import extract_list, extract_detail, is_detail, is_list

content = requests.get("https://www.qukuaiwang.com.cn/Index/ku_detail/kid/4812.html").text
if is_list(content):
    print(extract_list(content))

See error

Traceback (most recent call last):
  File "C:/Users/qinya/PycharmProjects/x7Nlp/1.py", line 7, in <module>
    print(extract_list(content))
  File "C:\Users\qinya\Anaconda3\envs\synonyms\lib\site-packages\gerapy_auto_extractor\extractors\list.py", line 246, in extract_list
    return list_extractor.extract(html, **kwargs)
  File "C:\Users\qinya\Anaconda3\envs\synonyms\lib\site-packages\gerapy_auto_extractor\extractors\base.py", line 45, in extract
    return self.process(element)
  File "C:\Users\qinya\Anaconda3\envs\synonyms\lib\site-packages\gerapy_auto_extractor\extractors\list.py", line 234, in process
    return self._extract_cluster(best_cluster)
  File "C:\Users\qinya\Anaconda3\envs\synonyms\lib\site-packages\gerapy_auto_extractor\extractors\list.py", line 187, in _extract_cluster
    best_path = max(probabilities_of_title_avg.items(), key=operator.itemgetter(1))[0]
ValueError: max() arg is an empty sequence

Expected behavior
该页面应该属于内容页面,但是使用is_list却返回true,调用extract_list函数应该返回空

Screenshots
页面截图:

Desktop (please complete the following information):

OS: win10
Python 3.7
Package Version :0.1.0

Wrong extract example

Some title were not extracted successfully.

只提取出了一部分，有一些标题没有提取出来。

can't remove element

此函数不起作用。
def remove_element(element: Element):
"""
remove child element from parent
:param element:
:return:
"""
if element is None:
return
p = element.getparent()
if p is not None:
p.remove(element)

Bug of Gerapy Auto Extractor 安装时出现问题

错误代码如下：
之前运行好好的，总是卡在，buliding wheels这里，我试了三台电脑，都是同样的问题。
能不能解释一下错误的原因和解决方案？stackoverflow上面说用conda安装就能解决，我试过依然不行。
非常感谢。
Building wheels for collected packages: lxml, numpy
Building wheel for lxml (setup.py) ... error
ERROR: Command errored out with exit status 1:
command: 'C:\Users\SinoCBD\AppData\Local\Programs\Python\Python38\python.exe' -u -c 'import io, os, sys, setuptools, tokenize; sys.argv[0] = '"'"'C:\Users\SinoCBD\AppData\Local\Temp\pip-install-gvkelvan\lxml_0b1625f8d4d64eccacc995673e930279\setup.py'"'"'; file='"'"'C:\Users\SinoCBD\AppData\Local\Temp\pip-install-gvkelvan\lxml_0b1625f8d4d64eccacc995673e930279\setup.py'"'"';f = getattr(tokenize, '"'"'open'"'"', open)(file) if os.path.exists(file) else io.StringIO('"'"'from setuptools import setup; setup()'"'"');code = f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, file, '"'"'exec'"'"'))' bdist_wheel -d 'C:\Users\SinoCBD\AppData\Local\Temp\pip-wheel-iu5h936t'
cwd: C:\Users\SinoCBD\AppData\Local\Temp\pip-install-gvkelvan\lxml_0b1625f8d4d64eccacc995673e930279
Complete output (97 lines):
Building lxml version 4.3.3.
Building without Cython.
ERROR: b"'xslt-config' \xb2\xbb\xca\xc7\xc4\xda\xb2\xbf\xbb\xf2\xcd\xe2\xb2\xbf\xc3\xfc\xc1\xee\xa3\xac\xd2\xb2\xb2\xbb\xca\xc7\xbf\xc9\xd4\xcb\xd0\xd0\xb5\xc4\xb3\xcc\xd0\xf2\r\n\xbb\xf2\xc5\xfa\xb4\xa6\xc0\xed\xce\xc4\xbc\xfe\xa1\xa3\r\n"
** make sure the development packages of libxml2 and libxslt are installed **

Using build configuration of libxslt
running bdist_wheel
running build
running build_py
creating build
creating build\lib.win-amd64-3.8
creating build\lib.win-amd64-3.8\lxml
copying src\lxml\builder.py -> build\lib.win-amd64-3.8\lxml
copying src\lxml\cssselect.py -> build\lib.win-amd64-3.8\lxml
copying src\lxml\doctestcompare.py -> build\lib.win-amd64-3.8\lxml
copying src\lxml\ElementInclude.py -> build\lib.win-amd64-3.8\lxml
copying src\lxml\pyclasslookup.py -> build\lib.win-amd64-3.8\lxml
copying src\lxml\sax.py -> build\lib.win-amd64-3.8\lxml
copying src\lxml\usedoctest.py -> build\lib.win-amd64-3.8\lxml
copying src\lxml_elementpath.py -> build\lib.win-amd64-3.8\lxml
copying src\lxml_init_.py -> build\lib.win-amd64-3.8\lxml
creating build\lib.win-amd64-3.8\lxml\includes
copying src\lxml\includes_init_.py -> build\lib.win-amd64-3.8\lxml\includes
creating build\lib.win-amd64-3.8\lxml\html
copying src\lxml\html\builder.py -> build\lib.win-amd64-3.8\lxml\html
copying src\lxml\html\clean.py -> build\lib.win-amd64-3.8\lxml\html
copying src\lxml\html\defs.py -> build\lib.win-amd64-3.8\lxml\html
copying src\lxml\html\diff.py -> build\lib.win-amd64-3.8\lxml\html
copying src\lxml\html\ElementSoup.py -> build\lib.win-amd64-3.8\lxml\html
copying src\lxml\html\formfill.py -> build\lib.win-amd64-3.8\lxml\html
copying src\lxml\html\html5parser.py -> build\lib.win-amd64-3.8\lxml\html
copying src\lxml\html\soupparser.py -> build\lib.win-amd64-3.8\lxml\html
copying src\lxml\html\usedoctest.py -> build\lib.win-amd64-3.8\lxml\html
copying src\lxml\html_diffcommand.py -> build\lib.win-amd64-3.8\lxml\html
copying src\lxml\html_html5builder.py -> build\lib.win-amd64-3.8\lxml\html
copying src\lxml\html_setmixin.py -> build\lib.win-amd64-3.8\lxml\html
copying src\lxml\html_init_.py -> build\lib.win-amd64-3.8\lxml\html
creating build\lib.win-amd64-3.8\lxml\isoschematron
copying src\lxml\isoschematron_init_.py -> build\lib.win-amd64-3.8\lxml\isoschematron
copying src\lxml\etree.h -> build\lib.win-amd64-3.8\lxml
copying src\lxml\etree_api.h -> build\lib.win-amd64-3.8\lxml
copying src\lxml\lxml.etree.h -> build\lib.win-amd64-3.8\lxml
copying src\lxml\lxml.etree_api.h -> build\lib.win-amd64-3.8\lxml
copying src\lxml\includes\c14n.pxd -> build\lib.win-amd64-3.8\lxml\includes
copying src\lxml\includes\config.pxd -> build\lib.win-amd64-3.8\lxml\includes
copying src\lxml\includes\dtdvalid.pxd -> build\lib.win-amd64-3.8\lxml\includes
copying src\lxml\includes\etreepublic.pxd -> build\lib.win-amd64-3.8\lxml\includes
copying src\lxml\includes\htmlparser.pxd -> build\lib.win-amd64-3.8\lxml\includes
copying src\lxml\includes\relaxng.pxd -> build\lib.win-amd64-3.8\lxml\includes
copying src\lxml\includes\schematron.pxd -> build\lib.win-amd64-3.8\lxml\includes
copying src\lxml\includes\tree.pxd -> build\lib.win-amd64-3.8\lxml\includes
copying src\lxml\includes\uri.pxd -> build\lib.win-amd64-3.8\lxml\includes
copying src\lxml\includes\xinclude.pxd -> build\lib.win-amd64-3.8\lxml\includes
copying src\lxml\includes\xmlerror.pxd -> build\lib.win-amd64-3.8\lxml\includes
copying src\lxml\includes\xmlparser.pxd -> build\lib.win-amd64-3.8\lxml\includes
copying src\lxml\includes\xmlschema.pxd -> build\lib.win-amd64-3.8\lxml\includes
copying src\lxml\includes\xpath.pxd -> build\lib.win-amd64-3.8\lxml\includes
copying src\lxml\includes\xslt.pxd -> build\lib.win-amd64-3.8\lxml\includes
copying src\lxml\includes_init_.pxd -> build\lib.win-amd64-3.8\lxml\includes
copying src\lxml\includes\etree_defs.h -> build\lib.win-amd64-3.8\lxml\includes
copying src\lxml\includes\lxml-version.h -> build\lib.win-amd64-3.8\lxml\includes
creating build\lib.win-amd64-3.8\lxml\isoschematron\resources
creating build\lib.win-amd64-3.8\lxml\isoschematron\resources\rng
copying src\lxml\isoschematron\resources\rng\iso-schematron.rng -> build\lib.win-amd64-3.8\lxml\isoschematron\resources\rng
creating build\lib.win-amd64-3.8\lxml\isoschematron\resources\xsl
copying src\lxml\isoschematron\resources\xsl\RNG2Schtrn.xsl -> build\lib.win-amd64-3.8\lxml\isoschematron\resources\xsl
copying src\lxml\isoschematron\resources\xsl\XSD2Schtrn.xsl -> build\lib.win-amd64-3.8\lxml\isoschematron\resources\xsl
creating build\lib.win-amd64-3.8\lxml\isoschematron\resources\xsl\iso-schematron-xslt1
copying src\lxml\isoschematron\resources\xsl\iso-schematron-xslt1\iso_abstract_expand.xsl -> build\lib.win-amd64-3.8\lxml\isoschematron\resources\xsl\iso-schematron-xslt1
copying src\lxml\isoschematron\resources\xsl\iso-schematron-xslt1\iso_dsdl_include.xsl -> build\lib.win-amd64-3.8\lxml\isoschematron\resources\xsl\iso-schematron-xslt1
copying src\lxml\isoschematron\resources\xsl\iso-schematron-xslt1\iso_schematron_message.xsl -> build\lib.win-amd64-3.8\lxml\isoschematron\resources\xsl\iso-schematron-xslt1
copying src\lxml\isoschematron\resources\xsl\iso-schematron-xslt1\iso_schematron_skeleton_for_xslt1.xsl -> build\lib.win-amd64-3.8\lxml\isoschematron\resources\xsl\iso-schematron-xslt1
copying src\lxml\isoschematron\resources\xsl\iso-schematron-xslt1\iso_svrl_for_xslt1.xsl -> build\lib.win-amd64-3.8\lxml\isoschematron\resources\xsl\iso-schematron-xslt1
copying src\lxml\isoschematron\resources\xsl\iso-schematron-xslt1\readme.txt -> build\lib.win-amd64-3.8\lxml\isoschematron\resources\xsl\iso-schematron-xslt1
running build_ext
building 'lxml.etree' extension
creating build\temp.win-amd64-3.8
creating build\temp.win-amd64-3.8\Release
creating build\temp.win-amd64-3.8\Release\src
creating build\temp.win-amd64-3.8\Release\src\lxml
C:\Program Files (x86)\Microsoft Visual Studio\2019\BuildTools\VC\Tools\MSVC\14.29.30133\bin\HostX86\x64\cl.exe /c /nologo /Ox /W3 /GL /DNDEBUG /MD -DCYTHON_CLINE_IN_TRACEBACK=0 -Isrc -Isrc\lxml\includes -IC:\Users\SinoCBD\AppData\Local\Programs\Python\Python38\include -IC:\Users\SinoCBD\AppData\Local\Programs\Python\Python38\include "-IC:\Program Files (x86)\Microsoft Visual Studio\2019\BuildTools\VC\Tools\MSVC\14.29.30133\include" /Tcsrc\lxml\etree.c /Fobuild\temp.win-amd64-3.8\Release\src\lxml\etree.obj -w
cl: 命令行 warning D9025 :正在重写“/W3”(用“/w”)
etree.c
C:\Users\SinoCBD\AppData\Local\Programs\Python\Python38\include\pyconfig.h(59): fatal error C1083: 无法打开包括文件: “io.h”: No such file or directory
Compile failed: command 'C:\Program Files (x86)\Microsoft Visual Studio\2019\BuildTools\VC\Tools\MSVC\14.29.30133\bin\HostX86\x64\cl.exe' failed with exit status 2
creating Users
creating Users\SinoCBD
creating Users\SinoCBD\AppData
creating Users\SinoCBD\AppData\Local
creating Users\SinoCBD\AppData\Local\Temp
C:\Program Files (x86)\Microsoft Visual Studio\2019\BuildTools\VC\Tools\MSVC\14.29.30133\bin\HostX86\x64\cl.exe /c /nologo /Ox /W3 /GL /DNDEBUG /MD -I/usr/include/libxml2 "-IC:\Program Files (x86)\Microsoft Visual Studio\2019\BuildTools\VC\Tools\MSVC\14.29.30133\include" /TcC:\Users\SinoCBD\AppData\Local\Temp\xmlXPathInita_eu5hb1.c /FoUsers\SinoCBD\AppData\Local\Temp\xmlXPathInita_eu5hb1.obj
xmlXPathInita_eu5hb1.c
C:\Users\SinoCBD\AppData\Local\Temp\xmlXPathInita_eu5hb1.c(1): fatal error C1083: 无法打开包括文件: “libxml/xpath.h”: No such file or directory
error: command 'C:\Program Files (x86)\Microsoft Visual Studio\2019\BuildTools\VC\Tools\MSVC\14.29.30133\bin\HostX86\x64\cl.exe' failed with exit status 2

Could not find function xmlCheckVersion in library libxml2. Is libxml2 installed?

ERROR: Failed building wheel for lxml
Running setup.py clean for lxml
Building wheel for numpy (setup.py) ... error
ERROR: Command errored out with exit status 1:
command: 'C:\Users\SinoCBD\AppData\Local\Programs\Python\Python38\python.exe' -u -c 'import io, os, sys, setuptools, tokenize; sys.argv[0] = '"'"'C:\Users\SinoCBD\AppData\Local\Temp\pip-install-gvkelvan\numpy_32d9c41759e64c63a571fd9253149241\setup.py'"'"'; file='"'"'C:\Users\SinoCBD\AppData\Local\Temp\pip-install-gvkelvan\numpy_32d9c41759e64c63a571fd9253149241\setup.py'"'"';f = getattr(tokenize, '"'"'open'"'"', open)(file) if os.path.exists(file) else io.StringIO('"'"'from setuptools import setup; setup()'"'"');code = f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, file, '"'"'exec'"'"'))' bdist_wheel -d 'C:\Users\SinoCBD\AppData\Local\Temp\pip-wheel-vxyalp67'
cwd: C:\Users\SinoCBD\AppData\Local\Temp\pip-install-gvkelvan\numpy_32d9c41759e64c63a571fd9253149241
Complete output (309 lines):
Running from numpy source directory.
C:\Users\SinoCBD\AppData\Local\Temp\pip-install-gvkelvan\numpy_32d9c41759e64c63a571fd9253149241\numpy\distutils\misc_util.py:476: SyntaxWarning: "is" with a literal. Did you mean "=="?
return is_string(s) and ('*' in s or '?' is s)
blas_opt_info:
blas_mkl_info:
No module named 'numpy.distutils._msvccompiler' in numpy.distutils; trying from distutils
customize MSVCCompiler
libraries mkl_rt not found in ['C:\Users\SinoCBD\AppData\Local\Programs\Python\Python38\lib', 'C:\', 'C:\Users\SinoCBD\AppData\Local\Programs\Python\Python38\libs']
NOT AVAILABLE

blis_info:
No module named 'numpy.distutils._msvccompiler' in numpy.distutils; trying from distutils
customize MSVCCompiler
libraries blis not found in ['C:\Users\SinoCBD\AppData\Local\Programs\Python\Python38\lib', 'C:\', 'C:\Users\SinoCBD\AppData\Local\Programs\Python\Python38\libs']
NOT AVAILABLE

openblas_info:
No module named 'numpy.distutils._msvccompiler' in numpy.distutils; trying from distutils
customize MSVCCompiler
No module named 'numpy.distutils._msvccompiler' in numpy.distutils; trying from distutils
customize MSVCCompiler
libraries openblas not found in ['C:\Users\SinoCBD\AppData\Local\Programs\Python\Python38\lib', 'C:\', 'C:\Users\SinoCBD\AppData\Local\Programs\Python\Python38\libs']
get_default_fcompiler: matching types: '['gnu', 'intelv', 'absoft', 'compaqv', 'intelev', 'gnu95', 'g95', 'intelvem', 'intelem', 'flang']'
customize GnuFCompiler
Could not locate executable g77
Could not locate executable f77
customize IntelVisualFCompiler
Could not locate executable ifort
Could not locate executable ifl
customize AbsoftFCompiler
Could not locate executable f90
customize CompaqVisualFCompiler
Could not locate executable DF
customize IntelItaniumVisualFCompiler
Could not locate executable efl
customize Gnu95FCompiler
Could not locate executable gfortran
Could not locate executable f95
customize G95FCompiler
Could not locate executable g95
customize IntelEM64VisualFCompiler
customize IntelEM64TFCompiler
Could not locate executable efort
Could not locate executable efc
customize PGroupFlangCompiler
Could not locate executable flang
don't know how to compile Fortran code on platform 'nt'
NOT AVAILABLE

atlas_3_10_blas_threads_info:
Setting PTATLAS=ATLAS
No module named 'numpy.distutils._msvccompiler' in numpy.distutils; trying from distutils
customize MSVCCompiler
libraries tatlas not found in ['C:\Users\SinoCBD\AppData\Local\Programs\Python\Python38\lib', 'C:\', 'C:\Users\SinoCBD\AppData\Local\Programs\Python\Python38\libs']
NOT AVAILABLE

atlas_3_10_blas_info:
No module named 'numpy.distutils._msvccompiler' in numpy.distutils; trying from distutils
customize MSVCCompiler
libraries satlas not found in ['C:\Users\SinoCBD\AppData\Local\Programs\Python\Python38\lib', 'C:\', 'C:\Users\SinoCBD\AppData\Local\Programs\Python\Python38\libs']
NOT AVAILABLE

atlas_blas_threads_info:
Setting PTATLAS=ATLAS
No module named 'numpy.distutils._msvccompiler' in numpy.distutils; trying from distutils
customize MSVCCompiler
libraries ptf77blas,ptcblas,atlas not found in ['C:\Users\SinoCBD\AppData\Local\Programs\Python\Python38\lib', 'C:\', 'C:\Users\SinoCBD\AppData\Local\Programs\Python\Python38\libs']
NOT AVAILABLE

atlas_blas_info:
No module named 'numpy.distutils._msvccompiler' in numpy.distutils; trying from distutils
customize MSVCCompiler
libraries f77blas,cblas,atlas not found in ['C:\Users\SinoCBD\AppData\Local\Programs\Python\Python38\lib', 'C:\', 'C:\Users\SinoCBD\AppData\Local\Programs\Python\Python38\libs']
NOT AVAILABLE

accelerate_info:
NOT AVAILABLE

C:\Users\SinoCBD\AppData\Local\Temp\pip-install-gvkelvan\numpy_32d9c41759e64c63a571fd9253149241\numpy\distutils\system_info.py:638: UserWarning:
Atlas (http://math-atlas.sourceforge.net/) libraries not found.
Directories to search for the libraries can be specified in the
numpy/distutils/site.cfg file (section [atlas]) or by setting
the ATLAS environment variable.
self.calc_info()
blas_info:
No module named 'numpy.distutils._msvccompiler' in numpy.distutils; trying from distutils
customize MSVCCompiler
libraries blas not found in ['C:\Users\SinoCBD\AppData\Local\Programs\Python\Python38\lib', 'C:\', 'C:\Users\SinoCBD\AppData\Local\Programs\Python\Python38\libs']
NOT AVAILABLE

C:\Users\SinoCBD\AppData\Local\Temp\pip-install-gvkelvan\numpy_32d9c41759e64c63a571fd9253149241\numpy\distutils\system_info.py:638: UserWarning:
Blas (http://www.netlib.org/blas/) libraries not found.
Directories to search for the libraries can be specified in the
numpy/distutils/site.cfg file (section [blas]) or by setting
the BLAS environment variable.
self.calc_info()
blas_src_info:
NOT AVAILABLE

C:\Users\SinoCBD\AppData\Local\Temp\pip-install-gvkelvan\numpy_32d9c41759e64c63a571fd9253149241\numpy\distutils\system_info.py:638: UserWarning:
Blas (http://www.netlib.org/blas/) sources not found.
Directories to search for the sources can be specified in the
numpy/distutils/site.cfg file (section [blas_src]) or by setting
the BLAS_SRC environment variable.
self.calc_info()
NOT AVAILABLE

'svnversion' 不是内部或外部命令，也不是可运行的程序
或批处理文件。
non-existing path in 'numpy\distutils': 'site.cfg'
lapack_opt_info:
lapack_mkl_info:
No module named 'numpy.distutils._msvccompiler' in numpy.distutils; trying from distutils
customize MSVCCompiler
libraries mkl_rt not found in ['C:\Users\SinoCBD\AppData\Local\Programs\Python\Python38\lib', 'C:\', 'C:\Users\SinoCBD\AppData\Local\Programs\Python\Python38\libs']
NOT AVAILABLE

openblas_lapack_info:
No module named 'numpy.distutils._msvccompiler' in numpy.distutils; trying from distutils
customize MSVCCompiler
No module named 'numpy.distutils._msvccompiler' in numpy.distutils; trying from distutils
customize MSVCCompiler
libraries openblas not found in ['C:\Users\SinoCBD\AppData\Local\Programs\Python\Python38\lib', 'C:\', 'C:\Users\SinoCBD\AppData\Local\Programs\Python\Python38\libs']
NOT AVAILABLE

openblas_clapack_info:
No module named 'numpy.distutils._msvccompiler' in numpy.distutils; trying from distutils
customize MSVCCompiler
No module named 'numpy.distutils._msvccompiler' in numpy.distutils; trying from distutils
customize MSVCCompiler
libraries openblas,lapack not found in ['C:\Users\SinoCBD\AppData\Local\Programs\Python\Python38\lib', 'C:\', 'C:\Users\SinoCBD\AppData\Local\Programs\Python\Python38\libs']
NOT AVAILABLE

atlas_3_10_threads_info:
Setting PTATLAS=ATLAS
No module named 'numpy.distutils._msvccompiler' in numpy.distutils; trying from distutils
customize MSVCCompiler
libraries lapack_atlas not found in C:\Users\SinoCBD\AppData\Local\Programs\Python\Python38\lib
No module named 'numpy.distutils._msvccompiler' in numpy.distutils; trying from distutils
customize MSVCCompiler
libraries tatlas,tatlas not found in C:\Users\SinoCBD\AppData\Local\Programs\Python\Python38\lib
No module named 'numpy.distutils._msvccompiler' in numpy.distutils; trying from distutils
customize MSVCCompiler
libraries lapack_atlas not found in C:
No module named 'numpy.distutils._msvccompiler' in numpy.distutils; trying from distutils
customize MSVCCompiler
libraries tatlas,tatlas not found in C:
No module named 'numpy.distutils._msvccompiler' in numpy.distutils; trying from distutils
customize MSVCCompiler
libraries lapack_atlas not found in C:\Users\SinoCBD\AppData\Local\Programs\Python\Python38\libs
No module named 'numpy.distutils._msvccompiler' in numpy.distutils; trying from distutils
customize MSVCCompiler
libraries tatlas,tatlas not found in C:\Users\SinoCBD\AppData\Local\Programs\Python\Python38\libs
<class 'numpy.distutils.system_info.atlas_3_10_threads_info'>
NOT AVAILABLE

atlas_3_10_info:
No module named 'numpy.distutils._msvccompiler' in numpy.distutils; trying from distutils
customize MSVCCompiler
libraries lapack_atlas not found in C:\Users\SinoCBD\AppData\Local\Programs\Python\Python38\lib
No module named 'numpy.distutils._msvccompiler' in numpy.distutils; trying from distutils
customize MSVCCompiler
libraries satlas,satlas not found in C:\Users\SinoCBD\AppData\Local\Programs\Python\Python38\lib
No module named 'numpy.distutils._msvccompiler' in numpy.distutils; trying from distutils
customize MSVCCompiler
libraries lapack_atlas not found in C:
No module named 'numpy.distutils._msvccompiler' in numpy.distutils; trying from distutils
customize MSVCCompiler
libraries satlas,satlas not found in C:
No module named 'numpy.distutils._msvccompiler' in numpy.distutils; trying from distutils
customize MSVCCompiler
libraries lapack_atlas not found in C:\Users\SinoCBD\AppData\Local\Programs\Python\Python38\libs
No module named 'numpy.distutils._msvccompiler' in numpy.distutils; trying from distutils
customize MSVCCompiler
libraries satlas,satlas not found in C:\Users\SinoCBD\AppData\Local\Programs\Python\Python38\libs
<class 'numpy.distutils.system_info.atlas_3_10_info'>
NOT AVAILABLE

atlas_threads_info:
Setting PTATLAS=ATLAS
No module named 'numpy.distutils._msvccompiler' in numpy.distutils; trying from distutils
customize MSVCCompiler
libraries lapack_atlas not found in C:\Users\SinoCBD\AppData\Local\Programs\Python\Python38\lib
No module named 'numpy.distutils._msvccompiler' in numpy.distutils; trying from distutils
customize MSVCCompiler
libraries ptf77blas,ptcblas,atlas not found in C:\Users\SinoCBD\AppData\Local\Programs\Python\Python38\lib
No module named 'numpy.distutils._msvccompiler' in numpy.distutils; trying from distutils
customize MSVCCompiler
libraries lapack_atlas not found in C:
No module named 'numpy.distutils._msvccompiler' in numpy.distutils; trying from distutils
customize MSVCCompiler
libraries ptf77blas,ptcblas,atlas not found in C:
No module named 'numpy.distutils._msvccompiler' in numpy.distutils; trying from distutils
customize MSVCCompiler
libraries lapack_atlas not found in C:\Users\SinoCBD\AppData\Local\Programs\Python\Python38\libs
No module named 'numpy.distutils._msvccompiler' in numpy.distutils; trying from distutils
customize MSVCCompiler
libraries ptf77blas,ptcblas,atlas not found in C:\Users\SinoCBD\AppData\Local\Programs\Python\Python38\libs
<class 'numpy.distutils.system_info.atlas_threads_info'>
NOT AVAILABLE

atlas_info:
No module named 'numpy.distutils._msvccompiler' in numpy.distutils; trying from distutils
customize MSVCCompiler
libraries lapack_atlas not found in C:\Users\SinoCBD\AppData\Local\Programs\Python\Python38\lib
No module named 'numpy.distutils._msvccompiler' in numpy.distutils; trying from distutils
customize MSVCCompiler
libraries f77blas,cblas,atlas not found in C:\Users\SinoCBD\AppData\Local\Programs\Python\Python38\lib
No module named 'numpy.distutils._msvccompiler' in numpy.distutils; trying from distutils
customize MSVCCompiler
libraries lapack_atlas not found in C:
No module named 'numpy.distutils._msvccompiler' in numpy.distutils; trying from distutils
customize MSVCCompiler
libraries f77blas,cblas,atlas not found in C:
No module named 'numpy.distutils._msvccompiler' in numpy.distutils; trying from distutils
customize MSVCCompiler
libraries lapack_atlas not found in C:\Users\SinoCBD\AppData\Local\Programs\Python\Python38\libs
No module named 'numpy.distutils._msvccompiler' in numpy.distutils; trying from distutils
customize MSVCCompiler
libraries f77blas,cblas,atlas not found in C:\Users\SinoCBD\AppData\Local\Programs\Python\Python38\libs
<class 'numpy.distutils.system_info.atlas_info'>
NOT AVAILABLE

lapack_info:
No module named 'numpy.distutils._msvccompiler' in numpy.distutils; trying from distutils
customize MSVCCompiler
libraries lapack not found in ['C:\Users\SinoCBD\AppData\Local\Programs\Python\Python38\lib', 'C:\', 'C:\Users\SinoCBD\AppData\Local\Programs\Python\Python38\libs']
NOT AVAILABLE

C:\Users\SinoCBD\AppData\Local\Temp\pip-install-gvkelvan\numpy_32d9c41759e64c63a571fd9253149241\numpy\distutils\system_info.py:638: UserWarning:
Lapack (http://www.netlib.org/lapack/) libraries not found.
Directories to search for the libraries can be specified in the
numpy/distutils/site.cfg file (section [lapack]) or by setting
the LAPACK environment variable.
self.calc_info()
lapack_src_info:
NOT AVAILABLE

C:\Users\SinoCBD\AppData\Local\Temp\pip-install-gvkelvan\numpy_32d9c41759e64c63a571fd9253149241\numpy\distutils\system_info.py:638: UserWarning:
Lapack (http://www.netlib.org/lapack/) sources not found.
Directories to search for the sources can be specified in the
numpy/distutils/site.cfg file (section [lapack_src]) or by setting
the LAPACK_SRC environment variable.
self.calc_info()
NOT AVAILABLE

C:\Users\SinoCBD\AppData\Local\Programs\Python\Python38\lib\distutils\dist.py:274: UserWarning: Unknown distribution option: 'define_macros'
warnings.warn(msg)
running bdist_wheel
running build
running config_cc
unifing config_cc, config, build_clib, build_ext, build commands --compiler options
running config_fc
unifing config_fc, config, build_clib, build_ext, build commands --fcompiler options
running build_src
build_src
building py_modules sources
creating build
creating build\src.win-amd64-3.8
creating build\src.win-amd64-3.8\numpy
creating build\src.win-amd64-3.8\numpy\distutils
building library "npymath" sources
No module named 'numpy.distutils._msvccompiler' in numpy.distutils; trying from distutils
C:\Program Files (x86)\Microsoft Visual Studio\2019\BuildTools\VC\Tools\MSVC\14.29.30133\bin\HostX86\x64\cl.exe /c /nologo /Ox /W3 /GL /DNDEBUG /MD -Inumpy\core\src\common -Inumpy\core\src -Inumpy\core -Inumpy\core\src\npymath -Inumpy\core\src\multiarray -Inumpy\core\src\umath -Inumpy\core\src\npysort -IC:\Users\SinoCBD\AppData\Local\Programs\Python\Python38\include -IC:\Users\SinoCBD\AppData\Local\Programs\Python\Python38\include -IC:\Program Files (x86)\Microsoft Visual Studio\2019\BuildTools\VC\Tools\MSVC\14.29.30133\include /Tc_configtest.c /Fo_configtest.obj
C:\Program Files (x86)\Microsoft Visual Studio\2019\BuildTools\VC\Tools\MSVC\14.29.30133\bin\HostX86\x64\link.exe /nologo /INCREMENTAL:NO /LTCG /MANIFEST:EMBED,ID=1 /LIBPATH:C:\Program Files (x86)\Microsoft Visual Studio\2019\BuildTools\VC\Tools\MSVC\14.29.30133\lib\x64 _configtest.obj /OUT:_configtest.exe
LINK : fatal error LNK1104: 无法打开文件“kernel32.lib”
failure.
removing: _configtest.c _configtest.obj configtest.obj.d
Traceback (most recent call last):
File "", line 1, in
File "C:\Users\SinoCBD\AppData\Local\Temp\pip-install-gvkelvan\numpy_32d9c41759e64c63a571fd9253149241\setup.py", line 415, in
setup_package()
File "C:\Users\SinoCBD\AppData\Local\Temp\pip-install-gvkelvan\numpy_32d9c41759e64c63a571fd9253149241\setup.py", line 407, in setup_package
setup(metadata)
File "C:\Users\SinoCBD\AppData\Local\Temp\pip-install-gvkelvan\numpy_32d9c41759e64c63a571fd9253149241\numpy\distutils\core.py", line 171, in setup
return old_setup(new_attr)
File "C:\Users\SinoCBD\AppData\Local\Programs\Python\Python38\lib\site-packages\setuptools_init.py", line 153, in setup
return distutils.core.setup(**attrs)
File "C:\Users\SinoCBD\AppData\Local\Programs\Python\Python38\lib\distutils\core.py", line 148, in setup
dist.run_commands()
File "C:\Users\SinoCBD\AppData\Local\Programs\Python\Python38\lib\distutils\dist.py", line 966, in run_commands
self.run_command(cmd)
File "C:\Users\SinoCBD\AppData\Local\Programs\Python\Python38\lib\distutils\dist.py", line 985, in run_command
cmd_obj.run()
File "C:\Users\SinoCBD\AppData\Local\Programs\Python\Python38\lib\site-packages\wheel\bdist_wheel.py", line 299, in run
self.run_command('build')
File "C:\Users\SinoCBD\AppData\Local\Programs\Python\Python38\lib\distutils\cmd.py", line 313, in run_command
self.distribution.run_command(command)
File "C:\Users\SinoCBD\AppData\Local\Programs\Python\Python38\lib\distutils\dist.py", line 985, in run_command
cmd_obj.run()
File "C:\Users\SinoCBD\AppData\Local\Temp\pip-install-gvkelvan\numpy_32d9c41759e64c63a571fd9253149241\numpy\distutils\command\build.py", line 47, in run
old_build.run(self)
File "C:\Users\SinoCBD\AppData\Local\Programs\Python\Python38\lib\distutils\command\build.py", line 135, in run
self.run_command(cmd_name)
File "C:\Users\SinoCBD\AppData\Local\Programs\Python\Python38\lib\distutils\cmd.py", line 313, in run_command
self.distribution.run_command(command)
File "C:\Users\SinoCBD\AppData\Local\Programs\Python\Python38\lib\distutils\dist.py", line 985, in run_command
cmd_obj.run()
File "C:\Users\SinoCBD\AppData\Local\Temp\pip-install-gvkelvan\numpy_32d9c41759e64c63a571fd9253149241\numpy\distutils\command\build_src.py", line 148, in run
self.build_sources()
File "C:\Users\SinoCBD\AppData\Local\Temp\pip-install-gvkelvan\numpy_32d9c41759e64c63a571fd9253149241\numpy\distutils\command\build_src.py", line 159, in build_sources
self.build_library_sources(*libname_info)
File "C:\Users\SinoCBD\AppData\Local\Temp\pip-install-gvkelvan\numpy_32d9c41759e64c63a571fd9253149241\numpy\distutils\command\build_src.py", line 292, in build_library_sources
sources = self.generate_sources(sources, (lib_name, build_info))
File "C:\Users\SinoCBD\AppData\Local\Temp\pip-install-gvkelvan\numpy_32d9c41759e64c63a571fd9253149241\numpy\distutils\command\build_src.py", line 375, in generate_sources
source = func(extension, build_dir)
File "numpy\core\setup.py", line 667, in get_mathlib_info
raise RuntimeError("Broken toolchain: cannot link a simple C program")
RuntimeError: Broken toolchain: cannot link a simple C program

ERROR: Failed building wheel for numpy
Running setup.py clean for numpy
ERROR: Command errored out with exit status 1:
command: 'C:\Users\SinoCBD\AppData\Local\Programs\Python\Python38\python.exe' -u -c 'import io, os, sys, setuptools, tokenize; sys.argv[0] = '"'"'C:\Users\SinoCBD\AppData\Local\Temp\pip-install-gvkelvan\numpy_32d9c41759e64c63a571fd9253149241\setup.py'"'"'; file='"'"'C:\Users\SinoCBD\AppData\Local\Temp\pip-install-gvkelvan\numpy_32d9c41759e64c63a571fd9253149241\setup.py'"'"';f = getattr(tokenize, '"'"'open'"'"', open)(file) if os.path.exists(file) else io.StringIO('"'"'from setuptools import setup; setup()'"'"');code = f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, file, '"'"'exec'"'"'))' clean --all
cwd: C:\Users\SinoCBD\AppData\Local\Temp\pip-install-gvkelvan\numpy_32d9c41759e64c63a571fd9253149241
Complete output (10 lines):
Running from numpy source directory.

setup.py clean is not supported, use one of the following instead:

- `git clean -xdf` (cleans all files)
- `git clean -Xdf` (cleans all versioned files, doesn't touch
                    files that aren't checked into the git repo)

Add --force to your command to use it anyway if you must (unsupported).

ERROR: Failed cleaning build dir for numpy
Failed to build lxml numpy
WARNING: Ignoring invalid distribution -umpy (c:\users\sinocbd\appdata\local\programs\python\python38\lib\site-packages)
WARNING: Ignoring invalid distribution -pype1 (c:\users\sinocbd\appdata\local\programs\python\python38\lib\site-packages)
Installing collected packages: numpy, scipy, joblib, scikit-learn, lxml, loguru, environs, dateparser, gerapy-auto-extractor
Attempting uninstall: numpy
WARNING: Ignoring invalid distribution -umpy (c:\users\sinocbd\appdata\local\programs\python\python38\lib\site-packages)
WARNING: Ignoring invalid distribution -pype1 (c:\users\sinocbd\appdata\local\programs\python\python38\lib\site-packages)
Found existing installation: numpy 1.21.2
Uninstalling numpy-1.21.2:
Successfully uninstalled numpy-1.21.2
Running setup.py install for numpy ... error
ERROR: Command errored out with exit status 1:
command: 'C:\Users\SinoCBD\AppData\Local\Programs\Python\Python38\python.exe' -u -c 'import io, os, sys, setuptools, tokenize; sys.argv[0] = '"'"'C:\Users\SinoCBD\AppData\Local\Temp\pip-install-gvkelvan\numpy_32d9c41759e64c63a571fd9253149241\setup.py'"'"'; file='"'"'C:\Users\SinoCBD\AppData\Local\Temp\pip-install-gvkelvan\numpy_32d9c41759e64c63a571fd9253149241\setup.py'"'"';f = getattr(tokenize, '"'"'open'"'"', open)(file) if os.path.exists(file) else io.StringIO('"'"'from setuptools import setup; setup()'"'"');code = f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, file, '"'"'exec'"'"'))' install --record 'C:\Users\SinoCBD\AppData\Local\Temp\pip-record-z7pop_qe\install-record.txt' --single-version-externally-managed --compile --install-headers 'C:\Users\SinoCBD\AppData\Local\Programs\Python\Python38\Include\numpy'
cwd: C:\Users\SinoCBD\AppData\Local\Temp\pip-install-gvkelvan\numpy_32d9c41759e64c63a571fd9253149241
Complete output (316 lines):
Running from numpy source directory.

Note: if you need reliable uninstall behavior, then install
with pip instead of using `setup.py install`:

  - `pip install .`       (from a git repo or downloaded source
                           release)
  - `pip install numpy`   (last NumPy release on PyPi)


blas_opt_info:
blas_mkl_info:
No module named 'numpy.distutils._msvccompiler' in numpy.distutils; trying from distutils
customize MSVCCompiler
  libraries mkl_rt not found in ['C:\\Users\\SinoCBD\\AppData\\Local\\Programs\\Python\\Python38\\lib', 'C:\\', 'C:\\Users\\SinoCBD\\AppData\\Local\\Programs\\Python\\Python38\\libs']
  NOT AVAILABLE

blis_info:
No module named 'numpy.distutils._msvccompiler' in numpy.distutils; trying from distutils
customize MSVCCompiler
  libraries blis not found in ['C:\\Users\\SinoCBD\\AppData\\Local\\Programs\\Python\\Python38\\lib', 'C:\\', 'C:\\Users\\SinoCBD\\AppData\\Local\\Programs\\Python\\Python38\\libs']
  NOT AVAILABLE

openblas_info:
No module named 'numpy.distutils._msvccompiler' in numpy.distutils; trying from distutils
customize MSVCCompiler
No module named 'numpy.distutils._msvccompiler' in numpy.distutils; trying from distutils
customize MSVCCompiler
  libraries openblas not found in ['C:\\Users\\SinoCBD\\AppData\\Local\\Programs\\Python\\Python38\\lib', 'C:\\', 'C:\\Users\\SinoCBD\\AppData\\Local\\Programs\\Python\\Python38\\libs']
get_default_fcompiler: matching types: '['gnu', 'intelv', 'absoft', 'compaqv', 'intelev', 'gnu95', 'g95', 'intelvem', 'intelem', 'flang']'
customize GnuFCompiler
Could not locate executable g77
Could not locate executable f77
customize IntelVisualFCompiler
Could not locate executable ifort
Could not locate executable ifl
customize AbsoftFCompiler
Could not locate executable f90
customize CompaqVisualFCompiler
Could not locate executable DF
customize IntelItaniumVisualFCompiler
Could not locate executable efl
customize Gnu95FCompiler
Could not locate executable gfortran
Could not locate executable f95
customize G95FCompiler
Could not locate executable g95
customize IntelEM64VisualFCompiler
customize IntelEM64TFCompiler
Could not locate executable efort
Could not locate executable efc
customize PGroupFlangCompiler
Could not locate executable flang
don't know how to compile Fortran code on platform 'nt'
  NOT AVAILABLE

atlas_3_10_blas_threads_info:
Setting PTATLAS=ATLAS
No module named 'numpy.distutils._msvccompiler' in numpy.distutils; trying from distutils
customize MSVCCompiler
  libraries tatlas not found in ['C:\\Users\\SinoCBD\\AppData\\Local\\Programs\\Python\\Python38\\lib', 'C:\\', 'C:\\Users\\SinoCBD\\AppData\\Local\\Programs\\Python\\Python38\\libs']
  NOT AVAILABLE

atlas_3_10_blas_info:
No module named 'numpy.distutils._msvccompiler' in numpy.distutils; trying from distutils
customize MSVCCompiler
  libraries satlas not found in ['C:\\Users\\SinoCBD\\AppData\\Local\\Programs\\Python\\Python38\\lib', 'C:\\', 'C:\\Users\\SinoCBD\\AppData\\Local\\Programs\\Python\\Python38\\libs']
  NOT AVAILABLE

atlas_blas_threads_info:
Setting PTATLAS=ATLAS
No module named 'numpy.distutils._msvccompiler' in numpy.distutils; trying from distutils
customize MSVCCompiler
  libraries ptf77blas,ptcblas,atlas not found in ['C:\\Users\\SinoCBD\\AppData\\Local\\Programs\\Python\\Python38\\lib', 'C:\\', 'C:\\Users\\SinoCBD\\AppData\\Local\\Programs\\Python\\Python38\\libs']
  NOT AVAILABLE

atlas_blas_info:
No module named 'numpy.distutils._msvccompiler' in numpy.distutils; trying from distutils
customize MSVCCompiler
  libraries f77blas,cblas,atlas not found in ['C:\\Users\\SinoCBD\\AppData\\Local\\Programs\\Python\\Python38\\lib', 'C:\\', 'C:\\Users\\SinoCBD\\AppData\\Local\\Programs\\Python\\Python38\\libs']
  NOT AVAILABLE

accelerate_info:
  NOT AVAILABLE

C:\Users\SinoCBD\AppData\Local\Temp\pip-install-gvkelvan\numpy_32d9c41759e64c63a571fd9253149241\numpy\distutils\system_info.py:638: UserWarning:
    Atlas (http://math-atlas.sourceforge.net/) libraries not found.
    Directories to search for the libraries can be specified in the
    numpy/distutils/site.cfg file (section [atlas]) or by setting
    the ATLAS environment variable.
  self.calc_info()
blas_info:
No module named 'numpy.distutils._msvccompiler' in numpy.distutils; trying from distutils
customize MSVCCompiler
  libraries blas not found in ['C:\\Users\\SinoCBD\\AppData\\Local\\Programs\\Python\\Python38\\lib', 'C:\\', 'C:\\Users\\SinoCBD\\AppData\\Local\\Programs\\Python\\Python38\\libs']
  NOT AVAILABLE

C:\Users\SinoCBD\AppData\Local\Temp\pip-install-gvkelvan\numpy_32d9c41759e64c63a571fd9253149241\numpy\distutils\system_info.py:638: UserWarning:
    Blas (http://www.netlib.org/blas/) libraries not found.
    Directories to search for the libraries can be specified in the
    numpy/distutils/site.cfg file (section [blas]) or by setting
    the BLAS environment variable.
  self.calc_info()
blas_src_info:
  NOT AVAILABLE

C:\Users\SinoCBD\AppData\Local\Temp\pip-install-gvkelvan\numpy_32d9c41759e64c63a571fd9253149241\numpy\distutils\system_info.py:638: UserWarning:
    Blas (http://www.netlib.org/blas/) sources not found.
    Directories to search for the sources can be specified in the
    numpy/distutils/site.cfg file (section [blas_src]) or by setting
    the BLAS_SRC environment variable.
  self.calc_info()
  NOT AVAILABLE

'svnversion' 不是内部或外部命令，也不是可运行的程序
或批处理文件。
non-existing path in 'numpy\\distutils': 'site.cfg'
lapack_opt_info:
lapack_mkl_info:
No module named 'numpy.distutils._msvccompiler' in numpy.distutils; trying from distutils
customize MSVCCompiler
  libraries mkl_rt not found in ['C:\\Users\\SinoCBD\\AppData\\Local\\Programs\\Python\\Python38\\lib', 'C:\\', 'C:\\Users\\SinoCBD\\AppData\\Local\\Programs\\Python\\Python38\\libs']
  NOT AVAILABLE

openblas_lapack_info:
No module named 'numpy.distutils._msvccompiler' in numpy.distutils; trying from distutils
customize MSVCCompiler
No module named 'numpy.distutils._msvccompiler' in numpy.distutils; trying from distutils
customize MSVCCompiler
  libraries openblas not found in ['C:\\Users\\SinoCBD\\AppData\\Local\\Programs\\Python\\Python38\\lib', 'C:\\', 'C:\\Users\\SinoCBD\\AppData\\Local\\Programs\\Python\\Python38\\libs']
  NOT AVAILABLE

openblas_clapack_info:
No module named 'numpy.distutils._msvccompiler' in numpy.distutils; trying from distutils
customize MSVCCompiler
No module named 'numpy.distutils._msvccompiler' in numpy.distutils; trying from distutils
customize MSVCCompiler
  libraries openblas,lapack not found in ['C:\\Users\\SinoCBD\\AppData\\Local\\Programs\\Python\\Python38\\lib', 'C:\\', 'C:\\Users\\SinoCBD\\AppData\\Local\\Programs\\Python\\Python38\\libs']
  NOT AVAILABLE

atlas_3_10_threads_info:
Setting PTATLAS=ATLAS
No module named 'numpy.distutils._msvccompiler' in numpy.distutils; trying from distutils
customize MSVCCompiler
  libraries lapack_atlas not found in C:\Users\SinoCBD\AppData\Local\Programs\Python\Python38\lib
No module named 'numpy.distutils._msvccompiler' in numpy.distutils; trying from distutils
customize MSVCCompiler
  libraries tatlas,tatlas not found in C:\Users\SinoCBD\AppData\Local\Programs\Python\Python38\lib
No module named 'numpy.distutils._msvccompiler' in numpy.distutils; trying from distutils
customize MSVCCompiler
  libraries lapack_atlas not found in C:\
No module named 'numpy.distutils._msvccompiler' in numpy.distutils; trying from distutils
customize MSVCCompiler
  libraries tatlas,tatlas not found in C:\
No module named 'numpy.distutils._msvccompiler' in numpy.distutils; trying from distutils
customize MSVCCompiler
  libraries lapack_atlas not found in C:\Users\SinoCBD\AppData\Local\Programs\Python\Python38\libs
No module named 'numpy.distutils._msvccompiler' in numpy.distutils; trying from distutils
customize MSVCCompiler
  libraries tatlas,tatlas not found in C:\Users\SinoCBD\AppData\Local\Programs\Python\Python38\libs
<class 'numpy.distutils.system_info.atlas_3_10_threads_info'>
  NOT AVAILABLE

atlas_3_10_info:
No module named 'numpy.distutils._msvccompiler' in numpy.distutils; trying from distutils
customize MSVCCompiler
  libraries lapack_atlas not found in C:\Users\SinoCBD\AppData\Local\Programs\Python\Python38\lib
No module named 'numpy.distutils._msvccompiler' in numpy.distutils; trying from distutils
customize MSVCCompiler
  libraries satlas,satlas not found in C:\Users\SinoCBD\AppData\Local\Programs\Python\Python38\lib
No module named 'numpy.distutils._msvccompiler' in numpy.distutils; trying from distutils
customize MSVCCompiler
  libraries lapack_atlas not found in C:\
No module named 'numpy.distutils._msvccompiler' in numpy.distutils; trying from distutils
customize MSVCCompiler
  libraries satlas,satlas not found in C:\
No module named 'numpy.distutils._msvccompiler' in numpy.distutils; trying from distutils
customize MSVCCompiler
  libraries lapack_atlas not found in C:\Users\SinoCBD\AppData\Local\Programs\Python\Python38\libs
No module named 'numpy.distutils._msvccompiler' in numpy.distutils; trying from distutils
customize MSVCCompiler
  libraries satlas,satlas not found in C:\Users\SinoCBD\AppData\Local\Programs\Python\Python38\libs
<class 'numpy.distutils.system_info.atlas_3_10_info'>
  NOT AVAILABLE

atlas_threads_info:
Setting PTATLAS=ATLAS
No module named 'numpy.distutils._msvccompiler' in numpy.distutils; trying from distutils
customize MSVCCompiler
  libraries lapack_atlas not found in C:\Users\SinoCBD\AppData\Local\Programs\Python\Python38\lib
No module named 'numpy.distutils._msvccompiler' in numpy.distutils; trying from distutils
customize MSVCCompiler
  libraries ptf77blas,ptcblas,atlas not found in C:\Users\SinoCBD\AppData\Local\Programs\Python\Python38\lib
No module named 'numpy.distutils._msvccompiler' in numpy.distutils; trying from distutils
customize MSVCCompiler
  libraries lapack_atlas not found in C:\
No module named 'numpy.distutils._msvccompiler' in numpy.distutils; trying from distutils
customize MSVCCompiler
  libraries ptf77blas,ptcblas,atlas not found in C:\
No module named 'numpy.distutils._msvccompiler' in numpy.distutils; trying from distutils
customize MSVCCompiler
  libraries lapack_atlas not found in C:\Users\SinoCBD\AppData\Local\Programs\Python\Python38\libs
No module named 'numpy.distutils._msvccompiler' in numpy.distutils; trying from distutils
customize MSVCCompiler
  libraries ptf77blas,ptcblas,atlas not found in C:\Users\SinoCBD\AppData\Local\Programs\Python\Python38\libs
<class 'numpy.distutils.system_info.atlas_threads_info'>
  NOT AVAILABLE

atlas_info:
No module named 'numpy.distutils._msvccompiler' in numpy.distutils; trying from distutils
customize MSVCCompiler
  libraries lapack_atlas not found in C:\Users\SinoCBD\AppData\Local\Programs\Python\Python38\lib
No module named 'numpy.distutils._msvccompiler' in numpy.distutils; trying from distutils
customize MSVCCompiler
  libraries f77blas,cblas,atlas not found in C:\Users\SinoCBD\AppData\Local\Programs\Python\Python38\lib
No module named 'numpy.distutils._msvccompiler' in numpy.distutils; trying from distutils
customize MSVCCompiler
  libraries lapack_atlas not found in C:\
No module named 'numpy.distutils._msvccompiler' in numpy.distutils; trying from distutils
customize MSVCCompiler
  libraries f77blas,cblas,atlas not found in C:\
No module named 'numpy.distutils._msvccompiler' in numpy.distutils; trying from distutils
customize MSVCCompiler
  libraries lapack_atlas not found in C:\Users\SinoCBD\AppData\Local\Programs\Python\Python38\libs
No module named 'numpy.distutils._msvccompiler' in numpy.distutils; trying from distutils
customize MSVCCompiler
  libraries f77blas,cblas,atlas not found in C:\Users\SinoCBD\AppData\Local\Programs\Python\Python38\libs
<class 'numpy.distutils.system_info.atlas_info'>
  NOT AVAILABLE

lapack_info:
No module named 'numpy.distutils._msvccompiler' in numpy.distutils; trying from distutils
customize MSVCCompiler
  libraries lapack not found in ['C:\\Users\\SinoCBD\\AppData\\Local\\Programs\\Python\\Python38\\lib', 'C:\\', 'C:\\Users\\SinoCBD\\AppData\\Local\\Programs\\Python\\Python38\\libs']
  NOT AVAILABLE

C:\Users\SinoCBD\AppData\Local\Temp\pip-install-gvkelvan\numpy_32d9c41759e64c63a571fd9253149241\numpy\distutils\system_info.py:638: UserWarning:
    Lapack (http://www.netlib.org/lapack/) libraries not found.
    Directories to search for the libraries can be specified in the
    numpy/distutils/site.cfg file (section [lapack]) or by setting
    the LAPACK environment variable.
  self.calc_info()
lapack_src_info:
  NOT AVAILABLE

C:\Users\SinoCBD\AppData\Local\Temp\pip-install-gvkelvan\numpy_32d9c41759e64c63a571fd9253149241\numpy\distutils\system_info.py:638: UserWarning:
    Lapack (http://www.netlib.org/lapack/) sources not found.
    Directories to search for the sources can be specified in the
    numpy/distutils/site.cfg file (section [lapack_src]) or by setting
    the LAPACK_SRC environment variable.
  self.calc_info()
  NOT AVAILABLE

C:\Users\SinoCBD\AppData\Local\Programs\Python\Python38\lib\distutils\dist.py:274: UserWarning: Unknown distribution option: 'define_macros'
  warnings.warn(msg)
running install
running build
running config_cc
unifing config_cc, config, build_clib, build_ext, build commands --compiler options
running config_fc
unifing config_fc, config, build_clib, build_ext, build commands --fcompiler options
running build_src
build_src
building py_modules sources
building library "npymath" sources
No module named 'numpy.distutils._msvccompiler' in numpy.distutils; trying from distutils
C:\Program Files (x86)\Microsoft Visual Studio\2019\BuildTools\VC\Tools\MSVC\14.29.30133\bin\HostX86\x64\cl.exe /c /nologo /Ox /W3 /GL /DNDEBUG /MD -Inumpy\core\src\common -Inumpy\core\src -Inumpy\core -Inumpy\core\src\npymath -Inumpy\core\src\multiarray -Inumpy\core\src\umath -Inumpy\core\src\npysort -IC:\Users\SinoCBD\AppData\Local\Programs\Python\Python38\include -IC:\Users\SinoCBD\AppData\Local\Programs\Python\Python38\include -IC:\Program Files (x86)\Microsoft Visual Studio\2019\BuildTools\VC\Tools\MSVC\14.29.30133\include /Tc_configtest.c /Fo_configtest.obj
C:\Program Files (x86)\Microsoft Visual Studio\2019\BuildTools\VC\Tools\MSVC\14.29.30133\bin\HostX86\x64\link.exe /nologo /INCREMENTAL:NO /LTCG /MANIFEST:EMBED,ID=1 /LIBPATH:C:\Program Files (x86)\Microsoft Visual Studio\2019\BuildTools\VC\Tools\MSVC\14.29.30133\lib\x64 _configtest.obj /OUT:_configtest.exe
LINK : fatal error LNK1104: 无法打开文件“kernel32.lib”
failure.
removing: _configtest.c _configtest.obj _configtest.obj.d
Traceback (most recent call last):
  File "<string>", line 1, in <module>
  File "C:\Users\SinoCBD\AppData\Local\Temp\pip-install-gvkelvan\numpy_32d9c41759e64c63a571fd9253149241\setup.py", line 415, in <module>
    setup_package()
  File "C:\Users\SinoCBD\AppData\Local\Temp\pip-install-gvkelvan\numpy_32d9c41759e64c63a571fd9253149241\setup.py", line 407, in setup_package
    setup(**metadata)
  File "C:\Users\SinoCBD\AppData\Local\Temp\pip-install-gvkelvan\numpy_32d9c41759e64c63a571fd9253149241\numpy\distutils\core.py", line 171, in setup
    return old_setup(**new_attr)
  File "C:\Users\SinoCBD\AppData\Local\Programs\Python\Python38\lib\site-packages\setuptools\__init__.py", line 153, in setup
    return distutils.core.setup(**attrs)
  File "C:\Users\SinoCBD\AppData\Local\Programs\Python\Python38\lib\distutils\core.py", line 148, in setup
    dist.run_commands()
  File "C:\Users\SinoCBD\AppData\Local\Programs\Python\Python38\lib\distutils\dist.py", line 966, in run_commands
    self.run_command(cmd)
  File "C:\Users\SinoCBD\AppData\Local\Programs\Python\Python38\lib\distutils\dist.py", line 985, in run_command
    cmd_obj.run()
  File "C:\Users\SinoCBD\AppData\Local\Temp\pip-install-gvkelvan\numpy_32d9c41759e64c63a571fd9253149241\numpy\distutils\command\install.py", line 62, in run
    r = self.setuptools_run()
  File "C:\Users\SinoCBD\AppData\Local\Temp\pip-install-gvkelvan\numpy_32d9c41759e64c63a571fd9253149241\numpy\distutils\command\install.py", line 36, in setuptools_run
    return distutils_install.run(self)
  File "C:\Users\SinoCBD\AppData\Local\Programs\Python\Python38\lib\distutils\command\install.py", line 545, in run
    self.run_command('build')
  File "C:\Users\SinoCBD\AppData\Local\Programs\Python\Python38\lib\distutils\cmd.py", line 313, in run_command
    self.distribution.run_command(command)
  File "C:\Users\SinoCBD\AppData\Local\Programs\Python\Python38\lib\distutils\dist.py", line 985, in run_command
    cmd_obj.run()
  File "C:\Users\SinoCBD\AppData\Local\Temp\pip-install-gvkelvan\numpy_32d9c41759e64c63a571fd9253149241\numpy\distutils\command\build.py", line 47, in run
    old_build.run(self)
  File "C:\Users\SinoCBD\AppData\Local\Programs\Python\Python38\lib\distutils\command\build.py", line 135, in run
    self.run_command(cmd_name)
  File "C:\Users\SinoCBD\AppData\Local\Programs\Python\Python38\lib\distutils\cmd.py", line 313, in run_command
    self.distribution.run_command(command)
  File "C:\Users\SinoCBD\AppData\Local\Programs\Python\Python38\lib\distutils\dist.py", line 985, in run_command
    cmd_obj.run()
  File "C:\Users\SinoCBD\AppData\Local\Temp\pip-install-gvkelvan\numpy_32d9c41759e64c63a571fd9253149241\numpy\distutils\command\build_src.py", line 148, in run
    self.build_sources()
  File "C:\Users\SinoCBD\AppData\Local\Temp\pip-install-gvkelvan\numpy_32d9c41759e64c63a571fd9253149241\numpy\distutils\command\build_src.py", line 159, in build_sources
    self.build_library_sources(*libname_info)
  File "C:\Users\SinoCBD\AppData\Local\Temp\pip-install-gvkelvan\numpy_32d9c41759e64c63a571fd9253149241\numpy\distutils\command\build_src.py", line 292, in build_library_sources
    sources = self.generate_sources(sources, (lib_name, build_info))
  File "C:\Users\SinoCBD\AppData\Local\Temp\pip-install-gvkelvan\numpy_32d9c41759e64c63a571fd9253149241\numpy\distutils\command\build_src.py", line 375, in generate_sources
    source = func(extension, build_dir)
  File "numpy\core\setup.py", line 667, in get_mathlib_info
    raise RuntimeError("Broken toolchain: cannot link a simple C program")
RuntimeError: Broken toolchain: cannot link a simple C program
----------------------------------------

Rolling back uninstall of numpy
Moving to c:\users\sinocbd\appdata\local\programs\python\python38\lib\site-packages\numpy-1.21.2.dist-info
from C:\Users\SinoCBD\AppData\Local\Programs\Python\Python38\Lib\site-packages~umpy-1.21.2.dist-info
Moving to c:\users\sinocbd\appdata\local\programs\python\python38\lib\site-packages\numpy
from C:\Users\SinoCBD\AppData\Local\Programs\Python\Python38\Lib\site-packages~-mpy
Moving to c:\users\sinocbd\appdata\local\programs\python\python38\scripts\f2py.exe
from C:\Users\SinoCBD\AppData\Local\Temp\pip-uninstall-0354tw02\f2py.exe
ERROR: Command errored out with exit status 1: 'C:\Users\SinoCBD\AppData\Local\Programs\Python\Python38\python.exe' -u -c 'import io, os, sys, setuptools, tokenize; sys.argv[0] = '"'"'C:\Users\SinoCBD\AppData\Local\Temp\pip-install-gvkelvan\numpy_32d9c41759e64c63a571fd9253149241\setup.py'"'"'; file='"'"'C:\Users\SinoCBD\AppData\Local\Temp\pip-install-gvkelvan\numpy_32d9c41759e64c63a571fd9253149241\setup.py'"'"';f = getattr(tokenize, '"'"'open'"'"', open)(file) if os.path.exists(file) else io.StringIO('"'"'from setuptools import setup; setup()'"'"');code = f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, file, '"'"'exec'"'"'))' install --record 'C:\Users\SinoCBD\AppData\Local\Temp\pip-record-z7pop_qe\install-record.txt' --single-version-externally-managed --compile --install-headers 'C:\Users\SinoCBD\AppData\Local\Programs\Python\Python38\Include\numpy' Check the logs for full command output.
WARNING: Ignoring invalid distribution -umpy (c:\users\sinocbd\appdata\local\programs\python\python38\lib\site-packages)
WARNING: Ignoring invalid distribution -pype1 (c:\users\sinocbd\appdata\local\programs\python\python38\lib\site-packages)
WARNING: Ignoring invalid distribution -umpy (c:\users\sinocbd\appdata\local\programs\python\python38\lib\site-packages)
WARNING: Ignoring invalid distribution -pype1 (c:\users\sinocbd\appdata\local\programs\python\python38\lib\site-packages)

大佬，更新起来啊

https://github.com/kingname/GeneralNewsExtractor/
隔壁的经常更新，你这都没更新，对比了源码，你这个更智能傻瓜式！！！

详情页只能提取到一个段落

测试 HTML：

cqie.html.zip

解析结果有问题

http://www.beijing.gov.cn/zhengce/zcjd/index_130.html

建议增加一个传入xpath，缩小提取范围的功能

我在提取列表页：aHR0cDovL3d3dy5zaGFuZ2hhaS5nb3YuY24vbncyL253MjMxNC9udzIzMTkvbncyNDA3L253NDg2NzgvaW5kZXguaHRtbA==(base64)的时候，会存在误提取的情况。

一些页面结构比较复杂，根据现有提取规则会存在误提取或者提取不到的情况，在想能否增加一个功能，不论是列表页提取还是详情页提取，用户自定义传入xpath，缩小提取范围后，再去提取，这样能大大增加提取精度呀。

崔哥加油

Extractor of author

Add feature for extracting author

https://www.econ.sdu.edu.cn/zxzx/tzgg.htm 类似这种带分类链接的能智能提取吗

https://www.econ.sdu.edu.cn/zxzx/tzgg.htm
这种网站，由于有2个链接，导致结果为空，大佬可以更新下吗

中文detail页面包含英文段落会导致识别准确度下降

描述
用的是“故宫低调点”的最新页面（见末尾附件），识别的结果为“特别声明”部分，而非文章实际内容。

原因
该部分主要为英文，导致“文本密度”比汉字节点的要高很多，英文的字数统计按照字符，而非单词，比如“hello world”字数为10，而非2，相比中文具有明显的字数优势，因此“文本密度”指标出现偏差，进而影响了节点的最终得分。具体数据如下：

方案
如果页面以中文为主，那么针对英文段落，其中字数的统计应该跟中文保持一致，标准统一，即按照单词数来统计，而非字符来统计。

我针对number_of_char和number_of_a_char这2个方法，按照如上思路进行了优化，得到了预期结果。如下：

附件
网页源代码，把后缀改为html即可
gugong_detail.txt

numpy版本问题

从1.18版本开始，numpy正式淘汰了numpy.testing.decorators这一系列模块名，作者可以更新一下相关的包，我用的numpy版本是1.19.5，出了点问题，如下：
D:\fsy\Anaconda\python.exe D:/python_study/IntelligentAnalysis/extract.py
Traceback (most recent call last):
File "D:/python_study/IntelligentAnalysis/extract.py", line 1, in
from gerapy_auto_extractor import extract_detail
File "D:\fsy\Anaconda\lib\site-packages\gerapy_auto_extractor_init_.py", line 4, in
from gerapy_auto_extractor.classifiers.list import is_list, probability_of_list
File "D:\fsy\Anaconda\lib\site-packages\gerapy_auto_extractor\classifiers\list.py", line 6, in
from sklearn.metrics import classification_report
File "D:\fsy\Anaconda\lib\site-packages\sklearn_init_.py", line 82, in
from .base import clone
File "D:\fsy\Anaconda\lib\site-packages\sklearn\base.py", line 17, in
from .utils import IS_32BIT
File "D:\fsy\Anaconda\lib\site-packages\sklearn\utils_init.py", line 23, in
from .class_weight import compute_class_weight, compute_sample_weight
File "D:\fsy\Anaconda\lib\site-packages\sklearn\utils\class_weight.py", line 7, in
from .validation import _deprecate_positional_args
File "D:\fsy\Anaconda\lib\site-packages\sklearn\utils\validation.py", line 26, in
from .fixes import object_dtype_isnan, parse_version
File "D:\fsy\Anaconda\lib\site-packages\sklearn\utils\fixes.py", line 20, in
import scipy.stats
File "D:\fsy\Anaconda\lib\site-packages\scipy\stats_init.py", line 345, in
from .morestats import *
File "D:\fsy\Anaconda\lib\site-packages\scipy\stats\morestats.py", line 12, in
from numpy.testing.decorators import setastest
ModuleNotFoundError: No module named 'numpy.testing.decorators'

对于分页页面爬取的建议

对于分页页面的爬取，有一些标签容易引入到list或detail中（例如一些浮动的列表元素等），而这些内容在每个分页都会重复，建议对这部分重复的数据进行自适应检测，这部分不属于爬取的内容，属于噪声。

numpy版本问题

从1.18版本开始，numpy正式淘汰了numpy.testing.decorators这一系列模块名（注意：只是淘汰了模块名，他们实质上包含的内容还在），并且贴心地告诉用户，要用umpy.testing.decorators里面的功能（命名空间），只需要import numpy.testing 就够了，自当前版本开始的testing自动会包含原有的testing.decorations等子模块
作者可以更新一下相关的包嘛？，我用的numpy版本是1.19.5，出了点问题，如下：
D:\fsy\Anaconda\python.exe D:/python_study/IntelligentAnalysis/extract.py
Traceback (most recent call last):
File "D:/python_study/IntelligentAnalysis/extract.py", line 1, in
from gerapy_auto_extractor import extract_detail
File "D:\fsy\Anaconda\lib\site-packages\gerapy_auto_extractor_init_.py", line 4, in
from gerapy_auto_extractor.classifiers.list import is_list, probability_of_list
File "D:\fsy\Anaconda\lib\site-packages\gerapy_auto_extractor\classifiers\list.py", line 6, in
from sklearn.metrics import classification_report
File "D:\fsy\Anaconda\lib\site-packages\sklearn_init_.py", line 82, in
from .base import clone
File "D:\fsy\Anaconda\lib\site-packages\sklearn\base.py", line 17, in
from .utils import IS_32BIT
File "D:\fsy\Anaconda\lib\site-packages\sklearn\utils_init.py", line 23, in
from .class_weight import compute_class_weight, compute_sample_weight
File "D:\fsy\Anaconda\lib\site-packages\sklearn\utils\class_weight.py", line 7, in
from .validation import _deprecate_positional_args
File "D:\fsy\Anaconda\lib\site-packages\sklearn\utils\validation.py", line 26, in
from .fixes import object_dtype_isnan, parse_version
File "D:\fsy\Anaconda\lib\site-packages\sklearn\utils\fixes.py", line 20, in
import scipy.stats
File "D:\fsy\Anaconda\lib\site-packages\scipy\stats_init.py", line 345, in
from .morestats import *
File "D:\fsy\Anaconda\lib\site-packages\scipy\stats\morestats.py", line 12, in
from numpy.testing.decorators import setastest
ModuleNotFoundError: No module named 'numpy.testing.decorators'

Bug of Gerapy Auto Extractor about similarity2

def similarity2(s1, s2):
"""
get similarity of two strings
:param s1:
:param s2:
:return:
"""
if not s1 or not s2:
return 0
s1_set = set(list(s1))
s2_set = set(list(s2))
intersection = s1_set.intersection(s2_set)
union = s1_set.intersection(s2_set)
return len(intersection) / len(union)

union = s1_set.intersection(s2_set)    # 这个应该是并集才对吧，源码里边应该是取错了

Recommend Projects

React

A declarative, efficient, and flexible JavaScript library for building user interfaces.
Vue.js

🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
Typescript

TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
TensorFlow

An Open Source Machine Learning Framework for Everyone
Django

The Web framework for perfectionists with deadlines.
Laravel

A PHP framework for web artisans
D3

Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

javascript

JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
web

Some thing interesting about web. New door for the world.
server

A server is a program made to process requests and deliver data to clients.
Machine learning

Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Visualization

Some thing interesting about visualization, use data art
Game

Some thing interesting about game, make everyone happy.

Recommend Org

Facebook

We are working to build community through open source technology. NB: members must have two-factor auth.
Microsoft

Open source projects and samples from Microsoft.
Google

Google ❤️ Open Source for everyone.
Alibaba

Alibaba Open Source for everyone
D3

Data-Driven Documents codes.
Tencent

China tencent open source team.

gerapy / gerapyautoextractor Goto Github PK

gerapyautoextractor's Introduction

Gerapy

Documentation

Support

Usage

Docker

TodoList

Communication

gerapyautoextractor's People

Contributors

Stargazers

Watchers

Forkers

gerapyautoextractor's Issues

Recommend Projects

Recommend Topics

Recommend Org