iclrandd / blackstone Goto Github PK

View Code? Open in Web Editor NEW

633.0 39.0 101.0 305 KB

:black_circle: A spaCy pipeline and model for NLP on unstructured legal text.

Home Page: https://research.iclr.co.uk

License: Apache License 2.0

Python 99.93% Dockerfile 0.07%

law caselaw nlp spacy-models legaltech

blackstone's People

Contributors

Stargazers

Watchers

Forkers

awesome-archive brianlv aymansalama trendingtechnology codeaudit adewin awye765 pkrouth kgeorgiadis-law bci-oshita zeichenkette ssitb stjordanis redfox1899 codefly13 ufukhurriyetoglu viralsteroids joshua888-cyrpto akki2825 tamarit sleclair0 fighting41love jonathanbowker databill86 richardbatstone robertomalatesta data-science-ai-open-source deneutoy omattuy houfu awoziji moritzpschwarz goooice grafoo legaleasellc vamsi1905 nndnha chetannitk fischtank44 lin-deng dylanhogg sree181 humdingers badbanana123 timjones1 walid-rahman2 pika97 stoic-signs arturtan houchangxi yahyaghani ryanmcdonough addleshaw-goddard arian-askari mikekennedyag taiphillips emersoncpp warhammer0 adeola23 swipswaps polymath-is bowwowden aropjoe bgpearman yykzgyy shwethamh ruhroh keshava danielhoadley robsteranium aahmadai techthiyanes cokelly amyconroy my-machine-learning-projects-ct gtthomas21 goro53467 onlycoca trrame reloadbrain darongdg chouhanashish18 creative-python c-bik tracywang95 tanerim xn18492 646e62 gwjr frankie336 nekoyokoshima thechrisoshow slaterpeter shubhamtheds gjb117 claycampbell cywsg andyjessen nolan-nan almighty-hardik

blackstone's Issues

Any particular reason why spacy is not been upgraded to 2.2+?

Hi ,

Spacy 2.1.8 is failing on my laptop but spacy 2.2+ install fine. Do you have any plans of upgrading the spacy version?

Abbreviation detection not working where short form contains a space followed by digits

The current implementation of the AbbreviationDetector() does not handle abbreviations that contain a short form followed by a space followed by a number

For example, in this scenario:

The Proceeds of Crime Act 2002 ("PoCA 2000")

The abbreviation is not matched.

The original implementation in scispaCy does not appear to have been built to handle instances in which the short form is bounded by quote marks).

neither segmenter, nor legislation linker, nor anything else works

blackstone.legislation_linker

AttributeError: module 'blackstone' has no attribute 'legislation_linker'

Additional Entity Types & Models

The prototype Blackstone model, en_blackstone_proto, was trained to detect six entity types that apply generally across legal texts (in the sense that they're not specific to any legal sub-discipline, such as criminal law, company law etc).

If you have any ideas for additional entity types that we should consider adding to future models, this is the place to add them.

Preferred method for setting out your ideas

For the sake of consistency, please add comments to this issue in the following format:

ENTITY TYPE:
ENTITY DESCRIPTION:
LEGAL TOPIC:
EXAMPLE:

For example, if you're submitting an idea for a new entity type that you think would apply generally across legal text (i.e. something that is not specific to any sub-discipline of law) you're comment should look like this:

ENTITY TYPE: Law Commission Report
ENTITY DESCRIPTION: Detects mentions of Law Commission Reports
LEGAL TOPIC: General
EXAMPLE: In addition, she considered the Law Commission Report on Contribution (Law Com No 79) (1977), which led to the enactment of the 1978 Act...

If, on the other hand, you're submitting an idea for a new entity type that you think applies to a particular sub-discipline, you're comment should look like this:

ENTITY TYPE: Indictment
ENTITY DESCRIPTION: Detects mentions of indictments
LEGAL TOPIC: Criminal law
EXAMPLE: On the other indictment (T20180081) there were included three counts of having an article with a blade or point, contrary to section 139(1) of the Criminal Justice Act 1988.

Pipenv installation failed

Using a clean pipfile and environment, a blackstone dependency failed. Looks like it's something about preshed, blis and unicode?

(base) peter@Peters-MBP14 pych-km-4 % pipenv shell
Creating a virtualenv for this project...
Pipfile: /Users/peter/pych-km-4/Pipfile
Using /Users/peter/opt/anaconda3/bin/python3 (3.9.12) to create virtualenv...
⠋ Creating virtual environment...created virtual environment CPython3.9.12.final.0-64 in 400ms
  creator CPython3Posix(dest=/Users/peter/.local/share/virtualenvs/pych-km-4-slCQK9Uf, clear=False, no_vcs_ignore=False, global=False)
  seeder FromAppData(download=False, pip=bundle, setuptools=bundle, wheel=bundle, via=copy, app_data_dir=/Users/peter/Library/Application Support/virtualenv)
    added seed packages: pip==22.1.2, setuptools==62.2.0, wheel==0.37.1
  activators BashActivator,CShellActivator,FishActivator,NushellActivator,PowerShellActivator,PythonActivator

✔ Successfully created virtual environment!
Virtualenv location: /Users/peter/.local/share/virtualenvs/pych-km-4-slCQK9Uf
Creating a Pipfile for this project...
Launching subshell in virtual environment...
Loading .zshrc
 . /Users/peter/.local/share/virtualenvs/pych-km-4-slCQK9Uf/bin/activate
(base) peter@Peters-MBP14 pych-km-4 %  . /Users/peter/.local/share/virtualenvs/pych-km-4-slCQK9Uf/bin/activate

Then:

(pych-km-4) (base) peter@Peters-MBP14 pych-km-4 % pipenv install blackstone
            Compiler gcc
            building 'blis.cy' extension
            creating build/temp.macosx-10.9-x86_64-cpython-39
            creating build/temp.macosx-10.9-x86_64-cpython-39/blis
            clang -Wno-unused-result -Wsign-compare -Wunreachable-code -DNDEBUG -fwrapv -O2 -Wall -fPIC -O2 -isystem /Users/peter/opt/anaconda3/include -arch x86_64 -I/Users/peter/opt/anaconda3/include -fPIC -O2 -isystem /Users/peter/opt/anaconda3/include -arch x86_64 -I/private/var/folders/dh/1ggfwgy164d06z3k4yf78ksc0000gn/T/pip-install-masm7svx/blis_8ee02e7df50f4e0c8da54b2d54349d4d/include -I/private/var/folders/dh/1ggfwgy164d06z3k4yf78ksc0000gn/T/pip-install-masm7svx/blis_8ee02e7df50f4e0c8da54b2d54349d4d/blis/_src/include/darwin-x86_64 -I/Users/peter/.local/share/virtualenvs/pych-km-3-CwEGPvOL/include -I/Users/peter/opt/anaconda3/include/python3.9 -c blis/cy.c -o build/temp.macosx-10.9-x86_64-cpython-39/blis/cy.o -std=c99
            blis/cy.c:2470:7: warning: code will never be executed [-Wunreachable-code]
                  PyErr_SetNone(PyExc_AssertionError);
                  ^~~~~~~~~~~~~
            blis/cy.c:2486:7: warning: code will never be executed [-Wunreachable-code]
                  PyErr_SetNone(PyExc_AssertionError);
                  ^~~~~~~~~~~~~
            blis/cy.c:2502:7: warning: code will never be executed [-Wunreachable-code]
                  PyErr_SetNone(PyExc_AssertionError);
                  ^~~~~~~~~~~~~
            blis/cy.c:2518:7: warning: code will never be executed [-Wunreachable-code]
                  PyErr_SetNone(PyExc_AssertionError);
                  ^~~~~~~~~~~~~
            blis/cy.c:2534:7: warning: code will never be executed [-Wunreachable-code]
                  PyErr_SetNone(PyExc_AssertionError);
                  ^~~~~~~~~~~~~
            blis/cy.c:2550:7: warning: code will never be executed [-Wunreachable-code]
                  PyErr_SetNone(PyExc_AssertionError);
                  ^~~~~~~~~~~~~
            blis/cy.c:2566:7: warning: code will never be executed [-Wunreachable-code]
                  PyErr_SetNone(PyExc_AssertionError);
                  ^~~~~~~~~~~~~
            blis/cy.c:2582:7: warning: code will never be executed [-Wunreachable-code]
                  PyErr_SetNone(PyExc_AssertionError);
                  ^~~~~~~~~~~~~
            blis/cy.c:2598:7: warning: code will never be executed [-Wunreachable-code]
                  PyErr_SetNone(PyExc_AssertionError);
                  ^~~~~~~~~~~~~
            blis/cy.c:2614:7: warning: code will never be executed [-Wunreachable-code]
                  PyErr_SetNone(PyExc_AssertionError);
                  ^~~~~~~~~~~~~
            blis/cy.c:2630:7: warning: code will never be executed [-Wunreachable-code]
                  PyErr_SetNone(PyExc_AssertionError);
                  ^~~~~~~~~~~~~
            blis/cy.c:2646:7: warning: code will never be executed [-Wunreachable-code]
                  PyErr_SetNone(PyExc_AssertionError);
                  ^~~~~~~~~~~~~
            blis/cy.c:2662:7: warning: code will never be executed [-Wunreachable-code]
                  PyErr_SetNone(PyExc_AssertionError);
                  ^~~~~~~~~~~~~
            blis/cy.c:5740:3: warning: code will never be executed [-Wunreachable-code]
              goto __pyx_L0;
              ^~~~~~~~~~~~~
            blis/cy.c:5871:3: warning: code will never be executed [-Wunreachable-code]
              goto __pyx_L0;
              ^~~~~~~~~~~~~
            blis/cy.c:21738:18: error: no member named 'tp_print' in 'struct _typeobject'
              __Pyx_EnumMeta.tp_print = 0;
              ~~~~~~~~~~~~~~ ^
            blis/cy.c:21747:26: error: no member named 'tp_print' in 'struct _typeobject'
              __pyx_type___pyx_array.tp_print = 0;
              ~~~~~~~~~~~~~~~~~~~~~~ ^
            blis/cy.c:21752:32: error: no member named 'tp_print' in 'struct _typeobject'
              __pyx_type___pyx_MemviewEnum.tp_print = 0;
              ~~~~~~~~~~~~~~~~~~~~~~~~~~~~ ^
            blis/cy.c:21767:31: error: no member named 'tp_print' in 'struct _typeobject'
              __pyx_type___pyx_memoryview.tp_print = 0;
              ~~~~~~~~~~~~~~~~~~~~~~~~~~~ ^
            blis/cy.c:21780:36: error: no member named 'tp_print' in 'struct _typeobject'
              __pyx_type___pyx_memoryviewslice.tp_print = 0;
              ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ ^
            blis/cy.c:24202:22: warning: '_PyUnicode_get_wstr_length' is deprecated [-Wdeprecated-declarations]
                                (PyUnicode_GET_SIZE(**name) != PyUnicode_GET_SIZE(key)) ? 1 :
                                 ^
            /Users/peter/opt/anaconda3/include/python3.9/cpython/unicodeobject.h:261:7: note: expanded from macro 'PyUnicode_GET_SIZE'
                  PyUnicode_WSTR_LENGTH(op) :                    \
                  ^
            /Users/peter/opt/anaconda3/include/python3.9/cpython/unicodeobject.h:451:35: note: expanded from macro 'PyUnicode_WSTR_LENGTH'
            #define PyUnicode_WSTR_LENGTH(op) _PyUnicode_get_wstr_length((PyObject*)op)
                                              ^
            /Users/peter/opt/anaconda3/include/python3.9/cpython/unicodeobject.h:445:1: note: '_PyUnicode_get_wstr_length' has been explicitly marked deprecated here
            Py_DEPRECATED(3.3)
            ^
            /Users/peter/opt/anaconda3/include/python3.9/pyport.h:508:54: note: expanded from macro 'Py_DEPRECATED'
            #define Py_DEPRECATED(VERSION_UNUSED) __attribute__((__deprecated__))
                                                                 ^
            blis/cy.c:24202:22: warning: 'PyUnicode_AsUnicode' is deprecated [-Wdeprecated-declarations]
                                (PyUnicode_GET_SIZE(**name) != PyUnicode_GET_SIZE(key)) ? 1 :
                                 ^
            /Users/peter/opt/anaconda3/include/python3.9/cpython/unicodeobject.h:262:14: note: expanded from macro 'PyUnicode_GET_SIZE'
                  ((void)PyUnicode_AsUnicode(_PyObject_CAST(op)),\
                         ^
            /Users/peter/opt/anaconda3/include/python3.9/cpython/unicodeobject.h:580:1: note: 'PyUnicode_AsUnicode' has been explicitly marked deprecated here
            Py_DEPRECATED(3.3) PyAPI_FUNC(Py_UNICODE *) PyUnicode_AsUnicode(
            ^
            /Users/peter/opt/anaconda3/include/python3.9/pyport.h:508:54: note: expanded from macro 'Py_DEPRECATED'
            #define Py_DEPRECATED(VERSION_UNUSED) __attribute__((__deprecated__))
                                                                 ^
            blis/cy.c:24202:22: warning: '_PyUnicode_get_wstr_length' is deprecated [-Wdeprecated-declarations]
                                (PyUnicode_GET_SIZE(**name) != PyUnicode_GET_SIZE(key)) ? 1 :
                                 ^
            /Users/peter/opt/anaconda3/include/python3.9/cpython/unicodeobject.h:264:8: note: expanded from macro 'PyUnicode_GET_SIZE'
                   PyUnicode_WSTR_LENGTH(op)))
                   ^
            /Users/peter/opt/anaconda3/include/python3.9/cpython/unicodeobject.h:451:35: note: expanded from macro 'PyUnicode_WSTR_LENGTH'
            #define PyUnicode_WSTR_LENGTH(op) _PyUnicode_get_wstr_length((PyObject*)op)
                                              ^
            /Users/peter/opt/anaconda3/include/python3.9/cpython/unicodeobject.h:445:1: note: '_PyUnicode_get_wstr_length' has been explicitly marked deprecated here
            Py_DEPRECATED(3.3)
            ^
            /Users/peter/opt/anaconda3/include/python3.9/pyport.h:508:54: note: expanded from macro 'Py_DEPRECATED'
            #define Py_DEPRECATED(VERSION_UNUSED) __attribute__((__deprecated__))
                                                                 ^
            blis/cy.c:24202:52: warning: '_PyUnicode_get_wstr_length' is deprecated [-Wdeprecated-declarations]
                                (PyUnicode_GET_SIZE(**name) != PyUnicode_GET_SIZE(key)) ? 1 :
                                                               ^
            /Users/peter/opt/anaconda3/include/python3.9/cpython/unicodeobject.h:261:7: note: expanded from macro 'PyUnicode_GET_SIZE'
                  PyUnicode_WSTR_LENGTH(op) :                    \
                  ^
            /Users/peter/opt/anaconda3/include/python3.9/cpython/unicodeobject.h:451:35: note: expanded from macro 'PyUnicode_WSTR_LENGTH'
            #define PyUnicode_WSTR_LENGTH(op) _PyUnicode_get_wstr_length((PyObject*)op)
                                              ^
            /Users/peter/opt/anaconda3/include/python3.9/cpython/unicodeobject.h:445:1: note: '_PyUnicode_get_wstr_length' has been explicitly marked deprecated here
            Py_DEPRECATED(3.3)
            ^
            /Users/peter/opt/anaconda3/include/python3.9/pyport.h:508:54: note: expanded from macro 'Py_DEPRECATED'
            #define Py_DEPRECATED(VERSION_UNUSED) __attribute__((__deprecated__))
                                                                 ^
            blis/cy.c:24202:52: warning: 'PyUnicode_AsUnicode' is deprecated [-Wdeprecated-declarations]
                                (PyUnicode_GET_SIZE(**name) != PyUnicode_GET_SIZE(key)) ? 1 :
                                                               ^
            /Users/peter/opt/anaconda3/include/python3.9/cpython/unicodeobject.h:262:14: note: expanded from macro 'PyUnicode_GET_SIZE'
                  ((void)PyUnicode_AsUnicode(_PyObject_CAST(op)),\
                         ^
            /Users/peter/opt/anaconda3/include/python3.9/cpython/unicodeobject.h:580:1: note: 'PyUnicode_AsUnicode' has been explicitly marked deprecated here
            Py_DEPRECATED(3.3) PyAPI_FUNC(Py_UNICODE *) PyUnicode_AsUnicode(
            ^
            /Users/peter/opt/anaconda3/include/python3.9/pyport.h:508:54: note: expanded from macro 'Py_DEPRECATED'
            #define Py_DEPRECATED(VERSION_UNUSED) __attribute__((__deprecated__))
                                                                 ^
            blis/cy.c:24202:52: warning: '_PyUnicode_get_wstr_length' is deprecated [-Wdeprecated-declarations]
                                (PyUnicode_GET_SIZE(**name) != PyUnicode_GET_SIZE(key)) ? 1 :
                                                               ^
            /Users/peter/opt/anaconda3/include/python3.9/cpython/unicodeobject.h:264:8: note: expanded from macro 'PyUnicode_GET_SIZE'
                   PyUnicode_WSTR_LENGTH(op)))
                   ^
            /Users/peter/opt/anaconda3/include/python3.9/cpython/unicodeobject.h:451:35: note: expanded from macro 'PyUnicode_WSTR_LENGTH'
            #define PyUnicode_WSTR_LENGTH(op) _PyUnicode_get_wstr_length((PyObject*)op)
                                              ^
            /Users/peter/opt/anaconda3/include/python3.9/cpython/unicodeobject.h:445:1: note: '_PyUnicode_get_wstr_length' has been explicitly marked deprecated here
            Py_DEPRECATED(3.3)
            ^
            /Users/peter/opt/anaconda3/include/python3.9/pyport.h:508:54: note: expanded from macro 'Py_DEPRECATED'
            #define Py_DEPRECATED(VERSION_UNUSED) __attribute__((__deprecated__))
                                                                 ^
            blis/cy.c:24218:26: warning: '_PyUnicode_get_wstr_length' is deprecated [-Wdeprecated-declarations]
                                    (PyUnicode_GET_SIZE(**argname) != PyUnicode_GET_SIZE(key)) ? 1 :
                                     ^
            /Users/peter/opt/anaconda3/include/python3.9/cpython/unicodeobject.h:261:7: note: expanded from macro 'PyUnicode_GET_SIZE'
                  PyUnicode_WSTR_LENGTH(op) :                    \
                  ^
            /Users/peter/opt/anaconda3/include/python3.9/cpython/unicodeobject.h:451:35: note: expanded from macro 'PyUnicode_WSTR_LENGTH'
            #define PyUnicode_WSTR_LENGTH(op) _PyUnicode_get_wstr_length((PyObject*)op)
                                              ^
            /Users/peter/opt/anaconda3/include/python3.9/cpython/unicodeobject.h:445:1: note: '_PyUnicode_get_wstr_length' has been explicitly marked deprecated here
            Py_DEPRECATED(3.3)
            ^
            /Users/peter/opt/anaconda3/include/python3.9/pyport.h:508:54: note: expanded from macro 'Py_DEPRECATED'
            #define Py_DEPRECATED(VERSION_UNUSED) __attribute__((__deprecated__))
                                                                 ^
            blis/cy.c:24218:26: warning: 'PyUnicode_AsUnicode' is deprecated [-Wdeprecated-declarations]
                                    (PyUnicode_GET_SIZE(**argname) != PyUnicode_GET_SIZE(key)) ? 1 :
                                     ^
            /Users/peter/opt/anaconda3/include/python3.9/cpython/unicodeobject.h:262:14: note: expanded from macro 'PyUnicode_GET_SIZE'
                  ((void)PyUnicode_AsUnicode(_PyObject_CAST(op)),\
                         ^
            /Users/peter/opt/anaconda3/include/python3.9/cpython/unicodeobject.h:580:1: note: 'PyUnicode_AsUnicode' has been explicitly marked deprecated here
            Py_DEPRECATED(3.3) PyAPI_FUNC(Py_UNICODE *) PyUnicode_AsUnicode(
            ^
            /Users/peter/opt/anaconda3/include/python3.9/pyport.h:508:54: note: expanded from macro 'Py_DEPRECATED'
            #define Py_DEPRECATED(VERSION_UNUSED) __attribute__((__deprecated__))
                                                                 ^
            blis/cy.c:24218:26: warning: '_PyUnicode_get_wstr_length' is deprecated [-Wdeprecated-declarations]
                                    (PyUnicode_GET_SIZE(**argname) != PyUnicode_GET_SIZE(key)) ? 1 :
                                     ^
            /Users/peter/opt/anaconda3/include/python3.9/cpython/unicodeobject.h:264:8: note: expanded from macro 'PyUnicode_GET_SIZE'
                   PyUnicode_WSTR_LENGTH(op)))
                   ^
            /Users/peter/opt/anaconda3/include/python3.9/cpython/unicodeobject.h:451:35: note: expanded from macro 'PyUnicode_WSTR_LENGTH'
            #define PyUnicode_WSTR_LENGTH(op) _PyUnicode_get_wstr_length((PyObject*)op)
                                              ^
            /Users/peter/opt/anaconda3/include/python3.9/cpython/unicodeobject.h:445:1: note: '_PyUnicode_get_wstr_length' has been explicitly marked deprecated here
            Py_DEPRECATED(3.3)
            ^
            /Users/peter/opt/anaconda3/include/python3.9/pyport.h:508:54: note: expanded from macro 'Py_DEPRECATED'
            #define Py_DEPRECATED(VERSION_UNUSED) __attribute__((__deprecated__))
                                                                 ^
            blis/cy.c:24218:59: warning: '_PyUnicode_get_wstr_length' is deprecated [-Wdeprecated-declarations]
                                    (PyUnicode_GET_SIZE(**argname) != PyUnicode_GET_SIZE(key)) ? 1 :
                                                                      ^
            /Users/peter/opt/anaconda3/include/python3.9/cpython/unicodeobject.h:261:7: note: expanded from macro 'PyUnicode_GET_SIZE'
                  PyUnicode_WSTR_LENGTH(op) :                    \
                  ^
            /Users/peter/opt/anaconda3/include/python3.9/cpython/unicodeobject.h:451:35: note: expanded from macro 'PyUnicode_WSTR_LENGTH'
            #define PyUnicode_WSTR_LENGTH(op) _PyUnicode_get_wstr_length((PyObject*)op)
                                              ^
            /Users/peter/opt/anaconda3/include/python3.9/cpython/unicodeobject.h:445:1: note: '_PyUnicode_get_wstr_length' has been explicitly marked deprecated here
            Py_DEPRECATED(3.3)
            ^
            /Users/peter/opt/anaconda3/include/python3.9/pyport.h:508:54: note: expanded from macro 'Py_DEPRECATED'
            #define Py_DEPRECATED(VERSION_UNUSED) __attribute__((__deprecated__))
                                                                 ^
            blis/cy.c:24218:59: warning: 'PyUnicode_AsUnicode' is deprecated [-Wdeprecated-declarations]
                                    (PyUnicode_GET_SIZE(**argname) != PyUnicode_GET_SIZE(key)) ? 1 :
                                                                      ^
            /Users/peter/opt/anaconda3/include/python3.9/cpython/unicodeobject.h:262:14: note: expanded from macro 'PyUnicode_GET_SIZE'
                  ((void)PyUnicode_AsUnicode(_PyObject_CAST(op)),\
                         ^
            /Users/peter/opt/anaconda3/include/python3.9/cpython/unicodeobject.h:580:1: note: 'PyUnicode_AsUnicode' has been explicitly marked deprecated here
            Py_DEPRECATED(3.3) PyAPI_FUNC(Py_UNICODE *) PyUnicode_AsUnicode(
            ^
            /Users/peter/opt/anaconda3/include/python3.9/pyport.h:508:54: note: expanded from macro 'Py_DEPRECATED'
            #define Py_DEPRECATED(VERSION_UNUSED) __attribute__((__deprecated__))
                                                                 ^
            blis/cy.c:24218:59: warning: '_PyUnicode_get_wstr_length' is deprecated [-Wdeprecated-declarations]
                                    (PyUnicode_GET_SIZE(**argname) != PyUnicode_GET_SIZE(key)) ? 1 :
                                                                      ^
            /Users/peter/opt/anaconda3/include/python3.9/cpython/unicodeobject.h:264:8: note: expanded from macro 'PyUnicode_GET_SIZE'
                   PyUnicode_WSTR_LENGTH(op)))
                   ^
            /Users/peter/opt/anaconda3/include/python3.9/cpython/unicodeobject.h:451:35: note: expanded from macro 'PyUnicode_WSTR_LENGTH'
            #define PyUnicode_WSTR_LENGTH(op) _PyUnicode_get_wstr_length((PyObject*)op)
                                              ^
            /Users/peter/opt/anaconda3/include/python3.9/cpython/unicodeobject.h:445:1: note: '_PyUnicode_get_wstr_length' has been explicitly marked deprecated here
            Py_DEPRECATED(3.3)
            ^
            /Users/peter/opt/anaconda3/include/python3.9/pyport.h:508:54: note: expanded from macro 'Py_DEPRECATED'
            #define Py_DEPRECATED(VERSION_UNUSED) __attribute__((__deprecated__))
                                                                 ^
            blis/cy.c:25025:16: warning: 'PyUnicode_FromUnicode' is deprecated [-Wdeprecated-declarations]
                    return PyUnicode_FromUnicode(NULL, 0);
                           ^
            /Users/peter/opt/anaconda3/include/python3.9/cpython/unicodeobject.h:551:1: note: 'PyUnicode_FromUnicode' has been explicitly marked deprecated here
            Py_DEPRECATED(3.3) PyAPI_FUNC(PyObject*) PyUnicode_FromUnicode(
            ^
            /Users/peter/opt/anaconda3/include/python3.9/pyport.h:508:54: note: expanded from macro 'Py_DEPRECATED'
            #define Py_DEPRECATED(VERSION_UNUSED) __attribute__((__deprecated__))
                                                                 ^
            28 warnings and 5 errors generated.
            error: command '/usr/bin/clang' failed with exit code 1
            [end of output]

        note: This error originates from a subprocess, and is likely not a problem with pip.
        ERROR: Failed building wheel for blis
        Running setup.py clean for blis
      Failed to build preshed blis
      Installing collected packages: wasabi, srsly, plac, murmurhash, cymem, wheel, tqdm, setuptools, preshed, numpy, Cython, blis, thinc
        Running setup.py install for preshed: started
        Running setup.py install for preshed: finished with status 'error'
        error: subprocess-exited-with-error

        × Running setup.py install for preshed did not run successfully.
        │ exit code: 1
        ╰─> [15 lines of output]
            /Users/peter/.local/share/virtualenvs/pych-km-3-CwEGPvOL/lib/python3.9/site-packages/setuptools/installer.py:27: SetuptoolsDeprecationWarning: setuptools.installer is deprecated. Requirements should be satisfied by a PEP 517 installer.
              warnings.warn(
            running install
            /Users/peter/.local/share/virtualenvs/pych-km-3-CwEGPvOL/lib/python3.9/site-packages/setuptools/command/install.py:34: SetuptoolsDeprecationWarning: setup.py install is deprecated. Use build and pip and other standards-based tools.
              warnings.warn(
            running build
            running build_py
            warning: build_py: byte-compiling is disabled, skipping.

            running build_ext
            building 'preshed.maps' extension
            clang -Wno-unused-result -Wsign-compare -Wunreachable-code -DNDEBUG -fwrapv -O2 -Wall -fPIC -O2 -isystem /Users/peter/opt/anaconda3/include -arch x86_64 -I/Users/peter/opt/anaconda3/include -fPIC -O2 -isystem /Users/peter/opt/anaconda3/include -arch x86_64 -I/Users/peter/opt/anaconda3/include/python3.9 -I/Users/peter/.local/share/virtualenvs/pych-km-3-CwEGPvOL/include -I/Users/peter/opt/anaconda3/include/python3.9 -c preshed/maps.cpp -o build/temp.macosx-10.9-x86_64-cpython-39/preshed/maps.o -O3 -Wno-strict-prototypes -Wno-unused-function
            clang: error: no such file or directory: 'preshed/maps.cpp'
            clang: error: no input files
            error: command '/usr/bin/clang' failed with exit code 1
            [end of output]

        note: This error originates from a subprocess, and is likely not a problem with pip.
      error: legacy-install-failure

      × Encountered error while trying to install package.
      ╰─> preshed

      note: This is an issue with the package mentioned above, not pip.
      hint: See above for output from the failure.
      [end of output]

  note: This error originates from a subprocess, and is likely not a problem with pip.
error: subprocess-exited-with-error

× pip subprocess to install build dependencies did not run successfully.
│ exit code: 1
╰─> See above for output.

note: This error originates from a subprocess, and is likely not a problem with pip.

Need some tests!

Speaks for itself. We really, really need some tests!

Add version to import package

Make it so we can track the version of blackstone at runtime via blackstone.__version__.

Add a `tests` directory

I recommend using https://docs.pytest.org/en/latest/

The tests directory should look like this:

tests/
- __init__.py (important)
- linker_test.py
- ...

Then add this file to your top level directory: https://github.com/allenai/scispacy/blob/master/pytest.ini

then you should be able to run all tests just by running pytest in the root of the project. The directory of your tests should look like the directory of the blackstone package. Tests will only be run if the class and method names begin or end with test.

config.cfg is missing from model

Error while loading the model : Could not read config.cfg

Stacktrace:

OSError Traceback (most recent call last)
in
1 # Load the model
----> 2 nlp = en_blackstone_proto.load()

env\lib\site-packages\en_blackstone_proto_init_.py in load(**overrides)
10
11 def load(**overrides):
---> 12 return load_model_from_init_py(file, **overrides)

env\lib\site-packages\spacy\util.py in load_model_from_init_py(init_file, vocab, disable, exclude, config)
512 if not model_path.exists():
513 raise IOError(Errors.E052.format(path=data_path))
--> 514 return load_model_from_path(
515 data_path,
516 vocab=vocab,

env\lib\site-packages\spacy\util.py in load_model_from_path(model_path, meta, vocab, disable, exclude, config)
386 meta = get_model_meta(model_path)
387 config_path = model_path / "config.cfg"
--> 388 config = load_config(config_path, overrides=dict_to_dot(config))
389 nlp = load_model_from_config(config, vocab=vocab, disable=disable, exclude=exclude)
390 return nlp.from_disk(model_path, exclude=exclude)

env\lib\site-packages\spacy\util.py in load_config(path, overrides, interpolate)
543 else:
544 if not config_path or not config_path.exists() or not config_path.is_file():
--> 545 raise IOError(Errors.E053.format(path=config_path, name="config.cfg"))
546 return config.from_disk(
547 config_path, overrides=overrides, interpolate=interpolate

OSError: [E053] Could not read config.cfg from env\lib\site-packages\en_blackstone_proto\en_blackstone_proto-0.0.1\config.cfg

Poorly maintained project: Upgrade package to support Python 3.10+

Hi, This only supports Python 3.6 it seems. We are in Py 3.10+ world. Can you please update your codebase so it can be compatible with Py 3.10+?

Thanks in advance.

ValueError - Unknown morphological feature: 'Person'

Hi.
After fresh installation I wanted to test sentence segmenter example but I get this error :

ValueError: [E167] Unknown morphological feature: 'Person' (2313063860588076218). This can happen if the tagger was trained with a different set of morphological features. If you're using a pretrained model, make sure that your models are up to date:
python -m spacy validate

python -m spacy validate show me:
TYPE NAME MODEL VERSION
package en-core-web-sm en_core_web_sm 2.2.0 ✔

I have tested with Spacy 2.2.1 and 2.2.0 (with Python 3.6.7 / Data Science Virtual Machine form Azure with GPU)

Thank you in advance for your help

requirements.txt is too big

requirements.txt typically only specifies the exact packages that are required to run a project, rather than an exhaustive list of your personal python environment.

https://github.com/allenai/scispacy/blob/master/requirements.in

Also, it's typical to only pin packages to a particular version if you know there is a problem in a newer version that will break stuff. The reason for this is that it makes it easier for people to install if you don't require precise version numbers.

Compatibility issue with Spacy v2.2

Please make it compatible with spacy v2.2.
Also, can you please share the dataset you trained this model on?

Add Sentence Segmenter

en_blackstone_proto, the model that ships with the prototype release of Blackstone, did not come with a sentence segmentation module. By and large, the current model does an okayish job splitting sentences, but it will get baffled on older material where punctation had a tendency to be crop dusted over the text.

We did sketch a sentence pipeline out during the development of the proto model (primarily to help with the data extraction for training the model itself), but it was put together in a bit of a flap.

I rather like scispaCy's implementation, which makes use of a list for abbreviations and section contractions.

General Python tidy

My point of view is that I have some Python experience, but very little domain experience.

Depending on who the contributors are, it might help things if we have a more robust Python environment:

virtual environment
runnable code
tests

If the imminent contributors aren't going to be Python-focussed, then that might not help, but if it does, then the benefit could be that everyone gets a common environment in which to play with this stuff early on.

Just a hello

Hi (Dan),

Just a hello, to say met with Paul & Paul on Friday. They gave us here at SCLQ a great chance to discuss all the cool stuff you've got going on. I am going to start playing with Blackstone and getting it to look at some of our Qld decisions.

One thing I'd like to discuss at some point is how to train it. Perhaps we can Skype sometime?

Custom modules are not getting loaded and giving error

from blackstone.pipeline.compound_cases import CompoundCases
ModuleNotFoundError: No module named 'blackstone.pipeline'; 'blackstone' is not a package

Is there anything to be done to load custom modules. Sorry , i could not find anything in the README regarding this.

Thanks,
Srijith

Compatibility with spaCy 2.1.9 & 2.2+

Hi Blackstone team,
at first, I want to thank you for your pre-trained models and your work in automatic legal text analysis. Especially your custom SentenceSegmenter and NER detections works very good with our dataset of legal texts.
Unfortunately this package still depends on spaCy 2.1 or more specifically on spaCy 2.1.8. This version currently has a major memory leak bug (explosion/spaCy#3618), which has been fixed with 2.1.9. I already modified the dependency files of Blackstone, so I'm able to install spaCy 2.1.9 instead of the required 2.1.8 which works flawlessly on my machine. You might consider changing your dependencies accordingly.
However, it would be even better if you could update to an even newer version of spaCy (e.g. 2.2+) to profit from several performance optimizations done by Explosion. There is already a pending pull request (#22) to address this issue, but without the corresponding training data you used to train the model there is no way to retrain ourselves.
It would be greatly appreciated if you could update your model & package to spaCy 2.2 - as this might take some time you update your package's dependencies to spaCy 2.1.9 in the meantime to circumvent memory leaks present in spaCy 2.1.9.

Unknown Morphological Feature

I am trying to get up to speed with the model but when I execute the example code given I hit the following error when nlp = spacy.load("en_blackstone_proto") is called.

[E167] Unknown morphological feature: 'ConjType' (9141427322507498425). This can happen if the tagger was trained with a different set of morphological features. If you're using a pretrained model, make sure that your models are up to date:
python -m spacy validate

Recommend Projects

React

A declarative, efficient, and flexible JavaScript library for building user interfaces.
Vue.js

🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
Typescript

TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
TensorFlow

An Open Source Machine Learning Framework for Everyone
Django

The Web framework for perfectionists with deadlines.
Laravel

A PHP framework for web artisans
D3

Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

javascript

JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
web

Some thing interesting about web. New door for the world.
server

A server is a program made to process requests and deliver data to clients.
Machine learning

Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Visualization

Some thing interesting about visualization, use data art
Game

Some thing interesting about game, make everyone happy.

Recommend Org

Facebook

We are working to build community through open source technology. NB: members must have two-factor auth.
Microsoft

Open source projects and samples from Microsoft.
Google

Google ❤️ Open Source for everyone.
Alibaba

Alibaba Open Source for everyone
D3

Data-Driven Documents codes.
Tencent

China tencent open source team.