Backport of hashlib.sha3 for 2.7 to 3.5
Home Page: https://docs.python.org/3/library/hashlib.html
License: Other
The pysha3 package has reached its end of life and is no longer supported.
The SHA-3 code was merged into Python 3.6 many years ago. Please use SHA-3 functions from hashlib module directly.
This is the same issue as #8 which was declared invalid, probably because it was missing crucial information on how to reproduce it. This user on StackOverflow managed to stumble upon it: https://stackoverflow.com/questions/46279121/how-can-i-find-keccak-256-hash-in-python
On Python 2.7:
install both sha3 and pysha3 via pip:
python -m pip install sha3 pysha3
Then try the example code:
import sha3
k = sha3.keccak_512()
k.update("data")
k.hexdigest()This seems to import the sha3 module from the sha3 package, not the pysha3 package.
Hi,
After installation according the manual I get this with new() function:
Python 2.7.16 (default, Mar 11 2019, 18:59:25)
[GCC 8.2.1 20181127] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> import hashlib
>>>
>>> import sha3
>>> hashlib.new('sha3_256')
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "/usr/lib64/python2.7/hashlib.py", line 130, in __hash_new
return __get_builtin_constructor(name)(string)
File "/usr/lib64/python2.7/hashlib.py", line 97, in __get_builtin_constructor
raise ValueError('unsupported hash type ' + name)
ValueError: unsupported hash type sha3_256
>>> hashlib
<module 'hashlib' from '/usr/lib64/python2.7/hashlib.pyc'>
>>> dir(hashlib)
['__all__', '__builtins__', '__doc__', '__file__', '__get_builtin_constructor', '__name__', '__package__', '_hashlib', 'algorithms', 'algorithms_available', 'algorithms_guaranteed', 'md5', 'new', 'pbkdf2_hmac', 'sha1', 'sha224', 'sha256', 'sha384', 'sha3_224', 'sha3_256', 'sha3_384', 'sha3_512', 'sha512', 'shake_128', 'shake_256']
>>> dir(hashlib.algorithms)
['__add__', '__class__', '__contains__', '__delattr__', '__doc__', '__eq__', '__format__', '__ge__', '__getattribute__', '__getitem__', '__getnewargs__', '__getslice__', '__gt__', '__hash__', '__init__', '__iter__', '__le__', '__len__', '__lt__', '__mul__', '__ne__', '__new__', '__reduce__', '__reduce_ex__', '__repr__', '__rmul__', '__setattr__', '__sizeof__', '__str__', '__subclasshook__', 'count', 'index']
>>> hashlib.algorithms
('md5', 'sha1', 'sha224', 'sha256', 'sha384', 'sha512')
>>> hashlib.algorithms_available
set(['sha3-512', 'shake128', 'sha3-384', 'sha3-256', 'sha512-224', 'blake2s256', 'shake256', 'blake2b512', 'md4', 'md5', 'sha1', 'sha224', 'sm3', 'sha512-256', 'mdc2', 'sha3-224', 'sha384', 'md5-sha1', 'sha256', 'sha512', 'ripemd160', 'whirlpool'])
>>>
In the readme:
>>> import sha3
>>> k = sha3.keccak_512()
>>> k.update(b"data")
>>> k.hexdigest()
'1065aceeded3a5e4412e2187e919bffeadf815f5bd73d37fe00d384fe29f55f08462fdabe1007b993ce5b8119630e7db93101d9425d6e352e22ffe3dcb56b825'
However, when I do this on my machine:
>>> import sha3
>>> k = sha3.keccak_512()
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
AttributeError: 'module' object has no attribute 'keccak_512'
Also looking a dir(sha3) I see nothing with "keccak"
>>> dir(sha3)
['SHA3224', 'SHA3256', 'SHA3384', 'SHA3512', 'SHAKE128', 'SHAKE256', '_SHA3Base'\
, '_SHAKEBase', '__all__', '__author__', '__builtins__', '__doc__', '__file__', \
'__name__', '__package__', '__path__', '__version__', '_sha3', 'binascii', 'copy\
', 'hashlib', 'sha3_224', 'sha3_256', 'sha3_384', 'sha3_512', 'shake128', 'shake\
256']
It would be awesome if you could also provide Linux "manylinux1" wheels.
This is relatively easy to do on travis-ci.
I got a problem when i installed.
I got error when I try to install it with pip3 install:
src/64/KeccakF-1600-opt64.c:250:2: error: #error "Not yet implemented"
But I can install it successfully with the source code.
Is there something I missed?
And I can see from the changes of version, the big endian has been supported.
My os is s390 and it is big endian.
When building with pypy, I get the following test errors:
$ python setup.py test
running test
running build
running build_py
running build_ext
test_basics (__main__.SHA3_224Tests) ... FAIL
test_hashlib (__main__.SHA3_224Tests) ... ok
test_hmac (__main__.SHA3_224Tests) ... ok
test_vectors (__main__.SHA3_224Tests) ... ok
test_vectors_unaligned (__main__.SHA3_224Tests) ... ok
test_basics (__main__.SHA3_256Tests) ... FAIL
test_hashlib (__main__.SHA3_256Tests) ... ok
test_hmac (__main__.SHA3_256Tests) ... ok
test_vectors (__main__.SHA3_256Tests) ... ok
test_vectors_unaligned (__main__.SHA3_256Tests) ... ok
test_basics (__main__.SHA3_384Tests) ... FAIL
test_hashlib (__main__.SHA3_384Tests) ... ok
test_hmac (__main__.SHA3_384Tests) ... ok
test_vectors (__main__.SHA3_384Tests) ... ok
test_vectors_unaligned (__main__.SHA3_384Tests) ... ok
test_basics (__main__.SHA3_512Tests) ... FAIL
test_hashlib (__main__.SHA3_512Tests) ... ok
test_hmac (__main__.SHA3_512Tests) ... ok
test_vectors (__main__.SHA3_512Tests) ... ok
test_vectors_unaligned (__main__.SHA3_512Tests) ... ok
test_basics (__main__.Shake_128Tests) ... FAIL
test_hashlib (__main__.Shake_128Tests) ... ok
test_hmac (__main__.Shake_128Tests) ... ok
test_vectors (__main__.Shake_128Tests) ... ok
test_vectors_unaligned (__main__.Shake_128Tests) ... ok
test_basics (__main__.Shake_256Tests) ... FAIL
test_hashlib (__main__.Shake_256Tests) ... ok
test_hmac (__main__.Shake_256Tests) ... ok
test_vectors (__main__.Shake_256Tests) ... ok
test_vectors_unaligned (__main__.Shake_256Tests) ... ok
test_basics (__main__.Keccak_224Tests) ... FAIL
test_hashlib (__main__.Keccak_224Tests) ... ok
test_hmac (__main__.Keccak_224Tests) ... ok
test_vectors (__main__.Keccak_224Tests) ... ok
test_vectors_unaligned (__main__.Keccak_224Tests) ... ok
test_basics (__main__.Keccak_256Tests) ... FAIL
test_hashlib (__main__.Keccak_256Tests) ... ok
test_hmac (__main__.Keccak_256Tests) ... ok
test_vectors (__main__.Keccak_256Tests) ... ok
test_vectors_unaligned (__main__.Keccak_256Tests) ... ok
test_basics (__main__.Keccak_384Tests) ... FAIL
test_hashlib (__main__.Keccak_384Tests) ... ok
test_hmac (__main__.Keccak_384Tests) ... ok
test_vectors (__main__.Keccak_384Tests) ... ok
test_vectors_unaligned (__main__.Keccak_384Tests) ... ok
test_basics (__main__.Keccak_512Tests) ... FAIL
test_hashlib (__main__.Keccak_512Tests) ... ok
test_hmac (__main__.Keccak_512Tests) ... ok
test_vectors (__main__.Keccak_512Tests) ... ok
test_vectors_unaligned (__main__.Keccak_512Tests) ... ok
======================================================================
FAIL: test_basics (__main__.SHA3_224Tests)
----------------------------------------------------------------------
Traceback (most recent call last):
File "tests.py", line 110, in test_basics
self.assertRaises(AttributeError, setattr, sha3, "digest", 3)
AssertionError: AttributeError not raised
======================================================================
FAIL: test_basics (__main__.SHA3_256Tests)
----------------------------------------------------------------------
Traceback (most recent call last):
File "tests.py", line 110, in test_basics
self.assertRaises(AttributeError, setattr, sha3, "digest", 3)
AssertionError: AttributeError not raised
======================================================================
FAIL: test_basics (__main__.SHA3_384Tests)
----------------------------------------------------------------------
Traceback (most recent call last):
File "tests.py", line 110, in test_basics
self.assertRaises(AttributeError, setattr, sha3, "digest", 3)
AssertionError: AttributeError not raised
======================================================================
FAIL: test_basics (__main__.SHA3_512Tests)
----------------------------------------------------------------------
Traceback (most recent call last):
File "tests.py", line 110, in test_basics
self.assertRaises(AttributeError, setattr, sha3, "digest", 3)
AssertionError: AttributeError not raised
======================================================================
FAIL: test_basics (__main__.Shake_128Tests)
----------------------------------------------------------------------
Traceback (most recent call last):
File "tests.py", line 110, in test_basics
self.assertRaises(AttributeError, setattr, sha3, "digest", 3)
AssertionError: AttributeError not raised
======================================================================
FAIL: test_basics (__main__.Shake_256Tests)
----------------------------------------------------------------------
Traceback (most recent call last):
File "tests.py", line 110, in test_basics
self.assertRaises(AttributeError, setattr, sha3, "digest", 3)
AssertionError: AttributeError not raised
======================================================================
FAIL: test_basics (__main__.Keccak_224Tests)
----------------------------------------------------------------------
Traceback (most recent call last):
File "tests.py", line 110, in test_basics
self.assertRaises(AttributeError, setattr, sha3, "digest", 3)
AssertionError: AttributeError not raised
======================================================================
FAIL: test_basics (__main__.Keccak_256Tests)
----------------------------------------------------------------------
Traceback (most recent call last):
File "tests.py", line 110, in test_basics
self.assertRaises(AttributeError, setattr, sha3, "digest", 3)
AssertionError: AttributeError not raised
======================================================================
FAIL: test_basics (__main__.Keccak_384Tests)
----------------------------------------------------------------------
Traceback (most recent call last):
File "tests.py", line 110, in test_basics
self.assertRaises(AttributeError, setattr, sha3, "digest", 3)
AssertionError: AttributeError not raised
======================================================================
FAIL: test_basics (__main__.Keccak_512Tests)
----------------------------------------------------------------------
Traceback (most recent call last):
File "tests.py", line 110, in test_basics
self.assertRaises(AttributeError, setattr, sha3, "digest", 3)
AssertionError: AttributeError not raised
----------------------------------------------------------------------
Ran 50 tests in 2.348s
FAILED (failures=10)
Traceback (most recent call last):
File "setup.py", line 123, in <module>
"Topic :: Security :: Cryptography",
File "/usr/lib64/pypy/lib-python/2.7/distutils/core.py", line 151, in setup
dist.run_commands()
File "/usr/lib64/pypy/lib-python/2.7/distutils/dist.py", line 953, in run_commands
self.run_command(cmd)
File "/usr/lib64/pypy/lib-python/2.7/distutils/dist.py", line 972, in run_command
cmd_obj.run()
File "setup.py", line 64, in run
errno = subprocess.check_call([sys.executable, "tests.py"], env=env)
File "/usr/lib64/pypy/lib-python/2.7/subprocess.py", line 186, in check_call
raise CalledProcessError(retcode, cmd)
CalledProcessError: Command '['/tmp/pysha3-1.0.2/_venv/bin/python', 'tests.py']' returned non-zero exit status 1
Hello,
There appears to be two issues with the pysha3 page on pypi:
Cheers
This is a tracking issue for PyPy3 support.
Currently, the missing component is PyUnicode_New. I posted that issue to the pypy project:
https://bitbucket.org/pypy/pypy/issues/2723/undefined-symbol-pyunicode_new-in-pypy3
Clearly this backport module is unmaintained these days. But there are still things using it (like https://github.com/opentimestamps/python-opentimestamps).
There was a security issue (CVE-2022-37454) found in the Keccak implementation bundled, see: python/cpython#98527
According to PyPy upstream (https://bitbucket.org/pypy/pypy/issues/2687/pypy3-is-missing-pystrhexh-breaking-pysha3) pystrhex.h is a private API of CPython and should not be used in the extensions. I would therefore suggest removing the conditional that uses it and always use the local implementation.
On aarch64, ‘pip install pysha3’ builds the wheels from source code and then installs it. It requires the user to have a development environment installed on his system. Also, it takes some time to build the wheels than downloading and extracting the wheels from pypi.
On aarch64, ‘pip install pysha3’ should download the wheels from pypi
@tiran and @Team Please let me know your interest in releasing aarch64 wheels. I can help in this. To start with could you please tell the steps/Ci you are using to publish the wheel in pypi.
#15 is fixed, but no update on https://pypi.org/project/pysha3/#history.
After porting python to our new platform (which forces data type alignment to their size) we have alignment problems with your library. Python3 ptests (Yocto) are failing.
The root cause seems to be in the function int SpongeAbsorb(SpongeInstance *instance, const unsigned char *data, size_t dataByteLen)
(Python-3.9.9/Modules/_sha3/kcp/KeccakSponge.inc)
In this part:
#ifdef SnP_FastLoop_Absorb
/* processing full blocks first */
if ((rateInBytes % (SnP_width/200)) == 0) {
/* fast lane: whole lane rate */
j = SnP_FastLoop_Absorb(instance->state, rateInBytes/(SnP_width/200), curData, dataByteLen - i);
i += j;
curData += j;
}
else {
#endif
curData is char pointer, ao it it goes to SnP_FastLoop_Absorb function, which is defined to KeccakF1600_FastLoop_Absorb(void *state, unsigned int laneCount, const unsigned char *data, size_t dataByteLen) (Python-3.9.9/Modules/_sha3/kcp/KeccakP-1600-opt64.c)
There it is re-typed to UINT64* (UINT64 inDataAsLanes = (UINT64)data;), then data are load from inDataAsLanes addres. There we got sigbus error, because compiler uses aligned load.
Can you make hint to compiler that this data can be unaligned or force the aligment?
Would it be possible to support Python 3.11? We currently get this error when installing pysha3 (no issues on Python 3.10):
× Running setup.py install for pysha3 did not run successfully.
│ exit code: 1
╰─> [20 lines of output]
running install
/opt/hostedtoolcache/Python/3.11.0/x64/lib/python3.11/site-packages/setuptools/command/install.py:34: SetuptoolsDeprecationWarning: setup.py install is deprecated. Use build and pip and other standards-based tools.
warnings.warn(
running build
running build_py
creating build
creating build/lib.linux-x86_64-cpython-311
copying sha3.py -> build/lib.linux-x86_64-cpython-311
running build_ext
building '_pysha3' extension
creating build/temp.linux-x86_64-cpython-311
creating build/temp.linux-x86_64-cpython-311/Modules
creating build/temp.linux-x86_64-cpython-311/Modules/_sha3
gcc -pthread -Wsign-compare -DNDEBUG -g -fwrapv -O3 -Wall -fPIC -DPY_WITH_KECCAK=1 -I/opt/hostedtoolcache/Python/3.11.0/x64/include/python3.11 -c Modules/_sha3/sha3module.c -o build/temp.linux-x86_64-cpython-311/Modules/_sha3/sha3module.o
In file included from Modules/_sha3/sha3module.c:20:
Modules/_sha3/backport.inc:78:10: fatal error: pystrhex.h: No such file or directory
78 | #include "pystrhex.h"
| ^~~~~~~~~~~~
compilation terminated.
error: command '/usr/bin/gcc' failed with exit code 1
[end of output]
note: This error originates from a subprocess, and is likely not a problem with pip.
error: legacy-install-failure
× Encountered error while trying to install package.
╰─> pysha3
Hi,
I'm trying to install pysha3 on windows using the following:
pip install sha3
I get the following error:
C:\Users\Mike\git\ProtoKeeper>pip install sha3 Collecting sha3 Using cached sha3-0.2.1.tar.gz Installing collected packages: sha3 Running setup.py install for sha3 ... error Complete output from command c:\prog\python\python36-32\python.exe -u -c "import setuptools, tokenize;__file__='C:\\Users\\Mike\\AppData\\Local\\Temp\\pip-build-_gzd3oe7\\sha3\\setup.py';f=getattr(tokenize, 'open', open)(__file__);code=f.read().replace('\r\n', '\n');f.close();exec(compile(code, __file__, 'exec'))" install --record C:\Users\Mike\AppData\Local\Temp\pip-3vd4i04y-record\install-record.txt --single-version-externally-managed --compile: running install running build running build_py creating build creating build\lib.win32-3.6 creating build\lib.win32-3.6\sha3 copying sha3\__init__.py -> build\lib.win32-3.6\sha3 running build_ext building '_sha3' extension creating build\temp.win32-3.6 creating build\temp.win32-3.6\Release creating build\temp.win32-3.6\Release\src creating build\temp.win32-3.6\Release\src\64 C:\Program Files (x86)\Microsoft Visual Studio\2017\BuildTools\VC\Tools\MSVC\14.13.26128\bin\HostX86\x86\cl.exe /c /nologo /Ox /W3 /GL /DNDEBUG /MD -Isrc/ -Isrc/64 -Ic:\prog\python\python36-32\include -Ic:\prog\python\python36-32\include "-IC:\Program Files (x86)\Microsoft Visual Studio\2017\BuildTools\VC\Tools\MSVC\14.13.26128\include" "-IC:\Program Files (x86)\Windows Kits\NETFXSDK\4.6.1\include\um" "-IC:\Program Files (x86)\Windows Kits\10\include\10.0.16299.0\ucrt" "-IC:\Program Files (x86)\Windows Kits\10\include\10.0.16299.0\shared" "-IC:\Program Files (x86)\Windows Kits\10\include\10.0.16299.0\um" "-IC:\Program Files (x86)\Windows Kits\10\include\10.0.16299.0\winrt" "-IC:\Program Files (x86)\Windows Kits\10\include\10.0.16299.0\cppwinrt" /Tcsha3.c /Fobuild\temp.win32-3.6\Release\sha3.obj sha3.c sha3.c(33): error C2016: C requires that a struct or union has at least one member error: command 'C:\\Program Files (x86)\\Microsoft Visual Studio\\2017\\BuildTools\\VC\\Tools\\MSVC\\14.13.26128\\bin\\HostX86\\x86\\cl.exe' failed with exit status 2
Am I doing something wrong or is this a bug?
Is it possible to get binary wheels build for python 3.8+?
The README and PyPi page make reference sha3_228(), this looks like a type-o, should be sha3_224()
>>> import hashlib
>>> import sha3
>>> s = hashlib.sha3_228()
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
AttributeError: 'module' object has no attribute 'sha3_228'
>>> s = hashlib.sha3_224()
The TestCommand (implemented in setup.py) subclasses the Distutils Command class directly. This is no longer correct: it should subclass the Setuptools implementation.
The Distutils library is deprecated upstream, and all implementations should instead use their replacements from Setuptools:
As Distutils is deprecated, any usage of functions or objects from distutils is similarly discouraged, and Setuptools aims to replace or deprecate all such uses. This section describes the recommended replacements.
On current Python versions, the use of distutils.core.Command causes the test suite to fail:
$ python3 ./setup.py test
./setup.py:5: DeprecationWarning: The distutils package is deprecated and slated for removal in Python 3.12. Use setuptools or check PEP 632 for potential alternatives
from distutils.core import Command
/usr/lib/python3/dist-packages/_distutils_hack/__init__.py:18: UserWarning: Distutils was imported before Setuptools, but importing Setuptools also replaces the `distutils` module in `sys.modules`. This may lead to undesirable behaviors or errors. To avoid these issues, avoid using distutils directly, ensure that setuptools is installed in the traditional way (e.g. not an editable install), and/or make sure that setuptools is always imported before distutils.
warnings.warn(
/usr/lib/python3/dist-packages/_distutils_hack/__init__.py:33: UserWarning: Setuptools is replacing distutils.
warnings.warn("Setuptools is replacing distutils.")
Traceback (most recent call last):
File "/<<PKGBUILDDIR>>/./setup.py", line 89, in <module>
setup(
[…]
File "/usr/lib/python3/dist-packages/setuptools/_distutils/dist.py", line 540, in _parse_command_opts
raise DistutilsClassError(
distutils.errors.DistutilsClassError: command class <class '__main__.TestCommand'> must subclass Command
For this package, the necessary replacement is: Import setuptools.Command and subclass from that.
The following error comes up when trying to import sha3 on Python 3.6, after using pip to install pysha3 (both 1.0.0 and 1.0.1):
Python 3.6.0 (default, Feb 2 2017, 12:13:44)
[GCC 4.2.1 Compatible Apple LLVM 8.0.0 (clang-800.0.42.1)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> import sha3
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "/Users/Brett/.pyenv/versions/bigchaindb-3.6/lib/python3.6/site-packages/sha3.py", line 8, in <module>
from _sha3 import keccak_224, keccak_256, keccak_384, keccak_512
ImportError: cannot import name 'keccak_224'Inspecting an imported _sha3:
>>> import _sha3
>>> dir(_sha3)
['__doc__', '__file__', '__loader__', '__name__', '__package__', '__spec__', 'implementation', 'keccakopt', 'sha3_224', 'sha3_256', 'sha3_384', 'sha3_512', 'shake_128', 'shake_256'](So everything but the keccak implementations is available)
To install, I used a normal pip install:
pip install pysha3This happens on native OSX (10.11), macOS (10.12), and on a docker container running the official 3.6 alpine image.
I'm not sure what's going on, since it looks like the PY_WITH_KECCAK flag is defined, but maybe it's not getting built with it?
Everything works fine on 3.5.
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
Django
The Web framework for perfectionists with deadlines.
Laravel
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
Microsoft
Open source projects and samples from Microsoft.
Google
Google ❤️ Open Source for everyone.
Alibaba
Alibaba Open Source for everyone
D3
Data-Driven Documents codes.
Tencent
China tencent open source team.