Python libzfs bindings
License: GNU Lesser General Public License v3.0
Python bindings for libzfs
py-libzfs is a fairly straight-forward set of Python bindings for libzfs for ZFS on Linux and FreeBSD.
INSTALLATION
./configure && make install
FEATURES:
QUICK HOWTO:
import libzfs
Get a list of pools:
pools = list(libzfs.ZFS().pools)
Get help:
help(libzfs)
The compilation on FreeBSD/HardenedBSD 11.1 fails with:
libzfs.c:45011:71: error: too few arguments to function call, expected 3, have 2
__pyx_v_ret = zpool_scan(__pyx_v_self->handle, POOL_SCAN_SCRUB);
I remeber compiling py-libzfs on FreeBSD 11.0 successfully, some time ago.
The feature to stop scrubbing, was not only ported to CURRENT, but also went into STABLE (11.1). I don't have the exact kernel version number, since when this happened. My machine is running:
uname -K
1101505I verified the source of fbsd and hbsd and the zpool_scan() indeed expects three arguments.
The problem can solved by checking FREEBSD_VERSION >= 1100000 instead of FREEBSD_VERSION >= 1200044 in the following:
https://github.com/freenas/py-libzfs/blob/3e31c3f01ec6648ec0a10ac6bda2955d4a8044d0/libzfs.pyx#L1869
https://github.com/freenas/py-libzfs/blob/3e31c3f01ec6648ec0a10ac6bda2955d4a8044d0/libzfs.pyx#L1883
https://github.com/freenas/py-libzfs/blob/3e31c3f01ec6648ec0a10ac6bda2955d4a8044d0/libzfs.pyx#L1869
However, when I do this I get the following compile error:
Error compiling Cython file:
------------------------------------------------------------
...
def stop_scrub(self):
cdef int ret
with nogil:
IF EXPERIMENTAL or FREEBSD_VERSION >= 1100044:
ret = libzfs.zpool_scan(self.handle, zfs.POOL_SCAN_NONE, zfs.POOL_SCRUB_NORMAL)
^
------------------------------------------------------------
libzfs.pyx:1884:76: Converting to Python object not allowed without gil
Am stuck at this point.
Hi.
I was thinking about adding github actions CI to this repo. It could produce a valid deb package as a artifact. Also do some basic testing and validation.
What do you think ?
I've been hacking on a rather crude port of this library for Linux, and noticed that certain features are not properly guarded by conditionals, despite the fact that we test for them!
here's my patch:
diff --git a/configure.ac b/configure.ac
index ce0a71b..5b048c8 100644
--- a/configure.ac
+++ b/configure.ac
@@ -14,27 +14,35 @@ define(SECTION_TITLE,
dnl works for checking define statements in c
m4_include([m4/AX_CHECK_DEFINE.m4])
-freebsd_src=${FREEBSD_SRC:=/usr/src}
-
-dnl Setting appropriate CFLAGS
-
-CFLAGS="-nostdinc
--I${freebsd_src}/cddl/lib/libumem
--I${freebsd_src}/sys/cddl/compat/opensolaris/
--I${freebsd_src}/sys/cddl/compat/opensolaris
--I${freebsd_src}/cddl/compat/opensolaris/include
--I${freebsd_src}/cddl/compat/opensolaris/lib/libumem
--I${freebsd_src}/cddl/contrib/opensolaris/lib/libzpool/common
--I${freebsd_src}/sys/cddl/contrib/opensolaris/common/zfs
--I${freebsd_src}/sys/cddl/contrib/opensolaris/uts/common/fs/zfs
--I${freebsd_src}/sys/cddl/contrib/opensolaris/uts/common/sys
--I${freebsd_src}/cddl/contrib/opensolaris/head
--I${freebsd_src}/sys/cddl/contrib/opensolaris/uts/common
--I${freebsd_src}/cddl/contrib/opensolaris/lib/libnvpair
--I${freebsd_src}/cddl/contrib/opensolaris/lib/libuutil/common
--I${freebsd_src}/cddl/contrib/opensolaris/lib/libzfs/common
--I${freebsd_src}/cddl/contrib/opensolaris/lib/libzfs_core/common
-${CFLAGS} -I/usr/include"
+if test `uname -s` == "Linux" ; then
+ ZFS_VERSION='0.7.5'
+ CFLAGS="
+ -I/usr/include/libspl
+ -I/usr/include/libzfs
+ "
+else
+ freebsd_src=${FREEBSD_SRC:=/usr/src}
+
+ dnl Setting appropriate CFLAGS
+
+ CFLAGS="-nostdinc
+ -I${freebsd_src}/cddl/lib/libumem
+ -I${freebsd_src}/sys/cddl/compat/opensolaris/
+ -I${freebsd_src}/sys/cddl/compat/opensolaris
+ -I${freebsd_src}/cddl/compat/opensolaris/include
+ -I${freebsd_src}/cddl/compat/opensolaris/lib/libumem
+ -I${freebsd_src}/cddl/contrib/opensolaris/lib/libzpool/common
+ -I${freebsd_src}/sys/cddl/contrib/opensolaris/common/zfs
+ -I${freebsd_src}/sys/cddl/contrib/opensolaris/uts/common/fs/zfs
+ -I${freebsd_src}/sys/cddl/contrib/opensolaris/uts/common/sys
+ -I${freebsd_src}/cddl/contrib/opensolaris/head
+ -I${freebsd_src}/sys/cddl/contrib/opensolaris/uts/common
+ -I${freebsd_src}/cddl/contrib/opensolaris/lib/libnvpair
+ -I${freebsd_src}/cddl/contrib/opensolaris/lib/libuutil/common
+ -I${freebsd_src}/cddl/contrib/opensolaris/lib/libzfs/common
+ -I${freebsd_src}/cddl/contrib/opensolaris/lib/libzfs_core/common
+ ${CFLAGS} -I/usr/include"
+fi
dnl Setting appropriate LDFLAGS
LDFLAGS="-lzfs -lnvpair -lzfs_core -luutil -lgeom -v"
@@ -93,12 +101,21 @@ AC_COMPILE_IFELSE([AC_LANG_PROGRAM(
SECTION_TITLE([RUNNING TESTS ON libzfs.h HEADER])
-AC_CHECK_HEADERS([libzfs.h], [], [AC_ERROR([A working libzfs header is required])], [
- #include <sys/types.h>
- #include <sys/mnttab.h>
- #include <sys/fs/zfs.h>
- #include <sys/zfs_ioctl.h>
-])
+if test `uname -s` == "Linux" ; then
+ AC_CHECK_HEADERS([libzfs.h], [], [AC_ERROR([A working libzfs header is required])], [
+ #include <sys/types.h>
+ #include <sys/mnttab.h>
+ #include <sys/fs/zfs.h>
+ ])
+else
+ AC_CHECK_HEADERS([libzfs.h], [], [AC_ERROR([A working libzfs header is required])], [
+ #include <sys/types.h>
+ #include <sys/mnttab.h>
+ #include <sys/fs/zfs.h>
+ #include <sys/zfs_ioctl.h>
+ ])
+fi
+
dnl Checking function signature of zpool_scanand here's the output from ./configure:
checking for gcc... gcc
checking whether the C compiler works... yes
checking for C compiler default output file name... a.out
checking for suffix of executables...
checking whether we are cross compiling... no
checking for suffix of object files... o
checking whether we are using the GNU C compiler... yes
checking whether gcc accepts -g... yes
checking for gcc option to accept ISO C89... none needed
-----------------------------------
RUNNING TESTS ON zfs.h HEADER
-----------------------------------
checking how to run the C preprocessor... gcc -E
checking for grep that handles long lines and -e... /bin/grep
checking for egrep... /bin/grep -E
checking for ANSI C header files... no
checking for sys/types.h... yes
checking for sys/stat.h... no
checking for stdlib.h... yes
checking for string.h... yes
checking for memory.h... yes
checking for strings.h... yes
checking for inttypes.h... yes
checking for stdint.h... yes
checking for unistd.h... yes
checking for sys/types.h... (cached) yes
checking for sys/fs/zfs.h... yes
checking for ZFS_MAX_DATASET_NAME_LEN defined in sys/fs/zfs.h... yes
configure: Checking existence of pss_pass_scrub_pause & pss_pass_scrub_spent_paused struct member
configure: OK
configure: Checking typedef pool_scrub_cmt_t in zfs header
configure: OK
--------------------------------------
RUNNING TESTS ON libzfs.h HEADER
--------------------------------------
checking for libzfs.h... yes
configure: Checking function signature of zpool_scan
configure: 2 params
configure: Checking function signature of zfs_send_one
configure: 4 params
configure: Checking EZFS_SCRUB_PAUSED presence
configure: OK
configure: Checking typedef sendflags_t in libzfs header
configure: OK
configure: Checking LZC_SEND_FLAG_EMBED_DATA presence
configure: OK
configure: Checking params for zfs_receive
configure: 6 params
configure: Checking existence of zfs_send_resume and zfs_send_resume_token_to_nvlist
configure: NOT FOUND
checking for ZFS_MAXNAMELEN defined in libzfs.h... no
checking for ZPOOL_MAXNAMELEN defined in libzfs.h... no
configure: Checking for resumable member in recvflags_t struct
configure: OK
configure: Checking for compress member in sendflags_t struct
configure: OK
-------------------------------------------
RUNNING TESTS ON libzfs_core.h HEADER
-------------------------------------------
checking for libzfs_core.h... yes
configure: Verifying lzc_bookmark existence
configure: NOT FOUND
configure: lzc_send_space function not found
-------------------------
GENERATING CONFIG.H
-------------------------
configure: creating ./config.status
config.status: creating pxd/config.pxi
config.status: creating Makefile
and here's the output from make:
/home/igalic/.pyenv/shims/python setup.py build
WARNING: '' not a valid package name; please use only .-separated package names in setup.py
running build
running build_py
running build_ext
cythoning libzfs.pyx to libzfs.c
warning: ./pxd/nvpair.pxd:75:2: 'DATA_TYPE_UINT8_ARRAY' redeclared
Error compiling Cython file:
------------------------------------------------------------
...
IF HAVE_LZC_SEND_FLAG_EMBED_DATA:
class SendFlags(enum.IntEnum):
EMBED_DATA = libzfs.LZC_SEND_FLAG_EMBED_DATA
^
------------------------------------------------------------
libzfs.pyx:323:27: cimported module has no attribute 'LZC_SEND_FLAG_EMBED_DATA'
Error compiling Cython file:
------------------------------------------------------------
...
with nogil:
IF HAVE_LZC_SEND_SPACE == 4:
ret = libzfs.lzc_send_space(c_name, cfromname, 0, &space)
ELSE:
ret = libzfs.lzc_send_space(c_name, cfromname, &space)
^
------------------------------------------------------------
libzfs.pyx:2242:28: cimported module has no attribute 'lzc_send_space'
Error compiling Cython file:
------------------------------------------------------------
...
bookmarks = NVList()
bookmarks['{0}#{1}'.format(self.parent.name, name)] = self.name
c_bookmarks = bookmarks.handle
with nogil:
ret = libzfs.lzc_bookmark(c_bookmarks, NULL)
^
------------------------------------------------------------
libzfs.pyx:2606:24: cimported module has no attribute 'lzc_bookmark'
Error compiling Cython file:
------------------------------------------------------------
...
with nogil:
IF HAVE_LZC_SEND_SPACE == 4:
ret = libzfs.lzc_send_space(c_name, cfromname, 0, &space)
ELSE:
ret = libzfs.lzc_send_space(c_name, cfromname, &space)
^
------------------------------------------------------------
libzfs.pyx:2242:43: Calling gil-requiring function not allowed without gil
Error compiling Cython file:
------------------------------------------------------------
...
bookmarks = NVList()
bookmarks['{0}#{1}'.format(self.parent.name, name)] = self.name
c_bookmarks = bookmarks.handle
with nogil:
ret = libzfs.lzc_bookmark(c_bookmarks, NULL)
^
------------------------------------------------------------
libzfs.pyx:2606:37: Calling gil-requiring function not allowed without gil
building 'libzfs' extension
creating build
creating build/temp.linux-x86_64-3.6
gcc…
libzfs.c:1:2: error: #error Do not use this file, it is the result of a failed Cython compilation.
#error Do not use this file, it is the result of a failed Cython compilation.
^~~~~
error: command 'gcc' failed with exit status 1
Makefile:5: recipe for target 'build' failed
make: *** [build] Error 1
The output of zpool iostat can be obtained with something like
import libzfs
pool = libzfs.ZFS().get("pool-name")
print(
sum(
pool
.root_vdev
.stats
.nvlist
.get("vdev_stats_ex")
.get("vdev_tot_r_lat_histo") # property name
)
)However, it appears this data is only loaded when the pool object is loaded. It appears zpool_refresh_stats will update the cached data in the pool handle to contain the latest stats. This will allow gathering the latest stats without calling libzfs.ZFS() each time
For me, ZFSPool.sync intermittently fails with EINVAL. Lately it's been happening almost every time, at least on some servers. The more frequent failures might've been triggered by upgrading from Python 3.8 to 3.9. Using pdb and gdb together shows that when it fails, the pool_name argument passed to lzc_sync is an empty string. However, pdb shows that in the lowest Python stack frame, the pool.name property looks fine. So somehow that property isn't being passed from Python to C correctly. Notably, pool.name is a property, not an instance variable. From experiment, I found that if I eliminate the property access from the sync method then everything works. But I don't know why. Is there a rule against accessing Python properties from Cython code?
I'll open a PR with my fix; I just don't know why it works.
The pkg py36-libzfs is currently not available in FreeBSD 12 CURRENT.
The package is missing in https://pkg.freebsd.org/FreeBSD:12:amd64/latest/All/ but available in https://pkg.freebsd.org/FreeBSD:11:amd64/latest/All/.
The following small python code leaks memory:
import libzfs
with libzfs.ZFS() as zfs:
for event in zfs.zpool_events(blocking=True, skip_existing_events=True):
pass
In parallell you should generate zfs events (e.g. create/destroy datasets, etc.). Then memory usage will increase.
sudo modprobe -r zfs
python -c "import libzfs; libzfs.ZFS()"
System info
Linux 5.16.11-200.fc35.x86_64 #1 SMP PREEMPT Wed Feb 23 17:08:49 UTC 2022 x86_64 GNU/Linuxbc687e9 added a global lock on zfs objects when used as context manager. Howewer, just when used as context manager. So, this may not protect in every case if the underlying libzfs is not thread safe.
Is this lock still needed? I mean, if I have an application with multiple threads, each thread managing its libzfs.ZFS object, will I still need a global lock to serialize them? Or perhaps, libzfs itself is threadsafe now. I suspect libzfs wants to be threadsafe, so if there is a known issue, then we should report this upstream.
On FreeBSD 12.4 (at least):
libzfs.c:97129:53: error: use of undeclared identifier 'ZFS_PROP_USEROBJUSED'; did you mean 'ZFS_PROP_USERUSED'?
libzfs.c:97141:53: error: use of undeclared identifier 'ZFS_PROP_USEROBJQUOTA'; did you mean 'ZFS_PROP_USERQUOTA'?
libzfs.c:97153:53: error: use of undeclared identifier 'ZFS_PROP_GROUPOBJUSED'; did you mean 'ZFS_PROP_GROUPUSED'?
libzfs.c:97165:53: error: use of undeclared identifier 'ZFS_PROP_GROUPOBJQUOTA'; did you mean 'ZFS_PROP_GROUPQUOTA'?
libzfs.c:97177:53: error: use of undeclared identifier 'ZFS_PROP_PROJECTUSED'; did you mean 'ZFS_PROP_GROUPUSED'?
libzfs.c:97189:53: error: use of undeclared identifier 'ZFS_PROP_PROJECTQUOTA'
libzfs.c:97201:53: error: use of undeclared identifier 'ZFS_PROP_PROJECTOBJUSED'
libzfs.c:97213:53: error: use of undeclared identifier 'ZFS_PROP_PROJECTOBJQUOTA'
See also FreeBSD bug 269308
Hi,
the build of py-libzfs fails on FreeBSD-current due to a missing zpool_Search_import(). As such iocage can not be build and the already existing iocage segfaults.
What's the replacement for zpool_search_import()?
Bye,
Alexander.
Here's the error log:
building 'libzfs' extension
creating build
creating build/temp.freebsd-14.0-CURRENT-amd64-3.8
cc -Wno-unused-result -Wsign-compare -Wunreachable-code -DNDEBUG -O2 -pipe -fstack-protector-strong -fno-strict-aliasing -O2 -pipe -fstack-protector-strong -fno-strict-aliasing -fPIC -I/usr/local/include/python3.8 -c libzfs.c -o build/temp.freebsd-14.0-CURRENT-a
libzfs.c:817:11: warning: 'likely' macro redefined [-Wmacro-redefined]
#define likely(x) __builtin_expect(!!(x), 1)
^
/usr/src/sys/contrib/openzfs/include/sys/zfs_context.h:131:9: note: previous definition is here
#define likely(x) __builtin_expect((x), 1)
^
libzfs.c:818:11: warning: 'unlikely' macro redefined [-Wmacro-redefined]
#define unlikely(x) __builtin_expect(!!(x), 0)
^
/usr/src/sys/contrib/openzfs/include/sys/zfs_context.h:132:9: note: previous definition is here
#define unlikely(x) __builtin_expect((x), 0)
^
libzfs.c:5474:9: error: expected ')'
CYTHON_FALLTHROUGH;
^
libzfs.c:276:49: note: expanded from macro 'CYTHON_FALLTHROUGH'
#define CYTHON_FALLTHROUGH __attribute__((fallthrough))
^
/usr/src/sys/contrib/openzfs/lib/libspl/include/sys/feature_tests.h:37:22: note: expanded from macro 'fallthrough'
#define fallthrough ((void)0)
^
libzfs.c:5476:9: error: expected ')'
CYTHON_FALLTHROUGH;
^
Hello!
I am opening this GitHub Issue just as an FYI, and so that Google can find it. I was interested in getting py-libzfs to work with ZFS on Linux (ZoL), and ran into build-breaking issues. I'm opening this issue to record my setup, the issues that I found, and (for what it's worth) my thoughts on those issues. I know that you guys are only using this on FreeBSD, so I'm not asking for you to work on it, I'm just leaving it in case anyone in the future find this and is interested!
All of my work was done on a CentOS 7.5 system running ZoL 0.7.11. Architecture was amd64.
In addition to the normal packages that come with a minimal system running ZoL, I had the install the following additional packages: git gcc autoconf python-devel Cython python-setuptools python2-pip libzfs2-devel kmod-zfs-devel. For CentOS 7, the system Python is 2.7. I did not check other Pythons, but the next target to check would likely be Python 3.4, because EPEL has pre-built Python 3.4 packages for setuptools and Cython.
Because some of CentOS 7's system Python packages were too old, I installed newer copies of setuptools and Cython using the command pip install --user --upgrade setuptools Cython.
That's my environment. Next up is the list of file changes that needed to be made, in order to get the build going.
First, configure.ac needed three changes:
• In CFLAGS, remove -nostdinc: The normal include paths can (and need to) be searched on Linux.
• In CFLAGS, add -D_LARGEFILE64_SOURCE. This is a feature test macro which, when defined, sets up the __rlim64_t typedef that is needed by ZoL.
• In LDFLAGS, remove -lgeom, which does not exist in Linux. Although I never got a successful build, I did not seem to run into anything that needed this library.
Next, in pxd/nvpair.pxd, one change was needed:
• Change from "nvpair.h" to from "sys/nvpair.h". This is because ZoL places nvpair.h into the sys sub-directory of its include dir.
Finally, before running configure and make, the CFLAGS environment variable needed to be set to -I/usr/include/libzfs -I/usr/include/libspl -I/usr/src/zfs-0.7.11/include. The first two paths are the paths for ZFS' and SPL's headers, and can be found at configure-time by running pkg-config libzfs --cflags. The last path is to where the kmod-zfs-devel package's headers are installed, and that package provides sys/zfs_ioctl.h. Maybe the ZoL team would be able to move sys/zfs_ioctl.h into the normal libzfs2-devel package?
Finally, the PYTHON environment variable should be set to the output of the which python command, so that the appropriate Python executable path is set to run setup.py.
With the environment specified, and the above changes, configure was able to run without errors. Cython was also able to run without errors. Unfortunately, the build still failed.
You can see the make output at https://gist.github.com/akkornel/0b2dd3ba3066706146d9c3c8b5a45e82
Some of the problems seem to be because ZoL has changed some function signatures:
• zfs_prop_valid_for_type has an additional argument, thanks to openzfs/zfs@962d524.
• zpool_scan has an additional argument, thanks to openzfs/zfs@0ea05c6
• renameflags_t does not exist. Instead, zfs_rename takes a zfs_handle_t *, a const char *, and two boolean_ts. The first boolean controls recursion, and the latter controls force-unmounting.
• zpool_read_label has an additional argument, thanks to openzfs/zfs@7d90f56
When packaged, ZoL uses the name libzfs2 for its library package. My assumption is that the above API changes are the reason for the library package numbering bump.
There are a few enum values that ZoL doesn't have:
• zpool_status_t does not have ZPOOL_STATUS_NON_NATIVE_ASHIFT as a possible value.
• vdev_aux_t does not have VDEV_AUX_ASHIFT_TOO_BIG as a possible value (although it does have VDEV_AUX_BAD_ASHIFT, which you do not).
Also, there are two instances of Cython doing something weird:
• lzc_send_space uses the four-argument version, but instead of sending an integer as the third parameter (LZC_SEND_FLAG_EMBED_DATA, the only member of enum lzc_send_flags), the .pyx code just has a bare 0, and Cython is turning that 0 into a call to PyLong_FromLong(0), which returns a PyObject *.
• In the call to libzfs.zfs_path_to_zhandle, something similar is happening: The last argument is supposed to be a zfs_type_t (an enum), but for some reason the expression DatasetType.FILESYSTEM.value is not being converted into an int, it's being converted into a struct PyObject *.
There's also one likely ZoL bug:
• For the call to zfs_destroy_snaps, you provide a const char * but ZoL takes a non-const char *. This is probably a bug in ZoL, because the string passed to zfs_destroy_snaps is immediately placed into a destroydata struct, which uses const char *.
So, it's a fair list of differences. Trying to fix this interests me, but I can't find the time for it right now, so I'm going to open and then close this issue. If you have any questions about my setup or what I'm doing, please let me know. Thanks much!
(originally filed with iocage project at iocage/iocage#602, but they say it's really a py-libzfs error, so I'll file it here.)
Iocage is assigned the zpool "iocage."
storm~;iocage --version
Version 1.0 ALPHA 1
storm~;uname -a
FreeBSD storm 12.0-ALPHA3 FreeBSD 12.0-ALPHA3 #9 r338296: Fri Aug 24 15:08:35 EDT 2018 root@storm:/usr/obj/usr/src/amd64.amd64/sys/GENERIC amd64
storm~;iocage list -r
+---------------+
| Bases fetched |
+===============+
| 11.2-RELEASE |
+---------------+
| 12.0-ALPHA2 |
+---------------+
storm~;iocage list
+-----+------+-------+--------------+---------------+
| JID | NAME | STATE | RELEASE | IP4 |
+=====+======+=======+==============+===============+
| 4 | wdb1 | up | 11.2-RELEASE | 203.0.113.236 |
+-----+------+-------+--------------+---------------+
| - | wdb2 | down | 12.0-ALPHA2 | 203.0.113.237 |
+-----+------+-------+--------------+---------------+
| 1 | www1 | up | 11.2-RELEASE | 203.0.113.234 |
+-----+------+-------+--------------+---------------+
| - | www2 | down | 12.0-ALPHA2 | 203.0.113.235 |
+-----+------+-------+--------------+---------------+
Supply the commands used, along with any steps to recreate it.
Traceback (most recent call last):
File "/usr/local/bin/iocage", line 10, in
sys.exit(cli())
File "/usr/local/lib/python3.6/site-packages/click/core.py", line 722, in call
return self.main(*args, **kwargs)
File "/usr/local/lib/python3.6/site-packages/click/core.py", line 697, in main
rv = self.invoke(ctx)
File "/usr/local/lib/python3.6/site-packages/click/core.py", line 1066, in invoke
return _process_result(sub_ctx.command.invoke(sub_ctx))
File "/usr/local/lib/python3.6/site-packages/click/core.py", line 895, in invoke
return ctx.invoke(self.callback, **ctx.params)
File "/usr/local/lib/python3.6/site-packages/click/core.py", line 535, in invoke
return callback(*args, **kwargs)
File "/usr/local/lib/python3.6/site-packages/iocage_cli/start.py", line 54, in cli
ioc.IOCage(jail=jail, rc=rc).start()
File "/usr/local/lib/python3.6/site-packages/iocage_lib/iocage.py", line 1582, in start
conf = ioc_json.IOCJson(path, silent=self.silent).json_load()
File "/usr/local/lib/python3.6/site-packages/iocage_lib/ioc_json.py", line 224, in json_load
if jail_dataset.mountpoint is None:
File "libzfs.pyx", line 2311, in libzfs.ZFSDataset.mountpoint.get
UnicodeDecodeError: 'ascii' codec can't decode byte 0xee in position 148: ordinal not in range(128)
storm~;iocage get -r boot
Traceback (most recent call last):
File "/usr/local/bin/iocage", line 10, in
sys.exit(cli())
File "/usr/local/lib/python3.6/site-packages/click/core.py", line 722, in call
return self.main(*args, **kwargs)
File "/usr/local/lib/python3.6/site-packages/click/core.py", line 697, in main
rv = self.invoke(ctx)
File "/usr/local/lib/python3.6/site-packages/click/core.py", line 1066, in invoke
return _process_result(sub_ctx.command.invoke(sub_ctx))
File "/usr/local/lib/python3.6/site-packages/click/core.py", line 895, in invoke
return ctx.invoke(self.callback, **ctx.params)
File "/usr/local/lib/python3.6/site-packages/click/core.py", line 535, in invoke
return callback(*args, **kwargs)
File "/usr/local/lib/python3.6/site-packages/iocage_cli/get.py", line 122, in cli
jails = ioc.IOCage().get(prop, recursive=True)
File "/usr/local/lib/python3.6/site-packages/iocage_lib/iocage.py", line 1102, in get
ioc_json.IOCJson(path).json_get_value(prop)
File "/usr/local/lib/python3.6/site-packages/iocage_lib/ioc_json.py", line 565, in json_get_value
conf = self.json_load()
File "/usr/local/lib/python3.6/site-packages/iocage_lib/ioc_json.py", line 224, in json_load
if jail_dataset.mountpoint is None:
File "libzfs.pyx", line 2311, in libzfs.ZFSDataset.mountpoint.get
UnicodeDecodeError: 'ascii' codec can't decode byte 0xf4 in position 135: ordinal not in range(128)
Traceback (most recent call last):
File "/usr/local/bin/iocage", line 10, in
sys.exit(cli())
File "/usr/local/lib/python3.6/site-packages/click/core.py", line 722, in call
return self.main(*args, **kwargs)
File "/usr/local/lib/python3.6/site-packages/click/core.py", line 697, in main
rv = self.invoke(ctx)
File "/usr/local/lib/python3.6/site-packages/click/core.py", line 1066, in invoke
return _process_result(sub_ctx.command.invoke(sub_ctx))
File "/usr/local/lib/python3.6/site-packages/click/core.py", line 895, in invoke
return ctx.invoke(self.callback, **ctx.params)
File "/usr/local/lib/python3.6/site-packages/click/core.py", line 535, in invoke
return callback(*args, **kwargs)
File "/usr/local/lib/python3.6/site-packages/iocage_cli/restart.py", line 41, in cli
ioc.IOCage(jail=jail).restart(soft)
File "/usr/local/lib/python3.6/site-packages/iocage_lib/iocage.py", line 1268, in restart
self.stop()
File "/usr/local/lib/python3.6/site-packages/iocage_lib/iocage.py", line 1641, in stop
conf = ioc_json.IOCJson(path, silent=self.silent, stop=True
File "/usr/local/lib/python3.6/site-packages/iocage_lib/ioc_json.py", line 224, in json_load
if jail_dataset.mountpoint is None:
File "libzfs.pyx", line 2311, in libzfs.ZFSDataset.mountpoint.get
UnicodeDecodeError: 'ascii' codec can't decode byte 0xee in position 148: ordinal not in range(128)
I destroy the 12.0-ALPHA2 jails, and I can restart www1.
storm~;iocage restart www1
I still can't do a get:
Traceback (most recent call last):
File "/usr/local/bin/iocage", line 10, in
sys.exit(cli())
File "/usr/local/lib/python3.6/site-packages/click/core.py", line 722, in call
return self.main(*args, **kwargs)
File "/usr/local/lib/python3.6/site-packages/click/core.py", line 697, in main
rv = self.invoke(ctx)
File "/usr/local/lib/python3.6/site-packages/click/core.py", line 1066, in invoke
return _process_result(sub_ctx.command.invoke(sub_ctx))
File "/usr/local/lib/python3.6/site-packages/click/core.py", line 895, in invoke
return ctx.invoke(self.callback, **ctx.params)
File "/usr/local/lib/python3.6/site-packages/click/core.py", line 535, in invoke
return callback(*args, **kwargs)
File "/usr/local/lib/python3.6/site-packages/iocage_cli/get.py", line 122, in cli
jails = ioc.IOCage().get(prop, recursive=True)
File "/usr/local/lib/python3.6/site-packages/iocage_lib/iocage.py", line 1102, in get
ioc_json.IOCJson(path).json_get_value(prop)
File "/usr/local/lib/python3.6/site-packages/iocage_lib/ioc_json.py", line 565, in json_get_value
conf = self.json_load()
File "/usr/local/lib/python3.6/site-packages/iocage_lib/ioc_json.py", line 224, in json_load
if jail_dataset.mountpoint is None:
File "libzfs.pyx", line 2311, in libzfs.ZFSDataset.mountpoint.get
UnicodeDecodeError: 'ascii' codec can't decode byte 0xf4 in position 135: ordinal not in range(128)
I'm trying to port https://github.com/volkerp/zfstui to py-libzfs, which uses some rather simple interactions with the CLI tools to show it's data. Are you open to expanding the coverage of the library?
I haven't gotten too far into it yet but what I'm immediately missing is
zpool history`checking for gcc... gcc
checking whether the C compiler works... yes
checking for C compiler default output file name... a.out
checking for suffix of executables...
checking whether we are cross compiling... no
checking for suffix of object files... o
checking whether we are using the GNU C compiler... yes
checking whether gcc accepts -g... yes
checking for gcc option to accept ISO C89... none needed
checking how to run the C preprocessor... gcc -E
checking for grep that handles long lines and -e... /usr/bin/grep
checking for egrep... /usr/bin/grep -E
checking for ANSI C header files... yes
checking for sys/types.h... yes
checking for sys/stat.h... yes
checking for stdlib.h... yes
checking for string.h... yes
checking for memory.h... yes
checking for strings.h... yes
checking for inttypes.h... yes
checking for stdint.h... yes
checking for unistd.h... yes
checking for sys/types.h... (cached) yes
checking for sys/fs/zfs.h... no
configure: error: A working zfs header is required
`
Pkg config:
pkg-config --cflags libzfs
-I/usr/include/libzfs -I/usr/include/libspl -I/usr/include/blkid -I/usr/include/uuid
find /usr/include/libzfs -name zfs.h
/usr/include/libzfs/sys/fs/zfs.h
/usr/include/libzfs/sys/fm/fs/zfs.h
With zfs-1.8.5 working fine...
I haven't found any stubs with type hintings for libzfs (neither here nor as 3rd-party package), so that static analysis tools like mypy can validate code which uses this library.
I'm currently writing a stubs file while developing a Python tool. So my question is if this project would accept a stubs file for libzfs? Then I would try to complete mine as good as possible and create a PR.
After the recent cb31b54 commit, which says the change is to "Properly handle ZPOOL_STATUS_INCOMPATIBLE_FEAT" (but does a lot more than that message documents), the build of py-libzfs breaks on systems which don't have ZPOOL_STATUS_ERRATA nor ZPOOL_STATUS_REBUILDING (such as FreeBSD 12).
After successfully mounting a ZFS dataset, the value of its 'mounted' property doesn't get updated from 'no' to 'yes'.
I think this is a bug, but if it's an expected behaviour, please close this issue.
Here's a minimal working example to demonstrate the issue:
#!/usr/bin/env python3
from libzfs import ZFS, DatasetType
zfs = ZFS()
pool = zfs.get("tank")
ds_name = "tank/mydataset"
# Create dataset
pool.create(ds_name, {}, fstype=DatasetType.FILESYSTEM, sparse_vol=False, create_ancestors=False)
ds = zfs.get_dataset(ds_name)
# Print 'mounted' property before mount
print(f"BEFORE mount: ds.properties['mounted'].value = {ds.properties['mounted'].value}")
# Mount dataset
ds.mount()
# Optional: force properties refresh
ds.properties['mounted'].refresh()
ds.properties.refresh()
# Print 'mounted' property after mount
print(f"AFTER mount: ds.properties['mounted'].value = {ds.properties['mounted'].value}")The output is the following:
BEFORE mount: ds.properties['mounted'].value = no
AFTER mount: ds.properties['mounted'].value = no
I'm not even sure if those refresh() methods actually do what I think they do, but I added them anyway. Even so, they don't seem to affect the 'mounted' property state.
I can reproduce the issue in the following environment:
How does this project relate to https://github.com/zfsonlinux/zfs/tree/master/contrib/pyzfs ?
Could pool.features be exposed a dict similar to pool.properties?
OS : Debian Bullseye (11)
Kernel : 5.10.0-15-amd64
ZFS Version : 2.1.4-1~bpo11+1
Impacted version : release/22.02.4
While using py-libzfs with Asyncio, accessing properties of several datasets concurrently causes a segfault in accessing getmntany to retrieve the 'mountpoint' property.
We were able to reproduce easily with this simple script.
from concurrent.futures import ThreadPoolExecutor
import time
import libzfs
dataset_name_fmt = 'pool/tank/my-dataset'
num = 200
def async_test(zfs_hdl, i):
ds_name = f"{dataset_name_fmt}-{i}"
ds = zfs_hdl.get_dataset(ds_name)
return ds.properties['used'].rawvalue
def main():
with ThreadPoolExecutor() as executor:
res = []
zfs_hdl = libzfs.ZFS()
for i in range(num):
res.append(executor.submit(async_test, zfs_hdl, i))
for r in res:
print(r.result())
if __name__ == '__main__':
main()
Below, the gdb --args python3 test_script.py output :
Thread 13 "python3" received signal SIGSEGV, Segmentation fault.
[Switching to Thread 0x7fffdcff9700 (LWP 634278)]
0x00007ffff7cd6b94 in ?? () from /lib/x86_64-linux-gnu/libc.so.6
(gdb) bt
#0 0x00007ffff7cd6b94 in ?? () from /lib/x86_64-linux-gnu/libc.so.6
#1 0x00007ffff7cb4a1c in _IO_getline_info () from /lib/x86_64-linux-gnu/libc.so.6
#2 0x00007ffff7cbe675 in fgets_unlocked () from /lib/x86_64-linux-gnu/libc.so.6
#3 0x00007ffff7d35736 in ?? () from /lib/x86_64-linux-gnu/libc.so.6
#4 0x00007ffff7d35fd4 in getmntent_r () from /lib/x86_64-linux-gnu/libc.so.6
#5 0x00007ffff74a2ce4 in _sol_getmntent () from /lib/x86_64-linux-gnu/libzfs_core.so.3
#6 0x00007ffff74a2d73 in getmntany () from /lib/x86_64-linux-gnu/libzfs_core.so.3
#7 0x00007ffff74c60d9 in libzfs_mnttab_find () from /lib/x86_64-linux-gnu/libzfs.so.4
#8 0x00007ffff74c6a18 in ?? () from /lib/x86_64-linux-gnu/libzfs.so.4
#9 0x00007ffff74c882e in zfs_prop_get () from /lib/x86_64-linux-gnu/libzfs.so.4
#10 0x00007ffff757d6cf in __pyx_pf_6libzfs_11ZFSProperty_8refresh (__pyx_v_self=0x7fffc406e580) at libzfs.c:39479
#11 __pyx_pw_6libzfs_11ZFSProperty_9refresh (__pyx_v_self=<libzfs.ZFSProperty at remote 0x7fffc406e580>, unused=<optimized out>) at libzfs.c:39405
#12 0x00007ffff7594af6 in __Pyx_PyObject_CallMethO (arg=0x0, func=<built-in method refresh of libzfs.ZFSProperty object at remote 0x7fffc406e580>)
at libzfs.c:108486
#13 __Pyx_PyObject_CallNoArg (func=<built-in method refresh of libzfs.ZFSProperty object at remote 0x7fffc406e580>) at libzfs.c:42976
#14 0x00007ffff75adf55 in __pyx_pf_6libzfs_15ZFSPropertyDict_2refresh (__pyx_v_self=0x7fffdc63fd10) at libzfs.c:64773
#15 __pyx_pw_6libzfs_15ZFSPropertyDict_3refresh (__pyx_v_self={}, unused=<optimized out>) at libzfs.c:64533
#16 0x00007ffff7594af6 in __Pyx_PyObject_CallMethO (arg=0x0, func=<built-in method refresh of libzfs.ZFSPropertyDict object at remote 0x7fffdc63fd10>)
at libzfs.c:108486
#17 __Pyx_PyObject_CallNoArg (func=<built-in method refresh of libzfs.ZFSPropertyDict object at remote 0x7fffdc63fd10>) at libzfs.c:42976
#18 0x00007ffff7594de8 in __pyx_pf_6libzfs_9ZFSObject_10properties___get__ (__pyx_v_self=<optimized out>) at libzfs.c:67695
#19 __pyx_pw_6libzfs_9ZFSObject_10properties_1__get__ (__pyx_v_self=<optimized out>) at libzfs.c:2096
#20 __pyx_getprop_6libzfs_9ZFSObject_properties (o=<optimized out>, x=<optimized out>) at libzfs.c:30081
#21 0x0000000000529f3e in getset_get (descr=0x7ffff768d640, obj=<libzfs.ZFSDataset at remote 0x7fffdc641200>, type=<optimized out>)
at ../Objects/descrobject.c:185
#22 0x0000000000526f68 in _PyObject_GenericGetAttrWithDict (obj=<optimized out>, name='properties', dict=<optimized out>, suppress=0)
at ../Objects/object.c:1201
#23 0x0000000000511ed1 in _PyEval_EvalFrameDefault (tstate=<optimized out>, f=<optimized out>, throwflag=<optimized out>) at ../Python/ceval.c:2994
#24 0x0000000000528b63 in _PyEval_EvalFrame (throwflag=0,
f=Frame 0x7ffff6f18200, for file /root/test_segv_lib.py, line 16, in test_coucou (zfs_hdl=<libzfs.ZFS at remote 0x7ffff6f61990>, i=56, v='test-constantin-56', ds=<libzfs.ZFSDataset at remote 0x7fffdc641200>), tstate=0xa8a5c0) at ../Include/internal/pycore_ceval.h:40
#25 function_code_fastcall (globals=<optimized out>, nargs=2, args=<optimized out>, co=<optimized out>, tstate=0xa8a5c0) at ../Objects/call.c:330
#26 _PyFunction_Vectorcall (func=<optimized out>, stack=<optimized out>, nargsf=<optimized out>, kwnames=<optimized out>) at ../Objects/call.c:367
#27 0x0000000000513e8b in do_call_core (kwdict={}, callargs=(<libzfs.ZFS at remote 0x7ffff6f61990>, 56), func=<function at remote 0x7ffff7873040>,
tstate=<optimized out>) at ../Python/ceval.c:5120
I was able to prevent the segfault by reactivating the GIL during libzfs.zfs_prop_get() call in ZFSProperty.refresh() and in ZFS.__dataset_handles()
I do believe that the nogil behaviour was intended. Could you please explain why the nogil is important for you here ? If you want I already have a patch and can provide a PR to re-acquire the GIL upon libzfs.zfs_prop_get() calls.
If any information is missing I remain at your disposal.
The way to lookup a dataset's properties is to do
dataset.properties['mypropname'].valueUnfortunately, the ZFSObject::properties property constructs a ZFSPropertyDict, which looks up all properties on the dataset, native and user. Some of these properties can be overridden at mount time, so libzfs calls getfsstat to lookup up mountpoint info. That's roughly equivalent to typing mount on the command line. For servers with many mounted file systems, it can take a long time. If your application needs to lookup a single property on, say, all snapshots of a dataset, it can take an extremely long time!
It would be very helpful of py-libzfs had a method that could retrieve the value of a single property. Something like this:
cdef class ZFSObject(object):
def property(self, propname):
""" Look up the value of a single property """
...What version of ZoL is this compatible with? I couldn't find it stated anywhere.
I get compile errors on 0.8, 0.7 seems to work;
checking for gcc... gcc
checking whether the C compiler works... yes
checking for C compiler default output file name... a.out
checking for suffix of executables...
checking whether we are cross compiling... no
checking for suffix of object files... o
checking whether we are using the GNU C compiler... yes
checking whether gcc accepts -g... yes
checking for gcc option to accept ISO C89... none needed
-----------------------------------
RUNNING TESTS ON zfs.h HEADER
-----------------------------------
checking how to run the C preprocessor... gcc -E
checking for grep that handles long lines and -e... /nix/store/wnjv27b3j6jfdl0968xpcymlc7chpqil-gnugrep-3.3/bin/grep
checking for egrep... /nix/store/wnjv27b3j6jfdl0968xpcymlc7chpqil-gnugrep-3.3/bin/grep -E
checking for ANSI C header files... yes
checking for sys/types.h... yes
checking for sys/stat.h... no
checking for stdlib.h... yes
checking for string.h... yes
checking for memory.h... yes
checking for strings.h... yes
checking for inttypes.h... yes
checking for stdint.h... yes
checking for unistd.h... yes
checking for sys/types.h... (cached) yes
checking for sys/fs/zfs.h... yes
checking for ZFS_MAX_DATASET_NAME_LEN defined in sys/fs/zfs.h... yes
configure: Checking signature of zfs_prop_valid_for_type in zfs header
configure: 3 PARAMS
configure: Checking existence of VDEV_AUX_ASHIFT_TOO_BIG enum in zfs header
configure: Not found
configure: Checking existence of pss_pass_scrub_pause & pss_pass_scrub_spent_paused struct member
configure: OK
configure: Checking existence of pss_pass_issued struct member
configure: OK
configure: Checking existence of zpool_errata_t
configure: OK
configure: Checking typedef pool_scrub_cmt_t in zfs header
configure: OK
configure: Verifying existence of zpool_rewind_policy_t
configure: NOT FOUND
configure: Verifying existence of zpool_load_policy_t
configure: OK
----------------------------------------
RUNNING TESTS ON libzutil.h HEADER
----------------------------------------
checking for libzutil.h... yes
configure: OK
configure: Checking existence of zpool_read_label in libzutil.h header
configure: OK
configure: Checking existence of zpool_search_import in libzutil.h header
configure: OK
--------------------------------------
RUNNING TESTS ON libzfs.h HEADER
--------------------------------------
checking for libzfs.h... yes
configure: Checking attribute name for verbose in sendflags_t struct
configure: Attribute named as verbose in sendflags_t struct
configure: Checking function signature zvol_volsize_to_reservation
configure: 2 PARAMS
configure: Checking existence of zpool_search_import in libzfs.h header
configure: NOT FOUND
configure: Checking zfs_iter_snapshots parameters
configure: 4 params
configure: Checking existence of zpool_read_label with 2 params in libzfs.h header
configure: NOT FOUND
configure: Checking existence of zpool_read_label with 3 params in libzfs.h header
configure: Checking existence of vs_*_ashift in fs/zfs.h header
configure: NOT FOUND
configure: Checking function signature of zpool_get_status
configure: 3 PARAMS
configure: Checking function signature of zpool_scan
configure: 3 params
configure: Checking function signature of zfs_send_one
configure: 4 params
configure: Checking EZFS_SCRUB_PAUSED presence
configure: OK
configure: Checking ZPOOL_STATUS_NON_NATIVE_ASHIFT presence
configure: NOT FOUND
configure: Checking typedef sendflags_t in libzfs header
configure: OK
configure: Checking LZC_SEND_FLAG_EMBED_DATA presence
configure: OK
configure: Checking params for zfs_receive
configure: 6 params
configure: Checking existence of zfs_send_resume and zfs_send_resume_token_to_nvlist
configure: OK
checking for ZFS_MAXNAMELEN defined in libzfs.h... no
configure: Checking existence of renameflags_t in libzfs.h
configure: NOT FOUND
checking for ZPOOL_MAXNAMELEN defined in libzfs.h... no
configure: Checking for resumable member in recvflags_t struct
configure: OK
configure: Checking for compress member in sendflags_t struct
configure: OK
-------------------------------------------
RUNNING TESTS ON libzfs_core.h HEADER
-------------------------------------------
checking for libzfs_core.h... yes
configure: Verifying lzc_bookmark existence
configure: OK
configure: Verifying existence and params of lzc_send_space
configure: 4 params
-----------------------------------------
RUNNING TESTS ON zfs_ioctl.h HEADER
-----------------------------------------
configure: Verifying existence of zfs_ioctl.h
checking for zfs_ioctl.h... no
configure: NOT FOUND
--------------------------------------
RUNNING TESTS ON nvpair.h HEADER
--------------------------------------
checking for sys/nvpair.h... yes
configure: Nvpair.h header under sys
-------------------------
GENERATING CONFIG.H
-------------------------
configure: GENERATED pxd/config.pxi
--------------------------
GENERATING config.py
--------------------------
configure: GENERATED config.py
configure: creating ./config.status
config.status: creating pxd/config.pxi
config.status: creating config.py
config.status: creating Makefile
running bdist_wheel
running build
running build_ext
cythoning libzfs.pyx to libzfs.c
building 'libzfs' extension
creating build
creating build/temp.linux-x86_64-3.7
gcc -Wno-unused-result -Wsign-compare -DNDEBUG -g -fwrapv -O3 -Wall -fPIC -I/nix/store/dwi186iqm6p3kfncfsl62fbfwfry1yci-python3-3.7.3/include/python3.7m -c libzfs.c -o build/temp.linux-x86_64-3.7/libzfs.o -g -O2 -I/nix/store/7h8v8sl8rz49w0nnywyian3xnk2c9qx3-zfs-user-0.8.1-dev/include/libzfs/ -I/nix/store/7h8v8sl8rz49w0nnywyian3xnk2c9qx3-zfs-user-0.8.1-dev/include/libspl/ -D_MACHINE_ENDIAN_H_ -DHAVE_ISSETUGID -Werror=implicit-function-declaration
libzfs.c: In function ‘__pyx_f_6libzfs_3ZFS___snapshot_details’:
libzfs.c:19064:12: error: too few arguments to function ‘zfs_iter_snapshots’
(void)(zfs_iter_snapshots(__pyx_v_handle, __pyx_v_simple_handle, __pyx_f_6libzfs_3ZFS___snapshot_details, ((void *)__pyx_v_snap_list)));
^~~~~~~~~~~~~~~~~~
In file included from libzfs.c:599:0:
/nix/store/7h8v8sl8rz49w0nnywyian3xnk2c9qx3-zfs-user-0.8.1-dev/include/libzfs/libzfs.h:592:12: note: declared here
extern int zfs_iter_snapshots(zfs_handle_t *, boolean_t, zfs_iter_f, void *,
^~~~~~~~~~~~~~~~~~
libzfs.c:19855:52: warning: passing argument 2 of ‘zfs_is_mounted’ from incompatible pointer type [-Wincompatible-pointer-types]
__pyx_v_ret = zfs_is_mounted(__pyx_v_handle, (&__pyx_v_mntpt));
^
In file included from libzfs.c:599:0:
/nix/store/7h8v8sl8rz49w0nnywyian3xnk2c9qx3-zfs-user-0.8.1-dev/include/libzfs/libzfs.h:770:18: note: expected ‘char **’ but argument is of type ‘const char **’
extern boolean_t zfs_is_mounted(zfs_handle_t *, char **);
^~~~~~~~~~~~~~
libzfs.c: In function ‘__pyx_gb_6libzfs_10ZFSDataset_9snapshots_2generator16’:
libzfs.c:59964:16: error: too few arguments to function ‘zfs_iter_snapshots’
(void)(zfs_iter_snapshots(__pyx_cur_scope->__pyx_v_self->__pyx_base.handle, 0, ((struct __pyx_vtabstruct_6libzfs_ZFSDataset *)__pyx_cur_scope->__pyx_v_self->__pyx_vtab)->__pyx___iterate, ((void *)(&__pyx_cur_scope->__pyx_v_iter))));
^~~~~~~~~~~~~~~~~~
In file included from libzfs.c:599:0:
/nix/store/7h8v8sl8rz49w0nnywyian3xnk2c9qx3-zfs-user-0.8.1-dev/include/libzfs/libzfs.h:592:12: note: declared here
extern int zfs_iter_snapshots(zfs_handle_t *, boolean_t, zfs_iter_f, void *,
^~~~~~~~~~~~~~~~~~
libzfs.c: In function ‘__pyx_pf_6libzfs_10ZFSDataset_2destroy_snapshot’:
libzfs.c:61689:74: warning: passing argument 2 of ‘zfs_destroy_snaps’ discards ‘const’ qualifier from pointer target type [-Wdiscarded-qualifiers]
__pyx_v_ret = zfs_destroy_snaps(__pyx_v_self->__pyx_base.handle, __pyx_v_c_name, 1);
^~~~~~~~~~~~~~
In file included from libzfs.c:599:0:
/nix/store/7h8v8sl8rz49w0nnywyian3xnk2c9qx3-zfs-user-0.8.1-dev/include/libzfs/libzfs.h:617:12: note: expected ‘char *’ but argument is of type ‘const char *’
extern int zfs_destroy_snaps(zfs_handle_t *, char *, boolean_t);
^~~~~~~~~~~~~~~~~
warning: ./pxd/nvpair.pxd:56:3: 'DATA_TYPE_UINT8_ARRAY' redeclared
warning: libzfs.pyx:576:24: Non-trivial type declarators in shared declaration (e.g. mix of pointers and values). Each pointer declaration should be on its own line.
warning: libzfs.pyx:576:31: Non-trivial type declarators in shared declaration (e.g. mix of pointers and values). Each pointer declaration should be on its own line.
warning: libzfs.pyx:577:29: Non-trivial type declarators in shared declaration (e.g. mix of pointers and values). Each pointer declaration should be on its own line.
warning: libzfs.pyx:577:35: Non-trivial type declarators in shared declaration (e.g. mix of pointers and values). Each pointer declaration should be on its own line.
The ZFS_IOC_SEND_PROGRESS ioctl requires the name of the snapshot currently being transferred. py-libzfs actually passes the name of the snapshot that was used to initiate the transfer. That works when sending a single snapshot, but fails when sending multiple snapshots (such as with zfs send's -I, -i, and -R options).
I have a patch for this. However, it still doesn't work to answer the question "how far along is this send that includes multiple snapshots?" because the calculated progress can exceed 100%. I think that's a limitation in libzfs, not py-libzfs. I've since abandoned ZFS_IOC_SEND_PROGRESS entirely so I'm not going to work on that patch anymore. Let me know if you would be interested in the partial solution.
Hi folks,
i've noticed that the tags in this repository aren't matching up with the versions released of the this library.
What makes it especially hard to match up which version of the code is causing which error when used, since the pyx source isn't distributed, only the generated C.
I'm using FreeNAS 11.3's new replication sub-system, I'm interested in using it's NETCAT feature to a remote Ubuntu Server with ZFS (SSH based replication to Ubuntu works fine). To use NETCAT, Freenas has this message about requiring "py-libzfs":
SSH+NETCAT uses SSH to establish a connection to the destination system, then uses py-libzfs to send an unencrypted data stream for higher transfer speeds. This only works when replicating to a FreeNAS, TrueNAS, or other system with py-libzfs installed.
That's not a lot to go on, and I was unable to find any additional documentation about this.
What I tried:
sudo apt install --yes --no-install-recommends gcc gcc-multilib libzfslinux-dev
cd ~
git clone https://github.com/freenas/py-libzfs
cd ~/py-libzfs
./configure && make install
It gets this far:
checking for gcc... gcc
checking whether the C compiler works... yes
checking for C compiler default output file name... a.out
checking for suffix of executables...
checking whether we are cross compiling... no
checking for suffix of object files... o
checking whether we are using the GNU C compiler... yes
checking whether gcc accepts -g... yes
checking for gcc option to accept ISO C89... none needed
-----------------------------------
RUNNING TESTS ON zfs.h HEADER
-----------------------------------
checking how to run the C preprocessor... gcc -E
checking for grep that handles long lines and -e... /bin/grep
checking for egrep... /bin/grep -E
checking for ANSI C header files... yes
checking for sys/types.h... yes
checking for sys/stat.h... yes
checking for stdlib.h... yes
checking for string.h... yes
checking for memory.h... yes
checking for strings.h... yes
checking for inttypes.h... yes
checking for stdint.h... yes
checking for unistd.h... yes
checking for sys/types.h... (cached) yes
checking for sys/fs/zfs.h... no
configure: error: A working zfs header is required
apt-file search sys/fs/zfs.h shows that the zfs.h file is in the libzfslinux-dev and zfs-dkms packages.
I though package "libzfslinux-dev" provided the needed working header? I don't want try / switch to zfs-dkms just to build py-libzfs.
Suggestions?
Hello py-libzfs-team,
I am trying to send snapshots to a backup device using the send - receive mechanism, but I cannot figure out how to use these methods correctly. Is there any documentation (what they do, what are the input parameters)? When I open the help (help(libzfs) ), I can see the different methods available, but I do not know how to find the description of input parameters.
class ZFSSnapshot(ZFSResource)
| Method resolution order:
| ZFSSnapshot
| ZFSResource
| ZFSObject
| builtins.object
|
| Methods defined here:
|
| __getstate__(...)
|
| __reduce_cython__(...)
|
| __setstate_cython__(...)
|
| bookmark(...)
|
| clone(...)
|
| delete(...)
|
| get_send_progress(...)
|
| hold(...)
|
| release(...)
|
| rollback(...)
|
| send(...)
In my implementation of the send-receive mechanism, I always get the Error libzfs.ZFSException: trailing slash in name although there is no trailing slash in the name of my dataset.
File "/usr/local/lib/python3.9/multiprocessing/process.py", line 315, in _bootstrap
self.run()
File "/usr/local/lib/python3.9/multiprocessing/process.py", line 108, in run
self._target(*self._args, **self._kwargs)
File "libzfs.pyx", line 1392, in libzfs.ZFS.receive
libzfs.ZFSException: trailing slash in name
On debian 10 (buster)
CFLAGS="-I/usr/include/libspl -I/usr/include/libzfs/ -I/usr/src/zfs-0.8.2/include/ -D_LARGEFILE64_SOURCE" ./configure
fails with config.log
with plain ./configure config.log
can you create a pip package?
Based on the success of #90 I figured I'd try to run make.
The tools I installed so far for this:
sudo apt install --yes --no-install-recommends gcc gcc-multilib libzfslinux-dev python2.7 python-setuptools cython
I had to over-ride the default phython 2.7 path such as:
sudo make PYTHON=$(which python2.7) install
This yielded the following output:
/usr/bin/python2.7 setup.py install --prefix /usr/local
running install
running bdist_egg
running egg_info
creating libzfs.egg-info
writing libzfs.egg-info/PKG-INFO
writing top-level names to libzfs.egg-info/top_level.txt
writing dependency_links to libzfs.egg-info/dependency_links.txt
writing manifest file 'libzfs.egg-info/SOURCES.txt'
reading manifest file 'libzfs.egg-info/SOURCES.txt'
writing manifest file 'libzfs.egg-info/SOURCES.txt'
installing library code to build/bdist.linux-x86_64/egg
running install_lib
running build_ext
cythoning libzfs.pyx to libzfs.c
warning: libzfs.pyx:637:24: Non-trivial type declarators in shared declaration (e.g. mix of pointers and values). Each pointer declaration should be on its own line.
warning: libzfs.pyx:637:31: Non-trivial type declarators in shared declaration (e.g. mix of pointers and values). Each pointer declaration should be on its own line.
warning: libzfs.pyx:638:29: Non-trivial type declarators in shared declaration (e.g. mix of pointers and values). Each pointer declaration should be on its own line.
warning: libzfs.pyx:638:35: Non-trivial type declarators in shared declaration (e.g. mix of pointers and values). Each pointer declaration should be on its own line.
Error compiling Cython file:
------------------------------------------------------------
...
cdef int block_flag = 0 if blocking else 1
zevent_fd = zfs_dev_fd
with nogil:
ret = libzfs.zpool_events_next(self.handle, &nvl, &dropped, block_flag, zevent_fd)
if ret != 0 or (nvl == NULL and block_flag == 0):
raise self.get_error()
^
------------------------------------------------------------
libzfs.pyx:418:20: Raising exception not allowed without gil
Error compiling Cython file:
------------------------------------------------------------
...
cdef int block_flag = 0 if blocking else 1
zevent_fd = zfs_dev_fd
with nogil:
ret = libzfs.zpool_events_next(self.handle, &nvl, &dropped, block_flag, zevent_fd)
if ret != 0 or (nvl == NULL and block_flag == 0):
raise self.get_error()
^
------------------------------------------------------------
libzfs.pyx:418:40: Calling gil-requiring function not allowed without gil
building 'libzfs' extension
creating build
creating build/temp.linux-x86_64-2.7
x86_64-linux-gnu-gcc -pthread -fno-strict-aliasing -Wdate-time -D_FORTIFY_SOURCE=2 -g -fdebug-prefix-map=/build/python2.7-5Z483E/python2.7-2.7.17=. -fstack-protector-strong -Wformat -Werror=format-security -fPIC -I/usr/include/python2.7 -c libzfs.c -o build/temp.linux-x86_64-2.7/libzfs.o -g -O2 -I/usr/include/libzfs/ -I/usr/include/libspl/ -D_MACHINE_ENDIAN_H_ -DHAVE_ISSETUGID -Werror=implicit-function-declaration
libzfs.c:1:2: error: #error Do not use this file, it is the result of a failed Cython compilation.
#error Do not use this file, it is the result of a failed Cython compilation.
^~~~~
error: command 'x86_64-linux-gnu-gcc' failed with exit status 1
Makefile:11: recipe for target 'install' failed
make: *** [install] Error 1
As to your suggestion to try the DEB package instead. I assume that means I can just grab the latest one, I don't need something specific to the version of ZFS on Linux (0.7.5) I'm running?
If you try to pickle a ZFSException object and then unpickle it, the original message gets unpickled as a tuple. For example
import pickle
from libzfs import ZFSException
e = ZFSException(42, "My error message")
with open("/tmp/zfsexception.pickle", "wb+") as f:
pickle.dump(e, f)
e2 = pickle.load(open("/tmp/zfsexception.pickle", "rb"))
raise e2$ python3.9 pickle_zfsexception.py
Traceback (most recent call last):
File "/usr/home/asomers/pickle_zfsexception.py", line 10, in <module>
raise e2
libzfs.ZFSException: ('My error message',)
Notice the parentheses. The correct output would've been libzfs.ZFSException: 'My error message'
This is a basically a copy of #90. OP closed their report due to finding a workaround, however the problem per se remains.
Run the following commands:
sudo apt install --yes --no-install-recommends gcc gcc-multilib libzfslinux-dev
cd ~
git clone https://github.com/freenas/py-libzfs
cd ~/py-libzfs
./configure
configure passes successfully.
checking for gcc... gcc
checking whether the C compiler works... yes
checking for C compiler default output file name... a.out
checking for suffix of executables...
checking whether we are cross compiling... no
checking for suffix of object files... o
checking whether we are using the GNU C compiler... yes
checking whether gcc accepts -g... yes
checking for gcc option to accept ISO C89... none needed
-----------------------------------
RUNNING TESTS ON zfs.h HEADER
-----------------------------------
checking how to run the C preprocessor... gcc -E
checking for grep that handles long lines and -e... /bin/grep
checking for egrep... /bin/grep -E
checking for ANSI C header files... yes
checking for sys/types.h... yes
checking for sys/stat.h... yes
checking for stdlib.h... yes
checking for string.h... yes
checking for memory.h... yes
checking for strings.h... yes
checking for inttypes.h... yes
checking for stdint.h... yes
checking for unistd.h... yes
checking for sys/types.h... (cached) yes
checking for sys/fs/zfs.h... no
configure: error: A working zfs header is required
As @skarekrow reported in iocage/iocage#797, the ZFS dataset deletion using py-libzfs is flaky.
Also switch over to subprocess zfs as occasionally the base dataset would not be destroyed properly by py-libzfs.
I wasn't able to confirm this obversation yet, but just in case, it is a serious issue that should be addressed in py-libzfs.
If you use snapshots_serialized and the only property in the list is createtxg, py-libzfs will not fetch it correctly. Instead of the usual parsed, source, rawvalue, value dict, it will simply return 0. However, if there are any other properties in the list, then all will be fetched correctly. From my spot-checking, only createtxg exhibits this behavior.
FreeBSD 13.1-RELEASE amd64, py-libzfs-1.1.2023020200. The bug must've been introduced sometime in the revision range 077cf6d..1a99a45.
Ensure that there is a dataset named "tank" with at least one snapshot. Then run this script:
from pprint import pprint
import libzfs
zfs = libzfs.ZFS()
badsnaps = zfs.snapshots_serialized(
props=["createtxg"],
datasets=["tank"],
recursive=False
)
goodsnaps = zfs.snapshots_serialized(
props=["used", "createtxg"],
datasets=["tank"],
recursive=False
)
print("Bad version of the snapshot list:")
pprint(badsnaps[0])
print("\nGood version of the snapshot list:")
pprint(goodsnaps[0])The output will look like this:
Bad version of the snapshot list:
{'createtxg': '0', <--- Incorrect value
'dataset': 'tank',
'id': 'tank@zrepl_20230119_070154_000',
'name': 'tank@zrepl_20230119_070154_000',
'pool': 'tank',
'snapshot_name': 'zrepl_20230119_070154_000',
'type': 'SNAPSHOT'} <--- Dict ends with no 'properties' section
Good version of the snapshot list:
{'createtxg': '53209141', <--- Correct value
'dataset': 'tank',
'id': 'tank@zrepl_20230119_070154_000',
'name': 'tank@zrepl_20230119_070154_000',
'pool': 'tank',
'properties': {'createtxg': {'parsed': '53209141', <--- Expected 'properties' section
'rawvalue': '53209141',
'source': 'NONE',
'value': '53209141'},
'used': {'parsed': 0,
'rawvalue': '0',
'source': 'NONE',
'value': '0B'}},
'snapshot_name': 'zrepl_20230119_070154_000',
'type': 'SNAPSHOT'}
we're currently debugging an issue in libioc where py-libzfs throws an exception when trying to check if a dataset is mounted bsdci/libioc#753 (comment)
apparently, the port has been updated, https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=242453 without a Tag, this makes it quite hard to see and debug.
I believe the current py-libzfs project is not compatible with cython 3.x. The following warnings and errors will appear:
warning: libzfs.pxd:7:0: The 'IF' statement is deprecated and will be removed in a future Cython version. Consider using runtime conditions or C macros instead. See https://github.com/cython/cython/issues/4310
warning: libzfs.pxd:16:4: The 'IF' statement is deprecated and will be removed in a future Cython version. Consider using runtime conditions or C macros instead. See https://github.com/cython/cython/issues/4310
warning: libzfs.pxd:23:4: The 'IF' statement is deprecated and will be removed in a future Cython version. Consider using runtime conditions or C macros instead. See https://github.com/cython/cython/issues/4310
warning: libzfs.pxd:25:4: The 'IF' statement is deprecated and will be removed in a future Cython version. Consider using runtime conditions or C macros instead. See https://github.com/cython/cython/issues/4310
warning: libzfs.pxd:27:4: The 'IF' statement is deprecated and will be removed in a future Cython version. Consider using runtime conditions or C macros instead. See https://github.com/cython/cython/issues/4310
warning: libzfs.pxd:31:0: The 'IF' statement is deprecated and will be removed in a future Cython version. Consider using runtime conditions or C macros instead. See https://github.com/cython/cython/issues/4310
warning: libzfs.pxd:40:8: The 'IF' statement is deprecated and will be removed in a future Cython version. Consider using runtime conditions or C macros instead. See https://github.com/cython/cython/issues/4310
warning: libzfs.pxd:43:8: The 'IF' statement is deprecated and will be removed in a future Cython version. Consider using runtime conditions or C macros instead. See https://github.com/cython/cython/issues/4310
warning: libzfs.pxd:69:4: The 'IF' statement is deprecated and will be removed in a future Cython version. Consider using runtime conditions or C macros instead. See https://github.com/cython/cython/issues/4310
warning: libzfs.pxd:74:4: The 'IF' statement is deprecated and will be removed in a future Cython version. Consider using runtime conditions or C macros instead. See https://github.com/cython/cython/issues/4310
warning: libzfs.pxd:156:4: The 'IF' statement is deprecated and will be removed in a future Cython version. Consider using runtime conditions or C macros instead. See https://github.com/cython/cython/issues/4310
warning: libzfs.pxd:206:4: The 'IF' statement is deprecated and will be removed in a future Cython version. Consider using runtime conditions or C macros instead. See https://github.com/cython/cython/issues/4310
warning: libzfs.pxd:218:4: The 'IF' statement is deprecated and will be removed in a future Cython version. Consider using runtime conditions or C macros instead. See https://github.com/cython/cython/issues/4310
warning: libzfs.pxd:250:4: The 'IF' statement is deprecated and will be removed in a future Cython version. Consider using runtime conditions or C macros instead. See https://github.com/cython/cython/issues/4310
warning: libzfs.pxd:253:4: The 'IF' statement is deprecated and will be removed in a future Cython version. Consider using runtime conditions or C macros instead. See https://github.com/cython/cython/issues/4310
warning: libzfs.pxd:282:4: The 'IF' statement is deprecated and will be removed in a future Cython version. Consider using runtime conditions or C macros instead. See https://github.com/cython/cython/issues/4310
warning: libzfs.pxd:396:4: The 'IF' statement is deprecated and will be removed in a future Cython version. Consider using runtime conditions or C macros instead. See https://github.com/cython/cython/issues/4310
warning: libzfs.pxd:400:4: The 'IF' statement is deprecated and will be removed in a future Cython version. Consider using runtime conditions or C macros instead. See https://github.com/cython/cython/issues/4310
warning: libzfs.pxd:404:4: The 'IF' statement is deprecated and will be removed in a future Cython version. Consider using runtime conditions or C macros instead. See https://github.com/cython/cython/issues/4310
warning: libzfs.pxd:408:4: The 'IF' statement is deprecated and will be removed in a future Cython version. Consider using runtime conditions or C macros instead. See https://github.com/cython/cython/issues/4310
warning: libzfs.pxd:412:4: The 'IF' statement is deprecated and will be removed in a future Cython version. Consider using runtime conditions or C macros instead. See https://github.com/cython/cython/issues/4310
warning: libzfs.pxd:416:4: The 'IF' statement is deprecated and will be removed in a future Cython version. Consider using runtime conditions or C macros instead. See https://github.com/cython/cython/issues/4310
warning: libzfs.pxd:442:4: The 'IF' statement is deprecated and will be removed in a future Cython version. Consider using runtime conditions or C macros instead. See https://github.com/cython/cython/issues/4310
warning: libzfs.pxd:443:8: The 'IF' statement is deprecated and will be removed in a future Cython version. Consider using runtime conditions or C macros instead. See https://github.com/cython/cython/issues/4310
warning: libzfs.pxd:454:8: The 'IF' statement is deprecated and will be removed in a future Cython version. Consider using runtime conditions or C macros instead. See https://github.com/cython/cython/issues/4310
warning: libzfs.pxd:468:4: The 'IF' statement is deprecated and will be removed in a future Cython version. Consider using runtime conditions or C macros instead. See https://github.com/cython/cython/issues/4310
warning: libzfs.pxd:473:4: The 'IF' statement is deprecated and will be removed in a future Cython version. Consider using runtime conditions or C macros instead. See https://github.com/cython/cython/issues/4310
warning: libzfs.pxd:483:4: The 'IF' statement is deprecated and will be removed in a future Cython version. Consider using runtime conditions or C macros instead. See https://github.com/cython/cython/issues/4310
warning: libzfs.pxd:511:4: The 'IF' statement is deprecated and will be removed in a future Cython version. Consider using runtime conditions or C macros instead. See https://github.com/cython/cython/issues/4310
warning: libzfs.pxd:539:4: The 'IF' statement is deprecated and will be removed in a future Cython version. Consider using runtime conditions or C macros instead. See https://github.com/cython/cython/issues/4310
warning: libzfs.pxd:578:4: The 'IF' statement is deprecated and will be removed in a future Cython version. Consider using runtime conditions or C macros instead. See https://github.com/cython/cython/issues/4310
warning: libzfs.pxd:579:8: The 'IF' statement is deprecated and will be removed in a future Cython version. Consider using runtime conditions or C macros instead. See https://github.com/cython/cython/issues/4310
warning: libzfs.pxd:605:4: The 'IF' statement is deprecated and will be removed in a future Cython version. Consider using runtime conditions or C macros instead. See https://github.com/cython/cython/issues/4310
warning: libzfs.pxd:610:4: The 'IF' statement is deprecated and will be removed in a future Cython version. Consider using runtime conditions or C macros instead. See https://github.com/cython/cython/issues/4310
warning: libzfs.pxd:618:4: The 'IF' statement is deprecated and will be removed in a future Cython version. Consider using runtime conditions or C macros instead. See https://github.com/cython/cython/issues/4310
warning: libzfs.pxd:630:4: The 'IF' statement is deprecated and will be removed in a future Cython version. Consider using runtime conditions or C macros instead. See https://github.com/cython/cython/issues/4310
warning: nvpair.pxd:7:0: The 'IF' statement is deprecated and will be removed in a future Cython version. Consider using runtime conditions or C macros instead. See https://github.com/cython/cython/issues/4310
warning: zfs.pxd:69:4: The 'IF' statement is deprecated and will be removed in a future Cython version. Consider using runtime conditions or C macros instead. See https://github.com/cython/cython/issues/4310
warning: zfs.pxd:90:4: The 'IF' statement is deprecated and will be removed in a future Cython version. Consider using runtime conditions or C macros instead. See https://github.com/cython/cython/issues/4310
warning: zfs.pxd:98:4: The 'IF' statement is deprecated and will be removed in a future Cython version. Consider using runtime conditions or C macros instead. See https://github.com/cython/cython/issues/4310
warning: zfs.pxd:101:4: The 'IF' statement is deprecated and will be removed in a future Cython version. Consider using runtime conditions or C macros instead. See https://github.com/cython/cython/issues/4310
warning: zfs.pxd:105:4: The 'IF' statement is deprecated and will be removed in a future Cython version. Consider using runtime conditions or C macros instead. See https://github.com/cython/cython/issues/4310
warning: zfs.pxd:135:4: The 'IF' statement is deprecated and will be removed in a future Cython version. Consider using runtime conditions or C macros instead. See https://github.com/cython/cython/issues/4310
warning: zfs.pxd:242:4: The 'IF' statement is deprecated and will be removed in a future Cython version. Consider using runtime conditions or C macros instead. See https://github.com/cython/cython/issues/4310
warning: zfs.pxd:248:4: The 'IF' statement is deprecated and will be removed in a future Cython version. Consider using runtime conditions or C macros instead. See https://github.com/cython/cython/issues/4310
warning: zfs.pxd:249:8: The 'IF' statement is deprecated and will be removed in a future Cython version. Consider using runtime conditions or C macros instead. See https://github.com/cython/cython/issues/4310
warning: zfs.pxd:267:8: The 'IF' statement is deprecated and will be removed in a future Cython version. Consider using runtime conditions or C macros instead. See https://github.com/cython/cython/issues/4310
warning: zfs.pxd:302:4: The 'IF' statement is deprecated and will be removed in a future Cython version. Consider using runtime conditions or C macros instead. See https://github.com/cython/cython/issues/4310
warning: zfs.pxd:430:4: The 'IF' statement is deprecated and will be removed in a future Cython version. Consider using runtime conditions or C macros instead. See https://github.com/cython/cython/issues/4310
warning: zfs.pxd:484:4: The 'IF' statement is deprecated and will be removed in a future Cython version. Consider using runtime conditions or C macros instead. See https://github.com/cython/cython/issues/4310
warning: zfs.pxd:563:0: The 'IF' statement is deprecated and will be removed in a future Cython version. Consider using runtime conditions or C macros instead. See https://github.com/cython/cython/issues/4310
warning: libzfs.pyx:45:4: The 'IF' statement is deprecated and will be removed in a future Cython version. Consider using runtime conditions or C macros instead. See https://github.com/cython/cython/issues/4310
warning: libzfs.pyx:50:4: The 'IF' statement is deprecated and will be removed in a future Cython version. Consider using runtime conditions or C macros instead. See https://github.com/cython/cython/issues/4310
warning: libzfs.pyx:130:4: The 'IF' statement is deprecated and will be removed in a future Cython version. Consider using runtime conditions or C macros instead. See https://github.com/cython/cython/issues/4310
warning: libzfs.pyx:135:4: The 'IF' statement is deprecated and will be removed in a future Cython version. Consider using runtime conditions or C macros instead. See https://github.com/cython/cython/issues/4310
warning: libzfs.pyx:181:4: The 'IF' statement is deprecated and will be removed in a future Cython version. Consider using runtime conditions or C macros instead. See https://github.com/cython/cython/issues/4310
warning: libzfs.pyx:220:4: The 'IF' statement is deprecated and will be removed in a future Cython version. Consider using runtime conditions or C macros instead. See https://github.com/cython/cython/issues/4310
warning: libzfs.pyx:231:4: The 'IF' statement is deprecated and will be removed in a future Cython version. Consider using runtime conditions or C macros instead. See https://github.com/cython/cython/issues/4310
warning: libzfs.pyx:233:4: The 'IF' statement is deprecated and will be removed in a future Cython version. Consider using runtime conditions or C macros instead. See https://github.com/cython/cython/issues/4310
warning: libzfs.pyx:236:4: The 'IF' statement is deprecated and will be removed in a future Cython version. Consider using runtime conditions or C macros instead. See https://github.com/cython/cython/issues/4310
warning: libzfs.pyx:238:4: The 'IF' statement is deprecated and will be removed in a future Cython version. Consider using runtime conditions or C macros instead. See https://github.com/cython/cython/issues/4310
warning: libzfs.pyx:259:0: The 'IF' statement is deprecated and will be removed in a future Cython version. Consider using runtime conditions or C macros instead. See https://github.com/cython/cython/issues/4310
warning: libzfs.pyx:279:4: The 'IF' statement is deprecated and will be removed in a future Cython version. Consider using runtime conditions or C macros instead. See https://github.com/cython/cython/issues/4310
warning: libzfs.pyx:292:4: The 'IF' statement is deprecated and will be removed in a future Cython version. Consider using runtime conditions or C macros instead. See https://github.com/cython/cython/issues/4310
warning: libzfs.pyx:294:4: The 'IF' statement is deprecated and will be removed in a future Cython version. Consider using runtime conditions or C macros instead. See https://github.com/cython/cython/issues/4310
warning: libzfs.pyx:296:4: The 'IF' statement is deprecated and will be removed in a future Cython version. Consider using runtime conditions or C macros instead. See https://github.com/cython/cython/issues/4310
warning: libzfs.pyx:298:4: The 'IF' statement is deprecated and will be removed in a future Cython version. Consider using runtime conditions or C macros instead. See https://github.com/cython/cython/issues/4310
warning: libzfs.pyx:300:4: The 'IF' statement is deprecated and will be removed in a future Cython version. Consider using runtime conditions or C macros instead. See https://github.com/cython/cython/issues/4310
warning: libzfs.pyx:302:4: The 'IF' statement is deprecated and will be removed in a future Cython version. Consider using runtime conditions or C macros instead. See https://github.com/cython/cython/issues/4310
warning: libzfs.pyx:304:4: The 'IF' statement is deprecated and will be removed in a future Cython version. Consider using runtime conditions or C macros instead. See https://github.com/cython/cython/issues/4310
warning: libzfs.pyx:324:0: The 'IF' statement is deprecated and will be removed in a future Cython version. Consider using runtime conditions or C macros instead. See https://github.com/cython/cython/issues/4310
warning: libzfs.pyx:455:0: The 'IF' statement is deprecated and will be removed in a future Cython version. Consider using runtime conditions or C macros instead. See https://github.com/cython/cython/issues/4310
warning: libzfs.pyx:544:4: The 'IF' statement is deprecated and will be removed in a future Cython version. Consider using runtime conditions or C macros instead. See https://github.com/cython/cython/issues/4310
warning: libzfs.pyx:589:8: The 'IF' statement is deprecated and will be removed in a future Cython version. Consider using runtime conditions or C macros instead. See https://github.com/cython/cython/issues/4310
warning: libzfs.pyx:623:8: The 'IF' statement is deprecated and will be removed in a future Cython version. Consider using runtime conditions or C macros instead. See https://github.com/cython/cython/issues/4310
warning: libzfs.pyx:631:8: The 'IF' statement is deprecated and will be removed in a future Cython version. Consider using runtime conditions or C macros instead. See https://github.com/cython/cython/issues/4310
warning: libzfs.pyx:639:8: The 'IF' statement is deprecated and will be removed in a future Cython version. Consider using runtime conditions or C macros instead. See https://github.com/cython/cython/issues/4310
warning: libzfs.pyx:647:8: The 'IF' statement is deprecated and will be removed in a future Cython version. Consider using runtime conditions or C macros instead. See https://github.com/cython/cython/issues/4310
warning: libzfs.pyx:671:12: The 'IF' statement is deprecated and will be removed in a future Cython version. Consider using runtime conditions or C macros instead. See https://github.com/cython/cython/issues/4310
warning: libzfs.pyx:702:16: The 'IF' statement is deprecated and will be removed in a future Cython version. Consider using runtime conditions or C macros instead. See https://github.com/cython/cython/issues/4310
warning: libzfs.pyx:714:8: The 'IF' statement is deprecated and will be removed in a future Cython version. Consider using runtime conditions or C macros instead. See https://github.com/cython/cython/issues/4310
warning: libzfs.pyx:809:12: The 'IF' statement is deprecated and will be removed in a future Cython version. Consider using runtime conditions or C macros instead. See https://github.com/cython/cython/issues/4310
warning: libzfs.pyx:905:8: The 'IF' statement is deprecated and will be removed in a future Cython version. Consider using runtime conditions or C macros instead. See https://github.com/cython/cython/issues/4310
warning: libzfs.pyx:910:8: The 'IF' statement is deprecated and will be removed in a future Cython version. Consider using runtime conditions or C macros instead. See https://github.com/cython/cython/issues/4310
warning: libzfs.pyx:926:8: The 'IF' statement is deprecated and will be removed in a future Cython version. Consider using runtime conditions or C macros instead. See https://github.com/cython/cython/issues/4310
warning: libzfs.pyx:938:8: The 'IF' statement is deprecated and will be removed in a future Cython version. Consider using runtime conditions or C macros instead. See https://github.com/cython/cython/issues/4310
warning: libzfs.pyx:952:4: The 'IF' statement is deprecated and will be removed in a future Cython version. Consider using runtime conditions or C macros instead. See https://github.com/cython/cython/issues/4310
warning: libzfs.pyx:982:16: The 'IF' statement is deprecated and will be removed in a future Cython version. Consider using runtime conditions or C macros instead. See https://github.com/cython/cython/issues/4310
warning: libzfs.pyx:1322:8: The 'IF' statement is deprecated and will be removed in a future Cython version. Consider using runtime conditions or C macros instead. See https://github.com/cython/cython/issues/4310
warning: libzfs.pyx:1329:12: The 'IF' statement is deprecated and will be removed in a future Cython version. Consider using runtime conditions or C macros instead. See https://github.com/cython/cython/issues/4310
warning: libzfs.pyx:1331:12: The 'IF' statement is deprecated and will be removed in a future Cython version. Consider using runtime conditions or C macros instead. See https://github.com/cython/cython/issues/4310
warning: libzfs.pyx:1337:12: The 'IF' statement is deprecated and will be removed in a future Cython version. Consider using runtime conditions or C macros instead. See https://github.com/cython/cython/issues/4310
warning: libzfs.pyx:1344:12: The 'IF' statement is deprecated and will be removed in a future Cython version. Consider using runtime conditions or C macros instead. See https://github.com/cython/cython/issues/4310
warning: libzfs.pyx:1365:4: The 'IF' statement is deprecated and will be removed in a future Cython version. Consider using runtime conditions or C macros instead. See https://github.com/cython/cython/issues/4310
warning: libzfs.pyx:1401:8: The 'IF' statement is deprecated and will be removed in a future Cython version. Consider using runtime conditions or C macros instead. See https://github.com/cython/cython/issues/4310
warning: libzfs.pyx:1412:8: The 'IF' statement is deprecated and will be removed in a future Cython version. Consider using runtime conditions or C macros instead. See https://github.com/cython/cython/issues/4310
warning: libzfs.pyx:1425:8: The 'IF' statement is deprecated and will be removed in a future Cython version. Consider using runtime conditions or C macros instead. See https://github.com/cython/cython/issues/4310
warning: libzfs.pyx:1543:8: The 'IF' statement is deprecated and will be removed in a future Cython version. Consider using runtime conditions or C macros instead. See https://github.com/cython/cython/issues/4310
warning: libzfs.pyx:1564:8: The 'IF' statement is deprecated and will be removed in a future Cython version. Consider using runtime conditions or C macros instead. See https://github.com/cython/cython/issues/4310
warning: libzfs.pyx:1625:8: The 'IF' statement is deprecated and will be removed in a future Cython version. Consider using runtime conditions or C macros instead. See https://github.com/cython/cython/issues/4310
warning: libzfs.pyx:1629:8: The 'IF' statement is deprecated and will be removed in a future Cython version. Consider using runtime conditions or C macros instead. See https://github.com/cython/cython/issues/4310
warning: libzfs.pyx:1654:16: The 'IF' statement is deprecated and will be removed in a future Cython version. Consider using runtime conditions or C macros instead. See https://github.com/cython/cython/issues/4310
warning: libzfs.pyx:1760:4: The 'IF' statement is deprecated and will be removed in a future Cython version. Consider using runtime conditions or C macros instead. See https://github.com/cython/cython/issues/4310
warning: libzfs.pyx:1778:4: The 'IF' statement is deprecated and will be removed in a future Cython version. Consider using runtime conditions or C macros instead. See https://github.com/cython/cython/issues/4310
warning: libzfs.pyx:2045:20: The 'IF' statement is deprecated and will be removed in a future Cython version. Consider using runtime conditions or C macros instead. See https://github.com/cython/cython/issues/4310
warning: libzfs.pyx:2153:8: The 'IF' statement is deprecated and will be removed in a future Cython version. Consider using runtime conditions or C macros instead. See https://github.com/cython/cython/issues/4310
warning: libzfs.pyx:2205:4: The 'IF' statement is deprecated and will be removed in a future Cython version. Consider using runtime conditions or C macros instead. See https://github.com/cython/cython/issues/4310
warning: libzfs.pyx:2302:12: The 'IF' statement is deprecated and will be removed in a future Cython version. Consider using runtime conditions or C macros instead. See https://github.com/cython/cython/issues/4310
warning: libzfs.pyx:2336:12: The 'IF' statement is deprecated and will be removed in a future Cython version. Consider using runtime conditions or C macros instead. See https://github.com/cython/cython/issues/4310
warning: libzfs.pyx:2718:8: The 'IF' statement is deprecated and will be removed in a future Cython version. Consider using runtime conditions or C macros instead. See https://github.com/cython/cython/issues/4310
warning: libzfs.pyx:2772:8: The 'IF' statement is deprecated and will be removed in a future Cython version. Consider using runtime conditions or C macros instead. See https://github.com/cython/cython/issues/4310
warning: libzfs.pyx:2782:8: The 'IF' statement is deprecated and will be removed in a future Cython version. Consider using runtime conditions or C macros instead. See https://github.com/cython/cython/issues/4310
warning: libzfs.pyx:2792:12: The 'IF' statement is deprecated and will be removed in a future Cython version. Consider using runtime conditions or C macros instead. See https://github.com/cython/cython/issues/4310
warning: libzfs.pyx:2798:12: The 'IF' statement is deprecated and will be removed in a future Cython version. Consider using runtime conditions or C macros instead. See https://github.com/cython/cython/issues/4310
warning: libzfs.pyx:2839:4: The 'IF' statement is deprecated and will be removed in a future Cython version. Consider using runtime conditions or C macros instead. See https://github.com/cython/cython/issues/4310
warning: libzfs.pyx:2856:12: The 'IF' statement is deprecated and will be removed in a future Cython version. Consider using runtime conditions or C macros instead. See https://github.com/cython/cython/issues/4310
warning: libzfs.pyx:2888:16: The 'IF' statement is deprecated and will be removed in a future Cython version. Consider using runtime conditions or C macros instead. See https://github.com/cython/cython/issues/4310
warning: libzfs.pyx:2973:12: The 'IF' statement is deprecated and will be removed in a future Cython version. Consider using runtime conditions or C macros instead. See https://github.com/cython/cython/issues/4310
warning: libzfs.pyx:2975:12: The 'IF' statement is deprecated and will be removed in a future Cython version. Consider using runtime conditions or C macros instead. See https://github.com/cython/cython/issues/4310
warning: libzfs.pyx:2978:12: The 'IF' statement is deprecated and will be removed in a future Cython version. Consider using runtime conditions or C macros instead. See https://github.com/cython/cython/issues/4310
warning: libzfs.pyx:2981:12: The 'IF' statement is deprecated and will be removed in a future Cython version. Consider using runtime conditions or C macros instead. See https://github.com/cython/cython/issues/4310
warning: libzfs.pyx:2985:12: The 'IF' statement is deprecated and will be removed in a future Cython version. Consider using runtime conditions or C macros instead. See https://github.com/cython/cython/issues/4310
warning: libzfs.pyx:3051:4: The 'IF' statement is deprecated and will be removed in a future Cython version. Consider using runtime conditions or C macros instead. See https://github.com/cython/cython/issues/4310
warning: libzfs.pyx:3067:4: The 'IF' statement is deprecated and will be removed in a future Cython version. Consider using runtime conditions or C macros instead. See https://github.com/cython/cython/issues/4310
warning: libzfs.pyx:3083:4: The 'IF' statement is deprecated and will be removed in a future Cython version. Consider using runtime conditions or C macros instead. See https://github.com/cython/cython/issues/4310
warning: libzfs.pyx:3121:8: The 'IF' statement is deprecated and will be removed in a future Cython version. Consider using runtime conditions or C macros instead. See https://github.com/cython/cython/issues/4310
warning: libzfs.pyx:3130:20: The 'IF' statement is deprecated and will be removed in a future Cython version. Consider using runtime conditions or C macros instead. See https://github.com/cython/cython/issues/4310
warning: libzfs.pyx:3160:8: The 'IF' statement is deprecated and will be removed in a future Cython version. Consider using runtime conditions or C macros instead. See https://github.com/cython/cython/issues/4310
warning: libzfs.pyx:3216:12: The 'IF' statement is deprecated and will be removed in a future Cython version. Consider using runtime conditions or C macros instead. See https://github.com/cython/cython/issues/4310
warning: libzfs.pyx:3230:12: The 'IF' statement is deprecated and will be removed in a future Cython version. Consider using runtime conditions or C macros instead. See https://github.com/cython/cython/issues/4310
warning: libzfs.pyx:3243:8: The 'IF' statement is deprecated and will be removed in a future Cython version. Consider using runtime conditions or C macros instead. See https://github.com/cython/cython/issues/4310
warning: libzfs.pyx:3471:8: The 'IF' statement is deprecated and will be removed in a future Cython version. Consider using runtime conditions or C macros instead. See https://github.com/cython/cython/issues/4310
warning: libzfs.pyx:3473:12: The 'IF' statement is deprecated and will be removed in a future Cython version. Consider using runtime conditions or C macros instead. See https://github.com/cython/cython/issues/4310
warning: libzfs.pyx:3481:16: The 'IF' statement is deprecated and will be removed in a future Cython version. Consider using runtime conditions or C macros instead. See https://github.com/cython/cython/issues/4310
warning: libzfs.pyx:3523:12: The 'IF' statement is deprecated and will be removed in a future Cython version. Consider using runtime conditions or C macros instead. See https://github.com/cython/cython/issues/4310
warning: libzfs.pyx:3661:8: The 'IF' statement is deprecated and will be removed in a future Cython version. Consider using runtime conditions or C macros instead. See https://github.com/cython/cython/issues/4310
warning: libzfs.pyx:3895:4: The 'IF' statement is deprecated and will be removed in a future Cython version. Consider using runtime conditions or C macros instead. See https://github.com/cython/cython/issues/4310
warning: libzfs.pyx:4093:4: The 'IF' statement is deprecated and will be removed in a future Cython version. Consider using runtime conditions or C macros instead. See https://github.com/cython/cython/issues/4310
warning: libzfs.pyx:4104:8: The 'IF' statement is deprecated and will be removed in a future Cython version. Consider using runtime conditions or C macros instead. See https://github.com/cython/cython/issues/4310
warning: libzfs.pyx:4274:4: The 'IF' statement is deprecated and will be removed in a future Cython version. Consider using runtime conditions or C macros instead. See https://github.com/cython/cython/issues/4310
warning: libzfs.pyx:4421:8: The 'IF' statement is deprecated and will be removed in a future Cython version. Consider using runtime conditions or C macros instead. See https://github.com/cython/cython/issues/4310
warning: libzfs.pyx:4462:4: The 'IF' statement is deprecated and will be removed in a future Cython version. Consider using runtime conditions or C macros instead. See https://github.com/cython/cython/issues/4310
warning: libzfs.pyx:4478:4: The 'IF' statement is deprecated and will be removed in a future Cython version. Consider using runtime conditions or C macros instead. See https://github.com/cython/cython/issues/4310
warning: libzfs.pyx:4500:4: The 'IF' statement is deprecated and will be removed in a future Cython version. Consider using runtime conditions or C macros instead. See https://github.com/cython/cython/issues/4310
warning: libzfs.pyx:4504:4: The 'IF' statement is deprecated and will be removed in a future Cython version. Consider using runtime conditions or C macros instead. See https://github.com/cython/cython/issues/4310
warning: libzfs.pyx:4508:4: The 'IF' statement is deprecated and will be removed in a future Cython version. Consider using runtime conditions or C macros instead. See https://github.com/cython/cython/issues/4310
warning: libzfs.pyx:4512:4: The 'IF' statement is deprecated and will be removed in a future Cython version. Consider using runtime conditions or C macros instead. See https://github.com/cython/cython/issues/4310
warning: libzfs.pyx:4516:4: The 'IF' statement is deprecated and will be removed in a future Cython version. Consider using runtime conditions or C macros instead. See https://github.com/cython/cython/issues/4310
warning: libzfs.pyx:4520:4: The 'IF' statement is deprecated and will be removed in a future Cython version. Consider using runtime conditions or C macros instead. See https://github.com/cython/cython/issues/4310
warning: libzfs.pyx:4550:4: The 'IF' statement is deprecated and will be removed in a future Cython version. Consider using runtime conditions or C macros instead. See https://github.com/cython/cython/issues/4310
Error compiling Cython file:
------------------------------------------------------------
...
proptypes = []
c_type = <zfs.zfs_type_t>t
iter.type = c_type
iter.props = <void *>proptypes
with nogil:
libzfs.zprop_iter(self.__iterate_props, <void*>&iter, True, True, c_type)
^
------------------------------------------------------------
libzfs.pyx:515:38: Cannot assign type 'int (int, void *) except * nogil' to 'zprop_func'. Exception values are incompatible. Suggest adding 'noexcept' to type 'int (int, void *) except * nogil'.
Error compiling Cython file:
------------------------------------------------------------
...
'source': PropertySource(<int>csource).name,
'source_info': str(csrcstr) if csource == zfs.ZPROP_SRC_INHERITED else None
}
if retrieve_children:
ZFS.__iterate_filesystems(handle, 0, ZFS.__dataset_handles, <void*>child_data)
^
------------------------------------------------------------
libzfs.pyx:802:52: Cannot assign type 'int (zfs_handle_t *, void *) except * nogil' to 'zfs_iter_f'. Exception values are incompatible. Suggest adding 'noexcept' to type 'int (zfs_handle_t *, void *) except * nogil'.
Error compiling Cython file:
------------------------------------------------------------
...
):
libzfs.zfs_close(handle)
return 0
libzfs.libzfs_add_handle(cb, handle)
ZFS.__iterate_filesystems(handle, 0, ZFS.__retrieve_mountable_datasets_handles, cb)
^
------------------------------------------------------------
libzfs.pyx:920:48: Cannot assign type 'int (zfs_handle_t *, void *) except * nogil' to 'zfs_iter_f'. Exception values are incompatible. Suggest adding 'noexcept' to type 'int (zfs_handle_t *, void *) except * nogil'.
Error compiling Cython file:
------------------------------------------------------------
...
# Gathering all handles first
ZFS.__retrieve_mountable_datasets_handles(handle, &cb)
# Mount all datasets
libzfs.zfs_foreach_mountpoint(
self.handle, cb.cb_handles, cb.cb_used, ZFS.mount_dataset, <void*>mount_data.handle, True
^
------------------------------------------------------------
libzfs.pyx:974:59: Cannot assign type 'int (zfs_handle_t *, void *) except * nogil' to 'zfs_iter_f'. Exception values are incompatible. Suggest adding 'noexcept' to type 'int (zfs_handle_t *, void *) except * nogil'.
Error compiling Cython file:
------------------------------------------------------------
...
)
# Share all datasets
if enable_shares:
libzfs.zfs_foreach_mountpoint(
self.handle, cb.cb_handles, cb.cb_used, ZFS.share_one_dataset, <void*>mount_results, False
^
------------------------------------------------------------
libzfs.pyx:980:63: Cannot assign type 'int (zfs_handle_t *, void *) except * nogil' to 'zfs_iter_f'. Exception values are incompatible. Suggest adding 'noexcept' to type 'int (zfs_handle_t *, void *) except * nogil'.
Error compiling Cython file:
------------------------------------------------------------
...
max_txg = configuration_data['max_txg']
properties = {}
simple_handle = set(props).issubset({'name', 'createtxg'})
snap_data = {}
libzfs.zfs_iter_snapshots(handle, simple_handle, ZFS.__snapshot_details, <void*>snap_list, min_txg, max_txg)
^
------------------------------------------------------------
libzfs.pyx:1033:60: Cannot assign type 'int (zfs_handle_t *, void *) except * nogil' to 'zfs_iter_f'. Exception values are incompatible. Suggest adding 'noexcept' to type 'int (zfs_handle_t *, void *) except * nogil'.
Error compiling Cython file:
------------------------------------------------------------
...
close_handle = snap_list[0]['close_handle']
recursive = snap_list[0]['recursive']
if is_dataset:
if recursive:
ZFS.__iterate_filesystems(handle, 0, ZFS.__datasets_snapshots, arg)
^
------------------------------------------------------------
libzfs.pyx:1145:56: Cannot assign type 'int (zfs_handle_t *, void *) except * nogil' to 'zfs_iter_f'. Exception values are incompatible. Suggest adding 'noexcept' to type 'int (zfs_handle_t *, void *) except * nogil'.
Error compiling Cython file:
------------------------------------------------------------
...
if not iter.array:
raise MemoryError()
iter.alloc = 32
libzfs.zpool_iter(self.handle, self.__iterate_pools, <void*>&iter)
^
------------------------------------------------------------
libzfs.pyx:1232:55: Cannot assign type 'int (zpool_handle_t *, void *) except * nogil' to 'zpool_iter_f'. Exception values are incompatible. Suggest adding 'noexcept' to type 'int (zpool_handle_t *, void *) except * nogil'.
Error compiling Cython file:
------------------------------------------------------------
...
cdef ZPoolProperty prop
proptypes = []
result = {}
with nogil:
libzfs.zprop_iter(self.__iterate_props, <void*>proptypes, True, True, zfs.ZFS_TYPE_POOL)
^
------------------------------------------------------------
libzfs.pyx:3005:38: Cannot assign type 'int (int, void *) except * nogil' to 'zprop_func'. Exception values are incompatible. Suggest adding 'noexcept' to type 'int (int, void *) except * nogil'.
Error compiling Cython file:
------------------------------------------------------------
...
iter.array = <uintptr_t *>malloc(128 * sizeof(uintptr_t))
if not iter.array:
raise MemoryError()
iter.alloc = 128
ZFS.__iterate_dependents(self.handle, 0, recursion, self.__iterate, <void*>&iter)
^
------------------------------------------------------------
libzfs.pyx:3569:68: Cannot assign type 'int (zfs_handle_t *, void *) except * nogil' to 'zfs_iter_f'. Exception values are incompatible. Suggest adding 'noexcept' to type 'int (zfs_handle_t *, void *) except * nogil'.
Error compiling Cython file:
------------------------------------------------------------
...
results = {}
for quota_prop in quota_props:
prop: zfs.zfs_userquota_prop_t = quota_prop.value
result = []
with nogil:
ret = libzfs.zfs_userspace(self.handle, prop, ZFSResource._userspace_cb, <void*>result)
^
------------------------------------------------------------
libzfs.pyx:3640:73: Cannot assign type 'int (void *, const char *, uint32_t, uint64_t) except * nogil' to 'zfs_userspace_cb_t'. Exception values are incompatible. Suggest adding 'noexcept' to type 'int (void *, const char *, uint32_t, uint64_t) except * nogil'.
Error compiling Cython file:
------------------------------------------------------------
...
with gil:
snap_config = <object> arg
spec_orig = snap_config['snapshot_specification']
err = ZFS.__iterate_snapspec(handle, 0, spec_orig, ZFSDataset.__snapshots_callback, arg)
^
------------------------------------------------------------
libzfs.pyx:3688:69: Cannot assign type 'int (zfs_handle_t *, void *) except * nogil' to 'zfs_iter_f'. Exception values are incompatible. Suggest adding 'noexcept' to type 'int (zfs_handle_t *, void *) except * nogil'.
Error compiling Cython file:
------------------------------------------------------------
...
if err not in (0, py_errno.ENOENT):
snap_config['failure'] = True
else:
if snap_config['recursive']:
with nogil:
err = ZFS.__iterate_filesystems(handle, 0, ZFSDataset.__gather_snapshots, arg)
^
------------------------------------------------------------
libzfs.pyx:3696:77: Cannot assign type 'int (zfs_handle_t *, void *) except * nogil' to 'zfs_iter_f'. Exception values are incompatible. Suggest adding 'noexcept' to type 'int (zfs_handle_t *, void *) except * nogil'.
Error compiling Cython file:
------------------------------------------------------------
...
iter.array = <uintptr_t *>malloc(128 * sizeof(uintptr_t))
if not iter.array:
raise MemoryError()
iter.alloc = 128
ZFS.__iterate_filesystems(self.handle, 0, self.__iterate, <void*>&iter)
^
------------------------------------------------------------
libzfs.pyx:3776:62: Cannot convert Python object to 'zfs_iter_f'
Error compiling Cython file:
------------------------------------------------------------
...
iter.array = <uintptr_t *>malloc(128 * sizeof(uintptr_t))
if not iter.array:
raise MemoryError()
iter.alloc = 128
libzfs.zfs_iter_snapshots(self.handle, False, self.__iterate, <void*>&iter, 0, 0)
^
------------------------------------------------------------
libzfs.pyx:3813:66: Cannot convert Python object to 'zfs_iter_f'
Error compiling Cython file:
------------------------------------------------------------
...
iter.array = <uintptr_t *>malloc(128 * sizeof(uintptr_t))
if not iter.array:
raise MemoryError()
iter.alloc = 128
ZFS.__iterate_bookmarks(self.handle, 0, self.__iterate, <void *>&iter)
^
------------------------------------------------------------
libzfs.pyx:3846:60: Cannot convert Python object to 'zfs_iter_f'
Error compiling Cython file:
------------------------------------------------------------
...
iter.array = <uintptr_t *>malloc(128 * sizeof(uintptr_t))
if not iter.array:
raise MemoryError()
iter.alloc = 128
ZFS.__iterate_filesystems(self.handle, 0, self.__iterate, <void*>&iter)
^
------------------------------------------------------------
libzfs.pyx:3776:62: Coercion from Python not allowed without the GIL
Error compiling Cython file:
------------------------------------------------------------
...
iter.array = <uintptr_t *>malloc(128 * sizeof(uintptr_t))
if not iter.array:
raise MemoryError()
iter.alloc = 128
ZFS.__iterate_filesystems(self.handle, 0, self.__iterate, <void*>&iter)
^
------------------------------------------------------------
libzfs.pyx:3776:62: Accessing Python attribute not allowed without gil
Error compiling Cython file:
------------------------------------------------------------
...
iter.array = <uintptr_t *>malloc(128 * sizeof(uintptr_t))
if not iter.array:
raise MemoryError()
iter.alloc = 128
libzfs.zfs_iter_snapshots(self.handle, False, self.__iterate, <void*>&iter, 0, 0)
^
------------------------------------------------------------
libzfs.pyx:3813:66: Coercion from Python not allowed without the GIL
Error compiling Cython file:
------------------------------------------------------------
...
iter.array = <uintptr_t *>malloc(128 * sizeof(uintptr_t))
if not iter.array:
raise MemoryError()
iter.alloc = 128
libzfs.zfs_iter_snapshots(self.handle, False, self.__iterate, <void*>&iter, 0, 0)
^
------------------------------------------------------------
libzfs.pyx:3813:66: Accessing Python attribute not allowed without gil
Error compiling Cython file:
------------------------------------------------------------
...
iter.array = <uintptr_t *>malloc(128 * sizeof(uintptr_t))
if not iter.array:
raise MemoryError()
iter.alloc = 128
ZFS.__iterate_bookmarks(self.handle, 0, self.__iterate, <void *>&iter)
^
------------------------------------------------------------
libzfs.pyx:3846:60: Coercion from Python not allowed without the GIL
Error compiling Cython file:
------------------------------------------------------------
...
iter.array = <uintptr_t *>malloc(128 * sizeof(uintptr_t))
if not iter.array:
raise MemoryError()
iter.alloc = 128
ZFS.__iterate_bookmarks(self.handle, 0, self.__iterate, <void *>&iter)
^
------------------------------------------------------------
Besides, the language_level derivative is also unset with current build system. It should be set to be one of 2, 3 or 3str. See https://cython.readthedocs.io/en/latest/src/userguide/migrating_to_cy30.html#python-3-syntax-semantics for more information.
Since cython3 3.x has been officially released, it would be great to make this project compatible with the new versions. Thanks!
Hey,
in order to work on iocage/iocage#831, I decided to set up a development environment on FreeBSD. But I can't install libzfs
(venv) lars@freebsd:/home/lars/PycharmProjects/iocage git:(master*) $ sudo pip install git+https://github.com/freenas/py-libzfs.git#egg=libzfs
Collecting libzfs from git+https://github.com/freenas/py-libzfs.git#egg=libzfs
Cloning https://github.com/freenas/py-libzfs.git to /tmp/pip-install-4ml0h6te/libzfs
Installing collected packages: libzfs
Running setup.py install for libzfs ... error
Complete output from command /home/lars/PycharmProjects/iocage/venv/bin/python -u -c "import setuptools, tokenize;__file__='/tmp/pip-install-4ml0h6te/libzfs/setup.py';f=getattr(tokenize, 'open', open)(__file__);code=f.read().replace('\r\n', '\n');f.close();exec(compile(code, __file__, 'exec'))" install --record /tmp/pip-record-7gsagzn5/install-record.txt --single-version-externally-managed --compile --install-headers /home/lars/PycharmProjects/iocage/venv/include/site/python3.7/libzfs:
WARNING: '' not a valid package name; please use only .-separated package names in setup.py
running install
running build
running build_py
creating build
creating build/lib.freebsd-12.0-RELEASE-p2-amd64-3.7
copying setup.py -> build/lib.freebsd-12.0-RELEASE-p2-amd64-3.7
running build_ext
cythoning libzfs.pyx to libzfs.c
Error compiling Cython file:
------------------------------------------------------------
...
# LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY
# OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF
# SUCH DAMAGE.
#
include "config.pxi"
^
------------------------------------------------------------
./pxd/libzfs.pxd:29:0: 'config.pxi' not found
Error compiling Cython file:
------------------------------------------------------------
...
cimport nvpair
cimport zfs
from types cimport *
IF HAVE_LZC_BOOKMARK:
^
------------------------------------------------------------
./pxd/libzfs.pxd:36:3: Compile-time name 'HAVE_LZC_BOOKMARK' not defined
Error compiling Cython file:
------------------------------------------------------------
...
cdef extern from "libzfs.h" nogil:
cdef enum:
MAXNAMELEN
MAXPATHLEN
IF HAVE_ZFS_MAXNAMELEN or HAVE_ZPOOL_MAXNAMELEN:
^
------------------------------------------------------------
./pxd/libzfs.pxd:53:7: Compile-time name 'HAVE_ZFS_MAXNAMELEN' not defined
Error compiling Cython file:
------------------------------------------------------------
...
cdef extern from "libzfs.h" nogil:
cdef enum:
MAXNAMELEN
MAXPATHLEN
IF HAVE_ZFS_MAXNAMELEN or HAVE_ZPOOL_MAXNAMELEN:
^
------------------------------------------------------------
./pxd/libzfs.pxd:53:30: Compile-time name 'HAVE_ZPOOL_MAXNAMELEN' not defined
Error compiling Cython file:
------------------------------------------------------------
...
IF HAVE_ZFS_MAXNAMELEN or HAVE_ZPOOL_MAXNAMELEN:
cdef enum:
ZFS_MAXNAMELEN
ZPOOL_MAXNAMELEN
IF HAVE_EZFS_SCRUB_PAUSED:
^
------------------------------------------------------------
./pxd/libzfs.pxd:58:7: Compile-time name 'HAVE_EZFS_SCRUB_PAUSED' not defined
Error compiling Cython file:
------------------------------------------------------------
...
extern int zpool_create(libzfs_handle_t *, const char *, nvpair.nvlist_t *,
nvpair.nvlist_t *, nvpair.nvlist_t *)
extern int zpool_destroy(zpool_handle_t *, const char *)
extern int zpool_add(zpool_handle_t *, nvpair.nvlist_t *)
IF HAVE_ZPOOL_SCAN == 3:
^
------------------------------------------------------------
./pxd/libzfs.pxd:185:7: Compile-time name 'HAVE_ZPOOL_SCAN' not defined
Error compiling Cython file:
------------------------------------------------------------
...
int forceunmount
extern int zfs_rename(zfs_handle_t *, const char *, const char *,
renameflags_t flags)
IF HAVE_SENDFLAGS_T_COMPRESS:
^
------------------------------------------------------------
./pxd/libzfs.pxd:426:7: Compile-time name 'HAVE_SENDFLAGS_T_COMPRESS' not defined
Error compiling Cython file:
------------------------------------------------------------
...
ctypedef int (*snapfilter_cb_t)(zfs_handle_t *, void *)
extern int zfs_send(zfs_handle_t *, const char *, const char *,
sendflags_t *, int, snapfilter_cb_t, void *, nvpair.nvlist_t **) nogil
IF HAVE_ZFS_SEND_ONE == 4:
^
------------------------------------------------------------
./pxd/libzfs.pxd:459:7: Compile-time name 'HAVE_ZFS_SEND_ONE' not defined
Error compiling Cython file:
------------------------------------------------------------
...
IF HAVE_ZFS_SEND_ONE == 4:
extern int zfs_send_one(zfs_handle_t *, const char *, int, int) nogil
ELSE:
extern int zfs_send_one(zfs_handle_t *, const char *, int) nogil
IF HAVE_ZFS_SEND_RESUME or HAVE_ZFS_SEND_RESUME_TOKEN_TO_NVLIST:
^
------------------------------------------------------------
./pxd/libzfs.pxd:464:7: Compile-time name 'HAVE_ZFS_SEND_RESUME' not defined
Error compiling Cython file:
------------------------------------------------------------
...
IF HAVE_ZFS_SEND_ONE == 4:
extern int zfs_send_one(zfs_handle_t *, const char *, int, int) nogil
ELSE:
extern int zfs_send_one(zfs_handle_t *, const char *, int) nogil
IF HAVE_ZFS_SEND_RESUME or HAVE_ZFS_SEND_RESUME_TOKEN_TO_NVLIST:
^
------------------------------------------------------------
./pxd/libzfs.pxd:464:31: Compile-time name 'HAVE_ZFS_SEND_RESUME_TOKEN_TO_NVLIST' not defined
Error compiling Cython file:
------------------------------------------------------------
...
ctypedef enum diff_flags_t:
ZFS_DIFF_PARSEABLE = 0x1,
ZFS_DIFF_TIMESTAMP = 0x2,
ZFS_DIFF_CLASSIFY = 0x4
IF HAVE_ZFS_RECEIVE == 7:
^
------------------------------------------------------------
./pxd/libzfs.pxd:495:7: Compile-time name 'HAVE_ZFS_RECEIVE' not defined
Error compiling Cython file:
------------------------------------------------------------
...
ZFS_DIFF_CLASSIFY = 0x4
IF HAVE_ZFS_RECEIVE == 7:
extern int zfs_receive(libzfs_handle_t *, const char *, recvflags_t *,
int, nvpair.nvlist_t *, nvpair.nvlist_t *, void *) # XXX: last argument should be avl_tree_t *
ELIF HAVE_ZFS_RECEIVE == 6:
^
------------------------------------------------------------
./pxd/libzfs.pxd:498:9: Compile-time name 'HAVE_ZFS_RECEIVE' not defined
Error compiling Cython file:
------------------------------------------------------------
...
from libc.stdlib cimport free, realloc
GLOBAL_CONTEXT_LOCK = threading.Lock()
include "config.pxi"
^
------------------------------------------------------------
libzfs.pyx:48:0: 'config.pxi' not found
Error compiling Cython file:
------------------------------------------------------------
...
DRYRUN = 5
PARSABLE = 6
PROGRESS = 7
LARGEBLOCK = 8
EMBED_DATA = 9
IF HAVE_SENDFLAGS_T_COMPRESS:
^
------------------------------------------------------------
libzfs.pyx:254:7: Compile-time name 'HAVE_SENDFLAGS_T_COMPRESS' not defined
Error compiling Cython file:
------------------------------------------------------------
...
DIRECTORY = '/'
SYMLINK = '@'
SOCKET = '='
IF HAVE_ZFS_MAX_DATASET_NAME_LEN:
^
------------------------------------------------------------
libzfs.pyx:274:3: Compile-time name 'HAVE_ZFS_MAX_DATASET_NAME_LEN' not defined
Error compiling Cython file:
------------------------------------------------------------
...
'oldpath': getattr(self, 'oldpath', None)
}
IF HAVE_LZC_SEND_FLAG_EMBED_DATA:
^
------------------------------------------------------------
libzfs.pyx:323:3: Compile-time name 'HAVE_LZC_SEND_FLAG_EMBED_DATA' not defined
Error compiling Cython file:
------------------------------------------------------------
...
flags.force = True
if nomount:
flags.nomount = True
IF HAVE_RECVFLAGS_T_RESUMABLE:
^
------------------------------------------------------------
libzfs.pyx:770:11: Compile-time name 'HAVE_RECVFLAGS_T_RESUMABLE' not defined
Error compiling Cython file:
------------------------------------------------------------
...
IF HAVE_RECVFLAGS_T_RESUMABLE:
if resumable:
flags.resumable = True
IF HAVE_ZFS_RECEIVE == 7:
^
------------------------------------------------------------
libzfs.pyx:774:11: Compile-time name 'HAVE_ZFS_RECEIVE' not defined
Error compiling Cython file:
------------------------------------------------------------
...
if props:
props_nvl = NVList(otherdict=props)
c_props_nvl = props_nvl.handle
with nogil:
IF HAVE_ZFS_RECEIVE == 6:
^
------------------------------------------------------------
libzfs.pyx:799:19: Compile-time name 'HAVE_ZFS_RECEIVE' not defined
Error compiling Cython file:
------------------------------------------------------------
...
out.append('log')
out.append(topology.get('log'))
return out
IF HAVE_SENDFLAGS_T_TYPEDEF and HAVE_ZFS_SEND_RESUME:
^
------------------------------------------------------------
libzfs.pyx:903:7: Compile-time name 'HAVE_SENDFLAGS_T_TYPEDEF' not defined
Error compiling Cython file:
------------------------------------------------------------
...
out.append('log')
out.append(topology.get('log'))
return out
IF HAVE_SENDFLAGS_T_TYPEDEF and HAVE_ZFS_SEND_RESUME:
^
------------------------------------------------------------
libzfs.pyx:903:36: Compile-time name 'HAVE_ZFS_SEND_RESUME' not defined
Error compiling Cython file:
------------------------------------------------------------
...
ret = libzfs.zfs_send_resume(self.handle, &cflags, c_fd, c_token)
if ret != 0:
raise ZFSException(self.errno, self.errstr)
IF HAVE_ZFS_SEND_RESUME_TOKEN_TO_NVLIST:
^
------------------------------------------------------------
libzfs.pyx:921:7: Compile-time name 'HAVE_ZFS_SEND_RESUME_TOKEN_TO_NVLIST' not defined
Error compiling Cython file:
------------------------------------------------------------
...
elapsed = (int(time.time()) - self.stat[10]) or 1
pass_exam = self.stat[9] or 1
rate = pass_exam / elapsed
return int((total - examined) / rate)
IF HAVE_POOL_SCAN_STAT_T_PAUSE:
^
------------------------------------------------------------
libzfs.pyx:1673:7: Compile-time name 'HAVE_POOL_SCAN_STAT_T_PAUSE' not defined
Error compiling Cython file:
------------------------------------------------------------
...
return None
if not self.bytes_to_scan:
return 0
IF HAVE_POOL_SCAN_STAT_T_PAUSE:
^
------------------------------------------------------------
libzfs.pyx:1707:15: Compile-time name 'HAVE_POOL_SCAN_STAT_T_PAUSE' not defined
Error compiling Cython file:
------------------------------------------------------------
...
def start_scrub(self):
cdef int ret
with nogil:
IF HAVE_ZPOOL_SCAN == 3:
^
------------------------------------------------------------
libzfs.pyx:2157:15: Compile-time name 'HAVE_ZPOOL_SCAN' not defined
Error compiling Cython file:
------------------------------------------------------------
...
def stop_scrub(self):
cdef int ret
with nogil:
IF HAVE_ZPOOL_SCAN == 3:
^
------------------------------------------------------------
libzfs.pyx:2171:15: Compile-time name 'HAVE_ZPOOL_SCAN' not defined
Error compiling Cython file:
------------------------------------------------------------
...
if fromname:
cfromname = fromname
with nogil:
IF HAVE_LZC_SEND_SPACE == 4:
^
------------------------------------------------------------
libzfs.pyx:2446:15: Compile-time name 'HAVE_LZC_SEND_SPACE' not defined
Error compiling Cython file:
------------------------------------------------------------
...
cflags.largeblock = 1
if SendFlag.EMBED_DATA in flags:
cflags.embed_data = 1
IF HAVE_SENDFLAGS_T_COMPRESS:
^
------------------------------------------------------------
libzfs.pyx:2997:7: Compile-time name 'HAVE_SENDFLAGS_T_COMPRESS' not defined
building 'libzfs' extension
creating build/temp.freebsd-12.0-RELEASE-p2-amd64-3.7
cc -pthread -Wno-unused-result -Wsign-compare -Wunreachable-code -DNDEBUG -O2 -pipe -fstack-protector -fno-strict-aliasing -fPIC -I/usr/src/cddl/lib/libumem -I/usr/src/sys/cddl/compat/opensolaris/ -I/usr/src/sys/cddl/compat/opensolaris -I/usr/src/cddl/compat/opensolaris/include -I/usr/src/cddl/compat/opensolaris/lib/libumem -I/usr/src/cddl/contrib/opensolaris/lib/libzpool/common -I/usr/src/sys/cddl/contrib/opensolaris/common/zfs -I/usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs -I/usr/src/sys/cddl/contrib/opensolaris/uts/common/sys -I/usr/src/cddl/contrib/opensolaris/head -I/usr/src/sys/cddl/contrib/opensolaris/uts/common -I/usr/src/cddl/contrib/opensolaris/lib/libnvpair -I/usr/src/cddl/contrib/opensolaris/lib/libuutil/common -I/usr/src/cddl/contrib/opensolaris/lib/libzfs/common -I/usr/src/cddl/contrib/opensolaris/lib/libzfs_core/common -I/home/lars/PycharmProjects/iocage/venv/include -I/usr/local/include/python3.7m -c libzfs.c -o build/temp.freebsd-12.0-RELEASE-p2-amd64-3.7/libzfs.o -DNEED_SOLARIS_BOOLEAN -D_XPG6 -g
libzfs.c:1:2: error: Do not use this file, it is the result of a failed Cython compilation.
#error Do not use this file, it is the result of a failed Cython compilation.
^
1 error generated.
error: command 'cc' failed with exit status 1
----------------------------------------
Command "/home/lars/PycharmProjects/iocage/venv/bin/python -u -c "import setuptools, tokenize;__file__='/tmp/pip-install-4ml0h6te/libzfs/setup.py';f=getattr(tokenize, 'open', open)(__file__);code=f.read().replace('\r\n', '\n');f.close();exec(compile(code, __file__, 'exec'))" install --record /tmp/pip-record-7gsagzn5/install-record.txt --single-version-externally-managed --compile --install-headers /home/lars/PycharmProjects/iocage/venv/include/site/python3.7/libzfs" failed with error code 1 in /tmp/pip-install-4ml0h6te/libzfs/
My FreeBSd is the following:
(venv) lars@freebsd:/home/lars/PycharmProjects/iocage git:(master*) $ freebsd-version
12.0-RELEASE-p2
(venv) lars@freebsd:/home/lars/PycharmProjects/iocage git:(master*) $ uname -a
FreeBSD freebsd 12.0-RELEASE-p2 FreeBSD 12.0-RELEASE-p2 GENERIC amd64
Could you please help me?
Best regards
Lars
I'm using py-libzfs to output a large json dump of ZFS status... just libzfs.ZFS().getstate() out to json. This works great. However, I've hit a situation where a test zpool with a raidz1 with file-based vdevs is unavailable, .getstate() returns 'KeyError: 'unsup_feat'
>>> zfs = libzfs.ZFS().__getstate__()
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "libzfs.pyx", line 417, in libzfs.ZFS.__getstate__
File "libzfs.pyx", line 2435, in libzfs.ZFSPool.__getstate__
File "libzfs.pyx", line 2656, in libzfs.ZFSPool.status_detail.__get__
File "libzfs.pyx", line 2617, in libzfs.ZFSPool.__unsup_features
KeyError: 'unsup_feat'
The test zpool in question has a status code of PoolStatus.MISSING_DEV_NR , I would expect getstate() to still work even though one of the zpools is in a fault condition. In fact, getstate should always work and shouldn't fail like this
Any ideas what's going on here?
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
Django
The Web framework for perfectionists with deadlines.
Laravel
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
Microsoft
Open source projects and samples from Microsoft.
Google
Google ❤️ Open Source for everyone.
Alibaba
Alibaba Open Source for everyone
D3
Data-Driven Documents codes.
Tencent
China tencent open source team.