Need man pages for all utilities.

I'm wondering whether it might be useful to include totsubdirs in the summary tables to complement totfiles and totlinks.

Once we have #149 completed, we'll need to develop a process by which we capture a longitudinal snapshot from the GUFI tree. There is both an initial snapshot process and there is potentially an incremental snapshot process. For example, here is one proposal:

1. Create an initial longitudinal snapshot by doing a roll-up of all the vrsummary tables into a single vrsummary table which includes a depth column and a pinode column.
   1.1. However, maybe this is not part of the longitudinal snap but is part of the GUFI tree build
2. Create a copy of this single augmented rolled-up summary table (SARUST).
3. At some future time, create a new copy of the SARUST.
4. Potentially reduce savings costs by reducing one of the copies into an increment off the other
   4.1. This should be a lot of savings since there may be 1B entries in the table (1B directories in a GUFI tree) but only a very small number of them will have changes between longitudinal snapshots

So, in this issue, we can define how we capture the initial snapshot as well as how we create the increment.

Some executables still use C-Thread-Pool instead of QueuePerThreadPool.

If a directory tree has special chars GUFI will throw warnings and skip those files

eg # or ?

Cannot attach database as "tree": unable to open database: file:/tmp/GUFI/wgsa/cgillies/wonderland/home/cgillies/workspace-ggts-3.4.0.RELEASE/genevetter_backup/node_modules/bower/node_modules/inquirer/node_modules/cli-color/node_modules/es5-ext/test/reg-exp/#/replace/db.db?mode=ro

Error: no such table: entries: /tmp/GUFI/KStorageLite/2016_DropSeq_Run1445/SingleCellData/KPMP_DiseaseBiopsies/Repeats_NotUsed/S-1908-000945-B_?/Sample_1125-EO-2/SC_RNA_COUNTER_CS/SC_RNA_COUNTER/SC_RNA_ANALYZER/CHOOSE_DIMENSION_REDUCTION_OUTPUT/fork0/chnk0-udc60c05d5e/files/pca_csv/10_components/db.db: "       INSERT INTO sument select uid, name, size, atime,       CASE WHEN size > 100 * 1024 * 1024 THEN size ELSE 0 END AS oversize     FROM entries    WHERE type='f';"

The queries still run but the logs are reports are flooded with these messages and the data in those folders are skipped in the query.

The most recent commit adding filesize distro to gufi_stats is very cool! Thanks for adding that.

I also noticed that there are a few useful predefined bins in vrsummary such as totltk,totmtk,totltm,totmtm,totmtg,totmtt. Would it be possible to add totzero as well so we can easily see the count of zero-byte files? Also, maybe a bit more granularity in the bins? Or, is the new gufi_stats -c filesize-log2-bins actually all that is needed here. Is it just as quick to use gufi_stats to get this as it is to query those fields in vrsummary?

Also, would it be possible to add zero-size files into the gufi_stats -c filesize-log2-bins output please?

Hello,

I didn't see any references to special characters in file names, so I thought it would be best to raise another issue (don't shoot the messenger, ha!).

I'm seeing messages related to opening a DB when there is a '%' in the file name (users going to user; probably doesn't know that's a reserved character in SQL):

I can always weed out that particular user's path if need be. At any rate, I figured the developers would want to know if they didn't already.

Thank you!
John DeSantis

Hello,

First and foremost, thank you for this software! It works incredibly well, and we're seeing significant differences querying the index file system (multiple PB's) vs. using the traditional find command.

I've run into an issue, unfortunately, in terms of memory usage during a re-index process. Basically, the gufi_dir2index process consumes too much memory leading to system instability. The test system in question has 256 GB of RAM available and the re-index has caused many OOM events.

Are there any controls and/or configuration options I can set on the GUFI side in order to not consume all available memory, leading to OOM events? For what it's worth, there doesn't seem to be a correlation between the number of files and/or space consumed on the file system that we're indexing. I built GUFI from the latest commit at the time, which was a8d9328.

Thank you!
John DeSantis

gufi_ls -R on the root directory breaks if the database file in the root directory was made by gufi_dir2index -z 0 /search /search since the recursive query joins on pinodes, and the /search directory will have an inode that is not the parent of any of the subdirectories.

Currently, multiple inputs to gufi_ls are processed one at a time. When the -S flag is used, the output of each input is sorted, but the expected output is all outputs being sorted as one set of data.

gufi_ls filename does not work

fb5a819 only works for directories

We want to enable a longitudinal study. This requires multiple phases. This issue is about the first phase. Subsequent issues will be created to describe subsequent phases so we can keep the conversations nicely organized.

We need to decide on what to capture in a longitudinal snapshot. One option would be merely to create a snapshot of the full GUFI tree but that's very large and includes a lot of information that wouldn't be necessary for a longitudinal study. Another option is to create a tool which copies a subset of the GUFI tree. It could either copy a subset of the GUFI tree files or it could do GUFI queries and save the output of those queries. That particular question is perhaps a subsequent phase.

Regardless of how we capture a longitudinal snapshot, we also have realized we first want to add additional information into the GUFI tree. Especially we want to capture histograms describing attributes of the entries within a directory into the summary tables. For example, how many files are size [0,1), [1-2), [2-4), etc. To collaboratively decide on the histograms required, we are defining that in this spreadsheet:

https://docs.google.com/spreadsheets/d/17PSZxHLVj731bI9PKY3E1V-TzSAHJiIahioc2nujo1U/edit#gid=0

With -size option if we pass '-ve' number along with size in k, M or G then it's giving an error,

example command which I tried,
gufi_find -size -10M

Fix rpm naming to the correct release, arch, etc.

Today we depend on --select in gufi_find to probe for more info in the dbs. This is fine for now, but some have suggested we may need a gufi_stat command that will get you the normal stat output and format accordingly.

A side effect of #37 is that empty database files are created when ATTACHing a non-existent file.

In looking to try out GUFI I've reviewed the repository thoroughly alongside various papers and presentations.
I got side tracked for quite a while trying to follow the installation instructions in this GUFI.docx file, which I realise now must be quite outdated.

After searching the repo on github for references to make I realised there were other references to cmake and in the end found the Quick Start instructions in README.md. I should have paid closer attention at the start, guess I overlooked it.

It would help to include a reference to that Quick Start section here, and a note in this page that the GUFI.docx is out of date.

Wonder if there are queries to list big directories with GUFI db.
Thanks !

So far found one issue related to gufi_stat. closedir on null causes crash in OSX and not on linux, moving closedir into the successful opendir block solves one issue. More issues exist (potentially related -- passing in non-existent dir)

Special chars are not handled in gufi_ls cmd, e.g. if my directory name containing special char like (dir#1@$) then gufi_ls is failing and not giving expected output.
I also verified by creating dirs having special chars. but gufi_ls is failing only dir. which are having '#' with their name.

Hi there,

I'm trying to compile this software and encountered the issues below when GPFS libraries were found.

I validated all dependencies and tried available GCC versions (4, 5, 6, 7, 9, 11, 12). I tried on CentOS 7 and Rocky 8.5, both of them having the same issue.

If I skip the GPFS part, the compilation finishes without any issue. I am wondering if there is anything we can do with the code?

Here is my output

[root@wmc040 build]# cmake ..
-- GUFI 0.6.4 main ad271c9
-- Using Python Interpreter: python3
-- Found TRUNCATE: /usr/bin/truncate
-- Found AWK: /usr/bin/awk
-- Found DIFF: /usr/bin/diff
-- Found FIND: /usr/bin/find
-- Found GREP: /usr/bin/grep
-- Found PATCH: /usr/bin/patch
-- Found SED: /usr/bin/sed
-- Found SETFATTR: /usr/bin/setfattr
-- Found PkgConfig: /usr/bin/pkg-config (found version "1.4.2")
-- Found XATTR: sys/xattr.h
-- xattr headers located in /usr/include
-- Looking for pthread.h
-- Looking for pthread.h - found
-- Performing Test CMAKE_HAVE_LIBC_PTHREAD
-- Performing Test CMAKE_HAVE_LIBC_PTHREAD - Failed
-- Looking for pthread_create in pthreads
-- Looking for pthread_create in pthreads - not found
-- Looking for pthread_create in pthread
-- Looking for pthread_create in pthread - found
-- Found Threads: TRUE
-- Checking for one of the modules 'libpcre2-8'
-- Checking for one of the modules 'zlib'
-- ZLIB headers located in /usr/include
-- ZLIB libraries located in /usr/lib64
-- Checking for one of the modules 'fuse'
-- FUSE not found. Not building bffuse and bfresultfuse
-- GPFS libraries found; building gpfs-scan-tool
-- Checking for one of the modules 'db2'
-- DB2 not found
-- Looking for localtime_r
-- Looking for localtime_r - found
-- LaTeX building is disabled (currently OFF).
-- shellcheck not found
-- pylint not found
-- checkstyle disabled
-- git version: 2.31.1
-- Not adding performance history framework (need PRINT_CUMULATIVE_TIMES enabled)
-- rpmbuild not found. 'make package' will not be available.
-- Configuring done
-- Generating done
-- Build files have been written to: /root/GUFI/build

[root@wmc040 build]# make
Installing SQLite3
Patching SQLite3 Pathname Length
patching file sqlite3.c
Patching SQLite3 Max Attach Count
patching file sqlite3.c
checking for a BSD-compatible install... /usr/bin/install -c
checking whether build environment is sane... yes
checking for a race-free mkdir -p... /usr/bin/mkdir -p
checking for gawk... gawk
checking whether make sets $(MAKE)... yes
checking whether make supports nested variables... yes
checking whether make supports the include directive... yes (GNU style)
checking for gcc... gcc
checking whether the C compiler works... yes
checking for C compiler default output file name... a.out
checking for suffix of executables...
checking whether we are cross compiling... no
checking for suffix of object files... o
checking whether the compiler supports GNU C... yes
checking whether gcc accepts -g... yes
checking for gcc option to enable C11 features... none needed
checking whether gcc understands -c and -o together... yes
checking dependency style of gcc... gcc3
checking for special C compiler options needed for large files... no
checking for _FILE_OFFSET_BITS value needed for large files... no
checking for gcc... (cached) gcc
checking whether the compiler supports GNU C... (cached) yes
checking whether gcc accepts -g... (cached) yes
checking for gcc option to enable C11 features... (cached) none needed
checking whether gcc understands -c and -o together... (cached) yes
checking dependency style of gcc... (cached) gcc3
checking build system type... x86_64-pc-linux-gnu
checking host system type... x86_64-pc-linux-gnu
checking how to print strings... printf
checking for a sed that does not truncate output... /usr/bin/sed
checking for grep that handles long lines and -e... /usr/bin/grep
checking for egrep... /usr/bin/grep -E
checking for fgrep... /usr/bin/grep -F
checking for ld used by gcc... /usr/bin/ld
checking if the linker (/usr/bin/ld) is GNU ld... yes
checking for BSD- or MS-compatible name lister (nm)... /usr/bin/nm -B
checking the name lister (/usr/bin/nm -B) interface... BSD nm
checking whether ln -s works... yes
checking the maximum length of command line arguments... 3458764513820540925
checking how to convert x86_64-pc-linux-gnu file names to x86_64-pc-linux-gnu format... func_convert_file_noop
checking how to convert x86_64-pc-linux-gnu file names to toolchain format... func_convert_file_noop
checking for /usr/bin/ld option to reload object files... -r
checking for objdump... objdump
checking how to recognize dependent libraries... pass_all
checking for dlltool... no
checking how to associate runtime and link libraries... printf %s\n
checking for ar... ar
checking for archiver @file support... @
checking for strip... strip
checking for ranlib... ranlib
checking command to parse /usr/bin/nm -B output from gcc object... ok
checking for sysroot... no
checking for a working dd... /usr/bin/dd
checking how to truncate binary pipes... /usr/bin/dd bs=4096 count=1
checking for mt... no
checking if : is a manifest tool... no
checking for stdio.h... yes
checking for stdlib.h... yes
checking for string.h... yes
checking for inttypes.h... yes
checking for stdint.h... yes
checking for strings.h... yes
checking for sys/stat.h... yes
checking for sys/types.h... yes
checking for unistd.h... yes
checking for dlfcn.h... yes
checking for objdir... .libs
checking if gcc supports -fno-rtti -fno-exceptions... no
checking for gcc option to produce PIC... -fPIC -DPIC
checking if gcc PIC flag -fPIC -DPIC works... yes
checking if gcc static flag -static works... no
checking if gcc supports -c -o file.o... yes
checking if gcc supports -c -o file.o... (cached) yes
checking whether the gcc linker (/usr/bin/ld -m elf_x86_64) supports shared libraries... yes
checking whether -lc should be explicitly linked in... no
checking dynamic linker characteristics... GNU/Linux ld.so
checking how to hardcode library paths into programs... immediate
checking whether stripping libraries is possible... yes
checking if libtool supports shared libraries... yes
checking whether to build shared libraries... yes
checking whether to build static libraries... yes
checking for fdatasync... yes
checking for usleep... yes
checking for fullfsync... no
checking for localtime_r... yes
checking for gmtime_r... yes
checking for gcc options needed to detect all undeclared functions... none needed
checking whether strerror_r is declared... yes
checking whether strerror_r returns char *... no
checking for editline/readline.h... no
checking for readline/readline.h... yes
checking for library containing tgetent... -ltermcap
checking for library containing readline... -lreadline
checking for library containing pthread_create... -lpthread
checking for library containing pthread_mutexattr_init... none required
checking for library containing dlopen... -ldl
checking for whether to support dynamic extensions... yes
checking SQL math functions... enabled
checking for library containing ceil... -lm
checking FTS4 extension... enabled
checking FTS3 extension... disabled
checking FTS5 extension... enabled
checking for library containing log... none required
checking RTREE extension... enabled
checking Session extension... disabled
checking Build type... release
checking for posix_fallocate... yes
checking for zlib.h... yes
checking for library containing deflate... -lz
checking for library containing system... none required
checking that generated files are newer than configure... done
configure: creating ./config.status
config.status: creating Makefile
config.status: creating sqlite3.pc
config.status: executing depfiles commands
config.status: executing libtool commands
In function ‘zipfileComparePath’,
inlined from ‘zipfileUpdate’ at ../shell.c:10105:11:
../shell.c:9964:17: warning: ‘__builtin_memcmp_eq’ specified size between 18446744071562067968 and 18446744073709551615 exceeds maximum object size 9223372036854775807 [-Wstringop-overflow=]
if( nA==nB && memcmp(zA, zB, nA)==0 ) return 0;
^~~~~~~~~~~~~~~~~~
libtool: link: gcc -D_REENTRANT=1 -DSQLITE_THREADSAFE=1 -DSQLITE_ENABLE_MATH_FUNCTIONS -DSQLITE_ENABLE_FTS4 -DSQLITE_ENABLE_FTS5 -DSQLITE_ENABLE_RTREE -DSQLITE_ENABLE_GEOPOLY -DSQLITE_HAVE_ZLIB -DSQLITE_ENABLE_EXPLAIN_COMMENTS -DSQLITE_DQS=0 -DSQLITE_ENABLE_DBPAGE_VTAB -DSQLITE_ENABLE_STMTVTAB -DSQLITE_ENABLE_DBSTAT_VTAB -DSQLITE_DEFAULT_AUTOMATIC_INDEX=0 -DSQLITE_DEFAULT_AUTOVACUUM=0 -DSQLITE_DEFAULT_CACHE_SIZE=16777216 -DSQLITE_DEFAULT_LOCKING_MODE=1 -DSQLITE_DEFAULT_MEMSTATUS=0 -DSQLITE_DEFAULT_SYNCHRONOUS=0 -DSQLITE_DEFAULT_WAL_SYNCHRONOUS=0 -DSQLITE_DQS=0 -DSQLITE_MAX_ATTACHED=254 -DSQLITE_MAX_EXPR_DEPTH=0 -USQLITE_THREADSAFE -DSQLITE_THREADSAFE=0 -DSQLITE_TEMP_STORE=3 -DSQLITE_USE_URI -USQLITE_ENABLE_FTS4 -USQLITE_ENABLE_FTS5 -USQLITE_ENABLE_GEOPOLY -DSQLITE_ENABLE_MATH_FUNCTIONS -USQLITE_ENABLE_RTREE -DSQLITE_OMIT_DEPRECATED -DSQLITE_OMIT_JSON -DSQLITE_OMIT_PROGRESS_CALLBACK -O3 -o sqlite3 sqlite3-shell.o sqlite3-sqlite3.o -lreadline -ltermcap -lz -lm -ldl -lpthread
libtool: compile: gcc -DPACKAGE_NAME="sqlite" -DPACKAGE_TARNAME="sqlite" -DPACKAGE_VERSION="3.43.1" "-DPACKAGE_STRING="sqlite 3.43.1"" -DPACKAGE_BUGREPORT="http://www.sqlite.org\" -DPACKAGE_URL="" -DPACKAGE="sqlite" -DVERSION="3.43.1" -DHAVE_STDIO_H=1 -DHAVE_STDLIB_H=1 -DHAVE_STRING_H=1 -DHAVE_INTTYPES_H=1 -DHAVE_STDINT_H=1 -DHAVE_STRINGS_H=1 -DHAVE_SYS_STAT_H=1 -DHAVE_SYS_TYPES_H=1 -DHAVE_UNISTD_H=1 -DSTDC_HEADERS=1 -DHAVE_DLFCN_H=1 -DLT_OBJDIR=".libs/" -DHAVE_FDATASYNC=1 -DHAVE_USLEEP=1 -DHAVE_LOCALTIME_R=1 -DHAVE_GMTIME_R=1 -DHAVE_DECL_STRERROR_R=1 -DHAVE_STRERROR_R=1 -DHAVE_READLINE_READLINE_H=1 -DHAVE_READLINE=1 -DHAVE_POSIX_FALLOCATE=1 -DHAVE_ZLIB_H=1 -I. -I.. -D_REENTRANT=1 -DSQLITE_THREADSAFE=1 -DSQLITE_ENABLE_MATH_FUNCTIONS -DSQLITE_ENABLE_FTS4 -DSQLITE_ENABLE_FTS5 -DSQLITE_ENABLE_RTREE -DSQLITE_ENABLE_GEOPOLY -DSQLITE_HAVE_ZLIB -DSQLITE_DEFAULT_AUTOMATIC_INDEX=0 -DSQLITE_DEFAULT_AUTOVACUUM=0 -DSQLITE_DEFAULT_CACHE_SIZE=16777216 -DSQLITE_DEFAULT_LOCKING_MODE=1 -DSQLITE_DEFAULT_MEMSTATUS=0 -DSQLITE_DEFAULT_SYNCHRONOUS=0 -DSQLITE_DEFAULT_WAL_SYNCHRONOUS=0 -DSQLITE_DQS=0 -DSQLITE_MAX_ATTACHED=254 -DSQLITE_MAX_EXPR_DEPTH=0 -USQLITE_THREADSAFE -DSQLITE_THREADSAFE=0 -DSQLITE_TEMP_STORE=3 -DSQLITE_USE_URI -USQLITE_ENABLE_FTS4 -USQLITE_ENABLE_FTS5 -USQLITE_ENABLE_GEOPOLY -DSQLITE_ENABLE_MATH_FUNCTIONS -USQLITE_ENABLE_RTREE -DSQLITE_OMIT_DEPRECATED -DSQLITE_OMIT_JSON -DSQLITE_OMIT_PROGRESS_CALLBACK -O3 -MT sqlite3.lo -MD -MP -MF .deps/sqlite3.Tpo -c ../sqlite3.c -fPIC -DPIC -o .libs/sqlite3.o
libtool: compile: gcc -DPACKAGE_NAME="sqlite" -DPACKAGE_TARNAME="sqlite" -DPACKAGE_VERSION="3.43.1" "-DPACKAGE_STRING="sqlite 3.43.1"" -DPACKAGE_BUGREPORT="http://www.sqlite.org\" -DPACKAGE_URL="" -DPACKAGE="sqlite" -DVERSION="3.43.1" -DHAVE_STDIO_H=1 -DHAVE_STDLIB_H=1 -DHAVE_STRING_H=1 -DHAVE_INTTYPES_H=1 -DHAVE_STDINT_H=1 -DHAVE_STRINGS_H=1 -DHAVE_SYS_STAT_H=1 -DHAVE_SYS_TYPES_H=1 -DHAVE_UNISTD_H=1 -DSTDC_HEADERS=1 -DHAVE_DLFCN_H=1 -DLT_OBJDIR=".libs/" -DHAVE_FDATASYNC=1 -DHAVE_USLEEP=1 -DHAVE_LOCALTIME_R=1 -DHAVE_GMTIME_R=1 -DHAVE_DECL_STRERROR_R=1 -DHAVE_STRERROR_R=1 -DHAVE_READLINE_READLINE_H=1 -DHAVE_READLINE=1 -DHAVE_POSIX_FALLOCATE=1 -DHAVE_ZLIB_H=1 -I. -I.. -D_REENTRANT=1 -DSQLITE_THREADSAFE=1 -DSQLITE_ENABLE_MATH_FUNCTIONS -DSQLITE_ENABLE_FTS4 -DSQLITE_ENABLE_FTS5 -DSQLITE_ENABLE_RTREE -DSQLITE_ENABLE_GEOPOLY -DSQLITE_HAVE_ZLIB -DSQLITE_DEFAULT_AUTOMATIC_INDEX=0 -DSQLITE_DEFAULT_AUTOVACUUM=0 -DSQLITE_DEFAULT_CACHE_SIZE=16777216 -DSQLITE_DEFAULT_LOCKING_MODE=1 -DSQLITE_DEFAULT_MEMSTATUS=0 -DSQLITE_DEFAULT_SYNCHRONOUS=0 -DSQLITE_DEFAULT_WAL_SYNCHRONOUS=0 -DSQLITE_DQS=0 -DSQLITE_MAX_ATTACHED=254 -DSQLITE_MAX_EXPR_DEPTH=0 -USQLITE_THREADSAFE -DSQLITE_THREADSAFE=0 -DSQLITE_TEMP_STORE=3 -DSQLITE_USE_URI -USQLITE_ENABLE_FTS4 -USQLITE_ENABLE_FTS5 -USQLITE_ENABLE_GEOPOLY -DSQLITE_ENABLE_MATH_FUNCTIONS -USQLITE_ENABLE_RTREE -DSQLITE_OMIT_DEPRECATED -DSQLITE_OMIT_JSON -DSQLITE_OMIT_PROGRESS_CALLBACK -O3 -MT sqlite3.lo -MD -MP -MF .deps/sqlite3.Tpo -c ../sqlite3.c -o sqlite3.o >/dev/null 2>&1
libtool: link: gcc -shared -fPIC -DPIC .libs/sqlite3.o -lz -lm -ldl -lpthread -O3 -Wl,-soname -Wl,libsqlite3.so.0 -o .libs/libsqlite3.so.0.8.6
libtool: link: (cd ".libs" && rm -f "libsqlite3.so.0" && ln -s "libsqlite3.so.0.8.6" "libsqlite3.so.0")
libtool: link: (cd ".libs" && rm -f "libsqlite3.so" && ln -s "libsqlite3.so.0.8.6" "libsqlite3.so")
libtool: link: ar cr .libs/libsqlite3.a sqlite3.o
libtool: link: ranlib .libs/libsqlite3.a
libtool: link: ( cd ".libs" && rm -f "libsqlite3.la" && ln -s "../libsqlite3.la" "libsqlite3.la" )
/usr/bin/mkdir -p '/root/GUFI/build/deps/sqlite3/lib'
/bin/sh ./libtool --mode=install /usr/bin/install -c libsqlite3.la '/root/GUFI/build/deps/sqlite3/lib'
libtool: install: /usr/bin/install -c .libs/libsqlite3.so.0.8.6 /root/GUFI/build/deps/sqlite3/lib/libsqlite3.so.0.8.6
libtool: install: (cd /root/GUFI/build/deps/sqlite3/lib && { ln -s -f libsqlite3.so.0.8.6 libsqlite3.so.0 || { rm -f libsqlite3.so.0 && ln -s libsqlite3.so.0.8.6 libsqlite3.so.0; }; })
libtool: install: (cd /root/GUFI/build/deps/sqlite3/lib && { ln -s -f libsqlite3.so.0.8.6 libsqlite3.so || { rm -f libsqlite3.so && ln -s libsqlite3.so.0.8.6 libsqlite3.so; }; })
libtool: install: /usr/bin/install -c .libs/libsqlite3.lai /root/GUFI/build/deps/sqlite3/lib/libsqlite3.la
libtool: install: /usr/bin/install -c .libs/libsqlite3.a /root/GUFI/build/deps/sqlite3/lib/libsqlite3.a
libtool: install: chmod 644 /root/GUFI/build/deps/sqlite3/lib/libsqlite3.a
libtool: install: ranlib /root/GUFI/build/deps/sqlite3/lib/libsqlite3.a
libtool: finish: PATH="/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/root/bin:/usr/lpp/mmfs/bin:/sbin" ldconfig -n /root/GUFI/build/deps/sqlite3/lib

Libraries have been installed in:
/root/GUFI/build/deps/sqlite3/lib

If you ever happen to want to link against installed libraries
in a given directory, LIBDIR, you must either use libtool, and
specify the full pathname of the library, or use the '-LLIBDIR'
flag during linking and do at least one of the following:

• add LIBDIR to the 'LD_LIBRARY_PATH' environment variable
  during execution
• add LIBDIR to the 'LD_RUN_PATH' environment variable
  during linking
• use the '-Wl,-rpath -Wl,LIBDIR' linker flag
• have your system administrator add LIBDIR to '/etc/ld.so.conf'

See any operating system documentation about shared libraries for
more information, such as the ld(1) and ld.so(8) manual pages.

/usr/bin/mkdir -p '/root/GUFI/build/deps/sqlite3/bin'
/bin/sh ./libtool --mode=install /usr/bin/install -c sqlite3 '/root/GUFI/build/deps/sqlite3/bin'
libtool: install: /usr/bin/install -c sqlite3 /root/GUFI/build/deps/sqlite3/bin/sqlite3
/usr/bin/mkdir -p '/root/GUFI/build/deps/sqlite3/include'
/usr/bin/install -c -m 644 ../sqlite3.h ../sqlite3ext.h '/root/GUFI/build/deps/sqlite3/include'
/usr/bin/mkdir -p '/root/GUFI/build/deps/sqlite3/share/man/man1'
/usr/bin/install -c -m 644 ../sqlite3.1 '/root/GUFI/build/deps/sqlite3/share/man/man1'
/usr/bin/mkdir -p '/root/GUFI/build/deps/sqlite3/lib/pkgconfig'
/usr/bin/install -c -m 644 sqlite3.pc '/root/GUFI/build/deps/sqlite3/lib/pkgconfig'
Installing SQLite3 PCRE
-- The C compiler identification is GNU 8.5.0
-- Detecting C compiler ABI info
-- Detecting C compiler ABI info - done
-- Check for working C compiler: /usr/bin/cc - skipped
-- Detecting C compile features
-- Detecting C compile features - done
-- Found PkgConfig: /usr/bin/pkg-config (found version "1.4.2")
-- Checking for one of the modules 'sqlite3'
-- Checking for one of the modules 'libpcre2-8'
-- Configuring done
-- Generating done
-- Build files have been written to: /root/GUFI/build/builds/sqlite3-pcre-pcre2/build
[ 25%] Building C object CMakeFiles/static.dir/pcre.c.o
[ 50%] Linking C static library libsqlite3-pcre.a
[ 50%] Built target static
[ 75%] Building C object CMakeFiles/shared.dir/pcre.c.o
[100%] Linking C shared library libsqlite3-pcre.so
[100%] Built target shared
Consolidate compiler generated dependencies of target static
[ 50%] Built target static
Consolidate compiler generated dependencies of target shared
[100%] Built target shared
Install the project...
-- Install configuration: ""
-- Installing: /root/GUFI/build/deps/sqlite3-pcre/pcre.h
-- Installing: /root/GUFI/build/deps/sqlite3-pcre/libsqlite3-pcre.so
-- Set runtime path of "/root/GUFI/build/deps/sqlite3-pcre/libsqlite3-pcre.so" to ""
-- Installing: /root/GUFI/build/deps/sqlite3-pcre/libsqlite3-pcre.a
Installing jemalloc
autoconf
./configure --enable-autogen
checking for xsltproc... /usr/bin/xsltproc
checking for gcc... gcc
checking whether the C compiler works... yes
checking for C compiler default output file name... a.out
checking for suffix of executables...
checking whether we are cross compiling... no
checking for suffix of object files... o
checking whether we are using the GNU C compiler... yes
checking whether gcc accepts -g... yes
checking for gcc option to accept ISO C89... none needed
checking whether compiler is cray... no
checking whether compiler supports -std=gnu11... yes
checking whether compiler supports -Wall... yes
checking whether compiler supports -Wextra... yes
checking whether compiler supports -Wshorten-64-to-32... no
checking whether compiler supports -Wsign-compare... yes
checking whether compiler supports -Wundef... yes
checking whether compiler supports -Wno-format-zero-length... yes
checking whether compiler supports -pipe... yes
checking whether compiler supports -g3... yes
checking how to run the C preprocessor... gcc -E
checking for g++... g++
checking whether we are using the GNU C++ compiler... yes
checking whether g++ accepts -g... yes
checking whether g++ supports C++14 features by default... yes
checking whether compiler supports -Wall... yes
checking whether compiler supports -Wextra... yes
checking whether compiler supports -g3... yes
checking whether libstdc++ linkage is compilable... yes
checking for grep that handles long lines and -e... /usr/bin/grep
checking for egrep... /usr/bin/grep -E
checking for ANSI C header files... yes
checking for sys/types.h... yes
checking for sys/stat.h... yes
checking for stdlib.h... yes
checking for string.h... yes
checking for memory.h... yes
checking for strings.h... yes
checking for inttypes.h... yes
checking for stdint.h... yes
checking for unistd.h... yes
checking whether byte ordering is bigendian... no
checking size of void ... 8
checking size of int... 4
checking size of long... 8
checking size of long long... 8
checking size of intmax_t... 8
checking build system type... x86_64-pc-linux-gnu
checking host system type... x86_64-pc-linux-gnu
checking whether pause instruction is compilable... yes
checking number of significant virtual address bits... 48
checking for ar... ar
checking for nm... nm
checking for gawk... gawk
Missing VERSION file, and unable to generate it; creating bogus VERSION
checking malloc.h usability... yes
checking malloc.h presence... yes
checking for malloc.h... yes
checking whether malloc_usable_size definition can use const argument... no
checking for library containing log... -lm
checking whether attribute syntax is compilable... yes
checking whether compiler supports -fvisibility=hidden... yes
checking whether compiler supports -fvisibility=hidden... yes
checking whether compiler supports -Werror... yes
checking whether compiler supports -herror_on_warning... no
checking whether tls_model attribute is compilable... yes
checking whether compiler supports -Werror... yes
checking whether compiler supports -herror_on_warning... no
checking whether alloc_size attribute is compilable... yes
checking whether compiler supports -Werror... yes
checking whether compiler supports -herror_on_warning... no
checking whether format(gnu_printf, ...) attribute is compilable... yes
checking whether compiler supports -Werror... yes
checking whether compiler supports -herror_on_warning... no
checking whether format(printf, ...) attribute is compilable... yes
checking whether compiler supports -Werror... yes
checking whether compiler supports -herror_on_warning... no
checking whether format(printf, ...) attribute is compilable... yes
checking for a BSD-compatible install... /usr/bin/install -c
checking for ranlib... ranlib
checking for ld... /usr/bin/ld
checking for autoconf... /usr/bin/autoconf
checking for memalign... yes
checking for valloc... yes
checking for __libc_calloc... yes
checking for __libc_free... yes
checking for __libc_malloc... yes
checking for __libc_memalign... yes
checking for __libc_realloc... yes
checking for __libc_valloc... yes
checking for __posix_memalign... no
checking whether compiler supports -O3... yes
checking whether compiler supports -O3... yes
checking whether compiler supports -funroll-loops... yes
checking configured backtracing method... N/A
checking for sbrk... yes
checking whether utrace(2) is compilable... no
checking whether a program using __builtin_unreachable is compilable... yes
checking whether a program using __builtin_ffsl is compilable... yes
checking whether a program using __builtin_popcountl is compilable... yes
checking LG_PAGE... 12
checking pthread.h usability... yes
checking pthread.h presence... yes
checking for pthread.h... yes
checking for pthread_create in -lpthread... yes
checking dlfcn.h usability... yes
checking dlfcn.h presence... yes
checking for dlfcn.h... yes
checking for dlsym... no
checking for dlsym in -ldl... yes
checking whether pthread_atfork(3) is compilable... yes
checking whether pthread_setname_np(3) is compilable... yes
checking for library containing clock_gettime... none required
checking whether clock_gettime(CLOCK_MONOTONIC_COARSE, ...) is compilable... yes
checking whether clock_gettime(CLOCK_MONOTONIC, ...) is compilable... yes
checking whether mach_absolute_time() is compilable... no
checking whether compiler supports -Werror... yes
checking whether syscall(2) is compilable... yes
checking for secure_getenv... yes
checking for sched_getcpu... yes
checking for sched_setaffinity... yes
checking for issetugid... no
checking for _malloc_thread_cleanup... no
checking for _pthread_mutex_init_calloc_cb... no
checking for TLS... yes
checking whether C11 atomics is compilable... yes
checking whether GCC __atomic atomics is compilable... yes
checking whether GCC 8-bit __atomic atomics is compilable... yes
checking whether GCC __sync atomics is compilable... yes
checking whether GCC 8-bit __sync atomics is compilable... yes
checking whether Darwin OSAtomic() is compilable... no
checking whether madvise(2) is compilable... yes
checking whether madvise(..., MADV_FREE) is compilable... yes
checking whether madvise(..., MADV_DONTNEED) is compilable... yes
checking whether madvise(..., MADV_DO[NT]DUMP) is compilable... yes
checking whether madvise(..., MADV_[NO]HUGEPAGE) is compilable... yes
checking for _builtin_clz... yes
checking whether Darwin os_unfair_lock*() is compilable... no
checking whether glibc malloc hook is compilable... yes
checking whether glibc memalign hook is compilable... yes
checking whether pthreads adaptive mutexes is compilable... yes
checking whether compiler supports -D_GNU_SOURCE... yes
checking whether compiler supports -Werror... yes
checking whether compiler supports -herror_on_warning... no
checking whether strerror_r returns char with gnu source is compilable... yes
checking for stdbool.h that conforms to C99... yes
checking for _Bool... yes
configure: creating ./config.status
config.status: creating Makefile
config.status: creating jemalloc.pc
config.status: creating doc/html.xsl
config.status: creating doc/manpages.xsl
config.status: creating doc/jemalloc.xml
config.status: creating include/jemalloc/jemalloc_macros.h
config.status: creating include/jemalloc/jemalloc_protos.h
config.status: creating include/jemalloc/jemalloc_typedefs.h
config.status: creating include/jemalloc/internal/jemalloc_preamble.h
config.status: creating test/test.sh
config.status: creating test/include/test/jemalloc_test.h
config.status: creating config.stamp
config.status: creating bin/jemalloc-config
config.status: creating bin/jemalloc.sh
config.status: creating bin/jeprof
config.status: creating include/jemalloc/jemalloc_defs.h
config.status: creating include/jemalloc/internal/jemalloc_internal_defs.h
config.status: creating test/include/test/jemalloc_test_defs.h
config.status: executing include/jemalloc/internal/public_symbols.txt commands
config.status: executing include/jemalloc/internal/private_symbols.awk commands
config.status: executing include/jemalloc/internal/private_symbols_jet.awk commands
config.status: executing include/jemalloc/internal/public_namespace.h commands
config.status: executing include/jemalloc/internal/public_unnamespace.h commands
config.status: executing include/jemalloc/jemalloc_protos_jet.h commands
config.status: executing include/jemalloc/jemalloc_rename.h commands
config.status: executing include/jemalloc/jemalloc_mangle.h commands
config.status: executing include/jemalloc/jemalloc_mangle_jet.h commands
config.status: executing include/jemalloc/jemalloc.h commands

jemalloc version : 0.0.0-0-g0000000000000000000000000000000000000000
library revision : 2

CONFIG : --enable-autogen
CC : gcc
CONFIGURE_CFLAGS : -std=gnu11 -Wall -Wextra -Wsign-compare -Wundef -Wno-format-zero-length -pipe -g3 -fvisibility=hidden -O3 -funroll-loops
SPECIFIED_CFLAGS :
EXTRA_CFLAGS :
CPPFLAGS : -D_GNU_SOURCE -D_REENTRANT
CXX : g++
CONFIGURE_CXXFLAGS : -Wall -Wextra -g3 -fvisibility=hidden -O3
SPECIFIED_CXXFLAGS :
EXTRA_CXXFLAGS :
LDFLAGS :
EXTRA_LDFLAGS :
DSO_LDFLAGS : -shared -Wl,-soname,$(@f)
LIBS : -lm -lstdc++ -pthread -ldl
RPATH_EXTRA :

XSLTPROC : /usr/bin/xsltproc
XSLROOT :

PREFIX : /usr/local
BINDIR : /usr/local/bin
DATADIR : /usr/local/share
INCLUDEDIR : /usr/local/include
LIBDIR : /usr/local/lib
MANDIR : /usr/local/share/man

srcroot :
abs_srcroot : /root/GUFI/build/builds/jemalloc-master/
objroot :
abs_objroot : /root/GUFI/build/builds/jemalloc-master/

JEMALLOC_PREFIX :
JEMALLOC_PRIVATE_NAMESPACE
: je_
install_suffix :
malloc_conf :
documentation : 1
shared libs : 1
static libs : 1
autogen : 1
debug : 0
stats : 1
experimetal_smallocx : 0
prof : 0
prof-libunwind : 0
prof-libgcc : 0
prof-gcc : 0
fill : 1
utrace : 0
xmalloc : 0
log : 0
lazy_lock : 0
cache-oblivious : 1
cxx : 1

checking for xsltproc... /usr/bin/xsltproc
checking for gcc... gcc
checking whether the C compiler works... yes
checking for C compiler default output file name... a.out
checking for suffix of executables...
checking whether we are cross compiling... no
checking for suffix of object files... o
checking whether we are using the GNU C compiler... yes
checking whether gcc accepts -g... yes
checking for gcc option to accept ISO C89... none needed
checking whether compiler is cray... no
checking whether compiler supports -std=gnu11... yes
checking whether compiler supports -Wall... yes
checking whether compiler supports -Wextra... yes
checking whether compiler supports -Wshorten-64-to-32... no
checking whether compiler supports -Wsign-compare... yes
checking whether compiler supports -Wundef... yes
checking whether compiler supports -Wno-format-zero-length... yes
checking whether compiler supports -pipe... yes
checking whether compiler supports -g3... yes
checking how to run the C preprocessor... gcc -E
checking for g++... g++
checking whether we are using the GNU C++ compiler... yes
checking whether g++ accepts -g... yes
checking whether g++ supports C++14 features by default... yes
checking whether compiler supports -Wall... yes
checking whether compiler supports -Wextra... yes
checking whether compiler supports -g3... yes
checking whether libstdc++ linkage is compilable... yes
checking for grep that handles long lines and -e... /usr/bin/grep
checking for egrep... /usr/bin/grep -E
checking for ANSI C header files... yes
checking for sys/types.h... yes
checking for sys/stat.h... yes
checking for stdlib.h... yes
checking for string.h... yes
checking for memory.h... yes
checking for strings.h... yes
checking for inttypes.h... yes
checking for stdint.h... yes
checking for unistd.h... yes
checking whether byte ordering is bigendian... no
checking size of void ... 8
checking size of int... 4
checking size of long... 8
checking size of long long... 8
checking size of intmax_t... 8
checking build system type... x86_64-pc-linux-gnu
checking host system type... x86_64-pc-linux-gnu
checking whether pause instruction is compilable... yes
checking number of significant virtual address bits... 48
checking for ar... ar
checking for nm... nm
checking for gawk... gawk
checking malloc.h usability... yes
checking malloc.h presence... yes
checking for malloc.h... yes
checking whether malloc_usable_size definition can use const argument... no
checking for library containing log... -lm
checking whether attribute syntax is compilable... yes
checking whether compiler supports -fvisibility=hidden... yes
checking whether compiler supports -fvisibility=hidden... yes
checking whether compiler supports -Werror... yes
checking whether compiler supports -herror_on_warning... no
checking whether tls_model attribute is compilable... yes
checking whether compiler supports -Werror... yes
checking whether compiler supports -herror_on_warning... no
checking whether alloc_size attribute is compilable... yes
checking whether compiler supports -Werror... yes
checking whether compiler supports -herror_on_warning... no
checking whether format(gnu_printf, ...) attribute is compilable... yes
checking whether compiler supports -Werror... yes
checking whether compiler supports -herror_on_warning... no
checking whether format(printf, ...) attribute is compilable... yes
checking whether compiler supports -Werror... yes
checking whether compiler supports -herror_on_warning... no
checking whether format(printf, ...) attribute is compilable... yes
checking for a BSD-compatible install... /usr/bin/install -c
checking for ranlib... ranlib
checking for ld... /usr/bin/ld
checking for autoconf... /usr/bin/autoconf
checking for memalign... yes
checking for valloc... yes
checking for __libc_calloc... yes
checking for __libc_free... yes
checking for __libc_malloc... yes
checking for __libc_memalign... yes
checking for __libc_realloc... yes
checking for __libc_valloc... yes
checking for __posix_memalign... no
checking whether compiler supports -O3... yes
checking whether compiler supports -O3... yes
checking whether compiler supports -funroll-loops... yes
checking configured backtracing method... N/A
checking for sbrk... yes
checking whether utrace(2) is compilable... no
checking whether a program using __builtin_unreachable is compilable... yes
checking whether a program using __builtin_ffsl is compilable... yes
checking whether a program using __builtin_popcountl is compilable... yes
checking LG_PAGE... 12
checking pthread.h usability... yes
checking pthread.h presence... yes
checking for pthread.h... yes
checking for pthread_create in -lpthread... yes
checking dlfcn.h usability... yes
checking dlfcn.h presence... yes
checking for dlfcn.h... yes
checking for dlsym... no
checking for dlsym in -ldl... yes
checking whether pthread_atfork(3) is compilable... yes
checking whether pthread_setname_np(3) is compilable... yes
checking for library containing clock_gettime... none required
checking whether clock_gettime(CLOCK_MONOTONIC_COARSE, ...) is compilable... yes
checking whether clock_gettime(CLOCK_MONOTONIC, ...) is compilable... yes
checking whether mach_absolute_time() is compilable... no
checking whether compiler supports -Werror... yes
checking whether syscall(2) is compilable... yes
checking for secure_getenv... yes
checking for sched_getcpu... yes
checking for sched_setaffinity... yes
checking for issetugid... no
checking for _malloc_thread_cleanup... no
checking for _pthread_mutex_init_calloc_cb... no
checking for TLS... yes
checking whether C11 atomics is compilable... yes
checking whether GCC __atomic atomics is compilable... yes
checking whether GCC 8-bit __atomic atomics is compilable... yes
checking whether GCC __sync atomics is compilable... yes
checking whether GCC 8-bit __sync atomics is compilable... yes
checking whether Darwin OSAtomic() is compilable... no
checking whether madvise(2) is compilable... yes
checking whether madvise(..., MADV_FREE) is compilable... yes
checking whether madvise(..., MADV_DONTNEED) is compilable... yes
checking whether madvise(..., MADV_DO[NT]DUMP) is compilable... yes
checking whether madvise(..., MADV_[NO]HUGEPAGE) is compilable... yes
checking for _builtin_clz... yes
checking whether Darwin os_unfair_lock*() is compilable... no
checking whether glibc malloc hook is compilable... yes
checking whether glibc memalign hook is compilable... yes
checking whether pthreads adaptive mutexes is compilable... yes
checking whether compiler supports -D_GNU_SOURCE... yes
checking whether compiler supports -Werror... yes
checking whether compiler supports -herror_on_warning... no
checking whether strerror_r returns char with gnu source is compilable... yes
checking for stdbool.h that conforms to C99... yes
checking for _Bool... yes
configure: creating ./config.status
config.status: creating Makefile
config.status: creating jemalloc.pc
config.status: creating doc/html.xsl
config.status: creating doc/manpages.xsl
config.status: creating doc/jemalloc.xml
config.status: creating include/jemalloc/jemalloc_macros.h
config.status: creating include/jemalloc/jemalloc_protos.h
config.status: creating include/jemalloc/jemalloc_typedefs.h
config.status: creating include/jemalloc/internal/jemalloc_preamble.h
config.status: creating test/test.sh
config.status: creating test/include/test/jemalloc_test.h
config.status: creating config.stamp
config.status: creating bin/jemalloc-config
config.status: creating bin/jemalloc.sh
config.status: creating bin/jeprof
config.status: creating include/jemalloc/jemalloc_defs.h
config.status: creating include/jemalloc/internal/jemalloc_internal_defs.h
config.status: creating test/include/test/jemalloc_test_defs.h
config.status: executing include/jemalloc/internal/public_symbols.txt commands
config.status: executing include/jemalloc/internal/private_symbols.awk commands
config.status: executing include/jemalloc/internal/private_symbols_jet.awk commands
config.status: executing include/jemalloc/internal/public_namespace.h commands
config.status: executing include/jemalloc/internal/public_unnamespace.h commands
config.status: executing include/jemalloc/jemalloc_protos_jet.h commands
config.status: executing include/jemalloc/jemalloc_rename.h commands
config.status: executing include/jemalloc/jemalloc_mangle.h commands
config.status: executing include/jemalloc/jemalloc_mangle_jet.h commands
config.status: executing include/jemalloc/jemalloc.h commands

jemalloc version : 0.0.0-0-g0000000000000000000000000000000000000000
library revision : 2

CONFIG : --prefix=/root/GUFI/build/deps/jemalloc
CC : gcc
CONFIGURE_CFLAGS : -std=gnu11 -Wall -Wextra -Wsign-compare -Wundef -Wno-format-zero-length -pipe -g3 -fvisibility=hidden -O3 -funroll-loops
SPECIFIED_CFLAGS :
EXTRA_CFLAGS :
CPPFLAGS : -D_GNU_SOURCE -D_REENTRANT
CXX : g++
CONFIGURE_CXXFLAGS : -Wall -Wextra -g3 -fvisibility=hidden -O3
SPECIFIED_CXXFLAGS :
EXTRA_CXXFLAGS :
LDFLAGS :
EXTRA_LDFLAGS :
DSO_LDFLAGS : -shared -Wl,-soname,$(@f)
LIBS : -lm -lstdc++ -pthread -ldl
RPATH_EXTRA :

XSLTPROC : /usr/bin/xsltproc
XSLROOT :

PREFIX : /root/GUFI/build/deps/jemalloc
BINDIR : /root/GUFI/build/deps/jemalloc/bin
DATADIR : /root/GUFI/build/deps/jemalloc/share
INCLUDEDIR : /root/GUFI/build/deps/jemalloc/include
LIBDIR : /root/GUFI/build/deps/jemalloc/lib
MANDIR : /root/GUFI/build/deps/jemalloc/share/man

srcroot : ../
abs_srcroot : /root/GUFI/build/builds/jemalloc-master/
objroot :
abs_objroot : /root/GUFI/build/builds/jemalloc-master/build/

JEMALLOC_PREFIX :
JEMALLOC_PRIVATE_NAMESPACE
: je_
install_suffix :
malloc_conf :
documentation : 1
shared libs : 1
static libs : 1
autogen : 0
debug : 0
stats : 1
experimetal_smallocx : 0
prof : 0
prof-libunwind : 0
prof-libgcc : 0
prof-gcc : 0
fill : 1
utrace : 0
xmalloc : 0
log : 0
lazy_lock : 0
cache-oblivious : 1
cxx : 1

cp include/jemalloc/internal/private_namespace.gen.h include/jemalloc/internal/private_namespace.gen.h
/usr/bin/install -c -m 755 bin/jemalloc-config /root/GUFI/build/deps/jemalloc/bin
/usr/bin/install -c -m 755 bin/jemalloc.sh /root/GUFI/build/deps/jemalloc/bin
/usr/bin/install -c -m 755 bin/jeprof /root/GUFI/build/deps/jemalloc/bin
/usr/bin/install -c -m 644 include/jemalloc/jemalloc.h /root/GUFI/build/deps/jemalloc/include/jemalloc
/usr/bin/install -c -m 755 lib/libjemalloc.a /root/GUFI/build/deps/jemalloc/lib
/usr/bin/install -c -m 755 lib/libjemalloc_pic.a /root/GUFI/build/deps/jemalloc/lib
/usr/bin/install -c -m 644 jemalloc.pc /root/GUFI/build/deps/jemalloc/lib/pkgconfig
Missing xsltproc. doc/jemalloc.html not (re)built.
Missing xsltproc. doc/jemalloc.3 not (re)built.
/usr/bin/install -c -m 644 doc/jemalloc.html /root/GUFI/build/deps/jemalloc/share/doc/jemalloc
/usr/bin/install -c -m 644 doc/jemalloc.3 /root/GUFI/build/deps/jemalloc/share/man/man3
Installing GoogleTest
-- The C compiler identification is GNU 8.5.0
-- The CXX compiler identification is GNU 8.5.0
-- Detecting C compiler ABI info
-- Detecting C compiler ABI info - done
-- Check for working C compiler: /usr/bin/cc - skipped
-- Detecting C compile features
-- Detecting C compile features - done
-- Detecting CXX compiler ABI info
-- Detecting CXX compiler ABI info - done
-- Check for working CXX compiler: /usr/bin/c++ - skipped
-- Detecting CXX compile features
-- Detecting CXX compile features - done
-- Found Python: /usr/bin/python3.6 (found version "3.6.8") found components: Interpreter
-- Looking for pthread.h
-- Looking for pthread.h - found
-- Performing Test CMAKE_HAVE_LIBC_PTHREAD
-- Performing Test CMAKE_HAVE_LIBC_PTHREAD - Failed
-- Looking for pthread_create in pthreads
-- Looking for pthread_create in pthreads - not found
-- Looking for pthread_create in pthread
-- Looking for pthread_create in pthread - found
-- Found Threads: TRUE
-- Configuring done
-- Generating done
-- Build files have been written to: /root/GUFI/build/builds/googletest-main/build
[ 25%] Building CXX object googletest/CMakeFiles/gtest.dir/src/gtest-all.cc.o
[ 50%] Linking CXX static library ../lib/libgtest.a
[ 50%] Built target gtest
[ 75%] Building CXX object googletest/CMakeFiles/gtest_main.dir/src/gtest_main.cc.o
[100%] Linking CXX static library ../lib/libgtest_main.a
[100%] Built target gtest_main
Consolidate compiler generated dependencies of target gtest
[ 50%] Built target gtest
Consolidate compiler generated dependencies of target gtest_main
[100%] Built target gtest_main
Install the project...
-- Install configuration: ""
-- Installing: /root/GUFI/build/deps/googletest/lib64/cmake/GTest/GTestTargets.cmake
-- Installing: /root/GUFI/build/deps/googletest/lib64/cmake/GTest/GTestTargets-noconfig.cmake
-- Installing: /root/GUFI/build/deps/googletest/lib64/cmake/GTest/GTestConfigVersion.cmake
-- Installing: /root/GUFI/build/deps/googletest/lib64/cmake/GTest/GTestConfig.cmake
-- Installing: /root/GUFI/build/deps/googletest/include
-- Installing: /root/GUFI/build/deps/googletest/include/gtest
-- Installing: /root/GUFI/build/deps/googletest/include/gtest/gtest-assertion-result.h
-- Installing: /root/GUFI/build/deps/googletest/include/gtest/gtest-death-test.h
-- Installing: /root/GUFI/build/deps/googletest/include/gtest/gtest-matchers.h
-- Installing: /root/GUFI/build/deps/googletest/include/gtest/gtest-message.h
-- Installing: /root/GUFI/build/deps/googletest/include/gtest/gtest-param-test.h
-- Installing: /root/GUFI/build/deps/googletest/include/gtest/gtest-printers.h
-- Installing: /root/GUFI/build/deps/googletest/include/gtest/gtest-spi.h
-- Installing: /root/GUFI/build/deps/googletest/include/gtest/gtest-test-part.h
-- Installing: /root/GUFI/build/deps/googletest/include/gtest/gtest-typed-test.h
-- Installing: /root/GUFI/build/deps/googletest/include/gtest/gtest.h
-- Installing: /root/GUFI/build/deps/googletest/include/gtest/gtest_pred_impl.h
-- Installing: /root/GUFI/build/deps/googletest/include/gtest/gtest_prod.h
-- Installing: /root/GUFI/build/deps/googletest/include/gtest/internal
-- Installing: /root/GUFI/build/deps/googletest/include/gtest/internal/custom
-- Installing: /root/GUFI/build/deps/googletest/include/gtest/internal/custom/README.md
-- Installing: /root/GUFI/build/deps/googletest/include/gtest/internal/custom/gtest-port.h
-- Installing: /root/GUFI/build/deps/googletest/include/gtest/internal/custom/gtest-printers.h
-- Installing: /root/GUFI/build/deps/googletest/include/gtest/internal/custom/gtest.h
-- Installing: /root/GUFI/build/deps/googletest/include/gtest/internal/gtest-death-test-internal.h
-- Installing: /root/GUFI/build/deps/googletest/include/gtest/internal/gtest-filepath.h
-- Installing: /root/GUFI/build/deps/googletest/include/gtest/internal/gtest-internal.h
-- Installing: /root/GUFI/build/deps/googletest/include/gtest/internal/gtest-param-util.h
-- Installing: /root/GUFI/build/deps/googletest/include/gtest/internal/gtest-port-arch.h
-- Installing: /root/GUFI/build/deps/googletest/include/gtest/internal/gtest-port.h
-- Installing: /root/GUFI/build/deps/googletest/include/gtest/internal/gtest-string.h
-- Installing: /root/GUFI/build/deps/googletest/include/gtest/internal/gtest-type-util.h
-- Installing: /root/GUFI/build/deps/googletest/lib64/libgtest.a
-- Installing: /root/GUFI/build/deps/googletest/lib64/libgtest_main.a
-- Installing: /root/GUFI/build/deps/googletest/lib64/pkgconfig/gtest.pc
-- Installing: /root/GUFI/build/deps/googletest/lib64/pkgconfig/gtest_main.pc
[ 0%] Built target install_dependencies
[ 1%] Building C object src/CMakeFiles/tsmepoch2time.dir/tsmepoch2time.c.o
[ 2%] Linking C executable tsmepoch2time
[ 2%] Built target tsmepoch2time
[ 3%] Building C object src/CMakeFiles/GUFI.dir/BottomUp.c.o
[ 4%] Building C object src/CMakeFiles/GUFI.dir/OutputBuffers.c.o
[ 5%] Building C object src/CMakeFiles/GUFI.dir/QueuePerThreadPool.c.o
[ 6%] Building C object src/CMakeFiles/GUFI.dir/SinglyLinkedList.c.o
[ 7%] Building C object src/CMakeFiles/GUFI.dir/bf.c.o
[ 8%] Building C object src/CMakeFiles/GUFI.dir/compress.c.o
[ 9%] Building C object src/CMakeFiles/GUFI.dir/dbutils.c.o
[ 10%] Building C object src/CMakeFiles/GUFI.dir/debug.c.o
[ 11%] Building C object src/CMakeFiles/GUFI.dir/external.c.o
[ 12%] Building C object src/CMakeFiles/GUFI.dir/histogram.c.o
[ 13%] Building C object src/CMakeFiles/GUFI.dir/print.c.o
[ 15%] Building C object src/CMakeFiles/GUFI.dir/template_db.c.o
[ 16%] Building C object src/CMakeFiles/GUFI.dir/trace.c.o
[ 17%] Building C object src/CMakeFiles/GUFI.dir/trie.c.o
[ 18%] Building C object src/CMakeFiles/GUFI.dir/utils.c.o
[ 19%] Building C object src/CMakeFiles/GUFI.dir/xattrs.c.o
[ 20%] Linking C static library libGUFI.a
[ 20%] Built target GUFI
[ 21%] Building C object src/CMakeFiles/gufi_query_lib.dir/gufi_query/PoolArgs.c.o
[ 22%] Building C object src/CMakeFiles/gufi_query_lib.dir/gufi_query/aggregate.c.o
[ 23%] Building C object src/CMakeFiles/gufi_query_lib.dir/gufi_query/external.c.o
[ 24%] Building C object src/CMakeFiles/gufi_query_lib.dir/gufi_query/processdir.c.o
[ 25%] Building C object src/CMakeFiles/gufi_query_lib.dir/gufi_query/process_queries.c.o
[ 26%] Building C object src/CMakeFiles/gufi_query_lib.dir/gufi_query/query.c.o
[ 27%] Building C object src/CMakeFiles/gufi_query_lib.dir/gufi_query/timers.c.o
[ 29%] Building C object src/CMakeFiles/gufi_query_lib.dir/gufi_query/validate_inputs.c.o
[ 30%] Building C object src/CMakeFiles/gufi_query_lib.dir/gufi_query/xattrs.c.o
[ 30%] Built target gufi_query_lib
[ 31%] Building C object src/CMakeFiles/gufi_query.dir/gufi_query/main.c.o
[ 32%] Linking C executable gufi_query
[ 32%] Built target gufi_query
[ 33%] Building C object src/CMakeFiles/gufi_dir2index.dir/gufi_dir2index.c.o
[ 34%] Linking C executable gufi_dir2index
[ 34%] Built target gufi_dir2index
[ 35%] Building C object src/CMakeFiles/gufi_treesummary_all.dir/gufi_treesummary_all.c.o
[ 36%] Linking C executable gufi_treesummary_all
[ 36%] Built target gufi_treesummary_all
[ 37%] Building C object src/CMakeFiles/parallel_rmr.dir/parallel_rmr.c.o
[ 38%] Linking C executable parallel_rmr
[ 38%] Built target parallel_rmr
[ 39%] Building C object src/CMakeFiles/dfw.dir/dfw.c.o
[ 40%] Linking C executable dfw
[ 40%] Built target dfw
[ 41%] Building C object src/CMakeFiles/tsmtime2epoch.dir/tsmtime2epoch.c.o
[ 43%] Linking C executable tsmtime2epoch
[ 43%] Built target tsmtime2epoch
[ 44%] Building C object src/CMakeFiles/gufi_unrollup.dir/gufi_unrollup.c.o
[ 45%] Linking C executable gufi_unrollup
[ 45%] Built target gufi_unrollup
[ 46%] Building C object src/CMakeFiles/gufi_stat_bin.dir/gufi_stat.c.o
[ 47%] Linking C executable gufi_stat_bin
[ 47%] Built target gufi_stat_bin
[ 48%] Building C object src/CMakeFiles/gpfs-scan-tool.dir/gpfs-scan-tool.c.o
/root/GUFI/src/gpfs-scan-tool.c: In function ‘proc_inodes’:
/root/GUFI/src/gpfs-scan-tool.c:182:9: error: ‘in’ undeclared (first use in this function); did you mean ‘int’?
delim=in.delim;
^~
int
/root/GUFI/src/gpfs-scan-tool.c:182:9: note: each undeclared identifier is reported only once for each function it appears in
/root/GUFI/src/gpfs-scan-tool.c: In function ‘validate_inputs’:
/root/GUFI/src/gpfs-scan-tool.c:317:57: error: ‘in’ undeclared (first use in this function); did you mean ‘int’?
printf("threadnum: %d; stride: %d; delimiter: %s;\n", in.maxthreads, in.stride, in.delim);
^~
int
/root/GUFI/src/gpfs-scan-tool.c: In function ‘main’:
/root/GUFI/src/gpfs-scan-tool.c:346:12: warning: initialization of ‘int’ from ‘void *’ makes integer from pointer without a cast [-Wint-conversion]
int rc = NULL;
^~~~
/root/GUFI/src/gpfs-scan-tool.c:360:69: error: ‘in’ undeclared (first use in this function); did you mean ‘i’?
int idx = parse_cmd_line(argc, argv, "hHn:d:g:", 1, "input_dir", &in);
^~
i
/root/GUFI/src/gpfs-scan-tool.c:378:5: error: too many arguments to function ‘INSTALL_STR’
INSTALL_STR(in.name, argv[idx++], MAXPATH, "input_dir");
^~~~~~~~~~~
In file included from /root/GUFI/src/gpfs-scan-tool.c:75:
/root/GUFI/include/bf.h:265:5: note: declared here
int INSTALL_STR(refstr_t *VAR, const char *SOURCE);
^~~~~~~~~~~
make[2]: *** [src/CMakeFiles/gpfs-scan-tool.dir/build.make:76: src/CMakeFiles/gpfs-scan-tool.dir/gpfs-scan-tool.c.o] Error 1
make[1]: *** [CMakeFiles/Makefile2:615: src/CMakeFiles/gpfs-scan-tool.dir/all] Error 2
make: *** [Makefile:146: all] Error 2

My gcc
[root@wmc040 build]# gcc -v
Using built-in specs.
COLLECT_GCC=gcc
COLLECT_LTO_WRAPPER=/usr/libexec/gcc/x86_64-redhat-linux/8/lto-wrapper
OFFLOAD_TARGET_NAMES=nvptx-none
OFFLOAD_TARGET_DEFAULT=1
Target: x86_64-redhat-linux
Configured with: ../configure --enable-bootstrap --enable-languages=c,c++,fortran,lto --prefix=/usr --mandir=/usr/share/man --infodir=/usr/share/info --with-bugurl=https://bugs.rockylinux.org/ --enable-shared --enable-threads=posix --enable-checking=release --enable-multilib --with-system-zlib --enable-__cxa_atexit --disable-libunwind-exceptions --enable-gnu-unique-object --enable-linker-build-id --with-gcc-major-version-only --with-linker-hash-style=gnu --enable-plugin --enable-initfini-array --with-isl --disable-libmpx --enable-offload-targets=nvptx-none --without-cuda-driver --enable-gnu-indirect-function --enable-cet --with-tune=generic --with-arch_32=x86-64 --build=x86_64-redhat-linux
Thread model: posix
gcc version 8.5.0 20210514 (Red Hat 8.5.0-18) (GCC)

[root@wmc040 build]# make -v
GNU Make 4.2.1
Built for x86_64-redhat-linux-gnu
Copyright (C) 1988-2016 Free Software Foundation, Inc.
License GPLv3+: GNU GPL version 3 or later http://gnu.org/licenses/gpl.html
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law.
[root@wmc040 build]# cat /etc/redhat-release
Rocky Linux release 8.5 (Green Obsidian)

Cheers
Dafei

find uses the fprintf flag for the output to file functionality. gnu find uses -o as the OR flag.

gufi_dir2trace can race on stdout when no per-thread-output prefix is provided. worktofile needs to be changed to buffer in OutputBuffers before writing to the actual file. processdir needs to be changed to atomically write stanzas.

Feature request: When indexing systems that have publicly viewable .snapshot or similar paths it complicates queries or causes confusion with multiple counting files etc.

An option would be to just delete .snapshot after the scan but that still means that GUFI indexed that entire path.

Thanks!

As of now we aren't handling type d at all. We fix using the summary table for each directory and be sure to use this same table for the --select and other flags following type d.

We've modified gufi_find quite a bit and gufi_ls has fallen behind some. Formatting definitely needs to be reworked to feel more like ls.

WIth the current build gufi_find looks for a config attribute DEFAULT_CONFIG_PATH which doesn't exist, it looks like this should be DEFAULT_PATH but not sure, but it breaks gufi_find while gufi_ls works just fine.

We need to document the config and provide a sample when installing via make or rpm. This should also create any assumed directory structure for the config file.

Similar to gufi_ls (#8), gufi_stats has also fallen behind.

I'm using "gufi_dir2index" to create index directory. I've done some file operations on my input directory's (i.e file system) sub-directory.
Now I want to update my existing index directory.

Is there any way to update existing index directory, please suggests.

Thank you!

When using the current repo building fails with:

/tmp/GUFI/contrib/deps/googletest.sh: line 21: cd: /tmp/GUFI/build/builds/googletest-main: No such file or directory
make[2]: *** [CMakeFiles/install_dependencies.dir/build.make:57: CMakeFiles/install_dependencies] Error 1
make[1]: *** [CMakeFiles/Makefile2:375: CMakeFiles/install_dependencies.dir/all] Error 2
make: *** [Makefile:141: all] Error 2

A quick fix for this is to delete the contrib/deps/googletest.tar.gz the google test script then downloads from the web and works just fine with a follow up make.

hi,

when I want to use GUFI, I always want to have the newest metadata lists. Do I need to execute gufi_dir2index whenever I want to check the newest metadata? Or there have some methods can update the metadata in a specific period without executing command manually？

We're having issues with folders with spaces in names:

eg

Project_RNA-Seq 5-AZA_2

Will create error messages like:

input-dir '/tmp/GUFI/umms-cspeers//Project_RNA-Seq' is not a directory
input-dir '/tmp/GUFI/umms-cspeers//5-AZA_2' is not a directory

The query being run is located: https://github.com/umich-arc/gufi-archive/blob/master/reports/dirsum.sh

I can confirm that the index does have the path correctly. The error appears to be in the query but simpler queries like
https://github.com/umich-arc/gufi-archive/blob/master/reports/totals.sh

Using GUFI Index located in: /tmp/GUFI/umms-cspeers/
Reporting on data older than 180 days last accessed

count                     sizeGB  oldsize  percent  
137661                    20173   5992     29%   

Work just fine.

The new commit which adds gufi_stats -c filesize-log2-bins is awesome. Thanks so much for adding that!

Would it be possible to add a similar gufi_stats -c dirsize_log2_bins please? Or let me know if you think that's something I might be able to do. Might be a good learning exercise for me. Be nice to get my name into the git commit history for GUFI also!

When gufi_query is called with -O (per thread database output files) or -e 0 (aggregate), the database files are opened through ATTACH, not sqlite3_open_v2 and can be modified with queries, even though they should have been opened in READONLY mode.

$ src/gufi_query -O out -E "create table tree.ABC (name TEXT)" /search/
$ sqlite3 /search/db.db ".tables"
ABC