homog_mem.py debug flag and max threading uxm build about uxm_deconv HOT 2 OPEN

Thank you for the helpful suggestions.
I fixed the debug flag problem (moved it from deconv.py to homog_mem.py).
About the threads issue - I couldn't reproduce it. It works for me with 100 threads. Did you get any error message/s?

from uxm_deconv.

Regarding the treading issue, I got the following message ([...] is a private path) while generating the memoization files:

[ wt homog ] [ GSM5652203_Heart-Fibroblasts-Z0000041X ] WARNING: all zeros!
WARNING: possibly failed in [...]/pats/pats_hg19/GSM5652203_Heart-Fibroblasts-Z0000041X.pat.gz - all 978 values are zero. memoization is not updated, to be safe

I got more of the same errors for other GSM_* files.

At the end I got:

multiprocessing.pool.RemoteTraceback:
"""
Traceback (most recent call last):
File "[...]/.conda/pkgs/UMX/lib/python3.10/multiprocessing/pool.py", line 125, in worker
result = (True, func(*args, **kwds))
File "[...]/.conda/pkgs/UMX/lib/python3.10/multiprocessing/pool.py", line 51, in starmapstar
return list(itertools.starmap(args[0], args[1]))
File "[...]/github_repos/UXM_deconv/src/homog_mem.py", line 227, in pat2homog
tmp_homog_path = wrap_cpp_tool(pat, remain_mrk, tmp_dir_l, rlen, verb, debug)
File "[...]/github_repos/UXM_deconv/src/homog_mem.py", line 167, in wrap_cpp_tool
subprocess.check_call(cmd, shell=True, stderr=so, stdout=so)
File "[...]/.conda/pkgs/UMX/lib/python3.10/subprocess.py", line 369, in check_call
raise CalledProcessError(retcode, cmd)
subprocess.CalledProcessError: Command 'wgbstools homog -f --rlen 4 -b [...]/UXM_deconv/tmp_dir/l4/GSM5652342_Bladder-Epithelial-Z000000QM.8deaqbh3.bed [...]/pats/pats_hg19/GSM5652342_Bladder-Epithelial-Z000000QM.pat.gz --prefix [...]/UXM_deconv/tmp_dir/l4/GSM5652342_Bladder-Epithelial-Z000000QM.8deaqbh3' died with <Signals.SIGSEGV: 11>.
"""

dmesg returns:

[336324.876686] show_signal_msg: 29 callbacks suppressed
[336324.876693] python3[108652]: segfault at 7fb51195d9d0 ip 00007fb535801f81 sp 00007ffd36f18990 error 4 in libpthread-2.17.so[7fb5357f9000+17000]
[336325.066166] python3[108646]: segfault at 7f452c8af9d0 ip 00007f4598753f81 sp 00007fffee689670 error 4 in libpthread-2.17.so[7f459874b000+17000]
[336328.202528] python3[112005]: segfault at 7fa0208a39d0 ip 00007fa03df5af81 sp 00007ffd55ae62e0 error 4 in libpthread-2.17.so[7fa03df52000+17000]
[336328.840650] python3[112353]: segfault at 7ff534b0f9d0 ip 00007ff5c29b7f81 sp 00007ffd753483a0 error 4 in libpthread-2.17.so[7ff5c29af000+17000]
[336329.939377] python3[112656]: segfault at 7f0fd9faa9d0 ip 00007f102464bf81 sp 00007ffcf6085110 error 4 in libpthread-2.17.so[7f1024643000+17000]
[336330.271708] python3[112700]: segfault at 7f74dba649d0 ip 00007f751e105f81 sp 00007ffe5cf92680 error 4 in libpthread-2.17.so[7f751e0fd000+17000]
[336330.624133] python3[112761]: segfault at 7f589746d9d0 ip 00007f58d9b0ef81 sp 00007ffea2845b30 error 4 in libpthread-2.17.so[7f58d9b06000+17000]
[336330.881537] python3[112823]: segfault at 7ff3eaec79d0 ip 00007ff415568f81 sp 00007ffe95303a60 error 4 in libpthread-2.17.so[7ff415560000+17000]
[336331.241338] python3[112883]: segfault at 7f3dde4329d0 ip 00007f3e40ad3f81 sp 00007fff580af5e0 error 4 in libpthread-2.17.so[7f3e40acb000+17000]
[336331.521982] python3[112941]: segfault at 7fd61d43d9d0 ip 00007fd647adef81 sp 00007ffe1d81b540 error 4 in libpthread-2.17.so[7fd647ad6000+17000]
[336331.633733] python3[113002]: segfault at 7fab158609d0 ip 00007fab6ff01f81 sp 00007ffca11ab750 error 4 in libpthread-2.17.so[7fab6fef9000+17000]
[336331.748252] python3[113021]: segfault at 7fccf7dde9d0 ip 00007fcd68c7cf81 sp 00007ffcce685830 error 4 in libpthread-2.17.so[7fcd68c74000+17000]
[336331.873550] python3[113040]: segfault at 7fcb38f339d0 ip 00007fcb91dd1f81 sp 00007ffc556accc0 error 4 in libpthread-2.17.so[7fcb91dc9000+17000]
[336332.010403] python3[113056]: segfault at 7f2a567a69d0 ip 00007f2ab7644f81 sp 00007ffe062450e0 error 4 in libpthread-2.17.so[7f2ab763c000+17000]

Here I used 50 CPUs and memory should not be an issue with 500Gb. However, it seems like there might be some issues specific to me and not necessarily a bug in your code.

from uxm_deconv.