Pre-populate and ship a Jitify cache? about cupy HOT 4 OPEN

IIRC the problem with pre-generated cache is that some headers depend on files under /usr/local/cuda/include which is user environment dependent. 😢

from cupy.

I am thinking about this too. I think the bigger problem is that the cache keys can contain includes with relative paths (or truncated paths in the bracket form <...> that requires the compiler to search), which makes it very hard to identify the source of the headers. If we normalize all keys to absolute path, maybe it'd make things easier/safer? For example, we can exclude CUDA headers just by checking their path prefix. However, we'll pay a price during cache lookup (de-normalizing an absolute path to relative/truncated path).

from cupy.

If we normalize all keys to absolute path, maybe it'd make things easier/safer?

I think there's no formal way to do that... If A/B/C.h has a #include <D.h> line, we only get D.h as a key, so we don't have knowledge of which file tried to include D.h nor which directory D.h is in.

from cupy.

Right, so when I noted

we might have to either implement it ourselves or use the WIP (NVIDIA/jitify#131).

I meant we'll no longer rely on NVRTC to find headers, but instead roll out a search logic that mimics the compiler behavior. This is the only way we can gain knowledge about header location. I think NVIDIA/jitify#131 did quite a bit of the work, I just don't know if in the long term we'd still want to use Jitify, or just vender that search loop implementation in CuPy and move on.

xref: NVIDIA/jitify#107 (comment)

from cupy.