mi_rezalloc() zero initialization is unsafe about mimalloc HOT 5 CLOSED

Ah this is a bit tricky: I now designed the zero interface, like mi_calloc , mi_zalloc , to only zero out the requested size, and for mi_rezalloc to only zero out the any additional requested size (if reallocating to a larger size).

A change we could make is to zero out always up to the useable size instead of requested size. I think that would be a good idea.

_It seems to me that the rezalloc feature requires to store the requested size or at least to zero out the implicit allocated range form requested size to allocated size in all allocations. _

It does do that already, zero out any additional space if reallocating to a larger area. Of course, the existing data should not zero out.

from mimalloc.

It's tricky indeed! Unfortunately, a change limited to the 'z' and 'c' allocation variants will not solve the problem. An example:

Initial allocation of 6 bytes by mi_malloc() (with debug initialization simulating an undefined state):
*p=xD0D0D0D0D0D0D0D0D0D0

The application copies its C string of length 5 plus a null byte as termination, i.e. it defines the state of the requested memory completely:
*p=x48656C6C6F00D0D0 ("Hello")

Another part of the application wants to append a sixth character to the C string and ensure termination by using mi_rezalloc(p, 7). The current implementation of _mi_realloc_zero() recognizes that the assigned block is large enough and delivers it unchanged. The application appends another character but no termination, since it used the initialization of the additional memory with zero. We end up with an unterminated C string with funny effects up to segmentation fault.
*p=x48656C6C6F21D0D0 ("Hello!ÐÐ...")

Can the implementation of mi_rezalloc() prevent this?
I don't see any way to do this without mi_rezalloc() knowing the size of the original allocation or initializing the extra memory available in all allocations (not just the 'z' and 'c' variants) to zero.

Note that even if the old block is copied to a larger new block, the undefined memory area is copied as well, since only the block size is known. The extension of the zero initialization by one word in the area of the old block is overwritten, so that the extended initialization remains ineffective! (see master commit 7b4f359; alloc.c; lines 336 and 339)

On possible solution without performance losses if the feature is not used would be the requirement to pass the originally requested size resp. count in mi_rezalloc() resp. mi_recalloc(). The application needs to know these values anyway in order to use the memory correctly, so why not make this missing information available? Aren't there even suggestions to call mi_free() itself with the requested size to allow better debug/consistency checks?

from mimalloc.

Ah, nice example. The way it should be I think is that (a) calloc, zalloc always zero out all the useable memory. rezalloc and recalloc are only well-defined for a previous calloc,zalloc,rezalloc,recalloc, but not if the previous memory was alloc'd with malloc. That would fix this I think. Still, we cannot enforce this and perhaps we should do away with the rezalloc variantcs...

from mimalloc.

> ...and perhaps we should do away with the rezalloc variantcs...
👍

from mimalloc.

removed rezalloc and variants in the latest dev branch. kept mi_recalloc as a Unix variant for convenience.

from mimalloc.