Trying to delete a table while a disk is in flood stage fails about crate HOT 5 CLOSED

Currently, blocks.read_only_allow_delete flag is switched automatically to true when the watermark is reached. At this point, user can only delete rows from tables, and nothing more. To drop a table you need to change Metadata, and currently blocks.read_only_allow_delete blocks such operations, probably because we assume that if there is no disk space left, changing the metadata could potentially cause corruptions.

Allowing only delete of rows, if space is exhausted, won't help, as deleting rows, means adding data to translog and then new segments, which will require more space (until afterwards, segments are merged and space is indeed freed up).

We should probably allow to drop tables when blocks.read_only_allow_delete is automatically set to true, to free up space, BUT we need to investigate and make sure that changing cluster state metadata in such conditions (very low, or even no space left on device situation) is safe.

Thank you @seut for the discussion on this topic.

from crate.

doesn't this relate to: #14963 ?

from crate.

doesn't this relate to: #14963 ?

How can I free up space, if I can't delete a table :)

from crate.

As we state now in the docs, if the table is read_only, you need to reset this flag, or set write to true to be able to delete. the blocks.read_only_allow_delete is a flag that is switched back and forth automatically if the watermark is reached/unershot.

from crate.

Relates a bit to #12710. I extended the info there to close this as duplicate

from crate.