Apache Hudi Auto-Size During Writes is not Working for Flink SQL about hudi HOT 4 CLOSED

That is because you are using the append mode, the append mode relies on the async clustering for small files merging. You can switch to upsert operation for the tests too, the upsert will merge the files during write.

from hudi.

That is because you are using the append mode, the append mode relies on the async clustering for small files merging. You can switch to upsert operation for the tests too, the upsert will merge the files during write.

Thank you. Looks like it's working fine. However, is it mentioned in documentation anywhere? All that I see is the note - the bulk_insert write operation does not have auto-sizing capabilities during ingestion

Another question, by adding the following property I was able to achieve the same for insert operation type:

'write.insert.cluster' = 'true',

This property above is not greatly documented either, I found it by digging in source code..

Is there any difference really? Or overall recommendation is to use async clustering?

Thanks in advance

from hudi.

'write.insert.cluster' = 'true' is actually inline clustering, that means it will do the clustering on each write, so the write throughput is low comparing to the async clustering.

Sorry it is not well documented.

from hudi.

Created a tracking JIRA to fix doc - https://issues.apache.org/jira/browse/HUDI-7396

Thanks @dannyhchen @vkhoroshko . Closing this.

from hudi.