Zim format detection does not work in 2.12.1 and 2.12.2 about markor HOT 12 CLOSED

Also solved in #2088 #2307

Edit. Sorry, wrong pr linked

from markor.

from markor.

testbuilsa of prs are at the github actions tab.

👍 Wow. Much CI. Such technology.

So I tested apk from #2307 and it indeed restored zim format handling

from markor.

Also solved in #2088

That PR is closed a year ago but the issue is a recent regression. I suppose it was introduced about a month or two ago

from markor.

I can confirm that with the latest version Zim files are not detected as Zim any more, just as plaintext.
Zim pages are in fact *.txt, but they start with the "header" lines e.g.

Content-Type: text/x-zim-wiki Wiki-Format: zim 0.6 Creation-Date: 2019-01-14T23:41:51+01:00

Probably this file content check for *.txt was broken somehow, so Markor just treats them as plain text files due to the ending.

If #2307 fixes the issue already, nice! :)

from markor.

@harshad1 I just took a look into #2307 , and if I didn't overlook something, I think it won't fix the issue, unfortunately.
I analyzed the problem a bit (just looked at the Github code via Octodroid on the smartphone, so I didn't test anything), and I think I found the reason for the problem.

It seems that this commit introduced the bug:
f35383a

In Document.java, you removed the if-elseif-Statements with a loop iteration over the newly introduced FormatRegistry.FORMATS:

The problem here is: Just using the lower-case filename fnlower in isFileOutOfThisFormat(fnlower) in a generic way is not feasible. In the original code most calls used fnlower as the argument, but Zim, Embedbinary and Orgmode used getPath(): isFileOutOfThisFormat(getPath()). And using the full path of the file is important, because Markor needs it to open the file, look into the content and decide if it contains a Zim header (don't know about the other two formats, but probably they need to be checked somehow too - so there could be similar bugs).
Moreover, the order of the file format checks is important: Originally, plaintext format was only used as fallback if no other format matches before. With the changed code, plaintext is at position 2 and Zim at position 4, so Zim files (ending *.txt just as plaintext) don't have any chance of being matched as Zim.
But like I said, it is necessary to also look into the file contents in addition to checking the line ending to decide if a file is Zim or just an "ordinary" plaintext file, so changing the order is not enough.

from markor.

Ah. You're right. I will fix this today

from markor.

I have this fixed and pushed up in #2307

Please test.

from markor.

@harshad1 Ah I saw that you already changed the order of the formats in a previous commit. 👍
Could work now with the change to the file path instead of just the lower case filename.

I just have my smartphone right now and cannot build the APK for testing, however, and I don't even have an Android dev environment set up on my computer at this time... ^^
But there's a valid Zim page under samples/Zim-Sample-Notebook/Root_Page.txt, so you could check if highlighting works if you open this file.

Maybe also @Self-Perfection could do a test if it works now.

Additionally I found a suspicious line in WikitextTextConverter - but this seems to have been there before in a similar form, so probably not exactly related to the issue here:
return Arrays.asList(new String[]{".wikitext"}).contains(ext);
I don't know what this ".wikitext" string could be - if it is supposed to be a file ending, then it's wrong, there is no such file ending as ".wikitext" in Zim, just ".txt". (The term "Wikitext" in general is not related to Zim in any way and not a good naming for the specific format IMO, but that's a different story.)

from markor.

Sorry, I can not test. Compiling from source code is beyond my capabilities now.

from markor.

Nice to know that the CI pipelines includes test apks. 👍
I tried it out as well and can confirm: the zim format detection works.

from markor.

Marking this done as the fix is in #2307

from markor.