Mount error: unknown entry type 0xca about exfat HOT 24 CLOSED

Thanks for the fix! I'm afraid the camera with the buggy firmware is meanwhile already broken :), but it's good to know there's an option in exfatfs that can fix it in case it might happen again somewhere.

Спасибо большое!

from exfat.

That's interesting, I've never seen such entry type before. Did you use this drive in a camera, media player or some other device (i.e. not a PC)? Could you post the output of dumpexfat -s /dev/sdb1?

from exfat.

ok I will
thanks for the reply
want to keep this distro on my yoga and don't want tis to be a deal breaker
works on older Toshiba using ubuntu lts with exfat-utilities xfce just fine

On Sun, Apr 10, 2016 at 12:17 AM, relan [email protected] wrote:

That's interesting, I've never seen such entry type before. Did you use
this drive in a camera, media player or some other device (i.e. not a PC)?
Could you post the output of dumpexfat -s /dev/sdb1?

—
You are receiving this because you authored the thread.
Reply to this email directly or view it on GitHub
#33 (comment)

from exfat.

I've got a similar error on an SDXC card formatted in a camera.

I think I've got a fix for it. :)

I analyzed the situation last night. The superblock says the
filesystem uses 512-byte sectors (9 Bits, OK), and 256-sector clusters
(8 Bits). This yields a clustersize of 128 KiB.

The root directory only has a few entries (aside from the mandatory
stuff, just the "DCIM" subdir), followed by a bunch of zeros.
However, after 64 KiB, some random data block appears – obviously not
zeroed out by the camera.

So whatever the first byte of that random data block is (0xE2 in my
case) causes the directory search to abort. In this resource:

http://ntfs.com/exfat-directory-structure.htm

it is told that an entry with type 0x00 is simply considered to be the
end of the directory marker, so no further scanning is due there.
Implementing this causes exfatfsck to pass (well, almost, it still
complains about an nonsensical value 240 for centiseconds, but that's
not a fatal error), and I can mount it and retrieve the data.

Here's the respective patch:

--- libexfat/node.c~    2016-05-23 21:27:33.911662917 +0200
+++ libexfat/node.c 2016-05-23 23:12:51.857185939 +0200
@@ -200,6 +200,11 @@
        }

        entry = get_entry_ptr(ef, it);
+       if (entry->type == 0)
+       {
+           rc = -ENOENT;
+           break;
+       }
        switch (entry->type)
        {
        case EXFAT_ENTRY_FILE:

libexfat-node-c-diff.txt

(Edit: sorry, there's one stray fprintf() too much in the patch file, left from debugging.)

from exfat.

Thanks for your analysis! This explains why fuse-exfat fails to mount this SD card.

However, the statement about EOD (0x00) entry in this document is incorrect. To check this I've created a fresh image and formatted it into exFAT in Windows 7 with 4 KB cluster. The root directory cluster looked like this (my comments on the right):

00043000  03 00 00 00 00 00 00 00  00 00 00 00 00 00 00 00  label
00043010  00 00 00 00 00 00 00 00  00 00 00 00 00 00 00 00
00043020  81 00 00 00 00 00 00 00  00 00 00 00 00 00 00 00  bitmap
00043030  00 00 00 00 02 00 00 00  1a 0c 00 00 00 00 00 00
00043040  82 00 00 00 0d d3 19 e6  00 00 00 00 00 00 00 00  upcase
00043050  00 00 00 00 03 00 00 00  cc 16 00 00 00 00 00 00
00043060  00 00 00 00 00 00 00 00  00 00 00 00 00 00 00 00  zeros
*

Then I've filled the last 1 KB of this cluster with random values:

00043000  03 00 00 00 00 00 00 00  00 00 00 00 00 00 00 00
00043010  00 00 00 00 00 00 00 00  00 00 00 00 00 00 00 00
00043020  81 00 00 00 00 00 00 00  00 00 00 00 00 00 00 00
00043030  00 00 00 00 02 00 00 00  1a 0c 00 00 00 00 00 00
00043040  82 00 00 00 0d d3 19 e6  00 00 00 00 00 00 00 00
00043050  00 00 00 00 03 00 00 00  cc 16 00 00 00 00 00 00
00043060  00 00 00 00 00 00 00 00  00 00 00 00 00 00 00 00
*
00043c00  e9 23 6b 0d 78 5a aa 41  c9 46 88 6d 11 2b 75 e8
00043c10  1d ba 82 89 0d 06 86 4b  d5 27 d6 e6 88 3f 6a 6c
00043c20  87 f3 c6 d5 3f 27 ce 53  2f a6 ac 6f 2c b6 f3 7d
00043c30  7c ab 48 93 ab 5a ae 5a  2f 41 c8 74 7f ad 8d 3a
00043c40  2e 05 bb 4b 92 5a 89 ba  90 66 41 8d 7e 2c 26 e1
00043c50  cf b7 a5 07 0d 07 7c c6  9d 1b 15 58 56 e8 08 b2
... skip ...
00043fe0  6f 56 1f a8 bb 0e 6e 49  ad ba ab 3a 3f 64 bf 96
00043ff0  78 54 32 6e 6c f9 e8 52  26 d9 31 11 ec 29 c6 ae
00044000  00 00 00 00 00 00 00 00  00 00 00 00 00 00 00 00
*

Windows' chkdsk reported a lot of errors on this FS despite zero entries after the last valid entry (0x82) in the directory.

I think the guys who wrote the doc assumed exFAT works with directories in the same way as FAT32. That's not the case: exFAT does not use EOD.

Could you dd your SD card with random data, format it to exFAT in your camera and run Windows' chkdsk against it?

from exfat.

Well, reformatting the card right now is not an option. We first
need to pull all the data from it.

Maybe Windows doesn't care if the garbage is simply beyond 64 KiB,
despite of the 128 KiB cluster size? I didn't try Windows (don't have
it), but OSX had no troubles reading the card.

I'd upload you a dump file containing the first few megabytes that
include the root directory and the block following it, but somehow,
github rejects the file (it keeps telling me I am supposed to upload
Zip files, but it is a Zip file, sigh).

I've (temporarily) uploaded it to

http://uriah.heep.sax.de/outgoing/exfat-data.bz2

(Unfortunately, since I did have it in OSX meanwhile, the root
directory is now filled with entries for the OSX Finder stuff. I
deleted these again, but the remnants are still there.)

from exfat.

Maybe Windows doesn't care if the garbage is simply beyond 64 KiB, despite of the 128 KiB cluster size?

I've made the same experiment with 128 KB cluster and got the same result: chkdsk reports a ton of errors.

http://uriah.heep.sax.de/outgoing/exfat-data.bz2

I've inspected this image an can confirm that it looks exactly as you described. So, the conclusion is: it's buggy camera software. Workaround: create filesystem using mkexfatfs.

from exfat.

I had a fellow coworker run chkdsk on that card on his Windows 7. Yes,
it complains about the garbage, trying to interpret it as directory
data.

Nevertheless, both Windows as well as OSX do not have troubles reading
the medium, and they don't try to display the garbage data in the 2nd
half of the root directory as any kind of directory entries.

We cannot fix that kind of buggy camera firmware, but as this bug
report (not opened by me) shows, we'll obviously always encounter it
in the field. So in order to support our users best, we somehow have
to cope with it (at least as well as Windows and OSX can cope with
it).

Since there is no point in searching for further valid directory
entries after the first 0x00 one (intermediate deleted entries are
not being zeroed out but only get their "valid" bit removed), I
suggest we stop reading the directory at that point for normal
operation (as my patch suggests).

However, exfatfsck might (and should) complain about the garbage, just
as MS' chkdsk does. That probably requires that exfatfsck being
written differently, but if it is ever considered that this tool might
also repair damaged filesystems, a different architecture would be
needed anyway for it.

(If you want me to supply a suggestion for this, just tell me.)

Спасибо большое за Вашая работа!

from exfat.

We cannot fix that kind of buggy camera firmware, but as this bug report (not opened by me) shows, we'll obviously always encounter it in the field.

That's true. I agree that it's better to make things just work even if a hack is needed for this. BTW what camera model is this?

Since there is no point in searching for further valid directory entries after the first 0x00 one (intermediate deleted entries are not being zeroed out but only get their "valid" bit removed), I suggest we stop reading the directory at that point for normal operation (as my patch suggests).

This approach will work fine for reading. But garbage in a directory should also be handled somehow when new entries are being created.

Right now I'm thinking of another "fix": forcefully zero out directory from the 1st zero (type 0x00) entry to the end of the directory. This would be quite simple to implement and test. Do other directories contain garbage too or it's only root directory that is broken?

from exfat.

BTW what camera model is this?

A compact camera from Rollei. Exact model I currently don't know. It's already a few years old now, so quite possible exFAT implementations were not that widely used already by the time of its development.

But garbage in a directory should also be handled somehow when new entries are being created.

I'd be reluctant to forbibly zero out something unless really needed.
So, when extending a directory, instead of zeroing out the entire remainder, only make sure there's at least one zeros-only entry past the current end of directory (i.e., make sure there are at least 32 zeros behind it).

Do other directories contain garbage too or it's only root directory that is broken?

From a glance at the next two directories (DCIM, and DCIM/103DICAM), they look OK.

from exfat.

I'd be reluctant to forbibly zero out something unless really needed.

I think it is really needed because working with a corrupted FS is asking for problems. It's easier to clear a minefield and then work safely than permanently keep in mind that there are mines over here.

So, when extending a directory, instead of zeroing out the entire remainder, only make sure there's at least one zeros-only entry past the current end of directory (i.e., make sure there are at least 32 zeros behind it).

In the early versions of fuse-exfat I did maintain a EOD (i.e. type 0x00) entry. Believe me, it's a mess. Microsoft's decision to get rid of it makes a lot of sense.

From a glance at the next two directories (DCIM, and DCIM/103DICAM), they look OK.

Great. This means that camera's FS creation code is buggy (not FS driver). Thus we need to handle only root directory in a special way.

from exfat.

I agree adding more hacks is not a good idea.

My suggestion:

• It ought to be possible to mount a corrupted filesystem read-only, and extract all the data from it without changing anything.
• If a filesystem is detected as being corrupt, all write operations should be rejected.
• Fixing the corrupted filesystem would better be referred to exfatfsck, rather than trying to perform "automatic" fixing internally.

from exfat.

Heh. I was just working on a patch to do a dangerous-read-only mount for data recovery.

In my case, I was getting corrupt checksums leading to unreadable directories which exfatfsck couldn't fix (windows could) and I just wanted my data back.

from exfat.

@dl8dtl, you suggest a more generic scheme. This makes sense, especially in the long run. An implementation will require a lot of efforts, though.

@ckuethe, that's quite a complex task.

from exfat.

An implementation will require a lot of efforts, though.

Я знаю ... можно помогать, немножко.
Just tell me what you need.

from exfat.

I called it "dangerous" for a reason. I disabled all the enforcement on error conditions just so that I could read my file system. Then I reformatted my device as FAT32 because I've had 3 corrupted exfat volumes this week and no unusable FAT32 in decades.

from exfat.

Note that exfatfsck does not currently 'fix' things -> issue #3

from exfat.

I stumbled into a similar issue last night. It's been working great for a couple of months but now I can't access the pen drive on my Linux machine. (worked after reboot though)

I have a USB 3.0 64GB pen drive which I have mounted in fstab:

/dev/sda1 /media/sda1 exfat-fuse auto,nofail,uid=33,gid=33,umask=0007 0 0

Here the output from dumpexfat:

dumpexfat 1.1.0
Volume serial number 0xc292d5a1
FS version 1.0
Sector size 512
Cluster size 131072
Sectors count 125304768
Clusters count 489447
First sector 64
FAT first sector 2048
FAT sectors count 3840
First cluster sector 6144
Root directory cluster 4
Volume state 0x0002
FATs count 1
Drive number 0x80
Allocated space 33%

This is what I've found in the syslog, around10 times every second:

Dec 30 19:39:37 localhost mount.exfat-fuse: unknown entry type 0x96

I also see that I have a huge amount of errors when running exfatfsck (1148...) of files which haven't been allocated. What's the reason for this?

from exfat.

Kalekulan, you are using a very old exfat-utils version (1.1, current is 1.2.5). I'd recommend to update.

Updating will hardly solve this particular issue though. Please post the output of exfatfsck /dev/sda1.

from exfat.

Thanks for the response.

I've updated now to 1.2.4. Version 1.1 is the one in the stable debian jesse rep.

I also see that I have a huge amount of errors when running exfatfsck (1148 of them...) of files which haven't been allocated (most of them are photo syncs from my iPhone 6).

Snip:
ERROR: cluster 0x23e64 of file 'Photo-2016-12-28-13-44-022276.JPG' is not allocated.

I put the pendrive in my Windows machine, trying to repair the allocation issues.
Now I get this output instead:
exfatfsck 1.2.4
WARN: volume was not unmounted cleanly.
Checking file system on /dev/sda1.
File system version 1.0
Sector size 512 bytes
Cluster size 128 KB
Volume size 60 GB
Used space 20 GB
Available space 40 GB
BUG: invalid cluster number 0.
Aborted

When I run mount.exfat-fuse -d I get this (snip):

getattr /123/abc/files/test.png
unique: 319, error: -2 (No such file or directory), outsize: 16
unique: 320, opcode: STATFS (17), nodeid: 5, insize: 40, pid: 1634
statfs /123/abc
unique: 320, success, outsize: 96

For the files which are unallocated.

from exfat.

I also see that I have a huge amount of errors when running exfatfsck (1148 of them...) of files which haven't been allocated

This means the volume is heavily damaged. Try photorec to recover the photos.

BUG: invalid cluster number 0.

This was fixed in v1.2.5: it'll report an error and continue checking.

Do you get unknown entry type 0x96 error with v1.2.4?

from exfat.

Ok, thanks!

No, the check stops when I get the invalid cluster number 0.

I've tried it with 1.2.5 now:

ERROR: 'Photo-2016-12-17-15-06-58_2246.JPG' directory starts with invalid cluster 0.
ERROR: 'Video-2016-12-17-13-13-19_2245.MOV' directory starts with invalid cluster 0.
ERROR: 'Photo-2016-12-19-11-50-52_2250.PNG' directory starts with invalid cluster 0.
ERROR: 'Photo-2016-12-24-14-46-56_2263.JPG' directory starts with invalid cluster 0.
ERROR: 'Video-2016-12-29-21-36-47_2277.MP4' directory starts with invalid cluster 0.
ERROR: 'Photo-2016-12-29-21-39-36_2278.JPG' directory starts with invalid cluster 0.
ERROR: 'Photo-2016-12-29-21-44-43_2279.JPG' directory starts with invalid cluster 0.
ERROR: 'Photo-2016-12-31-12-21-26_2288.JPG' directory starts with invalid cluster 0.

Totally 10463 directories and 37884 files.

from exfat.

The FS is damaged and I've no idea why. I'd recommend to save files from this USB stick (possibly with photorec) and format it. Keep an eye on it, this could be a hardware failure.

from exfat.

exfatfsck in v1.3.0 can now fix this kind of error.

from exfat.