picrap / exfat Goto Github PK

View Code? Open in Web Editor NEW

8.0 4.0 5.0 562 KB

exFAT read/write

License: MIT License

C# 100.00%

exfat partition filesystem

exfat's Introduction

ExFat

An exFAT accessor library.

Summary

ExFat allows to manipulate an exFAT formatted partition (provided as a System.IO.Stream). It comes with two packages:

The core package ExFat.Core available from NuGet, which allows simple exFAT management at three different levels (partition, entry and path).
The DiscUtils package ExFat.DiscUtils available from NuGet, which depends on DiscUtils package.

Currently, ExFat.Core does what it says: files/directories manipulation at any level. DiscUtils support is on its way and should be released in the very next few days.

ExFat.Core works at three levels:

Lowest level: partition access. This allows to manipulate clusters, allocation bitmap, directory entries and clusterr streams.
Middle level: entry access. Files/directories can be used to read/write content.
High level: path access. This works as you would expect using file paths.

ExFat.DiscUtils is also a high-level access (using paths) with implementation for DiscUtils.

Because it is still under development, you can see pending features state here.

Samples

All examples assume you have a Stream containing an exFAT partition.

// Access at partition-level. Most efficient, most dangerous.
// Integrity is not guaranteed at this level, 
// user needs to make all neceassary operations in right order.
using(var partition = new ExFatPartition(partitionStream))
{
    // returns all entries (including bitmap, volume label, etc.) from root directory
    var entries = partition.GetEntries(partition.RootDirectoryDataDescriptor);

    // returns all files/directories meta entries
    var metaEntries = partition.GetMetaEntries(partition.RootDirectoryDataDescriptor);

    // assuming there is one, of course (but we're in a sample)
    var someDirectory = metaEntries.First(e => e.IsDirectory);
    var directoryMetaEntries = partition.GetMetaEntries(someDirectory.DataDescriptor);

    var someFile = metaEntries.First(e => !e.IsDirectory);
    using(var dataStream = partition.OpenDataStream(someFile.DataDescriptor, FileAccess.Read))
    { }
}

// Access at entry level. Quite fast, since user has to track entries.
// Integrity is guaranteed. File attributes are not honored (maybe one day...)
using(var entryFilesystem = new ExFatEntryFilesystem(partitionStream))
{
    var someFileEntry = entryFilesystem.FindChild(filesystem.RootDirectory, "someFile");
    using(var fileStream = entryFilesystem.OpenFile(someFileEntry, FileAccess.Read)
    { }

    // finding one file in a directory requires two steps
    var someDirectoryEntry = entryFilesystem.FindChild(filesystem.RootDirectory, "someDirectory");
    var someChildFileEntry = entryFilesystem.FindChild(someDirectoryEntry, "someDirectory");
}

// Access at path level. Uses a path cache to retrieve entries, 
// so speed is not as good (but not that bad either)
// since there is not drive, paths only specify the directory chain
// (so "a\b\c" for example)
using(var pathFilesystem = new ExFatPathFilesystem(partitionStream))
{
    var rootEntries = pathFilesystem.EnumerateEntries("\"); // "" works too for root
    var childEntries = pathFilesystem.EnumerateEntries(@"\somedir"); // "somedir" works too
    using(var s = pathFilesystem.Open(@"a\b\c", FileMode.Open, FileAccess.Read)
    { }
}

Current build status (for people who care... If you ever meet one):

exfat's People

Contributors

Stargazers

Watchers

Forkers

radtek ltrdata jason6568 bangush icyson55

exfat's Issues

delay write of all metadata

will be good to have option to delay writing of all of metadata (Directory and FAT entries).
So it will be possible to sequential write data of many files without interrupting to write its metadata.
Sequential write is huge faster than write non-sequentially relatively small chunks.
Flush metadata either explicitly on convenient time or optionally by configurable timer.
Timer can be different for different operations, like 30 seconds for adding/moving/renaming/changing files and 1 second for deleting/shrinking files.
Either to have 2 streams, one for file data and second for metadata, so stream for metadata can be buffered.

disk corruption issues

Hello,

A couple of problems I encountered while using ExFat.DiscUtils with a vmdk virtual disk formatted ExFat:

The below code was used to copy files placed in a host directory to an empty directory in a VM.
Then, we power on the VM ,delete the files from the VM manually via file explorer, then power off the VM.
Doing this a couple of times (copying files to VM then deleting them and re-copying) with lots of files or directories causes disk corruption.
Specifically, fill the host directory with 300-400 files, execute the given code, then delete the files copied into the VM (manually).
After repeating this a couple of times the file system becomes corrupted.
This has been reproduced a couple of times but seems a bit random.
Copying an empty file (can be done with the below code) causes disk corruption.
It seems this happens every time we copy empty files.

Code used:

private static void Test(string vmdkPath, int partitionNumber, string pathInHost, string pathInVm)
{
	using (VirtualDisk disk = new DiscUtils.Vmdk.Disk(vmdkPath,FileAccess.ReadWrite))
	using (ExFatFileSystem fs = new ExFatFileSystem(disk.partitions[partitionNumber].Open()))
	{
		string[] hostDirFilePaths= Directory.GetFiles(pathInHost, "*.*", SearchOption.TopDirectoryOnly);
		
		foreach (string hostFilePath in hostDirFilePaths)
		{
			string vmFilePath = Path.Combine(pathInVm, Path.GetFileName(hostFilePath));
			
			string dst = Path.Combine(fs.Root.FullName, vmFilePath);
			
			using (Stream dstStream = fs.OpenFile(dst, FileMode.Create, FileAccess.ReadWrite))
			using (Stream sourceStream = File.Open(hostFilePath, FileMode.Open, FileAccess.Read))
			{
				CoptStream(sourceStream, dstStream);
			}
		}
	}
}

private static void CopyStream(Stream srcStream, Stream dstStream)
{
	int bufferLength = (int)Math.Min(srcStream.Length, 4*1024*1024);
	byte[] buffer = new byte[bufferLength];
	int bytesRead= 0;
	
	while((bytesRead = srcStream.Read(buffer,0,bufferLength))>0)
	{
		dstStream.Write(buffer,0,bytesRead);
		if(bytesRead < bufferLength)
		{
			break;
		}
	}
}

Thanks,
Chris

Second allocation bitmap

Currently, it is superbly ignored.

Thread-satefy

Currently it's randomly supported (nothing to be proud of).

Partition formatting

Using DiscUtils

DataLength != ValidDataLength

Make sure the DataDescriptor knows about this.

Flushing after file operations

Currently some dirty clusters can stay in memory for a while, which is not good.

Error when running the program - Error Failed to find TextTransform.exe tool

I just cloned the project and Im trying to run it in Visual Studio. However I get errors:

Warning A custom tool 'TextTemplatingFileGenerator' is associated with file 'ExFat.tt', but the output of the custom tool was not found in the project. You may try re-running the custom tool by right-clicking on the file in the Solution Explorer and choosing Run Custom Tool. ExFat.Core

Error Failed to find TextTransform.exe tool at 'C:\Program Files (x86)\Common Files\Microsoft Shared\TextTemplating\14.0\TextTransform.exe. ExFat.Core

Warning A custom tool 'TextTemplatingFileGenerator' is associated with file 'Properties\ProductInfo.tt', but the output of the custom tool was not found in the project. You may try re-running the custom tool by right-clicking on the file in the Solution Explorer and choosing Run Custom Tool. ExFat.Core

Warning A custom tool 'TextTemplatingFileGenerator' is associated with file 'ExFat.DiscUtils.tt', but the output of the custom tool was not found in the project. You may try re-running the custom tool by right-clicking on the file in the Solution Explorer and choosing Run Custom Tool. ExFat.DiscUtils