Giter Club home page Giter Club logo

projfs-managed-api's Introduction

ProjFS Managed API

Branch Functional Tests
main Build status

About ProjFS

ProjFS is short for Windows Projected File System. ProjFS allows a user-mode application called a "provider" to project hierarchical data into the file system, making it appear as files and directories in the file system. For example, a simple provider could project the Windows registry into the file system, making registry keys and values appear as files and directories, respectively. An example of a more complex provider is VFS for Git, used to virtualize very large git repos.

Conceptual documentation for ProjFS along with documentation of its Win32 API is at docs.microsoft.com.

Enabling ProjFS

ProjFS enablement is required for this library to work correctly. ProjFS ships as an optional component starting in Windows 10 version 1809.

About the ProjFS Managed API

The Windows SDK contains a native C API for ProjFS. The ProjFS Managed API provides a wrapper around the native API so that developers can write ProjFS providers using managed code.

Note that to use this library on a computer that does not have Visual Studio installed, you must install the Visual C++ redistributable.

Solution Layout

ProjectedFSLib.Managed project

This project contains the code that builds the API wrapper, ProjectedFSLib.Managed.dll. It is in the ProjectedFSLib.Managed.API directory.

SimpleProviderManaged project

This project builds a simple ProjFS provider, SimpleProviderManaged.exe, that uses the managed API. It projects the contents of one directory (the "source") into another one (the "virtualization root").

ProjectedFSLib.Managed.Test project

This project builds an NUnit test, ProjectedFSLib.Managed.Test.exe, that uses the SimpleProviderManaged provider to exercise the API wrapper. The managed API is a fairly thin wrapper around the native API, and the native API has its own much more comprehensive tests that are routinely executed at Microsoft in the normal course of OS development. So this managed test is just a basic functional test to get coverage of the managed wrapper API surface.

Building the ProjFS Managed API

  • Install Visual Studio 2022 Community Edition.
    • Include the following workloads:
      • .NET desktop development
      • Desktop development with C++
    • Ensure the following individual components are installed:
      • C++/CLI support
      • Windows 10 SDK (10.0.19041.0)
  • Create a folder to clone into, e.g. C:\Repos\ProjFS-Managed
  • Clone this repo into a subfolder named src, e.g. C:\Repos\ProjFS-Managed\src
  • Run src\scripts\BuildProjFS-Managed.bat
    • You can also build in Visual Studio by opening src\ProjectedFSLib.Managed.sln and building.

The build outputs will be placed under a BuildOutput subfolder, e.g. C:\Repos\ProjFS-Managed\BuildOutput.

Note: The Windows Projected File System optional component must be enabled in Windows before you can run SimpleProviderManaged.exe or a provider of your own devising. Refer to this page for instructions.

Dealing with BadImageFormatExceptions

The simplest cause for BadImageFormatExceptions is that you still need to enable ProjFS.

For .Net Core specific consumers, this can also occur when the .NET Core loader attempts to find Ijwhost.dll from the .NET Core runtime. To force this to be deployed with your application under MSBuild, add the following property to each csproj file that is importing the Microsoft.Windows.ProjFS package:

<PropertyGroup>
  <UseIJWHost>True</UseIJWHost>
</PropertyGroup>

Contributing

For details on how to contribute to this project, see the CONTRIBUTING.md file in this repository.

This project has adopted the Microsoft Open Source Code of Conduct. For more information see the Code of Conduct FAQ or contact [email protected] with any additional questions or comments.

projfs-managed-api's People

Contributors

abhijeet-gautam avatar brphelps avatar cgallred avatar erikmav avatar microsoft-github-policy-service[bot] avatar microsoftopensource avatar msftgits avatar narasamdya avatar yerudako avatar yerudakoms avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

projfs-managed-api's Issues

.NET Core support

Hello,

Currently, this library targets .NET Framework 4.6, and uses C++/CLI. I've heard that C++/CLI support will finally arrive with .NET Core 3.1.

Are there any plans to target it? We plan to move to .NET Core as soon as possible.

Thanks!

Proper way to handle renaming placeholder files

Hello,

I work on EdenFS (https://github.com/facebookexperimental/eden) and one of the bug I'm currently looking at is the surprising behavior of ProjectedFS when renaming a placeholder file:

% fsutil.exe reparsepoint query .\TARGETS
Reparse Tag Value : 0x9000001c
Tag value: Microsoft
Tag value: Directory

Reparse Data Length: 0x146
Reparse Data:
0000:  02 00 00 00 00 00 00 00  06 15 99 15 a7 08 19 44  ...............D
0010:  bc 72 50 5f 10 6d 85 0e  00 00 00 00 00 00 00 00  .rP_.m..........
0020:  00 00 00 00 00 00 00 00  00 00 00 00 00 00 00 00  ................
0030:  00 00 00 00 00 00 00 00  00 00 00 00 00 00 00 00  ................
0040:  00 00 00 00 00 00 00 00  00 00 00 00 00 00 00 00  ................
0050:  00 00 00 00 00 00 00 00  00 00 00 00 00 00 00 00  ................
0060:  00 00 00 00 00 00 00 00  00 00 00 00 00 00 00 00  ................
0070:  00 00 00 00 00 00 00 00  00 00 00 00 00 00 00 00  ................
0080:  00 00 00 00 00 00 00 00  00 00 00 00 00 00 00 00  ................
0090:  00 00 00 00 00 00 00 00  00 00 00 00 00 00 00 00  ................
00a0:  00 00 00 00 00 00 00 00  00 00 00 00 00 00 00 00  ................
00b0:  00 00 00 00 00 00 00 00  00 00 00 00 00 00 00 00  ................
00c0:  00 00 00 00 00 00 00 00  00 00 00 00 00 00 00 00  ................
00d0:  00 00 00 00 00 00 00 00  00 00 00 00 00 00 00 00  ................
00e0:  00 00 00 00 00 00 00 00  00 00 00 00 00 00 00 00  ................
00f0:  00 00 00 00 00 00 00 00  00 00 00 00 00 00 00 00  ................
0100:  00 00 00 00 00 00 00 00  00 00 00 00 00 00 00 00  ................
0110:  00 00 00 00 00 00 00 00  2c 00 66 00 62 00 63 00  ........,.f.b.c.
0120:  6f 00 64 00 65 00 5c 00  65 00 64 00 65 00 6e 00  o.d.e.\.e.d.e.n.
0130:  5c 00 66 00 73 00 5c 00  54 00 41 00 52 00 47 00  \.f.s.\.T.A.R.G.
0140:  45 00 54 00 53 00                                 E.T.S.
% mv .\TARGETS TARGETS2

% fsutil.exe reparsepoint query .\TARGETS2
Reparse Tag Value : 0x9000001c
Tag value: Microsoft
Tag value: Directory

Reparse Data Length: 0x146
Reparse Data:
0000:  02 00 00 00 08 00 00 00  06 15 99 15 a7 08 19 44  ...............D
0010:  bc 72 50 5f 10 6d 85 0e  00 00 00 00 00 00 00 00  .rP_.m..........
0020:  00 00 00 00 00 00 00 00  00 00 00 00 00 00 00 00  ................
0030:  00 00 00 00 00 00 00 00  00 00 00 00 00 00 00 00  ................
0040:  00 00 00 00 00 00 00 00  00 00 00 00 00 00 00 00  ................
0050:  00 00 00 00 00 00 00 00  00 00 00 00 00 00 00 00  ................
0060:  00 00 00 00 00 00 00 00  00 00 00 00 00 00 00 00  ................
0070:  00 00 00 00 00 00 00 00  00 00 00 00 00 00 00 00  ................
0080:  00 00 00 00 00 00 00 00  00 00 00 00 00 00 00 00  ................
0090:  00 00 00 00 00 00 00 00  00 00 00 00 00 00 00 00  ................
00a0:  00 00 00 00 00 00 00 00  00 00 00 00 00 00 00 00  ................
00b0:  00 00 00 00 00 00 00 00  00 00 00 00 00 00 00 00  ................
00c0:  00 00 00 00 00 00 00 00  00 00 00 00 00 00 00 00  ................
00d0:  00 00 00 00 00 00 00 00  00 00 00 00 00 00 00 00  ................
00e0:  00 00 00 00 00 00 00 00  00 00 00 00 00 00 00 00  ................
00f0:  00 00 00 00 00 00 00 00  00 00 00 00 00 00 00 00  ................
0100:  00 00 00 00 00 00 00 00  00 00 00 00 00 00 00 00  ................
0110:  00 00 00 00 00 00 00 00  2c 00 66 00 62 00 63 00  ........,.f.b.c.
0120:  6f 00 64 00 65 00 5c 00  65 00 64 00 65 00 6e 00  o.d.e.\.e.d.e.n.
0130:  5c 00 66 00 73 00 5c 00  54 00 41 00 52 00 47 00  \.f.s.\.T.A.R.G.
0140:  45 00 54 00 53 00                                 E.T.S.
% cat .\TARGETS2
cat : An internal error occurred.
At line:1 char:1
+ cat .\TARGETS2
+ ~~~~~~~~~~~~~~
    + CategoryInfo          : ReadError: (C:\open\fbsource\fbcode\eden\fs\TARGETS2:String) [Get-Content], IOException
    + FullyQualifiedErrorId : GetContentReaderIOError,Microsoft.PowerShell.Commands.GetContentCommand

In the logs of EdenFS, I can see that PRJ_GET_FILE_DATA_CB was called with: fbcode\\eden\\fs\\TARGETS, ie: the path prior to the rename, which makes for some fun behavior:

% echo foo > TARGETS
% cat .\TARGETS2
foo

To fix this this, I've tried calling PrjDeleteFile after rename to clear the placeholder which subsequent reads would recreate by using PRJ_GET_PLACEHOLDER_INFO_CB, but this doesn't work when the file is renamed to a directory that isn't a placeholder (like one that mkdir just created).

From what I can see, this really feels like a bug in ProjectedFS as the placeholder should be updated with its new filename. In the case where this isn't a bug, what is the proper way to handle renaming placeholders?

PrjGetOnDiskFileState not available?

Hello,

I need a way to tell if a given random file path string from a user/client application is as "real"/local file or if it is virtual (whether hydrated or not) quickly.

From the MSDN docs, it sounds like PrjGetOnDiskFileState() is exactly what I want here, but it doesn't appear to be exposed through the managed layer ... am I overlooking it, or is there a reason it was omitted?

I can just call File.Exists(), I suppose, but that seems pretty wasteful when I'm going to be in the code that owns the VirtualizationInstance anyway.

I could also track all locally created files via callbacks, I suppose, but then I'd have to persist that and keep it in memory, and it could be millions in degenerate cases, so definitely isn't ideal.

Thanks,
Brandon

P.S. - I'm just starting on a project using this and so far it is pretty sweet and easy to work with! ;)

Windows Server 2019 Support

I seem to be struggling to get this working on Server 2019 1809 (17763.678) but it is okay on Windows 10 1809 (17763.678)

I have built the sample and enabled ProjFS by running.

Enable-WindowsOptionalFeature -Online -FeatureName Client-ProjFS -NoRestart

I can see the PrjFlt driver is loaded when running fltmc. But on Windows Server I get the below error.

Failed to create SimpleProvider.
System.IO.FileNotFoundException: Could not load file or assembly 'ProjectedFSLib.Managed.dll' or one of its dependencies. The specified module could not be found.

Is this a limitation of Windows Server?

Pure virtual file system (without any kind of caching/hydrating)

Is there a way to stop ProjFS from making a physical file when a virtual file is read?

In my backing store, I have a list of virtual files.

public class VirtualFile
{
    public string Filename;
    public Stream Stream;
}

If the contents of a virtual file are requested, I can just read from the Stream to get the requested bytes. The issue is that ProjFS creates a physical file with those contents, which I want to avoid. This is because the Streams I'm dealing with are much larger than the HDD's capacity.

So is there a way to not create physical files, and instead just respond with populated buffers like we can in Dokany?

FILE_ATTRIBUTE_OFFLINE not preserved between provider and projection

This is probably a ProjFS issue rather than an Managed API issue - I hope you can forward this to the appropriate owner.

If you return a directory or file with FILE_ATTRIBUTE_OFFLINE set inside your ProjFS provider, it seems to preserve that flag for a second and then removes it. You can see explorer.exe showing the offline x's for a second, and then it goes away again.

I think what happens is that once the file has been accessed once, ProjFS swaps it for placeholder files - or it does some sort of other update of the enumeration. However, there is no callback of any kind to the provider at this point when it does that. When it does this update it stops returning the FILE_ATTRIBUTE_OFFLINE that the provider added originally, and explorer.exe takes the x's away.

This means explorer.exe now has free reign on the files and start fetching thumbnails for everything in sight. This unfortunately makes the ProjFS unusable against a slow backing store.

Dokan / FUSE works fine for these cases - if you return FILE_ATTRIBUTE_OFFLINE, explorer.exe respects it and doesn't try and create thumbnails.

The build agent needs an upgrade to build against the new ProjFS API

Context
The build agent used in the PR Build/Test pipeline runs Windows Server 10.0.17763. There are some ProjFS APIs that were introduced in later Windows versions, such as 2004. Notably, the symlink API (https://docs.microsoft.com/en-us/windows/win32/api/projectedfslib/ns-projectedfslib-prj_extended_info - 10.0; Build 19041) that was used in #73.

One approach to solving this could be using the windows-2022 image. It seems as if this would also need changes in the build scripts to allow builds on that environment (for example, .NET Framework 4.6.1 SDK is missing there, and the VS version is 2022 and not 2019 that current script expects). Christian and I tried a couple of minor changes, but it seems as if this requires additional investigation.

Please let me know if any additional details are needed here - I will be happy to help!

NotificationMapping constructor should accept "." when asking for notifications over the whole repo

The comment for NotificationMapping.NotificationMapping.notificationRoot says "The path to the notification root, relative to the virtualization root." But actually it's not always a relative path; if you want notifications over the whole tree, "" is actually what's asked for. That's described in the remarks section of NotificationMapping. For good or bad, nobody's going to read the . Intellisense is how documentation is served in almost all cases, so the comment in the is only going to be read by a dedicated few.

I think the best way to fix this is to allow "." as the value; preserver "" as a valid value for backwards compatibility (and maybe it makes sense to some people, I don't know). If you're really committed to having "" be the way, then add something to the tag on NotificationMapping.notificationRoot to that effect.

This is quite a serious land-mine, because the net result of using "." instead of "" is you silently get no notifications and no error either.

Question: is there a way to create virtual files? (like a procfs)

Hi,

Can virtual files be created with profjs?

I mean, suppose you want to create something like a directory to show info of running processes. Possible?

If not: expected?

I understand the basic functionality of procfs and I love it, but I'd like to have both "copied files" and virtual files.

Thanks!

Bypass File hydration based on the operation

Hi,
Would it be possible to bypass file hydration based on the file operation?
Eg. on right click to view properties invokes getFileDataCallBack() and hydrates the file. The process name is explorer.exe for all operations and hence cant block based on that.
Any recommendations?

VirtualizationInstance.StartVirtualizing fails on RS4

The issue appears to be in the code that creates tempHandle. It should be updated to:

        // Use a temp location to avoid e0158.
        pin_ptr<PRJ_NAMESPACE_VIRTUALIZATION_CONTEXT> tempHandle = &(m_virtualizationContext);
        auto instanceHandle = reinterpret_cast<PRJ_VIRTUALIZATIONINSTANCE_HANDLE*>(tempHandle);

Instead of:

auto tempHandle = reinterpret_cast<PRJ_VIRTUALIZATIONINSTANCE_HANDLE>(m_virtualizationContext);

So that the address of m_virtualizationContext is passed to the native API.

Odd file behavior in virtual branches, perhaps after installing Windows Updates

Hello,

My organization has been using a ProjFS-based file syncing setup since late last year. Since around March of this year, the number of machines using this system has grown from ~400 to ~700 now. This is the primary way people get and interact with the files they work on constantly every day.

I give this background because, as of yesterday, I've now seen what appears to be a new and very odd issue for a 2nd time (on a different machine) within the last ~week.

Nothing about the code managing ProjFS or the backing store information has changed in any substantial way for a few months now (some minor inspection improvements is about it, nothing that impacts interactions with ProjFS).

The only potentially interesting data point I have is that a few days before the first case our IT group rolled out a series of Windows Updates. That said, in both cases I had confirmed that the users did NOT have any pending windows updates at the time (they had installed and rebooted since), as that can cause all kinds of odd behavior.

The updates installed were KB5004245, KB5003539, KB5004748, KB5004772. The last batch of windows updates were installed a month prior (before this issue was first reported).

In both cases, the odd behavior mysteriously went away after a system restart.

Because that appears to fix it, and because our users sometimes like to attempt to fix things themselves and not let us know, it is possible that there have been other cases of this that I'm just not aware of. That said, the timing of 2 cases within a week, and just after the updates, combined with the reboot-as-a-fix, seems a little suspicious.

Anyway, on to the odd behavior...

The user notices the issue when a ProjFS file fails to be read, and so causes some user application to fail to start (if it is an exe/dll/etc) or fail to load some content file.

This behavior, though, only impacts a subset of the ProjFS files under a given virtual root, while other virtual files work perfectly fine (I am able to trigger placeholder creation, hydration, PrjDeleteFile back to virtual and repeat with other files). In the first case, all ProjFS files in a given directory were impacted (there were some full files mixed in the directory that worked normally).
I'm not sure if it was all files in the same directory for the 2nd case as the user rebooted before I could test that case thoroughly.

In the first case, though, I did have time to try a handful of things on these badly behaving files, with odd results:

  • The files are listed in File Explorer
  • I was able to right-click on them and open file properties
  • Attempting to delete the file (hitting delete key in File Explorer), though, failed oddly with "Item Not Found" "This is no longer located at XXX"

...then, in C#, I tried:
var path = // bad path
var fi = new FileInfo(path);
// fi.Exists => false
// fi.Attributes => 0xFFFFFFFF

var di = new DirectoryInfo(Path.GetDirectoryName(path));
var file = di.EnumerateFiles().First(f => string.Compare(f.FullName, path) == 0);
// file is NOT NULL ... it was enumerated
// file.Exists => TRUE !!
// file.Attributes => 0x00440001 (RECALL_ON_DATA_ACCESS | RECALL_ON_OPEN | READONLY) -- looks like a placeholder, but no REPARSE_POINT attribute??

using (var fs = file.OpenRead())
{
fs.ReadByte();
}
// throws FileNotFoundException

FindFirstFileEx w/ FIND_FIRST_EX_ON_DISK_ENTRIES_ONLY lists no items in the directory !? (full files had been deleted at this point, leaving only the badly behaving files in the directory)
FindFirstFileEx without that flag lists the bad file with the above attributes

Calling CreateFile() directly to try and get any reparse point data also failed to invalid path.

I also tried:

  • Stopping and restarting the virtual root
  • Clearing the ProjFS negative path cache

... and re-running all of the above tests, all with the same result.

Throughout all of this, my ProjFS application logged no errors, and I suspect got not callbacks (though I can't fully prove that).

The file in question had been newly created on this machine the day before, then submitted to version control and the user had later sync'd. So, the file should have transitioned from Full-Writable -> Full-ReadOnly -> PrjDeleteFile to make it virtual -> Placeholder (app-triggered) and broken. All of this occurred a day or two after taking the windows updates.

I didn't notice any obvious critical or error system events in that time period.

And then, as a bit of a last ditch, I asked the user to reboot ... and after they did, the badly behaving placeholder files suddenly worked perfectly!

Rebooting also "magically" fixed the 2nd case.

Any ideas?
Any other things you'd want me to try if/when I see the next case?

Thanks!

HResult.Handle result code from WriteFileData() and left a hydrated placeholder in a bad state?

Hello,

We had an odd situation today where one user was unable to launch one of their executables with a "The application was unable to start correctly (0xc0000005)" error.

In my logs I found that we got an HResult.Handle back from VirtualizationInstance.WriteFileData() during a GetFileDataCallback(). We returned HResult.InternalError as the result of the callback.

The user, though, still had this file as a hydrated placeholder in this bad/incomplete state on their drive which required manual intervention to remove and fix (calling VirtualizationInstance.DeleteFile was enough, the next hydration attempt succeeded as normal).

So my questions are:

  1. What would cause the HResult.Handle result code from WriteFileData()? Something have a non-shareWrite handle open to it or?

  2. Is there a different GetFileDataCallback() result code I should be returning to prevent that corrupt hydrated placeholder from being written / staying in that bad state? Or something else I can do from within the callback to handle this better? Or do I need to schedule a VirtualizationInstance.DeleteFile to occur in another thread shortly after the callback completes so it at least doesn't stay in a bad state?

This user oddly hit this same issue on 2 files that were likely being fetched at the same time, but among many others before/during/after that all worked fine.

Thanks!

Sample blows up on large files

I tried the "SimpleProvider" sample on some large files. It fails when a client app does a read at the beginning of a large file.

I haven't spent much time debugging, but in my repro, the client app opens a 34 GB file and does a 1 MB read at offset 0. ProjFS comes to the provider asking for range 0 - 2.2 GB. When the provider calls VirtualizationInstance::WriteFileData(), it returns 0x80070216 (ERROR_ARITHMETIC_OVERFLOW). From cursory inspection in the debugger, the arguments to the call seem reasonable (64K buffer).

It looks like the magic limit is unsigned int - just over 4 GB works fine.

Repro:

  1. Create directory with a large file (I used some VHD I had sitting around):
D:\>dir "D:\Hyper-V\DokanWin10\Virtual Hard Disks"
 Volume in drive D is SSD
 Volume Serial Number is 6058-2621

 Directory of D:\Hyper-V\DokanWin10\Virtual Hard Disks

11/12/2018  02:44 PM    <DIR>          .
11/12/2018  02:44 PM    <DIR>          ..
11/12/2018  02:44 PM    36,712,742,912 DokanWin10.vhdx
               1 File(s) 36,712,742,912 bytes
               2 Dir(s)  265,872,031,744 bytes free
  1. Project the directory using the sample:
    SimpleProviderManaged.exe --sourceroot D:\Hyper-V --virtroot D:\virt

  2. Do a read at the beginning of the file. For example:

using (FileStream fs = new FileStream(@"D:\virt\DokanWin10\Virtual Hard Disks\DokanWin10.vhdx", FileMode.Open, FileAccess.Read))
{
	byte[] buf = new byte[1024*1024];
	fs.Read(buf, 0, buf.Length);
}

How to tell if a tombstone exists for a file path?

Hello,

I need to know if a tombstone exists so some of my code can behave correctly in the situation where the local user has deleted a file that exists in the backing store.

Utils.TryGetOnDiskFileState() seems like what I want, but I can't get it to return OnDiskFileState.Tombstone ... in the above situation, it returns false with fileState of 0.

This function is returning what I want/expect for all other file situations I've tried, just not this locally deleted scenario.

I did notice this comment on the OnDiskFileState.Tombstone enum value...
"The item was a tombstone and the provider did not specify UpdateType.AllowTombstone."
...but the only api that uses UpdateType that I found was on DeleteFile() and UpdateFileIfNeeded(), and I've not called either of those, so maybe that comment is just misplaced?

Thanks!

Asynchronous QueryFileName?

The OnQueryFileName callback has the following signature:

HResult OnQueryFileName(string relativePath)

Since, unlike its native counterpart, it lacks a commandId parameter, we cannot treat this callback asynchronously and therefore return HResult.Pending to complete the request later.

Is there a reason why the managed version of this method works so differently from its native equivalent?

windows 10 projected file system issues

I am trying to virtualize our c++ unreal engine 5 project, including the UE5 engine source, so it is a gigantic project.

The project compiles and runs with no issue, I can run the editor, work on assets, and submit like normal.

The only issue I am having now is with Entrian code search. Entrian doesnt work at all in PrjFS environment, some initial debugging tells me for whatever reason PrjFS's get file data callback, when triggered by Entrian(ess.exe, invoked by the VS instance of my UE project) is always stuck in PrjWriteFileData, the function call will never return, unless I intentionally close my Visual Studio.

This feels like a projFS bug to me, or if not, any insights?
Btw I am working in c++, with PrjFS api on windows 10.

Thanks a lot!

ProjFS doesn't allow deletion of virtualization root directory

Note: I tried finding a ProjFS category in Feedback Hub because microsoft/VFSForGit#504 mentioned that one was in progress, but I couldn't find it.

Repro steps (I am running on Win10 1903):

  1. Add a virtualization root.
  2. Go into Explorer and try to delete the folder.

Expected results: I had expected ProjFS to support root deletion, specifically sending a PRJ_NOTIFICATION_PRE_DELETE for the root context so I can shut down my virtualization provider and return success or optionally block the deletion using an error code.
Actual results: Other than one directory enumeration callback, I don't receive a callback in my provider and Explorer shows an "unexpected error" dialog showing error code 0x80070032

This is important for my project because I am working on using ProjFS to virtualize my company's build output. I want to developers to be able to delete the root virtualization directory to do a completely clean build, but that doesn't work right now. Their workaround would be to run "rm -rf build/*" but that is confusing.

PreConvertToFull handler not notified

I tried to hook up a handler to get PreConvertToFull notification. which is not a member of RequiredCallback. I traced it down to the source level and the code seems to handle the request correctly (registration). However, the .NET handler was not notified at all when I tried to hydrate. Could someone take a look at this? Thank you,

Revert full state file keeping it hydrated

Is there a way to revert a file from Full state to a Hydrated Placeholder state?
Explaining: When open and edit a file for the first time, the PreConvertToFull callback is called, where I can control if the file is available for "Checkout" and subsequent allow user to edit the content or not. When user submit changes to the provider store, I want to mark the file as "synchronized" with the provider. This allows the PreConvertToFull callback to be called again in next time.
I tried using the UpdateFileIfNeeded for this, and to some extent that's what I need, however it discards the content, causing a hydrate to be called later. This is a disadvantage when I have to deal with really big files, I would like to avoid hydrate as much as possible, keeping the copy up-to-date in the user's location whenever possible.

Question: How to enable the rename of directories with content?

We tried the rename operation for directories in the windows mirror file system.

  • The rename of an empty directory works.
  • But the rename of a directory with content fails with the following error:
    Error 0x80070032: The request is not supported

Any help is welcome!

Thanks!

Is there a way to clean up ProjFS placeholders/markup without deleting everything?

Hello,

I need to support a user converting a directory back to not being virtual. Ideally this would go back to letting them manipulate the contents under that directory like normal (creating new files, etc).

The main reason for wanting this, versus just telling them to delete the virtual root, is that we want to preserve any local (Full) files they have under the root, and enable them to then work under that same path like normal.

I can write a utility to, after virtualization is disabled, go through an delete any non-Full files, that is easy.

Directories are the tricky bit.

I have found that I can convert a hydrated placeholder directory back to a normal looking/acting directory with DeviceIoCtrl(FSCTL_DELETE_REPARSE_POINT).

If it is just a NON-hydrated placeholder, though, this mostly works but leaves the RECALL_ON_DATA_ACCESS attribute, which I can't seem to clear (I see a note on MSDN about only being able to set this from kernel mode, which is presumably why).

That does allow me to create files in the directory once again, so seems like it gives me pretty much what I want...

So, I'm wondering:

  1. Is this method of de-virtualizing directories using FSCTL_DELETE_REPARSE_POINT okay to do (not leaving anything in a bad state, etc)?
  2. Is there a better way?
  3. Is there a way to get rid of the RECALL_ON_DATA_ACCESS attribute, and/or does it matter?

Thanks!

Determine "Who" performed an action.

Is it possible to have ProjFS return the username/SID of the user that deleted/created/updated a file?

Our scenario see's ProjFS running in folder under a shared location. This works well, but without implementing a FileSystem watcher we cannot determine who did what.

We can determine the creator of a new file by looking at the created file attributes, but im not aware that we can lookup who modified it.

Section on BadImageFormatExceptions also applies to FileNotFoundExceptions

I've recently had some trouble loading the ProjFS dll, and was experiencing a System.IO.FileNotFoundException trying to find the dll (in Visual Studio 2022). Following the instructions in the README for dealing with BadImageFormatExceptions (i.e., adding the UseIJWHost tag to my project) worked for me, even though I had never seen a BadImageFormatException. Perhaps the README could be updated to indicate that these steps can also help with FileNotFoundExceptions.

Is there a way to retrieve the ProviderId and ContentId from a placeholder?

Hello,

It would be helpful when debugging if I were able to get at this information ... is there a way to do so?

Also, similarly, it would be really nice if UpdateFileIfNeeded() could return a result that indicates if the file was actually updated (because providerId and/or contentId are different) or not. It currently appears to return "Ok" in both cases.

Thanks!

Handle 8.3 paths does not work

How to reproduce from console:

rem create file "VeryLongFileName.txt" in directory "C:\Test\Source\SomethingVeryLong\AnotherLongDirectoryName_1.0.0_Beta1.10\data" 
echo blah>C:\Test\Source\SomethingVeryLong\AnotherLongDirectoryName_1.0.0_Beta1.10\data\VeryLongFileName.txt
rem start projection
SimpleProviderManaged.exe --sourceroot C:\Test\Source\SomethingVeryLong\AnotherLongDirectoryName_1.0.0_Beta1.10\data --virtroot C:\Test\SomethingVeryLong\AnotherLongDirectoryName_1.0.0_Beta1.10\data

from another console:
rmdir /s /q C:\Test\SomethingVeryLong

expected result: directory "C:\Test\SomethingVeryLong" is removed
actual result: C:\Test\SomethingVeryLong\ANOTHE~1.10\data\VeryLongFileName.txt - The provider that supports file system virtualization is temporarily unavailable.

image

DirectoryEnumerationResults.Add symlink override is incompatible with EnumerateFileSystemInfos on .NET Framework

Context:
Symlink support is being added in #73. The DirectoryEnumerationResults.Add method calls PrjFillDirEntryBuffer2 native API method. For certain environment/symlink combinations, such as ones in these tests, PrjFillDirEntryBuffer2 returns ERROR_INSUFFICIENT_BUFFER for the first enumeration element.

It is expected of the provider to return ERROR_INSUFFICIENT_BUFFER as a callback result if the error happened on the first enumeration element. SimpleManagedProvider is an example of a provider doing this according to the guidelines.

Issue:
ERROR_INSUFFICIENT_BUFFER result coming from an enumeration callback is handled correctly in .NET Core, but not .NET Framework.

• Framework is using FindFirstFile()/FindNextFile(). There is no handling in the enumerator for buffer size results, in fact those functions’ doc pages do not discuss INSUFFICIENT_BUFFER as a possible result.
• NetCore is using NtQueryDirectoryFile() and related APIs, using resizing buffers.

.NET Framework:
directoryinfo.cs (microsoft.com)
filesystemenumerable.cs (microsoft.com)
.NET Core:
FileSystemEnumerator.Windows: runtime/FileSystemEnumerator.Windows.cs· dotnet/runtime (github.com)
FILE_FULL_DIR_INFORMATION derived from calling ntdll.dll : runtime/Interop.FILE_FULL_DIR_INFORMATION.cs dotnet/runtime (github.com)

Workaround:
.NET Framework 4.8 clients should not use the Add override with the symlink target.
If the symlink support is needed, .NET Core runtime could be used instead.

Test issue

There's a webhook that was automatically created for "issue comments". I want to see what it does.

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.