kevindockx / httpcacheheaders Goto Github PK

(cfr course comment: "However, when I tried following your instructions in the Dealing with Concurrency demo, after performing the PUT operation, it failed to update the ETag and retained the same ETag as in the two GET Operations.") => so the ETag isn't updated, which makes sense b/c MS's middleware doesn't generate ETags.

=> works when not using the response caching cache store middleware
=> works when using the response caching cache store middleware & sending a no-cache (so the store is bypassed)
=> doesn't work when using the middleware combined with marvin.cacheheaders: ETag isn't updated.

MS's cache store doesn't generate ETags, but it seems like it doesn't pass through the request either... should such requests always be sent with no-cache? Is this an issue with MS's cache store? Is it an issue with this middleware? To investigate.

I have just upgraded a fork to 2.1.1. It wasn't straightforward. Included is the diff.

It now builds against

$ dotnet --version
2.1.301

Some gotchas:

• Renamed package: Microsoft.AspNetCore.All --> Microsoft.AspNetCore.App
• Drop version for above package, except in the Test project (I can't explain why)
• Manually change Target Frameworks through text editing file from 2.0 --> 2.1

Index: test/Marvin.Cache.Headers.Test/Marvin.Cache.Headers.Test.csproj
IDEA additional info:
Subsystem: com.intellij.openapi.diff.impl.patch.CharsetEP
<+>UTF-8
===================================================================
--- test/Marvin.Cache.Headers.Test/Marvin.Cache.Headers.Test.csproj	(revision 99b980343d3d2b54b29208f8f54ab5b8e5a3f976)
+++ test/Marvin.Cache.Headers.Test/Marvin.Cache.Headers.Test.csproj	(date 1529885235000)
@@ -1,28 +1,25 @@
 <Project Sdk="Microsoft.NET.Sdk">
-
   <PropertyGroup>
-    <TargetFramework>netcoreapp2.0</TargetFramework>
+    <TargetFramework>netcoreapp2.1</TargetFramework>
     <AssemblyName>Marvin.Cache.Headers.Test</AssemblyName>
     <PackageId>Marvin.Cache.Headers.Test</PackageId>
     <GenerateRuntimeConfigurationFiles>false</GenerateRuntimeConfigurationFiles>
   </PropertyGroup>
-
   <ItemGroup>
-    <PackageReference Include="Microsoft.AspNetCore.All" Version="2.0.0"></PackageReference>
-    <PackageReference Include="Microsoft.AspNetCore.TestHost" Version="2.0.0" />
-    <PackageReference Include="Microsoft.NET.Test.Sdk" Version="15.3.0" />
+    <PackageReference Include="Microsoft.AspNetCore.App"  Version="2.1.1">
+    </PackageReference>
+    <PackageReference Include="Microsoft.AspNetCore.TestHost" Version="2.1.1" />
+    <PackageReference Include="Microsoft.AspNetCore.Hosting" Version="2.1.1" />
+    <PackageReference Include="Microsoft.NET.Test.Sdk" Version="15.7.2" />
     <PackageReference Include="xunit.runner.visualstudio" Version="2.3.1" />
-    <PackageReference Include="Moq" Version="4.7.142" />
+    <PackageReference Include="Moq" Version="4.8.3" />
     <PackageReference Include="xunit" Version="2.3.1" />
   </ItemGroup>
-
   <ItemGroup>
     <Folder Include="Properties\" />
   </ItemGroup>
-
   <ItemGroup>
     <ProjectReference Include="..\..\sample\Marvin.Cache.Headers.Sample\Marvin.Cache.Headers.Sample.csproj" />
     <ProjectReference Include="..\..\src\Marvin.Cache.Headers\Marvin.Cache.Headers.csproj" />
   </ItemGroup>
-
-</Project>
+</Project>
\ No newline at end of file
Index: sample/Marvin.Cache.Headers.Sample/Marvin.Cache.Headers.Sample.csproj
IDEA additional info:
Subsystem: com.intellij.openapi.diff.impl.patch.CharsetEP
<+>UTF-8
===================================================================
--- sample/Marvin.Cache.Headers.Sample/Marvin.Cache.Headers.Sample.csproj	(revision 99b980343d3d2b54b29208f8f54ab5b8e5a3f976)
+++ sample/Marvin.Cache.Headers.Sample/Marvin.Cache.Headers.Sample.csproj	(date 1529884438000)
@@ -1,21 +1,17 @@
 <Project Sdk="Microsoft.NET.Sdk.Web">
-
   <PropertyGroup>
-    <TargetFramework>netcoreapp2.0</TargetFramework>
+    <TargetFramework>netcoreapp2.1</TargetFramework>
   </PropertyGroup>
-
   <ItemGroup>
     <None Update="wwwroot\**\*">
       <CopyToPublishDirectory>PreserveNewest</CopyToPublishDirectory>
     </None>
   </ItemGroup>
-
   <ItemGroup>
-    <PackageReference Include="Microsoft.AspNetCore.All" Version="2.0.0"></PackageReference>
+    <PackageReference Include="Microsoft.AspNetCore.App">
+    </PackageReference>
   </ItemGroup>
-
   <ItemGroup>
     <ProjectReference Include="..\..\src\Marvin.Cache.Headers\Marvin.Cache.Headers.csproj" />
   </ItemGroup>
-
-</Project>
+</Project>
\ No newline at end of file
Index: src/Marvin.Cache.Headers/Marvin.Cache.Headers.csproj
IDEA additional info:
Subsystem: com.intellij.openapi.diff.impl.patch.CharsetEP
<+>UTF-8
===================================================================
--- src/Marvin.Cache.Headers/Marvin.Cache.Headers.csproj	(revision 99b980343d3d2b54b29208f8f54ab5b8e5a3f976)
+++ src/Marvin.Cache.Headers/Marvin.Cache.Headers.csproj	(date 1529882703000)
@@ -1,25 +1,22 @@
 <Project Sdk="Microsoft.NET.Sdk">
-
   <PropertyGroup>
-    <TargetFramework>netstandard1.6</TargetFramework>
+    <TargetFramework>netstandard2.0</TargetFramework>
     <AssemblyName>Marvin.Cache.Headers</AssemblyName>
     <PackageId>Marvin.Cache.Headers</PackageId>
-    <NetStandardImplicitPackageVersion>1.6.0</NetStandardImplicitPackageVersion>
+    <NetStandardImplicitPackageVersion>2.0.3</NetStandardImplicitPackageVersion>
     <PackageTargetFallback>$(PackageTargetFallback);dnxcore50</PackageTargetFallback>
     <GenerateAssemblyConfigurationAttribute>false</GenerateAssemblyConfigurationAttribute>
     <GenerateAssemblyCompanyAttribute>false</GenerateAssemblyCompanyAttribute>
     <GenerateAssemblyProductAttribute>false</GenerateAssemblyProductAttribute>
-    <Version>1.1.0</Version>
+    <Version>2.1.1</Version>
     <Description>ASP.NET Core middleware that adds HttpCache headers to responses (Cache-Control, Expires, ETag, Last-Modified), and implements cache expiration &amp; validation models.</Description>
   </PropertyGroup>
-
   <ItemGroup>
-    <PackageReference Include="Microsoft.AspNetCore.Http.Abstractions" Version="1.0.2" />
-    <PackageReference Include="Microsoft.AspNetCore.Mvc.Abstractions" Version="1.0.2" />
-    <PackageReference Include="Microsoft.Extensions.DependencyInjection.Abstractions" Version="1.0.2" />
-    <PackageReference Include="Microsoft.Net.Http.Headers" Version="1.0.2" />
-    <PackageReference Include="Microsoft.Extensions.Logging.Abstractions" Version="1.0.2" />
-    <PackageReference Include="Microsoft.Extensions.Options" Version="1.0.2" />
+    <PackageReference Include="Microsoft.AspNetCore.Http.Abstractions" Version="2.1.1" />
+    <PackageReference Include="Microsoft.AspNetCore.Mvc.Abstractions" Version="2.1.1" />
+    <PackageReference Include="Microsoft.Extensions.DependencyInjection.Abstractions" Version="2.1.1" />
+    <PackageReference Include="Microsoft.Net.Http.Headers" Version="2.1.1" />
+    <PackageReference Include="Microsoft.Extensions.Logging.Abstractions" Version="2.1.1" />
+    <PackageReference Include="Microsoft.Extensions.Options" Version="2.1.1" />
   </ItemGroup>
-
-</Project>
+</Project>
\ No newline at end of file

When i try to access documentation for my API created with swagger while HttpCacheHeaders is added in the HTTP request pipeline, the server returns a 500. When HttpCacheHeaders is disabled, it works fine.

ContactManager-20170529.txt

... so profiles can be applied to controllers/actions (instead of on a global level)

Currently, we only try to parse to/from RFC1123 pattern (https://msdn.microsoft.com/en-us/library/az4se3k1(v=vs.110).aspx).

Should we allow multiple formats? Cfr https://msdn.microsoft.com/en-us/library/bb301210(v=vs.110).aspx

The If-None-Match request-header field is used with a method to make it conditional. A client that has one or more entities previously obtained from the resource can verify that none of those entities is current by including a list of their associated entity tags in the If-None-Match header field. The purpose of this feature is to allow efficient updates of cached information with a minimum amount of transaction overhead. It is also used to prevent a method (e.g. PUT) from inadvertently modifying an existing resource when the client believes that the resource does not exist.

Examples:
If-None-Match: "xyzzy"
If-None-Match: W/"xyzzy"
If-None-Match: "xyzzy", "r2d2xxxx", "c3piozzzz"
If-None-Match: W/"xyzzy", W/"r2d2xxxx", W/"c3piozzzz"
If-None-Match: *

Cfr: https://tools.ietf.org/html/rfc2616#page-132

From the standard:

The If-Unmodified-Since request-header field is used with a method to make it conditional. If the requested resource has not been modified since the time specified in this field, the server SHOULD perform the requested operation as if the If-Unmodified-Since header were not present.

If the requested variant has been modified since the specified time, the server MUST NOT perform the requested operation, and MUST return a 412 (Precondition Failed).

  If-Unmodified-Since = "If-Unmodified-Since" ":" HTTP-date

An example of the field is:

   If-Unmodified-Since: Sat, 29 Oct 1994 19:43:31 GMT

If the request normally (i.e., without the If-Unmodified-Since header) would result in anything other than a 2xx or 412 status, the If-Unmodified-Since header SHOULD be ignored.

If the specified date is invalid, the header is ignored.

The result of a request having both an If-Unmodified-Since header field and either an If-None-Match or an If-Modified-Since header fields is undefined by this specification.

To check: shouldn't this only be used if the server can ensure those dates are strong validators? Might need additional design so users can signify this.

Currently when generating a key the path is taken into account, but not the query string. Take query string into account when generating the cache key.

https://github.com/KevinDockx/HttpCacheHeaders/blob/master/src/Marvin.Cache.Headers/HttpCacheHeadersMiddleware.cs#L215-L216 => should be (httpContext.Request.Method != HttpMethod.Get.ToString() && httpContext.Request.Method != HttpMethod.Head.ToString())

Currently, 304 responses & 412 responses take the ETag headers that are included in the request. This results in a possible list of ETags when, for example, an If-None-Match header with multiple ETags is submitted. Should change to a calculated ETag, or maybe the current one from the stored response (check standard to be sure).

Allow passing through general options object to turn off expiration and/or validations models. In ref to #60 & #5

A request intended to update a resource (e.g., a PUT) MAY include an If-Match header field to signal that the request method MUST NOT be applied if the entity corresponding to the If-Match value (a single entity tag) is no longer a representation of that resource. This allows the user to indicate that they do not wish the request to be successful if the resource has been changed without their knowledge.

Examples:
If-Match: "xyzzy"
If-Match: "xyzzy", "r2d2xxxx", "c3piozzzz"
If-Match: *

Cfr: https://tools.ietf.org/html/rfc2616#page-128

The meaning of "If-Match: *" is that the method SHOULD be performed if the representation selected by the origin server (or by a cache, possibly using the Vary mechanism) exists, and MUST NOT be performed if the representation does not exist.

Cfr: https://tools.ietf.org/html/rfc2616#page-128

Hi there,

I found that if I have an error in my application and I use the following order in startup:

app.UseDeveloperExceptionPage(); 
app.UseHttpCacheHeaders();
app.UseMvc();

I get a blank page instead of the exception details in the page.

If I use the following order then everything works correctly:

app.UseHttpCacheHeaders();
app.UseDeveloperExceptionPage(); 
app.UseMvc();

Could you please clarify the ordering needed, or perhaps your middleware needs to be bypassed when an error occurs?

Allow services.AddHttpCacheHeaders() for full default options.

Although the ETag header is included in the response, it cannot be accessed thru JavaScript when hosted from a different domain (foreign origin). According to a post in StackOverflow located at https://stackoverflow.com/a/25673446/4997224, there is a need to include the Access-Control-Expose-Headers to give explicit permissions for the client to read headers like the ETag header.

Here's what I did in my local copy of the HttpCacheHeaders. I added the last line of code after the code that sets/adds the ETag header and it seems to solve my problem. Unfortunately, the Access-Control-Expose-Headers string is not defined in the HeaderNames class so I had to hard-code it here.

// set the ETag header
headers[HeaderNames.ETag] = eTag.Value;
// expose ETag header over CORS
headers["Access-Control-Expose-Headers"] = "etag";

It would be nice if we can make this configurable.

https://github.com/KevinDockx/HttpCacheHeaders/blob/master/src/Marvin.Cache.Headers/HttpCacheHeadersMiddleware.cs#L610

Before doing #53 I would like to create which changes the string from the key to a type containing properties for the separate parts (and a custom ToString which returns the same as today)

This gives a bit more flexibility in future changes

Firstly great effort with the project.

I was wondering how would you handle the situation where you would like to cache non authenticated pages, but if a person logins then don't cache the page?

Is a validation attribute the way to go?

Add an example with a shared cache store. Ideal candidate (safe for the naming...) seems to be Microsoft's Response Caching middleware. However, at this time the current version doesn't look stable enough (no-cache issues, cache invalidation issues, revalidation issues, ...). Once it's finished (1.2.0?), add a sample so responses actually get cached :)

Currently: OR, should be AND

The meaning of "If-None-Match: *" is that the method MUST NOT be performed if the representation selected by the origin server (or by a cache, possibly using the Vary mechanism) exists, and SHOULD be performed if the representation does not exist. This feature is intended to be useful in preventing races between PUT operations.

Cfr: https://tools.ietf.org/html/rfc2616#page-132

https://github.com/KevinDockx/HttpCacheHeaders/blob/master/src/Marvin.Cache.Headers/HttpCacheHeadersMiddleware.cs#L85

I am trying to understand, why are we updating the Last Modified in the store? Shouldn't we let the store decide this?

Given an empty store and an initial request:

• Checks occur for caching
• None match, request goes as normal, response is built
• ETag is calculated and last modified is UtcNow
• This gets stored in the store and the response is sent

Given previous with a request for the same

• Checks for cache occur
• Found in store
• Update Last Modified in store
• Send back 302 not modified with a different last modified

Why are we doing the update? Image I have a redis store, and I want the last modified date be updated by another process, which knows a lot better when the object itself has changed (smarter cache invalidation), this would get overwritten now. Does that make sense?

I have resources that I know the last modified (which is not the time of generation) and I want to return this.

https://developer.mozilla.org/en-US/docs/Web/HTTP/Headers/Last-Modified

The Last-Modified response HTTP header contains the date and time at which the origin server believes the resource was last modified. It is used as a validator to determine if a resource received or stored is the same. Less accurate than an ETag header, it is a fallback mechanism. Conditional requests containing If-Modified-Since or If-Unmodified-Since headers make use of this field.

https://github.com/KevinDockx/HttpCacheHeaders/blob/master/src/Marvin.Cache.Headers/HttpCacheHeadersMiddleware.cs#L610

I would like to create a PR which allows a user to pass in his own method to create the key (which is later passed to the Store).

This would allow people to decide which values they like to take into account (and possibly modify).

Given the following exception handler
app.UseExceptionHandler(builder => { builder.Run(async context => { context.Response.StatusCode = 500; await context.Response.WriteAsync("An unexpected fault happened. Try again later"); }); });

If I'm using app.UseHttpCacheHeaders(); and i throw an un-handled exception I'll get the following exception
"exception": { "type": "ObjectDisposedException", "message": "Cannot access a closed Stream.", "method": "System.IO.__Error.StreamIsClosed()", "stacktrace": " at System.IO.__Error.StreamIsClosed()\r\n at System.IO.MemoryStream.Write(Byte[] buffer, Int32 offset, Int32 count)\r\n at System.IO.MemoryStream.WriteAsync(Byte[] buffer, Int32 offset, Int32 count, CancellationToken cancellationToken)\r\n--- End of stack trace from previous location where exception was thrown ---\r\n at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()\r\n at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)\r\n at System.Runtime.CompilerServices.TaskAwaiter.GetResult()\r\n at WebApiPattern.AspCore.Startup.<>c.<<Configure>b__1_1>d.MoveNext() in C:\\Workspace\\webapi_pattern\\WebApiPattern.AspCore\\Startup.cs:line 148\r\n--- End of stack trace from previous location where exception was thrown ---\r\n at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()\r\n at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)\r\n at Microsoft.AspNetCore.Diagnostics.ExceptionHandlerMiddleware.<Invoke>d__6.MoveNext()", "innerException": {}

Also I've been going through your PluralSight course and its great!

Hi,
this is what I get once I tryed to install the packages on my Asp.NET Core (Full .NET) project.

I'm using NLog to log exceptions and I've noticed that whenever a controller returns a No Content (204), the following is getting logged.

"exception": {
		"type": "InvalidOperationException",
		"message": "Write to non-body 204 response.",
		"method": "Microsoft.AspNetCore.Server.Kestrel.Internal.Http.Frame.HandleNonBodyResponseWrite()",
		"stacktrace": "   at Microsoft.AspNetCore.Server.Kestrel.Internal.Http.Frame.HandleNonBodyResponseWrite()\r\n   at Microsoft.AspNetCore.Server.Kestrel.Internal.Http.Frame.<WriteAsyncAwaited>d__183.MoveNext()\r\n--- End of stack trace from previous location where exception was thrown ---\r\n   at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()\r\n   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)\r\n   at Marvin.Cache.Headers.HttpCacheHeadersMiddleware.<Invoke>d__6.MoveNext()\r\n--- End of stack trace from previous location where exception was thrown ---\r\n   at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()\r\n   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)\r\n   at Microsoft.AspNetCore.Server.IISIntegration.IISMiddleware.<Invoke>d__8.MoveNext()\r\n--- End of stack trace from previous location where exception was thrown ---\r\n   at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()\r\n   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)\r\n   at Microsoft.AspNetCore.Hosting.Internal.RequestServicesContainerMiddleware.<Invoke>d__3.MoveNext()\r\n--- End of stack trace from previous location where exception was thrown ---\r\n   at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()\r\n   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)\r\n   at Microsoft.AspNetCore.Server.Kestrel.Internal.Http.Frame`1.<RequestProcessingAsync>d__2.MoveNext()",
		"innerException": {}

Commenting out the app.UseHttpCacheHeaders(); from Startup removes the error.

I've followed your course on Pluralsight and implemented this middleware in my application as follows:

services.AddHttpCacheHeaders( expirationModelOptions => { expirationModelOptions.MaxAge = 600; expirationModelOptions.SharedMaxAge = 300; }, validationModelOptions => { validationModelOptions.AddMustRevalidate = true; validationModelOptions.AddProxyRevalidate = true; });

However, I've got a sort, filter, pagination set up in my query string like: /api/objects?filter.name='a'
When I call the same url but with another querystring, for example to return all objects with a 'b' in their name : /api/objects/filter.name='b', I receive a '304 not modified'.

What can I do to get the urls with the filter query string revalidate if the query string changed?

This is useful when the server wants to input his own strong eTag-generating function. This use case is valid if the default function doesn't guarantee equality according to the server, or the other way around: if the generation of a strong eTag should not take into account the full response body. Eg: a modifiedby-field shouldn't be taken into account, even with different modifiedby-fields, responses are still regarded as equal In these cases, the user must provide his own strong eTag-generating functions.

Hello,

It would be nice to have an option to not generate ETag header for large files because ReadToEnd can be long in this case.