We should consider renaming them to make their intent clearer.

If a host doesn't support GetShareUrl the GetSharingUrlForFileWithUnknownType and GetSharingUrlForContainerWithUnknownType tests still run.

We should add a pre-req test that checks for the presence of the SupportedShareUrlTypes property in the CFI response (not a particular value in that property). If it’s not there at all, skip the test. That would fix the current bug.

The test case descriptions in the definition XML can have leading/trailing whitespace; that should be trimmed when importing the test cases from the XML.

They're defined as enums, but should arrays of enums.

ProofKeys section of tests fail because my server not receive the X-WOPI-Proof and X-WOPI-ProofOld HTTP headers.

Step for reproduce

1. clone this repo and compiled into exe with Visual Studio 2015.

2. run this command

Microsoft.Office.WopiValidator.exe -t MYTOKEN -l 0 -w http://localhost:3000/api/wopi/files/5d8488e930a07d2218d89d35  -e OfficeOnline -s -g ProofKeys

Test ProofKeys fails.

These are the received headers (only 2):

Authorization: Bearer eyJvIjoiQUU5RENGMTQ0MkNDMkUyM0ZBRUE2Mjk1MTg4NzI3QzgiLCJlIjoiRHROT2NHQ2o4NFM1TVhMVkU4VVlqeVNoWmp1cHFtbGMiLCJpIjoiNWQ4NDg4ZTkzMGEwN2QyMjE4ZDg5ZDM1In0=
Host: localhost:3000

no X-WOPI-Proof and X-WOPI-ProofOld headers.

What i'm doing wrong?

Copied from microsoft/Office-Online-Test-Tools-and-Documentation#359

This will enable partners to test and verify their request logging/debugging telemetry using the validator. As it is, partners only get those headers when testing with the apps.

For example, in the ProofKeys.CurrentValidSignedWithOldKey.OldInvalid test, the X-WOPI-ProofOld header is set to an invalid value, but the debugging info for the ProofOld value does not reflect this.

More information: microsoft/Office-Online-Test-Tools-and-Documentation#242

The host SHOULD provide a hash if they use chunked transfer to prevent a class of bugs where we think we got the whole file but didn't.

If hosts return a BaseFileName value of MyTestFile.WOPITEST, the pre-req test will fail because the EndsWith comparison is case-sensitive.

When integrating this tool into a CI system it would be great if this tool would return different exit codes depending on the state of the test cases which have been executed.

Would that be possible? THX

There is minimal logging support in the validator, but it is a custom interface (though it's simple). Integration with Extensions.Logging would enable package consumers to wire up their own log system, too.

I have a read-only WOPI host implementation. The CheckFileInfo endpoint returns a FileUrl, because it's possible to access the file without calling the GetFile endpoint. Because this property is present, the FileUrlUsage test group is executed, and it fails because it attempts to use Lock and PutFile, which my host doesn't implement.

The pre-requisites for FileUrlUsage should include SupportsUpdate and SupportsLock if it's going to use them.

Also, I think the test suite should include a read-only scenario for FileUrl.

The WOPI validator doesn't support session context at the moment. It should. :)

The problem is that we sign the requests in the validator using the wall-clock time, not the X-WOPI-Timestamp header we actually send in the request.

This means that when a host gets the request, it is not signed properly, because it was signed with a different Timestamp than the value on the request. So the host correctly returns a 500, but for the wrong reason. They should be returning a 500 because the X-WOPI-Timestamp value is older than 20 minutes, but they might also return a 500 simply because the proofs don't match due to the timestamp discrepancy.

The fix would be to ensure we're using the timestamp value that we use on the request when actually signing the request. That will ensure that when hosts check the proof, the signature will be correct, and if they still pass the test, it implies they're not checking the timestamp per the spec.

As of now I'm using the file's modification time and the file's content hash as version.

I think this is a quite reasonable implementation ...

But the tests are sometimes failing because the test tool puts identical content in consecutive calls.
https://github.com/Microsoft/wopi-validator-core/blob/e0ede96728f92b011c6c622c6945a265d3403b04/src/WopiValidator.Core/ResourceManagement/Resource.cs#L30

I'd propose to transmit some random generated file content.

Any objections? THX

Ideally this test would check two things:

1. That the SHA256 value was base64-encoded
2. That it matches the hash of the file retrieved via GetFile

Hi
How do I test against OOS.
What do i put into the Discovery
Does the wopitest doc have to include anything in particular.

Thanks

As described in microsoft/Office-Online-Test-Tools-and-Documentation#154, access tokens that are very long can cause problems with Office Online. A good starting length for the validation is 1000 characters I think.

The containers.EnumerateChildrenWithOneFilter and containers.EnumerateChildrenWithMultipleFilters tests don't actually verify that the filtering is done by the server. If that can't be done in the validator, it doesn't make much sense to have the tests, since a host who's not implemented filtering at all will still pass those tests.

The proof key tests can’t work as-is because there’s no signing key. We can disable the tests, but I think a better fix would be to provide a dummy WOPI discovery XML file in this repo and expose a key there. It could be a fixed key since it’s only used for testing.

Partners sometimes implement their WOPI endpoints incorrectly, and don't follow the URL schemes defined in the spec. For example, they provide a WopiSrc value like:

https://wopi.contoso.com/wopi/index/fileID/1212121

This will fail to boot the WAC apps because the URL doesn't include /files/. The validator doesn't currently check for this. It should.

System.ArgumentNullException: String reference not set to an instance of a String.
Parameter name: s
   at System.Text.Encoding.GetBytes(String s)
   at WopiValidator.Core.ProofKeysHelper.GetProofData(ProofKeyInput proofData)
   at WopiValidator.Core.ProofKeysHelper.GetSignedProofData(ProofKeyInput proofData, RSACryptoServiceProvider rsaAlg)
   at WopiValidator.Core.Requests.WopiRequest.GetProofKeyHeaders(String accessToken, Uri endpointUri, RSACryptoServiceProvider proofKeyProviderNew, RSACryptoServiceProvider proofKeyProviderOld)
   at WopiValidator.Core.Requests.WopiRequest.Execute(String endpointAddress, String accessToken, Int64 accessTokenTtl, ITestCase testCase, Dictionary`2 savedState, IResourceManager resourceManager, String userAgent, RSACryptoServiceProvider proofKeyProviderNew, RSACryptoServiceProvider proofKeyProviderOld)
   at WopiValidator.Core.TestCaseExecutor.ExecuteTestCase(ITestCase testCase)
   at Microsoft.Office.Web.WopiTestService.ExecuteTestMethod.ProcessRequest(HttpContext context)

Notes from chrisbro-MSFT:

I bet the Url property of the ContainerInfo objects returned by /files/EnumerateAncestors does not include an AccessToken. There are only two calls to GetBytes in ProofKeyHelper.GetProofData() and one of them is on a field that couldn’t be null or we would have crashed already. So it has to be access token. And that test is defined like so:

<Requests>
       <EnumerateAncestors>
              <SaveState>
                     <State Name="ParentContainerUrl" Source="AncestorsWithRootFirst[-1:].Url" />
              </SaveState>
       </EnumerateAncestors>
       <CheckContainer OverrideUrl="$State:ParentContainerUrl">

Have seen this a couple of times. Perhaps we should add a try..catch around WopiRequest.Execute() and have it return an IResponseData object that indicates the error. Test would still fail but it will fail properly and the request info can be seen in the inspector tabs.

Some tests use the foox or wopitestfoox file extension when creating new files. These should be changed to use file extensions like docx or pptx. The files created are cleaned up so it's not a big deal that the files' extensions won't match their type.

These tests should at least check the following:

• PutUserInfo shouldn't return a 409 (rather, it should only return the status codes outlined in the docs)
• PutUserInfo value is returned in subsequent Check calls

An empty array is a valid AncestorsWithRootFirst response for containers, when the EnumerateAncestors call is made on the root container.

This would help people better understand what tests are doing.

Related to #64.

Most importantly, that the optional HostEditUrl is provided.

An increasing number of implementers are returning JSON responses for operations that don't have a response body defined in WOPI. The validator should verify that responses that should be empty actually are empty.

Examples include the EnumerateChildren and EnumerateAncestors operations, both of which return URLs with access_tokens attached as a query param. The validator doesn't check that the access token is included in the URL as it should be.

While HTTP is permitted in the test environment, it is required in production. Having a validator test that fails when HTTP is used will help draw attention to this requirement.

The RenameFile test cases do not currently check that the renamed file has the correct file extension. This is because the RenameFile request takes the filename without the extension, and the response also returns the filename without the extension.

To address this, we need to add a new test that also calls CheckFileInfo and verifies that the BaseFileName returned matches what we expect.

Related to microsoft/Office-Online-Test-Tools-and-Documentation#313

These values should be different, because one is expected to open the file in view mode, and the other in edit mode.

This applies to all of the Host*Url properties, actually.

Affected tests that need the UserCanRename pre-requisite:

• containers.CreateAndRenameChildContainer
• containers.CreateAndRenameChildContainerAndVerifyReturnedContainerNameIsCorrectlyEncoded

Except in the unlocked 0-byte PutFile case, of course. This causes issues for some hosts. We can address this by making the files we Put as part of the validator > 0 bytes.

The X-WOPI-Proof and X-WOPI-ProofOld headers are never sent by the validator. How can we validate the requests without them? The ProofKeys tests are meaningless if they don't send those headers...

While this field isn't required in the protocol, we do require it for any host that supports co-authoring (which should be all of them).

Alternatively we could add a new test or test group rather than changing the existing FullCheckFileInfoSchema test.

I think commit 18d3894 was incomplete. Changes are needed to save state properly from Unlock requests (I think).

Related to microsoft/Office-Online-Test-Tools-and-Documentation#328

The test PutRelativeFile.IncludeHostUrls fails while no values are provided for HostEditUrl or HostViewUrl.

Those values are supposed to be optional therefore this test should not fail if no values are provided.
See: https://wopi.readthedocs.io/projects/wopirest/en/latest/files/PutRelativeFile.html#optional-response-properties
Either the specs are wrong or the test.

  Fail: PutRelativeFile.IncludeHostUrls
    PutRelativeFile, response code: 200 OK
      Incorrect value for 'HostEditUrl' property. Value is required but not provided.
      Incorrect value for 'HostViewUrl' property. Value is required but not provided.
    DeleteFile, response code: 0 0

The RenameFile tests currently require that the host implements either PutRelativeFile or CreateChildFile, because the test creates new files to rename, then deletes them.

We should consider changing the tests or adding new tests so that they don't rely on creating new files. This may have some unwanted side effects, so we should explore more.

We have additional logic at backend checking licence by additional parameters in URL. This definitely working Ok with Office Server before. But to implement Office online integration we should validate our's endpoint with wopi-core-validator. And it failing on some tests.
Actual problem:
Url parsing in methods BuildWopiUri and GetEndpointAddressOverride not always working correctly. If Url will already contain ? with parameters Validator will add /contents in the end of URI string and will break it.
Example:
Executing command: Microsoft.Office.Validator.exe ... -w http://localhost:5000/wopi/files/1111?customerId=123
Will create a request to next url: http://localhost:5000/wopi/files/1111?customerId=123/contents
Solution:
Check in both methods if URL already contains ? than insert PathOverride before ? else add to the end.

The docs mention UserPrincipalName in the properties of the CheckFileInfo response, but the validator rejects it as an unknown property:

Fail: FullCheckFileInfoSchema
CheckFileInfo, response code: 200 OK
Unknown Properties: UserPrincipalName
Re-run command: .\wopivalidator.exe -n FullCheckFileInfoSchema -w http://localhost:13958/api/tenant/foobar/wopi/files/d3301821-fddc-4751-b49f-5396cd6c6d6e -t eyJhbGciOiJSUzI1NiIsImtpZCI6IjA1NTU0MDM1QTk2MjA4NjMzMjczNEEyM0VDNTUzQzExRjFENDlGRTMiLCJ4NXQiOiJCVlZBTmFsaUNHTXljMG9qN0ZVOEVmSFVuLU0iLCJ0eXAiOiJKV1QifQ.eyJzdWIiOiI3OGIzMjc2ZS1iNWZlLTQzMjctYTE2Ni0wYjViNjUxNjU0ZTQiLCJmaWxlSWQiOiJkMzMwMTgyMS1mZGRjLTQ3NTEtYjQ5Zi01Mzk2Y2Q2YzZkNmUiLCJwZXJtaXNzaW9uIjoiUmVhZCIsIm5iZiI6MTU3MjQ1MzU3MywiZXhwIjoxNTcyNDU3MTczLCJpYXQiOjE1NzI0NTM1NzN9.B_8njZ8s9e3H_UYfJTmK97HSU_JiyUB0A3R-4QAxhjqhMy-r10LtoFE8Ao3LB3KnqXW8x0StAzIX_PEHFmcEMaxKMYVf9t_32F1Ho4gdz_AMfSB65DxxHiOGKWz-WCR1g3cyqiIc88aln9bgFTIsB5mW9suB9W-aTaiEuSAl6TeyBmkZ5dysVrFIvSblHYa-iZ8bUN7HRTnBS3-ky0FdtuZA4sCxbWNrO_K10kc6aSQU9ROL7MQlfLMpJMkBgJ7yqrBv3DcxzlMqRN--DAPNbRybVpBZfkI5-zdAgeT4mHAlsNRlZ_0HMiXC-vThPFMM-qwS9BUIX30WWVW1azdncw -l 1572457173870

Which is correct, the docs or the validator?