With or without the WebPages component installed, the Windows (NTLM) auth feature does not work (it is easier to test it with WebPages installed though).

• install sensenet Services and WebPages
• create a user in the repo for your domain account under a correct domain (sn, not builtin!)
• add a site in sensenet with a custom url with Windows auth defined on it
• register that url on your machine in the hosts file
• create an IIS site pointing to the web folder and register the url binding
• switch on Windows auth on the IIS site
• start the site in IIS and attach with VS

--> when the site starts to load, the browser pops up a login dialog (it should not), even if you are already logged in to Windows with your domain account. If you try to log in, it fails, the request arrives to SN, but without the authenticated user.

Possible causes:

• missing web.config entry?
• incorrect IIS configuration?
• something is wrong with our modules? (authenticate/portalcontext/authorize)

Related to #10

Create for CI that runs our tests and builds the latest codebase

Create an SN 6.5-compatible SnAdmin package for modifying task management settings for our internal sandbox builds.

If it can be added to SN7 easily, do that too.

If you have a workspace/doclib/file structure (e.g. /Root/ws/doclib/abc.docx) and try to access the file with wrong casing (e.g. /Root/WS/DocLib/abc.docx - note the uppercase letters), the result will be a plain html document that is displayed by the browser, instead of a downloaded file.

The incorrect response content type is text/html.

I followed /nuget/readme.md in the source code to try Sn7. And I download SnAdmin and compiled it and move it to SnAdminRuntime's output folder.
When I run the command: snadmin installnuget dataSource:. initialCatalog:sensenet, I got a Unhandled Exception:System.ApplicationException: Configurae the TargetDirectory.

• when referring to the product in source code (comments, messages, etc.), it should be sensenet ECM
• company name should remain Sense/Net

When something is wrong with the available types (e.g. there is a missing type or a wrong version of for example Newtonsoft.Json), SnAdmin (or SnAdminRuntime) fails to execute. The necessary information is displayed in the trace log, but not on screen (console window), because at that time even the logger module is not yet initialized.

We should handle type loader exceptions and display the LoaderException property (not InnerException!) on the console so that the user knows the error without starting DebugView.

When during AD sync a user needs to be removed from a group and the user has an invalid sync guid on the portal (e.g. not null, but empty string or contains an invalid guid string), the remove algorithm fails with an exception and prevents the tool from modifying the entire group - so the error affects not just that incorrect user content but the whole group.

Solution: protect the internals of that loop with try-catch and remove these kind of users from the group.

Tool: SyncAD2Portal
Method: UpdatePortalGroupProperties

There are a lots of warnings in the event log about a missing Portal settings file - which is there, but maybe a query does not return it.

SnAdmin packages have validation rules around package versions: for example you cannot install a product package that upgrades the product from version 1.2 to version 1.3 if the current installation is higher than version 1.2.

In case of application packages, the same version validation applies, but it is enforced per application. So currently only a single product is allowed in the system, but multiple applications (an application is intended to be a 3rd party project, developed by business solution developers).

Issue 1: versioning

Soon we will have multiple components (e.g. workflow or preview generation) built on the same SN core foundation and these components will have their own version history, with different version numbers. This will cause a problem when we release packages for these components: they are not product packages (because there can be only a single product version, and we also have to keep track of installed components), but not applications either.

Option 1

We introduce a new package type beside the currently existing Product and Application types: Component. These component packages should behave the same way as application packages, the only difference is that they are released as official components by Sense/Net.

Option 2

We allow multiple "products" in the system and enforce versioning validation rules based on product name - the same way as in case of applications.

Option 3

We remove the whole Product/Application feature: there will be only components (maybe even the core product can be treated as a component). In this case we could introduce a vendor/publisher info for these components that could distinguish 'official' Sense/Net packages from others'.

Issue 2: dependencies

We also have to take care of component dependencies (when a component can be installed only if another component is present). This is similar to NuGet package dependencies.

This can be solved by adding a dependency information to SnAdmin packages for these components. In the first stage it is enough to use this only for validation; later it can be used to acquire/resolve dependent packages automatically.

Epic for implementing Sensenet cloud compatibility. Should include Azure research and lots of component refactors.

Refactor the version info in the Content Explorer Root based on the new SnAdmin versioning.

Create components from the rest of the monolithic codebase.

The password generator demo package should be extended with the following functionality:

• save the generated passwords file into a secure list on sensenet.com (name: passwords-GUID.txt)
• save this link (url or content reference) to the appropriate sandbox content on sensenet.com so that administrators can access the demo site with those passwords later

Implement Json WebToken authentication in the core product

• Authorization should be in a separate module from the portal auth module
• Implement the token authentication inside the portal authentication module

Create a new component/package based on the content in TFS:

1. workspace content templates
2. default site / workspaces section, without the features that are published separately (e.g. workflows!)

• should be executable multiple times (see blocker issue #95 about executing Install packages multiple times)
• the second part is not imported by default, only if a switch (importdemo:true) is flipped
• lives in a separate repo here on github
• smart import: if the target sensenet ECM instance contains the WebPages component, the views and pages are imported. If not, they are ommitted. This requires a special conditional step that can check installed components.
• move the workspaces article here
• the outcome is a NuGet package
• update the sensenet-components.md doc with this new component in the main repo

Hi!
I could not filter a reference field's attribute e.g. I wanted to get all the workspaces, which was modified by a specific user:
"https://demo06.demo.sensenet.com/OData.svc/workspaces?metadata=no&$filter=ModifiedBy/Name%20eq%20%27Admin%27"

In a document management centric environment, it is often necessary for an end-user to view an audit trail of actions that occur with a specific document (created; modified; renamed; moved; deleted). This information is already stored in the JournalItems database table, but there appears to be no way to access it via the client APIs.

I'd like to request a feature to allow us to query a specific item and return an array of all rows from the JournalItems related.

Additionally, it would be useful for allow custom actions to be added to the JournalItems. For example, a desktop client app may want to add an entry for each time a document is printed or a document is emailed to an external user.

There are a couple of libraries currently in the main Services package that are ment to be referenced separately. For example a supporting 3rd party component does not need to (and should not) reference the full Services package, only a connector library, because in production the core libraries are not needed there (e.g. for a task executor or blob provider).

These libraries (that we should create a separate nuget package for) are the following:

• Preview
• Common
• Blob storage

The export tool help states that the default value of the source parameter is /Root. However if you execute the export without this parameter, it fails.

We should add /Root as the default value either in code (export step) or in the export package manifest (Admin\tools\export\manifest.xml)

• Azure SQL Database settings alteration
• Install database script sql authentication
• Don't forget to check if the settings are good for mssql as well
• Have another user install sensenet using azure sql
• Add Azure db info to the install from nuget docs

I would like to store my blobs in the Azure Blob Storage using the existing IBlobProvider interface.

• Importing into the blob storage works
• A 1 MB file is imported
• A 10 MB file is imported
• A 100 MB file is imported
• Azure blob storage documentation for sensenet ECM
• Create a benchmark environment for this as well and run the 50k benchmark scenario

I've started to install SensenetServices from nuget but I got en error, it says: Unable to find metadata of EntityFramework.6.1.2

If I install EF manually I got another error:
Unable to find metadata of SenseNet.SnAdmin.1.4.0

I'm using visual studio 17

Should I manually install the dependencies?

Port these articles from wiki, because they are referenced from new docs here on github:

• http://wiki.sensenet.com/Smart_Application_Model
• http://wiki.sensenet.com/OData_REST_API
• http://wiki.sensenet.com/Portlet
• http://wiki.sensenet.com/Webdav
• Update the links here in the install-sn-from-nuget.md and sensenet-components.md files.

When an Odata query can be translated to SQL query for faster execution the __count property always shows the invalid value. Actually it shows the filtered result count with skip, top because the SQL query cotains them and there is no second query for a count only execution.

Acceptance criteria

• Fix the bug in SN70Beta
• Fix the bug in SN 6.5.4 in a hotfix branch
• Report back in JIRA

Field controls

• Display Name: Has own control named "Display Name".
• Name: Has own control: "Name".
• Start date & Due date: DateTime type fields. The client is responsible for the UTC time format conversions to and from local time. (Can be set to show just the date or the whole date time)
• Assigned to: Reference field. Needs an own control specific to the User content type (it should inherit from the Reference grid control).
• Priority: Choice field, dropdown list.
• Status: Ditto.
• Completion: Number control - should be unified all over the place (we don't need currency or percentage controls, it should be derived from a parameter)
• Description: Wysiwyg editor.
• Modified by: Read-only reference field.
• Modification Date: DateTime field.
• Version: Read-only Choice field. (we may not need to display it)
• Index: Numer field - plain integer.

• buttons (save and cancel - not field controls)

this one is from @pietervanh as well:
I have some SyncTrees that use a Mapping that tries to use ADProperty(mail) and attach it to PortalProperty(Email)
Problem is that some users don't have ADProperty mail (or empty) but it tries to populate Email on portal anyways.
It should just not try to save Email on portal when ADProperty is empty.
It now generates an error (Cannot save Content Invalid Fields Email) and object is never updated.

This is really annoying cause I have many of these errors, and therefor I don't see the important sync errors anymore.

When I install the Services package (currently in the develop branch), it creates the db, but fails on the second sql script:

1. delete sn db if exists
2. install the SenseNet.Services nuget package into a new MVC app
3. execute the install command line tool: snadmin install-services ----> FAILs, but creates the db
4. execute it again ---> SUCCESS

Something is not right with the SQL connection after creating the db.

These kinds of office protocol links should work even without the UI layer:

ms-word:ofe|u|http://example.com/Root/Workspace/DocLib/Lorem.docx

Currently Word cannot open the file.

Create a repository for the Document viewer component and publish it as NuGet and/or npm packages.

Note: this is the user-facing viewer plugin, not the preview generator, which is an enterprise component and published differently.

• a JS plugin that can be used with the Services layer
• hide the Preview action in WebPages component but retain the old JS and ascx files so that it can be switched on manually.
• Refactor/rewrite the js plugin (remove jquery and other unnecessary dependencies, etc) in TypeScript and React(?)
• plugin can be used as a standalone viewer without sensenet

Tasks / Progress

• Pager widget (jump to fist / prev / next / last / specified page)
• Shapes
  • Delete shape from keyboard
  • Recalculate shape positions on rotation
  • Note formatting and updating
  • Create new shape
• Config
  • Endpoint for localized string resources
• DocumentViewerError component - localizable error messages instead of response dump
• Testing & docs
  • Enable completed-docs linting rule, fix all linting errors
  • Coverage to 80%+
• Finializing
  • publish example app to Netlify

Now it stores logs for every sync. There should be an option to logrotate... (save the last 24 detailed logs only)

If there is an unknown action in the system (e.g. there is an application content that refers to an unknown action class), and an odata response contains the list of available actions (which is the default), it will result in the following error (500 internal server eror):

InvalidContentActionException: Unknown action.

The operation (e.g. a content creation) may execute correctly, but the response is messed up with this error result because of the missing action. This should be a simple warning in the log.

The exception is thrown by the ActionFramework.GetActions method.

...which results in wakeup errors on NotifyContextChanged

This time to do with the wake up workflows to continue for when a Task changes that you are looking at with "WaitForContentChanged" workflowaction...

The problem is in the CreateBookmark I think, somehow the context in Execute (line 32 of WaitForContentChanged.cs) has the wrong WorkflowInstancePath (ContentWorkflowExtension) from another Running workflow, maybe .

Therefor the bookmark it creates is wrong cause it points to a a different workflow.

And so on NotifyContextChanged (at Save of Task) it can't load the workflowapp and continue

—

1 2016-10-27 10:49:00.97764 Workflow A:/LM/W3SVC/2/ROOT-1-131220189108567528 T:58 WF: NotifyContentChanged: Loaded notifications for Content#869782: count:1, items:[[668c6630-6402-42ea-8bcb-d77bd0a6240f]]
2 2016-10-27 10:49:00.97764 Workflow A:/LM/W3SVC/2/ROOT-1-131220189108567528 T:58 ERROR CreateWorkflowApplication: System.NullReferenceException: Object reference not set to an instance of an object... at SenseNet.Workflow.InstanceManager.CreateWorkflowApplication(WorkflowHandlerBase workflowMasterInstance, WorkflowApplicationCreationPurpose purpose, IDictionary2 parameters) in c:\Builds\5\SenseNet\PACKAGEEnterprise\Sources\Source\SenseNet\Workflow\InstanceManager.cs:line 162 3 2016-10-27 10:49:00.97764 Workflow A:/LM/W3SVC/2/ROOT-1-131220189108567528 T:58 ERROR FireNotification: System.NullReferenceException: Object reference not set to an instance of an object... at SenseNet.Workflow.InstanceManager.CreateWorkflowApplication(WorkflowHandlerBase workflowMasterInstance, WorkflowApplicationCreationPurpose purpose, IDictionary2 parameters) in c:\Builds\5\SenseNet\PACKAGEEnterprise\Sources\Source\SenseNet\Workflow\InstanceManager.cs:line 206.. at SenseNet.Workflow.InstanceManager.FireNotification(WorkflowNotification notification, WorkflowNotificationEventArgs eventArgs) in c:\Builds\5\SenseNet\PACKAGEEnterprise\Sources\Source\SenseNet\Workflow\InstanceManager.cs:line 429
4 2016-10-27 10:49:00.97764 Workflow A:/LM/W3SVC/2/ROOT-1-131220189108567528 T:58 ERROR NotifyContentChanged: System.NullReferenceException: Object reference not set to an instance of an object... at SenseNet.Workflow.InstanceManager.CreateWorkflowApplication(WorkflowHandlerBase workflowMasterInstance, WorkflowApplicationCreationPurpose purpose, IDictionary`2 parameters) in c:\Builds\5\SenseNet\PACKAGEEnterprise\Sources\Source\SenseNet\Workflow\InstanceManager.cs:line 206.. at SenseNet.Workflow.InstanceManager.FireNotification(WorkflowNotification notification, WorkflowNotificationEventArgs eventArgs) in c:\Builds\5\SenseNet\PACKAGEEnterprise\Sources\Source\SenseNet\Workflow\InstanceManager.cs:line 446.. at SenseNet.Workflow.InstanceManager.NotifyContentChanged(WorkflowNotificationEventArgs eventArgs) in c:\Builds\5\SenseNet\PACKAGEEnterprise\Sources\Source\SenseNet\Workflow\InstanceManager.cs:line 467

Which you can see from below select where I join the workflownotifications table with the instancestable.

SELECT TOP 1000 w.NotificationId
,w.NodeId
,w.WorkflowInstanceId
,w.WorkflowNodePath
,w.BookmarkName, n.BlockingBookmarks
FROM [SenseNetContentRepository].[dbo].[WorkflowNotification] as w
INNER JOIN [SenseNetContentRepository].[System.Activities.DurableInstancing].[InstancesTable] as n
ON w.WorkflowInstanceId = n.Id
where BookmarkName <> SUBSTRING(BlockingBookmarks,2,36)

Following Select gives me the results in the attachment. As you will see bookmark names aren't matching.

How this happens I have no clue... Somehow the Workflow must be running with bad WorkflowInstancePath when you execute WaitForContentChanged

Field controls

• Display Name: Has own control named "Display Name".
• Name: Has own control: "Name".
• Start date & Due date: DateTime type fields. The client is responsible for the UTC time format conversions to and from local time. (Can be set to show just the date or the whole date time)
• Priority: Choice field, dropdown list.
• Status: Ditto.
• Completion: Number control - should be unified all over the place (we don't need currency or percentage controls, it should be derived from a parameter)
• Description: Wysiwyg editor.
• Modified by: Read-only reference field.
• Modification Date: DateTime field.
• Version: Read-only Choice field. (we may not need to display it)
• Index: Numer field - plain integer.

• buttons (save and cancel - not field controls)

Some sql scripts fail in an environment where sql scripts are case sensitive. For example the proc_Node_LoadData_Batch stored procedure defines a table with a column called 'Id' (uppercase) and later tries to use it with 'id' (lowercase). This works in most environments, but in a db which uses a case-sensitive collation, this will fail.

Resolution: test the product for example with a Turkish case-sensitive collation and fix install and runtime (!) issues.

This is something we have to pay attention to in upgrade scripts too, but it is the responsibility of the developer creating a patch..

Acceptance crit:

• Case-sensitive scripts work for the most important features: create content, delete content, modify content types, batch file upload (with bigger files for generating chunks)
• Tested in Turkish case-sensitive collation

We should have a proof of concept of sensenet ECM working with Azure SQL

Acceptance criteria:

• We have an Azure sql db
• The db is filled with sn-specific data
• An on-premise installed sensenet can connect to the db
• It works

Currently SnAdminRuntime always loads custom patch dlls from the hardcoded "PackageCustomization" subfolder inside the executed package. It would be useful if we allowed the package maker to ship multiple different patch libraries for different phases inside a single package.

A typical use case for this is when we have to execute a custom step compiled with the old SN libraries, than other custom steps compiled with the new ones. These two types of custom steps cannot live in the same assembly and cannot be added to a single package now, because the runtime always loads custom dlls from the folder mentioned above.

Solution (updated): let the package creator define folders for each phase, containing custom libraries that are loaded only for that phase.

We keep the current behavior (if you put a folder called PackageCustomization into your package, we will load those dlls for all phases), and if you define a folder explicitly in the extensions attribute of a Phase, we will load those dlls only for that phase.

<Phase extensions="PackageRelativePath">

Create a repository (sn-notification) and NuGet packages for the Notification feature.

• a dll-only package (SenseNet.Notification)
• an install package (tables, content types, etc) (SenseNet.Notification.Install)
• a GUI package for portlets. Dependency: WebPages (SenseNet.Notification.Portlets)

Testing

• try the feature, event with a custom notification config
• The documentation is complete on community.sensenet.com/docs

todo

• Talk to Joe about their solution and discuss feature requests
• Check the documentation if it makes sense

I've failed to exec the following OData Action with JWT Token authentication:
http://sn-webpages/odata.svc/('Root')/GetVersionInfo
The request has a valid x-access-data header

Response:

{
  "error": {
    "code": "UnknownAction",
    "exceptiontype": "InvalidContentActionException",
    "message": {
      "lang": "en-us",
      "value": "Unknown action. Action: GetVersionInfo"
    },
    "innererror": {
      "trace": "ODataException: Unknown action. Action: GetVersionInfo\r\n\r\n---- Inner Exception:\r\nInvalidContentActionException: Unknown action. Action: GetVersionInfo\r\n   at SenseNet.Portal.OData.ODataFormatter.WriteOperationResult(PortalContext portalContext, ODataRequest odataReq) in C:\\Users\\gallayl\\Source\\Repos\\sensenet\\src\\Services\\OData\\ODataFormatter.cs:line 377\r\n   at SenseNet.Portal.OData.ODataFormatter.WriteContentProperty(String path, String propertyName, Boolean rawValue, PortalContext portalContext, ODataRequest req) in C:\\Users\\gallayl\\Source\\Repos\\sensenet\\src\\Services\\OData\\ODataFormatter.cs:line 316\r\n   at SenseNet.Portal.OData.ODataHandler.ProcessRequest(HttpContext context, String httpMethod, Stream inputStream) in C:\\Users\\gallayl\\Source\\Repos\\sensenet\\src\\Services\\OData\\ODataHandler.cs:line 142\r\n=====================\r\n"
    }
  }
}

I've noticed that at SenseNet.Portal.Virtualization.PortalAuthenticationModule.TokenAuthenticate_ (@line 339), after setting context.User, it will be SenseNet.ContentRepository.Storage.Security.StartupUser again.

It would be useful if we had an SnAdmin step designed to allow one or more child types in both scenarios:

• on a specific content
• in a content type definition

This step could be used to add additional content types to the current list, without removing anything from the existing list. For example:

<AllowChildTypes path="/Root" contentTypes="DocumentLibrary,MemoList" />
<AllowChildTypes contentType="Workspace" contentTypes="DocumentLibrary,MemoList" />

(names are negotiable as usual :) )

There are various message length limit that depends on the OS. https://support.microsoft.com/en-us/kb/957662
If we are trying to log an exception which is bigger that this then it is not logged at all.

Example:

var props = Newtonsoft.Json.JsonConvert.DeserializeObject<Dictionary<string, object>>(System.IO.File.ReadAllText(@"C:\temp\long exception.txt"), new JsonSerializerSettings() { TypeNameHandling = TypeNameHandling.All });
Microsoft.Practices.EnterpriseLibrary.Logging.Logger.Write(ex.Message, new List(), -1, 666, TraceEventType.Error, string.Empty, props);

This works:

var props = Newtonsoft.Json.JsonConvert.DeserializeObject<Dictionary<string, object>>(System.IO.File.ReadAllText(@"C:\temp\long exception.txt"), new JsonSerializerSettings() { TypeNameHandling = TypeNameHandling.All });
props.Remove("ContentRegistrationException/OldAspectDefinition");
Microsoft.Practices.EnterpriseLibrary.Logging.Logger.Write(ex.Message, new List(), -1, 666, TraceEventType.Error, string.Empty, props);

Acceptance criteria

• Fixed in SN70Beta
• Fixed in SN 6.5.4
• Reported back to JIRA

Create a separate repository for the workflow feature and publish it as NuGet packages.

• one dll-only package for the workflow libraries (SenseNet.Workflow)
• one install package (SenseNet.Workflow.Install) - creates the wf tables, imports content types etc.
• the install package adds views, pages (GUI-related stuff) only if the WebPages component is present.
• the install package adds certain items (e.g. Mailprocessor wf) only if the Workspaces component is present.
• another dll package for GUI classes that can be installed only if WebPages is present (SenseNet.Workflow.Portlets?)
• (later, optionally) set demo workflows on demo lists (under Default_Site/workspaces) as in previous demo systems (e.g. a predefined approval in one list)

Acceptance criteria

• Tried by 1 person other than @tusmester
• The install works
• Approval workflow works
• Public registration workflow runs without a hitch

Create an install package from the preview component.

• move the source code to its separate repo from TFS
  • DocViewer
  • Preview generator and server-side code
• cleanup DocViewer JS code in the ascx
• install packages

There should be a text extract functionality in Services, but it looks like it does not extract much text from pdf's. @kavics knows more about the issue, please share any relevant info here.

There is a new release of Newtonsoft.Json, we should upgrade our package references and test our components with that.

• update Services component (all others later)
  • package references
  • config files (assembly bindings)
  • nuspec files that define install packages
• update NuGet release readme.txt in Install packages.

The test project (with approximately 1300 tests) is not suitable for regular execution (run as per-commit test). Test cases within this project must be implemented in the appropriate component (mainly in the services). It would be better if this issue will generate more smaller stories (one theme per story e.g. odata or notification). The original project - after heavy cleaning - can continue to work as system tests (or beta tests or any other buzzword). In this case these tests must be executed before releasing the product.

ODataHandler, which is the front object of the backend, not unit testable because it works in httpcontext and uses other direct dependencies.

• Middleware
• Migrate http modules

This issue has been split up to followings: #733, #734, #735, #736, #737, #739, #740