force-net / deepcloner Goto Github PK

Howdy...

We spent the last 3 weeks working with MS on an issues without SOAP interface where we have integrated the DeepCloner tool. According to MS Support the DeepCloner is causing HEAP corruption which is then causing the Garbage Collector to throw errors due to Access Violations. This causes the dotNet6 framwork to crash and make the SOAP interface non-responsive.

Has anyone else see this issue? Any more info that the devs can use to fix this issue?

Is there a provision in the library to make shallow copy of few expensive objects, while we deep copy the remaining object graph. I doubt on reviewing the APIs, I do plan to make code change, by supplying an explicit Dictionary with expensive objects added to it and in case object is there in the dictionary then create a Shallow version else a regular code which creates a deep version. Please suggest if my understanding is correct and may be some guidance to make code modification.

To provide more clarity:

My object is of following type, where both ODPair and ServiceProduct are the custom types. ServiceProduct is an expensive object, for which I want to retain the Shallow copy, while rest of the object is deep copied

Dictionary<ODPair,Dictionary<string,List<ServiceProduct>>>

DeepClone is able to cause anonymous/untraceable errors in system code. I'm a few hours in now debugging an unpredictable exception. It has no stacktrace that would lead to my own code or to DeepClone.

The error has reported out as:

• a null reference exception in System.Private.CoreLib
• a null reference exception in System.Threading.Overlapped.Free
• an Access Violation exception in unspecified system code

By stepping through my code, I've finally narrowed the issue down to a usage of ActionExecutedContext.Result.DeepClone(). I'm sure it's my fault, I'm probably cloning something I shouldn't be (something that has a reference to itself or a pointer to an object held by some other process or a system object...I've been reading the other issues on this repo so I understand there are some things that shouldn't ever be cloned).

I'm okay with the cloning restrictions and I'll figure out what's going on, but it would have saved me some time if DeepCloner had some error handling built in. try/catch stuff. It wouldn't need to tell me exactly what the problem is, it would just be nice to know that the error is happening in DeepCloner and not somewhere else, since you can see that isn't clear at all from the errors I'm getting.

Is that a possibility for a future version?

EDIT: On further investigation, it's not at all certain that the error is happening during the DeepClone so this might be a red herring. Sorry for the trouble.

Hello, I have quite a few classes that implement INotifyPropertyChanged due to mvvm, cloning these classes seems to lead to a memory error and crash problem.

Is there a way to not clone events or specific properties?

Hi there, first of all, thanks for the great library :)

We have our own implementation of deep cloning and wanted to test out this library to compare performance, but most of our tests fail when nhibernate tries to save object with error:

System.InvalidCastException : Unable to cast object of type 'NHibernate.Engine.Query.QueryExpressionPlan' to type 'System.Transactions.SafeIUnknown'.
at System.Transactions.Transaction.JitSafeGetContextTransaction(ContextData contextData)
at System.Transactions.Transaction.FastGetTransaction(TransactionScope currentScope, ContextData contextData, Transaction& contextTransaction)
at System.Transactions.Transaction.get_Current()
at NHibernate.Transaction.AdoNetWithSystemTransactionFactory.EnlistInSystemTransactionIfNeeded(ISessionImplementor session)
at NHibernate.Impl.AbstractSessionImpl.CheckAndUpdateSessionStatus()
at NHibernate.Impl.SessionImpl..ctor(SessionFactoryImpl factory, ISessionCreationOptions options)
at NHibernate.Impl.SessionFactoryImpl.SessionBuilderImpl`1.OpenSession()

Our assumption is that DeepCloner tries to clone nhibernate proxy that is created on runtime, and that issues happen there.
So I wanted to ask is this maybe a known issue, and is there a way to handle it?

the program crashed when i deepclone opencvsahrp Mat for several times

I discovered several issues using your library.

Situation 1:
I have a parent form with datagridview. Mouse doubleclick on datagridview row retrieves binded record and open new child form that presents details of this record. In that child form I can click Save button to save updated record in database, after which child form is closed/disposed and parent form updates datagridview. In child form before saving to database I do deepclone using your library:

I removed everything unrelated.

' Property in form
Property Audit As AuditDefinition

' Entrypoint to form
Public Sub New(audit As AuditDefinition)
    InitializeComponent()
    Me.Audit = audit
End Sub

' Somewhere in form I await saving function and if return true I close the form
If Await SaveInDbAsync() then Close()

' Saving in database
Private Async Function SaveInDbAsync() As Task(Of Boolean)
    ' Cloning Audit property in case of saving in database failure (then restore changed child properties)
    cAudit = Audit.DeepClone
   ' Rest of code
End Function

Child form is properly initialized and destroyed with using/end using clause from parent form.

Problem:
Each next time when child form in created (with new record Audit) and deepclone is performed it is slower and slower. Example times:
132 ms (first use of deepclone in application)
15 ms
25 ms
53 ms
163 ms
291 ms
569 ms

What do you think about it?

Situation 2:
Since I switch making clone objects from my implementation to yours my application started to close unexpectedly with many different exceptions, mainly System.AccessViolationException in random System.xxxx.dll libraries.

For example during debugging situation 1 above I made a simple loop making cloning 10 times the same object receiving each time the same time (opposite to situation above where time is longer and longer with each call) but just after exiting the loop application receives AccessViolationException.

Dim cAudit As AuditDefinition

For i = 0 To 9
    Debug.WriteLine(i)
    cAudit = Audit.DeepClone
Next

Result:

0
1
2
3
4
5
6
7
8
Exception thrown: 'System.Runtime.InteropServices.SEHException' in System.Windows.Forms.dll
9
System.Runtime.InteropServices.SEHException (0x80004005): External component has thrown an exception.

And debugger can move to next lines of code but soon application is closed with

Cross-thread operation not valid: Control '{0}' accessed from a thread other than the thread it was created on.
Or
Attempted to read or write protected memory. This is often an indication that other memory is corrupt.

Exception occurs randomly after 4 to 9 cloning operations.
Moving operation to non-async function doesn't help.

What do you think about it?

Clone Action delegate is incorrect,
DeepClone appears to Clone the Action.Target object,
which it shouldn't be, because the delegate itself should just be a pointer to the target method.

Hi. I got this message by cloning standard DTO object, that inherits from INotifyPropertyChanged interface via Fody. I'm not sure could that be a problem. Thanks

This library has a significant amount of downloads and appears to be stable and reliable. Why does it still have major version 0?

This makes the library look rather unprofessional to consumers and actually raised a red flag when I saw it being added in one of our projects. Usually version 0.X are reserved for alpha and beta versions, basically things that are not yet ready to market.

Can the next release bump the version to 1.0.0 if the library is considered stable and production-ready?

Hello @force-net,
Would it be possible to publish the latest changes to NuGet? (commit 8f14040)

Thanks in advance

System.StackOverflowException:“Exception_WasThrown” on internal static Func<T, DeepCloneState, T> GetClonerForValueType().

public class Parent{};
public class Child : Parent {};
var parent = new Parent();
var child = new Child();
child.DeepCloneTo(parent); // fails to compile

Expected behavior: copy child properties which are present in/inherited from the parent to the parent instance and ignore child's own properties.

Here is a piece of code to recreate the bug, I hope.

using System.Collections.Generic;
using System.Linq;
using Force.DeepCloner;

namespace DeepClonerBug
{
internal class Program
{
static void Main(string[] args)
{
var org = new Dictionary<Location, Location[]>
{
{ new Location { Name = "A" }, new[] { new Location { Name = "1" } } }
};

        org[org.Keys.First()].ToString();

        var clone = org.DeepClone();

        // throws System.Collections.Generic.KeyNotFoundException: 'The given key was not present in the dictionary.'
        clone[clone.Keys.First()].ToString();
    }
}

public class Location
{
    public string Name { get; set; }
}

}

Expression must be writeable (Parameter 'left')
System.Linq.Expressions
at System.Linq.Expressions.Expression.RequiresCanWrite(Expression expression, String paramName)
at System.Linq.Expressions.Expression.Assign(Expression left, Expression right)
at Force.DeepCloner.Helpers.DeepClonerExprGenerator.GenerateProcessMethod(Type type, Boolean unboxStruct)
at Force.DeepCloner.Helpers.DeepClonerExprGenerator.GenerateClonerInternal(Type realType, Boolean asObject)
at Force.DeepCloner.Helpers.DeepClonerGenerator.GenerateCloner(Type t, Boolean asObject)
at Force.DeepCloner.Helpers.DeepClonerGenerator.<>c.b__1_0(Type t)
at Force.DeepCloner.Helpers.DeepClonerCache.GetOrAddClass[T](Type type, Func`2 adder)
at Force.DeepCloner.Helpers.DeepClonerGenerator.CloneClassRoot(Object obj)
at Force.DeepCloner.Helpers.DeepClonerGenerator.CloneObject[T](T obj)
at Force.DeepCloner.DeepClonerExtensions.DeepClone[T](T obj)

Is there an option, or configuration to evaluate the expression trees before deep copying a property? e.g. LINQ statements as nested property underneath an object.

JSONConvert, when serializing, evaluates the properties, and, I would like to achieve the same thing.

I have a class which implemented INotifyPropertyChanged interface.

How can I use deepClone without some property( PropertyChanged event )?

I have tried [field: NonSerialized] and [JsonIgnore], but it still clone the PropertyChanged event.

Example:

class C1 { public int Prop {get; set;} }
class C2 { public C1 Prop {get; set;} }

var target = new C2 { Prop = new C1() }
var source = new C2 { Prop = new C1() }

var targetC1Before = target.Prop;
source.DeepCloneTo(target);
var targetC1After = target.Prop;

Debug.Assert(object.ReferenceEquals(targetC1Before, targetC1After); // FAIL

Does DeepCloner use BinaryFormatter?

see: https://github.com/dotnet/designs/blob/main/accepted/2020/better-obsoletion/binaryformatter-obsoletion.md

I've run into a problem where using DeepClone to clone a TaskCancelledException is corrupting memory and leading to a crash of my application. The exception is being thrown by Task.Delay(int, CancellationToken) when the token is cancelled. I believe the source of the problem is when it goes to clone the TaskCancelledException.Task property. This particular task includes a System.Threading.TimerQueueTimer object that I believe has native resources stored in it.

Is it possible to have some way to skip or configure how certain properties are cloned? For example, you could have attributes such as [IgnoreClone] or [ShallowClone] to specify that a property should default to being shallow cloned instead of attempting to deep clone. This way I could mark my Exception property as [ShallowClone] and it wouldn't even attempt to clone something that might have unmanaged resources in it.

Preface

DBNull.Value represents ADO.NET NULL value and defined as:

public static readonly DBNull Value = new DBNull();

Problem

Code:

if(dataRow["col1"] != DBNull.Value && dataRow["col2"] == DBNull.Value) // ...

Let's assume both columns contain DBNull.Value. When dataRow is cloned by DeepCloner copy of DBNull.Value is created for both columns. After cloning first comparison is still true, second one changed result to false from true cause clone of DBNull is not DBNull.Value.

.NET FW has a lot of DBNull.Value comparisons itself which leads to a very strange effects, e.g. changing DBNull string in DataColumn from null to empty string.

Fix

Add DBNull to Force.DeepCloner.Helpers.DeepClonerSafeTypes.KnownTypes

Workaround

// Excluding DBNull from list of cloned types. DeepCloner creates DBNull.Value copy which is static readonly instance of DBNull itself which breaks DBNull.Value reference equality.
var assembly = Assembly.GetAssembly(typeof(DeepClonerExtensions));
var deepClonerSafeTypes = assembly.GetType("Force.DeepCloner.Helpers.DeepClonerSafeTypes");
var knownTypesField = deepClonerSafeTypes.GetField("KnownTypes", BindingFlags.Static | BindingFlags.NonPublic);
var knownTypes = (ConcurrentDictionary<Type, bool>)knownTypesField.GetValue(null);
knownTypes.TryAdd(typeof(DBNull), true);

DeepCloner is currently targeted at Netstandard 1.3, which, along with this version of netstandard, downloads the system.text.regularexpressions and system.net.http packages that have vulnerabilities.

Is it possible to target DeepCloner at Netstandard 2.0 to eliminate these vulnerabilities?

Hello:

Would you share the country of origin?

Please and thank-you,

Chad
.

Hi,

at the moment I'm investigating why my tests are unstable, i.e. unexpected aborts. Switched from mstest/vstests to nunit, but issue remained. Decided to run them in a smaller chunks, interesting enough that sometimes 50 tests could pass 4-5 runs and fail on 6th, but sometimes fail on every run or on every other run. Moreover, almost always it was a different test running when "abort" has happened. And by abort I mean (and according to event log) - an execution engine exception happening in the clr runtime:

The process was terminated due to an internal error in the .NET Runtime at IP 00007FF9C27D92E0 (00007FF9C2220000) with exit code 80131506.

Interesting enough another set of tests was always working as expected. Further and still a preliminary investigation indicated that failing tests (some of them) have DeepCloner in use. As the next step I ran nunit-console along with "Debug Diagnostic Tool" and with a single test that used DeepCloner. Then checked the logs, probably the most important findings are:

DetailID = 4
	Count:    1
	Type:     System.BadImageFormatException
	Message:  An attempt was made to load a program with an incorrect format. (Exception from HRESULT: 0x8007000B)
	Stack:    
		[InlinedCallFrame]
		[InlinedCallFrame]
		DomainNeutralILStubClass.IL_STUB_PInvoke(System.Reflection.RuntimeModule, Int32, IntPtr*, Int32, IntPtr*, Int32)
		System.ModuleHandle.ResolveMethodHandleInternalCore(System.Reflection.RuntimeModule, Int32, IntPtr[], Int32, IntPtr[], Int32)
		System.ModuleHandle.ResolveMethodHandleInternal(System.Reflection.RuntimeModule, Int32, System.RuntimeTypeHandle[], System.RuntimeTypeHandle[])
		System.Reflection.RuntimeModule.ResolveMethod(Int32, System.Type[], System.Type[])
		Force.DeepCloner.Helpers.DeepClonerMsilHelper.IsConstructorDoNothing(System.Type, System.Reflection.ConstructorInfo)
		Force.DeepCloner.Helpers.DeepClonerMsilGenerator.GenerateProcessMethod(System.Reflection.Emit.ILGenerator, System.Type, Boolean)
		Force.DeepCloner.Helpers.DeepClonerMsilGenerator.GenerateClonerInternal(System.Type, Boolean)
		Force.DeepCloner.Helpers.DeepClonerCache.GetOrAddClass[[System.__Canon, mscorlib]](System.Type, System.Func`2<System.Type,System.__Canon>)
		Force.DeepCloner.Helpers.DeepClonerGenerator.CloneClassInternal(System.Object, Force.DeepCloner.Helpers.DeepCloneState)
		DynamicClass.DeepObjectCloner_Dictionary`2_82(System.Object, Force.DeepCloner.Helpers.DeepCloneState)
		DynamicClass.DeepObjectCloner_ValidationFailure_81(System.Object, Force.DeepCloner.Helpers.DeepCloneState)
		DynamicClass.DeepObjectCloner_ValidationFailure[]_80(System.Object, Force.DeepCloner.Helpers.DeepCloneState)
		DynamicClass.DeepObjectCloner_List`1_79(System.Object, Force.DeepCloner.Helpers.DeepCloneState)
		DynamicClass.DeepObjectCloner_RequiredInventoryReservation_5(System.Object, Force.DeepCloner.Helpers.DeepCloneState)
		DynamicClass.DeepObjectCloner_RequiredInventoryReservation[]_4(System.Object, Force.DeepCloner.Helpers.DeepCloneState)
		DynamicClass.DeepObjectCloner_List`1_3(System.Object, Force.DeepCloner.Helpers.DeepCloneState)
		DynamicClass.DeepObjectCloner_RequiredInventoryReservationsCollection_2(System.Object, Force.DeepCloner.Helpers.DeepCloneState)
		DynamicClass.DeepObjectCloner_Inventory_1(System.Object, Force.DeepCloner.Helpers.DeepCloneState)
		Force.DeepCloner.Helpers.DeepClonerGenerator.CloneObject[[System.__Canon, mscorlib]](System.__Canon)
		REDACTED+<TEST_IN_QUESTION>d__27.MoveNext()
--- rest is redacted

and the following:

DetailID = 5
	Count:    1
	Type:     System.ArgumentException
	Message:  A BadImageFormatException has been thrown while parsing the signature. This is likely due to lack of a generic context. Ensure genericTypeArguments and genericMethodArguments are provided and contain enough context.
		Type:     System.BadImageFormatException
		Message:  An attempt was made to load a program with an incorrect format. (Exception from HRESULT: 0x8007000B)
	Stack:    
		[HelperMethodFrame]
		System.Reflection.RuntimeModule.ResolveMethod(Int32, System.Type[], System.Type[])
		[InlinedCallFrame]
		[InlinedCallFrame]
		DomainNeutralILStubClass.IL_STUB_PInvoke(System.Reflection.RuntimeModule, Int32, IntPtr*, Int32, IntPtr*, Int32)
		System.ModuleHandle.ResolveMethodHandleInternalCore(System.Reflection.RuntimeModule, Int32, IntPtr[], Int32, IntPtr[], Int32)
		System.ModuleHandle.ResolveMethodHandleInternal(System.Reflection.RuntimeModule, Int32, System.RuntimeTypeHandle[], System.RuntimeTypeHandle[])
		System.Reflection.RuntimeModule.ResolveMethod(Int32, System.Type[], System.Type[])
		Force.DeepCloner.Helpers.DeepClonerMsilHelper.IsConstructorDoNothing(System.Type, System.Reflection.ConstructorInfo)
		Force.DeepCloner.Helpers.DeepClonerMsilGenerator.GenerateProcessMethod(System.Reflection.Emit.ILGenerator, System.Type, Boolean)
		Force.DeepCloner.Helpers.DeepClonerMsilGenerator.GenerateClonerInternal(System.Type, Boolean)
		Force.DeepCloner.Helpers.DeepClonerCache.GetOrAddClass[[System.__Canon, mscorlib]](System.Type, System.Func`2<System.Type,System.__Canon>)
		Force.DeepCloner.Helpers.DeepClonerGenerator.CloneClassInternal(System.Object, Force.DeepCloner.Helpers.DeepCloneState)
		DynamicClass.DeepObjectCloner_Dictionary`2_82(System.Object, Force.DeepCloner.Helpers.DeepCloneState)
		DynamicClass.DeepObjectCloner_ValidationFailure_81(System.Object, Force.DeepCloner.Helpers.DeepCloneState)
		DynamicClass.DeepObjectCloner_ValidationFailure[]_80(System.Object, Force.DeepCloner.Helpers.DeepCloneState)
		DynamicClass.DeepObjectCloner_List`1_79(System.Object, Force.DeepCloner.Helpers.DeepCloneState)
		DynamicClass.DeepObjectCloner_RequiredInventoryReservation_5(System.Object, Force.DeepCloner.Helpers.DeepCloneState)
		DynamicClass.DeepObjectCloner_RequiredInventoryReservation[]_4(System.Object, Force.DeepCloner.Helpers.DeepCloneState)
		DynamicClass.DeepObjectCloner_List`1_3(System.Object, Force.DeepCloner.Helpers.DeepCloneState)
		DynamicClass.DeepObjectCloner_RequiredInventoryReservationsCollection_2(System.Object, Force.DeepCloner.Helpers.DeepCloneState)
		DynamicClass.DeepObjectCloner_Inventory_1(System.Object, Force.DeepCloner.Helpers.DeepCloneState)
		Force.DeepCloner.Helpers.DeepClonerGenerator.CloneObject[[System.__Canon, mscorlib]](System.__Canon)
		REDACTED+<TEST_IN_QUESTION>d__27.MoveNext()
--- rest is redacted

To verify the assumption I replaced DeepCloner with CloneExtensions for that test:

 var inventoryCopy = helper.Inventory.GetClone();//.DeepClone();

ran that particular test several times and it seems to be working as expected now.
As mentioned earlier, these are preliminary findings, yet they indicate(?) that DeepCloner generates a malformed IL that crashes the entire runtime. Next, I will migrate to CloneExtensions and run the entire test suite to confirm the assumption. Let's see...

Thank you!