microsoft / cci Goto Github PK

Common Compiler Infrastructure

License: MIT License

C# 100.00%

cci's Introduction

Common Compiler Infrastructure (CCI)

IMPORTANT

Moving forward, there will only be servicing investments in CCI to support existing uses. New projects that need related functionality should look to the following instead:

GitHub vs. CodePlex

This GitHub repository is the new central location for CCI and will replace https://ccimetadata.codeplex.com and https://cciast.codeplex.com.

cci's People

Contributors

Stargazers

Watchers

cci's Issues

MemberHelper.ResolveMethod fails with generic method instances

When calling ResolveMethod with a MethodReference and a containing type, the first thing it does is check to see if the method is a generic method instance. If so, it resolves the underlying generic method on the type, then tries to instantiate it. Unfortunately, it appears that this instantiation will always fail and return Dummy.MethodDefinition.

On line 634 we read:

var genericArgs = new ITypeReference[genericMethodInstanceReference.GenericParameterCount];

It then goes through and attempts to map the generic arguments from the input generic method instance onto the new generic method instance it's constructing, but it will bail out and return Dummy.MethodDefinition if the generic parameter count does not match.

Unfortunately, it appears that this will always fail due to the size of genericArgs always being 0, as generic method instances do not have GenericParameters. Assuming I'm not misreading the code, genericMethDef.GenericParameterCount should be used instead.

How should resource-emitting code be translated?

I'm working on porting a compiler over from Reflection.Emit to CCI. I've got most of it figured out by this point, but I can't seem to wrap my head around the Resource class.

The CLR managed resource model defines a Resource as a thing with a name that's a key to a list of name-value pairs. You call ModuleBuilder.DefineResource(), passing a Name and Description, to get an IResourceWriter, and then call AddResource 1 or more times, passing a Name for each individual resource within that IResourceWriter's named group.

The Microsoft.Cci.MutableCodeModel.Resource class appears to correspond to the individual sub-resources written by IResourceWriter, as it has a Name and a byte array for contents but no Description or collection of name-value pairs for sub-resources, but it also appears to correspond to the outer-level resource group, as it can refer to an external resource file, which can contain the name-value pairs within it. It's all very unclear as to what the correct way is to port code using the Reflection.Emit model.

What is the idiomatic way to create managed resource groups with CCI?

TypeHelper.GetMethod can't find methods with ref arguments

For a compiler to look up a method in an external assembly, the standard technique seems to be UnitHelper.GetType followed by TypeHelper.GetMethod. But this approach fails when looking up a method with a ref argument, because the code explicitly prevents matches if any parameter is ByRef, Out, or has modifiers.

What is the correct way to look up such a method?

Microsoft.DevDiv.Managed.Compiler.props

Where can I find Microsoft.DevDiv.Managed.Compiler.props?
It is referenced in Microsoft.Cci_NetCore.csproj:
<Import Project="$(_NTROOT)\Tools\Microsoft.DevDiv.Managed.Compiler.props" />

Or should I rely on dotnet migrate to build netstandard binaries?

Or does CCI not support netstandard?

Nuget Feed

Hi i see there are a lot of changes on these code, but the last date of publish an nuget was 2016-06-27, https://www.nuget.org/packages/Microsoft.Cci

do you have another nuget feed to use for get the lasted stable version of Microsoft.Cci?
also i have a code that use decompiler but in these nuget threre aren't dll that contains these class.

are you going to exclude these class?

Call is being emitted to the wrong method

I'm having a bizarre problem that I can't track down: when Main calls a certain method, everything in the emitting code looks right, but in the generated IL it's a call back to Main instead of to the method I'm trying to call. Or at least, that's what ILDasm says. PEVerify says

[IL]: Error: [C:\Users\Mason\AppData\Local\Temp\msbuild\testcase.exe : Array_polymorphic_foreachModule::Main] Method does not exist.

It's not clear whether that's referring to the method being called or the method in which the error occurs.

Do you have any idea how I could track down what's going wrong? At the point where I emit the call, it's passing the correct IMethodReference to _il.Emit(), and the target method does get generated and shows up as expected in ILDasm, so I'm kind of stuck here.

Error in module's CorFlags for Prefers32bits

It seems that the current sources don't handle the CorFlags bit field in the module header for the Prefers32bits bit. I have created a fix in the CorFlagsBug branch of my fork. It appears to fix the problem, but I'm worried that it won't do the right thing for older assemblies. If anyone has any ideas, please let me know!

MetadataReader does not work on ARM

I was trying to build corefx on Linux arm:

  Unhandled Exception: System.DataMisalignedException: A datatype misalignment was detected in a load or store instruction.
     at Microsoft.Cci.MetadataReader.PEFileToObjectModel.GetDefaultValue(MetadataObject metadataObject)
     at Microsoft.Cci.MetadataReader.ObjectModelImplementation.FieldDefinition.get_CompileTimeValue()
     at Microsoft.Cci.MutableCodeModel.FieldDefinition.Copy(IFieldDefinition fieldDefinition, IInternFactory internFactory)
     at Microsoft.Cci.MutableCodeModel.MetadataShallowCopier.CopyUnspecialized(IFieldDefinition fieldDefinition)
     at Microsoft.Cci.MutableCodeModel.MetadataShallowCopier.MetadataDispatcher.Visit(IFieldDefinition fieldDefinition)
     at Microsoft.Cci.MetadataReader.ObjectModelImplementation.FieldDefinition.Dispatch(IMetadataVisitor visitor)
     at Microsoft.Cci.MutableCodeModel.MetadataShallowCopier.Copy(IFieldDefinition fieldDefinition)
     at Microsoft.Cci.MutableCodeModel.MetadataDeepCopier.Populator.Visit(IFieldDefinition fieldDefinition)
     at Microsoft.Cci.MetadataReader.ObjectModelImplementation.FieldDefinition.Dispatch(IMetadataVisitor visitor)
     at Microsoft.Cci.MetadataTraverser.Traverse(IFieldDefinition fieldDefinition)
     at Microsoft.Cci.MetadataTraverser.Traverse(IEnumerable`1 fields)
     at Microsoft.Cci.MetadataTraverser.TraverseChildren(ITypeDefinition typeDefinition)
     at Microsoft.Cci.MetadataTraverser.TraverseChildren(INamedTypeDefinition namedTypeDefinition)
     at Microsoft.Cci.MetadataTraverser.TraverseChildren(INamespaceTypeDefinition namespaceTypeDefinition)
     at Microsoft.Cci.MetadataTraverser.Traverse(INamespaceTypeDefinition namespaceTypeDefinition)
     at Microsoft.Cci.MetadataTraverser.Dispatcher.Visit(INamespaceTypeDefinition namespaceTypeDefinition)
     at Microsoft.Cci.MetadataReader.ObjectModelImplementation.NamespaceType.Dispatch(IMetadataVisitor visitor)
     at Microsoft.Cci.MetadataTraverser.Traverse(IEnumerable`1 namespaceMembers)
     at Microsoft.Cci.MetadataTraverser.TraverseChildren(INamespaceDefinition namespaceDefinition)
     at Microsoft.Cci.MetadataTraverser.TraverseChildren(IUnitNamespace namespaceDefinition)
     at Microsoft.Cci.MetadataTraverser.TraverseChildren(INestedUnitNamespace nestedUnitNamespace)
     at Microsoft.Cci.MetadataTraverser.Traverse(INestedUnitNamespace nestedUnitNamespace)
     at Microsoft.Cci.MetadataTraverser.Dispatcher.Visit(INestedUnitNamespace nestedUnitNamespace)
     at Microsoft.Cci.MetadataReader.ObjectModelImplementation.NestedNamespace.Dispatch(IMetadataVisitor visitor)
     at Microsoft.Cci.MetadataTraverser.Traverse(IEnumerable`1 namespaceMembers)
     at Microsoft.Cci.MetadataTraverser.TraverseChildren(INamespaceDefinition namespaceDefinition)
     at Microsoft.Cci.MetadataTraverser.TraverseChildren(IUnitNamespace namespaceDefinition)
     at Microsoft.Cci.MetadataTraverser.TraverseChildren(IRootUnitNamespace rootUnitNamespace)
     at Microsoft.Cci.MetadataTraverser.Traverse(IRootUnitNamespace rootUnitNamespace)
     at Microsoft.Cci.MetadataTraverser.TraverseChildren(IModule module)
     at Microsoft.Cci.MetadataTraverser.TraverseChildren(IAssembly assembly)
     at Microsoft.Cci.MetadataTraverser.Traverse(IAssembly assembly)
     at Microsoft.Cci.MutableCodeModel.MetadataDeepCopier.Copy(IAssembly assembly)
     at GenFacades.Generator.Execute(String seeds, String contracts, String facadePath, Version assemblyFileVersion, Boolean clearBuildAndRevision, Boolean ignoreMissingTypes, Boolean ignoreBuildAndRevisionMismatch, Boolean buildDesignTimeFacades, String inclusionContracts, ErrorTreatment seedLoadErrorTreatment, ErrorTreatment contractLoadErrorTreatment, String[] seedTypePreferencesUnsplit, Boolean forceZeroVersionSeeds, Boolean producePdb, String partialFacadeAssemblyPath, Boolean buildPartialReferenceFacade)
     at GenFacades.Program.Main(String[] args)
  Aborted (core dumped)
/home/pi/github/wfurt-corefx2/src/shims/shims.proj(56,5): error MSB3073: The command ""/home/pi/github/wfurt-corefx2/Tools/dotnetcli/dotnet" "/home/pi/github/wfurt-corefx2/Tools/GenFacades.exe" -contracts:"/home/pi/github/wfurt-corefx2/bin/ref/netfx/mscorlib.dll;/home/pi/github/wfurt-corefx2/bin/ref/netfx/System.dll;/home/pi/github/wfurt-corefx2/bin/ref/netfx/System.Configuration.dll;/home/pi/github/wfurt-corefx2/bin/ref/netfx/System.Core.dll;/home/pi/github/wfurt-corefx2/bin/ref/netfx/System.Data.dll;/home/pi/github/wfurt-corefx2/bin/ref/netfx/System.Drawing.dll;/home/pi/github/wfurt-corefx2/bin/ref/netfx/System.IO.Compression.FileSystem.dll;/home/pi/github/wfurt-corefx2/bin/ref/netfx/System.Net.dll;/home/pi/github/wfurt-corefx2/bin/ref/netfx/System.Numerics.dll;/home/pi/github/wfurt-corefx2/bin/ref/netfx/System.Runtime.Serialization.dll;/home/pi/github/wfurt-corefx2/bin/ref/netfx/System.Security.dll;/home/pi/github/wfurt-corefx2/bin/ref/netfx/System.ServiceProcess.dll;/home/pi/github/wfurt-corefx2/bin/ref/netfx/System.Transactions.dll;/home/pi/github/wfurt-corefx2/bin/ref/netfx/System.Web.dll;/home/pi/github/wfurt-corefx2/bin/ref/netfx/System.Windows.dll;/home/pi/github/wfurt-corefx2/bin/ref/netfx/System.Xml.dll;/home/pi/github/wfurt-corefx2/bin/ref/netfx/System.Xml.Serialization.dll;/home/pi/github/wfurt-corefx2/bin/ref/netfx/System.Xml.Linq.dll;/home/pi/github/wfurt-corefx2/bin/ref/netfx/System.ComponentModel.DataAnnotations.dll;/home/pi/github/wfurt-corefx2/bin/ref/netfx/System.ServiceModel.Web.dll;/home/pi/github/wfurt-corefx2/bin/ref/netfx/WindowsBase.dll" @"/home/pi/github/wfurt-corefx2/bin/obj/shims/netcoreapp/genfacades.rsp"" exited with code 134.

Build FAILED.

/home/pi/github/wfurt-corefx2/src/shims/shims.proj(56,5): error MSB3073: The command ""/home/pi/github/wfurt-corefx2/Tools/dotnetcli/dotnet" "/home/pi/github/wfurt-corefx2/Tools/GenFacades.exe" -contracts:"/home/pi/github/wfurt-corefx2/bin/ref/netfx/mscorlib.dll;/home/pi/github/wfurt-corefx2/bin/ref/netfx/System.dll;/home/pi/github/wfurt-corefx2/bin/ref/netfx/System.Configuration.dll;/home/pi/github/wfurt-corefx2/bin/ref/netfx/System.Core.dll;/home/pi/github/wfurt-corefx2/bin/ref/netfx/System.Data.dll;/home/pi/github/wfurt-corefx2/bin/ref/netfx/System.Drawing.dll;/home/pi/github/wfurt-corefx2/bin/ref/netfx/System.IO.Compression.FileSystem.dll;/home/pi/github/wfurt-corefx2/bin/ref/netfx/System.Net.dll;/home/pi/github/wfurt-corefx2/bin/ref/netfx/System.Numerics.dll;/home/pi/github/wfurt-corefx2/bin/ref/netfx/System.Runtime.Serialization.dll;/home/pi/github/wfurt-corefx2/bin/ref/netfx/System.Security.dll;/home/pi/github/wfurt-corefx2/bin/ref/netfx/System.ServiceProcess.dll;/home/pi/github/wfurt-corefx2/bin/ref/netfx/System.Transactions.dll;/home/pi/github/wfurt-corefx2/bin/ref/netfx/System.Web.dll;/home/pi/github/wfurt-corefx2/bin/ref/netfx/System.Windows.dll;/home/pi/github/wfurt-corefx2/bin/ref/netfx/System.Xml.dll;/home/pi/github/wfurt-corefx2/bin/ref/netfx/System.Xml.Serialization.dll;/home/pi/github/wfurt-corefx2/bin/ref/netfx/System.Xml.Linq.dll;/home/pi/github/wfurt-corefx2/bin/ref/netfx/System.ComponentModel.DataAnnotations.dll;/home/pi/github/wfurt-corefx2/bin/ref/netfx/System.ServiceModel.Web.dll;/home/pi/github/wfurt-corefx2/bin/ref/netfx/WindowsBase.dll" @"/home/pi/github/wfurt-corefx2/bin/obj/shims/netcoreapp/genfacades.rsp"" exited with code 134.
    0 Warning(s)
    1 Error(s)

It seems like the correct code may be already there under #if COMPACTFX
However, there is no provision to enable that for ARM. Also the package targets ANY_CPU.
So it is really not sure up right now to provide multi-architecture support.
One option would be changing the compile-time checks to runtime option.

PEReaderAndWriter/PEReader/UnmanagedReadWrite.c:

nternal UInt32 PeekUInt32(
      uint offset
    ) {
#if COMPACTFX
      if ((int)(this.CurrentPointer + offset) % 4 != 0) return UnalignedPeekUInt32(offset);
#endif
      if (checked(this.CurrentPointer - this.Buffer + offset + sizeof(UInt32)) > this.Length)
        throw new ArgumentOutOfRangeException();
#if LITTLEENDIAN
      return *(UInt32*)(this.CurrentPointer + offset);
#elif BIGENDIAN
      uint uin = *(uint*)(this.CurrentPointer + offset);
      uin = (uin >> 16) | (uin << 16);
      uin = ((uin & 0xFF00FF00) >> 8) | ((uin & 0x00FF00FF) << 8);
      return uin;
#endif
    }

cc: @nguerrera @ericstj

Unable to read some PDB's

We have been noticing that occasionally ILSpy & JustDecompile are unable to decompile C++/CLI assemblies like (WPF's) PresentationCore.dll.

Initially we thought that it might have been a compiler or a linker bug, but turns out that (a) the DLL is fine, and (b) the PDB is what these decompilers seem to have a problem with and (c) the PDB is also well formed per folks in the C++ compiler/linker teams. We also found that dotPeek is able to handle these assemblies ok.

The call stack presented by ILSpy seems to point to CCi. Any ideas on what might be going on?

System.IndexOutOfRangeException: Index was outside the bounds of the array.
   at Microsoft.Cci.Pdb.BitAccess.ReadInt32(Int32& value)
   at Microsoft.Cci.Pdb.DbiSecCon..ctor(BitAccess bits)
   at Microsoft.Cci.Pdb.DbiModuleInfo..ctor(BitAccess bits, Boolean readStrings)
   at Microsoft.Cci.Pdb.PdbFile.LoadDbiStream(BitAccess bits, DbiModuleInfo[]& modules, DbiDbgHdr& header, Boolean readStrings)
   at Microsoft.Cci.Pdb.PdbFile.LoadFunctions(Stream read, Dictionary`2& tokenToSourceMapping, String& sourceServerData, Int32& age, Guid& guid)
   at Mono.Cecil.Pdb.NativePdbReader.PopulateFunctions()
   at Mono.Cecil.Pdb.NativePdbReader.ProcessDebugHeader(ImageDebugHeader header)
   at Mono.Cecil.ModuleDefinition.ReadSymbols(ISymbolReader reader, Boolean throwIfSymbolsAreNotMaching)
   at ICSharpCode.Decompiler.PdbProvider.Cecil.MonoCecilDebugInfoProvider..ctor(PEFile module, String pdbFileName, String description)
   at ICSharpCode.ILSpy.LoadedAssembly.LoadSymbols(PEFile module)
   at ICSharpCode.ILSpy.LoadedAssembly.LoadAssembly(Object state)
   at System.Threading.Tasks.Task`1.InnerInvoke()
   at System.Threading.Tasks.Task.Execute()

not-bad-presentationcore.zip

What is the correct way to call a generic method on a generic type?

Calling a method on a specialized generic type is done by obtaining a normal method definition and then wrapping it in a Microsoft.Cci.MutableCodeModel.SpecializedMethodDefinition with the ContainingTypeDefinition property set to the specialized generic type. This works fine for ordinary methods, but if I try to do it with a generic method, it breaks down.

PeWriter.GetMethodToken checks to see if this is a IGenericMethodInstanceReference, which it isn't (but the underlying UnspecializedVersion is!) and so instead of emitting a MethodSpec token, it ends up emitting a MemberRef token, resulting in a corrupt binary.

What's the right way to get a MethodSpec here?

Switch NuGet package to NetStandard 1.1 (or 1.2)

I've noticed that the latest NuGet packages (https://www.nuget.org/packages/Microsoft.Cci/4.0.0-rc3-24214-00 and https://dotnet.myget.org/feed/dotnet-core/package/nuget/Microsoft.Cci/4.0.0-rc4-24220-00) use NetStandard 1.3 (i.e. .NET 4.6) as minimum requirement while the project on GitHub still only requires .NET 4.0. Please publish a new version with a reduced version requirement since it appears that .NET 4.6 is not actually needed and significantly limits the use of Microsoft.Cci. Personally, I'll have to repackage the assembly and host it on my own to get around this issue.

I'd love to send a PR that fixes this, but it looks like there's no nuspec file where this could be configured uploaded on GitHub.

Collectible assembly generation

I'm working on porting a compiler over from Reflection.Emit to CCI. One of the things that this compiler is able to do is generate assemblies in memory, for use as an interpreter. With Reflection.Emit it's trivial to mark an assembly as collectible, but I don't see any equivalent here. What's the recommended way to create collectible assemblies with CCI?

PDB Writer

[Question] CI Builds?

I am running into the OutOfMemoryException when trying to use PdbReader with a pdb created using the new .NET core project types. I noticed that there are new checkins that have fixed this issue. Is there a NuGet feed I could use to test these builds? The latest build on https://dotnet.myget.org/feed/dotnet-core/package/nuget/Microsoft.Cci is a year old. :/

Thanks!