CSC : error CS0041: Unexpected error writing debug information -- 'Insufficient memory to continue the execution of the program.'

[TonyValenti]

Version Used:
VS 17.9.6

Steps to Reproduce:

1. I run this command: dotnet ef dbcontext optimize --context MigrationDbContext_SqlServer --namespace UniversalMigrator.Storage.Generated.SqlServer --output-dir Context.Generated\SqlServer\
2. I then try to compile my code.

Diagnostic Id:
CS0041

Expected Behavior:
The compiler does not crash.

Actual Behavior:
CSC OOMs.

Other Notes:
My PC is using 39 / 64GB or memory. I have plenty of RAM.
I'm not sure how to create a minimal repro of this.
If there are any logs or such I can provide, let me know how.
Also, I can jump on a Zoom/TEAMS with someone and give them access to my PC.

[jaredpar]

This is very likely not a real OOM scenario but likely a fragmentation issue or just bug in the DIA writing layer that the compiler is exposing. To debug though going to need to know more about what compilation is happening under the hood here.

@roji is there a way to get a binary log for the build that is happening here or surface the compilation in some way?

[roji]

/cc @AndriySvyryd, any idea how to get a binlog for dotnet ef dbcontext optimize (or some other way to diagnose this)?

[AndriySvyryd]

@roji @jaredpar OOM happens when compiling the generated code, so logs from dotnet ef dbcontext optimize wouldn't be very useful.

@TonyValenti Perhaps you can share the code generated by dotnet ef dbcontext optimize?

The current theory is that this is caused by the large number of unqualified lambdas (many of which are transformed into expression trees) in a single method.
See dotnet/efcore#33483

[TonyValenti]

Here you go.
Context.Generated.zip

[jaredpar]

@TonyValenti can you provide a compiler log of the build?

https://github.com/jaredpar/complog?tab=readme-ov-file#creating-compiler-logs

The generated code itself isn't going to be enough to track this down. Need to see it in the context of the greater build.

Note: that log will include basically all code in your project. So if it's not okay to share publicly please create a VS feedback ticket, share it there and put the link in this issue so I can assign it to my team.

[TonyValenti]

Here you go!
build.zip

[TonyValenti]

@jaredpar - Do you have everything you need?

[jaredpar]

@TonyValenti thanks for checking. I can repro this on my machine now with the provided compiler log.

[jaredpar]

@TonyValenti seems like my suspicion of this being an issue in DIA may be correct. Switching the compilation to use portable PDBs vs. Native PDBs causes the error to go away.

Short term you can flip to using <DebugType>Portable</DebugType> and that will get around the errorr

Note: is there a particular reason you're not already using portable PDBs? For .NET Core projects they're generally preferred.

[TonyValenti]

Hi @jaredpar -
I'm not explicitly setting DebugType type at all. But we do use PostSharp and it might set that (not sure though).

[jaredpar]

@TonyValenti the projects all seemingly have this import pattern at the top

	<Import Project="..\..\UniversalMigrator.Shared\__Shared\Shared.props" />

That Shared.props file has <DebugType>full</DebugType>

Note: I'm not familiar with PostSharp so unsure if this is / isn't related to that.

[jjonescz]

The method which is causing the OutOfMemoryException inside DiaSymWriter is really large, it has 2 851 608 bytes of IL instructions; 36 011 local variables; 98 782 sequence points. The method is CreateRelationalModel (generated by Entity Framework) in Context.Generated\SqlServer\MigrationDbContext_SqlServerModelBuilder.cs (Context.Generated.zip).

This is the stack trace of the exception:

SymUnmanagedWriterException:
   at Microsoft.DiaSymReader.SymUnmanagedWriterImpl.CloseMethod() in D:\roslyn-8\src\Compilers\Core\Portable\DiaSymReader\Writer\SymUnmanagedWriterImpl.cs:line 267
   at Microsoft.Cci.PdbWriter.CloseMethod(Int32 ilLength) in D:\roslyn-8\src\Compilers\Core\Portable\NativePdbWriter\PdbWriter.cs:line 640
   at Microsoft.Cci.PdbWriter.SerializeDebugInfo(IMethodBody methodBody, StandaloneSignatureHandle localSignatureHandleOpt, CustomDebugInfoWriter customDebugInfoWriter) in D:\roslyn-8\src\Compilers\Core\Portable\NativePdbWriter\PdbWriter.cs:line 154
   at Microsoft.Cci.MetadataWriter.SerializeMethodBodies(BlobBuilder ilBuilder, PdbWriter nativePdbWriterOpt, Blob& mvidStringFixup) in D:\roslyn-8\src\Compilers\Core\Portable\PEWriter\MetadataWriter.cs:line 2964
   at Microsoft.Cci.MetadataWriter.BuildMetadataAndIL(PdbWriter nativePdbWriterOpt, BlobBuilder ilBuilder, PooledBlobBuilder& mappedFieldDataBuilder, PooledBlobBuilder& managedResourceDataBuilder, Blob& mvidFixup, Blob& mvidStringFixup) in D:\roslyn-8\src\Compilers\Core\Portable\PEWriter\MetadataWriter.cs:line 1827
   at Microsoft.Cci.PeWriter.WritePeToStream(EmitContext context, CommonMessageProvider messageProvider, Func`1 getPeStream, Func`1 getPortablePdbStreamOpt, PdbWriter nativePdbWriterOpt, String pdbPathOpt, Boolean metadataOnly, Boolean isDeterministic, Boolean emitTestCoverageData, Nullable`1 privateKeyOpt, CancellationToken cancellationToken) in D:\roslyn-8\src\Compilers\Core\Portable\PEWriter\PeWriter.cs:line 111
   at Microsoft.CodeAnalysis.Compilation.SerializePeToStream(CommonPEModuleBuilder moduleBeingBuilt, DiagnosticBag metadataDiagnostics, CommonMessageProvider messageProvider, Func`1 getPeStream, Func`1 getMetadataPeStreamOpt, Func`1 getPortablePdbStreamOpt, PdbWriter nativePdbWriterOpt, String pdbPathOpt, RebuildData rebuildData, Boolean metadataOnly, Boolean includePrivateMembers, Boolean isDeterministic, Boolean emitTestCoverageData, Nullable`1 privateKeyOpt, CancellationToken cancellationToken) in D:\roslyn-8\src\Compilers\Core\Portable\Compilation\Compilation.cs:line 3361
   at Microsoft.CodeAnalysis.Compilation.SerializeToPeStream(CommonPEModuleBuilder moduleBeingBuilt, EmitStreamProvider peStreamProvider, EmitStreamProvider metadataPEStreamProvider, EmitStreamProvider pdbStreamProvider, RebuildData rebuildData, Func`2 testSymWriterFactory, DiagnosticBag diagnostics, EmitOptions emitOptions, Nullable`1 privateKeyOpt, CancellationToken cancellationToken) in D:\roslyn-8\src\Compilers\Core\Portable\Compilation\Compilation.cs:line 3260

Note: I've tried updating the Microsoft.DiaSymReader.Native.dll to 17.10.0-beta1.24272.1, that did not help.