Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

CopyOutputSymbolsToPublishDirectory is not respected #92188

Closed
MichalStrehovsky opened this issue Sep 17, 2023 · 5 comments · Fixed by #92315
Closed

CopyOutputSymbolsToPublishDirectory is not respected #92188

MichalStrehovsky opened this issue Sep 17, 2023 · 5 comments · Fixed by #92315

Comments

@MichalStrehovsky
Copy link
Member

It used to be the case that CopyOutputSymbolsToPublishDirectory=false would prevent copying PDB/symbols to the publish folder, but it looks like we dropped it in #89358.

Cc @agocke

@ghost ghost added the untriaged New issue has not been triaged by the area owner label Sep 17, 2023
@ghost
Copy link

ghost commented Sep 17, 2023

Tagging subscribers to this area: @agocke, @MichalStrehovsky, @jkotas
See info in area-owners.md if you want to be subscribed.

Issue Details

It used to be the case that CopyOutputSymbolsToPublishDirectory=false would prevent copying PDB/symbols to the publish folder, but it looks like we dropped it in #89358.

Cc @agocke

Author: MichalStrehovsky
Assignees: -
Labels:

area-NativeAOT-coreclr

Milestone: -

@agocke agocke added this to AppModel Sep 18, 2023
@agocke agocke assigned LakshanF and unassigned LakshanF Sep 19, 2023
@ghost ghost added the in-pr There is an active PR which will close this issue when it is merged label Sep 20, 2023
agocke added a commit to agocke/runtime that referenced this issue Sep 20, 2023
I accidentally removed this property from AOT compilation
when adding support for Mac dsym bundles. This change re-enables
support for suppressing debugging symbols in the output.

Fixes dotnet#92188
@richlander
Copy link
Member

Could we consider PublishDebugType or PublishSymbols as the convenience version of that. That property is a mouthful and bordering on sausage making.

@jkotas
Copy link
Member

jkotas commented Sep 20, 2023

The current plan where we simply drop the symbols is not great. It produces containers that are small, but they cannot be run with debugger attached and the crash dumps cannot be analyzed.

One way to solve this issue is to create two containers: Smaller production container without the symbols that is used for production, and layer over the small container that adds the symbols and that can be used to debug the app or analyze crash dumps.

How is this problem typically solved in other languages that produce fully native binaries (C/C++, Rust, golang, ...)?

@MichalStrehovsky
Copy link
Member Author

Could we consider PublishDebugType or PublishSymbols as the convenience version of that. That property is a mouthful and bordering on sausage making.

We didn't invent this name - this has been named like this since forever. The bug here is just that we don't respect the property for PublishAot (it is respected for other kinds of publishes).

If we want to create an alias, it would have to be a separate issue for the SDK repo. I don't know what's our policy on MSBuild aliases. Might be more palatable to expose it as an additional switch to dotnet publish (and e.g. we could add an option to dump the symbols to a different directory as well).

How is this problem typically solved in other languages that produce fully native binaries (C/C++, Rust, golang, ...)?

I tried to search around and asked bing chat as well but came empty handed. E.g. CMake release builds default to not generating any debug info, one needs to choose a RelWithDebInfo configuration to get that. So maybe it's solved by giving a blank stare if someone wants to debug a crash dump.

@richlander
Copy link
Member

@jkotas raises a great point. Seems like we should develop a pattern that we like and productize it (and validate it).

The smaller/bigger option would work in some scenarios but not others. For examples, Kubernetes deployments are immutable so you cannot deploy another container to a pod after the fact. The same is likely true for more bespoke services like Azure App Service. For dump debugging, that would work.

In terms of authoring, you could do one of two things, assuming Dockerfile.

  • Use two stages and build two different ways, relying on the builds being the same.
  • Build the app once to product the big image. docker build a second time, FROM the first image that just harvests the app (not symbols) from the big image and then copies those to the final base image.

agocke added a commit that referenced this issue Sep 20, 2023
I accidentally removed this property from AOT compilation
when adding support for Mac dsym bundles. This change re-enables
support for suppressing debugging symbols in the output.

Fixes #92188
@ghost ghost removed in-pr There is an active PR which will close this issue when it is merged untriaged New issue has not been triaged by the area owner labels Sep 20, 2023
github-actions bot pushed a commit that referenced this issue Sep 20, 2023
I accidentally removed this property from AOT compilation
when adding support for Mac dsym bundles. This change re-enables
support for suppressing debugging symbols in the output.

Fixes #92188
carlossanlop pushed a commit that referenced this issue Sep 29, 2023
* Bring back CopyOutputSymbolsToPublishDirectory

I accidentally removed this property from AOT compilation
when adding support for Mac dsym bundles. This change re-enables
support for suppressing debugging symbols in the output.

Fixes #92188

* Update src/coreclr/nativeaot/BuildIntegration/Microsoft.NETCore.Native.Publish.targets

Co-authored-by: Michal Strehovský <[email protected]>

---------

Co-authored-by: Andy Gocke <[email protected]>
Co-authored-by: Andy Gocke <[email protected]>
Co-authored-by: Michal Strehovský <[email protected]>
Artromskiy added a commit to Artromskiy/runtime that referenced this issue Oct 4, 2023
commit a5b75b8
Author: Jakob Botsch Nielsen <[email protected]>
Date:   Wed Sep 20 22:04:58 2023 +0200

    JIT: Fix invalid containment of vector broadcasts (dotnet#92333)

    The containment checks for vector broadcasts were missing a size check,
    meaning that a uint broadcast could contain a ubyte/ushort indirection.
    That would lead to out-of-bounds reads.

    Fix dotnet#83387

commit 614d864
Author: Stephen Toub <[email protected]>
Date:   Wed Sep 20 15:56:37 2023 -0400

    Use Utf8JsonWriterCache in JsonNode.To{Json}String (dotnet#92358)

commit c0b5150
Author: Andy Gocke <[email protected]>
Date:   Wed Sep 20 12:46:37 2023 -0700

    Bring back CopyOutputSymbolsToPublishDirectory (dotnet#92315)

    I accidentally removed this property from AOT compilation
    when adding support for Mac dsym bundles. This change re-enables
    support for suppressing debugging symbols in the output.

    Fixes dotnet#92188

commit b4be77b
Author: Kunal Pathak <[email protected]>
Date:   Wed Sep 20 10:17:22 2023 -0700

    Update the assert for BlendVariable (dotnet#92183)

    * Update the assert for BlendVariable

    * Add test cases

    * Add Sse41.IsSupported check

commit e235aef
Author: Miha Zupan <[email protected]>
Date:   Wed Sep 20 17:45:31 2023 +0200

    Set severity of rule CA1870 to warning (dotnet#92135)

    * Set severity of rule CA1870 to warning

    * Replace one more usage in nativeaot corelib

    * Set severity for tests as well

    * pragma disable the rule in nativeaot's reflection impl

commit 901f780
Author: Ilona Tomkowicz <[email protected]>
Date:   Wed Sep 20 17:45:01 2023 +0200

    [wasm][debugger] Add tests for indexing by object schema (dotnet#92268)

    * Indexing with object: works.

    * Update expected line numbers.

commit d6ff465
Author: Johan Lorensson <[email protected]>
Date:   Wed Sep 20 17:24:39 2023 +0200

    Add missing case for constrained gsharedvt call. (dotnet#92338)

    dotnet@1b788f4
    added a new value to our MonoRgctxInfoType enum type, but appears
    that all cases where not full adjusted. Running System.Buffers tests
    in full AOT hits the assert in info_equal about missing case,
    https://github.com/dotnet/runtime/blob/0dc5903679606b072adac70a268cdb77d1147b3e/src/mono/mono/mini/mini-generic-sharing.c#L2908.

    This commit adds the new enum value and align handling similar to other cases added
    by that commit.

commit 36ab905
Author: dotnet-maestro[bot] <42748379+dotnet-maestro[bot]@users.noreply.github.com>
Date:   Wed Sep 20 09:51:37 2023 -0500

    Update dependencies from https://github.com/dotnet/installer build 20230919.3 (dotnet#92339)

    Microsoft.Dotnet.Sdk.Internal
     From Version 9.0.100-alpha.1.23464.17 -> To Version 9.0.100-alpha.1.23469.3

    Co-authored-by: dotnet-maestro[bot] <dotnet-maestro[bot]@users.noreply.github.com>

commit 32c3355
Author: Stephen Toub <[email protected]>
Date:   Wed Sep 20 09:42:27 2023 -0400

    Fix downlevel build break in TensorPrimitives (dotnet#92269)

    * Fix downlevel build break in TensorPrimitives

    * Make net6.0 Tensors use ns2.0 implementation

    * Add net6.0 and net7.0 to Tensors temporarily since those are shipping in 8.0 branch

    * Only build net6.0 and net7.0 Tensors when not in source-build

    ---------

    Co-authored-by: Eric StJohn <[email protected]>

commit e8c3052
Author: Matt Thalman <[email protected]>
Date:   Wed Sep 20 07:45:55 2023 -0500

    Update Newtonsoft.Json from 13.0.1 to 13.0.3 (dotnet#92298)

commit b4912a7
Author: Zoltan Varga <[email protected]>
Date:   Wed Sep 20 08:12:52 2023 -0400

    [wasi] Fix llvm target triple. (dotnet#92256)

commit 0dc5903
Author: Artur Zgodziński <[email protected]>
Date:   Wed Sep 20 11:45:46 2023 +0100

    Fix trimming of DebuggerDisplay with Name (dotnet#92191)

    The `Name` and `Type` property of the `DebuggerDisplay` attribute accepts the
    same format string as its `Value` property, but does not prevent
    trimming members it references. Thanks to this fix, members referenced by
    any of these two properties are not trimmed and can be displayed by a
    debugger.

commit 521e1e6
Author: Marie Píchová <[email protected]>
Date:   Wed Sep 20 12:28:18 2023 +0200

    [QUIC] Throw ODE if connection/listener is disposed (dotnet#92215)

    * AcceptConnection/StreamAsync now throw ODE in case the listener/connection was stopped by DisposeAsync.

    * Fix exception type and make behavior stable for disposal

commit d411f50
Author: Stephen Toub <[email protected]>
Date:   Wed Sep 20 06:24:58 2023 -0400

    Avoid unnecessary array allocation in JsonHelpers.Utf8GetString on netstandard (dotnet#92304)

commit 5883b72
Author: Tarek Mahmoud Sayed <[email protected]>
Date:   Tue Sep 19 19:52:38 2023 -0700

    Fix options Validation with objects have indexers (dotnet#92309)

commit fcf7b11
Author: Sven Boemer <[email protected]>
Date:   Tue Sep 19 17:51:32 2023 -0700

    Prevent restoring illink for native-binplace.proj (dotnet#92289)

    Fixes dotnet#92194. The
    reference to illink from `native-binplace.proj`, built as a
    reference of `build-native.proj`, was hitting a nuget bug with
    static graph restore. The bug seems to be specific to something
    about the project file (maybe the language-specific targets,
    since `native-binplace.proj` imports the `Microsoft.NET.Sdk`, but
    doesn't have a `csproj` extension).

    Fixed by explicitly marking this as not a source project, which
    will prevent the import of illink.targets.

commit b049f42
Author: Egor Bogatov <[email protected]>
Date:   Wed Sep 20 01:39:30 2023 +0200

    Fix optSwitchConvert (dotnet#92249)

    Co-authored-by: Egor <[email protected]>

commit 41a8e39
Author: Tanner Gooding <[email protected]>
Date:   Tue Sep 19 15:09:19 2023 -0700

    Ensure VN handles both forms of the xarch shift instructions for SIMD (dotnet#91601)

commit 3b9b4fd
Author: Viktor Hofer <[email protected]>
Date:   Tue Sep 19 23:29:29 2023 +0200

    Move portable RID graph into runtime and clean-up (dotnet#92211)

    * Move portable RID graph into runtime and clean-up

    1. Move portable RID graph into runtime
    2. Allow updates to both the non-portable and portable RID graphs under
       source build.
    3. Clean-up project and remove hacks

    * Update README and delete test

    * Fix RID graph update when the key already exists

    * Update src/libraries/Microsoft.NETCore.Platforms/readme.md

    Co-authored-by: Jan Kotas <[email protected]>

    * Update src/libraries/Microsoft.NETCore.Platforms/readme.md

    Co-authored-by: Andy Gocke <[email protected]>

    ---------

    Co-authored-by: Jan Kotas <[email protected]>
    Co-authored-by: Andy Gocke <[email protected]>

commit 1185d19
Author: Tanner Gooding <[email protected]>
Date:   Tue Sep 19 13:41:15 2023 -0700

    Don't generate AddMask as it requires more explicit consideration of semantics (dotnet#92282)

commit a7cafec
Author: Carlos Sánchez López <[email protected]>
Date:   Tue Sep 19 12:14:24 2023 -0700

    [main] Bump Microsoft.Private.IntelliSense package version (dotnet#92255)

commit 094801e
Author: dotnet-maestro[bot] <42748379+dotnet-maestro[bot]@users.noreply.github.com>
Date:   Tue Sep 19 12:13:49 2023 -0700

    [main] Update dependencies from dotnet/runtime dotnet/emsdk dotnet/hotreload-utils dotnet/cecil dotnet/sdk dotnet/source-build-reference-packages (dotnet#92175)

    * Update dependencies from https://github.com/dotnet/emsdk build 20230915.3

    Microsoft.NET.Workload.Emscripten.Current.Manifest-9.0.100.Transport
     From Version 9.0.0-alpha.1.23457.3 -> To Version 9.0.0-alpha.1.23465.3

    * Update dependencies from https://github.com/dotnet/sdk build 20230915.37

    Microsoft.DotNet.ApiCompat.Task
     From Version 9.0.100-alpha.1.23465.4 -> To Version 9.0.100-alpha.1.23465.37

    * Update dependencies from https://github.com/dotnet/emsdk build 20230915.3

    Microsoft.NET.Workload.Emscripten.Current.Manifest-9.0.100.Transport
     From Version 9.0.0-alpha.1.23457.3 -> To Version 9.0.0-alpha.1.23465.3

    * Update dependencies from https://github.com/dotnet/sdk build 20230916.1

    Microsoft.DotNet.ApiCompat.Task
     From Version 9.0.100-alpha.1.23465.4 -> To Version 9.0.100-alpha.1.23466.1

    * Update dependencies from https://github.com/dotnet/emsdk build 20230915.3

    Microsoft.NET.Workload.Emscripten.Current.Manifest-9.0.100.Transport
     From Version 9.0.0-alpha.1.23457.3 -> To Version 9.0.0-alpha.1.23465.3

    * Update dependencies from https://github.com/dotnet/sdk build 20230918.4

    Microsoft.DotNet.ApiCompat.Task
     From Version 9.0.100-alpha.1.23465.4 -> To Version 9.0.100-alpha.1.23468.4

    * Update dependencies from https://github.com/dotnet/runtime build 20230916.6

    Microsoft.DotNet.ILCompiler , Microsoft.NET.ILLink.Tasks , Microsoft.NET.Sdk.IL , Microsoft.NETCore.App.Runtime.win-x64 , Microsoft.NETCore.ILAsm , runtime.native.System.IO.Ports , System.Text.Json
     From Version 9.0.0-alpha.1.23460.2 -> To Version 9.0.0-alpha.1.23466.6

    * Update dependencies from https://github.com/dotnet/source-build-reference-packages build 20230915.1

    Microsoft.SourceBuild.Intermediate.source-build-reference-packages
     From Version 8.0.0-alpha.1.23457.1 -> To Version 9.0.0-alpha.1.23465.1

    * Update dependencies from https://github.com/dotnet/emsdk build 20230915.3

    Microsoft.NET.Workload.Emscripten.Current.Manifest-9.0.100.Transport
     From Version 9.0.0-alpha.1.23457.3 -> To Version 9.0.0-alpha.1.23465.3

    * Update dependencies from https://github.com/dotnet/hotreload-utils build 20230918.2

    Microsoft.DotNet.HotReload.Utils.Generator.BuildTool
     From Version 8.0.0-alpha.0.23461.1 -> To Version 8.0.0-alpha.0.23468.2

    * Update dependencies from https://github.com/dotnet/cecil build 20230918.2

    Microsoft.DotNet.Cecil
     From Version 0.11.4-alpha.23461.1 -> To Version 0.11.4-alpha.23468.2

    * Update dependencies from https://github.com/dotnet/sdk build 20230918.31

    Microsoft.DotNet.ApiCompat.Task
     From Version 9.0.100-alpha.1.23465.4 -> To Version 9.0.100-alpha.1.23468.31

    * Update dependencies from https://github.com/dotnet/source-build-reference-packages build 20230918.3

    Microsoft.SourceBuild.Intermediate.source-build-reference-packages
     From Version 8.0.0-alpha.1.23457.1 -> To Version 9.0.0-alpha.1.23468.3

    ---------

    Co-authored-by: dotnet-maestro[bot] <dotnet-maestro[bot]@users.noreply.github.com>

commit 9bd0e0d
Author: Jeremy Koritzinsky <[email protected]>
Date:   Tue Sep 19 12:11:38 2023 -0700

    Remove "Is supported on this TFM" logic from marshalling generators and instead handle it during factory construction (dotnet#91768)

    Co-authored-by: Jackson Schuster <[email protected]>

commit 17eff3b
Author: Andy Ayers <[email protected]>
Date:   Tue Sep 19 11:37:48 2023 -0700

    JIT: generalize assert to handle SIMD64 (dotnet#92235)

    Fixes dotnet#91799.

commit 67dbbeb
Author: Andy Ayers <[email protected]>
Date:   Tue Sep 19 11:30:38 2023 -0700

    JIT: add missing xarch RMW case (dotnet#92252)

    Handle the case where we're indirectly updating a local with a value
    that is not a constant.

    Fixes dotnet#92218.
@ghost ghost locked as resolved and limited conversation to collaborators Oct 21, 2023
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Projects
Archived in project
Development

Successfully merging a pull request may close this issue.

4 participants