How does dependabot know a nuget dependency is a "development" dependency?

[BinToss]

I've taken a quick look through the NugetUpdater helper projects, but I couldn't find an answer within 30 minutes.

The only information I could find was the old pull requests that added support for NuGet development dependencies when the implementation of NuGet ecosystem support was still Ruby-only. And that implementation seems incorrect. MSBuild/dotnet/et cetera does not have any implementation for "DevelopmentDependency" items as variants of PackageReference items.

This faulty support may have been implemented this way due to the confusing and...poorly written documentation for NuGet's own DevelopmentDependency specification. I spent a few hours to cross-reference that spec doc to real world examples and what little other documentation is available to write this.

As stated in that write-up, setting the project configuration property "DevelopmentDependency" to true will do nothing but set the corresponding Nuspec property "developmentDependency" to true. When a .nupkg's .nuspec has this property set to true and a package manager adds the package as a project dependency, the resulting dependency is written to the (SDK-style) project file as a PackageReference with two changes:

• The PackageReference's PrivateAssets metadata (xml attr or child element) is set to "all", the package dependency will not flow transitively to parent projects.
• The PackageReference's ExcludeAssets (default: "none") is set to "compile" if the nupkg has a libs directory containing .dll files. In some cases, this is written to set IncludeAssets (default: "all") to "runtime; build; native; contentfiles; analyzers". Those .dll files are libraries the project is built against and will require at runtime. They may be included as-is, trimmed down, and/or weaved into a single executable or library for distribution.

---

Search Keywords:
deps-dev nuget

[BinToss]

2024-07-28 | Analysis - Part 1

CompatibilityChecker.CheckAsync gets an isDevDependency bool from CompatibilityChecker.GetPackageInfoAsync.

dependabot-core/nuget/helpers/lib/NuGetUpdater/NuGetUpdater.Core/Analyze/CompatabilityChecker.cs

Lines 20 to 33 in 16f4c43

	public static async Task<bool> CheckAsync(
	PackageIdentity package,
	ImmutableArray<NuGetFramework> projectFrameworks,
	NuGetContext nugetContext,
	Logger logger,
	CancellationToken cancellationToken)
	{
	var (isDevDependency, packageFrameworks) = await GetPackageInfoAsync(
	package,
	nugetContext,
	cancellationToken);
	return PerformCheck(package, projectFrameworks, isDevDependency, packageFrameworks, logger);
	}

In the body of CompatibilityChecker.GetPackageInfoAsync, isDevDependency is true if one of two conditions is met.

• The package's nuspec contains <developmentDependency>true</developmentDependency>. See https://github.com/NuGet/NuGet.Client/blob/5485ea697de98eee58746e0b0054cd478e33a1a5/src/NuGet.Core/NuGet.Packaging/Core/NuspecCoreReaderBase.cs#L123-L127. The package author may explicitly assign false to this property, but this is ignored if...
• The package has no TFMs (target frameworks) and no libs.
  

  dependabot-core/nuget/helpers/lib/NuGetUpdater/NuGetUpdater.Core/Analyze/CompatabilityChecker.cs

  Lines 73 to 113 in 16f4c43

  	internal static async Task<PackageInfo> GetPackageInfoAsync(
  	PackageIdentity package,
  	NuGetContext nugetContext,
  	CancellationToken cancellationToken)
  	{
  	var tempPackagePath = GetTempPackagePath(package, nugetContext);
  	var readers = File.Exists(tempPackagePath)
  	? ReadPackage(tempPackagePath)
  	: await DownloadPackageAsync(package, nugetContext, cancellationToken);
  	var nuspecStream = await readers.CoreReader.GetNuspecAsync(cancellationToken);
  	var reader = new NuspecReader(nuspecStream);
  	var isDevDependency = reader.GetDevelopmentDependency();
  	var tfms = reader.GetDependencyGroups()
  	.Select(d => d.TargetFramework)
  	.ToImmutableArray();
  	if (tfms.Length == 0)
  	{
  	// If the nuspec doesn't have any dependency groups,
  	// try to get the TargetFramework from files in the lib folder.
  	var libItems = (await readers.ContentReader.GetLibItemsAsync(cancellationToken)).ToList();
  	if (libItems.Count == 0)
  	{
  	// If there is no lib folder in this package, then assume it is a dev dependency.
  	isDevDependency = true;
  	}
  	tfms = libItems.Select(item => item.TargetFramework)
  	.Distinct()
  	.ToImmutableArray();
  	}
  	// The interfaces we given are not disposable but the underlying type can be.
  	// This will ensure we dispose of any resources that need to be cleaned up.
  	(readers.CoreReader as IDisposable)?.Dispose();
  	(readers.ContentReader as IDisposable)?.Dispose();
  	return (isDevDependency, tfms);
  	}

CompatibilityChecker.CheckAsync passes isDevDependency to CompatibilityChecker.PerformCheck wherein the method returns true if isDevDependency && packageFrameworks.Length == 0, among other things. packageFrameworks is an ImmutableArray<NuGetFramework> which are derived from the package the same way they are on NuGet.org.

dependabot-core/nuget/helpers/lib/NuGetUpdater/NuGetUpdater.Core/Analyze/CompatabilityChecker.cs

Lines 35 to 49 in 16f4c43

	internal static bool PerformCheck(
	PackageIdentity package,
	ImmutableArray<NuGetFramework> projectFrameworks,
	bool isDevDependency,
	ImmutableArray<NuGetFramework> packageFrameworks,
	Logger logger)
	{
	// development dependencies are packages such as analyzers which need to be compatible with the compiler not the
	// project itself, but some packages that report themselves as development dependencies still contain target
	// framework dependencies and should be checked for compatibility through the regular means
	if (isDevDependency && packageFrameworks.Length == 0)
	{
	return true;
	}

Following up the call chain(s), you end up at NuGetUpdater.Core.Analyze.AnalyzeWorker.RunAsync

Examples:

• https://www.nuget.org/packages/GitVersion.MsBuild#supportedframeworks-body-tab. This package matches both conditions for isDevDependency. Its nuspec has <developmentDependency>true</developmentDependency> and it has no TFMs and no libs.

---

From what I can tell, whether or not a package is a development dependency is determined by a package dependency. Metadata which can be used to override this is not checked. A project may use non-development-dependency packages as development dependencies, but NuGetUpdater does not check for this. Primarily because there is no standard in the NuGet ecosystem for a dependent to explicitly state a dependency will only be used as a development dependency. It doesn't help that there are multiple types of dependencies.

This is in stark contrast to the NPM ecosystem in which a project (i.e. package.json) categorizes dependencies in multiple ways:

• dependencies
• devDependencies
• peerDependencies (enhanced by separate peerDependenciesMeta)
• bundleDependencies
• optionalDependencies

Additionally, I still have not found any location at which a package's development dependency boolean is used by the commit message writer.

---

2024-07-31 | Analysis - Part 2

@brettfo

Your analysis in the previous comment is correct; we report something as a development dependency if we can't find any runtime files (e.g., anything under lib/, etc.)

As for surfacing that in the commit message, I'll have to yield to the core dependabot team, I'm not sure how that part of the code works.

I found it.

Dependabot::PullRequestCreator::MessageBuilder.dependencies

dependabot-core/common/lib/dependabot/pull_request_creator/message_builder.rb

Lines 31 to 32 in 8296fc2

	sig { returns(T::Array[Dependabot::Dependency]) }
	attr_reader :dependencies

Dependabot::Dependency.production?. If false, it's a dev dep.

dependabot-core/common/lib/dependabot/dependency.rb

Lines 180 to 189 in 8296fc2

	sig { returns(T::Boolean) }
	def production?
	return subdependency_production_check unless top_level?
	groups = requirements.flat_map { |r| r.fetch(:groups).map(&:to_s) }
	self.class
	.production_check_for_package_manager(package_manager)
	.call(groups)
	end

If top-level...

dependabot-core/common/lib/dependabot/dependency.rb

Lines 191 to 194 in 8296fc2

	sig { returns(T::Boolean) }
	def subdependency_production_check
	!subdependency_metadata&.all? { |h| h[:production] == false }
	end

If NOT top-level...

dependabot-core/common/lib/dependabot/dependency.rb

Lines 18 to 26 in 8296fc2

	sig do
	params(package_manager: String).returns(T.proc.params(arg0: T::Array[T.untyped]).returns(T::Boolean))
	end
	def self.production_check_for_package_manager(package_manager)
	production_check = @production_checks[package_manager]
	return production_check if production_check
	raise "Unsupported package_manager #{package_manager}"
	end

Dependabot NuGet implementation

NuGet checks the dependency groups for a group whose name is "dependencies" and not "devDependencies".

dependabot-core/nuget/lib/dependabot/nuget.rb

Lines 18 to 26 in 8296fc2

	require "dependabot/dependency"
	Dependabot::Dependency.register_production_check(
	"nuget",
	lambda do |groups|
	return true if groups.empty?
	groups.include?("dependencies")
	end
	)

See Line 117. A dependency is grouped in "devDependenciesifNativeDependencyFileDiscoveryfindsdependency_details.is_dev_dependency` is true. Else, the dependency is grouped in "dependencies".

dependabot-core/nuget/lib/dependabot/nuget/native_discovery/native_dependency_file_discovery.rb

Lines 104 to 127 in 8296fc2

	sig do
	params(file_name: String, dependency_details: NativeDependencyDetails)
	.returns(T.nilable(T::Hash[Symbol, T.untyped]))
	end
	def build_requirement(file_name, dependency_details)
	return if dependency_details.is_transitive
	version = dependency_details.version
	version = nil if version&.empty?
	requirement = {
	requirement: version,
	file: file_name,
	groups: [dependency_details.is_dev_dependency ? "devDependencies" : "dependencies"],
	source: nil
	}
	property_name = dependency_details.evaluation&.root_property_name
	return requirement unless property_name
	requirement[:metadata] = { property_name: property_name }
	requirement
	end
	end

NativeDependencyDetails directly correlates to NuGetUpdater.Core.Dependency

Dependabot::PullRequestCreator::MessageBuilder.pr_name_prefixer calls Dependabot::PullRequestCreator::PrNamePrefixer.

dependabot-core/common/lib/dependabot/pull_request_creator/message_builder.rb

Lines 809 to 822 in 8296fc2

	sig { returns(Dependabot::PullRequestCreator::PrNamePrefixer) }
	def pr_name_prefixer
	@pr_name_prefixer ||=
	T.let(
	PrNamePrefixer.new(
	source: source,
	dependencies: dependencies,
	credentials: credentials,
	commit_message_options: commit_message_options,
	security_fix: vulnerabilities_fixed.values.flatten.any?
	),
	T.nilable(Dependabot::PullRequestCreator::PrNamePrefixer)
	)
	end

Dependabot::PullRequestCreator::PrNamePrefixer.commit_prefix

dependabot-core/common/lib/dependabot/pull_request_creator/pr_name_prefixer.rb

Lines 91 to 103 in 8296fc2

	sig { returns(T.nilable(String)) }
	def commit_prefix
	# If a preferred prefix has been explicitly provided, use it
	return prefix_from_explicitly_provided_details if commit_message_options&.key?(:prefix)
	# Otherwise, if there is a previous Dependabot commit and it used a
	# known style, use that as our model for subsequent commits
	return prefix_for_last_dependabot_commit_style if last_dependabot_commit_style
	# Otherwise we need to detect the user's preferred style from the
	# existing commits on their repo
	build_commit_prefix_from_previous_commits
	end

Dependabot::PullRequestCreator::PrNamePrefixer.scope
If nuget updates are configured with commit-message: { include: scope }, then this scope property determines the commit message's scope. It checks if any of the affected dependencies are production?. If true, the scope is deps. Else, deps-dev.

dependabot-core/common/lib/dependabot/pull_request_creator/pr_name_prefixer.rb

Lines 164 to 166 in 8296fc2

	def scope
	dependencies.any?(&:production?) ? "deps" : "deps-dev"
	end

[brettfo]

Your analysis in the previous comment is correct; we report something as a development dependency if we can't find any runtime files (e.g., anything under lib/, etc.)

As for surfacing that in the commit message, I'll have to yield to the core dependabot team, I'm not sure how that part of the code works.