Expose target wildcard matching to Starlark

[aherrmann]

Description of the feature request:

Please expose Bazel's capability of matching target labels against wildcards to Starlark. Given a list of target patterns and a list of targets, return the list of all targets that match any of the given patterns. Patterns should have the same feature set as patterns allowed on the command line. E.g. specific //package:target, wild-card //package:all, or //package/.... Optionally, it could allow patterns relative to a specific target, e.g. :all matching all targets in the same package. Here's a suggestion of what the API might look like:

def match_targets(patterns, targets, relative_to = None):
    """
    Args:
      patterns: List of patterns.
      targets: List of target labels.
      relative_to: Match relative patterns relative to this target.
    
    Returns:
      List of labels contained in `targets` that match any of the given `patterns`.

    """

Feature requests: what underlying problem are you trying to solve with this feature?

The use-case that prompts this feature request comes from rules_haskell, where we would like to implement a REPL rule based on an aspect with the following interface:

haskell_repl(
    name = "my-repl",
    # Collect all transitive Haskell dependencies from these targets.
    deps = [
        "//package-a:target-1",
        "//package-b:target-2",
    ],
    # Load targets by source that match these patterns.
    from_source = [
        "//package-a/...",
        "//packaga-b/...",
        "//common/...",
    ],
    # Don't load targets by source that match these patterns, instead load their binaries.
    from_binary = [
        "//package-a/vendored/...",
    ],
)

See below for some background and motivation.

Currently, we re-implement a subset of Bazel's target pattern matching in Starlark to enable this interface. This is duplicated effort that should not be necessary. And we would prefer to offer our users Bazel's full pattern capabilities.

Note, that we cannot use genquery in this case, because it does not allow wild-cards, and also does not take configuration into account (query vs. cquery).

Background on Haskell REPL

With Haskell's REPL there is an important distinction between loading a target by source or as binary. If you load by source, this allows you to modify the source file, instruct the REPL to reload sources, and observe your code changes in the REPL right away, allowing for a fast feedback loop. Loading by binary instead means that code changes only become visible if you close the REPL, rebuild the binaries, and reopen the REPL. In practice this is too slow for productive development.

However, some targets should not be loaded from source, but instead as binary. Reasons for this could be that they require compiler flags that are incompatible with other modules, or you're not interested in modifying their source and want the speed benefit of loading them as binary.

What operating system are you running Bazel on?

NixOS

What's the output of bazel info release?

release 0.23.1- (@non-git)

If bazel info release returns "development version" or "(@non-git)", tell us how you built Bazel.

Using the Nix package manager.

Have you found anything relevant by searching the web?

Only genquery which is not applicable.

[laurentlb]

Currently, we re-implement a subset of Bazel's target pattern matching in Starlark to enable this interface. This is duplicated effort that should not be necessary. And we would prefer to offer our users Bazel's full pattern capabilities.

Can you detail which capabilities you'd like?

In the example:

    from_source = [
        "//package-a/...",
    ],

It seems like you only need to check a prefix: str.startswith("//package-a").

[aherrmann]

@laurentlb At the moment our own implementation supports absolute labels in the local workspace, e.g.
//some/package:some_target, with the wildcards ..., :all, and :*. Also the //some/package shortcut is implemented. Relative labels are not implemented. Relative labels would be nice. Otherwise, for the time being that seems sufficient.

Note, I'm not asking to add additional pattern matching capabilities to Bazel. Instead, my point is that this kind of pattern matching is something that Bazel already implements internally. It would be better if rules could just use Bazel's builtin pattern matching to make sure that the set of supported patterns is identical to what users expect from their experience with Bazel itself, and also so that the pattern matching stays in sync if it's ever extended in Bazel.

[adam-azarchs]

This would be useful for my use case as well. I want to create a test rule to verify that every file in some set of files in the source tree is depended on, at least transitively, by some test rule, in order to enforce that developers don't forget to update BUILD files when appropriate.

I can do this from the command line easily enough with bazel query but executing that within a test running on bazel doesn't work, as the work tree is locked by the parent bazel process. genquery as currently implemented doesn't work either, because the inherent nature of the test means we can't know the query scope in advance. Asking developers to add test_suite targets to every new package and then add those to the parent packages going up the tree would be a prohibitive burden for our organization. With the current state of affairs our choices seem to be either that or moving away from the goal of being able to say "if bazel test //... passes then the CI should pass too."

[martis42]

This feature would be helpful to me as well. I would like to write a rule which analyzes all packages below a given path.
Easy enough to do so via aspects/scripts and the command line, but sadly right now not possible as proper executable rule.

[github-actions]

Thank you for contributing to the Bazel repository! This issue has been marked as stale since it has not had any activity in the last 1+ years. It will be closed in the next 90 days unless any other activity occurs or one of the following labels is added: "not stale", "awaiting-bazeler". Please reach out to the triage team (@bazelbuild/triage) if you think this issue is still relevant or you are interested in getting the issue resolved.

[github-actions]

This issue has been automatically closed due to inactivity. If you're still interested in pursuing this, please post @bazelbuild/triage in a comment here and we'll take a look. Thanks!

[aaomidi]

Is this something that can be reopened?