crate_universe: Support git

[uhthomas]

Maybe I'm missing something, but when using git repositories in Cargo.toml, create_universe throws an error.

[dependencies]
regex = "1.5.4"
rure = {git = "https://github.com/rust-lang/regex", rev = "1.5.4"}

load("@rules_rust//crate_universe:bootstrap.bzl", "crate_universe_bootstrap")

crate_universe_bootstrap()

load("@rules_rust//crate_universe:defs.bzl", "crate_universe")

crate_universe(
    name = "crates",
    cargo_toml_files = ["//:Cargo.toml"],
    resolver = "@rules_rust_crate_universe_bootstrap//:crate_universe_resolver",
)

ERROR: An error occurred during the fetch of repository 'crates':
   Traceback (most recent call last):
        File "/private/var/tmp/_bazel_thomas/b6a3c999aae05297a54a9aa13d7a50c3/external/rules_rust/crate_universe/defs.bzl", line 147, column 13, in _crate_universe_resolve_impl
                fail("Error resolving crate_universe deps - see above output for more information")
Error in fail: Error resolving crate_universe deps - see above output for more information
ERROR: Error fetching repository: Traceback (most recent call last):
        File "/private/var/tmp/_bazel_thomas/b6a3c999aae05297a54a9aa13d7a50c3/external/rules_rust/crate_universe/defs.bzl", line 147, column 13, in _crate_universe_resolve_impl
                fail("Error resolving crate_universe deps - see above output for more information")
Error in fail: Error resolving crate_universe deps - see above output for more information

---

Additionally, is it possible to fetch private git repositories? cargo-raze uses Bazel's git rules which circumvents rust-lang/cargo#1851.

[shikhar]

try a commit hash instead of tag

[illicitonion]

I think the problem here is that you're trying to rename the dependency, which isn't supported by crate_universe; this works:

regex = {git = "https://github.com/rust-lang/regex", rev = "1.5.4"}

but this doesn't:

rure = {git = "https://github.com/rust-lang/regex", rev = "1.5.4"}

Additionally, is it possible to fetch private git repositories?

Yes, but you'll need to configure git in such a way that auth works - I think setting CARGO_NET_GIT_FETCH_WITH_CLI=true will do that - I don't think we have a way of conveniently consistently applying that (if it's set in your env, it'll get picked up), but we could probably make it easier to configure if that works for you?

[uhthomas]

To clarify, I'm currently using cargo-raze and the Cargo.toml works okay.

The line importing rure is not renaming the crate, it's specifically importing Rust's regex C API. See https://github.com/rust-lang/regex/blob/master/regex-capi/Cargo.toml#L2.

Admittedly, this is a very weird crate.

There is an entry for rure on crates.io but it looks quite out of date, so I wanted to compile it from scratch. I could be wrong!

it would be really great to see native support for fetching repos using the native git CLI. It's such a common workflow which is seemingly overlooked. Especially as it's inconsistent with the behaviour of other repository rules like bazel-gazelle which work perfectly fine.

Side note: Is it possible to get some better errors? Error resolving crate_universe deps - see above output for more information says nothing and implies that there should be some more information, even though there isn't.

Great work with the crate_universe repository rule by the way! It will make using rust with Bazel much easier.

[illicitonion]

Aha, right you are! So, the crux of the rure problem, I think, is that it's defined to be a crate-type we don't understand: https://github.com/rust-lang/regex/blob/5197f21287344d2994f9cf06758a3ea30f5a26c3/regex-capi/Cargo.toml#L18

Which we handle here:

rules_rust/crate_universe/src/templates/BUILD.crate.bazel.template

Lines 26 to 40 in 53b1de6

	{%- for target in crate.targets %}
	{%- set target_name_sanitized = target.name | replace(from="-", to="_") %}
	{%- if target.kind == "lib" %}
	{% include "templates/partials/rust_library.template" %}
	{%- elif target.kind == "bin" %}
	{% include "templates/partials/rust_binary.template" %}
	{%- elif target.kind == "proc-macro" %}
	{% include "templates/partials/rust_library.template" %}
	{%- elif target.kind == "dylib" %}
	{% include "templates/partials/rust_library.template" %}
	{%- elif target.kind == "rlib" %}
	{% include "templates/partials/rust_library.template" %}
	{%- else %}
	# Unsupported target "{{ target.name }}" with type "{{ target.kind }}" omitted
	{%- endif %}

As far as I can tell, cargo-raze does exactly the same target skipping:
https://github.com/google/cargo-raze/blob/0bc9b000d3cbbfde8ff2df02001a80faaa771552/impl/src/rendering/templates/crate.BUILD.template#L32-L52

and indeed using this Cargo.toml:

[package]
name = "rustplay"
version = "0.1.0"
edition = "2018"

[dependencies]
rure = {git = "https://github.com/rust-lang/regex", rev = "1.5.4"}

[package.metadata.raze]
workspace_path = "//cargo"

package_aliases_dir = "." 

targets = [ 
    "x86_64-apple-darwin",
    "x86_64-pc-windows-msvc",
    "x86_64-unknown-linux-gnu",
]

genmode = "Remote"

Running cargo-raze at HEAD generates this same BUILD file, which shouldn't be useful:

"""
@generated
cargo-raze crate build file.

DO NOT EDIT! Replaced on runs of cargo-raze
"""

# buildifier: disable=load
load("@bazel_skylib//lib:selects.bzl", "selects")

# buildifier: disable=load
load(
    "@rules_rust//rust:defs.bzl",
    "rust_binary",
    "rust_library",
    "rust_proc_macro",
    "rust_test",
)

package(default_visibility = [ 
    # Public for visibility by "@raze__crate__version//" targets.
    #   
    # Prefer access through "//cargo", which limits external
    # visibility to explicit Cargo.toml dependencies.
    "//visibility:public",
])

licenses([
    "notice",  # MIT from expression "MIT OR Apache-2.0"
])

# Generated Targets

# Unsupported target "rure" with type "cdylib" omitted

# Unsupported target "rure" with type "staticlib" omitted

[illicitonion]

Now, back to the git auth issue! @gibfahn put together a fantastic guide to making this Just Work after they spent quite a while delving into it: https://fahn.co/posts/cargo-auth-for-private-git-repos.html - possibly some of the content there may be useful to you? I definitely agree that it's sad that this stuff doesn't Just Work out of the box though, and if you have any insights as to how we could make it work more smoothly, I'd be very interested in working with you on them!

[shikhar]

trying to rename the dependency, which isn't supported by crate_universe

#904 is an open issue about that

[UebelAndre]

The new implementation of crate_universe supports git. I think the cargo aliases example demonstrates this. I'm gonna close this out for now but feel free to reopen if you find git support doesn't actually work. Try using release 0.1.0 though.