Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

move @cImport to the build system #20630

Open
1 of 3 tasks
Tracked by #16270 ...
andrewrk opened this issue Jul 14, 2024 · 26 comments
Open
1 of 3 tasks
Tracked by #16270 ...

move @cImport to the build system #20630

andrewrk opened this issue Jul 14, 2024 · 26 comments
Labels
accepted This proposal is planned. breaking Implementing this issue could cause existing code to no longer compile or have different behavior. proposal This issue suggests modifications. If it also has the "accepted" label then it is planned. zig build system std.Build, the build runner, `zig build` subcommand, package management
Milestone

Comments

@andrewrk
Copy link
Member

andrewrk commented Jul 14, 2024

This issue is to remove the @cImport language builtin, instead relying on the feature being provided by the build system (std.Build.Step.TranslateC).

This removes one dependency on libclang inside the Zig compiler executable, which is required for #16270. In the near-term future, the translate-c implementation will still be inside the Zig compiler executable, along with the ability to compile C/C++ source files. However, we can prepare users for this transition by solving the use case via the build system rather than a language feature.

As a consolation prize, the TranslateC build step can be enhanced with advanced settings, such as namespace stripping and validation of existing bindings. Since changes to this don't require changing the language, it's OK if the scope & complexity increase to some extent.

Blockers:

Related:

@andrewrk andrewrk added breaking Implementing this issue could cause existing code to no longer compile or have different behavior. proposal This issue suggests modifications. If it also has the "accepted" label then it is planned. accepted This proposal is planned. zig build system std.Build, the build runner, `zig build` subcommand, package management labels Jul 14, 2024
@andrewrk andrewrk added this to the 0.14.0 milestone Jul 14, 2024
@mlugg
Copy link
Member

mlugg commented Jul 14, 2024

A small extra note: this eliminates one quite common use case of usingnamespace in the wild, which is to have a c.zig containing a pub usingnamespace @cImport(...) declaration. Instead, you now use a TranslateC step as described above, and directly expose the generated file to Zig code as a module.

@149-code
Copy link

@andrewrk would this mean that in order to interoperate with c, you need to use the zig build system?

I have a small personal project written in c built using make files. Im currently extending it by writing new zig code that cIncludes header files and invoking the zig compiler from make (i.e zig build-obj).

Would this use case not be supported anymore?

@alexrp
Copy link
Member

alexrp commented Jul 16, 2024

You would just have to invoke zig translate-c in your makefile; that's what std.Build.Step.TranslateC does. The compiler also does the moral equivalent of that when you use @cImport().

@mlugg
Copy link
Member

mlugg commented Jul 16, 2024

@149-code, @alexrp -- note that once #16270 is implemented, zig translate-c will no longer be a subcommand, with this functionality instead existing entirely within the build system. In that case, you probably will need to use zig build.

In theory, you can have your build.zig purely do the C translation, so zig build effectively becomes your drop-in for zig translate-c. However, this will almost certainly be more awkward than just migrating your C project to use the Zig build system.

@alexrp
Copy link
Member

alexrp commented Jul 16, 2024

@mlugg

with this functionality instead existing entirely within the build system

Hm, that sounds a bit concerning. Not looking to relitigate #16270 all over again, but I'm on board with it because the promise was that existing use cases would still be supported - it just might involve downloading an extra package for all the C/C++ functionality (which is totally fair). But if translate-c can't even be accessed outside the Zig build system, that leaves my Zig MSBuild SDK in a very awkward spot. I'm specifically integrating Zig into another build system entirely (MSBuild), so I would have to generate and invoke a build.zig which adds a whole other layer of complexity, performance penalty, and moving parts for (AFAICT) no benefit other than accessing functionality that would have previously been easy to integrate.

Please don't get me wrong; I really like the Zig build system and I use it extensively in my "pure Zig" projects. But if we start making existing features of the zig binary wholly exclusive to the Zig build system, I think that's going to significantly harm Zig adoption in brownfield or polyglot settings.

@ifreund
Copy link
Member

ifreund commented Jul 16, 2024

I'm specifically integrating Zig into another build system entirely (MSBuild), so I would have to generate and invoke a build.zig which adds a whole other layer of complexity, performance penalty, and moving parts for (AFAICT) no benefit other than accessing functionality that would have previously been easy to integrate.

I don't think any code generation would be required. You could write a single, generic build.zig that exposes a CLI pretty much identical to the current zig translate-c. The only difference is that the binary exposing the CLI would not be the zig compiler.

One could probably even write a zig package that provides a translate-c binary in every way identical to the current zig translate-c subcommand except that it is not part of the zig compiler binary.

@mlugg
Copy link
Member

mlugg commented Jul 16, 2024

What ifreund says would work, but thinking about how the translate-c package is likely to work, it might not even be necessary. I would expect the translate-c package to expose an artifact, and users construct Run steps to run that artifact (of course, we would add any necessary helper functions to make this not-awkward). So, you could zig build the translate-c package directly to get a translate-c binary.

@alexrp
Copy link
Member

alexrp commented Jul 16, 2024

That sounds reasonable. My concern here was just the prospect of having to integrate all the machinery of the Zig build system as part of the build logic in my MSBuild SDK (which is currently just a zig build-exe/build-lib/test/cc/c++ invocation at the end of the day). Stacking build systems on top of each other is guaranteed pain, in my experience.

So as long as there's a way to invoke translate-c as a plain old CLI command - whether part of zig or a binary of its own - I'm a happy camper.

The only other point I would add here is that I think it would be good to distribute such a translate-c binary as part of a "C/C++ support" binary package on ziglang.org to reduce friction for cases like @149-code's (it would also make packaging a tiny bit easier for my SDK, but that's not essential by any means).

@andrewrk
Copy link
Member Author

Well, I think you have provided a quite interesting and valuable real world use case, so, let's work together to come up with a migration proposal that is appealing before proceeding with removal of @cImport functionality.

@mlugg
Copy link
Member

mlugg commented Jul 16, 2024

To be clear @andrewrk, the use case described above doesn't depend on @cImport, only on the zig translate-c subcommand. This proposal kind of has two parts: removing @cImport seems easy and unproblematic (at least, once we improve std.Build.Step.TranslateC), while migrating zig translate-c out of the compiler is what raised concerns above.

@andrewrk
Copy link
Member Author

Ah thanks for that clarification. Well, point stands for that follow-up issue but I will remove it from the blocker list above.

@steeve
Copy link

steeve commented Jul 22, 2024

So as long as there's a way to invoke translate-c as a plain old CLI command - whether part of zig or a binary of its own - I'm a happy camper.

Adding my 2cts to heavily support this. We're using zig via bazel, so we directly invoke zig build-{exe,static} and would love to keep translate-c as a CLI.

@WeijieH
Copy link

WeijieH commented Jul 22, 2024

I really like to use the zig run command to run some of my small projects/scripts (some are using @cImport) without dropping an exe at the current directory. So, after this change, is there way I can still simply call zig run or do I have to use a build.zig file?

@andrewrk
Copy link
Member Author

andrewrk commented Jul 22, 2024

Adding my 2cts to heavily support this. We're using zig via bazel, so we directly invoke zig build-{exe,static} and would love to keep translate-c as a CLI.

That's a given - the build system will invoke the translate-c tool as a subprocess using CLI args.

So, after this change, is there way I can still simply call zig run or do I have to use a build.zig file?

No, this use case will regress.

@agluszak
Copy link

What about the "C macro reflection" feature described here and discussed on HN?

@ifreund
Copy link
Member

ifreund commented Jul 30, 2024

What about the "C macro reflection" feature described here and discussed on HN?

This is unrelated to the reflection features described there. The only change would be how the translated C header files are accessed from zig code.

@ifreund
Copy link
Member

ifreund commented Jul 30, 2024

This is such a shame. Zig's main attraction is the easy & built-in interoperability with C.

Regressing features on that front is not a good sign.

@LouisLibre Your comment does not add anything of value to this discussion. A proposal like this weighs the benefits and drawbacks against each other to decide if the zig project as a whole would be improved by the proposal or not. Merely reiterating a single drawback with no nuance or discussion as you have done is useless noise.

@MatthiasPortzel
Copy link

For “hello world from C” and bug-reproduction demos, it is useful to have C-interop without having to set up the build system. (The ongoing HN thread on this topic has several anecdotes from people convinced to try out Zig because of such simple demos.) For someone new to the language, setting up the build system at all represents significant complexity.

However this use-case could be satisfied by adding an argument to zig run, zig build-exe, and friends that allowed the user to specify C files to be auto-included as modules.

This would continue to kick the can down the road with regards to #16270, but it may make sense in the short term (allowing both zig run to include C modules and removing cImport for incremental, if that was a concern).

@ziglang ziglang deleted a comment from LouisLoyaX Jul 30, 2024
@ityonemo
Copy link
Contributor

ityonemo commented Jul 30, 2024

Keeping incorporating c header imports as a feature accessible to the CLI is important to the semantic analysis pass that happens as a part of Zigler. https://github.com/E-xyza/zigler (I'm also using this in prod and this week it's saving the bacon of the startup I'm at)

@andrewrk
Copy link
Member Author

@ityonemo can you be more specific? Seems like you should be fine with the zig translate-c subcommand, no? It's more powerful than @cImport.

@ityonemo
Copy link
Contributor

I'm fine with deprecating @cImport, but just as the "zig module import" parameters (paths, deps) can be set via command line, if a module is imported as a c ABI, id want tobe able to do so via cli without resorting to build.zig. mentioning that zig run would regress is a bit worrying. For example, in 0.12 I can run zig run --dep foo -Mxyz.... I would have imagined something like zig run --h foo.h --define true=false -Mc_lib... etc.

I'm not exactly sure how the translate-c subcommands works (it sounds like it will? Maybe?) and I could probably figure it out, but I had lots of difficulty integrating a semantic pass with build.zig that lived side-by-side with the libs I generate, since I am doing sneaky things like substituting shims when running semantic analysis pass.

@andrewrk
Copy link
Member Author

andrewrk commented Jul 30, 2024

@ityonemo the @cImport feature effectively constructs a translate-c command which outputs to a file, and then that file is imported with @import. I can't see anything but a more powerful API offered to zigler by using the CLI rather than language feature. You won't be able to do it with 1 command though, you'll need to run translate-c separately and save the output somewhere. There's no plan to make run, build-exe, build-lib, or build-obj capable of executing a translate-c step automatically.

Anyway, here's a related issue I typed up just now. I think this is the actual issue that people are wanting to talk about today:

@ityonemo
Copy link
Contributor

ityonemo commented Jul 30, 2024

Good to know. More than one command is not ideal, but it's fine! I could probably also figure out how to do it with build.zig, I just had enough trouble with it that I gave up last time :)

@philip-peterson
Copy link

philip-peterson commented Sep 9, 2024

From the perspective of an outsider, the fact that @cImport is probably going away, while still having blog posts written about how great it is, it would be nice to preemptively learn the "new way" to write Zig that interops with C. If the documentation could just reflect the future (assuming of course that the future is somewhat known) it would be nice, since otherwise learning Zig now feels like it could be obsolete next week. (Of course this is always to be true of an experimental language but it just feels like a significant block is all.)

@andrewrk
Copy link
Member Author

andrewrk commented Sep 9, 2024

To be clear, there is an upgrade path, so if you learn how to use @cImport and then you have to move it to the build system, it doesn't really mean you wasted your time. It just means you have to do something like this diff.

@andrewrk
Copy link
Member Author

andrewrk commented Nov 7, 2024

https://github.com/ziglang/translate-c

FalsePattern added a commit to FalsePattern/zig-jni that referenced this issue Dec 11, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
accepted This proposal is planned. breaking Implementing this issue could cause existing code to no longer compile or have different behavior. proposal This issue suggests modifications. If it also has the "accepted" label then it is planned. zig build system std.Build, the build runner, `zig build` subcommand, package management
Projects
None yet
Development

No branches or pull requests