FreeBSD Port Team: Volunteers needed #4039
Comments
|
My experience with FreeBSD is making VM template for CloudStack, if this can qualify I would like to join the team and picking up this brilliant OS on the way. :) |
|
it's likely that you're already aware of this fork but if not it's worth checking out: https://github.com/ajensenwaud/coreclr |
|
Ugh. Was too fast on the comment button. Meant to attempt to summon @ajensenwaud to the thread. |
|
To provide some more context ... to make this project awesome, we'd want a core group of folks that felt ownership and accountability (in a good way) to make the project happen. In turn, the .NET Core team would be more compelled to invest in CI infrastructure for FreeBSD as a first class citizen. I like the idea of "port team". Some of very best people the CLR team has ever had were on the "X64 port team", 10+ years ago. They simply referred to themselves as "the port team". Not the type of folks you'd want to meet in (technical) dark allies ;) |
|
While I don't have the time to contribute significant resources to this, given the familiarities with OS X, I'm happy to support with code review, direction and suggestions. Feel free to @kangaroo me on issues. |
|
Long time FreeBSD user/supporter. I'd be keen to help out from an administrative/sysadmin/marketing/documentation role where needed. @qbit (OpenBSD), @josteink (FreeBSD), @Aesthetikx (FreeBSD), @ajensenwaud (FreeBSD) and @kangaroo (OSX) have been involved in previous work to bring CoreCLR to BSD. |
|
These PRs are relevant to this discussion: dotnet/coreclr#453, dotnet/coreclr#470. Nice to see that they have been merged. |
|
I mostly tried to help out in @ajensenwaud's repo as a sort of "challenge accepted" short-term project. As I don't have any up-to-date FreeBSD production-systems, I doubt I'm a good candidate for a longer term FreeBSD "port team". I do agree about the name though. It has a nice ring to it. Go for it! That said, I appreciate being summoned and namedropped in |
|
Hi all Thanks Jostein, appreciate your help so far! I am very keen to still assist with the porting and can also assist with test infrastructure, as required. How many are in for this? Anders
|
|
Maybe we should make a wiki with status of each port, next steps and hurdles? OpenBSD is stuck until libunwind is ready (unless coreclr people can confirm that isn't a hard dep?). |
|
You can start porting without the libunwind. It is required for exception handling and GC. So as long as an application doesn't throw an exception or initiate a GC, everything should work without the libunwind. |
|
👍 - I will keep hacking away then! |
|
Hi folks, I did not answer earlier as I want to know how big the feedback is. So far I am happy to see quite some people eager to join this project. One question we should answer though is whether we want separate *BSD teams (one for FreeBSD, one for OpenBSD, one for NetBSD) or one big BSD Port Team. As @richlander proposed I would start a PR with a new Wiki page, describing the intention of the team, current members, agenda and way to participate. Currently I see as members: Also @qbit for OpenBSD. |
|
Alright, have created an Azure virtual machine (2 cores, 3.5GB memory) running Post a public ssh-key here and I will create you an account on |
|
I'm pretty busy these days, so I'm definitely not making any promises. Anyway, I appreciate @ghuntley's initiative, and to show that, here's an SSH-key which means that in theory I can contribute should I ever get the time: Anyone else? |
From what I can tell, coreclr is pretty well abstracted and most (everything?) platform-specific goes through the Platform Abstraction Layer (PAL). If that builds cleanly, the build as a whole goes quite a bit further without too much trouble. The PAL seems focused on underlying implementations rather than platform (see context.cpp): That checks if this architecture has BSD regs, PT regs, GREG-sets, what register-mapping etc. Since I'm not sure if this is a problem at all, the best plan I can think of is to closely collaborate with the OpenBSD team (@qbit. anyone else?) and make sure the commited changes are mutually compatible. If we later discover there's any substantial difference between FreeBSD and OpenBSD (or for instance Darwin), we might need more differentiation between the platforms than just saying "it's BSD", but let's save that for later, if for nothing else, just to get something going.
If there's any global prerequisites/requirements will they be embedded in this script? According to the best of my knowledge @kangaroo's changes to support Darwin means clang3.7 (which of we will be cherry picking some) will be required to build this. Clang3.7 is not installed by default in FreeBSD 10.1, nor is it the default c-compiler. It would be nice to have stuff like this (and other things we may add incrementally) available to everyone. Basically a public build-script for the machine. Sounds doable? Which leads us to the next issue. Where do we work? Since I doubt we'll be given commit rights into coreclr directly, or have PR's accepted before we have something to show for, should we have a place of sorts to work where we can commit incremental improvements? If so @ajensenwaud already has something going. Should the work be based on that, or should we "start over", based on the current coreclr and just port the fixes we have so far? |
|
Thanks @ghuntley for the sponsoring of a porting machine. My SSH pub key is @josteink the major difference that came to my mind is the focus of OpenBSD on security. I read they introduced their own special APIs to make software more secure and less vulnerable to security related bugs. Admittedly though I do not know any details and just read that. If it works from a compatibility side, it certainly would be nice to see all three BSDs supported. Though I think at the beginning FreeBSD should be the priority in case something does not work on all three the same way. Clang 3.7 is the current development IIRC. Clang 3.5 should be fine for the moment. For the work, I was thinking about creating a Github "Organization", which could host our fork of the coreclr tree. @richlander how is your feeling about this? Also given that it would need a matching name with CoreCLR or dotnet in it. Maybe one member of the DotNet Foundation or Microsoft can set that up so it is guaranteed there also someone with administrative access at hand? In any way I think it would be better to have central tree somewhere and not everybody having their private trees. |
Fair enough. We'll stay on 3.5 unless it causes issues.
Seconding this. How about just "DotnetBSD"? What I'm not sure about is if we really do need central support for the DotNet Foundation or Microsoft to do this? What do we stand to gain? I'm not against it per se, but if it causes us delays or to lose momentum, that would be a shame. But then again this issue has @richlander's attention so it might be a non-issue altogether.
Absolutely positively agreed. |
|
Thanks @ghuntley !
This works for me. Also trying to recruit other OpenBSD devs.
This isn't so much of an issue - most of the changes have "standard" counterparts that give you stern warnings when using them. The bigger difference will be availability of ports / required things (libunwind..) / location of installed ports. |
|
My morning now, so time to reply. I'd be happy to setup a coreclr-freebsd (or some similar name) repo as a fork of coreclr within the dotnet org and give the port team commit rights to it. To my mind, that provide the following:
PRing your changes to upstream on a regular basis would still be a good idea, but I think that is your plan, anyway. I haven't asked anyone else's opinion on this, like @jkotas or @martinwoodward, but this is my proposal, and I think its a good one. |
|
I do not think we need a fork for the FreeBSD port. I expect that the changes for the FreeBSD port are going to non-controversial, and they can be submitted to master as they are made. |
|
I agree with @jkotas -- I expect the FreeBSD port will be far less invasive than the OS X one was. |
|
What do you think about @janhenke? |
|
My idea was a bit different. I thought about having a Port Team Organization and repo primarily to directly work in it. So I would no longer have my private janhenke/coreclr tree, but there would some dotnetbsdteam/coreclr, where I can commit code to and open feature branches. In that way someone else in the team can work together with me on a feature without needing PRs for every commit. PRs would only be used once we think a certain feature is ready so I can merged into the main sources. So I do not want to have "parallel tree" here, but I rather thought a working place where we can work cooperatively in feature branches until they are ready to be merged. So by definition all changes this tree has from upstream is consider unstable until a PR is opened. I also would not accept any PR into that tree, all PRs should go directly into dotnet/coreclr. Also the CI is only needed on the main dotnet repo, expanding the current bot simply with FreeBSD/OpenBSD/NetBSD instances. |
|
I'm pretty strongly against this direction, as I can see it spiraling out of control quickly. Having a separate tree per os? Per platform? Is there a compelling reason for the primary maintainer to not just have it on their personal fork, and accept PRs themselves and push to coreclr as needed? You can also give individuals push access to your tree. |
|
More centralization and overview what is being done. I do agree that it seems a matter of which kind of work flow we want to establish with this team. If the majority does not like the idea, it is fine too. As I said I just want us to agree one common procedure to organize the work flow and also somehow make it possible to create a team where we can recognize each other and issues that affect us more easily. In the end it would also be good to have one easy way to summon the team as a whole to an issue or PR if the opinion of a BSD porter is needed. |
|
I'm not sure why we're focusing on centralization outside the canonical repo. MS has made it clear they'd be happy to upstream the patches, so lets just centralize here like for all other OSs. In terms of having people recognize the issues, that seems like an easy candidate for a label? |
|
Dear all, Here is my ssh key: Please somebody kindly advise what shall I need to do/read to prepare for this project: e.g. read about FreeBSD, LLVM, Clang, or the source code, etc. Thanks all. |
|
Great idea with the VM on Azure! Here is my ssh key: ssh-dss On Fri, Mar 20, 2015 at 8:21 AM, Dong Xie [email protected] wrote:
|
|
On Fri, Mar 20, 2015 at 7:46 AM, Geoff Norton [email protected]
Agree, but unless all of us get commit access to coreclr, we are not going For sake of separation, I suggest building out github.com/dotnetbsd unless
|
|
Created seperate issue to discuss FPREGs, with some preliminary code. https://github.com/dotnet/coreclr/issues/594 Can you guys take a look, and provide feedback if you feel capable? |
|
Well done, Jostein. I will have a look at the FPREGs now, I don't know yet how we can tackle For other *BSDs I will spin up VMs up now to see how well it compiles on Cheers Anders On Sat, Mar 28, 2015 at 9:56 PM, Jostein Kjønigsen <[email protected]
|
|
Sounds great! When you first have VMs up and running, I would love it if you could also verify my preliminary work on fpregs other places than freebsd. Would be neat if we didn't have to do this messy register-work more than once :) |
|
Hello @janhenke and others... A number of developers here at http://www.interoute.com in London are interested in FreeBSD and your .NET port project. Our company would like to help out by offering free use of virtual machines in our cloud platform, Interoute VDC [http://cloudstore.interoute.com/main/WhatInterouteVDC]. We have a FreeBSD template already installed, and you can upload other templates as you require them (you need an OVA based on VMware vmdk image, or an ISO source). Can one of you get in touch with me directly [[email protected]] and we can discuss how much resources to set up.... Regards from London. |
|
OK, I have been working on setting up a NetBSD VM -- more work is required On Mon, Mar 30, 2015 at 5:27 PM, Jostein Kjønigsen <[email protected]
|
|
@phillipkent Thanks for the offer. Right now @ghuntley has our back covered as far as FreeBSD-instances go. We've also been offered Azure resources from @MattWhilden at Microsoft. OpenBSD and NetBSD is currently not able to run on Azure, so if you could host VMs for those instances, that might be useful. @ajensenwaud : Once "done" with your machines, do you think it would be possible to have them uploaded to interroute? That way others could help out testing/porting. |
|
Out of curiousity, I decided to cheat on the FPU registers, and just commented out the lines involving the remaing two (XMM etc). Basically just tosee what the next hurdle in the build is. Disappointingly so, I discovered that just like the FPU registers, other parts of coreclr is progressing faster than we're able to port it. PAL's context2.S (mostly assembly + macros) which used to build is now complaining about basic assembly instructions being invalid. I suspect some of the macros defined might be messing up evaluation on FreeBSD. Any assembly guys around wanting to take a look? Should we create a seperate issue for this as well? |
|
I made a separate issue for the assembly failing the build. I think it makes sense to allow discussion on a per bug basis, and it probably makes it easier to track if we're getting anywhere and at what speed. Assembly issue: https://github.com/dotnet/coreclr/issues/615 |
|
Highly recommend breaking these into separate issues. Smaller increments helps two ways:
Joking aside, it's really nice internally to MS to be able to easily articulate the work that y'all have undertaken. #everyonehasaboss #yallareawesome |
|
I agree with @MattWhilden and have created new issues ongoing as I've made further progress. @janhenke This issue seems to have managed to summon a team of sorts, and gotten ourselves bootstrapped. And as such this issue has achieved its mission, right? I think it's time to close it at this point. Agreed? |
|
@josteink For the initial mission I agree. Before it is closed though, I want to remark for any potential future reader that this is by no means a closed group. Any volunteer is welcome to continue making CoreCLR run smoothly on FreeBSD as well as the other BSDs. |
|
Yep. I'll try to keep things tagged appropriately so that new folks can know where to help out. Great start! |
|
Hi Jan, I am getting a hard error when trying to compile on OpenBSD because Commencing CoreCLR Repo build -- Looking for include file libunwind.h -- Configuring incomplete, errors occurred! On Thu, Mar 19, 2015 at 12:33 AM, Jan Vorlicek [email protected]
|
|
@qbit, I have set up an account for your on our new OpenBSD instance in IP: 103.232.35.201 (port 2201) With the SSH key above. Others -- please let me know if you would like access as well. cheers Anders On Fri, Mar 20, 2015 at 1:54 AM, Aaron Bieber [email protected]
|
|
I think the first step for OpenBSD is to have Clang 3.5 (or 3.6) and libunwind. libunwind is necessary for the exception handling afaik (stack unwinding to find the exception handler). How is the current status of LLVM/Clang on OpenBSD? |
|
LLVM/Clang can be installed as a binary package or from the Ports On Sun, Apr 19, 2015 at 8:58 PM, Jan Henke [email protected] wrote:
|
|
Which versions? CoreCLR currently compiles with 3.5. Is lldb available too? If yes we would need lldb 3.6. |
|
$ clang -v lldb is not in ports, but I will see if it builds manually. On Sun, Apr 19, 2015 at 9:24 PM, Jan Henke [email protected] wrote:
|
|
Looks like openbsd work is progressing nicely. Which is good news indeed. But I think it would be better if you created a separate issue for this, like "openbsd build-machine must be configured" or something similar. Sounds OK? |
|
Thanks @ajensenwaud ! I seem to be unable to access said machine.
|
|
Hi Aaron, not sure what the issue was -- can you please try again, this On Fri, Apr 24, 2015 at 11:42 PM, Aaron Bieber [email protected]
|
|
Hi There, Just starting to play with the new drop of Core 1.0. I don't see a lot of FreeBSD support in the documentation other than a port of the CLR, but maybe that's not significant as once the CLR is built the rest is managed code? |
|
It's not as simple as we'd like it to be. Initially coreclr (and corefx) on non-Windows platforms was mostly bootstrapped using mono. So when we did the original FreeBSD port (and I guess we got to around 98% or something like that, just a few platform-specific bits of corefx missing), we could basically leverage mono which was already available and working. At that point we kind of lost momentum for a variety of reasons and that's a shame. But .NET itself kept on moving (adding awesome things like the Dotnet CLI which is a massive improvement). Part of that process involved the .NET system to be rewritten to be self-bootstrapped. And with that it drew in lots of common .NET ecosystem dependencies: nuget, roslyn, etc. This has pros and cons. The pros are obvious: We no longer depend on mono to build .NET. The cons is that to build .NET Core from source, smoothy, without too much nonsense, you now need to download a prebuilt .NET Core SDK (including its own functional coreclr runner) to run the .NET toolchain used to build .NET. And that .NET toolchain is only available for already fully supported platforms (which does not include FreeBSD). Building this SDK chain yourself can be done, but it's not trivial. Some people are working on it here: https://github.com/dotnet/corefx/issues/10610 Others are trying to piece together what we have to create a proper FreeBSD ports package: https://github.com/dotnet/coreclr/issues/6115 From those 2 issues, you should be able to find references to other issues and leads which may get you the whole way through. But that's not exactly a as smooth ride as anyone should have to suffer. Basically this whole self-bootstrapping for everything good it does, has (for now) inadvertently set back a working FreeBSD-ports quite a bit. So that's the bad news. The good news is that any PR which improves the situation quickly gets accepted. If someone wants to pick up all the code we ported, and help finally land the thing, I'm sure everyone is going to be grateful about that. I'll be more than willing to assist with code-review and other non-intensive stuff. |
|
Thanks @josteink that brings me up to speed. I'll review everything carefully. As a curiosity, would it be possible to go back to bootstrapping with mono (no need to get too detailed in your answer)? |
|
It might be. It could definitely be worth testing and trying, although you'll probably end up hav ing to write mono wrapper-scripts for the entire SDK tool chain (Dotnet CLI, nuget, Roslyn, msbuild, etc). In a sense that would be creating a new SDK platform package, except the platform is mono, not a specific OS. That would probably be useful for all the platforms which aren't already bootstrapped. If that could be made, that might make an interesting fallback on currently "unsupported OSes". If that's more work or less than just getting a FreeBSD SDK building, I don't know. It would be starting from scratch, as opposed to finishing something already started. On the flip side, you would probably be able to get going at it without having to wait for anyone else to do work for you. Whatever you decide to do, I hope it works out :) @janvorli: do you think such a mono-based fallback approach and initiative would be something Microsoft could consider supporting at this stage? While it might seem like a step backwards, it would certainly assist putting up initial ports on new platforms. |
|
Ha, I just got Monodevelop 6.1 going on freebsd again. :P I need to get my head around the build toolchain first so let me go back over all the comments... |
ghost
locked as resolved and limited conversation to collaborators
Following the discussion on the chat, I am looking for volunteers to start a FreeBSD port team.
The purpose is to have a group of people working together on making coreclr working well on FreeBSD. This includes having more people with FreeBSD experience for reviewing patches and keeping an eye on PRs.
This issue is intended to discuss this idea and as a means to get into contact with each other. Also technical discussions about porting to FreeBSD are possible.
The text was updated successfully, but these errors were encountered: