Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Consider switching default port for ASP.NET Core to 8080 or 5000 #3968

Closed
richlander opened this issue Aug 5, 2022 · 33 comments
Closed

Consider switching default port for ASP.NET Core to 8080 or 5000 #3968

richlander opened this issue Aug 5, 2022 · 33 comments

Comments

@richlander
Copy link
Member

richlander commented Aug 5, 2022

Currently, the default port for our images is port 80: https://github.com/dotnet/dotnet-docker/blob/main/src/runtime-deps/6.0/jammy/amd64/Dockerfile#L19

It isn't possible for non-root images to bind to port 80 or 443: containerd/containerd#2516. We should consider using only non-root ports so that it is easy to use our assets for both root and non-root scenarios.

There are a few things to consider:

  • Folks will need to update script and manifests that do port mapping. As a result, it's an upgrade-time breaking change.
  • Folks can straightforwardly opt-out by re-setting the ENV to what it is today in their layer.
  • This change would bifurcate our runtime-deps Dockerfiles.
  • 8080 might not be the best port to choose. 5000 might be a much better idea since that's the default port for dev.

Is it worth it? I think so. The motivation is embracing non-root as a key scenario.

I'm thinking that this change is a bit late for .NET 7. It's probably best left until .NET 8 Preview 1.

Related: dotnet/designs#271

@richlander richlander added the bug label Aug 5, 2022
@mthalman mthalman moved this to Backlog in .NET Docker Aug 5, 2022
@richlander richlander changed the title Consider switching default port for ASP.NET Core to 8080 Consider switching default port for ASP.NET Core to 8080 or 5000 Aug 6, 2022
@richlander
Copy link
Member Author

@davidfowl
Copy link
Member

davidfowl commented Aug 6, 2022

How do other frameworks handle this? What do they recommended? Maybe we should just not set the port.

@richlander
Copy link
Member Author

richlander commented Aug 7, 2022

I just played with node.js in docker. It doesn't have this problem. I think it is because ASP.NET Core is being too fancy. AFAICT, it's security first posture is taking us down a bad path.

Today's behavior:

rich@kamloops:~$ docker run --rm -it -p 8000:80 mcr.microsoft.com/dotnet/samples:aspnetapp
      Now listening on: http://[::]:80

Don't set the ASPNETCORE_URLS ENV:

rich@MacBook-Air-2 ~ % docker run --rm -it -e ASPNETCORE_URLS= -p 5000:5000 mcr.microsoft.com/dotnet/samples:aspnetapp
      Now listening on: http://localhost:5000

I cannot access that port from outside of the container because that's a private loopback port.

I can outsmart that. I'll set the ENV but to 5000:

rich@MacBook-Air-2 ~ % docker run --rm -it -e ASPNETCORE_URLS=http://+:5000 -p 5000:5000 mcr.microsoft.com/dotnet/samples:aspnetapp
      Now listening on: http://[::]:5000

That works (obviously).

AFAICT, node.js is doing the same thing (not loopback) as this example but on port 3000. I'm far from a node.js expert. I followed this VS code tutorial -- https://code.visualstudio.com/docs/containers/quickstart-node -- and it worked perfectly w/o me needing to reason about the port.

rich@MacBook-Air-2 nod % docker run --rm -d -p 3000:3000 nodetest
d85b88fb88d613f8abaf68a12cfb875449db3df2a1c4a284c7bc15e31cce90e1
rich@MacBook-Air-2 nod % curl http://localhost:3000
<!DOCTYPE html><html><head><title>Express</title><link rel="stylesheet" href="/stylesheets/style.css"></head><body><h1>Express</h1><p>Welcome to Express</p></body></html>%  

I don't see any special ENVs in my Dockerfile or in the base image: https://hub.docker.com/_/node/. AFAICT, node.js does the right thing out of the box.

We made three distinct choices here that led to this poor outcome:

  • Choose loopback as the default experience
  • Make it complicated to say "upgrade from loopback to machine hosting on the default port".
  • Choose port 80 as the default Docker experience

The first one may well have been the right choice. The second two are objectively problematic. Personally, it took me ages to figure out the ASPNETCORE_URLS ENV. It is objectively overly complicated for something that everyone needs to deal with.

Now that I think of it, we should create a new boolean ENV called something like ASPNETCORE_EXPOSE_PORT or ASPNETCORE_LOOPBACK. In addition, we could apply more logic into DOTNET_RUNNING_IN_CONTAINER to also mean this.

The whole thing again, in compressed form:

rich@MacBook-Air-2 ~ % docker run --rm --name aspnetapp -d -p 8000:80 mcr.microsoft.com/dotnet/samples:aspnetapp 
ad04c7a80793f0c577c42c86bf3eac926125b10208bb7f184e20e87b6193fb40
rich@MacBook-Air-2 ~ % curl http://localhost:8000/Environment
{"runtimeVersion":".NET 6.0.6","osVersion":"Linux 5.10.104-linuxkit #1 SMP PREEMPT Thu Mar 17 17:05:54 UTC 2022","osArchitecture":"Arm64","processorCount":4,"totalAvailableMemoryBytes":4108652544,"memoryLimit":0,"memoryUsage":0}%           rich@MacBook-Air-2 ~ % docker kill aspnetapp                 
aspnetapp
rich@MacBook-Air-2 ~ % docker run --rm --name aspnetapp -d -e ASPNETCORE_URLS= -p 8000:80 mcr.microsoft.com/dotnet/samples:aspnetapp

d5c24528f7b8c5e958ddc280a7e44db9c42cf72942288ad261787463c73af099
rich@MacBook-Air-2 ~ % curl http://localhost:8000/Environment
curl: (52) Empty reply from server
rich@MacBook-Air-2 ~ % docker kill aspnetapp
aspnetapp
rich@MacBook-Air-2 ~ % docker run --rm --name aspnetapp -d -e ASPNETCORE_URLS= -p 8000:5000 mcr.microsoft.com/dotnet/samples:aspnetapp

0e6b23bfc2919f7cad770566828c3cffcd504610cb6ed61e34876c06f6f52bc7
rich@MacBook-Air-2 ~ % curl http://localhost:8000/Environment   
curl: (52) Empty reply from server
rich@MacBook-Air-2 ~ % docker kill aspnetapp
aspnetapp
rich@MacBook-Air-2 ~ % docker run --rm --name aspnetapp -d -e ASPNETCORE_URLS=http://+:5000 -p 8000:5000 mcr.microsoft.com/dotnet/samples:aspnetapp
651374707de7889677f37fc7559d9f1afac03d89e986676f2b12a2c58c500a4c
rich@MacBook-Air-2 ~ % curl http://localhost:8000/Environment
{"runtimeVersion":".NET 6.0.6","osVersion":"Linux 5.10.104-linuxkit #1 SMP PREEMPT Thu Mar 17 17:05:54 UTC 2022","osArchitecture":"Arm64","processorCount":4,"totalAvailableMemoryBytes":4108652544,"memoryLimit":0,"memoryUsage":0}%           rich@MacBook-Air-2 ~ % docker kill aspnetapp
aspnetapp
rich@MacBook-Air-2 ~ % 

@davidfowl
Copy link
Member

I thought the ASP.NET Core image set ASPNETCORE_URLS not runtime-deps, why is that?

ASPNETCORE_URLS is pretty nice to have out of the box as you can configure the scheme, listen address and port in a single url scheme without code changes. The other approach is putting it in code and knowing what the container's defaults are, this is what I've seen other frameworks lean towards.

I not against changing the default port in the container though from 80 to 5000 or something similar.

Is it worth it? I think so. The motivation is embracing non-root as a key scenario.

I don't have a good read, have we seen feedback pushing this scenario?

@richlander
Copy link
Member Author

richlander commented Aug 8, 2022

We set the ENV in the lowest layer due to self-contained apps and request for SDK scenarios.

I think you are missing my point. The whole need for this ENV is unnecessary complexity. It should be an advanced scenario, not required to get ASP.NET Core to work.

No call for this scenario yet because we have been pushing back on non-root to this point. We are now going to embrace it. I am looking at it for the first time and this is the first thing I noticed and now realize the ASP.NET Core has a design flaw that I am now seeing in a new light. I have always found this part of kestrel confusing and now I understand why.

This whole design point is informed by the (severe) mistake of IIS exposing port 80 to the network by default going back two decades. I think node has the right model and ASP.NET score is overcorrecting resulting in a bad trade off.

@davidfowl
Copy link
Member

davidfowl commented Aug 8, 2022

No call for this scenario yet because we have been pushing back on non-root to this point. We are now going to embrace it. I am looking at it for the first time and this is the first thing I noticed and now realize the ASP.NET Core has a design flaw that I am now seeing in a new light. I have always going this part of kestrel confusing and now I understand why.

The fix is simple right? Set ASPNETCORE_URLS, or hard code the listen urls in your application to listen on 0.0.0.0 (or + or *).

I think you are missing my point. The whole need for this ENV is unnecessary complexity. It should be an advanced scenario, not required to get ASP.NET Core to work.

I don't agree. Like I said the alternative is code in the application.

@richlander
Copy link
Member Author

So why is the ASP.NET Core design point better than what node is doing? That is super unclear to me. The node approach seems objectively better.

The thing you propose seems super odd to me. What are people supposed to interpret that as meaning? It is so cryptic.

The other thing I realized is that the ENV we set is only for raw/bare http. That seems like another oddity. Why do we make folks do extra work to host with TLS? I assume node had a nicer story here, too.

@richlander
Copy link
Member Author

richlander commented Aug 8, 2022

Plenty of hits on this topic at StackOverflow: https://stackoverflow.com/search?q=ASPNETCORE_URLS

No one using node needs to ask this question.

@davidfowl
Copy link
Member

The thing you propose seems super odd to me. What are people supposed to interpret that as meaning? It is so cryptic.

The meaning is to specify the port in code:

var builder = WebApplication.CreateBuilder(args);
var app = builder.Build();

var port = builder.Configuration["PORT"] ?? "5000";

app.MapGet("/", () => "Hello World");

app.Urls.Add($"http://0.0.0.0:{port}");

app.Run();

This is what the node approach looks like.

So why is the ASP.NET Core design point better than what node is doing? That is super unclear to me. The node approach seems objectively better.

I don't think other platforms have "an approach", like I said, the alternative is user code is responsible for doing this. If that's a "better" approach, then users can just start doing that. Hard code the port in their application along with 0.0.0.0 (like in the above example).

The other thing I realized is that the ENV we set is only for raw/bare http. That seems like another oddity. Why do we make folks do extra work to host with TLS? I assume node had a nicer story here, too.

TLS is hard everywhere. If you find it easier to configure TLS on another platform let me know, I'd be surprised.

Plenty of hits on this topic at StackOverflow: https://stackoverflow.com/search?q=ASPNETCORE_URLS

That's not proof of anything.

No one using node needs to ask this question.

You're just not looking for the right thing. https://stackoverflow.com/search?q=PORT+docker+nodejs

@richlander
Copy link
Member Author

Those node + docker issues seem to be folks who don't know how to use Docker. The ASP.NET Core issues seem to be people that don't know how to use ASP.NET Core.

Both Node.js and ASP.NET Core have default ports. The former is on a real port and the latter loopback. I don't grasp why we use loopback by default. It creates a usability challenge and doesn't address a real security concern.

Hard coding the port in code seems OK as long as it can be overriden by ASPNETCORE_URLS. However, it seems super odd for every ASP.NET Core app to override port 5000 to be port 5000. Most users just want a port to expose outside their environment. Requiring an easy mode for that would seem to be a very welcome feature.

@davidfowl
Copy link
Member

davidfowl commented Aug 8, 2022

OK lets start over. When you write an ASP.NET Core application and you don't specify any information, it will listen on default ports on localhost. This is assuming the developer didn't write any code to manually specify which addresses and ports to listen on. Starting with .NET 7 it'll only listen on an HTTP port 5000 but the address is always loopback.

If we're comparing apples to apples, nodejs has no default port. The default port is specified in user code and there's no way to override this port without manually authoring some code to read an environment variable (commonly PORT) or some other piece of configuration to specify what to listen on. A very common default port used in nodejs application is port 3000. Now the major difference is that nodejs when you don't specify an address/host to listen on, it will default to listening on all interfaces (0.0.0.0 or :: on IPV6 see https://nodejs.org/api/net.html#serverlistenport-host-backlog-callback).

That's the sticking point between the 2 technologies. ASP.NET Core defaults to listening to localhost not all interfaces. There's a long history behind why we default to localhost (like the fact that it prompts on windows) and maybe it deserves some reconsidering in a cloud native world. I don't think that changes ASPNETCORE_URLS at all though, just the defaults.

Most users just want a port to expose outside their environment. Requiring an easy mode for that would seem to be a very welcome feature.

Spit balling a little, maybe we can do this based on the environment even though I know that would make some people uncomfortable 😄 (it also wouldn't accomplish the goal if you were developing in containers).

One more thing to add, we don't have an API that lets you specify just a post, it's always a URL.

@davidfowl
Copy link
Member

Those node + docker issues seem to be folks who don't know how to use Docker. The ASP.NET Core issues seem to be people that don't know how to use ASP.NET Core.

No, there are people trying to make their application work in docker which means they need to get the port and host right in their application and get the port right in docker. That's why there are answers like this https://stackoverflow.com/a/57901602/45091

@richlander
Copy link
Member Author

node has no default port

I didn't see this at first. You are right. I was looking in the wrong files. This context is helpful.

var port = normalizePort(process.env.PORT || '3000');
app.set('port', port);

BTW: Wow is Express more verbose than ASP.NET Core for a simple app.

I don't think that changes ASPNETCORE_URLS at all though, just the defaults.

Fully agree with that.

we don't have an API that lets you specify just a post, it's always a URL
http://[::]:5000

That's a pretty cryptic URL. Let's be clear that this is a DSL for a URL pattern. That's why I have trouble with it and I'm certain others do, too.

There are two ends of the spectrum with an easy mode:

  • Explicit, like ASPNETCORE_HTTP_PORT=5000.
  • Implicit, like ASPNETCORE_HTTP_EXPOSE=true

The problem with the explicit one is that it significantly overlapping with ASPNETCORE_URLS. I like the implicit one since it's saying that a port will be exposed via a machine IP over HTTP. Since ASP.NET Core already knows about port 5000, we can just "upgrade" it.

The other option is honoring DOTNET_RUNNING_IN_CONTAINERS but even I think that's a bad idea.

I'm also reminded on this: https://github.com/dotnet/dotnet-docker/blob/main/samples/aspnetapp/aspnetapp/Properties/launchSettings.json#L26-L32. I created that years ago because I wanted a way to get a development experience across machines. Like testing the mobile experience of an app. We had a long conversation about it at the time. There are multiple scenarios where you'd want to expose your site on a real port.

@davidfowl
Copy link
Member

davidfowl commented Aug 8, 2022

That's a pretty cryptic URL. Let's be clear that this is a DSL for a URL pattern. That's why I have trouble with it and I'm certain others do, too.

Maybe? It's a DSL in URL form, (scheme, address, port). What's strange is the "+" and "*" characters. Those are HTTP.sys isms that leaked into the stack. That said, we support http://0.0.0.0:{port} as well and I don't think that's too crazy to understand. One simplification we could make is to also support omitting the scheme so host:port works as well.

The problem with the explicit one is that it significantly overlapping with ASPNETCORE_URLS. I like the implicit one since it's saying that a port will be exposed via a machine IP over HTTP.

So there are 2 problems:

  1. Do we look at changing the defaults. When you say nothing at all, what is the default port and default address?
    • To influence these defaults there would need to be another variable like ASPNETCORE_DEFAULT_HOST=0.0.0.0 and ASPNETCORE_DEFAULT_PORT=5000
    • Maybe we change the defaults? This is a breaking change and launching will now start to prompting on windows during dev.
  2. When explicitly specified, can we support more obvious syntax in ASPNETCORE_URLS
    • {host}:{port} - infer http for the scheme where the host and port are fully specified
    • :{port} or {port} - infer http and all addresses and specified port

PS: Golang supports a DSL as well in their ListenAndServe API (without the scheme)

@richlander
Copy link
Member Author

Let's transition the discussion to what we should do for .NET 8.

I propose we do the following (prior to Preview 1):

  • Add the app non-root user.
  • Create a new easy-mode way to set HTTP and HTTPS ports.
  • Use the following definition for HTTP: ASPNETCORE_DEFAULT_PORT=5000
  • Users could then override that with their own value or set ASPNETCORE_URLS, which would also override the easy-mode ENV.

We should decide on the port number before we publish the .NET 6+ Ubuntu Chiseled images. We want to avoid a breaking change with them later. They are port 8080, currently. It is up to the ASP.NET team to decide on that port value.

@tmds
Copy link
Member

tmds commented Sep 28, 2022

@richlander @davidfowl images published by Red Hat all default to port 8080.
If Microsoft images use the same port, it's easier to switch between the images.

8080 might not be the best port to choose. 5000 might be a much better idea since that's the default port for dev.

I don't understand what this means.
On the host, the container port can still be mapped to 5000 in dev scenarios.

@richlander
Copy link
Member Author

FYI: dotnet/aspnetcore#44194

@richlander
Copy link
Member Author

richlander commented Oct 13, 2022

Looks like we're almost done here and just need to pick the port (before updating .NET 8 Dockerfiles).

Here's some info:

The container team is happy to take more feedback, but the easiest thing for us to do is to use port 8080. If we don't hear anything, we'll use that when we generate the .NET 8 Dockerfiles.

@Tratcher, @davidfowl, @adityamandaleeka

@davidfowl
Copy link
Member

I think we should stick to 5000 for ASP.NET Core. 8080 is very popular, I think it's fine for containers since it's the only process in the container, but not as a default for the framework.

@adityamandaleeka
Copy link
Member

Yea, we were talking about this this morning. @sebastienros also feels 8080 is a good choice given that it's fairly standard for container images.

@davidfowl so you're fine with having the .NET containers configured by default to use 8080 right?

@richlander
Copy link
Member Author

Sounds like we're all in agreement.

@davidfowl
Copy link
Member

I should mention if we change this default, it'll likely break things like azure app service for containers/linux.

@richlander
Copy link
Member Author

Yep. We need to start talking to teams and also other clouds. This will be easier once we have images and a sample to test. I hope that is very soon.

@socketnorm

@normj
Copy link

normj commented Oct 17, 2022

I don't think any AWS service would be affected by the change directly, we do have some client deployment tooling that will need to be updated. For the most part I think this will cause users hiccups as they migrate to .NET 8 till they realize by upgrading to .NET 8 they have to also update the port mappings or Elastic Load Balancer port configuration. I would consider this a customer opt-in change by upgrading to .NET 8 but of course users are highly unlikely to realize this change as part of upgrading to .NET 8.

I imagine the experience for many upgrade to .NET 8 will be:

  • Upgrade project to target net8.0. Maybe run it locally without a container and everything looks good.
  • Update Dockerfile base image to the Microsoft provided .NET 8 images
  • Deployed but application is unresponsive
  • Check logs of deployed container
  • See no errors in the log but also no activity on then ASP.NET Core startup logging.
  • Then it goes 1 of 2 ways
    • Sees in the logs that binding port is different and makes the appropriate infrastructure changes
    • Doesn't notice the binding port is different or wasn't the original developer and didn't know what port was used before. Now spends unknown amount of time googling ".NET 8 broken in containers" and hopefully find a Stackoverflow post that tells them to fix their ports.

So my question back to you is how do we educate users of this change? Wrong port mapping silent failures are going to frustrate I suspect a lot of people. And due to containers being a hidden box from tooling point of view as far as used ports go it is going to challenging educate the user directly in the tooling.

I assume the driving factor for switching to port 8080 is the non-root feature. Is the non-root feature big enough to change the default behavior or should the non-root feature be a separate line of base images. That way when users opt into non-root base images they should be expecting more significant changes during upgrading if they choose to switch to the non-root base images.

@mehmetozkaya
Copy link

I came here from the problem when port-forwarding .net7 web api container pod in Minikube. It would be good to listen 0.0.0.0 url by default in asp.net to provide Kubernetes compatible for cloud-native deployments. It would be also good for Serverless Container Deployment tools like CloudRun, AWS CodeRunner and Azure Container instances.
For example, its obviously seen that only .net required to set url in code when deploying CloudRun;
https://cloud.google.com/run/docs/quickstarts/build-and-deploy/deploy-dotnet-service

Java and NodeJS is not required to url defintion:
https://cloud.google.com/run/docs/quickstarts/build-and-deploy/deploy-java-service
https://cloud.google.com/run/docs/quickstarts/build-and-deploy/deploy-nodejs-service

@davidfowl
Copy link
Member

@jskeet can you help clean up the google C# docs here? The code doesn’t need to be modified to listen on all interfaces (there are env variables that can be used instead)

@jskeet
Copy link

jskeet commented Apr 12, 2023

@davidfowl: Will see what I can do.

@meteatamel
Copy link

meteatamel commented Apr 13, 2023

Hi @davidfowl, could you clarify what you have in mind for Cloud Run C# docs?

To recap, Cloud Run contract states that the container must listen for requests on 0.0.0.0 on the PORT environment variable into the container. That's why we have the following in the code to dynamically listen for the port:

var port = Environment.GetEnvironmentVariable("PORT") ?? "8080";
var url = $"http://0.0.0.0:{port}";

I think you're suggesting we use ASPNETCORE_URLS=http://0.0.0.0:8080 somewhere in Dockerfile, instead? This means that every time the user changes the port of the service, they also need to rebuild the container. I'm not sure if this is better.

@Pablo-Lopez-Ponce
Copy link

So I guess you didn't eventually had that conversation with the Azure App Service folks because I just updated to .NET 8 and faced the exact frustration @normj anticipated more than a year ago when deploying it.

It's weird that app service is expecting an app in port 80 or 443 and then this image defaults to 8080. The solution was to set the Azure App service env variable WEBSITES_PORT=8080 if anyone ends up here for the same reason.

@schotime
Copy link

schotime commented Dec 5, 2023

@Pablo-Lopez-Ponce I just got this as well and solved it the same way. Talk about a breaking change.

@lbussell
Copy link
Contributor

lbussell commented Dec 6, 2023

@Pablo-Lopez-Ponce and @schotime, thanks for the feedback, we're aware of the port mismatch and are in contact with the Azure App Service team.

This breaking change is documented in our release announcement and the overall .NET 8 breaking change documentation.

@schotime
Copy link

schotime commented Dec 6, 2023

Yeh, I have no problem with the 8080 port change and it was document, its more how app service didn't automatically use the expose and there wasn't much information about that specifically

@lbussell
Copy link
Contributor

@schotime and @Pablo-Lopez-Ponce, the recommended fix from the Azure App Service team is to use the EXPOSE instruction in your Dockerfile to let App Service know which port your application is running on. It reads the image metadata and automatically opens the correct port. I reproduced your issue myself and added the EXPOSE 8080 instruction to my sample Dockerfile, and it works with App Service. Then you won't need to use the App Srevice WEBSITES_PORT setting.

We don't use EXPOSE in our ASP.NET Core base images as it's not a reversible instruction - if we EXPOSE a port by default, images that derive from that image can't un-set that. I'm going to add the EXPOSE instruction to our samples now thanks to your feedback.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
Status: Done
Development

No branches or pull requests