Ability to reload AWSOptions after ASP.NET Core Lambda Startup #219
-
The QuestionI am enhancing an existing ASP.NET Core Lambda solution to add multi-tenant support. Our multitenant setup would use the same codebase (front-end, ANguar, oaded from S3 buckets and back-end, .NET Lambdas) but separate database (either by schema, physical database or physical cluster - depnding on the client). Per tenant requests would be recognized by incoming host since each client would have their own domain (or be mapped to one of our tenant-specific subdomains). Configuration for our application resides in AWS Parameter Store and would be composed of a Common section (/myapplication/Commmon) which would contain mappings between Hosts and Logical Tenant Names. In addition, there would be tenant-specific sections, for each configured tenant, that would contain tenant-specific application configuration values (/myapplication/tenantA/, /myapplication/tenantB/). Processing piepeline in the ASP.NET Core 3.1 application would contain custom middleware which would do the initial mapping of Host to Tenant and then load the appropriate tenant specific configuration section. One problem I have noticed is that AWSOptions are loaded in the ConfigureServices section of my ASP.NET Core app while our middleware does not run until the next Configure section. AWSOptions are loaded from "the base" configuration "key (e.g. /myapplication/AWS/UserPoolClientId) and while I have tried to reload it from a tenant-specific section using this construct: configuration.Bind($"{tenantName}:AWS", awsOptions.Value); But it does not seem to work. My general question is whether there is a way to reload AWSOptions to support different Congito (and other AWS service) on a per-client/tenant basis rather than relying on the AWSOptions configured at the "base" level. Alternatively, is there a way to lazily configure Congito (or other AWS Services) to delay configuration until the client is known. Environment
This is a ❓ general question |
Beta Was this translation helpful? Give feedback.
Replies: 20 comments
-
Hi @IgorPietraszko, Good afternoon. Thanks for posting guidance question. Could you please share the sample code snippet to investigate further? As far as I know, there isn't an ability to reload AWSOptions in Amazon.AspNetCore.Identity.Cognito package. However, you can take some guidance from Amazon.Extensions.Configuration.SystemsManager which supports configuration reload, if a change in Systems Manager config is detected. Thanks, |
Beta Was this translation helpful? Give feedback.
-
Are you trying to persist AWS Service Clients between requests? It you're setting up a Multi Tenant Lambda function, where the Cognito Client needs to be configured differently based on request data (ie headers etc), why not recreate the Cognito Client at the beginning of every request? public APIGatewayProxyResponse Get(APIGatewayProxyRequest request, ILambdaContext context)
{
AmazonCognitoIdentityConfig config = LoadTenantSpecificCognitoConfi(request, context);
var client = new AmazonCognitoIdentityClient(config);
// perform Tenant specific work
} Alternatively, you could implement your own Client Factory to externalize some of the logic outside of your Lambda class. Something like this may work: public interface IMultiTenantCognitoClient
{
AmazonCognitoIdentiyClient GetCognitoClient(APIGatewayProxyRequest request, ILambdaContext context)
}
public class MultiTenantCognitoClient : IMultiTenantCognitoClient
{
// optional
private readonly AWSOptions _options;
public MultiTenantCognitoClient(AWSOptions) { _options = options; }
public AmazonCognitoIdentiyClient GetCognitoClient(APIGatewayProxyRequest request, ILambdaContext context)
{
// examine _options, request, and context to build a tenant specific AmazonCognitoIdentityConfig
var config = ...
return new AmazonCognitoIdentityClient(config);
}
} You can then setup the binding for IMultiTenantCognitoClient and then construct inject a IMultiTenantCognitoClient into your Lambda function. |
Beta Was this translation helpful? Give feedback.
-
I am using the LambdaEntryPoint class inheriting from APIGatewayProxyFunction as the entry point. This is what runs first and this is where I set up SystemsManager to access the config from two place: 1) /myapplication/Common and 2) /myapplication/<environment_name>. <environment_name> is set up as an Environment Variable on the lambda which allows SystemsManager to support multiple instances of the same Lambda as long as each is configured with a different <environment_name>. public class LambdaEntryPoint : Amazon.Lambda.AspNetCoreServer.APIGatewayProxyFunction
{
protected override void Init(IWebHostBuilder builder)
{
builder
.ConfigureLogging(logging => logging.AddLambdaLogger(loggerOptions))
.ConfigureAppConfiguration((context, config) =>
{
var env = context.HostingEnvironment;
config.AddSystemsManager($"/myapplication/common");
config.AddSystemsManager($"/myapplication/{env.EnvironmentName}",
optional: false,
reloadAfter: TimeSpan.FromMinutes(5));
})
.UseStartup<Startup>();
}
protected override void Init(IHostBuilder builder)
{
}
} Next, Startup() function runs which first configures services in ConfigureServices() and then configures the application in Configure(). AWS specific services like Cognito are added in the ConfigureServices() method. I make a decision about calling tenants based on the incoming Request's Host which is not resolved until the Configure() method. ConfigureServices() method sets up an empty TenantOption class which is then populated by Middleware invoked from Configure() method based on the Request's Host and mapped to an appropriate section in the SystemsManager (e.g. /myapplication/<environment_name/tenantA). Here are the relevant code pieces: public class Startup
{
public Startup(IConfiguration configuration, IWebHostEnvironment environment)
: base(configuration, environment) { }
public void ConfigureServices(IServiceCollection services)
{
// Set up Configuration with an empty TenantOptions, to be populated in ResolveMutitenantMiddleware
services.Configure<TenantOptions>(t => new TenantOptions());
var awsOption = Configuration.GetAWSOptions("AWS");
services.AddDefaultAWSOptions(awsOption);
services.AddAWSService<IAmazonSimpleEmailService>();
services.AddCognitoIdentity();
services.AddTransient<CognitoSignInManager<CognitoUser>>();
services.AddTransient<CognitoUserManager<CognitoUser>>();
}
public void Configure(IApplicationBuilder app, IWebHostEnvironment env,
ApplicationDbContext dbContext, ILogger<Startup> logger, IErrorReporter errorReporter)
{
// To run in multitenant setup, we need to extract the host
app.UseMultitenantSetup();
}
} Below code shows how I map configuration sections to TenantOptions based on the incoming Request: public class ResolveMutitenantMiddleware
{
private readonly RequestDelegate _next;
public ResolveMutitenantMiddleware(RequestDelegate next)
{
_next = next;
}
public async Task InvokeAsync(
HttpContext context,
IConfiguration configuration,
IOptions<TenantOptions> tenantOptions,
IOptions<AWSOptions> awsOptions)
{
// get the host from the request
var host = context.Request.Host;
// extract host to tenant mapping from config
var commonConfigSection = configuration.GetSection("Common");
// grab tenant name based on the host
var tenantName = commonConfigSection.GetValue<string>(host.Value);
// bind TenantOptions to tenant specific config section
configuration.Bind(tenantName, tenantOptions.Value);
// re-bind AWSOptions to tenant specific ones - does not work!
configuration.Bind($"{tenantName}:AWS", awsOptions.Value);
await _next(context);
}
} So rest of my code relying on TenantOptions works fine since configuration values in TenantOptions are not used until needed later in the request pipeline where I can inject them. However it seems that AWS services configured in the ConfigureServices() method cannot be re-jigged to load their configuration from a section other than /myapplication/<environment_name>/AWS (e.g. /myapplication/<environment_name>/tenantA/AWS) nor can I repopulate AWSOptions and "reset" all AWS Service to re-configure themselves based on the new options. Hope this sheds more light on my dilemma. |
Beta Was this translation helpful? Give feedback.
-
Your question makes a lot more sense now, thanks for the details; and what an intersting problem. You are correct that once a AWS Service is created it's not possible to influence the Service by mutating the Instead, we can borrow from aws/aws-sdk-net#1957 and late build I've created a small proof-of-concept that should have everything you need. Create a helper class:/// <summary>
/// DI Helping that allows Middleware to set a function for late binding <see cref="AWSOptions"/>.
/// NOTE: Binding for <see cref="IAWSOptionsFactory"/> as well as any services that want to consume it
/// need to use <see cref="ServiceLifetime.Scoped"/>
/// </summary>
public interface IAWSOptionsFactory
{
Func<IServiceProvider, AWSOptions> AWSOptionsBuilder { get; set; }
}
public class AWSOptionsFactory : IAWSOptionsFactory
{
public Func<IServiceProvider, AWSOptions> AWSOptionsBuilder { get; set; }
} Then in
|
Relative Url | Result |
---|---|
/api/Values?regionEndpoint="us-east-1" | ["US East (N. Virginia)"] |
/api/Values?regionEndpoint="us-east-2" | ["US East (Ohio)"] |
/api/values?regionEndpoint=eu-west-1 | ["Europe (Ireland)"] |
Beta Was this translation helpful? Give feedback.
-
Interested to hear if this works for your use case or if you have any follow up questions. |
Beta Was this translation helpful? Give feedback.
-
Looks very promising...trying it out... |
Beta Was this translation helpful? Give feedback.
-
Getting a //var awsOption = Configuration.GetAWSOptions("AWS");
//services.AddDefaultAWSOptions(awsOption);
// note: AWSOptionsFactory.AWSOptionsBuilder func will be populated in middleware
services.AddScoped<IAWSOptionsFactory, AWSOptionsFactory>();
-> services.Add(new ServiceDescriptor(typeof(AWSOptions), sp => sp.GetService<IAWSOptionsFactory>().AWSOptionsBuilder(sp), ServiceLifetime.Scoped));
services.AddAWSService<IAmazonSimpleEmailService>();
services.AddCognitoIdentity();
services.AddTransient<CognitoSignInManager<CognitoUser>>();
services.AddTransient<CognitoUserManager<CognitoUser>>(); I am not familiar with |
Beta Was this translation helpful? Give feedback.
-
This is just a guess without your full codebase, but I'm guessing In my example, it's the Middleware that populates What this means is, if the Service Collection needs to construct an You should have a couple options depending on how you're initializing your application and what is trying to take a dependency on |
Beta Was this translation helpful? Give feedback.
-
Thanks for your input. Will read up on Scope Scenarios and see where I get. |
Beta Was this translation helpful? Give feedback.
-
So I made small progress to see that what is happening is more or less what you have described. I am getting this exception now: Connection id "0HMFRRGHM3OQV", Request id "0HMFRRGHM3OQV:0000001B": An unhandled exception was thrown by the application.
System.InvalidOperationException: Cannot resolve scoped service 'Amazon.Extensions.NETCore.Setup.AWSOptions' from root provider.
at Microsoft.Extensions.DependencyInjection.ServiceLookup.CallSiteValidator.ValidateResolution(Type serviceType, IServiceScope scope, IServiceScope rootScope)
at Microsoft.Extensions.DependencyInjection.ServiceProvider.Microsoft.Extensions.DependencyInjection.ServiceLookup.IServiceProviderEngineCallback.OnResolve(Type serviceType, IServiceScope scope)
at Microsoft.Extensions.DependencyInjection.ServiceLookup.ServiceProviderEngine.GetService(Type serviceType, ServiceProviderEngineScope serviceProviderEngineScope)
at Microsoft.Extensions.DependencyInjection.ServiceLookup.ServiceProviderEngineScope.GetService(Type serviceType)
at Microsoft.Extensions.DependencyInjection.ServiceProviderServiceExtensions.GetService[T](IServiceProvider provider)
at Amazon.Extensions.NETCore.Setup.ClientFactory.CreateServiceClient(IServiceProvider provider) I guess one of my dependencies relies on a AWS service which is being created using CreateServiceClient(): internal object CreateServiceClient(IServiceProvider provider)
{
var loggerFactory = provider.GetService<Microsoft.Extensions.Logging.ILoggerFactory>();
var logger = loggerFactory?.CreateLogger("AWSSDK");
var options = _awsOptions ?? provider.GetService<AWSOptions>();
if(options == null)
{
var configuration = provider.GetService<IConfiguration>();
if(configuration != null)
{
options = configuration.GetAWSOptions();
if (options != null)
logger?.LogInformation("Found AWS options in IConfiguration");
}
}
return CreateServiceClient(logger, _serviceInterfaceType, options);
} which in turn is trying to get AWSOptions from the provider. Now, I am still a bit unclear on Scope Scenarios you have referred to but believe that provider referenced above exists in a different scope than AWSOptions I inject through the middleware. I also don't know which AWS Service is being created here. So still digging... |
Beta Was this translation helpful? Give feedback.
-
This issue has not received a response in 5 days. If you want to keep this issue open, please just leave a comment below and auto-close will be canceled. |
Beta Was this translation helpful? Give feedback.
-
Yes, I would like to keep this issue open. |
Beta Was this translation helpful? Give feedback.
-
Browsing through aws-aspnet-cognito-identity-provider, I noticed that CognitoUserPool is added to the DI container as a Singleton. Looking at this article (https://blog.steadycoding.com/using-singletons-in-net-core-in-aws-lambda/) I have a question about statefullness of Singletons in AWS Lambda? services.AddSingleton(t => new AWSCognitoClientOptions()); and then mutating it in my tenant resolution code: var awsCognitoClientOptions = context.RequestServices.GetService<AWSCognitoClientOptions>();
// re-bind AWSCognitoClientOptions to tenant specific ones
configuration.Bind($"{tenantName}:AWS", awsCognitoClientOptions); This allows me to have AWSCognitoClientOptions populated from a "general" config section and then mutated/changed to a tenant-specific configuration. This seems to be working locally but have to test it in lambda. The premise of this "solution" is that every tenant's request would get a new "instance" of "execution environment" with a brand new DI container and its own Singleton instance of CognitoUserPool. |
Beta Was this translation helpful? Give feedback.
-
I think I have a solution that does not require me messing around with how Cognito is declared in DI. This (https://referbruv.com/blog/posts/implementing-cognito-user-login-and-signup-in-aspnet-core-using-aws-sdk) solution does a "roll your own" authentication and thus instantiates CognitoUserPool and CognitoIdentityManager as a "hardcoded" dependency of a Scoped service called UserRepository, which in my case, would delay instantiation of these classes until I know which tenant the current request is for. |
Beta Was this translation helpful? Give feedback.
-
@IgorPietraszko Thanks for your response. Does this resolves your issue? Feel free to post the solution here for reference. |
Beta Was this translation helpful? Give feedback.
-
@ashishdhingra Its not a resolution but rather a workaround. I would still like, if possible, for someone on your or .NET/lambda team to elaborate on the question of Dependency Injection, Singletons and whether their instances survive between requests (warm startup) as is the case with CognitoUserPool which is added to DI as a Singleton. While there are some articles online from the point of view of Function invocation, I am specifically interested with ASP.NET (3.1 Core) Web API invocation when it runs in AWS Lambda. |
Beta Was this translation helpful? Give feedback.
-
In the above-mentioned workaround, I am running into a an issue trying to ListUsersAsync(). Getting an exception on Invalid Security Token. |
Beta Was this translation helpful? Give feedback.
-
Lambda & SingletonsYes, Lambda MAY keep a recently used instance to serve the next incoming request. This is known as a Warm Start and allows the lambda function to respond much quicker. In the case of ASP.NET, this means the Web Host persists between requests. As does the You can see this by adding a Singleton binding in public class SingletonModel
{
public string Data { get; set; }
}
public void ConfigureServices(IServiceCollection services)
{
services.AddSingleton<SingletonModel>(new SingletonModel{Data = $"Created on {DateTime.Now:s}"});
} Then update the API Controller to inject By comparing the Hash Code to a Scoped Service, we can see that the Singleton stays the same between requests while the Scoped doesn't public class ValuesController : ControllerBase
{
private readonly IAmazonSimpleEmailServiceV2 _amazonSimpleEmailService;
private readonly SingletonModel _singletonModel;
public ValuesController(
IAmazonSimpleEmailServiceV2 amazonSimpleEmailService,
SingletonModel singletonModel)
{
_amazonSimpleEmailService = amazonSimpleEmailService;
_singletonModel = singletonModel;
}
// GET api/values
[HttpGet]
public IEnumerable<string> Get()
{
return new string[]
{
_amazonSimpleEmailService.GetHashCode().ToString(),
$"Controller Hashcode: {GetHashCode()}",
$"SingletonModel: [Hash: {_singletonModel.GetHashCode()}, Data: {_singletonModel.Data}]"
};
}
}
Be careful with SingletonsDo be careful when deciding to bind a dependency as Singleton, as they can survive beyond servicing a single request. Additionally, asp.net in general is capable of servicing multiple requests concurrently, so Singletons need to be threadsafe. If your request pipeline relies on mutating Singleton you could open yourself up to a race condition where two concurrent requests could interfere with each other's state. UserRepository SolutionYou referenced the blog post https://referbruv.com/blog/posts/implementing-cognito-user-login-and-signup-in-aspnet-core-using-aws-sdk implementing a public class AmazonCognitoIdentityProviderFactory
{
private readonly IConfiguration _configuration;
public AmazonCognitoIdentityProviderFactory(IConfiguration config){
_configuration = config;
}
public IAmazonCognitoIdentityProvider BuildCognitoClient(string host)
{
// extract host to tenant mapping from config
var commonConfigSection = configuration.GetSection("Common");
// grab tenant name based on the host
var tenantName = commonConfigSection.GetValue<string>(host.Value);
var tenantOptions = new TenantOptions();
// bind TenantOptions to tenant specific config section
configuration.Bind(tenantName, tenantOptions);
// copy tenantOptions to AmazonCognitoIdentityProviderConfig
var config = new AmazonCognitoIdentityProviderConfig
{
RegionEndpoint = tenantOptions.RegionEndpoint;
// etc
}
return new AmazonCognitoIdentityProviderClient(config);
}
}
public class Startup
{
public void ConfigureServices(IServiceCollection services)
{
// note: in real application would recommend using a backing interface
services.AddSingleton<AmazonCognitoIdentityProviderFactory>();
}
} Then your API Controller methods would take the extra step of using the factory: public class ExampleController : ControllerBase
{
private readonly AmazonCognitoIdentityProviderFactory _cognitoFactory;
public ExampleController(AmazonCognitoIdentityProviderFactory cognitoFactory)
{
_cognitoFactory = cognitoFactory;
}
[HttpGet]
public IEnumerable<string> Get()
{
var cognitoClient = _cognitoFactory.BuildCognitoClient(Request.Host);
}
} This solution has a bit more overhead if you want to use multiple Services in so far as you'll either up creating a Factory per service, or adjusting your Factory to create multiple Services in one method go. However, it does have the advantage of not needing to carefully control the lifecycle of an Error using ListUsersAsync
Without a bit more context, I can't really help diagnose that 😄 . How did you end up building your |
Beta Was this translation helpful? Give feedback.
-
My solution is to plagiarize this code (https://github.com/aws/aws-aspnet-cognito-identity-provider/blob/master/src/Amazon.AspNetCore.Identity.Cognito/Extensions/CognitoServiceCollectionExtensions.cs) and changed the default ServiceLifetime to Scoped from Singleton. This should allow me to get a new instance of CognitoUserPool for every request thus differentiating between tenants (as each would be associated with a separate request). |
Beta Was this translation helpful? Give feedback.
-
Hello! Reopening this discussion to make it searchable. |
Beta Was this translation helpful? Give feedback.
Lambda & Singletons
Yes, Lambda MAY keep a recently used instance to serve the next incoming request. This is known as a Warm Start and allows the lambda function to respond much quicker.
In the case of ASP.NET, this means the Web Host persists between requests. As does the
IServiceCollection
that is populated inStartup.ConfigureServices
. Anything added as a Singleton in the initial request will therefor survive into subsequent requests.You can see this by adding a Singleton binding in
Startup