feat!: Align Support for Storage Emulation

[danielbankhead]

Today, baseUrl is equivalent to the STORAGE_EMULATOR_HOST, when available. This leads to limitations when using the client. Here's the relevant code snippet:

nodejs-storage/src/storage.ts

Lines 650 to 668 in 1447269

	let apiEndpoint = 'https://storage.googleapis.com';
	let customEndpoint = false;
	// Note: EMULATOR_HOST is an experimental configuration variable. Use apiEndpoint instead.
	const EMULATOR_HOST = process.env.STORAGE_EMULATOR_HOST;
	if (typeof EMULATOR_HOST === 'string') {
	apiEndpoint = Storage.sanitizeEndpoint(EMULATOR_HOST);
	customEndpoint = true;
	}
	if (options.apiEndpoint) {
	apiEndpoint = Storage.sanitizeEndpoint(options.apiEndpoint);
	customEndpoint = true;
	}
	options = Object.assign({}, options, {apiEndpoint});
	// Note: EMULATOR_HOST is an experimental configuration variable. Use apiEndpoint instead.
	const baseUrl = EMULATOR_HOST || `${options.apiEndpoint}/storage/v1`;

Ideally, STORAGE_EMULATOR_HOST should only be a host (without any path associations, e.g. http://localhost:9199) while the client calls the appropriate path on the host (e.g. /storage/v1). That implementation should look something like this: c75b8b8

This requires coordination with Firebase and updates to Firebase Storage Emulator. Additionally, we should explore practices to ensure reliability and a consistent experience between Firebase and Google Cloud Storage (e.g. code consolidation and integration testing).

Related:

• refactor!: Revamp apiEndpoint #2400

Background:

• baseurl is not quite right when using STORAGE_EMULATOR_HOST #1314
• STORAGE_EMULATOR_HOST not handled correctly #2069
• fix: correct STORAGE_EMULATOR_HOST handling (#2069, #1314) #2070
• Version 6.5.3 breaks emulator usage #2086
• Revert "fix: correct STORAGE_EMULATOR_HOST handling (#2069, #1314)" #2087
• fix: revert STORAGE_EMULATOR_HOST handling #2089

[mgabeler-lee-6rs]

The Go GCS client looks like it had some similar issues in the past. There's a fair bit in that repo if you search issues for STORAGE_EMULATOR_HOST, but this ticket is the closest to this and has two PRs linked to fix it: googleapis/google-cloud-go#2476

The changes there look functionally equivalent to what I proposed in #2070, so I'm wondering if the firebase emulator may already have problems with the Go client? I have no experience with the firebase emulator, so I couldn't say for sure.

A thought: as a stepping stone to avoid a "hard break", might it be useful to temporarily permit a second environment variable indicating how to interpret STORAGE_EMULATOR_HOST, with the initial default being the current behavior, and some specific value indicating to use the "new style", and at some point that default changes, and later that second env var is dropped?

[mgabeler-lee-6rs]

Friendly check-in to see if there's any progress on decisions for how this might move forwards?

[danielbankhead]

@mgabeler-lee-6rs hey, thanks for the inquiry; we plan to sync with peer teams on this and other issues over the next few months.

[mgabeler-lee-6rs]

Just checking in on status and confirming that this is still of interest.

[danielbankhead]

@mgabeler-lee-6rs We have alignment between teams and we're likely to resolve this in the current quarter.

[mgabeler-lee-6rs]

Obviously missed Q2, Q3 maybe? :)

[danielbankhead]

@mgabeler-lee-6rs yea, I’m aiming for this half :)

[danielbankhead]

Update: we had a few other high priority features to take care of; however we still have this as a key item to get to shortly.