Special characters in entrypoint paths are always forcibly replaced

[airhorns]

Currently, esbuild replaces certain characters in entrypoint filenames for the corresponding output filenames when bundle is false. I assume this is desirable in some circumstances, but, if there's anything else that is expecting those file names to be consistent, or if those special characters have special meaning to other systems, this replacement causes some strife!

For me, I am using esbuild to build a bunch of TypeScript files for a nodejs project into JS files for deployment to production. We use a next.js style filepath-based routing system, where we have a file named GET-/foo.ts that the router discovers and registers as a route at /foo in our webserver. We glob our whole package for .ts files and then submit all of the found paths as entrypoints to esbuild with bundle: false, and esbuild builds them into a mirror image of the same directory structure. The problem comes from esbuild's special character replacement -- our filepath based router also supports wildcards, like GET-*.ts, or dynamic parameter segments, like POST-:application_slug.ts. esbuild replaces the : character and the * character in the output filenames such that the semantic meaning of them to the downstream filepath based router changes.

We're currently working around this by asking for the metafile from esbuild, building a mapping of entrypoint to output file, and moving the written outputs back to place they started so that the only thing that's different is the extension and not the filename itself. This is annoying and tricky though!

Could we add an option to disable the special character replacement? Or to pass a smaller list of characters that will be replaced?

PS: Thanks as always for an amazing piece of open source software!

[evanw]

I have designed esbuild to be cross-platform. It deliberately avoids doing things that work only on some platforms but not others. This is one of those things. Colon characters in path names don't work on Windows according to https://docs.microsoft.com/en-us/windows/win32/fileio/naming-a-file:

• Use any character in the current code page for a name, including Unicode characters and characters in the extended character set (128–255), except for the following:

  • The following reserved characters:

    • < (less than)
    • > (greater than)
    • : (colon)
    • " (double quote)
    • / (forward slash)
    • \ (backslash)
    • | (vertical bar or pipe)
    • ? (question mark)
    • * (asterisk)

  • Integer value zero, sometimes referred to as the ASCII NUL character.

  • Characters whose integer representations are in the range from 1 through 31, except for alternate data streams where these characters are allowed. For more information about file streams, see File Streams.

  • Any other character that the target file system does not allow.

If you want to have cross-platform incompatible file names, you are welcome to use write: false and then write out the files yourself. It sounds like you are already doing that.

[airhorns]

I have designed esbuild to be cross-platform.

This makes sense, and yeah, there is a workaround, but, it is a little surprising that most but not all files end up with just a changed extension. For the unawares like me it was easy to assume that all files would follow the rule. Totally happy to chalk that up to me being thick and leave esbuild as is.

That said, is it important to you that the output is identical cross platform? Or just that esbuild works on windows? One option would be to turn on the character replacement on windows where it is necessary, but leave it off for *nix where it isn't. In my case, my input files wouldn't even be allowed on windows such that I don't think I could even feed esbuild these files in the first place. At first blush it doesn't seem that crazy to assume entrypoints with special characters mean that outputs with special characters wouldn't break anything on the machine esbuild is running on. If folks copied outputs between OSes it'd break though.

I actually ended up doing write: false and manually write the output files to the place where I want them to be, but it ends up with yet more downstream complexity trying to move the sourcemaps as well as the compiled outputs into the right spot. It is a little regrettable I have to stream all the content back into node to then write it out to the FS, but not the end of the world either. Here's what I have so far before getting the sourcemap part working just to demonstrate:

  const workspaceRoot = path.resolve(path.join(__dirname, "..", "..", ".."));

  const result = await build({
    platform: "node",
    format: "cjs",
    target: ["node14"],
    write: false,
    outdir: path.join(workspaceRoot, "packages"),
    bundle: false,
    sourcemap: true,
    metafile: true,
    entryPoints: await globby(
      [
        "packages/{@starscream,@gadgetinc,api,app-sandbox,docs,state-trees,api-client,gelly,web}/**/*.{ts,tsx}",
        "!**/node_modules",
        "!packages/api/spec/integration",
      ],
      { cwd: workspaceRoot }
    ),
    absWorkingDir: workspaceRoot,
  });

  const outputsByDestinationPath = keyBy(result.outputFiles, "path");

  /**
   * esbuild rewrites special characters out of filenames, replacing them with an underscore. The specific filenames are important to use for Starscream's routing syntax, so we need to write files to exactly the same path as the input file. By mapping the metafile that connects input files to output files we can ensure we keep the original filenames for any that might have been changed by esbuild.
   */
  const writes = [];
  for (const [destinationString, details] of Object.entries(result.metafile.outputs)) {
    if (details.entryPoint) {
      const entrypoint = path.parse(details.entryPoint);
      const absoluteDestination = path.join(workspaceRoot, destinationString);
      const buildResult = outputsByDestinationPath[absoluteDestination];
      if (!buildResult) {
        throw new Error(`no build result found for path ${absoluteDestination}`);
      }
      const esbuildDestination = path.parse(destinationString);

      const newDestination = path.join(workspaceRoot, entrypoint.dir, entrypoint.name + esbuildDestination.ext);
      writes.push(fs.writeFile(newDestination, buildResult.contents, "utf-8"));
    }
  }

  await Promise.all(writes);

If you are unconvinced feel free to close! Thanks for your consideration!

[nettybun]

@airhorns Did you try entryNames? https://esbuild.github.io/api/#entry-names Tthe documentation says:

[name]
This is the original file name of the entry point without the extension.

I'd hope "original" means unchanged original, and if not then maybe the documentation should be updated to say "This is the normalized file name ..."

@evanw not to advocate for feature creep but a backwards-compatible way to support this could be [name:unchanged] and [name:normalized] where [name] defaults to normalized.

[evanw]

I think it's actually already possible to do this using the alternate form of entryPoints that allows you to override the automatically-generated name with a custom name. For example:

$ touch 'GET-*.ts'
$ esbuild 'GET-*.ts' --outdir=out

  out/GET-_.js  25b 

$ esbuild 'GET-*=GET-*.ts' --outdir=out

  out/GET-*.js  25b 

This is documented here: https://esbuild.github.io/api/#entry-points. Closing as this is an edge case and there is a straightforward workaround.

[airhorns]

Nice, that worked well for me, thanks!