add enable_import_extensions option

[nvandamme]

No description provided.

[timostamm]

Thanks for this, Nicolas!

I think that CI fails because the old TypeScript version being used doesn't understand .js extensions in import statements.

If you were to bump TypeScript in packages/test-generated/package.json to a recent version, I think test-generated should pass, which will be a good proof that it works. Would you like to do so?

CI failure log:

@protobuf-ts/test-generated: Error: Cannot find module './google/protobuf/empty.js'
@protobuf-ts/test-generated: Require stack:
@protobuf-ts/test-generated: - /home/circleci/project/packages/test-generated/ts-out/exclude-options.ts
@protobuf-ts/test-generated: - /home/circleci/project/packages/test-generated/spec/exclude-options.spec.ts
@protobuf-ts/test-generated: - /home/circleci/project/packages/test-generated/node_modules/jasmine/lib/jasmine.js
@protobuf-ts/test-generated: - /home/circleci/project/packages/test-generated/node_modules/jasmine/bin/jasmine.js
@protobuf-ts/test-generated:     at Function.Module._resolveFilename (internal/modules/cjs/loader.js:1030:15)
@protobuf-ts/test-generated:     at Function.Module._resolveFilename (/home/circleci/project/packages/test-generated/node_modules/tsconfig-paths/lib/register.js:75:40)
@protobuf-ts/test-generated:     at Function.Module._load (internal/modules/cjs/loader.js:899:27)
@protobuf-ts/test-generated:     at Module.require (internal/modules/cjs/loader.js:1090:19)
@protobuf-ts/test-generated:     at require (internal/modules/cjs/helpers.js:75:18)
@protobuf-ts/test-generated:     at Object.<anonymous> (/home/circleci/project/packages/test-generated/ts-out/exclude-options.ts:4:1)
@protobuf-ts/test-generated:     at Module._compile (internal/modules/cjs/loader.js:1201:30)
@protobuf-ts/test-generated:     at Module.m._compile (/home/circleci/project/packages/test-generated/node_modules/ts-node/src/index.ts:858:23)
@protobuf-ts/test-generated:     at Module._extensions..js (internal/modules/cjs/loader.js:1221:10)
@protobuf-ts/test-generated:     at Object.require.extensions.<computed> [as .ts] (/home/circleci/project/packages/test-generated/node_modules/ts-node/src/index.ts:861:12)
@protobuf-ts/test-generated: Makefile:45: recipe for target 'test-spec' failed

[nvandamme]

Hi @timostamm ,

Thanks for the feedback, sure!

For now, i'm using a quick'n dirty script on my side to ensure all local imports has an extension.
This script is ran after protobuf-ts generation on all *.ts and *.js generated files :

import * as fs from "fs";
import glob from "glob";

function add_import_extensions(er, files) {
    files.forEach(file => {
        let data = fs.readFileSync(file, {encoding: "utf8"});
        data = data.replace(/from\ "\.\/(((?!\.js).)*)";$/gm, "from \"./$1.js\";");
        data = data.replace(/from\ "\.\.\/(((?!\.js).)*)";$/gm, "from \"../$1.js\";");
        fs.writeFileSync(file, data);
    });
}

glob("./api/js/**/*.ts", {}, add_import_extensions);
glob("./api/js/**/*.js", {}, add_import_extensions);

[debkanchan]

@nvandamme any updates on this? Since ts 4.7 released this is crucial

[faustbrian]

Any update on this? ESM is becoming more widespread so would be nice to have this added.

[sachaw]

@timostamm are you happy with this implementation, or are there changes you want performed, I've just hit a case where this is now a blocker, thanks.

[smnbbrv]

for those who suffers and cannot wait until this gets merged, just run this after generation

// proto-patch.js
import glob from "glob";
import fs from "fs";

const protoRoot = 'srv/proto';

glob(protoRoot + '/**/*.ts', async (err, files) => {
  files.forEach(file => {
    let content = fs.readFileSync(file, 'utf-8');

    content = content.split('\n').map(s => s.replace(/^(import .+? from ["']\..+?)(["'];)$/, '$1.js$2')).join('\n');

    fs.writeFileSync(file, content, 'utf-8')
  });
});

[hugebdu]

any updates?

[shrpne]

@timostamm can we move it forward? without .js extension generated imports are broken for now

[bbeesley]

@timostamm any update on when this can be merged? What's holding it back at the moment?

[jan-dolejsi]

@timostamm , if you still cannot get your test pipeline to work with this, please consider releasing this as experimental. You can make it obvious that there is no guarantee, if you name it --experimental_enable_import_extensions for example. The ESM is taking over the world and we all would love this component to come along.

[nebez]

Another version without npm dependencies while we await this feature to get merged. I use it as: buf generate && node ./patch-esm.js && tsc

#! /usr/bin/env node
import fs from 'node:fs';
import { dirname } from 'node:path';
import { fileURLToPath } from 'node:url';

const currentDir = dirname(fileURLToPath(import.meta.url));

const collectFiles = (directory, fileList = []) => {
    fs.readdirSync(directory)
        .map((innerFile) => `${directory}/${innerFile}`)
        .forEach((file) =>
            fs.statSync(file).isDirectory() ? collectFiles(file, fileList) : fileList.push(file)
        );
    return fileList;
};

const files = collectFiles(`${currentDir}/dist`).filter((file) => file.endsWith('.ts'));

files.forEach((file) => {
    const content = fs.readFileSync(file, 'utf8');
    const updated = content
        .split('\n')
        .map((s) => s.replace(/(} from ["']\..+(?<!\.js))(["'];)$/, '$1.js$2'))
        .join('\n');
    fs.writeFileSync(`${file}`, updated, 'utf-8');
});

[ngbrown]

Without the .js extensions being added to the imports and my package.json containing "type": "module", I am getting:

node:internal/errors:496
    ErrorCaptureStackTrace(err);
    ^

Error [ERR_MODULE_NOT_FOUND]: Cannot find module 'D:\dev\Grpc\GrpcGreeter\GrpcGreeterClientNodeJs\dist\gen\greet' imported from D:\dev\Grpc\GrpcGreeter\GrpcGreeterClientNodeJs\dist\gen\greet.client.js
    at new NodeError (node:internal/errors:405:5)
    at finalizeResolution (node:internal/modules/esm/resolve:327:11)
    at moduleResolve (node:internal/modules/esm/resolve:980:10)
    at defaultResolve (node:internal/modules/esm/resolve:1193:11)
    at ModuleLoader.defaultResolve (node:internal/modules/esm/loader:404:12)
    at ModuleLoader.resolve (node:internal/modules/esm/loader:373:25)
    at ModuleLoader.getModuleJob (node:internal/modules/esm/loader:250:38)
    at ModuleWrap.<anonymous> (node:internal/modules/esm/module_job:76:39)
    at link (node:internal/modules/esm/module_job:75:36) {
  url: 'file:///D:/dev/Grpc/GrpcGreeter/GrpcGreeterClientNodeJs/dist/gen/greet',
  code: 'ERR_MODULE_NOT_FOUND'
}

I put in the error message for easy searching, but this PR is over two years old. Is there going to a release with it soon?

[dionysiusmarquis]

I’m testing with the experimental node native typescript feature. In the past node required .js extensions for typescript imports. Now they are shifting to only allow .ts extensions, so it seems. At least I only could get it running with .ts extensions.