Skip to content

Commit

Permalink
Avoid using Promise.all to iterate through paths
Browse files Browse the repository at this point in the history
Originally, this was added to ensure maximum parallelism and
speed. But we are seeing out of memory errors on larger
repositories.

This changes avoids using Promise.all and instead uses a for loop.
  • Loading branch information
aeisenberg committed May 12, 2021
1 parent cf424ea commit 86d4428
Showing 1 changed file with 5 additions and 3 deletions.
8 changes: 5 additions & 3 deletions src/directory.ts
Original file line number Diff line number Diff line change
Expand Up @@ -166,15 +166,17 @@ export class LocDir {
};
} = {};

await Promise.all(paths.map(async (pathItem) => {
// We _could_ use Promise.all to count the files in parallel, but that
// would lead to out of memory errors when there are many files.
for (const pathItem of paths) {
const fullPath = slash(path.join(this.cwd, pathItem));
if (
!pathItem ||
this.ignoreLanguage(pathItem) ||
!(await fs.pathExists(fullPath)) ||
(await fs.stat(fullPath)).isDirectory()
) {
return;
continue;
}
const file = new LocFile(fullPath);
const fileLineInfo = await file.getFileInfo();
Expand All @@ -192,7 +194,7 @@ export class LocDir {
[fileLineInfo.languages]: language,
};
files.push(fullPath);
}));
}

return {
files,
Expand Down

0 comments on commit 86d4428

Please sign in to comment.