Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Automatically add / strip BOM #3040

Closed
vsemozhetbyt opened this issue Sep 24, 2015 · 3 comments
Closed

Automatically add / strip BOM #3040

vsemozhetbyt opened this issue Sep 24, 2015 · 3 comments
Labels
feature request Issues that request new features to be added to Node.js. fs Issues and PRs related to the fs subsystem / file system. i18n-api Issues and PRs related to the i18n implementation.

Comments

@vsemozhetbyt
Copy link
Contributor

Node does not add or strip BOM when reading or writing utf16le or utf8 files. User must do it manually in all the text IOs.

It is especially inconvenient with streams: for example, in reading huge file line by line, user must check the BOM or some variabe (alreadyStripped) for each line.

Would it be handy to entrust these actions to Node?
@brendanashworth brendanashworth added the fs Issues and PRs related to the fs subsystem / file system. label Sep 24, 2015
@mscdex mscdex added the question Issues that look for answers. label Sep 24, 2015
@sindresorhus
Copy link

You can use strip-bom or strip-bom-stream, but I agree it would be nice if it were in core.

@ChALkeR ChALkeR added semver-major PRs that contain breaking changes and should be released in the next major version. and removed semver-major PRs that contain breaking changes and should be released in the next major version. labels Sep 24, 2015
@vsemozhetbyt
Copy link
Contributor Author

vsemozhetbyt commented Sep 24, 2015 •
edited
Loading

@sindresorhus
Thank you.

Well, it do seems as a bit excessive way to add / remove just one character...

In text2file / file2text single gulping we can as well use '\uFEFF' + text / text.replace(/^\uFEFF/, '').

For strems to add the whole transform stream in the chain could be as resource consuming as an additional operation in each event handler call.

@ChALkeR ChALkeR added the feature request Issues that request new features to be added to Node.js. label Sep 24, 2015
@Fishrock123 Fishrock123 removed the question Issues that look for answers. label Oct 7, 2015
@srl295 srl295 added the i18n-api Issues and PRs related to the i18n implementation. label Oct 27, 2015
@evanlucas
Copy link
Contributor

Closing as this can be done easily in userland.

@vsemozhetbyt vsemozhetbyt mentioned this issue May 22, 2016
Closed
@vsemozhetbyt vsemozhetbyt mentioned this issue Mar 9, 2017
Closed
@dinoboff dinoboff mentioned this issue Jun 26, 2017
Closed
rmg added a commit to rmg/cipm that referenced this issue Sep 2, 2017
@rmg
strip BOM from JSON files being read
af3f08f 
fs.readFile* don't strip the BOM header, even when specifying 'utf8' as
the encoding, and JSON.parse() doesn't handle it either. There are
technically a bunch of BOM indicators, but this is the one seen most
commonly and actually appears in a number of package.json files in the
wild.

See nodejs/node#6924, nodejs/node#3040 for background.
rmg added a commit to rmg/cipm that referenced this issue Sep 2, 2017
@rmg
fix: strip BOM when reading JSON files
b87646c 
fs.readFile* don't strip the BOM header, even when specifying 'utf8' as
the encoding, and JSON.parse() doesn't handle it either. There are
technically a bunch of BOM indicators, but this is the one seen most
commonly and actually appears in a number of package.json files in the
wild.

See nodejs/node#6924, nodejs/node#3040 for background.
@rmg rmg mentioned this issue Sep 2, 2017
Merged
rmg added a commit to rmg/cipm that referenced this issue Sep 2, 2017
@rmg
fix: strip BOM when reading JSON files
3aa9b86 
fs.readFile* don't strip the BOM header, even when specifying 'utf8' as
the encoding, and JSON.parse() doesn't handle it either. There are
technically a bunch of BOM indicators, but this is the one seen most
commonly and actually appears in a number of package.json files in the
wild.

See nodejs/node#6924, nodejs/node#3040 for background.
rmg added a commit to rmg/cipm that referenced this issue Sep 3, 2017
@rmg
fix: strip BOM when reading JSON files
66a3363 
fs.readFile* don't strip the BOM header, even when specifying 'utf8' as
the encoding, and JSON.parse() doesn't handle it either. There are
technically a bunch of BOM indicators, but this is the one seen most
commonly and actually appears in a number of package.json files in the
wild.

See nodejs/node#6924, nodejs/node#3040 for background.
rmg added a commit to rmg/cipm that referenced this issue Sep 3, 2017
@rmg
fix: strip BOM when reading JSON files
6fb3db0 
fs.readFile* don't strip the BOM header, even when specifying 'utf8' as
the encoding, and JSON.parse() doesn't handle it either. There are
technically a bunch of BOM indicators, but this is the one seen most
commonly and actually appears in a number of package.json files in the
wild.

See nodejs/node#6924, nodejs/node#3040 for background.
rmg added a commit to rmg/cipm that referenced this issue Sep 5, 2017
@rmg
fix: strip BOM when reading JSON files
54cf9de 
fs.readFile* don't strip the BOM header, even when specifying 'utf8' as
the encoding, and JSON.parse() doesn't handle it either. There are
technically a bunch of BOM indicators, but this is the one seen most
commonly and actually appears in a number of package.json files in the
wild.

See nodejs/node#6924, nodejs/node#3040 for background.
zkat pushed a commit to zkat/cipm that referenced this issue Sep 6, 2017
@rmg @zkat
fix(json): strip BOM when reading JSON files (#8)
2529149 
fs.readFile* don't strip the BOM header, even when specifying 'utf8' as
the encoding, and JSON.parse() doesn't handle it either. There are
technically a bunch of BOM indicators, but this is the one seen most
commonly and actually appears in a number of package.json files in the
wild.

See nodejs/node#6924, nodejs/node#3040 for background.
This was referenced Sep 17, 2021
Open
Open
Open
Open
Open
Open
Open
Open
This was referenced Sep 18, 2021
Open
Open
Open
Closed
@lyonsil lyonsil mentioned this issue Aug 9, 2024
Open
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
feature request Issues that request new features to be added to Node.js. fs Issues and PRs related to the fs subsystem / file system. i18n-api Issues and PRs related to the i18n implementation.
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
8 participants
@mscdex @sindresorhus @ChALkeR @evanlucas @srl295 @Fishrock123 @brendanashworth @vsemozhetbyt