Remove LF, CRLF, CR from output to get a consistent [contenthash]

[peterhirn]

Plugin creates identical output files with different contenthash-values when input is LF vs CRLF.

I think the content-hash should be calculated based on the output?

[alexander-akait]

@peterhirn we already calculate contenthash on output, we need remove LF, CRLF, CR lines from output, for css it doesn't have mean, feel free to send a PR

[peterhirn]

Thank you for your response. I was looking at the output generate by optimize-css-assets-webpack-plugin, that's where I got the wrong assumption about the content-hash generation. I'll change the title.

[KingScooty]

I have a feeling i'm experiencing the same issue. We've got snapshots that keep failing due to changed CSS contenthashes, when the CSS rules in each file are exactly the same. What's weird is the hashes are the same when building on the same machine, but different when a new person clones the project down and does a build.

Suggests contenthashes are changing on a machine by machine basis :/

[alexander-akait]

@KingScooty yep, i think it is bug, anyway can you create minimum reproducible test repo to ensure problem in LF/CRLF and CR, thanks!

[alexander-akait]

Investigation (include other bundlers):

• webpack
• parcel
• rollup (using plugin)

All bundlers (list above) generate other hash (contenthash for webpack) if you add/remove spaces/newlines or you change LF, CRLF and CR to other value.

So it is normal what contenthash changes when you change line ending to other value.

Also i was wrong, it is not safe change line ending, it can potential break code. Also you usually use minimizer for production (for example cssnano) so you file(s) doesn't contain lines and contenthash should be same on difference computers. Using contenthash for dev purpose is not recommended (decrease compilation time).

If you still have difference contenthash on difference computers maybe some plugins (postcss or webpack) adding something new in source code based on your environment or your project use difference line ending styles (recommendation - put .editorconfig and use prettier or stylelint for catch problems with difference line ending in source code).

Anyway feel free to open new issue, if you still have difference contenthash on difference computers following the recommendations above.

[Lyrkan]

Anyway feel free to open new issue, if you still have difference contenthash on difference computers following the recommendations above.

@evilebottnawi I think the issue wasn't really related to LF/CRLF/CR, there is another problem leading to inconsistent [contenthash] since mini-css-extract-plugin@^0.4.4: #308

Since it basically gives a different hash depending on the root path (from what I understand) it is very likely to occur when building on different OS.

[alexander-akait]

@Lyrkan problem with source map, postcss generate absolute source map, so sources always have difference paths, known issue, we working on this

[redfox05]

We're also having this issue. Same hash is generated on one machine, but the content hash keeps changing when a different developer does a webpack build and nothing has changed in the content. Then it stays the same for that developer, but changes again when another developer does a build.

Any tips for working around this?

@evilebottnawi you mention that this is a known issue, do you have an open issue number we can track, as this one is closed, so I'm assuming the known issue is known by a different id?

[alexander-akait]

fixed in webpack@5, can't be fixed in webpack@4

[redfox05]

Thanks for the quick reply @evilebottnawi I just edited my reply to provide more context. We are actually running webpack 5.
Which version of your plugin is it fixed in for webpack 5?

Some relevant version numbers below for what we are running if it helps:

    "css-loader": "4.3.0",
    "css-minimizer-webpack-plugin": "1.1.5",
    "mini-css-extract-plugin": "1.0.0",
    "webpack": "5.1.3",
    "webpack-cli": "4.0.0",

[redfox05]

Noticed theres a few new versions released, updated now to the following, but the issue is still present.

    "css-loader": "5.0.0",
    "css-minimizer-webpack-plugin": "1.1.5",
    "mini-css-extract-plugin": "1.2.1",
    "webpack": "5.3.1",
    "webpack-cli": "4.1.0",

fixed in webpack@5, can't be fixed in webpack@4

@evilebottnawi, could you provide more context around what was fixed as per your comment above?

Thanks

[alexander-akait]

in production mode you will have contenthash === hash of content of your assets, you will try to calculate it using md4

[redfox05]

@evilebottnawi I'm afraid I don't quite follow. Could you clarify?
Do you mean we should set a flag of some sort in production mode? Would this be in the main webpack.optimizations, or in the plugin for mini-css-extract?
And are you saying its currently using md4, and thats the problem, or md4 is the solution? And if so, what do you suggest we do to fix our issue?

Thanks

[alexander-akait]

• we never change lf/crlf/cr to other values, because it is unsafe
• webpack@5 calculate real content hash of assets in production mode, you can enable this for development, but it reduce performance and unnecessary, if you don't use [hash]/[contenthash]/etc hashes in developers
• md4 is the faster algorithm, it is used for not cryptographic so it is safe to use
• what is the problem

[redfox05]

Thanks for the clarification.

This is in production mode.

The problem is as mentioned above, but I will go in more detail.
For our CSS files, mini-css-extract-plugin pulls the CSS out of the JS files as expected, and puts the CSS into .css files in the dist folder. So far so good. Even when runnning the build again, nothing is changed according to git, and everything is the same.

However, if we now commit all this to git, and another developer pulls this, and does a build. Something else happens.

The content hash on the .css file names change when a different developer does a webpack build and nothing has changed in the content. Then it stays the same for that developer, but changes again when another developer does a build.

This is what git shows has changed, even though no code has changed prior to running the build command on the second machine. Both source and sourcemap hash change.

Does that help clarify it, and is that the same as the original issue in this thread, or would you like me to create another issue on github?

Thanks

[alexander-akait]

Interesting, can you create reproducible test repo, do you use latest css-loader/postcss-loader/sass-loader/less-loader/etc?

[redfox05]

Yes, all the latest as per the ones I listed above

Noticed theres a few new versions released, updated now to the following, but the issue is still present.

    "css-loader": "5.0.0",
    "css-minimizer-webpack-plugin": "1.1.5",
    "mini-css-extract-plugin": "1.2.1",
    "webpack": "5.3.1",
    "webpack-cli": "4.1.0",

Bit tricky on a test repo, dont really know how to separate everything from ours.

[alexander-akait]

Without source maps hashes are same on different computers?

[redfox05]

Hmm, thats one I have yet to try, will try that now

[redfox05]

I did it locally and ofcourse the hashes change, as the contents now has no reference to a sourcemap.
I won't be able to see what it does on the other machine, as that one is in Canada, so will have to wait till they come online to try that and truly know.

We do need sourcemaps either way, although I agree its a good thing to rule out.

[alexander-akait]

We do need sourcemaps either way, although I agree its a good thing to rule out.

yes, I want to found the problem to focus on fixing

[redfox05]

We have come across line ending issues before, but seeing as this is CSS that is pulled into the JS, minified, and then extracted out again by your plugin, I dont see how line endings can come into it, as it would be lines generated by webpack/plugins. One machine is on Mac (mine), and the other Linux.

Will have to wait a couple more hours to test on the other machine.

[alexander-akait]

We have come across line ending issues before, but seeing as this is CSS that is pulled into the JS, minified, and then extracted out again by your plugin, I dont see how line endings can come into it, as it would be lines generated by webpack/plugins. One machine is on Mac (mine), and the other Linux.

yep, i think you faced with out problem

Will have to wait a couple more hours to test on the other machine.

Feel free to ping me

[redfox05]

@evilebottnawi they just came online and we did the test with no sourcemaps, error still occurs.
All packages up to date and matching on both machines, with npm install too.

The files are binary equal:

$ sha1sum app-styles-e81d36db.css app-styles-491e4748.css
9665fd0a858415545dff989426273fe326c41646  app-styles-e81d36db.css
9665fd0a858415545dff989426273fe326c41646  app-styles-491e4748.css

Yet something is still changing the hash, even though contents are identical. Thus can't be line endings either.

[alexander-akait]

need to look at plugins, maybe some change something, we have tests on this case and it is success on windows/macos/linux, can you show diff without source maps?

[redfox05]

The diff shows no difference between the files. As confirmed by the sha checksum being the same.

The route the css takes with regards to plugins:
.css file -> webpack -> css-loader -> MiniCssExtractPlugin

[alexander-akait]

Do you use mode: 'production'?

[redfox05]

Do you use mode: 'production'?

Yes

[alexander-akait]

Weird, hard to say why it happens, do you use sass/less/postcss?

[redfox05]

Weird, hard to say why it happens, do you use sass/less/postcss?

We dont use any of those, the build process is literally plain .css file -> webpack -> css-loader -> MiniCssExtractPlugin

We can also use Gitter chat to discuss if thats quicker/easier?

[alexander-akait]

Let's use twitter https://twitter.com/evilebottnawi, feel free to DM, but I think without reproducible test repo I can'y help 😞

[alexander-akait]

Maybe edge case