Sveltekit - Ineffective mark-compacts near heap limit Allocation failed - JavaScript heap out of memory #10589
Comments
github-actions
bot
added
the
Package: sveltekit
|
Hi @MSDev201 thanks for writing in! This is a somewhat known issue although we're not entirely sure what exactly is causing it. So far we've mostly pinned it down to
Unfortunately, we don't have a solution for this yet which is also the case because we haven't received a minimal reproducible example yet. If you're able to provide one, we can debug this further. If you have a highly customized build config by any chance, I suggest removing a couple of plugins to try to narrow it down. Let me know if this helps! |
|
@Lms24 I could narrow it down a little more. The Issue is happening here https://github.com/getsentry/sentry-javascript/blob/a7097d9ba2a74b2cb323da0ef22988a383782ffb/packages/sveltekit/src/vite/sourceMaps.ts#L169C7-L169C14 First of all, Im not sure if its correct to not await the flattening of the source maps. All processes are over when the flattening still is happening. for(const file of jsFiles) {
...
}This doesnt solve the problem thoug but it makes more clear that the issue is not happening after the source maps are uploaded, but before. To narrow it further down, I logged all files which where passed to sorcery. One of those files is a big 0.5mb js file with a 1mb source map. Sorcery appears to load also additional required source maps (possibly even circulating?) and I assume that this is where the out of memory error is caused. |
getsantry
bot
added
Waiting for: Product Owner
and removed
Waiting for: Community
labels
getsantry
bot
moved this from Waiting for: Community
to Waiting for: Product Owner
in GitHub Issues with 👀 2
oof you're completely right. This probably worked out by chance (more or less I'd guess), given the invocation of Sentry CLI to actually start uploading but it's definitely a bug. Thanks for flagging!
Hmm this is a good point. Originally, I added sorcery to flatten the weird sourcemaps created by the Node Adapter (#7811). Which adapter are you using @MSDev201? I wonder if we should simply scope flattening to the node adapter to avoid this for other ones. |
We are using @sveltejs/adapter-node version 2.0.0 |
We didn't properly await sourcemaps flattening via sorcery before proceeding to upload them. The reason is that the async callbacks in `forEach` weren't awaited. A `for` loop is the better approach here. Wondering if we should lint against async `forEach` callbacks. This behaviour could have caused various inconsistencies. My suspicion is that the timing worked _well enough_ in most cases but we definitely want to properly await this step. Thanks to @MSDev201 for bringing this up! Unfortunately this likely won't fix #10589 as a whole :(
|
So the issue in sorcery is that I get an infinite loop in the before mentioned source map. Quick simple example of my setup: someFile.js someCode();
//# sourceMappingURL=someFile.js.mapsomeFile.js.map {
"sources": [
"someFile.js"
]
// ...
}@Lms24 this issue was closed because of the |
|
@MSDev201 thanks a lot for investigating! To me, this looks like there's already a problem in the generated source map but sorcery also shouldn't cause an infinite loop because of this. Were you able to pin this to a specific source file that causes the self-reference? |
We didn't properly await sourcemaps flattening via sorcery before proceeding to upload them. The reason is that the async callbacks in `forEach` weren't awaited. A `for` loop is the better approach here. Wondering if we should lint against async `forEach` callbacks. This behaviour could have caused various inconsistencies. My suspicion is that the timing worked _well enough_ in most cases but we definitely want to properly await this step. Thanks to @MSDev201 for bringing this up! Unfortunately this likely won't fix #10589 as a whole :(
We didn't properly await sourcemaps flattening via sorcery before proceeding to upload them. The reason is that the async callbacks in `forEach` weren't awaited. A `for` loop is the better approach here. Wondering if we should lint against async `forEach` callbacks. This behaviour could have caused various inconsistencies. My suspicion is that the timing worked _well enough_ in most cases but we definitely want to properly await this step. Thanks to @MSDev201 for bringing this up! Unfortunately this likely won't fix #10589 as a whole :(
|
@Lms24 I was able to pin point which file caused it. Im not able to share it though. The file which causes the problem for us, has a source map which is not generated by The following will cause the infinite loop (tested on two sveltekit projects): static/test/test.js console.log("Hello world!");
//# sourceMappingURL=test.js.mapstatic/test/test.js.map {
"mappings": "ACAuB,QAAQ,GAAG,CAAC",
"sources": [
"<anon>",
"test.js"
],
"sourcesContent": [
"const $6c8bc724de647767$var$sayHello = ()=>console.log(\"Hello world!\");\n$6c8bc724de647767$var$sayHello();\n\n\n//# sourceMappingURL=test.js.map\n",
"const sayHello = () => console.log(\"Hello world!\");\nsayHello();\n"
],
"names": [
"console",
"log"
],
"version": 3,
"file": "test.js.map"
}Running the build now will cause the infinite loop in sorcery, but because the file is so much smaller, it doesnt run into the 'out of memory' error but will cause the following error: Regarding the parcel build (I dont think its anything special): "test": {
"source": "test.js",
"outputFormat": "esmodule",
"distDir": "../static/test",
"optimize": true,
"includeNodeModules": []
}and the source file const sayHello = () => console.log("Hello world!");
sayHello(); |
|
Hey @MSDev201 thanks so much for investigating this further! So this is a really painful problem because I'm afraid, to fix this on our end, we'll somehow need to check the source map content for such a circular reference which already smells brittle 😅 For some context: The reason we need sorcery to flatten the source map is because the node adapter, in its own build step, will emit source maps that don't point to the original source code but to the code of the intermediate sveltekit build output. So if the circular reference actually happens in this intermediate source map, our plugin would effectively need to trace the source map chain itself, which sounds even more brittle. Solution ideas:
UPDATE: found the issue in the sveltekit repo: sveltejs/kit#10040 |
Outstanding upstream issues need to be fixed first: - sveltejs/kit#10040 - getsentry/sentry-javascript#10589
Is there an existing issue for this?
How do you use Sentry?
Self-hosted/on-premise
Which SDK are you using?
@sentry/sveltekit
SDK Version
7.100.1
Framework Version
Sveltekit 2.4.0
Link to Sentry event
No response
SDK Setup
vite.config.js
Steps to Reproduce
npx @sentry/wizard@latest -i sveltekitand follow the promptsvite buildWe are having some circular dependencies in a few of our dependencies, not sure if this could cause the issue.
Expected Result
Vite build should finish with a success code and dont consume all of my memory.
Actual Result
Sourcemap and Release are successfully published and it looks like everything has finished successfully but right before stopping, it consumes infinite amount of memory. I tested with
NODE_OPTIONS=--max-old-space-size=16384but still get a 'out of memory' error.The text was updated successfully, but these errors were encountered: