Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Should not reach here: undefined #3217

Open
caramboleyo-no2fa opened this issue Nov 18, 2024 · 0 comments
Open

Should not reach here: undefined #3217

caramboleyo-no2fa opened this issue Nov 18, 2024 · 0 comments

Comments

@caramboleyo-no2fa
Copy link

I am using stream to move a big data set, but it seems there is a cap of ~460 iterations that causes an error i dont understand.

This is my code:

let i = 0;
const stream = connection.query('SELECT * FROM schedules ORDER BY id ASC').stream();
stream.on('data', result => {
	stream.pause();
	db.collection('packages').deleteOne({ id: result.id }).then(() => {
		db.collection('packages').insertOne(result).then(() => {
			i++;
			if (i % 1000 === 0) {
				console.log('Processed', i);
			}
			stream.resume();
		});
	});
});
stream.on('end', () => {
	connection.end();
	mongoClient.close();
	console.log('Finished streaming', i);
});
      at readLengthCodedNumberExt (/home/bun/app/node_modules/mysql2/lib/packets/packet.js:231:13)
      at readLengthCodedString (/home/bun/app/node_modules/mysql2/lib/packets/packet.js:381:22)
      at next (32:68)
      at row (/home/bun/app/node_modules/mysql2/lib/commands/query.js:242:29)
      at execute (/home/bun/app/node_modules/mysql2/lib/commands/command.js:45:22)
      at handlePacket (/home/bun/app/node_modules/mysql2/lib/connection.js:481:34)
      at /home/bun/app/node_modules/mysql2/lib/connection.js:97:12
      at executePayload (/home/bun/app/node_modules/mysql2/lib/packet_parser.js:139:14)
      at /home/bun/app/node_modules/mysql2/lib/connection.js:104:25
227 |       res = resNumber.toString() === resString ? resNumber : resString;
228 |       return bigNumberStrings ? resString : res;
229 |     }
230 |     // eslint-disable-next-line no-console
231 |     console.trace();
232 |     throw new Error(`Should not reach here: ${tag}`);
                ^
error: Should not reach here: undefined
      at readLengthCodedNumberExt (/home/bun/app/node_modules/mysql2/lib/packets/packet.js:232:11)
      at readLengthCodedString (/home/bun/app/node_modules/mysql2/lib/packets/packet.js:381:22)
      at next (32:68)
      at row (/home/bun/app/node_modules/mysql2/lib/commands/query.js:242:29)
      at execute (/home/bun/app/node_modules/mysql2/lib/commands/command.js:45:22)
      at handlePacket (/home/bun/app/node_modules/mysql2/lib/connection.js:481:34)
      at /home/bun/app/node_modules/mysql2/lib/connection.js:97:12
      at executePayload (/home/bun/app/node_modules/mysql2/lib/packet_parser.js:139:14)
      at /home/bun/app/node_modules/mysql2/lib/connection.js:104:25
      at emit (node:events:183:48)

if i use async/await instead of thenables i have the same result, only when i dont do both it runs through, but then 3mio db operations stuck up in memory and make dev ops kill it.

all i want is to do this in the background a little slower.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

2 participants