Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

deleteTopicRecords returns "KafkaJSNonRetriableError: Do not know how to serialize a BigInt" error while use logLevel = "DEBUG" #1726

Open
habi4ek opened this issue Nov 26, 2024 · 0 comments

Comments

@habi4ek
Copy link

habi4ek commented Nov 26, 2024

Describe the bug
deleteTopicRecords method returns error after setting "logLevel" = 'DEBUG'

To Reproduce

  1. set logLevel: logLevel.DEBUG in kafka config
  2. create a topic
  3. write a message into topic
  4. call deleteTopicRecords method

Expected behavior
All messages in the topic are deleted

Observed behavior
kafka throw an error "KafkaJSNonRetriableError: Do not know how to serialize a BigInt"

Environment:

  • OS: [Mac OS 15.1.1 (24B91)]
  • KafkaJS version [2.2.4]
  • Kafka version [inter.broker.protocol.version = 3.4-IV0]
  • NodeJS version [18.16.0]

Example

'use strict';

const {
    Kafka,
    logLevel
} = require('kafkajs');

const kafka = new Kafka({
    clientId: 'example-client',
    brokers:  ['localhost:9092'],
    logLevel: logLevel.DEBUG
});

const topicName = 'example-topic';

(async () => {
    const admin    = kafka.admin();
    const producer = kafka.producer();

    try {
        // Step 1: Connect to Admin and Create Topic
        await admin.connect();
        console.log(`Creating topic: ${topicName}`);
        await admin.createTopics({
            topics: [{ topic: topicName, numPartitions: 3 }],
        });

        console.log(`Topic "${topicName}" created.`);

        // Step 2: Produce a Message
        await producer.connect();
        const message = { key: 'example-key', value: 'Hello, KafkaJS!' };

        console.log(`Producing message: ${JSON.stringify(message)}`);
        await producer.send({
            topic:    topicName,
            messages: [message],
        });

        console.log(`Message produced to topic "${topicName}".`);

        // Step 3: Fetch Offsets
        const topicOffsets = await admin.fetchTopicOffsets(topicName);
        console.log('Current Topic Offsets:', topicOffsets);

        // Step 4: Delete All Messages
        const partitions = topicOffsets.map(({ partition }) => ({
            partition: parseInt(partition, 10),
            offset:    '-1', // Use -1 to delete all messages
        }));

        console.log(`Deleting records from topic "${topicName}"...`);
        await admin.deleteTopicRecords({
            topic: topicName,
            partitions,
        });

        console.log(`All messages deleted from topic "${topicName}".`);

        const topicOffsets2 = await admin.fetchTopicOffsets(topicName);
        console.log('Current Topic Offsets 2:', topicOffsets2);
    } catch (error) {
        console.error('Error:', error);
    } finally {
        // Cleanup
        await producer.disconnect();
        await admin.disconnect();
    }
})();

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant