Optionally log master secrets for TLS connections

[jsha]

Sometimes it's necessary to decrypt your own TLS connections to debug their contents. Wireshark supports this quite nicely with its decryption feature. For non-DH key agreement, you simply provide the private key of the server. However, for DH key agreement, or when you are acting only as a client, that doesn't work. Firefox and Chrome support the environment variable SSLKEYLOGFILE to write the master secrets used to a file, for decryption by Wireshark. It would be great to support this or a similar mechanism for logging master secrets in Node.

Key log format: https://developer.mozilla.org/en-US/docs/Mozilla/Projects/NSS/Key_Log_Format
Helpful Stack Exchange howto: https://security.stackexchange.com/questions/35639/decrypting-tls-in-wireshark-when-using-dhe-rsa-ciphersuites/42350#42350
Wireshark decryption docs: https://wiki.wireshark.org/SSL

(reposted from nodejs/node-convergence-archive#59).

[silverwind]

As this would basically throw out all security, the only way I see for a feature like that would be a compile-time option.

[jsha]

I think a compile-time option would satisfy the use case for this reasonably well, though it would be less convenient than the environment variable used by Firefox and Chrome.

I don't understand why you say this would throw out all security, though. The contents of the TLS connection are known to both endpoints, including the Node endpoint on which you would add this debugging variable when needed. This just allows the developer who runs the Node endpoint to store extra data about that connection. The idea is not that you would leave this on permanently in any production system.

[silverwind]

I'm thinking more of a scenario where a compromised server could be modified to provide these secrets to the attacker, while still serving the application. An attacker could for example modify the environment variables and force a restart on the node process.

I'm not really sure the benefits outweigth the risks here. Couldn't you just run a MITM proxy for your debugging needs?

[jsha]

compromised server could be modified to provide these secrets to the attacker, while still serving the application

This is entirely possible today. Anyone with access to the Node process' memory can exfiltrate key material.

I'm not really sure the benefits outweigth the risks here. Couldn't you just run a MITM proxy for your debugging needs?

For debugging of HTTP applications this is generally sufficient. But for debugging HTTP protocol issues, or especially HTTP/2 protocol issues, a proxy would not show the necessary information. The need for this flag came up for me specifically when trying to debug issues using HTTP/2 as a client against a remote server. The client knows the session key by definition, so allowing it to be (optionally) logged merely enables better debugging.

[silverwind]

Well, maybe a runtime flag could be done. I'm a bit more comfortable with a flag than a mere environment variable ;)

[jsha]

Yep, a runtime flag would be great too! I only suggested the env variable for similarity with Firefox and Chrome.

[jorangreef]

Sorry, but this is a terrible idea. It's a massive accident just waiting to happen. It opens so many possible attack vectors. If anyone needs to do this kind of thing they should just modify Node themselves.

[jsha]

@jorangreef: Can you go into more detail about the attack vectors or accidents you forsee? Keep in mind that this has been an option in Firefox and Chrome for many years and I have not heard a single example of anyone being compromised by it.

If I may guess a little: It sounds like you and @silverwind are most concerned by server operators misusing this flag. I think the most important debugging is is in TLS client code: if you control the server, you can force negotiation with a non-forward secret cipher, and since you also possess the private key, that allows the necessary analysis. However, for the client, there is no such option: the server controls the cipher negotiation and the private key.

Would you be less worried about accidental misuse of a flag that affects TLS clients only, and not servers?

[jorangreef]

@jsha there's probably an endless combination of attack vectors that an option to log the master key would support. But even if there's just one vector, that's enough, and @silverwind has already given a few.

Chrome and Firefox are desktop applications so the security threats are different.

I think your use-case would be solved if your Javascript could access the key from the actual TLS client instance directly, i.e. it's not necessary to ask Node to log it (via env flag or compile flag). But even then, it should only be exposed as a property by the instance if an option is explicitly provided.

Something like this should err on the side of being very difficult to do.

[lxe]

+1 This feature would immensely help with debugging.

I'm not really sure the benefits outweigth the risks here. Couldn't you just run a MITM proxy for your debugging needs?

Not at all. There are heisenbugs that require delicate and specific environment, and altering network flow with MITM proxies makes debugging impossible.

[larryboymi]

+1 ...
Realize it seems like more could happen server-side with exposure of client secrets, but this would immensely help debugging encrypted traffic.

If an attacker is able to get to the running process and manipulate it, the possibilities range widely regardless.

Java exposes via logging key parts which can be consumed by wireshark with a little manipulation. I know node is not java.

[tgvarik]

Wireshark >=1.6.0 supports NSS-format log files giving the session ID and master key in the following format:

RSA Session-ID:<32-byte session ID, hex encoded> Master-Key:<48-byte master key, hex encoded>

Both the session ID and master key can be obtained with TLSSocket.prototype.getSession(), which returns a DER-encoded ASN.1 structure described in ssl.h.

So just do something like this:

https.request(opts, cb)
.once('socket', (s) => {
  s.once('secureConnect', () =>
    let session = parseSession(s.getSession());
    // session.sessionId and session.masterKey should be hex strings
    fs.appendFileSync('sslkeylog.log', `RSA Session-ID:${session.sessionId} Master-Key:${session.masterKey}\n`);
  });
});

This works very well for me in Wireshark 2.2.4. How thoroughly to parse the session buffer is up to you, but even something as barebones as this will work in most cases:

function parseSession(buf) {
  return {
    sessionId: buf.slice(17, 17+32).toString('hex'),
    masterKey: buf.slice(51, 51+48).toString('hex')
  };
}

[jhford]

@tgvarik I found your comment really useful and wrote a simple library to implement that. I was wondering if you'd be OK with me publishing it to npm. If so, how would you like attribution?

https://github.com/jhford/node-https-wireshark

I'm also happy to delete the code :)

I changed it so that it should play nicely with older versions of node.

[tgvarik]

@jhford Thanks for writing that up. Please feel free to publish! No attribution needed, but a link to my comment above would be appreciated.

[joshperry]

We've been working to debug a binary, non-HTTP, TLS-secured protocol implementation with a custom wireshark dissector; we would find this feature extremely useful.

Even simple properties that gave these values without opaque buffer parsing would be very useful. I'm not sure what the negative security implications would be with this key material already available in-process from the session with no flags.

Thanks for the code, @tgvarik! Going to give it a shot here.

[silverwind]

I've been looking into extracting the keys from a server socket, and found that parsing the SSL_SESSION structure is insufficient with modern browser clients at least. For these type of connections, Wireshark supports the CLIENT_RANDOM <client_random> <master-secret> format (full details on supported formats).

SSL_SESSION only contains the master secret in these cases, so I assume SSL_get_client_random would need to be exposed as well to enable decryption of these type of sessions.

At this point, I think out-of-band decryption is something we should enable userland to do. It won't have to be easy or convenient, but we should provide access to the necessary data structures to make it possible.

cc: @nodejs/crypto

[mhdawson]

@sam-github

[Trott]

Is this being worked on? Should this stay open at this point?

[bnoordhuis]

I'll close it out. It's almost its 2nd anniversary and no one has put up a PR so far. If someone still wants to pursue this, open a PR and we'll take it from there.

[kolontsov]

Recently I published https://github.com/kolontsov/node-sslkeylog to implement SSLKEYLOG. It uses little hacky approach to get access to SSL_get_client_random(), but so far it works well for my debugging purporses.

I was thinking about submitting PR (to use SSL_get_client_random() without tricks), but I'm not yet sure how implementation should look like: we can write sslkeylog.txt if some env var exists, we can expose get_session_key() to be used with TLSSocket, etc..

OR we can wait until openssl 1.1.1 will be fully integrated and then just start using SSL_CTX_set_keylog_callback..

[mildsunrise]

@kolontsov A million thanks for your module, really easy to use.
For anyone wanting to debug client connections, the best way is to patch the used Agent:

function patchAgent(agent) {
  const { update_log } = require("sslkeylog");
  const patchSocket = socket => socket.on("secureConnect", () => update_log(socket));
  const patchIfSocket = socket => socket ? patchSocket(socket) : socket;
  const original = agent.createConnection.bind(agent);
  agent.createConnection = (options, callback) =>
    patchIfSocket(original(options, (err, socket) => callback(err, patchIfSocket(socket))));
}

To sniff all HTTPS connections, just patch the global agent when your program starts:

sslkeylog.set_log("/tmp/keys.log");
patchAgent(require("https").globalAgent);

[kolontsov]

@jmendeth thank you! I'm glad you found it useful. Also thanks for the code, I'll include it into example and/or readme.

[mildsunrise]

Good news! Node.JS 12.3.0 now has a native API for TLS secret logging.

Logging server connections

Subscribe to the keylog event and append the lines to a file:

const myServer = https.createServer(...);
// ...
myServer.on('keylog', line =>
    fs.appendFileSync('/tmp/secrets.log', line));

Then, as usual on Wireshark open Preferences → Protocols → SSL/TLS set «Pre-master-secret log filename» to /tmp/secrets.log. That's it!

This works on any instance of tls.Server, including https servers (as above) and http2 servers.

Logging client connections

The keylog event also works on TLSSockets, so you can use it with tls.connect():

const socket = tls.connect(...);
socket.on('keylog', line =>
    fs.appendFileSync('/tmp/secrets.log', line));

Or with http2.connect():

const session = http2.connect(...);
session.socket.on('keylog', line =>
    fs.appendFileSync('/tmp/secrets.log', line));

Logging HTTPS requests

This is tricky, because an existing connection may be reused for a request.
You can set agent: false to ensure this doesn't happen, and subscribe to the socket event:

const req = https.request("https://example.org", { agent: false });
req.on('socket', socket => {
  socket.on('keylog', line =>
      fs.appendFileSync('/tmp/secrets.log', line));
});

Tips

• You need to subscribe to keylog when the socket is created, not when it connects, because secrets might be emitted before that.
• Wireshark needs to capture the handshake in order to decrypt a stream. If the connection is already established when you start capturing, it won't work.
• Logging secrets from either client or server is enough--you don't need to log on both sides.
• It works on TLSv1.3 connections, it works with all key agreement schemes and it should support session resumption (i.e. 0-RTT) as well (can anybody confirm?)

[mildsunrise]

Thanks to #30055 there's now a --tls-keylog option that logs all TLS traffic (except of course, native addons that don't use the tls module). This is available starting at v12.15 and v13.2.

There's also the sslkeylog module mentioned above, which works with v10.0+. Do this at the start of your code:

require('sslkeylog').hookAll();

And again, all traffic will be logged if the SSLKEYLOGFILE environment variable is set.
But hookAll() works by patching Node.JS internals, so I recommend --tls-keylog if possible.

[luiso1979]

@mildsunrise It is a very good news. But I'm having a problem.
I have an express server and sometimes I also make some calls using request.
express: 4.13.3
request: 2.88.0

this is the server:

        this.server.on("keylog", (line: string) => {
            try {
                keylogStream.write(line);
            } catch (error) {
                LOGGER.error(error);
            }
        });

and this fragment is attached to the request call:

        .on("socket", (s) => {
                s.on("keylog", (line) => {
                    if (this.config.keylog_stream) {
                        this.config.keylog_stream.write(line);
                    }
                });
            });

doing this I get almost always all the keys I need to decrypt the traffic, but there are sometimes when some keys get lost (related to the calls arriving at server)

Hence I decided to use the --tls-keylog. This way it would dump everything, when I start the server and the calls (https) arrive the keylog is not even created, but I do get the log warning:

(node:37699) Warning: Using --tls-keylog makes TLS connections insecure by writing secret key material to file sslkeyfile.log 

Can anybody tell me what's going on or what I'm doing wrong?

Thanks in advance

[mildsunrise]

but there are sometimes when some keys get lost (related to the calls arriving at server)

This could happen if the agent is reusing sockets, see "Logging HTTPS requests" in my comment above.

Hence I decided to use the --tls-keylog. This way it would dump everything, when I start the server and the calls (https) arrive the keylog is not even created

That shouldn't be happening at all, could you open an issue with a small snippet to reproduce?

[mildsunrise]

@luiso1979 I found the bug you're hitting, opened #33366 for a fix

[luiso1979]

@mildsunrise thank you very much for the support!!!

[luiso1979]

@mildsunrise I tried to follow your advice regarding the agent reusing sockets, but my problem remains, since the lost key is not from a request I'm making but from a request that arrives to my server.

        this.server.on("keylog", (line: string) => {
            try {
                keylogStream.write(line);
            } catch (error) {
                LOGGER.error(error);
            }
        });

This is how I'm listening for keylogs, but sometimes certain arriving POSTs are not logged. It happens rarely and I find it difficult to reproduce it.

Do you have any insight that might help?

Thanks

[mildsunrise]

Without much more info, I'm afraid I can't tell... Make sure you're registering the handler before listen, and inspect your logfile to make sure it doesn't look corrupted.
(also, offtopic, line is actually a Buffer)

[mattfysh]

Has anyone managed to get this working with openssl v1.1.1? I'm using a shared library that makes use of the SSL_CTX_set_keylog_callback symbol, and have verified that it is not working for nodejs

libsslkeylog.so is built from https://git.lekensteyn.nl/peter/wireshark-notes/tree/src/sslkeylog.c

process.versions.openssl is '1.1.1n'

# this works
node --tls-keylog=node_opt.txt -e 'require("https").get("https://example.com", req => {})'

# this also works
SSLKEYLOGFILE=curl.txt LD_PRELOAD=libsslkeylog.so curl https://example.com

# this does _not_ work :*(
SSLKEYLOGFILE=node.txt LD_PRELOAD=libsslkeylog.so node -e 'require("https").get("https://example.com", req => {})'

[mildsunrise]

libsslkeylog.so is a 'hacky' way to add keylog support to things that don't have it natively.
Node.js has that native support through the --tls-keylog option as you said, so you should prefer this option to the libsslkeylog.so approach.

(If your reason to use libsslkeylog.so instead of --tls-keylog is because you need to set it through environment variables, read how to set Node.js options via environment)

[mattfysh]

Thanks @mildsunrise, I read this is caused by curl dynamically linking libssl whereas node statically links it so the symbols can't be patched that way.

I was trying to build a solution that works with arbitrary programs written in any language, but I can customize per runtime so I will use the --tls-keylog option for node as you pointed out. Thanks again :)