tls: Add PSK support (v9.0.0-pre) #14978

taylorzane · 2017-08-22T15:39:33Z

Checklist

make -j4 test (UNIX), or vcbuild test (Windows) passes
tests and/or benchmarks are included
documentation is changed or added
commit message follows commit guidelines

Affected core subsystem(s)

tls, crypto

This is an updated PR based off of #6701 that supports Node.js v9.0.0.

I understand that the Node.js team is waiting for FIPS support in OpenSSL-1.1, but this will get the ball rolling for TLS-PSK support in the latest Node.js versions.

jasnell · 2017-08-22T16:08:02Z

lib/_tls_common.js

+        if (typeof ret.psk === 'string') {
+          ret.psk = Buffer.from(ret.psk);
+        } else if (!Buffer.isBuffer(ret.psk)) {
+          throw new TypeError('Pre-shared key is not a string or buffer');


I'm not sure if the rest of this file has been converted to use the internal/errors mechanism, but new errors introduced definitely should.

jasnell · 2017-08-22T16:08:17Z

lib/_tls_common.js

+          throw new TypeError('Pre-shared key is not a string or buffer');
+        }
+        if (ret.psk.length > maxPskLen) {
+          throw new TypeError(`Pre-shared key exceed ${maxPskLen} bytes`);


This should likely be a RangeError

jasnell · 2017-08-22T16:08:33Z

lib/_tls_common.js

+            throw new TypeError('PSK identity is not a string');
+          }
+          if (ret.identity.length > maxIdentLen) {
+            throw new TypeError(`PSK identity exceeds ${maxIdentLen} bytes`);


RangeError

jasnell · 2017-08-22T16:10:28Z

src/node_crypto.cc

+void SecureContext::SetPskIdentity(const FunctionCallbackInfo<Value>& args) {
+  SecureContext* wrap = Unwrap<SecureContext>(args.Holder());
+
+  if (args.Length() != 1 || !args[0]->IsString()) {


Not convinced that this should error if more than 1 argument is provided. Perhaps just if (!args[0]->IsString()) {

jasnell · 2017-08-22T16:11:07Z

src/node_crypto.cc

+    return wrap->env()->ThrowTypeError("Argument must be a string");
+  }
+
+  String::Utf8Value identity(args[0]);


Please make sure the documentation indicates that the PSK should be UTF8 only.

jasnell · 2017-08-22T16:14:21Z

test/parallel/test-tls-psk-client.js

+  const s = tls.connect(common.PORT, {
+    ciphers: CIPHERS,
+    rejectUnauthorized: false,
+    pskCallback: (hint) => {


nit ... pskCallback(hint) {

taylorzane · 2017-08-22T16:26:32Z

I'll get these changes implemented right away, thanks @jasnell.

mscdex · 2017-08-22T17:03:31Z

/cc @nodejs/crypto

taylorzane · 2017-08-22T18:39:28Z

@jasnell I updated my changes as suggested. I have a few questions though, I'll add comments to my commit.

taylorzane · 2017-08-22T18:40:48Z

lib/_tls_common.js

+                                     ['string', 'buffer']);
+        }
+        if (ret.psk.length > maxPskLen) {
+          throw new errors.RangeError(


Is this the proper error to throw? There wasn't an existing error that satisfied the previous generic RangeError, so I just used ERR_ASSERTION.

Also, how is the formatting on these lines? It complies with eslint, but it's ugly as sin.

addaleax

Great work! Some of the more code here uses deprecated APIs, it would be nice if we could avoid that. Let me know if you need any help with it!

addaleax · 2017-08-22T16:18:49Z

doc/api/tls.md

+    <string>}`. Note that PSK ciphers are disabled by default, and using
+    TLS-PSK thus requires explicitly specifying a cipher suite with the
+    `ciphers` option. Additionally, it may be necessary to disable
+    `rejectUnauthorized` when not intending to use certificates.


“it may be necessary” sounds a bit vague … can you clarify when that is necessary?

addaleax · 2017-08-22T18:46:44Z

lib/_tls_common.js

+
+        if (typeof ret.psk === 'string') {
+          ret.psk = Buffer.from(ret.psk);
+        } else if (!Buffer.isBuffer(ret.psk)) {


Can you use ArrayBuffer.isVew(ret.psk) instead and adjust the thrown error accordingly? Like in
https://github.com/nodejs/node/pull/14807/files#diff-1c8a7b9222892c3457fa64cbbfab6573R56

addaleax · 2017-08-22T18:48:09Z

lib/_tls_common.js

+        // Only set for the client callback, which must provide an identity.
+        if (maxIdentLen) {
+          if (typeof ret.identity === 'string') {
+            ret.identity = Buffer.from(ret.identity + '\0');


I feel like it would be easier to pass a string to C++ instead? Am I missing something?

Well, the C++ method is using the identity in its buffer form, so we could pass a string and then convert it to a buffer in C++. Since Buffer::Data returns a char*, that's directly compatible with identity. I suppose we could convert the string directly to a char* as well. Seems like it would be about the same amount of work on either side.

One difference that this is always creating new intermediate JS objects in the process, which is part of why we don’t usually do that (I guess)

I agree, if we can avoid creating unnecessary (Buffer) objects and such in between then that would be better.

Sounds good, I'll get on this refactor in the morning.

addaleax · 2017-08-22T18:50:53Z

src/node_crypto.cc

+  Isolate* isolate = env->isolate();
+
+  Local<Value> argv[] = {
+    String::NewFromUtf8(isolate, identity),


Can you use the overload that uses v8::NewStringType/returns a MaybeLocal? This one is being deprecated by V8.

addaleax · 2017-08-22T18:52:50Z

src/node_crypto.cc

+                                        sc->object(),
+                                        env->onpskexchange_string(),
+                                        arraysize(argv),
+                                        argv);


I think this is only called synchronously, right (i.e. there is a JS stack somewhere below it)? In that case, it’s better to just use Function::Call instead of MakeCallback

addaleax · 2017-08-22T18:53:27Z

src/node_crypto.cc

+  Local<Object> obj = ret.As<Object>();
+
+  Local<Value> psk_buf = obj->Get(env->psk_string());
+  assert(Buffer::HasInstance(psk_buf));


nit: Can you use CHECK instead of assert? Also, this is equivalent to psk_buf->IsArrayBufferView(), just btw.

addaleax · 2017-08-22T18:53:40Z

src/node_crypto.cc

+  Local<Value> psk_buf = obj->Get(env->psk_string());
+  assert(Buffer::HasInstance(psk_buf));
+  size_t psk_len = Buffer::Length(psk_buf);
+  assert(psk_len <= max_psk_len);


nit: CHECK_LE?

addaleax · 2017-08-22T18:54:43Z

src/node_crypto.cc

+  }
+  Local<Object> obj = ret.As<Object>();
+
+  Local<Value> psk_buf = obj->Get(env->psk_string());


This variant of Get is being deprecated as well, the replacement API basically looks like obj->Get(env->context(), env->psk_string()).ToLocalChecked() in this case

addaleax · 2017-08-22T18:59:36Z

test/parallel/test-tls-psk-circuit.js

+}
+
+process.on('exit', () => {
+  assert.strictEqual(serverConnections, 3);


We have a test utility that’s called common.mustCall() and allows you to have a bit more code locality in tests; e.g. you could wrap the callback that increases serverConnections in common.mustCall() and wouldn’t need to have an explicit counter variable :)

addaleax · 2017-08-22T19:01:18Z

test/parallel/test-tls-psk-circuit.js

+if (!common.hasCrypto) {
+  console.log('1..0 # Skipped: missing crypto');
+  return;
+}


Just fyi, in most tests this looks like this by now:

node/test/parallel/test-benchmark-crypto.js

Lines 5 to 6 in d348512

if (!common.hasCrypto)

common.skip('missing crypto');

addaleax · 2017-08-22T19:05:42Z

CI: https://ci.nodejs.org/job/node-test-commit/11964/

taylorzane · 2017-08-22T19:26:02Z

Thanks for the follow-up, I'll get these changes implemented.

…g of pskCallback note.

taylorzane · 2017-08-22T21:19:00Z

I believe that all issues brought up by @addaleax have been resolved. Some code snippets were updated in both PskServerCallback and PskClientCallback even though they weren't explicitly mentioned. (Missed a couple asserts, fixed after this comment was made.)

addaleax

LGTM if @nodejs/crypto is happy

addaleax · 2017-08-22T21:26:51Z

src/node_crypto.cc

+  Local<Value> argv[] = {
+    String::NewFromUtf8(isolate,
+                        identity,
+                        String::kNormalString,


v8::NewStringType::kNormal (the overload that uses a Maybe – sorry, should have been clearer about that)

addaleax · 2017-08-22T21:27:08Z

src/node_crypto.cc

+  if (hint != nullptr) {
+    argv[0] = String::NewFromUtf8(isolate,
+                                  hint,
+                                  String::kNormalString,


addaleax · 2017-08-22T21:27:35Z

src/node_crypto.cc

+                                  String::kNormalString,
+                                  strlen(hint));
+  }
+  Local<Value> value = sc->object()->Get(env->onpskexchange_string());


This is the deprecated Get() overload (no context argument)

addaleax · 2017-08-22T21:28:10Z

src/node_crypto.cc

+  size_t identity_len = Buffer::Length(identity_buf);
+  CHECK_LE(identity_len, max_identity_len);
+  memcpy(identity, Buffer::Data(identity_buf), identity_len);
+  assert(identity[identity_len - 1] == '\0');


This is also a reason why using passing strings to C++ might be nice :)

taylorzane · 2017-08-22T21:51:15Z

Didn't realize I was using the wrong methods, I must have been looking at an old version of the V8 documentation. My MaybeLocal => Local logic seems right to me based on my experience with other languages with optional types.

taylorzane · 2017-08-23T18:26:52Z

Alright, TLS-PSK API now only accepts a hex-encoded string for psk (akin to the openssl s_server command); psk and identity are now passed as strings to C++ and converted to char arrays. I also updated a nit on a deprecated ->Get call.

jasnell · 2017-08-24T20:58:07Z

lib/_tls_common.js

@@ -132,6 +133,49 @@ exports.createSecureContext = function createSecureContext(options, context) {
    }
  }

+  if (options.pskCallback && c.context.enablePskCallback) {
+    c.context.onpskexchange = (identity, maxPskLen, maxIdentLen) => {


should there be a check here to ensure that pskCallback is a function?

jasnell · 2017-08-24T20:58:36Z

lib/_tls_common.js

+  if (options.pskCallback && c.context.enablePskCallback) {
+    c.context.onpskexchange = (identity, maxPskLen, maxIdentLen) => {
+      let ret = options.pskCallback(identity);
+      if (typeof ret !== 'object') {


what if ret === null? (I know the check before looks for that, but we should be clear here... just to be safe :-) ...)

jasnell · 2017-08-24T21:04:39Z

lib/_tls_common.js

+
+      return ret;
+    };
+    c.context.enablePskCallback();


If I'm reading the #ifdef statements write in the source below, there's a chance the enablePskCallback() will not be defined if OPENSSL_PSK_SUPPORT is switched off, correct? I may be missing something because I haven't done a complete walkthrough, but if that's the case, I'd recommend gating these with a flag in node_config.cc

There shouldn't be a chance that enablePskCallback is enabled, since the definition of that is also inside of an #ifdef, see here

Ok, I see, I was misreading the location of the closing brackets. This call is made inside the if that checks c.context.enablePskCallback is not falsy. :-)

jasnell · 2017-08-24T21:09:33Z

src/node_crypto.cc

+
+  MaybeLocal<Value> maybe_ret;
+  if (value->IsFunction()) {
+    Local<Function> func = Local<Function>::Cast(value);


value.As<Function>();?

jasnell · 2017-08-24T21:10:30Z

src/node_crypto.cc

+  HandleScope scope(isolate);
+
+  Local<Value> argv[] = {
+    Null(isolate),


May be helpful to add a comment here reminding folks that this is a error-back style callback, which explains why the first argument is passed as a Null

So that's actually the object passed to the pskCallback function, where the first argument (initially defined as Null) is the hint provided by OpenSSL/TLS. The hint is an optional field, which is why we're not erroring when it's not provided.

jasnell · 2017-08-24T21:12:37Z

test/parallel/test-tls-psk-circuit.js

+
+function startTest() {
+  function connectClient(options, next) {
+    const client = tls.connect(common.PORT, 'localhost', options, () => {


s/common.PORT/0

jasnell · 2017-08-24T21:13:00Z

test/parallel/test-tls-psk-circuit.js

+
+      client.end(TEST_DATA);
+
+      client.on('data', (data) => {


perhaps wrap this in a common.mustCallAtLeast()

jasnell · 2017-08-24T21:13:15Z

test/parallel/test-tls-psk-circuit.js

+        assert.strictEqual(data.toString(), TEST_DATA);
+      });
+
+      client.on('close', next);


common.mustCall(next)

jasnell · 2017-08-24T21:14:39Z

test/parallel/test-tls-psk-server.js

+let gotWorld = false;
+let opensslExitCode = -1;
+
+server.listen(common.PORT, () => {


s/common.PORT/0

note that in general we've been moving away from using common.PORT in the parallel tests.

…g of pskCallback note.

… included.

…eaned up some formatting.

…hange` to use `Function::Call`. Updated PSK callbacks to use new/non-deprecated functions.

…f8`. Updated PSK callbacks to use `MaybeLocal` overload of Function::Call.

…ty` arguments.

…ded some `mustCall` wrappers in TLS-PSK circuit test.

…d TLS-PSK callback to ensure that the returned value is not null.

… MaybeLocal to a Function.

…e the new error throwing format.

…context.

…tions.

Add the `pskCallback` client/server option, which resolves an identity or identity (hint) to a pre-shared key. The function signature on client and server is compatible. Add the `pskIdentity` server option to set the identity hint for the ServerKeyExchange message. Co-authored-by: Chris Osborn <[email protected]> Co-authored-by: stephank <[email protected]> Co-authored-by: Denys Otrishko <[email protected]>

taylorzane · 2018-10-01T05:04:18Z

Alright, I believe I've rebased correctly (again), and cherry-picked @lundibundi's commit. Though, I'm not seeing my changes appear in the "Commits" or "Files changed" tabs. Is there a way to resolve that issue?

lundibundi · 2018-10-01T09:02:48Z

@taylorzane you just had to reopen the PR. =)
Though it seems you didn't remove all of the previous commits (I squashed them in one there were becoming too much of them, therefore all of them are present in my commit, unless you were making changes and havent pushed them to github) and have a few commits from master here, did you perhaps use git merge and not git rebase?

taylorzane · 2018-10-01T14:48:18Z

Ah, I see what you did. I didn't actually remove the previous commits, I missed that part of your comment. Let me see what I can do about this this morning.

lundibundi · 2018-10-11T19:38:56Z

@taylorzane ping, will you have time for this?
Also, could you look through my note at the end here #23188 (comment) if you haven't seen it yet?

Trott · 2018-11-13T20:40:11Z

@jasnell @addaleax Do either of you want to rebase this and push it back up? Or should we close this PR?

lundibundi · 2018-11-14T08:57:44Z

@Trott I continued working on this in #23188.
I'll close this for now in favor of that PR.

nodejs-github-bot added c++ Issues and PRs that require attention from people who are familiar with C++. lib / src Issues and PRs related to general changes in the lib or src directory. labels Aug 22, 2017

jasnell reviewed Aug 22, 2017

View reviewed changes

taylorzane commented Aug 22, 2017

View reviewed changes

addaleax reviewed Aug 22, 2017

View reviewed changes

taylorzane added a commit to taylorzane/node that referenced this pull request Aug 22, 2017

tls: Added nodejs#14978 PR to the changes: section. Improved wordin…

f896c7c

…g of pskCallback note.

addaleax approved these changes Aug 22, 2017

View reviewed changes

addaleax mentioned this pull request Aug 23, 2017

tls: multiple PFX support in createSecureContext #14793

Closed

4 tasks

jasnell reviewed Aug 24, 2017

View reviewed changes

taylorzane and others added 22 commits September 30, 2018 23:53

tls: Updated formatting to conform with eslint rules.

45c120b

tls: Added nodejs#14978 PR to the changes: section. Improved wordin…

eef89b8

…g of pskCallback note.

tls: Updated tests to use the common.skip method when crypto is not…

6f095f8

… included.

tls: Updated onpskexchange callback to use ArrayBuffer.isView. Cl…

ba22b97

…eaned up some formatting.

tls: Added HandleScope to PSK callbacks. Updated calls to `onpskexc…

ba0727e

…hange` to use `Function::Call`. Updated PSK callbacks to use new/non-deprecated functions.

tls: Updated a few assert calls to use the proper CHECK call.

361b5ed

tls: Updated PSK callbacks to use MaybeLocal overload of `NewFromUt…

49a285f

…f8`. Updated PSK callbacks to use `MaybeLocal` overload of Function::Call.

tls: Updated docs and tests to reflect changes to the TLS-PSK API.

16921ab

tls: Updated TLS-PSK API to only accept strings for psk and `identi…

959887e

…ty` arguments.

tls: Updated TLS-PSK tests to not use common.PORT when possible. Ad…

65cb351

…ded some `mustCall` wrappers in TLS-PSK circuit test.

tls: Updated TLS to check that TLS-PSK callback is a function. Update…

7b43283

…d TLS-PSK callback to ensure that the returned value is not null.

tls: Updated PSKClientCallback to use a better method for casting a…

bbbb658

… MaybeLocal to a Function.

tls: Fixed linter error in _tls_common.js.

5136c93

tls: Updated PSKServerCallback to use a better method for casting a…

6ed9f92

… MaybeLocal to a Function.

tls: Renamed pskIdentity to pskIdentityHint. Updated errors to us…

97a71bd

…e the new error throwing format.

tls: Updated PSK callbacks to use the new syntax for getting the SSL …

b810f97

…context.

tls: Updated PSK tests to use new error messages, improved test asser…

92fc64e

…tions.

tls: Resolved lint errors.

4880511

tls: Updated PSK code to latest best practices.

40fae68

tls: Removed an unused env string.

001b3bd

tls: Updated PSK functions to use automatic memory management.

34e54c9

lundibundi reopened this Oct 1, 2018

lundibundi closed this Nov 14, 2018

tls: Add PSK support (v9.0.0-pre) #14978

tls: Add PSK support (v9.0.0-pre) #14978

Conversation

taylorzane commented Aug 22, 2017

Checklist

Affected core subsystem(s)

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

taylorzane commented Aug 22, 2017

mscdex commented Aug 22, 2017

taylorzane commented Aug 22, 2017

Choose a reason for hiding this comment

Choose a reason for hiding this comment

addaleax left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

mscdex Aug 23, 2017 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

addaleax commented Aug 22, 2017

taylorzane commented Aug 22, 2017

taylorzane commented Aug 22, 2017 • edited Loading

addaleax left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

taylorzane commented Aug 22, 2017

taylorzane commented Aug 23, 2017

Choose a reason for hiding this comment

jasnell Aug 24, 2017 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

taylorzane Aug 25, 2017 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

taylorzane commented Oct 1, 2018

lundibundi commented Oct 1, 2018 • edited Loading

taylorzane commented Oct 1, 2018

lundibundi commented Oct 11, 2018

Trott commented Nov 13, 2018

lundibundi commented Nov 14, 2018

mscdex Aug 23, 2017 •

edited

Loading

taylorzane commented Aug 22, 2017 •

edited

Loading

jasnell Aug 24, 2017 •

edited

Loading

taylorzane Aug 25, 2017 •

edited

Loading

lundibundi commented Oct 1, 2018 •

edited

Loading