SIGTERM not properly forwarded to node process

[buffpojken]

It seems like foreman is not properly forwarding signals to a running node-process.

Processes are started by the following Procfile:

rabbitmq: rabbitmq-server start
web: bundle exec rails s
node: node node/server.js
redis: redis-server /usr/local/etc/redis.conf

and starts as expected.

With the the Node-app, the following signal-trap is installed:

process.on('SIGTERM', function() {
  console.log('Received Signal');
    process.exit(1);
});

and is invoked as expected when running Node straight from the commandline (when bound on SIGINT). However, when running via foreman - no signal of any kind is propagated.

I've tried to read through the code for a hint but as far as I can see - the signals should be forwarded?

Testing was done on OS X 10.9.2

[ddollar]

What version of foreman are you using? Some significant changes were just released in the last couple of days.

[buffpojken]

Hmm, foreman was installed using a regular "gem install foreman"
and

foreman -v

returns 0.63.0.

I'll try forcing an update and will update this issue accordingly.

[buffpojken]

Okay, updated foreman to 0.66 - issue still remains exactly as described above.

[ddollar]

I did some testing and it looks like SIGINT does make it down to the node process but that no logging done inside that handler will show up.

What are you trying to do in the signal handler?

[ddollar]

It seems like the signals are being forwarded but that Node is not able to display output if you process.exit() right afterwards.

Try this script: https://gist.github.com/ddollar/cf9d3e01e2ba01750f79

[Offirmo]

Update : @ddollar has the correct analysis (see node issue nodejs/node-v0.x-archive#8329) but the link above is no longer usable (node 10.31 provides no flush on streams)

However, I found this module which does the job : https://www.npmjs.org/package/exit [update : or not :-/]

[Offirmo]

I wonder about this problem. The problem still holds. Here is my foreman output :

 foreman start
18:19:44 web.1  | started with pid 14115
18:19:44 web.1  | [node#14115] Hello from node.js : my pid is 14115
18:19:44 web.1  | [node#14115] Hello from node.js : is tty ? true
18:19:44 web.1  | [node#14115] Now listening on port 5000
18:19:44 web.1  | [node#14115] (Ctrl+C to stop)
^CSIGINT received
18:19:45 system | sending SIGTERM to all processes
18:19:47 web.1  | exited with code 102

My node.js is not able to output anything. However, when I send SIGINT/SIGTERM directly to the node process (kill ) I correctly get the node output :

foreman start
18:21:52 web.1  | started with pid 14146
18:21:53 web.1  | [node#14146] Hello from node.js : my pid is 14146
18:21:53 web.1  | [node#14146] Hello from node.js : is tty ? true
18:21:53 web.1  | [node#14146] Now listening on port 5000
18:21:53 web.1  | [node#14146] (Ctrl+C to stop)
18:21:57 web.1  | [node#14146] ! Caught signal SIGTERM "Termination signal"
18:21:57 web.1  | [node#14146] ! Exiting following signal SIGTERM with "Term" disposition...
18:21:57 web.1  | [node#14146] ! Exiting with 115
18:21:59 web.1  | exited with code 115
18:21:59 system | sending SIGTERM to all processes

How can we explain this ?

[Offirmo]

Since my tests involve Ctrl+C could it be related to #330 ?

[causztic]

still an issue even on the latest version.

when a process fails and SIGTERM is sent to the other node processes:
11:51:28 system | sending SIGTERM to all processes is shown but the other node processes did not receive the SIGTERM.

I manually patched this by:

• making foreman run a shell script instead
• use trap 'lsof -ti :8080 | xargs kill -15' EXIT to manually pass a SIGTERM signal to the corresponding process

[domenkozar]

I observed the same behavior.

Looking at the code

foreman/lib/foreman/engine.rb

Line 192 in 1445adc

system "sending #{signal} to #{name_for(pid)} at pid #{pid}"

should print that the signal is being sent but I haven't seen it.

EDIT: I was using outdated 0.78 verison