Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Hash initial server selection without save last server #18

Merged
merged 1 commit into from
Oct 3, 2015

Conversation

mvrogowski
Copy link

We use pen to load balancing RADIUS server that handles standard and PEAP requests.

The standard RADIUS request is a simple transaction that works fine with any load balancing algorithm. But the PEAP transaction is more complex: it needs the TLS tunnel to encrypt the traffic. To perform a PEAP transaction, it need to do more than one request to server. To get success, we need to ensure that all transaction packets will sent to same server. The only way to do it, is using the hash algorithm.

When hash algorithm is enabled, it redirects all request to same server, making one server hammered while others are on idle state.

To fix our problem, I added a new initial server selection algorithm called "Hash no server" that adds the source port to hash algorithm and don't save the last server the client used.

Could you analyze my solution? If it is a good one, I will be thankful if you add it to your repository. :)

I hope I have contributed to your project.

Thank's for your attention!

@UlricE
Copy link
Owner

UlricE commented Sep 29, 2015

Will have a look as soon as I get access to a computer.

UlricE added a commit that referenced this pull request Oct 3, 2015
Hash initial server selection without save last server
@UlricE UlricE merged commit 5e33606 into UlricE:master Oct 3, 2015
@mvrogowski
Copy link
Author

Could you add a new tar.gz release to http://siag.nu/pub/pen/ with this mods, please?

@UlricE
Copy link
Owner

UlricE commented Oct 6, 2015

Figuring out how to add this without breaking backward compatibility.

@UlricE
Copy link
Owner

UlricE commented Oct 6, 2015

About pull request #18:

One of the points of the hash algorithm is that it is deterministic:

  1. In a load-balanced pair of load balancers, the server selection is done the same way regardless of where the client ends up.
  2. A client will always be sent to the same server if it connects from the same IP address.

PR 18 maintains property 1 but not 2, so breaks the old hash algorithm.

The DSR code has its own hash algorithm which includes the port when the -r (roundrobin) option is used. It would be a good idea to use the same semantics for the non-DSR case.

Using -r will forego reusing the old server (see initial_server).

With these changes (see below), -hr will do the same thing as -N while maintaining backwards compatibility.

Have a look and if it makes sense I'll put this in 0.31.0.

diff --git a/client.c b/client.c
index cbdc327..79bd25a 100644
--- a/client.c
+++ b/client.c
@@ -84,10 +84,6 @@ int store_client(struct sockaddr_storage *cli)
        clients[i].addr = *cli;
        clients[i].connects++;

-       /* don't remember server */
-       if(server_alg & ALG_HASH_NO_SERVER) {
-               clients[i].server = NO_SERVER;
-       }

        DEBUG(2, "Client %s has index %d", pen_ntoa(cli), i);

diff --git a/pen.c b/pen.c
index 7f66433..eb346e7 100644
--- a/pen.c
+++ b/pen.c
@@ -2415,9 +2415,6 @@ static int options(int argc, char **argv)
                        }
                        break;
 #endif  /* HAVE_LIBSSL */
-               case 'N':
-                       server_alg |= ALG_HASH_NO_SERVER;
-                       break;
                case '?':
                default:
                        usage();
diff --git a/server.c b/server.c
index b9c016a..0238ef6 100644
--- a/server.c
+++ b/server.c
@@ -51,7 +51,11 @@ static int pen_hash(struct sockaddr_storage *a)
        switch (a->ss_family) {
        case AF_INET:
                si = (struct sockaddr_in *)a;
-               hash = (si->sin_addr.s_addr ^ si->sin_port) % nservers;
+               if (server_alg & ALG_ROUNDROBIN) {
+                       hash = (si->sin_addr.s_addr ^ si->sin_port) % nservers;
+               } else {
+                       hash = si->sin_addr.s_addr % nservers;
+               }

                DEBUG(2, "Hash: %d", hash);

@@ -206,7 +210,7 @@ int initial_server(int conn)
        }
        if (server_alg & ALG_PRIO) return server_by_prio();
        if (server_alg & ALG_WEIGHT) return server_by_weight();
-       if (server_alg & ALG_HASH || server_alg & ALG_HASH_NO_SERVER) return pen_hash(&clients[conns[conn].client].addr);
+       if (server_alg & ALG_HASH) return pen_hash(&clients[conns[conn].client].addr);
        return server_by_roundrobin();
 }

diff --git a/server.h b/server.h
index 53a67d8..b89d0aa 100644
--- a/server.h
+++ b/server.h
@@ -11,7 +11,6 @@
 #define ALG_PRIO 8
 #define ALG_HASH 16
 #define ALG_STUBBORN 32
-#define ALG_HASH_NO_SERVER 64

 #define EMERGENCY_SERVER (-1)
 #define ABUSE_SERVER (-2)

@mvrogowski
Copy link
Author

I will run some tests and reply as soon it's done.

@mvrogowski
Copy link
Author

I tested your solution and all works very well!

Please, tell me when you add to 0.31 and make it public.

Thank's, Ulric!

@UlricE
Copy link
Owner

UlricE commented Oct 12, 2015

Pushed and released. 👍

netbsd-srcmastr pushed a commit to NetBSD/pkgsrc that referenced this pull request Jan 10, 2019
- Corrected typo in pen.c per suggestion by Belinda Liu.
  This fixes issue #38.
- Merged pull request from Vincent Bernat for OpenSSL 1.1.0 compatibility.
  This fixes issue #28.
- Allow setting local address for upstream connections. This fixes issue #31.
- New penctl command "source" to set this option.
- Fixed issue #30: UDP not working in combination with a configuration file.
- In epoll.c: check for EPOLLHUP.
- In dsr.c: always use our real mac address, to avoid confusing switches.
- Cleaned up code residue surrounded by "#if 0".
- Added CS_HALFDEAD for UDP streams that haven't seen traffic in a while.
- Bug in pending_and_closing: don't modify the list we're looping over.
- Updated pen manpage.
- Deprecated -Q option (it didn't do anything since kqueue was already the
  default where it was available).
- Fixed error handling in epoll support.
- Added transparent UDP test case to testsuite.sh.
- Contribution from Talik Eichinger: add X-Forwarded-Proto when doing
  SSL decryption.
- Added tarpit test case to testsuite.sh.
- Tarpit functionality to be used with the DSR mode.
- pen.1: removed obsolete -S option, updated defaults for -x and -L.
- In failover_server: sanity checks to failover routine.
- In add_client: add the initial server to .client as well as .initial.
- In failover_server: changed abuse_server to ABUSE_SERVER and emerg_server
  to EMERG_SERVER, to handle their default NO_SERVER values.
  See issue #19 on Github.
- At the suggestion from Marcos Vinicius Rogowski, the hash algorith
  will now include the client port number if the -r (roundrobin)
  option is used. See UlricE/pen#18
- Fixed IP-based client tracking.
- Removed unnecessary #include <pen.h> in dlist.c
- Added UDP mode for Direct Server Return.
- Updated configure.ac for compatibility with CentOS 6.
- Added #ifdef around SSLv3 initialization code in ssl, as
  suggested by [email protected].
- Transparent reverse proxy support for Linux, FreeBSD and OpenBSD.
- Allow the client table size to be updated on the fly. Default size still 2048.
- Allow the connection table size to be updated in the fly. Default still 500.
- See penctl.1, options clients_max and conn_max.
- Introduced the macro NO_SERVER to be used instead of -1 to signify
  error conditions and such.
- Removed the fixed server table size along with the -S option.
- Fixed cosmetic bug in startup code which required port to be specified
  on backend servers even if it was the same as the listening port.
- Numerous updates to support the madness that is Windows.
- Fix from Vincent Bernat: segfault when not using SSL.
- DSR support using Netmap on FreeBSD.
- Unbroke DSR on Linux.
- Replaced all calls to perror with debug(..., strerror(errno);
- Updated penlog and penlogd to use diag.[ch].
- More refactoring: broke out conn.[ch], client.[ch], server.[ch],
  idler.[ch].
- Made a hash index such that the load balancer may balance load.
- Broke out Windows code from pen.c into windows.c. Added windows.h.
- Broke out public definitions for dsr into dsr.h.
- Broke out memory management into memory.[ch].
- Broke out dignostic and logging functions into diag.[ch].
- Broke out settings into settings.[ch].
- Broke out access lists into acl.[ch].
- Broke out event initialization into event.[ch].
- Added pen_epoll.h, pen_kqueue.h, pen_poll.h, pen_select.h.
- Broke out pen_aton et al into netconv.[ch].
- Added dsr.c
- Bug in copy_down affecting SSL connections fixed.
- Updated ocsp stapling to be compatible with server name indication.
- Added pen-ocsp.sh script.
- SSL code broken out into ssl.[ch]. SSL context creation broken
  out from ssl_init to ssl_create_context.
- Server Name Indication support. New command to enable:
  ssl_sni_path PATH
  where PATH is the name of a directory containing domain.key,
  domain.crt and domain.ca files for each domain.
- OCSP stapling. New command ssl_ocsp_response filename
  specifies the location of the ocsp response to be stapled.
  The response must be pre-fetched. The idea was borrowed
  from Rob Stradling.
- New command ssl_client_renegotiation_interval specifies the
  minimum number of seconds the client must wait between
  renegotiation requests. Default 3600.
- Enabled SSL session resumption.
- In do_cmd: don't print "ignoring command" for comments starting
  with '#'.
- Added ssl_option no_tlsv1.1 and ssl_option no_tlsv1.2 to disable
  SSL 1.1 and 1.2 respectively.
- Added autoconf check that the ECDHE is available and not disabled.
- Bumped default max connections and listen queue to 500.
- Support for ECDHE cipher suites.
- New commands ssl_option and ssl_ciphers to individually disable
  insecure protocols and ciphers.
- Updated penctl.1 with the new command.
- New knob to tweak max number of pending nonblocking connection
  attempts: pending_max N (default 100).
- Moved dlist prototypes to dlist.h.
- Added check to close idle connections after a period of inactivity.
- Penctl: idle_timeout N (default 0 = never close idle connections).
- Moved git repository to GitHub..
- New feature: dummy server. Rather than acting as a proxy,
  Pen will pretend to be a web server with just barely enough
  functionality to work as a test target.
- Penctl: dummy|no dummy.
- Yet Another command: abort_on_error|no abort_on_error makes
  Pen call abort() (or not) when encountering a fatal error.
- New feature: "reliable idling". Pen will make and maintain a
  number of idle connections to the backend servers. When a connection
  closes, a new one is made (hence "reliable"). Penctl: idlers [N].
- In do_cmd: return diagnostics to penctl so the user can see them,
  instead of uselessly sending them to syslog.
- New penctl commands:
    - socket N (print which connection the socket belongs to)
    - connection N (print info on the specified connection)
    - close N (forcibly close connection N)
- In open_listener: check that the requested port is in range.
- Fixed bug in dlist_insert.
- Even load distribution when a server is unavailable.
- Let pen save the settings for tcp_nodelay and tcp_fastclose.
- Make flush_up and flush_down return the correct value on error.
- Added config.h.win with reasonable settings for Windows.
- Better detection and blacklisting of unavailable servers.
- New penctl commands:
    - tcp_nodelay sets TCP_NODELAY on sockets. Turn off with no tcp_nodelay.
    - tcp_fastclose closes both upstream and downstream sockets if one of them
    - closes theirs. Will take the values up, down, both or off (default).
- Rather than making a table of pending connections every time through
  the main loop, keep them in a doubly linked list which is only updated
  as needed. O(n) -> O(1).
- A bug in udp mode: after successful "connect", do not event_add downfd,
  because it is equal to listenfd and epoll_ctl doesn't like that.
- Module kqueue.c updated.
- Module poll.c: set unused fd:s to -1, or Solaris will say ENOSYS.
- Enable diagnostic messages by default in configure.ac.
- Changed event bookkeeping from stateless to stateful.
- Made keepalive optional and added "keepalive / no keepalive" penctl command.
- Added windows.c and pen.h to the release tarball.
- More sensible autoconfiguration defaults: poll, kqueue, epoll, openssl and geoip
  are built if found unless explicitly excluded.
- New event management defaults: kqueue, epoll, poll, select in that order.
- New penctl commands: kqueue, epoll, poll, select.
- New command line option: -O cmd where cmd is any penctl command.
  E.g. -O select to use select instead of the compiled-in default.
- New penctl option "listen [address:]port" to allow listening address
  to be changed on the fly or via a configuration file.
- New pen options -i and -u to install and uninstall Pen as a Windows service.
- See pen manpage.
- Reduced default timeout to 3 seconds.
- New autoconf option --enable-debugging to enable debugging code.
- Lots of fixes for compatibility with Windows.
- Fixed bug in mainloop which kept trying to write 0 bytes.
- MinGW port. Use Makefile.win to compile.
- Event management code broken out into select.c, poll.c, kqueue.c and epoll.c.
- New command-line option -m to accept multiple incoming connections in a batch.
- New command-line option -q to set incoming pending connection queue length.
- Close upfd when failing over.
- Adjusted debug logging levels.
- Started on epoll support for Linux.
- Rewrote output_net and output_file to take a variable number of arguments.
- Handle timed out connection attempts in mainloop_kqueue.
- Fixed mainloop_kqueue.
- A lot of code broken out from mainloop_select into separate functions.
- Fixed mainloop_poll.
- Bugfixes related to the new backend connection logic.
- Cleaned up and simplified add_client() and associated circuitry.
- Connections to back end servers are now nonblocking and parallel.
- Removed the -n option and all code explicitly using blocking sockets.
- Removed the -D option and the "delayed forward" feature.
- Renamed server and client fields in the conn, client and server structures
  to better reflect what they are.
- Restructured the add_client, store_client, store_conn and try_server
  functions.
- Allow write_cfg to save IPv6 and GeoIP access lists.
- Fixed a bug in write_cfg, where Pen would try to write to an unwritable
  file. Reported by Steven Myint
- Return UDP replies from the server to the client.
- UDP load balancing code restructured and bugfixed.
- In mainloop_select: When there is a pending connection, keep accepting
  up to multi_accept times *or* until EAGAIN *or* connection table is full.
  This improves performance under load.
- Updated GeoIP support for IPv6.
- Servers can have ipv6 addresses. It is possible to use a mix of ipv4
  and ipv6 servers:
  ./pen -df -S 2 -r :::2222 [::1]:22 [127.0.0.1]:22
- In order to allow server addresses with : in them (i.e. ipv6), it is now
  possible to use square brackets around the address part of the server
  specification: [address]:port (e.g. [::1]:8080).
- Pen can now listen on ipv6 sockets in addition to ipv4 and unix ones.
  I.e. things like "pen ::1:2222 127.0.0.1:22" are now possible.
- snprintf format errors reported by Christopher Meng fixed in
  pen.c and penctl.c.
- Updated pen manpage to clarify what the control socket does.
- Resist opening control socket running as root.
- Remove the default file name for web log.
- New feature: unix domain listening sockets.
- Redesigned server and client structs to allow ipv6 addresses and require
  less casting (yuck) in the code.
- Updated penctl man page with syntax for IPv6 and GeoIP access lists.
- Fixed cosmetic signedness compiler warnings.
- Moved defines for ACE_IPV4 et al outside #ifdef HAVE_SSL clause.
  Otherwise pen won't compile without ssl.
- GeoIP access lists.
- Added "special exception" clause for linking with OpenSSL.
- Penlog ipv6 compatible.
- Modernized automake configuration.
- Penctl ipv6 compatible.
- Updated autoconf to 2.69.
- Updated SSL code. Protocol ssl2 removed. Default changed to tls1.
- Added UDP patch from Zen.
- Added patch from Debian that fixes some issues with penctl.cgi.
- Priority based server selection algorithm.
- Patch from Stephen P. Schaefer fixes several issues in write_cfg.
- In the server_by_weight function, multiply current connections
  by WEIGHT_FACTOR to make the selection mo fine grained when the
  number of connections is small.
- Patch from Dana Contreras: send stdio to /dev/null after forking.
- Fixed a bunch of cosmetic signedness compiler warnings.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants