MAJOR: config: remove parsing of the global "nbproc" directive

This one was deprecated in 2.3 and marked for removal in 2.5. It suffers
too many limitations compared to threads, and prevents some improvements
from being engaged. Instead of a bypassable startup error, there is now
a hard error.

The parsing code was removed, and very few obvious cases were as well.
The code is deeply rooted at certain places (e.g. "for" loops iterating
from 0 to nbproc) so it will not be that trivial to remove everywhere.
The "bind" and "bind-process" parsers will have to be adjusted, though
maybe not completely changed if we later want to support thread groups
for large NUMA machines. Some stats socket restrictions were removed,
and the doc was updated according to what was done. A few places in the
doc still refer to nbproc and will have to be revisited. The master-worker
code also refers to the process number to distinguish between master and
workers and will have to be carefully adjusted. The MAX_PROCS macro was
reset to 1, this will at least reduce the size of some remaining arrays.

Two regtests were dependieng on this directive, one with an explicit
"nbproc 1" and another one testing the master's CLI using nbproc 4.
Both were adapted.

admin/syntax-highlight/haproxy.vim

@@ -46,7 +46,7 @@ syn keyword hapParam     enabled
 syn keyword hapParam     fullconn
 syn keyword hapParam     gid group
 syn keyword hapParam     maxconn monitor-uri
-syn keyword hapParam     nbproc noepoll nopoll
+syn keyword hapParam     noepoll nopoll
 syn keyword hapParam     pidfile
 syn keyword hapParam     quiet
 syn keyword hapParam     redispatch retries

doc/configuration.txt

@@ -990,7 +990,6 @@ The following keywords are supported in the "global" section :
    - lua-load-per-thread
    - lua-prepend-path
    - mworker-max-reloads
-   - nbproc
    - nbthread
    - node
    - numa-cpu-mapping
@@ -1645,30 +1644,15 @@ mworker-max-reloads <number>
   SIGTERM. This option helps to keep under control the number of workers.
   See also "show proc" in the Management Guide.
 
-nbproc <number>  (deprecated)
-  Creates <number> processes when going daemon. This requires the "daemon"
-  mode. By default, only one process is created, which is the recommended mode
-  of operation. For systems limited to small sets of file descriptors per
-  process, it may be needed to fork multiple daemons. When set to a value
-  larger than 1, threads are automatically disabled. USING MULTIPLE PROCESSES
-  IS HARDER TO DEBUG AND IS REALLY DISCOURAGED. This directive is deprecated
-  and scheduled for removal in 2.5. Please use "nbthread" instead. See also
-  "daemon" and "nbthread".
-
 nbthread <number>
   This setting is only available when support for threads was built in. It
-  makes HAProxy run on <number> threads. This is exclusive with "nbproc". While
+  makes HAProxy run on <number> threads. "nbthread" also works when HAProxy is
-  "nbproc" historically used to be the only way to use multiple processors, it
+  started in foreground. On some platforms supporting CPU affinity, the default
-  also involved a number of shortcomings related to the lack of synchronization
+  "nbthread" value is automatically set to the number of CPUs the process is
-  between processes (health-checks, peers, stick-tables, stats, ...) which do
+  bound to upon startup. This means that the thread count can easily be
-  not affect threads. As such, any modern configuration is strongly encouraged
+  adjusted from the calling process using commands like "taskset" or "cpuset".
-  to migrate away from "nbproc" to "nbthread". "nbthread" also works when
+  Otherwise, this value defaults to 1. The default value is reported in the
-  HAProxy is started in foreground. On some platforms supporting CPU affinity,
+  output of "haproxy -vv".
-  when nbproc is not used, the default "nbthread" value is automatically set to
-  the number of CPUs the process is bound to upon startup. This means that the
-  thread count can easily be adjusted from the calling process using commands
-  like "taskset" or "cpuset". Otherwise, this value defaults to 1. The default
-  value is reported in the output of "haproxy -vv". See also "nbproc".
 
 numa-cpu-mapping
   By default, if running on Linux, HAProxy inspects on startup the CPU topology
@@ -2226,11 +2210,10 @@ maxcompcpuusage <number>
   Sets the maximum CPU usage HAProxy can reach before stopping the compression
   for new requests or decreasing the compression level of current requests.
   It works like 'maxcomprate' but measures CPU usage instead of incoming data
-  bandwidth. The value is expressed in percent of the CPU used by HAProxy. In
+  bandwidth. The value is expressed in percent of the CPU used by HAProxy. A
-  case of multiple processes (nbproc > 1), each process manages its individual
+  value of 100 disable the limit. The default value is 100. Setting a lower
-  usage. A value of 100 disable the limit. The default value is 100. Setting
+  value will prevent the compression work from slowing the whole process down
-  a lower value will prevent the compression work from slowing the whole
+  and from introducing high latencies.
-  process down and from introducing high latencies.
 
 maxpipes <number>
   Sets the maximum per-process number of pipes to <number>. Currently, pipes
@@ -2718,8 +2701,8 @@ tune.ssl.cachesize <number>
   idle entries are purged and reassigned. Higher values reduce the occurrence
   of such a purge, hence the number of CPU-intensive SSL handshakes by ensuring
   that all users keep their session as long as possible. All entries are
-  pre-allocated upon startup and are shared between all processes if "nbproc"
+  pre-allocated upon startup. Setting this value to 0 disables the SSL session
-  is greater than 1. Setting this value to 0 disables the SSL session cache.
+  cache.
 
 tune.ssl.force-private-cache
   This option disables SSL session cache sharing between all processes. It
@@ -4039,12 +4022,6 @@ bind /<path> [, ...] [param*]
                       only in log-forward sections.
                     - 'unix@'  -> address is a path to a local unix socket
                     - 'abns@'  -> address is in abstract namespace (Linux only).
-                      Note: since abstract sockets are not "rebindable", they
-                            do not cope well with multi-process mode during
-                            soft-restart, so it is better to avoid them if
-                            nbproc is greater than 1. The effect is that if the
-                            new process fails to start, only one of the old ones
-                            will be able to rebind to the socket.
                     - 'fd@<n>' -> use file descriptor <n> inherited from the
                       parent. The fd must be bound and may or may not already
                       be listening.
@@ -4187,7 +4164,7 @@ bind-process [ all | odd | even | <process_num>[-[<process_num>]] ] ...
             bind 10.0.0.4:80
             bind-process 1-4
 
-  See also : "nbproc" in global section, and "process" in section 5.1.
+  See also : "process" in section 5.1.
 
 
 capture cookie <name> len <length>
@@ -10497,10 +10474,6 @@ stats admin { if | unless } <cond>
   The admin level allows to enable/disable servers from the web interface. By
   default, statistics page is read-only for security reasons.
 
-  Note : Consider not using this feature in multi-process mode (nbproc > 1)
-         unless you know what you do : memory is not shared between the
-         processes, which can result in random behaviors.
-
   Currently, the POST request is limited to the buffer size minus the reserved
   buffer space, which means that if the list of servers is too long, the
   request won't be processed. It is recommended to alter few servers at a
@@ -10534,7 +10507,7 @@ stats admin { if | unless } <cond>
         stats http-request auth unless AUTH
         stats admin if AUTH_ADMIN
 
-  See also : "stats enable", "stats auth", "stats http-request", "nbproc",
+  See also : "stats enable", "stats auth", "stats http-request",
              "bind-process", section 3.4 about userlists and section 7 about
              ACL usage.
 
@@ -10998,10 +10971,6 @@ stick match <pattern> [table <table>] [{if | unless} <cond>]
   way, it becomes very easy to insert cookies and match on IP addresses in
   order to maintain stickiness between HTTP and HTTPS.
 
-  Note : Consider not using this feature in multi-process mode (nbproc > 1)
-         unless you know what you do : memory is not shared between the
-         processes, which can result in random behaviors.
-
   Example :
     # forward SMTP users to the same server they just used for POP in the
     # last 30 minutes
@@ -11020,7 +10989,7 @@ stick match <pattern> [table <table>] [{if | unless} <cond>]
         server s1 192.168.1.1:25
         server s2 192.168.1.1:25
 
-  See also : "stick-table", "stick on", "nbproc", "bind-process" and section 7
+  See also : "stick-table", "stick on", "bind-process" and section 7
              about ACLs and samples fetching.
 
 
@@ -11034,10 +11003,6 @@ stick on <pattern> [table <table>] [{if | unless} <condition>]
          to both keywords for details. It is only provided as a convenience
          for writing more maintainable configurations.
 
-  Note : Consider not using this feature in multi-process mode (nbproc > 1)
-         unless you know what you do : memory is not shared between the
-         processes, which can result in random behaviors.
-
   Examples :
     # The following form ...
     stick on src table pop if !localhost
@@ -11065,7 +11030,7 @@ stick on <pattern> [table <table>] [{if | unless} <condition>]
         server s1 192.168.1.1:443
         server s2 192.168.1.1:443
 
-  See also : "stick match", "stick store-request", "nbproc" and "bind-process".
+  See also : "stick match", "stick store-request" and "bind-process".
 
 
 stick store-request <pattern> [table <table>] [{if | unless} <condition>]
@@ -11131,10 +11096,6 @@ stick store-request <pattern> [table <table>] [{if | unless} <condition>]
   established, so that the table will contain the real server that processed
   the request.
 
-  Note : Consider not using this feature in multi-process mode (nbproc > 1)
-         unless you know what you do : memory is not shared between the
-         processes, which can result in random behaviors.
-
   Example :
     # forward SMTP users to the same server they just used for POP in the
     # last 30 minutes
@@ -11153,7 +11114,7 @@ stick store-request <pattern> [table <table>] [{if | unless} <condition>]
         server s1 192.168.1.1:25
         server s2 192.168.1.1:25
 
-  See also : "stick-table", "stick on", "nbproc", "bind-process" and section 7
+  See also : "stick-table", "stick on", "bind-process" and section 7
              about ACLs and sample fetching.
 
 
@@ -13770,7 +13731,7 @@ process <process-set>[/<thread-set>]
   the same IP:port but not the same process in a listener, so that the system
   can distribute the incoming connections into multiple queues and allow a
   smoother inter-process load balancing. Currently Linux 3.9 and above is known
-  for supporting this. See also "bind-process" and "nbproc".
+  for supporting this. See also "bind-process".
 
 proto <name>
   Forces the multiplexer's protocol to use for the incoming connections. It
@@ -17366,11 +17327,6 @@ lat_ns_tot : integer
 meth(<method>) : method
   Returns a method.
 
-nbproc : integer
-  Returns an integer value corresponding to the number of processes that were
-  started (it equals the global "nbproc" setting). This is useful for logging
-  and debugging purposes.
-
 nbsrv([<backend>]) : integer
   Returns an integer value corresponding to the number of usable servers of
   either the current backend or the named backend. This is mostly used with
@@ -17391,9 +17347,8 @@ prio_offset : integer
   set-priority-offset".
 
 proc : integer
-  Returns an integer value corresponding to the position of the process calling
+  Always returns value 1 (historically it would return the calling process
-  the function, between 1 and global.nbproc. This is useful for logging and
+  number).
-  debugging purposes.
 
 queue([<backend>]) : integer
   Returns the total number of queued connections of the designated backend,

examples/option-http_proxy.cfg

@@ -9,7 +9,6 @@ global
         uid             200
         gid             200
         chroot          /var/empty
-	nbproc		4
         daemon
 
 frontend test-proxy

include/haproxy/defaults.h

@@ -27,7 +27,7 @@
  * resources on embedded systems.
  */
 #ifndef MAX_PROCS
-#define MAX_PROCS LONGBITS
+#define MAX_PROCS 1
 #endif
 
 /* MAX_THREADS defines the highest limit for the global nbthread value. It

reg-tests/connection/proxy_protocol_random_fail.vtc

@@ -24,7 +24,6 @@ syslog Slog_1 -repeat 8 -level info {
 
 haproxy h1 -conf {
     global
-        nbproc 4
         tune.ssl.default-dh-param 2048
         stats bind-process 1
         log ${Slog_1_addr}:${Slog_1_port} len 2048 local0 debug err
@@ -37,13 +36,11 @@ haproxy h1 -conf {
         log global
 
     listen http
-        bind-process 1
         bind unix@"${tmpdir}/http.socket" accept-proxy name ssl-offload-http
         option forwardfor
 
     listen ssl-offload-http
         option httplog
-        bind-process 2-4
         bind "fd@${ssl}" ssl crt ${testdir}/common.pem ssl no-sslv3 alpn h2,http/1.1
         server http unix@"${tmpdir}/http.socket" send-proxy
 } -start

reg-tests/mcli/mcli_show_info.vtc

@@ -1,4 +1,4 @@
-varnishtest "Show info of process 3"
+varnishtest "Show info of process 1"
 
 #REQUIRE_VERSION=1.9
 
@@ -9,8 +9,6 @@ server s1 {
 } -start
 
 haproxy h1 -W -S -conf {
-    global
-        nbproc 4
     defaults
         mode http
         timeout connect 1s
@@ -26,6 +24,6 @@ haproxy h1 -W -S -conf {
 } -start
 
 haproxy h1 -mcli {
-    send "@3 show info"
+    send "@1 show info"
-    expect ~ ".*\nProcess_num: 3\n.*"
+    expect ~ ".*\nProcess_num: 1\n.*"
 } -wait

reg-tests/mcli/mcli_start_progs.vtc

@@ -10,8 +10,6 @@ server s1 {
 } -start
 
 haproxy h1 -W -S -conf {
-    global
-        nbproc 1
     defaults
         mode http
         timeout connect 1s

src/cfgparse-global.c

@@ -564,21 +564,9 @@ int cfg_parse_global(const char *file, int linenum, char **args, int kwm)
 	}
 	/* end of user/group name handling*/
 	else if (strcmp(args[0], "nbproc") == 0) {
-		if (alertif_too_many_args(1, file, linenum, args, &err_code))
+		ha_alert("parsing [%s:%d] : nbproc is not supported any more since HAProxy 2.5. Threads will automatically be used on multi-processor machines if available.\n", file, linenum);
-			goto out;
+		err_code |= ERR_ALERT | ERR_FATAL;
-		if (*(args[1]) == 0) {
+		goto out;
-			ha_alert("parsing [%s:%d] : '%s' expects an integer argument.\n", file, linenum, args[0]);
-			err_code |= ERR_ALERT | ERR_FATAL;
-			goto out;
-		}
-		global.nbproc = atol(args[1]);
-		all_proc_mask = nbits(global.nbproc);
-		if (global.nbproc < 1 || global.nbproc > MAX_PROCS) {
-			ha_alert("parsing [%s:%d] : '%s' must be between 1 and %d (was %d).\n",
-				 file, linenum, args[0], MAX_PROCS, global.nbproc);
-			err_code |= ERR_ALERT | ERR_FATAL;
-			goto out;
-		}
 	}
 	else if (strcmp(args[0], "nbthread") == 0) {
 		if (alertif_too_many_args(1, file, linenum, args, &err_code))

src/cfgparse.c

@@ -2646,7 +2646,7 @@ int check_config_validity()
 		global.nbthread = 1;
 
 #if defined(USE_THREAD)
-		if (global.nbproc == 1) {
+		{
 			int numa_cores = 0;
 #if defined(__linux__) && defined USE_CPU_AFFINITY
 			if (global.numa_cpu_mapping && !thread_cpu_mask_forced())
@@ -2659,12 +2659,6 @@ int check_config_validity()
 #endif
 	}
 
-	if (global.nbproc > 1 && global.nbthread > 1) {
-		ha_alert("cannot enable multiple processes if multiple threads are configured. Please use either nbproc or nbthread but not both.\n");
-		err_code |= ERR_ALERT | ERR_FATAL;
-		goto out;
-	}
-
 	pool_head_requri = create_pool("requri", global.tune.requri_len , MEM_F_SHARED);
 
 	pool_head_capture = create_pool("capture", global.tune.cookie_len, MEM_F_SHARED);
@@ -2731,10 +2725,6 @@ int check_config_validity()
 				ha_warning("Proxy '%s': the process specified on the 'bind-process' directive refers to a process number that is higher than global.nbproc. The proxy has been forced to run on process 1 only.\n", curproxy->id);
 				curproxy->bind_proc = 1;
 			}
-			else if (!curproxy->bind_proc && nbproc > 1) {
-				ha_warning("Proxy '%s': all processes specified on the 'bind-process' directive refer to numbers that are all higher than global.nbproc. The directive was ignored and the proxy will run on all processes.\n", curproxy->id);
-				curproxy->bind_proc = 0;
-			}
 		}
 
 		/* check and reduce the bind-proc of each listener */
@@ -3993,22 +3983,6 @@ out_uri_auth_compat:
 
 	/* Check multi-process mode compatibility */
 
-	if (global.nbproc > 1 && global.cli_fe) {
-		list_for_each_entry(bind_conf, &global.cli_fe->conf.bind, by_fe) {
-			unsigned long mask;
-
-			mask  = proc_mask(global.cli_fe->bind_proc) && all_proc_mask;
-			mask &= proc_mask(bind_conf->settings.bind_proc);
-
-			/* stop here if more than one process is used */
-			if (atleast2(mask))
-				break;
-		}
-		if (&bind_conf->by_fe != &global.cli_fe->conf.bind) {
-			ha_warning("stats socket will not work as expected in multi-process mode (nbproc > 1), you should force process binding globally using 'stats bind-process' or per socket using the 'process' attribute.\n");
-		}
-	}
-
 	/* Make each frontend inherit bind-process from its listeners when not specified. */
 	for (curproxy = proxies_list; curproxy; curproxy = curproxy->next) {
 		if (curproxy->bind_proc)

src/haproxy.c

@@ -1983,17 +1983,6 @@ static void init(int argc, char **argv)
 		ha_warning("a master CLI socket was defined, but master-worker mode (-W) is not enabled.\n");
 	}
 
-	if (global.nbproc > 1 && !global.nbthread) {
-		ha_warning("nbproc is deprecated!\n"
-			   "  | For suffering many limitations, the 'nbproc' directive is now deprecated\n"
-			   "  | and scheduled for removal in 2.5. Just comment it out: haproxy will use\n"
-			   "  | threads and will run on all allocated processors. You may also switch to\n"
-			   "  | 'nbthread %d' to keep the same number of processors. If you absolutely\n"
-			   "  | want to run in multi-process mode, you can silence this warning by adding\n"
-			   "  | 'nbthread 1', but then please report your use case to developers.\n",
-		           global.nbproc);
-	}
-
 	/* defaults sections are not needed anymore */
 	proxy_destroy_all_defaults();