1berry/haproxy
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

MEDIUM: vars: Add a per-process scope for variables

Browse Source 
Now it is possible to use variables attached to a process. The scope name is
'proc'. These variables are released only when HAProxy is stopped.

'tune.vars.proc-max-size' directive has been added to confiure the maximum
amount of memory used by "proc" variables. And because memory accounting is
hierachical for variables, memory for "proc" vars includes memory for "sess"
vars.
This commit is contained in:
Christopher Faulet 2016-11-09 11:36:17 +01:00 committed by Willy Tarreau
parent 09c9df286b
commit ff2613ed7a
5 changed files with 67 additions and 14 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

32
doc/configuration.txt
View File

@ -618,6 +618,7 @@ The following keywords are supported in the "global" section :
- tune.ssl.default-dh-param
- tune.ssl.ssl-ctx-cache-size
- tune.vars.global-max-size
- tune.vars.proc-max-size
- tune.vars.reqres-max-size
- tune.vars.sess-max-size
- tune.vars.txn-max-size
@ -1501,17 +1502,18 @@ tune.ssl.ssl-ctx-cache-size <number>
1000 entries.
tune.vars.global-max-size <size>
tune.vars.proc-max-size <size>
tune.vars.reqres-max-size <size>
tune.vars.sess-max-size <size>
tune.vars.txn-max-size <size>
These four tunes help to manage the maximum amount of memory used by the
variables system. "global" limits the overall amount of memory available
for all scopes. "sess" limits the memory for the session scope, "txn" for
the transaction scope, and "reqres" limits the memory for each request or
response processing.
Memory accounting is hierarchical, meaning more coarse grained limits
include the finer grained ones: "sess" includes "txn", and "txn" includes
"reqres".
These five tunes help to manage the maximum amount of memory used by the
variables system. "global" limits the overall amount of memory available for
all scopes. "proc" limits the memory for the process scope, "sess" limits the
memory for the session scope, "txn" for the transaction scope, and "reqres"
limits the memory for each request or response processing.
Memory accounting is hierarchical, meaning more coarse grained limits include
the finer grained ones: "proc" includes "sess", "sess" includes "txn", and
"txn" includes "reqres".
For example, when "tune.vars.sess-max-size" is limited to 100,
"tune.vars.txn-max-size" and "tune.vars.reqres-max-size" cannot exceed
@ -3941,6 +3943,7 @@ http-request { allow | tarpit | auth [realm <realm>] | redirect <rule> |
<var-name> The name of the variable starts with an indication about
its scope. The scopes allowed are:
"proc" : the variable is shared with the whole process
"sess" : the variable is shared with the whole session
"txn" : the variable is shared with the transaction
(request and response)
@ -4306,6 +4309,7 @@ http-response { allow | deny | add-header <name> <fmt> | set-nice <nice> |
<var-name> The name of the variable starts with an indication about
its scope. The scopes allowed are:
"proc" : the variable is shared with the whole process
"sess" : the variable is shared with the whole session
"txn" : the variable is shared with the transaction
(request and response)
@ -9068,6 +9072,7 @@ tcp-request content <action> [{if | unless} <condition>]
<var-name> The name of the variable starts with an indication about
its scope. The scopes allowed are:
"proc" : the variable is shared with the whole process
"sess" : the variable is shared with the whole session
"txn" : the variable is shared with the transaction
(request and response)
@ -9281,6 +9286,7 @@ tcp-response content <action> [{if | unless} <condition>]
<var-name> The name of the variable starts with an indication about
its scope. The scopes allowed are:
"proc" : the variable is shared with the whole process
"sess" : the variable is shared with the whole session
"txn" : the variable is shared with the transaction
(request and response)
@ -12105,6 +12111,7 @@ add(<value>)
result as a signed integer. <value> can be a numeric value or a variable
name. The name of the variable starts with an indication about its scope. The
scopes allowed are:
"proc" : the variable is shared with the whole process
"sess" : the variable is shared with the whole session
"txn" : the variable is shared with the transaction (request and response)
"req" : the variable is shared only during request processing
@ -12117,6 +12124,7 @@ and(<value>)
integer, and returns the result as an signed integer. <value> can be a
numeric value or a variable name. The name of the variable starts with an
indication about its scope. The scopes allowed are:
"proc" : the variable is shared with the whole process
"sess" : the variable is shared with the whole session
"txn" : the variable is shared with the transaction (request and response)
"req" : the variable is shared only during request processing
@ -12182,6 +12190,7 @@ div(<value>)
integer is returned (typically 2^63-1). <value> can be a numeric value or a
variable name. The name of the variable starts with an indication about its
scope. The scopes allowed are:
"proc" : the variable is shared with the whole process
"sess" : the variable is shared with the whole session
"txn" : the variable is shared with the transaction (request and response)
"req" : the variable is shared only during request processing
@ -12390,6 +12399,7 @@ mod(<value>)
remainder as an signed integer. If <value> is null, then zero is returned.
<value> can be a numeric value or a variable name. The name of the variable
starts with an indication about its scope. The scopes allowed are:
"proc" : the variable is shared with the whole process
"sess" : the variable is shared with the whole session
"txn" : the variable is shared with the transaction (request and response)
"req" : the variable is shared only during request processing
@ -12403,6 +12413,7 @@ mul(<value>)
value for the sign is returned so that the operation doesn't wrap around.
<value> can be a numeric value or a variable name. The name of the variable
starts with an indication about its scope. The scopes allowed are:
"proc" : the variable is shared with the whole process
"sess" : the variable is shared with the whole session
"txn" : the variable is shared with the transaction (request and response)
"req" : the variable is shared only during request processing
@ -12431,6 +12442,7 @@ or(<value>)
integer, and returns the result as an signed integer. <value> can be a
numeric value or a variable name. The name of the variable starts with an
indication about its scope. The scopes allowed are:
"proc" : the variable is shared with the whole process
"sess" : the variable is shared with the whole session
"txn" : the variable is shared with the transaction (request and response)
"req" : the variable is shared only during request processing
@ -12492,6 +12504,7 @@ set-var(<var name>)
Sets a variable with the input content and returns the content on the output as
is. The variable keeps the value and the associated input type. The name of the
variable starts with an indication about its scope. The scopes allowed are:
"proc" : the variable is shared with the whole process
"sess" : the variable is shared with the whole session
"txn" : the variable is shared with the transaction (request and
response),
@ -12506,6 +12519,7 @@ sub(<value>)
a constant, simply perform a "neg,add(value)". <value> can be a numeric value
or a variable name. The name of the variable starts with an indication about
its scope. The scopes allowed are:
"proc" : the variable is shared with the whole process
"sess" : the variable is shared with the whole session
"txn" : the variable is shared with the transaction (request and
response),
@ -12703,6 +12717,7 @@ xor(<value>)
of type signed integer, and returns the result as an signed integer.
<value> can be a numeric value or a variable name. The name of the variable
starts with an indication about its scope. The scopes allowed are:
"proc" : the variable is shared with the whole process
"sess" : the variable is shared with the whole session
"txn" : the variable is shared with the transaction (request and
response),
@ -12966,6 +12981,7 @@ var(<var-name>) : undefined
Returns a variable with the stored type. If the variable is not set, the
sample fetch fails. The name of the variable starts with an indication
about its scope. The scopes allowed are:
"proc" : the variable is shared with the whole process
"sess" : the variable is shared with the whole session
"txn" : the variable is shared with the transaction (request and
response),

2
include/types/global.h
View File

@ -31,6 +31,7 @@
#include <types/listener.h>
#include <types/proxy.h>
#include <types/task.h>
#include <types/vars.h>
#ifdef USE_51DEGREES
#include <import/51d.h>
@ -179,6 +180,7 @@ struct global {
unsigned long cpu_map[LONGBITS]; /* list of CPU masks for the 32/64 first processes */
#endif
struct proxy *stats_fe; /* the frontend holding the stats settings */
struct vars vars; /* list of variables for the process scope. */
#ifdef USE_DEVICEATLAS
struct {
void *atlasimgptr;

1
include/types/vars.h
View File

@ -10,6 +10,7 @@ enum vars_scope {
SCOPE_TXN,
SCOPE_REQ,
SCOPE_RES,
SCOPE_PROC,
};
struct vars {

6
src/haproxy.c
View File

@ -109,6 +109,7 @@
#include <proto/signal.h>
#include <proto/task.h>
#include <proto/dns.h>
#include <proto/vars.h>
#ifdef USE_OPENSSL
#include <proto/ssl_sock.h>
@ -734,6 +735,9 @@ void init(int argc, char **argv)
/* Initialise lua. */
hlua_init();
/* Initialize process vars */
vars_init(&global.vars, SCOPE_PROC);
global.tune.options |= GTUNE_USE_SELECT; /* select() is always available */
#if defined(ENABLE_POLL)
global.tune.options |= GTUNE_USE_POLL;
@ -1675,6 +1679,8 @@ void deinit(void)
free(wl);
}
vars_prune(&global.vars, NULL, NULL);
pool_destroy2(pool2_stream);
pool_destroy2(pool2_session);
pool_destroy2(pool2_connection);

40
src/vars.c
View File

@ -26,6 +26,7 @@ static int var_names_nb = 0;
/* This array of int contains the system limits per context. */
static unsigned int var_global_limit = 0;
static unsigned int var_global_size = 0;
static unsigned int var_proc_limit = 0;
static unsigned int var_sess_limit = 0;
static unsigned int var_txn_limit = 0;
static unsigned int var_reqres_limit = 0;
@ -45,7 +46,10 @@ static void var_accounting_diff(struct vars *vars, struct session *sess, struct
/* fall through */
case SCOPE_SESS:
sess->vars.size += size;
var_global_size += size;
/* fall through */
case SCOPE_PROC:
global.vars.size += size;
var_global_size += size;
}
}
@ -71,6 +75,10 @@ static int var_accounting_add(struct vars *vars, struct session *sess, struct st
case SCOPE_SESS:
if (var_sess_limit && sess->vars.size + size > var_sess_limit)
return 0;
/* fall through */
case SCOPE_PROC:
if (var_proc_limit && global.vars.size + size > var_proc_limit)
return 0;
if (var_global_limit && var_global_size + size > var_global_limit)
return 0;
}
@ -125,8 +133,9 @@ void vars_prune_per_sess(struct vars *vars)
pool_free2(var_pool, var);
size += sizeof(struct var);
}
vars->size -= size;
var_global_size -= size;
vars->size -= size;
global.vars.size -= size;
var_global_size -= size;
}
/* This function init a list of variabes. */
@ -162,7 +171,12 @@ static char *register_name(const char *name, int len, enum vars_scope *scope,
}
/* Check scope. */
if (len > 5 && strncmp(name, "sess.", 5) == 0) {
if (len > 5 && strncmp(name, "proc.", 5) == 0) {
name += 5;
len -= 5;
*scope = SCOPE_PROC;
}
else if (len > 5 && strncmp(name, "sess.", 5) == 0) {
name += 5;
len -= 5;
*scope = SCOPE_SESS;
@ -184,7 +198,7 @@ static char *register_name(const char *name, int len, enum vars_scope *scope,
}
else {
memprintf(err, "invalid variable name '%s'. A variable name must be start by its scope. "
"The scope can be 'sess', 'txn', 'req' or 'res'", name);
"The scope can be 'proc', 'sess', 'txn', 'req' or 'res'", name);
return NULL;
}
@ -246,6 +260,9 @@ static int smp_fetch_var(const struct arg *args, struct sample *smp, const char
/* Check the availibity of the variable. */
switch (var_desc->scope) {
case SCOPE_PROC:
vars = &global.vars;
break;
case SCOPE_SESS:
vars = &smp->sess->vars;
break;
@ -369,6 +386,7 @@ static inline int sample_store_stream(const char *name, enum vars_scope scope, s
struct vars *vars;
switch (scope) {
case SCOPE_PROC: vars = &global.vars; break;
case SCOPE_SESS: vars = &smp->sess->vars; break;
case SCOPE_TXN: vars = &smp->strm->vars_txn; break;
case SCOPE_REQ:
@ -461,6 +479,7 @@ int vars_get_by_name(const char *name, size_t len, struct sample *smp)
/* Select "vars" pool according with the scope. */
switch (scope) {
case SCOPE_PROC: vars = &global.vars; break;
case SCOPE_SESS: vars = &smp->sess->vars; break;
case SCOPE_TXN: vars = &smp->strm->vars_txn; break;
case SCOPE_REQ:
@ -494,6 +513,7 @@ int vars_get_by_desc(const struct var_desc *var_desc, struct sample *smp)
/* Select "vars" pool according with the scope. */
switch (var_desc->scope) {
case SCOPE_PROC: vars = &global.vars; break;
case SCOPE_SESS: vars = &smp->sess->vars; break;
case SCOPE_TXN: vars = &smp->strm->vars_txn; break;
case SCOPE_REQ:
@ -653,6 +673,13 @@ static int vars_max_size_global(char **args, int section_type, struct proxy *cur
return vars_max_size(args, section_type, curpx, defpx, file, line, err, &var_global_limit);
}
static int vars_max_size_proc(char **args, int section_type, struct proxy *curpx,
struct proxy *defpx, const char *file, int line,
char **err)
{
return vars_max_size(args, section_type, curpx, defpx, file, line, err, &var_proc_limit);
}
static int vars_max_size_sess(char **args, int section_type, struct proxy *curpx,
struct proxy *defpx, const char *file, int line,
char **err)
@ -676,7 +703,7 @@ static int vars_max_size_reqres(char **args, int section_type, struct proxy *cur
static struct sample_fetch_kw_list sample_fetch_keywords = {ILH, {
{ "var", smp_fetch_var, ARG1(1,STR), smp_check_var, SMP_T_STR, SMP_USE_L5CLI },
{ "var", smp_fetch_var, ARG1(1,STR), smp_check_var, SMP_T_STR, SMP_USE_L4CLI },
{ /* END */ },
}};
@ -712,6 +739,7 @@ static struct action_kw_list http_res_kws = { { }, {
static struct cfg_kw_list cfg_kws = {{ },{
{ CFG_GLOBAL, "tune.vars.global-max-size", vars_max_size_global },
{ CFG_GLOBAL, "tune.vars.proc-max-size", vars_max_size_proc },
{ CFG_GLOBAL, "tune.vars.sess-max-size", vars_max_size_sess },
{ CFG_GLOBAL, "tune.vars.txn-max-size", vars_max_size_txn },
{ CFG_GLOBAL, "tune.vars.reqres-max-size", vars_max_size_reqres },