logger: Expose max_lines and correct flag types (#5956)

Multiple flags are moved from FLAG to CLI_FLAG due to the way they are
implemented/used in code. If they were FLAG(s), meaning if they were
also configurable via configuration at runtime, the new values would
still be ignored.

These are:
  - logger_tls_endpoint
  - logger_tls_period
  - logger_tls_max

The flag logger_tls_max has been renamed to logger_tls_max_linesize and
an alias is added for compatibility.

A new flag is added, logger_tls_max_lines, to expose a previously
hardcoded value of 1024 for the maximum number of log lines to send
every period.
This commit is contained in:
Teddy Reed 2019-10-28 05:19:50 -04:00 committed by Alessandro Gario
parent 8b54ba9b16
commit 0c06658e45
2 changed files with 28 additions and 13 deletions

View File

@ -235,12 +235,16 @@ See the **tls**/[remote](../deployment/remote.md) plugin documentation. This is
Optionally enable GZIP compression for request bodies when sending. This is optional, and disabled by default, as the deployment must explicitly know that the logging endpoint supports GZIP for content encoding.
`--logger_tls_max=1048576`
`--logger_tls_max_linesize=1048576`
It is common for TLS/HTTPS servers to enforce a maximum request body size. The default behavior in osquery is to enforce each log line be under 1M bytes. This means each result line from a query's results cannot exceed 1M, this is very unlikely. Each log attempt will try to forward up to 1024 lines. If your service is limited request bodies, configure the client to limit the log line size.
Use this only in emergency situations as size violations are dropped. It is extremely uncommon for this to occur, as the `--value_max` for each column would need to be drastically larger, or the offending table would have to implement several hundred columns.
`--logger_tls_max_lines=1024`
This configures the max number of log lines to send every period (meaning every `logger_tls_period`).
`--distributed_tls_read_endpoint=`
The URI path which will be used, in conjunction with `--tls_hostname`, to create the remote URI for retrieving distributed queries when using the **tls** distributed plugin.

View File

@ -18,6 +18,7 @@
#include <osquery/enroll.h>
#include <osquery/flags.h>
#include <osquery/flagalias.h>
#include <osquery/registry.h>
#include <osquery/remote/serializers/json.h>
@ -27,20 +28,29 @@
namespace osquery {
constexpr size_t kTLSMaxLogLines = 1024;
CLI_FLAG(uint64,
logger_tls_max_lines,
1024,
"Max number of logs to send per period");
FLAG(string, logger_tls_endpoint, "", "TLS/HTTPS endpoint for results logging");
CLI_FLAG(string,
logger_tls_endpoint,
"",
"TLS/HTTPS endpoint for results logging");
CLI_FLAG(uint64,
logger_tls_period,
4,
"Seconds between flushing logs over TLS/HTTPS");
FLAG(uint64,
logger_tls_period,
4,
"Seconds between flushing logs over TLS/HTTPS");
FLAG(uint64,
logger_tls_max,
logger_tls_max_linesize,
1 * 1024 * 1024,
"Max size in bytes allowed per log line");
// The flag name logger_tls_max is deprecated.
FLAG_ALIAS(google::uint64, logger_tls_max, logger_tls_max_linesize);
FLAG(bool, logger_tls_compress, false, "GZip compress TLS/HTTPS request body");
REGISTER(TLSLoggerPlugin, "logger", "tls");
@ -49,7 +59,7 @@ TLSLogForwarder::TLSLogForwarder()
: BufferedLogForwarder("TLSLogForwarder",
"tls",
std::chrono::seconds(FLAGS_logger_tls_period),
kTLSMaxLogLines) {
FLAGS_logger_tls_max_lines) {
uri_ = TLSRequestHelper::makeURI(FLAGS_logger_tls_endpoint);
}
@ -98,8 +108,9 @@ Status TLSLogForwarder::send(std::vector<std::string>& log_data,
auto children = params.newArray();
iterate(log_data, ([&params, &children](std::string& item) {
// Enforce a max log line size for TLS logging.
if (item.size() > FLAGS_logger_tls_max) {
LOG(WARNING) << "Line exceeds TLS logger max: " << item.size();
if (item.size() > FLAGS_logger_tls_max_linesize) {
LOG(WARNING)
<< "Linesize exceeds TLS logger maximum: " << item.size();
return;
}
@ -123,4 +134,4 @@ Status TLSLogForwarder::send(std::vector<std::string>& log_data,
}
return TLSRequestHelper::go<JSONSerializer>(uri_, params, response);
}
}
} // namespace osquery