Log Collectors Benchmark

Throughput, resource usage and delivery correctness of log collectors that read log files on a Kubernetes Node, parse structured JSON logs, and enrich them with Pod metadata.

github.com/VictoriaMetrics/log-collectors-benchmark

Results

17 Sep 2026

Maximum throughput 8 passed, 1 failed

50 Pods, each increasing its log rate by 2 logs/sec every second until the collector stops keeping up.

Collector Status Max throughput (logs/sec) higher is better
vlagent v1.52.0 pass 177949
alloy v1.19.2 pass 40853
vector v0.58.0 pass 32057
grafana-agent v0.44.2 pass 31407
opentelemetry-collector v0.158.0 pass 27049
promtail v3.5.1 pass 26805
filebeat v9.5.3 pass 20133
fluentd v1.19 pass 7037
fluent-bit v5.0.9 fail: OOMKilled 41222

Resource usage at 10 000 logs/sec 7 passed, 1 failed

10 Pods produce 1 000 logs/sec each for 1 hour. Collectors unable to sustain this rate are omitted.

Collector Status Avg CPU (cores) lower is better Avg RAM (MiB) lower is better Max RAM (MiB) lower is better Net (bytes/log) lower is better Lost lower is better
vlagent v1.52.0 pass 0.05 31.6 44.5 33 0
fluent-bit v5.0.9 pass 0.20 53.5 105.4 39 3
alloy v1.19.2 pass 0.24 81.4 84.9 95 0
vector v0.58.0 pass 0.29 106.1 123.4 39 0
grafana-agent v0.44.2 pass 0.30 55.2 60.6 54 0
opentelemetry-collector v0.158.0 pass 0.35 106.6 125.5 81 0
promtail v3.5.1 pass 0.39 49.7 53.9 55 0
filebeat v9.5.3 fail: lost too many rows 0.53 89.6 133.2 191 9.3k

Methodology

Each collector runs on its own, with no other collectors on the Node. All log generator Pods produce the same kind of JSON records, see log format for details.

Every collector is limited to 1 CPU and 1 GiB of memory. Configs are based on the official Helm chart values and may include tuning to improve collector performance. The collector names in the tables link to the exact configs of each run.

Throughput is counted at the receiving end, not by the collectors themselves. A collector is marked as failed when its container restarts or when logs are lost. Its numbers still show what it reached before that.

The benchmark runs on a c2d-highcpu-8 virtual machine with a local SSD. Apart from the log generator and the collector under test, the Node only runs the benchmark components: log-verifier, VictoriaMetrics and Grafana.

Kubernetes log rotation settings are kept at their defaults: maximum log file size is 10 MiB, with checks running every 10 seconds.

Send an updated collector config if you know how to speed it up by more than 10% without introducing unusual production settings.

Transport config

Collector Protocol Compression
alloy Loki snappy
filebeat Elasticsearch none
fluent-bit JSON Lines gzip
fluentd JSON Lines gzip
grafana-agent Loki snappy
opentelemetry-collector OpenTelemetry gzip
promtail Loki snappy
vector JSON Lines zstd
vlagent JSON Lines zstd
Log format of each collector

original

{
  "generated_at": 1786383515121049424,
  "_msg": "Request processed successfully",
  "sequence_id": 1,
  "level": "DEBUG",
  "component": "notification-service",
  "method": "DELETE",
  "status_code": 404,
  "duration_ms": 705,
  "trace_id": "trace-a1b2c3d4"
}

alloy

{
  "_msg": "Request processed successfully",
  "_stream": "{collector=\"alloy\",kubernetes.pod_name=\"log-generator-7d7546994b-sgqrv\"}",
  "_time": "2026-08-10T17:38:35.121089019Z",
  "collector": "alloy",
  "component": "notification-service",
  "container": "log-generator",
  "duration_ms": "705",
  "filename": "/var/log/containers/log-generator-7d7546994b-sgqrv_log-generator_log-generator-53c6d5a877d762ec7f32bd346f87dcdf8734d3eddfa5a6fde69c15942ddc51f0.log",
  "generated_at": "1786383515121049424",
  "job": "log-generator/log-generator",
  "kubernetes.pod_name": "log-generator-7d7546994b-sgqrv",
  "level": "DEBUG",
  "method": "DELETE",
  "namespace": "log-generator",
  "sequence_id": "1",
  "sequence_id": "1",
  "status_code": "404",
  "stream": "stdout",
  "trace_id": "trace-a1b2c3d4"
}

fluent-bit

{
  "_msg": "Request processed successfully",
  "_p": "F",
  "_stream": "{collector=\"fluent-bit\",kubernetes.pod_name=\"log-generator-7d7546994b-sgqrv\"}",
  "_time": "2026-08-10T17:38:37.168911826Z",
  "collector": "fluent-bit",
  "component": "notification-service",
  "date": "1786383515.121089",
  "duration_ms": "705",
  "generated_at": "1786383515121049424",
  "kubernetes.container_hash": "sha256:b94e663657e4aaeceedbb53afe77ff0b37be195ed9ad91ae7d036947f99c76be",
  "kubernetes.container_image": "docker.io/library/log-generator:latest",
  "kubernetes.container_name": "log-generator",
  "kubernetes.docker_id": "53c6d5a877d762ec7f32bd346f87dcdf8734d3eddfa5a6fde69c15942ddc51f0",
  "kubernetes.host": "log-collectors-bench-control-plane",
  "kubernetes.labels.app": "log-generator",
  "kubernetes.labels.pod-template-hash": "7d7546994b",
  "kubernetes.namespace_name": "log-generator",
  "kubernetes.pod_id": "dbf56f0e-5f00-4a39-aef1-a6916e33fc11",
  "kubernetes.pod_ip": "10.244.0.39",
  "kubernetes.pod_name": "log-generator-7d7546994b-sgqrv",
  "level": "DEBUG",
  "method": "DELETE",
  "sequence_id": "1",
  "status_code": "404",
  "stream": "stdout",
  "time": "2026-08-10T17:38:35.121089019Z",
  "trace_id": "trace-a1b2c3d4"
}

fluentd

{
  "_msg": "Request processed successfully",
  "_stream": "{collector=\"fluentd\",kubernetes.pod_name=\"log-generator-7d7546994b-sgqrv\"}",
  "_time": "2026-08-10T17:40:09.456255437Z",
  "collector": "fluentd",
  "component": "notification-service",
  "docker.container_id": "53c6d5a877d762ec7f32bd346f87dcdf8734d3eddfa5a6fde69c15942ddc51f0",
  "duration_ms": "705",
  "generated_at": "1786383515121049424",
  "kubernetes.container_image": "docker.io/library/log-generator:latest",
  "kubernetes.container_image_id": "sha256:b94e663657e4aaeceedbb53afe77ff0b37be195ed9ad91ae7d036947f99c76be",
  "kubernetes.container_name": "log-generator",
  "kubernetes.host": "log-collectors-bench-control-plane",
  "kubernetes.labels.app": "log-generator",
  "kubernetes.labels.pod-template-hash": "7d7546994b",
  "kubernetes.namespace_name": "log-generator",
  "kubernetes.pod_id": "dbf56f0e-5f00-4a39-aef1-a6916e33fc11",
  "kubernetes.pod_ip": "10.244.0.39",
  "kubernetes.pod_name": "log-generator-7d7546994b-sgqrv",
  "level": "DEBUG",
  "method": "DELETE",
  "sequence_id": "1",
  "status_code": "404",
  "stream": "stdout",
  "trace_id": "trace-a1b2c3d4"
}

grafana-agent

{
  "_msg": "Request processed successfully",
  "_stream": "{collector=\"grafana-agent\",pod=\"log-generator-7d7546994b-sgqrv\"}",
  "_time": "2026-08-10T17:38:35.121089019Z",
  "collector": "grafana-agent",
  "component": "notification-service",
  "duration_ms": "705",
  "filename": "/var/log/containers/log-generator-7d7546994b-sgqrv_log-generator_log-generator-53c6d5a877d762ec7f32bd346f87dcdf8734d3eddfa5a6fde69c15942ddc51f0.log",
  "generated_at": "1786383515121049424",
  "level": "DEBUG",
  "method": "DELETE",
  "namespace": "log-generator",
  "node_name": "log-collectors-bench-control-plane",
  "pod": "log-generator-7d7546994b-sgqrv",
  "sequence_id": "1",
  "status_code": "404",
  "stream": "stdout",
  "trace_id": "trace-a1b2c3d4"
}

opentelemetry-collector

{
  "_msg": "Request processed successfully",
  "_msg": "{\"generated_at\":1786383515121049424,\"_msg\":\"Request processed successfully\",\"sequence_id\":1,\"level\":\"DEBUG\",\"component\":\"notification-service\",\"method\":\"DELETE\",\"status_code\":404,\"duration_ms\":705,\"trace_id\":\"trace-a1b2c3d4\"}",
  "_stream": "{collector=\"opentelemetry-collector\",k8s.pod.name=\"log-generator-7d7546994b-sgqrv\"}",
  "_time": "2026-08-10T17:38:35.121089019Z",
  "collector": "opentelemetry-collector",
  "component": "notification-service",
  "duration_ms": "705",
  "generated_at": "1786383515121049300",
  "k8s.container.name": "log-generator",
  "k8s.container.restart_count": "0",
  "k8s.namespace.name": "log-generator",
  "k8s.pod.name": "log-generator-7d7546994b-sgqrv",
  "k8s.pod.uid": "dbf56f0e-5f00-4a39-aef1-a6916e33fc11",
  "level": "DEBUG",
  "log.file.path": "/var/log/pods/log-generator_log-generator-7d7546994b-sgqrv_dbf56f0e-5f00-4a39-aef1-a6916e33fc11/log-generator/0.log",
  "log.iostream": "stdout",
  "logtag": "F",
  "method": "DELETE",
  "scope.name": "unknown",
  "scope.version": "unknown",
  "sequence_id": "1",
  "severity": "Unspecified",
  "status_code": "404",
  "trace_id": "trace-a1b2c3d4"
}

promtail

{
  "_msg": "Request processed successfully",
  "_stream": "{collector=\"promtail\",kubernetes.pod_name=\"log-generator-7d7546994b-sgqrv\"}",
  "_time": "2026-08-10T17:38:35.121089019Z",
  "app": "log-generator",
  "collector": "promtail",
  "component": "notification-service",
  "container": "log-generator",
  "duration_ms": "705",
  "filename": "/var/log/pods/log-generator_log-generator-7d7546994b-sgqrv_dbf56f0e-5f00-4a39-aef1-a6916e33fc11/log-generator/0.log",
  "generated_at": "1786383515121049424",
  "job": "log-generator/log-generator",
  "kubernetes.pod_name": "log-generator-7d7546994b-sgqrv",
  "level": "DEBUG",
  "method": "DELETE",
  "namespace": "log-generator",
  "node_name": "log-collectors-bench-control-plane",
  "sequence_id": "1",
  "status_code": "404",
  "stream": "stdout",
  "trace_id": "trace-a1b2c3d4"
}

vector

{
  "_msg": "Request processed successfully",
  "_stream": "{collector=\"vector\",kubernetes.pod_name=\"log-generator-7d7546994b-sgqrv\"}",
  "_time": "2026-08-10T17:38:40.489786168Z",
  "collector": "vector",
  "component": "notification-service",
  "duration_ms": "705",
  "file": "/var/log/pods/log-generator_log-generator-7d7546994b-sgqrv_dbf56f0e-5f00-4a39-aef1-a6916e33fc11/log-generator/0.log",
  "generated_at": "1786383515121049424",
  "kubernetes.container_id": "containerd://53c6d5a877d762ec7f32bd346f87dcdf8734d3eddfa5a6fde69c15942ddc51f0",
  "kubernetes.container_image": "log-generator:latest",
  "kubernetes.container_image_id": "sha256:b94e663657e4aaeceedbb53afe77ff0b37be195ed9ad91ae7d036947f99c76be",
  "kubernetes.container_name": "log-generator",
  "kubernetes.node_labels.beta.kubernetes.io/arch": "amd64",
  "kubernetes.node_labels.beta.kubernetes.io/os": "linux",
  "kubernetes.node_labels.kubernetes.io/arch": "amd64",
  "kubernetes.node_labels.kubernetes.io/hostname": "log-collectors-bench-control-plane",
  "kubernetes.node_labels.kubernetes.io/os": "linux",
  "kubernetes.pod_ip": "10.244.0.39",
  "kubernetes.pod_ips": "[\"10.244.0.39\"]",
  "kubernetes.pod_labels.app": "log-generator",
  "kubernetes.pod_labels.pod-template-hash": "7d7546994b",
  "kubernetes.pod_name": "log-generator-7d7546994b-sgqrv",
  "kubernetes.pod_namespace": "log-generator",
  "kubernetes.pod_node_name": "log-collectors-bench-control-plane",
  "kubernetes.pod_owner": "ReplicaSet/log-generator-7d7546994b",
  "kubernetes.pod_uid": "dbf56f0e-5f00-4a39-aef1-a6916e33fc11",
  "level": "DEBUG",
  "method": "DELETE",
  "sequence_id": "1",
  "source_type": "kubernetes_logs",
  "status_code": "404",
  "stream": "stdout",
  "timestamp": "2026-08-10T17:38:35.121089019Z",
  "trace_id": "trace-a1b2c3d4"
}

vlagent

{
  "_msg": "Request processed successfully",
  "_stream": "{collector=\"vlagent\",kubernetes.pod_name=\"log-generator-7d7546994b-sgqrv\"}",
  "_time": "2026-08-10T17:38:35.121089019Z",
  "collector": "vlagent",
  "component": "notification-service",
  "duration_ms": "705",
  "generated_at": "1786383515121049424",
  "kubernetes.container_id": "containerd://53c6d5a877d762ec7f32bd346f87dcdf8734d3eddfa5a6fde69c15942ddc51f0",
  "kubernetes.container_name": "log-generator",
  "kubernetes.node_labels.beta.kubernetes.io/arch": "amd64",
  "kubernetes.node_labels.beta.kubernetes.io/os": "linux",
  "kubernetes.node_labels.kubernetes.io/arch": "amd64",
  "kubernetes.node_labels.kubernetes.io/hostname": "log-collectors-bench-control-plane",
  "kubernetes.node_labels.kubernetes.io/os": "linux",
  "kubernetes.pod_ip": "10.244.0.39",
  "kubernetes.pod_labels.app": "log-generator",
  "kubernetes.pod_labels.pod-template-hash": "7d7546994b",
  "kubernetes.pod_name": "log-generator-7d7546994b-sgqrv",
  "kubernetes.pod_namespace": "log-generator",
  "kubernetes.pod_node_name": "log-collectors-bench-control-plane",
  "level": "DEBUG",
  "method": "DELETE",
  "sequence_id": "1",
  "status_code": "404",
  "trace_id": "trace-a1b2c3d4"
}

Previous runs

16 Sep 2026

Maximum throughput 8 passed, 1 failed

50 Pods, each increasing its log rate by 2 logs/sec every second until the collector stops keeping up.

Collector Status Max throughput (logs/sec) higher is better
vlagent v1.48.0 pass 176712
grafana-agent v0.44.2 pass 31972
promtail v3.5.1 pass 27589
opentelemetry-collector v0.146.1 pass 27134
alloy v1.13.2 pass 26060
vector v0.53.0 pass 17456
filebeat v9.3.1 pass 11054
fluentd v1.19 pass 7056
fluent-bit v4.2.3 fail: OOMKilled 39734

Resource usage at 10 000 logs/sec 7 passed, 1 failed

10 Pods produce 1 000 logs/sec each for 1 hour. Collectors unable to sustain this rate are omitted.

Collector Status Avg CPU (cores) lower is better Avg RAM (MiB) lower is better Max RAM (MiB) lower is better Net (bytes/log) lower is better Lost lower is better
vlagent v1.48.0 pass 0.05 27.6 44.2 33 0
fluent-bit v4.2.3 pass 0.21 64.3 132.4 39 1
grafana-agent v0.44.2 pass 0.30 55.7 62.7 54 0
opentelemetry-collector v0.146.1 pass 0.35 100.6 120.0 81 0
alloy v1.13.2 pass 0.36 67.1 82.0 96 0
promtail v3.5.1 pass 0.38 49.9 54.5 55 0
vector v0.53.0 pass 0.40 92.2 108.1 39 0
filebeat v9.3.1 fail: lost too many rows 0.84 215.5 274.2 216 579.3k
12 Aug 2026

Maximum throughput 8 passed, 1 failed

50 Pods, each increasing its log rate by 2 logs/sec every second until the collector stops keeping up.

Collector Status Max throughput (logs/sec) higher is better
vlagent v1.48.0 pass 178274
grafana-agent v0.44.2 pass 31964
opentelemetry-collector v0.146.1 pass 27308
promtail v3.5.1 pass 27192
alloy v1.13.2 pass 26478
vector v0.53.0 pass 17436
filebeat v9.3.1 pass 11120
fluentd v1.19 pass 7048
fluent-bit v4.2.3 fail: OOMKilled 40218

Resource usage at 10 000 logs/sec 7 passed, 1 failed

10 Pods produce 1 000 logs/sec each for 1 hour. Collectors unable to sustain this rate are omitted.

Collector Status Avg CPU (cores) lower is better Avg RAM (MiB) lower is better Max RAM (MiB) lower is better Net (bytes/log) lower is better Lost lower is better
vlagent v1.48.0 pass 0.05 29.6 40.5 33 0
fluent-bit v4.2.3 pass 0.21 60.0 115.1 39 0
grafana-agent v0.44.2 pass 0.30 56.3 62.8 54 0
opentelemetry-collector v0.146.1 pass 0.35 97.8 119.9 81 0
alloy v1.13.2 pass 0.36 65.9 80.1 96 0
vector v0.53.0 pass 0.40 92.3 111.2 39 0
promtail v3.5.1 pass 0.44 49.5 54.1 55 0
filebeat v9.3.1 fail: lost too many rows 0.89 216.7 278.9 237 434.0k