The OpenTelemetry Gateway is a bit of a chameleon, capable of acting as both an agent collecting data locally and a central aggregation point.
Let’s see it in action. Imagine you have a service running in a Kubernetes cluster. You want to collect its traces and metrics, process them, and send them to a backend like Jaeger.
# Deployment for the OpenTelemetry Collector (acting as an agent)
apiVersion: apps/v1
kind: Deployment
metadata:
name: my-service-otel-collector
namespace: default
spec:
replicas: 1
selector:
matchLabels:
app: my-service-otel-collector
template:
metadata:
labels:
app: my-service-otel-collector
spec:
containers:
- name: otel-collector
image: otel/opentelemetry-collector-contrib:0.80.0
ports:
- name: jaeger-thrift
containerPort: 14268
protocol: TCP
- name: prometheus
containerPort: 8888
- name: otlp
containerPort: 4317
volumeMounts:
- name: otel-collector-config-volume
mountPath: /etc/otelcol/
volumes:
- name: otel-collector-config-volume
configMap:
name: my-service-otel-collector-config
---
# ConfigMap for the OpenTelemetry Collector agent
apiVersion: v1
kind: ConfigMap
metadata:
name: my-service-otel-collector-config
namespace: default
data:
collector-config.yaml: |
receivers:
otlp:
protocols:
grpc:
http:
jaeger:
protocols:
thrift_compact:
thrift_binary:
thrift_grpc:
processors:
batch:
timeout: 100ms
send_batch_size: 1000
exporters:
logging:
verbosity: detailed
otlp:
endpoint: "otel-gateway.default.svc.cluster.local:4317" # Points to the central gateway
service:
pipelines:
traces:
receivers: [otlp, jaeger]
processors: [batch]
exporters: [logging, otlp]
metrics:
receivers: [otlp]
processors: [batch]
exporters: [logging, otlp]
In this setup, my-service-otel-collector is deployed alongside your application. It receives OTLP or Jaeger traces directly from your application (which would be configured to send data to localhost:4317 or similar if running in the same pod, or my-service-otel-collector.default.svc.cluster.local:4317 if in the same namespace). It then batches this data and forwards it to a central otel-gateway service.
Now, let’s look at the central gateway. This is where the aggregation and potentially more complex processing happens.
# Deployment for the OpenTelemetry Gateway
apiVersion: apps/v1
kind: Deployment
metadata:
name: otel-gateway
namespace: default
spec:
replicas: 2 # High availability for the gateway
selector:
matchLabels:
app: otel-gateway
template:
metadata:
labels:
app: otel-gateway
spec:
containers:
- name: otel-collector
image: otel/opentelemetry-collector-contrib:0.80.0
ports:
- name: otlp
containerPort: 4317
- name: prometheus
containerPort: 8888
volumeMounts:
- name: otel-gateway-config-volume
mountPath: /etc/otelcol/
volumes:
- name: otel-gateway-config-volume
configMap:
name: otel-gateway-config
---
# Service for the OpenTelemetry Gateway
apiVersion: v1
kind: Service
metadata:
name: otel-gateway
namespace: default
spec:
selector:
app: otel-gateway
ports:
- protocol: TCP
port: 4317
targetPort: 4317
name: otlp
type: ClusterIP # Or LoadBalancer if exposing externally
---
# ConfigMap for the OpenTelemetry Gateway
apiVersion: v1
kind: ConfigMap
metadata:
name: otel-gateway-config
namespace: default
data:
gateway-config.yaml: |
receivers:
otlp:
protocols:
grpc:
http:
processors:
batch:
timeout: 5s
send_batch_size: 1000
memory_limiter:
check_interval: 1s
limit_mib: 2000
spike_limit_mib: 500
attributes:
actions:
key: "service.name"
action: "insert"
value: "my-aggregated-service"
resource:
attributes:
- key: "deployment.environment"
value: "production"
action: "insert"
exporters:
logging:
verbosity: normal
jaeger:
endpoint: "http://jaeger-collector.observability.svc.cluster.local:14268/api/traces"
prometheus:
endpoint: "/metrics"
service:
pipelines:
traces:
receivers: [otlp]
processors: [memory_limiter, attributes, batch]
exporters: [logging, jaeger]
metrics:
receivers: [otlp]
processors: [memory_limiter, batch, resource]
exporters: [logging, prometheus]
The otel-gateway deployment runs the collector image. The otel-gateway service makes it discoverable within the cluster. Its configuration (gateway-config.yaml) receives data (via OTLP in this case) from the agent collectors. It then applies further processing like adding attributes, resource information, and a memory limiter before exporting to a backend like Jaeger.
The key distinction is the scope of data collection and processing. The agent-like collector is typically deployed per-application or per-host, focusing on collecting and forwarding raw data. The gateway collector is a central point designed to receive data from multiple agents, aggregate it, perform higher-level processing, and route it to various backends. You can even have multiple tiers of gateways if your scale demands it.
The resource processor is a powerful tool for enriching telemetry data at the gateway level. It allows you to consistently add attributes like deployment.environment or cluster.name to all telemetry signals passing through, providing crucial context for filtering and analysis in your observability backend.