Sign up
View stargazersSign up
Commits

README.md

Connectors

A connector is both an exporter and receiver. As the name suggests a Connector connects two pipelines: it emits data as an exporter at the end of one pipeline and consumes data as a receiver at the start of another pipeline. It may consume and emit data of the same data type, or of different data types. A connector may generate and emit data to summarize the consumed data, or it may simply replicate or route data.

Supported Data Types

Each type of connector is designed to work with one or more pairs of data types and may only be used to connect pipelines accordingly. (Recall that every pipeline is associated with a single data type, either traces, metrics, or logs.)

For example, the count connector counts traces, metrics, and logs, and reports the counts as a metric. Therefore, it may be used to connect the following types of pipelines.

Exporter Pipeline TypeReceiver Pipeline Type
tracesmetrics
metricsmetrics
logsmetrics

Another example, the router connector, is useful for routing data onto the appropriate pipeline so that it may be processed in distinct ways and/or exported to an appropriate backend. It does not alter the data it consumes in any ways, nor does it produce any additional data. Therefore, it may be used to connect the following types of pipelines.

Exporter Pipeline TypeReceiver Pipeline Type
tracestraces
metricsmetrics
logslogs

Configuration

Declaration

Connectors are defined within a dedicated connectors section at the top level of the collector config.

The count connector may be used with default settings.

yaml
1receivers:
2 foo:
3exporters:
4 bar:
5connectors:
6 count:
7 router:

Usage

Recall that a connector is an exporter and a receiver and that each connector MUST be used as both, in separate pipelines.

yaml
1receivers:
2 foo:
3exporters:
4 bar:
5connectors:
6 count:
7service:
8 pipelines:
9 traces:
10 receivers: [foo]
11 exporters: [count]
12 metrics:
13 receivers: [count]
14 exporters: [bar]

Connectors can be used alongside traditional exporters.

yaml
1receivers:
2 foo:
3exporters:
4 bar/traces_backend:
5 bar/metrics_backend:
6connectors:
7 count:
8service:
9 pipelines:
10 traces:
11 receivers: [foo]
12 exporters: [bar/traces_backend, count]
13 metrics:
14 receivers: [count]
15 exporters: [bar/metrics_backend]

Connectors can be used alongside traditional receivers.

yaml
1receivers:
2 foo/traces:
3 foo/metrics:
4exporters:
5 bar:
6connectors:
7 count:
8service:
9 pipelines:
10 traces:
11 receivers: [foo/traces]
12 exporters: [count]
13 metrics:
14 receivers: [foo/metrics, count]
15 exporters: [bar]

A connector can be an exporter in multiple pipelines.

yaml
1receivers:
2 foo/traces:
3 foo/metrics:
4 foo/logs:
5exporters:
6 bar/traces_backend:
7 bar/metrics_backend:
8 bar/logs_backend:
9connectors:
10 count:
11service:
12 pipelines:
13 traces:
14 receivers: [foo/traces]
15 exporters: [bar/traces_backend, count]
16 metrics:
17 receivers: [foo/metrics]
18 exporters: [bar/metrics_backend, count]
19 logs:
20 receivers: [foo/logs]
21 exporters: [bar/logs_backend, count]
22 metrics/counts:
23 receivers: [count]
24 exporters: [bar/metrics_backend]

A connector can be a receiver in multiple pipelines.

yaml
1receivers:
2 foo/traces:
3 foo/metrics:
4exporters:
5 bar/traces_backend:
6 bar/metrics_backend:
7 bar/metrics_backend/2:
8connectors:
9 count:
10service:
11 pipelines:
12 traces:
13 receivers: [foo/traces]
14 exporters: [bar/traces_backend, count]
15 metrics:
16 receivers: [count]
17 exporters: [bar/metrics_backend]
18 metrics/2:
19 receivers: [count]
20 exporters: [bar/metrics_backend/2]

Multiple connectors can be used in sequence.

yaml
1receivers:
2 foo:
3exporters:
4 bar:
5connectors:
6 count:
7 count/the_counts:
8service:
9 pipelines:
10 traces:
11 receivers: [foo]
12 exporters: [count]
13 metrics:
14 receivers: [count]
15 exporters: [bar/metrics_backend, count/the_counts]
16 metrics/count_the_counts:
17 receivers: [count/the_counts]
18 exporters: [bar]

A connector can only be used in a pair of pipelines when it supports the combination of Exporter Pipeline Type and Receiver Pipeline Type.

yaml
1receivers:
2 foo:
3exporters:
4 bar:
5connectors:
6 count:
7service:
8 pipelines:
9 traces:
10 receivers: [foo]
11 exporters: [count]
12 logs:
13 receivers: [count] # Invalid. The count connector does not support traces -> logs.
14 exporters: [bar]

Exporter Pipeline Type

The type of pipeline in which a connector is used as an exporter.

Receiver Pipeline Type

The type of pipeline in which the connector is used as a receiver.