OTLP Environment variables do not match specification

[jsuereth]

Currently, OTLP environment variables (OTEL_EXPORTER_OTLP_GRPC_ENDPOINT) does not match the specification.

Copied relevant sections:

Configuration Options

The following configuration options MUST be available to configure the OTLP exporter. Each configuration option MUST be overridable by a signal specific option.

Configuration Option	Description	Default	Env variable
Endpoint	Target to which the exporter is going to send spans or metrics. The endpoint MUST be a valid URL with scheme (http or https) and host, and MAY contain a port and path. A scheme of https indicates a secure connection. When using OTEL_EXPORTER_ENDPOINT with OTLP/HTTP, exporters SHOULD follow the collector convention of appending the version and signal to the path (e.g. v1/traces or v1/metrics). The per-signal endpoint configuration options take precedence and can be used to override this behavior. See the [OTLP Specification][otlphttp-req] for more details.	https://localhost:4317	OTEL_EXPORTER_OTLP_ENDPOINT OTEL_EXPORTER_OTLP_TRACES_ENDPOINT OTEL_EXPORTER_OTLP_METRICS_ENDPOINT
Certificate File	Path to certificate file for TLS credentials of gRPC client. Should only be used for a secure connection.	n/a	OTEL_EXPORTER_OTLP_CERTIFICATE OTEL_EXPORTER_OTLP_TRACES_CERTIFICATE OTEL_EXPORTER_OTLP_METRICS_CERTIFICATE
Headers	Key-value pairs to be used as headers associated with gRPC or HTTP requests. See Specifying headers for more details.	n/a	OTEL_EXPORTER_OTLP_HEADERS OTEL_EXPORTER_OTLP_TRACES_HEADERS OTEL_EXPORTER_OTLP_METRICS_HEADERS
Compression	Compression key for supported compression types. Supported compression: gzip	No value	OTEL_EXPORTER_OTLP_COMPRESSION OTEL_EXPORTER_OTLP_TRACES_COMPRESSION OTEL_EXPORTER_OTLP_METRICS_COMPRESSION
Timeout	Max waiting time for the backend to process each spans or metrics batch.	10s	OTEL_EXPORTER_OTLP_TIMEOUT OTEL_EXPORTER_OTLP_TRACES_TIMEOUT OTEL_EXPORTER_OTLP_METRICS_TIMEOUT

Supported values for OTEL_EXPORTER_OTLP_*COMPRESSION options:

• If the value is missing, then compression is disabled.
• gzip is the only specified compression method for now. Other options MAY be supported by language SDKs and should be documented for each particular language.

Example 1

The following configuration sends all signals to the same collector:

export OTEL_EXPORTER_OTLP_ENDPOINT=http://collector:4317

Example 2

Traces and metrics are sent to different collectors:

export OTEL_EXPORTER_OTLP_TRACES_ENDPOINT=http://collector:4317

export OTEL_EXPORTER_OTLP_METRICS_ENDPOINT=https://collector.example.com/v1/metrics

Specify Protocol

Currently, OTLP has more than one transport protocol it can support, e.g.
grpc, http/json, http/protobuf. As of 1.0 of the specification, there
is no specified default, or configuration via environment variables. We
reserve the following environment variables for configuration of protocols in
the future:

• OTEL_EXPORTER_OTLP_PROTOCOL
• OTEL_EXPORTER_OTLP_TRACES_PROTOCOL
• OTEL_EXPORTER_OTLP_METRICS_PROTOCOL

SDKs have an unspecified default, if no configuration is provided.

Specifying headers via environment variables

The OTEL_EXPORTER_OTLP_HEADERS, OTEL_EXPORTER_OTLP_TRACES_HEADERS, OTEL_EXPORTER_OTLP_METRICS_HEADERS environment variables will contain a list of key value pairs, and these are expected to be represented in a format matching to the W3C Correlation-Context, except that additional semi-colon delimited metadata is not supported, i.e.: key1=value1,key2=value2. All attribute values MUST be considered strings.

[lalitb]

Thanks for copying the relevant sections. This would mean that there can be only one OTLP exporter configured at given time ( as specified by OTEL_EXPORTER_OTLP_PROTOCOL ), and so all of them share the same env variables names.

[lalitb]

And to add further - otel-cpp doesn't support environment variables as defined in the specs, this is also indicated clearly in the compliance matrix :
https://github.com/open-telemetry/opentelemetry-specification/blob/main/spec-compliance-matrix.md#environment-variables

Feature	Go	Java	JS	Python	Ruby	Erlang	PHP	Rust	C++	.NET	Swift

OTEL_EXPORTER_OTLP_*	+	+		+	+	-	-	+	-	-	-

We would like to take exception for supporting the env-variables as per the specs, as the support can be added latter without breaking the API. The currently defined environment variables are proprietary to otel-cpp implementation and can co-exist along with specs env-variables once they are supported. @jsuereth - let us know if that would be fine ?

[lalitb]

@jsuereth - Sorry for multiple comments :) This was discussed in today's meeting. And as the env-var used in code are not same as required by specs, we planned to document them explicitly ( done in #974 ). Also mentioned there that both would be supported simultaneously in future.

[jsuereth]

Do you have users of these env variables now?

The specification denotes:

The following configuration options MUST be available to configure the OTLP exporter.

If you didn't have any environment variable configuration, that would make sense. However if you do provide environment variable configuration, then it should be consistent with otel. This is a major issue for consistency across the SDKs in OTel.

At a minimum, I think you should:

• Support OTEL_EXPORTER_OTLP_TRACES_ENDPOINT. This should ALWAYS override the existing OTEL_EXPORTER_OTLP_GRPC_ENDPOINT + OTEL_EXPORTER_OTLP_GRPC_SSL_ENABLE settings.
• Support OTEL_EXPORTER_OTLP_TRACES_CERTIFICATE which would override OTEL_EXPORTER_OTLP_GRPC_SSL_CERTIFICATE

What's the reason for (continued) divergence right now?

[lalitb]

Do you have users of these env variables now?

We don't know for sure, but we are trying best avoid doing breaking changes while in release candidate.

If you didn't have any environment variable configuration, that would make sense. However if you do provide environment variable configuration, then it should be consistent with otel. This is a major issue for consistency across the SDKs in OTel.

Good point. Will add support for OTEL_EXPORTER_OTLP_TRACES_ENDPOINT and OTEL_EXPORTER_OTLP_TRACES_CERTIFICATE as suggested by you.

[lalitb]

@jsuereth - Can you review this PR #974 for the changes for this issue. As mentioned as comment in PR:

It was creating more noise in code to support both legacy and compliant environment variables, and would further increase in future once we start supporting OTEL_EXPORTER_OTLP_[TRACES_]PROTOCOL. Keeping that in mind, have removed legacy environment variables. This would be breaking change ( hopefully won't affect lots of users ), and we will document this in release description.

Another alternate suggestion from @ThomsonTan - instead of any code changes for now, we can enable the NO_ENV macro, which basically for now disable all environment variables ( used here :

opentelemetry-cpp/sdk/include/opentelemetry/sdk/common/env_variables.h

Lines 16 to 18 in c8b35ff

	inline const std::string GetEnvironmentVariable(const char *env_var_name)
	{
	#if !defined(NO_GETENV)

)
, and then enable it once we are compliant with the specs