Skip to content
Platform > Operations > OpenTelemetry

Configuration

This page lists every property that controls OpenTelemetry export, and describes how export behaves when the collector is slow, rejects requests, or cannot be reached.

Overview

All export properties go in config/local.properties and are read once, at startup, so every change needs a restart. They are listed in the startup log, log/AmiCore.log, with the values in use; Sending metrics to a collector shows how to check.

A complete configuration for an authenticated collector looks like this:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
# Turn export on
ami.log.stats.otel.enabled=true

# Base URL of the collector; /v1/metrics and /v1/logs are appended
ami.log.stats.otel.endpoint=http://collector.example.com:4318

# The service.name reported with every request
ami.log.stats.otel.service.name=pricing-prod-01

# Extra HTTP headers, as a JSON object
ami.log.stats.otel.headers={"Authorization":"Bearer <token>"}

Keep each comment on a line of its own. A # after a value on the same line becomes part of the value.

Turning export on and choosing the collector

Export is off unless ami.log.stats.otel.enabled is true. With it on and nothing else set, data goes to http://localhost:4318.

ami.log.stats.otel.enabled

Default:
1
ami.log.stats.otel.enabled=false
Description:
  • Set to true to send runtime statistics to the collector at every statistics interval.
  • A value other than true or false turns export off and logs Invalid value for ami.log.stats.otel.enabled; expected true or false. OpenTelemetry export disabled.

ami.log.stats.otel.endpoint

Default:
1
ami.log.stats.otel.endpoint=http://localhost:4318
Description:
  • Base URL of the OTLP/HTTP receiver. Metrics are sent to <endpoint>/v1/metrics and log records to <endpoint>/v1/logs.
  • Must begin with a scheme such as http://. A value without one, such as localhost:4318, sends nothing, never turns export off, and logs only Could not create rest request: with java.net.MalformedURLException:unknown protocol: localhost.
  • A trailing / is accepted by the OpenTelemetry Collector.

Sending to an HTTPS collector

To send over HTTPS, set ami.log.stats.otel.endpoint to an https:// URL. The server checks the collector's certificate against the trust store of its Java runtime, the bundled runtime in the jre/ directory one level above the installation directory.

If that trust store does not trust the collector's certificate, every request fails, and export turns off after 5 failures:

1
OTel export request failed because sun.security.validator.ValidatorException: PKIX path building failed: sun.security.provider.certpath.SunCertPathBuilderException: unable to find valid certification path to requested target (1/5 consecutive failures)

To trust a collector whose certificate is issued by a private certificate authority, add the authority's certificate to the bundled runtime's trust store, then restart. Run from the directory that contains both jre/ and amicore/, replacing the file path with your authority's certificate:

1
.\jre\bin\keytool.exe -importcert -noprompt -alias collector-ca -file C:\certs\collector-ca.crt -keystore jre\lib\security\cacerts -storepass changeit

The command prints Certificate was added to keystore. changeit is the password of the bundled runtime's trust store.

Naming the service

Every request carries a service.name resource attribute. Set it so that each server can be told apart when more than one reports to the same collector.

ami.log.stats.otel.service.name

Default:
Not set.
Description:
  • The value sent as service.name.
  • When not set, the value of f1.logfilename is used, which is AmiCore on a default installation.

The other resource attributes are set automatically; see Resource attributes.

Authenticating to a collector

To send credentials, such as the access token a hosted collector requires, or any other HTTP header, set ami.log.stats.otel.headers to a JSON object of header names and values. The headers are added to every request.

ami.log.stats.otel.headers

Default:
Not set.
Description:
  • Extra HTTP headers as a JSON object, e.g: {"Authorization":"Bearer <token>"}.
  • Any other format, including the HTTP form Authorization: Bearer <token>, is ignored at startup with the warning below. Requests are then sent without the header, and a collector that requires it rejects them.
  • A header named exactly Content-Type replaces the default Content-Type: application/json.

A value that is not a JSON object logs this at startup:

1
Ignoring ami.log.stats.otel.headers: value is not a valid JSON object (expected e.g. {"Authorization":"Basic <base64>"}); StringFormatException

The header value is written to the startup log

The startup log lists every property with its value, including ami.log.stats.otel.headers. A token set there is stored in plain text in log/AmiCore.log. Restrict read access to the installation's log/ directory, and treat old log files as containing the credential.

When the collector is unavailable

A request that fails is not retried: its data is dropped. After too many failed requests in a row, export turns itself off for as long as the server runs, and only a restart turns it back on.

ami.log.stats.otel.max.retries

Default:
1
ami.log.stats.otel.max.retries=5
Description:
  • The number of consecutive failed requests after which export turns off until the next restart. It is not a retry count: no request is ever resent.
  • Each interval makes 2 requests, metrics then log records, so with the default the limit is reached in the third failing interval. That is about 30 seconds after the first failure when the collector answers with an error at once, and longer when requests are slow to fail: 38 seconds when the connection was refused, which took about 2 seconds per request.
  • A successful request resets the count. If the collector accepts log records but rejects metrics, or the reverse, the count never reaches the limit, export stays on, and the rejected data is lost at every interval.

ami.log.stats.otel.request.timeout

Default:
1
ami.log.stats.otel.request.timeout=10000
Description:
  • How long, in milliseconds, each request waits to connect and to receive a response before it fails with the request timed out.

Requests are sent from the thread that also writes most of the statistics file, so a collector that is slow, unresponsive, or unreachable delays those statistics too, until export turns off. Process statistics were recorded every 19 seconds instead of every 15 both with a 2-second timeout and a collector that did not respond, and with nothing listening at the endpoint.

When export turns off, the log shows:

1
OTel export disabled after 5 consecutive failed requests, most recently because <reason>; no further OpenTelemetry export attempts will be made for the remainder of this process.

Each earlier failure logs OTel export request failed because <reason> (<n>/5 consecutive failures). Troubleshooting lists the reasons and what to do about each.

When the server is stopped normally, it sends one last request of metrics and log records before it exits.

How often data is sent

Export has no interval of its own: it sends at every statistics interval, set by ami.amilog.stats.period. These properties belong to the statistics file and affect export as follows:

Property Default Effect on export
ami.amilog.stats.period 15 SECONDS One request of metrics and one of log records per period
ami.amilog.stats.os.period 1 MINUTE How often OperatingSystem.* and FileSystem.* statistics are collected
ami.center.log.stats.period.millis 15000 How often the Center's AmiCenter* statistics are collected

Statistics collected between intervals are held and sent with the next request.

What this does not support

  • Environment variables: the standard OTEL_* variables, such as OTEL_EXPORTER_OTLP_ENDPOINT, are not read. Use the properties on this page.
  • Choosing which metrics are sent: there is no property to include, exclude, or rename metrics.