Config File

Blip configuration is specified in a single config file that has three sections:

---
# Server config

# Monitor defaults

# Monitors
monitors: []

Server config

Server config is top-level sections that configure the Blip binary (blip), which is called the server because it runs the monitors and has an external API. Server config is optional.

Monitor defaults

Monitor defaults are top-level sections that configure default values for each monitor: hostname, username, and so on. Monitor defaults are commonly configured to avoid repeating the same config for each monitor.

Monitors

Monitors are a list of MySQL instances to monitor. Each monitor in the list inherits config from the monitor defaults. For example, if a monitor does not explicitly set the MySQL hostname, it inherits the value from monitor defaults if set.

The simplest possible Blip config for monitoring a single MySQL instance looks like:

monitors:
  - hostname: db.local
    username: blip
    password: "foo"

That uses the default plan to collect metrics. You will likely write your own custom plan, which is configured like:

plans:
  files: my-plan.yaml
monitors:
  - hostname: db.local
    username: blip
    password: "bar"

In this case, the monitor inherits the only plan. See Plans / Loading for details on how plans are loaded and shared.

To reference sections, subsections, and specific user-configurable variables within those sections:

SECTION

Any top-level section. For example: api, plans, and monitors.

SECTION[.SUB...].VAR

A user-configurable variable in the SECTION with optional subsections (SUB). For example: mysql.username (section = mysql, variable = username), or monitors.heartbeat.freq (section = monitors, subsection = heartbeat, variable = freq).

Note the difference between heartbeat and monitors.heartbeat: the former is the top-level monitor default; the latter is a monitor config that inherts the monitor defaults, if any.

In the Blip documentation outside this page, config file references begin with config.. For example, config.api refers to the api server config section of the Blip config file.

disable and disable-auto-... are used to disable features. There are no “enable” prefixes or variables. Instead, some features of either off or auto by default. If off by default, the feature is enabled by specifying a variable noted in the docs. For example, heartbeat is off by default and enabled when heartbeat.freq is specified. If auto by default, the feature is disabled by specifying disable-auto-FEATURE: true, where FEATURE is the feature name. For example, aws.disable-auto-region: true to disable auto-detecting the AWS region.

Blip uses lowercase-kebab-case for all sections and variable names.

The api section configures the Blip API.

api:
  bind: "127.0.0.1:7522"
  disable: false

The bind variable sets the interface address and port that the API listens on.

The disable variable disables the Blip API.

The monitor-loader section configures how Blip finds and loads MySQL instances.

monitor-loader:
  aws:
    regions: []
  files: []
  local:
    disable-auto: false
    disable-auto-root: false
  stop-loss: ""

The aws subsection of the monitor-loader section configure built-in support for loading Amazon RDS instances. By default, this feature is disabled. To enable, specify regions.

The regions variable sets which AWS regions to query for RDS instances. If auto is specified, Blip queries EC2 IMDS, which only works if Blip is running on an EC2 instance with an EC2 instance profile that allows rds:DescribeDBInstances.

The files variable specifies YAML files to load monitors from. Each file must have a monitors section, like:

---
monitors:
  - hostname: db.local

File paths are relative to the current working directory of blip.

The local subsection has only two variables:

disable-auto: true disable-auto-root: true

The stop-loss variable enables the stop-lost feature.

Monitor defaults are top-level sections that set default values for monitors that do not set an explicit value. Monitor defaults are useful when you have several MySQL instances to monitor and the configuration only differs by basic connection parameters, like hostname or socket.

For example, imagine that you have 10 monitors all with the same username and password. Instead of setting username and password in all 10 monitors, you can set these variables once in the top-level mysql section:

mysql:
  username: "defaultUser"
  password: "defaultPass"
monitors:
  - hostname: db1
  # ...
  - hostname: db10

The default username and password are applied to the 10 monitors because none of them explicitly set these variables. If a monitor explicitly sets one of the variables, then its explicit value is used instead of the default value.

Monitor defaults are convenient, but explicit monitor configuration is more clear, so use monitor defaults sparingly. The intended use case is for variables that must be consistent for all monitors. For example, if Blip monitors Amazon RDS instances in region us-east-1, then setting monitor default aws.region: "us-east-1" makes sense.

The aws section configures Amazon RDS for MySQL.

aws:
  disable-auto-region: false
  disable-auto-tls: false
  iam-auth: false
  password-secret: ""
  region: ""

The disable-auto-region variable enables/disables automatic detection of the AWS region.

The disable-auto-tls variables disables automatic use of the Amazon RDS certificate authority (CA). By default, Blip uses the Amazon RDS CA-2019 certificate, which is built-in (you don’t need to configure anything). See Cloud / AWS / TLS for details.

The iam-auth variable enables IAM database authentication. When enabled, an IAM authentication token is generated by Blip and used as the password.

See Cloud / AWS / IAM Authentication for details.

The password-secret variables sets the AWS Secrets Manager ARN that contains the MySQL user password.

The region variable sets the AWS region used to create an AWS configuration (which includes the AWS credentials).

See Cloud / AWS / IAM Authentication for details.

The exporter section configure Blip to emulate Prometheus mysqld_exporter.

exporter:
  flags:
    web.listen-address: "127.0.0.1:9104"
    web.telemetry-path: "/metrics"
  mode: ""

The flag variable is a key-value map of strings for certain Prometheus mysqld_exporter flags:

• web.listen-address (default: 127.0.0.1:9104)
• web.telemetry-path (default: /metrics)

The mode variables enables Prometheus emulation. When set to dual, Blip runs normally and emulates Prometheus. When set to legacy, Blip runs only emulates Prometheus. The feature is disabled by default.

The plan variables specifies which plan to load. The plan must have only 1 level. See Prometheus emulation for details.

The heartbeat section configures the Blip heartbeat.

heartbeat:
  freq: ""
  role: ""
  source-id: ""
  table: blip.heartbeat

The freq variable enables Blip heartbeats at the specified frequency. A frequency of “1s” or “2s” is suggested because heartbeat frequency does not determine replication lag accuracy or reporting. See Heartbeat > Accuracy for details.

See repl.lag metric collector for reporting replication lag.

To disable heartbeat, remove freq or set to an empty string (zero is not a valid value).

The role variable sets the role that the monitor reports in the heartbeat table. For example, this might be a region like “us-east-1”.

See Heartbeat > Topology to learn how role and source-id are used.

The source-id variable sets the source ID that the monitor reports in the heartbeat table. This overrides the default value, which is often necessary in the cloud where MySQL instances do not have user-defined hostnames, especially with respect to replication.

See Heartbeat > Topology to learn how role and source-id are used.

The table variable sets the Blip heartbeat table (where heartbeat are written). The default database is blip if the table name is not database-qualified like db.heartbeat.

The table must already exist; Blip does not create the table. See Heartbeat > Table for details.

The mysql section configures how to connect to MySQL.

mysql:
  hostname: ""
  mycnf: ""
  password: ""
  password-file: ""
  socket: ""
  timeout-connect: "10s"
  username: "blip"

As monitor defaults, this section is specified as shown above: top-level with the variables specified under mysql:. For each monitor in the monitors section, these variables are top-level (omit the mysql: header). For example:

monitors:
  - hostname: ""
    mycnf: ""
    # Other mysql section variables

These are the only variables the become top-level in monitors.

The hostname variable sets the MySQL hostname.

The mycnf variable sets a my.cnf file to read.

Blip reads the [client] section of the my.cnf file:

The username variable sets the MySQL username.

The password variable sets the MySQL password.

The password-file variable sets a file from which Blip reads the MySQL password.

The socket variable sets the MySQL socket.

The timeout-connect variable sets the connection timeout.

The plans section configures the source of plans.

plans:
  files:
    - plan1.yaml
    - plan2.yaml
  table: "blip.plans"
  monitor: {}
  change:
    # See below

The change subsection of the plan section configures plan changing based on the state of MySQL.

plans:
  change:
    offline:
      after: ""
      plan: ""
    standby:
      after: ""
      plan: ""
    read-only:
      after: ""
      plan: ""
    active:
      after: ""
      plan: ""

Each of the four sections—offline, standby, read-only, and active—have the same two variables:

The after variable sets how long before the state takes effect.

The plan variable sets the plan to load when the state takes effect.

The disable-default-plans variable enables/disables default plans.

The files variable is a list of file names from which to load plans. File paths are relative to the current working directory of blip. See Plans / Loading for details.

The monitor variable configures the MySQL instance from which the table is loaded.

The table variable is the MySQL table name from which plans are loaded. See Plans / Table.

The sinks section configures built-in metric sinks and custom metrics sinks. This section is a map of maps:

sinks:
  sink-foo:
    # Options for sink-foo:
    key1: value1
  sink-bar:
    # Options for sink-bar:
    key1: value1

Keys are sink names. Values for each are a key-value map (of strings) as options for the named sink.

The key-value options for each sink are sink-specific and passed directly to the sink. The sink validates the options. For built-in sinks, see Sinks for each one’s options. For custom sinks, the options are whatever you program the custom sink to accept.

Blip does not distinguish between built-in and custom sinks. The built-in sinks are merely sink plugins automatically registered on startup; they implement the same interface as custom sinks.

If no sinks are specified, Blip use the sink defined by package variable sink.Default, which is log.

The tags section sets user-defined key-value pairs (as strings) that are passed to each sink. For example (using interpolation):

tags:
  env: ${ENVIRONMENT:-dev}
  dc: ${DATACENTER:-local}
  hostname: %{monitor.hostname}

Blip calls these “tags”, but each sink might have a different term for the same concept. For example, with SignalFx these are called “dimensions”. But the concept is the same: metadata (usually string key-value pairs) attached to metrics that describe or annotate the metrics for grouping, aggregation, or filtering when display in graphs/charts.

The default sinks automatically send all tags with metrics. For example, the signalfx sink sends all tags as SignalFx dimensions.

The tls section configures TLS certificates.

tls:
  ca: ""
  cert: ""
  key: ""
  disable: false
  skip-verify: false

You can specify only tls.ca, or tls.cert and tls.key, or all three; any other combination is invalid.

By default, Blip does not use TLS for MySQL connections except when using AWS; see section aws or AWS.

The ca variables sets the certificate authority file.

The cert variables sets the public certificate file.

The key variables sets the private key file.

The disable variable disables TLS even if configured. ssl-mode=DISABLED in a mycnf file also disables TLS.

Do not verify the server address (MySQL hostname).

The monitors section is a list of MySQL instances to monitor. Each instance is a YAML dictionary containing any of the monitor default sections with one exception: mysql variables are top-level in a monitor. The example below shows two different MySQL instances to monitor.

monitors:

  - hostname: db1.local
    username: metrics
    password-file: "/secret/db-password"
    heartbeat:
      freq: 1s

  - mycnf: "/secret/my.cnf"
    exporter:
      mode: legacy

The first MySQL instance is configured in lines 3-7. (Note the single, leading hyphen on line 3 that denotes an item in a YAML list.) The first three variables—hostname, username, and password-file—are mysql variables but in a monitor they are top-level. But all other sections, like heartbeat and its variable freq, are exactly the same in a monitor.

The second MySQL instance is configured in lines 9-11. Variable mycnf belongs to section mysql, but again: in a monitor, mysql variables are top-level. Section exporter is exactly the same in a monitor.

Refer to Monitor Defaults for configuring MySQL instances, and remember: mysql variables are top-level in a monitor (omit mysql: and include the variables directly).

Monitors have three variables that only appear in monitors: id, meta, and plan.

The id variable uniquely identifies the MySQL instance in Blip.

Every monitor has a unique ID that, by default, Blip sets automatically. You can set monitor IDs manually, but it’s better to let Blip set them automatically to avoid duplicates (which causes a fatal error).

Blip uses monitor IDs to track and report each MySQL instance in its own output and API.

Blip does not use monitor IDs to identify MySQL instances for reporting metrics, but you can use them if you want. For example:

monitors:
  - id: db1
    hostname: db1.local
    tags:
      monitorId: %{monitor.id}

Since tags are passed to sinks (which report metrics), all sinks will receive the monitor ID. (Sinks receive the monitor ID at the code-level too, so technically this example is not necessary.)

Monitor IDs are not guaranteed to be stable—they might change between Blip versions. Therefore, do not rely on them outside of Blip for truly stable, unique MySQL instance identification.

The meta variable is a map of key-value strings for user-defined monitor metadata.

No part of Blip uses or requires monitor metadata. Unlike tags and metric metadata, Blip does not copy or send monitor metadata. This makes monitor metadata useful for advanced or automated configurations because it allows you to add custom configuration and reference it with interpolation.

Monitor metadata is optional. When useful, the Blip documentation will shown to use it.

The plan variable selects the shared plan for the monitor to use if change is not configured. The default (no value) selects a plan according to plan precedence.