# Victoria Metrics

[Victoria Metrics] is a powerful, scalable time-series database compatible
with Prometheus and its ecosystem of metrics exporters.


## Clustered Deployment

*Victoria Metrics* can run in a high-availability cluster, with the various
functions of the TSDB split into independently-scalable processes:

* `vmstorage`: Stores time series data.
* `vminsert`: Ingests metrics in various formats (e.g. Prometheus) and sends
  them to one or more `vmstorage` nodes.
* `vmselect`: Performs metrics queries, retrieving results from one or more
  `vmstorage` nodes.

The `vmstorage` processes are managed by a StatefulSet with a volume claim
template for persistent storage.  The number of replicas in the StatefulSet
must be $2n-1$ where $n$ is the value of the `replicationFactor` setting for
`vminsert`.

`vminsert` and `vmselect` processes are stateless and thus managed by a
Deployment.  There should be at least 2 replicas of each of these, so that
restarts, etc. can be performed without any downtime.


## vmagent

In a typical Victoria Metrics ecosystem, collecting metrics is handled
separately from the TSDB.  The [vmagent] process handles scraping and receiving
metrics and passing them to `vminsert`.  `vmagent` can cache received metrics
locally, in case no `vminsert` process is available, so it requires persistent
storage and is therefore managed by a StatefulSet.  Because there are multiple
`vmagent` processes scraping the same targets, the `vminsert` and `vmstorage`
processes MUST have the `dedup.minScrapeInterval` setting set to match the
`vmagent` scrape interval.  Jobs with scrape intervals longer than the
default will unfortunately have duplicate data points.


## Blackbox Exporter

Many applications and web sites are monitored via the [Blackbox Exporter],
which makes arbitrary HTTP, TCP, ICMP, etc. requests and reports Prometheus
metrics about them.  This is a stateless process, managed by a Deployment.


## vmalert

Victoria Metrics has a separate process for alerting, [vmalert].  This process
periodically executes the queries defined in its alerting rules and creates
alerts for matching results.  Alerts are stored in the Victoria Metrics TSDB.
Rules are defined in a YAML document, managed by a ConfigMap.  Notifications
are sent to Alertmanager.


## Alertmanager

[Alertmanager] receives notifications from `vmalert` and sends e.g. email
messages.  Multiple instances can be run in a cluster; each node needs to know
the host and port of every node in the cluster.


[Victoria Metrics]: https://new.docs.victoriametrics.com/
[vmagent]: https://new.docs.victoriametrics.com/vmagent/
[Blackbox Exporter]: https://github.com/prometheus/blackbox_exporter
[vmalert]: https://new.docs.victoriametrics.com/vmalert/
[Alertmanager]: https://prometheus.io/docs/alerting/latest/alertmanager/