How to monitor / discover more docker containers with prometheus
I am using prometheus, node-exporter and grafana locally with docker-compose to see how metrics can be displayed. Is it possible to add a container as a service (such as apache \ nginx ) in the docker-compose.yml and have the service discovered so that its metrics are also taken into account? The documentation is not too helpful in explaining this.
If everything needs to run locally (ie, both your Prometheus and the monitored entities are on the same machine), I would not bother with proper auto-discovery. Rather, I'd just have a static config file in Prometheus listing each of the services it needs to monitor. Each of these services would be running in a docker container, each with its own metric-exporting app (like the node-exporter or Telegraf). Yes, this means you'll need to extend your nginx-docker with telegraf (or other tool) to have a metric-emitting docker container. that way, Prometheus would be scraping n urls for n services. You could probably run all them in --net=host mode, but then each service would need to export its metrics in a different port. Keep in mind that when you gather metrics from within a docker container, some of metrics might be wrong, since the container has a limited view of the system. So OS/HW metrics might be off. But if you're interested in custom/app-level metrics, that should be fine. so, to sum: - each docker needs to run its own exporter (maybe exporting in a different port) - prometheus should be configured to know where to look for each of these services. if you really want auto-discovery, look into Consul and how it integrates with Prometheus. I'm told you can actually run consul-server AND consul-clients on the same machine, and with it you can setup Prometheus to read the list of services from Consul (meaning, have auto-discovery) without having to explicitly list each of them in the config file. Using zookeeper should also work: your containers would register with zookeeper and Prometheus would be aware of them from zk (disclaimer: I've never tried this particular configuration).
Files created by manage.py commands not chown'd from root user
Is it possible to create an intermediate image within docker-compose?
docker-compose service listen on specified interface
Does Docker-Compose allow overwriting of ports when using extends?
Jhipster Docker-compose conflict
Build multiple images with Docker Compose?
JHipster Cassandra Docker Build
Defining project name inside docker-compose configuration
Docker-compose named mounted volume
cannot ping linked docker container service
How to do variable substitution in env_file?
How to create stack in tutum with docker-compose.yml file?
ecs-cli docker compose version
dynamic link update in docker-compose
How to start arbitrarily many identical containers with docker-compose?
How to stop docker-compose run command