Monitoring with Prometheus and Grafana

In the Uyuni Web UI, open the details page of the system where Prometheus is to be installed, and navigate to the Formulas tab.

Check the Prometheus checkbox to enable monitoring formulas, and click Save.

Navigate to the Prometheus tab in the top menu.

In the Uyuni Server section, enter valid Uyuni API credentials. Make sure that the credentials you have entered allow access to the set of systems you want to monitor.

Customize any other configuration options according to your needs.

Click Save Formula.

Apply the highstate and confirm that it completes successfully.

Check that the Prometheus interface loads correctly. In your browser, navigate to the URL of the server where Prometheus is installed, on port 9090 (for example, http://example.com:9090).

On the monitoring server, install the golang-github-prometheus-prometheus package:

zypper in golang-github-prometheus-prometheus

Enable the Prometheus service:

systemctl enable --now prometheus

Check that the Prometheus interface loads correctly. In your browser, navigate to the URL of the server where Prometheus is installed, on port 9090 (for example, http://example.com:9090).

Open the configuration file at /etc/prometheus/prometheus.yml and add this configuration information. Replace server.url with your Uyuni server URL and adjust username and password fields to match your Uyuni credentials.

# {productname} self-health metrics
scrape_configs:
- job_name: 'mgr-server'
  static_configs:
    - targets:
      - 'server.url:9100'  # Node exporter
      - 'server.url:9187'  # PostgreSQL exporter
      - 'server.url:5556'  # JMX exporter (Tomcat)
      - 'server.url:5557'  # JMX exporter (Taskomatic)
      - 'server.url:9800'  # Taskomatic
    - targets:
      - 'server.url:80'    # Message queue
      labels:
        __metrics_path__: /rhn/metrics

# Managed systems metrics:
- job_name: 'mgr-clients'
  uyuni_sd_configs:
   - server: "http://server.url"
     username: "admin"
     password: "admin"
  relabel_configs:
   - source_labels: [__meta_uyuni_exporter]
     target_label: exporter
   - source_labels: [__address__]
     replacement: "No group"
     target_label: groups
   - source_labels: [__meta_uyuni_groups]
     regex: (.+)
     target_label: groups
   - source_labels: [__meta_uyuni_minion_hostname]
     target_label: hostname
   - source_labels: [__meta_uyuni_primary_fqdn]
     regex: (.+)
     target_label: hostname
   - source_labels: [hostname, __address__]
     regex: (.*);.*:(.*)
     replacement: ${1}:${2}
     target_label: __address__
   - source_labels: [__meta_uyuni_metrics_path]
     regex: (.+)
     target_label: __metrics_path__
   - source_labels: [__meta_uyuni_proxy_module]
     target_label: __param_module
   - source_labels: [__meta_uyuni_scheme]
     target_label: __scheme__

Save the configuration file.

Restart the Prometheus service:

systemctl restart prometheus

In the Uyuni Web UI, open the details page of the system where Grafana is to be installed, and navigate to the Formulas tab.

Check the Grafana checkbox to enable monitoring formulas, and click Save.

Navigate to the Grafana tab in the top menu.

In the Enable and configure Grafana section, enter the admin credentials you want to use to log in Grafana.

On the Datasources section, make sure that the Prometheus URL field points to the system where Prometheus is running.

Customize any other configuration options according to your needs.

Click Save Formula.

Apply the highstate and confirm that it completes successfully.

Check that the Grafana interface is loading correctly. In your browser, navigate to the URL of the server where Grafana is installed, on port 3000 (for example, http://example.com:3000).

Install the grafana package:

zypper in grafana

Enable the Grafana service:

systemctl enable --now grafana-server

In your browser, navigate to the URL of the server where Grafana is installed, on port 3000 (for example, http://example.com:3000).

On the login page, enter admin for username and password.

Click Log in. If login is successful, then you will see a prompt to change the password.

Click OK on the prompt, then change your password.

Move your cursor to the cog icon on the side menu which will show the configuration options.

Click Data sources.

Click Add data source to see a list of all supported data sources.

Choose the Prometheus data source.

Make sure to specify the correct URL of the Prometheus server.

Click Save & test.

To import a dashboard click the + icon in the side menu, and then click Import.

For Uyuni server overview load the dashboard ID: 17569.

For Uyuni clients overview load the dashboard ID: 17570.

In the Uyuni Web UI, navigate to Admin  Manager Configuration  Monitoring.

Click Enable services.

Restart Tomcat and Taskomatic.

Navigate to the URL of your Prometheus server, on port 9090 (for example, http://example.com:9090)

In the Prometheus UI, navigate to Status  Targets and confirm that all the endpoints on the mgr-server group are up.

If you have also installed Grafana with the Web UI, the server insights are visible on the Uyuni Server dashboard, in the Admin  Manager Configuration  Monitoring.

In the Uyuni Web UI, open the details page of the client to be monitored, and navigate to the Formulas tab.

Check the Enabled checkbox on the Prometheus Exporters formula.

Click Save.

Navigate to the Formulas  Prometheus Exporters tab.

Select the exporters you want to enable and customize arguments according to your needs. The Address field accepts either a port number preceded by a colon (:9100), or a fully resolvable address (example:9100).

Click Save Formula.

Apply the highstate.

In the Uyuni Web UI, open the details page of the system to be monitored, and navigate to the Formulas tab.

Check the Prometheus Exporters checkbox to enable the exporters formula, and click Save.

Navigate to the Prometheus Exporters tab in the top menu.

Check the Enable reverse proxy option, and enter a valid reverse proxy port number. For example, 9999.

Customize the other exporters according to your needs.

Click Save Formula.

Apply the highstate and confirm that it completes successfully.

On the Uyuni Server, at the command prompt, run following command:

rhn-ssl-tool --gen-server --dir="/root/ssl-build" --set-country="COUNTRY" \
--set-state="STATE" --set-city="CITY" --set-org="ORGANIZATION" \
--set-org-unit="ORGANIZATION UNIT" --set-email="name@example.com" \
--set-hostname="minion.example.com" --set-cname="minion.example.com" --no-rpm

Copy server.crt and server.key files to the Salt minion and provide read access for prometheus user.