Metrics¶
The Basics¶
Resources within OpenStack create numerous metrics that can be used to monitor the state of these resources. These metrics are stored in a scale-out metrics platform, providing for high-speed, scalable storage for massive metrics collection.
In the Genesis Public Cloud, most metrics are captured every 30 seconds which are then aggregated over time. 30 second measures are kept for 35 days, 5 minute measures are kept for 105 days, and 1 hour metrics are kept for 750 days.
The storage hierarchy is:
- Resources - OpenStack objects such as servers, networks, routers, load balancers, etc.
- Metrics - Object metrics such as a server’s cpu usage, memory usage, and vcpu count
- Measures - Metric values measured every 30 seconds
Resources¶
The resource ID in the metrics database is the same as the OpenStack resource ID. For example, if a server’s ID is 9d2810dd-63d2-4bef-a19f-2e38e8d4f925, the metric’s resource ID is also 9d2810dd-63d2-4bef-a19f-2e38e8d4f925.
To view a list of all resources in a project, run the following:
openstack metric resource list
The types of resources that are collected can be found using:
openstack metric resource-type list
The most common resource types include:
ceph_account
floating_ip
image
instance
instance_disk
instance_network_interface
network
volume
router
To filter the list of resources to a particular resource type (such as a VM, also called an instance), run the following:
openstack metric resource list --type instance
Metrics¶
Resources have metrics whose values are measured every 30 seconds. To view the metrics associated with a resource use the following:
openstack metric resource show <resource id>
For example, if you have a server with id 588858cf-cc5a-4a95-8ef1-74dc12f5d2fe, list the respective metrics using:
openstack metric resource show 588858cf-cc5a-4a95-8ef1-74dc12f5d2fe
which will produce:
+-----------------------+-------------------------------------------------------------------+
| Field | Value |
+-----------------------+-------------------------------------------------------------------+
| created_by_project_id | f0a94de0d6184e10814660d470721c31 |
| created_by_user_id | 39ef552637e74e32abdf25dd7454231f |
| creator | 39ef552637e74e32abdf25dd7454231f:f0a94de0d6184e10814660d470721c31 |
| ended_at | None |
| id | 588858cf-cc5a-4a95-8ef1-74dc12f5d2fe |
| metrics | cpu: 1ef0ab22-1332-4e7e-b30f-ecb5eb12c7d2 |
| | disk.root.size: 6ce00ba2-0c6a-4bb3-9d24-23b4ee291bb9 |
| | memory.usage: 18316275-0a54-48da-8362-dd1c3be48a1b |
| | powered_on_instance: 8e684330-bc4a-482f-aaf1-4e1d0607619a |
| | vcpus: 8c4fe067-f2ca-4e18-9549-f1bfe06a5ed4 |
| original_resource_id | 588858cf-cc5a-4a95-8ef1-74dc12f5d2fe |
| project_id | 5e79a78de75c4cbba82bd26d60119ccf |
| revision_end | None |
| revision_start | 2019-11-17T13:00:30.690133+00:00 |
| started_at | 2019-11-17T12:34:41.869975+00:00 |
| type | instance |
| user_id | abae2cd0ddc642c8b05ab95c4ff0697c |
+-----------------------+-------------------------------------------------------------------+
To view the properties of a metric, such as the “cpu” metric above, run:
openstack metric show 1ef0ab22-1332-4e7e-b30f-ecb5eb12c7d2
This will produce:
+--------------------------------+-------------------------------------------------------------------+
| Field | Value |
+--------------------------------+-------------------------------------------------------------------+
| archive_policy/name | ceilometer-high |
| creator | 39ef552637e74e32abdf25dd7454231f:f0a94de0d6184e10814660d470721c31 |
| id | 1ef0ab22-1332-4e7e-b30f-ecb5eb12c7d2 |
| name | cpu |
| resource/created_by_project_id | f0a94de0d6184e10814660d470721c31 |
| resource/created_by_user_id | 39ef552637e74e32abdf25dd7454231f |
| resource/creator | 39ef552637e74e32abdf25dd7454231f:f0a94de0d6184e10814660d470721c31 |
| resource/ended_at | None |
| resource/id | 588858cf-cc5a-4a95-8ef1-74dc12f5d2fe |
| resource/original_resource_id | 588858cf-cc5a-4a95-8ef1-74dc12f5d2fe |
| resource/project_id | 5e79a78de75c4cbba82bd26d60119ccf |
| resource/revision_end | None |
| resource/revision_start | 2019-11-17T13:00:30.690133+00:00 |
| resource/started_at | 2019-11-17T12:34:41.869975+00:00 |
| resource/type | instance |
| resource/user_id | abae2cd0ddc642c8b05ab95c4ff0697c |
| unit | ns |
+--------------------------------+-------------------------------------------------------------------+
The “unit” field indicates that this metric’s cpu is measures in nanoseconds, meaning the number of nanoseconds of CPU time that has passed.
Measures¶
To view the measured values for the cpu metric above, run:
openstack metric measures show 1ef0ab22-1332-4e7e-b30f-ecb5eb12c7d2
Depending on how long the server has been powered on, this list may be quite long.
To narrow down the list of measures, specify the start and stop date:
openstack metric measures show 1ef0ab22-1332-4e7e-b30f-ecb5eb12c7d2 \
--start "2019-11-18" \
--stop "2019-11-19"
The number of nanoseconds that have passed is not super useful - rather it would be better to retrieve the nanoseconds between each measure. OpenStack’s metric system will return the difference using the –aggregation parameter. For example:
openstack metric measures show 1ef0ab22-1332-4e7e-b30f-ecb5eb12c7d2 \
--start "2019-11-18" \
--stop "2019-11-19" \
--aggregation rate:mean
To view less granular measures, you can choose from 300 seconds or 3600 second granularities, such as:
openstack metric measures show 1ef0ab22-1332-4e7e-b30f-ecb5eb12c7d2 \
--start "2019-11-18" \
--stop "2019-11-19" \
--granularity 300
The aggregation method works with the granularity, so both can be used:
openstack metric measures show 1ef0ab22-1332-4e7e-b30f-ecb5eb12c7d2 \
--start "2019-11-18" \
--stop "2019-11-19" \
--aggregation rate:mean \
--granularity 300
However, what you get is the “mean” value of the 10 x 30-second intervals over the 300 seconds, which might not be what you are looking for. You may need to multiply the results by 10 since there are 10 x 30-second intervals in 300 seconds to get an accurate result.