Fix VM and volume metrics listing regressions #12284

winterhazel · 2025-12-18T03:56:56Z

Description

This PR fixes a regression in commit c8d44d9 that prevents non-admin accounts from listing the metrics of VMs and volumes that belong to a project (see #12237).

Previously, the following example condition was used in the listVirtualMachinesMetrics API in order to find VMs that could have their metrics listed by a user:

vm_instance.type='User' AND user_vm.id=88 AND vm_instance.removed IS NULL

In 4.20.2 and 4.22.0, the condition also restricts the result to VMs that belong to the calling account; hence, VMs that belong to a project are not returned:

WHERE vm_instance.type='User' AND vm_instance.account_id=6 AND user_vm.id=88 AND vm_instance.removed IS NULL

With this patch, the ID of projects that the user has access to is also included in the condition:

WHERE vm_instance.type='User' AND vm_instance.account_id IN (11,6)  AND user_vm.id=88 AND vm_instance.removed IS NULL

Types of changes

Breaking change (fix or feature that would cause existing functionality to change)
New feature (non-breaking change which adds functionality)
Bug fix (non-breaking change which fixes an issue)
Enhancement (improves an existing feature and functionality)
Cleanup (Code refactoring and cleanup, that may add test cases)
build/CI
test (unit or integration test code)

Feature/Enhancement Scale or Bug Severity

Bug Severity

Screenshots (if appropriate):

How Has This Been Tested?

I manually verified that, using both the id and ids parameters:

Root admins can access the metrics of VMs/volumes belonging to other users and projects
Domain admins can access the metrics of VMs/volumes belonging to users and projects of their domain
Users can access the metrics of VMs/volumes belonging to them and to projects they have access to

winterhazel · 2025-12-18T03:57:19Z

@blueorangutan package

blueorangutan · 2025-12-18T03:58:04Z

@winterhazel a [SL] Jenkins job has been kicked to build packages. It will be bundled with KVM, XenServer and VMware SystemVM templates. I'll keep you posted as I make progress.

codecov · 2025-12-18T04:07:05Z

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 16.23%. Comparing base (ba26d95) to head (67c5241).
⚠️ Report is 1 commits behind head on 4.20.

Additional details and impacted files

@@             Coverage Diff              @@
##               4.20   #12284      +/-   ##
============================================
+ Coverage     16.22%   16.23%   +0.01%     
- Complexity    13358    13379      +21     
============================================
  Files          5657     5657              
  Lines        498692   498865     +173     
  Branches      60530    60545      +15     
============================================
+ Hits          80932    81011      +79     
- Misses       408738   408820      +82     
- Partials       9022     9034      +12

Flag	Coverage Δ
uitests	`4.00% <ø> (ø)`
unittests	`17.09% <100.00%> (+0.01%)`	⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

blueorangutan · 2025-12-18T05:17:47Z

Packaging result [SF]: ✔️ el8 ✔️ el9 ✔️ el10 ✔️ debian ✔️ suse15. SL-JID 16080

kiranchavala

LGTM

Created a user in a domain
Login, as the user and Create a project
Deploy a vm in the project and check the metrics tab

Before fix

the API listVirtualMachinesUsageHistory has no response

After fix

the API listVirtualMachinesUsageHistory has response

DaanHoogland

clgtm, one remark/question though; two bits of code are repeated with a similar statement in between, Can we create a method for it?

winterhazel · 2025-12-18T12:53:31Z

@kiranchavala thanks for testing :]

@DaanHoogland yup, I will do some refactoring (and some extra tests to be sure) before marking it as ready.

…gnored when passing multiple IDs

winterhazel · 2025-12-18T20:11:14Z

@blueorangutan package

blueorangutan · 2025-12-18T20:12:04Z

@winterhazel a [SL] Jenkins job has been kicked to build packages. It will be bundled with KVM, XenServer and VMware SystemVM templates. I'll keep you posted as I make progress.

blueorangutan · 2025-12-18T20:12:58Z

Packaging result [SF]: ✖️ el8 ✖️ el9 ✖️ debian ✖️ suse15. SL-JID 16090

winterhazel · 2025-12-18T20:17:56Z

@DaanHoogland I extracted the code that builds the parameters to a method. Tests are also ok.

I noticed that there was another regression in which admins were not able to list the metrics of VMs/volumes belonging to projects via the ids parameter. It should be fixed now too.

Example

<before>
(admin) 🐢 > list virtualmachinesusagehistory id=69ecb2a6-dc91-46b9-a5d9-06e8222c5084
{
  "count": 1,
  "virtualmachine": [
    {
      "displayname": "k",
      "id": "69ecb2a6-dc91-46b9-a5d9-06e8222c5084",
      "name": "i-2-88-VM",
      "stats": [
        {
          "cpuused": "0%",
          "diskiopstotal": 0,
          "diskioread": 0,
          "diskiowrite": 0,
          "diskkbsread": 0,
          "diskkbswrite": 0,
          "memoryintfreekbs": 353728,
          "memorykbs": 524288,
          "memorytargetkbs": 524288,
          "networkkbsread": 0,
          "networkkbswrite": 0,
          "networkread": "0.00 MiB",
          "networkwrite": "0.00 MiB",
          "timestamp": "2025-12-18T19:21:54+0000"
        },
        {
          "cpuused": "20.77%",
          "diskiopstotal": 835,
          "diskioread": 640,
          "diskiowrite": 195,
          "diskkbsread": 10876,
          "diskkbswrite": 1612,
          "memoryintfreekbs": 353728,
          "memorykbs": 524288,
          "memorytargetkbs": 524288,
          "networkkbsread": 0,
          "networkkbswrite": 2,
          "networkread": "0.00 MiB",
          "networkwrite": "0.00 MiB",
          "timestamp": "2025-12-18T19:22:54+0000"
        },
        {
          "cpuused": "13.52%",
          "diskiopstotal": 0,
          "diskioread": 0,
          "diskiowrite": 0,
          "diskkbsread": 0,
          "diskkbswrite": 0,
          "memoryintfreekbs": 353728,
          "memorykbs": 524288,
          "memorytargetkbs": 524288,
          "networkkbsread": 0,
          "networkkbswrite": 0,
          "networkread": "0.00 MiB",
          "networkwrite": "0.00 MiB",
          "timestamp": "2025-12-18T19:23:54+0000"
        }
      ]
    }
  ]
}

(admin) 🐢 > list virtualmachinesusagehistory ids=69ecb2a6-dc91-46b9-a5d9-06e8222c5084
<empty>

<with the patch>
(admin) 🐢 > list virtualmachinesusagehistory ids=69ecb2a6-dc91-46b9-a5d9-06e8222c5084
{
  "count": 1,
  "virtualmachine": [
    {
      "displayname": "k",
      "id": "69ecb2a6-dc91-46b9-a5d9-06e8222c5084",
      "name": "i-2-88-VM",
      "stats": [
        {
          "cpuused": "0%",
          "diskiopstotal": 0,
          "diskioread": 0,
          "diskiowrite": 0,
          "diskkbsread": 0,
          "diskkbswrite": 0,
          "memoryintfreekbs": 353728,
          "memorykbs": 524288,
          "memorytargetkbs": 524288,
          "networkkbsread": 0,
          "networkkbswrite": 0,
          "networkread": "0.00 MiB",
          "networkwrite": "0.00 MiB",
          "timestamp": "2025-12-18T19:21:54+0000"
        },
        {
          "cpuused": "20.77%",
          "diskiopstotal": 835,
          "diskioread": 640,
          "diskiowrite": 195,
          "diskkbsread": 10876,
          "diskkbswrite": 1612,
          "memoryintfreekbs": 353728,
          "memorykbs": 524288,
          "memorytargetkbs": 524288,
          "networkkbsread": 0,
          "networkkbswrite": 2,
          "networkread": "0.00 MiB",
          "networkwrite": "0.00 MiB",
          "timestamp": "2025-12-18T19:22:54+0000"
        },
        {
          "cpuused": "13.52%",
          "diskiopstotal": 0,
          "diskioread": 0,
          "diskiowrite": 0,
          "diskkbsread": 0,
          "diskkbswrite": 0,
          "memoryintfreekbs": 353728,
          "memorykbs": 524288,
          "memorytargetkbs": 524288,
          "networkkbsread": 0,
          "networkkbswrite": 0,
          "networkread": "0.00 MiB",
          "networkwrite": "0.00 MiB",
          "timestamp": "2025-12-18T19:23:54+0000"
        },
        {
          "cpuused": "60.96%",
          "diskiopstotal": 1116,
          "diskioread": 902,
          "diskiowrite": 214,
          "diskkbsread": 67400,
          "diskkbswrite": 2040,
          "memoryintfreekbs": 353728,
          "memorykbs": 524288,
          "memorytargetkbs": 524288,
          "networkkbsread": 0,
          "networkkbswrite": 0,
          "networkread": "0.00 MiB",
          "networkwrite": "0.00 MiB",
          "timestamp": "2025-12-18T19:24:55+0000"
        },
        {
          "cpuused": "40.05%",
          "diskiopstotal": 277,
          "diskioread": 110,
          "diskiowrite": 167,
          "diskkbsread": 13304,
          "diskkbswrite": 38524,
          "memoryintfreekbs": 353728,
          "memorykbs": 524288,
          "memorytargetkbs": 524288,
          "networkkbsread": 0,
          "networkkbswrite": 0,
          "networkread": "0.00 MiB",
          "networkwrite": "0.00 MiB",
          "timestamp": "2025-12-18T19:25:56+0000"
        },
        {
          "cpuused": "26.1%",
          "diskiopstotal": 3,
          "diskioread": 0,
          "diskiowrite": 3,
          "diskkbsread": 0,
          "diskkbswrite": 16,
          "memoryintfreekbs": 353728,
          "memorykbs": 524288,
          "memorytargetkbs": 524288,
          "networkkbsread": 0,
          "networkkbswrite": 0,
          "networkread": "0.00 MiB",
          "networkwrite": "0.00 MiB",
          "timestamp": "2025-12-18T19:26:58+0000"
        },
        {
          "cpuused": "25.4%",
          "diskiopstotal": 1,
          "diskioread": 0,
          "diskiowrite": 1,
          "diskkbsread": 0,
          "diskkbswrite": 4,
          "memoryintfreekbs": 353728,
          "memorykbs": 524288,
          "memorytargetkbs": 524288,
          "networkkbsread": 0,
          "networkkbswrite": 0,
          "networkread": "0.00 MiB",
          "networkwrite": "0.00 MiB",
          "timestamp": "2025-12-18T19:27:59+0000"
        },
        {
          "cpuused": "24.83%",
          "diskiopstotal": 0,
          "diskioread": 0,
          "diskiowrite": 0,
          "diskkbsread": 0,
          "diskkbswrite": 0,
          "memoryintfreekbs": 353728,
          "memorykbs": 524288,
          "memorytargetkbs": 524288,
          "networkkbsread": 0,
          "networkkbswrite": 0,
          "networkread": "0.00 MiB",
          "networkwrite": "0.00 MiB",
          "timestamp": "2025-12-18T19:29:02+0000"
        },
        {
          "cpuused": "24.91%",
          "diskiopstotal": 0,
          "diskioread": 0,
          "diskiowrite": 0,
          "diskkbsread": 0,
          "diskkbswrite": 0,
          "memoryintfreekbs": 353728,
          "memorykbs": 524288,
          "memorytargetkbs": 524288,
          "networkkbsread": 0,
          "networkkbswrite": 0,
          "networkread": "0.00 MiB",
          "networkwrite": "0.00 MiB",
          "timestamp": "2025-12-18T19:30:04+0000"
        },
        {
          "cpuused": "25.87%",
          "diskiopstotal": 0,
          "diskioread": 0,
          "diskiowrite": 0,
          "diskkbsread": 0,
          "diskkbswrite": 0,
          "memoryintfreekbs": 353728,
          "memorykbs": 524288,
          "memorytargetkbs": 524288,
          "networkkbsread": 0,
          "networkkbswrite": 0,
          "networkread": "0.00 MiB",
          "networkwrite": "0.00 MiB",
          "timestamp": "2025-12-18T19:31:09+0000"
        },
        {
          "cpuused": "25.24%",
          "diskiopstotal": 0,
          "diskioread": 0,
          "diskiowrite": 0,
          "diskkbsread": 0,
          "diskkbswrite": 0,
          "memoryintfreekbs": 353728,
          "memorykbs": 524288,
          "memorytargetkbs": 524288,
          "networkkbsread": 0,
          "networkkbswrite": 0,
          "networkread": "0.00 MiB",
          "networkwrite": "0.00 MiB",
          "timestamp": "2025-12-18T19:32:11+0000"
        },
        {
          "cpuused": "20.79%",
          "diskiopstotal": 0,
          "diskioread": 0,
          "diskiowrite": 0,
          "diskkbsread": 0,
          "diskkbswrite": 0,
          "memoryintfreekbs": 353728,
          "memorykbs": 524288,
          "memorytargetkbs": 524288,
          "networkkbsread": 0,
          "networkkbswrite": 0,
          "networkread": "0.00 MiB",
          "networkwrite": "0.00 MiB",
          "timestamp": "2025-12-18T19:33:12+0000"
        },
        {
          "cpuused": "6.03%",
          "diskiopstotal": 0,
          "diskioread": 0,
          "diskiowrite": 0,
          "diskkbsread": 0,
          "diskkbswrite": 0,
          "memoryintfreekbs": 353728,
          "memorykbs": 524288,
          "memorytargetkbs": 524288,
          "networkkbsread": 0,
          "networkkbswrite": 0,
          "networkread": "0.00 MiB",
          "networkwrite": "0.00 MiB",
          "timestamp": "2025-12-18T19:34:13+0000"
        },
        {
          "cpuused": "5.77%",
          "diskiopstotal": 0,
          "diskioread": 0,
          "diskiowrite": 0,
          "diskkbsread": 0,
          "diskkbswrite": 0,
          "memoryintfreekbs": 353728,
          "memorykbs": 524288,
          "memorytargetkbs": 524288,
          "networkkbsread": 0,
          "networkkbswrite": 0,
          "networkread": "0.00 MiB",
          "networkwrite": "0.00 MiB",
          "timestamp": "2025-12-18T19:35:13+0000"
        },
        {
          "cpuused": "5.78%",
          "diskiopstotal": 0,
          "diskioread": 0,
          "diskiowrite": 0,
          "diskkbsread": 0,
          "diskkbswrite": 0,
          "memoryintfreekbs": 353728,
          "memorykbs": 524288,
          "memorytargetkbs": 524288,
          "networkkbsread": 0,
          "networkkbswrite": 0,
          "networkread": "0.00 MiB",
          "networkwrite": "0.00 MiB",
          "timestamp": "2025-12-18T19:36:13+0000"
        },
        {
          "cpuused": "7.01%",
          "diskiopstotal": 158,
          "diskioread": 103,
          "diskiowrite": 55,
          "diskkbsread": 412,
          "diskkbswrite": 472,
          "memoryintfreekbs": 353728,
          "memorykbs": 524288,
          "memorytargetkbs": 524288,
          "networkkbsread": 0,
          "networkkbswrite": 0,
          "networkread": "0.00 MiB",
          "networkwrite": "0.00 MiB",
          "timestamp": "2025-12-18T19:37:13+0000"
        },
        {
          "cpuused": "8.05%",
          "diskiopstotal": 3,
          "diskioread": 0,
          "diskiowrite": 3,
          "diskkbsread": 0,
          "diskkbswrite": 16,
          "memoryintfreekbs": 353728,
          "memorykbs": 524288,
          "memorytargetkbs": 524288,
          "networkkbsread": 0,
          "networkkbswrite": 0,
          "networkread": "0.00 MiB",
          "networkwrite": "0.00 MiB",
          "timestamp": "2025-12-18T19:38:14+0000"
        },
        {
          "cpuused": "8.79%",
          "diskiopstotal": 0,
          "diskioread": 0,
          "diskiowrite": 0,
          "diskkbsread": 0,
          "diskkbswrite": 0,
          "memoryintfreekbs": 353728,
          "memorykbs": 524288,
          "memorytargetkbs": 524288,
          "networkkbsread": 0,
          "networkkbswrite": 0,
          "networkread": "0.00 MiB",
          "networkwrite": "0.00 MiB",
          "timestamp": "2025-12-18T19:39:14+0000"
        },
        {
          "cpuused": "6.4%",
          "diskiopstotal": 0,
          "diskioread": 0,
          "diskiowrite": 0,
          "diskkbsread": 0,
          "diskkbswrite": 0,
          "memoryintfreekbs": 353728,
          "memorykbs": 524288,
          "memorytargetkbs": 524288,
          "networkkbsread": 0,
          "networkkbswrite": 0,
          "networkread": "0.00 MiB",
          "networkwrite": "0.00 MiB",
          "timestamp": "2025-12-18T19:40:14+0000"
        }
      ]
    }
  ]
}

Also, passing the ids parameter to listVolumesUsageHistory was throwing an exception. I took the opportunity to fix this here as well.

winterhazel · 2025-12-18T20:29:00Z

@blueorangutan package

blueorangutan · 2025-12-18T20:30:04Z

@winterhazel a [SL] Jenkins job has been kicked to build packages. It will be bundled with KVM, XenServer and VMware SystemVM templates. I'll keep you posted as I make progress.

blueorangutan · 2025-12-18T20:31:03Z

Packaging result [SF]: ✖️ el8 ✖️ el9 ✖️ debian ✖️ suse15. SL-JID 16091

winterhazel · 2025-12-18T20:49:55Z

@blueorangutan package

blueorangutan · 2025-12-18T20:52:04Z

@winterhazel a [SL] Jenkins job has been kicked to build packages. It will be bundled with KVM, XenServer and VMware SystemVM templates. I'll keep you posted as I make progress.

blueorangutan · 2025-12-18T20:52:58Z

Packaging result [SF]: ✖️ el8 ✖️ el9 ✖️ debian ✖️ suse15. SL-JID 16092

DaanHoogland · 2025-12-19T08:08:06Z

Packaging result [SF]: ✖️ el8 ✖️ el9 ✖️ debian ✖️ suse15. SL-JID 16092

we hit the github API limit @winterhazel , don’t worry about this.

weizhouapache · 2025-12-19T09:04:38Z

plugins/metrics/src/main/java/org/apache/cloudstack/metrics/MetricsServiceImpl.java

+
+        // If no ID was provided, then the listing will skip project resources (null); otherwise, project resources should
+        // be listed as well (any long allows this)
+        Long id = isIdProvided ? 1L : null;


is there side-effect when id is set to 1L ?

The value itself is not used, com.cloud.user.AccountManagerImpl#buildACLSearchParameters just checks whether the ID is null in order to decide whether project resources should be skipped or not, so no.

cloudstack/server/src/main/java/com/cloud/user/AccountManagerImpl.java

Lines 3376 to 3378 in eb93f01

if (id == null) {

domainIdRecursiveListProject.third(Project.ListProjectResourcesCriteria.SkipProjectResources);

}

blueorangutan · 2025-12-19T10:43:04Z

Packaging result [SF]: ✔️ el8 ✔️ el9 ✔️ el10 ✔️ debian ✔️ suse15. SL-JID 16103

DaanHoogland · 2025-12-19T16:47:15Z

@blueorangutan test

blueorangutan · 2025-12-19T16:48:04Z

@DaanHoogland a [SL] Trillian-Jenkins test job (ol8 mgmt + kvm-ol8) has been kicked to run smoke tests

Fix VM and volume metrics listing regression

c7989c0

kiranchavala approved these changes Dec 18, 2025

View reviewed changes

DaanHoogland approved these changes Dec 18, 2025

View reviewed changes

winterhazel added 2 commits December 18, 2025 17:03

Extract creation of search parameters & fix project resources being i…

c56594b

…gnored when passing multiple IDs

Remove unused import

fbf7551

winterhazel changed the title ~~Fix VM and volume metrics listing regression~~ Fix VM and volume metrics listing regressions Dec 18, 2025

winterhazel marked this pull request as ready for review December 18, 2025 20:11

winterhazel added type:regression component:metrics labels Dec 18, 2025

Remove another unused import

67c5241

weizhouapache reviewed Dec 19, 2025

View reviewed changes

	if (id == null) {
	domainIdRecursiveListProject.third(Project.ListProjectResourcesCriteria.SkipProjectResources);
	}

Fix VM and volume metrics listing regressions #12284

Are you sure you want to change the base?

Fix VM and volume metrics listing regressions #12284

Conversation

winterhazel commented Dec 18, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Description

Types of changes

Feature/Enhancement Scale or Bug Severity

Bug Severity

Screenshots (if appropriate):

How Has This Been Tested?

Uh oh!

winterhazel commented Dec 18, 2025

Uh oh!

blueorangutan commented Dec 18, 2025

Uh oh!

codecov bot commented Dec 18, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Codecov Report

Uh oh!

blueorangutan commented Dec 18, 2025

Uh oh!

kiranchavala left a comment

Choose a reason for hiding this comment

Uh oh!

DaanHoogland left a comment

Choose a reason for hiding this comment

Uh oh!

winterhazel commented Dec 18, 2025

Uh oh!

winterhazel commented Dec 18, 2025

Uh oh!

blueorangutan commented Dec 18, 2025

Uh oh!

blueorangutan commented Dec 18, 2025

Uh oh!

winterhazel commented Dec 18, 2025

Uh oh!

winterhazel commented Dec 18, 2025

Uh oh!

blueorangutan commented Dec 18, 2025

Uh oh!

blueorangutan commented Dec 18, 2025

Uh oh!

winterhazel commented Dec 18, 2025

Uh oh!

blueorangutan commented Dec 18, 2025

Uh oh!

blueorangutan commented Dec 18, 2025

Uh oh!

DaanHoogland commented Dec 19, 2025

Uh oh!

weizhouapache Dec 19, 2025

Choose a reason for hiding this comment

Uh oh!

winterhazel Dec 19, 2025

Choose a reason for hiding this comment

Uh oh!

blueorangutan commented Dec 19, 2025

Uh oh!

DaanHoogland commented Dec 19, 2025

Uh oh!

blueorangutan commented Dec 19, 2025

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

5 participants

winterhazel commented Dec 18, 2025 •

edited

Loading

codecov bot commented Dec 18, 2025 •

edited

Loading