kvm: fix exception in volume stats after storage migration by ustcweizhou · Pull Request #3884 · apache/cloudstack

ustcweizhou · 2020-02-12T21:34:29Z

Description

on kvm, the 'path' of volume is the file name on primary storage. we should use 'path' instead of 'uuid' in volume statistics.

Fixes: #3878

Types of changes

Breaking change (fix or feature that would cause existing functionality to change)
New feature (non-breaking change which adds functionality)
Bug fix (non-breaking change which fixes an issue)
Enhancement (improves an existing feature and functionality)
Cleanup (Code refactoring and cleanup, that may add test cases)

Screenshots (if appropriate):

How Has This Been Tested?

storage migration
volume statistics every 10 minutes (default)

yadvr

Does it matter for managed vs unmanaged?

yadvr · 2020-02-13T04:52:21Z

@blueorangutan package

yadvr

Nevermind, saw the PR that introduced the regression

blueorangutan · 2020-02-13T04:53:13Z

@rhtyd a Jenkins job has been kicked to build packages. I'll keep you posted as I make progress.

yadvr · 2020-02-13T04:53:24Z

@blueorangutan package

blueorangutan · 2020-02-13T04:54:13Z

@rhtyd a Jenkins job has been kicked to build packages. I'll keep you posted as I make progress.

blueorangutan · 2020-02-13T05:25:21Z

Packaging result: ✖centos6 ✔centos7 ✔debian. JID-848

blueorangutan · 2020-02-13T05:25:21Z

Packaging result: ✖centos6 ✔centos7 ✔debian. JID-849

yadvr · 2020-02-13T05:26:19Z

Packaging overlap failure, rekicking
@blueorangutan package

blueorangutan · 2020-02-13T05:27:12Z

@rhtyd a Jenkins job has been kicked to build packages. I'll keep you posted as I make progress.

blueorangutan · 2020-02-13T05:44:23Z

Packaging result: ✖centos6 ✔centos7 ✔debian. JID-850

yadvr · 2020-02-13T05:49:47Z

@blueorangutan test

blueorangutan · 2020-02-13T05:50:13Z

@rhtyd a Trillian-Jenkins test job (centos7 mgmt + kvm-centos7) has been kicked to run smoke tests

yadvr · 2020-02-14T06:30:07Z

@blueorangutan test

blueorangutan · 2020-02-14T06:30:17Z

@rhtyd a Trillian-Jenkins test job (centos7 mgmt + kvm-centos7) has been kicked to run smoke tests

DaanHoogland

looks good, I trust Wei, but as a matter of principle, did someone test?

andrijapanicsb · 2020-02-14T15:17:21Z

LGTM

haven't tested. @weizhouapache if you tested, I'm fine with it - thx for the fix.

weizhouapache · 2020-02-15T21:32:55Z

@DaanHoogland @andrijapanicsb @rhtyd

I have tested with qcow2 on NFS. Did not test local storage and Ceph RBD.

yadvr · 2020-02-17T01:13:29Z

@blueorangutan package

blueorangutan · 2020-02-17T01:14:30Z

@rhtyd a Jenkins job has been kicked to build packages. I'll keep you posted as I make progress.

blueorangutan · 2020-02-17T01:48:51Z

Packaging result: ✖centos6 ✔centos7 ✔debian. JID-876

DaanHoogland · 2020-02-17T09:50:36Z

@blueorangutan test

yadvr · 2020-02-17T15:40:09Z

@blueorangutan test

blueorangutan · 2020-02-17T15:40:33Z

@rhtyd a Trillian-Jenkins test job (centos7 mgmt + kvm-centos7) has been kicked to run smoke tests

blueorangutan · 2020-02-18T07:32:49Z

Trillian test result (tid-1011)
Environment: kvm-centos7 (x2), Advanced Networking with Mgmt server 7
Total time taken: 55716 seconds
Marvin logs: https://github.com/blueorangutan/acs-prs/releases/download/trillian/pr3884-t1011-kvm-centos7.zip
Intermittent failure detected: /marvin/tests/smoke/test_network.py
Intermittent failure detected: /marvin/tests/smoke/test_password_server.py
Intermittent failure detected: /marvin/tests/smoke/test_privategw_acl.py
Intermittent failure detected: /marvin/tests/smoke/test_router_dhcphosts.py
Intermittent failure detected: /marvin/tests/smoke/test_router_dns.py
Intermittent failure detected: /marvin/tests/smoke/test_router_dnsservice.py
Intermittent failure detected: /marvin/tests/smoke/test_routers_iptables_default_policy.py
Intermittent failure detected: /marvin/tests/smoke/test_routers_network_ops.py
Intermittent failure detected: /marvin/tests/smoke/test_service_offerings.py
Intermittent failure detected: /marvin/tests/smoke/test_templates.py
Intermittent failure detected: /marvin/tests/smoke/test_usage.py
Smoke tests completed. 66 look OK, 11 have error(s)
Only failed tests results shown below:

Test	Result	Time (s)	Test File
test_01_port_fwd_on_src_nat	`Failure`	813.94	test_network.py
test_02_port_fwd_on_non_src_nat	`Failure`	816.10	test_network.py
test_reboot_router	`Failure`	397.09	test_network.py
test_network_rules_acquired_public_ip_1_static_nat_rule	`Failure`	815.14	test_network.py
test_network_rules_acquired_public_ip_2_nat_rule	`Failure`	816.63	test_network.py
test_network_rules_acquired_public_ip_3_Load_Balancer_Rule	`Failure`	819.69	test_network.py
test_isolate_network_password_server	`Failure`	216.62	test_password_server.py
test_02_vpc_privategw_static_routes	`Failure`	941.47	test_privategw_acl.py
test_03_vpc_privategw_restart_vpc_cleanup	`Failure`	941.43	test_privategw_acl.py
test_04_rvpc_privategw_static_routes	`Failure`	996.36	test_privategw_acl.py
test_router_dhcphosts	`Failure`	216.53	test_router_dhcphosts.py
ContextSuite context=TestRouterDHCPHosts>:teardown	`Error`	225.79	test_router_dhcphosts.py
test_router_dns_guestipquery	`Failure`	612.66	test_router_dns.py
test_router_dns_guestipquery	`Failure`	613.73	test_router_dnsservice.py
test_01_single_VPC_iptables_policies	`Error`	916.52	test_routers_iptables_default_policy.py
test_01_isolate_network_FW_PF_default_routes_egress_true	`Failure`	263.10	test_routers_network_ops.py
test_02_isolate_network_FW_PF_default_routes_egress_false	`Failure`	261.07	test_routers_network_ops.py
test_01_RVR_Network_FW_PF_SSH_default_routes_egress_true	`Failure`	287.26	test_routers_network_ops.py
test_02_RVR_Network_FW_PF_SSH_default_routes_egress_false	`Failure`	284.19	test_routers_network_ops.py
test_01_service_offering_cpu_limit_use	`Error`	811.17	test_service_offerings.py
test_04_change_offering_small	`Failure`	910.99	test_service_offerings.py
test_04_extract_template	`Failure`	128.30	test_templates.py
ContextSuite context=TestISOUsage>:setup	`Error`	0.00	test_usage.py

yadvr · 2020-02-18T08:38:06Z

@skattoju3 can you review this?
@blueorangutan package

blueorangutan · 2020-02-18T08:38:35Z

@rhtyd a Jenkins job has been kicked to build packages. I'll keep you posted as I make progress.

blueorangutan · 2020-02-18T08:54:40Z

Packaging result: ✖centos6 ✔centos7 ✔debian. JID-894

yadvr · 2020-02-18T14:38:13Z

@blueorangutan test

blueorangutan · 2020-02-18T14:38:37Z

@rhtyd a Trillian-Jenkins test job (centos7 mgmt + kvm-centos7) has been kicked to run smoke tests

blueorangutan · 2020-02-18T22:04:34Z

Trillian test result (tid-1030)
Environment: kvm-centos7 (x2), Advanced Networking with Mgmt server 7
Total time taken: 25326 seconds
Marvin logs: https://github.com/blueorangutan/acs-prs/releases/download/trillian/pr3884-t1030-kvm-centos7.zip
Intermittent failure detected: /marvin/tests/smoke/test_privategw_acl.py
Smoke tests completed. 76 look OK, 1 have error(s)
Only failed tests results shown below:

Test	Result	Time (s)	Test File
test_02_vpc_privategw_static_routes	`Failure`	166.72	test_privategw_acl.py
test_03_vpc_privategw_restart_vpc_cleanup	`Failure`	164.54	test_privategw_acl.py
test_04_rvpc_privategw_static_routes	`Failure`	217.70	test_privategw_acl.py

) On kvm, the 'path' of volume is the file name on primary storage. we should use 'path' instead of 'uuid' in volume statistics. Fixes: apache#3878

…ck/pull/3884/files

* Applying PR apache#3884 from 4.13: https://github.com/apache/cloudstack/pull/3884/files * Applying PR apache#3884 from 4.13: https://github.com/apache/cloudstack/pull/3884/files

kvm: fix exception in volume statts after storage migration
dcfac40

ustcweizhou changed the base branch from master to 4.13February 12, 2020 21:34

weizhouapache mentioned this pull request Feb 12, 2020
Volume metrics doesn't update after volume migration #3878
Closed

weizhouapache changed the title ~~kvm: fix exception in volume statts after storage migration~~kvm: fix exception in volume stats after storage migrationFeb 12, 2020

yadvr reviewed Feb 13, 2020
View reviewed changes

yadvr closed this Feb 13, 2020

yadvr reopened this Feb 13, 2020

yadvr approved these changes Feb 13, 2020
View reviewed changes

yadvr added this to the 4.13.1.0 milestone Feb 13, 2020

yadvr closed this Feb 14, 2020

yadvr reopened this Feb 14, 2020

DaanHoogland approved these changes Feb 14, 2020
View reviewed changes

svenvogel added type:bug component:primary-storage labels Feb 16, 2020

yadvr closed this Feb 17, 2020

yadvr reopened this Feb 17, 2020

DaanHoogland closed this Feb 17, 2020

DaanHoogland reopened this Feb 17, 2020

yadvr closed this Feb 17, 2020

yadvr reopened this Feb 17, 2020

skattoju3 approved these changes Feb 18, 2020
View reviewed changes

yadvr merged commit 649ed45 into apache:4.13Feb 19, 2020

slavkap mentioned this pull request Mar 12, 2020
Fix: catch CloudRuntimeException in LibvirtGetVolumeStatsCommandWrapper.java #3949
Merged
4 tasks

weizhouapache mentioned this pull request Oct 7, 2020
fix NPE in volumes statistics #4388
Merged
3 tasks

nlgordon added a commit to ippathways/cloudstack that referenced this pull request Sep 14, 2021
Applying PRapache#3884from 4.13:https://github.com/apache/cloudsta…
f81d915
…ck/pull/3884/files

nlgordon added a commit to ippathways/cloudstack that referenced this pull request Sep 15, 2021
Applying PRapache#3884from 4.13:https://github.com/apache/cloudsta…
d7fa092
…ck/pull/3884/files

Uh oh!

Conversation

ustcweizhou commented Feb 12, 2020

Description

Types of changes

Screenshots (if appropriate):

How Has This Been Tested?

Uh oh!

yadvr left a comment

Choose a reason for hiding this comment

Uh oh!

yadvr commented Feb 13, 2020

Uh oh!

yadvr left a comment

Choose a reason for hiding this comment

Uh oh!

blueorangutan commented Feb 13, 2020

Uh oh!

yadvr commented Feb 13, 2020

Uh oh!

blueorangutan commented Feb 13, 2020

Uh oh!

blueorangutan commented Feb 13, 2020

Uh oh!

blueorangutan commented Feb 13, 2020

Uh oh!

yadvr commented Feb 13, 2020

Uh oh!

blueorangutan commented Feb 13, 2020

Uh oh!

blueorangutan commented Feb 13, 2020

Uh oh!

yadvr commented Feb 13, 2020

Uh oh!

blueorangutan commented Feb 13, 2020

Uh oh!

yadvr commented Feb 14, 2020

Uh oh!

blueorangutan commented Feb 14, 2020

Uh oh!

DaanHoogland left a comment

Choose a reason for hiding this comment

Uh oh!

andrijapanicsb commented Feb 14, 2020

Uh oh!

weizhouapache commented Feb 15, 2020

Uh oh!

yadvr commented Feb 17, 2020

Uh oh!

blueorangutan commented Feb 17, 2020

Uh oh!

blueorangutan commented Feb 17, 2020

Uh oh!

DaanHoogland commented Feb 17, 2020

Uh oh!

yadvr commented Feb 17, 2020

Uh oh!

blueorangutan commented Feb 17, 2020

Uh oh!

blueorangutan commented Feb 18, 2020

Uh oh!

yadvr commented Feb 18, 2020

Uh oh!

blueorangutan commented Feb 18, 2020

Uh oh!

blueorangutan commented Feb 18, 2020

Uh oh!

yadvr commented Feb 18, 2020

Uh oh!

blueorangutan commented Feb 18, 2020

Uh oh!

blueorangutan commented Feb 18, 2020

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!