only update powerstate if sure it is the latest by DaanHoogland · Pull Request #3743 · apache/cloudstack

DaanHoogland · 2019-12-04T16:45:59Z

Description

On update powerstate check if we have the latest knowledge before doing so.
When report is missing don't try to stop the VM on the host that doesn't know the VM.

Fixes: #3741

Types of changes

Breaking change (fix or feature that would cause existing functionality to change)
New feature (non-breaking change which adds functionality)
Bug fix (non-breaking change which fixes an issue)
Enhancement (improves an existing feature and functionality)
Cleanup (Code refactoring and cleanup, that may add test cases)

How Has This Been Tested?

This is purely based on log- and code analysis. intensiove review is requested. Also advise on environmental consequences is welcomed.
HA may be impacted do I suspect not in a negative way.

DaanHoogland · 2019-12-05T14:16:54Z

any vmware users or HA users are invited to have a look at this.

DaanHoogland · 2019-12-05T14:29:23Z

will be backporting to 4.13

andrijapanicsb · 2019-12-06T14:04:17Z

/CC @Doni7722

andrijapanicsb

I can only give LGTM based on the internal discussion we had - since it's impossible to test this edge case of race condition we are fixing here.
(can't comment the code, since not developer myself)

andrijapanicsb · 2019-12-31T13:13:14Z

This needs testing with VMware specifically

blueorangutan · 2019-12-31T13:14:12Z

@andrijapanicsb unsupported parameters provided. Supported mgmt server os are: centos6, centos7, ubuntu. Supported hypervisors are: kvm-centos6, kvm-centos7, kvm-ubuntu, xenserver-71, xenserver-65sp1, xenserver-62sp1, vmware-65u2, vmware-60u2, vmware-55u3, vmware-51u1, vmware-50u1

andrijapanicsb · 2019-12-31T22:07:58Z

@blueorangutan test centos7 vmware-65u2

nvazquez

Code LGTM. Thanks @DaanHoogland

DaanHoogland · 2020-01-03T08:12:20Z

revert snapshot error does not seem related but doing a re-package/re-test to be sure.
@blueorangutan package

andrijapanicsb · 2020-01-04T14:16:56Z

@blueorangutan test centos7 vmware-65u2

DaanHoogland · 2020-01-06T14:32:57Z

@blueorangutan package

blueorangutan · 2020-01-06T14:33:39Z

@DaanHoogland a Jenkins job has been kicked to build packages. I'll keep you posted as I make progress.

blueorangutan · 2020-01-06T14:57:18Z

Packaging result: ✔centos6 ✔centos7 ✔debian. JID-540

DaanHoogland · 2020-01-06T15:02:39Z

@blueorangutan test centos7 vmware-65u2

blueorangutan · 2020-01-06T15:03:39Z

@DaanHoogland a Trillian-Jenkins test job (centos7 mgmt + vmware-65u2) has been kicked to run smoke tests

blueorangutan · 2020-01-07T00:07:41Z

Trillian test result (tid-708)
Environment: vmware-65u2 (x2), Advanced Networking with Mgmt server 7
Total time taken: 30988 seconds
Marvin logs: https://github.com/blueorangutan/acs-prs/releases/download/trillian/pr3743-t708-vmware-65u2.zip
Smoke tests completed. 77 look OK, 0 have error(s)
Only failed tests results shown below:

Test	Result	Time (s)	Test File

* 4.13: only update powerstate if sure it is the latest (#3743) ui: fix migrate host form no host popup (#3682) client: jetty session timeout set after server is started (#3658) Increase DHCP lease time to infinite (#3662)

nvazquez · 2020-01-15T12:20:16Z

- ", power state: PowerReportMissing, last state update: " + vmStateUpdateTime.getTime());
+ if (s_logger.isInfoEnabled()) {
+ s_logger.info(
+ String.format("Detected missing VM. host: %l, vm id: %l(%s), power state: %s, last state update: %l"


Hi @DaanHoogland, I'm getting this error in latest master:
INFO [c.c.a.m.AgentManagerImpl] (AgentTaskPool-98:ctx-ad691272) (logid:1a451217) The agent from host 1 state determined is Up INFO [c.c.a.m.AgentManagerImpl] (AgentTaskPool-98:ctx-ad691272) (logid:1a451217) Agent is determined to be up and running WARN [c.c.a.m.AgentManagerImpl] (AgentManager-Handler-8:null) (logid:) Caught: java.util.UnknownFormatConversionException: Conversion = 'l' at java.util.Formatter$FormatSpecifier.conversion(Formatter.java:2691) at java.util.Formatter$FormatSpecifier.<init>(Formatter.java:2720) at java.util.Formatter.parse(Formatter.java:2560) at java.util.Formatter.format(Formatter.java:2501) at java.util.Formatter.format(Formatter.java:2455) at java.lang.String.format(String.java:2940) at com.cloud.vm.VirtualMachinePowerStateSyncImpl.processReport(VirtualMachinePowerStateSyncImpl.java:141) at com.cloud.vm.VirtualMachinePowerStateSyncImpl.processHostVmStatePingReport(VirtualMachinePowerStateSyncImpl.java:66) at com.cloud.vm.VirtualMachineManagerImpl.processCommands(VirtualMachineManagerImpl.java:3245) at com.cloud.agent.manager.AgentManagerImpl.handleCommands(AgentManagerImpl.java:309) at com.cloud.agent.manager.AgentManagerImpl$AgentHandler.processRequest(AgentManagerImpl.java:1287) at com.cloud.agent.manager.AgentManagerImpl$AgentHandler.doTask(AgentManagerImpl.java:1370) at com.cloud.agent.manager.ClusteredAgentManagerImpl$ClusteredAgentHandler.doTask(ClusteredAgentManagerImpl.java:702) at com.cloud.utils.nio.Task.call(Task.java:83) at com.cloud.utils.nio.Task.call(Task.java:29) at java.util.concurrent.FutureTask.run(FutureTask.java:266) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:748)

yes, i know, fixed in #3806

svenvogel added type:bug component:management-server labels Dec 4, 2019

DaanHoogland changed the title ~~only update powerstate is sure it is the latest~~only update powerstate if sure it is the latestDec 5, 2019

DaanHoogland marked this pull request as ready for review December 5, 2019 12:51

DaanHoogland requested review from andrijapanicsb and yadvr December 5, 2019 14:16

only update powerstate is sure it is the latest
ee718bc

DaanHoogland force-pushed the timingPowerState branch from c67c413 to ee718bcCompare December 5, 2019 15:24

DaanHoogland changed the base branch from master to 4.13December 5, 2019 15:25

andrijapanicsb approved these changes Dec 6, 2019
View reviewed changes

yadvr added this to the 4.13.1.0 milestone Dec 7, 2019

DaanHoogland requested review from GabrielBrascher, nvazquez, weizhouapache and wido December 23, 2019 09:47

nvazquez approved these changes Jan 2, 2020
View reviewed changes

apache deleted a comment from blueorangutan Jan 3, 2020

apache deleted a comment from andrijapanicsb Jan 3, 2020

apache deleted a comment from blueorangutan Jan 3, 2020

apache deleted a comment from borisstoyanov Jan 3, 2020

apache deleted a comment from blueorangutan Jan 3, 2020

apache deleted a comment from blueorangutan Jan 4, 2020

initial state npe
8682179

apache deleted a comment from blueorangutan Jan 6, 2020

DaanHoogland merged commit d44dc07 into apache:4.13Jan 7, 2020

DaanHoogland deleted the timingPowerState branch January 7, 2020 08:12

nvazquez reviewed Jan 15, 2020
View reviewed changes

ustcweizhou pushed a commit to ustcweizhou/cloudstack that referenced this pull request Feb 28, 2020
only update powerstate if sure it is the latest (apache#3743)
1d4457d

Uh oh!

Conversation

DaanHoogland commented Dec 4, 2019• edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Description

Types of changes

How Has This Been Tested?

Uh oh!

DaanHoogland commented Dec 5, 2019

Uh oh!

DaanHoogland commented Dec 5, 2019

Uh oh!

andrijapanicsb commented Dec 6, 2019

Uh oh!

andrijapanicsb left a comment

Choose a reason for hiding this comment

Uh oh!

andrijapanicsb commented Dec 31, 2019

Uh oh!

blueorangutan commented Dec 31, 2019

Uh oh!

andrijapanicsb commented Dec 31, 2019

Uh oh!

nvazquez left a comment

Choose a reason for hiding this comment

Uh oh!

DaanHoogland commented Jan 3, 2020

Uh oh!

andrijapanicsb commented Jan 4, 2020

Uh oh!

DaanHoogland commented Jan 6, 2020

Uh oh!

blueorangutan commented Jan 6, 2020

Uh oh!

blueorangutan commented Jan 6, 2020

Uh oh!

DaanHoogland commented Jan 6, 2020

Uh oh!

blueorangutan commented Jan 6, 2020

Uh oh!

blueorangutan commented Jan 7, 2020

Uh oh!

nvazquezJan 15, 2020

Choose a reason for hiding this comment

Uh oh!

DaanHooglandJan 15, 2020

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

6 participants

DaanHoogland commented Dec 4, 2019•
edited
Loading