Skip to content

Add new registers in guest_os - #4699

Merged
nvazquez merged 9 commits into
apache:mainfrom
GutoVeronezi:add-new-guest-os-registers-to-database
Aug 12, 2021
Merged

Add new registers in guest_os#4699
nvazquez merged 9 commits into
apache:mainfrom
GutoVeronezi:add-new-guest-os-registers-to-database

Conversation

@GutoVeronezi

Copy link
Copy Markdown
Contributor

Description

ACS has some APIs to manage guest_os and guest_os_hypervisor mapping entries; however, it is impracticable to use it due to the way we have been working.

Most times we add data to these tables via SQLs with a hardcoded id. (See schema-41200to41300.sql). Therefore, if an operator adds a new row to guest_os via the API, when he/she updates the version to any that insert a new row to guest_os, it might break the update due to duplicated id.

An INSERT IGNORE would avoid the breaking, but it would not insert the new row (if it contains different data/information/configuration); therefore, it does not seem like a good solution. Although it may not happen often, it could generate duplicated data (like display_name...).

This PR intends to add new data in guest_os and to suggest a new way to insert guest_os data via SQL, verifying if it already exists on the table (with insert... select ... where not exists).

Obs: we used a temporary table to avoid repeating the code.

Types of changes

  • Breaking change (fix or feature that would cause existing functionality to change)
  • New feature (non-breaking change which adds functionality)
  • Bug fix (non-breaking change which fixes an issue)
  • Enhancement (improves an existing feature and functionality)
  • Cleanup (Code refactoring and cleanup, that may add test cases)

Feature/Enhancement Scale or Bug Severity

Feature/Enhancement Scale

  • Major
  • Minor

How Has This Been Tested?

It has been tested locally on a test lab.

  1. I generated the packages and updated my lab;

@GutoVeronezi
GutoVeroneziforce-pushed the add-new-guest-os-registers-to-database branch from 015be09 to ffc3670CompareFebruary 17, 2021 20:27
@weizhouapache

Copy link
Copy Markdown
Member

@GutoVeronezi
it is better to create a mysql procedure which can be used for any hypervisor and guest os.

@shwstpprshwstppr added this to the 4.16.0.0 milestone Feb 18, 2021
@GutoVeronezi

Copy link
Copy Markdown
ContributorAuthor

@GutoVeronezi
it is better to create a mysql procedure which can be used for any hypervisor and guest os.

@weizhouapache I turned it into a mysql procedure, as you suggested.
Thanks for the review.

@shwstpprshwstppr left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@GutoVeronezi will it make sense to target these changes for 4.15.1?

Comment on lines +338 to +343
CALL ADD_GUEST_OS_AND_HYPERVISOR_MAPPING (1, 'Ubuntu 20.04 LTS', 'KVM', 'default', 'Ubuntu 20.04 LTS');
CALL ADD_GUEST_OS_AND_HYPERVISOR_MAPPING (1, 'Ubuntu 21.04', 'KVM', 'default', 'Ubuntu 21.04');
CALL ADD_GUEST_OS_AND_HYPERVISOR_MAPPING (1, 'pfSense 2.4', 'KVM', 'default', 'pfSense 2.4');
CALL ADD_GUEST_OS_AND_HYPERVISOR_MAPPING (1, 'OpenBSD 6.7', 'KVM', 'default', 'OpenBSD 6.7');
CALL ADD_GUEST_OS_AND_HYPERVISOR_MAPPING (1, 'OpenBSD 6.8', 'KVM', 'default', 'OpenBSD 6.8');
CALL ADD_GUEST_OS_AND_HYPERVISOR_MAPPING (1, 'AlmaLinux 8.3', 'KVM', 'default', 'AlmaLinux 8.3'); No newline at end of file

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

category_id used here for each entry is 1 while for Ubuntu releases we have 10. Not sure for others it should be 7 or 9. It is 9 for FreeBSD in current db

AND hypervisor.guest_os_id = guest_os.id
AND hypervisor.guest_os_name = guest_os_hypervisor_guest_os_name);
END;

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@GutoVeronezi
this procedure might work in mysql, but I think it will not in cloudstack upgrade.
";" should not be the last character of a line in a procedure in schema-XXX.sql.

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@GutoVeronezi
what I said is not clear.
I mean you need to change line 322 and 334 which have ";" at the end of line.
line 335 was ok (it is not ok now).

for line 322, you can move the ";" at the end of line, to be beginning of line 324.
for line 334, you can move the ";" to the position before "END;".

Copy link
Copy Markdown
ContributorAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@weizhouapache done.

I misunderstood it at first, but now i see.

I took a look into the file that reads and execute the SQL script and wondered why it was wrote that way.

@GabrielBrascherGabrielBrascher left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code LGTM.

@GutoVeronezi

Copy link
Copy Markdown
ContributorAuthor

@weizhouapache@shwstppr is there anything else to do?

Daniel Augusto Veronezi Salvadorand others added 5 commits April 19, 2021 18:05
@GutoVeronezi
GutoVeroneziforce-pushed the add-new-guest-os-registers-to-database branch from f8698c5 to fb26e23CompareApril 19, 2021 21:07
@GutoVeronezi

Copy link
Copy Markdown
ContributorAuthor

@weizhouapache@shwstppr@GabrielBrascher is there anything else to do?

@GutoVeronezi

Copy link
Copy Markdown
ContributorAuthor

@weizhouapache can you look at this?

@GutoVeronezi

Copy link
Copy Markdown
ContributorAuthor

@rhtyd as it can be a solution for issue #3609, could you take look at this?

@yadvr

Copy link
Copy Markdown
Member

cc @harikrishna-patnala I think you are working on a design doc on improving guest OS mapping support code-base wide, you may refer to this.

@harikrishna-patnala

Copy link
Copy Markdown
Member

@rhtyd yes, I've already referred this PR in that document. Thanks.

@GutoVeronezi

Copy link
Copy Markdown
ContributorAuthor

@GabrielBrascher@shwstppr@weizhouapache is there anything else to do?

@yadvr

Copy link
Copy Markdown
Member

@harikrishna-patnala can you review this PR - is this good to go or needs to wait for your design proposal/changes?

@GutoVeronezi

Copy link
Copy Markdown
ContributorAuthor

@harikrishna-patnala any update about @rhtyd's comment?

@nvazquez

Copy link
Copy Markdown
Contributor

@blueorangutan package

@blueorangutan

Copy link
Copy Markdown

@nvazquez a Jenkins job has been kicked to build packages. I'll keep you posted as I make progress.

@blueorangutan

Copy link
Copy Markdown

Packaging result: ✔️ el7 ✔️ el8 ✔️ debian. SL-JID 753

@nvazquez

Copy link
Copy Markdown
Contributor

@blueorangutan test

@blueorangutan

Copy link
Copy Markdown

@nvazquez a Trillian-Jenkins test job (centos7 mgmt + kvm-centos7) has been kicked to run smoke tests

@blueorangutan

Copy link
Copy Markdown

Trillian Build Failed (tid-1481)

@GutoVeronezi

Copy link
Copy Markdown
ContributorAuthor

@nvazquez could we try to run the tests again?

@nvazquez

Copy link
Copy Markdown
Contributor

@blueorangutan test

@blueorangutan

Copy link
Copy Markdown

@nvazquez a Trillian-Jenkins test job (centos7 mgmt + kvm-centos7) has been kicked to run smoke tests

@blueorangutan

Copy link
Copy Markdown

Trillian Build Failed (tid-1564)

@DaanHoogland

Copy link
Copy Markdown
Contributor

@blueorangutan package

@blueorangutan

Copy link
Copy Markdown

@DaanHoogland a Jenkins job has been kicked to build packages. I'll keep you posted as I make progress.

@blueorangutan

Copy link
Copy Markdown

Packaging result: ✔️ el7 ✔️ el8 ✔️ debian. SL-JID 839

@DaanHoogland

Copy link
Copy Markdown
Contributor

@blueorangutan test

@blueorangutan

Copy link
Copy Markdown

@DaanHoogland a Trillian-Jenkins test job (centos7 mgmt + kvm-centos7) has been kicked to run smoke tests

@blueorangutan

Copy link
Copy Markdown

Trillian Build Failed (tid-1603)

@DaanHoogland

Copy link
Copy Markdown
Contributor

@blueorangutan test keepEnv

@blueorangutan

Copy link
Copy Markdown

@DaanHoogland a Trillian-Jenkins test job (centos7 mgmt + kvm-centos7) has been kicked to run smoke tests

@blueorangutan

Copy link
Copy Markdown

Trillian Build Failed (tid-1605)

@nvazquez

Copy link
Copy Markdown
Contributor

Hi @GutoVeronezi the failures are consistent, I have checked is related to an error on the procedure creation, can you please check?

2021-08-11 16:28:18,001 ERROR [c.c.u.d.ScriptRunner] (main:null) (logid:) Error executing: CREATE PROCEDURE `cloud`.`ADD_GUEST_OS_AND_HYPERVISOR_MAPPING` ( IN guest_os_category_id bigint(20) unsigned, IN guest_os_display_name VARCHAR(255), IN guest_os_hypervisor_hypervisor_type VARCHAR(32), IN guest_os_hypervisor_hypervisor_version VARCHAR(32), IN guest_os_hypervisor_guest_os_name VARCHAR(255) ) BEGIN INSERT INTO cloud.guest_os (uuid, category_id, display_name, created) SELECT UUID(), guest_os_category_id, guest_os_display_name, now() WHERE not exists( SELECT 1 FROM cloud.guest_os WHERE cloud.guest_os.category_id = guest_os_category_id AND cloud.guest_os.display_name = guest_os_display_name) ;	INSERT INTO cloud.guest_os_hypervisor (uuid, hypervisor_type, hypervisor_version, guest_os_name, guest_os_id, created) SELECT UUID(), guest_os_hypervisor_hypervisor_type, guest_os_hypervisor_hypervisor_version, guest_os_hypervisor_guest_os_name, guest_os.id, now() FROM cloud.guest_os WHERE guest_os.category_id = guest_os_category_id AND guest_os.display_name = guest_os_display_name AND	NOT EXISTS (SELECT 1 FROM cloud.guest_os_hypervisor as hypervisor WHERE hypervisor_type = guest_os_hypervisor_hypervisor_type AND hypervisor_version = guest_os_hypervisor_hypervisor_version AND hypervisor.guest_os_id = guest_os.id AND hypervisor.guest_os_name = guest_os_hypervisor_guest_os_name) ;END 2021-08-11 16:28:18,001 ERROR [c.c.u.d.ScriptRunner] (main:null) (logid:) java.sql.SQLSyntaxErrorException: You have an error in your SQL syntax; check the manual that corresponds to your MariaDB server version for the right syntax to use near 'WHERE not exists( SELECT 1 FROM cloud.guest_os ' at line 1
2021-08-11 16:28:18,004 ERROR [c.c.u.DatabaseUpgradeChecker] (main:null) (logid:) Unable to execute upgrade script
java.sql.SQLSyntaxErrorException: You have an error in your SQL syntax; check the manual that corresponds to your MariaDB server version for the right syntax to use near 'WHERE not exists( SELECT 1 FROM cloud.guest_os ' at line 1
at com.cloud.utils.db.ScriptRunner.runScript(ScriptRunner.java:185)
at com.cloud.utils.db.ScriptRunner.runScript(ScriptRunner.java:87)
at com.cloud.upgrade.DatabaseUpgradeChecker.runScript(DatabaseUpgradeChecker.java:209)
at com.cloud.upgrade.DatabaseUpgradeChecker.upgrade(DatabaseUpgradeChecker.java:295)
at com.cloud.upgrade.DatabaseUpgradeChecker.check(DatabaseUpgradeChecker.java:379)
at org.apache.cloudstack.spring.lifecycle.CloudStackExtendedLifeCycle.checkIntegrity(CloudStackExtendedLifeCycle.java:64)
at org.apache.cloudstack.spring.lifecycle.CloudStackExtendedLifeCycle.start(CloudStackExtendedLifeCycle.java:54)
at org.springframework.context.support.DefaultLifecycleProcessor.doStart(DefaultLifecycleProcessor.java:178)
at org.springframework.context.support.DefaultLifecycleProcessor.access$200(DefaultLifecycleProcessor.java:54)
at org.springframework.context.support.DefaultLifecycleProcessor$LifecycleGroup.start(DefaultLifecycleProcessor.java:356)
at java.base/java.lang.Iterable.forEach(Iterable.java:75)
at org.springframework.context.support.DefaultLifecycleProcessor.startBeans(DefaultLifecycleProcessor.java:155)
at org.springframework.context.support.DefaultLifecycleProcessor.onRefresh(DefaultLifecycleProcessor.java:123)
at org.springframework.context.support.AbstractApplicationContext.finishRefresh(AbstractApplicationContext.java:940)
at org.springframework.context.support.AbstractApplicationContext.refresh(AbstractApplicationContext.java:591)
at org.apache.cloudstack.spring.module.model.impl.DefaultModuleDefinitionSet.loadContext(DefaultModuleDefinitionSet.java:144)
at org.apache.cloudstack.spring.module.model.impl.DefaultModuleDefinitionSet$2.with(DefaultModuleDefinitionSet.java:121)
at org.apache.cloudstack.spring.module.model.impl.DefaultModuleDefinitionSet.withModule(DefaultModuleDefinitionSet.java:244)
at org.apache.cloudstack.spring.module.model.impl.DefaultModuleDefinitionSet.withModule(DefaultModuleDefinitionSet.java:249)
at org.apache.cloudstack.spring.module.model.impl.DefaultModuleDefinitionSet.withModule(DefaultModuleDefinitionSet.java:232)
at org.apache.cloudstack.spring.module.model.impl.DefaultModuleDefinitionSet.loadContexts(DefaultModuleDefinitionSet.java:116)
at org.apache.cloudstack.spring.module.model.impl.DefaultModuleDefinitionSet.load(DefaultModuleDefinitionSet.java:78)
at org.apache.cloudstack.spring.module.factory.ModuleBasedContextFactory.loadModules(ModuleBasedContextFactory.java:37)
at org.apache.cloudstack.spring.module.factory.CloudStackSpringContext.init(CloudStackSpringContext.java:70)
at org.apache.cloudstack.spring.module.factory.CloudStackSpringContext.<init>(CloudStackSpringContext.java:57)
at org.apache.cloudstack.spring.module.factory.CloudStackSpringContext.<init>(CloudStackSpringContext.java:61)
at org.apache.cloudstack.spring.module.web.CloudStackContextLoaderListener.contextInitialized(CloudStackContextLoaderListener.java:51)
at org.eclipse.jetty.server.handler.ContextHandler.callContextInitialized(ContextHandler.java:1068)
at org.eclipse.jetty.servlet.ServletContextHandler.callContextInitialized(ServletContextHandler.java:572)
at org.eclipse.jetty.server.handler.ContextHandler.contextInitialized(ContextHandler.java:997)
at org.eclipse.jetty.servlet.ServletHandler.initialize(ServletHandler.java:746)
at org.eclipse.jetty.servlet.ServletContextHandler.startContext(ServletContextHandler.java:379)
at org.eclipse.jetty.webapp.WebAppContext.startWebapp(WebAppContext.java:1457)
at org.eclipse.jetty.webapp.WebAppContext.startContext(WebAppContext.java:1422)
at org.eclipse.jetty.server.handler.ContextHandler.doStart(ContextHandler.java:911)
at org.eclipse.jetty.servlet.ServletContextHandler.doStart(ServletContextHandler.java:288)
at org.eclipse.jetty.webapp.WebAppContext.doStart(WebAppContext.java:524)
at org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:73)
at org.eclipse.jetty.util.component.ContainerLifeCycle.start(ContainerLifeCycle.java:169)
at org.eclipse.jetty.util.component.ContainerLifeCycle.doStart(ContainerLifeCycle.java:110)
at org.eclipse.jetty.server.handler.AbstractHandler.doStart(AbstractHandler.java:97)
at org.eclipse.jetty.server.handler.gzip.GzipHandler.doStart(GzipHandler.java:426)
at org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:73)
at org.eclipse.jetty.util.component.ContainerLifeCycle.start(ContainerLifeCycle.java:169)
at org.eclipse.jetty.util.component.ContainerLifeCycle.doStart(ContainerLifeCycle.java:117)
at org.eclipse.jetty.server.handler.AbstractHandler.doStart(AbstractHandler.java:97)
at org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:73)
at org.eclipse.jetty.util.component.ContainerLifeCycle.start(ContainerLifeCycle.java:169)
at org.eclipse.jetty.server.Server.start(Server.java:423)
at org.eclipse.jetty.util.component.ContainerLifeCycle.doStart(ContainerLifeCycle.java:110)
at org.eclipse.jetty.server.handler.AbstractHandler.doStart(AbstractHandler.java:97)
at org.eclipse.jetty.server.Server.doStart(Server.java:387)
at org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:73)
at org.apache.cloudstack.ServerDaemon.start(ServerDaemon.java:188)
at org.apache.cloudstack.ServerDaemon.main(ServerDaemon.java:104)
2021-08-11 16:28:18,006 ERROR [c.c.u.DatabaseUpgradeChecker] (main:null) (logid:) Unable to upgrade the database
com.cloud.utils.exception.CloudRuntimeException: Unable to execute upgrade script
at com.cloud.upgrade.DatabaseUpgradeChecker.runScript(DatabaseUpgradeChecker.java:215)
at com.cloud.upgrade.DatabaseUpgradeChecker.upgrade(DatabaseUpgradeChecker.java:295)
at com.cloud.upgrade.DatabaseUpgradeChecker.check(DatabaseUpgradeChecker.java:379)
at org.apache.cloudstack.spring.lifecycle.CloudStackExtendedLifeCycle.checkIntegrity(CloudStackExtendedLifeCycle.java:64)
at org.apache.cloudstack.spring.lifecycle.CloudStackExtendedLifeCycle.start(CloudStackExtendedLifeCycle.java:54)
at org.springframework.context.support.DefaultLifecycleProcessor.doStart(DefaultLifecycleProcessor.java:178)
at org.springframework.context.support.DefaultLifecycleProcessor.access$200(DefaultLifecycleProcessor.java:54)
at org.springframework.context.support.DefaultLifecycleProcessor$LifecycleGroup.start(DefaultLifecycleProcessor.java:356)
at java.base/java.lang.Iterable.forEach(Iterable.java:75)
at org.springframework.context.support.DefaultLifecycleProcessor.startBeans(DefaultLifecycleProcessor.java:155)
at org.springframework.context.support.DefaultLifecycleProcessor.onRefresh(DefaultLifecycleProcessor.java:123)
at org.springframework.context.support.AbstractApplicationContext.finishRefresh(AbstractApplicationContext.java:940)
at org.springframework.context.support.AbstractApplicationContext.refresh(AbstractApplicationContext.java:591)
at org.apache.cloudstack.spring.module.model.impl.DefaultModuleDefinitionSet.loadContext(DefaultModuleDefinitionSet.java:144)
at org.apache.cloudstack.spring.module.model.impl.DefaultModuleDefinitionSet$2.with(DefaultModuleDefinitionSet.java:121)
at org.apache.cloudstack.spring.module.model.impl.DefaultModuleDefinitionSet.withModule(DefaultModuleDefinitionSet.java:244)
at org.apache.cloudstack.spring.module.model.impl.DefaultModuleDefinitionSet.withModule(DefaultModuleDefinitionSet.java:249)
at org.apache.cloudstack.spring.module.model.impl.DefaultModuleDefinitionSet.withModule(DefaultModuleDefinitionSet.java:232)
at org.apache.cloudstack.spring.module.model.impl.DefaultModuleDefinitionSet.loadContexts(DefaultModuleDefinitionSet.java:116)
at org.apache.cloudstack.spring.module.model.impl.DefaultModuleDefinitionSet.load(DefaultModuleDefinitionSet.java:78)
at org.apache.cloudstack.spring.module.factory.ModuleBasedContextFactory.loadModules(ModuleBasedContextFactory.java:37)
at org.apache.cloudstack.spring.module.factory.CloudStackSpringContext.init(CloudStackSpringContext.java:70)
at org.apache.cloudstack.spring.module.factory.CloudStackSpringContext.<init>(CloudStackSpringContext.java:57)
at org.apache.cloudstack.spring.module.factory.CloudStackSpringContext.<init>(CloudStackSpringContext.java:61)
at org.apache.cloudstack.spring.module.web.CloudStackContextLoaderListener.contextInitialized(CloudStackContextLoaderListener.java:51)
at org.eclipse.jetty.server.handler.ContextHandler.callContextInitialized(ContextHandler.java:1068)
at org.eclipse.jetty.servlet.ServletContextHandler.callContextInitialized(ServletContextHandler.java:572)
at org.eclipse.jetty.server.handler.ContextHandler.contextInitialized(ContextHandler.java:997)
at org.eclipse.jetty.servlet.ServletHandler.initialize(ServletHandler.java:746)
at org.eclipse.jetty.servlet.ServletContextHandler.startContext(ServletContextHandler.java:379)
at org.eclipse.jetty.webapp.WebAppContext.startWebapp(WebAppContext.java:1457)
at org.eclipse.jetty.webapp.WebAppContext.startContext(WebAppContext.java:1422)
at org.eclipse.jetty.server.handler.ContextHandler.doStart(ContextHandler.java:911)
at org.eclipse.jetty.servlet.ServletContextHandler.doStart(ServletContextHandler.java:288)
at org.eclipse.jetty.webapp.WebAppContext.doStart(WebAppContext.java:524)
at org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:73)
at org.eclipse.jetty.util.component.ContainerLifeCycle.start(ContainerLifeCycle.java:169)
at org.eclipse.jetty.util.component.ContainerLifeCycle.doStart(ContainerLifeCycle.java:110)
at org.eclipse.jetty.server.handler.AbstractHandler.doStart(AbstractHandler.java:97)
at org.eclipse.jetty.server.handler.gzip.GzipHandler.doStart(GzipHandler.java:426)
at org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:73)
at org.eclipse.jetty.util.component.ContainerLifeCycle.start(ContainerLifeCycle.java:169)
at org.eclipse.jetty.util.component.ContainerLifeCycle.doStart(ContainerLifeCycle.java:117)
at org.eclipse.jetty.server.handler.AbstractHandler.doStart(AbstractHandler.java:97)
at org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:73)
at org.eclipse.jetty.util.component.ContainerLifeCycle.start(ContainerLifeCycle.java:169)
at org.eclipse.jetty.server.Server.start(Server.java:423)
at org.eclipse.jetty.util.component.ContainerLifeCycle.doStart(ContainerLifeCycle.java:110)
at org.eclipse.jetty.server.handler.AbstractHandler.doStart(AbstractHandler.java:97)
at org.eclipse.jetty.server.Server.doStart(Server.java:387)
at org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:73)
at org.apache.cloudstack.ServerDaemon.start(ServerDaemon.java:188)
at org.apache.cloudstack.ServerDaemon.main(ServerDaemon.java:104)
Caused by: java.sql.SQLSyntaxErrorException: You have an error in your SQL syntax; check the manual that corresponds to your MariaDB server version for the right syntax to use near 'WHERE not exists( SELECT 1 FROM cloud.guest_os ' at line 1
at com.cloud.utils.db.ScriptRunner.runScript(ScriptRunner.java:185)
at com.cloud.utils.db.ScriptRunner.runScript(ScriptRunner.java:87)
at com.cloud.upgrade.DatabaseUpgradeChecker.runScript(DatabaseUpgradeChecker.java:209)
... 52 more
2021-08-11 16:28:18,031 DEBUG [c.c.u.d.T.Transaction] (main:null) (logid:) Rolling back the transaction: Time = 263 Name = Upgrade; called by -TransactionLegacy.rollback:888-TransactionLegacy.removeUpTo:831-TransactionLegacy.close:655-DatabaseUpgradeChecker.upgrade:310-DatabaseUpgradeChecker.check:379-CloudStackExtendedLifeCycle.checkIntegrity:64-CloudStackExtendedLifeCycle.start:54-DefaultLifecycleProcessor.doStart:178-DefaultLifecycleProcessor.access$200:54-DefaultLifecycleProcessor$LifecycleGroup.start:356-Iterable.forEach:75-DefaultLifecycleProcessor.startBeans:155

@GutoVeronezi

Copy link
Copy Markdown
ContributorAuthor

@nvazquez as we discussed offline, the environment where I ran the tests and the environment where the tests ran in the CI are different.

Mine uses:

mysql Ver 14.14 Distrib 5.7.35, for Linux (x86_64) using EditLine wrapper

Tests uses:

mysql Ver 15.1 Distrib 5.5.68-MariaDB, for Linux (x86_64) using readline 5.1

I did some tests with MariaDB and found the following situation:

MySQL (which I was using) doesn't requires a from clause with a dummy table, even if we are using a where clause MySQL doc. MariaDB (that the CI tests were using), although its doc says FROM DUAL could be used..., which would indicate that it is not mandatory, in reality requires a dummy table when we use a where clause.

Therefore, I added the FROM DUAL clause to allow it to run in MariaDB.

@nvazquez

Copy link
Copy Markdown
Contributor

Thanks for the quick fix @GutoVeronezi. Let's kick another round of tests
@blueorangutan package

@blueorangutan

Copy link
Copy Markdown

@nvazquez a Jenkins job has been kicked to build packages. I'll keep you posted as I make progress.

@blueorangutan

Copy link
Copy Markdown

Packaging result: ✔️ el7 ✔️ el8 ✔️ debian. SL-JID 850

@nvazquez

Copy link
Copy Markdown
Contributor

@blueorangutan test

@blueorangutan

Copy link
Copy Markdown

@nvazquez a Trillian-Jenkins test job (centos7 mgmt + kvm-centos7) has been kicked to run smoke tests

@blueorangutan

Copy link
Copy Markdown

Trillian test result (tid-1613)
Environment: kvm-centos7 (x2), Advanced Networking with Mgmt server 7
Total time taken: 52614 seconds
Marvin logs: https://github.com/blueorangutan/acs-prs/releases/download/trillian/pr4699-t1613-kvm-centos7.zip
Intermittent failure detected: /marvin/tests/smoke/test_routers_network_ops.py
Intermittent failure detected: /marvin/tests/smoke/test_vm_life_cycle.py
Intermittent failure detected: /marvin/tests/smoke/test_volumes.py
Intermittent failure detected: /marvin/tests/smoke/test_vpc_redundant.py
Intermittent failure detected: /marvin/tests/smoke/test_vpc_vpn.py
Smoke tests completed. 87 look OK, 2 have error(s)
Only failed tests results shown below:

TestResultTime (s)Test File
test_01_create_redundant_VPC_2tiers_4VMs_4IPs_4PF_ACLFailure741.53test_vpc_redundant.py
test_02_redundant_VPC_default_routesFailure942.32test_vpc_redundant.py
test_03_create_redundant_VPC_1tier_2VMs_2IPs_2PF_ACL_reboot_routersFailure635.85test_vpc_redundant.py
test_05_rvpc_multi_tiersFailure548.34test_vpc_redundant.py
test_01_redundant_vpc_site2site_vpnFailure414.67test_vpc_vpn.py

@nvazqueznvazquez left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@nvazquez
nvazquez merged commit 2786582 into apache:mainAug 12, 2021
Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

11 participants

@GutoVeronezi@weizhouapache@yadvr@harikrishna-patnala@DaanHoogland@nvazquez@blueorangutan@shwstppr@sureshanaparti@Spaceman1984@GabrielBrascher