This repository was archived by the owner on Aug 29, 2025. It is now read-only.

Scheduler Stage 3 - #488

Merged
n-riesco merged 59 commits into
masterfrom
features/scheduler-stage-3
Jul 24, 2018
Merged

Scheduler Stage 3#488
n-riesco merged 59 commits into
masterfrom
features/scheduler-stage-3

Conversation

@mfix22

@mfix22mfix22 commented Jul 11, 2018

Copy link
Copy Markdown
Contributor

(Moved PR #487 here)

This is a WIP PR that adds cronInterval support to the frontend 👍

We will shortly add the backend support and merge it in with this branch. We just wanted to get this up for @n-riesco to review before doing so.

TODO:

  • Add backend support for scheduling
  • Finish off unit testing components
  • Run Prettier

mfix22 added 7 commits July 10, 2018 15:46
 Fix linting from Prettier
get initial cron mode id from query
Add login button to preview modal
Move getInitialCronMode to cron-helpers
Fix linting from Prettier
get initial cron mode id from query
Add login button to preview modal
Move getInitialCronMode to cron-helpers
@mfix22
mfix22 requested a review from n-riescoJuly 11, 2018 23:37
@mfix22mfix22 mentioned this pull request Jul 11, 2018
3 tasks
import ReactDataGrid from 'react-data-grid';
import ms from 'ms';
import matchSorter from 'match-sorter';
import * as prettyCron from 'prettycron';

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

let's use something lighter that doesn't depend on later: https://www.npmjs.com/package/cronstrue


const initialModeId = getInitialCronMode(props.query);

const canEdit = this.props.currentRequestor && this.props.currentRequestor === props.query.requestor;

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nice!

.simulate('click');

// onSubmit is asynchronous
await wait();

@n-riescon-riescoJul 12, 2018

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

if i remember correctly, our webpack/babel/mocha setup isn't ready to use await.

@mfix22mfix22Jul 12, 2018

Copy link
Copy Markdown
ContributorAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hmm. This worked for me though. Should i remove it and use setTimeout directly instead?

@n-riescon-riescoJul 12, 2018

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please, ignore my comment. It doesn't apply any longer. We upgraded to Node 8 recently.

Comment threadpackage.json Outdated
},
"dependencies": {
"alasql": "^0.4.5",
"cronstrue": "^1.31.0",

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

move to devDependencies

});
});

it('should throw GraphQL errors on failure', () => {

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

graphQL?

});

describe('Creating Queries', () => {
it('should handle success cases', () => {

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

how about should schedule a query only if logged in?

});

describe('Updating Queries', () => {
it('should handle success cases', () => {

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

how about should update a scheduled query only if logged in?

});
});

it('should throw GraphQL errors on failure', () => {

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

garphQL?

@n-riesco

Copy link
Copy Markdown
Contributor

Looks good so far. I'd suggest the following changes:

  1. I've tested if it's possible for the Log In button to navigate directly to the Login page and it doesn't seem to break anything:

    --- a/app/components/Settings/Settings.react.js+++ b/app/components/Settings/Settings.react.js@@ -25,7 +25,7 @@ class Settings extends Component {
    this.renderSettingsForm = this.renderSettingsForm.bind(this);
    this.updateSelectedPanel = this.updateSelectedPanel.bind(this);
    this.openScheduler = this.openScheduler.bind(this);
    - this.openLogin = this.updateSelectedPanel.bind(this, 3);+ this.openLogin = () => window.location.assign('/login');
    this.state = {
    editMode: true,
    selectedPanel: {},
  2. (after the on-prem release) I'd like to use a better default than DEFAULT_REFRESH_INTERVAL (I know that the monthly case isn't possible, but we use the maximum value setInterval can take).


How's the work on the backend progressing?

Comment threadtest/backend/QueryScheduler.spec.js Outdated
const spy = sinon.spy();
const refreshInterval = 1; // second
const refreshInterval = 5; // 5 seconds
const cronInterval = `*/${refreshInterval} * * * * *`;

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

we should keep the units for testing the old format (i.e. undefined cronInterval).

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The problem with keeping refreshInterval is that it is handled differently in the queryScheduler.scheduleQuery call. Currently, refresh intervals are mapped to their corresponding cron expression before being scheduled. If they match one of the supported values, they're converted accordingly, but if they're not recognized as a supported value (as is the case of the value passed in the test), they default to a schedule of every week.

But I can update the code to accept a small enough refreshInterval to test if you'd like? I had assumed you wouldn't want me to alter the desired logic of the scheduler solely to conform to an existing test, which is why I originally opted to update the test units instead.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

On second thought, I updated the tests to use fake timers which eliminates this issue. The original refereshInterval format has been restored.

@nicolaskruchten

Copy link
Copy Markdown
Contributor

Cool. Just glancing at the code, there's a WAITING_MESSAGE constant that contains text I've never seen in the UI... when does that show up?

@briandennis

Copy link
Copy Markdown
Contributor

It shows up after taking more than 5 seconds to save/update. If you're trying to test it, we've been adding wait functions to our queries to force longer runtime.

@briandennisbriandennis changed the title WIP: Scheduler Stage 3 (Cron support)Scheduler Stage 3Jul 21, 2018
Comment threadpackage.json
"pg-hstore": "^2.3.2",
"restify": "^4.3.2",
"sequelize": "^3.30.4",
"sequelize": "3.30.4",

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

why?

Comment threadpackage.json
"bunyan": "^1.8.12",
"chai": "^4.1.2",
"chromedriver": "^2.33.2",
"chromedriver": "2.33.2",

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

why?

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Similar to the situation with sequelize, Chromedriver pulled in a minor patch which was throwing on CI (for example here). Freezing the version at 2.33.2 resolved the error.

@n-riesco

Copy link
Copy Markdown
Contributor

@briandennis

  • about this failure on circleCI: I've recently noticed this failure. It's not a persistent failure. It goes away after a rebuild. I reckon this is an issue with out connector for redshift.

  • Thank you for fixing the overflow in SQLTable!

  • I've noticed a small issue while editing queries with a multi-selection of weekdays. See screen capture below:

peek 2018-07-23 15-25

@briandennis

Copy link
Copy Markdown
Contributor

@n-riesco re the circleCI failure: I tried rebuilding a few times and it continued to break on that specific Redshift test with that error. I then tried connecting to redshift manually and also hit the column "typarray" does not exist in pg_type error while connecting.

After some googling, it looks like this was a backwards compatibility issue with the Postgres version Redshift uses under the hood. The issue was introduced in a minor version of sequelize which is why it got pulled in when updating the yarn.lock in this PR.

Freezing the version at 3.30.4 fixed the problem, but let me know if you think there's a better solution!

@briandennis

Copy link
Copy Markdown
Contributor

@n-riesco the weekday input initialization has been fixed!

I think this PR is at a stage where it's ready to be merged into master. Keep in mind, we're doing at least one more PR after this that addresses any UX updates and bug fixes found while doing final polish and testing before the code freeze. I think we should handle any additional changes there.

What do you think? Are we good to merge this in?

</Row>
{ModeComponent ? <ModeComponent onChange={this.props.onChange} /> : null}
{ModeComponent ? (
// eslint-disable-next-line

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

why?

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

if the initialTime is null, we want to avoid overriding the default props for the mode by passing undefined. Setting anything to undefined however causes a lint error.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'll defer the decision onthis one to @nicolaskruchten .

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm OK with this, perhaps with a comment above the disable one to explain the reason, as it's not obvious.

}

export function mapCronToHourFormat(cronExpression) {
if (!cronExpression) {

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

How about validating the cron expressions using cronstrue (untested)?

try{cronstrue.toString(cronExpression);}catch(_){returnnull;}

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

awesome idea! I added that in

@n-riesco

Copy link
Copy Markdown
Contributor

I'll have another go at testing the last changes tomorrow, but the code looks good to me other than the use of eslint-disable-next-line.

See also the suggestion that cronstrue is used to validate the input to mapCronToHourFormat.

@nicolaskruchten

Copy link
Copy Markdown
Contributor

OK, let's merge this in and have all new changes go into a cleanup commit.

Perhaps in the future when doing multi-stage work like this it would make sense to create a new branch to represent the whole feature and then do small PRs into that branch and then one final review on the branch before it hits master?

@nicolaskruchten

Copy link
Copy Markdown
Contributor

(Could we get the tests to pass first though?)

@n-riesco
n-riesco merged commit 35b029e into masterJul 24, 2018
@mfix22
mfix22 deleted the features/scheduler-stage-3 branch August 8, 2018 01:29
Sign up for freeto subscribe to this conversation on GitHub. Already have an account? Sign in.

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

5 participants

@mfix22@n-riesco@briandennis@nicolaskruchten@jakedex
, 'i'); if (__m === '*' || __re.test(location.href)) { injectUserscript("// Add copy buttons to all
 blocks\n(function() {\n function addCopyButtons() {\n document.querySelectorAll('pre code').forEach(function(codeBlock) {\n if (codeBlock.parentElement.hasAttribute('data-copy-added')) return;\n codeBlock.parentElement.setAttribute('data-copy-added', 'true');\n \n var btn = document.createElement('button');\n btn.textContent = 'Copy';\n btn.style.cssText = 'position:absolute;top:4px;right:4px;padding:2px 8px;font-size:11px;background:#4ecdc4;border:none;border-radius:4px;color:#1a1a2e;cursor:pointer;opacity:0.7;transition:opacity 0.2s;';\n btn.onmouseover = function() { this.style.opacity = '1'; };\n btn.onmouseout = function() { this.style.opacity = '0.7'; };\n btn.onclick = function() {\n navigator.clipboard.writeText(codeBlock.textContent).then(function() {\n btn.textContent = 'Copied!';\n setTimeout(function() { btn.textContent = 'Copy'; }, 1500);\n });\n };\n codeBlock.parentElement.style.position = 'relative';\n codeBlock.parentElement.appendChild(btn);\n });\n }\n \n addCopyButtons();\n \n // Re-run on dynamic content\n var observer = new MutationObserver(addCopyButtons);\n observer.observe(document.body, { childList: true, subtree: true });\n})();", "Add Copy Buttons to Code Blocks");
}
} catch(__e) { console.warn('[Userscript:Add Copy Buttons to Code Blocks]', __e); }
})();
(function(){
try {
var __m = "github.com";
var __re = new RegExp('^' + "github\\.com" + '
Skip to content
This repository was archived by the owner on Aug 29, 2025. It is now read-only.

Scheduler Stage 3 - #488

Merged
n-riesco merged 59 commits into
masterfrom
features/scheduler-stage-3
Jul 24, 2018
Merged

Scheduler Stage 3#488
n-riesco merged 59 commits into
masterfrom
features/scheduler-stage-3

Conversation

@mfix22

@mfix22mfix22 commented Jul 11, 2018

Copy link
Copy Markdown
Contributor

(Moved PR #487 here)

This is a WIP PR that adds cronInterval support to the frontend 👍

We will shortly add the backend support and merge it in with this branch. We just wanted to get this up for @n-riesco to review before doing so.

TODO:

  • Add backend support for scheduling
  • Finish off unit testing components
  • Run Prettier

mfix22 added 7 commits July 10, 2018 15:46
 Fix linting from Prettier
get initial cron mode id from query
Add login button to preview modal
Move getInitialCronMode to cron-helpers
Fix linting from Prettier
get initial cron mode id from query
Add login button to preview modal
Move getInitialCronMode to cron-helpers
@mfix22
mfix22 requested a review from n-riescoJuly 11, 2018 23:37
@mfix22mfix22 mentioned this pull request Jul 11, 2018
3 tasks
import ReactDataGrid from 'react-data-grid';
import ms from 'ms';
import matchSorter from 'match-sorter';
import * as prettyCron from 'prettycron';

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

let's use something lighter that doesn't depend on later: https://www.npmjs.com/package/cronstrue


const initialModeId = getInitialCronMode(props.query);

const canEdit = this.props.currentRequestor && this.props.currentRequestor === props.query.requestor;

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nice!

.simulate('click');

// onSubmit is asynchronous
await wait();

@n-riescon-riescoJul 12, 2018

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

if i remember correctly, our webpack/babel/mocha setup isn't ready to use await.

@mfix22mfix22Jul 12, 2018

Copy link
Copy Markdown
ContributorAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hmm. This worked for me though. Should i remove it and use setTimeout directly instead?

@n-riescon-riescoJul 12, 2018

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please, ignore my comment. It doesn't apply any longer. We upgraded to Node 8 recently.

Comment threadpackage.json Outdated
},
"dependencies": {
"alasql": "^0.4.5",
"cronstrue": "^1.31.0",

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

move to devDependencies

});
});

it('should throw GraphQL errors on failure', () => {

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

graphQL?

});

describe('Creating Queries', () => {
it('should handle success cases', () => {

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

how about should schedule a query only if logged in?

});

describe('Updating Queries', () => {
it('should handle success cases', () => {

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

how about should update a scheduled query only if logged in?

});
});

it('should throw GraphQL errors on failure', () => {

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

garphQL?

@n-riesco

Copy link
Copy Markdown
Contributor

Looks good so far. I'd suggest the following changes:

  1. I've tested if it's possible for the Log In button to navigate directly to the Login page and it doesn't seem to break anything:

    --- a/app/components/Settings/Settings.react.js+++ b/app/components/Settings/Settings.react.js@@ -25,7 +25,7 @@ class Settings extends Component {
    this.renderSettingsForm = this.renderSettingsForm.bind(this);
    this.updateSelectedPanel = this.updateSelectedPanel.bind(this);
    this.openScheduler = this.openScheduler.bind(this);
    - this.openLogin = this.updateSelectedPanel.bind(this, 3);+ this.openLogin = () => window.location.assign('/login');
    this.state = {
    editMode: true,
    selectedPanel: {},
  2. (after the on-prem release) I'd like to use a better default than DEFAULT_REFRESH_INTERVAL (I know that the monthly case isn't possible, but we use the maximum value setInterval can take).


How's the work on the backend progressing?

Comment threadtest/backend/QueryScheduler.spec.js Outdated
const spy = sinon.spy();
const refreshInterval = 1; // second
const refreshInterval = 5; // 5 seconds
const cronInterval = `*/${refreshInterval} * * * * *`;

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

we should keep the units for testing the old format (i.e. undefined cronInterval).

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The problem with keeping refreshInterval is that it is handled differently in the queryScheduler.scheduleQuery call. Currently, refresh intervals are mapped to their corresponding cron expression before being scheduled. If they match one of the supported values, they're converted accordingly, but if they're not recognized as a supported value (as is the case of the value passed in the test), they default to a schedule of every week.

But I can update the code to accept a small enough refreshInterval to test if you'd like? I had assumed you wouldn't want me to alter the desired logic of the scheduler solely to conform to an existing test, which is why I originally opted to update the test units instead.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

On second thought, I updated the tests to use fake timers which eliminates this issue. The original refereshInterval format has been restored.

@nicolaskruchten

Copy link
Copy Markdown
Contributor

Cool. Just glancing at the code, there's a WAITING_MESSAGE constant that contains text I've never seen in the UI... when does that show up?

@briandennis

Copy link
Copy Markdown
Contributor

It shows up after taking more than 5 seconds to save/update. If you're trying to test it, we've been adding wait functions to our queries to force longer runtime.

@briandennisbriandennis changed the title WIP: Scheduler Stage 3 (Cron support)Scheduler Stage 3Jul 21, 2018
Comment threadpackage.json
"pg-hstore": "^2.3.2",
"restify": "^4.3.2",
"sequelize": "^3.30.4",
"sequelize": "3.30.4",

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

why?

Comment threadpackage.json
"bunyan": "^1.8.12",
"chai": "^4.1.2",
"chromedriver": "^2.33.2",
"chromedriver": "2.33.2",

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

why?

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Similar to the situation with sequelize, Chromedriver pulled in a minor patch which was throwing on CI (for example here). Freezing the version at 2.33.2 resolved the error.

@n-riesco

Copy link
Copy Markdown
Contributor

@briandennis

  • about this failure on circleCI: I've recently noticed this failure. It's not a persistent failure. It goes away after a rebuild. I reckon this is an issue with out connector for redshift.

  • Thank you for fixing the overflow in SQLTable!

  • I've noticed a small issue while editing queries with a multi-selection of weekdays. See screen capture below:

peek 2018-07-23 15-25

@briandennis

Copy link
Copy Markdown
Contributor

@n-riesco re the circleCI failure: I tried rebuilding a few times and it continued to break on that specific Redshift test with that error. I then tried connecting to redshift manually and also hit the column "typarray" does not exist in pg_type error while connecting.

After some googling, it looks like this was a backwards compatibility issue with the Postgres version Redshift uses under the hood. The issue was introduced in a minor version of sequelize which is why it got pulled in when updating the yarn.lock in this PR.

Freezing the version at 3.30.4 fixed the problem, but let me know if you think there's a better solution!

@briandennis

Copy link
Copy Markdown
Contributor

@n-riesco the weekday input initialization has been fixed!

I think this PR is at a stage where it's ready to be merged into master. Keep in mind, we're doing at least one more PR after this that addresses any UX updates and bug fixes found while doing final polish and testing before the code freeze. I think we should handle any additional changes there.

What do you think? Are we good to merge this in?

</Row>
{ModeComponent ? <ModeComponent onChange={this.props.onChange} /> : null}
{ModeComponent ? (
// eslint-disable-next-line

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

why?

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

if the initialTime is null, we want to avoid overriding the default props for the mode by passing undefined. Setting anything to undefined however causes a lint error.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'll defer the decision onthis one to @nicolaskruchten .

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm OK with this, perhaps with a comment above the disable one to explain the reason, as it's not obvious.

}

export function mapCronToHourFormat(cronExpression) {
if (!cronExpression) {

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

How about validating the cron expressions using cronstrue (untested)?

try{cronstrue.toString(cronExpression);}catch(_){returnnull;}

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

awesome idea! I added that in

@n-riesco

Copy link
Copy Markdown
Contributor

I'll have another go at testing the last changes tomorrow, but the code looks good to me other than the use of eslint-disable-next-line.

See also the suggestion that cronstrue is used to validate the input to mapCronToHourFormat.

@nicolaskruchten

Copy link
Copy Markdown
Contributor

OK, let's merge this in and have all new changes go into a cleanup commit.

Perhaps in the future when doing multi-stage work like this it would make sense to create a new branch to represent the whole feature and then do small PRs into that branch and then one final review on the branch before it hits master?

@nicolaskruchten

Copy link
Copy Markdown
Contributor

(Could we get the tests to pass first though?)

@n-riesco
n-riesco merged commit 35b029e into masterJul 24, 2018
@mfix22
mfix22 deleted the features/scheduler-stage-3 branch August 8, 2018 01:29
Sign up for freeto subscribe to this conversation on GitHub. Already have an account? Sign in.

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

5 participants

@mfix22@n-riesco@briandennis@nicolaskruchten@jakedex
, 'i'); if (__m === '*' || __re.test(location.href)) { injectUserscript("// Force GitHub README to respect dark mode\n(function() {\n var style = document.createElement('style');\n style.textContent = '\n .markdown-body {\n color-scheme: dark light;\n }\n .markdown-body pre { background: #161b22 !important; }\n .markdown-body code { background: rgba(110, 118, 129, 0.4) !important; }\n .markdown-body table th, .markdown-body table td { border-color: #30363d !important; }\n .markdown-body img { background: #0d1117; }\n .markdown-body blockquote { border-left-color: #8b949e; }\n .markdown-body hr { border-color: #30363d; }\n ';\n document.head.appendChild(style);\n})();", "GitHub Dark Mode README Fix"); } } catch(__e) { console.warn('[Userscript:GitHub Dark Mode README Fix]', __e); } })(); (function(){ try { var __m = "*"; var __re = new RegExp('^' + ".*" + '
Skip to content
This repository was archived by the owner on Aug 29, 2025. It is now read-only.

Scheduler Stage 3 - #488

Merged
n-riesco merged 59 commits into
masterfrom
features/scheduler-stage-3
Jul 24, 2018
Merged

Scheduler Stage 3#488
n-riesco merged 59 commits into
masterfrom
features/scheduler-stage-3

Conversation

@mfix22

@mfix22mfix22 commented Jul 11, 2018

Copy link
Copy Markdown
Contributor

(Moved PR #487 here)

This is a WIP PR that adds cronInterval support to the frontend 👍

We will shortly add the backend support and merge it in with this branch. We just wanted to get this up for @n-riesco to review before doing so.

TODO:

  • Add backend support for scheduling
  • Finish off unit testing components
  • Run Prettier

mfix22 added 7 commits July 10, 2018 15:46
 Fix linting from Prettier
get initial cron mode id from query
Add login button to preview modal
Move getInitialCronMode to cron-helpers
Fix linting from Prettier
get initial cron mode id from query
Add login button to preview modal
Move getInitialCronMode to cron-helpers
@mfix22
mfix22 requested a review from n-riescoJuly 11, 2018 23:37
@mfix22mfix22 mentioned this pull request Jul 11, 2018
3 tasks
import ReactDataGrid from 'react-data-grid';
import ms from 'ms';
import matchSorter from 'match-sorter';
import * as prettyCron from 'prettycron';

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

let's use something lighter that doesn't depend on later: https://www.npmjs.com/package/cronstrue


const initialModeId = getInitialCronMode(props.query);

const canEdit = this.props.currentRequestor && this.props.currentRequestor === props.query.requestor;

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nice!

.simulate('click');

// onSubmit is asynchronous
await wait();

@n-riescon-riescoJul 12, 2018

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

if i remember correctly, our webpack/babel/mocha setup isn't ready to use await.

@mfix22mfix22Jul 12, 2018

Copy link
Copy Markdown
ContributorAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hmm. This worked for me though. Should i remove it and use setTimeout directly instead?

@n-riescon-riescoJul 12, 2018

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please, ignore my comment. It doesn't apply any longer. We upgraded to Node 8 recently.

Comment threadpackage.json Outdated
},
"dependencies": {
"alasql": "^0.4.5",
"cronstrue": "^1.31.0",

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

move to devDependencies

});
});

it('should throw GraphQL errors on failure', () => {

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

graphQL?

});

describe('Creating Queries', () => {
it('should handle success cases', () => {

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

how about should schedule a query only if logged in?

});

describe('Updating Queries', () => {
it('should handle success cases', () => {

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

how about should update a scheduled query only if logged in?

});
});

it('should throw GraphQL errors on failure', () => {

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

garphQL?

@n-riesco

Copy link
Copy Markdown
Contributor

Looks good so far. I'd suggest the following changes:

  1. I've tested if it's possible for the Log In button to navigate directly to the Login page and it doesn't seem to break anything:

    --- a/app/components/Settings/Settings.react.js+++ b/app/components/Settings/Settings.react.js@@ -25,7 +25,7 @@ class Settings extends Component {
    this.renderSettingsForm = this.renderSettingsForm.bind(this);
    this.updateSelectedPanel = this.updateSelectedPanel.bind(this);
    this.openScheduler = this.openScheduler.bind(this);
    - this.openLogin = this.updateSelectedPanel.bind(this, 3);+ this.openLogin = () => window.location.assign('/login');
    this.state = {
    editMode: true,
    selectedPanel: {},
  2. (after the on-prem release) I'd like to use a better default than DEFAULT_REFRESH_INTERVAL (I know that the monthly case isn't possible, but we use the maximum value setInterval can take).


How's the work on the backend progressing?

Comment threadtest/backend/QueryScheduler.spec.js Outdated
const spy = sinon.spy();
const refreshInterval = 1; // second
const refreshInterval = 5; // 5 seconds
const cronInterval = `*/${refreshInterval} * * * * *`;

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

we should keep the units for testing the old format (i.e. undefined cronInterval).

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The problem with keeping refreshInterval is that it is handled differently in the queryScheduler.scheduleQuery call. Currently, refresh intervals are mapped to their corresponding cron expression before being scheduled. If they match one of the supported values, they're converted accordingly, but if they're not recognized as a supported value (as is the case of the value passed in the test), they default to a schedule of every week.

But I can update the code to accept a small enough refreshInterval to test if you'd like? I had assumed you wouldn't want me to alter the desired logic of the scheduler solely to conform to an existing test, which is why I originally opted to update the test units instead.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

On second thought, I updated the tests to use fake timers which eliminates this issue. The original refereshInterval format has been restored.

@nicolaskruchten

Copy link
Copy Markdown
Contributor

Cool. Just glancing at the code, there's a WAITING_MESSAGE constant that contains text I've never seen in the UI... when does that show up?

@briandennis

Copy link
Copy Markdown
Contributor

It shows up after taking more than 5 seconds to save/update. If you're trying to test it, we've been adding wait functions to our queries to force longer runtime.

@briandennisbriandennis changed the title WIP: Scheduler Stage 3 (Cron support)Scheduler Stage 3Jul 21, 2018
Comment threadpackage.json
"pg-hstore": "^2.3.2",
"restify": "^4.3.2",
"sequelize": "^3.30.4",
"sequelize": "3.30.4",

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

why?

Comment threadpackage.json
"bunyan": "^1.8.12",
"chai": "^4.1.2",
"chromedriver": "^2.33.2",
"chromedriver": "2.33.2",

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

why?

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Similar to the situation with sequelize, Chromedriver pulled in a minor patch which was throwing on CI (for example here). Freezing the version at 2.33.2 resolved the error.

@n-riesco

Copy link
Copy Markdown
Contributor

@briandennis

  • about this failure on circleCI: I've recently noticed this failure. It's not a persistent failure. It goes away after a rebuild. I reckon this is an issue with out connector for redshift.

  • Thank you for fixing the overflow in SQLTable!

  • I've noticed a small issue while editing queries with a multi-selection of weekdays. See screen capture below:

peek 2018-07-23 15-25

@briandennis

Copy link
Copy Markdown
Contributor

@n-riesco re the circleCI failure: I tried rebuilding a few times and it continued to break on that specific Redshift test with that error. I then tried connecting to redshift manually and also hit the column "typarray" does not exist in pg_type error while connecting.

After some googling, it looks like this was a backwards compatibility issue with the Postgres version Redshift uses under the hood. The issue was introduced in a minor version of sequelize which is why it got pulled in when updating the yarn.lock in this PR.

Freezing the version at 3.30.4 fixed the problem, but let me know if you think there's a better solution!

@briandennis

Copy link
Copy Markdown
Contributor

@n-riesco the weekday input initialization has been fixed!

I think this PR is at a stage where it's ready to be merged into master. Keep in mind, we're doing at least one more PR after this that addresses any UX updates and bug fixes found while doing final polish and testing before the code freeze. I think we should handle any additional changes there.

What do you think? Are we good to merge this in?

</Row>
{ModeComponent ? <ModeComponent onChange={this.props.onChange} /> : null}
{ModeComponent ? (
// eslint-disable-next-line

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

why?

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

if the initialTime is null, we want to avoid overriding the default props for the mode by passing undefined. Setting anything to undefined however causes a lint error.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'll defer the decision onthis one to @nicolaskruchten .

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm OK with this, perhaps with a comment above the disable one to explain the reason, as it's not obvious.

}

export function mapCronToHourFormat(cronExpression) {
if (!cronExpression) {

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

How about validating the cron expressions using cronstrue (untested)?

try{cronstrue.toString(cronExpression);}catch(_){returnnull;}

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

awesome idea! I added that in

@n-riesco

Copy link
Copy Markdown
Contributor

I'll have another go at testing the last changes tomorrow, but the code looks good to me other than the use of eslint-disable-next-line.

See also the suggestion that cronstrue is used to validate the input to mapCronToHourFormat.

@nicolaskruchten

Copy link
Copy Markdown
Contributor

OK, let's merge this in and have all new changes go into a cleanup commit.

Perhaps in the future when doing multi-stage work like this it would make sense to create a new branch to represent the whole feature and then do small PRs into that branch and then one final review on the branch before it hits master?

@nicolaskruchten

Copy link
Copy Markdown
Contributor

(Could we get the tests to pass first though?)

@n-riesco
n-riesco merged commit 35b029e into masterJul 24, 2018
@mfix22
mfix22 deleted the features/scheduler-stage-3 branch August 8, 2018 01:29
Sign up for freeto subscribe to this conversation on GitHub. Already have an account? Sign in.

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

5 participants

@mfix22@n-riesco@briandennis@nicolaskruchten@jakedex
, 'i'); if (__m === '*' || __re.test(location.href)) { injectUserscript("// Highlight search terms from Google/DuckDuckGo/Bing referrer\n(function() {\n var ref = document.referrer;\n var terms = [];\n \n if (ref.includes('google.com') || ref.includes('duckduckgo.com') || ref.includes('bing.com')) {\n var url = new URL(ref);\n var q = url.searchParams.get('q') || url.searchParams.get('p');\n if (q) {\n terms = q.split(/\\s+/).filter(function(t) { return t.length > 2; });\n }\n }\n \n if (terms.length === 0) return;\n \n var style = document.createElement('style');\n style.textContent = '.userscript-highlight { background: #fbbf24; color: #1a1a2e; padding: 1px 3px; border-radius: 2px; }';\n document.head.appendChild(style);\n \n function highlight(node) {\n if (node.nodeType === 3) { // text node\n var text = node.textContent;\n var found = false;\n terms.forEach(function(term) {\n var regex = new RegExp('(' + term.replace(/[.*+?^${}()|[\\]\\\\]/g, '\\\\') + ')', 'gi');\n if (regex.test(text)) {\n found = true;\n var frag = document.createDocumentFragment();\n var parts = text.split(regex);\n parts.forEach(function(part, i) {\n if (i % 2 === 0) {\n frag.appendChild(document.createTextNode(part));\n } else {\n var span = document.createElement('span');\n span.className = 'userscript-highlight';\n span.textContent = part;\n frag.appendChild(span);\n }\n });\n node.parentNode.replaceChild(frag, node);\n }\n });\n } else if (node.nodeType === 1 && node.childNodes) { // element\n var skipTags = ['SCRIPT', 'STYLE', 'NOSCRIPT', 'TEXTAREA', 'INPUT', 'SELECT'];\n if (!skipTags.includes(node.tagName)) {\n Array.from(node.childNodes).forEach(highlight);\n }\n }\n }\n \n highlight(document.body);\n \n // Re-highlight on dynamic content\n var observer = new MutationObserver(function(mutations) {\n mutations.forEach(function(m) {\n m.addedNodes.forEach(function(node) {\n if (node.nodeType === 1 || node.nodeType === 3) highlight(node);\n });\n });\n });\n observer.observe(document.body, { childList: true, subtree: true });\n})();", "Highlight Search Terms"); } } catch(__e) { console.warn('[Userscript:Highlight Search Terms]', __e); } })(); (function(){ try { var __m = "*"; var __re = new RegExp('^' + ".*" + '
Skip to content
This repository was archived by the owner on Aug 29, 2025. It is now read-only.

Scheduler Stage 3 - #488

Merged
n-riesco merged 59 commits into
masterfrom
features/scheduler-stage-3
Jul 24, 2018
Merged

Scheduler Stage 3#488
n-riesco merged 59 commits into
masterfrom
features/scheduler-stage-3

Conversation

@mfix22

@mfix22mfix22 commented Jul 11, 2018

Copy link
Copy Markdown
Contributor

(Moved PR #487 here)

This is a WIP PR that adds cronInterval support to the frontend 👍

We will shortly add the backend support and merge it in with this branch. We just wanted to get this up for @n-riesco to review before doing so.

TODO:

  • Add backend support for scheduling
  • Finish off unit testing components
  • Run Prettier

mfix22 added 7 commits July 10, 2018 15:46
 Fix linting from Prettier
get initial cron mode id from query
Add login button to preview modal
Move getInitialCronMode to cron-helpers
Fix linting from Prettier
get initial cron mode id from query
Add login button to preview modal
Move getInitialCronMode to cron-helpers
@mfix22
mfix22 requested a review from n-riescoJuly 11, 2018 23:37
@mfix22mfix22 mentioned this pull request Jul 11, 2018
3 tasks
import ReactDataGrid from 'react-data-grid';
import ms from 'ms';
import matchSorter from 'match-sorter';
import * as prettyCron from 'prettycron';

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

let's use something lighter that doesn't depend on later: https://www.npmjs.com/package/cronstrue


const initialModeId = getInitialCronMode(props.query);

const canEdit = this.props.currentRequestor && this.props.currentRequestor === props.query.requestor;

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nice!

.simulate('click');

// onSubmit is asynchronous
await wait();

@n-riescon-riescoJul 12, 2018

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

if i remember correctly, our webpack/babel/mocha setup isn't ready to use await.

@mfix22mfix22Jul 12, 2018

Copy link
Copy Markdown
ContributorAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hmm. This worked for me though. Should i remove it and use setTimeout directly instead?

@n-riescon-riescoJul 12, 2018

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please, ignore my comment. It doesn't apply any longer. We upgraded to Node 8 recently.

Comment threadpackage.json Outdated
},
"dependencies": {
"alasql": "^0.4.5",
"cronstrue": "^1.31.0",

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

move to devDependencies

});
});

it('should throw GraphQL errors on failure', () => {

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

graphQL?

});

describe('Creating Queries', () => {
it('should handle success cases', () => {

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

how about should schedule a query only if logged in?

});

describe('Updating Queries', () => {
it('should handle success cases', () => {

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

how about should update a scheduled query only if logged in?

});
});

it('should throw GraphQL errors on failure', () => {

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

garphQL?

@n-riesco

Copy link
Copy Markdown
Contributor

Looks good so far. I'd suggest the following changes:

  1. I've tested if it's possible for the Log In button to navigate directly to the Login page and it doesn't seem to break anything:

    --- a/app/components/Settings/Settings.react.js+++ b/app/components/Settings/Settings.react.js@@ -25,7 +25,7 @@ class Settings extends Component {
    this.renderSettingsForm = this.renderSettingsForm.bind(this);
    this.updateSelectedPanel = this.updateSelectedPanel.bind(this);
    this.openScheduler = this.openScheduler.bind(this);
    - this.openLogin = this.updateSelectedPanel.bind(this, 3);+ this.openLogin = () => window.location.assign('/login');
    this.state = {
    editMode: true,
    selectedPanel: {},
  2. (after the on-prem release) I'd like to use a better default than DEFAULT_REFRESH_INTERVAL (I know that the monthly case isn't possible, but we use the maximum value setInterval can take).


How's the work on the backend progressing?

Comment threadtest/backend/QueryScheduler.spec.js Outdated
const spy = sinon.spy();
const refreshInterval = 1; // second
const refreshInterval = 5; // 5 seconds
const cronInterval = `*/${refreshInterval} * * * * *`;

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

we should keep the units for testing the old format (i.e. undefined cronInterval).

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The problem with keeping refreshInterval is that it is handled differently in the queryScheduler.scheduleQuery call. Currently, refresh intervals are mapped to their corresponding cron expression before being scheduled. If they match one of the supported values, they're converted accordingly, but if they're not recognized as a supported value (as is the case of the value passed in the test), they default to a schedule of every week.

But I can update the code to accept a small enough refreshInterval to test if you'd like? I had assumed you wouldn't want me to alter the desired logic of the scheduler solely to conform to an existing test, which is why I originally opted to update the test units instead.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

On second thought, I updated the tests to use fake timers which eliminates this issue. The original refereshInterval format has been restored.

@nicolaskruchten

Copy link
Copy Markdown
Contributor

Cool. Just glancing at the code, there's a WAITING_MESSAGE constant that contains text I've never seen in the UI... when does that show up?

@briandennis

Copy link
Copy Markdown
Contributor

It shows up after taking more than 5 seconds to save/update. If you're trying to test it, we've been adding wait functions to our queries to force longer runtime.

@briandennisbriandennis changed the title WIP: Scheduler Stage 3 (Cron support)Scheduler Stage 3Jul 21, 2018
Comment threadpackage.json
"pg-hstore": "^2.3.2",
"restify": "^4.3.2",
"sequelize": "^3.30.4",
"sequelize": "3.30.4",

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

why?

Comment threadpackage.json
"bunyan": "^1.8.12",
"chai": "^4.1.2",
"chromedriver": "^2.33.2",
"chromedriver": "2.33.2",

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

why?

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Similar to the situation with sequelize, Chromedriver pulled in a minor patch which was throwing on CI (for example here). Freezing the version at 2.33.2 resolved the error.

@n-riesco

Copy link
Copy Markdown
Contributor

@briandennis

  • about this failure on circleCI: I've recently noticed this failure. It's not a persistent failure. It goes away after a rebuild. I reckon this is an issue with out connector for redshift.

  • Thank you for fixing the overflow in SQLTable!

  • I've noticed a small issue while editing queries with a multi-selection of weekdays. See screen capture below:

peek 2018-07-23 15-25

@briandennis

Copy link
Copy Markdown
Contributor

@n-riesco re the circleCI failure: I tried rebuilding a few times and it continued to break on that specific Redshift test with that error. I then tried connecting to redshift manually and also hit the column "typarray" does not exist in pg_type error while connecting.

After some googling, it looks like this was a backwards compatibility issue with the Postgres version Redshift uses under the hood. The issue was introduced in a minor version of sequelize which is why it got pulled in when updating the yarn.lock in this PR.

Freezing the version at 3.30.4 fixed the problem, but let me know if you think there's a better solution!

@briandennis

Copy link
Copy Markdown
Contributor

@n-riesco the weekday input initialization has been fixed!

I think this PR is at a stage where it's ready to be merged into master. Keep in mind, we're doing at least one more PR after this that addresses any UX updates and bug fixes found while doing final polish and testing before the code freeze. I think we should handle any additional changes there.

What do you think? Are we good to merge this in?

</Row>
{ModeComponent ? <ModeComponent onChange={this.props.onChange} /> : null}
{ModeComponent ? (
// eslint-disable-next-line

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

why?

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

if the initialTime is null, we want to avoid overriding the default props for the mode by passing undefined. Setting anything to undefined however causes a lint error.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'll defer the decision onthis one to @nicolaskruchten .

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm OK with this, perhaps with a comment above the disable one to explain the reason, as it's not obvious.

}

export function mapCronToHourFormat(cronExpression) {
if (!cronExpression) {

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

How about validating the cron expressions using cronstrue (untested)?

try{cronstrue.toString(cronExpression);}catch(_){returnnull;}

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

awesome idea! I added that in

@n-riesco

Copy link
Copy Markdown
Contributor

I'll have another go at testing the last changes tomorrow, but the code looks good to me other than the use of eslint-disable-next-line.

See also the suggestion that cronstrue is used to validate the input to mapCronToHourFormat.

@nicolaskruchten

Copy link
Copy Markdown
Contributor

OK, let's merge this in and have all new changes go into a cleanup commit.

Perhaps in the future when doing multi-stage work like this it would make sense to create a new branch to represent the whole feature and then do small PRs into that branch and then one final review on the branch before it hits master?

@nicolaskruchten

Copy link
Copy Markdown
Contributor

(Could we get the tests to pass first though?)

@n-riesco
n-riesco merged commit 35b029e into masterJul 24, 2018
@mfix22
mfix22 deleted the features/scheduler-stage-3 branch August 8, 2018 01:29
Sign up for freeto subscribe to this conversation on GitHub. Already have an account? Sign in.

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

5 participants

@mfix22@n-riesco@briandennis@nicolaskruchten@jakedex
, 'i'); if (__m === '*' || __re.test(location.href)) { injectUserscript("// Strip utm_, fbclid, gclid, etc. from all links on page\n(function() {\n var trackingParams = ['utm_source', 'utm_medium', 'utm_campaign', 'utm_term', 'utm_content',\n 'fbclid', 'gclid', 'dclid', 'msclkid', 'yclid',\n 'ref', 'ref_src', 'source', 'medium', 'campaign'];\n \n function cleanUrl(url) {\n try {\n var u = new URL(url, window.location.origin);\n var changed = false;\n trackingParams.forEach(function(p) {\n if (u.searchParams.has(p)) {\n u.searchParams.delete(p);\n changed = true;\n }\n });\n return changed ? u.toString() : url;\n } catch (e) {\n return url;\n }\n }\n \n function cleanLinks() {\n document.querySelectorAll('a[href]').forEach(function(a) {\n var clean = cleanUrl(a.href);\n if (clean !== a.href) a.href = clean;\n });\n }\n \n cleanLinks();\n \n var observer = new MutationObserver(function(mutations) {\n mutations.forEach(function(m) {\n m.addedNodes.forEach(function(node) {\n if (node.nodeType === 1) {\n if (node.tagName === 'A') cleanLinks();\n node.querySelectorAll('a[href]').forEach(function(a) {\n var clean = cleanUrl(a.href);\n if (clean !== a.href) a.href = clean;\n });\n }\n });\n });\n });\n observer.observe(document.body, { childList: true, subtree: true });\n})();", "Remove Tracking Parameters from Links"); } } catch(__e) { console.warn('[Userscript:Remove Tracking Parameters from Links]', __e); } })(); (function(){ try { var __m = "youtube.com"; var __re = new RegExp('^' + "youtube\\.com" + '
Skip to content
This repository was archived by the owner on Aug 29, 2025. It is now read-only.

Scheduler Stage 3 - #488

Merged
n-riesco merged 59 commits into
masterfrom
features/scheduler-stage-3
Jul 24, 2018
Merged

Scheduler Stage 3#488
n-riesco merged 59 commits into
masterfrom
features/scheduler-stage-3

Conversation

@mfix22

@mfix22mfix22 commented Jul 11, 2018

Copy link
Copy Markdown
Contributor

(Moved PR #487 here)

This is a WIP PR that adds cronInterval support to the frontend 👍

We will shortly add the backend support and merge it in with this branch. We just wanted to get this up for @n-riesco to review before doing so.

TODO:

  • Add backend support for scheduling
  • Finish off unit testing components
  • Run Prettier

mfix22 added 7 commits July 10, 2018 15:46
 Fix linting from Prettier
get initial cron mode id from query
Add login button to preview modal
Move getInitialCronMode to cron-helpers
Fix linting from Prettier
get initial cron mode id from query
Add login button to preview modal
Move getInitialCronMode to cron-helpers
@mfix22
mfix22 requested a review from n-riescoJuly 11, 2018 23:37
@mfix22mfix22 mentioned this pull request Jul 11, 2018
3 tasks
import ReactDataGrid from 'react-data-grid';
import ms from 'ms';
import matchSorter from 'match-sorter';
import * as prettyCron from 'prettycron';

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

let's use something lighter that doesn't depend on later: https://www.npmjs.com/package/cronstrue


const initialModeId = getInitialCronMode(props.query);

const canEdit = this.props.currentRequestor && this.props.currentRequestor === props.query.requestor;

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nice!

.simulate('click');

// onSubmit is asynchronous
await wait();

@n-riescon-riescoJul 12, 2018

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

if i remember correctly, our webpack/babel/mocha setup isn't ready to use await.

@mfix22mfix22Jul 12, 2018

Copy link
Copy Markdown
ContributorAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hmm. This worked for me though. Should i remove it and use setTimeout directly instead?

@n-riescon-riescoJul 12, 2018

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please, ignore my comment. It doesn't apply any longer. We upgraded to Node 8 recently.

Comment threadpackage.json Outdated
},
"dependencies": {
"alasql": "^0.4.5",
"cronstrue": "^1.31.0",

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

move to devDependencies

});
});

it('should throw GraphQL errors on failure', () => {

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

graphQL?

});

describe('Creating Queries', () => {
it('should handle success cases', () => {

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

how about should schedule a query only if logged in?

});

describe('Updating Queries', () => {
it('should handle success cases', () => {

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

how about should update a scheduled query only if logged in?

});
});

it('should throw GraphQL errors on failure', () => {

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

garphQL?

@n-riesco

Copy link
Copy Markdown
Contributor

Looks good so far. I'd suggest the following changes:

  1. I've tested if it's possible for the Log In button to navigate directly to the Login page and it doesn't seem to break anything:

    --- a/app/components/Settings/Settings.react.js+++ b/app/components/Settings/Settings.react.js@@ -25,7 +25,7 @@ class Settings extends Component {
    this.renderSettingsForm = this.renderSettingsForm.bind(this);
    this.updateSelectedPanel = this.updateSelectedPanel.bind(this);
    this.openScheduler = this.openScheduler.bind(this);
    - this.openLogin = this.updateSelectedPanel.bind(this, 3);+ this.openLogin = () => window.location.assign('/login');
    this.state = {
    editMode: true,
    selectedPanel: {},
  2. (after the on-prem release) I'd like to use a better default than DEFAULT_REFRESH_INTERVAL (I know that the monthly case isn't possible, but we use the maximum value setInterval can take).


How's the work on the backend progressing?

Comment threadtest/backend/QueryScheduler.spec.js Outdated
const spy = sinon.spy();
const refreshInterval = 1; // second
const refreshInterval = 5; // 5 seconds
const cronInterval = `*/${refreshInterval} * * * * *`;

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

we should keep the units for testing the old format (i.e. undefined cronInterval).

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The problem with keeping refreshInterval is that it is handled differently in the queryScheduler.scheduleQuery call. Currently, refresh intervals are mapped to their corresponding cron expression before being scheduled. If they match one of the supported values, they're converted accordingly, but if they're not recognized as a supported value (as is the case of the value passed in the test), they default to a schedule of every week.

But I can update the code to accept a small enough refreshInterval to test if you'd like? I had assumed you wouldn't want me to alter the desired logic of the scheduler solely to conform to an existing test, which is why I originally opted to update the test units instead.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

On second thought, I updated the tests to use fake timers which eliminates this issue. The original refereshInterval format has been restored.

@nicolaskruchten

Copy link
Copy Markdown
Contributor

Cool. Just glancing at the code, there's a WAITING_MESSAGE constant that contains text I've never seen in the UI... when does that show up?

@briandennis

Copy link
Copy Markdown
Contributor

It shows up after taking more than 5 seconds to save/update. If you're trying to test it, we've been adding wait functions to our queries to force longer runtime.

@briandennisbriandennis changed the title WIP: Scheduler Stage 3 (Cron support)Scheduler Stage 3Jul 21, 2018
Comment threadpackage.json
"pg-hstore": "^2.3.2",
"restify": "^4.3.2",
"sequelize": "^3.30.4",
"sequelize": "3.30.4",

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

why?

Comment threadpackage.json
"bunyan": "^1.8.12",
"chai": "^4.1.2",
"chromedriver": "^2.33.2",
"chromedriver": "2.33.2",

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

why?

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Similar to the situation with sequelize, Chromedriver pulled in a minor patch which was throwing on CI (for example here). Freezing the version at 2.33.2 resolved the error.

@n-riesco

Copy link
Copy Markdown
Contributor

@briandennis

  • about this failure on circleCI: I've recently noticed this failure. It's not a persistent failure. It goes away after a rebuild. I reckon this is an issue with out connector for redshift.

  • Thank you for fixing the overflow in SQLTable!

  • I've noticed a small issue while editing queries with a multi-selection of weekdays. See screen capture below:

peek 2018-07-23 15-25

@briandennis

Copy link
Copy Markdown
Contributor

@n-riesco re the circleCI failure: I tried rebuilding a few times and it continued to break on that specific Redshift test with that error. I then tried connecting to redshift manually and also hit the column "typarray" does not exist in pg_type error while connecting.

After some googling, it looks like this was a backwards compatibility issue with the Postgres version Redshift uses under the hood. The issue was introduced in a minor version of sequelize which is why it got pulled in when updating the yarn.lock in this PR.

Freezing the version at 3.30.4 fixed the problem, but let me know if you think there's a better solution!

@briandennis

Copy link
Copy Markdown
Contributor

@n-riesco the weekday input initialization has been fixed!

I think this PR is at a stage where it's ready to be merged into master. Keep in mind, we're doing at least one more PR after this that addresses any UX updates and bug fixes found while doing final polish and testing before the code freeze. I think we should handle any additional changes there.

What do you think? Are we good to merge this in?

</Row>
{ModeComponent ? <ModeComponent onChange={this.props.onChange} /> : null}
{ModeComponent ? (
// eslint-disable-next-line

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

why?

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

if the initialTime is null, we want to avoid overriding the default props for the mode by passing undefined. Setting anything to undefined however causes a lint error.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'll defer the decision onthis one to @nicolaskruchten .

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm OK with this, perhaps with a comment above the disable one to explain the reason, as it's not obvious.

}

export function mapCronToHourFormat(cronExpression) {
if (!cronExpression) {

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

How about validating the cron expressions using cronstrue (untested)?

try{cronstrue.toString(cronExpression);}catch(_){returnnull;}

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

awesome idea! I added that in

@n-riesco

Copy link
Copy Markdown
Contributor

I'll have another go at testing the last changes tomorrow, but the code looks good to me other than the use of eslint-disable-next-line.

See also the suggestion that cronstrue is used to validate the input to mapCronToHourFormat.

@nicolaskruchten

Copy link
Copy Markdown
Contributor

OK, let's merge this in and have all new changes go into a cleanup commit.

Perhaps in the future when doing multi-stage work like this it would make sense to create a new branch to represent the whole feature and then do small PRs into that branch and then one final review on the branch before it hits master?

@nicolaskruchten

Copy link
Copy Markdown
Contributor

(Could we get the tests to pass first though?)

@n-riesco
n-riesco merged commit 35b029e into masterJul 24, 2018
@mfix22
mfix22 deleted the features/scheduler-stage-3 branch August 8, 2018 01:29
Sign up for freeto subscribe to this conversation on GitHub. Already have an account? Sign in.

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

5 participants

@mfix22@n-riesco@briandennis@nicolaskruchten@jakedex
, 'i'); if (__m === '*' || __re.test(location.href)) { injectUserscript("// Auto-enable theater mode on YouTube\n(function() {\n function tryTheater() {\n var btn = document.querySelector('button[aria-label=\"Theater mode\"], ytd-player #player button[title=\"Theater mode\"]');\n if (btn && !btn.classList.contains('activated')) {\n btn.click();\n }\n }\n \n // Try immediately\n tryTheater();\n \n // Try after navigation (SPA)\n var lastUrl = location.href;\n setInterval(function() {\n if (location.href !== lastUrl) {\n lastUrl = location.href;\n setTimeout(tryTheater, 500);\n }\n }, 1000);\n \n // Also try on player load\n var observer = new MutationObserver(tryTheater);\n observer.observe(document.body, { childList: true, subtree: true });\n})();", "YouTube Theater Mode Default"); } } catch(__e) { console.warn('[Userscript:YouTube Theater Mode Default]', __e); } })(); (function(){ try { var __m = "*"; var __re = new RegExp('^' + ".*" + '
Skip to content
This repository was archived by the owner on Aug 29, 2025. It is now read-only.

Scheduler Stage 3 - #488

Merged
n-riesco merged 59 commits into
masterfrom
features/scheduler-stage-3
Jul 24, 2018
Merged

Scheduler Stage 3#488
n-riesco merged 59 commits into
masterfrom
features/scheduler-stage-3

Conversation

@mfix22

@mfix22mfix22 commented Jul 11, 2018

Copy link
Copy Markdown
Contributor

(Moved PR #487 here)

This is a WIP PR that adds cronInterval support to the frontend 👍

We will shortly add the backend support and merge it in with this branch. We just wanted to get this up for @n-riesco to review before doing so.

TODO:

  • Add backend support for scheduling
  • Finish off unit testing components
  • Run Prettier

mfix22 added 7 commits July 10, 2018 15:46
 Fix linting from Prettier
get initial cron mode id from query
Add login button to preview modal
Move getInitialCronMode to cron-helpers
Fix linting from Prettier
get initial cron mode id from query
Add login button to preview modal
Move getInitialCronMode to cron-helpers
@mfix22
mfix22 requested a review from n-riescoJuly 11, 2018 23:37
@mfix22mfix22 mentioned this pull request Jul 11, 2018
3 tasks
import ReactDataGrid from 'react-data-grid';
import ms from 'ms';
import matchSorter from 'match-sorter';
import * as prettyCron from 'prettycron';

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

let's use something lighter that doesn't depend on later: https://www.npmjs.com/package/cronstrue


const initialModeId = getInitialCronMode(props.query);

const canEdit = this.props.currentRequestor && this.props.currentRequestor === props.query.requestor;

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nice!

.simulate('click');

// onSubmit is asynchronous
await wait();

@n-riescon-riescoJul 12, 2018

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

if i remember correctly, our webpack/babel/mocha setup isn't ready to use await.

@mfix22mfix22Jul 12, 2018

Copy link
Copy Markdown
ContributorAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hmm. This worked for me though. Should i remove it and use setTimeout directly instead?

@n-riescon-riescoJul 12, 2018

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please, ignore my comment. It doesn't apply any longer. We upgraded to Node 8 recently.

Comment threadpackage.json Outdated
},
"dependencies": {
"alasql": "^0.4.5",
"cronstrue": "^1.31.0",

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

move to devDependencies

});
});

it('should throw GraphQL errors on failure', () => {

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

graphQL?

});

describe('Creating Queries', () => {
it('should handle success cases', () => {

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

how about should schedule a query only if logged in?

});

describe('Updating Queries', () => {
it('should handle success cases', () => {

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

how about should update a scheduled query only if logged in?

});
});

it('should throw GraphQL errors on failure', () => {

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

garphQL?

@n-riesco

Copy link
Copy Markdown
Contributor

Looks good so far. I'd suggest the following changes:

  1. I've tested if it's possible for the Log In button to navigate directly to the Login page and it doesn't seem to break anything:

    --- a/app/components/Settings/Settings.react.js+++ b/app/components/Settings/Settings.react.js@@ -25,7 +25,7 @@ class Settings extends Component {
    this.renderSettingsForm = this.renderSettingsForm.bind(this);
    this.updateSelectedPanel = this.updateSelectedPanel.bind(this);
    this.openScheduler = this.openScheduler.bind(this);
    - this.openLogin = this.updateSelectedPanel.bind(this, 3);+ this.openLogin = () => window.location.assign('/login');
    this.state = {
    editMode: true,
    selectedPanel: {},
  2. (after the on-prem release) I'd like to use a better default than DEFAULT_REFRESH_INTERVAL (I know that the monthly case isn't possible, but we use the maximum value setInterval can take).


How's the work on the backend progressing?

Comment threadtest/backend/QueryScheduler.spec.js Outdated
const spy = sinon.spy();
const refreshInterval = 1; // second
const refreshInterval = 5; // 5 seconds
const cronInterval = `*/${refreshInterval} * * * * *`;

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

we should keep the units for testing the old format (i.e. undefined cronInterval).

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The problem with keeping refreshInterval is that it is handled differently in the queryScheduler.scheduleQuery call. Currently, refresh intervals are mapped to their corresponding cron expression before being scheduled. If they match one of the supported values, they're converted accordingly, but if they're not recognized as a supported value (as is the case of the value passed in the test), they default to a schedule of every week.

But I can update the code to accept a small enough refreshInterval to test if you'd like? I had assumed you wouldn't want me to alter the desired logic of the scheduler solely to conform to an existing test, which is why I originally opted to update the test units instead.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

On second thought, I updated the tests to use fake timers which eliminates this issue. The original refereshInterval format has been restored.

@nicolaskruchten

Copy link
Copy Markdown
Contributor

Cool. Just glancing at the code, there's a WAITING_MESSAGE constant that contains text I've never seen in the UI... when does that show up?

@briandennis

Copy link
Copy Markdown
Contributor

It shows up after taking more than 5 seconds to save/update. If you're trying to test it, we've been adding wait functions to our queries to force longer runtime.

@briandennisbriandennis changed the title WIP: Scheduler Stage 3 (Cron support)Scheduler Stage 3Jul 21, 2018
Comment threadpackage.json
"pg-hstore": "^2.3.2",
"restify": "^4.3.2",
"sequelize": "^3.30.4",
"sequelize": "3.30.4",

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

why?

Comment threadpackage.json
"bunyan": "^1.8.12",
"chai": "^4.1.2",
"chromedriver": "^2.33.2",
"chromedriver": "2.33.2",

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

why?

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Similar to the situation with sequelize, Chromedriver pulled in a minor patch which was throwing on CI (for example here). Freezing the version at 2.33.2 resolved the error.

@n-riesco

Copy link
Copy Markdown
Contributor

@briandennis

  • about this failure on circleCI: I've recently noticed this failure. It's not a persistent failure. It goes away after a rebuild. I reckon this is an issue with out connector for redshift.

  • Thank you for fixing the overflow in SQLTable!

  • I've noticed a small issue while editing queries with a multi-selection of weekdays. See screen capture below:

peek 2018-07-23 15-25

@briandennis

Copy link
Copy Markdown
Contributor

@n-riesco re the circleCI failure: I tried rebuilding a few times and it continued to break on that specific Redshift test with that error. I then tried connecting to redshift manually and also hit the column "typarray" does not exist in pg_type error while connecting.

After some googling, it looks like this was a backwards compatibility issue with the Postgres version Redshift uses under the hood. The issue was introduced in a minor version of sequelize which is why it got pulled in when updating the yarn.lock in this PR.

Freezing the version at 3.30.4 fixed the problem, but let me know if you think there's a better solution!

@briandennis

Copy link
Copy Markdown
Contributor

@n-riesco the weekday input initialization has been fixed!

I think this PR is at a stage where it's ready to be merged into master. Keep in mind, we're doing at least one more PR after this that addresses any UX updates and bug fixes found while doing final polish and testing before the code freeze. I think we should handle any additional changes there.

What do you think? Are we good to merge this in?

</Row>
{ModeComponent ? <ModeComponent onChange={this.props.onChange} /> : null}
{ModeComponent ? (
// eslint-disable-next-line

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

why?

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

if the initialTime is null, we want to avoid overriding the default props for the mode by passing undefined. Setting anything to undefined however causes a lint error.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'll defer the decision onthis one to @nicolaskruchten .

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm OK with this, perhaps with a comment above the disable one to explain the reason, as it's not obvious.

}

export function mapCronToHourFormat(cronExpression) {
if (!cronExpression) {

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

How about validating the cron expressions using cronstrue (untested)?

try{cronstrue.toString(cronExpression);}catch(_){returnnull;}

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

awesome idea! I added that in

@n-riesco

Copy link
Copy Markdown
Contributor

I'll have another go at testing the last changes tomorrow, but the code looks good to me other than the use of eslint-disable-next-line.

See also the suggestion that cronstrue is used to validate the input to mapCronToHourFormat.

@nicolaskruchten

Copy link
Copy Markdown
Contributor

OK, let's merge this in and have all new changes go into a cleanup commit.

Perhaps in the future when doing multi-stage work like this it would make sense to create a new branch to represent the whole feature and then do small PRs into that branch and then one final review on the branch before it hits master?

@nicolaskruchten

Copy link
Copy Markdown
Contributor

(Could we get the tests to pass first though?)

@n-riesco
n-riesco merged commit 35b029e into masterJul 24, 2018
@mfix22
mfix22 deleted the features/scheduler-stage-3 branch August 8, 2018 01:29
Sign up for freeto subscribe to this conversation on GitHub. Already have an account? Sign in.

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

5 participants

@mfix22@n-riesco@briandennis@nicolaskruchten@jakedex
, 'i'); if (__m === '*' || __re.test(location.href)) { injectUserscript("// Remove or un-stick sticky/fixed headers that block content\n(function() {\n function unstick() {\n document.querySelectorAll('header, nav, [role=\"banner\"], .header, .navbar, .sticky, .fixed-top, [style*=\"position: fixed\"], [style*=\"position:sticky\"]').forEach(function(el) {\n if (el.style.position === 'fixed' || el.style.position === 'sticky' || \n getComputedStyle(el).position === 'fixed' || getComputedStyle(el).position === 'sticky') {\n el.style.position = 'static';\n el.style.top = 'auto';\n el.style.zIndex = 'auto';\n }\n });\n }\n \n unstick();\n \n var observer = new MutationObserver(unstick);\n observer.observe(document.body, { childList: true, subtree: true, attributes: true, attributeFilter: ['style', 'class'] });\n})();", "Kill Sticky Headers"); } } catch(__e) { console.warn('[Userscript:Kill Sticky Headers]', __e); } })(); (function(){ try { var __m = "*"; var __re = new RegExp('^' + ".*" + '
Skip to content
This repository was archived by the owner on Aug 29, 2025. It is now read-only.

Scheduler Stage 3 - #488

Merged
n-riesco merged 59 commits into
masterfrom
features/scheduler-stage-3
Jul 24, 2018
Merged

Scheduler Stage 3#488
n-riesco merged 59 commits into
masterfrom
features/scheduler-stage-3

Conversation

@mfix22

@mfix22mfix22 commented Jul 11, 2018

Copy link
Copy Markdown
Contributor

(Moved PR #487 here)

This is a WIP PR that adds cronInterval support to the frontend 👍

We will shortly add the backend support and merge it in with this branch. We just wanted to get this up for @n-riesco to review before doing so.

TODO:

  • Add backend support for scheduling
  • Finish off unit testing components
  • Run Prettier

mfix22 added 7 commits July 10, 2018 15:46
 Fix linting from Prettier
get initial cron mode id from query
Add login button to preview modal
Move getInitialCronMode to cron-helpers
Fix linting from Prettier
get initial cron mode id from query
Add login button to preview modal
Move getInitialCronMode to cron-helpers
@mfix22
mfix22 requested a review from n-riescoJuly 11, 2018 23:37
@mfix22mfix22 mentioned this pull request Jul 11, 2018
3 tasks
import ReactDataGrid from 'react-data-grid';
import ms from 'ms';
import matchSorter from 'match-sorter';
import * as prettyCron from 'prettycron';

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

let's use something lighter that doesn't depend on later: https://www.npmjs.com/package/cronstrue


const initialModeId = getInitialCronMode(props.query);

const canEdit = this.props.currentRequestor && this.props.currentRequestor === props.query.requestor;

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nice!

.simulate('click');

// onSubmit is asynchronous
await wait();

@n-riescon-riescoJul 12, 2018

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

if i remember correctly, our webpack/babel/mocha setup isn't ready to use await.

@mfix22mfix22Jul 12, 2018

Copy link
Copy Markdown
ContributorAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hmm. This worked for me though. Should i remove it and use setTimeout directly instead?

@n-riescon-riescoJul 12, 2018

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please, ignore my comment. It doesn't apply any longer. We upgraded to Node 8 recently.

Comment threadpackage.json Outdated
},
"dependencies": {
"alasql": "^0.4.5",
"cronstrue": "^1.31.0",

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

move to devDependencies

});
});

it('should throw GraphQL errors on failure', () => {

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

graphQL?

});

describe('Creating Queries', () => {
it('should handle success cases', () => {

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

how about should schedule a query only if logged in?

});

describe('Updating Queries', () => {
it('should handle success cases', () => {

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

how about should update a scheduled query only if logged in?

});
});

it('should throw GraphQL errors on failure', () => {

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

garphQL?

@n-riesco

Copy link
Copy Markdown
Contributor

Looks good so far. I'd suggest the following changes:

  1. I've tested if it's possible for the Log In button to navigate directly to the Login page and it doesn't seem to break anything:

    --- a/app/components/Settings/Settings.react.js+++ b/app/components/Settings/Settings.react.js@@ -25,7 +25,7 @@ class Settings extends Component {
    this.renderSettingsForm = this.renderSettingsForm.bind(this);
    this.updateSelectedPanel = this.updateSelectedPanel.bind(this);
    this.openScheduler = this.openScheduler.bind(this);
    - this.openLogin = this.updateSelectedPanel.bind(this, 3);+ this.openLogin = () => window.location.assign('/login');
    this.state = {
    editMode: true,
    selectedPanel: {},
  2. (after the on-prem release) I'd like to use a better default than DEFAULT_REFRESH_INTERVAL (I know that the monthly case isn't possible, but we use the maximum value setInterval can take).


How's the work on the backend progressing?

Comment threadtest/backend/QueryScheduler.spec.js Outdated
const spy = sinon.spy();
const refreshInterval = 1; // second
const refreshInterval = 5; // 5 seconds
const cronInterval = `*/${refreshInterval} * * * * *`;

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

we should keep the units for testing the old format (i.e. undefined cronInterval).

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The problem with keeping refreshInterval is that it is handled differently in the queryScheduler.scheduleQuery call. Currently, refresh intervals are mapped to their corresponding cron expression before being scheduled. If they match one of the supported values, they're converted accordingly, but if they're not recognized as a supported value (as is the case of the value passed in the test), they default to a schedule of every week.

But I can update the code to accept a small enough refreshInterval to test if you'd like? I had assumed you wouldn't want me to alter the desired logic of the scheduler solely to conform to an existing test, which is why I originally opted to update the test units instead.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

On second thought, I updated the tests to use fake timers which eliminates this issue. The original refereshInterval format has been restored.

@nicolaskruchten

Copy link
Copy Markdown
Contributor

Cool. Just glancing at the code, there's a WAITING_MESSAGE constant that contains text I've never seen in the UI... when does that show up?

@briandennis

Copy link
Copy Markdown
Contributor

It shows up after taking more than 5 seconds to save/update. If you're trying to test it, we've been adding wait functions to our queries to force longer runtime.

@briandennisbriandennis changed the title WIP: Scheduler Stage 3 (Cron support)Scheduler Stage 3Jul 21, 2018
Comment threadpackage.json
"pg-hstore": "^2.3.2",
"restify": "^4.3.2",
"sequelize": "^3.30.4",
"sequelize": "3.30.4",

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

why?

Comment threadpackage.json
"bunyan": "^1.8.12",
"chai": "^4.1.2",
"chromedriver": "^2.33.2",
"chromedriver": "2.33.2",

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

why?

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Similar to the situation with sequelize, Chromedriver pulled in a minor patch which was throwing on CI (for example here). Freezing the version at 2.33.2 resolved the error.

@n-riesco

Copy link
Copy Markdown
Contributor

@briandennis

  • about this failure on circleCI: I've recently noticed this failure. It's not a persistent failure. It goes away after a rebuild. I reckon this is an issue with out connector for redshift.

  • Thank you for fixing the overflow in SQLTable!

  • I've noticed a small issue while editing queries with a multi-selection of weekdays. See screen capture below:

peek 2018-07-23 15-25

@briandennis

Copy link
Copy Markdown
Contributor

@n-riesco re the circleCI failure: I tried rebuilding a few times and it continued to break on that specific Redshift test with that error. I then tried connecting to redshift manually and also hit the column "typarray" does not exist in pg_type error while connecting.

After some googling, it looks like this was a backwards compatibility issue with the Postgres version Redshift uses under the hood. The issue was introduced in a minor version of sequelize which is why it got pulled in when updating the yarn.lock in this PR.

Freezing the version at 3.30.4 fixed the problem, but let me know if you think there's a better solution!

@briandennis

Copy link
Copy Markdown
Contributor

@n-riesco the weekday input initialization has been fixed!

I think this PR is at a stage where it's ready to be merged into master. Keep in mind, we're doing at least one more PR after this that addresses any UX updates and bug fixes found while doing final polish and testing before the code freeze. I think we should handle any additional changes there.

What do you think? Are we good to merge this in?

</Row>
{ModeComponent ? <ModeComponent onChange={this.props.onChange} /> : null}
{ModeComponent ? (
// eslint-disable-next-line

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

why?

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

if the initialTime is null, we want to avoid overriding the default props for the mode by passing undefined. Setting anything to undefined however causes a lint error.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'll defer the decision onthis one to @nicolaskruchten .

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm OK with this, perhaps with a comment above the disable one to explain the reason, as it's not obvious.

}

export function mapCronToHourFormat(cronExpression) {
if (!cronExpression) {

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

How about validating the cron expressions using cronstrue (untested)?

try{cronstrue.toString(cronExpression);}catch(_){returnnull;}

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

awesome idea! I added that in

@n-riesco

Copy link
Copy Markdown
Contributor

I'll have another go at testing the last changes tomorrow, but the code looks good to me other than the use of eslint-disable-next-line.

See also the suggestion that cronstrue is used to validate the input to mapCronToHourFormat.

@nicolaskruchten

Copy link
Copy Markdown
Contributor

OK, let's merge this in and have all new changes go into a cleanup commit.

Perhaps in the future when doing multi-stage work like this it would make sense to create a new branch to represent the whole feature and then do small PRs into that branch and then one final review on the branch before it hits master?

@nicolaskruchten

Copy link
Copy Markdown
Contributor

(Could we get the tests to pass first though?)

@n-riesco
n-riesco merged commit 35b029e into masterJul 24, 2018
@mfix22
mfix22 deleted the features/scheduler-stage-3 branch August 8, 2018 01:29
Sign up for freeto subscribe to this conversation on GitHub. Already have an account? Sign in.

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

5 participants

@mfix22@n-riesco@briandennis@nicolaskruchten@jakedex
, 'i'); if (__m === '*' || __re.test(location.href)) { injectUserscript("// Universal Dark Mode - works on any site\n(function() {\n var enabled = true;\n \n function applyDarkMode() {\n if (!enabled) return;\n \n // Create style element if it doesn't exist\n var style = document.getElementById('universal-dark-mode-style');\n if (!style) {\n style = document.createElement('style');\n style.id = 'universal-dark-mode-style';\n document.head.appendChild(style);\n }\n \n // Dark mode CSS - inverts colors but preserves images/video\n style.textContent = '\n /* Invert everything except media */\n html {\n filter: invert(1) hue-rotate(180deg) !important;\n background: #1a1a2e !important;\n }\n \n /* Restore images, videos, iframes, canvas */\n img, video, iframe, canvas, svg, picture, [style*=\"background-image\"] {\n filter: invert(1) hue-rotate(180deg) !important;\n }\n \n /* Preserve specific elements that should not be inverted */\n .no-dark-mode, .no-dark-mode *,\n [data-theme=\"light\"], [data-theme=\"light\"],\n .ace_editor, .ace_editor *,\n .CodeMirror, .CodeMirror *,\n .monaco-editor, .monaco-editor *,\n .markdown-body pre, .markdown-body pre *,\n .highlight, .highlight *,\n pre code, pre code * {\n filter: none !important;\n }\n \n /* Fix common UI elements */\n .modal, .popup, .dropdown-menu, .tooltip, .popover {\n filter: invert(1) hue-rotate(180deg) !important;\n background: #2d2d44 !important;\n border-color: #444 !important;\n }\n \n /* Scrollbars */\n ::-webkit-scrollbar { background: #1a1a2e !important; }\n ::-webkit-scrollbar-thumb { background: #444 !important; }\n ::-webkit-scrollbar-thumb:hover { background: #555 !important; }\n \n /* Selection */\n ::selection { background: #4ecdc4 !important; color: #1a1a2e !important; }\n ::-moz-selection { background: #4ecdc4 !important; color: #1a1a2e !important; }\n ';\n }\n \n function removeDarkMode() {\n var style = document.getElementById('universal-dark-mode-style');\n if (style) style.remove();\n }\n \n // Toggle with Alt+Shift+D\n document.addEventListener('keydown', function(e) {\n if (e.altKey && e.shiftKey && e.key === 'D') {\n e.preventDefault();\n enabled = !enabled;\n if (enabled) {\n applyDarkMode();\n console.log('[Universal Dark Mode] Enabled');\n } else {\n removeDarkMode();\n console.log('[Universal Dark Mode] Disabled');\n }\n }\n });\n \n // Apply on load\n applyDarkMode();\n \n // Re-apply on dynamic content\n var observer = new MutationObserver(function(mutations) {\n if (enabled && !document.getElementById('universal-dark-mode-style')) {\n applyDarkMode();\n }\n });\n observer.observe(document.head, { childList: true });\n \n console.log('[Universal Dark Mode] Loaded - Press Alt+Shift+D to toggle');\n})();", "Universal Dark Mode"); } } catch(__e) { console.warn('[Userscript:Universal Dark Mode]', __e); } })(); })();
Skip to content
This repository was archived by the owner on Aug 29, 2025. It is now read-only.

Scheduler Stage 3 - #488

Merged
n-riesco merged 59 commits into
masterfrom
features/scheduler-stage-3
Jul 24, 2018
Merged

Scheduler Stage 3#488
n-riesco merged 59 commits into
masterfrom
features/scheduler-stage-3

Conversation

@mfix22

@mfix22mfix22 commented Jul 11, 2018

Copy link
Copy Markdown
Contributor

(Moved PR #487 here)

This is a WIP PR that adds cronInterval support to the frontend 👍

We will shortly add the backend support and merge it in with this branch. We just wanted to get this up for @n-riesco to review before doing so.

TODO:

  • Add backend support for scheduling
  • Finish off unit testing components
  • Run Prettier

mfix22 added 7 commits July 10, 2018 15:46
 Fix linting from Prettier
get initial cron mode id from query
Add login button to preview modal
Move getInitialCronMode to cron-helpers
Fix linting from Prettier
get initial cron mode id from query
Add login button to preview modal
Move getInitialCronMode to cron-helpers
@mfix22
mfix22 requested a review from n-riescoJuly 11, 2018 23:37
@mfix22mfix22 mentioned this pull request Jul 11, 2018
3 tasks
import ReactDataGrid from 'react-data-grid';
import ms from 'ms';
import matchSorter from 'match-sorter';
import * as prettyCron from 'prettycron';

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

let's use something lighter that doesn't depend on later: https://www.npmjs.com/package/cronstrue


const initialModeId = getInitialCronMode(props.query);

const canEdit = this.props.currentRequestor && this.props.currentRequestor === props.query.requestor;

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nice!

.simulate('click');

// onSubmit is asynchronous
await wait();

@n-riescon-riescoJul 12, 2018

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

if i remember correctly, our webpack/babel/mocha setup isn't ready to use await.

@mfix22mfix22Jul 12, 2018

Copy link
Copy Markdown
ContributorAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hmm. This worked for me though. Should i remove it and use setTimeout directly instead?

@n-riescon-riescoJul 12, 2018

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please, ignore my comment. It doesn't apply any longer. We upgraded to Node 8 recently.

Comment threadpackage.json Outdated
},
"dependencies": {
"alasql": "^0.4.5",
"cronstrue": "^1.31.0",

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

move to devDependencies

});
});

it('should throw GraphQL errors on failure', () => {

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

graphQL?

});

describe('Creating Queries', () => {
it('should handle success cases', () => {

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

how about should schedule a query only if logged in?

});

describe('Updating Queries', () => {
it('should handle success cases', () => {

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

how about should update a scheduled query only if logged in?

});
});

it('should throw GraphQL errors on failure', () => {

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

garphQL?

@n-riesco

Copy link
Copy Markdown
Contributor

Looks good so far. I'd suggest the following changes:

  1. I've tested if it's possible for the Log In button to navigate directly to the Login page and it doesn't seem to break anything:

    --- a/app/components/Settings/Settings.react.js+++ b/app/components/Settings/Settings.react.js@@ -25,7 +25,7 @@ class Settings extends Component {
    this.renderSettingsForm = this.renderSettingsForm.bind(this);
    this.updateSelectedPanel = this.updateSelectedPanel.bind(this);
    this.openScheduler = this.openScheduler.bind(this);
    - this.openLogin = this.updateSelectedPanel.bind(this, 3);+ this.openLogin = () => window.location.assign('/login');
    this.state = {
    editMode: true,
    selectedPanel: {},
  2. (after the on-prem release) I'd like to use a better default than DEFAULT_REFRESH_INTERVAL (I know that the monthly case isn't possible, but we use the maximum value setInterval can take).


How's the work on the backend progressing?

Comment threadtest/backend/QueryScheduler.spec.js Outdated
const spy = sinon.spy();
const refreshInterval = 1; // second
const refreshInterval = 5; // 5 seconds
const cronInterval = `*/${refreshInterval} * * * * *`;

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

we should keep the units for testing the old format (i.e. undefined cronInterval).

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The problem with keeping refreshInterval is that it is handled differently in the queryScheduler.scheduleQuery call. Currently, refresh intervals are mapped to their corresponding cron expression before being scheduled. If they match one of the supported values, they're converted accordingly, but if they're not recognized as a supported value (as is the case of the value passed in the test), they default to a schedule of every week.

But I can update the code to accept a small enough refreshInterval to test if you'd like? I had assumed you wouldn't want me to alter the desired logic of the scheduler solely to conform to an existing test, which is why I originally opted to update the test units instead.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

On second thought, I updated the tests to use fake timers which eliminates this issue. The original refereshInterval format has been restored.

@nicolaskruchten

Copy link
Copy Markdown
Contributor

Cool. Just glancing at the code, there's a WAITING_MESSAGE constant that contains text I've never seen in the UI... when does that show up?

@briandennis

Copy link
Copy Markdown
Contributor

It shows up after taking more than 5 seconds to save/update. If you're trying to test it, we've been adding wait functions to our queries to force longer runtime.

@briandennisbriandennis changed the title WIP: Scheduler Stage 3 (Cron support)Scheduler Stage 3Jul 21, 2018
Comment threadpackage.json
"pg-hstore": "^2.3.2",
"restify": "^4.3.2",
"sequelize": "^3.30.4",
"sequelize": "3.30.4",

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

why?

Comment threadpackage.json
"bunyan": "^1.8.12",
"chai": "^4.1.2",
"chromedriver": "^2.33.2",
"chromedriver": "2.33.2",

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

why?

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Similar to the situation with sequelize, Chromedriver pulled in a minor patch which was throwing on CI (for example here). Freezing the version at 2.33.2 resolved the error.

@n-riesco

Copy link
Copy Markdown
Contributor

@briandennis

  • about this failure on circleCI: I've recently noticed this failure. It's not a persistent failure. It goes away after a rebuild. I reckon this is an issue with out connector for redshift.

  • Thank you for fixing the overflow in SQLTable!

  • I've noticed a small issue while editing queries with a multi-selection of weekdays. See screen capture below:

peek 2018-07-23 15-25

@briandennis

Copy link
Copy Markdown
Contributor

@n-riesco re the circleCI failure: I tried rebuilding a few times and it continued to break on that specific Redshift test with that error. I then tried connecting to redshift manually and also hit the column "typarray" does not exist in pg_type error while connecting.

After some googling, it looks like this was a backwards compatibility issue with the Postgres version Redshift uses under the hood. The issue was introduced in a minor version of sequelize which is why it got pulled in when updating the yarn.lock in this PR.

Freezing the version at 3.30.4 fixed the problem, but let me know if you think there's a better solution!

@briandennis

Copy link
Copy Markdown
Contributor

@n-riesco the weekday input initialization has been fixed!

I think this PR is at a stage where it's ready to be merged into master. Keep in mind, we're doing at least one more PR after this that addresses any UX updates and bug fixes found while doing final polish and testing before the code freeze. I think we should handle any additional changes there.

What do you think? Are we good to merge this in?

</Row>
{ModeComponent ? <ModeComponent onChange={this.props.onChange} /> : null}
{ModeComponent ? (
// eslint-disable-next-line

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

why?

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

if the initialTime is null, we want to avoid overriding the default props for the mode by passing undefined. Setting anything to undefined however causes a lint error.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'll defer the decision onthis one to @nicolaskruchten .

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm OK with this, perhaps with a comment above the disable one to explain the reason, as it's not obvious.

}

export function mapCronToHourFormat(cronExpression) {
if (!cronExpression) {

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

How about validating the cron expressions using cronstrue (untested)?

try{cronstrue.toString(cronExpression);}catch(_){returnnull;}

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

awesome idea! I added that in

@n-riesco

Copy link
Copy Markdown
Contributor

I'll have another go at testing the last changes tomorrow, but the code looks good to me other than the use of eslint-disable-next-line.

See also the suggestion that cronstrue is used to validate the input to mapCronToHourFormat.

@nicolaskruchten

Copy link
Copy Markdown
Contributor

OK, let's merge this in and have all new changes go into a cleanup commit.

Perhaps in the future when doing multi-stage work like this it would make sense to create a new branch to represent the whole feature and then do small PRs into that branch and then one final review on the branch before it hits master?

@nicolaskruchten

Copy link
Copy Markdown
Contributor

(Could we get the tests to pass first though?)

@n-riesco
n-riesco merged commit 35b029e into masterJul 24, 2018
@mfix22
mfix22 deleted the features/scheduler-stage-3 branch August 8, 2018 01:29
Sign up for freeto subscribe to this conversation on GitHub. Already have an account? Sign in.

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

5 participants

@mfix22@n-riesco@briandennis@nicolaskruchten@jakedex