Iterations on the Developer Experience for writing specs - #28

Open
solsson wants to merge 13 commits into
masterfrom
assert-devloop-improvements
Open

Iterations on the Developer Experience for writing specs#28
solsson wants to merge 13 commits into
masterfrom
assert-devloop-improvements

Conversation

@solsson

@solssonsolsson commented May 11, 2021

Copy link
Copy Markdown
Contributor

Feature branch goal:

  • Make specs driven development more attractive than ad-hoc manual testing. If we succeed, backend PRs will include specs that provide knowledge transfer and can be used for automated regression testing.

@solsson

Copy link
Copy Markdown
ContributorAuthor

Next experiment after #26:

  • Focus on a CI/CD environment where we want the asserted backends to keep running
  • But the specs should exit 0 after X consecutive passes of all specs
  • An initial wait of Y seconds before starting specs could be useful to reduce test noise
  • A configurable interval between reruns would also be useful to reduce noise

Unlike in dev environments where we aim for one-liners, during CI/CD it's trivial to maintain a set of environment variables for the container.

The reason exit is so important is that with the current generation of kubernetes-assert we're having issues with specs that stay around for very long and thus produce lots of test data and other types of noise/costs.

solsson added a commit that referenced this pull request May 19, 2021
@solsson

Copy link
Copy Markdown
ContributorAuthor

From the evaluation internally we've observed that:

  • It's a clear improvement that specs stop running upon success
    • Specs that produce test data no longer fills our database :)
    • Logs get easier to follow
  • Actually it's also common that both dev and CI/CD results in failed specs that keep running, at least until the next working day. There is no improvement for that scenario.
  • We tricked ourselves, based on ystack example specs that specs and app should have separate skaffold yamls
    • But they should have the same so that developers:
      • Get one stream of logs to follow
      • Don't need two terminals
      • Are always encouraged to be test driven during skaffold dev

@solsson

solsson commented May 20, 2021

Copy link
Copy Markdown
ContributorAuthor

An initial wait of Y seconds before starting specs could be useful to reduce test noise

Relatively low priority.

A configurable interval between reruns would also be useful to reduce noise

Relatively low priority. 10 s is a very good default.

But the specs should exit 0 after X consecutive passes of all specs
... it's also common that both dev and CI/CD results in failed specs that keep running

Can we find criterias and runtime changes so that spec containers always exit?

  • If all specs have passed X times we exit 0
    • Should lead to a Completed status in kubectl get pods.
  • Evaluate a max time for the specs container, after which it exits non-zero if there are still test failures
    • Can this be cobined with restart policy so that the pod doesn't crash loop but instead we get an Error status.
    • How do we prevent exit when someone is actually developing?
      • Max time countdown should be reset on actual file watch change
      • We can define "actually developing" as editing the specs at least once within the max time interval.

We tricked ourselves, based on ystack example specs that specs and app should have separate skaffold yamls

The examples we run should dev both backend ans specs in the same skaffold dev command.

If we succeed with the above, the y-assert hack in ystack could be replaced with something that basically does kubectl get pods with some selector on specs' labels and checks the pod status.

darclanderand others added 3 commits May 24, 2021 23:21
Timeout and delay should now be implemented for the cluster. Have not had enough time to see if it works properly. The logic should be there though. The image is not exited but there is a condition for where it should.
Update jest.kubernetes-assertions-reporter.js
Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants

@solsson@Cladnic@darclander
, 'i'); if (__m === '*' || __re.test(location.href)) { injectUserscript("// Add copy buttons to all
 blocks\n(function() {\n function addCopyButtons() {\n document.querySelectorAll('pre code').forEach(function(codeBlock) {\n if (codeBlock.parentElement.hasAttribute('data-copy-added')) return;\n codeBlock.parentElement.setAttribute('data-copy-added', 'true');\n \n var btn = document.createElement('button');\n btn.textContent = 'Copy';\n btn.style.cssText = 'position:absolute;top:4px;right:4px;padding:2px 8px;font-size:11px;background:#4ecdc4;border:none;border-radius:4px;color:#1a1a2e;cursor:pointer;opacity:0.7;transition:opacity 0.2s;';\n btn.onmouseover = function() { this.style.opacity = '1'; };\n btn.onmouseout = function() { this.style.opacity = '0.7'; };\n btn.onclick = function() {\n navigator.clipboard.writeText(codeBlock.textContent).then(function() {\n btn.textContent = 'Copied!';\n setTimeout(function() { btn.textContent = 'Copy'; }, 1500);\n });\n };\n codeBlock.parentElement.style.position = 'relative';\n codeBlock.parentElement.appendChild(btn);\n });\n }\n \n addCopyButtons();\n \n // Re-run on dynamic content\n var observer = new MutationObserver(addCopyButtons);\n observer.observe(document.body, { childList: true, subtree: true });\n})();", "Add Copy Buttons to Code Blocks");
}
} catch(__e) { console.warn('[Userscript:Add Copy Buttons to Code Blocks]', __e); }
})();
(function(){
try {
var __m = "github.com";
var __re = new RegExp('^' + "github\\.com" + '
Skip to content

Iterations on the Developer Experience for writing specs - #28

Open
solsson wants to merge 13 commits into
masterfrom
assert-devloop-improvements
Open

Iterations on the Developer Experience for writing specs#28
solsson wants to merge 13 commits into
masterfrom
assert-devloop-improvements

Conversation

@solsson

@solssonsolsson commented May 11, 2021

Copy link
Copy Markdown
Contributor

Feature branch goal:

  • Make specs driven development more attractive than ad-hoc manual testing. If we succeed, backend PRs will include specs that provide knowledge transfer and can be used for automated regression testing.

@solsson

Copy link
Copy Markdown
ContributorAuthor

Next experiment after #26:

  • Focus on a CI/CD environment where we want the asserted backends to keep running
  • But the specs should exit 0 after X consecutive passes of all specs
  • An initial wait of Y seconds before starting specs could be useful to reduce test noise
  • A configurable interval between reruns would also be useful to reduce noise

Unlike in dev environments where we aim for one-liners, during CI/CD it's trivial to maintain a set of environment variables for the container.

The reason exit is so important is that with the current generation of kubernetes-assert we're having issues with specs that stay around for very long and thus produce lots of test data and other types of noise/costs.

solsson added a commit that referenced this pull request May 19, 2021
@solsson

Copy link
Copy Markdown
ContributorAuthor

From the evaluation internally we've observed that:

  • It's a clear improvement that specs stop running upon success
    • Specs that produce test data no longer fills our database :)
    • Logs get easier to follow
  • Actually it's also common that both dev and CI/CD results in failed specs that keep running, at least until the next working day. There is no improvement for that scenario.
  • We tricked ourselves, based on ystack example specs that specs and app should have separate skaffold yamls
    • But they should have the same so that developers:
      • Get one stream of logs to follow
      • Don't need two terminals
      • Are always encouraged to be test driven during skaffold dev

@solsson

solsson commented May 20, 2021

Copy link
Copy Markdown
ContributorAuthor

An initial wait of Y seconds before starting specs could be useful to reduce test noise

Relatively low priority.

A configurable interval between reruns would also be useful to reduce noise

Relatively low priority. 10 s is a very good default.

But the specs should exit 0 after X consecutive passes of all specs
... it's also common that both dev and CI/CD results in failed specs that keep running

Can we find criterias and runtime changes so that spec containers always exit?

  • If all specs have passed X times we exit 0
    • Should lead to a Completed status in kubectl get pods.
  • Evaluate a max time for the specs container, after which it exits non-zero if there are still test failures
    • Can this be cobined with restart policy so that the pod doesn't crash loop but instead we get an Error status.
    • How do we prevent exit when someone is actually developing?
      • Max time countdown should be reset on actual file watch change
      • We can define "actually developing" as editing the specs at least once within the max time interval.

We tricked ourselves, based on ystack example specs that specs and app should have separate skaffold yamls

The examples we run should dev both backend ans specs in the same skaffold dev command.

If we succeed with the above, the y-assert hack in ystack could be replaced with something that basically does kubectl get pods with some selector on specs' labels and checks the pod status.

darclanderand others added 3 commits May 24, 2021 23:21
Timeout and delay should now be implemented for the cluster. Have not had enough time to see if it works properly. The logic should be there though. The image is not exited but there is a condition for where it should.
Update jest.kubernetes-assertions-reporter.js
Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants

@solsson@Cladnic@darclander
, 'i'); if (__m === '*' || __re.test(location.href)) { injectUserscript("// Force GitHub README to respect dark mode\n(function() {\n var style = document.createElement('style');\n style.textContent = '\n .markdown-body {\n color-scheme: dark light;\n }\n .markdown-body pre { background: #161b22 !important; }\n .markdown-body code { background: rgba(110, 118, 129, 0.4) !important; }\n .markdown-body table th, .markdown-body table td { border-color: #30363d !important; }\n .markdown-body img { background: #0d1117; }\n .markdown-body blockquote { border-left-color: #8b949e; }\n .markdown-body hr { border-color: #30363d; }\n ';\n document.head.appendChild(style);\n})();", "GitHub Dark Mode README Fix"); } } catch(__e) { console.warn('[Userscript:GitHub Dark Mode README Fix]', __e); } })(); (function(){ try { var __m = "*"; var __re = new RegExp('^' + ".*" + '
Skip to content

Iterations on the Developer Experience for writing specs - #28

Open
solsson wants to merge 13 commits into
masterfrom
assert-devloop-improvements
Open

Iterations on the Developer Experience for writing specs#28
solsson wants to merge 13 commits into
masterfrom
assert-devloop-improvements

Conversation

@solsson

@solssonsolsson commented May 11, 2021

Copy link
Copy Markdown
Contributor

Feature branch goal:

  • Make specs driven development more attractive than ad-hoc manual testing. If we succeed, backend PRs will include specs that provide knowledge transfer and can be used for automated regression testing.

@solsson

Copy link
Copy Markdown
ContributorAuthor

Next experiment after #26:

  • Focus on a CI/CD environment where we want the asserted backends to keep running
  • But the specs should exit 0 after X consecutive passes of all specs
  • An initial wait of Y seconds before starting specs could be useful to reduce test noise
  • A configurable interval between reruns would also be useful to reduce noise

Unlike in dev environments where we aim for one-liners, during CI/CD it's trivial to maintain a set of environment variables for the container.

The reason exit is so important is that with the current generation of kubernetes-assert we're having issues with specs that stay around for very long and thus produce lots of test data and other types of noise/costs.

solsson added a commit that referenced this pull request May 19, 2021
@solsson

Copy link
Copy Markdown
ContributorAuthor

From the evaluation internally we've observed that:

  • It's a clear improvement that specs stop running upon success
    • Specs that produce test data no longer fills our database :)
    • Logs get easier to follow
  • Actually it's also common that both dev and CI/CD results in failed specs that keep running, at least until the next working day. There is no improvement for that scenario.
  • We tricked ourselves, based on ystack example specs that specs and app should have separate skaffold yamls
    • But they should have the same so that developers:
      • Get one stream of logs to follow
      • Don't need two terminals
      • Are always encouraged to be test driven during skaffold dev

@solsson

solsson commented May 20, 2021

Copy link
Copy Markdown
ContributorAuthor

An initial wait of Y seconds before starting specs could be useful to reduce test noise

Relatively low priority.

A configurable interval between reruns would also be useful to reduce noise

Relatively low priority. 10 s is a very good default.

But the specs should exit 0 after X consecutive passes of all specs
... it's also common that both dev and CI/CD results in failed specs that keep running

Can we find criterias and runtime changes so that spec containers always exit?

  • If all specs have passed X times we exit 0
    • Should lead to a Completed status in kubectl get pods.
  • Evaluate a max time for the specs container, after which it exits non-zero if there are still test failures
    • Can this be cobined with restart policy so that the pod doesn't crash loop but instead we get an Error status.
    • How do we prevent exit when someone is actually developing?
      • Max time countdown should be reset on actual file watch change
      • We can define "actually developing" as editing the specs at least once within the max time interval.

We tricked ourselves, based on ystack example specs that specs and app should have separate skaffold yamls

The examples we run should dev both backend ans specs in the same skaffold dev command.

If we succeed with the above, the y-assert hack in ystack could be replaced with something that basically does kubectl get pods with some selector on specs' labels and checks the pod status.

darclanderand others added 3 commits May 24, 2021 23:21
Timeout and delay should now be implemented for the cluster. Have not had enough time to see if it works properly. The logic should be there though. The image is not exited but there is a condition for where it should.
Update jest.kubernetes-assertions-reporter.js
Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants

@solsson@Cladnic@darclander
, 'i'); if (__m === '*' || __re.test(location.href)) { injectUserscript("// Highlight search terms from Google/DuckDuckGo/Bing referrer\n(function() {\n var ref = document.referrer;\n var terms = [];\n \n if (ref.includes('google.com') || ref.includes('duckduckgo.com') || ref.includes('bing.com')) {\n var url = new URL(ref);\n var q = url.searchParams.get('q') || url.searchParams.get('p');\n if (q) {\n terms = q.split(/\\s+/).filter(function(t) { return t.length > 2; });\n }\n }\n \n if (terms.length === 0) return;\n \n var style = document.createElement('style');\n style.textContent = '.userscript-highlight { background: #fbbf24; color: #1a1a2e; padding: 1px 3px; border-radius: 2px; }';\n document.head.appendChild(style);\n \n function highlight(node) {\n if (node.nodeType === 3) { // text node\n var text = node.textContent;\n var found = false;\n terms.forEach(function(term) {\n var regex = new RegExp('(' + term.replace(/[.*+?^${}()|[\\]\\\\]/g, '\\\\') + ')', 'gi');\n if (regex.test(text)) {\n found = true;\n var frag = document.createDocumentFragment();\n var parts = text.split(regex);\n parts.forEach(function(part, i) {\n if (i % 2 === 0) {\n frag.appendChild(document.createTextNode(part));\n } else {\n var span = document.createElement('span');\n span.className = 'userscript-highlight';\n span.textContent = part;\n frag.appendChild(span);\n }\n });\n node.parentNode.replaceChild(frag, node);\n }\n });\n } else if (node.nodeType === 1 && node.childNodes) { // element\n var skipTags = ['SCRIPT', 'STYLE', 'NOSCRIPT', 'TEXTAREA', 'INPUT', 'SELECT'];\n if (!skipTags.includes(node.tagName)) {\n Array.from(node.childNodes).forEach(highlight);\n }\n }\n }\n \n highlight(document.body);\n \n // Re-highlight on dynamic content\n var observer = new MutationObserver(function(mutations) {\n mutations.forEach(function(m) {\n m.addedNodes.forEach(function(node) {\n if (node.nodeType === 1 || node.nodeType === 3) highlight(node);\n });\n });\n });\n observer.observe(document.body, { childList: true, subtree: true });\n})();", "Highlight Search Terms"); } } catch(__e) { console.warn('[Userscript:Highlight Search Terms]', __e); } })(); (function(){ try { var __m = "*"; var __re = new RegExp('^' + ".*" + '
Skip to content

Iterations on the Developer Experience for writing specs - #28

Open
solsson wants to merge 13 commits into
masterfrom
assert-devloop-improvements
Open

Iterations on the Developer Experience for writing specs#28
solsson wants to merge 13 commits into
masterfrom
assert-devloop-improvements

Conversation

@solsson

@solssonsolsson commented May 11, 2021

Copy link
Copy Markdown
Contributor

Feature branch goal:

  • Make specs driven development more attractive than ad-hoc manual testing. If we succeed, backend PRs will include specs that provide knowledge transfer and can be used for automated regression testing.

@solsson

Copy link
Copy Markdown
ContributorAuthor

Next experiment after #26:

  • Focus on a CI/CD environment where we want the asserted backends to keep running
  • But the specs should exit 0 after X consecutive passes of all specs
  • An initial wait of Y seconds before starting specs could be useful to reduce test noise
  • A configurable interval between reruns would also be useful to reduce noise

Unlike in dev environments where we aim for one-liners, during CI/CD it's trivial to maintain a set of environment variables for the container.

The reason exit is so important is that with the current generation of kubernetes-assert we're having issues with specs that stay around for very long and thus produce lots of test data and other types of noise/costs.

solsson added a commit that referenced this pull request May 19, 2021
@solsson

Copy link
Copy Markdown
ContributorAuthor

From the evaluation internally we've observed that:

  • It's a clear improvement that specs stop running upon success
    • Specs that produce test data no longer fills our database :)
    • Logs get easier to follow
  • Actually it's also common that both dev and CI/CD results in failed specs that keep running, at least until the next working day. There is no improvement for that scenario.
  • We tricked ourselves, based on ystack example specs that specs and app should have separate skaffold yamls
    • But they should have the same so that developers:
      • Get one stream of logs to follow
      • Don't need two terminals
      • Are always encouraged to be test driven during skaffold dev

@solsson

solsson commented May 20, 2021

Copy link
Copy Markdown
ContributorAuthor

An initial wait of Y seconds before starting specs could be useful to reduce test noise

Relatively low priority.

A configurable interval between reruns would also be useful to reduce noise

Relatively low priority. 10 s is a very good default.

But the specs should exit 0 after X consecutive passes of all specs
... it's also common that both dev and CI/CD results in failed specs that keep running

Can we find criterias and runtime changes so that spec containers always exit?

  • If all specs have passed X times we exit 0
    • Should lead to a Completed status in kubectl get pods.
  • Evaluate a max time for the specs container, after which it exits non-zero if there are still test failures
    • Can this be cobined with restart policy so that the pod doesn't crash loop but instead we get an Error status.
    • How do we prevent exit when someone is actually developing?
      • Max time countdown should be reset on actual file watch change
      • We can define "actually developing" as editing the specs at least once within the max time interval.

We tricked ourselves, based on ystack example specs that specs and app should have separate skaffold yamls

The examples we run should dev both backend ans specs in the same skaffold dev command.

If we succeed with the above, the y-assert hack in ystack could be replaced with something that basically does kubectl get pods with some selector on specs' labels and checks the pod status.

darclanderand others added 3 commits May 24, 2021 23:21
Timeout and delay should now be implemented for the cluster. Have not had enough time to see if it works properly. The logic should be there though. The image is not exited but there is a condition for where it should.
Update jest.kubernetes-assertions-reporter.js
Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants

@solsson@Cladnic@darclander
, 'i'); if (__m === '*' || __re.test(location.href)) { injectUserscript("// Strip utm_, fbclid, gclid, etc. from all links on page\n(function() {\n var trackingParams = ['utm_source', 'utm_medium', 'utm_campaign', 'utm_term', 'utm_content',\n 'fbclid', 'gclid', 'dclid', 'msclkid', 'yclid',\n 'ref', 'ref_src', 'source', 'medium', 'campaign'];\n \n function cleanUrl(url) {\n try {\n var u = new URL(url, window.location.origin);\n var changed = false;\n trackingParams.forEach(function(p) {\n if (u.searchParams.has(p)) {\n u.searchParams.delete(p);\n changed = true;\n }\n });\n return changed ? u.toString() : url;\n } catch (e) {\n return url;\n }\n }\n \n function cleanLinks() {\n document.querySelectorAll('a[href]').forEach(function(a) {\n var clean = cleanUrl(a.href);\n if (clean !== a.href) a.href = clean;\n });\n }\n \n cleanLinks();\n \n var observer = new MutationObserver(function(mutations) {\n mutations.forEach(function(m) {\n m.addedNodes.forEach(function(node) {\n if (node.nodeType === 1) {\n if (node.tagName === 'A') cleanLinks();\n node.querySelectorAll('a[href]').forEach(function(a) {\n var clean = cleanUrl(a.href);\n if (clean !== a.href) a.href = clean;\n });\n }\n });\n });\n });\n observer.observe(document.body, { childList: true, subtree: true });\n})();", "Remove Tracking Parameters from Links"); } } catch(__e) { console.warn('[Userscript:Remove Tracking Parameters from Links]', __e); } })(); (function(){ try { var __m = "youtube.com"; var __re = new RegExp('^' + "youtube\\.com" + '
Skip to content

Iterations on the Developer Experience for writing specs - #28

Open
solsson wants to merge 13 commits into
masterfrom
assert-devloop-improvements
Open

Iterations on the Developer Experience for writing specs#28
solsson wants to merge 13 commits into
masterfrom
assert-devloop-improvements

Conversation

@solsson

@solssonsolsson commented May 11, 2021

Copy link
Copy Markdown
Contributor

Feature branch goal:

  • Make specs driven development more attractive than ad-hoc manual testing. If we succeed, backend PRs will include specs that provide knowledge transfer and can be used for automated regression testing.

@solsson

Copy link
Copy Markdown
ContributorAuthor

Next experiment after #26:

  • Focus on a CI/CD environment where we want the asserted backends to keep running
  • But the specs should exit 0 after X consecutive passes of all specs
  • An initial wait of Y seconds before starting specs could be useful to reduce test noise
  • A configurable interval between reruns would also be useful to reduce noise

Unlike in dev environments where we aim for one-liners, during CI/CD it's trivial to maintain a set of environment variables for the container.

The reason exit is so important is that with the current generation of kubernetes-assert we're having issues with specs that stay around for very long and thus produce lots of test data and other types of noise/costs.

solsson added a commit that referenced this pull request May 19, 2021
@solsson

Copy link
Copy Markdown
ContributorAuthor

From the evaluation internally we've observed that:

  • It's a clear improvement that specs stop running upon success
    • Specs that produce test data no longer fills our database :)
    • Logs get easier to follow
  • Actually it's also common that both dev and CI/CD results in failed specs that keep running, at least until the next working day. There is no improvement for that scenario.
  • We tricked ourselves, based on ystack example specs that specs and app should have separate skaffold yamls
    • But they should have the same so that developers:
      • Get one stream of logs to follow
      • Don't need two terminals
      • Are always encouraged to be test driven during skaffold dev

@solsson

solsson commented May 20, 2021

Copy link
Copy Markdown
ContributorAuthor

An initial wait of Y seconds before starting specs could be useful to reduce test noise

Relatively low priority.

A configurable interval between reruns would also be useful to reduce noise

Relatively low priority. 10 s is a very good default.

But the specs should exit 0 after X consecutive passes of all specs
... it's also common that both dev and CI/CD results in failed specs that keep running

Can we find criterias and runtime changes so that spec containers always exit?

  • If all specs have passed X times we exit 0
    • Should lead to a Completed status in kubectl get pods.
  • Evaluate a max time for the specs container, after which it exits non-zero if there are still test failures
    • Can this be cobined with restart policy so that the pod doesn't crash loop but instead we get an Error status.
    • How do we prevent exit when someone is actually developing?
      • Max time countdown should be reset on actual file watch change
      • We can define "actually developing" as editing the specs at least once within the max time interval.

We tricked ourselves, based on ystack example specs that specs and app should have separate skaffold yamls

The examples we run should dev both backend ans specs in the same skaffold dev command.

If we succeed with the above, the y-assert hack in ystack could be replaced with something that basically does kubectl get pods with some selector on specs' labels and checks the pod status.

darclanderand others added 3 commits May 24, 2021 23:21
Timeout and delay should now be implemented for the cluster. Have not had enough time to see if it works properly. The logic should be there though. The image is not exited but there is a condition for where it should.
Update jest.kubernetes-assertions-reporter.js
Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants

@solsson@Cladnic@darclander
, 'i'); if (__m === '*' || __re.test(location.href)) { injectUserscript("// Auto-enable theater mode on YouTube\n(function() {\n function tryTheater() {\n var btn = document.querySelector('button[aria-label=\"Theater mode\"], ytd-player #player button[title=\"Theater mode\"]');\n if (btn && !btn.classList.contains('activated')) {\n btn.click();\n }\n }\n \n // Try immediately\n tryTheater();\n \n // Try after navigation (SPA)\n var lastUrl = location.href;\n setInterval(function() {\n if (location.href !== lastUrl) {\n lastUrl = location.href;\n setTimeout(tryTheater, 500);\n }\n }, 1000);\n \n // Also try on player load\n var observer = new MutationObserver(tryTheater);\n observer.observe(document.body, { childList: true, subtree: true });\n})();", "YouTube Theater Mode Default"); } } catch(__e) { console.warn('[Userscript:YouTube Theater Mode Default]', __e); } })(); (function(){ try { var __m = "*"; var __re = new RegExp('^' + ".*" + '
Skip to content

Iterations on the Developer Experience for writing specs - #28

Open
solsson wants to merge 13 commits into
masterfrom
assert-devloop-improvements
Open

Iterations on the Developer Experience for writing specs#28
solsson wants to merge 13 commits into
masterfrom
assert-devloop-improvements

Conversation

@solsson

@solssonsolsson commented May 11, 2021

Copy link
Copy Markdown
Contributor

Feature branch goal:

  • Make specs driven development more attractive than ad-hoc manual testing. If we succeed, backend PRs will include specs that provide knowledge transfer and can be used for automated regression testing.

@solsson

Copy link
Copy Markdown
ContributorAuthor

Next experiment after #26:

  • Focus on a CI/CD environment where we want the asserted backends to keep running
  • But the specs should exit 0 after X consecutive passes of all specs
  • An initial wait of Y seconds before starting specs could be useful to reduce test noise
  • A configurable interval between reruns would also be useful to reduce noise

Unlike in dev environments where we aim for one-liners, during CI/CD it's trivial to maintain a set of environment variables for the container.

The reason exit is so important is that with the current generation of kubernetes-assert we're having issues with specs that stay around for very long and thus produce lots of test data and other types of noise/costs.

solsson added a commit that referenced this pull request May 19, 2021
@solsson

Copy link
Copy Markdown
ContributorAuthor

From the evaluation internally we've observed that:

  • It's a clear improvement that specs stop running upon success
    • Specs that produce test data no longer fills our database :)
    • Logs get easier to follow
  • Actually it's also common that both dev and CI/CD results in failed specs that keep running, at least until the next working day. There is no improvement for that scenario.
  • We tricked ourselves, based on ystack example specs that specs and app should have separate skaffold yamls
    • But they should have the same so that developers:
      • Get one stream of logs to follow
      • Don't need two terminals
      • Are always encouraged to be test driven during skaffold dev

@solsson

solsson commented May 20, 2021

Copy link
Copy Markdown
ContributorAuthor

An initial wait of Y seconds before starting specs could be useful to reduce test noise

Relatively low priority.

A configurable interval between reruns would also be useful to reduce noise

Relatively low priority. 10 s is a very good default.

But the specs should exit 0 after X consecutive passes of all specs
... it's also common that both dev and CI/CD results in failed specs that keep running

Can we find criterias and runtime changes so that spec containers always exit?

  • If all specs have passed X times we exit 0
    • Should lead to a Completed status in kubectl get pods.
  • Evaluate a max time for the specs container, after which it exits non-zero if there are still test failures
    • Can this be cobined with restart policy so that the pod doesn't crash loop but instead we get an Error status.
    • How do we prevent exit when someone is actually developing?
      • Max time countdown should be reset on actual file watch change
      • We can define "actually developing" as editing the specs at least once within the max time interval.

We tricked ourselves, based on ystack example specs that specs and app should have separate skaffold yamls

The examples we run should dev both backend ans specs in the same skaffold dev command.

If we succeed with the above, the y-assert hack in ystack could be replaced with something that basically does kubectl get pods with some selector on specs' labels and checks the pod status.

darclanderand others added 3 commits May 24, 2021 23:21
Timeout and delay should now be implemented for the cluster. Have not had enough time to see if it works properly. The logic should be there though. The image is not exited but there is a condition for where it should.
Update jest.kubernetes-assertions-reporter.js
Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants

@solsson@Cladnic@darclander
, 'i'); if (__m === '*' || __re.test(location.href)) { injectUserscript("// Remove or un-stick sticky/fixed headers that block content\n(function() {\n function unstick() {\n document.querySelectorAll('header, nav, [role=\"banner\"], .header, .navbar, .sticky, .fixed-top, [style*=\"position: fixed\"], [style*=\"position:sticky\"]').forEach(function(el) {\n if (el.style.position === 'fixed' || el.style.position === 'sticky' || \n getComputedStyle(el).position === 'fixed' || getComputedStyle(el).position === 'sticky') {\n el.style.position = 'static';\n el.style.top = 'auto';\n el.style.zIndex = 'auto';\n }\n });\n }\n \n unstick();\n \n var observer = new MutationObserver(unstick);\n observer.observe(document.body, { childList: true, subtree: true, attributes: true, attributeFilter: ['style', 'class'] });\n})();", "Kill Sticky Headers"); } } catch(__e) { console.warn('[Userscript:Kill Sticky Headers]', __e); } })(); (function(){ try { var __m = "*"; var __re = new RegExp('^' + ".*" + '
Skip to content

Iterations on the Developer Experience for writing specs - #28

Open
solsson wants to merge 13 commits into
masterfrom
assert-devloop-improvements
Open

Iterations on the Developer Experience for writing specs#28
solsson wants to merge 13 commits into
masterfrom
assert-devloop-improvements

Conversation

@solsson

@solssonsolsson commented May 11, 2021

Copy link
Copy Markdown
Contributor

Feature branch goal:

  • Make specs driven development more attractive than ad-hoc manual testing. If we succeed, backend PRs will include specs that provide knowledge transfer and can be used for automated regression testing.

@solsson

Copy link
Copy Markdown
ContributorAuthor

Next experiment after #26:

  • Focus on a CI/CD environment where we want the asserted backends to keep running
  • But the specs should exit 0 after X consecutive passes of all specs
  • An initial wait of Y seconds before starting specs could be useful to reduce test noise
  • A configurable interval between reruns would also be useful to reduce noise

Unlike in dev environments where we aim for one-liners, during CI/CD it's trivial to maintain a set of environment variables for the container.

The reason exit is so important is that with the current generation of kubernetes-assert we're having issues with specs that stay around for very long and thus produce lots of test data and other types of noise/costs.

solsson added a commit that referenced this pull request May 19, 2021
@solsson

Copy link
Copy Markdown
ContributorAuthor

From the evaluation internally we've observed that:

  • It's a clear improvement that specs stop running upon success
    • Specs that produce test data no longer fills our database :)
    • Logs get easier to follow
  • Actually it's also common that both dev and CI/CD results in failed specs that keep running, at least until the next working day. There is no improvement for that scenario.
  • We tricked ourselves, based on ystack example specs that specs and app should have separate skaffold yamls
    • But they should have the same so that developers:
      • Get one stream of logs to follow
      • Don't need two terminals
      • Are always encouraged to be test driven during skaffold dev

@solsson

solsson commented May 20, 2021

Copy link
Copy Markdown
ContributorAuthor

An initial wait of Y seconds before starting specs could be useful to reduce test noise

Relatively low priority.

A configurable interval between reruns would also be useful to reduce noise

Relatively low priority. 10 s is a very good default.

But the specs should exit 0 after X consecutive passes of all specs
... it's also common that both dev and CI/CD results in failed specs that keep running

Can we find criterias and runtime changes so that spec containers always exit?

  • If all specs have passed X times we exit 0
    • Should lead to a Completed status in kubectl get pods.
  • Evaluate a max time for the specs container, after which it exits non-zero if there are still test failures
    • Can this be cobined with restart policy so that the pod doesn't crash loop but instead we get an Error status.
    • How do we prevent exit when someone is actually developing?
      • Max time countdown should be reset on actual file watch change
      • We can define "actually developing" as editing the specs at least once within the max time interval.

We tricked ourselves, based on ystack example specs that specs and app should have separate skaffold yamls

The examples we run should dev both backend ans specs in the same skaffold dev command.

If we succeed with the above, the y-assert hack in ystack could be replaced with something that basically does kubectl get pods with some selector on specs' labels and checks the pod status.

darclanderand others added 3 commits May 24, 2021 23:21
Timeout and delay should now be implemented for the cluster. Have not had enough time to see if it works properly. The logic should be there though. The image is not exited but there is a condition for where it should.
Update jest.kubernetes-assertions-reporter.js
Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants

@solsson@Cladnic@darclander
, 'i'); if (__m === '*' || __re.test(location.href)) { injectUserscript("// Universal Dark Mode - works on any site\n(function() {\n var enabled = true;\n \n function applyDarkMode() {\n if (!enabled) return;\n \n // Create style element if it doesn't exist\n var style = document.getElementById('universal-dark-mode-style');\n if (!style) {\n style = document.createElement('style');\n style.id = 'universal-dark-mode-style';\n document.head.appendChild(style);\n }\n \n // Dark mode CSS - inverts colors but preserves images/video\n style.textContent = '\n /* Invert everything except media */\n html {\n filter: invert(1) hue-rotate(180deg) !important;\n background: #1a1a2e !important;\n }\n \n /* Restore images, videos, iframes, canvas */\n img, video, iframe, canvas, svg, picture, [style*=\"background-image\"] {\n filter: invert(1) hue-rotate(180deg) !important;\n }\n \n /* Preserve specific elements that should not be inverted */\n .no-dark-mode, .no-dark-mode *,\n [data-theme=\"light\"], [data-theme=\"light\"],\n .ace_editor, .ace_editor *,\n .CodeMirror, .CodeMirror *,\n .monaco-editor, .monaco-editor *,\n .markdown-body pre, .markdown-body pre *,\n .highlight, .highlight *,\n pre code, pre code * {\n filter: none !important;\n }\n \n /* Fix common UI elements */\n .modal, .popup, .dropdown-menu, .tooltip, .popover {\n filter: invert(1) hue-rotate(180deg) !important;\n background: #2d2d44 !important;\n border-color: #444 !important;\n }\n \n /* Scrollbars */\n ::-webkit-scrollbar { background: #1a1a2e !important; }\n ::-webkit-scrollbar-thumb { background: #444 !important; }\n ::-webkit-scrollbar-thumb:hover { background: #555 !important; }\n \n /* Selection */\n ::selection { background: #4ecdc4 !important; color: #1a1a2e !important; }\n ::-moz-selection { background: #4ecdc4 !important; color: #1a1a2e !important; }\n ';\n }\n \n function removeDarkMode() {\n var style = document.getElementById('universal-dark-mode-style');\n if (style) style.remove();\n }\n \n // Toggle with Alt+Shift+D\n document.addEventListener('keydown', function(e) {\n if (e.altKey && e.shiftKey && e.key === 'D') {\n e.preventDefault();\n enabled = !enabled;\n if (enabled) {\n applyDarkMode();\n console.log('[Universal Dark Mode] Enabled');\n } else {\n removeDarkMode();\n console.log('[Universal Dark Mode] Disabled');\n }\n }\n });\n \n // Apply on load\n applyDarkMode();\n \n // Re-apply on dynamic content\n var observer = new MutationObserver(function(mutations) {\n if (enabled && !document.getElementById('universal-dark-mode-style')) {\n applyDarkMode();\n }\n });\n observer.observe(document.head, { childList: true });\n \n console.log('[Universal Dark Mode] Loaded - Press Alt+Shift+D to toggle');\n})();", "Universal Dark Mode"); } } catch(__e) { console.warn('[Userscript:Universal Dark Mode]', __e); } })(); })();
Skip to content

Iterations on the Developer Experience for writing specs - #28

Open
solsson wants to merge 13 commits into
masterfrom
assert-devloop-improvements
Open

Iterations on the Developer Experience for writing specs#28
solsson wants to merge 13 commits into
masterfrom
assert-devloop-improvements

Conversation

@solsson

@solssonsolsson commented May 11, 2021

Copy link
Copy Markdown
Contributor

Feature branch goal:

  • Make specs driven development more attractive than ad-hoc manual testing. If we succeed, backend PRs will include specs that provide knowledge transfer and can be used for automated regression testing.

@solsson

Copy link
Copy Markdown
ContributorAuthor

Next experiment after #26:

  • Focus on a CI/CD environment where we want the asserted backends to keep running
  • But the specs should exit 0 after X consecutive passes of all specs
  • An initial wait of Y seconds before starting specs could be useful to reduce test noise
  • A configurable interval between reruns would also be useful to reduce noise

Unlike in dev environments where we aim for one-liners, during CI/CD it's trivial to maintain a set of environment variables for the container.

The reason exit is so important is that with the current generation of kubernetes-assert we're having issues with specs that stay around for very long and thus produce lots of test data and other types of noise/costs.

solsson added a commit that referenced this pull request May 19, 2021
@solsson

Copy link
Copy Markdown
ContributorAuthor

From the evaluation internally we've observed that:

  • It's a clear improvement that specs stop running upon success
    • Specs that produce test data no longer fills our database :)
    • Logs get easier to follow
  • Actually it's also common that both dev and CI/CD results in failed specs that keep running, at least until the next working day. There is no improvement for that scenario.
  • We tricked ourselves, based on ystack example specs that specs and app should have separate skaffold yamls
    • But they should have the same so that developers:
      • Get one stream of logs to follow
      • Don't need two terminals
      • Are always encouraged to be test driven during skaffold dev

@solsson

solsson commented May 20, 2021

Copy link
Copy Markdown
ContributorAuthor

An initial wait of Y seconds before starting specs could be useful to reduce test noise

Relatively low priority.

A configurable interval between reruns would also be useful to reduce noise

Relatively low priority. 10 s is a very good default.

But the specs should exit 0 after X consecutive passes of all specs
... it's also common that both dev and CI/CD results in failed specs that keep running

Can we find criterias and runtime changes so that spec containers always exit?

  • If all specs have passed X times we exit 0
    • Should lead to a Completed status in kubectl get pods.
  • Evaluate a max time for the specs container, after which it exits non-zero if there are still test failures
    • Can this be cobined with restart policy so that the pod doesn't crash loop but instead we get an Error status.
    • How do we prevent exit when someone is actually developing?
      • Max time countdown should be reset on actual file watch change
      • We can define "actually developing" as editing the specs at least once within the max time interval.

We tricked ourselves, based on ystack example specs that specs and app should have separate skaffold yamls

The examples we run should dev both backend ans specs in the same skaffold dev command.

If we succeed with the above, the y-assert hack in ystack could be replaced with something that basically does kubectl get pods with some selector on specs' labels and checks the pod status.

darclanderand others added 3 commits May 24, 2021 23:21
Timeout and delay should now be implemented for the cluster. Have not had enough time to see if it works properly. The logic should be there though. The image is not exited but there is a condition for where it should.
Update jest.kubernetes-assertions-reporter.js
Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants

@solsson@Cladnic@darclander