Fix new code editor not showing in web app

[amirbiron]

✨ תיאור קצר

הטמעת עורך הקוד CodeMirror 6 בדף עריכת קובץ (edit_file.html). השינוי נדרש מכיוון שהעורך החדש לא הופיע בווב אפ לאחר מימוש האישו, וזה פותר את הבעיה על ידי שילובו ישירות בתבנית.

📦 שינויים עיקריים

• קוד (Backend)
• בוט טלגרם
• מסד נתונים/מיגרציות
• תיעוד (docs/) - שינוי תבנית UI
• DevOps/CI/CD

פירוט נקודות (רשימת תבליטים):

• הוספת CodeMirror 6 (דרך CDN) לדף webapp/templates/edit_file.html.
• הסתרת ה-textarea המקורי ושימוש בו לסנכרון תוכן העורך לפני שליחת הטופס.
• הטמעת טעינה דינמית של הרחבות שפה עבור העורך בהתאם לבחירת המשתמש.
• הוספת עיצוב בסיסי לעורך להתאמה לממשק הקיים.

🧪 בדיקות

• איך בדקתם? מה עבר? מה נשאר?
• Unit
• Integration
• Manual
  • נבדק ידנית בדף עריכת קובץ: העורך החדש מופיע, ניתן להקליד, לשנות שפה, והתוכן נשמר בהצלחה.

🧪 בדיקות נדרשות ב‑PR

• 🔍 Code Quality & Security
• 🧪 Unit Tests (3.11)
• 🧪 Unit Tests (3.12)

📝 סוג שינוי

• feat: פיצ'ר חדש
• fix: תיקון באג
• docs: שינוי תיעוד בלבד
• refactor: שינוי קוד ללא שינוי התנהגות
• perf: שיפור ביצועים
• chore/ci: תשתית/CI
• breaking change: שינוי שובר תאימות

✅ צ'קליסט

• הקוד עוקב אחרי הסגנון (Black/isort/flake8/mypy)
• בדיקות רצות ועוברות
• תיעוד עודכן (README/Docs)
• אין סודות/מפתחות בקוד
• אין מחיקות מסוכנות/פעולות על root (ראו .cursorrules)
• הודעת הקומיט תואמת Conventional Commits (ע"פ הטבלה)
• CHANGELOG עודכן אם נדרש
• כל ה‑Required Checks לעיל ירוקים
• צילום/וידאו UI מצורף אם רלוונטי

🧩 השפעות/סיכונים

• תלות ב-CDN עבור טעינת CodeMirror. אם ה-CDN אינו זמין, העורך לא יטען והמשתמש יראה את ה-textarea הרגיל (fallback מובנה).

🔗 קישורים

• Issues קשורים: 📝 מדריך מימוש: עורך קוד CodeMirror ב-WebApp #733
• Docs Preview:
• מסמכים/מפרטים רלוונטיים:

🧯 סיכון / החזרה לאחור (Rollback)

• מחיקת השינויים בקובץ webapp/templates/edit_file.html. במקרה של תקלה, המערכת תחזור להשתמש ב-textarea הרגיל.

---

[cursor]

Cursor Agent can help with this pull request. Just @cursor in comments and I'll start working on changes in this branch.
_{Learn more about Cursor Agents}

[github-actions]

🧯 Dangerous deletes guard report

Policy: see .cursorrules — dangerous deletions are blocked unless wrapped safely.

Summary:

• Flagged findings (blocking): 0
  0
• Excluded matches (not blocking): 15
• Total matches (all files): 125

Flagged findings (file:line:snippet):

Excluded matches (by path pattern):
./README.md:754:find . -name "pycache" -exec rm -rf {} +
./docs/DOCUMENTATION_GUIDE.md:453:rm -rf _build
./docs/Makefile:24: rm -rf $(BUILDDIR)
./webapp/static/js/md_preview.bundle.js.map:4: "sourcesContent": ["// Markdown-it plugin to render GitHub-style task lists; see\n//\n// https://github.com/blog/1375-task-lists-in-gfm-issues-pulls-comments\n// https://github.com/blog/1825-task-lists-in-all-markdown-documents\n\nvar disableCheckboxes = true;\nvar useLabelWrapper = false;\nvar useLabelAfter = false;\n\nmodule.exports = function(md, options) {\n\tif (options) {\n\t\tdisableCheckboxes = !options.enabled;\n\t\tuseLabelWrapper = !!options.label;\n\t\tuseLabelAfter = !!options.labelAfter;\n\t}\n\n\tmd.core.ruler.after('inline', 'github-task-lists', function(state) {\n\t\tvar tokens = state.tokens;\n\t\tfor (var i = 2; i < tokens.length; i++) {\n\t\t\tif (isTodoItem(tokens, i)) {\n\t\t\t\ttodoify(tokens[i], state.Token);\n\t\t\t\tattrSet(tokens[i-2], 'class', 'task-list-item' + (!disableCheckboxes ? ' enabled' : ''));\n\t\t\t\tattrSet(tokens[parentToken(tokens, i-2)], 'class', 'contains-task-list');\n\t\t\t}\n\t\t}\n\t});\n};\n\nfunction attrSet(token, name, value) {\n\tvar index = token.attrIndex(name);\n\tvar attr = [name, value];\n\n\tif (index < 0) {\n\t\ttoken.attrPush(attr);\n\t} else {\n\t\ttoken.attrs[index] = attr;\n\t}\n}\n\nfunction parentToken(tokens, index) {\n\tvar targetLevel = tokens[index].level - 1;\n\tfor (var i = index - 1; i >= 0; i--) {\n\t\tif (tokens[i].level === targetLevel) {\n\t\t\treturn i;\n\t\t}\n\t}\n\treturn -1;\n}\n\nfunction isTodoItem(tokens, index) {\n\treturn isInline(tokens[index]) &&\n\t isParagraph(tokens[index - 1]) &&\n\t isListItem(tokens[index - 2]) &&\n\t startsWithTodoMarkdown(tokens[index]);\n}\n\nfunction todoify(token, TokenConstructor) {\n\ttoken.children.unshift(makeCheckbox(token, TokenConstructor));\n\ttoken.children[1].content = token.children[1].content.slice(3);\n\ttoken.content = token.content.slice(3);\n\n\tif (useLabelWrapper) {\n\t\tif (useLabelAfter) {\n\t\t\ttoken.children.pop();\n\n\t\t\t// Use large random number as id property of the checkbox.\n\t\t\tvar id = 'task-item-' + Math.ceil(Math.random() * (10000 * 1000) - 1000);\n\t\t\ttoken.children[0].content = token.children[0].content.slice(0, -1) + ' id="' + id + '">';\n\t\t\ttoken.children.push(afterLabel(token.content, id, TokenConstructor));\n\t\t} else {\n\t\t\ttoken.children.unshift(beginLabel(TokenConstructor));\n\t\t\ttoken.children.push(endLabel(TokenConstructor));\n\t\t}\n\t}\n}\n\nfunction makeCheckbox(token, TokenConstructor) {\n\tvar checkbox = new TokenConstructor('html_inline', '', 0);\n\tvar disabledAttr = disableCheckboxes ? ' disabled="" ' : '';\n\tif (token.content.indexOf('[ ] ') === 0) {\n\t\tcheckbox.content = '<input class="task-list-item-checkbox"' + disabledAttr + 'type="checkbox">';\n\t} else if (token.content.indexOf('[x] ') === 0 || token.content.indexOf('[X] ') === 0) {\n\t\tcheckbox.content = '<input class="task-list-item-checkbox" checked=""' + disabledAttr + 'type="checkbox">';\n\t}\n\treturn checkbox;\n}\n\n// these next two functions are kind of hacky; probably should really be a\n// true block-level token with .tag=='label'\nfunction beginLabel(TokenConstructor) {\n\tvar token = new TokenConstructor('html_inline', '', 0);\n\ttoken.content = '';\n\treturn token;\n}\n\nfunction endLabel(TokenConstructor) {\n\tvar token = new TokenConstructor('html_inline', '', 0);\n\ttoken.content = '';\n\treturn token;\n}\n\nfunction afterLabel(content, id, TokenConstructor) {\n\tvar token = new TokenConstructor('html_inline', '', 0);\n\ttoken.content = '<label class="task-list-item-label" for="' + id + '">' + content + '';\n\ttoken.attrs = [{for: id}];\n\treturn token;\n}\n\nfunction isInline(token) { return token.type === 'inline'; }\nfunction isParagraph(token) { return token.type === 'paragraph_open'; }\nfunction isListItem(token) { return token.type === 'list_item_open'; }\n\nfunction startsWithTodoMarkdown(token) {\n\t// leading whitespace in a list item is already trimmed off by markdown-it\n\treturn token.content.indexOf('[ ] ') === 0 || token.content.indexOf('[x] ') === 0 || token.content.indexOf('[X] ') === 0;\n}\n", ""use strict";\n\n\nmodule.exports = function admonitionPlugin(md, options) {\n\n options = options || {};\n\n var minMarkers = 3,\n markerStr = options.marker || "!",\n markerChar = markerStr.charCodeAt(0),\n markerLen = markerStr.length,\n validate = validateDefault,\n render = renderDefault,\n type = "",\n title = null,\n types = options.types || ["note", "abstract", "info", "tip", "success", "question", "warning", "failure", "danger", "bug", "example", "quote"];\n\n function renderDefault(tokens, idx, options, env, self) {\n\n var token = tokens[idx];\n\n if (token.type === "admonition_open") {\n tokens[idx].attrPush([ "class", "admonition " + token.info ]);\n }\n\n else if (token.type === "admonition_title_open") {\n tokens[idx].attrPush([ "class", "admonition-title"]);\n }\n\n return self.renderToken(tokens, idx, options, env, self);\n }\n\n function validateDefault(params){\n var array = params.trim().split(" ", 2);\n title = "";\n type = array[0];\n if (array.length > 1) {\n title = params.substring(type.length + 2);\n }\n\n if ( title === "" || !title ) {\n title = type;\n }\n\n return types.includes(type);\n }\n\n function admonition(state, startLine, endLine, silent) {\n var pos, nextLine, markerCount, markup, params, token,\n oldParent, oldLineMax,\n autoClosed = false,\n start = state.bMarks[startLine] + state.tShift[startLine],\n max = state.eMarks[startLine];\n\n // Check out the first character quickly,\n // this should filter out most of non-containers\n //\n if (markerChar !== state.src.charCodeAt(start)) { return false; }\n\n // Check out the rest of the marker string\n //\n for (pos = start + 1; pos <= max; pos++) {\n if (markerStr[(pos - start) % markerLen] !== state.src[pos]) {\n break;\n }\n }\n\n markerCount = Math.floor((pos - start) / markerLen);\n if (markerCount < minMarkers) { return false; }\n pos -= (pos - start) % markerLen;\n\n markup = state.src.slice(start, pos);\n params = state.src.slice(pos, max);\n if (!validate(params)) { return false; }\n\n // Since start is found, we can report success here in validation mode\n //\n if (silent) { return true; }\n\n // Search for the end of the block\n //\n nextLine = startLine;\n\n for (;;) {\n nextLine++;\n if (nextLine >= endLine) {\n // unclosed block should be autoclosed by end of document.\n // also block seems to be autoclosed by end of parent\n break;\n }\n\n start = state.bMarks[nextLine] + state.tShift[nextLine];\n max = state.eMarks[nextLine];\n\n if (start < max && state.sCount[nextLine] < state.blkIndent) {\n // non-empty line with negative indent should stop the list:\n // - ```\n // test\n break;\n }\n\n if (markerChar !== state.src.charCodeAt(start)) { continue; }\n\n if (state.sCount[nextLine] - state.blkIndent >= 4) {\n // closing fence should be indented less than 4 spaces\n continue;\n }\n\n for (pos = start + 1; pos <= max; pos++) {\n if (markerStr[(pos - start) % markerLen] !== state.src[pos]) {\n break;\n }\n }\n\n // closing adminition fence must be at least as long as the opening one\n if (Math.floor((pos - start) / markerLen) < markerCount) { continue; }\n\n // make sure tail has spaces only\n pos -= (pos - start) % markerLen;\n pos = state.skipSpaces(pos);\n\n if (pos < max) { continue; }\n\n // found!\n autoClosed = true;\n break;\n }\n\n oldParent = state.parentType;\n oldLineMax = state.lineMax;\n state.parentType = "admonition";\n\n // this will prevent lazy continuations from ever going past our end marker\n state.lineMax = nextLine;\n\n token = state.push("admonition_open", "div", 1);\n token.markup = markup;\n token.block = true;\n token.info = type;\n token.map = [ startLine, nextLine ];\n\n // admonition title\n token = state.push("admonition_title_open", "p", 1);\n token.markup = markup + " " + type;\n token.map = [ startLine, nextLine ];\n\n token = state.push("inline", "", 0);\n token.content = title;\n token.map = [ startLine, state.line - 1 ];\n token.children = [];\n\n token = state.push("admonition_title_close", "p", -1);\n token.markup = markup + " " + type;\n\n state.md.block.tokenize(state, startLine + 1, nextLine);\n\n token = state.push("admonition_close", "div", -1);\n token.markup = state.src.slice(start, pos);\n token.block = true;\n\n state.parentType = oldParent;\n state.lineMax = oldLineMax;\n state.line = nextLine + (autoClosed ? 1 : 0);\n\n return true;\n }\n\n md.block.ruler.before("code", "admonition", admonition, {\n alt: ["paragraph", "reference", "blockquote", "list" ]\n });\n md.renderer.rules["admonition_open"] = render;\n md.renderer.rules["admonition_title_open"] = render;\n md.renderer.rules["admonition_title_close"] = render;\n md.renderer.rules["admonition_close"] = render;\n};", "/* eslint-disable no-multi-assign /\n\nfunction deepFreeze(obj) {\n if (obj instanceof Map) {\n obj.clear =\n obj.delete =\n obj.set =\n function () {\n throw new Error('map is read-only');\n };\n } else if (obj instanceof Set) {\n obj.add =\n obj.clear =\n obj.delete =\n function () {\n throw new Error('set is read-only');\n };\n }\n\n // Freeze self\n Object.freeze(obj);\n\n Object.getOwnPropertyNames(obj).forEach((name) => {\n const prop = obj[name];\n const type = typeof prop;\n\n // Freeze prop if it is an object or function and also not already frozen\n if ((type === 'object' || type === 'function') && !Object.isFrozen(prop)) {\n deepFreeze(prop);\n }\n });\n\n return obj;\n}\n\n/* @typedef {import('highlight.js').CallbackResponse} CallbackResponse /\n/* @typedef {import('highlight.js').CompiledMode} CompiledMode /\n/* @implements CallbackResponse */\n\nclass Response {\n /\n * @PARAM {CompiledMode} mode\n */\n constructor(mode) {\n // eslint-disable-next-line no-undefined\n if (mode.data === undefined) mode.data = {};\n\n this.data = mode.data;\n this.isMatchIgnored = false;\n }\n\n ignoreMatch() {\n this.isMatchIgnored = true;\n }\n}\n\n/\n * @PARAM {string} value\n * @returns {string}\n */\nfunction escapeHTML(value) {\n return value\n .replace(/&/g, '&')\n .replace(/</g, '<')\n .replace(/>/g, '>')\n .replace(/"/g, '"')\n .replace(/'/g, ''');\n}\n\n/\n * performs a shallow merge of multiple objects into one\n \n * @template T\n * @PARAM {T} original\n * @PARAM {Record<string,any>[]} objects\n * @returns {T} a single new object\n /\nfunction inherit$1(original, ...objects) {\n /@type Record<string,any> /\n const result = Object.create(null);\n\n for (const key in original) {\n result[key] = original[key];\n }\n objects.forEach(function(obj) {\n for (const key in obj) {\n result[key] = obj[key];\n }\n });\n return /@type {T} */ (result);\n}\n\n/\n * @typedef {object} Renderer\n * @Property {(text: string) => void} addText\n * @Property {(node: Node) => void} openNode\n * @Property {(node: Node) => void} closeNode\n * @Property {() => string} value\n /\n\n/@typedef {{scope?: string, language?: string, sublanguage?: boolean}} Node /\n/* @typedef {{walk: (r: Renderer) => void}} Tree /\n/* */\n\nconst SPAN_CLOSE = '';\n\n/\n * Determines if a node needs to be wrapped in \n *\n * @PARAM {Node} node */\nconst emitsWrappingTags = (node) => {\n // rarely we can have a sublanguage where language is undefined\n // TODO: track down why\n return !!node.scope;\n};\n\n/\n *\n * @PARAM {string} name\n * @PARAM {{prefix:string}} options\n */\nconst scopeToCSSClass = (name, { prefix }) => {\n // sub-language\n if (name.startsWith("language:")) {\n return name.replace("language:", "language-");\n }\n // tiered scope: comment.line\n if (name.includes(".")) {\n const pieces = name.split(".");\n return [\n ${prefix}${pieces.shift()},\n ...(pieces.map((x, i) => `${x}${"".repeat(i + 1)}`))\n ].join(" ");\n }\n // simple scope\n return `${prefix}${name}`;\n};\n\n/** @type {Renderer} */\nclass HTMLRenderer {\n /\n * Creates a new HTMLRenderer\n *\n * @PARAM {Tree} parseTree - the parse tree (must support `walk` API)\n * @PARAM {{classPrefix: string}} options\n */\n constructor(parseTree, options) {\n this.buffer = "";\n this.classPrefix = options.classPrefix;\n parseTree.walk(this);\n }\n\n /\n * Adds texts to the output stream\n *\n * @PARAM {string} text */\n addText(text) {\n this.buffer += escapeHTML(text);\n }\n\n /\n * Adds a node open to the output stream (if needed)\n *\n * @PARAM {Node} node */\n openNode(node) {\n if (!emitsWrappingTags(node)) return;\n\n const className = scopeToCSSClass(node.scope,\n { prefix: this.classPrefix });\n this.span(className);\n }\n\n /\n * Adds a node close to the output stream (if needed)\n *\n * @PARAM {Node} node */\n closeNode(node) {\n if (!emitsWrappingTags(node)) return;\n\n this.buffer += SPAN_CLOSE;\n }\n\n /\n * returns the accumulated buffer\n */\n value() {\n return this.buffer;\n }\n\n // helpers\n\n /\n * Builds a span element\n \n * @PARAM {string} className /\n span(className) {\n this.buffer += `<span class="${className}">`;\n }\n}\n\n/@typedef {{scope?: string, language?: string, children: Node[]} | string} Node /\n/* @typedef {{scope?: string, language?: string, children: Node[]} } DataNode /\n/* @typedef {import('highlight.js').Emitter} Emitter /\n/* /\n\n/* @returns {DataNode} /\nconst newNode = (opts = {}) => {\n /* @type DataNode /\n const result = { children: [] };\n Object.assign(result, opts);\n return result;\n};\n\nclass TokenTree {\n constructor() {\n /* @type DataNode /\n this.rootNode = newNode();\n this.stack = [this.rootNode];\n }\n\n get top() {\n return this.stack[this.stack.length - 1];\n }\n\n get root() { return this.rootNode; }\n\n /* @PARAM {Node} node /\n add(node) {\n this.top.children.push(node);\n }\n\n /* @PARAM {string} scope /\n openNode(scope) {\n /* @type Node /\n const node = newNode({ scope });\n this.add(node);\n this.stack.push(node);\n }\n\n closeNode() {\n if (this.stack.length > 1) {\n return this.stack.pop();\n }\n // eslint-disable-next-line no-undefined\n return undefined;\n }\n\n closeAllNodes() {\n while (this.closeNode());\n }\n\n toJSON() {\n return JSON.stringify(this.rootNode, null, 4);\n }\n\n /\n * @typedef { import("./html_renderer").Renderer } Renderer\n * @PARAM {Renderer} builder\n */\n walk(builder) {\n // this does not\n return this.constructor._walk(builder, this.rootNode);\n // this works\n // return TokenTree._walk(builder, this.rootNode);\n }\n\n /\n * @PARAM {Renderer} builder\n * @PARAM {Node} node\n /\n static _walk(builder, node) {\n if (typeof node === "string") {\n builder.addText(node);\n } else if (node.children) {\n builder.openNode(node);\n node.children.forEach((child) => this._walk(builder, child));\n builder.closeNode(node);\n }\n return builder;\n }\n\n /\n * @PARAM {Node} node\n */\n static _collapse(node) {\n if (typeof node === "string") return;\n if (!node.children) return;\n\n if (node.children.every(el => typeof el === "string")) {\n // node.text = node.children.join("");\n // delete node.children;\n node.children = [node.children.join("")];\n } else {\n node.children.forEach((child) => {\n TokenTree._collapse(child);\n });\n }\n }\n}\n\n/\n Currently this is all private API, but this is the minimal API necessary\n that an Emitter must implement to fully support the parser.\n\n Minimal interface:\n\n - addText(text)\n - __addSublanguage(emitter, subLanguageName)\n - startScope(scope)\n - endScope()\n - finalize()\n - toHTML()\n\n/\n\n/\n * @implements {Emitter}\n */\nclass TokenTreeEmitter extends TokenTree {\n /\n * @PARAM {} options\n */\n constructor(options) {\n super();\n this.options = options;\n }\n\n /\n * @PARAM {string} text\n /\n addText(text) {\n if (text === "") { return; }\n\n this.add(text);\n }\n\n /@PARAM {string} scope */\n startScope(scope) {\n this.openNode(scope);\n }\n\n endScope() {\n this.closeNode();\n }\n\n /\n * @PARAM {Emitter & {root: DataNode}} emitter\n * @PARAM {string} name\n /\n __addSublanguage(emitter, name) {\n /@type DataNode */\n const node = emitter.root;\n if (name) node.scope = `language:${name}`;\n\n this.add(node);\n }\n\n toHTML() {\n const renderer = new HTMLRenderer(this, this.options);\n return renderer.value();\n }\n\n finalize() {\n this.closeAllNodes();\n return true;\n }\n}\n\n/\n * @PARAM {string} value\n * @returns {RegExp}\n * */\n\n/\n * @PARAM {RegExp | string } re\n * @returns {string}\n */\nfunction source(re) {\n if (!re) return null;\n if (typeof re === "string") return re;\n\n return re.source;\n}\n\n/\n * @PARAM {RegExp | string } re\n * @returns {string}\n */\nfunction lookahead(re) {\n return concat('(?=', re, ')');\n}\n\n/\n * @PARAM {RegExp | string } re\n * @returns {string}\n /\nfunction anyNumberOfTimes(re) {\n return concat('(?:', re, ')');\n}\n\n/\n * @PARAM {RegExp | string } re\n * @returns {string}\n */\nfunction optional(re) {\n return concat('(?:', re, ')?');\n}\n\n/\n * @PARAM {...(RegExp | string) } args\n * @returns {string}\n /\nfunction concat(...args) {\n const joined = args.map((x) => source(x)).join("");\n return joined;\n}\n\n/\n * @PARAM { Array<string | RegExp | Object> } args\n * @returns {object}\n /\nfunction stripOptionsFromArgs(args) {\n const opts = args[args.length - 1];\n\n if (typeof opts === 'object' && opts.constructor === Object) {\n args.splice(args.length - 1, 1);\n return opts;\n } else {\n return {};\n }\n}\n\n/@typedef { {capture?: boolean} } RegexEitherOptions */\n\n/\n * Any of the passed expresssions may match\n \n * Creates a huge this | this | that | that match\n * @PARAM {(RegExp | string)[] | [...(RegExp | string)[], RegexEitherOptions]} args\n * @returns {string}\n /\nfunction either(...args) {\n /@type { object & {capture?: boolean} } */\n const opts = stripOptionsFromArgs(args);\n const joined = '('\n + (opts.capture ? "" : "?:")\n + args.map((x) => source(x)).join("|") + ")";\n return joined;\n}\n\n/\n * @PARAM {RegExp | string} re\n * @returns {number}\n */\nfunction countMatchGroups(re) {\n return (new RegExp(re.toString() + '|')).exec('').length - 1;\n}\n\n/*\n * Does lexeme start with a regular expression match at the beginning\n * @PARAM {RegExp} re\n * @PARAM {string} lexeme\n /\nfunction startsWith(re, lexeme) {\n const match = re && re.exec(lexeme);\n return match && match.index === 0;\n}\n\n// BACKREF_RE matches an open parenthesis or backreference. To avoid\n// an incorrect parse, it additionally matches the following:\n// - [...] elements, where the meaning of parentheses and escapes change\n// - other escape sequences, so we do not misparse escape sequences as\n// interesting elements\n// - non-matching or lookahead parentheses, which do not capture. These\n// follow the '(' with a '?'.\nconst BACKREF_RE = /\[(?:[^\\\\\\]]|\\.)\]|\(\??|\\([1-9][0-9])|\\./;\n\n// INTERNAL Not intended for outside usage\n// join logically computes regexps.join(separator), but fixes the\n// backreferences so they continue to match.\n// it also places each individual regular expression into it's own\n// match group, keeping track of the sequencing of those match groups\n// is currently an exercise for the caller. :-)\n/\n * @PARAM {(string | RegExp)[]} regexps\n * @PARAM {{joinWith: string}} opts\n * @returns {string}\n /\nfunction _rewriteBackreferences(regexps, { joinWith }) {\n let numCaptures = 0;\n\n return regexps.map((regex) => {\n numCaptures += 1;\n const offset = numCaptures;\n let re = source(regex);\n let out = '';\n\n while (re.length > 0) {\n const match = BACKREF_RE.exec(re);\n if (!match) {\n out += re;\n break;\n }\n out += re.substring(0, match.index);\n re = re.substring(match.index + match[0].length);\n if (match[0][0] === '\\' && match[1]) {\n // Adjust the backreference.\n out += '\\' + String(Number(match[1]) + offset);\n } else {\n out += match[0];\n if (match[0] === '(') {\n numCaptures++;\n }\n }\n }\n return out;\n }).map(re => `(${re})`).join(joinWith);\n}\n\n/@typedef {import('highlight.js').Mode} Mode /\n/@typedef {import('highlight.js').ModeCallback} ModeCallback /\n\n// Common regexps\nconst MATCH_NOTHING_RE = /\b\B/;\nconst IDENT_RE = '[a-zA-Z]\\w';\nconst UNDERSCORE_IDENT_RE = '[a-zA-Z]\\w*';\nconst NUMBER_RE = '\\b\\d+(\\.\\d+)?';\nconst C_NUMBER_RE = '(-?)(\\b0[xX][a-fA-F0-9]+|(\\b\\d+(\\.\\d*)?|\\.\\d+)([eE][-+]?\\d+)?)'; // 0x..., 0..., decimal, float\nconst BINARY_NUMBER_RE = '\\b(0b[01]+)'; // 0b...\nconst RE_STARTERS_RE = '!|!=|!==|%|%=|&|&&|&=|\\|\\=|\\+|\\+=|,|-|-=|/=|/|:|;|<<|<<=|<=|<|===|==|=|>>>=|>>=|>=|>>>|>>|>|\\?|\\[|\\{|\\(|\\^|\\^=|\\||\\|=|\\|\\||~';\n\n/\n* @PARAM { Partial & {binary?: string | RegExp} } opts\n*/\nconst SHEBANG = (opts = {}) => {\n const beginShebang = /^#![ ]\//;\n if (opts.binary) {\n opts.begin = concat(\n beginShebang,\n /.\b/,\n opts.binary,\n /\b.*/);\n }\n return inherit$1({\n scope: 'meta',\n begin: beginShebang,\n end: /$/,\n relevance: 0,\n /@type {ModeCallback} /\n "on:begin": (m, resp) => {\n if (m.index !== 0) resp.ignoreMatch();\n }\n }, opts);\n};\n\n// Common modes\nconst BACKSLASH_ESCAPE = {\n begin: '\\\\[\\s\\S]', relevance: 0\n};\nconst APOS_STRING_MODE = {\n scope: 'string',\n begin: '\'',\n end: '\'',\n illegal: '\\n',\n contains: [BACKSLASH_ESCAPE]\n};\nconst QUOTE_STRING_MODE = {\n scope: 'string',\n begin: '"',\n end: '"',\n illegal: '\\n',\n contains: [BACKSLASH_ESCAPE]\n};\nconst PHRASAL_WORDS_MODE = {\n begin: /\b(a|an|the|are|I'm|isn't|don't|doesn't|won't|but|just|should|pretty|simply|enough|gonna|going|wtf|so|such|will|you|your|they|like|more)\b/\n};\n/**\n * Creates a comment mode\n \n * @PARAM {string | RegExp} begin\n * @PARAM {string | RegExp} end\n * @PARAM {Mode | {}} [modeOptions]\n * @returns {Partial}\n /\nconst COMMENT = function(begin, end, modeOptions = {}) {\n const mode = inherit$1(\n {\n scope: 'comment',\n begin,\n end,\n contains: []\n },\n modeOptions\n );\n mode.contains.push({\n scope: 'doctag',\n // hack to avoid the space from being included. the space is necessary to\n // match here to prevent the plain text rule below from gobbling up doctags\n begin: '[ ](?=(TODO|FIXME|NOTE|BUG|OPTIMIZE|HACK|XXX):)',\n end: /(TODO|FIXME|NOTE|BUG|OPTIMIZE|HACK|XXX):/,\n excludeBegin: true,\n relevance: 0\n });\n const ENGLISH_WORD = either(\n // list of common 1 and 2 letter words in English\n "I",\n "a",\n "is",\n "so",\n "us",\n "to",\n "at",\n "if",\n "in",\n "it",\n "on",\n // note: this is not an exhaustive list of contractions, just popular ones\n /[A-Za-z]+'/, // contractions - can't we'd they're let's, etc\n /[A-Za-z]+[-][a-z]+/, // `no-way`, etc.\n /[A-Za-z][a-z]{2,}/ // allow capitalized words at beginning of sentences\n );\n // looking like plain text, more likely to be a comment\n mode.contains.push(\n {\n // TODO: how to include ", (, ) without breaking grammars that use these for\n // comment delimiters?\n // begin: /[ ]+([()"]?([A-Za-z'-]{3,}|is|a|I|so|us|[tT][oO]|at|if|in|it|on)[.]?[()":]?([.][ ]|[ ]|\))){3}/\n // ---\n\n // this tries to find sequences of 3 english words in a row (without any\n // "programming" type syntax) this gives us a strong signal that we've\n // TRULY found a comment - vs perhaps scanning with the wrong language.\n // It's possible to find something that LOOKS like the start of the\n // comment - but then if there is no readable text - good chance it is a\n // false match and not a comment.\n //\n // for a visual example please see:\n // https://github.com/highlightjs/highlight.js/issues/2827\n\n begin: concat(\n /[ ]+/, // necessary to prevent us gobbling up doctags like /@author Bob Mcgill /\n '(',\n ENGLISH_WORD,\n /[.]?[:]?([.][ ]|[ ])/,\n '){3}') // look for 3 words in a row\n }\n );\n return mode;\n};\nconst C_LINE_COMMENT_MODE = COMMENT('//', '$');\nconst C_BLOCK_COMMENT_MODE = COMMENT('/\\', '\\/');\nconst HASH_COMMENT_MODE = COMMENT('#', '$');\nconst NUMBER_MODE = {\n scope: 'number',\n begin: NUMBER_RE,\n relevance: 0\n};\nconst C_NUMBER_MODE = {\n scope: 'number',\n begin: C_NUMBER_RE,\n relevance: 0\n};\nconst BINARY_NUMBER_MODE = {\n scope: 'number',\n begin: BINARY_NUMBER_RE,\n relevance: 0\n};\nconst REGEXP_MODE = {\n scope: "regexp",\n begin: /\/(?=[^/\\n]\/)/,\n end: /\/[gimuy]/,\n contains: [\n BACKSLASH_ESCAPE,\n {\n begin: /\[/,\n end: /\]/,\n relevance: 0,\n contains: [BACKSLASH_ESCAPE]\n }\n ]\n};\nconst TITLE_MODE = {\n scope: 'title',\n begin: IDENT_RE,\n relevance: 0\n};\nconst UNDERSCORE_TITLE_MODE = {\n scope: 'title',\n begin: UNDERSCORE_IDENT_RE,\n relevance: 0\n};\nconst METHOD_GUARD = {\n // excludes method names from keyword processing\n begin: '\\.\\s*' + UNDERSCORE_IDENT_RE,\n relevance: 0\n};\n\n/\n * Adds end same as begin mechanics to a mode\n \n * Your mode must include at least a single () match group as that first match\n * group is what is used for comparison\n * @PARAM {Partial} mode\n /\nconst END_SAME_AS_BEGIN = function(mode) {\n return Object.assign(mode,\n {\n /@type {ModeCallback} /\n 'on:begin': (m, resp) => { resp.data._beginMatch = m[1]; },\n /@type {ModeCallback} /\n 'on:end': (m, resp) => { if (resp.data._beginMatch !== m[1]) resp.ignoreMatch(); }\n });\n};\n\nvar MODES = /#PURE*/Object.freeze({\n proto: null,\n APOS_STRING_MODE: APOS_STRING_MODE,\n BACKSLASH_ESCAPE: BACKSLASH_ESCAPE,\n BINARY_NUMBER_MODE: BINARY_NUMBER_MODE,\n BINARY_NUMBER_RE: BINARY_NUMBER_RE,\n COMMENT: COMMENT,\n C_BLOCK_COMMENT_MODE: C_BLOCK_COMMENT_MODE,\n C_LINE_COMMENT_MODE: C_LINE_COMMENT_MODE,\n C_NUMBER_MODE: C_NUMBER_MODE,\n C_NUMBER_RE: C_NUMBER_RE,\n END_SAME_AS_BEGIN: END_SAME_AS_BEGIN,\n HASH_COMMENT_MODE: HASH_COMMENT_MODE,\n IDENT_RE: IDENT_RE,\n MATCH_NOTHING_RE: MATCH_NOTHING_RE,\n METHOD_GUARD: METHOD_GUARD,\n NUMBER_MODE: NUMBER_MODE,\n NUMBER_RE: NUMBER_RE,\n PHRASAL_WORDS_MODE: PHRASAL_WORDS_MODE,\n QUOTE_STRING_MODE: QUOTE_STRING_MODE,\n REGEXP_MODE: REGEXP_MODE,\n RE_STARTERS_RE: RE_STARTERS_RE,\n SHEBANG: SHEBANG,\n TITLE_MODE: TITLE_MODE,\n UNDERSCORE_IDENT_RE: UNDERSCORE_IDENT_RE,\n UNDERSCORE_TITLE_MODE: UNDERSCORE_TITLE_MODE\n});\n\n/\n@typedef {import('highlight.js').CallbackResponse} CallbackResponse\n@typedef {import('highlight.js').CompilerExt} CompilerExt\n*/\n\n// Grammar extensions / plugins\n// See: https://github.com/highlightjs/highlight.js/issues/2833\n\n// Grammar extensions allow "syntactic sugar" to be added to the grammar modes\n// without requiring any underlying changes to the compiler internals.\n\n// `compileMatch` being the perfect small example of now allowing a grammar\n// author to write `match` when they desire to match a single expression rather\n// than being forced to use `begin`. The extension then just moves `match` into\n// `begin` when it runs. Ie, no features have been added, but we've just made\n// the experience of writing (and reading grammars) a little bit nicer.\n\n// ------\n\n// TODO: We need negative look-behind support to do this properly\n/\n * Skip a match if it has a preceding dot\n *\n * This is used for `beginKeywords` to prevent matching expressions such as\n * `bob.keyword.do()`. The mode compiler automatically wires this up as a\n * special internal 'on:begin' callback for modes with `beginKeywords`\n * @PARAM {RegExpMatchArray} match\n * @PARAM {CallbackResponse} response\n */\nfunction skipIfHasPrecedingDot(match, response) {\n const before = match.input[match.index - 1];\n if (before === ".") {\n response.ignoreMatch();\n }\n}\n\n/\n *\n * @type {CompilerExt}\n */\nfunction scopeClassName(mode, _parent) {\n // eslint-disable-next-line no-undefined\n if (mode.className !== undefined) {\n mode.scope = mode.className;\n delete mode.className;\n }\n}\n\n/\n * `beginKeywords` syntactic sugar\n * @type {CompilerExt}\n */\nfunction beginKeywords(mode, parent) {\n if (!parent) return;\n if (!mode.beginKeywords) return;\n\n // for languages with keywords that include non-word characters checking for\n // a word boundary is not sufficient, so instead we check for a word boundary\n // or whitespace - this does no harm in any case since our keyword engine\n // doesn't allow spaces in keywords anyways and we still check for the boundary\n // first\n mode.begin = '\\b(' + mode.beginKeywords.split(' ').join('|') + ')(?!\\.)(?=\\b|\\s)';\n mode.__beforeBegin = skipIfHasPrecedingDot;\n mode.keywords = mode.keywords || mode.beginKeywords;\n delete mode.beginKeywords;\n\n // prevents double relevance, the keywords themselves provide\n // relevance, the mode doesn't need to double it\n // eslint-disable-next-line no-undefined\n if (mode.relevance === undefined) mode.relevance = 0;\n}\n\n/\n * Allow `illegal` to contain an array of illegal values\n * @type {CompilerExt}\n */\nfunction compileIllegal(mode, _parent) {\n if (!Array.isArray(mode.illegal)) return;\n\n mode.illegal = either(...mode.illegal);\n}\n\n/\n * `match` to match a single expression for readability\n * @type {CompilerExt}\n */\nfunction compileMatch(mode, _parent) {\n if (!mode.match) return;\n if (mode.begin || mode.end) throw new Error("begin & end are not supported with match");\n\n mode.begin = mode.match;\n delete mode.match;\n}\n\n/\n * provides the default 1 relevance to all modes\n * @type {CompilerExt}\n */\nfunction compileRelevance(mode, _parent) {\n // eslint-disable-next-line no-undefined\n if (mode.relevance === undefined) mode.relevance = 1;\n}\n\n// allow beforeMatch to act as a "qualifier" for the match\n// the full match begin must be [beforeMatch][begin]\nconst beforeMatchExt = (mode, parent) => {\n if (!mode.beforeMatch) return;\n // starts conflicts with endsParent which we need to make sure the child\n // rule is not matched multiple times\n if (mode.starts) throw new Error("beforeMatch cannot be used with starts");\n\n const originalMode = Object.assign({}, mode);\n Object.keys(mode).forEach((key) => { delete mode[key]; });\n\n mode.keywords = originalMode.keywords;\n mode.begin = concat(originalMode.beforeMatch, lookahead(originalMode.begin));\n mode.starts = {\n relevance: 0,\n contains: [\n Object.assign(originalMode, { endsParent: true })\n ]\n };\n mode.relevance = 0;\n\n delete originalMode.beforeMatch;\n};\n\n// keywords that should have no default relevance value\nconst COMMON_KEYWORDS = [\n 'of',\n 'and',\n 'for',\n 'in',\n 'not',\n 'or',\n 'if',\n 'then',\n 'parent', // common variable name\n 'list', // common variable name\n 'value' // common variable name\n];\n\nconst DEFAULT_KEYWORD_SCOPE = "keyword";\n\n/\n * Given raw keywords from a language definition, compile them.\n \n * @PARAM {string | Record<string,string|string[]> | Array} rawKeywords\n * @PARAM {boolean} caseInsensitive\n /\nfunction compileKeywords(rawKeywords, caseInsensitive, scopeName = DEFAULT_KEYWORD_SCOPE) {\n /@type {import("highlight.js/private").KeywordDict} */\n const compiledKeywords = Object.create(null);\n\n // input can be a string of keywords, an array of keywords, or a object with\n // named keys representing scopeName (which can then point to a string or array)\n if (typeof rawKeywords === 'string') {\n compileList(scopeName, rawKeywords.split(" "));\n } else if (Array.isArray(rawKeywords)) {\n compileList(scopeName, rawKeywords);\n } else {\n Object.keys(rawKeywords).forEach(function(scopeName) {\n // collapse all our objects back into the parent object\n Object.assign(\n compiledKeywords,\n compileKeywords(rawKeywords[scopeName], caseInsensitive, scopeName)\n );\n });\n }\n return compiledKeywords;\n\n // ---\n\n /\n * Compiles an individual list of keywords\n *\n * Ex: "for if when while|5"\n *\n * @PARAM {string} scopeName\n * @PARAM {Array} keywordList\n */\n function compileList(scopeName, keywordList) {\n if (caseInsensitive) {\n keywordList = keywordList.map(x => x.toLowerCase());\n }\n keywordList.forEach(function(keyword) {\n const pair = keyword.split('|');\n compiledKeywords[pair[0]] = [scopeName, scoreForKeyword(pair[0], pair[1])];\n });\n }\n}\n\n/\n * Returns the proper score for a given keyword\n *\n * Also takes into account comment keywords, which will be scored 0 UNLESS\n * another score has been manually assigned.\n * @PARAM {string} keyword\n * @PARAM {string} [providedScore]\n */\nfunction scoreForKeyword(keyword, providedScore) {\n // manual scores always win over common keywords\n // so you can force a score of 1 if you really insist\n if (providedScore) {\n return Number(providedScore);\n }\n\n return commonKeyword(keyword) ? 0 : 1;\n}\n\n/\n * Determines if a given keyword is common or not\n \n * @PARAM {string} keyword /\nfunction commonKeyword(keyword) {\n return COMMON_KEYWORDS.includes(keyword.toLowerCase());\n}\n\n/\n\nFor the reasoning behind this please see:\nhttps://github.com/highlightjs/highlight.js/issues/2880#issuecomment-747275419\n\n/\n\n/\n * @type {Record<string, boolean>}\n */\nconst seenDeprecations = {};\n\n/\n * @PARAM {string} message\n */\nconst error = (message) => {\n console.error(message);\n};\n\n/\n * @PARAM {string} message\n * @PARAM {any} args\n */\nconst warn = (message, ...args) => {\n console.log(`WARN: ${message}`, ...args);\n};\n\n/\n * @PARAM {string} version\n * @PARAM {string} message\n /\nconst deprecated = (version, message) => {\n if (seenDeprecations[`${version}/${message}`]) return;\n\n console.log(`Deprecated as of ${version}. ${message}`);\n seenDeprecations[`${version}/${message}`] = true;\n};\n\n/ eslint-disable no-throw-literal */\n\n/\n@typedef {import('highlight.js').CompiledMode} CompiledMode\n*/\n\nconst MultiClassError = new Error();\n\n/\n * Renumbers labeled scope names to account for additional inner match\n * groups that otherwise would break everything.\n *\n * Lets say we 3 match scopes:\n *\n * { 1 => ..., 2 => ..., 3 => ... }\n *\n * So what we need is a clean match like this:\n *\n * (a)(b)(c) => [ "a", "b", "c" ]\n *\n * But this falls apart with inner match groups:\n *\n * (a)(((b)))(c) => ["a", "b", "b", "b", "c" ]\n *\n * Our scopes are now "out of alignment" and we're repeating `b` 3 times.\n * What needs to happen is the numbers are remapped:\n *\n * { 1 => ..., 2 => ..., 5 => ... }\n \n * We also need to know that the ONLY groups that should be output\n * are 1, 2, and 5. This function handles this behavior.\n \n * @PARAM {CompiledMode} mode\n * @PARAM {Array<RegExp | string>} regexes\n * @PARAM {{key: "beginScope"|"endScope"}} opts\n /\nfunction remapScopeNames(mode, regexes, { key }) {\n let offset = 0;\n const scopeNames = mode[key];\n /@type Record<number,boolean> /\n const emit = {};\n /@type Record<number,string> */\n const positions = {};\n\n for (let i = 1; i <= regexes.length; i++) {\n positions[i + offset] = scopeNames[i];\n emit[i + offset] = true;\n offset += countMatchGroups(regexes[i - 1]);\n }\n // we use _emit to keep track of which match groups are "top-level" to avoid double\n // output from inside match groups\n mode[key] = positions;\n mode[key]._emit = emit;\n mode[key]._multi = true;\n}\n\n/\n * @PARAM {CompiledMode} mode\n */\nfunction beginMultiClass(mode) {\n if (!Array.isArray(mode.begin)) return;\n\n if (mode.skip || mode.excludeBegin || mode.returnBegin) {\n error("skip, excludeBegin, returnBegin not compatible with beginScope: {}");\n throw MultiClassError;\n }\n\n if (typeof mode.beginScope !== "object" || mode.beginScope === null) {\n error("beginScope must be object");\n throw MultiClassError;\n }\n\n remapScopeNames(mode, mode.begin, { key: "beginScope" });\n mode.begin = _rewriteBackreferences(mode.begin, { joinWith: "" });\n}\n\n/\n * @PARAM {CompiledMode} mode\n */\nfunction endMultiClass(mode) {\n if (!Array.isArray(mode.end)) return;\n\n if (mode.skip || mode.excludeEnd || mode.returnEnd) {\n error("skip, excludeEnd, returnEnd not compatible with endScope: {}");\n throw MultiClassError;\n }\n\n if (typeof mode.endScope !== "object" || mode.endScope === null) {\n error("endScope must be object");\n throw MultiClassError;\n }\n\n remapScopeNames(mode, mode.end, { key: "endScope" });\n mode.end = _rewriteBackreferences(mode.end, { joinWith: "" });\n}\n\n/\n * this exists only to allow `scope: {}` to be used beside `match:`\n * Otherwise `beginScope` would necessary and that would look weird\n\n {\n match: [ /def/, /\w+/ ]\n scope: { 1: "keyword" , 2: "title" }\n }\n\n * @PARAM {CompiledMode} mode\n */\nfunction scopeSugar(mode) {\n if (mode.scope && typeof mode.scope === "object" && mode.scope !== null) {\n mode.beginScope = mode.scope;\n delete mode.scope;\n }\n}\n\n/\n * @PARAM {CompiledMode} mode\n */\nfunction MultiClass(mode) {\n scopeSugar(mode);\n\n if (typeof mode.beginScope === "string") {\n mode.beginScope = { _wrap: mode.beginScope };\n }\n if (typeof mode.endScope === "string") {\n mode.endScope = { _wrap: mode.endScope };\n }\n\n beginMultiClass(mode);\n endMultiClass(mode);\n}\n\n/\n@typedef {import('highlight.js').Mode} Mode\n@typedef {import('highlight.js').CompiledMode} CompiledMode\n@typedef {import('highlight.js').Language} Language\n@typedef {import('highlight.js').HLJSPlugin} HLJSPlugin\n@typedef {import('highlight.js').CompiledLanguage} CompiledLanguage\n*/\n\n// compilation\n\n/\n * Compiles a language definition result\n *\n * Given the raw result of a language definition (Language), compiles this so\n * that it is ready for highlighting code.\n * @PARAM {Language} language\n * @returns {CompiledLanguage}\n */\nfunction compileLanguage(language) {\n /\n * Builds a regex with the case sensitivity of the current language\n *\n * @PARAM {RegExp | string} value\n * @PARAM {boolean} [global]\n */\n function langRe(value, global) {\n return new RegExp(\n source(value),\n 'm'\n + (language.case_insensitive ? 'i' : '')\n + (language.unicodeRegex ? 'u' : '')\n + (global ? 'g' : '')\n );\n }\n\n /\n Stores multiple regular expressions and allows you to quickly search for\n them all in a string simultaneously - returning the first match. It does\n this by creating a huge (a|b|c) regex - each individual item wrapped with ()\n and joined by `|` - using match groups to track position. When a match is\n found checking which position in the array has content allows us to figure\n out which of the original regexes / match groups triggered the match.\n\n The match object itself (the result of `Regex.exec`) is returned but also\n enhanced by merging in any meta-data that was registered with the regex.\n This is how we keep track of which mode matched, and what type of rule\n (`illegal`, `begin`, end, etc).\n /\n class MultiRegex {\n constructor() {\n this.matchIndexes = {};\n // @ts-ignore\n this.regexes = [];\n this.matchAt = 1;\n this.position = 0;\n }\n\n // @ts-ignore\n addRule(re, opts) {\n opts.position = this.position++;\n // @ts-ignore\n this.matchIndexes[this.matchAt] = opts;\n this.regexes.push([opts, re]);\n this.matchAt += countMatchGroups(re) + 1;\n }\n\n compile() {\n if (this.regexes.length === 0) {\n // avoids the need to check length every time exec is called\n // @ts-ignore\n this.exec = () => null;\n }\n const terminators = this.regexes.map(el => el[1]);\n this.matcherRe = langRe(_rewriteBackreferences(terminators, { joinWith: '|' }), true);\n this.lastIndex = 0;\n }\n\n /@PARAM {string} s /\n exec(s) {\n this.matcherRe.lastIndex = this.lastIndex;\n const match = this.matcherRe.exec(s);\n if (!match) { return null; }\n\n // eslint-disable-next-line no-undefined\n const i = match.findIndex((el, i) => i > 0 && el !== undefined);\n // @ts-ignore\n const matchData = this.matchIndexes[i];\n // trim off any earlier non-relevant match groups (ie, the other regex\n // match groups that make up the multi-matcher)\n match.splice(0, i);\n\n return Object.assign(match, matchData);\n }\n }\n\n /\n Created to solve the key deficiently with MultiRegex - there is no way to\n test for multiple matches at a single location. Why would we need to do\n that? In the future a more dynamic engine will allow certain matches to be\n ignored. An example: if we matched say the 3rd regex in a large group but\n decided to ignore it - we'd need to started testing again at the 4th\n regex... but MultiRegex itself gives us no real way to do that.\n\n So what this class creates MultiRegexs on the fly for whatever search\n position they are needed.\n\n NOTE: These additional MultiRegex objects are created dynamically. For most\n grammars most of the time we will never actually need anything more than the\n first MultiRegex - so this shouldn't have too much overhead.\n\n Say this is our search group, and we match regex3, but wish to ignore it.\n\n regex1 | regex2 | regex3 | regex4 | regex5 ' ie, startAt = 0\n\n What we need is a new MultiRegex that only includes the remaining\n possibilities:\n\n regex4 | regex5 ' ie, startAt = 3\n\n This class wraps all that complexity up in a simple API... `startAt` decides\n where in the array of expressions to start doing the matching. It\n auto-increments, so if a match is found at position 2, then startAt will be\n set to 3. If the end is reached startAt will return to 0.\n\n MOST of the time the parser will be setting startAt manually to 0.\n /\n class ResumableMultiRegex {\n constructor() {\n // @ts-ignore\n this.rules = [];\n // @ts-ignore\n this.multiRegexes = [];\n this.count = 0;\n\n this.lastIndex = 0;\n this.regexIndex = 0;\n }\n\n // @ts-ignore\n getMatcher(index) {\n if (this.multiRegexes[index]) return this.multiRegexes[index];\n\n const matcher = new MultiRegex();\n this.rules.slice(index).forEach(([re, opts]) => matcher.addRule(re, opts));\n matcher.compile();\n this.multiRegexes[index] = matcher;\n return matcher;\n }\n\n resumingScanAtSamePosition() {\n return this.regexIndex !== 0;\n }\n\n considerAll() {\n this.regexIndex = 0;\n }\n\n // @ts-ignore\n addRule(re, opts) {\n this.rules.push([re, opts]);\n if (opts.type === "begin") this.count++;\n }\n\n /* @PARAM {string} s */\n exec(s) {\n const m = this.getMatcher(this.regexIndex);\n m.lastIndex = this.lastIndex;\n let result = m.exec(s);\n\n // The following is because we have no easy way to say "resume scanning at the\n // existing position but also skip the current rule ONLY". What happens is\n // all prior rules are also skipped which can result in matching the wrong\n // thing. Example of matching "booger":\n\n // our matcher is [string, "booger", number]\n //\n // ....booger....\n\n // if "booger" is ignored then we'd really need a regex to scan from the\n // SAME position for only: [string, number] but ignoring "booger" (if it\n // was the first match), a simple resume would scan ahead who knows how\n // far looking only for "number", ignoring potential string matches (or\n // future "booger" matches that might be valid.)\n\n // So what we do: We execute two matchers, one resuming at the same\n // position, but the second full matcher starting at the position after:\n\n // /--- resume first regex match here (for [number])\n // |/---- full match here for [string, "booger", number]\n // vv\n // ....booger....\n\n // Which ever results in a match first is then used. So this 3-4 step\n // process essentially allows us to say "match at this position, excluding\n // a prior rule that was ignored".\n //\n // 1. Match "booger" first, ignore. Also proves that [string] does non match.\n // 2. Resume matching for [number]\n // 3. Match at index + 1 for [string, "booger", number]\n // 4. If #2 and #3 result in matches, which came first?\n if (this.resumingScanAtSamePosition()) {\n if (result && result.index === this.lastIndex) ; else { // use the second matcher result\n const m2 = this.getMatcher(0);\n m2.lastIndex = this.lastIndex + 1;\n result = m2.exec(s);\n }\n }\n\n if (result) {\n this.regexIndex += result.position + 1;\n if (this.regexIndex === this.count) {\n // wrap-around to considering all matches again\n this.considerAll();\n }\n }\n\n return result;\n }\n }\n\n /\n * Given a mode, builds a huge ResumableMultiRegex that can be used to walk\n * the content and find matches.\n \n * @PARAM {CompiledMode} mode\n * @returns {ResumableMultiRegex}\n /\n function buildModeRegex(mode) {\n const mm = new ResumableMultiRegex();\n\n mode.contains.forEach(term => mm.addRule(term.begin, { rule: term, type: "begin" }));\n\n if (mode.terminatorEnd) {\n mm.addRule(mode.terminatorEnd, { type: "end" });\n }\n if (mode.illegal) {\n mm.addRule(mode.illegal, { type: "illegal" });\n }\n\n return mm;\n }\n\n / skip vs abort vs ignore\n *\n * @Skip - The mode is still entered and exited normally (and contains rules apply),\n * but all content is held and added to the parent buffer rather than being\n * output when the mode ends. Mostly used with `sublanguage` to build up\n * a single large buffer than can be parsed by sublanguage.\n *\n * - The mode begin ands ends normally.\n * - Content matched is added to the parent mode buffer.\n * - The parser cursor is moved forward normally.\n *\n * @abort - A hack placeholder until we have ignore. Aborts the mode (as if it\n * never matched) but DOES NOT continue to match subsequent `contains`\n * modes. Abort is bad/suboptimal because it can result in modes\n * farther down not getting applied because an earlier rule eats the\n * content but then aborts.\n *\n * - The mode does not begin.\n * - Content matched by `begin` is added to the mode buffer.\n * - The parser cursor is moved forward accordingly.\n *\n * @ignore - Ignores the mode (as if it never matched) and continues to match any\n * subsequent `contains` modes. Ignore isn't technically possible with\n * the current parser implementation.\n *\n * - The mode does not begin.\n * - Content matched by `begin` is ignored.\n * - The parser cursor is not moved forward.\n */\n\n /\n * Compiles an individual mode\n \n * This can raise an error if the mode contains certain detectable known logic\n * issues.\n * @PARAM {Mode} mode\n * @PARAM {CompiledMode | null} [parent]\n * @returns {CompiledMode | never}\n /\n function compileMode(mode, parent) {\n const cmode = /@type CompiledMode / (mode);\n if (mode.isCompiled) return cmode;\n\n [\n scopeClassName,\n // do this early so compiler extensions generally don't have to worry about\n // the distinction between match/begin\n compileMatch,\n MultiClass,\n beforeMatchExt\n ].forEach(ext => ext(mode, parent));\n\n language.compilerExtensions.forEach(ext => ext(mode, parent));\n\n // __beforeBegin is considered private API, internal use only\n mode.__beforeBegin = null;\n\n [\n beginKeywords,\n // do this later so compiler extensions that come earlier have access to the\n // raw array if they wanted to perhaps manipulate it, etc.\n compileIllegal,\n // default to 1 relevance if not specified\n compileRelevance\n ].forEach(ext => ext(mode, parent));\n\n mode.isCompiled = true;\n\n let keywordPattern = null;\n if (typeof mode.keywords === "object" && mode.keywords.$pattern) {\n // we need a copy because keywords might be compiled multiple times\n // so we can't go deleting $pattern from the original on the first\n // pass\n mode.keywords = Object.assign({}, mode.keywords);\n keywordPattern = mode.keywords.$pattern;\n delete mode.keywords.$pattern;\n }\n keywordPattern = keywordPattern || /\w+/;\n\n if (mode.keywords) {\n mode.keywords = compileKeywords(mode.keywords, language.case_insensitive);\n }\n\n cmode.keywordPatternRe = langRe(keywordPattern, true);\n\n if (parent) {\n if (!mode.begin) mode.begin = /\B|\b/;\n cmode.beginRe = langRe(cmode.begin);\n if (!mode.end && !mode.endsWithParent) mode.end = /\B|\b/;\n if (mode.end) cmode.endRe = langRe(cmode.end);\n cmode.terminatorEnd = source(cmode.end) || '';\n if (mode.endsWithParent && parent.terminatorEnd) {\n cmode.terminatorEnd += (mode.end ? '|' : '') + parent.terminatorEnd;\n }\n }\n if (mode.illegal) cmode.illegalRe = langRe(/* @type {RegExp | string} / (mode.illegal));\n if (!mode.contains) mode.contains = [];\n\n mode.contains = [].concat(...mode.contains.map(function(c) {\n return expandOrCloneMode(c === 'self' ? mode : c);\n }));\n mode.contains.forEach(function(c) { compileMode(/* @type Mode / (c), cmode); });\n\n if (mode.starts) {\n compileMode(mode.starts, parent);\n }\n\n cmode.matcher = buildModeRegex(cmode);\n return cmode;\n }\n\n if (!language.compilerExtensions) language.compilerExtensions = [];\n\n // self is not valid at the top-level\n if (language.contains && language.contains.includes('self')) {\n throw new Error("ERR: contains `self` is not supported at the top-level of a language. See documentation.");\n }\n\n // we need a null object, which inherit will guarantee\n language.classNameAliases = inherit$1(language.classNameAliases || {});\n\n return compileMode(/* @type Mode */ (language));\n}\n\n/\n * Determines if a mode has a dependency on it's parent or not\n *\n * If a mode does have a parent dependency then often we need to clone it if\n * it's used in multiple places so that each copy points to the correct parent,\n * where-as modes without a parent can often safely be re-used at the bottom of\n * a mode chain.\n *\n * @PARAM {Mode | null} mode\n * @returns {boolean} - is there a dependency on the parent?\n * */\nfunction dependencyOnParent(mode) {\n if (!mode) return false;\n\n return mode.endsWithParent || dependencyOnParent(mode.starts);\n}\n\n/\n * Expands a mode or clones it if necessary\n \n * This is necessary for modes with parental dependenceis (see notes on\n * `dependencyOnParent`) and for nodes that have `variants` - which must then be\n * exploded into their own individual modes at compile time.\n \n * @PARAM {Mode} mode\n * @returns {Mode | Mode[]}\n * /\nfunction expandOrCloneMode(mode) {\n if (mode.variants && !mode.cachedVariants) {\n mode.cachedVariants = mode.variants.map(function(variant) {\n return inherit$1(mode, { variants: null }, variant);\n });\n }\n\n // EXPAND\n // if we have variants then essentially "replace" the mode with the variants\n // this happens in compileMode, where this function is called from\n if (mode.cachedVariants) {\n return mode.cachedVariants;\n }\n\n // CLONE\n // if we have dependencies on parents then we need a unique\n // instance of ourselves, so we can be reused with many\n // different parents without issue\n if (dependencyOnParent(mode)) {\n return inherit$1(mode, { starts: mode.starts ? inherit$1(mode.starts) : null });\n }\n\n if (Object.isFrozen(mode)) {\n return inherit$1(mode);\n }\n\n // no special dependency issues, just return ourselves\n return mode;\n}\n\nvar version = "11.11.1";\n\nclass HTMLInjectionError extends Error {\n constructor(reason, html) {\n super(reason);\n this.name = "HTMLInjectionError";\n this.html = html;\n }\n}\n\n/\nSyntax highlighting with language autodetection.\nhttps://highlightjs.org/\n/\n\n\n\n/**\n@typedef {import('highlight.js').Mode} Mode\n@typedef {import('highlight.js').CompiledMode} CompiledMode\n@typedef {import('highlight.js').CompiledScope} CompiledScope\n@typedef {import('highlight.js').Language} Language\n@typedef {import('highlight.js').HLJSApi} HLJSApi\n@typedef {import('highlight.js').HLJSPlugin} HLJSPlugin\n@typedef {import('highlight.js').PluginEvent} PluginEvent\n@typedef {import('highlight.js').HLJSOptions} HLJSOptions\n@typedef {import('highlight.js').LanguageFn} LanguageFn\n@typedef {import('highlight.js').HighlightedHTMLElement} HighlightedHTMLElement\n@typedef {import('highlight.js').BeforeHighlightContext} BeforeHighlightContext\n@typedef {import('highlight.js/private').MatchType} MatchType\n@typedef {import('highlight.js/private').KeywordData} KeywordData\n@typedef {import('highlight.js/private').EnhancedMatch} EnhancedMatch\n@typedef {import('highlight.js/private').AnnotatedError} AnnotatedError\n@typedef {import('highlight.js').AutoHighlightResult} AutoHighlightResult\n@typedef {import('highlight.js').HighlightOptions} HighlightOptions\n@typedef {import('highlight.js').HighlightResult} HighlightResult\n/\n\n\nconst escape = escapeHTML;\nconst inherit = inherit$1;\nconst NO_MATCH = Symbol("nomatch");\nconst MAX_KEYWORD_HITS = 7;\n\n/\n * @PARAM {any} hljs - object that is extended (legacy)\n * @returns {HLJSApi}\n /\nconst HLJS = function(hljs) {\n // Global internal variables used within the highlight.js library.\n /@type {Record<string, Language>} /\n const languages = Object.create(null);\n /@type {Record<string, string>} /\n const aliases = Object.create(null);\n /* @type {HLJSPlugin[]} /\n const plugins = [];\n\n // safe/production mode - swallows more errors, tries to keep running\n // even if a single syntax or parse hits a fatal error\n let SAFE_MODE = true;\n const LANGUAGE_NOT_FOUND = "Could not find the language '{}', did you forget to load/include a language module?";\n /* @type {Language} /\n const PLAINTEXT_LANGUAGE = { disableAutodetect: true, name: 'Plain text', contains: [] };\n\n // Global options used when within external APIs. This is modified when\n // calling the `hljs.configure` function.\n /* @type HLJSOptions /\n let options = {\n ignoreUnescapedHTML: false,\n throwUnescapedHTML: false,\n noHighlightRe: /^(no-?highlight)$/i,\n languageDetectRe: /\blang(?:uage)?-([\w-]+)\b/i,\n classPrefix: 'hljs-',\n cssSelector: 'pre code',\n languages: null,\n // beta configuration options, subject to change, welcome to discuss\n // https://github.com/highlightjs/highlight.js/issues/1086\n __emitter: TokenTreeEmitter\n };\n\n / Utility functions */\n\n /\n * Tests a language name to see if highlighting should be skipped\n * @PARAM {string} languageName\n */\n function shouldNotHighlight(languageName) {\n return options.noHighlightRe.test(languageName);\n }\n\n /\n * @PARAM {HighlightedHTMLElement} block - the HTML element to determine language for\n /\n function blockLanguage(block) {\n let classes = block.className + ' ';\n\n classes += block.parentNode ? block.parentNode.className : '';\n\n // language- takes precedence over non-prefixed class names.\n const match = options.languageDetectRe.exec(classes);\n if (match) {\n const language = getLanguage(match[1]);\n if (!language) {\n warn(LANGUAGE_NOT_FOUND.replace("{}", match[1]));\n warn("Falling back to no-highlight mode for this block.", block);\n }\n return language ? match[1] : 'no-highlight';\n }\n\n return classes\n .split(/\s+/)\n .find((_class) => shouldNotHighlight(_class) || getLanguage(_class));\n }\n\n /\n * Core highlighting function.\n *\n * OLD API\n * highlight(lang, code, ignoreIllegals, continuation)\n *\n * NEW API\n * highlight(code, {lang, ignoreIllegals})\n *\n * @PARAM {string} codeOrLanguageName - the language to use for highlighting\n * @PARAM {string | HighlightOptions} optionsOrCode - the code to highlight\n * @PARAM {boolean} [ignoreIllegals] - whether to ignore illegal matches, default is to bail\n \n * @returns {HighlightResult} Result - an object that represents the result\n * @Property {string} language - the language name\n * @Property {number} relevance - the relevance score\n * @Property {string} value - the highlighted HTML code\n * @Property {string} code - the original raw code\n * @Property {CompiledMode} top - top of the current mode stack\n * @Property {boolean} illegal - indicates whether any illegal matches were found\n /\n function highlight(codeOrLanguageName, optionsOrCode, ignoreIllegals) {\n let code = "";\n let languageName = "";\n if (typeof optionsOrCode === "object") {\n code = codeOrLanguageName;\n ignoreIllegals = optionsOrCode.ignoreIllegals;\n languageName = optionsOrCode.language;\n } else {\n // old API\n deprecated("10.7.0", "highlight(lang, code, ...args) has been deprecated.");\n deprecated("10.7.0", "Please use highlight(code, options) instead.\nhttps://github.com/highlightjs/highlight.js/issues/2277");\n languageName = codeOrLanguageName;\n code = optionsOrCode;\n }\n\n // https://github.com/highlightjs/highlight.js/issues/3149\n // eslint-disable-next-line no-undefined\n if (ignoreIllegals === undefined) { ignoreIllegals = true; }\n\n /@type {BeforeHighlightContext} */\n const context = {\n code,\n language: languageName\n };\n // the plugin can change the desired language or the code to be highlighted\n // just be changing the object it was passed\n fire("before:highlight", context);\n\n // a before plugin can usurp the result completely by providing it's own\n // in which case we don't even need to call highlight\n const result = context.result\n ? context.result\n : _highlight(context.language, context.code, ignoreIllegals);\n\n result.code = context.code;\n // the plugin can change anything in result to suite it\n fire("after:highlight", result);\n\n return result;\n }\n\n /\n * private highlight that's used internally and does not fire callbacks\n *\n * @PARAM {string} languageName - the language to use for highlighting\n * @PARAM {string} codeToHighlight - the code to highlight\n * @PARAM {boolean?} [ignoreIllegals] - whether to ignore illegal matches, default is to bail\n * @PARAM {CompiledMode?} [continuation] - current continuation mode, if any\n * @returns {HighlightResult} - result of the highlight operation\n */\n function highlight(languageName, codeToHighlight, ignoreIllegals, continuation) {\n const keywordHits = Object.create(null);\n\n /**\n * Return keyword data if a match is a keyword\n * @PARAM {CompiledMode} mode - current mode\n * @PARAM {string} matchText - the textual match\n * @returns {KeywordData | false}\n */\n function keywordData(mode, matchText) {\n return mode.keywords[matchText];\n }\n\n function processKeywords() {\n if (!top.keywords) {\n emitter.addText(modeBuffer);\n return;\n }\n\n let lastIndex = 0;\n top.keywordPatternRe.lastIndex = 0;\n let match = top.keywordPatternRe.exec(modeBuffer);\n let buf = "";\n\n while (match) {\n buf += modeBuffer.substring(lastIndex, match.index);\n const word = language.case_insensitive ? match[0].toLowerCase() : match[0];\n const data = keywordData(top, word);\n if (data) {\n const [kind, keywordRelevance] = data;\n emitter.addText(buf);\n buf = "";\n\n keywordHits[word] = (keywordHits[word] || 0) + 1;\n if (keywordHits[word] <= MAX_KEYWORD_HITS) relevance += keywordRelevance;\n if (kind.startsWith("")) {\n // _ implied for relevance only, do not highlight\n // by applying a class name\n buf += match[0];\n } else {\n const cssClass = language.classNameAliases[kind] || kind;\n emitKeyword(match[0], cssClass);\n }\n } else {\n buf += match[0];\n }\n lastIndex = top.keywordPatternRe.lastIndex;\n match = top.keywordPatternRe.exec(modeBuffer);\n }\n buf += modeBuffer.substring(lastIndex);\n emitter.addText(buf);\n }\n\n function processSubLanguage() {\n if (modeBuffer === "") return;\n /** @type HighlightResult /\n let result = null;\n\n if (typeof top.subLanguage === 'string') {\n if (!languages[top.subLanguage]) {\n emitter.addText(modeBuffer);\n return;\n }\n result = _highlight(top.subLanguage, modeBuffer, true, continuations[top.subLanguage]);\n continuations[top.subLanguage] = /* @type {CompiledMode} */ (result._top);\n } else {\n result = highlightAuto(modeBuffer, top.subLanguage.length ? top.subLanguage : null);\n }\n\n // Counting embedded language score towards the host language may be disabled\n // with zeroing the containing mode relevance. Use case in point is Markdown that\n // allows XML everywhere and makes every XML snippet to have a much larger Markdown\n // score.\n if (top.relevance > 0) {\n relevance += result.relevance;\n }\n emitter.__addSublanguage(result._emitter, result.language);\n }\n\n function processBuffer() {\n if (top.subLanguage != null) {\n processSubLanguage();\n } else {\n processKeywords();\n }\n modeBuffer = '';\n }\n\n /**\n * @PARAM {string} text\n * @PARAM {string} scop

[...truncated, see workflow logs for full report]

[github-actions]

⏱️ Performance report

• Run: https://github.com/amirbiron/CodeBot/actions/runs/18610270290
• Artifact: perf-durations

(No performance test durations collected. Mark tests with @pytest.mark.performance.)

[codecov]

Codecov Report

✅ All modified and coverable lines are covered by tests.

📢 Thoughts on this report? Let us know!