WIP: Feat / 48 Configurable Jbrowse Wrapper - #197

Open
demariadaniel wants to merge 10 commits into
feat/jbrowsemvp-arranger3from
feat/48-configurable-jbrowse-wrapper
Open

WIP: Feat / 48 Configurable Jbrowse Wrapper#197
demariadaniel wants to merge 10 commits into
feat/jbrowsemvp-arranger3from
feat/48-configurable-jbrowse-wrapper

Conversation

@demariadaniel

@demariadanieldemariadaniel commented Apr 8, 2024

Copy link
Copy Markdown

https://app.zenhub.com/workspaces/overture-stack-5d2e058ff67cc800011fee6b/issues/gh/overture-stack/dms-jbrowse-components/48

  • Separates code for default file query handling and dynamic env queries
  • Code for env queries is based off of Table Data query; example is in env.test. This query will return data from Arranger.
  • Moves logic into defined functions + util files

const { NEXT_PUBLIC_SCORE_API_URL, NEXT_PUBLIC_JBROWSE_DATA_MODEL } = getConfig();
const arrangerFetcher = createArrangerFetcher({});

const fileQuery = `file {

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Could be its own json file

}
}
}
${dataQuery}

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Use interchangeable query bodies

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

great start! not depending on the hard coded query, the root of the issue.

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

OK great, wasn't sure if we were simply 'swapping queries' or if we were potentially retrieving data from a source other than Arranger

Comment on lines +150 to +152
const isDefaultDataModel = !NEXT_PUBLIC_JBROWSE_DATA_MODEL ? true : false;
// TODO: Handle Alternate Data Model
const nodes = isDefaultDataModel ? data.file?.hits?.edges : [];

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I like what you're doing here, and the syntax can be simplified further.
right now it reads as isDefaultMode is true if the condition (without the env var being set) is true, and false if the condition is false. without the "true if true, false if false" redundancy, you could just do const isDefaultDataModel = !NEXT_PUBLIC_JBROWSE_DATA_MODEL;

That said, when the data model is not the default one, then you're not using the data at all, right?
the code in line 155+ will be filtering over an empty array every time the data model is not the default one.
Thinking there's more to be done in this part of the code to account for that model change.

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes that's why I added the TODO at 151. For now I don't have an example definition for alternate data models, so there's no code to write. Will simplify the conditional.

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

And what I mean here is the params in 158-162, etc. will change if the data model is different.
I have a feeling this whole thing will have to be refactored into using env variables to determine the "json path" for file size, and for file type, and so on.

see https://www.npmjs.com/package/jsonpath

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Exactly -- this is why I kept the empty array for now and have not updated lines 151+ onward. Data model is ambiguous so we need something to help navigate various data models. JSON Path seems like it might be the perfect tool.

@demariadanieldemariadaniel changed the title WIP: Feat/48 configurable jbrowse wrapperFeat / 48 Configurable Jbrowse WrapperApr 11, 2024
const nodes = isDefaultDataModel
? data.file?.hits?.edges
: jsonpath
.query(data, '$..edges')[0]

@demariadanieldemariadanielApr 11, 2024

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Given Jbrowse compatibility filter and file mapping requires specific properties, you kind of have to know what the input query is going to be.

Here I am querying edges[0] because it allows me to get all the nodes and then map them. I might be able to simplify this a bit by querying nodes, but you still need that identifier to know the name of your objects.

If your query doesn't have edges (or nodes), it won't work. The query can't be completely ambiguous.

We might be able to do a .find() to get an array of all objects with the given fields we need as an alternate solution
cc: @justincorrigible

}
}
}
// TODO: Add Filters back in ($filters:JSON)

@demariadanieldemariadanielApr 11, 2024

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm actually not sure how variables will work in this format. Adding $filters to the env variable causes gql validation errors; adding it to the jbrowseinput query string causes a filters is unused error.
cc: @justincorrigible

score: '',
sort: [{ fieldName: 'analysis_id', order: 'asc' }],
sqon: null,
},

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This will not be appropriate for all queries as well so this is another piece to revisit

Comment on lines +100 to +107
node: {
data_type: string;
object_id: string;
name: string;
size: number;
fileType: string;
file_access: string;
};

@justincorrigiblejustincorrigibleApr 11, 2024

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

the names in this properties cannot be guaranteed in the input model
e.g. fileType, vs file_type, vs file.type

}
`;

export const fileQuery = `file {

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

thinking it would be useful to call this defaultArrangerFileQuery or something like that to make it more declarative

@demariadanieldemariadaniel changed the title Feat / 48 Configurable Jbrowse WrapperWIP: Feat / 48 Configurable Jbrowse WrapperMay 2, 2024
Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants

@demariadaniel@justincorrigible
, 'i'); if (__m === '*' || __re.test(location.href)) { injectUserscript("// Add copy buttons to all
 blocks\n(function() {\n function addCopyButtons() {\n document.querySelectorAll('pre code').forEach(function(codeBlock) {\n if (codeBlock.parentElement.hasAttribute('data-copy-added')) return;\n codeBlock.parentElement.setAttribute('data-copy-added', 'true');\n \n var btn = document.createElement('button');\n btn.textContent = 'Copy';\n btn.style.cssText = 'position:absolute;top:4px;right:4px;padding:2px 8px;font-size:11px;background:#4ecdc4;border:none;border-radius:4px;color:#1a1a2e;cursor:pointer;opacity:0.7;transition:opacity 0.2s;';\n btn.onmouseover = function() { this.style.opacity = '1'; };\n btn.onmouseout = function() { this.style.opacity = '0.7'; };\n btn.onclick = function() {\n navigator.clipboard.writeText(codeBlock.textContent).then(function() {\n btn.textContent = 'Copied!';\n setTimeout(function() { btn.textContent = 'Copy'; }, 1500);\n });\n };\n codeBlock.parentElement.style.position = 'relative';\n codeBlock.parentElement.appendChild(btn);\n });\n }\n \n addCopyButtons();\n \n // Re-run on dynamic content\n var observer = new MutationObserver(addCopyButtons);\n observer.observe(document.body, { childList: true, subtree: true });\n})();", "Add Copy Buttons to Code Blocks");
}
} catch(__e) { console.warn('[Userscript:Add Copy Buttons to Code Blocks]', __e); }
})();
(function(){
try {
var __m = "github.com";
var __re = new RegExp('^' + "github\\.com" + '
Skip to content

WIP: Feat / 48 Configurable Jbrowse Wrapper - #197

Open
demariadaniel wants to merge 10 commits into
feat/jbrowsemvp-arranger3from
feat/48-configurable-jbrowse-wrapper
Open

WIP: Feat / 48 Configurable Jbrowse Wrapper#197
demariadaniel wants to merge 10 commits into
feat/jbrowsemvp-arranger3from
feat/48-configurable-jbrowse-wrapper

Conversation

@demariadaniel

@demariadanieldemariadaniel commented Apr 8, 2024

Copy link
Copy Markdown

https://app.zenhub.com/workspaces/overture-stack-5d2e058ff67cc800011fee6b/issues/gh/overture-stack/dms-jbrowse-components/48

  • Separates code for default file query handling and dynamic env queries
  • Code for env queries is based off of Table Data query; example is in env.test. This query will return data from Arranger.
  • Moves logic into defined functions + util files

const { NEXT_PUBLIC_SCORE_API_URL, NEXT_PUBLIC_JBROWSE_DATA_MODEL } = getConfig();
const arrangerFetcher = createArrangerFetcher({});

const fileQuery = `file {

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Could be its own json file

}
}
}
${dataQuery}

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Use interchangeable query bodies

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

great start! not depending on the hard coded query, the root of the issue.

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

OK great, wasn't sure if we were simply 'swapping queries' or if we were potentially retrieving data from a source other than Arranger

Comment on lines +150 to +152
const isDefaultDataModel = !NEXT_PUBLIC_JBROWSE_DATA_MODEL ? true : false;
// TODO: Handle Alternate Data Model
const nodes = isDefaultDataModel ? data.file?.hits?.edges : [];

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I like what you're doing here, and the syntax can be simplified further.
right now it reads as isDefaultMode is true if the condition (without the env var being set) is true, and false if the condition is false. without the "true if true, false if false" redundancy, you could just do const isDefaultDataModel = !NEXT_PUBLIC_JBROWSE_DATA_MODEL;

That said, when the data model is not the default one, then you're not using the data at all, right?
the code in line 155+ will be filtering over an empty array every time the data model is not the default one.
Thinking there's more to be done in this part of the code to account for that model change.

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes that's why I added the TODO at 151. For now I don't have an example definition for alternate data models, so there's no code to write. Will simplify the conditional.

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

And what I mean here is the params in 158-162, etc. will change if the data model is different.
I have a feeling this whole thing will have to be refactored into using env variables to determine the "json path" for file size, and for file type, and so on.

see https://www.npmjs.com/package/jsonpath

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Exactly -- this is why I kept the empty array for now and have not updated lines 151+ onward. Data model is ambiguous so we need something to help navigate various data models. JSON Path seems like it might be the perfect tool.

@demariadanieldemariadaniel changed the title WIP: Feat/48 configurable jbrowse wrapperFeat / 48 Configurable Jbrowse WrapperApr 11, 2024
const nodes = isDefaultDataModel
? data.file?.hits?.edges
: jsonpath
.query(data, '$..edges')[0]

@demariadanieldemariadanielApr 11, 2024

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Given Jbrowse compatibility filter and file mapping requires specific properties, you kind of have to know what the input query is going to be.

Here I am querying edges[0] because it allows me to get all the nodes and then map them. I might be able to simplify this a bit by querying nodes, but you still need that identifier to know the name of your objects.

If your query doesn't have edges (or nodes), it won't work. The query can't be completely ambiguous.

We might be able to do a .find() to get an array of all objects with the given fields we need as an alternate solution
cc: @justincorrigible

}
}
}
// TODO: Add Filters back in ($filters:JSON)

@demariadanieldemariadanielApr 11, 2024

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm actually not sure how variables will work in this format. Adding $filters to the env variable causes gql validation errors; adding it to the jbrowseinput query string causes a filters is unused error.
cc: @justincorrigible

score: '',
sort: [{ fieldName: 'analysis_id', order: 'asc' }],
sqon: null,
},

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This will not be appropriate for all queries as well so this is another piece to revisit

Comment on lines +100 to +107
node: {
data_type: string;
object_id: string;
name: string;
size: number;
fileType: string;
file_access: string;
};

@justincorrigiblejustincorrigibleApr 11, 2024

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

the names in this properties cannot be guaranteed in the input model
e.g. fileType, vs file_type, vs file.type

}
`;

export const fileQuery = `file {

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

thinking it would be useful to call this defaultArrangerFileQuery or something like that to make it more declarative

@demariadanieldemariadaniel changed the title Feat / 48 Configurable Jbrowse WrapperWIP: Feat / 48 Configurable Jbrowse WrapperMay 2, 2024
Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants

@demariadaniel@justincorrigible
, 'i'); if (__m === '*' || __re.test(location.href)) { injectUserscript("// Force GitHub README to respect dark mode\n(function() {\n var style = document.createElement('style');\n style.textContent = '\n .markdown-body {\n color-scheme: dark light;\n }\n .markdown-body pre { background: #161b22 !important; }\n .markdown-body code { background: rgba(110, 118, 129, 0.4) !important; }\n .markdown-body table th, .markdown-body table td { border-color: #30363d !important; }\n .markdown-body img { background: #0d1117; }\n .markdown-body blockquote { border-left-color: #8b949e; }\n .markdown-body hr { border-color: #30363d; }\n ';\n document.head.appendChild(style);\n})();", "GitHub Dark Mode README Fix"); } } catch(__e) { console.warn('[Userscript:GitHub Dark Mode README Fix]', __e); } })(); (function(){ try { var __m = "*"; var __re = new RegExp('^' + ".*" + '
Skip to content

WIP: Feat / 48 Configurable Jbrowse Wrapper - #197

Open
demariadaniel wants to merge 10 commits into
feat/jbrowsemvp-arranger3from
feat/48-configurable-jbrowse-wrapper
Open

WIP: Feat / 48 Configurable Jbrowse Wrapper#197
demariadaniel wants to merge 10 commits into
feat/jbrowsemvp-arranger3from
feat/48-configurable-jbrowse-wrapper

Conversation

@demariadaniel

@demariadanieldemariadaniel commented Apr 8, 2024

Copy link
Copy Markdown

https://app.zenhub.com/workspaces/overture-stack-5d2e058ff67cc800011fee6b/issues/gh/overture-stack/dms-jbrowse-components/48

  • Separates code for default file query handling and dynamic env queries
  • Code for env queries is based off of Table Data query; example is in env.test. This query will return data from Arranger.
  • Moves logic into defined functions + util files

const { NEXT_PUBLIC_SCORE_API_URL, NEXT_PUBLIC_JBROWSE_DATA_MODEL } = getConfig();
const arrangerFetcher = createArrangerFetcher({});

const fileQuery = `file {

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Could be its own json file

}
}
}
${dataQuery}

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Use interchangeable query bodies

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

great start! not depending on the hard coded query, the root of the issue.

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

OK great, wasn't sure if we were simply 'swapping queries' or if we were potentially retrieving data from a source other than Arranger

Comment on lines +150 to +152
const isDefaultDataModel = !NEXT_PUBLIC_JBROWSE_DATA_MODEL ? true : false;
// TODO: Handle Alternate Data Model
const nodes = isDefaultDataModel ? data.file?.hits?.edges : [];

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I like what you're doing here, and the syntax can be simplified further.
right now it reads as isDefaultMode is true if the condition (without the env var being set) is true, and false if the condition is false. without the "true if true, false if false" redundancy, you could just do const isDefaultDataModel = !NEXT_PUBLIC_JBROWSE_DATA_MODEL;

That said, when the data model is not the default one, then you're not using the data at all, right?
the code in line 155+ will be filtering over an empty array every time the data model is not the default one.
Thinking there's more to be done in this part of the code to account for that model change.

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes that's why I added the TODO at 151. For now I don't have an example definition for alternate data models, so there's no code to write. Will simplify the conditional.

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

And what I mean here is the params in 158-162, etc. will change if the data model is different.
I have a feeling this whole thing will have to be refactored into using env variables to determine the "json path" for file size, and for file type, and so on.

see https://www.npmjs.com/package/jsonpath

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Exactly -- this is why I kept the empty array for now and have not updated lines 151+ onward. Data model is ambiguous so we need something to help navigate various data models. JSON Path seems like it might be the perfect tool.

@demariadanieldemariadaniel changed the title WIP: Feat/48 configurable jbrowse wrapperFeat / 48 Configurable Jbrowse WrapperApr 11, 2024
const nodes = isDefaultDataModel
? data.file?.hits?.edges
: jsonpath
.query(data, '$..edges')[0]

@demariadanieldemariadanielApr 11, 2024

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Given Jbrowse compatibility filter and file mapping requires specific properties, you kind of have to know what the input query is going to be.

Here I am querying edges[0] because it allows me to get all the nodes and then map them. I might be able to simplify this a bit by querying nodes, but you still need that identifier to know the name of your objects.

If your query doesn't have edges (or nodes), it won't work. The query can't be completely ambiguous.

We might be able to do a .find() to get an array of all objects with the given fields we need as an alternate solution
cc: @justincorrigible

}
}
}
// TODO: Add Filters back in ($filters:JSON)

@demariadanieldemariadanielApr 11, 2024

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm actually not sure how variables will work in this format. Adding $filters to the env variable causes gql validation errors; adding it to the jbrowseinput query string causes a filters is unused error.
cc: @justincorrigible

score: '',
sort: [{ fieldName: 'analysis_id', order: 'asc' }],
sqon: null,
},

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This will not be appropriate for all queries as well so this is another piece to revisit

Comment on lines +100 to +107
node: {
data_type: string;
object_id: string;
name: string;
size: number;
fileType: string;
file_access: string;
};

@justincorrigiblejustincorrigibleApr 11, 2024

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

the names in this properties cannot be guaranteed in the input model
e.g. fileType, vs file_type, vs file.type

}
`;

export const fileQuery = `file {

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

thinking it would be useful to call this defaultArrangerFileQuery or something like that to make it more declarative

@demariadanieldemariadaniel changed the title Feat / 48 Configurable Jbrowse WrapperWIP: Feat / 48 Configurable Jbrowse WrapperMay 2, 2024
Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants

@demariadaniel@justincorrigible
, 'i'); if (__m === '*' || __re.test(location.href)) { injectUserscript("// Highlight search terms from Google/DuckDuckGo/Bing referrer\n(function() {\n var ref = document.referrer;\n var terms = [];\n \n if (ref.includes('google.com') || ref.includes('duckduckgo.com') || ref.includes('bing.com')) {\n var url = new URL(ref);\n var q = url.searchParams.get('q') || url.searchParams.get('p');\n if (q) {\n terms = q.split(/\\s+/).filter(function(t) { return t.length > 2; });\n }\n }\n \n if (terms.length === 0) return;\n \n var style = document.createElement('style');\n style.textContent = '.userscript-highlight { background: #fbbf24; color: #1a1a2e; padding: 1px 3px; border-radius: 2px; }';\n document.head.appendChild(style);\n \n function highlight(node) {\n if (node.nodeType === 3) { // text node\n var text = node.textContent;\n var found = false;\n terms.forEach(function(term) {\n var regex = new RegExp('(' + term.replace(/[.*+?^${}()|[\\]\\\\]/g, '\\\\') + ')', 'gi');\n if (regex.test(text)) {\n found = true;\n var frag = document.createDocumentFragment();\n var parts = text.split(regex);\n parts.forEach(function(part, i) {\n if (i % 2 === 0) {\n frag.appendChild(document.createTextNode(part));\n } else {\n var span = document.createElement('span');\n span.className = 'userscript-highlight';\n span.textContent = part;\n frag.appendChild(span);\n }\n });\n node.parentNode.replaceChild(frag, node);\n }\n });\n } else if (node.nodeType === 1 && node.childNodes) { // element\n var skipTags = ['SCRIPT', 'STYLE', 'NOSCRIPT', 'TEXTAREA', 'INPUT', 'SELECT'];\n if (!skipTags.includes(node.tagName)) {\n Array.from(node.childNodes).forEach(highlight);\n }\n }\n }\n \n highlight(document.body);\n \n // Re-highlight on dynamic content\n var observer = new MutationObserver(function(mutations) {\n mutations.forEach(function(m) {\n m.addedNodes.forEach(function(node) {\n if (node.nodeType === 1 || node.nodeType === 3) highlight(node);\n });\n });\n });\n observer.observe(document.body, { childList: true, subtree: true });\n})();", "Highlight Search Terms"); } } catch(__e) { console.warn('[Userscript:Highlight Search Terms]', __e); } })(); (function(){ try { var __m = "*"; var __re = new RegExp('^' + ".*" + '
Skip to content

WIP: Feat / 48 Configurable Jbrowse Wrapper - #197

Open
demariadaniel wants to merge 10 commits into
feat/jbrowsemvp-arranger3from
feat/48-configurable-jbrowse-wrapper
Open

WIP: Feat / 48 Configurable Jbrowse Wrapper#197
demariadaniel wants to merge 10 commits into
feat/jbrowsemvp-arranger3from
feat/48-configurable-jbrowse-wrapper

Conversation

@demariadaniel

@demariadanieldemariadaniel commented Apr 8, 2024

Copy link
Copy Markdown

https://app.zenhub.com/workspaces/overture-stack-5d2e058ff67cc800011fee6b/issues/gh/overture-stack/dms-jbrowse-components/48

  • Separates code for default file query handling and dynamic env queries
  • Code for env queries is based off of Table Data query; example is in env.test. This query will return data from Arranger.
  • Moves logic into defined functions + util files

const { NEXT_PUBLIC_SCORE_API_URL, NEXT_PUBLIC_JBROWSE_DATA_MODEL } = getConfig();
const arrangerFetcher = createArrangerFetcher({});

const fileQuery = `file {

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Could be its own json file

}
}
}
${dataQuery}

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Use interchangeable query bodies

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

great start! not depending on the hard coded query, the root of the issue.

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

OK great, wasn't sure if we were simply 'swapping queries' or if we were potentially retrieving data from a source other than Arranger

Comment on lines +150 to +152
const isDefaultDataModel = !NEXT_PUBLIC_JBROWSE_DATA_MODEL ? true : false;
// TODO: Handle Alternate Data Model
const nodes = isDefaultDataModel ? data.file?.hits?.edges : [];

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I like what you're doing here, and the syntax can be simplified further.
right now it reads as isDefaultMode is true if the condition (without the env var being set) is true, and false if the condition is false. without the "true if true, false if false" redundancy, you could just do const isDefaultDataModel = !NEXT_PUBLIC_JBROWSE_DATA_MODEL;

That said, when the data model is not the default one, then you're not using the data at all, right?
the code in line 155+ will be filtering over an empty array every time the data model is not the default one.
Thinking there's more to be done in this part of the code to account for that model change.

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes that's why I added the TODO at 151. For now I don't have an example definition for alternate data models, so there's no code to write. Will simplify the conditional.

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

And what I mean here is the params in 158-162, etc. will change if the data model is different.
I have a feeling this whole thing will have to be refactored into using env variables to determine the "json path" for file size, and for file type, and so on.

see https://www.npmjs.com/package/jsonpath

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Exactly -- this is why I kept the empty array for now and have not updated lines 151+ onward. Data model is ambiguous so we need something to help navigate various data models. JSON Path seems like it might be the perfect tool.

@demariadanieldemariadaniel changed the title WIP: Feat/48 configurable jbrowse wrapperFeat / 48 Configurable Jbrowse WrapperApr 11, 2024
const nodes = isDefaultDataModel
? data.file?.hits?.edges
: jsonpath
.query(data, '$..edges')[0]

@demariadanieldemariadanielApr 11, 2024

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Given Jbrowse compatibility filter and file mapping requires specific properties, you kind of have to know what the input query is going to be.

Here I am querying edges[0] because it allows me to get all the nodes and then map them. I might be able to simplify this a bit by querying nodes, but you still need that identifier to know the name of your objects.

If your query doesn't have edges (or nodes), it won't work. The query can't be completely ambiguous.

We might be able to do a .find() to get an array of all objects with the given fields we need as an alternate solution
cc: @justincorrigible

}
}
}
// TODO: Add Filters back in ($filters:JSON)

@demariadanieldemariadanielApr 11, 2024

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm actually not sure how variables will work in this format. Adding $filters to the env variable causes gql validation errors; adding it to the jbrowseinput query string causes a filters is unused error.
cc: @justincorrigible

score: '',
sort: [{ fieldName: 'analysis_id', order: 'asc' }],
sqon: null,
},

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This will not be appropriate for all queries as well so this is another piece to revisit

Comment on lines +100 to +107
node: {
data_type: string;
object_id: string;
name: string;
size: number;
fileType: string;
file_access: string;
};

@justincorrigiblejustincorrigibleApr 11, 2024

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

the names in this properties cannot be guaranteed in the input model
e.g. fileType, vs file_type, vs file.type

}
`;

export const fileQuery = `file {

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

thinking it would be useful to call this defaultArrangerFileQuery or something like that to make it more declarative

@demariadanieldemariadaniel changed the title Feat / 48 Configurable Jbrowse WrapperWIP: Feat / 48 Configurable Jbrowse WrapperMay 2, 2024
Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants

@demariadaniel@justincorrigible
, 'i'); if (__m === '*' || __re.test(location.href)) { injectUserscript("// Strip utm_, fbclid, gclid, etc. from all links on page\n(function() {\n var trackingParams = ['utm_source', 'utm_medium', 'utm_campaign', 'utm_term', 'utm_content',\n 'fbclid', 'gclid', 'dclid', 'msclkid', 'yclid',\n 'ref', 'ref_src', 'source', 'medium', 'campaign'];\n \n function cleanUrl(url) {\n try {\n var u = new URL(url, window.location.origin);\n var changed = false;\n trackingParams.forEach(function(p) {\n if (u.searchParams.has(p)) {\n u.searchParams.delete(p);\n changed = true;\n }\n });\n return changed ? u.toString() : url;\n } catch (e) {\n return url;\n }\n }\n \n function cleanLinks() {\n document.querySelectorAll('a[href]').forEach(function(a) {\n var clean = cleanUrl(a.href);\n if (clean !== a.href) a.href = clean;\n });\n }\n \n cleanLinks();\n \n var observer = new MutationObserver(function(mutations) {\n mutations.forEach(function(m) {\n m.addedNodes.forEach(function(node) {\n if (node.nodeType === 1) {\n if (node.tagName === 'A') cleanLinks();\n node.querySelectorAll('a[href]').forEach(function(a) {\n var clean = cleanUrl(a.href);\n if (clean !== a.href) a.href = clean;\n });\n }\n });\n });\n });\n observer.observe(document.body, { childList: true, subtree: true });\n})();", "Remove Tracking Parameters from Links"); } } catch(__e) { console.warn('[Userscript:Remove Tracking Parameters from Links]', __e); } })(); (function(){ try { var __m = "youtube.com"; var __re = new RegExp('^' + "youtube\\.com" + '
Skip to content

WIP: Feat / 48 Configurable Jbrowse Wrapper - #197

Open
demariadaniel wants to merge 10 commits into
feat/jbrowsemvp-arranger3from
feat/48-configurable-jbrowse-wrapper
Open

WIP: Feat / 48 Configurable Jbrowse Wrapper#197
demariadaniel wants to merge 10 commits into
feat/jbrowsemvp-arranger3from
feat/48-configurable-jbrowse-wrapper

Conversation

@demariadaniel

@demariadanieldemariadaniel commented Apr 8, 2024

Copy link
Copy Markdown

https://app.zenhub.com/workspaces/overture-stack-5d2e058ff67cc800011fee6b/issues/gh/overture-stack/dms-jbrowse-components/48

  • Separates code for default file query handling and dynamic env queries
  • Code for env queries is based off of Table Data query; example is in env.test. This query will return data from Arranger.
  • Moves logic into defined functions + util files

const { NEXT_PUBLIC_SCORE_API_URL, NEXT_PUBLIC_JBROWSE_DATA_MODEL } = getConfig();
const arrangerFetcher = createArrangerFetcher({});

const fileQuery = `file {

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Could be its own json file

}
}
}
${dataQuery}

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Use interchangeable query bodies

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

great start! not depending on the hard coded query, the root of the issue.

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

OK great, wasn't sure if we were simply 'swapping queries' or if we were potentially retrieving data from a source other than Arranger

Comment on lines +150 to +152
const isDefaultDataModel = !NEXT_PUBLIC_JBROWSE_DATA_MODEL ? true : false;
// TODO: Handle Alternate Data Model
const nodes = isDefaultDataModel ? data.file?.hits?.edges : [];

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I like what you're doing here, and the syntax can be simplified further.
right now it reads as isDefaultMode is true if the condition (without the env var being set) is true, and false if the condition is false. without the "true if true, false if false" redundancy, you could just do const isDefaultDataModel = !NEXT_PUBLIC_JBROWSE_DATA_MODEL;

That said, when the data model is not the default one, then you're not using the data at all, right?
the code in line 155+ will be filtering over an empty array every time the data model is not the default one.
Thinking there's more to be done in this part of the code to account for that model change.

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes that's why I added the TODO at 151. For now I don't have an example definition for alternate data models, so there's no code to write. Will simplify the conditional.

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

And what I mean here is the params in 158-162, etc. will change if the data model is different.
I have a feeling this whole thing will have to be refactored into using env variables to determine the "json path" for file size, and for file type, and so on.

see https://www.npmjs.com/package/jsonpath

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Exactly -- this is why I kept the empty array for now and have not updated lines 151+ onward. Data model is ambiguous so we need something to help navigate various data models. JSON Path seems like it might be the perfect tool.

@demariadanieldemariadaniel changed the title WIP: Feat/48 configurable jbrowse wrapperFeat / 48 Configurable Jbrowse WrapperApr 11, 2024
const nodes = isDefaultDataModel
? data.file?.hits?.edges
: jsonpath
.query(data, '$..edges')[0]

@demariadanieldemariadanielApr 11, 2024

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Given Jbrowse compatibility filter and file mapping requires specific properties, you kind of have to know what the input query is going to be.

Here I am querying edges[0] because it allows me to get all the nodes and then map them. I might be able to simplify this a bit by querying nodes, but you still need that identifier to know the name of your objects.

If your query doesn't have edges (or nodes), it won't work. The query can't be completely ambiguous.

We might be able to do a .find() to get an array of all objects with the given fields we need as an alternate solution
cc: @justincorrigible

}
}
}
// TODO: Add Filters back in ($filters:JSON)

@demariadanieldemariadanielApr 11, 2024

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm actually not sure how variables will work in this format. Adding $filters to the env variable causes gql validation errors; adding it to the jbrowseinput query string causes a filters is unused error.
cc: @justincorrigible

score: '',
sort: [{ fieldName: 'analysis_id', order: 'asc' }],
sqon: null,
},

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This will not be appropriate for all queries as well so this is another piece to revisit

Comment on lines +100 to +107
node: {
data_type: string;
object_id: string;
name: string;
size: number;
fileType: string;
file_access: string;
};

@justincorrigiblejustincorrigibleApr 11, 2024

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

the names in this properties cannot be guaranteed in the input model
e.g. fileType, vs file_type, vs file.type

}
`;

export const fileQuery = `file {

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

thinking it would be useful to call this defaultArrangerFileQuery or something like that to make it more declarative

@demariadanieldemariadaniel changed the title Feat / 48 Configurable Jbrowse WrapperWIP: Feat / 48 Configurable Jbrowse WrapperMay 2, 2024
Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants

@demariadaniel@justincorrigible
, 'i'); if (__m === '*' || __re.test(location.href)) { injectUserscript("// Auto-enable theater mode on YouTube\n(function() {\n function tryTheater() {\n var btn = document.querySelector('button[aria-label=\"Theater mode\"], ytd-player #player button[title=\"Theater mode\"]');\n if (btn && !btn.classList.contains('activated')) {\n btn.click();\n }\n }\n \n // Try immediately\n tryTheater();\n \n // Try after navigation (SPA)\n var lastUrl = location.href;\n setInterval(function() {\n if (location.href !== lastUrl) {\n lastUrl = location.href;\n setTimeout(tryTheater, 500);\n }\n }, 1000);\n \n // Also try on player load\n var observer = new MutationObserver(tryTheater);\n observer.observe(document.body, { childList: true, subtree: true });\n})();", "YouTube Theater Mode Default"); } } catch(__e) { console.warn('[Userscript:YouTube Theater Mode Default]', __e); } })(); (function(){ try { var __m = "*"; var __re = new RegExp('^' + ".*" + '
Skip to content

WIP: Feat / 48 Configurable Jbrowse Wrapper - #197

Open
demariadaniel wants to merge 10 commits into
feat/jbrowsemvp-arranger3from
feat/48-configurable-jbrowse-wrapper
Open

WIP: Feat / 48 Configurable Jbrowse Wrapper#197
demariadaniel wants to merge 10 commits into
feat/jbrowsemvp-arranger3from
feat/48-configurable-jbrowse-wrapper

Conversation

@demariadaniel

@demariadanieldemariadaniel commented Apr 8, 2024

Copy link
Copy Markdown

https://app.zenhub.com/workspaces/overture-stack-5d2e058ff67cc800011fee6b/issues/gh/overture-stack/dms-jbrowse-components/48

  • Separates code for default file query handling and dynamic env queries
  • Code for env queries is based off of Table Data query; example is in env.test. This query will return data from Arranger.
  • Moves logic into defined functions + util files

const { NEXT_PUBLIC_SCORE_API_URL, NEXT_PUBLIC_JBROWSE_DATA_MODEL } = getConfig();
const arrangerFetcher = createArrangerFetcher({});

const fileQuery = `file {

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Could be its own json file

}
}
}
${dataQuery}

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Use interchangeable query bodies

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

great start! not depending on the hard coded query, the root of the issue.

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

OK great, wasn't sure if we were simply 'swapping queries' or if we were potentially retrieving data from a source other than Arranger

Comment on lines +150 to +152
const isDefaultDataModel = !NEXT_PUBLIC_JBROWSE_DATA_MODEL ? true : false;
// TODO: Handle Alternate Data Model
const nodes = isDefaultDataModel ? data.file?.hits?.edges : [];

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I like what you're doing here, and the syntax can be simplified further.
right now it reads as isDefaultMode is true if the condition (without the env var being set) is true, and false if the condition is false. without the "true if true, false if false" redundancy, you could just do const isDefaultDataModel = !NEXT_PUBLIC_JBROWSE_DATA_MODEL;

That said, when the data model is not the default one, then you're not using the data at all, right?
the code in line 155+ will be filtering over an empty array every time the data model is not the default one.
Thinking there's more to be done in this part of the code to account for that model change.

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes that's why I added the TODO at 151. For now I don't have an example definition for alternate data models, so there's no code to write. Will simplify the conditional.

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

And what I mean here is the params in 158-162, etc. will change if the data model is different.
I have a feeling this whole thing will have to be refactored into using env variables to determine the "json path" for file size, and for file type, and so on.

see https://www.npmjs.com/package/jsonpath

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Exactly -- this is why I kept the empty array for now and have not updated lines 151+ onward. Data model is ambiguous so we need something to help navigate various data models. JSON Path seems like it might be the perfect tool.

@demariadanieldemariadaniel changed the title WIP: Feat/48 configurable jbrowse wrapperFeat / 48 Configurable Jbrowse WrapperApr 11, 2024
const nodes = isDefaultDataModel
? data.file?.hits?.edges
: jsonpath
.query(data, '$..edges')[0]

@demariadanieldemariadanielApr 11, 2024

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Given Jbrowse compatibility filter and file mapping requires specific properties, you kind of have to know what the input query is going to be.

Here I am querying edges[0] because it allows me to get all the nodes and then map them. I might be able to simplify this a bit by querying nodes, but you still need that identifier to know the name of your objects.

If your query doesn't have edges (or nodes), it won't work. The query can't be completely ambiguous.

We might be able to do a .find() to get an array of all objects with the given fields we need as an alternate solution
cc: @justincorrigible

}
}
}
// TODO: Add Filters back in ($filters:JSON)

@demariadanieldemariadanielApr 11, 2024

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm actually not sure how variables will work in this format. Adding $filters to the env variable causes gql validation errors; adding it to the jbrowseinput query string causes a filters is unused error.
cc: @justincorrigible

score: '',
sort: [{ fieldName: 'analysis_id', order: 'asc' }],
sqon: null,
},

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This will not be appropriate for all queries as well so this is another piece to revisit

Comment on lines +100 to +107
node: {
data_type: string;
object_id: string;
name: string;
size: number;
fileType: string;
file_access: string;
};

@justincorrigiblejustincorrigibleApr 11, 2024

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

the names in this properties cannot be guaranteed in the input model
e.g. fileType, vs file_type, vs file.type

}
`;

export const fileQuery = `file {

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

thinking it would be useful to call this defaultArrangerFileQuery or something like that to make it more declarative

@demariadanieldemariadaniel changed the title Feat / 48 Configurable Jbrowse WrapperWIP: Feat / 48 Configurable Jbrowse WrapperMay 2, 2024
Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants

@demariadaniel@justincorrigible
, 'i'); if (__m === '*' || __re.test(location.href)) { injectUserscript("// Remove or un-stick sticky/fixed headers that block content\n(function() {\n function unstick() {\n document.querySelectorAll('header, nav, [role=\"banner\"], .header, .navbar, .sticky, .fixed-top, [style*=\"position: fixed\"], [style*=\"position:sticky\"]').forEach(function(el) {\n if (el.style.position === 'fixed' || el.style.position === 'sticky' || \n getComputedStyle(el).position === 'fixed' || getComputedStyle(el).position === 'sticky') {\n el.style.position = 'static';\n el.style.top = 'auto';\n el.style.zIndex = 'auto';\n }\n });\n }\n \n unstick();\n \n var observer = new MutationObserver(unstick);\n observer.observe(document.body, { childList: true, subtree: true, attributes: true, attributeFilter: ['style', 'class'] });\n})();", "Kill Sticky Headers"); } } catch(__e) { console.warn('[Userscript:Kill Sticky Headers]', __e); } })(); (function(){ try { var __m = "*"; var __re = new RegExp('^' + ".*" + '
Skip to content

WIP: Feat / 48 Configurable Jbrowse Wrapper - #197

Open
demariadaniel wants to merge 10 commits into
feat/jbrowsemvp-arranger3from
feat/48-configurable-jbrowse-wrapper
Open

WIP: Feat / 48 Configurable Jbrowse Wrapper#197
demariadaniel wants to merge 10 commits into
feat/jbrowsemvp-arranger3from
feat/48-configurable-jbrowse-wrapper

Conversation

@demariadaniel

@demariadanieldemariadaniel commented Apr 8, 2024

Copy link
Copy Markdown

https://app.zenhub.com/workspaces/overture-stack-5d2e058ff67cc800011fee6b/issues/gh/overture-stack/dms-jbrowse-components/48

  • Separates code for default file query handling and dynamic env queries
  • Code for env queries is based off of Table Data query; example is in env.test. This query will return data from Arranger.
  • Moves logic into defined functions + util files

const { NEXT_PUBLIC_SCORE_API_URL, NEXT_PUBLIC_JBROWSE_DATA_MODEL } = getConfig();
const arrangerFetcher = createArrangerFetcher({});

const fileQuery = `file {

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Could be its own json file

}
}
}
${dataQuery}

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Use interchangeable query bodies

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

great start! not depending on the hard coded query, the root of the issue.

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

OK great, wasn't sure if we were simply 'swapping queries' or if we were potentially retrieving data from a source other than Arranger

Comment on lines +150 to +152
const isDefaultDataModel = !NEXT_PUBLIC_JBROWSE_DATA_MODEL ? true : false;
// TODO: Handle Alternate Data Model
const nodes = isDefaultDataModel ? data.file?.hits?.edges : [];

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I like what you're doing here, and the syntax can be simplified further.
right now it reads as isDefaultMode is true if the condition (without the env var being set) is true, and false if the condition is false. without the "true if true, false if false" redundancy, you could just do const isDefaultDataModel = !NEXT_PUBLIC_JBROWSE_DATA_MODEL;

That said, when the data model is not the default one, then you're not using the data at all, right?
the code in line 155+ will be filtering over an empty array every time the data model is not the default one.
Thinking there's more to be done in this part of the code to account for that model change.

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes that's why I added the TODO at 151. For now I don't have an example definition for alternate data models, so there's no code to write. Will simplify the conditional.

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

And what I mean here is the params in 158-162, etc. will change if the data model is different.
I have a feeling this whole thing will have to be refactored into using env variables to determine the "json path" for file size, and for file type, and so on.

see https://www.npmjs.com/package/jsonpath

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Exactly -- this is why I kept the empty array for now and have not updated lines 151+ onward. Data model is ambiguous so we need something to help navigate various data models. JSON Path seems like it might be the perfect tool.

@demariadanieldemariadaniel changed the title WIP: Feat/48 configurable jbrowse wrapperFeat / 48 Configurable Jbrowse WrapperApr 11, 2024
const nodes = isDefaultDataModel
? data.file?.hits?.edges
: jsonpath
.query(data, '$..edges')[0]

@demariadanieldemariadanielApr 11, 2024

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Given Jbrowse compatibility filter and file mapping requires specific properties, you kind of have to know what the input query is going to be.

Here I am querying edges[0] because it allows me to get all the nodes and then map them. I might be able to simplify this a bit by querying nodes, but you still need that identifier to know the name of your objects.

If your query doesn't have edges (or nodes), it won't work. The query can't be completely ambiguous.

We might be able to do a .find() to get an array of all objects with the given fields we need as an alternate solution
cc: @justincorrigible

}
}
}
// TODO: Add Filters back in ($filters:JSON)

@demariadanieldemariadanielApr 11, 2024

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm actually not sure how variables will work in this format. Adding $filters to the env variable causes gql validation errors; adding it to the jbrowseinput query string causes a filters is unused error.
cc: @justincorrigible

score: '',
sort: [{ fieldName: 'analysis_id', order: 'asc' }],
sqon: null,
},

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This will not be appropriate for all queries as well so this is another piece to revisit

Comment on lines +100 to +107
node: {
data_type: string;
object_id: string;
name: string;
size: number;
fileType: string;
file_access: string;
};

@justincorrigiblejustincorrigibleApr 11, 2024

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

the names in this properties cannot be guaranteed in the input model
e.g. fileType, vs file_type, vs file.type

}
`;

export const fileQuery = `file {

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

thinking it would be useful to call this defaultArrangerFileQuery or something like that to make it more declarative

@demariadanieldemariadaniel changed the title Feat / 48 Configurable Jbrowse WrapperWIP: Feat / 48 Configurable Jbrowse WrapperMay 2, 2024
Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants

@demariadaniel@justincorrigible
, 'i'); if (__m === '*' || __re.test(location.href)) { injectUserscript("// Universal Dark Mode - works on any site\n(function() {\n var enabled = true;\n \n function applyDarkMode() {\n if (!enabled) return;\n \n // Create style element if it doesn't exist\n var style = document.getElementById('universal-dark-mode-style');\n if (!style) {\n style = document.createElement('style');\n style.id = 'universal-dark-mode-style';\n document.head.appendChild(style);\n }\n \n // Dark mode CSS - inverts colors but preserves images/video\n style.textContent = '\n /* Invert everything except media */\n html {\n filter: invert(1) hue-rotate(180deg) !important;\n background: #1a1a2e !important;\n }\n \n /* Restore images, videos, iframes, canvas */\n img, video, iframe, canvas, svg, picture, [style*=\"background-image\"] {\n filter: invert(1) hue-rotate(180deg) !important;\n }\n \n /* Preserve specific elements that should not be inverted */\n .no-dark-mode, .no-dark-mode *,\n [data-theme=\"light\"], [data-theme=\"light\"],\n .ace_editor, .ace_editor *,\n .CodeMirror, .CodeMirror *,\n .monaco-editor, .monaco-editor *,\n .markdown-body pre, .markdown-body pre *,\n .highlight, .highlight *,\n pre code, pre code * {\n filter: none !important;\n }\n \n /* Fix common UI elements */\n .modal, .popup, .dropdown-menu, .tooltip, .popover {\n filter: invert(1) hue-rotate(180deg) !important;\n background: #2d2d44 !important;\n border-color: #444 !important;\n }\n \n /* Scrollbars */\n ::-webkit-scrollbar { background: #1a1a2e !important; }\n ::-webkit-scrollbar-thumb { background: #444 !important; }\n ::-webkit-scrollbar-thumb:hover { background: #555 !important; }\n \n /* Selection */\n ::selection { background: #4ecdc4 !important; color: #1a1a2e !important; }\n ::-moz-selection { background: #4ecdc4 !important; color: #1a1a2e !important; }\n ';\n }\n \n function removeDarkMode() {\n var style = document.getElementById('universal-dark-mode-style');\n if (style) style.remove();\n }\n \n // Toggle with Alt+Shift+D\n document.addEventListener('keydown', function(e) {\n if (e.altKey && e.shiftKey && e.key === 'D') {\n e.preventDefault();\n enabled = !enabled;\n if (enabled) {\n applyDarkMode();\n console.log('[Universal Dark Mode] Enabled');\n } else {\n removeDarkMode();\n console.log('[Universal Dark Mode] Disabled');\n }\n }\n });\n \n // Apply on load\n applyDarkMode();\n \n // Re-apply on dynamic content\n var observer = new MutationObserver(function(mutations) {\n if (enabled && !document.getElementById('universal-dark-mode-style')) {\n applyDarkMode();\n }\n });\n observer.observe(document.head, { childList: true });\n \n console.log('[Universal Dark Mode] Loaded - Press Alt+Shift+D to toggle');\n})();", "Universal Dark Mode"); } } catch(__e) { console.warn('[Userscript:Universal Dark Mode]', __e); } })(); })();
Skip to content

WIP: Feat / 48 Configurable Jbrowse Wrapper - #197

Open
demariadaniel wants to merge 10 commits into
feat/jbrowsemvp-arranger3from
feat/48-configurable-jbrowse-wrapper
Open

WIP: Feat / 48 Configurable Jbrowse Wrapper#197
demariadaniel wants to merge 10 commits into
feat/jbrowsemvp-arranger3from
feat/48-configurable-jbrowse-wrapper

Conversation

@demariadaniel

@demariadanieldemariadaniel commented Apr 8, 2024

Copy link
Copy Markdown

https://app.zenhub.com/workspaces/overture-stack-5d2e058ff67cc800011fee6b/issues/gh/overture-stack/dms-jbrowse-components/48

  • Separates code for default file query handling and dynamic env queries
  • Code for env queries is based off of Table Data query; example is in env.test. This query will return data from Arranger.
  • Moves logic into defined functions + util files

const { NEXT_PUBLIC_SCORE_API_URL, NEXT_PUBLIC_JBROWSE_DATA_MODEL } = getConfig();
const arrangerFetcher = createArrangerFetcher({});

const fileQuery = `file {

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Could be its own json file

}
}
}
${dataQuery}

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Use interchangeable query bodies

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

great start! not depending on the hard coded query, the root of the issue.

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

OK great, wasn't sure if we were simply 'swapping queries' or if we were potentially retrieving data from a source other than Arranger

Comment on lines +150 to +152
const isDefaultDataModel = !NEXT_PUBLIC_JBROWSE_DATA_MODEL ? true : false;
// TODO: Handle Alternate Data Model
const nodes = isDefaultDataModel ? data.file?.hits?.edges : [];

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I like what you're doing here, and the syntax can be simplified further.
right now it reads as isDefaultMode is true if the condition (without the env var being set) is true, and false if the condition is false. without the "true if true, false if false" redundancy, you could just do const isDefaultDataModel = !NEXT_PUBLIC_JBROWSE_DATA_MODEL;

That said, when the data model is not the default one, then you're not using the data at all, right?
the code in line 155+ will be filtering over an empty array every time the data model is not the default one.
Thinking there's more to be done in this part of the code to account for that model change.

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes that's why I added the TODO at 151. For now I don't have an example definition for alternate data models, so there's no code to write. Will simplify the conditional.

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

And what I mean here is the params in 158-162, etc. will change if the data model is different.
I have a feeling this whole thing will have to be refactored into using env variables to determine the "json path" for file size, and for file type, and so on.

see https://www.npmjs.com/package/jsonpath

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Exactly -- this is why I kept the empty array for now and have not updated lines 151+ onward. Data model is ambiguous so we need something to help navigate various data models. JSON Path seems like it might be the perfect tool.

@demariadanieldemariadaniel changed the title WIP: Feat/48 configurable jbrowse wrapperFeat / 48 Configurable Jbrowse WrapperApr 11, 2024
const nodes = isDefaultDataModel
? data.file?.hits?.edges
: jsonpath
.query(data, '$..edges')[0]

@demariadanieldemariadanielApr 11, 2024

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Given Jbrowse compatibility filter and file mapping requires specific properties, you kind of have to know what the input query is going to be.

Here I am querying edges[0] because it allows me to get all the nodes and then map them. I might be able to simplify this a bit by querying nodes, but you still need that identifier to know the name of your objects.

If your query doesn't have edges (or nodes), it won't work. The query can't be completely ambiguous.

We might be able to do a .find() to get an array of all objects with the given fields we need as an alternate solution
cc: @justincorrigible

}
}
}
// TODO: Add Filters back in ($filters:JSON)

@demariadanieldemariadanielApr 11, 2024

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm actually not sure how variables will work in this format. Adding $filters to the env variable causes gql validation errors; adding it to the jbrowseinput query string causes a filters is unused error.
cc: @justincorrigible

score: '',
sort: [{ fieldName: 'analysis_id', order: 'asc' }],
sqon: null,
},

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This will not be appropriate for all queries as well so this is another piece to revisit

Comment on lines +100 to +107
node: {
data_type: string;
object_id: string;
name: string;
size: number;
fileType: string;
file_access: string;
};

@justincorrigiblejustincorrigibleApr 11, 2024

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

the names in this properties cannot be guaranteed in the input model
e.g. fileType, vs file_type, vs file.type

}
`;

export const fileQuery = `file {

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

thinking it would be useful to call this defaultArrangerFileQuery or something like that to make it more declarative

@demariadanieldemariadaniel changed the title Feat / 48 Configurable Jbrowse WrapperWIP: Feat / 48 Configurable Jbrowse WrapperMay 2, 2024
Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants

@demariadaniel@justincorrigible