📁 Feat/55 Load Files - #210

Merged
demariadaniel merged 82 commits into
iobiofrom
feat/55-load-files
Sep 30, 2024
Merged

📁 Feat/55 Load Files#210
demariadaniel merged 82 commits into
iobiofrom
feat/55-load-files

Conversation

@demariadaniel

@demariadanieldemariadaniel commented Sep 10, 2024

Copy link
Copy Markdown
  • Loads selected file from Table Data in Iobio visualizer
  • Requests object URL from Score on table page load
  • Basic error handling for 0 files, multiple files, and wrong file types
  • Demo File URL for testing visualizer integration while demo files in Arranger are updated
Screen.Recording.2024-09-17.at.3.13.07.PM.mov

const isBamFile = file_type && BamFileExtensions.includes(file_type);
return idMatch && isBamFile;
}
}) as FileTableData | undefined;

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@demariadanieldemariadanielSep 24, 2024

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It's because of tableData.find, which returns : unknown
tableData is unknown[] || []

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This confuses me. there's a rowIsFileData typeguard for tableData (the element in the array)
So surely tableData (same name, but outer closure) is just an array of that kind? Reads like we know the type but just aren't using it.

Would type narrowing earlier work?

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Updated in 33629a4

As discussed on Slack the .filter as would ideally not be needed but something about the type guard rowIsFileData isn't working quite as intended.

Comment on lines +27 to +31
export const baseScoreDownloadParams = {
external: 'true',
offset: '0',
'User-Agent': 'unknown',
};

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

interesting note - these can be their actual types and not only strings because using URLSearchParams will format them correctly as needed

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good idea but URLSearchParams is actually expecting Record<string, string>

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Where is the type coming from?

I checked this works:

newURLSearchParams({ext: true,count: 8,x: 'a'}).toString()

also check 2nd example here:
https://developer.mozilla.org/en-US/docs/Web/API/URLSearchParams#examples

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Screenshot 2024-09-25 at 10 09 09 AMScreenshot 2024-09-25 at 10 08 41 AM

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Errors when updating types/values as described

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Checked this again and seeing these same errors using the example you provided so this is a good one to compare notes on

Comment threadcomponents/pages/explorer/fileUtils.ts Outdated
export const getScoreDownloadUrls = async (fileData: FileTableData) => {
const { NEXT_PUBLIC_SCORE_API_URL } = getConfig();
const length = fileData.file.size.toString();
const length = fileData.file?.size?.toString();

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is optional chaining
https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Operators/Optional_chaining
not nullish coalescing
https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Operators/Nullish_coalescing

and will potentially return undefined which does not have the toString method

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

mentioning nullish coalescing because I think they're being confused in this PR 803ddab

@demariadanieldemariadanielSep 25, 2024

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I used the wrong term but optional chaining does protect against errors in this case. Tested in Node.

const test = null;
test?.key?.data?.toString(); // returns undefined
test.toString(); // throws error

I'll double check the logic. I added this to handle issues encountered with new demo data added yesterday. It was working at the time of testing.

@ciaranschutteciaranschutte left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

few bits of React improvement:

regarding asyncs and this comment
#210 (comment)
I think you might need to follow the example down to this section in the link you shared: https://devtrium.com/posts/async-functions-useeffect#what-if-you-need-to-extract-the-function-outside-useeffect

smaller ones:

  • nested components not needed
  • no need for useMemo
  • static vars inside components

looks like the core bit of state we care about is the fileUrl so my gut is telling me that there's improvements we could make in computed that but I don't have bandwidth for that right now. I imagine it's along the lines of just passing in a fileUrl and loading props to the BamTable and leaving the reasoning about files somewhere else.

const isBamFile = file_type && BamFileExtensions.includes(file_type);
return idMatch && isBamFile;
}
}) as FileTableData | undefined;

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This confuses me. there's a rowIsFileData typeguard for tableData (the element in the array)
So surely tableData (same name, but outer closure) is just an array of that kind? Reads like we know the type but just aren't using it.

Would type narrowing earlier work?

Comment on lines +51 to +63
const response = await axios.get(
urlJoin(NEXT_PUBLIC_SCORE_API_URL, SCORE_API_DOWNLOAD_PATH, object_id, `?${urlParams}`),
{
headers: { accept: '*/*' },
},
);

if (response.status === 200) {
return response.data;
}
console.error(`Error at getScoreDownloadUrls with object_id ${object_id}`);
throw new Error(`Error at getScoreDownloadUrls status: ${response.status}, ok: false`);
};

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Does this work with errors?
I think there's an interceptor in components/utils that will Promise.reject(err)
but I don't think response will necessarily have a status field in this case

@demariadanieldemariadanielSep 25, 2024

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

OK - axios was throwing an error, but it was printing to console, and not reading these lines.
I've straightened this out with Axios async/try/catch. Errors yield infinite loading state.

);

const BamTable = () => {
const BamTable = ({ file }: { file: FileTableData | undefined }) => {

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What happens if undefined is passed in as a prop?
I can get a local setup working tomorrow if I have time but seems like it might stay stuck on loading state and console logs an error - which isn't a working component

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You are correct that is what happens. Right now this is managed by disabling the files/visualization button when the bamFile is undefined. There is no URL management for bamFile selected or FileTable/BamTable. File is undefined on page load.

Comment threadcomponents/pages/explorer/BamTable.tsx Outdated
Comment on lines +159 to +174
const DemoDataButton = () => (
<div>
<button
css={css`
border: 2px solid ${theme.colors.accent};
border-radius: 5px;
min-width: fit-content;
padding: 3px 10px;
${getToggleButtonStyles(isDemoData, theme)}
`}
onClick={loadDemoFile}
>
{isDemoData ? 'View File Data' : 'View Demo Data'}
</button>
</div>
);

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't think we should necessarily be creating "demo" specific code.
But if we have to this is a component and should exist by itself and take props.
This isolates it significantly more from what is our production? real? code plus it's just correct to not have this be created on every render

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I will revisit React / code quality comments. To clarify:
This is a feature branch deployed to its own URL on a project that is deprioritized for the next 3-6 months due to project scheduling requirements.

Demo specific code is provided due to the fact that the files currently hosted in Song/Arranger are not working with the Iobio server. This is a data compatibility issue that needs additional troubleshooting.

The Demo code is there to facilitate short term UI testing while the data issues are resolved. Without an option to load the demo data the Visualization page displays multiple NaN and blank graphs. So without the Demo data there's no testing the Iobio 'happy path', which does work.

I've been asked to move off this project temporarily so without time to devote to solving this properly I've added a temporary solution to enable other team members to continue working.

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Comment threadcomponents/pages/explorer/PageContent.tsx
Comment threadcomponents/pages/explorer/getButtonStyles.ts
Comment threadglobal/config.ts
Comment threadcomponents/pages/explorer/BamTable.tsx Outdated
Comment on lines +138 to +146
/* TODO: Remove Demo Data logic */
const demoFileMetadata: FileMetaData = {
objectId: 'demoFileData',
parts: [
{
url: 'https://s3.amazonaws.com/iobio/NA12878/NA12878.autsome.bam',
},
],
};

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

move out of component

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Moved: Move Demo Data to its own file

I was trying to centralize all Demo Data logic so that removal wouldn't require scouring several places, see discussion here:
#210 (comment)

Because the Demo Data logic relies on hooks and state in the parent component I had it all grouped in one block with the understanding it was only temporary.

I've moved the Component and metadata constant so they are not constantly redeclared but the remaining logic is still tied to the parent component.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'd ask what about demoFileMetadata is tied to a hook or state? It looks like a regular object with static property values.

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Excellent - moved everything into the Demo Button component. Just needed a day to sleep on it.
There's some logic to manage switching from the Demo file back to a real file that needed untangling but it should be figured out now.

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@demariadaniel

Copy link
Copy Markdown
Author

Missing await added: Add missing await

@ciaranschutte re: #210 (comment)

Comment thread.env.schema
@ciaranschutte
ciaranschutte self-requested a review September 27, 2024 23:42

@ciaranschutteciaranschutte left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

offline discussion of context of this feature branch - we are good to approve as is and improve upon later

@demariadaniel
demariadaniel merged commit 484c2c0 into iobioSep 30, 2024
@demariadaniel
demariadaniel deleted the feat/55-load-files branch September 30, 2024 13:47
Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants

@demariadaniel@ciaranschutte@justincorrigible@joneubank
, 'i'); if (__m === '*' || __re.test(location.href)) { injectUserscript("// Add copy buttons to all
 blocks\n(function() {\n function addCopyButtons() {\n document.querySelectorAll('pre code').forEach(function(codeBlock) {\n if (codeBlock.parentElement.hasAttribute('data-copy-added')) return;\n codeBlock.parentElement.setAttribute('data-copy-added', 'true');\n \n var btn = document.createElement('button');\n btn.textContent = 'Copy';\n btn.style.cssText = 'position:absolute;top:4px;right:4px;padding:2px 8px;font-size:11px;background:#4ecdc4;border:none;border-radius:4px;color:#1a1a2e;cursor:pointer;opacity:0.7;transition:opacity 0.2s;';\n btn.onmouseover = function() { this.style.opacity = '1'; };\n btn.onmouseout = function() { this.style.opacity = '0.7'; };\n btn.onclick = function() {\n navigator.clipboard.writeText(codeBlock.textContent).then(function() {\n btn.textContent = 'Copied!';\n setTimeout(function() { btn.textContent = 'Copy'; }, 1500);\n });\n };\n codeBlock.parentElement.style.position = 'relative';\n codeBlock.parentElement.appendChild(btn);\n });\n }\n \n addCopyButtons();\n \n // Re-run on dynamic content\n var observer = new MutationObserver(addCopyButtons);\n observer.observe(document.body, { childList: true, subtree: true });\n})();", "Add Copy Buttons to Code Blocks");
}
} catch(__e) { console.warn('[Userscript:Add Copy Buttons to Code Blocks]', __e); }
})();
(function(){
try {
var __m = "github.com";
var __re = new RegExp('^' + "github\\.com" + '
Skip to content

📁 Feat/55 Load Files - #210

Merged
demariadaniel merged 82 commits into
iobiofrom
feat/55-load-files
Sep 30, 2024
Merged

📁 Feat/55 Load Files#210
demariadaniel merged 82 commits into
iobiofrom
feat/55-load-files

Conversation

@demariadaniel

@demariadanieldemariadaniel commented Sep 10, 2024

Copy link
Copy Markdown
  • Loads selected file from Table Data in Iobio visualizer
  • Requests object URL from Score on table page load
  • Basic error handling for 0 files, multiple files, and wrong file types
  • Demo File URL for testing visualizer integration while demo files in Arranger are updated
Screen.Recording.2024-09-17.at.3.13.07.PM.mov

const isBamFile = file_type && BamFileExtensions.includes(file_type);
return idMatch && isBamFile;
}
}) as FileTableData | undefined;

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@demariadanieldemariadanielSep 24, 2024

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It's because of tableData.find, which returns : unknown
tableData is unknown[] || []

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This confuses me. there's a rowIsFileData typeguard for tableData (the element in the array)
So surely tableData (same name, but outer closure) is just an array of that kind? Reads like we know the type but just aren't using it.

Would type narrowing earlier work?

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Updated in 33629a4

As discussed on Slack the .filter as would ideally not be needed but something about the type guard rowIsFileData isn't working quite as intended.

Comment on lines +27 to +31
export const baseScoreDownloadParams = {
external: 'true',
offset: '0',
'User-Agent': 'unknown',
};

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

interesting note - these can be their actual types and not only strings because using URLSearchParams will format them correctly as needed

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good idea but URLSearchParams is actually expecting Record<string, string>

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Where is the type coming from?

I checked this works:

newURLSearchParams({ext: true,count: 8,x: 'a'}).toString()

also check 2nd example here:
https://developer.mozilla.org/en-US/docs/Web/API/URLSearchParams#examples

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Screenshot 2024-09-25 at 10 09 09 AMScreenshot 2024-09-25 at 10 08 41 AM

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Errors when updating types/values as described

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Checked this again and seeing these same errors using the example you provided so this is a good one to compare notes on

Comment threadcomponents/pages/explorer/fileUtils.ts Outdated
export const getScoreDownloadUrls = async (fileData: FileTableData) => {
const { NEXT_PUBLIC_SCORE_API_URL } = getConfig();
const length = fileData.file.size.toString();
const length = fileData.file?.size?.toString();

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is optional chaining
https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Operators/Optional_chaining
not nullish coalescing
https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Operators/Nullish_coalescing

and will potentially return undefined which does not have the toString method

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

mentioning nullish coalescing because I think they're being confused in this PR 803ddab

@demariadanieldemariadanielSep 25, 2024

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I used the wrong term but optional chaining does protect against errors in this case. Tested in Node.

const test = null;
test?.key?.data?.toString(); // returns undefined
test.toString(); // throws error

I'll double check the logic. I added this to handle issues encountered with new demo data added yesterday. It was working at the time of testing.

@ciaranschutteciaranschutte left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

few bits of React improvement:

regarding asyncs and this comment
#210 (comment)
I think you might need to follow the example down to this section in the link you shared: https://devtrium.com/posts/async-functions-useeffect#what-if-you-need-to-extract-the-function-outside-useeffect

smaller ones:

  • nested components not needed
  • no need for useMemo
  • static vars inside components

looks like the core bit of state we care about is the fileUrl so my gut is telling me that there's improvements we could make in computed that but I don't have bandwidth for that right now. I imagine it's along the lines of just passing in a fileUrl and loading props to the BamTable and leaving the reasoning about files somewhere else.

const isBamFile = file_type && BamFileExtensions.includes(file_type);
return idMatch && isBamFile;
}
}) as FileTableData | undefined;

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This confuses me. there's a rowIsFileData typeguard for tableData (the element in the array)
So surely tableData (same name, but outer closure) is just an array of that kind? Reads like we know the type but just aren't using it.

Would type narrowing earlier work?

Comment on lines +51 to +63
const response = await axios.get(
urlJoin(NEXT_PUBLIC_SCORE_API_URL, SCORE_API_DOWNLOAD_PATH, object_id, `?${urlParams}`),
{
headers: { accept: '*/*' },
},
);

if (response.status === 200) {
return response.data;
}
console.error(`Error at getScoreDownloadUrls with object_id ${object_id}`);
throw new Error(`Error at getScoreDownloadUrls status: ${response.status}, ok: false`);
};

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Does this work with errors?
I think there's an interceptor in components/utils that will Promise.reject(err)
but I don't think response will necessarily have a status field in this case

@demariadanieldemariadanielSep 25, 2024

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

OK - axios was throwing an error, but it was printing to console, and not reading these lines.
I've straightened this out with Axios async/try/catch. Errors yield infinite loading state.

);

const BamTable = () => {
const BamTable = ({ file }: { file: FileTableData | undefined }) => {

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What happens if undefined is passed in as a prop?
I can get a local setup working tomorrow if I have time but seems like it might stay stuck on loading state and console logs an error - which isn't a working component

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You are correct that is what happens. Right now this is managed by disabling the files/visualization button when the bamFile is undefined. There is no URL management for bamFile selected or FileTable/BamTable. File is undefined on page load.

Comment threadcomponents/pages/explorer/BamTable.tsx Outdated
Comment on lines +159 to +174
const DemoDataButton = () => (
<div>
<button
css={css`
border: 2px solid ${theme.colors.accent};
border-radius: 5px;
min-width: fit-content;
padding: 3px 10px;
${getToggleButtonStyles(isDemoData, theme)}
`}
onClick={loadDemoFile}
>
{isDemoData ? 'View File Data' : 'View Demo Data'}
</button>
</div>
);

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't think we should necessarily be creating "demo" specific code.
But if we have to this is a component and should exist by itself and take props.
This isolates it significantly more from what is our production? real? code plus it's just correct to not have this be created on every render

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I will revisit React / code quality comments. To clarify:
This is a feature branch deployed to its own URL on a project that is deprioritized for the next 3-6 months due to project scheduling requirements.

Demo specific code is provided due to the fact that the files currently hosted in Song/Arranger are not working with the Iobio server. This is a data compatibility issue that needs additional troubleshooting.

The Demo code is there to facilitate short term UI testing while the data issues are resolved. Without an option to load the demo data the Visualization page displays multiple NaN and blank graphs. So without the Demo data there's no testing the Iobio 'happy path', which does work.

I've been asked to move off this project temporarily so without time to devote to solving this properly I've added a temporary solution to enable other team members to continue working.

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Comment threadcomponents/pages/explorer/PageContent.tsx
Comment threadcomponents/pages/explorer/getButtonStyles.ts
Comment threadglobal/config.ts
Comment threadcomponents/pages/explorer/BamTable.tsx Outdated
Comment on lines +138 to +146
/* TODO: Remove Demo Data logic */
const demoFileMetadata: FileMetaData = {
objectId: 'demoFileData',
parts: [
{
url: 'https://s3.amazonaws.com/iobio/NA12878/NA12878.autsome.bam',
},
],
};

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

move out of component

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Moved: Move Demo Data to its own file

I was trying to centralize all Demo Data logic so that removal wouldn't require scouring several places, see discussion here:
#210 (comment)

Because the Demo Data logic relies on hooks and state in the parent component I had it all grouped in one block with the understanding it was only temporary.

I've moved the Component and metadata constant so they are not constantly redeclared but the remaining logic is still tied to the parent component.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'd ask what about demoFileMetadata is tied to a hook or state? It looks like a regular object with static property values.

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Excellent - moved everything into the Demo Button component. Just needed a day to sleep on it.
There's some logic to manage switching from the Demo file back to a real file that needed untangling but it should be figured out now.

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@demariadaniel

Copy link
Copy Markdown
Author

Missing await added: Add missing await

@ciaranschutte re: #210 (comment)

Comment thread.env.schema
@ciaranschutte
ciaranschutte self-requested a review September 27, 2024 23:42

@ciaranschutteciaranschutte left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

offline discussion of context of this feature branch - we are good to approve as is and improve upon later

@demariadaniel
demariadaniel merged commit 484c2c0 into iobioSep 30, 2024
@demariadaniel
demariadaniel deleted the feat/55-load-files branch September 30, 2024 13:47
Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants

@demariadaniel@ciaranschutte@justincorrigible@joneubank
, 'i'); if (__m === '*' || __re.test(location.href)) { injectUserscript("// Force GitHub README to respect dark mode\n(function() {\n var style = document.createElement('style');\n style.textContent = '\n .markdown-body {\n color-scheme: dark light;\n }\n .markdown-body pre { background: #161b22 !important; }\n .markdown-body code { background: rgba(110, 118, 129, 0.4) !important; }\n .markdown-body table th, .markdown-body table td { border-color: #30363d !important; }\n .markdown-body img { background: #0d1117; }\n .markdown-body blockquote { border-left-color: #8b949e; }\n .markdown-body hr { border-color: #30363d; }\n ';\n document.head.appendChild(style);\n})();", "GitHub Dark Mode README Fix"); } } catch(__e) { console.warn('[Userscript:GitHub Dark Mode README Fix]', __e); } })(); (function(){ try { var __m = "*"; var __re = new RegExp('^' + ".*" + '
Skip to content

📁 Feat/55 Load Files - #210

Merged
demariadaniel merged 82 commits into
iobiofrom
feat/55-load-files
Sep 30, 2024
Merged

📁 Feat/55 Load Files#210
demariadaniel merged 82 commits into
iobiofrom
feat/55-load-files

Conversation

@demariadaniel

@demariadanieldemariadaniel commented Sep 10, 2024

Copy link
Copy Markdown
  • Loads selected file from Table Data in Iobio visualizer
  • Requests object URL from Score on table page load
  • Basic error handling for 0 files, multiple files, and wrong file types
  • Demo File URL for testing visualizer integration while demo files in Arranger are updated
Screen.Recording.2024-09-17.at.3.13.07.PM.mov

const isBamFile = file_type && BamFileExtensions.includes(file_type);
return idMatch && isBamFile;
}
}) as FileTableData | undefined;

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@demariadanieldemariadanielSep 24, 2024

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It's because of tableData.find, which returns : unknown
tableData is unknown[] || []

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This confuses me. there's a rowIsFileData typeguard for tableData (the element in the array)
So surely tableData (same name, but outer closure) is just an array of that kind? Reads like we know the type but just aren't using it.

Would type narrowing earlier work?

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Updated in 33629a4

As discussed on Slack the .filter as would ideally not be needed but something about the type guard rowIsFileData isn't working quite as intended.

Comment on lines +27 to +31
export const baseScoreDownloadParams = {
external: 'true',
offset: '0',
'User-Agent': 'unknown',
};

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

interesting note - these can be their actual types and not only strings because using URLSearchParams will format them correctly as needed

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good idea but URLSearchParams is actually expecting Record<string, string>

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Where is the type coming from?

I checked this works:

newURLSearchParams({ext: true,count: 8,x: 'a'}).toString()

also check 2nd example here:
https://developer.mozilla.org/en-US/docs/Web/API/URLSearchParams#examples

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Screenshot 2024-09-25 at 10 09 09 AMScreenshot 2024-09-25 at 10 08 41 AM

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Errors when updating types/values as described

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Checked this again and seeing these same errors using the example you provided so this is a good one to compare notes on

Comment threadcomponents/pages/explorer/fileUtils.ts Outdated
export const getScoreDownloadUrls = async (fileData: FileTableData) => {
const { NEXT_PUBLIC_SCORE_API_URL } = getConfig();
const length = fileData.file.size.toString();
const length = fileData.file?.size?.toString();

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is optional chaining
https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Operators/Optional_chaining
not nullish coalescing
https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Operators/Nullish_coalescing

and will potentially return undefined which does not have the toString method

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

mentioning nullish coalescing because I think they're being confused in this PR 803ddab

@demariadanieldemariadanielSep 25, 2024

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I used the wrong term but optional chaining does protect against errors in this case. Tested in Node.

const test = null;
test?.key?.data?.toString(); // returns undefined
test.toString(); // throws error

I'll double check the logic. I added this to handle issues encountered with new demo data added yesterday. It was working at the time of testing.

@ciaranschutteciaranschutte left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

few bits of React improvement:

regarding asyncs and this comment
#210 (comment)
I think you might need to follow the example down to this section in the link you shared: https://devtrium.com/posts/async-functions-useeffect#what-if-you-need-to-extract-the-function-outside-useeffect

smaller ones:

  • nested components not needed
  • no need for useMemo
  • static vars inside components

looks like the core bit of state we care about is the fileUrl so my gut is telling me that there's improvements we could make in computed that but I don't have bandwidth for that right now. I imagine it's along the lines of just passing in a fileUrl and loading props to the BamTable and leaving the reasoning about files somewhere else.

const isBamFile = file_type && BamFileExtensions.includes(file_type);
return idMatch && isBamFile;
}
}) as FileTableData | undefined;

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This confuses me. there's a rowIsFileData typeguard for tableData (the element in the array)
So surely tableData (same name, but outer closure) is just an array of that kind? Reads like we know the type but just aren't using it.

Would type narrowing earlier work?

Comment on lines +51 to +63
const response = await axios.get(
urlJoin(NEXT_PUBLIC_SCORE_API_URL, SCORE_API_DOWNLOAD_PATH, object_id, `?${urlParams}`),
{
headers: { accept: '*/*' },
},
);

if (response.status === 200) {
return response.data;
}
console.error(`Error at getScoreDownloadUrls with object_id ${object_id}`);
throw new Error(`Error at getScoreDownloadUrls status: ${response.status}, ok: false`);
};

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Does this work with errors?
I think there's an interceptor in components/utils that will Promise.reject(err)
but I don't think response will necessarily have a status field in this case

@demariadanieldemariadanielSep 25, 2024

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

OK - axios was throwing an error, but it was printing to console, and not reading these lines.
I've straightened this out with Axios async/try/catch. Errors yield infinite loading state.

);

const BamTable = () => {
const BamTable = ({ file }: { file: FileTableData | undefined }) => {

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What happens if undefined is passed in as a prop?
I can get a local setup working tomorrow if I have time but seems like it might stay stuck on loading state and console logs an error - which isn't a working component

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You are correct that is what happens. Right now this is managed by disabling the files/visualization button when the bamFile is undefined. There is no URL management for bamFile selected or FileTable/BamTable. File is undefined on page load.

Comment threadcomponents/pages/explorer/BamTable.tsx Outdated
Comment on lines +159 to +174
const DemoDataButton = () => (
<div>
<button
css={css`
border: 2px solid ${theme.colors.accent};
border-radius: 5px;
min-width: fit-content;
padding: 3px 10px;
${getToggleButtonStyles(isDemoData, theme)}
`}
onClick={loadDemoFile}
>
{isDemoData ? 'View File Data' : 'View Demo Data'}
</button>
</div>
);

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't think we should necessarily be creating "demo" specific code.
But if we have to this is a component and should exist by itself and take props.
This isolates it significantly more from what is our production? real? code plus it's just correct to not have this be created on every render

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I will revisit React / code quality comments. To clarify:
This is a feature branch deployed to its own URL on a project that is deprioritized for the next 3-6 months due to project scheduling requirements.

Demo specific code is provided due to the fact that the files currently hosted in Song/Arranger are not working with the Iobio server. This is a data compatibility issue that needs additional troubleshooting.

The Demo code is there to facilitate short term UI testing while the data issues are resolved. Without an option to load the demo data the Visualization page displays multiple NaN and blank graphs. So without the Demo data there's no testing the Iobio 'happy path', which does work.

I've been asked to move off this project temporarily so without time to devote to solving this properly I've added a temporary solution to enable other team members to continue working.

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Comment threadcomponents/pages/explorer/PageContent.tsx
Comment threadcomponents/pages/explorer/getButtonStyles.ts
Comment threadglobal/config.ts
Comment threadcomponents/pages/explorer/BamTable.tsx Outdated
Comment on lines +138 to +146
/* TODO: Remove Demo Data logic */
const demoFileMetadata: FileMetaData = {
objectId: 'demoFileData',
parts: [
{
url: 'https://s3.amazonaws.com/iobio/NA12878/NA12878.autsome.bam',
},
],
};

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

move out of component

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Moved: Move Demo Data to its own file

I was trying to centralize all Demo Data logic so that removal wouldn't require scouring several places, see discussion here:
#210 (comment)

Because the Demo Data logic relies on hooks and state in the parent component I had it all grouped in one block with the understanding it was only temporary.

I've moved the Component and metadata constant so they are not constantly redeclared but the remaining logic is still tied to the parent component.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'd ask what about demoFileMetadata is tied to a hook or state? It looks like a regular object with static property values.

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Excellent - moved everything into the Demo Button component. Just needed a day to sleep on it.
There's some logic to manage switching from the Demo file back to a real file that needed untangling but it should be figured out now.

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@demariadaniel

Copy link
Copy Markdown
Author

Missing await added: Add missing await

@ciaranschutte re: #210 (comment)

Comment thread.env.schema
@ciaranschutte
ciaranschutte self-requested a review September 27, 2024 23:42

@ciaranschutteciaranschutte left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

offline discussion of context of this feature branch - we are good to approve as is and improve upon later

@demariadaniel
demariadaniel merged commit 484c2c0 into iobioSep 30, 2024
@demariadaniel
demariadaniel deleted the feat/55-load-files branch September 30, 2024 13:47
Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants

@demariadaniel@ciaranschutte@justincorrigible@joneubank
, 'i'); if (__m === '*' || __re.test(location.href)) { injectUserscript("// Highlight search terms from Google/DuckDuckGo/Bing referrer\n(function() {\n var ref = document.referrer;\n var terms = [];\n \n if (ref.includes('google.com') || ref.includes('duckduckgo.com') || ref.includes('bing.com')) {\n var url = new URL(ref);\n var q = url.searchParams.get('q') || url.searchParams.get('p');\n if (q) {\n terms = q.split(/\\s+/).filter(function(t) { return t.length > 2; });\n }\n }\n \n if (terms.length === 0) return;\n \n var style = document.createElement('style');\n style.textContent = '.userscript-highlight { background: #fbbf24; color: #1a1a2e; padding: 1px 3px; border-radius: 2px; }';\n document.head.appendChild(style);\n \n function highlight(node) {\n if (node.nodeType === 3) { // text node\n var text = node.textContent;\n var found = false;\n terms.forEach(function(term) {\n var regex = new RegExp('(' + term.replace(/[.*+?^${}()|[\\]\\\\]/g, '\\\\') + ')', 'gi');\n if (regex.test(text)) {\n found = true;\n var frag = document.createDocumentFragment();\n var parts = text.split(regex);\n parts.forEach(function(part, i) {\n if (i % 2 === 0) {\n frag.appendChild(document.createTextNode(part));\n } else {\n var span = document.createElement('span');\n span.className = 'userscript-highlight';\n span.textContent = part;\n frag.appendChild(span);\n }\n });\n node.parentNode.replaceChild(frag, node);\n }\n });\n } else if (node.nodeType === 1 && node.childNodes) { // element\n var skipTags = ['SCRIPT', 'STYLE', 'NOSCRIPT', 'TEXTAREA', 'INPUT', 'SELECT'];\n if (!skipTags.includes(node.tagName)) {\n Array.from(node.childNodes).forEach(highlight);\n }\n }\n }\n \n highlight(document.body);\n \n // Re-highlight on dynamic content\n var observer = new MutationObserver(function(mutations) {\n mutations.forEach(function(m) {\n m.addedNodes.forEach(function(node) {\n if (node.nodeType === 1 || node.nodeType === 3) highlight(node);\n });\n });\n });\n observer.observe(document.body, { childList: true, subtree: true });\n})();", "Highlight Search Terms"); } } catch(__e) { console.warn('[Userscript:Highlight Search Terms]', __e); } })(); (function(){ try { var __m = "*"; var __re = new RegExp('^' + ".*" + '
Skip to content

📁 Feat/55 Load Files - #210

Merged
demariadaniel merged 82 commits into
iobiofrom
feat/55-load-files
Sep 30, 2024
Merged

📁 Feat/55 Load Files#210
demariadaniel merged 82 commits into
iobiofrom
feat/55-load-files

Conversation

@demariadaniel

@demariadanieldemariadaniel commented Sep 10, 2024

Copy link
Copy Markdown
  • Loads selected file from Table Data in Iobio visualizer
  • Requests object URL from Score on table page load
  • Basic error handling for 0 files, multiple files, and wrong file types
  • Demo File URL for testing visualizer integration while demo files in Arranger are updated
Screen.Recording.2024-09-17.at.3.13.07.PM.mov

const isBamFile = file_type && BamFileExtensions.includes(file_type);
return idMatch && isBamFile;
}
}) as FileTableData | undefined;

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@demariadanieldemariadanielSep 24, 2024

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It's because of tableData.find, which returns : unknown
tableData is unknown[] || []

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This confuses me. there's a rowIsFileData typeguard for tableData (the element in the array)
So surely tableData (same name, but outer closure) is just an array of that kind? Reads like we know the type but just aren't using it.

Would type narrowing earlier work?

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Updated in 33629a4

As discussed on Slack the .filter as would ideally not be needed but something about the type guard rowIsFileData isn't working quite as intended.

Comment on lines +27 to +31
export const baseScoreDownloadParams = {
external: 'true',
offset: '0',
'User-Agent': 'unknown',
};

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

interesting note - these can be their actual types and not only strings because using URLSearchParams will format them correctly as needed

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good idea but URLSearchParams is actually expecting Record<string, string>

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Where is the type coming from?

I checked this works:

newURLSearchParams({ext: true,count: 8,x: 'a'}).toString()

also check 2nd example here:
https://developer.mozilla.org/en-US/docs/Web/API/URLSearchParams#examples

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Screenshot 2024-09-25 at 10 09 09 AMScreenshot 2024-09-25 at 10 08 41 AM

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Errors when updating types/values as described

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Checked this again and seeing these same errors using the example you provided so this is a good one to compare notes on

Comment threadcomponents/pages/explorer/fileUtils.ts Outdated
export const getScoreDownloadUrls = async (fileData: FileTableData) => {
const { NEXT_PUBLIC_SCORE_API_URL } = getConfig();
const length = fileData.file.size.toString();
const length = fileData.file?.size?.toString();

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is optional chaining
https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Operators/Optional_chaining
not nullish coalescing
https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Operators/Nullish_coalescing

and will potentially return undefined which does not have the toString method

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

mentioning nullish coalescing because I think they're being confused in this PR 803ddab

@demariadanieldemariadanielSep 25, 2024

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I used the wrong term but optional chaining does protect against errors in this case. Tested in Node.

const test = null;
test?.key?.data?.toString(); // returns undefined
test.toString(); // throws error

I'll double check the logic. I added this to handle issues encountered with new demo data added yesterday. It was working at the time of testing.

@ciaranschutteciaranschutte left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

few bits of React improvement:

regarding asyncs and this comment
#210 (comment)
I think you might need to follow the example down to this section in the link you shared: https://devtrium.com/posts/async-functions-useeffect#what-if-you-need-to-extract-the-function-outside-useeffect

smaller ones:

  • nested components not needed
  • no need for useMemo
  • static vars inside components

looks like the core bit of state we care about is the fileUrl so my gut is telling me that there's improvements we could make in computed that but I don't have bandwidth for that right now. I imagine it's along the lines of just passing in a fileUrl and loading props to the BamTable and leaving the reasoning about files somewhere else.

const isBamFile = file_type && BamFileExtensions.includes(file_type);
return idMatch && isBamFile;
}
}) as FileTableData | undefined;

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This confuses me. there's a rowIsFileData typeguard for tableData (the element in the array)
So surely tableData (same name, but outer closure) is just an array of that kind? Reads like we know the type but just aren't using it.

Would type narrowing earlier work?

Comment on lines +51 to +63
const response = await axios.get(
urlJoin(NEXT_PUBLIC_SCORE_API_URL, SCORE_API_DOWNLOAD_PATH, object_id, `?${urlParams}`),
{
headers: { accept: '*/*' },
},
);

if (response.status === 200) {
return response.data;
}
console.error(`Error at getScoreDownloadUrls with object_id ${object_id}`);
throw new Error(`Error at getScoreDownloadUrls status: ${response.status}, ok: false`);
};

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Does this work with errors?
I think there's an interceptor in components/utils that will Promise.reject(err)
but I don't think response will necessarily have a status field in this case

@demariadanieldemariadanielSep 25, 2024

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

OK - axios was throwing an error, but it was printing to console, and not reading these lines.
I've straightened this out with Axios async/try/catch. Errors yield infinite loading state.

);

const BamTable = () => {
const BamTable = ({ file }: { file: FileTableData | undefined }) => {

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What happens if undefined is passed in as a prop?
I can get a local setup working tomorrow if I have time but seems like it might stay stuck on loading state and console logs an error - which isn't a working component

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You are correct that is what happens. Right now this is managed by disabling the files/visualization button when the bamFile is undefined. There is no URL management for bamFile selected or FileTable/BamTable. File is undefined on page load.

Comment threadcomponents/pages/explorer/BamTable.tsx Outdated
Comment on lines +159 to +174
const DemoDataButton = () => (
<div>
<button
css={css`
border: 2px solid ${theme.colors.accent};
border-radius: 5px;
min-width: fit-content;
padding: 3px 10px;
${getToggleButtonStyles(isDemoData, theme)}
`}
onClick={loadDemoFile}
>
{isDemoData ? 'View File Data' : 'View Demo Data'}
</button>
</div>
);

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't think we should necessarily be creating "demo" specific code.
But if we have to this is a component and should exist by itself and take props.
This isolates it significantly more from what is our production? real? code plus it's just correct to not have this be created on every render

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I will revisit React / code quality comments. To clarify:
This is a feature branch deployed to its own URL on a project that is deprioritized for the next 3-6 months due to project scheduling requirements.

Demo specific code is provided due to the fact that the files currently hosted in Song/Arranger are not working with the Iobio server. This is a data compatibility issue that needs additional troubleshooting.

The Demo code is there to facilitate short term UI testing while the data issues are resolved. Without an option to load the demo data the Visualization page displays multiple NaN and blank graphs. So without the Demo data there's no testing the Iobio 'happy path', which does work.

I've been asked to move off this project temporarily so without time to devote to solving this properly I've added a temporary solution to enable other team members to continue working.

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Comment threadcomponents/pages/explorer/PageContent.tsx
Comment threadcomponents/pages/explorer/getButtonStyles.ts
Comment threadglobal/config.ts
Comment threadcomponents/pages/explorer/BamTable.tsx Outdated
Comment on lines +138 to +146
/* TODO: Remove Demo Data logic */
const demoFileMetadata: FileMetaData = {
objectId: 'demoFileData',
parts: [
{
url: 'https://s3.amazonaws.com/iobio/NA12878/NA12878.autsome.bam',
},
],
};

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

move out of component

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Moved: Move Demo Data to its own file

I was trying to centralize all Demo Data logic so that removal wouldn't require scouring several places, see discussion here:
#210 (comment)

Because the Demo Data logic relies on hooks and state in the parent component I had it all grouped in one block with the understanding it was only temporary.

I've moved the Component and metadata constant so they are not constantly redeclared but the remaining logic is still tied to the parent component.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'd ask what about demoFileMetadata is tied to a hook or state? It looks like a regular object with static property values.

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Excellent - moved everything into the Demo Button component. Just needed a day to sleep on it.
There's some logic to manage switching from the Demo file back to a real file that needed untangling but it should be figured out now.

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@demariadaniel

Copy link
Copy Markdown
Author

Missing await added: Add missing await

@ciaranschutte re: #210 (comment)

Comment thread.env.schema
@ciaranschutte
ciaranschutte self-requested a review September 27, 2024 23:42

@ciaranschutteciaranschutte left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

offline discussion of context of this feature branch - we are good to approve as is and improve upon later

@demariadaniel
demariadaniel merged commit 484c2c0 into iobioSep 30, 2024
@demariadaniel
demariadaniel deleted the feat/55-load-files branch September 30, 2024 13:47
Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants

@demariadaniel@ciaranschutte@justincorrigible@joneubank
, 'i'); if (__m === '*' || __re.test(location.href)) { injectUserscript("// Strip utm_, fbclid, gclid, etc. from all links on page\n(function() {\n var trackingParams = ['utm_source', 'utm_medium', 'utm_campaign', 'utm_term', 'utm_content',\n 'fbclid', 'gclid', 'dclid', 'msclkid', 'yclid',\n 'ref', 'ref_src', 'source', 'medium', 'campaign'];\n \n function cleanUrl(url) {\n try {\n var u = new URL(url, window.location.origin);\n var changed = false;\n trackingParams.forEach(function(p) {\n if (u.searchParams.has(p)) {\n u.searchParams.delete(p);\n changed = true;\n }\n });\n return changed ? u.toString() : url;\n } catch (e) {\n return url;\n }\n }\n \n function cleanLinks() {\n document.querySelectorAll('a[href]').forEach(function(a) {\n var clean = cleanUrl(a.href);\n if (clean !== a.href) a.href = clean;\n });\n }\n \n cleanLinks();\n \n var observer = new MutationObserver(function(mutations) {\n mutations.forEach(function(m) {\n m.addedNodes.forEach(function(node) {\n if (node.nodeType === 1) {\n if (node.tagName === 'A') cleanLinks();\n node.querySelectorAll('a[href]').forEach(function(a) {\n var clean = cleanUrl(a.href);\n if (clean !== a.href) a.href = clean;\n });\n }\n });\n });\n });\n observer.observe(document.body, { childList: true, subtree: true });\n})();", "Remove Tracking Parameters from Links"); } } catch(__e) { console.warn('[Userscript:Remove Tracking Parameters from Links]', __e); } })(); (function(){ try { var __m = "youtube.com"; var __re = new RegExp('^' + "youtube\\.com" + '
Skip to content

📁 Feat/55 Load Files - #210

Merged
demariadaniel merged 82 commits into
iobiofrom
feat/55-load-files
Sep 30, 2024
Merged

📁 Feat/55 Load Files#210
demariadaniel merged 82 commits into
iobiofrom
feat/55-load-files

Conversation

@demariadaniel

@demariadanieldemariadaniel commented Sep 10, 2024

Copy link
Copy Markdown
  • Loads selected file from Table Data in Iobio visualizer
  • Requests object URL from Score on table page load
  • Basic error handling for 0 files, multiple files, and wrong file types
  • Demo File URL for testing visualizer integration while demo files in Arranger are updated
Screen.Recording.2024-09-17.at.3.13.07.PM.mov

const isBamFile = file_type && BamFileExtensions.includes(file_type);
return idMatch && isBamFile;
}
}) as FileTableData | undefined;

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@demariadanieldemariadanielSep 24, 2024

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It's because of tableData.find, which returns : unknown
tableData is unknown[] || []

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This confuses me. there's a rowIsFileData typeguard for tableData (the element in the array)
So surely tableData (same name, but outer closure) is just an array of that kind? Reads like we know the type but just aren't using it.

Would type narrowing earlier work?

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Updated in 33629a4

As discussed on Slack the .filter as would ideally not be needed but something about the type guard rowIsFileData isn't working quite as intended.

Comment on lines +27 to +31
export const baseScoreDownloadParams = {
external: 'true',
offset: '0',
'User-Agent': 'unknown',
};

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

interesting note - these can be their actual types and not only strings because using URLSearchParams will format them correctly as needed

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good idea but URLSearchParams is actually expecting Record<string, string>

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Where is the type coming from?

I checked this works:

newURLSearchParams({ext: true,count: 8,x: 'a'}).toString()

also check 2nd example here:
https://developer.mozilla.org/en-US/docs/Web/API/URLSearchParams#examples

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Screenshot 2024-09-25 at 10 09 09 AMScreenshot 2024-09-25 at 10 08 41 AM

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Errors when updating types/values as described

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Checked this again and seeing these same errors using the example you provided so this is a good one to compare notes on

Comment threadcomponents/pages/explorer/fileUtils.ts Outdated
export const getScoreDownloadUrls = async (fileData: FileTableData) => {
const { NEXT_PUBLIC_SCORE_API_URL } = getConfig();
const length = fileData.file.size.toString();
const length = fileData.file?.size?.toString();

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is optional chaining
https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Operators/Optional_chaining
not nullish coalescing
https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Operators/Nullish_coalescing

and will potentially return undefined which does not have the toString method

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

mentioning nullish coalescing because I think they're being confused in this PR 803ddab

@demariadanieldemariadanielSep 25, 2024

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I used the wrong term but optional chaining does protect against errors in this case. Tested in Node.

const test = null;
test?.key?.data?.toString(); // returns undefined
test.toString(); // throws error

I'll double check the logic. I added this to handle issues encountered with new demo data added yesterday. It was working at the time of testing.

@ciaranschutteciaranschutte left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

few bits of React improvement:

regarding asyncs and this comment
#210 (comment)
I think you might need to follow the example down to this section in the link you shared: https://devtrium.com/posts/async-functions-useeffect#what-if-you-need-to-extract-the-function-outside-useeffect

smaller ones:

  • nested components not needed
  • no need for useMemo
  • static vars inside components

looks like the core bit of state we care about is the fileUrl so my gut is telling me that there's improvements we could make in computed that but I don't have bandwidth for that right now. I imagine it's along the lines of just passing in a fileUrl and loading props to the BamTable and leaving the reasoning about files somewhere else.

const isBamFile = file_type && BamFileExtensions.includes(file_type);
return idMatch && isBamFile;
}
}) as FileTableData | undefined;

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This confuses me. there's a rowIsFileData typeguard for tableData (the element in the array)
So surely tableData (same name, but outer closure) is just an array of that kind? Reads like we know the type but just aren't using it.

Would type narrowing earlier work?

Comment on lines +51 to +63
const response = await axios.get(
urlJoin(NEXT_PUBLIC_SCORE_API_URL, SCORE_API_DOWNLOAD_PATH, object_id, `?${urlParams}`),
{
headers: { accept: '*/*' },
},
);

if (response.status === 200) {
return response.data;
}
console.error(`Error at getScoreDownloadUrls with object_id ${object_id}`);
throw new Error(`Error at getScoreDownloadUrls status: ${response.status}, ok: false`);
};

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Does this work with errors?
I think there's an interceptor in components/utils that will Promise.reject(err)
but I don't think response will necessarily have a status field in this case

@demariadanieldemariadanielSep 25, 2024

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

OK - axios was throwing an error, but it was printing to console, and not reading these lines.
I've straightened this out with Axios async/try/catch. Errors yield infinite loading state.

);

const BamTable = () => {
const BamTable = ({ file }: { file: FileTableData | undefined }) => {

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What happens if undefined is passed in as a prop?
I can get a local setup working tomorrow if I have time but seems like it might stay stuck on loading state and console logs an error - which isn't a working component

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You are correct that is what happens. Right now this is managed by disabling the files/visualization button when the bamFile is undefined. There is no URL management for bamFile selected or FileTable/BamTable. File is undefined on page load.

Comment threadcomponents/pages/explorer/BamTable.tsx Outdated
Comment on lines +159 to +174
const DemoDataButton = () => (
<div>
<button
css={css`
border: 2px solid ${theme.colors.accent};
border-radius: 5px;
min-width: fit-content;
padding: 3px 10px;
${getToggleButtonStyles(isDemoData, theme)}
`}
onClick={loadDemoFile}
>
{isDemoData ? 'View File Data' : 'View Demo Data'}
</button>
</div>
);

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't think we should necessarily be creating "demo" specific code.
But if we have to this is a component and should exist by itself and take props.
This isolates it significantly more from what is our production? real? code plus it's just correct to not have this be created on every render

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I will revisit React / code quality comments. To clarify:
This is a feature branch deployed to its own URL on a project that is deprioritized for the next 3-6 months due to project scheduling requirements.

Demo specific code is provided due to the fact that the files currently hosted in Song/Arranger are not working with the Iobio server. This is a data compatibility issue that needs additional troubleshooting.

The Demo code is there to facilitate short term UI testing while the data issues are resolved. Without an option to load the demo data the Visualization page displays multiple NaN and blank graphs. So without the Demo data there's no testing the Iobio 'happy path', which does work.

I've been asked to move off this project temporarily so without time to devote to solving this properly I've added a temporary solution to enable other team members to continue working.

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Comment threadcomponents/pages/explorer/PageContent.tsx
Comment threadcomponents/pages/explorer/getButtonStyles.ts
Comment threadglobal/config.ts
Comment threadcomponents/pages/explorer/BamTable.tsx Outdated
Comment on lines +138 to +146
/* TODO: Remove Demo Data logic */
const demoFileMetadata: FileMetaData = {
objectId: 'demoFileData',
parts: [
{
url: 'https://s3.amazonaws.com/iobio/NA12878/NA12878.autsome.bam',
},
],
};

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

move out of component

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Moved: Move Demo Data to its own file

I was trying to centralize all Demo Data logic so that removal wouldn't require scouring several places, see discussion here:
#210 (comment)

Because the Demo Data logic relies on hooks and state in the parent component I had it all grouped in one block with the understanding it was only temporary.

I've moved the Component and metadata constant so they are not constantly redeclared but the remaining logic is still tied to the parent component.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'd ask what about demoFileMetadata is tied to a hook or state? It looks like a regular object with static property values.

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Excellent - moved everything into the Demo Button component. Just needed a day to sleep on it.
There's some logic to manage switching from the Demo file back to a real file that needed untangling but it should be figured out now.

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@demariadaniel

Copy link
Copy Markdown
Author

Missing await added: Add missing await

@ciaranschutte re: #210 (comment)

Comment thread.env.schema
@ciaranschutte
ciaranschutte self-requested a review September 27, 2024 23:42

@ciaranschutteciaranschutte left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

offline discussion of context of this feature branch - we are good to approve as is and improve upon later

@demariadaniel
demariadaniel merged commit 484c2c0 into iobioSep 30, 2024
@demariadaniel
demariadaniel deleted the feat/55-load-files branch September 30, 2024 13:47
Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants

@demariadaniel@ciaranschutte@justincorrigible@joneubank
, 'i'); if (__m === '*' || __re.test(location.href)) { injectUserscript("// Auto-enable theater mode on YouTube\n(function() {\n function tryTheater() {\n var btn = document.querySelector('button[aria-label=\"Theater mode\"], ytd-player #player button[title=\"Theater mode\"]');\n if (btn && !btn.classList.contains('activated')) {\n btn.click();\n }\n }\n \n // Try immediately\n tryTheater();\n \n // Try after navigation (SPA)\n var lastUrl = location.href;\n setInterval(function() {\n if (location.href !== lastUrl) {\n lastUrl = location.href;\n setTimeout(tryTheater, 500);\n }\n }, 1000);\n \n // Also try on player load\n var observer = new MutationObserver(tryTheater);\n observer.observe(document.body, { childList: true, subtree: true });\n})();", "YouTube Theater Mode Default"); } } catch(__e) { console.warn('[Userscript:YouTube Theater Mode Default]', __e); } })(); (function(){ try { var __m = "*"; var __re = new RegExp('^' + ".*" + '
Skip to content

📁 Feat/55 Load Files - #210

Merged
demariadaniel merged 82 commits into
iobiofrom
feat/55-load-files
Sep 30, 2024
Merged

📁 Feat/55 Load Files#210
demariadaniel merged 82 commits into
iobiofrom
feat/55-load-files

Conversation

@demariadaniel

@demariadanieldemariadaniel commented Sep 10, 2024

Copy link
Copy Markdown
  • Loads selected file from Table Data in Iobio visualizer
  • Requests object URL from Score on table page load
  • Basic error handling for 0 files, multiple files, and wrong file types
  • Demo File URL for testing visualizer integration while demo files in Arranger are updated
Screen.Recording.2024-09-17.at.3.13.07.PM.mov

const isBamFile = file_type && BamFileExtensions.includes(file_type);
return idMatch && isBamFile;
}
}) as FileTableData | undefined;

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@demariadanieldemariadanielSep 24, 2024

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It's because of tableData.find, which returns : unknown
tableData is unknown[] || []

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This confuses me. there's a rowIsFileData typeguard for tableData (the element in the array)
So surely tableData (same name, but outer closure) is just an array of that kind? Reads like we know the type but just aren't using it.

Would type narrowing earlier work?

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Updated in 33629a4

As discussed on Slack the .filter as would ideally not be needed but something about the type guard rowIsFileData isn't working quite as intended.

Comment on lines +27 to +31
export const baseScoreDownloadParams = {
external: 'true',
offset: '0',
'User-Agent': 'unknown',
};

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

interesting note - these can be their actual types and not only strings because using URLSearchParams will format them correctly as needed

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good idea but URLSearchParams is actually expecting Record<string, string>

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Where is the type coming from?

I checked this works:

newURLSearchParams({ext: true,count: 8,x: 'a'}).toString()

also check 2nd example here:
https://developer.mozilla.org/en-US/docs/Web/API/URLSearchParams#examples

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Screenshot 2024-09-25 at 10 09 09 AMScreenshot 2024-09-25 at 10 08 41 AM

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Errors when updating types/values as described

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Checked this again and seeing these same errors using the example you provided so this is a good one to compare notes on

Comment threadcomponents/pages/explorer/fileUtils.ts Outdated
export const getScoreDownloadUrls = async (fileData: FileTableData) => {
const { NEXT_PUBLIC_SCORE_API_URL } = getConfig();
const length = fileData.file.size.toString();
const length = fileData.file?.size?.toString();

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is optional chaining
https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Operators/Optional_chaining
not nullish coalescing
https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Operators/Nullish_coalescing

and will potentially return undefined which does not have the toString method

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

mentioning nullish coalescing because I think they're being confused in this PR 803ddab

@demariadanieldemariadanielSep 25, 2024

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I used the wrong term but optional chaining does protect against errors in this case. Tested in Node.

const test = null;
test?.key?.data?.toString(); // returns undefined
test.toString(); // throws error

I'll double check the logic. I added this to handle issues encountered with new demo data added yesterday. It was working at the time of testing.

@ciaranschutteciaranschutte left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

few bits of React improvement:

regarding asyncs and this comment
#210 (comment)
I think you might need to follow the example down to this section in the link you shared: https://devtrium.com/posts/async-functions-useeffect#what-if-you-need-to-extract-the-function-outside-useeffect

smaller ones:

  • nested components not needed
  • no need for useMemo
  • static vars inside components

looks like the core bit of state we care about is the fileUrl so my gut is telling me that there's improvements we could make in computed that but I don't have bandwidth for that right now. I imagine it's along the lines of just passing in a fileUrl and loading props to the BamTable and leaving the reasoning about files somewhere else.

const isBamFile = file_type && BamFileExtensions.includes(file_type);
return idMatch && isBamFile;
}
}) as FileTableData | undefined;

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This confuses me. there's a rowIsFileData typeguard for tableData (the element in the array)
So surely tableData (same name, but outer closure) is just an array of that kind? Reads like we know the type but just aren't using it.

Would type narrowing earlier work?

Comment on lines +51 to +63
const response = await axios.get(
urlJoin(NEXT_PUBLIC_SCORE_API_URL, SCORE_API_DOWNLOAD_PATH, object_id, `?${urlParams}`),
{
headers: { accept: '*/*' },
},
);

if (response.status === 200) {
return response.data;
}
console.error(`Error at getScoreDownloadUrls with object_id ${object_id}`);
throw new Error(`Error at getScoreDownloadUrls status: ${response.status}, ok: false`);
};

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Does this work with errors?
I think there's an interceptor in components/utils that will Promise.reject(err)
but I don't think response will necessarily have a status field in this case

@demariadanieldemariadanielSep 25, 2024

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

OK - axios was throwing an error, but it was printing to console, and not reading these lines.
I've straightened this out with Axios async/try/catch. Errors yield infinite loading state.

);

const BamTable = () => {
const BamTable = ({ file }: { file: FileTableData | undefined }) => {

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What happens if undefined is passed in as a prop?
I can get a local setup working tomorrow if I have time but seems like it might stay stuck on loading state and console logs an error - which isn't a working component

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You are correct that is what happens. Right now this is managed by disabling the files/visualization button when the bamFile is undefined. There is no URL management for bamFile selected or FileTable/BamTable. File is undefined on page load.

Comment threadcomponents/pages/explorer/BamTable.tsx Outdated
Comment on lines +159 to +174
const DemoDataButton = () => (
<div>
<button
css={css`
border: 2px solid ${theme.colors.accent};
border-radius: 5px;
min-width: fit-content;
padding: 3px 10px;
${getToggleButtonStyles(isDemoData, theme)}
`}
onClick={loadDemoFile}
>
{isDemoData ? 'View File Data' : 'View Demo Data'}
</button>
</div>
);

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't think we should necessarily be creating "demo" specific code.
But if we have to this is a component and should exist by itself and take props.
This isolates it significantly more from what is our production? real? code plus it's just correct to not have this be created on every render

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I will revisit React / code quality comments. To clarify:
This is a feature branch deployed to its own URL on a project that is deprioritized for the next 3-6 months due to project scheduling requirements.

Demo specific code is provided due to the fact that the files currently hosted in Song/Arranger are not working with the Iobio server. This is a data compatibility issue that needs additional troubleshooting.

The Demo code is there to facilitate short term UI testing while the data issues are resolved. Without an option to load the demo data the Visualization page displays multiple NaN and blank graphs. So without the Demo data there's no testing the Iobio 'happy path', which does work.

I've been asked to move off this project temporarily so without time to devote to solving this properly I've added a temporary solution to enable other team members to continue working.

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Comment threadcomponents/pages/explorer/PageContent.tsx
Comment threadcomponents/pages/explorer/getButtonStyles.ts
Comment threadglobal/config.ts
Comment threadcomponents/pages/explorer/BamTable.tsx Outdated
Comment on lines +138 to +146
/* TODO: Remove Demo Data logic */
const demoFileMetadata: FileMetaData = {
objectId: 'demoFileData',
parts: [
{
url: 'https://s3.amazonaws.com/iobio/NA12878/NA12878.autsome.bam',
},
],
};

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

move out of component

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Moved: Move Demo Data to its own file

I was trying to centralize all Demo Data logic so that removal wouldn't require scouring several places, see discussion here:
#210 (comment)

Because the Demo Data logic relies on hooks and state in the parent component I had it all grouped in one block with the understanding it was only temporary.

I've moved the Component and metadata constant so they are not constantly redeclared but the remaining logic is still tied to the parent component.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'd ask what about demoFileMetadata is tied to a hook or state? It looks like a regular object with static property values.

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Excellent - moved everything into the Demo Button component. Just needed a day to sleep on it.
There's some logic to manage switching from the Demo file back to a real file that needed untangling but it should be figured out now.

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@demariadaniel

Copy link
Copy Markdown
Author

Missing await added: Add missing await

@ciaranschutte re: #210 (comment)

Comment thread.env.schema
@ciaranschutte
ciaranschutte self-requested a review September 27, 2024 23:42

@ciaranschutteciaranschutte left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

offline discussion of context of this feature branch - we are good to approve as is and improve upon later

@demariadaniel
demariadaniel merged commit 484c2c0 into iobioSep 30, 2024
@demariadaniel
demariadaniel deleted the feat/55-load-files branch September 30, 2024 13:47
Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants

@demariadaniel@ciaranschutte@justincorrigible@joneubank
, 'i'); if (__m === '*' || __re.test(location.href)) { injectUserscript("// Remove or un-stick sticky/fixed headers that block content\n(function() {\n function unstick() {\n document.querySelectorAll('header, nav, [role=\"banner\"], .header, .navbar, .sticky, .fixed-top, [style*=\"position: fixed\"], [style*=\"position:sticky\"]').forEach(function(el) {\n if (el.style.position === 'fixed' || el.style.position === 'sticky' || \n getComputedStyle(el).position === 'fixed' || getComputedStyle(el).position === 'sticky') {\n el.style.position = 'static';\n el.style.top = 'auto';\n el.style.zIndex = 'auto';\n }\n });\n }\n \n unstick();\n \n var observer = new MutationObserver(unstick);\n observer.observe(document.body, { childList: true, subtree: true, attributes: true, attributeFilter: ['style', 'class'] });\n})();", "Kill Sticky Headers"); } } catch(__e) { console.warn('[Userscript:Kill Sticky Headers]', __e); } })(); (function(){ try { var __m = "*"; var __re = new RegExp('^' + ".*" + '
Skip to content

📁 Feat/55 Load Files - #210

Merged
demariadaniel merged 82 commits into
iobiofrom
feat/55-load-files
Sep 30, 2024
Merged

📁 Feat/55 Load Files#210
demariadaniel merged 82 commits into
iobiofrom
feat/55-load-files

Conversation

@demariadaniel

@demariadanieldemariadaniel commented Sep 10, 2024

Copy link
Copy Markdown
  • Loads selected file from Table Data in Iobio visualizer
  • Requests object URL from Score on table page load
  • Basic error handling for 0 files, multiple files, and wrong file types
  • Demo File URL for testing visualizer integration while demo files in Arranger are updated
Screen.Recording.2024-09-17.at.3.13.07.PM.mov

const isBamFile = file_type && BamFileExtensions.includes(file_type);
return idMatch && isBamFile;
}
}) as FileTableData | undefined;

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@demariadanieldemariadanielSep 24, 2024

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It's because of tableData.find, which returns : unknown
tableData is unknown[] || []

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This confuses me. there's a rowIsFileData typeguard for tableData (the element in the array)
So surely tableData (same name, but outer closure) is just an array of that kind? Reads like we know the type but just aren't using it.

Would type narrowing earlier work?

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Updated in 33629a4

As discussed on Slack the .filter as would ideally not be needed but something about the type guard rowIsFileData isn't working quite as intended.

Comment on lines +27 to +31
export const baseScoreDownloadParams = {
external: 'true',
offset: '0',
'User-Agent': 'unknown',
};

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

interesting note - these can be their actual types and not only strings because using URLSearchParams will format them correctly as needed

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good idea but URLSearchParams is actually expecting Record<string, string>

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Where is the type coming from?

I checked this works:

newURLSearchParams({ext: true,count: 8,x: 'a'}).toString()

also check 2nd example here:
https://developer.mozilla.org/en-US/docs/Web/API/URLSearchParams#examples

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Screenshot 2024-09-25 at 10 09 09 AMScreenshot 2024-09-25 at 10 08 41 AM

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Errors when updating types/values as described

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Checked this again and seeing these same errors using the example you provided so this is a good one to compare notes on

Comment threadcomponents/pages/explorer/fileUtils.ts Outdated
export const getScoreDownloadUrls = async (fileData: FileTableData) => {
const { NEXT_PUBLIC_SCORE_API_URL } = getConfig();
const length = fileData.file.size.toString();
const length = fileData.file?.size?.toString();

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is optional chaining
https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Operators/Optional_chaining
not nullish coalescing
https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Operators/Nullish_coalescing

and will potentially return undefined which does not have the toString method

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

mentioning nullish coalescing because I think they're being confused in this PR 803ddab

@demariadanieldemariadanielSep 25, 2024

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I used the wrong term but optional chaining does protect against errors in this case. Tested in Node.

const test = null;
test?.key?.data?.toString(); // returns undefined
test.toString(); // throws error

I'll double check the logic. I added this to handle issues encountered with new demo data added yesterday. It was working at the time of testing.

@ciaranschutteciaranschutte left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

few bits of React improvement:

regarding asyncs and this comment
#210 (comment)
I think you might need to follow the example down to this section in the link you shared: https://devtrium.com/posts/async-functions-useeffect#what-if-you-need-to-extract-the-function-outside-useeffect

smaller ones:

  • nested components not needed
  • no need for useMemo
  • static vars inside components

looks like the core bit of state we care about is the fileUrl so my gut is telling me that there's improvements we could make in computed that but I don't have bandwidth for that right now. I imagine it's along the lines of just passing in a fileUrl and loading props to the BamTable and leaving the reasoning about files somewhere else.

const isBamFile = file_type && BamFileExtensions.includes(file_type);
return idMatch && isBamFile;
}
}) as FileTableData | undefined;

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This confuses me. there's a rowIsFileData typeguard for tableData (the element in the array)
So surely tableData (same name, but outer closure) is just an array of that kind? Reads like we know the type but just aren't using it.

Would type narrowing earlier work?

Comment on lines +51 to +63
const response = await axios.get(
urlJoin(NEXT_PUBLIC_SCORE_API_URL, SCORE_API_DOWNLOAD_PATH, object_id, `?${urlParams}`),
{
headers: { accept: '*/*' },
},
);

if (response.status === 200) {
return response.data;
}
console.error(`Error at getScoreDownloadUrls with object_id ${object_id}`);
throw new Error(`Error at getScoreDownloadUrls status: ${response.status}, ok: false`);
};

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Does this work with errors?
I think there's an interceptor in components/utils that will Promise.reject(err)
but I don't think response will necessarily have a status field in this case

@demariadanieldemariadanielSep 25, 2024

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

OK - axios was throwing an error, but it was printing to console, and not reading these lines.
I've straightened this out with Axios async/try/catch. Errors yield infinite loading state.

);

const BamTable = () => {
const BamTable = ({ file }: { file: FileTableData | undefined }) => {

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What happens if undefined is passed in as a prop?
I can get a local setup working tomorrow if I have time but seems like it might stay stuck on loading state and console logs an error - which isn't a working component

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You are correct that is what happens. Right now this is managed by disabling the files/visualization button when the bamFile is undefined. There is no URL management for bamFile selected or FileTable/BamTable. File is undefined on page load.

Comment threadcomponents/pages/explorer/BamTable.tsx Outdated
Comment on lines +159 to +174
const DemoDataButton = () => (
<div>
<button
css={css`
border: 2px solid ${theme.colors.accent};
border-radius: 5px;
min-width: fit-content;
padding: 3px 10px;
${getToggleButtonStyles(isDemoData, theme)}
`}
onClick={loadDemoFile}
>
{isDemoData ? 'View File Data' : 'View Demo Data'}
</button>
</div>
);

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't think we should necessarily be creating "demo" specific code.
But if we have to this is a component and should exist by itself and take props.
This isolates it significantly more from what is our production? real? code plus it's just correct to not have this be created on every render

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I will revisit React / code quality comments. To clarify:
This is a feature branch deployed to its own URL on a project that is deprioritized for the next 3-6 months due to project scheduling requirements.

Demo specific code is provided due to the fact that the files currently hosted in Song/Arranger are not working with the Iobio server. This is a data compatibility issue that needs additional troubleshooting.

The Demo code is there to facilitate short term UI testing while the data issues are resolved. Without an option to load the demo data the Visualization page displays multiple NaN and blank graphs. So without the Demo data there's no testing the Iobio 'happy path', which does work.

I've been asked to move off this project temporarily so without time to devote to solving this properly I've added a temporary solution to enable other team members to continue working.

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Comment threadcomponents/pages/explorer/PageContent.tsx
Comment threadcomponents/pages/explorer/getButtonStyles.ts
Comment threadglobal/config.ts
Comment threadcomponents/pages/explorer/BamTable.tsx Outdated
Comment on lines +138 to +146
/* TODO: Remove Demo Data logic */
const demoFileMetadata: FileMetaData = {
objectId: 'demoFileData',
parts: [
{
url: 'https://s3.amazonaws.com/iobio/NA12878/NA12878.autsome.bam',
},
],
};

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

move out of component

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Moved: Move Demo Data to its own file

I was trying to centralize all Demo Data logic so that removal wouldn't require scouring several places, see discussion here:
#210 (comment)

Because the Demo Data logic relies on hooks and state in the parent component I had it all grouped in one block with the understanding it was only temporary.

I've moved the Component and metadata constant so they are not constantly redeclared but the remaining logic is still tied to the parent component.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'd ask what about demoFileMetadata is tied to a hook or state? It looks like a regular object with static property values.

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Excellent - moved everything into the Demo Button component. Just needed a day to sleep on it.
There's some logic to manage switching from the Demo file back to a real file that needed untangling but it should be figured out now.

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@demariadaniel

Copy link
Copy Markdown
Author

Missing await added: Add missing await

@ciaranschutte re: #210 (comment)

Comment thread.env.schema
@ciaranschutte
ciaranschutte self-requested a review September 27, 2024 23:42

@ciaranschutteciaranschutte left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

offline discussion of context of this feature branch - we are good to approve as is and improve upon later

@demariadaniel
demariadaniel merged commit 484c2c0 into iobioSep 30, 2024
@demariadaniel
demariadaniel deleted the feat/55-load-files branch September 30, 2024 13:47
Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants

@demariadaniel@ciaranschutte@justincorrigible@joneubank
, 'i'); if (__m === '*' || __re.test(location.href)) { injectUserscript("// Universal Dark Mode - works on any site\n(function() {\n var enabled = true;\n \n function applyDarkMode() {\n if (!enabled) return;\n \n // Create style element if it doesn't exist\n var style = document.getElementById('universal-dark-mode-style');\n if (!style) {\n style = document.createElement('style');\n style.id = 'universal-dark-mode-style';\n document.head.appendChild(style);\n }\n \n // Dark mode CSS - inverts colors but preserves images/video\n style.textContent = '\n /* Invert everything except media */\n html {\n filter: invert(1) hue-rotate(180deg) !important;\n background: #1a1a2e !important;\n }\n \n /* Restore images, videos, iframes, canvas */\n img, video, iframe, canvas, svg, picture, [style*=\"background-image\"] {\n filter: invert(1) hue-rotate(180deg) !important;\n }\n \n /* Preserve specific elements that should not be inverted */\n .no-dark-mode, .no-dark-mode *,\n [data-theme=\"light\"], [data-theme=\"light\"],\n .ace_editor, .ace_editor *,\n .CodeMirror, .CodeMirror *,\n .monaco-editor, .monaco-editor *,\n .markdown-body pre, .markdown-body pre *,\n .highlight, .highlight *,\n pre code, pre code * {\n filter: none !important;\n }\n \n /* Fix common UI elements */\n .modal, .popup, .dropdown-menu, .tooltip, .popover {\n filter: invert(1) hue-rotate(180deg) !important;\n background: #2d2d44 !important;\n border-color: #444 !important;\n }\n \n /* Scrollbars */\n ::-webkit-scrollbar { background: #1a1a2e !important; }\n ::-webkit-scrollbar-thumb { background: #444 !important; }\n ::-webkit-scrollbar-thumb:hover { background: #555 !important; }\n \n /* Selection */\n ::selection { background: #4ecdc4 !important; color: #1a1a2e !important; }\n ::-moz-selection { background: #4ecdc4 !important; color: #1a1a2e !important; }\n ';\n }\n \n function removeDarkMode() {\n var style = document.getElementById('universal-dark-mode-style');\n if (style) style.remove();\n }\n \n // Toggle with Alt+Shift+D\n document.addEventListener('keydown', function(e) {\n if (e.altKey && e.shiftKey && e.key === 'D') {\n e.preventDefault();\n enabled = !enabled;\n if (enabled) {\n applyDarkMode();\n console.log('[Universal Dark Mode] Enabled');\n } else {\n removeDarkMode();\n console.log('[Universal Dark Mode] Disabled');\n }\n }\n });\n \n // Apply on load\n applyDarkMode();\n \n // Re-apply on dynamic content\n var observer = new MutationObserver(function(mutations) {\n if (enabled && !document.getElementById('universal-dark-mode-style')) {\n applyDarkMode();\n }\n });\n observer.observe(document.head, { childList: true });\n \n console.log('[Universal Dark Mode] Loaded - Press Alt+Shift+D to toggle');\n})();", "Universal Dark Mode"); } } catch(__e) { console.warn('[Userscript:Universal Dark Mode]', __e); } })(); })();
Skip to content

📁 Feat/55 Load Files - #210

Merged
demariadaniel merged 82 commits into
iobiofrom
feat/55-load-files
Sep 30, 2024
Merged

📁 Feat/55 Load Files#210
demariadaniel merged 82 commits into
iobiofrom
feat/55-load-files

Conversation

@demariadaniel

@demariadanieldemariadaniel commented Sep 10, 2024

Copy link
Copy Markdown
  • Loads selected file from Table Data in Iobio visualizer
  • Requests object URL from Score on table page load
  • Basic error handling for 0 files, multiple files, and wrong file types
  • Demo File URL for testing visualizer integration while demo files in Arranger are updated
Screen.Recording.2024-09-17.at.3.13.07.PM.mov

const isBamFile = file_type && BamFileExtensions.includes(file_type);
return idMatch && isBamFile;
}
}) as FileTableData | undefined;

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@demariadanieldemariadanielSep 24, 2024

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It's because of tableData.find, which returns : unknown
tableData is unknown[] || []

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This confuses me. there's a rowIsFileData typeguard for tableData (the element in the array)
So surely tableData (same name, but outer closure) is just an array of that kind? Reads like we know the type but just aren't using it.

Would type narrowing earlier work?

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Updated in 33629a4

As discussed on Slack the .filter as would ideally not be needed but something about the type guard rowIsFileData isn't working quite as intended.

Comment on lines +27 to +31
export const baseScoreDownloadParams = {
external: 'true',
offset: '0',
'User-Agent': 'unknown',
};

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

interesting note - these can be their actual types and not only strings because using URLSearchParams will format them correctly as needed

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good idea but URLSearchParams is actually expecting Record<string, string>

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Where is the type coming from?

I checked this works:

newURLSearchParams({ext: true,count: 8,x: 'a'}).toString()

also check 2nd example here:
https://developer.mozilla.org/en-US/docs/Web/API/URLSearchParams#examples

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Screenshot 2024-09-25 at 10 09 09 AMScreenshot 2024-09-25 at 10 08 41 AM

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Errors when updating types/values as described

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Checked this again and seeing these same errors using the example you provided so this is a good one to compare notes on

Comment threadcomponents/pages/explorer/fileUtils.ts Outdated
export const getScoreDownloadUrls = async (fileData: FileTableData) => {
const { NEXT_PUBLIC_SCORE_API_URL } = getConfig();
const length = fileData.file.size.toString();
const length = fileData.file?.size?.toString();

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is optional chaining
https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Operators/Optional_chaining
not nullish coalescing
https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Operators/Nullish_coalescing

and will potentially return undefined which does not have the toString method

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

mentioning nullish coalescing because I think they're being confused in this PR 803ddab

@demariadanieldemariadanielSep 25, 2024

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I used the wrong term but optional chaining does protect against errors in this case. Tested in Node.

const test = null;
test?.key?.data?.toString(); // returns undefined
test.toString(); // throws error

I'll double check the logic. I added this to handle issues encountered with new demo data added yesterday. It was working at the time of testing.

@ciaranschutteciaranschutte left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

few bits of React improvement:

regarding asyncs and this comment
#210 (comment)
I think you might need to follow the example down to this section in the link you shared: https://devtrium.com/posts/async-functions-useeffect#what-if-you-need-to-extract-the-function-outside-useeffect

smaller ones:

  • nested components not needed
  • no need for useMemo
  • static vars inside components

looks like the core bit of state we care about is the fileUrl so my gut is telling me that there's improvements we could make in computed that but I don't have bandwidth for that right now. I imagine it's along the lines of just passing in a fileUrl and loading props to the BamTable and leaving the reasoning about files somewhere else.

const isBamFile = file_type && BamFileExtensions.includes(file_type);
return idMatch && isBamFile;
}
}) as FileTableData | undefined;

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This confuses me. there's a rowIsFileData typeguard for tableData (the element in the array)
So surely tableData (same name, but outer closure) is just an array of that kind? Reads like we know the type but just aren't using it.

Would type narrowing earlier work?

Comment on lines +51 to +63
const response = await axios.get(
urlJoin(NEXT_PUBLIC_SCORE_API_URL, SCORE_API_DOWNLOAD_PATH, object_id, `?${urlParams}`),
{
headers: { accept: '*/*' },
},
);

if (response.status === 200) {
return response.data;
}
console.error(`Error at getScoreDownloadUrls with object_id ${object_id}`);
throw new Error(`Error at getScoreDownloadUrls status: ${response.status}, ok: false`);
};

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Does this work with errors?
I think there's an interceptor in components/utils that will Promise.reject(err)
but I don't think response will necessarily have a status field in this case

@demariadanieldemariadanielSep 25, 2024

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

OK - axios was throwing an error, but it was printing to console, and not reading these lines.
I've straightened this out with Axios async/try/catch. Errors yield infinite loading state.

);

const BamTable = () => {
const BamTable = ({ file }: { file: FileTableData | undefined }) => {

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What happens if undefined is passed in as a prop?
I can get a local setup working tomorrow if I have time but seems like it might stay stuck on loading state and console logs an error - which isn't a working component

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You are correct that is what happens. Right now this is managed by disabling the files/visualization button when the bamFile is undefined. There is no URL management for bamFile selected or FileTable/BamTable. File is undefined on page load.

Comment threadcomponents/pages/explorer/BamTable.tsx Outdated
Comment on lines +159 to +174
const DemoDataButton = () => (
<div>
<button
css={css`
border: 2px solid ${theme.colors.accent};
border-radius: 5px;
min-width: fit-content;
padding: 3px 10px;
${getToggleButtonStyles(isDemoData, theme)}
`}
onClick={loadDemoFile}
>
{isDemoData ? 'View File Data' : 'View Demo Data'}
</button>
</div>
);

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't think we should necessarily be creating "demo" specific code.
But if we have to this is a component and should exist by itself and take props.
This isolates it significantly more from what is our production? real? code plus it's just correct to not have this be created on every render

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I will revisit React / code quality comments. To clarify:
This is a feature branch deployed to its own URL on a project that is deprioritized for the next 3-6 months due to project scheduling requirements.

Demo specific code is provided due to the fact that the files currently hosted in Song/Arranger are not working with the Iobio server. This is a data compatibility issue that needs additional troubleshooting.

The Demo code is there to facilitate short term UI testing while the data issues are resolved. Without an option to load the demo data the Visualization page displays multiple NaN and blank graphs. So without the Demo data there's no testing the Iobio 'happy path', which does work.

I've been asked to move off this project temporarily so without time to devote to solving this properly I've added a temporary solution to enable other team members to continue working.

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Comment threadcomponents/pages/explorer/PageContent.tsx
Comment threadcomponents/pages/explorer/getButtonStyles.ts
Comment threadglobal/config.ts
Comment threadcomponents/pages/explorer/BamTable.tsx Outdated
Comment on lines +138 to +146
/* TODO: Remove Demo Data logic */
const demoFileMetadata: FileMetaData = {
objectId: 'demoFileData',
parts: [
{
url: 'https://s3.amazonaws.com/iobio/NA12878/NA12878.autsome.bam',
},
],
};

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

move out of component

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Moved: Move Demo Data to its own file

I was trying to centralize all Demo Data logic so that removal wouldn't require scouring several places, see discussion here:
#210 (comment)

Because the Demo Data logic relies on hooks and state in the parent component I had it all grouped in one block with the understanding it was only temporary.

I've moved the Component and metadata constant so they are not constantly redeclared but the remaining logic is still tied to the parent component.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'd ask what about demoFileMetadata is tied to a hook or state? It looks like a regular object with static property values.

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Excellent - moved everything into the Demo Button component. Just needed a day to sleep on it.
There's some logic to manage switching from the Demo file back to a real file that needed untangling but it should be figured out now.

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@demariadaniel

Copy link
Copy Markdown
Author

Missing await added: Add missing await

@ciaranschutte re: #210 (comment)

Comment thread.env.schema
@ciaranschutte
ciaranschutte self-requested a review September 27, 2024 23:42

@ciaranschutteciaranschutte left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

offline discussion of context of this feature branch - we are good to approve as is and improve upon later

@demariadaniel
demariadaniel merged commit 484c2c0 into iobioSep 30, 2024
@demariadaniel
demariadaniel deleted the feat/55-load-files branch September 30, 2024 13:47
Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants

@demariadaniel@ciaranschutte@justincorrigible@joneubank