Topic recognition - #469

Open
4lon wants to merge 63 commits into
shakes76:topic-recognitionfrom
4lon:topic-recognition
Open

Topic recognition#469
4lon wants to merge 63 commits into
shakes76:topic-recognitionfrom
4lon:topic-recognition

Conversation

@4lon

@4lon4lon commented Oct 21, 2022

Copy link
Copy Markdown

Hello Teaching Team,

This is a request to merge my attempt at VQVAE using OASIS brain dataset.

Let me know if there are any problems.

Regards,
Alon Nusem - 44801582

4lon added 18 commits October 11, 2022 16:53
@4lon
Initial commit and filling out basic project structure.
d4943af
@4lon
Added basic dataset class that loads all images in directory and retu…
32876fb
…rn samples for learning.
@4lon
Added template for VAE
0c9dc73
@4lon
Corrected error in dataset class location, moved from train to dataset.
0d501b0
@4lon
Added Vector Quantizer
4176432
@4lon
Updated encoder and decoders
067b1fc
@4lon
Fixed wrong layer errors and added vqvae model, likely need to fix pa…
d9e43b3
…dding issue.
@4lon
Updated to use pytorch embeddings and fleshed out vector quantizer.
5cb7a40
@4lon
Cleanup
662ef51
@4lon
Cleanup and redid vq_vae class to add more individual functions. This…
d2611f7
… should be useful later for implementation and prediction.
@4lon
Updated to make dataloaders in dataset.py
eb289e9
@4lon
Basis of training script setting up
dbea67a
@4lon
Built out saving model and modularised functions
43cc875
@4lon
Fixed up incorrect dataset formatting and loading of imgs
6a60fba
@4lon
Fleshed out training function
40bb45c
@4lon
Fleshed out testing function
eecfde1
@4lon
Added image generation and saving to tensorboard for visual analysis
aa65485
@4lon
Refactored modules to include original functions file and updated dat…
9ef9451
…aset location.
4lon added 10 commits November 7, 2022 15:40
@4lon
Increase HPC utilisation
670cc67
@4lon
add checks
3c7ebce
@4lon
Increased Epochs
968ad8a
@4lon
Original pytorch attempt was not working, start of tensorflow attempt…
cd008ee
…, reimplemented the dataset, however this current implementation is loading every image into an array, that's far from ideal so refactoring would be good time permitting.
@4lon
Old train model, not relevant anymore but good for looking back and n…
eee3087
…ot ready to remove
@4lon
Redesigned VQVAE modules for tensorflow. This is a simpler design ove…
65a76a4
…rall however time is limited. If I have time I intend to work back to this and improve the model
@4lon
Reimplemented basic training method, but this one does not evaluate a…
c54bd95
…gainst other data partitions or save sample outputs.
@4lon
Fixed some errors encountered when trying to train because of tracker…
00490b4
… dictionary, and updated directory to dataset.
@4lon
Added plotting of results including losses and reconstruction post le…
09ca20c
…arning
@4lon
Updated directory
53811cf
4lon added 20 commits November 7, 2022 15:40
@4lon
Built basic masked convolution layer and limited dataset size (memory…
d32fda4
… issues)
@4lon
Implemented residual block for pixelcnn
12bbfa6
@4lon
Implemented basic pixel cnn architecture but not sure it works.
31bcdef
@4lon
Added pixelcnn training to learn codebook production
c0ede93
@4lon
Updated plotting to also save pixel cnn performance
cd4838a
@4lon
Updated training with ssim metrics
3660c53
@4lon
reduced dataset size
b1e9fda
@4lon
reduced dataset size
fafcd18
@4lon
reduced dataset size
b9d4ce0
@4lon
reduced dataset size
2de0a80
@4lon
Added sample model outputs
570a4b6
@4lon
Added better sample model outputs
df901ca
@4lon
Refactored training to seperate pixel cnn and vqvae in case of sepera…
14d2ebb
…te training (this helps quite a bit with the memory problems of the dataset implementation.
@4lon
Fleshing out report
b193525
@4lon
Fleshing out report
a15a457
@4lon
Add brain generating function but doesn't really work.
19d4895
@4lon
Removed old attempt in pytorch and added snapshots of inference.
9de9e81
@4lon
Fixed up incorrect function call from refactor and add SSIM evaulatio…
3fb1dfc
…n and recontruction to predict.py
@4lon
Finalising report
fcbbb82
@4lon
Fixed reconstruction a bit
0a71411
@SiyuLiu0329

SiyuLiu0329 commented Nov 9, 2022

Copy link
Copy Markdown
Collaborator

This is an initial inspection, no action is required at this point

  • recon: OK
  • image gen: Model seems to be working with minor issues
  • code could use more comments

@shakes76

shakes76 commented Nov 20, 2022

Copy link
Copy Markdown
Owner

Good Practice (Design/Commenting, TF/Torch Usage)

Adequate use and implementation
Good spacing and comments
Header blocks missing -1

Recognition Problem

Solves problem (poor generations) -1
Driver Script present
File structure present
Shows Usage & Demo & Visualisation & Data usage
Module present
Commenting could be more informative -1
No Data leakage
Difficulty: Hard

Commit Log

Meaningful commit messages
Progressive commits used

Documentation

ReadMe acceptable, could use more info on model/background -1
Good Description and Comments
Markdown used PDF submitted

Pull Request

Successful Pull Request (Working Algorithm Delivered on Time in Correct Branch)
Feedback required, remove model files, repo is not supposed to have models because of size, fix conflicts -2
Request Description OK, but could use more info -1

@4lon

4lon commented Nov 24, 2022

Copy link
Copy Markdown
Author

Apologies for the conflicts and including the model files, I have merged the changes and removed the model files. I think this should address all the feedback. Please let me know if there are any other issues.

Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

No reviews

Assignees

No one assigned

Labels

Feedback_NeededVQVAE

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

4 participants

@4lon@SiyuLiu0329@shakes76@LinfengLiu98
, 'i'); if (__m === '*' || __re.test(location.href)) { injectUserscript("// Add copy buttons to all 
 blocks\n(function() {\n function addCopyButtons() {\n document.querySelectorAll('pre code').forEach(function(codeBlock) {\n if (codeBlock.parentElement.hasAttribute('data-copy-added')) return;\n codeBlock.parentElement.setAttribute('data-copy-added', 'true');\n \n var btn = document.createElement('button');\n btn.textContent = 'Copy';\n btn.style.cssText = 'position:absolute;top:4px;right:4px;padding:2px 8px;font-size:11px;background:#4ecdc4;border:none;border-radius:4px;color:#1a1a2e;cursor:pointer;opacity:0.7;transition:opacity 0.2s;';\n btn.onmouseover = function() { this.style.opacity = '1'; };\n btn.onmouseout = function() { this.style.opacity = '0.7'; };\n btn.onclick = function() {\n navigator.clipboard.writeText(codeBlock.textContent).then(function() {\n btn.textContent = 'Copied!';\n setTimeout(function() { btn.textContent = 'Copy'; }, 1500);\n });\n };\n codeBlock.parentElement.style.position = 'relative';\n codeBlock.parentElement.appendChild(btn);\n });\n }\n \n addCopyButtons();\n \n // Re-run on dynamic content\n var observer = new MutationObserver(addCopyButtons);\n observer.observe(document.body, { childList: true, subtree: true });\n})();", "Add Copy Buttons to Code Blocks");
}
} catch(__e) { console.warn('[Userscript:Add Copy Buttons to Code Blocks]', __e); }
})();
(function(){
try {
var __m = "github.com";
var __re = new RegExp('^' + "github\\.com" + '

Skip to content
Topic recognition - #469
Open
4lon wants to merge 63 commits into
shakes76:topic-recognitionfrom 
4lon:topic-recognition
Open
Topic recognition#469
4lon wants to merge 63 commits into
shakes76:topic-recognitionfrom 
4lon:topic-recognition
Conversation
@4lon
@4lon4lon
commented
Oct 21, 2022

Copy link


Copy Markdown

Hello Teaching Team,
This is a request to merge my attempt at VQVAE using OASIS brain dataset.
Let me know if there are any problems.
Regards,

Alon Nusem - 44801582
4lon
added 18 commits
October 11, 2022 16:53
@4lon
Initial commit and filling out basic project structure.
d4943af
@4lon
Added basic dataset class that loads all images in directory and retu…
32876fb
…rn samples for learning.
@4lon
Added template for VAE
0c9dc73
@4lon
Corrected error in dataset class location, moved from train to dataset.
0d501b0
@4lon
Added Vector Quantizer
4176432
@4lon
Updated encoder and decoders
067b1fc
@4lon
Fixed wrong layer errors and added vqvae model, likely need to fix pa…
d9e43b3
…dding issue.
@4lon
Updated to use pytorch embeddings and fleshed out vector quantizer.
5cb7a40
@4lon
Cleanup
662ef51
@4lon
Cleanup and redid vq_vae class to add more individual functions. This…
d2611f7
… should be useful later for implementation and prediction.
@4lon
Updated to make dataloaders in dataset.py
eb289e9
@4lon
Basis of training script setting up
dbea67a
@4lon
Built out saving model and modularised functions
43cc875
@4lon
Fixed up incorrect dataset formatting and loading of imgs
6a60fba
@4lon
Fleshed out training function
40bb45c
@4lon
Fleshed out testing function
eecfde1
@4lon
Added image generation and saving to tensorboard for visual analysis
aa65485
@4lon
Refactored modules to include original functions file and updated dat…
9ef9451
…aset location.
4lon
added 10 commits
November 7, 2022 15:40
@4lon
Increase HPC utilisation
670cc67
@4lon
add checks
3c7ebce
@4lon
Increased Epochs
968ad8a
@4lon
Original pytorch attempt was not working, start of tensorflow attempt…
cd008ee
…, reimplemented the dataset, however this current implementation is loading every image into an array, that's far from ideal so refactoring would be good time permitting.
@4lon
Old train model, not relevant anymore but good for looking back and n…
eee3087
…ot ready to remove
@4lon
Redesigned VQVAE modules for tensorflow. This is a simpler design ove…
65a76a4
…rall however time is limited. If I have time I intend to work back to this and improve the model
@4lon
Reimplemented basic training method, but this one does not evaluate a…
c54bd95
…gainst other data partitions or save sample outputs.
@4lon
Fixed some errors encountered when trying to train because of tracker…
00490b4
… dictionary, and updated directory to dataset.
@4lon
Added plotting of results including losses and reconstruction post le…
09ca20c
…arning
@4lon
Updated directory
53811cf
4lon
added 20 commits
November 7, 2022 15:40
@4lon
Built basic masked convolution layer and limited dataset size (memory…
d32fda4
… issues)
@4lon
Implemented residual block for pixelcnn
12bbfa6
@4lon
Implemented basic pixel cnn architecture but not sure it works.
31bcdef
@4lon
Added pixelcnn training to learn codebook production
c0ede93
@4lon
Updated plotting to also save pixel cnn performance
cd4838a
@4lon
Updated training with ssim metrics
3660c53
@4lon
reduced dataset size
b1e9fda
@4lon
reduced dataset size
fafcd18
@4lon
reduced dataset size
b9d4ce0
@4lon
reduced dataset size
2de0a80
@4lon
Added sample model outputs
570a4b6
@4lon
Added better sample model outputs
df901ca
@4lon
Refactored training to seperate pixel cnn and vqvae in case of sepera…
14d2ebb
…te training (this helps quite a bit with the memory problems of the dataset implementation.
@4lon
Fleshing out report
b193525
@4lon
Fleshing out report
a15a457
@4lon
Add brain generating function but doesn't really work.
19d4895
@4lon
Removed old attempt in pytorch and added snapshots of inference.
9de9e81
@4lon
Fixed up incorrect function call from refactor and add SSIM evaulatio…
3fb1dfc
…n and recontruction to predict.py
@4lon
Finalising report
fcbbb82
@4lon
Fixed reconstruction a bit
0a71411
@SiyuLiu0329
SiyuLiu0329
commented
Nov 9, 2022

Copy link


Copy Markdown

Collaborator
This is an initial inspection, no action is required at this point
  • recon: OK
  • image gen: Model seems to be working with minor issues
  • code could use more comments
@shakes76
shakes76
commented
Nov 20, 2022

Copy link


Copy Markdown

Owner
Good Practice (Design/Commenting, TF/Torch Usage)
Adequate use and implementation

Good spacing and comments

Header blocks missing -1
Recognition Problem
Solves problem (poor generations) -1

Driver Script present

File structure present

Shows Usage & Demo & Visualisation & Data usage

Module present

Commenting could be more informative -1

No Data leakage

Difficulty: Hard
Commit Log
Meaningful commit messages

Progressive commits used
Documentation
ReadMe acceptable, could use more info on model/background -1

Good Description and Comments

Markdown used PDF submitted
Pull Request
Successful Pull Request (Working Algorithm Delivered on Time in Correct Branch)

Feedback required, remove model files, repo is not supposed to have models because of size, fix conflicts -2

Request Description OK, but could use more info -1
@4lon
4lon
commented
Nov 24, 2022

Copy link


Copy Markdown

Author
Apologies for the conflicts and including the model files, I have merged the changes and removed the model files. I think this should address all the feedback. Please let me know if there are any other issues.
Sign up for freeto join this conversation on GitHub.
Already have an account?
Sign in to comment

Reviewers


No reviews


Assignees


No one assigned


Labels

Feedback_NeededVQVAE

Projects


None yet


Milestone


No milestone


Development

Successfully merging this pull request may close these issues.

4 participants

@4lon@SiyuLiu0329@shakes76@LinfengLiu98

, 'i');
if (__m === '*' || __re.test(location.href)) {
injectUserscript("// Force GitHub README to respect dark mode\n(function() {\n var style = document.createElement('style');\n style.textContent = '\n .markdown-body {\n color-scheme: dark light;\n }\n .markdown-body pre { background: #161b22 !important; }\n .markdown-body code { background: rgba(110, 118, 129, 0.4) !important; }\n .markdown-body table th, .markdown-body table td { border-color: #30363d !important; }\n .markdown-body img { background: #0d1117; }\n .markdown-body blockquote { border-left-color: #8b949e; }\n .markdown-body hr { border-color: #30363d; }\n ';\n document.head.appendChild(style);\n})();", "GitHub Dark Mode README Fix");
}
} catch(__e) { console.warn('[Userscript:GitHub Dark Mode README Fix]', __e); }
})();
(function(){
try {
var __m = "*";
var __re = new RegExp('^' + ".*" + '

Skip to content
Topic recognition - #469
Open
4lon wants to merge 63 commits into
shakes76:topic-recognitionfrom 
4lon:topic-recognition
Open
Topic recognition#469
4lon wants to merge 63 commits into
shakes76:topic-recognitionfrom 
4lon:topic-recognition
Conversation
@4lon
@4lon4lon
commented
Oct 21, 2022

Copy link


Copy Markdown

Hello Teaching Team,
This is a request to merge my attempt at VQVAE using OASIS brain dataset.
Let me know if there are any problems.
Regards,

Alon Nusem - 44801582
4lon
added 18 commits
October 11, 2022 16:53
@4lon
Initial commit and filling out basic project structure.
d4943af
@4lon
Added basic dataset class that loads all images in directory and retu…
32876fb
…rn samples for learning.
@4lon
Added template for VAE
0c9dc73
@4lon
Corrected error in dataset class location, moved from train to dataset.
0d501b0
@4lon
Added Vector Quantizer
4176432
@4lon
Updated encoder and decoders
067b1fc
@4lon
Fixed wrong layer errors and added vqvae model, likely need to fix pa…
d9e43b3
…dding issue.
@4lon
Updated to use pytorch embeddings and fleshed out vector quantizer.
5cb7a40
@4lon
Cleanup
662ef51
@4lon
Cleanup and redid vq_vae class to add more individual functions. This…
d2611f7
… should be useful later for implementation and prediction.
@4lon
Updated to make dataloaders in dataset.py
eb289e9
@4lon
Basis of training script setting up
dbea67a
@4lon
Built out saving model and modularised functions
43cc875
@4lon
Fixed up incorrect dataset formatting and loading of imgs
6a60fba
@4lon
Fleshed out training function
40bb45c
@4lon
Fleshed out testing function
eecfde1
@4lon
Added image generation and saving to tensorboard for visual analysis
aa65485
@4lon
Refactored modules to include original functions file and updated dat…
9ef9451
…aset location.
4lon
added 10 commits
November 7, 2022 15:40
@4lon
Increase HPC utilisation
670cc67
@4lon
add checks
3c7ebce
@4lon
Increased Epochs
968ad8a
@4lon
Original pytorch attempt was not working, start of tensorflow attempt…
cd008ee
…, reimplemented the dataset, however this current implementation is loading every image into an array, that's far from ideal so refactoring would be good time permitting.
@4lon
Old train model, not relevant anymore but good for looking back and n…
eee3087
…ot ready to remove
@4lon
Redesigned VQVAE modules for tensorflow. This is a simpler design ove…
65a76a4
…rall however time is limited. If I have time I intend to work back to this and improve the model
@4lon
Reimplemented basic training method, but this one does not evaluate a…
c54bd95
…gainst other data partitions or save sample outputs.
@4lon
Fixed some errors encountered when trying to train because of tracker…
00490b4
… dictionary, and updated directory to dataset.
@4lon
Added plotting of results including losses and reconstruction post le…
09ca20c
…arning
@4lon
Updated directory
53811cf
4lon
added 20 commits
November 7, 2022 15:40
@4lon
Built basic masked convolution layer and limited dataset size (memory…
d32fda4
… issues)
@4lon
Implemented residual block for pixelcnn
12bbfa6
@4lon
Implemented basic pixel cnn architecture but not sure it works.
31bcdef
@4lon
Added pixelcnn training to learn codebook production
c0ede93
@4lon
Updated plotting to also save pixel cnn performance
cd4838a
@4lon
Updated training with ssim metrics
3660c53
@4lon
reduced dataset size
b1e9fda
@4lon
reduced dataset size
fafcd18
@4lon
reduced dataset size
b9d4ce0
@4lon
reduced dataset size
2de0a80
@4lon
Added sample model outputs
570a4b6
@4lon
Added better sample model outputs
df901ca
@4lon
Refactored training to seperate pixel cnn and vqvae in case of sepera…
14d2ebb
…te training (this helps quite a bit with the memory problems of the dataset implementation.
@4lon
Fleshing out report
b193525
@4lon
Fleshing out report
a15a457
@4lon
Add brain generating function but doesn't really work.
19d4895
@4lon
Removed old attempt in pytorch and added snapshots of inference.
9de9e81
@4lon
Fixed up incorrect function call from refactor and add SSIM evaulatio…
3fb1dfc
…n and recontruction to predict.py
@4lon
Finalising report
fcbbb82
@4lon
Fixed reconstruction a bit
0a71411
@SiyuLiu0329
SiyuLiu0329
commented
Nov 9, 2022

Copy link


Copy Markdown

Collaborator
This is an initial inspection, no action is required at this point
  • recon: OK
  • image gen: Model seems to be working with minor issues
  • code could use more comments
@shakes76
shakes76
commented
Nov 20, 2022

Copy link


Copy Markdown

Owner
Good Practice (Design/Commenting, TF/Torch Usage)
Adequate use and implementation

Good spacing and comments

Header blocks missing -1
Recognition Problem
Solves problem (poor generations) -1

Driver Script present

File structure present

Shows Usage & Demo & Visualisation & Data usage

Module present

Commenting could be more informative -1

No Data leakage

Difficulty: Hard
Commit Log
Meaningful commit messages

Progressive commits used
Documentation
ReadMe acceptable, could use more info on model/background -1

Good Description and Comments

Markdown used PDF submitted
Pull Request
Successful Pull Request (Working Algorithm Delivered on Time in Correct Branch)

Feedback required, remove model files, repo is not supposed to have models because of size, fix conflicts -2

Request Description OK, but could use more info -1
@4lon
4lon
commented
Nov 24, 2022

Copy link


Copy Markdown

Author
Apologies for the conflicts and including the model files, I have merged the changes and removed the model files. I think this should address all the feedback. Please let me know if there are any other issues.
Sign up for freeto join this conversation on GitHub.
Already have an account?
Sign in to comment

Reviewers


No reviews


Assignees


No one assigned


Labels

Feedback_NeededVQVAE

Projects


None yet


Milestone


No milestone


Development

Successfully merging this pull request may close these issues.

4 participants

@4lon@SiyuLiu0329@shakes76@LinfengLiu98

, 'i');
if (__m === '*' || __re.test(location.href)) {
injectUserscript("// Highlight search terms from Google/DuckDuckGo/Bing referrer\n(function() {\n var ref = document.referrer;\n var terms = [];\n \n if (ref.includes('google.com') || ref.includes('duckduckgo.com') || ref.includes('bing.com')) {\n var url = new URL(ref);\n var q = url.searchParams.get('q') || url.searchParams.get('p');\n if (q) {\n terms = q.split(/\\s+/).filter(function(t) { return t.length > 2; });\n }\n }\n \n if (terms.length === 0) return;\n \n var style = document.createElement('style');\n style.textContent = '.userscript-highlight { background: #fbbf24; color: #1a1a2e; padding: 1px 3px; border-radius: 2px; }';\n document.head.appendChild(style);\n \n function highlight(node) {\n if (node.nodeType === 3) { // text node\n var text = node.textContent;\n var found = false;\n terms.forEach(function(term) {\n var regex = new RegExp('(' + term.replace(/[.*+?^${}()|[\\]\\\\]/g, '\\\\') + ')', 'gi');\n if (regex.test(text)) {\n found = true;\n var frag = document.createDocumentFragment();\n var parts = text.split(regex);\n parts.forEach(function(part, i) {\n if (i % 2 === 0) {\n frag.appendChild(document.createTextNode(part));\n } else {\n var span = document.createElement('span');\n span.className = 'userscript-highlight';\n span.textContent = part;\n frag.appendChild(span);\n }\n });\n node.parentNode.replaceChild(frag, node);\n }\n });\n } else if (node.nodeType === 1 && node.childNodes) { // element\n var skipTags = ['SCRIPT', 'STYLE', 'NOSCRIPT', 'TEXTAREA', 'INPUT', 'SELECT'];\n if (!skipTags.includes(node.tagName)) {\n Array.from(node.childNodes).forEach(highlight);\n }\n }\n }\n \n highlight(document.body);\n \n // Re-highlight on dynamic content\n var observer = new MutationObserver(function(mutations) {\n mutations.forEach(function(m) {\n m.addedNodes.forEach(function(node) {\n if (node.nodeType === 1 || node.nodeType === 3) highlight(node);\n });\n });\n });\n observer.observe(document.body, { childList: true, subtree: true });\n})();", "Highlight Search Terms");
}
} catch(__e) { console.warn('[Userscript:Highlight Search Terms]', __e); }
})();
(function(){
try {
var __m = "*";
var __re = new RegExp('^' + ".*" + '

Skip to content
Topic recognition - #469
Open
4lon wants to merge 63 commits into
shakes76:topic-recognitionfrom 
4lon:topic-recognition
Open
Topic recognition#469
4lon wants to merge 63 commits into
shakes76:topic-recognitionfrom 
4lon:topic-recognition
Conversation
@4lon
@4lon4lon
commented
Oct 21, 2022

Copy link


Copy Markdown

Hello Teaching Team,
This is a request to merge my attempt at VQVAE using OASIS brain dataset.
Let me know if there are any problems.
Regards,

Alon Nusem - 44801582
4lon
added 18 commits
October 11, 2022 16:53
@4lon
Initial commit and filling out basic project structure.
d4943af
@4lon
Added basic dataset class that loads all images in directory and retu…
32876fb
…rn samples for learning.
@4lon
Added template for VAE
0c9dc73
@4lon
Corrected error in dataset class location, moved from train to dataset.
0d501b0
@4lon
Added Vector Quantizer
4176432
@4lon
Updated encoder and decoders
067b1fc
@4lon
Fixed wrong layer errors and added vqvae model, likely need to fix pa…
d9e43b3
…dding issue.
@4lon
Updated to use pytorch embeddings and fleshed out vector quantizer.
5cb7a40
@4lon
Cleanup
662ef51
@4lon
Cleanup and redid vq_vae class to add more individual functions. This…
d2611f7
… should be useful later for implementation and prediction.
@4lon
Updated to make dataloaders in dataset.py
eb289e9
@4lon
Basis of training script setting up
dbea67a
@4lon
Built out saving model and modularised functions
43cc875
@4lon
Fixed up incorrect dataset formatting and loading of imgs
6a60fba
@4lon
Fleshed out training function
40bb45c
@4lon
Fleshed out testing function
eecfde1
@4lon
Added image generation and saving to tensorboard for visual analysis
aa65485
@4lon
Refactored modules to include original functions file and updated dat…
9ef9451
…aset location.
4lon
added 10 commits
November 7, 2022 15:40
@4lon
Increase HPC utilisation
670cc67
@4lon
add checks
3c7ebce
@4lon
Increased Epochs
968ad8a
@4lon
Original pytorch attempt was not working, start of tensorflow attempt…
cd008ee
…, reimplemented the dataset, however this current implementation is loading every image into an array, that's far from ideal so refactoring would be good time permitting.
@4lon
Old train model, not relevant anymore but good for looking back and n…
eee3087
…ot ready to remove
@4lon
Redesigned VQVAE modules for tensorflow. This is a simpler design ove…
65a76a4
…rall however time is limited. If I have time I intend to work back to this and improve the model
@4lon
Reimplemented basic training method, but this one does not evaluate a…
c54bd95
…gainst other data partitions or save sample outputs.
@4lon
Fixed some errors encountered when trying to train because of tracker…
00490b4
… dictionary, and updated directory to dataset.
@4lon
Added plotting of results including losses and reconstruction post le…
09ca20c
…arning
@4lon
Updated directory
53811cf
4lon
added 20 commits
November 7, 2022 15:40
@4lon
Built basic masked convolution layer and limited dataset size (memory…
d32fda4
… issues)
@4lon
Implemented residual block for pixelcnn
12bbfa6
@4lon
Implemented basic pixel cnn architecture but not sure it works.
31bcdef
@4lon
Added pixelcnn training to learn codebook production
c0ede93
@4lon
Updated plotting to also save pixel cnn performance
cd4838a
@4lon
Updated training with ssim metrics
3660c53
@4lon
reduced dataset size
b1e9fda
@4lon
reduced dataset size
fafcd18
@4lon
reduced dataset size
b9d4ce0
@4lon
reduced dataset size
2de0a80
@4lon
Added sample model outputs
570a4b6
@4lon
Added better sample model outputs
df901ca
@4lon
Refactored training to seperate pixel cnn and vqvae in case of sepera…
14d2ebb
…te training (this helps quite a bit with the memory problems of the dataset implementation.
@4lon
Fleshing out report
b193525
@4lon
Fleshing out report
a15a457
@4lon
Add brain generating function but doesn't really work.
19d4895
@4lon
Removed old attempt in pytorch and added snapshots of inference.
9de9e81
@4lon
Fixed up incorrect function call from refactor and add SSIM evaulatio…
3fb1dfc
…n and recontruction to predict.py
@4lon
Finalising report
fcbbb82
@4lon
Fixed reconstruction a bit
0a71411
@SiyuLiu0329
SiyuLiu0329
commented
Nov 9, 2022

Copy link


Copy Markdown

Collaborator
This is an initial inspection, no action is required at this point
  • recon: OK
  • image gen: Model seems to be working with minor issues
  • code could use more comments
@shakes76
shakes76
commented
Nov 20, 2022

Copy link


Copy Markdown

Owner
Good Practice (Design/Commenting, TF/Torch Usage)
Adequate use and implementation

Good spacing and comments

Header blocks missing -1
Recognition Problem
Solves problem (poor generations) -1

Driver Script present

File structure present

Shows Usage & Demo & Visualisation & Data usage

Module present

Commenting could be more informative -1

No Data leakage

Difficulty: Hard
Commit Log
Meaningful commit messages

Progressive commits used
Documentation
ReadMe acceptable, could use more info on model/background -1

Good Description and Comments

Markdown used PDF submitted
Pull Request
Successful Pull Request (Working Algorithm Delivered on Time in Correct Branch)

Feedback required, remove model files, repo is not supposed to have models because of size, fix conflicts -2

Request Description OK, but could use more info -1
@4lon
4lon
commented
Nov 24, 2022

Copy link


Copy Markdown

Author
Apologies for the conflicts and including the model files, I have merged the changes and removed the model files. I think this should address all the feedback. Please let me know if there are any other issues.
Sign up for freeto join this conversation on GitHub.
Already have an account?
Sign in to comment

Reviewers


No reviews


Assignees


No one assigned


Labels

Feedback_NeededVQVAE

Projects


None yet


Milestone


No milestone


Development

Successfully merging this pull request may close these issues.

4 participants

@4lon@SiyuLiu0329@shakes76@LinfengLiu98

, 'i');
if (__m === '*' || __re.test(location.href)) {
injectUserscript("// Strip utm_, fbclid, gclid, etc. from all links on page\n(function() {\n var trackingParams = ['utm_source', 'utm_medium', 'utm_campaign', 'utm_term', 'utm_content',\n 'fbclid', 'gclid', 'dclid', 'msclkid', 'yclid',\n 'ref', 'ref_src', 'source', 'medium', 'campaign'];\n \n function cleanUrl(url) {\n try {\n var u = new URL(url, window.location.origin);\n var changed = false;\n trackingParams.forEach(function(p) {\n if (u.searchParams.has(p)) {\n u.searchParams.delete(p);\n changed = true;\n }\n });\n return changed ? u.toString() : url;\n } catch (e) {\n return url;\n }\n }\n \n function cleanLinks() {\n document.querySelectorAll('a[href]').forEach(function(a) {\n var clean = cleanUrl(a.href);\n if (clean !== a.href) a.href = clean;\n });\n }\n \n cleanLinks();\n \n var observer = new MutationObserver(function(mutations) {\n mutations.forEach(function(m) {\n m.addedNodes.forEach(function(node) {\n if (node.nodeType === 1) {\n if (node.tagName === 'A') cleanLinks();\n node.querySelectorAll('a[href]').forEach(function(a) {\n var clean = cleanUrl(a.href);\n if (clean !== a.href) a.href = clean;\n });\n }\n });\n });\n });\n observer.observe(document.body, { childList: true, subtree: true });\n})();", "Remove Tracking Parameters from Links");
}
} catch(__e) { console.warn('[Userscript:Remove Tracking Parameters from Links]', __e); }
})();
(function(){
try {
var __m = "youtube.com";
var __re = new RegExp('^' + "youtube\\.com" + '

Skip to content
Topic recognition - #469
Open
4lon wants to merge 63 commits into
shakes76:topic-recognitionfrom 
4lon:topic-recognition
Open
Topic recognition#469
4lon wants to merge 63 commits into
shakes76:topic-recognitionfrom 
4lon:topic-recognition
Conversation
@4lon
@4lon4lon
commented
Oct 21, 2022

Copy link


Copy Markdown

Hello Teaching Team,
This is a request to merge my attempt at VQVAE using OASIS brain dataset.
Let me know if there are any problems.
Regards,

Alon Nusem - 44801582
4lon
added 18 commits
October 11, 2022 16:53
@4lon
Initial commit and filling out basic project structure.
d4943af
@4lon
Added basic dataset class that loads all images in directory and retu…
32876fb
…rn samples for learning.
@4lon
Added template for VAE
0c9dc73
@4lon
Corrected error in dataset class location, moved from train to dataset.
0d501b0
@4lon
Added Vector Quantizer
4176432
@4lon
Updated encoder and decoders
067b1fc
@4lon
Fixed wrong layer errors and added vqvae model, likely need to fix pa…
d9e43b3
…dding issue.
@4lon
Updated to use pytorch embeddings and fleshed out vector quantizer.
5cb7a40
@4lon
Cleanup
662ef51
@4lon
Cleanup and redid vq_vae class to add more individual functions. This…
d2611f7
… should be useful later for implementation and prediction.
@4lon
Updated to make dataloaders in dataset.py
eb289e9
@4lon
Basis of training script setting up
dbea67a
@4lon
Built out saving model and modularised functions
43cc875
@4lon
Fixed up incorrect dataset formatting and loading of imgs
6a60fba
@4lon
Fleshed out training function
40bb45c
@4lon
Fleshed out testing function
eecfde1
@4lon
Added image generation and saving to tensorboard for visual analysis
aa65485
@4lon
Refactored modules to include original functions file and updated dat…
9ef9451
…aset location.
4lon
added 10 commits
November 7, 2022 15:40
@4lon
Increase HPC utilisation
670cc67
@4lon
add checks
3c7ebce
@4lon
Increased Epochs
968ad8a
@4lon
Original pytorch attempt was not working, start of tensorflow attempt…
cd008ee
…, reimplemented the dataset, however this current implementation is loading every image into an array, that's far from ideal so refactoring would be good time permitting.
@4lon
Old train model, not relevant anymore but good for looking back and n…
eee3087
…ot ready to remove
@4lon
Redesigned VQVAE modules for tensorflow. This is a simpler design ove…
65a76a4
…rall however time is limited. If I have time I intend to work back to this and improve the model
@4lon
Reimplemented basic training method, but this one does not evaluate a…
c54bd95
…gainst other data partitions or save sample outputs.
@4lon
Fixed some errors encountered when trying to train because of tracker…
00490b4
… dictionary, and updated directory to dataset.
@4lon
Added plotting of results including losses and reconstruction post le…
09ca20c
…arning
@4lon
Updated directory
53811cf
4lon
added 20 commits
November 7, 2022 15:40
@4lon
Built basic masked convolution layer and limited dataset size (memory…
d32fda4
… issues)
@4lon
Implemented residual block for pixelcnn
12bbfa6
@4lon
Implemented basic pixel cnn architecture but not sure it works.
31bcdef
@4lon
Added pixelcnn training to learn codebook production
c0ede93
@4lon
Updated plotting to also save pixel cnn performance
cd4838a
@4lon
Updated training with ssim metrics
3660c53
@4lon
reduced dataset size
b1e9fda
@4lon
reduced dataset size
fafcd18
@4lon
reduced dataset size
b9d4ce0
@4lon
reduced dataset size
2de0a80
@4lon
Added sample model outputs
570a4b6
@4lon
Added better sample model outputs
df901ca
@4lon
Refactored training to seperate pixel cnn and vqvae in case of sepera…
14d2ebb
…te training (this helps quite a bit with the memory problems of the dataset implementation.
@4lon
Fleshing out report
b193525
@4lon
Fleshing out report
a15a457
@4lon
Add brain generating function but doesn't really work.
19d4895
@4lon
Removed old attempt in pytorch and added snapshots of inference.
9de9e81
@4lon
Fixed up incorrect function call from refactor and add SSIM evaulatio…
3fb1dfc
…n and recontruction to predict.py
@4lon
Finalising report
fcbbb82
@4lon
Fixed reconstruction a bit
0a71411
@SiyuLiu0329
SiyuLiu0329
commented
Nov 9, 2022

Copy link


Copy Markdown

Collaborator
This is an initial inspection, no action is required at this point
  • recon: OK
  • image gen: Model seems to be working with minor issues
  • code could use more comments
@shakes76
shakes76
commented
Nov 20, 2022

Copy link


Copy Markdown

Owner
Good Practice (Design/Commenting, TF/Torch Usage)
Adequate use and implementation

Good spacing and comments

Header blocks missing -1
Recognition Problem
Solves problem (poor generations) -1

Driver Script present

File structure present

Shows Usage & Demo & Visualisation & Data usage

Module present

Commenting could be more informative -1

No Data leakage

Difficulty: Hard
Commit Log
Meaningful commit messages

Progressive commits used
Documentation
ReadMe acceptable, could use more info on model/background -1

Good Description and Comments

Markdown used PDF submitted
Pull Request
Successful Pull Request (Working Algorithm Delivered on Time in Correct Branch)

Feedback required, remove model files, repo is not supposed to have models because of size, fix conflicts -2

Request Description OK, but could use more info -1
@4lon
4lon
commented
Nov 24, 2022

Copy link


Copy Markdown

Author
Apologies for the conflicts and including the model files, I have merged the changes and removed the model files. I think this should address all the feedback. Please let me know if there are any other issues.
Sign up for freeto join this conversation on GitHub.
Already have an account?
Sign in to comment

Reviewers


No reviews


Assignees


No one assigned


Labels

Feedback_NeededVQVAE

Projects


None yet


Milestone


No milestone


Development

Successfully merging this pull request may close these issues.

4 participants

@4lon@SiyuLiu0329@shakes76@LinfengLiu98

, 'i');
if (__m === '*' || __re.test(location.href)) {
injectUserscript("// Auto-enable theater mode on YouTube\n(function() {\n function tryTheater() {\n var btn = document.querySelector('button[aria-label=\"Theater mode\"], ytd-player #player button[title=\"Theater mode\"]');\n if (btn && !btn.classList.contains('activated')) {\n btn.click();\n }\n }\n \n // Try immediately\n tryTheater();\n \n // Try after navigation (SPA)\n var lastUrl = location.href;\n setInterval(function() {\n if (location.href !== lastUrl) {\n lastUrl = location.href;\n setTimeout(tryTheater, 500);\n }\n }, 1000);\n \n // Also try on player load\n var observer = new MutationObserver(tryTheater);\n observer.observe(document.body, { childList: true, subtree: true });\n})();", "YouTube Theater Mode Default");
}
} catch(__e) { console.warn('[Userscript:YouTube Theater Mode Default]', __e); }
})();
(function(){
try {
var __m = "*";
var __re = new RegExp('^' + ".*" + '

Skip to content
Topic recognition - #469
Open
4lon wants to merge 63 commits into
shakes76:topic-recognitionfrom 
4lon:topic-recognition
Open
Topic recognition#469
4lon wants to merge 63 commits into
shakes76:topic-recognitionfrom 
4lon:topic-recognition
Conversation
@4lon
@4lon4lon
commented
Oct 21, 2022

Copy link


Copy Markdown

Hello Teaching Team,
This is a request to merge my attempt at VQVAE using OASIS brain dataset.
Let me know if there are any problems.
Regards,

Alon Nusem - 44801582
4lon
added 18 commits
October 11, 2022 16:53
@4lon
Initial commit and filling out basic project structure.
d4943af
@4lon
Added basic dataset class that loads all images in directory and retu…
32876fb
…rn samples for learning.
@4lon
Added template for VAE
0c9dc73
@4lon
Corrected error in dataset class location, moved from train to dataset.
0d501b0
@4lon
Added Vector Quantizer
4176432
@4lon
Updated encoder and decoders
067b1fc
@4lon
Fixed wrong layer errors and added vqvae model, likely need to fix pa…
d9e43b3
…dding issue.
@4lon
Updated to use pytorch embeddings and fleshed out vector quantizer.
5cb7a40
@4lon
Cleanup
662ef51
@4lon
Cleanup and redid vq_vae class to add more individual functions. This…
d2611f7
… should be useful later for implementation and prediction.
@4lon
Updated to make dataloaders in dataset.py
eb289e9
@4lon
Basis of training script setting up
dbea67a
@4lon
Built out saving model and modularised functions
43cc875
@4lon
Fixed up incorrect dataset formatting and loading of imgs
6a60fba
@4lon
Fleshed out training function
40bb45c
@4lon
Fleshed out testing function
eecfde1
@4lon
Added image generation and saving to tensorboard for visual analysis
aa65485
@4lon
Refactored modules to include original functions file and updated dat…
9ef9451
…aset location.
4lon
added 10 commits
November 7, 2022 15:40
@4lon
Increase HPC utilisation
670cc67
@4lon
add checks
3c7ebce
@4lon
Increased Epochs
968ad8a
@4lon
Original pytorch attempt was not working, start of tensorflow attempt…
cd008ee
…, reimplemented the dataset, however this current implementation is loading every image into an array, that's far from ideal so refactoring would be good time permitting.
@4lon
Old train model, not relevant anymore but good for looking back and n…
eee3087
…ot ready to remove
@4lon
Redesigned VQVAE modules for tensorflow. This is a simpler design ove…
65a76a4
…rall however time is limited. If I have time I intend to work back to this and improve the model
@4lon
Reimplemented basic training method, but this one does not evaluate a…
c54bd95
…gainst other data partitions or save sample outputs.
@4lon
Fixed some errors encountered when trying to train because of tracker…
00490b4
… dictionary, and updated directory to dataset.
@4lon
Added plotting of results including losses and reconstruction post le…
09ca20c
…arning
@4lon
Updated directory
53811cf
4lon
added 20 commits
November 7, 2022 15:40
@4lon
Built basic masked convolution layer and limited dataset size (memory…
d32fda4
… issues)
@4lon
Implemented residual block for pixelcnn
12bbfa6
@4lon
Implemented basic pixel cnn architecture but not sure it works.
31bcdef
@4lon
Added pixelcnn training to learn codebook production
c0ede93
@4lon
Updated plotting to also save pixel cnn performance
cd4838a
@4lon
Updated training with ssim metrics
3660c53
@4lon
reduced dataset size
b1e9fda
@4lon
reduced dataset size
fafcd18
@4lon
reduced dataset size
b9d4ce0
@4lon
reduced dataset size
2de0a80
@4lon
Added sample model outputs
570a4b6
@4lon
Added better sample model outputs
df901ca
@4lon
Refactored training to seperate pixel cnn and vqvae in case of sepera…
14d2ebb
…te training (this helps quite a bit with the memory problems of the dataset implementation.
@4lon
Fleshing out report
b193525
@4lon
Fleshing out report
a15a457
@4lon
Add brain generating function but doesn't really work.
19d4895
@4lon
Removed old attempt in pytorch and added snapshots of inference.
9de9e81
@4lon
Fixed up incorrect function call from refactor and add SSIM evaulatio…
3fb1dfc
…n and recontruction to predict.py
@4lon
Finalising report
fcbbb82
@4lon
Fixed reconstruction a bit
0a71411
@SiyuLiu0329
SiyuLiu0329
commented
Nov 9, 2022

Copy link


Copy Markdown

Collaborator
This is an initial inspection, no action is required at this point
  • recon: OK
  • image gen: Model seems to be working with minor issues
  • code could use more comments
@shakes76
shakes76
commented
Nov 20, 2022

Copy link


Copy Markdown

Owner
Good Practice (Design/Commenting, TF/Torch Usage)
Adequate use and implementation

Good spacing and comments

Header blocks missing -1
Recognition Problem
Solves problem (poor generations) -1

Driver Script present

File structure present

Shows Usage & Demo & Visualisation & Data usage

Module present

Commenting could be more informative -1

No Data leakage

Difficulty: Hard
Commit Log
Meaningful commit messages

Progressive commits used
Documentation
ReadMe acceptable, could use more info on model/background -1

Good Description and Comments

Markdown used PDF submitted
Pull Request
Successful Pull Request (Working Algorithm Delivered on Time in Correct Branch)

Feedback required, remove model files, repo is not supposed to have models because of size, fix conflicts -2

Request Description OK, but could use more info -1
@4lon
4lon
commented
Nov 24, 2022

Copy link


Copy Markdown

Author
Apologies for the conflicts and including the model files, I have merged the changes and removed the model files. I think this should address all the feedback. Please let me know if there are any other issues.
Sign up for freeto join this conversation on GitHub.
Already have an account?
Sign in to comment

Reviewers


No reviews


Assignees


No one assigned


Labels

Feedback_NeededVQVAE

Projects


None yet


Milestone


No milestone


Development

Successfully merging this pull request may close these issues.

4 participants

@4lon@SiyuLiu0329@shakes76@LinfengLiu98

, 'i');
if (__m === '*' || __re.test(location.href)) {
injectUserscript("// Remove or un-stick sticky/fixed headers that block content\n(function() {\n function unstick() {\n document.querySelectorAll('header, nav, [role=\"banner\"], .header, .navbar, .sticky, .fixed-top, [style*=\"position: fixed\"], [style*=\"position:sticky\"]').forEach(function(el) {\n if (el.style.position === 'fixed' || el.style.position === 'sticky' || \n getComputedStyle(el).position === 'fixed' || getComputedStyle(el).position === 'sticky') {\n el.style.position = 'static';\n el.style.top = 'auto';\n el.style.zIndex = 'auto';\n }\n });\n }\n \n unstick();\n \n var observer = new MutationObserver(unstick);\n observer.observe(document.body, { childList: true, subtree: true, attributes: true, attributeFilter: ['style', 'class'] });\n})();", "Kill Sticky Headers");
}
} catch(__e) { console.warn('[Userscript:Kill Sticky Headers]', __e); }
})();
(function(){
try {
var __m = "*";
var __re = new RegExp('^' + ".*" + '

Skip to content
Topic recognition - #469
Open
4lon wants to merge 63 commits into
shakes76:topic-recognitionfrom 
4lon:topic-recognition
Open
Topic recognition#469
4lon wants to merge 63 commits into
shakes76:topic-recognitionfrom 
4lon:topic-recognition
Conversation
@4lon
@4lon4lon
commented
Oct 21, 2022

Copy link


Copy Markdown

Hello Teaching Team,
This is a request to merge my attempt at VQVAE using OASIS brain dataset.
Let me know if there are any problems.
Regards,

Alon Nusem - 44801582
4lon
added 18 commits
October 11, 2022 16:53
@4lon
Initial commit and filling out basic project structure.
d4943af
@4lon
Added basic dataset class that loads all images in directory and retu…
32876fb
…rn samples for learning.
@4lon
Added template for VAE
0c9dc73
@4lon
Corrected error in dataset class location, moved from train to dataset.
0d501b0
@4lon
Added Vector Quantizer
4176432
@4lon
Updated encoder and decoders
067b1fc
@4lon
Fixed wrong layer errors and added vqvae model, likely need to fix pa…
d9e43b3
…dding issue.
@4lon
Updated to use pytorch embeddings and fleshed out vector quantizer.
5cb7a40
@4lon
Cleanup
662ef51
@4lon
Cleanup and redid vq_vae class to add more individual functions. This…
d2611f7
… should be useful later for implementation and prediction.
@4lon
Updated to make dataloaders in dataset.py
eb289e9
@4lon
Basis of training script setting up
dbea67a
@4lon
Built out saving model and modularised functions
43cc875
@4lon
Fixed up incorrect dataset formatting and loading of imgs
6a60fba
@4lon
Fleshed out training function
40bb45c
@4lon
Fleshed out testing function
eecfde1
@4lon
Added image generation and saving to tensorboard for visual analysis
aa65485
@4lon
Refactored modules to include original functions file and updated dat…
9ef9451
…aset location.
4lon
added 10 commits
November 7, 2022 15:40
@4lon
Increase HPC utilisation
670cc67
@4lon
add checks
3c7ebce
@4lon
Increased Epochs
968ad8a
@4lon
Original pytorch attempt was not working, start of tensorflow attempt…
cd008ee
…, reimplemented the dataset, however this current implementation is loading every image into an array, that's far from ideal so refactoring would be good time permitting.
@4lon
Old train model, not relevant anymore but good for looking back and n…
eee3087
…ot ready to remove
@4lon
Redesigned VQVAE modules for tensorflow. This is a simpler design ove…
65a76a4
…rall however time is limited. If I have time I intend to work back to this and improve the model
@4lon
Reimplemented basic training method, but this one does not evaluate a…
c54bd95
…gainst other data partitions or save sample outputs.
@4lon
Fixed some errors encountered when trying to train because of tracker…
00490b4
… dictionary, and updated directory to dataset.
@4lon
Added plotting of results including losses and reconstruction post le…
09ca20c
…arning
@4lon
Updated directory
53811cf
4lon
added 20 commits
November 7, 2022 15:40
@4lon
Built basic masked convolution layer and limited dataset size (memory…
d32fda4
… issues)
@4lon
Implemented residual block for pixelcnn
12bbfa6
@4lon
Implemented basic pixel cnn architecture but not sure it works.
31bcdef
@4lon
Added pixelcnn training to learn codebook production
c0ede93
@4lon
Updated plotting to also save pixel cnn performance
cd4838a
@4lon
Updated training with ssim metrics
3660c53
@4lon
reduced dataset size
b1e9fda
@4lon
reduced dataset size
fafcd18
@4lon
reduced dataset size
b9d4ce0
@4lon
reduced dataset size
2de0a80
@4lon
Added sample model outputs
570a4b6
@4lon
Added better sample model outputs
df901ca
@4lon
Refactored training to seperate pixel cnn and vqvae in case of sepera…
14d2ebb
…te training (this helps quite a bit with the memory problems of the dataset implementation.
@4lon
Fleshing out report
b193525
@4lon
Fleshing out report
a15a457
@4lon
Add brain generating function but doesn't really work.
19d4895
@4lon
Removed old attempt in pytorch and added snapshots of inference.
9de9e81
@4lon
Fixed up incorrect function call from refactor and add SSIM evaulatio…
3fb1dfc
…n and recontruction to predict.py
@4lon
Finalising report
fcbbb82
@4lon
Fixed reconstruction a bit
0a71411
@SiyuLiu0329
SiyuLiu0329
commented
Nov 9, 2022

Copy link


Copy Markdown

Collaborator
This is an initial inspection, no action is required at this point
  • recon: OK
  • image gen: Model seems to be working with minor issues
  • code could use more comments
@shakes76
shakes76
commented
Nov 20, 2022

Copy link


Copy Markdown

Owner
Good Practice (Design/Commenting, TF/Torch Usage)
Adequate use and implementation

Good spacing and comments

Header blocks missing -1
Recognition Problem
Solves problem (poor generations) -1

Driver Script present

File structure present

Shows Usage & Demo & Visualisation & Data usage

Module present

Commenting could be more informative -1

No Data leakage

Difficulty: Hard
Commit Log
Meaningful commit messages

Progressive commits used
Documentation
ReadMe acceptable, could use more info on model/background -1

Good Description and Comments

Markdown used PDF submitted
Pull Request
Successful Pull Request (Working Algorithm Delivered on Time in Correct Branch)

Feedback required, remove model files, repo is not supposed to have models because of size, fix conflicts -2

Request Description OK, but could use more info -1
@4lon
4lon
commented
Nov 24, 2022

Copy link


Copy Markdown

Author
Apologies for the conflicts and including the model files, I have merged the changes and removed the model files. I think this should address all the feedback. Please let me know if there are any other issues.
Sign up for freeto join this conversation on GitHub.
Already have an account?
Sign in to comment

Reviewers


No reviews


Assignees


No one assigned


Labels

Feedback_NeededVQVAE

Projects


None yet


Milestone


No milestone


Development

Successfully merging this pull request may close these issues.

4 participants

@4lon@SiyuLiu0329@shakes76@LinfengLiu98

, 'i');
if (__m === '*' || __re.test(location.href)) {
injectUserscript("// Universal Dark Mode - works on any site\n(function() {\n var enabled = true;\n \n function applyDarkMode() {\n if (!enabled) return;\n \n // Create style element if it doesn't exist\n var style = document.getElementById('universal-dark-mode-style');\n if (!style) {\n style = document.createElement('style');\n style.id = 'universal-dark-mode-style';\n document.head.appendChild(style);\n }\n \n // Dark mode CSS - inverts colors but preserves images/video\n style.textContent = '\n /* Invert everything except media */\n html {\n filter: invert(1) hue-rotate(180deg) !important;\n background: #1a1a2e !important;\n }\n \n /* Restore images, videos, iframes, canvas */\n img, video, iframe, canvas, svg, picture, [style*=\"background-image\"] {\n filter: invert(1) hue-rotate(180deg) !important;\n }\n \n /* Preserve specific elements that should not be inverted */\n .no-dark-mode, .no-dark-mode *,\n [data-theme=\"light\"], [data-theme=\"light\"],\n .ace_editor, .ace_editor *,\n .CodeMirror, .CodeMirror *,\n .monaco-editor, .monaco-editor *,\n .markdown-body pre, .markdown-body pre *,\n .highlight, .highlight *,\n pre code, pre code * {\n filter: none !important;\n }\n \n /* Fix common UI elements */\n .modal, .popup, .dropdown-menu, .tooltip, .popover {\n filter: invert(1) hue-rotate(180deg) !important;\n background: #2d2d44 !important;\n border-color: #444 !important;\n }\n \n /* Scrollbars */\n ::-webkit-scrollbar { background: #1a1a2e !important; }\n ::-webkit-scrollbar-thumb { background: #444 !important; }\n ::-webkit-scrollbar-thumb:hover { background: #555 !important; }\n \n /* Selection */\n ::selection { background: #4ecdc4 !important; color: #1a1a2e !important; }\n ::-moz-selection { background: #4ecdc4 !important; color: #1a1a2e !important; }\n ';\n }\n \n function removeDarkMode() {\n var style = document.getElementById('universal-dark-mode-style');\n if (style) style.remove();\n }\n \n // Toggle with Alt+Shift+D\n document.addEventListener('keydown', function(e) {\n if (e.altKey && e.shiftKey && e.key === 'D') {\n e.preventDefault();\n enabled = !enabled;\n if (enabled) {\n applyDarkMode();\n console.log('[Universal Dark Mode] Enabled');\n } else {\n removeDarkMode();\n console.log('[Universal Dark Mode] Disabled');\n }\n }\n });\n \n // Apply on load\n applyDarkMode();\n \n // Re-apply on dynamic content\n var observer = new MutationObserver(function(mutations) {\n if (enabled && !document.getElementById('universal-dark-mode-style')) {\n applyDarkMode();\n }\n });\n observer.observe(document.head, { childList: true });\n \n console.log('[Universal Dark Mode] Loaded - Press Alt+Shift+D to toggle');\n})();", "Universal Dark Mode");
}
} catch(__e) { console.warn('[Userscript:Universal Dark Mode]', __e); }
})();
})();

Skip to content
Topic recognition - #469
Open
4lon wants to merge 63 commits into
shakes76:topic-recognitionfrom 
4lon:topic-recognition
Open
Topic recognition#469
4lon wants to merge 63 commits into
shakes76:topic-recognitionfrom 
4lon:topic-recognition
Conversation
@4lon
@4lon4lon
commented
Oct 21, 2022

Copy link


Copy Markdown

Hello Teaching Team,
This is a request to merge my attempt at VQVAE using OASIS brain dataset.
Let me know if there are any problems.
Regards,

Alon Nusem - 44801582
4lon
added 18 commits
October 11, 2022 16:53
@4lon
Initial commit and filling out basic project structure.
d4943af
@4lon
Added basic dataset class that loads all images in directory and retu…
32876fb
…rn samples for learning.
@4lon
Added template for VAE
0c9dc73
@4lon
Corrected error in dataset class location, moved from train to dataset.
0d501b0
@4lon
Added Vector Quantizer
4176432
@4lon
Updated encoder and decoders
067b1fc
@4lon
Fixed wrong layer errors and added vqvae model, likely need to fix pa…
d9e43b3
…dding issue.
@4lon
Updated to use pytorch embeddings and fleshed out vector quantizer.
5cb7a40
@4lon
Cleanup
662ef51
@4lon
Cleanup and redid vq_vae class to add more individual functions. This…
d2611f7
… should be useful later for implementation and prediction.
@4lon
Updated to make dataloaders in dataset.py
eb289e9
@4lon
Basis of training script setting up
dbea67a
@4lon
Built out saving model and modularised functions
43cc875
@4lon
Fixed up incorrect dataset formatting and loading of imgs
6a60fba
@4lon
Fleshed out training function
40bb45c
@4lon
Fleshed out testing function
eecfde1
@4lon
Added image generation and saving to tensorboard for visual analysis
aa65485
@4lon
Refactored modules to include original functions file and updated dat…
9ef9451
…aset location.
4lon
added 10 commits
November 7, 2022 15:40
@4lon
Increase HPC utilisation
670cc67
@4lon
add checks
3c7ebce
@4lon
Increased Epochs
968ad8a
@4lon
Original pytorch attempt was not working, start of tensorflow attempt…
cd008ee
…, reimplemented the dataset, however this current implementation is loading every image into an array, that's far from ideal so refactoring would be good time permitting.
@4lon
Old train model, not relevant anymore but good for looking back and n…
eee3087
…ot ready to remove
@4lon
Redesigned VQVAE modules for tensorflow. This is a simpler design ove…
65a76a4
…rall however time is limited. If I have time I intend to work back to this and improve the model
@4lon
Reimplemented basic training method, but this one does not evaluate a…
c54bd95
…gainst other data partitions or save sample outputs.
@4lon
Fixed some errors encountered when trying to train because of tracker…
00490b4
… dictionary, and updated directory to dataset.
@4lon
Added plotting of results including losses and reconstruction post le…
09ca20c
…arning
@4lon
Updated directory
53811cf
4lon
added 20 commits
November 7, 2022 15:40
@4lon
Built basic masked convolution layer and limited dataset size (memory…
d32fda4
… issues)
@4lon
Implemented residual block for pixelcnn
12bbfa6
@4lon
Implemented basic pixel cnn architecture but not sure it works.
31bcdef
@4lon
Added pixelcnn training to learn codebook production
c0ede93
@4lon
Updated plotting to also save pixel cnn performance
cd4838a
@4lon
Updated training with ssim metrics
3660c53
@4lon
reduced dataset size
b1e9fda
@4lon
reduced dataset size
fafcd18
@4lon
reduced dataset size
b9d4ce0
@4lon
reduced dataset size
2de0a80
@4lon
Added sample model outputs
570a4b6
@4lon
Added better sample model outputs
df901ca
@4lon
Refactored training to seperate pixel cnn and vqvae in case of sepera…
14d2ebb
…te training (this helps quite a bit with the memory problems of the dataset implementation.
@4lon
Fleshing out report
b193525
@4lon
Fleshing out report
a15a457
@4lon
Add brain generating function but doesn't really work.
19d4895
@4lon
Removed old attempt in pytorch and added snapshots of inference.
9de9e81
@4lon
Fixed up incorrect function call from refactor and add SSIM evaulatio…
3fb1dfc
…n and recontruction to predict.py
@4lon
Finalising report
fcbbb82
@4lon
Fixed reconstruction a bit
0a71411
@SiyuLiu0329
SiyuLiu0329
commented
Nov 9, 2022

Copy link


Copy Markdown

Collaborator
This is an initial inspection, no action is required at this point
  • recon: OK
  • image gen: Model seems to be working with minor issues
  • code could use more comments
@shakes76
shakes76
commented
Nov 20, 2022

Copy link


Copy Markdown

Owner
Good Practice (Design/Commenting, TF/Torch Usage)
Adequate use and implementation

Good spacing and comments

Header blocks missing -1
Recognition Problem
Solves problem (poor generations) -1

Driver Script present

File structure present

Shows Usage & Demo & Visualisation & Data usage

Module present

Commenting could be more informative -1

No Data leakage

Difficulty: Hard
Commit Log
Meaningful commit messages

Progressive commits used
Documentation
ReadMe acceptable, could use more info on model/background -1

Good Description and Comments

Markdown used PDF submitted
Pull Request
Successful Pull Request (Working Algorithm Delivered on Time in Correct Branch)

Feedback required, remove model files, repo is not supposed to have models because of size, fix conflicts -2

Request Description OK, but could use more info -1
@4lon
4lon
commented
Nov 24, 2022

Copy link


Copy Markdown

Author
Apologies for the conflicts and including the model files, I have merged the changes and removed the model files. I think this should address all the feedback. Please let me know if there are any other issues.
Sign up for freeto join this conversation on GitHub.
Already have an account?
Sign in to comment

Reviewers


No reviews


Assignees


No one assigned


Labels

Feedback_NeededVQVAE

Projects


None yet


Milestone


No milestone


Development

Successfully merging this pull request may close these issues.

4 participants

@4lon@SiyuLiu0329@shakes76@LinfengLiu98