Restructuring 2 - Add Utility functions and Run all Tests by kartikdutt18 · Pull Request #12 · mlpack/models

kartikdutt18 · 2020-05-26T06:23:51Z

This PR is transferred from #8, Since FFN weights some time cause invalid memory access which seems related to mlpack/mlpack#2314. So I am adding utility function to unzip files here and I'll continue working on image dataloader and then we can get back to the LeNet model.
I hope this makes sense.
This PR provides the following :

Unzip Utility Function.
Some bug fixes.
Runs all tests.
(Probably we should get mnist_test/train in mnist.tar.gz since mnist test and train are fixed).
ListDir Utility function.

Looking forward to your suggestions.
Regards.

CC: @KimSangYeon-DGU

kartikdutt18 · 2020-05-27T15:09:35Z

This works on windows too now, Cleaning commits and outputs.

zoq · 2020-05-27T21:07:22Z

(Probably we should get mnist_test/train in mnist.tar.gz since mnist test and train are fixed).

Sure, we can do that, is there anything else you like to add here, this looks pretty good to me.

kartikdutt18 · 2020-05-28T03:09:33Z

Sure, we can do that, is there anything else you like to add here, this looks pretty good to me.

I am done with this PR from my side aside from placing mnist_train/test in mnist.tar.gz. If this makes sense I'll ask on the IRC once I incorporate all suggestions.

zoq · 2020-05-28T21:30:33Z

Makes sense to me.

zoq

From my side this looks ready to go.

KimSangYeon-DGU · 2020-05-28T23:30:14Z

Looks good to me as well, please let us know if this is ready to go :)

Trained a lenet1 model Add All Weights

Completed everything Fix build, Style fixes next Use force local for windows while unzipping tar files Use force local for windows while unzipping tar files

Added Utility Function Style Fix Fix Typo causing build error in windows Fix Typo causing build error in windows Fix const issue for windows Extract in data folder Reposition force local This should work Print Path in windows for debugging Print Path in windows for debugging Print Path in windows for debugging Stip components of tar Stip components of tar strip componenets Relative Path Relative Path Relative Path Add List Dir utility function Print Print again Print again Squash this Update utils.hpp Squash this Clean Up Added more tests Style Fix Style Fix

kartikdutt18 · 2020-05-29T16:57:38Z

This is ready from my side. Just tested it everything works fine. I think the builds will result in the same output.
Regards.

kartikdutt18 · 2020-05-30T10:21:39Z

Sorry pushed in the wrong branch, reverting now. Fixed now.

KimSangYeon-DGU · 2020-05-31T07:17:02Z

+
+

Can you use 1 newline and add a comment about what we're checking in this block?
And, perhaps, I missed something... can you let me know why the data were distributed like below?
Train : 8,400
Valid: 33,600
Test: 28,000

This is the same split as mlpack::data::split. Here 0.8 means 80% testing and 20 % training data. mlpack takes in test ratio. I though it would be best not to invert. Let me know if I should do that or keep it same as mlpack. Here is the link for data file.

Ok, then, it would be better to change the parameter name ratio to testRatio for clarification in DataLoader class :)

Sure, I'll make the change.

Also, kindly update the parameter description as well in here

Ah, ok. I thought it's from the same file :)

Should I use validRatio instead I think that makes even more sense.

Nice. that makes sense to me as well.

Right, making the changes.

KimSangYeon-DGU

Ah, ok, I totally understand 👍 :)

Looks good to me.

kartikdutt18 · 2020-05-31T09:15:56Z

Great. Thanks a lot for the reviews.

mlpack-bot

Second approval provided automatically after 24 hours. 👍

KimSangYeon-DGU · 2020-06-02T02:06:59Z

Thanks! @kartikdutt18 👍

kartikdutt18 · 2020-06-02T02:57:42Z

Thanks @KimSangYeon-DGU, @zoq for the reviews and all the help.

mlpack-botBot added s: needs review s: unanswered s: unlabeled labels May 26, 2020

kartikdutt18 force-pushed the Restructuring-2-a-UtilityFunctions branch 4 times, most recently from 7a3666b to 07d732aCompare May 27, 2020 14:15

kartikdutt18 force-pushed the Restructuring-2-a-UtilityFunctions branch from 07d732a to 3d74ee1Compare May 27, 2020 15:11

zoq added t: added feature and removed s: unanswered s: unlabeled labels May 27, 2020

zoq reviewed May 28, 2020
View reviewed changes

KimSangYeon-DGU reviewed May 28, 2020
View reviewed changes

Comment threadtests/dataloader_tests.cpp Outdated
Comment threadutils/utils.hpp Outdated

kartikdutt18 added 2 commits May 29, 2020 22:04

Add basic definition of models, Needs to be trained and tested
c4113eb
Trained a lenet1 model Add All Weights

Add unzip utility function, fix linux build
78aa733
Completed everything Fix build, Style fixes next Use force local for windows while unzipping tar files Use force local for windows while unzipping tar files

kartikdutt18 force-pushed the Restructuring-2-a-UtilityFunctions branch from 3d74ee1 to 9d90a21Compare May 29, 2020 16:35

kartikdutt18 force-pushed the Restructuring-2-a-UtilityFunctions branch from 585905f to 6ae432eCompare May 29, 2020 16:56

kartikdutt18 force-pushed the Restructuring-2-a-UtilityFunctions branch from e51aaa9 to 6ae432eCompare May 30, 2020 10:23

kartikdutt18 requested a review from KimSangYeon-DGU May 31, 2020 04:24

KimSangYeon-DGU reviewed May 31, 2020
View reviewed changes

Change variable to test ratio and add more comments in tests
040decb

KimSangYeon-DGU reviewed May 31, 2020
View reviewed changes

Comment threadtests/dataloader_tests.cpp Outdated

kartikdutt18 added 2 commits May 31, 2020 14:39

Rename to validRatio
bb66ce8

Rename to validRatio
ed7a643

kartikdutt18 force-pushed the Restructuring-2-a-UtilityFunctions branch from 66cd0d7 to ed7a643Compare May 31, 2020 09:14

KimSangYeon-DGU approved these changes May 31, 2020
View reviewed changes

KimSangYeon-DGU added s: needs review and removed s: needs review labels May 31, 2020

mlpack-botBot approved these changes Jun 1, 2020
View reviewed changes

mlpack-botBot removed the s: needs review label Jun 1, 2020

KimSangYeon-DGU merged commit f4d4a17 into mlpack:masterJun 2, 2020

KimSangYeon-DGU mentioned this pull request Jun 2, 2020
Add Image Dataloader for PASCAL VOC and CIFAR 10 dataset. #13
Closed
6 tasks

Uh oh!

Conversation

kartikdutt18 commented May 26, 2020• edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

kartikdutt18 commented May 27, 2020• edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

zoq commented May 27, 2020

Uh oh!

kartikdutt18 commented May 28, 2020

Uh oh!

zoq commented May 28, 2020

Uh oh!

zoq left a comment

Choose a reason for hiding this comment

Uh oh!

KimSangYeon-DGU commented May 28, 2020

Uh oh!

Uh oh!

Uh oh!

kartikdutt18 commented May 29, 2020

Uh oh!

kartikdutt18 commented May 30, 2020• edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Uh oh!

KimSangYeon-DGUMay 31, 2020• edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

kartikdutt18May 31, 2020• edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

KimSangYeon-DGUMay 31, 2020• edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

kartikdutt18May 31, 2020

Choose a reason for hiding this comment

Uh oh!

KimSangYeon-DGUMay 31, 2020• edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

KimSangYeon-DGUMay 31, 2020• edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

kartikdutt18May 31, 2020

Choose a reason for hiding this comment

Uh oh!

KimSangYeon-DGUMay 31, 2020

Choose a reason for hiding this comment

Uh oh!

kartikdutt18May 31, 2020• edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

kartikdutt18May 31, 2020

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

KimSangYeon-DGU left a comment • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

kartikdutt18 commented May 31, 2020

Uh oh!

mlpack-botBot left a comment

Choose a reason for hiding this comment

Uh oh!

KimSangYeon-DGU commented Jun 2, 2020

Uh oh!

kartikdutt18 commented Jun 2, 2020

Uh oh!

Reviewers

Assignees

kartikdutt18 commented May 26, 2020•
edited
Loading

kartikdutt18 commented May 27, 2020•
edited
Loading

kartikdutt18 commented May 30, 2020•
edited
Loading

KimSangYeon-DGUMay 31, 2020•
edited
Loading

kartikdutt18May 31, 2020•
edited
Loading

KimSangYeon-DGUMay 31, 2020•
edited
Loading

KimSangYeon-DGUMay 31, 2020•
edited
Loading

KimSangYeon-DGUMay 31, 2020•
edited
Loading

kartikdutt18May 31, 2020•
edited
Loading

KimSangYeon-DGU left a comment •
edited
Loading