update by aihao2000 · Pull Request #7067 · huggingface/diffusers

aihao2000 · 2024-02-23T07:19:32Z

What does this PR do?

load_dataset data_dir
Supports custom local datasets. Just like add load_datasete data_dir parameter #5747 (comment)
Update fingerprint hash parameters. Avoid irrelevant parameters causing repeated calculations in the map cache

Fixes # (issue)

Before submitting

This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
Did you read the contributor guideline?
Did you read our philosophy doc (important for complex PRs)?
Was this discussed/approved via a GitHub issue or the forum? Please add a link to it if that's the case.
Did you make sure to update the documentation with your changes? Here are the
documentation guidelines, and
here are tips on formatting docstrings.
Did you write any new necessary tests?

Who can review?

Anyone in the community is free to review the PR once the tests have passed. Feel free to tag
members/contributors who may be interested in your PR.

@sayakpaul and @patrickvonplaten

sayakpaul · 2024-02-23T07:26:27Z

Thanks for your contributions! Can we please break this PR into multiple PRs as it targets multiple mutually exclusive areas?

aihao2000 · 2024-02-23T09:05:37Z

@sayakpaul Yes, feel free to change it.

sayakpaul · 2024-02-23T09:17:54Z

Sorry for not being clear. I was asking you to tackle this in multiple PRs

aihao2000 · 2024-02-26T16:38:50Z

@sayakpaul I'm not familiar with git and it seems a little troublesome. Are there any doubtful changes? I can restore that part.

sayakpaul · 2024-02-26T16:41:12Z

It's a general good practice to not introduce multiple changes in a single PR. And I would like to follow that. For example, for each of the points you enlisted in the PR description, you could create PRs (four PRs).

github-actions · 2024-03-24T15:03:05Z

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the contributing guidelines are likely to be ignored.

sayakpaul · 2024-03-24T15:14:21Z

@aihao2000 a gentle ping here

aihao2000 · 2024-03-29T08:24:30Z

@sayakpaul IIt is now only about the addition of the data_dir parameter. Is this okay?

HuggingFaceDocBuilderDev · 2024-03-29T10:51:09Z

The docs for this PR live here. All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.

lhoestq

Cool :) ust one comment:

lhoestq · 2024-03-29T11:16:43Z

- if args.dataset_name is None and args.train_data_dir is None:
- raise ValueError("Need either a dataset name or a training folder.")


maybe keep this one ?

@lhoestq oh my bad

github-actions · 2024-04-22T15:03:50Z

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the contributing guidelines are likely to be ignored.

yiyixuxu · 2024-04-22T18:56:23Z

@aihao2000 can we address the last comment and get this merged?

aihao2000 · 2024-04-23T03:31:17Z

@yiyixuxu Should I submit a new commit to restore this code?

github-actions · 2024-09-14T15:18:29Z

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the contributing guidelines are likely to be ignored.

yiyixuxu · 2024-09-17T21:58:26Z

@aihao2000 yes

github-actions · 2024-10-12T15:09:09Z

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the contributing guidelines are likely to be ignored.

yiyixuxu · 2024-12-03T04:17:32Z

can you run make style?

yiyixuxu · 2024-12-03T21:26:56Z

thanks @hlky !

* add data_dir parameter to load_dataset --------- Co-authored-by: Sayak Paul <spsayakpaul@gmail.com> Co-authored-by: YiYi Xu <yixu310@gmail.com> Co-authored-by: hlky <hlky@hlky.ac>

github-actionsBot added the stale Issues that haven't received updates label Mar 24, 2024

github-actionsBot removed the stale Issues that haven't received updates label Mar 25, 2024

add data_dir parameter to load_dataset
77697d4

aihao2000 force-pushed the update-train-scripts branch from 3432404 to 77697d4Compare March 29, 2024 08:18

sayakpaul requested a review from lhoestq March 29, 2024 10:43

Merge branch 'main' into update-train-scripts
032b14f

lhoestq approved these changes Mar 29, 2024
View reviewed changes

github-actionsBot added the stale Issues that haven't received updates label Apr 22, 2024

yiyixuxu removed the stale Issues that haven't received updates label Apr 22, 2024

restore sanity checks
fcf5364

github-actionsBot added the stale Issues that haven't received updates label Sep 14, 2024

Merge branch 'main' into update-train-scripts
034b914

github-actionsBot removed the stale Issues that haven't received updates label Sep 18, 2024

github-actionsBot added the stale Issues that haven't received updates label Oct 12, 2024

yiyixuxu added close-to-merge and removed stale Issues that haven't received updates labels Dec 3, 2024

make
a98b946

yiyixuxu merged commit b58f67f into huggingface:mainDec 3, 2024

yiyixuxu removed the close-to-merge label Dec 3, 2024

sayakpaul added a commit that referenced this pull request Dec 23, 2024
update (#7067)
70557b9
* add data_dir parameter to load_dataset --------- Co-authored-by: Sayak Paul <spsayakpaul@gmail.com> Co-authored-by: YiYi Xu <yixu310@gmail.com> Co-authored-by: hlky <hlky@hlky.ac>

		if args.dataset_name is None and args.train_data_dir is None:
		raise ValueError("Need either a dataset name or a training folder.")

Uh oh!

Conversation

aihao2000 commented Feb 23, 2024• edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

What does this PR do?

Before submitting

Who can review?

Uh oh!

sayakpaul commented Feb 23, 2024

Uh oh!

aihao2000 commented Feb 23, 2024

Uh oh!

sayakpaul commented Feb 23, 2024

Uh oh!

aihao2000 commented Feb 26, 2024

Uh oh!

sayakpaul commented Feb 26, 2024

Uh oh!

github-actionsBot commented Mar 24, 2024

Uh oh!

sayakpaul commented Mar 24, 2024

Uh oh!

aihao2000 commented Mar 29, 2024

Uh oh!

HuggingFaceDocBuilderDev commented Mar 29, 2024

Uh oh!

lhoestq left a comment

Choose a reason for hiding this comment

Uh oh!

lhoestqMar 29, 2024

Choose a reason for hiding this comment

Uh oh!

aihao2000Mar 29, 2024

Choose a reason for hiding this comment

Uh oh!

github-actionsBot commented Apr 22, 2024

Uh oh!

yiyixuxu commented Apr 22, 2024

Uh oh!

aihao2000 commented Apr 23, 2024

Uh oh!

github-actionsBot commented Sep 14, 2024

Uh oh!

yiyixuxu commented Sep 17, 2024

Uh oh!

github-actionsBot commented Oct 12, 2024

Uh oh!

yiyixuxu commented Dec 3, 2024

Uh oh!

yiyixuxu commented Dec 3, 2024

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

6 participants

aihao2000 commented Feb 23, 2024•
edited
Loading