Skip to content

Monitored Loss Calculations - #4030

Merged
mrwyattii merged 6 commits into
masterfrom
jomayeri/issue-3769
Jul 25, 2023
Merged

Monitored Loss Calculations#4030
mrwyattii merged 6 commits into
masterfrom
jomayeri/issue-3769

Conversation

@jomayeri

@jomayeri jomayeri commented Jul 25, 2023

Copy link
Copy Markdown
Collaborator

Updating the monitored loss calculation.
Steps:

  1. Divide batch loss tensor by gradient accumulation steps.
  2. Reduce batch loss tensor to scalar using mean.
  3. Accumulate loss for gradient accumulation steps.

@jomayeri
jomayeri requested review from jeffra and tjruwase as code owners July 25, 2023 00:21
@mrwyattii
mrwyattii enabled auto-merge July 25, 2023 17:04
@mrwyattii
mrwyattii added this pull request to the merge queue Jul 25, 2023
@jomayeri jomayeri linked an issue Jul 25, 2023 that may be closed by this pull request
[BUG] monitor loss seems not being correctly computed. #3769
Closed
Merged via the queue into master with commit eeab613 Jul 25, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@tjruwase tjruwase tjruwase approved these changes
@jeffra jeffra Awaiting requested review from jeffra

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

[BUG] monitor loss seems not being correctly computed.

3 participants

@jomayeri @tjruwase @mrwyattii