Skip to content

Fix autocast v fp16 + add benchmark config instructions in benchmark.md - #11

Draft
eolecvk wants to merge 5 commits into
mainfrom
eole/complete-benchmark-doc
Draft

Fix autocast v fp16 + add benchmark config instructions in benchmark.md#11
eolecvk wants to merge 5 commits into
mainfrom
eole/complete-benchmark-doc

Conversation

@eolecvk

@eolecvkeolecvk commented Oct 12, 2022

Copy link
Copy Markdown
Contributor
  • Fixed conditional logic when using autocast in benchmark code.
  • Updated diffusers==0.3.0 to diffusers==0.4.2
  • @chuanli11 : Any reason to not include torch and torchvision in the requirements?

Removing autocast does seem to reduce the latency and memory in a big way!
Here is what the new benchmark results for autocast v fp16 look like.

deviceprecisionautocastruntimen_sampleslatencymemory
NVIDIA RTX A6000singleFalsepytorch17.377.73
NVIDIA RTX A6000halfTruepytorch14.394.52
NVIDIA RTX A6000halfFalsepytorch13.343.38
NVIDIA RTX A6000singleFalsepytorch214.09.57
NVIDIA RTX A6000halfTruepytorch28.298.74
NVIDIA RTX A6000halfFalsepytorch26.174.92
NVIDIA RTX A6000singleFalsepytorch427.2518.4
NVIDIA RTX A6000halfTruepytorch415.7311.4
NVIDIA RTX A6000halfFalsepytorch411.598.82

Also added benchmark config instructions in benchmark.md

@chuanli11

Copy link
Copy Markdown
Contributor

@eolecvk seems torch and torchvision are included in the setup.py.

I don't know if @justinpinkney has particular reason for doing it that way.

@eolecvk

Copy link
Copy Markdown
ContributorAuthor

@eolecvk seems torch and torchvision are included in the setup.py.

I don't know if @justinpinkney has particular reason for doing it that way.

It's because the original repo was setup that way:
https://github.com/CompVis/stable-diffusion/blob/69ae4b35e0a0f6ee1af8bb9a5d0016ccb27e36dc/setup.py

...but even in their case, they install pytorch and torchvision in conda's environment.yaml:
https://github.com/CompVis/stable-diffusion/blob/69ae4b35e0a0f6ee1af8bb9a5d0016ccb27e36dc/environment.yaml

Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants

@eolecvk@chuanli11