Skip to content

web ui: display in-progress images #181

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
lstein merged 2 commits into from
Aug 30, 2022
Merged

web ui: display in-progress images #181

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
0173d3a
stream images
bakkot Aug 29, 2022
18597ad
fix bug in pngwriter
bakkot Aug 29, 2022
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions ldm/dream/pngwriter.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -73,6 +73,7 @@ def unique_filename(self, seed, upscaled=False, previouspath=None):
if upscaled:
break
filename = f'{basecount:06}.{seed}.{series:02}.png'
path = os.path.join(self.outdir, filename)
finished = not os.path.exists(path)
return os.path.join(self.outdir, filename)

Expand Down
18 changes: 15 additions & 3 deletions ldm/dream/server.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -64,7 +64,8 @@ def do_POST(self):
upscale_level = post_data['upscale_level']
upscale_strength = post_data['upscale_strength']
upscale = [int(upscale_level),float(upscale_strength)] if upscale_level != '' else None
seed = None if int(post_data['seed']) == -1 else int(post_data['seed'])
progress_images = 'progress_images' in post_data
seed = self.model.seed if int(post_data['seed']) == -1 else int(post_data['seed'])

print(f"Request to generate with prompt: {prompt}")
# In order to handle upscaled images, the PngWriter needs to maintain state
Expand Down Expand Up @@ -116,9 +117,20 @@ def image_done(image, seed, upscaled=False):
{'event':action,'processed_file_cnt':f'{x}/{iterations}'}

@TesseractCat TesseractCat Aug 29, 2022

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Probably should be processed_file_count, also since this is intended to be consumed by JS, maybe this should be processedFileCount.

@bakkot bakkot Aug 29, 2022

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

That's not related to this PR.

) + '\n',"utf-8"))

def image_progress(image, step):
# TODO: refactor PngWriter:
# it doesn't need to know if batch_size > 1, just if this is _part of a batch_
step_writer = PngWriter('./outputs/intermediates/', prompt, 2)
def image_progress(sample, step):
url = None
# since rendering images is moderately expensive, only render every 5th image
# and don't bother with the last one, since it'll render anyway
if progress_images and step % 5 == 0 and step < steps - 1:
images = self.model._samples_to_images(sample)
image = images[0]
step_writer.write_image(image, seed) # TODO PngWriter to return path
url = step_writer.filepath
self.wfile.write(bytes(json.dumps(
{'event':'step', 'step':step}
{'event':'step', 'step':step, 'url': url}
) + '\n',"utf-8"))

if initimg is None:
Expand Down
9 changes: 8 additions & 1 deletion static/dream_web/index.css
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -46,7 +46,7 @@ fieldset {
margin: auto;
padding-top: 10px;
}
img {
#results img {
cursor: pointer;
height: 30vh;
border-radius: 5px;
Expand All @@ -67,3 +67,10 @@ hr {
label {
white-space: nowrap;
}
#progress-section {
display: none;
}
#progress-image {
width: 30vh;
height: 30vh;
}
14 changes: 10 additions & 4 deletions static/dream_web/index.html
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -67,6 +67,9 @@ <h2 id="header">Stable Diffusion Dream Server</h2>
<button type="button" id="reset-seed">&olarr;</button>
<span>&bull;</span>
<button type="button" id="reset-all">Reset to Defaults</button>
<br>
<label for="progress_images">Display in-progress images (slows down generation):</label>
<input type="checkbox" name="progress_images" id="progress_images">

@TesseractCat TesseractCat Aug 29, 2022

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Quick consistency nitpick, can this be named progress-images?

@bakkot bakkot Aug 29, 2022

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sure. The underscore was just for consistency with gfpgan_strength immediately below.

@bakkot bakkot Aug 29, 2022

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actually, this file seems to consistently use underscores for names, so I've left it as-is for now.

@lstein lstein Aug 30, 2022

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@TesseractCat are there still issues, or are we good to go? (I've re-requested a review. Hope that's etiquette.)

@bakkot bakkot Aug 30, 2022

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@lstein could this get merged as-is, and then @TesseractCat can submit any further style nits in a followup PR? I'd like to stop needing to rebase this PR, and to get #182 in.

<div id="gfpgan">
<p><em>The options below require the GFPGAN and ESRGAN packages to be installed</em></p>
<label title="Strength of the gfpgan (face fixing) algorithm." for="gfpgan_strength">GPFGAN Strength:</label>
Expand All @@ -83,10 +86,13 @@ <h2 id="header">Stable Diffusion Dream Server</h2>
</fieldset>
</form>
<div id="about">For news and support for this web service, visit our <a href="http://github.com/lstein/stable-diffusion">GitHub site</a></div>
<br>
<progress id="progress" value="0" max="1"></progress>
<div id="scaling-inprocess-message">
<i><span>Postprocessing...</span><span id="processing_cnt">1/3</span></i>
<div id="progress-section">
<progress id="progress-bar" value="0" max="1"></progress>
<br>
<img id="progress-image" src='data:image/svg+xml,<svg xmlns="http://www.w3.org/2000/svg"/>'></img>
<div id="scaling-inprocess-message">
<i><span>Postprocessing...</span><span id="processing_cnt">1/3</span></i>
</div>
</div>
</div>
<div id="results">
Expand Down
35 changes: 26 additions & 9 deletions static/dream_web/index.js
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -53,14 +53,22 @@ function clearFields(form) {
form.prompt.value = prompt;
}

const BLANK_IMAGE_URL = 'data:image/svg+xml,<svg xmlns="http://www.w3.org/2000/svg"/>';
async function generateSubmit(form) {
const prompt = document.querySelector("#prompt").value;

// Convert file data to base64
let formData = Object.fromEntries(new FormData(form));
formData.initimg = formData.initimg.name !== '' ? await toBase64(formData.initimg) : null;

document.querySelector('progress').setAttribute('max', formData.steps);
let progressSectionEle = document.querySelector('#progress-section');
progressSectionEle.style.display = 'initial';
let progressEle = document.querySelector('#progress-bar');
progressEle.setAttribute('max', formData.steps);
let progressImageEle = document.querySelector('#progress-image');
progressImageEle.src = BLANK_IMAGE_URL;

progressImageEle.style.display = {}.hasOwnProperty.call(formData, 'progress_images') ? 'initial': 'none';

@TesseractCat TesseractCat Aug 29, 2022

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why not just do formData.hasOwnProperty?

@bakkot bakkot Aug 29, 2022

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You should never do foo.hasOwnProperty. That will break if foo has an own property named hasOwnProperty, or if it has a null prototype. In this particular instance we know it won't, but this is still the idiomatic way to write this test.


// Post as JSON, using Fetch streaming to get results
fetch(form.action, {
Expand All @@ -73,22 +81,31 @@ async function generateSubmit(form) {
while (true) {
let {value, done} = await reader.read();
value = new TextDecoder().decode(value);
if (done) break;
if (done) {
progressSectionEle.style.display = 'none';
break;
}

for (let event of value.split('\n').filter(e => e !== '')) {
const data = JSON.parse(event);

if (data.event == 'result') {
noOutputs = false;
document.querySelector("#no-results-message")?.remove();
appendOutput(data.files[0],data.files[1],data.config)
} else if (data.event == 'upscaling-started') {
document.getElementById("processing_cnt").textContent=data.processed_file_cnt;
document.getElementById("scaling-inprocess-message").style.display = "block";
} else if (data.event == 'upscaling-done') {
document.getElementById("scaling-inprocess-message").style.display = "none";
appendOutput(data.files[0],data.files[1],data.config);
progressEle.setAttribute('value', 0);
progressEle.setAttribute('max', formData.steps);
progressImageEle.src = BLANK_IMAGE_URL;
} else if (data.event == 'upscaling-started') {
document.getElementById("processing_cnt").textContent=data.processed_file_cnt;
document.getElementById("scaling-inprocess-message").style.display = "block";
} else if (data.event == 'upscaling-done') {
document.getElementById("scaling-inprocess-message").style.display = "none";
} else if (data.event == 'step') {
document.querySelector('progress').setAttribute('value', data.step.toString());
progressEle.setAttribute('value', data.step);
if (data.url) {
progressImageEle.src = data.url;
}
}
}
}
Expand Down