Skip to content

ControlNet crashes with high-resolution reference images when using Canny preprocessor #642

Description

@rmatif

ControlNet crashes when processing high-resolution reference images with the Canny preprocessor enabled. The issue only occurs with large images while smaller resolutions (like 512x512) work fine

./build/bin/sd -m models/realisticVisionV60B1_v51HyperVAE.safetensors --control-net models/control_v11p_sd15_canny.pth --control-image input_test.png --canny -W 640 -H 512 -v -p "cute cat, lego style" --type q8_0
Option: n_threads: 2
mode: txt2img
model_path: models/realisticVisionV60B1_v51HyperVAE.safetensors
wtype: q8_0
clip_l_path: clip_g_path: t5xxl_path: diffusion_model_path: vae_path: taesd_path: esrgan_path: controlnet_path: models/control_v11p_sd15_canny.pth
embeddings_path: stacked_id_embeddings_path: input_id_images_path: style ratio: 20.00
normalize input image :false
output_path: output.png
init_img: mask_img: control_image: input_test.png
clip on cpu: false
controlnet cpu: false
vae decoder on cpu:false
diffusion flash attention:false
strength(control): 0.90
prompt: cute cat, lego style
negative_prompt: min_cfg: 1.00
cfg_scale: 7.00
slg_scale: 0.00
guidance: 3.50
eta: 0.00
clip_skip: -1
width: 640
height: 512
sample_method: euler_a
schedule: default
sample_steps: 20
strength(img2img): 0.75
rng: cuda
seed: 42
batch_count: 1
vae_tiling: false
upscale_repeats: 1
System Info: SSE3 = 1
AVX = 1
AVX2 = 1
AVX512 = 1
AVX512_VBMI = 1
AVX512_VNNI = 1
FMA = 1
NEON = 0
ARM_FMA = 0
F16C = 1
FP16_VA = 0
WASM_SIMD = 0
VSX = 0
[DEBUG] stable-diffusion.cpp:188 - Using CPU backend
[INFO ] stable-diffusion.cpp:197 - loading model from 'models/realisticVisionV60B1_v51HyperVAE.safetensors'
[INFO ] model.cpp:908 - load models/realisticVisionV60B1_v51HyperVAE.safetensors using safetensors format
[DEBUG] model.cpp:979 - init from 'models/realisticVisionV60B1_v51HyperVAE.safetensors'
[INFO ] stable-diffusion.cpp:244 - Version: SD 1.x [INFO ] stable-diffusion.cpp:277 - Weight type: q8_0
[INFO ] stable-diffusion.cpp:278 - Conditioner weight type: q8_0
[INFO ] stable-diffusion.cpp:279 - Diffusion model weight type: q8_0
[INFO ] stable-diffusion.cpp:280 - VAE weight type: q8_0
[DEBUG] stable-diffusion.cpp:282 - ggml tensor size = 400 bytes
[DEBUG] clip.hpp:171 - vocab size: 49408
[DEBUG] clip.hpp:182 - trigger word img already in vocab
[DEBUG] ggml_extend.hpp:1174 - clip params backend buffer size = 231.50 MB(RAM) (196 tensors)
[DEBUG] ggml_extend.hpp:1174 - unet params backend buffer size = 1400.64 MB(RAM) (686 tensors)
[DEBUG] ggml_extend.hpp:1174 - vae params backend buffer size = 94.47 MB(RAM) (140 tensors)
[DEBUG] stable-diffusion.cpp:419 - loading weights
[DEBUG] model.cpp:1727 - loading tensors from models/realisticVisionV60B1_v51HyperVAE.safetensors
|==================================================| 1130/1130 - 125.00it/s
[INFO ] control.hpp:434 - loading control net from 'models/control_v11p_sd15_canny.pth'
[DEBUG] ggml_extend.hpp:1174 - control_net params backend buffer size = 937.60 MB(RAM) (340 tensors)
[INFO ] model.cpp:911 - load models/control_v11p_sd15_canny.pth using checkpoint format
[DEBUG] model.cpp:1445 - init from 'models/control_v11p_sd15_canny.pth'
ZIP 0, name = control_v11p_sd15_canny/data.pkl, dir = control_v11p_sd15_canny/ [DEBUG] model.cpp:1727 - loading tensors from models/control_v11p_sd15_canny.pth
|==================================================| 340/340 - 0.00it/s
[INFO ] control.hpp:453 - control net model loaded
[INFO ] stable-diffusion.cpp:503 - total params memory size = 2664.21MB (VRAM 937.60MB, RAM 1726.61MB): clip 231.50MB(RAM), unet 1400.64MB(RAM), vae 94.47MB(RAM), controlnet 937.60MB(VRAM), pmid 0.00MB(RAM)
[INFO ] stable-diffusion.cpp:522 - loading model from 'models/realisticVisionV60B1_v51HyperVAE.safetensors' completed, taking 15.86s
[INFO ] stable-diffusion.cpp:556 - running in eps-prediction mode
[DEBUG] stable-diffusion.cpp:600 - finished loaded file
ggml_new_object: not enough space in the context's memory pool (needed 10489856, available 10485760)/workspaces/stable-diffusion.cpp/ggml/src/ggml.c:1597: GGML_ASSERT(obj_new) failedCould not attach to process. If your uid matches the uid of the targetprocess, check the setting of /proc/sys/kernel/yama/ptrace_scope, or tryagain as the root user. For more details, see /etc/sysctl.d/10-ptrace.confptrace: Operation not permitted.No stack.The program is not being run.Aborted (core dumped)

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions