Csharp ⚙️ HEIC/HEIF → JPEG

Csharp is the HEIC/HEIF conversion dependency for PhotoSort / PhotoSweep. The repositories remain separate: Csharp creates safe JPEG derivatives; PhotoSweep performs inventory, OCR, faces, GPS, raster analysis, visual similarity, and review.

New here? Start with one file 🚀

cmake -S . -B build
cmake --build build -j
./build/csharp --backend cpu input.heic output.jpg

The CPU backend is the portable starting point. The original HEIC is preserved, and the JPEG is written atomically. After conversion, PhotoSweep can analyze the source or derivative tree; its comparison tool is the human duplicate-review gate.

📥 HEIC/HEIF → Csharp validation/conversion → JPEG derivative
→ PhotoSweep 🔤 OCR · 🙂 faces · 📍 GPS · 🎨 swatch · 🖼️ raster
→ human duplicate review → optional cleanup

Experimental GPU-accelerated HEIC/HEIF → JPEG conversion for NVIDIA GPUs.

Goal

Keep HEIF container handling on the CPU, hand supported HEVC image payloads to NVIDIA NVDEC, keep decoded frames on the GPU where practical, and eventually encode JPEG with nvJPEG.

HEIF container
↓
libheif / container handling
↓
HEVC coded image
↓
NVDEC
↓
GPU surface
↓
nvJPEG
↓
JPEG

Csharp has two performance goals, and they are deliberately separate:

1. maximize images/second when GPU acceleration actually wins;
2. provide a low-CPU decode path when preserving CPU headroom is more valuable than minimum wall time.

A GPU path may therefore still be useful when it is slower in elapsed time if it materially reduces CPU-seconds consumed while PhotoSort or other workloads need the CPU.

Phase 1

heicprobe establishes the boring, safe foundation before GPU decode work:

• open HEIC/HEIF through libheif
• locate the primary image
• report dimensions and basic image properties
• report EXIF/XMP presence
• classify whether the file is a candidate for the future GPU fast path
• fail cleanly on malformed/unsupported input

Complex HEIF features should eventually fall back to libheif's normal CPU decode rather than being guessed at.

Build

Requires CMake, a C compiler, pkg-config, libheif development headers, and FFmpeg development libraries:

sudo apt install libheif-dev libjpeg-dev libavcodec-dev libavutil-dev libswscale-dev

cmake -S . -B build
cmake --build build -j
./build/heicprobe image.heic

The v0.1 single-file converter uses libjpeg-turbo/libjpeg's raw YCbCr API and writes through a temporary sibling before the final rename:

./build/csharp --backend direct --quality 90 input.heic output.jpg
./build/csharp --backend cpu --overwrite input.heic output.jpg

The direct converter uses NVDEC for ordinary opaque 8-bit 4:2:0 HEIC stills and automatically falls back to libheif RGB/RGBA decoding for other primary formats, including alpha, depth/auxiliary sidecars, unusual chroma, and higher bit depth inputs that libheif can convert to 8-bit JPEG output. Alpha is composited onto white because JPEG has no alpha channel. It copies supported EXIF, XMP, and ICC data, normalizes EXIF orientation because libheif supplies display-oriented pixels, refuses an existing destination unless --overwrite is supplied, and removes temporary output after failures. Build with -DCSHARP_ENABLE_DIRECT_NVDEC=ON for the accelerated backend; otherwise use --backend cpu.

The focused NVDEC proof mode decodes the same disposable image first through the normal libheif CPU decoder and then through the registered csharp-nvdec decoder plugin:

./build/heicprobe --compare-nvdec image.heic

The persistent decode-only benchmark loads its inputs into memory once and round-robins them in one process. It reports cold startup separately from warm repeated decodes:

./build/heicprobe --benchmark 100 sample1.heic sample2.heic

Benchmark output includes wall time, process CPU time, images/sec, CPU-ms/image, and average CPU cores consumed. This lets Csharp distinguish a throughput win from a CPU-headroom win.

The current NVDEC implementation shares one CUDA device and pools AVCodecContext/NVDEC instances by HEVC parameter-set signature. This avoids reinitializing compatible decoders for every image while keeping incompatible streams isolated.

The plugin accepts libheif's length-prefixed HEVC stream, converts it to Annex-B NAL units, decodes with FFmpeg's hevc_cuvid, downloads the CUDA NV12 frame, and returns an 8-bit YCbCr 4:2:0 heif_image. The current scope is ordinary opaque 8-bit HEVC images; unsupported formats must use the normal libheif path.

Next decoder experiments

The current hevc_cuvid route is the reference GPU backend. Two experiments are intentionally kept separate from it so the working path remains available:

• FFmpeg native HEVC decoder with CUDA/NVDEC hardware frames, avoiding the legacy CUVID decoder wrapper where possible.
• Direct NVIDIA Video Codec SDK/NVDECODE integration using a long-lived decoder and decoder reconfiguration for changing still-image dimensions.

Both must preserve the existing exact CPU/NVDEC YCbCr validation before they can replace the reference backend.

Status

Validated v0.1 hardening snapshot:

• 48 real iCloud HEICs converted 48/48 with both CPU and direct-request modes.
• The tested corpus observed 48 direct NVDEC paths and zero decode-failure fallbacks.
• EXIF, standalone XMP, ICC, and orientation normalization were independently checked; output JPEGs were independently decoded with ImageMagick.
• A 1,008-conversion CPU lifecycle stress completed 1008/1008 with no invalid JPEGs, temporary-file accumulation, or observed FD/RSS growth.
• The current corpus does not cover 10-bit, HDR, alpha-primary, grid/tiled, or unusual chroma inputs; those remain explicit fallback/fixture work rather than claims of coverage.

Use --report PATH for a one-row TSV conversion result containing the selected path, fallback reason, dimensions, source depth/chroma, metadata counts, status, and error stage. Use --no-fallback when a direct-request test must fail instead of using CPU fallback.

The direct path is suitable for continued v0.1 testing, but the project is not yet a drop-in archive migration tool until the missing format fixtures and longer direct-mode stress coverage are completed.

PhotoSort integration 🔗

Build Csharp separately, convert only the HEIC/HEIF files that need JPEG derivatives, and point PhotoSweep at the selected source or derivative tree. Csharp preserves inputs, uses direct NVDEC only for its supported fast path, falls back to libheif CPU decoding for unsupported formats, preserves supported metadata/orientation, and atomically creates output. See the PhotoSort integration guide.