Tutorial

Input Data

You can map an already existing native RGB image source. BGR is mainly implemented because of OpenCV API where you can reference a BGR Mat. You can also reference a managed byte array.

IntPtrnativeRGBImage= ..some native source;varrgba=newRgbImage(H264Sharp.ImageFormat.Rgb,w,h,nativeRGBImage);// Or managedbyte[]managedImageBytes=some managed source..
var rgba =newRgbImage(H264Sharp.ImageFormat.Rgb,w,h,managedImageBytes,offset,count);

YUV Image Input

You can reference a native source YUV I420 planar image.

IntPtrY= ..;IntPtrU= ..;IntPtrV= ..;varyuvp=newYUVImagePointer(Y,U,V,width,height,strideY,strideUV);IntPtrYuv= ..some YUV Source;varyuvp=newYUVImagePointer(Yuv,width,height);

YUV NV12 Image Input

In this format, U and V are interleaved. This is an extremely common native camera output format.

IntPtrY= ..;IntPtrUV= ..;varyuvNv12=newYUVNV12ImagePointer(Y,UV,width,height,strideY,strideUV);IntPtrYuvNV12Contiguous= ..;varyuvNv12=newYUVNV12ImagePointer(YuvNV12Contiguous,width,height);

YUV formats provide more efficient encoding because they bypass the RGB-to-YUV conversion.

---

Output Data

For output from the decoder, you can allocate an RGB format image container.

// Will allocate w*h*4 native bytesvarrgba=newRgbImage(H264Sharp.ImageFormat.Rgba,w,h);varbgra=newRgbImage(H264Sharp.ImageFormat.Bgra,w,h);// Will allocate w*h*3 native bytesvarrgb=newRgbImage(H264Sharp.ImageFormat.Rgb,w,h);varbgr=newRgbImage(H264Sharp.ImageFormat.Bgr,w,h);

You can also refer to existing native memory or a managed byte array.

For YUV output where you want to own the bytes:

// Will allocate (width * height) for Y + ((width * height) / 2) bytes for U and VYuvImageyuvOut=newYuvImage(width,height);

Otherwise, you can use YUVImagePointer to refer to the internal memory of the decoder.

---

Converter Configuration

Optionally, configure your converter if needed. The converter is used by both the decoder and encoder, and settings are global.

// What's inside the native side currentlyvarconfig=Converter.GetCurrentConfig();config.EnableSSE=1;config.EnableNeon=1;config.EnableAvx2=1;config.NumThreads=32;config.EnableCustomThreadPool=1;Converter.SetConfig(config);

---

Encoder and Decoder Initialization

Create your encoder, decoder, or both.

H264Encoderencoder=newH264Encoder();H264Decoderdecoder=newH264Decoder();

You can change the path of the OpenH264 DLL from Cisco. The default one is stored in Defines.CiscoDllName, and you can assign it globally or locally.

// For globalDefines.CiscoDllName="/yourPath/openh264-2.4.1-win64.dll";// For localH264Encoderencoder=newH264Encoder("/yourPath/openh264-2.4.1-win64.dll");H264Decoderdecoder=newH264Decoder("/yourPath/openh264-2.4.1-win64.dll");

It is recommended to use the default decoder initialization.

decoder.Initialize();encoder.Initialize(width,height,bitrate:2_500_000,fps:30,ConfigType.CameraCaptureAdvanced);

---

Encoding

Encode your source images.

List<byte[]>encodedFrames=newList<byte[]>();if(!encoder.Encode(source,outEncodedData[]ec)){foreach(varencodedinec){encodedFrames.Add(encoded.GetBytes());}}

---

Decoding

Decode your encoded frames.

RgbImagergb=newRgbImage(H264Sharp.ImageFormat.Rgb,w,h);foreach(byte[]encodedinencodedFrames){if(decoder.Decode(encoded,0,encoded.Length,noDelay:true,outDecodingStateds,refrgb)){// Do something with rgb}}

WPF

You can write your rgb output to an image on WPF

<Imagex:Name="Decoded"HorizontalAlignment="Left"VerticalAlignment="Top"Margin="10,30,10,10"></Image>

voidDrawEncodedImg(RgbImageframe){Dispatcher.BeginInvoke(newAction(()=>{if(Decoded.Source==null){Decoded.Source=newWriteableBitmap(frame.Width,frame.Height,96,96,PixelFormats.Bgr24,null);}vardst=(WriteableBitmap)Decoded.Source;dst.Lock();intwidth=frame.Width;intheight=frame.Height;intstep=frame.Stride;intrange=frame.Stride*frame.Height;dst.WritePixels(newInt32Rect(0,0,width,height),frame.NativeBytes,range,step);dst.Unlock();// You should pool the RgbImage containers since this switches to the UI thread. // Or sencronize and reuse the same containerpool.Add(frame);}));}

OpenCV

You can convert your Mat into input for the encoder without any copy. Here using OpenCvSharp with camera capture:

varcapture=newVideoCapture(0,VideoCaptureAPIs.WINRT);capture.Open(0);Matframe=newMat();while(captureActive){if(capture.Read(frame)){varg=newRgbImage(ImageFormat.Bgr,frame.Width,frame.Height,(int)frame.Step(),newIntPtr(frame.DataPointer));encodedSuccess=encoder.Encode(g,outEncodedData[]ec);}}