VideoLab

README 中文版本框架设计与实现介绍

High-performance and flexible video editing and effects framework, based on AVFoundation and Metal.

Framework design and implementation

Features

• High-performance real-time video editing and exporting.
• Highly free combination of video, image, audio.
• Support audio pitch setting and volume adjustment.
• Support CALayer vector animations, so complex text animations are supported.
• Support keyframe animation.
• Support After Effect-like pre-compose.
• Support transitions.
• Support custom effects. Such as LUT filter, zoom blur, etc.

The following are some GIFs of features(multiple layers, text animation, keyframe animation, pre compose, and transition)

Requirements

• iOS 11.0+
• Swift 5.0+

Installation

VideoLab is available through CocoaPods. Specify the following in your Podfile:

source'https://github.com/CocoaPods/Specs.git'platform:ios,'11.0'use_frameworks!target'<Your Target>'dopod'VideoLab'end

Usage

Basic Concept

RenderLayer

RenderLayer is the most basic unit in the VideoLab framework. A video, image, audio can be a RenderLayer, or even just an effect can be a RenderLayer. RenderLayer is more like the concept of the layer in After Effect.

RenderComposition

RenderComposition works as a composite, can set frame rate, canvas size, contains multiple RenderLayers, can set CALayer to support vector animations.

VideoLab

VideoLab can be considered as a lab where AVPlayerItem, AVAssetExportSession, AVAssetImageGenerator can be generated according to RenderComposition.

Basic Usage

// 1. Layer 1
varurl=Bundle.main.url(forResource:"video1", withExtension:"MOV")varasset=AVAsset(url: url!)varsource=AVAssetSource(asset: asset)
source.selectedTimeRange =CMTimeRange(start:CMTime.zero, duration: asset.duration)vartimeRange= source.selectedTimeRange
letrenderLayer1=RenderLayer(timeRange: timeRange, source: source)
// 1. Layer 2
url =Bundle.main.url(forResource:"video2", withExtension:"MOV")
asset =AVAsset(url: url!)
source =AVAssetSource(asset: asset)
source.selectedTimeRange =CMTimeRange(start:CMTime.zero, duration: asset.duration)
timeRange = source.selectedTimeRange
timeRange.start =CMTimeRangeGetEnd(renderLayer1.timeRange)letrenderLayer2=RenderLayer(timeRange: timeRange, source: source)
// 2. Composition
letcomposition=RenderComposition()
composition.renderSize =CGSize(width:1280, height:720)
composition.layers =[renderLayer1, renderLayer2]
// 3. VideoLab
letvideoLab=VideoLab(renderComposition: composition)
// 4. Make playerItem
letplayerItem= videoLab.makePlayerItem()

1. Create RenderLayer
2. Create RenderComposition, set renderSize and layers
3. Create VideoLab with renderComposition
4. Make AVPlayerItem or AVAssetExportSession

More Advanced Usage

Transform

varcenter=CGPoint(x:0.25, y:0.25)vartransform=Transform(center: center, rotation:0, scale:0.5)
renderLayer1.transform = transform

1. Create Transform with center, rotation and scale
2. RenderLayer set transform

Audio Configuration

letaudioConfiguration=AudioConfiguration()letvolumeRampTimeRange=CMTimeRange(start:CMTime.zero, duration:CMTime(seconds:5, preferredTimescale:600))letvolumeRamp1=VolumeRamp(startVolume:0.0, endVolume:0.0, timeRange: volumeRampTimeRange)
audioConfiguration.volumeRamps =[volumeRamp1]
renderLayer2.audioConfiguration = audioConfiguration

1. Create AudioConfiguration
2. Create VolumeRamp with startVolume, endVolume and timeRange
3. AudioConfiguration set volumeRamps
4. RenderLayer set audioConfiguration

CALayer Animation

For exporting set your customized CALayer for RenderComposition

composition.animationLayer =<Your customized CALayer>

For playback add AVSynchronizedLayer to your view's layer, See more detail in Text Animation Demo.

Keyframe Animation

// 1. Keyframe animation
letkeyTimes=[CMTime(seconds:2, preferredTimescale:600),CMTime(seconds:4, preferredTimescale:600),CMTime(seconds:6, preferredTimescale:600)]letanimation=KeyframeAnimation(keyPath:"blendOpacity",
values:[1.0,0.2,1.0],
keyTimes: keyTimes, timingFunctions:[.linear,.linear])
renderLayer1.animations =[animation]vartransform=Transform.identity
letanimation1=KeyframeAnimation(keyPath:"scale",
values:[1.0,1.3,1.0],
keyTimes: keyTimes, timingFunctions:[.quadraticEaseInOut,.quadraticEaseInOut])letanimation2=KeyframeAnimation(keyPath:"rotation",
values:[0,Float.pi /2.0,0],
keyTimes: keyTimes, timingFunctions:[.quadraticEaseInOut,.quadraticEaseInOut])
transform.animations =[animation1, animation2]
renderLayer1.transform = transform

1. Create KeyframeAnimation with keyPath, values, keyTimes and timingFunctions
2. Set animations for a struct or class that implements the Animatable protocol (e.g. Transform struct, RenderLayer class)

RenderLayerGroup (After Effect-like pre-compose)

letlayerGroup=RenderLayerGroup(timeRange: timeRange)
layerGroup.layers =[renderLayer1, renderLayer2]

1. Create RenderLayerGroup with timeRange
2. Set sub layers for layerGroup. See more detail in Layer Group Demo.

Transition

We don't have a transition layer, so instead, you can add a transform or operations to each RenderLayer to create a transition. See more detail in Transition Demo.

Custom Effects

// Filter
varfilter=LookupFilter()
filter.addTexture(lutTextures[0], at:0)
renderLayer.operations =[filter]
// Zoom Blur
varzoomblur=ZoomBlur()
animation =KeyframeAnimation(keyPath:"blurSize",
values:[0.0,3.0],
keyTimes: keyTimes, timingFunctions:[.quarticEaseOut])
zoomblur.animations =[animation]
layerGroup1.operations =[zoomblur]

1. Create customize Operation inherited from BasicOperation. BasicOperation also conforms to the Animatable protocol
2. Set operations for RenderLayer.

TODO

• Support Open GL render
• Add speed adjustment for RenderLayer.
• Provide a more convenient way to use transitions, possibly providing TransitionLayer.
• Add log system.
• Improve the demo and provide UI interactions.

Author

BearRuan, ruanjingxiong@gmail.com

License

VideoLab is available under the MIT license. See the LICENSE file for more info.