GitHub - NextLevel/NextLevel: ⬆️ Rad Media Capture in Swift
source link: https://github.com/NextLevel/NextLevel
Go to the source link to view the article. You can view the picture content, updated content and better typesetting reading experience. If the link is broken, please click the button below to view the snapshot at that time.
README.md
NextLevel
is a Swift camera system designed for easy integration, customized media capture, and image streaming for iOS.
It enables fast integration and customization leveraging AVFoundation
or optionally ARKit
.
Features ? “Vine-like” video clip recording and editing ? photo capture (raw, jpeg, and video frame) ? customizable gestural interaction and interface ? ARKit integration (beta) ? dual, wide angle, telephoto, & true depth support ? adjustable frame rate on supported hardware (ie fast/slow motion capture) ? depth data capture support ? video zoom ⚖ white balance, focus, and exposure adjustment ? flash and torch support ? mirroring support ☀ low light boost ? smooth auto-focus ⚙ configurable encoding and compression settings ? simple media capture and editing API ? extensible API for image processing and CV ? Swift 4
Need a different version of Swift?
4.2
- Target your Podfile to theswift4.2
branch4.0
- Target your Podfile to the latest release or master3.2
- Target your Podfile to release0.6.3
or theswift3.2
branch
Quick Start
# CocoaPods swift_version = "4.0" pod "NextLevel", "~> 0.9.5" # Carthage github "nextlevel/NextLevel" ~> 0.9.5 # Swift PM let package = Package( dependencies: [ .Package(url: "https://github.com/nextlevel/NextLevel", majorVersion: 0) ] )
Alternatively, drop the NextLevel source files or project file into your Xcode project.
ARKit Capture
Starting with release 0.8.0
, NextLevel provides a mode for ARKit capture in addition to AVFoundation. This enables a variety of new camera features while leveraging the existing recording capabilities and media management of NextLevel.
The ARKit mode is in beta, so please help out if you encounter an issue or by contributing ideas for improvement. It is not compiled by default without the inclusion of the Swift compiler flag USE_ARKIT
, since Apple will reject apps that link ARKit and do not use it. To try it out, setup the AppDelegate to load the MixedRealityViewController
class and include the xcode build settings flag.
If you use Cocoapods, you can include -DUSE_ARKIT
with the following podfile addition or by adding it to your Xcode build settings.
installer.pods_project.targets.each do |target| # setup NextLevel for ARKit use if target.name == 'NextLevel' target.build_configurations.each do |config| config.build_settings['OTHER_SWIFT_FLAGS'] = '-DUSE_ARKIT' end end end
Overview
Before starting, ensure that permission keys have been added to your app's Info.plist
.
<key>NSCameraUsageDescription</key> <string>Allowing access to the camera lets you take photos and videos.</string> <key>NSMicrophoneUsageDescription</key> <string>Allowing access to the microphone lets you record audio.</string>
Recording Video Clips
Import the library.
import NextLevel
Setup the camera preview.
let screenBounds = UIScreen.main.bounds self.previewView = UIView(frame: screenBounds) if let previewView = self.previewView { previewView.autoresizingMask = [.flexibleWidth, .flexibleHeight] previewView.backgroundColor = UIColor.black NextLevel.shared.previewLayer.frame = previewView.bounds previewView.layer.addSublayer(NextLevel.shared.previewLayer) self.view.addSubview(previewView) }
Configure the capture session.
override func viewDidLoad() { NextLevel.shared.delegate = self NextLevel.shared.deviceDelegate = self NextLevel.shared.videoDelegate = self NextLevel.shared.photoDelegate = self // modify .videoConfiguration, .audioConfiguration, .photoConfiguration properties // Compression, resolution, and maximum recording time options are available NextLevel.shared.videoConfiguration.maximumCaptureDuration = CMTimeMakeWithSeconds(5, 600) NextLevel.shared.audioConfiguration.bitRate = 44000 }
Start/stop the session when appropriate. These methods create a new "session" instance for 'NextLevel.shared.session' when called.
override func viewWillAppear(_ animated: Bool) { super.viewWillAppear(animated) NextLevel.shared.start() // … }
override func viewWillDisappear(_ animated: Bool) { super.viewWillDisappear(animated) NextLevel.shared.stop() // … }
Video record/pause.
// record NextLevel.shared.record() // pause NextLevel.shared.pause()
Editing Recorded Clips
Editing and finalizing the recorded session.
if let session = NextLevel.shared.session { //.. // undo session.removeLastClip() // various editing operations can be done using the NextLevelSession methods // export session.mergeClips(usingPreset: AVAssetExportPresetHighestQuality, completionHandler: { (url: URL?, error: Error?) in if let _ = url { // } else if let _ = error { // } }) //.. }
Videos can also be processed using the NextLevelSessionExporter, a media transcoding library in Swift.
Custom Buffer Rendering
‘NextLevel’ was designed for sample buffer analysis and custom modification in real-time along side a rich set of camera features.
Just to note, modifications performed on a buffer and provided back to NextLevel may potentially effect frame rate.
Enable custom rendering.
NextLevel.shared.isVideoCustomContextRenderingEnabled = true
Optional hook that allows reading sampleBuffer
for analysis.
extension CameraViewController: NextLevelVideoDelegate { // ... // video frame processing public func nextLevel(_ nextLevel: NextLevel, willProcessRawVideoSampleBuffer sampleBuffer: CMSampleBuffer) { // Use the sampleBuffer parameter in your system for continual analysis }
Another optional hook for reading buffers for modification, imageBuffer
. This is also the recommended place to provide the buffer back to NextLevel for recording.
extension CameraViewController: NextLevelVideoDelegate { // ... // enabled by isCustomContextVideoRenderingEnabled public func nextLevel(_ nextLevel: NextLevel, renderToCustomContextWithImageBuffer imageBuffer: CVPixelBuffer, onQueue queue: DispatchQueue) { // provide the frame back to NextLevel for recording if let frame = self._availableFrameBuffer { nextLevel.videoCustomContextImageBuffer = frame } }
NextLevel will check this property when writing buffers to a destination file. This works for both video and photos with capturePhotoFromVideo
.
nextLevel.videoCustomContextImageBuffer = modifiedFrame
About
NextLevel was initally just a weekend project but has grown into a open community of camera enthusists. The software provides foundational components for managing media recording, camera interface customization, gestural interaction customization, and image streaming for iOS. The same capabilities can also be found in apps such as Snapchat, Instagram, and Vine.
The goal is to continue to provide a good foundation for quick integration (enabling projects to be taken to the next level) – allowing focus to placed on functionality that matters most whether it's realtime image processing, computer vision methods, augmented reality, or even new cinematographic recording techniques.
Documentation
You can find the docs here. Documentation is generated with jazzy and hosted on GitHub-Pages.
Stickers
If you found this project to be helpful, check out the Next Level stickers.
Project
NextLevel is a community – contributions and discussions are welcome!
- Feature idea? Open an issue.
- Found a bug? Open an issue.
- Need help? Use Stack Overflow with the tag ’nextlevel’.
- Questions? Use Stack Overflow with the tag 'nextlevel'.
- Want to contribute? Submit a pull request.
Related Projects
- Player (Swift), video player in Swift
- PBJVideoPlayer (obj-c), video player in obj-c
- GPUImage2, image processing library
- SCRecorder, obj-c capture library
- PBJVision, obj-c capture library
- NextLevelSessionExporter, media transcoding in Swift
Resources
- iOS Device Camera Summary
- AV Foundation Programming Guide
- AV Foundation Framework Reference
- Swift Evolution
- objc.io Camera and Photos
- objc.io Video
- objc.io Core Image and Video
- Cameras, ecommerce and machine learning
- Again, iPhone is the default camera
License
NextLevel is available under the MIT license, see the LICENSE file for more information.
Recommend
About Joyk
Aggregate valuable and interesting links.
Joyk means Joy of geeK