GitHub - hexagons/PixelKit: Live Graphics Framework for iOS & macOS
source link: https://github.com/hexagons/PixelKit
Go to the source link to view the article. You can view the picture content, updated content and better typesetting reading experience. If the link is broken, please click the button below to view the snapshot at that time.
README.md
PixelKit
a Live Graphics Framework for iOS and macOS
powered by Metal - inspired by TouchDesigner
Content:
Camera -
Image -
Video -
Screen Capture-
Color -
Circle -
Rectangle -
Polygon -
Gradient -
Noise -
Text
Effects:
Levels -
Blur -
Edge -
Threshold -
Quantize -
Transform -
Kaleidoscope -
Twirl -
Feedback -
Channel Mix -
Chroma Key -
Corner Pin -
HueSat -
Crop -
Flip Flop -
Range -
Sharpen -
Slope -
Cross -
Blend -
Lookup -
Displace -
Remap -
Reorder -
Blends
Examples:
Camera Effects -
Green Screen -
Hello Pixels App -
Code Examples
Info:
Coordinate Space -
Blend Operators -
Effect Convenience Funcs -
File IO -
High Bit Mode -
Apps
Install
pod 'PixelKit'
import PixelKit
Note that PixelKit dose not have simulator support. Metal for iOS can only run on a physical device.
To gain camera access, on macOS, check Camera in the App Sandbox in your Xcode project settings under Capabilities.
Docs
Tutorials
Getting started with PixelKit in Swift
Getting started with Metal in PixelKit
Green Screen in Swift & PixelKit
Particles in VertexKit & PixelKit
Examples
Hello Pixels App (iOS & macOS)
Code Examples of all PIXs
Example: Camera Effects
import PixelKit
let camera = CameraPIX() let levels = LevelsPIX() levels.inPix = camera levels.brightness = 1.5 levels.gamma = 0.5 let hueSat = HueSatPIX() hueSat.inPix = levels hueSat.sat = 0.5 let blur = BlurPIX() blur.inPix = hueSat blur.radius = 0.25 let res: PIX.Res = .custom(w: 1500, h: 1000) let circle = CirclePIX(res: res) circle.radius = 0.45 circle.bgColor = .clear let finalPix: PIX = blur & (camera * circle) finalPix.view.frame = view.bounds view.addSubview(finalPix.view)
This can also be done with Effect Convenience Funcs:
let pix = CameraPIX()._brightness(1.5)._gamma(0.5)._saturation(0.5)._blur(0.25)
Remeber to add NSCameraUsageDescription
to your info.plist
Example: Green Screen
import PixelKit
let cityImage = ImagePIX() cityImage.image = UIImage(named: "city") let supermanVideo = VideoPIX() supermanVideo.load(fileNamed: "superman", withExtension: "mov") let supermanKeyed = ChromaKeyPIX() supermanKeyed.inPix = supermanVideo supermanKeyed.keyColor = .green let blendPix = BlendPIX() blendPix.blendingMode = .over blendPix.inPixA = cityImage blendPix.inPixB = supermanKeyed let finalPix: PIX = blendPix finalPix.view.frame = view.bounds view.addSubview(finalPix.view)
This can also be done with Blend Operators and Effect Convenience Funcs:
let pix = ImagePIX("city") & VideoPIX("superman.mov")._chromaKey(.green)
This is a representation of the Pixel Nodes Green Screen project.
Coordinate Space
PixelKit coordinate space is normailzed to the vertical axis (1.0 in height) with the origin (0.0, 0.0) in the center.
Note that compared to native UIKit views the vertical axis is flipped and origin is moved, this is more convinent when working with graphics is PixelKit.
A full rotation is defined by 1.0
Center: CGPoint(x: 0, y: 0)
Bottom Left: CGPoint(x: -0.5 * aspectRatio, y: -0.5)
Top Right: CGPoint(x: 0.5 * aspectRatio, y: 0.5)
Tip: PIX.Res
has an .aspect
property:let aspectRatio: LiveFloat = PIX.Res._1080p.aspect
Blend Operators
A quick and convenient way to blend PIXs
These are the supported PIX.BlendingMode
operators:
&
!&
+
-
*
**
!**
%
<>
><
--
~
.over
.under
.add
.subtract
.multiply
.power
.gamma
.difference
.minimum
.maximum
.subtractWithAlpha
.average
let blendPix = (CameraPIX() !** NoisePIX(res: .fullHD(.portrait))) * CirclePIX(res: .fullHD(.portrait))
Note when using Live values, one line if else statments are written with <?>
& <=>
:
let a: LiveFloat = 1.0 let b: LiveFloat = 2.0 let isOdd: LiveBool = .seconds % 2.0 < 1.0 let ab: LiveFloat = isOdd <?> a <=> b
The default global blend operator fill mode is .aspectFit
, change it like this:PIX.blendOperators.globalPlacement = .aspectFill
Live Values
Live values can be a constant (let
) and still have changin values.
Live values are ease to animate with the .live
or .seconds
static properites.
The Live Values:
CGFloat
-->LiveFloat
Int
-->LiveInt
Bool
-->LiveBool
CGPoint
-->LivePoint
CGSize
-->LiveSize
CGRect
-->LiveRect
Static properites:
LiveFloat.live
LiveFloat.seconds
LiveFloat.secondsSince1970
LiveFloat.touch
/LiveFloat.mouseLeft
LiveFloat.touchX
/LiveFloat.mouseX
LiveFloat.touchY
/LiveFloat.mouseY
LivePoint.touchXY
/LiveFloat.mouseXY
Functions:
liveFloat.delay(seconds: 1.0)
liveFloat.filter(seconds: 1.0)
liveFloat.filter(frames: 60)
Static functions:
LiveFloat.noise(range: -1.0...1.0, for: 1.0)
LiveFloat.wave(range: -1.0...1.0, for: 1.0)
Effect Convenience Funcs
- pix._reRes(to: ._1080p * 0.5) -> ResPIX
- pix._reRes(by: 0.5) -> ResPIX
- pix._brightness(0.5) -> LevelsPIX
- pix._darkness(0.5) -> LevelsPIX
- pix._contrast(0.5) -> LevelsPIX
- pix._gamma(0.5) -> LevelsPIX
- pix._invert() -> LevelsPIX
- pix._opacity(0.5) -> LevelsPIX
- pix._blur(0.5) -> BlurPIX
- pix._edge() -> EdgePIX
- pix._threshold(at: 0.5) -> ThresholdPIX
- pix._quantize(by: 0.5) -> QuantizePIX
- pix._position(at: CGPoint(x: 0.5, y: 0.5)) -> TransformPIX
- pix._rotatate(to: .pi) -> TransformPIX
- pix._scale(by: 0.5) -> TransformPIX
- pix._kaleidoscope() -> KaleidoscopePIX
- pix._twirl(0.5) -> TwirlPIX
- pix._swap(.red, .blue) -> ChannelMixPIX
- pix._key(.green) -> ChromaKeyPIX
- pix._hue(0.5) -> HueSatPIX
- pix._saturation(0.5) -> HueSatPIX
- pix._crop(CGRect(x: 0.5, y 0.5, width: 0.5, height: 0.5)) -> CropPIX
- pix._flipX() -> FlipFlopPIX
- pix._flipY() -> FlipFlopPIX
- pix._flopLeft() -> FlipFlopPIX
- pix._flopRight() -> FlipFlopPIX
- pix._range(inLow: 0.0, inHigh: 0.5, outLow: 0.5, outHigh: 1.0) -> RangePIX
- pix._range(inLow: .clear, inHigh: .gray, outLow: .gray, outHigh: .white) -> RangePIX
- pix._sharpen() -> SharpenPIX
- pix._slope() - > SlopePIX
- pixA._lookup(pix: pixB, axis: .x) -> LookupPIX
- pixA._lumaBlur(pix: pixB, radius: 0.5) -> LumaBlurPIX
- pixA._displace(pix: pixB, distance: 0.5) -> DisplacePIX
- pixA._remap(pix: pixB) -> RemapPIX
Keep in mind that these funcs will create new PIXs.
Be careful of overloading GPU memory if in a loop.
MIDI
Here's an example of live midi values in range 0.0 to 1.0.
let circle = CirclePIX(res: ._1024)
circle.radius = .midi("13")
circle.color = .midi("17")
You can find the addresses by enabeling logging like this:
MIDI.main.log = true
High Bit Mode
Some effects like DisplacePIX and SlopePIX can benefit from a higher bit depth.
The default is 8 bits. Change it like this:
PixelKit.main.bits = ._16
Enable high bit mode before you create any PIXs.
Note resources do not support higher bits yet.
There is currently there is some gamma offset with resources.
MetalPIXs
let metalPix = MetalPIX(res: ._1080p, code: """ pix = float4(u, v, 0.0, 1.0); """ )
let metalEffectPix = MetalEffectPIX(code: """ float gamma = 0.25; pix = pow(inPix, 1.0 / gamma); """ ) metalEffectPix.inPix = CameraPIX()
let metalMergerEffectPix = MetalMergerEffectPIX(code: """ pix = pow(inPixA, 1.0 / inPixB); """ ) metalMergerEffectPix.inPixA = CameraPIX() metalMergerEffectPix.inPixB = ImagePIX("img_name")
let metalMultiEffectPix = MetalMultiEffectPIX(code: """ float4 inPixA = inTexs.sample(s, uv, 0); float4 inPixB = inTexs.sample(s, uv, 1); float4 inPixC = inTexs.sample(s, uv, 2); pix = inPixA + inPixB + inPixC; """ ) metalMultiEffectPix.inPixs = [ImagePIX("img_a"), ImagePIX("img_b"), ImagePIX("img_c")]
Uniforms:
var lumUniform = MetalUniform(name: "lum") let metalPix = MetalPIX(res: ._1080p, code: """ pix = float4(in.lum, in.lum, in.lum, 1.0); """, uniforms: [lumUniform] ) lumUniform.value = 0.5
Apps
Pixel Nodes
a Live Graphics Node Editor for iPad
powered by PixelKit
Layer Cam
a camera app lets you live layer filters of your choice.
combine effects to create new cool styles.
VJLive
VJLive is a dual deck asset playback system with effects.
Assets can be loaded from Photos. Live camera support. AirPlay support.
by Anton Heestand, Hexagons
Recommend
About Joyk
Aggregate valuable and interesting links.
Joyk means Joy of geeK