<Prev | Content | Next>

14. UI pt.1: UIKit/CoreAnimation

As you may see from previous episodes, Metal is a super powerful instrument for GPU computing. But as a graphics framework, its main purpose is the visual representation of data: from UI elements to interactive scenes in video games or graphic editors. In this episode, I’ll explore Metal’s approach to UI, how to implement your own Metal view, and how to use MTKView.

CoreAnimation

Fundamentally, all native UI on Apple platforms is rendered with CoreAnimation. Basically, the visual representation of UI is a CALayer, which may have:

  • masking layers,
  • child layers,
  • filters,
  • transforms,
  • etc.

These layers are rendered directly with Metal, or baked with CoreGraphics (as Metal isn’t great at vector graphics) and then placed into the screen buffer:

Layers hierarchy

Though there’re lots of ready-to-use layers whose implementations are hidden from you, there’s also CAMetalLayer, which contains CAMetalDrawable, which contains an MTLTexture, which you can render into.

  • UIView
    • CAMetalLayer
      • CAMetalDrawable
        • MTLTexture

Also, as CALayer may have a masking layer, you can render into that too for some special masking effect.

To use CAMetalLayer instead of default layer, add this to your view implementation:

// ...
override class var layerClass: AnyClass {
    return CAMetalLayer.self
}
// ...

Rendering synchronisation

To synchronize rendering with device frames, you need to use CADisplayLink.

Starting from iOS 17 (macOS 14), you can use CAMetalDisplayLink, which offers more control over FPS and makes rendering smoother and more efficient.

In the display link, you can set the preferred FPS, incorporate a rendering loop, and manage it (for example, pause it).

// ...
var displayLink: CADisplayLink

func createDisplayLink() {
    displayLink = CADisplayLink(target: self, selector: #selector(render))
    displayLink.preferredFramesPerSecond = 60
    displayLink.isPaused = false
    displayLink.add(to: .current, forMode: RunLoop.Mode.default)
}

@objc
func render(displayLink: CADisplayLink) {
    // Render here
}
// ...

Drawing to drawable

As you know from previous episodes, you need a texture to render into. As mentioned above, you can find the texture in CAMetalDrawable:

// ...
@objc
func render(displaylink: CADisplayLink) {
    guard let metalLayer = self.layer as? CAMetalLayer,             // (1)    
          let drawable = metalLayer.nextDrawable(),                 // (2)
          let commandBuffer = self.commandQueue.makeCommandBuffer() // (3)
    else {
        return
    }

    let targetTexture = drawable.texture                            // (4)
    // rendering into the texture

    commandBuffer.present(drawable)                                 // (5)
    commandBuffer.commit()
}
// ...
  1. Getting the view’s layer as a Metal layer.
  2. Getting the next available drawable from the layer (there can be multiple drawables for buffering). To avoid blocking when a drawable isn’t immediately free, set allowsNextDrawableTimeout to true.
  3. Creating a command buffer for rendering (in this and other snippets, some class field details are omitted).
  4. Getting the drawable’s texture.
  5. Presenting the updated drawable on screen. You can only call this method before calling the command buffer’s commit() method.

NOTE: Make sure to retrieve the correct pixel format and sampling parameters from the drawable so your pipelines remain compatible with the layer.

MTKView

Though the manual way isn’t particularly complex, you can still take an easier route by just using MTKView from MetalKit. It covers most of these details, and in most use cases, you only need to implement MTKViewDelegate:

// ...
func draw(in view: MTKView) {
    guard let commandBuffer = commandQueue.makeCommandBuffer(),
          let encoderDescriptor = view.currentRenderPassDescriptor  // (1)
    else {
        return
    }

    if let renderEncoder = commandBuffer.makeRenderCommandEncoder(descriptor: encoderDescriptor) {
        // render commands...
        renderEncoder.endEncoding()
    }

    if let drawable = view.currentDrawable {                        // (2)
        commandBuffer.present(drawable)
    }
    commandBuffer.commit()
}

func mtkView(_ view: MTKView, drawableSizeWillChange size: CGSize) {
    // (3)
}
// ...
  1. Acquire the render pass descriptor from the MTKView, which is already prepared.
  2. Retrieve the current drawable from the MTKView for presenting.
  3. Handle changes in the drawable’s size (note that it’s in pixels and already accounts for the content scale factor).

Additionally, you can directly adjust pixel formats, preferred FPS, and other parameters on the view.

Conclusion

  • Everything is drawn with Metal, but hidden in CoreAnimation: (CoreAnimation → CALayer → CAMetalLayer → CAMetalDrawable → MTLTexture)
  • For rendering sync, there’s CADisplayLink.
  • If you don’t want to wire up everything yourself, just use MetalKit.
  • SwiftUI has its own interesting features, but I'll discuss them in the next episode.

<Prev | Content | Next>