Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

RTCMTLVideoView freezes issue. #2

Open
Meonardo opened this issue Jun 4, 2021 · 3 comments
Open

RTCMTLVideoView freezes issue. #2

Meonardo opened this issue Jun 4, 2021 · 3 comments
Labels
bug Something isn't working help wanted Extra attention is needed

Comments

@Meonardo
Copy link
Owner

Meonardo commented Jun 4, 2021

Known issue:
In landscape mode, using RTCMTLVideoView as localVideoTrack renderer and set its transform will freeze the application.
Please see the issue link.

Alternatively, we can use RTCEAGLVideoView as localVideoTrack renderer.

@Meonardo Meonardo added bug Something isn't working help wanted Extra attention is needed labels Jun 4, 2021
Meonardo pushed a commit that referenced this issue Jun 4, 2021
Meonardo pushed a commit that referenced this issue Jun 4, 2021
@Meonardo
Copy link
Owner Author

Meonardo commented Jun 4, 2021

The issue is not fixed,
but the alternative way to achieve local front camera renderer mirror effect is:

  1. Update renderer rotationOverride according to the current orentation of the device;
  2. Update the camera output orientation in file RTCCameraVideoCapturer,
- (void)captureOutput:(AVCaptureOutput *)captureOutput
    didOutputSampleBuffer:(CMSampleBufferRef)sampleBuffer
           fromConnection:(AVCaptureConnection *)connection
``` .

@fukemy
Copy link

fukemy commented Oct 9, 2021

just change to using RTCEAGLVideoView

final class WebRTCView: UIView, RTCVideoViewDelegate{
    
    var videoView = RTCEAGLVideoView(frame: .zero)
    
    var videoSize = CGSize.zero
    var isFillView = false
    
    @IBInspectable public var cornerRadius: CGFloat = 0 {
        didSet {
            layer.cornerRadius = cornerRadius
//            layer.borderWidth = cornerRadius > 0 ? 1 : 0
//            layer.borderColor = UIColor.white.cgColor
        }
    }

    override init(frame: CGRect) {
        super.init(frame: frame)
        initView()
    }

    required init?(coder: NSCoder) {
        super.init(coder: coder)
        initView()
    }
    
    func initView(){
        videoView.delegate = self
        clipsToBounds = true
        addSubview(videoView)
    }
    
    func videoView(_ videoView: RTCVideoRenderer, didChangeVideoSize size: CGSize) {
        self.videoSize = size
        setNeedsLayout()
    }

    override func layoutSubviews() {
        super.layoutSubviews()
        updateLayout()
    }
    
    func updateLayout(){
        guard videoSize.width > 0 && videoSize.height > 0 else {
            videoView.frame = bounds
            return
        }

        var videoFrame : CGRect = .zero
            
        if !isFillView{
            videoFrame = AVMakeRect(aspectRatio: videoSize, insideRect: bounds)
            let scale = videoFrame.size.aspectFitScale(in: bounds.size)
            videoFrame.size.width = videoFrame.size.width * CGFloat(scale)
            videoFrame.size.height = videoFrame.size.height * CGFloat(scale)
            
        }else{
            let newSize = aspectFill(aspectRatio: videoSize, minimumSize: bounds.size)
            videoFrame.size.width = newSize.width
            videoFrame.size.height = newSize.height
            
        }
        

        videoView.frame = videoFrame
        videoView.center = CGPoint(x: bounds.midX, y: bounds.midY)
    }
    
    func aspectFill(aspectRatio : CGSize, minimumSize: CGSize) -> CGSize {
        let mW = minimumSize.width / aspectRatio.width;
        let mH = minimumSize.height / aspectRatio.height;
        
        var temp = minimumSize
        if( mH > mW ) {
            temp.width = minimumSize.height / aspectRatio.height * aspectRatio.width;
        }
        else if( mW > mH ) {
            temp.height = minimumSize.width / aspectRatio.width * aspectRatio.height;
        }
        
        return temp;
    }
    
}


extension CGSize {
    func aspectFitScale(in container: CGSize) -> CGFloat {

        if height <= container.height && width > container.width {
            return container.width / width
        }
        if height > container.height && width > container.width {
            return min(container.width / width, container.height / height)
        }
        if height > container.height && width <= container.width {
            return container.height / height
        }
        if height <= container.height && width <= container.width {
            return min(container.width / width, container.height / height)
        }
        return 1.0
    }
}

@Meonardo
Copy link
Owner Author

Meonardo commented Oct 9, 2021

just change to using RTCEAGLVideoView

final class WebRTCView: UIView, RTCVideoViewDelegate{
    
    var videoView = RTCEAGLVideoView(frame: .zero)
    
    var videoSize = CGSize.zero
    var isFillView = false
    
    @IBInspectable public var cornerRadius: CGFloat = 0 {
        didSet {
            layer.cornerRadius = cornerRadius
//            layer.borderWidth = cornerRadius > 0 ? 1 : 0
//            layer.borderColor = UIColor.white.cgColor
        }
    }

    override init(frame: CGRect) {
        super.init(frame: frame)
        initView()
    }

    required init?(coder: NSCoder) {
        super.init(coder: coder)
        initView()
    }
    
    func initView(){
        videoView.delegate = self
        clipsToBounds = true
        addSubview(videoView)
    }
    
    func videoView(_ videoView: RTCVideoRenderer, didChangeVideoSize size: CGSize) {
        self.videoSize = size
        setNeedsLayout()
    }

    override func layoutSubviews() {
        super.layoutSubviews()
        updateLayout()
    }
    
    func updateLayout(){
        guard videoSize.width > 0 && videoSize.height > 0 else {
            videoView.frame = bounds
            return
        }

        var videoFrame : CGRect = .zero
            
        if !isFillView{
            videoFrame = AVMakeRect(aspectRatio: videoSize, insideRect: bounds)
            let scale = videoFrame.size.aspectFitScale(in: bounds.size)
            videoFrame.size.width = videoFrame.size.width * CGFloat(scale)
            videoFrame.size.height = videoFrame.size.height * CGFloat(scale)
            
        }else{
            let newSize = aspectFill(aspectRatio: videoSize, minimumSize: bounds.size)
            videoFrame.size.width = newSize.width
            videoFrame.size.height = newSize.height
            
        }
        

        videoView.frame = videoFrame
        videoView.center = CGPoint(x: bounds.midX, y: bounds.midY)
    }
    
    func aspectFill(aspectRatio : CGSize, minimumSize: CGSize) -> CGSize {
        let mW = minimumSize.width / aspectRatio.width;
        let mH = minimumSize.height / aspectRatio.height;
        
        var temp = minimumSize
        if( mH > mW ) {
            temp.width = minimumSize.height / aspectRatio.height * aspectRatio.width;
        }
        else if( mW > mH ) {
            temp.height = minimumSize.width / aspectRatio.width * aspectRatio.height;
        }
        
        return temp;
    }
    
}


extension CGSize {
    func aspectFitScale(in container: CGSize) -> CGFloat {

        if height <= container.height && width > container.width {
            return container.width / width
        }
        if height > container.height && width > container.width {
            return min(container.width / width, container.height / height)
        }
        if height > container.height && width <= container.width {
            return container.height / height
        }
        if height <= container.height && width <= container.width {
            return min(container.width / width, container.height / height)
        }
        return 1.0
    }
}

That's great, use RTCEAGLVideoView will work.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working help wanted Extra attention is needed
Projects
None yet
Development

No branches or pull requests

2 participants