如何通过HarmonyOS编解码播放Camera的实时预览流

奶盖
发布于 2021-7-20 11:36
浏览
1收藏

1. 介绍

 

视频编解码的主要工作:

  • 编码,即将原始的视频信息压缩为既定格式的数据。
  • 解码,即将已知格式的数据还原为视频信息。

本教程将通过启动相机捕获预览帧,转换为视频原始数据并使用HamonyOS视频解码能力播放预览画面。
通过本教程,你将实现不受视频格式限制、不受视频完整性的影响、确保设备可以实时播放视频流数据,也可以以此为基础实现分布式相机预览、直播、视频聊天等功能。
实时播放预览流界面,效果图如下:如何通过HarmonyOS编解码播放Camera的实时预览流-鸿蒙开发者社区

 

2. 代码结构解读

 

如何通过HarmonyOS编解码播放Camera的实时预览流-鸿蒙开发者社区

 

  • camera:封装了HarmonyOS camera,通过自定义的CameraView和控制器CameraController,实现了Model和View的解耦。
  • codec:是应用于视频编解码Codec的封装,包括编码器CodecEncoder和解码器CodecDecoder,方便开发者使用编码和解码。
  • manager:是视频编解码播放器的封装,用于slice和编解码能力分离。
  • media:是camera视频录制所使用recorder的封装,用于去CameraController代码复杂度。
  • utils:工具类
    • LogUtil是日志打印类,对HiLog日志进行了封装。
    • ScreenUtils是获取其设备屏幕宽高和分辨率的工具类。
  • CodecAbility:自定义视频编解码功能入口。
  • MainAbility:主程序入口,DevEco Studio生成,未添加逻辑,无需变更。
  • MyApplication:DevEco Studio生成,无需变更。

 

3. HarmonyOS Camera介绍

 

本应用通过鸿蒙Camera捕获预览帧,并实现了设置自拍镜像和切换摄像头的功能。

 

 ● 自拍镜像
通过FrameConfig.Builder设置返回帧参数接口可以设置镜像功能,代码如下:

frameConfigBuilder.setParameter(ParameterKey.IMAGE_MIRROR, true);

 

 ● 捕获预览帧数据
HarmonyOS编码器需要传入视频原始数据,开发者可以通过设置帧接收器的格式为YUV420_888来获取帧的原始数据,

 

步骤如下:

 

1.创建帧接收器。

imageReceiver =ImageReceiver.create( 
				Math.max(resoluteX, resoluteY), 
				Math.min(resoluteX, resoluteY), 
				ImageFormat.YUV420_888, 
				CameraConst.IMAGE_RCV_CAPACITY);

 

 

2.设置帧接收器回调接口。

imageReceiver.setImageArrivalListener(new ImageReceiver.IImageArrivalListener() { 
	@Override 
	public void onImageArrival(ImageReceiver imageReceiver) { 
	    //帧数据回调 
	} 
});

 

3.开始连续捕获模式。

frameConfigBuilder.addSurface(imageReceiver.getRecevingSurface()); 
try { 
	cameraDevice.triggerLoopingCapture(frameConfigBuilder.build()); 
} catch (IllegalArgumentException e) { 
	LogUtil.info(TAG, "pushFlow is failed," + e.getMessage()); 
}

 

 ● 切换摄像头
通过CameraKit获取手机的摄像头硬件id,通过id创建Camera实例,代码如下:

private void cameraInit() { 
	CameraKit camerakit = CameraKit.getInstance(context.getApplicationContext()); 
	if (camerakit == null || camerakit.getCameraIds().length <= 0) { 
		return; 
	} 
	String cameraId = camerakit.getCameraIds()[0]; 
	if (camerakit.getCameraIds().length > 1) { 
		cameraId = isFrontCamera ? camerakit.getCameraIds()[1] : camerakit.getCameraIds()[0]; 
	} 
	cameraSupportor = camerakit.getCameraAbility(cameraId); 
	CameraStateCallback cameraStateCallback = new MyCameraStatuCallback(); 
	camerakit.createCamera(cameraId, cameraStateCallback, eventHandler); 
}

 

4. YUV编码

 

使用HarmonyOS编码器Codec对Camera获取的视频YUV数据进行编码,步骤如下:
步骤 1 - 初始化编码器。

Format fmt = new Format();
fmt.putStringValue(Format.MIME,Format.VIDEO_AVC);
fmt.putIntValue(Format.WIDTH, controller.getResolution().height); 
fmt.putIntValue(Format.HEIGHT, controller.getResolution().width); 
fmt.putIntValue(Format.BIT_RATE, MediaConst.RECORDER_BIT_RATE); 
fmt.putIntValue(Format.COLOR_MODEL, CodecConst.CODEC_COLOR_MODEL); 
fmt.putIntValue(Format.FRAME_RATE, MediaConst.RECORDER_FRAME_RATE); 
fmt.putIntValue(Format.FRAME_INTERVAL, CodecConst.CODEC_FRAME_INTERVAL); 
fmt.putIntValue(Format.BITRATE_MODE, CodecConst.CODEC_BITRATE_MODE);

 

步骤 2 - videoEncoder= new CodecEncoder.Builder().setFormat(fmt).create();启动编码器。

videoEncoder.openEncoder();

 

步骤 3 - 开始编码,传入Camera获取的YUV帧数据。

videoEncoder.startEncode(frame);

 

 

步骤 4 - 设置编码成功回调,在回调中返回编码后数据,在本应用中在此回调中对数据解码播放。

videoEncoder.setEncodeListener((byteBuffer, bufferInfo) -> { 
		byte[] buffers = new byte[bufferInfo.size]; 
		byteBuffer.clear(); 
		byteBuffer.get(buffers); 
		videoDecoder.startDecode(buffers); 
	});

 

—-结束

 

5. 视频源解码播放

 

编码回调返回自定义编码格式数据,使用Codec对视频源进行播放,步骤如下:


步骤 1 - 初始化解码器,需要传入surface作为视频承载。

videoDecode=r new CodecDecoder.Builder()
.setFormat(fmt)
.setSurface(surface).create();

 

步骤 2 - 启动解码器。

videoDecoder.openDecoder();

 

步骤 3 - 开始解码。

byte[] buffers = new byte[bufferInfo.size]; 
byteBuffer.clear(); 
byteBuffer.get(buffers); 
videoDecoder.startDecode(buffers);

—-结束

 

 

6. 完整示例

 

CodecAbilitySlice完整示例代码如下:

/* 
 * Copyright (c) Huawei Technologies Co., Ltd. 2021-2021. All rights reserved. 
 * 
 * Licensed under the Apache License, Version 2.0 (the "License"); 
 * you may not use this file except in compliance with the License. 
 * You may obtain a copy of the License at 
 * 
 * http://www.apache.org/licenses/LICENSE-2.0 
 * 
 * Unless required by applicable law or agreed to in writing, software 
 * distributed under the License is distributed on an "AS IS" BASIS, 
 * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. 
 * See the License for the specific language governing permissions and 
 * limitations under the License. 
 */ 
 
package com.huawei.codecdemo.slice; 
 
import com.huawei.codecdemo.ResourceTable; 
import com.huawei.codecdemo.camera.api.CameraListener; 
import com.huawei.codecdemo.camera.constant.CaptureMode; 
import com.huawei.codecdemo.camera.view.CameraView; 
import com.huawei.codecdemo.camera.view.CaptureButton; 
import com.huawei.codecdemo.manager.CodecPlayer; 
 
import ohos.aafwk.ability.AbilitySlice; 
import ohos.aafwk.content.Intent; 
import ohos.agp.components.Image; 
import ohos.agp.components.Switch; 
import ohos.agp.components.surfaceprovider.SurfaceProvider; 
import ohos.agp.graphics.SurfaceOps; 
import ohos.app.dispatcher.task.TaskPriority; 
 
import java.util.Optional; 
 
/** 
 * CodecAbilitySlice 
 * 
 * @since 2021-04-09 
 */ 
public class CodecAbilitySlice extends AbilitySlice { 
    private static final int NUMBER_INT_1000 = 1000; 
    private CameraListener cameraController; 
    private CodecPlayer codecPlayer; 
    private CaptureButton captureButton; 
    private Image cameraSwitchButton; 
    private Switch mirrorSwitch; 
 
    @Override 
    protected void onStart(Intent intent) { 
        super.onStart(intent); 
        super.setUIContent(ResourceTable.Layout_ability_codec); 
        initView(); 
        initListener(); 
    } 
 
    private void initView() { 
        if (findComponentById(ResourceTable.Id_button_capture) instanceof CaptureButton) { 
            captureButton = (CaptureButton) findComponentById(ResourceTable.Id_button_capture); 
        } 
        if (findComponentById(ResourceTable.Id_image_camera_switch) instanceof Image) { 
            cameraSwitchButton = (Image) findComponentById(ResourceTable.Id_image_camera_switch); 
        } 
        if (findComponentById(ResourceTable.Id_mirror_switch) instanceof Switch) { 
            mirrorSwitch = (Switch) findComponentById(ResourceTable.Id_mirror_switch); 
        } 
        if (findComponentById(ResourceTable.Id_cameraview) instanceof CameraView) { 
            CameraView cameraView = (CameraView) findComponentById(ResourceTable.Id_cameraview); 
            cameraController = cameraView.getController(); 
            cameraController.setMode(CaptureMode.PUSH_FLOW); 
        } 
        if (findComponentById(ResourceTable.Id_remote_player) instanceof SurfaceProvider) { 
            SurfaceProvider remoteSurfaceView = (SurfaceProvider) findComponentById(ResourceTable.Id_remote_player); 
            remoteSurfaceView.pinToZTop(true); 
            Optional<SurfaceOps> optional = remoteSurfaceView.getSurfaceOps(); 
            optional.ifPresent(surfaceOps -> surfaceOps.addCallback( 
                    new SurfaceOps.Callback() { 
                        @Override 
                        public void surfaceCreated(SurfaceOps surfaceOps) { 
                            codecPlayer = new CodecPlayer(surfaceOps.getSurface()); 
                            cameraController.setCameraListener(codecPlayer); 
                        } 
 
                        @Override 
                        public void surfaceChanged( 
                                SurfaceOps surfaceOps, int tag, int width, int height) { 
                        } 
 
                        @Override 
                        public void surfaceDestroyed(SurfaceOps surfaceOps) { 
                        } 
                    })); 
        } 
    } 
 
    private void initListener() { 
        if (mirrorSwitch != null) { 
            mirrorSwitch.setCheckedStateChangedListener((absButton, isMirror) -> 
                    cameraController.setMirrorEffect(isMirror)); 
        } 
        if (cameraSwitchButton != null) { 
            cameraSwitchButton.setClickedListener(component -> { 
                codecPlayer.stop(); 
                cameraController.switchCamera(); 
                getGlobalTaskDispatcher(TaskPriority.DEFAULT) 
                        .delayDispatch(() -> cameraController.capture(), NUMBER_INT_1000); 
            }); 
        } 
        if (captureButton != null) { 
            captureButton.setClickedListener(component -> { 
                captureToggle(); 
            }); 
        } 
    } 
 
    private void captureToggle() { 
        if (cameraController.isRecording()) { 
            cameraController.stopRecord(); 
            captureButton.capture2round(); 
        } else { 
            cameraController.capture(); 
            if (cameraController.getCaptureMode() == CaptureMode.VIDEO_RECORD) { 
                captureButton.capture2Rect(); 
            } 
        } 
    } 
 
    @Override 
    protected void onActive() { 
        super.onActive(); 
    } 
 
    @Override 
    protected void onForeground(Intent intent) { 
        super.onForeground(intent); 
    } 
 
    @Override 
    protected void onStop() { 
        super.onStop(); 
    } 
}

 

其中,页面布局文件为ability_codec.xml,示例代码如下:

<?xml version="1.0" encoding="utf-8"?> 
<DependentLayout 
    xmlns:ohos="http://schemas.huawei.com/res/ohos" 
    ohos:id="$+id:prent_layout" 
    ohos:height="match_parent" 
    ohos:width="match_parent" 
    ohos:alignment="center" 
    ohos:background_element="#000000"> 
 
    <com.huawei.codecdemo.camera.view.CameraView 
        ohos:id="$+id:cameraview" 
        ohos:height="300vp" 
        ohos:width="match_parent"/> 
 
    <SurfaceProvider 
        ohos:id="$+id:remote_player" 
        ohos:height="300vp" 
        ohos:width="match_parent" 
        ohos:below="$id:cameraview" 
        ohos:top_margin="5vp"/> 
 
    <DependentLayout 
        ohos:height="match_content" 
        ohos:width="match_parent" 
        ohos:align_parent_bottom="true" 
        ohos:bottom_margin="15vp"> 
 
        <Text 
            ohos:id="$+id:mirror_text" 
            ohos:height="match_content" 
            ohos:width="match_content" 
            ohos:left_margin="10vp" 
            ohos:text="自拍镜像:" 
            ohos:text_color="#ffffff" 
            ohos:text_size="14vp" 
            ohos:vertical_center="true"/> 
 
        <Switch 
            ohos:id="$+id:mirror_switch" 
            ohos:height="20vp" 
            ohos:width="40vp" 
            ohos:right_of="$id:mirror_text" 
            ohos:text_state_off="off" 
            ohos:text_state_on="on" 
            ohos:thumb_element="$graphic:thumb" 
            ohos:track_element="$graphic:track" 
            ohos:vertical_center="true"/> 
 
        <com.huawei.codecdemo.camera.view.CaptureButton 
            ohos:id="$+id:button_capture" 
            ohos:height="50vp" 
            ohos:width="50vp" 
            ohos:background_element="$graphic:shape_take_picture_bac" 
            ohos:center_in_parent="true" 
            ohos:layout_alignment="horizontal_center"/> 
 
        <Image 
            ohos:id="$+id:image_camera_switch" 
            ohos:height="40vp" 
            ohos:width="40vp" 
            ohos:image_src="$media:ic_camera_switch" 
            ohos:left_margin="70vp" 
            ohos:right_of="$id:button_capture" 
            ohos:scale_mode="stretch" 
            ohos:vertical_center="true"/> 
 
    </DependentLayout> 
 
</DependentLayout>

 

 说明:
以上代码仅demo演示参考使用,产品化的代码需要考虑数据校验和国际化

 

7. 恭喜您

到这里您已经成功学习了如何通过HarmonyOS编解码播放Camera的实时预览流。

已于2021-7-20 11:36:52修改
3
收藏 1
回复
举报
3条回复
按时间正序
/
按时间倒序
爱吃土豆丝的打工人
爱吃土豆丝的打工人

CameraView.java 里面的东西有没有详细的可以看一看不

回复
2021-7-20 14:44:05
chaoxiaoshu
chaoxiaoshu

有源码吗?

视频解码部分我不是很明白

回复
2021-7-20 16:19:17
wx669503af01ba9
wx669503af01ba9

求源码谢谢


回复
2024-8-2 16:00:22
回复
    相关推荐