如何通过HarmonyOS编解码播放Camera的实时预览流
奶盖
发布于 2021-7-20 11:36
浏览
1收藏
1. 介绍
视频编解码的主要工作：
- 编码，即将原始的视频信息压缩为既定格式的数据。
- 解码，即将已知格式的数据还原为视频信息。
本教程将通过启动相机捕获预览帧，转换为视频原始数据并使用HamonyOS视频解码能力播放预览画面。
通过本教程，你将实现不受视频格式限制、不受视频完整性的影响、确保设备可以实时播放视频流数据，也可以以此为基础实现分布式相机预览、直播、视频聊天等功能。
实时播放预览流界面,效果图如下：
2. 代码结构解读
- camera：封装了HarmonyOS camera，通过自定义的CameraView和控制器CameraController，实现了Model和View的解耦。
- codec：是应用于视频编解码Codec的封装，包括编码器CodecEncoder和解码器CodecDecoder，方便开发者使用编码和解码。
- manager：是视频编解码播放器的封装，用于slice和编解码能力分离。
- media：是camera视频录制所使用recorder的封装，用于去CameraController代码复杂度。
- utils：工具类
- LogUtil是日志打印类，对HiLog日志进行了封装。
- ScreenUtils是获取其设备屏幕宽高和分辨率的工具类。
- CodecAbility：自定义视频编解码功能入口。
- MainAbility：主程序入口，DevEco Studio生成，未添加逻辑，无需变更。
- MyApplication：DevEco Studio生成，无需变更。
3. HarmonyOS Camera介绍
本应用通过鸿蒙Camera捕获预览帧，并实现了设置自拍镜像和切换摄像头的功能。
● 自拍镜像
通过FrameConfig.Builder设置返回帧参数接口可以设置镜像功能，代码如下：
frameConfigBuilder.setParameter(ParameterKey.IMAGE_MIRROR, true);
● 捕获预览帧数据
HarmonyOS编码器需要传入视频原始数据，开发者可以通过设置帧接收器的格式为YUV420_888来获取帧的原始数据，
步骤如下：
1.创建帧接收器。
imageReceiver =ImageReceiver.create(
Math.max(resoluteX, resoluteY),
Math.min(resoluteX, resoluteY),
ImageFormat.YUV420_888,
CameraConst.IMAGE_RCV_CAPACITY);
2.设置帧接收器回调接口。
imageReceiver.setImageArrivalListener(new ImageReceiver.IImageArrivalListener() {
@Override
public void onImageArrival(ImageReceiver imageReceiver) {
//帧数据回调
}
});
3.开始连续捕获模式。
frameConfigBuilder.addSurface(imageReceiver.getRecevingSurface());
try {
cameraDevice.triggerLoopingCapture(frameConfigBuilder.build());
} catch (IllegalArgumentException e) {
LogUtil.info(TAG, "pushFlow is failed," + e.getMessage());
}
● 切换摄像头
通过CameraKit获取手机的摄像头硬件id，通过id创建Camera实例，代码如下：
private void cameraInit() {
CameraKit camerakit = CameraKit.getInstance(context.getApplicationContext());
if (camerakit == null || camerakit.getCameraIds().length <= 0) {
return;
}
String cameraId = camerakit.getCameraIds()[0];
if (camerakit.getCameraIds().length > 1) {
cameraId = isFrontCamera ? camerakit.getCameraIds()[1] : camerakit.getCameraIds()[0];
}
cameraSupportor = camerakit.getCameraAbility(cameraId);
CameraStateCallback cameraStateCallback = new MyCameraStatuCallback();
camerakit.createCamera(cameraId, cameraStateCallback, eventHandler);
}
4. YUV编码
使用HarmonyOS编码器Codec对Camera获取的视频YUV数据进行编码，步骤如下：
步骤 1 - 初始化编码器。
Format fmt = new Format();
fmt.putStringValue(Format.MIME,Format.VIDEO_AVC);
fmt.putIntValue(Format.WIDTH, controller.getResolution().height);
fmt.putIntValue(Format.HEIGHT, controller.getResolution().width);
fmt.putIntValue(Format.BIT_RATE, MediaConst.RECORDER_BIT_RATE);
fmt.putIntValue(Format.COLOR_MODEL, CodecConst.CODEC_COLOR_MODEL);
fmt.putIntValue(Format.FRAME_RATE, MediaConst.RECORDER_FRAME_RATE);
fmt.putIntValue(Format.FRAME_INTERVAL, CodecConst.CODEC_FRAME_INTERVAL);
fmt.putIntValue(Format.BITRATE_MODE, CodecConst.CODEC_BITRATE_MODE);
步骤 2 - videoEncoder= new CodecEncoder.Builder().setFormat(fmt).create();启动编码器。
videoEncoder.openEncoder();
步骤 3 - 开始编码，传入Camera获取的YUV帧数据。
videoEncoder.startEncode(frame);
步骤 4 - 设置编码成功回调，在回调中返回编码后数据，在本应用中在此回调中对数据解码播放。
videoEncoder.setEncodeListener((byteBuffer, bufferInfo) -> {
byte[] buffers = new byte[bufferInfo.size];
byteBuffer.clear();
byteBuffer.get(buffers);
videoDecoder.startDecode(buffers);
});
—-结束
5. 视频源解码播放
编码回调返回自定义编码格式数据，使用Codec对视频源进行播放，步骤如下：
步骤 1 - 初始化解码器，需要传入surface作为视频承载。
videoDecode=r new CodecDecoder.Builder()
.setFormat(fmt)
.setSurface(surface).create();
步骤 2 - 启动解码器。
videoDecoder.openDecoder();
步骤 3 - 开始解码。
byte[] buffers = new byte[bufferInfo.size];
byteBuffer.clear();
byteBuffer.get(buffers);
videoDecoder.startDecode(buffers);—-结束
6. 完整示例
CodecAbilitySlice完整示例代码如下：
/*
* Copyright (c) Huawei Technologies Co., Ltd. 2021-2021. All rights reserved.
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package com.huawei.codecdemo.slice;
import com.huawei.codecdemo.ResourceTable;
import com.huawei.codecdemo.camera.api.CameraListener;
import com.huawei.codecdemo.camera.constant.CaptureMode;
import com.huawei.codecdemo.camera.view.CameraView;
import com.huawei.codecdemo.camera.view.CaptureButton;
import com.huawei.codecdemo.manager.CodecPlayer;
import ohos.aafwk.ability.AbilitySlice;
import ohos.aafwk.content.Intent;
import ohos.agp.components.Image;
import ohos.agp.components.Switch;
import ohos.agp.components.surfaceprovider.SurfaceProvider;
import ohos.agp.graphics.SurfaceOps;
import ohos.app.dispatcher.task.TaskPriority;
import java.util.Optional;
/**
* CodecAbilitySlice
*
* @since 2021-04-09
*/
public class CodecAbilitySlice extends AbilitySlice {
private static final int NUMBER_INT_1000 = 1000;
private CameraListener cameraController;
private CodecPlayer codecPlayer;
private CaptureButton captureButton;
private Image cameraSwitchButton;
private Switch mirrorSwitch;
@Override
protected void onStart(Intent intent) {
super.onStart(intent);
super.setUIContent(ResourceTable.Layout_ability_codec);
initView();
initListener();
}
private void initView() {
if (findComponentById(ResourceTable.Id_button_capture) instanceof CaptureButton) {
captureButton = (CaptureButton) findComponentById(ResourceTable.Id_button_capture);
}
if (findComponentById(ResourceTable.Id_image_camera_switch) instanceof Image) {
cameraSwitchButton = (Image) findComponentById(ResourceTable.Id_image_camera_switch);
}
if (findComponentById(ResourceTable.Id_mirror_switch) instanceof Switch) {
mirrorSwitch = (Switch) findComponentById(ResourceTable.Id_mirror_switch);
}
if (findComponentById(ResourceTable.Id_cameraview) instanceof CameraView) {
CameraView cameraView = (CameraView) findComponentById(ResourceTable.Id_cameraview);
cameraController = cameraView.getController();
cameraController.setMode(CaptureMode.PUSH_FLOW);
}
if (findComponentById(ResourceTable.Id_remote_player) instanceof SurfaceProvider) {
SurfaceProvider remoteSurfaceView = (SurfaceProvider) findComponentById(ResourceTable.Id_remote_player);
remoteSurfaceView.pinToZTop(true);
Optional<SurfaceOps> optional = remoteSurfaceView.getSurfaceOps();
optional.ifPresent(surfaceOps -> surfaceOps.addCallback(
new SurfaceOps.Callback() {
@Override
public void surfaceCreated(SurfaceOps surfaceOps) {
codecPlayer = new CodecPlayer(surfaceOps.getSurface());
cameraController.setCameraListener(codecPlayer);
}
@Override
public void surfaceChanged(
SurfaceOps surfaceOps, int tag, int width, int height) {
}
@Override
public void surfaceDestroyed(SurfaceOps surfaceOps) {
}
}));
}
}
private void initListener() {
if (mirrorSwitch != null) {
mirrorSwitch.setCheckedStateChangedListener((absButton, isMirror) ->
cameraController.setMirrorEffect(isMirror));
}
if (cameraSwitchButton != null) {
cameraSwitchButton.setClickedListener(component -> {
codecPlayer.stop();
cameraController.switchCamera();
getGlobalTaskDispatcher(TaskPriority.DEFAULT)
.delayDispatch(() -> cameraController.capture(), NUMBER_INT_1000);
});
}
if (captureButton != null) {
captureButton.setClickedListener(component -> {
captureToggle();
});
}
}
private void captureToggle() {
if (cameraController.isRecording()) {
cameraController.stopRecord();
captureButton.capture2round();
} else {
cameraController.capture();
if (cameraController.getCaptureMode() == CaptureMode.VIDEO_RECORD) {
captureButton.capture2Rect();
}
}
}
@Override
protected void onActive() {
super.onActive();
}
@Override
protected void onForeground(Intent intent) {
super.onForeground(intent);
}
@Override
protected void onStop() {
super.onStop();
}
}
其中，页面布局文件为ability_codec.xml，示例代码如下：
<?xml version="1.0" encoding="utf-8"?>
<DependentLayout
xmlns:ohos="http://schemas.huawei.com/res/ohos"
ohos:id="$+id:prent_layout"
ohos:height="match_parent"
ohos:width="match_parent"
ohos:alignment="center"
ohos:background_element="#000000">
<com.huawei.codecdemo.camera.view.CameraView
ohos:id="$+id:cameraview"
ohos:height="300vp"
ohos:width="match_parent"/>
<SurfaceProvider
ohos:id="$+id:remote_player"
ohos:height="300vp"
ohos:width="match_parent"
ohos:below="$id:cameraview"
ohos:top_margin="5vp"/>
<DependentLayout
ohos:height="match_content"
ohos:width="match_parent"
ohos:align_parent_bottom="true"
ohos:bottom_margin="15vp">
<Text
ohos:id="$+id:mirror_text"
ohos:height="match_content"
ohos:width="match_content"
ohos:left_margin="10vp"
ohos:text="自拍镜像："
ohos:text_color="#ffffff"
ohos:text_size="14vp"
ohos:vertical_center="true"/>
<Switch
ohos:id="$+id:mirror_switch"
ohos:height="20vp"
ohos:width="40vp"
ohos:right_of="$id:mirror_text"
ohos:text_state_off="off"
ohos:text_state_on="on"
ohos:thumb_element="$graphic:thumb"
ohos:track_element="$graphic:track"
ohos:vertical_center="true"/>
<com.huawei.codecdemo.camera.view.CaptureButton
ohos:id="$+id:button_capture"
ohos:height="50vp"
ohos:width="50vp"
ohos:background_element="$graphic:shape_take_picture_bac"
ohos:center_in_parent="true"
ohos:layout_alignment="horizontal_center"/>
<Image
ohos:id="$+id:image_camera_switch"
ohos:height="40vp"
ohos:width="40vp"
ohos:image_src="$media:ic_camera_switch"
ohos:left_margin="70vp"
ohos:right_of="$id:button_capture"
ohos:scale_mode="stretch"
ohos:vertical_center="true"/>
</DependentLayout>
</DependentLayout>
说明：
以上代码仅demo演示参考使用，产品化的代码需要考虑数据校验和国际化
7. 恭喜您
到这里您已经成功学习了如何通过HarmonyOS编解码播放Camera的实时预览流。
已于2021-7-20 11:36:52修改
赞
3
收藏 1
回复
相关推荐
CameraView.java 里面的东西有没有详细的可以看一看不
有源码吗？
视频解码部分我不是很明白