[HarmonyOS Next] 使用Core Vision Kit基础视觉服务骨骼点检测样例原创

第一小趴菜

发布于 2024-12-3 18:52

浏览

1收藏

前言

Core Vision Kit（基础视觉服务）是机器视觉相关的基础能力，例如通用文字识别（即OCR，Optical Character Recognition，也称为光学字符识别）、人脸检测、人脸比对以及主体分割等能力。
本章给大家一个识别骨骼点的一个样例
版本： HarmonyOS Beta(api12)

实现方式

1.导入图片

我们通过以下方法将本地的图片导入应用，openPhoto方法选取本地图片的连接，通过loadImage将图片加载到我们的应用中

  private async selectImage() {
    let uri = await this.openPhoto()
    if (uri === undefined) {
      hilog.error(0x0000, 'skeletonDetectSample', "Failed to defined uri.");
    }
    this.loadImage(uri)
  }

  private openPhoto(): Promise<string> {
    return new Promise<string>((resolve, reject) => {
      let photoPicker: photoAccessHelper.PhotoViewPicker = new photoAccessHelper.PhotoViewPicker();
      photoPicker.select({
        MIMEType: photoAccessHelper.PhotoViewMIMETypes.IMAGE_TYPE, maxSelectNumber: 1
      }).then(res => {
        resolve(res.photoUris[0])
      }).catch((err: BusinessError) => {
        hilog.error(0x0000, 'skeletonDetectSample', `Failed to get photo image uri. code：${err.code}，message：${err.message}`);
        reject('')
      })
    })
  }

  private loadImage(name: string) {
    setTimeout(async () => {
      let fileSource = await fileIo.open(name, fileIo.OpenMode.READ_ONLY);
      this.imageSource = image.createImageSource(fileSource.fd);
      this.chooseImage = await this.imageSource.createPixelMap();
    }, 100)
  }
1.
2.
3.
4.
5.
6.
7.
8.
9.
10.
11.
12.
13.
14.
15.
16.
17.
18.
19.
20.
21.
22.
23.
24.
25.
26.
27.
28.
29.

2.识别

          if(!this.chooseImage) {
            hilog.error(0x0000, 'skeletonDetectSample', `Failed to choose image. chooseImage: ${this.chooseImage}`);
            return;
          }
          let request: visionBase.Request = {
            inputData: { pixelMap: this.chooseImage, }
          };
          // Call the skeleton detection API.
          let data: skeletonDetection.SkeletonDetectionResponse = await (await skeletonDetection.SkeletonDetector.create()).process(request);
          let poseJson = JSON.stringify(data);
          hilog.info(0x0000, 'skeletonDetectSample', `Succeeded in face detect：${poseJson}`);
          this.dataValues = poseJson;
1.
2.
3.
4.
5.
6.
7.
8.
9.
10.
11.
12.

效果展示

上方为我们导入的图片的样子，而下方为我们导入图片的骨骼点坐标集合，分别为：
支持17个关键点的识别，具体为鼻子，左右眼，左右耳，左右肩，左右肘、左右手腕、左右髋、左右膝、左右脚踝。

[HarmonyOS Next] 使用Core Vision Kit基础视觉服务骨骼点检测样例-鸿蒙开发者社区

完整代码

import { image } from '@kit.ImageKit';
import { hilog } from '@kit.PerformanceAnalysisKit';
import { BusinessError } from '@kit.BasicServicesKit';
import { fileIo } from '@kit.CoreFileKit';
import { skeletonDetection, visionBase } from '@kit.CoreVisionKit';
import { photoAccessHelper } from '@kit.MediaLibraryKit';

@Entry
@Component
struct Index {
  private imageSource: image.ImageSource | undefined = undefined;
  @State chooseImage: PixelMap | undefined = undefined
  @State dataValues: string = ''

  build() {
    Column() {
      Image(this.chooseImage)
        .objectFit(ImageFit.Contain)
        .height('60%')


      Scroll() {
        Text(this.dataValues)
          .copyOption(CopyOptions.LocalDevice)
          .margin(10)
          .width('60%')
      }
      .height('15%')
      .scrollable(ScrollDirection.Vertical)

      Button('导入图片')
        .type(ButtonType.Capsule)
        .fontColor(Color.White)
        .alignSelf(ItemAlign.Center)
        .width('80%')
        .margin(10)
        .onClick(() => {
          // Pull up the gallery.
          this.selectImage()
        })

      Button('识别')
        .type(ButtonType.Capsule)
        .fontColor(Color.White)
        .alignSelf(ItemAlign.Center)
        .width('80%')
        .margin(10)
        .onClick(async () => {
          if(!this.chooseImage) {
            hilog.error(0x0000, 'skeletonDetectSample', `Failed to choose image. chooseImage: ${this.chooseImage}`);
            return;
          }
          let request: visionBase.Request = {
            inputData: { pixelMap: this.chooseImage, }
          };
          // Call the skeleton detection API.
          let data: skeletonDetection.SkeletonDetectionResponse = await (await skeletonDetection.SkeletonDetector.create()).process(request);
          let poseJson = JSON.stringify(data);
          hilog.info(0x0000, 'skeletonDetectSample', `Succeeded in face detect：${poseJson}`);
          this.dataValues = poseJson;
        })
    }
    .width('100%')
    .height('100%')
    .justifyContent(FlexAlign.Center)
  }

  private async selectImage() {
    let uri = await this.openPhoto()
    if (uri === undefined) {
      hilog.error(0x0000, 'skeletonDetectSample', "Failed to defined uri.");
    }
    this.loadImage(uri)
  }

  private openPhoto(): Promise<string> {
    return new Promise<string>((resolve, reject) => {
      let photoPicker: photoAccessHelper.PhotoViewPicker = new photoAccessHelper.PhotoViewPicker();
      photoPicker.select({
        MIMEType: photoAccessHelper.PhotoViewMIMETypes.IMAGE_TYPE, maxSelectNumber: 1
      }).then(res => {
        resolve(res.photoUris[0])
      }).catch((err: BusinessError) => {
        hilog.error(0x0000, 'skeletonDetectSample', `Failed to get photo image uri. code：${err.code}，message：${err.message}`);
        reject('')
      })
    })
  }

  private loadImage(name: string) {
    setTimeout(async () => {
      let fileSource = await fileIo.open(name, fileIo.OpenMode.READ_ONLY);
      this.imageSource = image.createImageSource(fileSource.fd);
      this.chooseImage = await this.imageSource.createPixelMap();
    }, 100)
  }
}

1.
2.
3.
4.
5.
6.
7.
8.
9.
10.
11.
12.
13.
14.
15.
16.
17.
18.
19.
20.
21.
22.
23.
24.
25.
26.
27.
28.
29.
30.
31.
32.
33.
34.
35.
36.
37.
38.
39.
40.
41.
42.
43.
44.
45.
46.
47.
48.
49.
50.
51.
52.
53.
54.
55.
56.
57.
58.
59.
60.
61.
62.
63.
64.
65.
66.
67.
68.
69.
70.
71.
72.
73.
74.
75.
76.
77.
78.
79.
80.
81.
82.
83.
84.
85.
86.
87.
88.
89.
90.
91.
92.
93.
94.
95.
96.
97.
98.

分类

HarmonyOS

其他

标签

HarmonyOS

已于2024-12-3 18:52:34修改

51CTO

51CTO博客

51CTO学堂

[HarmonyOS Next] 使用Core Vision Kit基础视觉服务骨骼点检测样例原创

前言

实现方式

1.导入图片

2.识别

效果展示

完整代码

目录

订阅鸿蒙技术特刊，精选内容抢先看

51CTO

51CTO博客

51CTO学堂

[HarmonyOS Next] 使用Core Vision Kit基础视觉服务 骨骼点检测样例 原创

前言

实现方式

1.导入图片

2.识别

效果展示

完整代码

目录

订阅鸿蒙技术特刊，精选内容抢先看

[HarmonyOS Next] 使用Core Vision Kit基础视觉服务骨骼点检测样例原创