ZF/SmartJavaAI
1
0
Fork 0
mirror of https://github.com/geekwenjie/SmartJavaAI.git synced 2026-09-10 03:28:49 +00:00
Code Issues Packages Projects Releases Wiki Activity

- 【人脸检测】新增6个模型(MTCNN、YOLOV5、RetinaFace小尺寸版)，大幅提升性能

Browse Source 
- 【人脸识别】新增Seetaface6轻量模型
- 【目标检测】支持视频流目标检测（rtsp、视频文件等）
- 【目标检测】支持tensorflow2目标检测模型
- 【目标检测】新增行人检测模型(yolo-person)
- 【通用视觉】新增4个动作识别模型
- 【通用视觉】新增语义分割模型
- 【通用视觉】新增5个实例分割模型(含yolov8-seg、yolov11-seg)
- 【通用视觉】新增yolo-obb11旋转框检测(含yolov11-obb)
- 【通用视觉】新增5个姿态估计模型(含yolov8-pose、yolov11-pose)
This commit is contained in:
dengwenjie
2025-09-07 17:19:19 +08:00
parent 2b044fda29
commit a8e7ce6c4e
102 changed files with 4473 additions and 1368 deletions
Download Patch File Download Diff File Expand all files Collapse all files

58
examples/vision-example/README.md Normal file
View File

@@ -0,0 +1,58 @@
# 目标检测示例
## 📁 项目结构
```
objectdetection-example/
├── src/
│ ├── main/
│ │ ├── java/
│ │ │ └── smartai/examples/objectdetection/
│ │ │ ├── ObjectDetection.java
│ │ │ └── ViewerFrame.java
```
---
## 🧩 功能模块说明
### 1. 目标检测 [ObjectDetection.java]
- **功能**：核心目标检测类，包含多个测试方法，展示了如何使用不同的模型进行目标检测
---
## ⚙️ 配置要求
- **运行环境**：
- JDK 1.8 或更高版本
- IntelliJ IDEA 推荐作为开发 IDE
- **依赖库**：
- OpenCV、DJL、SmartJavaAI SDK
- **模型路径**：
- 所有模型需下载并配置正确的路径（参考各 demo 注释中的链接）
---
## 🚀 快速开始
1. 克隆项目到本地：
2. 导入项目至 IntelliJ IDEA。
3. 根据需要修改模型路径（见各 demo 中注释）。
4. 运行对应的 JUnit 测试类方法即可体验各项功能。
---
## 📄 文档
有关完整使用说明，请查阅 SmartJavaAI 官方文档：
[http://doc.smartjavaai.cn](http://doc.smartjavaai.cn)
---

352
examples/vision-example/pom.xml Normal file
View File

@@ -0,0 +1,352 @@
<?xml version="1.0" encoding="UTF-8"?>
<project xmlns="http://maven.apache.org/POM/4.0.0"
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
<modelVersion>4.0.0</modelVersion>
<groupId>cn.smartjavaai</groupId>
<artifactId>examples</artifactId>
<version>1.0.0-SNAPSHOT</version>
<properties>
<maven.compiler.source>11</maven.compiler.source>
<maven.compiler.target>11</maven.compiler.target>
<project.build.sourceEncoding>UTF-8</project.build.sourceEncoding>
<smartjavaai.version>1.0.24</smartjavaai.version>
<!--如果打包运行，需要替换成你的main-->
<exec.mainClass>smartai.examples.vision.ObjectDetectionDemo</exec.mainClass>
<javacv.version>1.5.10</javacv.version>
<javacv.platform.macosx-arm64>macosx-arm64</javacv.platform.macosx-arm64>
<javacv.platform.linux-x86_64>linux-x86_64</javacv.platform.linux-x86_64>
<javacv.platform.linux-arm64>linux-arm64</javacv.platform.linux-arm64>
<javacv.platform.windows-x86_64>windows-x86_64</javacv.platform.windows-x86_64>
<djl.platform.windows-x86_64>win-x86_64</djl.platform.windows-x86_64>
<djl.platform.linux-x86_64>linux-x86_64</djl.platform.linux-x86_64>
<djl.platform.linux-aarch64>linux-aarch64</djl.platform.linux-aarch64>
<djl.platform.osx-aarch64>osx-aarch64</djl.platform.osx-aarch64>
</properties>
<dependencyManagement>
<dependencies>
<dependency>
<groupId>cn.smartjavaai</groupId>
<artifactId>bom</artifactId>
<version>${smartjavaai.version}</version>
<type>pom</type>
<!-- 注意这里是import -->
<scope>import</scope>
</dependency>
</dependencies>
</dependencyManagement>
<dependencies>
<dependency>
<groupId>commons-cli</groupId>
<artifactId>commons-cli</artifactId>
<version>1.9.0</version>
</dependency>
<dependency>
<groupId>commons-io</groupId>
<artifactId>commons-io</artifactId>
<version>2.17.0</version>
</dependency>
<dependency>
<groupId>org.testng</groupId>
<artifactId>testng</artifactId>
<version>7.10.2</version>
<scope>test</scope>
</dependency>
<dependency>
<groupId>ch.qos.logback</groupId>
<artifactId>logback-classic</artifactId>
<version>1.2.3</version>
</dependency>
<dependency>
<groupId>com.alibaba</groupId>
<artifactId>fastjson</artifactId>
<version>1.2.83</version>
</dependency>
<dependency>
<groupId>junit</groupId>
<artifactId>junit</artifactId>
<version>4.13.2</version>
</dependency>
<!--目标检测模块-->
<dependency>
<groupId>cn.smartjavaai</groupId>
<artifactId>vision</artifactId>
</dependency>
<dependency>
<groupId>ai.djl.pytorch</groupId>
<artifactId>pytorch-jni</artifactId>
<version>2.5.1-0.32.0</version>
<scope>runtime</scope>
</dependency>
<!-- windows平台 (保留对应平台的配置，可以减小包大小)-->
<dependency>
<groupId>org.bytedeco</groupId>
<artifactId>javacpp</artifactId>
<version>${javacv.version}</version>
<classifier>${javacv.platform.windows-x86_64}</classifier>
</dependency>
<dependency>
<groupId>org.bytedeco</groupId>
<artifactId>ffmpeg</artifactId>
<version>6.1.1-1.5.10</version>
<classifier>${javacv.platform.windows-x86_64}</classifier>
</dependency>
<dependency>
<groupId>org.bytedeco</groupId>
<artifactId>openblas</artifactId>
<version>0.3.26-1.5.10</version>
<classifier>${javacv.platform.windows-x86_64}</classifier>
</dependency>
<dependency>
<groupId>org.bytedeco</groupId>
<artifactId>opencv</artifactId>
<version>4.9.0-1.5.10</version>
<classifier>${javacv.platform.windows-x86_64}</classifier>
</dependency>
<dependency>
<groupId>ai.djl.pytorch</groupId>
<artifactId>pytorch-native-cpu</artifactId>
<classifier>${djl.platform.windows-x86_64}</classifier>
<version>2.5.1</version>
<scope>runtime</scope>
</dependency>
<dependency>
<groupId>ai.djl.tensorflow</groupId>
<artifactId>tensorflow-native-cpu</artifactId>
<classifier>win-x86_64</classifier>
<scope>runtime</scope>
<version>2.16.1</version>
</dependency>
<dependency>
<groupId>ai.djl.mxnet</groupId>
<artifactId>mxnet-native-mkl</artifactId>
<classifier>win-x86_64</classifier>
<scope>runtime</scope>
<version>1.9.1</version>
</dependency>
<!-- linux x86 平台 (保留对应平台的配置，可以减小包大小)-->
<dependency>
<groupId>org.bytedeco</groupId>
<artifactId>javacpp</artifactId>
<version>${javacv.version}</version>
<classifier>${javacv.platform.linux-x86_64}</classifier>
</dependency>
<dependency>
<groupId>org.bytedeco</groupId>
<artifactId>ffmpeg</artifactId>
<version>6.1.1-1.5.10</version>
<classifier>${javacv.platform.linux-x86_64}</classifier>
</dependency>
<dependency>
<groupId>org.bytedeco</groupId>
<artifactId>openblas</artifactId>
<version>0.3.26-1.5.10</version>
<classifier>${javacv.platform.linux-x86_64}</classifier>
</dependency>
<dependency>
<groupId>org.bytedeco</groupId>
<artifactId>opencv</artifactId>
<version>4.9.0-1.5.10</version>
<classifier>${javacv.platform.linux-x86_64}</classifier>
</dependency>
<dependency>
<groupId>ai.djl.pytorch</groupId>
<artifactId>pytorch-native-cpu</artifactId>
<classifier>${djl.platform.linux-x86_64}</classifier>
<version>2.5.1</version>
<scope>runtime</scope>
</dependency>
<dependency>
<groupId>ai.djl.tensorflow</groupId>
<artifactId>tensorflow-native-cpu</artifactId>
<classifier>linux-x86_64</classifier>
<scope>runtime</scope>
<version>2.16.1</version>
</dependency>
<dependency>
<groupId>ai.djl.mxnet</groupId>
<artifactId>mxnet-native-mkl</artifactId>
<classifier>linux-x86_64</classifier>
<scope>runtime</scope>
<version>1.9.1</version>
</dependency>
<dependency>
<groupId>ai.djl.pytorch</groupId>
<artifactId>pytorch-native-cpu-precxx11</artifactId>
<classifier>${djl.platform.linux-x86_64}</classifier>
<version>2.5.1</version>
<scope>runtime</scope>
</dependency>
<!-- macOS M系列 平台 (保留对应平台的配置，可以减小包大小)-->
<dependency>
<groupId>org.bytedeco</groupId>
<artifactId>javacpp</artifactId>
<version>${javacv.version}</version>
<classifier>${javacv.platform.macosx-arm64}</classifier>
</dependency>
<dependency>
<groupId>org.bytedeco</groupId>
<artifactId>ffmpeg</artifactId>
<version>6.1.1-1.5.10</version>
<classifier>${javacv.platform.macosx-arm64}</classifier>
</dependency>
<dependency>
<groupId>org.bytedeco</groupId>
<artifactId>openblas</artifactId>
<version>0.3.26-1.5.10</version>
<classifier>${javacv.platform.macosx-arm64}</classifier>
</dependency>
<dependency>
<groupId>org.bytedeco</groupId>
<artifactId>opencv</artifactId>
<version>4.9.0-1.5.10</version>
<classifier>${javacv.platform.macosx-arm64}</classifier>
</dependency>
<dependency>
<groupId>ai.djl.pytorch</groupId>
<artifactId>pytorch-native-cpu</artifactId>
<classifier>${djl.platform.osx-aarch64}</classifier>
<version>2.5.1</version>
<scope>runtime</scope>
</dependency>
<dependency>
<groupId>ai.djl.tensorflow</groupId>
<artifactId>tensorflow-native-cpu</artifactId>
<classifier>osx-aarch64</classifier>
<version>2.16.1</version>
<scope>runtime</scope>
</dependency>
<dependency>
<groupId>ai.djl.tensorflow</groupId>
<artifactId>tensorflow-native-cpu</artifactId>
<classifier>osx-aarch64</classifier>
<version>2.16.1</version>
<scope>runtime</scope>
</dependency>
<dependency>
<groupId>ai.djl.mxnet</groupId>
<artifactId>mxnet-native-mkl</artifactId>
<classifier>osx-x86_64</classifier>
<version>1.9.1</version>
<scope>runtime</scope>
</dependency>
<!-- linux aarch64 平台 (保留对应平台的配置，可以减小包大小)-->
<dependency>
<groupId>org.bytedeco</groupId>
<artifactId>javacpp</artifactId>
<version>${javacv.version}</version>
<classifier>${javacv.platform.linux-arm64}</classifier>
</dependency>
<dependency>
<groupId>org.bytedeco</groupId>
<artifactId>ffmpeg</artifactId>
<version>6.1.1-1.5.10</version>
<classifier>${javacv.platform.linux-arm64}</classifier>
</dependency>
<dependency>
<groupId>org.bytedeco</groupId>
<artifactId>openblas</artifactId>
<version>0.3.26-1.5.10</version>
<classifier>${javacv.platform.linux-arm64}</classifier>
</dependency>
<dependency>
<groupId>org.bytedeco</groupId>
<artifactId>opencv</artifactId>
<version>4.9.0-1.5.10</version>
<classifier>${javacv.platform.linux-arm64}</classifier>
</dependency>
</dependencies>
<build>
<plugins>
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-shade-plugin</artifactId>
<version>3.5.0</version>
<executions>
<execution>
<phase>package</phase>
<goals><goal>shade</goal></goals>
<configuration>
<createDependencyReducedPom>false</createDependencyReducedPom>
<transformers>
<transformer implementation="org.apache.maven.plugins.shade.resource.ServicesResourceTransformer"/>
<transformer implementation="org.apache.maven.plugins.shade.resource.ManifestResourceTransformer">
<mainClass>${exec.mainClass}</mainClass>
</transformer>
</transformers>
</configuration>
</execution>
</executions>
</plugin>
</plugins>
</build>
<repositories>
<!-- <repository>-->
<!-- <id>aliyunmaven</id>-->
<!-- <name>阿里云公共仓库</name>-->
<!-- <url>https://maven.aliyun.com/repository/public</url>-->
<!-- <releases>-->
<!-- <enabled>true</enabled>-->
<!-- </releases>-->
<!-- <snapshots>-->
<!-- <enabled>false</enabled>-->
<!-- </snapshots>-->
<!-- </repository>-->
<repository>
<id>central</id>
<url>https://repo1.maven.org/maven2/</url>
</repository>
</repositories>
</project>

92
examples/vision-example/src/main/java/smartai/examples/vision/ActionRecognizeDemo.java Normal file
View File

@@ -0,0 +1,92 @@
package smartai.examples.vision;
import ai.djl.modality.Classifications;
import ai.djl.modality.cv.Image;
import cn.smartjavaai.action.config.ActionRecModelConfig;
import cn.smartjavaai.action.enums.ActionRecModelEnum;
import cn.smartjavaai.action.model.ActionRecModel;
import cn.smartjavaai.action.model.ActionRecModelFactory;
import cn.smartjavaai.common.cv.SmartImageFactory;
import cn.smartjavaai.common.entity.R;
import cn.smartjavaai.common.enums.DeviceEnum;
import lombok.extern.slf4j.Slf4j;
import org.apache.commons.collections.CollectionUtils;
import org.junit.BeforeClass;
import org.junit.Test;
import java.io.IOException;
import java.nio.file.Paths;
import java.util.Arrays;
/**
* 动作识别Demo
* 模型下载地址：https://pan.baidu.com/s/17doY4pgZM9EbtSIaoCWWCA?pwd=1234 提取码: 1234
* 文档地址：http://doc.smartjavaai.cn/
* @author dwj
*/
@Slf4j
public class ActionRecognizeDemo {
//设备类型
public static DeviceEnum device = DeviceEnum.CPU;
@BeforeClass
public static void beforeAll() throws IOException {
//修改缓存路径
// Config.setCachePath("/Users/xxx/smartjavaai_cache");
}
/**
* 获取动作识别模型
* 注意事项：
* 1、不同模型支持的动作类别不同，请查看文档：http://doc.smartjavaai.cn
*/
public ActionRecModel getModel(){
ActionRecModelConfig config = new ActionRecModelConfig();
//动作识别模型切换时，需要同时更新 modelEnum 和 modelPath。其中部分 modelEnum 对应多个模型文件，可通过指定 modelPath 来选择具体的模型。
config.setModelEnum(ActionRecModelEnum.INCEPTIONV3_KINETICS400_ONNX);
//模型所在路径
config.setModelPath("/Users/wenjie/Documents/develop/model/action/gluoncv-inceptionv3_kinetics400-695477a5.onnx");
config.setDevice(device);
//置信度阈值
config.setThreshold(0.5f);
//指定允许的类别
// config.setAllowedClasses(Arrays.asList("dancing_ballet"));
return ActionRecModelFactory.getInstance().getModel(config);
}
/**
* 动作识别
* 注意事项：
* 1、不同模型支持的动作类别不同，请查看文档：http://doc.smartjavaai.cn
* 2、图片中应该只包含单一动作人物
* 3、动作识别，只做图片分类，并不做人物定位
*/
@Test
public void actionRecognition(){
try {
ActionRecModel detectorModel = getModel();
//创建Image对象，可以从文件、url、InputStream创建、BufferedImage、Base64创建，具体使用方法可以查看文档
Image image = SmartImageFactory.getInstance().fromFile(Paths.get("src/main/resources/action/dance.jpg"));
R<Classifications> result = detectorModel.detect(image);
if(result.isSuccess()){
if(CollectionUtils.isNotEmpty(result.getData().getClassNames())){
//分数最高分类
log.info("动作识别结果：{}", result.getData().best().toString());
//按分数排序前5个结果
// log.info("动作识别结果：{}", result.getData().topK(5).toString());
}else{
log.info("未识别到动作");
}
}else{
log.info("动作识别失败：{}", result.getMessage());
}
} catch (Exception e) {
e.printStackTrace();
}
}
}

135
examples/vision-example/src/main/java/smartai/examples/vision/InstanceSegDemo.java Normal file
View File

@@ -0,0 +1,135 @@
package smartai.examples.vision;
import ai.djl.modality.Classifications;
import ai.djl.modality.cv.Image;
import cn.smartjavaai.action.model.ActionRecModel;
import cn.smartjavaai.common.cv.SmartImageFactory;
import cn.smartjavaai.common.entity.DetectionResponse;
import cn.smartjavaai.common.entity.R;
import cn.smartjavaai.common.enums.DeviceEnum;
import cn.smartjavaai.common.utils.ImageUtils;
import cn.smartjavaai.instanceseg.config.InstanceSegModelConfig;
import cn.smartjavaai.instanceseg.enums.InstanceSegModelEnum;
import cn.smartjavaai.instanceseg.model.InstanceSegModel;
import cn.smartjavaai.instanceseg.model.InstanceSegModelFactory;
import cn.smartjavaai.objectdetection.model.person.PersonDetModel;
import com.alibaba.fastjson.JSONObject;
import lombok.extern.slf4j.Slf4j;
import org.junit.Assert;
import org.junit.BeforeClass;
import org.junit.Test;
import javax.imageio.ImageIO;
import java.awt.image.BufferedImage;
import java.io.File;
import java.io.IOException;
import java.nio.file.Paths;
/**
* 实例分割 Demo
* 模型下载地址：https://pan.baidu.com/s/12nRRY9JFNDwLeg63jfBerA?pwd=1234 提取码: 1234
* 文档地址：http://doc.smartjavaai.cn/
* @author dwj
*/
@Slf4j
public class InstanceSegDemo {
//设备类型
public static DeviceEnum device = DeviceEnum.CPU;
@BeforeClass
public static void beforeAll() throws IOException {
//修改缓存路径
// Config.setCachePath("/Users/xxx/smartjavaai_cache");
}
/**
* 获取实例分割模型
* 注意事项：
* 1、更多模型请查看文档：http://doc.smartjavaai.cn
*/
public InstanceSegModel getModel(){
InstanceSegModelConfig config = new InstanceSegModelConfig();
//实例分割模型，切换模型需要同时修改modelEnum及modelPath
config.setModelEnum(InstanceSegModelEnum.SEG_YOLO11N_ONNX);
//模型所在路径，synset.txt也需要放在同目录下
config.setModelPath("/Users/wenjie/Documents/develop/model/vision/instance/yolo11n-seg-onnx/yolo11n-seg.onnx");
// 指定允许的类别
// config.setAllowedClasses(Arrays.asList("person","car"));
//指定返回检测数量
config.setDevice(device);
//置信度阈值
config.setThreshold(0.5f);
return InstanceSegModelFactory.getInstance().getModel(config);
}
/**
* 实例分割
*/
@Test
public void instanceSegmentation(){
try {
InstanceSegModel detectorModel = getModel();
//创建Image对象，可以从文件、url、InputStream创建、BufferedImage、Base64创建，具体使用方法可以查看文档
Image image = SmartImageFactory.getInstance().fromFile(Paths.get("src/main/resources/dog_bike_car.jpg"));
R<DetectionResponse> result = detectorModel.detect(image);
if(result.isSuccess()){
log.info("实例分割结果：{}", result.getData());
}else{
log.info("实例分割失败：{}", result.getMessage());
}
} catch (Exception e) {
e.printStackTrace();
}
}
/**
* 实例分割并绘制检测结果
*/
@Test
public void instanceSegmentationAndDraw(){
try {
InstanceSegModel detectorModel = getModel();
R<DetectionResponse> result = detectorModel.detectAndDraw("src/main/resources/dog_bike_car.jpg","output/dog_bike_car_detected.png");
if(result.isSuccess()){
log.info("实例分割结果：{}", JSONObject.toJSONString(result.getData()));
}else{
log.info("实例分割失败：{}", result.getMessage());
}
} catch (Exception e) {
e.printStackTrace();
}
}
/**
* 实例分割并绘制检测结果
*/
@Test
public void instanceSegmentationAndDraw2(){
try {
InstanceSegModel detectorModel = getModel();
//创建Image对象，可以从文件、url、InputStream创建、BufferedImage、Base64创建，具体使用方法可以查看文档
Image image = SmartImageFactory.getInstance().fromFile(Paths.get("src/main/resources/dog_bike_car.jpg"));
//可以根据后续业务场景使用detectedImage
R<DetectionResponse> result = detectorModel.detectAndDraw(image);
if(result.isSuccess()){
log.info("实例分割结果：{}", JSONObject.toJSONString(result.getData()));
//保存图片
ImageUtils.saveImage(result.getData().getDrawnImage(), "dog_bike_car_detected.png", "output");
}else{
log.info("实例分割失败：{}", result.getMessage());
}
} catch (Exception e) {
e.printStackTrace();
}
}
}

129
examples/vision-example/src/main/java/smartai/examples/vision/ObbDetDemo.java Normal file
View File

@@ -0,0 +1,129 @@
package smartai.examples.vision;
import ai.djl.modality.cv.Image;
import cn.smartjavaai.common.cv.SmartImageFactory;
import cn.smartjavaai.common.entity.DetectionResponse;
import cn.smartjavaai.common.entity.R;
import cn.smartjavaai.common.enums.DeviceEnum;
import cn.smartjavaai.common.utils.ImageUtils;
import cn.smartjavaai.instanceseg.config.InstanceSegModelConfig;
import cn.smartjavaai.instanceseg.enums.InstanceSegModelEnum;
import cn.smartjavaai.instanceseg.model.InstanceSegModelFactory;
import cn.smartjavaai.obb.config.ObbDetModelConfig;
import cn.smartjavaai.obb.enums.ObbDetModelEnum;
import cn.smartjavaai.obb.model.ObbDetModel;
import cn.smartjavaai.obb.model.ObbDetModelFactory;
import com.alibaba.fastjson.JSONObject;
import lombok.extern.slf4j.Slf4j;
import org.junit.BeforeClass;
import org.junit.Test;
import java.io.IOException;
import java.nio.file.Paths;
import java.util.Arrays;
/**
* obb旋转框检测demo
* 模型下载地址：https://pan.baidu.com/s/1-tC0u-aha3tnMQwy8FKy1Q?pwd=1234 提取码: 1234
* 文档地址：http://doc.smartjavaai.cn/
* @author dwj
*/
@Slf4j
public class ObbDetDemo {
//设备类型
public static DeviceEnum device = DeviceEnum.CPU;
@BeforeClass
public static void beforeAll() throws IOException {
//修改缓存路径
// Config.setCachePath("/Users/xxx/smartjavaai_cache");
}
/**
* 获取旋转框检测模型
* 注意事项：
* 1、更多模型请查看文档：http://doc.smartjavaai.cn
* 2、模型可检测物体请查看：模型同目录文件synset.txt
*/
public ObbDetModel getModel(){
ObbDetModelConfig config = new ObbDetModelConfig();
//旋转框检测模型，切换模型需要同时修改modelEnum及modelPath
config.setModelEnum(ObbDetModelEnum.YOLOV11);
//模型所在路径，synset.txt也需要放在同目录下
config.setModelPath("/Users/wenjie/Documents/develop/model/vision/obb/yolo11n-obb.onnx");
// 指定允许的类别
// config.setAllowedClasses(Arrays.asList("plane","ship"));
//指定返回检测数量
config.setDevice(device);
//置信度阈值
config.setThreshold(0.5f);
return ObbDetModelFactory.getInstance().getModel(config);
}
/**
* 旋转框检测
*/
@Test
public void obbDet(){
try {
ObbDetModel detectorModel = getModel();
//创建Image对象，可以从文件、url、InputStream创建、BufferedImage、Base64创建，具体使用方法可以查看文档
Image image = SmartImageFactory.getInstance().fromFile(Paths.get("src/main/resources/obb/boats.jpg"));
R<DetectionResponse> result = detectorModel.detect(image);
if(result.isSuccess()){
log.info("旋转框检测结果：{}", JSONObject.toJSONString(result.getData()));
}else{
log.info("旋转框检测失败：{}", result.getMessage());
}
} catch (Exception e) {
e.printStackTrace();
}
}
/**
* 旋转框检测并绘制检测结果
*/
@Test
public void obbDetAndDraw(){
try {
ObbDetModel detectorModel = getModel();
R<DetectionResponse> result = detectorModel.detectAndDraw("src/main/resources/obb/boats.jpg","output/boats_detected.png");
if(result.isSuccess()){
log.info("旋转框检测结果：{}", JSONObject.toJSONString(result.getData()));
}else{
log.info("旋转框检测失败：{}", result.getMessage());
}
} catch (Exception e) {
e.printStackTrace();
}
}
/**
* 旋转框检测并绘制检测结果
*/
@Test
public void obbDetAndDraw2(){
try {
ObbDetModel detectorModel = getModel();
//创建Image对象，可以从文件、url、InputStream创建、BufferedImage、Base64创建，具体使用方法可以查看文档
Image image = SmartImageFactory.getInstance().fromFile(Paths.get("src/main/resources/obb/boats.jpg"));
//可以根据后续业务场景使用detectedImage
R<DetectionResponse> result = detectorModel.detectAndDraw(image);
if(result.isSuccess()){
log.info("旋转框检测结果：{}", JSONObject.toJSONString(result.getData()));
//保存图片
ImageUtils.saveImage(result.getData().getDrawnImage(), "boats_obb_detected.png", "output");
}else{
log.info("旋转框检测失败：{}", result.getMessage());
}
} catch (Exception e) {
e.printStackTrace();
}
}
}

420
examples/vision-example/src/main/java/smartai/examples/vision/ObjectDetectionDemo.java Normal file
View File

@@ -0,0 +1,420 @@
package smartai.examples.vision;
import ai.djl.modality.cv.Image;
import ai.djl.modality.cv.ImageFactory;
import ai.djl.util.JsonUtils;
import cn.hutool.core.date.LocalDateTimeUtil;
import cn.hutool.core.lang.UUID;
import cn.smartjavaai.common.entity.DetectionInfo;
import cn.smartjavaai.common.entity.DetectionRectangle;
import cn.smartjavaai.common.entity.DetectionResponse;
import cn.smartjavaai.common.enums.DeviceEnum;
import cn.smartjavaai.common.enums.VideoSourceType;
import cn.smartjavaai.common.utils.ImageUtils;
import cn.smartjavaai.common.utils.OpenCVUtils;
import cn.smartjavaai.objectdetection.config.DetectorModelConfig;
import cn.smartjavaai.objectdetection.enums.DetectorModelEnum;
import cn.smartjavaai.objectdetection.model.DetectorModel;
import cn.smartjavaai.objectdetection.model.ObjectDetectionModelFactory;
import cn.smartjavaai.objectdetection.stream.StreamDetectionListener;
import cn.smartjavaai.objectdetection.stream.StreamDetector;
import com.alibaba.fastjson.JSONObject;
import lombok.extern.slf4j.Slf4j;
import nu.pattern.OpenCV;
import org.junit.Assert;
import org.junit.BeforeClass;
import org.junit.Test;
import org.opencv.core.Mat;
import org.opencv.core.Size;
import org.opencv.imgproc.Imgproc;
import org.opencv.videoio.VideoCapture;
import org.opencv.videoio.Videoio;
import javax.imageio.ImageIO;
import javax.swing.*;
import java.awt.*;
import java.awt.image.BufferedImage;
import java.io.File;
import java.io.IOException;
import java.nio.file.Paths;
import java.util.*;
import java.util.List;
import java.util.concurrent.CountDownLatch;
/**
* 目标检测模型demo
* 模型下载地址：https://pan.baidu.com/s/10aTOLBlR6EG-sq6g0OkAWg?pwd=1234 提取码: 1234
* 文档地址：http://doc.smartjavaai.cn/
* @author dwj
*/
@Slf4j
public class ObjectDetectionDemo {
//设备类型
public static DeviceEnum device = DeviceEnum.CPU;
@BeforeClass
public static void beforeAll() throws IOException {
//修改缓存路径
// Config.setCachePath("/Users/xxx/smartjavaai_cache");
}
/**
* 获取目标检测模型
* 注意事项：
* 1、更多模型请查看文档：http://doc.smartjavaai.cn/objectdetect.html
*/
public DetectorModel getModel(){
DetectorModelConfig config = new DetectorModelConfig();
//目标检测模型，切换模型需要同时修改modelEnum及modelPath
config.setModelEnum(DetectorModelEnum.YOLOV12_OFFICIAL_ONNX);
//模型所在路径，synset.txt也需要放在同目录下
config.setModelPath("/Users/wenjie/Documents/develop/model/vision/object/yolov12/yolov12n.onnx");
// 指定允许的类别
// config.setAllowedClasses(Arrays.asList("person","car"));
//指定返回检测数量
config.setTopK(100);
config.setDevice(device);
//置信度阈值
config.setThreshold(0.5f);
return ObjectDetectionModelFactory.getInstance().getModel(config);
}
/**
* 目标检测
*/
@Test
public void objectDetection(){
try {
DetectorModel detectorModel = getModel();
DetectionResponse detectionResponse = detectorModel.detect("src/main/resources/object_detection.jpg");
log.info("目标检测结果：{}", JSONObject.toJSONString(detectionResponse));
} catch (Exception e) {
e.printStackTrace();
}
}
/**
* 目标检测并绘制检测结果
*/
@Test
public void objectDetectionAndDraw(){
try {
DetectorModel detectorModel = getModel();
detectorModel.detectAndDraw("src/main/resources/object_detection.jpg","output/object_detection_detected.png");
} catch (Exception e) {
e.printStackTrace();
}
}
/**
* 目标检测并绘制检测结果,返回BufferedImage
*/
@Test
public void objectDetectionAndDraw2(){
try {
DetectorModel detectorModel = getModel();
String imagePath = "src/main/resources/object_detection.jpg";
BufferedImage image = ImageIO.read(new File(Paths.get(imagePath).toAbsolutePath().toString()));
//可以根据后续业务场景使用detectedImage
BufferedImage detectedImage = detectorModel.detectAndDraw(image);
Assert.assertNotNull("detectedImage null", detectedImage);
} catch (Exception e) {
e.printStackTrace();
}
}
/**
* 使用自己训练的模型检测
*/
@Test
public void objectDetectionWithCustomModel(){
try {
DetectorModelConfig config = new DetectorModelConfig();
//目标检测模型，切换模型需要同时修改modelEnum及modelPath
config.setModelEnum(DetectorModelEnum.YOLOV12_CUSTOM_ONNX);
//模型所在路径，synset.txt也需要放在同目录下(分类文件，具体请看文档：http://doc.smartjavaai.cn/objectdetect.html#%E4%BD%BF%E7%94%A8%E8%87%AA%E5%B7%B1%E8%AE%AD%E7%BB%83%E7%9A%84%E6%A8%A1%E5%9E%8B%E6%A3%80%E6%B5%8B)
config.setModelPath("/Users/xxx/Documents/develop/fire_model/best.onnx");
//模型训练时图片宽度
config.putCustomParam("width", 640);//resize 宽
//模型训练时图片高度
config.putCustomParam("height", 640);// resize 高
config.putCustomParam("nmsThreshold", 0.5f);
// 指定允许的类别
// config.setAllowedClasses(Arrays.asList("person"));
//指定返回检测数量
config.setTopK(100);
config.setDevice(device);
DetectorModel detectorModel = ObjectDetectionModelFactory.getInstance().getModel(config);
DetectionResponse detect = detectorModel.detect("src/main/resources/dog_bike_car.jpg");
log.info("目标检测结果：{}", JSONObject.toJSONString(detect));
} catch (Exception e) {
e.printStackTrace();
}
}
/**
* tensorflow2目标检测
* 注意事项：
* 1、百度网盘只提供部分模型，更多tensorflow模型可以前往官网下载：https://github.com/tensorflow/models/blob/master/research/object_detection/g3doc/tf2_detection_zoo.md
*/
@Test
public void objectDetection3(){
try {
DetectorModelConfig config = new DetectorModelConfig();
//指定模型枚举，可以通过modelPath指定不同tensorflow模型
config.setModelEnum(DetectorModelEnum.TENSORFLOW2_OFFICIAL);
//模型路径，需解压模型压缩包，可以通过modelPath指定不同tensorflow模型
config.setModelPath("/Users/wenjie/Documents/develop/model/tensorflow/ssd_mobilenet_v2_320x320_coco17_tpu-8");
// config.putCustomParam("synsetUrl", "https://raw.githubusercontent.com/tensorflow/models/master/research/object_detection/data/mscoco_label_map.pbtxt");
// config.putCustomParam("synsetPath", "/Users/wenjie/Downloads/mscoco_label_map.pbtxt.txt");
//分类文件，需下载放入模型路径下
config.putCustomParam("synsetFileName", "mscoco.pbtxt");
// 指定允许的类别
// config.setAllowedClasses(Arrays.asList("person"));
//指定返回检测数量
config.setTopK(100);
config.setDevice(device);
DetectorModel detectorModel = ObjectDetectionModelFactory.getInstance().getModel(config);
DetectionResponse detectionResponse = detectorModel.detect("src/main/resources/dog_bike_car.jpg");
//检测并保存绘制结果
detectorModel.detectAndDraw("src/main/resources/dog_bike_car.jpg", "output/dog_bike_car_detect.jpg");
log.info("目标检测结果：{}", JSONObject.toJSONString(detectionResponse));
} catch (Exception e) {
e.printStackTrace();
}
}
/**
* 视频流目标检测
*/
@Test
public void testStream(){
StreamDetector detector = new StreamDetector.Builder()
//视频源类型：支持视频流、本地摄像头、视频文件
.sourceType(VideoSourceType.STREAM)
//视频流地址，支持rtsp、rtmp、http等常见视频流
.streamUrl("rtsp://username:password@ip:port/Streaming/Channels/101")
//每隔多少帧检测一次（需要根据模型检测速度决定）
.frameDetectionInterval(10)
//目标检测模型
.detectorModel(getModel())
//回调函数：检测到指定目标时触发（getModel中可指定模型检测的物体）
.listener(new StreamDetectionListener() {
/**
* 建议把耗时操作放到新线程里执行
* @param detectionInfoList 目标信息列表
* @param image 检测到的图片
*/
@Override
public void onObjectDetected(List<DetectionInfo> detectionInfoList, Image image) {
log.info("时间：" + LocalDateTimeUtil.now().toString());
log.info("检测结果：{}", JsonUtils.toJson(detectionInfoList));
//绘制检测结果
OpenCVUtils.drawRectAndText(image, detectionInfoList);
//保存图片
ImageUtils.saveImage(image, "test"+ UUID.fastUUID().toString() +".png","/Users/wenjie/Downloads");
if (image != null){
((Mat)image.getWrappedImage()).release();
}
}
@Override
public void onStreamEnded() {
log.info("视频流检测结束");
}
@Override
public void onStreamDisconnected() {
log.info("视频流断开连接");
}
}).build();
detector.startDetection();
//阻塞主线程
CountDownLatch latch = new CountDownLatch(1);
try {
latch.await(); // 一直阻塞，直到被 countDown
} catch (InterruptedException e) {
throw new RuntimeException(e);
}
}
/**
* 本地摄像头目标检测
*/
@Test
public void testLocalCamera(){
StreamDetector detector = new StreamDetector.Builder()
//视频源类型：支持视频流、本地摄像头、视频文件
.sourceType(VideoSourceType.CAMERA)
//摄像头序号
.cameraIndex(0)
//每隔多少帧检测一次（需要根据模型检测速度决定）
.frameDetectionInterval(5)
//目标检测模型
.detectorModel(getModel())
//回调函数：检测到指定目标时触发（getModel中可指定模型检测的物体）
.listener(new StreamDetectionListener() {
@Override
public void onObjectDetected(List<DetectionInfo> detectionInfoList, Image image) {
log.info("时间：" + LocalDateTimeUtil.now().toString());
log.info("检测结果：{}", JsonUtils.toJson(detectionInfoList));
//绘制检测结果
OpenCVUtils.drawRectAndText(image, detectionInfoList);
//保存图片
ImageUtils.saveImage(image, "test"+ UUID.fastUUID().toString() +".png","/Users/wenjie/Downloads");
}
@Override
public void onStreamEnded() {
log.info("视频流检测结束");
}
@Override
public void onStreamDisconnected() {
log.info("视频流断开连接");
}
}).build();
detector.startDetection();
//阻塞主线程
CountDownLatch latch = new CountDownLatch(1);
try {
latch.await(); // 一直阻塞，直到被 countDown
} catch (InterruptedException e) {
throw new RuntimeException(e);
}
}
/**
* 视频文件目标检测
*/
@Test
public void testVideoFile(){
StreamDetector detector = new StreamDetector.Builder()
//视频源类型：支持视频流、本地摄像头、视频文件
.sourceType(VideoSourceType.FILE)
//摄像头序号
.streamUrl("girl.mp4")
//每隔多少帧检测一次（需要根据模型检测速度决定）
.frameDetectionInterval(5)
//目标检测模型
.detectorModel(getModel())
//同物体重复检测时间间隔，单位s
.repeatGap(5)
//回调函数：检测到指定目标时触发（getModel中可指定模型检测的物体）
.listener(new StreamDetectionListener() {
@Override
public void onObjectDetected(List<DetectionInfo> detectionInfoList, Image image) {
log.info("时间：" + LocalDateTimeUtil.now().toString());
log.info("检测结果：{}", JsonUtils.toJson(detectionInfoList));
//绘制检测结果
OpenCVUtils.drawRectAndText(image, detectionInfoList);
//保存图片
ImageUtils.saveImage(image, "test"+ UUID.fastUUID().toString() +".png","/Users/wenjie/Downloads");
}
@Override
public void onStreamEnded() {
log.info("视频流检测结束");
}
@Override
public void onStreamDisconnected() {
log.info("视频流断开连接");
}
}).build();
detector.startDetection();
//阻塞主线程
CountDownLatch latch = new CountDownLatch(1);
try {
latch.await(); // 一直阻塞，直到被 countDown
} catch (InterruptedException e) {
throw new RuntimeException(e);
}
}
/**
* 摄像头目标检测并实时预览
* 注意事项：如果视频比较卡，可以使用更轻量的检测模型
*/
@Test
public void testDetectCamera(){
try {
DetectorModel detectorModel = getModel();
OpenCV.loadShared();
VideoCapture capture = new VideoCapture(0);
if (!capture.isOpened()) {
System.out.println("No camera detected");
return;
}
double ratio =
capture.get(Videoio.CAP_PROP_FRAME_WIDTH)
/ capture.get(Videoio.CAP_PROP_FRAME_HEIGHT);
Dimension screenSize = Toolkit.getDefaultToolkit().getScreenSize();
int height = (int) (screenSize.height * 0.65f);
int width = (int) (height * ratio);
if (width > screenSize.width) {
width = screenSize.width;
}
Mat image = new Mat();
boolean captured = false;
for (int i = 0; i < 10; ++i) {
captured = capture.read(image);
if (captured) {
break;
}
try {
Thread.sleep(50);
} catch (InterruptedException ignore) {
// ignore
}
}
if (!captured) {
JOptionPane.showConfirmDialog(null, "Failed to capture image from WebCam.");
}
ViewerFrame frame = new ViewerFrame(width, height);
ImageFactory factory = ImageFactory.getInstance();
Size size = new Size(width, height);
while (capture.isOpened()) {
if (!capture.read(image)) {
break;
}
Mat resizeImage = new Mat();
Imgproc.resize(image, resizeImage, size);
Image img = factory.fromImage(resizeImage);
BufferedImage bufferedImage = OpenCVUtils.mat2Image(resizeImage);
DetectionResponse detectedResult = detectorModel.detect(bufferedImage);
if (Objects.isNull(detectedResult) || Objects.isNull(detectedResult.getDetectionInfoList()) || detectedResult.getDetectionInfoList().size() == 0){
log.debug("未检测到物体");
continue;
}
for(DetectionInfo detectionInfo : detectedResult.getDetectionInfoList()){
DetectionRectangle detectionRectangle = detectionInfo.getDetectionRectangle();
String text = detectionInfo.getObjectDetInfo().getClassName();
ImageUtils.drawImageRectWithText(bufferedImage, detectionRectangle, text, Color.RED);
}
frame.showImage(bufferedImage);
}
capture.release();
System.exit(0);
} catch (Exception e) {
throw new RuntimeException(e);
}
}
}

121
examples/vision-example/src/main/java/smartai/examples/vision/PersonDetectDemo.java Normal file
View File

@@ -0,0 +1,121 @@
package smartai.examples.vision;
import ai.djl.modality.cv.Image;
import cn.smartjavaai.common.cv.SmartImageFactory;
import cn.smartjavaai.common.entity.DetectionResponse;
import cn.smartjavaai.common.entity.R;
import cn.smartjavaai.common.enums.DeviceEnum;
import cn.smartjavaai.common.utils.ImageUtils;
import cn.smartjavaai.objectdetection.config.PersonDetModelConfig;
import cn.smartjavaai.objectdetection.enums.PersonDetectorModelEnum;
import cn.smartjavaai.objectdetection.model.person.PersonDetModel;
import cn.smartjavaai.objectdetection.model.person.PersonDetModelFactory;
import com.alibaba.fastjson.JSONObject;
import lombok.extern.slf4j.Slf4j;
import org.junit.BeforeClass;
import org.junit.Test;
import java.io.IOException;
import java.nio.file.Paths;
/**
* 行人检测案例
* 模型下载地址：https://pan.baidu.com/s/1EWfExw7pYjKEH5uR5wf3Rw?pwd=1234 提取码: 1234
* 文档地址：http://doc.smartjavaai.cn/
* @author dwj
*/
@Slf4j
public class PersonDetectDemo {
//设备类型
public static DeviceEnum device = DeviceEnum.CPU;
@BeforeClass
public static void beforeAll() throws IOException {
//修改缓存路径
// Config.setCachePath("/Users/xxx/smartjavaai_cache");
}
/**
* 获取行人检测模型
*/
public PersonDetModel getModel(){
PersonDetModelConfig config = new PersonDetModelConfig();
//行人检测模型，切换模型需要同时修改modelEnum及modelPath
config.setModelEnum(PersonDetectorModelEnum.YOLOV8_PERSON);
//模型所在路径
config.setModelPath("/Users/wenjie/Documents/develop/model/person/yolov8n-person.onnx");
//指定返回检测数量
config.setTopK(100);
config.setDevice(device);
//置信度阈值
config.setThreshold(0.5f);
return PersonDetModelFactory.getInstance().getModel(config);
}
/**
* 行人检测
*/
@Test
public void objectDetection(){
try {
PersonDetModel detectorModel = getModel();
//创建Image对象，可以从文件、url、InputStream创建、BufferedImage、Base64创建，具体使用方法可以查看文档
Image image = SmartImageFactory.getInstance().fromFile(Paths.get("src/main/resources/person/person.png"));
R<DetectionResponse> result = detectorModel.detect(image);
if(result.isSuccess()){
log.info("行人检测结果：{}", JSONObject.toJSONString(result.getData()));
}else{
log.info("行人检测失败：{}", result.getMessage());
}
} catch (Exception e) {
e.printStackTrace();
}
}
/**
* 行人检测并绘制检测结果
*/
@Test
public void objectDetectionAndDraw(){
try {
PersonDetModel detectorModel = getModel();
//保存绘制后图片以及返回检测结果
R<DetectionResponse> result = detectorModel.detectAndDraw("src/main/resources/person/person.png","output/person_detected.png");
if(result.isSuccess()){
log.info("行人检测结果：{}", JSONObject.toJSONString(result.getData()));
}else{
log.info("行人检测失败：{}", result.getMessage());
}
} catch (Exception e) {
e.printStackTrace();
}
}
/**
* 行人检测并绘制检测结果
*/
@Test
public void objectDetectionAndDraw2(){
try {
PersonDetModel detectorModel = getModel();
//创建Image对象，可以从文件、url、InputStream创建、BufferedImage、Base64创建，具体使用方法可以查看文档
Image image = SmartImageFactory.getInstance().fromFile(Paths.get("src/main/resources/person/person.png"));
//可以根据后续业务场景使用detectedImage
R<DetectionResponse> result = detectorModel.detectAndDraw(image);
if(result.isSuccess()){
log.info("行人检测结果：{}", JSONObject.toJSONString(result.getData()));
//保存图片
ImageUtils.saveImage(result.getData().getDrawnImage(), "person_result.png", "output");
}else{
log.info("行人检测失败：{}", result.getMessage());
}
} catch (Exception e) {
e.printStackTrace();
}
}
}

118
examples/vision-example/src/main/java/smartai/examples/vision/PoseDetDemo.java Normal file
View File

@@ -0,0 +1,118 @@
package smartai.examples.vision;
import ai.djl.modality.cv.Image;
import ai.djl.modality.cv.output.Joints;
import cn.smartjavaai.common.cv.SmartImageFactory;
import cn.smartjavaai.common.entity.DetectionResponse;
import cn.smartjavaai.common.entity.R;
import cn.smartjavaai.common.enums.DeviceEnum;
import cn.smartjavaai.common.utils.ImageUtils;
import cn.smartjavaai.pose.config.PoseModelConfig;
import cn.smartjavaai.pose.enums.PoseModelEnum;
import cn.smartjavaai.pose.model.PoseDetModelFactory;
import cn.smartjavaai.pose.model.PoseModel;
import com.alibaba.fastjson.JSONObject;
import lombok.extern.slf4j.Slf4j;
import org.junit.BeforeClass;
import org.junit.Test;
import java.io.IOException;
import java.nio.file.Paths;
/**
* 姿态估计demo
* 模型下载地址：https://pan.baidu.com/s/1pPYyl1V2CpcMYCO8CJQHGg?pwd=1234 提取码: 1234
* 文档地址：http://doc.smartjavaai.cn/
* @author dwj
*/
@Slf4j
public class PoseDetDemo {
//设备类型
public static DeviceEnum device = DeviceEnum.CPU;
@BeforeClass
public static void beforeAll() throws IOException {
//修改缓存路径
// Config.setCachePath("/Users/xxx/smartjavaai_cache");
}
/**
* 获取姿态估计模型
* 注意事项：
* 1、更多模型请查看文档：http://doc.smartjavaai.cn
* 2、模型可检测物体请查看：模型同目录文件synset.txt
*/
public PoseModel getModel(){
PoseModelConfig config = new PoseModelConfig();
//姿态估计模型，切换模型需要同时修改modelEnum及modelPath
config.setModelEnum(PoseModelEnum.YOLOV8N_POSE_PT);
//模型所在路径，synset.txt也需要放在同目录下
config.setModelPath("/Users/wenjie/Documents/develop/model/vision/pose/yolo11n-pose-onnx/yolo11n-pose.onnx");
config.setDevice(device);
//置信度阈值
config.setThreshold(0.25f);
return PoseDetModelFactory.getInstance().getModel(config);
}
/**
* 姿态估计
*/
@Test
public void poseDet(){
try {
PoseModel detectorModel = getModel();
//创建Image对象，可以从文件、url、InputStream创建、BufferedImage、Base64创建，具体使用方法可以查看文档
Image image = SmartImageFactory.getInstance().fromFile(Paths.get("src/main/resources/pose/pose_soccer.png"));
R<Joints[]> result = detectorModel.detect(image);
if(result.isSuccess()){
log.info("姿态估计结果：{}", JSONObject.toJSONString(result.getData()));
}else{
log.info("姿态估计失败：{}", result.getMessage());
}
} catch (Exception e) {
e.printStackTrace();
}
}
/**
* 姿态估计并绘制检测结果
*/
@Test
public void poseDetAndDraw(){
try {
PoseModel detectorModel = getModel();
R<Joints[]> result = detectorModel.detectAndDraw("src/main/resources/pose/pose_soccer.png","output/pose_detected.png");
if(result.isSuccess()){
log.info("姿态估计结果：{}", JSONObject.toJSONString(result.getData()));
}else{
log.info("姿态估计失败：{}", result.getMessage());
}
} catch (Exception e) {
e.printStackTrace();
}
}
/**
* 姿态估计并绘制检测结果
*/
@Test
public void poseDetAndDraw2(){
try {
PoseModel detectorModel = getModel();
//创建Image对象，可以从文件、url、InputStream创建、BufferedImage、Base64创建，具体使用方法可以查看文档
Image image = SmartImageFactory.getInstance().fromFile(Paths.get("src/main/resources/pose/pose_soccer.png"));
//可以根据后续业务场景使用detectedImage
Image drawImage = detectorModel.detectAndDraw(image);
//保存图片
ImageUtils.saveImage(drawImage, "pose_detected.png", "output");
} catch (Exception e) {
e.printStackTrace();
}
}
}

123
examples/vision-example/src/main/java/smartai/examples/vision/SemSegDemo.java Normal file
View File

@@ -0,0 +1,123 @@
package smartai.examples.vision;
import ai.djl.modality.cv.Image;
import ai.djl.modality.cv.output.CategoryMask;
import cn.smartjavaai.common.cv.SmartImageFactory;
import cn.smartjavaai.common.entity.DetectionResponse;
import cn.smartjavaai.common.entity.R;
import cn.smartjavaai.common.enums.DeviceEnum;
import cn.smartjavaai.common.utils.ImageUtils;
import cn.smartjavaai.semseg.config.SemSegModelConfig;
import cn.smartjavaai.semseg.enums.SemSegModelEnum;
import cn.smartjavaai.semseg.model.SemSegModel;
import cn.smartjavaai.semseg.model.SemSegModelFactory;
import com.alibaba.fastjson.JSONObject;
import lombok.extern.slf4j.Slf4j;
import org.junit.BeforeClass;
import org.junit.Test;
import java.io.IOException;
import java.nio.file.Paths;
import java.util.Arrays;
/**
* 语义分割 Demo 通过网盘分享的文件：语义分割（semantic_segmentation）
* 模型下载地址：https://pan.baidu.com/s/18gs9E5h_d9imPmNLHuDo9A?pwd=1234 提取码: 1234
* 文档地址：http://doc.smartjavaai.cn/
* @author dwj
*/
@Slf4j
public class SemSegDemo {
//设备类型
public static DeviceEnum device = DeviceEnum.CPU;
@BeforeClass
public static void beforeAll() throws IOException {
//修改缓存路径
// Config.setCachePath("/Users/xxx/smartjavaai_cache");
}
/**
* 获取语义分割模型
* 注意事项：
* 1、更多模型请查看文档：http://doc.smartjavaai.cn
*/
public SemSegModel getModel(){
SemSegModelConfig config = new SemSegModelConfig();
//语义分割模型，切换模型需要同时修改modelEnum及modelPath
config.setModelEnum(SemSegModelEnum.DEEPLABV3);
//模型所在路径，synset.txt也需要放在同目录下
config.setModelPath("/Users/wenjie/Documents/develop/model/vision/semseg/deeplabv3/deeplabv3.pt");
// 指定允许的类别
// config.setAllowedClasses(Arrays.asList("person","car"));
//指定返回检测数量
config.setDevice(device);
return SemSegModelFactory.getInstance().getModel(config);
}
/**
* 语义分割
*/
@Test
public void semSeg(){
try {
SemSegModel detectorModel = getModel();
//创建Image对象，可以从文件、url、InputStream创建、BufferedImage、Base64创建，具体使用方法可以查看文档
Image image = SmartImageFactory.getInstance().fromFile(Paths.get("src/main/resources/dog_bike_car.jpg"));
R<CategoryMask> result = detectorModel.detect(image);
if(result.isSuccess()){
log.info("语义分割结果：{}", result.getData());
}else{
log.info("语义分割失败：{}", result.getMessage());
}
} catch (Exception e) {
e.printStackTrace();
}
}
/**
* 语义分割并绘制检测结果
*/
@Test
public void semSegAndDraw(){
try {
SemSegModel detectorModel = getModel();
R<CategoryMask> result = detectorModel.detectAndDraw("src/main/resources/dog_bike_car.jpg","output/dog_bike_car_semseg.png");
if(result.isSuccess()){
log.info("语义分割结果：{}", result.getData());
}else{
log.info("语义分割失败：{}", result.getMessage());
}
} catch (Exception e) {
e.printStackTrace();
}
}
/**
* 语义分割并绘制检测结果
*/
@Test
public void semSegAndDraw2(){
try {
SemSegModel detectorModel = getModel();
//创建Image对象，可以从文件、url、InputStream创建、BufferedImage、Base64创建，具体使用方法可以查看文档
Image image = SmartImageFactory.getInstance().fromFile(Paths.get("src/main/resources/dog_bike_car.jpg"));
//可以根据后续业务场景使用detectedImage
Image dretectedImage = detectorModel.detectAndDraw(image);
//保存
ImageUtils.saveImage(dretectedImage, "dog_bike_car_detected.png", "output");
} catch (Exception e) {
e.printStackTrace();
}
}
}

70
examples/vision-example/src/main/java/smartai/examples/vision/ViewerFrame.java Normal file
View File

@@ -0,0 +1,70 @@
/*
* Copyright 2020 Amazon.com, Inc. or its affiliates. All Rights Reserved.
*
* Licensed under the Apache License, Version 2.0 (the "License"). You may not use this file except in compliance
* with the License. A copy of the License is located at
*
* http://aws.amazon.com/apache2.0/
*
* or in the "license" file accompanying this file. This file is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES
* OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions
* and limitations under the License.
*/
package smartai.examples.vision;
import javax.swing.*;
import java.awt.*;
import java.awt.image.BufferedImage;
public class ViewerFrame {
private JFrame frame;
private ImagePanel imagePanel;
public ViewerFrame(int width, int height) {
frame = new JFrame("Demo");
imagePanel = new ImagePanel();
frame.setLayout(new BorderLayout());
frame.add(BorderLayout.CENTER, imagePanel);
JOptionPane.setRootFrame(frame);
Dimension screenSize = Toolkit.getDefaultToolkit().getScreenSize();
if (width > screenSize.width) {
width = screenSize.width;
}
Dimension frameSize = new Dimension(width, height);
frame.setSize(frameSize);
frame.setLocation((screenSize.width - width) / 2, (screenSize.height - height) / 2);
frame.setDefaultCloseOperation(WindowConstants.EXIT_ON_CLOSE);
frame.setVisible(true);
}
public void showImage(BufferedImage image) {
imagePanel.setImage(image);
SwingUtilities.invokeLater(
() -> {
frame.repaint();
frame.pack();
});
}
private static final class ImagePanel extends JPanel {
private BufferedImage image;
void setImage(BufferedImage image) {
this.image = image;
}
@Override
public void paintComponent(Graphics g) {
super.paintComponent(g);
if (image == null) {
return;
}
g.drawImage(image, 0, 0, null);
setPreferredSize(new Dimension(image.getWidth(), image.getHeight()));
}
}
}

3
examples/vision-example/src/main/resources/META-INF/MANIFEST.MF Normal file
View File

@@ -0,0 +1,3 @@
Manifest-Version: 1.0
Main-Class: smartai.examples.vision.ObjectDetectionDemo

BIN
examples/vision-example/src/main/resources/action/calling.jpeg Normal file
View File

Binary file not shown.
Side by Side

After

Width:  |  Height:  |  Size: 69 KiB

BIN
examples/vision-example/src/main/resources/action/dance.jpg Normal file
View File

Binary file not shown.
Side by Side

After

Width:  |  Height:  |  Size: 18 KiB

BIN
examples/vision-example/src/main/resources/dog_bike_car.jpg Normal file
View File

Binary file not shown.
Side by Side

After

Width:  |  Height:  |  Size: 160 KiB

BIN
examples/vision-example/src/main/resources/largest_selfie.jpg Normal file
View File

Binary file not shown.
Side by Side

After

Width:  |  Height:  |  Size: 463 KiB

14
examples/vision-example/src/main/resources/logback.xml Normal file
View File

@@ -0,0 +1,14 @@
<?xml version="1.0" encoding="UTF-8"?>
<!-- 步骤2: 配置文件 (src/main/resources/logback.xml) -->
<configuration scan="true" scanPeriod="30 seconds">
<!-- 控制台日志输出 -->
<appender name="CONSOLE" class="ch.qos.logback.core.ConsoleAppender">
<encoder>
<pattern>%d{yyyy-MM-dd HH:mm:ss.SSS} [%thread] %highlight(%-5level) %cyan(%logger{36}) - %msg%n</pattern>
</encoder>
</appender>
<root level="DEBUG">
<appender-ref ref="CONSOLE" />
</root>
</configuration>

BIN
examples/vision-example/src/main/resources/obb/boats.jpg Normal file
View File

Binary file not shown.
Side by Side

After

Width:  |  Height:  |  Size: 190 KiB

BIN
examples/vision-example/src/main/resources/object_detection.jpg Normal file
View File

Binary file not shown.
Side by Side

After

Width:  |  Height:  |  Size: 1.4 MiB

BIN
examples/vision-example/src/main/resources/person/person.png Normal file
View File

Binary file not shown.
Side by Side

After

Width:  |  Height:  |  Size: 440 KiB

BIN
examples/vision-example/src/main/resources/pose/pose_soccer.png Normal file
View File

Binary file not shown.
Side by Side

After

Width:  |  Height:  |  Size: 1.5 MiB

BIN
examples/vision-example/src/main/resources/segmentation.jpg Normal file
View File

Binary file not shown.
Side by Side

After

Width:  |  Height:  |  Size: 89 KiB

BIN
examples/vision-example/src/main/resources/zero/000000039769.jpg Normal file
View File

Binary file not shown.
Side by Side

After

Width:  |  Height:  |  Size: 169 KiB