1、集成车牌识别模型,支持车牌检测与识别
2、新增 Milvus 身份验证支持 3、目标检测功能升级:可指定类别及topk 4、支持自定义线程池线程数量
92
README.md
@@ -192,6 +192,25 @@ SmartJavaAI是专为JAVA 开发者打造的一个功能丰富、开箱即用的
|
||||
</div>
|
||||
</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td>
|
||||
<div align="left">
|
||||
<p>车牌识别</p>
|
||||
- 单层/双层检测 <br>
|
||||
- 车牌颜色识别 <br>
|
||||
</div>
|
||||
</td>
|
||||
<td>
|
||||
<div align="center">
|
||||
<img src="https://cdn.jsdelivr.net/gh/geekwenjie/SmartJavaAI-Site/images/ocr/plate_recognized.jpg" width = "500px"/>
|
||||
</div>
|
||||
</td>
|
||||
<td>
|
||||
<div align="center">
|
||||
<img src="https://cdn.jsdelivr.net/gh/geekwenjie/SmartJavaAI-Site/images/ocr/plate_recognized2.jpg" width = "500px"/>
|
||||
</div>
|
||||
</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td>
|
||||
<div align="left">
|
||||
@@ -251,21 +270,22 @@ SmartJavaAI是专为JAVA 开发者打造的一个功能丰富、开箱即用的
|
||||
- 支持任意角度识别,方向校准
|
||||
- 支持通用文字识别,通用手写字识别
|
||||
- 支持表格识别
|
||||
- 支持中文车牌识别:单层/双层检测,颜色识别,支持12种中文车牌
|
||||
- **机器翻译**
|
||||
- 集成NLLB-200模型:支持200+语言互相翻译
|
||||
|
||||
|
||||
## 🌟 AI集成方式对比
|
||||
|
||||
| 方案 | 技术特点 | 优点 | 缺点 |
|
||||
|---------------------|--------------------------------------------------------------------------|---------------------------------------------------------------------|---------------------------------------------------------------------|
|
||||
| **OpenCV** | 传统图像处理方案 | ✅ 提供java接口<br>✅ 轻量级部署<br>✅ 社区资源丰富 | ❌ 基于传统算法精度低(60%-75%)<br>❌ 需本地安装环境 |
|
||||
| ‌**商业闭源SDK(如虹软等)**‌ | 商业级闭源解决方案 | ✅ 开箱即用<br>✅ 提供完整文档和SDK<br>✅ 支持离线活体检测 | ❌ 免费版需年度授权更新<br>❌ 商业授权费用高<br>❌ 代码不可控 |
|
||||
| ‌**云API(阿里云)**‌ | SaaS化云端服务 | ✅ 零部署成本<br>✅ 支持高并发<br>✅ 自带模型迭代 | ❌ 网络延迟风险(200-800ms)<br>❌ 按调用量计费<br>❌ 有数据安全风险 |
|
||||
| 方案 | 技术特点 | 优点 | 缺点 |
|
||||
|---------------------|--------------------------------------------------------------------------|---------------------------------------------------------------------|-----------------------------------------------|
|
||||
| **OpenCV** | 传统图像处理方案 | ✅ 提供java接口<br>✅ 轻量级部署<br>✅ 社区资源丰富 | ❌ 基于传统算法精度低(60%-75%)<br>❌ 需本地安装环境 |
|
||||
| ‌**商业闭源SDK(如虹软等)**‌ | 商业级闭源解决方案 | ✅ 开箱即用<br>✅ 提供完整文档和SDK<br>✅ 支持离线活体检测 | ❌ 免费版需年度授权更新<br>❌ 商业授权费用高<br>❌ 代码不可控 |
|
||||
| ‌**云API(阿里云)**‌ | SaaS化云端服务 | ✅ 零部署成本<br>✅ 支持高并发<br>✅ 自带模型迭代 | ❌ 网络延迟风险(200-800ms)<br>❌ 按调用量计费<br>❌ 有数据安全风险 |
|
||||
| ‌**Python混合调用**‌ | 跨语言调用方案 | ✅ 可集成PyTorch/TF等框架<br>✅ 支持自定义算法<br>✅ 识别精度高 | ❌ 需维护双语言环境<br>❌ 进程通信性能损耗(30%+)<br>❌ 异常处理复杂度翻倍 |
|
||||
| ‌**JNI/JNA**‌ | 跨语言底层调用方案 | ✅ 直接调用 C/C++ 高性能算法库✅ 支持调用各种原生成熟库✅ 可封装成通用工具Jar |❌ 开发成本高,JNI更复杂❌ 跨平台兼容性差 |
|
||||
| ‌**DJL框架**‌ | 深度学习框架 | ✅ 纯Java实现<br>✅ 支持主流深度学习框架<br>✅ 可加载预训练模型(99%+) | ❌ 需掌握DL知识<br>❌ 需处理模型加载、预处理、后处理等复杂技术细节 |
|
||||
| ‌**SmartJavaAI**‌ | java深度学习工具包 | ✅ 支持主流深度学习框架<br>✅ 提供丰富、开箱即用API<br> ✅ 上手简单,单一Jar包集成 | ❌要求JDK版本11及以上 |
|
||||
| ‌**JNI/JNA**‌ | 跨语言底层调用方案 | ✅ 直接调用 C/C++ 高性能算法库✅ 支持调用各种原生成熟库✅ 可封装成通用工具Jar | ❌ 开发成本高,JNI更复杂❌ 跨平台兼容性差 |
|
||||
| ‌**DJL框架**‌ | 深度学习框架 | ✅ 纯Java实现<br>✅ 支持主流深度学习框架<br>✅ 可加载预训练模型(99%+) | ❌ 需掌握DL知识<br>❌ 需处理模型加载、预处理、后处理等复杂技术细节 |
|
||||
| ‌**SmartJavaAI**‌ | java深度学习工具包 | ✅ 支持主流深度学习框架<br>✅ 提供丰富、开箱即用API<br> ✅ 上手简单,单一Jar包集成 | 无 |
|
||||
|
||||
|
||||
|
||||
@@ -300,7 +320,7 @@ SmartJavaAI是专为JAVA 开发者打造的一个功能丰富、开箱即用的
|
||||
<dependency>
|
||||
<groupId>cn.smartjavaai</groupId>
|
||||
<artifactId>smartjavaai-all</artifactId>
|
||||
<version>1.0.20</version>
|
||||
<version>1.0.22</version>
|
||||
</dependency>
|
||||
```
|
||||
### 3、完整示例代码
|
||||
@@ -406,16 +426,6 @@ SmartJavaAI是专为JAVA 开发者打造的一个功能丰富、开箱即用的
|
||||
- 自训练模型推理
|
||||
- yolov3~yolov12 系列
|
||||
|
||||
**SSD 系列**
|
||||
|
||||
| 模型名称 | 引擎 | 骨干网络 | 输入尺寸 | <div style="width: 60pt">训练数据集</div> | 精度(mAP) | <div style="width: 50pt">推理速度</div> | <div style="width: 150pt">适用场景</div>|
|
||||
| :-------- |------------------| ------------- | ----------- |--------------------------------------| -------------|-------------------------------------| -------------|
|
||||
|SSD_300_RESNET5| PyTorch | ResNet‑50 | 300×300 | COCO | 中等 | 快 | 精度需求一般|
|
||||
|SSD_512_RESNET50_V1_VOC| PyTorch | ResNet‑50 | 512×512 | Pascal VOC | 稍高 | 中等 | 精度优先、可接受略低速度的场景|
|
||||
|SSD_512_VGG16_ATROUS_COCO| MXNet | VGG‑16 | 512×512 | COCO | 较高 | 中等 | 通用场景;对小目标有一定提升|
|
||||
|SSD_300_VGG16_ATROUS_VOC| MXNet | VGG‑16 | 300×300 | Pascal VOC | 中等偏上 | 快 | VOC 数据集同类任务;资源受限时使用|
|
||||
|SSD_512_MOBILENET1_VOC| MXNet | MobileNet‑1.0 | 512×512 | Pascal VOC | 中等 | 快 | 嵌入式/移动端设备;算力和内存都很有限|
|
||||
|
||||
**YOLO 系列**
|
||||
|
||||
|模型名称 | 引擎 | 版本 | 大小(Backbone) | <div style="width: 60pt">数据集</div> | <div style="width: 50pt">精度</div> | <div style="width: 50pt">速度</div> | <div style="width: 150pt">适用场景</div> |
|
||||
@@ -427,6 +437,18 @@ SmartJavaAI是专为JAVA 开发者打造的一个功能丰富、开箱即用的
|
||||
|YOLOV5S_ONNXRUNTIME| OnnxRuntime | v5 | 小型 | COCO | 较高 | 加速(需 ONNX 支持) | Windows/Linux 通用加速部署 |
|
||||
|YOLO (MXNet / 通用模型) | MXNet | v3 | DarkNet‑53 | COCO | 较高 | 快 | 需要 MXNet 生态或复现老项目时使用 |
|
||||
|
||||
|
||||
**SSD 系列**
|
||||
|
||||
| 模型名称 | 引擎 | 骨干网络 | 输入尺寸 | <div style="width: 60pt">训练数据集</div> | 精度(mAP) | <div style="width: 50pt">推理速度</div> | <div style="width: 150pt">适用场景</div>|
|
||||
| :-------- |------------------| ------------- | ----------- |--------------------------------------| -------------|-------------------------------------| -------------|
|
||||
|SSD_300_RESNET5| PyTorch | ResNet‑50 | 300×300 | COCO | 中等 | 快 | 精度需求一般|
|
||||
|SSD_512_RESNET50_V1_VOC| PyTorch | ResNet‑50 | 512×512 | Pascal VOC | 稍高 | 中等 | 精度优先、可接受略低速度的场景|
|
||||
|SSD_512_VGG16_ATROUS_COCO| MXNet | VGG‑16 | 512×512 | COCO | 较高 | 中等 | 通用场景;对小目标有一定提升|
|
||||
|SSD_300_VGG16_ATROUS_VOC| MXNet | VGG‑16 | 300×300 | Pascal VOC | 中等偏上 | 快 | VOC 数据集同类任务;资源受限时使用|
|
||||
|SSD_512_MOBILENET1_VOC| MXNet | MobileNet‑1.0 | 512×512 | Pascal VOC | 中等 | 快 | 嵌入式/移动端设备;算力和内存都很有限|
|
||||
|
||||
|
||||
**YOLOv3 变体系列**
|
||||
|
||||
|模型名称| 引擎 | 骨干网络 | <div style="width: 60pt">数据集</div> | 输入尺寸 | <div style="width: 50pt">精度</div> | <div style="width: 50pt">速度</div> | <div style="width: 200pt">适用场景</div> |
|
||||
@@ -486,6 +508,21 @@ SmartJavaAI是专为JAVA 开发者打造的一个功能丰富、开箱即用的
|
||||
| SLANet_plus | (增强版)该模型通过轻量级骨干 PP-LCNet、CSP-PAN 融合与 SLA Head 解码,有效提升表格结构识别的精度与速度。 | [Github](https://paddlepaddle.github.io/PaddleOCR/v3.1.0/version3.x/module_usage/table_structure_recognition.html#_3) |
|
||||
|
||||
|
||||
**车牌检测模型(License Plate Detection)**
|
||||
|
||||
| 模型名称 | 模型简介 | 模型开源网站 |
|
||||
|--------|----------------------|------------|
|
||||
| YOLOV5 | 基于YOLOV5训练,支持12种中文车牌 | [Github](https://github.com/we0091234/Chinese_license_plate_detection_recognition) |
|
||||
| YOLOV7 | 基于YOLOV7训练,支持12种中文车牌 | [Github](https://github.com/we0091234/Chinese_license_plate_detection_recognition) |
|
||||
|
||||
|
||||
**车牌识别模型(License Plate Recognition)**
|
||||
|
||||
| 模型名称 | 模型简介 | 模型开源网站 |
|
||||
|--------|------------|------------|
|
||||
| PLATE_REC_CRNN | CRNN中文字符识别 | [Github](https://github.com/Sierkinhane/CRNN_Chinese_Characters_Rec) |
|
||||
|
||||
|
||||
---
|
||||
|
||||
#### 机器翻译模型
|
||||
@@ -519,7 +556,22 @@ SmartJavaAI是专为JAVA 开发者打造的一个功能丰富、开箱即用的
|
||||
|
||||
## 近期更新日志
|
||||
|
||||
## [v1.0.20] - 2025-07-06
|
||||
## [v1.0.22] - 2025-07-28
|
||||
- 新增 Milvus 身份验证支持
|
||||
- 集成车牌识别模型,支持车牌检测与识别
|
||||
- 目标检测功能升级:可指定类别及topk
|
||||
- 支持自定义线程池线程数量
|
||||
|
||||
|
||||
## [v1.0.20] - 2025-07-18
|
||||
- OCR:新增表格识别模型
|
||||
- OCR:新增9个通用模型
|
||||
- OCR:支持批量检测识别
|
||||
- OCR:新增更多参数,使用更加灵活
|
||||
- 人脸识别:支持ID查询及分页获取人脸信息
|
||||
- 活体检测:视频检测支持设置最大帧数
|
||||
|
||||
## [v1.0.19] - 2025-07-06
|
||||
- 人脸模块:新增小视科技(MiniVision)活体检测模型
|
||||
- 人脸模块:新增阿里通义工作室活体检测模型
|
||||
- 人脸模块:新增 2 个表情识别模型
|
||||
|
||||
@@ -12,7 +12,7 @@
|
||||
<maven.compiler.source>11</maven.compiler.source>
|
||||
<maven.compiler.target>11</maven.compiler.target>
|
||||
<project.build.sourceEncoding>UTF-8</project.build.sourceEncoding>
|
||||
<smartjavaai.version>1.0.20</smartjavaai.version>
|
||||
<smartjavaai.version>1.0.22</smartjavaai.version>
|
||||
<!--如果打包运行,需要替换成你的main-->
|
||||
<exec.mainClass>smartai.examples.face.facedet.FaceDetDemo</exec.mainClass>
|
||||
|
||||
@@ -95,6 +95,13 @@
|
||||
<dependency>
|
||||
<groupId>cn.smartjavaai</groupId>
|
||||
<artifactId>smartjavaai-face</artifactId>
|
||||
<version>1.0.22</version>
|
||||
</dependency>
|
||||
|
||||
<dependency>
|
||||
<groupId>cn.smartjavaai</groupId>
|
||||
<artifactId>smartjavaai-all</artifactId>
|
||||
<version>1.0.22</version>
|
||||
</dependency>
|
||||
|
||||
|
||||
|
||||
@@ -0,0 +1,25 @@
|
||||
package smartai.examples.face;
|
||||
|
||||
import cn.smartjavaai.common.utils.VideoUtils;
|
||||
import org.bytedeco.ffmpeg.global.avcodec;
|
||||
import org.bytedeco.javacv.FFmpegFrameGrabber;
|
||||
import org.bytedeco.javacv.FFmpegFrameRecorder;
|
||||
|
||||
/**
|
||||
* 视频预处理
|
||||
* @author dwj
|
||||
* @date 2025/7/17
|
||||
*/
|
||||
public class VideoDemo {
|
||||
|
||||
public static void main(String[] args) {
|
||||
try {
|
||||
VideoUtils.rotateVideo("/Users/wenjie/Downloads/girl.mp4", "/Users/wenjie/Downloads/girl_rotate.mp4", 180,"mp4", avcodec.AV_CODEC_ID_H264);
|
||||
} catch (FFmpegFrameRecorder.Exception e) {
|
||||
throw new RuntimeException(e);
|
||||
} catch (FFmpegFrameGrabber.Exception e) {
|
||||
throw new RuntimeException(e);
|
||||
}
|
||||
}
|
||||
|
||||
}
|
||||
@@ -57,7 +57,8 @@ public class FaceAttributeDetDemo {
|
||||
*/
|
||||
@Test
|
||||
public void testFaceAttributeDetect(){
|
||||
try (FaceAttributeModel faceAttributeModel = getFaceAttributeModel()){
|
||||
try {
|
||||
FaceAttributeModel faceAttributeModel = getFaceAttributeModel();
|
||||
DetectionResponse detectionResponse = faceAttributeModel.detect("src/main/resources/iu_1.jpg");
|
||||
//绘制并导出人脸属性图片,小人脸仅有人脸框
|
||||
BufferedImage image = ImageIO.read(new File(Paths.get("src/main/resources/iu_1.jpg").toAbsolutePath().toString()));
|
||||
@@ -73,7 +74,8 @@ public class FaceAttributeDetDemo {
|
||||
*/
|
||||
@Test
|
||||
public void testFaceAttributeDetect2(){
|
||||
try (FaceAttributeModel faceAttributeModel = getFaceAttributeModel()){
|
||||
try {
|
||||
FaceAttributeModel faceAttributeModel = getFaceAttributeModel();
|
||||
FaceAttribute faceAttribute = faceAttributeModel.detectTopFace("src/main/resources/iu_1.jpg");
|
||||
log.info("人脸属性检测结果:{}", JSONObject.toJSONString(faceAttribute));
|
||||
} catch (Exception e) {
|
||||
@@ -86,7 +88,8 @@ public class FaceAttributeDetDemo {
|
||||
*/
|
||||
@Test
|
||||
public void testFaceAttributeDetect3(){
|
||||
try (FaceAttributeModel faceAttributeModel = getFaceAttributeModel()){
|
||||
try {
|
||||
FaceAttributeModel faceAttributeModel = getFaceAttributeModel();
|
||||
FaceAttribute faceAttribute = faceAttributeModel.detectTopFace("src/main/resources/iu_1.jpg");
|
||||
log.info("人脸属性检测结果:{}", JSONObject.toJSONString(faceAttribute));
|
||||
} catch (Exception e) {
|
||||
@@ -103,8 +106,9 @@ public class FaceAttributeDetDemo {
|
||||
*/
|
||||
@Test
|
||||
public void testFaceAttributeDetect4(){
|
||||
try (FaceDetModel faceDetModel = getFaceDetModel();
|
||||
FaceAttributeModel faceAttributeModel = getFaceAttributeModel()){
|
||||
try {
|
||||
FaceDetModel faceDetModel = getFaceDetModel();
|
||||
FaceAttributeModel faceAttributeModel = getFaceAttributeModel();
|
||||
//人脸检测
|
||||
BufferedImage image = ImageIO.read(new File(Paths.get("src/main/resources/iu_1.jpg").toAbsolutePath().toString()));
|
||||
R<DetectionResponse> detectionResponse = faceDetModel.detect(image);
|
||||
|
||||
@@ -88,7 +88,8 @@ public class ExpressionRecDemo {
|
||||
*/
|
||||
@Test
|
||||
public void testExpressionDetect() {
|
||||
try (ExpressionModel model = getExpressionModel()){
|
||||
try {
|
||||
ExpressionModel model = getExpressionModel();
|
||||
R<ExpressionResult> result = model.detectTopFace("src/main/resources/emotion/happy.png");
|
||||
if(result.isSuccess()){
|
||||
log.info("识别结果:{}", JSONObject.toJSONString(result.getData().getExpression().getDescription()));
|
||||
@@ -106,7 +107,8 @@ public class ExpressionRecDemo {
|
||||
*/
|
||||
@Test
|
||||
public void testExpressionDetect2() {
|
||||
try (ExpressionModel model = getExpressionModel()){
|
||||
try {
|
||||
ExpressionModel model = getExpressionModel();
|
||||
R<DetectionResponse> result = model.detect("src/main/resources/emotion/happy.png");
|
||||
if(result.isSuccess()){
|
||||
//log.info("识别结果:{}", JSONObject.toJSONString(result.getData()));
|
||||
@@ -128,8 +130,9 @@ public class ExpressionRecDemo {
|
||||
*/
|
||||
@Test
|
||||
public void testExpressionDetect3() {
|
||||
try (FaceDetModel faceDetModel = getFaceDetModel();
|
||||
ExpressionModel model = getExpressionModel()){
|
||||
try {
|
||||
FaceDetModel faceDetModel = getFaceDetModel();
|
||||
ExpressionModel model = getExpressionModel();
|
||||
BufferedImage image = ImageIO.read(new File(Paths.get("src/main/resources/emotion/happy.png").toAbsolutePath().toString()));
|
||||
R<DetectionResponse> detResult = faceDetModel.detect(image);
|
||||
if(detResult.isSuccess()){
|
||||
@@ -156,8 +159,9 @@ public class ExpressionRecDemo {
|
||||
*/
|
||||
@Test
|
||||
public void testExpressionDetect4() {
|
||||
try (FaceDetModel faceDetModel = getFaceDetModel();
|
||||
ExpressionModel model = getExpressionModel()){
|
||||
try {
|
||||
FaceDetModel faceDetModel = getFaceDetModel();
|
||||
ExpressionModel model = getExpressionModel();
|
||||
BufferedImage image = ImageIO.read(new File(Paths.get("src/main/resources/emotion/happy.png").toAbsolutePath().toString()));
|
||||
R<DetectionResponse> detResult = faceDetModel.detect(image);
|
||||
if(detResult.isSuccess()){
|
||||
@@ -182,7 +186,8 @@ public class ExpressionRecDemo {
|
||||
*/
|
||||
@Test
|
||||
public void testExpressionDetectAndDraw(){
|
||||
try (ExpressionModel model = getExpressionModel()){
|
||||
try {
|
||||
ExpressionModel model = getExpressionModel();
|
||||
BufferedImage image = ImageIO.read(new File(Paths.get("src/main/resources/emotion/surprise.png").toAbsolutePath().toString()));
|
||||
R<DetectionResponse> result = model.detect(image);
|
||||
if(result.isSuccess()){
|
||||
@@ -206,7 +211,8 @@ public class ExpressionRecDemo {
|
||||
*/
|
||||
@Test
|
||||
public void testExpressionDetectCamera(){
|
||||
try (ExpressionModel expressionModel = getExpressionModel()){
|
||||
try {
|
||||
ExpressionModel expressionModel = getExpressionModel();
|
||||
OpenCV.loadShared();
|
||||
VideoCapture capture = new VideoCapture(0);
|
||||
if (!capture.isOpened()) {
|
||||
|
||||
@@ -82,7 +82,8 @@ public class FaceDetDemo {
|
||||
*/
|
||||
@Test
|
||||
public void testFaceDetect(){
|
||||
try (FaceDetModel faceModel = FaceDetModelFactory.getInstance().getModel()) {
|
||||
try {
|
||||
FaceDetModel faceModel = FaceDetModelFactory.getInstance().getModel();
|
||||
R<DetectionResponse> detectedResult = faceModel.detect(imgPath);
|
||||
if(detectedResult.isSuccess()){
|
||||
log.info("人脸检测结果:{}", JSONObject.toJSONString(detectedResult.getData()));
|
||||
@@ -100,7 +101,8 @@ public class FaceDetDemo {
|
||||
*/
|
||||
@Test
|
||||
public void testFaceDetectCustomConfig(){
|
||||
try (FaceDetModel faceModel = getFaceDetModel()){
|
||||
try {
|
||||
FaceDetModel faceModel = getFaceDetModel();
|
||||
R<DetectionResponse> detectedResult = faceModel.detect(imgPath);
|
||||
if(detectedResult.isSuccess()){
|
||||
log.info("人脸检测结果:{}", JSONObject.toJSONString(detectedResult.getData()));
|
||||
@@ -118,7 +120,8 @@ public class FaceDetDemo {
|
||||
*/
|
||||
@Test
|
||||
public void testFaceDetectAndDraw(){
|
||||
try (FaceDetModel faceModel = getFaceDetModel()){
|
||||
try {
|
||||
FaceDetModel faceModel = getFaceDetModel();
|
||||
faceModel.detectAndDraw("src/main/resources/largest_selfie.jpg","output/largest_selfie_detected.png");
|
||||
} catch (Exception e) {
|
||||
throw new RuntimeException(e);
|
||||
@@ -131,7 +134,8 @@ public class FaceDetDemo {
|
||||
*/
|
||||
@Test
|
||||
public void testFaceDetectAndDraw2(){
|
||||
try (FaceDetModel faceModel = getFaceDetModel()){
|
||||
try {
|
||||
FaceDetModel faceModel = getFaceDetModel();
|
||||
BufferedImage image = null;
|
||||
String imagePath = "src/main/resources/largest_selfie.jpg";
|
||||
image = ImageIO.read(new File(Paths.get(imagePath).toAbsolutePath().toString()));
|
||||
@@ -154,11 +158,12 @@ public class FaceDetDemo {
|
||||
*/
|
||||
@Test
|
||||
public void testDetectFaceOffine(){
|
||||
FaceDetConfig config = new FaceDetConfig();
|
||||
config.setModelEnum(FaceDetModelEnum.RETINA_FACE);//人脸模型
|
||||
//模型路径,不同模型下载路径请参看文档
|
||||
config.setModelPath("/Users/xxx/Documents/develop/face_model/retinaface.pt");
|
||||
try (FaceDetModel faceModel = FaceDetModelFactory.getInstance().getModel(config)) {
|
||||
try {
|
||||
FaceDetConfig config = new FaceDetConfig();
|
||||
config.setModelEnum(FaceDetModelEnum.RETINA_FACE);//人脸模型
|
||||
//模型路径,不同模型下载路径请参看文档
|
||||
config.setModelPath("/Users/xxx/Documents/develop/face_model/retinaface.pt");
|
||||
FaceDetModel faceModel = FaceDetModelFactory.getInstance().getModel(config);
|
||||
R<DetectionResponse> detectedResult = faceModel.detect(imgPath);
|
||||
if(detectedResult.isSuccess()){
|
||||
log.info("人脸检测结果:{}", JSONObject.toJSONString(detectedResult.getData()));
|
||||
@@ -175,10 +180,11 @@ public class FaceDetDemo {
|
||||
*/
|
||||
@Test
|
||||
public void testDetectFaceGPU(){
|
||||
FaceDetConfig config = new FaceDetConfig();
|
||||
config.setModelEnum(FaceDetModelEnum.RETINA_FACE);//人脸模型
|
||||
config.setDevice(DeviceEnum.GPU);
|
||||
try (FaceDetModel faceModel = FaceDetModelFactory.getInstance().getModel(config)) {
|
||||
try {
|
||||
FaceDetConfig config = new FaceDetConfig();
|
||||
config.setModelEnum(FaceDetModelEnum.RETINA_FACE);//人脸模型
|
||||
config.setDevice(DeviceEnum.GPU);
|
||||
FaceDetModel faceModel = FaceDetModelFactory.getInstance().getModel(config);
|
||||
R<DetectionResponse> detectedResult = faceModel.detect(imgPath);
|
||||
if(detectedResult.isSuccess()){
|
||||
log.info("人脸检测结果:{}", JSONObject.toJSONString(detectedResult.getData()));
|
||||
@@ -196,7 +202,8 @@ public class FaceDetDemo {
|
||||
*/
|
||||
@Test
|
||||
public void testFaceDetectSeetaface6(){
|
||||
try (FaceDetModel faceModel = getSeetaface6DetModel()){
|
||||
try {
|
||||
FaceDetModel faceModel = getSeetaface6DetModel();
|
||||
R<DetectionResponse> detectedResult = faceModel.detect(imgPath);
|
||||
if(detectedResult.isSuccess()){
|
||||
log.info("人脸检测结果:{}", JSONObject.toJSONString(detectedResult.getData()));
|
||||
@@ -215,7 +222,8 @@ public class FaceDetDemo {
|
||||
*/
|
||||
@Test
|
||||
public void testDetectCamera(){
|
||||
try (FaceDetModel faceModel = getFaceDetModel()){
|
||||
try {
|
||||
FaceDetModel faceModel = getFaceDetModel();
|
||||
OpenCV.loadShared();
|
||||
VideoCapture capture = new VideoCapture(0);
|
||||
if (!capture.isOpened()) {
|
||||
|
||||
@@ -95,6 +95,8 @@ public class FaceRecDemo {
|
||||
MilvusConfig vectorDBConfig = new MilvusConfig();
|
||||
vectorDBConfig.setHost("127.0.0.1");
|
||||
vectorDBConfig.setPort(19530);
|
||||
//vectorDBConfig.setUsername("root");
|
||||
//vectorDBConfig.setPassword("Milvus");
|
||||
//vectorDBConfig.setCollectionName("face5");
|
||||
//ID策略:自动生成
|
||||
vectorDBConfig.setIdStrategy(IdStrategy.AUTO);
|
||||
@@ -135,7 +137,8 @@ public class FaceRecDemo {
|
||||
*/
|
||||
@Test
|
||||
public void testExtractFeatures(){
|
||||
try (FaceRecModel faceRecModel = getFaceRecModel()){
|
||||
try {
|
||||
FaceRecModel faceRecModel = getFaceRecModel();
|
||||
//提取图片中所有人脸特征
|
||||
R<DetectionResponse> faceResult = faceRecModel.extractFeatures("src/main/resources/iu_1.jpg");
|
||||
if(faceResult.isSuccess()){
|
||||
@@ -158,7 +161,8 @@ public class FaceRecDemo {
|
||||
*/
|
||||
@Test
|
||||
public void featureComparison(){
|
||||
try (FaceRecModel faceRecModel = getFaceRecModel()){
|
||||
try {
|
||||
FaceRecModel faceRecModel = getFaceRecModel();
|
||||
//基于图像直接比对人脸特征
|
||||
R<Float> similarResult = faceRecModel.featureComparison("src/main/resources/iu_1.jpg","src/main/resources/iu_2.jpg");
|
||||
if(similarResult.isSuccess()){
|
||||
@@ -183,7 +187,8 @@ public class FaceRecDemo {
|
||||
*/
|
||||
@Test
|
||||
public void featureComparison2(){
|
||||
try (FaceRecModel faceRecModel = getFaceRecModel()){
|
||||
try {
|
||||
FaceRecModel faceRecModel = getFaceRecModel();
|
||||
//特征提取(提取分数最高人脸特征),适用于单人脸场景
|
||||
R<float[]> featureResult1 = faceRecModel.extractTopFaceFeature("src/main/resources/iu_1.jpg");
|
||||
if(featureResult1.isSuccess()){
|
||||
@@ -220,7 +225,8 @@ public class FaceRecDemo {
|
||||
*/
|
||||
@Test
|
||||
public void searchFace(){
|
||||
try (FaceRecModel faceRecModel = getFaceRecModelWithDbConfig()){
|
||||
try {
|
||||
FaceRecModel faceRecModel = getFaceRecModelWithDbConfig();
|
||||
//等待加载人脸库结束
|
||||
while (!faceRecModel.isLoadFaceCompleted()){
|
||||
Thread.sleep(100);
|
||||
@@ -296,7 +302,8 @@ public class FaceRecDemo {
|
||||
*/
|
||||
@Test
|
||||
public void searchFace2(){
|
||||
try (FaceRecModel faceRecModel = getFaceRecModelWithSQLiteConfig()){
|
||||
try {
|
||||
FaceRecModel faceRecModel = getFaceRecModelWithSQLiteConfig();
|
||||
//等待加载人脸库结束
|
||||
while (!faceRecModel.isLoadFaceCompleted()){
|
||||
Thread.sleep(100);
|
||||
@@ -367,7 +374,8 @@ public class FaceRecDemo {
|
||||
@Test
|
||||
public void getFaceInfo(){
|
||||
//使用ID获取人脸信息
|
||||
try (FaceRecModel faceRecModel = getFaceRecModelWithSQLiteConfig()){
|
||||
try {
|
||||
FaceRecModel faceRecModel = getFaceRecModelWithSQLiteConfig();
|
||||
//等待加载人脸库结束
|
||||
while (!faceRecModel.isLoadFaceCompleted()){
|
||||
Thread.sleep(100);
|
||||
@@ -390,7 +398,8 @@ public class FaceRecDemo {
|
||||
@Test
|
||||
public void listFaces(){
|
||||
//使用ID获取人脸信息
|
||||
try (FaceRecModel faceRecModel = getFaceRecModelWithDbConfig()){
|
||||
try {
|
||||
FaceRecModel faceRecModel = getFaceRecModelWithDbConfig();
|
||||
//等待加载人脸库结束
|
||||
while (!faceRecModel.isLoadFaceCompleted()){
|
||||
Thread.sleep(100);
|
||||
|
||||
@@ -133,7 +133,8 @@ public class LivenessDetDemo {
|
||||
*/
|
||||
@Test
|
||||
public void testLivenessDetect(){
|
||||
try (LivenessDetModel livenessDetModel = getLivenessDetModel()){
|
||||
try {
|
||||
LivenessDetModel livenessDetModel = getLivenessDetModel();
|
||||
R<DetectionResponse> response = livenessDetModel.detect("src/main/resources/liveness/1.jpg");
|
||||
if(response.isSuccess()){
|
||||
for (DetectionInfo detectionInfo : response.getData().getDetectionInfoList()){
|
||||
@@ -152,7 +153,8 @@ public class LivenessDetDemo {
|
||||
*/
|
||||
@Test
|
||||
public void testLivenessDetectAndDraw(){
|
||||
try (LivenessDetModel livenessDetModel = getLivenessDetModel()){
|
||||
try {
|
||||
LivenessDetModel livenessDetModel = getLivenessDetModel();
|
||||
BufferedImage image = ImageIO.read(new File(Paths.get("src/main/resources/liveness/1.jpg").toAbsolutePath().toString()));
|
||||
R<DetectionResponse> response = livenessDetModel.detect(image);
|
||||
if(response.isSuccess()){
|
||||
@@ -175,7 +177,8 @@ public class LivenessDetDemo {
|
||||
*/
|
||||
@Test
|
||||
public void testLivenessDetect2(){
|
||||
try (LivenessDetModel livenessDetModel = getLivenessDetModel()){
|
||||
try {
|
||||
LivenessDetModel livenessDetModel = getLivenessDetModel();
|
||||
//指定文件夹路径
|
||||
File dir = new File("face-example/src/main/resources/liveness");
|
||||
File[] files = dir.listFiles();
|
||||
@@ -198,8 +201,9 @@ public class LivenessDetDemo {
|
||||
*/
|
||||
@Test
|
||||
public void testLivenessDetect3(){
|
||||
try (FaceDetModel faceDetectModel = getFaceDetModel();
|
||||
LivenessDetModel livenessDetModel = getLivenessDetModel()){
|
||||
try {
|
||||
FaceDetModel faceDetectModel = getFaceDetModel();
|
||||
LivenessDetModel livenessDetModel = getLivenessDetModel();
|
||||
// 将图片路径转换为 BufferedImage
|
||||
BufferedImage image = ImageIO.read(new File(Paths.get("src/main/resources/liveness/1.jpg").toAbsolutePath().toString()));
|
||||
//人脸检测
|
||||
@@ -230,8 +234,9 @@ public class LivenessDetDemo {
|
||||
*/
|
||||
@Test
|
||||
public void testLivenessDetect4(){
|
||||
try (FaceDetModel faceDetModel = getFaceDetModel();
|
||||
LivenessDetModel livenessDetModel = getMiniVisionLivenessDetModel()){
|
||||
try {
|
||||
FaceDetModel faceDetModel = getFaceDetModel();
|
||||
LivenessDetModel livenessDetModel = getMiniVisionLivenessDetModel();
|
||||
// 将图片路径转换为 BufferedImage
|
||||
BufferedImage image = ImageIO.read(new File(Paths.get("src/main/resources/liveness/1.jpg").toAbsolutePath().toString()));
|
||||
R<DetectionResponse> detResult = faceDetModel.detect(image);
|
||||
@@ -259,7 +264,8 @@ public class LivenessDetDemo {
|
||||
*/
|
||||
@Test
|
||||
public void testLivenessDetectVideo(){
|
||||
try (LivenessDetModel livenessDetModel = getLivenessDetModel()){
|
||||
try {
|
||||
LivenessDetModel livenessDetModel = getLivenessDetModel();
|
||||
//视频路径
|
||||
R<LivenessResult> livenessStatus = livenessDetModel.detectVideo("video.mp4");
|
||||
if (livenessStatus.isSuccess()){
|
||||
@@ -278,7 +284,8 @@ public class LivenessDetDemo {
|
||||
*/
|
||||
@Test
|
||||
public void testLivenessDetectCamera(){
|
||||
try (LivenessDetModel livenessDetModel = getLivenessDetModel()){
|
||||
try {
|
||||
LivenessDetModel livenessDetModel = getLivenessDetModel();
|
||||
OpenCV.loadShared();
|
||||
VideoCapture capture = new VideoCapture(0);
|
||||
if (!capture.isOpened()) {
|
||||
|
||||
@@ -78,8 +78,9 @@ public class FaceQualityDetDemo {
|
||||
*/
|
||||
@Test
|
||||
public void evaluateBrightness(){
|
||||
try (FaceQualityModel faceQualityModel = getFaceQualityModel();
|
||||
FaceDetModel faceDetModel = getFaceDetModel()){
|
||||
try {
|
||||
FaceQualityModel faceQualityModel = getFaceQualityModel();
|
||||
FaceDetModel faceDetModel = getFaceDetModel();
|
||||
//人脸检测
|
||||
BufferedImage image = ImageIO.read(new File(Paths.get("src/main/resources/iu_1.jpg").toAbsolutePath().toString()));
|
||||
R<DetectionResponse> detectionResponse = faceDetModel.detect(image);
|
||||
@@ -110,8 +111,9 @@ public class FaceQualityDetDemo {
|
||||
*/
|
||||
@Test
|
||||
public void evaluateCompleteness(){
|
||||
try (FaceQualityModel faceQualityModel = getFaceQualityModel();
|
||||
FaceDetModel faceDetModel = getFaceDetModel()){
|
||||
try {
|
||||
FaceQualityModel faceQualityModel = getFaceQualityModel();
|
||||
FaceDetModel faceDetModel = getFaceDetModel();
|
||||
//人脸检测
|
||||
BufferedImage image = ImageIO.read(new File(Paths.get("src/main/resources/iu_1.jpg").toAbsolutePath().toString()));
|
||||
R<DetectionResponse> detectionResponse = faceDetModel.detect(image);
|
||||
@@ -142,8 +144,9 @@ public class FaceQualityDetDemo {
|
||||
*/
|
||||
@Test
|
||||
public void evaluateClarity(){
|
||||
try (FaceQualityModel faceQualityModel = getFaceQualityModel();
|
||||
FaceDetModel faceDetModel = getFaceDetModel()){
|
||||
try {
|
||||
FaceQualityModel faceQualityModel = getFaceQualityModel();
|
||||
FaceDetModel faceDetModel = getFaceDetModel();
|
||||
//人脸检测
|
||||
BufferedImage image = ImageIO.read(new File(Paths.get("src/main/resources/iu_1.jpg").toAbsolutePath().toString()));
|
||||
R<DetectionResponse> detectionResponse = faceDetModel.detect(image);
|
||||
@@ -174,8 +177,9 @@ public class FaceQualityDetDemo {
|
||||
*/
|
||||
@Test
|
||||
public void evaluatePose(){
|
||||
try (FaceQualityModel faceQualityModel = getFaceQualityModel();
|
||||
FaceDetModel faceDetModel = getFaceDetModel()){
|
||||
try {
|
||||
FaceQualityModel faceQualityModel = getFaceQualityModel();
|
||||
FaceDetModel faceDetModel = getFaceDetModel();
|
||||
//人脸检测
|
||||
BufferedImage image = ImageIO.read(new File(Paths.get("src/main/resources/iu_1.jpg").toAbsolutePath().toString()));
|
||||
R<DetectionResponse> detectionResponse = faceDetModel.detect(image);
|
||||
@@ -207,8 +211,9 @@ public class FaceQualityDetDemo {
|
||||
*/
|
||||
@Test
|
||||
public void evaluateResolution(){
|
||||
try (FaceQualityModel faceQualityModel = getFaceQualityModel();
|
||||
FaceDetModel faceDetModel = getFaceDetModel()){
|
||||
try {
|
||||
FaceQualityModel faceQualityModel = getFaceQualityModel();
|
||||
FaceDetModel faceDetModel = getFaceDetModel();
|
||||
//人脸检测
|
||||
BufferedImage image = ImageIO.read(new File(Paths.get("src/main/resources/iu_1.jpg").toAbsolutePath().toString()));
|
||||
R<DetectionResponse> detectionResponse = faceDetModel.detect(image);
|
||||
@@ -240,8 +245,9 @@ public class FaceQualityDetDemo {
|
||||
*/
|
||||
@Test
|
||||
public void evaluateAll(){
|
||||
try (FaceQualityModel faceQualityModel = getFaceQualityModel();
|
||||
FaceDetModel faceDetModel = getFaceDetModel()){
|
||||
try {
|
||||
FaceQualityModel faceQualityModel = getFaceQualityModel();
|
||||
FaceDetModel faceDetModel = getFaceDetModel();
|
||||
//人脸检测
|
||||
BufferedImage image = ImageIO.read(new File(Paths.get("src/main/resources/iu_1.jpg").toAbsolutePath().toString()));
|
||||
R<DetectionResponse> detectionResponse = faceDetModel.detect(image);
|
||||
|
||||
@@ -12,7 +12,7 @@
|
||||
<maven.compiler.source>11</maven.compiler.source>
|
||||
<maven.compiler.target>11</maven.compiler.target>
|
||||
<project.build.sourceEncoding>UTF-8</project.build.sourceEncoding>
|
||||
<smartjavaai.version>1.0.20</smartjavaai.version>
|
||||
<smartjavaai.version>1.0.22</smartjavaai.version>
|
||||
<!--如果打包运行,需要替换成你的main-->
|
||||
<exec.mainClass>smartai.examples.objectdetection.ObjectDetection</exec.mainClass>
|
||||
|
||||
|
||||
@@ -41,10 +41,8 @@ import java.awt.image.BufferedImage;
|
||||
import java.io.File;
|
||||
import java.io.IOException;
|
||||
import java.nio.file.Paths;
|
||||
import java.util.ArrayList;
|
||||
import java.util.Iterator;
|
||||
import java.util.*;
|
||||
import java.util.List;
|
||||
import java.util.Objects;
|
||||
import java.util.concurrent.Callable;
|
||||
import java.util.concurrent.ExecutorService;
|
||||
import java.util.concurrent.Executors;
|
||||
@@ -71,7 +69,8 @@ public class ObjectDetection {
|
||||
@Test
|
||||
public void objectDetection(){
|
||||
//默认cpu
|
||||
try (DetectorModel detectorModel = ObjectDetectionModelFactory.getInstance().getModel()){
|
||||
try {
|
||||
DetectorModel detectorModel = ObjectDetectionModelFactory.getInstance().getModel();
|
||||
DetectionResponse detectionResponse = detectorModel.detect("src/main/resources/object_detection.jpg");
|
||||
log.info("目标检测结果:{}", JSONObject.toJSONString(detectionResponse));
|
||||
} catch (Exception e) {
|
||||
@@ -84,10 +83,15 @@ public class ObjectDetection {
|
||||
*/
|
||||
@Test
|
||||
public void objectDetection2(){
|
||||
DetectorModelConfig config = new DetectorModelConfig();
|
||||
config.setModelEnum(DetectorModelEnum.SSD_300_RESNET50);//检测模型,目前支持19种预置模型
|
||||
config.setDevice(device);
|
||||
try (DetectorModel detectorModel = ObjectDetectionModelFactory.getInstance().getModel(config)){
|
||||
try {
|
||||
DetectorModelConfig config = new DetectorModelConfig();
|
||||
config.setModelEnum(DetectorModelEnum.SSD_300_RESNET50);//检测模型,目前支持19种预置模型
|
||||
// 指定允许的类别
|
||||
// config.setAllowedClasses(Arrays.asList("person"));
|
||||
//指定返回检测数量
|
||||
config.setTopK(100);
|
||||
config.setDevice(device);
|
||||
DetectorModel detectorModel = ObjectDetectionModelFactory.getInstance().getModel(config);
|
||||
DetectionResponse detectionResponse = detectorModel.detect("src/main/resources/dog_bike_car.jpg");
|
||||
log.info("目标检测结果:{}", JSONObject.toJSONString(detectionResponse));
|
||||
} catch (Exception e) {
|
||||
@@ -100,7 +104,8 @@ public class ObjectDetection {
|
||||
*/
|
||||
@Test
|
||||
public void objectDetectionAndDraw(){
|
||||
try (DetectorModel detectorModel = ObjectDetectionModelFactory.getInstance().getModel()){
|
||||
try {
|
||||
DetectorModel detectorModel = ObjectDetectionModelFactory.getInstance().getModel();
|
||||
detectorModel.detectAndDraw("src/main/resources/object_detection.jpg","output/object_detection_detected.png");
|
||||
} catch (Exception e) {
|
||||
e.printStackTrace();
|
||||
@@ -112,7 +117,8 @@ public class ObjectDetection {
|
||||
*/
|
||||
@Test
|
||||
public void objectDetectionAndDraw2(){
|
||||
try (DetectorModel detectorModel = ObjectDetectionModelFactory.getInstance().getModel()){
|
||||
try {
|
||||
DetectorModel detectorModel = ObjectDetectionModelFactory.getInstance().getModel();
|
||||
String imagePath = "src/main/resources/object_detection.jpg";
|
||||
BufferedImage image = ImageIO.read(new File(Paths.get(imagePath).toAbsolutePath().toString()));
|
||||
//可以根据后续业务场景使用detectedImage
|
||||
@@ -131,15 +137,16 @@ public class ObjectDetection {
|
||||
*/
|
||||
@Test
|
||||
public void objectDetectionWithOfficialModel(){
|
||||
DetectorModelConfig config = new DetectorModelConfig();
|
||||
config.setThreshold(0.3f);
|
||||
//也支持YoloV8:YOLOV8_OFFICIAL 模型可以从文档中提供的地址下载
|
||||
config.setModelEnum(DetectorModelEnum.YOLOV12_OFFICIAL);//检测模型,目前支持19种模型
|
||||
// 指定模型路径,需要更改为自己的模型路径
|
||||
config.setModelPath("/Users/xxx/Documents/yolov12n.onnx");
|
||||
config.setDevice(device);
|
||||
//一定要将yolo官方的类别文件:synset.txt(文档中下载)放在模型同目录下,否则报错
|
||||
try (DetectorModel detectorModel = ObjectDetectionModelFactory.getInstance().getModel(config)){
|
||||
try {
|
||||
DetectorModelConfig config = new DetectorModelConfig();
|
||||
config.setThreshold(0.3f);
|
||||
//也支持YoloV8:YOLOV8_OFFICIAL 模型可以从文档中提供的地址下载
|
||||
config.setModelEnum(DetectorModelEnum.YOLOV12_OFFICIAL);//检测模型,目前支持19种模型
|
||||
// 指定模型路径,需要更改为自己的模型路径
|
||||
config.setModelPath("/Users/xxx/Documents/yolov12n.onnx");
|
||||
config.setDevice(device);
|
||||
//一定要将yolo官方的类别文件:synset.txt(文档中下载)放在模型同目录下,否则报错
|
||||
DetectorModel detectorModel = ObjectDetectionModelFactory.getInstance().getModel(config);
|
||||
DetectionResponse detect = detectorModel.detect("src/main/resources/dog_bike_car.jpg");
|
||||
log.info("目标检测结果:{}", JSONObject.toJSONString(detect));
|
||||
} catch (Exception e) {
|
||||
@@ -152,16 +159,21 @@ public class ObjectDetection {
|
||||
*/
|
||||
@Test
|
||||
public void objectDetectionWithCustomModel(){
|
||||
DetectorModelConfig config = new DetectorModelConfig();
|
||||
//也支持YoloV8:YOLOV8_CUSTOM 模型需要自己训练,训练教程可以查看文档
|
||||
config.setModelEnum(DetectorModelEnum.YOLOV12_CUSTOM);//自定义YOLOV12模型
|
||||
// 指定模型路径,需要更改为自己的模型路径
|
||||
config.setModelPath("/Users/xxx/Documents/develop/fire_model/best.onnx");
|
||||
config.putCustomParam("width", 640);//resize 宽
|
||||
config.putCustomParam("height", 640);// resize 高
|
||||
config.putCustomParam("nmsThreshold", 0.5f);
|
||||
config.setDevice(device);
|
||||
try (DetectorModel detectorModel = ObjectDetectionModelFactory.getInstance().getModel(config)){
|
||||
try {
|
||||
DetectorModelConfig config = new DetectorModelConfig();
|
||||
//也支持YoloV8:YOLOV8_CUSTOM 模型需要自己训练,训练教程可以查看文档
|
||||
config.setModelEnum(DetectorModelEnum.YOLOV12_CUSTOM);//自定义YOLOV12模型
|
||||
// 指定模型路径,需要更改为自己的模型路径
|
||||
config.setModelPath("/Users/xxx/Documents/develop/fire_model/best.onnx");
|
||||
config.putCustomParam("width", 640);//resize 宽
|
||||
config.putCustomParam("height", 640);// resize 高
|
||||
config.putCustomParam("nmsThreshold", 0.5f);
|
||||
// 指定允许的类别
|
||||
// config.setAllowedClasses(Arrays.asList("person"));
|
||||
//指定返回检测数量
|
||||
config.setTopK(100);
|
||||
config.setDevice(device);
|
||||
DetectorModel detectorModel = ObjectDetectionModelFactory.getInstance().getModel(config);
|
||||
DetectionResponse detect = detectorModel.detect("src/main/resources/dog_bike_car.jpg");
|
||||
log.info("目标检测结果:{}", JSONObject.toJSONString(detect));
|
||||
} catch (Exception e) {
|
||||
@@ -176,7 +188,8 @@ public class ObjectDetection {
|
||||
*/
|
||||
@Test
|
||||
public void testDetectCamera(){
|
||||
try (DetectorModel detectorModel = ObjectDetectionModelFactory.getInstance().getModel()){
|
||||
try {
|
||||
DetectorModel detectorModel = ObjectDetectionModelFactory.getInstance().getModel();
|
||||
OpenCV.loadShared();
|
||||
VideoCapture capture = new VideoCapture(0);
|
||||
if (!capture.isOpened()) {
|
||||
|
||||
@@ -12,7 +12,7 @@
|
||||
<maven.compiler.source>11</maven.compiler.source>
|
||||
<maven.compiler.target>11</maven.compiler.target>
|
||||
<project.build.sourceEncoding>UTF-8</project.build.sourceEncoding>
|
||||
<smartjavaai.version>1.0.20</smartjavaai.version>
|
||||
<smartjavaai.version>1.0.22</smartjavaai.version>
|
||||
<!--如果打包运行,需要替换成你的main-->
|
||||
<exec.mainClass>smartai.examples.ocr.common.OcrRecognizeDemo</exec.mainClass>
|
||||
|
||||
|
||||
@@ -11,7 +11,6 @@ import cn.smartjavaai.ocr.entity.OcrInfo;
|
||||
import cn.smartjavaai.ocr.enums.CommonDetModelEnum;
|
||||
import cn.smartjavaai.ocr.factory.OcrModelFactory;
|
||||
import cn.smartjavaai.ocr.model.common.detect.OcrCommonDetModel;
|
||||
import cn.smartjavaai.ocr.opencv.OcrOpenCVUtils;
|
||||
import cn.smartjavaai.ocr.utils.OcrUtils;
|
||||
import com.alibaba.fastjson.JSONObject;
|
||||
import lombok.extern.slf4j.Slf4j;
|
||||
@@ -60,7 +59,8 @@ public class OcrDetectionDemo {
|
||||
*/
|
||||
@Test
|
||||
public void detect(){
|
||||
try (OcrCommonDetModel model = getDetectionModel()){
|
||||
try {
|
||||
OcrCommonDetModel model = getDetectionModel();
|
||||
List<OcrBox> boxes = model.detect("src/main/resources/ocr_1.jpg");
|
||||
log.info("OCR检测结果:{}", JSONObject.toJSONString(boxes));
|
||||
} catch (Exception e) {
|
||||
@@ -77,7 +77,8 @@ public class OcrDetectionDemo {
|
||||
*/
|
||||
@Test
|
||||
public void detectAndDraw(){
|
||||
try (OcrCommonDetModel model = getDetectionModel()){
|
||||
try {
|
||||
OcrCommonDetModel model = getDetectionModel();
|
||||
model.detectAndDraw("src/main/resources/ocr_1.jpg", "output/ocr_1_detected.jpg");
|
||||
} catch (Exception e) {
|
||||
e.printStackTrace();
|
||||
@@ -94,7 +95,8 @@ public class OcrDetectionDemo {
|
||||
*/
|
||||
@Test
|
||||
public void batchDetect(){
|
||||
try (OcrCommonDetModel model = getDetectionModel()){
|
||||
try {
|
||||
OcrCommonDetModel model = getDetectionModel();
|
||||
//批量检测要求图片宽高一致
|
||||
String folderPath = "/Users/xxx/Downloads/testing33";
|
||||
//读取文件夹中所有图片
|
||||
|
||||
@@ -67,7 +67,8 @@ public class OcrDirectionDetDemo {
|
||||
*/
|
||||
@Test
|
||||
public void detect(){
|
||||
try (OcrDirectionModel directionModel = getDirectionModel()){
|
||||
try {
|
||||
OcrDirectionModel directionModel = getDirectionModel();
|
||||
List<OcrItem> itemList = directionModel.detect("src/main/resources/ocr_1.jpg");
|
||||
log.info("OCR方向检测结果1:{}", JSONObject.toJSONString(itemList));
|
||||
} catch (Exception e) {
|
||||
@@ -84,7 +85,8 @@ public class OcrDirectionDetDemo {
|
||||
*/
|
||||
@Test
|
||||
public void detectAndDraw(){
|
||||
try (OcrDirectionModel directionModel = getDirectionModel()){
|
||||
try {
|
||||
OcrDirectionModel directionModel = getDirectionModel();
|
||||
directionModel.detectAndDraw("src/main/resources/ocr_3.jpg", "output/ocr_3_detected.png");
|
||||
} catch (Exception e) {
|
||||
e.printStackTrace();
|
||||
|
||||
@@ -106,7 +106,8 @@ public class OcrRecognizeDemo {
|
||||
*/
|
||||
@Test
|
||||
public void recognize(){
|
||||
try (OcrCommonRecModel recModel = getRecModel()){
|
||||
try {
|
||||
OcrCommonRecModel recModel = getRecModel();
|
||||
//不带方向矫正,分行返回文本
|
||||
OcrRecOptions options = new OcrRecOptions(false, true);
|
||||
OcrInfo ocrInfo = recModel.recognize("src/main/resources/ocr_2.jpg",options);
|
||||
@@ -127,7 +128,8 @@ public class OcrRecognizeDemo {
|
||||
*/
|
||||
@Test
|
||||
public void recognizeHandWriting(){
|
||||
try (OcrCommonRecModel recModel = getRecModel()){
|
||||
try {
|
||||
OcrCommonRecModel recModel = getRecModel();
|
||||
OcrInfo ocrInfo = recModel.recognize("src/main/resources/handwriting_1.jpg",new OcrRecOptions());
|
||||
log.info("OCR识别结果:{}", JSONObject.toJSONString(ocrInfo));
|
||||
} catch (Exception e) {
|
||||
@@ -146,7 +148,8 @@ public class OcrRecognizeDemo {
|
||||
*/
|
||||
@Test
|
||||
public void recognize2(){
|
||||
try (OcrCommonRecModel recModel = getRecModelWithDirection()){
|
||||
try {
|
||||
OcrCommonRecModel recModel = getRecModelWithDirection();
|
||||
//带方向矫正,分行返回文本
|
||||
OcrRecOptions options = new OcrRecOptions(true, true);
|
||||
OcrInfo ocrInfo = recModel.recognize("src/main/resources/ocr_3.jpg",options);
|
||||
@@ -168,7 +171,8 @@ public class OcrRecognizeDemo {
|
||||
*/
|
||||
@Test
|
||||
public void recognizeAndDraw(){
|
||||
try (OcrCommonRecModel recModel = getRecModelWithDirection()){
|
||||
try {
|
||||
OcrCommonRecModel recModel = getRecModelWithDirection();
|
||||
int fontSize = 18;
|
||||
recModel.recognizeAndDraw("src/main/resources/general_ocr_002.png", "output/ocr_4_recognized.jpg", fontSize, new OcrRecOptions());
|
||||
} catch (Exception e) {
|
||||
@@ -184,7 +188,8 @@ public class OcrRecognizeDemo {
|
||||
*/
|
||||
@Test
|
||||
public void batchRecognize(){
|
||||
try (OcrCommonRecModel recModel = getRecModelWithDirection()){
|
||||
try {
|
||||
OcrCommonRecModel recModel = getRecModelWithDirection();
|
||||
//批量检测要求图片宽高一致
|
||||
String folderPath = "/Users/xxx/Downloads/testing33";
|
||||
//读取文件夹中所有图片
|
||||
|
||||
@@ -0,0 +1,81 @@
|
||||
package smartai.examples.ocr.plate;
|
||||
|
||||
import ai.djl.util.JsonUtils;
|
||||
import cn.smartjavaai.common.entity.R;
|
||||
import cn.smartjavaai.common.enums.DeviceEnum;
|
||||
import cn.smartjavaai.common.utils.ImageUtils;
|
||||
import cn.smartjavaai.ocr.config.PlateDetModelConfig;
|
||||
import cn.smartjavaai.ocr.config.PlateRecModelConfig;
|
||||
import cn.smartjavaai.ocr.entity.PlateInfo;
|
||||
import cn.smartjavaai.ocr.enums.PlateDetModelEnum;
|
||||
import cn.smartjavaai.ocr.enums.PlateRecModelEnum;
|
||||
import cn.smartjavaai.ocr.factory.PlateModelFactory;
|
||||
import cn.smartjavaai.ocr.model.plate.PlateDetModel;
|
||||
import cn.smartjavaai.ocr.model.plate.PlateRecModel;
|
||||
import lombok.extern.slf4j.Slf4j;
|
||||
import org.junit.Test;
|
||||
|
||||
import java.io.File;
|
||||
import java.util.List;
|
||||
|
||||
/**
|
||||
* @author dwj
|
||||
*/
|
||||
@Slf4j
|
||||
public class PlateRecDemo {
|
||||
|
||||
//设备类型
|
||||
public static DeviceEnum device = DeviceEnum.CPU;
|
||||
|
||||
/**
|
||||
* 获取车牌检测模型
|
||||
* @return
|
||||
*/
|
||||
public PlateDetModel getPlateDetModel() {
|
||||
PlateDetModelConfig config = new PlateDetModelConfig();
|
||||
config.setModelEnum(PlateDetModelEnum.YOLOV5);
|
||||
config.setModelPath("/Users/wenjie/Downloads/modelscope-2b3747db53adb48370c6edaccac56eb1ca0da1b5/onnx_export/model.onnx");
|
||||
// config.setModelPath("/Users/wenjie/Documents/develop/model/plate/yolov8s.onnx");
|
||||
config.setModelPath("/Users/wenjie/Downloads/数据集/车牌/onnx/plate_detect.onnx");
|
||||
config.setPredictorPoolSize(3);
|
||||
config.setDevice(device);
|
||||
return PlateModelFactory.getInstance().getDetModel(config);
|
||||
}
|
||||
|
||||
/**
|
||||
* 获取车牌识别模型
|
||||
* @return
|
||||
*/
|
||||
public PlateRecModel getPlateRecModel() {
|
||||
PlateRecModelConfig recModelConfig = new PlateRecModelConfig();
|
||||
recModelConfig.setModelEnum(PlateRecModelEnum.PLATE_REC_CRNN);
|
||||
recModelConfig.setModelPath("/Users/wenjie/Downloads/数据集/车牌/onnx/plate_rec_color.onnx");
|
||||
recModelConfig.setPlateDetModel(getPlateDetModel());
|
||||
return PlateModelFactory.getInstance().getRecModel(recModelConfig);
|
||||
}
|
||||
|
||||
@Test
|
||||
public void testDetect() {
|
||||
PlateRecModel plateRecModel = getPlateRecModel();
|
||||
R<List<PlateInfo>> result = plateRecModel.recognize("src/main/resources/plate/Quicker_20220930_180856.png");
|
||||
if(result.isSuccess()){
|
||||
log.info("车牌识别结果:{}", JsonUtils.toJson(result.getData()));
|
||||
}else{
|
||||
log.error("车牌识别失败:{}", result.getMessage());
|
||||
}
|
||||
}
|
||||
|
||||
@Test
|
||||
public void recognizeAndDraw() {
|
||||
PlateRecModel plateRecModel = getPlateRecModel();
|
||||
R<Void> result = plateRecModel.recognizeAndDraw("src/main/resources/plate/single_green.jpg", "output/plate_recognized2.jpg");
|
||||
if(result.isSuccess()){
|
||||
log.info("车牌识别成功");
|
||||
}else{
|
||||
log.error("车牌识别失败:{}", result.getMessage());
|
||||
}
|
||||
|
||||
}
|
||||
|
||||
|
||||
}
|
||||
@@ -113,10 +113,11 @@ public class TableRecDemo {
|
||||
*/
|
||||
@Test
|
||||
public void recognize(){
|
||||
try (TableStructureModel tableStructureModel = getTableStructureModel();
|
||||
OcrCommonDetModel detModel = getDetectionModel();
|
||||
OcrCommonRecModel recModel = getRecModel();
|
||||
OcrDirectionModel directionModel = getDirectionModel()){
|
||||
try {
|
||||
TableStructureModel tableStructureModel = getTableStructureModel();
|
||||
OcrCommonDetModel detModel = getDetectionModel();
|
||||
OcrCommonRecModel recModel = getRecModel();
|
||||
OcrDirectionModel directionModel = getDirectionModel();
|
||||
//创建表格识别器
|
||||
TableRecognizer tableRecognizer = TableRecognizer.builder()
|
||||
.withStructureModel(tableStructureModel)
|
||||
|
||||
|
After Width: | Height: | Size: 1.4 MiB |
|
After Width: | Height: | Size: 1.0 MiB |
|
After Width: | Height: | Size: 241 KiB |
|
After Width: | Height: | Size: 328 KiB |
BIN
examples/ocr-examples/src/main/resources/plate/double_yellow.jpg
Normal file
|
After Width: | Height: | Size: 29 KiB |
BIN
examples/ocr-examples/src/main/resources/plate/hongkang1.jpg
Normal file
|
After Width: | Height: | Size: 571 KiB |
BIN
examples/ocr-examples/src/main/resources/plate/moto.png
Normal file
|
After Width: | Height: | Size: 400 KiB |
BIN
examples/ocr-examples/src/main/resources/plate/police.jpg
Normal file
|
After Width: | Height: | Size: 382 KiB |
BIN
examples/ocr-examples/src/main/resources/plate/shi_lin_guan.jpg
Normal file
|
After Width: | Height: | Size: 47 KiB |
BIN
examples/ocr-examples/src/main/resources/plate/single_blue.jpg
Normal file
|
After Width: | Height: | Size: 1.8 MiB |
BIN
examples/ocr-examples/src/main/resources/plate/single_green.jpg
Normal file
|
After Width: | Height: | Size: 903 KiB |
BIN
examples/ocr-examples/src/main/resources/plate/single_yellow.jpg
Normal file
|
After Width: | Height: | Size: 85 KiB |
BIN
examples/ocr-examples/src/main/resources/plate/tmp8F1F.png
Normal file
|
After Width: | Height: | Size: 932 KiB |
BIN
examples/ocr-examples/src/main/resources/plate/tmpA5E3.png
Normal file
|
After Width: | Height: | Size: 513 KiB |
BIN
examples/ocr-examples/src/main/resources/plate/xue.jpg
Normal file
|
After Width: | Height: | Size: 999 KiB |
BIN
examples/ocr-examples/src/main/resources/table/medal_table.png
Normal file
|
After Width: | Height: | Size: 943 KiB |
BIN
examples/ocr-examples/src/main/resources/table/table_ch1.png
Normal file
|
After Width: | Height: | Size: 219 KiB |
@@ -12,7 +12,7 @@
|
||||
<maven.compiler.source>11</maven.compiler.source>
|
||||
<maven.compiler.target>11</maven.compiler.target>
|
||||
<project.build.sourceEncoding>UTF-8</project.build.sourceEncoding>
|
||||
<smartjavaai.version>1.0.20</smartjavaai.version>
|
||||
<smartjavaai.version>1.0.22</smartjavaai.version>
|
||||
<!--如果打包运行,需要替换成你的main-->
|
||||
<exec.mainClass>smartai.examples.nlp.translation.TranslationDemo</exec.mainClass>
|
||||
|
||||
|
||||
2
pom.xml
@@ -7,7 +7,7 @@
|
||||
<name>SmartJavaAI</name>
|
||||
<groupId>cn.smartjavaai</groupId>
|
||||
<artifactId>smartjavaai-parent</artifactId>
|
||||
<version>1.0.20</version>
|
||||
<version>1.0.22</version>
|
||||
<packaging>pom</packaging>
|
||||
<description>SmartJavaAI</description>
|
||||
<modules>
|
||||
|
||||
@@ -6,11 +6,11 @@
|
||||
<parent>
|
||||
<groupId>cn.smartjavaai</groupId>
|
||||
<artifactId>smartjavaai-parent</artifactId>
|
||||
<version>1.0.20</version>
|
||||
<version>1.0.22</version>
|
||||
</parent>
|
||||
|
||||
<artifactId>smartjavaai-all</artifactId>
|
||||
<version>1.0.20</version>
|
||||
<version>1.0.22</version>
|
||||
<name>${project.artifactId}</name>
|
||||
<description>SmartJavaAI</description>
|
||||
<url>https://github.com/geekwenjie/SmartJavaAI</url>
|
||||
|
||||
@@ -6,10 +6,10 @@
|
||||
<parent>
|
||||
<groupId>cn.smartjavaai</groupId>
|
||||
<artifactId>smartjavaai-parent</artifactId>
|
||||
<version>1.0.20</version>
|
||||
<version>1.0.22</version>
|
||||
</parent>
|
||||
|
||||
<version>1.0.20</version>
|
||||
<version>1.0.22</version>
|
||||
<artifactId>smartjavaai-bom</artifactId>
|
||||
<name>smartjavaai-bom</name>
|
||||
<description>统一版本管理的 BOM 包,同时支持 import 和全量依赖</description>
|
||||
|
||||
@@ -6,7 +6,7 @@
|
||||
<parent>
|
||||
<groupId>cn.smartjavaai</groupId>
|
||||
<artifactId>smartjavaai-parent</artifactId>
|
||||
<version>1.0.20</version>
|
||||
<version>1.0.22</version>
|
||||
</parent>
|
||||
|
||||
<name>smartjavaai-common</name>
|
||||
|
||||
@@ -29,6 +29,11 @@ public class ModelConfig {
|
||||
*/
|
||||
private String batchifier;
|
||||
|
||||
/**
|
||||
* 模型预测器池大小(默认为cpu核心数)
|
||||
*/
|
||||
private int predictorPoolSize;
|
||||
|
||||
/**
|
||||
* 个性化配置(按模型类型动态解析)
|
||||
*/
|
||||
|
||||
@@ -57,6 +57,7 @@ public class R<T> {
|
||||
NO_FACE_DETECTED(3, "未检测到人脸"),
|
||||
PARAM_ERROR(4, "参数错误"),
|
||||
INVALID_VIDEO(5, "视频无效"),
|
||||
NO_OBJECT_DETECTED(6, "未检测到目标"),
|
||||
Unknown(-1, "未知错误");
|
||||
|
||||
private final int code;
|
||||
|
||||
@@ -3,6 +3,7 @@ package cn.smartjavaai.common.pool;
|
||||
import ai.djl.inference.Predictor;
|
||||
import ai.djl.repository.zoo.ZooModel;
|
||||
import ai.djl.translate.Translator;
|
||||
import lombok.extern.slf4j.Slf4j;
|
||||
import org.apache.commons.pool2.BasePooledObjectFactory;
|
||||
import org.apache.commons.pool2.PooledObject;
|
||||
import org.apache.commons.pool2.impl.DefaultPooledObject;
|
||||
@@ -12,6 +13,7 @@ import org.apache.commons.pool2.impl.DefaultPooledObject;
|
||||
* @author dwj
|
||||
* @date 2025/4/8
|
||||
*/
|
||||
@Slf4j
|
||||
public class PredictorFactory<I, O> extends BasePooledObjectFactory<Predictor<I, O>> {
|
||||
private final ZooModel<I, O> model;
|
||||
|
||||
@@ -21,6 +23,7 @@ public class PredictorFactory<I, O> extends BasePooledObjectFactory<Predictor<I,
|
||||
|
||||
@Override
|
||||
public Predictor<I, O> create() {
|
||||
log.debug("create predictor");
|
||||
return model.newPredictor();
|
||||
}
|
||||
|
||||
@@ -31,6 +34,7 @@ public class PredictorFactory<I, O> extends BasePooledObjectFactory<Predictor<I,
|
||||
|
||||
@Override
|
||||
public void destroyObject(PooledObject<Predictor<I, O>> p) {
|
||||
log.debug("close predictor");
|
||||
p.getObject().close();
|
||||
}
|
||||
}
|
||||
|
||||
@@ -8,6 +8,9 @@ import ai.djl.ndarray.NDArray;
|
||||
import cn.smartjavaai.common.entity.DetectionRectangle;
|
||||
import cn.smartjavaai.common.entity.DetectionResponse;
|
||||
import org.opencv.core.Mat;
|
||||
import org.opencv.core.Scalar;
|
||||
import org.opencv.core.Size;
|
||||
import org.opencv.imgproc.Imgproc;
|
||||
|
||||
import javax.imageio.ImageIO;
|
||||
import java.awt.*;
|
||||
@@ -364,6 +367,7 @@ public class ImageUtils {
|
||||
int width = metrics.stringWidth(text) + padding * 2 - stroke / 2;
|
||||
int height = metrics.getHeight() + metrics.getDescent();
|
||||
int ascent = metrics.getAscent();
|
||||
y = Math.max(0, y - height);
|
||||
java.awt.Rectangle background = new java.awt.Rectangle(x, y, width, height);
|
||||
g.fill(background);
|
||||
g.setPaint(Color.WHITE);
|
||||
@@ -468,6 +472,33 @@ public class ImageUtils {
|
||||
return true;
|
||||
}
|
||||
|
||||
/**
|
||||
* 在图像上绘制带白色背景、黑色文字的文本
|
||||
*/
|
||||
public static void putTextWithBackground(Mat image, String text, org.opencv.core.Point origin, Scalar textColor, Scalar backgroundColor, int padding) {
|
||||
// 默认字体
|
||||
int font = Imgproc.FONT_HERSHEY_SCRIPT_SIMPLEX;
|
||||
// 默认字体缩放大小
|
||||
double fontScale = 1.0;
|
||||
//线条粗细
|
||||
int thickness = 2;
|
||||
//获取文字大小
|
||||
int[] baseLine = new int[1];
|
||||
Size textSize = Imgproc.getTextSize(text, font, fontScale, thickness, baseLine);
|
||||
int textWidth = (int) textSize.width;
|
||||
int textHeight = (int) textSize.height;
|
||||
|
||||
//计算带padding的背景框
|
||||
org.opencv.core.Point bgTopLeft = new org.opencv.core.Point(origin.x - padding, origin.y - textHeight - padding);
|
||||
org.opencv.core.Point bgBottomRight = new org.opencv.core.Point(origin.x + textWidth + padding, origin.y + baseLine[0] + padding);
|
||||
|
||||
//绘制背景矩形
|
||||
Imgproc.rectangle(image, bgTopLeft, bgBottomRight, backgroundColor, Imgproc.FILLED);
|
||||
|
||||
//绘制文字(黑色)
|
||||
Imgproc.putText(image, text, origin, font, fontScale, textColor, thickness);
|
||||
}
|
||||
|
||||
|
||||
|
||||
}
|
||||
|
||||
@@ -0,0 +1,124 @@
|
||||
package cn.smartjavaai.common.utils;
|
||||
|
||||
import ai.djl.modality.cv.util.NDImageUtils;
|
||||
import ai.djl.ndarray.NDArray;
|
||||
import ai.djl.ndarray.NDManager;
|
||||
import ai.djl.ndarray.index.NDIndex;
|
||||
import ai.djl.ndarray.types.DataType;
|
||||
import ai.djl.ndarray.types.Shape;
|
||||
|
||||
import java.util.Arrays;
|
||||
|
||||
/**
|
||||
* 按比例缩放,剩余空间用指定颜色填充
|
||||
* @author dwj
|
||||
*/
|
||||
public class LetterBoxUtils {
|
||||
|
||||
public enum PaddingPosition {
|
||||
CENTER, LEFT_TOP, RIGHT_BOTTOM
|
||||
}
|
||||
|
||||
public static class ResizeResult {
|
||||
public NDArray image;
|
||||
public float r;
|
||||
public int left;
|
||||
public int top;
|
||||
}
|
||||
|
||||
public static ResizeResult letterboxWithMeta(NDArray paddingImg, float r, int left, int top) {
|
||||
// ... letterbox 逻辑不变
|
||||
ResizeResult result = new ResizeResult();
|
||||
result.image = paddingImg;
|
||||
result.r = r;
|
||||
result.left = left;
|
||||
result.top = top;
|
||||
return result;
|
||||
}
|
||||
|
||||
|
||||
/**
|
||||
* 按比例缩放 + padding
|
||||
*
|
||||
* @param img 原图 NDArray HWC
|
||||
* @param targetW 目标宽度
|
||||
* @param targetH 目标高度
|
||||
* @param padColor padding 填充颜色,RGB 归一化 0-1
|
||||
* @param position padding 位置:CENTER / LEFT_TOP / RIGHT_BOTTOM
|
||||
* @return 处理后的 NDArray
|
||||
*/
|
||||
public static ResizeResult letterbox(NDManager manager, NDArray img, int targetW, int targetH, float padColor, PaddingPosition position) {
|
||||
long origH = img.getShape().get(0);
|
||||
long origW = img.getShape().get(1);
|
||||
|
||||
float r = Math.min(targetW / (float) origW, targetH / (float) origH);
|
||||
int newW = Math.round(origW * r);
|
||||
int newH = Math.round(origH * r);
|
||||
|
||||
img = NDImageUtils.resize(img, newW, newH); // HWC 0~1
|
||||
|
||||
// NDArray paddingImg = manager
|
||||
// .full(new Shape(targetW, targetH, 3), padColor, DataType.UINT8);
|
||||
|
||||
NDArray paddingImg = manager.zeros(new Shape(targetW, targetH, 3), DataType.FLOAT32);
|
||||
paddingImg = paddingImg.add(114);
|
||||
|
||||
int padW = targetW - newW;
|
||||
int padH = targetH - newH;
|
||||
int top = 0, left = 0;
|
||||
|
||||
switch (position) {
|
||||
case CENTER:
|
||||
left = padW / 2;
|
||||
top = padH / 2;
|
||||
break;
|
||||
case LEFT_TOP:
|
||||
left = 0;
|
||||
top = 0;
|
||||
break;
|
||||
case RIGHT_BOTTOM:
|
||||
left = padW;
|
||||
top = padH;
|
||||
break;
|
||||
}
|
||||
paddingImg.set(new NDIndex(String.format("%d:%d,%d:%d", top, top + newH, left, left + newW)), img);
|
||||
return letterboxWithMeta(paddingImg, r, left, top);
|
||||
}
|
||||
|
||||
|
||||
/**
|
||||
* 恢复缩放后的 box
|
||||
* @param boxes
|
||||
* @param scaleRatio
|
||||
* @param left
|
||||
* @param top
|
||||
* @param keypointStart
|
||||
* @param keypointDim
|
||||
* @return
|
||||
*/
|
||||
public static NDArray restoreBox(NDArray boxes, float scaleRatio, float left, float top, int keypointStart, int keypointDim) {
|
||||
// 处理 bbox
|
||||
NDArray x1 = boxes.get(":, 0").sub(left).div(scaleRatio);
|
||||
NDArray y1 = boxes.get(":, 1").sub(top).div(scaleRatio);
|
||||
NDArray x2 = boxes.get(":, 2").sub(left).div(scaleRatio);
|
||||
NDArray y2 = boxes.get(":, 3").sub(top).div(scaleRatio);
|
||||
|
||||
boxes.set(new NDIndex(":, 0"), x1);
|
||||
boxes.set(new NDIndex(":, 1"), y1);
|
||||
boxes.set(new NDIndex(":, 2"), x2);
|
||||
boxes.set(new NDIndex(":, 3"), y2);
|
||||
|
||||
if (keypointDim > 0) {
|
||||
for (int i = 0; i < keypointDim; i += 2) {
|
||||
int xIdx = keypointStart + i;
|
||||
int yIdx = keypointStart + i + 1;
|
||||
NDArray keyX = boxes.get(":, " + xIdx).sub(left).div(scaleRatio);
|
||||
NDArray keyY = boxes.get(":, " + yIdx).sub(top).div(scaleRatio);
|
||||
boxes.set(new NDIndex(":, " + xIdx), keyX);
|
||||
boxes.set(new NDIndex(":, " + yIdx), keyY);
|
||||
}
|
||||
}
|
||||
return boxes;
|
||||
}
|
||||
|
||||
}
|
||||
@@ -0,0 +1,67 @@
|
||||
package cn.smartjavaai.common.utils;
|
||||
|
||||
import ai.djl.ndarray.NDArray;
|
||||
|
||||
import java.util.ArrayList;
|
||||
import java.util.List;
|
||||
|
||||
/**
|
||||
* @author dwj
|
||||
* @date 2025/7/23
|
||||
*/
|
||||
public class NMSUtils {
|
||||
|
||||
/**
|
||||
* 通用 NMS 方法,输入 NDArray 形式的 boxes 和 scores,返回保留的索引列表
|
||||
*
|
||||
* @param boxes NDArray 形状为 (N, 4),格式为 [x1, y1, x2, y2]
|
||||
* @param scores NDArray 形状为 (N,) 或 (N,1),每个 box 的置信度
|
||||
* @param iouThreshold IOU 阈值,超过该阈值则认为有重叠
|
||||
* @return 保留框的索引列表
|
||||
*/
|
||||
public static int[] nms(NDArray boxes, NDArray scores, float iouThreshold) {
|
||||
if (boxes.isEmpty()) {
|
||||
return new int[0];
|
||||
}
|
||||
|
||||
NDArray x1 = boxes.get(":, 0");
|
||||
NDArray y1 = boxes.get(":, 1");
|
||||
NDArray x2 = boxes.get(":, 2");
|
||||
NDArray y2 = boxes.get(":, 3");
|
||||
|
||||
NDArray areas = x2.sub(x1).add(1).mul(y2.sub(y1).add(1));
|
||||
|
||||
// 按照置信度降序排序
|
||||
NDArray order = scores.argSort().flip(0);
|
||||
|
||||
List<Integer> keep = new ArrayList<>();
|
||||
|
||||
while (order.size() > 0) {
|
||||
int idx = (int)order.getLong(0);
|
||||
keep.add(idx);
|
||||
|
||||
if (order.size() == 1) break;
|
||||
|
||||
NDArray currentBox = boxes.get(idx);
|
||||
NDArray others = boxes.get(order);
|
||||
|
||||
NDArray xx1 = x1.get(order).maximum(x1.get(idx));
|
||||
NDArray yy1 = y1.get(order).maximum(y1.get(idx));
|
||||
NDArray xx2 = x2.get(order).minimum(x2.get(idx));
|
||||
NDArray yy2 = y2.get(order).minimum(y2.get(idx));
|
||||
|
||||
NDArray w = xx2.sub(xx1).add(1).maximum(0);
|
||||
NDArray h = yy2.sub(yy1).add(1).maximum(0);
|
||||
NDArray inter = w.mul(h);
|
||||
|
||||
NDArray remAreas = areas.get(order);
|
||||
NDArray union = remAreas.add(areas.get(idx)).sub(inter);
|
||||
NDArray iou = inter.div(union);
|
||||
|
||||
NDArray mask = iou.lte(iouThreshold);
|
||||
order = order.get(mask);
|
||||
}
|
||||
return keep.stream().mapToInt(i -> i).toArray();
|
||||
}
|
||||
|
||||
}
|
||||
@@ -124,4 +124,20 @@ public class OpenCVUtils {
|
||||
mat.put(0, 0, data);
|
||||
return mat;
|
||||
}
|
||||
|
||||
/**
|
||||
* 透视变换
|
||||
*
|
||||
* @param src
|
||||
* @param srcPoints
|
||||
* @param dstPoints
|
||||
* @return
|
||||
*/
|
||||
public static Mat perspectiveTransform(Mat src, Mat srcPoints, Mat dstPoints) {
|
||||
Mat dst = src.clone();
|
||||
Mat warp_mat = Imgproc.getPerspectiveTransform(srcPoints, dstPoints);
|
||||
Imgproc.warpPerspective(src, dst, warp_mat, dst.size());
|
||||
warp_mat.release();
|
||||
return dst;
|
||||
}
|
||||
}
|
||||
|
||||
@@ -0,0 +1,70 @@
|
||||
package cn.smartjavaai.common.utils;
|
||||
|
||||
import ai.djl.modality.cv.output.Point;
|
||||
|
||||
import java.util.Arrays;
|
||||
import java.util.Comparator;
|
||||
import java.util.List;
|
||||
|
||||
/**
|
||||
* @author dwj
|
||||
*/
|
||||
public class PointUtils {
|
||||
|
||||
/**
|
||||
* 对 4 个关键点进行排序,顺序为:
|
||||
* 左上、右上、右下、左下
|
||||
*/
|
||||
public static List<Point> orderPoints(List<Point> points) {
|
||||
if (points == null || points.size() != 4) {
|
||||
throw new IllegalArgumentException("必须提供 4 个点");
|
||||
}
|
||||
|
||||
// 按 X 坐标升序排列
|
||||
points.sort(Comparator.comparingDouble(Point::getX));
|
||||
|
||||
List<Point> left = points.subList(0, 2);
|
||||
List<Point> right = points.subList(2, 4);
|
||||
|
||||
// 左侧两点按 Y 排序:上为 tl,下为 bl
|
||||
Point tl = left.get(0).getY() < left.get(1).getY() ? left.get(0) : left.get(1);
|
||||
Point bl = left.get(0).getY() >= left.get(1).getY() ? left.get(0) : left.get(1);
|
||||
|
||||
// 右侧两点按 Y 排序:上为 tr,下为 br
|
||||
Point tr = right.get(0).getY() < right.get(1).getY() ? right.get(0) : right.get(1);
|
||||
Point br = right.get(0).getY() >= right.get(1).getY() ? right.get(0) : right.get(1);
|
||||
|
||||
return Arrays.asList(tl, tr, br, bl);
|
||||
}
|
||||
|
||||
/**
|
||||
* 欧式距离计算
|
||||
*
|
||||
* @param point1
|
||||
* @param point2
|
||||
* @return
|
||||
*/
|
||||
public static float distance(float[] point1, float[] point2) {
|
||||
float disX = point1[0] - point2[0];
|
||||
float disY = point1[1] - point2[1];
|
||||
float dis = (float) Math.sqrt(disX * disX + disY * disY);
|
||||
return dis;
|
||||
}
|
||||
|
||||
/**
|
||||
* 欧式距离计算
|
||||
*
|
||||
* @param point1
|
||||
* @param point2
|
||||
* @return
|
||||
*/
|
||||
public static float distance(Point point1, Point point2) {
|
||||
double disX = point1.getX() - point2.getX();
|
||||
double disY = point1.getY() - point2.getY();
|
||||
float dis = (float) Math.sqrt(disX * disX + disY * disY);
|
||||
return dis;
|
||||
}
|
||||
|
||||
|
||||
|
||||
}
|
||||
@@ -6,11 +6,11 @@
|
||||
<parent>
|
||||
<groupId>cn.smartjavaai</groupId>
|
||||
<artifactId>smartjavaai-parent</artifactId>
|
||||
<version>1.0.20</version>
|
||||
<version>1.0.22</version>
|
||||
</parent>
|
||||
|
||||
<artifactId>smartjavaai-face</artifactId>
|
||||
<version>1.0.20</version>
|
||||
<version>1.0.22</version>
|
||||
<name>smartjavaai-face</name>
|
||||
<description>SmartJavaAI</description>
|
||||
<url>https://github.com/geekwenjie/SmartJavaAI</url>
|
||||
|
||||
@@ -1,5 +1,6 @@
|
||||
package cn.smartjavaai.face.config;
|
||||
|
||||
import cn.smartjavaai.common.config.ModelConfig;
|
||||
import cn.smartjavaai.common.enums.DeviceEnum;
|
||||
import cn.smartjavaai.face.enums.FaceAttributeModelEnum;
|
||||
import lombok.Data;
|
||||
@@ -9,7 +10,7 @@ import lombok.Data;
|
||||
* @author dwj
|
||||
*/
|
||||
@Data
|
||||
public class FaceAttributeConfig {
|
||||
public class FaceAttributeConfig extends ModelConfig {
|
||||
|
||||
/**
|
||||
* 人脸属性识别模型枚举
|
||||
@@ -21,16 +22,6 @@ public class FaceAttributeConfig {
|
||||
*/
|
||||
private String modelPath;
|
||||
|
||||
/**
|
||||
* 设备类型
|
||||
*/
|
||||
private DeviceEnum device;
|
||||
|
||||
/**
|
||||
* gpu设备ID 当device为GPU时生效
|
||||
*/
|
||||
private int gpuId = 0;
|
||||
|
||||
/**
|
||||
* 是否启用年龄检测
|
||||
*/
|
||||
|
||||
@@ -1,5 +1,6 @@
|
||||
package cn.smartjavaai.face.config;
|
||||
|
||||
import cn.smartjavaai.common.config.ModelConfig;
|
||||
import cn.smartjavaai.common.enums.DeviceEnum;
|
||||
import cn.smartjavaai.face.constant.FaceDetectConstant;
|
||||
import cn.smartjavaai.face.enums.FaceDetModelEnum;
|
||||
@@ -13,7 +14,7 @@ import java.util.Map;
|
||||
* @author dwj
|
||||
*/
|
||||
@Data
|
||||
public class FaceDetConfig {
|
||||
public class FaceDetConfig extends ModelConfig {
|
||||
|
||||
/**
|
||||
* 人脸检测模型枚举
|
||||
@@ -36,16 +37,6 @@ public class FaceDetConfig {
|
||||
*/
|
||||
private String modelPath;
|
||||
|
||||
/**
|
||||
* 设备类型
|
||||
*/
|
||||
private DeviceEnum device;
|
||||
|
||||
/**
|
||||
* 个性化配置(按模型类型动态解析)
|
||||
*/
|
||||
private Map<String, Object> customParams = new HashMap<>();
|
||||
|
||||
|
||||
public FaceDetConfig() {
|
||||
}
|
||||
@@ -59,19 +50,4 @@ public class FaceDetConfig {
|
||||
this.modelPath = modelPath;
|
||||
}
|
||||
|
||||
public <T> T getCustomParam(String key, Class<T> clazz) {
|
||||
Object value = customParams.get(key);
|
||||
if (value == null) return null;
|
||||
return clazz.cast(value);
|
||||
}
|
||||
|
||||
/**
|
||||
* 添加个性化配置项
|
||||
*/
|
||||
public void putCustomParam(String key, Object value) {
|
||||
if (customParams == null) {
|
||||
customParams = new HashMap<>();
|
||||
}
|
||||
customParams.put(key, value);
|
||||
}
|
||||
}
|
||||
|
||||
@@ -1,5 +1,6 @@
|
||||
package cn.smartjavaai.face.config;
|
||||
|
||||
import cn.smartjavaai.common.config.ModelConfig;
|
||||
import cn.smartjavaai.common.enums.DeviceEnum;
|
||||
import cn.smartjavaai.face.enums.ExpressionModelEnum;
|
||||
import cn.smartjavaai.face.model.facedect.FaceDetModel;
|
||||
@@ -14,7 +15,7 @@ import java.util.Map;
|
||||
* @date 2025/7/1
|
||||
*/
|
||||
@Data
|
||||
public class FaceExpressionConfig {
|
||||
public class FaceExpressionConfig extends ModelConfig {
|
||||
|
||||
/**
|
||||
* 模型枚举
|
||||
@@ -26,11 +27,6 @@ public class FaceExpressionConfig {
|
||||
*/
|
||||
private String modelPath;
|
||||
|
||||
/**
|
||||
* 设备类型
|
||||
*/
|
||||
private DeviceEnum device;
|
||||
|
||||
/**
|
||||
* 人脸检测模型
|
||||
*/
|
||||
|
||||
@@ -1,5 +1,6 @@
|
||||
package cn.smartjavaai.face.config;
|
||||
|
||||
import cn.smartjavaai.common.config.ModelConfig;
|
||||
import cn.smartjavaai.common.enums.DeviceEnum;
|
||||
import cn.smartjavaai.face.constant.FaceDetectConstant;
|
||||
import cn.smartjavaai.face.enums.FaceRecModelEnum;
|
||||
@@ -16,7 +17,7 @@ import java.util.Map;
|
||||
* @author dwj
|
||||
*/
|
||||
@Data
|
||||
public class FaceRecConfig {
|
||||
public class FaceRecConfig extends ModelConfig {
|
||||
|
||||
/**
|
||||
* 人脸模型枚举
|
||||
@@ -28,11 +29,6 @@ public class FaceRecConfig {
|
||||
*/
|
||||
private String modelPath;
|
||||
|
||||
/**
|
||||
* 设备类型
|
||||
*/
|
||||
private DeviceEnum device;
|
||||
|
||||
|
||||
/**
|
||||
* 向量数据库配置
|
||||
@@ -60,10 +56,6 @@ public class FaceRecConfig {
|
||||
*/
|
||||
private FaceDetModel detectModel;
|
||||
|
||||
/**
|
||||
* 个性化配置(按模型类型动态解析)
|
||||
*/
|
||||
private Map<String, Object> customParams = new HashMap<>();
|
||||
|
||||
public FaceRecConfig() {
|
||||
}
|
||||
@@ -77,19 +69,5 @@ public class FaceRecConfig {
|
||||
this.modelPath = modelPath;
|
||||
}
|
||||
|
||||
public <T> T getCustomParam(String key, Class<T> clazz) {
|
||||
Object value = customParams.get(key);
|
||||
if (value == null) return null;
|
||||
return clazz.cast(value);
|
||||
}
|
||||
|
||||
/**
|
||||
* 添加个性化配置项
|
||||
*/
|
||||
public void putCustomParam(String key, Object value) {
|
||||
if (customParams == null) {
|
||||
customParams = new HashMap<>();
|
||||
}
|
||||
customParams.put(key, value);
|
||||
}
|
||||
}
|
||||
|
||||
@@ -1,5 +1,6 @@
|
||||
package cn.smartjavaai.face.config;
|
||||
|
||||
import cn.smartjavaai.common.config.ModelConfig;
|
||||
import cn.smartjavaai.common.enums.DeviceEnum;
|
||||
import cn.smartjavaai.face.constant.LivenessConstant;
|
||||
import cn.smartjavaai.face.enums.LivenessModelEnum;
|
||||
@@ -15,7 +16,7 @@ import java.util.Map;
|
||||
* @author dwj
|
||||
*/
|
||||
@Data
|
||||
public class LivenessConfig {
|
||||
public class LivenessConfig extends ModelConfig {
|
||||
|
||||
/**
|
||||
* 活体检测模型枚举
|
||||
@@ -27,20 +28,12 @@ public class LivenessConfig {
|
||||
*/
|
||||
private String modelPath;
|
||||
|
||||
/**
|
||||
* 设备类型
|
||||
*/
|
||||
private DeviceEnum device;
|
||||
|
||||
/**
|
||||
* 人脸检测模型
|
||||
*/
|
||||
private FaceDetModel detectModel;
|
||||
|
||||
/**
|
||||
* 个性化配置(按模型类型动态解析)
|
||||
*/
|
||||
private Map<String, Object> customParams = new HashMap<>();
|
||||
|
||||
|
||||
/**
|
||||
@@ -74,20 +67,4 @@ public class LivenessConfig {
|
||||
this.modelPath = modelPath;
|
||||
}
|
||||
|
||||
// 可选封装方法,便于类型转换和调用
|
||||
public <T> T getCustomParam(String key, Class<T> clazz) {
|
||||
Object value = customParams.get(key);
|
||||
if (value == null) return null;
|
||||
return clazz.cast(value);
|
||||
}
|
||||
|
||||
/**
|
||||
* 添加个性化配置项
|
||||
*/
|
||||
public void putCustomParam(String key, Object value) {
|
||||
if (customParams == null) {
|
||||
customParams = new HashMap<>();
|
||||
}
|
||||
customParams.put(key, value);
|
||||
}
|
||||
}
|
||||
|
||||
@@ -1,5 +1,6 @@
|
||||
package cn.smartjavaai.face.config;
|
||||
|
||||
import cn.smartjavaai.common.config.ModelConfig;
|
||||
import cn.smartjavaai.common.enums.DeviceEnum;
|
||||
import cn.smartjavaai.face.constant.LivenessConstant;
|
||||
import cn.smartjavaai.face.enums.LivenessModelEnum;
|
||||
@@ -11,7 +12,7 @@ import lombok.Data;
|
||||
* @author dwj
|
||||
*/
|
||||
@Data
|
||||
public class QualityConfig {
|
||||
public class QualityConfig extends ModelConfig {
|
||||
|
||||
/**
|
||||
* 活体检测模型枚举
|
||||
@@ -23,16 +24,6 @@ public class QualityConfig {
|
||||
*/
|
||||
private String modelPath;
|
||||
|
||||
/**
|
||||
* 设备类型
|
||||
*/
|
||||
private DeviceEnum device;
|
||||
|
||||
/**
|
||||
* gpu设备ID 当device为GPU时生效
|
||||
*/
|
||||
private int gpuId = 0;
|
||||
|
||||
|
||||
public QualityConfig() {
|
||||
}
|
||||
|
||||
@@ -1,5 +1,6 @@
|
||||
package cn.smartjavaai.face.model.attribute;
|
||||
|
||||
import ai.djl.engine.Engine;
|
||||
import cn.smartjavaai.common.entity.*;
|
||||
import cn.smartjavaai.common.entity.face.FaceAttribute;
|
||||
import cn.smartjavaai.common.entity.face.FaceInfo;
|
||||
@@ -102,6 +103,19 @@ public class Seetaface6FaceAttributeModel implements FaceAttributeModel {
|
||||
this.eyeStateDetectorPool = new EyeStateDetectorPool(eyeStateDetectorPoolConfSetting);
|
||||
this.maskDetectorPool = new MaskDetectorPool(maskDetectorPoolConfSetting);
|
||||
this.poseEstimatorPool = new PoseEstimatorPool(poseEstimatorPoolConfSetting);
|
||||
|
||||
int predictorPoolSize = config.getPredictorPoolSize();
|
||||
if(config.getPredictorPoolSize() <= 0){
|
||||
predictorPoolSize = Runtime.getRuntime().availableProcessors(); // 默认等于CPU核心数
|
||||
}
|
||||
faceDetectorPool.setMaxTotal(predictorPoolSize);
|
||||
faceLandmarkerPool.setMaxTotal(predictorPoolSize);
|
||||
genderPredictorPool.setMaxTotal(predictorPoolSize);
|
||||
agePredictorPool.setMaxTotal(predictorPoolSize);
|
||||
eyeStateDetectorPool.setMaxTotal(predictorPoolSize);
|
||||
maskDetectorPool.setMaxTotal(predictorPoolSize);
|
||||
poseEstimatorPool.setMaxTotal(predictorPoolSize);
|
||||
log.debug("模型推理器线程池最大数量: " + predictorPoolSize);
|
||||
} catch (FileNotFoundException e) {
|
||||
throw new FaceException(e);
|
||||
}
|
||||
@@ -471,6 +485,35 @@ public class Seetaface6FaceAttributeModel implements FaceAttributeModel {
|
||||
}
|
||||
}
|
||||
|
||||
|
||||
public FaceDetectorPool getFaceDetectorPool() {
|
||||
return faceDetectorPool;
|
||||
}
|
||||
|
||||
public GenderPredictorPool getGenderPredictorPool() {
|
||||
return genderPredictorPool;
|
||||
}
|
||||
|
||||
public FaceLandmarkerPool getFaceLandmarkerPool() {
|
||||
return faceLandmarkerPool;
|
||||
}
|
||||
|
||||
public AgePredictorPool getAgePredictorPool() {
|
||||
return agePredictorPool;
|
||||
}
|
||||
|
||||
public EyeStateDetectorPool getEyeStateDetectorPool() {
|
||||
return eyeStateDetectorPool;
|
||||
}
|
||||
|
||||
public MaskDetectorPool getMaskDetectorPool() {
|
||||
return maskDetectorPool;
|
||||
}
|
||||
|
||||
public PoseEstimatorPool getPoseEstimatorPool() {
|
||||
return poseEstimatorPool;
|
||||
}
|
||||
|
||||
@Override
|
||||
public void close() throws Exception {
|
||||
if(Objects.nonNull(faceDetectorPool)){
|
||||
|
||||
@@ -2,6 +2,7 @@ package cn.smartjavaai.face.model.expression;
|
||||
|
||||
import ai.djl.Device;
|
||||
import ai.djl.MalformedModelException;
|
||||
import ai.djl.engine.Engine;
|
||||
import ai.djl.inference.Predictor;
|
||||
import ai.djl.modality.Classifications;
|
||||
import ai.djl.modality.cv.Image;
|
||||
@@ -32,6 +33,7 @@ import lombok.extern.slf4j.Slf4j;
|
||||
import org.apache.commons.lang3.StringUtils;
|
||||
import org.apache.commons.pool2.ObjectPool;
|
||||
import org.apache.commons.pool2.impl.GenericObjectPool;
|
||||
import org.opencv.core.Mat;
|
||||
import org.opencv.face.Face;
|
||||
|
||||
import javax.imageio.ImageIO;
|
||||
@@ -56,7 +58,7 @@ public class CommonEmotionModel implements ExpressionModel{
|
||||
|
||||
private ZooModel<Image, Classifications> model;
|
||||
|
||||
private ObjectPool<Predictor<Image, Classifications>> predictorPool;
|
||||
private GenericObjectPool<Predictor<Image, Classifications>> predictorPool;
|
||||
|
||||
@Override
|
||||
public void loadModel(FaceExpressionConfig config) {
|
||||
@@ -73,6 +75,14 @@ public class CommonEmotionModel implements ExpressionModel{
|
||||
try {
|
||||
model = criteria.loadModel();
|
||||
this.predictorPool = new GenericObjectPool<>(new PredictorFactory<>(model));
|
||||
int predictorPoolSize = config.getPredictorPoolSize();
|
||||
if(config.getPredictorPoolSize() <= 0){
|
||||
predictorPoolSize = Runtime.getRuntime().availableProcessors(); // 默认等于CPU核心数
|
||||
}
|
||||
predictorPool.setMaxTotal(predictorPoolSize);
|
||||
log.debug("当前设备: " + model.getNDManager().getDevice());
|
||||
log.debug("当前引擎: " + Engine.getInstance().getEngineName());
|
||||
log.debug("模型推理器线程池最大数量: " + predictorPoolSize);
|
||||
} catch (IOException | ModelNotFoundException | MalformedModelException e) {
|
||||
throw new FaceException("DenseNetEmotionModel模型加载失败", e);
|
||||
}
|
||||
@@ -153,6 +163,7 @@ public class CommonEmotionModel implements ExpressionModel{
|
||||
result.setClassifications(classifications);
|
||||
faceInfo.setExpressionResult(result);
|
||||
}
|
||||
((Mat)djlImage.getWrappedImage()).release();
|
||||
return faceDetectionResponse;
|
||||
}
|
||||
|
||||
@@ -226,6 +237,7 @@ public class CommonEmotionModel implements ExpressionModel{
|
||||
result.setClassifications(classifications);
|
||||
expressionResults.add(result);
|
||||
}
|
||||
((Mat)djlImage.getWrappedImage()).release();
|
||||
return R.ok(expressionResults);
|
||||
}
|
||||
|
||||
@@ -276,6 +288,7 @@ public class CommonEmotionModel implements ExpressionModel{
|
||||
FacialExpression expression = FacialExpression.fromLabel(bestClass.getClassName());
|
||||
ExpressionResult result = new ExpressionResult(expression, (float)bestClass.getProbability());
|
||||
result.setClassifications(classifications);
|
||||
((Mat)djlImage.getWrappedImage()).release();
|
||||
return R.ok(result);
|
||||
|
||||
}
|
||||
@@ -342,7 +355,10 @@ public class CommonEmotionModel implements ExpressionModel{
|
||||
return detectTopFace(imageData);
|
||||
}
|
||||
|
||||
|
||||
@Override
|
||||
public GenericObjectPool<Predictor<Image, Classifications>> getPool() {
|
||||
return predictorPool;
|
||||
}
|
||||
|
||||
@Override
|
||||
public void close() {
|
||||
|
||||
@@ -1,11 +1,16 @@
|
||||
package cn.smartjavaai.face.model.expression;
|
||||
|
||||
import ai.djl.inference.Predictor;
|
||||
import ai.djl.modality.Classifications;
|
||||
import ai.djl.modality.cv.Image;
|
||||
import ai.djl.modality.cv.output.DetectedObjects;
|
||||
import cn.smartjavaai.common.entity.DetectionRectangle;
|
||||
import cn.smartjavaai.common.entity.DetectionResponse;
|
||||
import cn.smartjavaai.common.entity.Point;
|
||||
import cn.smartjavaai.common.entity.R;
|
||||
import cn.smartjavaai.common.entity.face.ExpressionResult;
|
||||
import cn.smartjavaai.face.config.FaceExpressionConfig;
|
||||
import org.apache.commons.pool2.impl.GenericObjectPool;
|
||||
|
||||
import java.awt.image.BufferedImage;
|
||||
import java.util.List;
|
||||
@@ -190,6 +195,9 @@ public interface ExpressionModel extends AutoCloseable{
|
||||
|
||||
|
||||
|
||||
default GenericObjectPool<Predictor<Image, Classifications>> getPool(){
|
||||
throw new UnsupportedOperationException("默认不支持该功能");
|
||||
}
|
||||
|
||||
|
||||
}
|
||||
|
||||
@@ -26,7 +26,7 @@ public class EmotionCriteriaFactory {
|
||||
public static Criteria<Image, Classifications> createCriteria(FaceExpressionConfig config) {
|
||||
Device device = null;
|
||||
if(!Objects.isNull(config.getDevice())){
|
||||
device = config.getDevice() == DeviceEnum.CPU ? Device.cpu() : Device.gpu();
|
||||
device = config.getDevice() == DeviceEnum.CPU ? Device.cpu() : Device.gpu(config.getGpuId());
|
||||
}
|
||||
Criteria<Image, Classifications> criteria = null;
|
||||
if(config.getModelEnum() == ExpressionModelEnum.DensNet121){
|
||||
@@ -50,6 +50,7 @@ public class EmotionCriteriaFactory {
|
||||
.optModelPath(Paths.get(config.getModelPath()))
|
||||
.optTranslator(new FrEmotionTranslator(224))
|
||||
.optProgress(new ProgressBar())
|
||||
.optDevice(device)
|
||||
.build();
|
||||
}
|
||||
return criteria;
|
||||
|
||||
@@ -24,6 +24,7 @@ import lombok.extern.slf4j.Slf4j;
|
||||
import org.apache.commons.lang3.StringUtils;
|
||||
import org.apache.commons.pool2.ObjectPool;
|
||||
import org.apache.commons.pool2.impl.GenericObjectPool;
|
||||
import org.opencv.core.Mat;
|
||||
|
||||
import javax.imageio.ImageIO;
|
||||
import java.awt.image.BufferedImage;
|
||||
@@ -43,7 +44,7 @@ import java.util.Objects;
|
||||
@Slf4j
|
||||
public class CommonFaceDetModel implements FaceDetModel{
|
||||
|
||||
private ObjectPool<Predictor<Image, DetectedObjects>> predictorPool;
|
||||
private GenericObjectPool<Predictor<Image, DetectedObjects>> predictorPool;
|
||||
|
||||
private ZooModel<Image, DetectedObjects> model;
|
||||
|
||||
@@ -58,8 +59,14 @@ public class CommonFaceDetModel implements FaceDetModel{
|
||||
try {
|
||||
model = criteria.loadModel();
|
||||
this.predictorPool = new GenericObjectPool<>(new PredictorFactory<>(model));
|
||||
int predictorPoolSize = config.getPredictorPoolSize();
|
||||
if(config.getPredictorPoolSize() <= 0){
|
||||
predictorPoolSize = Runtime.getRuntime().availableProcessors(); // 默认等于CPU核心数
|
||||
}
|
||||
predictorPool.setMaxTotal(predictorPoolSize);
|
||||
log.debug("当前设备: " + model.getNDManager().getDevice());
|
||||
log.debug("当前引擎: " + Engine.getInstance().getEngineName());
|
||||
log.debug("模型推理器线程池最大数量: " + predictorPoolSize);
|
||||
} catch (IOException | ModelNotFoundException | MalformedModelException e) {
|
||||
throw new FaceException("人脸检测模型加载失败", e);
|
||||
}
|
||||
@@ -81,11 +88,16 @@ public class CommonFaceDetModel implements FaceDetModel{
|
||||
Image img = null;
|
||||
try {
|
||||
img = ImageFactory.getInstance().fromFile(Paths.get(imagePath));
|
||||
DetectedObjects detection = detect(img);
|
||||
return R.ok(FaceUtils.convertToDetectionResponse(detection,img));
|
||||
} catch (IOException e) {
|
||||
throw new FaceException("无效的图片", e);
|
||||
} finally {
|
||||
if (img != null) {
|
||||
((Mat)img.getWrappedImage()).release();
|
||||
}
|
||||
}
|
||||
DetectedObjects detection = detect(img);
|
||||
return R.ok(FaceUtils.convertToDetectionResponse(detection,img));
|
||||
|
||||
}
|
||||
|
||||
/**
|
||||
@@ -99,12 +111,17 @@ public class CommonFaceDetModel implements FaceDetModel{
|
||||
if(Objects.isNull(imageInputStream)){
|
||||
return R.fail(R.Status.INVALID_IMAGE);
|
||||
}
|
||||
Image img = null;
|
||||
try {
|
||||
Image img = ImageFactory.getInstance().fromInputStream(imageInputStream);
|
||||
img = ImageFactory.getInstance().fromInputStream(imageInputStream);
|
||||
DetectedObjects detection = detect(img);
|
||||
return R.ok(FaceUtils.convertToDetectionResponse(detection,img));
|
||||
} catch (IOException e) {
|
||||
throw new FaceException("无效图片输入流", e);
|
||||
} finally {
|
||||
if (img != null) {
|
||||
((Mat)img.getWrappedImage()).release();
|
||||
}
|
||||
}
|
||||
|
||||
}
|
||||
@@ -114,9 +131,19 @@ public class CommonFaceDetModel implements FaceDetModel{
|
||||
if(!ImageUtils.isImageValid(image)){
|
||||
return R.fail(R.Status.INVALID_IMAGE);
|
||||
}
|
||||
Image img = ImageFactory.getInstance().fromImage(OpenCVUtils.image2Mat(image));
|
||||
DetectedObjects detection = detect(img);
|
||||
return R.ok(FaceUtils.convertToDetectionResponse(detection,img));
|
||||
Image img = null;
|
||||
try {
|
||||
img = ImageFactory.getInstance().fromImage(OpenCVUtils.image2Mat(image));
|
||||
DetectedObjects detection = detect(img);
|
||||
return R.ok(FaceUtils.convertToDetectionResponse(detection,img));
|
||||
} catch (Exception e) {
|
||||
throw new FaceException(e);
|
||||
} finally {
|
||||
if (img != null) {
|
||||
((Mat)img.getWrappedImage()).release();
|
||||
}
|
||||
}
|
||||
|
||||
}
|
||||
|
||||
@Override
|
||||
@@ -124,11 +151,7 @@ public class CommonFaceDetModel implements FaceDetModel{
|
||||
if(Objects.isNull(imageData)){
|
||||
return R.fail(R.Status.INVALID_IMAGE);
|
||||
}
|
||||
try {
|
||||
return detect(ImageIO.read(new ByteArrayInputStream(imageData)));
|
||||
} catch (IOException e) {
|
||||
throw new FaceException("错误的图像", e);
|
||||
}
|
||||
return detect(new ByteArrayInputStream(imageData));
|
||||
}
|
||||
|
||||
@Override
|
||||
@@ -145,8 +168,9 @@ public class CommonFaceDetModel implements FaceDetModel{
|
||||
if(!FileUtils.isFileExists(imagePath)){
|
||||
return R.fail(R.Status.FILE_NOT_FOUND);
|
||||
}
|
||||
Image img = null;
|
||||
try {
|
||||
Image img = ImageFactory.getInstance().fromFile(Paths.get(imagePath));
|
||||
img = ImageFactory.getInstance().fromFile(Paths.get(imagePath));
|
||||
DetectedObjects detectedObjects = detect(img);
|
||||
if(Objects.isNull(detectedObjects) || detectedObjects.getNumberOfObjects() == 0){
|
||||
return R.fail(R.Status.NO_FACE_DETECTED);
|
||||
@@ -158,6 +182,10 @@ public class CommonFaceDetModel implements FaceDetModel{
|
||||
return R.ok();
|
||||
} catch (IOException e) {
|
||||
throw new FaceException(e);
|
||||
} finally {
|
||||
if (img != null){
|
||||
((Mat)img.getWrappedImage()).release();
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
@@ -181,6 +209,10 @@ public class CommonFaceDetModel implements FaceDetModel{
|
||||
return R.ok(ImageIO.read(new ByteArrayInputStream(imageBytes)));
|
||||
} catch (IOException e) {
|
||||
throw new FaceException("导出图片失败", e);
|
||||
} finally {
|
||||
if (img != null){
|
||||
((Mat)img.getWrappedImage()).release();
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
@@ -213,6 +245,11 @@ public class CommonFaceDetModel implements FaceDetModel{
|
||||
}
|
||||
|
||||
|
||||
@Override
|
||||
public GenericObjectPool<Predictor<Image, DetectedObjects>> getPool() {
|
||||
return predictorPool;
|
||||
}
|
||||
|
||||
@Override
|
||||
public void close() {
|
||||
try {
|
||||
|
||||
@@ -1,8 +1,13 @@
|
||||
package cn.smartjavaai.face.model.facedect;
|
||||
|
||||
import ai.djl.inference.Predictor;
|
||||
import ai.djl.modality.Classifications;
|
||||
import ai.djl.modality.cv.Image;
|
||||
import ai.djl.modality.cv.output.DetectedObjects;
|
||||
import cn.smartjavaai.common.entity.DetectionResponse;
|
||||
import cn.smartjavaai.common.entity.R;
|
||||
import cn.smartjavaai.face.config.FaceDetConfig;
|
||||
import org.apache.commons.pool2.impl.GenericObjectPool;
|
||||
|
||||
import java.awt.image.BufferedImage;
|
||||
import java.io.InputStream;
|
||||
@@ -83,6 +88,10 @@ public interface FaceDetModel extends AutoCloseable{
|
||||
throw new UnsupportedOperationException("默认不支持该功能");
|
||||
}
|
||||
|
||||
default GenericObjectPool<Predictor<Image, DetectedObjects>> getPool(){
|
||||
throw new UnsupportedOperationException("默认不支持该功能");
|
||||
}
|
||||
|
||||
|
||||
|
||||
}
|
||||
|
||||
@@ -1,5 +1,6 @@
|
||||
package cn.smartjavaai.face.model.facedect;
|
||||
|
||||
import ai.djl.engine.Engine;
|
||||
import cn.smartjavaai.common.entity.DetectionResponse;
|
||||
import cn.smartjavaai.common.entity.R;
|
||||
import cn.smartjavaai.common.enums.DeviceEnum;
|
||||
@@ -48,13 +49,9 @@ public class SeetaFace6FaceDetModel implements FaceDetModel{
|
||||
String[] faceDetectorModelPath = {config.getModelPath() + File.separator + "face_detector.csta"};
|
||||
String[] faceLandmarkerModelPath = {config.getModelPath() + File.separator + "face_landmarker_pts5.csta"};
|
||||
SeetaDevice device = SeetaDevice.SEETA_DEVICE_AUTO;
|
||||
int gpuId = 0;
|
||||
int gpuId = config.getGpuId();
|
||||
if(Objects.nonNull(config.getDevice())){
|
||||
device = config.getDevice() == DeviceEnum.CPU ? SeetaDevice.SEETA_DEVICE_CPU : SeetaDevice.SEETA_DEVICE_GPU;
|
||||
Integer gpuIdValue = config.getCustomParam("gpuId", Integer.class);
|
||||
if(Objects.nonNull(gpuIdValue) && device == SeetaDevice.SEETA_DEVICE_GPU){
|
||||
gpuId = gpuIdValue;
|
||||
}
|
||||
}
|
||||
try {
|
||||
SeetaModelSetting faceDetectorPoolSetting = new SeetaModelSetting(gpuId, faceDetectorModelPath, device);
|
||||
@@ -65,6 +62,14 @@ public class SeetaFace6FaceDetModel implements FaceDetModel{
|
||||
|
||||
this.faceDetectorPool = new FaceDetectorPool(faceDetectorPoolConfSetting);
|
||||
this.faceLandmarkerPool = new FaceLandmarkerPool(faceLandmarkerPoolConfSetting);
|
||||
|
||||
int predictorPoolSize = config.getPredictorPoolSize();
|
||||
if(config.getPredictorPoolSize() <= 0){
|
||||
predictorPoolSize = Runtime.getRuntime().availableProcessors(); // 默认等于CPU核心数
|
||||
}
|
||||
faceDetectorPool.setMaxTotal(predictorPoolSize);
|
||||
faceLandmarkerPool.setMaxTotal(predictorPoolSize);
|
||||
log.debug("模型推理器线程池最大数量: " + predictorPoolSize);
|
||||
} catch (FileNotFoundException e) {
|
||||
throw new FaceException(e);
|
||||
}
|
||||
@@ -212,8 +217,13 @@ public class SeetaFace6FaceDetModel implements FaceDetModel{
|
||||
}
|
||||
|
||||
|
||||
public FaceDetectorPool getFaceDetectorPool() {
|
||||
return faceDetectorPool;
|
||||
}
|
||||
|
||||
|
||||
public FaceLandmarkerPool getFaceLandmarkerPool() {
|
||||
return faceLandmarkerPool;
|
||||
}
|
||||
|
||||
@Override
|
||||
public void close() throws Exception {
|
||||
|
||||
@@ -31,7 +31,7 @@ public class FaceDetCriteriaFactory {
|
||||
public static Criteria<Image, DetectedObjects> createCriteria(FaceDetConfig config) {
|
||||
Device device = null;
|
||||
if(!Objects.isNull(config.getDevice())){
|
||||
device = config.getDevice() == DeviceEnum.CPU ? Device.cpu() : Device.gpu();
|
||||
device = config.getDevice() == DeviceEnum.CPU ? Device.cpu() : Device.gpu(config.getGpuId());
|
||||
}
|
||||
Criteria<Image, DetectedObjects> criteria = null;
|
||||
if(config.getModelEnum() == FaceDetModelEnum.RETINA_FACE){
|
||||
|
||||
@@ -9,6 +9,7 @@ import ai.djl.ndarray.NDManager;
|
||||
import ai.djl.repository.zoo.Criteria;
|
||||
import ai.djl.repository.zoo.ModelNotFoundException;
|
||||
import ai.djl.repository.zoo.ZooModel;
|
||||
import cn.hutool.core.lang.UUID;
|
||||
import cn.smartjavaai.common.entity.*;
|
||||
import cn.smartjavaai.common.entity.face.FaceInfo;
|
||||
import cn.smartjavaai.common.entity.face.FaceSearchResult;
|
||||
@@ -41,6 +42,7 @@ import lombok.extern.slf4j.Slf4j;
|
||||
import org.apache.commons.lang3.StringUtils;
|
||||
import org.apache.commons.pool2.ObjectPool;
|
||||
import org.apache.commons.pool2.impl.GenericObjectPool;
|
||||
import org.opencv.core.Mat;
|
||||
|
||||
import javax.imageio.ImageIO;
|
||||
import java.awt.image.BufferedImage;
|
||||
@@ -68,7 +70,7 @@ public class CommonFaceRecModel implements FaceRecModel{
|
||||
private static volatile boolean isLoadCompleted = false;
|
||||
|
||||
|
||||
private ObjectPool<Predictor<Image, float[]>> predictorPool;
|
||||
private GenericObjectPool<Predictor<Image, float[]>> predictorPool;
|
||||
|
||||
|
||||
private ZooModel<Image, float[]> model;
|
||||
@@ -101,8 +103,14 @@ public class CommonFaceRecModel implements FaceRecModel{
|
||||
model = faceFeatureCriteria.loadModel();
|
||||
// 创建池子:每个线程独享 Predictor
|
||||
this.predictorPool = new GenericObjectPool<>(new PredictorFactory<>(model));
|
||||
int predictorPoolSize = config.getPredictorPoolSize();
|
||||
if(config.getPredictorPoolSize() <= 0){
|
||||
predictorPoolSize = Runtime.getRuntime().availableProcessors(); // 默认等于CPU核心数
|
||||
}
|
||||
predictorPool.setMaxTotal(predictorPoolSize);
|
||||
log.debug("当前设备: " + model.getNDManager().getDevice());
|
||||
log.debug("当前引擎: " + Engine.getInstance().getEngineName());
|
||||
log.debug("模型推理器线程池最大数量: " + predictorPoolSize);
|
||||
} catch (IOException | ModelNotFoundException | MalformedModelException e) {
|
||||
throw new FaceException("模型加载失败", e);
|
||||
}
|
||||
@@ -286,6 +294,7 @@ public class CommonFaceRecModel implements FaceRecModel{
|
||||
faceInfo.setFeature(features);
|
||||
}
|
||||
}
|
||||
((Mat)djlImage.getWrappedImage()).release();
|
||||
return detectedResult;
|
||||
}
|
||||
|
||||
@@ -350,6 +359,7 @@ public class CommonFaceRecModel implements FaceRecModel{
|
||||
}
|
||||
features = featureExtraction(subImage);
|
||||
}
|
||||
((Mat)djlImage.getWrappedImage()).release();
|
||||
return Objects.isNull(features) ? R.fail(R.Status.Unknown) : R.ok(features);
|
||||
}
|
||||
|
||||
@@ -703,5 +713,8 @@ public class CommonFaceRecModel implements FaceRecModel{
|
||||
}
|
||||
|
||||
|
||||
|
||||
@Override
|
||||
public GenericObjectPool<Predictor<Image, float[]>> getPool() {
|
||||
return predictorPool;
|
||||
}
|
||||
}
|
||||
|
||||
@@ -1,5 +1,7 @@
|
||||
package cn.smartjavaai.face.model.facerec;
|
||||
|
||||
import ai.djl.inference.Predictor;
|
||||
import ai.djl.modality.cv.Image;
|
||||
import cn.smartjavaai.common.entity.DetectionResponse;
|
||||
import cn.smartjavaai.common.entity.R;
|
||||
import cn.smartjavaai.face.config.FaceRecConfig;
|
||||
@@ -7,6 +9,7 @@ import cn.smartjavaai.face.entity.FaceRegisterInfo;
|
||||
import cn.smartjavaai.face.entity.FaceSearchParams;
|
||||
import cn.smartjavaai.common.entity.face.FaceSearchResult;
|
||||
import cn.smartjavaai.face.vector.entity.FaceVector;
|
||||
import org.apache.commons.pool2.impl.GenericObjectPool;
|
||||
|
||||
import java.awt.image.BufferedImage;
|
||||
import java.io.InputStream;
|
||||
@@ -367,4 +370,9 @@ public interface FaceRecModel extends AutoCloseable{
|
||||
throw new UnsupportedOperationException("默认不支持该功能");
|
||||
}
|
||||
|
||||
|
||||
default GenericObjectPool<Predictor<Image, float[]>> getPool() {
|
||||
throw new UnsupportedOperationException("默认不支持该功能");
|
||||
}
|
||||
|
||||
}
|
||||
|
||||
@@ -1,5 +1,6 @@
|
||||
package cn.smartjavaai.face.model.facerec;
|
||||
|
||||
import ai.djl.engine.Engine;
|
||||
import cn.smartjavaai.common.entity.DetectionInfo;
|
||||
import cn.smartjavaai.common.entity.DetectionResponse;
|
||||
import cn.smartjavaai.common.entity.R;
|
||||
@@ -89,13 +90,9 @@ public class SeetaFace6FaceRecModel implements FaceRecModel{
|
||||
String[] faceRecognizerModelPath = {config.getModelPath() + File.separator + "face_recognizer.csta"};
|
||||
String[] faceLandmarkerModelPath = {config.getModelPath() + File.separator + "face_landmarker_pts5.csta"};
|
||||
SeetaDevice device = SeetaDevice.SEETA_DEVICE_AUTO;
|
||||
int gpuId = 0;
|
||||
int gpuId = config.getGpuId();
|
||||
if(Objects.nonNull(config.getDevice())){
|
||||
device = config.getDevice() == DeviceEnum.CPU ? SeetaDevice.SEETA_DEVICE_CPU : SeetaDevice.SEETA_DEVICE_GPU;
|
||||
Integer gpuIdValue = config.getCustomParam("gpuId", Integer.class);
|
||||
if(Objects.nonNull(gpuIdValue) && device == SeetaDevice.SEETA_DEVICE_GPU){
|
||||
gpuId = gpuIdValue;
|
||||
}
|
||||
}
|
||||
try {
|
||||
SeetaModelSetting faceDetectorPoolSetting = new SeetaModelSetting(gpuId, faceDetectorModelPath, device);
|
||||
@@ -115,6 +112,16 @@ public class SeetaFace6FaceRecModel implements FaceRecModel{
|
||||
this.faceLandmarkerPool = new FaceLandmarkerPool(faceLandmarkerPoolConfSetting);
|
||||
this.faceDatabasePool = new FaceDatabasePool(faceDatabasePoolConfSetting);
|
||||
|
||||
int predictorPoolSize = config.getPredictorPoolSize();
|
||||
if(config.getPredictorPoolSize() <= 0){
|
||||
predictorPoolSize = Runtime.getRuntime().availableProcessors(); // 默认等于CPU核心数
|
||||
}
|
||||
faceDetectorPool.setMaxTotal(predictorPoolSize);
|
||||
faceRecognizerPool.setMaxTotal(predictorPoolSize);
|
||||
faceLandmarkerPool.setMaxTotal(predictorPoolSize);
|
||||
faceDatabasePool.setMaxTotal(predictorPoolSize);
|
||||
log.debug("模型推理器线程池最大数量: " + predictorPoolSize);
|
||||
|
||||
|
||||
//初始化人脸库
|
||||
if(config.getVectorDBConfig() != null && config.getVectorDBConfig().getType() != null){
|
||||
@@ -739,6 +746,36 @@ public class SeetaFace6FaceRecModel implements FaceRecModel{
|
||||
}
|
||||
}
|
||||
|
||||
/**
|
||||
* 特征提取
|
||||
* @param image
|
||||
* @return
|
||||
*/
|
||||
public float[] featureExtraction(BufferedImage image){
|
||||
FaceRecognizer faceRecognizer = null;
|
||||
try {
|
||||
faceRecognizer = faceRecognizerPool.borrowObject();
|
||||
SeetaImageData imageData = new SeetaImageData(image.getWidth(), image.getHeight(), 3);
|
||||
imageData.data = ImageUtils.getMatrixBGR(image);
|
||||
//提取特征
|
||||
float[] features = new float[faceRecognizer.GetExtractFeatureSize()];
|
||||
faceRecognizer.ExtractCroppedFace(imageData, features);
|
||||
return features;
|
||||
} catch (FaceException e) {
|
||||
throw e;
|
||||
} catch (Exception e) {
|
||||
throw new FaceException("目标检测错误", e);
|
||||
}finally {
|
||||
if (faceRecognizer != null) {
|
||||
try {
|
||||
faceRecognizerPool.returnObject(faceRecognizer); //归还
|
||||
} catch (Exception e) {
|
||||
log.warn("归还Predictor失败", e);
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
@Override
|
||||
public R<float[]> extractTopFaceFeature(String imagePath) {
|
||||
if(!FileUtils.isFileExists(imagePath)){
|
||||
@@ -878,4 +915,19 @@ public class SeetaFace6FaceRecModel implements FaceRecModel{
|
||||
}
|
||||
|
||||
|
||||
public FaceDetectorPool getFaceDetectorPool() {
|
||||
return faceDetectorPool;
|
||||
}
|
||||
|
||||
public FaceRecognizerPool getFaceRecognizerPool() {
|
||||
return faceRecognizerPool;
|
||||
}
|
||||
|
||||
public FaceLandmarkerPool getFaceLandmarkerPool() {
|
||||
return faceLandmarkerPool;
|
||||
}
|
||||
|
||||
public FaceDatabasePool getFaceDatabasePool() {
|
||||
return faceDatabasePool;
|
||||
}
|
||||
}
|
||||
|
||||
@@ -27,7 +27,7 @@ public class FaceRecCriteriaFactory {
|
||||
public static Criteria<Image, float[]> createCriteria(FaceRecConfig config) {
|
||||
Device device = null;
|
||||
if(!Objects.isNull(config.getDevice())){
|
||||
device = config.getDevice() == DeviceEnum.CPU ? Device.cpu() : Device.gpu();
|
||||
device = config.getDevice() == DeviceEnum.CPU ? Device.cpu() : Device.gpu(config.getGpuId());
|
||||
}
|
||||
Criteria<Image, float[]> criteria = null;
|
||||
if(config.getModelEnum() == FaceRecModelEnum.FACENET_MODEL){
|
||||
@@ -58,6 +58,7 @@ public class FaceRecCriteriaFactory {
|
||||
.optTranslator(new FaceFeatureTranslator())
|
||||
.optEngine("PyTorch") // Use PyTorch engine
|
||||
.optProgress(new ProgressBar())
|
||||
.optDevice(device)
|
||||
.build();
|
||||
}else if (config.getModelEnum() == FaceRecModelEnum.INSIGHT_FACE_IRSE50_MODEL){
|
||||
if(StringUtils.isBlank(config.getModelPath())){
|
||||
@@ -73,6 +74,7 @@ public class FaceRecCriteriaFactory {
|
||||
// .optArgument("resize", "112,112")
|
||||
.optTranslator(new FaceFeatureTranslator())
|
||||
.optEngine("PyTorch") // Use PyTorch engine
|
||||
.optDevice(device)
|
||||
.optProgress(new ProgressBar())
|
||||
.build();
|
||||
}else if (config.getModelEnum() == FaceRecModelEnum.ELASTIC_FACE_MODEL){
|
||||
@@ -89,6 +91,7 @@ public class FaceRecCriteriaFactory {
|
||||
// .optArgument("resize", "112,112")
|
||||
.optTranslator(new FaceFeatureTranslator())
|
||||
.optEngine("PyTorch") // Use PyTorch engine
|
||||
.optDevice(device)
|
||||
.optProgress(new ProgressBar())
|
||||
.build();
|
||||
}
|
||||
|
||||
@@ -31,7 +31,7 @@ public final class FaceFeatureTranslator implements Translator<Image, float[]> {
|
||||
NDArray array = input.toNDArray(ctx.getNDManager(), Image.Flag.COLOR);
|
||||
Pipeline pipeline = new Pipeline();
|
||||
if(input.getWidth() != 112 || input.getHeight() != 112){
|
||||
pipeline.add(new Resize(112));
|
||||
pipeline.add(new Resize(112,112));
|
||||
}
|
||||
pipeline
|
||||
.add(new ToTensor())
|
||||
|
||||
@@ -2,6 +2,7 @@ package cn.smartjavaai.face.model.liveness;
|
||||
|
||||
import ai.djl.Device;
|
||||
import ai.djl.MalformedModelException;
|
||||
import ai.djl.engine.Engine;
|
||||
import ai.djl.inference.Predictor;
|
||||
import ai.djl.modality.cv.Image;
|
||||
import ai.djl.modality.cv.ImageFactory;
|
||||
@@ -35,6 +36,7 @@ import org.apache.commons.pool2.impl.GenericObjectPool;
|
||||
import org.bytedeco.javacv.FFmpegFrameGrabber;
|
||||
import org.bytedeco.javacv.Frame;
|
||||
import org.bytedeco.javacv.Java2DFrameUtils;
|
||||
import org.opencv.core.Mat;
|
||||
|
||||
import javax.imageio.ImageIO;
|
||||
import java.awt.image.BufferedImage;
|
||||
@@ -54,7 +56,7 @@ import java.util.*;
|
||||
@Slf4j
|
||||
public class CommonLivenessModel implements LivenessDetModel{
|
||||
|
||||
protected ObjectPool<Predictor<Image, Float>> predictorPool;
|
||||
protected GenericObjectPool<Predictor<Image, Float>> predictorPool;
|
||||
|
||||
protected LivenessConfig config;
|
||||
|
||||
@@ -78,6 +80,15 @@ public class CommonLivenessModel implements LivenessDetModel{
|
||||
} catch (IOException | ModelNotFoundException | MalformedModelException e) {
|
||||
throw new FaceException("阿里通义实验室活体检测模型加载失败", e);
|
||||
}
|
||||
|
||||
int predictorPoolSize = config.getPredictorPoolSize();
|
||||
if(config.getPredictorPoolSize() <= 0){
|
||||
predictorPoolSize = Runtime.getRuntime().availableProcessors(); // 默认等于CPU核心数
|
||||
}
|
||||
predictorPool.setMaxTotal(predictorPoolSize);
|
||||
log.debug("当前设备: " + model.getNDManager().getDevice());
|
||||
log.debug("当前引擎: " + Engine.getInstance().getEngineName());
|
||||
log.debug("模型推理器线程池最大数量: " + predictorPoolSize);
|
||||
}
|
||||
|
||||
|
||||
@@ -87,6 +98,7 @@ public class CommonLivenessModel implements LivenessDetModel{
|
||||
throw new FaceException("图像无效");
|
||||
}
|
||||
Predictor<Image, Float> predictor = null;
|
||||
Image djlImage = null;
|
||||
try {
|
||||
predictor = predictorPool.borrowObject();
|
||||
//预处理图片
|
||||
@@ -101,7 +113,7 @@ public class CommonLivenessModel implements LivenessDetModel{
|
||||
.setCenterCropSize(112)
|
||||
.process();
|
||||
}
|
||||
Image djlImage = ImageFactory.getInstance().fromImage(OpenCVUtils.image2Mat(processedImage));
|
||||
djlImage = ImageFactory.getInstance().fromImage(OpenCVUtils.image2Mat(processedImage));
|
||||
Float result = predictor.predict(djlImage);
|
||||
if(result >= config.getRealityThreshold()){
|
||||
return R.ok(new LivenessResult(LivenessStatus.LIVE, result));
|
||||
@@ -124,6 +136,10 @@ public class CommonLivenessModel implements LivenessDetModel{
|
||||
}
|
||||
}
|
||||
}
|
||||
if (djlImage != null){
|
||||
((Mat)djlImage.getWrappedImage()).release();
|
||||
}
|
||||
|
||||
}
|
||||
}
|
||||
|
||||
@@ -403,6 +419,12 @@ public class CommonLivenessModel implements LivenessDetModel{
|
||||
return R.fail(R.Status.Unknown);
|
||||
}
|
||||
|
||||
|
||||
@Override
|
||||
public GenericObjectPool<Predictor<Image, Float>> getPool() {
|
||||
return predictorPool;
|
||||
}
|
||||
|
||||
@Override
|
||||
public void close() throws Exception {
|
||||
try {
|
||||
|
||||
@@ -1,11 +1,14 @@
|
||||
package cn.smartjavaai.face.model.liveness;
|
||||
|
||||
import ai.djl.inference.Predictor;
|
||||
import ai.djl.modality.cv.Image;
|
||||
import cn.smartjavaai.common.entity.DetectionRectangle;
|
||||
import cn.smartjavaai.common.entity.DetectionResponse;
|
||||
import cn.smartjavaai.common.entity.Point;
|
||||
import cn.smartjavaai.common.entity.R;
|
||||
import cn.smartjavaai.common.entity.face.LivenessResult;
|
||||
import cn.smartjavaai.face.config.LivenessConfig;
|
||||
import org.apache.commons.pool2.impl.GenericObjectPool;
|
||||
|
||||
import java.awt.image.BufferedImage;
|
||||
import java.io.InputStream;
|
||||
@@ -292,6 +295,9 @@ public interface LivenessDetModel extends AutoCloseable{
|
||||
|
||||
|
||||
|
||||
default GenericObjectPool<Predictor<Image, Float>> getPool() {
|
||||
throw new UnsupportedOperationException("默认不支持该功能");
|
||||
}
|
||||
|
||||
|
||||
|
||||
|
||||
@@ -2,6 +2,7 @@ package cn.smartjavaai.face.model.liveness;
|
||||
|
||||
import ai.djl.Device;
|
||||
import ai.djl.MalformedModelException;
|
||||
import ai.djl.engine.Engine;
|
||||
import ai.djl.inference.Predictor;
|
||||
import ai.djl.modality.cv.Image;
|
||||
import ai.djl.modality.cv.ImageFactory;
|
||||
@@ -33,6 +34,7 @@ import org.apache.commons.pool2.impl.GenericObjectPool;
|
||||
import org.bytedeco.javacv.FFmpegFrameGrabber;
|
||||
import org.bytedeco.javacv.Frame;
|
||||
import org.bytedeco.javacv.Java2DFrameUtils;
|
||||
import org.opencv.core.Mat;
|
||||
|
||||
import javax.imageio.ImageIO;
|
||||
import java.awt.image.BufferedImage;
|
||||
@@ -58,9 +60,9 @@ public class MiniVisionLivenessModel extends CommonLivenessModel{
|
||||
*/
|
||||
private static final String SE_MODEL_PATH_KEY = "seModelPath";
|
||||
|
||||
private ObjectPool<Predictor<Image, float[]>> predictorPool;
|
||||
private GenericObjectPool<Predictor<Image, float[]>> predictorPool;
|
||||
|
||||
private ObjectPool<Predictor<Image, float[]>> sePredictorPool;
|
||||
private GenericObjectPool<Predictor<Image, float[]>> sePredictorPool;
|
||||
|
||||
|
||||
/**
|
||||
@@ -87,7 +89,7 @@ public class MiniVisionLivenessModel extends CommonLivenessModel{
|
||||
this.config.setRealityThreshold(realityThreshold);
|
||||
Device device = null;
|
||||
if(!Objects.isNull(config.getDevice())){
|
||||
device = config.getDevice() == DeviceEnum.CPU ? Device.cpu() : Device.gpu();
|
||||
device = config.getDevice() == DeviceEnum.CPU ? Device.cpu() : Device.gpu(config.getGpuId());
|
||||
}
|
||||
if(StringUtils.isNotBlank(config.getModelPath()) && StringUtils.isBlank(seModelPath)){
|
||||
//2.7_80x80_MiniFASNetV2
|
||||
@@ -109,6 +111,7 @@ public class MiniVisionLivenessModel extends CommonLivenessModel{
|
||||
.optModelPath(Paths.get(config.getModelPath()))
|
||||
.optTranslator(new MiniVisionTranslator())
|
||||
.optProgress(new ProgressBar())
|
||||
.optDevice(device)
|
||||
.build();
|
||||
try {
|
||||
model = criteria.loadModel();
|
||||
@@ -128,6 +131,7 @@ public class MiniVisionLivenessModel extends CommonLivenessModel{
|
||||
.optModelPath(Paths.get(seModelPath))
|
||||
.optTranslator(new MiniVisionTranslator())
|
||||
.optProgress(new ProgressBar())
|
||||
.optDevice(device)
|
||||
.build();
|
||||
try {
|
||||
seModel = seCriteria.loadModel();
|
||||
@@ -136,6 +140,16 @@ public class MiniVisionLivenessModel extends CommonLivenessModel{
|
||||
throw new FaceException("MiniFASNetV1SE模型加载失败", e);
|
||||
}
|
||||
}
|
||||
|
||||
int predictorPoolSize = config.getPredictorPoolSize();
|
||||
if(config.getPredictorPoolSize() <= 0){
|
||||
predictorPoolSize = Runtime.getRuntime().availableProcessors(); // 默认等于CPU核心数
|
||||
}
|
||||
predictorPool.setMaxTotal(predictorPoolSize);
|
||||
sePredictorPool.setMaxTotal(predictorPoolSize);
|
||||
log.debug("当前设备: " + model.getNDManager().getDevice());
|
||||
log.debug("当前引擎: " + Engine.getInstance().getEngineName());
|
||||
log.debug("模型推理器线程池最大数量: " + predictorPoolSize);
|
||||
}
|
||||
|
||||
|
||||
@@ -160,6 +174,8 @@ public class MiniVisionLivenessModel extends CommonLivenessModel{
|
||||
predictor = predictorPool.borrowObject();
|
||||
Image djlImage = ImageFactory.getInstance().fromImage(OpenCVUtils.image2Mat(processedImage));
|
||||
result = predictor.predict(djlImage);
|
||||
((Mat)djlImage.getWrappedImage()).release();
|
||||
|
||||
}
|
||||
if(Objects.nonNull(sePredictorPool)){
|
||||
//预处理图片
|
||||
@@ -172,6 +188,7 @@ public class MiniVisionLivenessModel extends CommonLivenessModel{
|
||||
Image djlImage = ImageFactory.getInstance().fromImage(OpenCVUtils.image2Mat(processedImage));
|
||||
sePredictor = sePredictorPool.borrowObject();
|
||||
seResult = sePredictor.predict(djlImage);
|
||||
((Mat)djlImage.getWrappedImage()).release();
|
||||
}
|
||||
if(Objects.isNull(result) && Objects.isNull(seResult)){
|
||||
throw new FaceException("活体检测错误");
|
||||
@@ -221,6 +238,14 @@ public class MiniVisionLivenessModel extends CommonLivenessModel{
|
||||
}
|
||||
}
|
||||
|
||||
public GenericObjectPool<Predictor<Image, float[]>> getPredictorPool() {
|
||||
return predictorPool;
|
||||
}
|
||||
|
||||
public GenericObjectPool<Predictor<Image, float[]>> getSePredictorPool() {
|
||||
return sePredictorPool;
|
||||
}
|
||||
|
||||
@Override
|
||||
public void close() throws Exception {
|
||||
try {
|
||||
|
||||
@@ -1,5 +1,6 @@
|
||||
package cn.smartjavaai.face.model.liveness;
|
||||
|
||||
import ai.djl.engine.Engine;
|
||||
import cn.smartjavaai.common.entity.*;
|
||||
import cn.smartjavaai.common.entity.face.FaceInfo;
|
||||
import cn.smartjavaai.common.entity.face.LivenessResult;
|
||||
@@ -56,13 +57,9 @@ public class Seetaface6LivenessModel implements LivenessDetModel{
|
||||
String[] faceAntiSpoofingModelPath = {config.getModelPath() + File.separator + "fas_first.csta",config.getModelPath() + File.separator + "fas_second.csta"};
|
||||
String[] faceLandmarkerModelPath = {config.getModelPath() + File.separator + "face_landmarker_pts5.csta"};
|
||||
SeetaDevice device = SeetaDevice.SEETA_DEVICE_AUTO;
|
||||
int gpuId = 0;
|
||||
int gpuId = config.getGpuId();
|
||||
if(Objects.nonNull(config.getDevice())){
|
||||
device = config.getDevice() == DeviceEnum.CPU ? SeetaDevice.SEETA_DEVICE_CPU : SeetaDevice.SEETA_DEVICE_GPU;
|
||||
Integer gpuIdValue = config.getCustomParam("gpuId", Integer.class);
|
||||
if(Objects.nonNull(gpuIdValue) && device == SeetaDevice.SEETA_DEVICE_GPU){
|
||||
gpuId = gpuIdValue;
|
||||
}
|
||||
}
|
||||
|
||||
try {
|
||||
@@ -79,6 +76,16 @@ public class Seetaface6LivenessModel implements LivenessDetModel{
|
||||
this.faceAntiSpoofingPool = new FaceAntiSpoofingPool(faceAntiSpoofingPoolConfSetting);
|
||||
this.faceLandmarkerPool = new FaceLandmarkerPool(faceLandmarkerPoolConfSetting);
|
||||
|
||||
int predictorPoolSize = config.getPredictorPoolSize();
|
||||
if(config.getPredictorPoolSize() <= 0){
|
||||
predictorPoolSize = Runtime.getRuntime().availableProcessors(); // 默认等于CPU核心数
|
||||
}
|
||||
|
||||
faceDetectorPool.setMaxTotal(predictorPoolSize);
|
||||
faceAntiSpoofingPool.setMaxTotal(predictorPoolSize);
|
||||
faceLandmarkerPool.setMaxTotal(predictorPoolSize);
|
||||
log.debug("模型推理器线程池最大数量: " + predictorPoolSize);
|
||||
|
||||
//初始化模型参数
|
||||
initConfig();
|
||||
} catch (FileNotFoundException e) {
|
||||
@@ -550,6 +557,18 @@ public class Seetaface6LivenessModel implements LivenessDetModel{
|
||||
return R.fail(1000, "有效帧数量不足,无法完成活体检测");
|
||||
}
|
||||
|
||||
public FaceDetectorPool getFaceDetectorPool() {
|
||||
return faceDetectorPool;
|
||||
}
|
||||
|
||||
public FaceAntiSpoofingPool getFaceAntiSpoofingPool() {
|
||||
return faceAntiSpoofingPool;
|
||||
}
|
||||
|
||||
public FaceLandmarkerPool getFaceLandmarkerPool() {
|
||||
return faceLandmarkerPool;
|
||||
}
|
||||
|
||||
@Override
|
||||
public void close() throws Exception {
|
||||
try {
|
||||
|
||||
@@ -31,7 +31,7 @@ public class LivenessCriteriaFactory {
|
||||
public static Criteria<Image, Float> createCriteria(LivenessConfig config) {
|
||||
Device device = null;
|
||||
if(!Objects.isNull(config.getDevice())){
|
||||
device = config.getDevice() == DeviceEnum.CPU ? Device.cpu() : Device.gpu();
|
||||
device = config.getDevice() == DeviceEnum.CPU ? Device.cpu() : Device.gpu(config.getGpuId());
|
||||
}
|
||||
Criteria<Image, Float> criteria = null;
|
||||
if(config.getModelEnum() == LivenessModelEnum.IIC_FL_MODEL){
|
||||
|
||||
@@ -1,5 +1,6 @@
|
||||
package cn.smartjavaai.face.model.quality;
|
||||
|
||||
import ai.djl.engine.Engine;
|
||||
import cn.smartjavaai.common.entity.*;
|
||||
import cn.smartjavaai.common.enums.DeviceEnum;
|
||||
import cn.smartjavaai.common.utils.FileUtils;
|
||||
@@ -74,6 +75,8 @@ public class Seetaface6QualityModel implements FaceQualityModel {
|
||||
*/
|
||||
private QualityOfResolutionPool qualityOfResolutionPool;
|
||||
|
||||
int predictorPoolSize = 0;
|
||||
|
||||
|
||||
@Override
|
||||
public void loadModel(QualityConfig config) {
|
||||
@@ -84,6 +87,11 @@ public class Seetaface6QualityModel implements FaceQualityModel {
|
||||
//加载依赖库
|
||||
NativeLoader.loadNativeLibraries(device);
|
||||
this.config = config;
|
||||
int predictorPoolSize = config.getPredictorPoolSize();
|
||||
if(config.getPredictorPoolSize() <= 0){
|
||||
predictorPoolSize = Runtime.getRuntime().availableProcessors(); // 默认等于CPU核心数
|
||||
}
|
||||
log.debug("模型推理器线程池最大数量: " + predictorPoolSize);
|
||||
log.debug("Loading seetaFace6 library successfully.");
|
||||
}
|
||||
|
||||
@@ -104,6 +112,7 @@ public class Seetaface6QualityModel implements FaceQualityModel {
|
||||
try {
|
||||
if(Objects.isNull(this.qualityOfBrightnessPool)){
|
||||
this.qualityOfBrightnessPool = new QualityOfBrightnessPool(new SeetaConfSetting());
|
||||
qualityOfBrightnessPool.setMaxTotal(predictorPoolSize);
|
||||
}
|
||||
qualityOfBrightness = qualityOfBrightnessPool.borrowObject();
|
||||
SeetaImageData imageData = new SeetaImageData(image.getWidth(), image.getHeight(), 3);
|
||||
@@ -170,6 +179,7 @@ public class Seetaface6QualityModel implements FaceQualityModel {
|
||||
try {
|
||||
if(Objects.isNull(this.qualityOfClarityPool)){
|
||||
this.qualityOfClarityPool = new QualityOfClarityPool(new SeetaConfSetting());
|
||||
qualityOfClarityPool.setMaxTotal(predictorPoolSize);
|
||||
}
|
||||
qualityOfClarity = qualityOfClarityPool.borrowObject();
|
||||
SeetaImageData imageData = new SeetaImageData(image.getWidth(), image.getHeight(), 3);
|
||||
@@ -236,6 +246,7 @@ public class Seetaface6QualityModel implements FaceQualityModel {
|
||||
try {
|
||||
if(Objects.isNull(this.qualityOfIntegrityPool)){
|
||||
this.qualityOfIntegrityPool = new QualityOfIntegrityPool(new SeetaConfSetting());
|
||||
qualityOfIntegrityPool.setMaxTotal(predictorPoolSize);
|
||||
}
|
||||
qualityOfIntegrity = qualityOfIntegrityPool.borrowObject();
|
||||
SeetaImageData imageData = new SeetaImageData(image.getWidth(), image.getHeight(), 3);
|
||||
@@ -302,6 +313,7 @@ public class Seetaface6QualityModel implements FaceQualityModel {
|
||||
try {
|
||||
if(Objects.isNull(this.qualityOfPosePool)){
|
||||
this.qualityOfPosePool = new QualityOfPosePool(new SeetaConfSetting());
|
||||
qualityOfPosePool.setMaxTotal(predictorPoolSize);
|
||||
}
|
||||
qualityOfPose = qualityOfPosePool.borrowObject();
|
||||
SeetaImageData imageData = new SeetaImageData(image.getWidth(), image.getHeight(), 3);
|
||||
@@ -368,6 +380,7 @@ public class Seetaface6QualityModel implements FaceQualityModel {
|
||||
try {
|
||||
if(Objects.isNull(this.qualityOfResolutionPool)){
|
||||
this.qualityOfResolutionPool = new QualityOfResolutionPool(new SeetaConfSetting());
|
||||
qualityOfResolutionPool.setMaxTotal(predictorPoolSize);
|
||||
}
|
||||
qualityOfResolution = qualityOfResolutionPool.borrowObject();
|
||||
SeetaImageData imageData = new SeetaImageData(image.getWidth(), image.getHeight(), 3);
|
||||
@@ -438,6 +451,7 @@ public class Seetaface6QualityModel implements FaceQualityModel {
|
||||
}
|
||||
SeetaConfSetting setting = getClarityMLSetting();
|
||||
this.qualityOfLBNPool = new QualityOfLBNPool(setting);
|
||||
qualityOfLBNPool.setMaxTotal(predictorPoolSize);
|
||||
}
|
||||
qualityOfLBN = qualityOfLBNPool.borrowObject();
|
||||
SeetaImageData imageData = new SeetaImageData(image.getWidth(), image.getHeight(), 3);
|
||||
@@ -509,6 +523,7 @@ public class Seetaface6QualityModel implements FaceQualityModel {
|
||||
}
|
||||
SeetaConfSetting setting = getPoseMLSetting();
|
||||
this.qualityOfPoseExPool = new QualityOfPoseExPool(setting);
|
||||
qualityOfPoseExPool.setMaxTotal(predictorPoolSize);
|
||||
}
|
||||
qualityOfPoseEx = qualityOfPoseExPool.borrowObject();
|
||||
SeetaImageData imageData = new SeetaImageData(image.getWidth(), image.getHeight(), 3);
|
||||
@@ -628,18 +643,23 @@ public class Seetaface6QualityModel implements FaceQualityModel {
|
||||
try {
|
||||
if(Objects.isNull(this.qualityOfBrightnessPool)){
|
||||
this.qualityOfBrightnessPool = new QualityOfBrightnessPool(new SeetaConfSetting());
|
||||
qualityOfBrightnessPool.setMaxTotal(predictorPoolSize);
|
||||
}
|
||||
if(Objects.isNull(this.qualityOfClarityPool)){
|
||||
this.qualityOfClarityPool = new QualityOfClarityPool(new SeetaConfSetting());
|
||||
qualityOfClarityPool.setMaxTotal(predictorPoolSize);
|
||||
}
|
||||
if(Objects.isNull(this.qualityOfIntegrityPool)){
|
||||
this.qualityOfIntegrityPool = new QualityOfIntegrityPool(new SeetaConfSetting());
|
||||
qualityOfIntegrityPool.setMaxTotal(predictorPoolSize);
|
||||
}
|
||||
if(Objects.isNull(this.qualityOfPosePool)){
|
||||
this.qualityOfPosePool = new QualityOfPosePool(new SeetaConfSetting());
|
||||
qualityOfPosePool.setMaxTotal(predictorPoolSize);
|
||||
}
|
||||
if(Objects.isNull(this.qualityOfResolutionPool)){
|
||||
this.qualityOfResolutionPool = new QualityOfResolutionPool(new SeetaConfSetting());
|
||||
qualityOfResolutionPool.setMaxTotal(predictorPoolSize);
|
||||
}
|
||||
FaceQualitySummary summary = new FaceQualitySummary();
|
||||
qualityOfBrightness = qualityOfBrightnessPool.borrowObject();
|
||||
@@ -690,6 +710,34 @@ public class Seetaface6QualityModel implements FaceQualityModel {
|
||||
}
|
||||
}
|
||||
|
||||
public QualityOfBrightnessPool getQualityOfBrightnessPool() {
|
||||
return qualityOfBrightnessPool;
|
||||
}
|
||||
|
||||
public QualityOfClarityPool getQualityOfClarityPool() {
|
||||
return qualityOfClarityPool;
|
||||
}
|
||||
|
||||
public QualityOfLBNPool getQualityOfLBNPool() {
|
||||
return qualityOfLBNPool;
|
||||
}
|
||||
|
||||
public QualityOfIntegrityPool getQualityOfIntegrityPool() {
|
||||
return qualityOfIntegrityPool;
|
||||
}
|
||||
|
||||
public QualityOfPosePool getQualityOfPosePool() {
|
||||
return qualityOfPosePool;
|
||||
}
|
||||
|
||||
public QualityOfPoseExPool getQualityOfPoseExPool() {
|
||||
return qualityOfPoseExPool;
|
||||
}
|
||||
|
||||
public QualityOfResolutionPool getQualityOfResolutionPool() {
|
||||
return qualityOfResolutionPool;
|
||||
}
|
||||
|
||||
@Override
|
||||
public void close() throws Exception {
|
||||
if(Objects.nonNull(qualityOfBrightnessPool)){
|
||||
|
||||
@@ -60,15 +60,13 @@ public class DJLImagePreprocessor {
|
||||
// 处理流程
|
||||
public Image process() {
|
||||
Image result = image;
|
||||
|
||||
if (enableCrop) {
|
||||
result = result.getSubImage(cropRect.x, cropRect.y, cropRect.width, cropRect.height);
|
||||
}
|
||||
|
||||
if (enableAffine) {
|
||||
if(enableAffine){
|
||||
result = warpAffine(keyPoints, affineTargetWidth, affineTargetHeight);
|
||||
}else {
|
||||
if(enableCrop){
|
||||
result = result.getSubImage(cropRect.x, cropRect.y, cropRect.width, cropRect.height);
|
||||
}
|
||||
}
|
||||
|
||||
return result;
|
||||
}
|
||||
|
||||
|
||||
@@ -62,6 +62,16 @@ public class MilvusConfig extends VectorDBConfig {
|
||||
*/
|
||||
private boolean useMemoryCache = true;
|
||||
|
||||
/**
|
||||
* 用户名
|
||||
*/
|
||||
private String username;
|
||||
|
||||
/**
|
||||
* 密码
|
||||
*/
|
||||
private String password;
|
||||
|
||||
/**
|
||||
* 构造函数
|
||||
*/
|
||||
|
||||
@@ -49,10 +49,13 @@ public class MilvusClient implements VectorDBClient {
|
||||
@Override
|
||||
public void initialize() {
|
||||
try {
|
||||
ConnectParam connectParam = ConnectParam.newBuilder()
|
||||
ConnectParam.Builder builder = ConnectParam.newBuilder()
|
||||
.withHost(config.getHost())
|
||||
.withPort(config.getPort())
|
||||
.build();
|
||||
.withPort(config.getPort());
|
||||
if (StringUtils.isNotBlank(config.getUsername()) && StringUtils.isNotBlank(config.getPassword())) {
|
||||
builder.withAuthorization(config.getUsername(), config.getPassword());
|
||||
}
|
||||
ConnectParam connectParam = builder.build();
|
||||
serviceClient = new MilvusServiceClient(connectParam);
|
||||
collectionName = StringUtils.isNotBlank(config.getCollectionName()) ? config.getCollectionName() : VectorDBConstants.Defaults.DEFAULT_COLLECTION_NAME;
|
||||
createCollection(collectionName, config.getDimension());
|
||||
|
||||
@@ -6,11 +6,11 @@
|
||||
<parent>
|
||||
<groupId>cn.smartjavaai</groupId>
|
||||
<artifactId>smartjavaai-parent</artifactId>
|
||||
<version>1.0.20</version>
|
||||
<version>1.0.22</version>
|
||||
</parent>
|
||||
|
||||
<artifactId>smartjavaai-objectdetection</artifactId>
|
||||
<version>1.0.20</version>
|
||||
<version>1.0.22</version>
|
||||
<name>smartjavaai-objectdetection</name>
|
||||
<description>SmartJavaAI</description>
|
||||
<url>https://github.com/geekwenjie/SmartJavaAI</url>
|
||||
|
||||
@@ -1,11 +1,13 @@
|
||||
package cn.smartjavaai.objectdetection.config;
|
||||
|
||||
import cn.smartjavaai.common.config.ModelConfig;
|
||||
import cn.smartjavaai.common.enums.DeviceEnum;
|
||||
import cn.smartjavaai.objectdetection.constant.DetectorConstant;
|
||||
import cn.smartjavaai.objectdetection.enums.DetectorModelEnum;
|
||||
import lombok.Data;
|
||||
|
||||
import java.util.HashMap;
|
||||
import java.util.List;
|
||||
import java.util.Map;
|
||||
|
||||
/**
|
||||
@@ -15,7 +17,7 @@ import java.util.Map;
|
||||
* @date 2025/4/4
|
||||
*/
|
||||
@Data
|
||||
public class DetectorModelConfig {
|
||||
public class DetectorModelConfig extends ModelConfig {
|
||||
|
||||
/**
|
||||
* 模型
|
||||
@@ -27,10 +29,6 @@ public class DetectorModelConfig {
|
||||
*/
|
||||
private float threshold = DetectorConstant.DEFAULT_THRESHOLD;
|
||||
|
||||
/**
|
||||
* 设备类型
|
||||
*/
|
||||
private DeviceEnum device;
|
||||
|
||||
/**
|
||||
* 模型路径
|
||||
@@ -45,35 +43,27 @@ public class DetectorModelConfig {
|
||||
private int maxBox;
|
||||
|
||||
/**
|
||||
* 个性化配置
|
||||
* 允许的类列表
|
||||
*/
|
||||
private Map<String, Object> customParams = new HashMap<>();
|
||||
private List<String> allowedClasses;
|
||||
|
||||
/**
|
||||
* 检测结果数量
|
||||
*/
|
||||
private int topK;
|
||||
|
||||
|
||||
public DetectorModelConfig() {
|
||||
}
|
||||
|
||||
public DetectorModelConfig(DetectorModelEnum modelEnum, DeviceEnum device) {
|
||||
this.modelEnum = modelEnum;
|
||||
this.device = device;
|
||||
setDevice(device);
|
||||
}
|
||||
|
||||
public DetectorModelConfig(DetectorModelEnum modelEnum) {
|
||||
this.modelEnum = modelEnum;
|
||||
}
|
||||
|
||||
public <T> T getCustomParam(String key, Class<T> clazz) {
|
||||
Object value = customParams.get(key);
|
||||
if (value == null) return null;
|
||||
return clazz.cast(value);
|
||||
}
|
||||
|
||||
/**
|
||||
* 添加个性化配置项
|
||||
*/
|
||||
public void putCustomParam(String key, Object value) {
|
||||
if (customParams == null) {
|
||||
customParams = new HashMap<>();
|
||||
}
|
||||
customParams.put(key, value);
|
||||
}
|
||||
}
|
||||
|
||||
@@ -25,7 +25,7 @@ public class DJLModelCriteriaBuilder implements CriteriaBuilderStrategy {
|
||||
public Criteria<Image, DetectedObjects> buildCriteria(DetectorModelConfig config) {
|
||||
Device device = null;
|
||||
if(!Objects.isNull(config.getDevice())){
|
||||
device = config.getDevice() == DeviceEnum.CPU ? Device.cpu() : Device.gpu();
|
||||
device = config.getDevice() == DeviceEnum.CPU ? Device.cpu() : Device.gpu(config.getGpuId());
|
||||
}
|
||||
Criteria<Image, DetectedObjects> criteria = Criteria.builder()
|
||||
.optApplication(Application.CV.OBJECT_DETECTION)
|
||||
|
||||
@@ -25,7 +25,7 @@ public class YoloCriteriaBuilder implements CriteriaBuilderStrategy {
|
||||
public Criteria<Image, DetectedObjects> buildCriteria(DetectorModelConfig config) {
|
||||
Device device = null;
|
||||
if(!Objects.isNull(config.getDevice())){
|
||||
device = config.getDevice() == DeviceEnum.CPU ? Device.cpu() : Device.gpu();
|
||||
device = config.getDevice() == DeviceEnum.CPU ? Device.cpu() : Device.gpu(config.getGpuId());
|
||||
}
|
||||
|
||||
Map<String, Object> customParams = getDefaultConfig();
|
||||
|
||||
@@ -1,16 +1,22 @@
|
||||
package cn.smartjavaai.objectdetection.model;
|
||||
|
||||
import ai.djl.MalformedModelException;
|
||||
import ai.djl.engine.Engine;
|
||||
import ai.djl.inference.Predictor;
|
||||
import ai.djl.modality.cv.Image;
|
||||
import ai.djl.modality.cv.ImageFactory;
|
||||
import ai.djl.modality.cv.output.BoundingBox;
|
||||
import ai.djl.modality.cv.output.DetectedObjects;
|
||||
import ai.djl.modality.cv.translator.YoloV8TranslatorFactory;
|
||||
import ai.djl.repository.zoo.Criteria;
|
||||
import ai.djl.repository.zoo.ModelNotFoundException;
|
||||
import ai.djl.repository.zoo.ZooModel;
|
||||
import ai.djl.training.util.ProgressBar;
|
||||
import cn.smartjavaai.common.entity.DetectionInfo;
|
||||
import cn.smartjavaai.common.entity.DetectionRectangle;
|
||||
import cn.smartjavaai.common.entity.DetectionResponse;
|
||||
import cn.smartjavaai.common.entity.Point;
|
||||
import cn.smartjavaai.common.entity.face.FaceInfo;
|
||||
import cn.smartjavaai.common.pool.PredictorFactory;
|
||||
import cn.smartjavaai.common.utils.FileUtils;
|
||||
import cn.smartjavaai.common.utils.ImageUtils;
|
||||
@@ -21,39 +27,53 @@ import cn.smartjavaai.objectdetection.criteria.CriteriaBuilderFactory;
|
||||
import cn.smartjavaai.objectdetection.exception.DetectionException;
|
||||
import cn.smartjavaai.objectdetection.utils.DetectorUtils;
|
||||
import lombok.extern.slf4j.Slf4j;
|
||||
import org.apache.commons.collections.CollectionUtils;
|
||||
import org.apache.commons.lang3.StringUtils;
|
||||
import org.apache.commons.pool2.ObjectPool;
|
||||
import org.apache.commons.pool2.impl.GenericObjectPool;
|
||||
import org.opencv.core.Mat;
|
||||
|
||||
import javax.imageio.ImageIO;
|
||||
import java.awt.image.BufferedImage;
|
||||
import java.io.*;
|
||||
import java.nio.file.Paths;
|
||||
import java.util.ArrayList;
|
||||
import java.util.Iterator;
|
||||
import java.util.List;
|
||||
import java.util.Objects;
|
||||
import java.util.stream.Collectors;
|
||||
|
||||
/**
|
||||
* 目标检测模型
|
||||
* @author dwj
|
||||
* @date 2025/4/4
|
||||
*/
|
||||
@Slf4j
|
||||
public class DetectorModel implements AutoCloseable{
|
||||
|
||||
private ZooModel<Image, DetectedObjects> model;
|
||||
|
||||
private ObjectPool<Predictor<Image, DetectedObjects>> predictorPool;
|
||||
private GenericObjectPool<Predictor<Image, DetectedObjects>> predictorPool;
|
||||
|
||||
private DetectorModelConfig config;
|
||||
|
||||
public void loadModel(DetectorModelConfig config){
|
||||
if(Objects.isNull(config.getModelEnum())){
|
||||
throw new DetectionException("未配置模型枚举");
|
||||
}
|
||||
Criteria<Image, DetectedObjects> criteria = CriteriaBuilderFactory.createCriteria(config);
|
||||
|
||||
this.config = config;
|
||||
try {
|
||||
model = criteria.loadModel();
|
||||
// 创建池子:每个线程独享 Predictor
|
||||
this.predictorPool = new GenericObjectPool<>(new PredictorFactory<>(model));
|
||||
int predictorPoolSize = config.getPredictorPoolSize();
|
||||
if(config.getPredictorPoolSize() <= 0){
|
||||
predictorPoolSize = Runtime.getRuntime().availableProcessors(); // 默认等于CPU核心数
|
||||
}
|
||||
predictorPool.setMaxTotal(predictorPoolSize);
|
||||
log.debug("当前设备: " + model.getNDManager().getDevice());
|
||||
log.debug("当前引擎: " + Engine.getInstance().getEngineName());
|
||||
log.debug("模型推理器线程池最大数量: " + predictorPoolSize);
|
||||
} catch (IOException | ModelNotFoundException | MalformedModelException e) {
|
||||
throw new DetectionException("模型加载失败", e);
|
||||
}
|
||||
@@ -72,11 +92,16 @@ public class DetectorModel implements AutoCloseable{
|
||||
Image image = null;
|
||||
try {
|
||||
image = ImageFactory.getInstance().fromFile(Paths.get(imagePath));
|
||||
} catch (IOException e) {
|
||||
throw new DetectionException("图片转换错误", e);
|
||||
DetectedObjects detectedObjects = detect(image);
|
||||
return DetectorUtils.convertToDetectionResponse(detectedObjects, image);
|
||||
} catch (Exception e) {
|
||||
throw new DetectionException(e);
|
||||
} finally {
|
||||
if (image != null){
|
||||
((Mat)image.getWrappedImage()).release();
|
||||
}
|
||||
}
|
||||
DetectedObjects detectedObjects = detect(image);
|
||||
return DetectorUtils.convertToDetectionResponse(detectedObjects, image);
|
||||
|
||||
}
|
||||
|
||||
|
||||
@@ -89,8 +114,9 @@ public class DetectorModel implements AutoCloseable{
|
||||
if(!FileUtils.isFileExists(imagePath)){
|
||||
throw new DetectionException("图像文件不存在");
|
||||
}
|
||||
Image img = null;
|
||||
try {
|
||||
Image img = ImageFactory.getInstance().fromFile(Paths.get(imagePath));
|
||||
img = ImageFactory.getInstance().fromFile(Paths.get(imagePath));
|
||||
DetectedObjects detectedObjects = detect(img);
|
||||
img.drawBoundingBoxes(detectedObjects);
|
||||
ByteArrayOutputStream outputStream = new ByteArrayOutputStream();
|
||||
@@ -98,6 +124,10 @@ public class DetectorModel implements AutoCloseable{
|
||||
img.save(new FileOutputStream(Paths.get(outputPath).toAbsolutePath().toString()), "png");
|
||||
} catch (IOException e) {
|
||||
throw new DetectionException(e);
|
||||
} finally {
|
||||
if (img != null){
|
||||
((Mat)img.getWrappedImage()).release();
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
@@ -129,9 +159,19 @@ public class DetectorModel implements AutoCloseable{
|
||||
if(!ImageUtils.isImageValid(image)){
|
||||
throw new DetectionException("图像无效");
|
||||
}
|
||||
Image img = ImageFactory.getInstance().fromImage(OpenCVUtils.image2Mat(image));
|
||||
DetectedObjects detectedObjects = detect(img);
|
||||
return DetectorUtils.convertToDetectionResponse(detectedObjects, img);
|
||||
Image img = null;
|
||||
try {
|
||||
img = ImageFactory.getInstance().fromImage(OpenCVUtils.image2Mat(image));
|
||||
DetectedObjects detectedObjects = detect(img);
|
||||
return DetectorUtils.convertToDetectionResponse(detectedObjects, img);
|
||||
} catch (Exception e) {
|
||||
throw new DetectionException(e);
|
||||
} finally {
|
||||
if (img != null) {
|
||||
((Mat)img.getWrappedImage()).release();
|
||||
}
|
||||
}
|
||||
|
||||
}
|
||||
|
||||
/**
|
||||
@@ -155,6 +195,10 @@ public class DetectorModel implements AutoCloseable{
|
||||
return ImageIO.read(new ByteArrayInputStream(imageBytes));
|
||||
} catch (IOException e) {
|
||||
throw new DetectionException("导出图片失败", e);
|
||||
} finally {
|
||||
if (img != null) {
|
||||
((Mat)img.getWrappedImage()).release();
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
@@ -163,11 +207,13 @@ public class DetectorModel implements AutoCloseable{
|
||||
* @param image
|
||||
* @return
|
||||
*/
|
||||
private DetectedObjects detect(Image image){
|
||||
public DetectedObjects detect(Image image){
|
||||
Predictor<Image, DetectedObjects> predictor = null;
|
||||
try {
|
||||
predictor = predictorPool.borrowObject();
|
||||
return predictor.predict(image);
|
||||
DetectedObjects detectedObjects = predictor.predict(image);
|
||||
detectedObjects = filterDetections(detectedObjects);
|
||||
return detectedObjects;
|
||||
} catch (Exception e) {
|
||||
throw new DetectionException("目标检测错误", e);
|
||||
}finally {
|
||||
@@ -187,6 +233,50 @@ public class DetectorModel implements AutoCloseable{
|
||||
}
|
||||
}
|
||||
|
||||
/**
|
||||
* 筛选检测结果
|
||||
* @param detectedObjects
|
||||
* @return
|
||||
*/
|
||||
private DetectedObjects filterDetections(DetectedObjects detectedObjects) {
|
||||
if (Objects.isNull(detectedObjects) || detectedObjects.getNumberOfObjects() == 0) {
|
||||
return detectedObjects;
|
||||
}
|
||||
List<DetectedObjects.DetectedObject> items = detectedObjects.items();
|
||||
// 按照允许的类别进行过滤
|
||||
List<DetectedObjects.DetectedObject> filtered = new ArrayList<>();
|
||||
//过滤类别
|
||||
if(!CollectionUtils.isEmpty(config.getAllowedClasses())){
|
||||
for (DetectedObjects.DetectedObject obj : items) {
|
||||
if(config.getAllowedClasses().contains(obj.getClassName())){
|
||||
filtered.add(obj);
|
||||
}
|
||||
}
|
||||
}else{
|
||||
filtered = items;
|
||||
}
|
||||
// 按照概率进行排序
|
||||
filtered.sort((o1, o2) -> Double.compare(o2.getProbability(), o1.getProbability()));
|
||||
if(config.getTopK() > 0 && filtered.size() > config.getTopK()){
|
||||
filtered = filtered.subList(0, config.getTopK());
|
||||
}
|
||||
// 构建新的 DetectedObjects 返回
|
||||
List<String> names = new ArrayList<>();
|
||||
List<Double> probs = new ArrayList<>();
|
||||
List<BoundingBox> boxes = new ArrayList<>();
|
||||
|
||||
for (DetectedObjects.DetectedObject obj : filtered) {
|
||||
names.add(obj.getClassName());
|
||||
probs.add(obj.getProbability());
|
||||
boxes.add(obj.getBoundingBox());
|
||||
}
|
||||
return new DetectedObjects(names, probs, boxes);
|
||||
}
|
||||
|
||||
public GenericObjectPool<Predictor<Image, DetectedObjects>> getPool() {
|
||||
return predictorPool;
|
||||
}
|
||||
|
||||
|
||||
/**
|
||||
* 显式释放资源
|
||||
|
||||
@@ -6,7 +6,7 @@
|
||||
<parent>
|
||||
<groupId>cn.smartjavaai</groupId>
|
||||
<artifactId>smartjavaai-parent</artifactId>
|
||||
<version>1.0.20</version>
|
||||
<version>1.0.22</version>
|
||||
</parent>
|
||||
|
||||
<artifactId>smartjavaai-ocr</artifactId>
|
||||
@@ -42,7 +42,7 @@
|
||||
</dependency>
|
||||
</dependencies>
|
||||
|
||||
<version>1.0.20</version>
|
||||
<version>1.0.22</version>
|
||||
<name>smartjavaai-ocr</name>
|
||||
<description>SmartJavaAI</description>
|
||||
<url>https://github.com/geekwenjie/SmartJavaAI</url>
|
||||
|
||||
@@ -0,0 +1,41 @@
|
||||
package cn.smartjavaai.ocr.config;
|
||||
|
||||
import cn.smartjavaai.common.config.ModelConfig;
|
||||
import cn.smartjavaai.ocr.enums.CommonDetModelEnum;
|
||||
import cn.smartjavaai.ocr.enums.PlateDetModelEnum;
|
||||
import lombok.Data;
|
||||
|
||||
/**
|
||||
* 车牌检测模型配置
|
||||
* @author dwj
|
||||
*/
|
||||
@Data
|
||||
public class PlateDetModelConfig extends ModelConfig {
|
||||
|
||||
/**
|
||||
* 模型
|
||||
*/
|
||||
private PlateDetModelEnum modelEnum;
|
||||
|
||||
/**
|
||||
* 检测模型路径
|
||||
*/
|
||||
private String modelPath;
|
||||
|
||||
/**
|
||||
* 置信度阈值
|
||||
*/
|
||||
private float confidenceThreshold;
|
||||
|
||||
/**
|
||||
* iou阈值
|
||||
*/
|
||||
private float iouThreshold;
|
||||
|
||||
/**
|
||||
* 检测结果数量
|
||||
*/
|
||||
private int topK;
|
||||
|
||||
|
||||
}
|
||||
@@ -0,0 +1,47 @@
|
||||
package cn.smartjavaai.ocr.config;
|
||||
|
||||
import cn.smartjavaai.common.config.ModelConfig;
|
||||
import cn.smartjavaai.ocr.enums.PlateDetModelEnum;
|
||||
import cn.smartjavaai.ocr.enums.PlateRecModelEnum;
|
||||
import cn.smartjavaai.ocr.model.plate.PlateDetModel;
|
||||
import lombok.Data;
|
||||
|
||||
/**
|
||||
* 车牌识别模型配置
|
||||
* @author dwj
|
||||
*/
|
||||
@Data
|
||||
public class PlateRecModelConfig extends ModelConfig {
|
||||
|
||||
/**
|
||||
* 模型
|
||||
*/
|
||||
private PlateRecModelEnum modelEnum;
|
||||
|
||||
/**
|
||||
* 检测模型路径
|
||||
*/
|
||||
private String modelPath;
|
||||
|
||||
/**
|
||||
* 车牌检测模型
|
||||
*/
|
||||
private PlateDetModel plateDetModel;
|
||||
|
||||
/**
|
||||
* 置信度阈值
|
||||
*/
|
||||
private float confidenceThreshold;
|
||||
|
||||
/**
|
||||
* iou阈值
|
||||
*/
|
||||
private float iouThreshold;
|
||||
|
||||
/**
|
||||
* 检测结果数量
|
||||
*/
|
||||
private int topK;
|
||||
|
||||
|
||||
}
|
||||
@@ -0,0 +1,45 @@
|
||||
package cn.smartjavaai.ocr.entity;
|
||||
|
||||
import cn.smartjavaai.common.entity.DetectionRectangle;
|
||||
import cn.smartjavaai.ocr.enums.PlateType;
|
||||
import lombok.Data;
|
||||
|
||||
/**
|
||||
* 车牌识别信息
|
||||
* @author dwj
|
||||
*/
|
||||
@Data
|
||||
public class PlateInfo {
|
||||
|
||||
/**
|
||||
* 车牌类型
|
||||
*/
|
||||
private PlateType plateType;
|
||||
|
||||
/**
|
||||
* 车牌号码
|
||||
*/
|
||||
private String plateNumber;
|
||||
|
||||
/**
|
||||
* 车牌颜色
|
||||
*/
|
||||
private String plateColor;
|
||||
|
||||
/**
|
||||
* 检测位置信息
|
||||
*/
|
||||
private DetectionRectangle detectionRectangle;
|
||||
|
||||
/**
|
||||
* 车牌4角坐标
|
||||
*/
|
||||
private OcrBox box;
|
||||
|
||||
/**
|
||||
* 检测得分
|
||||
*/
|
||||
private float score;
|
||||
|
||||
|
||||
}
|
||||
@@ -0,0 +1,35 @@
|
||||
package cn.smartjavaai.ocr.entity;
|
||||
|
||||
import lombok.Data;
|
||||
|
||||
/**
|
||||
* @author dwj
|
||||
*/
|
||||
@Data
|
||||
public class PlateResult {
|
||||
|
||||
/**
|
||||
* 车牌号码
|
||||
*/
|
||||
private String plateNo;
|
||||
|
||||
/**
|
||||
* 车牌颜色
|
||||
*/
|
||||
private String plateColor;
|
||||
|
||||
public PlateResult(String plateNo, String plateColor) {
|
||||
this.plateNo = plateNo;
|
||||
this.plateColor = plateColor;
|
||||
}
|
||||
|
||||
|
||||
@Override
|
||||
public String toString() {
|
||||
return "PlateResult{" +
|
||||
"plateNo='" + plateNo + '\'' +
|
||||
", plateColor='" + plateColor + '\'' +
|
||||
'}';
|
||||
}
|
||||
}
|
||||
|
||||
@@ -0,0 +1,28 @@
|
||||
package cn.smartjavaai.ocr.enums;
|
||||
|
||||
/**
|
||||
* 车牌检测模型枚举
|
||||
* @author dwj
|
||||
*/
|
||||
public enum PlateDetModelEnum {
|
||||
|
||||
YOLOV5,
|
||||
|
||||
YOLOV7;
|
||||
|
||||
|
||||
/**
|
||||
* 根据名称获取枚举 (忽略大小写和下划线变体)
|
||||
*/
|
||||
public static PlateDetModelEnum fromName(String name) {
|
||||
String formatted = name.trim().toUpperCase().replaceAll("[-_]", "");
|
||||
for (PlateDetModelEnum model : values()) {
|
||||
if (model.name().replaceAll("_", "").equals(formatted)) {
|
||||
return model;
|
||||
}
|
||||
}
|
||||
throw new IllegalArgumentException("未知模型名称: " + name);
|
||||
}
|
||||
|
||||
|
||||
}
|
||||
@@ -0,0 +1,26 @@
|
||||
package cn.smartjavaai.ocr.enums;
|
||||
|
||||
/**
|
||||
* 车牌识别模型枚举
|
||||
* @author dwj
|
||||
*/
|
||||
public enum PlateRecModelEnum {
|
||||
|
||||
PLATE_REC_CRNN;
|
||||
|
||||
|
||||
/**
|
||||
* 根据名称获取枚举 (忽略大小写和下划线变体)
|
||||
*/
|
||||
public static PlateRecModelEnum fromName(String name) {
|
||||
String formatted = name.trim().toUpperCase().replaceAll("[-_]", "");
|
||||
for (PlateRecModelEnum model : values()) {
|
||||
if (model.name().replaceAll("_", "").equals(formatted)) {
|
||||
return model;
|
||||
}
|
||||
}
|
||||
throw new IllegalArgumentException("未知模型名称: " + name);
|
||||
}
|
||||
|
||||
|
||||
}
|
||||
@@ -0,0 +1,45 @@
|
||||
package cn.smartjavaai.ocr.enums;
|
||||
|
||||
/**
|
||||
* @author dwj
|
||||
*/
|
||||
public enum PlateType {
|
||||
|
||||
SINGLE("single", "单层"),
|
||||
DOUBLE("double", "双层"),
|
||||
UNKNOWN("unknown", "未知");
|
||||
|
||||
private final String className;
|
||||
private final String description;
|
||||
|
||||
PlateType(String className, String description) {
|
||||
this.className = className;
|
||||
this.description = description;
|
||||
}
|
||||
|
||||
public String getClassName() {
|
||||
return className;
|
||||
}
|
||||
|
||||
public String getDescription() {
|
||||
return description;
|
||||
}
|
||||
|
||||
|
||||
|
||||
|
||||
/**
|
||||
* 根据value获取对应的PlateType
|
||||
* @param className
|
||||
* @return PlateType
|
||||
*/
|
||||
public static PlateType fromClassName(String className) {
|
||||
for (PlateType type : values()) {
|
||||
if (type.className.equals(className)) {
|
||||
return type;
|
||||
}
|
||||
}
|
||||
return null;
|
||||
}
|
||||
|
||||
}
|
||||
@@ -0,0 +1,168 @@
|
||||
package cn.smartjavaai.ocr.factory;
|
||||
|
||||
import cn.smartjavaai.common.config.Config;
|
||||
import cn.smartjavaai.ocr.config.PlateDetModelConfig;
|
||||
import cn.smartjavaai.ocr.config.PlateRecModelConfig;
|
||||
import cn.smartjavaai.ocr.config.TableStructureConfig;
|
||||
import cn.smartjavaai.ocr.enums.PlateDetModelEnum;
|
||||
import cn.smartjavaai.ocr.enums.PlateRecModelEnum;
|
||||
import cn.smartjavaai.ocr.enums.TableStructureModelEnum;
|
||||
import cn.smartjavaai.ocr.exception.OcrException;
|
||||
import cn.smartjavaai.ocr.model.plate.CRNNPlateRecModel;
|
||||
import cn.smartjavaai.ocr.model.plate.PlateDetModel;
|
||||
import cn.smartjavaai.ocr.model.plate.PlateRecModel;
|
||||
import cn.smartjavaai.ocr.model.plate.Yolov5PlateDetModel;
|
||||
import cn.smartjavaai.ocr.model.table.CommonTableStructureModel;
|
||||
import cn.smartjavaai.ocr.model.table.TableStructureModel;
|
||||
import lombok.extern.slf4j.Slf4j;
|
||||
|
||||
import java.util.Map;
|
||||
import java.util.Objects;
|
||||
import java.util.concurrent.ConcurrentHashMap;
|
||||
|
||||
/**
|
||||
* 车牌识别模型工厂
|
||||
* @author dwj
|
||||
*/
|
||||
@Slf4j
|
||||
public class PlateModelFactory {
|
||||
|
||||
// 使用 volatile 和双重检查锁定来确保线程安全的单例模式
|
||||
private static volatile PlateModelFactory instance;
|
||||
|
||||
/**
|
||||
* 模型缓存
|
||||
*/
|
||||
private static final ConcurrentHashMap<PlateDetModelEnum, PlateDetModel> detModelMap = new ConcurrentHashMap<>();
|
||||
|
||||
/**
|
||||
* 模型缓存
|
||||
*/
|
||||
private static final ConcurrentHashMap<PlateRecModelEnum, PlateRecModel> recModelMap = new ConcurrentHashMap<>();
|
||||
|
||||
|
||||
/**
|
||||
* 模型注册表
|
||||
*/
|
||||
private static final Map<PlateDetModelEnum, Class<? extends PlateDetModel>> detModelRegistry =
|
||||
new ConcurrentHashMap<>();
|
||||
|
||||
/**
|
||||
* 模型注册表
|
||||
*/
|
||||
private static final Map<PlateRecModelEnum, Class<? extends PlateRecModel>> recModelRegistry =
|
||||
new ConcurrentHashMap<>();
|
||||
|
||||
|
||||
public static PlateModelFactory getInstance() {
|
||||
if (instance == null) {
|
||||
synchronized (PlateModelFactory.class) {
|
||||
if (instance == null) {
|
||||
instance = new PlateModelFactory();
|
||||
}
|
||||
}
|
||||
}
|
||||
return instance;
|
||||
}
|
||||
|
||||
|
||||
|
||||
/**
|
||||
* 注册模型
|
||||
* @param plateDetModelEnum
|
||||
* @param clazz
|
||||
*/
|
||||
private static void registerDetModel(PlateDetModelEnum plateDetModelEnum, Class<? extends PlateDetModel> clazz) {
|
||||
detModelRegistry.put(plateDetModelEnum, clazz);
|
||||
}
|
||||
|
||||
/**
|
||||
* 注册模型
|
||||
* @param plateRecModelEnum
|
||||
* @param clazz
|
||||
*/
|
||||
private static void registerRecModel(PlateRecModelEnum plateRecModelEnum, Class<? extends PlateRecModel> clazz) {
|
||||
recModelRegistry.put(plateRecModelEnum, clazz);
|
||||
}
|
||||
|
||||
|
||||
/**
|
||||
* 获取模型
|
||||
* @param config
|
||||
* @return
|
||||
*/
|
||||
public PlateDetModel getDetModel(PlateDetModelConfig config) {
|
||||
if(Objects.isNull(config) || Objects.isNull(config.getModelEnum())){
|
||||
throw new OcrException("未配置OCR模型");
|
||||
}
|
||||
return detModelMap.computeIfAbsent(config.getModelEnum(), k -> {
|
||||
return createDetModel(config);
|
||||
});
|
||||
}
|
||||
|
||||
/**
|
||||
* 获取模型
|
||||
* @param config
|
||||
* @return
|
||||
*/
|
||||
public PlateRecModel getRecModel(PlateRecModelConfig config) {
|
||||
if(Objects.isNull(config) || Objects.isNull(config.getModelEnum())){
|
||||
throw new OcrException("未配置OCR模型");
|
||||
}
|
||||
return recModelMap.computeIfAbsent(config.getModelEnum(), k -> {
|
||||
return createRecModel(config);
|
||||
});
|
||||
}
|
||||
|
||||
|
||||
|
||||
/**
|
||||
* 创建检测模型
|
||||
* @param config
|
||||
* @return
|
||||
*/
|
||||
private PlateDetModel createDetModel(PlateDetModelConfig config) {
|
||||
Class<?> clazz = detModelRegistry.get(config.getModelEnum());
|
||||
if(clazz == null){
|
||||
throw new OcrException("Unsupported model");
|
||||
}
|
||||
PlateDetModel model = null;
|
||||
try {
|
||||
model = (PlateDetModel) clazz.newInstance();
|
||||
} catch (InstantiationException | IllegalAccessException e) {
|
||||
throw new OcrException(e);
|
||||
}
|
||||
model.loadModel(config);
|
||||
return model;
|
||||
}
|
||||
|
||||
/**
|
||||
* 创建识别模型
|
||||
* @param config
|
||||
* @return
|
||||
*/
|
||||
private PlateRecModel createRecModel(PlateRecModelConfig config) {
|
||||
Class<?> clazz = recModelRegistry.get(config.getModelEnum());
|
||||
if(clazz == null){
|
||||
throw new OcrException("Unsupported model");
|
||||
}
|
||||
PlateRecModel model = null;
|
||||
try {
|
||||
model = (PlateRecModel) clazz.newInstance();
|
||||
} catch (InstantiationException | IllegalAccessException e) {
|
||||
throw new OcrException(e);
|
||||
}
|
||||
model.loadModel(config);
|
||||
return model;
|
||||
}
|
||||
|
||||
|
||||
// 初始化默认算法
|
||||
static {
|
||||
registerDetModel(PlateDetModelEnum.YOLOV5, Yolov5PlateDetModel.class);
|
||||
registerDetModel(PlateDetModelEnum.YOLOV7, Yolov5PlateDetModel.class);
|
||||
registerRecModel(PlateRecModelEnum.PLATE_REC_CRNN, CRNNPlateRecModel.class);
|
||||
log.debug("缓存目录:{}", Config.getCachePath());
|
||||
}
|
||||
|
||||
}
|
||||
@@ -1,8 +1,11 @@
|
||||
package cn.smartjavaai.ocr.model.common.detect;
|
||||
|
||||
import ai.djl.inference.Predictor;
|
||||
import ai.djl.modality.cv.Image;
|
||||
import ai.djl.ndarray.NDList;
|
||||
import cn.smartjavaai.ocr.config.OcrDetModelConfig;
|
||||
import cn.smartjavaai.ocr.entity.OcrBox;
|
||||
import org.apache.commons.pool2.impl.GenericObjectPool;
|
||||
|
||||
import java.awt.image.BufferedImage;
|
||||
import java.util.List;
|
||||
@@ -93,5 +96,9 @@ public interface OcrCommonDetModel extends AutoCloseable{
|
||||
throw new UnsupportedOperationException("默认不支持该功能");
|
||||
}
|
||||
|
||||
default GenericObjectPool<Predictor<Image, NDList>> getPool(){
|
||||
throw new UnsupportedOperationException("默认不支持该功能");
|
||||
}
|
||||
|
||||
|
||||
}
|
||||
|
||||
@@ -43,7 +43,7 @@ import java.util.*;
|
||||
@Slf4j
|
||||
public class OcrCommonDetModelImpl implements OcrCommonDetModel{
|
||||
|
||||
private ObjectPool<Predictor<Image, NDList>> detPredictorPool;
|
||||
private GenericObjectPool<Predictor<Image, NDList>> detPredictorPool;
|
||||
|
||||
private ZooModel<Image, NDList> detectionModel;
|
||||
|
||||
@@ -61,8 +61,14 @@ public class OcrCommonDetModelImpl implements OcrCommonDetModel{
|
||||
detectionModel = ModelZoo.loadModel(detCriteria);
|
||||
// 创建池子:每个线程独享 Predictor
|
||||
this.detPredictorPool = new GenericObjectPool<>(new PredictorFactory<>(detectionModel));
|
||||
int predictorPoolSize = config.getPredictorPoolSize();
|
||||
if(config.getPredictorPoolSize() <= 0){
|
||||
predictorPoolSize = Runtime.getRuntime().availableProcessors(); // 默认等于CPU核心数
|
||||
}
|
||||
detPredictorPool.setMaxTotal(predictorPoolSize);
|
||||
log.debug("当前设备: " + detectionModel.getNDManager().getDevice());
|
||||
log.debug("当前引擎: " + Engine.getInstance().getEngineName());
|
||||
log.debug("模型推理器线程池最大数量: " + predictorPoolSize);
|
||||
} catch (IOException | ModelNotFoundException | MalformedModelException e) {
|
||||
throw new OcrException("检测模型加载失败", e);
|
||||
}
|
||||
@@ -209,6 +215,11 @@ public class OcrCommonDetModelImpl implements OcrCommonDetModel{
|
||||
}
|
||||
}
|
||||
|
||||
@Override
|
||||
public GenericObjectPool<Predictor<Image, NDList>> getPool() {
|
||||
return detPredictorPool;
|
||||
}
|
||||
|
||||
@Override
|
||||
public void close() throws Exception {
|
||||
try {
|
||||
|
||||
@@ -25,7 +25,7 @@ public class OcrCommonDetCriterialFactory {
|
||||
public static Criteria<Image, NDList> createCriteria(OcrDetModelConfig config) {
|
||||
Device device = null;
|
||||
if(!Objects.isNull(config.getDevice())){
|
||||
device = config.getDevice() == DeviceEnum.CPU ? Device.cpu() : Device.gpu();
|
||||
device = config.getDevice() == DeviceEnum.CPU ? Device.cpu() : Device.gpu(config.getGpuId());
|
||||
}
|
||||
Criteria<Image, NDList> criteria = null;
|
||||
ConcurrentHashMap params = new ConcurrentHashMap<String, String>();
|
||||
|
||||
@@ -9,6 +9,7 @@ import cn.smartjavaai.ocr.entity.OcrBox;
|
||||
import cn.smartjavaai.ocr.entity.OcrInfo;
|
||||
import cn.smartjavaai.ocr.entity.OcrItem;
|
||||
import cn.smartjavaai.ocr.model.common.detect.OcrCommonDetModel;
|
||||
import org.apache.commons.pool2.impl.GenericObjectPool;
|
||||
import org.opencv.core.Mat;
|
||||
|
||||
import java.awt.image.BufferedImage;
|
||||
@@ -106,4 +107,8 @@ public interface OcrDirectionModel extends AutoCloseable{
|
||||
throw new UnsupportedOperationException("默认不支持该功能");
|
||||
}
|
||||
|
||||
default GenericObjectPool<Predictor<Image, DirectionInfo>> getPool() {
|
||||
throw new UnsupportedOperationException("默认不支持该功能");
|
||||
}
|
||||
|
||||
}
|
||||
|
||||
@@ -55,7 +55,7 @@ import java.util.concurrent.ConcurrentHashMap;
|
||||
public class PPOCRMobileV2ClsModel implements OcrDirectionModel {
|
||||
|
||||
|
||||
private ObjectPool<Predictor<Image, DirectionInfo>> predictorPool;
|
||||
private GenericObjectPool<Predictor<Image, DirectionInfo>> predictorPool;
|
||||
|
||||
private DirectionModelConfig config;
|
||||
|
||||
@@ -71,10 +71,6 @@ public class PPOCRMobileV2ClsModel implements OcrDirectionModel {
|
||||
}
|
||||
this.config = config;
|
||||
this.textDetModel = config.getTextDetModel();
|
||||
Device device = null;
|
||||
if(!Objects.isNull(config.getDevice())){
|
||||
device = config.getDevice() == DeviceEnum.CPU ? Device.cpu() : Device.gpu();
|
||||
}
|
||||
ConcurrentHashMap params = new ConcurrentHashMap<String, String>();
|
||||
if(StringUtils.isNotBlank(config.getBatchifier())){
|
||||
params.put("batchifier", config.getBatchifier());
|
||||
@@ -84,8 +80,14 @@ public class PPOCRMobileV2ClsModel implements OcrDirectionModel {
|
||||
model = ModelZoo.loadModel(criteria);
|
||||
// 创建池子:每个线程独享 Predictor
|
||||
this.predictorPool = new GenericObjectPool<>(new PredictorFactory<>(model));
|
||||
int predictorPoolSize = config.getPredictorPoolSize();
|
||||
if(config.getPredictorPoolSize() <= 0){
|
||||
predictorPoolSize = Runtime.getRuntime().availableProcessors(); // 默认等于CPU核心数
|
||||
}
|
||||
predictorPool.setMaxTotal(predictorPoolSize);
|
||||
log.debug("当前设备: " + model.getNDManager().getDevice());
|
||||
log.debug("当前引擎: " + Engine.getInstance().getEngineName());
|
||||
log.debug("模型推理器线程池最大数量: " + predictorPoolSize);
|
||||
} catch (IOException | ModelNotFoundException | MalformedModelException e) {
|
||||
throw new OcrException("模型加载失败", e);
|
||||
}
|
||||
@@ -369,6 +371,11 @@ public class PPOCRMobileV2ClsModel implements OcrDirectionModel {
|
||||
return textDetModel;
|
||||
}
|
||||
|
||||
@Override
|
||||
public GenericObjectPool<Predictor<Image, DirectionInfo>> getPool() {
|
||||
return predictorPool;
|
||||
}
|
||||
|
||||
@Override
|
||||
public void close() throws Exception {
|
||||
try {
|
||||
|
||||
@@ -28,7 +28,7 @@ public class DirectionCriteriaFactory {
|
||||
public static Criteria<Image, DirectionInfo> createCriteria(DirectionModelConfig config) {
|
||||
Device device = null;
|
||||
if(!Objects.isNull(config.getDevice())){
|
||||
device = config.getDevice() == DeviceEnum.CPU ? Device.cpu() : Device.gpu();
|
||||
device = config.getDevice() == DeviceEnum.CPU ? Device.cpu() : Device.gpu(config.getGpuId());
|
||||
}
|
||||
Criteria<Image, DirectionInfo> criteria = null;
|
||||
ConcurrentHashMap params = new ConcurrentHashMap<String, String>();
|
||||
|
||||
@@ -1,5 +1,6 @@
|
||||
package cn.smartjavaai.ocr.model.common.recognize;
|
||||
|
||||
import ai.djl.inference.Predictor;
|
||||
import ai.djl.modality.cv.Image;
|
||||
import cn.smartjavaai.ocr.config.OcrDetModelConfig;
|
||||
import cn.smartjavaai.ocr.config.OcrRecModelConfig;
|
||||
@@ -8,6 +9,7 @@ import cn.smartjavaai.ocr.entity.OcrBox;
|
||||
import cn.smartjavaai.ocr.entity.OcrInfo;
|
||||
import cn.smartjavaai.ocr.model.common.detect.OcrCommonDetModel;
|
||||
import cn.smartjavaai.ocr.model.common.direction.OcrDirectionModel;
|
||||
import org.apache.commons.pool2.impl.GenericObjectPool;
|
||||
|
||||
import java.awt.image.BufferedImage;
|
||||
import java.util.List;
|
||||
@@ -106,4 +108,8 @@ public interface OcrCommonRecModel extends AutoCloseable{
|
||||
throw new UnsupportedOperationException("默认不支持该功能");
|
||||
}
|
||||
|
||||
default GenericObjectPool<Predictor<Image, String>> getPool() {
|
||||
throw new UnsupportedOperationException("默认不支持该功能");
|
||||
}
|
||||
|
||||
}
|
||||
|
||||
@@ -24,7 +24,6 @@ import cn.smartjavaai.ocr.exception.OcrException;
|
||||
import cn.smartjavaai.ocr.model.common.detect.OcrCommonDetModel;
|
||||
import cn.smartjavaai.ocr.model.common.direction.OcrDirectionModel;
|
||||
import cn.smartjavaai.ocr.model.common.recognize.criteria.OcrCommonRecCriterialFactory;
|
||||
import cn.smartjavaai.ocr.opencv.OcrOpenCVUtils;
|
||||
import cn.smartjavaai.ocr.utils.OcrUtils;
|
||||
import lombok.extern.slf4j.Slf4j;
|
||||
import org.apache.commons.collections.CollectionUtils;
|
||||
@@ -49,7 +48,7 @@ import java.util.stream.Collectors;
|
||||
@Slf4j
|
||||
public class OcrCommonRecModelImpl implements OcrCommonRecModel {
|
||||
|
||||
private ObjectPool<Predictor<Image, String>> recPredictorPool;
|
||||
private GenericObjectPool<Predictor<Image, String>> recPredictorPool;
|
||||
|
||||
private OcrRecModelConfig config;
|
||||
|
||||
@@ -72,8 +71,14 @@ public class OcrCommonRecModelImpl implements OcrCommonRecModel {
|
||||
try{
|
||||
recognitionModel = ModelZoo.loadModel(recCriteria);
|
||||
this.recPredictorPool = new GenericObjectPool<>(new PredictorFactory<>(recognitionModel));
|
||||
int predictorPoolSize = config.getPredictorPoolSize();
|
||||
if(config.getPredictorPoolSize() <= 0){
|
||||
predictorPoolSize = Runtime.getRuntime().availableProcessors(); // 默认等于CPU核心数
|
||||
}
|
||||
recPredictorPool.setMaxTotal(predictorPoolSize);
|
||||
log.debug("当前设备: " + recognitionModel.getNDManager().getDevice());
|
||||
log.debug("当前引擎: " + Engine.getInstance().getEngineName());
|
||||
log.debug("模型推理器线程池最大数量: " + predictorPoolSize);
|
||||
} catch (IOException | ModelNotFoundException | MalformedModelException e) {
|
||||
throw new OcrException("识别模型加载失败", e);
|
||||
}
|
||||
@@ -238,7 +243,7 @@ public class OcrCommonRecModelImpl implements OcrCommonRecModel {
|
||||
throw new OcrException("未检测到文字");
|
||||
}
|
||||
Mat wrappedImage = (Mat) img.getWrappedImage();
|
||||
BufferedImage bufferedImage = OcrOpenCVUtils.mat2Image(wrappedImage);
|
||||
BufferedImage bufferedImage = OpenCVUtils.mat2Image(wrappedImage);
|
||||
OcrUtils.drawRectWithText(bufferedImage, ocrInfo, fontSize);
|
||||
ImageUtils.saveImage(bufferedImage, outputPath);
|
||||
wrappedImage.release();
|
||||
@@ -443,6 +448,11 @@ public class OcrCommonRecModelImpl implements OcrCommonRecModel {
|
||||
return directionModel;
|
||||
}
|
||||
|
||||
|
||||
public GenericObjectPool<Predictor<Image, String>> getRecPredictorPool() {
|
||||
return recPredictorPool;
|
||||
}
|
||||
|
||||
@Override
|
||||
public void close() throws Exception {
|
||||
try {
|
||||
|
||||
@@ -24,7 +24,7 @@ public class OcrCommonRecCriterialFactory {
|
||||
public static Criteria<Image, String> createCriteria(OcrRecModelConfig config) {
|
||||
Device device = null;
|
||||
if(!Objects.isNull(config.getDevice())){
|
||||
device = config.getDevice() == DeviceEnum.CPU ? Device.cpu() : Device.gpu();
|
||||
device = config.getDevice() == DeviceEnum.CPU ? Device.cpu() : Device.gpu(config.getGpuId());
|
||||
}
|
||||
Criteria<Image, String> criteria = null;
|
||||
ConcurrentHashMap params = new ConcurrentHashMap<String, String>();
|
||||
|
||||
@@ -0,0 +1,342 @@
|
||||
package cn.smartjavaai.ocr.model.plate;
|
||||
|
||||
import ai.djl.MalformedModelException;
|
||||
import ai.djl.engine.Engine;
|
||||
import ai.djl.inference.Predictor;
|
||||
import ai.djl.modality.cv.Image;
|
||||
import ai.djl.modality.cv.ImageFactory;
|
||||
import ai.djl.modality.cv.output.BoundingBox;
|
||||
import ai.djl.modality.cv.output.DetectedObjects;
|
||||
import ai.djl.repository.zoo.Criteria;
|
||||
import ai.djl.repository.zoo.ModelNotFoundException;
|
||||
import ai.djl.repository.zoo.ModelZoo;
|
||||
import ai.djl.repository.zoo.ZooModel;
|
||||
import cn.hutool.core.lang.UUID;
|
||||
import cn.hutool.core.lang.generator.UUIDGenerator;
|
||||
import cn.smartjavaai.common.entity.DetectionRectangle;
|
||||
import cn.smartjavaai.common.entity.R;
|
||||
import cn.smartjavaai.common.pool.PredictorFactory;
|
||||
import cn.smartjavaai.common.utils.Base64ImageUtils;
|
||||
import cn.smartjavaai.common.utils.FileUtils;
|
||||
import cn.smartjavaai.common.utils.ImageUtils;
|
||||
import cn.smartjavaai.common.utils.OpenCVUtils;
|
||||
import cn.smartjavaai.ocr.config.PlateDetModelConfig;
|
||||
import cn.smartjavaai.ocr.config.PlateRecModelConfig;
|
||||
import cn.smartjavaai.ocr.entity.PlateInfo;
|
||||
import cn.smartjavaai.ocr.entity.PlateResult;
|
||||
import cn.smartjavaai.ocr.enums.PlateType;
|
||||
import cn.smartjavaai.ocr.exception.OcrException;
|
||||
import cn.smartjavaai.ocr.model.plate.criteria.PlateDetCriterialFactory;
|
||||
import cn.smartjavaai.ocr.model.plate.criteria.PlateRecCriterialFactory;
|
||||
import cn.smartjavaai.ocr.utils.OcrUtils;
|
||||
import lombok.extern.slf4j.Slf4j;
|
||||
import org.apache.commons.collections.CollectionUtils;
|
||||
import org.apache.commons.lang3.StringUtils;
|
||||
import org.apache.commons.pool2.ObjectPool;
|
||||
import org.apache.commons.pool2.impl.GenericObjectPool;
|
||||
import org.opencv.core.Core;
|
||||
import org.opencv.core.Mat;
|
||||
import org.opencv.core.Rect;
|
||||
import org.opencv.core.Size;
|
||||
import org.opencv.imgproc.Imgproc;
|
||||
|
||||
import javax.imageio.ImageIO;
|
||||
import java.awt.*;
|
||||
import java.awt.image.BufferedImage;
|
||||
import java.io.*;
|
||||
import java.nio.file.Files;
|
||||
import java.nio.file.Path;
|
||||
import java.nio.file.Paths;
|
||||
import java.util.ArrayList;
|
||||
import java.util.Iterator;
|
||||
import java.util.List;
|
||||
import java.util.Objects;
|
||||
|
||||
/**
|
||||
* @author dwj
|
||||
*/
|
||||
@Slf4j
|
||||
public class CRNNPlateRecModel implements PlateRecModel{
|
||||
|
||||
|
||||
private GenericObjectPool<Predictor<Image, PlateResult>> recPredictorPool;
|
||||
|
||||
private ZooModel<Image, PlateResult> recModel;
|
||||
|
||||
private PlateRecModelConfig config;
|
||||
|
||||
@Override
|
||||
public void loadModel(PlateRecModelConfig config) {
|
||||
if(StringUtils.isBlank(config.getModelPath())){
|
||||
throw new OcrException("modelPath is null");
|
||||
}
|
||||
this.config = config;
|
||||
//初始化 检测Criteria
|
||||
Criteria<Image, PlateResult> detCriteria = PlateRecCriterialFactory.createCriteria(config);
|
||||
try{
|
||||
recModel = ModelZoo.loadModel(detCriteria);
|
||||
// 创建池子:每个线程独享 Predictor
|
||||
this.recPredictorPool = new GenericObjectPool<>(new PredictorFactory<>(recModel));
|
||||
int predictorPoolSize = config.getPredictorPoolSize();
|
||||
if(config.getPredictorPoolSize() <= 0){
|
||||
predictorPoolSize = Runtime.getRuntime().availableProcessors(); // 默认等于CPU核心数
|
||||
}
|
||||
recPredictorPool.setMaxTotal(predictorPoolSize);
|
||||
log.debug("当前设备: " + recModel.getNDManager().getDevice());
|
||||
log.debug("当前引擎: " + Engine.getInstance().getEngineName());
|
||||
log.debug("模型推理器线程池最大数量: " + predictorPoolSize);
|
||||
} catch (IOException | ModelNotFoundException | MalformedModelException e) {
|
||||
throw new OcrException("检测模型加载失败", e);
|
||||
}
|
||||
}
|
||||
|
||||
@Override
|
||||
public R<List<PlateInfo>> recognize(String imagePath) {
|
||||
if(!FileUtils.isFileExists(imagePath)){
|
||||
return R.fail(R.Status.FILE_NOT_FOUND);
|
||||
}
|
||||
Image img = null;
|
||||
try {
|
||||
img = ImageFactory.getInstance().fromFile(Paths.get(imagePath));
|
||||
R<List<PlateInfo>> plateResult = recognize(img);
|
||||
return plateResult;
|
||||
} catch (IOException e) {
|
||||
throw new OcrException("无效的图片", e);
|
||||
} finally {
|
||||
((Mat)img.getWrappedImage()).release();
|
||||
}
|
||||
}
|
||||
|
||||
@Override
|
||||
public R<List<PlateInfo>> recognizeBase64(String base64Image) {
|
||||
if(StringUtils.isBlank(base64Image)){
|
||||
return R.fail(R.Status.INVALID_IMAGE);
|
||||
}
|
||||
byte[] imageData = Base64ImageUtils.base64ToImage(base64Image);
|
||||
return recognize(imageData);
|
||||
}
|
||||
|
||||
@Override
|
||||
public R<List<PlateInfo>> recognize(BufferedImage image) {
|
||||
if(!ImageUtils.isImageValid(image)){
|
||||
return R.fail(R.Status.INVALID_IMAGE);
|
||||
}
|
||||
Image img = ImageFactory.getInstance().fromImage(OpenCVUtils.image2Mat(image));
|
||||
R<List<PlateInfo>> plateResult = recognize(img);
|
||||
((Mat)img.getWrappedImage()).release();
|
||||
return plateResult;
|
||||
}
|
||||
|
||||
@Override
|
||||
public R<List<PlateInfo>> recognize(byte[] imageData) {
|
||||
if(Objects.isNull(imageData)){
|
||||
return R.fail(R.Status.INVALID_IMAGE);
|
||||
}
|
||||
return recognize(new ByteArrayInputStream(imageData));
|
||||
}
|
||||
|
||||
@Override
|
||||
public R<List<PlateInfo>> recognize(Image image) {
|
||||
if(Objects.isNull(config.getPlateDetModel())){
|
||||
return R.fail(R.Status.PARAM_ERROR.getCode(), "未指定车牌检测模型");
|
||||
}
|
||||
DetectedObjects detectedObjects = config.getPlateDetModel().detect(image);
|
||||
if(Objects.isNull(detectedObjects) || detectedObjects.getNumberOfObjects() == 0){
|
||||
return R.fail(R.Status.NO_OBJECT_DETECTED);
|
||||
}
|
||||
List<PlateInfo> plateInfoList = OcrUtils.convertToPlateInfo(detectedObjects, image);
|
||||
Predictor<Image, PlateResult> predictor = null;
|
||||
try {
|
||||
predictor = recPredictorPool.borrowObject();
|
||||
for (PlateInfo plateInfo : plateInfoList){
|
||||
DetectionRectangle detectionRectangle = plateInfo.getDetectionRectangle();
|
||||
// Image subImage = image.getSubImage(detectionRectangle.getX(), detectionRectangle.getY(), detectionRectangle.getWidth(), detectionRectangle.getHeight());
|
||||
//透视变换
|
||||
Image subImage = OcrUtils.transformAndCrop((Mat)image.getWrappedImage(), plateInfo.getBox());
|
||||
//双层车牌
|
||||
if(plateInfo.getPlateType() == PlateType.DOUBLE){
|
||||
Mat mergeImage = getSplitMerge((Mat)subImage.getWrappedImage());
|
||||
subImage = ImageFactory.getInstance().fromImage(mergeImage);
|
||||
}
|
||||
PlateResult plateResult = predictor.predict(subImage);
|
||||
if(Objects.nonNull(plateResult)){
|
||||
plateInfo.setPlateNumber(plateResult.getPlateNo());
|
||||
plateInfo.setPlateColor(plateResult.getPlateColor());
|
||||
}
|
||||
}
|
||||
return R.ok(plateInfoList);
|
||||
} catch (Exception e) {
|
||||
throw new OcrException("车牌识别错误", e);
|
||||
}finally {
|
||||
if (predictor != null) {
|
||||
try {
|
||||
recPredictorPool.returnObject(predictor); //归还
|
||||
} catch (Exception e) {
|
||||
log.warn("归还Predictor失败", e);
|
||||
try {
|
||||
predictor.close(); // 归还失败才销毁
|
||||
} catch (Exception ex) {
|
||||
log.error("关闭Predictor失败", ex);
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
/**
|
||||
* 双层车牌进行分割后识别
|
||||
* @param img
|
||||
* @return
|
||||
*/
|
||||
private Mat getSplitMerge(Mat img) {
|
||||
int h = img.rows();
|
||||
int w = img.cols();
|
||||
|
||||
// 上半部分:高度的前 5/12
|
||||
Rect upperRect = new Rect(0, 0, w, (int)(5.0 / 12 * h));
|
||||
Mat imgUpper = new Mat(img, upperRect);
|
||||
|
||||
// 下半部分:高度从 1/3 开始
|
||||
Rect lowerRect = new Rect(0, (int)(1.0 / 3 * h), w, h - (int)(1.0 / 3 * h));
|
||||
Mat imgLower = new Mat(img, lowerRect);
|
||||
|
||||
// 将上半部分 resize 到与下半部分相同大小
|
||||
Mat resizedUpper = new Mat();
|
||||
Size lowerSize = imgLower.size();
|
||||
Imgproc.resize(imgUpper, resizedUpper, lowerSize);
|
||||
|
||||
// 水平拼接(将上下拼成左右)
|
||||
List<Mat> mergeList = new ArrayList<>();
|
||||
mergeList.add(resizedUpper);
|
||||
mergeList.add(imgLower);
|
||||
|
||||
Mat merged = new Mat();
|
||||
Core.hconcat(mergeList, merged);
|
||||
return merged;
|
||||
}
|
||||
|
||||
@Override
|
||||
public PlateResult recognizeCropped(Image image) {
|
||||
Predictor<Image, PlateResult> predictor = null;
|
||||
try {
|
||||
predictor = recPredictorPool.borrowObject();
|
||||
return predictor.predict(image);
|
||||
} catch (Exception e) {
|
||||
throw new OcrException("车牌检测错误", e);
|
||||
}finally {
|
||||
if (predictor != null) {
|
||||
try {
|
||||
recPredictorPool.returnObject(predictor); //归还
|
||||
} catch (Exception e) {
|
||||
log.warn("归还Predictor失败", e);
|
||||
try {
|
||||
predictor.close(); // 归还失败才销毁
|
||||
} catch (Exception ex) {
|
||||
log.error("关闭Predictor失败", ex);
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
}
|
||||
}
|
||||
|
||||
@Override
|
||||
public R<List<PlateInfo>> recognize(InputStream inputStream) {
|
||||
if(Objects.isNull(inputStream)){
|
||||
return R.fail(R.Status.INVALID_IMAGE);
|
||||
}
|
||||
Image img = null;
|
||||
try {
|
||||
img = ImageFactory.getInstance().fromInputStream(inputStream);
|
||||
return recognize(img);
|
||||
} catch (IOException e) {
|
||||
throw new OcrException("无效图片输入流", e);
|
||||
} finally {
|
||||
if (img != null){
|
||||
((Mat)img.getWrappedImage()).release();
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
@Override
|
||||
public R<Void> recognizeAndDraw(String imagePath, String outputPath) {
|
||||
if(!FileUtils.isFileExists(imagePath)){
|
||||
return R.fail(R.Status.FILE_NOT_FOUND);
|
||||
}
|
||||
Image img = null;
|
||||
try {
|
||||
img = ImageFactory.getInstance().fromFile(Paths.get(imagePath));
|
||||
R<List<PlateInfo>> plateResult = recognize(img);
|
||||
if(!plateResult.isSuccess()){
|
||||
return R.fail(plateResult.getCode(), plateResult.getMessage());
|
||||
}
|
||||
if(CollectionUtils.isEmpty(plateResult.getData())){
|
||||
return R.fail(R.Status.NO_OBJECT_DETECTED);
|
||||
}
|
||||
BufferedImage bufferedImage = OpenCVUtils.mat2Image((Mat)img.getWrappedImage());
|
||||
OcrUtils.drawPlateInfo(bufferedImage, plateResult.getData());
|
||||
ImageIO.write(bufferedImage, "jpg", new File(outputPath));
|
||||
return R.ok();
|
||||
} catch (IOException e) {
|
||||
throw new OcrException(e);
|
||||
} finally {
|
||||
if (img != null){
|
||||
((Mat)img.getWrappedImage()).release();
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
@Override
|
||||
public R<BufferedImage> recognizeAndDraw(BufferedImage sourceImage) {
|
||||
if(!ImageUtils.isImageValid(sourceImage)){
|
||||
return R.fail(R.Status.INVALID_IMAGE);
|
||||
}
|
||||
Image img = ImageFactory.getInstance().fromImage(OpenCVUtils.image2Mat(sourceImage));
|
||||
try {
|
||||
R<List<PlateInfo>> plateResult = recognize(img);
|
||||
if(!plateResult.isSuccess()){
|
||||
return R.fail(plateResult.getCode(), plateResult.getMessage());
|
||||
}
|
||||
if(CollectionUtils.isEmpty(plateResult.getData())){
|
||||
return R.fail(R.Status.NO_OBJECT_DETECTED);
|
||||
}
|
||||
OcrUtils.drawPlateInfo((Mat)img.getWrappedImage(), plateResult.getData());
|
||||
ByteArrayOutputStream outputStream = new ByteArrayOutputStream();
|
||||
// 调用 save 方法将 Image 写入字节流
|
||||
img.save(outputStream, "png");
|
||||
// 将字节流转换为 BufferedImage
|
||||
byte[] imageBytes = outputStream.toByteArray();
|
||||
return R.ok(ImageIO.read(new ByteArrayInputStream(imageBytes)));
|
||||
} catch (IOException e) {
|
||||
throw new OcrException("导出图片失败", e);
|
||||
} finally {
|
||||
if (img != null){
|
||||
((Mat)img.getWrappedImage()).release();
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
|
||||
@Override
|
||||
public GenericObjectPool<Predictor<Image, PlateResult>> getPool() {
|
||||
return recPredictorPool;
|
||||
}
|
||||
|
||||
@Override
|
||||
public void close() throws Exception {
|
||||
try {
|
||||
if (recPredictorPool != null) {
|
||||
recPredictorPool.close();
|
||||
}
|
||||
} catch (Exception e) {
|
||||
log.warn("关闭 predictorPool 失败", e);
|
||||
}
|
||||
try {
|
||||
if (recModel != null) {
|
||||
recModel.close();
|
||||
}
|
||||
} catch (Exception e) {
|
||||
log.warn("关闭 model 失败", e);
|
||||
}
|
||||
}
|
||||
}
|
||||