1、集成车牌识别模型,支持车牌检测与识别

2、新增 Milvus 身份验证支持
3、目标检测功能升级:可指定类别及topk
4、支持自定义线程池线程数量
This commit is contained in:
dengwenjie
2025-07-28 12:04:02 +08:00
parent 1bd74d1bb8
commit 1d45bc597d
117 changed files with 3490 additions and 437 deletions

View File

@@ -192,6 +192,25 @@ SmartJavaAI是专为JAVA 开发者打造的一个功能丰富、开箱即用的
</div>
</td>
</tr>
<tr>
<td>
<div align="left">
<p>车牌识别</p>
- 单层/双层检测 <br>
- 车牌颜色识别 <br>
</div>
</td>
<td>
<div align="center">
<img src="https://cdn.jsdelivr.net/gh/geekwenjie/SmartJavaAI-Site/images/ocr/plate_recognized.jpg" width = "500px"/>
</div>
</td>
<td>
<div align="center">
<img src="https://cdn.jsdelivr.net/gh/geekwenjie/SmartJavaAI-Site/images/ocr/plate_recognized2.jpg" width = "500px"/>
</div>
</td>
</tr>
<tr>
<td>
<div align="left">
@@ -251,21 +270,22 @@ SmartJavaAI是专为JAVA 开发者打造的一个功能丰富、开箱即用的
- 支持任意角度识别,方向校准
- 支持通用文字识别,通用手写字识别
- 支持表格识别
- 支持中文车牌识别:单层/双层检测颜色识别支持12种中文车牌
- **机器翻译**
- 集成NLLB-200模型支持200+语言互相翻译
## 🌟 AI集成方式对比
| 方案 | 技术特点 | 优点 | 缺点 |
|---------------------|--------------------------------------------------------------------------|---------------------------------------------------------------------|---------------------------------------------------------------------|
| **OpenCV** | 传统图像处理方案 | ✅ 提供java接口<br>✅ 轻量级部署<br>✅ 社区资源丰富 | ❌ 基于传统算法精度低(60%-75%)<br>❌ 需本地安装环境 |
| &zwnj;**商业闭源SDK如虹软等**&zwnj; | 商业级闭源解决方案 | ✅ 开箱即用<br>✅ 提供完整文档和SDK<br>✅ 支持离线活体检测 | ❌ 免费版需年度授权更新<br>❌ 商业授权费用高<br>❌ 代码不可控 |
| &zwnj;**云API(阿里云)**&zwnj; | SaaS化云端服务 | ✅ 零部署成本<br>✅ 支持高并发<br>✅ 自带模型迭代 | ❌ 网络延迟风险(200-800ms)<br>❌ 按调用量计费<br>❌ 有数据安全风险 |
| 方案 | 技术特点 | 优点 | 缺点 |
|---------------------|--------------------------------------------------------------------------|---------------------------------------------------------------------|-----------------------------------------------|
| **OpenCV** | 传统图像处理方案 | ✅ 提供java接口<br>✅ 轻量级部署<br>✅ 社区资源丰富 | ❌ 基于传统算法精度低(60%-75%)<br>❌ 需本地安装环境 |
| &zwnj;**商业闭源SDK如虹软等**&zwnj; | 商业级闭源解决方案 | ✅ 开箱即用<br>✅ 提供完整文档和SDK<br>✅ 支持离线活体检测 | ❌ 免费版需年度授权更新<br>❌ 商业授权费用高<br>❌ 代码不可控 |
| &zwnj;**云API(阿里云)**&zwnj; | SaaS化云端服务 | ✅ 零部署成本<br>✅ 支持高并发<br>✅ 自带模型迭代 | ❌ 网络延迟风险(200-800ms)<br>❌ 按调用量计费<br>❌ 有数据安全风险 |
| &zwnj;**Python混合调用**&zwnj; | 跨语言调用方案 | ✅ 可集成PyTorch/TF等框架<br>✅ 支持自定义算法<br>✅ 识别精度高 | ❌ 需维护双语言环境<br>❌ 进程通信性能损耗(30%+)<br>❌ 异常处理复杂度翻倍 |
| &zwnj;**JNI/JNA**&zwnj; | 跨语言底层调用方案 | ✅ 直接调用 C/C++ 高性能算法库✅ 支持调用各种原生成熟库✅ 可封装成通用工具Jar |❌ 开发成本高JNI更复杂❌ 跨平台兼容性差 |
| &zwnj;**DJL框架**&zwnj; | 深度学习框架 | ✅ 纯Java实现<br>✅ 支持主流深度学习框架<br>✅ 可加载预训练模型(99%+) | ❌ 需掌握DL知识<br>❌ 需处理模型加载、预处理、后处理等复杂技术细节 |
| &zwnj;**SmartJavaAI**&zwnj; | java深度学习工具包 | ✅ 支持主流深度学习框架<br>✅ 提供丰富、开箱即用API<br> ✅ 上手简单单一Jar包集成 | ❌要求JDK版本11及以上 |
| &zwnj;**JNI/JNA**&zwnj; | 跨语言底层调用方案 | ✅ 直接调用 C/C++ 高性能算法库✅ 支持调用各种原生成熟库✅ 可封装成通用工具Jar | ❌ 开发成本高JNI更复杂❌ 跨平台兼容性差 |
| &zwnj;**DJL框架**&zwnj; | 深度学习框架 | ✅ 纯Java实现<br>✅ 支持主流深度学习框架<br>✅ 可加载预训练模型(99%+) | ❌ 需掌握DL知识<br>❌ 需处理模型加载、预处理、后处理等复杂技术细节 |
| &zwnj;**SmartJavaAI**&zwnj; | java深度学习工具包 | ✅ 支持主流深度学习框架<br>✅ 提供丰富、开箱即用API<br> ✅ 上手简单单一Jar包集成 | |
@@ -300,7 +320,7 @@ SmartJavaAI是专为JAVA 开发者打造的一个功能丰富、开箱即用的
<dependency>
<groupId>cn.smartjavaai</groupId>
<artifactId>smartjavaai-all</artifactId>
<version>1.0.20</version>
<version>1.0.22</version>
</dependency>
```
### 3、完整示例代码
@@ -406,16 +426,6 @@ SmartJavaAI是专为JAVA 开发者打造的一个功能丰富、开箱即用的
- 自训练模型推理
- yolov3~yolov12 系列
**SSD 系列**
| 模型名称 | 引擎 | 骨干网络 | 输入尺寸 | <div style="width: 60pt">训练数据集</div> | 精度mAP | <div style="width: 50pt">推理速度</div> | <div style="width: 150pt">适用场景</div>|
| :-------- |------------------| ------------- | ----------- |--------------------------------------| -------------|-------------------------------------| -------------|
|SSD_300_RESNET5| PyTorch | ResNet50 | 300×300 | COCO | 中等 | 快 | 精度需求一般|
|SSD_512_RESNET50_V1_VOC| PyTorch | ResNet50 | 512×512 | Pascal VOC | 稍高 | 中等 | 精度优先、可接受略低速度的场景|
|SSD_512_VGG16_ATROUS_COCO| MXNet | VGG16 | 512×512 | COCO | 较高 | 中等 | 通用场景;对小目标有一定提升|
|SSD_300_VGG16_ATROUS_VOC| MXNet | VGG16 | 300×300 | Pascal VOC | 中等偏上 | 快 | VOC 数据集同类任务;资源受限时使用|
|SSD_512_MOBILENET1_VOC| MXNet | MobileNet1.0 | 512×512 | Pascal VOC | 中等 | 快 | 嵌入式/移动端设备;算力和内存都很有限|
**YOLO 系列**
|模型名称 | 引擎 | 版本 | 大小Backbone | <div style="width: 60pt">数据集</div> | <div style="width: 50pt">精度</div> | <div style="width: 50pt">速度</div> | <div style="width: 150pt">适用场景</div> |
@@ -427,6 +437,18 @@ SmartJavaAI是专为JAVA 开发者打造的一个功能丰富、开箱即用的
|YOLOV5S_ONNXRUNTIME| OnnxRuntime | v5 | 小型 | COCO | 较高 | 加速(需 ONNX 支持) | Windows/Linux 通用加速部署 |
|YOLO (MXNet / 通用模型) | MXNet | v3 | DarkNet53 | COCO | 较高 | 快 | 需要 MXNet 生态或复现老项目时使用 |
**SSD 系列**
| 模型名称 | 引擎 | 骨干网络 | 输入尺寸 | <div style="width: 60pt">训练数据集</div> | 精度mAP | <div style="width: 50pt">推理速度</div> | <div style="width: 150pt">适用场景</div>|
| :-------- |------------------| ------------- | ----------- |--------------------------------------| -------------|-------------------------------------| -------------|
|SSD_300_RESNET5| PyTorch | ResNet50 | 300×300 | COCO | 中等 | 快 | 精度需求一般|
|SSD_512_RESNET50_V1_VOC| PyTorch | ResNet50 | 512×512 | Pascal VOC | 稍高 | 中等 | 精度优先、可接受略低速度的场景|
|SSD_512_VGG16_ATROUS_COCO| MXNet | VGG16 | 512×512 | COCO | 较高 | 中等 | 通用场景;对小目标有一定提升|
|SSD_300_VGG16_ATROUS_VOC| MXNet | VGG16 | 300×300 | Pascal VOC | 中等偏上 | 快 | VOC 数据集同类任务;资源受限时使用|
|SSD_512_MOBILENET1_VOC| MXNet | MobileNet1.0 | 512×512 | Pascal VOC | 中等 | 快 | 嵌入式/移动端设备;算力和内存都很有限|
**YOLOv3 变体系列**
|模型名称| 引擎 | 骨干网络 | <div style="width: 60pt">数据集</div> | 输入尺寸 | <div style="width: 50pt">精度</div> | <div style="width: 50pt">速度</div> | <div style="width: 200pt">适用场景</div> |
@@ -486,6 +508,21 @@ SmartJavaAI是专为JAVA 开发者打造的一个功能丰富、开箱即用的
| SLANet_plus | (增强版)该模型通过轻量级骨干 PP-LCNet、CSP-PAN 融合与 SLA Head 解码,有效提升表格结构识别的精度与速度。 | [Github](https://paddlepaddle.github.io/PaddleOCR/v3.1.0/version3.x/module_usage/table_structure_recognition.html#_3) |
**车牌检测模型(License Plate Detection)**
| 模型名称 | 模型简介 | 模型开源网站 |
|--------|----------------------|------------|
| YOLOV5 | 基于YOLOV5训练支持12种中文车牌 | [Github](https://github.com/we0091234/Chinese_license_plate_detection_recognition) |
| YOLOV7 | 基于YOLOV7训练支持12种中文车牌 | [Github](https://github.com/we0091234/Chinese_license_plate_detection_recognition) |
**车牌识别模型(License Plate Recognition)**
| 模型名称 | 模型简介 | 模型开源网站 |
|--------|------------|------------|
| PLATE_REC_CRNN | CRNN中文字符识别 | [Github](https://github.com/Sierkinhane/CRNN_Chinese_Characters_Rec) |
---
#### 机器翻译模型
@@ -519,7 +556,22 @@ SmartJavaAI是专为JAVA 开发者打造的一个功能丰富、开箱即用的
## 近期更新日志
## [v1.0.20] - 2025-07-06
## [v1.0.22] - 2025-07-28
- 新增 Milvus 身份验证支持
- 集成车牌识别模型,支持车牌检测与识别
- 目标检测功能升级可指定类别及topk
- 支持自定义线程池线程数量
## [v1.0.20] - 2025-07-18
- OCR新增表格识别模型
- OCR新增9个通用模型
- OCR支持批量检测识别
- OCR新增更多参数使用更加灵活
- 人脸识别支持ID查询及分页获取人脸信息
- 活体检测:视频检测支持设置最大帧数
## [v1.0.19] - 2025-07-06
- 人脸模块新增小视科技MiniVision活体检测模型
- 人脸模块:新增阿里通义工作室活体检测模型
- 人脸模块:新增 2 个表情识别模型

View File

@@ -12,7 +12,7 @@
<maven.compiler.source>11</maven.compiler.source>
<maven.compiler.target>11</maven.compiler.target>
<project.build.sourceEncoding>UTF-8</project.build.sourceEncoding>
<smartjavaai.version>1.0.20</smartjavaai.version>
<smartjavaai.version>1.0.22</smartjavaai.version>
<!--如果打包运行需要替换成你的main-->
<exec.mainClass>smartai.examples.face.facedet.FaceDetDemo</exec.mainClass>
@@ -95,6 +95,13 @@
<dependency>
<groupId>cn.smartjavaai</groupId>
<artifactId>smartjavaai-face</artifactId>
<version>1.0.22</version>
</dependency>
<dependency>
<groupId>cn.smartjavaai</groupId>
<artifactId>smartjavaai-all</artifactId>
<version>1.0.22</version>
</dependency>

View File

@@ -0,0 +1,25 @@
package smartai.examples.face;
import cn.smartjavaai.common.utils.VideoUtils;
import org.bytedeco.ffmpeg.global.avcodec;
import org.bytedeco.javacv.FFmpegFrameGrabber;
import org.bytedeco.javacv.FFmpegFrameRecorder;
/**
* 视频预处理
* @author dwj
* @date 2025/7/17
*/
public class VideoDemo {
public static void main(String[] args) {
try {
VideoUtils.rotateVideo("/Users/wenjie/Downloads/girl.mp4", "/Users/wenjie/Downloads/girl_rotate.mp4", 180,"mp4", avcodec.AV_CODEC_ID_H264);
} catch (FFmpegFrameRecorder.Exception e) {
throw new RuntimeException(e);
} catch (FFmpegFrameGrabber.Exception e) {
throw new RuntimeException(e);
}
}
}

View File

@@ -57,7 +57,8 @@ public class FaceAttributeDetDemo {
*/
@Test
public void testFaceAttributeDetect(){
try (FaceAttributeModel faceAttributeModel = getFaceAttributeModel()){
try {
FaceAttributeModel faceAttributeModel = getFaceAttributeModel();
DetectionResponse detectionResponse = faceAttributeModel.detect("src/main/resources/iu_1.jpg");
//绘制并导出人脸属性图片,小人脸仅有人脸框
BufferedImage image = ImageIO.read(new File(Paths.get("src/main/resources/iu_1.jpg").toAbsolutePath().toString()));
@@ -73,7 +74,8 @@ public class FaceAttributeDetDemo {
*/
@Test
public void testFaceAttributeDetect2(){
try (FaceAttributeModel faceAttributeModel = getFaceAttributeModel()){
try {
FaceAttributeModel faceAttributeModel = getFaceAttributeModel();
FaceAttribute faceAttribute = faceAttributeModel.detectTopFace("src/main/resources/iu_1.jpg");
log.info("人脸属性检测结果:{}", JSONObject.toJSONString(faceAttribute));
} catch (Exception e) {
@@ -86,7 +88,8 @@ public class FaceAttributeDetDemo {
*/
@Test
public void testFaceAttributeDetect3(){
try (FaceAttributeModel faceAttributeModel = getFaceAttributeModel()){
try {
FaceAttributeModel faceAttributeModel = getFaceAttributeModel();
FaceAttribute faceAttribute = faceAttributeModel.detectTopFace("src/main/resources/iu_1.jpg");
log.info("人脸属性检测结果:{}", JSONObject.toJSONString(faceAttribute));
} catch (Exception e) {
@@ -103,8 +106,9 @@ public class FaceAttributeDetDemo {
*/
@Test
public void testFaceAttributeDetect4(){
try (FaceDetModel faceDetModel = getFaceDetModel();
FaceAttributeModel faceAttributeModel = getFaceAttributeModel()){
try {
FaceDetModel faceDetModel = getFaceDetModel();
FaceAttributeModel faceAttributeModel = getFaceAttributeModel();
//人脸检测
BufferedImage image = ImageIO.read(new File(Paths.get("src/main/resources/iu_1.jpg").toAbsolutePath().toString()));
R<DetectionResponse> detectionResponse = faceDetModel.detect(image);

View File

@@ -88,7 +88,8 @@ public class ExpressionRecDemo {
*/
@Test
public void testExpressionDetect() {
try (ExpressionModel model = getExpressionModel()){
try {
ExpressionModel model = getExpressionModel();
R<ExpressionResult> result = model.detectTopFace("src/main/resources/emotion/happy.png");
if(result.isSuccess()){
log.info("识别结果:{}", JSONObject.toJSONString(result.getData().getExpression().getDescription()));
@@ -106,7 +107,8 @@ public class ExpressionRecDemo {
*/
@Test
public void testExpressionDetect2() {
try (ExpressionModel model = getExpressionModel()){
try {
ExpressionModel model = getExpressionModel();
R<DetectionResponse> result = model.detect("src/main/resources/emotion/happy.png");
if(result.isSuccess()){
//log.info("识别结果:{}", JSONObject.toJSONString(result.getData()));
@@ -128,8 +130,9 @@ public class ExpressionRecDemo {
*/
@Test
public void testExpressionDetect3() {
try (FaceDetModel faceDetModel = getFaceDetModel();
ExpressionModel model = getExpressionModel()){
try {
FaceDetModel faceDetModel = getFaceDetModel();
ExpressionModel model = getExpressionModel();
BufferedImage image = ImageIO.read(new File(Paths.get("src/main/resources/emotion/happy.png").toAbsolutePath().toString()));
R<DetectionResponse> detResult = faceDetModel.detect(image);
if(detResult.isSuccess()){
@@ -156,8 +159,9 @@ public class ExpressionRecDemo {
*/
@Test
public void testExpressionDetect4() {
try (FaceDetModel faceDetModel = getFaceDetModel();
ExpressionModel model = getExpressionModel()){
try {
FaceDetModel faceDetModel = getFaceDetModel();
ExpressionModel model = getExpressionModel();
BufferedImage image = ImageIO.read(new File(Paths.get("src/main/resources/emotion/happy.png").toAbsolutePath().toString()));
R<DetectionResponse> detResult = faceDetModel.detect(image);
if(detResult.isSuccess()){
@@ -182,7 +186,8 @@ public class ExpressionRecDemo {
*/
@Test
public void testExpressionDetectAndDraw(){
try (ExpressionModel model = getExpressionModel()){
try {
ExpressionModel model = getExpressionModel();
BufferedImage image = ImageIO.read(new File(Paths.get("src/main/resources/emotion/surprise.png").toAbsolutePath().toString()));
R<DetectionResponse> result = model.detect(image);
if(result.isSuccess()){
@@ -206,7 +211,8 @@ public class ExpressionRecDemo {
*/
@Test
public void testExpressionDetectCamera(){
try (ExpressionModel expressionModel = getExpressionModel()){
try {
ExpressionModel expressionModel = getExpressionModel();
OpenCV.loadShared();
VideoCapture capture = new VideoCapture(0);
if (!capture.isOpened()) {

View File

@@ -82,7 +82,8 @@ public class FaceDetDemo {
*/
@Test
public void testFaceDetect(){
try (FaceDetModel faceModel = FaceDetModelFactory.getInstance().getModel()) {
try {
FaceDetModel faceModel = FaceDetModelFactory.getInstance().getModel();
R<DetectionResponse> detectedResult = faceModel.detect(imgPath);
if(detectedResult.isSuccess()){
log.info("人脸检测结果:{}", JSONObject.toJSONString(detectedResult.getData()));
@@ -100,7 +101,8 @@ public class FaceDetDemo {
*/
@Test
public void testFaceDetectCustomConfig(){
try (FaceDetModel faceModel = getFaceDetModel()){
try {
FaceDetModel faceModel = getFaceDetModel();
R<DetectionResponse> detectedResult = faceModel.detect(imgPath);
if(detectedResult.isSuccess()){
log.info("人脸检测结果:{}", JSONObject.toJSONString(detectedResult.getData()));
@@ -118,7 +120,8 @@ public class FaceDetDemo {
*/
@Test
public void testFaceDetectAndDraw(){
try (FaceDetModel faceModel = getFaceDetModel()){
try {
FaceDetModel faceModel = getFaceDetModel();
faceModel.detectAndDraw("src/main/resources/largest_selfie.jpg","output/largest_selfie_detected.png");
} catch (Exception e) {
throw new RuntimeException(e);
@@ -131,7 +134,8 @@ public class FaceDetDemo {
*/
@Test
public void testFaceDetectAndDraw2(){
try (FaceDetModel faceModel = getFaceDetModel()){
try {
FaceDetModel faceModel = getFaceDetModel();
BufferedImage image = null;
String imagePath = "src/main/resources/largest_selfie.jpg";
image = ImageIO.read(new File(Paths.get(imagePath).toAbsolutePath().toString()));
@@ -154,11 +158,12 @@ public class FaceDetDemo {
*/
@Test
public void testDetectFaceOffine(){
FaceDetConfig config = new FaceDetConfig();
config.setModelEnum(FaceDetModelEnum.RETINA_FACE);//人脸模型
//模型路径,不同模型下载路径请参看文档
config.setModelPath("/Users/xxx/Documents/develop/face_model/retinaface.pt");
try (FaceDetModel faceModel = FaceDetModelFactory.getInstance().getModel(config)) {
try {
FaceDetConfig config = new FaceDetConfig();
config.setModelEnum(FaceDetModelEnum.RETINA_FACE);//人脸模型
//模型路径,不同模型下载路径请参看文档
config.setModelPath("/Users/xxx/Documents/develop/face_model/retinaface.pt");
FaceDetModel faceModel = FaceDetModelFactory.getInstance().getModel(config);
R<DetectionResponse> detectedResult = faceModel.detect(imgPath);
if(detectedResult.isSuccess()){
log.info("人脸检测结果:{}", JSONObject.toJSONString(detectedResult.getData()));
@@ -175,10 +180,11 @@ public class FaceDetDemo {
*/
@Test
public void testDetectFaceGPU(){
FaceDetConfig config = new FaceDetConfig();
config.setModelEnum(FaceDetModelEnum.RETINA_FACE);//人脸模型
config.setDevice(DeviceEnum.GPU);
try (FaceDetModel faceModel = FaceDetModelFactory.getInstance().getModel(config)) {
try {
FaceDetConfig config = new FaceDetConfig();
config.setModelEnum(FaceDetModelEnum.RETINA_FACE);//人脸模型
config.setDevice(DeviceEnum.GPU);
FaceDetModel faceModel = FaceDetModelFactory.getInstance().getModel(config);
R<DetectionResponse> detectedResult = faceModel.detect(imgPath);
if(detectedResult.isSuccess()){
log.info("人脸检测结果:{}", JSONObject.toJSONString(detectedResult.getData()));
@@ -196,7 +202,8 @@ public class FaceDetDemo {
*/
@Test
public void testFaceDetectSeetaface6(){
try (FaceDetModel faceModel = getSeetaface6DetModel()){
try {
FaceDetModel faceModel = getSeetaface6DetModel();
R<DetectionResponse> detectedResult = faceModel.detect(imgPath);
if(detectedResult.isSuccess()){
log.info("人脸检测结果:{}", JSONObject.toJSONString(detectedResult.getData()));
@@ -215,7 +222,8 @@ public class FaceDetDemo {
*/
@Test
public void testDetectCamera(){
try (FaceDetModel faceModel = getFaceDetModel()){
try {
FaceDetModel faceModel = getFaceDetModel();
OpenCV.loadShared();
VideoCapture capture = new VideoCapture(0);
if (!capture.isOpened()) {

View File

@@ -95,6 +95,8 @@ public class FaceRecDemo {
MilvusConfig vectorDBConfig = new MilvusConfig();
vectorDBConfig.setHost("127.0.0.1");
vectorDBConfig.setPort(19530);
//vectorDBConfig.setUsername("root");
//vectorDBConfig.setPassword("Milvus");
//vectorDBConfig.setCollectionName("face5");
//ID策略自动生成
vectorDBConfig.setIdStrategy(IdStrategy.AUTO);
@@ -135,7 +137,8 @@ public class FaceRecDemo {
*/
@Test
public void testExtractFeatures(){
try (FaceRecModel faceRecModel = getFaceRecModel()){
try {
FaceRecModel faceRecModel = getFaceRecModel();
//提取图片中所有人脸特征
R<DetectionResponse> faceResult = faceRecModel.extractFeatures("src/main/resources/iu_1.jpg");
if(faceResult.isSuccess()){
@@ -158,7 +161,8 @@ public class FaceRecDemo {
*/
@Test
public void featureComparison(){
try (FaceRecModel faceRecModel = getFaceRecModel()){
try {
FaceRecModel faceRecModel = getFaceRecModel();
//基于图像直接比对人脸特征
R<Float> similarResult = faceRecModel.featureComparison("src/main/resources/iu_1.jpg","src/main/resources/iu_2.jpg");
if(similarResult.isSuccess()){
@@ -183,7 +187,8 @@ public class FaceRecDemo {
*/
@Test
public void featureComparison2(){
try (FaceRecModel faceRecModel = getFaceRecModel()){
try {
FaceRecModel faceRecModel = getFaceRecModel();
//特征提取(提取分数最高人脸特征),适用于单人脸场景
R<float[]> featureResult1 = faceRecModel.extractTopFaceFeature("src/main/resources/iu_1.jpg");
if(featureResult1.isSuccess()){
@@ -220,7 +225,8 @@ public class FaceRecDemo {
*/
@Test
public void searchFace(){
try (FaceRecModel faceRecModel = getFaceRecModelWithDbConfig()){
try {
FaceRecModel faceRecModel = getFaceRecModelWithDbConfig();
//等待加载人脸库结束
while (!faceRecModel.isLoadFaceCompleted()){
Thread.sleep(100);
@@ -296,7 +302,8 @@ public class FaceRecDemo {
*/
@Test
public void searchFace2(){
try (FaceRecModel faceRecModel = getFaceRecModelWithSQLiteConfig()){
try {
FaceRecModel faceRecModel = getFaceRecModelWithSQLiteConfig();
//等待加载人脸库结束
while (!faceRecModel.isLoadFaceCompleted()){
Thread.sleep(100);
@@ -367,7 +374,8 @@ public class FaceRecDemo {
@Test
public void getFaceInfo(){
//使用ID获取人脸信息
try (FaceRecModel faceRecModel = getFaceRecModelWithSQLiteConfig()){
try {
FaceRecModel faceRecModel = getFaceRecModelWithSQLiteConfig();
//等待加载人脸库结束
while (!faceRecModel.isLoadFaceCompleted()){
Thread.sleep(100);
@@ -390,7 +398,8 @@ public class FaceRecDemo {
@Test
public void listFaces(){
//使用ID获取人脸信息
try (FaceRecModel faceRecModel = getFaceRecModelWithDbConfig()){
try {
FaceRecModel faceRecModel = getFaceRecModelWithDbConfig();
//等待加载人脸库结束
while (!faceRecModel.isLoadFaceCompleted()){
Thread.sleep(100);

View File

@@ -133,7 +133,8 @@ public class LivenessDetDemo {
*/
@Test
public void testLivenessDetect(){
try (LivenessDetModel livenessDetModel = getLivenessDetModel()){
try {
LivenessDetModel livenessDetModel = getLivenessDetModel();
R<DetectionResponse> response = livenessDetModel.detect("src/main/resources/liveness/1.jpg");
if(response.isSuccess()){
for (DetectionInfo detectionInfo : response.getData().getDetectionInfoList()){
@@ -152,7 +153,8 @@ public class LivenessDetDemo {
*/
@Test
public void testLivenessDetectAndDraw(){
try (LivenessDetModel livenessDetModel = getLivenessDetModel()){
try {
LivenessDetModel livenessDetModel = getLivenessDetModel();
BufferedImage image = ImageIO.read(new File(Paths.get("src/main/resources/liveness/1.jpg").toAbsolutePath().toString()));
R<DetectionResponse> response = livenessDetModel.detect(image);
if(response.isSuccess()){
@@ -175,7 +177,8 @@ public class LivenessDetDemo {
*/
@Test
public void testLivenessDetect2(){
try (LivenessDetModel livenessDetModel = getLivenessDetModel()){
try {
LivenessDetModel livenessDetModel = getLivenessDetModel();
//指定文件夹路径
File dir = new File("face-example/src/main/resources/liveness");
File[] files = dir.listFiles();
@@ -198,8 +201,9 @@ public class LivenessDetDemo {
*/
@Test
public void testLivenessDetect3(){
try (FaceDetModel faceDetectModel = getFaceDetModel();
LivenessDetModel livenessDetModel = getLivenessDetModel()){
try {
FaceDetModel faceDetectModel = getFaceDetModel();
LivenessDetModel livenessDetModel = getLivenessDetModel();
// 将图片路径转换为 BufferedImage
BufferedImage image = ImageIO.read(new File(Paths.get("src/main/resources/liveness/1.jpg").toAbsolutePath().toString()));
//人脸检测
@@ -230,8 +234,9 @@ public class LivenessDetDemo {
*/
@Test
public void testLivenessDetect4(){
try (FaceDetModel faceDetModel = getFaceDetModel();
LivenessDetModel livenessDetModel = getMiniVisionLivenessDetModel()){
try {
FaceDetModel faceDetModel = getFaceDetModel();
LivenessDetModel livenessDetModel = getMiniVisionLivenessDetModel();
// 将图片路径转换为 BufferedImage
BufferedImage image = ImageIO.read(new File(Paths.get("src/main/resources/liveness/1.jpg").toAbsolutePath().toString()));
R<DetectionResponse> detResult = faceDetModel.detect(image);
@@ -259,7 +264,8 @@ public class LivenessDetDemo {
*/
@Test
public void testLivenessDetectVideo(){
try (LivenessDetModel livenessDetModel = getLivenessDetModel()){
try {
LivenessDetModel livenessDetModel = getLivenessDetModel();
//视频路径
R<LivenessResult> livenessStatus = livenessDetModel.detectVideo("video.mp4");
if (livenessStatus.isSuccess()){
@@ -278,7 +284,8 @@ public class LivenessDetDemo {
*/
@Test
public void testLivenessDetectCamera(){
try (LivenessDetModel livenessDetModel = getLivenessDetModel()){
try {
LivenessDetModel livenessDetModel = getLivenessDetModel();
OpenCV.loadShared();
VideoCapture capture = new VideoCapture(0);
if (!capture.isOpened()) {

View File

@@ -78,8 +78,9 @@ public class FaceQualityDetDemo {
*/
@Test
public void evaluateBrightness(){
try (FaceQualityModel faceQualityModel = getFaceQualityModel();
FaceDetModel faceDetModel = getFaceDetModel()){
try {
FaceQualityModel faceQualityModel = getFaceQualityModel();
FaceDetModel faceDetModel = getFaceDetModel();
//人脸检测
BufferedImage image = ImageIO.read(new File(Paths.get("src/main/resources/iu_1.jpg").toAbsolutePath().toString()));
R<DetectionResponse> detectionResponse = faceDetModel.detect(image);
@@ -110,8 +111,9 @@ public class FaceQualityDetDemo {
*/
@Test
public void evaluateCompleteness(){
try (FaceQualityModel faceQualityModel = getFaceQualityModel();
FaceDetModel faceDetModel = getFaceDetModel()){
try {
FaceQualityModel faceQualityModel = getFaceQualityModel();
FaceDetModel faceDetModel = getFaceDetModel();
//人脸检测
BufferedImage image = ImageIO.read(new File(Paths.get("src/main/resources/iu_1.jpg").toAbsolutePath().toString()));
R<DetectionResponse> detectionResponse = faceDetModel.detect(image);
@@ -142,8 +144,9 @@ public class FaceQualityDetDemo {
*/
@Test
public void evaluateClarity(){
try (FaceQualityModel faceQualityModel = getFaceQualityModel();
FaceDetModel faceDetModel = getFaceDetModel()){
try {
FaceQualityModel faceQualityModel = getFaceQualityModel();
FaceDetModel faceDetModel = getFaceDetModel();
//人脸检测
BufferedImage image = ImageIO.read(new File(Paths.get("src/main/resources/iu_1.jpg").toAbsolutePath().toString()));
R<DetectionResponse> detectionResponse = faceDetModel.detect(image);
@@ -174,8 +177,9 @@ public class FaceQualityDetDemo {
*/
@Test
public void evaluatePose(){
try (FaceQualityModel faceQualityModel = getFaceQualityModel();
FaceDetModel faceDetModel = getFaceDetModel()){
try {
FaceQualityModel faceQualityModel = getFaceQualityModel();
FaceDetModel faceDetModel = getFaceDetModel();
//人脸检测
BufferedImage image = ImageIO.read(new File(Paths.get("src/main/resources/iu_1.jpg").toAbsolutePath().toString()));
R<DetectionResponse> detectionResponse = faceDetModel.detect(image);
@@ -207,8 +211,9 @@ public class FaceQualityDetDemo {
*/
@Test
public void evaluateResolution(){
try (FaceQualityModel faceQualityModel = getFaceQualityModel();
FaceDetModel faceDetModel = getFaceDetModel()){
try {
FaceQualityModel faceQualityModel = getFaceQualityModel();
FaceDetModel faceDetModel = getFaceDetModel();
//人脸检测
BufferedImage image = ImageIO.read(new File(Paths.get("src/main/resources/iu_1.jpg").toAbsolutePath().toString()));
R<DetectionResponse> detectionResponse = faceDetModel.detect(image);
@@ -240,8 +245,9 @@ public class FaceQualityDetDemo {
*/
@Test
public void evaluateAll(){
try (FaceQualityModel faceQualityModel = getFaceQualityModel();
FaceDetModel faceDetModel = getFaceDetModel()){
try {
FaceQualityModel faceQualityModel = getFaceQualityModel();
FaceDetModel faceDetModel = getFaceDetModel();
//人脸检测
BufferedImage image = ImageIO.read(new File(Paths.get("src/main/resources/iu_1.jpg").toAbsolutePath().toString()));
R<DetectionResponse> detectionResponse = faceDetModel.detect(image);

View File

@@ -12,7 +12,7 @@
<maven.compiler.source>11</maven.compiler.source>
<maven.compiler.target>11</maven.compiler.target>
<project.build.sourceEncoding>UTF-8</project.build.sourceEncoding>
<smartjavaai.version>1.0.20</smartjavaai.version>
<smartjavaai.version>1.0.22</smartjavaai.version>
<!--如果打包运行需要替换成你的main-->
<exec.mainClass>smartai.examples.objectdetection.ObjectDetection</exec.mainClass>

View File

@@ -41,10 +41,8 @@ import java.awt.image.BufferedImage;
import java.io.File;
import java.io.IOException;
import java.nio.file.Paths;
import java.util.ArrayList;
import java.util.Iterator;
import java.util.*;
import java.util.List;
import java.util.Objects;
import java.util.concurrent.Callable;
import java.util.concurrent.ExecutorService;
import java.util.concurrent.Executors;
@@ -71,7 +69,8 @@ public class ObjectDetection {
@Test
public void objectDetection(){
//默认cpu
try (DetectorModel detectorModel = ObjectDetectionModelFactory.getInstance().getModel()){
try {
DetectorModel detectorModel = ObjectDetectionModelFactory.getInstance().getModel();
DetectionResponse detectionResponse = detectorModel.detect("src/main/resources/object_detection.jpg");
log.info("目标检测结果:{}", JSONObject.toJSONString(detectionResponse));
} catch (Exception e) {
@@ -84,10 +83,15 @@ public class ObjectDetection {
*/
@Test
public void objectDetection2(){
DetectorModelConfig config = new DetectorModelConfig();
config.setModelEnum(DetectorModelEnum.SSD_300_RESNET50);//检测模型目前支持19种预置模型
config.setDevice(device);
try (DetectorModel detectorModel = ObjectDetectionModelFactory.getInstance().getModel(config)){
try {
DetectorModelConfig config = new DetectorModelConfig();
config.setModelEnum(DetectorModelEnum.SSD_300_RESNET50);//检测模型目前支持19种预置模型
// 指定允许的类别
// config.setAllowedClasses(Arrays.asList("person"));
//指定返回检测数量
config.setTopK(100);
config.setDevice(device);
DetectorModel detectorModel = ObjectDetectionModelFactory.getInstance().getModel(config);
DetectionResponse detectionResponse = detectorModel.detect("src/main/resources/dog_bike_car.jpg");
log.info("目标检测结果:{}", JSONObject.toJSONString(detectionResponse));
} catch (Exception e) {
@@ -100,7 +104,8 @@ public class ObjectDetection {
*/
@Test
public void objectDetectionAndDraw(){
try (DetectorModel detectorModel = ObjectDetectionModelFactory.getInstance().getModel()){
try {
DetectorModel detectorModel = ObjectDetectionModelFactory.getInstance().getModel();
detectorModel.detectAndDraw("src/main/resources/object_detection.jpg","output/object_detection_detected.png");
} catch (Exception e) {
e.printStackTrace();
@@ -112,7 +117,8 @@ public class ObjectDetection {
*/
@Test
public void objectDetectionAndDraw2(){
try (DetectorModel detectorModel = ObjectDetectionModelFactory.getInstance().getModel()){
try {
DetectorModel detectorModel = ObjectDetectionModelFactory.getInstance().getModel();
String imagePath = "src/main/resources/object_detection.jpg";
BufferedImage image = ImageIO.read(new File(Paths.get(imagePath).toAbsolutePath().toString()));
//可以根据后续业务场景使用detectedImage
@@ -131,15 +137,16 @@ public class ObjectDetection {
*/
@Test
public void objectDetectionWithOfficialModel(){
DetectorModelConfig config = new DetectorModelConfig();
config.setThreshold(0.3f);
//也支持YoloV8YOLOV8_OFFICIAL 模型可以从文档中提供的地址下载
config.setModelEnum(DetectorModelEnum.YOLOV12_OFFICIAL);//检测模型目前支持19种模型
// 指定模型路径,需要更改为自己的模型路径
config.setModelPath("/Users/xxx/Documents/yolov12n.onnx");
config.setDevice(device);
//一定要将yolo官方的类别文件synset.txt文档中下载放在模型同目录下否则报错
try (DetectorModel detectorModel = ObjectDetectionModelFactory.getInstance().getModel(config)){
try {
DetectorModelConfig config = new DetectorModelConfig();
config.setThreshold(0.3f);
//也支持YoloV8YOLOV8_OFFICIAL 模型可以从文档中提供的地址下载
config.setModelEnum(DetectorModelEnum.YOLOV12_OFFICIAL);//检测模型目前支持19种模型
// 指定模型路径,需要更改为自己的模型路径
config.setModelPath("/Users/xxx/Documents/yolov12n.onnx");
config.setDevice(device);
//一定要将yolo官方的类别文件synset.txt文档中下载放在模型同目录下否则报错
DetectorModel detectorModel = ObjectDetectionModelFactory.getInstance().getModel(config);
DetectionResponse detect = detectorModel.detect("src/main/resources/dog_bike_car.jpg");
log.info("目标检测结果:{}", JSONObject.toJSONString(detect));
} catch (Exception e) {
@@ -152,16 +159,21 @@ public class ObjectDetection {
*/
@Test
public void objectDetectionWithCustomModel(){
DetectorModelConfig config = new DetectorModelConfig();
//也支持YoloV8YOLOV8_CUSTOM 模型需要自己训练,训练教程可以查看文档
config.setModelEnum(DetectorModelEnum.YOLOV12_CUSTOM);//自定义YOLOV12模型
// 指定模型路径,需要更改为自己的模型路径
config.setModelPath("/Users/xxx/Documents/develop/fire_model/best.onnx");
config.putCustomParam("width", 640);//resize 宽
config.putCustomParam("height", 640);// resize
config.putCustomParam("nmsThreshold", 0.5f);
config.setDevice(device);
try (DetectorModel detectorModel = ObjectDetectionModelFactory.getInstance().getModel(config)){
try {
DetectorModelConfig config = new DetectorModelConfig();
//也支持YoloV8YOLOV8_CUSTOM 模型需要自己训练,训练教程可以查看文档
config.setModelEnum(DetectorModelEnum.YOLOV12_CUSTOM);//自定义YOLOV12模型
// 指定模型路径,需要更改为自己的模型路径
config.setModelPath("/Users/xxx/Documents/develop/fire_model/best.onnx");
config.putCustomParam("width", 640);//resize
config.putCustomParam("height", 640);// resize 高
config.putCustomParam("nmsThreshold", 0.5f);
// 指定允许的类别
// config.setAllowedClasses(Arrays.asList("person"));
//指定返回检测数量
config.setTopK(100);
config.setDevice(device);
DetectorModel detectorModel = ObjectDetectionModelFactory.getInstance().getModel(config);
DetectionResponse detect = detectorModel.detect("src/main/resources/dog_bike_car.jpg");
log.info("目标检测结果:{}", JSONObject.toJSONString(detect));
} catch (Exception e) {
@@ -176,7 +188,8 @@ public class ObjectDetection {
*/
@Test
public void testDetectCamera(){
try (DetectorModel detectorModel = ObjectDetectionModelFactory.getInstance().getModel()){
try {
DetectorModel detectorModel = ObjectDetectionModelFactory.getInstance().getModel();
OpenCV.loadShared();
VideoCapture capture = new VideoCapture(0);
if (!capture.isOpened()) {

View File

@@ -12,7 +12,7 @@
<maven.compiler.source>11</maven.compiler.source>
<maven.compiler.target>11</maven.compiler.target>
<project.build.sourceEncoding>UTF-8</project.build.sourceEncoding>
<smartjavaai.version>1.0.20</smartjavaai.version>
<smartjavaai.version>1.0.22</smartjavaai.version>
<!--如果打包运行需要替换成你的main-->
<exec.mainClass>smartai.examples.ocr.common.OcrRecognizeDemo</exec.mainClass>

View File

@@ -11,7 +11,6 @@ import cn.smartjavaai.ocr.entity.OcrInfo;
import cn.smartjavaai.ocr.enums.CommonDetModelEnum;
import cn.smartjavaai.ocr.factory.OcrModelFactory;
import cn.smartjavaai.ocr.model.common.detect.OcrCommonDetModel;
import cn.smartjavaai.ocr.opencv.OcrOpenCVUtils;
import cn.smartjavaai.ocr.utils.OcrUtils;
import com.alibaba.fastjson.JSONObject;
import lombok.extern.slf4j.Slf4j;
@@ -60,7 +59,8 @@ public class OcrDetectionDemo {
*/
@Test
public void detect(){
try (OcrCommonDetModel model = getDetectionModel()){
try {
OcrCommonDetModel model = getDetectionModel();
List<OcrBox> boxes = model.detect("src/main/resources/ocr_1.jpg");
log.info("OCR检测结果{}", JSONObject.toJSONString(boxes));
} catch (Exception e) {
@@ -77,7 +77,8 @@ public class OcrDetectionDemo {
*/
@Test
public void detectAndDraw(){
try (OcrCommonDetModel model = getDetectionModel()){
try {
OcrCommonDetModel model = getDetectionModel();
model.detectAndDraw("src/main/resources/ocr_1.jpg", "output/ocr_1_detected.jpg");
} catch (Exception e) {
e.printStackTrace();
@@ -94,7 +95,8 @@ public class OcrDetectionDemo {
*/
@Test
public void batchDetect(){
try (OcrCommonDetModel model = getDetectionModel()){
try {
OcrCommonDetModel model = getDetectionModel();
//批量检测要求图片宽高一致
String folderPath = "/Users/xxx/Downloads/testing33";
//读取文件夹中所有图片

View File

@@ -67,7 +67,8 @@ public class OcrDirectionDetDemo {
*/
@Test
public void detect(){
try (OcrDirectionModel directionModel = getDirectionModel()){
try {
OcrDirectionModel directionModel = getDirectionModel();
List<OcrItem> itemList = directionModel.detect("src/main/resources/ocr_1.jpg");
log.info("OCR方向检测结果1{}", JSONObject.toJSONString(itemList));
} catch (Exception e) {
@@ -84,7 +85,8 @@ public class OcrDirectionDetDemo {
*/
@Test
public void detectAndDraw(){
try (OcrDirectionModel directionModel = getDirectionModel()){
try {
OcrDirectionModel directionModel = getDirectionModel();
directionModel.detectAndDraw("src/main/resources/ocr_3.jpg", "output/ocr_3_detected.png");
} catch (Exception e) {
e.printStackTrace();

View File

@@ -106,7 +106,8 @@ public class OcrRecognizeDemo {
*/
@Test
public void recognize(){
try (OcrCommonRecModel recModel = getRecModel()){
try {
OcrCommonRecModel recModel = getRecModel();
//不带方向矫正,分行返回文本
OcrRecOptions options = new OcrRecOptions(false, true);
OcrInfo ocrInfo = recModel.recognize("src/main/resources/ocr_2.jpg",options);
@@ -127,7 +128,8 @@ public class OcrRecognizeDemo {
*/
@Test
public void recognizeHandWriting(){
try (OcrCommonRecModel recModel = getRecModel()){
try {
OcrCommonRecModel recModel = getRecModel();
OcrInfo ocrInfo = recModel.recognize("src/main/resources/handwriting_1.jpg",new OcrRecOptions());
log.info("OCR识别结果{}", JSONObject.toJSONString(ocrInfo));
} catch (Exception e) {
@@ -146,7 +148,8 @@ public class OcrRecognizeDemo {
*/
@Test
public void recognize2(){
try (OcrCommonRecModel recModel = getRecModelWithDirection()){
try {
OcrCommonRecModel recModel = getRecModelWithDirection();
//带方向矫正,分行返回文本
OcrRecOptions options = new OcrRecOptions(true, true);
OcrInfo ocrInfo = recModel.recognize("src/main/resources/ocr_3.jpg",options);
@@ -168,7 +171,8 @@ public class OcrRecognizeDemo {
*/
@Test
public void recognizeAndDraw(){
try (OcrCommonRecModel recModel = getRecModelWithDirection()){
try {
OcrCommonRecModel recModel = getRecModelWithDirection();
int fontSize = 18;
recModel.recognizeAndDraw("src/main/resources/general_ocr_002.png", "output/ocr_4_recognized.jpg", fontSize, new OcrRecOptions());
} catch (Exception e) {
@@ -184,7 +188,8 @@ public class OcrRecognizeDemo {
*/
@Test
public void batchRecognize(){
try (OcrCommonRecModel recModel = getRecModelWithDirection()){
try {
OcrCommonRecModel recModel = getRecModelWithDirection();
//批量检测要求图片宽高一致
String folderPath = "/Users/xxx/Downloads/testing33";
//读取文件夹中所有图片

View File

@@ -0,0 +1,81 @@
package smartai.examples.ocr.plate;
import ai.djl.util.JsonUtils;
import cn.smartjavaai.common.entity.R;
import cn.smartjavaai.common.enums.DeviceEnum;
import cn.smartjavaai.common.utils.ImageUtils;
import cn.smartjavaai.ocr.config.PlateDetModelConfig;
import cn.smartjavaai.ocr.config.PlateRecModelConfig;
import cn.smartjavaai.ocr.entity.PlateInfo;
import cn.smartjavaai.ocr.enums.PlateDetModelEnum;
import cn.smartjavaai.ocr.enums.PlateRecModelEnum;
import cn.smartjavaai.ocr.factory.PlateModelFactory;
import cn.smartjavaai.ocr.model.plate.PlateDetModel;
import cn.smartjavaai.ocr.model.plate.PlateRecModel;
import lombok.extern.slf4j.Slf4j;
import org.junit.Test;
import java.io.File;
import java.util.List;
/**
* @author dwj
*/
@Slf4j
public class PlateRecDemo {
//设备类型
public static DeviceEnum device = DeviceEnum.CPU;
/**
* 获取车牌检测模型
* @return
*/
public PlateDetModel getPlateDetModel() {
PlateDetModelConfig config = new PlateDetModelConfig();
config.setModelEnum(PlateDetModelEnum.YOLOV5);
config.setModelPath("/Users/wenjie/Downloads/modelscope-2b3747db53adb48370c6edaccac56eb1ca0da1b5/onnx_export/model.onnx");
// config.setModelPath("/Users/wenjie/Documents/develop/model/plate/yolov8s.onnx");
config.setModelPath("/Users/wenjie/Downloads/数据集/车牌/onnx/plate_detect.onnx");
config.setPredictorPoolSize(3);
config.setDevice(device);
return PlateModelFactory.getInstance().getDetModel(config);
}
/**
* 获取车牌识别模型
* @return
*/
public PlateRecModel getPlateRecModel() {
PlateRecModelConfig recModelConfig = new PlateRecModelConfig();
recModelConfig.setModelEnum(PlateRecModelEnum.PLATE_REC_CRNN);
recModelConfig.setModelPath("/Users/wenjie/Downloads/数据集/车牌/onnx/plate_rec_color.onnx");
recModelConfig.setPlateDetModel(getPlateDetModel());
return PlateModelFactory.getInstance().getRecModel(recModelConfig);
}
@Test
public void testDetect() {
PlateRecModel plateRecModel = getPlateRecModel();
R<List<PlateInfo>> result = plateRecModel.recognize("src/main/resources/plate/Quicker_20220930_180856.png");
if(result.isSuccess()){
log.info("车牌识别结果:{}", JsonUtils.toJson(result.getData()));
}else{
log.error("车牌识别失败:{}", result.getMessage());
}
}
@Test
public void recognizeAndDraw() {
PlateRecModel plateRecModel = getPlateRecModel();
R<Void> result = plateRecModel.recognizeAndDraw("src/main/resources/plate/single_green.jpg", "output/plate_recognized2.jpg");
if(result.isSuccess()){
log.info("车牌识别成功");
}else{
log.error("车牌识别失败:{}", result.getMessage());
}
}
}

View File

@@ -113,10 +113,11 @@ public class TableRecDemo {
*/
@Test
public void recognize(){
try (TableStructureModel tableStructureModel = getTableStructureModel();
OcrCommonDetModel detModel = getDetectionModel();
OcrCommonRecModel recModel = getRecModel();
OcrDirectionModel directionModel = getDirectionModel()){
try {
TableStructureModel tableStructureModel = getTableStructureModel();
OcrCommonDetModel detModel = getDetectionModel();
OcrCommonRecModel recModel = getRecModel();
OcrDirectionModel directionModel = getDirectionModel();
//创建表格识别器
TableRecognizer tableRecognizer = TableRecognizer.builder()
.withStructureModel(tableStructureModel)

Binary file not shown.

After

Width:  |  Height:  |  Size: 1.4 MiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 1.0 MiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 241 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 328 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 29 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 571 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 400 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 382 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 47 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 1.8 MiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 903 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 85 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 932 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 513 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 999 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 943 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 219 KiB

View File

@@ -12,7 +12,7 @@
<maven.compiler.source>11</maven.compiler.source>
<maven.compiler.target>11</maven.compiler.target>
<project.build.sourceEncoding>UTF-8</project.build.sourceEncoding>
<smartjavaai.version>1.0.20</smartjavaai.version>
<smartjavaai.version>1.0.22</smartjavaai.version>
<!--如果打包运行需要替换成你的main-->
<exec.mainClass>smartai.examples.nlp.translation.TranslationDemo</exec.mainClass>

View File

@@ -7,7 +7,7 @@
<name>SmartJavaAI</name>
<groupId>cn.smartjavaai</groupId>
<artifactId>smartjavaai-parent</artifactId>
<version>1.0.20</version>
<version>1.0.22</version>
<packaging>pom</packaging>
<description>SmartJavaAI</description>
<modules>

View File

@@ -6,11 +6,11 @@
<parent>
<groupId>cn.smartjavaai</groupId>
<artifactId>smartjavaai-parent</artifactId>
<version>1.0.20</version>
<version>1.0.22</version>
</parent>
<artifactId>smartjavaai-all</artifactId>
<version>1.0.20</version>
<version>1.0.22</version>
<name>${project.artifactId}</name>
<description>SmartJavaAI</description>
<url>https://github.com/geekwenjie/SmartJavaAI</url>

View File

@@ -6,10 +6,10 @@
<parent>
<groupId>cn.smartjavaai</groupId>
<artifactId>smartjavaai-parent</artifactId>
<version>1.0.20</version>
<version>1.0.22</version>
</parent>
<version>1.0.20</version>
<version>1.0.22</version>
<artifactId>smartjavaai-bom</artifactId>
<name>smartjavaai-bom</name>
<description>统一版本管理的 BOM 包,同时支持 import 和全量依赖</description>

View File

@@ -6,7 +6,7 @@
<parent>
<groupId>cn.smartjavaai</groupId>
<artifactId>smartjavaai-parent</artifactId>
<version>1.0.20</version>
<version>1.0.22</version>
</parent>
<name>smartjavaai-common</name>

View File

@@ -29,6 +29,11 @@ public class ModelConfig {
*/
private String batchifier;
/**
* 模型预测器池大小(默认为cpu核心数)
*/
private int predictorPoolSize;
/**
* 个性化配置(按模型类型动态解析)
*/

View File

@@ -57,6 +57,7 @@ public class R<T> {
NO_FACE_DETECTED(3, "未检测到人脸"),
PARAM_ERROR(4, "参数错误"),
INVALID_VIDEO(5, "视频无效"),
NO_OBJECT_DETECTED(6, "未检测到目标"),
Unknown(-1, "未知错误");
private final int code;

View File

@@ -3,6 +3,7 @@ package cn.smartjavaai.common.pool;
import ai.djl.inference.Predictor;
import ai.djl.repository.zoo.ZooModel;
import ai.djl.translate.Translator;
import lombok.extern.slf4j.Slf4j;
import org.apache.commons.pool2.BasePooledObjectFactory;
import org.apache.commons.pool2.PooledObject;
import org.apache.commons.pool2.impl.DefaultPooledObject;
@@ -12,6 +13,7 @@ import org.apache.commons.pool2.impl.DefaultPooledObject;
* @author dwj
* @date 2025/4/8
*/
@Slf4j
public class PredictorFactory<I, O> extends BasePooledObjectFactory<Predictor<I, O>> {
private final ZooModel<I, O> model;
@@ -21,6 +23,7 @@ public class PredictorFactory<I, O> extends BasePooledObjectFactory<Predictor<I,
@Override
public Predictor<I, O> create() {
log.debug("create predictor");
return model.newPredictor();
}
@@ -31,6 +34,7 @@ public class PredictorFactory<I, O> extends BasePooledObjectFactory<Predictor<I,
@Override
public void destroyObject(PooledObject<Predictor<I, O>> p) {
log.debug("close predictor");
p.getObject().close();
}
}

View File

@@ -8,6 +8,9 @@ import ai.djl.ndarray.NDArray;
import cn.smartjavaai.common.entity.DetectionRectangle;
import cn.smartjavaai.common.entity.DetectionResponse;
import org.opencv.core.Mat;
import org.opencv.core.Scalar;
import org.opencv.core.Size;
import org.opencv.imgproc.Imgproc;
import javax.imageio.ImageIO;
import java.awt.*;
@@ -364,6 +367,7 @@ public class ImageUtils {
int width = metrics.stringWidth(text) + padding * 2 - stroke / 2;
int height = metrics.getHeight() + metrics.getDescent();
int ascent = metrics.getAscent();
y = Math.max(0, y - height);
java.awt.Rectangle background = new java.awt.Rectangle(x, y, width, height);
g.fill(background);
g.setPaint(Color.WHITE);
@@ -468,6 +472,33 @@ public class ImageUtils {
return true;
}
/**
* 在图像上绘制带白色背景、黑色文字的文本
*/
public static void putTextWithBackground(Mat image, String text, org.opencv.core.Point origin, Scalar textColor, Scalar backgroundColor, int padding) {
// 默认字体
int font = Imgproc.FONT_HERSHEY_SCRIPT_SIMPLEX;
// 默认字体缩放大小
double fontScale = 1.0;
//线条粗细
int thickness = 2;
//获取文字大小
int[] baseLine = new int[1];
Size textSize = Imgproc.getTextSize(text, font, fontScale, thickness, baseLine);
int textWidth = (int) textSize.width;
int textHeight = (int) textSize.height;
//计算带padding的背景框
org.opencv.core.Point bgTopLeft = new org.opencv.core.Point(origin.x - padding, origin.y - textHeight - padding);
org.opencv.core.Point bgBottomRight = new org.opencv.core.Point(origin.x + textWidth + padding, origin.y + baseLine[0] + padding);
//绘制背景矩形
Imgproc.rectangle(image, bgTopLeft, bgBottomRight, backgroundColor, Imgproc.FILLED);
//绘制文字(黑色)
Imgproc.putText(image, text, origin, font, fontScale, textColor, thickness);
}
}

View File

@@ -0,0 +1,124 @@
package cn.smartjavaai.common.utils;
import ai.djl.modality.cv.util.NDImageUtils;
import ai.djl.ndarray.NDArray;
import ai.djl.ndarray.NDManager;
import ai.djl.ndarray.index.NDIndex;
import ai.djl.ndarray.types.DataType;
import ai.djl.ndarray.types.Shape;
import java.util.Arrays;
/**
* 按比例缩放,剩余空间用指定颜色填充
* @author dwj
*/
public class LetterBoxUtils {
public enum PaddingPosition {
CENTER, LEFT_TOP, RIGHT_BOTTOM
}
public static class ResizeResult {
public NDArray image;
public float r;
public int left;
public int top;
}
public static ResizeResult letterboxWithMeta(NDArray paddingImg, float r, int left, int top) {
// ... letterbox 逻辑不变
ResizeResult result = new ResizeResult();
result.image = paddingImg;
result.r = r;
result.left = left;
result.top = top;
return result;
}
/**
* 按比例缩放 + padding
*
* @param img 原图 NDArray HWC
* @param targetW 目标宽度
* @param targetH 目标高度
* @param padColor padding 填充颜色RGB 归一化 0-1
* @param position padding 位置CENTER / LEFT_TOP / RIGHT_BOTTOM
* @return 处理后的 NDArray
*/
public static ResizeResult letterbox(NDManager manager, NDArray img, int targetW, int targetH, float padColor, PaddingPosition position) {
long origH = img.getShape().get(0);
long origW = img.getShape().get(1);
float r = Math.min(targetW / (float) origW, targetH / (float) origH);
int newW = Math.round(origW * r);
int newH = Math.round(origH * r);
img = NDImageUtils.resize(img, newW, newH); // HWC 0~1
// NDArray paddingImg = manager
// .full(new Shape(targetW, targetH, 3), padColor, DataType.UINT8);
NDArray paddingImg = manager.zeros(new Shape(targetW, targetH, 3), DataType.FLOAT32);
paddingImg = paddingImg.add(114);
int padW = targetW - newW;
int padH = targetH - newH;
int top = 0, left = 0;
switch (position) {
case CENTER:
left = padW / 2;
top = padH / 2;
break;
case LEFT_TOP:
left = 0;
top = 0;
break;
case RIGHT_BOTTOM:
left = padW;
top = padH;
break;
}
paddingImg.set(new NDIndex(String.format("%d:%d,%d:%d", top, top + newH, left, left + newW)), img);
return letterboxWithMeta(paddingImg, r, left, top);
}
/**
* 恢复缩放后的 box
* @param boxes
* @param scaleRatio
* @param left
* @param top
* @param keypointStart
* @param keypointDim
* @return
*/
public static NDArray restoreBox(NDArray boxes, float scaleRatio, float left, float top, int keypointStart, int keypointDim) {
// 处理 bbox
NDArray x1 = boxes.get(":, 0").sub(left).div(scaleRatio);
NDArray y1 = boxes.get(":, 1").sub(top).div(scaleRatio);
NDArray x2 = boxes.get(":, 2").sub(left).div(scaleRatio);
NDArray y2 = boxes.get(":, 3").sub(top).div(scaleRatio);
boxes.set(new NDIndex(":, 0"), x1);
boxes.set(new NDIndex(":, 1"), y1);
boxes.set(new NDIndex(":, 2"), x2);
boxes.set(new NDIndex(":, 3"), y2);
if (keypointDim > 0) {
for (int i = 0; i < keypointDim; i += 2) {
int xIdx = keypointStart + i;
int yIdx = keypointStart + i + 1;
NDArray keyX = boxes.get(":, " + xIdx).sub(left).div(scaleRatio);
NDArray keyY = boxes.get(":, " + yIdx).sub(top).div(scaleRatio);
boxes.set(new NDIndex(":, " + xIdx), keyX);
boxes.set(new NDIndex(":, " + yIdx), keyY);
}
}
return boxes;
}
}

View File

@@ -0,0 +1,67 @@
package cn.smartjavaai.common.utils;
import ai.djl.ndarray.NDArray;
import java.util.ArrayList;
import java.util.List;
/**
* @author dwj
* @date 2025/7/23
*/
public class NMSUtils {
/**
* 通用 NMS 方法,输入 NDArray 形式的 boxes 和 scores返回保留的索引列表
*
* @param boxes NDArray 形状为 (N, 4),格式为 [x1, y1, x2, y2]
* @param scores NDArray 形状为 (N,) 或 (N,1),每个 box 的置信度
* @param iouThreshold IOU 阈值,超过该阈值则认为有重叠
* @return 保留框的索引列表
*/
public static int[] nms(NDArray boxes, NDArray scores, float iouThreshold) {
if (boxes.isEmpty()) {
return new int[0];
}
NDArray x1 = boxes.get(":, 0");
NDArray y1 = boxes.get(":, 1");
NDArray x2 = boxes.get(":, 2");
NDArray y2 = boxes.get(":, 3");
NDArray areas = x2.sub(x1).add(1).mul(y2.sub(y1).add(1));
// 按照置信度降序排序
NDArray order = scores.argSort().flip(0);
List<Integer> keep = new ArrayList<>();
while (order.size() > 0) {
int idx = (int)order.getLong(0);
keep.add(idx);
if (order.size() == 1) break;
NDArray currentBox = boxes.get(idx);
NDArray others = boxes.get(order);
NDArray xx1 = x1.get(order).maximum(x1.get(idx));
NDArray yy1 = y1.get(order).maximum(y1.get(idx));
NDArray xx2 = x2.get(order).minimum(x2.get(idx));
NDArray yy2 = y2.get(order).minimum(y2.get(idx));
NDArray w = xx2.sub(xx1).add(1).maximum(0);
NDArray h = yy2.sub(yy1).add(1).maximum(0);
NDArray inter = w.mul(h);
NDArray remAreas = areas.get(order);
NDArray union = remAreas.add(areas.get(idx)).sub(inter);
NDArray iou = inter.div(union);
NDArray mask = iou.lte(iouThreshold);
order = order.get(mask);
}
return keep.stream().mapToInt(i -> i).toArray();
}
}

View File

@@ -124,4 +124,20 @@ public class OpenCVUtils {
mat.put(0, 0, data);
return mat;
}
/**
* 透视变换
*
* @param src
* @param srcPoints
* @param dstPoints
* @return
*/
public static Mat perspectiveTransform(Mat src, Mat srcPoints, Mat dstPoints) {
Mat dst = src.clone();
Mat warp_mat = Imgproc.getPerspectiveTransform(srcPoints, dstPoints);
Imgproc.warpPerspective(src, dst, warp_mat, dst.size());
warp_mat.release();
return dst;
}
}

View File

@@ -0,0 +1,70 @@
package cn.smartjavaai.common.utils;
import ai.djl.modality.cv.output.Point;
import java.util.Arrays;
import java.util.Comparator;
import java.util.List;
/**
* @author dwj
*/
public class PointUtils {
/**
* 对 4 个关键点进行排序,顺序为:
* 左上、右上、右下、左下
*/
public static List<Point> orderPoints(List<Point> points) {
if (points == null || points.size() != 4) {
throw new IllegalArgumentException("必须提供 4 个点");
}
// 按 X 坐标升序排列
points.sort(Comparator.comparingDouble(Point::getX));
List<Point> left = points.subList(0, 2);
List<Point> right = points.subList(2, 4);
// 左侧两点按 Y 排序:上为 tl下为 bl
Point tl = left.get(0).getY() < left.get(1).getY() ? left.get(0) : left.get(1);
Point bl = left.get(0).getY() >= left.get(1).getY() ? left.get(0) : left.get(1);
// 右侧两点按 Y 排序:上为 tr下为 br
Point tr = right.get(0).getY() < right.get(1).getY() ? right.get(0) : right.get(1);
Point br = right.get(0).getY() >= right.get(1).getY() ? right.get(0) : right.get(1);
return Arrays.asList(tl, tr, br, bl);
}
/**
* 欧式距离计算
*
* @param point1
* @param point2
* @return
*/
public static float distance(float[] point1, float[] point2) {
float disX = point1[0] - point2[0];
float disY = point1[1] - point2[1];
float dis = (float) Math.sqrt(disX * disX + disY * disY);
return dis;
}
/**
* 欧式距离计算
*
* @param point1
* @param point2
* @return
*/
public static float distance(Point point1, Point point2) {
double disX = point1.getX() - point2.getX();
double disY = point1.getY() - point2.getY();
float dis = (float) Math.sqrt(disX * disX + disY * disY);
return dis;
}
}

View File

@@ -6,11 +6,11 @@
<parent>
<groupId>cn.smartjavaai</groupId>
<artifactId>smartjavaai-parent</artifactId>
<version>1.0.20</version>
<version>1.0.22</version>
</parent>
<artifactId>smartjavaai-face</artifactId>
<version>1.0.20</version>
<version>1.0.22</version>
<name>smartjavaai-face</name>
<description>SmartJavaAI</description>
<url>https://github.com/geekwenjie/SmartJavaAI</url>

View File

@@ -1,5 +1,6 @@
package cn.smartjavaai.face.config;
import cn.smartjavaai.common.config.ModelConfig;
import cn.smartjavaai.common.enums.DeviceEnum;
import cn.smartjavaai.face.enums.FaceAttributeModelEnum;
import lombok.Data;
@@ -9,7 +10,7 @@ import lombok.Data;
* @author dwj
*/
@Data
public class FaceAttributeConfig {
public class FaceAttributeConfig extends ModelConfig {
/**
* 人脸属性识别模型枚举
@@ -21,16 +22,6 @@ public class FaceAttributeConfig {
*/
private String modelPath;
/**
* 设备类型
*/
private DeviceEnum device;
/**
* gpu设备ID 当device为GPU时生效
*/
private int gpuId = 0;
/**
* 是否启用年龄检测
*/

View File

@@ -1,5 +1,6 @@
package cn.smartjavaai.face.config;
import cn.smartjavaai.common.config.ModelConfig;
import cn.smartjavaai.common.enums.DeviceEnum;
import cn.smartjavaai.face.constant.FaceDetectConstant;
import cn.smartjavaai.face.enums.FaceDetModelEnum;
@@ -13,7 +14,7 @@ import java.util.Map;
* @author dwj
*/
@Data
public class FaceDetConfig {
public class FaceDetConfig extends ModelConfig {
/**
* 人脸检测模型枚举
@@ -36,16 +37,6 @@ public class FaceDetConfig {
*/
private String modelPath;
/**
* 设备类型
*/
private DeviceEnum device;
/**
* 个性化配置(按模型类型动态解析)
*/
private Map<String, Object> customParams = new HashMap<>();
public FaceDetConfig() {
}
@@ -59,19 +50,4 @@ public class FaceDetConfig {
this.modelPath = modelPath;
}
public <T> T getCustomParam(String key, Class<T> clazz) {
Object value = customParams.get(key);
if (value == null) return null;
return clazz.cast(value);
}
/**
* 添加个性化配置项
*/
public void putCustomParam(String key, Object value) {
if (customParams == null) {
customParams = new HashMap<>();
}
customParams.put(key, value);
}
}

View File

@@ -1,5 +1,6 @@
package cn.smartjavaai.face.config;
import cn.smartjavaai.common.config.ModelConfig;
import cn.smartjavaai.common.enums.DeviceEnum;
import cn.smartjavaai.face.enums.ExpressionModelEnum;
import cn.smartjavaai.face.model.facedect.FaceDetModel;
@@ -14,7 +15,7 @@ import java.util.Map;
* @date 2025/7/1
*/
@Data
public class FaceExpressionConfig {
public class FaceExpressionConfig extends ModelConfig {
/**
* 模型枚举
@@ -26,11 +27,6 @@ public class FaceExpressionConfig {
*/
private String modelPath;
/**
* 设备类型
*/
private DeviceEnum device;
/**
* 人脸检测模型
*/

View File

@@ -1,5 +1,6 @@
package cn.smartjavaai.face.config;
import cn.smartjavaai.common.config.ModelConfig;
import cn.smartjavaai.common.enums.DeviceEnum;
import cn.smartjavaai.face.constant.FaceDetectConstant;
import cn.smartjavaai.face.enums.FaceRecModelEnum;
@@ -16,7 +17,7 @@ import java.util.Map;
* @author dwj
*/
@Data
public class FaceRecConfig {
public class FaceRecConfig extends ModelConfig {
/**
* 人脸模型枚举
@@ -28,11 +29,6 @@ public class FaceRecConfig {
*/
private String modelPath;
/**
* 设备类型
*/
private DeviceEnum device;
/**
* 向量数据库配置
@@ -60,10 +56,6 @@ public class FaceRecConfig {
*/
private FaceDetModel detectModel;
/**
* 个性化配置(按模型类型动态解析)
*/
private Map<String, Object> customParams = new HashMap<>();
public FaceRecConfig() {
}
@@ -77,19 +69,5 @@ public class FaceRecConfig {
this.modelPath = modelPath;
}
public <T> T getCustomParam(String key, Class<T> clazz) {
Object value = customParams.get(key);
if (value == null) return null;
return clazz.cast(value);
}
/**
* 添加个性化配置项
*/
public void putCustomParam(String key, Object value) {
if (customParams == null) {
customParams = new HashMap<>();
}
customParams.put(key, value);
}
}

View File

@@ -1,5 +1,6 @@
package cn.smartjavaai.face.config;
import cn.smartjavaai.common.config.ModelConfig;
import cn.smartjavaai.common.enums.DeviceEnum;
import cn.smartjavaai.face.constant.LivenessConstant;
import cn.smartjavaai.face.enums.LivenessModelEnum;
@@ -15,7 +16,7 @@ import java.util.Map;
* @author dwj
*/
@Data
public class LivenessConfig {
public class LivenessConfig extends ModelConfig {
/**
* 活体检测模型枚举
@@ -27,20 +28,12 @@ public class LivenessConfig {
*/
private String modelPath;
/**
* 设备类型
*/
private DeviceEnum device;
/**
* 人脸检测模型
*/
private FaceDetModel detectModel;
/**
* 个性化配置(按模型类型动态解析)
*/
private Map<String, Object> customParams = new HashMap<>();
/**
@@ -74,20 +67,4 @@ public class LivenessConfig {
this.modelPath = modelPath;
}
// 可选封装方法,便于类型转换和调用
public <T> T getCustomParam(String key, Class<T> clazz) {
Object value = customParams.get(key);
if (value == null) return null;
return clazz.cast(value);
}
/**
* 添加个性化配置项
*/
public void putCustomParam(String key, Object value) {
if (customParams == null) {
customParams = new HashMap<>();
}
customParams.put(key, value);
}
}

View File

@@ -1,5 +1,6 @@
package cn.smartjavaai.face.config;
import cn.smartjavaai.common.config.ModelConfig;
import cn.smartjavaai.common.enums.DeviceEnum;
import cn.smartjavaai.face.constant.LivenessConstant;
import cn.smartjavaai.face.enums.LivenessModelEnum;
@@ -11,7 +12,7 @@ import lombok.Data;
* @author dwj
*/
@Data
public class QualityConfig {
public class QualityConfig extends ModelConfig {
/**
* 活体检测模型枚举
@@ -23,16 +24,6 @@ public class QualityConfig {
*/
private String modelPath;
/**
* 设备类型
*/
private DeviceEnum device;
/**
* gpu设备ID 当device为GPU时生效
*/
private int gpuId = 0;
public QualityConfig() {
}

View File

@@ -1,5 +1,6 @@
package cn.smartjavaai.face.model.attribute;
import ai.djl.engine.Engine;
import cn.smartjavaai.common.entity.*;
import cn.smartjavaai.common.entity.face.FaceAttribute;
import cn.smartjavaai.common.entity.face.FaceInfo;
@@ -102,6 +103,19 @@ public class Seetaface6FaceAttributeModel implements FaceAttributeModel {
this.eyeStateDetectorPool = new EyeStateDetectorPool(eyeStateDetectorPoolConfSetting);
this.maskDetectorPool = new MaskDetectorPool(maskDetectorPoolConfSetting);
this.poseEstimatorPool = new PoseEstimatorPool(poseEstimatorPoolConfSetting);
int predictorPoolSize = config.getPredictorPoolSize();
if(config.getPredictorPoolSize() <= 0){
predictorPoolSize = Runtime.getRuntime().availableProcessors(); // 默认等于CPU核心数
}
faceDetectorPool.setMaxTotal(predictorPoolSize);
faceLandmarkerPool.setMaxTotal(predictorPoolSize);
genderPredictorPool.setMaxTotal(predictorPoolSize);
agePredictorPool.setMaxTotal(predictorPoolSize);
eyeStateDetectorPool.setMaxTotal(predictorPoolSize);
maskDetectorPool.setMaxTotal(predictorPoolSize);
poseEstimatorPool.setMaxTotal(predictorPoolSize);
log.debug("模型推理器线程池最大数量: " + predictorPoolSize);
} catch (FileNotFoundException e) {
throw new FaceException(e);
}
@@ -471,6 +485,35 @@ public class Seetaface6FaceAttributeModel implements FaceAttributeModel {
}
}
public FaceDetectorPool getFaceDetectorPool() {
return faceDetectorPool;
}
public GenderPredictorPool getGenderPredictorPool() {
return genderPredictorPool;
}
public FaceLandmarkerPool getFaceLandmarkerPool() {
return faceLandmarkerPool;
}
public AgePredictorPool getAgePredictorPool() {
return agePredictorPool;
}
public EyeStateDetectorPool getEyeStateDetectorPool() {
return eyeStateDetectorPool;
}
public MaskDetectorPool getMaskDetectorPool() {
return maskDetectorPool;
}
public PoseEstimatorPool getPoseEstimatorPool() {
return poseEstimatorPool;
}
@Override
public void close() throws Exception {
if(Objects.nonNull(faceDetectorPool)){

View File

@@ -2,6 +2,7 @@ package cn.smartjavaai.face.model.expression;
import ai.djl.Device;
import ai.djl.MalformedModelException;
import ai.djl.engine.Engine;
import ai.djl.inference.Predictor;
import ai.djl.modality.Classifications;
import ai.djl.modality.cv.Image;
@@ -32,6 +33,7 @@ import lombok.extern.slf4j.Slf4j;
import org.apache.commons.lang3.StringUtils;
import org.apache.commons.pool2.ObjectPool;
import org.apache.commons.pool2.impl.GenericObjectPool;
import org.opencv.core.Mat;
import org.opencv.face.Face;
import javax.imageio.ImageIO;
@@ -56,7 +58,7 @@ public class CommonEmotionModel implements ExpressionModel{
private ZooModel<Image, Classifications> model;
private ObjectPool<Predictor<Image, Classifications>> predictorPool;
private GenericObjectPool<Predictor<Image, Classifications>> predictorPool;
@Override
public void loadModel(FaceExpressionConfig config) {
@@ -73,6 +75,14 @@ public class CommonEmotionModel implements ExpressionModel{
try {
model = criteria.loadModel();
this.predictorPool = new GenericObjectPool<>(new PredictorFactory<>(model));
int predictorPoolSize = config.getPredictorPoolSize();
if(config.getPredictorPoolSize() <= 0){
predictorPoolSize = Runtime.getRuntime().availableProcessors(); // 默认等于CPU核心数
}
predictorPool.setMaxTotal(predictorPoolSize);
log.debug("当前设备: " + model.getNDManager().getDevice());
log.debug("当前引擎: " + Engine.getInstance().getEngineName());
log.debug("模型推理器线程池最大数量: " + predictorPoolSize);
} catch (IOException | ModelNotFoundException | MalformedModelException e) {
throw new FaceException("DenseNetEmotionModel模型加载失败", e);
}
@@ -153,6 +163,7 @@ public class CommonEmotionModel implements ExpressionModel{
result.setClassifications(classifications);
faceInfo.setExpressionResult(result);
}
((Mat)djlImage.getWrappedImage()).release();
return faceDetectionResponse;
}
@@ -226,6 +237,7 @@ public class CommonEmotionModel implements ExpressionModel{
result.setClassifications(classifications);
expressionResults.add(result);
}
((Mat)djlImage.getWrappedImage()).release();
return R.ok(expressionResults);
}
@@ -276,6 +288,7 @@ public class CommonEmotionModel implements ExpressionModel{
FacialExpression expression = FacialExpression.fromLabel(bestClass.getClassName());
ExpressionResult result = new ExpressionResult(expression, (float)bestClass.getProbability());
result.setClassifications(classifications);
((Mat)djlImage.getWrappedImage()).release();
return R.ok(result);
}
@@ -342,7 +355,10 @@ public class CommonEmotionModel implements ExpressionModel{
return detectTopFace(imageData);
}
@Override
public GenericObjectPool<Predictor<Image, Classifications>> getPool() {
return predictorPool;
}
@Override
public void close() {

View File

@@ -1,11 +1,16 @@
package cn.smartjavaai.face.model.expression;
import ai.djl.inference.Predictor;
import ai.djl.modality.Classifications;
import ai.djl.modality.cv.Image;
import ai.djl.modality.cv.output.DetectedObjects;
import cn.smartjavaai.common.entity.DetectionRectangle;
import cn.smartjavaai.common.entity.DetectionResponse;
import cn.smartjavaai.common.entity.Point;
import cn.smartjavaai.common.entity.R;
import cn.smartjavaai.common.entity.face.ExpressionResult;
import cn.smartjavaai.face.config.FaceExpressionConfig;
import org.apache.commons.pool2.impl.GenericObjectPool;
import java.awt.image.BufferedImage;
import java.util.List;
@@ -190,6 +195,9 @@ public interface ExpressionModel extends AutoCloseable{
default GenericObjectPool<Predictor<Image, Classifications>> getPool(){
throw new UnsupportedOperationException("默认不支持该功能");
}
}

View File

@@ -26,7 +26,7 @@ public class EmotionCriteriaFactory {
public static Criteria<Image, Classifications> createCriteria(FaceExpressionConfig config) {
Device device = null;
if(!Objects.isNull(config.getDevice())){
device = config.getDevice() == DeviceEnum.CPU ? Device.cpu() : Device.gpu();
device = config.getDevice() == DeviceEnum.CPU ? Device.cpu() : Device.gpu(config.getGpuId());
}
Criteria<Image, Classifications> criteria = null;
if(config.getModelEnum() == ExpressionModelEnum.DensNet121){
@@ -50,6 +50,7 @@ public class EmotionCriteriaFactory {
.optModelPath(Paths.get(config.getModelPath()))
.optTranslator(new FrEmotionTranslator(224))
.optProgress(new ProgressBar())
.optDevice(device)
.build();
}
return criteria;

View File

@@ -24,6 +24,7 @@ import lombok.extern.slf4j.Slf4j;
import org.apache.commons.lang3.StringUtils;
import org.apache.commons.pool2.ObjectPool;
import org.apache.commons.pool2.impl.GenericObjectPool;
import org.opencv.core.Mat;
import javax.imageio.ImageIO;
import java.awt.image.BufferedImage;
@@ -43,7 +44,7 @@ import java.util.Objects;
@Slf4j
public class CommonFaceDetModel implements FaceDetModel{
private ObjectPool<Predictor<Image, DetectedObjects>> predictorPool;
private GenericObjectPool<Predictor<Image, DetectedObjects>> predictorPool;
private ZooModel<Image, DetectedObjects> model;
@@ -58,8 +59,14 @@ public class CommonFaceDetModel implements FaceDetModel{
try {
model = criteria.loadModel();
this.predictorPool = new GenericObjectPool<>(new PredictorFactory<>(model));
int predictorPoolSize = config.getPredictorPoolSize();
if(config.getPredictorPoolSize() <= 0){
predictorPoolSize = Runtime.getRuntime().availableProcessors(); // 默认等于CPU核心数
}
predictorPool.setMaxTotal(predictorPoolSize);
log.debug("当前设备: " + model.getNDManager().getDevice());
log.debug("当前引擎: " + Engine.getInstance().getEngineName());
log.debug("模型推理器线程池最大数量: " + predictorPoolSize);
} catch (IOException | ModelNotFoundException | MalformedModelException e) {
throw new FaceException("人脸检测模型加载失败", e);
}
@@ -81,11 +88,16 @@ public class CommonFaceDetModel implements FaceDetModel{
Image img = null;
try {
img = ImageFactory.getInstance().fromFile(Paths.get(imagePath));
DetectedObjects detection = detect(img);
return R.ok(FaceUtils.convertToDetectionResponse(detection,img));
} catch (IOException e) {
throw new FaceException("无效的图片", e);
} finally {
if (img != null) {
((Mat)img.getWrappedImage()).release();
}
}
DetectedObjects detection = detect(img);
return R.ok(FaceUtils.convertToDetectionResponse(detection,img));
}
/**
@@ -99,12 +111,17 @@ public class CommonFaceDetModel implements FaceDetModel{
if(Objects.isNull(imageInputStream)){
return R.fail(R.Status.INVALID_IMAGE);
}
Image img = null;
try {
Image img = ImageFactory.getInstance().fromInputStream(imageInputStream);
img = ImageFactory.getInstance().fromInputStream(imageInputStream);
DetectedObjects detection = detect(img);
return R.ok(FaceUtils.convertToDetectionResponse(detection,img));
} catch (IOException e) {
throw new FaceException("无效图片输入流", e);
} finally {
if (img != null) {
((Mat)img.getWrappedImage()).release();
}
}
}
@@ -114,9 +131,19 @@ public class CommonFaceDetModel implements FaceDetModel{
if(!ImageUtils.isImageValid(image)){
return R.fail(R.Status.INVALID_IMAGE);
}
Image img = ImageFactory.getInstance().fromImage(OpenCVUtils.image2Mat(image));
DetectedObjects detection = detect(img);
return R.ok(FaceUtils.convertToDetectionResponse(detection,img));
Image img = null;
try {
img = ImageFactory.getInstance().fromImage(OpenCVUtils.image2Mat(image));
DetectedObjects detection = detect(img);
return R.ok(FaceUtils.convertToDetectionResponse(detection,img));
} catch (Exception e) {
throw new FaceException(e);
} finally {
if (img != null) {
((Mat)img.getWrappedImage()).release();
}
}
}
@Override
@@ -124,11 +151,7 @@ public class CommonFaceDetModel implements FaceDetModel{
if(Objects.isNull(imageData)){
return R.fail(R.Status.INVALID_IMAGE);
}
try {
return detect(ImageIO.read(new ByteArrayInputStream(imageData)));
} catch (IOException e) {
throw new FaceException("错误的图像", e);
}
return detect(new ByteArrayInputStream(imageData));
}
@Override
@@ -145,8 +168,9 @@ public class CommonFaceDetModel implements FaceDetModel{
if(!FileUtils.isFileExists(imagePath)){
return R.fail(R.Status.FILE_NOT_FOUND);
}
Image img = null;
try {
Image img = ImageFactory.getInstance().fromFile(Paths.get(imagePath));
img = ImageFactory.getInstance().fromFile(Paths.get(imagePath));
DetectedObjects detectedObjects = detect(img);
if(Objects.isNull(detectedObjects) || detectedObjects.getNumberOfObjects() == 0){
return R.fail(R.Status.NO_FACE_DETECTED);
@@ -158,6 +182,10 @@ public class CommonFaceDetModel implements FaceDetModel{
return R.ok();
} catch (IOException e) {
throw new FaceException(e);
} finally {
if (img != null){
((Mat)img.getWrappedImage()).release();
}
}
}
@@ -181,6 +209,10 @@ public class CommonFaceDetModel implements FaceDetModel{
return R.ok(ImageIO.read(new ByteArrayInputStream(imageBytes)));
} catch (IOException e) {
throw new FaceException("导出图片失败", e);
} finally {
if (img != null){
((Mat)img.getWrappedImage()).release();
}
}
}
@@ -213,6 +245,11 @@ public class CommonFaceDetModel implements FaceDetModel{
}
@Override
public GenericObjectPool<Predictor<Image, DetectedObjects>> getPool() {
return predictorPool;
}
@Override
public void close() {
try {

View File

@@ -1,8 +1,13 @@
package cn.smartjavaai.face.model.facedect;
import ai.djl.inference.Predictor;
import ai.djl.modality.Classifications;
import ai.djl.modality.cv.Image;
import ai.djl.modality.cv.output.DetectedObjects;
import cn.smartjavaai.common.entity.DetectionResponse;
import cn.smartjavaai.common.entity.R;
import cn.smartjavaai.face.config.FaceDetConfig;
import org.apache.commons.pool2.impl.GenericObjectPool;
import java.awt.image.BufferedImage;
import java.io.InputStream;
@@ -83,6 +88,10 @@ public interface FaceDetModel extends AutoCloseable{
throw new UnsupportedOperationException("默认不支持该功能");
}
default GenericObjectPool<Predictor<Image, DetectedObjects>> getPool(){
throw new UnsupportedOperationException("默认不支持该功能");
}
}

View File

@@ -1,5 +1,6 @@
package cn.smartjavaai.face.model.facedect;
import ai.djl.engine.Engine;
import cn.smartjavaai.common.entity.DetectionResponse;
import cn.smartjavaai.common.entity.R;
import cn.smartjavaai.common.enums.DeviceEnum;
@@ -48,13 +49,9 @@ public class SeetaFace6FaceDetModel implements FaceDetModel{
String[] faceDetectorModelPath = {config.getModelPath() + File.separator + "face_detector.csta"};
String[] faceLandmarkerModelPath = {config.getModelPath() + File.separator + "face_landmarker_pts5.csta"};
SeetaDevice device = SeetaDevice.SEETA_DEVICE_AUTO;
int gpuId = 0;
int gpuId = config.getGpuId();
if(Objects.nonNull(config.getDevice())){
device = config.getDevice() == DeviceEnum.CPU ? SeetaDevice.SEETA_DEVICE_CPU : SeetaDevice.SEETA_DEVICE_GPU;
Integer gpuIdValue = config.getCustomParam("gpuId", Integer.class);
if(Objects.nonNull(gpuIdValue) && device == SeetaDevice.SEETA_DEVICE_GPU){
gpuId = gpuIdValue;
}
}
try {
SeetaModelSetting faceDetectorPoolSetting = new SeetaModelSetting(gpuId, faceDetectorModelPath, device);
@@ -65,6 +62,14 @@ public class SeetaFace6FaceDetModel implements FaceDetModel{
this.faceDetectorPool = new FaceDetectorPool(faceDetectorPoolConfSetting);
this.faceLandmarkerPool = new FaceLandmarkerPool(faceLandmarkerPoolConfSetting);
int predictorPoolSize = config.getPredictorPoolSize();
if(config.getPredictorPoolSize() <= 0){
predictorPoolSize = Runtime.getRuntime().availableProcessors(); // 默认等于CPU核心数
}
faceDetectorPool.setMaxTotal(predictorPoolSize);
faceLandmarkerPool.setMaxTotal(predictorPoolSize);
log.debug("模型推理器线程池最大数量: " + predictorPoolSize);
} catch (FileNotFoundException e) {
throw new FaceException(e);
}
@@ -212,8 +217,13 @@ public class SeetaFace6FaceDetModel implements FaceDetModel{
}
public FaceDetectorPool getFaceDetectorPool() {
return faceDetectorPool;
}
public FaceLandmarkerPool getFaceLandmarkerPool() {
return faceLandmarkerPool;
}
@Override
public void close() throws Exception {

View File

@@ -31,7 +31,7 @@ public class FaceDetCriteriaFactory {
public static Criteria<Image, DetectedObjects> createCriteria(FaceDetConfig config) {
Device device = null;
if(!Objects.isNull(config.getDevice())){
device = config.getDevice() == DeviceEnum.CPU ? Device.cpu() : Device.gpu();
device = config.getDevice() == DeviceEnum.CPU ? Device.cpu() : Device.gpu(config.getGpuId());
}
Criteria<Image, DetectedObjects> criteria = null;
if(config.getModelEnum() == FaceDetModelEnum.RETINA_FACE){

View File

@@ -9,6 +9,7 @@ import ai.djl.ndarray.NDManager;
import ai.djl.repository.zoo.Criteria;
import ai.djl.repository.zoo.ModelNotFoundException;
import ai.djl.repository.zoo.ZooModel;
import cn.hutool.core.lang.UUID;
import cn.smartjavaai.common.entity.*;
import cn.smartjavaai.common.entity.face.FaceInfo;
import cn.smartjavaai.common.entity.face.FaceSearchResult;
@@ -41,6 +42,7 @@ import lombok.extern.slf4j.Slf4j;
import org.apache.commons.lang3.StringUtils;
import org.apache.commons.pool2.ObjectPool;
import org.apache.commons.pool2.impl.GenericObjectPool;
import org.opencv.core.Mat;
import javax.imageio.ImageIO;
import java.awt.image.BufferedImage;
@@ -68,7 +70,7 @@ public class CommonFaceRecModel implements FaceRecModel{
private static volatile boolean isLoadCompleted = false;
private ObjectPool<Predictor<Image, float[]>> predictorPool;
private GenericObjectPool<Predictor<Image, float[]>> predictorPool;
private ZooModel<Image, float[]> model;
@@ -101,8 +103,14 @@ public class CommonFaceRecModel implements FaceRecModel{
model = faceFeatureCriteria.loadModel();
// 创建池子:每个线程独享 Predictor
this.predictorPool = new GenericObjectPool<>(new PredictorFactory<>(model));
int predictorPoolSize = config.getPredictorPoolSize();
if(config.getPredictorPoolSize() <= 0){
predictorPoolSize = Runtime.getRuntime().availableProcessors(); // 默认等于CPU核心数
}
predictorPool.setMaxTotal(predictorPoolSize);
log.debug("当前设备: " + model.getNDManager().getDevice());
log.debug("当前引擎: " + Engine.getInstance().getEngineName());
log.debug("模型推理器线程池最大数量: " + predictorPoolSize);
} catch (IOException | ModelNotFoundException | MalformedModelException e) {
throw new FaceException("模型加载失败", e);
}
@@ -286,6 +294,7 @@ public class CommonFaceRecModel implements FaceRecModel{
faceInfo.setFeature(features);
}
}
((Mat)djlImage.getWrappedImage()).release();
return detectedResult;
}
@@ -350,6 +359,7 @@ public class CommonFaceRecModel implements FaceRecModel{
}
features = featureExtraction(subImage);
}
((Mat)djlImage.getWrappedImage()).release();
return Objects.isNull(features) ? R.fail(R.Status.Unknown) : R.ok(features);
}
@@ -703,5 +713,8 @@ public class CommonFaceRecModel implements FaceRecModel{
}
@Override
public GenericObjectPool<Predictor<Image, float[]>> getPool() {
return predictorPool;
}
}

View File

@@ -1,5 +1,7 @@
package cn.smartjavaai.face.model.facerec;
import ai.djl.inference.Predictor;
import ai.djl.modality.cv.Image;
import cn.smartjavaai.common.entity.DetectionResponse;
import cn.smartjavaai.common.entity.R;
import cn.smartjavaai.face.config.FaceRecConfig;
@@ -7,6 +9,7 @@ import cn.smartjavaai.face.entity.FaceRegisterInfo;
import cn.smartjavaai.face.entity.FaceSearchParams;
import cn.smartjavaai.common.entity.face.FaceSearchResult;
import cn.smartjavaai.face.vector.entity.FaceVector;
import org.apache.commons.pool2.impl.GenericObjectPool;
import java.awt.image.BufferedImage;
import java.io.InputStream;
@@ -367,4 +370,9 @@ public interface FaceRecModel extends AutoCloseable{
throw new UnsupportedOperationException("默认不支持该功能");
}
default GenericObjectPool<Predictor<Image, float[]>> getPool() {
throw new UnsupportedOperationException("默认不支持该功能");
}
}

View File

@@ -1,5 +1,6 @@
package cn.smartjavaai.face.model.facerec;
import ai.djl.engine.Engine;
import cn.smartjavaai.common.entity.DetectionInfo;
import cn.smartjavaai.common.entity.DetectionResponse;
import cn.smartjavaai.common.entity.R;
@@ -89,13 +90,9 @@ public class SeetaFace6FaceRecModel implements FaceRecModel{
String[] faceRecognizerModelPath = {config.getModelPath() + File.separator + "face_recognizer.csta"};
String[] faceLandmarkerModelPath = {config.getModelPath() + File.separator + "face_landmarker_pts5.csta"};
SeetaDevice device = SeetaDevice.SEETA_DEVICE_AUTO;
int gpuId = 0;
int gpuId = config.getGpuId();
if(Objects.nonNull(config.getDevice())){
device = config.getDevice() == DeviceEnum.CPU ? SeetaDevice.SEETA_DEVICE_CPU : SeetaDevice.SEETA_DEVICE_GPU;
Integer gpuIdValue = config.getCustomParam("gpuId", Integer.class);
if(Objects.nonNull(gpuIdValue) && device == SeetaDevice.SEETA_DEVICE_GPU){
gpuId = gpuIdValue;
}
}
try {
SeetaModelSetting faceDetectorPoolSetting = new SeetaModelSetting(gpuId, faceDetectorModelPath, device);
@@ -115,6 +112,16 @@ public class SeetaFace6FaceRecModel implements FaceRecModel{
this.faceLandmarkerPool = new FaceLandmarkerPool(faceLandmarkerPoolConfSetting);
this.faceDatabasePool = new FaceDatabasePool(faceDatabasePoolConfSetting);
int predictorPoolSize = config.getPredictorPoolSize();
if(config.getPredictorPoolSize() <= 0){
predictorPoolSize = Runtime.getRuntime().availableProcessors(); // 默认等于CPU核心数
}
faceDetectorPool.setMaxTotal(predictorPoolSize);
faceRecognizerPool.setMaxTotal(predictorPoolSize);
faceLandmarkerPool.setMaxTotal(predictorPoolSize);
faceDatabasePool.setMaxTotal(predictorPoolSize);
log.debug("模型推理器线程池最大数量: " + predictorPoolSize);
//初始化人脸库
if(config.getVectorDBConfig() != null && config.getVectorDBConfig().getType() != null){
@@ -739,6 +746,36 @@ public class SeetaFace6FaceRecModel implements FaceRecModel{
}
}
/**
* 特征提取
* @param image
* @return
*/
public float[] featureExtraction(BufferedImage image){
FaceRecognizer faceRecognizer = null;
try {
faceRecognizer = faceRecognizerPool.borrowObject();
SeetaImageData imageData = new SeetaImageData(image.getWidth(), image.getHeight(), 3);
imageData.data = ImageUtils.getMatrixBGR(image);
//提取特征
float[] features = new float[faceRecognizer.GetExtractFeatureSize()];
faceRecognizer.ExtractCroppedFace(imageData, features);
return features;
} catch (FaceException e) {
throw e;
} catch (Exception e) {
throw new FaceException("目标检测错误", e);
}finally {
if (faceRecognizer != null) {
try {
faceRecognizerPool.returnObject(faceRecognizer); //归还
} catch (Exception e) {
log.warn("归还Predictor失败", e);
}
}
}
}
@Override
public R<float[]> extractTopFaceFeature(String imagePath) {
if(!FileUtils.isFileExists(imagePath)){
@@ -878,4 +915,19 @@ public class SeetaFace6FaceRecModel implements FaceRecModel{
}
public FaceDetectorPool getFaceDetectorPool() {
return faceDetectorPool;
}
public FaceRecognizerPool getFaceRecognizerPool() {
return faceRecognizerPool;
}
public FaceLandmarkerPool getFaceLandmarkerPool() {
return faceLandmarkerPool;
}
public FaceDatabasePool getFaceDatabasePool() {
return faceDatabasePool;
}
}

View File

@@ -27,7 +27,7 @@ public class FaceRecCriteriaFactory {
public static Criteria<Image, float[]> createCriteria(FaceRecConfig config) {
Device device = null;
if(!Objects.isNull(config.getDevice())){
device = config.getDevice() == DeviceEnum.CPU ? Device.cpu() : Device.gpu();
device = config.getDevice() == DeviceEnum.CPU ? Device.cpu() : Device.gpu(config.getGpuId());
}
Criteria<Image, float[]> criteria = null;
if(config.getModelEnum() == FaceRecModelEnum.FACENET_MODEL){
@@ -58,6 +58,7 @@ public class FaceRecCriteriaFactory {
.optTranslator(new FaceFeatureTranslator())
.optEngine("PyTorch") // Use PyTorch engine
.optProgress(new ProgressBar())
.optDevice(device)
.build();
}else if (config.getModelEnum() == FaceRecModelEnum.INSIGHT_FACE_IRSE50_MODEL){
if(StringUtils.isBlank(config.getModelPath())){
@@ -73,6 +74,7 @@ public class FaceRecCriteriaFactory {
// .optArgument("resize", "112,112")
.optTranslator(new FaceFeatureTranslator())
.optEngine("PyTorch") // Use PyTorch engine
.optDevice(device)
.optProgress(new ProgressBar())
.build();
}else if (config.getModelEnum() == FaceRecModelEnum.ELASTIC_FACE_MODEL){
@@ -89,6 +91,7 @@ public class FaceRecCriteriaFactory {
// .optArgument("resize", "112,112")
.optTranslator(new FaceFeatureTranslator())
.optEngine("PyTorch") // Use PyTorch engine
.optDevice(device)
.optProgress(new ProgressBar())
.build();
}

View File

@@ -31,7 +31,7 @@ public final class FaceFeatureTranslator implements Translator<Image, float[]> {
NDArray array = input.toNDArray(ctx.getNDManager(), Image.Flag.COLOR);
Pipeline pipeline = new Pipeline();
if(input.getWidth() != 112 || input.getHeight() != 112){
pipeline.add(new Resize(112));
pipeline.add(new Resize(112,112));
}
pipeline
.add(new ToTensor())

View File

@@ -2,6 +2,7 @@ package cn.smartjavaai.face.model.liveness;
import ai.djl.Device;
import ai.djl.MalformedModelException;
import ai.djl.engine.Engine;
import ai.djl.inference.Predictor;
import ai.djl.modality.cv.Image;
import ai.djl.modality.cv.ImageFactory;
@@ -35,6 +36,7 @@ import org.apache.commons.pool2.impl.GenericObjectPool;
import org.bytedeco.javacv.FFmpegFrameGrabber;
import org.bytedeco.javacv.Frame;
import org.bytedeco.javacv.Java2DFrameUtils;
import org.opencv.core.Mat;
import javax.imageio.ImageIO;
import java.awt.image.BufferedImage;
@@ -54,7 +56,7 @@ import java.util.*;
@Slf4j
public class CommonLivenessModel implements LivenessDetModel{
protected ObjectPool<Predictor<Image, Float>> predictorPool;
protected GenericObjectPool<Predictor<Image, Float>> predictorPool;
protected LivenessConfig config;
@@ -78,6 +80,15 @@ public class CommonLivenessModel implements LivenessDetModel{
} catch (IOException | ModelNotFoundException | MalformedModelException e) {
throw new FaceException("阿里通义实验室活体检测模型加载失败", e);
}
int predictorPoolSize = config.getPredictorPoolSize();
if(config.getPredictorPoolSize() <= 0){
predictorPoolSize = Runtime.getRuntime().availableProcessors(); // 默认等于CPU核心数
}
predictorPool.setMaxTotal(predictorPoolSize);
log.debug("当前设备: " + model.getNDManager().getDevice());
log.debug("当前引擎: " + Engine.getInstance().getEngineName());
log.debug("模型推理器线程池最大数量: " + predictorPoolSize);
}
@@ -87,6 +98,7 @@ public class CommonLivenessModel implements LivenessDetModel{
throw new FaceException("图像无效");
}
Predictor<Image, Float> predictor = null;
Image djlImage = null;
try {
predictor = predictorPool.borrowObject();
//预处理图片
@@ -101,7 +113,7 @@ public class CommonLivenessModel implements LivenessDetModel{
.setCenterCropSize(112)
.process();
}
Image djlImage = ImageFactory.getInstance().fromImage(OpenCVUtils.image2Mat(processedImage));
djlImage = ImageFactory.getInstance().fromImage(OpenCVUtils.image2Mat(processedImage));
Float result = predictor.predict(djlImage);
if(result >= config.getRealityThreshold()){
return R.ok(new LivenessResult(LivenessStatus.LIVE, result));
@@ -124,6 +136,10 @@ public class CommonLivenessModel implements LivenessDetModel{
}
}
}
if (djlImage != null){
((Mat)djlImage.getWrappedImage()).release();
}
}
}
@@ -403,6 +419,12 @@ public class CommonLivenessModel implements LivenessDetModel{
return R.fail(R.Status.Unknown);
}
@Override
public GenericObjectPool<Predictor<Image, Float>> getPool() {
return predictorPool;
}
@Override
public void close() throws Exception {
try {

View File

@@ -1,11 +1,14 @@
package cn.smartjavaai.face.model.liveness;
import ai.djl.inference.Predictor;
import ai.djl.modality.cv.Image;
import cn.smartjavaai.common.entity.DetectionRectangle;
import cn.smartjavaai.common.entity.DetectionResponse;
import cn.smartjavaai.common.entity.Point;
import cn.smartjavaai.common.entity.R;
import cn.smartjavaai.common.entity.face.LivenessResult;
import cn.smartjavaai.face.config.LivenessConfig;
import org.apache.commons.pool2.impl.GenericObjectPool;
import java.awt.image.BufferedImage;
import java.io.InputStream;
@@ -292,6 +295,9 @@ public interface LivenessDetModel extends AutoCloseable{
default GenericObjectPool<Predictor<Image, Float>> getPool() {
throw new UnsupportedOperationException("默认不支持该功能");
}

View File

@@ -2,6 +2,7 @@ package cn.smartjavaai.face.model.liveness;
import ai.djl.Device;
import ai.djl.MalformedModelException;
import ai.djl.engine.Engine;
import ai.djl.inference.Predictor;
import ai.djl.modality.cv.Image;
import ai.djl.modality.cv.ImageFactory;
@@ -33,6 +34,7 @@ import org.apache.commons.pool2.impl.GenericObjectPool;
import org.bytedeco.javacv.FFmpegFrameGrabber;
import org.bytedeco.javacv.Frame;
import org.bytedeco.javacv.Java2DFrameUtils;
import org.opencv.core.Mat;
import javax.imageio.ImageIO;
import java.awt.image.BufferedImage;
@@ -58,9 +60,9 @@ public class MiniVisionLivenessModel extends CommonLivenessModel{
*/
private static final String SE_MODEL_PATH_KEY = "seModelPath";
private ObjectPool<Predictor<Image, float[]>> predictorPool;
private GenericObjectPool<Predictor<Image, float[]>> predictorPool;
private ObjectPool<Predictor<Image, float[]>> sePredictorPool;
private GenericObjectPool<Predictor<Image, float[]>> sePredictorPool;
/**
@@ -87,7 +89,7 @@ public class MiniVisionLivenessModel extends CommonLivenessModel{
this.config.setRealityThreshold(realityThreshold);
Device device = null;
if(!Objects.isNull(config.getDevice())){
device = config.getDevice() == DeviceEnum.CPU ? Device.cpu() : Device.gpu();
device = config.getDevice() == DeviceEnum.CPU ? Device.cpu() : Device.gpu(config.getGpuId());
}
if(StringUtils.isNotBlank(config.getModelPath()) && StringUtils.isBlank(seModelPath)){
//2.7_80x80_MiniFASNetV2
@@ -109,6 +111,7 @@ public class MiniVisionLivenessModel extends CommonLivenessModel{
.optModelPath(Paths.get(config.getModelPath()))
.optTranslator(new MiniVisionTranslator())
.optProgress(new ProgressBar())
.optDevice(device)
.build();
try {
model = criteria.loadModel();
@@ -128,6 +131,7 @@ public class MiniVisionLivenessModel extends CommonLivenessModel{
.optModelPath(Paths.get(seModelPath))
.optTranslator(new MiniVisionTranslator())
.optProgress(new ProgressBar())
.optDevice(device)
.build();
try {
seModel = seCriteria.loadModel();
@@ -136,6 +140,16 @@ public class MiniVisionLivenessModel extends CommonLivenessModel{
throw new FaceException("MiniFASNetV1SE模型加载失败", e);
}
}
int predictorPoolSize = config.getPredictorPoolSize();
if(config.getPredictorPoolSize() <= 0){
predictorPoolSize = Runtime.getRuntime().availableProcessors(); // 默认等于CPU核心数
}
predictorPool.setMaxTotal(predictorPoolSize);
sePredictorPool.setMaxTotal(predictorPoolSize);
log.debug("当前设备: " + model.getNDManager().getDevice());
log.debug("当前引擎: " + Engine.getInstance().getEngineName());
log.debug("模型推理器线程池最大数量: " + predictorPoolSize);
}
@@ -160,6 +174,8 @@ public class MiniVisionLivenessModel extends CommonLivenessModel{
predictor = predictorPool.borrowObject();
Image djlImage = ImageFactory.getInstance().fromImage(OpenCVUtils.image2Mat(processedImage));
result = predictor.predict(djlImage);
((Mat)djlImage.getWrappedImage()).release();
}
if(Objects.nonNull(sePredictorPool)){
//预处理图片
@@ -172,6 +188,7 @@ public class MiniVisionLivenessModel extends CommonLivenessModel{
Image djlImage = ImageFactory.getInstance().fromImage(OpenCVUtils.image2Mat(processedImage));
sePredictor = sePredictorPool.borrowObject();
seResult = sePredictor.predict(djlImage);
((Mat)djlImage.getWrappedImage()).release();
}
if(Objects.isNull(result) && Objects.isNull(seResult)){
throw new FaceException("活体检测错误");
@@ -221,6 +238,14 @@ public class MiniVisionLivenessModel extends CommonLivenessModel{
}
}
public GenericObjectPool<Predictor<Image, float[]>> getPredictorPool() {
return predictorPool;
}
public GenericObjectPool<Predictor<Image, float[]>> getSePredictorPool() {
return sePredictorPool;
}
@Override
public void close() throws Exception {
try {

View File

@@ -1,5 +1,6 @@
package cn.smartjavaai.face.model.liveness;
import ai.djl.engine.Engine;
import cn.smartjavaai.common.entity.*;
import cn.smartjavaai.common.entity.face.FaceInfo;
import cn.smartjavaai.common.entity.face.LivenessResult;
@@ -56,13 +57,9 @@ public class Seetaface6LivenessModel implements LivenessDetModel{
String[] faceAntiSpoofingModelPath = {config.getModelPath() + File.separator + "fas_first.csta",config.getModelPath() + File.separator + "fas_second.csta"};
String[] faceLandmarkerModelPath = {config.getModelPath() + File.separator + "face_landmarker_pts5.csta"};
SeetaDevice device = SeetaDevice.SEETA_DEVICE_AUTO;
int gpuId = 0;
int gpuId = config.getGpuId();
if(Objects.nonNull(config.getDevice())){
device = config.getDevice() == DeviceEnum.CPU ? SeetaDevice.SEETA_DEVICE_CPU : SeetaDevice.SEETA_DEVICE_GPU;
Integer gpuIdValue = config.getCustomParam("gpuId", Integer.class);
if(Objects.nonNull(gpuIdValue) && device == SeetaDevice.SEETA_DEVICE_GPU){
gpuId = gpuIdValue;
}
}
try {
@@ -79,6 +76,16 @@ public class Seetaface6LivenessModel implements LivenessDetModel{
this.faceAntiSpoofingPool = new FaceAntiSpoofingPool(faceAntiSpoofingPoolConfSetting);
this.faceLandmarkerPool = new FaceLandmarkerPool(faceLandmarkerPoolConfSetting);
int predictorPoolSize = config.getPredictorPoolSize();
if(config.getPredictorPoolSize() <= 0){
predictorPoolSize = Runtime.getRuntime().availableProcessors(); // 默认等于CPU核心数
}
faceDetectorPool.setMaxTotal(predictorPoolSize);
faceAntiSpoofingPool.setMaxTotal(predictorPoolSize);
faceLandmarkerPool.setMaxTotal(predictorPoolSize);
log.debug("模型推理器线程池最大数量: " + predictorPoolSize);
//初始化模型参数
initConfig();
} catch (FileNotFoundException e) {
@@ -550,6 +557,18 @@ public class Seetaface6LivenessModel implements LivenessDetModel{
return R.fail(1000, "有效帧数量不足,无法完成活体检测");
}
public FaceDetectorPool getFaceDetectorPool() {
return faceDetectorPool;
}
public FaceAntiSpoofingPool getFaceAntiSpoofingPool() {
return faceAntiSpoofingPool;
}
public FaceLandmarkerPool getFaceLandmarkerPool() {
return faceLandmarkerPool;
}
@Override
public void close() throws Exception {
try {

View File

@@ -31,7 +31,7 @@ public class LivenessCriteriaFactory {
public static Criteria<Image, Float> createCriteria(LivenessConfig config) {
Device device = null;
if(!Objects.isNull(config.getDevice())){
device = config.getDevice() == DeviceEnum.CPU ? Device.cpu() : Device.gpu();
device = config.getDevice() == DeviceEnum.CPU ? Device.cpu() : Device.gpu(config.getGpuId());
}
Criteria<Image, Float> criteria = null;
if(config.getModelEnum() == LivenessModelEnum.IIC_FL_MODEL){

View File

@@ -1,5 +1,6 @@
package cn.smartjavaai.face.model.quality;
import ai.djl.engine.Engine;
import cn.smartjavaai.common.entity.*;
import cn.smartjavaai.common.enums.DeviceEnum;
import cn.smartjavaai.common.utils.FileUtils;
@@ -74,6 +75,8 @@ public class Seetaface6QualityModel implements FaceQualityModel {
*/
private QualityOfResolutionPool qualityOfResolutionPool;
int predictorPoolSize = 0;
@Override
public void loadModel(QualityConfig config) {
@@ -84,6 +87,11 @@ public class Seetaface6QualityModel implements FaceQualityModel {
//加载依赖库
NativeLoader.loadNativeLibraries(device);
this.config = config;
int predictorPoolSize = config.getPredictorPoolSize();
if(config.getPredictorPoolSize() <= 0){
predictorPoolSize = Runtime.getRuntime().availableProcessors(); // 默认等于CPU核心数
}
log.debug("模型推理器线程池最大数量: " + predictorPoolSize);
log.debug("Loading seetaFace6 library successfully.");
}
@@ -104,6 +112,7 @@ public class Seetaface6QualityModel implements FaceQualityModel {
try {
if(Objects.isNull(this.qualityOfBrightnessPool)){
this.qualityOfBrightnessPool = new QualityOfBrightnessPool(new SeetaConfSetting());
qualityOfBrightnessPool.setMaxTotal(predictorPoolSize);
}
qualityOfBrightness = qualityOfBrightnessPool.borrowObject();
SeetaImageData imageData = new SeetaImageData(image.getWidth(), image.getHeight(), 3);
@@ -170,6 +179,7 @@ public class Seetaface6QualityModel implements FaceQualityModel {
try {
if(Objects.isNull(this.qualityOfClarityPool)){
this.qualityOfClarityPool = new QualityOfClarityPool(new SeetaConfSetting());
qualityOfClarityPool.setMaxTotal(predictorPoolSize);
}
qualityOfClarity = qualityOfClarityPool.borrowObject();
SeetaImageData imageData = new SeetaImageData(image.getWidth(), image.getHeight(), 3);
@@ -236,6 +246,7 @@ public class Seetaface6QualityModel implements FaceQualityModel {
try {
if(Objects.isNull(this.qualityOfIntegrityPool)){
this.qualityOfIntegrityPool = new QualityOfIntegrityPool(new SeetaConfSetting());
qualityOfIntegrityPool.setMaxTotal(predictorPoolSize);
}
qualityOfIntegrity = qualityOfIntegrityPool.borrowObject();
SeetaImageData imageData = new SeetaImageData(image.getWidth(), image.getHeight(), 3);
@@ -302,6 +313,7 @@ public class Seetaface6QualityModel implements FaceQualityModel {
try {
if(Objects.isNull(this.qualityOfPosePool)){
this.qualityOfPosePool = new QualityOfPosePool(new SeetaConfSetting());
qualityOfPosePool.setMaxTotal(predictorPoolSize);
}
qualityOfPose = qualityOfPosePool.borrowObject();
SeetaImageData imageData = new SeetaImageData(image.getWidth(), image.getHeight(), 3);
@@ -368,6 +380,7 @@ public class Seetaface6QualityModel implements FaceQualityModel {
try {
if(Objects.isNull(this.qualityOfResolutionPool)){
this.qualityOfResolutionPool = new QualityOfResolutionPool(new SeetaConfSetting());
qualityOfResolutionPool.setMaxTotal(predictorPoolSize);
}
qualityOfResolution = qualityOfResolutionPool.borrowObject();
SeetaImageData imageData = new SeetaImageData(image.getWidth(), image.getHeight(), 3);
@@ -438,6 +451,7 @@ public class Seetaface6QualityModel implements FaceQualityModel {
}
SeetaConfSetting setting = getClarityMLSetting();
this.qualityOfLBNPool = new QualityOfLBNPool(setting);
qualityOfLBNPool.setMaxTotal(predictorPoolSize);
}
qualityOfLBN = qualityOfLBNPool.borrowObject();
SeetaImageData imageData = new SeetaImageData(image.getWidth(), image.getHeight(), 3);
@@ -509,6 +523,7 @@ public class Seetaface6QualityModel implements FaceQualityModel {
}
SeetaConfSetting setting = getPoseMLSetting();
this.qualityOfPoseExPool = new QualityOfPoseExPool(setting);
qualityOfPoseExPool.setMaxTotal(predictorPoolSize);
}
qualityOfPoseEx = qualityOfPoseExPool.borrowObject();
SeetaImageData imageData = new SeetaImageData(image.getWidth(), image.getHeight(), 3);
@@ -628,18 +643,23 @@ public class Seetaface6QualityModel implements FaceQualityModel {
try {
if(Objects.isNull(this.qualityOfBrightnessPool)){
this.qualityOfBrightnessPool = new QualityOfBrightnessPool(new SeetaConfSetting());
qualityOfBrightnessPool.setMaxTotal(predictorPoolSize);
}
if(Objects.isNull(this.qualityOfClarityPool)){
this.qualityOfClarityPool = new QualityOfClarityPool(new SeetaConfSetting());
qualityOfClarityPool.setMaxTotal(predictorPoolSize);
}
if(Objects.isNull(this.qualityOfIntegrityPool)){
this.qualityOfIntegrityPool = new QualityOfIntegrityPool(new SeetaConfSetting());
qualityOfIntegrityPool.setMaxTotal(predictorPoolSize);
}
if(Objects.isNull(this.qualityOfPosePool)){
this.qualityOfPosePool = new QualityOfPosePool(new SeetaConfSetting());
qualityOfPosePool.setMaxTotal(predictorPoolSize);
}
if(Objects.isNull(this.qualityOfResolutionPool)){
this.qualityOfResolutionPool = new QualityOfResolutionPool(new SeetaConfSetting());
qualityOfResolutionPool.setMaxTotal(predictorPoolSize);
}
FaceQualitySummary summary = new FaceQualitySummary();
qualityOfBrightness = qualityOfBrightnessPool.borrowObject();
@@ -690,6 +710,34 @@ public class Seetaface6QualityModel implements FaceQualityModel {
}
}
public QualityOfBrightnessPool getQualityOfBrightnessPool() {
return qualityOfBrightnessPool;
}
public QualityOfClarityPool getQualityOfClarityPool() {
return qualityOfClarityPool;
}
public QualityOfLBNPool getQualityOfLBNPool() {
return qualityOfLBNPool;
}
public QualityOfIntegrityPool getQualityOfIntegrityPool() {
return qualityOfIntegrityPool;
}
public QualityOfPosePool getQualityOfPosePool() {
return qualityOfPosePool;
}
public QualityOfPoseExPool getQualityOfPoseExPool() {
return qualityOfPoseExPool;
}
public QualityOfResolutionPool getQualityOfResolutionPool() {
return qualityOfResolutionPool;
}
@Override
public void close() throws Exception {
if(Objects.nonNull(qualityOfBrightnessPool)){

View File

@@ -60,15 +60,13 @@ public class DJLImagePreprocessor {
// 处理流程
public Image process() {
Image result = image;
if (enableCrop) {
result = result.getSubImage(cropRect.x, cropRect.y, cropRect.width, cropRect.height);
}
if (enableAffine) {
if(enableAffine){
result = warpAffine(keyPoints, affineTargetWidth, affineTargetHeight);
}else {
if(enableCrop){
result = result.getSubImage(cropRect.x, cropRect.y, cropRect.width, cropRect.height);
}
}
return result;
}

View File

@@ -62,6 +62,16 @@ public class MilvusConfig extends VectorDBConfig {
*/
private boolean useMemoryCache = true;
/**
* 用户名
*/
private String username;
/**
* 密码
*/
private String password;
/**
* 构造函数
*/

View File

@@ -49,10 +49,13 @@ public class MilvusClient implements VectorDBClient {
@Override
public void initialize() {
try {
ConnectParam connectParam = ConnectParam.newBuilder()
ConnectParam.Builder builder = ConnectParam.newBuilder()
.withHost(config.getHost())
.withPort(config.getPort())
.build();
.withPort(config.getPort());
if (StringUtils.isNotBlank(config.getUsername()) && StringUtils.isNotBlank(config.getPassword())) {
builder.withAuthorization(config.getUsername(), config.getPassword());
}
ConnectParam connectParam = builder.build();
serviceClient = new MilvusServiceClient(connectParam);
collectionName = StringUtils.isNotBlank(config.getCollectionName()) ? config.getCollectionName() : VectorDBConstants.Defaults.DEFAULT_COLLECTION_NAME;
createCollection(collectionName, config.getDimension());

View File

@@ -6,11 +6,11 @@
<parent>
<groupId>cn.smartjavaai</groupId>
<artifactId>smartjavaai-parent</artifactId>
<version>1.0.20</version>
<version>1.0.22</version>
</parent>
<artifactId>smartjavaai-objectdetection</artifactId>
<version>1.0.20</version>
<version>1.0.22</version>
<name>smartjavaai-objectdetection</name>
<description>SmartJavaAI</description>
<url>https://github.com/geekwenjie/SmartJavaAI</url>

View File

@@ -1,11 +1,13 @@
package cn.smartjavaai.objectdetection.config;
import cn.smartjavaai.common.config.ModelConfig;
import cn.smartjavaai.common.enums.DeviceEnum;
import cn.smartjavaai.objectdetection.constant.DetectorConstant;
import cn.smartjavaai.objectdetection.enums.DetectorModelEnum;
import lombok.Data;
import java.util.HashMap;
import java.util.List;
import java.util.Map;
/**
@@ -15,7 +17,7 @@ import java.util.Map;
* @date 2025/4/4
*/
@Data
public class DetectorModelConfig {
public class DetectorModelConfig extends ModelConfig {
/**
* 模型
@@ -27,10 +29,6 @@ public class DetectorModelConfig {
*/
private float threshold = DetectorConstant.DEFAULT_THRESHOLD;
/**
* 设备类型
*/
private DeviceEnum device;
/**
* 模型路径
@@ -45,35 +43,27 @@ public class DetectorModelConfig {
private int maxBox;
/**
* 个性化配置
* 允许的类列表
*/
private Map<String, Object> customParams = new HashMap<>();
private List<String> allowedClasses;
/**
* 检测结果数量
*/
private int topK;
public DetectorModelConfig() {
}
public DetectorModelConfig(DetectorModelEnum modelEnum, DeviceEnum device) {
this.modelEnum = modelEnum;
this.device = device;
setDevice(device);
}
public DetectorModelConfig(DetectorModelEnum modelEnum) {
this.modelEnum = modelEnum;
}
public <T> T getCustomParam(String key, Class<T> clazz) {
Object value = customParams.get(key);
if (value == null) return null;
return clazz.cast(value);
}
/**
* 添加个性化配置项
*/
public void putCustomParam(String key, Object value) {
if (customParams == null) {
customParams = new HashMap<>();
}
customParams.put(key, value);
}
}

View File

@@ -25,7 +25,7 @@ public class DJLModelCriteriaBuilder implements CriteriaBuilderStrategy {
public Criteria<Image, DetectedObjects> buildCriteria(DetectorModelConfig config) {
Device device = null;
if(!Objects.isNull(config.getDevice())){
device = config.getDevice() == DeviceEnum.CPU ? Device.cpu() : Device.gpu();
device = config.getDevice() == DeviceEnum.CPU ? Device.cpu() : Device.gpu(config.getGpuId());
}
Criteria<Image, DetectedObjects> criteria = Criteria.builder()
.optApplication(Application.CV.OBJECT_DETECTION)

View File

@@ -25,7 +25,7 @@ public class YoloCriteriaBuilder implements CriteriaBuilderStrategy {
public Criteria<Image, DetectedObjects> buildCriteria(DetectorModelConfig config) {
Device device = null;
if(!Objects.isNull(config.getDevice())){
device = config.getDevice() == DeviceEnum.CPU ? Device.cpu() : Device.gpu();
device = config.getDevice() == DeviceEnum.CPU ? Device.cpu() : Device.gpu(config.getGpuId());
}
Map<String, Object> customParams = getDefaultConfig();

View File

@@ -1,16 +1,22 @@
package cn.smartjavaai.objectdetection.model;
import ai.djl.MalformedModelException;
import ai.djl.engine.Engine;
import ai.djl.inference.Predictor;
import ai.djl.modality.cv.Image;
import ai.djl.modality.cv.ImageFactory;
import ai.djl.modality.cv.output.BoundingBox;
import ai.djl.modality.cv.output.DetectedObjects;
import ai.djl.modality.cv.translator.YoloV8TranslatorFactory;
import ai.djl.repository.zoo.Criteria;
import ai.djl.repository.zoo.ModelNotFoundException;
import ai.djl.repository.zoo.ZooModel;
import ai.djl.training.util.ProgressBar;
import cn.smartjavaai.common.entity.DetectionInfo;
import cn.smartjavaai.common.entity.DetectionRectangle;
import cn.smartjavaai.common.entity.DetectionResponse;
import cn.smartjavaai.common.entity.Point;
import cn.smartjavaai.common.entity.face.FaceInfo;
import cn.smartjavaai.common.pool.PredictorFactory;
import cn.smartjavaai.common.utils.FileUtils;
import cn.smartjavaai.common.utils.ImageUtils;
@@ -21,39 +27,53 @@ import cn.smartjavaai.objectdetection.criteria.CriteriaBuilderFactory;
import cn.smartjavaai.objectdetection.exception.DetectionException;
import cn.smartjavaai.objectdetection.utils.DetectorUtils;
import lombok.extern.slf4j.Slf4j;
import org.apache.commons.collections.CollectionUtils;
import org.apache.commons.lang3.StringUtils;
import org.apache.commons.pool2.ObjectPool;
import org.apache.commons.pool2.impl.GenericObjectPool;
import org.opencv.core.Mat;
import javax.imageio.ImageIO;
import java.awt.image.BufferedImage;
import java.io.*;
import java.nio.file.Paths;
import java.util.ArrayList;
import java.util.Iterator;
import java.util.List;
import java.util.Objects;
import java.util.stream.Collectors;
/**
* 目标检测模型
* @author dwj
* @date 2025/4/4
*/
@Slf4j
public class DetectorModel implements AutoCloseable{
private ZooModel<Image, DetectedObjects> model;
private ObjectPool<Predictor<Image, DetectedObjects>> predictorPool;
private GenericObjectPool<Predictor<Image, DetectedObjects>> predictorPool;
private DetectorModelConfig config;
public void loadModel(DetectorModelConfig config){
if(Objects.isNull(config.getModelEnum())){
throw new DetectionException("未配置模型枚举");
}
Criteria<Image, DetectedObjects> criteria = CriteriaBuilderFactory.createCriteria(config);
this.config = config;
try {
model = criteria.loadModel();
// 创建池子:每个线程独享 Predictor
this.predictorPool = new GenericObjectPool<>(new PredictorFactory<>(model));
int predictorPoolSize = config.getPredictorPoolSize();
if(config.getPredictorPoolSize() <= 0){
predictorPoolSize = Runtime.getRuntime().availableProcessors(); // 默认等于CPU核心数
}
predictorPool.setMaxTotal(predictorPoolSize);
log.debug("当前设备: " + model.getNDManager().getDevice());
log.debug("当前引擎: " + Engine.getInstance().getEngineName());
log.debug("模型推理器线程池最大数量: " + predictorPoolSize);
} catch (IOException | ModelNotFoundException | MalformedModelException e) {
throw new DetectionException("模型加载失败", e);
}
@@ -72,11 +92,16 @@ public class DetectorModel implements AutoCloseable{
Image image = null;
try {
image = ImageFactory.getInstance().fromFile(Paths.get(imagePath));
} catch (IOException e) {
throw new DetectionException("图片转换错误", e);
DetectedObjects detectedObjects = detect(image);
return DetectorUtils.convertToDetectionResponse(detectedObjects, image);
} catch (Exception e) {
throw new DetectionException(e);
} finally {
if (image != null){
((Mat)image.getWrappedImage()).release();
}
}
DetectedObjects detectedObjects = detect(image);
return DetectorUtils.convertToDetectionResponse(detectedObjects, image);
}
@@ -89,8 +114,9 @@ public class DetectorModel implements AutoCloseable{
if(!FileUtils.isFileExists(imagePath)){
throw new DetectionException("图像文件不存在");
}
Image img = null;
try {
Image img = ImageFactory.getInstance().fromFile(Paths.get(imagePath));
img = ImageFactory.getInstance().fromFile(Paths.get(imagePath));
DetectedObjects detectedObjects = detect(img);
img.drawBoundingBoxes(detectedObjects);
ByteArrayOutputStream outputStream = new ByteArrayOutputStream();
@@ -98,6 +124,10 @@ public class DetectorModel implements AutoCloseable{
img.save(new FileOutputStream(Paths.get(outputPath).toAbsolutePath().toString()), "png");
} catch (IOException e) {
throw new DetectionException(e);
} finally {
if (img != null){
((Mat)img.getWrappedImage()).release();
}
}
}
@@ -129,9 +159,19 @@ public class DetectorModel implements AutoCloseable{
if(!ImageUtils.isImageValid(image)){
throw new DetectionException("图像无效");
}
Image img = ImageFactory.getInstance().fromImage(OpenCVUtils.image2Mat(image));
DetectedObjects detectedObjects = detect(img);
return DetectorUtils.convertToDetectionResponse(detectedObjects, img);
Image img = null;
try {
img = ImageFactory.getInstance().fromImage(OpenCVUtils.image2Mat(image));
DetectedObjects detectedObjects = detect(img);
return DetectorUtils.convertToDetectionResponse(detectedObjects, img);
} catch (Exception e) {
throw new DetectionException(e);
} finally {
if (img != null) {
((Mat)img.getWrappedImage()).release();
}
}
}
/**
@@ -155,6 +195,10 @@ public class DetectorModel implements AutoCloseable{
return ImageIO.read(new ByteArrayInputStream(imageBytes));
} catch (IOException e) {
throw new DetectionException("导出图片失败", e);
} finally {
if (img != null) {
((Mat)img.getWrappedImage()).release();
}
}
}
@@ -163,11 +207,13 @@ public class DetectorModel implements AutoCloseable{
* @param image
* @return
*/
private DetectedObjects detect(Image image){
public DetectedObjects detect(Image image){
Predictor<Image, DetectedObjects> predictor = null;
try {
predictor = predictorPool.borrowObject();
return predictor.predict(image);
DetectedObjects detectedObjects = predictor.predict(image);
detectedObjects = filterDetections(detectedObjects);
return detectedObjects;
} catch (Exception e) {
throw new DetectionException("目标检测错误", e);
}finally {
@@ -187,6 +233,50 @@ public class DetectorModel implements AutoCloseable{
}
}
/**
* 筛选检测结果
* @param detectedObjects
* @return
*/
private DetectedObjects filterDetections(DetectedObjects detectedObjects) {
if (Objects.isNull(detectedObjects) || detectedObjects.getNumberOfObjects() == 0) {
return detectedObjects;
}
List<DetectedObjects.DetectedObject> items = detectedObjects.items();
// 按照允许的类别进行过滤
List<DetectedObjects.DetectedObject> filtered = new ArrayList<>();
//过滤类别
if(!CollectionUtils.isEmpty(config.getAllowedClasses())){
for (DetectedObjects.DetectedObject obj : items) {
if(config.getAllowedClasses().contains(obj.getClassName())){
filtered.add(obj);
}
}
}else{
filtered = items;
}
// 按照概率进行排序
filtered.sort((o1, o2) -> Double.compare(o2.getProbability(), o1.getProbability()));
if(config.getTopK() > 0 && filtered.size() > config.getTopK()){
filtered = filtered.subList(0, config.getTopK());
}
// 构建新的 DetectedObjects 返回
List<String> names = new ArrayList<>();
List<Double> probs = new ArrayList<>();
List<BoundingBox> boxes = new ArrayList<>();
for (DetectedObjects.DetectedObject obj : filtered) {
names.add(obj.getClassName());
probs.add(obj.getProbability());
boxes.add(obj.getBoundingBox());
}
return new DetectedObjects(names, probs, boxes);
}
public GenericObjectPool<Predictor<Image, DetectedObjects>> getPool() {
return predictorPool;
}
/**
* 显式释放资源

View File

@@ -6,7 +6,7 @@
<parent>
<groupId>cn.smartjavaai</groupId>
<artifactId>smartjavaai-parent</artifactId>
<version>1.0.20</version>
<version>1.0.22</version>
</parent>
<artifactId>smartjavaai-ocr</artifactId>
@@ -42,7 +42,7 @@
</dependency>
</dependencies>
<version>1.0.20</version>
<version>1.0.22</version>
<name>smartjavaai-ocr</name>
<description>SmartJavaAI</description>
<url>https://github.com/geekwenjie/SmartJavaAI</url>

View File

@@ -0,0 +1,41 @@
package cn.smartjavaai.ocr.config;
import cn.smartjavaai.common.config.ModelConfig;
import cn.smartjavaai.ocr.enums.CommonDetModelEnum;
import cn.smartjavaai.ocr.enums.PlateDetModelEnum;
import lombok.Data;
/**
* 车牌检测模型配置
* @author dwj
*/
@Data
public class PlateDetModelConfig extends ModelConfig {
/**
* 模型
*/
private PlateDetModelEnum modelEnum;
/**
* 检测模型路径
*/
private String modelPath;
/**
* 置信度阈值
*/
private float confidenceThreshold;
/**
* iou阈值
*/
private float iouThreshold;
/**
* 检测结果数量
*/
private int topK;
}

View File

@@ -0,0 +1,47 @@
package cn.smartjavaai.ocr.config;
import cn.smartjavaai.common.config.ModelConfig;
import cn.smartjavaai.ocr.enums.PlateDetModelEnum;
import cn.smartjavaai.ocr.enums.PlateRecModelEnum;
import cn.smartjavaai.ocr.model.plate.PlateDetModel;
import lombok.Data;
/**
* 车牌识别模型配置
* @author dwj
*/
@Data
public class PlateRecModelConfig extends ModelConfig {
/**
* 模型
*/
private PlateRecModelEnum modelEnum;
/**
* 检测模型路径
*/
private String modelPath;
/**
* 车牌检测模型
*/
private PlateDetModel plateDetModel;
/**
* 置信度阈值
*/
private float confidenceThreshold;
/**
* iou阈值
*/
private float iouThreshold;
/**
* 检测结果数量
*/
private int topK;
}

View File

@@ -0,0 +1,45 @@
package cn.smartjavaai.ocr.entity;
import cn.smartjavaai.common.entity.DetectionRectangle;
import cn.smartjavaai.ocr.enums.PlateType;
import lombok.Data;
/**
* 车牌识别信息
* @author dwj
*/
@Data
public class PlateInfo {
/**
* 车牌类型
*/
private PlateType plateType;
/**
* 车牌号码
*/
private String plateNumber;
/**
* 车牌颜色
*/
private String plateColor;
/**
* 检测位置信息
*/
private DetectionRectangle detectionRectangle;
/**
* 车牌4角坐标
*/
private OcrBox box;
/**
* 检测得分
*/
private float score;
}

View File

@@ -0,0 +1,35 @@
package cn.smartjavaai.ocr.entity;
import lombok.Data;
/**
* @author dwj
*/
@Data
public class PlateResult {
/**
* 车牌号码
*/
private String plateNo;
/**
* 车牌颜色
*/
private String plateColor;
public PlateResult(String plateNo, String plateColor) {
this.plateNo = plateNo;
this.plateColor = plateColor;
}
@Override
public String toString() {
return "PlateResult{" +
"plateNo='" + plateNo + '\'' +
", plateColor='" + plateColor + '\'' +
'}';
}
}

View File

@@ -0,0 +1,28 @@
package cn.smartjavaai.ocr.enums;
/**
* 车牌检测模型枚举
* @author dwj
*/
public enum PlateDetModelEnum {
YOLOV5,
YOLOV7;
/**
* 根据名称获取枚举 (忽略大小写和下划线变体)
*/
public static PlateDetModelEnum fromName(String name) {
String formatted = name.trim().toUpperCase().replaceAll("[-_]", "");
for (PlateDetModelEnum model : values()) {
if (model.name().replaceAll("_", "").equals(formatted)) {
return model;
}
}
throw new IllegalArgumentException("未知模型名称: " + name);
}
}

View File

@@ -0,0 +1,26 @@
package cn.smartjavaai.ocr.enums;
/**
* 车牌识别模型枚举
* @author dwj
*/
public enum PlateRecModelEnum {
PLATE_REC_CRNN;
/**
* 根据名称获取枚举 (忽略大小写和下划线变体)
*/
public static PlateRecModelEnum fromName(String name) {
String formatted = name.trim().toUpperCase().replaceAll("[-_]", "");
for (PlateRecModelEnum model : values()) {
if (model.name().replaceAll("_", "").equals(formatted)) {
return model;
}
}
throw new IllegalArgumentException("未知模型名称: " + name);
}
}

View File

@@ -0,0 +1,45 @@
package cn.smartjavaai.ocr.enums;
/**
* @author dwj
*/
public enum PlateType {
SINGLE("single", "单层"),
DOUBLE("double", "双层"),
UNKNOWN("unknown", "未知");
private final String className;
private final String description;
PlateType(String className, String description) {
this.className = className;
this.description = description;
}
public String getClassName() {
return className;
}
public String getDescription() {
return description;
}
/**
* 根据value获取对应的PlateType
* @param className
* @return PlateType
*/
public static PlateType fromClassName(String className) {
for (PlateType type : values()) {
if (type.className.equals(className)) {
return type;
}
}
return null;
}
}

View File

@@ -0,0 +1,168 @@
package cn.smartjavaai.ocr.factory;
import cn.smartjavaai.common.config.Config;
import cn.smartjavaai.ocr.config.PlateDetModelConfig;
import cn.smartjavaai.ocr.config.PlateRecModelConfig;
import cn.smartjavaai.ocr.config.TableStructureConfig;
import cn.smartjavaai.ocr.enums.PlateDetModelEnum;
import cn.smartjavaai.ocr.enums.PlateRecModelEnum;
import cn.smartjavaai.ocr.enums.TableStructureModelEnum;
import cn.smartjavaai.ocr.exception.OcrException;
import cn.smartjavaai.ocr.model.plate.CRNNPlateRecModel;
import cn.smartjavaai.ocr.model.plate.PlateDetModel;
import cn.smartjavaai.ocr.model.plate.PlateRecModel;
import cn.smartjavaai.ocr.model.plate.Yolov5PlateDetModel;
import cn.smartjavaai.ocr.model.table.CommonTableStructureModel;
import cn.smartjavaai.ocr.model.table.TableStructureModel;
import lombok.extern.slf4j.Slf4j;
import java.util.Map;
import java.util.Objects;
import java.util.concurrent.ConcurrentHashMap;
/**
* 车牌识别模型工厂
* @author dwj
*/
@Slf4j
public class PlateModelFactory {
// 使用 volatile 和双重检查锁定来确保线程安全的单例模式
private static volatile PlateModelFactory instance;
/**
* 模型缓存
*/
private static final ConcurrentHashMap<PlateDetModelEnum, PlateDetModel> detModelMap = new ConcurrentHashMap<>();
/**
* 模型缓存
*/
private static final ConcurrentHashMap<PlateRecModelEnum, PlateRecModel> recModelMap = new ConcurrentHashMap<>();
/**
* 模型注册表
*/
private static final Map<PlateDetModelEnum, Class<? extends PlateDetModel>> detModelRegistry =
new ConcurrentHashMap<>();
/**
* 模型注册表
*/
private static final Map<PlateRecModelEnum, Class<? extends PlateRecModel>> recModelRegistry =
new ConcurrentHashMap<>();
public static PlateModelFactory getInstance() {
if (instance == null) {
synchronized (PlateModelFactory.class) {
if (instance == null) {
instance = new PlateModelFactory();
}
}
}
return instance;
}
/**
* 注册模型
* @param plateDetModelEnum
* @param clazz
*/
private static void registerDetModel(PlateDetModelEnum plateDetModelEnum, Class<? extends PlateDetModel> clazz) {
detModelRegistry.put(plateDetModelEnum, clazz);
}
/**
* 注册模型
* @param plateRecModelEnum
* @param clazz
*/
private static void registerRecModel(PlateRecModelEnum plateRecModelEnum, Class<? extends PlateRecModel> clazz) {
recModelRegistry.put(plateRecModelEnum, clazz);
}
/**
* 获取模型
* @param config
* @return
*/
public PlateDetModel getDetModel(PlateDetModelConfig config) {
if(Objects.isNull(config) || Objects.isNull(config.getModelEnum())){
throw new OcrException("未配置OCR模型");
}
return detModelMap.computeIfAbsent(config.getModelEnum(), k -> {
return createDetModel(config);
});
}
/**
* 获取模型
* @param config
* @return
*/
public PlateRecModel getRecModel(PlateRecModelConfig config) {
if(Objects.isNull(config) || Objects.isNull(config.getModelEnum())){
throw new OcrException("未配置OCR模型");
}
return recModelMap.computeIfAbsent(config.getModelEnum(), k -> {
return createRecModel(config);
});
}
/**
* 创建检测模型
* @param config
* @return
*/
private PlateDetModel createDetModel(PlateDetModelConfig config) {
Class<?> clazz = detModelRegistry.get(config.getModelEnum());
if(clazz == null){
throw new OcrException("Unsupported model");
}
PlateDetModel model = null;
try {
model = (PlateDetModel) clazz.newInstance();
} catch (InstantiationException | IllegalAccessException e) {
throw new OcrException(e);
}
model.loadModel(config);
return model;
}
/**
* 创建识别模型
* @param config
* @return
*/
private PlateRecModel createRecModel(PlateRecModelConfig config) {
Class<?> clazz = recModelRegistry.get(config.getModelEnum());
if(clazz == null){
throw new OcrException("Unsupported model");
}
PlateRecModel model = null;
try {
model = (PlateRecModel) clazz.newInstance();
} catch (InstantiationException | IllegalAccessException e) {
throw new OcrException(e);
}
model.loadModel(config);
return model;
}
// 初始化默认算法
static {
registerDetModel(PlateDetModelEnum.YOLOV5, Yolov5PlateDetModel.class);
registerDetModel(PlateDetModelEnum.YOLOV7, Yolov5PlateDetModel.class);
registerRecModel(PlateRecModelEnum.PLATE_REC_CRNN, CRNNPlateRecModel.class);
log.debug("缓存目录:{}", Config.getCachePath());
}
}

View File

@@ -1,8 +1,11 @@
package cn.smartjavaai.ocr.model.common.detect;
import ai.djl.inference.Predictor;
import ai.djl.modality.cv.Image;
import ai.djl.ndarray.NDList;
import cn.smartjavaai.ocr.config.OcrDetModelConfig;
import cn.smartjavaai.ocr.entity.OcrBox;
import org.apache.commons.pool2.impl.GenericObjectPool;
import java.awt.image.BufferedImage;
import java.util.List;
@@ -93,5 +96,9 @@ public interface OcrCommonDetModel extends AutoCloseable{
throw new UnsupportedOperationException("默认不支持该功能");
}
default GenericObjectPool<Predictor<Image, NDList>> getPool(){
throw new UnsupportedOperationException("默认不支持该功能");
}
}

View File

@@ -43,7 +43,7 @@ import java.util.*;
@Slf4j
public class OcrCommonDetModelImpl implements OcrCommonDetModel{
private ObjectPool<Predictor<Image, NDList>> detPredictorPool;
private GenericObjectPool<Predictor<Image, NDList>> detPredictorPool;
private ZooModel<Image, NDList> detectionModel;
@@ -61,8 +61,14 @@ public class OcrCommonDetModelImpl implements OcrCommonDetModel{
detectionModel = ModelZoo.loadModel(detCriteria);
// 创建池子:每个线程独享 Predictor
this.detPredictorPool = new GenericObjectPool<>(new PredictorFactory<>(detectionModel));
int predictorPoolSize = config.getPredictorPoolSize();
if(config.getPredictorPoolSize() <= 0){
predictorPoolSize = Runtime.getRuntime().availableProcessors(); // 默认等于CPU核心数
}
detPredictorPool.setMaxTotal(predictorPoolSize);
log.debug("当前设备: " + detectionModel.getNDManager().getDevice());
log.debug("当前引擎: " + Engine.getInstance().getEngineName());
log.debug("模型推理器线程池最大数量: " + predictorPoolSize);
} catch (IOException | ModelNotFoundException | MalformedModelException e) {
throw new OcrException("检测模型加载失败", e);
}
@@ -209,6 +215,11 @@ public class OcrCommonDetModelImpl implements OcrCommonDetModel{
}
}
@Override
public GenericObjectPool<Predictor<Image, NDList>> getPool() {
return detPredictorPool;
}
@Override
public void close() throws Exception {
try {

View File

@@ -25,7 +25,7 @@ public class OcrCommonDetCriterialFactory {
public static Criteria<Image, NDList> createCriteria(OcrDetModelConfig config) {
Device device = null;
if(!Objects.isNull(config.getDevice())){
device = config.getDevice() == DeviceEnum.CPU ? Device.cpu() : Device.gpu();
device = config.getDevice() == DeviceEnum.CPU ? Device.cpu() : Device.gpu(config.getGpuId());
}
Criteria<Image, NDList> criteria = null;
ConcurrentHashMap params = new ConcurrentHashMap<String, String>();

View File

@@ -9,6 +9,7 @@ import cn.smartjavaai.ocr.entity.OcrBox;
import cn.smartjavaai.ocr.entity.OcrInfo;
import cn.smartjavaai.ocr.entity.OcrItem;
import cn.smartjavaai.ocr.model.common.detect.OcrCommonDetModel;
import org.apache.commons.pool2.impl.GenericObjectPool;
import org.opencv.core.Mat;
import java.awt.image.BufferedImage;
@@ -106,4 +107,8 @@ public interface OcrDirectionModel extends AutoCloseable{
throw new UnsupportedOperationException("默认不支持该功能");
}
default GenericObjectPool<Predictor<Image, DirectionInfo>> getPool() {
throw new UnsupportedOperationException("默认不支持该功能");
}
}

View File

@@ -55,7 +55,7 @@ import java.util.concurrent.ConcurrentHashMap;
public class PPOCRMobileV2ClsModel implements OcrDirectionModel {
private ObjectPool<Predictor<Image, DirectionInfo>> predictorPool;
private GenericObjectPool<Predictor<Image, DirectionInfo>> predictorPool;
private DirectionModelConfig config;
@@ -71,10 +71,6 @@ public class PPOCRMobileV2ClsModel implements OcrDirectionModel {
}
this.config = config;
this.textDetModel = config.getTextDetModel();
Device device = null;
if(!Objects.isNull(config.getDevice())){
device = config.getDevice() == DeviceEnum.CPU ? Device.cpu() : Device.gpu();
}
ConcurrentHashMap params = new ConcurrentHashMap<String, String>();
if(StringUtils.isNotBlank(config.getBatchifier())){
params.put("batchifier", config.getBatchifier());
@@ -84,8 +80,14 @@ public class PPOCRMobileV2ClsModel implements OcrDirectionModel {
model = ModelZoo.loadModel(criteria);
// 创建池子:每个线程独享 Predictor
this.predictorPool = new GenericObjectPool<>(new PredictorFactory<>(model));
int predictorPoolSize = config.getPredictorPoolSize();
if(config.getPredictorPoolSize() <= 0){
predictorPoolSize = Runtime.getRuntime().availableProcessors(); // 默认等于CPU核心数
}
predictorPool.setMaxTotal(predictorPoolSize);
log.debug("当前设备: " + model.getNDManager().getDevice());
log.debug("当前引擎: " + Engine.getInstance().getEngineName());
log.debug("模型推理器线程池最大数量: " + predictorPoolSize);
} catch (IOException | ModelNotFoundException | MalformedModelException e) {
throw new OcrException("模型加载失败", e);
}
@@ -369,6 +371,11 @@ public class PPOCRMobileV2ClsModel implements OcrDirectionModel {
return textDetModel;
}
@Override
public GenericObjectPool<Predictor<Image, DirectionInfo>> getPool() {
return predictorPool;
}
@Override
public void close() throws Exception {
try {

View File

@@ -28,7 +28,7 @@ public class DirectionCriteriaFactory {
public static Criteria<Image, DirectionInfo> createCriteria(DirectionModelConfig config) {
Device device = null;
if(!Objects.isNull(config.getDevice())){
device = config.getDevice() == DeviceEnum.CPU ? Device.cpu() : Device.gpu();
device = config.getDevice() == DeviceEnum.CPU ? Device.cpu() : Device.gpu(config.getGpuId());
}
Criteria<Image, DirectionInfo> criteria = null;
ConcurrentHashMap params = new ConcurrentHashMap<String, String>();

View File

@@ -1,5 +1,6 @@
package cn.smartjavaai.ocr.model.common.recognize;
import ai.djl.inference.Predictor;
import ai.djl.modality.cv.Image;
import cn.smartjavaai.ocr.config.OcrDetModelConfig;
import cn.smartjavaai.ocr.config.OcrRecModelConfig;
@@ -8,6 +9,7 @@ import cn.smartjavaai.ocr.entity.OcrBox;
import cn.smartjavaai.ocr.entity.OcrInfo;
import cn.smartjavaai.ocr.model.common.detect.OcrCommonDetModel;
import cn.smartjavaai.ocr.model.common.direction.OcrDirectionModel;
import org.apache.commons.pool2.impl.GenericObjectPool;
import java.awt.image.BufferedImage;
import java.util.List;
@@ -106,4 +108,8 @@ public interface OcrCommonRecModel extends AutoCloseable{
throw new UnsupportedOperationException("默认不支持该功能");
}
default GenericObjectPool<Predictor<Image, String>> getPool() {
throw new UnsupportedOperationException("默认不支持该功能");
}
}

View File

@@ -24,7 +24,6 @@ import cn.smartjavaai.ocr.exception.OcrException;
import cn.smartjavaai.ocr.model.common.detect.OcrCommonDetModel;
import cn.smartjavaai.ocr.model.common.direction.OcrDirectionModel;
import cn.smartjavaai.ocr.model.common.recognize.criteria.OcrCommonRecCriterialFactory;
import cn.smartjavaai.ocr.opencv.OcrOpenCVUtils;
import cn.smartjavaai.ocr.utils.OcrUtils;
import lombok.extern.slf4j.Slf4j;
import org.apache.commons.collections.CollectionUtils;
@@ -49,7 +48,7 @@ import java.util.stream.Collectors;
@Slf4j
public class OcrCommonRecModelImpl implements OcrCommonRecModel {
private ObjectPool<Predictor<Image, String>> recPredictorPool;
private GenericObjectPool<Predictor<Image, String>> recPredictorPool;
private OcrRecModelConfig config;
@@ -72,8 +71,14 @@ public class OcrCommonRecModelImpl implements OcrCommonRecModel {
try{
recognitionModel = ModelZoo.loadModel(recCriteria);
this.recPredictorPool = new GenericObjectPool<>(new PredictorFactory<>(recognitionModel));
int predictorPoolSize = config.getPredictorPoolSize();
if(config.getPredictorPoolSize() <= 0){
predictorPoolSize = Runtime.getRuntime().availableProcessors(); // 默认等于CPU核心数
}
recPredictorPool.setMaxTotal(predictorPoolSize);
log.debug("当前设备: " + recognitionModel.getNDManager().getDevice());
log.debug("当前引擎: " + Engine.getInstance().getEngineName());
log.debug("模型推理器线程池最大数量: " + predictorPoolSize);
} catch (IOException | ModelNotFoundException | MalformedModelException e) {
throw new OcrException("识别模型加载失败", e);
}
@@ -238,7 +243,7 @@ public class OcrCommonRecModelImpl implements OcrCommonRecModel {
throw new OcrException("未检测到文字");
}
Mat wrappedImage = (Mat) img.getWrappedImage();
BufferedImage bufferedImage = OcrOpenCVUtils.mat2Image(wrappedImage);
BufferedImage bufferedImage = OpenCVUtils.mat2Image(wrappedImage);
OcrUtils.drawRectWithText(bufferedImage, ocrInfo, fontSize);
ImageUtils.saveImage(bufferedImage, outputPath);
wrappedImage.release();
@@ -443,6 +448,11 @@ public class OcrCommonRecModelImpl implements OcrCommonRecModel {
return directionModel;
}
public GenericObjectPool<Predictor<Image, String>> getRecPredictorPool() {
return recPredictorPool;
}
@Override
public void close() throws Exception {
try {

View File

@@ -24,7 +24,7 @@ public class OcrCommonRecCriterialFactory {
public static Criteria<Image, String> createCriteria(OcrRecModelConfig config) {
Device device = null;
if(!Objects.isNull(config.getDevice())){
device = config.getDevice() == DeviceEnum.CPU ? Device.cpu() : Device.gpu();
device = config.getDevice() == DeviceEnum.CPU ? Device.cpu() : Device.gpu(config.getGpuId());
}
Criteria<Image, String> criteria = null;
ConcurrentHashMap params = new ConcurrentHashMap<String, String>();

View File

@@ -0,0 +1,342 @@
package cn.smartjavaai.ocr.model.plate;
import ai.djl.MalformedModelException;
import ai.djl.engine.Engine;
import ai.djl.inference.Predictor;
import ai.djl.modality.cv.Image;
import ai.djl.modality.cv.ImageFactory;
import ai.djl.modality.cv.output.BoundingBox;
import ai.djl.modality.cv.output.DetectedObjects;
import ai.djl.repository.zoo.Criteria;
import ai.djl.repository.zoo.ModelNotFoundException;
import ai.djl.repository.zoo.ModelZoo;
import ai.djl.repository.zoo.ZooModel;
import cn.hutool.core.lang.UUID;
import cn.hutool.core.lang.generator.UUIDGenerator;
import cn.smartjavaai.common.entity.DetectionRectangle;
import cn.smartjavaai.common.entity.R;
import cn.smartjavaai.common.pool.PredictorFactory;
import cn.smartjavaai.common.utils.Base64ImageUtils;
import cn.smartjavaai.common.utils.FileUtils;
import cn.smartjavaai.common.utils.ImageUtils;
import cn.smartjavaai.common.utils.OpenCVUtils;
import cn.smartjavaai.ocr.config.PlateDetModelConfig;
import cn.smartjavaai.ocr.config.PlateRecModelConfig;
import cn.smartjavaai.ocr.entity.PlateInfo;
import cn.smartjavaai.ocr.entity.PlateResult;
import cn.smartjavaai.ocr.enums.PlateType;
import cn.smartjavaai.ocr.exception.OcrException;
import cn.smartjavaai.ocr.model.plate.criteria.PlateDetCriterialFactory;
import cn.smartjavaai.ocr.model.plate.criteria.PlateRecCriterialFactory;
import cn.smartjavaai.ocr.utils.OcrUtils;
import lombok.extern.slf4j.Slf4j;
import org.apache.commons.collections.CollectionUtils;
import org.apache.commons.lang3.StringUtils;
import org.apache.commons.pool2.ObjectPool;
import org.apache.commons.pool2.impl.GenericObjectPool;
import org.opencv.core.Core;
import org.opencv.core.Mat;
import org.opencv.core.Rect;
import org.opencv.core.Size;
import org.opencv.imgproc.Imgproc;
import javax.imageio.ImageIO;
import java.awt.*;
import java.awt.image.BufferedImage;
import java.io.*;
import java.nio.file.Files;
import java.nio.file.Path;
import java.nio.file.Paths;
import java.util.ArrayList;
import java.util.Iterator;
import java.util.List;
import java.util.Objects;
/**
* @author dwj
*/
@Slf4j
public class CRNNPlateRecModel implements PlateRecModel{
private GenericObjectPool<Predictor<Image, PlateResult>> recPredictorPool;
private ZooModel<Image, PlateResult> recModel;
private PlateRecModelConfig config;
@Override
public void loadModel(PlateRecModelConfig config) {
if(StringUtils.isBlank(config.getModelPath())){
throw new OcrException("modelPath is null");
}
this.config = config;
//初始化 检测Criteria
Criteria<Image, PlateResult> detCriteria = PlateRecCriterialFactory.createCriteria(config);
try{
recModel = ModelZoo.loadModel(detCriteria);
// 创建池子:每个线程独享 Predictor
this.recPredictorPool = new GenericObjectPool<>(new PredictorFactory<>(recModel));
int predictorPoolSize = config.getPredictorPoolSize();
if(config.getPredictorPoolSize() <= 0){
predictorPoolSize = Runtime.getRuntime().availableProcessors(); // 默认等于CPU核心数
}
recPredictorPool.setMaxTotal(predictorPoolSize);
log.debug("当前设备: " + recModel.getNDManager().getDevice());
log.debug("当前引擎: " + Engine.getInstance().getEngineName());
log.debug("模型推理器线程池最大数量: " + predictorPoolSize);
} catch (IOException | ModelNotFoundException | MalformedModelException e) {
throw new OcrException("检测模型加载失败", e);
}
}
@Override
public R<List<PlateInfo>> recognize(String imagePath) {
if(!FileUtils.isFileExists(imagePath)){
return R.fail(R.Status.FILE_NOT_FOUND);
}
Image img = null;
try {
img = ImageFactory.getInstance().fromFile(Paths.get(imagePath));
R<List<PlateInfo>> plateResult = recognize(img);
return plateResult;
} catch (IOException e) {
throw new OcrException("无效的图片", e);
} finally {
((Mat)img.getWrappedImage()).release();
}
}
@Override
public R<List<PlateInfo>> recognizeBase64(String base64Image) {
if(StringUtils.isBlank(base64Image)){
return R.fail(R.Status.INVALID_IMAGE);
}
byte[] imageData = Base64ImageUtils.base64ToImage(base64Image);
return recognize(imageData);
}
@Override
public R<List<PlateInfo>> recognize(BufferedImage image) {
if(!ImageUtils.isImageValid(image)){
return R.fail(R.Status.INVALID_IMAGE);
}
Image img = ImageFactory.getInstance().fromImage(OpenCVUtils.image2Mat(image));
R<List<PlateInfo>> plateResult = recognize(img);
((Mat)img.getWrappedImage()).release();
return plateResult;
}
@Override
public R<List<PlateInfo>> recognize(byte[] imageData) {
if(Objects.isNull(imageData)){
return R.fail(R.Status.INVALID_IMAGE);
}
return recognize(new ByteArrayInputStream(imageData));
}
@Override
public R<List<PlateInfo>> recognize(Image image) {
if(Objects.isNull(config.getPlateDetModel())){
return R.fail(R.Status.PARAM_ERROR.getCode(), "未指定车牌检测模型");
}
DetectedObjects detectedObjects = config.getPlateDetModel().detect(image);
if(Objects.isNull(detectedObjects) || detectedObjects.getNumberOfObjects() == 0){
return R.fail(R.Status.NO_OBJECT_DETECTED);
}
List<PlateInfo> plateInfoList = OcrUtils.convertToPlateInfo(detectedObjects, image);
Predictor<Image, PlateResult> predictor = null;
try {
predictor = recPredictorPool.borrowObject();
for (PlateInfo plateInfo : plateInfoList){
DetectionRectangle detectionRectangle = plateInfo.getDetectionRectangle();
// Image subImage = image.getSubImage(detectionRectangle.getX(), detectionRectangle.getY(), detectionRectangle.getWidth(), detectionRectangle.getHeight());
//透视变换
Image subImage = OcrUtils.transformAndCrop((Mat)image.getWrappedImage(), plateInfo.getBox());
//双层车牌
if(plateInfo.getPlateType() == PlateType.DOUBLE){
Mat mergeImage = getSplitMerge((Mat)subImage.getWrappedImage());
subImage = ImageFactory.getInstance().fromImage(mergeImage);
}
PlateResult plateResult = predictor.predict(subImage);
if(Objects.nonNull(plateResult)){
plateInfo.setPlateNumber(plateResult.getPlateNo());
plateInfo.setPlateColor(plateResult.getPlateColor());
}
}
return R.ok(plateInfoList);
} catch (Exception e) {
throw new OcrException("车牌识别错误", e);
}finally {
if (predictor != null) {
try {
recPredictorPool.returnObject(predictor); //归还
} catch (Exception e) {
log.warn("归还Predictor失败", e);
try {
predictor.close(); // 归还失败才销毁
} catch (Exception ex) {
log.error("关闭Predictor失败", ex);
}
}
}
}
}
/**
* 双层车牌进行分割后识别
* @param img
* @return
*/
private Mat getSplitMerge(Mat img) {
int h = img.rows();
int w = img.cols();
// 上半部分:高度的前 5/12
Rect upperRect = new Rect(0, 0, w, (int)(5.0 / 12 * h));
Mat imgUpper = new Mat(img, upperRect);
// 下半部分:高度从 1/3 开始
Rect lowerRect = new Rect(0, (int)(1.0 / 3 * h), w, h - (int)(1.0 / 3 * h));
Mat imgLower = new Mat(img, lowerRect);
// 将上半部分 resize 到与下半部分相同大小
Mat resizedUpper = new Mat();
Size lowerSize = imgLower.size();
Imgproc.resize(imgUpper, resizedUpper, lowerSize);
// 水平拼接(将上下拼成左右)
List<Mat> mergeList = new ArrayList<>();
mergeList.add(resizedUpper);
mergeList.add(imgLower);
Mat merged = new Mat();
Core.hconcat(mergeList, merged);
return merged;
}
@Override
public PlateResult recognizeCropped(Image image) {
Predictor<Image, PlateResult> predictor = null;
try {
predictor = recPredictorPool.borrowObject();
return predictor.predict(image);
} catch (Exception e) {
throw new OcrException("车牌检测错误", e);
}finally {
if (predictor != null) {
try {
recPredictorPool.returnObject(predictor); //归还
} catch (Exception e) {
log.warn("归还Predictor失败", e);
try {
predictor.close(); // 归还失败才销毁
} catch (Exception ex) {
log.error("关闭Predictor失败", ex);
}
}
}
}
}
@Override
public R<List<PlateInfo>> recognize(InputStream inputStream) {
if(Objects.isNull(inputStream)){
return R.fail(R.Status.INVALID_IMAGE);
}
Image img = null;
try {
img = ImageFactory.getInstance().fromInputStream(inputStream);
return recognize(img);
} catch (IOException e) {
throw new OcrException("无效图片输入流", e);
} finally {
if (img != null){
((Mat)img.getWrappedImage()).release();
}
}
}
@Override
public R<Void> recognizeAndDraw(String imagePath, String outputPath) {
if(!FileUtils.isFileExists(imagePath)){
return R.fail(R.Status.FILE_NOT_FOUND);
}
Image img = null;
try {
img = ImageFactory.getInstance().fromFile(Paths.get(imagePath));
R<List<PlateInfo>> plateResult = recognize(img);
if(!plateResult.isSuccess()){
return R.fail(plateResult.getCode(), plateResult.getMessage());
}
if(CollectionUtils.isEmpty(plateResult.getData())){
return R.fail(R.Status.NO_OBJECT_DETECTED);
}
BufferedImage bufferedImage = OpenCVUtils.mat2Image((Mat)img.getWrappedImage());
OcrUtils.drawPlateInfo(bufferedImage, plateResult.getData());
ImageIO.write(bufferedImage, "jpg", new File(outputPath));
return R.ok();
} catch (IOException e) {
throw new OcrException(e);
} finally {
if (img != null){
((Mat)img.getWrappedImage()).release();
}
}
}
@Override
public R<BufferedImage> recognizeAndDraw(BufferedImage sourceImage) {
if(!ImageUtils.isImageValid(sourceImage)){
return R.fail(R.Status.INVALID_IMAGE);
}
Image img = ImageFactory.getInstance().fromImage(OpenCVUtils.image2Mat(sourceImage));
try {
R<List<PlateInfo>> plateResult = recognize(img);
if(!plateResult.isSuccess()){
return R.fail(plateResult.getCode(), plateResult.getMessage());
}
if(CollectionUtils.isEmpty(plateResult.getData())){
return R.fail(R.Status.NO_OBJECT_DETECTED);
}
OcrUtils.drawPlateInfo((Mat)img.getWrappedImage(), plateResult.getData());
ByteArrayOutputStream outputStream = new ByteArrayOutputStream();
// 调用 save 方法将 Image 写入字节流
img.save(outputStream, "png");
// 将字节流转换为 BufferedImage
byte[] imageBytes = outputStream.toByteArray();
return R.ok(ImageIO.read(new ByteArrayInputStream(imageBytes)));
} catch (IOException e) {
throw new OcrException("导出图片失败", e);
} finally {
if (img != null){
((Mat)img.getWrappedImage()).release();
}
}
}
@Override
public GenericObjectPool<Predictor<Image, PlateResult>> getPool() {
return recPredictorPool;
}
@Override
public void close() throws Exception {
try {
if (recPredictorPool != null) {
recPredictorPool.close();
}
} catch (Exception e) {
log.warn("关闭 predictorPool 失败", e);
}
try {
if (recModel != null) {
recModel.close();
}
} catch (Exception e) {
log.warn("关闭 model 失败", e);
}
}
}

Some files were not shown because too many files have changed in this diff Show More