diff --git a/README.md b/README.md index 5da7e3f..ec8f975 100644 --- a/README.md +++ b/README.md @@ -1,25 +1,36 @@ -# SmartJavaAI：JAVA深度学习算法工具包 +

+ +

+

+ 🍬JAVA轻量级深度学习算法库，该库致力于构建Java生态与AI模型之间的高效桥梁 +

+## 📚简介 + +`SmartJavaAI`是专为Java 开发者打造的一个功能丰富、开箱即用的 Java 算法工具包，致力于帮助Java开发者高效集成各类智能算法。SmartJavaAI通过对多种主流算法的统一封装，开发者无需深入了解底层实现，即可轻松在 Java 代码中调用人脸识别、目标检测、OCR 等功能。目前已支持部分人脸识别与目标检测算法，底层实现涵盖了 C++、Python 等语言的深度学习模型。后续将持续扩展更多算法，最终将构建一个面向 Java 开发者的通用智能工具库。 -![retinaface_detected](examples/output/retinaface_detected.jpg) +## 🚀 核心亮点 +针对 Java 开发者在集成智能算法时常见的两大痛点： +- 🐍 主流AI深度学习框架（PyTorch/TensorFlow）的Python生态与Java工程体系割裂 -**SmartJavaAI**是基于 **DJL（Deep Java Library）** 封装的轻量级深度学习算法库，依托DJL的自动模型管理和跨框架特性，**无需安装Python环境**且无需手动下载模型文件（模型由DJL内部自动从云端加载），该库致力于构建Java生态与AI模型之间的高效桥梁。针对Java开发者面临的两大痛点： - -- 🐍 主流AI框架（PyTorch/TensorFlow）的Python生态与Java工程体系割裂 - -- ⚙️ 直接使用DJL需处理模型加载、预处理、后处理等复杂技术细节 +- ⚙️ 现有算法方案分散杂乱，封装不统一，使用门槛高，不易直接服务于 Java 业务开发 我们实现了： -✅ **开箱即用** - 两行代码完成人脸检测/识别 -✅ **多模型支持** - 集成RetinaFace/Ultra-Light-Fast-Generic-Face-Detector双检测模型（即将支持OCR/目标检测） -✅ **跨平台兼容** - 完美支持Windows/Linux/macOS系统（x86 & ARM架构） + +✅ **开箱即用** - 两行代码即可调用算法 + +✅ **支持多种深度学习引擎** - Pytorch、Tensorflow、MXNet、ONNX Runtime + +✅ **功能丰富** - 当前支持人脸识别与目标检测，未来将陆续支持 OCR、图像分类、NLP 等多个 AI 领域任务，构建全面的智能算法体系。 + +✅ **跨平台兼容** - 支持Windows/Linux/macOS系统（x86 & ARM架构） -## 🌟 核心优势 +## 🌟 AI集成方式对比 | 维度 | Python生态 | 原生DJL | 本工具包 | |------------|---------------------|-----------------|----------------| | 开发效率 | 需搭建Python环境 | 需实现完整AI Pipeline | 提供即用API | @@ -27,229 +38,115 @@ | 部署复杂度 | 需维护多语言服务 | 需处理底层资源调度 | 单一Jar包集成 | | 性能表现 | 原生高性能 | 依赖开发者优化经验 | 内置生产级调优 | + + ## 📌 支持功能 -### ✅ 已实现功能 +#### ✅ 已实现功能 - **人脸检测** - 支持图片/视频流中的多面孔定位与质量评估 + 人脸检测、人脸识别、人脸比对1:1、人脸比对1:N、人脸库注册、人脸库、人脸库删除 +- **目标检测** + 支持通用目标检测，能够识别图像中的多种物体类别，返回物体位置与类别信息 -- **人脸特征提取** - 基于深度学习算法生成512维特征向量 - -- **人脸特征比对** （1：1） - -- **人脸查询** （1：N） - 人脸库注册/人脸库查询/人脸库删除 - -### ⌛ 规划中功能 +#### ⌛ 规划中功能 - **OCR文字识别** - 即将支持身份证/银行卡/车牌等关键信息提取，适配复杂背景与模糊文本 - -- **目标检测** - 计划集成YOLOv9模型，支持车辆检测/安全帽识别/工业质检等场景 - + 即将支持身份证/银行卡/车牌等关键信息提取 - **图像分割** - - **语音识别** 基于Transformer的语音转文本引擎，支持中文/英文多语种识别 -## 目前已集成的人脸算法模型 -- **RetinaFace 模型**[[GitHub]](https://github.com/deepinsight/insightface/tree/master/detection/retinaface)：一个高效的深度学习人脸检测模型，支持高精度的人脸检测，但目前不支持人脸比对 -- **Ultra-Light-Fast-Generic-Face-Detector-1MB** [[GitHub\]](https://github.com/Linzaer/Ultra-Light-Fast-Generic-Face-Detector-1MB)：一个轻量级的人脸检测模型，适用于需要较低延迟和较小模型尺寸的应用场景。 -- **Seetaface6** [[GitHub\]](https://github.com/seetafaceengine/SeetaFace6)：是中科视拓最新开放的商业正式级版本，支持人脸检测、关键点定位、人脸识别。同时增加了活体检测、质量评估、年龄性别估计。并且响应时事，开放了口罩检测以及戴口罩的人脸识别模型 -- **[facenet-pytorch](https://github.com/timesler/facenet-pytorch)** [[GitHub\]](https://github.com/seetafaceengine/SeetaFace6)：这是 pytorch 中 Inception Resnet (V1) 模型的存储库，在 VGGFace2 和 CASIA-Webface 上进行了预训练。Pytorch 模型权重使用从 David Sandberg 的 [tensorflow Facenet repo](https://github.com/davidsandberg/facenet) 移植的参数进行初始化。该存储库中还包含 MTCNN 的高效 pytorch 实现，用于推理之前的人脸检测。这些模型也是经过预训练的。据我们所知，这是最快的 MTCNN 实现。 +## 🛠️包含组件 -### 人脸模型对比及下载地址 +| 模块 | 介绍 | +| -------------------|---------------------------------------------------------------------------------- | +| smartjavaai-common | 基础通用模块，封装了公共功能，供各算法模块共享使用 | +| smartjavaai-face | 人脸功能模块 | +| smartjavaai-objectdetection | 目标检测模块 | +| smartjavaai-seetaface6-lib | seetaface6人脸算法JNI接口封装 | -| 模型名称 | 下载地址 | 文件大小 | 适用场景 | 兼容系统 | -| :-----------------------: | :----------------------------------------------------------: | :------: | :---------------: | ------------------- | -| retinaface | [下载](https://resources.djl.ai/test-models/pytorch/retinaface.zip) | 110MB | 高精度人脸检测 | Windows/Linux/MacOS | -| ultralightfastgenericface | [下载](https://resources.djl.ai/test-models/pytorch/ultranet.zip) | 1.7MB | 高速人脸检测 | Windows/Linux/MacOS | -| seetaface6 | [下载](https://pan.baidu.com/s/1hfNacA8ISV2qHrycjOkgqA?pwd=1234) | 288MB | 人脸检测/人脸识别 | Windows | -| facenet-pytorch | [下载](https://resources.djl.ai/test-models/pytorch/face_feature.zip) | 104MB | 人脸识别 | Windows/Linux/MacOS | +可以根据需求对每个模块单独引入 -## 环境要求 +------------------------------------------------------------------------------- + + + +## 📦 安装 + +#### 1、环境要求 - Java 版本：**JDK 11或更高版本** - 操作系统：不同模型支持的系统不一样，具体请查看文档 -## 使用步骤 - -> -> 📌 **运行提示**： -> -> （1）默认算法（RetinaFace）或轻量算法（Ultra-Light-Fast-Generic-Face-Detector ）都为python算法，兼容 Windows、Linux、MacOS，Android 等系统，SmartJavaAI首次启动将自动下载模型到及依赖库到本地（.djl.ai隐藏文件夹），建议保持网络畅通。初始化完成后，后续启动可实现毫秒级响应。在无网络环境下，可指定本地模型路径（需提前下载模型包）。目前，这两种算法不支持人脸识别或人脸比对功能。 -> -> （2）Seetaface6 采用 C++ 编写，兼容 Windows、CentOS、Ubuntu 等系统，虽然Seetaface6 支持linux，但是我们目前仅实现了windows，如果后续对linux需求多，我们将兼容linux。Seetaface6 支持全功能人脸处理（人脸检测、人脸比对 1:1 或 1:N）。SmartJavaAI 通过 JNI 调用 C++ 接口，不支持在线下载模型，需手动下载并存储至本地。使用人脸比对等功能时，需要将项目中db/faces-data.db存放到您本地路径下并在config中指定人脸库路径。 - -### 1. 安装人脸算法依赖 - -在 Maven 项目的 `pom.xml` 中添加 SmartJavaAI的人脸算法依赖： +#### 2、Maven +在项目的pom.xml的dependencies中加入以下内容（全部功能），也可以根据需求对每个模块单独引入: ```xml - - - ink.numberone - smartjavaai-face - 1.0.6 - - + + ink.numberone + smartjavaai-all + 1.0.8 + ``` +#### 3、人脸检测运行流程 -### 2. 人脸检测代码示例 +（1）人脸模型下载（使用人脸相关功能） -```java -//创建人脸算法 -FaceAlgorithm currentAlgorithm = FaceAlgorithmFactory.createFaceAlgorithm(); -//使用图片路径检测 -FaceDetectedResult result = currentAlgorithm.detect("src/main/resources/largest_selfie.jpg"); -``` +如果在有网环境下使用，不需要下载模型(SeetaFace6模型除外) -### 3. 轻量人脸检测代码示例 +| 模型名称 | 下载地址 | 文件大小 | 适用场景 | 兼容系统 | +| :-----------------------: | :----------------------------------------------------------: | :------: | :----------------------------------------: | ------------------- | +| retinaface | [下载](https://resources.djl.ai/test-models/pytorch/retinaface.zip) | 110MB | 高精度人脸检测 | Windows/Linux/MacOS | +| ultralightfastgenericface | [下载](https://resources.djl.ai/test-models/pytorch/ultranet.zip) | 1.7MB | 高速人脸检测 | Windows/Linux/MacOS | +| seetaface6 | [下载](https://pan.baidu.com/s/1hfNacA8ISV2qHrycjOkgqA?pwd=1234) | 288MB | 人脸检测、人脸比对、人脸库注册、人脸库查询 | Windows | +| facenet | [下载](https://resources.djl.ai/test-models/pytorch/face_feature.zip) | 104MB | 人脸特征提取、人脸比对 | Windows/Linux/MacOS | -```java -//创建人脸算法 -FaceAlgorithm currentAlgorithm = FaceAlgorithmFactory.createLightFaceAlgorithm(); -//使用图片路径检测 -FaceDetectedResult result = currentAlgorithm.detect("src/main/resources/largest_selfie.jpg"); -``` +（2）人脸库下载（使用人脸库相关功能：人脸注册、人脸查询） -### 4. 使用图片输入流检测 +目前仅SeetaFace6人脸算法支持人脸库注册，查询等功能，所以只有使用SeetaFace6模型时才需要下载`face.db`，`face.db` 是 一个SQLite 数据库，程序启动并使用相关功能时会自动操作该数据库，用于存储人脸特征数据及其对应的唯一标识 Key，支持后续的人脸注册、查询和比对等操作。 -```java -//支持各种输入流方式检测图片 -File input = new File("src/main/resources/largest_selfie.jpg"); -FaceDetectedResult result = currentAlgorithm.detect(new FileInputStream(input)); -``` +下载链接: https://pan.baidu.com/s/1DzE1rDkFnjEXQbIasIdFrA?pwd=1234 提取码: 1234 -### 5. 人脸比对（1：1） +（3）下载示例代码 -> -> （1）将项目中db/faces-data.db存放到您本地路径下 -> -> （2）下载模型到本地路径，下载地址：https://pan.baidu.com/s/1hfNacA8ISV2qHrycjOkgqA?pwd=1234 提取码：1234 -> -> （3）如果网盘地址请联系本人，文档最后有联系方式 +https://gitee.com/dengwenjie/SmartJavaAI-Demo -```java -// 初始化配置 -ModelConfig config = new ModelConfig(); -config.setAlgorithmName("seetaface6");//目前支持人脸比对的算法只有：seetaface6 -//人脸库路径 如果不指定人脸库，无法使用 1:N人脸搜索 -config.setFaceDbPath("C:/Users/Administrator/Downloads/faces-data.db"); -//改为模型存放路径 -config.setModelPath("/opt/sf3.0_models"); -//创建人脸算法 -FaceAlgorithm currentAlgorithm = FaceAlgorithmFactory.createFaceAlgorithm(config); -//自动裁剪人脸并比对人脸特征 -float similar = currentAlgorithm.featureComparison("src/main/resources/kana1.jpg","src/main/resources/kana2.jpg"); -log.info("相似度：{}", similar); -``` +`📁 src/main/java/smartai/examples/face +└── 📄[RetinaFaceDemo.java](https://gitee.com/dengwenjie/SmartJavaAI-Demo/blob/master/src/main/java/smartai/examples/face/RetinaFaceDemo.java) *（人脸模型：RetinaFace示例代码）* +└── 📄[LightFaceDemo](https://gitee.com/dengwenjie/SmartJavaAI-Demo/blob/master/src/main/java/smartai/examples/face/LightFaceDemo.java) *（人脸模型：UltraLightFastGenericFaceModel示例代码）* +└── 📄[SeetaFace6Demo.java](https://gitee.com/dengwenjie/SmartJavaAI-Demo/blob/master/src/main/java/smartai/examples/face/SeetaFace6Demo.java) *（人脸模型：SeetaFace6示例代码）* -### 6. 人脸特征提取及比对（seetaface6） +└── 📄[FaceNetDemo.java](https://gitee.com/dengwenjie/SmartJavaAI-Demo/blob/master/src/main/java/smartai/examples/face/FaceNetDemo.java) *（人脸模型：FaceNet示例代码）* -```java -// 初始化配置 -ModelConfig config = new ModelConfig(); -config.setAlgorithmName("seetaface6"); -//人脸库路径 如果不指定人脸库，无法使用 1:N人脸搜索 -config.setFaceDbPath("C:/Users/Administrator/Downloads/faces-data.db"); -//改为模型存放路径 -config.setModelPath("C:/Users/Administrator/Downloads/sf3.0_models/sf3.0_models"); -//创建人脸算法 -FaceAlgorithm currentAlgorithm = FaceAlgorithmFactory.createFaceAlgorithm(config); -//提取图像中最大人脸的特征 -float[] feature1 = currentAlgorithm.featureExtraction("src/main/resources/kana1.jpg"); -float[] feature2 = currentAlgorithm.featureExtraction("src/main/resources/kana2.jpg"); -float similar = currentAlgorithm.calculSimilar(feature1, feature2); -log.info("相似度：{}", similar); -``` +（4）离线使用方法 -### 7. 人脸特征提取及比对（facenet-pytorch） +程序首次运行时，会自动下载所需的底层依赖库到默认的缓存路径。不同操作系统的默认缓存路径如下： -```java -//创建脸算法 -FaceAlgorithm featureAlgorithm = FaceAlgorithmFactory.createFaceFeatureAlgorithm(); -//提取身份证人脸特征 -float[] feature1 = featureAlgorithm.featureExtraction("src/main/resources/kana1.jpg"); -float[] feature2 = featureAlgorithm.featureExtraction("src/main/resources/kana2.jpg"); -if (feature1 != null && feature2 != null) { - //相似度在0.8至0.85及以上时，可判定为同一人，但具体阈值可能因图片而异，存在一定误差。 - float similar = featureAlgorithm.calculSimilar(feature1, feature2); - log.info("相似度：{}", similar); -} else { - log.warn("人脸特征提取失败"); -} -``` +\{user}需要替换成您当前登录的用户名 -### 8. 注册及搜索人脸（1：N） +| | 依赖库及缓存目录 | +| ------- | ------------------------ | +| windows | C:/Users/\{user}/.djl.ai | +| linux | /home/{user}/.djl.ai | +| macos | /Users/{user}/.djl.ai | -```java -// 初始化配置 -ModelConfig config = new ModelConfig(); -config.setAlgorithmName("seetaface6"); -//人脸库路径 如果不指定人脸库，无法使用 1:N人脸搜索 -config.setFaceDbPath("C:/Users/Administrator/Downloads/faces-data.db"); -//改为模型存放路径 -config.setModelPath("C:/Users/Administrator/Downloads/sf3.0_models/sf3.0_models"); -//创建人脸算法 自动将人脸库加载到内存中 -FaceAlgorithm currentAlgorithm = FaceAlgorithmFactory.createFaceAlgorithm(config); -//等待人脸库加载完毕 -Thread.sleep(1000); -//注册kana1人脸，参数key建议设置为人名 -boolean isSuccss = currentAlgorithm.register("kana1","src/main/resources/kana1.jpg"); -//注册jsy人脸，参数key建议设置为人名 -isSuccss = currentAlgorithm.register("jsy","src/main/resources/jsy.jpg"); -FaceResult faceResult = currentAlgorithm.search("src/main/resources/kana2.jpg"); -if(faceResult != null){ - log.info("查询到人脸：{}", faceResult.toString()); -}else{ - log.info("未查询到人脸"); -} -``` +对于需要在离线环境中使用的情况，可以在联网环境中运行程序一次，确保所需的依赖库已下载。然后，将上述缓存目录复制到离线环境中相同的路径下，即可实现离线使用。 -### 9. 人脸检测（离线下载模型） +请注意，SeetaFace6 默认支持离线使用，无需上述操作即可在离线环境中运行。 -```java -// 初始化配置 -ModelConfig config = new ModelConfig(); -config.setAlgorithmName("retinaface");//人脸算法模型，目前支持：retinaface/ultralightfastgenericface/seetaface6 -//config.setAlgorithmName("ultralightfastgenericface");//轻量模型 -config.setConfidenceThreshold(FaceConfig.DEFAULT_CONFIDENCE_THRESHOLD);//置信度阈值 -config.setMaxFaceCount(FaceConfig.MAX_FACE_LIMIT);//每张特征图保留的最大候选框数量 -//nms阈值:控制重叠框的合并程度,取值越低，合并越多重叠框（减少误检但可能漏检）；取值越高，保留更多框（增加检出但可能引入冗余） -config.setNmsThresh(FaceConfig.NMS_THRESHOLD); -//模型下载地址： -//retinaface: https://resources.djl.ai/test-models/pytorch/retinaface.zip -//ultralightfastgenericface: https://resources.djl.ai/test-models/pytorch/ultranet.zip -//改为模型存放路径 -config.setModelPath("/Users/xxx/Documents/develop/face_model/retinaface.pt"); -//创建人脸算法 -FaceAlgorithm currentAlgorithm = FaceAlgorithmFactory.createFaceAlgorithm(config); -//使用图片路径检测 -FaceDetectedResult result = currentAlgorithm.detect("src/main/resources/largest_selfie.jpg"); -logger.info("人脸检测结果：{}", JSONObject.toJSONString(result)); -//使用图片流检测 -File input = new File("src/main/resources/largest_selfie.jpg"); -//FaceDetectedResult result = currentAlgorithm.detect(new FileInputStream(input)); -//logger.info("人脸检测结果：{}", JSONObject.toJSONString(result)); -BufferedImage image = ImageIO.read(input); -//创建保存路径 -Path imagePath = Paths.get("output").resolve("retinaface_detected.jpg"); -//绘制人脸框 -ImageUtils.drawBoundingBoxes(image, result, imagePath.toAbsolutePath().toString()); -``` +#### 4、目标检测运行流程 -## 完整代码 +（1）下载示例代码 -`📁 examples/src/main/java/smartai/examples/face` -└── 📄[FaceDemo.java](https://github.com/geekwenjie/SmartJavaAI/blob/master/examples/src/main/java/smartai/examples/face/FaceDemo.java) *（基于JDK11构建的完整可执行示例）* +https://gitee.com/dengwenjie/SmartJavaAI-Demo + +`📁 src/main/java/smartai/examples/objectdetection +└── 📄[ObjectDetection.java](https://gitee.com/dengwenjie/SmartJavaAI-Demo/blob/master/src/main/java/smartai/examples/objectdetection/ObjectDetection.java) *（目标检测示例代码）* @@ -259,7 +156,8 @@ ImageUtils.drawBoundingBoxes(image, result, imagePath.toAbsolutePath().toString( 本项目在开发过程中借鉴或使用了以下优秀开源项目，特此致谢： - **[Seetaface6JNI](https://gitee.com/cnsugar/seetaface6JNI)** ---- +- **[Deep Java Library](https://docs.djl.ai)** + ## 联系方式 @@ -276,6 +174,13 @@ ImageUtils.drawBoundingBoxes(image, result, imagePath.toAbsolutePath().toString( ## 更新日志 + +## [v1.0.8] - 2025-04-13 +- 新增目标检测功能 +- 模型调用接口统一封装 +- 修复若干已知问题 +- 支持自定义选择使用 GPU 或 CPU 运算 +- 人脸识别模块新增多种接口，功能更加完善 ## [v1.0.6] - 2025-04-01 - 修复人脸识别算法facenet-pytorch实现方式 - 优化Seetaface6算法，兼容jdk高版本 diff --git a/db/faces-data.db b/db/faces-data.db deleted file mode 100644 index cbbc660..0000000 Binary files a/db/faces-data.db and /dev/null differ diff --git a/examples/output/retinaface_detected.jpg b/examples/output/retinaface_detected.jpg deleted file mode 100644 index 50ec51d..0000000 Binary files a/examples/output/retinaface_detected.jpg and /dev/null differ diff --git a/examples/pom.xml b/examples/pom.xml deleted file mode 100644 index ea30c65..0000000 --- a/examples/pom.xml +++ /dev/null @@ -1,115 +0,0 @@ - - - 4.0.0 - - ink.numberone - examples - 1.0.0-SNAPSHOT - - - 11 - 11 - UTF-8 - 1.0.0-SNAPSHOT - smartai.examples.face.FaceDemo - - - - - - commons-cli - commons-cli - 1.9.0 - - - commons-io - commons-io - 2.17.0 - - - org.apache.logging.log4j - log4j-slf4j2-impl - 2.24.1 - - - org.testng - testng - 7.10.2 - test - - - - ink.numberone - smartjavaai-face - 1.0.6 - - - - ch.qos.logback - logback-classic - 1.2.3 - - - org.slf4j - slf4j-api - 1.7.30 - - - - com.alibaba - fastjson - 1.2.83 - - - - ai.djl.onnxruntime - onnxruntime-engine - 0.20.0 - - - - - example - - - org.apache.maven.plugins - maven-assembly-plugin - 2.3 - - - false - - jar-with-dependencies - - - - - true - - lib/ - - smartai.examples.face.FaceDemo - - - - - - make-assembly - - package - - assembly - - - - - - - - - - - - diff --git a/examples/src/main/java/smartai/examples/face/FaceDemo.java b/examples/src/main/java/smartai/examples/face/FaceDemo.java deleted file mode 100644 index 431344d..0000000 --- a/examples/src/main/java/smartai/examples/face/FaceDemo.java +++ /dev/null @@ -1,278 +0,0 @@ -package smartai.examples.face; - -import cn.smartjavaai.common.entity.Rectangle; -import cn.smartjavaai.face.*; -import cn.smartjavaai.face.entity.FaceResult; -import com.alibaba.fastjson.JSONObject; -import lombok.extern.slf4j.Slf4j; -import org.apache.commons.lang3.time.StopWatch; -import org.slf4j.Logger; -import org.slf4j.LoggerFactory; -import smartai.examples.utils.ImageUtils; - -import javax.imageio.ImageIO; -import java.awt.*; -import java.awt.image.BufferedImage; -import java.awt.image.RasterFormatException; -import java.io.File; -import java.io.FileInputStream; -import java.io.IOException; -import java.net.URL; -import java.nio.file.Files; -import java.nio.file.LinkOption; -import java.nio.file.Path; -import java.nio.file.Paths; - -/** - * @author dwj - */ -@Slf4j -public class FaceDemo { - - - public static void main(String[] args) { - try { - featureExtractionAndCompare2(); - //detectFace2(); - //verifyIDCard(); - } catch (Exception e) { - e.printStackTrace(); - } - } - - /** - * 人脸检测（服务端模型） - * 人脸模型：retinaface - * 特点：识别精度高，高速 - * 应用场景：如监控摄像头、智能安防系统等需要高精度检测的场合 - */ - public static void detectFace(){ - try { - //创建人脸算法 - FaceAlgorithm currentAlgorithm = FaceAlgorithmFactory.createFaceAlgorithm(); - //使用图片路径检测 - FaceDetectedResult result = currentAlgorithm.detect("src/main/resources/largest_selfie.jpg"); - log.info("人脸检测结果：{}", JSONObject.toJSONString(result)); - //使用图片流检测 - File input = new File("src/main/resources/largest_selfie.jpg"); - //FaceDetectedResult result = currentAlgorithm.detect(new FileInputStream(input)); - //log.info("人脸检测结果：{}", JSONObject.toJSONString(result)); - BufferedImage image = ImageIO.read(input); - //创建保存路径 - Path imagePath = Paths.get("output").resolve("retinaface_detected.jpg"); - //绘制人脸框 - ImageUtils.drawBoundingBoxes(image, result, imagePath.toAbsolutePath().toString()); - } catch (Exception e) { - e.printStackTrace(); - } - - } - - - /** - * 人脸检测（轻量模型） - * 人脸模型：Ultra-Light-Fast-Generic-Face-Detector-1MB - * 特点：高速，准确率略低 - * 应用场景：如监控摄像头、智能安防系统等需要高精度检测的场合 - */ - public static void detectFace2(){ - try { - //创建轻量人脸算法 - FaceAlgorithm currentAlgorithm = FaceAlgorithmFactory.createLightFaceAlgorithm(); - //使用图片路径检测 - FaceDetectedResult result = currentAlgorithm.detect("src/main/resources/largest_selfie.jpg"); - log.info("轻量人脸检测结果：{}", JSONObject.toJSONString(result)); - //使用图片流检测 - //File imageFile = new File("/Users/wenjie/Downloads/djl-master/examples/src/test/resources/largest_selfie.jpg"); - //FaceDetectedResult result = currentAlgorithm.detect(new FileInputStream(imageFile)); - File input = new File("src/main/resources/largest_selfie.jpg"); - BufferedImage image = ImageIO.read(input); - //创建保存路径 - Path imagePath = Paths.get("output").resolve("retinaface_detected.jpg"); - //绘制人脸框 - ImageUtils.drawBoundingBoxes(image, result, imagePath.toAbsolutePath().toString()); - } catch (Exception e) { - e.printStackTrace(); - } - } - - - - /** - * 人脸检测（离线模型） - * 人脸模型：retinaface - * 特点：识别精度高，高速 - * 应用场景：如监控摄像头、智能安防系统等需要高精度检测的场合 - */ - public static void detectFaceOffine(){ - try { - // 初始化配置 - ModelConfig config = new ModelConfig(); - config.setAlgorithmName("retinaface");//人脸算法模型，目前支持：retinaface/ultralightfastgenericface/seetaface6 - //config.setAlgorithmName("ultralightfastgenericface");//轻量模型 - config.setConfidenceThreshold(FaceConfig.DEFAULT_CONFIDENCE_THRESHOLD);//置信度阈值 - config.setMaxFaceCount(FaceConfig.MAX_FACE_LIMIT);//每张特征图保留的最大候选框数量 - //nms阈值:控制重叠框的合并程度,取值越低，合并越多重叠框（减少误检但可能漏检）；取值越高，保留更多框（增加检出但可能引入冗余） - config.setNmsThresh(FaceConfig.NMS_THRESHOLD); - //模型下载地址： - //retinaface: https://resources.djl.ai/test-models/pytorch/retinaface.zip - //ultralightfastgenericface: https://resources.djl.ai/test-models/pytorch/ultranet.zip - //改为模型存放路径 - config.setModelPath("/Users/wenjie/Documents/develop/face_model/retinaface.pt"); - //创建人脸算法 - FaceAlgorithm currentAlgorithm = FaceAlgorithmFactory.createFaceAlgorithm(config); - //使用图片路径检测 - FaceDetectedResult result = currentAlgorithm.detect("src/main/resources/largest_selfie.jpg"); - log.info("人脸检测结果：{}", JSONObject.toJSONString(result)); - //使用图片流检测 - File input = new File("src/main/resources/largest_selfie.jpg"); - //FaceDetectedResult result = currentAlgorithm.detect(new FileInputStream(input)); - //logger.info("人脸检测结果：{}", JSONObject.toJSONString(result)); - BufferedImage image = ImageIO.read(input); - //创建保存路径 - Path imagePath = Paths.get("output").resolve("retinaface_detected.jpg"); - //绘制人脸框 - ImageUtils.drawBoundingBoxes(image, result, imagePath.toAbsolutePath().toString()); - } catch (Exception e) { - e.printStackTrace(); - } - } - - /** - * 人脸比对（1：1） - * @throws Exception - */ - public static void featureComparison(){ - try { - // 初始化配置 - ModelConfig config = new ModelConfig(); - config.setAlgorithmName("seetaface6");//目前支持人脸比对的算法只有：seetaface6 - //人脸库路径 如果不指定人脸库，无法使用 1:N人脸搜索 - config.setFaceDbPath("C:/Users/Administrator/Downloads/faces-data.db"); - //改为模型存放路径 - config.setModelPath("/opt/sf3.0_models"); - //创建人脸算法 - FaceAlgorithm currentAlgorithm = FaceAlgorithmFactory.createFaceAlgorithm(config); - //自动裁剪人脸并比对人脸特征 - float similar = currentAlgorithm.featureComparison("src/main/resources/kana1.jpg","src/main/resources/kana2.jpg"); - log.info("相似度：{}", similar); - } - catch (Exception e){ - e.printStackTrace(); - } - } - - /** - * seetaface6人脸特征提取及比对（可人证核验） - * 目前仅支持windows 64位系统，如需支持其他操作系统可参考方法：featureExtractionAndCompare2 - */ - public static void featureExtractionAndCompare(){ - try { - // 初始化配置 - ModelConfig config = new ModelConfig(); - config.setAlgorithmName("seetaface6"); - //人脸库路径 如果不指定人脸库，无法使用 1:N人脸搜索 - config.setFaceDbPath("C:/Users/Administrator/Downloads/faces-data.db"); - //改为模型存放路径 - config.setModelPath("C:/Users/Administrator/Downloads/sf3.0_models/sf3.0_models"); - //创建人脸算法 - FaceAlgorithm currentAlgorithm = FaceAlgorithmFactory.createFaceAlgorithm(config); - //提取图像中最大人脸的特征 - float[] feature1 = currentAlgorithm.featureExtraction("src/main/resources/kana1.jpg"); - float[] feature2 = currentAlgorithm.featureExtraction("src/main/resources/kana2.jpg"); - if(feature1 != null && feature2 != null){ - float similar = currentAlgorithm.calculSimilar(feature1, feature2); - log.info("相似度：{}", similar); - }else{ - log.warn("人脸特征提取失败"); - } - } - catch (Exception e){ - e.printStackTrace(); - } - } - - /** - * facenet-pytorch 人脸特征提取及比对（可人证核验） - * 支持windows，linux，macos - */ - public static void featureExtractionAndCompare2(){ - try { - //创建脸算法 - FaceAlgorithm featureAlgorithm = FaceAlgorithmFactory.createFaceFeatureAlgorithm(); - //提取身份证人脸特征 - float[] feature1 = featureAlgorithm.featureExtraction("src/main/resources/kana1.jpg"); - float[] feature2 = featureAlgorithm.featureExtraction("src/main/resources/kana2.jpg"); - if (feature1 != null && feature2 != null) { - //相似度在0.8至0.85及以上时，可判定为同一人，但具体阈值可能因图片而异，存在一定误差。 - float similar = featureAlgorithm.calculSimilar(feature1, feature2); - log.info("相似度：{}", similar); - } else { - log.warn("人脸特征提取失败"); - } - } catch (Exception e) { - e.printStackTrace(); - } - } - - /** - * 注册人脸及搜索人脸（1：N） - */ - public static void registerAndSearchFace(){ - try { - // 初始化配置 - ModelConfig config = new ModelConfig(); - config.setAlgorithmName("seetaface6"); - //人脸库路径 如果不指定人脸库，无法使用 1:N人脸搜索 - config.setFaceDbPath("C:/Users/Administrator/Downloads/faces-data.db"); - //改为模型存放路径 - config.setModelPath("C:/Users/Administrator/Downloads/sf3.0_models/sf3.0_models"); - //创建人脸算法 自动将人脸库加载到内存中 - FaceAlgorithm currentAlgorithm = FaceAlgorithmFactory.createFaceAlgorithm(config); - //等待人脸库加载完毕 - Thread.sleep(1000); - //注册kana1人脸，参数key建议设置为人名 - boolean isSuccss = currentAlgorithm.register("kana1","src/main/resources/kana1.jpg"); - //注册jsy人脸，参数key建议设置为人名 - isSuccss = currentAlgorithm.register("jsy","src/main/resources/jsy.jpg"); - FaceResult faceResult = currentAlgorithm.search("src/main/resources/kana2.jpg"); - if(faceResult != null){ - log.info("查询到人脸：{}", faceResult.toString()); - }else{ - log.info("未查询到人脸"); - } - - } - catch (Exception e){ - e.printStackTrace(); - } - } - - /** - * 删除已注册人脸 - */ - public static void removeRegisterFace(){ - try { - // 初始化配置 - ModelConfig config = new ModelConfig(); - config.setAlgorithmName("seetaface6"); - //人脸库路径 如果不指定人脸库，无法使用 1:N人脸搜索 - config.setFaceDbPath("C:/Users/Administrator/Downloads/faces-data.db"); - //改为模型存放路径 - config.setModelPath("C:/Users/Administrator/Downloads/sf3.0_models/sf3.0_models"); - //创建人脸算法 自动将人脸库加载到内存中 - FaceAlgorithm currentAlgorithm = FaceAlgorithmFactory.createFaceAlgorithm(config); - //等待人脸库加载完毕 - Thread.sleep(1000); - //使用注册人脸时的key值删除，可一次性删除单个 - long num = currentAlgorithm.removeRegister("kana1"); - //删除全部人脸 - //long num = currentAlgorithm.clearFace(); - log.info("删除成功数量：" + num); - } - catch (Exception e){ - e.printStackTrace(); - } - } - -} diff --git a/examples/src/main/java/smartai/examples/utils/ImageUtils.java b/examples/src/main/java/smartai/examples/utils/ImageUtils.java deleted file mode 100644 index 1914760..0000000 --- a/examples/src/main/java/smartai/examples/utils/ImageUtils.java +++ /dev/null @@ -1,56 +0,0 @@ -package smartai.examples.utils; - -import cn.smartjavaai.face.FaceDetectedResult; - -import javax.imageio.ImageIO; -import java.awt.*; -import java.awt.image.BufferedImage; -import java.io.File; -import java.io.IOException; -import java.nio.file.Files; -import java.nio.file.Path; -import java.nio.file.Paths; - -/** - * @author dwj - */ -public class ImageUtils { - - /** - * 绘制人脸框 - * @param sourceImage - * @param faceDetectedResult - * @param savePath - * @throws IOException - */ - public static void drawBoundingBoxes(BufferedImage sourceImage, FaceDetectedResult faceDetectedResult,String savePath) throws IOException { - Graphics2D graphics = sourceImage.createGraphics(); - graphics.setColor(Color.RED);// 边框颜色 - graphics.setStroke(new BasicStroke(2)); // 线宽2像素 - graphics.setRenderingHint(RenderingHints.KEY_ANTIALIASING, - RenderingHints.VALUE_ANTIALIAS_ON); // 抗锯齿 - int stroke = 2; - for(cn.smartjavaai.common.entity.Rectangle rectangle : faceDetectedResult.getRectangles()){ - graphics.setColor(Color.RED);// 边框颜色 - graphics.drawRect(rectangle.getPointList().get(0).getX(), - rectangle.getPointList().get(0).getY(), rectangle.getWidth(), rectangle.getHeight()); - drawText(graphics, "face", rectangle.getPointList().get(0).getX(), rectangle.getPointList().get(0).getY(), stroke, 4); - } - graphics.dispose(); - ImageIO.write(sourceImage, "jpg", new File(savePath)); - - } - - private static void drawText(Graphics2D g, String text, int x, int y, int stroke, int padding) { - FontMetrics metrics = g.getFontMetrics(); - x += stroke / 2; - y += stroke / 2; - int width = metrics.stringWidth(text) + padding * 2 - stroke / 2; - int height = metrics.getHeight() + metrics.getDescent(); - int ascent = metrics.getAscent(); - Rectangle background = new Rectangle(x, y, width, height); - g.fill(background); - g.setPaint(Color.WHITE); - g.drawString(text, x + padding, y + ascent); - } -} diff --git a/examples/src/main/resources/jsy.jpg b/examples/src/main/resources/jsy.jpg deleted file mode 100644 index c4640e7..0000000 Binary files a/examples/src/main/resources/jsy.jpg and /dev/null differ diff --git a/examples/src/main/resources/kana1.jpg b/examples/src/main/resources/kana1.jpg deleted file mode 100644 index ef364e0..0000000 Binary files a/examples/src/main/resources/kana1.jpg and /dev/null differ diff --git a/examples/src/main/resources/kana2.jpg b/examples/src/main/resources/kana2.jpg deleted file mode 100644 index 59c9e52..0000000 Binary files a/examples/src/main/resources/kana2.jpg and /dev/null differ diff --git a/examples/src/main/resources/largest_selfie.jpg b/examples/src/main/resources/largest_selfie.jpg deleted file mode 100644 index 605ec97..0000000 Binary files a/examples/src/main/resources/largest_selfie.jpg and /dev/null differ diff --git a/examples/src/main/resources/logback.xml b/examples/src/main/resources/logback.xml deleted file mode 100644 index 8f006d9..0000000 --- a/examples/src/main/resources/logback.xml +++ /dev/null @@ -1,14 +0,0 @@ - - - - - - - %d{yyyy-MM-dd HH:mm:ss.SSS} [%thread] %highlight(%-5level) %cyan(%logger{36}) - %msg%n - - - - - - - diff --git a/pom.xml b/pom.xml index 97803f1..3630892 100644 --- a/pom.xml +++ b/pom.xml @@ -6,13 +6,15 @@ ink.numberone smartjavaai-parent - 1.0.6 + 1.0.8 pom SmartJavaAI smartjavaai-face smartjavaai-common smartjavaai-seetaface6-lib + smartjavaai-objectdetection + smartjavaai-all @@ -35,13 +37,13 @@ ink.numberone smartjavaai-common - 1.0.6 + 1.0.8 ink.numberone smartjavaai-face - 1.0.6 + 1.0.8 @@ -51,56 +53,46 @@ ai.djl api - ${djl.version} ai.djl basicdataset - ${djl.version} ai.djl model-zoo - ${djl.version} ai.djl.timeseries timeseries - ${djl.version} ai.djl.huggingface tokenizers - ${djl.version} ai.djl.audio audio - ${djl.version} ai.djl.mxnet mxnet-model-zoo - ${djl.version} ai.djl.pytorch pytorch-model-zoo - ${djl.version} ai.djl.tensorflow tensorflow-model-zoo - ${djl.version} ai.djl.onnxruntime onnxruntime-engine - ${djl.version} org.testng @@ -121,47 +113,6 @@ 3.9 - - ai.djl.pytorch - pytorch-engine - 0.32.0 - runtime - - - - ai.djl.pytorch - pytorch-native-cpu - win-x86_64 - runtime - 2.5.1 - - - ai.djl.pytorch - pytorch-native-cpu - linux-x86_64 - runtime - 2.5.1 - - - ai.djl.pytorch - pytorch-native-cpu-precxx11 - linux-aarch64 - runtime - 2.5.1 - - - ai.djl.pytorch - pytorch-native-cpu - osx-aarch64 - 2.5.1 - runtime - - - ai.djl.pytorch - pytorch-jni - 2.5.1-0.32.0 - runtime - @@ -194,8 +145,52 @@ 2.17.0 + + + org.apache.commons + commons-pool2 + 2.12.0 + + + + + + ai.djl.pytorch + pytorch-engine + runtime + + + ai.djl.tensorflow + tensorflow-engine + runtime + + + + ai.djl.mxnet + mxnet-engine + runtime + + + ai.djl.onnxruntime + onnxruntime-engine + + + ai.djl.ml.xgboost + xgboost + runtime + + + ai.djl.tensorrt + tensorrt + runtime + + + + + + https://github.com/geekwenjie/SmartJavaAI diff --git a/smartjavaai-all/pom.xml b/smartjavaai-all/pom.xml new file mode 100644 index 0000000..b3d4953 --- /dev/null +++ b/smartjavaai-all/pom.xml @@ -0,0 +1,139 @@ + + + 4.0.0 + + ink.numberone + smartjavaai-parent + 1.0.8 + + + smartjavaai-all + 1.0.8 + smartjavaai-all + SmartJavaAI + https://github.com/geekwenjie/SmartJavaAI + + + MIT License + https://opensource.org/licenses/MIT + + + + + + ink.numberone + smartjavaai-face + ${project.version} + + + + ink.numberone + smartjavaai-objectdetection + ${project.version} + + + + + 11 + 11 + UTF-8 + true + + + + + + org.sonatype.central + central-publishing-maven-plugin + 0.4.0 + true + + dengwenjie + true + ${project.groupId}:${project.artifactId}:${project.version} + + + + + org.apache.maven.plugins + maven-source-plugin + 3.1.0 + + + attach-sources + + jar-no-fork + + + + + + org.apache.maven.plugins + maven-javadoc-plugin + 3.1.0 + + ${java.home}/bin/javadoc + none + + -Xdoclint:none + + + + + attach-javadocs + + jar + + + + + + org.apache.maven.plugins + maven-gpg-plugin + 3.1.0 + + + sign-artifacts + verify + + sign + + + + + + + + + + scm:git:git://github.com/geekwenjie/SmartJavaAI.git + scm:git:ssh://github.com/geekwenjie/SmartJavaAI.git + http://github.com/geekwenjie/SmartJavaAI/tree/master + + + + + + dengwenjie + https://s01.oss.sonatype.org/content/repositories/snapshots + + + dengwenjie + https://s01.oss.sonatype.org/service/local/staging/deploy/maven2/ + + + + + + dengwenjie + 775747758@qq.com + + Project Manager + Architect + + + + + diff --git a/smartjavaai-common/pom.xml b/smartjavaai-common/pom.xml index bffeb9b..b0ef058 100644 --- a/smartjavaai-common/pom.xml +++ b/smartjavaai-common/pom.xml @@ -6,7 +6,7 @@ ink.numberone smartjavaai-parent - 1.0.6 + 1.0.8 smartjavaai-common diff --git a/smartjavaai-common/src/main/java/cn/smartjavaai/common/entity/DetectionRectangle.java b/smartjavaai-common/src/main/java/cn/smartjavaai/common/entity/DetectionRectangle.java new file mode 100644 index 0000000..3a4b2e2 --- /dev/null +++ b/smartjavaai-common/src/main/java/cn/smartjavaai/common/entity/DetectionRectangle.java @@ -0,0 +1,84 @@ +package cn.smartjavaai.common.entity; + +/** + * 检测结果-矩形区域 + * @author dwj + */ +public class DetectionRectangle { + + public int x; + public int y; + public int width; + public int height; + public float score; + + public String className; + + public DetectionRectangle() { + } + + public DetectionRectangle(int x, int y, int width, int height, float score) { + this.x = x; + this.y = y; + this.width = width; + this.height = height; + this.score = score; + } + + public DetectionRectangle(int x, int y, int width, int height, float score, String className) { + this.x = x; + this.y = y; + this.width = width; + this.height = height; + this.score = score; + this.className = className; + } + + public int getX() { + return x; + } + + public void setX(int x) { + this.x = x; + } + + public int getY() { + return y; + } + + public void setY(int y) { + this.y = y; + } + + public int getWidth() { + return width; + } + + public void setWidth(int width) { + this.width = width; + } + + public int getHeight() { + return height; + } + + public void setHeight(int height) { + this.height = height; + } + + public float getScore() { + return score; + } + + public void setScore(float score) { + this.score = score; + } + + public String getClassName() { + return className; + } + + public void setClassName(String className) { + this.className = className; + } +} diff --git a/smartjavaai-common/src/main/java/cn/smartjavaai/common/entity/DetectionResponse.java b/smartjavaai-common/src/main/java/cn/smartjavaai/common/entity/DetectionResponse.java new file mode 100644 index 0000000..7ee6f48 --- /dev/null +++ b/smartjavaai-common/src/main/java/cn/smartjavaai/common/entity/DetectionResponse.java @@ -0,0 +1,21 @@ +package cn.smartjavaai.common.entity; + +import java.util.List; + +/** + * 检测结果 + * @author dwj + * @date 2025/4/12 + */ +public class DetectionResponse { + + private List rectangleList; + + public List getRectangleList() { + return rectangleList; + } + + public void setRectangleList(List rectangleList) { + this.rectangleList = rectangleList; + } +} diff --git a/smartjavaai-common/src/main/java/cn/smartjavaai/common/entity/Rectangle.java b/smartjavaai-common/src/main/java/cn/smartjavaai/common/entity/Rectangle.java deleted file mode 100644 index dad5e90..0000000 --- a/smartjavaai-common/src/main/java/cn/smartjavaai/common/entity/Rectangle.java +++ /dev/null @@ -1,49 +0,0 @@ -package cn.smartjavaai.common.entity; - -import java.util.List; - -/** - * 矩形区域 - * @author dwj - */ -public class Rectangle { - - /** - * 矩形区域点集合 - */ - List pointList; - - /** - * 矩形区域宽度 - */ - int width; - - /** - * 矩形区域高度 - */ - int height; - - public int getWidth() { - return width; - } - - public void setWidth(int width) { - this.width = width; - } - - public int getHeight() { - return height; - } - - public void setHeight(int height) { - this.height = height; - } - - public List getPointList() { - return pointList; - } - - public void setPointList(List pointList) { - this.pointList = pointList; - } -} diff --git a/smartjavaai-common/src/main/java/cn/smartjavaai/common/enums/DeviceEnum.java b/smartjavaai-common/src/main/java/cn/smartjavaai/common/enums/DeviceEnum.java new file mode 100644 index 0000000..afbc40f --- /dev/null +++ b/smartjavaai-common/src/main/java/cn/smartjavaai/common/enums/DeviceEnum.java @@ -0,0 +1,14 @@ +package cn.smartjavaai.common.enums; + +/** + * 目标检测模型枚举 + * @author dwj + * @date 2025/4/4 + */ +public enum DeviceEnum { + + // resnet50 系列 + CPU, + GPU; + +} diff --git a/smartjavaai-common/src/main/java/cn/smartjavaai/common/pool/ModelPredictorPoolManager.java b/smartjavaai-common/src/main/java/cn/smartjavaai/common/pool/ModelPredictorPoolManager.java new file mode 100644 index 0000000..27ae467 --- /dev/null +++ b/smartjavaai-common/src/main/java/cn/smartjavaai/common/pool/ModelPredictorPoolManager.java @@ -0,0 +1,71 @@ +package cn.smartjavaai.common.pool; + +import ai.djl.inference.Predictor; +import ai.djl.repository.zoo.ZooModel; +import org.apache.commons.pool2.PooledObject; +import org.apache.commons.pool2.BasePooledObjectFactory; +import org.apache.commons.pool2.impl.DefaultPooledObject; +import org.apache.commons.pool2.impl.GenericObjectPool; +import org.apache.commons.pool2.impl.GenericObjectPoolConfig; + +import java.util.Map; +import java.util.concurrent.ConcurrentHashMap; + + +/** + * 模型共享池管理器 + * @author dwj + * @date 2025/4/8 + */ +public class ModelPredictorPoolManager { + + // 每个模型的唯一key -> 对应Predictor池 + private final Map>> poolMap = new ConcurrentHashMap<>(); + + /** + * 注册模型池 + * @param key 模型标识符（自定义，如模型路径、模型名等） + * @param model 模型本体 + * @param config 池配置（可选） + */ + public void registerModel(String key, ZooModel model, GenericObjectPoolConfig> config) { + PredictorFactory factory = new PredictorFactory<>(model); + GenericObjectPool> pool = new GenericObjectPool<>(factory, config); + poolMap.put(key, pool); + } + + /** + * 借出一个 Predictor + */ + @SuppressWarnings("unchecked") + public Predictor borrowPredictor(String key) throws Exception { + GenericObjectPool> pool = (GenericObjectPool>) poolMap.get(key); + if (pool == null) { + throw new IllegalArgumentException("模型未注册: " + key); + } + return pool.borrowObject(); + } + + /** + * 归还一个 Predictor + */ + @SuppressWarnings("unchecked") + public void returnPredictor(String key, Predictor predictor) { + GenericObjectPool> pool = (GenericObjectPool>) poolMap.get(key); + if (pool != null) { + pool.returnObject(predictor); + } + } + + /** + * 销毁全部池 + */ + public void closeAll() { + for (GenericObjectPool> pool : poolMap.values()) { + pool.close(); + } + poolMap.clear(); + } + + +} diff --git a/smartjavaai-common/src/main/java/cn/smartjavaai/common/pool/PredictorFactory.java b/smartjavaai-common/src/main/java/cn/smartjavaai/common/pool/PredictorFactory.java new file mode 100644 index 0000000..582ddca --- /dev/null +++ b/smartjavaai-common/src/main/java/cn/smartjavaai/common/pool/PredictorFactory.java @@ -0,0 +1,35 @@ +package cn.smartjavaai.common.pool; + +import ai.djl.inference.Predictor; +import ai.djl.repository.zoo.ZooModel; +import org.apache.commons.pool2.BasePooledObjectFactory; +import org.apache.commons.pool2.PooledObject; +import org.apache.commons.pool2.impl.DefaultPooledObject; + +/** + * Predictor 工厂类 + * @author dwj + * @date 2025/4/8 + */ +public class PredictorFactory extends BasePooledObjectFactory> { + private final ZooModel model; + + public PredictorFactory(ZooModel model) { + this.model = model; + } + + @Override + public Predictor create() { + return model.newPredictor(); + } + + @Override + public PooledObject> wrap(Predictor predictor) { + return new DefaultPooledObject<>(predictor); + } + + @Override + public void destroyObject(PooledObject> p) { + p.getObject().close(); + } +} diff --git a/smartjavaai-common/src/main/java/cn/smartjavaai/common/utils/FileUtils.java b/smartjavaai-common/src/main/java/cn/smartjavaai/common/utils/FileUtils.java new file mode 100644 index 0000000..67afd6f --- /dev/null +++ b/smartjavaai-common/src/main/java/cn/smartjavaai/common/utils/FileUtils.java @@ -0,0 +1,21 @@ +package cn.smartjavaai.common.utils; + +import java.io.File; + +/** + * 文件操作工具类 + * @author dwj + * @date 2025/4/4 + */ +public class FileUtils { + + /** + * 检查文件是否存在 + * @param filePath + * @return + */ + public static boolean isFileExists(String filePath) { + File file = new File(filePath); + return file.exists() && !file.isDirectory(); // 确保是文件且存在 + } +} diff --git a/smartjavaai-common/src/main/java/cn/smartjavaai/common/utils/ImageUtils.java b/smartjavaai-common/src/main/java/cn/smartjavaai/common/utils/ImageUtils.java index 6cfd4a1..a9e9d92 100644 --- a/smartjavaai-common/src/main/java/cn/smartjavaai/common/utils/ImageUtils.java +++ b/smartjavaai-common/src/main/java/cn/smartjavaai/common/utils/ImageUtils.java @@ -1,8 +1,12 @@ package cn.smartjavaai.common.utils; +import javax.imageio.ImageIO; +import java.awt.*; import java.awt.image.BufferedImage; //import java.awt.image.ColorConvertOp; import java.awt.image.ComponentSampleModel; +import java.io.File; +import java.io.IOException; import java.util.Arrays; /** @@ -74,4 +78,18 @@ public class ImageUtils { return image; } + /** + * 检查图像是否有效 + * @param image + * @return + */ + public static boolean isImageValid(BufferedImage image) { + // 检查是否为 null 或尺寸异常（如宽高为0） + return image != null && image.getWidth() > 0 && image.getHeight() > 0; + } + + + + + } diff --git a/smartjavaai-face/pom.xml b/smartjavaai-face/pom.xml index 0f22a62..6bb03a7 100644 --- a/smartjavaai-face/pom.xml +++ b/smartjavaai-face/pom.xml @@ -6,11 +6,11 @@ ink.numberone smartjavaai-parent - 1.0.6 + 1.0.8 smartjavaai-face - 1.0.6 + 1.0.8 smartjavaai-face SmartJavaAI https://github.com/geekwenjie/SmartJavaAI diff --git a/smartjavaai-face/src/main/java/cn/smartjavaai/face/AbstractFaceAlgorithm.java b/smartjavaai-face/src/main/java/cn/smartjavaai/face/AbstractFaceAlgorithm.java deleted file mode 100644 index b58f914..0000000 --- a/smartjavaai-face/src/main/java/cn/smartjavaai/face/AbstractFaceAlgorithm.java +++ /dev/null @@ -1,87 +0,0 @@ -package cn.smartjavaai.face; - -import cn.smartjavaai.face.entity.FaceResult; - -import java.io.IOException; -import java.io.InputStream; - -/** - * 人脸识别算法 - * @author dwj - */ -public abstract class AbstractFaceAlgorithm implements FaceAlgorithm{ - @Override - public void loadModel(ModelConfig config) throws Exception { - throw new UnsupportedOperationException("默认不支持该功能"); - } - - @Override - public void loadFaceFeatureModel(ModelConfig config) throws Exception { - throw new UnsupportedOperationException("默认不支持该功能"); - } - - @Override - public FaceDetectedResult detect(String imagePath) throws Exception { - throw new UnsupportedOperationException("默认不支持该功能"); - } - - @Override - public FaceDetectedResult detect(InputStream imageInputStream) throws Exception { - return null; - } - - @Override - public float[] featureExtraction(String imagePath) throws Exception { - throw new UnsupportedOperationException("默认不支持该功能"); - } - - @Override - public float[] featureExtraction(InputStream inputStream) throws Exception { - throw new UnsupportedOperationException("默认不支持该功能"); - } - - @Override - public float calculSimilar(float[] feature1, float[] feature2) throws Exception { - throw new UnsupportedOperationException("默认不支持该功能"); - } - - @Override - public float featureComparison(String imagePath1, String imagePath2) throws Exception { - throw new UnsupportedOperationException("默认不支持该功能"); - } - - @Override - public float featureComparison(InputStream inputStream1, InputStream inputStream2) throws Exception { - throw new UnsupportedOperationException("默认不支持该功能"); - } - - @Override - public boolean register(String key, String imagePath) throws Exception { - throw new UnsupportedOperationException("默认不支持该功能"); - } - - @Override - public boolean register(String key, InputStream inputStream) throws Exception { - throw new UnsupportedOperationException("默认不支持该功能"); - } - - @Override - public FaceResult search(String imagePath) throws Exception{ - throw new UnsupportedOperationException("默认不支持该功能"); - } - - @Override - public FaceResult search(InputStream inputStream) throws Exception{ - throw new UnsupportedOperationException("默认不支持该功能"); - } - - @Override - public long removeRegister(String... keys) throws Exception{ - throw new UnsupportedOperationException("默认不支持该功能"); - } - - @Override - public long clearFace() throws Exception{ - throw new UnsupportedOperationException("默认不支持该功能"); - } -} diff --git a/smartjavaai-face/src/main/java/cn/smartjavaai/face/AbstractFaceModel.java b/smartjavaai-face/src/main/java/cn/smartjavaai/face/AbstractFaceModel.java new file mode 100644 index 0000000..f4a5a8d --- /dev/null +++ b/smartjavaai-face/src/main/java/cn/smartjavaai/face/AbstractFaceModel.java @@ -0,0 +1,143 @@ +package cn.smartjavaai.face; + +import cn.smartjavaai.common.entity.DetectionResponse; +import cn.smartjavaai.face.entity.FaceResult; + +import java.awt.image.BufferedImage; +import java.io.InputStream; + +/** + * 人脸识别算法 + * @author dwj + */ +public abstract class AbstractFaceModel implements FaceModel { + @Override + public void loadModel(FaceModelConfig config) { + throw new UnsupportedOperationException("默认不支持该功能"); + } + + @Override + public DetectionResponse detect(String imagePath) { + throw new UnsupportedOperationException("默认不支持该功能"); + } + + @Override + public DetectionResponse detect(InputStream imageInputStream) { + throw new UnsupportedOperationException("默认不支持该功能"); + } + + @Override + public DetectionResponse detect(BufferedImage image) { + throw new UnsupportedOperationException("默认不支持该功能"); + } + + @Override + public DetectionResponse detect(byte[] imageData) { + throw new UnsupportedOperationException("默认不支持该功能"); + } + + @Override + public void detectAndDraw(String imagePath, String outputPath) { + throw new UnsupportedOperationException("默认不支持该功能"); + } + + @Override + public BufferedImage detectAndDraw(BufferedImage sourceImage) { + throw new UnsupportedOperationException("默认不支持该功能"); + } + + @Override + public float[] featureExtraction(String imagePath) { + throw new UnsupportedOperationException("默认不支持该功能"); + } + + @Override + public float[] featureExtraction(InputStream inputStream) { + throw new UnsupportedOperationException("默认不支持该功能"); + } + + @Override + public float calculSimilar(float[] feature1, float[] feature2) { + throw new UnsupportedOperationException("默认不支持该功能"); + } + + @Override + public float featureComparison(String imagePath1, String imagePath2) { + throw new UnsupportedOperationException("默认不支持该功能"); + } + + @Override + public float featureComparison(InputStream inputStream1, InputStream inputStream2) { + throw new UnsupportedOperationException("默认不支持该功能"); + } + + @Override + public boolean register(String key, String imagePath) { + throw new UnsupportedOperationException("默认不支持该功能"); + } + + @Override + public boolean register(String key, InputStream inputStream) { + throw new UnsupportedOperationException("默认不支持该功能"); + } + + @Override + public boolean register(String key, BufferedImage sourceImage) { + throw new UnsupportedOperationException("默认不支持该功能"); + } + + @Override + public boolean register(String key, byte[] imageData) { + throw new UnsupportedOperationException("默认不支持该功能"); + } + + @Override + public FaceResult search(String imagePath) { + throw new UnsupportedOperationException("默认不支持该功能"); + } + + @Override + public FaceResult search(InputStream inputStream) { + throw new UnsupportedOperationException("默认不支持该功能"); + } + + @Override + public long removeRegister(String... keys) { + throw new UnsupportedOperationException("默认不支持该功能"); + } + + @Override + public long clearFace() { + throw new UnsupportedOperationException("默认不支持该功能"); + } + + @Override + public float[] featureExtraction(BufferedImage sourceImage) { + throw new UnsupportedOperationException("默认不支持该功能"); + } + + @Override + public float[] featureExtraction(byte[] imageData) { + throw new UnsupportedOperationException("默认不支持该功能"); + } + + @Override + public float featureComparison(BufferedImage sourceImage1, BufferedImage sourceImag2) { + throw new UnsupportedOperationException("默认不支持该功能"); + } + + @Override + public float featureComparison(byte[] imageData1, byte[] imageData2) { + throw new UnsupportedOperationException("默认不支持该功能"); + } + + @Override + public FaceResult search(BufferedImage sourceImage) { + throw new UnsupportedOperationException("默认不支持该功能"); + } + + @Override + public FaceResult search(byte[] imageData) { + throw new UnsupportedOperationException("默认不支持该功能"); + } +} diff --git a/smartjavaai-face/src/main/java/cn/smartjavaai/face/FaceAlgorithm.java b/smartjavaai-face/src/main/java/cn/smartjavaai/face/FaceAlgorithm.java deleted file mode 100644 index b42dcd8..0000000 --- a/smartjavaai-face/src/main/java/cn/smartjavaai/face/FaceAlgorithm.java +++ /dev/null @@ -1,133 +0,0 @@ -package cn.smartjavaai.face; - -import ai.djl.MalformedModelException; -import ai.djl.repository.zoo.ModelNotFoundException; -import ai.djl.translate.TranslateException; -import cn.smartjavaai.face.entity.FaceResult; - -import java.awt.image.BufferedImage; -import java.io.IOException; -import java.io.InputStream; - -/** - * 人脸识别算法 - * @author dwj - */ -public interface FaceAlgorithm { - - /** - * 加载模型 - * @param config - * @throws Exception - */ - void loadModel(ModelConfig config) throws Exception; // 加载模型 - - /** - * 加载人脸特征提取模型 - * @param config - * @throws Exception - */ - void loadFaceFeatureModel(ModelConfig config) throws Exception; // 加载模型 - - /** - * 人脸检测 - * @param imagePath 图片路径 - * @return - * @throws Exception - */ - FaceDetectedResult detect(String imagePath) throws Exception; - - /** - * 人脸检测 - * @param imageInputStream 图片输入流 - * @return - * @throws Exception - */ - FaceDetectedResult detect(InputStream imageInputStream) throws Exception; - - /** - * 特征提取 - * @param imagePath 图片路径 - * @return - * @throws Exception - */ - float[] featureExtraction(String imagePath) throws Exception; - - /** - * 特征提取 - * @param inputStream 输入流 - * @return - * @throws Exception - */ - float[] featureExtraction(InputStream inputStream) throws Exception; - - /** - * 计算相似度 - * @param feature1 图1特征 - * @param feature2 图2特征 - * @return - * @throws Exception - */ - float calculSimilar(float[] feature1, float[] feature2) throws Exception; - - /** - * 特征比较 - * @param imagePath1 图1路径 - * @param imagePath2 图2路径 - * @return - * @throws Exception - */ - float featureComparison(String imagePath1, String imagePath2) throws Exception; - - /** - * 特征比较 - * @param inputStream1 图1输入流 - * @param inputStream2 图2输入流 - * @return - * @throws Exception - */ - float featureComparison(InputStream inputStream1, InputStream inputStream2) throws Exception; - - /** - * 注册人脸 - * @param key - * @param imagePath - * @return - */ - boolean register(String key, String imagePath) throws Exception; - - /** - * 注册人脸 - * @param key - * @param inputStream - * @return - */ - boolean register(String key, InputStream inputStream) throws Exception; - - /** - * 查询人脸 - * @param imagePath - * @return - */ - FaceResult search(String imagePath) throws Exception; - - /** - * 查询人脸 - * @param inputStream - * @return - */ - FaceResult search(InputStream inputStream) throws Exception; - - /** - * 删除已标记人脸 - * @param keys - * @return - */ - long removeRegister(String... keys) throws Exception; - - /** - * 清空人脸库数据 - */ - long clearFace() throws Exception; - -} diff --git a/smartjavaai-face/src/main/java/cn/smartjavaai/face/FaceAlgorithmFactory.java b/smartjavaai-face/src/main/java/cn/smartjavaai/face/FaceAlgorithmFactory.java deleted file mode 100644 index 437e316..0000000 --- a/smartjavaai-face/src/main/java/cn/smartjavaai/face/FaceAlgorithmFactory.java +++ /dev/null @@ -1,122 +0,0 @@ -package cn.smartjavaai.face; - -import cn.smartjavaai.face.algo.FeatureExtractionAlgo; -import cn.smartjavaai.face.algo.RetinaFace; -import cn.smartjavaai.face.algo.SeetaFace6Algo; -import cn.smartjavaai.face.algo.UltraLightFastGenericFace; - -import java.util.Map; -import java.util.concurrent.ConcurrentHashMap; - -/** - * 人脸算法工厂 - * @author dwj - */ -public class FaceAlgorithmFactory { - - /** - * 算法注册表 - */ - private static final Map> registry = - new ConcurrentHashMap<>(); - - /** - * 注册算法 - * @param name - * @param clazz - */ - public static void registerAlgorithm(String name, Class clazz) { - registry.put(name.toLowerCase(), clazz); - } - - /** - * 使用ModelConfig创建算法 - * @param config - * @return - * @throws Exception - */ - public static FaceAlgorithm createFaceAlgorithm(ModelConfig config) throws Exception { - Class clazz = registry.get(config.getAlgorithmName().toLowerCase()); - if(clazz == null){ - System.out.println("No such algorithm: " + config.getAlgorithmName().toLowerCase()); - throw new IllegalArgumentException("Unsupported algorithm"); - } - FaceAlgorithm algorithm = (FaceAlgorithm) clazz.newInstance(); - algorithm.loadModel(config); - return algorithm; - } - - /** - * 创建默认算法 - * @return - * @throws Exception - */ - public static FaceAlgorithm createFaceAlgorithm() throws Exception { - // 初始化配置 - ModelConfig config = new ModelConfig(); - config.setAlgorithmName("retinaface"); - config.setConfidenceThreshold(FaceConfig.DEFAULT_CONFIDENCE_THRESHOLD); - config.setMaxFaceCount(FaceConfig.MAX_FACE_LIMIT); - config.setNmsThresh(FaceConfig.NMS_THRESHOLD); - return createFaceAlgorithm(config); - } - - /** - * 创建轻量级算法 - * @return - * @throws Exception - */ - public static FaceAlgorithm createLightFaceAlgorithm() throws Exception { - // 初始化配置 - ModelConfig config = new ModelConfig(); - config.setAlgorithmName("ultralightfastgenericface"); - config.setConfidenceThreshold(FaceConfig.DEFAULT_CONFIDENCE_THRESHOLD); - config.setMaxFaceCount(FaceConfig.MAX_FACE_LIMIT); - config.setNmsThresh(FaceConfig.NMS_THRESHOLD); - Class clazz = registry.get(config.getAlgorithmName().toLowerCase()); - if(clazz == null){ - throw new IllegalArgumentException("Unsupported algorithm"); - } - FaceAlgorithm algorithm = (FaceAlgorithm) clazz.newInstance(); - algorithm.loadModel(config); - return algorithm; - } - - /** - * 使用ModelConfig创建人脸特征提取算法 - * @param config - * @return - * @throws Exception - */ - public static FaceAlgorithm createFaceFeatureAlgorithm(ModelConfig config) throws Exception { - Class clazz = registry.get(config.getAlgorithmName().toLowerCase()); - if(clazz == null){ - throw new IllegalArgumentException("Unsupported algorithm"); - } - FaceAlgorithm algorithm = (FaceAlgorithm) clazz.newInstance(); - algorithm.loadFaceFeatureModel(config); - return algorithm; - } - - /** - * 创建人脸特征提取算法 - * @return - * @throws Exception - */ - public static FaceAlgorithm createFaceFeatureAlgorithm() throws Exception { - // 初始化配置 - ModelConfig config = new ModelConfig(); - config.setAlgorithmName("featureExtraction"); - return createFaceFeatureAlgorithm(config); - } - - // 初始化默认算法 - static { - registerAlgorithm("retinaface", RetinaFace.class); - registerAlgorithm("ultralightfastgenericface", UltraLightFastGenericFace.class); - //人脸特征提取 - registerAlgorithm("featureExtraction", FeatureExtractionAlgo.class); - registerAlgorithm("seetaface6", SeetaFace6Algo.class); - } - -} diff --git a/smartjavaai-face/src/main/java/cn/smartjavaai/face/FaceDetectedResult.java b/smartjavaai-face/src/main/java/cn/smartjavaai/face/FaceDetectedResult.java deleted file mode 100644 index e04d0c3..0000000 --- a/smartjavaai-face/src/main/java/cn/smartjavaai/face/FaceDetectedResult.java +++ /dev/null @@ -1,40 +0,0 @@ -package cn.smartjavaai.face; - - -import cn.smartjavaai.common.entity.Rectangle; - -import java.util.List; - -/** - * 人脸检测结果 - * @author dwj - */ -public class FaceDetectedResult { - - - /** - * 置信度 - */ - private List probabilities; - - /** - * 人脸框 - */ - private List rectangles; - - public List getProbabilities() { - return probabilities; - } - - public void setProbabilities(List probabilities) { - this.probabilities = probabilities; - } - - public List getRectangles() { - return rectangles; - } - - public void setRectangles(List rectangles) { - this.rectangles = rectangles; - } -} diff --git a/smartjavaai-face/src/main/java/cn/smartjavaai/face/FaceModel.java b/smartjavaai-face/src/main/java/cn/smartjavaai/face/FaceModel.java new file mode 100644 index 0000000..bab6128 --- /dev/null +++ b/smartjavaai-face/src/main/java/cn/smartjavaai/face/FaceModel.java @@ -0,0 +1,206 @@ +package cn.smartjavaai.face; + +import cn.smartjavaai.common.entity.DetectionResponse; +import cn.smartjavaai.face.entity.FaceResult; + +import java.awt.image.BufferedImage; +import java.io.InputStream; + +/** + * 人脸识别算法 + * @author dwj + */ +public interface FaceModel { + + /** + * 加载模型 + * @param config + */ + void loadModel(FaceModelConfig config); // 加载模型 + + + /** + * 人脸检测 + * @param imagePath 图片路径 + * @return + */ + DetectionResponse detect(String imagePath); + + /** + * 人脸检测 + * @param imageInputStream 图片输入流 + * @return + */ + DetectionResponse detect(InputStream imageInputStream); + + /** + * 人脸检测 + * @param image BufferedImage + * @return + */ + DetectionResponse detect(BufferedImage image); + + /** + * 人脸检测 + * @param imageData + * @return + */ + DetectionResponse detect(byte[] imageData); + + /** + * 检测并绘制人脸 + * @param imagePath 图片输入路径（包含文件名称） + * @param outputPath 图片输出路径（包含文件名称） + */ + void detectAndDraw(String imagePath, String outputPath); + + /** + * 检测并绘制人脸 + * @param sourceImage + * @return + */ + BufferedImage detectAndDraw(BufferedImage sourceImage); + + /** + * 特征提取 + * @param imagePath 图片路径 + * @return + */ + float[] featureExtraction(String imagePath); + + /** + * 特征提取 + * @param inputStream 输入流 + * @return + */ + float[] featureExtraction(InputStream inputStream); + + /** + * 特征提取 + * @param sourceImage BufferedImage图片数据 + * @return + */ + float[] featureExtraction(BufferedImage sourceImage); + + /** + * 特征提取 + * @param imageData 图片字节流 + * @return + */ + float[] featureExtraction(byte[] imageData); + + /** + * 计算相似度 + * @param feature1 图1特征 + * @param feature2 图2特征 + * @return + */ + float calculSimilar(float[] feature1, float[] feature2); + + /** + * 特征比较 + * @param imagePath1 图1路径 + * @param imagePath2 图2路径 + * @return + */ + float featureComparison(String imagePath1, String imagePath2); + + /** + * 特征比较 + * @param sourceImage1 图1BufferedImage + * @param sourceImag2 图2BufferedImage + * @return + */ + float featureComparison(BufferedImage sourceImage1, BufferedImage sourceImag2); + + /** + * 特征比较 + * @param inputStream1 图1输入流 + * @param inputStream2 图2输入流 + * @return + */ + float featureComparison(InputStream inputStream1, InputStream inputStream2); + + + /** + * 特征比较 + * @param imageData1 + * @param imageData2 + * @return + */ + float featureComparison(byte[] imageData1, byte[] imageData2); + + /** + * 注册人脸 + * @param key + * @param imagePath + * @return + */ + boolean register(String key, String imagePath); + + /** + * 注册人脸 + * @param key + * @param inputStream + * @return + */ + boolean register(String key, InputStream inputStream); + + /** + * 注册人脸 + * @param key + * @param sourceImage + * @return + */ + boolean register(String key, BufferedImage sourceImage); + + + /** + * 注册人脸 + * @param key + * @param imageData + * @return + */ + boolean register(String key, byte[] imageData); + + /** + * 查询人脸 + * @param imagePath + * @return + */ + FaceResult search(String imagePath); + + /** + * 查询人脸 + * @param inputStream + * @return + */ + FaceResult search(InputStream inputStream); + + /** + * 查询人脸 + * @param sourceImage + * @return + */ + FaceResult search(BufferedImage sourceImage); + + /** + * 查询人脸 + * @param imageData + * @return + */ + FaceResult search(byte[] imageData); + + /** + * 删除已标记人脸 + * @param keys + * @return + */ + long removeRegister(String... keys); + + /** + * 清空人脸库数据 + */ + long clearFace(); + +} diff --git a/smartjavaai-face/src/main/java/cn/smartjavaai/face/FaceModelConfig.java b/smartjavaai-face/src/main/java/cn/smartjavaai/face/FaceModelConfig.java new file mode 100644 index 0000000..a527c95 --- /dev/null +++ b/smartjavaai-face/src/main/java/cn/smartjavaai/face/FaceModelConfig.java @@ -0,0 +1,45 @@ +package cn.smartjavaai.face; + +import cn.smartjavaai.common.enums.DeviceEnum; +import lombok.Data; + +/** + * 模型配置 + * @author dwj + */ +@Data +public class FaceModelConfig { + + /** + * 人脸算法名称 + */ + private FaceModelEnum modelEnum; + + /** + * 置信度阈值 + */ + private double confidenceThreshold = FaceConfig.DEFAULT_CONFIDENCE_THRESHOLD; + + /** + * 非极大抑制阈值 作用：消除重叠检测框，保留最优结果 + */ + private double nmsThresh = FaceConfig.NMS_THRESHOLD; + + /** + * 模型路径 + */ + private String modelPath; + + /** + * 人脸库路径 + */ + private String faceDbPath; + + /** + * 设备类型 + */ + private DeviceEnum device; + + + +} diff --git a/smartjavaai-face/src/main/java/cn/smartjavaai/face/FaceModelEnum.java b/smartjavaai-face/src/main/java/cn/smartjavaai/face/FaceModelEnum.java new file mode 100644 index 0000000..3dc23af --- /dev/null +++ b/smartjavaai-face/src/main/java/cn/smartjavaai/face/FaceModelEnum.java @@ -0,0 +1,44 @@ +package cn.smartjavaai.face; + +import cn.smartjavaai.face.model.FeatureExtractionModel; +import cn.smartjavaai.face.model.RetinaFaceModel; +import cn.smartjavaai.face.model.SeetaFace6Model; +import cn.smartjavaai.face.model.UltraLightFastGenericFaceModel; + +/** + * 人脸模型枚举 + * @author dwj + * @date 2025/4/10 + */ +public enum FaceModelEnum { + + RETINA_FACE("RetinaFaceModel"), + ULTRA_LIGHT_FAST_GENERIC_FACE("UltraLightFastGenericFaceModel"), + FACENET_FEATURE_EXTRACTION("FeatureExtractionModel"), + SEETA_FACE6_MODEL("SeetaFace6Model"); + + private final String modelClassName; + + FaceModelEnum(String modelClassName) { + this.modelClassName = modelClassName; + } + + public String getModelClassName() { + return modelClassName; + } + + /** + * 根据名称获取枚举 (忽略大小写和下划线变体) + */ + public static FaceModelEnum fromName(String name) { + String formatted = name.trim().toUpperCase().replaceAll("[-_]", ""); + for (FaceModelEnum model : values()) { + if (model.name().replaceAll("_", "").equals(formatted)) { + return model; + } + } + throw new IllegalArgumentException("未知模型名称: " + name); + } + + +} diff --git a/smartjavaai-face/src/main/java/cn/smartjavaai/face/FaceModelFactory.java b/smartjavaai-face/src/main/java/cn/smartjavaai/face/FaceModelFactory.java new file mode 100644 index 0000000..1474dbe --- /dev/null +++ b/smartjavaai-face/src/main/java/cn/smartjavaai/face/FaceModelFactory.java @@ -0,0 +1,127 @@ +package cn.smartjavaai.face; + +import cn.smartjavaai.face.exception.FaceException; +import cn.smartjavaai.face.model.FeatureExtractionModel; +import cn.smartjavaai.face.model.RetinaFaceModel; +import cn.smartjavaai.face.model.SeetaFace6Model; +import cn.smartjavaai.face.model.UltraLightFastGenericFaceModel; +import lombok.extern.slf4j.Slf4j; + +import java.util.Map; +import java.util.Objects; +import java.util.concurrent.ConcurrentHashMap; + +/** + * 人脸算法工厂 + * @author dwj + */ +@Slf4j +public class FaceModelFactory { + + // 使用 volatile 和双重检查锁定来确保线程安全的单例模式 + private static volatile FaceModelFactory instance; + + private static final ConcurrentHashMap modelMap = new ConcurrentHashMap<>(); + + /** + * 算法注册表 + */ + private static final Map> registry = + new ConcurrentHashMap<>(); + + + public static FaceModelFactory getInstance() { + if (instance == null) { + synchronized (FaceModelFactory.class) { + if (instance == null) { + instance = new FaceModelFactory(); + } + } + } + return instance; + } + + + + /** + * 注册算法 + * @param name + * @param clazz + */ + private static void registerAlgorithm(String name, Class clazz) { + registry.put(name.toLowerCase(), clazz); + } + + + /** + * 获取模型（通过配置） + * @param config + * @return + */ + public FaceModel getModel(FaceModelConfig config) { + if(Objects.isNull(config) || Objects.isNull(config.getModelEnum())){ + throw new FaceException("未配置人脸模型"); + } + return modelMap.computeIfAbsent(config.getModelEnum().name(), k -> { + return createFaceModel(config); + }); + } + + /** + * 获取默认模型 + * @return + */ + public FaceModel getModel() { + // 初始化默认配置 + FaceModelConfig config = new FaceModelConfig(); + config.setModelEnum(FaceModelEnum.RETINA_FACE); + config.setConfidenceThreshold(FaceConfig.DEFAULT_CONFIDENCE_THRESHOLD); + config.setNmsThresh(FaceConfig.NMS_THRESHOLD); + return getModel(config); + } + + /** + * 使用ModelConfig创建算法 + * @param config + * @return + */ + private FaceModel createFaceModel(FaceModelConfig config) { + Class clazz = registry.get(config.getModelEnum().getModelClassName().toLowerCase()); + if(clazz == null){ + throw new FaceException("Unsupported algorithm"); + } + FaceModel algorithm = null; + try { + algorithm = (FaceModel) clazz.newInstance(); + } catch (InstantiationException | IllegalAccessException e) { + throw new FaceException(e); + } + algorithm.loadModel(config); + return algorithm; + } + + + /** + * 获取轻量级人脸模型 + * @return + */ + public FaceModel getLightFaceModel() { + // 初始化默认配置 + FaceModelConfig config = new FaceModelConfig(); + config.setModelEnum(FaceModelEnum.ULTRA_LIGHT_FAST_GENERIC_FACE); + config.setConfidenceThreshold(FaceConfig.DEFAULT_CONFIDENCE_THRESHOLD); + config.setNmsThresh(FaceConfig.NMS_THRESHOLD); + return getModel(config); + } + + + // 初始化默认算法 + static { + registerAlgorithm("retinafacemodel", RetinaFaceModel.class); + registerAlgorithm("ultralightfastgenericfacemodel", UltraLightFastGenericFaceModel.class); + //人脸特征提取 + registerAlgorithm("featureextractionmodel", FeatureExtractionModel.class); + registerAlgorithm("seetaface6model", SeetaFace6Model.class); + } + +} diff --git a/smartjavaai-face/src/main/java/cn/smartjavaai/face/ModelConfig.java b/smartjavaai-face/src/main/java/cn/smartjavaai/face/ModelConfig.java deleted file mode 100644 index d8685e9..0000000 --- a/smartjavaai-face/src/main/java/cn/smartjavaai/face/ModelConfig.java +++ /dev/null @@ -1,88 +0,0 @@ -package cn.smartjavaai.face; - -/** - * 模型配置 - * @author dwj - */ - -public class ModelConfig { - - /** - * 人脸算法名称 - */ - private String algorithmName; - - /** - * 置信度阈值 - */ - private double confidenceThreshold; - - /** - * 非极大抑制阈值 作用：消除重叠检测框，保留最优结果 - */ - private double nmsThresh; - - /** - * 最大检测人脸数量 - */ - private int maxFaceCount; - - /** - * 模型路径 - */ - private String modelPath; - - /** - * 人脸库路径 - */ - private String faceDbPath; - - - public String getAlgorithmName() { - return algorithmName; - } - - public void setAlgorithmName(String algorithmName) { - this.algorithmName = algorithmName; - } - - public double getConfidenceThreshold() { - return confidenceThreshold; - } - - public void setConfidenceThreshold(double confidenceThreshold) { - this.confidenceThreshold = confidenceThreshold; - } - - public double getNmsThresh() { - return nmsThresh; - } - - public void setNmsThresh(double nmsThresh) { - this.nmsThresh = nmsThresh; - } - - public int getMaxFaceCount() { - return maxFaceCount; - } - - public void setMaxFaceCount(int maxFaceCount) { - this.maxFaceCount = maxFaceCount; - } - - public String getModelPath() { - return modelPath; - } - - public void setModelPath(String modelPath) { - this.modelPath = modelPath; - } - - public String getFaceDbPath() { - return faceDbPath; - } - - public void setFaceDbPath(String faceDbPath) { - this.faceDbPath = faceDbPath; - } -} diff --git a/smartjavaai-face/src/main/java/cn/smartjavaai/face/algo/FeatureExtractionAlgo.java b/smartjavaai-face/src/main/java/cn/smartjavaai/face/algo/FeatureExtractionAlgo.java deleted file mode 100644 index 734474c..0000000 --- a/smartjavaai-face/src/main/java/cn/smartjavaai/face/algo/FeatureExtractionAlgo.java +++ /dev/null @@ -1,162 +0,0 @@ -package cn.smartjavaai.face.algo; - -import ai.djl.Device; -import ai.djl.inference.Predictor; -import ai.djl.modality.cv.Image; -import ai.djl.modality.cv.ImageFactory; -import ai.djl.modality.cv.output.DetectedObjects; -import ai.djl.modality.cv.translator.ImageFeatureExtractorFactory; -import ai.djl.repository.zoo.Criteria; -import ai.djl.repository.zoo.ZooModel; -import ai.djl.training.util.ProgressBar; -import cn.smartjavaai.common.entity.Point; -import cn.smartjavaai.common.entity.Rectangle; -import cn.smartjavaai.face.AbstractFaceAlgorithm; -import cn.smartjavaai.face.FaceDetectedResult; -import cn.smartjavaai.face.FaceDetectionTranslator; -import cn.smartjavaai.face.ModelConfig; -import cn.smartjavaai.face.translator.FaceFeatureTranslator; -import org.apache.commons.lang3.StringUtils; - -import java.io.InputStream; -import java.nio.file.Path; -import java.nio.file.Paths; -import java.util.ArrayList; -import java.util.Arrays; -import java.util.List; -import java.util.stream.Collectors; - -/** - * @author dwj - */ -public class FeatureExtractionAlgo extends AbstractFaceAlgorithm { - - - private Criteria faceFeatureCriteria; - - private Predictor predictor; - - private ZooModel model; - - public static final List mean = - Arrays.asList( - 127.5f / 255.0f, - 127.5f / 255.0f, - 127.5f / 255.0f, - 128.0f / 255.0f, - 128.0f / 255.0f, - 128.0f / 255.0f); - - - /** - * 加载人脸特征提取模型 - * @param config - * @throws Exception - */ - @Override - public void loadFaceFeatureModel(ModelConfig config) throws Exception { - String normalize = mean.stream().map(Object::toString).collect(Collectors.joining(",")); - faceFeatureCriteria = - Criteria.builder() - .setTypes(Image.class, float[].class) - .optModelName("face_feature") // specify model file prefix - .optModelUrls(StringUtils.isNotBlank(config.getModelPath()) ? null : - "https://resources.djl.ai/test-models/pytorch/face_feature.zip") - .optModelPath(StringUtils.isNotBlank(config.getModelPath()) ? Paths.get(config.getModelPath()) : null) - .optTranslator(new FaceFeatureTranslator()) - .optArgument("normalize", normalize) - .optEngine("PyTorch") // Use PyTorch engine - .optProgress(new ProgressBar()) - .build(); - - model = faceFeatureCriteria.loadModel(); - predictor = model.newPredictor(); - } - - - /** - * 特征提取 - * @param imagePath 图片路径 - * @return - * @throws Exception - */ - @Override - public float[] featureExtraction(String imagePath) throws Exception { - Path imageFile = Paths.get(imagePath); - Image img = ImageFactory.getInstance().fromFile(imageFile); - img.getWrappedImage(); - return predictor.predict(img); - } - - /** - * 特征提取 - * @param inputStream 输入流 - * @return - * @throws Exception - */ - @Override - public float[] featureExtraction(InputStream inputStream) throws Exception { - Image img = ImageFactory.getInstance().fromInputStream(inputStream); - img.getWrappedImage(); - return predictor.predict(img); - } - - /** - * 计算相似度 - * @param feature1 图1特征 - * @param feature2 图2特征 - * @return - * @throws Exception - */ - @Override - public float calculSimilar(float[] feature1, float[] feature2) throws Exception { - float ret = 0.0f; - float mod1 = 0.0f; - float mod2 = 0.0f; - int length = feature1.length; - for (int i = 0; i < length; ++i) { - ret += feature1[i] * feature2[i]; - mod1 += feature1[i] * feature1[i]; - mod2 += feature2[i] * feature2[i]; - } - return (float) ((ret / Math.sqrt(mod1) / Math.sqrt(mod2) + 1) / 2.0f); - } - - /** - * 特征比较 - * @param imagePath1 图1路径 - * @param imagePath2 图2路径 - * @return - * @throws Exception - */ - @Override - public float featureComparison(String imagePath1, String imagePath2) throws Exception { - float[] feature1 = featureExtraction(imagePath1); - float[] feature2 = featureExtraction(imagePath2); - return calculSimilar(feature1, feature2); - } - - /** - * 特征比较 - * @param inputStream1 图1输入流 - * @param inputStream2 图2输入流 - * @return - * @throws Exception - */ - @Override - public float featureComparison(InputStream inputStream1, InputStream inputStream2) throws Exception { - float[] feature1 = featureExtraction(inputStream1); - float[] feature2 = featureExtraction(inputStream2); - return calculSimilar(feature1, feature2); - } - - /*@Override - public float[] recognize(FaceRegion region) { - return new float[0]; - }*/ - - /*@Override - public void loadModel(ModelConfig config) throws Exception { - - }*/ -} diff --git a/smartjavaai-face/src/main/java/cn/smartjavaai/face/algo/RetinaFace.java b/smartjavaai-face/src/main/java/cn/smartjavaai/face/algo/RetinaFace.java deleted file mode 100644 index 98b2512..0000000 --- a/smartjavaai-face/src/main/java/cn/smartjavaai/face/algo/RetinaFace.java +++ /dev/null @@ -1,144 +0,0 @@ -package cn.smartjavaai.face.algo; - -import ai.djl.MalformedModelException; -import ai.djl.inference.Predictor; -import ai.djl.modality.cv.Image; -import ai.djl.modality.cv.ImageFactory; -import ai.djl.modality.cv.output.DetectedObjects; -import ai.djl.modality.cv.translator.ImageFeatureExtractorFactory; -import ai.djl.repository.zoo.Criteria; -import ai.djl.repository.zoo.ModelNotFoundException; -import ai.djl.repository.zoo.ZooModel; -import ai.djl.training.util.ProgressBar; -import ai.djl.translate.TranslateException; -import cn.smartjavaai.common.entity.Point; -import cn.smartjavaai.common.entity.Rectangle; -import cn.smartjavaai.face.*; -import org.apache.commons.lang3.StringUtils; - - -import java.io.IOException; -import java.io.InputStream; -import java.lang.reflect.InvocationTargetException; -import java.nio.file.Path; -import java.nio.file.Paths; -import java.util.ArrayList; -import java.util.Arrays; -import java.util.List; -import java.util.stream.Collectors; -import java.util.stream.StreamSupport; - -/** - * RetinaFace实现 - * @author dwj - */ -public class RetinaFace extends AbstractFaceAlgorithm { - - private Criteria criteria; - - private Criteria faceFeatureCriteria; - - private Predictor predictor; - - private ZooModel model; - - /** - * 特征图层的基础缩放比例 - */ - public static final int[][] scales = {{16, 32}, {64, 128}, {256, 512}}; - /** - * 特征图相对于原图的采样步长 - */ - public static final int[] steps = {8, 16, 32}; - /** - * 缩放系数 - */ - public static final double[] variance = {0.1f, 0.2f}; - - - /** - * 加载模型 - * @param config - */ - @Override - public void loadModel(ModelConfig config) throws ModelNotFoundException, MalformedModelException, IOException { - FaceDetectionTranslator translator = - new FaceDetectionTranslator(config.getConfidenceThreshold(), config.getNmsThresh(), variance, config.getMaxFaceCount(), scales, steps); - criteria = - Criteria.builder() - .setTypes(Image.class, DetectedObjects.class) - .optModelUrls(StringUtils.isNotBlank(config.getModelPath()) ? null : "https://resources.djl.ai/test-models/pytorch/retinaface.zip") - // Load model from local file, e.g: - .optModelPath(StringUtils.isNotBlank(config.getModelPath()) ? Paths.get(config.getModelPath()) : null) - .optModelName(StringUtils.isNotBlank(config.getAlgorithmName()) ? config.getAlgorithmName() : "retinaface") // specify model file prefix - .optTranslator(translator) - .optProgress(new ProgressBar()) - .optEngine("PyTorch") // Use PyTorch engine - .build(); - model = criteria.loadModel(); - predictor = model.newPredictor(); - } - - - - /** - * 检测人脸 - * @param imagePath 图片路径 - * @return - * @throws Exception - */ - @Override - public FaceDetectedResult detect(String imagePath) throws Exception{ - Path facePath = Paths.get(imagePath); - Image img = ImageFactory.getInstance().fromFile(facePath); - DetectedObjects detection = predictor.predict(img); - return convertToFaceDetectedResult(detection,img); - } - - /** - * 检测人脸 - * @param imageInputStream 图片流 - * @return - * @throws Exception - */ - @Override - public FaceDetectedResult detect(InputStream imageInputStream) throws Exception { - Image img = ImageFactory.getInstance().fromInputStream(imageInputStream); - DetectedObjects detection = predictor.predict(img); - return convertToFaceDetectedResult(detection,img); - } - - /** - * 转换为FaceDetectedResult - * @param detection - * @param img - * @return - */ - private FaceDetectedResult convertToFaceDetectedResult(DetectedObjects detection, Image img){ - FaceDetectedResult faceDetectedResult = new FaceDetectedResult(); - List probabilities = new ArrayList<>(detection.getProbabilities()); - List detectedObjectList = detection.items(); - List RectangleList = detectedObjectList.parallelStream() - .map(obj -> { - Rectangle rectangle = new Rectangle(); - List pointList = new ArrayList<>(); - ai.djl.modality.cv.output.Rectangle rectangleDjl = obj.getBoundingBox().getBounds(); - int x = (int)(rectangleDjl.getX() * (double)img.getWidth()); - int y = (int)(rectangleDjl.getY() * (double)img.getHeight()); - int width = (int)(rectangleDjl.getWidth() * (double)img.getWidth()); - int height = (int)(rectangleDjl.getHeight() * (double)img.getHeight()); - pointList.add(new Point(x,y)); - pointList.add(new Point(x + width,y)); - pointList.add(new Point(x,y + height)); - pointList.add(new Point(x + width,y + height)); - rectangle.setPointList(pointList); - rectangle.setHeight(height); - rectangle.setWidth(width); - return rectangle; - }) - .collect(Collectors.toList()); - faceDetectedResult.setProbabilities(probabilities); - faceDetectedResult.setRectangles(RectangleList); - return faceDetectedResult; - } -} diff --git a/smartjavaai-face/src/main/java/cn/smartjavaai/face/algo/SeetaFace6Algo.java b/smartjavaai-face/src/main/java/cn/smartjavaai/face/algo/SeetaFace6Algo.java deleted file mode 100644 index e06ba05..0000000 --- a/smartjavaai-face/src/main/java/cn/smartjavaai/face/algo/SeetaFace6Algo.java +++ /dev/null @@ -1,319 +0,0 @@ -package cn.smartjavaai.face.algo; - -import cn.smartjavaai.common.entity.Point; -import cn.smartjavaai.common.entity.Rectangle; -import cn.smartjavaai.common.utils.ImageUtils; -import cn.smartjavaai.face.AbstractFaceAlgorithm; -import cn.smartjavaai.face.FaceDetectedResult; -import cn.smartjavaai.face.ModelConfig; -import cn.smartjavaai.face.dao.FaceDao; -import cn.smartjavaai.face.entity.FaceData; -import cn.smartjavaai.face.entity.FaceResult; -import com.seetaface.NativeLoader; -import com.seetaface.SeetaFace6JNI; -import com.seetaface.model.RecognizeResult; -import com.seetaface.model.SeetaImageData; -import com.seetaface.model.SeetaRect; -import lombok.extern.slf4j.Slf4j; -import org.apache.commons.lang3.StringUtils; - -import javax.imageio.ImageIO; -import java.awt.image.BufferedImage; -import java.io.*; -import java.nio.file.Path; -import java.nio.file.Paths; -import java.sql.SQLException; -import java.util.ArrayList; -import java.util.List; -import java.util.Objects; - -/** - * SeetaFace6 人脸算法 - * @author dwj - */ -@Slf4j -public class SeetaFace6Algo extends AbstractFaceAlgorithm { - - - private ModelConfig config; - - - @Override - public void loadModel(ModelConfig config) throws Exception { - this.config = config; - if (NativeLoader.seetaFace6SDK == null) { - synchronized (SeetaFace6JNI.class) { - if(StringUtils.isBlank(config.getModelPath())){ - throw new Exception("modelPath is null"); - } - //加载依赖库 - NativeLoader.loadNativeLibraries(config.getModelPath()); - log.info("Loading seetaFace6 library successfully."); - NativeLoader.seetaFace6SDK = new SeetaFace6JNI(); - //加载模型 - boolean isSuccess = NativeLoader.seetaFace6SDK.initModel(config.getModelPath()); - if(!isSuccess){ - throw new Exception("seetaFace6模型初始化失败," + config.getModelPath()); - } - log.info("Load seetaFace6 model success!"); - new Thread(new Runnable() { - public void run() { - try { - log.info("start load faceDb..."); - loadFaceDb(); - log.info("Load faceDb success!"); - } catch (Exception e) { - e.printStackTrace(); - } - } - }).start(); - } - } - } - - @Override - public FaceDetectedResult detect(String imagePath) throws Exception { - // 将图片路径转换为 BufferedImage - BufferedImage image = ImageIO.read(new File(Paths.get(imagePath).toAbsolutePath().toString())); - SeetaImageData imageData = new SeetaImageData(image.getWidth(), image.getHeight(), 3); - imageData.data = ImageUtils.getMatrixBGR(image); - SeetaRect[] seetaResult = NativeLoader.seetaFace6SDK.detect(imageData); - return convertToFaceDetectedResult(seetaResult); - } - - @Override - public FaceDetectedResult detect(InputStream imageInputStream) throws Exception { - BufferedImage image = ImageIO.read(imageInputStream); - SeetaImageData imageData = new SeetaImageData(image.getWidth(), image.getHeight(), 3); - imageData.data = ImageUtils.getMatrixBGR(image); - SeetaRect[] seetaResult = NativeLoader.seetaFace6SDK.detect(imageData); - return convertToFaceDetectedResult(seetaResult); - } - - @Override - public float[] featureExtraction(String imagePath) throws Exception { - BufferedImage image = ImageIO.read(new File(Paths.get(imagePath).toAbsolutePath().toString())); - SeetaImageData imageData = new SeetaImageData(image.getWidth(), image.getHeight(), 3); - imageData.data = ImageUtils.getMatrixBGR(image); - return NativeLoader.seetaFace6SDK.extractMaxFace(imageData); - } - - @Override - public float[] featureExtraction(InputStream inputStream) throws Exception { - BufferedImage image = ImageIO.read(inputStream); - SeetaImageData imageData = new SeetaImageData(image.getWidth(), image.getHeight(), 3); - imageData.data = ImageUtils.getMatrixBGR(image); - return NativeLoader.seetaFace6SDK.extractMaxFace(imageData); - } - - @Override - public float calculSimilar(float[] feature1, float[] feature2) throws Exception { - return NativeLoader.seetaFace6SDK.calculateSimilarity(feature1, feature2); - } - - @Override - public float featureComparison(String imagePath1, String imagePath2) throws Exception { - return featureComparison(new FileInputStream(Paths.get(imagePath1).toAbsolutePath().toString()), - new FileInputStream(Paths.get(imagePath2).toAbsolutePath().toString())); - } - - @Override - public float featureComparison(InputStream inputStream1, InputStream inputStream2) throws Exception { - BufferedImage image1 = ImageIO.read(inputStream1); - BufferedImage image2 = ImageIO.read(inputStream2); - SeetaImageData imageData1 = new SeetaImageData(image1.getWidth(), image1.getHeight(), 3); - imageData1.data = ImageUtils.getMatrixBGR(image1); - - SeetaImageData imageData2 = new SeetaImageData(image2.getWidth(), image2.getHeight(), 3); - imageData2.data = ImageUtils.getMatrixBGR(image2); - //裁剪 - byte[][] cropImg1 = NativeLoader.seetaFace6SDK.crop(imageData1); - byte[][] cropImg2 = NativeLoader.seetaFace6SDK.crop(imageData2); - if(cropImg1 == null || cropImg1.length == 0){ - throw new Exception("未发现人脸"); - } - if(cropImg2 == null || cropImg2.length == 0){ - throw new Exception("未发现人脸"); - } - - BufferedImage cropImage1 = ImageUtils.bgrToBufferedImage(cropImg1[0], 256, 256); - BufferedImage cropImage2 = ImageUtils.bgrToBufferedImage(cropImg2[0], 256, 256); - - SeetaImageData cropImageData1 = new SeetaImageData(cropImage1.getWidth(), cropImage1.getHeight(), 3); - cropImageData1.data = ImageUtils.getMatrixBGR(cropImage1); - SeetaImageData cropImageData2 = new SeetaImageData(cropImage2.getWidth(), cropImage2.getHeight(), 3); - cropImageData2.data = ImageUtils.getMatrixBGR(cropImage2); - return NativeLoader.seetaFace6SDK.compare(cropImageData1, cropImageData2); - } - - - /** - * 转换为FaceDetectedResult - * @param seetaResult - * @return - */ - private FaceDetectedResult convertToFaceDetectedResult(SeetaRect[] seetaResult){ - FaceDetectedResult faceDetectedResult = new FaceDetectedResult(); - List RectangleList = new ArrayList(); - List probabilities = new ArrayList(); - if(seetaResult != null && seetaResult.length > 0){ - for(SeetaRect rect : seetaResult){ - Rectangle rectangle = new Rectangle(); - List pointList = new ArrayList<>(); - pointList.add(new Point(rect.x,rect.y)); - pointList.add(new Point(rect.x + rect.width,rect.y)); - pointList.add(new Point(rect.x,rect.y + rect.height)); - pointList.add(new Point(rect.x + rect.width,rect.y + rect.height)); - rectangle.setPointList(pointList); - rectangle.setHeight(rect.height); - rectangle.setWidth(rect.width); - RectangleList.add(rectangle); - probabilities.add(new Double(rect.score)); - } - } - faceDetectedResult.setProbabilities(probabilities); - faceDetectedResult.setRectangles(RectangleList); - return faceDetectedResult; - } - - @Override - public boolean register(String key, String imagePath) throws Exception { - return register(key, new FileInputStream(Paths.get(imagePath).toAbsolutePath().toString())); - } - - - @Override - public boolean register(String key, InputStream inputStream) throws Exception { - if(!checkFaceDb()){ - throw new Exception("未找到人脸库，无法使用此功能（请检查是否配置人脸库路径）"); - } - //裁剪人脸 - BufferedImage image = ImageIO.read(inputStream); - SeetaImageData imageData = new SeetaImageData(image.getWidth(), image.getHeight(), 3); - imageData.data = ImageUtils.getMatrixBGR(image); - byte[][] bytes = NativeLoader.seetaFace6SDK.crop(imageData); - if (bytes == null || bytes.length == 0) { - log.info("register face fail: key={}, error=no valid face", key); - return false; - } - long index = NativeLoader.seetaFace6SDK.registerCroppedFace(bytes[0]); - if (index < 0) { - log.info("register face fail: key={}, index={}", key, index); - return false; - } - //持久化到sqlite数据库 - FaceData face = new FaceData(); - face.setKey(key); - face.setIndex(index); - face.setImgData(bytes[0]); - new FaceDao(config.getFaceDbPath()).save(face); - return true; - } - - public boolean register(String key, FaceData faceData) throws Exception { - long index = NativeLoader.seetaFace6SDK.registerCroppedFace(faceData.getImgData()); - if (index < 0) { - log.info("register face fail: key={}, index={}", key, index); - return false; - } - int rows = new FaceDao(config.getFaceDbPath()).updateIndex(index, faceData); - return rows > 0; - } - - - - @Override - public FaceResult search(String imagePath) throws Exception { - return search(new FileInputStream(Paths.get(imagePath).toAbsolutePath().toString())); - } - - @Override - public FaceResult search(InputStream inputStream) throws Exception{ - if(!checkFaceDb()){ - throw new Exception("未找到人脸库，无法使用此功能（请检查是否配置人脸库路径）"); - } - BufferedImage image = ImageIO.read(inputStream); - SeetaImageData imageData = new SeetaImageData(image.getWidth(), image.getHeight(), 3); - imageData.data = ImageUtils.getMatrixBGR(image); - RecognizeResult recognizeResult = NativeLoader.seetaFace6SDK.query(imageData); - return searchFaceDb(recognizeResult); - - } - - @Override - public long removeRegister(String... keys) throws Exception { - if(!checkFaceDb()){ - throw new Exception("未找到人脸库，无法使用此功能（请检查是否配置人脸库路径）"); - } - List list = new FaceDao(config.getFaceDbPath()).findIndexList(keys); - if (list == null) { - return 0; - } - long[] array = list.stream().mapToLong(Long::longValue).toArray(); - long rows = NativeLoader.seetaFace6SDK.delete(array); - new FaceDao(config.getFaceDbPath()).deleteFace(keys); - return rows; - } - - @Override - public long clearFace() throws Exception{ - if(!checkFaceDb()){ - throw new Exception("未找到人脸库，无法使用此功能（请检查是否配置人脸库路径）"); - } - long rows = NativeLoader.seetaFace6SDK.delete(new long[]{-1}); - new FaceDao(config.getFaceDbPath()).deleteAll(); - return rows; - } - - /** - * 检查是否存在人脸库 - * @return - */ - private boolean checkFaceDb(){ - if(Objects.nonNull(config) && StringUtils.isNotBlank(config.getFaceDbPath())){ - File file = new File(config.getFaceDbPath()); - return file.exists() && file.isFile(); - } - return false; - } - - private FaceResult searchFaceDb(RecognizeResult recognizeResult) throws SQLException, ClassNotFoundException { - if(recognizeResult != null && recognizeResult.index >= 0){ - String key = new FaceDao(config.getFaceDbPath()).findKeyByIndex(recognizeResult.index); - return new FaceResult(key, recognizeResult.similar); - } - return null; - } - - - /** - * 加载人脸库 - * @throws SQLException - * @throws ClassNotFoundException - */ - private void loadFaceDb() throws SQLException, ClassNotFoundException { - if(!checkFaceDb()){ - log.info("未配置人脸库"); - return; - } - //分页查询人脸库 - int pageNo = 0, pageSize = 100; - while (true) { - List list = new FaceDao(config.getFaceDbPath()).findFace(pageNo, pageSize); - if (list == null) { - break; - } - list.forEach(face -> { - try { - register(face.getKey(), face); - } catch (Exception e) { - e.printStackTrace(); - } - }); - if (list.size() < pageSize) { - break; - } - pageNo++; - } - } -} diff --git a/smartjavaai-face/src/main/java/cn/smartjavaai/face/algo/UltraLightFastGenericFace.java b/smartjavaai-face/src/main/java/cn/smartjavaai/face/algo/UltraLightFastGenericFace.java deleted file mode 100644 index 1468654..0000000 --- a/smartjavaai-face/src/main/java/cn/smartjavaai/face/algo/UltraLightFastGenericFace.java +++ /dev/null @@ -1,135 +0,0 @@ -package cn.smartjavaai.face.algo; - -import ai.djl.MalformedModelException; -import ai.djl.inference.Predictor; -import ai.djl.modality.cv.Image; -import ai.djl.modality.cv.ImageFactory; -import ai.djl.modality.cv.output.DetectedObjects; -import ai.djl.modality.cv.translator.ImageFeatureExtractorFactory; -import ai.djl.repository.zoo.Criteria; -import ai.djl.repository.zoo.ModelNotFoundException; -import ai.djl.repository.zoo.ZooModel; -import ai.djl.training.util.ProgressBar; -import cn.smartjavaai.common.entity.Point; -import cn.smartjavaai.common.entity.Rectangle; -import cn.smartjavaai.face.*; - -import java.io.IOException; -import java.io.InputStream; -import java.nio.file.Path; -import java.nio.file.Paths; -import java.util.ArrayList; -import java.util.Arrays; -import java.util.List; -import java.util.stream.Collectors; - -/** - * @author dwj - */ -public class UltraLightFastGenericFace extends AbstractFaceAlgorithm { - - - private Criteria criteria; - - /** - * 特征图层的基础缩放比例 - */ - private static final int[][] scales = {{10, 16, 24}, {32, 48}, {64, 96}, {128, 192, 256}}; - /** - * 特征图相对于原图的采样步长 - */ - private static final int[] steps = {8, 16, 32, 64}; - /** - * 缩放系数 - */ - private static final double[] variance = {0.1f, 0.2f}; - - private Predictor predictor; - - private ZooModel model; - - - - - /** - * 加载模型 - * @param config - */ - @Override - public void loadModel(ModelConfig config) throws ModelNotFoundException, MalformedModelException, IOException { - FaceDetectionTranslator translator = - new FaceDetectionTranslator(config.getConfidenceThreshold(), config.getNmsThresh(), variance, config.getMaxFaceCount(), scales, steps); - criteria = - Criteria.builder() - .setTypes(Image.class, DetectedObjects.class) - .optModelUrls("https://resources.djl.ai/test-models/pytorch/ultranet.zip") - .optTranslator(translator) - .optProgress(new ProgressBar()) - .optEngine("PyTorch") // Use PyTorch engine - .build(); - model = criteria.loadModel(); - predictor = model.newPredictor(); - } - - /** - * 检测人脸 - * @param imagePath 图片路径 - * @return - * @throws Exception - */ - @Override - public FaceDetectedResult detect(String imagePath) throws Exception{ - Path facePath = Paths.get(imagePath); - Image img = ImageFactory.getInstance().fromFile(facePath); - DetectedObjects detection = predictor.predict(img); - return convertToFaceDetectedResult(detection,img); - } - - /** - * 检测人脸 - * @param imageInputStream 图片输入流 - * @return - * @throws Exception - */ - @Override - public FaceDetectedResult detect(InputStream imageInputStream) throws Exception { - Image img = ImageFactory.getInstance().fromInputStream(imageInputStream); - DetectedObjects detection = predictor.predict(img); - return convertToFaceDetectedResult(detection,img); - } - - /** - * 转换检测结果 - * @param detection - * @param img - * @return - */ - private FaceDetectedResult convertToFaceDetectedResult(DetectedObjects detection, Image img){ - FaceDetectedResult faceDetectedResult = new FaceDetectedResult(); - List probabilities = new ArrayList<>(detection.getProbabilities()); - List detectedObjectList = detection.items(); - List RectangleList = detectedObjectList.parallelStream() - .map(obj -> { - Rectangle rectangle = new Rectangle(); - List pointList = new ArrayList<>(); - ai.djl.modality.cv.output.Rectangle rectangleDjl = obj.getBoundingBox().getBounds(); - int x = (int)(rectangleDjl.getX() * (double)img.getWidth()); - int y = (int)(rectangleDjl.getY() * (double)img.getHeight()); - int width = (int)(rectangleDjl.getWidth() * (double)img.getWidth()); - int height = (int)(rectangleDjl.getHeight() * (double)img.getHeight()); - pointList.add(new Point(x,y)); - pointList.add(new Point(x + width,y)); - pointList.add(new Point(x,y + height)); - pointList.add(new Point(x + width,y + height)); - rectangle.setPointList(pointList); - rectangle.setHeight(height); - rectangle.setWidth(width); - return rectangle; - }) - .collect(Collectors.toList()); - faceDetectedResult.setProbabilities(probabilities); - faceDetectedResult.setRectangles(RectangleList); - return faceDetectedResult; - } - -} diff --git a/smartjavaai-face/src/main/java/cn/smartjavaai/face/exception/FaceException.java b/smartjavaai-face/src/main/java/cn/smartjavaai/face/exception/FaceException.java new file mode 100644 index 0000000..8fcedab --- /dev/null +++ b/smartjavaai-face/src/main/java/cn/smartjavaai/face/exception/FaceException.java @@ -0,0 +1,30 @@ +package cn.smartjavaai.face.exception; + +/** + * 人脸检测异常 + * @author dwj + * @date 2025/4/4 + */ +public class FaceException extends RuntimeException{ + + public FaceException() { + super(); + } + + public FaceException(String message, Throwable cause, boolean enableSuppression, boolean writableStackTrace) { + super(message, cause, enableSuppression, writableStackTrace); + } + + public FaceException(String message, Throwable cause) { + super(message, cause); + } + + public FaceException(String message) { + super(message); + } + + public FaceException(Throwable cause) { + super(cause); + } + +} diff --git a/smartjavaai-face/src/main/java/cn/smartjavaai/face/factory/SeetaFace6Factory.java b/smartjavaai-face/src/main/java/cn/smartjavaai/face/factory/SeetaFace6Factory.java new file mode 100644 index 0000000..5d56f85 --- /dev/null +++ b/smartjavaai-face/src/main/java/cn/smartjavaai/face/factory/SeetaFace6Factory.java @@ -0,0 +1,34 @@ +package cn.smartjavaai.face.factory; + +import ai.djl.inference.Predictor; +import ai.djl.repository.zoo.ZooModel; +import cn.smartjavaai.face.model.SeetaFace6Model; +import com.seetaface.SeetaFace6JNI; +import org.apache.commons.pool2.BasePooledObjectFactory; +import org.apache.commons.pool2.PooledObject; +import org.apache.commons.pool2.impl.DefaultPooledObject; + +/** + * Predictor 工厂类 + * @author dwj + * @date 2025/4/8 + */ +public class SeetaFace6Factory extends BasePooledObjectFactory { + + @Override + public SeetaFace6JNI create() { + return new SeetaFace6JNI(); + } + + @Override + public PooledObject wrap(SeetaFace6JNI obj) { + return new DefaultPooledObject<>(obj); + } + + @Override + public void destroyObject(PooledObject p) { + //p.getObject().dispose(); // 如果需要释放 native 资源 + SeetaFace6JNI object = p.getObject(); + object = null; + } +} diff --git a/smartjavaai-face/src/main/java/cn/smartjavaai/face/model/FeatureExtractionModel.java b/smartjavaai-face/src/main/java/cn/smartjavaai/face/model/FeatureExtractionModel.java new file mode 100644 index 0000000..af377c1 --- /dev/null +++ b/smartjavaai-face/src/main/java/cn/smartjavaai/face/model/FeatureExtractionModel.java @@ -0,0 +1,259 @@ +package cn.smartjavaai.face.model; + +import ai.djl.Device; +import ai.djl.MalformedModelException; +import ai.djl.inference.Predictor; +import ai.djl.modality.cv.Image; +import ai.djl.modality.cv.ImageFactory; +import ai.djl.repository.zoo.Criteria; +import ai.djl.repository.zoo.ModelNotFoundException; +import ai.djl.repository.zoo.ZooModel; +import ai.djl.training.util.ProgressBar; +import cn.smartjavaai.common.enums.DeviceEnum; +import cn.smartjavaai.common.pool.PredictorFactory; +import cn.smartjavaai.common.utils.FileUtils; +import cn.smartjavaai.common.utils.ImageUtils; +import cn.smartjavaai.face.AbstractFaceModel; +import cn.smartjavaai.face.FaceModelConfig; +import cn.smartjavaai.face.exception.FaceException; +import cn.smartjavaai.face.translator.FaceFeatureTranslator; +import lombok.extern.slf4j.Slf4j; +import org.apache.commons.lang3.StringUtils; +import org.apache.commons.pool2.ObjectPool; +import org.apache.commons.pool2.impl.GenericObjectPool; +import org.apache.commons.pool2.impl.GenericObjectPoolConfig; + +import javax.imageio.ImageIO; +import java.awt.image.BufferedImage; +import java.io.ByteArrayInputStream; +import java.io.IOException; +import java.io.InputStream; +import java.nio.file.Paths; +import java.time.Duration; +import java.util.Arrays; +import java.util.List; +import java.util.Objects; +import java.util.stream.Collectors; + +/** + * @author dwj + */ +@Slf4j +public class FeatureExtractionModel extends AbstractFaceModel implements AutoCloseable{ + + + private ObjectPool> predictorPool; + + + private ZooModel model; + + public static final List mean = + Arrays.asList( + 127.5f / 255.0f, + 127.5f / 255.0f, + 127.5f / 255.0f, + 128.0f / 255.0f, + 128.0f / 255.0f, + 128.0f / 255.0f); + + + /** + * 加载人脸特征提取模型 + * @param config + */ + @Override + public void loadModel(FaceModelConfig config) { + Device device = null; + if(!Objects.isNull(config.getDevice())){ + device = config.getDevice() == DeviceEnum.CPU ? Device.cpu() : Device.gpu(); + } + String normalize = mean.stream().map(Object::toString).collect(Collectors.joining(",")); + Criteria faceFeatureCriteria = + Criteria.builder() + .setTypes(Image.class, float[].class) + .optModelName("face_feature") // specify model file prefix + .optModelUrls(StringUtils.isNotBlank(config.getModelPath()) ? null : + "https://resources.djl.ai/test-models/pytorch/face_feature.zip") + .optModelPath(StringUtils.isNotBlank(config.getModelPath()) ? Paths.get(config.getModelPath()) : null) + .optTranslator(new FaceFeatureTranslator()) + .optArgument("normalize", normalize) + .optDevice(device) + .optEngine("PyTorch") // Use PyTorch engine + .optProgress(new ProgressBar()) + .build(); + + try { + model = faceFeatureCriteria.loadModel(); + // 创建池子：每个线程独享 Predictor + this.predictorPool = new GenericObjectPool<>(new PredictorFactory<>(model)); + log.info("当前设备: " + model.getNDManager().getDevice()); + } catch (IOException | ModelNotFoundException | MalformedModelException e) { + throw new FaceException("模型加载失败", e); + } + } + + private float[] featureExtraction(Image image){ + image.getWrappedImage(); + Predictor predictor = null; + try { + predictor = predictorPool.borrowObject(); + return predictor.predict(image); + } catch (Exception e) { + throw new FaceException("目标检测错误", e); + }finally { + if (predictor != null) { + try { + predictorPool.returnObject(predictor); //归还 + log.info("释放资源"); + } catch (Exception e) { + log.warn("归还Predictor失败", e); + try { + predictor.close(); // 归还失败才销毁 + } catch (Exception ex) { + log.error("关闭Predictor失败", ex); + } + } + } + } + } + + + /** + * 特征提取 + * @param imagePath 图片路径 + * @return + */ + @Override + public float[] featureExtraction(String imagePath) { + if(!FileUtils.isFileExists(imagePath)){ + throw new FaceException("图像文件不存在"); + } + Image img = null; + try { + img = ImageFactory.getInstance().fromFile(Paths.get(imagePath)); + } catch (IOException e) { + throw new FaceException("无效图片", e); + } + return featureExtraction(img); + } + + /** + * 特征提取 + * @param inputStream 输入流 + * @return + */ + @Override + public float[] featureExtraction(InputStream inputStream) { + if(Objects.isNull(inputStream)){ + throw new FaceException("图像输入流无效"); + } + Image img = null; + try { + img = ImageFactory.getInstance().fromInputStream(inputStream); + } catch (IOException e) { + throw new FaceException("无效图片输入流", e); + } + return featureExtraction(img); + } + + @Override + public float[] featureExtraction(BufferedImage sourceImage) { + if(!ImageUtils.isImageValid(sourceImage)){ + throw new FaceException("图像无效"); + } + Image img = ImageFactory.getInstance().fromImage(sourceImage); + return featureExtraction(img); + } + + @Override + public float[] featureExtraction(byte[] imageData) { + if(Objects.isNull(imageData)){ + throw new FaceException("图像无效"); + } + try { + return featureExtraction(ImageIO.read(new ByteArrayInputStream(imageData))); + } catch (IOException e) { + throw new FaceException("无效图片字节流", e); + } + } + + /** + * 计算相似度 + * @param feature1 图1特征 + * @param feature2 图2特征 + * @return + */ + @Override + public float calculSimilar(float[] feature1, float[] feature2) { + float ret = 0.0f; + float mod1 = 0.0f; + float mod2 = 0.0f; + int length = feature1.length; + for (int i = 0; i < length; ++i) { + ret += feature1[i] * feature2[i]; + mod1 += feature1[i] * feature1[i]; + mod2 += feature2[i] * feature2[i]; + } + return (float) ((ret / Math.sqrt(mod1) / Math.sqrt(mod2) + 1) / 2.0f); + } + + /** + * 特征比较 + * @param imagePath1 图1路径 + * @param imagePath2 图2路径 + * @return + */ + @Override + public float featureComparison(String imagePath1, String imagePath2) { + if(!FileUtils.isFileExists(imagePath1) || !FileUtils.isFileExists(imagePath2)){ + throw new FaceException("图像文件不存在"); + } + float[] feature1 = featureExtraction(imagePath1); + float[] feature2 = featureExtraction(imagePath2); + return calculSimilar(feature1, feature2); + } + + /** + * 特征比较 + * @param inputStream1 图1输入流 + * @param inputStream2 图2输入流 + * @return + */ + @Override + public float featureComparison(InputStream inputStream1, InputStream inputStream2) { + if(Objects.isNull(inputStream1) || Objects.isNull(inputStream2)){ + throw new FaceException("图像输入流无效"); + } + float[] feature1 = featureExtraction(inputStream1); + float[] feature2 = featureExtraction(inputStream2); + return calculSimilar(feature1, feature2); + } + + @Override + public float featureComparison(BufferedImage sourceImage1, BufferedImage sourceImag2) { + if(!ImageUtils.isImageValid(sourceImage1) || !ImageUtils.isImageValid(sourceImag2)){ + throw new FaceException("图像无效"); + } + float[] feature1 = featureExtraction(sourceImage1); + float[] feature2 = featureExtraction(sourceImag2); + return calculSimilar(feature1, feature2); + } + + @Override + public float featureComparison(byte[] imageData1, byte[] imageData2) { + if(Objects.isNull(imageData1) || Objects.isNull(imageData2)){ + throw new FaceException("图像无效"); + } + float[] feature1 = featureExtraction(imageData1); + float[] feature2 = featureExtraction(imageData2); + return calculSimilar(feature1, feature2); + } + + @Override + public void close() { + if (predictorPool != null) { + predictorPool.close(); + } + } + +} diff --git a/smartjavaai-face/src/main/java/cn/smartjavaai/face/model/RetinaFaceModel.java b/smartjavaai-face/src/main/java/cn/smartjavaai/face/model/RetinaFaceModel.java new file mode 100644 index 0000000..f98dfea --- /dev/null +++ b/smartjavaai-face/src/main/java/cn/smartjavaai/face/model/RetinaFaceModel.java @@ -0,0 +1,243 @@ +package cn.smartjavaai.face.model; + +import ai.djl.Device; +import ai.djl.MalformedModelException; +import ai.djl.inference.Predictor; +import ai.djl.modality.cv.Image; +import ai.djl.modality.cv.ImageFactory; +import ai.djl.modality.cv.output.DetectedObjects; +import ai.djl.repository.zoo.Criteria; +import ai.djl.repository.zoo.ModelNotFoundException; +import ai.djl.repository.zoo.ZooModel; +import ai.djl.training.util.ProgressBar; +import cn.smartjavaai.common.entity.DetectionResponse; +import cn.smartjavaai.common.enums.DeviceEnum; +import cn.smartjavaai.common.pool.PredictorFactory; +import cn.smartjavaai.common.utils.FileUtils; +import cn.smartjavaai.common.utils.ImageUtils; +import cn.smartjavaai.face.*; +import cn.smartjavaai.face.exception.FaceException; +import cn.smartjavaai.face.translator.FaceDetectionTranslator; +import cn.smartjavaai.face.utils.FaceUtils; +import lombok.extern.slf4j.Slf4j; +import org.apache.commons.lang3.StringUtils; +import org.apache.commons.pool2.ObjectPool; +import org.apache.commons.pool2.impl.GenericObjectPool; +import org.apache.commons.pool2.impl.GenericObjectPoolConfig; + + +import javax.imageio.ImageIO; +import java.awt.image.BufferedImage; +import java.io.*; +import java.nio.file.Files; +import java.nio.file.Path; +import java.nio.file.Paths; +import java.time.Duration; +import java.util.Objects; + +/** + * RetinaFace实现 + * @author dwj + */ +@Slf4j +public class RetinaFaceModel extends AbstractFaceModel implements AutoCloseable{ + + + private ObjectPool> predictorPool; + + private ZooModel model; + + /** + * 特征图层的基础缩放比例 + */ + public static final int[][] scales = {{16, 32}, {64, 128}, {256, 512}}; + /** + * 特征图相对于原图的采样步长 + */ + public static final int[] steps = {8, 16, 32}; + /** + * 缩放系数 + */ + public static final double[] variance = {0.1f, 0.2f}; + + + /** + * 加载模型 + * @param config + */ + @Override + public void loadModel(FaceModelConfig config){ + Device device = null; + if(!Objects.isNull(config.getDevice())){ + device = config.getDevice() == DeviceEnum.CPU ? Device.cpu() : Device.gpu(); + } + FaceDetectionTranslator translator = + new FaceDetectionTranslator(config.getConfidenceThreshold(), config.getNmsThresh(), variance, FaceConfig.MAX_FACE_LIMIT, scales, steps); + Criteria criteria = + Criteria.builder() + .setTypes(Image.class, DetectedObjects.class) + .optModelUrls(StringUtils.isNotBlank(config.getModelPath()) ? null : "https://resources.djl.ai/test-models/pytorch/retinaface.zip") + // Load model from local file, e.g: + .optModelPath(StringUtils.isNotBlank(config.getModelPath()) ? Paths.get(config.getModelPath()) : null) + .optModelName("retinaface") // specify model file prefix + .optTranslator(translator) + .optDevice(device) + .optProgress(new ProgressBar()) + .optEngine("PyTorch") // Use PyTorch engine + .build(); + try { + model = criteria.loadModel(); + // 创建池子：每个线程独享 Predictor + this.predictorPool = new GenericObjectPool<>(new PredictorFactory<>(model)); + log.info("当前设备: " + model.getNDManager().getDevice()); + } catch (IOException | ModelNotFoundException | MalformedModelException e) { + throw new FaceException("模型加载失败", e); + } + } + + + + /** + * 检测人脸 + * @param imagePath 图片路径 + * @return + * @throws Exception + */ + @Override + public DetectionResponse detect(String imagePath){ + if(!FileUtils.isFileExists(imagePath)){ + throw new FaceException("图像文件不存在"); + } + Image img = null; + try { + img = ImageFactory.getInstance().fromFile(Paths.get(imagePath)); + } catch (IOException e) { + throw new FaceException("无效的图片", e); + } + DetectedObjects detection = detect(img); + return FaceUtils.convertToDetectionResponse(detection,img); + } + + /** + * 检测人脸 + * @param imageInputStream 图片流 + * @return + * @throws Exception + */ + @Override + public DetectionResponse detect(InputStream imageInputStream){ + if(Objects.isNull(imageInputStream)){ + throw new FaceException("图像输入流无效"); + } + try { + Image img = ImageFactory.getInstance().fromInputStream(imageInputStream); + DetectedObjects detection = detect(img); + return FaceUtils.convertToDetectionResponse(detection,img); + } catch (IOException e) { + throw new FaceException("无效图片输入流", e); + } + + } + + @Override + public DetectionResponse detect(BufferedImage image) { + if(!ImageUtils.isImageValid(image)){ + throw new FaceException("图像无效"); + } + Image img = ImageFactory.getInstance().fromImage(image); + DetectedObjects detection = detect(img); + return FaceUtils.convertToDetectionResponse(detection,img); + } + + @Override + public DetectionResponse detect(byte[] imageData) { + if(Objects.isNull(imageData)){ + throw new FaceException("图像无效"); + } + try { + return detect(ImageIO.read(new ByteArrayInputStream(imageData))); + } catch (IOException e) { + throw new FaceException("错误的图像", e); + } + } + + @Override + public void detectAndDraw(String imagePath, String outputPath) { + if(!FileUtils.isFileExists(imagePath)){ + throw new FaceException("图像文件不存在"); + } + try { + Image img = ImageFactory.getInstance().fromFile(Paths.get(imagePath)); + DetectedObjects detectedObjects = detect(img); + if(Objects.isNull(detectedObjects) || detectedObjects.getNumberOfObjects() == 0){ + throw new FaceException("未识别到人脸"); + } + img.drawBoundingBoxes(detectedObjects); + Path output = Paths.get(outputPath); + log.info("Saving to {}", output.toAbsolutePath().toString()); + img.save(Files.newOutputStream(output), "png"); + } catch (IOException e) { + throw new FaceException(e); + } + } + + @Override + public BufferedImage detectAndDraw(BufferedImage sourceImage) { + if(!ImageUtils.isImageValid(sourceImage)){ + throw new FaceException("图像无效"); + } + Image img = ImageFactory.getInstance().fromImage(sourceImage); + DetectedObjects detectedObjects = detect(img); + if(Objects.isNull(detectedObjects) || detectedObjects.getNumberOfObjects() == 0){ + throw new FaceException("未识别到人脸"); + } + img.drawBoundingBoxes(detectedObjects); + try { + ByteArrayOutputStream outputStream = new ByteArrayOutputStream(); + // 调用 save 方法将 Image 写入字节流 + img.save(outputStream, "png"); + // 将字节流转换为 BufferedImage + byte[] imageBytes = outputStream.toByteArray(); + return ImageIO.read(new ByteArrayInputStream(imageBytes)); + } catch (IOException e) { + throw new FaceException("导出图片失败", e); + } + } + + /** + * 人脸检测 + * @param image + * @return + */ + private DetectedObjects detect(Image image){ + Predictor predictor = null; + try { + predictor = predictorPool.borrowObject(); + return predictor.predict(image); + } catch (Exception e) { + throw new FaceException("目标检测错误", e); + }finally { + if (predictor != null) { + try { + predictorPool.returnObject(predictor); //归还 + log.info("释放资源"); + } catch (Exception e) { + log.warn("归还Predictor失败", e); + try { + predictor.close(); // 归还失败才销毁 + } catch (Exception ex) { + log.error("关闭Predictor失败", ex); + } + } + } + } + } + + + @Override + public void close() { + if (predictorPool != null) { + predictorPool.close(); + } + } +} diff --git a/smartjavaai-face/src/main/java/cn/smartjavaai/face/model/SeetaFace6Model.java b/smartjavaai-face/src/main/java/cn/smartjavaai/face/model/SeetaFace6Model.java new file mode 100644 index 0000000..1030f2a --- /dev/null +++ b/smartjavaai-face/src/main/java/cn/smartjavaai/face/model/SeetaFace6Model.java @@ -0,0 +1,585 @@ +package cn.smartjavaai.face.model; + +import cn.smartjavaai.common.entity.DetectionResponse; +import cn.smartjavaai.common.utils.FileUtils; +import cn.smartjavaai.common.utils.ImageUtils; +import cn.smartjavaai.face.AbstractFaceModel; +import cn.smartjavaai.face.FaceModelConfig; +import cn.smartjavaai.face.dao.FaceDao; +import cn.smartjavaai.face.entity.FaceData; +import cn.smartjavaai.face.entity.FaceResult; +import cn.smartjavaai.face.exception.FaceException; +import cn.smartjavaai.face.utils.FaceUtils; +import com.seetaface.NativeLoader; +import com.seetaface.SeetaFace6JNI; +import com.seetaface.model.RecognizeResult; +import com.seetaface.model.SeetaImageData; +import com.seetaface.model.SeetaRect; +import lombok.extern.slf4j.Slf4j; +import org.apache.commons.lang3.StringUtils; + +import javax.imageio.ImageIO; +import java.awt.image.BufferedImage; +import java.io.*; +import java.nio.file.Path; +import java.nio.file.Paths; +import java.sql.SQLException; +import java.util.List; +import java.util.Objects; + +/** + * SeetaFace6 人脸算法 + * @author dwj + */ +@SuppressWarnings("AliMissingOverrideAnnotation") +@Slf4j +public class SeetaFace6Model extends AbstractFaceModel { + + + private FaceModelConfig config; + + private static final Object lock = new Object(); // 全局锁 + + + @Override + public void loadModel(FaceModelConfig config) { + this.config = config; + if (NativeLoader.seetaFace6SDK == null) { + synchronized (lock) { + if(StringUtils.isBlank(config.getModelPath())){ + throw new FaceException("modelPath is null"); + } + //加载依赖库 + NativeLoader.loadNativeLibraries(config.getModelPath()); + log.info("Loading seetaFace6 library successfully."); + NativeLoader.seetaFace6SDK = new SeetaFace6JNI(); + //加载模型 + boolean isSuccess = NativeLoader.seetaFace6SDK.initModel(config.getModelPath()); + if(!isSuccess){ + throw new FaceException("seetaFace6模型初始化失败," + config.getModelPath()); + } + log.info("Load seetaFace6 model success!"); + new Thread(new Runnable() { + public void run() { + try { + log.info("start load faceDb..."); + loadFaceDb(); + log.info("Load faceDb success!"); + } catch (Exception e) { + e.printStackTrace(); + } + } + }).start(); + } + } + } + + @Override + public DetectionResponse detect(String imagePath) { + if(!FileUtils.isFileExists(imagePath)){ + throw new FaceException("图像文件不存在"); + } + // 将图片路径转换为 BufferedImage + BufferedImage image = null; + try { + image = ImageIO.read(new File(Paths.get(imagePath).toAbsolutePath().toString())); + } catch (IOException e) { + throw new FaceException("无效图片路径", e); + } + return detect(image); + } + + @Override + public DetectionResponse detect(InputStream imageInputStream) { + if(Objects.isNull(imageInputStream)){ + throw new FaceException("图像输入流无效"); + } + BufferedImage image = null; + try { + image = ImageIO.read(imageInputStream); + } catch (IOException e) { + throw new FaceException("无效图片输入流", e); + } + return detect(image); + } + + @Override + public DetectionResponse detect(BufferedImage image) { + if(!ImageUtils.isImageValid(image)){ + throw new FaceException("图像无效"); + } + SeetaImageData imageData = new SeetaImageData(image.getWidth(), image.getHeight(), 3); + imageData.data = ImageUtils.getMatrixBGR(image); + synchronized (lock) { + SeetaRect[] seetaResult = NativeLoader.seetaFace6SDK.detect(imageData); + return FaceUtils.convertToDetectionResponse(seetaResult, config); + } + } + + @Override + public DetectionResponse detect(byte[] imageData) { + if(Objects.isNull(imageData)){ + throw new FaceException("图像无效"); + } + try { + return detect(ImageIO.read(new ByteArrayInputStream(imageData))); + } catch (IOException e) { + throw new FaceException("错误的图像", e); + } + } + + @Override + public void detectAndDraw(String imagePath, String outputPath) { + if(!FileUtils.isFileExists(imagePath)){ + throw new FaceException("图像文件不存在"); + } + try { + //创建保存路径 + Path imageOutputPath = Paths.get(outputPath); + BufferedImage image = null; + try { + image = ImageIO.read(new File(Paths.get(imagePath).toAbsolutePath().toString())); + } catch (IOException e) { + throw new FaceException("无效图片路径", e); + } + DetectionResponse result = detect(image); + if(Objects.isNull(result) || Objects.isNull(result.getRectangleList()) || result.getRectangleList().isEmpty()){ + throw new FaceException("未识别到人脸"); + } + //绘制人脸框 + FaceUtils.drawBoundingBoxes(image, result, imageOutputPath.toAbsolutePath().toString()); + } catch (IOException e) { + throw new FaceException(e); + } + } + + @Override + public BufferedImage detectAndDraw(BufferedImage sourceImage) { + if(!ImageUtils.isImageValid(sourceImage)){ + throw new FaceException("图像无效"); + } + DetectionResponse detectedObjects = detect(sourceImage); + if(Objects.isNull(detectedObjects) || Objects.isNull(detectedObjects.getRectangleList()) || detectedObjects.getRectangleList().isEmpty()){ + throw new FaceException("未识别到人脸"); + } + //绘制人脸框 + try { + return FaceUtils.drawBoundingBoxes(sourceImage, detectedObjects); + } catch (IOException e) { + throw new RuntimeException(e); + } + } + + @Override + public float[] featureExtraction(BufferedImage image) { + if(!ImageUtils.isImageValid(image)){ + throw new FaceException("图像无效"); + } + SeetaImageData imageData = new SeetaImageData(image.getWidth(), image.getHeight(), 3); + imageData.data = ImageUtils.getMatrixBGR(image); + synchronized (lock) { + return NativeLoader.seetaFace6SDK.extractMaxFace(imageData); + } + + } + + @Override + public float[] featureExtraction(String imagePath) { + if(!FileUtils.isFileExists(imagePath)){ + throw new FaceException("图像文件不存在"); + } + BufferedImage image = null; + try { + image = ImageIO.read(new File(Paths.get(imagePath).toAbsolutePath().toString())); + } catch (IOException e) { + throw new FaceException("无效图片路径", e); + } + return featureExtraction(image); + } + + @Override + public float[] featureExtraction(InputStream inputStream) { + if(Objects.isNull(inputStream)){ + throw new FaceException("图像输入流无效"); + } + BufferedImage image = null; + try { + image = ImageIO.read(inputStream); + } catch (IOException e) { + throw new FaceException("无效图片输入流", e); + } + return featureExtraction(image); + } + + @Override + public float[] featureExtraction(byte[] imageData) { + if(Objects.isNull(imageData)){ + throw new FaceException("图像无效"); + } + try { + return featureExtraction(ImageIO.read(new ByteArrayInputStream(imageData))); + } catch (IOException e) { + throw new FaceException("错误的图像", e); + } + } + + @Override + public float calculSimilar(float[] feature1, float[] feature2) { + if(Objects.isNull(feature1) || Objects.isNull(feature2)){ + throw new FaceException("特征向量无效"); + } + synchronized (lock) { + return NativeLoader.seetaFace6SDK.calculateSimilarity(feature1, feature2); + } + } + + @Override + public float featureComparison(String imagePath1, String imagePath2) { + if(!FileUtils.isFileExists(imagePath1) || !FileUtils.isFileExists(imagePath2)){ + throw new FaceException("图像文件不存在"); + } + BufferedImage image1 = null; + BufferedImage image2 = null; + try { + image1 = ImageIO.read(new File(Paths.get(imagePath1).toAbsolutePath().toString())); + image2 = ImageIO.read(new File(Paths.get(imagePath2).toAbsolutePath().toString())); + } catch (IOException e) { + throw new FaceException("无效图片路径", e); + } + return featureComparison(image1, image2); + } + + @Override + public float featureComparison(InputStream inputStream1, InputStream inputStream2) { + if(Objects.isNull(inputStream1) || Objects.isNull(inputStream2)){ + throw new FaceException("图像输入流无效"); + } + BufferedImage image1 = null; + BufferedImage image2 = null; + try { + image1 = ImageIO.read(inputStream1); + image2 = ImageIO.read(inputStream2); + } catch (IOException e) { + throw new FaceException("无效图片输入流", e); + } + return featureComparison(image1, image2); + } + + + @Override + public float featureComparison(BufferedImage image1, BufferedImage image2) { + if(!ImageUtils.isImageValid(image1) || !ImageUtils.isImageValid(image2)){ + throw new FaceException("图像无效"); + } + SeetaImageData imageData1 = new SeetaImageData(image1.getWidth(), image1.getHeight(), 3); + imageData1.data = ImageUtils.getMatrixBGR(image1); + + SeetaImageData imageData2 = new SeetaImageData(image2.getWidth(), image2.getHeight(), 3); + imageData2.data = ImageUtils.getMatrixBGR(image2); + synchronized (lock) { + //裁剪 + byte[][] cropImg1 = NativeLoader.seetaFace6SDK.crop(imageData1); + byte[][] cropImg2 = NativeLoader.seetaFace6SDK.crop(imageData2); + if(cropImg1 == null || cropImg1.length == 0){ + throw new FaceException("未发现人脸"); + } + if(cropImg2 == null || cropImg2.length == 0){ + throw new FaceException("未发现人脸"); + } + BufferedImage cropImage1 = ImageUtils.bgrToBufferedImage(cropImg1[0], 256, 256); + BufferedImage cropImage2 = ImageUtils.bgrToBufferedImage(cropImg2[0], 256, 256); + + SeetaImageData cropImageData1 = new SeetaImageData(cropImage1.getWidth(), cropImage1.getHeight(), 3); + cropImageData1.data = ImageUtils.getMatrixBGR(cropImage1); + SeetaImageData cropImageData2 = new SeetaImageData(cropImage2.getWidth(), cropImage2.getHeight(), 3); + cropImageData2.data = ImageUtils.getMatrixBGR(cropImage2); + return NativeLoader.seetaFace6SDK.compare(cropImageData1, cropImageData2); + } + } + + @Override + public float featureComparison(byte[] imageData1, byte[] imageData2) { + if(Objects.isNull(imageData1) || Objects.isNull(imageData2)){ + throw new FaceException("图像无效"); + } + BufferedImage image1 = null; + BufferedImage image2 = null; + try { + image1 = ImageIO.read(new ByteArrayInputStream(imageData1)); + image2 = ImageIO.read(new ByteArrayInputStream(imageData2)); + } catch (IOException e) { + throw new FaceException("无效图片", e); + } + return featureComparison(image1, image2); + } + + @Override + public boolean register(String key, String imagePath) { + if(!FileUtils.isFileExists(imagePath)){ + throw new FaceException("图像文件不存在"); + } + BufferedImage bufferedImage = null; + try { + bufferedImage = ImageIO.read(new File(Paths.get(imagePath).toAbsolutePath().toString())); + } catch (IOException e) { + throw new FaceException("无效图片路径", e); + } + return register(key, bufferedImage); + } + + + @Override + public boolean register(String key, BufferedImage image) { + if(!checkFaceDb()){ + throw new FaceException("未找到人脸库，无法使用此功能（请检查是否配置人脸库路径）"); + } + if(!ImageUtils.isImageValid(image)){ + throw new FaceException("图像无效"); + } + SeetaImageData imageData = new SeetaImageData(image.getWidth(), image.getHeight(), 3); + imageData.data = ImageUtils.getMatrixBGR(image); + synchronized (lock) { + byte[][] bytes = NativeLoader.seetaFace6SDK.crop(imageData); + if (bytes == null || bytes.length == 0) { + log.info("register face fail: key={}, error=no valid face", key); + return false; + } + long index = NativeLoader.seetaFace6SDK.registerCroppedFace(bytes[0]); + if (index < 0) { + log.info("register face fail: key={}, index={}", key, index); + return false; + } + //持久化到sqlite数据库 + FaceData face = new FaceData(); + face.setKey(key); + face.setIndex(index); + face.setImgData(bytes[0]); + try { + new FaceDao(config.getFaceDbPath()).save(face); + } catch (SQLException | ClassNotFoundException e) { + throw new FaceException("保存人脸库失败", e); + } + return true; + } + } + + @Override + public boolean register(String key, byte[] imageData) { + if(Objects.isNull(imageData)){ + throw new FaceException("图像无效"); + } + BufferedImage bufferedImage = null; + try { + bufferedImage = ImageIO.read(new ByteArrayInputStream(imageData)); + } catch (IOException e) { + throw new FaceException(e); + } + return register(key, bufferedImage); + } + + @Override + public boolean register(String key, InputStream inputStream) { + if(!checkFaceDb()){ + throw new FaceException("未找到人脸库，无法使用此功能（请检查是否配置人脸库路径）"); + } + if(Objects.isNull(inputStream)){ + throw new FaceException("图像输入流无效"); + } + BufferedImage image = null; + try { + image = ImageIO.read(inputStream); + } catch (IOException e) { + throw new FaceException("无效的图片输入流", e); + } + return register(key, image); + } + + /** + * 注册已裁剪后人脸 + * @param key + * @param faceData + * @return + */ + private boolean register(String key, FaceData faceData) { + synchronized (lock) { + long index = NativeLoader.seetaFace6SDK.registerCroppedFace(faceData.getImgData()); + if (index < 0) { + log.info("register face fail: key={}, index={}", key, index); + return false; + } + int rows = 0; + try { + rows = new FaceDao(config.getFaceDbPath()).updateIndex(index, faceData); + } catch (SQLException | ClassNotFoundException e) { + throw new FaceException(e); + } + return rows > 0; + } + } + + + + @Override + public FaceResult search(String imagePath) { + if(!FileUtils.isFileExists(imagePath)){ + throw new FaceException("图像文件不存在"); + } + BufferedImage bufferedImage = null; + try { + bufferedImage = ImageIO.read(new File(Paths.get(imagePath).toAbsolutePath().toString())); + } catch (IOException e) { + throw new FaceException("无效图片路径", e); + } + return search(bufferedImage); + } + + @Override + public FaceResult search(InputStream inputStream) { + if(Objects.isNull(inputStream)){ + throw new FaceException("图像输入流无效"); + } + if(!checkFaceDb()){ + throw new FaceException("未找到人脸库，无法使用此功能（请检查是否配置人脸库路径）"); + } + BufferedImage image = null; + try { + image = ImageIO.read(inputStream); + } catch (IOException e) { + throw new FaceException("无效图片输入流", e); + } + return search(image); + } + + @Override + public FaceResult search(BufferedImage image) { + if(!ImageUtils.isImageValid(image)){ + throw new FaceException("图像无效"); + } + SeetaImageData imageData = new SeetaImageData(image.getWidth(), image.getHeight(), 3); + imageData.data = ImageUtils.getMatrixBGR(image); + synchronized (lock) { + RecognizeResult recognizeResult = NativeLoader.seetaFace6SDK.query(imageData); + return searchFaceDb(recognizeResult); + } + } + + @Override + public FaceResult search(byte[] imageData) { + if(Objects.isNull(imageData)){ + throw new FaceException("图像无效"); + } + BufferedImage bufferedImage = null; + try { + bufferedImage = ImageIO.read(new ByteArrayInputStream(imageData)); + } catch (IOException e) { + throw new FaceException(e); + } + return search(bufferedImage); + } + + @Override + public long removeRegister(String... keys) { + if(keys == null || keys.length == 0){ + throw new FaceException("keys不允许为空"); + } + if(!checkFaceDb()){ + throw new FaceException("未找到人脸库，无法使用此功能（请检查是否配置人脸库路径）"); + } + synchronized (lock) { + try { + List list = new FaceDao(config.getFaceDbPath()).findIndexList(keys); + if (list == null) { + return 0; + } + long[] array = list.stream().mapToLong(Long::longValue).toArray(); + long rows = NativeLoader.seetaFace6SDK.delete(array); + new FaceDao(config.getFaceDbPath()).deleteFace(keys); + return rows; + } catch (SQLException | ClassNotFoundException e) { + throw new FaceException(e); + } + } + + } + + @Override + public long clearFace(){ + if(!checkFaceDb()){ + throw new FaceException("未找到人脸库，无法使用此功能（请检查是否配置人脸库路径）"); + } + synchronized (lock) { + long rows = NativeLoader.seetaFace6SDK.delete(new long[]{-1}); + try { + new FaceDao(config.getFaceDbPath()).deleteAll(); + } catch (SQLException | ClassNotFoundException e) { + throw new FaceException("删除人脸库失败", e); + } + return rows; + } + + } + + /** + * 检查是否存在人脸库 + * @return + */ + private boolean checkFaceDb(){ + if(Objects.nonNull(config) && StringUtils.isNotBlank(config.getFaceDbPath())){ + File file = new File(config.getFaceDbPath()); + return file.exists() && file.isFile(); + } + return false; + } + + private FaceResult searchFaceDb(RecognizeResult recognizeResult) { + if(recognizeResult != null && recognizeResult.index >= 0){ + String key = null; + synchronized (lock) { + try { + key = new FaceDao(config.getFaceDbPath()).findKeyByIndex(recognizeResult.index); + } catch (SQLException | ClassNotFoundException e) { + throw new FaceException("查询人脸库失败", e); + } + return new FaceResult(key, recognizeResult.similar); + } + } + return null; + } + + + /** + * 加载人脸库 + * @throws SQLException + * @throws ClassNotFoundException + */ + private void loadFaceDb() { + if(!checkFaceDb()){ + log.info("未配置人脸库"); + return; + } + //分页查询人脸库 + int pageNo = 0, pageSize = 100; + while (true) { + List list = null; + try { + list = new FaceDao(config.getFaceDbPath()).findFace(pageNo, pageSize); + } catch (SQLException | ClassNotFoundException e) { + throw new FaceException("查询人脸库失败", e); + } + if (list == null) { + break; + } + list.forEach(face -> { + try { + register(face.getKey(), face); + } catch (Exception e) { + e.printStackTrace(); + } + }); + if (list.size() < pageSize) { + break; + } + pageNo++; + } + } +} diff --git a/smartjavaai-face/src/main/java/cn/smartjavaai/face/model/UltraLightFastGenericFaceModel.java b/smartjavaai-face/src/main/java/cn/smartjavaai/face/model/UltraLightFastGenericFaceModel.java new file mode 100644 index 0000000..4c83a6a --- /dev/null +++ b/smartjavaai-face/src/main/java/cn/smartjavaai/face/model/UltraLightFastGenericFaceModel.java @@ -0,0 +1,232 @@ +package cn.smartjavaai.face.model; + +import ai.djl.Device; +import ai.djl.MalformedModelException; +import ai.djl.inference.Predictor; +import ai.djl.modality.cv.Image; +import ai.djl.modality.cv.ImageFactory; +import ai.djl.modality.cv.output.DetectedObjects; +import ai.djl.repository.zoo.Criteria; +import ai.djl.repository.zoo.ModelNotFoundException; +import ai.djl.repository.zoo.ZooModel; +import ai.djl.training.util.ProgressBar; +import cn.smartjavaai.common.entity.DetectionResponse; +import cn.smartjavaai.common.enums.DeviceEnum; +import cn.smartjavaai.common.pool.PredictorFactory; +import cn.smartjavaai.common.utils.FileUtils; +import cn.smartjavaai.common.utils.ImageUtils; +import cn.smartjavaai.face.*; +import cn.smartjavaai.face.exception.FaceException; +import cn.smartjavaai.face.translator.FaceDetectionTranslator; +import cn.smartjavaai.face.utils.FaceUtils; +import lombok.extern.slf4j.Slf4j; +import org.apache.commons.lang3.StringUtils; +import org.apache.commons.pool2.ObjectPool; +import org.apache.commons.pool2.impl.GenericObjectPool; +import org.apache.commons.pool2.impl.GenericObjectPoolConfig; + +import javax.imageio.ImageIO; +import java.awt.image.BufferedImage; +import java.io.*; +import java.nio.file.Paths; +import java.time.Duration; +import java.util.Objects; + +/** + * @author dwj + */ +@Slf4j +public class UltraLightFastGenericFaceModel extends AbstractFaceModel implements AutoCloseable{ + + + private ObjectPool> predictorPool; + + /** + * 特征图层的基础缩放比例 + */ + private static final int[][] scales = {{10, 16, 24}, {32, 48}, {64, 96}, {128, 192, 256}}; + /** + * 特征图相对于原图的采样步长 + */ + private static final int[] steps = {8, 16, 32, 64}; + /** + * 缩放系数 + */ + private static final double[] variance = {0.1f, 0.2f}; + + + private ZooModel model; + + + + + /** + * 加载模型 + * @param config + */ + @Override + public void loadModel(FaceModelConfig config) { + Device device = null; + if(!Objects.isNull(config.getDevice())){ + device = config.getDevice() == DeviceEnum.CPU ? Device.cpu() : Device.gpu(); + } + FaceDetectionTranslator translator = + new FaceDetectionTranslator(config.getConfidenceThreshold(), config.getNmsThresh(), variance, FaceConfig.MAX_FACE_LIMIT, scales, steps); + Criteria criteria = + Criteria.builder() + .setTypes(Image.class, DetectedObjects.class) + .optModelUrls(StringUtils.isNotBlank(config.getModelPath()) ? null : "https://resources.djl.ai/test-models/pytorch/ultranet.zip") + .optModelPath(StringUtils.isNotBlank(config.getModelPath()) ? Paths.get(config.getModelPath()) : null) + .optTranslator(translator) + .optProgress(new ProgressBar()) + .optDevice(device) + .optEngine("PyTorch") // Use PyTorch engine + .build(); + try { + model = criteria.loadModel(); + // 创建池子：每个线程独享 Predictor + this.predictorPool = new GenericObjectPool<>(new PredictorFactory<>(model)); + log.info("当前设备: " + model.getNDManager().getDevice()); + } catch (IOException | ModelNotFoundException | MalformedModelException e) { + throw new FaceException("模型加载失败", e); + } + } + + /** + * 检测人脸 + * @param imagePath 图片路径 + * @return + * @throws Exception + */ + @Override + public DetectionResponse detect(String imagePath){ + if(!FileUtils.isFileExists(imagePath)){ + throw new FaceException("图像文件不存在"); + } + Image img = null; + try { + img = ImageFactory.getInstance().fromFile(Paths.get(imagePath)); + } catch (IOException e) { + throw new FaceException("无效的图片", e); + } + DetectedObjects detection = detect(img); + return FaceUtils.convertToDetectionResponse(detection,img); + } + + /** + * 检测人脸 + * @param imageInputStream 图片流 + * @return + * @throws Exception + */ + @Override + public DetectionResponse detect(InputStream imageInputStream){ + try { + Image img = ImageFactory.getInstance().fromInputStream(imageInputStream); + DetectedObjects detection = detect(img); + return FaceUtils.convertToDetectionResponse(detection,img); + } catch (IOException e) { + throw new FaceException("无效图片输入流", e); + } + + } + + @Override + public DetectionResponse detect(BufferedImage image) { + Image img = ImageFactory.getInstance().fromImage(image); + DetectedObjects detection = detect(img); + return FaceUtils.convertToDetectionResponse(detection,img); + } + + @Override + public DetectionResponse detect(byte[] imageData) { + if(Objects.isNull(imageData)){ + throw new FaceException("图像无效"); + } + try { + return detect(ImageIO.read(new ByteArrayInputStream(imageData))); + } catch (IOException e) { + throw new FaceException("错误的图像", e); + } + } + + @Override + public void detectAndDraw(String imagePath, String outputPath) { + if(!FileUtils.isFileExists(imagePath)){ + throw new FaceException("图像文件不存在"); + } + try { + Image img = ImageFactory.getInstance().fromFile(Paths.get(imagePath)); + DetectedObjects detectedObjects = detect(img); + if(Objects.isNull(detectedObjects) || detectedObjects.getNumberOfObjects() == 0){ + throw new FaceException("未识别到人脸"); + } + img.drawBoundingBoxes(detectedObjects); + ByteArrayOutputStream outputStream = new ByteArrayOutputStream(); + // 调用 save 方法将 Image 写入字节流 + img.save(new FileOutputStream(Paths.get(outputPath).toAbsolutePath().toString()), "png"); + } catch (IOException e) { + throw new FaceException(e); + } + } + + @Override + public BufferedImage detectAndDraw(BufferedImage sourceImage) { + if(!ImageUtils.isImageValid(sourceImage)){ + throw new FaceException("图像无效"); + } + Image img = ImageFactory.getInstance().fromImage(sourceImage); + DetectedObjects detectedObjects = detect(img); + if(Objects.isNull(detectedObjects) || detectedObjects.getNumberOfObjects() == 0){ + throw new FaceException("未识别到人脸"); + } + img.drawBoundingBoxes(detectedObjects); + try { + ByteArrayOutputStream outputStream = new ByteArrayOutputStream(); + // 调用 save 方法将 Image 写入字节流 + img.save(outputStream, "png"); + // 将字节流转换为 BufferedImage + byte[] imageBytes = outputStream.toByteArray(); + return ImageIO.read(new ByteArrayInputStream(imageBytes)); + } catch (IOException e) { + throw new FaceException("导出图片失败", e); + } + } + + /** + * 人脸检测 + * @param image + * @return + */ + private DetectedObjects detect(Image image){ + Predictor predictor = null; + try { + predictor = predictorPool.borrowObject(); + return predictor.predict(image); + } catch (Exception e) { + throw new FaceException("目标检测错误", e); + }finally { + if (predictor != null) { + try { + predictorPool.returnObject(predictor); //归还 + log.info("释放资源"); + } catch (Exception e) { + log.warn("归还Predictor失败", e); + try { + predictor.close(); // 归还失败才销毁 + } catch (Exception ex) { + log.error("关闭Predictor失败", ex); + } + } + } + } + } + + + @Override + public void close() { + if (predictorPool != null) { + predictorPool.close(); + } + } +} diff --git a/smartjavaai-face/src/main/java/cn/smartjavaai/face/FaceDetectionTranslator.java b/smartjavaai-face/src/main/java/cn/smartjavaai/face/translator/FaceDetectionTranslator.java similarity index 99% rename from smartjavaai-face/src/main/java/cn/smartjavaai/face/FaceDetectionTranslator.java rename to smartjavaai-face/src/main/java/cn/smartjavaai/face/translator/FaceDetectionTranslator.java index f70b567..341524e 100644 --- a/smartjavaai-face/src/main/java/cn/smartjavaai/face/FaceDetectionTranslator.java +++ b/smartjavaai-face/src/main/java/cn/smartjavaai/face/translator/FaceDetectionTranslator.java @@ -10,7 +10,7 @@ * OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions * and limitations under the License. */ -package cn.smartjavaai.face; +package cn.smartjavaai.face.translator; import ai.djl.modality.cv.Image; import ai.djl.modality.cv.output.*; diff --git a/smartjavaai-face/src/main/java/cn/smartjavaai/face/translator/FaceFeatureTranslator.java b/smartjavaai-face/src/main/java/cn/smartjavaai/face/translator/FaceFeatureTranslator.java index ec2d430..99a4628 100644 --- a/smartjavaai-face/src/main/java/cn/smartjavaai/face/translator/FaceFeatureTranslator.java +++ b/smartjavaai-face/src/main/java/cn/smartjavaai/face/translator/FaceFeatureTranslator.java @@ -12,7 +12,7 @@ import ai.djl.translate.Translator; import ai.djl.translate.TranslatorContext; /** - * @author 邓文杰 + * @author dwj * @date 2025/3/31 */ public final class FaceFeatureTranslator implements Translator { diff --git a/smartjavaai-face/src/main/java/cn/smartjavaai/face/utils/FaceUtils.java b/smartjavaai-face/src/main/java/cn/smartjavaai/face/utils/FaceUtils.java new file mode 100644 index 0000000..692a2d0 --- /dev/null +++ b/smartjavaai-face/src/main/java/cn/smartjavaai/face/utils/FaceUtils.java @@ -0,0 +1,167 @@ +package cn.smartjavaai.face.utils; + +import ai.djl.modality.cv.Image; +import ai.djl.modality.cv.output.BoundingBox; +import ai.djl.modality.cv.output.DetectedObjects; +import cn.smartjavaai.common.entity.DetectionResponse; +import cn.smartjavaai.common.entity.Point; +import cn.smartjavaai.common.entity.DetectionRectangle; +import cn.smartjavaai.common.utils.ImageUtils; +import cn.smartjavaai.face.FaceModelConfig; +import cn.smartjavaai.face.exception.FaceException; +import com.seetaface.model.SeetaRect; + +import javax.imageio.ImageIO; +import java.awt.*; +import java.awt.image.BufferedImage; +import java.io.File; +import java.io.IOException; +import java.util.ArrayList; +import java.util.Iterator; +import java.util.List; +import java.util.Objects; +import java.util.stream.Collectors; + +/** + * 人脸检测相关工具类 + * @author dwj + * @date 2025/4/9 + */ +public class FaceUtils { + + + /** + * 转换为FaceDetectedResult + * @param detection + * @param img + * @return + */ + public static DetectionResponse convertToDetectionResponse(DetectedObjects detection, Image img){ + if(Objects.isNull(detection) || Objects.isNull(detection.getProbabilities()) + || detection.getProbabilities().isEmpty() || Objects.isNull(detection.items()) || detection.items().isEmpty()){ + return null; + } + DetectionResponse detectionResponse = new DetectionResponse(); + List detectedObjectList = detection.items(); + List rectangleList = new ArrayList(); + Iterator iterator = detectedObjectList.iterator(); + int index = 0; + while(iterator.hasNext()) { + DetectedObjects.DetectedObject result = (DetectedObjects.DetectedObject)iterator.next(); + BoundingBox box = result.getBoundingBox(); + int x = (int)(box.getBounds().getX() * (double)img.getWidth()); + int y = (int)(box.getBounds().getY() * (double)img.getHeight()); + int width = (int)(box.getBounds().getWidth() * (double)img.getWidth()); + int height = (int)(box.getBounds().getHeight() * (double)img.getHeight()); + DetectionRectangle rectangle = new DetectionRectangle(x, y, width, height, detection.getProbabilities().get(index).floatValue()); + rectangleList.add(rectangle); + index++; + } + detectionResponse.setRectangleList(rectangleList); + return detectionResponse; + } + + /** + * 转换为FaceDetectedResult + * @param seetaResult + * @return + */ + public static DetectionResponse convertToDetectionResponse(SeetaRect[] seetaResult, FaceModelConfig config){ + if(Objects.isNull(seetaResult) || seetaResult.length == 0){ + return null; + } + DetectionResponse detectionResponse = new DetectionResponse(); + List rectangleList = new ArrayList(); + for(SeetaRect rect : seetaResult){ + //过滤置信度 + if(config.getConfidenceThreshold() > 0 && rect.score < config.getConfidenceThreshold()){ + continue; + } + DetectionRectangle rectangle = new DetectionRectangle(rect.x, rect.y, rect.width, rect.height, rect.score); + rectangleList.add(rectangle); + } + detectionResponse.setRectangleList(rectangleList); + return detectionResponse; + } + + /** + * 绘制人脸框 + * @param sourceImage + * @param detectionResponse + * @param savePath + * @throws IOException + */ + public static void drawBoundingBoxes(BufferedImage sourceImage, DetectionResponse detectionResponse, String savePath) throws IOException { + if(!ImageUtils.isImageValid(sourceImage)){ + throw new FaceException("图像无效"); + } + if(Objects.isNull(detectionResponse) || Objects.isNull(detectionResponse.getRectangleList()) || detectionResponse.getRectangleList().isEmpty()){ + throw new FaceException("无目标数据"); + } + Graphics2D graphics = sourceImage.createGraphics(); + graphics.setColor(Color.RED);// 边框颜色 + graphics.setStroke(new BasicStroke(2)); // 线宽2像素 + graphics.setRenderingHint(RenderingHints.KEY_ANTIALIASING, + RenderingHints.VALUE_ANTIALIAS_ON); // 抗锯齿 + int stroke = 2; + for(DetectionRectangle rectangle : detectionResponse.getRectangleList()){ + graphics.setColor(Color.RED);// 边框颜色 + graphics.drawRect(rectangle.getX(), rectangle.getY(), rectangle.getWidth(), rectangle.getHeight()); + drawText(graphics, "face", rectangle.getX(), rectangle.getY(), stroke, 4); + } + graphics.dispose(); + ImageIO.write(sourceImage, "jpg", new File(savePath)); + } + + /** + * 绘制人脸框 + * @param sourceImage + * @param detectionResponse + * @throws IOException + */ + public static BufferedImage drawBoundingBoxes(BufferedImage sourceImage, DetectionResponse detectionResponse) throws IOException { + if(!ImageUtils.isImageValid(sourceImage)){ + throw new FaceException("图像无效"); + } + if(Objects.isNull(detectionResponse) || Objects.isNull(detectionResponse.getRectangleList()) || detectionResponse.getRectangleList().isEmpty()){ + throw new FaceException("无目标数据"); + } + Graphics2D graphics = sourceImage.createGraphics(); + graphics.setColor(Color.RED);// 边框颜色 + graphics.setStroke(new BasicStroke(2)); // 线宽2像素 + graphics.setRenderingHint(RenderingHints.KEY_ANTIALIASING, + RenderingHints.VALUE_ANTIALIAS_ON); // 抗锯齿 + int stroke = 2; + for(DetectionRectangle rectangle : detectionResponse.getRectangleList()){ + graphics.setColor(Color.RED);// 边框颜色 + graphics.drawRect(rectangle.getX(), rectangle.getY(), rectangle.getWidth(), rectangle.getHeight()); + drawText(graphics, "face", rectangle.getX(), rectangle.getY(), stroke, 4); + } + graphics.dispose(); + return sourceImage; + } + + /** + * 绘制文字 + * @param g + * @param text + * @param x + * @param y + * @param stroke + * @param padding + */ + private static void drawText(Graphics2D g, String text, int x, int y, int stroke, int padding) { + FontMetrics metrics = g.getFontMetrics(); + x += stroke / 2; + y += stroke / 2; + int width = metrics.stringWidth(text) + padding * 2 - stroke / 2; + int height = metrics.getHeight() + metrics.getDescent(); + int ascent = metrics.getAscent(); + java.awt.Rectangle background = new java.awt.Rectangle(x, y, width, height); + g.fill(background); + g.setPaint(Color.WHITE); + g.drawString(text, x + padding, y + ascent); + } + + +} diff --git a/smartjavaai-face/src/main/java/com/seetaface/SeetaFace6JNI.java b/smartjavaai-face/src/main/java/com/seetaface/SeetaFace6JNI.java index e3cfd6d..935eb93 100644 --- a/smartjavaai-face/src/main/java/com/seetaface/SeetaFace6JNI.java +++ b/smartjavaai-face/src/main/java/com/seetaface/SeetaFace6JNI.java @@ -124,4 +124,12 @@ public class SeetaFace6JNI { */ public native int predictImage(SeetaImageData img); + public native void dispose(); + + @Override + protected void finalize() throws Throwable { + super.finalize(); + this.dispose(); + } + } diff --git a/smartjavaai-objectdetection/pom.xml b/smartjavaai-objectdetection/pom.xml new file mode 100644 index 0000000..bc29850 --- /dev/null +++ b/smartjavaai-objectdetection/pom.xml @@ -0,0 +1,137 @@ + + + 4.0.0 + + ink.numberone + smartjavaai-parent + 1.0.8 + + + smartjavaai-objectdetection + 1.0.8 + smartjavaai-objectdetection + SmartJavaAI + https://github.com/geekwenjie/SmartJavaAI + + + MIT License + https://opensource.org/licenses/MIT + + + + + + 11 + 11 + UTF-8 + true + + + + + ink.numberone + smartjavaai-common + ${project.version} + + + + + + + + + + org.sonatype.central + central-publishing-maven-plugin + 0.4.0 + true + + dengwenjie + true + ${project.groupId}:${project.artifactId}:${project.version} + + + + + org.apache.maven.plugins + maven-source-plugin + 3.1.0 + + + attach-sources + + jar-no-fork + + + + + + org.apache.maven.plugins + maven-javadoc-plugin + 3.1.0 + + ${java.home}/bin/javadoc + none + + -Xdoclint:none + + + + + attach-javadocs + + jar + + + + + + org.apache.maven.plugins + maven-gpg-plugin + 3.1.0 + + + sign-artifacts + verify + + sign + + + + + + + + + + scm:git:git://github.com/geekwenjie/SmartJavaAI.git + scm:git:ssh://github.com/geekwenjie/SmartJavaAI.git + http://github.com/geekwenjie/SmartJavaAI/tree/master + + + + + + dengwenjie + https://s01.oss.sonatype.org/content/repositories/snapshots + + + dengwenjie + https://s01.oss.sonatype.org/service/local/staging/deploy/maven2/ + + + + + + dengwenjie + 775747758@qq.com + + Project Manager + Architect + + + + + diff --git a/smartjavaai-objectdetection/src/main/java/cn/smartjavaai/objectdetection/DetectorConfig.java b/smartjavaai-objectdetection/src/main/java/cn/smartjavaai/objectdetection/DetectorConfig.java new file mode 100644 index 0000000..a7502ac --- /dev/null +++ b/smartjavaai-objectdetection/src/main/java/cn/smartjavaai/objectdetection/DetectorConfig.java @@ -0,0 +1,14 @@ +package cn.smartjavaai.objectdetection; + +/** + * @author dwj + * @date 2025/4/7 + */ +public class DetectorConfig { + + /** + * 置信度阈值 + */ + public static final float DEFAULT_THRESHOLD = 0.5F; + +} diff --git a/smartjavaai-objectdetection/src/main/java/cn/smartjavaai/objectdetection/DetectorModelConfig.java b/smartjavaai-objectdetection/src/main/java/cn/smartjavaai/objectdetection/DetectorModelConfig.java new file mode 100644 index 0000000..c69d8c0 --- /dev/null +++ b/smartjavaai-objectdetection/src/main/java/cn/smartjavaai/objectdetection/DetectorModelConfig.java @@ -0,0 +1,29 @@ +package cn.smartjavaai.objectdetection; + +import cn.smartjavaai.common.enums.DeviceEnum; +import lombok.Data; + +/** + * 目标检测模型参数配置 + * + * @author dwj + * @date 2025/4/4 + */ +@Data +public class DetectorModelConfig { + + /** + * 模型名称 + */ + private DetectorModelEnum modelEnum; + + /** + * 置信度阈值 + */ + private float threshold = DetectorConfig.DEFAULT_THRESHOLD; + + /** + * 设备类型 + */ + private DeviceEnum device; +} diff --git a/smartjavaai-objectdetection/src/main/java/cn/smartjavaai/objectdetection/DetectorModelEnum.java b/smartjavaai-objectdetection/src/main/java/cn/smartjavaai/objectdetection/DetectorModelEnum.java new file mode 100644 index 0000000..8a8d15a --- /dev/null +++ b/smartjavaai-objectdetection/src/main/java/cn/smartjavaai/objectdetection/DetectorModelEnum.java @@ -0,0 +1,62 @@ +package cn.smartjavaai.objectdetection; + +/** + * 目标检测模型枚举 + * @author dwj + * @date 2025/4/4 + */ +public enum DetectorModelEnum { + + // resnet50 系列 + SSD_300_RESNET50("ai.djl.pytorch/ssd/0.0.1/ssd_300_resnet50"), + SSD_512_RESNET50_V1_VOC("ai.djl.mxnet/ssd/0.0.1/ssd_512_resnet50_v1_voc"), + + // vgg16 系列 + SSD_512_VGG16_ATROUS_COCO("ai.djl.mxnet/ssd/0.0.1/ssd_512_vgg16_atrous_coco"), + SSD_300_VGG16_ATROUS_VOC("ai.djl.mxnet/ssd/0.0.1/ssd_300_vgg16_atrous_voc"), + + // mobilenet 系列 + SSD_512_MOBILENET1_VOC("ai.djl.mxnet/ssd/0.0.1/ssd_512_mobilenet1.0_voc"), + + // YOLO 系列 + YOLOV8N("ai.djl.pytorch/yolov8n/0.0.1/yolov8n"), + YOLO11N("ai.djl.pytorch/yolo11n/0.0.1/yolo11n"), + YOLOV5S("ai.djl.pytorch/yolo5s/0.0.1/yolov5s"), + YOLOV5S_ONNXRUNTIME("ai.djl.onnxruntime/yolo5s/0.0.1/yolo5s"), + YOLO("ai.djl.mxnet/yolo/0.0.1/yolo"), + + // YOLOv3 变体 + YOLO3_DARKNET_VOC_416("ai.djl.mxnet/yolo/0.0.1/yolo3_darknet_voc_416"), + YOLO3_MOBILENET_VOC_320("ai.djl.mxnet/yolo/0.0.1/yolo3_mobilenet_voc_320"), + YOLO3_MOBILENET_VOC_416("ai.djl.mxnet/yolo/0.0.1/yolo3_mobilenet_voc_41"), + YOLO3_DARKNET_COCO_320("ai.djl.mxnet/yolo/0.0.1/yolo3_darknet_coco_320"), + YOLO3_DARKNET_COCO_416("ai.djl.mxnet/yolo/0.0.1/yolo3_darknet_coco_416"), + YOLO3_DARKNET_COCO_608("ai.djl.mxnet/yolo/0.0.1/yolo3_darknet_coco_608"), + YOLO3_MOBILENET_COCO_320("ai.djl.mxnet/yolo/0.0.1/yolo3_mobilenet_coco_320"), + YOLO3_MOBILENET_COCO_416("ai.djl.mxnet/yolo/0.0.1/yolo3_mobilenet_coco_416"), + YOLO3_MOBILENET_COCO_608("ai.djl.mxnet/yolo/0.0.1/yolo3_mobilenet_coco_608"); + + /** + * 根据名称获取枚举 (忽略大小写和下划线变体) + */ + public static DetectorModelEnum fromName(String name) { + String formatted = name.trim().toUpperCase().replaceAll("[-_]", ""); + for (DetectorModelEnum model : values()) { + if (model.name().replaceAll("_", "").equals(formatted)) { + return model; + } + } + throw new IllegalArgumentException("未知模型名称: " + name); + } + + private final String modelUri; + + DetectorModelEnum(String modelUri) { + this.modelUri = modelUri; + } + + public String getModelUri() { + return modelUri; + } + +} diff --git a/smartjavaai-objectdetection/src/main/java/cn/smartjavaai/objectdetection/exception/DetectionException.java b/smartjavaai-objectdetection/src/main/java/cn/smartjavaai/objectdetection/exception/DetectionException.java new file mode 100644 index 0000000..de73605 --- /dev/null +++ b/smartjavaai-objectdetection/src/main/java/cn/smartjavaai/objectdetection/exception/DetectionException.java @@ -0,0 +1,30 @@ +package cn.smartjavaai.objectdetection.exception; + +/** + * 目标检测异常 + * @author dwj + * @date 2025/4/4 + */ +public class DetectionException extends RuntimeException{ + + public DetectionException() { + super(); + } + + public DetectionException(String message, Throwable cause, boolean enableSuppression, boolean writableStackTrace) { + super(message, cause, enableSuppression, writableStackTrace); + } + + public DetectionException(String message, Throwable cause) { + super(message, cause); + } + + public DetectionException(String message) { + super(message); + } + + public DetectionException(Throwable cause) { + super(cause); + } + +} diff --git a/smartjavaai-objectdetection/src/main/java/cn/smartjavaai/objectdetection/model/DetectorModel.java b/smartjavaai-objectdetection/src/main/java/cn/smartjavaai/objectdetection/model/DetectorModel.java new file mode 100644 index 0000000..307aa92 --- /dev/null +++ b/smartjavaai-objectdetection/src/main/java/cn/smartjavaai/objectdetection/model/DetectorModel.java @@ -0,0 +1,216 @@ +package cn.smartjavaai.objectdetection.model; + +import ai.djl.Application; +import ai.djl.Device; +import ai.djl.MalformedModelException; +import ai.djl.inference.Predictor; +import ai.djl.modality.cv.Image; +import ai.djl.modality.cv.ImageFactory; +import ai.djl.modality.cv.output.DetectedObjects; +import ai.djl.repository.zoo.Criteria; +import ai.djl.repository.zoo.ModelNotFoundException; +import ai.djl.repository.zoo.ZooModel; +import ai.djl.training.util.ProgressBar; +import ai.djl.translate.TranslateException; +import cn.smartjavaai.common.entity.DetectionResponse; +import cn.smartjavaai.common.enums.DeviceEnum; +import cn.smartjavaai.common.pool.ModelPredictorPoolManager; +import cn.smartjavaai.common.pool.PredictorFactory; +import cn.smartjavaai.common.utils.FileUtils; +import cn.smartjavaai.common.utils.ImageUtils; +import cn.smartjavaai.objectdetection.DetectorConfig; +import cn.smartjavaai.objectdetection.DetectorModelConfig; +import cn.smartjavaai.objectdetection.exception.DetectionException; +import cn.smartjavaai.objectdetection.utils.DetectorUtils; +import lombok.extern.slf4j.Slf4j; +import org.apache.commons.lang3.Validate; +import org.apache.commons.pool2.ObjectPool; +import org.apache.commons.pool2.impl.GenericObjectPool; +import org.apache.commons.pool2.impl.GenericObjectPoolConfig; + +import javax.imageio.ImageIO; +import java.awt.image.BufferedImage; +import java.io.*; +import java.nio.file.Path; +import java.nio.file.Paths; +import java.time.Duration; +import java.util.Objects; + +/** + * 目标检测模型 + * @author dwj + * @date 2025/4/4 + */ +@Slf4j +public class DetectorModel implements AutoCloseable{ + + private ZooModel model; + + //private Predictor predictor; + + private static final String DJL_MODEL_PREFIX = "djl://"; + + private ObjectPool> predictorPool; + + public void loadModel(DetectorModelConfig config){ + Device device = null; + if(!Objects.isNull(config.getDevice())){ + device = config.getDevice() == DeviceEnum.CPU ? Device.cpu() : Device.gpu(); + } + Criteria criteria = Criteria.builder() + .optApplication(Application.CV.OBJECT_DETECTION) + .setTypes(Image.class, DetectedObjects.class) + .optArgument("threshold", config.getThreshold() > 0 ? config.getThreshold() : DetectorConfig.DEFAULT_THRESHOLD) + .optModelUrls(DJL_MODEL_PREFIX + config.getModelEnum().getModelUri()) + .optDevice(device) + .optProgress(new ProgressBar()) + .build(); + try { + model = criteria.loadModel(); + // 创建池子：每个线程独享 Predictor + this.predictorPool = new GenericObjectPool<>(new PredictorFactory<>(model)); + log.info("当前设备: " + model.getNDManager().getDevice()); + } catch (IOException | ModelNotFoundException | MalformedModelException e) { + throw new DetectionException("模型加载失败", e); + } + } + + /** + * 目标检测 + * @param imagePath + * @return + * @throws Exception + */ + public DetectionResponse detect(String imagePath){ + if(!FileUtils.isFileExists(imagePath)){ + throw new DetectionException("图像文件不存在"); + } + Image image = null; + try { + image = ImageFactory.getInstance().fromFile(Paths.get(imagePath)); + } catch (IOException e) { + throw new DetectionException("图片转换错误", e); + } + DetectedObjects detectedObjects = detect(image); + return DetectorUtils.convertToDetectionResponse(detectedObjects, image); + } + + + /** + * 目标检测-将检测结果绘制到原图 + * @param imagePath + * @return + */ + public void detectAndDraw(String imagePath, String outputPath){ + if(!FileUtils.isFileExists(imagePath)){ + throw new DetectionException("图像文件不存在"); + } + try { + Image img = ImageFactory.getInstance().fromFile(Paths.get(imagePath)); + DetectedObjects detectedObjects = detect(img); + img.drawBoundingBoxes(detectedObjects); + ByteArrayOutputStream outputStream = new ByteArrayOutputStream(); + // 调用 save 方法将 Image 写入字节流 + img.save(new FileOutputStream(Paths.get(outputPath).toAbsolutePath().toString()), "png"); + } catch (IOException e) { + throw new DetectionException(e); + } + } + + /** + * 目标检测 + * @param imageData + * @return + */ + public DetectionResponse detect(byte[] imageData){ + if(Objects.isNull(imageData)){ + throw new DetectionException("图像无效"); + } + try { + BufferedImage image = ImageIO.read(new ByteArrayInputStream(imageData)); + return detect(image); + } catch (IOException e) { + throw new DetectionException("错误的图像", e); + } + + } + + + /** + * 目标检测 + * @param image + * @return + */ + public DetectionResponse detect(BufferedImage image){ + if(!ImageUtils.isImageValid(image)){ + throw new DetectionException("图像无效"); + } + Image img = ImageFactory.getInstance().fromImage(image); + DetectedObjects detectedObjects = detect(img); + return DetectorUtils.convertToDetectionResponse(detectedObjects, img); + } + + /** + * 目标检测-将检测结果绘制到原图 + * @param sourceImage + * @return + */ + public BufferedImage detectAndDraw(BufferedImage sourceImage){ + if(!ImageUtils.isImageValid(sourceImage)){ + throw new DetectionException("图像无效"); + } + Image img = ImageFactory.getInstance().fromImage(sourceImage); + DetectedObjects detectedObjects = detect(img); + img.drawBoundingBoxes(detectedObjects); + try { + ByteArrayOutputStream outputStream = new ByteArrayOutputStream(); + // 调用 save 方法将 Image 写入字节流 + img.save(outputStream, "png"); + // 将字节流转换为 BufferedImage + byte[] imageBytes = outputStream.toByteArray(); + return ImageIO.read(new ByteArrayInputStream(imageBytes)); + } catch (IOException e) { + throw new DetectionException("导出图片失败", e); + } + } + + /** + * 目标检测 + * @param image + * @return + */ + private DetectedObjects detect(Image image){ + Predictor predictor = null; + try { + predictor = predictorPool.borrowObject(); + return predictor.predict(image); + } catch (Exception e) { + throw new DetectionException("目标检测错误", e); + }finally { + if (predictor != null) { + try { + predictorPool.returnObject(predictor); //归还 + log.info("释放资源"); + } catch (Exception e) { + log.warn("归还Predictor失败", e); + try { + predictor.close(); // 归还失败才销毁 + } catch (Exception ex) { + log.error("关闭Predictor失败", ex); + } + } + } + } + } + + + /** + * 显式释放资源（必须调用！） + */ + @Override + public void close() { + if (predictorPool != null) { + predictorPool.close(); + } + } +} diff --git a/smartjavaai-objectdetection/src/main/java/cn/smartjavaai/objectdetection/model/ObjectDetectionModelFactory.java b/smartjavaai-objectdetection/src/main/java/cn/smartjavaai/objectdetection/model/ObjectDetectionModelFactory.java new file mode 100644 index 0000000..48c9d23 --- /dev/null +++ b/smartjavaai-objectdetection/src/main/java/cn/smartjavaai/objectdetection/model/ObjectDetectionModelFactory.java @@ -0,0 +1,73 @@ +package cn.smartjavaai.objectdetection.model; + +import cn.smartjavaai.objectdetection.DetectorModelConfig; +import cn.smartjavaai.objectdetection.DetectorModelEnum; +import cn.smartjavaai.objectdetection.exception.DetectionException; +import lombok.extern.slf4j.Slf4j; + +import java.util.Map; +import java.util.Objects; +import java.util.concurrent.ConcurrentHashMap; + +/** + * 目标检测 模型工厂 + * @author dwj + */ +@Slf4j +public class ObjectDetectionModelFactory { + + // 使用 volatile 和双重检查锁定来确保线程安全的单例模式 + private static volatile ObjectDetectionModelFactory instance; + + private static final ConcurrentHashMap modelMap = new ConcurrentHashMap<>(); + + // 私有构造函数，防止外部创建实例 + private ObjectDetectionModelFactory() {} + + // 双重检查锁定的单例方法 + public static ObjectDetectionModelFactory getInstance() { + if (instance == null) { + synchronized (ObjectDetectionModelFactory.class) { + if (instance == null) { + instance = new ObjectDetectionModelFactory(); + } + } + } + return instance; + } + + /** + * 获取模型（通过配置） + * @param config + * @return + */ + public DetectorModel getModel(DetectorModelConfig config) { + if(Objects.isNull(config) || Objects.isNull(config.getModelEnum())){ + throw new DetectionException("未配置模型"); + } + return modelMap.computeIfAbsent(config.getModelEnum().name(), k -> { + DetectorModel model = new DetectorModel(); + model.loadModel(config); + return model; + }); + } + + /** + * 获取默认模型 + * @return + */ + public DetectorModel getModel() { + // 初始化默认配置 + DetectorModelConfig config = new DetectorModelConfig(); + config.setModelEnum(DetectorModelEnum.YOLO11N); + return getModel(config); + } + + /** + * 关闭所有已加载的模型 + */ + public void closeAll() { + modelMap.values().forEach(DetectorModel::close); + } +} + diff --git a/smartjavaai-objectdetection/src/main/java/cn/smartjavaai/objectdetection/utils/DetectorUtils.java b/smartjavaai-objectdetection/src/main/java/cn/smartjavaai/objectdetection/utils/DetectorUtils.java new file mode 100644 index 0000000..0f06279 --- /dev/null +++ b/smartjavaai-objectdetection/src/main/java/cn/smartjavaai/objectdetection/utils/DetectorUtils.java @@ -0,0 +1,62 @@ +package cn.smartjavaai.objectdetection.utils; + +import ai.djl.modality.cv.Image; +import ai.djl.modality.cv.output.BoundingBox; +import ai.djl.modality.cv.output.DetectedObjects; +import cn.smartjavaai.common.entity.DetectionRectangle; +import cn.smartjavaai.common.entity.DetectionResponse; +import cn.smartjavaai.common.utils.ImageUtils; + +import javax.imageio.ImageIO; +import java.awt.*; +import java.awt.image.BufferedImage; +import java.io.File; +import java.io.IOException; +import java.util.ArrayList; +import java.util.Iterator; +import java.util.List; +import java.util.Objects; + +/** + * 目标检测相关工具类 + * @author dwj + * @date 2025/4/9 + */ +public class DetectorUtils { + + + /** + * 转换为FaceDetectedResult + * @param detection + * @param img + * @return + */ + public static DetectionResponse convertToDetectionResponse(DetectedObjects detection, Image img){ + if(Objects.isNull(detection) || Objects.isNull(detection.getProbabilities()) + || detection.getProbabilities().isEmpty() || Objects.isNull(detection.items()) || detection.items().isEmpty()){ + return null; + } + DetectionResponse detectionResponse = new DetectionResponse(); + List rectangleList = new ArrayList(); + List detectedObjectList = detection.items(); + Iterator iterator = detectedObjectList.iterator(); + int index = 0; + while(iterator.hasNext()) { + DetectedObjects.DetectedObject result = (DetectedObjects.DetectedObject)iterator.next(); + String className = result.getClassName(); + BoundingBox box = result.getBoundingBox(); + int x = (int)(box.getBounds().getX() * (double)img.getWidth()); + int y = (int)(box.getBounds().getY() * (double)img.getHeight()); + int width = (int)(box.getBounds().getWidth() * (double)img.getWidth()); + int height = (int)(box.getBounds().getHeight() * (double)img.getHeight()); + DetectionRectangle rectangle = new DetectionRectangle(x, y, width, height, detection.getProbabilities().get(index).floatValue(),className); + rectangleList.add(rectangle); + index++; + } + detectionResponse.setRectangleList(rectangleList); + return detectionResponse; + } + + + +} diff --git a/smartjavaai-seetaface6-lib/pom.xml b/smartjavaai-seetaface6-lib/pom.xml index de5d13e..865a063 100644 --- a/smartjavaai-seetaface6-lib/pom.xml +++ b/smartjavaai-seetaface6-lib/pom.xml @@ -6,7 +6,7 @@ ink.numberone smartjavaai-parent - 1.0.6 + 1.0.8 smartjavaai-seetaface6-lib