mirror of
https://github.com/geekwenjie/SmartJavaAI.git
synced 2026-09-09 19:18:52 +00:00
- 新增 语音识别模块,集成 OpenAI 开源的 Whisper 和 Vosk
- 修复 质量评估模型的 Bug - 修复 OCR 模块 recognizeAndDraw 方法的 Bug - 修复 车牌识别在未检测到车牌时的报错问题 - 优化 OCR 表格识别功能,新增导出方式
This commit is contained in:
80
README.md
80
README.md
@@ -205,7 +205,7 @@ SmartJavaAI是专为JAVA 开发者打造的一个功能丰富、开箱即用的
|
||||
<img src="https://cdn.jsdelivr.net/gh/geekwenjie/SmartJavaAI-Site/images/ocr/plate_recognized.jpg" width = "500px"/>
|
||||
</div>
|
||||
</td>
|
||||
<td>
|
||||
<td>
|
||||
<div align="center">
|
||||
<img src="https://cdn.jsdelivr.net/gh/geekwenjie/SmartJavaAI-Site/images/ocr/plate_recognized2.jpg" width = "500px"/>
|
||||
</div>
|
||||
@@ -224,6 +224,19 @@ SmartJavaAI是专为JAVA 开发者打造的一个功能丰富、开箱即用的
|
||||
</div>
|
||||
</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td>
|
||||
<div align="left">
|
||||
<p>语音识别</p>
|
||||
- 支持100种语言
|
||||
</div>
|
||||
</td>
|
||||
<td>
|
||||
<div align="center">
|
||||
<img src="https://cdn.jsdelivr.net/gh/geekwenjie/SmartJavaAI-Site/images/speech/asr.png" width = "500px"/>
|
||||
</div>
|
||||
</td>
|
||||
</tr>
|
||||
</table>
|
||||
</div>
|
||||
|
||||
@@ -273,6 +286,9 @@ SmartJavaAI是专为JAVA 开发者打造的一个功能丰富、开箱即用的
|
||||
- 支持中文车牌识别:单层/双层检测,颜色识别,支持12种中文车牌
|
||||
- **机器翻译**
|
||||
- 集成NLLB-200模型:支持200+语言互相翻译
|
||||
- **语音识别**
|
||||
- 集成openai的whisper模型:支持100种语言
|
||||
- 集成vosk语音识别
|
||||
|
||||
|
||||
## 🌟 AI集成方式对比
|
||||
@@ -291,13 +307,15 @@ SmartJavaAI是专为JAVA 开发者打造的一个功能丰富、开箱即用的
|
||||
|
||||
## 🛠️包含组件
|
||||
|
||||
| 模块 | 介绍 |
|
||||
|-----------------------------|---------------------------|
|
||||
| smartjavaai-common | 基础通用模块,封装了公共功能,供各算法模块共享使用 |
|
||||
| smartjavaai-face | 人脸功能模块 |
|
||||
| smartjavaai-objectdetection | 目标检测模块 |
|
||||
| smartjavaai-ocr | OCR文字识别模块 |
|
||||
| smartjavaai-translate | 机器翻译模块 |
|
||||
| 模块 | 介绍 |
|
||||
|-----------------------------|------------------------------|
|
||||
| smartjavaai-common | 基础通用模块,封装了公共功能,供各算法模块共享使用 |
|
||||
| smartjavaai-bom | 依赖管理模块 |
|
||||
| smartjavaai-face | 人脸功能模块 |
|
||||
| smartjavaai-objectdetection | 目标检测模块 |
|
||||
| smartjavaai-ocr | OCR文字识别模块 |
|
||||
| smartjavaai-translate | 机器翻译模块 |
|
||||
| smartjavaai-speech | 语音功能模块,包含 ASR 和 TTS |
|
||||
|
||||
可以根据需求对每个模块单独引入,也可以通过引入`smartjavaai-all`方式引入所有模块。
|
||||
|
||||
@@ -325,7 +343,7 @@ SmartJavaAI是专为JAVA 开发者打造的一个功能丰富、开箱即用的
|
||||
<dependency>
|
||||
<groupId>cn.smartjavaai</groupId>
|
||||
<artifactId>smartjavaai-all</artifactId>
|
||||
<version>1.0.22</version>
|
||||
<version>1.0.23</version>
|
||||
</dependency>
|
||||
```
|
||||
### 3、完整示例代码
|
||||
@@ -540,6 +558,18 @@ SmartJavaAI是专为JAVA 开发者打造的一个功能丰富、开箱即用的
|
||||
| ------------------------------ | ------------------------ |-----------------------------------------------|
|
||||
| NLLB-200 | Meta AI 开发的一个先进的单一多语言机器翻译模型 | [Github](https://github.com/facebookresearch/fairseq/tree/nllb) |
|
||||
|
||||
---
|
||||
|
||||
#### 语音识别模型
|
||||
|
||||
这里仅介绍模型的开源项目,每个开源项目通常包含多个具体模型,本文不逐一列出。
|
||||
|
||||
| 模型名称 | 模型简介 | 模型官网 |
|
||||
|---------| ------------------------ |-----------------------------------------------|
|
||||
| Whisper | OpenAI 开源的通用语音识别(ASR)模型,支持多语言转写和翻译,具有较高的识别精度,尤其在嘈杂环境中表现良好,适合离线和批量音频处理。 | [Github](https://github.com/ggml-org/whisper.cpp) |
|
||||
| Vosk | 一个轻量级离线语音识别工具包,支持多种语言和平台(包括移动端与嵌入式设备),可在低资源环境中运行,适合实时语音识别场景。 | [Github](https://github.com/alphacep/vosk-api) |
|
||||
|
||||
|
||||
---
|
||||
|
||||
## 🙏 鸣谢
|
||||
@@ -559,9 +589,24 @@ SmartJavaAI是专为JAVA 开发者打造的一个功能丰富、开箱即用的
|
||||
|
||||
🚀 **如果这个项目对你有帮助,别忘了点个 Star ⭐!你的支持是我持续优化升级的动力!** ❤️
|
||||
|
||||
## 献代码的步骤
|
||||
|
||||
1、在Gitee或者Github/Gitcode上fork项目到自己的repo
|
||||
2、把fork过去的项目也就是你的项目clone到你的本地
|
||||
3、修改代码(记得一定要修改dev分支)
|
||||
4、commit后push到自己的库(dev分支)
|
||||
5、登录Gitee或Github/Gitcode在你首页可以看到一个 pull request 按钮,点击它,填写一些说明信息,然后提交即可。
|
||||
6、等待维护者合并
|
||||
|
||||
## 近期更新日志
|
||||
|
||||
## [v1.0.23] - 2025-08-09
|
||||
- 新增 语音识别模块,集成 OpenAI 开源的 Whisper 和 Vosk
|
||||
- 修复 质量评估模型的 Bug
|
||||
- 修复 OCR 模块 recognizeAndDraw 方法的 Bug
|
||||
- 修复 车牌识别在未检测到车牌时的报错问题
|
||||
- 优化 OCR 表格识别功能,新增导出方式
|
||||
|
||||
## [v1.0.22] - 2025-07-28
|
||||
- 新增 Milvus 身份验证支持
|
||||
- 集成车牌识别模型,支持车牌检测与识别
|
||||
@@ -590,21 +635,4 @@ SmartJavaAI是专为JAVA 开发者打造的一个功能丰富、开箱即用的
|
||||
- 人脸模块:修复人脸更新后的缓存异常问题
|
||||
- 其他:优化部分功能与细节体验
|
||||
|
||||
## [v1.0.17] - 2025-06-18
|
||||
- 新增机器翻译模块:支持 200+ 种语言之间的相互翻译
|
||||
- 人脸识别模块:修复批量删除人脸数据时的异常问题
|
||||
- 人脸识别模块:修复人脸检索 Top大于 1 时报异常问题
|
||||
|
||||
## [v1.0.16] - 2025-06-09
|
||||
- 人脸模块:人脸查询支持 Milvus 和 SQLite
|
||||
- 人脸模块:FaceNet人脸模型也支持人脸注册,查询等功能
|
||||
- 人脸模块:Seetaface6 自动下载人脸库
|
||||
- 人脸模块:Seetaface6解决依赖库重复下载问题
|
||||
- 人脸模块:支持手动加载人脸库
|
||||
- 人脸模块:人脸识别相关功能支持更多参数
|
||||
|
||||
|
||||
## [v1.0.15] - 2025-05-17
|
||||
- 新增OCR文字识别模块:支持最新 PP-OCRv5
|
||||
- OCR文本识别:支持文字方向检测与自动校正
|
||||
|
||||
|
||||
@@ -12,7 +12,7 @@
|
||||
<maven.compiler.source>11</maven.compiler.source>
|
||||
<maven.compiler.target>11</maven.compiler.target>
|
||||
<project.build.sourceEncoding>UTF-8</project.build.sourceEncoding>
|
||||
<smartjavaai.version>1.0.22</smartjavaai.version>
|
||||
<smartjavaai.version>1.0.23</smartjavaai.version>
|
||||
<!--如果打包运行,需要替换成你的main-->
|
||||
<exec.mainClass>smartai.examples.face.facedet.FaceDetDemo</exec.mainClass>
|
||||
|
||||
@@ -95,16 +95,8 @@
|
||||
<dependency>
|
||||
<groupId>cn.smartjavaai</groupId>
|
||||
<artifactId>smartjavaai-face</artifactId>
|
||||
<version>1.0.22</version>
|
||||
</dependency>
|
||||
|
||||
<dependency>
|
||||
<groupId>cn.smartjavaai</groupId>
|
||||
<artifactId>smartjavaai-all</artifactId>
|
||||
<version>1.0.22</version>
|
||||
</dependency>
|
||||
|
||||
|
||||
|
||||
<dependency>
|
||||
<groupId>ai.djl.pytorch</groupId>
|
||||
|
||||
@@ -1,5 +1,6 @@
|
||||
package smartai.examples.face.attribute;
|
||||
|
||||
import cn.smartjavaai.common.config.Config;
|
||||
import cn.smartjavaai.common.entity.DetectionInfo;
|
||||
import cn.smartjavaai.common.entity.DetectionResponse;
|
||||
import cn.smartjavaai.common.entity.R;
|
||||
@@ -16,6 +17,7 @@ import cn.smartjavaai.face.model.facedect.FaceDetModel;
|
||||
import cn.smartjavaai.face.utils.FaceUtils;
|
||||
import com.alibaba.fastjson.JSONObject;
|
||||
import lombok.extern.slf4j.Slf4j;
|
||||
import org.junit.BeforeClass;
|
||||
import org.junit.Test;
|
||||
|
||||
import javax.imageio.ImageIO;
|
||||
@@ -33,6 +35,12 @@ import java.util.List;
|
||||
@Slf4j
|
||||
public class FaceAttributeDetDemo {
|
||||
|
||||
@BeforeClass
|
||||
public static void beforeAll() throws IOException {
|
||||
//修改缓存路径
|
||||
// Config.setCachePath("/Users/xxx/smartjavaai_cache");
|
||||
}
|
||||
|
||||
|
||||
public FaceAttributeModel getFaceAttributeModel() {
|
||||
FaceAttributeConfig config = new FaceAttributeConfig();
|
||||
|
||||
@@ -2,6 +2,7 @@ package smartai.examples.face.expression;
|
||||
|
||||
import ai.djl.modality.cv.Image;
|
||||
import ai.djl.modality.cv.ImageFactory;
|
||||
import cn.smartjavaai.common.config.Config;
|
||||
import cn.smartjavaai.common.entity.DetectionInfo;
|
||||
import cn.smartjavaai.common.entity.DetectionRectangle;
|
||||
import cn.smartjavaai.common.entity.DetectionResponse;
|
||||
@@ -26,6 +27,7 @@ import cn.smartjavaai.face.model.liveness.LivenessDetModel;
|
||||
import com.alibaba.fastjson.JSONObject;
|
||||
import lombok.extern.slf4j.Slf4j;
|
||||
import nu.pattern.OpenCV;
|
||||
import org.junit.BeforeClass;
|
||||
import org.junit.Test;
|
||||
import org.opencv.core.Mat;
|
||||
import org.opencv.core.Size;
|
||||
@@ -54,6 +56,12 @@ public class ExpressionRecDemo {
|
||||
//设备类型
|
||||
public static DeviceEnum device = DeviceEnum.CPU;
|
||||
|
||||
@BeforeClass
|
||||
public static void beforeAll() throws IOException {
|
||||
//修改缓存路径
|
||||
// Config.setCachePath("/Users/xxx/smartjavaai_cache");
|
||||
}
|
||||
|
||||
/**
|
||||
* 获取人脸检测模型
|
||||
* @return
|
||||
|
||||
@@ -2,6 +2,7 @@ package smartai.examples.face.facedet;
|
||||
|
||||
import ai.djl.modality.cv.Image;
|
||||
import ai.djl.modality.cv.ImageFactory;
|
||||
import cn.smartjavaai.common.config.Config;
|
||||
import cn.smartjavaai.common.entity.DetectionInfo;
|
||||
import cn.smartjavaai.common.entity.DetectionRectangle;
|
||||
import cn.smartjavaai.common.entity.DetectionResponse;
|
||||
@@ -19,6 +20,7 @@ import cn.smartjavaai.face.model.liveness.LivenessDetModel;
|
||||
import com.alibaba.fastjson.JSONObject;
|
||||
import lombok.extern.slf4j.Slf4j;
|
||||
import nu.pattern.OpenCV;
|
||||
import org.junit.BeforeClass;
|
||||
import org.junit.Test;
|
||||
import org.opencv.core.Mat;
|
||||
import org.opencv.core.Size;
|
||||
@@ -32,6 +34,7 @@ import javax.swing.*;
|
||||
import java.awt.*;
|
||||
import java.awt.image.BufferedImage;
|
||||
import java.io.File;
|
||||
import java.io.IOException;
|
||||
import java.nio.file.Paths;
|
||||
|
||||
/**
|
||||
@@ -47,6 +50,12 @@ public class FaceDetDemo {
|
||||
|
||||
public static String imgPath = "src/main/resources/iu_1.jpg";
|
||||
|
||||
@BeforeClass
|
||||
public static void beforeAll() throws IOException {
|
||||
//修改缓存路径
|
||||
// Config.setCachePath("/Users/xxx/smartjavaai_cache");
|
||||
}
|
||||
|
||||
|
||||
/**
|
||||
* 获取人脸检测模型
|
||||
@@ -55,7 +64,10 @@ public class FaceDetDemo {
|
||||
*/
|
||||
public FaceDetModel getFaceDetModel(){
|
||||
FaceDetConfig config = new FaceDetConfig();
|
||||
//高精度模型,速度慢
|
||||
config.setModelEnum(FaceDetModelEnum.RETINA_FACE);//人脸检测模型
|
||||
//下载模型并替换本地路径,下载地址:https://pan.baidu.com/s/10l22x5fRz_gwLr8EAHa1Jg?pwd=1234 提取码: 1234
|
||||
config.setModelPath("/Users/xxx/Documents/develop/model/retinaface.pt");
|
||||
config.setConfidenceThreshold(FaceDetectConstant.DEFAULT_CONFIDENCE_THRESHOLD);//只返回相似度大于该值的人脸
|
||||
config.setNmsThresh(FaceDetectConstant.NMS_THRESHOLD);//用于去除重复的人脸框,当两个框的重叠度超过该值时,只保留一个
|
||||
return FaceDetModelFactory.getInstance().getModel(config);
|
||||
@@ -70,7 +82,7 @@ public class FaceDetDemo {
|
||||
FaceDetConfig config = new FaceDetConfig();
|
||||
//指定模型
|
||||
config.setModelEnum(FaceDetModelEnum.SEETA_FACE6_MODEL);
|
||||
//指定模型路径:请根据实际情况替换为本地模型文件的绝对路径(模型下载地址请查看文档)
|
||||
//指定模型路径:请根据实际情况替换为本地模型文件的绝对路径(下载地址:https://pan.baidu.com/s/10l22x5fRz_gwLr8EAHa1Jg?pwd=1234 提取码: 1234)
|
||||
config.setModelPath("C:/Users/Administrator/Downloads/sf3.0_models/sf3.0_models");
|
||||
return FaceDetModelFactory.getInstance().getModel(config);
|
||||
}
|
||||
@@ -153,28 +165,6 @@ public class FaceDetDemo {
|
||||
|
||||
}
|
||||
|
||||
/**
|
||||
* 人脸检测(离线模型)
|
||||
*/
|
||||
@Test
|
||||
public void testDetectFaceOffine(){
|
||||
try {
|
||||
FaceDetConfig config = new FaceDetConfig();
|
||||
config.setModelEnum(FaceDetModelEnum.RETINA_FACE);//人脸模型
|
||||
//模型路径,不同模型下载路径请参看文档
|
||||
config.setModelPath("/Users/xxx/Documents/develop/face_model/retinaface.pt");
|
||||
FaceDetModel faceModel = FaceDetModelFactory.getInstance().getModel(config);
|
||||
R<DetectionResponse> detectedResult = faceModel.detect(imgPath);
|
||||
if(detectedResult.isSuccess()){
|
||||
log.info("人脸检测结果:{}", JSONObject.toJSONString(detectedResult.getData()));
|
||||
}else{
|
||||
log.info("人脸检测失败:{}", detectedResult.getMessage());
|
||||
}
|
||||
} catch (Exception e) {
|
||||
e.printStackTrace();
|
||||
}
|
||||
}
|
||||
|
||||
/**
|
||||
* 人脸检测(GPU模式)
|
||||
*/
|
||||
@@ -182,7 +172,10 @@ public class FaceDetDemo {
|
||||
public void testDetectFaceGPU(){
|
||||
try {
|
||||
FaceDetConfig config = new FaceDetConfig();
|
||||
config.setModelEnum(FaceDetModelEnum.RETINA_FACE);//人脸模型
|
||||
//高精度模型,速度慢
|
||||
config.setModelEnum(FaceDetModelEnum.RETINA_FACE);//人脸检测模型
|
||||
//下载模型并替换本地路径,下载地址:https://pan.baidu.com/s/10l22x5fRz_gwLr8EAHa1Jg?pwd=1234 提取码: 1234
|
||||
config.setModelPath("/Users/xxx/Documents/develop/model/retinaface.pt");
|
||||
config.setDevice(DeviceEnum.GPU);
|
||||
FaceDetModel faceModel = FaceDetModelFactory.getInstance().getModel(config);
|
||||
R<DetectionResponse> detectedResult = faceModel.detect(imgPath);
|
||||
@@ -223,7 +216,7 @@ public class FaceDetDemo {
|
||||
@Test
|
||||
public void testDetectCamera(){
|
||||
try {
|
||||
FaceDetModel faceModel = getFaceDetModel();
|
||||
FaceDetModel faceModel = getSeetaface6DetModel();
|
||||
OpenCV.loadShared();
|
||||
VideoCapture capture = new VideoCapture(0);
|
||||
if (!capture.isOpened()) {
|
||||
|
||||
@@ -1,5 +1,6 @@
|
||||
package smartai.examples.face.facerec;
|
||||
|
||||
import cn.smartjavaai.common.config.Config;
|
||||
import cn.smartjavaai.common.entity.DetectionResponse;
|
||||
import cn.smartjavaai.common.entity.R;
|
||||
import cn.smartjavaai.common.entity.face.FaceSearchResult;
|
||||
@@ -24,8 +25,10 @@ import cn.smartjavaai.face.vector.entity.FaceVector;
|
||||
import com.alibaba.fastjson.JSONArray;
|
||||
import com.alibaba.fastjson.JSONObject;
|
||||
import lombok.extern.slf4j.Slf4j;
|
||||
import org.junit.BeforeClass;
|
||||
import org.junit.Test;
|
||||
|
||||
import java.io.IOException;
|
||||
import java.util.List;
|
||||
|
||||
/**
|
||||
@@ -42,14 +45,25 @@ public class FaceRecDemo {
|
||||
//设备类型
|
||||
public static DeviceEnum device = DeviceEnum.CPU;
|
||||
|
||||
@BeforeClass
|
||||
public static void beforeAll() throws IOException {
|
||||
//修改缓存路径
|
||||
// Config.setCachePath("/Users/xxx/smartjavaai_cache");
|
||||
}
|
||||
|
||||
|
||||
/**
|
||||
* 获取人脸检测模型
|
||||
* 获取人脸检测模型(高精度,速度慢)
|
||||
* 追求准确度可以使用
|
||||
* 也可以使用其他模型,具体其他模型参数可以查看文档:http://doc.smartjavaai.cn/face.html
|
||||
* @return
|
||||
*/
|
||||
public FaceDetModel getFaceDetModel(){
|
||||
public FaceDetModel getHighAccuracyDetModel(){
|
||||
FaceDetConfig config = new FaceDetConfig();
|
||||
//高精度模型,速度慢
|
||||
config.setModelEnum(FaceDetModelEnum.RETINA_FACE);//人脸检测模型
|
||||
//下载模型并替换本地路径,下载地址:https://pan.baidu.com/s/10l22x5fRz_gwLr8EAHa1Jg?pwd=1234 提取码: 1234
|
||||
config.setModelPath("/Users/xxx/Documents/develop/model/retinaface.pt");
|
||||
config.setConfidenceThreshold(FaceDetectConstant.DEFAULT_CONFIDENCE_THRESHOLD);//只返回相似度大于该值的人脸
|
||||
config.setNmsThresh(FaceDetectConstant.NMS_THRESHOLD);//用于去除重复的人脸框,当两个框的重叠度超过该值时,只保留一个
|
||||
config.setDevice(device);
|
||||
@@ -57,21 +71,62 @@ public class FaceRecDemo {
|
||||
}
|
||||
|
||||
/**
|
||||
* 获取人脸识别模型
|
||||
* 获取人脸检测模型(高速模型,精度一般)
|
||||
* 追求速度可以使用
|
||||
* 也可以使用其他模型,具体其他模型参数可以查看文档:http://doc.smartjavaai.cn/face.html
|
||||
* @return
|
||||
*/
|
||||
public FaceRecModel getFaceRecModel(){
|
||||
public FaceDetModel getHighSpeedDetModel(){
|
||||
FaceDetConfig config = new FaceDetConfig();
|
||||
//高速模型,速度快,精度一般
|
||||
config.setModelEnum(FaceDetModelEnum.SEETA_FACE6_MODEL);
|
||||
//下载模型并替换本地路径,下载地址:https://pan.baidu.com/s/10l22x5fRz_gwLr8EAHa1Jg?pwd=1234 提取码: 1234
|
||||
config.setModelPath("/Users/xxx/Documents/develop/model/sf3.0_models");
|
||||
config.setDevice(device);
|
||||
return FaceDetModelFactory.getInstance().getModel(config);
|
||||
}
|
||||
|
||||
/**
|
||||
* 获取人脸识别模型(高精度,速度慢)
|
||||
* 追求准确度可以使用
|
||||
* 也可以使用其他模型,具体其他模型参数可以查看文档:http://doc.smartjavaai.cn/face.html
|
||||
* @return
|
||||
*/
|
||||
public FaceRecModel getHighAccuracyFaceRecModel(){
|
||||
FaceRecConfig config = new FaceRecConfig();
|
||||
config.setModelEnum(FaceRecModelEnum.FACENET_MODEL);
|
||||
// config.setModelPath("/Users/xxx/Documents/develop/model/elasticface.pt");
|
||||
// config.setModelPath("/Users/xxx/Documents/develop/model/InsightFace/model_mobilefacenet.pt");
|
||||
//高精度模型,速度慢
|
||||
config.setModelEnum(FaceRecModelEnum.ELASTIC_FACE_MODEL);
|
||||
//模型路径,请下载模型并替换为本地路径:https://pan.baidu.com/s/10l22x5fRz_gwLr8EAHa1Jg?pwd=1234 提取码: 1234
|
||||
config.setModelPath("/Users/xxx/Documents/develop/model/elasticface.pt");
|
||||
//裁剪人脸:如果图片已经是裁剪过的,则请将此参数设置为false
|
||||
config.setCropFace(true);
|
||||
//开启人脸对齐:适用于人脸不正的场景,开启将提升人脸特征准确度,关闭可以提升性能
|
||||
config.setAlign(true);
|
||||
config.setDevice(device);
|
||||
//指定人脸检测模型
|
||||
config.setDetectModel(getFaceDetModel());
|
||||
config.setDetectModel(getHighAccuracyDetModel());
|
||||
return FaceRecModelFactory.getInstance().getModel(config);
|
||||
}
|
||||
|
||||
/**
|
||||
* 获取人脸识别模型(高速模型,精度一般)
|
||||
* 追求速度可以使用
|
||||
* 也可以使用其他模型,具体其他模型参数可以查看文档:http://doc.smartjavaai.cn/face.html
|
||||
* @return
|
||||
*/
|
||||
public FaceRecModel getHighSpeedFaceRecModel(){
|
||||
FaceRecConfig config = new FaceRecConfig();
|
||||
//高精度模型,速度慢
|
||||
config.setModelEnum(FaceRecModelEnum.SEETA_FACE6_MODEL);
|
||||
//模型路径,请下载模型并替换为本地路径:https://pan.baidu.com/s/10l22x5fRz_gwLr8EAHa1Jg?pwd=1234 提取码: 1234
|
||||
config.setModelPath("/Users/xxx/Documents/develop/model/sf3.0_models");
|
||||
//裁剪人脸:如果图片已经是裁剪过的,则请将此参数设置为false
|
||||
config.setCropFace(true);
|
||||
//开启人脸对齐:适用于人脸不正的场景,开启将提升人脸特征准确度,关闭可以提升性能
|
||||
config.setAlign(false);
|
||||
config.setDevice(device);
|
||||
//指定人脸检测模型
|
||||
config.setDetectModel(getHighSpeedDetModel());
|
||||
return FaceRecModelFactory.getInstance().getModel(config);
|
||||
}
|
||||
|
||||
@@ -81,14 +136,15 @@ public class FaceRecDemo {
|
||||
*/
|
||||
public FaceRecModel getFaceRecModelWithDbConfig(){
|
||||
FaceRecConfig config = new FaceRecConfig();
|
||||
//高精度模型,速度慢,追求速度请更换高速模型,具体其他模型参数可以查看文档:http://doc.smartjavaai.cn/face.html
|
||||
config.setModelEnum(FaceRecModelEnum.ELASTIC_FACE_MODEL);//人脸检测模型
|
||||
config.setModelPath("/Users/xxx/Documents/develop/model/elasticface.pt");
|
||||
//裁剪人脸:如果图片已经是裁剪过的,则请将此参数设置为false
|
||||
config.setCropFace(true);
|
||||
//开启人脸对齐:适用于人脸不正的场景,开启将提升人脸特征准确度,关闭可以提升性能
|
||||
config.setAlign(true);
|
||||
//指定人脸检测模型
|
||||
config.setDetectModel(getFaceDetModel());
|
||||
//指定人脸检测模型,高精度模型,速度慢,追求速度请更换高速模型getHighSpeedDetModel
|
||||
config.setDetectModel(getHighAccuracyDetModel());
|
||||
config.setDevice(device);
|
||||
|
||||
//初始化向量数据库:Milvus数据库配置
|
||||
@@ -112,13 +168,15 @@ public class FaceRecDemo {
|
||||
*/
|
||||
public FaceRecModel getFaceRecModelWithSQLiteConfig(){
|
||||
FaceRecConfig config = new FaceRecConfig();
|
||||
config.setModelEnum(FaceRecModelEnum.FACENET_MODEL);//人脸检测模型
|
||||
//高精度模型,速度慢, 追求速度请更换高速模型,具体其他模型参数可以查看文档:http://doc.smartjavaai.cn/face.html
|
||||
config.setModelEnum(FaceRecModelEnum.ELASTIC_FACE_MODEL);//人脸检测模型
|
||||
config.setModelPath("/Users/xxx/Documents/develop/model/elasticface.pt");
|
||||
//裁剪人脸:如果图片已经是裁剪过的,则请将此参数设置为false
|
||||
config.setCropFace(true);
|
||||
//开启人脸对齐:适用于人脸不正的场景,开启将提升人脸特征准确度,关闭可以提升性能
|
||||
config.setAlign(true);
|
||||
//指定人脸检测模型
|
||||
config.setDetectModel(getFaceDetModel());
|
||||
//指定人脸检测模型,高精度模型,速度慢,追求速度请更换高速模型getHighSpeedDetModel
|
||||
config.setDetectModel(getHighAccuracyDetModel());
|
||||
config.setDevice(device);
|
||||
|
||||
//初始化SQLite数据库
|
||||
@@ -138,7 +196,8 @@ public class FaceRecDemo {
|
||||
@Test
|
||||
public void testExtractFeatures(){
|
||||
try {
|
||||
FaceRecModel faceRecModel = getFaceRecModel();
|
||||
//高精度模型,速度慢, 追求速度请更换高速模型: getHighSpeedFaceRecModel
|
||||
FaceRecModel faceRecModel = getHighAccuracyFaceRecModel();
|
||||
//提取图片中所有人脸特征
|
||||
R<DetectionResponse> faceResult = faceRecModel.extractFeatures("src/main/resources/iu_1.jpg");
|
||||
if(faceResult.isSuccess()){
|
||||
@@ -162,7 +221,8 @@ public class FaceRecDemo {
|
||||
@Test
|
||||
public void featureComparison(){
|
||||
try {
|
||||
FaceRecModel faceRecModel = getFaceRecModel();
|
||||
//高精度模型,速度慢, 追求速度请更换高速模型: getHighSpeedFaceRecModel
|
||||
FaceRecModel faceRecModel = getHighAccuracyFaceRecModel();
|
||||
//基于图像直接比对人脸特征
|
||||
R<Float> similarResult = faceRecModel.featureComparison("src/main/resources/iu_1.jpg","src/main/resources/iu_2.jpg");
|
||||
if(similarResult.isSuccess()){
|
||||
@@ -188,7 +248,8 @@ public class FaceRecDemo {
|
||||
@Test
|
||||
public void featureComparison2(){
|
||||
try {
|
||||
FaceRecModel faceRecModel = getFaceRecModel();
|
||||
//高精度模型,速度慢, 追求速度请更换高速模型: getHighSpeedFaceRecModel
|
||||
FaceRecModel faceRecModel = getHighAccuracyFaceRecModel();
|
||||
//特征提取(提取分数最高人脸特征),适用于单人脸场景
|
||||
R<float[]> featureResult1 = faceRecModel.extractTopFaceFeature("src/main/resources/iu_1.jpg");
|
||||
if(featureResult1.isSuccess()){
|
||||
@@ -226,6 +287,7 @@ public class FaceRecDemo {
|
||||
@Test
|
||||
public void searchFace(){
|
||||
try {
|
||||
//高精度模型,速度慢, 追求速度请更换高速模型
|
||||
FaceRecModel faceRecModel = getFaceRecModelWithDbConfig();
|
||||
//等待加载人脸库结束
|
||||
while (!faceRecModel.isLoadFaceCompleted()){
|
||||
@@ -303,6 +365,7 @@ public class FaceRecDemo {
|
||||
@Test
|
||||
public void searchFace2(){
|
||||
try {
|
||||
//高精度模型,速度慢, 追求速度请更换高速模型
|
||||
FaceRecModel faceRecModel = getFaceRecModelWithSQLiteConfig();
|
||||
//等待加载人脸库结束
|
||||
while (!faceRecModel.isLoadFaceCompleted()){
|
||||
@@ -419,4 +482,7 @@ public class FaceRecDemo {
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
}
|
||||
|
||||
@@ -3,6 +3,7 @@ package smartai.examples.face.liveness;
|
||||
import ai.djl.modality.cv.Image;
|
||||
import ai.djl.modality.cv.ImageFactory;
|
||||
import cn.hutool.core.lang.UUID;
|
||||
import cn.smartjavaai.common.config.Config;
|
||||
import cn.smartjavaai.common.entity.DetectionInfo;
|
||||
import cn.smartjavaai.common.entity.DetectionRectangle;
|
||||
import cn.smartjavaai.common.entity.DetectionResponse;
|
||||
@@ -32,6 +33,7 @@ import nu.pattern.OpenCV;
|
||||
import org.bytedeco.javacv.FFmpegFrameGrabber;
|
||||
import org.bytedeco.javacv.Frame;
|
||||
import org.bytedeco.javacv.Java2DFrameUtils;
|
||||
import org.junit.BeforeClass;
|
||||
import org.junit.Test;
|
||||
import org.opencv.core.Mat;
|
||||
import org.opencv.core.Size;
|
||||
@@ -61,6 +63,12 @@ public class LivenessDetDemo {
|
||||
//设备类型
|
||||
public static DeviceEnum device = DeviceEnum.CPU;
|
||||
|
||||
@BeforeClass
|
||||
public static void beforeAll() throws IOException {
|
||||
//修改缓存路径
|
||||
// Config.setCachePath("/Users/xxx/smartjavaai_cache");
|
||||
}
|
||||
|
||||
|
||||
|
||||
/**
|
||||
|
||||
@@ -1,5 +1,6 @@
|
||||
package smartai.examples.face.quality;
|
||||
|
||||
import cn.smartjavaai.common.config.Config;
|
||||
import cn.smartjavaai.common.entity.DetectionInfo;
|
||||
import cn.smartjavaai.common.entity.DetectionResponse;
|
||||
import cn.smartjavaai.common.entity.R;
|
||||
@@ -23,11 +24,13 @@ import cn.smartjavaai.face.model.quality.FaceQualityModel;
|
||||
import cn.smartjavaai.face.utils.FaceUtils;
|
||||
import com.alibaba.fastjson.JSONObject;
|
||||
import lombok.extern.slf4j.Slf4j;
|
||||
import org.junit.BeforeClass;
|
||||
import org.junit.Test;
|
||||
|
||||
import javax.imageio.ImageIO;
|
||||
import java.awt.image.BufferedImage;
|
||||
import java.io.File;
|
||||
import java.io.IOException;
|
||||
import java.nio.file.Paths;
|
||||
|
||||
/**
|
||||
@@ -41,6 +44,12 @@ public class FaceQualityDetDemo {
|
||||
//设备类型
|
||||
public static DeviceEnum device = DeviceEnum.CPU;
|
||||
|
||||
@BeforeClass
|
||||
public static void beforeAll() throws IOException {
|
||||
//修改缓存路径
|
||||
// Config.setCachePath("/Users/xxx/smartjavaai_cache");
|
||||
}
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
@@ -12,7 +12,7 @@
|
||||
<maven.compiler.source>11</maven.compiler.source>
|
||||
<maven.compiler.target>11</maven.compiler.target>
|
||||
<project.build.sourceEncoding>UTF-8</project.build.sourceEncoding>
|
||||
<smartjavaai.version>1.0.22</smartjavaai.version>
|
||||
<smartjavaai.version>1.0.23</smartjavaai.version>
|
||||
<!--如果打包运行,需要替换成你的main-->
|
||||
<exec.mainClass>smartai.examples.objectdetection.ObjectDetection</exec.mainClass>
|
||||
|
||||
|
||||
@@ -11,6 +11,7 @@ import ai.djl.repository.zoo.ModelNotFoundException;
|
||||
import ai.djl.repository.zoo.ModelZoo;
|
||||
import ai.djl.repository.zoo.ZooModel;
|
||||
import ai.djl.training.util.ProgressBar;
|
||||
import cn.smartjavaai.common.config.Config;
|
||||
import cn.smartjavaai.common.entity.DetectionInfo;
|
||||
import cn.smartjavaai.common.entity.DetectionRectangle;
|
||||
import cn.smartjavaai.common.entity.DetectionResponse;
|
||||
@@ -27,6 +28,7 @@ import com.alibaba.fastjson.JSONObject;
|
||||
import lombok.extern.slf4j.Slf4j;
|
||||
import nu.pattern.OpenCV;
|
||||
import org.junit.Assert;
|
||||
import org.junit.BeforeClass;
|
||||
import org.junit.Test;
|
||||
import org.opencv.core.Mat;
|
||||
import org.opencv.core.Size;
|
||||
@@ -47,6 +49,7 @@ import java.util.concurrent.Callable;
|
||||
import java.util.concurrent.ExecutorService;
|
||||
import java.util.concurrent.Executors;
|
||||
import java.util.concurrent.Future;
|
||||
import java.util.stream.Collectors;
|
||||
|
||||
/**
|
||||
* 目标检测模型demo
|
||||
@@ -61,6 +64,12 @@ public class ObjectDetection {
|
||||
//设备类型
|
||||
public static DeviceEnum device = DeviceEnum.CPU;
|
||||
|
||||
@BeforeClass
|
||||
public static void beforeAll() throws IOException {
|
||||
//修改缓存路径
|
||||
// Config.setCachePath("/Users/xxx/smartjavaai_cache");
|
||||
}
|
||||
|
||||
|
||||
|
||||
/**
|
||||
@@ -139,15 +148,30 @@ public class ObjectDetection {
|
||||
public void objectDetectionWithOfficialModel(){
|
||||
try {
|
||||
DetectorModelConfig config = new DetectorModelConfig();
|
||||
config.setThreshold(0.3f);
|
||||
// config.setThreshold(0.3f);
|
||||
//也支持YoloV8:YOLOV8_OFFICIAL 模型可以从文档中提供的地址下载
|
||||
config.setModelEnum(DetectorModelEnum.YOLOV12_OFFICIAL);//检测模型,目前支持19种模型
|
||||
// 指定模型路径,需要更改为自己的模型路径
|
||||
config.setModelPath("/Users/xxx/Documents/yolov12n.onnx");
|
||||
config.setModelPath("/Users/wenjie/Documents/develop/face_model/yolo11n.torchscript");
|
||||
config.setDevice(device);
|
||||
config.putCustomParam("width", 640);//resize 宽
|
||||
config.putCustomParam("height", 640);// resize 高
|
||||
config.putCustomParam("resize", true);
|
||||
config.putCustomParam("toTensor", true);
|
||||
config.putCustomParam("applyRatio", true);
|
||||
config.putCustomParam("threshold", 0.6f);
|
||||
// for performance optimization maxBox parameter can reduce number of
|
||||
// considered boxes from 8400
|
||||
config.putCustomParam("maxBox", 8400);
|
||||
// config.putCustomParam("pad", 114d);
|
||||
// List<Float> mean = Arrays.asList(0.5f,0.5f,0.5f,0.5f,0.5f,0.5f);
|
||||
// String normalize = mean.stream().map(Object::toString).collect(Collectors.joining(","));
|
||||
// config.putCustomParam("normalize", normalize);
|
||||
// config.putCustomParam("flag", Image.Flag.COLOR);
|
||||
// config.putCustomParam("pad", 114);
|
||||
//一定要将yolo官方的类别文件:synset.txt(文档中下载)放在模型同目录下,否则报错
|
||||
DetectorModel detectorModel = ObjectDetectionModelFactory.getInstance().getModel(config);
|
||||
DetectionResponse detect = detectorModel.detect("src/main/resources/dog_bike_car.jpg");
|
||||
DetectionResponse detect = detectorModel.detect("src/main/resources/object_detection.jpg");
|
||||
log.info("目标检测结果:{}", JSONObject.toJSONString(detect));
|
||||
} catch (Exception e) {
|
||||
e.printStackTrace();
|
||||
|
||||
@@ -12,7 +12,7 @@
|
||||
<maven.compiler.source>11</maven.compiler.source>
|
||||
<maven.compiler.target>11</maven.compiler.target>
|
||||
<project.build.sourceEncoding>UTF-8</project.build.sourceEncoding>
|
||||
<smartjavaai.version>1.0.22</smartjavaai.version>
|
||||
<smartjavaai.version>1.0.23</smartjavaai.version>
|
||||
<!--如果打包运行,需要替换成你的main-->
|
||||
<exec.mainClass>smartai.examples.ocr.common.OcrRecognizeDemo</exec.mainClass>
|
||||
|
||||
|
||||
@@ -1,6 +1,7 @@
|
||||
package smartai.examples.ocr.common;
|
||||
|
||||
import ai.djl.modality.cv.Image;
|
||||
import cn.smartjavaai.common.config.Config;
|
||||
import cn.smartjavaai.common.entity.DetectionResponse;
|
||||
import cn.smartjavaai.common.enums.DeviceEnum;
|
||||
import cn.smartjavaai.common.utils.ImageUtils;
|
||||
@@ -14,12 +15,14 @@ import cn.smartjavaai.ocr.model.common.detect.OcrCommonDetModel;
|
||||
import cn.smartjavaai.ocr.utils.OcrUtils;
|
||||
import com.alibaba.fastjson.JSONObject;
|
||||
import lombok.extern.slf4j.Slf4j;
|
||||
import org.junit.BeforeClass;
|
||||
import org.junit.Test;
|
||||
import org.opencv.core.Mat;
|
||||
|
||||
import javax.imageio.ImageIO;
|
||||
import java.awt.image.BufferedImage;
|
||||
import java.io.File;
|
||||
import java.io.IOException;
|
||||
import java.nio.file.Paths;
|
||||
import java.util.List;
|
||||
|
||||
@@ -32,9 +35,17 @@ import java.util.List;
|
||||
public class OcrDetectionDemo {
|
||||
|
||||
|
||||
|
||||
|
||||
//设备类型
|
||||
public static DeviceEnum device = DeviceEnum.CPU;
|
||||
|
||||
@BeforeClass
|
||||
public static void beforeAll() throws IOException {
|
||||
//修改缓存路径
|
||||
// Config.setCachePath("/Users/xxx/smartjavaai_cache");
|
||||
}
|
||||
|
||||
/**
|
||||
* 获取文本检测模型
|
||||
* @return
|
||||
|
||||
@@ -1,5 +1,6 @@
|
||||
package smartai.examples.ocr.common;
|
||||
|
||||
import cn.smartjavaai.common.config.Config;
|
||||
import cn.smartjavaai.common.enums.DeviceEnum;
|
||||
import cn.smartjavaai.ocr.config.DirectionModelConfig;
|
||||
import cn.smartjavaai.ocr.config.OcrDetModelConfig;
|
||||
@@ -12,8 +13,10 @@ import cn.smartjavaai.ocr.model.common.detect.OcrCommonDetModel;
|
||||
import cn.smartjavaai.ocr.model.common.direction.OcrDirectionModel;
|
||||
import com.alibaba.fastjson.JSONObject;
|
||||
import lombok.extern.slf4j.Slf4j;
|
||||
import org.junit.BeforeClass;
|
||||
import org.junit.Test;
|
||||
|
||||
import java.io.IOException;
|
||||
import java.util.List;
|
||||
|
||||
/**
|
||||
@@ -28,6 +31,12 @@ public class OcrDirectionDetDemo {
|
||||
//设备类型
|
||||
public static DeviceEnum device = DeviceEnum.CPU;
|
||||
|
||||
@BeforeClass
|
||||
public static void beforeAll() throws IOException {
|
||||
//修改缓存路径
|
||||
// Config.setCachePath("/Users/xxx/smartjavaai_cache");
|
||||
}
|
||||
|
||||
/**
|
||||
* 获取方向检测模型
|
||||
* @return
|
||||
|
||||
@@ -1,6 +1,10 @@
|
||||
package smartai.examples.ocr.common;
|
||||
|
||||
import ai.djl.modality.cv.Image;
|
||||
import ai.djl.util.JsonUtils;
|
||||
import cn.hutool.core.img.ImgUtil;
|
||||
import cn.hutool.core.io.FileUtil;
|
||||
import cn.smartjavaai.common.config.Config;
|
||||
import cn.smartjavaai.common.enums.DeviceEnum;
|
||||
import cn.smartjavaai.common.utils.ImageUtils;
|
||||
import cn.smartjavaai.ocr.config.DirectionModelConfig;
|
||||
@@ -18,9 +22,15 @@ import cn.smartjavaai.ocr.model.common.direction.OcrDirectionModel;
|
||||
import cn.smartjavaai.ocr.model.common.recognize.OcrCommonRecModel;
|
||||
import com.alibaba.fastjson.JSONObject;
|
||||
import lombok.extern.slf4j.Slf4j;
|
||||
import org.junit.BeforeClass;
|
||||
import org.junit.Test;
|
||||
|
||||
import javax.imageio.ImageIO;
|
||||
import java.awt.image.BufferedImage;
|
||||
import java.io.File;
|
||||
import java.io.IOException;
|
||||
import java.nio.file.Path;
|
||||
import java.nio.file.Paths;
|
||||
import java.util.List;
|
||||
|
||||
/**
|
||||
@@ -35,6 +45,13 @@ public class OcrRecognizeDemo {
|
||||
//设备类型
|
||||
public static DeviceEnum device = DeviceEnum.CPU;
|
||||
|
||||
|
||||
@BeforeClass
|
||||
public static void beforeAll() throws IOException {
|
||||
//修改缓存路径
|
||||
//Config.setCachePath("/Users/xxx/smartjavaai_cache");
|
||||
}
|
||||
|
||||
/**
|
||||
* 获取通用识别模型(不带方向矫正)
|
||||
* @return
|
||||
@@ -44,7 +61,7 @@ public class OcrRecognizeDemo {
|
||||
//指定文本识别模型
|
||||
recModelConfig.setRecModelEnum(CommonRecModelEnum.PP_OCR_V5_MOBILE_REC_MODEL);
|
||||
//指定识别模型位置,需要更改为自己的模型路径(下载地址请查看文档)
|
||||
recModelConfig.setRecModelPath("/Users/xxx/Documents/develop/model/ocr/PP-OCRv5_mobile_rec_infer/PP-OCRv5_mobile_rec_infer.onnx");
|
||||
recModelConfig.setRecModelPath("/Users/wenjie/Documents/develop/model/ocr/PP-OCRv5_mobile_rec_infer/PP-OCRv5_mobile_rec_infer.onnx");
|
||||
recModelConfig.setDevice(device);
|
||||
recModelConfig.setTextDetModel(getDetectionModel());
|
||||
return OcrModelFactory.getInstance().getRecModel(recModelConfig);
|
||||
@@ -59,7 +76,7 @@ public class OcrRecognizeDemo {
|
||||
//指定检测模型
|
||||
config.setModelEnum(CommonDetModelEnum.PP_OCR_V5_MOBILE_DET_MODEL);
|
||||
//指定模型位置,需要更改为自己的模型路径(下载地址请查看文档)
|
||||
config.setDetModelPath("/Users/xxx/Documents/develop/model/ocr/PP-OCRv5_mobile_det_infer/PP-OCRv5_mobile_det_infer.onnx");
|
||||
config.setDetModelPath("/Users/wenjie/Documents/develop/model/ocr/PP-OCRv5_mobile_det_infer/PP-OCRv5_mobile_det_infer.onnx");
|
||||
config.setDevice(device);
|
||||
return OcrModelFactory.getInstance().getDetModel(config);
|
||||
}
|
||||
@@ -180,6 +197,59 @@ public class OcrRecognizeDemo {
|
||||
}
|
||||
}
|
||||
|
||||
@Test
|
||||
public void recognizeAndDraw2(){
|
||||
try {
|
||||
OcrCommonRecModel recModel = getRecModel();
|
||||
int fontSize = 18;
|
||||
//创建保存路径
|
||||
Path inputImagePath = Paths.get("src/main/resources/general_ocr_002.png");
|
||||
Path imageOutputPath = Paths.get("output/ocr_4_recognized.jpg");
|
||||
BufferedImage image = null;
|
||||
image = ImageIO.read(new File(inputImagePath.toAbsolutePath().toString()));
|
||||
BufferedImage resultImage = recModel.recognizeAndDraw(image, fontSize, new OcrRecOptions());
|
||||
ImageUtils.saveImage(resultImage, imageOutputPath.toAbsolutePath().toString());
|
||||
} catch (Exception e) {
|
||||
e.printStackTrace();
|
||||
}
|
||||
}
|
||||
|
||||
/**
|
||||
* 文本识别并绘制结果(返回base64)
|
||||
*/
|
||||
@Test
|
||||
public void recognizeAndDrawToBase64(){
|
||||
try {
|
||||
OcrCommonRecModel recModel = getRecModel();
|
||||
int fontSize = 18;
|
||||
//创建保存路径
|
||||
Path inputImagePath = Paths.get("src/main/resources/general_ocr_002.png");
|
||||
byte[] imageBytes = FileUtil.readBytes(inputImagePath);
|
||||
String base64 = recModel.recognizeAndDrawToBase64(imageBytes, fontSize, new OcrRecOptions());
|
||||
log.info("base64:{}", base64);
|
||||
} catch (Exception e) {
|
||||
e.printStackTrace();
|
||||
}
|
||||
}
|
||||
|
||||
/**
|
||||
* 文本识别并绘制结果(返回OcrInfo,OcrInfo中包含base64)
|
||||
*/
|
||||
@Test
|
||||
public void recognizeAndDraw3(){
|
||||
try {
|
||||
OcrCommonRecModel recModel = getRecModel();
|
||||
int fontSize = 18;
|
||||
//创建保存路径
|
||||
Path inputImagePath = Paths.get("src/main/resources/general_ocr_002.png");
|
||||
byte[] imageBytes = FileUtil.readBytes(inputImagePath);
|
||||
OcrInfo ocrInfo = recModel.recognizeAndDraw(imageBytes, fontSize, new OcrRecOptions());
|
||||
log.info("ocrInfo:{}", JsonUtils.toJson(ocrInfo));
|
||||
} catch (Exception e) {
|
||||
e.printStackTrace();
|
||||
}
|
||||
}
|
||||
|
||||
/**
|
||||
* 批量识别
|
||||
* 注意事项:
|
||||
|
||||
@@ -1,6 +1,7 @@
|
||||
package smartai.examples.ocr.plate;
|
||||
|
||||
import ai.djl.util.JsonUtils;
|
||||
import cn.smartjavaai.common.config.Config;
|
||||
import cn.smartjavaai.common.entity.R;
|
||||
import cn.smartjavaai.common.enums.DeviceEnum;
|
||||
import cn.smartjavaai.common.utils.ImageUtils;
|
||||
@@ -13,9 +14,11 @@ import cn.smartjavaai.ocr.factory.PlateModelFactory;
|
||||
import cn.smartjavaai.ocr.model.plate.PlateDetModel;
|
||||
import cn.smartjavaai.ocr.model.plate.PlateRecModel;
|
||||
import lombok.extern.slf4j.Slf4j;
|
||||
import org.junit.BeforeClass;
|
||||
import org.junit.Test;
|
||||
|
||||
import java.io.File;
|
||||
import java.io.IOException;
|
||||
import java.util.List;
|
||||
|
||||
/**
|
||||
@@ -27,6 +30,12 @@ public class PlateRecDemo {
|
||||
//设备类型
|
||||
public static DeviceEnum device = DeviceEnum.CPU;
|
||||
|
||||
@BeforeClass
|
||||
public static void beforeAll() throws IOException {
|
||||
//修改缓存路径
|
||||
// Config.setCachePath("/Users/xxx/smartjavaai_cache");
|
||||
}
|
||||
|
||||
/**
|
||||
* 获取车牌检测模型
|
||||
* @return
|
||||
|
||||
@@ -2,6 +2,7 @@ package smartai.examples.ocr.table;
|
||||
|
||||
import ai.djl.modality.cv.Image;
|
||||
import cn.hutool.core.io.FileUtil;
|
||||
import cn.smartjavaai.common.config.Config;
|
||||
import cn.smartjavaai.common.entity.R;
|
||||
import cn.smartjavaai.common.enums.DeviceEnum;
|
||||
import cn.smartjavaai.common.utils.ImageUtils;
|
||||
@@ -21,17 +22,21 @@ import cn.smartjavaai.ocr.model.table.TableRecognizer;
|
||||
import cn.smartjavaai.ocr.model.table.TableStructureModel;
|
||||
import com.alibaba.fastjson.JSONObject;
|
||||
import lombok.extern.slf4j.Slf4j;
|
||||
import org.junit.BeforeClass;
|
||||
import org.junit.Test;
|
||||
|
||||
import javax.imageio.ImageIO;
|
||||
import java.awt.image.BufferedImage;
|
||||
import java.io.File;
|
||||
import java.io.IOException;
|
||||
import java.io.OutputStream;
|
||||
import java.nio.file.Files;
|
||||
import java.nio.file.Path;
|
||||
import java.nio.file.Paths;
|
||||
import java.util.List;
|
||||
|
||||
/**
|
||||
* OCR 文本识别 示例
|
||||
* OCR 表格识别 示例
|
||||
* 模型下载地址:https://pan.baidu.com/s/1MLfd73Vjdpnuls9-oqc9uw?pwd=1234 提取码: 1234
|
||||
* @author dwj
|
||||
* @date 2025/5/25
|
||||
@@ -39,6 +44,13 @@ import java.util.List;
|
||||
@Slf4j
|
||||
public class TableRecDemo {
|
||||
|
||||
|
||||
@BeforeClass
|
||||
public static void beforeAll() throws IOException {
|
||||
//修改缓存路径
|
||||
// Config.setCachePath("/Users/xxx/smartjavaai_cache");
|
||||
}
|
||||
|
||||
//设备类型
|
||||
public static DeviceEnum device = DeviceEnum.CPU;
|
||||
|
||||
@@ -51,7 +63,7 @@ public class TableRecDemo {
|
||||
//指定文本识别模型
|
||||
recModelConfig.setRecModelEnum(CommonRecModelEnum.PP_OCR_V5_MOBILE_REC_MODEL);
|
||||
//指定识别模型位置,需要更改为自己的模型路径(下载地址请查看文档)
|
||||
recModelConfig.setRecModelPath("/Users/xxx/Documents/develop/model/ocr/PP-OCRv5_mobile_rec_infer/PP-OCRv5_mobile_rec_infer.onnx");
|
||||
recModelConfig.setRecModelPath("/Users/wenjie/Documents/develop/model/ocr/PP-OCRv5_mobile_rec_infer/PP-OCRv5_mobile_rec_infer.onnx");
|
||||
recModelConfig.setDevice(device);
|
||||
recModelConfig.setTextDetModel(getDetectionModel());
|
||||
return OcrModelFactory.getInstance().getRecModel(recModelConfig);
|
||||
@@ -66,7 +78,7 @@ public class TableRecDemo {
|
||||
//指定检测模型
|
||||
config.setModelEnum(CommonDetModelEnum.PP_OCR_V5_MOBILE_DET_MODEL);
|
||||
//指定模型位置,需要更改为自己的模型路径(下载地址请查看文档)
|
||||
config.setDetModelPath("/Users/xxx/Documents/develop/model/ocr/PP-OCRv5_mobile_det_infer/PP-OCRv5_mobile_det_infer.onnx");
|
||||
config.setDetModelPath("/Users/wenjie/Documents/develop/model/ocr/PP-OCRv5_mobile_det_infer/PP-OCRv5_mobile_det_infer.onnx");
|
||||
// config.setDetModelPath("/Users/xxx/Documents/develop/model/ocr/PP-OCRv5_server_det_infer/PP-OCRv5_server_det.onnx");
|
||||
config.setDevice(device);
|
||||
return OcrModelFactory.getInstance().getDetModel(config);
|
||||
@@ -81,7 +93,7 @@ public class TableRecDemo {
|
||||
//指定行文本方向检测模型
|
||||
directionModelConfig.setModelEnum(DirectionModelEnum.PP_LCNET_X0_25);
|
||||
//指定行文本方向检测模型路径,需要更改为自己的模型路径(下载地址请查看文档)
|
||||
directionModelConfig.setModelPath("/Users/xxx/Documents/develop/model/ocr/PP-LCNet_x0_25_textline_ori_infer/PP-LCNet_x0_25_textline_ori_infer.onnx");
|
||||
directionModelConfig.setModelPath("/Users/wenjie/Documents/develop/model/ocr/PP-LCNet_x0_25_textline_ori_infer/PP-LCNet_x0_25_textline_ori_infer.onnx");
|
||||
directionModelConfig.setDevice(device);
|
||||
return OcrModelFactory.getInstance().getDirectionModel(directionModelConfig);
|
||||
}
|
||||
@@ -95,7 +107,7 @@ public class TableRecDemo {
|
||||
//指定行文本方向检测模型
|
||||
config.setModelEnum(TableStructureModelEnum.SLANET_PLUS);
|
||||
//指定行文本方向检测模型路径,需要更改为自己的模型路径(下载地址请查看文档)
|
||||
config.setModelPath("/Users/xxx/Documents/develop/model/ocr/slanet-plus/slanet-plus.onnx");
|
||||
config.setModelPath("/Users/wenjie/Documents/develop/model/ocr/slanet-plus/slanet-plus.onnx");
|
||||
// config.setModelPath("/Users/xxx/Documents/develop/model/ocr/SLANet_infer/SLANet.onnx");
|
||||
config.setDevice(device);
|
||||
return TableRecModelFactory.getInstance().getTableStructureModel(config);
|
||||
@@ -142,4 +154,47 @@ public class TableRecDemo {
|
||||
}
|
||||
}
|
||||
|
||||
|
||||
/**
|
||||
* 表格识别
|
||||
* 仅支持简单表格
|
||||
* 流程:表格结构识别 -> 文本检测 -> 文本识别 -> 合成html table
|
||||
* 注意事项:
|
||||
* 1、批量检测时,模型应统一放在外层 try 中使用,避免重复加载,自动释放资源更安全。
|
||||
* 2、模型文件需要放在单独文件夹
|
||||
*/
|
||||
@Test
|
||||
public void recognize2(){
|
||||
try {
|
||||
TableStructureModel tableStructureModel = getTableStructureModel();
|
||||
OcrCommonDetModel detModel = getDetectionModel();
|
||||
OcrCommonRecModel recModel = getRecModel();
|
||||
OcrDirectionModel directionModel = getDirectionModel();
|
||||
//创建表格识别器
|
||||
TableRecognizer tableRecognizer = TableRecognizer.builder()
|
||||
.withStructureModel(tableStructureModel)
|
||||
.withTextDetModel(detModel)
|
||||
// .withDirectionModel(getDirectionModel()) //如果表格中存在旋转的文字,可以使用方向分类模型
|
||||
.withTextRecModel(recModel).build();
|
||||
String imagePath = "src/main/resources/table/table_ch1.png";
|
||||
BufferedImage image = ImageIO.read(new File(Paths.get(imagePath).toAbsolutePath().toString()));
|
||||
R<TableStructureResult> result = tableRecognizer.recognize(image);
|
||||
if(result.isSuccess()){
|
||||
log.info("result: {}", result.getData().getHtml());
|
||||
//导出html内容到文件
|
||||
Path outputPath = Paths.get("output/table_ch2_result.html");
|
||||
FileUtil.writeUtf8String(result.getData().getHtml(), outputPath.toAbsolutePath().toString());
|
||||
//绘制表格结构
|
||||
BufferedImage resultImage = tableRecognizer.drawTable(result.getData(), image);
|
||||
ImageUtils.saveImage(resultImage, "output/table_ch2_result.jpg");
|
||||
//导出excel,如果导出失败,可能是因为表格结果识别的结果是错乱的
|
||||
try (OutputStream out = Files.newOutputStream(Paths.get("output/table_ch2_result2.xls"))) {
|
||||
tableRecognizer.exportExcel(result.getData().getHtml(), out);
|
||||
}
|
||||
}
|
||||
} catch (Exception e) {
|
||||
e.printStackTrace();
|
||||
}
|
||||
}
|
||||
|
||||
}
|
||||
|
||||
7
examples/speech-examples/.gitignore
vendored
Normal file
7
examples/speech-examples/.gitignore
vendored
Normal file
@@ -0,0 +1,7 @@
|
||||
.idea
|
||||
.idea/
|
||||
target
|
||||
log
|
||||
*.iml
|
||||
/.settings/
|
||||
/logging.file_IS_UNDEFINED/
|
||||
43
examples/speech-examples/README.md
Normal file
43
examples/speech-examples/README.md
Normal file
@@ -0,0 +1,43 @@
|
||||
# OCR文字识别示例
|
||||
|
||||
|
||||
我来为你写一个 `speech-examples` 项目的 README.md 文件。
|
||||
|
||||
## 🎤 语音识别示例
|
||||
|
||||
本项目展示了如何使用 SmartJavaAI SDK 进行语音识别,支持 Whisper 和 Vosk 两种语音识别引擎。
|
||||
|
||||
## 📁 项目结构
|
||||
|
||||
```
|
||||
src
|
||||
├── main
|
||||
│ ├── java
|
||||
│ │ └── smartai/examples/speech/asr
|
||||
│ │ └── SpeechRecognizeDemo.java # 语音识别示例
|
||||
│ └── resources
|
||||
│ ├── logback.xml # 日志配置文件
|
||||
│ ├── speech_zh.mp3 # 中文测试音频
|
||||
│ ├── lff_zh.mp3 # 中文测试音频
|
||||
│ └── jfk_en.wav # 英文测试音频
|
||||
└── test
|
||||
```
|
||||
|
||||
## 🚀 快速开始
|
||||
|
||||
1. 克隆项目到本地:
|
||||
|
||||
2. 导入项目至 IntelliJ IDEA。
|
||||
|
||||
3. 根据需要修改模型路径(见各 demo 中注释)。
|
||||
|
||||
4. 运行对应的 JUnit 测试类方法即可体验各项功能。
|
||||
|
||||
---
|
||||
|
||||
## 📄 文档
|
||||
|
||||
有关完整使用说明,请查阅 SmartJavaAI 官方文档:
|
||||
[http://doc.smartjavaai.cn](http://doc.smartjavaai.cn)
|
||||
|
||||
---
|
||||
214
examples/speech-examples/pom.xml
Normal file
214
examples/speech-examples/pom.xml
Normal file
@@ -0,0 +1,214 @@
|
||||
<?xml version="1.0" encoding="UTF-8"?>
|
||||
<project xmlns="http://maven.apache.org/POM/4.0.0"
|
||||
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
|
||||
xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
|
||||
<modelVersion>4.0.0</modelVersion>
|
||||
|
||||
<groupId>cn.smartjavaai</groupId>
|
||||
<artifactId>examples-speech</artifactId>
|
||||
<version>1.0.0-SNAPSHOT</version>
|
||||
|
||||
<properties>
|
||||
<maven.compiler.source>11</maven.compiler.source>
|
||||
<maven.compiler.target>11</maven.compiler.target>
|
||||
<project.build.sourceEncoding>UTF-8</project.build.sourceEncoding>
|
||||
<smartjavaai.version>1.0.23</smartjavaai.version>
|
||||
<!--如果打包运行,需要替换成你的main-->
|
||||
<exec.mainClass>smartai.examples.speech.asr.common.OcrRecognizeDemo</exec.mainClass>
|
||||
|
||||
</properties>
|
||||
|
||||
<dependencyManagement>
|
||||
<dependencies>
|
||||
<dependency>
|
||||
<groupId>cn.smartjavaai</groupId>
|
||||
<artifactId>smartjavaai-bom</artifactId>
|
||||
<version>${smartjavaai.version}</version>
|
||||
<type>pom</type>
|
||||
<!-- 注意这里是import -->
|
||||
<scope>import</scope>
|
||||
</dependency>
|
||||
</dependencies>
|
||||
</dependencyManagement>
|
||||
|
||||
|
||||
<dependencies>
|
||||
<dependency>
|
||||
<groupId>commons-cli</groupId>
|
||||
<artifactId>commons-cli</artifactId>
|
||||
<version>1.9.0</version>
|
||||
</dependency>
|
||||
<dependency>
|
||||
<groupId>commons-io</groupId>
|
||||
<artifactId>commons-io</artifactId>
|
||||
<version>2.17.0</version>
|
||||
</dependency>
|
||||
<dependency>
|
||||
<groupId>org.apache.logging.log4j</groupId>
|
||||
<artifactId>log4j-slf4j2-impl</artifactId>
|
||||
<version>2.24.1</version>
|
||||
</dependency>
|
||||
<dependency>
|
||||
<groupId>org.testng</groupId>
|
||||
<artifactId>testng</artifactId>
|
||||
<version>7.10.2</version>
|
||||
<scope>test</scope>
|
||||
</dependency>
|
||||
|
||||
|
||||
<dependency>
|
||||
<groupId>ch.qos.logback</groupId>
|
||||
<artifactId>logback-classic</artifactId>
|
||||
<version>1.2.3</version>
|
||||
</dependency>
|
||||
<dependency>
|
||||
<groupId>org.slf4j</groupId>
|
||||
<artifactId>slf4j-api</artifactId>
|
||||
<version>1.7.30</version>
|
||||
</dependency>
|
||||
|
||||
<dependency>
|
||||
<groupId>com.alibaba</groupId>
|
||||
<artifactId>fastjson</artifactId>
|
||||
<version>1.2.83</version>
|
||||
</dependency>
|
||||
|
||||
<dependency>
|
||||
<groupId>junit</groupId>
|
||||
<artifactId>junit</artifactId>
|
||||
<version>4.13.2</version>
|
||||
</dependency>
|
||||
|
||||
|
||||
|
||||
<!--语音识别模块-->
|
||||
<dependency>
|
||||
<groupId>cn.smartjavaai</groupId>
|
||||
<artifactId>smartjavaai-speech</artifactId>
|
||||
<exclusions>
|
||||
<exclusion>
|
||||
<groupId>com.microsoft.onnxruntime</groupId>
|
||||
<artifactId>onnxruntime</artifactId>
|
||||
</exclusion>
|
||||
<exclusion>
|
||||
<groupId>org.openpnp</groupId>
|
||||
<artifactId>opencv</artifactId>
|
||||
</exclusion>
|
||||
<exclusion>
|
||||
<groupId>ai.djl.huggingface</groupId>
|
||||
<artifactId>tokenizers</artifactId>
|
||||
</exclusion>
|
||||
<exclusion>
|
||||
<groupId>ai.djl.ml.xgboost</groupId>
|
||||
<artifactId>xgboost</artifactId>
|
||||
</exclusion>
|
||||
</exclusions>
|
||||
</dependency>
|
||||
|
||||
<!--ffmpeg库 引用所有平台库-->
|
||||
<dependency>
|
||||
<groupId>ws.schild</groupId>
|
||||
<artifactId>jave-all-deps</artifactId>
|
||||
<version>3.5.0</version>
|
||||
</dependency>
|
||||
|
||||
|
||||
<!-- windows平台 (保留对应平台的配置,可以减小包大小)-->
|
||||
<!-- <dependency>-->
|
||||
<!-- <groupId>ws.schild</groupId>-->
|
||||
<!-- <artifactId>jave-nativebin-win64</artifactId>-->
|
||||
<!-- <version>3.5.0</version>-->
|
||||
<!-- </dependency>-->
|
||||
|
||||
|
||||
|
||||
<!-- linux x86 平台 (保留对应平台的配置,可以减小包大小)-->
|
||||
<!-- <dependency>-->
|
||||
<!-- <groupId>ws.schild</groupId>-->
|
||||
<!-- <artifactId>jave-nativebin-linux64</artifactId>-->
|
||||
<!-- <version>3.5.0</version>-->
|
||||
<!-- </dependency>-->
|
||||
|
||||
<!-- linux arm64 平台 (保留对应平台的配置,可以减小包大小)-->
|
||||
<!-- <dependency>-->
|
||||
<!-- <groupId>ws.schild</groupId>-->
|
||||
<!-- <artifactId>jave-nativebin-linux-arm64</artifactId>-->
|
||||
<!-- <version>3.5.0</version>-->
|
||||
<!-- </dependency>-->
|
||||
|
||||
<!-- linux arm32 平台 (保留对应平台的配置,可以减小包大小)-->
|
||||
<!-- <dependency>-->
|
||||
<!-- <groupId>ws.schild</groupId>-->
|
||||
<!-- <artifactId>jave-nativebin-linux-arm32</artifactId>-->
|
||||
<!-- <version>3.5.0</version>-->
|
||||
<!-- </dependency>-->
|
||||
|
||||
|
||||
<!-- macOS osx64 平台 (保留对应平台的配置,可以减小包大小)-->
|
||||
<!-- <dependency>-->
|
||||
<!-- <groupId>ws.schild</groupId>-->
|
||||
<!-- <artifactId>jave-nativebin-osx64</artifactId>-->
|
||||
<!-- <version>3.5.0</version>-->
|
||||
<!-- </dependency>-->
|
||||
|
||||
<!-- macOS M系列 平台 (保留对应平台的配置,可以减小包大小)-->
|
||||
<!-- <dependency>-->
|
||||
<!-- <groupId>ws.schild</groupId>-->
|
||||
<!-- <artifactId>jave-nativebin-osxm1</artifactId>-->
|
||||
<!-- <version>3.5.0</version>-->
|
||||
<!-- </dependency>-->
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
</dependencies>
|
||||
|
||||
<build>
|
||||
<plugins>
|
||||
<plugin>
|
||||
<groupId>org.apache.maven.plugins</groupId>
|
||||
<artifactId>maven-shade-plugin</artifactId>
|
||||
<version>3.5.0</version>
|
||||
<executions>
|
||||
<execution>
|
||||
<phase>package</phase>
|
||||
<goals><goal>shade</goal></goals>
|
||||
<configuration>
|
||||
<createDependencyReducedPom>false</createDependencyReducedPom>
|
||||
<transformers>
|
||||
<transformer implementation="org.apache.maven.plugins.shade.resource.ServicesResourceTransformer"/>
|
||||
<transformer implementation="org.apache.maven.plugins.shade.resource.ManifestResourceTransformer">
|
||||
<mainClass>${exec.mainClass}</mainClass>
|
||||
</transformer>
|
||||
</transformers>
|
||||
</configuration>
|
||||
</execution>
|
||||
</executions>
|
||||
</plugin>
|
||||
</plugins>
|
||||
</build>
|
||||
|
||||
<repositories>
|
||||
<repository>
|
||||
<id>aliyunmaven</id>
|
||||
<name>阿里云公共仓库</name>
|
||||
<url>https://maven.aliyun.com/repository/public</url>
|
||||
<releases>
|
||||
<enabled>true</enabled>
|
||||
</releases>
|
||||
<snapshots>
|
||||
<enabled>false</enabled>
|
||||
</snapshots>
|
||||
</repository>
|
||||
</repositories>
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
</project>
|
||||
@@ -0,0 +1,311 @@
|
||||
package smartai.examples.speech.asr;
|
||||
|
||||
import ai.djl.util.JsonUtils;
|
||||
import cn.hutool.core.io.FileUtil;
|
||||
import cn.smartjavaai.common.entity.Language;
|
||||
import cn.smartjavaai.common.entity.R;
|
||||
import cn.smartjavaai.speech.asr.config.AsrModelConfig;
|
||||
import cn.smartjavaai.speech.asr.entity.AsrResult;
|
||||
import cn.smartjavaai.speech.asr.entity.VoskParams;
|
||||
import cn.smartjavaai.speech.asr.entity.WhisperParams;
|
||||
import cn.smartjavaai.speech.asr.enums.AsrModelEnum;
|
||||
import cn.smartjavaai.speech.asr.factory.SpeechRecognizerFactory;
|
||||
import cn.smartjavaai.speech.asr.model.SpeechRecognizer;
|
||||
import cn.smartjavaai.speech.asr.model.VoskRecognizer;
|
||||
import cn.smartjavaai.speech.asr.model.WhisperRecognizer;
|
||||
import io.github.givimad.whisperjni.WhisperFullParams;
|
||||
import io.github.givimad.whisperjni.WhisperGrammar;
|
||||
import io.github.givimad.whisperjni.WhisperSamplingStrategy;
|
||||
import io.github.givimad.whisperjni.WhisperState;
|
||||
import lombok.extern.slf4j.Slf4j;
|
||||
import org.apache.commons.io.FileUtils;
|
||||
import org.junit.Test;
|
||||
import org.vosk.Recognizer;
|
||||
|
||||
import javax.sound.sampled.AudioSystem;
|
||||
import java.io.BufferedInputStream;
|
||||
import java.io.File;
|
||||
import java.io.FileInputStream;
|
||||
import java.io.InputStream;
|
||||
import java.nio.file.Paths;
|
||||
|
||||
/**
|
||||
*
|
||||
* 语音识别demo
|
||||
* @author dwj
|
||||
* @date 2025/8/6
|
||||
*/
|
||||
@Slf4j
|
||||
public class SpeechRecognizeDemo {
|
||||
|
||||
/**
|
||||
* 获取Whisper模型
|
||||
* 模型下载网盘:通过网盘分享的文件:https://pan.baidu.com/s/1kiMF5MF641R7LTn1GpB2lQ?pwd=1234 提取码: 1234
|
||||
* 更多模型下载地址:https://huggingface.co/ggerganov/whisper.cpp/tree/main
|
||||
* @return
|
||||
*/
|
||||
public SpeechRecognizer getWhisperRecognizer() {
|
||||
AsrModelConfig config = new AsrModelConfig();
|
||||
config.setModelEnum(AsrModelEnum.WHISPER);
|
||||
//模型下载地址:https://huggingface.co/ggerganov/whisper.cpp/tree/main
|
||||
config.setModelPath("/Users/xxx/Documents/develop/model/speech/ggml-medium.bin");
|
||||
return SpeechRecognizerFactory.getInstance().getModel(config);
|
||||
}
|
||||
|
||||
|
||||
/**
|
||||
* Whisper 语音识别
|
||||
* 多语言模型支持100种语言
|
||||
* 注意事项:
|
||||
* 1、不支持centos7
|
||||
* 2、模型越大越准确
|
||||
* 3、暂不支持GPU使用,如需GPU使用需要自行编译:https://github.com/ggml-org/whisper.cpp/tree/master?tab=readme-ov-file#nvidia-gpu-support
|
||||
*/
|
||||
@Test
|
||||
public void testWhisper() {
|
||||
try {
|
||||
SpeechRecognizer recognizer = getWhisperRecognizer();
|
||||
WhisperParams params = new WhisperParams();
|
||||
//语言:中文
|
||||
params.setLanguage(Language.ZH);
|
||||
R<AsrResult> result = recognizer.recognize("src/main/resources/speech_zh.mp3", params);
|
||||
if (result.isSuccess()){
|
||||
log.info("识别成功:{}", JsonUtils.toJson(result.getData()));
|
||||
}else{
|
||||
log.info("识别失败:{}", result.getMessage());
|
||||
}
|
||||
} catch (Exception e) {
|
||||
e.printStackTrace();
|
||||
}
|
||||
}
|
||||
|
||||
/**
|
||||
* Whisper 语音识别(使用个性化配置)
|
||||
* 多语言模型支持100种语言
|
||||
* 注意事项:
|
||||
* 1、不支持centos7
|
||||
* 2、模型越大越准确
|
||||
* 3、暂不支持GPU使用,如需GPU使用需要自行编译:https://github.com/ggml-org/whisper.cpp/tree/master?tab=readme-ov-file#nvidia-gpu-support
|
||||
*/
|
||||
@Test
|
||||
public void testWhisperWithCustomConfig() {
|
||||
try {
|
||||
SpeechRecognizer recognizer = getWhisperRecognizer();
|
||||
WhisperParams params = new WhisperParams();
|
||||
//语言:中文
|
||||
params.setLanguage(Language.ZH);
|
||||
/**
|
||||
* 解码搜索策略类型:
|
||||
* GREEDY - 贪婪解码,逐步选择概率最高的结果;
|
||||
* BEAN_SEARCH - Beam 搜索,保留多个候选路径以提高准确性。
|
||||
*/
|
||||
WhisperFullParams fullParams = new WhisperFullParams(WhisperSamplingStrategy.BEAN_SEARCH);
|
||||
//语言
|
||||
fullParams.language = Language.ZH.getCode();
|
||||
//线程数,设为 0 表示使用最大核心数。
|
||||
fullParams.nThreads = 0;
|
||||
//解码器使用的历史文本作为提示的最大 token 数。
|
||||
fullParams.nMaxTextCtx = 16384;
|
||||
//解码起始偏移(毫秒)
|
||||
fullParams.offsetMs = 0;
|
||||
//解码持续时长(毫秒),超过此长度的音频将被截断
|
||||
fullParams.durationMs = 0;
|
||||
//是否翻译为英文
|
||||
fullParams.translate = false;
|
||||
// 初始提示,用于提供上下文或样例,帮助模型更准确地理解语音内容
|
||||
fullParams.initialPrompt = "简体中文";
|
||||
//禁用上下文链接,不使用前一段解码结果作为上下文
|
||||
fullParams.noContext = true;
|
||||
//是否强制仅输出一个段落(适用于短语音)
|
||||
fullParams.singleSegment = false;
|
||||
//是否打印特殊标记
|
||||
fullParams.printSpecial = false;
|
||||
//是否直接从 whisper.cpp 中打印结果(不推荐,建议使用回调方式替代)
|
||||
fullParams.printRealtime = false;
|
||||
//抑制非语音 token输出
|
||||
fullParams.suppressNonSpeechTokens = false;
|
||||
//更多参数请查看官网:https://github.com/GiviMAD/whisper-jni/blob/33854520b1f0b3697106a7932a2fd64e8191bca9/src/main/java/io/github/givimad/whisperjni/WhisperFullParams.java
|
||||
params.setParams(fullParams);
|
||||
//建议上传 WAV 格式音频。其他格式将自动转换为 WAV,可能影响处理速度
|
||||
R<AsrResult> result = recognizer.recognize("src/main/resources/speech_zh.mp3", params);
|
||||
if (result.isSuccess()){
|
||||
log.info("识别成功:{}", JsonUtils.toJson(result.getData()));
|
||||
}else{
|
||||
log.info("识别失败:{}", result.getMessage());
|
||||
}
|
||||
} catch (Exception e) {
|
||||
e.printStackTrace();
|
||||
}
|
||||
}
|
||||
|
||||
/**
|
||||
* Whisper 语音识别(使用Grammar语法规则)
|
||||
* 多语言模型支持100种语言
|
||||
* 注意事项:
|
||||
* 1、不支持centos7
|
||||
* 2、模型越大越准确
|
||||
* 3、暂不支持GPU使用,如需GPU使用需要自行编译:https://github.com/ggml-org/whisper.cpp/tree/master?tab=readme-ov-file#nvidia-gpu-support
|
||||
*/
|
||||
@Test
|
||||
public void testWhisperWithGrammar() {
|
||||
try {
|
||||
WhisperRecognizer whisperRecognizer = (WhisperRecognizer)getWhisperRecognizer();
|
||||
//语法规则
|
||||
String grammarText = "root ::= \" And so, my fellow American, ask not what your country can do for you, ask what you can do for your country.\"";
|
||||
try (WhisperGrammar grammar = whisperRecognizer.parseGrammar(grammarText)){
|
||||
WhisperParams params = new WhisperParams();
|
||||
WhisperFullParams fullParams = new WhisperFullParams(WhisperSamplingStrategy.BEAN_SEARCH);
|
||||
//语言:英文
|
||||
fullParams.language = Language.EN.getCode();
|
||||
fullParams.grammar = grammar;
|
||||
params.setParams(fullParams);
|
||||
//建议上传 WAV 格式音频。其他格式将自动转换为 WAV,可能影响处理速度
|
||||
R<AsrResult> result = whisperRecognizer.recognize("src/main/resources/jfk_en.wav", params);
|
||||
if (result.isSuccess()){
|
||||
log.info("识别成功:{}", JsonUtils.toJson(result.getData()));
|
||||
}else{
|
||||
log.info("识别失败:{}", result.getMessage());
|
||||
}
|
||||
} catch (Exception e) {
|
||||
throw new RuntimeException(e);
|
||||
}
|
||||
|
||||
} catch (Exception e) {
|
||||
e.printStackTrace();
|
||||
}
|
||||
}
|
||||
|
||||
|
||||
/**
|
||||
* 获取Vosk模型(中文)
|
||||
* 模型下载网盘:通过网盘分享的文件:https://pan.baidu.com/s/1kiMF5MF641R7LTn1GpB2lQ?pwd=1234 提取码: 1234
|
||||
* 更多模型下载地址:https://alphacephei.com/vosk/models
|
||||
* @return
|
||||
*/
|
||||
public SpeechRecognizer geVoskRecognizer() {
|
||||
AsrModelConfig config = new AsrModelConfig();
|
||||
config.setModelEnum(AsrModelEnum.VOSK);
|
||||
/**
|
||||
* 每个模型只支持一种语言,请下载对应语音的模型,模型下载地址:https://alphacephei.com/vosk/models
|
||||
* 将模型解压后,将模型目录位置填写到此处
|
||||
*/
|
||||
config.setModelPath("/Users/xxx/Documents/develop/model/speech/vosk-model-cn-0.22");
|
||||
/**
|
||||
* macos m系列芯片需要手动下载依赖库,并指定位置,其他平台不需要
|
||||
* 下载地址:https://pan.baidu.com/s/1LZ_EX1XdTTp_f5ruud82MA?pwd=1234 提取码: 1234
|
||||
*/
|
||||
// config.setLibPath(Paths.get("/Users/xxx/Downloads/vosk-arrch64-dylib-main/libvosk.dylib"));
|
||||
return SpeechRecognizerFactory.getInstance().getModel(config);
|
||||
}
|
||||
|
||||
/**
|
||||
* 获取Vosk模型(英文)
|
||||
* 模型下载网盘:通过网盘分享的文件:https://pan.baidu.com/s/1kiMF5MF641R7LTn1GpB2lQ?pwd=1234 提取码: 1234
|
||||
* 更多模型下载地址:https://alphacephei.com/vosk/models
|
||||
* @return
|
||||
*/
|
||||
public SpeechRecognizer geEnVoskRecognizer() {
|
||||
AsrModelConfig config = new AsrModelConfig();
|
||||
config.setModelEnum(AsrModelEnum.VOSK);
|
||||
/**
|
||||
* 每个模型只支持一种语言,请下载对应语音的模型,模型下载地址:https://alphacephei.com/vosk/models
|
||||
* 将模型解压后,将模型目录位置填写到此处
|
||||
*/
|
||||
config.setModelPath("/Users/xxx/Documents/develop/model/speech/vosk-model-small-en-us-0.15");
|
||||
// config.setLibPath(Paths.get("/Users/xxx/Downloads/vosk-arrch64-dylib-main/libvosk.dylib"));
|
||||
return SpeechRecognizerFactory.getInstance().getModel(config);
|
||||
}
|
||||
|
||||
|
||||
/**
|
||||
* Vosk 语音识别
|
||||
* 支持 20 多种语言和方言——英语、印度英语、德语、法语、西班牙语、葡萄牙语、中文、俄语、土耳其语、越南语、意大利语、荷兰语、加泰罗尼亚语、阿拉伯语、希腊语、波斯语、菲律宾语、乌克兰语、哈萨克语、瑞典语、日语、世界语、印地语、捷克语、波兰语等
|
||||
* 注意事项:
|
||||
* 1、每个模型只支持一种语言,请下载对应语言的模型
|
||||
* 2、如果音频中存在多种语言,不推荐使用vosk,可以使用Whisper
|
||||
* 3、模型越大越准确
|
||||
* 4、暂不支持GPU使用,如需GPU使用需要自行编译:https://alphacephei.com/vosk/install
|
||||
*/
|
||||
@Test
|
||||
public void testVosk() {
|
||||
try {
|
||||
SpeechRecognizer recognizer = geVoskRecognizer();
|
||||
//建议上传 WAV 格式音频。其他格式将自动转换为 WAV,可能影响处理速度
|
||||
R<AsrResult> result = recognizer.recognize("src/main/resources/lff_zh.mp3");
|
||||
if (result.isSuccess()){
|
||||
log.info("识别成功:{}", JsonUtils.toJson(result.getData()));
|
||||
}else{
|
||||
log.info("识别失败:{}", result.getMessage());
|
||||
}
|
||||
} catch (Exception e) {
|
||||
e.printStackTrace();
|
||||
}
|
||||
}
|
||||
|
||||
/**
|
||||
* Vosk 语音识别(使用Grammar语法规则)
|
||||
* 支持 20 多种语言和方言——英语、印度英语、德语、法语、西班牙语、葡萄牙语、中文、俄语、土耳其语、越南语、意大利语、荷兰语、加泰罗尼亚语、阿拉伯语、希腊语、波斯语、菲律宾语、乌克兰语、哈萨克语、瑞典语、日语、世界语、印地语、捷克语、波兰语等
|
||||
* 注意事项:
|
||||
* 1、每个模型只支持一种语言,请下载对应语言的模型
|
||||
* 2、如果音频中存在多种语言,不推荐使用vosk,可以使用Whisper
|
||||
* 3、模型越大越准确
|
||||
* 4、暂不支持GPU使用,如需GPU使用需要自行编译:https://alphacephei.com/vosk/install
|
||||
*/
|
||||
@Test
|
||||
public void testVoskWithGrammar() {
|
||||
try {
|
||||
//获取英文模型
|
||||
SpeechRecognizer recognizer = geEnVoskRecognizer();
|
||||
VoskParams voskParams = new VoskParams();
|
||||
//英文
|
||||
voskParams.setLanguage(Language.EN);
|
||||
voskParams.setGrammar("[\"one two three four five six seven eight nine zero oh\"]");
|
||||
//建议上传 WAV 格式音频。其他格式将自动转换为 WAV,可能影响处理速度
|
||||
R<AsrResult> result = recognizer.recognize("src/main/resources/test_en.wav",voskParams);
|
||||
if (result.isSuccess()){
|
||||
log.info("识别成功:{}", JsonUtils.toJson(result.getData()));
|
||||
}else{
|
||||
log.info("识别失败:{}", result.getMessage());
|
||||
}
|
||||
} catch (Exception e) {
|
||||
e.printStackTrace();
|
||||
}
|
||||
}
|
||||
|
||||
/**
|
||||
* Vosk 语音识别(使用Vosk内部识别器)
|
||||
* 支持 20 多种语言和方言——英语、印度英语、德语、法语、西班牙语、葡萄牙语、中文、俄语、土耳其语、越南语、意大利语、荷兰语、加泰罗尼亚语、阿拉伯语、希腊语、波斯语、菲律宾语、乌克兰语、哈萨克语、瑞典语、日语、世界语、印地语、捷克语、波兰语等
|
||||
* 注意事项:
|
||||
* 1、每个模型只支持一种语言,请下载对应语言的模型
|
||||
* 2、如果音频中存在多种语言,不推荐使用vosk,可以使用Whisper
|
||||
* 3、模型越大越准确
|
||||
* 4、暂不支持GPU使用,如需GPU使用需要自行编译:https://alphacephei.com/vosk/install
|
||||
*/
|
||||
@Test
|
||||
public void testVoskAdvanced() {
|
||||
try {
|
||||
VoskRecognizer recognizer = (VoskRecognizer)geVoskRecognizer();
|
||||
//使用vosk内部接口,需要指定识别音频的采样率
|
||||
Recognizer voskRecognizer = recognizer.createAdvancedRecognizer(16000);
|
||||
voskRecognizer.setWords(true);
|
||||
voskRecognizer.setPartialWords(true);
|
||||
// 使用vosk内部接口,只支持wav格式
|
||||
String audioPath = "src/main/resources/lff_zh.wav";
|
||||
InputStream ais = AudioSystem.getAudioInputStream(new BufferedInputStream(new FileInputStream(audioPath)));
|
||||
int nbytes;
|
||||
byte[] b = new byte[4096];
|
||||
while ((nbytes = ais.read(b)) >= 0) {
|
||||
if (voskRecognizer.acceptWaveForm(b, nbytes)) {
|
||||
log.info(voskRecognizer.getResult());
|
||||
} else {
|
||||
log.info(voskRecognizer.getPartialResult());
|
||||
}
|
||||
}
|
||||
log.info(voskRecognizer.getFinalResult());
|
||||
} catch (Exception e) {
|
||||
e.printStackTrace();
|
||||
}
|
||||
}
|
||||
|
||||
|
||||
}
|
||||
@@ -0,0 +1,3 @@
|
||||
Manifest-Version: 1.0
|
||||
Main-Class: smartai.examples.face.SeetaFace6LinuxDemo
|
||||
|
||||
BIN
examples/speech-examples/src/main/resources/jfk_en.wav
Normal file
BIN
examples/speech-examples/src/main/resources/jfk_en.wav
Normal file
Binary file not shown.
BIN
examples/speech-examples/src/main/resources/lff_zh.mp3
Normal file
BIN
examples/speech-examples/src/main/resources/lff_zh.mp3
Normal file
Binary file not shown.
BIN
examples/speech-examples/src/main/resources/lff_zh.wav
Normal file
BIN
examples/speech-examples/src/main/resources/lff_zh.wav
Normal file
Binary file not shown.
14
examples/speech-examples/src/main/resources/logback.xml
Normal file
14
examples/speech-examples/src/main/resources/logback.xml
Normal file
@@ -0,0 +1,14 @@
|
||||
<?xml version="1.0" encoding="UTF-8"?>
|
||||
<!-- 步骤2: 配置文件 (src/main/resources/logback.xml) -->
|
||||
<configuration scan="true" scanPeriod="30 seconds">
|
||||
<!-- 控制台日志输出 -->
|
||||
<appender name="CONSOLE" class="ch.qos.logback.core.ConsoleAppender">
|
||||
<encoder>
|
||||
<pattern>%d{yyyy-MM-dd HH:mm:ss.SSS} [%thread] %highlight(%-5level) %cyan(%logger{36}) - %msg%n</pattern>
|
||||
</encoder>
|
||||
</appender>
|
||||
|
||||
<root level="DEBUG">
|
||||
<appender-ref ref="CONSOLE" />
|
||||
</root>
|
||||
</configuration>
|
||||
BIN
examples/speech-examples/src/main/resources/speech_zh.mp3
Normal file
BIN
examples/speech-examples/src/main/resources/speech_zh.mp3
Normal file
Binary file not shown.
BIN
examples/speech-examples/src/main/resources/test_en.wav
Normal file
BIN
examples/speech-examples/src/main/resources/test_en.wav
Normal file
Binary file not shown.
@@ -12,7 +12,7 @@
|
||||
<maven.compiler.source>11</maven.compiler.source>
|
||||
<maven.compiler.target>11</maven.compiler.target>
|
||||
<project.build.sourceEncoding>UTF-8</project.build.sourceEncoding>
|
||||
<smartjavaai.version>1.0.22</smartjavaai.version>
|
||||
<smartjavaai.version>1.0.23</smartjavaai.version>
|
||||
<!--如果打包运行,需要替换成你的main-->
|
||||
<exec.mainClass>smartai.examples.nlp.translation.TranslationDemo</exec.mainClass>
|
||||
|
||||
|
||||
@@ -1,6 +1,7 @@
|
||||
package smartai.examples.nlp.translation;
|
||||
|
||||
import ai.djl.util.JsonUtils;
|
||||
import cn.smartjavaai.common.config.Config;
|
||||
import cn.smartjavaai.common.entity.R;
|
||||
import cn.smartjavaai.common.enums.DeviceEnum;
|
||||
import cn.smartjavaai.translation.config.TranslationModelConfig;
|
||||
@@ -11,8 +12,11 @@ import cn.smartjavaai.translation.factory.TranslationModelFactory;
|
||||
import cn.smartjavaai.translation.model.TranslationModel;
|
||||
import lombok.Data;
|
||||
import lombok.extern.slf4j.Slf4j;
|
||||
import org.junit.BeforeClass;
|
||||
import org.junit.Test;
|
||||
|
||||
import java.io.IOException;
|
||||
|
||||
/**
|
||||
* 翻译Demo
|
||||
* 支持 Meta AI 开源的 NLLB-200 模型,实现 200 多种语言之间的高质量互译。
|
||||
@@ -23,6 +27,12 @@ import org.junit.Test;
|
||||
@Slf4j
|
||||
public class TranslationDemo {
|
||||
|
||||
@BeforeClass
|
||||
public static void beforeAll() throws IOException {
|
||||
//修改缓存路径
|
||||
// Config.setCachePath("/Users/xxx/smartjavaai_cache");
|
||||
}
|
||||
|
||||
|
||||
/**
|
||||
* 翻译
|
||||
|
||||
33
pom.xml
33
pom.xml
@@ -7,7 +7,7 @@
|
||||
<name>SmartJavaAI</name>
|
||||
<groupId>cn.smartjavaai</groupId>
|
||||
<artifactId>smartjavaai-parent</artifactId>
|
||||
<version>1.0.22</version>
|
||||
<version>1.0.23</version>
|
||||
<packaging>pom</packaging>
|
||||
<description>SmartJavaAI</description>
|
||||
<modules>
|
||||
@@ -170,11 +170,11 @@
|
||||
<artifactId>opencv</artifactId>
|
||||
</dependency>
|
||||
|
||||
<dependency>
|
||||
<groupId>ai.djl.ml.xgboost</groupId>
|
||||
<artifactId>xgboost</artifactId>
|
||||
<scope>runtime</scope>
|
||||
</dependency>
|
||||
<!-- <dependency>-->
|
||||
<!-- <groupId>ai.djl.ml.xgboost</groupId>-->
|
||||
<!-- <artifactId>xgboost</artifactId>-->
|
||||
<!-- <scope>runtime</scope>-->
|
||||
<!-- </dependency>-->
|
||||
<dependency>
|
||||
<groupId>ai.djl.tensorrt</groupId>
|
||||
<artifactId>tensorrt</artifactId>
|
||||
@@ -281,6 +281,27 @@
|
||||
</execution>
|
||||
</executions>
|
||||
</plugin>
|
||||
|
||||
<!-- 让编译/测试插件使用 toolchain 选择的 JDK -->
|
||||
<plugin>
|
||||
<groupId>org.apache.maven.plugins</groupId>
|
||||
<artifactId>maven-toolchains-plugin</artifactId>
|
||||
<version>3.2.0</version>
|
||||
<executions>
|
||||
<execution>
|
||||
<goals>
|
||||
<goal>toolchain</goal>
|
||||
</goals>
|
||||
<configuration>
|
||||
<toolchains>
|
||||
<jdk>
|
||||
<version>1.8</version>
|
||||
</jdk>
|
||||
</toolchains>
|
||||
</configuration>
|
||||
</execution>
|
||||
</executions>
|
||||
</plugin>
|
||||
</plugins>
|
||||
</build>
|
||||
|
||||
|
||||
@@ -6,11 +6,11 @@
|
||||
<parent>
|
||||
<groupId>cn.smartjavaai</groupId>
|
||||
<artifactId>smartjavaai-parent</artifactId>
|
||||
<version>1.0.22</version>
|
||||
<version>1.0.23</version>
|
||||
</parent>
|
||||
|
||||
<artifactId>smartjavaai-all</artifactId>
|
||||
<version>1.0.22</version>
|
||||
<version>1.0.23</version>
|
||||
<name>${project.artifactId}</name>
|
||||
<description>SmartJavaAI</description>
|
||||
<url>https://github.com/geekwenjie/SmartJavaAI</url>
|
||||
@@ -55,6 +55,12 @@
|
||||
<version>${project.version}</version>
|
||||
</dependency>
|
||||
|
||||
<dependency>
|
||||
<groupId>cn.smartjavaai</groupId>
|
||||
<artifactId>smartjavaai-speech</artifactId>
|
||||
<version>${project.version}</version>
|
||||
</dependency>
|
||||
|
||||
<!-- 引用所有平台包 -->
|
||||
<!-- <dependency>-->
|
||||
<!-- <groupId>org.bytedeco</groupId>-->
|
||||
|
||||
@@ -6,10 +6,10 @@
|
||||
<parent>
|
||||
<groupId>cn.smartjavaai</groupId>
|
||||
<artifactId>smartjavaai-parent</artifactId>
|
||||
<version>1.0.22</version>
|
||||
<version>1.0.23</version>
|
||||
</parent>
|
||||
|
||||
<version>1.0.22</version>
|
||||
<version>1.0.23</version>
|
||||
<artifactId>smartjavaai-bom</artifactId>
|
||||
<name>smartjavaai-bom</name>
|
||||
<description>统一版本管理的 BOM 包,同时支持 import 和全量依赖</description>
|
||||
@@ -43,6 +43,11 @@
|
||||
<artifactId>smartjavaai-translate</artifactId>
|
||||
<version>${project.parent.version}</version>
|
||||
</dependency>
|
||||
<dependency>
|
||||
<groupId>cn.smartjavaai</groupId>
|
||||
<artifactId>smartjavaai-speech</artifactId>
|
||||
<version>${project.parent.version}</version>
|
||||
</dependency>
|
||||
</dependencies>
|
||||
</dependencyManagement>
|
||||
|
||||
|
||||
@@ -6,7 +6,7 @@
|
||||
<parent>
|
||||
<groupId>cn.smartjavaai</groupId>
|
||||
<artifactId>smartjavaai-parent</artifactId>
|
||||
<version>1.0.22</version>
|
||||
<version>1.0.23</version>
|
||||
</parent>
|
||||
|
||||
<name>smartjavaai-common</name>
|
||||
|
||||
@@ -83,4 +83,5 @@ public class Config {
|
||||
}
|
||||
|
||||
|
||||
|
||||
}
|
||||
|
||||
@@ -53,7 +53,7 @@ public class R<T> {
|
||||
public enum Status {
|
||||
SUCCESS(0, "成功"),
|
||||
INVALID_IMAGE(1, "图像无效"),
|
||||
FILE_NOT_FOUND(2, "图像文件不存在"),
|
||||
FILE_NOT_FOUND(2, "文件不存在"),
|
||||
NO_FACE_DETECTED(3, "未检测到人脸"),
|
||||
PARAM_ERROR(4, "参数错误"),
|
||||
INVALID_VIDEO(5, "视频无效"),
|
||||
|
||||
@@ -6,11 +6,11 @@
|
||||
<parent>
|
||||
<groupId>cn.smartjavaai</groupId>
|
||||
<artifactId>smartjavaai-parent</artifactId>
|
||||
<version>1.0.22</version>
|
||||
<version>1.0.23</version>
|
||||
</parent>
|
||||
|
||||
<artifactId>smartjavaai-face</artifactId>
|
||||
<version>1.0.22</version>
|
||||
<version>1.0.23</version>
|
||||
<name>smartjavaai-face</name>
|
||||
<description>SmartJavaAI</description>
|
||||
<url>https://github.com/geekwenjie/SmartJavaAI</url>
|
||||
|
||||
@@ -6,11 +6,11 @@
|
||||
<parent>
|
||||
<groupId>cn.smartjavaai</groupId>
|
||||
<artifactId>smartjavaai-parent</artifactId>
|
||||
<version>1.0.22</version>
|
||||
<version>1.0.23</version>
|
||||
</parent>
|
||||
|
||||
<artifactId>smartjavaai-objectdetection</artifactId>
|
||||
<version>1.0.22</version>
|
||||
<version>1.0.23</version>
|
||||
<name>smartjavaai-objectdetection</name>
|
||||
<description>SmartJavaAI</description>
|
||||
<url>https://github.com/geekwenjie/SmartJavaAI</url>
|
||||
|
||||
@@ -6,7 +6,7 @@
|
||||
<parent>
|
||||
<groupId>cn.smartjavaai</groupId>
|
||||
<artifactId>smartjavaai-parent</artifactId>
|
||||
<version>1.0.22</version>
|
||||
<version>1.0.23</version>
|
||||
</parent>
|
||||
|
||||
<artifactId>smartjavaai-ocr</artifactId>
|
||||
@@ -42,7 +42,7 @@
|
||||
</dependency>
|
||||
</dependencies>
|
||||
|
||||
<version>1.0.22</version>
|
||||
<version>1.0.23</version>
|
||||
<name>smartjavaai-ocr</name>
|
||||
<description>SmartJavaAI</description>
|
||||
<url>https://github.com/geekwenjie/SmartJavaAI</url>
|
||||
|
||||
@@ -36,6 +36,7 @@ import java.awt.image.BufferedImage;
|
||||
import java.io.ByteArrayInputStream;
|
||||
import java.io.File;
|
||||
import java.io.IOException;
|
||||
import java.io.OutputStream;
|
||||
import java.nio.file.Paths;
|
||||
import java.util.*;
|
||||
import java.util.List;
|
||||
@@ -180,6 +181,25 @@ public class TableRecognizer {
|
||||
ImageUtils.saveImage(image, savePath);
|
||||
}
|
||||
|
||||
|
||||
/**
|
||||
* 绘制表格
|
||||
* @param tableStructureResult
|
||||
* @param image
|
||||
* @return
|
||||
*/
|
||||
public BufferedImage drawTable(TableStructureResult tableStructureResult, BufferedImage image){
|
||||
if(Objects.isNull(tableStructureResult) || CollectionUtils.isEmpty(tableStructureResult.getTableTagList())){
|
||||
throw new OcrException("表格结构为空");
|
||||
}
|
||||
for (int i = 0; i < tableStructureResult.getOcrItemList().size(); i++){
|
||||
OcrItem item = tableStructureResult.getOcrItemList().get(i);
|
||||
DetectionRectangle detectionRectangle = item.getOcrBox().toDetectionRectangle();
|
||||
ImageUtils.drawImageRectWithText(image, detectionRectangle, i + "", Color.RED);
|
||||
}
|
||||
return image;
|
||||
}
|
||||
|
||||
/**
|
||||
* 删除 HTML 中第一个 <style> ... </style> 段落
|
||||
* @param html 原始 HTML
|
||||
@@ -200,17 +220,34 @@ public class TableRecognizer {
|
||||
return html.substring(0, styleStart) + html.substring(styleEnd);
|
||||
}
|
||||
|
||||
|
||||
/**
|
||||
* 导出 Excel
|
||||
* @param html
|
||||
* @param out
|
||||
*/
|
||||
public void exportExcel(String html, OutputStream out){
|
||||
String content = removeStyleBlock(html);
|
||||
content = content.replace("<html><body>", "");
|
||||
content = content.replace("</body></html>", "");
|
||||
try (HSSFWorkbook workbook = ConvertHtml2Excel.table2Excel(content)){
|
||||
workbook.write(out);
|
||||
out.flush();
|
||||
} catch (Exception e) {
|
||||
throw new OcrException("导出excel失败,请检查表结构是否识别正确");
|
||||
}
|
||||
}
|
||||
|
||||
/**
|
||||
* 导出 Excel
|
||||
* @param html
|
||||
* @param savePath
|
||||
*/
|
||||
public void exportExcel(String html, String savePath){
|
||||
try {
|
||||
String content = removeStyleBlock(html);
|
||||
content = content.replace("<html><body>", "");
|
||||
content = content.replace("</body></html>", "");
|
||||
HSSFWorkbook workbook = ConvertHtml2Excel.table2Excel(content);
|
||||
String content = removeStyleBlock(html);
|
||||
content = content.replace("<html><body>", "");
|
||||
content = content.replace("</body></html>", "");
|
||||
try (HSSFWorkbook workbook = ConvertHtml2Excel.table2Excel(content)){
|
||||
workbook.write(new File(savePath));
|
||||
} catch (Exception e) {
|
||||
throw new OcrException("导出excel失败,请检查表结构是否识别正确");
|
||||
|
||||
@@ -6,7 +6,7 @@
|
||||
<parent>
|
||||
<groupId>cn.smartjavaai</groupId>
|
||||
<artifactId>smartjavaai-parent</artifactId>
|
||||
<version>1.0.22</version>
|
||||
<version>1.0.23</version>
|
||||
</parent>
|
||||
|
||||
<artifactId>smartjavaai-speech</artifactId>
|
||||
@@ -48,16 +48,10 @@
|
||||
<artifactId>jave-core</artifactId>
|
||||
<version>3.5.0</version>
|
||||
</dependency>
|
||||
|
||||
<dependency>
|
||||
<groupId>ws.schild</groupId>
|
||||
<artifactId>jave-nativebin-osxm1</artifactId>
|
||||
<version>3.5.0</version>
|
||||
</dependency>
|
||||
</dependencies>
|
||||
|
||||
|
||||
<version>1.0.22</version>
|
||||
<version>1.0.23</version>
|
||||
<name>smartjavaai-speech</name>
|
||||
<description>SmartJavaAI</description>
|
||||
<url>https://github.com/geekwenjie/SmartJavaAI</url>
|
||||
@@ -70,6 +64,15 @@
|
||||
|
||||
<build>
|
||||
<plugins>
|
||||
<plugin>
|
||||
<groupId>org.apache.maven.plugins</groupId>
|
||||
<artifactId>maven-compiler-plugin</artifactId>
|
||||
<version>3.10.1</version>
|
||||
<configuration>
|
||||
<source>${maven.compiler.source}</source>
|
||||
<target>${maven.compiler.target}</target>
|
||||
</configuration>
|
||||
</plugin>
|
||||
<plugin>
|
||||
<groupId>org.sonatype.central</groupId>
|
||||
<artifactId>central-publishing-maven-plugin</artifactId>
|
||||
@@ -129,6 +132,25 @@
|
||||
</execution>
|
||||
</executions>
|
||||
</plugin>
|
||||
<plugin>
|
||||
<groupId>org.apache.maven.plugins</groupId>
|
||||
<artifactId>maven-toolchains-plugin</artifactId>
|
||||
<version>3.2.0</version>
|
||||
<executions>
|
||||
<execution>
|
||||
<goals>
|
||||
<goal>toolchain</goal>
|
||||
</goals>
|
||||
<configuration>
|
||||
<toolchains>
|
||||
<jdk>
|
||||
<version>11</version>
|
||||
</jdk>
|
||||
</toolchains>
|
||||
</configuration>
|
||||
</execution>
|
||||
</executions>
|
||||
</plugin>
|
||||
</plugins>
|
||||
</build>
|
||||
|
||||
|
||||
@@ -4,7 +4,10 @@ import cn.smartjavaai.common.config.ModelConfig;
|
||||
import cn.smartjavaai.speech.asr.enums.AsrModelEnum;
|
||||
import lombok.Data;
|
||||
|
||||
import java.nio.file.Path;
|
||||
|
||||
/**
|
||||
* Asr模型配置
|
||||
* @author dwj
|
||||
* @date 2025/7/31
|
||||
*/
|
||||
@@ -14,4 +17,9 @@ public class AsrModelConfig extends ModelConfig {
|
||||
private AsrModelEnum modelEnum;
|
||||
|
||||
private String modelPath;
|
||||
|
||||
/**
|
||||
* 依赖库目录
|
||||
*/
|
||||
private Path libPath;
|
||||
}
|
||||
|
||||
@@ -12,19 +12,9 @@ import lombok.Data;
|
||||
public class VoskParams extends RecParams{
|
||||
|
||||
/**
|
||||
* 最大候选结果数
|
||||
*/
|
||||
private int maxAlternatives;
|
||||
|
||||
/**
|
||||
* 限定词汇表 例:["yes", "no", "hello"]
|
||||
* 限定词汇表
|
||||
*/
|
||||
private String grammar;
|
||||
|
||||
/**
|
||||
* 是否返回词级别的识别结果(包含每个词的开始/结束时间和置信度)。
|
||||
*/
|
||||
private boolean words = true;
|
||||
|
||||
|
||||
}
|
||||
|
||||
@@ -0,0 +1,103 @@
|
||||
package cn.smartjavaai.speech.asr.factory;
|
||||
|
||||
import cn.smartjavaai.common.config.Config;
|
||||
import cn.smartjavaai.speech.asr.config.AsrModelConfig;
|
||||
import cn.smartjavaai.speech.asr.enums.AsrModelEnum;
|
||||
import cn.smartjavaai.speech.asr.exception.AsrException;
|
||||
import cn.smartjavaai.speech.asr.model.SpeechRecognizer;
|
||||
import cn.smartjavaai.speech.asr.model.VoskRecognizer;
|
||||
import cn.smartjavaai.speech.asr.model.WhisperRecognizer;
|
||||
import lombok.extern.slf4j.Slf4j;
|
||||
|
||||
import java.util.Map;
|
||||
import java.util.Objects;
|
||||
import java.util.concurrent.ConcurrentHashMap;
|
||||
|
||||
/**
|
||||
* 语音识别模型工厂
|
||||
* @author dwj
|
||||
*/
|
||||
@Slf4j
|
||||
public class SpeechRecognizerFactory {
|
||||
|
||||
// 使用 volatile 和双重检查锁定来确保线程安全的单例模式
|
||||
private static volatile SpeechRecognizerFactory instance;
|
||||
|
||||
/**
|
||||
* 模型缓存
|
||||
*/
|
||||
private static final ConcurrentHashMap<AsrModelEnum, SpeechRecognizer> modelMap = new ConcurrentHashMap<>();
|
||||
|
||||
/**
|
||||
* 模型注册表
|
||||
*/
|
||||
private static final Map<AsrModelEnum, Class<? extends SpeechRecognizer>> registry =
|
||||
new ConcurrentHashMap<>();
|
||||
|
||||
|
||||
public static SpeechRecognizerFactory getInstance() {
|
||||
if (instance == null) {
|
||||
synchronized (SpeechRecognizerFactory.class) {
|
||||
if (instance == null) {
|
||||
instance = new SpeechRecognizerFactory();
|
||||
}
|
||||
}
|
||||
}
|
||||
return instance;
|
||||
}
|
||||
|
||||
|
||||
|
||||
/**
|
||||
* 注册模型
|
||||
* @param expressionModelEnum
|
||||
* @param clazz
|
||||
*/
|
||||
private static void registerModel(AsrModelEnum expressionModelEnum, Class<? extends SpeechRecognizer> clazz) {
|
||||
registry.put(expressionModelEnum, clazz);
|
||||
}
|
||||
|
||||
|
||||
/**
|
||||
* 获取模型(通过配置)
|
||||
* @param config
|
||||
* @return
|
||||
*/
|
||||
public SpeechRecognizer getModel(AsrModelConfig config) {
|
||||
if(Objects.isNull(config) || Objects.isNull(config.getModelEnum())){
|
||||
throw new AsrException("未配置语音识别模型枚举");
|
||||
}
|
||||
return modelMap.computeIfAbsent(config.getModelEnum(), k -> {
|
||||
return createFaceModel(config);
|
||||
});
|
||||
}
|
||||
|
||||
/**
|
||||
* 使用ModelConfig创建模型
|
||||
* @param config
|
||||
* @return
|
||||
*/
|
||||
private SpeechRecognizer createFaceModel(AsrModelConfig config) {
|
||||
Class<?> clazz = registry.get(config.getModelEnum());
|
||||
if(clazz == null){
|
||||
throw new AsrException("Unsupported model");
|
||||
}
|
||||
SpeechRecognizer model = null;
|
||||
try {
|
||||
model = (SpeechRecognizer) clazz.newInstance();
|
||||
} catch (InstantiationException | IllegalAccessException e) {
|
||||
throw new AsrException(e);
|
||||
}
|
||||
model.loadModel(config);
|
||||
return model;
|
||||
}
|
||||
|
||||
|
||||
// 初始化默认算法
|
||||
static {
|
||||
registerModel(AsrModelEnum.WHISPER, WhisperRecognizer.class);
|
||||
registerModel(AsrModelEnum.VOSK, VoskRecognizer.class);
|
||||
log.debug("缓存目录:{}", Config.getCachePath());
|
||||
}
|
||||
|
||||
}
|
||||
@@ -13,9 +13,6 @@ import cn.smartjavaai.speech.asr.pool.WhisperStatePool;
|
||||
import cn.smartjavaai.speech.utils.AudioUtils;
|
||||
import com.google.gson.JsonArray;
|
||||
import com.google.gson.JsonObject;
|
||||
import io.github.givimad.whisperjni.WhisperContext;
|
||||
import io.github.givimad.whisperjni.WhisperJNI;
|
||||
import io.github.givimad.whisperjni.WhisperState;
|
||||
import lombok.extern.slf4j.Slf4j;
|
||||
import org.apache.commons.lang3.StringUtils;
|
||||
import org.vosk.LibVosk;
|
||||
@@ -39,6 +36,8 @@ import java.nio.file.Path;
|
||||
import java.nio.file.Paths;
|
||||
import java.util.ArrayList;
|
||||
import java.util.List;
|
||||
import java.util.Objects;
|
||||
import java.util.stream.Collectors;
|
||||
|
||||
import static ai.djl.util.JsonUtils.GSON;
|
||||
|
||||
@@ -61,6 +60,10 @@ public class VoskRecognizer implements SpeechRecognizer{
|
||||
throw new AsrException("Missing model file: " + testModelPath.toAbsolutePath());
|
||||
}
|
||||
try {
|
||||
//加载自定义依赖库
|
||||
if(Objects.nonNull(config.getLibPath())){
|
||||
System.load(config.getLibPath().toAbsolutePath().toString());
|
||||
}
|
||||
model = new Model(config.getModelPath());
|
||||
LibVosk.setLogLevel(LogLevel.DEBUG);
|
||||
log.debug("Vosk init success");
|
||||
@@ -86,7 +89,7 @@ public class VoskRecognizer implements SpeechRecognizer{
|
||||
return recognize(audioStream, new VoskParams());
|
||||
}
|
||||
|
||||
private R<AsrResult> recognizeAudioStream(AudioInputStream ais,RecParams params) {
|
||||
private R<AsrResult> recognizeAudioStream(AudioInputStream ais, RecParams params) {
|
||||
try (Recognizer recognizer = buildRecognizer(params, ais.getFormat().getSampleRate())){
|
||||
AudioFormat audioFormat = ais.getFormat();
|
||||
log.debug("sampleRate:{}", audioFormat.getSampleRate());
|
||||
@@ -95,14 +98,49 @@ public class VoskRecognizer implements SpeechRecognizer{
|
||||
byte[] b = new byte[4096];
|
||||
List<AsrSegment> segments = new ArrayList<AsrSegment>();
|
||||
StringBuilder text = new StringBuilder();
|
||||
String temp = "";
|
||||
while ((nbytes = ais.read(b)) >= 0) {
|
||||
if (recognizer.acceptWaveForm(b, nbytes)) {
|
||||
String result = recognizer.getResult();
|
||||
AsrSegment segment = parseSegment(result);
|
||||
segments.add(segment);
|
||||
// log.info("result:{}", result);
|
||||
AsrSegment segment = parseSegment(result, params);
|
||||
if(segment != null){
|
||||
segments.add(segment);
|
||||
text.append(segment.getText());
|
||||
}
|
||||
}else{
|
||||
temp = recognizer.getPartialResult();
|
||||
// log.info("temp:{}", temp);
|
||||
}
|
||||
}
|
||||
return R.ok(new AsrResult(text.toString(), segments));
|
||||
if(StringUtils.isNotBlank(temp)){
|
||||
AsrSegment segment = parsePartialSegment(temp, params);
|
||||
if(segment != null){
|
||||
segments.add(segment);
|
||||
text.append(segment.getText());
|
||||
}
|
||||
}
|
||||
//补全结果
|
||||
String finalText = recognizer.getFinalResult();
|
||||
if(StringUtils.isNotBlank(text.toString()) && StringUtils.isNotBlank(finalText)){
|
||||
AsrSegment finalSegment = parseSegment(finalText, params);
|
||||
if(finalSegment != null){
|
||||
//需要补全
|
||||
if(!text.toString().endsWith(finalSegment.getText())){
|
||||
AsrSegment alignSegment = VoskRecognizer.alignSegment(segments.get(segments.size() - 1), finalSegment);
|
||||
//如果匹配失败,则直接使用最终片段
|
||||
if(alignSegment != null){
|
||||
segments.set(segments.size() - 1,alignSegment);
|
||||
}else{
|
||||
segments.set(segments.size() - 1,finalSegment);
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
String result = segments.stream()
|
||||
.map(AsrSegment::getText)
|
||||
.collect(Collectors.joining("\n"));
|
||||
return R.ok(new AsrResult(result, segments));
|
||||
} catch (IOException e) {
|
||||
throw new AsrException(e);
|
||||
}
|
||||
@@ -113,18 +151,65 @@ public class VoskRecognizer implements SpeechRecognizer{
|
||||
* @param segment
|
||||
* @return
|
||||
*/
|
||||
private AsrSegment parseSegment(String segment) {
|
||||
private AsrSegment parseSegment(String segment, RecParams params) {
|
||||
JsonObject json = GSON.fromJson(segment, JsonObject.class);
|
||||
JsonArray resultArray = json.getAsJsonArray("result");
|
||||
if(Objects.isNull(resultArray) || resultArray.size() == 0){
|
||||
return null;
|
||||
}
|
||||
double segmentStart = resultArray.get(0).getAsJsonObject().get("start").getAsDouble();
|
||||
double segmentEnd = resultArray.get(resultArray.size() - 1).getAsJsonObject().get("end").getAsDouble();
|
||||
long startMs = Math.round(segmentStart * 1000);
|
||||
long endMs = Math.round(segmentEnd * 1000);
|
||||
String text = json.get("text").getAsString();
|
||||
String noSpaces = text.replace(" ", "");
|
||||
return new AsrSegment(noSpaces, startMs, endMs);
|
||||
if(Objects.nonNull(params.getLanguage()) && params.getLanguage() == Language.ZH){
|
||||
text = text.replace(" ", "");
|
||||
}
|
||||
return new AsrSegment(text, startMs, endMs);
|
||||
}
|
||||
|
||||
/**
|
||||
* 解析结果
|
||||
* @param segment
|
||||
* @return
|
||||
*/
|
||||
private AsrSegment parsePartialSegment(String segment, RecParams params) {
|
||||
JsonObject json = GSON.fromJson(segment, JsonObject.class);
|
||||
JsonArray resultArray = json.getAsJsonArray("partial_result");
|
||||
if(Objects.isNull(resultArray) || resultArray.size() == 0){
|
||||
return null;
|
||||
}
|
||||
double segmentStart = resultArray.get(0).getAsJsonObject().get("start").getAsDouble();
|
||||
double segmentEnd = resultArray.get(resultArray.size() - 1).getAsJsonObject().get("end").getAsDouble();
|
||||
long startMs = Math.round(segmentStart * 1000);
|
||||
long endMs = Math.round(segmentEnd * 1000);
|
||||
String text = json.get("partial").getAsString();
|
||||
if(Objects.nonNull(params.getLanguage()) && params.getLanguage() == Language.ZH){
|
||||
text = text.replace(" ", "");
|
||||
}
|
||||
return new AsrSegment(text, startMs, endMs);
|
||||
}
|
||||
|
||||
/**
|
||||
* 补全
|
||||
* @param shortSeg
|
||||
* @param longSeg
|
||||
* @return
|
||||
*/
|
||||
public static AsrSegment alignSegment(AsrSegment shortSeg, AsrSegment longSeg) {
|
||||
String shortText = shortSeg.getText();
|
||||
String longText = longSeg.getText();
|
||||
|
||||
int index = longText.indexOf(shortText);
|
||||
if (index == -1) {
|
||||
// log.debug("短文本不在长文本中");
|
||||
return null;
|
||||
}
|
||||
String resultText = longText.substring(index);
|
||||
return new AsrSegment(resultText, shortSeg.getStartTime(), longSeg.getEndTime());
|
||||
}
|
||||
|
||||
|
||||
/**
|
||||
* 创建识别器
|
||||
* @param params
|
||||
@@ -146,8 +231,8 @@ public class VoskRecognizer implements SpeechRecognizer{
|
||||
// }
|
||||
//暂时只支持返回一个结果
|
||||
// recognizer.setMaxAlternatives(1);
|
||||
recognizer.setWords(voskParams.isWords());
|
||||
//recognizer.setPartialWords(true);
|
||||
recognizer.setWords(true);
|
||||
recognizer.setPartialWords(true);
|
||||
return recognizer;
|
||||
}
|
||||
|
||||
@@ -187,6 +272,7 @@ public class VoskRecognizer implements SpeechRecognizer{
|
||||
tryStream = new BufferedInputStream(new ByteArrayInputStream(allBytes));
|
||||
conversionStream = new BufferedInputStream(new ByteArrayInputStream(allBytes));
|
||||
ais = AudioSystem.getAudioInputStream(tryStream);
|
||||
return recognizeAudioStream(ais, params);
|
||||
} catch (UnsupportedAudioFileException e) {
|
||||
log.debug("Unsupported Audio file, Conversion to WAV is required");
|
||||
needConversion = true;
|
||||
@@ -207,6 +293,7 @@ public class VoskRecognizer implements SpeechRecognizer{
|
||||
if(needConversion){
|
||||
try {
|
||||
tempFile = AudioUtils.audioFormatConversion(conversionStream, "wav");
|
||||
// log.info("tempFile:{}", tempFile.getAbsolutePath());
|
||||
} catch (EncoderException | IOException e) {
|
||||
throw new AsrException(e);
|
||||
}
|
||||
@@ -228,7 +315,7 @@ public class VoskRecognizer implements SpeechRecognizer{
|
||||
}
|
||||
}
|
||||
}
|
||||
return recognizeAudioStream(ais, params);
|
||||
return R.fail(R.Status.Unknown);
|
||||
}
|
||||
|
||||
@Override
|
||||
|
||||
@@ -51,6 +51,10 @@ public class WhisperRecognizer implements SpeechRecognizer{
|
||||
throw new AsrException("Missing model file: " + testModelPath.toAbsolutePath());
|
||||
}
|
||||
try {
|
||||
//加载自定义依赖库
|
||||
if(Objects.nonNull(config.getLibPath())){
|
||||
System.setProperty("io.github.givimad.whisperjni.libdir",config.getLibPath().toAbsolutePath().toString());
|
||||
}
|
||||
WhisperJNI.loadLibrary();
|
||||
WhisperJNI.setLibraryLogger(null);
|
||||
whisper = new WhisperJNI();
|
||||
@@ -116,6 +120,9 @@ public class WhisperRecognizer implements SpeechRecognizer{
|
||||
StringBuilder text = new StringBuilder();
|
||||
try {
|
||||
WhisperParams whisperParams = (WhisperParams) params;
|
||||
if(Objects.isNull(whisperParams.getParams().language)){
|
||||
return R.fail(1003, "请指定语言");
|
||||
}
|
||||
//不是英语,需要检查是否是多语言模型
|
||||
if(!Language.EN.getCode().equals(whisperParams.getParams().language)){
|
||||
if(!whisper.isMultilingual(ctx)){
|
||||
@@ -190,7 +197,7 @@ public class WhisperRecognizer implements SpeechRecognizer{
|
||||
* 获取一个WhisperState对象
|
||||
* @return
|
||||
*/
|
||||
private WhisperState getWhisperState(){
|
||||
public WhisperState getWhisperState(){
|
||||
try {
|
||||
return statePool.borrowObject();
|
||||
} catch (Exception e) {
|
||||
|
||||
@@ -20,7 +20,7 @@
|
||||
</dependency>
|
||||
</dependencies>
|
||||
|
||||
<version>1.0.22</version>
|
||||
<version>1.0.23</version>
|
||||
<name>smartjavaai-translate</name>
|
||||
<description>SmartJavaAI</description>
|
||||
<url>https://github.com/geekwenjie/SmartJavaAI</url>
|
||||
|
||||
Reference in New Issue
Block a user