63 Commits

Author SHA1 Message Date
github-actions[bot]
2fea3d980e chore: Release v4.0.0rc2 2026-08-07 06:48:31 +00:00
yakhyo
bb8553f6b6 fix: Match CenterFace preprocessing to upstream 2026-08-07 15:32:40 +09:00
yakhyo
d42778296b fix: Keep BlazeFace boxes unclipped for MediaPipe parity 2026-08-07 14:26:52 +09:00
Yakhyokhuja Valikhujaev
4d6f29a113 feat: Release v4.0.0 (#130)
* docs: Minor info changes

* refactor: Remove deprecated factory functions

Removes the 10 factory functions deprecated in v3.7.0:
create_detector, list_available_detectors, create_recognizer,
create_face_parser, create_gaze_estimator, create_head_pose_estimator,
create_landmarker, create_matting_model, create_spoofer,
create_attribute_predictor.

Instantiate model classes directly instead, e.g.
`from uniface.detection import SCRFD; SCRFD()`.

Also drops the now-unused typing_extensions dependency and updates
docstring examples that referenced the factories.

* refactor: Remove MODEL_URLS and MODEL_SHA256 back-compat mirrors

Use MODEL_REGISTRY instead of the MODEL_URLS and MODEL_SHA256 dicts in
uniface.constants.

* chore: Add .claude directory to gitignore

* ci: Deploy docs via GitHub Pages artifact instead of gh-pages branch

* feat: Add FaceAttribNet face state prediction model

Integrate Qualcomm's FaceAttribNet (Facial-Attribute-Detection) as a new
attribute model predicting five independent binary face states from a
face crop: left/right eye openness, eyeglasses, mask, and sunglasses.

- Add FaceAttribNet class with letterbox preprocessing (normalization is
  baked into the ONNX graph) and multi-label FaceStateResult output
- Register FaceAttribNetWeights with dynamic-batch ONNX weights (opset 17)
- Enrich Face with left_eye_open, right_eye_open, eyeglasses, mask,
  and sunglasses fields
- Rename AttributeResult to DemographyResult and DDAMFNWeights to
  EmotionWeights so 'attribute' unambiguously means the model category
- Add tests and docs

* docs: Add face attributes notebook, demo assets, and README updates

* feat: Add CenterFace face detection model

* docs: Add CONTEXT.md glossary and ADRs for v4 naming decisions

ADR-0001: Face.landmarks keeps its v3 name for the 5-point set; the
future dense-landmark slot will be Face.dense_landmarks.
ADR-0002: Attribute stays the per-face predictor base class name;
FaceAnalyzer(attributes=...) is unchanged.

* refactor: Rename FaceAnalyzer attributes parameter to predictors

The pipeline parameter names the role (per-face predictors), so future
head pose, gaze, spoofing, and quality models can be passed without the
semantic mismatch of being listed as attributes. The Attribute base
class keeps its name per ADR-0002.

BREAKING CHANGE: FaceAnalyzer(attributes=...) is now
FaceAnalyzer(predictors=...); FaceAnalyzer.attributes is now
FaceAnalyzer.predictors.

* docs: Update docs and notebooks for predictors parameter rename

* docs: Remove CONTEXT.md

* docs: Restore README badges

* docs: Remove ADR folder

* refactor: Rename Attribute base class to BaseAttribute

* docs: Replace thick README separators with heading underlines

* docs: Put README tagline in a callout

* fix: Use predictors= in the analyze tool

* refactor: Make supports_landmarks a class attribute on BaseDetector

Replaces the private _supports_landmarks instance flag and its read-only
property with a plain class attribute, mirroring supports_alignment. Every
detector set the flag to True in __init__, so the False default never
survived and the hasattr guard in the property was dead weight.

Also drops a stale Attributes entry in RetinaFace for a field that no
longer exists, and removes comments that restated the call beneath them.

* docs: Normalize docstring markup and document missing raises

Audited every docstring and comment in uniface/ against the detection
package, which is the most internally consistent one.

- Replace 13 Sphinx roles and 159 RST double-backticks with markdown
  backticks; nothing renders docstrings, there is no Sphinx, and mkdocs
  has no mkdocstrings plugin
- Rewrite 59 legacy typing generics (List, Tuple, Optional, Union) to
  PEP 585, each checked against the real signature rather than
  mechanically substituted
- Add Raises sections to download_file, compute_similarity and
  parse_with_inverse, which raised undocumented exceptions
- Drop the RuntimeError Raises block from six abstract _initialize_model
  methods whose body raises NotImplementedError; the contract belongs to
  the implementer and is now stated as prose
- Fold Properties into Attributes with a Read-only prefix, singularize
  Examples, and fix the copyright year in faceattribnet

No executable line changed; the bulk pass rewrote only string and comment
tokens matched by byte offset.

* fix: Raise ValueError instead of asserting in estimate_norm

Both input checks used bare assert statements, which python -O strips.
The docstring promised AssertionError as part of the contract, so under an
optimized build the validation silently disappeared and an unsupported
image_size flowed into the alignment maths instead of failing fast.

Both are now ValueError, and the messages report the offending value.

* docs: Replace RST double-backticks in face_utils

* docs: Document Raises on every model class

The detection package documents ValueError and RuntimeError on the class
docstring; no model class in any other package did, even though they all
call verify_model_weights in __init__ and reach an _initialize_model that
wraps session creation in RuntimeError.

Verified per class before adding: each target reaches both raise sites,
inheriting _initialize_model from its base where it does not define one.

* fix: Replace the last two bare asserts with explicit raises

python -O strips assert statements, so both checks silently disappeared in
optimized builds. BaseRecognizer would then accept a multi-output model and
mis-slice its results, and get_meanface_info would build neighbour tables
from a meanface of the wrong length.

Behaviour is unchanged in normal builds: the recognizer check already sat
inside the try that re-raises as RuntimeError, and the meanface check now
raises the ValueError its docstring already documented.

No bare asserts remain in uniface/.

* style: Put multi-line docstring summaries on the first line

Card 6 of the docstring audit: class docstrings opened two different ways,
45 with a leading newline and 37 on the same line as the quotes.

Settled by the repo's own config rather than by taste. pyproject already
declares pydocstyle convention = google, under which ruff resolves D212
(multi-line-summary-first-line) as the active rule and explicitly ignores
the incompatible D213. Same-line was therefore already the declared
standard, even though the detection package -- the audit's exemplar -- sat
on the other side of the split.

Applied with ruff --fix, then the summaries that ended up glued to their
description got the blank line D205 wants.

* style: Put multi-line docstring summaries on the first line (constants, test_utils)

Same D212/D205 pass as the previous commit, applied to the two files that
were held back from it.

* build: Enforce D205 and D212 in ruff

The docstring audit found the opening style split 45/37 with nothing to
hold it in place: pyproject declared pydocstyle convention = google, but D
was absent from select, so the convention setting was inert.

Selecting the two rules the codebase now satisfies makes the decision
checkable instead of conventional. The rest of D is left off deliberately
-- D100 alone would flag 63 files, since this project's convention is to
have no module docstrings.

* feat: Add BlazeFace detector and MediaPipe Face Mesh landmarker

BlazeFace is MediaPipe's short-range SSD face detector: two anchor grids
over a 128x128 letterboxed image, decoded with MediaPipe's weighted NMS
where overlapping candidates are score-averaged rather than discarded.

It emits 6 MediaPipe keypoints, not the 5-point alignment template, so
BaseDetector gains a supports_alignment flag. It is True everywhere else
and False only here. FaceAnalyzer checks it and disables recognition with
a warning instead of producing broken embeddings, and estimate_norm's
error now names the constraint when a caller tries to align anyway.

FaceMesh predicts 468 dense 3D landmarks from a face crop and is
detector-agnostic: the ROI follows MediaPipe's recipe, a square region at
1.5x the detector box rotated so the eye line is horizontal, built from
whichever two eye points the detector provides. Every face in an image
runs in one session call. roi_from_box and warp_roi are public so
video-mode tracking can be built on top.

Also adds FaceMeshResult, draw_mesh with full and partial tessellations,
weight entries for both models, the facemesh and facestate tools, example
notebook 15, and tests for both models.

Tessellation tables are vendored from PINTO0309/facemesh_onnx_tensorrt
(Apache-2.0); provenance is recorded in docs/license-attribution.md.

* docs: List BlazeFace and Face Mesh in the remaining model indexes

The feature commit updated the detection and landmarks module pages, the
model zoo and the concept pages, but four summary listings still predated
both models:

- README examples table stopped at notebook 14
- README training-dataset table omitted both
- quickstart model summary omitted both
- index.md feature cards described detection as 5-point only, which
  BlazeFace's 6 keypoints contradict

* feat: Add MediaPipe Face Landmarker, a 478-point mesh with irises

Google's Face Landmarker is the successor to Face Mesh: one network predicting
478 landmarks from a 256x256 crop, where the first 468 keep the Face Mesh
topology and ordering and the last ten are the irises. It ships in the Tasks
API bundle; the earlier 478-point attention model is not ported, since it needs
three MediaPipe-only TFLite custom ops that no standard runtime can load.

FaceMesh needs no logic change to serve it. Input size and landmark count were
already read from the ONNX graph, so 192->256 and 468->478 are absorbed at load
time. Verified end to end at 0.007% of inter-ocular distance against MediaPipe's
own pipeline, with the existing ROI recipe unchanged - Google's model card
specifies the same 25% per-side margin the default already used.

Iris membership follows MediaPipe's own convention: the result stays a flat
array and the region lives in named constants next to the topology it indexes,
rather than becoming accessors on FaceMeshResult. draw_mesh renders the irises
distinctly, so the library and tools/facemesh.py agree by construction.

BREAKING CHANGE: FaceMeshWeights.DEFAULT is replaced by V1_468 and V2_478.
Default behaviour is unchanged - the constructor still loads the 468-point
model - but code naming DEFAULT explicitly must be updated. Every other
multi-variant weights enum already names its members descriptively.

* feat: Add Hugging Face mirror fallback for weight downloads

* docs: Document the Hugging Face mirror fallback

* fix: Resolve bugs found in whole-codebase review

* refactor: Publish validate_image and normalize line endings

* fix: Guarantee weighted NMS terminates

* refactor: Make model constructors keyword-only

* refactor: Read input_size from the ONNX model

* fix: Normalize and validate attribute input_size

* refactor: Replace detector **kwargs with explicit options

* fix: Reject unusable images in BlazeFace and FaceMesh

* feat: Make detector capability flags opt-in

* fix: Show every populated face state in Face repr

* docs: Clarify inline comments

* docs: Update docs and notebooks for v4

* refactor: Make remaining public constructors keyword-only

* feat: Export BaseAttribute and document custom predictors

* test: Cover FaceAnalyzer predictors

* fix: Pin the Hugging Face mirror to an immutable revision

* docs: Add changelog with v4 migration guide

* ci: Deploy release docs via the Pages artifact flow

* build: Require setuptools 77 for the SPDX license field

* docs: Fix anchors, dataset table, and tool options

* chore: Release v4.0.0rc1

---------

Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
2026-08-04 02:47:26 +09:00
Yakhyokhuja Valikhujaev
a79d113791 feat: Add RetinaFace with ResNet50 backbone (#128)
* Added RESNET50 for Retinaface

* chore: Change model hosting to uniface github release

---------

Co-authored-by: Bert <maxcooncat75@gmail.com>
2026-07-13 18:03:02 +09:00
github-actions[bot]
d1df387274 chore: Release v3.7.1 2026-06-16 13:42:27 +00:00
Yakhyokhuja Valikhujaev
1c74e5884f chore: Clean up redundant files and update README (#126) 2026-06-15 00:53:47 +09:00
Yakhyokhuja Valikhujaev
9484e9e6e1 fix: Fix download file function with writing to tempfile for downloading file (#125) 2026-06-14 23:29:56 +09:00
Yakhyokhuja Valikhujaev
b12c929f9d docs: Fix old documentation issues (#124) 2026-06-14 22:17:15 +09:00
github-actions[bot]
50c67507e3 chore: Release v3.7.0 2026-05-27 12:21:16 +00:00
github-actions[bot]
1584a9d857 chore: Release v3.7.0rc1 2026-05-26 14:36:21 +00:00
Yakhyokhuja Valikhujaev
446e3629fb feat: Add face image quality assessment (#122)
* feat: Add face image quality assessment functionality

* chore: Add hover animation for landing page components
2026-05-26 23:29:27 +09:00
Yakhyokhuja Valikhujaev
5af87dcd56 feat: Deprecate factory function and pre-commit update (#121)
* refactor: Align BGR contract for vis_parsing_maps and tighten public docstrings

* refactor: Deprecate warning for factory functions

* Update uv.lock

* feat: Catch changes to pyproject in pre-commit to keep  uv.lock updated
2026-05-25 00:44:27 +09:00
Yakhyokhuja Valikhujaev
7882ec5cb4 chore: Update docstrings and comments (#119) 2026-05-11 01:07:14 +09:00
dependabot[bot]
d51d030545 chore(deps): bump gitpython from 3.1.49 to 3.1.50 (#118)
Bumps [gitpython](https://github.com/gitpython-developers/GitPython) from 3.1.49 to 3.1.50.
- [Release notes](https://github.com/gitpython-developers/GitPython/releases)
- [Changelog](https://github.com/gitpython-developers/GitPython/blob/main/CHANGES)
- [Commits](https://github.com/gitpython-developers/GitPython/compare/3.1.49...3.1.50)

---
updated-dependencies:
- dependency-name: gitpython
  dependency-version: 3.1.50
  dependency-type: indirect
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2026-05-09 19:16:08 +09:00
github-actions[bot]
5a767847da chore: Release v3.6.0 2026-05-08 12:25:34 +00:00
github-actions[bot]
4a22f903f0 chore: Release v3.6.0rc2 2026-05-08 12:20:29 +00:00
Yakhyokhuja Valikhujaev
43a46e11df ci: Refresh uv.lock during release pipeline (#117) 2026-05-08 21:13:44 +09:00
github-actions[bot]
025b93ab8b chore: Release v3.6.0rc1 2026-05-08 03:27:30 +00:00
Yakhyokhuja Valikhujaev
8bf87d958f feat: Add PIPNet for facial landmarks detection (#116)
* docs: Add PipNet model documentation

* feat: Add PipNet for face landmark detection
2026-05-08 12:25:00 +09:00
Yakhyokhuja Valikhujaev
b813dc2ee7 ref: Update package mngt and optimize the vector store functions (#115)
* ref: Update download and hash chunk sizes to speed up

* build: Adopt uv with uv.lock and drop requirements.txt

* ref: Centralize softmax helper and minor cleanups
2026-05-06 01:47:27 +09:00
Yakhyokhuja Valikhujaev
73fc291930 ci: Resolve deprecation warnings in pipeline (#114)
* ci: Resolve deprecation warnings in pipeline
2026-04-28 00:52:46 +09:00
github-actions[bot]
400bb72217 chore: Release v3.5.3 2026-04-27 15:24:18 +00:00
Yakhyokhuja Valikhujaev
a0a12d5eca fix: Fix pypi publish re-run issue (#113) 2026-04-28 00:22:12 +09:00
github-actions[bot]
a34f376da0 chore: Release v3.5.2 2026-04-27 15:04:20 +00:00
Yakhyokhuja Valikhujaev
2b29706615 ci: Add end-to-end deployment pipeline and fix docs auto-trigger (#112) 2026-04-27 23:59:09 +09:00
github-actions[bot]
f6d3cf33f0 chore: Release v3.5.1 2026-04-27 11:53:21 +00:00
Yakhyokhuja Valikhujaev
0eb042425c chore: Minor changes to workflow names and docs (#111) 2026-04-27 20:51:50 +09:00
github-actions[bot]
35c0b6d539 chore: Release v3.5.1rc1 2026-04-25 15:17:54 +00:00
Yakhyokhuja Valikhujaev
13c4ac83d8 feat: Update the release workflow and package installation command (#110)
* fix: Fix installation conflict between onnxruntime and onnxruntime-gpu

* fix: Fix CI, notebooks, type hints, and packaging issues found in audit

* feat: Add new release config

* ci: Automate release pipeline and document release process
2026-04-25 23:59:00 +09:00
Yakhyokhuja Valikhujaev
6ce397b811 feat: Add MODNet portrait matting (#108)
* feat: Add MODNet portrait matting

* docs: Update docs and example of portrait matting

* fix: Fix linting issue
2026-04-11 23:30:32 +09:00
Yakhyokhuja Valikhujaev
9bf54f5f78 feat: Add EdgeFace recognition model (#105)
* refactor: Split recognition models into separate files

* feat: Add EdgeFace recognition model

* release: Bump version to v3.4.0
2026-04-04 20:11:28 +09:00
Yakhyokhuja Valikhujaev
c87ec1ad0f docs: Add example images and update MkDocs files (#104)
* chore: Add example inference results

* docs: Update MkDocs and README files
2026-04-04 18:28:27 +09:00
Yakhyokhuja Valikhujaev
9e56a86963 chore: Update docs and clean up notebook outputs before before commit (#102)
* chore: Add links for repo and docs on example notebooks

* ref: Compress jupyter notebook sizes

* ci: Add nbstripout pre-commit hook for notebook output stripping

* docs: Add coding agent docs and commit message tag
2026-04-03 10:10:51 +09:00
Yakhyokhuja Valikhujaev
426bd71505 release: Release UniFace v3.3.0 - Python 3.10 support, stores refactor, docs and examples refresh (#101)
* docs: Update docs and examples

* chore: Update tools folder testing for development

* feat: Update indexing to stores and drawing logic

* chore: Update the release version to 3.3.0

* feat: Add python 3.10 support

* build: Add python support for worklows and publishing

* chore: Update all example notebooks
2026-03-28 22:30:56 +09:00
LiberiFatali
ede8b27091 chore: Add example notebook for face recognition (#100) 2026-03-28 05:27:27 +09:00
Yakhyokhuja Valikhujaev
02c77ce5db feat: Add head pose estimation model (#99)
* feat: Add Head Pose Estimation  with 6 different models

* chore: Update jupyter notebook examples

* docs: Update head pose estimation related docs
2026-03-26 22:57:05 +09:00
Yakhyokhuja Valikhujaev
d70d6a254f ref: Unify attribute/detector base classes and fix tools reliability (#98)
* refactor: unify attribute API, deduplicate detectors, and fix embedding shape

* refactor: unify attribute API and deduplicate detector code

* chore: Update docs page build on tags and frame validation before flip
2026-03-25 23:43:56 +09:00
Yakhyokhuja Valikhujaev
7d37633b1a chore: drop Python 3.10 support, bump scikit-image to >=0.26.0 (#96) 2026-03-19 10:04:52 +09:00
Yakhyokhuja Valikhujaev
bc413df4a8 docs: Add release changelog markdown file (#92) 2026-03-19 09:46:16 +09:00
Marc-Antoine BERTIN
8db0577991 feat: Add Python 3.14 support (#95)
- Relax requires-python upper bound from <3.14 to <3.15
- Add Python 3.14 classifier to pyproject.toml
- Add Python 3.14 to CI test matrix (ubuntu-latest)
- Fix SimilarityTransform.estimate() deprecation warning (scikit-image >=0.26)
  by switching to SimilarityTransform.from_estimate() class constructor

All 147 tests pass on Python 3.14.3 with no warnings.

Co-authored-by: marc-antoine <marcantoine.bertin@storyzy.com>
2026-03-19 09:41:44 +09:00
Yakhyokhuja Valikhujaev
3682a2124f release: Release UniFace version v3.1.0 (#91)
* release: Release UniFace version v3.1.0

* docs: Change classifiers to stable from beta
2026-03-11 12:21:33 +09:00
Yakhyokhuja Valikhujaev
2ef6a1ebe8 refactor: Use dataclass-based model info in model management (#90)
- Refactor model management section: Using data classes for more robust model management.
2026-03-11 12:05:43 +09:00
Yakhyokhuja Valikhujaev
78a2dba7c7 feat: Add FAISS vectore database for fast face search (#88) 2026-03-05 22:46:03 +09:00
Yakhyokhuja Valikhujaev
87e496d1f5 feat: Add FAISS vector DB support for fast search (#86)
* feat: Add FAISS: VectorDB for face embedding search

* docs: Update Documentation
2026-03-03 12:12:05 +09:00
Yakhyokhuja Valikhujaev
5604ebf4f1 docs: Add datasets information in the docs (#85) 2026-02-18 16:02:37 +09:00
Yakhyokhuja Valikhujaev
971775b2e8 feat: Update API format and gaze estimation models (#82)
* docs: Update documentation

* fix: Update several missing docs and tests

* docs: Clean up and remove redundants

* fix: Fix the gaze output formula and change the output order

* chore: Update model weights for gaze estimation

* release: Update release version to v3.0.0
2026-02-14 23:54:51 +09:00
Yakhyokhuja Valikhujaev
c520ea2df2 faet: Add ByteTrack - Multi-Object Tracking by Associating Every Detection Box (#81)
* feat: Add BYTETrack for face/person tracking

* docs: Update documentation

* ref: Update tools folder file naming and imports

* docs: Update jupyter notebook examples

* ref: Rename the file and remove duplicate codes

* docs: Update README.md

* chore: Update description in mkdocs, add keywords for face tracking

* docs: Add announcement section

* feat: Remove expand bbox for tracking and update docs
2026-02-12 00:20:23 +09:00
Yakhyokhuja Valikhujaev
2a8cb54d31 feat: Add get and set for cache dir (#80) 2026-02-09 23:32:02 +09:00
Yakhyokhuja Valikhujaev
331f46be7c release: Update release version and docs (#79) 2026-02-05 21:45:28 +09:00
Yakhyokhuja Valikhujaev
9991fae62a docs: Update UniFace library documentation and README.md (#78)
* docs: Update wrong/missing references

* docs: Update README.md
2026-02-04 20:45:02 +09:00
Yakhyokhuja Valikhujaev
b74ab95d39 docs: Update UniFace github image (#75) 2026-01-25 17:07:40 +09:00
Yakhyokhuja Valikhujaev
d2b0303bfe docs: Add additional badges to README.md (#74)
* Update badges in README.md
* Update ci.yml
2026-01-24 22:25:09 +09:00
Yakhyokhuja Valikhujaev
5f74487eb3 feat: Add XSeg for Face Segmentation (#72)
* feat: Add XSeg for Face Segmentation DeepFaceLab

* docs: Update model inference related reference

* chore: Update jupyter notebook example for face segmentation
2026-01-22 22:33:31 +09:00
Yakhyokhuja Valikhujaev
f897482d26 release: Release UniFace v2.2.1 (#69) 2026-01-18 22:38:15 +09:00
Yakhyokhuja Valikhujaev
f3d81eb201 feat: Add providers for chosing inference backend (#68)
* feat: Add providers for chosing inference backend

* docs: Update Python version
2026-01-18 22:29:15 +09:00
Yakhyokhuja Valikhujaev
ea0b56f7e0 fix: Add cache dir check (#67) 2026-01-15 18:07:45 +09:00
Yakhyokhuja Valikhujaev
edbab5f7bf fix: use Python 3.11 in validate job for tomllib support (#65) 2026-01-07 00:29:48 +09:00
Yakhyokhuja Valikhujaev
cd8077e460 feat: Update release to v2.2.0 (#64) 2026-01-07 00:16:29 +09:00
Yakhyokhuja Valikhujaev
452b3381a2 Update badge links in README.md (#63) 2026-01-06 23:32:36 +09:00
Yakhyokhuja Valikhujaev
07c8bd7b24 feat: Add YOLOv8 Face Detection model support (#62)
* docs: Update UniFace documentation

* feat: Add YOLOv8 face detection model
2026-01-03 19:08:41 +09:00
Yakhyokhuja Valikhujaev
68179d1e2d feat: Add AdaFace: Quality Adaptive Margin for Face Recognition (#61)
* feat: Add AdaFace model

* release: Update release version to v2.1.0
2026-01-02 00:23:24 +09:00
Yakhyokhuja Valikhujaev
99b35dddb4 chore: Add google analytics (#57) 2025-12-31 19:45:49 +09:00
212 changed files with 20635 additions and 5495 deletions

Binary file not shown.

Before

Width:  |  Height:  |  Size: 716 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 673 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 826 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 563 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 33 KiB

BIN
.github/logos/uniface.png vendored Normal file

Binary file not shown.

After

Width:  |  Height:  |  Size: 1.7 MiB

BIN
.github/logos/uniface_rounded_q80.webp vendored Normal file

Binary file not shown.

After

Width:  |  Height:  |  Size: 62 KiB

View File

@@ -4,11 +4,9 @@ on:
push:
branches:
- main
- develop
pull_request:
branches:
- main
- develop
concurrency:
group: ${{ github.workflow }}-${{ github.ref }}
@@ -19,10 +17,10 @@ jobs:
runs-on: ubuntu-latest
timeout-minutes: 5
steps:
- uses: actions/checkout@v4
- uses: actions/setup-python@v5
- uses: actions/checkout@v5
- uses: actions/setup-python@v6
with:
python-version: '3.11'
python-version: "3.11"
- uses: pre-commit/action@v3.0.1
test:
@@ -33,33 +31,44 @@ jobs:
strategy:
fail-fast: false
matrix:
os: [ubuntu-latest, macos-latest, windows-latest]
python-version: ["3.11", "3.13"]
include:
# Full Python range on Linux (fastest runner)
- os: ubuntu-latest
python-version: "3.10"
- os: ubuntu-latest
python-version: "3.11"
- os: ubuntu-latest
python-version: "3.12"
- os: ubuntu-latest
python-version: "3.13"
- os: ubuntu-latest
python-version: "3.14"
- os: macos-latest
python-version: "3.13"
- os: windows-latest
python-version: "3.13"
steps:
- name: Checkout code
uses: actions/checkout@v4
uses: actions/checkout@v5
- name: Set up Python ${{ matrix.python-version }}
uses: actions/setup-python@v5
- name: Install uv
uses: astral-sh/setup-uv@v6
with:
enable-cache: true
python-version: ${{ matrix.python-version }}
cache: "pip"
- name: Install dependencies
run: |
python -m pip install --upgrade pip
python -m pip install .[dev]
run: uv sync --locked --extra cpu --extra dev
- name: Check ONNX Runtime providers
run: |
python -c "import onnxruntime as ort; print('Available providers:', ort.get_available_providers())"
run: uv run python -c "import onnxruntime as ort; print('Available providers:', ort.get_available_providers())"
- name: Run tests
run: pytest -v --tb=short
run: uv run pytest -v --tb=short
- name: Test package imports
run: python -c "import uniface; print(f'uniface {uniface.__version__} loaded with {len(uniface.__all__)} exports')"
run: uv run python -c "import uniface; print(f'uniface {uniface.__version__} loaded with {len(uniface.__all__)} exports')"
build:
runs-on: ubuntu-latest
@@ -68,10 +77,10 @@ jobs:
steps:
- name: Checkout code
uses: actions/checkout@v4
uses: actions/checkout@v5
- name: Set up Python
uses: actions/setup-python@v5
uses: actions/setup-python@v6
with:
python-version: "3.11"
cache: "pip"

View File

@@ -1,38 +1,50 @@
name: Deploy docs
name: Deploy Documentation
on:
push:
branches: [main]
workflow_dispatch:
permissions:
contents: write
contents: read
pages: write
id-token: write
concurrency:
group: pages
cancel-in-progress: false
jobs:
deploy:
runs-on: ubuntu-latest
environment:
name: github-pages
url: ${{ steps.deployment.outputs.page_url }}
steps:
- uses: actions/checkout@v4
- uses: actions/checkout@v5
with:
fetch-depth: 0 # Fetch full history for git-committers and git-revision-date plugins
fetch-depth: 0
- uses: actions/setup-python@v5
- name: Install uv
uses: astral-sh/setup-uv@v6
with:
enable-cache: true
python-version: "3.11"
- name: Install dependencies
run: |
python -m pip install --upgrade pip
pip install mkdocs-material pymdown-extensions mkdocs-git-committers-plugin-2 mkdocs-git-revision-date-localized-plugin
run: uv sync --locked --extra docs
- name: Build docs
env:
MKDOCS_GIT_COMMITTERS_APIKEY: ${{ secrets.MKDOCS_GIT_COMMITTERS_APIKEY }}
run: mkdocs build --strict
run: uv run mkdocs build --strict
- name: Configure GitHub Pages
uses: actions/configure-pages@v5
- name: Upload site artifact
uses: actions/upload-pages-artifact@v4
with:
path: ./site
- name: Deploy to GitHub Pages
uses: peaceiris/actions-gh-pages@v4
with:
github_token: ${{ secrets.GITHUB_TOKEN }}
publish_dir: ./site
destination_dir: docs
id: deployment
uses: actions/deploy-pages@v4

240
.github/workflows/pipeline.yml vendored Normal file
View File

@@ -0,0 +1,240 @@
name: Release Pipeline
on:
workflow_dispatch:
inputs:
version:
description: 'Version (e.g. 3.6.0, 3.6.0b1, 3.6.0rc1)'
required: true
concurrency:
group: pipeline
cancel-in-progress: false
jobs:
validate:
runs-on: ubuntu-latest
timeout-minutes: 5
outputs:
is_prerelease: ${{ steps.prerelease.outputs.is_prerelease }}
steps:
- name: Checkout code
uses: actions/checkout@v5
- name: Set up Python
uses: actions/setup-python@v6
with:
python-version: "3.11"
- name: Validate version (PEP 440)
run: |
python - <<'EOF'
import re, sys
v = "${{ inputs.version }}"
if not re.fullmatch(r'\d+\.\d+\.\d+((a|b|rc)\d+|\.dev\d+)?', v):
print(f"Invalid version: {v}")
print("Expected forms: 3.6.0, 3.6.0a1, 3.6.0b1, 3.6.0rc1, 3.6.0.dev1")
sys.exit(1)
EOF
- name: Check tag does not exist
run: |
if git rev-parse "v${{ inputs.version }}" >/dev/null 2>&1; then
echo "Tag v${{ inputs.version }} already exists."
exit 1
fi
- name: Detect pre-release
id: prerelease
run: |
if [[ "${{ inputs.version }}" =~ (a|b|rc|\.dev)[0-9]+ ]]; then
echo "is_prerelease=true" >> $GITHUB_OUTPUT
else
echo "is_prerelease=false" >> $GITHUB_OUTPUT
fi
test:
runs-on: ubuntu-latest
timeout-minutes: 15
needs: validate
strategy:
fail-fast: false
matrix:
python-version: ["3.10", "3.11", "3.12", "3.13", "3.14"]
steps:
- name: Checkout code
uses: actions/checkout@v5
- name: Install uv
uses: astral-sh/setup-uv@v6
with:
enable-cache: true
python-version: ${{ matrix.python-version }}
- name: Install dependencies
run: uv sync --locked --extra cpu --extra dev
- name: Run tests
run: uv run pytest -v --tb=short
release:
runs-on: ubuntu-latest
timeout-minutes: 5
needs: test
permissions:
contents: write
steps:
- name: Checkout code
uses: actions/checkout@v5
with:
fetch-depth: 0
token: ${{ secrets.RELEASE_TOKEN }}
- name: Set up Python
uses: actions/setup-python@v6
with:
python-version: "3.11"
- name: Update pyproject.toml
run: |
python - <<'EOF'
import re, pathlib
p = pathlib.Path('pyproject.toml')
text = p.read_text()
new = re.sub(r'^version\s*=\s*".*"', f'version = "${{ inputs.version }}"', text, count=1, flags=re.M)
if new == text:
raise SystemExit("Failed to update version in pyproject.toml")
p.write_text(new)
EOF
- name: Update uniface/__init__.py
run: |
python - <<'EOF'
import re, pathlib
p = pathlib.Path('uniface/__init__.py')
text = p.read_text()
new = re.sub(r"^__version__\s*=\s*'.*'", f"__version__ = '${{ inputs.version }}'", text, count=1, flags=re.M)
if new == text:
raise SystemExit("Failed to update __version__ in uniface/__init__.py")
p.write_text(new)
EOF
- name: Install uv
uses: astral-sh/setup-uv@v6
with:
enable-cache: true
python-version: "3.11"
- name: Refresh uv.lock with new project version
run: uv lock --upgrade-package uniface
- name: Commit, tag, push
run: |
git config user.name "github-actions[bot]"
git config user.email "41898282+github-actions[bot]@users.noreply.github.com"
git add pyproject.toml uniface/__init__.py uv.lock
git commit -m "chore: Release v${{ inputs.version }}"
git tag "v${{ inputs.version }}"
git push origin HEAD:${{ github.ref_name }}
git push origin "v${{ inputs.version }}"
publish:
runs-on: ubuntu-latest
timeout-minutes: 10
needs: [validate, release]
permissions:
contents: write
id-token: write
environment:
name: pypi
url: https://pypi.org/project/uniface/
steps:
- name: Checkout tag
uses: actions/checkout@v5
with:
ref: v${{ inputs.version }}
- name: Set up Python
uses: actions/setup-python@v6
with:
python-version: "3.11"
cache: 'pip'
- name: Install build tools
run: |
python -m pip install --upgrade pip
python -m pip install build twine
- name: Build package
run: python -m build
- name: Check package
run: twine check dist/*
- name: Publish to PyPI
env:
TWINE_USERNAME: __token__
TWINE_PASSWORD: ${{ secrets.PYPI_API_TOKEN }}
run: twine upload dist/*
- name: Create GitHub Release
uses: softprops/action-gh-release@v2
with:
tag_name: v${{ inputs.version }}
files: dist/*
generate_release_notes: true
prerelease: ${{ needs.validate.outputs.is_prerelease }}
docs:
runs-on: ubuntu-latest
timeout-minutes: 10
needs: [validate, publish]
if: needs.validate.outputs.is_prerelease == 'false'
permissions:
contents: read
pages: write
id-token: write
environment:
name: github-pages
url: ${{ steps.deployment.outputs.page_url }}
concurrency:
group: pages
cancel-in-progress: false
steps:
- name: Checkout tag
uses: actions/checkout@v5
with:
ref: v${{ inputs.version }}
fetch-depth: 0
- name: Install uv
uses: astral-sh/setup-uv@v6
with:
enable-cache: true
python-version: "3.11"
- name: Install dependencies
run: uv sync --locked --extra docs
- name: Build docs
env:
MKDOCS_GIT_COMMITTERS_APIKEY: ${{ secrets.MKDOCS_GIT_COMMITTERS_APIKEY }}
run: uv run mkdocs build --strict
- name: Configure GitHub Pages
uses: actions/configure-pages@v5
- name: Upload site artifact
uses: actions/upload-pages-artifact@v4
with:
path: ./site
- name: Deploy to GitHub Pages
id: deployment
uses: actions/deploy-pages@v4

View File

@@ -1,119 +0,0 @@
name: Publish to PyPI
on:
push:
tags:
- "v*.*.*" # Trigger only on version tags like v0.1.9
concurrency:
group: ${{ github.workflow }}-${{ github.ref }}
cancel-in-progress: true
jobs:
validate:
runs-on: ubuntu-latest
timeout-minutes: 5
outputs:
version: ${{ steps.get_version.outputs.version }}
tag_version: ${{ steps.get_version.outputs.tag_version }}
steps:
- name: Checkout code
uses: actions/checkout@v4
- name: Set up Python
uses: actions/setup-python@v5
with:
python-version: "3.11"
- name: Get version from tag and pyproject.toml
id: get_version
run: |
TAG_VERSION=${GITHUB_REF#refs/tags/v}
echo "tag_version=$TAG_VERSION" >> $GITHUB_OUTPUT
PYPROJECT_VERSION=$(python -c "import tomllib; print(tomllib.load(open('pyproject.toml','rb'))['project']['version'])")
echo "version=$PYPROJECT_VERSION" >> $GITHUB_OUTPUT
echo "Tag version: v$TAG_VERSION"
echo "pyproject.toml version: $PYPROJECT_VERSION"
- name: Verify version match
run: |
if [ "${{ steps.get_version.outputs.tag_version }}" != "${{ steps.get_version.outputs.version }}" ]; then
echo "Error: Tag version (${{ steps.get_version.outputs.tag_version }}) does not match pyproject.toml version (${{ steps.get_version.outputs.version }})"
exit 1
fi
echo "Version validation passed: ${{ steps.get_version.outputs.version }}"
test:
runs-on: ubuntu-latest
timeout-minutes: 15
needs: validate
strategy:
fail-fast: false
matrix:
python-version: ["3.11", "3.13"]
steps:
- name: Checkout code
uses: actions/checkout@v4
- name: Set up Python ${{ matrix.python-version }}
uses: actions/setup-python@v5
with:
python-version: ${{ matrix.python-version }}
cache: 'pip'
- name: Install dependencies
run: |
python -m pip install --upgrade pip
python -m pip install .[dev]
- name: Run tests
run: pytest -v
publish:
runs-on: ubuntu-latest
timeout-minutes: 10
needs: [validate, test]
permissions:
contents: write
id-token: write
environment:
name: pypi
url: https://pypi.org/project/uniface/
steps:
- name: Checkout code
uses: actions/checkout@v4
- name: Set up Python
uses: actions/setup-python@v5
with:
python-version: "3.10"
cache: 'pip'
- name: Install build tools
run: |
python -m pip install --upgrade pip
python -m pip install build twine
- name: Build package
run: python -m build
- name: Check package
run: twine check dist/*
- name: Publish to PyPI
env:
TWINE_USERNAME: __token__
TWINE_PASSWORD: ${{ secrets.PYPI_API_TOKEN }}
run: twine upload dist/*
- name: Create GitHub Release
uses: softprops/action-gh-release@v1
with:
files: dist/*
generate_release_notes: true

2
.gitignore vendored
View File

@@ -1,4 +1,6 @@
tmp_*
.vscode/
.claude
# Byte-compiled / optimized / DLL files
__pycache__/

View File

@@ -18,6 +18,13 @@ repos:
- id: debug-statements
- id: check-ast
# Strip Jupyter notebook outputs
- repo: https://github.com/kynan/nbstripout
rev: 0.9.1
hooks:
- id: nbstripout
files: ^examples/
# Ruff - Fast Python linter and formatter
- repo: https://github.com/astral-sh/ruff-pre-commit
rev: v0.14.10
@@ -35,6 +42,12 @@ repos:
additional_dependencies: ['bandit[toml]']
exclude: ^tests/
# Keep uv.lock in sync with pyproject.toml
- repo: https://github.com/astral-sh/uv-pre-commit
rev: 0.9.0
hooks:
- id: uv-lock
# Configuration
ci:
autofix_commit_msg: 'style: auto-fix by pre-commit hooks'

6
AGENTS.md Normal file
View File

@@ -0,0 +1,6 @@
<!-- Cursor agent instructions — shared with CLAUDE.md -->
<!-- See CLAUDE.md for full project instructions for AI coding agents. -->
# AGENTS.md
Please read and follow all instructions in [CLAUDE.md](./CLAUDE.md).

64
CHANGELOG.md Normal file
View File

@@ -0,0 +1,64 @@
# Changelog
Notable changes to UniFace are documented here. Earlier releases are covered by
the autogenerated notes on the [GitHub releases page](https://github.com/yakhyo/uniface/releases).
## 4.0.0 - 2026-08-03
### Breaking changes
- **Factory functions are gone.** `create_detector()` and its siblings no longer
exist. Construct model classes directly.
- **`FaceAnalyzer(attributes=...)` is now `predictors=`.** The parameter accepts
any list of `BaseAttribute` subclasses.
- **`Attribute` is renamed `BaseAttribute`**, matching `BaseDetector`,
`BaseRecognizer`, and the other base classes.
- **Constructors are keyword-only.** Every model class, plus `FaceAnalyzer`,
`BlurFace`, `EllipticalBlur`, `FAISS`, and `BYTETracker`, rejects positional
arguments. `SCRFD(model_name)` raises `TypeError`; write
`SCRFD(model_name=...)`.
- **Detector `**kwargs` replaced with explicit options.** `RetinaFace` takes
`pre_nms_topk`, `post_nms_topk`, and `dynamic_size`; `YOLOv5Face` and
`YOLOv8Face` take `max_det`. Misspelled options now fail instead of being
silently ignored.
- **`input_size` removed from `MobileGaze`, `HeadPose`, `BiSeNet`, and
`Landmark106`.** These models always resized to the size in the ONNX graph;
the parameter had no effect and is gone.
- **Inputs are validated.** Detectors and landmarkers require 3-channel `uint8`
BGR images. Float or grayscale arrays now raise `ValueError` instead of
silently returning no faces.
- **Detector capability flags are opt-in.** `supports_landmarks` and
`supports_alignment` default to `False` on `BaseDetector`. Third-party
detector subclasses that produce 5-point alignment landmarks must now declare
both flags; boxes-only subclasses need no declaration.
### Migrating from 3.x
| 3.x | 4.0 |
| --- | --- |
| `create_detector('retinaface', ...)` | `RetinaFace(...)` |
| `RetinaFace(RetinaFaceWeights.MNET_V2)` | `RetinaFace(model_name=RetinaFaceWeights.MNET_V2)` |
| `FaceAnalyzer(attributes=[AgeGender()])` | `FaceAnalyzer(predictors=[AgeGender()])` |
| `class MyPredictor(Attribute)` | `class MyPredictor(BaseAttribute)` |
| `RetinaFace(**{'pre_nms_topk': 1000})` | `RetinaFace(pre_nms_topk=1000)` |
| `HeadPose(input_size=(224, 224))` | `HeadPose()` |
### Added
- `BlazeFace` detector: MediaPipe short-range face detector with 6 keypoints.
- `CenterFace` detector.
- `FaceMesh` landmarker: MediaPipe dense 3D face mesh, 468 points or 478 with
irises (`FaceMeshWeights.V2_478`).
- `FaceAttribNet` predictor: eye openness, eyeglasses, sunglasses, and mask
probabilities as `FaceStateResult`, enriching `Face` in-place.
- Hugging Face mirror fallback for weight downloads when GitHub Releases is
unreachable, pinned to an immutable revision.
- `uniface.common.validate_image` is public for use in custom models.
### Fixed
- `BlazeFace` weighted NMS could loop forever with `iou_threshold=1.0` or a
zero-area box; it now always terminates.
- `Face.__repr__` no longer assumes all five face-state fields are set together.
- Attribute models normalize `input_size` and warn when it disagrees with the
ONNX metadata; `FaceAttribNet` rejects non-square sizes.

81
CLAUDE.md Normal file
View File

@@ -0,0 +1,81 @@
# CLAUDE.md
Project instructions for AI coding agents.
## Project Overview
UniFace is a Python library for face detection, recognition, tracking, landmark analysis, face parsing, gaze estimation, age/gender detection. It uses ONNX Runtime for inference.
## Code Style
- Python 3.10+ with type hints
- Line length: 120
- Single quotes for strings, double quotes for docstrings
- Google-style docstrings
- Formatter/linter: Ruff (config in `pyproject.toml`)
- Run `ruff format .` and `ruff check . --fix` before committing
## Commit Messages
Follow [Conventional Commits](https://www.conventionalcommits.org/) with a **capitalized** description:
```
<type>: <Capitalized short description>
```
Types: `feat`, `fix`, `docs`, `style`, `refactor`, `perf`, `test`, `build`, `ci`, `chore`
Examples:
- `feat: Add gaze estimation model`
- `fix: Correct bounding box scaling for non-square images`
- `ci: Add nbstripout pre-commit hook`
- `docs: Update installation instructions`
- `refactor: Unify attribute/detector base classes`
## Testing
```bash
pytest -v --tb=short
```
Tests live in `tests/`. Run the full suite before submitting changes.
## Pre-commit
Pre-commit hooks handle formatting, linting, security checks, and notebook output stripping. Always run:
```bash
pre-commit install
pre-commit run --all-files
```
## Project Structure
```
uniface/ # Main package
detection/ # Face detection models (SCRFD, RetinaFace, YOLOv5, YOLOv8)
recognition/ # Face recognition/verification (AdaFace, ArcFace, EdgeFace, MobileFace, SphereFace)
landmark/ # Facial landmark models
tracking/ # Object tracking (ByteTrack)
parsing/ # Face parsing/segmentation (BiSeNet, XSeg)
gaze/ # Gaze estimation
headpose/ # Head pose estimation
attribute/ # Age, gender, emotion detection
spoofing/ # Anti-spoofing (MiniFASNet)
privacy/ # Face anonymization
stores/ # Vector stores (FAISS)
constants.py # Model weight URLs and checksums
model_store.py # Model download/cache management
analyzer.py # High-level FaceAnalyzer API
types.py # Shared type definitions
tests/ # Unit tests
examples/ # Jupyter notebooks (outputs are auto-stripped)
docs/ # MkDocs documentation
```
## Key Conventions
- New models: add class in submodule, register weights in `constants.py`, export in `__init__.py`
- Dependencies: managed in `pyproject.toml`
- All ONNX models are downloaded on demand with SHA256 verification
- Do not commit notebook outputs; `nbstripout` pre-commit hook handles this

View File

@@ -21,25 +21,31 @@ Thank you for considering contributing to UniFace! We welcome contributions of a
## Development Setup
We use [uv](https://docs.astral.sh/uv/) for reproducible dev installs. The committed `uv.lock` pins every transitive dependency so contributors and CI resolve to identical versions.
```bash
# Install uv (https://docs.astral.sh/uv/getting-started/installation/)
curl -LsSf https://astral.sh/uv/install.sh | sh
git clone https://github.com/yakhyo/uniface.git
cd uniface
pip install -e ".[dev]"
# Sync runtime + cpu + dev extras from uv.lock (use --extra gpu instead of cpu for CUDA)
uv sync --extra cpu --extra dev
```
`uv sync` creates a project-local `.venv/` and installs everything pinned in `uv.lock`. Run commands with `uv run <cmd>` (e.g. `uv run pytest`), or activate the venv with `source .venv/bin/activate`.
### Setting Up Pre-commit Hooks
We use [pre-commit](https://pre-commit.com/) to ensure code quality and consistency. Install and configure it:
We use [pre-commit](https://pre-commit.com/) to ensure code quality and consistency. `pre-commit` is included in the `[dev]` extra, so it's already installed after `uv sync`.
```bash
# Install pre-commit
pip install pre-commit
# Install the git hooks
pre-commit install
uv run pre-commit install
# (Optional) Run against all files
pre-commit run --all-files
uv run pre-commit run --all-files
```
Once installed, pre-commit will automatically run on every commit to check:
@@ -59,12 +65,12 @@ This project uses [Ruff](https://docs.astral.sh/ruff/) for linting and formattin
#### General Rules
- **Line length:** 120 characters maximum
- **Python version:** 3.11+ (use modern syntax)
- **Python version:** 3.10+ (use modern syntax)
- **Quote style:** Single quotes for strings, double quotes for docstrings
#### Type Hints
Use modern Python 3.11+ type hints (PEP 585 and PEP 604):
Use modern Python 3.10+ type hints (PEP 585 and PEP 604):
```python
# Preferred (modern)
@@ -82,23 +88,23 @@ def process(items: List[str], config: Optional[Dict[str, int]] = None) -> Tuple[
Use [Google-style docstrings](https://google.github.io/styleguide/pyguide.html#38-comments-and-docstrings) for all public APIs:
```python
def detect_faces(image: np.ndarray, threshold: float = 0.5) -> list[Face]:
def detect(self, image: np.ndarray, **kwargs: Any) -> list[Face]:
"""Detect faces in an image.
Args:
image: Input image as a numpy array with shape (H, W, C) in BGR format.
threshold: Confidence threshold for filtering detections. Defaults to 0.5.
image: Input image as numpy array with shape (H, W, C) in BGR format.
**kwargs: Additional detection parameters.
Returns:
List of Face objects containing bounding boxes, confidence scores,
and facial landmarks.
List of detected Face objects.
Raises:
ValueError: If the input image has invalid dimensions.
ValueError: If the image is empty, not 3-channel BGR, or not uint8.
Example:
>>> from uniface import detect_faces
>>> faces = detect_faces(image, threshold=0.8)
>>> from uniface import RetinaFace
>>> detector = RetinaFace(confidence_threshold=0.8)
>>> faces = detector.detect(image)
>>> print(f"Found {len(faces)} faces")
"""
```
@@ -153,7 +159,7 @@ pytest tests/
pytest tests/ -v
# Run specific test file
pytest tests/test_factory.py
pytest tests/test_scrfd.py
# Run with coverage
pytest tests/ --cov=uniface --cov-report=html
@@ -174,16 +180,62 @@ When adding a new model or feature:
Example notebooks demonstrating library usage:
| Example | Notebook |
|---------|----------|
| Face Detection | [01_face_detection.ipynb](examples/01_face_detection.ipynb) |
| Face Alignment | [02_face_alignment.ipynb](examples/02_face_alignment.ipynb) |
| Face Verification | [03_face_verification.ipynb](examples/03_face_verification.ipynb) |
| Face Search | [04_face_search.ipynb](examples/04_face_search.ipynb) |
| Face Analyzer | [05_face_analyzer.ipynb](examples/05_face_analyzer.ipynb) |
| Face Parsing | [06_face_parsing.ipynb](examples/06_face_parsing.ipynb) |
| Example | Notebook |
| ------------------ | ------------------------------------------------------------------- |
| Face Detection | [01_face_detection.ipynb](examples/01_face_detection.ipynb) |
| Face Alignment | [02_face_alignment.ipynb](examples/02_face_alignment.ipynb) |
| Face Verification | [03_face_verification.ipynb](examples/03_face_verification.ipynb) |
| Face Search | [04_face_search.ipynb](examples/04_face_search.ipynb) |
| Face Analyzer | [05_face_analyzer.ipynb](examples/05_face_analyzer.ipynb) |
| Face Parsing | [06_face_parsing.ipynb](examples/06_face_parsing.ipynb) |
| Face Anonymization | [07_face_anonymization.ipynb](examples/07_face_anonymization.ipynb) |
| Gaze Estimation | [08_gaze_estimation.ipynb](examples/08_gaze_estimation.ipynb) |
| Gaze Estimation | [08_gaze_estimation.ipynb](examples/08_gaze_estimation.ipynb) |
| Face Segmentation | [09_face_segmentation.ipynb](examples/09_face_segmentation.ipynb) |
| Face Vector Store | [10_face_vector_store.ipynb](examples/10_face_vector_store.ipynb) |
| Head Pose Estimation | [11_head_pose_estimation.ipynb](examples/11_head_pose_estimation.ipynb) |
| Face Recognition | [12_face_recognition.ipynb](examples/12_face_recognition.ipynb) |
| Portrait Matting | [13_portrait_matting.ipynb](examples/13_portrait_matting.ipynb) |
| Face Attributes | [14_face_attributes.ipynb](examples/14_face_attributes.ipynb) |
| Face Mesh | [15_face_mesh.ipynb](examples/15_face_mesh.ipynb) |
## Release Process
Releases are fully automated via GitHub Actions. Only maintainers with branch-protection bypass privileges on `main` can trigger a release.
### Cutting a release
1. Go to **Actions → Release Pipeline → Run workflow** on GitHub.
2. Enter the version following [PEP 440](https://peps.python.org/pep-0440/):
- Stable: `0.7.0`, `1.0.0`
- Pre-release: `0.7.0rc1`, `0.7.0b1`, `0.7.0a1`, `0.7.0.dev1`
3. Click **Run workflow**.
### What happens automatically
The `Release Pipeline` workflow runs all stages in sequence:
1. **Validate** — checks the version string against PEP 440 and confirms the tag does not already exist.
2. **Test** — runs the test suite on Python 3.103.14.
3. **Release** — updates `pyproject.toml` and `uniface/__init__.py`, commits `chore: Release vX.Y.Z` to `main`, creates and pushes tag `vX.Y.Z`.
4. **Publish** — builds the package, uploads to PyPI, and creates a GitHub Release (flagged as pre-release for `a`/`b`/`rc`/`.dev` versions).
5. **Deploy docs** — runs only for **stable** versions. Pre-releases do not update the live documentation site.
### Verifying a release
- PyPI: <https://pypi.org/project/uniface/>
- GitHub Releases: <https://github.com/yakhyo/uniface/releases>
- Docs (stable only): <https://yakhyo.github.io/uniface/>
### Installing a pre-release
End users can opt in to pre-releases with the `--pre` flag:
```bash
pip install uniface --pre # latest pre-release
pip install uniface==0.7.0rc1 # specific pre-release
```
Without `--pre`, `pip install uniface` always resolves to the latest stable version.
## Questions?

340
README.md
View File

@@ -1,67 +1,142 @@
# UniFace: All-in-One Face Analysis Library
<h1 align="center">UniFace: A Unified Face Analysis Library for Python</h1>
<div align="center">
[![PyPI](https://img.shields.io/pypi/v/uniface.svg)](https://pypi.org/project/uniface/)
[![Python](https://img.shields.io/badge/Python-3.11%2B-blue)](https://www.python.org/)
[![PyPI Version](https://img.shields.io/pypi/v/uniface.svg?label=Version)](https://pypi.org/project/uniface/)
[![Python Version](https://img.shields.io/badge/Python-3.10%2B-blue)](https://www.python.org/)
[![License](https://img.shields.io/badge/License-MIT-blue.svg)](https://opensource.org/licenses/MIT)
[![CI](https://github.com/yakhyo/uniface/actions/workflows/ci.yml/badge.svg)](https://github.com/yakhyo/uniface/actions)
[![Downloads](https://static.pepy.tech/badge/uniface)](https://pepy.tech/project/uniface)
[![Docs](https://img.shields.io/badge/Docs-UniFace-blue.svg)](https://yakhyo.github.io/uniface/)
[![Github Build Status](https://github.com/yakhyo/uniface/actions/workflows/ci.yml/badge.svg)](https://github.com/yakhyo/uniface/actions)
[![PyPI Downloads](https://static.pepy.tech/personalized-badge/uniface?period=total&units=INTERNATIONAL_SYSTEM&left_color=GRAY&right_color=BLUE&left_text=Downloads)](https://pepy.tech/projects/uniface)
[![UniFace Documentation](https://img.shields.io/badge/Docs-UniFace-blue.svg)](https://yakhyo.github.io/uniface/)
[![Kaggle Badge](https://img.shields.io/badge/Notebooks-Kaggle?label=Kaggle&color=blue)](https://www.kaggle.com/yakhyokhuja/code)
</div>
<div align="center">
<img src=".github/logos/logo_web.webp" width=80%>
<img src="https://raw.githubusercontent.com/yakhyo/uniface/main/.github/logos/uniface_rounded_q80.webp" width="90%" alt="UniFace - A Unified Face Analysis Library for Python">
</div>
**UniFace** is a lightweight, production-ready face analysis library built on ONNX Runtime. It provides high-performance face detection, recognition, landmark detection, face parsing, gaze estimation, and attribute analysis with hardware acceleration support across platforms.
> 💬 **Have questions?** [Chat with this codebase on DeepWiki](https://deepwiki.com/yakhyo/uniface) - AI-powered docs that let you ask anything about UniFace.
---
> [!NOTE]
> **UniFace** is a lightweight, production-ready Python library for face detection, recognition, tracking, landmark analysis, face parsing, gaze estimation, and face attributes.
## Features
- **Face Detection** — RetinaFace, SCRFD, and YOLOv5-Face with 5-point landmarks
- **Face Recognition** — ArcFace, MobileFace, and SphereFace embeddings
- **Facial Landmarks** — 106-point landmark localization
- **Face Parsing** — BiSeNet semantic segmentation (19 classes)
- **Gaze Estimation** — Real-time gaze direction with MobileGaze
- **Attribute Analysis** — Age, gender, race (FairFace), and emotion
- **Anti-Spoofing** — Face liveness detection with MiniFASNet
- **Face Anonymization** — 5 blur methods for privacy protection
- **Hardware Acceleration** — ARM64 (Apple Silicon), CUDA (NVIDIA), CPU
| Feature | Details |
| --------------------------- | ------------------------------------------------------------------------------------------------ |
| **Face Detection** | RetinaFace, SCRFD, CenterFace, YOLOv5-Face, and YOLOv8-Face with 5-point landmarks; BlazeFace (MediaPipe, 6 keypoints) |
| **Face Recognition** | AdaFace, ArcFace, EdgeFace, MobileFace, and SphereFace embeddings |
| **Face Tracking** | Multi-object tracking with [BYTETracker](https://github.com/yakhyo/bytetrack-tracker) for persistent IDs across video frames |
| **Facial Landmarks** | 106-point (2d106det), 98 / 68-point (PIPNet), and 468 / 478-point dense 3D mesh (MediaPipe Face Mesh, the 478 variant adding irises) — separate from the 5-point detector landmarks |
| **Face Parsing** | BiSeNet semantic segmentation (19 classes), XSeg face masking |
| **Portrait Matting** | Trimap-free alpha matte with MODNet (background removal, green screen, compositing) |
| **Gaze Estimation** | Real-time gaze direction with MobileGaze |
| **Head Pose Estimation** | 3D head orientation (pitch, yaw, roll) with 6D rotation representation |
| **Attribute Analysis** | Age, gender, race (FairFace), emotion, and face states (eye openness, glasses, sunglasses, mask with FaceAttribNet) |
| **Vector Store** | FAISS-backed embedding store for fast multi-identity search |
| **Anti-Spoofing** | Face liveness detection with MiniFASNet |
| **Face Quality Assessment** | eDifFIQA single-score quality (T/S/M/L, NIST FATE-Quality #1 with the L variant) |
| **Face Anonymization** | 5 blur methods for privacy protection |
| **Hardware Acceleration** | ARM64 (Apple Silicon), CUDA (NVIDIA), CPU |
---
## Visual Examples
<table>
<tr>
<td align="center"><b>Face Detection</b><br><img src="https://raw.githubusercontent.com/yakhyo/uniface/main/assets/demos/detection.jpg" width="100%"></td>
<td align="center"><b>Gaze Estimation</b><br><img src="https://raw.githubusercontent.com/yakhyo/uniface/main/assets/demos/gaze.jpg" width="100%"></td>
</tr>
<tr>
<td align="center"><b>Head Pose Estimation</b><br><img src="https://raw.githubusercontent.com/yakhyo/uniface/main/assets/demos/headpose.jpg" width="100%"></td>
<td align="center"><b>Age & Gender</b><br><img src="https://raw.githubusercontent.com/yakhyo/uniface/main/assets/demos/age_gender.jpg" width="100%"></td>
</tr>
<tr>
<td align="center" colspan="2"><b>Face Verification</b><br><img src="https://raw.githubusercontent.com/yakhyo/uniface/main/assets/demos/verification.jpg" width="80%"></td>
</tr>
<tr>
<td align="center"><b>106-Point Landmarks</b><br><img src="https://raw.githubusercontent.com/yakhyo/uniface/main/assets/demos/landmarks.jpg" width="70%"></td>
<td align="center"><b>98-Point Landmarks (PIPNet)</b><br><img src="https://raw.githubusercontent.com/yakhyo/uniface/main/assets/demos/landmarks_pipnet.jpg" width="70%"></td>
</tr>
<tr>
<td align="center" colspan="2"><b>Face Parsing</b><br><img src="https://raw.githubusercontent.com/yakhyo/uniface/main/assets/demos/parsing.jpg" width="80%"></td>
</tr>
<tr>
<td align="center" colspan="2"><b>Face Segmentation</b><br><img src="https://raw.githubusercontent.com/yakhyo/uniface/main/assets/demos/segmentation.jpg" width="80%"></td>
</tr>
<tr>
<td align="center" colspan="2"><b>Portrait Matting</b><br><img src="https://raw.githubusercontent.com/yakhyo/uniface/main/assets/demos/matting.jpg" width="100%"></td>
</tr>
<tr>
<td align="center" colspan="2"><b>Face Anonymization</b><br><img src="https://raw.githubusercontent.com/yakhyo/uniface/main/assets/demos/anonymization.jpg" width="100%"></td>
</tr>
</table>
## Installation
**CPU / Apple Silicon**
```bash
# Standard installation
pip install uniface
# GPU support (CUDA)
pip install uniface[gpu]
# From source
git clone https://github.com/yakhyo/uniface.git
cd uniface && pip install -e .
pip install uniface[cpu]
```
---
**GPU support (NVIDIA CUDA)**
## Quick Example
```bash
pip install uniface[gpu]
```
> **Why separate extras?** `onnxruntime` and `onnxruntime-gpu` conflict when both are installed — they own the same Python namespace. Installing only the extra you need prevents that conflict entirely.
**From source (latest version)**
```bash
git clone https://github.com/yakhyo/uniface.git
cd uniface && pip install -e ".[cpu]" # or .[gpu] for CUDA
```
**FAISS vector store**
```bash
pip install faiss-cpu # or faiss-gpu for CUDA
```
**Optional dependencies**
- Emotion model uses TorchScript and requires `torch`:
`pip install torch` (choose the correct build for your OS/CUDA)
- YOLOv5-Face and YOLOv8-Face support faster NMS with `torchvision`:
`pip install torch torchvision` then use `nms_mode='torchvision'`
## Model Downloads and Cache
Models are downloaded automatically on first use and verified via SHA-256.
Default cache location: `~/.uniface/models`
Override with the programmatic API or environment variable:
```python
from uniface.model_store import get_cache_dir, set_cache_dir
set_cache_dir('/data/models')
print(get_cache_dir()) # /data/models
```
```bash
export UNIFACE_CACHE_DIR=/data/models
```
## Quick Example (Detection)
```python
import cv2
from uniface import RetinaFace
from uniface.detection import RetinaFace
# Initialize detector (models auto-download on first use)
detector = RetinaFace()
# Detect faces
image = cv2.imread("photo.jpg")
if image is None:
raise ValueError("Failed to load image. Check the path to 'photo.jpg'.")
faces = detector.detect(image)
for face in faces:
@@ -70,56 +145,181 @@ for face in faces:
print(f"Landmarks: {face.landmarks.shape}")
```
<div align="center">
<img src="assets/test_result.png">
</div>
## Example (Face Analyzer)
---
```python
import cv2
from uniface import FaceAnalyzer
# Zero-config: uses SCRFD (500M) + ArcFace (MobileNet) by default
analyzer = FaceAnalyzer()
image = cv2.imread("photo.jpg")
if image is None:
raise ValueError("Failed to load image. Check the path to 'photo.jpg'.")
faces = analyzer.analyze(image)
for face in faces:
print(face.bbox, face.embedding.shape if face.embedding is not None else None)
```
With attributes:
```python
from uniface import FaceAnalyzer, AgeGender
analyzer = FaceAnalyzer(predictors=[AgeGender()])
faces = analyzer.analyze(image)
for face in faces:
print(f"{face.sex}, {face.age}y, embedding={face.embedding.shape}")
```
## Example (Portrait Matting)
```python
import cv2
import numpy as np
from uniface.matting import MODNet
matting = MODNet()
image = cv2.imread("portrait.jpg")
matte = matting.predict(image) # (H, W) float32 in [0, 1]
# Transparent PNG
rgba = cv2.cvtColor(image, cv2.COLOR_BGR2BGRA)
rgba[:, :, 3] = (matte * 255).astype(np.uint8)
cv2.imwrite("transparent.png", rgba)
# Green screen
matte_3ch = matte[:, :, np.newaxis]
bg = np.full_like(image, (0, 177, 64), dtype=np.uint8)
result = (image * matte_3ch + bg * (1 - matte_3ch)).astype(np.uint8)
cv2.imwrite("green_screen.jpg", result)
```
## Jupyter Notebooks
| Example | Colab | Description |
| ---------------------------------------------------------------------- | :-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------: | --------------------------------------- |
| [01_face_detection.ipynb](examples/01_face_detection.ipynb) | [![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/yakhyo/uniface/blob/main/examples/01_face_detection.ipynb) | Face detection and landmarks |
| [02_face_alignment.ipynb](examples/02_face_alignment.ipynb) | [![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/yakhyo/uniface/blob/main/examples/02_face_alignment.ipynb) | Face alignment for recognition |
| [03_face_verification.ipynb](examples/03_face_verification.ipynb) | [![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/yakhyo/uniface/blob/main/examples/03_face_verification.ipynb) | Compare faces for identity |
| [04_face_search.ipynb](examples/04_face_search.ipynb) | [![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/yakhyo/uniface/blob/main/examples/04_face_search.ipynb) | Find a person in group photos |
| [05_face_analyzer.ipynb](examples/05_face_analyzer.ipynb) | [![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/yakhyo/uniface/blob/main/examples/05_face_analyzer.ipynb) | Unified face analysis |
| [06_face_parsing.ipynb](examples/06_face_parsing.ipynb) | [![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/yakhyo/uniface/blob/main/examples/06_face_parsing.ipynb) | Semantic face segmentation |
| [07_face_anonymization.ipynb](examples/07_face_anonymization.ipynb) | [![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/yakhyo/uniface/blob/main/examples/07_face_anonymization.ipynb) | Privacy-preserving blur |
| [08_gaze_estimation.ipynb](examples/08_gaze_estimation.ipynb) | [![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/yakhyo/uniface/blob/main/examples/08_gaze_estimation.ipynb) | Gaze direction estimation |
| [09_face_segmentation.ipynb](examples/09_face_segmentation.ipynb) | [![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/yakhyo/uniface/blob/main/examples/09_face_segmentation.ipynb) | Face segmentation with XSeg |
| [10_face_vector_store.ipynb](examples/10_face_vector_store.ipynb) | [![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/yakhyo/uniface/blob/main/examples/10_face_vector_store.ipynb) | FAISS-backed face database |
| [11_head_pose_estimation.ipynb](examples/11_head_pose_estimation.ipynb) | [![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/yakhyo/uniface/blob/main/examples/11_head_pose_estimation.ipynb) | Head pose estimation (pitch, yaw, roll) |
| [12_face_recognition.ipynb](examples/12_face_recognition.ipynb) | [![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/yakhyo/uniface/blob/main/examples/12_face_recognition.ipynb) | Standalone face recognition pipeline |
| [13_portrait_matting.ipynb](examples/13_portrait_matting.ipynb) | [![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/yakhyo/uniface/blob/main/examples/13_portrait_matting.ipynb) | Portrait matting with MODNet |
| [14_face_attributes.ipynb](examples/14_face_attributes.ipynb) | [![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/yakhyo/uniface/blob/main/examples/14_face_attributes.ipynb) | Face states (eyes, glasses, sunglasses, mask) |
| [15_face_mesh.ipynb](examples/15_face_mesh.ipynb) | [![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/yakhyo/uniface/blob/main/examples/15_face_mesh.ipynb) | 468 / 478-point dense 3D face mesh |
## Documentation
📚 **Full documentation**: [yakhyo.github.io/uniface](https://yakhyo.github.io/uniface/)
Full documentation: https://yakhyo.github.io/uniface/
| Resource | Description |
|----------|-------------|
| [Quickstart](https://yakhyo.github.io/uniface/quickstart/) | Get up and running in 5 minutes |
| [Model Zoo](https://yakhyo.github.io/uniface/models/) | All models, benchmarks, and selection guide |
| [API Reference](https://yakhyo.github.io/uniface/modules/detection/) | Detailed module documentation |
| [Tutorials](https://yakhyo.github.io/uniface/recipes/image-pipeline/) | Step-by-step workflow examples |
| [Guides](https://yakhyo.github.io/uniface/concepts/overview/) | Architecture and design principles |
| Resource | Description |
| -------------------------------------------------------------------- | ------------------------------------------- |
| [Quickstart](https://yakhyo.github.io/uniface/quickstart/) | Get up and running in 5 minutes |
| [Model Zoo](https://yakhyo.github.io/uniface/models/) | All models, benchmarks, and selection guide |
| [API Reference](https://yakhyo.github.io/uniface/modules/detection/) | Detailed module documentation |
| [Tutorials](https://yakhyo.github.io/uniface/recipes/image-pipeline/) | Step-by-step workflow examples |
| [Guides](https://yakhyo.github.io/uniface/concepts/overview/) | Architecture and design principles |
| [Datasets](https://yakhyo.github.io/uniface/datasets/) | Training data and evaluation benchmarks |
### Jupyter Notebooks
## Execution Providers (ONNX Runtime)
| Example | Colab | Description |
|---------|:-----:|-------------|
| [01_face_detection.ipynb](examples/01_face_detection.ipynb) | [![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/yakhyo/uniface/blob/main/examples/01_face_detection.ipynb) | Face detection and landmarks |
| [02_face_alignment.ipynb](examples/02_face_alignment.ipynb) | [![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/yakhyo/uniface/blob/main/examples/02_face_alignment.ipynb) | Face alignment for recognition |
| [03_face_verification.ipynb](examples/03_face_verification.ipynb) | [![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/yakhyo/uniface/blob/main/examples/03_face_verification.ipynb) | Compare faces for identity |
| [04_face_search.ipynb](examples/04_face_search.ipynb) | [![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/yakhyo/uniface/blob/main/examples/04_face_search.ipynb) | Find a person in group photos |
| [05_face_analyzer.ipynb](examples/05_face_analyzer.ipynb) | [![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/yakhyo/uniface/blob/main/examples/05_face_analyzer.ipynb) | All-in-one analysis |
| [06_face_parsing.ipynb](examples/06_face_parsing.ipynb) | [![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/yakhyo/uniface/blob/main/examples/06_face_parsing.ipynb) | Semantic face segmentation |
| [07_face_anonymization.ipynb](examples/07_face_anonymization.ipynb) | [![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/yakhyo/uniface/blob/main/examples/07_face_anonymization.ipynb) | Privacy-preserving blur |
| [08_gaze_estimation.ipynb](examples/08_gaze_estimation.ipynb) | [![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/yakhyo/uniface/blob/main/examples/08_gaze_estimation.ipynb) | Gaze direction estimation |
```python
from uniface.detection import RetinaFace
---
# Force CPU-only inference
detector = RetinaFace(providers=["CPUExecutionProvider"])
```
See more in the docs:
https://yakhyo.github.io/uniface/concepts/execution-providers/
## Datasets
| Task | Training Dataset | Models |
| ----------- | --------------------------- | ------------------------------------------- |
| Detection | WIDER FACE | RetinaFace, SCRFD, CenterFace, YOLOv5-Face, YOLOv8-Face |
| Recognition | MS1MV2 | MobileFace, SphereFace |
| Recognition | WebFace600K | ArcFace |
| Recognition | WebFace4M / 12M | AdaFace, EdgeFace |
| Landmarks | WFLW, 300W+CelebA | PIPNet (98 / 68 pts) |
| Detection | Proprietary (Google) | BlazeFace (short-range) |
| Landmarks | Proprietary (Google) | Face Mesh (468 / 478 pts) |
| Gaze | Gaze360 | MobileGaze |
| Head Pose | 300W-LP | HeadPose (ResNet, MobileNet) |
| Parsing | CelebAMask-HQ | BiSeNet |
| Attributes | CelebA, FairFace, AffectNet | AgeGender, FairFace, Emotion |
| Attributes | Proprietary (Qualcomm) | FaceAttribNet |
> See [Datasets documentation](https://yakhyo.github.io/uniface/datasets/) for download links, benchmarks, and details.
## Licensing and Model Usage
UniFace is MIT-licensed, but several pretrained models carry their own licenses.
Review: https://yakhyo.github.io/uniface/license-attribution/
Notable examples:
- YOLOv5-Face and YOLOv8-Face weights are GPL-3.0
- FairFace weights are CC BY 4.0
- FaceAttribNet weights are BSD-3-Clause (© Qualcomm Technologies, Inc.)
- BlazeFace and Face Mesh weights are Apache-2.0 (© Google, from MediaPipe)
If you plan commercial use, verify model license compatibility.
## References
- [yakhyo/retinaface-pytorch](https://github.com/yakhyo/retinaface-pytorch) — RetinaFace training
- [yakhyo/yolov5-face-onnx-inference](https://github.com/yakhyo/yolov5-face-onnx-inference) — YOLOv5-Face ONNX
- [yakhyo/face-recognition](https://github.com/yakhyo/face-recognition) — ArcFace, MobileFace, SphereFace
- [yakhyo/face-parsing](https://github.com/yakhyo/face-parsing) — BiSeNet face parsing
- [yakhyo/gaze-estimation](https://github.com/yakhyo/gaze-estimation) — MobileGaze training
- [yakhyo/face-anti-spoofing](https://github.com/yakhyo/face-anti-spoofing) — MiniFASNet inference
- [yakhyo/fairface-onnx](https://github.com/yakhyo/fairface-onnx) — FairFace attributes
- [deepinsight/insightface](https://github.com/deepinsight/insightface) — Model architectures
| Feature | Repository | Training | Description |
| ------------- | --------------------------------------------------------------------------------------- | :------: | ------------------------------------ |
| Detection | [retinaface-pytorch](https://github.com/yakhyo/retinaface-pytorch) | ✓ | RetinaFace PyTorch Training & Export |
| Detection | [yolov5-face-onnx-inference](https://github.com/yakhyo/yolov5-face-onnx-inference) | - | YOLOv5-Face Inference |
| Detection | [yolov8-face-onnx-inference](https://github.com/yakhyo/yolov8-face-onnx-inference) | - | YOLOv8-Face Inference |
| Detection | [Star-Clouds/CenterFace](https://github.com/Star-Clouds/CenterFace) | - | CenterFace Original Weights |
| Tracking | [bytetrack-tracker](https://github.com/yakhyo/bytetrack-tracker) | - | BYTETracker Multi-Object Tracking |
| Recognition | [face-recognition](https://github.com/yakhyo/face-recognition) | ✓ | MobileFace, SphereFace Training |
| Recognition | [edgeface-onnx](https://github.com/yakhyo/edgeface-onnx) | - | EdgeFace Inference |
| Landmarks | [pipnet-onnx](https://github.com/yakhyo/pipnet-onnx) | - | PIPNet 98 / 68-point Inference |
| Landmarks | [mediapipe-face-mesh-onnx](https://github.com/yakhyo/mediapipe-face-mesh-onnx) | - | Face Mesh 468 / 478-point Inference |
| Parsing | [face-parsing](https://github.com/yakhyo/face-parsing) | ✓ | BiSeNet Face Parsing |
| Parsing | [face-segmentation](https://github.com/yakhyo/face-segmentation) | - | XSeg Face Segmentation |
| Gaze | [gaze-estimation](https://github.com/yakhyo/gaze-estimation) | ✓ | MobileGaze Training |
| Head Pose | [head-pose-estimation](https://github.com/yakhyo/head-pose-estimation) | ✓ | Head Pose Training (6DRepNet-style) |
| Matting | [modnet](https://github.com/yakhyo/modnet) | - | MODNet Portrait Matting |
| Anti-Spoofing | [face-anti-spoofing](https://github.com/yakhyo/face-anti-spoofing) | - | MiniFASNet Inference |
| Quality | [face-image-quality-assessment](https://github.com/yakhyo/face-image-quality-assessment) | - | eDifFIQA Inference |
| Attributes | [fairface-onnx](https://github.com/yakhyo/fairface-onnx) | - | FairFace Inference |
| Attributes | [face-attribute](https://github.com/yakhyo/face-attribute) | - | FaceAttribNet Inference |
---
*SCRFD and ArcFace models are from [InsightFace](https://github.com/deepinsight/insightface).
## Contributing
Contributions are welcome! Please see [CONTRIBUTING.md](CONTRIBUTING.md) for guidelines.
Contributions are welcome. Please see [CONTRIBUTING.md](CONTRIBUTING.md).
## Support
If you find this project useful, consider giving it a ⭐ on GitHub — it helps others discover it!
Questions or feedback:
- Discord: https://discord.gg/wdzrjr7R5j
- GitHub Issues: https://github.com/yakhyo/uniface/issues
- DeepWiki Q&A: https://deepwiki.com/yakhyo/uniface
## License
This project is licensed under the [MIT License](LICENSE).
> **Disclaimer:** This project is not affiliated with or related to
> [Uniface](https://uniface.com/) by Rocket Software.

Binary file not shown.

Before

Width:  |  Height:  |  Size: 996 KiB

BIN
assets/demos/age_gender.jpg Normal file

Binary file not shown.

After

Width:  |  Height:  |  Size: 206 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 1.5 MiB

BIN
assets/demos/detection.jpg Normal file

Binary file not shown.

After

Width:  |  Height:  |  Size: 341 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 936 KiB

BIN
assets/demos/gaze.jpg Normal file

Binary file not shown.

After

Width:  |  Height:  |  Size: 212 KiB

BIN
assets/demos/headpose.jpg Normal file

Binary file not shown.

After

Width:  |  Height:  |  Size: 233 KiB

BIN
assets/demos/landmarks.jpg Normal file

Binary file not shown.

After

Width:  |  Height:  |  Size: 345 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 344 KiB

BIN
assets/demos/matting.jpg Normal file

Binary file not shown.

After

Width:  |  Height:  |  Size: 938 KiB

BIN
assets/demos/parsing.jpg Normal file

Binary file not shown.

After

Width:  |  Height:  |  Size: 712 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 851 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 171 KiB

BIN
assets/demos/src_man1.jpg Normal file

Binary file not shown.

After

Width:  |  Height:  |  Size: 63 KiB

BIN
assets/demos/src_man2.jpg Normal file

Binary file not shown.

After

Width:  |  Height:  |  Size: 220 KiB

BIN
assets/demos/src_man3.jpg Normal file

Binary file not shown.

After

Width:  |  Height:  |  Size: 146 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 96 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 208 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 121 KiB

View File

Before

Width:  |  Height:  |  Size: 1.3 MiB

After

Width:  |  Height:  |  Size: 1.3 MiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 345 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 127 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 260 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 5.8 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 1.1 MiB

View File

@@ -88,12 +88,22 @@ landmarks = face.landmarks # Shape: (5, 2)
3 ● ● 4
```
This ordering is the *alignment template* (`uniface.face_utils.reference_alignment`)
that recognition, quality scoring, and XSeg parsing all require.
!!! warning "BlazeFace uses a different layout"
`BlazeFace` returns 6 keypoints — right eye, left eye, nose tip, mouth **center**,
right ear tragion, left ear tragion — named from the subject's perspective, so rows
0/1 are still the viewer-left and viewer-right eye. With no mouth corners they
cannot be fitted to the template above, which is why `BlazeFace.supports_alignment`
is `False`.
### 106-Point Landmarks
Returned by `Landmark106`:
```python
from uniface import Landmark106
from uniface.landmark import Landmark106
landmarker = Landmark106()
landmarks = landmarker.get_landmarks(image, face.bbox)
@@ -110,6 +120,48 @@ landmarks = landmarker.get_landmarks(image, face.bbox)
| 63-86 | Eyes | 24 |
| 87-105 | Mouth | 19 |
### 98 / 68-Point Landmarks (PIPNet)
Returned by `PIPNet`. The variant determines the layout:
```python
from uniface.constants import PIPNetWeights
from uniface.landmark import PIPNet
# 98-point WFLW layout (default)
landmarks = PIPNet().get_landmarks(image, face.bbox)
# Shape: (98, 2)
# 68-point 300W layout
landmarks = PIPNet(model_name=PIPNetWeights.DW300_CELEBA_68).get_landmarks(image, face.bbox)
# Shape: (68, 2)
```
The 98-point output follows the standard [WFLW](https://wywu.github.io/projects/LAB/WFLW.html) layout
(33 face-contour points, eyebrow/eye/nose/mouth groups). The 68-point output follows the standard
[300W / iBUG](https://ibug.doc.ic.ac.uk/resources/300-W/) layout. Coordinates are in original-image
pixel space, identical in convention to `Landmark106`.
### 468-Point Landmarks (Face Mesh)
`FaceMesh` is the only model in UniFace that returns **three** coordinates per point:
```python
from uniface import FaceMesh
results = FaceMesh().predict(image, faces)
results[0].landmarks # Shape: (468, 3)
```
| Axis | Meaning |
|------|---------|
| `x`, `y` | Original-image pixel coordinates, same convention as every other landmarker |
| `z` | **Relative** depth, on the same pixel scale as `x`/`y`. Smaller is closer to the camera |
`z` has no absolute origin — it is only meaningful *within* one face, for comparing which
features sit nearer the camera. It is not a distance measurement and is not comparable
between faces or between images. Use `results[0].points_2d` to drop it.
---
## Face Crop
@@ -174,11 +226,11 @@ yaw = -90° ────┼──── yaw = +90°
Face alignment uses 5-point landmarks to normalize face orientation:
```python
from uniface import face_alignment
from uniface.face_utils import face_alignment
# Align face to standard template
aligned_face = face_alignment(image, face.landmarks)
# Output: 112x112 aligned face image
aligned_face, _ = face_alignment(image, face.landmarks)
# Output: (112x112 aligned face image, inverse transform matrix)
```
The alignment transforms faces to a canonical pose for better recognition accuracy.

View File

@@ -1,6 +1,6 @@
# Execution Providers
UniFace uses ONNX Runtime for model inference, which supports multiple hardware acceleration backends.
UniFace automatically runs each model on the best available hardware — Apple Silicon (CoreML), NVIDIA GPU (CUDA), or CPU. Under the hood this is handled by ONNX Runtime execution providers.
---
@@ -9,7 +9,7 @@ UniFace uses ONNX Runtime for model inference, which supports multiple hardware
UniFace automatically selects the optimal execution provider based on available hardware:
```python
from uniface import RetinaFace
from uniface.detection import RetinaFace
# Automatically uses best available provider
detector = RetinaFace()
@@ -17,12 +17,44 @@ detector = RetinaFace()
**Priority order:**
1. **CUDAExecutionProvider** - NVIDIA GPU
2. **CoreMLExecutionProvider** - Apple Silicon
1. **CoreMLExecutionProvider** - Apple Silicon
2. **CUDAExecutionProvider** - NVIDIA GPU
3. **CPUExecutionProvider** - Fallback
---
## Explicit Provider Selection
You can specify which execution provider to use by passing the `providers` parameter:
```python
from uniface.detection import RetinaFace
from uniface.recognition import ArcFace
# Force CPU execution (even if GPU is available)
detector = RetinaFace(providers=['CPUExecutionProvider'])
recognizer = ArcFace(providers=['CPUExecutionProvider'])
# Use CUDA with CPU fallback
detector = RetinaFace(providers=['CUDAExecutionProvider', 'CPUExecutionProvider'])
```
All **ONNX-based** model classes accept the `providers` parameter:
- Detection: `RetinaFace`, `SCRFD`, `YOLOv5Face`, `YOLOv8Face`
- Recognition: `ArcFace`, `AdaFace`, `MobileFace`, `SphereFace`
- Landmarks: `Landmark106`, `PIPNet`
- Gaze: `MobileGaze`
- Parsing: `BiSeNet`, `XSeg`
- Attributes: `AgeGender`, `FairFace`
- Anti-Spoofing: `MiniFASNet`
!!! note "Non-ONNX components"
- **Emotion** uses TorchScript and selects its device automatically (`mps` / `cuda` / `cpu`). It does **not** accept the `providers` parameter.
- **BlurFace** is a pure OpenCV utility and does not load any model.
---
## Check Available Providers
```python
@@ -61,7 +93,7 @@ print("Available providers:", providers)
No additional setup required. ARM64 optimizations are built into `onnxruntime`:
```bash
pip install uniface
pip install uniface[cpu]
```
Verify ARM64:
@@ -78,7 +110,7 @@ python -c "import platform; print(platform.machine())"
### NVIDIA GPU (CUDA)
Install with GPU support:
Install with GPU support (this installs `onnxruntime-gpu`, which already includes CPU fallback):
```bash
pip install uniface[gpu]
@@ -108,7 +140,7 @@ else:
CPU execution is always available:
```bash
pip install uniface
pip install uniface[cpu]
```
Works on all platforms without additional configuration.
@@ -147,7 +179,7 @@ pip install uniface[gpu]
Smaller input sizes are faster but may reduce accuracy:
```python
from uniface import RetinaFace
from uniface.detection import RetinaFace
# Faster, lower accuracy
detector = RetinaFace(input_size=(320, 320))
@@ -183,7 +215,7 @@ for image_path in image_paths:
3. Reinstall with GPU support:
```bash
pip uninstall onnxruntime onnxruntime-gpu
pip uninstall onnxruntime onnxruntime-gpu -y
pip install uniface[gpu]
```

View File

@@ -28,6 +28,20 @@ print(f"Dtype: {image.dtype}") # uint8
bgr_image = np.array(pil_image)[:, :, ::-1] # RGB → BGR
```
!!! warning "Dtype"
Images must be `uint8` in `[0, 255]`, three-channel. A float `[0, 1]` array or a
grayscale frame raises `ValueError` rather than returning quietly wrong results:
```python
detector.detect(image.astype(np.float32) / 255.0)
# ValueError: Expected dtype uint8, got float32. Scale to [0, 255] and cast with .astype(np.uint8).
detector.detect(cv2.cvtColor(image, cv2.COLOR_BGR2GRAY))
# ValueError: Expected a BGR image of shape (H, W, 3), got (480, 640). Convert with cv2.cvtColor.
```
Normalization is each model's job — pass the raw `cv2.imread` array through.
---
## Output: Face Dataclass
@@ -43,7 +57,8 @@ class Face:
# Required (from detection)
bbox: np.ndarray # [x1, y1, x2, y2]
confidence: float # 0.0 to 1.0
landmarks: np.ndarray # (5, 2) or (106, 2)
landmarks: np.ndarray # (5, 2) from detectors — except BlazeFace, which returns (6, 2).
# Dense landmarkers return (106, 2), (98, 2), (68, 2), (468, 3), or (478, 3).
# Optional (enriched by analyzers)
embedding: np.ndarray | None = None
@@ -53,6 +68,13 @@ class Face:
race: str | None = None # "East Asian", etc.
emotion: str | None = None # "Happy", etc.
emotion_confidence: float | None = None
left_eye_open: float | None = None # [0, 1] probability, from FaceAttribNet
right_eye_open: float | None = None # [0, 1] probability, from FaceAttribNet
eyeglasses: float | None = None # [0, 1] probability, from FaceAttribNet
mask: float | None = None # [0, 1] probability, from FaceAttribNet
sunglasses: float | None = None # [0, 1] probability, from FaceAttribNet
quality: float | None = None # [0, 1] quality score from eDifFIQA
track_id: int | None = None # Persistent ID from tracker
```
### Properties
@@ -76,6 +98,9 @@ similarity = face1.compute_similarity(face2)
# Convert to dictionary
face_dict = face.to_dict()
# Convert to JSON string
face_json = face.to_json(indent=2)
```
---
@@ -105,6 +130,27 @@ print(f"Yaw: {np.degrees(result.yaw):.1f}°")
---
### HeadPoseResult
```python
@dataclass(frozen=True)
class HeadPoseResult:
pitch: float # Rotation around X-axis (degrees), + = looking down
yaw: float # Rotation around Y-axis (degrees), + = looking right
roll: float # Rotation around Z-axis (degrees), + = tilting clockwise
```
**Usage:**
```python
result = head_pose.estimate(face_crop)
print(f"Pitch: {result.pitch:.1f}°")
print(f"Yaw: {result.yaw:.1f}°")
print(f"Roll: {result.roll:.1f}°")
```
---
### SpoofingResult
```python
@@ -124,11 +170,11 @@ print(f"{label}: {result.confidence:.1%}")
---
### AttributeResult
### DemographyResult
```python
@dataclass(frozen=True)
class AttributeResult:
class DemographyResult:
gender: int # 0=Female, 1=Male
age: int | None # Years (AgeGender model)
age_group: str | None # "20-29" (FairFace model)
@@ -143,11 +189,11 @@ class AttributeResult:
```python
# AgeGender model
result = age_gender.predict(image, face.bbox)
result = age_gender.predict(image, face)
print(f"{result.sex}, {result.age} years old")
# FairFace model
result = fairface.predict(image, face.bbox)
result = fairface.predict(image, face)
print(f"{result.sex}, {result.age_group}, {result.race}")
```
@@ -164,20 +210,73 @@ class EmotionResult:
---
### QualityResult
```python
@dataclass(frozen=True)
class QualityResult:
score: float # 0.0 to 1.0, higher = better quality
```
---
### FaceStateResult
```python
@dataclass(frozen=True)
class FaceStateResult:
left_eye_open: float # Probability the left eye is open
right_eye_open: float # Probability the right eye is open
eyeglasses: float # Probability eyeglasses are present
mask: float # Probability a face mask is present
sunglasses: float # Probability sunglasses are present
```
The five values come from independent binary heads: they do not sum to 1 and several can
be high at once. Threshold each attribute separately; never `argmax`.
**Usage:**
```python
result = face_attrib_net.predict(image, face)
print(result.as_dict()) # {'left_eye_open': 0.98, ...}
print(result.labels(0.5)) # ['left_eye_open', 'right_eye_open', 'eyeglasses']
```
---
### FaceMeshResult
```python
@dataclass(frozen=True)
class FaceMeshResult:
landmarks: np.ndarray # (468, 3) or (478, 3); x, y in image pixels, z is relative depth
score: float # Face presence, 0.0 to 1.0
@property
def points_2d(self) -> np.ndarray:
return self.landmarks[:, :2] # (N, 2), depth dropped
```
`score` saturates near 1.0 for anything plausible. It confirms the model ran; it is not a
discriminative confidence, so do not threshold on it.
---
## Embeddings
Face recognition models return normalized 512-dimensional embeddings:
```python
embedding = recognizer.get_normalized_embedding(image, landmarks)
print(f"Shape: {embedding.shape}") # (1, 512)
print(f"Shape: {embedding.shape}") # (512,)
print(f"Norm: {np.linalg.norm(embedding):.4f}") # ~1.0
```
### Similarity Computation
```python
from uniface import compute_similarity
from uniface.face_utils import compute_similarity
similarity = compute_similarity(embedding1, embedding2)
# Returns: float between -1 and 1 (cosine similarity)
@@ -199,16 +298,16 @@ print(f"Classes: {np.unique(mask)}") # [0, 1, 2, ...]
| ID | Class | ID | Class |
|----|-------|----|-------|
| 0 | Background | 10 | Ear Ring |
| 1 | Skin | 11 | Nose |
| 2 | Left Eyebrow | 12 | Mouth |
| 3 | Right Eyebrow | 13 | Upper Lip |
| 4 | Left Eye | 14 | Lower Lip |
| 5 | Right Eye | 15 | Neck |
| 6 | Eye Glasses | 16 | Neck Lace |
| 7 | Left Ear | 17 | Cloth |
| 8 | Right Ear | 18 | Hair |
| 9 | Hat | | |
| 0 | Background | 10 | Nose |
| 1 | Skin | 11 | Mouth |
| 2 | Left Eyebrow | 12 | Upper Lip |
| 3 | Right Eyebrow | 13 | Lower Lip |
| 4 | Left Eye | 14 | Neck |
| 5 | Right Eye | 15 | Necklace |
| 6 | Eyeglasses | 16 | Cloth |
| 7 | Left Ear | 17 | Hair |
| 8 | Right Ear | 18 | Hat |
| 9 | Earring | | |
---

View File

@@ -9,7 +9,7 @@ UniFace automatically downloads and caches models. This page explains how model
Models are downloaded on first use:
```python
from uniface import RetinaFace
from uniface.detection import RetinaFace
# First run: downloads model to cache
detector = RetinaFace() # ~3.5 MB download
@@ -18,6 +18,10 @@ detector = RetinaFace() # ~3.5 MB download
detector = RetinaFace() # Instant
```
Weights come from GitHub Releases, with a [Hugging Face mirror](https://huggingface.co/yakhyo/uniface-weights)
as an automatic fallback when GitHub is unreachable. There is nothing to configure, and no
Hugging Face token is needed.
---
## Cache Location
@@ -32,9 +36,9 @@ Default cache directory:
```
~/.uniface/models/
├── retinaface_mv2.onnx
├── w600k_mbf.onnx
├── 2d106det.onnx
├── retinaface_mnet_v2.onnx
├── arcface_mnet.onnx
├── 2d_106.onnx
├── gaze_resnet34.onnx
├── parsing_resnet18.onnx
└── ...
@@ -44,44 +48,57 @@ Default cache directory:
## Custom Cache Directory
Specify a custom cache location:
Use the programmatic API to change the cache location at runtime:
```python
from uniface.model_store import verify_model_weights
from uniface.constants import RetinaFaceWeights
from uniface.model_store import get_cache_dir, set_cache_dir
# Download to custom directory
model_path = verify_model_weights(
RetinaFaceWeights.MNET_V2,
root='./my_models'
)
print(f"Model at: {model_path}")
# Set a custom cache directory
set_cache_dir('/data/models')
# Verify the current path
print(get_cache_dir()) # /data/models
# All subsequent model loads use the new directory
from uniface.detection import RetinaFace
detector = RetinaFace() # Downloads to /data/models/
```
Or set the `UNIFACE_CACHE_DIR` environment variable (see [Environment Variables](#environment-variables) below).
---
## Pre-Download Models
Download models before deployment:
Download models before deployment using the concurrent downloader:
```python
from uniface.model_store import verify_model_weights
from uniface.model_store import download_models
from uniface.constants import (
RetinaFaceWeights,
ArcFaceWeights,
AgeGenderWeights,
)
# Download all needed models
models = [
# Download multiple models concurrently (defaults to min(CPU count, 8) threads)
paths = download_models([
RetinaFaceWeights.MNET_V2,
ArcFaceWeights.MNET,
AgeGenderWeights.DEFAULT,
]
])
for model in models:
path = verify_model_weights(model)
print(f"Downloaded: {path}")
for model, path in paths.items():
print(f"{model.value} -> {path}")
```
Or download one at a time:
```python
from uniface.model_store import verify_model_weights
from uniface.constants import RetinaFaceWeights
path = verify_model_weights(RetinaFaceWeights.MNET_V2)
print(f"Downloaded: {path}")
```
Or use the CLI tool:
@@ -115,11 +132,20 @@ print(f"Copy from: {path}")
scp -r ~/.uniface/models/ user@offline-machine:~/.uniface/models/
```
### 3. Use normally
### 3. Point to the cache (if non-default location)
```python
from uniface.model_store import set_cache_dir
# Only needed if the models are not at ~/.uniface/models/
set_cache_dir('/path/to/copied/models')
```
### 4. Use normally
```python
# Models load from local cache
from uniface import RetinaFace
from uniface.detection import RetinaFace
detector = RetinaFace() # No network required
```
@@ -130,14 +156,16 @@ detector = RetinaFace() # No network required
Models are verified with SHA-256 checksums:
```python
from uniface.constants import MODEL_SHA256, RetinaFaceWeights
from uniface.constants import MODEL_REGISTRY, RetinaFaceWeights
# Check expected checksum
expected = MODEL_SHA256[RetinaFaceWeights.MNET_V2]
expected = MODEL_REGISTRY[RetinaFaceWeights.MNET_V2].sha256
print(f"Expected SHA256: {expected}")
```
If a model fails verification, it's re-downloaded automatically.
If a model fails verification, it's re-downloaded automatically. The same checksum is enforced
whichever source served the file, so a corrupted or stale mirror falls through to the other
source instead of being cached.
---
@@ -155,6 +183,8 @@ If a model fails verification, it's re-downloaded automatically.
| YOLOv5n-Face | 11 MB | ✅ |
| YOLOv5s-Face | 28 MB | ✅ |
| YOLOv5m-Face | 82 MB | ✅ |
| YOLOv8-Lite-S | 7.4 MB | ✅ |
| YOLOv8n-Face | 12 MB | ✅ |
### Recognition Models
@@ -170,6 +200,8 @@ If a model fails verification, it's re-downloaded automatically.
| Model | Size | Download |
|-------|------|----------|
| Landmark106 | 14 MB | ✅ |
| PIPNet WFLW-98 | 47 MB | ✅ |
| PIPNet 300W+CelebA-68 | 46 MB | ✅ |
| AgeGender | 8 MB | ✅ |
| FairFace | 44 MB | ✅ |
| Gaze ResNet34 | 82 MB | ✅ |
@@ -180,14 +212,19 @@ If a model fails verification, it's re-downloaded automatically.
## Clear Cache
Remove cached models:
Find and remove cached models:
```python
from uniface.model_store import get_cache_dir
print(get_cache_dir()) # shows the active cache path
```
```bash
# Remove all cached models
rm -rf ~/.uniface/models/
# Remove specific model
rm ~/.uniface/models/retinaface_mv2.onnx
rm ~/.uniface/models/retinaface_mnet_v2.onnx
```
Models will be re-downloaded on next use.
@@ -196,20 +233,35 @@ Models will be re-downloaded on next use.
## Environment Variables
Set custom cache location via environment variable:
There are three equivalent ways to configure the cache directory:
```bash
export UNIFACE_CACHE_DIR=/path/to/custom/cache
**1. Programmatic API (recommended)**
```python
from uniface.model_store import get_cache_dir, set_cache_dir
set_cache_dir('/path/to/custom/cache')
print(get_cache_dir()) # /path/to/custom/cache
```
**2. Direct environment variable (Python)**
```python
import os
os.environ['UNIFACE_CACHE_DIR'] = '/path/to/custom/cache'
from uniface import RetinaFace
from uniface.detection import RetinaFace
detector = RetinaFace() # Uses custom cache
```
**3. Shell environment variable**
```bash
export UNIFACE_CACHE_DIR=/path/to/custom/cache
```
All three methods set the same `UNIFACE_CACHE_DIR` environment variable under the hood. `get_cache_dir()` always returns the resolved path.
---
## Next Steps

View File

@@ -15,7 +15,7 @@ graph TB
end
subgraph Detection
DET[RetinaFace / SCRFD / YOLOv5Face]
DET[RetinaFace / SCRFD / YOLOv5Face / YOLOv8Face]
end
subgraph Analysis
@@ -23,35 +23,52 @@ graph TB
LMK[Landmarks]
ATTR[Attributes]
GAZE[Gaze]
HPOSE[Head Pose]
PARSE[Parsing]
SPOOF[Anti-Spoofing]
QUAL[Quality]
MATT[Matting]
PRIV[Privacy]
end
subgraph Tracking
TRK[BYTETracker]
end
subgraph Stores
IDX[FAISS Vector Store]
end
subgraph Output
FACE[Face Objects]
end
IMG --> DET
IMG --> MATT
DET --> REC
DET --> LMK
DET --> ATTR
DET --> GAZE
DET --> HPOSE
DET --> PARSE
DET --> SPOOF
DET --> QUAL
DET --> PRIV
DET --> TRK
REC --> IDX
REC --> FACE
LMK --> FACE
ATTR --> FACE
TRK --> FACE
```
---
## Design Principles
### 1. ONNX-First
### 1. Cross-Platform Inference
All models use ONNX Runtime for inference:
UniFace gives you consistent, hardware-accelerated face analysis across macOS, Linux, and Windows from a single API. The optimal hardware backend is selected automatically; models run on ONNX Runtime, with PyTorch for a few optional ones.
- **Cross-platform**: Same models work on macOS, Linux, Windows
- **Hardware acceleration**: Automatic selection of optimal provider
@@ -64,22 +81,26 @@ Core dependencies are kept minimal:
```
numpy # Array operations
opencv-python # Image processing
onnxruntime # Model inference
scikit-image # Geometric transforms (face alignment)
scipy # Scientific computing
requests # Model download
tqdm # Progress bars
```
ONNX Runtime is installed separately via the `uniface[cpu]` or `uniface[gpu]` extra (see [Installation](../installation.md)).
### 3. Simple API
Factory functions and direct instantiation:
Direct class instantiation:
```python
# Factory function
detector = create_detector('retinaface')
from uniface.detection import CenterFace, RetinaFace, SCRFD
# Direct instantiation (recommended)
from uniface import RetinaFace
detector = RetinaFace()
# or
detector = SCRFD()
# or
detector = CenterFace()
```
### 4. Type Safety
@@ -97,19 +118,24 @@ def detect(self, image: np.ndarray) -> list[Face]:
```
uniface/
├── detection/ # Face detection (RetinaFace, SCRFD, YOLOv5Face)
├── recognition/ # Face recognition (ArcFace, MobileFace, SphereFace)
├── landmark/ # 106-point landmarks
├── attribute/ # Age, gender, emotion, race
├── detection/ # Face detection (BlazeFace, CenterFace, RetinaFace, SCRFD, YOLOv5Face, YOLOv8Face)
├── recognition/ # Face recognition (AdaFace, ArcFace, EdgeFace, MobileFace, SphereFace)
├── tracking/ # Multi-object tracking (BYTETracker)
├── landmark/ # Dense landmarks (Landmark106 = 106 pts, PIPNet = 98 / 68 pts, FaceMesh = 468 / 478 pts)
├── attribute/ # Age, gender, emotion, race, face states
├── parsing/ # Face semantic segmentation
├── matting/ # Portrait matting (MODNet)
├── gaze/ # Gaze estimation
├── spoofing/ # Anti-spoofing
├── headpose/ # Head pose estimation
├── spoofing/ # Anti-spoofing (MiniFASNet)
├── quality/ # Face image quality assessment (eDifFIQA)
├── privacy/ # Face anonymization
├── types.py # Dataclasses (Face, GazeResult, etc.)
├── stores/ # Vector stores (FAISS)
├── types.py # Dataclasses (Face, GazeResult, HeadPoseResult, etc.)
├── constants.py # Model weights and URLs
├── model_store.py # Model download and caching
├── onnx_utils.py # ONNX Runtime utilities
└── visualization.py # Drawing utilities
└── draw.py # Drawing utilities
```
---
@@ -120,7 +146,9 @@ A typical face analysis workflow:
```python
import cv2
from uniface import RetinaFace, ArcFace, AgeGender
from uniface.attribute import AgeGender
from uniface.detection import RetinaFace
from uniface.recognition import ArcFace
# 1. Initialize models
detector = RetinaFace()
@@ -139,7 +167,7 @@ for face in faces:
embedding = recognizer.get_normalized_embedding(image, face.landmarks)
# Attributes
attrs = age_gender.predict(image, face.bbox)
attrs = age_gender.predict(image, face)
print(f"Face: {attrs.sex}, {attrs.age} years")
```
@@ -151,12 +179,20 @@ for face in faces:
For convenience, `FaceAnalyzer` combines multiple modules:
```python
from uniface import FaceAnalyzer
from uniface.analyzer import FaceAnalyzer
from uniface.attribute import AgeGender, FairFace
from uniface.detection import RetinaFace
from uniface.recognition import ArcFace
detector = RetinaFace()
recognizer = ArcFace()
age_gender = AgeGender()
fairface = FairFace()
analyzer = FaceAnalyzer(
detect=True,
recognize=True,
attributes=True
detector=detector,
recognizer=recognizer,
predictors=[age_gender, fairface],
)
faces = analyzer.analyze(image)
@@ -169,8 +205,8 @@ for face in faces:
## Model Lifecycle
1. **First use**: Model is downloaded from GitHub releases
2. **Cached**: Stored in `~/.uniface/models/`
1. **First use**: Model is downloaded from GitHub Releases, with a [Hugging Face mirror](https://huggingface.co/yakhyo/uniface-weights) as an automatic fallback when GitHub is unreachable
2. **Cached**: Stored in `~/.uniface/models/` (configurable via `set_cache_dir()` or `UNIFACE_CACHE_DIR`)
3. **Verified**: SHA-256 checksum validation
4. **Loaded**: ONNX Runtime session created
5. **Inference**: Hardware-accelerated execution
@@ -179,6 +215,11 @@ for face in faces:
# Models auto-download on first use
detector = RetinaFace() # Downloads if not cached
# Optionally configure cache location
from uniface.model_store import get_cache_dir, set_cache_dir
set_cache_dir('/data/models')
print(get_cache_dir()) # /data/models
# Or manually pre-download
from uniface.model_store import verify_model_weights
from uniface.constants import RetinaFaceWeights

View File

@@ -11,7 +11,7 @@ This page explains how to tune detection and recognition thresholds for your use
Controls minimum confidence for face detection:
```python
from uniface import RetinaFace
from uniface.detection import RetinaFace
# Default (balanced)
detector = RetinaFace(confidence_threshold=0.5)
@@ -81,7 +81,7 @@ For identity verification (same person check):
```python
import numpy as np
from uniface import compute_similarity
from uniface.face_utils import compute_similarity
similarity = compute_similarity(embedding1, embedding2)
@@ -128,7 +128,7 @@ def calibrate_threshold(same_pairs, diff_pairs, recognizer, detector):
if faces1 and faces2:
emb1 = recognizer.get_normalized_embedding(img1, faces1[0].landmarks)
emb2 = recognizer.get_normalized_embedding(img2, faces2[0].landmarks)
same_scores.append(np.dot(emb1, emb2.T)[0][0])
same_scores.append(float(np.dot(emb1, emb2)))
# Compute similarities for different-person pairs
for img1_path, img2_path in diff_pairs:
@@ -183,7 +183,7 @@ else:
### Emotion
```python
result = emotion_predictor.predict(image, landmarks)
result = emotion_predictor.predict(image, face)
# Filter low-confidence predictions
if result.confidence > 0.6:
@@ -199,19 +199,13 @@ else:
For drawing detections, filter by confidence:
```python
from uniface.visualization import draw_detections
# Only draw high-confidence detections
bboxes = [f.bbox for f in faces if f.confidence > 0.7]
scores = [f.confidence for f in faces if f.confidence > 0.7]
landmarks = [f.landmarks for f in faces if f.confidence > 0.7]
from uniface.draw import draw_detections
# Only draw high-confidence detections (confidence ≥ vis_threshold)
draw_detections(
image=image,
bboxes=bboxes,
scores=scores,
landmarks=landmarks,
vis_threshold=0.6 # Additional visualization filter
faces=faces,
vis_threshold=0.7,
)
```

View File

@@ -6,16 +6,20 @@ Thank you for contributing to UniFace!
## Quick Start
We use [uv](https://docs.astral.sh/uv/) for reproducible dev installs (lockfile-pinned).
```bash
# Install uv first: https://docs.astral.sh/uv/getting-started/installation/
# Clone
git clone https://github.com/yakhyo/uniface.git
cd uniface
# Install dev dependencies
pip install -e ".[dev]"
# Install runtime + cpu + dev extras from uv.lock (--extra gpu for CUDA)
uv sync --extra cpu --extra dev
# Run tests
pytest
uv run pytest
```
---
@@ -32,17 +36,50 @@ ruff check . --fix
**Guidelines:**
- Line length: 120
- Python 3.11+ type hints
- Python 3.10+ type hints
- Google-style docstrings
---
## Pre-commit Hooks
`pre-commit` is included in the `[dev]` extra, so `uv sync` already installs it.
```bash
pip install pre-commit
pre-commit install
pre-commit run --all-files
uv run pre-commit install
uv run pre-commit run --all-files
```
---
## Commit Messages
We follow [Conventional Commits](https://www.conventionalcommits.org/):
```
<type>: <short description>
```
| Type | When to use |
|--------------|--------------------------------------------------|
| **feat** | New feature or capability |
| **fix** | Bug fix |
| **docs** | Documentation changes |
| **style** | Formatting, whitespace (no logic change) |
| **refactor** | Code restructuring without changing behavior |
| **perf** | Performance improvement |
| **test** | Adding or updating tests |
| **build** | Build system or dependencies |
| **ci** | CI/CD and pre-commit configuration |
| **chore** | Routine maintenance and tooling |
**Examples:**
```
feat: Add gaze estimation model
fix: Correct bounding box scaling for non-square images
ci: Add nbstripout pre-commit hook
docs: Update installation instructions
```
---
@@ -67,6 +104,14 @@ pre-commit run --all-files
---
## Releases
Releases are automated via GitHub Actions. Maintainers trigger **Actions → Release Pipeline → Run workflow** with a [PEP 440](https://peps.python.org/pep-0440/) version (e.g. `0.7.0`, `0.7.0rc1`). The pipeline runs tests, bumps `pyproject.toml` + `uniface/__init__.py`, tags the commit, publishes to PyPI, and creates a GitHub Release. Docs redeploy only for stable releases.
See [CONTRIBUTING.md](https://github.com/yakhyo/uniface/blob/main/CONTRIBUTING.md#release-process) for the full process.
---
## Questions?
Open an issue on [GitHub](https://github.com/yakhyo/uniface/issues).

387
docs/datasets.md Normal file
View File

@@ -0,0 +1,387 @@
# Datasets
Overview of all training datasets and evaluation benchmarks used by UniFace models.
---
## Quick Reference
| Task | Dataset | Scale | Models |
| ----------- | ------------------------------------------------ | ---------------------- | ------------------------------------------- |
| Detection | [WIDER FACE](#wider-face) | 32K images | RetinaFace, SCRFD, CenterFace, YOLOv5-Face, YOLOv8-Face |
| Detection | Proprietary (Google) | - | BlazeFace |
| Recognition | [MS1MV2](#ms1mv2) | 5.8M images, 85.7K IDs | MobileFace, SphereFace |
| Recognition | [WebFace600K](#webface600k) | 600K images | ArcFace |
| Recognition | [WebFace4M / WebFace12M](#webface4m-webface12m) | 4M / 12M images | AdaFace, EdgeFace |
| Landmarks | [WFLW](#wflw) / [300W+CelebA](#300w-celeba) | 10K / 3.8K labeled + 202.6K unlabeled | PIPNet (98 / 68 pts) |
| Landmarks | Proprietary (Google) | - | Face Mesh (468 / 478 pts) |
| Attributes | Proprietary (Qualcomm) | - | FaceAttribNet |
| Gaze | [Gaze360](#gaze360) | 238 subjects | MobileGaze |
| Parsing | [CelebAMask-HQ](#celebamask-hq) | 30K images | BiSeNet |
| Attributes | [CelebA](#celeba) | 200K images | AgeGender |
| Attributes | [FairFace](#fairface) | Balanced demographics | FairFace |
| Attributes | [AffectNet](#affectnet) | Emotion labels | Emotion |
---
## Training Datasets
### Face Detection
#### WIDER FACE
Large-scale face detection benchmark with images across 61 event categories. Contains faces with a high degree of variability in scale, pose, occlusion, expression, and illumination.
| Property | Value |
| -------- | ------------------------------------------- |
| Images | ~32,000 (train/val/test split) |
| Faces | ~394,000 annotated |
| Subsets | Easy, Medium, Hard |
| Used by | RetinaFace, SCRFD, YOLOv5-Face, YOLOv8-Face |
!!! info "Download & References"
**Paper**: [WIDER FACE: A Face Detection Benchmark](https://arxiv.org/abs/1511.06523)
**Download**: [http://shuoyang1213.me/WIDERFACE/](http://shuoyang1213.me/WIDERFACE/)
---
### Face Recognition
#### MS1MV2
Refined version of the MS-Celeb-1M dataset, cleaned by InsightFace. Widely used for training face recognition models.
| Property | Value |
| ---------- | ------------------------------ |
| Identities | 85.7K |
| Images | 5.8M |
| Format | Aligned and cropped to 112x112 |
| Used by | MobileFace, SphereFace |
!!! info "Download"
**Kaggle (aligned 112x112)**: [ms1m-arcface-dataset](https://www.kaggle.com/datasets/yakhyokhuja/ms1m-arcface-dataset) (from InsightFace)
**Training code**: [yakhyo/face-recognition](https://github.com/yakhyo/face-recognition)
---
#### WebFace600K
Medium-scale face recognition dataset from the WebFace series.
| Property | Value |
| -------- | ------- |
| Images | ~600K |
| Used by | ArcFace |
!!! info "Source"
**Origin**: [InsightFace](https://github.com/deepinsight/insightface)
**Paper**: [ArcFace: Additive Angular Margin Loss for Deep Face Recognition](https://arxiv.org/abs/1801.07698)
---
#### WebFace4M / WebFace12M
Large-scale face recognition datasets from the WebFace260M collection. Used for training AdaFace models with adaptive quality-aware margin, and EdgeFace models for edge devices.
| Property | WebFace4M | WebFace12M |
| -------- | ------------- | -------------- |
| Images | ~4M | ~12M |
| Used by | AdaFace IR_18 | AdaFace IR_101 |
!!! info "Source"
**Paper**: [AdaFace: Quality Adaptive Margin for Face Recognition](https://arxiv.org/abs/2204.00964)
**Original code**: [mk-minchul/AdaFace](https://github.com/mk-minchul/AdaFace)
---
#### CASIA-WebFace
Smaller-scale face recognition dataset suitable for academic research and lighter training runs.
| Property | Value |
| ---------- | ------------------------------ |
| Identities | 10.6K |
| Images | 491K |
| Format | Aligned and cropped to 112x112 |
| Used by | Alternative training set |
!!! info "Download"
**Kaggle (aligned 112x112)**: [webface-112x112](https://www.kaggle.com/datasets/yakhyokhuja/webface-112x112) (from OpenSphere)
---
#### VGGFace2
Large-scale dataset with wide variations in pose, age, illumination, ethnicity, and profession.
| Property | Value |
| ---------- | ------------------------------ |
| Identities | 8.6K |
| Images | 3.1M |
| Format | Aligned and cropped to 112x112 |
| Used by | Alternative training set |
!!! info "Download"
**Kaggle (aligned 112x112)**: [vggface2-112x112](https://www.kaggle.com/datasets/yakhyokhuja/vggface2-112x112) (from OpenSphere)
---
### Facial Landmarks
#### WFLW
Wider Facial Landmarks in-the-Wild — a 98-point landmark dataset whose images come from
WIDER FACE. Used to train the supervised PIPNet 98-point variant shipped with UniFace.
| Property | Value |
| ---------- | -------------------------------------- |
| Images | 10,000 (7,500 train / 2,500 test) |
| Annotation | 98 manually labeled landmarks per face |
| Used by | PIPNet WFLW-98 |
!!! info "Reference"
**Project page**: [WFLW dataset](https://wywu.github.io/projects/LAB/WFLW.html)
---
#### 300W + CelebA
The 68-point PIPNet variant is trained in a generalizable semi-supervised setting (GSSL):
labeled images come from 300W and unlabeled images come from CelebA.
| Property | Value |
| --------------- | -------------------------------------------------------------------------------- |
| Labeled images | 3,837 (3,148 train: LFPW train + HELEN train + AFW; 689 test: LFPW test + HELEN test + iBUG) |
| Unlabeled | 202,599 (full CelebA; bounding boxes from RetinaFace per the PIPNet paper) |
| Annotation | 68-point iBUG layout |
| Used by | PIPNet 300W+CelebA-68 |
!!! info "Reference"
**Paper**: [PIPNet (Pixel-in-Pixel Net)](https://arxiv.org/abs/2003.03771) (IJCV 2021)
---
### Gaze Estimation
#### Gaze360
Large-scale gaze estimation dataset collected in indoor and outdoor environments with diverse head poses and wide gaze ranges (up to 360 degrees).
| Property | Value |
| ----------- | --------------------- |
| Subjects | 238 |
| Environment | Indoor and outdoor |
| Used by | All MobileGaze models |
!!! info "Download & Preprocessing"
**Download**: [gaze360.csail.mit.edu/download.php](https://gaze360.csail.mit.edu/download.php)
**Preprocessing**: [GazeHub - Gaze360](https://phi-ai.buaa.edu.cn/Gazehub/3D-dataset/#gaze360)
!!! note "UniFace Models"
All MobileGaze models shipped with UniFace are trained exclusively on Gaze360 for 200 epochs.
**Dataset structure:**
```
data/
└── Gaze360/
├── Image/
└── Label/
```
---
#### MPIIFaceGaze
Dataset for appearance-based gaze estimation from laptop webcam images of participants during everyday laptop usage. Supported by the gaze estimation training code but not used for the UniFace pretrained weights.
| Property | Value |
| ----------- | ---------------------------------------- |
| Subjects | 15 |
| Environment | Everyday laptop usage |
| Used by | Supported (not used for UniFace weights) |
!!! info "Download & Preprocessing"
**Download**: [MPIIFaceGaze download page](https://www.mpi-inf.mpg.de/departments/computer-vision-and-machine-learning/research/gaze-based-human-computer-interaction/its-written-all-over-your-face-full-face-appearance-based-gaze-estimation)
**Preprocessing**: [GazeHub - MPIIFaceGaze](https://phi-ai.buaa.edu.cn/Gazehub/3D-dataset/#mpiifacegaze)
**Dataset structure:**
```
data/
└── MPIIFaceGaze/
├── Image/
└── Label/
```
---
### Head Pose Estimation
#### 300W-LP
Large-scale synthesized face dataset with large pose variations, generated from 300W by face profiling. Used for training head pose estimation models.
| Property | Value |
| ----------- | ----------------------------- |
| Images | ~122,000 (synthesized) |
| Source | 300W (profiled) |
| Pose range | ±90° yaw |
| Evaluation | AFLW2000 |
| Used by | All HeadPose models |
!!! info "Download & Reference"
**Paper**: [Face Alignment Across Large Poses: A 3D Solution](https://arxiv.org/abs/1511.07212)
**Training code**: [yakhyo/head-pose-estimation](https://github.com/yakhyo/head-pose-estimation)
!!! note "UniFace Models"
All HeadPose models shipped with UniFace are trained on 300W-LP and evaluated on AFLW2000.
---
### Face Parsing
#### CelebAMask-HQ
High-quality face parsing dataset with pixel-level annotations for 19 facial component classes.
| Property | Value |
| ---------- | ---------------------------- |
| Images | 30,000 |
| Classes | 19 facial components |
| Resolution | High quality |
| Used by | BiSeNet (ResNet18, ResNet34) |
!!! info "Source"
**GitHub**: [switchablenorms/CelebAMask-HQ](https://github.com/switchablenorms/CelebAMask-HQ)
**Training code**: [yakhyo/face-parsing](https://github.com/yakhyo/face-parsing)
**Dataset structure:**
```
dataset/
├── images/ # Input face images
│ ├── image1.jpg
│ └── ...
└── labels/ # Segmentation masks
├── image1.png
└── ...
```
---
### Attribute Analysis
#### CelebA
Large-scale face attributes dataset widely used for training age and gender prediction models.
| Property | Value |
| ---------- | -------------------- |
| Images | ~200K |
| Attributes | 40 binary attributes |
| Used by | AgeGender |
!!! info "Reference"
**Paper**: [Deep Learning Face Attributes in the Wild](https://arxiv.org/abs/1411.7766)
---
#### FairFace
Face attribute dataset designed for balanced representation across race, gender, and age groups. Provides more equitable predictions compared to imbalanced datasets.
| Property | Value |
| ---------- | ----------------------------------- |
| Attributes | Race (7), Gender (2), Age Group (9) |
| Used by | FairFace |
| License | CC BY 4.0 |
!!! info "Reference"
**Paper**: [FairFace: Face Attribute Dataset for Balanced Race, Gender, and Age](https://arxiv.org/abs/1908.04913)
**ONNX inference**: [yakhyo/fairface-onnx](https://github.com/yakhyo/fairface-onnx)
---
#### AffectNet
Large-scale facial expression dataset for emotion recognition training.
| Property | Value |
| -------- | ----------------------------------------------------------------------- |
| Classes | 7 or 8 (Neutral, Happy, Sad, Surprise, Fear, Disgust, Angry + Contempt) |
| Used by | Emotion (AFFECNET7, AFFECNET8) |
!!! info "Reference"
**Paper**: [AffectNet: A Database for Facial Expression, Valence, and Arousal Computing in the Wild](https://ieeexplore.ieee.org/document/8013713)
---
## Evaluation Benchmarks
### Face Detection
#### WIDER FACE Validation Set
The standard benchmark for face detection models. Results are reported across three difficulty subsets.
| Subset | Criteria |
| ------ | --------------------------------------------- |
| Easy | Large, clear, unoccluded faces |
| Medium | Moderate scale and occlusion |
| Hard | Small, heavily occluded, or challenging faces |
See [Model Zoo - Detection](models.md#face-detection-models) for per-model accuracy on each subset.
---
### Face Recognition
Recognition models are evaluated across multiple benchmarks. Aligned 112x112 validation datasets are available as a single download.
!!! info "Download"
**Kaggle**: [agedb-30-calfw-cplfw-lfw-aligned-112x112](https://www.kaggle.com/datasets/yakhyokhuja/agedb-30-calfw-cplfw-lfw-aligned-112x112)
| Benchmark | Description | Used by |
| ------------ | ----------------------------------------------------------------- | ------------------------------- |
| **LFW** | Labeled Faces in the Wild - standard face verification benchmark | ArcFace, MobileFace, SphereFace |
| **CALFW** | Cross-Age LFW - face verification across age gaps | MobileFace, SphereFace |
| **CPLFW** | Cross-Pose LFW - face verification across pose variations | MobileFace, SphereFace |
| **AgeDB-30** | Age database with 30-year age gaps | ArcFace, MobileFace, SphereFace |
| **CFP-FP** | Celebrities in Frontal-Profile - frontal vs. profile verification | ArcFace |
| **IJB-B** | IARPA Janus Benchmark B - TAR@FAR=0.01% | AdaFace |
| **IJB-C** | IARPA Janus Benchmark C - TAR@FAR=1e-4 | AdaFace, ArcFace |
See [Model Zoo - Recognition](models.md#face-recognition-models) for per-model accuracy on each benchmark.
---
### Gaze Estimation
| Benchmark | Metric | Description |
| -------------------- | ------------- | -------------------------------------------- |
| **Gaze360 test set** | MAE (degrees) | Mean Absolute Error in gaze angle prediction |
See [Model Zoo - Gaze](models.md#gaze-estimation-models) for per-model MAE scores.
---
## Training Repositories
For training your own models or reproducing results, see the following repositories:
| Task | Repository | Datasets Supported |
| ----------- | ------------------------------------------------------------------------- | ------------------------------- |
| Detection | [yakhyo/retinaface-pytorch](https://github.com/yakhyo/retinaface-pytorch) | WIDER FACE |
| Recognition | [yakhyo/face-recognition](https://github.com/yakhyo/face-recognition) | MS1MV2, CASIA-WebFace, VGGFace2 |
| Gaze | [yakhyo/gaze-estimation](https://github.com/yakhyo/gaze-estimation) | Gaze360, MPIIFaceGaze |
| Parsing | [yakhyo/face-parsing](https://github.com/yakhyo/face-parsing) | CelebAMask-HQ |

View File

@@ -10,15 +10,15 @@ template: home.html
# UniFace { .hero-title }
<p class="hero-subtitle">A lightweight, production-ready face analysis library built on ONNX Runtime</p>
<p class="hero-subtitle">A Unified Face Analysis Library for Python</p>
[![PyPI](https://img.shields.io/pypi/v/uniface.svg)](https://pypi.org/project/uniface/)
[![Python](https://img.shields.io/badge/Python-3.11%2B-blue)](https://www.python.org/)
[![PyPI Version](https://img.shields.io/pypi/v/uniface.svg?label=Version)](https://pypi.org/project/uniface/)
[![PyPI Downloads](https://static.pepy.tech/personalized-badge/uniface?period=total&units=INTERNATIONAL_SYSTEM&left_color=GRAY&right_color=BLUE&left_text=Downloads)](https://pepy.tech/projects/uniface)
[![License](https://img.shields.io/badge/License-MIT-blue.svg)](https://opensource.org/licenses/MIT)
[![Downloads](https://static.pepy.tech/badge/uniface)](https://pepy.tech/project/uniface)
[![Github Build Status](https://github.com/yakhyo/uniface/actions/workflows/ci.yml/badge.svg)](https://github.com/yakhyo/uniface/actions)
[Get Started](quickstart.md){ .md-button .md-button--primary }
[View on GitHub](https://github.com/yakhyo/uniface){ .md-button }
[:material-rocket-launch: Get Started](quickstart.md){ .md-button .md-button--primary }
[:material-github: View on GitHub](https://github.com/yakhyo/uniface){ .md-button }
</div>
@@ -26,42 +26,80 @@ template: home.html
<div class="feature-card" markdown>
### :material-face-recognition: Face Detection
ONNX-optimized RetinaFace, SCRFD, and YOLOv5-Face models with 5-point landmarks.
RetinaFace, SCRFD, CenterFace, and YOLO detectors with 5-point landmarks, plus BlazeFace for short-range MediaPipe parity.
<a class="feature-card-link" href="modules/detection/" aria-label="Face Detection"></a>
</div>
<div class="feature-card" markdown>
### :material-account-check: Face Recognition
ArcFace, MobileFace, and SphereFace embeddings for identity verification.
AdaFace, ArcFace, EdgeFace, MobileFace, and SphereFace embeddings for identity verification.
<a class="feature-card-link" href="modules/recognition/" aria-label="Face Recognition"></a>
</div>
<div class="feature-card" markdown>
### :material-map-marker: Landmarks
Accurate 106-point facial landmark localization for detailed face analysis.
Dense facial landmark localization — 106-point (2d106det), 98 / 68-point (PIPNet), and 468/478-point 3D mesh (Face Mesh, the 478 variant adding irises) variants.
<a class="feature-card-link" href="modules/landmarks/" aria-label="Landmarks"></a>
</div>
<div class="feature-card" markdown>
### :material-account-details: Attributes
Age, gender, race (FairFace), and emotion detection from faces.
Age, gender, race (FairFace), emotion, and face states (FaceAttribNet) from faces.
<a class="feature-card-link" href="modules/attributes/" aria-label="Attributes"></a>
</div>
<div class="feature-card" markdown>
### :material-face-man-shimmer: Face Parsing
BiSeNet semantic segmentation with 19 facial component classes.
BiSeNet semantic segmentation with 19 facial component classes, plus XSeg face masking.
<a class="feature-card-link" href="modules/parsing/" aria-label="Face Parsing"></a>
</div>
<div class="feature-card" markdown>
### :material-image-off: Portrait Matting
Trimap-free alpha matte with MODNet for background removal and compositing.
<a class="feature-card-link" href="modules/matting/" aria-label="Portrait Matting"></a>
</div>
<div class="feature-card" markdown>
### :material-eye: Gaze Estimation
Real-time gaze direction prediction with MobileGaze models.
<a class="feature-card-link" href="modules/gaze/" aria-label="Gaze Estimation"></a>
</div>
<div class="feature-card" markdown>
### :material-axis-arrow: Head Pose
3D head orientation (pitch, yaw, roll) estimation with 6D rotation models.
<a class="feature-card-link" href="modules/headpose/" aria-label="Head Pose"></a>
</div>
<div class="feature-card" markdown>
### :material-motion-play: Tracking
Multi-object tracking with BYTETracker for persistent face IDs across video frames.
<a class="feature-card-link" href="modules/tracking/" aria-label="Tracking"></a>
</div>
<div class="feature-card" markdown>
### :material-shield-check: Anti-Spoofing
Face liveness detection with MiniFASNet to prevent fraud.
<a class="feature-card-link" href="modules/spoofing/" aria-label="Anti-Spoofing"></a>
</div>
<div class="feature-card" markdown>
### :material-star-check: Face Quality
eDifFIQA scalar quality score to filter or rank faces before recognition.
<a class="feature-card-link" href="modules/quality/" aria-label="Face Quality"></a>
</div>
<div class="feature-card" markdown>
### :material-blur: Privacy
Face anonymization with 5 blur methods for privacy protection.
<a class="feature-card-link" href="modules/privacy/" aria-label="Privacy"></a>
</div>
<div class="feature-card" markdown>
### :material-database-search: Vector Indexing
FAISS-backed embedding store for fast multi-identity face search.
<a class="feature-card-link" href="modules/stores/" aria-label="Vector Indexing"></a>
</div>
</div>
@@ -70,25 +108,24 @@ Face anonymization with 5 blur methods for privacy protection.
## Installation
=== "Standard"
UniFace runs out of the box on macOS, Linux, and Windows, with automatic hardware acceleration on Apple Silicon, NVIDIA CUDA, and CPU. Inference runs on **ONNX Runtime** (with **PyTorch** for a few optional models) — installed automatically by the extra you choose below.
```bash
pip install uniface
```
**CPU / Apple Silicon**
```bash
pip install uniface[cpu]
```
=== "GPU (CUDA)"
**GPU (NVIDIA CUDA)**
```bash
pip install uniface[gpu]
```
```bash
pip install uniface[gpu]
```
=== "From Source"
```bash
git clone https://github.com/yakhyo/uniface.git
cd uniface
pip install -e .
```
**From Source**
```bash
git clone https://github.com/yakhyo/uniface.git
cd uniface
pip install -e ".[cpu]" # or .[gpu] for CUDA
```
---
@@ -99,29 +136,25 @@ Face anonymization with 5 blur methods for privacy protection.
<div class="feature-card" markdown>
### :material-rocket-launch: Quickstart
Get up and running in 5 minutes with common use cases.
[Quickstart Guide →](quickstart.md)
<a class="feature-card-link" href="quickstart/" aria-label="Quickstart"></a>
</div>
<div class="feature-card" markdown>
### :material-school: Tutorials
Step-by-step examples for common workflows.
[View Tutorials →](recipes/image-pipeline.md)
<a class="feature-card-link" href="recipes/image-pipeline/" aria-label="Tutorials"></a>
</div>
<div class="feature-card" markdown>
### :material-api: API Reference
Explore individual modules and their APIs.
[Browse API →](modules/detection.md)
<a class="feature-card-link" href="modules/detection/" aria-label="API Reference"></a>
</div>
<div class="feature-card" markdown>
### :material-book-open-variant: Guides
Learn about the architecture and design principles.
[Read Guides →](concepts/overview.md)
<a class="feature-card-link" href="concepts/overview/" aria-label="Guides"></a>
</div>
</div>

View File

@@ -6,20 +6,32 @@ This guide covers all installation options for UniFace.
## Requirements
- **Python**: 3.11 or higher
- **Python**: 3.10 3.14
- **Operating Systems**: macOS, Linux, Windows
---
## Why Two Extras?
`onnxruntime` (CPU) and `onnxruntime-gpu` (CUDA) both own the same Python namespace.
Installing both at the same time causes file conflicts and silent provider mismatches.
UniFace exposes them as separate, mutually exclusive extras so you install exactly one.
---
## Quick Install
The simplest way to install UniFace:
=== "CPU / Apple Silicon"
```bash
pip install uniface
```
```bash
pip install uniface[cpu]
```
This installs the CPU version with all core dependencies.
=== "NVIDIA GPU (CUDA)"
```bash
pip install uniface[gpu]
```
---
@@ -27,14 +39,16 @@ This installs the CPU version with all core dependencies.
### macOS (Apple Silicon - M1/M2/M3/M4)
For Apple Silicon Macs, the standard installation automatically includes ARM64 optimizations:
The `[cpu]` extra pulls in the standard `onnxruntime` package, which has native ARM64 support
built in since version 1.13. No additional setup is needed for CoreML acceleration.
```bash
pip install uniface
pip install uniface[cpu]
```
!!! tip "Native Performance"
The base `onnxruntime` package has native Apple Silicon support with ARM64 optimizations built-in since version 1.13+. No additional configuration needed.
`onnxruntime` 1.13+ includes ARM64 optimizations out of the box.
UniFace automatically detects and enables `CoreMLExecutionProvider` on Apple Silicon.
Verify ARM64 installation:
@@ -47,19 +61,22 @@ python -c "import platform; print(platform.machine())"
### Linux/Windows with NVIDIA GPU
For CUDA acceleration on NVIDIA GPUs:
```bash
pip install uniface[gpu]
```
This installs `onnxruntime-gpu`, which includes both `CUDAExecutionProvider` and
`CPUExecutionProvider` — no separate CPU package is needed.
**Requirements:**
- CUDA 11.x or 12.x
- NVIDIA driver compatible with your CUDA version
- CUDA 11.x or 12.x toolkit
- cuDNN 8.x
!!! info "CUDA Compatibility"
See [ONNX Runtime GPU requirements](https://onnxruntime.ai/docs/execution-providers/CUDA-ExecutionProvider.html) for detailed compatibility matrix.
See the [ONNX Runtime GPU compatibility matrix](https://onnxruntime.ai/docs/execution-providers/CUDA-ExecutionProvider.html)
for matching CUDA and cuDNN versions.
Verify GPU installation:
@@ -74,7 +91,7 @@ print("Available providers:", ort.get_available_providers())
### CPU-Only (All Platforms)
```bash
pip install uniface
pip install uniface[cpu]
```
Works on all platforms with automatic CPU fallback.
@@ -88,29 +105,60 @@ For development or the latest features:
```bash
git clone https://github.com/yakhyo/uniface.git
cd uniface
pip install -e .
pip install -e ".[cpu]" # CPU / Apple Silicon
pip install -e ".[gpu]" # NVIDIA GPU
```
With development dependencies:
```bash
pip install -e ".[dev]"
pip install -e ".[cpu,dev]"
```
---
## FAISS Vector Store
For fast multi-identity face search using a FAISS vector store:
```bash
pip install faiss-cpu # CPU
pip install faiss-gpu # NVIDIA GPU (CUDA)
```
See the [Stores module](modules/stores.md) for usage.
---
## Dependencies
UniFace has minimal dependencies:
UniFace has minimal core dependencies:
| Package | Purpose |
|---------|---------|
| `numpy` | Array operations |
| `opencv-python` | Image processing |
| `onnxruntime` | Model inference |
| `scikit-image` | Geometric transforms |
| `scipy` | Signal processing |
| `requests` | Model download |
| `tqdm` | Progress bars |
**Runtime extras (install exactly one):**
| Extra | Package | Use case |
|-------|---------|---------|
| `uniface[cpu]` | `onnxruntime` | CPU inference, Apple Silicon |
| `uniface[gpu]` | `onnxruntime-gpu` | NVIDIA CUDA inference |
**Other optional packages:**
| Package | Install | Purpose |
|---------|---------|---------|
| `faiss-cpu` / `faiss-gpu` | `pip install faiss-cpu` | FAISS vector store |
| `torch` | `pip install torch` | Emotion model (TorchScript) |
| `torchvision` | `pip install torchvision` | Faster NMS for YOLO detectors |
---
## Verify Installation
@@ -126,24 +174,88 @@ import onnxruntime as ort
print(f"Available providers: {ort.get_available_providers()}")
# Quick test
from uniface import RetinaFace
from uniface.detection import RetinaFace
detector = RetinaFace()
print("Installation successful!")
```
---
## Upgrading
When upgrading UniFace, stay consistent with your runtime extra:
```bash
pip install --upgrade uniface[cpu] # or uniface[gpu]
```
If you are switching from CPU to GPU (or vice versa):
```bash
pip uninstall onnxruntime onnxruntime-gpu -y
pip install uniface[gpu] # install the one you want
```
---
## Pre-release Versions
UniFace ships release candidates and betas to PyPI ahead of stable releases (versions like `4.0.0rc1`, `4.0.0b1`, `4.0.0a1`). These let you try upcoming features before they're finalized.
`pip install uniface` always installs the latest **stable** release. To opt in to pre-releases:
```bash
# Latest pre-release (if newer than latest stable)
pip install uniface[cpu] --pre
# A specific pre-release
pip install uniface[cpu]==4.0.0rc1
```
Pre-releases are not recommended for production — APIs may still change before the stable release.
---
## Troubleshooting
### onnxruntime Not Found
If you see:
```
ImportError: onnxruntime is not installed. Install it with one of:
pip install uniface[cpu] # CPU / Apple Silicon
pip install uniface[gpu] # NVIDIA GPU (CUDA)
```
You installed uniface without an extra. Run the appropriate command above.
---
### Both onnxruntime and onnxruntime-gpu Installed
If you previously ran `pip install uniface[gpu]` on top of a `pip install uniface[cpu]`
(or vice versa), you may have both packages installed simultaneously, which causes conflicts.
Fix it with:
```bash
pip uninstall onnxruntime onnxruntime-gpu -y
pip install uniface[gpu] # or uniface[cpu]
```
---
### Import Errors
If you encounter import errors, ensure you're using Python 3.11+:
Ensure you're using Python 3.10+:
```bash
python --version
# Should show: Python 3.11.x or higher
# Should show: Python 3.10.x or higher
```
---
### Model Download Issues
Models are automatically downloaded on first use. If downloads fail:
@@ -157,6 +269,25 @@ model_path = verify_model_weights(RetinaFaceWeights.MNET_V2)
print(f"Model downloaded to: {model_path}")
```
---
### CUDA Not Detected
1. Verify CUDA installation:
```bash
nvidia-smi
```
2. Check CUDA version compatibility with ONNX Runtime.
3. Reinstall the GPU extra cleanly:
```bash
pip uninstall onnxruntime onnxruntime-gpu -y
pip install uniface[gpu]
```
---
### Performance Issues on Mac
Verify you're using the ARM64 build (not x86_64 via Rosetta):

View File

@@ -12,11 +12,19 @@ UniFace is released under the [MIT License](https://opensource.org/licenses/MIT)
|-------|--------|---------|
| RetinaFace | [yakhyo/retinaface-pytorch](https://github.com/yakhyo/retinaface-pytorch) | MIT |
| SCRFD | [InsightFace](https://github.com/deepinsight/insightface) | MIT |
| CenterFace | [Star-Clouds/CenterFace](https://github.com/Star-Clouds/CenterFace) | MIT |
| BlazeFace | [yakhyo/mediapipe-face-mesh-onnx](https://github.com/yakhyo/mediapipe-face-mesh-onnx) — architecture & weights from Google [MediaPipe](https://github.com/google-ai-edge/mediapipe) | Apache-2.0 |
| YOLOv5-Face | [yakhyo/yolov5-face-onnx-inference](https://github.com/yakhyo/yolov5-face-onnx-inference) | GPL-3.0 |
| YOLOv8-Face | [yakhyo/yolov8-face-onnx-inference](https://github.com/yakhyo/yolov8-face-onnx-inference) | GPL-3.0 |
| AdaFace | [yakhyo/adaface-onnx](https://github.com/yakhyo/adaface-onnx) | MIT |
| ArcFace | [InsightFace](https://github.com/deepinsight/insightface) | MIT |
| MobileFace | [yakhyo/face-recognition](https://github.com/yakhyo/face-recognition) | MIT |
| SphereFace | [yakhyo/face-recognition](https://github.com/yakhyo/face-recognition) | MIT |
| BiSeNet | [yakhyo/face-parsing](https://github.com/yakhyo/face-parsing) | MIT |
| MobileGaze | [yakhyo/gaze-estimation](https://github.com/yakhyo/gaze-estimation) | MIT |
| MODNet | [yakhyo/modnet](https://github.com/yakhyo/modnet) | Apache-2.0 |
| MiniFASNet | [yakhyo/face-anti-spoofing](https://github.com/yakhyo/face-anti-spoofing) | Apache-2.0 |
| FairFace | [yakhyo/fairface-onnx](https://github.com/yakhyo/fairface-onnx) | CC BY 4.0 |
| FaceAttribNet | [yakhyo/face-attribute](https://github.com/yakhyo/face-attribute) — architecture & weights © Qualcomm Technologies, Inc. ([qualcomm/ai-hub-models](https://github.com/qualcomm/ai-hub-models)) | BSD-3-Clause |
| PIPNet | [yakhyo/pipnet-onnx](https://github.com/yakhyo/pipnet-onnx) — meanface tables vendored from [jhb86253817/PIPNet](https://github.com/jhb86253817/PIPNet) | MIT |
| Face Mesh | [yakhyo/mediapipe-face-mesh-onnx](https://github.com/yakhyo/mediapipe-face-mesh-onnx) — topology & weights from Google [MediaPipe](https://github.com/google-ai-edge/mediapipe), sourced via [PINTO0309's ONNX conversion](https://github.com/PINTO0309/facemesh_onnx_tensorrt); tessellation tables vendored from the same source | Apache-2.0 |

View File

@@ -1,6 +1,6 @@
# Model Zoo
Complete guide to all available models, their performance characteristics, and selection criteria.
Complete guide to all available models and their performance characteristics.
---
@@ -8,110 +8,209 @@ Complete guide to all available models, their performance characteristics, and s
### RetinaFace Family
RetinaFace models are trained on the WIDER FACE dataset and provide excellent accuracy-speed tradeoffs.
RetinaFace models are trained on the [WIDER FACE](datasets.md#wider-face) dataset.
| Model Name | Params | Size | Easy | Medium | Hard | Use Case |
| -------------- | ------ | ----- | ------ | ------ | ------ | ----------------------------- |
| `MNET_025` | 0.4M | 1.7MB | 88.48% | 87.02% | 80.61% | Mobile/Edge devices |
| `MNET_050` | 1.0M | 2.6MB | 89.42% | 87.97% | 82.40% | Mobile/Edge devices |
| `MNET_V1` | 3.5M | 3.8MB | 90.59% | 89.14% | 84.13% | Balanced mobile |
| `MNET_V2` :material-check-circle: | 3.2M | 3.5MB | 91.70% | 91.03% | 86.60% | **Default** |
| `RESNET18` | 11.7M | 27MB | 92.50% | 91.02% | 86.63% | Server/High accuracy |
| `RESNET34` | 24.8M | 56MB | 94.16% | 93.12% | 88.90% | Maximum accuracy |
| Model Name | Params | Size | Easy | Medium | Hard |
| -------------- | ------ | ----- | ------ | ------ | ------ |
| `MNET_025` | 0.4M | 1.7MB | 88.48% | 87.02% | 80.61% |
| `MNET_050` | 1.0M | 2.6MB | 89.42% | 87.97% | 82.40% |
| `MNET_V1` | 3.5M | 3.8MB | 90.59% | 89.14% | 84.13% |
| `MNET_V2` :material-check-circle: | 3.2M | 3.5MB | 91.70% | 91.03% | 86.60% |
| `RESNET18` | 11.7M | 27MB | 92.50% | 91.02% | 86.63% |
| `RESNET34` | 24.8M | 56MB | 94.16% | 93.12% | 88.90% |
| `RESNET50` | 27.4M | 104MB | 94.72%* | 93.67%* | 88.82%* |
*\* `RESNET50` weights come from [HivisionIDPhotos](https://huggingface.co/TheEeeeLin/HivisionIDPhotos_matting/blob/main/retinaface-resnet50.onnx);
its scores are measured with [widerface_evaluation](https://github.com/yakhyo/widerface_evaluation) rather than quoted
from the paper. It trails `RESNET34` on Hard despite being ~2x the size.*
!!! info "Accuracy & Benchmarks"
**Accuracy**: WIDER FACE validation set (Easy/Medium/Hard subsets) - from [RetinaFace paper](https://arxiv.org/abs/1905.00641)
**Speed**: Benchmark on your own hardware using `python tools/detection.py --source <image> --iterations 100`
**Speed**: Benchmark on your own hardware using `python tools/detect.py --source <image>`
---
### SCRFD Family
SCRFD (Sample and Computation Redistribution for Efficient Face Detection) models offer state-of-the-art speed-accuracy tradeoffs.
SCRFD (Sample and Computation Redistribution for Efficient Face Detection) models trained on [WIDER FACE](datasets.md#wider-face) dataset.
| Model Name | Params | Size | Easy | Medium | Hard | Use Case |
| ---------------- | ------ | ----- | ------ | ------ | ------ | ------------------------------- |
| `SCRFD_500M` | 0.6M | 2.5MB | 90.57% | 88.12% | 68.51% | Real-time applications |
| `SCRFD_10G` :material-check-circle: | 4.2M | 17MB | 95.16% | 93.87% | 83.05% | **High accuracy + speed** |
| Model Name | Params | Size | Easy | Medium | Hard |
| ---------------- | ------ | ----- | ------ | ------ | ------ |
| `SCRFD_500M_KPS` | 0.6M | 2.5MB | 90.57% | 88.12% | 68.51% |
| `SCRFD_10G_KPS` :material-check-circle: | 4.2M | 17MB | 95.16% | 93.87% | 83.05% |
!!! info "Accuracy & Benchmarks"
**Accuracy**: WIDER FACE validation set - from [SCRFD paper](https://arxiv.org/abs/2105.04714)
**Speed**: Benchmark on your own hardware using `python tools/detection.py --source <image> --iterations 100`
**Speed**: Benchmark on your own hardware using `python tools/detect.py --source <image>`
---
### CenterFace
CenterFace is an anchor-free detector (MobileNetV2 + FPN) that treats faces as center points, with joint 5-point landmark prediction. Trained on [WIDER FACE](datasets.md#wider-face) dataset.
| Model Name | Size | Easy | Medium | Hard |
| ---------- | ----- | ------ | ------ | ------ |
| `DEFAULT` :material-check-circle: | 7.0MB | 92.2%* | 91.1%* | 78.2%* |
*\* Single inference on the original image (SIO). With multi-scale and flip testing the
[original repo](https://github.com/Star-Clouds/CenterFace) reports 93.5% / 92.4% / 87.5%.*
!!! info "Accuracy & Benchmarks"
**Accuracy**: WIDER FACE validation set - from [CenterFace paper](https://arxiv.org/abs/1911.03599)
**Speed**: Benchmark on your own hardware using `python tools/detect.py --source <image> --method centerface`
!!! note "Input Size"
The ONNX model has dynamic spatial dimensions. `input_size` is an upper bound
(default 640×640), not a fixed shape: larger images are scaled down to fit and smaller
ones are left alone, preserving aspect ratio. Each side is then rounded up to a
multiple of 32. Pass `input_size=None` to always run at native resolution, like upstream.
!!! warning "Weakest landmarks for rotated faces"
CenterFace decodes all five points from a single stride-4 cell, and its landmark
accuracy is the lowest in this library at every in-plane rotation — roughly 8.5% NME at
30° and 11.7% at 45°, against 5.4% / 6.2% for RetinaFace-R50. Past ~60° all WIDER
FACE-trained detectors here degrade sharply. Prefer `RetinaFace` or `SCRFD` when faces
may be tilted; this is a property of the model, not of the port.
---
### BlazeFace
Google MediaPipe's short-range SSD detector — the one `mp.solutions.face_mesh` runs
internally. Pairing it with [Face Mesh](#face-mesh-468-or-478-points) reproduces MediaPipe's own output.
| Model Name | Input | Keypoints | Size |
| ---------- | ----- | --------- | ---- |
| `DEFAULT` :material-check-circle: | 128×128 | 6 | 0.5MB |
!!! warning "Not for recognition pipelines"
BlazeFace emits 6 MediaPipe keypoints (right eye, left eye, nose tip, mouth **center**,
right/left ear tragion), not the 5-point alignment template. With no mouth corners they
cannot drive face alignment, so `supports_alignment` is `False` and `FaceAnalyzer`
disables recognition for it.
!!! note "Short range"
Tuned for faces within roughly 2m. Not benchmarked on WIDER FACE, and less accurate
than SCRFD or YOLOv8 on small or distant faces. Choose it for its 0.5MB footprint or
for MediaPipe parity.
!!! info "Reference"
**Paper**: [BlazeFace: Sub-millisecond Neural Face Detection on Mobile GPUs](https://arxiv.org/abs/1907.05047)
**Source**: [yakhyo/mediapipe-face-mesh-onnx](https://github.com/yakhyo/mediapipe-face-mesh-onnx) — ONNX export of Google [MediaPipe](https://github.com/google-ai-edge/mediapipe)
---
### YOLOv5-Face Family
YOLOv5-Face models provide excellent detection accuracy with 5-point facial landmarks, optimized for real-time applications.
YOLOv5-Face models provide detection with 5-point facial landmarks, trained on [WIDER FACE](datasets.md#wider-face) dataset.
| Model Name | Size | Easy | Medium | Hard | Use Case |
| -------------- | ---- | ------ | ------ | ------ | ------------------------------ |
| `YOLOV5N` | 11MB | 93.61% | 91.52% | 80.53% | Lightweight/Mobile |
| `YOLOV5S` :material-check-circle: | 28MB | 94.33% | 92.61% | 83.15% | **Real-time + accuracy** |
| `YOLOV5M` | 82MB | 95.30% | 93.76% | 85.28% | High accuracy |
| Model Name | Size | Easy | Medium | Hard |
| -------------- | ---- | ------ | ------ | ------ |
| `YOLOV5N` | 11MB | 93.61% | 91.52% | 80.53% |
| `YOLOV5S` :material-check-circle: | 28MB | 94.33% | 92.61% | 83.15% |
| `YOLOV5M` | 82MB | 95.30% | 93.76% | 85.28% |
!!! info "Accuracy & Benchmarks"
**Accuracy**: WIDER FACE validation set - from [YOLOv5-Face paper](https://arxiv.org/abs/2105.12931)
**Speed**: Benchmark on your own hardware using `python tools/detection.py --source <image> --iterations 100`
**Speed**: Benchmark on your own hardware using `python tools/detect.py --source <image>`
!!! note "Fixed Input Size"
All YOLOv5-Face models use a fixed input size of 640×640. Models exported to ONNX from [deepcam-cn/yolov5-face](https://github.com/deepcam-cn/yolov5-face).
All YOLOv5-Face models use a fixed input size of 640×640.
---
### YOLOv8-Face Family
YOLOv8-Face models use anchor-free design with DFL (Distribution Focal Loss) for bbox regression. Provides detection with 5-point facial landmarks.
| Model Name | Size | Easy | Medium | Hard |
| ---------------- | ------ | ------ | ------ | ------ |
| `YOLOV8_LITE_S`| 7.4MB | 93.4% | 91.2% | 78.6% |
| `YOLOV8N` :material-check-circle: | 12MB | 94.6% | 92.3% | 79.6% |
!!! info "Accuracy & Benchmarks"
**Accuracy**: WIDER FACE validation set (Easy/Medium/Hard subsets)
**Speed**: Benchmark on your own hardware using `python tools/detect.py --source <image> --method yolov8face`
!!! note "Fixed Input Size"
All YOLOv8-Face models use a fixed input size of 640×640.
---
## Face Recognition Models
### AdaFace
Face recognition using adaptive margin based on image quality.
| Model Name | Backbone | Dataset | Size | IJB-B TAR | IJB-C TAR |
| ----------- | -------- | ----------- | ------ | --------- | --------- |
| `IR_18` :material-check-circle: | IR-18 | WebFace4M | 92 MB | 93.03% | 94.99% |
| `IR_101` | IR-101 | WebFace12M | 249 MB | - | 97.66% |
!!! info "Training Data & Accuracy"
**Dataset**: [WebFace4M / WebFace12M](datasets.md#webface4m-webface12m) (4M / 12M images)
**Accuracy**: IJB-B and IJB-C benchmarks, TAR@FAR=0.01%
!!! tip "Key Innovation"
AdaFace introduces adaptive margin that adjusts based on image quality, providing better performance on low-quality images compared to fixed-margin approaches.
---
### ArcFace
State-of-the-art face recognition using additive angular margin loss.
Face recognition using additive angular margin loss.
| Model Name | Backbone | Params | Size | Use Case |
| ----------- | --------- | ------ | ----- | -------------------------------- |
| `MNET` :material-check-circle: | MobileNet | 2.0M | 8MB | **Balanced (recommended)** |
| `RESNET` | ResNet50 | 43.6M | 166MB | Maximum accuracy |
| Model Name | Backbone | Params | Size | LFW | CFP-FP | AgeDB-30 | IJB-C |
| ----------- | --------- | ------ | ----- | ------ | ------ | -------- | ----- |
| `MNET` :material-check-circle: | MobileNet | 2.0M | 8MB | 99.70% | 98.00% | 96.58% | 95.02% |
| `RESNET` | ResNet50 | 43.6M | 166MB | 99.83% | 99.33% | 98.23% | 97.25% |
!!! info "Training Data"
**Dataset**: Trained on MS1M-V2 (5.8M images, 85K identities)
**Dataset**: Trained on [WebFace600K](datasets.md#webface600k) (600K images)
**Accuracy**: Benchmark on your own dataset or use standard face verification benchmarks
**Accuracy**: IJB-C accuracy reported as TAR@FAR=1e-4
---
### MobileFace
Lightweight face recognition optimized for mobile devices.
Lightweight face recognition models with MobileNet backbones.
| Model Name | Backbone | Params | Size | LFW | CALFW | CPLFW | AgeDB-30 | Use Case |
| ----------------- | ---------------- | ------ | ---- | ------ | ------ | ------ | -------- | --------------------- |
| `MNET_025` | MobileNetV1 0.25 | 0.36M | 1MB | 98.76% | 92.02% | 82.37% | 90.02% | Ultra-lightweight |
| `MNET_V2` :material-check-circle: | MobileNetV2 | 2.29M | 4MB | 99.55% | 94.87% | 86.89% | 95.16% | **Mobile/Edge** |
| `MNET_V3_SMALL` | MobileNetV3-S | 1.25M | 3MB | 99.30% | 93.77% | 85.29% | 92.79% | Mobile optimized |
| `MNET_V3_LARGE` | MobileNetV3-L | 3.52M | 10MB | 99.53% | 94.56% | 86.79% | 95.13% | Balanced mobile |
| Model Name | Backbone | Params | Size | LFW | CALFW | CPLFW | AgeDB-30 |
| ----------------- | ---------------- | ------ | ---- | ------ | ------ | ------ | -------- |
| `MNET_025` | MobileNetV1 0.25 | 0.36M | 1MB | 98.76% | 92.02% | 82.37% | 90.02% |
| `MNET_V2` :material-check-circle: | MobileNetV2 | 2.29M | 4MB | 99.55% | 94.87% | 86.89% | 95.16% |
| `MNET_V3_SMALL` | MobileNetV3-S | 1.25M | 3MB | 99.30% | 93.77% | 85.29% | 92.79% |
| `MNET_V3_LARGE` | MobileNetV3-L | 3.52M | 10MB | 99.53% | 94.56% | 86.79% | 95.13% |
!!! info "Training Data"
**Dataset**: Trained on MS1M-V2 (5.8M images, 85K identities)
**Dataset**: Trained on [MS1MV2](datasets.md#ms1mv2) (5.8M images, 85K identities)
**Accuracy**: Evaluated on LFW, CALFW, CPLFW, and AgeDB-30 benchmarks
!!! tip "Use Case"
These models are lightweight alternatives to ArcFace for resource-constrained environments.
---
### SphereFace
Face recognition using angular softmax loss.
| Model Name | Backbone | Params | Size | LFW | CALFW | CPLFW | AgeDB-30 | Use Case |
| ------------ | -------- | ------ | ---- | ------ | ------ | ------ | -------- | ------------------- |
| `SPHERE20` | Sphere20 | 24.5M | 50MB | 99.67% | 95.61% | 88.75% | 96.58% | Research/Comparison |
| `SPHERE36` | Sphere36 | 34.6M | 92MB | 99.72% | 95.64% | 89.92% | 96.83% | Research/Comparison |
| Model Name | Backbone | Params | Size | LFW | CALFW | CPLFW | AgeDB-30 |
| ------------ | -------- | ------ | ---- | ------ | ------ | ------ | -------- |
| `SPHERE20` | Sphere20 | 24.5M | 50MB | 99.67% | 95.61% | 88.75% | 96.58% |
| `SPHERE36` | Sphere36 | 34.6M | 92MB | 99.72% | 95.64% | 89.92% | 96.83% |
!!! info "Training Data"
**Dataset**: Trained on MS1M-V2 (5.8M images, 85K identities)
**Dataset**: Trained on [MS1MV2](datasets.md#ms1mv2) (5.8M images, 85K identities)
**Accuracy**: Evaluated on LFW, CALFW, CPLFW, and AgeDB-30 benchmarks
@@ -120,15 +219,33 @@ Face recognition using angular softmax loss.
---
### EdgeFace
Efficient face recognition designed for edge devices, using EdgeNeXt backbone with optional LoRA compression.
| Model Name | Backbone | Params | MFLOPs | Size | LFW | CALFW | CPLFW | CFP-FP | AgeDB-30 |
| --------------- | -------- | ------ | ------ | ----- | ------ | ------ | ------ | ------ | -------- |
| `XXS` :material-check-circle: | EdgeNeXt | 1.24M | 94 | ~5 MB | 99.57% | 94.83% | 90.27% | 93.63% | 94.92% |
| `XS_GAMMA_06` | EdgeNeXt | 1.77M | 154 | ~7 MB | 99.73% | 95.28% | 91.58% | 94.71% | 96.08% |
| `S_GAMMA_05` | EdgeNeXt | 3.65M | 306 | ~14 MB | 99.78% | 95.55% | 92.48% | 95.74% | 97.03% |
| `BASE` | EdgeNeXt | 18.2M | 1399 | ~70 MB | 99.83% | 96.07% | 93.75% | 97.01% | 97.60% |
!!! info "Training Data & Reference"
**Paper**: [EdgeFace: Efficient Face Recognition Model for Edge Devices](https://arxiv.org/abs/2307.01838v2) (IEEE T-BIOM 2024)
**Source**: [github.com/otroshi/edgeface](https://github.com/otroshi/edgeface) | [github.com/yakhyo/edgeface-onnx](https://github.com/yakhyo/edgeface-onnx)
---
## Facial Landmark Models
### 106-Point Landmark Detection
High-precision facial landmark localization.
Facial landmark localization model.
| Model Name | Points | Params | Size | Use Case |
| ---------- | ------ | ------ | ---- | ------------------------ |
| `2D106` | 106 | 3.7M | 14MB | Face alignment, analysis |
| Model Name | Points | Params | Size |
| ---------- | ------ | ------ | ---- |
| `DEFAULT` (2d_106) | 106 | 3.7M | 14MB |
**Landmark Groups:**
@@ -142,16 +259,63 @@ High-precision facial landmark localization.
---
### PIPNet (98 / 68 points)
PIPNet (Pixel-in-Pixel Net) facial landmark detector. ResNet-18 backbone, 256×256 input.
| Model Name | Points | Backbone | Dataset | Size |
| ---------- | ------ | -------- | ------- | ---- |
| `WFLW_98` :material-check-circle: | 98 | ResNet-18 | WFLW (supervised) | 47 MB |
| `DW300_CELEBA_68` | 68 | ResNet-18 | 300W+CelebA (GSSL) | 46 MB |
!!! info "Reference"
**Paper**: [PIPNet: Towards Efficient Facial Landmark Detection in the Wild](https://arxiv.org/abs/2003.03771) (IJCV 2021)
**Source**: [yakhyo/pipnet-onnx](https://github.com/yakhyo/pipnet-onnx) — ONNX export from [jhb86253817/PIPNet](https://github.com/jhb86253817/PIPNet)
!!! note "Auto-selected meanface"
Both variants share the same architecture; the number of landmarks (and the matching
meanface table) is inferred from the ONNX output channel count.
---
### Face Mesh (468 or 478 points)
Google MediaPipe's dense mesh. The only UniFace landmarker that returns 3D points and a
face-presence score, and the only model that batches every face of an image into one
inference call.
| Model Name | Points | Input | Size |
| ---------- | ------ | ----- | ---- |
| `V1_468` :material-check-circle: | 468 (3D) | 192×192 | 2.4MB |
| `V2_478` | 478 (3D, with irises) | 256×256 | 4.6MB |
!!! info "Reference"
**Paper**: [Real-time Facial Surface Geometry from Monocular Video on Mobile GPUs](https://arxiv.org/abs/1907.06724)
**Source**: [yakhyo/mediapipe-face-mesh-onnx](https://github.com/yakhyo/mediapipe-face-mesh-onnx) — architectures recovered from Google [MediaPipe](https://github.com/google-ai-edge/mediapipe). `V1_468`'s weights came via [PINTO0309's conversion](https://github.com/PINTO0309/facemesh_onnx_tensorrt); `V2_478`'s were read directly from the `.tflite` in Google's `face_landmarker.task` bundle
!!! note "Works with any detector"
Face Mesh needs a bounding box plus the first two landmarks (the eyes) to align its
crop. Rows 0/1 are the viewer-left and viewer-right eye in both the 5-point template
and BlazeFace's 6-point layout, so every UniFace detector can seed it.
!!! note "Relative depth"
The `z` coordinate is relative depth on the same pixel scale as `x`/`y` (smaller is
closer). It has no absolute origin and is not comparable between faces or images.
---
## Attribute Analysis Models
### Age & Gender Detection
| Model Name | Attributes | Params | Size | Use Case |
| ----------- | ----------- | ------ | ---- | --------------- |
| `AgeGender` | Age, Gender | 2.1M | 8MB | General purpose |
| Model Name | Attributes | Params | Size |
| ----------- | ----------- | ------ | ---- |
| `DEFAULT` | Age, Gender | 2.1M | 8MB |
!!! info "Training Data"
**Dataset**: Trained on CelebA
**Dataset**: Trained on [CelebA](datasets.md#celeba)
!!! warning "Accuracy Note"
Accuracy varies by demographic and image quality. Test on your specific use case.
@@ -160,12 +324,12 @@ High-precision facial landmark localization.
### FairFace Attributes
| Model Name | Attributes | Params | Size | Use Case |
| ----------- | --------------------- | ------ | ----- | --------------------------- |
| `FairFace` | Race, Gender, Age Group | - | 44MB | Balanced demographic prediction |
| Model Name | Attributes | Params | Size |
| ----------- | --------------------- | ------ | ----- |
| `DEFAULT` | Race, Gender, Age Group | - | 44MB |
!!! info "Training Data"
**Dataset**: Trained on FairFace dataset with balanced demographics
**Dataset**: Trained on [FairFace](datasets.md#fairface) dataset with balanced demographics
!!! tip "Equitable Predictions"
FairFace provides more equitable predictions across different racial and gender groups.
@@ -178,41 +342,58 @@ High-precision facial landmark localization.
### Emotion Detection
| Model Name | Classes | Params | Size | Use Case |
| ------------- | ------- | ------ | ---- | --------------- |
| `AFFECNET7` | 7 | 0.5M | 2MB | 7-class emotion |
| `AFFECNET8` | 8 | 0.5M | 2MB | 8-class emotion |
| Model Name | Classes | Params | Size |
| ------------- | ------- | ------ | ---- |
| `AFFECNET7` | 7 | 0.5M | 2MB |
| `AFFECNET8` | 8 | 0.5M | 2MB |
**Classes (7)**: Neutral, Happy, Sad, Surprise, Fear, Disgust, Anger
**Classes (7)**: Neutral, Happy, Sad, Surprise, Fear, Disgust, Angry
**Classes (8)**: Above + Contempt
!!! info "Training Data"
**Dataset**: Trained on AffectNet
**Dataset**: Trained on [AffectNet](datasets.md#affectnet)
!!! note "Accuracy Note"
Emotion detection accuracy depends heavily on facial expression clarity and cultural context.
---
### Face State Detection (FaceAttribNet)
| Model Name | Attributes | Params | Size |
| --------------- | ------------------------------------------------------- | ------ | ---- |
| `DEFAULT` | Eye openness (L/R), Eyeglasses, Mask, Sunglasses | 10.8M | 41MB |
Qualcomm's [Facial-Attribute-Detection](https://github.com/qualcomm/ai-hub-models/tree/main/src/qai_hub_models/models/face_attrib_net) model. Outputs five independent binary probabilities from a 128x128 face crop.
!!! info "Training Data"
**Dataset**: Trained by Qualcomm on a proprietary face dataset
!!! warning "Multi-label Output"
The five probabilities come from independent binary heads: they do not sum to 1
and several can be high at once. Threshold each attribute separately; never `argmax`.
---
## Gaze Estimation Models
### MobileGaze Family
Real-time gaze direction prediction models trained on Gaze360 dataset. Returns pitch (vertical) and yaw (horizontal) angles in radians.
Gaze direction prediction models trained on [Gaze360](datasets.md#gaze360) dataset. Returns pitch (vertical) and yaw (horizontal) angles in radians.
| Model Name | Params | Size | MAE* | Use Case |
| -------------- | ------ | ------- | ----- | ----------------------------- |
| `RESNET18` | 11.7M | 43 MB | 12.84 | Balanced accuracy/speed |
| `RESNET34` :material-check-circle: | 24.8M | 81.6 MB | 11.33 | **Default** |
| `RESNET50` | 25.6M | 91.3 MB | 11.34 | High accuracy |
| `MOBILENET_V2` | 3.5M | 9.59 MB | 13.07 | Mobile/Edge devices |
| `MOBILEONE_S0` | 2.1M | 4.8 MB | 12.58 | Lightweight/Real-time |
| Model Name | Params | Size | MAE* |
| -------------- | ------ | ------- | ----- |
| `RESNET18` | 11.7M | 43 MB | 12.84 |
| `RESNET34` :material-check-circle: | 24.8M | 81.6 MB | 11.33 |
| `RESNET50` | 25.6M | 91.3 MB | 11.34 |
| `MOBILENET_V2` | 3.5M | 9.59 MB | 13.07 |
| `MOBILEONE_S0` | 2.1M | 4.8 MB | 12.58 |
*MAE (Mean Absolute Error) in degrees on Gaze360 test set - lower is better
!!! info "Training Data"
**Dataset**: Trained on Gaze360 (indoor/outdoor scenes with diverse head poses)
**Dataset**: Trained on [Gaze360](datasets.md#gaze360) (indoor/outdoor scenes with diverse head poses)
**Training**: 200 epochs with classification-based approach (binned angles)
@@ -221,19 +402,46 @@ Real-time gaze direction prediction models trained on Gaze360 dataset. Returns p
---
## Head Pose Estimation Models
### HeadPose Family
Head pose estimation models using 6D rotation representation. Trained on [300W-LP](datasets.md#300w-lp) dataset, evaluated on AFLW2000. Returns pitch, yaw, and roll angles in degrees.
| Model Name | Backbone | Size | MAE* |
| -------------- | -------- | ------- | ----- |
| `RESNET18` :material-check-circle: | ResNet18 | 43 MB | 5.22° |
| `RESNET34` | ResNet34 | 82 MB | 5.07° |
| `RESNET50` | ResNet50 | 91 MB | 4.83° |
| `MOBILENET_V2` | MobileNetV2 | 9.6 MB | 5.72° |
| `MOBILENET_V3_SMALL` | MobileNetV3-Small | 4.8 MB | 6.31° |
| `MOBILENET_V3_LARGE` | MobileNetV3-Large | 16 MB | 5.58° |
*MAE (Mean Absolute Error) in degrees on AFLW2000 test set — lower is better
!!! info "Training Data"
**Dataset**: Trained on [300W-LP](datasets.md#300w-lp) (synthesized large-pose faces from 300W)
**Method**: 6D rotation representation (rotation matrix → Euler angles)
!!! note "Input Requirements"
Requires face crop as input. Use face detection first to obtain bounding boxes.
---
## Face Parsing Models
### BiSeNet Family
BiSeNet (Bilateral Segmentation Network) models for semantic face parsing. Segments face images into 19 facial component classes.
| Model Name | Params | Size | Classes | Use Case |
| -------------- | ------ | ------- | ------- | ----------------------------- |
| `RESNET18` :material-check-circle: | 13.3M | 50.7 MB | 19 | **Default** |
| `RESNET34` | 24.1M | 89.2 MB | 19 | Higher accuracy |
| Model Name | Params | Size | Classes |
| -------------- | ------ | ------- | ------- |
| `RESNET18` :material-check-circle: | 13.3M | 50.7 MB | 19 |
| `RESNET34` | 24.1M | 89.2 MB | 19 |
!!! info "Training Data"
**Dataset**: Trained on CelebAMask-HQ
**Dataset**: Trained on [CelebAMask-HQ](datasets.md#celebamask-hq)
**Architecture**: BiSeNet with ResNet backbone
@@ -243,13 +451,13 @@ BiSeNet (Bilateral Segmentation Network) models for semantic face parsing. Segme
| # | Class | # | Class | # | Class |
|---|-------|---|-------|---|-------|
| 1 | Background | 8 | Left Ear | 15 | Neck |
| 2 | Skin | 9 | Right Ear | 16 | Neck Lace |
| 3 | Left Eyebrow | 10 | Ear Ring | 17 | Cloth |
| 4 | Right Eyebrow | 11 | Nose | 18 | Hair |
| 5 | Left Eye | 12 | Mouth | 19 | Hat |
| 6 | Right Eye | 13 | Upper Lip | | |
| 7 | Eye Glasses | 14 | Lower Lip | | |
| 0 | Background | 7 | Left Ear | 14 | Neck |
| 1 | Skin | 8 | Right Ear | 15 | Neck Lace |
| 2 | Left Eyebrow | 9 | Ear Ring | 16 | Cloth |
| 3 | Right Eyebrow | 10 | Nose | 17 | Hair |
| 4 | Left Eye | 11 | Mouth | 18 | Hat |
| 5 | Right Eye | 12 | Upper Lip | | |
| 6 | Eye Glasses | 13 | Lower Lip | | |
**Applications:**
@@ -264,22 +472,99 @@ BiSeNet (Bilateral Segmentation Network) models for semantic face parsing. Segme
---
### XSeg
XSeg from DeepFaceLab outputs masks for face regions. Requires 5-point landmarks for face alignment.
| Model Name | Size | Output |
|------------|--------|--------|
| `DEFAULT` | 67 MB | Mask [0, 1] |
!!! info "Model Details"
**Origin**: DeepFaceLab
**Input**: NHWC format, normalized to [0, 1]
**Alignment**: Requires 5-point landmarks (not bbox crops)
**Applications:**
- Face region extraction
- Face swapping pipelines
- Occlusion handling
!!! note "Input Requirements"
Requires 5-point facial landmarks. Use a face detector like RetinaFace to obtain landmarks first.
---
## Portrait Matting Models
### MODNet
MODNet (Real-Time Trimap-Free Portrait Matting) produces soft alpha mattes from full images without requiring a trimap. Uses MobileNetV2 backbone with low-resolution, high-resolution, and fusion branches.
| Model Name | Variant | Size | Use Case |
| ---------- | ------- | ---- | -------- |
| `PHOTOGRAPHIC` :material-check-circle: | High-quality | 25 MB | Portrait photos |
| `WEBCAM` | Real-time | 25 MB | Webcam feeds |
!!! info "Model Details"
**Paper**: [MODNet: Real-Time Trimap-Free Portrait Matting via Objective Decomposition](https://arxiv.org/abs/2011.11961) (AAAI 2022)
**Source**: [yakhyo/modnet](https://github.com/yakhyo/modnet) — ported weights and clean inference codebase
**Output**: Alpha matte `(H, W)` in `[0, 1]`
**Applications:**
- Background removal / replacement
- Green screen compositing
- Video conferencing virtual backgrounds
- Portrait editing
!!! note "Input Requirements"
Operates on full images (not face crops). No trimap or face detection required.
---
## Anti-Spoofing Models
### MiniFASNet Family
Lightweight face anti-spoofing models for liveness detection. Detect if a face is real (live) or fake (photo, video replay, mask).
Face anti-spoofing models for liveness detection. Detect if a face is real (live) or fake (photo, video replay, mask).
| Model Name | Size | Scale | Use Case |
| ---------- | ------ | ----- | ----------------------------- |
| `V1SE` | 1.2 MB | 4.0 | Squeeze-and-excitation variant |
| `V2` :material-check-circle: | 1.2 MB | 2.7 | **Default** |
| Model Name | Size | Scale |
| ---------- | ------ | ----- |
| `V1SE` | 1.2 MB | 4.0 |
| `V2` :material-check-circle: | 1.2 MB | 2.7 |
!!! info "Output Format"
**Output**: Returns `SpoofingResult(is_real, confidence)` where is_real: True=Real, False=Fake
!!! note "Input Requirements"
Requires face bounding box from a detector. Use with RetinaFace, SCRFD, or YOLOv5Face.
Requires face bounding box from a detector.
---
## Face Image Quality Models
### eDifFIQA Family
Face image quality assessment. Predicts a single scalar score from an aligned 112x112 face crop — higher = better. eDifFIQA-L ranks first on the NIST FATE-Quality Kiosk-to-Entry track.
| Model Name | Backbone | Params | Size |
| ---------- | -------- | ------ | ---- |
| `T` :material-check-circle: | MobileFaceNet | 1.7M | 6.6 MB |
| `S` | IResNet-18 | 24.6M | 93.7 MB |
| `M` | IResNet-50 | 44.1M | 168.3 MB |
| `L` | IResNet-100 | 65.7M | 250.6 MB |
!!! info "Output Format"
**Output**: Returns `QualityResult(score)` — float, roughly in `[0, 1]`. Not a calibrated probability; calibrate your threshold on a held-out set.
!!! note "Input Requirements"
Requires 5-point landmarks from a detector (e.g. SCRFD, RetinaFace) for alignment, or a pre-aligned 112x112 crop via `score_aligned()`.
---
@@ -287,10 +572,15 @@ Lightweight face anti-spoofing models for liveness detection. Detect if a face i
Models are automatically downloaded and cached on first use.
- **Cache location**: `~/.uniface/models/`
- **Verification**: Models are verified with SHA-256 checksums
- **Sources**: GitHub Releases, falling back automatically to a [Hugging Face mirror](https://huggingface.co/yakhyo/uniface-weights) when GitHub is unreachable
- **Cache location**: `~/.uniface/models/` (configurable via `set_cache_dir()` or `UNIFACE_CACHE_DIR` env var)
- **Inspect cache path**: `get_cache_dir()` returns the resolved active path
- **Verification**: Models are verified with SHA-256 checksums, whichever source served them
- **Concurrent download**: `download_models([...])` fetches multiple models in parallel
- **Manual download**: Use `python tools/download_model.py` to pre-download models
See [Model Cache & Offline Use](concepts/model-cache-offline.md) for full details.
---
## References
@@ -300,11 +590,21 @@ Models are automatically downloaded and cached on first use.
- **RetinaFace Training**: [yakhyo/retinaface-pytorch](https://github.com/yakhyo/retinaface-pytorch) - PyTorch implementation and training code
- **YOLOv5-Face Original**: [deepcam-cn/yolov5-face](https://github.com/deepcam-cn/yolov5-face) - Original PyTorch implementation
- **YOLOv5-Face ONNX**: [yakhyo/yolov5-face-onnx-inference](https://github.com/yakhyo/yolov5-face-onnx-inference) - ONNX inference implementation
- **YOLOv8-Face Original**: [derronqi/yolov8-face](https://github.com/derronqi/yolov8-face) - Original PyTorch implementation
- **YOLOv8-Face ONNX**: [yakhyo/yolov8-face-onnx-inference](https://github.com/yakhyo/yolov8-face-onnx-inference) - ONNX inference implementation
- **AdaFace Original**: [mk-minchul/AdaFace](https://github.com/mk-minchul/AdaFace) - Original PyTorch implementation
- **AdaFace ONNX**: [yakhyo/adaface-onnx](https://github.com/yakhyo/adaface-onnx) - ONNX export and inference
- **Face Recognition Training**: [yakhyo/face-recognition](https://github.com/yakhyo/face-recognition) - ArcFace, MobileFace, SphereFace training code
- **Gaze Estimation Training**: [yakhyo/gaze-estimation](https://github.com/yakhyo/gaze-estimation) - MobileGaze training code and pretrained weights
- **Head Pose Estimation**: [yakhyo/head-pose-estimation](https://github.com/yakhyo/head-pose-estimation) - 6D rotation head pose estimation training and ONNX models
- **Face Parsing Training**: [yakhyo/face-parsing](https://github.com/yakhyo/face-parsing) - BiSeNet training code and pretrained weights
- **Face Segmentation**: [yakhyo/face-segmentation](https://github.com/yakhyo/face-segmentation) - XSeg ONNX Inference
- **Portrait Matting**: [yakhyo/modnet](https://github.com/yakhyo/modnet) - MODNet ported weights and inference (from [ZHKKKe/MODNet](https://github.com/ZHKKKe/MODNet))
- **Face Anti-Spoofing**: [yakhyo/face-anti-spoofing](https://github.com/yakhyo/face-anti-spoofing) - MiniFASNet ONNX inference (weights from [minivision-ai/Silent-Face-Anti-Spoofing](https://github.com/minivision-ai/Silent-Face-Anti-Spoofing))
- **Face Image Quality Assessment**: [yakhyo/face-image-quality-assessment](https://github.com/yakhyo/face-image-quality-assessment) - eDifFIQA PyTorch inference, ONNX export and inference
- **FairFace**: [yakhyo/fairface-onnx](https://github.com/yakhyo/fairface-onnx) - FairFace ONNX inference for race, gender, age prediction
- **FaceAttribNet**: [yakhyo/face-attribute](https://github.com/yakhyo/face-attribute) - FaceAttribNet ONNX export and inference (weights from [qualcomm/ai-hub-models](https://github.com/qualcomm/ai-hub-models/tree/main/src/qai_hub_models/models/face_attrib_net))
- **PIPNet**: [yakhyo/pipnet-onnx](https://github.com/yakhyo/pipnet-onnx) - PIPNet ONNX export and inference (from [jhb86253817/PIPNet](https://github.com/jhb86253817/PIPNet))
- **InsightFace**: [deepinsight/insightface](https://github.com/deepinsight/insightface) - Model architectures and pretrained weights
### Papers
@@ -312,6 +612,9 @@ Models are automatically downloaded and cached on first use.
- **RetinaFace**: [Single-Shot Multi-Level Face Localisation in the Wild](https://arxiv.org/abs/1905.00641)
- **SCRFD**: [Sample and Computation Redistribution for Efficient Face Detection](https://arxiv.org/abs/2105.04714)
- **YOLOv5-Face**: [YOLO5Face: Why Reinventing a Face Detector](https://arxiv.org/abs/2105.12931)
- **AdaFace**: [AdaFace: Quality Adaptive Margin for Face Recognition](https://arxiv.org/abs/2204.00964)
- **ArcFace**: [Additive Angular Margin Loss for Deep Face Recognition](https://arxiv.org/abs/1801.07698)
- **SphereFace**: [Deep Hypersphere Embedding for Face Recognition](https://arxiv.org/abs/1704.08063)
- **MODNet**: [Real-Time Trimap-Free Portrait Matting via Objective Decomposition](https://arxiv.org/abs/2011.11961)
- **BiSeNet**: [Bilateral Segmentation Network for Real-time Semantic Segmentation](https://arxiv.org/abs/1808.00897)
- **PIPNet**: [Towards Efficient Facial Landmark Detection in the Wild](https://arxiv.org/abs/2003.03771)

View File

@@ -1,6 +1,11 @@
# Attributes
Facial attribute analysis for age, gender, race, and emotion detection.
Facial attribute analysis for age, gender, race, emotion, and face state detection (eye openness, glasses, mask).
<figure markdown="span">
![Age & Gender Prediction](https://raw.githubusercontent.com/yakhyo/uniface/main/assets/demos/age_gender.jpg){ width="100%" }
<figcaption>Age and gender prediction with detection bounding boxes</figcaption>
</figure>
---
@@ -11,6 +16,7 @@ Facial attribute analysis for age, gender, race, and emotion detection.
| **AgeGender** | Age, Gender | 8 MB | Exact age prediction |
| **FairFace** | Gender, Age Group, Race | 44 MB | Balanced demographics |
| **Emotion** | 7-8 emotions | 2 MB | Requires PyTorch |
| **FaceAttribNet** | Eye openness, Eyeglasses, Mask, Sunglasses | 41 MB | Multi-label (independent probabilities) |
---
@@ -21,7 +27,8 @@ Predicts exact age and binary gender.
### Basic Usage
```python
from uniface import RetinaFace, AgeGender
from uniface.attribute import AgeGender
from uniface.detection import RetinaFace
detector = RetinaFace()
age_gender = AgeGender()
@@ -29,15 +36,16 @@ age_gender = AgeGender()
faces = detector.detect(image)
for face in faces:
result = age_gender.predict(image, face.bbox)
result = age_gender.predict(image, face)
print(f"Gender: {result.sex}") # "Female" or "Male"
print(f"Age: {result.age} years")
# face.gender and face.age are also set automatically
```
### Output
```python
# AttributeResult fields
# DemographyResult fields
result.gender # 0=Female, 1=Male
result.sex # "Female" or "Male" (property)
result.age # int, age in years
@@ -54,7 +62,8 @@ Predicts gender, age group, and race with balanced demographics.
### Basic Usage
```python
from uniface import RetinaFace, FairFace
from uniface.attribute import FairFace
from uniface.detection import RetinaFace
detector = RetinaFace()
fairface = FairFace()
@@ -62,16 +71,17 @@ fairface = FairFace()
faces = detector.detect(image)
for face in faces:
result = fairface.predict(image, face.bbox)
result = fairface.predict(image, face)
print(f"Gender: {result.sex}")
print(f"Age Group: {result.age_group}")
print(f"Race: {result.race}")
# face.gender, face.age_group, face.race are also set automatically
```
### Output
```python
# AttributeResult fields
# DemographyResult fields
result.gender # 0=Female, 1=Male
result.sex # "Female" or "Male"
result.age # None (not provided by this model)
@@ -120,17 +130,17 @@ Predicts facial emotions. Requires PyTorch.
### Basic Usage
```python
from uniface import RetinaFace
from uniface.detection import RetinaFace
from uniface.attribute import Emotion
from uniface.constants import DDAMFNWeights
from uniface.constants import EmotionWeights
detector = RetinaFace()
emotion = Emotion(model_name=DDAMFNWeights.AFFECNET7)
emotion = Emotion(model_name=EmotionWeights.AFFECNET7)
faces = detector.detect(image)
for face in faces:
result = emotion.predict(image, face.landmarks)
result = emotion.predict(image, face)
print(f"Emotion: {result.emotion}")
print(f"Confidence: {result.confidence:.2%}")
```
@@ -147,7 +157,7 @@ for face in faces:
| Surprise |
| Fear |
| Disgust |
| Anger |
| Angry |
=== "8-Class (AFFECNET8)"
@@ -159,20 +169,82 @@ for face in faces:
| Surprise |
| Fear |
| Disgust |
| Anger |
| Angry |
| Contempt |
### Model Variants
```python
from uniface.attribute import Emotion
from uniface.constants import DDAMFNWeights
from uniface.constants import EmotionWeights
# 7-class emotion
emotion = Emotion(model_name=DDAMFNWeights.AFFECNET7)
emotion = Emotion(model_name=EmotionWeights.AFFECNET7)
# 8-class emotion
emotion = Emotion(model_name=DDAMFNWeights.AFFECNET8)
emotion = Emotion(model_name=EmotionWeights.AFFECNET8)
```
---
## FaceAttribNet
Predicts five independent binary face states from a face crop: left/right eye openness, eyeglasses, face mask, and sunglasses. Based on Qualcomm's [Facial-Attribute-Detection](https://github.com/qualcomm/ai-hub-models/tree/main/src/qai_hub_models/models/face_attrib_net) model.
<figure markdown="span">
![Face Attribute Detection](https://raw.githubusercontent.com/yakhyo/uniface/main/assets/demos/face_attributes.png){ width="100%" }
<figcaption>Face state prediction: per-attribute True/False with probabilities</figcaption>
</figure>
!!! warning "Multi-label output"
The five values come from independent binary heads: they do not sum to 1 and
several can be high at once (a face can wear both sunglasses and a mask).
Threshold each attribute separately; never `argmax`.
### Basic Usage
```python
from uniface.attribute import FaceAttribNet
from uniface.detection import RetinaFace
detector = RetinaFace()
face_attrib = FaceAttribNet()
faces = detector.detect(image)
for face in faces:
result = face_attrib.predict(image, face)
print(result.as_dict()) # {'left_eye_open': 0.99, 'right_eye_open': 0.98, ...}
print(result.labels(threshold=0.5)) # e.g. ['left_eye_open', 'right_eye_open', 'eyeglasses']
# face.left_eye_open, face.right_eye_open, face.eyeglasses,
# face.mask, face.sunglasses are also set automatically
```
### Output
```python
# FaceStateResult fields (all probabilities in [0, 1])
result.left_eye_open # Probability the left eye is open
result.right_eye_open # Probability the right eye is open
result.eyeglasses # Probability eyeglasses are present
result.mask # Probability a face mask is present
result.sunglasses # Probability sunglasses are present
result.as_dict() # name -> probability mapping
result.labels(threshold) # names of attributes above the threshold
```
---
## Available Attribute Models
```python
from uniface.attribute import AgeGender, Emotion, FaceAttribNet, FairFace
age_gender = AgeGender()
fairface = FairFace()
emotion = Emotion() # requires the optional `torch` dependency
face_attrib = FaceAttribNet()
```
---
@@ -182,7 +254,8 @@ emotion = Emotion(model_name=DDAMFNWeights.AFFECNET8)
### Full Attribute Analysis
```python
from uniface import RetinaFace, AgeGender, FairFace
from uniface.attribute import AgeGender, FairFace
from uniface.detection import RetinaFace
detector = RetinaFace()
age_gender = AgeGender()
@@ -192,10 +265,10 @@ faces = detector.detect(image)
for face in faces:
# Get exact age from AgeGender
ag_result = age_gender.predict(image, face.bbox)
ag_result = age_gender.predict(image, face)
# Get race from FairFace
ff_result = fairface.predict(image, face.bbox)
ff_result = fairface.predict(image, face)
print(f"Gender: {ag_result.sex}")
print(f"Exact Age: {ag_result.age}")
@@ -206,12 +279,13 @@ for face in faces:
### Using FaceAnalyzer
```python
from uniface import FaceAnalyzer
from uniface.analyzer import FaceAnalyzer
from uniface.attribute import AgeGender
from uniface.detection import RetinaFace
analyzer = FaceAnalyzer(
detect=True,
recognize=False,
attributes=True # Uses AgeGender
detector=RetinaFace(),
predictors=[AgeGender()],
)
faces = analyzer.analyze(image)
@@ -253,7 +327,7 @@ def draw_attributes(image, face, result):
# Usage
for face in faces:
result = age_gender.predict(image, face.bbox)
result = age_gender.predict(image, face)
image = draw_attributes(image, face, result)
cv2.imwrite("attributes.jpg", image)
@@ -267,6 +341,7 @@ cv2.imwrite("attributes.jpg", image)
- **AgeGender**: Trained on CelebA; accuracy varies by demographic
- **FairFace**: Trained for balanced demographics; better cross-racial accuracy
- **Emotion**: Accuracy depends on facial expression clarity
- **FaceAttribNet**: Trained by Qualcomm on a proprietary dataset; tinted eyeglasses may register as sunglasses
Always test on your specific use case and consider cultural context.
@@ -277,3 +352,4 @@ cv2.imwrite("attributes.jpg", image)
- [Parsing](parsing.md) - Face semantic segmentation
- [Gaze](gaze.md) - Gaze estimation
- [Image Pipeline Recipe](../recipes/image-pipeline.md) - Complete workflow
- [CLI Tools](https://github.com/yakhyo/uniface/blob/main/tools/README.md) - Command-line scripts for all UniFace modules

View File

@@ -1,27 +1,60 @@
# Detection
Face detection is the first step in any face analysis pipeline. UniFace provides three detection models.
Face detection is the first step in any face analysis pipeline. UniFace provides six detection models.
<figure markdown="span">
![Face Detection](https://raw.githubusercontent.com/yakhyo/uniface/main/assets/demos/detection.jpg){ width="100%" }
<figcaption>SCRFD detection with corner-style bounding boxes and 5-point landmarks</figcaption>
</figure>
---
## Available Models
| Model | Backbone | Size | WIDER FACE (Easy/Medium/Hard) | Best For |
|-------|----------|------|-------------------------------|----------|
| **RetinaFace** | MobileNet V2 | 3.5 MB | 91.7% / 91.0% / 86.6% | Balanced (recommended) |
| **SCRFD** | SCRFD-10G | 17 MB | 95.2% / 93.9% / 83.1% | High accuracy |
| **YOLOv5-Face** | YOLOv5s | 28 MB | 94.3% / 92.6% / 83.2% | Real-time |
| Model | Backbone | Size | Easy | Medium | Hard | Landmarks |
|-------|----------|------|------|--------|------|:---------:|
| **RetinaFace** | MobileNet V2 | 3.5 MB | 91.7% | 91.0% | 86.6% | :material-check: |
| **SCRFD** | SCRFD-10G | 17 MB | 95.2% | 93.9% | 83.1% | :material-check: |
| **CenterFace** | MobileNet V2 | 7.0 MB | 92.2% | 91.1% | 78.2% | :material-check: |
| **YOLOv5-Face** | YOLOv5s | 28 MB | 94.3% | 92.6% | 83.2% | :material-check: |
| **YOLOv8-Face** | YOLOv8n | 12 MB | 94.6% | 92.3% | 79.6% | :material-check: |
| **BlazeFace** | BlazeFace (short-range) | 0.5 MB | — | — | — | 6-point |
!!! note "Dataset"
All models except BlazeFace are trained on the WIDERFACE dataset and benchmarked against it.
BlazeFace comes from Google MediaPipe and is not benchmarked on WIDERFACE.
!!! warning "BlazeFace landmarks are not alignment landmarks"
Every other detector returns the 5-point alignment template — left eye, right eye,
nose, left mouth corner, right mouth corner — which
[recognition](recognition.md), [quality](quality.md), and XSeg
[parsing](parsing.md) all consume.
BlazeFace returns **6** MediaPipe keypoints whose fourth point is a mouth
*center*, not corners, so they cannot be fitted to that template. It leaves
`supports_alignment = False`, and `FaceAnalyzer` disables recognition
with a warning rather than producing broken embeddings.
```python
detector.supports_alignment # False for BlazeFace, True for every other detector
detector.supports_landmarks # True for every built-in detector
```
`supports_landmarks` answers the earlier question: does the detector fill
`Face.landmarks` at all? Every built-in detector does. Both flags are opt-in on
`BaseDetector`, so a custom boxes-only detector reports `False` for both without
declaring anything — see [Custom Models](../recipes/custom-models.md).
---
## RetinaFace
The recommended detector for most use cases.
Single-shot face detector with multi-scale feature pyramid.
### Basic Usage
```python
from uniface import RetinaFace
from uniface.detection import RetinaFace
detector = RetinaFace()
faces = detector.detect(image)
@@ -35,7 +68,7 @@ for face in faces:
### Model Variants
```python
from uniface import RetinaFace
from uniface.detection import RetinaFace
from uniface.constants import RetinaFaceWeights
# Lightweight (mobile/edge)
@@ -56,6 +89,10 @@ detector = RetinaFace(model_name=RetinaFaceWeights.RESNET34)
| **MNET_V2** :material-check-circle: | 3.2M | 3.5 MB | 91.7% | 91.0% | 86.6% |
| RESNET18 | 11.7M | 27 MB | 92.5% | 91.0% | 86.6% |
| RESNET34 | 24.8M | 56 MB | 94.2% | 93.1% | 88.9% |
| RESNET50 | 27.4M | 104 MB | 94.7%* | 93.7%* | 88.8%* |
*\* `RESNET50` weights come from [HivisionIDPhotos](https://huggingface.co/TheEeeeLin/HivisionIDPhotos_matting/blob/main/retinaface-resnet50.onnx);
its scores are measured, not quoted from the paper. See [Models](../models.md#retinaface-family).*
### Configuration
@@ -65,7 +102,8 @@ detector = RetinaFace(
confidence_threshold=0.5, # Min confidence
nms_threshold=0.4, # NMS IoU threshold
input_size=(640, 640), # Input resolution
dynamic_size=False # Enable dynamic input size
dynamic_size=False, # Enable dynamic input size
providers=None, # Auto-detect, or ['CPUExecutionProvider']
)
```
@@ -78,7 +116,7 @@ State-of-the-art detection with excellent accuracy-speed tradeoff.
### Basic Usage
```python
from uniface import SCRFD
from uniface.detection import SCRFD
detector = SCRFD()
faces = detector.detect(image)
@@ -87,7 +125,7 @@ faces = detector.detect(image)
### Model Variants
```python
from uniface import SCRFD
from uniface.detection import SCRFD
from uniface.constants import SCRFDWeights
# Real-time (lightweight)
@@ -109,7 +147,57 @@ detector = SCRFD(
model_name=SCRFDWeights.SCRFD_10G_KPS,
confidence_threshold=0.5,
nms_threshold=0.4,
input_size=(640, 640)
input_size=(640, 640),
providers=None, # Auto-detect, or ['CPUExecutionProvider']
)
```
---
## CenterFace
Anchor-free detection that treats faces as center points (MobileNetV2 + FPN), with joint 5-point landmark prediction. Lightweight and fast on CPU.
Paper: [CenterFace: Joint Face Detection and Alignment Using Face as Point](https://arxiv.org/abs/1911.03599)
### Basic Usage
```python
from uniface.detection import CenterFace
detector = CenterFace()
faces = detector.detect(image)
```
| Variant | Size | Easy | Medium | Hard |
|---------|------|------|--------|------|
| **DEFAULT** :material-check-circle: | 7.0 MB | 92.2% | 91.1% | 78.2% |
!!! note "Benchmark schema"
Scores are WIDER FACE val with single inference on the original image (SIO).
With multi-scale and flip testing the [original repo](https://github.com/Star-Clouds/CenterFace)
reports 93.5% / 92.4% / 87.5%.
!!! warning "Limitations"
- **Landmark precision**: landmarks are decoded from a single coarse feature-map cell
per face, so they are less precise than SCRFD or RetinaFace (roughly 5% of box size
deviation on upright faces). For alignment-critical recognition, prefer SCRFD/RetinaFace,
or refine with [PIPNet / Landmark106](landmarks.md) on CenterFace boxes.
- **Rotated faces**: detection recall and landmark accuracy drop faster than SCRFD as
in-plane rotation increases (noticeable beyond ~20-30 degrees). Best suited for
roughly upright faces (webcams, portraits, surveillance).
### Configuration
```python
from uniface.constants import CenterFaceWeights
detector = CenterFace(
model_name=CenterFaceWeights.DEFAULT,
confidence_threshold=0.35,
nms_threshold=0.3,
input_size=(640, 640), # width and height must be multiples of 32
providers=None, # Auto-detect, or ['CPUExecutionProvider']
)
```
@@ -122,7 +210,7 @@ YOLO-based detection optimized for faces.
### Basic Usage
```python
from uniface import YOLOv5Face
from uniface.detection import YOLOv5Face
detector = YOLOv5Face()
faces = detector.detect(image)
@@ -131,7 +219,7 @@ faces = detector.detect(image)
### Model Variants
```python
from uniface import YOLOv5Face
from uniface.detection import YOLOv5Face
from uniface.constants import YOLOv5FaceWeights
# Lightweight
@@ -159,44 +247,127 @@ detector = YOLOv5Face(model_name=YOLOv5FaceWeights.YOLOV5M)
detector = YOLOv5Face(
model_name=YOLOv5FaceWeights.YOLOV5S,
confidence_threshold=0.6,
nms_threshold=0.5
nms_threshold=0.5,
nms_mode='numpy', # or 'torchvision' for faster NMS
providers=None, # Auto-detect, or ['CPUExecutionProvider']
)
```
---
## Factory Function
## YOLOv8-Face
Create detectors dynamically:
Anchor-free detection with DFL (Distribution Focal Loss) for accurate bbox regression.
### Basic Usage
```python
from uniface import create_detector
from uniface.detection import YOLOv8Face
detector = create_detector('retinaface')
# or
detector = create_detector('scrfd')
# or
detector = create_detector('yolov5face')
detector = YOLOv8Face()
faces = detector.detect(image)
```
### Model Variants
```python
from uniface.detection import YOLOv8Face
from uniface.constants import YOLOv8FaceWeights
# Lightweight
detector = YOLOv8Face(model_name=YOLOv8FaceWeights.YOLOV8_LITE_S)
# Recommended (default)
detector = YOLOv8Face(model_name=YOLOv8FaceWeights.YOLOV8N)
```
| Variant | Size | Easy | Medium | Hard |
|---------|------|------|--------|------|
| YOLOV8_LITE_S | 7.4 MB | 93.4% | 91.2% | 78.6% |
| **YOLOV8N** :material-check-circle: | 12 MB | 94.6% | 92.3% | 79.6% |
!!! note "Fixed Input Size"
YOLOv8-Face uses a fixed input size of 640×640.
### Configuration
```python
detector = YOLOv8Face(
model_name=YOLOv8FaceWeights.YOLOV8N,
confidence_threshold=0.5,
nms_threshold=0.45,
nms_mode='numpy', # or 'torchvision' for faster NMS
providers=None, # Auto-detect, or ['CPUExecutionProvider']
)
```
---
## High-Level API
## BlazeFace
One-line detection:
Google MediaPipe's short-range SSD detector — the one `mp.solutions.face_mesh` runs
internally. Pair it with [FaceMesh](landmarks.md#face-mesh-468-or-478-points-3d) to reproduce MediaPipe's
own output exactly.
At 0.5 MB it is by far the smallest detector here, but it is tuned for faces within
roughly 2 m and is less accurate than SCRFD or YOLOv8 on small or distant faces.
Choose it for its footprint or for MediaPipe parity — not as a general-purpose detector.
### Basic Usage
```python
from uniface import detect_faces
from uniface.detection import BlazeFace
faces = detect_faces(
image,
method='retinaface',
confidence_threshold=0.5
detector = BlazeFace()
faces = detector.detect(image)
for face in faces:
print(f"Confidence: {face.confidence:.2f}")
print(f"Keypoints: {face.landmarks.shape}") # (6, 2), not (5, 2)
```
### Keypoint Layout
```python
# [right_eye, left_eye, nose_tip, mouth_center, right_ear, left_ear]
# Named from the subject's perspective, so rows 0/1 are the viewer-left
# and viewer-right eye — the same geometric order the 5-point template uses.
```
### Configuration
```python
detector = BlazeFace(
confidence_threshold=0.5,
nms_threshold=0.3, # MediaPipe blends overlapping boxes rather than dropping them
providers=None, # Auto-detect, or ['CPUExecutionProvider']
)
```
---
## Available Detectors
Import the detector class you need:
```python
from uniface.detection import BlazeFace, CenterFace, RetinaFace, SCRFD, YOLOv5Face, YOLOv8Face
detector = RetinaFace()
# or
detector = SCRFD()
# or
detector = CenterFace()
# or
detector = YOLOv5Face()
# or
detector = YOLOv8Face()
# or
detector = BlazeFace() # 6 keypoints; supports_alignment is False
```
---
## Output Format
All detectors return `list[Face]`:
@@ -209,9 +380,11 @@ for face in faces:
# Detection confidence (0-1)
confidence = face.confidence
# 5-point landmarks (5, 2)
# Landmarks (K, 2)
landmarks = face.landmarks
# [left_eye, right_eye, nose, left_mouth, right_mouth]
# K == 5 for every detector except BlazeFace:
# [left_eye, right_eye, nose, left_mouth, right_mouth]
# BlazeFace returns K == 6 in MediaPipe's own order; see the warning above.
```
---
@@ -219,14 +392,12 @@ for face in faces:
## Visualization
```python
from uniface.visualization import draw_detections
from uniface.draw import draw_detections
draw_detections(
image=image,
bboxes=[f.bbox for f in faces],
scores=[f.confidence for f in faces],
landmarks=[f.landmarks for f in faces],
vis_threshold=0.6
faces=faces,
vis_threshold=0.6,
)
cv2.imwrite("result.jpg", image)
@@ -239,7 +410,7 @@ cv2.imwrite("result.jpg", image)
Benchmark on your hardware:
```bash
python tools/detection.py --source image.jpg --iterations 100
python tools/detect.py --source image.jpg
```
---
@@ -247,6 +418,7 @@ python tools/detection.py --source image.jpg --iterations 100
## See Also
- [Recognition Module](recognition.md) - Extract embeddings from detected faces
- [Landmarks Module](landmarks.md) - Get 106-point landmarks
- [Landmarks Module](landmarks.md) - Get 106 / 98 / 68-point dense landmarks
- [Image Pipeline Recipe](../recipes/image-pipeline.md) - Complete detection workflow
- [Concepts: Thresholds](../concepts/thresholds-calibration.md) - Tuning detection parameters
- [CLI Tools](https://github.com/yakhyo/uniface/blob/main/tools/README.md) - Command-line scripts for all UniFace modules

View File

@@ -2,17 +2,22 @@
Gaze estimation predicts where a person is looking (pitch and yaw angles).
<figure markdown="span">
![Gaze Estimation](https://raw.githubusercontent.com/yakhyo/uniface/main/assets/demos/gaze.jpg){ width="100%" }
<figcaption>Gaze direction arrows with pitch/yaw angle labels</figcaption>
</figure>
---
## Available Models
| Model | Backbone | Size | MAE* | Best For |
|-------|----------|------|------|----------|
| ResNet18 | ResNet18 | 43 MB | 12.84° | Balanced |
| **ResNet34** :material-check-circle: | ResNet34 | 82 MB | 11.33° | Recommended |
| ResNet50 | ResNet50 | 91 MB | 11.34° | High accuracy |
| MobileNetV2 | MobileNetV2 | 9.6 MB | 13.07° | Mobile |
| MobileOne-S0 | MobileOne | 4.8 MB | 12.58° | Lightweight |
| Model | Backbone | Size | MAE* |
|-------|----------|------|------|
| ResNet18 | ResNet18 | 43 MB | 12.84° |
| **ResNet34** :material-check-circle: | ResNet34 | 82 MB | 11.33° |
| ResNet50 | ResNet50 | 91 MB | 11.34° |
| MobileNetV2 | MobileNetV2 | 9.6 MB | 13.07° |
| MobileOne-S0 | MobileOne | 4.8 MB | 12.58° |
*MAE = Mean Absolute Error on Gaze360 test set (lower is better)
@@ -23,7 +28,8 @@ Gaze estimation predicts where a person is looking (pitch and yaw angles).
```python
import cv2
import numpy as np
from uniface import RetinaFace, MobileGaze
from uniface.detection import RetinaFace
from uniface.gaze import MobileGaze
detector = RetinaFace()
gaze_estimator = MobileGaze()
@@ -52,7 +58,7 @@ for face in faces:
## Model Variants
```python
from uniface import MobileGaze
from uniface.gaze import MobileGaze
from uniface.constants import GazeWeights
# Default (ResNet34, recommended)
@@ -102,7 +108,7 @@ yaw = -90° ────┼──── yaw = +90°
## Visualization
```python
from uniface.visualization import draw_gaze
from uniface.draw import draw_gaze
# Detect faces
faces = detector.detect(image)
@@ -154,8 +160,9 @@ def draw_gaze_custom(image, bbox, pitch, yaw, length=100, color=(0, 255, 0)):
```python
import cv2
import numpy as np
from uniface import RetinaFace, MobileGaze
from uniface.visualization import draw_gaze
from uniface.detection import RetinaFace
from uniface.gaze import MobileGaze
from uniface.draw import draw_gaze
detector = RetinaFace()
gaze_estimator = MobileGaze()
@@ -253,18 +260,20 @@ print(f"Looking: {direction}")
---
## Factory Function
## Available Estimators
```python
from uniface import create_gaze_estimator
from uniface.gaze import MobileGaze
gaze = create_gaze_estimator() # Returns MobileGaze
gaze = MobileGaze()
```
---
## Next Steps
- [Head Pose Estimation](headpose.md) - 3D head orientation
- [Anti-Spoofing](spoofing.md) - Face liveness detection
- [Privacy](privacy.md) - Face anonymization
- [Video Recipe](../recipes/video-webcam.md) - Real-time processing
- [CLI Tools](https://github.com/yakhyo/uniface/blob/main/tools/README.md) - Command-line scripts for all UniFace modules

238
docs/modules/headpose.md Normal file
View File

@@ -0,0 +1,238 @@
# Head Pose Estimation
Head pose estimation predicts the 3D orientation of a person's head (pitch, yaw, and roll angles).
<figure markdown="span">
![Head Pose Estimation](https://raw.githubusercontent.com/yakhyo/uniface/main/assets/demos/headpose.jpg){ width="100%" }
<figcaption>3D head pose visualization with pitch, yaw, and roll angles</figcaption>
</figure>
---
## Available Models
| Model | Backbone | Size | MAE* |
|-------|----------|------|------|
| **ResNet18** :material-check-circle: | ResNet18 | 43 MB | 5.22° |
| ResNet34 | ResNet34 | 82 MB | 5.07° |
| ResNet50 | ResNet50 | 91 MB | 4.83° |
| MobileNetV2 | MobileNetV2 | 9.6 MB | 5.72° |
| MobileNetV3-Small | MobileNetV3 | 4.8 MB | 6.31° |
| MobileNetV3-Large | MobileNetV3 | 16 MB | 5.58° |
*MAE = Mean Absolute Error on AFLW2000 test set (lower is better)
---
## Basic Usage
```python
import cv2
from uniface.detection import RetinaFace
from uniface.headpose import HeadPose
detector = RetinaFace()
head_pose = HeadPose()
image = cv2.imread("photo.jpg")
faces = detector.detect(image)
for face in faces:
# Crop face
x1, y1, x2, y2 = map(int, face.bbox)
face_crop = image[y1:y2, x1:x2]
if face_crop.size > 0:
# Estimate head pose
result = head_pose.estimate(face_crop)
print(f"Pitch: {result.pitch:.1f}°, Yaw: {result.yaw:.1f}°, Roll: {result.roll:.1f}°")
```
---
## Model Variants
```python
from uniface.headpose import HeadPose
from uniface.constants import HeadPoseWeights
# Default (ResNet18, recommended balance of speed and accuracy)
hp = HeadPose()
# Lightweight for mobile/edge
hp = HeadPose(model_name=HeadPoseWeights.MOBILENET_V3_SMALL)
# Higher accuracy
hp = HeadPose(model_name=HeadPoseWeights.RESNET50)
```
---
## Output Format
```python
result = head_pose.estimate(face_crop)
# HeadPoseResult dataclass
result.pitch # Rotation around X-axis in degrees
result.yaw # Rotation around Y-axis in degrees
result.roll # Rotation around Z-axis in degrees
```
### Angle Convention
```
pitch > 0 (looking down)
yaw < 0 ─────┼───── yaw > 0
(looking left) │ (looking right)
pitch < 0 (looking up)
roll > 0 = clockwise tilt
roll < 0 = counter-clockwise tilt
```
- **Pitch**: Rotation around X-axis (positive = looking down)
- **Yaw**: Rotation around Y-axis (positive = looking right)
- **Roll**: Rotation around Z-axis (positive = tilting clockwise)
---
## Visualization
### 3D Cube (default)
The default visualization draws a wireframe cube oriented to match the head pose.
```python
from uniface.draw import draw_head_pose
faces = detector.detect(image)
for face in faces:
x1, y1, x2, y2 = map(int, face.bbox)
face_crop = image[y1:y2, x1:x2]
if face_crop.size > 0:
result = head_pose.estimate(face_crop)
# Draw cube on image (default)
draw_head_pose(image, face.bbox, result.pitch, result.yaw, result.roll)
cv2.imwrite("headpose_output.jpg", image)
```
### Axis Visualization
```python
from uniface.draw import draw_head_pose
# X/Y/Z coordinate axes
draw_head_pose(image, face.bbox, result.pitch, result.yaw, result.roll, draw_type='axis')
```
### Low-Level Drawing Functions
```python
from uniface.draw import draw_head_pose_cube, draw_head_pose_axis
# Draw cube directly
draw_head_pose_cube(image, [100, 100, 250, 280], pitch=-5.0, yaw=10.0, roll=2.0)
# Draw axes directly
draw_head_pose_axis(image, [100, 100, 250, 280], pitch=-5.0, yaw=10.0, roll=2.0)
```
---
## Real-Time Head Pose Tracking
```python
import cv2
from uniface.detection import RetinaFace
from uniface.headpose import HeadPose
from uniface.draw import draw_head_pose
detector = RetinaFace()
head_pose = HeadPose()
cap = cv2.VideoCapture(0)
while True:
ret, frame = cap.read()
if not ret:
break
faces = detector.detect(frame)
for face in faces:
x1, y1, x2, y2 = map(int, face.bbox)
face_crop = frame[y1:y2, x1:x2]
if face_crop.size > 0:
result = head_pose.estimate(face_crop)
draw_head_pose(frame, face.bbox, result.pitch, result.yaw, result.roll)
cv2.imshow("Head Pose Estimation", frame)
if cv2.waitKey(1) & 0xFF == ord('q'):
break
cap.release()
cv2.destroyAllWindows()
```
---
## Use Cases
### Driver Drowsiness Detection
```python
def is_head_drooping(result, pitch_threshold=-15):
"""Check if the head is drooping (looking down significantly)."""
return result.pitch < pitch_threshold
result = head_pose.estimate(face_crop)
if is_head_drooping(result):
print("Warning: Head drooping detected")
```
### Attention Monitoring
```python
def is_facing_forward(result, threshold=20):
"""Check if the person is facing roughly forward."""
return (
abs(result.pitch) < threshold
and abs(result.yaw) < threshold
and abs(result.roll) < threshold
)
result = head_pose.estimate(face_crop)
if is_facing_forward(result):
print("Facing forward")
else:
print("Looking away")
```
---
## Available Estimators
```python
from uniface.headpose import HeadPose
hp = HeadPose()
```
---
## Next Steps
- [Gaze Estimation](gaze.md) - Eye gaze direction
- [Anti-Spoofing](spoofing.md) - Face liveness detection
- [Video Recipe](../recipes/video-webcam.md) - Real-time processing
- [CLI Tools](https://github.com/yakhyo/uniface/blob/main/tools/README.md) - Command-line scripts for all UniFace modules

View File

@@ -2,16 +2,27 @@
Facial landmark detection provides precise localization of facial features.
<figure markdown="span">
![106-Point Landmarks](https://raw.githubusercontent.com/yakhyo/uniface/main/assets/demos/landmarks.jpg){ width="40%" }
![98-Point Landmarks (PIPNet)](https://raw.githubusercontent.com/yakhyo/uniface/main/assets/demos/landmarks_pipnet.jpg){ width="40%" }
<figcaption>106-point (left) and PIPNet 98-point (right) facial landmark localization</figcaption>
</figure>
---
## Available Models
| Model | Points | Size | Use Case |
|-------|--------|------|----------|
| **Landmark106** | 106 | 14 MB | Detailed face analysis |
| Model | Points | Size |
|-------|--------|------|
| **Landmark106** | 106 | 14 MB |
| **PIPNet (WFLW-98)** | 98 | 47 MB |
| **PIPNet (300W+CelebA-68)** | 68 | 46 MB |
| **FaceMesh** (`V1_468`) | 468 (3D) | 2.4 MB |
| **FaceMesh** (`V2_478`) | 478 (3D, with irises) | 4.6 MB |
!!! info "5-Point Landmarks"
Basic 5-point landmarks are included with all detection models (RetinaFace, SCRFD, YOLOv5-Face).
Basic 5-point landmarks are included with all detection models (RetinaFace, SCRFD, CenterFace, YOLOv5-Face, YOLOv8-Face).
BlazeFace is the exception — it returns 6 MediaPipe keypoints instead; see [Detection](detection.md#blazeface).
---
@@ -20,7 +31,8 @@ Facial landmark detection provides precise localization of facial features.
### Basic Usage
```python
from uniface import RetinaFace, Landmark106
from uniface.detection import RetinaFace
from uniface.landmark import Landmark106
detector = RetinaFace()
landmarker = Landmark106()
@@ -73,12 +85,168 @@ mouth = landmarks[87:106]
---
## 5-Point Landmarks (Detection)
## PIPNet (98 / 68 points)
All detection models provide 5-point landmarks:
PIPNet (Pixel-in-Pixel Net) is a high-accuracy facial landmark detector. UniFace ships
two ONNX variants that share a ResNet-18 backbone and 256×256 input — the only difference
is the number of points and the dataset they were trained on.
### Basic Usage
```python
from uniface import RetinaFace
from uniface.detection import RetinaFace
from uniface.landmark import PIPNet
detector = RetinaFace()
landmarker = PIPNet() # Default: 98 points (WFLW)
faces = detector.detect(image)
if faces:
landmarks = landmarker.get_landmarks(image, faces[0].bbox)
print(f"Landmarks shape: {landmarks.shape}") # (98, 2)
```
### 68-Point Variant (300W+CelebA, GSSL)
```python
from uniface.constants import PIPNetWeights
from uniface.landmark import PIPNet
landmarker = PIPNet(model_name=PIPNetWeights.DW300_CELEBA_68)
landmarks = landmarker.get_landmarks(image, face.bbox)
print(landmarks.shape) # (68, 2)
```
### Notes
- The number of landmarks is read from the ONNX output and the matching meanface
table is selected automatically — there is no `num_lms=` argument.
- PIPNet uses an asymmetric crop around the bbox (+10% left / right / bottom,
10% top) and ImageNet normalization. This is handled internally.
- Output landmarks are in original-image pixel coordinates as `float32`.
---
## Face Mesh (468 or 478 points, 3D)
Google MediaPipe's dense mesh. Unlike the other landmarkers it returns **3D** points
and a face-presence score, and it processes every face in an image in a single batched
inference call.
### Basic Usage
Works with any detector — it needs a bounding box, plus the first two landmarks (the
eyes) to align the crop:
```python
from uniface import SCRFD, FaceMesh
detector, mesher = SCRFD(), FaceMesh()
faces = detector.detect(image)
results = mesher.predict(image, faces) # one batched call for all faces
results[0].landmarks.shape # (468, 3) — x, y in image pixels; z is relative depth
results[0].points_2d.shape # (468, 2) — depth dropped
results[0].score # face presence, [0, 1]
```
Without a detector, pass boxes directly:
```python
results = mesher.predict(image, bboxes=[[x1, y1, x2, y2]])
```
### Iris landmarks
`V2_478` is MediaPipe's Face Landmarker: the same 468 mesh points in the same order,
plus ten iris points. Everything else — the detector, the ROI, the drawing edges — is
unchanged, so switching is a one-word change.
Following MediaPipe, the result stays a flat array and region membership lives in
named constants:
```python
from uniface import SCRFD, FaceMesh
from uniface.constants import FaceMeshWeights
from uniface.landmark import IRIS_LEFT, IRIS_RIGHT, NUM_MESH_LANDMARKS
mesher = FaceMesh(model_name=FaceMeshWeights.V2_478)
result = mesher.predict(image, SCRFD().detect(image))[0]
result.landmarks.shape # (478, 3)
result.landmarks[:NUM_MESH_LANDMARKS] # the 468 mesh points, as V1_468 returns them
result.landmarks[IRIS_LEFT] # (5, 3) — center, right, top, left, bottom
result.landmarks[IRIS_RIGHT] # (5, 3)
left_pupil = result.landmarks[IRIS_LEFT][0, :2]
```
Each iris is ordered center-first, so the mean distance from point 0 to the other four
gives the iris radius. A human iris is close to 11.7 mm across regardless of the
person, which makes that radius usable as a scale reference for real-world distance.
`V2_478` runs at 256×256 against `V1_468`'s 192×192 — 113 against 35 MMac. The
wall-clock gap is narrower than that ratio since neither model saturates a modern CPU,
so benchmark your own target. It is an addition, not a replacement: stay on `V1_468`
unless you need the irises.
### Drop-in Use
`FaceMesh` implements the same interface as `Landmark106` and `PIPNet`, so it can be
swapped into existing code that expects 2D points:
```python
landmarks = mesher.get_landmarks(image, face.bbox) # (478, 2) for the V2_478 mesher above
```
### MediaPipe Parity
Seeding the mesh with [BlazeFace](detection.md#blazeface) — the detector MediaPipe uses
internally — reproduces MediaPipe's own output:
```python
from uniface import BlazeFace, FaceMesh
detector, mesher = BlazeFace(), FaceMesh()
results = mesher.predict(image, detector.detect(image))
```
### Visualization
```python
from uniface.draw import draw_mesh
draw_mesh(image, results[0].landmarks) # 'partial': contours + points
draw_mesh(image, results[0].landmarks, mode='full') # dense 2556-edge tessellation
draw_mesh(image, results[0].landmarks, mode='points') # points only
```
!!! tip "Use `partial` or `points` for video"
`mode='full'` issues 2556 line draws per face. It is the most detailed view but
noticeably slower — prefer the other two for real-time work.
### Notes
- The crop follows MediaPipe's ROI rule: a square region at 1.5× the detector box,
rotated so the eye line is horizontal. Tune it with `margin=` if the mesh clips.
- Passing `Face` objects roll-normalizes the crop automatically. With bare `bboxes`
the crop is axis-aligned, which degrades the mesh on tilted heads.
- `roi_from_box` and `warp_roi` are public, so you can build MediaPipe's video-mode
ROI tracking (ROI from the previous frame's mesh) on top of the model.
- `score` saturates near 1.0 for anything plausible. It confirms the model ran; it is
not a discriminative confidence, so do not threshold on it.
---
## 5-Point Landmarks (Detection)
All detection models except BlazeFace provide 5-point landmarks. BlazeFace returns 6
MediaPipe keypoints instead, so its `supports_alignment` is `False`; see
[Detection](detection.md#blazeface).
```python
from uniface.detection import RetinaFace
detector = RetinaFace()
faces = detector.detect(image)
@@ -152,11 +320,11 @@ def draw_landmarks_with_connections(image, landmarks):
### Face Alignment
```python
from uniface import face_alignment
from uniface.face_utils import face_alignment
# Align face using 5-point landmarks
aligned = face_alignment(image, faces[0].landmarks)
# Returns: 112x112 aligned face
aligned, _ = face_alignment(image, faces[0].landmarks)
# Returns: (112x112 aligned face, inverse transform matrix)
```
### Eye Aspect Ratio (Blink Detection)
@@ -233,12 +401,26 @@ def estimate_head_pose(landmarks, image_shape):
---
## Factory Function
## Available Landmarkers
```python
from uniface import create_landmarker
from uniface.constants import FaceMeshWeights, PIPNetWeights
from uniface.landmark import FaceMesh, Landmark106, PIPNet
landmarker = create_landmarker() # Returns Landmark106
# Default: 106-point InsightFace model
landmarker = Landmark106()
# 98-point PIPNet (WFLW)
landmarker = PIPNet()
# 68-point PIPNet (300W+CelebA)
landmarker = PIPNet(model_name=PIPNetWeights.DW300_CELEBA_68)
# 468-point dense 3D mesh (MediaPipe Face Mesh)
landmarker = FaceMesh()
# 478-point dense 3D mesh with irises (MediaPipe Face Landmarker)
landmarker = FaceMesh(model_name=FaceMeshWeights.V2_478)
```
---
@@ -249,3 +431,4 @@ landmarker = create_landmarker() # Returns Landmark106
- [Attributes Module](attributes.md) - Age, gender, emotion
- [Gaze Module](gaze.md) - Gaze estimation
- [Concepts: Coordinate Systems](../concepts/coordinate-systems.md) - Landmark formats
- [CLI Tools](https://github.com/yakhyo/uniface/blob/main/tools/README.md) - Command-line scripts for all UniFace modules

156
docs/modules/matting.md Normal file
View File

@@ -0,0 +1,156 @@
# Portrait Matting
Portrait matting produces a soft alpha matte separating the foreground (person) from the background — no trimap needed.
<figure markdown="span">
![Portrait Matting](https://raw.githubusercontent.com/yakhyo/uniface/main/assets/demos/matting.jpg){ width="100%" }
<figcaption>MODNet: Input → Matte → Green Screen</figcaption>
</figure>
---
## Available Models
| Model | Variant | Size | Use Case |
|-------|---------|------|----------|
| **MODNet Photographic** :material-check-circle: | PHOTOGRAPHIC | 25 MB | High-quality portrait photos |
| MODNet Webcam | WEBCAM | 25 MB | Real-time webcam feeds |
---
## Basic Usage
```python
import cv2
from uniface.matting import MODNet
matting = MODNet()
image = cv2.imread("photo.jpg")
matte = matting.predict(image)
print(f"Matte shape: {matte.shape}") # (H, W)
print(f"Matte dtype: {matte.dtype}") # float32
print(f"Matte range: [{matte.min():.2f}, {matte.max():.2f}]") # [0, 1]
```
---
## Model Variants
```python
from uniface.matting import MODNet
from uniface.constants import MODNetWeights
# Photographic (default) — best for photos
matting = MODNet()
# Webcam — optimized for real-time
matting = MODNet(model_name=MODNetWeights.WEBCAM)
# Custom input size
matting = MODNet(input_size=256)
```
| Parameter | Default | Description |
|-----------|---------|-------------|
| `model_name` | `PHOTOGRAPHIC` | Model variant to load |
| `input_size` | `512` | Target shorter-side size for preprocessing |
| `providers` | `None` | ONNX Runtime execution providers |
---
## Applications
### Transparent Background (RGBA)
```python
import cv2
import numpy as np
matting = MODNet()
image = cv2.imread("photo.jpg")
matte = matting.predict(image)
rgba = cv2.cvtColor(image, cv2.COLOR_BGR2BGRA)
rgba[:, :, 3] = (matte * 255).astype(np.uint8)
cv2.imwrite("transparent.png", rgba)
```
### Green Screen
```python
import numpy as np
matte_3ch = matte[:, :, np.newaxis]
bg = np.full_like(image, (0, 177, 64), dtype=np.uint8)
green = (image * matte_3ch + bg * (1 - matte_3ch)).astype(np.uint8)
cv2.imwrite("green_screen.jpg", green)
```
### Custom Background
```python
import cv2
import numpy as np
background = cv2.imread("beach.jpg")
background = cv2.resize(background, (image.shape[1], image.shape[0]))
matte_3ch = matte[:, :, np.newaxis]
result = (image * matte_3ch + background * (1 - matte_3ch)).astype(np.uint8)
cv2.imwrite("custom_bg.jpg", result)
```
### Webcam Matting
```python
import cv2
import numpy as np
from uniface.matting import MODNet
from uniface.constants import MODNetWeights
matting = MODNet(model_name=MODNetWeights.WEBCAM)
cap = cv2.VideoCapture(0)
while True:
ret, frame = cap.read()
if not ret:
break
matte = matting.predict(frame)
matte_3ch = matte[:, :, np.newaxis]
bg = np.full_like(frame, (0, 177, 64), dtype=np.uint8)
result = (frame * matte_3ch + bg * (1 - matte_3ch)).astype(np.uint8)
cv2.imshow("Matting", np.hstack([frame, result]))
if cv2.waitKey(1) & 0xFF == ord("q"):
break
cap.release()
cv2.destroyAllWindows()
```
---
## Available Matting Models
```python
from uniface.constants import MODNetWeights
from uniface.matting import MODNet
# Default (Photographic)
matting = MODNet()
# Webcam variant
matting = MODNet(model_name=MODNetWeights.WEBCAM)
```
---
## Next Steps
- [Parsing](parsing.md) - Face semantic segmentation
- [Privacy](privacy.md) - Face anonymization
- [Detection](detection.md) - Face detection
- [CLI Tools](https://github.com/yakhyo/uniface/blob/main/tools/README.md) - Command-line scripts for all UniFace modules

View File

@@ -1,15 +1,26 @@
# Parsing
Face parsing segments faces into semantic components (skin, eyes, nose, mouth, hair, etc.).
Face parsing segments faces into semantic components or face regions.
<figure markdown="span">
![Face Parsing](https://raw.githubusercontent.com/yakhyo/uniface/main/assets/demos/parsing.jpg){ width="80%" }
<figcaption>BiSeNet face parsing with 19 semantic component classes</figcaption>
</figure>
<figure markdown="span">
![Face Segmentation](https://raw.githubusercontent.com/yakhyo/uniface/main/assets/demos/segmentation.jpg){ width="80%" }
<figcaption>XSeg face region segmentation mask</figcaption>
</figure>
---
## Available Models
| Model | Backbone | Size | Classes | Best For |
|-------|----------|------|---------|----------|
| **BiSeNet ResNet18** :material-check-circle: | ResNet18 | 51 MB | 19 | Balanced (recommended) |
| **BiSeNet ResNet34** | ResNet34 | 89 MB | 19 | Higher accuracy |
| Model | Backbone | Size | Output |
|-------|----------|------|--------|
| **BiSeNet ResNet18** :material-check-circle: | ResNet18 | 51 MB | 19 classes |
| BiSeNet ResNet34 | ResNet34 | 89 MB | 19 classes |
| XSeg | - | 67 MB | Mask |
---
@@ -18,7 +29,7 @@ Face parsing segments faces into semantic components (skin, eyes, nose, mouth, h
```python
import cv2
from uniface.parsing import BiSeNet
from uniface.visualization import vis_parsing_maps
from uniface.draw import vis_parsing_maps
# Initialize parser
parser = BiSeNet()
@@ -30,13 +41,11 @@ face_image = cv2.imread("face.jpg")
mask = parser.parse(face_image)
print(f"Mask shape: {mask.shape}") # (H, W)
# Visualize
face_rgb = cv2.cvtColor(face_image, cv2.COLOR_BGR2RGB)
vis_result = vis_parsing_maps(face_rgb, mask, save_image=False)
# Visualize (BGR in, BGR out — same convention as cv2)
vis_result = vis_parsing_maps(face_image, mask, save_image=False)
# Save result
vis_bgr = cv2.cvtColor(vis_result, cv2.COLOR_RGB2BGR)
cv2.imwrite("parsed.jpg", vis_bgr)
cv2.imwrite("parsed.jpg", vis_result)
```
---
@@ -45,16 +54,16 @@ cv2.imwrite("parsed.jpg", vis_bgr)
| ID | Class | ID | Class |
|----|-------|----|-------|
| 0 | Background | 10 | Ear Ring |
| 1 | Skin | 11 | Nose |
| 2 | Left Eyebrow | 12 | Mouth |
| 3 | Right Eyebrow | 13 | Upper Lip |
| 4 | Left Eye | 14 | Lower Lip |
| 5 | Right Eye | 15 | Neck |
| 6 | Eye Glasses | 16 | Neck Lace |
| 7 | Left Ear | 17 | Cloth |
| 8 | Right Ear | 18 | Hair |
| 9 | Hat | | |
| 0 | Background | 10 | Nose |
| 1 | Skin | 11 | Mouth |
| 2 | Left Eyebrow | 12 | Upper Lip |
| 3 | Right Eyebrow | 13 | Lower Lip |
| 4 | Left Eye | 14 | Neck |
| 5 | Right Eye | 15 | Necklace |
| 6 | Eyeglasses | 16 | Cloth |
| 7 | Left Ear | 17 | Hair |
| 8 | Right Ear | 18 | Hat |
| 9 | Earring | | |
---
@@ -71,10 +80,10 @@ parser = BiSeNet()
parser = BiSeNet(model_name=ParsingWeights.RESNET34)
```
| Variant | Params | Size | Notes |
|---------|--------|------|-------|
| **RESNET18** :material-check-circle: | 13.3M | 51 MB | Recommended |
| RESNET34 | 24.1M | 89 MB | Higher accuracy |
| Variant | Params | Size |
|---------|--------|------|
| **RESNET18** :material-check-circle: | 13.3M | 51 MB |
| RESNET34 | 24.1M | 89 MB |
---
@@ -84,9 +93,9 @@ parser = BiSeNet(model_name=ParsingWeights.RESNET34)
```python
import cv2
from uniface import RetinaFace
from uniface.detection import RetinaFace
from uniface.parsing import BiSeNet
from uniface.visualization import vis_parsing_maps
from uniface.draw import vis_parsing_maps
detector = RetinaFace()
parser = BiSeNet()
@@ -102,13 +111,11 @@ for i, face in enumerate(faces):
# Parse
mask = parser.parse(face_crop)
# Visualize
face_rgb = cv2.cvtColor(face_crop, cv2.COLOR_BGR2RGB)
vis_result = vis_parsing_maps(face_rgb, mask, save_image=False)
# Visualize (BGR in, BGR out)
vis_result = vis_parsing_maps(face_crop, mask, save_image=False)
# Save
vis_bgr = cv2.cvtColor(vis_result, cv2.COLOR_RGB2BGR)
cv2.imwrite(f"face_{i}_parsed.jpg", vis_bgr)
cv2.imwrite(f"face_{i}_parsed.jpg", vis_result)
```
---
@@ -125,7 +132,7 @@ mask = parser.parse(face_image)
# Extract specific component
SKIN = 1
HAIR = 18
HAIR = 17
LEFT_EYE = 4
RIGHT_EYE = 5
@@ -148,10 +155,10 @@ mask = parser.parse(face_image)
component_names = {
0: 'Background', 1: 'Skin', 2: 'L-Eyebrow', 3: 'R-Eyebrow',
4: 'L-Eye', 5: 'R-Eye', 6: 'Glasses', 7: 'L-Ear', 8: 'R-Ear',
9: 'Hat', 10: 'Earring', 11: 'Nose', 12: 'Mouth',
13: 'U-Lip', 14: 'L-Lip', 15: 'Neck', 16: 'Necklace',
17: 'Cloth', 18: 'Hair'
4: 'L-Eye', 5: 'R-Eye', 6: 'Eyeglasses', 7: 'L-Ear', 8: 'R-Ear',
9: 'Earring', 10: 'Nose', 11: 'Mouth',
12: 'U-Lip', 13: 'L-Lip', 14: 'Neck', 15: 'Necklace',
16: 'Cloth', 17: 'Hair', 18: 'Hat'
}
for class_id in np.unique(mask):
@@ -176,23 +183,19 @@ def apply_lip_color(image, mask, color=(180, 50, 50)):
"""Apply lip color using parsing mask."""
result = image.copy()
# Get lip mask (upper + lower lip)
lip_mask = ((mask == 13) | (mask == 14)).astype(np.uint8)
# Get lip mask (upper lip=12, lower lip=13)
lip_mask = ((mask == 12) | (mask == 13)).astype(np.uint8)
# Create color overlay
overlay = np.zeros_like(image)
overlay[:] = color
# Blend with original
lip_region = cv2.bitwise_and(overlay, overlay, mask=lip_mask)
non_lip = cv2.bitwise_and(result, result, mask=1 - lip_mask)
# Combine with alpha blending
# Alpha blend lip region
alpha = 0.4
result = cv2.addWeighted(result, 1 - alpha * lip_mask[:,:,np.newaxis] / 255,
lip_region, alpha, 0)
mask_3ch = lip_mask[:, :, np.newaxis]
result = np.where(mask_3ch, (image * (1 - alpha) + overlay * alpha).astype(np.uint8), result)
return result.astype(np.uint8)
return result
```
### Background Replacement
@@ -218,7 +221,7 @@ def replace_background(image, mask, background):
```python
def get_hair_mask(mask):
"""Extract clean hair mask."""
hair_mask = (mask == 18).astype(np.uint8) * 255
hair_mask = (mask == 17).astype(np.uint8) * 255
# Clean up with morphological operations
kernel = np.ones((5, 5), np.uint8)
@@ -233,14 +236,14 @@ def get_hair_mask(mask):
## Visualization Options
```python
from uniface.visualization import vis_parsing_maps
from uniface.draw import vis_parsing_maps
# Default visualization
vis_result = vis_parsing_maps(face_rgb, mask)
# Default visualization (BGR in, BGR out)
vis_result = vis_parsing_maps(face_image, mask)
# With different parameters
vis_result = vis_parsing_maps(
face_rgb,
face_image,
mask,
save_image=False, # Don't save to file
)
@@ -248,12 +251,83 @@ vis_result = vis_parsing_maps(
---
## Factory Function
## XSeg
XSeg outputs a mask for face regions. Unlike BiSeNet which works on bbox crops, XSeg requires 5-point landmarks for face alignment.
### Basic Usage
```python
from uniface import create_face_parser
import cv2
from uniface.detection import RetinaFace
from uniface.parsing import XSeg
parser = create_face_parser() # Returns BiSeNet
detector = RetinaFace()
parser = XSeg()
image = cv2.imread("photo.jpg")
faces = detector.detect(image)
for face in faces:
if face.landmarks is not None:
mask = parser.parse(image, landmarks=face.landmarks)
print(f"Mask shape: {mask.shape}") # (H, W), values in [0, 1]
```
### Parameters
```python
from uniface.parsing import XSeg
# Default settings
parser = XSeg()
# Custom settings
parser = XSeg(
align_size=256, # Face alignment size
blur_sigma=5, # Gaussian blur for smoothing (0 = raw)
)
```
| Parameter | Default | Description |
|-----------|---------|-------------|
| `align_size` | 256 | Face alignment output size |
| `blur_sigma` | 0 | Mask smoothing (0 = no blur) |
### Methods
```python
# Full pipeline: align -> segment -> warp back to original space
mask = parser.parse(image, landmarks=landmarks)
# For pre-aligned face crops
mask = parser.parse_aligned(face_crop)
# Get mask + crop + inverse matrix for custom warping
mask, face_crop, inverse_matrix = parser.parse_with_inverse(image, landmarks)
```
### BiSeNet vs XSeg
| Feature | BiSeNet | XSeg |
|---------|---------|------|
| Output | 19 class labels | Mask [0, 1] |
| Input | Bbox crop | Requires landmarks |
| Use case | Facial components | Face region extraction |
---
## Available Parsers
```python
from uniface.constants import ParsingWeights, XSegWeights
from uniface.parsing import BiSeNet, XSeg
# BiSeNet (default)
parser = BiSeNet(model_name=ParsingWeights.RESNET18)
# XSeg
parser = XSeg(model_name=XSegWeights.DEFAULT)
```
---
@@ -263,3 +337,4 @@ parser = create_face_parser() # Returns BiSeNet
- [Gaze](gaze.md) - Gaze estimation
- [Privacy](privacy.md) - Face anonymization
- [Detection](detection.md) - Face detection
- [CLI Tools](https://github.com/yakhyo/uniface/blob/main/tools/README.md) - Command-line scripts for all UniFace modules

View File

@@ -2,41 +2,29 @@
Face anonymization protects privacy by blurring or obscuring faces in images and videos.
<figure markdown="span">
![Face Anonymization](https://raw.githubusercontent.com/yakhyo/uniface/main/assets/demos/anonymization.jpg){ width="100%" }
<figcaption>Five anonymization methods: pixelate, gaussian, blackout, elliptical, and median</figcaption>
</figure>
---
## Available Methods
| Method | Description | Use Case |
|--------|-------------|----------|
| **pixelate** | Blocky pixelation | News media standard |
| **gaussian** | Smooth blur | Natural appearance |
| **blackout** | Solid color fill | Maximum privacy |
| **elliptical** | Oval-shaped blur | Natural face shape |
| **median** | Edge-preserving blur | Artistic effect |
| Method | Description |
|--------|-------------|
| **pixelate** | Blocky pixelation |
| **gaussian** | Smooth blur |
| **blackout** | Solid color fill |
| **elliptical** | Oval-shaped blur |
| **median** | Edge-preserving blur |
---
## Quick Start
### One-Line Anonymization
```python
from uniface.privacy import anonymize_faces
import cv2
image = cv2.imread("group_photo.jpg")
anonymized = anonymize_faces(image, method='pixelate')
cv2.imwrite("anonymized.jpg", anonymized)
```
---
## BlurFace Class
For more control, use the `BlurFace` class:
```python
from uniface import RetinaFace
from uniface.detection import RetinaFace
from uniface.privacy import BlurFace
import cv2
@@ -59,12 +47,12 @@ cv2.imwrite("anonymized.jpg", anonymized)
Blocky pixelation effect (common in news media):
```python
blurrer = BlurFace(method='pixelate', pixel_blocks=10)
blurrer = BlurFace(method='pixelate', pixel_blocks=15)
```
| Parameter | Default | Description |
|-----------|---------|-------------|
| `pixel_blocks` | 10 | Number of blocks (lower = more pixelated) |
| `pixel_blocks` | 15 | Number of blocks (lower = more pixelated) |
### Gaussian
@@ -103,6 +91,15 @@ blurrer = BlurFace(method='elliptical', blur_strength=3.0, margin=20)
| `blur_strength` | 3.0 | Blur intensity |
| `margin` | 20 | Margin around face |
The elliptical blur is also available as a standalone class for direct use on bounding boxes:
```python
from uniface.privacy import EllipticalBlur
blur = EllipticalBlur(blur_strength=3.0, margin=20)
result = blur(image, bboxes=[face.bbox for face in faces])
```
### Median
Edge-preserving blur with artistic effect:
@@ -137,7 +134,7 @@ result = blurrer.anonymize(image, faces, inplace=True)
```python
import cv2
from uniface import RetinaFace
from uniface.detection import RetinaFace
from uniface.privacy import BlurFace
detector = RetinaFace()
@@ -166,7 +163,7 @@ cv2.destroyAllWindows()
```python
import cv2
from uniface import RetinaFace
from uniface.detection import RetinaFace
from uniface.privacy import BlurFace
detector = RetinaFace()
@@ -211,7 +208,7 @@ def anonymize_except(image, all_faces, exclude_embeddings, recognizer, threshold
# Check if should be excluded
should_exclude = False
for ref_emb in exclude_embeddings:
similarity = np.dot(embedding, ref_emb.T)[0][0]
similarity = np.dot(embedding, ref_emb)
if similarity > threshold:
should_exclude = True
break
@@ -238,7 +235,7 @@ def anonymize_low_confidence(image, faces, blurrer, confidence_threshold=0.8):
```python
import cv2
from uniface import RetinaFace
from uniface.detection import RetinaFace
from uniface.privacy import BlurFace
detector = RetinaFace()
@@ -255,23 +252,9 @@ for method in methods:
---
## Command-Line Tool
```bash
# Anonymize image with pixelation
python tools/face_anonymize.py --source photo.jpg
# Real-time webcam
python tools/face_anonymize.py --source 0 --method gaussian
# Custom blur strength
python tools/face_anonymize.py --source photo.jpg --method gaussian --blur-strength 5.0
```
---
## Next Steps
- [Anonymize Stream Recipe](../recipes/anonymize-stream.md) - Video pipeline
- [Detection](detection.md) - Face detection options
- [Batch Processing Recipe](../recipes/batch-processing.md) - Process multiple files
- [CLI Tools](https://github.com/yakhyo/uniface/blob/main/tools/README.md) - Command-line scripts for all UniFace modules

204
docs/modules/quality.md Normal file
View File

@@ -0,0 +1,204 @@
# Face Image Quality Assessment
Face image quality assessment predicts a single scalar score from an aligned face crop. Higher score = better quality (sharpness, frontalness, illumination, low occlusion). Use the score to filter or rank faces before recognition.
UniFace ships [eDifFIQA](https://github.com/yakhyo/face-image-quality-assessment), which ranks first on the NIST [FATE-Quality](https://pages.nist.gov/frvt/html/frvt_quality.html) Kiosk-to-Entry track with the L variant.
---
## Available Models
| Variant | Backbone | Params | Size |
|---------|----------|--------|------|
| **eDifFIQA-T** :material-check-circle: | MobileFaceNet | 1.7M | 6.6 MB |
| eDifFIQA-S | IResNet-18 | 24.6M | 93.7 MB |
| eDifFIQA-M | IResNet-50 | 44.1M | 168.3 MB |
| eDifFIQA-L | IResNet-100 | 65.7M | 250.6 MB |
All variants take an aligned 112x112 face crop and output a scalar score in roughly [0, 1].
---
## Basic Usage
```python
import cv2
from uniface.detection import SCRFD
from uniface.quality import EDifFIQA
# Lower SCRFD's confidence threshold so quality scoring sees every plausible
# face — including the low-confidence ones that quality scoring is meant to flag.
detector = SCRFD(confidence_threshold=0.3)
quality = EDifFIQA()
image = cv2.imread("photo.jpg")
faces = detector.detect(image)
for face in faces:
result = quality.predict(image, face.landmarks)
print(f"Quality: {result.score:.4f}")
```
`predict` aligns the face using the 5-point landmarks and runs inference. To skip alignment (pre-aligned 112x112 crop), call `score_aligned` instead:
```python
result = quality.score_aligned(aligned_face)
```
!!! tip "Detector threshold"
SCRFD's default `confidence_threshold` is 0.5, which can miss tightly-cropped or partial faces — exactly the cases you want quality scoring to surface. Drop it to ~0.3 for quality use cases.
---
## Output Format
```python
result = quality.predict(image, face.landmarks)
# QualityResult dataclass
result.score # float, higher = better
```
Scores are roughly in `[0, 1]` but eDifFIQA is **not** a calibrated probability — the absolute number is only meaningful relative to other faces from the same model. As a rough guide from the upstream demo (eDifFIQA-T):
| Score | Typical face |
|-------|--------------|
| ≥ 0.7 | Sharp, frontal, well-lit |
| 0.4 0.7 | Acceptable (some blur / off-angle / low-res) |
| < 0.3 | Heavy blur, occlusion, or severe degradation |
Calibrate your own threshold on a held-out set rather than reusing these bands verbatim.
---
## Model Variants
```python
from uniface.quality import EDifFIQA
from uniface.constants import EDifFIQAWeights
# Default (T, smallest, recommended for real-time)
quality = EDifFIQA()
# More accurate variants
quality = EDifFIQA(model_name=EDifFIQAWeights.S)
quality = EDifFIQA(model_name=EDifFIQAWeights.M)
quality = EDifFIQA(model_name=EDifFIQAWeights.L)
```
---
## Visualization
```python
from uniface.draw import draw_quality_score
for face in faces:
result = quality.predict(image, face.landmarks)
draw_quality_score(image, face.bbox, result.score)
cv2.imwrite("quality_output.jpg", image)
```
The label is color-coded by score: red (< 0.3), orange (0.3-0.6), green (>= 0.6). Adjust with `low_threshold` and `high_threshold` kwargs.
---
## Use Cases
### Filter Low-Quality Faces Before Recognition
```python
from uniface.detection import SCRFD
from uniface.quality import EDifFIQA
from uniface.recognition import ArcFace
detector = SCRFD(confidence_threshold=0.3)
quality = EDifFIQA()
recognizer = ArcFace()
MIN_QUALITY = 0.5 # tune on your data
for face in detector.detect(image):
score = quality.predict(image, face.landmarks).score
if score < MIN_QUALITY:
continue
face.embedding = recognizer.get_normalized_embedding(image, face.landmarks)
```
### Rank Faces in a Video
```python
best = (None, -1.0) # (frame_idx, score)
for idx, frame in enumerate(frames):
for face in detector.detect(frame):
score = quality.predict(frame, face.landmarks).score
if score > best[1]:
best = (idx, score)
print(f"Best frame: {best[0]} (quality {best[1]:.4f})")
```
### Best-of-N Enrollment
Pick the sharpest frame from a short capture for face enrollment:
```python
def best_face_from_capture(frames, detector, quality):
"""Return the (frame, face) with the highest quality across frames."""
best = (None, None, -1.0)
for frame in frames:
for face in detector.detect(frame):
score = quality.predict(frame, face.landmarks).score
if score > best[2]:
best = (frame, face, score)
return best # (frame, face, score)
```
---
## Real-Time Webcam
```python
import cv2
from uniface.detection import SCRFD
from uniface.quality import EDifFIQA
from uniface.draw import draw_quality_score
detector = SCRFD(confidence_threshold=0.3)
quality = EDifFIQA()
cap = cv2.VideoCapture(0)
while True:
ret, frame = cap.read()
if not ret:
break
for face in detector.detect(frame):
result = quality.predict(frame, face.landmarks)
draw_quality_score(frame, face.bbox, result.score)
cv2.imshow("Face Quality", frame)
if cv2.waitKey(1) & 0xFF == ord('q'):
break
cap.release()
cv2.destroyAllWindows()
```
---
## Reference
- Paper: ["eDifFIQA: Towards Efficient Face Image Quality Assessment based on Denoising Diffusion Probabilistic Models"](https://ieeexplore.ieee.org/document/10468647) (Babnik et al., IEEE T-BIOM 2024)
- Code: [yakhyo/face-image-quality-assessment](https://github.com/yakhyo/face-image-quality-assessment) — PyTorch inference, ONNX export, and ONNX Runtime inference. ONNX weights for UniFace are mirrored here.
---
## Next Steps
- [Recognition](recognition.md) - Run recognition only on high-quality faces
- [Anti-Spoofing](spoofing.md) - Liveness check
- [Detection](detection.md) - Face detection
- [CLI Tools](https://github.com/yakhyo/uniface/blob/main/tools/README.md) - Command-line scripts for all UniFace modules

View File

@@ -2,26 +2,82 @@
Face recognition extracts embeddings for identity verification and face search.
<figure markdown="span">
![Face Verification](https://raw.githubusercontent.com/yakhyo/uniface/main/assets/demos/verification.jpg){ width="80%" }
<figcaption>Pairwise face verification with cosine similarity scores</figcaption>
</figure>
---
## Available Models
| Model | Backbone | Size | Embedding Dim | Best For |
|-------|----------|------|---------------|----------|
| **ArcFace** | MobileNet/ResNet | 8-166 MB | 512 | General use (recommended) |
| **MobileFace** | MobileNet V2/V3 | 1-10 MB | 512 | Mobile/Edge |
| **SphereFace** | Sphere20/36 | 50-92 MB | 512 | Research |
| Model | Backbone | Size | Embedding Dim |
|-------|----------|------|---------------|
| **AdaFace** | IR-18/IR-101 | 92-249 MB | 512 |
| **ArcFace** | MobileNet/ResNet | 8-166 MB | 512 |
| **EdgeFace** | EdgeNeXt/LoRA | 5-70 MB | 512 |
| **MobileFace** | MobileNet V2/V3 | 1-10 MB | 512 |
| **SphereFace** | Sphere20/36 | 50-92 MB | 512 |
---
## AdaFace
Face recognition using adaptive margin based on image quality.
### Basic Usage
```python
from uniface.detection import RetinaFace
from uniface.recognition import AdaFace
detector = RetinaFace()
recognizer = AdaFace()
# Detect face
faces = detector.detect(image)
# Extract embedding
if faces:
embedding = recognizer.get_normalized_embedding(image, faces[0].landmarks)
print(f"Embedding shape: {embedding.shape}") # (512,)
```
### Model Variants
```python
from uniface.recognition import AdaFace
from uniface.constants import AdaFaceWeights
# Lightweight (default)
recognizer = AdaFace(model_name=AdaFaceWeights.IR_18)
# High accuracy
recognizer = AdaFace(model_name=AdaFaceWeights.IR_101)
# Force CPU execution
recognizer = AdaFace(providers=['CPUExecutionProvider'])
```
| Variant | Dataset | Size | IJB-B | IJB-C |
|---------|---------|------|-------|-------|
| **IR_18** :material-check-circle: | WebFace4M | 92 MB | 93.03% | 94.99% |
| IR_101 | WebFace12M | 249 MB | - | 97.66% |
!!! info "Benchmark Metrics"
IJB-B and IJB-C accuracy reported as TAR@FAR=0.01%
---
## ArcFace
State-of-the-art recognition using additive angular margin loss.
Face recognition using additive angular margin loss.
### Basic Usage
```python
from uniface import RetinaFace, ArcFace
from uniface.detection import RetinaFace
from uniface.recognition import ArcFace
detector = RetinaFace()
recognizer = ArcFace()
@@ -32,13 +88,13 @@ faces = detector.detect(image)
# Extract embedding
if faces:
embedding = recognizer.get_normalized_embedding(image, faces[0].landmarks)
print(f"Embedding shape: {embedding.shape}") # (1, 512)
print(f"Embedding shape: {embedding.shape}") # (512,)
```
### Model Variants
```python
from uniface import ArcFace
from uniface.recognition import ArcFace
from uniface.constants import ArcFaceWeights
# Lightweight (default)
@@ -46,23 +102,89 @@ recognizer = ArcFace(model_name=ArcFaceWeights.MNET)
# High accuracy
recognizer = ArcFace(model_name=ArcFaceWeights.RESNET)
# Force CPU execution
recognizer = ArcFace(providers=['CPUExecutionProvider'])
```
| Variant | Backbone | Size | Use Case |
|---------|----------|------|----------|
| **MNET** :material-check-circle: | MobileNet | 8 MB | Balanced (recommended) |
| RESNET | ResNet50 | 166 MB | Maximum accuracy |
| Variant | Backbone | Size | LFW | CFP-FP | AgeDB-30 | IJB-C |
|---------|----------|------|-----|--------|----------|-------|
| **MNET** :material-check-circle: | MobileNet | 8 MB | 99.70% | 98.00% | 96.58% | 95.02% |
| RESNET | ResNet50 | 166 MB | 99.83% | 99.33% | 98.23% | 97.25% |
!!! info "Training Data & Metrics"
**Dataset**: Trained on WebFace600K (600K images)
**Accuracy**: IJB-C reported as TAR@FAR=1e-4
---
## EdgeFace
Efficient face recognition designed for edge devices, using an EdgeNeXt backbone with optional LoRA low-rank compression. Competition-winning entry (compact track) at EFaR 2023, IJCB.
### Basic Usage
```python
from uniface.detection import RetinaFace
from uniface.recognition import EdgeFace
detector = RetinaFace()
recognizer = EdgeFace()
# Detect face
faces = detector.detect(image)
# Extract embedding
if faces:
embedding = recognizer.get_normalized_embedding(image, faces[0].landmarks)
print(f"Embedding shape: {embedding.shape}") # (512,)
```
### Model Variants
```python
from uniface.recognition import EdgeFace
from uniface.constants import EdgeFaceWeights
# Ultra-compact (default)
recognizer = EdgeFace(model_name=EdgeFaceWeights.XXS)
# Compact with LoRA
recognizer = EdgeFace(model_name=EdgeFaceWeights.XS_GAMMA_06)
# Small with LoRA
recognizer = EdgeFace(model_name=EdgeFaceWeights.S_GAMMA_05)
# Full-size
recognizer = EdgeFace(model_name=EdgeFaceWeights.BASE)
# Force CPU execution
recognizer = EdgeFace(providers=['CPUExecutionProvider'])
```
| Variant | Params | MFLOPs | Size | LFW | CALFW | CPLFW | CFP-FP | AgeDB-30 |
|---------|--------|--------|------|-----|-------|-------|--------|----------|
| **XXS** :material-check-circle: | 1.24M | 94 | ~5 MB | 99.57% | 94.83% | 90.27% | 93.63% | 94.92% |
| XS_GAMMA_06 | 1.77M | 154 | ~7 MB | 99.73% | 95.28% | 91.58% | 94.71% | 96.08% |
| S_GAMMA_05 | 3.65M | 306 | ~14 MB | 99.78% | 95.55% | 92.48% | 95.74% | 97.03% |
| BASE | 18.2M | 1399 | ~70 MB | 99.83% | 96.07% | 93.75% | 97.01% | 97.60% |
!!! info "Reference"
**Paper**: [EdgeFace: Efficient Face Recognition Model for Edge Devices](https://arxiv.org/abs/2307.01838v2) (IEEE T-BIOM 2024)
**Source**: [github.com/otroshi/edgeface](https://github.com/otroshi/edgeface)
---
## MobileFace
Lightweight recognition for resource-constrained environments.
Lightweight face recognition models with MobileNet backbones.
### Basic Usage
```python
from uniface import MobileFace
from uniface.recognition import MobileFace
recognizer = MobileFace()
embedding = recognizer.get_normalized_embedding(image, landmarks)
@@ -71,7 +193,7 @@ embedding = recognizer.get_normalized_embedding(image, landmarks)
### Model Variants
```python
from uniface import MobileFace
from uniface.recognition import MobileFace
from uniface.constants import MobileFaceWeights
# Ultra-lightweight
@@ -84,33 +206,33 @@ recognizer = MobileFace(model_name=MobileFaceWeights.MNET_V2)
recognizer = MobileFace(model_name=MobileFaceWeights.MNET_V3_LARGE)
```
| Variant | Params | Size | LFW | Use Case |
|---------|--------|------|-----|----------|
| MNET_025 | 0.36M | 1 MB | 98.8% | Ultra-lightweight |
| **MNET_V2** :material-check-circle: | 2.29M | 4 MB | 99.6% | Mobile/Edge |
| MNET_V3_SMALL | 1.25M | 3 MB | 99.3% | Mobile optimized |
| MNET_V3_LARGE | 3.52M | 10 MB | 99.5% | Balanced mobile |
| Variant | Params | Size | LFW | CALFW | CPLFW | AgeDB-30 |
|---------|--------|------|-----|-------|-------|----------|
| MNET_025 | 0.36M | 1 MB | 98.76% | 92.02% | 82.37% | 90.02% |
| **MNET_V2** :material-check-circle: | 2.29M | 4 MB | 99.55% | 94.87% | 86.89% | 95.16% |
| MNET_V3_SMALL | 1.25M | 3 MB | 99.30% | 93.77% | 85.29% | 92.79% |
| MNET_V3_LARGE | 3.52M | 10 MB | 99.53% | 94.56% | 86.79% | 95.13% |
---
## SphereFace
Recognition using angular softmax loss (A-Softmax).
Face recognition using angular softmax loss (A-Softmax).
### Basic Usage
```python
from uniface import SphereFace
from uniface.recognition import SphereFace
from uniface.constants import SphereFaceWeights
recognizer = SphereFace(model_name=SphereFaceWeights.SPHERE20)
embedding = recognizer.get_normalized_embedding(image, landmarks)
```
| Variant | Params | Size | LFW | Use Case |
|---------|--------|------|-----|----------|
| SPHERE20 | 24.5M | 50 MB | 99.7% | Research |
| SPHERE36 | 34.6M | 92 MB | 99.7% | Research |
| Variant | Params | Size | LFW | CALFW | CPLFW | AgeDB-30 |
|---------|--------|------|-----|-------|-------|----------|
| SPHERE20 | 24.5M | 50 MB | 99.67% | 95.61% | 88.75% | 96.58% |
| SPHERE36 | 34.6M | 92 MB | 99.72% | 95.64% | 89.92% | 96.83% |
---
@@ -119,7 +241,7 @@ embedding = recognizer.get_normalized_embedding(image, landmarks)
### Compute Similarity
```python
from uniface import compute_similarity
from uniface.face_utils import compute_similarity
import numpy as np
# Extract embeddings
@@ -129,8 +251,8 @@ emb2 = recognizer.get_normalized_embedding(image2, landmarks2)
# Method 1: Using utility function
similarity = compute_similarity(emb1, emb2)
# Method 2: Direct computation
similarity = np.dot(emb1, emb2.T)[0][0]
# Method 2: Direct computation (embeddings are L2-normalized 1-D vectors)
similarity = np.dot(emb1, emb2)
print(f"Similarity: {similarity:.4f}")
```
@@ -155,10 +277,10 @@ Recognition models require aligned faces. UniFace handles this internally:
embedding = recognizer.get_normalized_embedding(image, landmarks)
# Or manually align
from uniface import face_alignment
from uniface.face_utils import face_alignment
aligned_face = face_alignment(image, landmarks)
# Returns: 112x112 aligned face image
aligned_face, _ = face_alignment(image, landmarks)
# Returns: (112x112 aligned face, inverse transform matrix)
```
---
@@ -167,7 +289,8 @@ aligned_face = face_alignment(image, landmarks)
```python
import numpy as np
from uniface import RetinaFace, ArcFace
from uniface.detection import RetinaFace
from uniface.recognition import ArcFace
detector = RetinaFace()
recognizer = ArcFace()
@@ -203,7 +326,7 @@ def search_face(query_embedding, database, threshold=0.6):
best_similarity = -1
for person_id, db_embedding in database.items():
similarity = np.dot(query_embedding, db_embedding.T)[0][0]
similarity = np.dot(query_embedding, db_embedding)
if similarity > best_similarity and similarity > threshold:
best_similarity = similarity
@@ -223,12 +346,16 @@ else:
---
## Factory Function
## Available Recognizers
```python
from uniface import create_recognizer
from uniface.recognition import ArcFace, AdaFace, EdgeFace, MobileFace, SphereFace
recognizer = create_recognizer('arcface')
recognizer = ArcFace()
# or
recognizer = AdaFace()
# or
recognizer = EdgeFace()
```
---
@@ -238,3 +365,4 @@ recognizer = create_recognizer('arcface')
- [Detection Module](detection.md) - Detect faces first
- [Face Search Recipe](../recipes/face-search.md) - Complete search system
- [Thresholds](../concepts/thresholds-calibration.md) - Calibration guide
- [CLI Tools](https://github.com/yakhyo/uniface/blob/main/tools/README.md) - Command-line scripts for all UniFace modules

View File

@@ -6,10 +6,10 @@ Face anti-spoofing detects whether a face is real (live) or fake (photo, video r
## Available Models
| Model | Size | Notes |
|-------|------|-------|
| MiniFASNet V1SE | 1.2 MB | Squeeze-and-Excitation variant |
| **MiniFASNet V2** :material-check-circle: | 1.2 MB | Improved version (recommended) |
| Model | Size |
|-------|------|
| MiniFASNet V1SE | 1.2 MB |
| **MiniFASNet V2** :material-check-circle: | 1.2 MB |
---
@@ -17,7 +17,7 @@ Face anti-spoofing detects whether a face is real (live) or fake (photo, video r
```python
import cv2
from uniface import RetinaFace
from uniface.detection import RetinaFace
from uniface.spoofing import MiniFASNet
detector = RetinaFace()
@@ -69,20 +69,21 @@ spoofer = MiniFASNet(model_name=MiniFASNetWeights.V1SE)
## Confidence Thresholds
The default threshold is 0.5. Adjust for your use case:
`result.is_real` is based on the model's top predicted class (argmax). If you want stricter behavior,
apply your own confidence threshold:
```python
result = spoofer.predict(image, face.bbox)
# High security (fewer false accepts)
HIGH_THRESHOLD = 0.7
if result.confidence > HIGH_THRESHOLD:
if result.is_real and result.confidence > HIGH_THRESHOLD:
print("Real (high confidence)")
else:
print("Suspicious")
# Balanced
if result.is_real: # Uses default 0.5 threshold
# Balanced (argmax decision)
if result.is_real:
print("Real")
else:
print("Fake")
@@ -127,7 +128,7 @@ cv2.imwrite("spoofing_result.jpg", image)
```python
import cv2
from uniface import RetinaFace
from uniface.detection import RetinaFace
from uniface.spoofing import MiniFASNet
detector = RetinaFace()
@@ -237,24 +238,12 @@ MiniFASNet can detect various spoof attacks:
---
## Command-Line Tool
```bash
# Image
python tools/spoofing.py --source photo.jpg
# Webcam
python tools/spoofing.py --source 0
```
---
## Factory Function
## Available Spoofers
```python
from uniface import create_spoofer
from uniface.spoofing import MiniFASNet
spoofer = create_spoofer() # Returns MiniFASNet
spoofer = MiniFASNet()
```
---
@@ -264,3 +253,4 @@ spoofer = create_spoofer() # Returns MiniFASNet
- [Privacy](privacy.md) - Face anonymization
- [Detection](detection.md) - Face detection
- [Recognition](recognition.md) - Face recognition
- [CLI Tools](https://github.com/yakhyo/uniface/blob/main/tools/README.md) - Command-line scripts for all UniFace modules

173
docs/modules/stores.md Normal file
View File

@@ -0,0 +1,173 @@
# Stores
FAISS-backed vector store for fast similarity search over embeddings.
!!! info "Optional dependency"
```bash
pip install faiss-cpu
```
---
## FAISS
```python
from uniface.stores import FAISS
```
A thin wrapper around a FAISS `IndexFlatIP` (inner-product) index. Vectors
**must** be L2-normalised before adding so that inner product equals cosine
similarity. The store does not normalise internally.
Each vector is paired with a metadata `dict` that can carry any
JSON-serialisable payload (person ID, name, source path, etc.).
### Constructor
```python
store = FAISS(embedding_size=512, db_path="./vector_index")
```
| Parameter | Type | Default | Description |
|-----------|------|---------|-------------|
| `embedding_size` | `int` | `512` | Dimension of embedding vectors |
| `db_path` | `str` | `"./vector_index"` | Directory for persisting index and metadata |
---
### Methods
#### `add(embedding, metadata)`
Add a single embedding with associated metadata.
```python
store.add(embedding, {"person_id": "alice", "source": "photo.jpg"})
```
| Parameter | Type | Description |
|-----------|------|-------------|
| `embedding` | `np.ndarray` | L2-normalised embedding vector |
| `metadata` | `dict[str, Any]` | Arbitrary JSON-serialisable key-value pairs |
---
#### `search(embedding, threshold=0.4)`
Find the closest match for a query embedding.
```python
result, similarity = store.search(query_embedding, threshold=0.4)
if result:
print(result["person_id"], similarity)
```
| Parameter | Type | Default | Description |
|-----------|------|---------|-------------|
| `embedding` | `np.ndarray` | — | L2-normalised query vector |
| `threshold` | `float` | `0.4` | Minimum cosine similarity to accept a match |
**Returns:** `(metadata, similarity)` if a match is found, or `(None, similarity)` when below threshold or the index is empty.
---
#### `remove(key, value)`
Remove all entries where `metadata[key] == value` and rebuild the index.
```python
removed = store.remove("person_id", "bob")
print(f"Removed {removed} entries")
```
| Parameter | Type | Description |
|-----------|------|-------------|
| `key` | `str` | Metadata key to match |
| `value` | `Any` | Value to match |
**Returns:** Number of entries removed.
---
#### `save()`
Persist the FAISS index and metadata to disk.
```python
store.save()
```
Writes two files to `db_path`:
- `faiss_index.bin` — binary FAISS index
- `metadata.json` — JSON array of metadata dicts
---
#### `load()`
Load a previously saved index and metadata.
```python
store = FAISS(db_path="./vector_index")
loaded = store.load() # True if files exist
```
**Returns:** `True` if loaded successfully, `False` if files are missing.
**Raises:** `RuntimeError` if files exist but cannot be read.
---
### Properties
| Property | Type | Description |
|----------|------|-------------|
| `size` | `int` | Number of vectors in the index |
| `len(store)` | `int` | Same as `size` |
---
## Example: End-to-End
```python
import cv2
from uniface.detection import RetinaFace
from uniface.recognition import ArcFace
from uniface.stores import FAISS
detector = RetinaFace()
recognizer = ArcFace()
# Build
store = FAISS(db_path="./my_index")
image = cv2.imread("alice.jpg")
faces = detector.detect(image)
embedding = recognizer.get_normalized_embedding(image, faces[0].landmarks)
store.add(embedding, {"person_id": "alice"})
store.save()
# Search
store2 = FAISS(db_path="./my_index")
store2.load()
query = cv2.imread("unknown.jpg")
faces = detector.detect(query)
emb = recognizer.get_normalized_embedding(query, faces[0].landmarks)
result, sim = store2.search(emb)
if result:
print(f"Matched: {result['person_id']} (similarity: {sim:.3f})")
else:
print(f"No match (similarity: {sim:.3f})")
```
---
## See Also
- [Face Search Recipe](../recipes/face-search.md) - Building and querying indexes
- [Recognition Module](recognition.md) - Embedding extraction
- [Thresholds Guide](../concepts/thresholds-calibration.md) - Tuning similarity thresholds
- [CLI Tools](https://github.com/yakhyo/uniface/blob/main/tools/README.md) - Command-line scripts for all UniFace modules

243
docs/modules/tracking.md Normal file
View File

@@ -0,0 +1,243 @@
# Tracking
Multi-object tracking using [BYTETracker](https://github.com/yakhyo/bytetrack-tracker) with Kalman filtering and IoU-based association. The tracker assigns persistent IDs to detected objects across video frames using a two-stage association strategy — first matching high-confidence detections, then low-confidence ones.
---
## How It Works
BYTETracker takes detection bounding boxes as input and returns tracked bounding boxes with persistent IDs. It does not depend on any specific detector — any source of `[x1, y1, x2, y2, score]` arrays will work.
Each frame, the tracker:
1. Splits detections into high-confidence and low-confidence groups
2. Matches high-confidence detections to existing tracks using IoU
3. Matches remaining tracks to low-confidence detections (second chance)
4. Starts new tracks for unmatched high-confidence detections
5. Removes tracks that have been lost for too long
The Kalman filter predicts where each track will be in the next frame, which helps maintain associations even when detections are noisy.
---
## Basic Usage
```python
import cv2
import numpy as np
from uniface.common import xyxy_to_cxcywh
from uniface.detection import SCRFD
from uniface.tracking import BYTETracker
from uniface.draw import draw_tracks
detector = SCRFD()
tracker = BYTETracker(track_thresh=0.5, track_buffer=30)
cap = cv2.VideoCapture("video.mp4")
while cap.isOpened():
ret, frame = cap.read()
if not ret:
break
# 1. Detect faces
faces = detector.detect(frame)
# 2. Build detections array: [x1, y1, x2, y2, score]
dets = np.array([[*f.bbox, f.confidence] for f in faces])
dets = dets if len(dets) > 0 else np.empty((0, 5))
# 3. Update tracker
tracks = tracker.update(dets)
# 4. Map track IDs back to face objects
if len(tracks) > 0 and len(faces) > 0:
face_bboxes = np.array([f.bbox for f in faces], dtype=np.float32)
track_ids = tracks[:, 4].astype(int)
face_centers = xyxy_to_cxcywh(face_bboxes)[:, :2]
track_centers = xyxy_to_cxcywh(tracks[:, :4])[:, :2]
for ti in range(len(tracks)):
dists = (track_centers[ti, 0] - face_centers[:, 0]) ** 2 + (track_centers[ti, 1] - face_centers[:, 1]) ** 2
faces[int(np.argmin(dists))].track_id = track_ids[ti]
# 5. Draw
tracked_faces = [f for f in faces if f.track_id is not None]
draw_tracks(image=frame, faces=tracked_faces)
cv2.imshow("Tracking", frame)
if cv2.waitKey(1) & 0xFF == ord('q'):
break
cap.release()
cv2.destroyAllWindows()
```
Each track ID gets a deterministic color via golden-ratio hue stepping, so the same person keeps the same color across the entire video.
---
## Webcam Tracking
```python
import cv2
import numpy as np
from uniface.common import xyxy_to_cxcywh
from uniface.detection import SCRFD
from uniface.tracking import BYTETracker
from uniface.draw import draw_tracks
detector = SCRFD()
tracker = BYTETracker(track_thresh=0.5, track_buffer=30)
cap = cv2.VideoCapture(0)
while True:
ret, frame = cap.read()
if not ret:
break
faces = detector.detect(frame)
dets = np.array([[*f.bbox, f.confidence] for f in faces])
dets = dets if len(dets) > 0 else np.empty((0, 5))
tracks = tracker.update(dets)
if len(tracks) > 0 and len(faces) > 0:
face_bboxes = np.array([f.bbox for f in faces], dtype=np.float32)
track_ids = tracks[:, 4].astype(int)
face_centers = xyxy_to_cxcywh(face_bboxes)[:, :2]
track_centers = xyxy_to_cxcywh(tracks[:, :4])[:, :2]
for ti in range(len(tracks)):
dists = (track_centers[ti, 0] - face_centers[:, 0]) ** 2 + (track_centers[ti, 1] - face_centers[:, 1]) ** 2
faces[int(np.argmin(dists))].track_id = track_ids[ti]
draw_tracks(image=frame, faces=[f for f in faces if f.track_id is not None])
cv2.imshow("Face Tracking - Press 'q' to quit", frame)
if cv2.waitKey(1) & 0xFF == ord('q'):
break
cap.release()
cv2.destroyAllWindows()
```
---
## Parameters
```python
from uniface.tracking import BYTETracker
tracker = BYTETracker(
track_thresh=0.5,
track_buffer=30,
match_thresh=0.8,
low_thresh=0.1,
)
```
| Parameter | Default | Description |
|-----------|---------|-------------|
| `track_thresh` | 0.5 | Detections above this score go through first-pass association |
| `track_buffer` | 30 | How many frames to keep a lost track before removing it |
| `match_thresh` | 0.8 | IoU threshold for matching tracks to detections |
| `low_thresh` | 0.1 | Detections below this score are discarded entirely |
---
## Input / Output
**Input**`(N, 5)` numpy array with `[x1, y1, x2, y2, confidence]` per detection:
```python
detections = np.array([
[100, 50, 200, 160, 0.95],
[300, 80, 380, 200, 0.87],
])
```
**Output**`(M, 5)` numpy array with `[x1, y1, x2, y2, track_id]` per active track:
```python
tracks = tracker.update(detections)
# array([[101.2, 51.3, 199.8, 159.8, 1.],
# [300.5, 80.2, 379.7, 200.1, 2.]])
```
The output bounding boxes come from the Kalman filter prediction, so they may differ slightly from the input. Track IDs are integers that persist across frames for the same object.
---
## Resetting the Tracker
When switching to a different video or scene, reset the tracker to clear all internal state:
```python
tracker.reset()
```
This clears all active, lost, and removed tracks, resets the frame counter, and resets the ID counter back to zero.
---
## Visualization
`draw_tracks` draws bounding boxes color-coded by track ID:
```python
from uniface.draw import draw_tracks
draw_tracks(
image=frame,
faces=tracked_faces,
draw_landmarks=True,
draw_id=True,
corner_bbox=True,
)
```
---
## Small Face Performance
!!! warning "Tracking performance with small faces"
The tracker relies on IoU (Intersection over Union) to match detections across
frames. When faces occupy a small portion of the image — for example in
surveillance footage or wide-angle cameras — even slight movement between frames
can cause a large drop in IoU. This makes it harder for the tracker to maintain
consistent IDs, and you may see IDs switching or resetting more often than expected.
This is not specific to BYTETracker; it applies to any IoU-based tracker. A few
things that can help:
- **Lower `match_thresh`** (e.g. `0.5` or `0.6`) so the tracker accepts lower
overlap as a valid match.
- **Increase `track_buffer`** (e.g. `60` or higher) to hold onto lost tracks
longer before discarding them.
- **Use a higher-resolution input** if possible, so face bounding boxes are
larger in pixel terms.
```python
tracker = BYTETracker(
track_thresh=0.4,
track_buffer=60,
match_thresh=0.6,
)
```
---
## References
- [yakhyo/bytetrack-tracker](https://github.com/yakhyo/bytetrack-tracker) — standalone BYTETracker implementation used in UniFace
- [ByteTrack paper](https://arxiv.org/abs/2110.06864) — Zhang et al., "ByteTrack: Multi-Object Tracking by Associating Every Detection Box"
---
## See Also
- [Detection](detection.md) — face detection models
- [Video & Webcam](../recipes/video-webcam.md) — video processing patterns
- [Inputs & Outputs](../concepts/inputs-outputs.md) — data types and formats
- [CLI Tools](https://github.com/yakhyo/uniface/blob/main/tools/README.md) — command-line scripts for all UniFace modules

View File

@@ -12,10 +12,17 @@ Run UniFace examples directly in your browser with Google Colab, or download and
| [Face Alignment](https://github.com/yakhyo/uniface/blob/main/examples/02_face_alignment.ipynb) | [![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/yakhyo/uniface/blob/main/examples/02_face_alignment.ipynb) | Align faces for recognition |
| [Face Verification](https://github.com/yakhyo/uniface/blob/main/examples/03_face_verification.ipynb) | [![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/yakhyo/uniface/blob/main/examples/03_face_verification.ipynb) | Compare faces for identity |
| [Face Search](https://github.com/yakhyo/uniface/blob/main/examples/04_face_search.ipynb) | [![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/yakhyo/uniface/blob/main/examples/04_face_search.ipynb) | Find a person in group photos |
| [Face Analyzer](https://github.com/yakhyo/uniface/blob/main/examples/05_face_analyzer.ipynb) | [![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/yakhyo/uniface/blob/main/examples/05_face_analyzer.ipynb) | All-in-one face analysis |
| [Face Analyzer](https://github.com/yakhyo/uniface/blob/main/examples/05_face_analyzer.ipynb) | [![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/yakhyo/uniface/blob/main/examples/05_face_analyzer.ipynb) | Unified face analysis |
| [Face Parsing](https://github.com/yakhyo/uniface/blob/main/examples/06_face_parsing.ipynb) | [![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/yakhyo/uniface/blob/main/examples/06_face_parsing.ipynb) | Semantic face segmentation |
| [Face Anonymization](https://github.com/yakhyo/uniface/blob/main/examples/07_face_anonymization.ipynb) | [![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/yakhyo/uniface/blob/main/examples/07_face_anonymization.ipynb) | Privacy-preserving blur |
| [Gaze Estimation](https://github.com/yakhyo/uniface/blob/main/examples/08_gaze_estimation.ipynb) | [![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/yakhyo/uniface/blob/main/examples/08_gaze_estimation.ipynb) | Gaze direction estimation |
| [Face Segmentation](https://github.com/yakhyo/uniface/blob/main/examples/09_face_segmentation.ipynb) | [![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/yakhyo/uniface/blob/main/examples/09_face_segmentation.ipynb) | Face segmentation with XSeg |
| [Face Vector Store](https://github.com/yakhyo/uniface/blob/main/examples/10_face_vector_store.ipynb) | [![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/yakhyo/uniface/blob/main/examples/10_face_vector_store.ipynb) | FAISS-backed face database |
| [Head Pose Estimation](https://github.com/yakhyo/uniface/blob/main/examples/11_head_pose_estimation.ipynb) | [![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/yakhyo/uniface/blob/main/examples/11_head_pose_estimation.ipynb) | 3D head orientation estimation |
| [Face Recognition](https://github.com/yakhyo/uniface/blob/main/examples/12_face_recognition.ipynb) | [![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/yakhyo/uniface/blob/main/examples/12_face_recognition.ipynb) | Standalone face recognition pipeline |
| [Portrait Matting](https://github.com/yakhyo/uniface/blob/main/examples/13_portrait_matting.ipynb) | [![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/yakhyo/uniface/blob/main/examples/13_portrait_matting.ipynb) | Portrait matting with MODNet |
| [Face Attributes](https://github.com/yakhyo/uniface/blob/main/examples/14_face_attributes.ipynb) | [![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/yakhyo/uniface/blob/main/examples/14_face_attributes.ipynb) | Face states (eyes, glasses, sunglasses, mask) |
| [Face Mesh](https://github.com/yakhyo/uniface/blob/main/examples/15_face_mesh.ipynb) | [![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/yakhyo/uniface/blob/main/examples/15_face_mesh.ipynb) | 468 / 478-point dense 3D mesh |
---
@@ -29,7 +36,7 @@ git clone https://github.com/yakhyo/uniface.git
cd uniface
# Install dependencies
pip install uniface jupyter
pip install "uniface[cpu]" jupyter # or uniface[gpu] for CUDA
# Launch Jupyter
jupyter notebook examples/

7
docs/overrides/main.html Normal file
View File

@@ -0,0 +1,7 @@
{% extends "base.html" %}
{% block announce %}
<a href="https://github.com/yakhyo/uniface" target="_blank" rel="noopener">
Support our work &mdash; give UniFace a <span class="twemoji">{% include ".icons/octicons/star-fill-16.svg" %}</span> on <strong>GitHub</strong> and help us reach more developers!
</a>
{% endblock %}

View File

@@ -10,7 +10,7 @@ Detect faces in an image:
```python
import cv2
from uniface import RetinaFace
from uniface.detection import RetinaFace
# Load image
image = cv2.imread("photo.jpg")
@@ -46,27 +46,16 @@ Draw bounding boxes and landmarks:
```python
import cv2
from uniface import RetinaFace
from uniface.visualization import draw_detections
from uniface.detection import RetinaFace
from uniface.draw import draw_detections
# Detect faces
detector = RetinaFace()
image = cv2.imread("photo.jpg")
faces = detector.detect(image)
# Extract visualization data
bboxes = [f.bbox for f in faces]
scores = [f.confidence for f in faces]
landmarks = [f.landmarks for f in faces]
# Draw on image
draw_detections(
image=image,
bboxes=bboxes,
scores=scores,
landmarks=landmarks,
vis_threshold=0.6,
)
draw_detections(image=image, faces=faces, vis_threshold=0.6)
# Save result
cv2.imwrite("output.jpg", image)
@@ -80,8 +69,8 @@ Compare two faces:
```python
import cv2
import numpy as np
from uniface import RetinaFace, ArcFace
from uniface.detection import RetinaFace
from uniface.recognition import ArcFace
# Initialize models
detector = RetinaFace()
@@ -96,12 +85,13 @@ faces1 = detector.detect(image1)
faces2 = detector.detect(image2)
if faces1 and faces2:
# Extract embeddings
# Extract embeddings (normalized 1-D vectors)
emb1 = recognizer.get_normalized_embedding(image1, faces1[0].landmarks)
emb2 = recognizer.get_normalized_embedding(image2, faces2[0].landmarks)
# Compute similarity (cosine similarity)
similarity = np.dot(emb1, emb2.T)[0][0]
# Compute cosine similarity
from uniface import compute_similarity
similarity = compute_similarity(emb1, emb2, normalized=True)
# Interpret result
if similarity > 0.6:
@@ -121,7 +111,8 @@ if faces1 and faces2:
```python
import cv2
from uniface import RetinaFace, AgeGender
from uniface.attribute import AgeGender
from uniface.detection import RetinaFace
# Initialize models
detector = RetinaFace()
@@ -133,7 +124,7 @@ faces = detector.detect(image)
# Predict attributes
for i, face in enumerate(faces):
result = age_gender.predict(image, face.bbox)
result = age_gender.predict(image, face)
print(f"Face {i+1}: {result.sex}, {result.age} years old")
```
@@ -152,7 +143,8 @@ Detect race, gender, and age group:
```python
import cv2
from uniface import RetinaFace, FairFace
from uniface.attribute import FairFace
from uniface.detection import RetinaFace
detector = RetinaFace()
fairface = FairFace()
@@ -161,7 +153,7 @@ image = cv2.imread("photo.jpg")
faces = detector.detect(image)
for i, face in enumerate(faces):
result = fairface.predict(image, face.bbox)
result = fairface.predict(image, face)
print(f"Face {i+1}: {result.sex}, {result.age_group}, {result.race}")
```
@@ -174,21 +166,24 @@ Face 2: Female, 20-29, White
---
## Facial Landmarks (106 Points)
## Facial Landmarks (106 / 98 / 68 / 468 / 478 Points)
UniFace ships three dense-landmark families. Pick whichever fits your downstream task:
```python
import cv2
from uniface import RetinaFace, Landmark106
from uniface.detection import RetinaFace
from uniface.landmark import Landmark106
detector = RetinaFace()
landmarker = Landmark106()
landmarker = Landmark106() # 106-point InsightFace 2d106det model
image = cv2.imread("photo.jpg")
faces = detector.detect(image)
if faces:
landmarks = landmarker.get_landmarks(image, faces[0].bbox)
print(f"Detected {len(landmarks)} landmarks")
print(f"Detected {len(landmarks)} landmarks") # 106
# Draw landmarks
for x, y in landmarks.astype(int):
@@ -197,6 +192,35 @@ if faces:
cv2.imwrite("landmarks.jpg", image)
```
**PIPNet (98 / 68 points)** — ResNet-18 backbone trained on WFLW (98 pts) or 300W+CelebA (68 pts):
```python
from uniface.constants import PIPNetWeights
from uniface.landmark import PIPNet
# 98-point WFLW model (default)
landmarker_98 = PIPNet()
# 68-point 300W+CelebA model
landmarker_68 = PIPNet(model_name=PIPNetWeights.DW300_CELEBA_68)
landmarks = landmarker_98.get_landmarks(image, faces[0].bbox) # (98, 2)
```
**Face Mesh (468 / 478 points, 3D)** is MediaPipe's dense mesh, run over every detected face
in a single batched call. Pass `FaceMeshWeights.V2_478` for the 478-point variant with irises:
```python
from uniface.landmark import FaceMesh
mesher = FaceMesh() # default: 468 points
results = mesher.predict(image, faces)
print(results[0].landmarks.shape) # (468, 3), x/y in image pixels, z is relative depth
print(results[0].points_2d.shape) # (468, 2), depth dropped
print(results[0].score) # face presence, [0, 1]
```
---
## Gaze Estimation
@@ -204,8 +228,9 @@ if faces:
```python
import cv2
import numpy as np
from uniface import RetinaFace, MobileGaze
from uniface.visualization import draw_gaze
from uniface.detection import RetinaFace
from uniface.gaze import MobileGaze
from uniface.draw import draw_gaze
detector = RetinaFace()
gaze_estimator = MobileGaze()
@@ -229,6 +254,36 @@ cv2.imwrite("gaze_output.jpg", image)
---
## Head Pose Estimation
```python
import cv2
from uniface.detection import RetinaFace
from uniface.headpose import HeadPose
from uniface.draw import draw_head_pose
detector = RetinaFace()
head_pose = HeadPose()
image = cv2.imread("photo.jpg")
faces = detector.detect(image)
for i, face in enumerate(faces):
x1, y1, x2, y2 = map(int, face.bbox[:4])
face_crop = image[y1:y2, x1:x2]
if face_crop.size > 0:
result = head_pose.estimate(face_crop)
print(f"Face {i+1}: pitch={result.pitch:.1f}°, yaw={result.yaw:.1f}°, roll={result.roll:.1f}°")
# Draw 3D cube visualization
draw_head_pose(image, face.bbox, result.pitch, result.yaw, result.roll)
cv2.imwrite("headpose_output.jpg", image)
```
---
## Face Parsing
Segment face into semantic components:
@@ -237,7 +292,7 @@ Segment face into semantic components:
import cv2
import numpy as np
from uniface.parsing import BiSeNet
from uniface.visualization import vis_parsing_maps
from uniface.draw import vis_parsing_maps
parser = BiSeNet()
@@ -247,40 +302,65 @@ face_image = cv2.imread("face.jpg")
# Parse face into 19 components
mask = parser.parse(face_image)
# Visualize with overlay
face_rgb = cv2.cvtColor(face_image, cv2.COLOR_BGR2RGB)
vis_result = vis_parsing_maps(face_rgb, mask, save_image=False)
# Visualize with overlay (BGR in, BGR out — same convention as cv2)
vis_result = vis_parsing_maps(face_image, mask, save_image=False)
print(f"Detected {len(np.unique(mask))} facial components")
```
---
## Portrait Matting
Remove backgrounds without a trimap:
```python
import cv2
import numpy as np
from uniface.matting import MODNet
matting = MODNet()
image = cv2.imread("portrait.jpg")
matte = matting.predict(image) # (H, W) float32 in [0, 1]
# Transparent PNG
rgba = cv2.cvtColor(image, cv2.COLOR_BGR2BGRA)
rgba[:, :, 3] = (matte * 255).astype(np.uint8)
cv2.imwrite("transparent.png", rgba)
# Green screen
matte_3ch = matte[:, :, np.newaxis]
bg = np.full_like(image, (0, 177, 64), dtype=np.uint8)
result = (image * matte_3ch + bg * (1 - matte_3ch)).astype(np.uint8)
cv2.imwrite("green_screen.jpg", result)
```
---
## Face Anonymization
Blur faces for privacy protection:
```python
from uniface.privacy import anonymize_faces
import cv2
# One-liner: automatic detection and blurring
image = cv2.imread("group_photo.jpg")
anonymized = anonymize_faces(image, method='pixelate')
cv2.imwrite("anonymized.jpg", anonymized)
```
**Manual control:**
```python
from uniface import RetinaFace
from uniface.detection import RetinaFace
from uniface.privacy import BlurFace
detector = RetinaFace()
blurrer = BlurFace(method='gaussian', blur_strength=5.0)
blurrer = BlurFace(method='pixelate')
image = cv2.imread("group_photo.jpg")
faces = detector.detect(image)
anonymized = blurrer.anonymize(image, faces)
cv2.imwrite("anonymized.jpg", anonymized)
```
**Custom blur settings:**
```python
blurrer = BlurFace(method='gaussian', blur_strength=5.0)
anonymized = blurrer.anonymize(image, faces)
```
**Available methods:**
@@ -301,7 +381,7 @@ Detect real vs. fake faces:
```python
import cv2
from uniface import RetinaFace
from uniface.detection import RetinaFace
from uniface.spoofing import MiniFASNet
detector = RetinaFace()
@@ -318,14 +398,38 @@ for i, face in enumerate(faces):
---
## Face Image Quality Assessment
Score how usable each face is for downstream recognition:
```python
import cv2
from uniface.detection import SCRFD
from uniface.quality import EDifFIQA
detector = SCRFD(confidence_threshold=0.3)
quality = EDifFIQA()
image = cv2.imread("photo.jpg")
faces = detector.detect(image)
for i, face in enumerate(faces):
result = quality.predict(image, face.landmarks)
print(f"Face {i+1}: quality={result.score:.4f}")
```
Higher = better. Use it to filter or rank faces before recognition.
---
## Webcam Demo
Real-time face detection:
```python
import cv2
from uniface import RetinaFace
from uniface.visualization import draw_detections
from uniface.detection import RetinaFace
from uniface.draw import draw_detections
detector = RetinaFace()
cap = cv2.VideoCapture(0)
@@ -339,10 +443,7 @@ while True:
faces = detector.detect(frame)
bboxes = [f.bbox for f in faces]
scores = [f.confidence for f in faces]
landmarks = [f.landmarks for f in faces]
draw_detections(image=frame, bboxes=bboxes, scores=scores, landmarks=landmarks)
draw_detections(image=frame, faces=faces)
cv2.imshow("UniFace - Press 'q' to quit", frame)
@@ -355,20 +456,95 @@ cv2.destroyAllWindows()
---
## Face Tracking
Track faces across video frames with persistent IDs:
```python
import cv2
import numpy as np
from uniface.common import xyxy_to_cxcywh
from uniface.detection import SCRFD
from uniface.tracking import BYTETracker
from uniface.draw import draw_tracks
detector = SCRFD()
tracker = BYTETracker(track_thresh=0.5, track_buffer=30)
cap = cv2.VideoCapture("video.mp4")
while cap.isOpened():
ret, frame = cap.read()
if not ret:
break
faces = detector.detect(frame)
dets = np.array([[*f.bbox, f.confidence] for f in faces])
dets = dets if len(dets) > 0 else np.empty((0, 5))
tracks = tracker.update(dets)
# Assign track IDs to faces
if len(tracks) > 0 and len(faces) > 0:
face_bboxes = np.array([f.bbox for f in faces], dtype=np.float32)
track_ids = tracks[:, 4].astype(int)
face_centers = xyxy_to_cxcywh(face_bboxes)[:, :2]
track_centers = xyxy_to_cxcywh(tracks[:, :4])[:, :2]
for ti in range(len(tracks)):
dists = (track_centers[ti, 0] - face_centers[:, 0]) ** 2 + (track_centers[ti, 1] - face_centers[:, 1]) ** 2
faces[int(np.argmin(dists))].track_id = track_ids[ti]
tracked_faces = [f for f in faces if f.track_id is not None]
draw_tracks(image=frame, faces=tracked_faces)
cv2.imshow("Tracking", frame)
if cv2.waitKey(1) & 0xFF == ord('q'):
break
cap.release()
cv2.destroyAllWindows()
```
For more details, see the [Tracking module](modules/tracking.md).
---
## Model Selection
For detailed model comparisons, benchmarks, and selection guidance, see the [Model Zoo](models.md).
For detailed model comparisons and benchmarks, see the [Model Zoo](models.md).
**Quick recommendations:**
**Available models by task:**
| Task | Recommended Model | Alternative |
|------|-------------------|-------------|
| Detection (balanced) | `RetinaFace` (MNET_V2) | `YOLOv5Face` (YOLOV5S) |
| Detection (speed) | `RetinaFace` (MNET_025) | `SCRFD` (SCRFD_500M) |
| Detection (accuracy) | `SCRFD` (SCRFD_10G) | `RetinaFace` (RESNET34) |
| Recognition | `ArcFace` (MNET) | `MobileFace` (MNET_V2) |
| Gaze | `MobileGaze` (RESNET34) | `MobileGaze` (MOBILEONE_S0) |
| Parsing | `BiSeNet` (RESNET18) | `BiSeNet` (RESNET34) |
| Task | Available Models |
|------|------------------|
| Detection | `RetinaFace`, `SCRFD`, `CenterFace`, `YOLOv5Face`, `YOLOv8Face`, `BlazeFace` (short-range, 6 keypoints) |
| Recognition | `ArcFace`, `AdaFace`, `EdgeFace`, `MobileFace`, `SphereFace` |
| Landmarks | `Landmark106` (106 pts), `PIPNet` (98 / 68 pts), `FaceMesh` (468 or 478 pts, 3D) |
| Tracking | `BYTETracker` |
| Gaze | `MobileGaze` (ResNet18/34/50, MobileNetV2, MobileOneS0) |
| Head Pose | `HeadPose` (ResNet18/34/50, MobileNetV2/V3) |
| Parsing | `BiSeNet` (ResNet18/34), `XSeg` |
| Matting | `MODNet` |
| Attributes | `AgeGender`, `FairFace`, `Emotion`, `FaceAttribNet` (face states) |
| Anti-Spoofing | `MiniFASNet` (V1SE, V2) |
| Quality | `EDifFIQA` (T, S, M, L) |
| Privacy | `BlurFace` (5 blur methods) |
| Vector Store | `FAISS` |
---
## Verbose Logging
Enable logging to see what happens during model loading and inference (useful for debugging):
```python
import logging
from uniface import enable_logging
enable_logging() # INFO level
enable_logging(level=logging.DEBUG) # DEBUG level
```
---
@@ -407,13 +583,21 @@ python -c "import platform; print(platform.machine())"
### Import Errors
```python
# Correct imports
from uniface.detection import RetinaFace
from uniface.recognition import ArcFace
from uniface.landmark import Landmark106
# Also works (re-exported at package level)
from uniface import RetinaFace, ArcFace, Landmark106
from uniface.detection import BlazeFace, CenterFace, RetinaFace, SCRFD, YOLOv5Face, YOLOv8Face
from uniface.recognition import ArcFace, AdaFace
from uniface.attribute import AgeGender, Emotion, FaceAttribNet, FairFace
from uniface.landmark import FaceMesh, Landmark106, PIPNet
from uniface.gaze import MobileGaze
from uniface.headpose import HeadPose
from uniface.matting import MODNet
from uniface.parsing import BiSeNet, XSeg
from uniface.privacy import BlurFace
from uniface.quality import EDifFIQA
from uniface.spoofing import MiniFASNet
from uniface.tracking import BYTETracker
from uniface.analyzer import FaceAnalyzer
from uniface.stores import FAISS # pip install faiss-cpu
from uniface.draw import draw_detections, draw_tracks
```
---

View File

@@ -11,7 +11,7 @@ Blur faces in real-time video streams for privacy protection.
```python
import cv2
from uniface import RetinaFace
from uniface.detection import RetinaFace
from uniface.privacy import BlurFace
detector = RetinaFace()
@@ -40,7 +40,7 @@ cv2.destroyAllWindows()
```python
import cv2
from uniface import RetinaFace
from uniface.detection import RetinaFace
from uniface.privacy import BlurFace
detector = RetinaFace()
@@ -52,7 +52,7 @@ w, h = int(cap.get(3)), int(cap.get(4))
out = cv2.VideoWriter('output.mp4', cv2.VideoWriter_fourcc(*'mp4v'), fps, (w, h))
while cap.read()[0]:
while True:
ret, frame = cap.read()
if not ret:
break
@@ -67,14 +67,19 @@ out.release()
---
## One-Liner for Images
## Single Image
```python
from uniface.privacy import anonymize_faces
import cv2
from uniface.detection import RetinaFace
from uniface.privacy import BlurFace
detector = RetinaFace()
blurrer = BlurFace(method='pixelate')
image = cv2.imread("photo.jpg")
result = anonymize_faces(image, method='pixelate')
faces = detector.detect(image)
result = blurrer.anonymize(image, faces)
cv2.imwrite("anonymized.jpg", result)
```
@@ -84,7 +89,7 @@ cv2.imwrite("anonymized.jpg", result)
| Method | Usage |
|--------|-------|
| Pixelate | `BlurFace(method='pixelate', pixel_blocks=10)` |
| Pixelate | `BlurFace(method='pixelate', pixel_blocks=15)` |
| Gaussian | `BlurFace(method='gaussian', blur_strength=3.0)` |
| Blackout | `BlurFace(method='blackout', color=(0,0,0))` |
| Elliptical | `BlurFace(method='elliptical', margin=20)` |

View File

@@ -12,7 +12,7 @@ Process multiple images efficiently.
```python
import cv2
from pathlib import Path
from uniface import RetinaFace
from uniface.detection import RetinaFace
detector = RetinaFace()
@@ -54,7 +54,8 @@ for image_path in tqdm(image_files, desc="Processing"):
## Extract Embeddings
```python
from uniface import RetinaFace, ArcFace
from uniface.detection import RetinaFace
from uniface.recognition import ArcFace
import numpy as np
detector = RetinaFace()

View File

@@ -26,30 +26,41 @@ from uniface.types import Face
import numpy as np
class MyDetector(BaseDetector):
# Both flags default to False, so a boxes-only detector declares neither.
# Opt in if your detector fills Face.landmarks.
supports_landmarks = True
# Opt in only if those landmarks ARE the 5-point alignment template
# (left eye, right eye, nose, left mouth corner, right mouth corner).
# Left False, FaceAnalyzer disables recognition instead of producing
# broken embeddings.
supports_alignment = True
def __init__(self, model_path: str, confidence_threshold: float = 0.5):
super().__init__(confidence_threshold=confidence_threshold)
self.session = create_onnx_session(model_path)
self.threshold = confidence_threshold
def preprocess(self, image: np.ndarray) -> np.ndarray:
# Your preprocessing logic
# e.g., resize, normalize, transpose
raise NotImplementedError
def postprocess(self, outputs, shape) -> list[Face]:
# Your postprocessing logic
# e.g., decode boxes, apply NMS, create Face objects
raise NotImplementedError
def detect(self, image: np.ndarray) -> list[Face]:
# 1. Preprocess image
input_tensor = self._preprocess(image)
input_tensor = self.preprocess(image)
# 2. Run inference
outputs = self.session.run(None, {'input': input_tensor})
# 3. Postprocess outputs to Face objects
faces = self._postprocess(outputs, image.shape)
return faces
def _preprocess(self, image):
# Your preprocessing logic
# e.g., resize, normalize, transpose
pass
def _postprocess(self, outputs, shape):
# Your postprocessing logic
# e.g., decode boxes, apply NMS, create Face objects
pass
return self.postprocess(outputs, image.shape)
```
---
@@ -57,36 +68,48 @@ class MyDetector(BaseDetector):
## Add Custom Recognition Model
```python
from uniface.recognition.base import BaseRecognizer
from uniface.onnx_utils import create_onnx_session
from uniface import face_alignment
import numpy as np
from uniface.recognition.base import BaseRecognizer, PreprocessConfig
class MyRecognizer(BaseRecognizer):
def __init__(self, model_path: str):
self.session = create_onnx_session(model_path)
def __init__(self, model_path: str, providers=None):
preprocessing = PreprocessConfig(input_mean=127.5, input_std=127.5, input_size=(112, 112))
super().__init__(model_path=model_path, preprocessing=preprocessing, providers=providers)
def get_normalized_embedding(
self,
image: np.ndarray,
landmarks: np.ndarray
) -> np.ndarray:
# 1. Align face
aligned = face_alignment(image, landmarks)
# Optional: override preprocess() if your model expects custom normalization.
```
# 2. Preprocess
input_tensor = self._preprocess(aligned)
---
# 3. Run inference
embedding = self.session.run(None, {'input': input_tensor})[0]
## Add Custom Per-Face Predictor
# 4. Normalize
embedding = embedding / np.linalg.norm(embedding)
return embedding
`FaceAnalyzer` runs any `BaseAttribute` subclass on each detected face via the
`predictors=` list. Implement `predict(image, face)` to read what you need from
the `Face` (bbox, landmarks), run inference, and write results back:
def _preprocess(self, image):
# Your preprocessing logic
pass
```python
from uniface.attribute import BaseAttribute
class MyPredictor(BaseAttribute):
def _initialize_model(self):
... # load your model
def preprocess(self, image, *args):
... # crop and normalize
def postprocess(self, prediction):
... # raw output to a result object
def predict(self, image, face):
result = self.postprocess(self._run(self.preprocess(image, face.bbox)))
face.age = result.age # enrich the Face in-place
return result
```
```python
from uniface import FaceAnalyzer
analyzer = FaceAnalyzer(predictors=[MyPredictor()])
faces = analyzer.analyze(image)
```
---

View File

@@ -1,178 +1,166 @@
# Face Search
Build a face search system for finding people in images.
Find and identify people in images and video streams.
!!! note "Work in Progress"
This page contains example code patterns. Test thoroughly before using in production.
UniFace supports two search approaches:
| Approach | Use case | Tool |
| -------------------- | ------------------------------------------------ | ----------------------- |
| **Reference search** | "Is this specific person in the video?" | `tools/search.py` |
| **Vector search** | "Who is this?" against a database of known faces | `tools/faiss_search.py` |
---
## Basic Face Database
## Reference Search (single image)
Compare every detected face against a single reference photo:
```python
import cv2
import numpy as np
from uniface.detection import RetinaFace
from uniface.recognition import ArcFace
from uniface.face_utils import compute_similarity
detector = RetinaFace()
recognizer = ArcFace()
ref_image = cv2.imread("reference.jpg")
ref_faces = detector.detect(ref_image)
ref_embedding = recognizer.get_normalized_embedding(ref_image, ref_faces[0].landmarks)
query_image = cv2.imread("group_photo.jpg")
faces = detector.detect(query_image)
for face in faces:
embedding = recognizer.get_normalized_embedding(query_image, face.landmarks)
sim = compute_similarity(ref_embedding, embedding)
label = f"Match ({sim:.2f})" if sim > 0.4 else f"Unknown ({sim:.2f})"
print(label)
```
**CLI tool:**
```bash
python tools/search.py --reference ref.jpg --source video.mp4
python tools/search.py --reference ref.jpg --source 0 # webcam
```
---
## Vector Search (FAISS index)
For identifying faces against a database of many known people, use the
[`FAISS`](../modules/stores.md) vector store.
!!! info "Install extra"
`bash
pip install faiss-cpu
`
### Build an index
Organise face images in person sub-folders:
```
dataset/
├── alice/
│ ├── 001.jpg
│ └── 002.jpg
├── bob/
│ └── 001.jpg
└── charlie/
├── 001.jpg
└── 002.jpg
```
```python
import cv2
from pathlib import Path
from uniface import RetinaFace, ArcFace
from uniface.detection import RetinaFace
from uniface.recognition import ArcFace
from uniface.stores import FAISS
class FaceDatabase:
def __init__(self):
self.detector = RetinaFace()
self.recognizer = ArcFace()
self.embeddings = {}
detector = RetinaFace()
recognizer = ArcFace()
store = FAISS(db_path="./my_index")
def add_face(self, person_id, image):
"""Add a face to the database."""
faces = self.detector.detect(image)
if not faces:
raise ValueError(f"No face found for {person_id}")
for person_dir in sorted(Path("dataset").iterdir()):
if not person_dir.is_dir():
continue
for img_path in person_dir.glob("*.jpg"):
image = cv2.imread(str(img_path))
faces = detector.detect(image)
if faces:
emb = recognizer.get_normalized_embedding(image, faces[0].landmarks)
store.add(emb, {"person_id": person_dir.name, "source": str(img_path)})
face = max(faces, key=lambda f: f.confidence)
embedding = self.recognizer.get_normalized_embedding(image, face.landmarks)
self.embeddings[person_id] = embedding
return True
def search(self, image, threshold=0.6):
"""Search for faces in an image."""
faces = self.detector.detect(image)
results = []
for face in faces:
embedding = self.recognizer.get_normalized_embedding(image, face.landmarks)
best_match = None
best_similarity = -1
for person_id, db_embedding in self.embeddings.items():
similarity = np.dot(embedding, db_embedding.T)[0][0]
if similarity > best_similarity:
best_similarity = similarity
best_match = person_id
results.append({
'bbox': face.bbox,
'match': best_match if best_similarity >= threshold else None,
'similarity': best_similarity
})
return results
def save(self, path):
"""Save database to file."""
np.savez(path, embeddings=dict(self.embeddings))
def load(self, path):
"""Load database from file."""
data = np.load(path, allow_pickle=True)
self.embeddings = data['embeddings'].item()
# Usage
db = FaceDatabase()
# Add faces
for image_path in Path("known_faces/").glob("*.jpg"):
person_id = image_path.stem
image = cv2.imread(str(image_path))
try:
db.add_face(person_id, image)
print(f"Added: {person_id}")
except ValueError as e:
print(f"Skipped: {e}")
# Save database
db.save("face_database.npz")
# Search
query_image = cv2.imread("group_photo.jpg")
results = db.search(query_image)
for r in results:
if r['match']:
print(f"Found: {r['match']} (similarity: {r['similarity']:.3f})")
store.save()
print(f"Index saved: {store}")
```
---
**CLI tool:**
## Visualization
```bash
python tools/faiss_search.py build --faces-dir dataset/ --db-path ./my_index
```
### Search against the index
```python
import cv2
from uniface.detection import RetinaFace
from uniface.recognition import ArcFace
from uniface.stores import FAISS
def visualize_search_results(image, results):
"""Draw search results on image."""
for r in results:
x1, y1, x2, y2 = map(int, r['bbox'])
detector = RetinaFace()
recognizer = ArcFace()
if r['match']:
color = (0, 255, 0) # Green for match
label = f"{r['match']} ({r['similarity']:.2f})"
else:
color = (0, 0, 255) # Red for unknown
label = f"Unknown ({r['similarity']:.2f})"
store = FAISS(db_path="./my_index")
store.load()
cv2.rectangle(image, (x1, y1), (x2, y2), color, 2)
cv2.putText(image, label, (x1, y1 - 10),
cv2.FONT_HERSHEY_SIMPLEX, 0.6, color, 2)
image = cv2.imread("query.jpg")
faces = detector.detect(image)
return image
for face in faces:
embedding = recognizer.get_normalized_embedding(image, face.landmarks)
result, similarity = store.search(embedding, threshold=0.4)
# Usage
results = db.search(image)
annotated = visualize_search_results(image.copy(), results)
cv2.imwrite("search_result.jpg", annotated)
if result:
print(f"Matched: {result['person_id']} ({similarity:.2f})")
else:
print(f"Unknown ({similarity:.2f})")
```
---
**CLI tool:**
## Real-Time Search
```bash
python tools/faiss_search.py run --db-path ./my_index --source video.mp4
python tools/faiss_search.py run --db-path ./my_index --source 0 # webcam
```
### Manage the index
```python
import cv2
from uniface.stores import FAISS
def realtime_search(db):
"""Real-time face search from webcam."""
cap = cv2.VideoCapture(0)
store = FAISS(db_path="./my_index")
store.load()
while True:
ret, frame = cap.read()
if not ret:
break
print(f"Total vectors: {len(store)}")
results = db.search(frame, threshold=0.5)
removed = store.remove("person_id", "bob")
print(f"Removed {removed} entries")
for r in results:
x1, y1, x2, y2 = map(int, r['bbox'])
if r['match']:
color = (0, 255, 0)
label = r['match']
else:
color = (0, 0, 255)
label = "Unknown"
cv2.rectangle(frame, (x1, y1), (x2, y2), color, 2)
cv2.putText(frame, label, (x1, y1 - 10),
cv2.FONT_HERSHEY_SIMPLEX, 0.6, color, 2)
cv2.imshow("Face Search", frame)
if cv2.waitKey(1) & 0xFF == ord('q'):
break
cap.release()
cv2.destroyAllWindows()
# Usage
db = FaceDatabase()
db.load("face_database.npz")
realtime_search(db)
store.save()
```
---
## See Also
- [Stores Module](../modules/stores.md) - Full `FAISS` API reference
- [Recognition Module](../modules/recognition.md) - Face recognition details
- [Batch Processing](batch-processing.md) - Process multiple files
- [Video & Webcam](video-webcam.md) - Real-time processing
- [Concepts: Thresholds](../concepts/thresholds-calibration.md) - Tuning similarity thresholds

View File

@@ -8,8 +8,10 @@ A complete pipeline for processing images with detection, recognition, and attri
```python
import cv2
from uniface import RetinaFace, ArcFace, AgeGender
from uniface.visualization import draw_detections
from uniface.attribute import AgeGender
from uniface.detection import RetinaFace
from uniface.recognition import ArcFace
from uniface.draw import draw_detections
# Initialize models
detector = RetinaFace()
@@ -32,7 +34,7 @@ def process_image(image_path):
embedding = recognizer.get_normalized_embedding(image, face.landmarks)
# Step 3: Predict attributes
attrs = age_gender.predict(image, face.bbox)
attrs = age_gender.predict(image, face)
results.append({
'face_id': i,
@@ -46,12 +48,7 @@ def process_image(image_path):
print(f" Face {i+1}: {attrs.sex}, {attrs.age} years old")
# Visualize
draw_detections(
image=image,
bboxes=[f.bbox for f in faces],
scores=[f.confidence for f in faces],
landmarks=[f.landmarks for f in faces]
)
draw_detections(image=image, faces=faces)
return image, results
@@ -67,14 +64,21 @@ cv2.imwrite("result.jpg", result_image)
For convenience, use the built-in `FaceAnalyzer`:
```python
from uniface import FaceAnalyzer
from uniface.analyzer import FaceAnalyzer
from uniface.attribute import AgeGender
from uniface.detection import RetinaFace
from uniface.recognition import ArcFace
import cv2
# Initialize with desired modules
detector = RetinaFace()
recognizer = ArcFace()
age_gender = AgeGender()
analyzer = FaceAnalyzer(
detect=True,
recognize=True,
attributes=True
detector=detector,
recognizer=recognizer,
predictors=[age_gender],
)
# Process image
@@ -97,13 +101,15 @@ Complete pipeline with all modules:
```python
import cv2
import numpy as np
from uniface import (
RetinaFace, ArcFace, AgeGender, FairFace,
Landmark106, MobileGaze
)
from uniface.attribute import AgeGender, FairFace
from uniface.detection import RetinaFace
from uniface.gaze import MobileGaze
from uniface.headpose import HeadPose
from uniface.landmark import Landmark106
from uniface.recognition import ArcFace
from uniface.parsing import BiSeNet
from uniface.spoofing import MiniFASNet
from uniface.visualization import draw_detections, draw_gaze
from uniface.draw import draw_detections, draw_gaze, draw_head_pose
class FaceAnalysisPipeline:
def __init__(self):
@@ -114,6 +120,7 @@ class FaceAnalysisPipeline:
self.fairface = FairFace()
self.landmarker = Landmark106()
self.gaze = MobileGaze()
self.head_pose = HeadPose()
self.parser = BiSeNet()
self.spoofer = MiniFASNet()
@@ -135,12 +142,12 @@ class FaceAnalysisPipeline:
)
# Attributes
ag_result = self.age_gender.predict(image, face.bbox)
ag_result = self.age_gender.predict(image, face)
result['age'] = ag_result.age
result['gender'] = ag_result.sex
# FairFace attributes
ff_result = self.fairface.predict(image, face.bbox)
ff_result = self.fairface.predict(image, face)
result['age_group'] = ff_result.age_group
result['race'] = ff_result.race
@@ -157,6 +164,13 @@ class FaceAnalysisPipeline:
result['gaze_pitch'] = gaze_result.pitch
result['gaze_yaw'] = gaze_result.yaw
# Head pose estimation
if face_crop.size > 0:
hp_result = self.head_pose.estimate(face_crop)
result['head_pitch'] = hp_result.pitch
result['head_yaw'] = hp_result.yaw
result['head_roll'] = hp_result.roll
# Face parsing
if face_crop.size > 0:
result['parsing_mask'] = self.parser.parse(face_crop)
@@ -179,6 +193,7 @@ for i, r in enumerate(results):
print(f" Gender: {r['gender']}, Age: {r['age']}")
print(f" Race: {r['race']}, Age Group: {r['age_group']}")
print(f" Gaze: pitch={np.degrees(r['gaze_pitch']):.1f}°")
print(f" Head Pose: P={r['head_pitch']:.1f}° Y={r['head_yaw']:.1f}° R={r['head_roll']:.1f}°")
print(f" Real: {r['is_real']} ({r['spoof_confidence']:.1%})")
```
@@ -189,8 +204,10 @@ for i, r in enumerate(results):
```python
import cv2
import numpy as np
from uniface import RetinaFace, AgeGender, MobileGaze
from uniface.visualization import draw_detections, draw_gaze
from uniface.attribute import AgeGender
from uniface.detection import RetinaFace
from uniface.gaze import MobileGaze
from uniface.draw import draw_detections, draw_gaze
def visualize_analysis(image_path, output_path):
"""Create annotated visualization of face analysis."""
@@ -208,7 +225,7 @@ def visualize_analysis(image_path, output_path):
cv2.rectangle(image, (x1, y1), (x2, y2), (0, 255, 0), 2)
# Age and gender
attrs = age_gender.predict(image, face.bbox)
attrs = age_gender.predict(image, face)
label = f"{attrs.sex}, {attrs.age}y"
cv2.putText(image, label, (x1, y1 - 10),
cv2.FONT_HERSHEY_SIMPLEX, 0.6, (0, 255, 0), 2)
@@ -256,6 +273,11 @@ def results_to_json(results):
'gaze': {
'pitch_deg': float(np.degrees(r['gaze_pitch'])) if 'gaze_pitch' in r else None,
'yaw_deg': float(np.degrees(r['gaze_yaw'])) if 'gaze_yaw' in r else None
},
'head_pose': {
'pitch': float(r['head_pitch']) if 'head_pitch' in r else None,
'yaw': float(r['head_yaw']) if 'head_yaw' in r else None,
'roll': float(r['head_roll']) if 'head_roll' in r else None
}
}
output.append(item)
@@ -279,3 +301,4 @@ with open('results.json', 'w') as f:
- [Face Search](face-search.md) - Build a search system
- [Detection Module](../modules/detection.md) - Detection options
- [Recognition Module](../modules/recognition.md) - Recognition details
- [Head Pose Module](../modules/headpose.md) - Head orientation estimation

View File

@@ -11,8 +11,8 @@ Real-time face analysis for video streams.
```python
import cv2
from uniface import RetinaFace
from uniface.visualization import draw_detections
from uniface.detection import RetinaFace
from uniface.draw import draw_detections
detector = RetinaFace()
cap = cv2.VideoCapture(0)
@@ -26,12 +26,7 @@ while True:
faces = detector.detect(frame)
draw_detections(
image=frame,
bboxes=[f.bbox for f in faces],
scores=[f.confidence for f in faces],
landmarks=[f.landmarks for f in faces]
)
draw_detections(image=frame, faces=faces)
cv2.imshow("Face Detection", frame)
@@ -48,7 +43,7 @@ cv2.destroyAllWindows()
```python
import cv2
from uniface import RetinaFace
from uniface.detection import RetinaFace
def process_video(input_path, output_path):
"""Process a video file."""
@@ -64,7 +59,7 @@ def process_video(input_path, output_path):
fourcc = cv2.VideoWriter_fourcc(*'mp4v')
out = cv2.VideoWriter(output_path, fourcc, fps, (width, height))
while cap.read()[0]:
while True:
ret, frame = cap.read()
if not ret:
break
@@ -83,6 +78,57 @@ process_video("input.mp4", "output.mp4")
---
## Webcam Tracking
To track faces across frames with persistent IDs, pair a detector with `BYTETracker`:
```python
import cv2
import numpy as np
from uniface.common import xyxy_to_cxcywh
from uniface.detection import SCRFD
from uniface.tracking import BYTETracker
from uniface.draw import draw_tracks
detector = SCRFD()
tracker = BYTETracker(track_thresh=0.5, track_buffer=30)
cap = cv2.VideoCapture(0)
while True:
ret, frame = cap.read()
if not ret:
break
faces = detector.detect(frame)
dets = np.array([[*f.bbox, f.confidence] for f in faces])
dets = dets if len(dets) > 0 else np.empty((0, 5))
tracks = tracker.update(dets)
if len(tracks) > 0 and len(faces) > 0:
face_bboxes = np.array([f.bbox for f in faces], dtype=np.float32)
track_ids = tracks[:, 4].astype(int)
face_centers = xyxy_to_cxcywh(face_bboxes)[:, :2]
track_centers = xyxy_to_cxcywh(tracks[:, :4])[:, :2]
for ti in range(len(tracks)):
dists = (track_centers[ti, 0] - face_centers[:, 0]) ** 2 + (track_centers[ti, 1] - face_centers[:, 1]) ** 2
faces[int(np.argmin(dists))].track_id = track_ids[ti]
draw_tracks(image=frame, faces=[f for f in faces if f.track_id is not None])
cv2.imshow("Face Tracking", frame)
if cv2.waitKey(1) & 0xFF == ord('q'):
break
cap.release()
cv2.destroyAllWindows()
```
For more details on tracker parameters and tuning, see [Tracking](../modules/tracking.md).
---
## Performance Tips
### Skip Frames
@@ -119,7 +165,9 @@ while True:
## See Also
- [Tracking Module](../modules/tracking.md) - Face tracking with BYTETracker
- [Anonymize Stream](anonymize-stream.md) - Privacy protection in video
- [Batch Processing](batch-processing.md) - Process multiple files
- [Detection Module](../modules/detection.md) - Detection options
- [Gaze Module](../modules/gaze.md) - Gaze tracking
- [Gaze Module](../modules/gaze.md) - Gaze estimation
- [Head Pose Module](../modules/headpose.md) - Head orientation estimation

View File

@@ -1,5 +1,45 @@
/* UniFace Documentation - Custom Styles */
/* ===== Announcement Bar ===== */
/* Light theme: indigo gradient, white text. */
[data-md-color-scheme="default"] .md-banner {
background: linear-gradient(90deg, #3f51b5 0%, #7c4dff 100%);
color: #ffffff;
}
[data-md-color-scheme="default"] .md-banner a {
color: #ffffff;
text-decoration: none;
}
[data-md-color-scheme="default"] .md-banner a:hover {
text-decoration: underline;
}
[data-md-color-scheme="default"] .md-banner .twemoji svg {
fill: #ffd54f;
}
/* Dark theme: deeper purple gradient, soft gold star. */
[data-md-color-scheme="slate"] .md-banner {
background: linear-gradient(90deg, #4527a0 0%, #7c4dff 100%);
color: #ffffff;
}
[data-md-color-scheme="slate"] .md-banner a {
color: #ffffff;
text-decoration: none;
}
[data-md-color-scheme="slate"] .md-banner a:hover {
text-decoration: underline;
}
[data-md-color-scheme="slate"] .md-banner .twemoji svg {
fill: #ffd54f;
}
/* ===== Hero Section ===== */
.md-content .hero {
@@ -19,13 +59,6 @@
background-clip: text;
}
.hero-tagline {
font-size: 1.5rem;
color: var(--md-default-fg-color);
margin-bottom: 0.5rem !important;
font-weight: 500;
}
.hero-subtitle {
font-size: 1rem;
color: var(--md-default-fg-color--light);
@@ -36,34 +69,45 @@
.hero .md-button {
margin: 0.5rem 0.25rem;
padding: 0.7rem 1.5rem;
font-weight: 600;
border-radius: 8px;
transition: all 0.2s ease;
padding: 0.6rem 1.25rem;
font-weight: 500;
font-size: 0.9rem;
border-radius: 6px;
display: inline-flex;
align-items: center;
gap: 0.4rem;
transition: background-color 0.15s ease, border-color 0.15s ease, color 0.15s ease;
}
.hero .md-button .twemoji,
.hero .md-button svg {
width: 1em;
height: 1em;
fill: currentColor;
}
.hero .md-button--primary {
background: linear-gradient(135deg, var(--md-primary-fg-color) 0%, #5c6bc0 100%);
border: none;
box-shadow: 0 4px 14px rgba(63, 81, 181, 0.4);
background: var(--md-primary-fg-color);
border: 1px solid var(--md-primary-fg-color);
color: #ffffff;
}
.hero .md-button--primary:hover {
transform: translateY(-2px);
box-shadow: 0 6px 20px rgba(63, 81, 181, 0.5);
background: var(--md-primary-fg-color--dark, #303f9f);
border-color: var(--md-primary-fg-color--dark, #303f9f);
color: #ffffff;
}
.hero .md-button:not(.md-button--primary) {
border: 2px solid var(--md-primary-fg-color);
background: transparent;
color: var(--md-primary-fg-color);
border: 1px solid var(--md-default-fg-color--lighter);
color: var(--md-default-fg-color);
}
.hero .md-button:not(.md-button--primary):hover {
background: var(--md-primary-fg-color);
border-color: var(--md-primary-fg-color);
color: white;
transform: translateY(-2px);
background: var(--md-default-fg-color--lightest);
border-color: var(--md-default-fg-color--light);
color: var(--md-default-fg-color);
}
/* Badge styling in hero */
@@ -85,31 +129,21 @@
border-radius: 12px;
background: var(--md-code-bg-color);
border: 1px solid var(--md-default-fg-color--lightest);
transition: all 0.3s ease;
position: relative;
overflow: hidden;
}
.feature-card::before {
content: '';
position: absolute;
top: 0;
left: 0;
right: 0;
height: 3px;
background: linear-gradient(90deg, var(--md-primary-fg-color), #7c4dff);
opacity: 0;
transition: opacity 0.3s ease;
transition: border-color 0.2s ease;
}
.feature-card:hover {
transform: translateY(-4px);
box-shadow: 0 12px 24px rgba(0, 0, 0, 0.1);
border-color: var(--md-primary-fg-color--light);
border-color: var(--md-accent-fg-color);
}
.feature-card:hover::before {
opacity: 1;
/* Transparent overlay link — makes the whole card clickable. */
.feature-card-link {
position: absolute;
inset: 0;
z-index: 1;
border-radius: inherit;
text-decoration: none;
}
.feature-card h3 {
@@ -129,13 +163,6 @@
line-height: 1.5;
}
.feature-card a {
display: inline-block;
margin-top: 0.75rem;
font-weight: 500;
font-size: 0.875rem;
}
/* ===== Next Steps Grid (2 columns) ===== */
.next-steps-grid {
display: grid;
@@ -160,30 +187,28 @@
background-clip: text;
}
[data-md-color-scheme="slate"] .feature-card:hover {
box-shadow: 0 12px 24px rgba(0, 0, 0, 0.3);
}
[data-md-color-scheme="slate"] .hero .md-button--primary {
background: linear-gradient(135deg, #7c4dff 0%, #b388ff 100%);
box-shadow: 0 4px 14px rgba(124, 77, 255, 0.4);
background: #ffffff;
border-color: #ffffff;
color: #1a1a2e;
}
[data-md-color-scheme="slate"] .hero .md-button--primary:hover {
box-shadow: 0 6px 20px rgba(124, 77, 255, 0.5);
background: #e6e6e6;
border-color: #e6e6e6;
color: #1a1a2e;
}
[data-md-color-scheme="slate"] .hero .md-button:not(.md-button--primary) {
border: 2px solid rgba(255, 255, 255, 0.3);
background: rgba(255, 255, 255, 0.05);
background: transparent;
border: 1px solid rgba(255, 255, 255, 0.25);
color: rgba(255, 255, 255, 0.9);
}
[data-md-color-scheme="slate"] .hero .md-button:not(.md-button--primary):hover {
background: rgba(255, 255, 255, 0.1);
border-color: rgba(255, 255, 255, 0.5);
color: white;
transform: translateY(-2px);
background: rgba(255, 255, 255, 0.08);
border-color: rgba(255, 255, 255, 0.4);
color: #ffffff;
}
/* ===== Responsive Design ===== */

File diff suppressed because one or more lines are too long

File diff suppressed because one or more lines are too long

File diff suppressed because one or more lines are too long

File diff suppressed because one or more lines are too long

File diff suppressed because one or more lines are too long

File diff suppressed because one or more lines are too long

File diff suppressed because one or more lines are too long

File diff suppressed because one or more lines are too long

View File

@@ -0,0 +1,417 @@
{
"cells": [
{
"cell_type": "markdown",
"metadata": {},
"source": [
"# XSeg Face Segmentation\n",
"\n",
"<div style=\"display:flex; flex-wrap:wrap; align-items:center;\">\n",
" <a style=\"margin-right:10px; margin-bottom:6px;\" href=\"https://pepy.tech/projects/uniface\"><img alt=\"PyPI Downloads\" src=\"https://static.pepy.tech/personalized-badge/uniface?period=total&units=international_system&left_color=grey&right_color=blue&left_text=Downloads\"></a>\n",
" <a style=\"margin-right:10px; margin-bottom:6px;\" href=\"https://pypi.org/project/uniface/\"><img alt=\"PyPI Version\" src=\"https://img.shields.io/pypi/v/uniface.svg\"></a>\n",
" <a style=\"margin-right:10px; margin-bottom:6px;\" href=\"https://opensource.org/licenses/MIT\"><img alt=\"License\" src=\"https://img.shields.io/badge/License-MIT-blue.svg\"></a>\n",
" <a style=\"margin-bottom:6px;\" href=\"https://github.com/yakhyo/uniface\"><img alt=\"GitHub Stars\" src=\"https://img.shields.io/github/stars/yakhyo/uniface.svg?style=social\"></a>\n",
"</div>\n",
"\n",
"**UniFace** is a lightweight, production-ready Python library for face detection, recognition, tracking, landmark analysis, face parsing, gaze estimation, and face attributes.\n",
"\n",
"🔗 **GitHub**: [github.com/yakhyo/uniface](https://github.com/yakhyo/uniface) | 📚 **Docs**: [yakhyo.github.io/uniface](https://yakhyo.github.io/uniface)\n",
"\n",
"---\n",
"\n",
"This notebook demonstrates face segmentation using the **XSeg** model from DeepFaceLab.\n",
"\n",
"XSeg outputs a mask for face regions. Unlike BiSeNet which works on bbox crops, XSeg requires 5-point landmarks for face alignment.\n",
"\n",
"## 1. Install UniFace"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"%pip install -q \"uniface[cpu]\"\n",
"\n",
"# Clone repo for assets (Colab only)\n",
"import os\n",
"if 'COLAB_GPU' in os.environ or 'COLAB_RELEASE_TAG' in os.environ:\n",
" if not os.path.exists('uniface'):\n",
" !git clone --depth 1 https://github.com/yakhyo/uniface.git\n",
" os.chdir('uniface/examples')"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## 2. Import Libraries"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"import cv2\n",
"import numpy as np\n",
"import matplotlib.pyplot as plt\n",
"from pathlib import Path\n",
"\n",
"import uniface\n",
"from uniface.detection import RetinaFace\n",
"from uniface.parsing import XSeg\n",
"\n",
"print(f\"UniFace version: {uniface.__version__}\")"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## 3. Initialize Models\n",
"\n",
"XSeg requires face detection with landmarks. We use RetinaFace for detection and XSeg for segmentation."
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"# Initialize detector and parser\n",
"detector = RetinaFace()\n",
"parser = XSeg()\n",
"\n",
"print(f\"XSeg input size: {parser.input_size}\")\n",
"print(f\"Align size: {parser.align_size}\")\n",
"print(f\"Blur sigma: {parser.blur_sigma}\")"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## 4. Helper Functions"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"def apply_mask_overlay(image, mask, color=(0, 255, 0), alpha=0.5):\n",
" \"\"\"Apply colored mask overlay on image.\"\"\"\n",
" overlay = image.copy().astype(np.float32)\n",
"\n",
" # Create colored overlay where mask is positive\n",
" color_overlay = np.zeros_like(image, dtype=np.float32)\n",
" color_overlay[:] = color\n",
"\n",
" mask_3ch = mask[..., np.newaxis]\n",
" overlay = overlay * (1 - mask_3ch * alpha) + color_overlay * mask_3ch * alpha\n",
"\n",
" return overlay.clip(0, 255).astype(np.uint8)\n",
"\n",
"\n",
"def show_results(original, mask, result, title=\"XSeg Result\"):\n",
" \"\"\"Display original, mask, and result side by side.\"\"\"\n",
" fig, axes = plt.subplots(1, 3, figsize=(15, 5))\n",
"\n",
" axes[0].imshow(cv2.cvtColor(original, cv2.COLOR_BGR2RGB))\n",
" axes[0].set_title(\"Original\")\n",
" axes[0].axis(\"off\")\n",
"\n",
" axes[1].imshow(mask, cmap=\"gray\")\n",
" axes[1].set_title(\"Mask\")\n",
" axes[1].axis(\"off\")\n",
"\n",
" axes[2].imshow(cv2.cvtColor(result, cv2.COLOR_BGR2RGB))\n",
" axes[2].set_title(\"Overlay\")\n",
" axes[2].axis(\"off\")\n",
"\n",
" plt.suptitle(title)\n",
" plt.tight_layout()\n",
" plt.show()"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## 5. Process Single Image"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"# Load image\n",
"image_path = \"../assets/einstein.png\"\n",
"image = cv2.imread(image_path)\n",
"print(f\"Image shape: {image.shape}\")\n",
"\n",
"# Detect faces\n",
"faces = detector.detect(image)\n",
"print(f\"Detected {len(faces)} face(s)\")\n",
"\n",
"# Parse first face\n",
"if len(faces) > 0 and faces[0].landmarks is not None:\n",
" face = faces[0]\n",
" mask = parser.parse(image, landmarks=face.landmarks)\n",
"\n",
" print(f\"Mask shape: {mask.shape}\")\n",
" print(f\"Mask range: [{mask.min():.3f}, {mask.max():.3f}]\")\n",
"\n",
" # Visualize\n",
" result = apply_mask_overlay(image, mask)\n",
" show_results(image, mask, result, \"Single Face Segmentation\")"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## 6. Configurable Parameters\n",
"\n",
"XSeg has two main parameters:\n",
"- `align_size`: Face alignment output size (default: 256)\n",
"- `blur_sigma`: Gaussian blur for mask smoothing (default: 0 = raw output)"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"# Load image\n",
"image_path = \"../assets/einstein.png\"\n",
"image = cv2.imread(image_path)\n",
"\n",
"# Detect face\n",
"faces = detector.detect(image)\n",
"landmarks = faces[0].landmarks\n",
"\n",
"# Compare different blur settings\n",
"blur_values = [0, 3, 5]\n",
"\n",
"fig, axes = plt.subplots(1, len(blur_values), figsize=(15, 5))\n",
"\n",
"for i, blur in enumerate(blur_values):\n",
" parser_test = XSeg(blur_sigma=blur)\n",
" mask = parser_test.parse(image, landmarks=landmarks)\n",
"\n",
" axes[i].imshow(mask, cmap=\"gray\")\n",
" axes[i].set_title(f\"blur_sigma={blur}\")\n",
" axes[i].axis(\"off\")\n",
"\n",
"plt.suptitle(\"Effect of blur_sigma\")\n",
"plt.tight_layout()\n",
"plt.show()"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## 7. Using parse_aligned\n",
"\n",
"If you already have aligned face crops, use `parse_aligned()` directly."
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"from uniface.face_utils import face_alignment\n",
"\n",
"# Load and detect\n",
"image = cv2.imread(\"../assets/einstein.png\")\n",
"faces = detector.detect(image)\n",
"landmarks = faces[0].landmarks\n",
"\n",
"# Align face manually\n",
"aligned_face, inverse_matrix = face_alignment(image, landmarks, image_size=256)\n",
"print(f\"Aligned face shape: {aligned_face.shape}\")\n",
"\n",
"# Parse aligned crop directly\n",
"mask = parser.parse_aligned(aligned_face)\n",
"print(f\"Mask shape: {mask.shape}\")\n",
"\n",
"# Visualize\n",
"result = apply_mask_overlay(aligned_face, mask)\n",
"\n",
"fig, axes = plt.subplots(1, 3, figsize=(12, 4))\n",
"axes[0].imshow(cv2.cvtColor(aligned_face, cv2.COLOR_BGR2RGB))\n",
"axes[0].set_title(\"Aligned Face\")\n",
"axes[0].axis(\"off\")\n",
"\n",
"axes[1].imshow(mask, cmap=\"gray\")\n",
"axes[1].set_title(\"Mask\")\n",
"axes[1].axis(\"off\")\n",
"\n",
"axes[2].imshow(cv2.cvtColor(result, cv2.COLOR_BGR2RGB))\n",
"axes[2].set_title(\"Overlay\")\n",
"axes[2].axis(\"off\")\n",
"\n",
"plt.suptitle(\"parse_aligned() on pre-aligned crop\")\n",
"plt.tight_layout()\n",
"plt.show()"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## 8. XSeg vs BiSeNet\n",
"\n",
"| Feature | XSeg | BiSeNet |\n",
"|---------|------|--------|\n",
"| Output | Mask [0, 1] | 19 class labels |\n",
"| Input | Requires landmarks | Works on bbox crops |\n",
"| Use case | Face region extraction | Facial component parsing |\n",
"| Origin | DeepFaceLab | CelebAMask-HQ |"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"from uniface.parsing import BiSeNet\n",
"from uniface.draw import vis_parsing_maps\n",
"\n",
"# Load image and detect\n",
"image = cv2.imread(\"../assets/einstein.png\")\n",
"faces = detector.detect(image)\n",
"face = faces[0]\n",
"\n",
"# XSeg: requires landmarks\n",
"xseg_mask = parser.parse(image, landmarks=face.landmarks)\n",
"\n",
"# BiSeNet: works on bbox crop\n",
"bisenet = BiSeNet()\n",
"x1, y1, x2, y2 = map(int, face.bbox[:4])\n",
"face_crop = image[y1:y2, x1:x2]\n",
"bisenet_mask = bisenet.parse(face_crop)\n",
"\n",
"# Visualize comparison\n",
"fig, axes = plt.subplots(1, 3, figsize=(15, 5))\n",
"\n",
"axes[0].imshow(cv2.cvtColor(image, cv2.COLOR_BGR2RGB))\n",
"axes[0].set_title(\"Original\")\n",
"axes[0].axis(\"off\")\n",
"\n",
"axes[1].imshow(xseg_mask, cmap=\"gray\")\n",
"axes[1].set_title(\"XSeg\")\n",
"axes[1].axis(\"off\")\n",
"\n",
"face_rgb = cv2.cvtColor(face_crop, cv2.COLOR_BGR2RGB)\n",
"bisenet_vis = vis_parsing_maps(face_rgb, bisenet_mask, save_image=False)\n",
"axes[2].imshow(bisenet_vis)\n",
"axes[2].set_title(\"BiSeNet (19 classes)\")\n",
"axes[2].axis(\"off\")\n",
"\n",
"plt.suptitle(\"XSeg vs BiSeNet\")\n",
"plt.tight_layout()\n",
"plt.show()"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## 9. Application: Face Masking\n",
"\n",
"Use XSeg mask to extract or replace face regions."
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"# Load image\n",
"image = cv2.imread(\"../assets/einstein.png\")\n",
"faces = detector.detect(image)\n",
"mask = parser.parse(image, landmarks=faces[0].landmarks)\n",
"\n",
"# Extract face only\n",
"mask_3ch = np.stack([mask] * 3, axis=-1)\n",
"face_only = (image * mask_3ch).astype(np.uint8)\n",
"\n",
"# Replace background with white\n",
"white_bg = np.ones_like(image) * 255\n",
"face_on_white = (image * mask_3ch + white_bg * (1 - mask_3ch)).astype(np.uint8)\n",
"\n",
"# Visualize\n",
"fig, axes = plt.subplots(1, 3, figsize=(15, 5))\n",
"\n",
"axes[0].imshow(cv2.cvtColor(image, cv2.COLOR_BGR2RGB))\n",
"axes[0].set_title(\"Original\")\n",
"axes[0].axis(\"off\")\n",
"\n",
"axes[1].imshow(cv2.cvtColor(face_only, cv2.COLOR_BGR2RGB))\n",
"axes[1].set_title(\"Face Extracted\")\n",
"axes[1].axis(\"off\")\n",
"\n",
"axes[2].imshow(cv2.cvtColor(face_on_white, cv2.COLOR_BGR2RGB))\n",
"axes[2].set_title(\"White Background\")\n",
"axes[2].axis(\"off\")\n",
"\n",
"plt.suptitle(\"Face Masking Applications\")\n",
"plt.tight_layout()\n",
"plt.show()"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## Summary\n",
"\n",
"XSeg provides face segmentation using landmark-based alignment:\n",
"\n",
"- **`parse(image, landmarks=landmarks)`** - Full pipeline: align, segment, warp back\n",
"- **`parse_aligned(face_crop)`** - For pre-aligned crops\n",
"- **`parse_with_inverse(image, landmarks)`** - Returns mask + crop + inverse matrix\n",
"\n",
"Parameters:\n",
"- `align_size` - Face alignment size (default: 256)\n",
"- `blur_sigma` - Mask smoothing (default: 0 = raw)"
]
}
],
"metadata": {
"kernelspec": {
"display_name": "base",
"language": "python",
"name": "python3"
},
"language_info": {
"codemirror_mode": {
"name": "ipython",
"version": 3
},
"file_extension": ".py",
"mimetype": "text/x-python",
"name": "python",
"nbconvert_exporter": "python",
"pygments_lexer": "ipython3",
"version": "3.13.5"
}
},
"nbformat": 4,
"nbformat_minor": 4
}

View File

@@ -0,0 +1,291 @@
{
"cells": [
{
"cell_type": "markdown",
"metadata": {},
"source": [
"# Face Vector Store with FAISS\n",
"\n",
"<div style=\"display:flex; flex-wrap:wrap; align-items:center;\">\n",
" <a style=\"margin-right:10px; margin-bottom:6px;\" href=\"https://pepy.tech/projects/uniface\"><img alt=\"PyPI Downloads\" src=\"https://static.pepy.tech/personalized-badge/uniface?period=total&units=international_system&left_color=grey&right_color=blue&left_text=Downloads\"></a>\n",
" <a style=\"margin-right:10px; margin-bottom:6px;\" href=\"https://pypi.org/project/uniface/\"><img alt=\"PyPI Version\" src=\"https://img.shields.io/pypi/v/uniface.svg\"></a>\n",
" <a style=\"margin-right:10px; margin-bottom:6px;\" href=\"https://opensource.org/licenses/MIT\"><img alt=\"License\" src=\"https://img.shields.io/badge/License-MIT-blue.svg\"></a>\n",
" <a style=\"margin-bottom:6px;\" href=\"https://github.com/yakhyo/uniface\"><img alt=\"GitHub Stars\" src=\"https://img.shields.io/github/stars/yakhyo/uniface.svg?style=social\"></a>\n",
"</div>\n",
"\n",
"**UniFace** is a lightweight, production-ready Python library for face detection, recognition, tracking, landmark analysis, face parsing, gaze estimation, and face attributes.\n",
"\n",
"🔗 **GitHub**: [github.com/yakhyo/uniface](https://github.com/yakhyo/uniface) | 📚 **Docs**: [yakhyo.github.io/uniface](https://yakhyo.github.io/uniface)\n",
"\n",
"---\n",
"\n",
"This notebook demonstrates how to build a persistent face database using the **FAISS** vector store in UniFace.\n",
"\n",
"Unlike direct pairwise comparison (see `04_face_search`), a vector store lets you efficiently index\n",
"thousands of face embeddings and retrieve the closest match in sub-millisecond time.\n",
"\n",
"## 1. Install UniFace"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"%pip install -q \"uniface[cpu]\" faiss-cpu\n",
"\n",
"# Clone repo for assets (Colab only)\n",
"import os\n",
"if 'COLAB_GPU' in os.environ or 'COLAB_RELEASE_TAG' in os.environ:\n",
" if not os.path.exists('uniface'):\n",
" !git clone --depth 1 https://github.com/yakhyo/uniface.git\n",
" os.chdir('uniface/examples')"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## 2. Import Libraries"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"import cv2\n",
"import matplotlib.pyplot as plt\n",
"import shutil\n",
"\n",
"import uniface\n",
"from uniface.analyzer import FaceAnalyzer\n",
"from uniface.detection import RetinaFace\n",
"from uniface.recognition import ArcFace\n",
"from uniface.stores import FAISS\n",
"\n",
"print(f'UniFace version: {uniface.__version__}')"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## 3. Initialize Models and Vector Store"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"analyzer = FaceAnalyzer(\n",
" detector=RetinaFace(confidence_threshold=0.5),\n",
" recognizer=ArcFace(),\n",
")\n",
"\n",
"DB_PATH = './demo_face_index'\n",
"store = FAISS(embedding_size=512, db_path=DB_PATH)\n",
"print(store)"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## 4. Enroll Faces into the Store\n",
"\n",
"We detect faces in the test images and add each embedding with metadata."
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"enrollment_images = {\n",
" '../assets/test_images/image0.jpg': 'person_0',\n",
" '../assets/test_images/image1.jpg': 'person_1',\n",
" '../assets/test_images/image2.jpg': 'person_2',\n",
" '../assets/test_images/image3.jpg': 'person_3',\n",
" '../assets/test_images/image4.jpg': 'person_4',\n",
"}\n",
"\n",
"for path, label in enrollment_images.items():\n",
" image = cv2.imread(path)\n",
" faces = analyzer.analyze(image)\n",
" if faces:\n",
" store.add(\n",
" embedding=faces[0].embedding,\n",
" metadata={'label': label, 'source': path},\n",
" )\n",
" print(f'Enrolled {label} from {path}')\n",
"\n",
"print(f'\\nStore size: {store.size} vectors')"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## 5. Search the Store\n",
"\n",
"Use a query image to find the closest match in the database."
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"query_image = cv2.imread('../assets/test_images/image0.jpg')\n",
"query_faces = analyzer.analyze(query_image)\n",
"\n",
"if query_faces:\n",
" result, similarity = store.search(query_faces[0].embedding, threshold=0.4)\n",
"\n",
" if result:\n",
" print(f'Match found: {result[\"label\"]} (similarity: {similarity:.4f})')\n",
" print(f'Source: {result[\"source\"]}')\n",
" else:\n",
" print(f'No match above threshold (best similarity: {similarity:.4f})')"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"if query_faces and result:\n",
" matched_image = cv2.imread(result['source'])\n",
"\n",
" fig, axes = plt.subplots(1, 2, figsize=(10, 4))\n",
" axes[0].imshow(cv2.cvtColor(query_image, cv2.COLOR_BGR2RGB))\n",
" axes[0].set_title('Query', fontsize=12)\n",
" axes[1].imshow(cv2.cvtColor(matched_image, cv2.COLOR_BGR2RGB))\n",
" axes[1].set_title(f'Match: {result[\"label\"]} ({similarity:.3f})', fontsize=12)\n",
" for ax in axes:\n",
" ax.axis('off')\n",
" plt.tight_layout()\n",
" plt.show()"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## 6. Save and Reload the Index\n",
"\n",
"The index and metadata can be persisted to disk and loaded later."
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"store.save()\n",
"\n",
"# Create a fresh store and load the saved data\n",
"store_reloaded = FAISS(embedding_size=512, db_path=DB_PATH)\n",
"loaded = store_reloaded.load()\n",
"print(f'Load successful: {loaded}')\n",
"print(f'Reloaded store size: {store_reloaded.size} vectors')\n",
"\n",
"# Verify search still works after reload\n",
"if query_faces:\n",
" result, similarity = store_reloaded.search(query_faces[0].embedding, threshold=0.4)\n",
" if result:\n",
" print(f'Search after reload: {result[\"label\"]} ({similarity:.4f})')"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## 7. Remove Entries\n",
"\n",
"Remove all entries matching a metadata key-value pair."
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"print(f'Before removal: {store.size} vectors')\n",
"\n",
"removed = store.remove(key='label', value='person_0')\n",
"print(f'Removed {removed} entry')\n",
"print(f'After removal: {store.size} vectors')\n",
"\n",
"# Searching for the removed person should now return a different (lower) match\n",
"if query_faces:\n",
" result, similarity = store.search(query_faces[0].embedding, threshold=0.4)\n",
" if result:\n",
" print(f'\\nClosest remaining match: {result[\"label\"]} ({similarity:.4f})')\n",
" else:\n",
" print(f'\\nNo match above threshold (best similarity: {similarity:.4f})')"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## 8. Cleanup"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"shutil.rmtree(DB_PATH, ignore_errors=True)\n",
"print('Cleaned up demo index.')"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## Notes\n",
"\n",
"- Embeddings **must** be L2-normalised before adding (ArcFace already produces normalised embeddings)\n",
"- The default threshold of `0.4` works for most cases; raise it for stricter matching\n",
"- `save()` / `load()` persist the FAISS index and metadata as files in `db_path`\n",
"- For GPU-accelerated search install `faiss-gpu` instead of `faiss-cpu`\n",
"- The store uses `IndexFlatIP` (inner product = cosine similarity for normalised vectors)"
]
}
],
"metadata": {
"kernelspec": {
"display_name": "base",
"language": "python",
"name": "python3"
},
"language_info": {
"codemirror_mode": {
"name": "ipython",
"version": 3
},
"file_extension": ".py",
"mimetype": "text/x-python",
"name": "python",
"nbconvert_exporter": "python",
"pygments_lexer": "ipython3",
"version": "3.13.5"
}
},
"nbformat": 4,
"nbformat_minor": 4
}

View File

@@ -0,0 +1,223 @@
{
"cells": [
{
"cell_type": "markdown",
"metadata": {},
"source": [
"# Head Pose Estimation with UniFace\n",
"\n",
"<div style=\"display:flex; flex-wrap:wrap; align-items:center;\">\n",
" <a style=\"margin-right:10px; margin-bottom:6px;\" href=\"https://pepy.tech/projects/uniface\"><img alt=\"PyPI Downloads\" src=\"https://static.pepy.tech/personalized-badge/uniface?period=total&units=international_system&left_color=grey&right_color=blue&left_text=Downloads\"></a>\n",
" <a style=\"margin-right:10px; margin-bottom:6px;\" href=\"https://pypi.org/project/uniface/\"><img alt=\"PyPI Version\" src=\"https://img.shields.io/pypi/v/uniface.svg\"></a>\n",
" <a style=\"margin-right:10px; margin-bottom:6px;\" href=\"https://opensource.org/licenses/MIT\"><img alt=\"License\" src=\"https://img.shields.io/badge/License-MIT-blue.svg\"></a>\n",
" <a style=\"margin-bottom:6px;\" href=\"https://github.com/yakhyo/uniface\"><img alt=\"GitHub Stars\" src=\"https://img.shields.io/github/stars/yakhyo/uniface.svg?style=social\"></a>\n",
"</div>\n",
"\n",
"**UniFace** is a lightweight, production-ready Python library for face detection, recognition, tracking, landmark analysis, face parsing, gaze estimation, and face attributes.\n",
"\n",
"🔗 **GitHub**: [github.com/yakhyo/uniface](https://github.com/yakhyo/uniface) | 📚 **Docs**: [yakhyo.github.io/uniface](https://yakhyo.github.io/uniface)\n",
"\n",
"---\n",
"\n",
"This notebook demonstrates head pose estimation using the **UniFace** library.\n",
"\n",
"## 1. Install UniFace"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"%pip install -q \"uniface[cpu]\"\n",
"\n",
"# Clone repo for assets (Colab only)\n",
"import os\n",
"if 'COLAB_GPU' in os.environ or 'COLAB_RELEASE_TAG' in os.environ:\n",
" if not os.path.exists('uniface'):\n",
" !git clone --depth 1 https://github.com/yakhyo/uniface.git\n",
" os.chdir('uniface/examples')"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## 2. Import Libraries"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"import cv2\n",
"import numpy as np\n",
"import matplotlib.pyplot as plt\n",
"from pathlib import Path\n",
"from PIL import Image\n",
"\n",
"import uniface\n",
"from uniface.detection import RetinaFace\n",
"from uniface.headpose import HeadPose\n",
"from uniface.draw import draw_head_pose\n",
"\n",
"print(f\"UniFace version: {uniface.__version__}\")"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## 3. Initialize Models"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"# Initialize face detector\n",
"detector = RetinaFace(confidence_threshold=0.5)\n",
"\n",
"# Initialize head pose estimator (default: ResNet18 backbone)\n",
"head_pose = HeadPose()\n",
"\n",
"print(\"Models initialized successfully!\")"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## 4. Process All Test Images\n",
"\n",
"Display original images in the first row and head-pose-annotated images in the second row."
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"# Get all test images\n",
"test_images_dir = Path('../assets/test_images')\n",
"test_images = sorted(test_images_dir.glob('*.jpg'))\n",
"\n",
"original_images = []\n",
"annotated_images = []\n",
"\n",
"for img_path in test_images:\n",
" image = cv2.imread(str(img_path))\n",
" if image is None:\n",
" continue\n",
"\n",
" # Store original (BGR -> RGB for display)\n",
" original_images.append(cv2.cvtColor(image, cv2.COLOR_BGR2RGB))\n",
"\n",
" # Detect faces and estimate head pose\n",
" faces = detector.detect(image)\n",
"\n",
" for face in faces:\n",
" x1, y1, x2, y2 = map(int, face.bbox)\n",
" face_crop = image[y1:y2, x1:x2]\n",
"\n",
" if face_crop.size == 0:\n",
" continue\n",
"\n",
" result = head_pose.estimate(face_crop)\n",
" draw_head_pose(image, face.bbox, result.pitch, result.yaw, result.roll)\n",
"\n",
" print(f\"{img_path.name}: pitch={result.pitch:.1f}°, yaw={result.yaw:.1f}°, roll={result.roll:.1f}°\")\n",
"\n",
" annotated_images.append(cv2.cvtColor(image, cv2.COLOR_BGR2RGB))\n",
"\n",
"print(f\"\\nProcessed {len(original_images)} images\")"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## 5. Visualize Results\n",
"\n",
"**First row**: Original images \n",
"**Second row**: Images with head pose 3D cube overlay"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"num_images = len(original_images)\n",
"\n",
"# Create figure with 2 rows\n",
"fig, axes = plt.subplots(2, num_images, figsize=(5 * num_images, 10))\n",
"\n",
"if num_images == 1:\n",
" axes = axes.reshape(2, 1)\n",
"\n",
"for i in range(num_images):\n",
" axes[0, i].imshow(original_images[i])\n",
" axes[0, i].set_title('Original', fontsize=12)\n",
" axes[0, i].axis('off')\n",
"\n",
" axes[1, i].imshow(annotated_images[i])\n",
" axes[1, i].set_title('Head Pose', fontsize=12)\n",
" axes[1, i].axis('off')\n",
"\n",
"plt.tight_layout()\n",
"plt.show()"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## Notes\n",
"\n",
"- **Input**: Head pose estimation requires a face crop (obtained from face detection)\n",
"- **Output**: `HeadPoseResult` with pitch, yaw, and roll angles in **degrees**\n",
"- **Visualization**: Two modes available — `'cube'` (3D wireframe) and `'axis'` (X/Y/Z coordinate axes)\n",
"- **Models**: 6 backbone variants available via `HeadPoseWeights` enum\n",
"- **Method**: Uses 6D rotation representation converted to Euler angles\n",
"\n",
"### Available Backbones\n",
"\n",
"```python\n",
"from uniface.constants import HeadPoseWeights\n",
"\n",
"# Options: RESNET18, RESNET34, RESNET50, MOBILENET_V2, MOBILENET_V3_SMALL, MOBILENET_V3_LARGE\n",
"head_pose = HeadPose(model_name=HeadPoseWeights.RESNET50)\n",
"```"
]
}
],
"metadata": {
"kernelspec": {
"display_name": "base",
"language": "python",
"name": "python3"
},
"language_info": {
"codemirror_mode": {
"name": "ipython",
"version": 3
},
"file_extension": ".py",
"mimetype": "text/x-python",
"name": "python",
"nbconvert_exporter": "python",
"pygments_lexer": "ipython3",
"version": "3.13.5"
}
},
"nbformat": 4,
"nbformat_minor": 4
}

View File

@@ -0,0 +1,356 @@
{
"cells": [
{
"cell_type": "markdown",
"id": "0",
"metadata": {},
"source": [
"# Face Recognition: RetinaFace → Align → ArcFace\n",
"\n",
"<div style=\"display:flex; flex-wrap:wrap; align-items:center;\">\n",
" <a style=\"margin-right:10px; margin-bottom:6px;\" href=\"https://pepy.tech/projects/uniface\"><img alt=\"PyPI Downloads\" src=\"https://static.pepy.tech/personalized-badge/uniface?period=total&units=international_system&left_color=grey&right_color=blue&left_text=Downloads\"></a>\n",
" <a style=\"margin-right:10px; margin-bottom:6px;\" href=\"https://pypi.org/project/uniface/\"><img alt=\"PyPI Version\" src=\"https://img.shields.io/pypi/v/uniface.svg\"></a>\n",
" <a style=\"margin-right:10px; margin-bottom:6px;\" href=\"https://opensource.org/licenses/MIT\"><img alt=\"License\" src=\"https://img.shields.io/badge/License-MIT-blue.svg\"></a>\n",
" <a style=\"margin-bottom:6px;\" href=\"https://github.com/yakhyo/uniface\"><img alt=\"GitHub Stars\" src=\"https://img.shields.io/github/stars/yakhyo/uniface.svg?style=social\"></a>\n",
"</div>\n",
"\n",
"**UniFace** is a lightweight, production-ready Python library for face detection, recognition, tracking, landmark analysis, face parsing, gaze estimation, and face attributes.\n",
"\n",
"🔗 **GitHub**: [github.com/yakhyo/uniface](https://github.com/yakhyo/uniface) | 📚 **Docs**: [yakhyo.github.io/uniface](https://yakhyo.github.io/uniface)\n",
"\n",
"---\n",
"\n",
"This notebook demonstrates face recognition **without** the high-level `FaceAnalyzer` wrapper. Each step is handled manually:\n",
"\n",
"1. **RetinaFace**: Detects faces and extracts 5-point landmarks.\n",
"2. **Face Alignment**: Warps each face into a standardized 112x112 crop using the landmarks.\n",
"3. **ArcFace**: Generates a 512-D L2-normalized embedding from the aligned crop.\n",
"\n",
"We compare three test images: `image0.jpg`, `image1.jpg`, and `image5.jpg`."
]
},
{
"cell_type": "markdown",
"id": "1",
"metadata": {},
"source": [
"## 1. Install UniFace"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "2",
"metadata": {},
"outputs": [],
"source": [
"%pip install -q \"uniface[cpu]\"\n",
"\n",
"# Clone repo for assets (Colab only)\n",
"import os\n",
"if 'COLAB_GPU' in os.environ or 'COLAB_RELEASE_TAG' in os.environ:\n",
" if not os.path.exists('uniface'):\n",
" !git clone --depth 1 https://github.com/yakhyo/uniface.git\n",
" os.chdir('uniface/examples')"
]
},
{
"cell_type": "markdown",
"id": "3",
"metadata": {},
"source": [
"## 2. Import Libraries"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "4",
"metadata": {},
"outputs": [],
"source": [
"import cv2\n",
"import numpy as np\n",
"import matplotlib.pyplot as plt\n",
"import matplotlib.patches as patches\n",
"\n",
"import uniface\n",
"from uniface.detection import RetinaFace\n",
"from uniface.recognition import ArcFace\n",
"from uniface.face_utils import face_alignment\n",
"\n",
"print(f\"UniFace version: {uniface.__version__}\")"
]
},
{
"cell_type": "markdown",
"id": "5",
"metadata": {},
"source": [
"## 3. Configuration"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "6",
"metadata": {},
"outputs": [],
"source": [
"IMAGE_PATHS = {\n",
" \"image0\": \"../assets/test_images/image0.jpg\",\n",
" \"image1\": \"../assets/test_images/image1.jpg\",\n",
" \"image5\": \"../assets/test_images/image5.jpg\",\n",
"}\n",
"THRESHOLD = 0.4 # Cosine similarity threshold for \"same person\""
]
},
{
"cell_type": "markdown",
"id": "7",
"metadata": {},
"source": [
"## 4. Initialize Models"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "8",
"metadata": {},
"outputs": [],
"source": [
"detector = RetinaFace(confidence_threshold=0.5)\n",
"recognizer = ArcFace()"
]
},
{
"cell_type": "markdown",
"id": "9",
"metadata": {},
"source": [
"## 5. Load Images & Detect Faces\n",
"\n",
"We use the detector to find faces and their landmarks in each image."
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "10",
"metadata": {},
"outputs": [],
"source": [
"images = {}\n",
"faces = {}\n",
"\n",
"for name, path in IMAGE_PATHS.items():\n",
" img = cv2.imread(path)\n",
" if img is None:\n",
" raise FileNotFoundError(f\"Cannot read: {path}\")\n",
"\n",
" detected = detector.detect(img)\n",
" if not detected:\n",
" raise RuntimeError(f\"No face detected in: {path}\")\n",
"\n",
" images[name] = img\n",
" faces[name] = detected[0] # Keep highest-confidence face\n",
" print(f\"{name:8s} | {len(detected)} face(s) detected | confidence={faces[name].confidence:.3f}\")"
]
},
{
"cell_type": "markdown",
"id": "11",
"metadata": {},
"source": [
"## 6. Visualize Detections"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "12",
"metadata": {},
"outputs": [],
"source": [
"LM_COLORS = [\"red\", \"blue\", \"green\", \"cyan\", \"magenta\"]\n",
"\n",
"fig, axes = plt.subplots(1, 3, figsize=(15, 5))\n",
"fig.suptitle(\"Detected Faces & 5-Point Landmarks\", fontweight=\"bold\", fontsize=16)\n",
"\n",
"for ax, (name, img) in zip(axes, images.items()):\n",
" face = faces[name]\n",
" ax.imshow(cv2.cvtColor(img, cv2.COLOR_BGR2RGB))\n",
" ax.set_title(f\"{name}\\nconf={face.confidence:.3f}\", fontsize=12)\n",
" ax.axis(\"off\")\n",
"\n",
" # Bounding box\n",
" x1, y1, x2, y2 = face.bbox.astype(int)\n",
" ax.add_patch(patches.Rectangle(\n",
" (x1, y1), x2 - x1, y2 - y1,\n",
" linewidth=2, edgecolor=\"lime\", facecolor=\"none\"))\n",
"\n",
" # Landmarks\n",
" for (lx, ly), c in zip(face.landmarks, LM_COLORS):\n",
" ax.plot(lx, ly, \"o\", color=c, markersize=6)\n",
"\n",
"plt.tight_layout()\n",
"plt.show()"
]
},
{
"cell_type": "markdown",
"id": "13",
"metadata": {},
"source": [
"## 7. Face Alignment\n",
"\n",
"We warp the detected faces into a standardized 112x112 size. This improves recognition accuracy."
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "14",
"metadata": {},
"outputs": [],
"source": [
"aligned = {}\n",
"\n",
"for name, img in images.items():\n",
" lm = faces[name].landmarks\n",
" crop, _ = face_alignment(img, lm, image_size=(112, 112))\n",
" aligned[name] = crop\n",
"\n",
"fig, axes = plt.subplots(1, 3, figsize=(12, 4))\n",
"fig.suptitle(\"Aligned Face Crops (112x112)\", fontweight=\"bold\", fontsize=14)\n",
"\n",
"for ax, (name, crop) in zip(axes, aligned.items()):\n",
" ax.imshow(cv2.cvtColor(crop, cv2.COLOR_BGR2RGB))\n",
" ax.set_title(name, fontsize=12)\n",
" ax.axis(\"off\")\n",
"\n",
"plt.tight_layout()\n",
"plt.show()"
]
},
{
"cell_type": "markdown",
"id": "15",
"metadata": {},
"source": [
"## 8. Extract Embeddings\n",
"\n",
"We pass the aligned crops to ArcFace to get the 512-D vectors."
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "16",
"metadata": {},
"outputs": [],
"source": [
"embeddings = {}\n",
"\n",
"for name, crop in aligned.items():\n",
" # landmarks=None because image is already aligned\n",
" emb = recognizer.get_normalized_embedding(crop, landmarks=None)\n",
" embeddings[name] = emb\n",
" print(f\"{name:8s} | embedding shape={emb.shape} | L2-norm={np.linalg.norm(emb):.4f}\")"
]
},
{
"cell_type": "markdown",
"id": "17",
"metadata": {},
"source": [
"## 9. Pairwise Cosine Similarity\n",
"\n",
"Since embeddings are normalized, cosine similarity is just the dot product."
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "18",
"metadata": {},
"outputs": [],
"source": [
"names = list(embeddings.keys())\n",
"n = len(names)\n",
"sim_matrix = np.zeros((n, n))\n",
"\n",
"for i, ni in enumerate(names):\n",
" for j, nj in enumerate(names):\n",
" # Use squeeze() to handle (1, 512) shapes if present\n",
" sim_matrix[i, j] = float(np.dot(embeddings[ni].squeeze(), embeddings[nj].squeeze()))\n",
"\n",
"# Print comparison results\n",
"pairs = [(names[i], names[j]) for i in range(n) for j in range(i + 1, n)]\n",
"for a, b in pairs:\n",
" s = float(np.dot(embeddings[a].squeeze(), embeddings[b].squeeze()))\n",
" verdict = \"✓ Same person\" if s >= THRESHOLD else \"✗ Different people\"\n",
" print(f\"{a} vs {b}: similarity={s:.4f} → {verdict}\")"
]
},
{
"cell_type": "markdown",
"id": "19",
"metadata": {},
"source": [
"## 10. Similarity Heatmap"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "20",
"metadata": {},
"outputs": [],
"source": [
"fig, ax = plt.subplots(figsize=(8, 6))\n",
"im = ax.imshow(sim_matrix, vmin=0, vmax=1, cmap=\"viridis\")\n",
"plt.colorbar(im, ax=ax, label=\"Cosine similarity\")\n",
"\n",
"ax.set_xticks(range(n))\n",
"ax.set_yticks(range(n))\n",
"ax.set_xticklabels(names, rotation=30, ha=\"right\")\n",
"ax.set_yticklabels(names)\n",
"ax.set_title(\"Pairwise Face Similarity (ArcFace)\", fontweight=\"bold\")\n",
"\n",
"for i in range(n):\n",
" for j in range(n):\n",
" val = sim_matrix[i, j]\n",
" ax.text(j, i, f\"{val:.2f}\",\n",
" ha=\"center\", va=\"center\",\n",
" color=\"black\" if val >= 0.6 else \"white\",\n",
" fontsize=12, fontweight=\"bold\")\n",
"\n",
"plt.tight_layout()\n",
"plt.show()"
]
}
],
"metadata": {
"kernelspec": {
"display_name": "base",
"language": "python",
"name": "python3"
},
"language_info": {
"codemirror_mode": {
"name": "ipython",
"version": 3
},
"file_extension": ".py",
"mimetype": "text/x-python",
"name": "python",
"nbconvert_exporter": "python",
"pygments_lexer": "ipython3",
"version": "3.13.5"
}
},
"nbformat": 4,
"nbformat_minor": 5
}

View File

@@ -0,0 +1,265 @@
{
"cells": [
{
"cell_type": "markdown",
"metadata": {},
"source": [
"# Portrait Matting with MODNet\n",
"\n",
"<div style=\"display:flex; flex-wrap:wrap; align-items:center;\">\n",
" <a style=\"margin-right:10px; margin-bottom:6px;\" href=\"https://pepy.tech/projects/uniface\"><img alt=\"PyPI Downloads\" src=\"https://static.pepy.tech/personalized-badge/uniface?period=total&units=international_system&left_color=grey&right_color=blue&left_text=Downloads\"></a>\n",
" <a style=\"margin-right:10px; margin-bottom:6px;\" href=\"https://pypi.org/project/uniface/\"><img alt=\"PyPI Version\" src=\"https://img.shields.io/pypi/v/uniface.svg\"></a>\n",
" <a style=\"margin-right:10px; margin-bottom:6px;\" href=\"https://opensource.org/licenses/MIT\"><img alt=\"License\" src=\"https://img.shields.io/badge/License-MIT-blue.svg\"></a>\n",
" <a style=\"margin-bottom:6px;\" href=\"https://github.com/yakhyo/uniface\"><img alt=\"GitHub Stars\" src=\"https://img.shields.io/github/stars/yakhyo/uniface.svg?style=social\"></a>\n",
"</div>\n",
"\n",
"**UniFace** is a lightweight, production-ready Python library for face detection, recognition, tracking, landmark analysis, face parsing, gaze estimation, and face attributes.\n",
"\n",
"🔗 **GitHub**: [github.com/yakhyo/uniface](https://github.com/yakhyo/uniface) | 📚 **Docs**: [yakhyo.github.io/uniface](https://yakhyo.github.io/uniface)\n",
"\n",
"---\n",
"\n",
"This notebook demonstrates portrait matting using **MODNet** — a trimap-free model that produces soft alpha mattes from full images. No face detection or cropping required.\n",
"\n",
"## 1. Install UniFace"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"%pip install -q \"uniface[cpu]\"\n",
"\n",
"# Clone repo for assets (Colab only)\n",
"import os\n",
"if 'COLAB_GPU' in os.environ or 'COLAB_RELEASE_TAG' in os.environ:\n",
" if not os.path.exists('uniface'):\n",
" !git clone --depth 1 https://github.com/yakhyo/uniface.git\n",
" os.chdir('uniface/examples')"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## 2. Import Libraries"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"import cv2\n",
"import numpy as np\n",
"import matplotlib.pyplot as plt\n",
"\n",
"import uniface\n",
"from uniface.matting import MODNet\n",
"\n",
"print(f\"UniFace version: {uniface.__version__}\")"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## 3. Initialize Model\n",
"\n",
"MODNet has two variants:\n",
"- **PHOTOGRAPHIC** (default): optimized for high-quality portrait photos\n",
"- **WEBCAM**: optimized for real-time webcam feeds"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"matting = MODNet()\n",
"\n",
"print(f\"Input size: {matting.input_size}\")\n",
"print(f\"Input name: {matting.input_name}\")\n",
"print(f\"Output names: {matting.output_names}\")"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## 4. Helper Functions"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"def compose(image, matte, background=None):\n",
" \"\"\"Composite foreground over a background using the alpha matte.\"\"\"\n",
" h, w = image.shape[:2]\n",
" matte_3ch = matte[:, :, np.newaxis]\n",
"\n",
" if background is None:\n",
" bg = np.full_like(image, (0, 177, 64), dtype=np.uint8)\n",
" else:\n",
" bg = cv2.resize(background, (w, h), interpolation=cv2.INTER_AREA)\n",
"\n",
" return (image * matte_3ch + bg * (1 - matte_3ch)).astype(np.uint8)\n",
"\n",
"\n",
"def show_results(image, matte):\n",
" \"\"\"Display original, matte, and green screen as a single merged image.\"\"\"\n",
" matte_vis = cv2.cvtColor((matte * 255).astype(np.uint8), cv2.COLOR_GRAY2BGR)\n",
" green = compose(image, matte)\n",
" merged = np.hstack([image, matte_vis, green])\n",
"\n",
" plt.figure(figsize=(18, 6))\n",
" plt.imshow(cv2.cvtColor(merged, cv2.COLOR_BGR2RGB))\n",
" plt.axis(\"off\")\n",
" plt.tight_layout()\n",
" plt.show()"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## 5. Basic Matting"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"image = cv2.imread(\"../assets/demos/src_portrait1.jpg\")\n",
"print(f\"Image shape: {image.shape}\")\n",
"\n",
"matte = matting.predict(image)\n",
"print(f\"Matte shape: {matte.shape}\")\n",
"print(f\"Matte dtype: {matte.dtype}\")\n",
"print(f\"Matte range: [{matte.min():.3f}, {matte.max():.3f}]\")\n",
"\n",
"show_results(image, matte)"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## 6. Transparent Background (RGBA)"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"alpha = (matte * 255).astype(np.uint8)\n",
"rgba = cv2.cvtColor(image, cv2.COLOR_BGR2BGRA)\n",
"rgba[:, :, 3] = alpha\n",
"\n",
"# Checkerboard background to visualize transparency\n",
"h, w = image.shape[:2]\n",
"checker = np.zeros((h, w, 3), dtype=np.uint8)\n",
"block = 20\n",
"for y in range(0, h, block):\n",
" for x in range(0, w, block):\n",
" if (y // block + x // block) % 2 == 0:\n",
" checker[y:y+block, x:x+block] = 200\n",
" else:\n",
" checker[y:y+block, x:x+block] = 255\n",
"\n",
"matte_3ch = matte[:, :, np.newaxis]\n",
"rgba_vis = (image * matte_3ch + checker * (1 - matte_3ch)).astype(np.uint8)\n",
"\n",
"merged = np.hstack([image, rgba_vis])\n",
"\n",
"plt.figure(figsize=(16, 5))\n",
"plt.imshow(cv2.cvtColor(merged, cv2.COLOR_BGR2RGB))\n",
"plt.axis(\"off\")\n",
"plt.tight_layout()\n",
"plt.show()\n",
"\n",
"print(f\"RGBA shape: {rgba.shape}\")"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## 7. Custom Background"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"# Create a gradient background\n",
"h, w = image.shape[:2]\n",
"gradient = np.zeros((h, w, 3), dtype=np.uint8)\n",
"for y in range(h):\n",
" ratio = y / h\n",
" gradient[y, :] = [int(180 * (1 - ratio)), int(100 + 80 * ratio), int(220 * ratio)]\n",
"\n",
"custom_bg = compose(image, matte, gradient)\n",
"green_bg = compose(image, matte)\n",
"\n",
"merged = np.hstack([image, green_bg, custom_bg])\n",
"\n",
"plt.figure(figsize=(18, 6))\n",
"plt.imshow(cv2.cvtColor(merged, cv2.COLOR_BGR2RGB))\n",
"plt.axis(\"off\")\n",
"plt.tight_layout()\n",
"plt.show()"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## Summary\n",
"\n",
"MODNet provides trimap-free portrait matting:\n",
"\n",
"- **`predict(image)`** — returns `(H, W)` float32 alpha matte in `[0, 1]`\n",
"- **No face detection needed** — works on full images directly\n",
"- **Two variants** — `PHOTOGRAPHIC` for photos, `WEBCAM` for real-time\n",
"- **Compositing** — use the matte for transparent PNGs, green screen, or custom backgrounds\n",
"\n",
"For more details, see the [Matting docs](https://yakhyo.github.io/uniface/modules/matting/)."
]
}
],
"metadata": {
"kernelspec": {
"display_name": "base",
"language": "python",
"name": "python3"
},
"language_info": {
"codemirror_mode": {
"name": "ipython",
"version": 3
},
"file_extension": ".py",
"mimetype": "text/x-python",
"name": "python",
"nbconvert_exporter": "python",
"pygments_lexer": "ipython3",
"version": "3.13.5"
}
},
"nbformat": 4,
"nbformat_minor": 4
}

View File

@@ -0,0 +1,260 @@
{
"cells": [
{
"cell_type": "markdown",
"metadata": {},
"source": [
"# Face Attribute Detection with UniFace\n",
"\n",
"<div style=\"display:flex; flex-wrap:wrap; align-items:center;\">\n",
" <a style=\"margin-right:10px; margin-bottom:6px;\" href=\"https://pepy.tech/projects/uniface\"><img alt=\"PyPI Downloads\" src=\"https://static.pepy.tech/personalized-badge/uniface?period=total&units=international_system&left_color=grey&right_color=blue&left_text=Downloads\"></a>\n",
" <a style=\"margin-right:10px; margin-bottom:6px;\" href=\"https://pypi.org/project/uniface/\"><img alt=\"PyPI Version\" src=\"https://img.shields.io/pypi/v/uniface.svg\"></a>\n",
" <a style=\"margin-right:10px; margin-bottom:6px;\" href=\"https://opensource.org/licenses/MIT\"><img alt=\"License\" src=\"https://img.shields.io/badge/License-MIT-blue.svg\"></a>\n",
" <a style=\"margin-bottom:6px;\" href=\"https://github.com/yakhyo/uniface\"><img alt=\"GitHub Stars\" src=\"https://img.shields.io/github/stars/yakhyo/uniface.svg?style=social\"></a>\n",
"</div>\n",
"\n",
"**UniFace** is a lightweight, production-ready Python library for face detection, recognition, tracking, landmark analysis, face parsing, gaze estimation, and face attributes.\n",
"\n",
"🔗 **GitHub**: [github.com/yakhyo/uniface](https://github.com/yakhyo/uniface) | 📚 **Docs**: [yakhyo.github.io/uniface](https://yakhyo.github.io/uniface)\n",
"\n",
"---\n",
"\n",
"This notebook demonstrates face attribute detection with **FaceAttribNet** — five independent binary face states per face: left/right eye openness, eyeglasses, face mask, and sunglasses.\n",
"\n",
"## 1. Install UniFace"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"%pip install -q \"uniface[cpu]\"\n",
"\n",
"# Clone repo for assets (Colab only)\n",
"import os\n",
"if 'COLAB_GPU' in os.environ or 'COLAB_RELEASE_TAG' in os.environ:\n",
" if not os.path.exists('uniface'):\n",
" !git clone --depth 1 https://github.com/yakhyo/uniface.git\n",
" os.chdir('uniface/examples')"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## 2. Import Libraries"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"import cv2\n",
"import matplotlib.pyplot as plt\n",
"from pathlib import Path\n",
"\n",
"import uniface\n",
"from uniface.attribute import FaceAttribNet\n",
"from uniface.detection import RetinaFace\n",
"\n",
"print(f\"UniFace version: {uniface.__version__}\")"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## 3. Initialize Models"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"# Initialize face detector\n",
"detector = RetinaFace(confidence_threshold=0.5)\n",
"\n",
"# Initialize face attribute model\n",
"face_attrib = FaceAttribNet()\n",
"\n",
"print(\"Models initialized successfully!\")"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## 4. Process All Test Images\n",
"\n",
"Each face gets a `FaceStateResult` with five **independent** probabilities: `left_eye_open`, `right_eye_open`, `eyeglasses`, `mask`, `sunglasses`.\n",
"\n",
"> ⚠️ These come from independent binary heads — they do not sum to 1 and several can be high at once (a face can wear both sunglasses and a mask). Threshold each attribute separately; never `argmax`."
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"THRESHOLD = 0.5\n",
"\n",
"\n",
"def annotate(image, face, result):\n",
" \"\"\"Draw the face box and the attributes that exceed THRESHOLD.\"\"\"\n",
" x1, y1, x2, y2 = map(int, face.bbox)\n",
" cv2.rectangle(image, (x1, y1), (x2, y2), (0, 255, 0), 2)\n",
"\n",
" scale = max(0.6, (x2 - x1) / 250)\n",
" for i, label in enumerate(result.labels(THRESHOLD)):\n",
" y_text = y1 + int((i + 1) * 32 * scale)\n",
" cv2.putText(image, label, (x2 + 8, y_text), cv2.FONT_HERSHEY_SIMPLEX, scale, (0, 255, 0), 2, cv2.LINE_AA)\n",
"\n",
"\n",
"test_images_dir = Path('../assets/test_images/attributes')\n",
"test_images = sorted(test_images_dir.glob('*.jpg'))\n",
"\n",
"original_images = []\n",
"annotated_images = []\n",
"titles = []\n",
"\n",
"for img_path in test_images:\n",
" image = cv2.imread(str(img_path))\n",
" if image is None:\n",
" continue\n",
"\n",
" original_images.append(cv2.cvtColor(image, cv2.COLOR_BGR2RGB))\n",
"\n",
" faces = detector.detect(image)\n",
" for face in faces:\n",
" result = face_attrib.predict(image, face)\n",
" active = ', '.join(result.labels(THRESHOLD)) or 'none'\n",
" print(f\"{img_path.name}: {active}\")\n",
" annotate(image, face, result)\n",
"\n",
" annotated_images.append(cv2.cvtColor(image, cv2.COLOR_BGR2RGB))\n",
" titles.append(img_path.stem)\n",
"\n",
"print(f\"\\nProcessed {len(original_images)} images\")"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## 5. Visualize Results\n",
"\n",
"**First row**: Original images\n",
"**Second row**: Faces annotated with the attributes above the threshold"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"num_images = len(original_images)\n",
"\n",
"fig, axes = plt.subplots(2, num_images, figsize=(4 * num_images, 9))\n",
"\n",
"if num_images == 1:\n",
" axes = axes.reshape(2, 1)\n",
"\n",
"for i in range(num_images):\n",
" axes[0, i].imshow(original_images[i])\n",
" axes[0, i].set_title(titles[i], fontsize=12)\n",
" axes[0, i].axis('off')\n",
"\n",
" axes[1, i].imshow(annotated_images[i])\n",
" axes[1, i].set_title('Attributes', fontsize=12)\n",
" axes[1, i].axis('off')\n",
"\n",
"plt.tight_layout()\n",
"plt.show()"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## 6. Inspect a Single Prediction\n",
"\n",
"`predict()` returns a `FaceStateResult` and also writes the probabilities back onto the `Face` object (`face.left_eye_open`, `face.right_eye_open`, `face.eyeglasses`, `face.mask`, `face.sunglasses`)."
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"image = cv2.imread(str(test_images_dir / 'sunglasses.jpg'))\n",
"face = detector.detect(image)[0]\n",
"\n",
"result = face_attrib.predict(image, face)\n",
"\n",
"print(result)\n",
"print()\n",
"for name, prob in result.as_dict().items():\n",
" marker = '✔' if prob > THRESHOLD else '✘'\n",
" print(f\" {marker} {name:15s} {prob:.4f}\")\n",
"\n",
"print()\n",
"print(f\"Face enriched: sunglasses={face.sunglasses:.4f}, left_eye_open={face.left_eye_open:.4f}\")"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## Notes\n",
"\n",
"- **Input**: full image + detected `Face` (the model crops by `face.bbox` internally, letterboxes to 128×128)\n",
"- **Output**: `FaceStateResult` with five independent probabilities in `[0, 1]`\n",
"- **Multi-label**: threshold each attribute separately (`result.labels(threshold)`); never `argmax`\n",
"- **Face enrichment**: `predict()` also sets `face.left_eye_open`, `face.right_eye_open`, `face.eyeglasses`, `face.mask`, `face.sunglasses`\n",
"- **Model**: Qualcomm's [FaceAttribNet](https://github.com/qualcomm/ai-hub-models/tree/main/src/qai_hub_models/models/face_attrib_net) (\"Facial-Attribute-Detection\"), ONNX export from [yakhyo/face-attribute](https://github.com/yakhyo/face-attribute)\n",
"\n",
"### With FaceAnalyzer\n",
"\n",
"```python\n",
"from uniface import FaceAnalyzer, FaceAttribNet\n",
"\n",
"analyzer = FaceAnalyzer(predictors=[FaceAttribNet()])\n",
"faces = analyzer.analyze(image)\n",
"\n",
"for face in faces:\n",
" print(face.eyeglasses, face.mask, face.sunglasses)\n",
"```"
]
}
],
"metadata": {
"kernelspec": {
"display_name": "base",
"language": "python",
"name": "python3"
},
"language_info": {
"codemirror_mode": {
"name": "ipython",
"version": 3
},
"file_extension": ".py",
"mimetype": "text/x-python",
"name": "python",
"nbconvert_exporter": "python",
"pygments_lexer": "ipython3",
"version": "3.13.5"
}
},
"nbformat": 4,
"nbformat_minor": 4
}

287
examples/15_face_mesh.ipynb Normal file
View File

@@ -0,0 +1,287 @@
{
"cells": [
{
"cell_type": "markdown",
"metadata": {},
"source": [
"# Dense Face Mesh with UniFace\n",
"\n",
"<div style=\"display:flex; flex-wrap:wrap; align-items:center;\">\n",
" <a style=\"margin-right:10px; margin-bottom:6px;\" href=\"https://pepy.tech/projects/uniface\"><img alt=\"PyPI Downloads\" src=\"https://static.pepy.tech/personalized-badge/uniface?period=total&units=international_system&left_color=grey&right_color=blue&left_text=Downloads\"></a>\n",
" <a style=\"margin-right:10px; margin-bottom:6px;\" href=\"https://pypi.org/project/uniface/\"><img alt=\"PyPI Version\" src=\"https://img.shields.io/pypi/v/uniface.svg\"></a>\n",
" <a style=\"margin-right:10px; margin-bottom:6px;\" href=\"https://opensource.org/licenses/MIT\"><img alt=\"License\" src=\"https://img.shields.io/badge/License-MIT-blue.svg\"></a>\n",
" <a style=\"margin-bottom:6px;\" href=\"https://github.com/yakhyo/uniface\"><img alt=\"GitHub Stars\" src=\"https://img.shields.io/github/stars/yakhyo/uniface.svg?style=social\"></a>\n",
"</div>\n",
"\n",
"**UniFace** is a lightweight, production-ready Python library for face detection, recognition, tracking, landmark analysis, face parsing, gaze estimation, and face attributes.\n",
"\n",
"🔗 **GitHub**: [github.com/yakhyo/uniface](https://github.com/yakhyo/uniface) | 📚 **Docs**: [yakhyo.github.io/uniface](https://yakhyo.github.io/uniface)\n",
"\n",
"---\n",
"\n",
"This notebook demonstrates **MediaPipe Face Mesh** — 468 dense 3D landmarks per face.\n",
"\n",
"Unlike the other landmark models it returns **three** coordinates per point and a face-presence score, and it runs every face in an image through a **single batched inference call**.\n",
"\n",
"## 1. Install UniFace"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"%pip install -q \"uniface[cpu]\"\n",
"\n",
"# Clone repo for assets (Colab only)\n",
"import os\n",
"if 'COLAB_GPU' in os.environ or 'COLAB_RELEASE_TAG' in os.environ:\n",
" if not os.path.exists('uniface'):\n",
" !git clone --depth 1 https://github.com/yakhyo/uniface.git\n",
" os.chdir('uniface/examples')"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## 2. Import Libraries"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"import cv2\n",
"import matplotlib.pyplot as plt\n",
"import numpy as np\n",
"from pathlib import Path\n",
"\n",
"import uniface\n",
"from uniface.detection import SCRFD, BlazeFace\n",
"from uniface.draw import draw_mesh\n",
"from uniface.landmark import FaceMesh\n",
"\n",
"print(f\"UniFace version: {uniface.__version__}\")"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## 3. Initialize Models"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"# Any detector works — Face Mesh only needs a box plus the two eye points\n",
"detector = SCRFD()\n",
"mesher = FaceMesh()\n",
"\n",
"print(f\"Landmarks: {mesher.num_landmarks}\")\n",
"print(f\"Input size: {mesher.input_size}x{mesher.input_size}\")"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## 4. Detect and Mesh\n",
"\n",
"Face Mesh needs a bounding box plus the first two landmarks (the eyes), which it uses to rotate the crop so the eye line is horizontal. Passing `Face` objects does this automatically."
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"image = cv2.imread('../assets/einstein.png')\n",
"\n",
"faces = detector.detect(image)\n",
"print(f\"Detected {len(faces)} face(s)\")\n",
"\n",
"# One batched call for every face in the image\n",
"results = mesher.predict(image, faces)\n",
"\n",
"result = results[0]\n",
"print(f\"Landmarks shape: {result.landmarks.shape}\") # (468, 3)\n",
"print(f\"2D only: {result.points_2d.shape}\") # (468, 2)\n",
"print(f\"Presence score: {result.score:.4f}\")"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## 5. Visualize\n",
"\n",
"`draw_mesh` has three render modes. `full` draws the dense 2556-edge tessellation — the most detailed, but noticeably slower. Prefer `partial` or `points` for video."
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"fig, axes = plt.subplots(1, 3, figsize=(18, 6))\n",
"\n",
"for ax, mode in zip(axes, ['points', 'partial', 'full']):\n",
" canvas = image.copy()\n",
" draw_mesh(canvas, result.landmarks, mode=mode)\n",
" ax.imshow(cv2.cvtColor(canvas, cv2.COLOR_BGR2RGB))\n",
" ax.set_title(f\"mode='{mode}'\")\n",
" ax.axis('off')\n",
"\n",
"plt.tight_layout()\n",
"plt.show()"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## 6. The Depth Coordinate\n",
"\n",
"`z` is **relative** depth on the same pixel scale as `x`/`y`, where smaller is closer to the camera. It has no absolute origin, so it is only meaningful *within* one face — not between faces or between images."
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"points = result.landmarks\n",
"\n",
"plt.figure(figsize=(7, 7))\n",
"scatter = plt.scatter(points[:, 0], points[:, 1], c=points[:, 2], cmap='viridis', s=8)\n",
"plt.colorbar(scatter, label='z (relative depth, smaller = closer)')\n",
"plt.gca().invert_yaxis()\n",
"plt.gca().set_aspect('equal')\n",
"plt.title('468 landmarks coloured by depth')\n",
"plt.show()\n",
"\n",
"print(f\"z range: {points[:, 2].min():.1f} .. {points[:, 2].max():.1f}\")\n",
"print(\"The nose tip is the closest point; the ears and jaw edges sit furthest back.\")"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## 7. MediaPipe Parity\n",
"\n",
"Seeding the mesh with **BlazeFace** reproduces MediaPipe's own pipeline exactly.\n",
"\n",
"> ⚠️ BlazeFace returns **6** MediaPipe keypoints whose 4th point is a mouth *center*, not corners, so they cannot be fitted to the 5-point alignment template. It declares this with `supports_alignment = False`, and `FaceAnalyzer` disables recognition for it rather than producing broken embeddings."
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"# BlazeFace is the detector mp.solutions.face_mesh runs internally\n",
"blazeface = BlazeFace()\n",
"\n",
"bf_faces = blazeface.detect(image)\n",
"bf_results = mesher.predict(image, bf_faces)\n",
"\n",
"print(f\"BlazeFace keypoints: {bf_faces[0].landmarks.shape}\") # (6, 2), not (5, 2)\n",
"print(f\"supports_alignment: {blazeface.supports_alignment}\") # False\n",
"\n",
"canvas = image.copy()\n",
"draw_mesh(canvas, bf_results[0].landmarks, mode='full')\n",
"\n",
"plt.figure(figsize=(7, 7))\n",
"plt.imshow(cv2.cvtColor(canvas, cv2.COLOR_BGR2RGB))\n",
"plt.title('MediaPipe parity: BlazeFace + Face Mesh')\n",
"plt.axis('off')\n",
"plt.show()"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## 8. Other Ways to Call It"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"# Face Mesh implements the same interface as Landmark106 and PIPNet\n",
"landmarks_2d = mesher.get_landmarks(image, faces[0].bbox)\n",
"print(f\"get_landmarks: {landmarks_2d.shape}\")\n",
"\n",
"# Without a detector, pass boxes directly\n",
"x1, y1, x2, y2 = faces[0].bbox\n",
"manual = mesher.predict(image, bboxes=[[x1, y1, x2, y2]])\n",
"print(f\"From a raw box: {manual[0].landmarks.shape}\")\n",
"\n",
"# Tune the crop with margin= if the mesh clips on unusual framing\n",
"tight = mesher.predict(image, faces, margin=0.1)[0]\n",
"print(f\"Tighter crop shifts the fit: {not np.allclose(tight.landmarks, result.landmarks)}\")"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## Summary\n",
"\n",
"| | |\n",
"|---|---|\n",
"| **Output** | `(468, 3)` — `x`/`y` in image pixels, `z` relative depth on the same scale |\n",
"| **Presence score** | Saturates near 1.0; confirms the model ran, not a discriminative confidence |\n",
"| **Detectors** | Works with any of them — needs a box plus the two eye points |\n",
"| **Batching** | `predict()` runs all faces in one inference call |\n",
"| **Drop-in** | `get_landmarks()` matches `Landmark106` / `PIPNet` |\n",
"\n",
"### Next steps\n",
"\n",
"- [Landmarks documentation](https://yakhyo.github.io/uniface/modules/landmarks/)\n",
"- [Detection documentation](https://yakhyo.github.io/uniface/modules/detection/) — BlazeFace and the 6-keypoint layout\n",
"- [Coordinate systems](https://yakhyo.github.io/uniface/concepts/coordinate-systems/) — what `z` means\n",
"\n",
"> Face Mesh and BlazeFace weights originate from Google MediaPipe and are Apache-2.0 licensed.\n",
"> See [Licenses & Attribution](https://yakhyo.github.io/uniface/license-attribution/)."
]
}
],
"metadata": {
"kernelspec": {
"display_name": "base",
"language": "python",
"name": "python3"
},
"language_info": {
"codemirror_mode": {
"name": "ipython",
"version": 3
},
"file_extension": ".py",
"mimetype": "text/x-python",
"name": "python",
"nbconvert_exporter": "python",
"pygments_lexer": "ipython3",
"version": "3.13.5"
}
},
"nbformat": 4,
"nbformat_minor": 4
}

View File

@@ -1,5 +1,5 @@
site_name: UniFace
site_description: All-in-One Face Analysis Library with ONNX Runtime
site_description: A Unified Face Analysis Library for Python
site_author: Yakhyokhuja Valikhujaev
site_url: https://yakhyo.github.io/uniface
@@ -7,7 +7,7 @@ repo_name: yakhyo/uniface
repo_url: https://github.com/yakhyo/uniface
edit_uri: edit/main/docs/
copyright: Copyright &copy; 2025 Yakhyokhuja Valikhujaev
copyright: Copyright &copy; 2026 Yakhyokhuja Valikhujaev
theme:
name: material
@@ -15,7 +15,7 @@ theme:
palette:
- media: "(prefers-color-scheme)"
toggle:
icon: material/link
icon: material/brightness-auto
name: Switch to light mode
- media: "(prefers-color-scheme: light)"
scheme: default
@@ -48,6 +48,7 @@ theme:
- content.action.edit
- content.action.view
- content.tabs.link
- announce.dismiss
- toc.follow
icon:
@@ -77,7 +78,7 @@ extra:
link: https://x.com/y_valikhujaev
analytics:
provider: google
property: G-XXXXXXXXXX
property: G-FGEHR2K5ZE
extra_css:
- stylesheets/extra.css
@@ -134,6 +135,7 @@ nav:
- Quickstart: quickstart.md
- Notebooks: notebooks.md
- Model Zoo: models.md
- Datasets: datasets.md
- Tutorials:
- Image Pipeline: recipes/image-pipeline.md
- Video & Webcam: recipes/video-webcam.md
@@ -144,12 +146,17 @@ nav:
- API Reference:
- Detection: modules/detection.md
- Recognition: modules/recognition.md
- Tracking: modules/tracking.md
- Landmarks: modules/landmarks.md
- Attributes: modules/attributes.md
- Parsing: modules/parsing.md
- Matting: modules/matting.md
- Gaze: modules/gaze.md
- Head Pose: modules/headpose.md
- Anti-Spoofing: modules/spoofing.md
- Quality: modules/quality.md
- Privacy: modules/privacy.md
- Stores: modules/stores.md
- Guides:
- Overview: concepts/overview.md
- Inputs & Outputs: concepts/inputs-outputs.md

View File

@@ -1,7 +1,7 @@
[project]
name = "uniface"
version = "2.0.2"
description = "UniFace: A Comprehensive Library for Face Detection, Recognition, Landmark Analysis, Face Parsing, Gaze Estimation, Age, and Gender Detection"
version = "4.0.0rc2"
description = "UniFace: A Unified Face Analysis Library for Python"
readme = "README.md"
license = "MIT"
authors = [{ name = "Yakhyokhuja Valikhujaev", email = "yakhyo9696@gmail.com" }]
@@ -9,10 +9,11 @@ maintainers = [
{ name = "Yakhyokhuja Valikhujaev", email = "yakhyo9696@gmail.com" },
]
requires-python = ">=3.11,<3.14"
requires-python = ">=3.10,<3.15"
keywords = [
"face-detection",
"face-recognition",
"face-tracking",
"facial-landmarks",
"face-parsing",
"face-segmentation",
@@ -28,39 +29,53 @@ keywords = [
]
classifiers = [
"Development Status :: 4 - Beta",
"Development Status :: 5 - Production/Stable",
"Intended Audience :: Developers",
"Intended Audience :: Science/Research",
"Operating System :: OS Independent",
"Programming Language :: Python :: 3",
"Programming Language :: Python :: 3.10",
"Programming Language :: Python :: 3.11",
"Programming Language :: Python :: 3.12",
"Programming Language :: Python :: 3.13",
"Programming Language :: Python :: 3.14",
]
dependencies = [
"numpy>=1.21.0",
"opencv-python>=4.5.0",
"onnx>=1.12.0",
"onnxruntime>=1.16.0",
"scikit-image>=0.19.0",
"scikit-image>=0.22.0",
"scipy>=1.7.0",
"requests>=2.28.0",
"tqdm>=4.64.0",
]
[project.optional-dependencies]
dev = ["pytest>=7.0.0", "ruff>=0.4.0"]
gpu = ["onnxruntime-gpu>=1.16.0"]
cpu = [
"onnxruntime>=1.16.0; python_version >= '3.11'",
"onnxruntime>=1.16.0,<1.24; python_version < '3.11'",
]
gpu = [
"onnxruntime-gpu>=1.16.0; python_version >= '3.11'",
"onnxruntime-gpu>=1.16.0,<1.24; python_version < '3.11'",
]
dev = ["pytest>=7.0.0", "ruff>=0.4.0", "pre-commit>=3.0.0"]
docs = [
"mkdocs-material",
"pymdown-extensions",
"mkdocs-git-committers-plugin-2",
"mkdocs-git-revision-date-localized-plugin",
]
[project.urls]
Homepage = "https://github.com/yakhyo/uniface"
Repository = "https://github.com/yakhyo/uniface"
Documentation = "https://github.com/yakhyo/uniface/blob/main/README.md"
"Quick Start" = "https://github.com/yakhyo/uniface/blob/main/QUICKSTART.md"
"Model Zoo" = "https://github.com/yakhyo/uniface/blob/main/MODELS.md"
Documentation = "https://yakhyo.github.io/uniface"
"Quick Start" = "https://yakhyo.github.io/uniface/quickstart/"
"Model Zoo" = "https://yakhyo.github.io/uniface/models/"
[build-system]
requires = ["setuptools>=64", "wheel"]
requires = ["setuptools>=77"]
build-backend = "setuptools.build_meta"
[tool.setuptools]
@@ -71,7 +86,7 @@ uniface = ["py.typed"]
[tool.ruff]
line-length = 120
target-version = "py311"
target-version = "py310"
exclude = [
".git",
".ruff_cache",
@@ -101,6 +116,8 @@ select = [
"C4", # flake8-comprehensions
"SIM", # flake8-simplify
"RUF", # Ruff-specific rules
"D205", # pydocstyle: blank line between docstring summary and description
"D212", # pydocstyle: multi-line docstring summary starts on the first line
]
ignore = [
"E501", # Line too long (handled by formatter)
@@ -127,15 +144,6 @@ section-order = [
[tool.ruff.lint.pydocstyle]
convention = "google"
[tool.mypy]
python_version = "3.11"
warn_return_any = false
warn_unused_ignores = true
ignore_missing_imports = true
exclude = ["tests/", "scripts/", "examples/"]
# Disable strict return type checking for numpy operations
disable_error_code = ["no-any-return"]
[tool.bandit]
exclude_dirs = ["tests", "scripts", "examples"]
skips = ["B101", "B614"] # B101: assert, B614: torch.jit.load (models are SHA256 verified)

View File

@@ -1,8 +0,0 @@
numpy>=1.21.0
opencv-python>=4.5.0
onnx>=1.12.0
onnxruntime>=1.16.0
scikit-image>=0.19.0
requests>=2.28.0
pytest>=7.0.0
tqdm>=4.64.0

View File

@@ -1,15 +1,22 @@
# Copyright 2025 Yakhyokhuja Valikhujaev
# Copyright 2025-2026 Yakhyokhuja Valikhujaev
# Author: Yakhyokhuja Valikhujaev
# GitHub: https://github.com/yakhyo
"""Tests for AgeGender attribute predictor."""
from __future__ import annotations
import numpy as np
import pytest
from uniface.attribute import AgeGender, AttributeResult
from uniface.attribute import AgeGender, DemographyResult
from uniface.types import Face
def _make_face(bbox: list[int] | np.ndarray) -> Face:
"""Helper: build a minimal Face from a bounding box."""
bbox = np.asarray(bbox)
landmarks = np.zeros((5, 2), dtype=np.float32)
return Face(bbox=bbox, confidence=0.99, landmarks=landmarks)
@pytest.fixture
@@ -23,30 +30,30 @@ def mock_image():
@pytest.fixture
def mock_bbox():
return [100, 100, 300, 300]
def mock_face():
return _make_face([100, 100, 300, 300])
def test_model_initialization(age_gender_model):
assert age_gender_model is not None, 'AgeGender model initialization failed.'
def test_prediction_output_format(age_gender_model, mock_image, mock_bbox):
result = age_gender_model.predict(mock_image, mock_bbox)
assert isinstance(result, AttributeResult), f'Result should be AttributeResult, got {type(result)}'
def test_prediction_output_format(age_gender_model, mock_image, mock_face):
result = age_gender_model.predict(mock_image, mock_face)
assert isinstance(result, DemographyResult), f'Result should be DemographyResult, got {type(result)}'
assert isinstance(result.gender, int), f'Gender should be int, got {type(result.gender)}'
assert isinstance(result.age, int), f'Age should be int, got {type(result.age)}'
assert isinstance(result.sex, str), f'Sex should be str, got {type(result.sex)}'
def test_gender_values(age_gender_model, mock_image, mock_bbox):
result = age_gender_model.predict(mock_image, mock_bbox)
def test_gender_values(age_gender_model, mock_image, mock_face):
result = age_gender_model.predict(mock_image, mock_face)
assert result.gender in [0, 1], f'Gender should be 0 (Female) or 1 (Male), got {result.gender}'
assert result.sex in ['Female', 'Male'], f'Sex should be Female or Male, got {result.sex}'
def test_age_range(age_gender_model, mock_image, mock_bbox):
result = age_gender_model.predict(mock_image, mock_bbox)
def test_age_range(age_gender_model, mock_image, mock_face):
result = age_gender_model.predict(mock_image, mock_face)
assert 0 <= result.age <= 120, f'Age should be between 0 and 120, got {result.age}'
@@ -58,39 +65,52 @@ def test_different_bbox_sizes(age_gender_model, mock_image):
]
for bbox in test_bboxes:
result = age_gender_model.predict(mock_image, bbox)
face = _make_face(bbox)
result = age_gender_model.predict(mock_image, face)
assert result.gender in [0, 1], f'Failed for bbox {bbox}'
assert 0 <= result.age <= 120, f'Age out of range for bbox {bbox}'
def test_different_image_sizes(age_gender_model, mock_bbox):
def test_different_image_sizes(age_gender_model):
test_sizes = [(480, 640, 3), (720, 1280, 3), (1080, 1920, 3)]
face = _make_face([100, 100, 300, 300])
for size in test_sizes:
mock_image = np.random.randint(0, 255, size, dtype=np.uint8)
result = age_gender_model.predict(mock_image, mock_bbox)
result = age_gender_model.predict(mock_image, face)
assert result.gender in [0, 1], f'Failed for image size {size}'
assert 0 <= result.age <= 120, f'Age out of range for image size {size}'
def test_consistency(age_gender_model, mock_image, mock_bbox):
result1 = age_gender_model.predict(mock_image, mock_bbox)
result2 = age_gender_model.predict(mock_image, mock_bbox)
def test_consistency(age_gender_model, mock_image, mock_face):
result1 = age_gender_model.predict(mock_image, mock_face)
result2 = age_gender_model.predict(mock_image, mock_face)
assert result1.gender == result2.gender, 'Same input should produce same gender prediction'
assert result1.age == result2.age, 'Same input should produce same age prediction'
def test_face_enrichment(age_gender_model, mock_image, mock_face):
"""predict() must write gender & age back to the Face object."""
assert mock_face.gender is None
assert mock_face.age is None
result = age_gender_model.predict(mock_image, mock_face)
assert mock_face.gender == result.gender
assert mock_face.age == result.age
def test_bbox_list_format(age_gender_model, mock_image):
bbox_list = [100, 100, 300, 300]
result = age_gender_model.predict(mock_image, bbox_list)
face = _make_face([100, 100, 300, 300])
result = age_gender_model.predict(mock_image, face)
assert result.gender in [0, 1], 'Should work with bbox as list'
assert 0 <= result.age <= 120, 'Age should be in valid range'
def test_bbox_array_format(age_gender_model, mock_image):
bbox_array = np.array([100, 100, 300, 300])
result = age_gender_model.predict(mock_image, bbox_array)
face = _make_face(np.array([100, 100, 300, 300]))
result = age_gender_model.predict(mock_image, face)
assert result.gender in [0, 1], 'Should work with bbox as numpy array'
assert 0 <= result.age <= 120, 'Age should be in valid range'
@@ -104,7 +124,8 @@ def test_multiple_predictions(age_gender_model, mock_image):
results = []
for bbox in bboxes:
result = age_gender_model.predict(mock_image, bbox)
face = _make_face(bbox)
result = age_gender_model.predict(mock_image, face)
results.append(result)
assert len(results) == 3, 'Should have 3 predictions'
@@ -113,28 +134,26 @@ def test_multiple_predictions(age_gender_model, mock_image):
assert 0 <= result.age <= 120
def test_age_is_positive(age_gender_model, mock_image, mock_bbox):
def test_age_is_positive(age_gender_model, mock_image, mock_face):
for _ in range(5):
result = age_gender_model.predict(mock_image, mock_bbox)
result = age_gender_model.predict(mock_image, mock_face)
assert result.age >= 0, f'Age should be non-negative, got {result.age}'
def test_output_format_for_visualization(age_gender_model, mock_image, mock_bbox):
result = age_gender_model.predict(mock_image, mock_bbox)
def test_output_format_for_visualization(age_gender_model, mock_image, mock_face):
result = age_gender_model.predict(mock_image, mock_face)
text = f'{result.sex}, {result.age}y'
assert isinstance(text, str), 'Should be able to format as string'
assert 'Male' in text or 'Female' in text, 'Text should contain gender'
assert 'y' in text, "Text should contain 'y' for years"
def test_attribute_result_fields(age_gender_model, mock_image, mock_bbox):
"""Test that AttributeResult has correct fields for AgeGender model."""
result = age_gender_model.predict(mock_image, mock_bbox)
def test_attribute_result_fields(age_gender_model, mock_image, mock_face):
"""Test that DemographyResult has correct fields for AgeGender model."""
result = age_gender_model.predict(mock_image, mock_face)
# AgeGender should set gender and age
assert result.gender is not None
assert result.age is not None
# AgeGender should NOT set race and age_group (FairFace only)
assert result.race is None
assert result.age_group is None

123
tests/test_analyzer.py Normal file
View File

@@ -0,0 +1,123 @@
# Copyright 2025-2026 Yakhyokhuja Valikhujaev
# Author: Yakhyokhuja Valikhujaev
# GitHub: https://github.com/yakhyo
from __future__ import annotations
import numpy as np
import pytest
from uniface.analyzer import FaceAnalyzer
from uniface.attribute import BaseAttribute
from uniface.detection.base import BaseDetector
from uniface.types import Face
class StubDetector(BaseDetector):
"""Returns a fixed number of faces without any model or download."""
supports_landmarks = True
supports_alignment = True
def __init__(self, num_faces: int = 2) -> None:
super().__init__()
self.num_faces = num_faces
def detect(self, image: np.ndarray, **kwargs) -> list[Face]:
return [
Face(
bbox=np.array([10.0 * i, 10.0, 50.0 + 10.0 * i, 60.0]),
confidence=0.9,
landmarks=np.zeros((5, 2)),
)
for i in range(self.num_faces)
]
def preprocess(self, image: np.ndarray) -> np.ndarray:
return image
def postprocess(self, outputs, **kwargs):
return outputs
class AgeStampPredictor(BaseAttribute):
"""Enriches each face in-place and counts its calls."""
def __init__(self) -> None:
self.calls = 0
def _initialize_model(self) -> None: ...
def preprocess(self, image, *args):
return image
def postprocess(self, prediction):
return prediction
def predict(self, image: np.ndarray, face: Face) -> None:
self.calls += 1
face.age = 30
class FailingPredictor(BaseAttribute):
"""Always raises, to exercise the analyzer's per-predictor error handling."""
def _initialize_model(self) -> None: ...
def preprocess(self, image, *args):
return image
def postprocess(self, prediction):
return prediction
def predict(self, image: np.ndarray, face: Face) -> None:
raise RuntimeError('boom')
@pytest.fixture
def image():
return np.zeros((240, 320, 3), dtype=np.uint8)
def test_predictors_run_once_per_face(image):
predictor = AgeStampPredictor()
analyzer = FaceAnalyzer(detector=StubDetector(num_faces=3), recognizer=None, predictors=[predictor])
faces = analyzer.analyze(image)
assert predictor.calls == 3
assert all(face.age == 30 for face in faces)
def test_failing_predictor_does_not_break_the_pipeline(image):
"""One bad predictor must not abort analysis or starve the others."""
stamper = AgeStampPredictor()
analyzer = FaceAnalyzer(
detector=StubDetector(num_faces=2),
recognizer=None,
predictors=[FailingPredictor(), stamper],
)
faces = analyzer.analyze(image)
assert len(faces) == 2
assert stamper.calls == 2
def test_predictors_default_to_empty(image):
analyzer = FaceAnalyzer(detector=StubDetector(), recognizer=None)
assert analyzer.predictors == []
assert len(analyzer.analyze(image)) == 2
def test_repr_lists_predictors():
analyzer = FaceAnalyzer(detector=StubDetector(), recognizer=None, predictors=[AgeStampPredictor()])
assert 'predictors=[AgeStampPredictor]' in repr(analyzer)
assert 'recognizer' not in repr(analyzer)
def test_constructor_is_keyword_only():
with pytest.raises(TypeError):
FaceAnalyzer(StubDetector())

274
tests/test_blazeface.py Normal file
View File

@@ -0,0 +1,274 @@
# Copyright 2025-2026 Yakhyokhuja Valikhujaev
# Author: Yakhyokhuja Valikhujaev
# GitHub: https://github.com/yakhyo
from __future__ import annotations
from pathlib import Path
import threading
import cv2
import numpy as np
import pytest
from uniface.constants import BlazeFaceWeights
from uniface.detection import BlazeFace
from uniface.detection.blazeface import _weighted_nms
TEST_IMAGE = Path(__file__).resolve().parent.parent / 'assets' / 'einstein.png'
@pytest.fixture(scope='module')
def blazeface_model():
return BlazeFace(model_name=BlazeFaceWeights.DEFAULT)
@pytest.fixture(scope='module')
def face_image():
image = cv2.imread(str(TEST_IMAGE))
assert image is not None, f'Missing test asset: {TEST_IMAGE}'
return image
def test_model_initialization(blazeface_model):
assert blazeface_model is not None
assert blazeface_model.input_size == 128
assert blazeface_model.anchors.shape == (896, 2)
def test_does_not_support_alignment(blazeface_model):
"""The contract that keeps BlazeFace out of recognition pipelines."""
assert blazeface_model.supports_alignment is False
assert blazeface_model.supports_landmarks is True
assert blazeface_model.get_info()['supports_alignment'] is False
def test_detect_returns_faces(blazeface_model, face_image):
faces = blazeface_model.detect(face_image)
assert isinstance(faces, list)
assert len(faces) >= 1
for face in faces:
assert 0.0 <= face.confidence <= 1.0
def test_emits_six_keypoints(blazeface_model, face_image):
"""MediaPipe's layout, not the 5-point alignment template."""
face = blazeface_model.detect(face_image)[0]
assert face.landmarks.shape == (6, 2)
assert face.landmarks.dtype == np.float32
def test_first_two_keypoints_are_the_eyes(blazeface_model, face_image):
"""Row 0 is the viewer-left eye, row 1 the viewer-right — what FaceMesh reads."""
face = blazeface_model.detect(face_image)[0]
assert face.landmarks[0][0] < face.landmarks[1][0]
def test_bbox_is_well_formed(blazeface_model, face_image):
"""Boxes are ordered and non-degenerate, but not clipped to the frame."""
for face in blazeface_model.detect(face_image):
x1, y1, x2, y2 = face.bbox
assert x1 < x2 and y1 < y2
def test_bbox_stays_square_when_the_face_runs_off_frame(blazeface_model, face_image):
"""The box is left unclipped so FaceMesh's square-ROI rule still holds.
MediaPipe's `detection_to_roi` expands a *square* box by 1.5x. Clipping to the
frame breaks the squareness and shifts the centre, which drags the mesh off the
mouth on webcam close-ups — the exact case where the box overflows.
"""
# Crop tight around the face so the decoded box overflows the frame.
frame = cv2.resize(face_image[400:870, 380:850], (640, 480))
height, width = frame.shape[:2]
faces = blazeface_model.detect(frame)
assert faces, 'expected a detection on the close-up crop'
x1, y1, x2, y2 = faces[0].bbox
assert x1 < 0 or y1 < 0 or x2 > width or y2 > height, 'crop is not tight enough to exercise overflow'
assert (y2 - y1) / (x2 - x1) == pytest.approx(1.0, abs=0.02)
def test_no_faces_in_blank_images(blazeface_model):
"""Also covers letterboxing in both orientations."""
for shape in ((480, 640, 3), (640, 480, 3)):
assert blazeface_model.detect(np.zeros(shape, dtype=np.uint8)) == []
# Captured from mediapipe 0.10.14, mp.solutions.face_detection.FaceDetection(
# model_selection=0, min_detection_confidence=0.5), pixel coords.
# That is the blaze_face_short_range graph this class reimplements. Regenerating needs a
# Python 3.12 venv: 0.10.35 dropped mp.solutions.
MEDIAPIPE_REFERENCE = {
'einstein.png': {
'bbox': (335.2566, 346.1731, 891.4519, 902.2785),
'score': 0.852732,
'keypoints': (
(549.8562, 509.9917),
(777.7883, 505.4715),
(706.1156, 653.1544),
(686.1717, 762.5642),
(364.3153, 543.7503),
(839.8786, 537.1355),
),
},
'test_images/image0.jpg': {
'bbox': (115.3043, 111.4395, 447.7941, 443.9094),
'score': 0.883393,
'keypoints': (
(191.1302, 276.5923),
(328.5422, 206.7665),
(299.1591, 347.9716),
(333.2188, 389.0749),
(143.1687, 295.0384),
(419.6207, 154.5568),
),
},
'demos/src_man1.jpg': {
'bbox': (284.8197, 223.9135, 961.3036, 900.3185),
'score': 0.944123,
'keypoints': (
(492.2396, 430.3650),
(759.8115, 440.1673),
(623.2774, 621.8229),
(619.6940, 738.1563),
(349.4137, 456.7012),
(898.0755, 473.6671),
),
},
}
@pytest.mark.parametrize('asset', list(MEDIAPIPE_REFERENCE))
def test_matches_mediapipe_reference_output(blazeface_model, asset):
"""Pin the port to MediaPipe's own numbers so preprocessing drift cannot pass silently.
MediaPipe parity is the reason to pick this detector over SCRFD or RetinaFace, but
nothing else in the suite enforces it. Measured agreement is ~0.15px on boxes and
~0.24px on keypoints; the 1.5px tolerance leaves room for ONNX Runtime provider and
platform differences while still catching real drift. Verified to fail on all three
assets when the letterbox resampler is swapped for INTER_AREA, which shifts boxes
4-16px. These faces all sit inside the frame, so re-clipping is *not* caught here —
`test_bbox_stays_square_when_the_face_runs_off_frame` covers that.
"""
expected = MEDIAPIPE_REFERENCE[asset]
image = cv2.imread(str(TEST_IMAGE.parent / asset))
assert image is not None, f'Missing test asset: {asset}'
faces = blazeface_model.detect(image)
assert len(faces) == 1, f'MediaPipe finds exactly one face in {asset}'
face = faces[0]
assert face.bbox == pytest.approx(expected['bbox'], abs=1.5)
assert face.confidence == pytest.approx(expected['score'], abs=0.01)
assert face.landmarks == pytest.approx(np.array(expected['keypoints']), abs=1.5)
@pytest.mark.parametrize(
('label', 'bad_image'),
[
('float [0, 1]', np.random.rand(480, 640, 3).astype(np.float32)),
('grayscale', np.zeros((480, 640), dtype=np.uint8)),
('empty', np.zeros((0, 0, 3), dtype=np.uint8)),
],
)
def test_detect_rejects_unusable_images(blazeface_model, label, bad_image):
"""This detector warps its own input, bypassing the shared resize helpers.
A float image used to return zero faces silently and a grayscale one raised
an opaque IndexError, both unlike every other detector.
"""
with pytest.raises(ValueError):
blazeface_model.detect(bad_image)
def test_no_overflow_warning_on_extreme_logits(blazeface_model, face_image):
"""Anchor logits are clipped before the sigmoid; numpy must not warn."""
with np.errstate(all='raise'):
blazeface_model.detect(face_image)
def test_weighted_nms_blends_overlapping_boxes():
"""MediaPipe averages overlapping candidates rather than discarding them."""
columns = 5 + 2 * 6
rows = np.zeros((2, columns))
rows[0, :5] = [0.9, 0.50, 0.50, 0.20, 0.20]
rows[1, :5] = [0.3, 0.52, 0.52, 0.20, 0.20]
merged = _weighted_nms(rows, iou_threshold=0.3)
assert len(merged) == 1
assert merged[0, 0] == pytest.approx(0.9)
# Pulled toward the weaker box rather than ignoring it.
assert 0.50 < merged[0, 1] < 0.52
def test_weighted_nms_keeps_disjoint_boxes():
columns = 5 + 2 * 6
rows = np.zeros((2, columns))
rows[0, :5] = [0.9, 0.20, 0.20, 0.10, 0.10]
rows[1, :5] = [0.8, 0.80, 0.80, 0.10, 0.10]
assert len(_weighted_nms(rows, iou_threshold=0.3)) == 2
def _nms_within(rows: np.ndarray, iou_threshold: float, seconds: float = 5.0) -> np.ndarray:
"""Run `_weighted_nms` on a worker thread so a non-terminating loop fails instead of hanging."""
result: list[np.ndarray] = []
worker = threading.Thread(target=lambda: result.append(_weighted_nms(rows, iou_threshold)), daemon=True)
worker.start()
worker.join(seconds)
assert not worker.is_alive(), f'_weighted_nms did not terminate within {seconds}s'
return result[0]
def test_weighted_nms_terminates_when_threshold_excludes_self_overlap():
"""iou_threshold=1.0 fails the strict `>` even for the winner's self-IoU of 1.0."""
columns = 5 + 2 * 6
rows = np.zeros((2, columns))
rows[0, :5] = [0.9, 0.50, 0.50, 0.20, 0.20]
rows[1, :5] = [0.3, 0.52, 0.52, 0.20, 0.20]
with np.errstate(all='raise'):
merged = _nms_within(rows, iou_threshold=1.0)
# Nothing overlaps enough to blend, so both survive untouched.
assert len(merged) == 2
assert merged[0, 1] == pytest.approx(0.50)
assert merged[1, 1] == pytest.approx(0.52)
def test_weighted_nms_terminates_on_zero_area_box():
"""A degenerate box scores an IoU of 0 against itself."""
columns = 5 + 2 * 6
rows = np.zeros((1, columns))
rows[0, :5] = [0.9, 0.50, 0.50, 0.0, 0.0]
with np.errstate(all='raise'):
merged = _nms_within(rows, iou_threshold=0.3)
assert len(merged) == 1
assert merged[0, :5] == pytest.approx([0.9, 0.50, 0.50, 0.0, 0.0])
def test_analyzer_disables_recognition(blazeface_model, face_image):
"""The regression test for the alignment seam.
An unalignable detector must not crash the analyzer, must not silently
produce broken embeddings, and must say so once rather than per face.
"""
from uniface.analyzer import FaceAnalyzer
from uniface.recognition import ArcFace
analyzer = FaceAnalyzer(detector=blazeface_model, recognizer=ArcFace())
assert analyzer.recognizer is None
faces = analyzer.analyze(face_image)
assert all(face.embedding is None for face in faces)

106
tests/test_centerface.py Normal file
View File

@@ -0,0 +1,106 @@
# Copyright 2025-2026 Yakhyokhuja Valikhujaev
# Author: Yakhyokhuja Valikhujaev
# GitHub: https://github.com/yakhyo
from __future__ import annotations
import numpy as np
import pytest
from uniface.constants import CenterFaceWeights
from uniface.detection import CenterFace
@pytest.fixture
def centerface_model():
return CenterFace(
model_name=CenterFaceWeights.DEFAULT,
confidence_threshold=0.35,
nms_threshold=0.3,
)
def test_model_initialization(centerface_model):
assert centerface_model is not None, 'Model initialization failed.'
def test_invalid_input_size():
with pytest.raises(ValueError, match='strictly positive'):
CenterFace(input_size=(0, 480))
@pytest.mark.parametrize('size', [(480, 640), (519, 713), (720, 1280)])
def test_inference_shape_is_padded_to_multiple_of_32(centerface_model, size):
"""The FPN needs both sides divisible by 32, whatever the caller passes in."""
image = np.zeros((*size, 3), dtype=np.uint8)
resized, scale_w, scale_h = centerface_model._resize(image)
assert resized.shape[0] % 32 == 0
assert resized.shape[1] % 32 == 0
assert scale_w == pytest.approx(resized.shape[1] / size[1])
assert scale_h == pytest.approx(resized.shape[0] / size[0])
def test_large_input_is_capped_but_small_input_is_not_upscaled(centerface_model):
"""input_size bounds cost on huge images without touching ordinary frames."""
detector = CenterFace(input_size=(640, 640))
big, _, _ = detector._resize(np.zeros((2000, 4000, 3), dtype=np.uint8))
assert big.shape[1] <= 640 + 31 and big.shape[0] <= 320 + 31
# A 640x480 camera frame runs at its own resolution, not letterboxed to 640x640
frame, scale_w, scale_h = detector._resize(np.zeros((480, 640, 3), dtype=np.uint8))
assert frame.shape[:2] == (480, 640)
assert (scale_w, scale_h) == (1.0, 1.0)
def test_native_input_size_never_rescales(centerface_model):
"""input_size=None reproduces upstream: native resolution, rounded up only."""
detector = CenterFace(input_size=None)
resized, _, _ = detector._resize(np.zeros((1080, 1920, 3), dtype=np.uint8))
assert resized.shape[:2] == (1088, 1920)
def test_inference_on_640x640_image(centerface_model):
mock_image = np.random.randint(0, 255, (640, 640, 3), dtype=np.uint8)
faces = centerface_model.detect(mock_image)
assert isinstance(faces, list), 'Detections should be a list.'
for face in faces:
# Face is a dataclass, check attributes exist
assert hasattr(face, 'bbox'), "Each detection should have a 'bbox' attribute."
assert hasattr(face, 'confidence'), "Each detection should have a 'confidence' attribute."
assert hasattr(face, 'landmarks'), "Each detection should have a 'landmarks' attribute."
bbox = face.bbox
assert len(bbox) == 4, 'BBox should have 4 values (x1, y1, x2, y2).'
landmarks = face.landmarks
assert len(landmarks) == 5, 'Should have 5 landmark points.'
assert all(len(pt) == 2 for pt in landmarks), 'Each landmark should be (x, y).'
def test_confidence_threshold(centerface_model):
mock_image = np.random.randint(0, 255, (640, 640, 3), dtype=np.uint8)
faces = centerface_model.detect(mock_image)
for face in faces:
confidence = face.confidence
assert confidence >= 0.35, f'Detection has confidence {confidence} below threshold 0.35'
def test_no_faces_detected(centerface_model):
empty_image = np.zeros((640, 640, 3), dtype=np.uint8)
faces = centerface_model.detect(empty_image)
assert len(faces) == 0, 'Should detect no faces in a blank image.'
def test_different_input_sizes(centerface_model):
test_sizes = [(480, 640, 3), (720, 1280, 3), (1080, 1920, 3)]
for size in test_sizes:
mock_image = np.random.randint(0, 255, size, dtype=np.uint8)
faces = centerface_model.detect(mock_image)
assert isinstance(faces, list), f'Should return list for size {size}'

Some files were not shown because too many files have changed in this diff Show More