mirror of
https://github.com/yakhyo/uniface.git
synced 2026-05-16 05:27:53 +00:00
231 lines
7.3 KiB
Plaintext
231 lines
7.3 KiB
Plaintext
{
|
|
"cells": [
|
|
{
|
|
"cell_type": "markdown",
|
|
"metadata": {},
|
|
"source": [
|
|
"# Gaze Estimation with UniFace\n",
|
|
"\n",
|
|
"<div style=\"display:flex; flex-wrap:wrap; align-items:center;\">\n",
|
|
" <a style=\"margin-right:10px; margin-bottom:6px;\" href=\"https://pepy.tech/projects/uniface\"><img alt=\"PyPI Downloads\" src=\"https://static.pepy.tech/personalized-badge/uniface?period=total&units=international_system&left_color=grey&right_color=blue&left_text=Downloads\"></a>\n",
|
|
" <a style=\"margin-right:10px; margin-bottom:6px;\" href=\"https://pypi.org/project/uniface/\"><img alt=\"PyPI Version\" src=\"https://img.shields.io/pypi/v/uniface.svg\"></a>\n",
|
|
" <a style=\"margin-right:10px; margin-bottom:6px;\" href=\"https://opensource.org/licenses/MIT\"><img alt=\"License\" src=\"https://img.shields.io/badge/License-MIT-blue.svg\"></a>\n",
|
|
" <a style=\"margin-bottom:6px;\" href=\"https://github.com/yakhyo/uniface\"><img alt=\"GitHub Stars\" src=\"https://img.shields.io/github/stars/yakhyo/uniface.svg?style=social\"></a>\n",
|
|
"</div>\n",
|
|
"\n",
|
|
"**UniFace** is a lightweight, production-ready Python library for face detection, recognition, tracking, landmark analysis, face parsing, gaze estimation, and face attributes.\n",
|
|
"\n",
|
|
"🔗 **GitHub**: [github.com/yakhyo/uniface](https://github.com/yakhyo/uniface) | 📚 **Docs**: [yakhyo.github.io/uniface](https://yakhyo.github.io/uniface)\n",
|
|
"\n",
|
|
"---\n",
|
|
"\n",
|
|
"This notebook demonstrates gaze estimation using the **UniFace** library.\n",
|
|
"\n",
|
|
"## 1. Install UniFace"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": null,
|
|
"metadata": {},
|
|
"outputs": [],
|
|
"source": [
|
|
"%pip install -q \"uniface[cpu]\"\n",
|
|
"\n",
|
|
"# Clone repo for assets (Colab only)\n",
|
|
"import os\n",
|
|
"if 'COLAB_GPU' in os.environ or 'COLAB_RELEASE_TAG' in os.environ:\n",
|
|
" if not os.path.exists('uniface'):\n",
|
|
" !git clone --depth 1 https://github.com/yakhyo/uniface.git\n",
|
|
" os.chdir('uniface/examples')"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"metadata": {},
|
|
"source": [
|
|
"## 2. Import Libraries"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": null,
|
|
"metadata": {},
|
|
"outputs": [],
|
|
"source": [
|
|
"import cv2\n",
|
|
"import numpy as np\n",
|
|
"import matplotlib.pyplot as plt\n",
|
|
"from pathlib import Path\n",
|
|
"from PIL import Image\n",
|
|
"\n",
|
|
"import uniface\n",
|
|
"from uniface.detection import RetinaFace\n",
|
|
"from uniface.gaze import MobileGaze\n",
|
|
"from uniface.draw import draw_gaze\n",
|
|
"\n",
|
|
"print(f\"UniFace version: {uniface.__version__}\")"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"metadata": {},
|
|
"source": [
|
|
"## 3. Initialize Models"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": null,
|
|
"metadata": {},
|
|
"outputs": [],
|
|
"source": [
|
|
"# Initialize face detector\n",
|
|
"detector = RetinaFace(confidence_threshold=0.5)\n",
|
|
"\n",
|
|
"# Initialize gaze estimator (uses ResNet34 by default)\n",
|
|
"gaze_estimator = MobileGaze()"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"metadata": {},
|
|
"source": [
|
|
"## 4. Process All Test Images\n",
|
|
"\n",
|
|
"Display original images in the first row and gaze-annotated images in the second row."
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": null,
|
|
"metadata": {},
|
|
"outputs": [],
|
|
"source": [
|
|
"# Get all test images\n",
|
|
"test_images_dir = Path('../assets/test_images')\n",
|
|
"test_images = sorted(test_images_dir.glob('*.jpg'))\n",
|
|
"\n",
|
|
"# Store original and processed images\n",
|
|
"original_images = []\n",
|
|
"processed_images = []\n",
|
|
"\n",
|
|
"for image_path in test_images:\n",
|
|
" print(f\"Processing: {image_path.name}\")\n",
|
|
"\n",
|
|
" # Load image\n",
|
|
" image = cv2.imread(str(image_path))\n",
|
|
" original = image.copy()\n",
|
|
"\n",
|
|
" # Detect faces\n",
|
|
" faces = detector.detect(image)\n",
|
|
" print(f' Detected {len(faces)} face(s)')\n",
|
|
"\n",
|
|
" # Estimate gaze for each face\n",
|
|
" for i, face in enumerate(faces):\n",
|
|
" x1, y1, x2, y2 = map(int, face.bbox[:4])\n",
|
|
" face_crop = image[y1:y2, x1:x2]\n",
|
|
"\n",
|
|
" if face_crop.size > 0:\n",
|
|
" gaze = gaze_estimator.estimate(face_crop)\n",
|
|
" pitch_deg = np.degrees(gaze.pitch)\n",
|
|
" yaw_deg = np.degrees(gaze.yaw)\n",
|
|
"\n",
|
|
" print(f' Face {i+1}: pitch={pitch_deg:.1f}°, yaw={yaw_deg:.1f}°')\n",
|
|
"\n",
|
|
" # Draw gaze without angle text\n",
|
|
" draw_gaze(image, face.bbox, gaze.pitch, gaze.yaw, draw_angles=False)\n",
|
|
"\n",
|
|
" # Convert BGR to RGB for display\n",
|
|
" original_rgb = cv2.cvtColor(original, cv2.COLOR_BGR2RGB)\n",
|
|
" processed_rgb = cv2.cvtColor(image, cv2.COLOR_BGR2RGB)\n",
|
|
"\n",
|
|
" original_images.append(original_rgb)\n",
|
|
" processed_images.append(processed_rgb)\n",
|
|
"\n",
|
|
"print(f\"\\nProcessed {len(test_images)} images\")"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"metadata": {},
|
|
"source": [
|
|
"## 5. Visualize Results\n",
|
|
"\n",
|
|
"**First row**: Original images \n",
|
|
"**Second row**: Images with gaze direction arrows"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": null,
|
|
"metadata": {},
|
|
"outputs": [],
|
|
"source": [
|
|
"num_images = len(original_images)\n",
|
|
"\n",
|
|
"# Create figure with 2 rows\n",
|
|
"fig, axes = plt.subplots(2, num_images, figsize=(4*num_images, 8))\n",
|
|
"\n",
|
|
"# Handle case where there's only one image\n",
|
|
"if num_images == 1:\n",
|
|
" axes = axes.reshape(2, 1)\n",
|
|
"\n",
|
|
"# First row: Original images\n",
|
|
"for i, img in enumerate(original_images):\n",
|
|
" axes[0, i].imshow(img)\n",
|
|
" axes[0, i].set_title(f'Original {i}', fontsize=12)\n",
|
|
" axes[0, i].axis('off')\n",
|
|
"\n",
|
|
"# Second row: Gaze-annotated images\n",
|
|
"for i, img in enumerate(processed_images):\n",
|
|
" axes[1, i].imshow(img)\n",
|
|
" axes[1, i].set_title(f'Gaze Estimation {i}', fontsize=12)\n",
|
|
" axes[1, i].axis('off')\n",
|
|
"\n",
|
|
"plt.tight_layout()\n",
|
|
"plt.show()"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"metadata": {},
|
|
"source": [
|
|
"## Notes\n",
|
|
"\n",
|
|
"- **Input**: Gaze estimation requires a face crop (obtained from face detection)\n",
|
|
"- **Output**: Returns a `GazeResult` object with `pitch` and `yaw` attributes (angles in radians)\n",
|
|
"- **Visualization**: `draw_gaze()` automatically draws bounding box and gaze arrow\n",
|
|
"- **Models**: Trained on Gaze360 dataset with diverse head poses\n",
|
|
"- **Performance**: MAE (Mean Absolute Error) ranges from 11-13 degrees\n",
|
|
"\n",
|
|
"### Tips for Best Results\n",
|
|
"- Ensure faces are clearly visible and well-lit\n",
|
|
"- Works best with frontal to semi-profile faces\n",
|
|
"- Accuracy may vary with extreme head poses or occlusions"
|
|
]
|
|
}
|
|
],
|
|
"metadata": {
|
|
"kernelspec": {
|
|
"display_name": "base",
|
|
"language": "python",
|
|
"name": "python3"
|
|
},
|
|
"language_info": {
|
|
"codemirror_mode": {
|
|
"name": "ipython",
|
|
"version": 3
|
|
},
|
|
"file_extension": ".py",
|
|
"mimetype": "text/x-python",
|
|
"name": "python",
|
|
"nbconvert_exporter": "python",
|
|
"pygments_lexer": "ipython3",
|
|
"version": "3.13.5"
|
|
}
|
|
},
|
|
"nbformat": 4,
|
|
"nbformat_minor": 4
|
|
}
|