9147dfe2f3
* Travis OpenCV4 update, testing Ubuntu with new OpenCV * Fix to Ubuntu travis * Another attempt at OpenCV 4.0 for Ubuntu * And another OpenCV attempt. * Simplifying the travis script * Ubuntu OpenCV 4 support. * Updating to OpenCV 4, for x64 windows. * Fixes to move to OpenCV 4 on windows. * Travis fix for OpenCV 4 on OSX * Renaming a lib. * Travis opencv4 fix. * Building OpenCV4 versions using appveyor. * Attempt mac travis fix. * Small travis fix. * Travis fix attempt. * First iteration in boost removal and upgrade to C++17 * Test with ocv 4.0 * Moving filesystem out of stdafx * Some more boost testing with cmake. * More CMAKE options * More compiler flag changes * Another attempt at compiler options. * Another attempt. * More filesystem stuff. * Linking to filesystem. * Cmake fix with target linking. * Attempting travis with g++-8 * Attempting to setup g++8 on travis linux. * Another travis change. * Adding OpenBLAS to travis and removing g++-8 * Fixing typo * More travis experiments. * More travis debugging. * A small directory change. * Adding some more travis changes. * travis typo fix. * Some reordering of travis, for cleaner yml * Removing `using namespace std` in order to avoid clash with byte and to make the code more consistent. * Working towards removing std::filesystem requirement, allow boost::filesystem as well. * Making boost an optional dependency * Fixing std issue. * Fixing cmake issue. * Fixing the precompiled header issue. * Another cmake boost fix. * Including missing files. * Removing unnecessary includes. * Removing more includes. * Changes to appveyor build, proper removal of VS2015 * If boost is present, do not need to link to filesystem. * Removing un-needed link library. * oops * Mac attempt at opencv4 travis. * Upgrading OCV to 4.1 on VS2018 * Downloading OpenCV binaries through a script * Triger an appveyor build. * Upgrading VS version. * Attempting VS2017 build * Adding win-32 libraries for OpenCV 4.1 * Adding OpenCV 32 bit libraries.
OpenFace 2.1.0: an open source facial behavior analysis toolkit
Over the past few years, there has been an increased interest in automatic facial behavior analysis and understanding. We present OpenFace – a tool intended for computer vision and machine learning researchers, affective computing community and people interested in building interactive applications based on facial behavior analysis. OpenFace is the first toolkit capable of facial landmark detection, head pose estimation, facial action unit recognition, and eye-gaze estimation with available source code for both running and training the models. The computer vision algorithms which represent the core of OpenFace demonstrate state-of-the-art results in all of the above mentioned tasks. Furthermore, our tool is capable of real-time performance and is able to run from a simple webcam without any specialist hardware.
OpenFace is an implementation of a number of research papers from the Multicomp group, Language Technologies Institute at the Carnegie Mellon University and Rainbow Group, Computer Laboratory, University of Cambridge. The founder of the project and main developer is Tadas Baltrušaitis.
Special thanks goes to Louis-Philippe Morency and his MultiComp Lab at Carnegie Mellon University for help in writing and testing the code, Erroll Wood for the gaze estimation work, and Amir Zadeh and Yao Chong Lim on work on the CE-CLM model.
WIKI
For instructions of how to install/compile/use the project please see WIKI
Functionality
The system is capable of performing a number of facial analysis tasks:
- Facial Landmark Detection
- Facial Landmark and head pose tracking (links to YouTube videos)
- Facial Action Unit Recognition
- Gaze tracking (image of it in action)
- Facial Feature Extraction (aligned faces and HOG features)
Citation
If you use any of the resources provided on this page in any of your publications we ask you to cite the following work and the work for a relevant submodule you used.
Overall system
OpenFace 2.0: Facial Behavior Analysis Toolkit Tadas Baltrušaitis, Amir Zadeh, Yao Chong Lim, and Louis-Philippe Morency, IEEE International Conference on Automatic Face and Gesture Recognition, 2018
Facial landmark detection and tracking
Convolutional experts constrained local model for facial landmark detection A. Zadeh, T. Baltrušaitis, and Louis-Philippe Morency. Computer Vision and Pattern Recognition Workshops, 2017
Constrained Local Neural Fields for robust facial landmark detection in the wild Tadas Baltrušaitis, Peter Robinson, and Louis-Philippe Morency. in IEEE Int. Conference on Computer Vision Workshops, 300 Faces in-the-Wild Challenge, 2013.
Eye gaze tracking
Rendering of Eyes for Eye-Shape Registration and Gaze Estimation Erroll Wood, Tadas Baltrušaitis, Xucong Zhang, Yusuke Sugano, Peter Robinson, and Andreas Bulling in IEEE International Conference on Computer Vision (ICCV), 2015
Facial Action Unit detection
Cross-dataset learning and person-specific normalisation for automatic Action Unit detection Tadas Baltrušaitis, Marwa Mahmoud, and Peter Robinson in Facial Expression Recognition and Analysis Challenge, IEEE International Conference on Automatic Face and Gesture Recognition, 2015
Commercial license
For inquiries about the commercial licensing of the OpenFace toolkit please visit https://www.flintbox.com/public/project/50632/
Final remarks
I did my best to make sure that the code runs out of the box but there are always issues and I would be grateful for your understanding that this is research code and a research project. If you encounter any problems/bugs/issues please contact me on github or by emailing me at tadyla@gmail.com for any bug reports/questions/suggestions. I prefer questions and bug reports on github as that provides visibility to others who might be encountering same issues or who have the same questions.
Copyright
Copyright can be found in the Copyright.txt
You have to respect boost, dlib, OpenBLAS, and OpenCV licenses.
Furthermore you have to respect the licenses of the datasets used for model training - https://github.com/TadasBaltrusaitis/OpenFace/wiki/Datasets