Facelift Package¶

`facelift.types`¶

Contains module-wide used types.

facelift.types.Frame¶

An aliased type for a basic numpy array that gets given to use via OpenCV.

Type: NDArray[(Any, Any, 3), UInt8]

facelift.types.Point¶

A single x, y coordinate that describes a single positional point.

Type: NDArray[(2,), Int32]

facelift.types.PointSequence¶

A sequence of points that is typically used to describe a face feature or a line during rendering.

Type: NDArray[(Any, 2), Int32]

facelift.types.Encoding¶

A 128 dimension encoding of a detected face for a given frame.

Type: NDArray[(128,), Int32]

facelift.types.Detector¶

Callable that takes a frame and an upsample count and discovers the bounds of a face within the frame.

Type: Callable[[Frame, int], PointSequence]

facelift.types.Predictor¶

Callable which takes a frame and detected face bounds to discover the shape and features within the face.

Type: Callable[[Frame, dlib.rectangle], dlib.full_object_detection]

class facelift.types.Encoder(*args, **kwds)[source]¶

Protocol class for dlib.face_recognition_model_v1..

compute_face_descriptor(frame, face, num_jitters=0, padding=0.25)[source]¶

Compute a descriptor for a detected face frame.

Parameters

frame (Frame) – The frame containing just the detected face.
face (dlib.full_object_detection) – The raw detected face bounds within the given face frame.
num_jitters (int) – The number of jitters to run through the dector projection. Defaults to 0.
padding (float) – The default padding around the face. Defaults to 0.25.

Returns

The face descriptor (encoding).

Return type

dlib.vector

class facelift.types.Face(raw, landmarks, frame)[source]¶

Describes a detected face.

Parameters

raw (dlib.full_object_detection) – The raw dlib object detection container.
landmarks (Dict[FaceFeature, PointSequence]) – Mapping of extracted face features to the sequence of points describing those features.
frame (Frame) – The base non-normalized cropped frame of just the face.

property rectangle¶

Point sequence representation of the detected face’s bounds.

This property is useful for properly positioning text around the detected face as the draw_text() needs a text container to be defined.

Returns: A sequence of 2 points indicating the top-left and bottom-right corners of the detected face’s bounds.
Return type: PointSequence

class facelift.types.FaceFeature(value)[source]¶

Enumeration of features of a face that we can detect.

NOSE¶: The nose of a face.

JAW¶: The jaw line of a face.

MOUTH¶: The external bounds of the mouth of a face.

INNER_MOUTH¶: The internal bounds of the mouth of a face.

RIGHT_EYE¶: The external and internal bounds of the right eye of a face.

LEFT_EYE¶: The external and internal bounds of the left eye of a face.

RIGHT_EYEBROW¶: The right eyebrow of a face.

LEFT_EYEBROW¶: The left eyebrow of a face.

FOREHEAD¶: The forehead curvature of a face.

class facelift.types.MediaType(value)[source]¶

Enumeration of acceptable media types for processing.

IMAGE¶: Defines media that contains just a single frame to process.

VIDEO¶: Defines media that contains a known number of frames to process.

STREAM¶: Defines media that contains an unknown number of frames to process.

`facelift.capture`¶

Contains helpers and managers for capturing content from various sources.

Among the included functions, iter_media_frames() and iter_stream_frames() should really be all you ever care about. With these two functions you can iterate over either some image or video (as supported by OpenCV) or frames streamed directly from a webcam. The frames output by these generators are numpy arrays that are considered Frame instances and are used throughout the project.

For example, if I had a video file ~/my-file.mp4 and wanted to iterate over all available frames within the video, I would do use iter_media_frames() like the following:

from pathlib import Path
from facelift.capture import iter_media_frames

MY_FILE = Path("~/my-file.mp4").expanduser()
for frame in iter_media_frames(MY_FILE):
    print(frame)

The same works for images, however only 1 frame will ever be yielded from the generator.

If you want to instead iterate over the frames from a webcam, you should use the iter_stream_frames() like the following:

from facelift.capture import iter_stream_frames
# will default the device id to a value of 0
# this means OpenCV will attempt to discover the first available webcam
for frame in iter_stream_frames():
    print(frame)

# if you have 2 webcams enabled and want to instead use the 2nd one, you should
# specify a device index of 1 like this
for frame in iter_stream_frames(1):
    print(frame)

facelift.capture.file_capture(filepath)[source]¶

Context manager to open a given filepath for frame capture.

This is just a simple context manager wrapper around the base media_capture() manager to ensure that a given filepath exists and is a supported media type before attempting to build a capture around it.

Examples

>>> from pathlib import Path
>>> from facelift.capture import file_capture
>>> MY_FILEPATH = Path("~/my-file.mp4").expanduser()
>>> with file_capture(MY_FILEPATH) as capture:
...     print(capture)
<VideoCapture 0x1234567890>

Parameters

filepath (Path) – The filepath to open for capture

Raises

FileNotFoundError – When the given filepath doesn’t exist
ValueError – When the given filepath is not a supported media type

Yields

cv2.VideoCapture – A capturer that allows for reading frames from the given media filepath

Return type

Generator[VideoCapture, None, None]

facelift.capture.iter_media_frames(media_filepath, loop=False)[source]¶

Iterate over frames from a given supported media file.

Examples

>>> from pathlib import Path
>>> from facelift.capture import iter_media_frames
>>> MEDIA_PATH = Path("~/my-media.mp4").expanduser()
>>> for frame in iter_media_frames(MEDIA_PATH):
...     # do something with the frame

Parameters

media_filepath (Path) – The filepath to the media to read frames from.
loop (bool) – Flag that indicates if capture should reset to starting frame once all frames have been read. Defaults to False

Yields

Frame – A frame read from the given media file

Return type

Generator[Type[ndarray], None, None]

facelift.capture.iter_stream_frames(stream_type=None)[source]¶

Iterate over frames from a given streaming device.

By default this iterator will attempt to connect to the first available webcam and yield the webcam’s streamed frames. You can specify the appropriate device index 0-99 (0 being the default), or a custom stream type defined by the OpenCV video IO enum.

Examples

>>> from facelift.capture import iter_stream_frames
>>> # iterate over frames available from the second available webcam
>>> for frame in iter_stream_frames(1):
...     # do something with the frame

Parameters: stream_type (Optional[int], optional) – The stream type to attempt to open.
Yields: Frame – A read frame from the given streaming device
Return type: Generator[Type[ndarray], None, None]

facelift.capture.media_capture(media, media_type)[source]¶

General purpose media capture context manager.

This context manager is basically just a wrapper around the provided VideoCapture constructor along with some capturing destruction logic. The provided media can either be a filepath to capture frames off of or a device id as defined by the OpenCV video IO enum.

In most all cases where you just want to build a capture off of your default webcam, you should just be giving a media of 0.

Examples

>>> # build a media capture for a specific media file
>>> from facelift.capture import media_capture
>>> with media_capture("/home/a-user/Desktop/test.mp4") as capture:
...     print(capture)
<VideoCapture 0x1234567890>

>>> # build a media capture around the first available webcam
>>> from facelift.capture import media_capture
>>> with media_capture(0) as capture:
...     print(capture)
<VideoCapture 0x1234567890>

Parameters

media (Union[str, int]) – The media to build a capturer for
media_type (MediaType) –

Raises

ValueError – On failure to open the given media for capture

Yields

cv2.VideoCapture – A capturer that allows for reading sequential frames

Return type

Generator[VideoCapture, None, None]

facelift.capture.stream_capture(stream_type=None)[source]¶

Context manager to open a stream for frame capture.

By default this context manager will just attempt to connect to open capturing on any available webcams or connected cameras. You can get more specific about what device you would like to open a capturer on by supplying a different stream type. These stream types come directly from the OpenCV video IO enum.

Examples

>>> # build a frame capture from the first available webcam
>>> from facelift.capture import stream_capture
>>> with stream_capture() as capture:
...     print(capture)
<VideoCapture 0x1234567890>

>>> # build a frame capture from the second available webcam
>>> from facelift.capture import stream_capture
>>> with stream_capture(1) as capture:
...     print(capture)
<VideoCapture 0x1234567890>

Parameters: stream_type (Optional[int], optional) – The stream type to open
Raises: ValueError – When the given stream device fails to be opened for capture
Yields: cv2.VideoCapture – A capturer that allows for reading frames from the defined stream type
Return type: Generator[VideoCapture, None, None]

`facelift.transform`¶

Contains some common necessary frame transformation helper methods.

These transformation methods are useful for optimizing face detection in frames. Typically face detection takes much longer the more pixels there are to consider. Therefore, using scale() or resize() will help you speed up detection.

These helper transforms can be composed together to produce apply multiple operations on a single frame. For example, if we wanted to first downscale by half and then rotate a frame by 90 degrees, we could do something like the following:

from facelift.transform import rotate, scale
transformed_frame = rotate(scale(frame, 0.5), 90)

facelift.transform.DEFAULT_INTERPOLATION¶

The default type of interpolation to use in transforms that require an interpolation method. Defaults to cv2.INTER_AREA.

Type: int

facelift.transform.adjust(frame, brightness=None, sharpness=None)[source]¶

Adjust the brightness or sharpness of a frame.

Examples

>>> from facelift.transform import adjust
>>> sharper_frame = adjust(frame, sharpness=1.4)
>>> brighter_frame = adjust(frame, brightness=10)
>>> sharper_and_brighter_frame = adjust(frame, sharpness=1.4, brightness=10)

Parameters

frame (Frame) – The frame to adjust
brightness (Optional[int], optional) – The new brightness of the frame (can be negative, default is 0). Defaults to 0.
sharpness (Optional[float], optional) – The new sharpness of the frame (0.0 is black, default is 1.0). Defaults to 1.0.

Returns

The newly adjusted frame

Return type

Frame

facelift.transform.copy(frame)[source]¶

Copy the given frame to a new location in memory.

Examples

>>> from facelift.transform import copy
>>> copied_frame = copy(frame)
>>> assert frame == copied_frame
>>> assert frame is not copied_frame

Parameters: frame (Frame) – The frame to copy
Returns: An exact copy of the given frame
Return type: Frame

facelift.transform.crop(frame, start, end)[source]¶

Crop the given frame between two top-left to bottom-right points.

Examples

Crop a frame from the first pixel to the center pixel.

>>> from facelift.transform import crop
>>> assert frame.shape[:1] == [512, 512]
>>> cropped_frame = crop(frame, (0, 0), (256, 256))
>>> assert cropped_frame.shape[:1] == [256, 256]

Parameters

frame (Frame) – The frame to crop
start (Tuple[int, int]) – The top-left point to start the crop at
end (Tuple[int, int]) – The bottom-right point to end the crop at

Raises

ValueError – When the given starting crop point appears after the given ending crop point

Returns

The newly cropped frame

Return type

Frame

facelift.transform.flip(frame, x_axis=False, y_axis=False)[source]¶

Flip the given frame over either or both the x and y axis.

Examples

>>> from facelift.transform import flip
>>> vertically_flipped_frame = flip(frame, x_axis=True)
>>> horizontally_flipped_frame = flip(frame, y_axis=True)
>>> inverted_frame = flip(frame, x_axis=True, y_axis=True)

Parameters

frame (Frame) – The frame to flip
x_axis (bool, optional) – Flag indicating the frame should be flipped vertically. Defaults to False.
y_axis (bool, optional) – Flag indicating the frame should be flipped horizontally. Defaults to False.

Returns

The newly flipped frame

Return type

Frame

facelift.transform.grayscale(frame)[source]¶

Convert the given frame to grayscale.

This helper is useful sometimes for classification as color doesn’t matter as much during face encoding.

Examples

>>> from facelift.transform import grayscale
>>> grayscale_frame = grayscale(bgr_frame)

Parameters: frame (Frame) – The BGR frame to convert to grayscale
Returns: The newly grayscaled frame
Return type: Frame

facelift.transform.resize(frame, width=None, height=None, lock_aspect=True, interpolation=cv2.INTER_AREA)[source]¶

Resize a given frame to a given width and/or height.

If both width and height are given, the frame will be resized accordingly.
If only one of width or height is given, the frame will be resized according to the provided dimension (either width or height).
- As long as lock_aspect is truthy, the unprovided dimension will be adjusted to maintain the original aspect-ratio of the frame.
- If lock_aspect is falsy, the resize operation will only scale the provided dimension while keeping the original size of the unprovided dimension.

Examples

Resize a frame’s width while keeping the height relative:

>>> from facelift.transform import resize
>>> assert frame.shape[:1] == [512, 512]
>>> resized_frame = resize(frame, width=256, lock_aspect=True)
>>> assert resized_frame.shape[:1] == [256, 256]

Resize a frame’s width while keeping the original height:

>>> from facelift.transform import resize
>>> assert frame.shape[:1] == [512, 512]
>>> resized_frame = resize(frame, width=256, lock_aspect=False)
>>> assert resized_frame.shape[:1] == [512, 256]

Resize both a frame’s width and height:

>>> from facelift.transform import resize
>>> assert frame.shape[:1] == [512, 512]
>>> resized_frame = resize(frame, width=256, height=128)
>>> assert resized_frame.shape[:1] == [128, 256]

Parameters

frame (Frame) – The frame to resize
width (Optional[int], optional) – The exact width to resize the frame to.
height (Optional[int], optional) – The exact height to resize the frame to.
lock_aspect (bool, optional) – Whether to keep the width and height relative when only given one value. Defaults to True.
interpolation (int, optional) – The type of interpolation to use in the resize operation. Defaults to DEFAULT_INTERPOLATION.

Returns

The newly resized frame

Return type

Frame

facelift.transform.rgb(frame)[source]¶

Convert the given frame to RGB.

This helper transform is typically needed when working with other image processing libraries such as pillow as they work in RGB coordinates while OpenCV works in BGR coordinates.

Examples

>>> from facelift.transform import rgb
>>> rgb_frame = rgb(bgr_frame)

Parameters: frame (Frame) – The BGR frame to convert to RGB
Returns: The new RGB frame
Return type: Frame

facelift.transform.rotate(frame, degrees, interpolation=cv2.INTER_AREA)[source]¶

Rotate a frame while keeping the whole frame visible.

Examples

>>> from facelift.transform import rotate
>>> rotated_90 = rotate(frame, 90)
>>> rotated_neg_90 = rotate(frame, -90)

Warning

This transform typically will produce larger frames since we are producing a rotated frame while keeping the original frame completely visible. This means if we do a perfect 45 degree rotation on a 512x512 frame we will produce a 724x724 frame since the 512x512 frame is now on a angle that requires a larger container.

Be cautious when using rotation. Most of the time you do not need to rotate on any angles other than 90, 180, and 270 for decent face detection. However, this isn’t always true.

Parameters

frame (Frame) – The frame to rotate
degrees (int) – The number of degrees to rotate the given frame
interpolation (int, optional) – The type of interpolation to use in the produced rotation matrix. Defaults to DEFAULT_INTERPOLATION.

Returns

The newly rotated frame

Return type

Frame

facelift.transform.scale(frame, factor, interpolation=cv2.INTER_AREA)[source]¶

Scale a given frame down or up depending on the given scale factor.

Examples

Downscaling a frame can be performed with a scale factor >0 and <1. For example, scaling a frame to half of its original size would require a scale factor of 0.5.

>>> from facelift.transform import scale
>>> assert frame.shape[:1] == [512, 512]
>>> downscaled_frame = scale(frame, 0.5)
>>> assert downscaled_frame.shape[:1] == [256, 256]

Upscaling a frame with this method is very naive and suboptimal. However, any value >1 will result in a upscaled frame. For example, scaling a frame to double its original size would require a scale factor of 2.

>>> from facelift.transform import scale
>>> assert frame.shape[:1] == [512, 512]
>>> upscaled_frame = scale(frame, 2)
>>> assert upscaled_frame.shape[:1] == [1024, 1024]

Following this logic, a scale factor of 1 would result in absolutely no change to the given frame.

Warning

This transformation will return the exact same frame instance as the one provided through the frame parameter in the following cases:

If a factor of exactly 1 is given. In this case the scale operation would result in no change.

The given frame has factor less than 1 a width or height of 1px. In this case we are attempting to scale down the given frame and we cannot scale down the frame any further without producing a 0px frame.

Parameters

frame (Frame) – The frame to scale
factor (float) – The factor to scale the given frame
interpolation (Optional[int], optional) – The type of interpolation to use in the scale operation. Defaults to DEFAULT_INTERPOLATION.

Raises

ValueError – When the given scale factor is not positive

Returns

The newly scaled frame

Return type

Frame

facelift.transform.translate(frame, delta_x=None, delta_y=None, interpolation=cv2.INTER_AREA)[source]¶

Translate the given frame a specific distance away from its origin.

Examples

>>> from facelift.transform import translate
>>> translated_neg_90_x_frame = translate(frame, delta_x=-90)

Important

This translation retains the original size of the given frame. So a 512x512 frame translated 90px on the x-axis will still be 512x512 and space where the frame use to take up will be essentially nulled out.

Parameters

frame (Frame) – The frame to translate
delta_x (Optional[int], optional) – The pixel distance to translate the frame on the x-axis.
delta_y (Optional[int], optional) – The pixel distance to translate the frame on the y-axis.
interpolation (int, optional) – The type of interpolation to use during the translation. Defaults to DEFAULT_INTERPOLATION.

Returns

The newly translated frame

Return type

Frame

`facelift.magic`¶

Contains helpers and enums used to guess the type of media that is being processed.

This module utilizes python-magic which in turn uses libmagic to guess the appropriate mimetype of some byte buffer.

facelift.magic.DEFAULT_MAGIC_BUFFER_SIZE¶

The default number of bytes to try and read from when making a guess at the mimetype of some file.

Type: int

facelift.magic.get_media_type(media_filepath, buffer_size=None, validate=False)[source]¶

Try and determine the media type for content at the given filepath.

Parameters

media_filepath (Path) – The filepath to guess the media type of
buffer_size (Optional[int], optional) – The number of bytes to use for guessing the media type of the given file. Defaults to the value of DEFAULT_MAGIC_BUFFER_SIZE.
validate (bool, optional) – If truthy, a ValueError will be raised if the given file’s mimetype does not match a supported MediaType. Defaults to False.

Raises

FileNotFoundError – When the provided filepath does not exist
ValueError – When validate is truthy and the given filepath does not match a supported MediaType

Returns

The appropriate media type enum attribute for the given filepath, if a successful guess and media type match is made

Return type

Optional[MediaType]

facelift.magic.get_mimetype(media_filepath, buffer_size=None)[source]¶

Try and determine the mimetype for content at the given filepath.

Parameters

media_filepath (Path) – The filepath to guess the mimetype of
buffer_size (Optional[int], optional) – The number of bytes to use for guessing the mimetype of the given file. Defaults to the value of DEFAULT_MAGIC_BUFFER_SIZE.

Raises

FileNotFoundError – When the provided filepath does not exist

Returns

The guessed mimetype if a guess can be safely made

Return type

Optional[str]

`facelift.detect`¶

Contains the available bulitin face detectors.

The included detectors will handle the necessary process for taking a read frame and discovering all the available faces and included landmarks. If you have a custom face_landmarks model, you can inherit from BaseLandmarkDetector to detect and return faces using a custom model.

Examples

>>> from facelift.detect import BasicFaceDetector
>>> from facelift.capture import iter_media_frames
>>> detector = BasicFaceDetector()
>>> for frame in iter_media_frames(MEDIA_FILEPATH):
...     for face in detector.iter_faces(frame):
...         print(face)

class facelift.detect.BaseLandmarkDetector[source]¶

An abstract landmark detector class that each landmark model should inherit from.

Raises

NotImplementedError – If the model_filepath property is not implemented
NotImplementedError – If the landmark_slices property is not implemented

detector¶

Detector to use in face bounds detection.

Returns: The detector callable.
Return type: Detector

get_landmarks(points)[source]¶

Get the mapping of face features and point sequences for extracted points.

Parameters: points (PointSequence) – The sequence of extracted points from dlib.
Returns: The dictionary of face features and point sequences.
Return type: Dict[FaceFeature, PointSequence]

iter_faces(frame, upsample=0)[source]¶

Iterate over detected faces within a given Frame.

Examples

Get detected faces from the first available webcam.

>>> from facelift.capture import iter_stream_frames
>>> from facelift.detect import BasicFaceDetector
>>> detector = BasicFaceDetector()
>>> for frame in iter_stream_frames():
...     for face in detector.iter_faces(frame):
...         print(face)

Parameters

frame (Frame) – The frame to detect faces in.
upsample (int, optional) – The number of times to scale up the image before detecting faces. Defaults to 0.

Yields

Face – A detected face within the image, this has no guarantee of order if multiple faces are detected

Return type

Generator[Face, None, None]

abstract property landmark_slices¶

Property mapping of facial features to face point slices.

Raises: NotImplementedError – Must be implemented by subclasses
Return type: Dict[FaceFeature, Tuple[int, int]]

abstract property model_filepath¶

Property filepath to the landmarks model that should be used for detection.

Raises: NotImplementedError – Must be implemented by subclasses
Return type: Path

predictor¶

Predictor to use in face landmark detection.

Returns: The predictor callable.
Return type: Predictor

static shape_to_points(shape, dtype='int')[source]¶

Convert dlib shapes to point sequences.

Example

After getting a detected face shape from dlib, we need to convert it back into a numpy.ndarray so OpenCV can use it.

>>> from facelift.detect import BasicFaceDetector
>>> detector = BasicFaceDetector()
>>> for face_bounds in detector.detector(frame, 0):
...     face_shape = detector.predictor(frame, face_bounds)
...     face_features = detector.shape_to_points(face_shape)

Parameters

shape (dlib.full_object_detection) – The detected dlib shape.
dtype (str, optional) – The point type to use when converting the given shape to points. Defaults to “int”.

Returns

The newly created sequence of points.

Return type

PointSequence

static slices_to_landmarks(points, feature_slices)[source]¶

Group point sequences to features based on point index slices.

Helper function to automatically group features when given the feature slice definition. This feature slice definition is a basic way to easily categorize the features discovered from the dlib predictor as an actual FaceFeature.

Examples

>>> from facelift.detect import BasicFaceDetector
>>> detector = BasicFaceDetector()
>>> for face_bounds in detector.detector(frame, 0):
...     face_shape = detector.predictor(frame, face_bounds)
...     face_features = detector.shape_to_points(face_shape)
...     grouped_features = detector.slices_to_landmarks(face_features)

Parameters

points (PointSequence) – The points to extract feature sequences from.
feature_slices (Dict[FaceFeature, Tuple[int, int]]) – A dictionary of FaceFeature and slice tuples.

Returns

The dictionary of features and grouped point sequences.

Return type

Dict[FaceFeature, PointSequence]

class facelift.detect.BasicFaceDetector[source]¶

Basic face detector.

This face detector gives single point positions for the outside of both eyes and the philtrum (right beneath the nose). For rendering, these facial features must be rendered as points rather than lines.

This model is useful for just finding faces and getting normalized face frames. This model is not useful for emotion, perspective, or face state detection.

class facelift.detect.FullFaceDetector[source]¶

Full face detector.

This face detector detects all available frontal face features. This model can be used for most anything, but may not be the most efficient. If you are just trying to detect faces, you should probably use the BasicFaceDetector instead.

get_landmarks(points)[source]¶

Get the mapping of face features and point sequences for extracted points.

Parameters: points (PointSequence) – The sequence of extracted points from dlib.
Returns: The dictionary of face features and point sequences.
Return type: Dict[FaceFeature, PointSequence]

class facelift.detect.PartialFaceDetector[source]¶

Partial face detector.

This face detector detects all features of a face except for the forehead. This model is useful for most any purpose. However, if all you are doing is detecting faces, you should probably use the BasicFaceDetector instead.

facelift.detect.get_detector()[source]¶

Build the generic detector callable.

This detector comes directly from the dlib FHOG frontal face detector.

Returns: The new callable to detect face bounds
Return type: Detector

facelift.detect.get_predictor(model_filepath)[source]¶

Build a predictor callable for a given landmark model.

Parameters: model_filepath (Path) – The path to the landmark model
Raises: FileNotFoundError – If the given model filepath does not exist
Returns: The new callable to predict face shapes
Return type: Predictor

`facelift.encode`¶

Contains the available builtin face encoders.

The included encoders will handle the necessary steps to take a given frame and detected face to generate an encoding that can be used for future recognition. I highly recommend that you use the BasicFaceDetector if attempting to encode faces as it is lightweight and other detectors don’t provide any added benefit to face recognition.

Examples

>>> from facelift.capture import iter_media_frames
>>> from facelift.detect import BasicFaceDetector
>>> from facelift.encode import BasicFaceEncoder
>>> detector = BasicFaceDetector()
>>> encoder = BasicFaceEncoder()
>>> for frame in iter_media_frames(MEDIA_FILEPATH):
...     for face in detector.iter_faces(frame):
...         face_encoding = encoder.get_encoding(frame, face)

Important

Faces detected from the FullFaceDetector cannot be encoded as the model this detector uses is trained by a third party and not able to be processed by dlib’s default ResNet model. Please only use faces detected using the BasicFaceDetector or the PartialFaceDetector for building face encodings.

I would highly recommend that you use the BasicFaceDetector in all cases where you are performing encoding. The trained detection model for this basic detector is ~5MB whereas the alternative is >90MB. Using a heavier model will cause slowdown when simply trying to recognize multiple faces in a single frame.

facelift.encode.DEFAULT_ENCODING_JITTER¶

The default amount of jitter to apply to produced encodings.

Type: int

facelift.encode.DEFAULT_ENCODING_PADDING¶

The default padding expected to exist around the detected face frame.

Type: float

class facelift.encode.BaseEncoder[source]¶

An abstract encoder class that each encoder should inherit from.

Raises: NotImplementedError – If the model_filepath property is not implemented

get_encoding(frame, face, jitter=0, padding=0.25)[source]¶

Calculate the encoding for a given frame and detected face.

Examples

>>> from facelift.capture import iter_media_frames
>>> from facelift.detect import BasicFaceDetector
>>> from facelift.encode import BasicFaceEncoder
>>> detector = BasicFaceDetector()
>>> encoder = BasicFaceEncoder()
>>> for frame in iter_media_frames(MEDIA_FILEPATH):
...     for face in detector.iter_faces(frame):
...         face_encoding = encoder.get_encoding(frame, face)

Parameters

frame (Frame) – The frame the face was detected in
face (Face) – The detected face from the given frame
jitter (int, optional) – The amount of jitter to apply during encoding. This can help provide more accurate encodings for frames containing the same face. Defaults to DEFAULT_ENCODING_JITTER.
padding (float, optional) – The amount of padding to apply to the face frame during encoding. Defaults to DEFAULT_ENCODING_PADDING.

Returns

The encoding of the provided face for the given frame

Return type

Encoding

abstract property model_filepath¶

Property filepath to the encoding model that should be used for encoding.

Raises: NotImplementedError – Must be implemented by subclasses
Return type: Path

score_encoding(source_encoding, known_encodings)[source]¶

Score a source encoding against a list of known encodings.

Important

This score is the average Euclidian distance between the given encodings. So the most similar encodings will result in a score closest to 0.0.

If no encodings are given, then we will default to using math.inf as it is the greatest distance from 0.0 that we can define.

Examples

>>> from facelift.capture import iter_media_frames
>>> from facelift.detect import BasicFaceDetector
>>> from facelift.encode import BasicFaceEncoder
>>> detector = BasicFaceDetector()
>>> encoder = BasicFaceEncoder()
>>> # A list of previously encoded faces for a single person
>>> KNOWN_FACES = [...]
>>> for frame in iter_media_frames(MEDIA_FILEPATH):
...     for face in detector.iter_faces(frame):
...         face_encoding = encoder.get_encoding(frame, face)
...         score = encoder.score_encoding(face_encoding, KNOWN_FACES)

Parameters

source_encoding (Encoding) – The unknown encoding we are attempting to score.
known_encodings (List[Encoding]) – A list of known encodings we are scoring against. These encodings should all encodings from a single person’s face.

Returns

The score of a given encoding against a list of known encodings. This value should be greater than 0.0 (lower is better).

Return type

float

class facelift.encode.BasicFaceEncoder[source]¶

Encode faces detected by the BasicFaceDetector.

This face encoder can handle faces detected by both the BasicFaceDetector and the PartialFaceDetector. However, you should likely only ever be encoding faces for recognition from the lightest model available (BasicFaceDetector).

Important

This encoder can not handle faces detected using the FullFaceDetector. If we determine we are using a face detected by this detector, the get_encoding() method will raise a ValueError.

get_encoding(frame, face, jitter=0, padding=0.25)[source]¶

Calculate the encoding for a given frame and detected face.

Examples

>>> from facelift.capture import iter_media_frames
>>> from facelift.detect import BasicFaceDetector
>>> from facelift.encode import BasicFaceEncoder
>>> detector = BasicFaceDetector()
>>> encoder = BasicFaceEncoder()
>>> for frame in iter_media_frames(MEDIA_FILEPATH):
...     for face in detector.iter_faces(frame):
...         face_encoding = encoder.get_encoding(frame, face)

Parameters

frame (Frame) – The frame the face was detected in
face (Face) – The detected face from the given frame
jitter (int, optional) – The amount of jitter to apply during encoding. This can help provide more accurate encodings for frames containing the same face. Defaults to DEFAULT_ENCODING_JITTER.
padding (float, optional) – The amount of padding to apply to the face frame during encoding. Defaults to DEFAULT_ENCODING_PADDING.

Raises

ValueError – When the given face was detected with the FullFaceDetector.

Returns

The encoding of the provided face for the given frame

Return type

Encoding

facelift.encode.get_encoder(model_filepath)[source]¶

Build an encoder for the given dlib ResNet model.

Parameters: model_filepath (Path) – The path to the encoder model
Raises: FileNotFoundError – If the given model filepath does not exist
Returns: The encoder to use for encoding face frames
Return type: Encoder

`facelift.helpers`¶

Contains mechanisms to extract details or normalized details for detected faces.

facelift.helpers.DEFAULT_NORMALIZED_FACE_SIZE¶

The default size of the normalized face frame. Defaults to 256.

Type: int

facelift.helpers.DEFAULT_NORMALIZED_LEFT_EYE_POSITION¶

The default percentage (0.0-1.0) where the left eye should be placed in the normalized face frame. Defaults to (0.35, 0.35).

Type: Tuple[float, float]

facelift.helpers.get_eye_angle(face)[source]¶

Get the angle the eyes are currently at for the given face.

Parameters: face (Face) – The face to get the eye angle from.
Returns: The floating point value describing the angle of the eyes in the face.
Return type: numpy.float64

facelift.helpers.get_eye_center_position(face)[source]¶

Get the center position between the eyes of the given face.

Parameters: face (Face) – The face to extract the center position from.
Returns: The position directly between the eyes of the face
Return type: Tuple[numpy.int64, numpy.int64]

facelift.helpers.get_eye_deltas(face)[source]¶

Get the difference between eye positions of the given face.

Parameters: face (Face) – The face to get the eye deltas from.
Returns: A tuple of (x delta, y delta) for the given face’s eyes
Return type: Tuple[numpy.int64, numpy.int64]

facelift.helpers.get_eye_distance(face)[source]¶

Get the distance between the eyes of the given face.

Parameters: face (Face) – The face to get the eye distance from.
Returns: A floating point value describing the distance between the face’s eye.
Return type: numpy.float64

facelift.helpers.get_eye_positions(face)[source]¶

Get the center position tuples of eyes from the given face.

Parameters: face (Face) – The face to extract eye positions from.
Raises: ValueError – If the given face is missing either left or right eye landmarks
Return type: Tuple[Tuple[int64, int64], Tuple[int64, int64]]
Returns: A tuple of (left eye position, right eye position)

facelift.helpers.get_normalized_frame(frame, face, desired_width=None, desired_height=None, desired_left_eye_position=None)[source]¶

Get a normalized face frame where the face is aligned, cropped, and positioned.

Examples

Get a normalized face frame from a detected face from the given frame:

>>> from facelift.helpers import get_normalized_frame
>>> normalized_frame = get_normalized_frame(frame, face)

Parameters

frame (Frame) – The original frame the face was detected from.
face (Face) – The detected face to use when extracting a normalized face frame.
desired_width (Optional[int], optional) – The desired width of the normalized frame. Defaults to None.
desired_height (Optional[int], optional) – The desired height of the normalized frame. Defaults to None.
desired_left_eye_position (Optional[Tuple[float, float]], optional) – The desired position point for the left eye. This position is a value between 0.0 and 1.0 indicating the percentage of the frame. Defaults to None.

Returns

The normalized face frame.

Return type

Frame

`facelift.render`¶

Contains some very basic wrappers around drawing things onto frames.

When detecting faces, it is kinda nice to be able to see what features are being detected and where inaccuracies are being detected. With a combination of the window module and some of these helper functions, we can easily visualize what features are being detected.

For example, if we wanted to draw lines for each detected feature from the PartialFaceDetector we can do the following:

>>> from facelift.capture import iter_stream_frames
>>> from facelift.window import opencv_window
>>> from facelift.detect import PartialFaceDetector
>>> from facelift.render import draw_line
>>> detector = PartialFaceDetector()
>>> with opencv_window() as window:
...     for frame in iter_stream_frames():
...         for face in detector.iter_faces(frame):
...             for _, points in face.landmarks.items():
...                 frame = draw_line(frame, points)
...         window.render(frame)

facelift.render.DEFAULT_COLOR¶

The default color for all draw helper functions. Defaults to (255, 255, 255), or white.

Type: Tuple[int, int, int]

facelift.render.DEFAULT_FONT¶

The default OpenCV HERSHEY font to use for rendering text. Defaults to cv2.FONT_HERSHEY_SIMPLEX

Type: int

class facelift.render.LineType(value)[source]¶

Enumeration of the different available PointSequence types for OpenCV.

FILLED¶: Filled line (useful for single points).

CONNECTED_4¶: A 4-point connected line.

CONNECTED_8¶: An 8-point connected line.

ANTI_ALIASED¶: An anti-aliased line (good for drawing curves).

class facelift.render.Position(value)[source]¶

Enumeration of available relative positions.

START¶: Positioned content appears at the left of the container.

END¶: Positioned content appears at the right of the container.

CENTER¶: Positioned content appears in the middle of the container.

facelift.render.draw_contour(frame, line, color=255, 255, 255, thickness=- 1, line_type=cv2.LINE_AA)[source]¶

Form and draw a contour for the given line on a frame.

Examples

Draw a contour between multiple points.

>>> from facelift.render import draw_contour
>>> frame = draw_contour(frame, [(10, 10), (20, 20)])

Parameters

frame (Frame) – The frame to draw the contour on.
line (PointSequence) – The array of points to use to form the contour.
color (Tuple[int, int, int], optional) – The color of the contour.. Defaults to DEFAULT_COLOR.
thickness (int, optional) – The thickness of the contour. Defaults to -1.
line_type (LineType, optional) – The line type to use for the contour. Defaults to LineType.ANTI_ALIASED.

Return type

Type[ndarray]

Returns

Frame The frame with the contour drawn on it

facelift.render.draw_line(frame, line, sequence=None, color=255, 255, 255, thickness=1, line_type=cv2.LINE_AA)[source]¶

Draw a sequence of connected points on a given frame.

Examples

Draw a line between a sequence of points.

>>> from facelift.render import draw_line
>>> frame = draw_line(frame, [(10, 10), (20, 20)])

Parameters

frame (Frame) – The frame to draw the line on.
line (PointSequence) – The array of points to draw on the given frame
sequence (Optional[List[Tuple[int, int]]], optional) – An optional custom sequence for drawing the given line points. Defaults to None.
color (Tuple[int, int, int], optional) – The color of the line. Defaults to DEFAULT_COLOR.
thickness (int, optional) – The thickness of the line. Defaults to 1.
line_type (LineType, optional) – The type of the line. Defaults to LineType.FILLED.

Return type

Type[ndarray]

Returns

Frame The frame with the line drawn on it

facelift.render.draw_point(frame, point, size=1, color=255, 255, 255, thickness=- 1, line_type=cv2.FILLED)[source]¶

Draw a single point on a given frame.

Examples

Draw a single point a position (10, 10) on a given frame.

>>> from facelift.render import draw_point
>>> frame = draw_point(frame, (10, 10))

Parameters

frame (Frame) – The frame to draw the point
point (Point) – The pixel coordinates to draw the point
size (int, optional) – The size of the point. Defaults to 1.
color (Tuple[int, int, int], optional) – The color of the point. Defaults to DEFAULT_COLOR.
thickness (int, optional) – The thickness of the point. Defaults to -1.
line_type (LineType, optional) – The type of line type to use for the point. Defaults to LineType.FILLED.

Return type

Type[ndarray]

Returns

Frame The frame with the point drawn on it

facelift.render.draw_points(frame, points, size=1, color=255, 255, 255, thickness=- 1, line_type=cv2.FILLED)[source]¶

Draw multiple points on a given frame.

Examples

Draw a sequence of points to a given frame.

>>> from facelift.render import draw_points
>>> frame = draw_points(frame, [(10, 10), (20, 20)])

Parameters

frame (Frame) – The frame to draw the points on.
points (PointSequence) – The sequence of points to draw.
size (int, optional) – The size of the points. Defaults to 1.
color (Tuple[int, int, int], optional) – The color of the points. Defaults to DEFAULT_COLOR.
thickness (int, optional) – The thickness of the points. Defaults to -1.
line_type (LineType, optional) – The type of line type to use for the points. Defaults to LineType.FILLED.

Return type

Type[ndarray]

Returns

Frame The frame with the points drawn on it

facelift.render.draw_rectangle(frame, start, end, color=255, 255, 255, thickness=1, line_type=cv2.LINE_AA)[source]¶

Draw a rectangle on the given frame.

Examples

Draw a rectangle starting at (10, 10) and ending at (20, 20).

>>> from facelift.render import draw_rectangle
>>> frame = draw_rectangle(frame, (10, 10), (20, 20))

Parameters

frame (Frame) – The frame to draw the rectangle on.
start (Point) – The starting point of the rectangle.
end (Point) – The ending point of the rectangle.
color (Tuple[int, int, int], optional) – The color of the rectangle. Defaults to DEFAULT_COLOR.
thickness (int, optional) – The thickness of the rectangle. Defaults to 1.
line_type (LineType, optional) – The line type to use when drawing the lines of the rectangle. Defaults to LineType.ANTI_ALIASED.

Return type

Type[ndarray]

Returns

Frame The frame with the rectangle drawn on it

facelift.render.draw_text(frame, text, start, end, color=(255, 255, 255), font=cv2.FONT_HERSHEY_SIMPLEX, font_scale=1, thickness=1, line_type=cv2.LINE_AA, x_position=<Position.START: 'start'>, y_position=<Position.START: 'start'>, x_offset=0, y_offset=0, allow_overflow=False)[source]¶

Draw some text on the given frame.

Examples

Draw the text “Hello, World!” right-aligned within the text rectangle from (10, 10) to (20, 20).

>>> from facelift.render import draw_text, Position
>>> frame = draw_text(
...     frame,
...     "Hello, World",
...     (10, 10),
...     (20, 20),
...     x_position=Position.END
... )

Parameters

frame (Frame) – The frame to draw some text on
text (str) – The text to draw on the frame
start (Point) – The starting point of the text container
end (Point) – The ending point of the text container
color (Tuple[int, int, int], optional) – The color of the text. Defaults to DEFAULT_COLOR.
font (int, optional) – The OpenCV hershey font to draw the text with. Defaults to DEFAULT_FONT.
font_scale (float, optional) – The scale of the font. Defaults to 1.
thickness (int, optional) – The thickness of the font. Defaults to 1.
line_type (LineType, optional) – The line type of the font. Defaults to LineType.ANTI_ALIASED.
x_position (Position, optional) – The x-axis position to draw the text in relative to the text container. Defaults to Position.START.
y_position (Position, optional) – The y-axis position to draw the text in relative to the text container. Defaults to Position.START.
x_offset (int, optional) – The x-axis offset from the text container to add to the calculated relative position. Defaults to 0.
y_offset (int, optional) – The y-axis offset from the text container to add to the calculated relative position. Defaults to 0.
allow_overflow (bool, optional) – If set to True, the provided text will start drawing at the given start and end points without obeying them as a bounding text container. Defaults to False.

Return type

Type[ndarray]

Returns

Frame The frame with the text drawn on it

`facelift.window`¶

Contains some helper abstractions for OpenCV windows and frame rendering.

This collection of window helpers is just to help standardize and cleanup how to interact with OpenCV window displays. The opencv_window context manager is very easy to use for getting a quick window for rendering frames as they are produced.

For example:

>>> from pathlib import Path
>>> from facelift.window import opencv_window
>>> from facelift.capture import iter_media_frames
>>> with opencv_window() as window:
...     for frame in iter_media_frames(Path("~/my-file.mp4")):
...         window.render(frame)

This context manager will produce a new window for rendering the frames read from my-file.mp4 and will destroy the window once the context is exited.

I wouldn’t recommend using this for any kind of production use; mostly the OpenCV window is just useful for debugging.

facelift.window.DEFAULT_WINDOW_TITLE¶

The default OpenCV window title if none is supplied. Defaults to “Facelift”.

Type: str

facelift.window.DEFAULT_WINDOW_DELAY¶

The default number of milliseconds to wait between showing frames. Defaults to 1.

Type: int

facelift.window.DEFAULT_WINDOW_STEP_KEY¶

The default ASCII key index to use as the step key when step is enabled. Defaults to 0x20 (Space).

Type: int

class facelift.window.WindowStyle[source]¶

Object namespace of available OpenCV window styles.

DEFAULT¶

The default OpenCV window style.

Type: int

AUTOSIZE¶

Automatically fit window size on creation.

Type: int

GUI_NORMAL¶

Window with a basic GUI experience.

Type: int

GUI_EXPANDED¶

Window with an expanded GUI experience.

Type: int

FULLSCREEN¶

Window that displays frames fullscreen (full-canvas).

Type: int

FREE_RATIO¶

Window that allows for any window ratio.

Type: int

KEEP_RATIO¶

Window that maintains the original window ratio.

Type: int

OPENGL¶

Window rendered via OpenGL. May not work for some machines and will only work if OpenCV is compiled with GPU support.

Type: int

class facelift.window.opencv_window(title='Facelift', style=cv2.WINDOW_NORMAL, delay=1, step=False, step_key=32)[source]¶

Create an OpenCV window that closes once the context exits.

Examples

Easy usage of OpenCV’s provided window to display read frames from a webcam.

>>> from facelift.window import opencv_window
>>> with opencv_window() as window:
...     for frame in iter_stream_frames():
...         window.render(frame)

Parameters

title (str) – The title of the OpenCV window.
style (int) – The style of the OpenCV window.
delay (float) – The number of milliseconds to delay between displaying frames.
step (bool) – Flag that indicates if the window should wait for a press of the defined step_key before releasing the render call. Defaults to False.
step_key (int) – The ASCII integer index of the key to wait for press when step is True. Defaults to 0x20 (Space).

Raises

ValueError – If the given window title is an empty string
ValueError – If the given window delay is less or equal to 0

__enter__()[source]¶: Initialize the context of the window.

__exit__(exc_type, exc_value, traceback)[source]¶

Destroy the context of the window.

Parameters

exc_type (Optional[Type[BaseException]]) –
exc_value (Optional[BaseException]) –
traceback (Optional[traceback]) –

Return type

Optional[bool]

close()[source]¶: Destroy the window with the current context’s title.

create()[source]¶: Create a new window with the current context’s title and style.

render(frame)[source]¶

Render a given frame in the current window.

Parameters: frame (Frame) – The frame to render within the window

`facelift._data`¶

Helpers for fetching the pre-trained models this project is built around.

Due to the size of the models that we are building this project around, we need to fetch the models outside of the standard PyPi installation. The following methods handle building an asset manifest that should be released with each GitHub release. This asset manifest will then further inform the little downloading script we have provided where to find and place the assets in the installed package.

This helper utility currently expects the following of the GitHub release:

A data-manifest.json is provided as a GitHub release asset.
All models within the asset manifest are included as GitHub release assets.

Important

The data-manifest.json must following the following structure:

{
    "relative filepath from package root for asset": [
        "download url of asset",
        "md5 hash of asset"
    ]
}

As an example:

{
    "data/encoders/dlib_face_recognition_resnet_model_v1.dat": [
        "https://github.com/stephen-bunn/facelift/releases/download/v0.1.0/dlib_face_recognition_resnet_model_v1.dat",
        "2316b25ae80acf4ad9b620b00071c423"
    ]
}

Examples

>>> from facelift._data import download_data
>>> download_data(display_progress=True)
https://... [123 / 456] 26.97%
Downloaded https://... to ./... (1234567890)

facelift._data.build_manifest(release_tag, *asset_filepaths)[source]¶

Build the manifest content for a proposed release and defined assets.

Parameters

release_tag (str) – The release tag the manifest is being built for.
asset_filepaths (pathlib.Path) – Multiple existing local asset filepaths.

Raises

FileNotFoundError – When a given asset filepath does not exist.
ValueError – When a checksum cannot be calculated for one of the given filepaths.

Returns

The manifest JSON-serializable dictionary

Return type

Dict[str, Tuple[str, str]]

facelift._data.download_data(display_progress=False, release_tag=None, chunk_size=4096, validate=True)[source]¶

Download the data from a fetched remote release manifest.

Parameters

display_progress (bool, optional) – Flag that indicates if you want to display the download progress for assets. Defaults to False.
release_tag (Optional[str], optional) – The release tag of the assets you want to download. Defaults to None which will fetch the latest release assets.
chunk_size (int, optional) – The chunk size to use when downloading assets. Defaults to DOWNLOAD_CHUNK_SIZE.
validate (bool, optional) – If False, will skip checksum validation for all downloaded assets. Defaults to True.

Raises

FileExistsError – If a file already exists at one of the assets relative file locations.
ValueError – If the downloaded assets fails checksum validation.

facelift._data.get_remote_manifest(release_tag=None)[source]¶

Get the manifest content from a GitHub release.

Parameters: release_tag (Optional[str], optional) – The release tag of the manifest to fetch. Defaults to None which fetches the latest release manifest.
Returns: The manifest JSON-serializable dictionary
Return type: Dict[str, Tuple[str, str]]

Facelift Package¶

facelift.types¶

facelift.capture¶

facelift.transform¶

facelift.magic¶

facelift.detect¶

facelift.encode¶

facelift.helpers¶

facelift.render¶

facelift.window¶

facelift._data¶

`facelift.types`¶

`facelift.capture`¶

`facelift.transform`¶

`facelift.magic`¶

`facelift.detect`¶

`facelift.encode`¶

`facelift.helpers`¶

`facelift.render`¶

`facelift.window`¶

`facelift._data`¶