Privacy Preserving Face recognition

Facial recognition is, and is ought to be an application of AI for which we should be especially sensitive for as it penetrates even further into our daily lives. Like any kind of inference models targeting biometric data, face recognition models require trining on large amounts of privacy-sensitive datasets (in this case, face images), with the dependence on such sensitive data points continuing along inference time.

Preserving the privacy of subjects whose data are used both during training and inference phases is therefore an important objective, one that has been attracting the focus of researchers and industry workers.

For the purposes of a work related project, I was recently involved into the development of a face recognition system that preserves the privacy of the end user while also maintaining high recognition accuracy. While there are multiple methodologies and frameworks employed by practtioners, most of those have no theoretical guarantees that can, at least in theory, protect the privacy of the user against all current and all possible future adversaries. Many of the practices are mostly grounded in protection against the current state-of-the-art attacks, but as we know, these are costantly evolving. The emergence of more powerful AI systems such as LLMs is also a factor that could be utilised against privacy preservation.

Transformation based privacy

One family of privacy preserving methods is based on transforming the input data’s features in a way that they become practically inrecognisable to both the human eye and reconstruction algorithms. Training the model on these transformed features results in a model that is able to produce predictions when presented with these transformed input features instead of the original ones. These methods can perform well given that currently available reconstruction methods are not able to reproduce the transformed images back to their original versions, or at least reveal identifying features without necesarilly perfectly reconstructing the image. Given this, these methods do not offer theoretical guarantees on their privacy preservation, it is all a matter of when a good enough adversary can reproduce transformed data. As attacks get more powerful, such methods are getting weaker.

Differential Privacy (DP)

Differential privacy has been around since xx, introduced by Goodfellow et. al. Its mechanisms are based around training a neural network model using a variant of the Stochastic Gradient Descent (SGD) algorithm called DP-SGD. This modified version clamps the gradients of parameters as to limit the sensitivity of the model to specific training points. Along that, it also employs the use of additive random noise to the training data, such noise is also added to an input image during inference. A man-in-the-middle interjecting the communication between user and server while a user is posting an image of themselves for identification would therefore look corrupted to the attacker’s naked eye.

The main parameters to think about when developing a dofferentialy private model is the privacy budget, mostly expressed as epsilon. This parameter directly controls the magnitude of the noise added to the features of the training data , and while greater epislon is (meaning more additive noise to our training data) intuitevly better, the whole objective of someone buding such a model is to find the ideal tradeoff between privacy and accuracy. The more privacy/noise we add to our data, the hardest will be for the model to adapt and still produce accurate results, since features become less informative the more they get corrupted with noise. It is therefore the ultimate goal elevate the question of “How to make my model privacy preserving by adding noise?” to “How much privacy can I add before my model starts degrading too much?”.

Face recognition

Face identification and recognition nowadays usually based on computer vision. More specfically, convolutional neural networks are trained on large numbers of face images to detect faces within them. This is essentially a task of taking an image with faces as input and producing an output with the exact coordinates of the location of a face. See below for an example.

After face identification and localisation within an image comes the task of actually recognising the person whose face the model detected. In the simplest context, in which we know we only need to identify a few specific people without the need to ever add new people to this set, one could gather a bunch of captured images of for each of those people and train a convolutional neural network for classifying between them. In production systems where one needs to idetify among a large number of people and also be able to add new instances to the set of recognisable faces, the method described above becomes extremely inefficient, since we would have to fine-tune the model every time we wanted it to recongnise additional people. Instead, in this case we can train our model to not specifically classify between known faces, but generate highly faithful signatures of faces in the form of low-dimensional latent representations. A model trained on a large/expressice enough face dataset would then be able to encode a raw images of the same person into similar signatures, and upon deployment phase, we simply identify a face based on its closest known signature within an identifiable set. This method still needs the presence of an identifiable set which will need to be expanded iteratively, but drops the need to re-train or tune the model all the time, which is surely the most inconvenient and inefficient thing we would have to do.

See an illustration of the face recognition pipeline below:

Privacy preserving face recognition

Now that we have our playing cards in hand (the differential privacy mechanism and the CV based face recognition system), the question is now how we combine these two to get a system which can identify faces while preserving the privacy of the subjects using differential privacy?

Exxcitngly enough for us, there is nothing prohibiting us from employing both a transformation based and a differentially private method for a model. One could use transformed versions of the raw image features and still use the additive noise mechanism to enhance privacy as well as train the recognition model with the DP-SGD algorithm. Many open source model architectures are available, each one utilising a different set of options. In our case, the one we are going to look into which I actually used for my system is called DCTDP, an approach developed by Tencent researchers as a part of their “trusty” face analysis research platform called TFace.

At a glance, DCTDP is an approach consisted of three parts:

1. A transformation layer: The input image goes through a frequency domain transformation called DCT, which turns the raw RGB features of the image into frequency domain pixels.
2. The differential privacy layer: Here, the features received from step 1 are perturbated with noise. The amount of noise applied to each feature is learnable, meaning that we have paraeters associated with the amount noise added to each feature. During training, these parameters adapt so that noise is added according to each feature’s sensitivity (i.e. How much information about the original image each feature reveals.)
3. A face recognition neural network responsible for producing the low-dimensional image signatures.

So, given the above components, DCTDP has established an architecture that puts both transformations and additive noise to work. The smart thing about it being the fact that the model learns the right amount of noise to add per feature, specifically, adding more noise to the locations of the image (or features) that carry the most identifying information.

The mechanism behind the learnable privacy is simple. The noise added to each input feature is randomly sampled from a Laplace distribution which is controlled by a scale parameter $\sigma$, which dictates the spread of the values sampled (e.g. larger $\sigma$-> larger samples can be sampled and vice versa). So the natural next step is to parameterise these scales and make them learnnable during training.

Specifically, for DCTDP, this is done as follows:

1. Initialise parameter tensor of the same shape as the number of frequenc features and set to 0.
2. The parameter tensor is then passed through a softmax activation, resulting in learned values that add up to 1.
3. These are multiplied by global epsilon (the privacy budget) to get the per-feature epsilon, therefore each of the parameters is scaled with the privacy budget and all of them add up to global epsilon.
4. This value is now used as the scale to sample Laplace distributed noise values to add to the input features.

But there is still one very important detail that is being introduced here. Remember that the purpose is to conceal the most relevant features for identification more than others. In an image, a lot of the space is just noisy signal that does not carry information that actually helps a model identify a face (or create an expressive representation of it). We should therefore aim to assign most of the privacy budget to the informative features. To control this, we empirically compute a sensitivity value r for each feature by computing the difference between its greatest value found in the training data (after transformation) and its smallest value. This is simply a range value, telling us how much a feature varies within our data. A feature that barely varies is obviously not that useful (probably some feature in the background of the image that is always very dark). These values are then used between step 3 and 4 above, by taking the values from (3) to compute per_feature_r/per_feature_epsilon. This is actually what is used as the scale of the Laplaca sampled noise values.

Backpropagation will then do its magic and help adapt the values of those parameters with respect to the minimisation of the network’s loss function.

In the image above, we see the raw images on the left and noise pertrubed images in column 3 and 4 for privacy budgets 4 and 0.5. Of course, the images with lower privacy budget (epsilon) end up having more noise added to them, since the scales (per_feature_r/per_feature_epsilon) are larger.

The privacy preserving model has obviously done some work, the features are clearly corrupted with additive noise making the images very hard to identify with the naked eye while also following DP practices regarding privacy budget allocation. But can you observe something specific? From the visualisations, we can see that, for both budgets, there is much more noise added to the background of the images and not so much added in the actual faces.

Which is kind of expected. Gradient descent optimisation using backpropagation will do everyhting it can to minimise the loss, it doesn’t really understand the whole context within which we use our parameters, it only sees how each individual param affects the loss and acts accordingly. The sensitivity used to compute the Laplace scales is a simple trick we use to control that but at the end of the day these are fixed empirical values, and the optimiser is of course very likely to adapt the softmax layer params so that the per feature epsilons are the largest for the features with high sensitivity to drive their noise scale values down.

Regularising the model to force more budget to be allocated to those high sensitivity features is a natural next step. One could try complementing the overall loss function with a term that punishes the model if the allocation of the budget is overwhelmingly large towards those low-sensitivity features. As a low effort experiment to combat our challenge, we introduced the following regularisation term into the already existing loss.

• We take the fractions of budget allocated to each feature (softmax outputs multiplied with the privacy budget value) as they are stored in a tensor called eps , multiply it with the set of sensitivity values (which are precomputed and fixed) and then take the mean of the result. This term is larger whenever high sensitivity values get assigned larger budgets (therefore less noise) and is smaller when high budgets are allocated to the lower-sensitivity features. Plugging this term into the already existing loss is therefore going to penalise our model when this unwanted behavior takes place, therefore steering the model towards avoiding this penalty during training.
• Another term that can act as a possible remedy is to try and control the distribution of the allocated epsilon. We ideally want to prevent the distribution to collapse towards specific features. A collapsing epsilon distribution means that epsilon spreads only to a restricted set of features (in our case, the low sensiivity features), leaving all else without any allocated epsilon. A way to handle this is to keep track of the entropy. By keeping entropy large enough, we ensure that the distribution’s variance does not shrink and collapses towards specific values. In a loss functions which needs to get minimised, we can simply introduce a negative entropy term, therefore minimising that term (in the moedls effort to minimise the overall loss) during training, ends up in the model trying to keep the entropy as large as possible.

Closely observing the figure bove, where we visualise some of the inputs channels after applying the DCT transformation, we can see that some revealing detail still showing at the unregularised models, completely vasnishes when using the regularised one.