I'll be using Azure Cognitive services to create a simple web app that recognizes and groups faces found in uploaded photos. As a bonus the user will be able to upload a photo and find other photos with similar faces.

The finished demo is on github.

Prerequisites

If you want to get this running you'll need an Azure account (a free trial account will do just fine) and a Windows machine running Visual Studio 2017 Preview 3 with the latest Azure functions tooling preview installed. Instructions for installing and using the tooling are here

Getting started

First, let's define a few simple endpoints that we'll need:

• POST /api/faces This endpoint is to upload a picture containing faces
• GET /api/faces This endpoint lists the unique faces that have been recognized
• GET /api/faces/{id} This endpoint lists all photos for a given face id
• POST /api/faces/similar This endpoint accepts a photo and returns a list of faces found in it

In addition to these we'll have a function that monitors a Azure Blob Storage folder for new images. It is this function that will do most of the work

Defining functions

With the new functions tooling comes a new way of defining functions and their inputs. The previous method of defining a function.json with all the data is gone. Now the metadata is defined in code and the function.json files are generated at build time. This is a significant improvement over the previous tooling. In addition functions can now be run and debugged in Visual Studio with very little effort, in fact it works the same way that any other dotnet project does - click the run button and everything just works. In reality there are a few issues with the preview tooling but these should be sorted before final release.

Api Endpoints

I'll go into a bit of detail on the image upload endpoint as that contains most of the interesting stuff

[FunctionName("UploadFaces")]
public static async Task<HttpResponseMessage> UploadFaces(
// represents the http request and defines the route
[HttpTrigger(AuthorizationLevel.Function, "post", Route = "faces")]HttpRequestMessage req,
// this allows us to do dynamic bindings in the function body
Binder binder,
// for logging
TraceWriter log)
{
  ...
  // here I create a dynamic binding to a storage blob - this isn't really needed in this case, but it serves as a nice example of how to do it
  var cloudBlob = await binder.BindAsync<CloudBlockBlob>(new BlobAttribute($"faces/{Guid.NewGuid()}", FileAccess.ReadWrite));
  // this just uploads the file and sets the content type correctly (the image is converted to png earlier in this method)
  await cloudBlob.UploadFromStreamAsync(memoryStream).ConfigureAwait(false);
  cloudBlob.Properties.ContentType = "image/png";
  await cloudBlob.SetPropertiesAsync().ConfigureAwait(false);
}

The full code for this method is on Github

The other api functions are pretty simple and can be seen on Github

Face detection

Face detection is done in a background task that gets triggered by an image being uploaded to the storage folder in the api function above.

// this function gets triggered whenever a file is uploaded to the 'faces' container in blob storage
[FunctionName("FindFaces")]
public static async Task Run([BlobTrigger("faces/{name}", Connection = "AzureWebJobsStorage")]ICloudBlob imageFile,
                             string name,
                             [Table("faces", Connection = "AzureWebJobsStorage")]IAsyncCollector<FaceRectangle> outTable,
                             Binder binder,
                             TraceWriter log)
{
  using (var imageStream = imageFile.OpenRead())
  {
    // defined here: https://github.com/JimiSmith/face-finder-demo/blob/master/FunctionsFaceDemo/FaceApi.cs#L80
    var faces = await FaceApi.GetFaces(imageStream);
    foreach (var face in faces)
    {
      // generate a thumbnail for each face using ImageResizer and upload to blob storage
      ...
      // add the face to the face list
      await FaceApi.EnsureFaceListExists(FACE_GROUP);
      var similarFaces = await FaceApi.FindSimilarFaces(face.FaceId.ToString(), FACE_GROUP);
      if (similarFaces == null || similarFaces.Count() == 0)
      {
          // no similar faces in face group, add it
          var persistedFace = await FaceApi.AddFaceToFaceList(imageStream, FACE_GROUP, face.FaceRectangle);
          face.FaceId = persistedFace.PersistedFaceId;
      }
      else
      {
          face.FaceId = similarFaces.First().PersistedFaceId;
      }

      // save the face to table storage to be accessed by api functions
      var faceRectangle = face.FaceRectangle;
      faceRectangle.RowKey = Convert.ToBase64String(System.Text.Encoding.UTF8.GetBytes(DateTimeOffset.UtcNow.ToString()));
      faceRectangle.PartitionKey = face.FaceId.ToString();
      faceRectangle.FaceUrl = imageFile.Uri.ToString();
      faceRectangle.FaceThumbnailUrl = $"https://{imageFile.Uri.Host}/faces/thumbnail/{thumbnailFileName}";
      await outTable.AddAsync(faceRectangle);
    }
  }
}

The full code for this method is on Github

At this point we now have a functional api for uploading images and recognizing faces. In the next blog post I'll go over the process of creating a simple frontend for this api

Update: VSTS now has support for deploying Azure functions with the new tooling