Cognitive Services Computer Vision In Xamarin Forms

Before jumping into the cognitive service, we first have an idea of what the cognitive services do in the real life. So let us take a real life example of a Blind Person. Saqib Sheikh is blind from both eyes and he lost his vision at the age of seven. However, thanks to Microsoft Cognitive services he is still feels the emotions as well as picture of everything by using the Microsoft Cognitive Services. So today, we are talking about the World best Cognitive Services from the Microsoft end and latter we will implement this in Xamarin Forms so every single person can enjoy the Cognitive Services.

So let’s get started, The very first thing you need to know is Restful Api and then you simply switch to https://www.microsoft.com/cognitive-services where we have couple of restful API’s . So in short, we will consume the rest API’s in Xamarin Forms by using the Vision Nugget Package. So let’s switch to Visual Studio 2015 or Xamarin Studio and create a new project, add a few xaml lines to your xaml file i.e.

<StackLayout Padding="50">

    <Button x:Name="BtnImage" Text="Pick Image" Clicked="BtnImage_OnClicked" />

    <Image x:Name="Img" WidthRequest="200" HeightRequest="200"/>

    <Label x:Name="LblResult" FontSize="32"  />

  </StackLayout>

Then import a following nugget package Microsoft.ProjectOxford.Vision

After this add one more package for Camera that is Xam.Plugin.Media.

Now just initialize the Camera and add a few lines of code so you could access the camera of mobile and desktop devices.

var media = Plugin.Media.CrossMedia.Current;

await media.Initialize();

var file = await media.TakePhotoAsync(new StoreCameraMediaOptions { SaveToAlbum = false });

Img.Source = ImageSource.FromStream(() => file.GetStream());

Then it’s time to call a vision client just like HttpClient.

Therefore, what I am going to do is I will simply add few lines of code here for accessing the Cognitive Services calling and feedback response and then populate the actual result into the Label.

  var visionclient = new VisionServiceClient(visionKey);

  var result = await visionclient.DescribeAsync(file.GetStream());

  LblResult.Text = result.Description.Captions.First().Text;

Then it's time to call the Dispose function to avoid memory leakage and memory flaws in your applications by using the following line i.e. 

file.Dispose();

Now all you need is just a device or an internet connection for accessing the JSON response and calling to the Microsoft cognitive services API.

Let us see how it looks like when I take a picture from camera and what comes via JSON response.