Emojify Me

Project Synopsis

Develop a mobile app that takes a picture of people using Firebase Mobile Vision identify all the faces in the photo. Lastly, replace all the faces with the proper emoji.

Goals

• Understand the basics of how artificial intelligence works in particular the field of vision.

• Make use of the Firbase MLKit Vision Api to build a face detector.

• Determine how many faces there are on a photo.

• Identify facial characteristics for each face:

  • Right eye open or close
  • Left eye open or close
  • Smiling or frown

• Replace each face with the proper emoji on top of the original photo.

  • Smile:

  

  • Frown:

  

App overview

This section provides an overview of the code that is provided. The started code allows you to concentrate on the Vision api aspect. However, it is important to understand how this code is architecture and how it works.

Single Activity

The application has a single activity. The activity is the entry point of an Android application. The activity is launched once the user click on the application icon.

Screens

The application defines two screens or fragments. A fragment defines its own UI and encapsulates its own functionality.

Home Fragment

The home fragment defines a bottom app bar with a single FAB button that triggers the camera to take a picture.

Layout

The home fragment is defined on the home_fragment.xml as follows:

• Defines a Coordinator Layout as the main container to respond to scrolling techniques from Toolbars.
• Defines an AppBar to displays information or actions relating to the current screen.
• Defines a Material Tool Bar that provides the activity title and can declare other interactive items.
• Defines a Text View to display the title of the application.
• Defines an include layout with the value of home_content.xml
• Defines a Bottom App Bar to define one main FAB located in the center of the bar.
• Floating Action Button (FAB) to denote the primary action of the screen in this case to take a photo.

The home_content.xml is defined as follows:

• Defines a Constraint Layout that allows placing components according to relationships between sibling views and the parent layout.
• Defines two horizontal guidelines the first one at 35% and the second one at 70% of total height of the screen. These guidelines allow to properly place the Material Card View and the Text View.
• Defines a Material Card View that renders a gif image.
• Defines a Text View that renders the text "Take a Selfie".

HomeFragment.java

This section describes the main aspects of the behavior defined for the home_fragment.xml.

Emojify FAB

Here is the code snippet that is called when the user taps on the emojify FAB:

@OnClick(R.id.fab_emojify)
publicvoidonClickEmojifyFAB() {
// Check for the external storage permission// Verify if the app has permission to store a photo on the phoneif (ContextCompat.checkSelfPermission(getContext(), Manifest.permission.WRITE_EXTERNAL_STORAGE)
!= PackageManager.PERMISSION_GRANTED) {
// If you do not have permission, request itActivityCompat.requestPermissions(
getActivity(),
newString[] {Manifest.permission.WRITE_EXTERNAL_STORAGE},
REQUEST_STORAGE_PERMISSION);
} else {
// Launch the camera if the permission existslaunchCameraIntent();
}
}

The snippet above performs the following actions:

• Verify that the user has permission to store files on the phone.
• In case the user does not have permissions a dialog will be shown asking to grant a permission to store files.
• However, if the user does have permissions then launchCameraIntent() will be invoked.

The goal of method launchCameraIntent() is to launch an intent
that allows the user to take a photo.

launchCameraIntent

/** Creates a temporary file in which a picture will be store. */privatevoidlaunchCameraIntent() {
// Create the capture image intentIntenttakePictureIntent = newIntent(MediaStore.ACTION_IMAGE_CAPTURE);
// Ensure that there's a camera activity to handle the intentif (takePictureIntent.resolveActivity(getActivity().getPackageManager()) != null) {
// Create the temporary File where the photo should goFilephotoFile = null;
try {
photoFile = BitmapUtils.createTempImageFile(getContext());
} catch (IOExceptionex) {
// Error occurred while creating the Fileex.printStackTrace();
}
// Continue only if the File was successfully createdif (photoFile != null) {
// Get the path of the temporary fileviewModel.setPhotoPath(photoFile.getAbsolutePath());
// Get the content URI for the image fileUriphotoURI = FileProvider.getUriForFile(getContext(), FILE_PROVIDER_AUTHORITY, photoFile);
// Add the URI so the camera can store the imagetakePictureIntent.putExtra(MediaStore.EXTRA_OUTPUT, photoURI);
// Launch the camera activitystartActivityForResult(takePictureIntent, REQUEST_IMAGE_CAPTURE);
}
}
}

The snippet above performs the following actions:

• Create an intent that allows the user to take a photo called takePictureIntent.
• Validates that there is an app for taking photos installed on the phone.
• Creates a temporary file in where the photo will be stored photoURI.
• On the intent takePictureIntent specify as an extra where the temporary file on where the photo should be store photoURI.
• Launch the intentstartActivityForResult(takePictureIntent, REQUEST_IMAGE_CAPTURE).

The call of startActivityForResult() resolves the intent to an app that can handle the intent and starts its corresponding Activity. However, If there is more than one app that can handle the intent, Android presents the user with a dialog to pick which app to use.

onActivityResult

Lastly, once the picture has been taken and temporarily saved. The camera intent will be completed and Android will call the life cycle method onActivityResult:

@OverridepublicvoidonActivityResult(intrequestCode, intresultCode, @NullableIntentdata) {
super.onActivityResult(requestCode, resultCode, data);
// If the image capture activity was called and was successfulif (requestCode == REQUEST_IMAGE_CAPTURE && resultCode == Activity.RESULT_OK) {
// Resample the saved image to fit the ImageViewBitmapphoto = BitmapUtils.resamplePic(getContext(), viewModel.getPhotoPath());
// Save the photo on the view modelviewModel.setPhoto(photo);
// Navigate to the Photo fragment to see the picture takenNavigation.findNavController(getView()).navigate(R.id.action_mainFragment_to_photoFragment);
} else {
// Otherwise, delete the temporary image fileBitmapUtils.deleteImageFile(getContext(), viewModel.getPhotoPath());
}
}

The snippet above performs the following actions:

• If intent completed successfully the requestCode will be RESULT_OK. And the temporary file will be stored on the ViewModel. Lastly, the navigation component will render the photo_fragment.xml.
• However, if the intent was not successful the temporary file will be deleted.

Photo Fragment

The photo fragment defines a bottom app bar with a FAB button that saves a picture and two more buttons one to share a photo the second one to delete it.

Layout

The photo fragment is defined on the photo_fragment.xml as follows:

• Defines a Coordinator Layout as the main container to respond to scrolling techniques from Toolbars.
• Defines an AppBar to displays information or actions relating to the current screen.
• Defines a Material Tool Bar that provides the activity title and declares an interactive item to close the current fragment and go back to the home_fragment.xml.
• Defines a Text View to display the title of the application in this case "Photo".
• Defines an include layout with the value of photo_content.xml
• Defines a Bottom App Bar to define one main FAB located in the end of the bar.
• Floating Action Button (FAB) to denote the primary action of the screen in this case to save the current photo on the phone.

The photo_content.xml is defined as follows:

• Defines a Constraint Layout that allows placing components according to relationships between sibling views and the parent layout.
• Defines a horizontal guidelines at 50% of total height of the screen. This guidelines allow to properly place the Material Card View.
• Defines a Material Card View that renders the photo that was take with the camera.

PhotoFragment.java

This section describes the main aspects of the behavior defined for the photo_fragment.xml.

Save FAB

Here is the code snippet that is called when the user taps on the save FAB:

/** OnClick method for the save button. */@OnClick(R.id.fab_save)
publicvoidonClickSaveFAB() {
// Save the imageStringphotoPath = BitmapUtils.saveImage(getContext(), viewModel.getPhoto());
viewModel.setPhotoPath(photoPath);
}

The snippet above performs the following actions:

• Retrieves the photo from the ViewModel and saved it on the phones gallery.
• Upon successfully storing the photo the path is stored on the ViewModel.

Share Action Menu

Here is the code snippet that is called when the user taps on the share action menu:

/** OnClick method share action menu. */privatevoidonShareMenuSelected() {
onClickSaveFAB();
// Share the imageBitmapUtils.shareImage(getContext(), viewModel.getPhotoPath());
}

The snippet above performs the following actions:

• Calls the onClickSaveFAB to save the photo.
• Lastly, a share intent is trigger to allow the user share a photo through a list of different apps such as Google Photos, Google Drive to name a few.

Delete Action Menu

Here is the code snippet that is called when the user taps on the delete action menu:

/** OnClick method delete action menu. */privatevoidonDeleteMenuSelected() {
// Delete the temporary image fileBitmapUtils.deleteImageFile(getContext(), viewModel.getPhotoPath());
}

The snippet above performs the following actions:

• Deletes the temporary file where the photo is stored.

MaterialToolbar Navigation OnClickListener

Here is the code snippet that is called when the user taps on the Material Tool Bar Navigation:

// Specifies the close navigation listener to go back to the home fragmenttopToolbar.setNavigationOnClickListener(
v ->
Navigation.findNavController(getView())
.navigate(R.id.action_photoFragment_to_mainFragment));

The snippet above performs the following actions:

• Closes the photo_fragment.xml and navigates back to the home_fragment.xml.

Navigation between fragments

The application make use of navigation component to navigate between the screens. Navigation refers to the interactions that allow users to navigate across, into, and back out from the different screens of an application.

Main Activity Layout

One of the core parts of the Navigation component is the navigation host.
The navigation host is an empty container where destinations are swapped in and out as a user navigates through your app.

<androidx.fragment.app.FragmentContainerView
android:id="@+id/nav_host_fragment"android:name="androidx.navigation.fragment.NavHostFragment"android:layout_width="0dp"android:layout_height="0dp"app:defaultNavHost="true"app:layout_constraintBottom_toBottomOf="parent"app:layout_constraintLeft_toLeftOf="parent"app:layout_constraintRight_toRightOf="parent"app:layout_constraintTop_toTopOf="parent"app:navGraph="@navigation/nav_graph" />

Add destinations to the navigation graph

The destinations are defined on the nav_graph.xml as follows:

<?xml version="1.0" encoding="utf-8"?>
<navigationxmlns:android="http://schemas.android.com/apk/res/android"xmlns:app="http://schemas.android.com/apk/res-auto"xmlns:tools="http://schemas.android.com/tools"android:id="@+id/nav_graph"app:startDestination="@id/mainFragment">
<fragmentandroid:id="@+id/mainFragment"android:name="com.google.codenext.emojify.ui.HomeFragment"android:label="main_fragment"tools:layout="@layout/home_fragment" >
<actionandroid:id="@+id/action_mainFragment_to_photoFragment"app:destination="@id/photoFragment" />
</fragment>
<fragmentandroid:id="@+id/photoFragment"android:name="com.google.codenext.emojify.ui.PhotoFragment"android:label="photo_fragment"tools:layout="@layout/photo_fragment" >
<actionandroid:id="@+id/action_photoFragment_to_mainFragment"app:destination="@id/mainFragment" />
</fragment>
</navigation>

Navigate to a destination

Navigating to a destination is done using a NavController, an object that manages app navigation within a NavHost. Therefore, to navigate from one destination to another is done via the following method:

Navigation.findNavController(View)

• To Navigate from photo_fragment.xml to home_fragment.xml is accomplished as follow:

Navigation.findNavController(getView())
.navigate(R.id.action_photoFragment_to_mainFragment));

• To Navigate from home_fragment.xml to photo_fragment.xml is accomplished as follow:

Navigation.findNavController(getView())
.navigate(R.id.action_mainFragment_to_photoFragment);

ViewModel that stores information

A ViewModel is a class designed to store and manage UI-related data in a lifecycle conscious way. The ViewModel class allows data to survive configuration changes such as screen rotations.

MainActivityViewModel

The ViewModel allows to share the photo and the location where the photo is stored on the phone to allow the fragments communicated among them.

publicclassMainActivityViewModelextendsViewModel {
privatestaticfinalStringTAG = "MainActivityViewModel";
privateStringphotoPath;
privateBitmapphoto;
/** Provides the file path of the where a photo is stored on the device. */publicStringgetPhotoPath() {
returnphotoPath;
}
/** Stores the file path of where a given photo is stored on the device. */publicvoidsetPhotoPath(StringphotoPath) {
Timber.tag(TAG).d(photoPath);
this.photoPath = photoPath;
}
/** Provides a {@link Bitmap} JPG photo. */publicBitmapgetPhoto() {
returnphoto;
}
/** Stores a JPG photo as a {@link Bitmap} */publicvoidsetPhoto(Bitmapphoto) {
this.photo = photo;
}
}

Share data between fragments

It's very common that two or more fragments in an activity need to communicate with each other. For our app we need to share the photo taken with the camera as well as its location on where is stored on the phone.

To accomplish this these fragments share a ViewModel using MainActivity scope to handle this communication.

• This is defined in the HomeFragment as follows:

/** Home fragment initiates the camera that captures a selfie in landscape. */publicclassHomeFragmentextendsFragment {
privateMainActivityViewModelviewModel;
@OverridepublicvoidonCreate(@NullableBundlesavedInstanceState) {
super.onCreate(savedInstanceState);
viewModel = ViewModelProviders.of(getActivity()).get(MainActivityViewModel.class);
}
...

• This is defined in the PhotoFragment as follows:

/** Photo fragment that allows to emojify a selfie as well as allow to save it, share it or discard it. */publicclassPhotoFragmentextendsFragment {
privateMainActivityViewModelviewModel;
@OverridepublicvoidonCreate(BundlesavedInstanceState) {
super.onCreate(savedInstanceState);
viewModel = ViewModelProviders.of(getActivity()).get(MainActivityViewModel.class);
}
...