June 18, 2020

Firebase machine learning kit tutorial

By Mohit Agrawal

Easy tutorial for integrating Firebase machine learning kit in android

Overview

There was a time when learning and implementing machine learning was not an easy task. And if we talk about implementing the machine learning in the mobile devices then it was not possible only because the execution of the heavy algorithm needs heigh computing power. But as we know, mobile technology has grown exponentially in the past few years.

Firebase is one of them. It has recently announced a new feature which is Firebase Machine Learning kit. In this tutorial, I will explain everything about it in detail. I will also show you how to Integrate the Firebase machine learning kit in your android app.

What is the Firebase Machine Learning kit?

ML kit is one of the new features of the firebase. It is a mobile SDK that provides machine learning features to the mobile app. Firebase has made machine learning for mobile devices so easy that even if you don’t have the experience with the neural network then you can build it with just a few lines of code.

Firebase Machine learning kit comes in three variants. They are:

  • APIs: These are pre-trained models by google. We just need to pass the input data and we get the result. Let’s take an example to understand. Suppose you are using a ‘Text Recognition” API then you just need to pass an image to the API and it will give all the text recognized by the model.
  • Custom: In this feature, you can upload your own custom model to the firebase portal and then serve them to your mobile device.
  • AutoML: Train high-quality custom machine learning models with minimum effort and machine learning expertise.

Integration of Firebase Machine learning kit tutorial

Now let’s see what are the different SDK’s under API variant.

Firebase Machine learning kit

We get many following pre-trained machine learning model in APIs variant:

  • Text recognition: Recognize and extract text from images.
  • Image labeling: To identify objects, places, human, writing, and actions in images.
  • Landmark recognition: To recognize landmarks in a given image.
  • Face detection: It detects faces and facial landmarks with contours.
  • Barcode scanning: It scans the barcodes.
  • Translation: It translates the text from one language to another.
  • Smart Reply: To automatically generate relevant replies to messages.
  • Image labeling: To identify objects, locations, activities, animal species, and much more.
  • Object detection & tracking: It detects and tracks the object in camera and static images.

Let’s build a text recognition android app

I will show you the process of integrating the Firebase ML kit in your android project. And we will use text recognition API to identity all the text written on an image.

STEP 1 Create a new android project and connect it with your firebase account. You can follow my latest firebase tutorial to learn how to connect the android studio with firebase.

STEP 2 Open the app level gradle file and add this firebase dependency.

dependencies {
  // ...

  implementation 'com.google.android.gms:play-services-mlkit-text-recognition:16.0.0'
}

STEP 3 Now open AndroidManifest.xml and add this before the closing of the application tag.

<application ...>
  ...
  <meta-data
      android:name="com.google.mlkit.vision.DEPENDENCIES"
      android:value="ocr" />
  <!-- To use multiple models: android:value="ocr,model2,model3" -->
</application>

Google says this is an optional but recommended. If you do this step then the model will be downloaded when the app starts the first time itself.

Now add the permission to access the image from the gallery.

<uses-permission android:name="android.permission.INTERNET"/>
    <uses-permission android:name="android.permission.WRITE_EXTERNAL_STORAGE" />
    <uses-permission android:name="android.permission.READ_EXTERNAL_STORAGE" />

STEP 4 Let’s build the UI for the app. Open activity_main.xml and add the this code.

<?xml version="1.0" encoding="utf-8"?>
<androidx.constraintlayout.widget.ConstraintLayout xmlns:android="http://schemas.android.com/apk/res/android"
    xmlns:app="http://schemas.android.com/apk/res-auto"
    xmlns:tools="http://schemas.android.com/tools"
    android:layout_width="match_parent"
    android:layout_height="match_parent"
    android:layout_centerInParent="false"
    android:layout_centerHorizontal="true"
    android:layout_centerVertical="false"
    tools:context=".MainActivity">

    <LinearLayout
        android:layout_width="0dp"
        android:layout_height="0dp"
        android:baselineAligned="true"
        android:orientation="vertical"
        app:layout_constraintBottom_toBottomOf="parent"
        app:layout_constraintEnd_toEndOf="parent"
        app:layout_constraintStart_toStartOf="parent"
        app:layout_constraintTop_toTopOf="parent">

        <ImageView
            android:id="@+id/imageView"
            android:layout_width="300dp"
            android:layout_height="300dp"
            android:layout_gravity="center"
            android:visibility="visible"
            app:srcCompat="@android:drawable/ic_menu_gallery"
            tools:srcCompat="@tools:sample/backgrounds/scenic" />

        <Button
            android:id="@+id/button"
            android:layout_width="wrap_content"
            android:layout_height="48dp"
            android:layout_gravity="center"
            android:text="Open Gallary" />

        <TextView
            android:id="@+id/textView"
            android:layout_width="match_parent"
            android:layout_height="match_parent"
            android:layout_gravity="center"
            android:gravity="center"
            android:text="Choose an image from the gallary." />
    </LinearLayout>
</androidx.constraintlayout.widget.ConstraintLayout>

The UI should look like the below screenshot. It has one image view, one button to open the gallery, and one text view to display all the texts identified by the model.

Firebase machine learning kit app ui
Firebase machine learning kit app ui

STEP 5 This is the step where we actually use the API code in the MainActivity.kt. Copy the below code to your file.

package com.example.textrecognition

import android.content.Intent
import android.database.Cursor
import android.graphics.BitmapFactory
import android.net.Uri
import android.os.Bundle
import android.provider.MediaStore
import android.util.Log
import androidx.appcompat.app.AppCompatActivity
import com.google.mlkit.vision.common.InputImage
import com.google.mlkit.vision.text.Text
import com.google.mlkit.vision.text.TextRecognition
import kotlinx.android.synthetic.main.activity_main.*
import java.net.URI


var RESULT_LOAD_IMAGE = 1

class MainActivity : AppCompatActivity() {
    override fun onCreate(savedInstanceState: Bundle?) {
        super.onCreate(savedInstanceState)
        setContentView(R.layout.activity_main)

        button.setOnClickListener {
            val i = Intent(
                    Intent.ACTION_PICK,
                    MediaStore.Images.Media.EXTERNAL_CONTENT_URI)
            startActivityForResult(i, RESULT_LOAD_IMAGE)
        }
    }

    override fun onActivityResult(requestCode: Int, resultCode: Int, data: Intent?) {
        super.onActivityResult(requestCode, resultCode, data)

        if (requestCode == RESULT_LOAD_IMAGE && resultCode == RESULT_OK && null != data) {
            var uri = data.data
            val filePathColumn = arrayOf(MediaStore.Images.Media.DATA)
            val cursor: Cursor? = contentResolver.query(uri!!,
                    filePathColumn, null, null, null)
            cursor?.moveToFirst()
            val columnIndex: Int = cursor!!.getColumnIndex(filePathColumn[0])
            val picturePath: String = cursor.getString(columnIndex)
            cursor?.close()
            imageView.setImageBitmap(BitmapFactory.decodeFile(picturePath));
            runTextRecognition(uri)
        }
    }

    private fun runTextRecognition(uri: Uri) {
        val image = InputImage.fromFilePath(this, uri)
        val recognizer = TextRecognition.getClient()
        button.isEnabled = false
        recognizer.process(image)
                .addOnSuccessListener { texts ->
                    button.isEnabled = true
                    processTextRecognitionResult(texts)
                }
                .addOnFailureListener { e -> // Task failed with an exception
                    button.isEnabled = true
                    e.printStackTrace()
                }
    }

    private fun processTextRecognitionResult(texts: Text) {
        Log.i("my", texts.text)
        //Toast.makeText(this, texts.toString(), Toast.LENGTH_LONG).show();
        textView.text = texts.text
    }
}

As you can see in the above code I am simply opening the gallery to pick an image then I am passing the URI of the selected image to the function runTextRecognition(). In this function, I am calling the text recognition API which is val recognizer = TextRecognition.getClient(). So this is how you can use the firebase machine learning in your android app.

If you have done everything right till here then your app should work like this.

Source code link.

Subscribe YouTube: More tutorials like this

I hope this blog post is useful for you, do let me know your opinion in the comment section below.
I will be happy to see your comments down below 👏.
Thanks for reading!!!

  1. Waiting for the text recognition model to be downloaded. Please wait. [Solved]

    Just wait for around 10 minutes after running the code. It will download the model in background. But if the issue stills persist then run it on a physical device to solve the issue.

Recent posts.

android android beginners tutorial android tutorial Animation API call API call in iOS async asynchronous cardview custom switch custom ui dark web dark web links deep web deep web links Dynamic gradient view facebook messenger firebase hosting firebase hosting tutorial firebase remote config firebase tutorials flutter flutter for beginner flutter tutorial flutter UI tutorial hacking hacking tutorial hashcat iOS ios tutorial ios ui tutorial kotlin listview in flutter Lottie animation Messenger for kids review onion links Roll the Dice app swift tech news tor tor browser tor links UISwitch wifi wifi hacking