Code Coverage on Android

[subrat_mandal]

How to find code coverage on android for the test running using Appium .

[harigovind]

You may take a look at this

http://dtmilano.blogspot.in/2011/11/obtaining-code-coverage-of-running.html thanks to Diego Torres Milano

This is for Emma code coverage tool ,But i believe you are using Jacoco . No worries jacoco team was kind enough to add a compatibility class for Emma, So this method still works . They have RT class from Emma still included . You can use reflection to RT class

So here you go

1. You need to create a BroadcastReceiver class and give the logic to dump coverage data
2. Add jacoco jar reference to Gradle file
3. include your broadcastreceiver class to your build
4. generate apk as any normal build
5. use this apk to run test
6. Send broadcast to dump coverage , you will get a file in your mentioned path (coverage.ec)
7. Now rename coverage.ec to coverage.exec
8. Go to AndroidStudio ->Analyze->Show codecoverage->select coverage.exec file
   and you should be able to see the coverage

CATCH : Make sure you haven't made any other build from the android studio till you generate coverage. Because jacoco need same Binary to process data ( basically class files in bin folder )

If you want to do step 8 via code then refer : http://www.eclemma.org/jacoco/trunk/doc/examples/java/ReportGenerator.java

[subrat_mandal]

@harigovind can you post the code for BroadcastReceiver class

[harigovind]

@subrat_mandal Sorry for the late reply had caught up with some personal work .

For the code please refer to the first link in the previous post , Only deference is link has an activity class .

copy the same code from OnCreate method to OnReceive method in the broadcast listener class

here you go

public class CodeCoverageReceiver extends BroadcastReceiver {

    private final Bundle mResults = new Bundle();
    private static final String TAG = "EMMA";
    private static final boolean LOGD = true;

    private java.io.File coverageFile = null;

    @Override
    public void onReceive(Context context, Intent intent) {
        // TODO Auto-generated method stub

        Tracer.d(TAG, "Dump coverage intent received >>  " + intent.getAction());

        getcodeCoveragePath();
        generateCoverageReport();

        if (coverageFile.canRead() && coverageFile.exists()) {
            Tracer.d(TAG, "ec file generated and it is valid....");
        }

    }

    private void getcodeCoveragePath() {

        String baseDir = Environment.getExternalStorageDirectory().getAbsolutePath();
        String fileName = "coverage.ec";
        coverageFile = new File(baseDir + File.separator + fileName);

    }


    private void generateCoverageReport() {
        if (LOGD)
            Tracer.d(TAG, "generateCoverageReport()");


        Tracer.d(TAG, "coverageFile = " + coverageFile.getAbsolutePath());

        Class<?> emmaRTClass = Class.forName("com.vladium.emma.rt.RT");

        Method dumpCoverageMethod = emmaRTClass.getMethod(
                "dumpCoverageData", coverageFile.getClass(), boolean.class,
                boolean.class);
        dumpCoverageMethod.invoke(null, coverageFile, false, false);

        while (!coverageFile.canRead()) {
            Tracer.d(TAG, "File is getting created.........");
        }

        Tracer.d(TAG, "Coverage.ec File is created at " + coverageFile.getAbsolutePath());

    } catch(
    ClassNotFoundException e)

    {
        reportEmmaError("Is emma jar on classpath?", e);
    } catch(
    SecurityException e)

    {
        reportEmmaError(e);
    } catch(
    NoSuchMethodException e)

    {
        reportEmmaError(e);
    } catch(
    IllegalArgumentException e)

    {
        reportEmmaError(e);
    } catch(
    IllegalAccessException e)

    {
        reportEmmaError(e);
    } catch(
    InvocationTargetException e)

    {
        reportEmmaError(e);
    }

}


    private void reportEmmaError(Exception e) {
        reportEmmaError("", e);
    }

    private void reportEmmaError(String hint, Exception e) {
        String msg = "Failed to generate emma coverage. " + hint;
        Tracer.d(TAG, msg, e);
        mResults.putString(Instrumentation.REPORT_KEY_STREAMRESULT, "\nError: "
                + msg);
    }


}

[jgray]

Hi, do you know the helper class that you referenced in the earlier post? I am having a difficult time trying to figure out how to produce code coverage files with Appium and I hope your method will work for me. We are using JoCoCo but I cannot the method "dumpCoverageMethod" anywhere in the library. Is it called somewhere else or is there any other information you could provide?

[harigovind]

You may have to try with an older version of Jacoco library. I remember RT class from Emma was deprecated in Jacoco library I have used.. Sorry not able to recall exact version.. I will check and update this thread.

[jgray]

Thanks, I appreciate the help!

[harigovind]

I had used , Jacoco-0.7.10.2xxxxx for my project .

[jgray]

Thank you! Also, in your project were you using the com.vladium.emma.rt.RT.dumpCoverageData method or a different method from the JaCoCo library like getExecutionData?

There seems to be an issue with running either. I am receiving an error ( listed below ) whenever I try calling the methods. I am able to run the test through Appium and the test passes. After the test I try creating the coverage file but receive the following error.

Unable to start receiver com.xxx.xxxxx.CodeCoverageReceiver: java.lang.IllegalStateException: JaCoCo agent not started.

[harigovind]

Hmm , I have used com.vladium.emma.rt.RT.dumpCoverageData which anyway calls out.write(org.jacoco.agent.rt.RT.getAgent().getExecutionData() internally .

Did you add Jacocoagent jar to your project ?
http://search.maven.org/remotecontent?filepath=org/jacoco/jacoco/0.7.9/jacoco-0.7.9.zip (will be in lib folder after unzip)

For this setup to work you need to add Jacocoagent.jar as a library to your project (May be copy it on to ur lib folder and right click on it to find add as library.. option. Basically your apk should have Jacocoagent as well bundled .

Please check if you have done that.

[anand-vb]

I am also facing the same issue “java.lang.IllegalStateException: JaCoCo agent not started” and I tried adding jacocoagent.jar file as library in Android studio, but still the same. Is there anything else I need to do to get rid off this issue?

[harigovind]

hmm , look like from here , Agent getInstance() will be thowing IllegalStateException . We should create an agent instance with valid AgentOptions.

Bit tricky though , to find out what are the required VALID_OPTIONS . Will give it a try sometime . Meanwhile if you get a breakthrough please share .

[anand-vb]

Actually when I did clean and rebuild the solution and generated the debug apk. Then it worked for me.

[harigovind]

Just an update to this post . After spending some time i realized that ,what we are missing all along is the instrumentation of the class files with jacocoant . According to the documentation here , “The first instrumented class loaded will trigger the initialization of the JaCoCo runtime. If no instrumented class is loaded the JaCoCo runtime will not get started at all.” .

So that concludes , we need to instrument the class files using jacocoant before proceeding to make the apk .
Now there is no straightforward options to instrument using gradle . But there are few options available like this . I have tried out and was able to instrument the classes but final apk didn’t include instrumented code.

But there is a workaround which worked for me , posting it for others

1 .Generate apk as mentioned earlier on this post ( remember to add jacocoagent jar on the class path)
2. decompile apk’s dex files using dex2jar tool
3. instrument classes using jacocoant instrument task
4. convert classes back to dex files using dx.bat from android build-tools
5. put it back into apk

That’s it. Now you can use that apk for automation / manual testing and send broadcast command to dump coverage file . Also you can used recompiled classes before instrumentation for generating report .

There are useful jacocoant tasks for merge , report etc please do checkout

[vnvbsudhakar]

Will this work for cordova based application?

[harigovind]

Unfortunately it will work only till the point your app calls the Webviews , what happens in Webview is not covered