Can I merge Emma coverage data from unit and integration test targets?_问答_开发者

We have our TeamCity builds set up using a build chain, so that our unit tests开发者_JS百科 and and integration tests can run in parallel when triggered by a commit:

Build Chain - dependant on:
- Unit tests
- Integration tests

I am looking for a way that we can combine/merge the coverage data generated by the unit and integration tests in the build chain, so that we can get a better picture of how much actual code is covered by the two combined.

The plan then is to be able to monitor changes in coverage of committed code, and perhaps failing builds if percentages fall!

I have set up the 'build chain' target so that the coverage files (*.em, *.ec) from the unit and integration targets are available to it.

I created an ant build file specifically for the build chain target (with help from the emma doco!):

<project name="coverage-merge" basedir="." default="all">
    <!-- directory that contains emma.jar and emma_ant.jar: -->
    <property name="emma.dir" value="${basedir}/lib"/>
    <property name="coverage.dir" location="${basedir}/coverage"/>

    <path id="emma.lib">
        <pathelement location="${emma.dir}/emma-teamcity-3.1.1.jar"/>
        <pathelement location="${emma.dir}/emma_ant-2.0.5312.jar"/>
    </path>

    <taskdef resource="emma_ant.properties" classpathref="emma.lib"/>

    <target name="all" depends="-report"/>

    <target name="-report">
        <emma>
            <report sourcepath="${src.dir}" sort="+block,+name,+method,+class" 
                    metrics="method:70,block:80,line:80,class:100">
                <infileset dir="${coverage.dir}" includes="**/*.em, **/*.ec"/>

                <!-- for every type of report desired, configure a nested
                     element; various report parameters
                     can be inherited from the parent <report>
                     and individually overridden for each report type:
                -->
                <txt outfile="${coverage.dir}/coverage.txt" depth="package" 
                        columns="class,method,block,line,name"/>
                <xml outfile="${coverage.dir}/coverage.xml" depth="package"/>
                <html outfile="${coverage.dir}/coverage.html" depth="method" 
                        columns="name,class,method,block,line"/>
            </report>
        </emma>
    </target>
</project>

...which merges all the coverage files into a single report!

The metrics parameter of report sets the highlight threshold for the html report, so that the percentages against packages and files that are lower than the threshold are highlighted in red.

Modifying the xml output will allow me to use something like andariel to run an xpath over the results, and then force the build to fail if thresholds are not met!

Per TC's Emma doc

All coverage.* files are removed in the beginning of the build, so you have to ensure that full recompilation of sources is performed in the build to have the actual coverage.em file.

What I did to workaround this is below:

Use -out emma.em in teamcity build steps config, and make sure merge option is set to true to preserve the instrumented data.
In the last step when coverage report is generated, use ant's move task <move file="$YOUR_PATH/emma.em" tofile="$YOUR_PATH/coverage.em"/> to rename to the default one.
The emma report will pick up the default em file to generate report.

Hope this helps whoever whats to have an accumulated emma coverage report.

Most of the code coverage tools I've encountered do not seem to have a way to combine test results from different or overlapping subsystems. As you have pointed out, this is a very useful ability.

Our SD Test Coverage tools do have this ability and are available for Java, C, C++, C#, PHP and COBOL. In fact, the SD test coverage tools can combine test coverage data from multiple languages into a single monolithic result, so that you can get an overview of test coverage for your multi-lingual applications. It is able to show the coverage on all the source langauges involved, as well as provide summary reports.