What is Code Coverage???
Code Coverage is a measurement of how many lines/blocks/arcs of your code are executed while the automated tests are running.
Code coverage is collected by using a specialized tool to instrument the binaries to add tracing calls and run a full set of automated tests against the instrumented product. A good tool will give you not only the percentage of the code that is executed, but also will allow you to drill into the data and see exactly which lines of code were executed during particular test.
Btw, while code coverage is a good metric of how much testing you are doing, it is not necessarily a good metric of how well you are testing your product. There are other metrics you should use along with code coverage to ensure the quality.
Code coverage basically tests that how much of your code is covered under tests. So, if you have 90% code coverage than it means there is 10% of code that is not covered under tests. I know you might be thinking that 90% of the code is covered but you have to look from a different angle. What is stopping you to get 100% code coverage?
A good example will be this:
if(customer.IsOldCustomer())
{
}
else
{
}
Now, in the above code there are two paths/branches. If you are always hitting the “YES” branch then you are not covering the else part and it will be shown in the Code Coverage results. This is good because now you know that what is not covered and you can write a test to cover the else part. If there was no code coverage then you are just sitting on a time bomb to explode.
NCover is a good tool to measure code coverage.
Just remember, having “100% code-coverage” doesn’t mean everything is tested completely – while it means every line of code is tested, it doesn’t mean they are tested under every (common) situation..
I would use code-coverage to highlight bits of code that I should probably write tests for. For example, if whatever code-coverage tool shows myImportantFunction() isn’t executed while running my current unit-tests, they should probably be improved.
Basically, 100% code-coverage doesn’t mean your code is perfect. Use it as a guide to write more comprehensive (unit-)tests.
Complimenting few points to many of the answers above.
Code coverage means, how well your test set is covering your source code. i.e. to what extent is the source code covered by the set of Test cases.
As mentioned in above answers, there are various coverage criteria, like paths, conditions, functions, statements, etc. But additional criteria to be covered are
- Condition coverage: All boolean expression to be evaluated for true and false.
- Decision coverage: Not just boolean expression to be evaluated for true and false once, but to cover all subsequent if-elseif-else body.
- Loop Coverage: means, Has every possible loop been executed one time, more than once and zero time. Also, if we have assumption on max limit, then, if feasible, test maximum limit times and, one more than maximum limit times.
- Entry and Exit Coverage: Test for all possible call and its return value.
- Parameter Value Coverage(PVC). To check if all possible values for a parameter are tested. For example, a string could be any of these commonly: a) null, b) empty, c) whitespace (space, tabs, new line), d) valid string, e) invalid string, f) single-byte string, g) double-byte string. Failure to test each possible parameter value may leave a bug. Testing only one of these could result in 100% code coverage as each line is covered, but as only one of seven options are tested, means, only 14.2% coverage of parameter value.
- Inheritance Coverage: In case of object oriented source, when returning a derived object referred by base class, coverage to evaluate, if sibling object is returned, should be tested.
Note: Static code analysis will find if there are any unreachable code or hanging code, i.e. code not covered by any other function call. And also other static coverage. Even if static code analysis reports that 100% code is covered, it does not give reports about your testing set if all possible code coverage is tested.
3 tools to determine code-coverage.
- JTest – a proprietary tool built over JUnit. (It generates unit tests as well)
- Cobertura – an open source code coverage tool that can easily be coupled with JUnit tests to generate reports.
- Emma – another – this one we’ve used for a slightly different purpose than unit testing. It has been used to generate coverage reports when the web-application is accessed by end-users. This coupled with web-testing tools (ex: Canoo) can give you very useful coverage reports which tell you how much code is covered during typical end user usage.We use these tools to
- . Review that developers have written good unit tests
- . Ensure that all code is traversed during black-box testing
https://stackoverflow.com/questions/195008/what-is-code-coverage-and-how-do-you-measure-it
https://www.safaribooksonline.com/library/view/learning-path-android/9781788292993/video2_25.html
Running with Coverage
IntelliJ IDEA provides a dedicated action that allows you to perform run with code coverage measurement. The code coverage data are processed according to the option selected in the Coverage page of the Settings/Preferences dialog box.
To run with code coverage measurement
- Do one of the following:
- Open the desired file in the editor, and choose on the context menu. When running tests with coverage, note that you can run the entire test class, or each individual test method, depending on the caret location.
- Select the desired run/debug configuration, and then on the main menu choose .
- On the main toolbar, click
. This will launch the selected run/debug configuration.
- If the Show options before applying coverage to the editor checkbox has been selected in the Coverage page of the Settings/Preferences dialog box, choose whether you want to replace the active coverage suites, or add the collected data to the active suites, or do not want not apply coverage data. You can also opt to skip this dialog in the future.
In case any other option has been selected, the respective action will be performed silently.
- Explore the collected coverage data in the Coverage Tool Window.
https://www.jetbrains.com/help/idea/2017.3/running-with-coverage.html