GSOC Project for Branch Coverage reporting for Python

As there appears to be some interest in branch coverage and the cool testing/metrics/quality tools that can be built upon it, I've proposed a GSOC project to start the ball rolling. (I assume it will start rolling either way, but allowing someone to devote more time to it is always nice).

I put a short writeup on the Python SOC wiki page. Here's a little more info. There's only so much one can do with line coverage, but that's the best python has at the moment because the one can only trace at a line level. If tracing were enabled to happen at a condition level, then branch (and path) coverage can be reported. For people who want to go hog wild and try to do full path coverage, they can go ahead. For those who want a little more than branch coverage, there is Structured Testing. (For people who don't want to go that far, perhaps you have a bunch of tests that take a while to run, if we have path data, it would be able to tell you which tests were redundant and you could cut your testing time).

Here's my thoughts on what the project might include:

• Update sys.settrace to enable branch coverage (or add a new function)
• Create library to implement branch coverage using above tool
• Create command line app using said library to implement branch tracing (aggregating subsequent runs) and dump/pickle data
• Create command line app to load and report on data
• (Test above)
• Bonus stuff
• Calculate flow graphs
• Calculate basis paths through graphs
• Take branch reports/basis paths and determine missing paths/tests
• Create visualization for flow graphs/extend Ned's work
• Create visualization for basis paths (in pygame?) that links code to flow graphs and tests to executed paths
• Redundant test finder
• .... cool stuff you think of

I'm willing to mentor this. If you are interested please or have suggestions please let me know. Comments? Other ideas?

tags: branch coverage  gsoc  python  coveragepy  figleaf  complexity  mccabe  

links: digg this    del.icio.us    reddit