CDash Custom Dynamic Analysis

I'm trying to integrate custom dynamic analysis tools to CDash. Such as KWStyle, CppCheck and Visual Leak Detector.

I'v figured out that I need to generate a DynamicAnalysis.xml file and submit it to CDash, from CTest scripts.

I think I know how to run the external tool as a part of the ctest script.

Either by using these variables to change how ctest_memcheck() works

CTEST_MEMORYCHECK_COMMAND

CTEST_MEMORYCHECK_SUPPRESSIONS_FILE

CTEST_MEMORYCHECK_COMMAND_OPTIONS

or by running the tool from the execute_process() command.

But I'm a bit uncertain which one to use.

The main problem I think I have is, how can I extract errors from the output of the custom tool and include that information into the DynamicAnalysis.xml to submit?

The extreme solution i see is that i'd need to make a program that generates a valid DynamicAnalysis.xml file.

But the problem is that I don't know the syntax of the DefectList element in the XML file. I have found no answer from google and even the XML Schema for that file is unhelpful.

EDIT:

Looking at this:

http://www.cdash.org/CDash/viewDynamicAnalysis.php?buildid=987149

What draws my attention are the labels, especially the empty ones. I don't see how these would come from the DynamicAnalysis.xml file. Maybe it tracks any labels that have ever appearred? Can i create my own custom labels somehow?

Does CDash create the labels automatically, depending on the tool type? Does this block custom defect types?

I'm just guessing here, so the question is; can i create custom labels for my custom tool, just by generating a DynamicAnalysis.xml - file.

It occurred to me that the am开发者_如何学Pythonount of different errors from CppCheck (static code analysis) is huge, compared to valgrind for instance. I'm not that certain that I should use the dynamic analysis. Maybe a custom build type (Continuous / Experimental / Nightly) thing would work better. Like this:

http://www.cdash.org/CDash/buildSummary.php?buildid=930174

I have no idea how to do this, i guess it requires meddling around with CDash code?

Which one would work better?

---

If you are using valgrind, you can simply set CTEST_MEMORYCHECK_COMMAND to the full path to valgrind, and ctest will generate the DynamicAnalysis.xml file for you from the valgrind output when you call ctest_memcheck.

The best way to understand the possible values that can appear in the DynamicAnalysis.xml file is to analyze the source code of CTest.

The file CMake/Source/CTest/cmCTestMemCheckHandler.cxx has the list of defect types in a variable named "cmCTestMemCheckResultLongStrings". Search through that file for references to that variable to see what the possible values are and how they are used to generate "<Defect/>" xml elements.

EDIT (for additional information):

You can also easily see what XML elements CDash is expecting by inspecting its source code. Specifically, the file "CDash/xml_handlers/dynamic_analysis_handler.php".

---

From what I'v learned so far, is that for a tool that runs on the tests made in the cmake script, the Dynamic Analysis is the thing.

For tools that run on the entire program, a custom Build.xml is the thing you need.

I found out that i can commit those files from the ctest_submit command by using the FILES parameter.

I also found out that you can add custom "build names" to the side of Continuous, Nightly, and others.

And that you can set the builds from certain machines to be automatically transferred under these.

The custom labels under DynamicAnalysis did come from somewhere in CDash, i can't remember where anymore.