Best way to comparing perormance of different version of library

Currently for each call in library,I am doing multiple iteration , Measuring time taken for each call and then calculating :

1. Average time taken for ea开发者_如何学Pythonch call
2. Min time take
3. Max time taken
4. Standard Deviation

But it seems its not a good method . As these time depends upon state of machine ,As if CPU is busy with other process result will be impacted.

Please suggest me still if its a good way.Or i can use some better method.

---

It is a good measure of the basic speed of the algorithm but finding total performance is dependent on how you use something. The number of instructions is not useful, either.

For example, say you have some func doMath() written in C, and some templated doMath call that takes 3-4 templated arguments. The C func will often show worse performance in a simple case. However, in a medium to large program the templates will grow to dozens or hundreds or even thousands instead of just one. This will choke out the instruction cache. Same applies for data - some tree implementations will have much more memory efficiency and the more bloated ones will perform better on simple tests, but in actual use they will perform much more poorly.

Similarly, it may seem like a tree has performance through basic tests, but you could find over time it fragments the memory so much that performance drops and drops over time until your program barely functions.

So, performance always depends on your actual use of something, not just its implementation. Not to say it's always a tradeoff, some implementations are better than others. Ultimately, though, the only way to get great performance is to know to a deep level exactly what the computer is doing and exactly what the compiler generates out or else it will always be a mystery to you.

That's part of why people suggest profiling, but profiling will only tell you so much, basically point the way. You will see the symptoms but often not the cause. That's because performance is based on the whole system and it's not just a simple matter of adding up the instruction count.