Parallel programming is difficult. You need to understand what bottlenecks are holding your app performance back. There is a new tool by Microsoft which can help profile your app. It can assist in finding places where parallelization makes sense. Let’s first understand some theory so we can appreciate the tool.
One CPU actually has many microprocessors in it. This physical core can run multiple threads. The official name for this is Simultaneous Multithreading (SMT). Intel markets this as hyper threading. Logical cores refer to the number of threads that can be running at the same time. This is related by different than the number of physical cores.
So what does this new Concurrency Visualizer tool show you? You can see where your app spends a lot of serial CPU time. You can also inspect how parallel the app already is. Finally you can see whether other applications are affecting your app’s performance.
Here is one note. Heavy file I/O can drastically impact performance. But it may be hard to tell the exact source of the problem. I/O is generally buffered. So the problems may build up before you actually see them. By then it is difficult to get to the root cause.
Multithreading is good up until your thread gets blocked. These can be many different causes for a block. Your own code may be doing synchronization. Or your thread might get preempted by another. Your thread could explicitly do a sleep. Or file I/O could cause you to wait. Finally you might be doing some GUI work to make sure the app seems responsive.
There is a lot more I can say about Concurrency Visualizer. But I am out of time. Just beware that the trace logs might get huge. And it is best to run it first with a small number of other apps running. Go try the tool out for yourself.
