Our architect came into my office recently with a conundrum. We’d recently jumped into concurrency as part of a performance-tuning overhaul. After some profiling, we identified a few hotspots in program initialization and in paths that need to maintain user-responsiveness. The solution was deceptively simple: adopt a parallel for-each in place of the existing sequential for-each. After debugging the thread-unsafe parts of the legacy code invoked from the parallel for-each, we achieved success. Or so we thought. It turns out that ensuring thread-safety along all code paths executed from the parallel for-each, through the course of routine maintenance, is a challenging task. Another programmer on the team later made an innocent change to the code invoked by the parallel for-each, introducing a race condition. It wasn’t clear whose responsibility this was – the breaking change could have been arbitrarily distant from the parallel for-each.
In functional languages, immutability and pure functions (non side-effecting) are core concepts. Many argue that these lead to code that is easier to reason about, and that concurrency is nearly a natural consequence. As Herb Sutter says, “shared state leads to contention”. The corollary is that avoiding shared state can eliminate concurrency concerns (the idea behind lock-free algorithms). Sharing of state occurs not just in obvious places, like global variables. It also occurs in local mutable variables (or members), which are within scope for more than one thread of execution.
The problem is that idiomatic C++ is littered with such mutable state and side-effecting functions. I say “idiomatic” and not “legacy” C++. In this new world order of threading support in the STL, it is oh-so-easy to manage threads – their creation, synchronization, invocation, etc. What’s lacking is a best practice for ensuring that the code executed by threads is safe. Some C++ shops have adopted a pure functional style of programming to ensure thread safety. This is noble and impressive, but in my view impractical. Such code may be thread-safe itself, but must still interface with other C++ code (libraries) which is still idiomatic and thus, not thread-safe. How is thread-safety to be achieved then? The fact that even the venerable go-to collection class, vector, must essentially be rewritten is instructive.
The architect concluded that we had two options: roll back all concurrency work, or train all the developers on staff to be aware of threading and to write ALL of their code to be thread-safe. The former was not an option – we had crossed the Rubicon. In the end, we identified two techniques to achieve the latter: (1) a “top down” (from the parallel for-each) analysis of const correctness; and (2) a “bottom up” analysis of “mutable correctness” (synchronizing access to shared state). But the lesson for us was that with concurrency, you cannot “buy it by the yard” or delegate the responsibility to the local subject matter expert. In other words, the move to concurrency in an organization must be atomic. In that sense, as Sutter has also said, the free lunch is still over.