Multithreaded operators

How can an operator +, -, *, /, etc... be multithreaded?

In this paper on the language IDL, it claims that all operators use the 'thread pool' for increased execution speed.

How is it that one can use multiple threads to execute a statement such as 'b = a - a' (as on page 42) of the paper?

Can someone explain this? (I currently consider IDL a total ripoff, but maybe someone can change my mind.)

(Really this appl开发者_运维知识库ies to any language, how can an operator by mulithreaded in any computer programming language?)

---

I think it's important to also consider that not all operations with + are created equal. If you're using some sort of bignum library, for example, you might be able to seperate a large number into smaller parts, do distinct sums of integers( in parallel), then carry over. In any case, it's not going to be a single-cycle addition of to integers. multiplication involves a couple steps, and division, a lot of steps.

In the example given, the floating points(floating point means a non-trivial adding process) had "4.2 million data points": I doubt they were storing that in a small 32-bit register. The "simple" operation for addition has suddenly become a huge iterative process... or maybe something a lot faster if they are able to do it in parallel.

While simple operations with small integers might not be worth threading, it's worthwhile to note that B=A+A, while seeming simple, could actually lead to many calculations. 1 line of code doesn't necessarily mean 1 operation.

---

I don't know about IDL, but it's certainly possible if you have some higher level types. For instance you could conveniently parallelize array operations. Presumably that's what the "4200000 pts" refers to, although someone decided to make the graphs really hard to read.

For comparison, in C (with possible OpenMP parallelization) you might have something like:

#pragma omp parallel for
for (int i=0; i<sizeof(B)/sizeof(B[0]); i++) {
    B[i]-=A[i];
}

In a higher level language, such as NumPy, Matlab or C++, it could be just B=B-A. All that said, B=A-A sounds confusingly like B=0 to me.

You asked for a parallel operator in a favorite language? Here's a bit of Haskell:

import Control.Parallel

pmap _ [] = []
pmap f (x:xs) =
  let rest=pmap f xs
  in rest `par` (f x):rest

parOp op a b = pmap (uncurry op) (zip a b)

parAdd = parOp (+)

main = do
  putStrLn$show ([0..300] `parAdd` [500..800])

Yes, it's still a loop. Multitude of operations (not operators) is key to this type of parallelism.

---

Primitive operations on matrices, arrays, etc. can be parallelised - scroll up to page 41 and you'll find:

For system comparison plots, the results are reported for arrays of 4.2 million elements.

Edit: Assume you have an array A = [1, 2, 3, 4, 5, 6].

Calculating B = A - A = [0, 0, 0, 0, 0, 0] involves 6 subtraction operations (1-1, 2-2, etc).

• with a single CPU, regardless of number of threads, the subtractions must be performed in series.
• with multiple CPUs but only one thread, the subtractions are also performed in series - that's by definition of a thread.
• multiple CPUs, multiple threads - the subtractions can be divided amongst threads/CPUs and thus occur simultaneously (up to the number of CPUs available).