how to make unix command use multiple CPUs

How can I make a unix command (say a one-liner using cut and awk on a HUGE file) use all 16 cores instead of just 1? This isn't a program where I can use -j and specify the number of CPUs to us开发者_JAVA技巧e...

Have you tried GNU parallel to parallelize jobs? See http://www.gnu.org/software/parallel/

One possible way is to split your input file into a number of pieces and then launch separate shell pipelines for each piece. Multiple processes will take multiple cores.

If you use both cut and awk in the same pipeline, they'll probably run on different CPUs, so you will use two of them. There isn't a simple way to set up a short pipeline like that to use more CPUs than the number of steps in the pipeline.