PelicanHPC provides MPITB for GNU/Octave (homepage at
http://atc.ugr.es/javier-bin/mpitb) which lets you use MPI with Octave. MPI is the "message passing interface", one means of doing parallel computing. OpenMP is a different beast entirely. It is directed toward using all CPUs in a shared memory situation, like a multi-core CPU. It is not meant for distributed memory situations, such as a cluster, where MPI is the most widely used method. I don't have much experience with OpenMP. It could probably be used with Octave in a way similar to how MPITB works, but I don't recall seeing any references, You should probably check on the Octave mail list. If you're interested in MPI and Octave, then PelicanHPC is probably the easiest way to get started.