Ask HN: Are MPI implementations simply a wrapper on shared memory for intra-node

2 points by systemskid 18 hours ago | 4 comments

HPC noob here: Skimming through Open MPI implementation, it seems like it uses SM for intra-node communication? Isn't one of the points of MPI to use message queues for such communication, which differentiates it from shared memory based approaches?

PaulHoule 18 hours ago |
The culture of MPI does not value simplicity of implementation but rather optimizing every possible case so I would expect MPI to have an optimized implementation if two processes are on the same node.
dekhn 18 hours ago |
No. MPI is an interface, free to use a wide range of implementations.
systemskid 17 hours ago |
Interesting. So since shared memory is faster, it became the de facto for intra-node comm? I always thought shared memory was like an added feature and you needed separate APIs like MPI_allocate_shared_mem to make MPI use SM.
dekhn 14 hours ago |
I believe shared memory has been used for intra node in MPI implementations for a long time.
You're confusing what MPI exposes as part of its interface (API call to allocate shared memory) and how an MPI implementation uses the hardware to achieve its goals.
MPI is subtle, complex, and surprising, and implementations even subtle, complex, and surprising. It's designed to eke the last bit of performance out of systems and thus the implementations may use any range of system details so long as they don't violate the contracts in the MPI design. I don't think the original MPI designers really considered intra-node communication as particularly important, at the time most machines were single core and the parallelism was between machines. Then massively multicore with coherent memory models showed up, and the MPI implementers (both OpenMPI and MPICH; I don't know about any commercial implementations) added explicit shared memory to the API (the call you reference) and adopted shared memory as an internal transport.