AB's prototype distributed computing implementation

Go back: Horace framework redesign ideas

---

Matlab based distributed implementation

AB has written several documents which are in the Horace and Herbert trees regarding a proposed architecture for implementing parallelisation of Horace/Herbert on a distributed computing system:

• Parallel Horace.pptx
• ParallelFrameworkDesign.docx

The presentation is a briefer summary of the ideas in the docx, but includes information (including implemented syntax) of the prototype "MPI"-like class written by AB.

An even briefer summary:

• A distributed parallel "prototype" framework has been implemented.
• It only works for the user facing gen_sqw command at present.
  • Extending it to work for cut_sqw and sqw_eval is planned and will not take too much effort.
• It can use one of two "back-ends" for inter-node communications:
  • A pure Matlab "poor-man's MPI" class which uses temporary files to pass data.
  • The Matlab parallel computing toolkit if the user has this installed.
    • This itself can use either a (true) MPI backend (needs user configuration and a MPICH2 compatible library installed)
    • Or a file-based fallback.
  • A C++ (mex) backend which wraps an MPI library without needing the parallel computing toolkit is planned. This mex interface will run a compiled Matlab instance on the child nodes to do the actual processing.
• In all cases the design assumes that Horace is running on a system with a shared filesystem (that is, all nodes should have read/write access to the same location, including access to data/sqw files as well as temporary files).

---

Go back: Horace framework redesign ideas