starpu-max/doc/doxygen/chapters/21simgrid.doxy

/*
 * This file is part of the StarPU Handbook.
 * Copyright (C) 2009--2011  Universit@'e de Bordeaux
 * Copyright (C) 2010, 2011, 2012, 2013, 2014  CNRS
 * Copyright (C) 2011, 2012 INRIA
 * See the file version.doxy for copying conditions.
 */

/*! \page SimGridSupport SimGrid Support

StarPU can use Simgrid in order to simulate execution on an arbitrary
platform.

\section Preparing Preparing your application for simulation.

There are a few technical details which need to be handled for an application to
be simulated through Simgrid.

If the application uses <c>gettimeofday</c> to make its
performance measurements, the real time will be used, which will be bogus. To
get the simulated time, it has to use starpu_timing_now() which returns the
virtual timestamp in us.

For some technical reason, the application's .c file which contains main() has
to be recompiled with starpu_simgrid_wrap.h, which in the simgrid case will # define main()
into starpu_main(), and it is libstarpu which will provide the real main() and
will call the application's main().

To be able to test with crazy data sizes, one may want to only allocate
application data if STARPU_SIMGRID is not defined.  Passing a NULL pointer to
starpu_data_register functions is fine, data will never be read/written to by
StarPU in Simgrid mode anyway.

To be able to run the application with e.g. CUDA simulation on a system which
does not have CUDA installed, one can fill the cuda_funcs with (void*)1, to
express that there is a CUDA implementation, even if one does not actually
provide it. StarPU will never actually run it in Simgrid mode anyway.

\section Calibration Calibration

The idea is to first compile StarPU normally, and run the application,
so as to automatically benchmark the bus and the codelets.

\verbatim
$ ./configure && make
$ STARPU_SCHED=dmda ./examples/matvecmult/matvecmult
[starpu][_starpu_load_history_based_model] Warning: model matvecmult
   is not calibrated, forcing calibration for this run. Use the
   STARPU_CALIBRATE environment variable to control this.
$ ...
$ STARPU_SCHED=dmda ./examples/matvecmult/matvecmult
TEST PASSED
\endverbatim

Note that we force to use the scheduler <c>dmda</c> to generate
performance models for the application. The application may need to be
run several times before the model is calibrated.

\section Simulation Simulation

Then, recompile StarPU, passing \ref enable-simgrid "--enable-simgrid"
to <c>./configure</c>.

\verbatim
$ ./configure --enable-simgrid
\endverbatim

To specify the location of SimGrid, you can either set the environment
variables SIMGRID_CFLAGS and SIMGRID_LIBS, or use the configure
options \ref with-simgrid-dir "--with-simgrid-dir",
\ref with-simgrid-include-dir "--with-simgrid-include-dir" and
\ref with-simgrid-lib-dir "--with-simgrid-lib-dir", for example

\verbatim
$ ./configure --with-simgrid-dir=/opt/local/simgrid
\endverbatim

You can then re-run the application.

\verbatim
$ make
$ STARPU_SCHED=dmda ./examples/matvecmult/matvecmult
TEST FAILED !!!
\endverbatim

It is normal that the test fails: since the computation are not actually done
(that is the whole point of simgrid), the result is wrong, of course.

If the performance model is not calibrated enough, the following error
message will be displayed

\verbatim
$ STARPU_SCHED=dmda ./examples/matvecmult/matvecmult
[starpu][_starpu_load_history_based_model] Warning: model matvecmult
    is not calibrated, forcing calibration for this run. Use the
    STARPU_CALIBRATE environment variable to control this.
[starpu][_starpu_simgrid_execute_job][assert failure] Codelet
    matvecmult does not have a perfmodel, or is not calibrated enough
\endverbatim

The number of devices can be chosen as usual with \ref STARPU_NCPU, \ref
STARPU_NCUDA, and \ref STARPU_NOPENCL, and the amount of GPU memory
with \ref STARPU_LIMIT_CUDA_MEM, \ref STARPU_LIMIT_CUDA_devid_MEM, \ref
STARPU_LIMIT_OPENCL_MEM, and \ref STARPU_LIMIT_OPENCL_devid_MEM.

\section SimulationOnAnotherMachine Simulation On Another Machine

The simgrid support even permits to perform simulations on another machine, your
desktop, typically. To achieve this, one still needs to perform the Calibration
step on the actual machine to be simulated, then copy them to your desktop
machine (the <c>$STARPU_HOME/.starpu</c> directory). One can then perform the
Simulation step on the desktop machine, by setting the environment
variable \ref STARPU_HOSTNAME to the name of the actual machine, to
make StarPU use the performance models of the simulated machine even
on the desktop machine.

If the desktop machine does not have CUDA or OpenCL, StarPU is still able to
use simgrid to simulate execution with CUDA/OpenCL devices, but the application
source code will probably disable the CUDA and OpenCL codelets in thatcd sc
case. Since during simgrid execution, the functions of the codelet are actually
not called, one can use dummy functions such as the following to still permit
CUDA or OpenCL execution:

\section SimulationExamples Simulation examples

StarPU ships a few performance models for a couple of systems: attila and
mirage. See section \ref SimulatedBenchmarks for the details.

\section Tweaking simulation

The simulation can be tweaked, to be able to tune it between a very accurate
simulation and a very simple simulation (which is thus close to scheduling
theory results), see the \ref STARPU_SIMGRID_CUDA_MALLOC_COST and \ref
STARPU_SIMGRID_CUDA_QUEUE_COST environment variables.

\section MPI applications

StarPU-MPI applications can also be run in simgrid mode. It needs to be compiled
with smpicc, and run using the starpu_smpirun script, for instance:

\verbatim
$ STARPU_SCHED=dmda starpu_smpirun -platform cluster.xml -hostfile hostfile ./mpi/tests/pingpong
\endverbatim

Where cluster.xml is a Simgrid-MPI platform description, and hostfile the
list of MPI nodes to be used. StarPU currently only supports homogeneous MPI
clusters: for each MPI node it will just replicate the architecture referred by
\ref STARPU_HOSTNAME.

\section Debugging applications

By default, simgrid uses its own implementation of threads, which prevents gdb
from being able to inspect stacks of all threads.  To be able to fully debug an
application running with simgrid, pass the <c>--cfg=contexts/factory:thread</c>
option to the application, to make simgrid use system threads, which gdb will be
able to manipulate as usual.

\snippet simgrid.c To be included. You should update doxygen if you see this text.


*/