starpu-max/doc/doxygen/chapters/430_mic_scc_support.doxy

/*
 * This file is part of the StarPU Handbook.
 * Copyright (C) 2009--2011  Universit@'e de Bordeaux
 * Copyright (C) 2010, 2011, 2012, 2013, 2016, 2017  CNRS
 * Copyright (C) 2011, 2012 INRIA
 * See the file version.doxy for copying conditions.
 */

/*! \page MICSCCSupport MIC Xeon Phi / SCC Support

\section Compilation Compilation

SCC support just needs the presence of the RCCE library.

MIC Xeon Phi support actually needs two compilations of StarPU, one for the host and one for
the device. The <c>PATH</c> environment variable has to include the path to the
cross-compilation toolchain, for instance <c>/usr/linux-k1om-4.7/bin</c> .
The <c>SINK_PKG_CONFIG_PATH</c> environment variable should include the path to the
cross-compiled <c>hwloc.pc</c>.
The script <c>mic-configure</c> can then be used to achieve the two compilations: it basically
calls <c>configure</c> as appropriate from two new directories: <c>build_mic</c> and
<c>build_host</c>. <c>make</c> and <c>make install</c> can then be used as usual and will
recurse into both directories. If different configuration options are needed
for the host and for the mic, one can use <c>--with-host-param=--with-fxt</c>
for instance to specify the <c>--with-fxt</c> option for the host only, or
<c>--with-mic-param=--with-fxt</c> for the mic only.

One can also run StarPU just natively on the Xeon Phi, i.e. it will only run
directly on the Phi without any exchange with the host CPU. The binaries in
<c>build_mic</c> can be run that way.

For MPI support, you will probably have to specify different MPI compiler path
or option for the host and the device builds, for instance:

\verbatim
./mic-configure --with-mic-param=--with-mpicc="/.../mpiicc -mmic" \
    --with-host-param=--with-mpicc=/.../mpiicc
\endverbatim

In case you have troubles with the coi or scif libraries (the Intel paths are
really not standard, it seems...), you can still make a build in native mode
only, by using <c>mic-configure --enable-native-mic</c> (and notably without
<c>--enable-mic</c> since in that case we don't need mic offloading support).

\section PortingApplicationsToMICSCC Porting Applications To MIC Xeon Phi / SCC

The simplest way to port an application to MIC Xeon Phi or SCC is to set the field
starpu_codelet::cpu_funcs_name, to provide StarPU with the function
name of the CPU implementation, so for instance:

\verbatim
struct starpu_codelet cl =
{
    .cpu_funcs = {myfunc},
    .cpu_funcs_name = {"myfunc"},
    .nbuffers = 1,
}
\endverbatim

StarPU will thus simply use the
existing CPU implementation (cross-rebuilt in the MIC Xeon Phi case). The
functions have to be globally-visible (i.e. not <c>static</c>) for
StarPU to be able to look them up, and -rdynamic must be passed to gcc (or
-export-dynamic to ld) so that symbols of the main program are visible.

If you have used the <c>.where</c> field, you additionally need to add in it
<c>STARPU_MIC</c> for the Xeon Phi, and/or <c>STARPU_SCC</c> for the SCC.

For non-native MIC Xeon Phi execution, the 'main' function of the application, on the sink, should call starpu_init() immediately upon start-up; the starpu_init() function never returns. On the host, the 'main' function may freely perform application related initialization calls as usual, before calling starpu_init().

For MIC Xeon Phi, the application may programmatically detect whether executing on the sink or on the host, by checking whether the STARPU_SINK environment variable is defined (on the sink) or not (on the host).

For SCC execution, the function starpu_initialize() also has to be
used instead of starpu_init(), so as to pass <c>argc</c> and
<c>argv</c>.

\section LaunchingPrograms Launching Programs

SCC programs are started through RCCE.

MIC programs are started from the host. StarPU automatically
starts the same program on MIC devices. It however needs to get
the MIC-cross-built binary. It will look for the file given by the
environment variable \ref STARPU_MIC_SINK_PROGRAM_NAME or in the
directory given by the environment variable \ref STARPU_MIC_SINK_PROGRAM_PATH,
or in the field
starpu_conf::mic_sink_program_path. It will also look in the current
directory for the same binary name plus the suffix <c>-mic</c> or
<c>_mic</c>.

The testsuite can be started by simply running <c>make check</c> from the
top directory. It will recurse into both <c>build_host</c> to run tests with only
the host, and into <c>build_mic</c> to run tests with both the host and the MIC
devices. Single tests with the host and the MIC can be run by starting
<c>./loader-cross.sh ./the_test</c> from <c>build_mic/tests</c>.

*/