/* StarPU --- Runtime system for heterogeneous multicore architectures.
 *
 * Copyright (C) 2009-2020  Université de Bordeaux, CNRS (LaBRI UMR 5800), Inria
 *
 * StarPU is free software; you can redistribute it and/or modify
 * it under the terms of the GNU Lesser General Public License as published by
 * the Free Software Foundation; either version 2.1 of the License, or (at
 * your option) any later version.
 *
 * StarPU is distributed in the hope that it will be useful, but
 * WITHOUT ANY WARRANTY; without even the implied warranty of
 * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.
 *
 * See the GNU Lesser General Public License in COPYING.LGPL for more details.
 */

/*! \page MICSupport MIC Xeon Phi Support

\section MICCompilation Compilation

MIC Xeon Phi support actually needs two compilations of StarPU, one for the host and one for
the device. The <c>PATH</c> environment variable has to include the path to the
cross-compilation toolchain, for instance <c>/usr/linux-k1om-4.7/bin</c> .
The <c>SINK_PKG_CONFIG_PATH</c> environment variable should include the path to the
cross-compiled <c>hwloc.pc</c>.
The script <c>mic-configure</c> can then be used to achieve the two compilations: it basically
calls <c>configure</c> as appropriate from two new directories: <c>build_mic</c> and
<c>build_host</c>. <c>make</c> and <c>make install</c> can then be used as usual and will
recurse into both directories. If different \c configure options are needed
for the host and for the mic, one can use <c>--with-host-param=--with-fxt</c>
for instance to specify the <c>--with-fxt</c> option for the host only, or
<c>--with-mic-param=--with-fxt</c> for the mic only.

One can also run StarPU just natively on the Xeon Phi, i.e. it will only run
directly on the Phi without any exchange with the host CPU. The binaries in
<c>build_mic</c> can be run that way.

For MPI support, you will probably have to specify different MPI compiler path
or option for the host and the device builds, for instance:

\verbatim
./mic-configure --with-mic-param=--with-mpicc="/.../mpiicc -mmic" \
    --with-host-param=--with-mpicc=/.../mpiicc
\endverbatim

In case you have troubles with the \c coi or \c scif libraries (the Intel paths are
really not standard, it seems...), you can still make a build in native mode
only, by using <c>mic-configure --enable-native-mic</c> (and notably without
<c>--enable-mic</c> since in that case we don't need \c mic offloading support).

\section PortingApplicationsToMIC Porting Applications To MIC Xeon Phi

The simplest way to port an application to MIC Xeon Phi is to set the field
starpu_codelet::cpu_funcs_name, to provide StarPU with the function
name of the CPU implementation, so for instance:

\verbatim
struct starpu_codelet cl =
{
    .cpu_funcs = {myfunc},
    .cpu_funcs_name = {"myfunc"},
    .nbuffers = 1,
}
\endverbatim

StarPU will thus simply use the
existing CPU implementation (cross-rebuilt in the MIC Xeon Phi case). The
functions have to be globally-visible (i.e. not <c>static</c>) for
StarPU to be able to look them up, and \c -rdynamic must be passed to \c gcc (or
\c -export-dynamic to \c ld) so that symbols of the main program are visible.

If you have used the starpu_codelet::where field, you additionally need to add in it
::STARPU_MIC for the Xeon Phi.

For non-native MIC Xeon Phi execution, the 'main' function of the application, on the sink, should call starpu_init() immediately upon start-up; the starpu_init() function never returns. On the host, the 'main' function may freely perform application related initialization calls as usual, before calling starpu_init().

For MIC Xeon Phi, the application may programmatically detect whether executing on the sink or on the host, by checking whether the \ref STARPU_SINK environment variable is defined (on the sink) or not (on the host).

\section LaunchingPrograms Launching Programs

MIC programs are started from the host. StarPU automatically
starts the same program on MIC devices. It however needs to get
the MIC-cross-built binary. It will look for the file given by the
environment variable \ref STARPU_MIC_SINK_PROGRAM_NAME or in the
directory given by the environment variable \ref STARPU_MIC_SINK_PROGRAM_PATH,
or in the field
starpu_conf::mic_sink_program_path. It will also look in the current
directory for the same binary name plus the suffix <c>-mic</c> or
<c>_mic</c>.

The testsuite can be started by simply running <c>make check</c> from the
top directory. It will recurse into both <c>build_host</c> to run tests with only
the host, and into <c>build_mic</c> to run tests with both the host and the MIC
devices. Single tests with the host and the MIC can be run by starting
<c>./loader-cross.sh ./the_test</c> from <c>build_mic/tests</c>.

*/