Compilation configuration
=========================

The following arguments can be given to the configure script.

Common configuration
--------------------

--enable-debug
..............

Enable debugging messages.

--enable-fast
.............

Do not enforce assertions, saves a lot of time spent to compute them otherwise.

--enable-verbose
................

Augment the verbosity of the debugging messages. This can be disabled at runtime by setting the environment variable STARPU_SILENT to any value.

     % STARPU_SILENT=1 ./vector_scal

--enable-coverage
.................

Enable flags for the gcov coverage tool.

Configuring workers
-------------------

--enable-maxcpus=<number>
.........................

Define the maximum number of CPU cores that StarPU will support, then available as the STARPU_MAXCPUS macro.

--disable-cpu
.............

Disable the use of CPUs of the machine. Only GPUs etc. will be used.

--enable-maxcudadev=<number>
............................

Define the maximum number of CUDA devices that StarPU will support, then available as the STARPU_MAXCUDADEVS macro.

--disable-cuda
..............

Disable the use of CUDA, even if a valid CUDA installation was detected.

--with-cuda-dir=<path>
......................

Specify the directory where CUDA is installed. This directory should notably contain include/cuda.h.

--with-cuda-include-dir=<path>
..............................

Specify the directory where CUDA headers are installed. This directory should notably contain cuda.h. This defaults to /include appended to the value given to --with-cuda-dir.

--with-cuda-lib-dir=<path>
..........................

Specify the directory where the CUDA library is installed. This directory should notably contain the CUDA shared libraries (e.g. libcuda.so). This defaults to /lib appended to the value given to --with-cuda-dir.

--disable-cuda-memcpy-peer
..........................

Explicitely disable peer transfers when using CUDA 4.0

--enable-maxopencldev=<number>
..............................

Define the maximum number of OpenCL devices that StarPU will support, then available as the STARPU_MAXOPENCLDEVS macro.

--disable-opencl
................

Disable the use of OpenCL, even if the SDK is detected.

--with-opencl-dir=<path>
........................

Specify the location of the OpenCL SDK. This directory should notably contain include/CL/cl.h (or include/OpenCL/cl.h on Mac OS).

--with-opencl-include-dir=<path>
................................

Specify the location of OpenCL headers. This directory should notably contain CL/cl.h (or OpenCL/cl.h on Mac OS). This defaults to /include appended to the value given to --with-opencl-dir.

--with-opencl-lib-dir=<path>
............................

Specify the location of the OpenCL library. This directory should notably contain the OpenCL shared libraries (e.g. libOpenCL.so). This defaults to /lib appended to the value given to --with-opencl-dir.

--enable-gordon
...............

Enable the use of the Gordon runtime for Cell SPUs.

--with-gordon-dir=<path>
........................

Specify the location of the Gordon SDK.

--enable-maximplementations=<number>
....................................

Define the number of implementations that can be defined for a single kind of device. It is then available as the STARPU_MAXIMPLEMENTATIONS macro.

Advanced configuration
----------------------

--enable-perf-debug
...................

Enable performance debugging through gprof.

--enable-model-debug
....................

Enable performance model debugging.

--enable-stats
..............

Enable statistics.

--enable-maxbuffers=<nbuffers>
..............................

Define the maximum number of buffers that tasks will be able to take as parameters, then available as the STARPU_NMAXBUFS macro.

--enable-allocation-cache
.........................

Enable the use of a data allocation cache to avoid the cost of it with CUDA. Still experimental.

--enable-opengl-render
......................

Enable the use of OpenGL for the rendering of some examples.

--enable-blas-lib=<name>
........................

Specify the blas library to be used by some of the examples. The library has to be 'atlas' or 'goto'.

--disable-starpufft
...................

Disable the build of libstarpufft, even if fftw or cuFFT is available.

--with-magma=<path>
...................

Specify where magma is installed. This directory should notably contain include/magmablas.h.

--with-fxt=<path>
.................

Specify the location of FxT (for generating traces and rendering them using ViTE). This directory should notably contain include/fxt/fxt.h.

--with-perf-model-dir=<dir>
...........................

Specify where performance models should be stored (instead of defaulting to the current user's home).

--with-mpicc=<path to mpicc>
............................

Specify the location of the mpicc compiler to be used for starpumpi.

--with-goto-dir=<dir>
.....................

Specify the location of GotoBLAS.

--with-atlas-dir=<dir>
......................

Specify the location of ATLAS. This directory should notably contain include/cblas.h.

--with-mkl-cflags=<cflags>
..........................

Specify the compilation flags for the MKL Library.

--with-mkl-ldflags=<ldflags>
............................

Specify the linking flags for the MKL Library. Note that the http://software.intel.com/en-us/articles/intel-mkl-link-line-advisor/ website provides a script to determine the linking flags.

Execution configuration through environment variables
=====================================================

Note: the values given in starpu_conf structure passed when calling starpu_init will override the values of the environment variables.

Configuring workers
-------------------

STARPU_NCPUS -- Number of CPU workers
.....................................

Specify the number of CPU workers (thus not including workers dedicated to control acceleratores). Note that by default, StarPU will not allocate more CPU workers than there are physical CPUs, and that some CPUs are used to control the accelerators.

STARPU_NCUDA -- Number of CUDA workers
......................................

Specify the number of CUDA devices that StarPU can use. If STARPU_NCUDA is lower than the number of physical devices, it is possible to select which CUDA devices should be used by the means of the STARPU_WORKERS_CUDAID environment variable. By default, StarPU will create as many CUDA workers as there are CUDA devices.

STARPU_NOPENCL -- Number of OpenCL workers
..........................................

OpenCL equivalent of the STARPU_NCUDA environment variable.

STARPU_NGORDON -- Number of SPU workers (Cell)
..............................................

Specify the number of SPUs that StarPU can use.

STARPU_WORKERS_CPUID -- Bind workers to specific CPUs
.....................................................

Passing an array of integers (starting from 0) in STARPU_WORKERS_CPUID specifies on which logical CPU the different workers should be bound. For instance, if STARPU_WORKERS_CPUID = "0 1 4 5", the first worker will be bound to logical CPU #0, the second CPU worker will be bound to logical CPU #1 and so on.  Note that the logical ordering of the CPUs is either determined by the OS, or provided by the hwloc library in case it is available.

Note that the first workers correspond to the CUDA workers, then come the OpenCL and the SPU, and finally the CPU workers. For example if we have STARPU_NCUDA=1, STARPU_NOPENCL=1, STARPU_NCPUS=2 and STARPU_WORKERS_CPUID = "0 2 1 3", the CUDA device will be controlled by logical CPU #0, the OpenCL device will be controlled by logical CPU #2, and the logical CPUs #1 and #3 will be used by the CPU workers.

If the number of workers is larger than the array given in STARPU_WORKERS_CPUID, the workers are bound to the logical CPUs in a round-robin fashion: if STARPU_WORKERS_CPUID = "0 1", the first and the third (resp. second and fourth) workers will be put on CPU #0 (resp. CPU #1).

This variable is ignored if the use_explicit_workers_bindid flag of the starpu_conf structure passed to starpu_init is set.

STARPU_WORKERS_CUDAID -- Select specific CUDA devices
.....................................................

Similarly to the STARPU_WORKERS_CPUID environment variable, it is possible to select which CUDA devices should be used by StarPU. On a machine equipped with 4 GPUs, setting STARPU_WORKERS_CUDAID = "1 3" and STARPU_NCUDA=2 specifies that 2 CUDA workers should be created, and that they should use CUDA devices #1 and #3 (the logical ordering of the devices is the one reported by CUDA).

This variable is ignored if the use_explicit_workers_cuda_gpuid flag of the starpu_conf structure passed to starpu_init is set.

STARPU_WORKERS_OPENCLID -- Select specific OpenCL devices
.........................................................

OpenCL equivalent of the STARPU_WORKERS_CUDAID environment variable.

This variable is ignored if the use_explicit_workers_opencl_gpuid flag of the starpu_conf structure passed to starpu_init is set.

Configuring the Scheduling engine
---------------------------------

STARPU_SCHED -- Scheduling policy
.................................

Choose between the different scheduling policies proposed by StarPU: work random, stealing, greedy, with performance models, etc.

Use STARPU_SCHED=help to get the list of available schedulers.

STARPU_CALIBRATE -- Calibrate performance models
................................................

If this variable is set to 1, the performance models are calibrated during the execution. If it is set to 2, the previous values are dropped to restart calibration from scratch. Setting this variable to 0 disable calibration, this is the default behaviour.

Note: this currently only applies to dm, dmda and heft scheduling policies.

STARPU_PREFETCH -- Use data prefetch
....................................

This variable indicates whether data prefetching should be enabled (0 means that it is disabled). If prefetching is enabled, when a task is scheduled to be executed e.g. on a GPU, StarPU will request an asynchronous transfer in advance, so that data is already present on the GPU when the task starts. As a result, computation and data transfers are overlapped. Note that prefetching is enabled by default in StarPU.

STARPU_SCHED_ALPHA -- Computation factor
........................................

To estimate the cost of a task StarPU takes into account the estimated computation time (obtained thanks to performance models). The alpha factor is the coefficient to be applied to it before adding it to the communication part.

STARPU_SCHED_BETA -- Communication factor
.........................................

To estimate the cost of a task StarPU takes into account the estimated data transfer time (obtained thanks to performance models). The beta factor is the coefficient to be applied to it before adding it to the computation part.

Miscellaneous and debug
-----------------------

STARPU_SILENT -- Disable verbose mode
.....................................

This variable allows to disable verbose mode at runtime when StarPU has been configured with the option --enable-verbose.

STARPU_LOGFILENAME -- Select debug file name
............................................

This variable specifies in which file the debugging output should be saved to.

STARPU_FXT_PREFIX -- FxT trace location
.......................................

This variable specifies in which directory to save the trace generated if FxT is enabled. It needs to have a trailing '/' character.

STARPU_LIMIT_GPU_MEM -- Restrict memory size on the GPUs
........................................................

This variable specifies the maximum number of megabytes that should be available to the application on each GPUs. In case this value is smaller than the size of the memory of a GPU, StarPU pre-allocates a buffer to waste memory on the device. This variable is intended to be used for experimental purposes as it emulates devices that have a limited amount of memory.

STARPU_GENERATE_TRACE -- Generate a Paje trace when StarPU is shut down
.......................................................................

When set to 1, this variable indicates that StarPU should automatically generate a Paje trace when starpu_shutdown is called.