starpu-max/doc/chapters/tips-tricks.texi

@c -*-texinfo-*-

@c This file is part of the StarPU Handbook.
@c Copyright (C) 2009--2011  Universit@'e de Bordeaux 1
@c Copyright (C) 2010, 2011, 2012, 2013  Centre National de la Recherche Scientifique
@c Copyright (C) 2011 Institut National de Recherche en Informatique et Automatique
@c See the file starpu.texi for copying conditions.

@menu
* Per-worker library initialization::  How to initialize a computation library once for each worker?
* Limit memory::
* Thread Binding on NetBSD::
@end menu

@node Per-worker library initialization
@section How to initialize a computation library once for each worker?

Some libraries need to be initialized once for each concurrent instance that
may run on the machine. For instance, a C++ computation class which is not
thread-safe by itself, but for which several instanciated objects of that class
can be used concurrently. This can be used in StarPU by initializing one such
object per worker. For instance, the libstarpufft example does the following to
be able to use FFTW.

Some global array stores the instanciated objects:

@cartouche
@smallexample
fftw_plan plan_cpu[STARPU_NMAXWORKERS];
@end smallexample
@end cartouche

At initialisation time of libstarpu, the objects are initialized:

@cartouche
@smallexample
int workerid;
for (workerid = 0; workerid < starpu_worker_get_count(); workerid++) @{
    switch (starpu_worker_get_type(workerid)) @{
        case STARPU_CPU_WORKER:
            plan_cpu[workerid] = fftw_plan(...);
            break;
    @}
@}
@end smallexample
@end cartouche

And in the codelet body, they are used:

@cartouche
@smallexample
static void fft(void *descr[], void *_args)
@{
    int workerid = starpu_worker_get_id();
    fftw_plan plan = plan_cpu[workerid];
    ...

    fftw_execute(plan, ...);
@}
@end smallexample
@end cartouche

Another way to go which may be needed is to execute some code from the workers
themselves thanks to @code{starpu_execute_on_each_worker}. This may be required
by CUDA to behave properly due to threading issues. For instance, StarPU's
@code{starpu_cublas_init} looks like the following to call
@code{cublasInit} from the workers themselves:

@cartouche
@smallexample
static void init_cublas_func(void *args STARPU_ATTRIBUTE_UNUSED)
@{
    cublasStatus cublasst = cublasInit();
    cublasSetKernelStream(starpu_cuda_get_local_stream());
@}
void starpu_cublas_init(void)
@{
    starpu_execute_on_each_worker(init_cublas_func, NULL, STARPU_CUDA);
@}
@end smallexample
@end cartouche

@node Limit memory
@section How to limit memory per node

TODO

Talk about
@code{STARPU_LIMIT_CUDA_devid_MEM}, @code{STARPU_LIMIT_CUDA_MEM},
@code{STARPU_LIMIT_OPENCL_devid_MEM}, @code{STARPU_LIMIT_OPENCL_MEM}
and @code{STARPU_LIMIT_CPU_MEM}

@code{starpu_memory_get_available}

@node Thread Binding on NetBSD
@section Thread Binding on NetBSD

When using StarPU on a NetBSD machine, if the topology
discovery library @code{hwloc} is used, thread binding will fail. To
prevent the problem, you should at least use the version 1.7 of
@code{hwloc}, and also issue the following call:

@example
$ sysctl -w security.models.extensions.user_set_cpu_affinity=1
@end example

Or add the following line in the file @code{/etc/sysctl.conf}

@example
security.models.extensions.user_set_cpu_affinity=1
@end example