101_building.doxy 16 KB

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960616263646566676869707172737475767778798081828384858687888990919293949596979899100101102103104105106107108109110111112113114115116117118119120121122123124125126127128129130131132133134135136137138139140141142143144145146147148149150151152153154155156157158159160161162163164165166167168169170171172173174175176177178179180181182183184185186187188189190191192193194195196197198199200201202203204205206207208209210211212213214215216217218219220221222223224225226227228229230231232233234235236237238239240241242243244245246247248249250251252253254255256257258259260261262263264265266267268269270271272273274275276277278279280281282283284285286287288289290291292293294295296297298299300301302303304305306307308309310311312313314315316317318319320321322323324325326327328329330331332333334335336337338339340341342343344345346347348349350351352353354355356357358359360361362363364365366367368369370371372373374375376377378379380381382383384385386387388389390391392393394395396397398399400401402403404405406407408409410411412413414415416417418419420421422423424425426427428429430431432433434435436437438439440441442443444445446447448449450451452453454455456457458459460461462463464465466467468469470471472473474475
  1. /* StarPU --- Runtime system for heterogeneous multicore architectures.
  2. *
  3. * Copyright (C) 2010-2018 CNRS
  4. * Copyright (C) 2009-2011,2013-2016 Université de Bordeaux
  5. * Copyright (C) 2011-2012,2018 Inria
  6. *
  7. * StarPU is free software; you can redistribute it and/or modify
  8. * it under the terms of the GNU Lesser General Public License as published by
  9. * the Free Software Foundation; either version 2.1 of the License, or (at
  10. * your option) any later version.
  11. *
  12. * StarPU is distributed in the hope that it will be useful, but
  13. * WITHOUT ANY WARRANTY; without even the implied warranty of
  14. * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.
  15. *
  16. * See the GNU Lesser General Public License in COPYING.LGPL for more details.
  17. */
  18. /*! \page BuildingAndInstallingStarPU Building and Installing StarPU
  19. \section InstallingABinaryPackage Installing a Binary Package
  20. One of the StarPU developers being a Debian Developer, the packages
  21. are well integrated and very uptodate. To see which packages are
  22. available, simply type:
  23. \verbatim
  24. $ apt-cache search starpu
  25. \endverbatim
  26. To install what you need, type for example:
  27. \verbatim
  28. $ sudo apt-get install libstarpu-1.3 libstarpu-dev
  29. \endverbatim
  30. \section InstallingFromSource Installing from Source
  31. StarPU can be built and installed by the standard means of the GNU
  32. autotools. The following chapter is intended to briefly remind how these tools
  33. can be used to install StarPU.
  34. \subsection OptionalDependencies Optional Dependencies
  35. The <c>hwloc</c> (http://www.open-mpi.org/software/hwloc) topology
  36. discovery library is not mandatory to use StarPU but strongly
  37. recommended. It allows for topology aware scheduling, which improves
  38. performance. <c>libhwloc</c> is available in major free operating system
  39. distributions, and for most operating systems.
  40. If <c>libhwloc</c> is not available on your system, the option
  41. \ref without-hwloc "--without-hwloc" should be explicitely given when calling the
  42. <c>configure</c> script. If <c>libhwloc</c> is installed in a standard
  43. location, no option is required, it will be detected automatically,
  44. otherwise \ref with-hwloc "--with-hwloc=<directory>" should be used to specify its
  45. location.
  46. \subsection GettingSources Getting Sources
  47. StarPU's sources can be obtained from the download page of
  48. the StarPU website (http://starpu.gforge.inria.fr/files/).
  49. All releases and the development tree of StarPU are freely available
  50. on Inria's gforge under the LGPL license. Some releases are available
  51. under the BSD license.
  52. The latest release can be downloaded from the Inria's gforge (http://gforge.inria.fr/frs/?group_id=1570) or
  53. directly from the StarPU download page (http://starpu.gforge.inria.fr/files/).
  54. The latest nightly snapshot can be downloaded from the StarPU gforge website (http://starpu.gforge.inria.fr/testing/).
  55. \verbatim
  56. $ wget http://starpu.gforge.inria.fr/testing/starpu-nightly-latest.tar.gz
  57. \endverbatim
  58. And finally, current development version is also accessible via git.
  59. It should be used only if you need the very latest changes (i.e. less
  60. than a day!).
  61. \verbatim
  62. $ git clone https://scm.gforge.inria.fr/anonscm/git/starpu/starpu.git
  63. \endverbatim
  64. \subsection ConfiguringStarPU Configuring StarPU
  65. Running <c>autogen.sh</c> is not necessary when using the tarball
  66. releases of StarPU. If you are using the source code from the git
  67. repository, you first need to generate the configure scripts and the
  68. Makefiles. This requires the availability of <c>autoconf</c> and
  69. <c>automake</c> >= 2.60.
  70. \verbatim
  71. $ ./autogen.sh
  72. \endverbatim
  73. You then need to configure StarPU. Details about options that are
  74. useful to give to <c>./configure</c> are given in \ref CompilationConfiguration.
  75. \verbatim
  76. $ ./configure
  77. \endverbatim
  78. If <c>configure</c> does not detect some software or produces errors, please
  79. make sure to post the contents of the file <c>config.log</c> when
  80. reporting the issue.
  81. By default, the files produced during the compilation are placed in
  82. the source directory. As the compilation generates a lot of files, it
  83. is advised to put them all in a separate directory. It is then
  84. easier to cleanup, and this allows to compile several configurations
  85. out of the same source tree. For that, simply enter the directory
  86. where you want the compilation to produce its files, and invoke the
  87. <c>configure</c> script located in the StarPU source directory.
  88. \verbatim
  89. $ mkdir build
  90. $ cd build
  91. $ ../configure
  92. \endverbatim
  93. By default, StarPU will be installed in <c>/usr/local/bin</c>,
  94. <c>/usr/local/lib</c>, etc. You can specify an installation prefix
  95. other than <c>/usr/local</c> using the option <c>--prefix</c>, for
  96. instance:
  97. \verbatim
  98. $ ../configure --prefix=$HOME/starpu
  99. \endverbatim
  100. \subsection BuildingStarPU Building StarPU
  101. \verbatim
  102. $ make
  103. \endverbatim
  104. Once everything is built, you may want to test the result. An
  105. extensive set of regression tests is provided with StarPU. Running the
  106. tests is done by calling <c>make check</c>. These tests are run every night
  107. and the result from the main profile is publicly available (http://starpu.gforge.inria.fr/testing/).
  108. \verbatim
  109. $ make check
  110. \endverbatim
  111. \subsection InstallingStarPU Installing StarPU
  112. In order to install StarPU at the location that was specified during
  113. configuration:
  114. \verbatim
  115. $ make install
  116. \endverbatim
  117. Libtool interface versioning information are included in
  118. libraries names (<c>libstarpu-1.3.so</c>, <c>libstarpumpi-1.3.so</c> and
  119. <c>libstarpufft-1.3.so</c>).
  120. \section SettingUpYourOwnCode Setting up Your Own Code
  121. \subsection SettingFlagsForCompilingLinkingAndRunningApplications Setting Flags for Compiling, Linking and Running Applications
  122. StarPU provides a <c>pkg-config</c> executable to obtain relevant compiler
  123. and linker flags. As compiling and linking an application against
  124. StarPU may require to use specific flags or libraries (for instance
  125. <c>CUDA</c> or <c>libspe2</c>).
  126. If StarPU was not installed at some standard location, the path of StarPU's
  127. library must be specified in the environment variable <c>PKG_CONFIG_PATH</c> so
  128. that <c>pkg-config</c> can find it. For example if StarPU was installed in
  129. <c>$STARPU_PATH</c>:
  130. \verbatim
  131. $ PKG_CONFIG_PATH=$PKG_CONFIG_PATH:$STARPU_PATH/lib/pkgconfig
  132. \endverbatim
  133. The flags required to compile or link against StarPU are then
  134. accessible with the following commands:
  135. \verbatim
  136. $ pkg-config --cflags starpu-1.3 # options for the compiler
  137. $ pkg-config --libs starpu-1.3 # options for the linker
  138. \endverbatim
  139. Note that it is still possible to use the API provided in the version
  140. 1.0 of StarPU by calling <c>pkg-config</c> with the <c>starpu-1.0</c> package.
  141. Similar packages are provided for <c>starpumpi-1.0</c> and <c>starpufft-1.0</c>.
  142. It is also possible to use the API provided in the version
  143. 0.9 of StarPU by calling <c>pkg-config</c> with the <c>libstarpu</c> package.
  144. Similar packages are provided for <c>libstarpumpi</c> and <c>libstarpufft</c>.
  145. Make sure that <c>pkg-config --libs starpu-1.3</c> actually produces some output
  146. before going further: <c>PKG_CONFIG_PATH</c> has to point to the place where
  147. <c>starpu-1.3.pc</c> was installed during <c>make install</c>.
  148. Also pass the option <c>--static</c> if the application is to be
  149. linked statically.
  150. It is also necessary to set the environment variable <c>LD_LIBRARY_PATH</c> to
  151. locate dynamic libraries at runtime.
  152. \verbatim
  153. $ LD_LIBRARY_PATH=$STARPU_PATH/lib:$LD_LIBRARY_PATH
  154. \endverbatim
  155. When using a Makefile, the following lines can be added to set the
  156. options for the compiler and the linker:
  157. \verbatim
  158. CFLAGS += $$(pkg-config --cflags starpu-1.3)
  159. LDFLAGS += $$(pkg-config --libs starpu-1.3)
  160. \endverbatim
  161. \subsection IntegratingStarPUInABuildSystem Integrating StarPU in a Build System
  162. \subsubsection StarPUInCMake Integrating StarPU in a CMake Build System
  163. This section shows a minimal example integrating StarPU in an existing application's CMake build system.
  164. Let's assume we want to build an executable from the following source code using CMake:
  165. \code{.c}
  166. #include <starpu.h>
  167. int main(void)
  168. {
  169. int ret;
  170. ret = starpu_init(NULL);
  171. if (ret != 0)
  172. {
  173. return 1;
  174. }
  175. starpu_shutdown();
  176. return 0;
  177. }
  178. \endcode
  179. The \c CMakeLists.txt file below uses the Pkg-Config support from CMake to
  180. autodetect the StarPU installation and library dependences (such as
  181. <c>libhwloc</c>) provided that the <c>PKG_CONFIG_PATH</c> variable is set, and
  182. is sufficient to build a statically-linked executable. This example has been
  183. successfully tested with CMake 3.2, though it may work with earlier CMake 3.x
  184. versions.
  185. \code{CMakeLists.txt}
  186. cmake_minimum_required (VERSION 3.2)
  187. project (hello_starpu)
  188. find_package(PkgConfig)
  189. pkg_check_modules(STARPU REQUIRED starpu-1.3)
  190. if (STARPU_FOUND)
  191. include_directories (${STARPU_INCLUDE_DIRS})
  192. link_directories (${STARPU_STATIC_LIBRARY_DIRS})
  193. link_libraries (${STARPU_STATIC_LIBRARIES})
  194. else (STARPU_FOUND)
  195. message(FATAL_ERROR "StarPU not found")
  196. endif()
  197. add_executable(hello_starpu hello_starpu.c)
  198. \endcode
  199. The following \c CMakeLists.txt implements an alternative, more complex
  200. strategy, still relying on PkgConfig, but also taking into account additional
  201. flags. While more complete, this approach makes CMake's build types (Debug,
  202. Release, ...) unavailable because of the direct affectation to variable
  203. <c>CMAKE_C_FLAGS</c>. If both the full flags support and the build types
  204. support are needed, the \c CMakeLists.txt below may be altered to work with
  205. <c>CMAKE_C_FLAGS_RELEASE</c>, <c>CMAKE_C_FLAGS_DEBUG</c>, and others as needed.
  206. This example has been successfully tested with CMake 3.2, though it may work
  207. with earlier CMake 3.x versions.
  208. \code{CMakeLists.txt}
  209. cmake_minimum_required (VERSION 3.2)
  210. project (hello_starpu)
  211. find_package(PkgConfig)
  212. pkg_check_modules(STARPU REQUIRED starpu-1.3)
  213. # This section must appear before 'add_executable'
  214. if (STARPU_FOUND)
  215. # CFLAGS other than -I
  216. foreach(CFLAG ${STARPU_CFLAGS_OTHER})
  217. set (CMAKE_C_FLAGS "${CMAKE_C_FLAGS} ${CFLAG}")
  218. endforeach()
  219. # Static LDFLAGS other than -L
  220. foreach(LDFLAG ${STARPU_STATIC_LDFLAGS_OTHER})
  221. set (CMAKE_EXE_LINKER_FLAGS "${CMAKE_EXE_LINKER_FLAGS} ${LDFLAG}")
  222. endforeach()
  223. # -L directories
  224. link_directories(${STARPU_STATIC_LIBRARY_DIRS})
  225. else (STARPU_FOUND)
  226. message(FATAL_ERROR "StarPU not found")
  227. endif()
  228. add_executable(hello_starpu hello_starpu.c)
  229. # This section must appear after 'add_executable'
  230. if (STARPU_FOUND)
  231. # -I directories
  232. target_include_directories(hello_starpu PRIVATE ${STARPU_INCLUDE_DIRS})
  233. # Static -l libs
  234. target_link_libraries(hello_starpu PRIVATE ${STARPU_STATIC_LIBRARIES})
  235. endif()
  236. \endcode
  237. \subsection RunningABasicStarPUApplication Running a Basic StarPU Application
  238. Basic examples using StarPU are built in the directory
  239. <c>examples/basic_examples/</c> (and installed in
  240. <c>$STARPU_PATH/lib/starpu/examples/</c>). You can for example run the example
  241. <c>vector_scal</c>.
  242. \verbatim
  243. $ ./examples/basic_examples/vector_scal
  244. BEFORE: First element was 1.000000
  245. AFTER: First element is 3.140000
  246. \endverbatim
  247. When StarPU is used for the first time, the directory
  248. <c>$STARPU_HOME/.starpu/</c> is created, performance models will be stored in
  249. that directory (\ref STARPU_HOME).
  250. Please note that buses are benchmarked when StarPU is launched for the
  251. first time. This may take a few minutes, or less if <c>libhwloc</c> is
  252. installed. This step is done only once per user and per machine.
  253. \subsection RunningABasicStarPUApplicationOnMicrosoft Running a Basic StarPU Application on Microsoft Visual C
  254. Batch files are provided to run StarPU applications under Microsoft
  255. Visual C. They are installed in <c>$STARPU_PATH/bin/msvc</c>.
  256. To execute a StarPU application, you first need to set the environment
  257. variable \ref STARPU_PATH.
  258. \verbatim
  259. c:\....> cd c:\cygwin\home\ci\starpu\
  260. c:\....> set STARPU_PATH=c:\cygwin\home\ci\starpu\
  261. c:\....> cd bin\msvc
  262. c:\....> starpu_open.bat starpu_simple.c
  263. \endverbatim
  264. The batch script will run Microsoft Visual C with a basic project file
  265. to run the given application.
  266. The batch script <c>starpu_clean.bat</c> can be used to delete all
  267. compilation generated files.
  268. The batch script <c>starpu_exec.bat</c> can be used to compile and execute a
  269. StarPU application from the command prompt.
  270. \verbatim
  271. c:\....> cd c:\cygwin\home\ci\starpu\
  272. c:\....> set STARPU_PATH=c:\cygwin\home\ci\starpu\
  273. c:\....> cd bin\msvc
  274. c:\....> starpu_exec.bat ..\..\..\..\examples\basic_examples\hello_world.c
  275. \endverbatim
  276. \verbatim
  277. MSVC StarPU Execution
  278. ...
  279. /out:hello_world.exe
  280. ...
  281. Hello world (params = {1, 2.00000})
  282. Callback function got argument 0000042
  283. c:\....>
  284. \endverbatim
  285. \subsection KernelThreadsStartedByStarPU Kernel Threads Started by StarPU
  286. StarPU automatically binds one thread per CPU core. It does not use
  287. SMT/hyperthreading because kernels are usually already optimized for using a
  288. full core, and using hyperthreading would make kernel calibration rather random.
  289. Since driving GPUs is a CPU-consuming task, StarPU dedicates one core
  290. per GPU.
  291. While StarPU tasks are executing, the application is not supposed to do
  292. computations in the threads it starts itself, tasks should be used instead.
  293. TODO: add a StarPU function to bind an application thread (e.g. the main thread)
  294. to a dedicated core (and thus disable the corresponding StarPU CPU worker).
  295. \subsection EnablingOpenCL Enabling OpenCL
  296. When both CUDA and OpenCL drivers are enabled, StarPU will launch an
  297. OpenCL worker for NVIDIA GPUs only if CUDA is not already running on them.
  298. This design choice was necessary as OpenCL and CUDA can not run at the
  299. same time on the same NVIDIA GPU, as there is currently no interoperability
  300. between them.
  301. To enable OpenCL, you need either to disable CUDA when configuring StarPU:
  302. \verbatim
  303. $ ./configure --disable-cuda
  304. \endverbatim
  305. or when running applications:
  306. \verbatim
  307. $ STARPU_NCUDA=0 ./application
  308. \endverbatim
  309. OpenCL will automatically be started on any device not yet used by
  310. CUDA. So on a machine running 4 GPUS, it is therefore possible to
  311. enable CUDA on 2 devices, and OpenCL on the 2 other devices by doing
  312. so:
  313. \verbatim
  314. $ STARPU_NCUDA=2 ./application
  315. \endverbatim
  316. \section BenchmarkingStarPU Benchmarking StarPU
  317. Some interesting benchmarks are installed among examples in
  318. <c>$STARPU_PATH/lib/starpu/examples/</c>. Make sure to try various
  319. schedulers, for instance <c>STARPU_SCHED=dmda</c>.
  320. \subsection TaskSizeOverhead Task Size Overhead
  321. This benchmark gives a glimpse into how long a task should be (in µs) for StarPU overhead
  322. to be low enough to keep efficiency. Running
  323. <c>tasks_size_overhead.sh</c> generates a plot
  324. of the speedup of tasks of various sizes, depending on the number of CPUs being
  325. used.
  326. \image html tasks_size_overhead.png
  327. \image latex tasks_size_overhead.eps "" width=\textwidth
  328. \subsection DataTransferLatency Data Transfer Latency
  329. <c>local_pingpong</c> performs a ping-pong between the first two CUDA nodes, and
  330. prints the measured latency.
  331. \subsection MatrixMatrixMultiplication Matrix-Matrix Multiplication
  332. <c>sgemm</c> and <c>dgemm</c> perform a blocked matrix-matrix
  333. multiplication using BLAS and cuBLAS. They output the obtained GFlops.
  334. \subsection CholeskyFactorization Cholesky Factorization
  335. <c>cholesky_*</c> perform a Cholesky factorization (single precision). They use different dependency primitives.
  336. \subsection LUFactorization LU Factorization
  337. <c>lu_*</c> perform an LU factorization. They use different dependency primitives.
  338. \subsection SimulatedBenchmarks Simulated benchmarks
  339. It can also be convenient to try simulated benchmarks, if you want to give a try
  340. at CPU-GPU scheduling without actually having a GPU at hand. This can be done by
  341. using the simgrid version of StarPU: first install the simgrid simulator from
  342. http://simgrid.gforge.inria.fr/ (we tested with simgrid 3.11, 3.12 and 3.13, other versions
  343. may have compatibility issues), then configure StarPU with \ref enable-simgrid
  344. "--enable-simgrid" and rebuild and install it, and then you can simulate the performance for a
  345. few virtualized systems shipped along StarPU: attila, mirage, idgraf, and sirocco.
  346. For instance:
  347. \verbatim
  348. $ export STARPU_PERF_MODEL_DIR=$STARPU_PATH/share/starpu/perfmodels/sampling
  349. $ export STARPU_HOSTNAME=attila
  350. $ $STARPU_PATH/lib/starpu/examples/cholesky_implicit -size $((960*20)) -nblocks 20
  351. \endverbatim
  352. Will show the performance of the cholesky factorization with the attila
  353. system. It will be interesting to try with different matrix sizes and
  354. schedulers.
  355. Performance models are available for cholesky_*, lu_*, *gemm, with block sizes
  356. 320, 640, or 960 (plus 1440 for sirocco), and for stencil with block size 128x128x128, 192x192x192, and
  357. 256x256x256.
  358. */