501_environment_variables.doxy 42 KB

1234567891011121314151617181920212223242526272829303132333435363738394041424344454647484950515253545556575859606162636465666768697071727374757677787980818283848586878889909192939495969798991001011021031041051061071081091101111121131141151161171181191201211221231241251261271281291301311321331341351361371381391401411421431441451461471481491501511521531541551561571581591601611621631641651661671681691701711721731741751761771781791801811821831841851861871881891901911921931941951961971981992002012022032042052062072082092102112122132142152162172182192202212222232242252262272282292302312322332342352362372382392402412422432442452462472482492502512522532542552562572582592602612622632642652662672682692702712722732742752762772782792802812822832842852862872882892902912922932942952962972982993003013023033043053063073083093103113123133143153163173183193203213223233243253263273283293303313323333343353363373383393403413423433443453463473483493503513523533543553563573583593603613623633643653663673683693703713723733743753763773783793803813823833843853863873883893903913923933943953963973983994004014024034044054064074084094104114124134144154164174184194204214224234244254264274284294304314324334344354364374384394404414424434444454464474484494504514524534544554564574584594604614624634644654664674684694704714724734744754764774784794804814824834844854864874884894904914924934944954964974984995005015025035045055065075085095105115125135145155165175185195205215225235245255265275285295305315325335345355365375385395405415425435445455465475485495505515525535545555565575585595605615625635645655665675685695705715725735745755765775785795805815825835845855865875885895905915925935945955965975985996006016026036046056066076086096106116126136146156166176186196206216226236246256266276286296306316326336346356366376386396406416426436446456466476486496506516526536546556566576586596606616626636646656666676686696706716726736746756766776786796806816826836846856866876886896906916926936946956966976986997007017027037047057067077087097107117127137147157167177187197207217227237247257267277287297307317327337347357367377387397407417427437447457467477487497507517527537547557567577587597607617627637647657667677687697707717727737747757767777787797807817827837847857867877887897907917927937947957967977987998008018028038048058068078088098108118128138148158168178188198208218228238248258268278288298308318328338348358368378388398408418428438448458468478488498508518528538548558568578588598608618628638648658668678688698708718728738748758768778788798808818828838848858868878888898908918928938948958968978988999009019029039049059069079089099109119129139149159169179189199209219229239249259269279289299309319329339349359369379389399409419429439449459469479489499509519529539549559569579589599609619629639649659669679689699709719729739749759769779789799809819829839849859869879889899909919929939949959969979989991000100110021003100410051006100710081009101010111012101310141015101610171018101910201021102210231024102510261027102810291030103110321033103410351036103710381039104010411042104310441045104610471048104910501051105210531054105510561057105810591060106110621063106410651066106710681069107010711072107310741075107610771078107910801081108210831084108510861087108810891090109110921093109410951096109710981099110011011102110311041105110611071108110911101111111211131114111511161117111811191120112111221123112411251126112711281129113011311132113311341135113611371138113911401141114211431144114511461147114811491150115111521153115411551156115711581159116011611162116311641165116611671168116911701171117211731174117511761177117811791180118111821183118411851186118711881189119011911192119311941195119611971198119912001201120212031204120512061207120812091210121112121213121412151216121712181219122012211222122312241225122612271228122912301231123212331234123512361237123812391240124112421243
  1. /* StarPU --- Runtime system for heterogeneous multicore architectures.
  2. *
  3. * Copyright (C) 2009-2011 Universit@'e de Bordeaux
  4. * Copyright (C) 2010, 2011, 2012, 2013, 2014, 2015, 2016, 2017 CNRS
  5. * Copyright (C) 2011, 2012, 2016, 2017 Inria
  6. * Copyright (C) 2016 Uppsala University
  7. *
  8. * StarPU is free software; you can redistribute it and/or modify
  9. * it under the terms of the GNU Lesser General Public License as published by
  10. * the Free Software Foundation; either version 2.1 of the License, or (at
  11. * your option) any later version.
  12. *
  13. * StarPU is distributed in the hope that it will be useful, but
  14. * WITHOUT ANY WARRANTY; without even the implied warranty of
  15. * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.
  16. *
  17. * See the GNU Lesser General Public License in COPYING.LGPL for more details.
  18. */
  19. /*! \page ExecutionConfigurationThroughEnvironmentVariables Execution Configuration Through Environment Variables
  20. The behavior of the StarPU library and tools may be tuned thanks to
  21. the following environment variables.
  22. \section ConfiguringWorkers Configuring Workers
  23. <dl>
  24. <dt>STARPU_NCPU</dt>
  25. <dd>
  26. \anchor STARPU_NCPU
  27. \addindex __env__STARPU_NCPU
  28. Specify the number of CPU workers (thus not including workers
  29. dedicated to control accelerators). Note that by default, StarPU will
  30. not allocate more CPU workers than there are physical CPUs, and that
  31. some CPUs are used to control the accelerators.
  32. </dd>
  33. <dt>STARPU_NCPUS</dt>
  34. <dd>
  35. \anchor STARPU_NCPUS
  36. \addindex __env__STARPU_NCPUS
  37. This variable is deprecated. You should use \ref STARPU_NCPU.
  38. </dd>
  39. <dt>STARPU_NCUDA</dt>
  40. <dd>
  41. \anchor STARPU_NCUDA
  42. \addindex __env__STARPU_NCUDA
  43. Specify the number of CUDA devices that StarPU can use. If
  44. \ref STARPU_NCUDA is lower than the number of physical devices, it is
  45. possible to select which CUDA devices should be used by the means of the
  46. environment variable \ref STARPU_WORKERS_CUDAID. By default, StarPU will
  47. create as many CUDA workers as there are CUDA devices.
  48. </dd>
  49. <dt>STARPU_NWORKER_PER_CUDA</dt>
  50. <dd>
  51. \anchor STARPU_NWORKER_PER_CUDA
  52. \addindex __env__STARPU_NWORKER_PER_CUDA
  53. Specify the number of workers per CUDA device, and thus the number of kernels
  54. which will be concurrently running on the devices. The default value is 1.
  55. </dd>
  56. <dt>STARPU_CUDA_THREAD_PER_WORKER</dt>
  57. <dd>
  58. \anchor STARPU_CUDA_THREAD_PER_WORKER
  59. \addindex __env__STARPU_CUDA_THREAD_PER_WORKER
  60. Specify if the cuda driver should provide a thread per stream or a single thread
  61. dealing with all the streams. 0 if one thread per stream, 1 otherwise. The default
  62. value is 0. Setting it to 1 is contradictory with setting STARPU_CUDA_THREAD_PER_DEV to 1.
  63. </dd>
  64. <dt>STARPU_CUDA_THREAD_PER_DEV</dt>
  65. <dd>
  66. \anchor STARPU_CUDA_THREAD_PER_DEV
  67. \addindex __env__STARPU_CUDA_THREAD_PER_DEV
  68. Specify if the cuda driver should provide a thread per device or a single thread
  69. dealing with all the devices. 0 if one thread per device, 1 otherwise. The default
  70. value is 1, unless STARPU_CUDA_THREAD_PER_WORKER is set to 1. Setting it to 1 is
  71. contradictory with setting STARPU_CUDA_THREAD_PER_WORKER to 1.
  72. </dd>
  73. <dt>STARPU_CUDA_PIPELINE</dt>
  74. <dd>
  75. \anchor STARPU_CUDA_PIPELINE
  76. \addindex __env__STARPU_CUDA_PIPELINE
  77. Specify how many asynchronous tasks are submitted in advance on CUDA
  78. devices. This for instance permits to overlap task management with the execution
  79. of previous tasks, but it also allows concurrent execution on Fermi cards, which
  80. otherwise bring spurious synchronizations. The default is 2. Setting the value to 0 forces a synchronous
  81. execution of all tasks.
  82. </dd>
  83. <dt>STARPU_NOPENCL</dt>
  84. <dd>
  85. \anchor STARPU_NOPENCL
  86. \addindex __env__STARPU_NOPENCL
  87. OpenCL equivalent of the environment variable \ref STARPU_NCUDA.
  88. </dd>
  89. <dt>STARPU_OPENCL_PIPELINE</dt>
  90. <dd>
  91. \anchor STARPU_OPENCL_PIPELINE
  92. \addindex __env__STARPU_OPENCL_PIPELINE
  93. Specify how many asynchronous tasks are submitted in advance on OpenCL
  94. devices. This for instance permits to overlap task management with the execution
  95. of previous tasks, but it also allows concurrent execution on Fermi cards, which
  96. otherwise bring spurious synchronizations. The default is 2. Setting the value to 0 forces a synchronous
  97. execution of all tasks.
  98. </dd>
  99. <dt>STARPU_OPENCL_ON_CPUS</dt>
  100. <dd>
  101. \anchor STARPU_OPENCL_ON_CPUS
  102. \addindex __env__STARPU_OPENCL_ON_CPUS
  103. By default, the OpenCL driver only enables GPU and accelerator
  104. devices. By setting the environment variable \ref STARPU_OPENCL_ON_CPUS
  105. to 1, the OpenCL driver will also enable CPU devices.
  106. </dd>
  107. <dt>STARPU_OPENCL_ONLY_ON_CPUS</dt>
  108. <dd>
  109. \anchor STARPU_OPENCL_ONLY_ON_CPUS
  110. \addindex __env__STARPU_OPENCL_ONLY_ON_CPUS
  111. By default, the OpenCL driver enables GPU and accelerator
  112. devices. By setting the environment variable \ref STARPU_OPENCL_ONLY_ON_CPUS
  113. to 1, the OpenCL driver will ONLY enable CPU devices.
  114. </dd>
  115. <dt>STARPU_NMIC</dt>
  116. <dd>
  117. \anchor STARPU_NMIC
  118. \addindex __env__STARPU_NMIC
  119. MIC equivalent of the environment variable \ref STARPU_NCUDA, i.e. the number of
  120. MIC devices to use.
  121. </dd>
  122. <dt>STARPU_NMICTHREADS</dt>
  123. <dd>
  124. \anchor STARPU_NMICTHREADS
  125. \addindex __env__STARPU_NMICTHREADS
  126. Number of threads to use on the MIC devices.
  127. </dd>
  128. <dt>STARPU_NMPI_MS</dt>
  129. <dd>
  130. \anchor STARPU_NMPI_MS
  131. \addindex __env__STARPU_NMPI_MS
  132. MPI Master Slave equivalent of the environment variable \ref STARPU_NCUDA, i.e. the number of
  133. MPI Master Slave devices to use.
  134. </dd>
  135. <dt>STARPU_NMPIMSTHREADS</dt>
  136. <dd>
  137. \anchor STARPU_NMPIMSTHREADS
  138. \addindex __env__STARPU_NMPIMSTHREADS
  139. Number of threads to use on the MPI Slave devices.
  140. </dd>
  141. <dt>STARPU_MPI_MASTER_NODE</dt>
  142. <dd>
  143. \anchor STARPU_MPI_MASTER_NODE
  144. \addindex __env__STARPU_MPI_MASTER_NODE
  145. This variable allows to chose which MPI node (with the MPI ID) will be the master.
  146. </dd>
  147. <dt>STARPU_NSCC</dt>
  148. <dd>
  149. \anchor STARPU_NSCC
  150. \addindex __env__STARPU_NSCC
  151. SCC equivalent of the environment variable \ref STARPU_NCUDA.
  152. </dd>
  153. <dt>STARPU_WORKERS_NOBIND</dt>
  154. <dd>
  155. \anchor STARPU_WORKERS_NOBIND
  156. \addindex __env__STARPU_WORKERS_NOBIND
  157. Setting it to non-zero will prevent StarPU from binding its threads to
  158. CPUs. This is for instance useful when running the testsuite in parallel.
  159. </dd>
  160. <dt>STARPU_WORKERS_CPUID</dt>
  161. <dd>
  162. \anchor STARPU_WORKERS_CPUID
  163. \addindex __env__STARPU_WORKERS_CPUID
  164. Passing an array of integers in \ref STARPU_WORKERS_CPUID
  165. specifies on which logical CPU the different workers should be
  166. bound. For instance, if <c>STARPU_WORKERS_CPUID = "0 1 4 5"</c>, the first
  167. worker will be bound to logical CPU #0, the second CPU worker will be bound to
  168. logical CPU #1 and so on. Note that the logical ordering of the CPUs is either
  169. determined by the OS, or provided by the library <c>hwloc</c> in case it is
  170. available. Ranges can be provided: for instance, <c>STARPU_WORKERS_CPUID = "1-3
  171. 5"</c> will bind the first three workers on logical CPUs #1, #2, and #3, and the
  172. fourth worker on logical CPU #5. Unbound ranges can also be provided:
  173. <c>STARPU_WORKERS_CPUID = "1-"</c> will bind the workers starting from logical
  174. CPU #1 up to last CPU.
  175. Note that the first workers correspond to the CUDA workers, then come the
  176. OpenCL workers, and finally the CPU workers. For example if
  177. we have <c>STARPU_NCUDA=1</c>, <c>STARPU_NOPENCL=1</c>, <c>STARPU_NCPU=2</c>
  178. and <c>STARPU_WORKERS_CPUID = "0 2 1 3"</c>, the CUDA device will be controlled
  179. by logical CPU #0, the OpenCL device will be controlled by logical CPU #2, and
  180. the logical CPUs #1 and #3 will be used by the CPU workers.
  181. If the number of workers is larger than the array given in
  182. \ref STARPU_WORKERS_CPUID, the workers are bound to the logical CPUs in a
  183. round-robin fashion: if <c>STARPU_WORKERS_CPUID = "0 1"</c>, the first
  184. and the third (resp. second and fourth) workers will be put on CPU #0
  185. (resp. CPU #1).
  186. This variable is ignored if the field
  187. starpu_conf::use_explicit_workers_bindid passed to starpu_init() is
  188. set.
  189. </dd>
  190. <dt>STARPU_MAIN_THREAD_CPUID</dt>
  191. <dd>
  192. \anchor STARPU_MAIN_THREAD_CPUID
  193. \addindex __env__STARPU_MAIN_THREAD_CPUID
  194. When defined, this make StarPU bind the thread that calls starpu_initialize() to
  195. the given CPU ID.
  196. </dd>
  197. <dt>STARPU_MPI_THREAD_CPUID</dt>
  198. <dd>
  199. \anchor STARPU_MPI_THREAD_CPUID
  200. \addindex __env__STARPU_MPI_THREAD_CPUID
  201. When defined, this make StarPU bind its MPI thread to the given CPU ID.
  202. </dd>
  203. <dt>STARPU_WORKERS_CUDAID</dt>
  204. <dd>
  205. \anchor STARPU_WORKERS_CUDAID
  206. \addindex __env__STARPU_WORKERS_CUDAID
  207. Similarly to the \ref STARPU_WORKERS_CPUID environment variable, it is
  208. possible to select which CUDA devices should be used by StarPU. On a machine
  209. equipped with 4 GPUs, setting <c>STARPU_WORKERS_CUDAID = "1 3"</c> and
  210. <c>STARPU_NCUDA=2</c> specifies that 2 CUDA workers should be created, and that
  211. they should use CUDA devices #1 and #3 (the logical ordering of the devices is
  212. the one reported by CUDA).
  213. This variable is ignored if the field
  214. starpu_conf::use_explicit_workers_cuda_gpuid passed to starpu_init()
  215. is set.
  216. </dd>
  217. <dt>STARPU_WORKERS_OPENCLID</dt>
  218. <dd>
  219. \anchor STARPU_WORKERS_OPENCLID
  220. \addindex __env__STARPU_WORKERS_OPENCLID
  221. OpenCL equivalent of the \ref STARPU_WORKERS_CUDAID environment variable.
  222. This variable is ignored if the field
  223. starpu_conf::use_explicit_workers_opencl_gpuid passed to starpu_init()
  224. is set.
  225. </dd>
  226. <dt>STARPU_WORKERS_MICID</dt>
  227. <dd>
  228. \anchor STARPU_WORKERS_MICID
  229. \addindex __env__STARPU_WORKERS_MICID
  230. MIC equivalent of the \ref STARPU_WORKERS_CUDAID environment variable.
  231. This variable is ignored if the field
  232. starpu_conf::use_explicit_workers_mic_deviceid passed to starpu_init()
  233. is set.
  234. </dd>
  235. <dt>STARPU_WORKERS_SCCID</dt>
  236. <dd>
  237. \anchor STARPU_WORKERS_SCCID
  238. \addindex __env__STARPU_WORKERS_SCCID
  239. SCC equivalent of the \ref STARPU_WORKERS_CUDAID environment variable.
  240. This variable is ignored if the field
  241. starpu_conf::use_explicit_workers_scc_deviceid passed to starpu_init()
  242. is set.
  243. </dd>
  244. <dt>STARPU_WORKER_TREE</dt>
  245. <dd>
  246. \anchor STARPU_WORKER_TREE
  247. \addindex __env__STARPU_WORKER_TREE
  248. Define to 1 to enable the tree iterator in schedulers.
  249. </dd>
  250. <dt>STARPU_SINGLE_COMBINED_WORKER</dt>
  251. <dd>
  252. \anchor STARPU_SINGLE_COMBINED_WORKER
  253. \addindex __env__STARPU_SINGLE_COMBINED_WORKER
  254. If set, StarPU will create several workers which won't be able to work
  255. concurrently. It will by default create combined workers which size goes from 1
  256. to the total number of CPU workers in the system. \ref STARPU_MIN_WORKERSIZE
  257. and \ref STARPU_MAX_WORKERSIZE can be used to change this default.
  258. </dd>
  259. <dt>STARPU_MIN_WORKERSIZE</dt>
  260. <dd>
  261. \anchor STARPU_MIN_WORKERSIZE
  262. \addindex __env__STARPU_MIN_WORKERSIZE
  263. \ref STARPU_MIN_WORKERSIZE
  264. permits to specify the minimum size of the combined workers (instead of the default 2)
  265. </dd>
  266. <dt>STARPU_MAX_WORKERSIZE</dt>
  267. <dd>
  268. \anchor STARPU_MAX_WORKERSIZE
  269. \addindex __env__STARPU_MAX_WORKERSIZE
  270. \ref STARPU_MAX_WORKERSIZE
  271. permits to specify the minimum size of the combined workers (instead of the
  272. number of CPU workers in the system)
  273. </dd>
  274. <dt>STARPU_SYNTHESIZE_ARITY_COMBINED_WORKER</dt>
  275. <dd>
  276. \anchor STARPU_SYNTHESIZE_ARITY_COMBINED_WORKER
  277. \addindex __env__STARPU_SYNTHESIZE_ARITY_COMBINED_WORKER
  278. Let the user decide how many elements are allowed between combined workers
  279. created from hwloc information. For instance, in the case of sockets with 6
  280. cores without shared L2 caches, if \ref STARPU_SYNTHESIZE_ARITY_COMBINED_WORKER is
  281. set to 6, no combined worker will be synthesized beyond one for the socket
  282. and one per core. If it is set to 3, 3 intermediate combined workers will be
  283. synthesized, to divide the socket cores into 3 chunks of 2 cores. If it set to
  284. 2, 2 intermediate combined workers will be synthesized, to divide the the socket
  285. cores into 2 chunks of 3 cores, and then 3 additional combined workers will be
  286. synthesized, to divide the former synthesized workers into a bunch of 2 cores,
  287. and the remaining core (for which no combined worker is synthesized since there
  288. is already a normal worker for it).
  289. The default, 2, thus makes StarPU tend to building a binary trees of combined
  290. workers.
  291. </dd>
  292. <dt>STARPU_DISABLE_ASYNCHRONOUS_COPY</dt>
  293. <dd>
  294. \anchor STARPU_DISABLE_ASYNCHRONOUS_COPY
  295. \addindex __env__STARPU_DISABLE_ASYNCHRONOUS_COPY
  296. Disable asynchronous copies between CPU and GPU devices.
  297. The AMD implementation of OpenCL is known to
  298. fail when copying data asynchronously. When using this implementation,
  299. it is therefore necessary to disable asynchronous data transfers.
  300. </dd>
  301. <dt>STARPU_DISABLE_ASYNCHRONOUS_CUDA_COPY</dt>
  302. <dd>
  303. \anchor STARPU_DISABLE_ASYNCHRONOUS_CUDA_COPY
  304. \addindex __env__STARPU_DISABLE_ASYNCHRONOUS_CUDA_COPY
  305. Disable asynchronous copies between CPU and CUDA devices.
  306. </dd>
  307. <dt>STARPU_DISABLE_ASYNCHRONOUS_OPENCL_COPY</dt>
  308. <dd>
  309. \anchor STARPU_DISABLE_ASYNCHRONOUS_OPENCL_COPY
  310. \addindex __env__STARPU_DISABLE_ASYNCHRONOUS_OPENCL_COPY
  311. Disable asynchronous copies between CPU and OpenCL devices.
  312. The AMD implementation of OpenCL is known to
  313. fail when copying data asynchronously. When using this implementation,
  314. it is therefore necessary to disable asynchronous data transfers.
  315. </dd>
  316. <dt>STARPU_DISABLE_ASYNCHRONOUS_MIC_COPY</dt>
  317. <dd>
  318. \anchor STARPU_DISABLE_ASYNCHRONOUS_MIC_COPY
  319. \addindex __env__STARPU_DISABLE_ASYNCHRONOUS_MIC_COPY
  320. Disable asynchronous copies between CPU and MIC devices.
  321. </dd>
  322. <dt>STARPU_DISABLE_ASYNCHRONOUS_MPI_MS_COPY</dt>
  323. <dd>
  324. \anchor STARPU_DISABLE_ASYNCHRONOUS_MPI_MS_COPY
  325. \addindex __env__STARPU_DISABLE_ASYNCHRONOUS_MPI_MS_COPY
  326. Disable asynchronous copies between CPU and MPI Slave devices.
  327. </dd>
  328. <dt>STARPU_ENABLE_CUDA_GPU_GPU_DIRECT</dt>
  329. <dd>
  330. \anchor STARPU_ENABLE_CUDA_GPU_GPU_DIRECT
  331. \addindex __env__STARPU_ENABLE_CUDA_GPU_GPU_DIRECT
  332. Enable (1) or Disable (0) direct CUDA transfers from GPU to GPU, without copying
  333. through RAM. The default is Enabled.
  334. This permits to test the performance effect of GPU-Direct.
  335. </dd>
  336. <dt>STARPU_DISABLE_PINNING</dt>
  337. <dd>
  338. \anchor STARPU_DISABLE_PINNING
  339. \addindex __env__STARPU_DISABLE_PINNING
  340. Disable (1) or Enable (0) pinning host memory allocated through starpu_malloc, starpu_memory_pin
  341. and friends. The default is Enabled.
  342. This permits to test the performance effect of memory pinning.
  343. </dd>
  344. <dt>STARPU_MIC_SINK_PROGRAM_NAME</dt>
  345. <dd>
  346. \anchor STARPU_MIC_SINK_PROGRAM_NAME
  347. \addindex __env__STARPU_MIC_SINK_PROGRAM_NAME
  348. todo
  349. </dd>
  350. <dt>STARPU_MIC_SINK_PROGRAM_PATH</dt>
  351. <dd>
  352. \anchor STARPU_MIC_SINK_PROGRAM_PATH
  353. \addindex __env__STARPU_MIC_SINK_PROGRAM_PATH
  354. todo
  355. </dd>
  356. <dt>STARPU_MIC_PROGRAM_PATH</dt>
  357. <dd>
  358. \anchor STARPU_MIC_PROGRAM_PATH
  359. \addindex __env__STARPU_MIC_PROGRAM_PATH
  360. todo
  361. </dd>
  362. </dl>
  363. \section ConfiguringTheSchedulingEngine Configuring The Scheduling Engine
  364. <dl>
  365. <dt>STARPU_SCHED</dt>
  366. <dd>
  367. \anchor STARPU_SCHED
  368. \addindex __env__STARPU_SCHED
  369. Choose between the different scheduling policies proposed by StarPU: work
  370. random, stealing, greedy, with performance models, etc.
  371. Use <c>STARPU_SCHED=help</c> to get the list of available schedulers.
  372. </dd>
  373. <dt>STARPU_MIN_PRIO</dt>
  374. <dd>
  375. \anchor STARPU_MIN_PRIO_env
  376. \addindex __env__STARPU_MIN_PRIO
  377. Set the mininum priority used by priorities-aware schedulers.
  378. </dd>
  379. <dt>STARPU_MAX_PRIO</dt>
  380. <dd>
  381. \anchor STARPU_MAX_PRIO_env
  382. \addindex __env__STARPU_MAX_PRIO
  383. Set the maximum priority used by priorities-aware schedulers.
  384. </dd>
  385. <dt>STARPU_CALIBRATE</dt>
  386. <dd>
  387. \anchor STARPU_CALIBRATE
  388. \addindex __env__STARPU_CALIBRATE
  389. If this variable is set to 1, the performance models are calibrated during
  390. the execution. If it is set to 2, the previous values are dropped to restart
  391. calibration from scratch. Setting this variable to 0 disable calibration, this
  392. is the default behaviour.
  393. Note: this currently only applies to <c>dm</c> and <c>dmda</c> scheduling policies.
  394. </dd>
  395. <dt>STARPU_CALIBRATE_MINIMUM</dt>
  396. <dd>
  397. \anchor STARPU_CALIBRATE_MINIMUM
  398. \addindex __env__STARPU_CALIBRATE_MINIMUM
  399. This defines the minimum number of calibration measurements that will be made
  400. before considering that the performance model is calibrated. The default value is 10.
  401. </dd>
  402. <dt>STARPU_BUS_CALIBRATE</dt>
  403. <dd>
  404. \anchor STARPU_BUS_CALIBRATE
  405. \addindex __env__STARPU_BUS_CALIBRATE
  406. If this variable is set to 1, the bus is recalibrated during intialization.
  407. </dd>
  408. <dt>STARPU_PREFETCH</dt>
  409. <dd>
  410. \anchor STARPU_PREFETCH
  411. \addindex __env__STARPU_PREFETCH
  412. This variable indicates whether data prefetching should be enabled (0 means
  413. that it is disabled). If prefetching is enabled, when a task is scheduled to be
  414. executed e.g. on a GPU, StarPU will request an asynchronous transfer in
  415. advance, so that data is already present on the GPU when the task starts. As a
  416. result, computation and data transfers are overlapped.
  417. Note that prefetching is enabled by default in StarPU.
  418. </dd>
  419. <dt>STARPU_SCHED_ALPHA</dt>
  420. <dd>
  421. \anchor STARPU_SCHED_ALPHA
  422. \addindex __env__STARPU_SCHED_ALPHA
  423. To estimate the cost of a task StarPU takes into account the estimated
  424. computation time (obtained thanks to performance models). The alpha factor is
  425. the coefficient to be applied to it before adding it to the communication part.
  426. </dd>
  427. <dt>STARPU_SCHED_BETA</dt>
  428. <dd>
  429. \anchor STARPU_SCHED_BETA
  430. \addindex __env__STARPU_SCHED_BETA
  431. To estimate the cost of a task StarPU takes into account the estimated
  432. data transfer time (obtained thanks to performance models). The beta factor is
  433. the coefficient to be applied to it before adding it to the computation part.
  434. </dd>
  435. <dt>STARPU_SCHED_GAMMA</dt>
  436. <dd>
  437. \anchor STARPU_SCHED_GAMMA
  438. \addindex __env__STARPU_SCHED_GAMMA
  439. Define the execution time penalty of a joule (\ref Energy-basedScheduling).
  440. </dd>
  441. <dt>STARPU_IDLE_POWER</dt>
  442. <dd>
  443. \anchor STARPU_IDLE_POWER
  444. \addindex __env__STARPU_IDLE_POWER
  445. Define the idle power of the machine (\ref Energy-basedScheduling).
  446. </dd>
  447. <dt>STARPU_PROFILING</dt>
  448. <dd>
  449. \anchor STARPU_PROFILING
  450. \addindex __env__STARPU_PROFILING
  451. Enable on-line performance monitoring (\ref EnablingOn-linePerformanceMonitoring).
  452. </dd>
  453. </dl>
  454. \section Extensions Extensions
  455. <dl>
  456. <dt>SOCL_OCL_LIB_OPENCL</dt>
  457. <dd>
  458. \anchor SOCL_OCL_LIB_OPENCL
  459. \addindex __env__SOCL_OCL_LIB_OPENCL
  460. THE SOCL test suite is only run when the environment variable
  461. \ref SOCL_OCL_LIB_OPENCL is defined. It should contain the location
  462. of the file <c>libOpenCL.so</c> of the OCL ICD implementation.
  463. </dd>
  464. <dt>OCL_ICD_VENDORS</dt>
  465. <dd>
  466. \anchor OCL_ICD_VENDORS
  467. \addindex __env__OCL_ICD_VENDORS
  468. When using SOCL with OpenCL ICD
  469. (https://forge.imag.fr/projects/ocl-icd/), this variable may be used
  470. to point to the directory where ICD files are installed. The default
  471. directory is <c>/etc/OpenCL/vendors</c>. StarPU installs ICD
  472. files in the directory <c>$prefix/share/starpu/opencl/vendors</c>.
  473. </dd>
  474. <dt>STARPU_COMM_STATS</dt>
  475. <dd>
  476. \anchor STARPU_COMM_STATS
  477. \addindex __env__STARPU_COMM_STATS
  478. Communication statistics for starpumpi (\ref MPISupport)
  479. will be enabled when the environment variable \ref STARPU_COMM_STATS
  480. is defined to an value other than 0.
  481. </dd>
  482. <dt>STARPU_MPI_CACHE</dt>
  483. <dd>
  484. \anchor STARPU_MPI_CACHE
  485. \addindex __env__STARPU_MPI_CACHE
  486. Communication cache for starpumpi (\ref MPISupport) will be
  487. disabled when the environment variable \ref STARPU_MPI_CACHE is set
  488. to 0. It is enabled by default or for any other values of the variable
  489. \ref STARPU_MPI_CACHE.
  490. </dd>
  491. <dt>STARPU_MPI_COMM</dt>
  492. <dd>
  493. \anchor STARPU_MPI_COMM
  494. \addindex __env__STARPU_MPI_COMM
  495. Communication trace for starpumpi (\ref MPISupport) will be
  496. enabled when the environment variable \ref STARPU_MPI_COMM is set
  497. to 1, and StarPU has been configured with the option
  498. \ref enable-verbose "--enable-verbose".
  499. </dd>
  500. <dt>STARPU_MPI_CACHE_STATS</dt>
  501. <dd>
  502. \anchor STARPU_MPI_CACHE_STATS
  503. \addindex __env__STARPU_MPI_CACHE_STATS
  504. When set to 1, statistics are enabled for the communication cache (\ref MPISupport). For now,
  505. it prints messages on the standard output when data are added or removed from the received
  506. communication cache.
  507. </dd>
  508. <dt>STARPU_MPI_FAKE_SIZE</dt>
  509. <dd>
  510. \anchor STARPU_MPI_FAKE_SIZE
  511. \addindex __env__STARPU_MPI_FAKE_SIZE
  512. Setting to a number makes StarPU believe that there are as many MPI nodes, even
  513. if it was run on only one MPI node. This allows e.g. to simulate the execution
  514. of one of the nodes of a big cluster without actually running the rest.
  515. It of course does not provide computation results and timing.
  516. </dd>
  517. <dt>STARPU_MPI_FAKE_RANK</dt>
  518. <dd>
  519. \anchor STARPU_MPI_FAKE_RANK
  520. \addindex __env__STARPU_MPI_FAKE_RANK
  521. Setting to a number makes StarPU believe that it runs the given MPI node, even
  522. if it was run on only one MPI node. This allows e.g. to simulate the execution
  523. of one of the nodes of a big cluster without actually running the rest.
  524. It of course does not provide computation results and timing.
  525. </dd>
  526. <dt>STARPU_SIMGRID_CUDA_MALLOC_COST</dt>
  527. <dd>
  528. \anchor STARPU_SIMGRID_CUDA_MALLOC_COST
  529. \addindex __env__STARPU_SIMGRID_CUDA_MALLOC_COST
  530. When set to 1 (which is the default), CUDA malloc costs are taken into account
  531. in simgrid mode.
  532. </dd>
  533. <dt>STARPU_SIMGRID_CUDA_QUEUE_COST</dt>
  534. <dd>
  535. \anchor STARPU_SIMGRID_CUDA_QUEUE_COST
  536. \addindex __env__STARPU_SIMGRID_CUDA_QUEUE_COST
  537. When set to 1 (which is the default), CUDA task and transfer queueing costs are
  538. taken into account in simgrid mode.
  539. </dd>
  540. <dt>STARPU_PCI_FLAT</dt>
  541. <dd>
  542. \anchor STARPU_PCI_FLAT
  543. \addindex __env__STARPU_PCI_FLAT
  544. When unset or set to 0, the platform file created for simgrid will
  545. contain PCI bandwidths and routes.
  546. </dd>
  547. <dt>STARPU_SIMGRID_QUEUE_MALLOC_COST</dt>
  548. <dd>
  549. \anchor STARPU_SIMGRID_QUEUE_MALLOC_COST
  550. \addindex __env__STARPU_SIMGRID_QUEUE_MALLOC_COST
  551. When unset or set to 1, simulate within simgrid the GPU transfer queueing.
  552. </dd>
  553. <dt>STARPU_MALLOC_SIMULATION_FOLD</dt>
  554. <dd>
  555. \anchor STARPU_MALLOC_SIMULATION_FOLD
  556. \addindex __env__STARPU_MALLOC_SIMULATION_FOLD
  557. This defines the size of the file used for folding virtual allocation, in
  558. MiB. The default is 1, thus allowing 64GiB virtual memory when Linux's
  559. <c>sysctl vm.max_map_count</c> value is the default 65535.
  560. </dd>
  561. <dt>STARPU_SIMGRID_TASK_SUBMIT_COST</dt>
  562. <dd>
  563. \anchor STARPU_SIMGRID_TASK_SUBMIT_COST
  564. \addindex __env__STARPU_SIMGRID_TASK_SUBMIT_COST
  565. When set to 1 (which is the default), task submission costs are taken into
  566. account in simgrid mode. This provides more accurate simgrid predictions,
  567. especially for the beginning of the execution.
  568. </dd>
  569. <dt>STARPU_SIMGRID_FETCHING_INPUT_COST</dt>
  570. <dd>
  571. \anchor STARPU_SIMGRID_FETCHING_INPUT_COST
  572. \addindex __env__STARPU_SIMGRID_FETCHING_INPUT_COST
  573. When set to 1 (which is the default), fetching input costs are taken into
  574. account in simgrid mode. This provides more accurate simgrid predictions,
  575. especially regarding data transfers.
  576. </dd>
  577. <dt>STARPU_SIMGRID_SCHED_COST</dt>
  578. <dd>
  579. \anchor STARPU_SIMGRID_SCHED_COST
  580. \addindex __env__STARPU_SIMGRID_SCHED_COST
  581. When set to 1 (0 is the default), scheduling costs are taken into
  582. account in simgrid mode. This provides more accurate simgrid predictions,
  583. and allows studying scheduling overhead of the runtime system. However,
  584. it also makes simulation non-deterministic.
  585. </dd>
  586. </dl>
  587. \section MiscellaneousAndDebug Miscellaneous And Debug
  588. <dl>
  589. <dt>STARPU_HOME</dt>
  590. <dd>
  591. \anchor STARPU_HOME
  592. \addindex __env__STARPU_HOME
  593. This specifies the main directory in which StarPU stores its
  594. configuration files. The default is <c>$HOME</c> on Unix environments,
  595. and <c>$USERPROFILE</c> on Windows environments.
  596. </dd>
  597. <dt>STARPU_PATH</dt>
  598. <dd>
  599. \anchor STARPU_PATH
  600. \addindex __env__STARPU_PATH
  601. Only used on Windows environments.
  602. This specifies the main directory in which StarPU is installed
  603. (\ref RunningABasicStarPUApplicationOnMicrosoft)
  604. </dd>
  605. <dt>STARPU_PERF_MODEL_DIR</dt>
  606. <dd>
  607. \anchor STARPU_PERF_MODEL_DIR
  608. \addindex __env__STARPU_PERF_MODEL_DIR
  609. This specifies the main directory in which StarPU stores its
  610. performance model files. The default is <c>$STARPU_HOME/.starpu/sampling</c>.
  611. </dd>
  612. <dt>STARPU_PERF_MODEL_HOMOGENEOUS_CPU</dt>
  613. <dd>
  614. \anchor STARPU_PERF_MODEL_HOMOGENEOUS_CPU
  615. \addindex __env__STARPU_PERF_MODEL_HOMOGENEOUS_CPU
  616. When this is set to 0, StarPU will assume that CPU devices do not have the same
  617. performance, and thus use different performance models for them, thus making
  618. kernel calibration much longer, since measurements have to be made for each CPU
  619. core.
  620. </dd>
  621. <dt>STARPU_PERF_MODEL_HOMOGENEOUS_CUDA</dt>
  622. <dd>
  623. \anchor STARPU_PERF_MODEL_HOMOGENEOUS_CUDA
  624. \addindex __env__STARPU_PERF_MODEL_HOMOGENEOUS_CUDA
  625. When this is set to 1, StarPU will assume that all CUDA devices have the same
  626. performance, and thus share performance models for them, thus allowing kernel
  627. calibration to be much faster, since measurements only have to be once for all
  628. CUDA GPUs.
  629. </dd>
  630. <dt>STARPU_PERF_MODEL_HOMOGENEOUS_OPENCL</dt>
  631. <dd>
  632. \anchor STARPU_PERF_MODEL_HOMOGENEOUS_OPENCL
  633. \addindex __env__STARPU_PERF_MODEL_HOMOGENEOUS_OPENCL
  634. When this is set to 1, StarPU will assume that all OPENCL devices have the same
  635. performance, and thus share performance models for them, thus allowing kernel
  636. calibration to be much faster, since measurements only have to be once for all
  637. OPENCL GPUs.
  638. </dd>
  639. <dt>STARPU_PERF_MODEL_HOMOGENEOUS_MIC</dt>
  640. <dd>
  641. \anchor STARPU_PERF_MODEL_HOMOGENEOUS_MIC
  642. \addindex __env__STARPU_PERF_MODEL_HOMOGENEOUS_MIC
  643. When this is set to 1, StarPU will assume that all MIC devices have the same
  644. performance, and thus share performance models for them, thus allowing kernel
  645. calibration to be much faster, since measurements only have to be once for all
  646. MIC GPUs.
  647. </dd>
  648. <dt>STARPU_PERF_MODEL_HOMOGENEOUS_MPI_MS</dt>
  649. <dd>
  650. \anchor STARPU_PERF_MODEL_HOMOGENEOUS_MPI_MS
  651. \addindex __env__STARPU_PERF_MODEL_HOMOGENEOUS_MPI_MS
  652. When this is set to 1, StarPU will assume that all MPI Slave devices have the same
  653. performance, and thus share performance models for them, thus allowing kernel
  654. calibration to be much faster, since measurements only have to be once for all
  655. MPI Slaves.
  656. </dd>
  657. <dt>STARPU_PERF_MODEL_HOMOGENEOUS_SCC</dt>
  658. <dd>
  659. \anchor STARPU_PERF_MODEL_HOMOGENEOUS_SCC
  660. \addindex __env__STARPU_PERF_MODEL_HOMOGENEOUS_SCC
  661. When this is set to 1, StarPU will assume that all SCC devices have the same
  662. performance, and thus share performance models for them, thus allowing kernel
  663. calibration to be much faster, since measurements only have to be once for all
  664. SCC GPUs.
  665. </dd>
  666. <dt>STARPU_HOSTNAME</dt>
  667. <dd>
  668. \anchor STARPU_HOSTNAME
  669. \addindex __env__STARPU_HOSTNAME
  670. When set, force the hostname to be used when dealing performance model
  671. files. Models are indexed by machine name. When running for example on
  672. a homogenenous cluster, it is possible to share the models between
  673. machines by setting <c>export STARPU_HOSTNAME=some_global_name</c>.
  674. </dd>
  675. <dt>STARPU_OPENCL_PROGRAM_DIR</dt>
  676. <dd>
  677. \anchor STARPU_OPENCL_PROGRAM_DIR
  678. \addindex __env__STARPU_OPENCL_PROGRAM_DIR
  679. This specifies the directory where the OpenCL codelet source files are
  680. located. The function starpu_opencl_load_program_source() looks
  681. for the codelet in the current directory, in the directory specified
  682. by the environment variable \ref STARPU_OPENCL_PROGRAM_DIR, in the
  683. directory <c>share/starpu/opencl</c> of the installation directory of
  684. StarPU, and finally in the source directory of StarPU.
  685. </dd>
  686. <dt>STARPU_SILENT</dt>
  687. <dd>
  688. \anchor STARPU_SILENT
  689. \addindex __env__STARPU_SILENT
  690. This variable allows to disable verbose mode at runtime when StarPU
  691. has been configured with the option \ref enable-verbose "--enable-verbose". It also
  692. disables the display of StarPU information and warning messages.
  693. </dd>
  694. <dt>STARPU_LOGFILENAME</dt>
  695. <dd>
  696. \anchor STARPU_LOGFILENAME
  697. \addindex __env__STARPU_LOGFILENAME
  698. This variable specifies in which file the debugging output should be saved to.
  699. </dd>
  700. <dt>STARPU_FXT_PREFIX</dt>
  701. <dd>
  702. \anchor STARPU_FXT_PREFIX
  703. \addindex __env__STARPU_FXT_PREFIX
  704. This variable specifies in which directory to save the trace generated if FxT is enabled. It needs to have a trailing '/' character.
  705. </dd>
  706. <dt>STARPU_FXT_TRACE</dt>
  707. <dd>
  708. \anchor STARPU_FXT_TRACE
  709. \addindex __env__STARPU_FXT_TRACE
  710. This variable specifies whether to generate (1) or not (0) the FxT trace in /tmp/prof_file_XXX_YYY . The default is 1 (generate it)
  711. </dd>
  712. <dt>STARPU_LIMIT_CUDA_devid_MEM</dt>
  713. <dd>
  714. \anchor STARPU_LIMIT_CUDA_devid_MEM
  715. \addindex __env__STARPU_LIMIT_CUDA_devid_MEM
  716. This variable specifies the maximum number of megabytes that should be
  717. available to the application on the CUDA device with the identifier
  718. <c>devid</c>. This variable is intended to be used for experimental
  719. purposes as it emulates devices that have a limited amount of memory.
  720. When defined, the variable overwrites the value of the variable
  721. \ref STARPU_LIMIT_CUDA_MEM.
  722. </dd>
  723. <dt>STARPU_LIMIT_CUDA_MEM</dt>
  724. <dd>
  725. \anchor STARPU_LIMIT_CUDA_MEM
  726. \addindex __env__STARPU_LIMIT_CUDA_MEM
  727. This variable specifies the maximum number of megabytes that should be
  728. available to the application on each CUDA devices. This variable is
  729. intended to be used for experimental purposes as it emulates devices
  730. that have a limited amount of memory.
  731. </dd>
  732. <dt>STARPU_LIMIT_OPENCL_devid_MEM</dt>
  733. <dd>
  734. \anchor STARPU_LIMIT_OPENCL_devid_MEM
  735. \addindex __env__STARPU_LIMIT_OPENCL_devid_MEM
  736. This variable specifies the maximum number of megabytes that should be
  737. available to the application on the OpenCL device with the identifier
  738. <c>devid</c>. This variable is intended to be used for experimental
  739. purposes as it emulates devices that have a limited amount of memory.
  740. When defined, the variable overwrites the value of the variable
  741. \ref STARPU_LIMIT_OPENCL_MEM.
  742. </dd>
  743. <dt>STARPU_LIMIT_OPENCL_MEM</dt>
  744. <dd>
  745. \anchor STARPU_LIMIT_OPENCL_MEM
  746. \addindex __env__STARPU_LIMIT_OPENCL_MEM
  747. This variable specifies the maximum number of megabytes that should be
  748. available to the application on each OpenCL devices. This variable is
  749. intended to be used for experimental purposes as it emulates devices
  750. that have a limited amount of memory.
  751. </dd>
  752. <dt>STARPU_LIMIT_CPU_MEM</dt>
  753. <dd>
  754. \anchor STARPU_LIMIT_CPU_MEM
  755. \addindex __env__STARPU_LIMIT_CPU_MEM
  756. This variable specifies the maximum number of megabytes that should be
  757. available to the application in the main CPU memory. Setting it enables allocation
  758. cache in main memory. Setting it to zero lets StarPU overflow memory.
  759. </dd>
  760. <dt>STARPU_LIMIT_CPU_NUMA_devid_MEM</dt>
  761. <dd>
  762. \anchor STARPU_LIMIT_CPU_NUMA_devid_MEM
  763. \addindex __env__STARPU_LIMIT_CPU_NUMA_devid_MEM
  764. This variable specifies the maximum number of megabytes that should be
  765. available to the application on the NUMA node with the OS identifier <c>devid</c>.
  766. </dd>
  767. <dt>STARPU_MINIMUM_AVAILABLE_MEM</dt>
  768. <dd>
  769. \anchor STARPU_MINIMUM_AVAILABLE_MEM
  770. \addindex __env__STARPU_MINIMUM_AVAILABLE_MEM
  771. This specifies the minimum percentage of memory that should be available in GPUs
  772. (or in main memory, when using out of core), below which a reclaiming pass is
  773. performed. The default is 0%.
  774. </dd>
  775. <dt>STARPU_TARGET_AVAILABLE_MEM</dt>
  776. <dd>
  777. \anchor STARPU_TARGET_AVAILABLE_MEM
  778. \addindex __env__STARPU_TARGET_AVAILABLE_MEM
  779. This specifies the target percentage of memory that should be reached in
  780. GPUs (or in main memory, when using out of core), when performing a periodic
  781. reclaiming pass. The default is 0%.
  782. </dd>
  783. <dt>STARPU_MINIMUM_CLEAN_BUFFERS</dt>
  784. <dd>
  785. \anchor STARPU_MINIMUM_CLEAN_BUFFERS
  786. \addindex __env__STARPU_MINIMUM_CLEAN_BUFFERS
  787. This specifies the minimum percentage of number of buffers that should be clean in GPUs
  788. (or in main memory, when using out of core), below which asynchronous writebacks will be
  789. issued. The default is 5%.
  790. </dd>
  791. <dt>STARPU_TARGET_CLEAN_BUFFERS</dt>
  792. <dd>
  793. \anchor STARPU_TARGET_CLEAN_BUFFERS
  794. \addindex __env__STARPU_TARGET_CLEAN_BUFFERS
  795. This specifies the target percentage of number of buffers that should be reached in
  796. GPUs (or in main memory, when using out of core), when performing an asynchronous
  797. writeback pass. The default is 10%.
  798. </dd>
  799. <dt>STARPU_DIDUSE_BARRIER</dt>
  800. <dd>
  801. \anchor STARPU_DIDUSE_BARRIER
  802. \addindex __env__STARPU_DIDUSE_BARRIER
  803. When set to 1, StarPU will never evict a piece of data if it has not been used
  804. by at least one task. This avoids odd behaviors under high memory pressure, but
  805. can lead to deadlocks, so is to be considered experimental only.
  806. </dd>
  807. <dt>STARPU_DISK_SWAP</dt>
  808. <dd>
  809. \anchor STARPU_DISK_SWAP
  810. \addindex __env__STARPU_DISK_SWAP
  811. This specifies a path where StarPU can push data when the main memory is getting
  812. full.
  813. </dd>
  814. <dt>STARPU_DISK_SWAP_BACKEND</dt>
  815. <dd>
  816. \anchor STARPU_DISK_SWAP_BACKEND
  817. \addindex __env__STARPU_DISK_SWAP_BACKEND
  818. This specifies then backend to be used by StarPU to push data when the main
  819. memory is getting full. The default is unistd (i.e. using read/write functions),
  820. other values are stdio (i.e. using fread/fwrite), unistd_o_direct (i.e. using
  821. read/write with O_DIRECT), leveldb (i.e. using a leveldb database), and hdf5
  822. (i.e. using HDF5 library).
  823. </dd>
  824. <dt>STARPU_DISK_SWAP_SIZE</dt>
  825. <dd>
  826. \anchor STARPU_DISK_SWAP_SIZE
  827. \addindex __env__STARPU_DISK_SWAP_SIZE
  828. This specifies then maximum size in MiB to be used by StarPU to push data when the main
  829. memory is getting full. The default is unlimited.
  830. </dd>
  831. <dt>STARPU_LIMIT_MAX_SUBMITTED_TASKS</dt>
  832. <dd>
  833. \anchor STARPU_LIMIT_MAX_SUBMITTED_TASKS
  834. \addindex __env__STARPU_LIMIT_MAX_SUBMITTED_TASKS
  835. This variable allows the user to control the task submission flow by specifying
  836. to StarPU a maximum number of submitted tasks allowed at a given time, i.e. when
  837. this limit is reached task submission becomes blocking until enough tasks have
  838. completed, specified by \ref STARPU_LIMIT_MIN_SUBMITTED_TASKS.
  839. Setting it enables allocation cache buffer reuse in main memory.
  840. </dd>
  841. <dt>STARPU_LIMIT_MIN_SUBMITTED_TASKS</dt>
  842. <dd>
  843. \anchor STARPU_LIMIT_MIN_SUBMITTED_TASKS
  844. \addindex __env__STARPU_LIMIT_MIN_SUBMITTED_TASKS
  845. This variable allows the user to control the task submission flow by specifying
  846. to StarPU a submitted task threshold to wait before unblocking task submission. This
  847. variable has to be used in conjunction with \ref STARPU_LIMIT_MAX_SUBMITTED_TASKS
  848. which puts the task submission thread to
  849. sleep. Setting it enables allocation cache buffer reuse in main memory.
  850. </dd>
  851. <dt>STARPU_TRACE_BUFFER_SIZE</dt>
  852. <dd>
  853. \anchor STARPU_TRACE_BUFFER_SIZE
  854. \addindex __env__STARPU_TRACE_BUFFER_SIZE
  855. This sets the buffer size for recording trace events in MiB. Setting it to a big
  856. size allows to avoid pauses in the trace while it is recorded on the disk. This
  857. however also consumes memory, of course. The default value is 64.
  858. </dd>
  859. <dt>STARPU_GENERATE_TRACE</dt>
  860. <dd>
  861. \anchor STARPU_GENERATE_TRACE
  862. \addindex __env__STARPU_GENERATE_TRACE
  863. When set to <c>1</c>, this variable indicates that StarPU should automatically
  864. generate a Paje trace when starpu_shutdown() is called.
  865. </dd>
  866. <dt>STARPU_ENABLE_STATS</dt>
  867. <dd>
  868. \anchor STARPU_ENABLE_STATS
  869. \addindex __env__STARPU_ENABLE_STATS
  870. When defined, enable gathering various data statistics (\ref DataStatistics).
  871. </dd>
  872. <dt>STARPU_MEMORY_STATS</dt>
  873. <dd>
  874. \anchor STARPU_MEMORY_STATS
  875. \addindex __env__STARPU_MEMORY_STATS
  876. When set to 0, disable the display of memory statistics on data which
  877. have not been unregistered at the end of the execution (\ref MemoryFeedback).
  878. </dd>
  879. <dt>STARPU_MAX_MEMORY_USE</dt>
  880. <dd>
  881. \anchor STARPU_MAX_MEMORY_USE
  882. \addindex __env__STARPU_MAX_MEMORY_USE
  883. When set to 1, display at the end of the execution the maximum memory used by
  884. StarPU for internal data structures during execution.
  885. </dd>
  886. <dt>STARPU_BUS_STATS</dt>
  887. <dd>
  888. \anchor STARPU_BUS_STATS
  889. \addindex __env__STARPU_BUS_STATS
  890. When defined, statistics about data transfers will be displayed when calling
  891. starpu_shutdown() (\ref Profiling).
  892. </dd>
  893. <dt>STARPU_WORKER_STATS</dt>
  894. <dd>
  895. \anchor STARPU_WORKER_STATS
  896. \addindex __env__STARPU_WORKER_STATS
  897. When defined, statistics about the workers will be displayed when calling
  898. starpu_shutdown() (\ref Profiling). When combined with the
  899. environment variable \ref STARPU_PROFILING, it displays the energy
  900. consumption (\ref Energy-basedScheduling).
  901. </dd>
  902. <dt>STARPU_STATS</dt>
  903. <dd>
  904. \anchor STARPU_STATS
  905. \addindex __env__STARPU_STATS
  906. When set to 0, data statistics will not be displayed at the
  907. end of the execution of an application (\ref DataStatistics).
  908. </dd>
  909. <dt>STARPU_WATCHDOG_TIMEOUT</dt>
  910. <dd>
  911. \anchor STARPU_WATCHDOG_TIMEOUT
  912. \addindex __env__STARPU_WATCHDOG_TIMEOUT
  913. When set to a value other than 0, allows to make StarPU print an error
  914. message whenever StarPU does not terminate any task for the given time (in µs),
  915. but lets the application continue normally. Should
  916. be used in combination with \ref STARPU_WATCHDOG_CRASH
  917. (see \ref DetectionStuckConditions).
  918. </dd>
  919. <dt>STARPU_WATCHDOG_CRASH</dt>
  920. <dd>
  921. \anchor STARPU_WATCHDOG_CRASH
  922. \addindex __env__STARPU_WATCHDOG_CRASH
  923. When set to a value other than 0, it triggers a crash when the watch
  924. dog is reached, thus allowing to catch the situation in gdb, etc
  925. (see \ref DetectionStuckConditions)
  926. </dd>
  927. <dt>STARPU_WATCHDOG_DELAY</dt>
  928. <dd>
  929. \anchor STARPU_WATCHDOG_DELAY
  930. \addindex __env__STARPU_WATCHDOG_DELAY
  931. This delays the activation of the watchdog by the given time (in µs). This can
  932. be convenient for letting the application initialize data etc. before starting
  933. to look for idle time.
  934. </dd>
  935. <dt>STARPU_TASK_BREAK_ON_PUSH</dt>
  936. <dd>
  937. \anchor STARPU_TASK_BREAK_ON_PUSH
  938. \addindex __env__STARPU_TASK_BREAK_ON_PUSH
  939. When this variable contains a job id, StarPU will raise SIGTRAP when the task
  940. with that job id is being pushed to the scheduler, which will be nicely catched by debuggers
  941. (see \ref DebuggingScheduling)
  942. </dd>
  943. <dt>STARPU_TASK_BREAK_ON_SCHED</dt>
  944. <dd>
  945. \anchor STARPU_TASK_BREAK_ON_SCHED
  946. \addindex __env__STARPU_TASK_BREAK_ON_SCHED
  947. When this variable contains a job id, StarPU will raise SIGTRAP when the task
  948. with that job id is being scheduled by the scheduler (at a scheduler-specific
  949. point), which will be nicely catched by debuggers.
  950. This only works for schedulers which have such a scheduling point defined
  951. (see \ref DebuggingScheduling)
  952. </dd>
  953. <dt>STARPU_TASK_BREAK_ON_POP</dt>
  954. <dd>
  955. \anchor STARPU_TASK_BREAK_ON_POP
  956. \addindex __env__STARPU_TASK_BREAK_ON_POP
  957. When this variable contains a job id, StarPU will raise SIGTRAP when the task
  958. with that job id is being popped from the scheduler, which will be nicely catched by debuggers
  959. (see \ref DebuggingScheduling)
  960. </dd>
  961. <dt>STARPU_TASK_BREAK_ON_EXEC</dt>
  962. <dd>
  963. \anchor STARPU_TASK_BREAK_ON_EXEC
  964. \addindex __env__STARPU_TASK_BREAK_ON_EXEC
  965. When this variable contains a job id, StarPU will raise SIGTRAP when the task
  966. with that job id is being executed, which will be nicely catched by debuggers
  967. (see \ref DebuggingScheduling)
  968. </dd>
  969. <dt>STARPU_DISABLE_KERNELS</dt>
  970. <dd>
  971. \anchor STARPU_DISABLE_KERNELS
  972. \addindex __env__STARPU_DISABLE_KERNELS
  973. When set to a value other than 1, it disables actually calling the kernel
  974. functions, thus allowing to quickly check that the task scheme is working
  975. properly, without performing the actual application-provided computation.
  976. </dd>
  977. <dt>STARPU_HISTORY_MAX_ERROR</dt>
  978. <dd>
  979. \anchor STARPU_HISTORY_MAX_ERROR
  980. \addindex __env__STARPU_HISTORY_MAX_ERROR
  981. History-based performance models will drop measurements which are really far
  982. froom the measured average. This specifies the allowed variation. The default is
  983. 50 (%), i.e. the measurement is allowed to be x1.5 faster or /1.5 slower than the
  984. average.
  985. </dd>
  986. <dt>STARPU_RAND_SEED</dt>
  987. <dd>
  988. \anchor STARPU_RAND_SEED
  989. \addindex __env__STARPU_RAND_SEED
  990. The random scheduler and some examples use random numbers for their own
  991. working. Depending on the examples, the seed is by default juste always 0 or
  992. the current time() (unless simgrid mode is enabled, in which case it is always
  993. 0). \ref STARPU_RAND_SEED allows to set the seed to a specific value.
  994. </dd>
  995. <dt>STARPU_IDLE_TIME</dt>
  996. <dd>
  997. \anchor STARPU_IDLE_TIME
  998. \addindex __env__STARPU_IDLE_TIME
  999. When set to a value being a valid filename, a corresponding file
  1000. will be created when shutting down StarPU. The file will contain the
  1001. sum of all the workers' idle time.
  1002. </dd>
  1003. <dt>STARPU_GLOBAL_ARBITER</dt>
  1004. <dd>
  1005. \anchor STARPU_GLOBAL_ARBITER
  1006. \addindex __env__STARPU_GLOBAL_ARBITER
  1007. When set to a positive value, StarPU will create a arbiter, which
  1008. implements an advanced but centralized management of concurrent data
  1009. accesses (see \ref ConcurrentDataAccess).
  1010. </dd>
  1011. <dt>STARPU_USE_NUMA</dt>
  1012. <dd>
  1013. \anchor STARPU_USE_NUMA
  1014. \addindex __env__STARPU_USE_NUMA
  1015. When defined, NUMA nodes are taking into account by StarPU. Otherwise, memory
  1016. is considered as only one node. This is experimental for now.
  1017. When enabled, STARPU_MAIN_MEMORY is a pointer to the NUMA node associated to the
  1018. first CPU worker if it exists, the NUMA node associated to the first GPU discovered otherwise.
  1019. If StarPU doesn't find any NUMA node after these step, STARPU_MAIN_MEMORY is the first NUMA node
  1020. discovered by StarPU.
  1021. </dd>
  1022. </dl>
  1023. \section ConfiguringTheHypervisor Configuring The Hypervisor
  1024. <dl>
  1025. <dt>SC_HYPERVISOR_POLICY</dt>
  1026. <dd>
  1027. \anchor SC_HYPERVISOR_POLICY
  1028. \addindex __env__SC_HYPERVISOR_POLICY
  1029. Choose between the different resizing policies proposed by StarPU for the hypervisor:
  1030. idle, app_driven, feft_lp, teft_lp; ispeed_lp, throughput_lp etc.
  1031. Use <c>SC_HYPERVISOR_POLICY=help</c> to get the list of available policies for the hypervisor
  1032. </dd>
  1033. <dt>SC_HYPERVISOR_TRIGGER_RESIZE</dt>
  1034. <dd>
  1035. \anchor SC_HYPERVISOR_TRIGGER_RESIZE
  1036. \addindex __env__SC_HYPERVISOR_TRIGGER_RESIZE
  1037. Choose how should the hypervisor be triggered: <c>speed</c> if the resizing algorithm should
  1038. be called whenever the speed of the context does not correspond to an optimal precomputed value,
  1039. <c>idle</c> it the resizing algorithm should be called whenever the workers are idle for a period
  1040. longer than the value indicated when configuring the hypervisor.
  1041. </dd>
  1042. <dt>SC_HYPERVISOR_START_RESIZE</dt>
  1043. <dd>
  1044. \anchor SC_HYPERVISOR_START_RESIZE
  1045. \addindex __env__SC_HYPERVISOR_START_RESIZE
  1046. Indicate the moment when the resizing should be available. The value correspond to the percentage
  1047. of the total time of execution of the application. The default value is the resizing frame.
  1048. </dd>
  1049. <dt>SC_HYPERVISOR_MAX_SPEED_GAP</dt>
  1050. <dd>
  1051. \anchor SC_HYPERVISOR_MAX_SPEED_GAP
  1052. \addindex __env__SC_HYPERVISOR_MAX_SPEED_GAP
  1053. Indicate the ratio of speed difference between contexts that should trigger the hypervisor.
  1054. This situation may occur only when a theoretical speed could not be computed and the hypervisor
  1055. has no value to compare the speed to. Otherwise the resizing of a context is not influenced by the
  1056. the speed of the other contexts, but only by the the value that a context should have.
  1057. </dd>
  1058. <dt>SC_HYPERVISOR_STOP_PRINT</dt>
  1059. <dd>
  1060. \anchor SC_HYPERVISOR_STOP_PRINT
  1061. \addindex __env__SC_HYPERVISOR_STOP_PRINT
  1062. By default the values of the speed of the workers is printed during the execution
  1063. of the application. If the value 1 is given to this environment variable this printing
  1064. is not done.
  1065. </dd>
  1066. <dt>SC_HYPERVISOR_LAZY_RESIZE</dt>
  1067. <dd>
  1068. \anchor SC_HYPERVISOR_LAZY_RESIZE
  1069. \addindex __env__SC_HYPERVISOR_LAZY_RESIZE
  1070. By default the hypervisor resizes the contexts in a lazy way, that is workers are firstly added to a new context
  1071. before removing them from the previous one. Once this workers are clearly taken into account
  1072. into the new context (a task was poped there) we remove them from the previous one. However if the application
  1073. would like that the change in the distribution of workers should change right away this variable should be set to 0
  1074. </dd>
  1075. <dt>SC_HYPERVISOR_SAMPLE_CRITERIA</dt>
  1076. <dd>
  1077. \anchor SC_HYPERVISOR_SAMPLE_CRITERIA
  1078. \addindex __env__SC_HYPERVISOR_SAMPLE_CRITERIA
  1079. By default the hypervisor uses a sample of flops when computing the speed of the contexts and of the workers.
  1080. If this variable is set to <c>time</c> the hypervisor uses a sample of time (10% of an aproximation of the total
  1081. execution time of the application)
  1082. </dd>
  1083. </dl>
  1084. */