501_environment_variables.doxy 43 KB

12345678910111213141516171819202122232425262728293031323334353637383940414243444546474849505152535455565758596061626364656667686970717273747576777879808182838485868788899091929394959697989910010110210310410510610710810911011111211311411511611711811912012112212312412512612712812913013113213313413513613713813914014114214314414514614714814915015115215315415515615715815916016116216316416516616716816917017117217317417517617717817918018118218318418518618718818919019119219319419519619719819920020120220320420520620720820921021121221321421521621721821922022122222322422522622722822923023123223323423523623723823924024124224324424524624724824925025125225325425525625725825926026126226326426526626726826927027127227327427527627727827928028128228328428528628728828929029129229329429529629729829930030130230330430530630730830931031131231331431531631731831932032132232332432532632732832933033133233333433533633733833934034134234334434534634734834935035135235335435535635735835936036136236336436536636736836937037137237337437537637737837938038138238338438538638738838939039139239339439539639739839940040140240340440540640740840941041141241341441541641741841942042142242342442542642742842943043143243343443543643743843944044144244344444544644744844945045145245345445545645745845946046146246346446546646746846947047147247347447547647747847948048148248348448548648748848949049149249349449549649749849950050150250350450550650750850951051151251351451551651751851952052152252352452552652752852953053153253353453553653753853954054154254354454554654754854955055155255355455555655755855956056156256356456556656756856957057157257357457557657757857958058158258358458558658758858959059159259359459559659759859960060160260360460560660760860961061161261361461561661761861962062162262362462562662762862963063163263363463563663763863964064164264364464564664764864965065165265365465565665765865966066166266366466566666766866967067167267367467567667767867968068168268368468568668768868969069169269369469569669769869970070170270370470570670770870971071171271371471571671771871972072172272372472572672772872973073173273373473573673773873974074174274374474574674774874975075175275375475575675775875976076176276376476576676776876977077177277377477577677777877978078178278378478578678778878979079179279379479579679779879980080180280380480580680780880981081181281381481581681781881982082182282382482582682782882983083183283383483583683783883984084184284384484584684784884985085185285385485585685785885986086186286386486586686786886987087187287387487587687787887988088188288388488588688788888989089189289389489589689789889990090190290390490590690790890991091191291391491591691791891992092192292392492592692792892993093193293393493593693793893994094194294394494594694794894995095195295395495595695795895996096196296396496596696796896997097197297397497597697797897998098198298398498598698798898999099199299399499599699799899910001001100210031004100510061007100810091010101110121013101410151016101710181019102010211022102310241025102610271028102910301031103210331034103510361037103810391040104110421043104410451046104710481049105010511052105310541055105610571058105910601061106210631064106510661067106810691070107110721073107410751076107710781079108010811082108310841085108610871088108910901091109210931094109510961097109810991100110111021103110411051106110711081109111011111112111311141115111611171118111911201121112211231124112511261127112811291130113111321133113411351136113711381139114011411142114311441145114611471148114911501151115211531154115511561157115811591160116111621163116411651166116711681169117011711172117311741175117611771178117911801181118211831184118511861187118811891190119111921193119411951196119711981199120012011202120312041205120612071208120912101211121212131214121512161217121812191220122112221223122412251226122712281229123012311232123312341235123612371238123912401241124212431244124512461247124812491250125112521253125412551256125712581259126012611262126312641265126612671268
  1. /* StarPU --- Runtime system for heterogeneous multicore architectures.
  2. *
  3. * Copyright (C) 2011-2013,2015-2017 Inria
  4. * Copyright (C) 2010-2018 CNRS
  5. * Copyright (C) 2009-2011,2013-2017 Université de Bordeaux
  6. * Copyright (C) 2016 Uppsala University
  7. *
  8. * StarPU is free software; you can redistribute it and/or modify
  9. * it under the terms of the GNU Lesser General Public License as published by
  10. * the Free Software Foundation; either version 2.1 of the License, or (at
  11. * your option) any later version.
  12. *
  13. * StarPU is distributed in the hope that it will be useful, but
  14. * WITHOUT ANY WARRANTY; without even the implied warranty of
  15. * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.
  16. *
  17. * See the GNU Lesser General Public License in COPYING.LGPL for more details.
  18. */
  19. /*! \page ExecutionConfigurationThroughEnvironmentVariables Execution Configuration Through Environment Variables
  20. The behavior of the StarPU library and tools may be tuned thanks to
  21. the following environment variables.
  22. \section ConfiguringWorkers Configuring Workers
  23. <dl>
  24. <dt>STARPU_NCPU</dt>
  25. <dd>
  26. \anchor STARPU_NCPU
  27. \addindex __env__STARPU_NCPU
  28. Specify the number of CPU workers (thus not including workers
  29. dedicated to control accelerators). Note that by default, StarPU will
  30. not allocate more CPU workers than there are physical CPUs, and that
  31. some CPUs are used to control the accelerators.
  32. </dd>
  33. <dt>STARPU_NCPUS</dt>
  34. <dd>
  35. \anchor STARPU_NCPUS
  36. \addindex __env__STARPU_NCPUS
  37. This variable is deprecated. You should use \ref STARPU_NCPU.
  38. </dd>
  39. <dt>STARPU_NCUDA</dt>
  40. <dd>
  41. \anchor STARPU_NCUDA
  42. \addindex __env__STARPU_NCUDA
  43. Specify the number of CUDA devices that StarPU can use. If
  44. \ref STARPU_NCUDA is lower than the number of physical devices, it is
  45. possible to select which CUDA devices should be used by the means of the
  46. environment variable \ref STARPU_WORKERS_CUDAID. By default, StarPU will
  47. create as many CUDA workers as there are CUDA devices.
  48. </dd>
  49. <dt>STARPU_NWORKER_PER_CUDA</dt>
  50. <dd>
  51. \anchor STARPU_NWORKER_PER_CUDA
  52. \addindex __env__STARPU_NWORKER_PER_CUDA
  53. Specify the number of workers per CUDA device, and thus the number of kernels
  54. which will be concurrently running on the devices. The default value is 1.
  55. </dd>
  56. <dt>STARPU_CUDA_THREAD_PER_WORKER</dt>
  57. <dd>
  58. \anchor STARPU_CUDA_THREAD_PER_WORKER
  59. \addindex __env__STARPU_CUDA_THREAD_PER_WORKER
  60. Specify if the cuda driver should provide a thread per stream or a single thread
  61. dealing with all the streams. 0 if one thread per stream, 1 otherwise. The default
  62. value is 0. Setting it to 1 is contradictory with setting STARPU_CUDA_THREAD_PER_DEV to 1.
  63. </dd>
  64. <dt>STARPU_CUDA_THREAD_PER_DEV</dt>
  65. <dd>
  66. \anchor STARPU_CUDA_THREAD_PER_DEV
  67. \addindex __env__STARPU_CUDA_THREAD_PER_DEV
  68. Specify if the cuda driver should provide a thread per device or a single thread
  69. dealing with all the devices. 0 if one thread per device, 1 otherwise. The default
  70. value is 1, unless STARPU_CUDA_THREAD_PER_WORKER is set to 1. Setting it to 1 is
  71. contradictory with setting STARPU_CUDA_THREAD_PER_WORKER to 1.
  72. </dd>
  73. <dt>STARPU_CUDA_PIPELINE</dt>
  74. <dd>
  75. \anchor STARPU_CUDA_PIPELINE
  76. \addindex __env__STARPU_CUDA_PIPELINE
  77. Specify how many asynchronous tasks are submitted in advance on CUDA
  78. devices. This for instance permits to overlap task management with the execution
  79. of previous tasks, but it also allows concurrent execution on Fermi cards, which
  80. otherwise bring spurious synchronizations. The default is 2. Setting the value to 0 forces a synchronous
  81. execution of all tasks.
  82. </dd>
  83. <dt>STARPU_NOPENCL</dt>
  84. <dd>
  85. \anchor STARPU_NOPENCL
  86. \addindex __env__STARPU_NOPENCL
  87. OpenCL equivalent of the environment variable \ref STARPU_NCUDA.
  88. </dd>
  89. <dt>STARPU_OPENCL_PIPELINE</dt>
  90. <dd>
  91. \anchor STARPU_OPENCL_PIPELINE
  92. \addindex __env__STARPU_OPENCL_PIPELINE
  93. Specify how many asynchronous tasks are submitted in advance on OpenCL
  94. devices. This for instance permits to overlap task management with the execution
  95. of previous tasks, but it also allows concurrent execution on Fermi cards, which
  96. otherwise bring spurious synchronizations. The default is 2. Setting the value to 0 forces a synchronous
  97. execution of all tasks.
  98. </dd>
  99. <dt>STARPU_OPENCL_ON_CPUS</dt>
  100. <dd>
  101. \anchor STARPU_OPENCL_ON_CPUS
  102. \addindex __env__STARPU_OPENCL_ON_CPUS
  103. By default, the OpenCL driver only enables GPU and accelerator
  104. devices. By setting the environment variable \ref STARPU_OPENCL_ON_CPUS
  105. to 1, the OpenCL driver will also enable CPU devices.
  106. </dd>
  107. <dt>STARPU_OPENCL_ONLY_ON_CPUS</dt>
  108. <dd>
  109. \anchor STARPU_OPENCL_ONLY_ON_CPUS
  110. \addindex __env__STARPU_OPENCL_ONLY_ON_CPUS
  111. By default, the OpenCL driver enables GPU and accelerator
  112. devices. By setting the environment variable \ref STARPU_OPENCL_ONLY_ON_CPUS
  113. to 1, the OpenCL driver will ONLY enable CPU devices.
  114. </dd>
  115. <dt>STARPU_NMIC</dt>
  116. <dd>
  117. \anchor STARPU_NMIC
  118. \addindex __env__STARPU_NMIC
  119. MIC equivalent of the environment variable \ref STARPU_NCUDA, i.e. the number of
  120. MIC devices to use.
  121. </dd>
  122. <dt>STARPU_NMICTHREADS</dt>
  123. <dd>
  124. \anchor STARPU_NMICTHREADS
  125. \addindex __env__STARPU_NMICTHREADS
  126. Number of threads to use on the MIC devices.
  127. </dd>
  128. <dt>STARPU_NMPI_MS</dt>
  129. <dd>
  130. \anchor STARPU_NMPI_MS
  131. \addindex __env__STARPU_NMPI_MS
  132. MPI Master Slave equivalent of the environment variable \ref STARPU_NCUDA, i.e. the number of
  133. MPI Master Slave devices to use.
  134. </dd>
  135. <dt>STARPU_NMPIMSTHREADS</dt>
  136. <dd>
  137. \anchor STARPU_NMPIMSTHREADS
  138. \addindex __env__STARPU_NMPIMSTHREADS
  139. Number of threads to use on the MPI Slave devices.
  140. </dd>
  141. <dt>STARPU_MPI_MASTER_NODE</dt>
  142. <dd>
  143. \anchor STARPU_MPI_MASTER_NODE
  144. \addindex __env__STARPU_MPI_MASTER_NODE
  145. This variable allows to chose which MPI node (with the MPI ID) will be the master.
  146. </dd>
  147. <dt>STARPU_NSCC</dt>
  148. <dd>
  149. \anchor STARPU_NSCC
  150. \addindex __env__STARPU_NSCC
  151. SCC equivalent of the environment variable \ref STARPU_NCUDA.
  152. </dd>
  153. <dt>STARPU_WORKERS_NOBIND</dt>
  154. <dd>
  155. \anchor STARPU_WORKERS_NOBIND
  156. \addindex __env__STARPU_WORKERS_NOBIND
  157. Setting it to non-zero will prevent StarPU from binding its threads to
  158. CPUs. This is for instance useful when running the testsuite in parallel.
  159. </dd>
  160. <dt>STARPU_WORKERS_CPUID</dt>
  161. <dd>
  162. \anchor STARPU_WORKERS_CPUID
  163. \addindex __env__STARPU_WORKERS_CPUID
  164. Passing an array of integers in \ref STARPU_WORKERS_CPUID
  165. specifies on which logical CPU the different workers should be
  166. bound. For instance, if <c>STARPU_WORKERS_CPUID = "0 1 4 5"</c>, the first
  167. worker will be bound to logical CPU #0, the second CPU worker will be bound to
  168. logical CPU #1 and so on. Note that the logical ordering of the CPUs is either
  169. determined by the OS, or provided by the library <c>hwloc</c> in case it is
  170. available. Ranges can be provided: for instance, <c>STARPU_WORKERS_CPUID = "1-3
  171. 5"</c> will bind the first three workers on logical CPUs #1, #2, and #3, and the
  172. fourth worker on logical CPU #5. Unbound ranges can also be provided:
  173. <c>STARPU_WORKERS_CPUID = "1-"</c> will bind the workers starting from logical
  174. CPU #1 up to last CPU.
  175. Note that the first workers correspond to the CUDA workers, then come the
  176. OpenCL workers, and finally the CPU workers. For example if
  177. we have <c>STARPU_NCUDA=1</c>, <c>STARPU_NOPENCL=1</c>, <c>STARPU_NCPU=2</c>
  178. and <c>STARPU_WORKERS_CPUID = "0 2 1 3"</c>, the CUDA device will be controlled
  179. by logical CPU #0, the OpenCL device will be controlled by logical CPU #2, and
  180. the logical CPUs #1 and #3 will be used by the CPU workers.
  181. If the number of workers is larger than the array given in
  182. \ref STARPU_WORKERS_CPUID, the workers are bound to the logical CPUs in a
  183. round-robin fashion: if <c>STARPU_WORKERS_CPUID = "0 1"</c>, the first
  184. and the third (resp. second and fourth) workers will be put on CPU #0
  185. (resp. CPU #1).
  186. This variable is ignored if the field
  187. starpu_conf::use_explicit_workers_bindid passed to starpu_init() is
  188. set.
  189. </dd>
  190. <dt>STARPU_MAIN_THREAD_CPUID</dt>
  191. <dd>
  192. \anchor STARPU_MAIN_THREAD_CPUID
  193. \addindex __env__STARPU_MAIN_THREAD_CPUID
  194. When defined, this make StarPU bind the thread that calls starpu_initialize() to
  195. the given CPU ID.
  196. </dd>
  197. <dt>STARPU_MPI_THREAD_CPUID</dt>
  198. <dd>
  199. \anchor STARPU_MPI_THREAD_CPUID
  200. \addindex __env__STARPU_MPI_THREAD_CPUID
  201. When defined, this make StarPU bind its MPI thread to the given CPU ID.
  202. </dd>
  203. <dt>STARPU_WORKERS_CUDAID</dt>
  204. <dd>
  205. \anchor STARPU_WORKERS_CUDAID
  206. \addindex __env__STARPU_WORKERS_CUDAID
  207. Similarly to the \ref STARPU_WORKERS_CPUID environment variable, it is
  208. possible to select which CUDA devices should be used by StarPU. On a machine
  209. equipped with 4 GPUs, setting <c>STARPU_WORKERS_CUDAID = "1 3"</c> and
  210. <c>STARPU_NCUDA=2</c> specifies that 2 CUDA workers should be created, and that
  211. they should use CUDA devices #1 and #3 (the logical ordering of the devices is
  212. the one reported by CUDA).
  213. This variable is ignored if the field
  214. starpu_conf::use_explicit_workers_cuda_gpuid passed to starpu_init()
  215. is set.
  216. </dd>
  217. <dt>STARPU_WORKERS_OPENCLID</dt>
  218. <dd>
  219. \anchor STARPU_WORKERS_OPENCLID
  220. \addindex __env__STARPU_WORKERS_OPENCLID
  221. OpenCL equivalent of the \ref STARPU_WORKERS_CUDAID environment variable.
  222. This variable is ignored if the field
  223. starpu_conf::use_explicit_workers_opencl_gpuid passed to starpu_init()
  224. is set.
  225. </dd>
  226. <dt>STARPU_WORKERS_MICID</dt>
  227. <dd>
  228. \anchor STARPU_WORKERS_MICID
  229. \addindex __env__STARPU_WORKERS_MICID
  230. MIC equivalent of the \ref STARPU_WORKERS_CUDAID environment variable.
  231. This variable is ignored if the field
  232. starpu_conf::use_explicit_workers_mic_deviceid passed to starpu_init()
  233. is set.
  234. </dd>
  235. <dt>STARPU_WORKERS_SCCID</dt>
  236. <dd>
  237. \anchor STARPU_WORKERS_SCCID
  238. \addindex __env__STARPU_WORKERS_SCCID
  239. SCC equivalent of the \ref STARPU_WORKERS_CUDAID environment variable.
  240. This variable is ignored if the field
  241. starpu_conf::use_explicit_workers_scc_deviceid passed to starpu_init()
  242. is set.
  243. </dd>
  244. <dt>STARPU_WORKER_TREE</dt>
  245. <dd>
  246. \anchor STARPU_WORKER_TREE
  247. \addindex __env__STARPU_WORKER_TREE
  248. Define to 1 to enable the tree iterator in schedulers.
  249. </dd>
  250. <dt>STARPU_SINGLE_COMBINED_WORKER</dt>
  251. <dd>
  252. \anchor STARPU_SINGLE_COMBINED_WORKER
  253. \addindex __env__STARPU_SINGLE_COMBINED_WORKER
  254. If set, StarPU will create several workers which won't be able to work
  255. concurrently. It will by default create combined workers which size goes from 1
  256. to the total number of CPU workers in the system. \ref STARPU_MIN_WORKERSIZE
  257. and \ref STARPU_MAX_WORKERSIZE can be used to change this default.
  258. </dd>
  259. <dt>STARPU_MIN_WORKERSIZE</dt>
  260. <dd>
  261. \anchor STARPU_MIN_WORKERSIZE
  262. \addindex __env__STARPU_MIN_WORKERSIZE
  263. \ref STARPU_MIN_WORKERSIZE
  264. permits to specify the minimum size of the combined workers (instead of the default 2)
  265. </dd>
  266. <dt>STARPU_MAX_WORKERSIZE</dt>
  267. <dd>
  268. \anchor STARPU_MAX_WORKERSIZE
  269. \addindex __env__STARPU_MAX_WORKERSIZE
  270. \ref STARPU_MAX_WORKERSIZE
  271. permits to specify the minimum size of the combined workers (instead of the
  272. number of CPU workers in the system)
  273. </dd>
  274. <dt>STARPU_SYNTHESIZE_ARITY_COMBINED_WORKER</dt>
  275. <dd>
  276. \anchor STARPU_SYNTHESIZE_ARITY_COMBINED_WORKER
  277. \addindex __env__STARPU_SYNTHESIZE_ARITY_COMBINED_WORKER
  278. Let the user decide how many elements are allowed between combined workers
  279. created from hwloc information. For instance, in the case of sockets with 6
  280. cores without shared L2 caches, if \ref STARPU_SYNTHESIZE_ARITY_COMBINED_WORKER is
  281. set to 6, no combined worker will be synthesized beyond one for the socket
  282. and one per core. If it is set to 3, 3 intermediate combined workers will be
  283. synthesized, to divide the socket cores into 3 chunks of 2 cores. If it set to
  284. 2, 2 intermediate combined workers will be synthesized, to divide the the socket
  285. cores into 2 chunks of 3 cores, and then 3 additional combined workers will be
  286. synthesized, to divide the former synthesized workers into a bunch of 2 cores,
  287. and the remaining core (for which no combined worker is synthesized since there
  288. is already a normal worker for it).
  289. The default, 2, thus makes StarPU tend to building a binary trees of combined
  290. workers.
  291. </dd>
  292. <dt>STARPU_DISABLE_ASYNCHRONOUS_COPY</dt>
  293. <dd>
  294. \anchor STARPU_DISABLE_ASYNCHRONOUS_COPY
  295. \addindex __env__STARPU_DISABLE_ASYNCHRONOUS_COPY
  296. Disable asynchronous copies between CPU and GPU devices.
  297. The AMD implementation of OpenCL is known to
  298. fail when copying data asynchronously. When using this implementation,
  299. it is therefore necessary to disable asynchronous data transfers.
  300. </dd>
  301. <dt>STARPU_DISABLE_ASYNCHRONOUS_CUDA_COPY</dt>
  302. <dd>
  303. \anchor STARPU_DISABLE_ASYNCHRONOUS_CUDA_COPY
  304. \addindex __env__STARPU_DISABLE_ASYNCHRONOUS_CUDA_COPY
  305. Disable asynchronous copies between CPU and CUDA devices.
  306. </dd>
  307. <dt>STARPU_DISABLE_ASYNCHRONOUS_OPENCL_COPY</dt>
  308. <dd>
  309. \anchor STARPU_DISABLE_ASYNCHRONOUS_OPENCL_COPY
  310. \addindex __env__STARPU_DISABLE_ASYNCHRONOUS_OPENCL_COPY
  311. Disable asynchronous copies between CPU and OpenCL devices.
  312. The AMD implementation of OpenCL is known to
  313. fail when copying data asynchronously. When using this implementation,
  314. it is therefore necessary to disable asynchronous data transfers.
  315. </dd>
  316. <dt>STARPU_DISABLE_ASYNCHRONOUS_MIC_COPY</dt>
  317. <dd>
  318. \anchor STARPU_DISABLE_ASYNCHRONOUS_MIC_COPY
  319. \addindex __env__STARPU_DISABLE_ASYNCHRONOUS_MIC_COPY
  320. Disable asynchronous copies between CPU and MIC devices.
  321. </dd>
  322. <dt>STARPU_DISABLE_ASYNCHRONOUS_MPI_MS_COPY</dt>
  323. <dd>
  324. \anchor STARPU_DISABLE_ASYNCHRONOUS_MPI_MS_COPY
  325. \addindex __env__STARPU_DISABLE_ASYNCHRONOUS_MPI_MS_COPY
  326. Disable asynchronous copies between CPU and MPI Slave devices.
  327. </dd>
  328. <dt>STARPU_ENABLE_CUDA_GPU_GPU_DIRECT</dt>
  329. <dd>
  330. \anchor STARPU_ENABLE_CUDA_GPU_GPU_DIRECT
  331. \addindex __env__STARPU_ENABLE_CUDA_GPU_GPU_DIRECT
  332. Enable (1) or Disable (0) direct CUDA transfers from GPU to GPU, without copying
  333. through RAM. The default is Enabled.
  334. This permits to test the performance effect of GPU-Direct.
  335. </dd>
  336. <dt>STARPU_DISABLE_PINNING</dt>
  337. <dd>
  338. \anchor STARPU_DISABLE_PINNING
  339. \addindex __env__STARPU_DISABLE_PINNING
  340. Disable (1) or Enable (0) pinning host memory allocated through starpu_malloc, starpu_memory_pin
  341. and friends. The default is Enabled.
  342. This permits to test the performance effect of memory pinning.
  343. </dd>
  344. <dt>STARPU_MIC_SINK_PROGRAM_NAME</dt>
  345. <dd>
  346. \anchor STARPU_MIC_SINK_PROGRAM_NAME
  347. \addindex __env__STARPU_MIC_SINK_PROGRAM_NAME
  348. todo
  349. </dd>
  350. <dt>STARPU_MIC_SINK_PROGRAM_PATH</dt>
  351. <dd>
  352. \anchor STARPU_MIC_SINK_PROGRAM_PATH
  353. \addindex __env__STARPU_MIC_SINK_PROGRAM_PATH
  354. todo
  355. </dd>
  356. <dt>STARPU_MIC_PROGRAM_PATH</dt>
  357. <dd>
  358. \anchor STARPU_MIC_PROGRAM_PATH
  359. \addindex __env__STARPU_MIC_PROGRAM_PATH
  360. todo
  361. </dd>
  362. </dl>
  363. \section ConfiguringTheSchedulingEngine Configuring The Scheduling Engine
  364. <dl>
  365. <dt>STARPU_SCHED</dt>
  366. <dd>
  367. \anchor STARPU_SCHED
  368. \addindex __env__STARPU_SCHED
  369. Choose between the different scheduling policies proposed by StarPU: work
  370. random, stealing, greedy, with performance models, etc.
  371. Use <c>STARPU_SCHED=help</c> to get the list of available schedulers.
  372. </dd>
  373. <dt>STARPU_MIN_PRIO</dt>
  374. <dd>
  375. \anchor STARPU_MIN_PRIO_env
  376. \addindex __env__STARPU_MIN_PRIO
  377. Set the mininum priority used by priorities-aware schedulers.
  378. </dd>
  379. <dt>STARPU_MAX_PRIO</dt>
  380. <dd>
  381. \anchor STARPU_MAX_PRIO_env
  382. \addindex __env__STARPU_MAX_PRIO
  383. Set the maximum priority used by priorities-aware schedulers.
  384. </dd>
  385. <dt>STARPU_CALIBRATE</dt>
  386. <dd>
  387. \anchor STARPU_CALIBRATE
  388. \addindex __env__STARPU_CALIBRATE
  389. If this variable is set to 1, the performance models are calibrated during
  390. the execution. If it is set to 2, the previous values are dropped to restart
  391. calibration from scratch. Setting this variable to 0 disable calibration, this
  392. is the default behaviour.
  393. Note: this currently only applies to <c>dm</c> and <c>dmda</c> scheduling policies.
  394. </dd>
  395. <dt>STARPU_CALIBRATE_MINIMUM</dt>
  396. <dd>
  397. \anchor STARPU_CALIBRATE_MINIMUM
  398. \addindex __env__STARPU_CALIBRATE_MINIMUM
  399. This defines the minimum number of calibration measurements that will be made
  400. before considering that the performance model is calibrated. The default value is 10.
  401. </dd>
  402. <dt>STARPU_BUS_CALIBRATE</dt>
  403. <dd>
  404. \anchor STARPU_BUS_CALIBRATE
  405. \addindex __env__STARPU_BUS_CALIBRATE
  406. If this variable is set to 1, the bus is recalibrated during intialization.
  407. </dd>
  408. <dt>STARPU_PREFETCH</dt>
  409. <dd>
  410. \anchor STARPU_PREFETCH
  411. \addindex __env__STARPU_PREFETCH
  412. This variable indicates whether data prefetching should be enabled (0 means
  413. that it is disabled). If prefetching is enabled, when a task is scheduled to be
  414. executed e.g. on a GPU, StarPU will request an asynchronous transfer in
  415. advance, so that data is already present on the GPU when the task starts. As a
  416. result, computation and data transfers are overlapped.
  417. Note that prefetching is enabled by default in StarPU.
  418. </dd>
  419. <dt>STARPU_SCHED_ALPHA</dt>
  420. <dd>
  421. \anchor STARPU_SCHED_ALPHA
  422. \addindex __env__STARPU_SCHED_ALPHA
  423. To estimate the cost of a task StarPU takes into account the estimated
  424. computation time (obtained thanks to performance models). The alpha factor is
  425. the coefficient to be applied to it before adding it to the communication part.
  426. </dd>
  427. <dt>STARPU_SCHED_BETA</dt>
  428. <dd>
  429. \anchor STARPU_SCHED_BETA
  430. \addindex __env__STARPU_SCHED_BETA
  431. To estimate the cost of a task StarPU takes into account the estimated
  432. data transfer time (obtained thanks to performance models). The beta factor is
  433. the coefficient to be applied to it before adding it to the computation part.
  434. </dd>
  435. <dt>STARPU_SCHED_GAMMA</dt>
  436. <dd>
  437. \anchor STARPU_SCHED_GAMMA
  438. \addindex __env__STARPU_SCHED_GAMMA
  439. Define the execution time penalty of a joule (\ref Energy-basedScheduling).
  440. </dd>
  441. <dt>STARPU_IDLE_POWER</dt>
  442. <dd>
  443. \anchor STARPU_IDLE_POWER
  444. \addindex __env__STARPU_IDLE_POWER
  445. Define the idle power of the machine (\ref Energy-basedScheduling).
  446. </dd>
  447. <dt>STARPU_PROFILING</dt>
  448. <dd>
  449. \anchor STARPU_PROFILING
  450. \addindex __env__STARPU_PROFILING
  451. Enable on-line performance monitoring (\ref EnablingOn-linePerformanceMonitoring).
  452. </dd>
  453. </dl>
  454. \section Extensions Extensions
  455. <dl>
  456. <dt>SOCL_OCL_LIB_OPENCL</dt>
  457. <dd>
  458. \anchor SOCL_OCL_LIB_OPENCL
  459. \addindex __env__SOCL_OCL_LIB_OPENCL
  460. THE SOCL test suite is only run when the environment variable
  461. \ref SOCL_OCL_LIB_OPENCL is defined. It should contain the location
  462. of the file <c>libOpenCL.so</c> of the OCL ICD implementation.
  463. </dd>
  464. <dt>OCL_ICD_VENDORS</dt>
  465. <dd>
  466. \anchor OCL_ICD_VENDORS
  467. \addindex __env__OCL_ICD_VENDORS
  468. When using SOCL with OpenCL ICD
  469. (https://forge.imag.fr/projects/ocl-icd/), this variable may be used
  470. to point to the directory where ICD files are installed. The default
  471. directory is <c>/etc/OpenCL/vendors</c>. StarPU installs ICD
  472. files in the directory <c>$prefix/share/starpu/opencl/vendors</c>.
  473. </dd>
  474. <dt>STARPU_COMM_STATS</dt>
  475. <dd>
  476. \anchor STARPU_COMM_STATS
  477. \addindex __env__STARPU_COMM_STATS
  478. Communication statistics for starpumpi (\ref MPISupport)
  479. will be enabled when the environment variable \ref STARPU_COMM_STATS
  480. is defined to an value other than 0.
  481. </dd>
  482. <dt>STARPU_MPI_CACHE</dt>
  483. <dd>
  484. \anchor STARPU_MPI_CACHE
  485. \addindex __env__STARPU_MPI_CACHE
  486. Communication cache for starpumpi (\ref MPISupport) will be
  487. disabled when the environment variable \ref STARPU_MPI_CACHE is set
  488. to 0. It is enabled by default or for any other values of the variable
  489. \ref STARPU_MPI_CACHE.
  490. </dd>
  491. <dt>STARPU_MPI_COMM</dt>
  492. <dd>
  493. \anchor STARPU_MPI_COMM
  494. \addindex __env__STARPU_MPI_COMM
  495. Communication trace for starpumpi (\ref MPISupport) will be
  496. enabled when the environment variable \ref STARPU_MPI_COMM is set
  497. to 1, and StarPU has been configured with the option
  498. \ref enable-verbose "--enable-verbose".
  499. </dd>
  500. <dt>STARPU_MPI_CACHE_STATS</dt>
  501. <dd>
  502. \anchor STARPU_MPI_CACHE_STATS
  503. \addindex __env__STARPU_MPI_CACHE_STATS
  504. When set to 1, statistics are enabled for the communication cache (\ref MPISupport). For now,
  505. it prints messages on the standard output when data are added or removed from the received
  506. communication cache.
  507. </dd>
  508. <dt>STARPU_MPI_PRIORITIES</dt>
  509. <dd>
  510. \anchor STARPU_MPI_PRIORITIES
  511. \addindex __env__STARPU_MPI_PRIORITIES
  512. When set to 0, the use of priorities to order MPI communications is disabled
  513. (\ref MPISupport).
  514. </dd>
  515. <dt>STARPU_MPI_FAKE_SIZE</dt>
  516. <dd>
  517. \anchor STARPU_MPI_FAKE_SIZE
  518. \addindex __env__STARPU_MPI_FAKE_SIZE
  519. Setting to a number makes StarPU believe that there are as many MPI nodes, even
  520. if it was run on only one MPI node. This allows e.g. to simulate the execution
  521. of one of the nodes of a big cluster without actually running the rest.
  522. It of course does not provide computation results and timing.
  523. </dd>
  524. <dt>STARPU_MPI_FAKE_RANK</dt>
  525. <dd>
  526. \anchor STARPU_MPI_FAKE_RANK
  527. \addindex __env__STARPU_MPI_FAKE_RANK
  528. Setting to a number makes StarPU believe that it runs the given MPI node, even
  529. if it was run on only one MPI node. This allows e.g. to simulate the execution
  530. of one of the nodes of a big cluster without actually running the rest.
  531. It of course does not provide computation results and timing.
  532. </dd>
  533. <dt>STARPU_SIMGRID_CUDA_MALLOC_COST</dt>
  534. <dd>
  535. \anchor STARPU_SIMGRID_CUDA_MALLOC_COST
  536. \addindex __env__STARPU_SIMGRID_CUDA_MALLOC_COST
  537. When set to 1 (which is the default), CUDA malloc costs are taken into account
  538. in simgrid mode.
  539. </dd>
  540. <dt>STARPU_SIMGRID_CUDA_QUEUE_COST</dt>
  541. <dd>
  542. \anchor STARPU_SIMGRID_CUDA_QUEUE_COST
  543. \addindex __env__STARPU_SIMGRID_CUDA_QUEUE_COST
  544. When set to 1 (which is the default), CUDA task and transfer queueing costs are
  545. taken into account in simgrid mode.
  546. </dd>
  547. <dt>STARPU_PCI_FLAT</dt>
  548. <dd>
  549. \anchor STARPU_PCI_FLAT
  550. \addindex __env__STARPU_PCI_FLAT
  551. When unset or set to 0, the platform file created for simgrid will
  552. contain PCI bandwidths and routes.
  553. </dd>
  554. <dt>STARPU_SIMGRID_QUEUE_MALLOC_COST</dt>
  555. <dd>
  556. \anchor STARPU_SIMGRID_QUEUE_MALLOC_COST
  557. \addindex __env__STARPU_SIMGRID_QUEUE_MALLOC_COST
  558. When unset or set to 1, simulate within simgrid the GPU transfer queueing.
  559. </dd>
  560. <dt>STARPU_MALLOC_SIMULATION_FOLD</dt>
  561. <dd>
  562. \anchor STARPU_MALLOC_SIMULATION_FOLD
  563. \addindex __env__STARPU_MALLOC_SIMULATION_FOLD
  564. This defines the size of the file used for folding virtual allocation, in
  565. MiB. The default is 1, thus allowing 64GiB virtual memory when Linux's
  566. <c>sysctl vm.max_map_count</c> value is the default 65535.
  567. </dd>
  568. <dt>STARPU_SIMGRID_TASK_SUBMIT_COST</dt>
  569. <dd>
  570. \anchor STARPU_SIMGRID_TASK_SUBMIT_COST
  571. \addindex __env__STARPU_SIMGRID_TASK_SUBMIT_COST
  572. When set to 1 (which is the default), task submission costs are taken into
  573. account in simgrid mode. This provides more accurate simgrid predictions,
  574. especially for the beginning of the execution.
  575. </dd>
  576. <dt>STARPU_SIMGRID_FETCHING_INPUT_COST</dt>
  577. <dd>
  578. \anchor STARPU_SIMGRID_FETCHING_INPUT_COST
  579. \addindex __env__STARPU_SIMGRID_FETCHING_INPUT_COST
  580. When set to 1 (which is the default), fetching input costs are taken into
  581. account in simgrid mode. This provides more accurate simgrid predictions,
  582. especially regarding data transfers.
  583. </dd>
  584. <dt>STARPU_SIMGRID_SCHED_COST</dt>
  585. <dd>
  586. \anchor STARPU_SIMGRID_SCHED_COST
  587. \addindex __env__STARPU_SIMGRID_SCHED_COST
  588. When set to 1 (0 is the default), scheduling costs are taken into
  589. account in simgrid mode. This provides more accurate simgrid predictions,
  590. and allows studying scheduling overhead of the runtime system. However,
  591. it also makes simulation non-deterministic.
  592. </dd>
  593. </dl>
  594. \section MiscellaneousAndDebug Miscellaneous And Debug
  595. <dl>
  596. <dt>STARPU_HOME</dt>
  597. <dd>
  598. \anchor STARPU_HOME
  599. \addindex __env__STARPU_HOME
  600. This specifies the main directory in which StarPU stores its
  601. configuration files. The default is <c>$HOME</c> on Unix environments,
  602. and <c>$USERPROFILE</c> on Windows environments.
  603. </dd>
  604. <dt>STARPU_PATH</dt>
  605. <dd>
  606. \anchor STARPU_PATH
  607. \addindex __env__STARPU_PATH
  608. Only used on Windows environments.
  609. This specifies the main directory in which StarPU is installed
  610. (\ref RunningABasicStarPUApplicationOnMicrosoft)
  611. </dd>
  612. <dt>STARPU_PERF_MODEL_DIR</dt>
  613. <dd>
  614. \anchor STARPU_PERF_MODEL_DIR
  615. \addindex __env__STARPU_PERF_MODEL_DIR
  616. This specifies the main directory in which StarPU stores its
  617. performance model files. The default is <c>$STARPU_HOME/.starpu/sampling</c>.
  618. </dd>
  619. <dt>STARPU_PERF_MODEL_HOMOGENEOUS_CPU</dt>
  620. <dd>
  621. \anchor STARPU_PERF_MODEL_HOMOGENEOUS_CPU
  622. \addindex __env__STARPU_PERF_MODEL_HOMOGENEOUS_CPU
  623. When this is set to 0, StarPU will assume that CPU devices do not have the same
  624. performance, and thus use different performance models for them, thus making
  625. kernel calibration much longer, since measurements have to be made for each CPU
  626. core.
  627. </dd>
  628. <dt>STARPU_PERF_MODEL_HOMOGENEOUS_CUDA</dt>
  629. <dd>
  630. \anchor STARPU_PERF_MODEL_HOMOGENEOUS_CUDA
  631. \addindex __env__STARPU_PERF_MODEL_HOMOGENEOUS_CUDA
  632. When this is set to 1, StarPU will assume that all CUDA devices have the same
  633. performance, and thus share performance models for them, thus allowing kernel
  634. calibration to be much faster, since measurements only have to be once for all
  635. CUDA GPUs.
  636. </dd>
  637. <dt>STARPU_PERF_MODEL_HOMOGENEOUS_OPENCL</dt>
  638. <dd>
  639. \anchor STARPU_PERF_MODEL_HOMOGENEOUS_OPENCL
  640. \addindex __env__STARPU_PERF_MODEL_HOMOGENEOUS_OPENCL
  641. When this is set to 1, StarPU will assume that all OPENCL devices have the same
  642. performance, and thus share performance models for them, thus allowing kernel
  643. calibration to be much faster, since measurements only have to be once for all
  644. OPENCL GPUs.
  645. </dd>
  646. <dt>STARPU_PERF_MODEL_HOMOGENEOUS_MIC</dt>
  647. <dd>
  648. \anchor STARPU_PERF_MODEL_HOMOGENEOUS_MIC
  649. \addindex __env__STARPU_PERF_MODEL_HOMOGENEOUS_MIC
  650. When this is set to 1, StarPU will assume that all MIC devices have the same
  651. performance, and thus share performance models for them, thus allowing kernel
  652. calibration to be much faster, since measurements only have to be once for all
  653. MIC GPUs.
  654. </dd>
  655. <dt>STARPU_PERF_MODEL_HOMOGENEOUS_MPI_MS</dt>
  656. <dd>
  657. \anchor STARPU_PERF_MODEL_HOMOGENEOUS_MPI_MS
  658. \addindex __env__STARPU_PERF_MODEL_HOMOGENEOUS_MPI_MS
  659. When this is set to 1, StarPU will assume that all MPI Slave devices have the same
  660. performance, and thus share performance models for them, thus allowing kernel
  661. calibration to be much faster, since measurements only have to be once for all
  662. MPI Slaves.
  663. </dd>
  664. <dt>STARPU_PERF_MODEL_HOMOGENEOUS_SCC</dt>
  665. <dd>
  666. \anchor STARPU_PERF_MODEL_HOMOGENEOUS_SCC
  667. \addindex __env__STARPU_PERF_MODEL_HOMOGENEOUS_SCC
  668. When this is set to 1, StarPU will assume that all SCC devices have the same
  669. performance, and thus share performance models for them, thus allowing kernel
  670. calibration to be much faster, since measurements only have to be once for all
  671. SCC GPUs.
  672. </dd>
  673. <dt>STARPU_HOSTNAME</dt>
  674. <dd>
  675. \anchor STARPU_HOSTNAME
  676. \addindex __env__STARPU_HOSTNAME
  677. When set, force the hostname to be used when dealing performance model
  678. files. Models are indexed by machine name. When running for example on
  679. a homogenenous cluster, it is possible to share the models between
  680. machines by setting <c>export STARPU_HOSTNAME=some_global_name</c>.
  681. </dd>
  682. <dt>STARPU_OPENCL_PROGRAM_DIR</dt>
  683. <dd>
  684. \anchor STARPU_OPENCL_PROGRAM_DIR
  685. \addindex __env__STARPU_OPENCL_PROGRAM_DIR
  686. This specifies the directory where the OpenCL codelet source files are
  687. located. The function starpu_opencl_load_program_source() looks
  688. for the codelet in the current directory, in the directory specified
  689. by the environment variable \ref STARPU_OPENCL_PROGRAM_DIR, in the
  690. directory <c>share/starpu/opencl</c> of the installation directory of
  691. StarPU, and finally in the source directory of StarPU.
  692. </dd>
  693. <dt>STARPU_SILENT</dt>
  694. <dd>
  695. \anchor STARPU_SILENT
  696. \addindex __env__STARPU_SILENT
  697. This variable allows to disable verbose mode at runtime when StarPU
  698. has been configured with the option \ref enable-verbose "--enable-verbose". It also
  699. disables the display of StarPU information and warning messages.
  700. </dd>
  701. <dt>STARPU_LOGFILENAME</dt>
  702. <dd>
  703. \anchor STARPU_LOGFILENAME
  704. \addindex __env__STARPU_LOGFILENAME
  705. This variable specifies in which file the debugging output should be saved to.
  706. </dd>
  707. <dt>STARPU_FXT_PREFIX</dt>
  708. <dd>
  709. \anchor STARPU_FXT_PREFIX
  710. \addindex __env__STARPU_FXT_PREFIX
  711. This variable specifies in which directory to save the trace generated if FxT is enabled. It needs to have a trailing '/' character.
  712. </dd>
  713. <dt>STARPU_FXT_TRACE</dt>
  714. <dd>
  715. \anchor STARPU_FXT_TRACE
  716. \addindex __env__STARPU_FXT_TRACE
  717. This variable specifies whether to generate (1) or not (0) the FxT trace in /tmp/prof_file_XXX_YYY . The default is 1 (generate it)
  718. </dd>
  719. <dt>STARPU_LIMIT_CUDA_devid_MEM</dt>
  720. <dd>
  721. \anchor STARPU_LIMIT_CUDA_devid_MEM
  722. \addindex __env__STARPU_LIMIT_CUDA_devid_MEM
  723. This variable specifies the maximum number of megabytes that should be
  724. available to the application on the CUDA device with the identifier
  725. <c>devid</c>. This variable is intended to be used for experimental
  726. purposes as it emulates devices that have a limited amount of memory.
  727. When defined, the variable overwrites the value of the variable
  728. \ref STARPU_LIMIT_CUDA_MEM.
  729. </dd>
  730. <dt>STARPU_LIMIT_CUDA_MEM</dt>
  731. <dd>
  732. \anchor STARPU_LIMIT_CUDA_MEM
  733. \addindex __env__STARPU_LIMIT_CUDA_MEM
  734. This variable specifies the maximum number of megabytes that should be
  735. available to the application on each CUDA devices. This variable is
  736. intended to be used for experimental purposes as it emulates devices
  737. that have a limited amount of memory.
  738. </dd>
  739. <dt>STARPU_LIMIT_OPENCL_devid_MEM</dt>
  740. <dd>
  741. \anchor STARPU_LIMIT_OPENCL_devid_MEM
  742. \addindex __env__STARPU_LIMIT_OPENCL_devid_MEM
  743. This variable specifies the maximum number of megabytes that should be
  744. available to the application on the OpenCL device with the identifier
  745. <c>devid</c>. This variable is intended to be used for experimental
  746. purposes as it emulates devices that have a limited amount of memory.
  747. When defined, the variable overwrites the value of the variable
  748. \ref STARPU_LIMIT_OPENCL_MEM.
  749. </dd>
  750. <dt>STARPU_LIMIT_OPENCL_MEM</dt>
  751. <dd>
  752. \anchor STARPU_LIMIT_OPENCL_MEM
  753. \addindex __env__STARPU_LIMIT_OPENCL_MEM
  754. This variable specifies the maximum number of megabytes that should be
  755. available to the application on each OpenCL devices. This variable is
  756. intended to be used for experimental purposes as it emulates devices
  757. that have a limited amount of memory.
  758. </dd>
  759. <dt>STARPU_LIMIT_CPU_MEM</dt>
  760. <dd>
  761. \anchor STARPU_LIMIT_CPU_MEM
  762. \addindex __env__STARPU_LIMIT_CPU_MEM
  763. This variable specifies the maximum number of megabytes that should be
  764. available to the application in the main CPU memory. Setting it enables allocation
  765. cache in main memory. Setting it to zero lets StarPU overflow memory.
  766. </dd>
  767. <dt>STARPU_LIMIT_CPU_NUMA_devid_MEM</dt>
  768. <dd>
  769. \anchor STARPU_LIMIT_CPU_NUMA_devid_MEM
  770. \addindex __env__STARPU_LIMIT_CPU_NUMA_devid_MEM
  771. This variable specifies the maximum number of megabytes that should be
  772. available to the application on the NUMA node with the OS identifier <c>devid</c>.
  773. </dd>
  774. <dt>STARPU_MINIMUM_AVAILABLE_MEM</dt>
  775. <dd>
  776. \anchor STARPU_MINIMUM_AVAILABLE_MEM
  777. \addindex __env__STARPU_MINIMUM_AVAILABLE_MEM
  778. This specifies the minimum percentage of memory that should be available in GPUs
  779. (or in main memory, when using out of core), below which a reclaiming pass is
  780. performed. The default is 0%.
  781. </dd>
  782. <dt>STARPU_TARGET_AVAILABLE_MEM</dt>
  783. <dd>
  784. \anchor STARPU_TARGET_AVAILABLE_MEM
  785. \addindex __env__STARPU_TARGET_AVAILABLE_MEM
  786. This specifies the target percentage of memory that should be reached in
  787. GPUs (or in main memory, when using out of core), when performing a periodic
  788. reclaiming pass. The default is 0%.
  789. </dd>
  790. <dt>STARPU_MINIMUM_CLEAN_BUFFERS</dt>
  791. <dd>
  792. \anchor STARPU_MINIMUM_CLEAN_BUFFERS
  793. \addindex __env__STARPU_MINIMUM_CLEAN_BUFFERS
  794. This specifies the minimum percentage of number of buffers that should be clean in GPUs
  795. (or in main memory, when using out of core), below which asynchronous writebacks will be
  796. issued. The default is 5%.
  797. </dd>
  798. <dt>STARPU_TARGET_CLEAN_BUFFERS</dt>
  799. <dd>
  800. \anchor STARPU_TARGET_CLEAN_BUFFERS
  801. \addindex __env__STARPU_TARGET_CLEAN_BUFFERS
  802. This specifies the target percentage of number of buffers that should be reached in
  803. GPUs (or in main memory, when using out of core), when performing an asynchronous
  804. writeback pass. The default is 10%.
  805. </dd>
  806. <dt>STARPU_DIDUSE_BARRIER</dt>
  807. <dd>
  808. \anchor STARPU_DIDUSE_BARRIER
  809. \addindex __env__STARPU_DIDUSE_BARRIER
  810. When set to 1, StarPU will never evict a piece of data if it has not been used
  811. by at least one task. This avoids odd behaviors under high memory pressure, but
  812. can lead to deadlocks, so is to be considered experimental only.
  813. </dd>
  814. <dt>STARPU_DISK_SWAP</dt>
  815. <dd>
  816. \anchor STARPU_DISK_SWAP
  817. \addindex __env__STARPU_DISK_SWAP
  818. This specifies a path where StarPU can push data when the main memory is getting
  819. full.
  820. </dd>
  821. <dt>STARPU_DISK_SWAP_BACKEND</dt>
  822. <dd>
  823. \anchor STARPU_DISK_SWAP_BACKEND
  824. \addindex __env__STARPU_DISK_SWAP_BACKEND
  825. This specifies then backend to be used by StarPU to push data when the main
  826. memory is getting full. The default is unistd (i.e. using read/write functions),
  827. other values are stdio (i.e. using fread/fwrite), unistd_o_direct (i.e. using
  828. read/write with O_DIRECT), leveldb (i.e. using a leveldb database), and hdf5
  829. (i.e. using HDF5 library).
  830. </dd>
  831. <dt>STARPU_DISK_SWAP_SIZE</dt>
  832. <dd>
  833. \anchor STARPU_DISK_SWAP_SIZE
  834. \addindex __env__STARPU_DISK_SWAP_SIZE
  835. This specifies then maximum size in MiB to be used by StarPU to push data when the main
  836. memory is getting full. The default is unlimited.
  837. </dd>
  838. <dt>STARPU_LIMIT_MAX_SUBMITTED_TASKS</dt>
  839. <dd>
  840. \anchor STARPU_LIMIT_MAX_SUBMITTED_TASKS
  841. \addindex __env__STARPU_LIMIT_MAX_SUBMITTED_TASKS
  842. This variable allows the user to control the task submission flow by specifying
  843. to StarPU a maximum number of submitted tasks allowed at a given time, i.e. when
  844. this limit is reached task submission becomes blocking until enough tasks have
  845. completed, specified by \ref STARPU_LIMIT_MIN_SUBMITTED_TASKS.
  846. Setting it enables allocation cache buffer reuse in main memory.
  847. </dd>
  848. <dt>STARPU_LIMIT_MIN_SUBMITTED_TASKS</dt>
  849. <dd>
  850. \anchor STARPU_LIMIT_MIN_SUBMITTED_TASKS
  851. \addindex __env__STARPU_LIMIT_MIN_SUBMITTED_TASKS
  852. This variable allows the user to control the task submission flow by specifying
  853. to StarPU a submitted task threshold to wait before unblocking task submission. This
  854. variable has to be used in conjunction with \ref STARPU_LIMIT_MAX_SUBMITTED_TASKS
  855. which puts the task submission thread to
  856. sleep. Setting it enables allocation cache buffer reuse in main memory.
  857. </dd>
  858. <dt>STARPU_TRACE_BUFFER_SIZE</dt>
  859. <dd>
  860. \anchor STARPU_TRACE_BUFFER_SIZE
  861. \addindex __env__STARPU_TRACE_BUFFER_SIZE
  862. This sets the buffer size for recording trace events in MiB. Setting it to a big
  863. size allows to avoid pauses in the trace while it is recorded on the disk. This
  864. however also consumes memory, of course. The default value is 64.
  865. </dd>
  866. <dt>STARPU_GENERATE_TRACE</dt>
  867. <dd>
  868. \anchor STARPU_GENERATE_TRACE
  869. \addindex __env__STARPU_GENERATE_TRACE
  870. When set to <c>1</c>, this variable indicates that StarPU should automatically
  871. generate a Paje trace when starpu_shutdown() is called.
  872. </dd>
  873. <dt>STARPU_GENERATE_TRACE_OPTIONS</dt>
  874. <dd>
  875. \anchor STARPU_GENERATE_TRACE_OPTIONS
  876. \addindex __env__STARPU_GENERATE_TRACE_OPTIONS
  877. When the variable \ref STARPU_GENERATE_TRACE is set to <c>1</c> to
  878. generate a Paje trace, this variable can be set to specify options (see
  879. <c>starpu_fxt_tool --help</c>).
  880. </dd>
  881. <dt>STARPU_ENABLE_STATS</dt>
  882. <dd>
  883. \anchor STARPU_ENABLE_STATS
  884. \addindex __env__STARPU_ENABLE_STATS
  885. When defined, enable gathering various data statistics (\ref DataStatistics).
  886. </dd>
  887. <dt>STARPU_MEMORY_STATS</dt>
  888. <dd>
  889. \anchor STARPU_MEMORY_STATS
  890. \addindex __env__STARPU_MEMORY_STATS
  891. When set to 0, disable the display of memory statistics on data which
  892. have not been unregistered at the end of the execution (\ref MemoryFeedback).
  893. </dd>
  894. <dt>STARPU_MAX_MEMORY_USE</dt>
  895. <dd>
  896. \anchor STARPU_MAX_MEMORY_USE
  897. \addindex __env__STARPU_MAX_MEMORY_USE
  898. When set to 1, display at the end of the execution the maximum memory used by
  899. StarPU for internal data structures during execution.
  900. </dd>
  901. <dt>STARPU_BUS_STATS</dt>
  902. <dd>
  903. \anchor STARPU_BUS_STATS
  904. \addindex __env__STARPU_BUS_STATS
  905. When defined, statistics about data transfers will be displayed when calling
  906. starpu_shutdown() (\ref Profiling).
  907. </dd>
  908. <dt>STARPU_WORKER_STATS</dt>
  909. <dd>
  910. \anchor STARPU_WORKER_STATS
  911. \addindex __env__STARPU_WORKER_STATS
  912. When defined, statistics about the workers will be displayed when calling
  913. starpu_shutdown() (\ref Profiling). When combined with the
  914. environment variable \ref STARPU_PROFILING, it displays the energy
  915. consumption (\ref Energy-basedScheduling).
  916. </dd>
  917. <dt>STARPU_STATS</dt>
  918. <dd>
  919. \anchor STARPU_STATS
  920. \addindex __env__STARPU_STATS
  921. When set to 0, data statistics will not be displayed at the
  922. end of the execution of an application (\ref DataStatistics).
  923. </dd>
  924. <dt>STARPU_WATCHDOG_TIMEOUT</dt>
  925. <dd>
  926. \anchor STARPU_WATCHDOG_TIMEOUT
  927. \addindex __env__STARPU_WATCHDOG_TIMEOUT
  928. When set to a value other than 0, allows to make StarPU print an error
  929. message whenever StarPU does not terminate any task for the given time (in µs),
  930. but lets the application continue normally. Should
  931. be used in combination with \ref STARPU_WATCHDOG_CRASH
  932. (see \ref DetectionStuckConditions).
  933. </dd>
  934. <dt>STARPU_WATCHDOG_CRASH</dt>
  935. <dd>
  936. \anchor STARPU_WATCHDOG_CRASH
  937. \addindex __env__STARPU_WATCHDOG_CRASH
  938. When set to a value other than 0, it triggers a crash when the watch
  939. dog is reached, thus allowing to catch the situation in gdb, etc
  940. (see \ref DetectionStuckConditions)
  941. </dd>
  942. <dt>STARPU_WATCHDOG_DELAY</dt>
  943. <dd>
  944. \anchor STARPU_WATCHDOG_DELAY
  945. \addindex __env__STARPU_WATCHDOG_DELAY
  946. This delays the activation of the watchdog by the given time (in µs). This can
  947. be convenient for letting the application initialize data etc. before starting
  948. to look for idle time.
  949. </dd>
  950. <dt>STARPU_TASK_BREAK_ON_PUSH</dt>
  951. <dd>
  952. \anchor STARPU_TASK_BREAK_ON_PUSH
  953. \addindex __env__STARPU_TASK_BREAK_ON_PUSH
  954. When this variable contains a job id, StarPU will raise SIGTRAP when the task
  955. with that job id is being pushed to the scheduler, which will be nicely catched by debuggers
  956. (see \ref DebuggingScheduling)
  957. </dd>
  958. <dt>STARPU_TASK_BREAK_ON_SCHED</dt>
  959. <dd>
  960. \anchor STARPU_TASK_BREAK_ON_SCHED
  961. \addindex __env__STARPU_TASK_BREAK_ON_SCHED
  962. When this variable contains a job id, StarPU will raise SIGTRAP when the task
  963. with that job id is being scheduled by the scheduler (at a scheduler-specific
  964. point), which will be nicely catched by debuggers.
  965. This only works for schedulers which have such a scheduling point defined
  966. (see \ref DebuggingScheduling)
  967. </dd>
  968. <dt>STARPU_TASK_BREAK_ON_POP</dt>
  969. <dd>
  970. \anchor STARPU_TASK_BREAK_ON_POP
  971. \addindex __env__STARPU_TASK_BREAK_ON_POP
  972. When this variable contains a job id, StarPU will raise SIGTRAP when the task
  973. with that job id is being popped from the scheduler, which will be nicely catched by debuggers
  974. (see \ref DebuggingScheduling)
  975. </dd>
  976. <dt>STARPU_TASK_BREAK_ON_EXEC</dt>
  977. <dd>
  978. \anchor STARPU_TASK_BREAK_ON_EXEC
  979. \addindex __env__STARPU_TASK_BREAK_ON_EXEC
  980. When this variable contains a job id, StarPU will raise SIGTRAP when the task
  981. with that job id is being executed, which will be nicely catched by debuggers
  982. (see \ref DebuggingScheduling)
  983. </dd>
  984. <dt>STARPU_DISABLE_KERNELS</dt>
  985. <dd>
  986. \anchor STARPU_DISABLE_KERNELS
  987. \addindex __env__STARPU_DISABLE_KERNELS
  988. When set to a value other than 1, it disables actually calling the kernel
  989. functions, thus allowing to quickly check that the task scheme is working
  990. properly, without performing the actual application-provided computation.
  991. </dd>
  992. <dt>STARPU_HISTORY_MAX_ERROR</dt>
  993. <dd>
  994. \anchor STARPU_HISTORY_MAX_ERROR
  995. \addindex __env__STARPU_HISTORY_MAX_ERROR
  996. History-based performance models will drop measurements which are really far
  997. froom the measured average. This specifies the allowed variation. The default is
  998. 50 (%), i.e. the measurement is allowed to be x1.5 faster or /1.5 slower than the
  999. average.
  1000. </dd>
  1001. <dt>STARPU_RAND_SEED</dt>
  1002. <dd>
  1003. \anchor STARPU_RAND_SEED
  1004. \addindex __env__STARPU_RAND_SEED
  1005. The random scheduler and some examples use random numbers for their own
  1006. working. Depending on the examples, the seed is by default juste always 0 or
  1007. the current time() (unless simgrid mode is enabled, in which case it is always
  1008. 0). \ref STARPU_RAND_SEED allows to set the seed to a specific value.
  1009. </dd>
  1010. <dt>STARPU_IDLE_TIME</dt>
  1011. <dd>
  1012. \anchor STARPU_IDLE_TIME
  1013. \addindex __env__STARPU_IDLE_TIME
  1014. When set to a value being a valid filename, a corresponding file
  1015. will be created when shutting down StarPU. The file will contain the
  1016. sum of all the workers' idle time.
  1017. </dd>
  1018. <dt>STARPU_GLOBAL_ARBITER</dt>
  1019. <dd>
  1020. \anchor STARPU_GLOBAL_ARBITER
  1021. \addindex __env__STARPU_GLOBAL_ARBITER
  1022. When set to a positive value, StarPU will create a arbiter, which
  1023. implements an advanced but centralized management of concurrent data
  1024. accesses (see \ref ConcurrentDataAccess).
  1025. </dd>
  1026. <dt>STARPU_USE_NUMA</dt>
  1027. <dd>
  1028. \anchor STARPU_USE_NUMA
  1029. \addindex __env__STARPU_USE_NUMA
  1030. When defined, NUMA nodes are taking into account by StarPU. Otherwise, memory
  1031. is considered as only one node. This is experimental for now.
  1032. When enabled, STARPU_MAIN_MEMORY is a pointer to the NUMA node associated to the
  1033. first CPU worker if it exists, the NUMA node associated to the first GPU discovered otherwise.
  1034. If StarPU doesn't find any NUMA node after these step, STARPU_MAIN_MEMORY is the first NUMA node
  1035. discovered by StarPU.
  1036. </dd>
  1037. <dt>STARPU_IDLE_FILE</dt>
  1038. <dd>
  1039. \anchor STARPU_IDLE_FILE
  1040. \addindex __env__STARPU_IDLE_FILE
  1041. If the environment variable STARPU_IDLE_FILE is defined, a file named after its contents will be created at the end of the execution.
  1042. The file will contain the sum of the idle times of all the workers.
  1043. </dd>
  1044. </dl>
  1045. \section ConfiguringTheHypervisor Configuring The Hypervisor
  1046. <dl>
  1047. <dt>SC_HYPERVISOR_POLICY</dt>
  1048. <dd>
  1049. \anchor SC_HYPERVISOR_POLICY
  1050. \addindex __env__SC_HYPERVISOR_POLICY
  1051. Choose between the different resizing policies proposed by StarPU for the hypervisor:
  1052. idle, app_driven, feft_lp, teft_lp; ispeed_lp, throughput_lp etc.
  1053. Use <c>SC_HYPERVISOR_POLICY=help</c> to get the list of available policies for the hypervisor
  1054. </dd>
  1055. <dt>SC_HYPERVISOR_TRIGGER_RESIZE</dt>
  1056. <dd>
  1057. \anchor SC_HYPERVISOR_TRIGGER_RESIZE
  1058. \addindex __env__SC_HYPERVISOR_TRIGGER_RESIZE
  1059. Choose how should the hypervisor be triggered: <c>speed</c> if the resizing algorithm should
  1060. be called whenever the speed of the context does not correspond to an optimal precomputed value,
  1061. <c>idle</c> it the resizing algorithm should be called whenever the workers are idle for a period
  1062. longer than the value indicated when configuring the hypervisor.
  1063. </dd>
  1064. <dt>SC_HYPERVISOR_START_RESIZE</dt>
  1065. <dd>
  1066. \anchor SC_HYPERVISOR_START_RESIZE
  1067. \addindex __env__SC_HYPERVISOR_START_RESIZE
  1068. Indicate the moment when the resizing should be available. The value correspond to the percentage
  1069. of the total time of execution of the application. The default value is the resizing frame.
  1070. </dd>
  1071. <dt>SC_HYPERVISOR_MAX_SPEED_GAP</dt>
  1072. <dd>
  1073. \anchor SC_HYPERVISOR_MAX_SPEED_GAP
  1074. \addindex __env__SC_HYPERVISOR_MAX_SPEED_GAP
  1075. Indicate the ratio of speed difference between contexts that should trigger the hypervisor.
  1076. This situation may occur only when a theoretical speed could not be computed and the hypervisor
  1077. has no value to compare the speed to. Otherwise the resizing of a context is not influenced by the
  1078. the speed of the other contexts, but only by the the value that a context should have.
  1079. </dd>
  1080. <dt>SC_HYPERVISOR_STOP_PRINT</dt>
  1081. <dd>
  1082. \anchor SC_HYPERVISOR_STOP_PRINT
  1083. \addindex __env__SC_HYPERVISOR_STOP_PRINT
  1084. By default the values of the speed of the workers is printed during the execution
  1085. of the application. If the value 1 is given to this environment variable this printing
  1086. is not done.
  1087. </dd>
  1088. <dt>SC_HYPERVISOR_LAZY_RESIZE</dt>
  1089. <dd>
  1090. \anchor SC_HYPERVISOR_LAZY_RESIZE
  1091. \addindex __env__SC_HYPERVISOR_LAZY_RESIZE
  1092. By default the hypervisor resizes the contexts in a lazy way, that is workers are firstly added to a new context
  1093. before removing them from the previous one. Once this workers are clearly taken into account
  1094. into the new context (a task was poped there) we remove them from the previous one. However if the application
  1095. would like that the change in the distribution of workers should change right away this variable should be set to 0
  1096. </dd>
  1097. <dt>SC_HYPERVISOR_SAMPLE_CRITERIA</dt>
  1098. <dd>
  1099. \anchor SC_HYPERVISOR_SAMPLE_CRITERIA
  1100. \addindex __env__SC_HYPERVISOR_SAMPLE_CRITERIA
  1101. By default the hypervisor uses a sample of flops when computing the speed of the contexts and of the workers.
  1102. If this variable is set to <c>time</c> the hypervisor uses a sample of time (10% of an aproximation of the total
  1103. execution time of the application)
  1104. </dd>
  1105. </dl>
  1106. */