370_online_performance_tools.doxy 47 KB

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960616263646566676869707172737475767778798081828384858687888990919293949596979899100101102103104105106107108109110111112113114115116117118119120121122123124125126127128129130131132133134135136137138139140141142143144145146147148149150151152153154155156157158159160161162163164165166167168169170171172173174175176177178179180181182183184185186187188189190191192193194195196197198199200201202203204205206207208209210211212213214215216217218219220221222223224225226227228229230231232233234235236237238239240241242243244245246247248249250251252253254255256257258259260261262263264265266267268269270271272273274275276277278279280281282283284285286287288289290291292293294295296297298299300301302303304305306307308309310311312313314315316317318319320321322323324325326327328329330331332333334335336337338339340341342343344345346347348349350351352353354355356357358359360361362363364365366367368369370371372373374375376377378379380381382383384385386387388389390391392393394395396397398399400401402403404405406407408409410411412413414415416417418419420421422423424425426427428429430431432433434435436437438439440441442443444445446447448449450451452453454455456457458459460461462463464465466467468469470471472473474475476477478479480481482483484485486487488489490491492493494495496497498499500501502503504505506507508509510511512513514515516517518519520521522523524525526527528529530531532533534535536537538539540541542543544545546547548549550551552553554555556557558559560561562563564565566567568569570571572573574575576577578579580581582583584585586587588589590591592593594595596597598599600601602603604605606607608609610611612613614615616617618619620621622623624625626627628629630631632633634635636637638639640641642643644645646647648649650651652653654655656657658659660661662663664665666667668669670671672673674675676677678679680681682683684685686687688689690691692693694695696697698699700701702703704705706707708709710711712713714715716717718719720721722723724725726727728729730731732733734735736737738739740741742743744745746747748749750751752753754755756757758759760761762763764765766767768769770771772773774775776777778779780781782783784785786787788789790791792793794795796797798799800801802803804805806807808809810811812813814815816817818819820821822823824825826827828829830831832833834835836837838839840841842843844845846847848849850851852853854855856857858859860861862863864865866867868869870871872873874875876877878879880881882883884885886887888889890891892893894895896897898899900901902903904905906907908909910911912913914915916917918919920921922923924925
  1. /* StarPU --- Runtime system for heterogeneous multicore architectures.
  2. *
  3. * Copyright (C) 2011,2012,2016 Inria
  4. * Copyright (C) 2010-2020 CNRS
  5. * Copyright (C) 2009-2011,2014,2016,2018-2019 Université de Bordeaux
  6. *
  7. * StarPU is free software; you can redistribute it and/or modify
  8. * it under the terms of the GNU Lesser General Public License as published by
  9. * the Free Software Foundation; either version 2.1 of the License, or (at
  10. * your option) any later version.
  11. *
  12. * StarPU is distributed in the hope that it will be useful, but
  13. * WITHOUT ANY WARRANTY; without even the implied warranty of
  14. * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.
  15. *
  16. * See the GNU Lesser General Public License in COPYING.LGPL for more details.
  17. */
  18. /*! \page OnlinePerformanceTools Online Performance Tools
  19. \section On-linePerformanceFeedback On-line Performance Feedback
  20. \subsection EnablingOn-linePerformanceMonitoring Enabling On-line Performance Monitoring
  21. In order to enable online performance monitoring, the application can
  22. call starpu_profiling_status_set() with the parameter
  23. ::STARPU_PROFILING_ENABLE. It is possible to detect whether monitoring
  24. is already enabled or not by calling starpu_profiling_status_get().
  25. Enabling monitoring also reinitialize all previously collected
  26. feedback. The environment variable \ref STARPU_PROFILING can also be
  27. set to <c>1</c> to achieve the same effect. The function
  28. starpu_profiling_init() can also be called during the execution to
  29. reinitialize performance counters and to start the profiling if the
  30. environment variable \ref STARPU_PROFILING is set to <c>1</c>.
  31. Likewise, performance monitoring is stopped by calling
  32. starpu_profiling_status_set() with the parameter
  33. ::STARPU_PROFILING_DISABLE. Note that this does not reset the
  34. performance counters so that the application may consult them later
  35. on.
  36. More details about the performance monitoring API are available in \ref API_Profiling.
  37. \subsection Per-taskFeedback Per-task Feedback
  38. If profiling is enabled, a pointer to a structure
  39. starpu_profiling_task_info is put in the field
  40. starpu_task::profiling_info when a task terminates. This structure is
  41. automatically destroyed when the task structure is destroyed, either
  42. automatically or by calling starpu_task_destroy().
  43. The structure starpu_profiling_task_info indicates the date when the
  44. task was submitted (starpu_profiling_task_info::submit_time), started
  45. (starpu_profiling_task_info::start_time), and terminated
  46. (starpu_profiling_task_info::end_time), relative to the initialization
  47. of StarPU with starpu_init(). It also specifies the identifier of the worker
  48. that has executed the task (starpu_profiling_task_info::workerid).
  49. These date are stored as <c>timespec</c> structures which the user may convert
  50. into micro-seconds using the helper function
  51. starpu_timing_timespec_to_us().
  52. It it worth noting that the application may directly access this structure from
  53. the callback executed at the end of the task. The structure starpu_task
  54. associated to the callback currently being executed is indeed accessible with
  55. the function starpu_task_get_current().
  56. \subsection Per-codeletFeedback Per-codelet Feedback
  57. The field starpu_codelet::per_worker_stats is
  58. an array of counters. The <c>i</c>-th entry of the array is incremented every time a
  59. task implementing the codelet is executed on the <c>i</c>-th worker.
  60. This array is not reinitialized when profiling is enabled or disabled.
  61. \subsection Per-workerFeedback Per-worker Feedback
  62. The second argument returned by the function
  63. starpu_profiling_worker_get_info() is a structure
  64. starpu_profiling_worker_info that gives statistics about the specified
  65. worker. This structure specifies when StarPU started collecting
  66. profiling information for that worker
  67. (starpu_profiling_worker_info::start_time), the
  68. duration of the profiling measurement interval
  69. (starpu_profiling_worker_info::total_time), the time spent executing
  70. kernels (starpu_profiling_worker_info::executing_time), the time
  71. spent sleeping because there is no task to execute at all
  72. (starpu_profiling_worker_info::sleeping_time), and the number of tasks that were executed
  73. while profiling was enabled. These values give an estimation of the
  74. proportion of time spent do real work, and the time spent either
  75. sleeping because there are not enough executable tasks or simply
  76. wasted in pure StarPU overhead.
  77. Calling starpu_profiling_worker_get_info() resets the profiling
  78. information associated to a worker.
  79. To easily display all this information, the environment variable
  80. \ref STARPU_WORKER_STATS can be set to <c>1</c> (in addition to setting
  81. \ref STARPU_PROFILING to 1). A summary will then be displayed at
  82. program termination. To display the summary in a file instead of the
  83. standard error stream, use the environment variable \ref STARPU_WORKER_STATS_FILE.
  84. \verbatim
  85. Worker stats:
  86. CUDA 0.0 (4.7 GiB)
  87. 480 task(s)
  88. total: 1574.82 ms executing: 1510.72 ms sleeping: 0.00 ms overhead 64.10 ms
  89. 325.217970 GFlop/s
  90. CPU 0
  91. 22 task(s)
  92. total: 1574.82 ms executing: 1364.81 ms sleeping: 0.00 ms overhead 210.01 ms
  93. 7.512057 GFlop/s
  94. CPU 1
  95. 14 task(s)
  96. total: 1574.82 ms executing: 1500.13 ms sleeping: 0.00 ms overhead 74.69 ms
  97. 6.675853 GFlop/s
  98. CPU 2
  99. 14 task(s)
  100. total: 1574.82 ms executing: 1553.12 ms sleeping: 0.00 ms overhead 21.70 ms
  101. 7.152886 GFlop/s
  102. \endverbatim
  103. The number of GFlops is available because the starpu_task::flops field of the
  104. tasks were filled (or \ref STARPU_FLOPS used in starpu_task_insert()).
  105. When an FxT trace is generated (see \ref GeneratingTracesWithFxT), it is also
  106. possible to use the tool <c>starpu_workers_activity</c> (see
  107. \ref MonitoringActivity) to generate a graphic showing the evolution of
  108. these values during the time, for the different workers.
  109. \subsection Bus-relatedFeedback Bus-related Feedback
  110. // how to enable/disable performance monitoring
  111. // what kind of information do we get ?
  112. The bus speed measured by StarPU can be displayed by using the tool
  113. <c>starpu_machine_display</c>, for instance:
  114. \verbatim
  115. StarPU has found:
  116. 3 CUDA devices
  117. CUDA 0 (Tesla C2050 02:00.0)
  118. CUDA 1 (Tesla C2050 03:00.0)
  119. CUDA 2 (Tesla C2050 84:00.0)
  120. from to RAM to CUDA 0 to CUDA 1 to CUDA 2
  121. RAM 0.000000 5176.530428 5176.492994 5191.710722
  122. CUDA 0 4523.732446 0.000000 2414.074751 2417.379201
  123. CUDA 1 4523.718152 2414.078822 0.000000 2417.375119
  124. CUDA 2 4534.229519 2417.069025 2417.060863 0.000000
  125. \endverbatim
  126. Statistics about the data transfers which were performed and temporal average
  127. of bandwidth usage can be obtained by setting the environment variable
  128. \ref STARPU_BUS_STATS to <c>1</c>; a summary will then be displayed at
  129. program termination. To display the summary in a file instead of the
  130. standard error stream, use the environment variable \ref STARPU_BUS_STATS_FILE.
  131. \verbatim
  132. Data transfer stats:
  133. RAM 0 -> CUDA 0 319.92 MB 213.10 MB/s (transfers : 91 - avg 3.52 MB)
  134. CUDA 0 -> RAM 0 214.45 MB 142.85 MB/s (transfers : 61 - avg 3.52 MB)
  135. RAM 0 -> CUDA 1 302.34 MB 201.39 MB/s (transfers : 86 - avg 3.52 MB)
  136. CUDA 1 -> RAM 0 133.59 MB 88.99 MB/s (transfers : 38 - avg 3.52 MB)
  137. CUDA 0 -> CUDA 1 144.14 MB 96.01 MB/s (transfers : 41 - avg 3.52 MB)
  138. CUDA 1 -> CUDA 0 130.08 MB 86.64 MB/s (transfers : 37 - avg 3.52 MB)
  139. RAM 0 -> CUDA 2 312.89 MB 208.42 MB/s (transfers : 89 - avg 3.52 MB)
  140. CUDA 2 -> RAM 0 133.59 MB 88.99 MB/s (transfers : 38 - avg 3.52 MB)
  141. CUDA 0 -> CUDA 2 151.17 MB 100.69 MB/s (transfers : 43 - avg 3.52 MB)
  142. CUDA 2 -> CUDA 0 105.47 MB 70.25 MB/s (transfers : 30 - avg 3.52 MB)
  143. CUDA 1 -> CUDA 2 175.78 MB 117.09 MB/s (transfers : 50 - avg 3.52 MB)
  144. CUDA 2 -> CUDA 1 203.91 MB 135.82 MB/s (transfers : 58 - avg 3.52 MB)
  145. Total transfers: 2.27 GB
  146. \endverbatim
  147. \subsection MPI-relatedFeedback MPI-related Feedback
  148. Statistics about the data transfers which were performed over MPI can be
  149. obtained by setting the environment variable \ref STARPU_COMM_STATS to <c>1</c>;
  150. a summary will then be displayed at program termination:
  151. \verbatim
  152. [starpu_comm_stats][1] TOTAL: 456.000000 B 0.000435 MB 0.000188 B/s 0.000000 MB/s
  153. [starpu_comm_stats][1:0] 456.000000 B 0.000435 MB 0.000188 B/s 0.000000 MB/s
  154. [starpu_comm_stats][0] TOTAL: 456.000000 B 0.000435 MB 0.000188 B/s 0.000000 MB/s
  155. [starpu_comm_stats][0:1] 456.000000 B 0.000435 MB 0.000188 B/s 0.000000 MB/s
  156. \endverbatim
  157. These statistics can be plotted as heatmaps using StarPU tool <c>starpu_mpi_comm_matrix.py</c> (see \ref MPIDebug).
  158. \section TaskAndWorkerProfiling Task And Worker Profiling
  159. A full example showing how to use the profiling API is available in
  160. the StarPU sources in the directory <c>examples/profiling/</c>.
  161. \code{.c}
  162. struct starpu_task *task = starpu_task_create();
  163. task->cl = &cl;
  164. task->synchronous = 1;
  165. /* We will destroy the task structure by hand so that we can
  166. * query the profiling info before the task is destroyed. */
  167. task->destroy = 0;
  168. /* Submit and wait for completion (since synchronous was set to 1) */
  169. starpu_task_submit(task);
  170. /* The task is finished, get profiling information */
  171. struct starpu_profiling_task_info *info = task->profiling_info;
  172. /* How much time did it take before the task started ? */
  173. double delay += starpu_timing_timespec_delay_us(&info->submit_time, &info->start_time);
  174. /* How long was the task execution ? */
  175. double length += starpu_timing_timespec_delay_us(&info->start_time, &info->end_time);
  176. /* We no longer need the task structure */
  177. starpu_task_destroy(task);
  178. \endcode
  179. \code{.c}
  180. /* Display the occupancy of all workers during the test */
  181. int worker;
  182. for (worker = 0; worker < starpu_worker_get_count(); worker++)
  183. {
  184. struct starpu_profiling_worker_info worker_info;
  185. int ret = starpu_profiling_worker_get_info(worker, &worker_info);
  186. STARPU_ASSERT(!ret);
  187. double total_time = starpu_timing_timespec_to_us(&worker_info.total_time);
  188. double executing_time = starpu_timing_timespec_to_us(&worker_info.executing_time);
  189. double sleeping_time = starpu_timing_timespec_to_us(&worker_info.sleeping_time);
  190. double overhead_time = total_time - executing_time - sleeping_time;
  191. float executing_ratio = 100.0*executing_time/total_time;
  192. float sleeping_ratio = 100.0*sleeping_time/total_time;
  193. float overhead_ratio = 100.0 - executing_ratio - sleeping_ratio;
  194. char workername[128];
  195. starpu_worker_get_name(worker, workername, 128);
  196. fprintf(stderr, "Worker %s:\n", workername);
  197. fprintf(stderr, "\ttotal time: %.2lf ms\n", total_time*1e-3);
  198. fprintf(stderr, "\texec time: %.2lf ms (%.2f %%)\n", executing_time*1e-3, executing_ratio);
  199. fprintf(stderr, "\tblocked time: %.2lf ms (%.2f %%)\n", sleeping_time*1e-3, sleeping_ratio);
  200. fprintf(stderr, "\toverhead time: %.2lf ms (%.2f %%)\n", overhead_time*1e-3, overhead_ratio);
  201. }
  202. \endcode
  203. \section PerformanceModelExample Performance Model Example
  204. To achieve good scheduling, StarPU scheduling policies need to be able to
  205. estimate in advance the duration of a task. This is done by giving to codelets
  206. a performance model, by defining a structure starpu_perfmodel and
  207. providing its address in the field starpu_codelet::model. The fields
  208. starpu_perfmodel::symbol and starpu_perfmodel::type are mandatory, to
  209. give a name to the model, and the type of the model, since there are
  210. several kinds of performance models. For compatibility, make sure to
  211. initialize the whole structure to zero, either by using explicit
  212. memset(), or by letting the compiler implicitly do it as examplified
  213. below.
  214. <ul>
  215. <li>
  216. Measured at runtime (model type ::STARPU_HISTORY_BASED). This assumes that for a
  217. given set of data input/output sizes, the performance will always be about the
  218. same. This is very true for regular kernels on GPUs for instance (<0.1% error),
  219. and just a bit less true on CPUs (~=1% error). This also assumes that there are
  220. few different sets of data input/output sizes. StarPU will then keep record of
  221. the average time of previous executions on the various processing units, and use
  222. it as an estimation. History is done per task size, by using a hash of the input
  223. and ouput sizes as an index.
  224. It will also save it in <c>$STARPU_HOME/.starpu/sampling/codelets</c>
  225. for further executions, and can be observed by using the tool
  226. <c>starpu_perfmodel_display</c>, or drawn by using
  227. the tool <c>starpu_perfmodel_plot</c> (\ref PerformanceModelCalibration). The
  228. models are indexed by machine name. To
  229. share the models between machines (e.g. for a homogeneous cluster), use
  230. <c>export STARPU_HOSTNAME=some_global_name</c>. Measurements are only done
  231. when using a task scheduler which makes use of it, such as
  232. <c>dmda</c>. Measurements can also be provided explicitly by the application, by
  233. using the function starpu_perfmodel_update_history().
  234. The following is a small code example.
  235. If e.g. the code is recompiled with other compilation options, or several
  236. variants of the code are used, the <c>symbol</c> string should be changed to reflect
  237. that, in order to recalibrate a new model from zero. The <c>symbol</c> string can even
  238. be constructed dynamically at execution time, as long as this is done before
  239. submitting any task using it.
  240. \code{.c}
  241. static struct starpu_perfmodel mult_perf_model =
  242. {
  243. .type = STARPU_HISTORY_BASED,
  244. .symbol = "mult_perf_model"
  245. };
  246. struct starpu_codelet cl =
  247. {
  248. .cpu_funcs = { cpu_mult },
  249. .cpu_funcs_name = { "cpu_mult" },
  250. .nbuffers = 3,
  251. .modes = { STARPU_R, STARPU_R, STARPU_W },
  252. /* for the scheduling policy to be able to use performance models */
  253. .model = &mult_perf_model
  254. };
  255. \endcode
  256. </li>
  257. <li>
  258. Measured at runtime and refined by regression (model types
  259. ::STARPU_REGRESSION_BASED and ::STARPU_NL_REGRESSION_BASED). This
  260. still assumes performance regularity, but works
  261. with various data input sizes, by applying regression over observed
  262. execution times. ::STARPU_REGRESSION_BASED uses an <c>a*n^b</c> regression
  263. form, ::STARPU_NL_REGRESSION_BASED uses an <c>a*n^b+c</c> (more precise than
  264. ::STARPU_REGRESSION_BASED, but costs a lot more to compute).
  265. For instance,
  266. <c>tests/perfmodels/regression_based.c</c> uses a regression-based performance
  267. model for the function memset().
  268. Of course, the application has to issue
  269. tasks with varying size so that the regression can be computed. StarPU will not
  270. trust the regression unless there is at least 10% difference between the minimum
  271. and maximum observed input size. It can be useful to set the
  272. environment variable \ref STARPU_CALIBRATE to <c>1</c> and run the application
  273. on varying input sizes with \ref STARPU_SCHED set to <c>dmda</c> scheduler,
  274. so as to feed the performance model for a variety of
  275. inputs. The application can also provide the measurements explictly by
  276. using the function starpu_perfmodel_update_history(). The tools
  277. <c>starpu_perfmodel_display</c> and <c>starpu_perfmodel_plot</c> can
  278. be used to observe how much the performance model is calibrated
  279. (\ref PerformanceModelCalibration); when their output look good,
  280. \ref STARPU_CALIBRATE can be reset to <c>0</c> to let
  281. StarPU use the resulting performance model without recording new measures, and
  282. \ref STARPU_SCHED can be set to <c>dmda</c> to benefit from the performance models. If
  283. the data input sizes vary a lot, it is really important to set
  284. \ref STARPU_CALIBRATE to <c>0</c>, otherwise StarPU will continue adding the
  285. measures, and result with a very big performance model, which will take time a
  286. lot of time to load and save.
  287. For non-linear regression, since computing it
  288. is quite expensive, it is only done at termination of the application. This
  289. means that the first execution of the application will use only history-based
  290. performance model to perform scheduling, without using regression.
  291. </li>
  292. <li>
  293. Another type of model is ::STARPU_MULTIPLE_REGRESSION_BASED, which
  294. is based on multiple linear regression. In this model, the user
  295. defines both the relevant parameters and the equation for computing the
  296. task duration.
  297. \f[
  298. T_{kernel} = a + b(M^{\alpha_1} * N^{\beta_1} * K^{\gamma_1}) + c(M^{\alpha_2} * N^{\beta_2} * K^{\gamma_2}) + ...
  299. \f]
  300. \f$M, N, K\f$ are the parameters of the task, added at the task
  301. creation. These need to be extracted by the <c>cl_perf_func</c>
  302. function, which should be defined by the user. \f$\alpha, \beta,
  303. \gamma\f$ are the exponents defined by the user in
  304. <c>model->combinations</c> table. Finally, coefficients \f$a, b, c\f$
  305. are computed automatically by the StarPU at the end of the execution, using least
  306. squares method of the <c>dgels_</c> LAPACK function.
  307. <c>examples/mlr/mlr.c</c> example provides more details on
  308. the usage of ::STARPU_MULTIPLE_REGRESSION_BASED models.
  309. Coefficients computation is done at the end of the execution, and the
  310. results are stored in standard codelet perfmodel files. Additional
  311. files containing the duration of task together with the value of each
  312. parameter are stored in <c>.starpu/sampling/codelets/tmp/</c>
  313. directory. These files are reused when \ref STARPU_CALIBRATE
  314. environment variable is set to <c>1</c>, to recompute coefficients
  315. based on the current, but also on the previous
  316. executions. Additionally, when multiple linear regression models are
  317. disabled (using \ref disable-mlr "--disable-mlr" configure option) or when the
  318. <c>model->combinations</c> are not defined, StarPU will still write
  319. output files into <c>.starpu/sampling/codelets/tmp/</c> to allow
  320. performing an analysis. This analysis typically aims at finding the
  321. most appropriate equation for the codelet and
  322. <c>tools/starpu_mlr_analysis</c> script provides an example of how to
  323. perform such study.
  324. </li>
  325. <li>
  326. Provided as an estimation from the application itself (model type
  327. ::STARPU_COMMON and field starpu_perfmodel::cost_function),
  328. see for instance
  329. <c>examples/common/blas_model.h</c> and <c>examples/common/blas_model.c</c>.
  330. </li>
  331. <li>
  332. Provided explicitly by the application (model type ::STARPU_PER_ARCH):
  333. either field starpu_perfmodel::arch_cost_function, or
  334. the fields <c>.per_arch[arch][nimpl].cost_function</c> have to be
  335. filled with pointers to functions which return the expected duration
  336. of the task in micro-seconds, one per architecture, see for instance
  337. <c>tests/datawizard/locality.c</c>
  338. </li>
  339. </ul>
  340. For ::STARPU_HISTORY_BASED, ::STARPU_REGRESSION_BASED, and
  341. ::STARPU_NL_REGRESSION_BASED, the dimensions of task data (both input
  342. and output) are used as an index by default. ::STARPU_HISTORY_BASED uses a CRC
  343. hash of the dimensions as an index to distinguish histories, and
  344. ::STARPU_REGRESSION_BASED and ::STARPU_NL_REGRESSION_BASED use the total
  345. size as an index for the regression.
  346. The starpu_perfmodel::size_base and starpu_perfmodel::footprint fields however
  347. permit the application to override that, when for instance some of the data
  348. do not matter for task cost (e.g. mere reference table), or when using sparse
  349. structures (in which case it is the number of non-zeros which matter), or when
  350. there is some hidden parameter such as the number of iterations, or when the
  351. application actually has a very good idea of the complexity of the algorithm,
  352. and just not the speed of the processor, etc. The example in the directory
  353. <c>examples/pi</c> uses this to include the number of iterations in the base
  354. size. starpu_perfmodel::size_base should be used when the variance of the actual
  355. performance is known (i.e. bigger return value is longer execution
  356. time), and thus particularly useful for ::STARPU_REGRESSION_BASED or
  357. ::STARPU_NL_REGRESSION_BASED. starpu_perfmodel::footprint can be used when the
  358. variance of the actual performance is unknown (irregular performance behavior,
  359. etc.), and thus only useful for ::STARPU_HISTORY_BASED.
  360. starpu_task_data_footprint() can be used as a base and combined with other
  361. parameters through starpu_hash_crc32c_be() for instance.
  362. StarPU will automatically determine when the performance model is calibrated,
  363. or rather, it will assume the performance model is calibrated until the
  364. application submits a task for which the performance can not be predicted. For
  365. ::STARPU_HISTORY_BASED, StarPU will require 10 (STARPU_CALIBRATE_MINIMUM)
  366. measurements for a given size before estimating that an average can be taken as
  367. estimation for further executions with the same size. For
  368. ::STARPU_REGRESSION_BASED and ::STARPU_NL_REGRESSION_BASED, StarPU will require
  369. 10 (STARPU_CALIBRATE_MINIMUM) measurements, and that the minimum measured
  370. data size is smaller than 90% of the maximum measured data size (i.e. the
  371. measurement interval is large enough for a regression to have a meaning).
  372. Calibration can also be forced by setting the \ref STARPU_CALIBRATE environment
  373. variable to <c>1</c>, or even reset by setting it to <c>2</c>.
  374. How to use schedulers which can benefit from such performance model is explained
  375. in \ref TaskSchedulingPolicy.
  376. The same can be done for task energy consumption estimation, by setting
  377. the field starpu_codelet::energy_model the same way as the field
  378. starpu_codelet::model. Note: for now, the application has to give to
  379. the energy consumption performance model a name which is different from
  380. the execution time performance model.
  381. The application can request time estimations from the StarPU performance
  382. models by filling a task structure as usual without actually submitting
  383. it. The data handles can be created by calling any of the functions
  384. <c>starpu_*_data_register</c> with a <c>NULL</c> pointer and <c>-1</c>
  385. node and the desired data sizes, and need to be unregistered as usual.
  386. The functions starpu_task_expected_length() and
  387. starpu_task_expected_energy() can then be called to get an estimation
  388. of the task cost on a given arch. starpu_task_footprint() can also be
  389. used to get the footprint used for indexing history-based performance
  390. models. starpu_task_destroy() needs to be called to destroy the dummy
  391. task afterwards. See <c>tests/perfmodels/regression_based.c</c> for an example.
  392. The application can also request an on-the-fly XML report of the performance
  393. model, by calling starpu_perfmodel_dump_xml() to print the report to a
  394. <c>FILE*</c>.
  395. \section Performance Monitoring Counters
  396. This section presents the StarPU performance monitoring framework. It summarizes the objectives of the framework. It then introduces the entities involved in the framework. It presents the API of the framework, as well as some implementation details. It exposes the typical sequence of operations to plug an external tool to monitor a performance counter of StarPU.
  397. \subsection Objectives
  398. The objectives of this framework are to let external tools interface with StarPU to collect various performance metrics at runtime, in a generic, safe, extensible way. For that, it enables such tools to discover the available performance metrics in a particular StarPU build as well as the type of each performance counter value. It lets these tools build sets of performance counters to monitor, and then register listener callbacks to collect the measurement samples of these sets of performance counters at runtime.
  399. \subsection Entities
  400. The performance monitoring framework is built on a series of concepts and items, organized in a consistent way. The corresponding C language objects should be considered opaque by external tools, and should only be manipulated through proper function calls and accessors.
  401. \subsubsection Performance Counter
  402. The performance counter entity is the fundamental object of the framework, representing one piece of performance metrics, such as for instance the total number of tasks submitted so far, that is exported by StarPU and can be collected through the framework at runtime. A performance counter has a type and belongs to a scope. A performance counter is designated by a unique name and unique ID integer.
  403. \subsubsection Performance Counter Type
  404. A performance counter has a type. A type is designated by a unique name and unique ID number. Currently supported types include:
  405. \verbatim
  406. Type Name Type Definition
  407. "int32" 32-bit signed integers
  408. "int64" 64-bit signed integers
  409. "float" 32-bit single-precision floating point
  410. "double" 64-bit double-precision floating point
  411. \endverbatim
  412. \subsubsection Performance Counter Scope
  413. A performance counter belongs to a scope. The scope of a counter defines the context considered for computing the corresponding performance counter. A scope is designated with a unique name and unique ID number. Currently defined scopes include:
  414. \verbatim
  415. Scope Name Scope Definition
  416. "global" Counter is global to the StarPU instance
  417. "per_worker" Counter is within the scope of a thread worker
  418. "per_codelet" Counter is within the scope of a task codelet
  419. \endverbatim
  420. \subsubsection Performance Counter Set
  421. A performance counter set is a subset of the performance counters belonging to the same scope. Each counter of the scope can be in the enabled or disabled state in a performance counter set. A performance counter set enables a performance monitoring tool to indicate the set of counters to be collected for a particular listener callback.
  422. \subsubsection Performance Counter Sample
  423. A performance counter sample corresponds to one sample of collected measurement values of a performance counter set. Only the values corresponding to enabled counters in the sample's counter set should be observed by the listener callback. Whether the sample contains valid values for counters disabled in the set is unspecified.
  424. \subsubsection Performance Counter Listener
  425. A performance counter listener is a callback function registered by some external tool to monitor a set of performance counters in a particular scope. It is called each time a new performance counter sample is ready to be observed. The sample object should not be accessed outside of the callback.
  426. \subsubsection Application Programming Interface
  427. The API of the performance monitoring framework is defined in the "starpu_perf_monitoring.h" public header file of StarPU. This header file is automatically included with "starpu.h". An example of use of the routines is given in Section 3.6.
  428. \subsubsection Scope Related Routines
  429. \verbatim
  430. Function Name Function Definition
  431. starpu_perf_counter_scope_name_to_id Translate scope name constant string to scope id
  432. starpu_perf_counter_scope_id_to_name Translate scope id to scope name constant string
  433. \endverbatim
  434. \subsubsection Type Related Routines
  435. \verbatim
  436. Function Name Function Definition
  437. starpu_perf_counter_type_name_to_id Translate type name constant string to type id
  438. starpu_perf_counter_type_id_to_name Translate type id to type name constant string
  439. \endverbatim
  440. \subsubsection Counter Related Routines
  441. \verbatim
  442. Function Name Function Definition
  443. starpu_perf_counter_nb Return the number of performance counters for the given scope
  444. starpu_perf_counter_name_to_id Translate a performance counter name to its id
  445. starpu_perf_counter_nth_to_id Translate a performance counter rank in its scope to its counter id
  446. starpu_perf_counter_id_to_name Translate a counter id to its name constant string
  447. starpu_perf_counter_get_type_id Return the counter's type id
  448. starpu_perf_counter_get_help_string Return the counter's help string
  449. starpu_perf_counter_list_avail Display the list of counters defined in the given scope
  450. starpu_perf_counter_list_all_avail Display the list of counters defined in all scopes
  451. \endverbatim
  452. \subsubsection Counter Set Related Routines
  453. \verbatim
  454. Function Name Function Definition
  455. starpu_perf_counter_set_alloc Allocate a new performance counter set
  456. starpu_perf_counter_set_free Free a performance counter set
  457. starpu_perf_counter_set_enable_id Enable a given counter in the set
  458. starpu_perf_counter_set_disable_id Disable a given counter in the set
  459. \endverbatim
  460. \subsubsection Listener Related Routines
  461. \verbatim
  462. Function Name Function Definition
  463. starpu_perf_counter_listener_init Initialize a new performance counter listener
  464. starpu_perf_counter_listener_exit End a performance counter listener
  465. starpu_perf_counter_set_global_listener Set a listener for the global scope
  466. starpu_perf_counter_set_per_worker_listener Set a listener for the per_worker scope on a given worker
  467. starpu_perf_counter_set_all_per_worker_listeners Set a common listener for all workers
  468. starpu_perf_counter_set_per_codelet_listener Set a per_codelet listener for a codelet
  469. starpu_perf_counter_unset_global_listener Unset the global listener
  470. starpu_perf_counter_unset_per_worker_listener Unset the per_worker listener
  471. starpu_perf_counter_unset_all_per_worker_listeners Unset all per_worker listeners
  472. starpu_perf_counter_unset_per_codelet_listener Unset a per_codelet listener
  473. \endverbatim
  474. \subsubsection Sample Related Routines
  475. \verbatim
  476. Function Name Function Definition
  477. starpu_perf_counter_sample_get_int32_value Read an int32 counter value from a sample
  478. starpu_perf_counter_sample_get_int64_value Read an int64 counter value from a sample
  479. starpu_perf_counter_sample_get_float_value Read a float counter value from a sample
  480. starpu_perf_counter_sample_get_double_value Read a double counter value from a sample
  481. \endverbatim
  482. \subsection Implementation Details
  483. \subsubsection Performance Counter Registration
  484. Each module of StarPU can export performance counters. In order to do so, modules that need to export some counters define a registration function that is called at StarPU initialization time. This function is responsible for calling the "_starpu_perf_counter_register()" function once for each counter it exports, to let the framework know about the list of counters managed by the module. It also registers performance sample updater callbacks for the module, one for each scope for which it exports counters.
  485. \subsubsection Performance Sample Updaters
  486. The updater callback for a module and scope combination is internally called every time a sample for a set of performance counter must be updated. Thus, the updated callback is responsible for filling the sample's selected counters with the counter values found at the time of the call.
  487. Global updaters are currently called at task submission time, as well as any blocking tasks management function of the StarPU API, such as the "starpu_task_wait_for_all()", which waits for the completion of all tasks submitted up to this point.
  488. Per-worker updaters are currently called at the level of StarPU's drivers, that is, the modules in charge of task execution of hardware-specific worker threads. The actual calls occur in-between the execution of tasks.
  489. Per-codelet updaters are currently called both at task submission time, and at the level of StarPU's drivers together with the per-worker updaters.
  490. A performance sample object is locked during the sample collection. The locking prevents the following issues:
  491. <ul>
  492. <li>The listener of sample being changed during sample collection;
  493. <li>The set of counters enabled for a sample being changed;
  494. <li>Conflicting concurrent updates;
  495. <li>Updates while the sample is being read by the listener.
  496. </ul>
  497. The location of the updaters' calls is chosen to minimize the sequentialization effect of the locking, in order to limit the level of interference of the monitoring process. For Global updaters, the calls are performed only on the application thread(s) in charge of submitting tasks. Since, in most cases, only a single application thread submits tasks, the sequentialization effect is moderate. Per-worker updates are local to their worker, thus here again the sample lock is un-contented, unless the external monitoring tool frequently changes the set of enabled counters in the sample.
  498. \subsubsection Counter operations
  499. In practice the sample updaters only take snapshots of the actual performance counters. The performance counters themselves are updated with ad-hoc procedures depending on each counter. Such procedures typically involve atomic operations. While operations such as atomic increments or decrements on integer values are readily available, this is not the case for more complex operations such as min/max for computing peak value counters (for instance in the global and per-codelet counters for peak number of submitted tasks and peak number of ready tasks waiting for execution), and this is also not the case for computations on floating point data (used for instance in computing cumulated execution time of tasks, either per worker or per codelet). The performance monitoring framework therefore supplies such missing routines, for the internal use of StarPU.
  500. \subsubsection Runtime checks
  501. The performance monitoring framework features a comprehensive set of runtime checks to verify that both StarPU and some external tool do not access a performance counter with the wrong typed routines, to quickly detect situations of mismatch that can result from the evolution of multiple pieces of software at distinct paces. Moreover, no StarPU data structure is accessed directly either by the external code making use of the performance monitoring framework. The use of the C enum constants is optional; referring to values through constant strings is available when more robustness is desired. These runtime checks enable the framework to be extensible. Moreover, while the framework's counters currently are permanently compiled in, they could be made optional at compile time, for instance to suppress any overhead once the analysis and optimization process has been completed by the programmer. Thanks to the runtime discovery of available counters, the applicative code, or an intermediate layer such as skeleton layer acting on its behalf, would then be able to adapt to performance analysis builds versus optimized builds.
  502. \subsection Exported Counters
  503. \subsubsection Global Scope
  504. \verbatim
  505. Counter Name Counter Definition
  506. starpu.task.g_total_submitted Total number of tasks submitted
  507. starpu.task.g_peak_submitted Maximum number of tasks submitted, waiting for dependencies resolution at any time
  508. starpu.task.g_peak_ready Maximum number of tasks ready for execution, waiting for an execution slot at any time
  509. \endverbatim
  510. \subsubsection Per-worker Scope
  511. \verbatim
  512. Counter Name Counter Definition
  513. starpu.task.w_total_executed Total number of tasks executed on a given worker
  514. starpu.task.w_cumul_execution_time Cumulated execution time of tasks executed on a given worker
  515. \endverbatim
  516. \subsubsection Per-Codelet Scope
  517. \verbatim
  518. Counter Name Counter Definition
  519. starpu.task.c_total_submitted Total number of submitted tasks for a given codelet
  520. starpu.task.c_peak_submitted Maximum number of submitted tasks for a given codelet waiting for dependencies resolution at any time
  521. starpu.task.c_peak_ready Maximum number of ready tasks for a given codelet waiting for an execution slot at any time
  522. starpu.task.c_total_executed Total number of executed tasks for a given codelet
  523. starpu.task.c_cumul_execution_time Cumulated execution time of tasks for a given codelet
  524. \endverbatim
  525. \subsection Sequence of operations
  526. This section presents a typical sequence of operations to interface an external tool with some StarPU performance counters. In this example, the counters monitored are the per-worker total number of executed tasks ("starpu.task.w_total_executed") and the tasks' cumulated execution time ("starpu.task.w_cumul_execution_time").
  527. <b>Step 0: Initialize StarPU</b>
  528. StarPU must first be initialized, by a call to starpu_init(), for performance counters to become available, since each module of StarPU registers the performance counters it exports during that initialization phase.
  529. \verbatim
  530. int ret = starpu_init(NULL);
  531. \endverbatim
  532. <b>Step 1: Allocate a counter set</b>
  533. A counter set has to be allocated on the per-worker scope. The per-worker scope id can be obtained by name, or with the pre-defined enum value starpu_perf_counter_scope_per_worker.
  534. \verbatim
  535. enum starpu_perf_counter_scope w_scope = starpu_perf_counter_scope_per_worker;
  536. struct starpu_perf_counter_set *w_set = starpu_perf_counter_set_alloc(w_scope);
  537. \endverbatim
  538. <b>Step 2: Get the counter IDs</b>
  539. Each performance counter has a unique ID used to refer to it in subsequent calls to the performance monitoring framework.
  540. \verbatim
  541. int id_w_total_executed = starpu_perf_counter_name_to_id(w_scope,
  542. "starpu.task.w_total_executed");
  543. int id_w_cumul_execution_time = starpu_perf_counter_name_to_id(w_scope,
  544. "starpu.task.w_cumul_execution_time");
  545. \endverbatim
  546. <b>Step 3: Enable the counters in the counter set</b>
  547. This step indicates which counters will be collected into performance monitoring samples for the listeners referring to this counter set.
  548. \verbatim
  549. starpu_perf_counter_set_enable_id(w_set, id_w_total_executed);
  550. starpu_perf_counter_set_enable_id(w_set, id_w_cumul_execution_time);
  551. \endverbatim
  552. <b>Step 4: Write a listener callback</b>
  553. This callback will be triggered when a sample becomes available. Upon execution, it reads the values for the two counters from the sample and displays these values, for the sake of the example.
  554. \verbatim
  555. void w_listener_cb(struct starpu_perf_counter_listener *listener,
  556. struct starpu_perf_counter_sample *sample,
  557. void *context)
  558. {
  559. int32_t w_total_executed =
  560. starpu_perf_counter_sample_get_int32_value(sample, id_w_total_executed);
  561. double w_cumul_execution_time =
  562. starpu_perf_counter_sample_get_double_value(sample, id_w_cumul_execution_time);
  563. printf("worker[%d]: w_total_executed = %d, w_cumul_execution_time = %lf\n",
  564. starpu_worker_get_id(),
  565. w_total_executed,
  566. w_cumul_execution_time);
  567. }
  568. \endverbatim
  569. <b>Step 5: Initialize the listener</b>
  570. This step allocates the listener structure and prepares it to listen to the selected set of per-worker counters. However, it is not actually active until Step 6, once it is attached to one or more worker.
  571. \verbatim
  572. struct starpu_perf_counter_listener * w_listener =
  573. starpu_perf_counter_listener_init(w_set, w_listener_cb, NULL);
  574. \endverbatim
  575. <b>Step 6: Set the listener on all workers</b>
  576. This step actually makes the listener active, in this case on every StarPU worker thread.
  577. \verbatim
  578. starpu_perf_counter_set_all_per_worker_listeners(w_listener);
  579. \endverbatim
  580. After this step, any task assigned to a worker will be counted in that worker selected performance counters, and reported to the listener.
  581. \section Performance Steering Knobs
  582. This section presents the StarPU performance steering framework. It summarizes the objectives of the framework. It introduces the entities involved in the framework, and then details the API, implementation and sequence of operations.
  583. \subsection Objectives
  584. The objectives of this framework are to let external tools interface with StarPU, observe, and act at runtime on actionable performance steering knobs exported by StarPU, in a generic, safe, extensible way. It defines an API to let such external tools discover the available performance steering knobs in a particular StarPU revision of build, as well as the type of each knob.
  585. \subsection Entities
  586. \subsubsection Performance Steering Knob
  587. The performance steering knob entity designates one runtime-actionable knob exported by StarPU. It may represent some setting, or some constant used within StarPU for a given purpose. The value of the knob is typed, it can be obtained or modified with the appropriate getter/setter routine. The knob belongs to a scope. A performance steering knob is designated with a unique name and unique ID number.
  588. \subsubsection Knob Type
  589. A performance steering knob has a type. A type is designated by a unique name and unique ID number. Currently supported types include:
  590. \verbatim
  591. Type Name Type Definition
  592. "int32" 32-bit signed integers
  593. "int64" 64-bit signed integers
  594. "float" 32-bit single precision floating point
  595. "double" 64-bit double precision floating point
  596. \endverbatim
  597. On/Off knobs are defined as "int32" type, with value 0 for Off and value !0 for On, unless otherwise specified.
  598. \subsubsection Knob Scope
  599. A performance steering knob belongs to a scope. The scope of a knob defines the context considered for computing the corresponding knob. A scope is designated with a unique name and unique ID number. Currently defined scopes include:
  600. \verbatim
  601. Scope Name Scope Definition
  602. "global" Knob is global to the StarPU instance
  603. "per_worker" Knob is within the scope of a thread worker
  604. "per_scheduler" Knob is within the scope of a scheduling policy instance
  605. \endverbatim
  606. \subsubsection Knob Group
  607. The notion of Performance Steering Knob Group is currently internal to StarPU. It defines a series of knobs that are handled by the same couple of setter/getter functions internally. A knob group belongs to a knob scope.
  608. \subsection Application Programming Interface
  609. \subsubsection Scope Related Routines
  610. \verbatim
  611. Function Name Function Definition
  612. starpu_perf_knob_scope_name_to_id Translate scope name constant string to scope id
  613. starpu_perf_knob_scope_id_to_name Translate scope id to scope name constant string
  614. \endverbatim
  615. \subsubsection Type Related Routines
  616. \verbatim
  617. Function Name Function Definition
  618. starpu_perf_knob_type_name_to_id Translate type name constant string to type id
  619. starpu_perf_knob_type_id_to_name Translate type id to type name constant string
  620. \endverbatim
  621. \subsubsection Performance Steering Knob Related Routines
  622. \verbatim
  623. Function Name Function Definition
  624. starpu_perf_knob_nb Return the number of performance steering knobs for the given scope
  625. starpu_perf_knob_name_to_id Translate a performance knob name to its id
  626. starpu_perf_knob_nth_to_id Translate a performance knob rank in its scope to its knob id
  627. starpu_perf_knob_id_to_name Translate a knob id to its name constant string
  628. starpu_perf_knob_get_type_id Return the knob's type id
  629. starpu_perf_knob_get_help_string Return the knob's help string
  630. starpu_perf_knob_list_avail Display the list of knobs defined in the given scope
  631. starpu_perf_knob_list_all_avail Display the list of knobs defined in all scopes
  632. starpu_perf_knob_get_<SCOPE>_<TYPE>_value Get knob value for given scope and type
  633. starpu_perf_knob_set_<SCOPE>_<TYPE>_value Set knob value for given scope and type
  634. \endverbatim
  635. \subsection Implementation Details
  636. While the APIs of the monitoring and the steering frameworks share a similar design philosophy, the internals are significantly different. Since the effect of the steering knobs varies widely, there is no global locking scheme in place shared for all knobs. Instead, each knob gets its own procedures to get the value of a setting, or change it. To prevent code duplication, some related knobs may share getter/setter routines as knob groups.
  637. The steering framework does not involve callback routines. Knob get operations proceed immediately, except for the possible delay in getting access to the knob value. Knob set operations also proceed immediately, not counting the exclusive access time, though their action result may be observed with some latency, depending on the knob and on the current workload. For instance, acting on a per-worker "starpu.worker.w_enable_worker_knob" to disable a worker thread may be observed only after the corresponding worker's assigned task queue becomes empty, since its actual effect is to prevent additional tasks to be queued to the worker, and not to migrate already queued tasks to another worker. Such design choices aim at providing a compromise between offering some steering capabilities and keeping the cost of supporting such steering capabilities to an acceptable level.
  638. The framework is designed to be easily extensible. At StarPU initialization time, the framework calls initialization functions if StarPU modules to initialize the set of knobs they export. Knob get/set accessors can be shared among multiple knobs in a knob group. Thus, exporting a new knob is basically a matter of declaring it at initialization time, by specifying its name and value type, and either add its handling to an existing getter/setter pair of accessors in a knob group, or create a new group. As the performance monitoring framework, the performance steering framework is currently permanently enabled, but could be made optional at compile-time to separate testing builds from production builds.
  639. \subsection Exported Steering Knobs
  640. \subsubsection Global Scope
  641. \verbatim
  642. Knob Name Knob Definition
  643. starpu.global.g_calibrate_knob
  644. Enable/disable the calibration of performance models
  645. starpu.global.g_enable_catch_signal_knob Enable/disable the catching of UNIX signals
  646. \endverbatim
  647. \subsubsection Per-worker Scope
  648. \verbatim
  649. Knob Name Knob Definition
  650. starpu.worker.w_bind_to_pu_knob Change the processing unit to which a worker thread is bound
  651. starpu.worker.w_enable_worker_knob Disable/re-enable a worker thread to be selected
  652. for task execution
  653. \endverbatim
  654. \subsubsection Per-Scheduler Scope
  655. \verbatim
  656. Knob Name Knob Definition
  657. starpu.task.s_max_priority_cap_knob Set a capping maximum priority value for subsequently submitted tasks
  658. starpu.task.s_min_priority_cap_knob Set a capping minimum priority value for subsequently submitted tasks
  659. starpu.dmda.s_alpha_knob Scaling factor for the Alpha constant for Deque Model schedulers to alter the weight of the estimated task execution time
  660. starpu.dmda.s_beta_knob Scaling factor for the Beta constant for Deque Model schedulers to alter the weight of the estimated data transfer time for the task's input(s)
  661. starpu.dmda.s_gamma_knob Scaling factor for the Gamma constant for Deque Model schedulers to alter the weight of the estimated power consumption of the task
  662. starpu.dmda.s_idle_power_knob Scaling factor for the baseline Idle power consumption estimation of the corresponding processing unit
  663. \endverbatim
  664. \subsection Sequence of operations
  665. This section presents an example of sequence of operations representing a typical use of the performance steering knobs exported by StarPU. In this example, a worker thread is temporarily barred from executing tasks. For that, the corresponding "starpu.worker.w_enable_worker_knob" of the worker, initially set to 1 (= enabled) is changed to 0 (= disabled).
  666. <b>Step 0: Initialize StarPU</b>
  667. StarPU must first be initialized, by a call to starpu_init(). Performance steering knobs only become available after this step, since each module of StarPU registers the knobs it exports during that initialization phase.
  668. \verbatim
  669. int ret = starpu_init(NULL);
  670. \endverbatim
  671. <b>Step 1: Get the knob ID</b>
  672. Each performance steering knob has a unique ID used to refer to it in subsequent calls to the performance steering framework. The knob belongs to the "per_worker" scope.
  673. \verbatim
  674. int w_scope = starpu_perf_knob_scope_name_to_id("per_worker");
  675. int w_enable_id = starpu_perf_knob_name_to_id(w_scope,
  676. "starpu.worker.w_enable_worker_knob");
  677. \endverbatim
  678. <b>Step 2: Get the knob current value</b>
  679. This knob is an On/Off knob. Its value type is therefore a 32-bit integer, with value 0 for Off and value !0 for On. The getter functions for per-worker knobs expect the knob ID as first argument, and the worker ID as second argument. Here the getter call obtains the value of worker 5.
  680. \verbatim
  681. int32_t val = starpu_perf_knob_get_per_worker_int32_value(w_enable_id, 5);
  682. \endverbatim
  683. <b>Step 3: Set the knob current value</b>
  684. The setter functions for per-worker knobs expect the knob ID as first argument, the worker ID as second argument, and the new value as third argument. Here, the value for worker 5 is set to 0 to temporarily bar the worker thread from accepting new tasks for execution.
  685. \verbatim
  686. starpu_perf_knob_set_per_worker_int32_value(w_enable_id, 5, 0);
  687. \endverbatim
  688. Subsequently setting the value of the knob back to 1 enables the corresponding to accept new tasks for execution again.
  689. \verbatim
  690. starpu_perf_knob_set_per_worker_int32_value(w_enable_id, 5, 1);
  691. \endverbatim
  692. */