05check_list_performance.doxy 9.6 KB

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960616263646566676869707172737475767778798081828384858687888990919293949596979899100101102103104105106107108109110111112113114115116117118119120121122123124125126127128129130131132133134135136137138139140141142143144145146147148149150151152153154155156157158159160161162163164165166167168169170171172173174175176177178179180181182183184185186187188189190191192193194195196197198199200201202203204205206207208209210211212213214215216217218219220221222223224225226227228229230231232233234235
  1. /*
  2. * This file is part of the StarPU Handbook.
  3. * Copyright (C) 2009--2011 Universit@'e de Bordeaux 1
  4. * Copyright (C) 2010, 2011, 2012, 2013, 2014 Centre National de la Recherche Scientifique
  5. * Copyright (C) 2011, 2012 Institut National de Recherche en Informatique et Automatique
  6. * See the file version.doxy for copying conditions.
  7. */
  8. /*! \page CheckListWhenPerformanceAreNotThere Check List When Performance Are Not There
  9. TODO: improve!
  10. Simply encapsulating application kernels into tasks already permits to
  11. seamlessly support CPU and GPUs at the same time. To achieve good
  12. performance, we give below a list of features which should be checked.
  13. \section DataRelatedFeaturesToImprovePerformance Data Related Features That May Improve Performance
  14. link to \ref DataManagement
  15. link to \ref DataPrefetch
  16. \section TaskRelatedFeaturesToImprovePerformance Task Related Features That May Improve Performance
  17. link to \ref TaskGranularity
  18. link to \ref TaskSubmission
  19. link to \ref TaskPriorities
  20. \section SchedulingRelatedFeaturesToImprovePerformance Scheduling Related Features That May Improve Performance
  21. link to \ref TaskSchedulingPolicy
  22. link to \ref TaskDistributionVsDataTransfer
  23. link to \ref Power-basedScheduling
  24. link to \ref StaticScheduling
  25. \section CUDA-specificOptimizations CUDA-specific Optimizations
  26. Due to CUDA limitations, StarPU will have a hard time overlapping its own
  27. communications and the codelet computations if the application does not use a
  28. dedicated CUDA stream for its computations instead of the default stream,
  29. which synchronizes all operations of the GPU. StarPU provides one by the use
  30. of starpu_cuda_get_local_stream() which can be used by all CUDA codelet
  31. operations to avoid this issue. For instance:
  32. \code{.c}
  33. func <<<grid,block,0,starpu_cuda_get_local_stream()>>> (foo, bar);
  34. cudaStreamSynchronize(starpu_cuda_get_local_stream());
  35. \endcode
  36. Unfortunately, some CUDA libraries do not have stream variants of
  37. kernels. That will lower the potential for overlapping.
  38. StarPU already does appropriate calls for the CUBLAS library.
  39. If the kernel can be made to only use this local stream or other self-allocated
  40. streams, i.e. the whole kernel submission can be made asynchronous, then
  41. one should enable asynchronous execution of the kernel. That means setting
  42. the STARPU_CUDA_ASYNC flag in cuda_flags[] in the codelet, and dropping the
  43. cudaStreamSynchronize() call at the end of the cuda_func function, so that it
  44. returns immediately after having queued the kernel to the local stream. That way, StarPU will be
  45. able to submit and complete data transfers while kernels are executing, instead of only at each
  46. kernel submission. The kernel just has to make sure that StarPU can use the
  47. local stream to synchronize with the kernel startup and completion.
  48. Using the STARPU_CUDA_ASYNC flag also permits to enable concurrent kernel
  49. execution, on cards which support it (Kepler and later, notably). This is
  50. enabled by setting the STARPU_NWORKER_PER_CUDA environment variable to the
  51. number of kernels to execute concurrently. This is useful when kernels are
  52. small and do not feed the whole GPU with threads to run.
  53. \section OpenCL-specificOptimizations OpenCL-specific Optimizations
  54. If the kernel can be made to only use the StarPU-provided command queue or other self-allocated
  55. queues, i.e. the whole kernel submission can be made asynchronous, then
  56. one should enable asynchronous execution of the kernel. This means setting
  57. the corresponding opencl_flags[] flag in the codelet and dropping the
  58. clFinish() and starpu_opencl_collect_stats() calls at the end of the kernel, so
  59. that it returns immediately after having queued the kernel to the provided queue.
  60. That way, StarPU will be able to submit and complete data transfers while kernels are executing, instead of
  61. only at each kernel submission. The kernel just has to make sure
  62. that StarPU can use the command queue it has provided to synchronize with the
  63. kernel startup and completion.
  64. \section DetectionStuckConditions Detection Stuck Conditions
  65. It may happen that for some reason, StarPU does not make progress for a long
  66. period of time. Reason are sometimes due to contention inside StarPU, but
  67. sometimes this is due to external reasons, such as stuck MPI driver, or CUDA
  68. driver, etc.
  69. <c>export STARPU_WATCHDOG_TIMEOUT=10000</c>
  70. allows to make StarPU print an error message whenever StarPU does not terminate
  71. any task for 10ms. In addition to that,
  72. <c>export STARPU_WATCHDOG_CRASH=1</c>
  73. triggers a crash in that condition, thus allowing to catch the situation in gdb
  74. etc.
  75. \section HowToLimitMemoryPerNode How to limit memory per node
  76. TODO
  77. Talk about
  78. \ref STARPU_LIMIT_CUDA_devid_MEM, \ref STARPU_LIMIT_CUDA_MEM,
  79. \ref STARPU_LIMIT_OPENCL_devid_MEM, \ref STARPU_LIMIT_OPENCL_MEM
  80. and \ref STARPU_LIMIT_CPU_MEM
  81. starpu_memory_get_total()
  82. starpu_memory_get_available()
  83. \section PerformanceModelCalibration Performance Model Calibration
  84. Most schedulers are based on an estimation of codelet duration on each kind
  85. of processing unit. For this to be possible, the application programmer needs
  86. to configure a performance model for the codelets of the application (see
  87. \ref PerformanceModelExample for instance). History-based performance models
  88. use on-line calibration. StarPU will automatically calibrate codelets
  89. which have never been calibrated yet, and save the result in
  90. <c>$STARPU_HOME/.starpu/sampling/codelets</c>.
  91. The models are indexed by machine name. To share the models between
  92. machines (e.g. for a homogeneous cluster), use <c>export
  93. STARPU_HOSTNAME=some_global_name</c>. To force continuing calibration,
  94. use <c>export STARPU_CALIBRATE=1</c> . This may be necessary if your application
  95. has not-so-stable performance. StarPU will force calibration (and thus ignore
  96. the current result) until 10 (<c>_STARPU_CALIBRATION_MINIMUM</c>) measurements have been
  97. made on each architecture, to avoid badly scheduling tasks just because the
  98. first measurements were not so good. Details on the current performance model status
  99. can be obtained from the command <c>starpu_perfmodel_display</c>: the <c>-l</c>
  100. option lists the available performance models, and the <c>-s</c> option permits
  101. to choose the performance model to be displayed. The result looks like:
  102. \verbatim
  103. $ starpu_perfmodel_display -s starpu_slu_lu_model_11
  104. performance model for cpu_impl_0
  105. # hash size flops mean dev n
  106. 914f3bef 1048576 0.000000e+00 2.503577e+04 1.982465e+02 8
  107. 3e921964 65536 0.000000e+00 5.527003e+02 1.848114e+01 7
  108. e5a07e31 4096 0.000000e+00 1.717457e+01 5.190038e+00 14
  109. ...
  110. \endverbatim
  111. Which shows that for the LU 11 kernel with a 1MiB matrix, the average
  112. execution time on CPUs was about 25ms, with a 0.2ms standard deviation, over
  113. 8 samples. It is a good idea to check this before doing actual performance
  114. measurements.
  115. A graph can be drawn by using the tool <c>starpu_perfmodel_plot</c>:
  116. \verbatim
  117. $ starpu_perfmodel_plot -s starpu_slu_lu_model_11
  118. 4096 16384 65536 262144 1048576 4194304
  119. $ gnuplot starpu_starpu_slu_lu_model_11.gp
  120. $ gv starpu_starpu_slu_lu_model_11.eps
  121. \endverbatim
  122. \image html starpu_starpu_slu_lu_model_11.png
  123. \image latex starpu_starpu_slu_lu_model_11.eps "" width=\textwidth
  124. If a kernel source code was modified (e.g. performance improvement), the
  125. calibration information is stale and should be dropped, to re-calibrate from
  126. start. This can be done by using <c>export STARPU_CALIBRATE=2</c>.
  127. Note: due to CUDA limitations, to be able to measure kernel duration,
  128. calibration mode needs to disable asynchronous data transfers. Calibration thus
  129. disables data transfer / computation overlapping, and should thus not be used
  130. for eventual benchmarks. Note 2: history-based performance models get calibrated
  131. only if a performance-model-based scheduler is chosen.
  132. The history-based performance models can also be explicitly filled by the
  133. application without execution, if e.g. the application already has a series of
  134. measurements. This can be done by using starpu_perfmodel_update_history(),
  135. for instance:
  136. \code{.c}
  137. static struct starpu_perfmodel perf_model = {
  138. .type = STARPU_HISTORY_BASED,
  139. .symbol = "my_perfmodel",
  140. };
  141. struct starpu_codelet cl = {
  142. .cuda_funcs = { cuda_func1, cuda_func2, NULL },
  143. .nbuffers = 1,
  144. .modes = {STARPU_W},
  145. .model = &perf_model
  146. };
  147. void feed(void) {
  148. struct my_measure *measure;
  149. struct starpu_task task;
  150. starpu_task_init(&task);
  151. task.cl = &cl;
  152. for (measure = &measures[0]; measure < measures[last]; measure++) {
  153. starpu_data_handle_t handle;
  154. starpu_vector_data_register(&handle, -1, 0, measure->size, sizeof(float));
  155. task.handles[0] = handle;
  156. starpu_perfmodel_update_history(&perf_model, &task,
  157. STARPU_CUDA_DEFAULT + measure->cudadev, 0,
  158. measure->implementation, measure->time);
  159. starpu_task_clean(&task);
  160. starpu_data_unregister(handle);
  161. }
  162. }
  163. \endcode
  164. Measurement has to be provided in milliseconds for the completion time models,
  165. and in Joules for the energy consumption models.
  166. \section Profiling Profiling
  167. A quick view of how many tasks each worker has executed can be obtained by setting
  168. <c>export STARPU_WORKER_STATS=1</c> This is a convenient way to check that
  169. execution did happen on accelerators, without penalizing performance with
  170. the profiling overhead.
  171. A quick view of how much data transfers have been issued can be obtained by setting
  172. <c>export STARPU_BUS_STATS=1</c> .
  173. More detailed profiling information can be enabled by using <c>export STARPU_PROFILING=1</c> or by
  174. calling starpu_profiling_status_set() from the source code.
  175. Statistics on the execution can then be obtained by using <c>export
  176. STARPU_BUS_STATS=1</c> and <c>export STARPU_WORKER_STATS=1</c> .
  177. More details on performance feedback are provided by the next chapter.
  178. */