|
|
@@ -753,66 +753,90 @@ to a less optimal solution. This increases even more computation time.
|
|
|
|
|
|
\section starvz Trace visualization with StarVZ
|
|
|
|
|
|
-Creating views with StarVZ (see: https://github.com/schnorr/starvz) is made up of two steps. The initial
|
|
|
-stage consists of a pre-processing of the traces generated by the application.
|
|
|
-The second step consists of the analysis itself and is carried out with the
|
|
|
-aid of R packages. To download and install StarVZ, it is necessary to have R,
|
|
|
-pajeng and the following packages:
|
|
|
+Creating views with StarVZ (see: https://github.com/schnorr/starvz) is
|
|
|
+made up of two steps. The initial stage consists of a pre-processing
|
|
|
+of the traces generated by the application, while the second one
|
|
|
+consists of the analysis itself and is carried out with R packages'
|
|
|
+aid. StarVZ is available at CRAN
|
|
|
+(https://cran.r-project.org/package=starvz) and depends on pj_dump
|
|
|
+(from pajeng) and rec2csv (from recutils).
|
|
|
+
|
|
|
+To download and install StarVZ, it is necessary to have R,
|
|
|
+pajeng, and recutils:
|
|
|
|
|
|
\verbatim
|
|
|
-# For pajeng
|
|
|
-apt install -y git cmake build-essential libboost-dev asciidoc flex bison
|
|
|
-git clone git://github.com/schnorr/pajeng.git
|
|
|
-mkdir -p pajeng/b ; cd pajeng/b
|
|
|
-cmake ..
|
|
|
-make
|
|
|
+# For pj_dump and rec2csv
|
|
|
+apt install -y pajeng recutils
|
|
|
|
|
|
-# For R tidyverse
|
|
|
+# For R
|
|
|
apt install -y r-base libxml2-dev libssl-dev libcurl4-openssl-dev libgit2-dev libboost-dev
|
|
|
\endverbatim
|
|
|
|
|
|
-To install the StarVZ the following commands can be used:
|
|
|
+To install the StarVZ, the following command can be used:
|
|
|
|
|
|
\verbatim
|
|
|
-git clone https://github.com/schnorr/starvz.git
|
|
|
-echo "install.packages(c('tidyverse', 'devtools'), repos = 'https://cloud.r-project.org')" | R --vanilla
|
|
|
-echo "library(devtools); devtools::install_local(path='./starvz/R_package')" | R --vanilla
|
|
|
+echo "install.packages('starvz', repos = 'https://cloud.r-project.org')" | R --vanilla
|
|
|
\endverbatim
|
|
|
|
|
|
-To generate traces from an application, it is necessary to set \ref STARPU_GENERATE_TRACE.
|
|
|
-and build StarPU with FxT. Then, Step 1 of StarVZ can be used on a folder with
|
|
|
-StarPU FxT traces:
|
|
|
+To generate traces from an application, it is necessary to set \ref STARPU_GENERATE_TRACE
|
|
|
+and build StarPU with FxT. Then, StarVZ can be used on a folder with
|
|
|
+StarPU FxT traces to produce a default view:
|
|
|
+
|
|
|
+\verbatim
|
|
|
+export PATH=$(Rscript -e 'cat(system.file("tools/", package = "starvz"), sep="\n")'):$PATH
|
|
|
+
|
|
|
+starvz /foo/path-to-fxt-files
|
|
|
+\endverbatim
|
|
|
+
|
|
|
+An example of default view:
|
|
|
+
|
|
|
+\image html starvz_visu.png
|
|
|
+\image latex starvz_visu.pdf "" width=\textwidth
|
|
|
|
|
|
+One can also use existing trace files (paje.trace, tasks.rec,
|
|
|
+data.rec, papi.rec and dag.dot) skipping the StarVZ internal call to
|
|
|
+starpu_fxt_tool with:
|
|
|
\verbatim
|
|
|
-export PATH=starvz/:$PATH
|
|
|
-export PATH=pajeng/b:$PATH
|
|
|
-export PATH=$STARPU_HOME/bin:$PATH
|
|
|
+starvz --use-paje-trace /foo/path-to-trace-files
|
|
|
+\endverbatim
|
|
|
|
|
|
-./starvz/src/phase1-workflow.sh /tmp/ ""
|
|
|
+Alternatively, each StarVZ step can be executed separately. Step 1 can
|
|
|
+be used on a folder with:
|
|
|
+\verbatim
|
|
|
+starvz -1 /foo/path-to-fxt-files
|
|
|
\endverbatim
|
|
|
|
|
|
-Then the second step can be executed directly in R, StarVZ enables a set of
|
|
|
-different plots that can be configured on a .yaml file. A default file is provided
|
|
|
-<c>full_config.yaml</c>; also the options can be changed directly in R.
|
|
|
+Then the second step can be
|
|
|
+executed directly in R. StarVZ enables a set of different plots that
|
|
|
+can be configured on a .yaml file. A default file is provided
|
|
|
+(<c>default.yaml</c>); also, the options can be changed directly in
|
|
|
+R.
|
|
|
|
|
|
\verbatim
|
|
|
library(starvz)
|
|
|
-dtrace <- the_fast_reader_function("./")
|
|
|
+library(dplyr)
|
|
|
+
|
|
|
+dtrace <- starvz_read("./", selective = FALSE)
|
|
|
|
|
|
-pajer <- config::get(file = "starvz/full_config.yaml")
|
|
|
+# show idleness ratio
|
|
|
+dtrace$config$st$idleness = TRUE
|
|
|
|
|
|
-pajer$starpu$active = TRUE
|
|
|
-pajer$submitted$active = TRUE
|
|
|
-pajer$st$abe$active = TRUE
|
|
|
+# show ABE bound
|
|
|
+dtrace$config$st$abe$active = TRUE
|
|
|
|
|
|
-plot <- the_master_function(dtrace)
|
|
|
+# find the last task with dplyr
|
|
|
+dtrace$config$st$tasks$list = dtrace$Application %>% filter(End == max(End)) %>% .$JobId
|
|
|
+# show last task dependencies
|
|
|
+dtrace$config$st$tasks$active = TRUE
|
|
|
+dtrace$config$st$tasks$levels = 50
|
|
|
+
|
|
|
+plot <- starvz_plot(dtrace)
|
|
|
\endverbatim
|
|
|
|
|
|
An example of visualization follows:
|
|
|
|
|
|
-\image html starvz_visu.png
|
|
|
-\image latex starvz_visu.eps "" width=\textwidth
|
|
|
-
|
|
|
+\image html starvz_visu_r.png
|
|
|
+\image latex starvz_visu_r.pdf "" width=\textwidth
|
|
|
|
|
|
\section MemoryFeedback Memory Feedback
|
|
|
|
Vinicius Garcia Pinto
