New environment variables STARPU_BUS_STATS_FILE and STARPU_WORKER_STATS_FILE to specify files in which to display statistics about data transfers and workers.

ChangeLog

@@ -2,7 +2,7 @@
 #
 # Copyright (C) 2012-2014,2016-2018                      Inria
 # Copyright (C) 2009-2019                                Université de Bordeaux
-# Copyright (C) 2010-2019                                CNRS
+# Copyright (C) 2010-2020                                CNRS
 #
 # StarPU is free software; you can redistribute it and/or modify
 # it under the terms of the GNU Lesser General Public License as published by
@@ -30,6 +30,14 @@ New features:
 Small changes:
   * Use the S4U interface of Simgrid instead of xbt and MSG.
 
+StarPU 1.3.4 (git revision xxx)
+==============================================
+
+Small features:
+  * New environment variables STARPU_BUS_STATS_FILE and
+    STARPU_WORKER_STATS_FILE to specify files in which to display
+    statistics about data transfers and workers.
+
 StarPU 1.3.3 (git revision xxx)
 ==============================================
 

doc/doxygen/chapters/210_check_list_performance.doxy

@@ -1,7 +1,7 @@
 /* StarPU --- Runtime system for heterogeneous multicore architectures.
  *
  * Copyright (C) 2011-2013,2015,2017                      Inria
- * Copyright (C) 2010-2019                                CNRS
+ * Copyright (C) 2010-2020                                CNRS
  * Copyright (C) 2009-2011,2013-2019                      Université de Bordeaux
  *
  * StarPU is free software; you can redistribute it and/or modify
@@ -440,10 +440,14 @@ and in Joules for the energy consumption models.
 A quick view of how many tasks each worker has executed can be obtained by setting
 <c>export STARPU_WORKER_STATS=1</c> (\ref STARPU_WORKER_STATS). This is a convenient way to check that
 execution did happen on accelerators, without penalizing performance with
-the profiling overhead.
+the profiling overhead. \ref STARPU_WORKER_STATS_FILE can be defined
+to specify a filename in which to display statistics, by default
+statistics are printed on the standard error stream.
 
 A quick view of how much data transfers have been issued can be obtained by setting
-<c>export STARPU_BUS_STATS=1</c> (\ref STARPU_BUS_STATS).
+<c>export STARPU_BUS_STATS=1</c> (\ref STARPU_BUS_STATS). \ref
+STARPU_BUS_STATS_FILE can be defined to specify a filename in which to
+display statistics, by default statistics are printed on the standard error stream.
 
 More detailed profiling information can be enabled by using <c>export STARPU_PROFILING=1</c> (\ref STARPU_PROFILING)
 or by

doc/doxygen/chapters/320_scheduling.doxy

@@ -1,6 +1,6 @@
 /* StarPU --- Runtime system for heterogeneous multicore architectures.
  *
- * Copyright (C) 2010-2019                                CNRS
+ * Copyright (C) 2010-2020                                CNRS
  * Copyright (C) 2011,2012,2016                           Inria
  * Copyright (C) 2009-2011,2014-2019                      Université de Bordeaux
  *
@@ -168,7 +168,7 @@ processing units), the idle power of the machine should be given by setting
 be obtained from the machine power supplier.
 
 The energy actually consumed by the total execution can be displayed by setting
-<c>export STARPU_PROFILING=1 STARPU_WORKER_STATS=1</c> .
+<c>export STARPU_PROFILING=1 STARPU_WORKER_STATS=1</c> (\ref STARPU_PROFILING and \ref STARPU_WORKER_STATS).
 
 For OpenCL devices, on-line task consumption measurement is currently supported through the
 <c>CL_PROFILING_POWER_CONSUMED</c> OpenCL extension, implemented in the MoviSim

doc/doxygen/chapters/370_online_performance_tools.doxy

@@ -1,7 +1,7 @@
 /* StarPU --- Runtime system for heterogeneous multicore architectures.
  *
  * Copyright (C) 2011,2012,2016                           Inria
- * Copyright (C) 2010-2019                                CNRS
+ * Copyright (C) 2010-2020                                CNRS
  * Copyright (C) 2009-2011,2014,2016,2018-2019            Université de Bordeaux
  *
  * StarPU is free software; you can redistribute it and/or modify
@@ -94,7 +94,9 @@ information associated to a worker.
 
 To easily display all this information, the environment variable
 \ref STARPU_WORKER_STATS can be set to <c>1</c> (in addition to setting
-\ref STARPU_PROFILING to 1). A summary will then be displayed at program termination:
+\ref STARPU_PROFILING to 1). A summary will then be displayed at
+program termination.  To display the summary in a file instead of the
+standard error stream, use the environment variable \ref STARPU_WORKER_STATS_FILE.
 
 \verbatim
 Worker stats:
@@ -150,7 +152,9 @@ CUDA 2  4534.229519     2417.069025     2417.060863     0.000000
 
 Statistics about the data transfers which were performed and temporal average
 of bandwidth usage can be obtained by setting the environment variable
-\ref STARPU_BUS_STATS to <c>1</c>; a summary will then be displayed at program termination:
+\ref STARPU_BUS_STATS to <c>1</c>; a summary will then be displayed at
+program termination. To display the summary in a file instead of the
+standard error stream, use the environment variable \ref STARPU_BUS_STATS_FILE.
 
 \verbatim
 Data transfer stats:

doc/doxygen/chapters/401_out_of_core.doxy

@@ -1,6 +1,6 @@
 /* StarPU --- Runtime system for heterogeneous multicore architectures.
  *
- * Copyright (C) 2013,2014,2016-2019                      CNRS
+ * Copyright (C) 2013,2014,2016-2020                      CNRS
  * Copyright (C) 2013,2014,2017,2018-2019                 Université de Bordeaux
  * Copyright (C) 2013                                     Inria
  * Copyright (C) 2013                                     Corentin Salingue
@@ -91,7 +91,7 @@ system daemons, and application data).
 When the register call is made, StarPU will benchmark the disk. This can
 take some time.
 
-<strong>Warning: the size thus has to be at least \ref STARPU_DISK_SIZE_MIN bytes ! </strong> 
+<strong>Warning: the size thus has to be at least \ref STARPU_DISK_SIZE_MIN bytes ! </strong>
 
 StarPU will then automatically try to evict unused data to this new disk. One
 can also use the standard StarPU memory node API to prefetch data etc., see the
@@ -127,7 +127,7 @@ value right after this call, and thus the very first task using the handle needs
 to use the ::STARPU_W mode like above, ::STARPU_R or ::STARPU_RW would not make
 sense.
 
-By default, StarPU will try to push any data handle to the disk. 
+By default, StarPU will try to push any data handle to the disk.
 To specify whether a given handle should be pushed to the disk,
 starpu_data_set_ooc_flag() should be used.
 
@@ -174,7 +174,9 @@ work on this area in the coming future.
 
 Beyond pure performance feedback, some figures are interesting to have a look at.
 
-Using <c>export STARPU_BUS_STATS=1</c> gives an overview of the data
+Using <c>export STARPU_BUS_STATS=1</c> (\ref STARPU_BUS_STATS and \ref STARPU_BUS_STATS_FILE
+to define a filename in which to display statistics, by default the
+standard error stream is used) gives an overview of the data
 transfers which were needed. The values can also be obtained at runtime
 by using starpu_bus_get_profiling_info(). An example can be read in
 <c>src/profiling/profiling_helpers.c</c>.
@@ -188,7 +190,7 @@ Data transfer speed for /tmp/sthibault-disk-DJzhAj (node 1):
 1 -> 0: 23858 µs
 
 #---------------------
-TEST DISK MEMORY 
+TEST DISK MEMORY
 
 #---------------------
 Data transfer stats:

doc/doxygen/chapters/501_environment_variables.doxy

@@ -1,7 +1,7 @@
 /* StarPU --- Runtime system for heterogeneous multicore architectures.
  *
  * Copyright (C) 2011-2013,2015-2017                      Inria
- * Copyright (C) 2010-2019                                CNRS
+ * Copyright (C) 2010-2020                                CNRS
  * Copyright (C) 2009-2011,2013-2020                      Université de Bordeaux
  * Copyright (C) 2016                                     Uppsala University
  *
@@ -1093,7 +1093,17 @@ StarPU for internal data structures during execution.
 \anchor STARPU_BUS_STATS
 \addindex __env__STARPU_BUS_STATS
 When defined, statistics about data transfers will be displayed when calling
-starpu_shutdown() (\ref Profiling).
+starpu_shutdown() (\ref Profiling). By default, statistics are printed
+on the standard error stream, use the environement variable \ref
+STARPU_BUS_STATS_FILE to define another filename.
+</dd>
+
+<dt>STARPU_BUS_STATS_FILE</dt>
+<dd>
+\anchor STARPU_BUS_STATS_FILE
+\addindex __env__STARPU_BUS_STATS_FILE
+Define the name of the file where to display data transfers
+statistics, see \ref STARPU_BUS_STATS.
 </dd>
 
 <dt>STARPU_WORKER_STATS</dt>
@@ -1103,7 +1113,17 @@ starpu_shutdown() (\ref Profiling).
 When defined, statistics about the workers will be displayed when calling
 starpu_shutdown() (\ref Profiling). When combined with the
 environment variable \ref STARPU_PROFILING, it displays the energy
-consumption (\ref Energy-basedScheduling).
+consumption (\ref Energy-basedScheduling).  By default, statistics are
+printed on the standard error stream, use the environement variable
+\ref STARPU_WORKER_STATS_FILE to define another filename.
+</dd>
+
+<dt>STARPU_WORKER_STATS_FILE</dt>
+<dd>
+\anchor STARPU_WORKER_STATS_FILE
+\addindex __env__STARPU_WORKER_STATS_FILE
+Define the name of the file where to display workers statistics, see
+\ref STARPU_WORKER_STATS.
 </dd>
 
 <dt>STARPU_STATS</dt>

src/profiling/profiling_helpers.c

@@ -1,8 +1,8 @@
 /* StarPU --- Runtime system for heterogeneous multicore architectures.
  *
  * Copyright (C) 2016,2017                                Inria
- * Copyright (C) 2010-2013,2016,2017                      CNRS
- * Copyright (C) 2010,2011,2013-2016, 2019                      Université de Bordeaux
+ * Copyright (C) 2010-2013,2016,2017,2020                 CNRS
+ * Copyright (C) 2010,2011,2013-2016, 2019                Université de Bordeaux
  *
  * StarPU is free software; you can redistribute it and/or modify
  * it under the terms of the GNU Lesser General Public License as published by
@@ -63,7 +63,7 @@ void _starpu_profiling_bus_helper_display_summary(FILE *stream)
 
 		sum_transferred += transferred;
 	}
-	
+
 	double d = convert_to_GB(sum_transferred);
 
 	fprintf(stream, "Total transfers: %.4lf %s\n", d, "GB");
@@ -74,7 +74,17 @@ void starpu_profiling_bus_helper_display_summary(void)
 {
 	const char *stats;
 	if (!((stats = starpu_getenv("STARPU_BUS_STATS")) && atoi(stats))) return;
-	_starpu_profiling_bus_helper_display_summary(stderr);
+	const char *filename = starpu_getenv("STARPU_BUS_STATS_FILE");
+	if (filename==NULL)
+		_starpu_profiling_bus_helper_display_summary(stderr);
+	else
+	{
+		FILE *sfile = fopen(filename, "w+");
+		STARPU_ASSERT_MSG(sfile, "Could not open file %s for displaying bus stats (%s). You can specify another file destination with the STARPU_BUS_STATS_FILE environment variable", filename, strerror(errno));
+		_starpu_profiling_bus_helper_display_summary(sfile);
+		fclose(sfile);
+	}
+
 }
 
 void _starpu_profiling_worker_helper_display_summary(FILE *stream)
@@ -140,5 +150,14 @@ void starpu_profiling_worker_helper_display_summary(void)
 {
 	const char *stats;
 	if (!((stats = starpu_getenv("STARPU_WORKER_STATS")) && atoi(stats))) return;
-	_starpu_profiling_worker_helper_display_summary(stderr);
+	const char *filename = starpu_getenv("STARPU_WORKER_STATS_FILE");
+	if (filename==NULL)
+		_starpu_profiling_worker_helper_display_summary(stderr);
+	else
+	{
+		FILE *sfile = fopen(filename, "w+");
+		STARPU_ASSERT_MSG(sfile, "Could not open file %s for displaying worker stats (%s). You can specify another file destination with the STARPU_WORKER_STATS_FILE environment variable", filename, strerror(errno));
+		_starpu_profiling_worker_helper_display_summary(sfile);
+		fclose(sfile);
+	}
 }