|
|
@@ -1,3 +1,17 @@
|
|
|
+## Generating the 300 SPARK configurations
|
|
|
+In order to generate the 300 spark configurations, we use the OpenTuner library. The available range of values for all the parameters can be found in ```src/python/spark-config-generator.py``` file.
|
|
|
+To generate the 300 configs we execute the following:
|
|
|
+```bash
|
|
|
+mkdir configs
|
|
|
+for i in $(seq 1 1 300)
|
|
|
+do
|
|
|
+ python3 spark-config-generator.py
|
|
|
+ mv temp configs/$i
|
|
|
+done
|
|
|
+```
|
|
|
+The process may take a while. The 300 configurations will be available in the configs folder.
|
|
|
+
|
|
|
+## Known issues
|
|
|
### spark-config-generator.py
|
|
|
This script generates a random SPARK configuration based on 107 SPARK parameters using the opentuner library.
|
|
|
|
