# Uses of Class
org.apache.sysds.resource.cost.RDDStats

Packages that use RDDStats
Package
Description
org.apache.sysds.resource.cost

## Uses of RDDStats in org.apache.sysds.resource.cost

Methods in org.apache.sysds.resource.cost that return RDDStats
Modifier and Type
Method
Description
`RDDStats`
VarStats.`getRddStats()`
Meant to be used at testing
Methods in org.apache.sysds.resource.cost with parameters of type RDDStats
Modifier and Type
Method
Description
`static double`
SparkCostUtils.`getCPUTime(long nflop, int numPartitions, IOCostUtils.IOMetrics executorMetrics, RDDStats output, RDDStats... inputs)`
Computes an estimate for the time needed by the CPU to execute (including memory access) an instruction by providing number of floating operations.
`static double`
IOCostUtils.`getMemReadTime(RDDStats stats, IOCostUtils.IOMetrics metrics)`
Estimate time to scan distributed data sets in memory on Spark.
`static double`
IOCostUtils.`getMemWriteTime(RDDStats stats, IOCostUtils.IOMetrics metrics)`
Estimate time to write distributed data set on memory in CP.
`static double`
IOCostUtils.`getSparkCollectTime(RDDStats output, IOCostUtils.IOMetrics driverMetrics, IOCostUtils.IOMetrics executorMetrics)`
Estimates the time for collecting Spark Job output; The output RDD is transferred to the Spark driver at the end of each ResultStage; time = transfer time (overlaps and dominates the read and deserialization times);
`static double`
IOCostUtils.`getSparkParallelizeTime(RDDStats output, IOCostUtils.IOMetrics driverMetrics, IOCostUtils.IOMetrics executorMetrics)`
Estimates the time to parallelize a local object to Spark.
`static double`
IOCostUtils.`getSparkShuffleReadStaticTime(RDDStats input, IOCostUtils.IOMetrics metrics)`
Estimates the time for reading distributed RDD input at the beginning of a Stage when a wide-transformation is partition preserving: only local disk reads
`static double`
IOCostUtils.`getSparkShuffleReadTime(RDDStats input, IOCostUtils.IOMetrics metrics)`
Estimates the time for reading distributed RDD input at the beginning of a Stage; time = transfer time (overlaps and dominates the read and deserialization times); For simplification it is assumed that the whole dataset is shuffled;
`static double`
IOCostUtils.`getSparkShuffleTime(RDDStats output, IOCostUtils.IOMetrics metrics, boolean withDistribution)`
Combines the shuffle write and read time since these are being typically added in one place to the general data transmission for instruction estimation.
`static double`
IOCostUtils.`getSparkShuffleWriteTime(RDDStats output, IOCostUtils.IOMetrics metrics)`
Estimates the time for writing the RDD output to the local system at the end of a ShuffleMapStage; time = disk write time (overlaps and dominates the serialization time) The whole data set is being written to shuffle files even if 1 executor is utilized;
`void`
VarStats.`setRddStats(RDDStats rddStats)`
Meant to be used at testing