Skip to content

Commit 32a744d

Browse files
authored
Update ExternalAppendOnlyUnsafeRowArray.scala
1 parent 8023fd3 commit 32a744d

File tree

1 file changed

+5
-5
lines changed

1 file changed

+5
-5
lines changed

sql/core/src/main/scala/org/apache/spark/sql/execution/ExternalAppendOnlyUnsafeRowArray.scala

Lines changed: 5 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -36,14 +36,14 @@ import org.apache.spark.util.collection.unsafe.sort.{UnsafeExternalSorter, Unsaf
3636
* An append-only array for [[UnsafeRow]]s that strictly keeps content in an in-memory array
3737
* until [[numRowsInMemoryBufferThreshold]] or [[sizeInBytesInMemoryBufferThreshold]] is reached
3838
* post which it will switch to a mode (backed by [[UnsafeExternalSorter]]) which would flush to
39-
* disk after [[numRowsSpillThreshold]] is met (or before if there is excessive memory consumption).
40-
* Setting these threshold involves following trade-offs:
39+
* disk after [[numRowsSpillThreshold]] or [[sizeInBytesSpillThreshold]] is met (or before if there
40+
* is excessive memory consumption). Setting these threshold involves following trade-offs:
4141
*
4242
* - If [[numRowsInMemoryBufferThreshold]] and [[sizeInBytesInMemoryBufferThreshold]] are too high,
4343
* the in-memory array may occupy more memory than is available, resulting in OOM.
44-
* - If [[numRowsSpillThreshold]] is too low, data will be spilled frequently and lead to
45-
* excessive disk writes. This may lead to a performance regression compared to the normal case
46-
* of using an [[ArrayBuffer]] or [[Array]].
44+
* - If [[numRowsSpillThreshold]] or [[sizeInBytesSpillThreshold]] is too low, data will be spilled
45+
* frequently and lead to excessive disk writes. This may lead to a performance regression compared
46+
* to the normal case of using an [[ArrayBuffer]] or [[Array]].
4747
*/
4848
class ExternalAppendOnlyUnsafeRowArray(
4949
taskMemoryManager: TaskMemoryManager,

0 commit comments

Comments
 (0)