Skip to content

Commit

Permalink
fixed docs
Browse files Browse the repository at this point in the history
  • Loading branch information
zhenghh04 committed Feb 7, 2025
1 parent 178429e commit ada7fee
Showing 1 changed file with 6 additions and 2 deletions.
8 changes: 6 additions & 2 deletions docs/source/config.rst
Original file line number Diff line number Diff line change
Expand Up @@ -41,7 +41,9 @@ The characteristics of a workload is specified through a YAML file. This file wi
epochs_between_checkpoints: 2
model_size: 499153191
A `DLIO` YAML configuration file contains following sections:

* **model** - specifying the name of the model. This is simply an indentifyer of the configuration file. It does not have impact on the actual simulation.
* **framework** - specifying the framework to use for the benchmark, available options: tensorflow, pytorch
* **workflow** - specifying what workflow operations to execute in the pipeline. Workflow operations include: dataset generation (``generate_data``), training (``train``), evaluation (``evaluation``), checkpointing (``checkpoint``), debugging (``debug``), etc.
Expand All @@ -56,7 +58,6 @@ More built-in examples can be found in the `workload`_ folder. One can also crea

model
------------------
One can specify the name of the model as
.. list-table::
:widths: 15 10 30
:header-rows: 1
Expand Down Expand Up @@ -103,12 +104,15 @@ optimization_groups and layer_parameters.
Please note that if optimization_groups and layer_parameters are specified, the transformer configuration will be ignored. But we
always suggest to specify the transformer configuration for better readability.

Please also note that ZeRO stage 3 is not compatiable with ``parallelism.pipeline == 3``.
Please also note that ZeRO stage 3 is not compatiable with ``parallelism.pipeline > 1``.

.. list-table::
:widths: 15 10 30
:header-rows: 1

* - name
- default
- The name of the model
* - hidden_size
- 2048
- Hidden dimension of the transformer layer.
Expand Down

0 comments on commit ada7fee

Please sign in to comment.