Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Docs Update for diction Issues#1056 #1230

Merged
merged 3 commits into from
Dec 10, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion docs/02_notebooks/L0_overview.ipynb
Original file line number Diff line number Diff line change
Expand Up @@ -12,7 +12,7 @@
},
"source": [
"# Overview\n",
"Before we get started, we must first install Tianshou's library and Gym environment by running the commands below. This tutorials will always keep up with the latest version of Tianshou since they also serve as a test for the latest version. If you are using an older version of Tianshou, please refer to the [documentation](https://tianshou.readthedocs.io/en/latest/) of your version.\n"
"To begin, ensure you have Tianshou and the Gym environment installed by executing the following commands. This tutorials will always keep up with the latest version of Tianshou since they also serve as a test for the latest version. For users on older versions of Tianshou, please consult the [documentation](https://tianshou.readthedocs.io/en/latest/) corresponding to your version..\n"
]
},
{
Expand Down
6 changes: 3 additions & 3 deletions docs/02_notebooks/L1_Batch.ipynb
Original file line number Diff line number Diff line change
Expand Up @@ -7,7 +7,7 @@
},
"source": [
"# Batch\n",
"In this tutorial, we will introduce the **Batch** to you, which is the most basic data structure in Tianshou. You can consider Batch as a numpy version of python dictionary. It is also similar to pytorch's tensordict,\n",
"In this tutorial, we will introduce the **Batch** to you, which serves as the fundamental data structure in Tianshou. Think of Batch as a numpy-enhanced version of a Python dictionary. It is also similar to pytorch's tensordict,\n",
"although with a somehow different type structure."
]
},
Expand Down Expand Up @@ -62,7 +62,7 @@
"\n",
"## Why do we need Batch in Tianshou?\n",
"The motivation behind the implementation of Batch module is simple. In DRL, you need to handle a lot of dictionary-format data. For instance, most algorithms would require you to store state, action, and reward data for every step when interacting with the environment. All of them can be organized as a dictionary, and the\n",
" Batch class helps Tianshou in unifying the interfaces of a diverse set of algorithms. In addition, Batch supports advanced indexing, concatenation and splitting, formatting print just like any other numpy array, which proved to be helpful for developers.\n",
" Batch class helps Tianshou in unifying the interfaces of a diverse set of algorithms. In addition, Batch supports advanced indexing, concatenation, and splitting, as well as printing formatted outputs akin to standard numpy arrays, proving invaluable for developers.\n",
"<div align=center>\n",
"<img src=\"https://tianshou.readthedocs.io/en/master/_images/concepts_arch.png\", title=\"The data flow is converted into a Batch in Tianshou\">\n",
"\n",
Expand Down Expand Up @@ -145,7 +145,7 @@
},
"source": [
"### Getting access to data\n",
"You can conveniently search or change the key-value pair in a Batch just as if it were a python dictionary."
"You can effortlessly search for or modify key-value pairs within a Batch, much like interacting with a Python dictionary."
]
},
{
Expand Down
8 changes: 4 additions & 4 deletions docs/02_notebooks/L2_Buffer.ipynb
Original file line number Diff line number Diff line change
Expand Up @@ -22,7 +22,7 @@
},
"source": [
"# Buffer\n",
"Replay Buffer is a very common module in DRL implementations. In Tianshou, you can consider Buffer module as as a specialized form of Batch, which helps you track all data trajectories and provide utilities such as sampling method besides the basic storage.\n",
"Replay Buffer is a very common module in DRL implementations. In Tianshou, the Buffer module can be viewed as a specialized form of Batch, designed to track all data trajectories and offering utilities like sampling methods beyond basic storage.\n",
"\n",
"There are many kinds of Buffer modules in Tianshou, two most basic ones are ReplayBuffer and VectorReplayBuffer. The later one is specially designed for parallelized environments (will introduce in tutorial [Vectorized Environment](https://tianshou.readthedocs.io/en/master/02_notebooks/L3_Vectorized__Environment.html)). In this tutorial, we will focus on ReplayBuffer."
]
Expand All @@ -43,7 +43,7 @@
},
"source": [
"### Basic usages as a batch\n",
"Usually a buffer stores all the data in a batch with circular-queue style."
"Typically, a buffer stores all data in batches, employing a circular-queue mechanism."
]
},
{
Expand Down Expand Up @@ -164,7 +164,7 @@
},
"source": [
"### Data sampling\n",
"We keep a replay buffer in DRL for one purpose: sample data from it for training. `ReplayBuffer.sample()` and `ReplayBuffer.split(..., shuffle=True)` can both fulfill this need."
"The primary purpose of maintaining a replay buffer in DRL is to sample data for training. `ReplayBuffer.sample()` and `ReplayBuffer.split(..., shuffle=True)` can both fulfill this need."
]
},
{
Expand Down Expand Up @@ -395,7 +395,7 @@
"* CachedReplayBuffer, one main buffer with several cached buffers (higher sample efficiency in some scenarios)\n",
"* ReplayBufferManager, A base class that can be inherited (may help you manage multiple buffers).\n",
"\n",
"Check the documentation and the source code for more details.\n",
"Refer to the documentation and source code for further details.\n",
"\n",
"### Support for steps stacking to use RNN in DRL.\n",
"There is an option called `stack_num` (default to 1) when initializing the ReplayBuffer, which may help you use RNN in your algorithm. Check the documentation for details."
Expand Down
4 changes: 2 additions & 2 deletions docs/02_notebooks/L3_Vectorized__Environment.ipynb
Original file line number Diff line number Diff line change
Expand Up @@ -7,7 +7,7 @@
},
"source": [
"# Vectorized Environment\n",
"In reinforcement learning, the agent interacts with environments to improve itself. In this tutorial we will concentrate on the environment part. Although there are many kinds of environments or their libraries in DRL research, Tianshou chooses to keep a consistent API with [OPENAI Gym](https://gym.openai.com/).\n",
"In reinforcement learning, an agent engages with environments to enhance its performance. In this tutorial we will concentrate on the environment part. Although there are many kinds of environments or their libraries in DRL research, Tianshou chooses to keep a consistent API with [OPENAI Gym](https://gym.openai.com/).\n",
"\n",
"<div align=center>\n",
"<img src=\"https://tianshou.readthedocs.io/en/master/_images/rl-loop.jpg\", title=\"The agents interacting with the environment\">\n",
Expand Down Expand Up @@ -200,7 +200,7 @@
"Check the [documentation](https://tianshou.org/en/master/03_api/env/venvs.html) for details.\n",
"\n",
"### Difference between synchronous and asynchronous mode (How to choose?)\n",
"Explanation can be found at the [Parallel Sampling](https://tianshou.org/en/master/01_tutorials/07_cheatsheet.html#parallel-sampling) tutorial."
"For further insights, refer to the [Parallel Sampling](https://tianshou.org/en/master/01_tutorials/07_cheatsheet.html#parallel-sampling) tutorial."
]
}
],
Expand Down
Loading