Skip to content

Commit a186aa0

Browse files
swpfYZeyi-Lin
andauthored
docs: access swanlab cloud on compute node (#107)
* docs: access swanlab cloud on compute node 新增在内网计算节点上访问SwanLab Cloud的方法 * docs: add example for explanation * tmux code * move location --------- Co-authored-by: ZeYi Lin <[email protected]> Co-authored-by: Ze-Yi LIN <[email protected]>
1 parent 7c66964 commit a186aa0

File tree

6 files changed

+121
-0
lines changed

6 files changed

+121
-0
lines changed

.vitepress/en.ts

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -133,6 +133,7 @@ function sidebarGuideCloud(): DefaultTheme.SidebarItem[] {
133133
{ text: 'Jupyter Notebook', link: 'experiment_track/jupyter-notebook' },
134134
{ text: 'Limitations and Performance', link: 'experiment_track/limit-and-performance' },
135135
{ text: 'Experiment metadata', link: 'experiment_track/experiment-metadata' },
136+
{ text: 'Access SwanLab on internal computing nodes', link: 'experiment_track/ssh-portforwarding' },
136137
{ text: 'FAQ', link: 'experiment_track/FAQ' },
137138
]
138139
},

.vitepress/zh.ts

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -154,7 +154,9 @@ function sidebarGuideCloud(): DefaultTheme.SidebarItem[] {
154154
{ text: '用Jupyter Notebook跟踪实验', link: 'experiment_track/jupyter-notebook' },
155155
{ text: '限制与性能', link: 'experiment_track/limit-and-performance' },
156156
{ text: '实验元数据', link: 'experiment_track/experiment-metadata' },
157+
{ text: '内网计算节点访问SwanLab', link: 'experiment_track/ssh-portforwarding' },
157158
{ text: '常见问题', link: 'experiment_track/FAQ' },
159+
158160
]
159161
},
160162
{
Lines changed: 59 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,59 @@
1+
# Accessing SwanLab Cloud from Internal Computing Nodes
2+
3+
Typically, computing nodes in a computing cluster cannot connect to the internet, and external development machines must connect to computing nodes through a jump server. If you cannot connect to the public network, you won't be able to upload data to SwanLab Cloud. However, since the jump server acts as an "intermediary" that can connect to the internet, we can utilize it to proxy the computing nodes' connection to the public network environment.
4+
5+
<img src="./ssh-portforwarding/cluster-network_en.png" alt="cluster-network" style="zoom:30%;" />
6+
7+
We can use SSH proxy forwarding to enable computing nodes to connect to [SwanLab Cloud](https://swanlab.cn/).
8+
9+
## Enable Proxy Forwarding Network
10+
11+
> Ensure that your computing node can connect to the jump server via SSH
12+
13+
Execute the following command on the computing node to connect to the jump server:
14+
15+
```bash
16+
ssh -D {port} {user}@{ip}
17+
```
18+
19+
- `port` parameter is the port used for proxy forwarding, for example `2015`
20+
- `user` and `ip` parameters are the username and internal IP address of the jump server
21+
22+
For example: `ssh -D 2015 [email protected]`
23+
24+
After successfully connecting to the jump server, a SOCKS proxy tunnel is opened on the corresponding port. You can then configure the proxy by setting environment variables in the terminal, for example:
25+
26+
```bash
27+
export http_proxy=socks5://127.0.0.1:{port} https_proxy=socks5://127.0.0.1:{port}
28+
```
29+
30+
> Note: Replace the corresponding `port` with your configured port, the protocol is [socks5](https://en.wikipedia.org/wiki/SOCKS)
31+
32+
After configuration, you can test if you're correctly connected to the public network using the following command:
33+
34+
```bash
35+
curl ipinfo.io
36+
```
37+
38+
Once configured successfully, you can happily use SwanLab Cloud version 🥳.
39+
40+
Note that the SSH connection must not be disconnected, and closing the terminal session will cause the connection to drop. You can use [tmux](https://github.com/tmux/tmux/wiki) to keep the SSH connection command running in the background.
41+
42+
```bash
43+
tmux
44+
# Execute the SSH connection command in tmux
45+
ssh -D {port} {user}@{ip}
46+
```
47+
48+
New terminal sessions require reconfiguring environment variables. Of course, you can write the above environment variable export commands into the `.bashrc` file to automatically set the environment variables each time a new terminal session is opened. For example:
49+
```bash
50+
echo "export http_proxy=socks5://127.0.0.1:{port}" >> ~/.bashrc
51+
echo "export https_proxy=socks5://127.0.0.1:{port}" >> ~/.bashrc
52+
```
53+
> Note Replace '{port}' with the port you set
54+
55+
## Working Principle
56+
57+
The above implementation relies on [SSH Dynamic Port Forwarding](https://en.wikipedia.org/wiki/Port_forwarding#Dynamic_port_forwarding), which turns the SSH server into a SOCKS proxy server that applications on your computer can use as an intermediary to connect to remote servers.
58+
59+
> **Note:** The program must support SOCKS type proxies for you to be able to route traffic from that application using dynamic port forwarding.
Loading
Lines changed: 59 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,59 @@
1+
# 在内网计算节点访问SwanLab Cloud
2+
3+
通常,在算力集群中计算节点无法连接到互联网,外部开发机也必须通过跳板机才能连接到计算节点。如果无法连接到公网那就无法将数据上传到 SwanLab 云端。但是跳板机作为“中间人”角色,可以连接到互联网,那么就可以利用跳板机来实现代理计算节点连接到公网环境。
4+
5+
<img src="./ssh-portforwarding/cluster-network.png" alt="cluster-network" style="zoom:30%;" />
6+
7+
我们可以通过使用SSH代理转发来实现让计算节点也能连接上 [SwanLab Cloud](https://swanlab.cn/)
8+
9+
## 开启代理转发网络
10+
11+
> 确保你的计算节点能通过SSH连接上跳板机
12+
13+
在计算节点上执行以下命令连接到跳板机:
14+
15+
```bash
16+
ssh -D {port} {user}@{ip}
17+
```
18+
19+
- `port` 参数为用于代理转发的端口,例如 `2015`
20+
- `user``ip` 参数为跳板机服务器对应的用户名和内网IP地址
21+
22+
例如:`ssh -D 2015 [email protected]`
23+
24+
连接到跳板机成功后,即在对应的端口开启了一个SOCKS代理通道,那么可以直接在终端设置环境变量来配置代理,例如:
25+
26+
```bash
27+
export http_proxy=socks5://127.0.0.1:{port} https_proxy=socks5://127.0.0.1:{port}
28+
```
29+
30+
> 注意将对应的 `port` 更换为自己设置的端口,协议为 [socks5](https://en.wikipedia.org/wiki/SOCKS)
31+
32+
配置成功后可以使用以下命令测试是否正确连接到公网:
33+
34+
```bash
35+
curl ipinfo.io
36+
```
37+
38+
配置成功后就可以愉快地使用SwanLab云端版了🥳。
39+
40+
注意SSH连接不能断开,关闭终端会话会导致连接断开,那么可以使用 [tmux](https://github.com/tmux/tmux/wiki) 将SSH连接命令放置在后台。
41+
42+
```bash
43+
tmux
44+
# 在tmux中执行SSH连接命令
45+
ssh -D {port} {user}@{ip}
46+
```
47+
48+
新开终端会话必须重新配置环境变量,当然可以将上述导入环境变量的命令写入 `.bashrc` 文件中实现每次开启新终端会话时自动写入环境变量。例如:
49+
```bash
50+
echo "export http_proxy=socks5://127.0.0.1:{port}" >> ~/.bashrc
51+
echo "export https_proxy=socks5://127.0.0.1:{port}" >> ~/.bashrc
52+
```
53+
> 注意将 `{port}` 替换为自己设置的端口
54+
55+
## 实现原理
56+
57+
上述实现借助于 [SSH 动态转发](https://en.wikipedia.org/wiki/Port_forwarding#Dynamic_port_forwarding)功能,SSH 动态端口转发将 SSH 服务器变成 SOCKS 代理服务器,您计算机上的应用程序可以将其用作连接远程服务器的中介。
58+
59+
> **注意:**程序必须支持 SOCKS 类型的代理,您才能使用动态端口转发从该应用程序路由流量。
Loading

0 commit comments

Comments
 (0)