diff --git a/readme-cn.md b/readme-cn.md index 42a84f5..9ca682a 100644 --- a/readme-cn.md +++ b/readme-cn.md @@ -193,40 +193,42 @@ 11. 一些模型的加载,以及微调和量化功能需要`flash-attn`和`bitsandbytes`库,请使用如下方法安装。 - ```bash - # Linux版本安装命令,需要大约1小时编译安装时间。 - pip install flash-attn - pip install bitsandbytes + ```bash + # Linux版本安装命令,需要大约1小时编译安装时间。 + pip install flash-attn + pip install bitsandbytes - # Windows版本下载URL,请选择正确的CUDA Tookit版本,Torch版本和Python版本。 - https://github.com/bdashore3/flash-attention/releases - https://github.com/jllllll/bitsandbytes-windows-webui/releases - # 下载后请使用以下命令安装 - pip install *.whl + # Windows版本下载URL,请选择正确的CUDA Tookit版本,Torch版本和Python版本。 + https://github.com/bdashore3/flash-attention/releases + https://github.com/jllllll/bitsandbytes-windows-webui/releases + # 下载后请使用以下命令安装 + pip install *.whl - # MacOS版本不支持 - ``` + # MacOS版本不支持 + ``` 12. 如果仅想在本地进行部署,可以使用python启动WebUI,http://127.0.0.1:8818 - ```bash - python __webgui_server__.py --webui - ``` + + ```bash + python __webgui_server__.py --webui + ``` 13. 如果需要在云服务器上部署,并在本地访问WebUI,请使用反向代理,并以HTTPS协议启动WebUI。在本地请使用https://127.0.0.1:4480 打开WebUI,在远端使用 https://[server ip]:4480 打开WebUI。 - ```bash - // 批处理内部默认使用的虚拟环境是 keras-llm-robot,如果想使用其它的虚拟环境名称,请自行修改批处理文件 - webui-startup-windows.bat - // ubuntu(linux)平台 - python __webgui_server__.py --webui - chmod +x ./tools/ssl-proxy-linux - ./tools/ssl-proxy-linux -from 0.0.0.0:4480 -to 127.0.0.1:8818 - - // MacOS平台 - python __webgui_server__.py --webui - chmod +x ./tools/ssl-proxy-darwin - ./tools/ssl-proxy-darwin -from 0.0.0.0:4480 -to 127.0.0.1:8818 - ``` + ```bash + // 批处理内部默认使用的虚拟环境是 keras-llm-robot,如果想使用其它的虚拟环境名称,请自行修改批处理文件 + webui-startup-windows.bat + + // ubuntu(linux)平台 + python __webgui_server__.py --webui + chmod +x ./tools/ssl-proxy-linux + ./tools/ssl-proxy-linux -from 0.0.0.0:4480 -to 127.0.0.1:8818 + + // MacOS平台 + python __webgui_server__.py --webui + chmod +x ./tools/ssl-proxy-darwin + ./tools/ssl-proxy-darwin -from 0.0.0.0:4480 -to 127.0.0.1:8818 + ``` ## 功能介绍 diff --git a/readme.md b/readme.md index e4bec4c..7ac79c8 100644 --- a/readme.md +++ b/readme.md @@ -188,46 +188,49 @@ Auxiliary models, such as retrieval, code execution, text-to-speech, speech-to-t pip install kornia pip install open_clip_torch ``` + 11. Some model loading, as well as fine-tuning and quantization features require the `flash-attn` and `bitsandbytes` libraries, please install them using the following method. - ```bash - # For Linux,Takes about 1 hour to compile and install。 - pip install flash-attn - pip install bitsandbytes + ```bash + # For Linux,Takes about 1 hour to compile and install。 + pip install flash-attn + pip install bitsandbytes - # The Download URL of Windows, Please take care to select the correct CUDA Tookit version, Torch version and Python version. - https://github.com/bdashore3/flash-attention/releases - https://github.com/jllllll/bitsandbytes-windows-webui/releases - # After downloading, please use the following command to install - pip install *.whl + # The Download URL of Windows, Please take care to select the correct CUDA Tookit version, Torch version and Python version. + https://github.com/bdashore3/flash-attention/releases + https://github.com/jllllll/bitsandbytes-windows-webui/releases + # After downloading, please use the following command to install + pip install *.whl - # MacOS version is not supported - ``` + # MacOS version is not supported + ``` 12. If run locally, start the Web UI using Python at http://127.0.0.1:8818: - ```bash - python __webgui_server__.py --webui - ``` + + ```bash + python __webgui_server__.py --webui + ``` 13. If deploying on a cloud server and accessing the Web UI locally, use reverse proxy and start the Web UI with HTTPS. Access using https://127.0.0.1:4480 on locally, and use the https interface at https://[server ip]:4480 on remotely: - ```bash - // By default, the batch file uses the virtual environment named keras-llm-robot, - // Modify the batch file if using a different virtual environment name. - - webui-startup-windows.bat - // ubuntu(linux) + ```bash + // By default, the batch file uses the virtual environment named keras-llm-robot, + // Modify the batch file if using a different virtual environment name. - python __webgui_server__.py --webui - chmod +x ./tools/ssl-proxy-linux - ./tools/ssl-proxy-linux -from 0.0.0.0:4480 -to 127.0.0.1:8818 + webui-startup-windows.bat + + // ubuntu(linux) - // MacOS + python __webgui_server__.py --webui + chmod +x ./tools/ssl-proxy-linux + ./tools/ssl-proxy-linux -from 0.0.0.0:4480 -to 127.0.0.1:8818 - python __webgui_server__.py --webui - chmod +x ./tools/ssl-proxy-darwin - ./tools/ssl-proxy-darwin -from 0.0.0.0:4480 -to 127.0.0.1:8818 - ``` + // MacOS + + python __webgui_server__.py --webui + chmod +x ./tools/ssl-proxy-darwin + ./tools/ssl-proxy-darwin -from 0.0.0.0:4480 -to 127.0.0.1:8818 + ``` ## Feature Overview