Update v1.2.0-alpha

Author: YoshitakaMo

README.md

@@ -2,6 +2,10 @@
 
 [ColabFold / AlphaFold2_advanced](https://colab.research.google.com/github/sokrypton/ColabFold/blob/main/beta/AlphaFold2_advanced.ipynb) on your local PC (or macOS)
 
+## New Updates
+
+- 04Dec2021, LocalColabFold is now compatible with the latest [pip installable ColabFold](https://github.com/sokrypton/ColabFold#running-locally). In this repository, I will provide a script to install ColabFold with some external parameter files to perform relaxation with AMBER. The weight parameters of AlphaFold and AlphaFold-Multimer will be downloaded automatically at your first run.
+
 ## Installation
 
 ### For Linux
@@ -14,10 +18,9 @@ Built on Mon_Oct_12_20:09:46_PDT_2020
 Cuda compilation tools, release 11.1, V11.1.105
 Build cuda_11.1.TC455_06.29190527_0
 </pre>DO NOT use `nvidia-smi` for checking the version.<br>See [NVIDIA CUDA Installation Guide for Linux](https://docs.nvidia.com/cuda/cuda-installation-guide-linux/index.html) if you haven't installed it.
-1. Download `install_colabfold_linux.sh` from this repository:<pre>$ wget https://raw.githubusercontent.com/YoshitakaMo/localcolabfold/main/install_colabfold_linux.sh</pre> and run it in the directory where you want to install:<pre>$ bash install_colabfold_linux.sh</pre>About 5 minutes later, `colabfold` directory will be created. Do not move this directory after the installation.
-1. Type `cd colabfold` to enter the directory.
-1. Modify the variables such as `sequence = 'PIAQIHILEGRSDEQKETLIREVSEAISRSLDAPLTSVRVIITEMAKGHFGIGGELASK'`, `jobname = "test"`, and etc. in `runner.py` for your prediction. For more information, please refer to the original [ColabFold / AlphaFold2_advanced](https://colab.research.google.com/github/sokrypton/ColabFold/blob/main/beta/AlphaFold2_advanced.ipynb).
-1. To run the prediction, type <pre>$ colabfold-conda/bin/python3.7 runner.py</pre>in the `colabfold` directory. The result files will be created in the `predition_<jobname>_<hash>` in the `colabfold` directory. After the prediction finished, you may move the results from the `colabfold` directory.
+1. Download `install_colabbatch_linux.sh` from this repository:<pre>$ wget https://raw.githubusercontent.com/YoshitakaMo/localcolabfold/main/install_colabbatch_linux.sh</pre> and run it in the directory where you want to install:<pre>$ bash install_colabbatch_linux.sh</pre>About 5 minutes later, `colabfold_batch` directory will be created. Do not move this directory after the installation.
+1. Add environment variable PATH:<pre># bash<br>export PATH="<COLABFOLDBATCH_DIR>/bin:\$PATH"<br>#i.e. export PATH="/home/moriwaki/Desktop/colabfold_batch/bin:\$PATH"</pre>
+2. To run the prediction, type <pre>colabfold_batch --amber --templates --num-recycle 3 inputfile outputdir/ </pre>The result files will be created in the `outputdir`. For more details, see `colabfold_batch --help`.
 
 ### For macOS
 
@@ -36,54 +39,16 @@ Please use the correct installer for your Mac.
 #### For Mac with Intel CPU
 
 1. Install [Homebrew](https://brew.sh/index_ja) if not present:<pre>$ /bin/bash -c "$(curl -fsSL https://raw.githubusercontent.com/Homebrew/install/HEAD/install.sh)"</pre>
-1. Install `wget` command using Homebrew:<pre>$ brew install wget</pre>
-1. Download `install_colabfold_intelmac.sh` from this repository:<pre>$ wget https://raw.githubusercontent.com/YoshitakaMo/localcolabfold/main/install_colabfold_intelmac.sh</pre> and run it in the directory where you want to install:<pre>$ bash install_colabfold_intelmac.sh</pre>About 5 minutes later, `colabfold` directory will be created. Do not move this directory after the installation.
+1. Install `wget` command using Homebrew:<pre>$ brew install wget gnu-sed<br>\$ brew install brewsci/bio/hh-suite brewsci/bio/kalign</pre>
+1. Download `install_colabbatch_intelmac.sh` from this repository:<pre>$ wget https://raw.githubusercontent.com/YoshitakaMo/localcolabfold/main/install_colabbatch_intelmac.sh</pre> and run it in the directory where you want to install:<pre>$ bash install_colabbatch_intelmac.sh</pre>About 5 minutes later, `colabfold_batch` directory will be created. Do not move this directory after the installation.
 1. The rest procedure is the same as "For Linux".
 
 #### For Mac with Apple Silicon (M1 chip)
 
-**Note: This installer is experimental because most of the dependent packages are not fully tested on Apple Silicon Mac.**
-
-1. Install [Homebrew](https://brew.sh/index_ja) if not present:<pre>$ /bin/bash -c "$(curl -fsSL https://raw.githubusercontent.com/Homebrew/install/HEAD/install.sh)"</pre>
-1. Install `wget` and `cmake` commands using Homebrew:<pre>$ brew install wget cmake</pre>
-1. Install `miniforge` command using Homebrew:<pre>$ brew install --cask miniforge</pre>
-1. Download `install_colabfold_M1mac.sh` from this repository:<pre>$ wget https://raw.githubusercontent.com/YoshitakaMo/localcolabfold/main/install_colabfold_M1mac.sh</pre> and run it in the directory where you want to install:<pre>$ bash install_colabfold_M1mac.sh</pre>About 5 minutes later, `colabfold` directory will be created. Do not move this directory after the installation.
-1. Type `cd colabfold` to enter the directory.
-1. Modify the variables such as `sequence = 'PIAQIHILEGRSDEQKETLIREVSEAISRSLDAPLTSVRVIITEMAKGHFGIGGELASK'`, `jobname = "test"`, and etc. in `runner.py` for your prediction. For more information, please refer to the original [ColabFold / AlphaFold2_advanced](https://colab.research.google.com/github/sokrypton/ColabFold/blob/main/beta/AlphaFold2_advanced.ipynb).
-1. To run the prediction, type <pre>$ colabfold-conda/bin/python3.8 runner.py</pre>in the `colabfold` directory. The result files will be created in the `predition_<jobname>_<hash>` in the `colabfold` directory. After the prediction finished, you may move the results from the `colabfold` directory.
-
-A Warning message appeared when you run the prediction:
-```
-You are using an experimental build of OpenMM v7.5.1.
-This is NOT SUITABLE for production!
-It has not been properly tested on this platform and we cannot guarantee it provides accurate results.
-```
-
-This message is due to Apple Silicon, but I think we can ignore it.
-
-## Usage of `colabfold` shell script (Linux)
-
-An executable `colabfold` shell script is installed in `/path/to/colabfold/bin` directory. This is more helpful for installation on a shared computer and users who want to predict many sequences.
-
-1. Prepare a FASTA file containing the amino acid sequence for which you want to predict the structure (e.g. `6x9z.fasta`).<pre>>6X9Z_1|Chain A|Transmembrane beta-barrels|synthetic construct (32630)
-MEQKPGTLMVYVVVGYNTDNTVDVVGGAQYAVSPYLFLDVGYGWNNSSLNFLEVGGGVSYKVSPDLEPYVKAGFEYNTDNTIKPTAGAGALYRVSPNLALMVEYGWNNSSLQKVAIGIAYKVKD</pre>
-2. Type `export PATH="/path/to/colabfold/bin:$PATH"` to add a path to the PATH environment variable. For example, `export PATH="/home/foo/bar/colabfold/bin:$PATH"` if you installed localcolabfold on `/home/foo/bar/colabfold`.
-3. Run colabfold command with your FASTA file. For example,<pre>$ colabfold --input 6x9z.fasta \\
-   --output_dir 6x9z \\
-   --max_recycle 18 \\
-   --use_ptm \\
-   --use_turbo \\
-   --num_relax Top5</pre>This will predict a protein structure [6x9z](https://www.rcsb.org/structure/6x9z) with increasing the number of 'recycling' to 18. This may be effective for *de novo* structure prediction. For another example, [PDB: 3KUD](https://www.rcsb.org/structure/3KUD), <pre>$ colabfold --input 3kud_complex.fasta \\
-   --output_dir 3kud \\
-   --homooligomer 1:1 \\
-   --use_ptm \\
-   --use_turbo \\
-   --max_recycle 3 \\
-   --num_relax Top5</pre>where the input sequence `3kud_complex.fasta` is<pre>>3KUD_complex
-   MTEYKLVVVGAGGVGKSALTIQLIQNHFVDEYDPTIEDSYRKQVVIDGETCLLDILDTAGQEEYSAMRDQYMRTGEGFLCVFAINNTKSFEDIHQYREQIKRVKDSDDVPMVLVGNKCDLAARTVESRQAQDLARSYGIPYIETSAKTRQGVEDAFYTLVREIRQH:
-   PSKTSNTIRVFLPNKQRTVVNVRNGMSLHDCLMKALKVRGLQPECCAVFRLLHEHKGKKARLDWNTDAASLIGEELQVDFL</pre>This will predict a heterooligomer. For more information about the options, type `colabfold --help` or refer to the original [ColabFold / AlphaFold2_advanced](https://colab.research.google.com/github/sokrypton/ColabFold/blob/main/beta/AlphaFold2_advanced.ipynb).
+Currently not supported.
 
 ## Advantages of LocalColabFold
+
 - **Structure inference and relaxation will be accelerated if your PC has Nvidia GPU and CUDA drivers.**
 - **No Time out (90 minutes and 12 hours)**
 - **No GPU limitations**
@@ -100,16 +65,15 @@ MEQKPGTLMVYVVVGYNTDNTVDVVGGAQYAVSPYLFLDVGYGWNNSSLNFLEVGGGVSYKVSPDLEPYVKAGFEYNTDN
   - Yes, the sequence input is the same as ColabFold. See [ColabFold / AlphaFold2_advanced](https://colab.research.google.com/github/sokrypton/ColabFold/blob/main/beta/AlphaFold2_advanced.ipynb).
 - Is it possible to create MSA by jackhmmer?
   - **No, it is not currently supported**.
-- I want to run the predictions step-by-step like Google Colab.
-  - You can use VSCode and Python plugin to do the same. See https://code.visualstudio.com/docs/python/jupyter-support-py.
 - I want to use multiple GPUs to perform the prediction.
-  - You need to set the environment variables `TF_FORCE_UNIFIED_MEMORY`,`XLA_PYTHON_CLIENT_MEM_FRACTION` before execution. See [this discussion](https://github.com/YoshitakaMo/localcolabfold/issues/7#issuecomment-923027641).
-- I want to solve the `ResourceExhausted` error when trying to predict for a sequence with > 1000 residues.
-  - See the same discussion as above.
+  - **AlphaFold and ColabFold does not support multiple GPUs**. Only One GPU can model your protein.
 - I got an error message `CUDA_ERROR_ILLEGAL_ADDRESS: an illegal memory access was encountered`.
   - You may not have updated to CUDA 11.1 or later. Please check the version of Cuda compiler with `nvcc --version` command, not `nvidia-smi`.
 - Is this available on Windows 10?
   - You can run LocalColabFold on your Windows 10 with [WSL2](https://docs.microsoft.com/en-us/windows/wsl/install-win10).
+- I want to use a custom MSA file in the format of a3m.
+  - **ColabFold can accept various input files now**. See the help messsage. You can set your own A3M file, a fasta file that contains multiple sequences (in FASTA format), or a directory that contains multiple fasta files.
+
 
 ## Tutorials & Presentations
 
@@ -121,7 +85,16 @@ MEQKPGTLMVYVVVGYNTDNTVDVVGGAQYAVSPYLFLDVGYGWNNSSLNFLEVGGGVSYKVSPDLEPYVKAGFEYNTDN
 
 ## How do I reference this work?
 
-- Mirdita M, Schuetze K, Moriwaki Y, Heo L, Ovchinnikov S and Steinegger M. ColabFold - Making protein folding accessible to all. *bioRxiv*, doi: [10.1101/2021.08.15.456425](https://www.biorxiv.org/content/10.1101/2021.08.15.456425v2) (2021)
-- John Jumper, Richard Evans, Alexander Pritzel, et al. -  Highly accurate protein structure prediction with AlphaFold. *Nature*, 1–11, doi: [10.1038/s41586-021-03819-2](https://www.nature.com/articles/s41586-021-03819-2) (2021)
+- Mirdita M, Schütze K, Moriwaki Y, Heo L, Ovchinnikov S and Steinegger M. ColabFold - Making protein folding accessible to all. <br />
+  bioRxiv (2021) doi: [10.1101/2021.08.15.456425](https://www.biorxiv.org/content/10.1101/2021.08.15.456425v2)
+- If you’re using **AlphaFold**, please also cite: <br />
+  Jumper et al. "Highly accurate protein structure prediction with AlphaFold." <br />
+  Nature (2021) doi: [10.1038/s41586-021-03819-2](https://doi.org/10.1038/s41586-021-03819-2)
+- If you’re using **AlphaFold-multimer**, please also cite: <br />
+  Evans et al. "Protein complex prediction with AlphaFold-Multimer." <br />
+  biorxiv (2021) doi: [10.1101/2021.10.04.463034v1](https://www.biorxiv.org/content/10.1101/2021.10.04.463034v1)
+- If you are using **RoseTTAFold**, please also cite: <br />
+  Minkyung et al. "Accurate prediction of protein structures and interactions using a three-track neural network." <br />
+  Science (2021) doi: [10.1126/science.abj8754](https://doi.org/10.1126/science.abj8754)
 
 [![DOI](https://zenodo.org/badge/doi/10.5281/zenodo.5123296.svg)](https://doi.org/10.5281/zenodo.5123296)

install_colabbatch_intelmac.sh

@@ -0,0 +1,69 @@
+#!/bin/bash
+
+# check commands
+type wget || { echo "wget command is not installed. Please install it at first using Homebrew." ; exit 1 ; }
+type gsed || { echo "gnu-sed command is not installed. Please install it at first using Homebrew." ; exit 1 ; }
+type hhsearch || { echo "hhsearch command is not installed. Please install it at first using Homebrew." ; exit 1 ; }
+type kalign || { echo "kalign command is not installed. Please install it at first using Homebrew." ; exit 1 ; }
+
+# check whether Apple Silicon (M1 mac) or Intel Mac
+arch_name="$(uname -m)"
+
+if [ "${arch_name}" = "x86_64" ]; then
+    if [ "$(sysctl -in sysctl.proc_translated)" = "1" ]; then
+        echo "Running on Rosetta 2"
+    else
+        echo "Running on native Intel"
+    fi
+elif [ "${arch_name}" = "arm64" ]; then
+    echo "Running on Apple Silicon (M1 mac)"
+    echo "This installer is only for intel Mac. Use install_colabfold_M1mac.sh to install on this Mac."
+    exit 1
+else
+    echo "Unknown architecture: ${arch_name}"
+    exit 1
+fi
+
+CURRENTPATH=`pwd`
+COLABFOLDDIR="${CURRENTPATH}/colabfold_batch"
+
+mkdir -p ${COLABFOLDDIR}
+cd ${COLABFOLDDIR}
+wget https://git.scicore.unibas.ch/schwede/openstructure/-/raw/7102c63615b64735c4941278d92b554ec94415f8/modules/mol/alg/src/stereo_chemical_props.txt --no-check-certificate
+wget -q -P . https://repo.anaconda.com/miniconda/Miniconda3-latest-MacOSX-x86_64.sh
+bash ./Miniconda3-latest-MacOSX-x86_64.sh -b -p ${COLABFOLDDIR}/conda
+rm Miniconda3-latest-MacOSX-x86_64.sh
+. "${COLABFOLDDIR}/conda/etc/profile.d/conda.sh"
+export PATH="${COLABFOLDDIR}/conda/condabin:${PATH}"
+conda create -p $COLABFOLDDIR/colabfold-conda python=3.7 -y
+conda activate $COLABFOLDDIR/colabfold-conda
+conda update -n base conda -y
+conda install -c conda-forge python=3.7 openmm==7.5.1 pdbfixer -y
+# patch to openmm
+wget -qnc https://raw.githubusercontent.com/deepmind/alphafold/main/docker/openmm.patch --no-check-certificate
+(cd ${COLABFOLDDIR}/colabfold-conda/lib/python3.7/site-packages; patch -s -p0 < ${COLABFOLDDIR}/openmm.patch)
+rm openmm.patch
+# install ColabFold and Jaxlib
+colabfold-conda/bin/python3.7 -m pip install "colabfold[alphafold] @ git+https://github.com/sokrypton/ColabFold"
+colabfold-conda/bin/python3.7 -m pip install https://storage.googleapis.com/jax-releases/mac/jaxlib-0.1.74-cp37-none-macosx_10_9_x86_64.whl
+
+# bin directory to run
+mkdir -p $COLABFOLDDIR/bin
+cd $COLABFOLDDIR/bin
+cat << EOF > colabfold_batch
+#!/bin/sh
+$COLABFOLDDIR/colabfold-conda/bin/colabfold_batch --cpu \$@
+EOF
+chmod +x colabfold_batch
+
+# hack to share the parameter files in a workstation.
+gsed -i -e "s#props_path = \"stereo_chemical_props.txt\"#props_path = \"${COLABFOLDDIR}/stereo_chemical_props.txt\"#" ${COLABFOLDDIR}/colabfold-conda/lib/python3.7/site-packages/colabfold/batch.py
+gsed -i -e "s#kalign_binary_path=\"kalign\"#kalign_binary_path=\"/usr/local/bin/kalign\"#g" ${COLABFOLDDIR}/colabfold-conda/lib/python3.7/site-packages/colabfold/batch.py
+gsed -i -e "s#binary_path=\"hhsearch\"#binary_path=\"/usr/local/bin/hhsearch\"#g" ${COLABFOLDDIR}/colabfold-conda/lib/python3.7/site-packages/colabfold/batch.py
+gsed -i -e "s#Path(appdirs.user_cache_dir(__package__ or \"colabfold\"))#${COLABFOLDDIR}#g" ${COLABFOLDDIR}/colabfold-conda/lib/python3.7/site-packages/colabfold/download.py
+
+echo "Installation of colabFold_batch finished."
+echo "Note: AlphaFold2 weight parameters will be donwloaded at ${COLABFOLDDIR}/params directory in the first run."
+echo "Please set your PATH to ${COLABFOLDDIR}/bin to run 'colabfold_batch'."
+echo "i.e. For Bash, export PATH=\"${COLABFOLDDIR}/bin:\$PATH\""
+echo "For more details, please type 'colabfold_batch --help'."

install_colabbatch_linux.sh

@@ -0,0 +1,55 @@
+#!/bin/bash
+
+type wget || { echo "wget command is not installed. Please install it at first using apt or yum." ; exit 1 ; }
+type curl || { echo "curl command is not installed. Please install it at first using apt or yum. " ; exit 1 ; }
+
+CURRENTPATH=`pwd`
+COLABFOLDDIR="${CURRENTPATH}/colabfold_batch"
+
+mkdir -p ${COLABFOLDDIR}
+cd ${COLABFOLDDIR}
+wget https://git.scicore.unibas.ch/schwede/openstructure/-/raw/7102c63615b64735c4941278d92b554ec94415f8/modules/mol/alg/src/stereo_chemical_props.txt --no-check-certificate
+wget -q -P . https://repo.anaconda.com/miniconda/Miniconda3-latest-Linux-x86_64.sh
+bash ./Miniconda3-latest-Linux-x86_64.sh -b -p ${COLABFOLDDIR}/conda
+rm Miniconda3-latest-Linux-x86_64.sh
+. "${COLABFOLDDIR}/conda/etc/profile.d/conda.sh"
+export PATH="${COLABFOLDDIR}/conda/condabin:${PATH}"
+conda create -p $COLABFOLDDIR/colabfold-conda python=3.7 -y
+conda activate $COLABFOLDDIR/colabfold-conda
+conda update -n base conda -y
+conda install -c conda-forge python=3.7 cudnn==8.2.1.32 cudatoolkit==11.1.1 openmm==7.5.1 pdbfixer -y
+# patch to openmm
+wget -qnc https://raw.githubusercontent.com/deepmind/alphafold/main/docker/openmm.patch --no-check-certificate
+(cd ${COLABFOLDDIR}/colabfold-conda/lib/python3.7/site-packages; patch -s -p0 < ${COLABFOLDDIR}/openmm.patch)
+rm openmm.patch
+# install alignment tools
+conda install -c conda-forge -c bioconda kalign3=3.2.2 hhsuite=3.3.0 -y
+# install ColabFold and Jaxlib
+colabfold-conda/bin/python3.7 -m pip install "colabfold[alphafold] @ git+https://github.com/sokrypton/ColabFold"
+colabfold-conda/bin/python3.7 -m pip install https://storage.googleapis.com/jax-releases/cuda111/jaxlib-0.1.72+cuda111-cp37-none-manylinux2010_x86_64.whl
+
+# bin directory to run
+mkdir -p $COLABFOLDDIR/bin
+cd $COLABFOLDDIR/bin
+cat << EOF > colabfold_batch
+#!/bin/sh
+export TF_FORCE_UNIFIED_MEMORY="1"
+export XLA_PYTHON_CLIENT_MEM_FRACTION="4.0"
+$COLABFOLDDIR/colabfold-conda/bin/colabfold_batch \$@
+EOF
+chmod +x colabfold_batch
+
+# hack to share the parameter files in a workstation.
+cd ${COLABFOLDDIR}/colabfold-conda/lib/python3.7/site-packages/colabfold
+sed -i -e "s#props_path = \"stereo_chemical_props.txt\"#props_path = \"${COLABFOLDDIR}/stereo_chemical_props.txt\"#" batch.py
+sed -i -e "s#kalign_binary_path=\"kalign\"#kalign_binary_path=\"${COLABFOLDDIR}/colabfold-conda/bin/kalign\"#g" ${COLABFOLDDIR}/colabfold-conda/lib/python3.7/site-packages/colabfold/batch.py
+sed -i -e "s#binary_path=\"hhsearch\"#binary_path=\"${COLABFOLDDIR}/colabfold-conda/bin/hhsearch\"#g" ${COLABFOLDDIR}/colabfold-conda/lib/python3.7/site-packages/colabfold/batch.py
+sed -i -e "s#Path(appdirs.user_cache_dir(__package__ or \"colabfold\"))#\"${COLABFOLDDIR}\"#g" download.py
+cd ${COLABFOLDDIR}/colabfold-conda/lib/python3.7/site-packages/alphafold/relax
+sed -i -e 's/CPU/CUDA/g' amber_minimize.py
+
+echo "Installation of colabFold_batch finished."
+echo "Note: AlphaFold2 weight parameters will be donwloaded at ${COLABFOLDDIR}/params directory in the first run."
+echo "Please set your PATH to ${COLABFOLDDIR}/bin to run 'colabfold_batch'."
+echo "i.e. For Bash, export PATH=\"${COLABFOLDDIR}/bin:\$PATH\""
+echo "For more details, please type 'colabfold_batch --help'."