TypeError: '<' not supported between instances of 'Chromosome' and 'Chromosome'

[zhangrengang]

I got an exception when there are multiple chromosomes. It seemed that self.__chromosomes.sort() can not be sorted when self.__chromosomes have more than one Chromosome values.

Traceback (most recent call last):
  File "/share/home/app/.local/python3/bin/run_pypmag.py", line 4, in <module>
    __import__('pkg_resources').run_script('arachis==0.42.2', 'run_pypmag.py')
  File "/share/home/app/.local/python3/lib/python3.6/site-packages/pkg_resources/__init__.py", line 666, in run_script
    self.require(requires)[0].run_script(script_name, ns)
  File "/share/home/app/.local/python3/lib/python3.6/site-packages/pkg_resources/__init__.py", line 1469, in run_script
    exec(script_code, namespace, namespace)
  File "/share/home/app/.local/python3/lib/python3.6/site-packages/arachis-0.42.2-py3.6.egg/EGG-INFO/scripts/run_pypmag.py", line 561, in <module>
  File "/share/home/app/.local/python3/lib/python3.6/site-packages/arachis-0.42.2-py3.6.egg/EGG-INFO/scripts/run_pypmag.py", line 556, in main
  File "/share/home/app/.local/python3/lib/python3.6/site-packages/arachis-0.42.2-py3.6.egg/EGG-INFO/scripts/run_pypmag.py", line 287, in reconstruct_ancestral_genomes
  File "/share/home/app/.local/python3/lib/python3.6/site-packages/arachis-0.42.2-py3.6.egg/arachis/genomeClass.py", line 1358, in __init__
  File "/share/home/app/.local/python3/lib/python3.6/site-packages/arachis-0.42.2-py3.6.egg/arachis/genomeClass.py", line 1381, in parse_grimm
  File "/share/home/app/.local/python3/lib/python3.6/site-packages/arachis-0.42.2-py3.6.egg/arachis/genomeClass.py", line 1272, in __init__
TypeError: '<' not supported between instances of 'Chromosome' and 'Chromosome'

[Kinggerm]

Thanks for reporting this issue. Fixed!
Please let me know the updates!

[zhangrengang]

Another issue:

Block names: 4039 5742 6517 81 7845 6038 5965 6039 8471 3266 5248 5247 3031 107 8009 16 7680 6035 1350 2712 6146 1490 685 5921 246 6681 4436 5547 1878 3833 691 809 6345 3043 3492 1383 1847 4406 7404 505 5664 3929 969 3466 3006 5696 5868 7180 2203 1385 7181 7182 4114 6798 8292 5773 4833 6417 3816 528 .......
Block state: 1
Ancestors: A1 A2 A3
raxmlHPC -f A -c 6 -m MULTIGAMMA -p 12345 -t output3/rerooted.A3 -s output3/Content.encoding -n Content.A3 -w output3
raxmlHPC -f A -c 6 -m BINGAMMA -p 12345 -t output3/rerooted.A3 -s output3/Order.encoding -n Order.A3 -w output3
A3 block end nodes: 0 - 17984
A3 terminal nodes: 17985 - 17996
A3 in-block nodes: 17997 - 26988
raxmlHPC -f A -c 6 -m MULTIGAMMA -p 12345 -t output3/rerooted.A1 -s output3/Content.encoding -n Content.A1 -w output3
raxmlHPC -f A -c 6 -m BINGAMMA -p 12345 -t output3/rerooted.A1 -s output3/Order.encoding -n Order.A1 -w output3
A1 block end nodes: 0 - 17984
A1 terminal nodes: 17985 - 17998
A1 in-block nodes: 17999 - 26990
raxmlHPC -f A -c 6 -m MULTIGAMMA -p 12345 -t output3/rerooted.A2 -s output3/Content.encoding -n Content.A2 -w output3
raxmlHPC -f A -c 6 -m BINGAMMA -p 12345 -t output3/rerooted.A2 -s output3/Order.encoding -n Order.A2 -w output3
A2 block end nodes: 0 - 17984
A2 terminal nodes: 17985 - 18000
A2 in-block nodes: 18001 - 26992
Traceback (most recent call last):
  File "/share/home/app/.local/python3/bin/run_pypmag.py", line 4, in <module>
    __import__('pkg_resources').run_script('arachis==0.42.2', 'run_pypmag.py')
  File "/share/home/app/.local/python3/lib/python3.6/site-packages/pkg_resources/__init__.py", line 666, in run_script
    self.require(requires)[0].run_script(script_name, ns)
  File "/share/home/app/.local/python3/lib/python3.6/site-packages/pkg_resources/__init__.py", line 1469, in run_script
    exec(script_code, namespace, namespace)
  File "/share/home/app/.local/python3/lib/python3.6/site-packages/arachis-0.42.2-py3.6.egg/EGG-INFO/scripts/run_pypmag.py", line 561, in <module>
  File "/share/home/app/.local/python3/lib/python3.6/site-packages/arachis-0.42.2-py3.6.egg/EGG-INFO/scripts/run_pypmag.py", line 556, in main
  File "/share/home/app/.local/python3/lib/python3.6/site-packages/arachis-0.42.2-py3.6.egg/EGG-INFO/scripts/run_pypmag.py", line 349, in reconstruct_ancestral_genomes
FileNotFoundError: [Errno 2] No such file or directory: 'output3/OutputGeneOrder_A1'

But it is ok with the test data.

[Kinggerm]

Could you provide the data (grimm and tree) you are using? Thanks

[zhangrengang]

I have sent the data file to your email.

[Kinggerm]

I've received the data. It seems to be an issue with parallelization (-p). The default is 4 processes, with which I got the same error as you posted. But If I turn off the parallelization (-p 1), your data could finish successfully.

I'm gonna fix this issue, but before that, please use -p 1 to get around this issue.

[zhangrengang]

OK. Thanks.

[zhangrengang]

Using -p 1, an other exception raised:

Block names: 4039 5742 6517 81 7845 6038 5965 6039 8471 3266 5248 5247 3031 107 8009 16 7680 6035 1350 2712 6146 ......
Block state: 1
Ancestors: A1 A2 A3
raxmlHPC -f A -c 6 -m MULTIGAMMA -p 12345 -t output5/rerooted.A1 -s output5/Content.encoding -n Content.A1 -w output5
raxmlHPC -f A -c 6 -m BINGAMMA -p 12345 -t output5/rerooted.A1 -s output5/Order.encoding -n Order.A1 -w output5
A1 block end nodes: 0 - 17984
A1 terminal nodes: 17985 - 17998
A1 in-block nodes: 17999 - 26990
concorde -s 12345 -o output5/pymag.TSPSolution.A1 output5/pymag.TSPLIB.A1
Traceback (most recent call last):
  File "/share/home/app/build/Arachis/scripts/run_pypmag.py", line 562, in <module>
    main()
  File "/share/home/app/build/Arachis/scripts/run_pypmag.py", line 557, in main
    options.resume, options.verbose, options.keep_temp, options.processing_num)
  File "/share/home/app/build/Arachis/scripts/run_pypmag.py", line 347, in reconstruct_ancestral_genomes
    which_concorde, which_raxml, random_seed, verbose, keep_temp)
  File "/share/home/app/build/Arachis/scripts/run_pypmag.py", line 506, in sub_reconstruct_anc_genomes
    tsp_path = read_tsp_path(solution_file)
  File "/share/home/app/build/Arachis/scripts/run_pypmag.py", line 253, in read_tsp_path
    with open(solution_file) as file_handler:
FileNotFoundError: [Errno 2] No such file or directory: 'output5/pymag.TSPSolution.A1'

It is seemed that concorde is failed. If I run concorde -s 12345 -o output5/pymag.TSPSolution.A1 output5/pymag.TSPLIB.A1 alone, the following is reported:

concorde -s 12345 -o output5/pymag.TSPSolution.A1 output5/pymag.TSPLIB.A1
Host: cluster.hpc.org  Current process id: 28755
Using random seed 12345
output5/pymag.TSPLIB.A1: Value too large for defined data type
Unable to open output5/pymag.TSPLIB.A1 for input
CCutil_gettsplib failed

The file output5/pymag.TSPLIB.A1 exists but is seemed too large.

[zhangrengang]

A minor issue: verbose is not included in run_concorde(tsp_file, solution_file, out_dir, which_concorde, random_seed) in run_pypmag.py. It should be run_concorde(tsp_file, solution_file, out_dir, which_concorde, random_seed, verbose).

[Kinggerm]

A minor issue: verbose is not included in run_concorde(tsp_file, solution_file, out_dir, which_concorde, random_seed) in run_pypmag.py. It should be run_concorde(tsp_file, solution_file, out_dir, which_concorde, random_seed, verbose).

Thanks. I've noticed this during checking the parallelization. It will be fixed in the coming version.

[Kinggerm]

Using -p 1, an other exception raised:

Block names: 4039 5742 6517 81 7845 6038 5965 6039 8471 3266 5248 5247 3031 107 8009 16 7680 6035 1350 2712 6146 ......
Block state: 1
Ancestors: A1 A2 A3
raxmlHPC -f A -c 6 -m MULTIGAMMA -p 12345 -t output5/rerooted.A1 -s output5/Content.encoding -n Content.A1 -w output5
raxmlHPC -f A -c 6 -m BINGAMMA -p 12345 -t output5/rerooted.A1 -s output5/Order.encoding -n Order.A1 -w output5
A1 block end nodes: 0 - 17984
A1 terminal nodes: 17985 - 17998
A1 in-block nodes: 17999 - 26990
concorde -s 12345 -o output5/pymag.TSPSolution.A1 output5/pymag.TSPLIB.A1
Traceback (most recent call last):
  File "/share/home/app/build/Arachis/scripts/run_pypmag.py", line 562, in <module>
    main()
  File "/share/home/app/build/Arachis/scripts/run_pypmag.py", line 557, in main
    options.resume, options.verbose, options.keep_temp, options.processing_num)
  File "/share/home/app/build/Arachis/scripts/run_pypmag.py", line 347, in reconstruct_ancestral_genomes
    which_concorde, which_raxml, random_seed, verbose, keep_temp)
  File "/share/home/app/build/Arachis/scripts/run_pypmag.py", line 506, in sub_reconstruct_anc_genomes
    tsp_path = read_tsp_path(solution_file)
  File "/share/home/app/build/Arachis/scripts/run_pypmag.py", line 253, in read_tsp_path
    with open(solution_file) as file_handler:
FileNotFoundError: [Errno 2] No such file or directory: 'output5/pymag.TSPSolution.A1'

It is seemed that concorde is failed. If I run concorde -s 12345 -o output5/pymag.TSPSolution.A1 output5/pymag.TSPLIB.A1 alone, the following is reported:

concorde -s 12345 -o output5/pymag.TSPSolution.A1 output5/pymag.TSPLIB.A1
Host: cluster.hpc.org  Current process id: 28755
Using random seed 12345
output5/pymag.TSPLIB.A1: Value too large for defined data type
Unable to open output5/pymag.TSPLIB.A1 for input
CCutil_gettsplib failed

The file output5/pymag.TSPLIB.A1 exists but is seemed too large.

my command: run_pypmag.py -t sp.tre -d chroms.grimm -o test_out-p1 -p 1
The screen output:

Block state: 1 
Ancestors: A1 A2 A3
raxmlHPC -f A -c 6 -m MULTIGAMMA -p 12345 -t test_out-p1/rerooted.A1 -s test_out-p1/Content.encoding -n Content.A1 -w /Users/JJJ/Documents/Codes/Python/Arachis/issues/2020-08-23-zhangrengang/test_out-p1
raxmlHPC -f A -c 6 -m BINGAMMA -p 12345 -t test_out-p1/rerooted.A1 -s test_out-p1/Order.encoding -n Order.A1 -w /Users/JJJ/Documents/Codes/Python/Arachis/issues/2020-08-23-zhangrengang/test_out-p1
A1 block end nodes: 0 - 17984
A1 terminal nodes: 17985 - 17998
A1 in-block nodes: 17999 - 26990
A1 cost: 652.45s
raxmlHPC -f A -c 6 -m MULTIGAMMA -p 12345 -t test_out-p1/rerooted.A2 -s test_out-p1/Content.encoding -n Content.A2 -w /Users/JJJ/Documents/Codes/Python/Arachis/issues/2020-08-23-zhangrengang/test_out-p1
raxmlHPC -f A -c 6 -m BINGAMMA -p 12345 -t test_out-p1/rerooted.A2 -s test_out-p1/Order.encoding -n Order.A2 -w /Users/JJJ/Documents/Codes/Python/Arachis/issues/2020-08-23-zhangrengang/test_out-p1
A2 block end nodes: 0 - 17984
A2 terminal nodes: 17985 - 18000
A2 in-block nodes: 18001 - 26992
A2 cost: 585.61s
raxmlHPC -f A -c 6 -m MULTIGAMMA -p 12345 -t test_out-p1/rerooted.A3 -s test_out-p1/Content.encoding -n Content.A3 -w /Users/JJJ/Documents/Codes/Python/Arachis/issues/2020-08-23-zhangrengang/test_out-p1
raxmlHPC -f A -c 6 -m BINGAMMA -p 12345 -t test_out-p1/rerooted.A3 -s test_out-p1/Order.encoding -n Order.A3 -w /Users/JJJ/Documents/Codes/Python/Arachis/issues/2020-08-23-zhangrengang/test_out-p1
A3 block end nodes: 0 - 17984
A3 terminal nodes: 17985 - 17996
A3 in-block nodes: 17997 - 26988
A3 cost: 812.83s
Cost: 2105.96s

I've sent the output files to your email.

Sorry for the delay. Gonna look into the details soon. Could you provide the versions of concorde and python?

[zhangrengang]

python 3.6 and concorde "concorde-linux | Concorde for Red Hat Linux 8.0" in http://www.math.uwaterloo.ca/tsp/concorde/downloads/downloads.htm.