Skip to content
This repository has been archived by the owner on Jan 27, 2022. It is now read-only.

Testing Avalon Proxy model with Besu 404 Client Error: Not Found for url: http://rpc.node1.avalon.local:8555 #541

Open
justalittlenoob opened this issue Jun 17, 2020 · 7 comments
Open

Testing Avalon Proxy model with Besu 404 Client Error: Not Found for url: http://rpc.node1.avalon.local:8555 #541

justalittlenoob opened this issue Jun 17, 2020 · 7 comments

Comments

@justalittlenoob
Copy link

I am trying to test proxy model with besu according to this doc. There is an error when i try to run ./eth_generic_client.py -b ethereum --workload_id "echo-result" -o --in_data "Hello" in avalon-shell container. Any advice would be appreciated.

Here are some information may be helpful.

vi /etc/environment
PATH="/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/usr/local/games"
http_proxy=http://abc-proxy.abc.com:914
https_proxy=http://abc-proxy.abc.com:914
ftp_proxy=http://abc-proxy.abc.com:914
no_proxy=10.96.0.0/12,10.67.117.0/24,10.244.0.0/16,127.0.0.1,localhost,node1.avalon.local,rpc.node1.avalon.local,node2.avalon.local, rpc.node2.avalon.local

docker ps -a
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
d3556bde412c avalon-blockchain-connector-eth-dev "bash -c '\n avalon_…" 23 minutes ago Up 23 minutes avalon-blockchain-connector-eth

e4c0dfc4387c avalon-shell-dev "bash -c 'tail -f /d…" 24 minutes ago Up 24 minutes avalon-shell

cc6238a6b6e2 avalon-enclave-manager-dev "bash -c '\n enclave…" 24 minutes ago Up 24 minutes 5555/tcp avalon-enclave-manager

f406125301fe avalon-listener-dev "bash -c '\n avalon_…" 24 minutes ago Up 24 minutes 1947/tcp, 5555/tcp avalon-listener

893ed7ebb9b0 avalon-lmdb-dev "bash -c '\n kv_stor…" 24 minutes ago Up 24 minutes 9090/tcp avalon-lmdb

eb8890062d4d truffle-envt:latest "docker-entrypoint.s…" 59 minutes ago Exited (0) 56 minutes ago truffle-envt-besu-setup

628cfd271c56 pegasyseng/ethsigner:0.4.0 "/opt/ethsigner/bin/…" 59 minutes ago Up 59 minutes 8545/tcp, 0.0.0.0:23001->8555/tcp besu_rpc.node2.avalon.local_1

c765e9e6c2c2 hyperledger/besu:1.4.3 "besu --data-path=/v…" 59 minutes ago Up 59 minutes 8547/tcp, 30303/tcp, 0.0.0.0:23011->8545/tcp, 0.0.0.0:23002->8546/tcp besu_node2.avalon.local_1

9681e68d4374 pegasyseng/ethsigner:0.4.0 "/opt/ethsigner/bin/…" 59 minutes ago Up 59 minutes 8545/tcp, 0.0.0.0:22001->8555/tcp besu_rpc.node1.avalon.local_1

1df85be814df hyperledger/besu:1.4.3 "besu --data-path=/v…" 59 minutes ago Up 59 minutes 8547/tcp, 30303/tcp, 0.0.0.0:22011->8545/tcp, 0.0.0.0:22002->8546/tcp besu_node1.avalon.local_1

/project/avalon/examples/apps/generic_client# ./eth_generic_client.py -b ethereum --workload_id "echo-result" -o --in_data "Hello"
[18:58:34 INFO main] ******* Hyperledger Avalon Generic client *******
[18:58:34 INFO solcx] Using solc version v0.5.15
[18:58:37 WARNING STDERR] Traceback (most recent call last):
[18:58:37 WARNING STDERR] File "./eth_generic_client.py", line 577, in
[18:58:37 WARNING STDERR] Main()
[18:58:37 WARNING STDERR] File "./eth_generic_client.py", line 477, in Main
[18:58:37 WARNING STDERR] worker_id, config)
[18:58:37 WARNING STDERR] File "./eth_generic_client.py", line 337, in _get_first_active_worker
[18:58:37 WARNING STDERR] jrpc_req_id
[18:58:37 WARNING STDERR] File "/usr/local/lib/python3.6/dist-packages/avalon_sdk/connector/blockchains/ethereum/ethereum_worker_registry.py", line 101, in worker_lookup
[18:58:37 WARNING STDERR] worker_type.value, org_id, application_id).call()
[18:58:37 WARNING STDERR] File "/usr/local/lib/python3.6/dist-packages/web3/contract.py", line 959, in call
[18:58:37 WARNING STDERR] **self.kwargs
[18:58:37 WARNING STDERR] File "/usr/local/lib/python3.6/dist-packages/web3/contract.py", line 1493, in call_contract_function
[18:58:37 WARNING STDERR] return_data = web3.eth.call(call_transaction, block_identifier=block_id)
[18:58:37 WARNING STDERR] File "/usr/local/lib/python3.6/dist-packages/eth_utils/functional.py", line 45, in inner
[18:58:37 WARNING STDERR] return callback(fn(*args, **kwargs))
[18:58:37 WARNING STDERR] File "/usr/local/lib/python3.6/dist-packages/web3/eth.py", line 430, in call
[18:58:37 WARNING STDERR] [transaction, block_identifier],
[18:58:37 WARNING STDERR] File "/usr/local/lib/python3.6/dist-packages/web3/manager.py", line 149, in request_blocking
[18:58:37 WARNING STDERR] response = self._make_request(method, params)
[18:58:37 WARNING STDERR] File "/usr/local/lib/python3.6/dist-packages/web3/manager.py", line 128, in _make_request
[18:58:37 WARNING STDERR] return request_func(method, params)
[18:58:37 WARNING STDERR] File "cytoolz/functoolz.pyx", line 250, in cytoolz.functoolz.curry.call
[18:58:37 WARNING STDERR] File "/usr/local/lib/python3.6/dist-packages/web3/middleware/formatting.py", line 75, in apply_formatters
[18:58:37 WARNING STDERR] response = make_request(method, params)
[18:58:37 WARNING STDERR] File "/usr/local/lib/python3.6/dist-packages/web3/middleware/gas_price_strategy.py", line 34, in middleware
[18:58:37 WARNING STDERR] return make_request(method, params)
[18:58:37 WARNING STDERR] File "cytoolz/functoolz.pyx", line 250, in cytoolz.functoolz.curry.call
[18:58:37 WARNING STDERR] File "/usr/local/lib/python3.6/dist-packages/web3/middleware/formatting.py", line 73, in apply_formatters
[18:58:37 WARNING STDERR] response = make_request(method, formatted_params)
[18:58:37 WARNING STDERR] File "/usr/local/lib/python3.6/dist-packages/web3/middleware/attrdict.py", line 33, in middleware
[18:58:37 WARNING STDERR] response = make_request(method, params)
[18:58:37 WARNING STDERR] File "cytoolz/functoolz.pyx", line 250, in cytoolz.functoolz.curry.call
[18:58:37 WARNING STDERR] File "/usr/local/lib/python3.6/dist-packages/web3/middleware/formatting.py", line 73, in apply_formatters
[18:58:37 WARNING STDERR] response = make_request(method, formatted_params)
[18:58:37 WARNING STDERR] File "/usr/local/lib/python3.6/dist-packages/web3/middleware/normalize_errors.py", line 25, in middleware
[18:58:37 WARNING STDERR] result = make_request(method, params)
[18:58:37 WARNING STDERR] File "cytoolz/functoolz.pyx", line 250, in cytoolz.functoolz.curry.call
[18:58:37 WARNING STDERR] File "/usr/local/lib/python3.6/dist-packages/web3/middleware/formatting.py", line 73, in apply_formatters
[18:58:37 WARNING STDERR] response = make_request(method, formatted_params)
[18:58:37 WARNING STDERR] File "cytoolz/functoolz.pyx", line 250, in cytoolz.functoolz.curry.call
[18:58:37 WARNING STDERR] File "/usr/local/lib/python3.6/dist-packages/web3/middleware/formatting.py", line 73, in apply_formatters
[18:58:37 WARNING STDERR] response = make_request(method, formatted_params)
[18:58:37 WARNING STDERR] File "/usr/local/lib/python3.6/dist-packages/web3/middleware/exception_retry_request.py", line 105, in middleware
[18:58:37 WARNING STDERR] return make_request(method, params)
[18:58:37 WARNING STDERR] File "/usr/local/lib/python3.6/dist-packages/web3/providers/rpc.py", line 83, in make_request
[18:58:37 WARNING STDERR] **self.get_request_kwargs()
[18:58:37 WARNING STDERR] File "/usr/local/lib/python3.6/dist-packages/web3/_utils/request.py", line 35, in make_post_request
[18:58:37 WARNING STDERR] response.raise_for_status()
[18:58:37 WARNING STDERR] File "/usr/local/lib/python3.6/dist-packages/requests/models.py", line 940, in raise_for_status
[18:58:37 WARNING STDERR] raise HTTPError(http_error_msg, response=self)
[18:58:37 WARNING STDERR] requests.exceptions
[18:58:37 WARNING STDERR] .
[18:58:37 WARNING STDERR] HTTPError
[18:58:37 WARNING STDERR] :
[18:58:37 WARNING STDERR] 404 Client Error: Not Found for url: http://rpc.node1.avalon.local:8555/
@rranjan3
Copy link
Contributor

rranjan3 commented Jun 22, 2020 •
edited
Loading

If you are behind a proxy, you might need to set the proxy inside docker containers as well.
Try echo $no_proxy from within shell container. If that does not work, you need to ensure the no_proxy is set within your container environment.

One way to do this would be to update ~/.docker/config.json. Refer - https://docs.docker.com/network/proxy/#configure-the-docker-client

@justalittlenoob
Copy link
Author

@rranjan3
Thank you very much, your advice indeed help. After adding no_proxy in the shell container, it works.

Besides, i have the following confuse.

  1. Is this(https://github.com/hyperledger/avalon/blob/master/docs/TestingBesuProxyModel.rst) running in HW mode? If not, how to change it to HW mode?
  2. How to know whether avalon runs in HW mode or SIM mode?

@danintel
Copy link
Contributor

danintel commented Jun 23, 2020 via email

@rranjan3
Copy link
Contributor

Adding to what @danintel suggested.
Once you are done with the SGX specific setup and you want to run the enclave-manager in SGX HW mode with dockers,
For Step 7 in the docs, you could use the below command -

docker-compose -f docker-compose.yaml -f docker-compose-sgx.yaml -f docker-compose-eth-besu.yaml up -d --build

@justalittlenoob
Copy link
Author

@danintel Hi Dan, your advice taught me a lot. Thank you very much.
@rranjan3 I changed my steps according to you advice and a new problem(no workers found) appeared in HW mode. Do you have any suggestions?

sgx sdk/psw: https://github.com/intel/linux-sgx master branch(proxy type = manual and aesm proxy = http://abc-proxy.abc.com:914 are added to /etc/aesmd.conf )
sgx driver: https://github.com/intel/linux-sgx-driver master branch
full log: eth_hw_log_1.txt

docker-compose -f docker-compose.yaml -f docker-compose-sgx.yaml -f docker-compose-eth-besu.yaml up --build
avalon-blockchain-connector-eth | 2020-06-24 17:54:01,750 - INFO - Blockchain to be used as proxy : ethereum
avalon-blockchain-connector-eth | 2020-06-24 17:54:01,751 - INFO - About to start Ethereum connector service
avalon-blockchain-connector-eth | 2020-06-24 17:54:01,754 - INFO - Using solc version v0.5.15
avalon-blockchain-connector-eth | 2020-06-24 17:54:01,824 - INFO - Using solc version v0.5.15
avalon-blockchain-connector-eth | 2020-06-24 17:54:01,924 - INFO - Using solc version v0.5.15
avalon-blockchain-connector-eth | 2020-06-24 17:54:01,966 - INFO - Ethereum Connector service started
avalon-blockchain-connector-eth | 2020-06-24 17:54:01,990 - INFO - Worker lookup response from blockchain: {
avalon-blockchain-connector-eth | "result": {
avalon-blockchain-connector-eth | "totalCount": 0,
avalon-blockchain-connector-eth | "lookupTag": 0,
avalon-blockchain-connector-eth | "ids": []
avalon-blockchain-connector-eth | }
avalon-blockchain-connector-eth | }
avalon-blockchain-connector-eth |
avalon-blockchain-connector-eth | 2020-06-24 17:54:01,990 - ERROR - No workers found in ethereum blockchain
...
avalon-blockchain-connector-eth | 2020-06-24 17:54:01,997 - INFO -
avalon-blockchain-connector-eth | Worker lookup response from kv storage : {
avalon-blockchain-connector-eth | "result": {
avalon-blockchain-connector-eth | "totalCount": 0,
avalon-blockchain-connector-eth | "lookupTag": "",
avalon-blockchain-connector-eth | "ids": []
avalon-blockchain-connector-eth | },
avalon-blockchain-connector-eth | "id": 73207,
avalon-blockchain-connector-eth | "jsonrpc": "2.0"
avalon-blockchain-connector-eth | }
avalon-blockchain-connector-eth |
avalon-blockchain-connector-eth | 2020-06-24 17:54:01,997 - ERROR - No workers found in kv storage
avalon-blockchain-connector-eth | 2020-06-24 17:54:02,013 - INFO - Started listener for events from blockchain
avalon-blockchain-connector-eth | 2020-06-24 17:54:02,016 - INFO - Started handler to handle events
...
avalon-enclave-manager | [17:54:03 WARNING avalon_enclave_manager.base_enclave_info] failure GROUP_OUT_OF_DATE (update your BIOS/microcode!!!) keep going
...
avalon-enclave-manager | [17:54:03 INFO avalon_enclave_manager.work_order_kv_delegate] No workorder entries found in wo-processing table, skipping Cleanup
avalon-enclave-manager | [17:54:03 INFO avalon_enclave_manager.singleton.singleton_enclave_manager] --------------- Boot time flow Complete ----------------
avalon-enclave-manager | [17:54:03 INFO avalon_enclave_manager.singleton.singleton_enclave_manager] ZMQ Port hosted by Enclave
avalon-enclave-manager | [17:54:03 INFO avalon_enclave_manager.singleton.singleton_enclave_manager] Enclave Manager waiting for next request

./eth_generic_client.py -b ethereum --workload_id "echo-result" -o --in_data "Hello"
[17:58:12 INFO main] ******* Hyperledger Avalon Generic client *******
[17:58:12 INFO solcx] Using solc version v0.5.15
[17:58:12 INFO main]
Worker lookup response: {
"result": {
"totalCount": 0,
"lookupTag": 0,
"ids": []
}
}

[17:58:12 ERROR main] No workers found
[17:58:12 ERROR main] Failed to lookup worker
[17:58:12 ERROR main] Cannot proceed without a valid worker

@rranjan3
Copy link
Contributor

@justalittlenoob It must be a race condition we need to fix. The HW mode would take more time for bootupby when the blockchain connector would have started. This would lead to a situation where the connector does not find a worker in the KV storage and hence nothing gets synced up in the proxy blockchain.

Try the command below with a sleep induced (for now) in the blockchain connector container at -
https://github.com/hyperledger/avalon/blob/master/docker-compose-eth-besu.yaml#L34

      bash -c "
        sleep 30
        avalon_ethereum_connector -u http://avalon-listener:1947
        tail -f /dev/null
        "

If you have the images already built, you could ignore the trialing --build.

@justalittlenoob
Copy link
Author

@rranjan3
Thank you very much for your patience. The walk around method make the ERROR - No workers found in kv storage gone. But the No workers found problem in avalon-shell still exist.

After running docker-compose -f docker-compose.yaml -f docker-compose-sgx.yaml -f docker-compose-eth-besu.yaml up -d --build, there will be the following warnings and errors.(full log: eth_hw_log_2.txt)

avalon-enclave-manager | [14:25:14 WARNING STDERR] /usr/lib/python3/dist-packages/urllib3/connectionpool.py:860: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings
avalon-enclave-manager | [14:25:14 WARNING STDERR] InsecureRequestWarning)
avalon-enclave-manager | [14:25:16 WARNING avalon_enclave_manager.base_enclave_info] failure GROUP_OUT_OF_DATE (update your BIOS/microcode!!!) keep going

avalon-enclave-manager | [14:25:16 INFO avalon_enclave_manager.work_order_kv_delegate] No workorder entries found in wo-processing table, skipping Cleanup
avalon-blockchain-connector-eth | 2020-06-29 14:26:14,061 - ERROR - No workers found in ethereum blockchain

root@bb185bc97ec3:/project/avalon/examples/apps/generic_client# ./eth_generic_client.py -b ethereum --workload_id "echo-result" -o --in_data "Hello"

[15:54:52 INFO main] ******* Hyperledger Avalon Generic client *******
[15:54:52 INFO solcx] Using solc version v0.5.15
[15:54:52 INFO main]
Worker lookup response: {
"result": {
"totalCount": 0,
"lookupTag": 0,
"ids": []
}
}
[15:54:52 ERROR main] No workers found
[15:54:52 ERROR main] Failed to lookup worker
[15:54:52 ERROR main] Cannot proceed without a valid worker

Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Assignees
No one assigned
Labels
None yet
Milestone
No milestone
Development

No branches or pull requests
3 participants
@justalittlenoob @danintel @rranjan3