-
Notifications
You must be signed in to change notification settings - Fork 48
Count the number of transactions from files containing Bitcoin Blockchain data
This is a MapReduce application demonstrating some of the capabilities of the hadoopcryptoledger library. It takes as input a set of files on HDFS containing Bitcoin Blockchain data. As output it returns the number of transactions found in the blockchain data. It has successfully been tested with the Cloudera Quickstart VM 5.5 and HDP Sandbox 2.5, but other Hadoop distributions should work equally well.
See here how to fetch Bitcoin blockchain data.
After it has been copied you are ready to use the example.
Execute
git clone https://github.com/ZuInnoTe/hadoopcryptoledger.git hadoopcryptoledger
You can build the application by changing to the directory hadoopcryptoledger/examples/mapreduce-bitcoinblock and using the following command:
../gradlew clean build integrationTest
This will also execute the integration tests validating readiness of the application
Make sure that the output directory is clean:
hadoop fs -rm -R /user/bitcoin/output
Execute the following command
hadoop jar ./build/libs/example-hcl-mr-bitcoinblock-0.1.0.jar org.zuinnote.hadoop.bitcoin.example.driver.BitcoinBlockCounterDriver /user/bitcoin/input /user/bitcoin/output
After the map/reduce job has completed, you find the result in /user/bitcoin/output. You can display it using the following command:
hadoop fs -cat /user/bitcoin/output/part-r-00000
Understanding the structure of Bitcoin data:
Blocks: https://en.bitcoin.it/wiki/Block
Transactions: https://en.bitcoin.it/wiki/Transactions