Ethereum ETL lets you convert blockchain data into convenient formats like CSVs and relational databases.

Overview

Ethereum ETL

Build Status Join the chat at https://gitter.im/ethereum-eth Telegram Discord

Ethereum ETL lets you convert blockchain data into convenient formats like CSVs and relational databases.

Do you just want to query Ethereum data right away? Use the public dataset in BigQuery.

Full documentation available here.

Quickstart

Install Ethereum ETL:

pip3 install ethereum-etl

Export blocks and transactions (Schema, Reference):

> ethereumetl export_blocks_and_transactions --start-block 0 --end-block 500000 \
--blocks-output blocks.csv --transactions-output transactions.csv \
--provider-uri https://mainnet.infura.io/v3/7aef3f0cd1f64408b163814b22cc643c

Export ERC20 and ERC721 transfers (Schema, Reference):

> ethereumetl export_token_transfers --start-block 0 --end-block 500000 \
--provider-uri file://$HOME/Library/Ethereum/geth.ipc --output token_transfers.csv

Export traces (Schema, Reference):

> ethereumetl export_traces --start-block 0 --end-block 500000 \
--provider-uri file://$HOME/Library/Ethereum/parity.ipc --output traces.csv

Stream blocks, transactions, logs, token_transfers continually to console (Reference):

> pip3 install ethereum-etl[streaming]
> ethereumetl stream --start-block 500000 -e block,transaction,log,token_transfer --log-file log.txt \
--provider-uri https://mainnet.infura.io/v3/7aef3f0cd1f64408b163814b22cc643c

Find other commands here.

For the latest version, check out the repo and call

> pip3 install -e . 
> python3 ethereumetl.py

Useful Links

Running Tests

> pip3 install -e .[dev,streaming]
> export ETHEREUM_ETL_RUN_SLOW_TESTS=True
> pytest -vv

Running Tox Tests

> pip3 install tox
> tox

Running in Docker

  1. Install Docker https://docs.docker.com/install/

  2. Build a docker image

     > docker build -t ethereum-etl:latest .
     > docker image ls
    
  3. Run a container out of the image

     > docker run -v $HOME/output:/ethereum-etl/output ethereum-etl:latest export_all -s 0 -e 5499999 -b 100000 -p https://mainnet.infura.io
     > docker run -v $HOME/output:/ethereum-etl/output ethereum-etl:latest export_all -s 2018-01-01 -e 2018-01-01 -p https://mainnet.infura.io
    
  4. Run streaming to console or Pub/Sub

     > docker build -t ethereum-etl:latest -f Dockerfile .
     > echo "Stream to console"
     > docker run ethereum-etl:latest stream --start-block 500000 --log-file log.txt
     > echo "Stream to Pub/Sub"
     > docker run -v /path_to_credentials_file/:/ethereum-etl/ --env GOOGLE_APPLICATION_CREDENTIALS=/ethereum-etl/credentials_file.json ethereum-etl:latest stream --start-block 500000 --output projects//topics/crypto_ethereum
    

Projects using Ethereum ETL

  • Google - Public BigQuery Ethereum datasets
  • Nansen by D5 - Analytics platform for Ethereum
Comments
  • Add support for internal transactions

    Add support for internal transactions

    Parity provides APIs for retrieving internal transactions. trace_filter API allows retrieving traces for a range of blocks. This method is already supported by web3.py parity module.

    The scope of this task is to

    1. Design the model (the field names from this PR can be used as a basis: https://github.com/blockchain-etl/ethereum-etl/pull/72).
    2. Implement the export job (the job implementation will mostly resemble the flow in ExportTokenTransfersJob.
    3. Write unit tests (an example can be found here)
    4. Add documentation to README (an example can be found here) (5. Add instructions for how to start parity/geth i.e. what options to use).

    =======================

    Remarks:

    • There will be another task to add the BigQuery schema here
    • Geth also provides support for tracing transactions. There will be another task for implementing internal transactions exporter job for Geth. https://github.com/ethereum/go-ethereum/pull/15516, https://www.linkedin.com/pulse/geth-18-iceberg-michael-hunt-jr-
    opened by medvedev1088 21
  • Add Dockerfile

    Add Dockerfile

    The scope for this task is:

    1. Add Dockerfile (this can be used as a basis). Both ipc and rpc should be supported.
    2. Test it.
    3. Update documentation in README with instructions on how to use it.
    help wanted 
    opened by medvedev1088 15
  • Installing Ethereum-ETL

    Installing Ethereum-ETL

    Dear all,

    unfortunately I can't install ethereum-etl based on the instructions in the readme. Running "pip3 install ethereum-etl" as well as cloning the repo and running "pip3 install -e ." always results in:

    Retrying (Retry(total=4, connect=None, read=None, redirect=None, status=None)) after connection broken by 'ConnectTimeoutError(<urllib3.connection.VerifiedHTTPSConnection object at 0x7f0b79bd0750>, 'Connection to pypi.org timed out. (connect timeout=15)')': /simple/ethereum-etl/

    Could not find a version that satisfies the requirement ethereum-etl (from versions: ) No matching distribution found for ethereum-etl

    Does anybody has an idea what the problem is?

    Thanks and greets Peter

    opened by PeHo89 13
  • Add support for geth traces

    Add support for geth traces

    The scope of this task:

    1. Design the model for geth traces (trace.py can be used as a basis).
    2. Implement the export and extract jobs. 2.a The export job should use traceBlockByNumber (https://github.com/ethereum/go-ethereum/pull/15516) and output raw JSON responses. It should use callTracer, an example JS API call: debug.traceBlockByNumber('0x250252', {'tracer': 'callTracer'}); the same is available via JSON RPC. The job should use BatchWeb3Provider similar to export_blocks_job.py. 2.b The extract job should take the file with raw JSON responses, parses them, map to our domain entity and output to CSV file.
    3. Write unit tests (real mainnet responses should be used in test cases, so the same tests can be run against geth node).
    4. Add documentation to README (an example can be found here). Include instructions for how to start geth i.e. what options to use (e.g. geth --gcmode archive)

    ======================= Notes:

    • Parity traces are already supported https://github.com/blockchain-etl/ethereum-etl/pull/104.
    opened by medvedev1088 13
  • web3 upgrade

    web3 upgrade

    Here are the changes to upgrade web3.py for issue #308.

    • One major change was dropping the support for python 3.6 on ethereum etl since web3.py dropped 3.6 support
    • You may want to take a look at these notes on upgrading from web3.py v4 to v5 to see if anything jumps out at you.
    • I did still have the infura test falling locally but I think I needed to just dig in a little more and see why my setup wasn't working. If I have time tomorrow I will look at that.
    • I had to add a few of those mock response files. I copied ones you already had in there and modified them a little.
    • Also, I did not test the docker file but noticed it was using python3.6 so just changed the image it was pulling from to python3.7 hope that works.

    Let me know what you think, happy to tweak this some more if needed.

    opened by dbfreem 11
  • Windows - SyntaxError: invalid syntax

    Windows - SyntaxError: invalid syntax

    C:\Users\XPS\Documents\Python-Ethereum\ethereum-etl-master>python export_blocks_and_transactions.py --start-block 0 --end-block 500000 \ --ipc-path ~/Library/Ethereum/geth.ipc --blocks-output blocks.csv --transactions-output transactions.csv Traceback (most recent call last): File "export_blocks_and_transactions.py", line 4, in from ethereumetl.ipc import IPCWrapper File "C:\Users\XPS\Documents\Python-Ethereum\ethereum-etl-master\ethereumetl\ipc.py", line 5, in from web3.providers.ipc import get_default_ipc_path, PersistantSocket File "C:\Users\XPS\AppData\Local\Programs\Python\Python37-32\lib\site-packages\web3_init_.py", line 8, in from web3.main import Web3 # noqa: E402 File "C:\Users\XPS\AppData\Local\Programs\Python\Python37-32\lib\site-packages\web3\main.py", line 26, in from web3.providers.eth_tester import ( File "C:\Users\XPS\AppData\Local\Programs\Python\Python37-32\lib\site-packages\web3\providers_init_.py", line 8, in from .websocket import WebsocketProvider # noqa: F401 File "C:\Users\XPS\AppData\Local\Programs\Python\Python37-32\lib\site-packages\web3\providers\websocket.py", line 9, in import websockets File "C:\Users\XPS\AppData\Local\Programs\Python\Python37-32\lib\site-packages\websockets_init_.py", line 3, in from .client import * File "C:\Users\XPS\AppData\Local\Programs\Python\Python37-32\lib\site-packages\websockets\client.py", line 20, in from .protocol import WebSocketCommonProtocol File "C:\Users\XPS\AppData\Local\Programs\Python\Python37-32\lib\site-packages\websockets\protocol.py", line 18, in from .compatibility import asyncio_ensure_future File "C:\Users\XPS\AppData\Local\Programs\Python\Python37-32\lib\site-packages\websockets\compatibility.py", line 15 asyncio_ensure_future = asyncio.async # Python < 3.5 ^ SyntaxError: invalid syntax

    bug duplicate 
    opened by RaianeQC 11
  • adds support for non-mainnet in etl stream

    adds support for non-mainnet in etl stream

    relates #178

    DEMO on ronin network, an eth sidechain. Notice that latest block can now be read, thanks to middleware injection.

    2021-08-12 19:24:34,321 - ProgressLogger [INFO] - Started work. Items to process: 1.
    2021-08-12 19:24:34,375 - ProgressLogger [INFO] - 1 items processed. Progress is 100%.
    2021-08-12 19:24:34,375 - ProgressLogger [INFO] - Finished work. Total items processed: 1. Took 0:00:00.054600.
    2021-08-12 19:24:34,376 - ProgressLogger [INFO] - Started work.
    2021-08-12 19:24:34,376 - ProgressLogger [INFO] - Finished work. Total items processed: 0. Took 0:00:00.000169.
    2021-08-12 19:24:34,376 - ProgressLogger [INFO] - Started work.
    2021-08-12 19:24:34,376 - ProgressLogger [INFO] - Finished work. Total items processed: 0. Took 0:00:00.000143.
    
    opened by numonedad 10
  • Ethereum data export error

    Ethereum data export error

    When i use “ethereumetl export_all -s 0 -e 5999999 -b 100000 -p file://$HOME/Library/Ethereum/geth.ipc -o output”,The following error occurred:“ValueError: Unknown uri scheme /datalake/blockdata/geth.ipc”,Ask what is the reason

    opened by sznariOsmosis 10
  • Add support for extracting Origin Protocol data

    Add support for extracting Origin Protocol data

    This implements a prototype that extracts data generated by Origin Protocol's open source platform for building decentralized marketplaces on Ethereum and IPFS.

    Fixes https://github.com/blockchain-etl/ethereum-etl/issues/215

    opened by franckc 10
  • ValueError: result is None in response (export_all)

    ValueError: result is None in response (export_all)

    I run it on ubuntu

    ethereumetl export_all -s 4900000 -e 4999999 -b 100000 -p https://mainnet.infura.io -o output

    but after n time it has error:

    File "/usr/local/lib/python3.6/dist-packages/ethereumetl/jobs/export_blocks_job.py", line 70, in _export │ File "/usr/local/lib/python3.6/dist-packages/ethereumetl/jobs/export_blocks_job.py", line 70, in _export total_items=self.end_block - self.start_block + 1 │ total_items=self.end_block - self.start_block + 1 File "/usr/local/lib/python3.6/dist-packages/ethereumetl/executors/batch_work_executor.py", line 48, in execute │ File "/usr/local/lib/python3.6/dist-packages/ethereumetl/executors/batch_work_executor.py", line 48, in execute self.executor.submit(self._fail_safe_execute, work_handler, batch) │ self.executor.submit(self._fail_safe_execute, work_handler, batch) File "/usr/local/lib/python3.6/dist-packages/ethereumetl/executors/fail_safe_executor.py", line 31, in submit │ File "/usr/local/lib/python3.6/dist-packages/ethereumetl/executors/fail_safe_executor.py", line 31, in submit self._check_completed_futures() │ self._check_completed_futures() File "/usr/local/lib/python3.6/dist-packages/ethereumetl/executors/fail_safe_executor.py", line 47, in _check_compl│ File "/usr/local/lib/python3.6/dist-packages/ethereumetl/executors/fail_safe_executor.py", line 47, in _check_comp eted_futures │leted_futures future.result() │ future.result() File "/usr/lib/python3.6/concurrent/futures/_base.py", line 425, in result │ File "/usr/lib/python3.6/concurrent/futures/_base.py", line 425, in result return self.__get_result() │ return self.__get_result() File "/usr/lib/python3.6/concurrent/futures/_base.py", line 384, in __get_result │ File "/usr/lib/python3.6/concurrent/futures/_base.py", line 384, in __get_result raise self._exception │ raise self._exception File "/usr/lib/python3.6/concurrent/futures/thread.py", line 56, in run │ File "/usr/lib/python3.6/concurrent/futures/thread.py", line 56, in run result = self.fn(*self.args, **self.kwargs) │ result = self.fn(*self.args, **self.kwargs) File "/usr/local/lib/python3.6/dist-packages/ethereumetl/executors/batch_work_executor.py", line 53, in _fail_safe_│ File "/usr/local/lib/python3.6/dist-packages/ethereumetl/executors/batch_work_executor.py", line 53, in _fail_safe execute │_execute work_handler(batch) │ work_handler(batch) File "/usr/local/lib/python3.6/dist-packages/ethereumetl/jobs/export_blocks_job.py", line 77, in _export_batch │ File "/usr/local/lib/python3.6/dist-packages/ethereumetl/jobs/export_blocks_job.py", line 77, in _export_batch blocks = [self.block_mapper.json_dict_to_block(result) for result in results] │ blocks = [self.block_mapper.json_dict_to_block(result) for result in results] File "/usr/local/lib/python3.6/dist-packages/ethereumetl/jobs/export_blocks_job.py", line 77, in <listcomp> │ File "/usr/local/lib/python3.6/dist-packages/ethereumetl/jobs/export_blocks_job.py", line 77, in <listcomp> blocks = [self.block_mapper.json_dict_to_block(result) for result in results] │ blocks = [self.block_mapper.json_dict_to_block(result) for result in results] File "/usr/local/lib/python3.6/dist-packages/ethereumetl/utils.py", line 58, in rpc_response_batch_to_results │ File "/usr/local/lib/python3.6/dist-packages/ethereumetl/utils.py", line 58, in rpc_response_batch_to_results yield rpc_response_to_result(response_item) │ yield rpc_response_to_result(response_item) File "/usr/local/lib/python3.6/dist-packages/ethereumetl/utils.py", line 67, in rpc_response_to_result │ File "/usr/local/lib/python3.6/dist-packages/ethereumetl/utils.py", line 67, in rpc_response_to_result raise ValueError(error_message) │ raise ValueError(error_message) ValueError: result is None in response {'jsonrpc': '2.0', 'id': 0, 'error': {'code': -32603, 'message': 'request proc│ValueError: result is None in response {'jsonrpc': '2.0', 'id': 0, 'error': {'code': -32603, 'message': 'request pro essing failed'}}.

    opened by minhtrih 10
  • Error in exporting traces using Parity

    Error in exporting traces using Parity

    @medvedev1088 We used Parity to export traces. It works perfectly for almost every block. BUT error happens in some block ranges, such as 2,280,000~2,400,000 and 3800000~4,000,000. Error:web3.utils.threads.Timeout: 60 seconds image

    Our server has far more than 8 GB memory. Maybe we could change default time. But we could not figure out the correct command

    opened by cxd1995 10
  • Add decoding log functions here

    Add decoding log functions here

    https://github.com/blockchain-etl/ethereum-etl/blob/6710e6b894789d83ce3c28db79b4ec5bf2ec3845/ethereumetl/jobs/export_receipts_job.py#L78

    Add feature to decode logs in the exportation of receipts

    opened by pauline-thrackle 0
  • [Errno 101]

    [Errno 101]

    Command: ethereumetl export_blocks_and_transactions --start-block 0 --end-block 500000
    --blocks-output blocks.csv --transactions-output transactions.csv
    --provider-uri https://mainnet.infura.io/v3/7aef3f0cd1f64408b163814b22cc643c Error: urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='mainnet.infura.io', port=443): Max retries exceeded with url: /v3/7aef3f0cd1f64408b163814b22cc643c (Caused by NewConnectionError('<urllib3.connection.HTTPSConnection object at 0x7f4b8fd63910>: Failed to establish a new connection: [Errno 101] Network is unreachable')) requests.exceptions.ConnectionError: HTTPSConnectionPool(host='mainnet.infura.io', port=443): Max retries exceeded with url: /v3/7aef3f0cd1f64408b163814b22cc643c (Caused by NewConnectionError('<urllib3.connection.HTTPSConnection object at 0x7f4b8fd621d0>: Failed to establish a new connection: [Errno 101] Network is unreachable')) how to solve this problem?

    opened by Facebeside 0
  • M1 mac issue Symbolic Execution not available: No module named 'mythril.ether'

    M1 mac issue Symbolic Execution not available: No module named 'mythril.ether'

    System:

    • M1 Mac
    • Python 3.9.15
    • pip 22.1.2
    • Poetry (version 1.2.2)

    pyproject.toml

    [tool.poetry]
    name = "asd"
    version = "0.1.0"
    description = ""
    authors = ["Your Name <[email protected]>"]
    readme = "README.md"
    packages = [{include = "asd"}]
    
    [tool.poetry.dependencies]
    python = "^3.9"
    ethereum-etl = "^2.1.1"
    pyetherchain = "^0.3.2"
    blake2b-py = { git = "https://github.com/ethereum/blake2b-py.git", tag="v0.1.4" }
    mythril = {extras = ["mythril"], version = "^0.23.12"}
    
    
    
    [build-system]
    requires = ["poetry-core"]
    build-backend = "poetry.core.masonry.api"
    

    reprod script

    from ethereumetl.service.eth_contract_service import EthContractService
    
    bytecode_as_str = '608060405234801561001057600080fd5b506040516111893803806111898339818101604052602081101561003357600080fd5b810190808051906020019092919050505060016000803373ffffffffffffffffffffffffffffffffffffffff1673ffffffffffffffffffffffffffffffffffffffff1681526020019081526020016000208190555080600260006101000a81548173ffffffffffffffffffffffffffffffffffffffff021916908373ffffffffffffffffffffffffffffffffffffffff1602179055506b033b2e3c9fd0803ce8000000600381905550600160048190555050611095806100f46000396000f3fe608060405234801561001057600080fd5b50600436106100b45760003560e01c80636924500911610071578063692450091461020d578063957aa58c146102175780639c52a7f114610235578063bf353dbb14610279578063d9638d36146102d1578063ebecb39d14610346576100b4565b80631504460f146100b95780631a0b287e146100e757806329ae81141461012957806336569e7714610161578063495d32cb146101ab57806365fae35e146101c9575b600080fd5b6100e5600480360360208110156100cf57600080fd5b810190808035906020019092919050505061039e565b005b610127600480360360608110156100fd57600080fd5b810190808035906020019092919080359060200190929190803590602001909291905050506105bd565b005b61015f6004803603604081101561013f57600080fd5b8101908080359060200190929190803590602001909291905050506107d7565b005b6101696109dc565b604051808273ffffffffffffffffffffffffffffffffffffffff1673ffffffffffffffffffffffffffffffffffffffff16815260200191505060405180910390f35b6101b3610a02565b6040518082815260200191505060405180910390f35b61020b600480360360208110156101df57600080fd5b81019080803573ffffffffffffffffffffffffffffffffffffffff169060200190929190505050610a08565b005b610215610b36565b005b61021f610c27565b6040518082815260200191505060405180910390f35b6102776004803603602081101561024b57600080fd5b81019080803573ffffffffffffffffffffffffffffffffffffffff169060200190929190505050610c2d565b005b6102bb6004803603602081101561028f57600080fd5b81019080803573ffffffffffffffffffffffffffffffffffffffff169060200190929190505050610d5b565b6040518082815260200191505060405180910390f35b6102fd600480360360208110156102e757600080fd5b8101908080359060200190929190505050610d73565b604051808373ffffffffffffffffffffffffffffffffffffffff1673ffffffffffffffffffffffffffffffffffffffff1681526020018281526020019250505060405180910390f35b61039c6004803603606081101561035c57600080fd5b810190808035906020019092919080359060200190929190803573ffffffffffffffffffffffffffffffffffffffff169060200190929190505050610db7565b005b6000806001600084815260200190815260200160002060000160009054906101000a900473ffffffffffffffffffffffffffffffffffffffff1673ffffffffffffffffffffffffffffffffffffffff166359e02dd76040518163ffffffff1660e01b81526004016040805180830381600087803b15801561041e57600080fd5b505af1158015610432573d6000803e3d6000fd5b505050506040513d604081101561044857600080fd5b810190808051906020019092919080519060200190929190505050915091506000816104755760006104b1565b6104b061049461048c8560001c633b9aca0061100b565b600354611037565b6001600087815260200190815260200160002060010154611037565b5b9050600260009054906101000a900473ffffffffffffffffffffffffffffffffffffffff1673ffffffffffffffffffffffffffffffffffffffff16631a0b287e85836040518363ffffffff1660e01b815260040180838152602001807f73706f740000000000000000000000000000000000000000000000000000000081525060200182815260200192505050600060405180830381600087803b15801561055857600080fd5b505af115801561056c573d6000803e3d6000fd5b505050507fdfd7467e425a8107cfd368d159957692c25085aacbcf5228ce08f10f2146486e84848360405180848152602001838152602001828152602001935050505060405180910390a150505050565b60016000803373ffffffffffffffffffffffffffffffffffffffff1673ffffffffffffffffffffffffffffffffffffffff1681526020019081526020016000205414610671576040517f08c379a00000000000000000000000000000000000000000000000000000000081526004018080602001828103825260168152602001807f53706f747465722f6e6f742d617574686f72697a65640000000000000000000081525060200191505060405180910390fd5b6001600454146106e9576040517f08c379a00000000000000000000000000000000000000000000000000000000081526004018080602001828103825260108152602001807f53706f747465722f6e6f742d6c6976650000000000000000000000000000000081525060200191505060405180910390fd5b7f6d617400000000000000000000000000000000000000000000000000000000008214156107315780600160008581526020019081526020016000206001018190555061079f565b6040517f08c379a000000000000000000000000000000000000000000000000000000000815260040180806020018281038252601f8152602001807f53706f747465722f66696c652d756e7265636f676e697a65642d706172616d0081525060200191505060405180910390fd5b5961012081016040526020815260e0602082015260e0600060408301376024356004353360003560e01c60e01b61012085a450505050565b60016000803373ffffffffffffffffffffffffffffffffffffffff1673ffffffffffffffffffffffffffffffffffffffff168152602001908152602001600020541461088b576040517f08c379a00000000000000000000000000000000000000000000000000000000081526004018080602001828103825260168152602001807f53706f747465722f6e6f742d617574686f72697a65640000000000000000000081525060200191505060405180910390fd5b600160045414610903576040517f08c379a00000000000000000000000000000000000000000000000000000000081526004018080602001828103825260108152602001807f53706f747465722f6e6f742d6c6976650000000000000000000000000000000081525060200191505060405180910390fd5b7f706172000000000000000000000000000000000000000000000000000000000082141561093757806003819055506109a5565b6040517f08c379a000000000000000000000000000000000000000000000000000000000815260040180806020018281038252601f8152602001807f53706f747465722f66696c652d756e7265636f676e697a65642d706172616d0081525060200191505060405180910390fd5b5961012081016040526020815260e0602082015260e0600060408301376024356004353360003560e01c60e01b61012085a4505050565b600260009054906101000a900473ffffffffffffffffffffffffffffffffffffffff1681565b60035481565b60016000803373ffffffffffffffffffffffffffffffffffffffff1673ffffffffffffffffffffffffffffffffffffffff1681526020019081526020016000205414610abc576040517f08c379a00000000000000000000000000000000000000000000000000000000081526004018080602001828103825260168152602001807f53706f747465722f6e6f742d617574686f72697a65640000000000000000000081525060200191505060405180910390fd5b60016000808373ffffffffffffffffffffffffffffffffffffffff1673ffffffffffffffffffffffffffffffffffffffff168152602001908152602001600020819055505961012081016040526020815260e0602082015260e0600060408301376024356004353360003560e01c60e01b61012085a45050565b60016000803373ffffffffffffffffffffffffffffffffffffffff1673ffffffffffffffffffffffffffffffffffffffff1681526020019081526020016000205414610bea576040517f08c379a00000000000000000000000000000000000000000000000000000000081526004018080602001828103825260168152602001807f53706f747465722f6e6f742d617574686f72697a65640000000000000000000081525060200191505060405180910390fd5b60006004819055505961012081016040526020815260e0602082015260e0600060408301376024356004353360003560e01c60e01b61012085a450565b60045481565b60016000803373ffffffffffffffffffffffffffffffffffffffff1673ffffffffffffffffffffffffffffffffffffffff1681526020019081526020016000205414610ce1576040517f08c379a00000000000000000000000000000000000000000000000000000000081526004018080602001828103825260168152602001807f53706f747465722f6e6f742d617574686f72697a65640000000000000000000081525060200191505060405180910390fd5b60008060008373ffffffffffffffffffffffffffffffffffffffff1673ffffffffffffffffffffffffffffffffffffffff168152602001908152602001600020819055505961012081016040526020815260e0602082015260e0600060408301376024356004353360003560e01c60e01b61012085a45050565b60006020528060005260406000206000915090505481565b60016020528060005260406000206000915090508060000160009054906101000a900473ffffffffffffffffffffffffffffffffffffffff16908060010154905082565b60016000803373ffffffffffffffffffffffffffffffffffffffff1673ffffffffffffffffffffffffffffffffffffffff1681526020019081526020016000205414610e6b576040517f08c379a00000000000000000000000000000000000000000000000000000000081526004018080602001828103825260168152602001807f53706f747465722f6e6f742d617574686f72697a65640000000000000000000081525060200191505060405180910390fd5b600160045414610ee3576040517f08c379a00000000000000000000000000000000000000000000000000000000081526004018080602001828103825260108152602001807f53706f747465722f6e6f742d6c6976650000000000000000000000000000000081525060200191505060405180910390fd5b7f7069700000000000000000000000000000000000000000000000000000000000821415610f6557806001600085815260200190815260200160002060000160006101000a81548173ffffffffffffffffffffffffffffffffffffffff021916908373ffffffffffffffffffffffffffffffffffffffff160217905550610fd3565b6040517f08c379a000000000000000000000000000000000000000000000000000000000815260040180806020018281038252601f8152602001807f53706f747465722f66696c652d756e7265636f676e697a65642d706172616d0081525060200191505060405180910390fd5b5961012081016040526020815260e0602082015260e0600060408301376024356004353360003560e01c60e01b61012085a450505050565b600080821480611028575082828385029250828161102557fe5b04145b61103157600080fd5b92915050565b600081611050846b033b2e3c9fd0803ce800000061100b565b8161105757fe5b0490509291505056fea265627a7a72315820b20103aad12e7c2a10372688d31c070bc7aa268459f8bc3a58d4298c7f6bdf0964736f6c634300050c003200000000000000000000000035d1b3f3d7966a1dfe207aa4514c12a259a0492b'
    
    def main():
        contract_service = EthContractService()
        func_sigs = contract_service.get_function_sighashes(bytecode_as_str)
        print(func_sigs)
    
    
    if __name__ == '__main__':
        main()
    
    opened by 007vasy 0
  • stream command failed to extract contract information

    stream command failed to extract contract information

    Command : ethereumetl stream --start-block 15789281 -e block,transaction,log,token_transfer,contract --log-file log.txt --provider-uri https://mainnet.infura.io/v3/9aa3d95b3bc440fa88ea12eaa4456161

    Error : 2022-11-30 16:07:55,003 - root [ERROR] - An exception occurred while syncing block data. Traceback (most recent call last): File "/Users/chintanpatel/Library/Python/3.8/lib/python/site-packages/blockchainetl/streaming/streamer.py", line 77, in _do_stream synced_blocks = self._sync_cycle() File "/Users/chintanpatel/Library/Python/3.8/lib/python/site-packages/blockchainetl/streaming/streamer.py", line 98, in _sync_cycle self.blockchain_streamer_adapter.export_all(self.last_synced_block + 1, target_block) File "/Users/chintanpatel/Library/Python/3.8/lib/python/site-packages/ethereumetl/streaming/eth_streamer_adapter.py", line 62, in export_all traces = self._export_traces(start_block, end_block) File "/Users/chintanpatel/Library/Python/3.8/lib/python/site-packages/ethereumetl/streaming/eth_streamer_adapter.py", line 159, in _export_traces job.run() File "/Users/chintanpatel/Library/Python/3.8/lib/python/site-packages/blockchainetl/jobs/base_job.py", line 30, in run self._end() File "/Users/chintanpatel/Library/Python/3.8/lib/python/site-packages/ethereumetl/jobs/export_traces_job.py", line 106, in _end self.batch_work_executor.shutdown() File "/Users/chintanpatel/Library/Python/3.8/lib/python/site-packages/ethereumetl/executors/batch_work_executor.py", line 97, in shutdown self.executor.shutdown() File "/Users/chintanpatel/Library/Python/3.8/lib/python/site-packages/ethereumetl/executors/fail_safe_executor.py", line 39, in shutdown self._check_completed_futures() File "/Users/chintanpatel/Library/Python/3.8/lib/python/site-packages/ethereumetl/executors/fail_safe_executor.py", line 47, in _check_completed_futures future.result() File "/Library/Developer/CommandLineTools/Library/Frameworks/Python3.framework/Versions/3.8/lib/python3.8/concurrent/futures/_base.py", line 437, in result return self.__get_result() File "/Library/Developer/CommandLineTools/Library/Frameworks/Python3.framework/Versions/3.8/lib/python3.8/concurrent/futures/_base.py", line 389, in __get_result raise self._exception File "/Library/Developer/CommandLineTools/Library/Frameworks/Python3.framework/Versions/3.8/lib/python3.8/concurrent/futures/thread.py", line 57, in run result = self.fn(*self.args, **self.kwargs) File "/Users/chintanpatel/Library/Python/3.8/lib/python/site-packages/ethereumetl/executors/batch_work_executor.py", line 63, in _fail_safe_execute work_handler(batch) File "/Users/chintanpatel/Library/Python/3.8/lib/python/site-packages/ethereumetl/jobs/export_traces_job.py", line 90, in _export_batch json_traces = self.web3.parity.traceBlock(block_number) File "/Users/chintanpatel/Library/Python/3.8/lib/python/site-packages/web3/module.py", line 57, in caller result = w3.manager.request_blocking(method_str, File "/Users/chintanpatel/Library/Python/3.8/lib/python/site-packages/web3/manager.py", line 198, in request_blocking return self.formatted_response(response, File "/Users/chintanpatel/Library/Python/3.8/lib/python/site-packages/web3/manager.py", line 171, in formatted_response raise ValueError(response["error"]) ValueError: {'code': -32601, 'message': 'the method trace_block does not exist/is not available'}

    opened by infocodiste 1
  • can't support proxy contract erc20

    can't support proxy contract erc20

    I looked at the source code and found that bytecode is used to judge whether it is erc20, but many contracts currently use the proxy contract upgrade solution to deploy the contract, which makes it impossible to judge whether it is erc20

    opened by SuperEdison 2
Releases(v2.1.2)
Owner
Blockchain ETL
Facilitating data science on blockchain data. Available in Google BigQuery https://goo.gl/oY5BCQ
Blockchain ETL
Smart-contracts - open sourcing our upcoming smart contracts for better security and transparency

Smart-contracts - open sourcing our upcoming smart contracts for better security and transparency

Rand Gallery 16 Jul 10, 2022
PeGuard - Windows PE crypter and packing utility

PEGUARD PEGUARD is a file crypter and packing utility. This project was original

11 Nov 28, 2022
dashboard to track crypto prices and change via the coinmarketcap APIs

crypto-dashboard Dashboard to track crypto prices and change via the coinmarketcap APIs. Uses chart.js and ag-grid. Requirements: python 3 (was writte

4 Nov 09, 2021
A community effort to bring back Duino-Coin

Duino-Coin-Revived A community effort to bring back Duino-Coin! Along with reviving the cryptocurrency, we will add many improvements to it, including

1 Dec 22, 2021
Python Encryption Name Game

Python 3.9.7 Encryption Name Game Encrypt a name with numbers using a Caesar cipher! You can choose different numbers to encrypt your name from 1 to o

Armand Brunelle 3 Dec 24, 2021
Active github repos of all cryptocurrencies

This repo is to maintain the list of active repositories for all cryptocurrencies that https://codemask.org uses. The active list will be automaticall

CodeMask 5 May 20, 2022
A python-jvm wrapper for interacting with the Ergo blockchain.

ergpy A python-jvm wrapper for interacting with the Ergo blockchain. Usage Requires Java 8 or higher to be installed on system pip install JPype1 pip

mgpai 16 Oct 05, 2022
Hide secret texts inside an image, optionally encrypt them with a password using AES-256.

Hide secret texts/messages inside an image. You can optionally encrypt your texts with a password using AES-256 before encoding into the image.

Teja Swaroop 97 Dec 29, 2022
Lottery by Ethereum Blockchain

Lottery by Ethereum Blockchain Set your web3 provider url in .env PROVIDER=https://mainnet.infura.io/v3/YOUR-INFURA-TOKEN Create your source file .

John Torres 3 Dec 23, 2021
The leading native Python SSHv2 protocol library.

Paramiko Paramiko: Python SSH module Copyright: Copyright (c) 2009 Robey Pointer 8.1k Jan 08, 2023

E2EE disabling plugin for Synapse

E2EE disabling plugin for Synapse This Pluggable Module disables end-to-end encryption in a self-hosted Synapse servers. It works by stripping out req

Konstantin Sharlaimov 9 Nov 30, 2022
Simple encryption/decryption utility using Pycryptodome module. Working with AES and RSA algorithms.

EncypherUtil Simple encryption/decryption utility using PyCryptodome module. Working with AES and RSA algorithms. THIS UTILITY IS NOT LICENSED AS CRYP

Egor Yakubovich 0 Jun 14, 2022
Technical_indicators_cryptos - Using technical indicators to find optimal trading strategies to deploy onto trading bot.

technical_indicators_cryptos Using technical indicators to find optimal trading strategies to deploy onto trading bot. In the Jup Notebook you wil

Van 4 Jul 03, 2022
SimpleWallet - Simple wallet for Bitcoin

Simple Wallet This is a basic python starter package to be used as a template fo

Mystic 1 Jan 08, 2022
Simple bitcoin ticker for the Pimorono Inky pHAT Red.

bitcoin-ticker Simple bitcoin ticker for the Pimorono Inky pHAT Red. Equipment Raspberry Pi Zero W v1.1 or Pi 2 model b v1.1 Pimorono Inky pHAT Red (S

2 Mar 15, 2022
Простой шифратор работающий по ключам.

deCryptor Что это такое? Простой шифратор работающий по ключам и без них. Как пользоваться? СМОТРЕТЬ ИЗОБРАЖЕНИЕ Разработчики Роман Слабицкий - написа

Romanin 2 May 31, 2022
Enchpyter, is able to encrypt and decrypt words as you determine, of course, according to the alphabet.

Enchpyter Enchpyter is a program do encrypt and decrypt any word you want (just letters). You enter how many letters jumps and write the word, so, the

João Assalim 2 Oct 10, 2022
Distributed, blockchain based hashtables middleware for deduplication of file uploads to the cloud

distributed-blockchain-based-secure-file-dedupe Searching is Distributed, Block and Access List for each upload is unique and it is stored in a single

Abhishek Tangod 1 Dec 02, 2021
Linear encryption software programmed with python

Echoder linear encryption software programmed with python How does it work? The text in the text section runs a function with two keys entered keys mu

Emre Orhan 4 Dec 20, 2021
Simple python crypto bot to trade crypto on Binance based on RSI. Utilizing web sockets to get real-time prices

Py Crypto Bot Using Binance WebSocket API to get real-time price data for cryptocurrencies. Using the TA-Lib library to calculate the RSI and execute

Kennedy Ngugi Mwaura 15 Jan 04, 2023