Flare Network Ftso V2 Setup

Table of Contents

FTSO V2 Data Provider Information

Markdown Code

# ftso-v2-provider-deployment

## FTSOV2 Overview

![Overview](Overview.png)

A data provider system for FTSOv2 consists of the following components:

1. **Flare System Client**. Responsible for all interactions with the FTSOv2 smart contracts, including data collection and submission, voter registration, and additional system tasks.
2. **Data Provider**. Provides commit, reveal, and median result data to System Client for submission.
3. **Feed Value Provider**. Provides current values (prices) for a given set of feeds.
4. **Indexer**. Monitors the blockchain and records all FTSOv2 related transactions and events.
5. **Fast Updates Client**. Responsible for interaction with Fast Updates contracts, including data collection and submission and other system tasks.

Reference implementations are provided for **Indexer**, **Flare System Client**, **Data Provider**, **Fast Updats Client**, and providers are expected to plug in their own **Feed Value Provider** implementing a specific REST API (there is an sample implementation for testing).

### Operation

The following is a very simplified description of a single voting round operation.

**System Client** runs a scheduler which triggers voting actions every round (90s):
- On voting round start, obtain reveal data for the previous round from **Data Provider** and send to `Submission` smart contract.
- Once the reveal deadline passes (45s), obtain median result Merkle root from **Data Provider**, sign, and send to `Submission` smart contract. 
- Before the end of the voting round, obtain a feed value commit hash for the current round from the **Data Provider** (which will get processed in the following round).
- There is finalizer process which monitors the indexer database for signature transactions, and once enough signature weight for a voting round is gathered, submits the set of signatures to the `Relay` smart contract. If signature verification passes, the voting round is considered finalized. The `Relay` contract is the authoritative storage of confirmed voting round result Merkle roots.

Additionally, once in a reward epoch the **System Client** triggers voter registration, which allows participating in the following reward epoch.

**Data Provider** obtains all commit and reveal data straight from encoded transaction calldata recorded in the indexer database. All calls to `Submission` contract are simply empty function invocations, with the actual submission data provided as additional calldata on the transaction.

## Deployment

### Entity registration

You will need to setup your Entity by following [REGISTRATION.md](docs/REGISTRATION.md).

### Dependencies

You will also need to install some dependencies on your system:
- [jq](https://jqlang.github.io/jq/)
- [envsubst](https://www.gnu.org/software/gettext/manual/html_node/envsubst-Invocation.html)
- [docker](https://www.docker.com/)
- [bash] (macOs only for updated version) `brew install bash`

### Stack deployment
Use `.env.example` to create `.env` file (eg.: using `cp .env.example .env`) and fill out all values.

Populate configs for provider stack by running `./populate_config.sh`. **NOTE: You'll need to rerun this command if you change your `.env` file.**

Since docker-compose.yaml is provided you can start everything with `docker compose up -d` and stop everything with `docker compose down`. Database is persisted in a named docker volume. If you need to wipe database you need to remove the volume manually. When upstream codebase is changed new images need to be pulled with `docker compose pull`.

### Feed value provider

Start your own feed value provider or alternatively use example provider:
```bash
docker run --rm -it \
  --publish "0.0.0.0:3101:3101" \
  --network "ftso-v2-deployment_default" \
  ghcr.io/flare-foundation/ftso-v2-example-value-provider

Once the container is running, you can find the API spec at: http://localhost:3101/api-doc.

Note: some users reported issues with getting the provider to start. For initial testing a “fixed” value provider can be used that simply returns a constant instead of reading data from exchanges. It can be started by setting an extra env variable VALUE_PROVIDER_IMPL=fixed:

docker run --rm -it \
  --env VALUE_PROVIDER_IMPL=fixed \
  --publish "0.0.0.0:3101:3101" \
  --network "ftso-v2-deployment_default" \
  ghcr.io/flare-foundation/ftso-v2-example-value-provider

You should see the following in the logs:

WARN [FixedFeed] Initializing FixedFeed, will return 0.01 for all feeds.

How do I know it’s working

You will see various errors initially on ftso-scaling and flare-system-client, since the data provider will not be registered as a voter for the current reward epoch. There is a time window for voter registration on every reward epoch, and if you leave things running you should eventually see RegisterVoter success logged by flare-system-client. It should then start submitting data successfully in the following reward epoch.

Here are log samples indicating successful operation (flare-system-client):

[03-04|06:58:20.000]	DEBUG	protocol/submitter.go:142	submitter submit1 running epoch 567838
[03-04|06:58:20.000]	DEBUG	protocol/submitter.go:143	  epoch is [2024-03-04 06:57:00 +0000 UTC, 2024-03-04 06:58:30 +0000 UTC], now is 2024-03-04 06:58:20.000923016 +0000 UTC m=+234317.457953448
[03-04|06:58:20.001]	DEBUG	protocol/protocol_utils.go:55	Calling protocol client API: http://ftso-scaling-00:3100/submit1/567838/0x5579C824e5550ae24ceFe41B129472c3EC70be5c
[03-04|06:58:20.081]	DEBUG	chain/tx_utils.go:120	Sending signed tx: 0xd4e677859190afca0c5f287e735b959ced12d8f5107bcb14cc31add09cbc92ec
[03-04|06:58:20.212]	DEBUG	chain/tx_utils.go:128	Waiting for tx to be mined...
[03-04|06:58:22.244]	DEBUG	chain/tx_utils.go:134	Tx mined, getting receipt 0xd4e677859190afca0c5f287e735b959ced12d8f5107bcb14cc31add09cbc92ec
[03-04|06:58:22.271]	DEBUG	chain/tx_utils.go:139	Receipt status: 1
[03-04|06:58:22.272]	INFO	protocol/submitter.go:78	submitter submit1 submitted tx
[03-04|06:58:35.001]	DEBUG	protocol/submitter.go:142	submitter submit2 running epoch 567839
[03-04|06:58:35.001]	DEBUG	protocol/submitter.go:143	  epoch is [2024-03-04 06:58:30 +0000 UTC, 2024-03-04 07:00:00 +0000 UTC], now is 2024-03-04 06:58:35.001392224 +0000 UTC m=+234332.458422666
[03-04|06:58:35.001]	DEBUG	protocol/protocol_utils.go:55	Calling protocol client API: http://ftso-scaling-00:3100/submit2/567838/0x5579C824e5550ae24ceFe41B129472c3EC70be5c
[03-04|06:58:35.074]	DEBUG	chain/tx_utils.go:120	Sending signed tx: 0x56fe0f0d60b876122ea13d2ae902c4ad777f26d3e2d44c19742d7fb0a1ae25af
[03-04|06:58:35.197]	DEBUG	chain/tx_utils.go:128	Waiting for tx to be mined...
[03-04|06:58:37.241]	DEBUG	chain/tx_utils.go:134	Tx mined, getting receipt 0x56fe0f0d60b876122ea13d2ae902c4ad777f26d3e2d44c19742d7fb0a1ae25af
[03-04|06:58:37.274]	DEBUG	chain/tx_utils.go:139	Receipt status: 1
[03-04|06:58:37.274]	INFO	protocol/submitter.go:78	submitter submit2 submitted tx
[03-04|06:59:20.000]	DEBUG	protocol/submitter.go:208	signatureSubmitter submitSignatures running epoch 567839
[03-04|06:59:20.003]	DEBUG	protocol/submitter.go:209	  epoch is [2024-03-04 06:58:30 +0000 UTC, 2024-03-04 07:00:00 +0000 UTC], now is 2024-03-04 06:59:20.003337094 +0000 UTC m=+234377.460367537
[03-04|06:59:20.003]	DEBUG	protocol/protocol_utils.go:55	Calling protocol client API: http://ftso-scaling-00:3100/submitSignatures/567838/0x6Bc692221B1feff64218eF6Fb3D1D2cE077a64D3
[03-04|06:59:20.502]	DEBUG	chain/tx_utils.go:120	Sending signed tx: 0xb371cae856bf1f37f52f9db556a346d0de35e2b02b73f87ec2dad63f044d7e8a
[03-04|06:59:20.532]	DEBUG	chain/tx_utils.go:128	Waiting for tx to be mined...
[03-04|06:59:21.564]	DEBUG	chain/tx_utils.go:134	Tx mined, getting receipt 0xb371cae856bf1f37f52f9db556a346d0de35e2b02b73f87ec2dad63f044d7e8a
[03-04|06:59:21.604]	DEBUG	chain/tx_utils.go:139	Receipt status: 1

Here are log samples indicating successful operation (fast-updates):

Note: depending on your weight it may take some time until you are selected for the fast-updates

ftso-v2-deployment-fast-updates  | [04-25|09:00:32.456] INFO    provider/feed_provider.go:65    deltas: +++++++++++++++++++++-++++0+
ftso-v2-deployment-fast-updates  | [04-25|09:00:32.456] INFO    client/client_requests.go:205   submitting update for block 14266248 replicate 0: +++++++++++++++++++++-++++0+
ftso-v2-deployment-fast-updates  | [04-25|09:00:33.496] INFO    client/client_requests.go:239   successful update for block 14266248 replicate 0 in block 14266250

So from the Readme it covers the following


Overview


System Components

Flare System Client.

Responsible for all interactions with the FTSOv2 smart contracts, including data collection and submission, voter registration, and additional system tasks.

Data Provider.

Provides commit, reveal, and median result data to System Client for submission.

Feed Value Provider.

Provides current values (prices) for a given set of feeds.

Indexer.

Monitors the blockchain and records all FTSOv2 related transactions and events.

Fast Updates Client.

Responsible for interaction with Fast Updates contracts, including data collection and submission and other system tasks.


Flare System Client

Responsible for all interactions with the FTSOv2 smart contracts, including data collection and submission, voter registration, and additional system tasks.


Data Provider

Provides commit, reveal, and median result data to System Client for submission.


Feed Value Provider.

Provides current values (prices) for a given set of feeds.


Entity Registration

Entity registration

You will need 5 addresses to be able to participate in the protocol:

  • Identity address: entity’s main address should be stored in a cold storage as it will only ever be needed in the initial setup and any future rotation of other keys
  • Submit address: used for submit1, submit2 and submit3 transactions can be stored in a hot storage as its private key will be added to .env file during deployment
  • Submit signatures: used for submitSignatures transactions can be stored in a hot storage as its private key will be added to .env file during deployment
  • Signing policy: used for reward epoch registration and relay transactions as well as payload signing in some parts of the system can be stored in a hot storage as its private key will be added to .env file during deployment
  • Delegation address: used for wrapping your tokens and getting delegation should be stored in a cold storage as it will not be needed in day to day protocol participation
Human: Can you create a python script to generate a BIP-44 wallet generating the 24 word pass phrase/words, and showing the resulting other addresses in the wallet
AI (Claude):

Certainly, I can help you create a Python script to generate a BIP-44 wallet with a 24-word mnemonic phrase and display the resulting addresses. This script will use the `bip_utils` library, which provides the necessary functionality for BIP-39 mnemonic generation and BIP-44 wallet derivation.



```python
import bip_utils
from bip_utils import Bip39SeedGenerator, Bip44, Bip44Coins, Bip44Changes
from typing import List

def generate_mnemonic() -> str:
    return bip_utils.Bip39MnemonicGenerator().FromWordsNumber(bip_utils.Bip39WordsNum.WORDS_NUM_24)

def derive_addresses(mnemonic: str, num_addresses: int = 5) -> List[str]:
    # Generate seed from mnemonic
    seed_bytes = Bip39SeedGenerator(mnemonic).Generate()

    # Create BIP44 object
    bip44_mst_ctx = Bip44.FromSeed(seed_bytes, Bip44Coins.BITCOIN)

    addresses = []
    for i in range(num_addresses):
        # Derive account 0, external chain
        bip44_acc_ctx = bip44_mst_ctx.Purpose().Coin().Account(0)
        bip44_chg_ctx = bip44_acc_ctx.Change(Bip44Changes.CHAIN_EXT)
        bip44_addr_ctx = bip44_chg_ctx.AddressIndex(i)

        addresses.append(bip44_addr_ctx.PublicKey().ToAddress())

    return addresses

def main():
    # Generate mnemonic
    mnemonic = generate_mnemonic()
    print("Generated 24-word mnemonic:")
    print(mnemonic)
    print()

    # Derive addresses
    addresses = derive_addresses(mnemonic)
    print("Derived Bitcoin addresses:")
    for i, address in enumerate(addresses):
        print(f"Address {i}: {address}")

if __name__ == "__main__":
    main()

```

This script does the following:

1. Generates a 24-word mnemonic phrase using BIP-39.
2. Uses the mnemonic to create a BIP-44 wallet for Bitcoin.
3. Derives the first 5 addresses from the wallet.

To run this script, you'll need to install the `bip_utils` library. You can do this by running:

```
pip install bip-utils
```

When you run the script, it will output the generated 24-word mnemonic and the first 5 derived Bitcoin addresses.

Please note that this script generates a real mnemonic and derives real addresses. If you plan to use this for actual cryptocurrency storage, make sure to keep the mnemonic phrase secure and private.

Would you like me to explain any part of the code in more detail?

Model version: 3.5
Conversation date: 2024-09-19

That’s some text with a footnote.1


  1. And that’s the footnote. ↩︎