Using User-hosted Secrets in Requests

This tutorial shows you how to send a request to a Decentralized Oracle Network to call the Coinmarketcap API. After OCR completes offchain computation and aggregation, it returns the BTC/USD asset price to your smart contract. Because the API requires you to provide an API key, this guide will also show you how to encrypt, sign your API key, and share the encrypted secret offchain with a Decentralized Oracle Network (DON).

The encrypted secrets are never stored onchain. This tutorial uses the threshold decryption feature. This tutorial shows you how to share encrypted secrets offchain with a Decentralized Oracle Network (DON) using a storage platform such as AWS S3, Google Drive, IPFS, or any other service where the DON can fetch secrets via HTTP. Read the Secrets Management page to learn more.

Read the Using User-hosted (gist) Secrets in Requests tutorial before you follow the steps in this example. This tutorial uses the same example but with a slightly different process:

  1. Instead of relying on storing the encrypted secrets on gist, you will host your encrypted secrets on AWS S3.
  2. Include the encrypted secrets in an offchain-secrets.json file.
  3. Host the secrets file offchain (AWS S3).
  4. Encrypt the S3 HTTPs URL .
  5. Include the encrypted URL in your Chainlink Functions request.

Prerequisites

Set up your environment

You must provide the private key from a testnet wallet to run the examples in this documentation. Install a Web3 wallet, configure Node.js, clone the smartcontractkit/smart-contract-examples repository, and configure a .env.enc file with the required environment variables.

Install and configure your Web3 wallet for Ethereum Sepolia:

  1. Install Deno so you can compile and simulate your Functions source code on your local machine.

  2. Install the MetaMask wallet or other Ethereum Web3 wallet.

  3. Set the network for your wallet to the Sepolia testnet. If you need to add Sepolia to your wallet, you can find the chain ID and the LINK token contract address on the LINK Token Contracts page.

  4. Request testnet LINK and ETH from faucets.chain.link/sepolia.

Install the required frameworks and dependencies:

  1. Install the latest release of Node.js 20. Optionally, you can use the nvm package to switch between Node.js versions with nvm use 20.

    Note: To ensure you are running the correct version in a terminal, type node -v.

    node -v

    $ node -v
v20.9.0

  2. In a terminal, clone the smart-contract examples repository and change directories. This example repository imports the Chainlink Functions Toolkit NPM package. You can import this package to your own projects to enable them to work with Chainlink Functions.

    git clone https://github.com/smartcontractkit/smart-contract-examples.git && \
cd ./smart-contract-examples/functions-examples/

  3. Run npm install to install the dependencies.

    npm install

  4. For higher security, the examples repository encrypts your environment variables at rest.

    1. Set an encryption password for your environment variables.

      npx env-enc set-pw

    2. Run npx env-enc set to configure a .env.enc file with the basic variables that you need to send your requests to the Sepolia network.

      • ETHEREUM_SEPOLIA_RPC_URL: Set a URL for the Sepolia testnet. You can sign up for a personal endpoint from Alchemy, Infura, or another node provider service.

      • PRIVATE_KEY: Find the private key for your testnet wallet. If you use MetaMask, follow the instructions to Export a Private Key. Note: Your private key is needed to sign any transactions you make such as making requests.

      npx env-enc set

Configure your onchain resources

After you configure your local environment, configure some onchain resources to process your requests, receive the responses, and pay for the work done by the DON.

Deploy a Functions consumer contract on Sepolia

  1. Open the FunctionsConsumerExample.sol contract in Remix.

    Open in Remix What is Remix?

  2. Compile the contract.

  3. Open MetaMask and select the Sepolia network.

  4. In Remix under the Deploy & Run Transactions tab, select Injected Provider - MetaMask in the Environment list. Remix will use the MetaMask wallet to communicate with Sepolia.

  5. Under the Deploy section, fill in the router address for your specific blockchain. You can find both of these addresses on the Supported Networks page. For Sepolia, the router address is 0xb83E47C2bC239B3bf370bc41e1459A34b41238D0.

  6. Click the Deploy button to deploy the contract. MetaMask prompts you to confirm the transaction. Check the transaction details to make sure you are deploying the contract to Sepolia.

  7. After you confirm the transaction, the contract address appears in the Deployed Contracts list. Copy the contract address.

Create a subscription

Follow the Managing Functions Subscriptions guide to accept the Chainlink Functions Terms of Service (ToS), create a subscription, fund it, then add your consumer contract address to it.

You can find the Chainlink Functions Subscription Manager at functions.chain.link.

Tutorial

This tutorial is configured to get the BTC/USD price with a request that requires API keys. For a detailed explanation of the code example, read the Examine the code section.

You can locate the scripts used in this tutorial in the examples/7-use-secrets-url directory.

  1. Get a free API key from CoinMarketCap.

  2. Run npx env-enc set to add an encrypted COINMARKETCAP_API_KEY to your .env.enc file.

    npx env-enc set

  3. Prepare the store for your encrypted secrets file.

    1. Create a AWS free tier account.
    2. Follow these steps to create a AWS S3 bucket. Choose a name for your bucket, set ACLs enabled, and turn off Block all public access.

Build Offchain Secrets

Before you make a request, prepare the secrets file and host it offchain:

  1. Encrypt the secrets and store them in the offchain-secrets.json file using the gen-offchain-secrets script of the 7-use-secrets-url folder.

    node examples/7-use-secrets-url/gen-offchain-secrets.js

    Example:

    $ node examples/7-use-secrets-url/gen-offchain-secrets.js
secp256k1 unavailable, reverting to browser version
Encrypted secrets object written to /functions-examples/offchain-secrets.json

  2. Follow these steps to upload the file offchain-secrets.json to your AWS S3 bucket.

  3. To make the file publically accessible without authentication:

    1. Find the file in the bucket list, and click on it to open the object overview.
    2. Click on the Permissions tab to display the Access control list (ACL).
    3. Click on Edit.
    4. Set Everyone (public access) Objects read, then confirm. This action makes the object readable by anyone on the internet.
    5. Note the object URL.
    6. To verify that the URL is publicly readable without authentication, open a new browser tab and copy/paste the object URL in the browser location bar. After you hit Enter , the browser will display the content of your encrypted secrets file.

  4. Note the URL. You will need it in the following section. For example: https://clfunctions.s3.eu-north-1.amazonaws.com/offchain-secrets.json.

Send a Request

To run the example:

  1. Open the file request.js, which is located in the 7-use-secrets-url folder.

  2. Replace the consumer contract address and the subscription ID with your own values.

    const consumerAddress = "0x8dFf78B7EE3128D00E90611FBeD20A71397064D9" // REPLACE this with your Functions consumer address
const subscriptionId = 3 // REPLACE this with your subscription ID

  3. Replace the secretsUrls with your AWS S3 URL:

    const secretsUrls = ["https://clfunctions.s3.eu-north-1.amazonaws.com/offchain-secrets.json"] // REPLACE WITH YOUR VALUES after running gen-offchain-secrets.js and uploading offchain-secrets.json to a public URL

  4. Make a request:

    node examples/7-use-secrets-url/request.js

    The script runs your function in a sandbox environment before making an onchain transaction:

    $ node examples/7-use-secrets-url/request.js
secp256k1 unavailable, reverting to browser version
Encrypted secrets object written to /Users/crystalgomes/smart-contract-examples/functions-examples/offchain-secrets.json
crystalgomes@MB-CY16VK6DPG functions-examples % node examples/7-use-secrets-url/request.js

secp256k1 unavailable, reverting to browser version
Start simulation...
Simulation result {
capturedTerminalOutput: 'Price: 68196.20 USD\n',
responseBytesHexstring: '0x0000000000000000000000000000000000000000000000000000000000680f24'
}
✅ Decoded response to uint256:  6819620n

Estimate request costs...
Fulfillment cost estimated to 1.018348822253235 LINK

Make request...

Encrypt the URLs..

✅ Functions request sent! Transaction hash 0xadc0db0ddea7b9836b86a9c9e008bc97d47e5f92b0dcec9694d3944d0065c789. Waiting for a response...
See your request in the explorer https://sepolia.etherscan.io/tx/0xadc0db0ddea7b9836b86a9c9e008bc97d47e5f92b0dcec9694d3944d0065c789

✅ Request 0xb308ca293859dab47d8848578291e687a0d9373274d1451a9c9667dc4bba5fca successfully fulfilled. Cost is 0.260029208488139025 LINK.Complete reponse:  {
requestId: '0xb308ca293859dab47d8848578291e687a0d9373274d1451a9c9667dc4bba5fca',
subscriptionId: 2303,
totalCostInJuels: 260029208488139025n,
responseBytesHexstring: '0x000000000000000000000000000000000000000000000000000000000068146a',
errorString: '',
returnDataBytesHexstring: '0x',
fulfillmentCode: 0
}

✅ Decoded response to uint256:  6820970n

    The output of the example gives you the following information:

    • Your request is first run on a sandbox environment to ensure it is correctly configured.

    • The fulfillment costs are estimated before making the request.

    • The AWS S3 URL is encrypted before sending it in the request.

    • Your request was successfully sent to Chainlink Functions. The transaction in this example is 0xadc0db0ddea7b9836b86a9c9e008bc97d47e5f92b0dcec9694d3944d0065c789 and the request ID is 0xb308ca293859dab47d8848578291e687a0d9373274d1451a9c9667dc4bba5fca.

    • The DON successfully fulfilled your request. The total cost was: 0.260029208488139025 LINK.

    • The consumer contract received a response in bytes with a value of 0x000000000000000000000000000000000000000000000000000000000068146a. Decoding it offchain to uint256 gives you a result: 6820970.

Examine the code

FunctionsConsumerExample.sol

// SPDX-License-Identifier: MIT
pragma solidity 0.8.19;

import {FunctionsClient} from "@chainlink/contracts/src/v0.8/functions/v1_0_0/FunctionsClient.sol";
import {ConfirmedOwner} from "@chainlink/contracts/src/v0.8/shared/access/ConfirmedOwner.sol";
import {FunctionsRequest} from "@chainlink/contracts/src/v0.8/functions/v1_0_0/libraries/FunctionsRequest.sol";

/**
 * THIS IS AN EXAMPLE CONTRACT THAT USES HARDCODED VALUES FOR CLARITY.
 * THIS IS AN EXAMPLE CONTRACT THAT USES UN-AUDITED CODE.
 * DO NOT USE THIS CODE IN PRODUCTION.
 */
contract FunctionsConsumerExample is FunctionsClient, ConfirmedOwner {
    using FunctionsRequest for FunctionsRequest.Request;

    bytes32 public s_lastRequestId;
    bytes public s_lastResponse;
    bytes public s_lastError;

    error UnexpectedRequestID(bytes32 requestId);

    event Response(bytes32 indexed requestId, bytes response, bytes err);

    constructor(
        address router
    ) FunctionsClient(router) ConfirmedOwner(msg.sender) {}

    /**
     * @notice Send a simple request
     * @param source JavaScript source code
     * @param encryptedSecretsUrls Encrypted URLs where to fetch user secrets
     * @param donHostedSecretsSlotID Don hosted secrets slotId
     * @param donHostedSecretsVersion Don hosted secrets version
     * @param args List of arguments accessible from within the source code
     * @param bytesArgs Array of bytes arguments, represented as hex strings
     * @param subscriptionId Billing ID
     */
    function sendRequest(
        string memory source,
        bytes memory encryptedSecretsUrls,
        uint8 donHostedSecretsSlotID,
        uint64 donHostedSecretsVersion,
        string[] memory args,
        bytes[] memory bytesArgs,
        uint64 subscriptionId,
        uint32 gasLimit,
        bytes32 donID
    ) external onlyOwner returns (bytes32 requestId) {
        FunctionsRequest.Request memory req;
        req.initializeRequestForInlineJavaScript(source);
        if (encryptedSecretsUrls.length > 0)
            req.addSecretsReference(encryptedSecretsUrls);
        else if (donHostedSecretsVersion > 0) {
            req.addDONHostedSecrets(
                donHostedSecretsSlotID,
                donHostedSecretsVersion
            );
        }
        if (args.length > 0) req.setArgs(args);
        if (bytesArgs.length > 0) req.setBytesArgs(bytesArgs);
        s_lastRequestId = _sendRequest(
            req.encodeCBOR(),
            subscriptionId,
            gasLimit,
            donID
        );
        return s_lastRequestId;
    }

    /**
     * @notice Send a pre-encoded CBOR request
     * @param request CBOR-encoded request data
     * @param subscriptionId Billing ID
     * @param gasLimit The maximum amount of gas the request can consume
     * @param donID ID of the job to be invoked
     * @return requestId The ID of the sent request
     */
    function sendRequestCBOR(
        bytes memory request,
        uint64 subscriptionId,
        uint32 gasLimit,
        bytes32 donID
    ) external onlyOwner returns (bytes32 requestId) {
        s_lastRequestId = _sendRequest(
            request,
            subscriptionId,
            gasLimit,
            donID
        );
        return s_lastRequestId;
    }

    /**
     * @notice Store latest result/error
     * @param requestId The request ID, returned by sendRequest()
     * @param response Aggregated response from the user code
     * @param err Aggregated error from the user code or from the execution pipeline
     * Either response or error parameter will be set, but never both
     */
    function fulfillRequest(
        bytes32 requestId,
        bytes memory response,
        bytes memory err
    ) internal override {
        if (s_lastRequestId != requestId) {
            revert UnexpectedRequestID(requestId);
        }
        s_lastResponse = response;
        s_lastError = err;
        emit Response(requestId, s_lastResponse, s_lastError);
    }
}
Open in Remix What is Remix?

  • To write a Chainlink Functions consumer contract, your contract must import FunctionsClient.sol and FunctionsRequest.sol. You can read the API references: FunctionsClient and FunctionsRequest.

    These contracts are available in an NPM package, so you can import them from within your project.

    import {FunctionsClient} from "@chainlink/contracts/src/v0.8/functions/v1_0_0/FunctionsClient.sol";
import {FunctionsRequest} from "@chainlink/contracts/src/v0.8/functions/v1_0_0/libraries/FunctionsRequest.sol";

  • Use the FunctionsRequest.sol library to get all the functions needed for building a Chainlink Functions request.

    using FunctionsRequest for FunctionsRequest.Request;

  • The latest request id, latest received response, and latest received error (if any) are defined as state variables:

    bytes32 public s_lastRequestId;
bytes public s_lastResponse;
bytes public s_lastError;

  • We define the Response event that your smart contract will emit during the callback

    event Response(bytes32 indexed requestId, bytes response, bytes err);

  • Pass the router address for your network when you deploy the contract:

    constructor(address router) FunctionsClient(router)

  • The three remaining functions are:

    • sendRequest for sending a request. It receives the JavaScript source code, encrypted secretsUrls (in case the encrypted secrets are hosted by the user), DON hosted secrets slot id and version (in case the encrypted secrets are hosted by the DON), list of arguments to pass to the source code, subscription id, and callback gas limit as parameters. Then:

      • It uses the FunctionsRequestlibrary to initialize the request and add any passed encrypted secrets reference or arguments. You can read the API Reference for Initializing a request, adding user hosted secrets, adding DON hosted secrets, adding arguments, and adding bytes arguments.

        FunctionsRequest.Request memory req;
req.initializeRequestForInlineJavaScript(source);
if (encryptedSecretsUrls.length > 0)
    req.addSecretsReference(encryptedSecretsUrls);
else if (donHostedSecretsVersion > 0) {
    req.addDONHostedSecrets(
        donHostedSecretsSlotID,
        donHostedSecretsVersion
    );
}
if (args.length > 0) req.setArgs(args);
if (bytesArgs.length > 0) req.setBytesArgs(bytesArgs);

      • It sends the request to the router by calling the FunctionsClient sendRequest function. You can read the API reference for sending a request. Finally, it stores the request id in s_lastRequestId then return it.

        s_lastRequestId = _sendRequest(
    req.encodeCBOR(),
    subscriptionId,
    gasLimit,
    jobId
);
return s_lastRequestId;

        Note: _sendRequest accepts requests encoded in bytes. Therefore, you must encode it using encodeCBOR.

    • sendRequestCBOR for sending a request already encoded in bytes. It receives the request object encoded in bytes, subscription id, and callback gas limit as parameters. Then, it sends the request to the router by calling the FunctionsClient sendRequest function. Note: This function is helpful if you want to encode a request offchain before sending it, saving gas when submitting the request.

  • fulfillRequest to be invoked during the callback. This function is defined in FunctionsClient as virtual (read fulfillRequest API reference). So, your smart contract must override the function to implement the callback. The implementation of the callback is straightforward: the contract stores the latest response and error in s_lastResponse and s_lastError before emitting the Response event.

    s_lastResponse = response;
s_lastError = err;
emit Response(requestId, s_lastResponse, s_lastError);

JavaScript example

source.js

The JavaScript code is similar to the Using Secrets in Requests tutorial.

gen-offchain-secrets.js

This explanation focuses on the gen-offchain-secrets.js script and shows how to use the Chainlink Functions NPM package in your own JavaScript/TypeScript project to encrypts your secrets. After encryption, the script saves the encrypted secrets on a local file, offchain-secrets.json. You can then upload the file to your storage of choice (AWS S3 in this example).

The script imports:

  • path and fs : Used to read the source file.
  • ethers: Ethers.js library, enables the script to interact with the blockchain.
  • @chainlink/functions-toolkit: Chainlink Functions NPM package. All its utilities are documented in the NPM README.
  • @chainlink/env-enc: A tool for loading and storing encrypted environment variables. Read the official documentation to learn more.

The primary function that the script executes is generateOffchainSecretsFile. This function can be broken into three main parts:

  1. Definition of necessary identifiers:

    • routerAddress: Chainlink Functions router address on Sepolia.
    • donId: Identifier of the DON that will fulfill your requests on Sepolia.
    • secrets: The secrets object.
    • Initialization of ethers signer and provider objects. The Chainlink NPM package uses the signer to sign the encrypted secrets with your private key.

  2. Encrypt the secrets:

    • Initialize a SecretsManager instance from the Chainlink Functions NPM package.
    • Call the encryptSecrets function from the created instance to encrypt the secrets.

  3. Use the fs library to store the encrypted secrets on a local file, offchain-secrets.json.

request.js

This explanation focuses on the request.js script and shows how to use the Chainlink Functions NPM package in your own JavaScript/TypeScript project to send requests to a DON. The code is self-explanatory and has comments to help you understand all the steps.

The script imports:

  • path and fs : Used to read the source file.
  • ethers: Ethers.js library, enables the script to interact with the blockchain.
  • @chainlink/functions-toolkit: Chainlink Functions NPM package. All its utilities are documented in the NPM README.
  • @chainlink/env-enc: A tool for loading and storing encrypted environment variables. Read the official documentation to learn more.
  • ../abi/functionsClient.json: The abi of the contract your script will interact with. Note: The script was tested with this FunctionsConsumerExample contract.

The script has two hardcoded values that you have to change using your own Functions consumer contract and subscription ID:

const consumerAddress = "0x8dFf78B7EE3128D00E90611FBeD20A71397064D9" // REPLACE this with your Functions consumer address
const subscriptionId = 3 // REPLACE this with your subscription ID

The primary function that the script executes is makeRequestSepolia. This function can be broken into six main parts:

  1. Definition of necessary identifiers:

    • routerAddress: Chainlink Functions router address on Sepolia.
    • donId: Identifier of the DON that will fulfill your requests on Sepolia.
    • explorerUrl: Block explorer URL of the Sepolia testnet.
    • source: The source code must be a string object. That's why we use fs.readFileSync to read source.js and then call toString() to get the content as a string object.
    • args: During the execution of your function, These arguments are passed to the source code. The args value is ["1", "USD"], which fetches the BTC/USD price.
    • secrets: The secrets object. Note: Because we are sharing the URL of the encrypted secrets with the DON, the secrets object is only used during simulation.
    • secretsUrls: The URL of the encrypted secrets object.
    • gasLimit: Maximum gas that Chainlink Functions can use when transmitting the response to your contract.
    • Initialization of ethers signer and provider objects. The signer is used to make transactions on the blockchain, and the provider reads data from the blockchain.

  2. Simulating your request in a local sandbox environment:

    • Use simulateScript from the Chainlink Functions NPM package.
    • Read the response of the simulation. If successful, use the Functions NPM package decodeResult function and ReturnType enum to decode the response to the expected returned type (ReturnType.uint256 in this example).

  3. Estimating the costs:

    • Initialize a SubscriptionManager from the Functions NPM package, then call the estimateFunctionsRequestCost function.
    • The response is returned in Juels (1 LINK = 10**18 Juels). Use the ethers.utils.formatEther utility function to convert the output to LINK.

  4. Encrypt the secrets, then create a gist containing the encrypted secrets object. This is done in two steps:

    • Initialize a SecretsManager instance from the Functions NPM package, then call the encryptSecrets function.
    • Call the encryptedSecretsUrls function of the SecretsManager instance. This function encrypts the secrets URL. Note: The encrypted URL will be sent to the DON when making a request.

  5. Making a Chainlink Functions request:

    • Initialize your functions consumer contract using the contract address, abi, and ethers signer.
    • Call the sendRequest function of your consumer contract.

  6. Waiting for the response:

    • Initialize a ResponseListener from the Functions NPM package and then call the listenForResponseFromTransaction function to wait for a response. By default, this function waits for five minutes.
    • Upon reception of the response, use the Functions NPM package decodeResult function and ReturnType enum to decode the response to the expected returned type (ReturnType.uint256 in this example).

Get the latest Chainlink content straight to your inbox.