WEP 11: Script execution with continuation

Abstract

We propose to remove the limitation on the complexity of a callable function of a dApp script. For this purpose, we suggest adding a new transaction type: Continuation. If the complexity exceeds 4000, the execution of the callable function is split into several stages. The first stage of calculations is performed within the Invoke Script transaction; the further stages are performed within Continuation transactions created automatically by block generators. The sender of the Invoke Script transaction should specify a transaction fee sufficient for the estimated number of stages.

Motivation and purposes

At the moment, a single callable function can perform calculations with a complexity of up to 4000. To implement a more complex logic, a developer is forced to decompose it into several transactions manually.

The idea is to split a large execution into multiple transactions and blocks automatically by the blockchain’s execution environment. The computation cost in each transaction remains predictable, and heavy calculations do not lock the blockchain. A more detailed motivation is given in the Unlimited dApp script complexity via Continuations article.

Specification

Script execution

A block generator that adds an Invoke Script transaction to a microblock performs calculations with a total complexity up to 4000 and saves intermediate results in the internal database. Further, the block generator, the same or another if a new key block is already created, detects an uncompleted calculation sequence, creates a Continuation transaction, and adds it to the microblock, performing the next stage of calculations. The process continues until the script is completely executed or fails.

image

Suspension of other transactions involving dApp

Until the calculation sequence is completed, transactions that can reduce the dApp balance can be added to the UTX pool but cannot be added to the block:

  • Transactions that are sent on behalf of the dApp.
  • Transactions that invoke the dApp.
  • Exchange transactions in which the dApp is one of the order senders.
  • Transactions with a fee in a sponsored asset issued by dApp.

Meanwhile, any transfers in favor of dApp, leases to dApp, and cancellations of leases to dApp are allowed.

Blockchain data usage

During all the stages of calculations, the dApp script operates the same blockchain data.

1) Entries of dApp’s own data storage

Due to the suspension of other transactions involving the dApp (see above), the dApp’s data storage entries do not change from the start of the script execution until the end. We add new built-in functions to Ride that allow the dApp to read data from its own data storage:

  • getBinary(key: String): ByteVector|Unit
  • getBinaryValue(key: String): ByteVector
  • getBoolean(key: String): Boolean|Unit
  • getBooleanValue(key: String): Boolean
  • getInteger(key: String): Int|Unit
  • getIntegerValue(key: String): Int
  • getString(key: String): String|Unit
  • getStringValue(key: String): String

The dApp script can call these functions at any stage, that is, in any transaction of the calculation sequence.

2) External data

All the blockchain data, except for the dApp’s data storage entries, are considered external. External blockchain data include:

  • Entries of data storages of other accounts.
  • Account balances, including the dApp account itself.
  • Current blockchain height.
  • Asset parameters, block headers, transactions, etc.

All external blockchain data used by the dApp script must be obtained at the first stage of calculations. The complexity of the script’s part from the beginning to the last function that reads external data, inclusive, should not exceed 4000.

If the script contains branches, it is not known in advance which branch will be executed. Therefore, the total complexity for all branches is taken into account. Consider the following scheme:

image

If the operations op2, op-a2 and op-b3 read the external blockchain data, then the total complexity of the operations op1 + op2 + op-a1 + op-a2 + op-b1 + op-b2 + op-b3 must not exceed 4000, otherwise the dApp script cannot be assigned to an account (Set Script transaction would be rejected).

Continuation transaction

Continuation is a new transaction type designed to execute a callable function of a dApp script in several stages. A block generator creates the Continuation transaction if there is an uncompleted calculation sequence. A user cannot send a Continuation transaction, so it does not have the senderPublicKey and proofs fields.

New version of Invoke Script transaction; extraFeePerStep

Invoke Script transaction version 3 contains the extraFeePerStep field that indicates the extra fee for each stage of calculations. extraFeePerStep can be 0. The sender can specify extraFeePerStep > 0 in order to raise the processing priority of transactions of the calculation sequence.

Fee

The sender should specify a fee taking into account the maximum possible number of calculation stages and script actions.

The minimum fee in WAVES for an Invoke Script transaction is calculated as follows:

Fee = (0.005 + E) × ⌈С / 4000⌉ + S + 0.004 × P + 0.004 × A + 1 × I,

Where:

E is the extra fee specified in the extraFeePerStep field,

С is the complexity of the callable function. С/4000 rounded up to the nearest integer is the number of stages in the calculation sequence.

S = 0.004 if the transaction sender is a dApp or a smart account, otherwise 0.

P is the number of payments in smart assets attached to the transaction.

A is the number of script actions (transfers, reissues, burnings) with smart assets.

I is the number of issued assets that are not NFT.

The entire fee amount specified in the transaction is charged to the sender when the Invoke Script transaction is added to a microblock. If the fee is indicated in the sponsored asset, the sponsor will be charged the fee equivalent in WAVES.

The fee is distributed as follows:

  1. For the Invoke Script transaction: 0.005 + E + S.
  2. For each Continuation transaction except the last one: 0.005 + E.
  3. For the last transaction of the continuation: 0.005 + E + 0.004 × P + 0.004 × A + 1 × I.

(By the Waves-NG protocol, the generator of the block to which the transaction is added receives 40% of the transaction fee, and the next block generator receives 60% of the fee.)

After the script is completely executed or fails, the fee’s unused portion (for stages and asset scripts whose execution was not started; see example below) is returned to the sender. If the fee is indicated in the sponsored asset, the WAVES equivalent of this portion of the fee is returned to the sponsor.

Note: the threshold for saving failed transactions is applicable only to Invoke Script transactions. If the script fails at one of the subsequent stages, the Continuation transaction is saved on the blockchain, and a fee is charged for it.

Backward compatibility

Script execution with continuation is only available if the dApp script uses Standard library version 5 and is called by the Invoke Script transaction version 3. In the earlier version of Standard library or Invoke Script transaction, the limit of 4000 complexity is applied.

Examples and implementation

Distribution of complexity between stages

Consider the complexity of the callable function is 10,000.

1st stage: the block generator that processes the Invoke Script transaction stops script execution before the total complexity exceeds 4000. For example, at complexity 3900, if the next operation complexity is 120.

2nd stage: the block generator that processes the Continuation transaction stops script execution before the total complexity exceeds 8000. For example, at complexity 7950, if the next operation complexity is 80. The complexity of the stage is 7950 – 3900 = 4050.

3rd stage: the block generator that processes the next Continuation transaction executes the remaining operations with a complexity of 10,000 – 7950 = 2050.

Distribution of fee

Consider the previous example, supposing that:

  • The transaction sender is a smart account: S = 0.004.
  • The Invoke Script Transaction contains 1 payment in a smart asset: P = 1.
  • The callable function performs 2 smart asset transfers and 1 token issue: A = 2 and I = 1.
  • The transaction sender specifies extraFeePerStep: E = 0.002

The minimum fee for the Invoke Script transaction is (0.005 + 0.002) × 3 + 0.004 + 0.004 × 1+ 0.004 × 2 + 1 = 1.037 WAVES. If all transactions are successful, the fee will be distributed as follows:

  • for the Invoke Script transaction: 0.005 + 0.002 + 0.004,
  • for the first Continuation transaction: 0.005 + 0.002,
  • for the last Continuation transaction: 0.005 + 0.002 + 0.004 × 1 + 0.004 × 2 + 1.

Note: if the sender specifies the transaction fee more than the minimum of 1.037 WAVES, the remainder will be returned to them even if all stages are successfully completed.

If the Invoke Script transaction fails after the calculations’ complexity exceeds the threshold for saving failed transactions, the sender pays 0.005 + 0.002 + 0.004 = 0.011 WAVES. The unused fee of 1.026 WAVES is returned to the sender.

If the first Continuation transaction fails, then the sender pays (0.005 + 0.002) × 2 + 0.004 = 0.018 WAVES. The unused fee of 1.019 WAVES is returned.

If the last Continuation transaction fails:

  • If the callable function execution fails, then the sender pays (0.005 + 0.002) × 3 + 0.004 = 0.025 WAVES. The unused fee of 1.012 WAVES is returned.
  • If the callable function result is successfully calculated, but one of the asset scripts denied the transaction, the sender also pays 0.004 for each asset script actually executed. For example, if the first asset script returned true, the second asset script returned false or failed, and the third asset script has not started execution, then the sender pays (0.005 + 0.002) × 3 + 0.004 + 0.004 × 2 = 0.033 WAVES. The unused fee of 1.004 WAVES is returned.
2 Likes

Questions:

  1. Will this full fee calculation be provided at all stages of execution via stateChanges?
  2. Do we get our transaction via txId at final or first step? How can we track execution process of our transaction?
  3. Which data is saved as intermediate to the blockchain, is it enough to really start from a mid? For example the script can read many data from other accounts at first step, because this data is subject to change it must be saved as intermediate, but it can be even bigger than a block.
  4. Is there a mechanism of emergency unblocking an account. If something went wrong an account can be frozen in endless continuation process.
  5. Do these continuations have separate limits in a block or they will share limits with regular invocation transactions?

Global idea is interesting, but how I will know how much steps I will need as a user of a service when I click on a button to execute a dApp?

Hi @deemru :slight_smile:

  1. could you rephrase the question?
  2. i think it would be great to enrich tx with additional txids of continuations, execution state, etc.
  3. since ride is an expression-based, the algorithm simply evaluates all nodes of expression tree that depend on external state during the first step, so it’s tree-traversal + some knowledge of which functions are impure.
    so during the first step, script
(3 + getIntger(someAcc, "x") + getInteger("x")) 

would be reduced to

(3 + 42 + getInteger("x")))
  1. it’s the limit of fees attached to execution, so eventually the computation will either result with success of fail, or run out of “gas”.

  2. they share the same limits for block. one computation can flow through multiple blocks. however I don’t expect this happening too often, the miners are incentivised to include as much txs into the block as possible.

you can’t. imagine you need to traverse a queue of payouts: you can’t know the amount of steps upfront, so you either pay max possible or what is actually being computed. one way around is to require as max, but return what hasn’t been used.

in the end, computation cost depends not just on the computation itself, but on the data. this has to be respected.

  1. stateChanges should provide fees diff information at every step, right?
  2. But how this first step state would be saved in blockchain? The integer 42 seems must be saved somehow to continue execution. Otherwise you have to rollback to the initial state and execute from beginning to continue.
  3. I mean something like stopExecutionTransaction which will force the original transaction to fail.
  1. hmm, maybe it’s good idea, not exactly obvious how it can be useful though
  2. in between the steps, the updated expression is saved to blockchain state(not the blockchain)
  3. we don’t have plans on this. usually there won’t be any time to do that: miners will fill the microblock with continuations instantly.
  1. It is needed for explorers to work correctly. Otherwise they would have to do all these fees nuances on their own.
  2. Got it, continuations work on their internal hot cache.
  3. Ok.