Transcript

Intro

Hey, this is Sohrab, head of engineering for Lab Eleven.

Today, I want to go through the basics of Solana transactions and I will be using Better Call Sol as a visual aid.

You can find me on Twitter by clicking this little laptop icon at the bottom of bettercallsol.dev.

This is going to be at the beginner level so treat it as a starting point, not a comprehensive guide. I will leave some references in the video description if you like to explore things further.

Let’s get started.

Overview

To perform any action on-chain, you need to submit a transaction to a Solana cluster.
A cluster is basically any group of Solana validators that share the same genesis block.

There are 3 major clusters in the Solana ecosystem:

1. Mainnet or Mainnet-beta, to be more precise, is “the real” cluster, where money is real.

2. Then there is Testnet, primarily used by the validators to try out new versions and test things out

3. And finally, there is Devnet, which is for developers to test out their Solana programs and Dapps.

As a beginner, Devnet is where we will be spending most of our time.

When a validator is elected as leader, it looks at the pending transactions and picks a subset to execute.

But we don't submit transactions directly to the validators. Instead, we submit them to RPC servers, and those servers forward them to the validators.

These URLs that use to communicate with Solana, are actually these RPC endpoints.

Better Call Sol is already populated with the public RPC endpoints that Solana Labs maintains. These endpoints are good for basic testing but once you build a Dapp, you would want to sign up for a third-party RPC provider that gives you some reliability and performance guarantees. You can add those custom RPC endpoints in the settings.

By the way, RPC (or Remote Procedure Call) is a protocol for running commands on a remote server. In this case, the command is communicated over HTTPS in a JSON format. You can see examples of these if you got to the RPC JSON API specs in the Solana docs.

As you can see, transactions are encoded as a base58 or preferably as a base64 string and sent to the RPC endpoint.

A transaction, itself, is made up of one or more instructions. In Better Call Sol, each instruction is indicated with a purple box.

An instruction essentially tells a validator to invoke an on-chain program.

The instruction is made up of 3 parts:

1. The address of the program that is being invoked

2. The other Solana accounts that are read or written to

3. And finally, the instruction data or the parameters to be provided to the program

Both the program and the accounts are specified using their public key. Each account on Solana is associated with an Ed25519 keypair.

The public key of the keypair is used as the address and the private key is kept... well... private and is used for signing transactions.

When we supply accounts to an instruction, we need to indicate whether they will be used as read-only or if we are planning to write to them.

Declaring these upfront helps Solana to schedule the transactions. For example, if multiple transactions only read an account, then those transactions can be executed in parallel. But if an account is written by multiple transactions, then each transaction has to be processed sequentially, where it would lock the account, write to it and release the lock for the next transaction.

Accounts can also be signers. This is handy when we need to give permission to a transaction to update an account and we need to prove we have authority over that account. We prove this by signing the transaction with the private key for that account, which proves we own the keypair for that account.

When it comes to instruction data, under the hood, we are providing a byte array to the program. As long as both we, the client, and the on-chain program have agreed on what this byte array represents, we can use it to provide additional arguments to the instruction.

For example, if the program expects a UTF-8 encoded string to come through, then we just need to make sure that's what we are sending it.

Raw binary data is good but for more complex interactions, the programs tend to define a schema for how they expect the instruction data to be laid out.

Two of the most popular ways of defining these schemas or layouts include:

• Buffer layout, which you will find in various SPL programs, and

• Borsh, which is prevalent in Anchor programs

Btw SPL stands for Solana Program Library and is basically a set of programs, developed by Solana Labs, to streamline some standard operations like creating tokens, token swaps, etc. Think of SPL programs as the standard lib of Solana.

The agreed-upon layouts normally consist of one or more fields of predefined types. During the encoding of a transaction, the value of each field is encoded into bytes and laid out one after another to make up the instruction data.

Setup

Now Let's look at executing some transactions and see what happens.

To send a transaction, we need to pay a transaction fee. This fee is paid to the validator to reimburse them for executing the transaction.

We pay the fee in SOLs, the native currency of Solana, and we do so from our wallet. A wallet is just yet another Solana account that we have the keypair for.

We sign transactions with our wallet keypair to give the validator permission to deduct the transaction fee from our SOL balance in that account.

We can generate a wallet using the Solana CLI but in this case, since we're in the browser, we will be using a wallet browser extension.

Here you can see a few popular Solana wallets. I will be using the Phantom wallet for this example.

By the way, please always make sure to use a throw-away wallet when you are dealing with a Dapp that you have never used or may not trust. It’s always better to be safe.

Since we are going to be mostly working in Devnet, we need to also switch the wallet to Devnet.

And since it is going to be paying for transactions we need to fund the wallet with some SOLs.

Here is a little trick: copy the wallet public key into a random account field in Better Call SOL. As long as you are in Devnet or Testnet, the tool gives you the opportunity to airdrop some SOLs into that wallet.

It costs very little to send a transaction so 1 SOL should be plenty.

Please note that Solana has placed some restrictions on the number of SOLs you can airdrop so don't go overboard with it.

The airdrop itself is actually a Solana transaction as well. Which you can view in a Solana explorer.

Solana explorers are platforms that parse on-chain transactions and give you a human-readable view of what happened on-chain. Here you can see that 1 SOL was transferred to our wallet.

Once the transaction is confirmed by the chain, you should see that your wallet balance has been updated with 1 SOL.

Memo Transaction: Simulate

We're finally ready to run a transaction. Let's start with the most basic transaction that you can run on the chain: a memo

When you load an example into Better Call Sol, it will come with some annotations to help you understand the transaction better. But that's what I'm here for so let's collapse those for more screen real estate.

Memo program is an SPL program that essentially lets you attach a memo to a transaction, by logging whatever you provide to it as instruction data.

As you can see, the program address is populated for us. This instruction takes no accounts since it does not need to read or write to any Solana accounts. And the program expects the instruction data to be provided as a UTF-8 string.

Now we can send this transaction to the chain and see what happens. But at times, especially in mainnet, you want to first simulate the transaction.

Given the deterministic nature of Solana transactions, you can ask an RPC server to simulate what happens if we run the transaction. The simulation is usually pretty close to the real thing, down to the program logs that may be generated.

This does not cost any transaction fees so it is a risk-free way of ensuring the transaction will work before sending it through. It also helps reduce the traffic in the cluster as fewer buggy transactions will make it through.

Let's try simulating this transaction first.

So it looks like this transaction will likely be successful and here are the log entries that the program will produce if the transaction is run.

The memo program essentially just logs any instruction data that we send it so you can see that would happen here.

The simulation also identifies which slot was used for the simulation. A slot is the period of time that a given validator is the leader and going to produce a block. It only lasts a few seconds and then the next slot starts with a new leader.

You can also see how many CUs or Compute Units will be consumed by running this transaction. Each transaction is budgeted for 200,000 compute units. This can be increased by paying more transaction fees. But for our purposes, the default is enough.

One interesting point is that printing a single line of program logs has consumed about 3% of the compute budget. Logging is quite expensive on Solana so please keep that in mind when you start creating your own on-chain programs.

Memo Transaction: Send

I think we are ready to send this transaction for real. So let's do that.

Before the transaction is sent, your wallet extension prompts you to "approve" the transaction. This essentially signs the transaction with your wallet keypair, authorising the validator to deduct the transaction fees. You can see this described as "network fees" in this wallet.

Once you approve the transaction, it is sent to the chain. And then Better Call Sol starts to poll the chain to monitor whether the transaction is being confirmed or not.

A transaction goes through a few different states till it is finalised. First, it is submitted to the RPC provider, who in turn submits it to the cluster. Then a validator picks up the transaction and processes it. It broadcasts the block, that contains our transactions (amongst others) to all other validators so they can confirm it. Once a majority of the validators have confirmed the block, the transaction is considered confirmed. There is one final commitment where enough other blocks are built on top of our block, which makes it impossible for a fork to discard our transaction. In other words, the transaction is finalised at that point.

When we submit a transaction to the cluster, we generate a transaction signature. This signature is a unique identifier for the transaction, which we can use to look it up after the fact.

Transaction signatures are also Ed25519 signatures that encode the transaction payload including the instructions. That is kinda handy because if you, by accident, double-submit the same transaction, the cluster can easily dedup that for you.

Another way that we prevent double-spending is by providing a recent blockhash when we send the transaction. We query the chain for a recently finalised block and include its identifier or hash in the transaction header.

If for any reason, the transaction doesn’t get picked up by the validator till much later, the validator checks how old this blockhash is. If too many blocks have been produced since when we submitted the transaction, then the transaction is too old and the validator will simply discard it.

This means that we can confidently re-submit the same transaction after the blockhash expires, knowing that if the old transaction was stuck somewhere and gets picked up by a validator, it won’t be processed.

Transfer SOLs Transaction

This was a relatively simple transaction. Let’s try something slightly more complex.

This time, we send a transaction that again consists of a single instruction. It will transfer some SOLs from one account to another.

For this, we need to use the system program. The system program is a native program to Solana blockchain, whose address you can see in the program field.

A transfer instruction takes two accounts: 1) the sender of funds, and 2) the receiver of funds.

Btw these account labels are provided for readability in Better Call Sol. They are not sent as part of the transaction. Solana distinguishes between accounts, purely based on the order they have been defined.

Both accounts are marked as writable since the balance of both accounts will change as a result of this instruction. The amount to be transferred will be deducted from the first one and added to the balance of the second one.

The first account has to sign the transaction to indicate that it agrees for the funds to be removed from its balance.

On Solana, there is no consent when receiving funds and tokens so the second account does NOT have to sign this transaction.

We will use our connected wallet as the sender of funds. Better Call Sol already knows this so you can just ask it to auto-fill the public key of your wallet in the first account field.

Remember that you can airdrop more SOLs using the airdrop button if you need to. Here, I don’t need to.

The second account is an arbitrary account I have created for this example.

Moving down to the instruction data, we can see that a buffer layout structure is expected.

System Program has 13 or so different operations implemented within a single program. So we need to identify which of those operations we intend to invoke.

The third operation, or index 2 in the 0-indexed list, is the transfer operation that we are after. So this is set for the first field in the instruction data.

We also need to provide the amount of funds we like to transfer. The amount is described in lamports and stored in a 64-bit unsigned integer. A billion lamports are equal to 1 SOL. Here we have decided to transfer 100,000 lamports across.

Ok, we are now ready to send this transaction.

This time the wallet indicates that some fund transfer is about to take place, besides the usual network fees.

Once we sign the transaction by approving it in our wallet, it is sent to the cluster and we wait for the results.

System program is invoked a lot as part of transactions so they tend not to log much so as not to consume transaction budgets.

But you can switch to looking at the account balances to see what exactly happened.

Here both the transaction fee and 100,000 lamports were deducted from the first account and 100,000 lamports were added to the second account’s balance.

You should be able to see this by opening each account in the Solana explorer as well.

Create Account Transaction

Now let’s step things up by doing another type of transaction. This time, we like to create a Solana account on-chain.

Create Account is yet another operation in the System program so we have filled in the system program public key. Also, it is the first operation so you will note the index 0 in the first instruction data field.

When an account is created on-chain, we pay rent for that account. The rent is the fee we pay validators for keeping the account and its content around.

At the start of each epoch, the rent due is deducted from account balances. This rent is calculated based on the size of the account.

The convention is to deposit 2 years' worth of rent in an account. Once we do that, the account becomes rent-exempt. In other words, the validators no longer will charge rent on that account.

So in this instruction, we need to indicate the account that will be paying the rent for our new account. In this case, we will use our wallet to pay for that so let’s auto-fill our wallet’s public key for the first account.

As mentioned previously, each account in Solana is associated with a keypair. So in order to create a new account, we need to create a new keypair and then use the private key of that keypair to sign for this transaction.

Again, we can simply use the auto-fill to achieve this. The new public key of our keypair becomes the address for the new account we will create.

As expected both accounts need to sign for this transaction and they will be both written to.

Coming down to the instruction data, we can see that this operation takes 3 fields, besides the instruction indicator.

They are in order:

• The amount of lamports we will deposit in the new account.

  • Here I have chosen an amount that is larger than the rent-exempt amount required

• The amount of space, in bytes, that this account will have

• And finally, which account will be set as the owner of this account

Account ownership can be a confusing concept in Solana. The owner of an account is the only one that can modify that account, for example, change its balance or data content. Ownership is typically set to a program that will be responsible for actually updating the account.

In this case, setting the owner to the system program makes sense, since we will be using the system program operations such as transfer SOL to interact with this account anyway.

This is different from the authority on the account. Previously, we generated a keypair for this account, which is how we prove we have authority over the account by signing future transactions with it.

The authority gives us the ability to request changes to be made to the account. But System Program is the one actually making changes. And before it does so, it makes sure that we have authority over the account.

Anyway, let’s send this transaction to Devnet.

Once the transaction is confirmed, we can look at the account that we just created.

As expected, it has the balance we paid for from our wallet and it has 64 bytes in size. The owner is also set to the System program.

You also note that this is not an executable account. This is called a data account and it is used to normally hold data, in this case, 64 bytes worth of data.

Solana programs are stateless so these data accounts are how the state is persisted across transactions.

Outro

Ok, I think we leave things here. I have shown you some basic ways of composing Solana transactions using Better Call Sol. Now it is your turn to go to bettercallsol.dev and try out some transactions of your own.

I hope you found this video useful. Please let me know on YouTube or Twitter if you want to see more content like this and I’ll see if I can be bothered.

Thank you for watching.

References

• Solana Docs: Terminology

• Solana Docs: Solana Clusters

• Solana Docs: JSON RPC API

• Solana Cookbook: Core Concepts: Transactions

• Solana Cookbook: Core Concepts: Accounts

• Solana Program Library

• Solana Docs: Native Programs

• Solana Docs: Transaction Fees

• Solana Docs: Transaction Confirmation

• Solana Docs: Commitment Status

In these scenarios, we miss having different log levels, as we are used to with other application frameworks. In this micro-post, we will implement this feature for both Anchor and plain Solana applications.

We will try to keep things simple: introduce a macro that includes or omits logs based on a Cargo feature.

Anchor

The debug macro, for Anchor programs, is implemented as below:

#[macro_export]
macro_rules! debug {
    ($($rest:tt)*) => {
        #[cfg(feature="verbose")]
        anchor_lang::prelude::msg!($($rest)*)
    };
}

The macro checks for the verbose Cargo feature and includes the logging statement if it is enabled.

For this to work, you will need to declare a verbose feature in the program's Cargo.toml:

[features]
...
verbose = []
...

Now you can use this feature in your code:

debug!("some super detailed info");
debug!("the secret is {}", secret);

We can build binaries with or without debug logging using the following Anchor commands:

# with debug logging
anchor build -- --features verbose
# without debug logging
anchor build

Unfortunately, both anchor build and anchor test run Cargo builds with --release flag so we have to be explicit about enabling the feature.

Solana

The differences are minimal between Anchor and non-Anchor versions.

The debug macro is defined as below:

#[macro_export]
macro_rules! debug {
    ($($rest:tt)*) => {
        #[cfg(debug_assertions)]
        solana_program::msg!($($rest)*)
    };
}

Here we use the standard debug_assertions conditional compilation instead. This removes the need for defining custom features and can be controlled by the standard Cargo profiles.

And cargo commands are used for the build:

# with debug logging
cargo build
# without debug logging
cargo build --release

This article has been written with Anchor v0.24.x in mind. We will endeavour to keep this information updated as things evolve.

Anchor framework is quickly becoming the de facto approach to building Solana on-chain programs. Its abstractions offer security and ergonomics that a typical Solana developer may miss while chewing glass.

Programs are called either by clients off-chain or by other programs on-chain, i.e. Cross-Program Invocations (CPIs). This unlocks composability, the key to building sophisticated solutions from small battle-tested building blocks.

Anchor simplifies how programs receive invocations or invoke others. One can simply read through the book and learn how to use these features; however, it is less obvious how it all hangs together.

Abstractions are great for productivity but to reach an expert level in any framework, it is essential to understand what happens under the hood. This post will serve as the proverbial popping up the hood and having a poke around.

Invoking a Program

Each instruction, in a transaction submitted to Solana Runtime, is made up of a program_id, an accounts array and an instruction_data byte array. The Runtime invokes the given program, passing in the accounts and instruction data as input.

This makes sense if we look at what a standard Solana program's entry point looks like:

entrypoint!(process_instruction);

fn process_instruction(
    program_id: &Pubkey,
    accounts: &[AccountInfo],
    instruction_data: &[u8],
) -> ProgramResult {
    // TODO check program_id
    // TODO determine which method to call
    // TODO deserialise instruction_data
    // TODO deserialise accounts
    // TODO validate accounts
    // TODO map accounts into an ergonomic struct
    // TODO perform business function
    // TODO map errors if needed
}

Things get more complicated once we have to implement the body of this function. I have suggested some actions that a typical program may take with the help of some TODOs. A lot to needs to be built and it has to be done for each and every program you write.

This is where Anchor comes in, picking up all that slack. Check out this Anchor program, which we will use as an example for the rest of this post:

declare_id!("Fg6PaFpoGXkYsidMpWTK6W2BeZ7FEfcYkg476zPFsLnS");

#[program]
pub mod hello_world {
    use super::*;

    pub fn do_something<'info>(
        ctx: Context<DoSomething<'info>>, 
        x: u8
    ) -> Result<()> {
        ...
    }

    pub fn do_another_thing<'info>(
        ctx: Context<DoAnotherThing<'info>>,
    ) -> Result<()> {
        ...
    }
}

#[derive(Accounts)]
pub struct DoSomething<'info> {
    account: Account<'info, TokenAccount>,
    token_program: Program<'info, Token>,
    rent: Sysvar<'info, Rent>,
}

#[derive(Accounts)]
pub struct DoAnotherThing<'info> {
    ...
}

This program defines two different entry points to the program. There are corresponding structs for all the input accounts and do_something() even receives an argument x.

It may not look like it but this Anchor program is implementing all those TODOs I mentioned earlier.

The Anchor program specifies the program's ID to help to ensure that the program binary is only run at the given address. This is checked as soon as the instruction arrives at the program. I would have preferred if the ID was specified in the config, rather than the code, but here we are.

Another interesting difference with the Solana program is how Anchor allows you to define multiple methods within the program. Anchor achieves this by encoding each method's signature into a unique SHA256 hash and storing it at the first 8 bytes of instruction_data.

So when a new instruction arrives at the entry point, the program compares the hash to each method signature and dispatches the instruction to the relevant method.

ℹ️ Anchor also supports a fallback method, for cases where the incoming instruction does not match any of the defined methods. Simply create a new method without the Context argument and it will be designated as the fallback one.

Once the instruction is routed to the correct method, the provided accounts and instruction_data arrays are deserialised and passed as the Context and the arguments accordingly.

Again, the first 8 bytes of each provided account are used to distinguish it from others and map correctly into the struct.

Helpfully, Anchor also performs further checks based on the provided account types and constraints to protect against incorrect and/or malicious input. Some of these include whether an account is a token account or is associated with an expected mint.

Cross-Program Invocation

Now that we understand how a program receives instructions, let's see what it takes to invoke from another program.

If we were to invoke the above Anchor program in plain Solana, we would do something similar to below:

// prepare instruction data
struct DoSomething {
    x: u8,
}
let instruction_data: Vec<u8> = sighash_of_the_method();
instruction_data.append(BorshSerialize.try_to_vec(DoSomething { x: 2 }));

let accounts = vec![
    AccountMeta::new(*token_account, false),
    AccountMeta::new_readonly(*signer, true),
    AccountMeta::new_readonly(*rent, false),
];

// prepare account infos
let account_infos = vec![
    token_account.info.clone(),
    signer.info.clone(),
    rent.info.clone(),
];

// prepare the instruction
let instruction = solana_program::instruction::Instruction {
    program_id: *program_id,
    accounts,
    instruction_data,
};

// invoke the program, providing the instruction
solana_program::program::invoke_signed(
   &instruction, 
   &account_infos, 
   signer_seeds
)?

Predictably, we create an instruction, made up of the program ID, an array of accounts and an instruction data byte array. This instruction is then signed and submitted to the Runtime, along with the list of accounts.

You may note that we also need to pack the instructions data as a byte array, remembering to include the corresponding method hash at the beginning.

Over in Anchor land, things seem much simpler:

let accounts = hello_world::cpi::accounts::DoSomething {
    account: ..., 
    token_program: ..., 
    rent: ...,
}

let context = CpiContext::new(program_id, accounts);
hello_world::cpi::do_something(context, x);

This is because Anchor macros generate a lot of the boiler-plate for us.

Among those are:

• hello_world::cpi::accounts::DoSomething that normalises our Account structs into AccountInfo types so they can be plugged into the CPI call
• hello_world::cpi:do_something(), which essentially performs the same steps as seen in our plain Solana CPI call
• CpiContext that knows how to convert our accounts to AccountMeta or AccountInfo, as required

That concludes this quick deep-dive into Solana and Anchor program invocation. It goes without saying that this is a small part of what Anchor can do for us. To achieve that next level of proficiency and self-sufficiency, it is worthwhile to dig into each of these features.

Watch this space for more posts about that!

Local Development

The book already calls out the required software for building Anchor programs.

I highly recommend Linux, macOS or Windows Subsystem for Linux v2 (WSL2) as the development environments.

For IDE, Visual Studio Code provides the optimal development experience at the best price (free!). Here are my suggestions for VS Code extensions to install:

• rust-analyzer: full support for Rust programming. I would avoid the RLS extension as it has been deprecated in favour of the rust-analyzer and has not received an update in a few years.
• CodeLLDB: for debugging Rust programs. For those brave souls with a pure (non-WSL) Windows development environment, you may want to look at the C/C++ extension instead.

TypeScript support comes out of the box. Some other nice-to-have extensions include Better TOML, Prettier and Remote - WSL (mandatory if working on WSL2).

I should note if you are more comfortable with the JetBrains extended universe, Rust support can be installed into your IDE as well. However, to enable Rust debugging, you will need CLion or other paid variants with Native Debugging Support.

If you are after an IntelliJ IDE that has both Rust and TypeScript support, you are limited to the paid IntelliJ IDEA Ultimate.

VS Code Workspace for Mono-Repos

Anchor projects are made up of a Cargo workspace, as specified in the root Cargo.toml file, and one or more programs in their own Cargo crates, as specified in programs/*/Cargo.toml.

rust-analyzer currently only supports one Cargo workspace per VS Code workspace by default. You may face errors like the below if you open a mono-repo with multiple Anchor projects in it.

[ERROR rust_analyzer] failed to find any projects in [AbsPathBuf("...")]
[ERROR rust_analyzer::lsp_utils] rust-analyzer failed to discover workspace

The recommended way to get around this is to turn off the project auto-discovery of the plugin and specify each Anchor project manually. This can be done in .vscode/settings.json of your VS Code workspace:

{
  "rust-analyzer.linkedProjects": [
    "an_anchor_project/Cargo.toml", 
    "another_anchor_project/Cargo.toml"
  ]
}

Learning Rust

Rust programming can be one of the bigger hurdles when moving into the Solana ecosystem, especially if you do not have a system programming background.

Everyone learns differently so I cannot possibly recommend one efficient way to get upskilled. But here are some tips:

• For those that learn by reading, the Rust book is indispensable.
• For those that learn by doing, Rust by Example can be the way to go.
• For those that learn by watching, you are spoiled for choice: there are plenty of YouTube videos and courses on educational sites, such as Udemy.

I personally find the first two options more effective. As someone who knows a few programming languages already, I can skip and skim through chapters and only learn what I do not already know.

Regardless of how you learn, you do not need to learn every aspect of Rust to be a Solana developer (though I highly recommend learning as much as you can - Rust is a great language!). Here are some topics to focus on:

• Basics of the language: variables, data types, functions, control flow, etc.
• Basics of cargo: crates, modules, command line, etc.
• Ownership, references and borrowing
• Structs and implementations
• Common collections, e.g. vectors
• Enums and pattern matching
• Error handling
• Generic types, traits and lifetimes
• Smart pointers, especially Box<>
• Basics of Macros

Here are some topics that are either not applicable or you will not need at the beginner or even intermediate level:

• I/O: filesystem or network
• Fearless concurrency: threads, process, futures, sync/send, etc.
• Developing macros
• OS-specific libraries
• Unsafe Rust

Learning Anchor

Anchor documentation is evolving rapidly and is actively being updated. Please keep that in mind when reading this section.

Anchor framework is a good on-ramp onto Solana development since it provides a few safety nets and ergonomics that you may miss otherwise.

A quick search will bring up a raft of tutorials written by the community. Most of them cover the basics well. The only thing to watch out for is the date when the tutorial was published. Some of the older ones may no longer be valid.

The Anchor book itself is a good starting place that covers topics in a mix of theory and practice. It is missing some topics and could use a few runs of edit but otherwise, it is great.

I personally learn more from fully working examples so Anchor GitHub repo is the place to spend time in:

• The examples folder contains the accompanied code for the tutorials of the now deprecated original documentation.
• More substantial examples can be found in the tests folder, covering a wide range of topics.

The tests are set up differently than normal Anchor projects. If you want to run them locally, consult the repo's GitHub Actions for hints.

So far, we haven't spoken much about learning Solana itself. There is a lot to learn there, that will require its own post. But in the meanwhile, I would at the minimum recommend getting your head around the basics via either their official docs or the cookbook.

Community

As is the case with any other project in this space, the community is motivated for the projects to succeed so there is plenty of support available.

Anchor Discord is quite active and both users and contributors are more than happy to help each other out. Following Anchor on Twitter can also be worthwhile to keep on top of the latest updates.

Solana has recently achieved sufficient commitment for a StackOverflow site so hopefully soon, there will also be a specialised Q&A site to make community support even smoother!

Table stakes

Blockchain has the power to be a disruptive technology, but it cannot possibly reach its potential if it cannot deliver experiences that are virtually indistinguishable from what we are used to today.

We can’t have all the goodness of web3 unless it feels like web2.

Above all else, this means that any blockchain technology that seeks mass adoption must be fast and cheap. All other attributes are subservient if the goal is for meaningful, broad impact and adoption - and for us, it is.

Solana’s been a victim of its own success with recent congestion issues, but there’s no denying it is both fast and cheap. In fact, Solana is currently the most used blockchain in the world - it regularly does 25M to 50M transactions per day, at an average of >1,000 transactions per second (https://pro.solana.fm/statistics). And it is currently the cheapest blockchain in the world, each of those 25M+ transactions costing only 0.025c each (0.000005 SOL @ $50 USD per SOL as of 21 May 2022).

But this is merely table stakes, and there are other chains that make different trade offs to Solana and have the potential to be both fast enough and cheap enough. This includes Avalanche, NEAR, Cosmos / Tendermint and the various Ethereum L2 scaling solutions.

So given that, why Solana above these other options?

Designed for scale

In traditional databases, there are two characteristics to achieving scale in both reads and writes.

1. Vertical scalability (scaling up ⇑) characterises throughput on a single machine, and
2. Horizontal scalability (scaling out ⇔) characterises throughput that can be achieved in a cluster of machines.

Vertical scalability hit its upper limits sooner, as there are physical limits to what can be done on a single machine. Horizontal scalability provides more headroom, but must make some tradeoffs to allow machines to operate in a clustered environment.

More often than not, this means introducing a degree of sharding which breaks the database into smaller sections that can operate somewhat independently, especially on write loads. However, sharding is not a decision to be made lightly. Once sharded, the database is no longer a logical whole, and transactions cannot cross the sharding boundary.

Blockchains are logically very similar. There’s a natural limit to how much throughput they can handle before the single blockchain needs to split into logical shards, and then has to deal with the sharding tradeoff.

Sharding is the main strategy for managing speed and cost in all fast and cheap chains:

• Ethereum shard chains are its path to scalability, expected to ship sometime in the next few years
• Ethereum L2s are a form of sharding with particularly high costs for crossing between the L2 and L1 boundary
• Avalanche has sharding with its subnet concept
• Cosmos / Tendermint uses sharding by promoting independent chains with interchain communication (IBC)
• NEARs path beyond 800-1000 TPS requires sharding with Nightshade

It might seem that sharding is necessary to achieve high speed at low cost. Well, not for Solana.

Solana has been designed from the ground up to maximise the throughput of the chain without requiring any sharding. In fact, the design is such that Solana’s cluster throughput is expected to converge on the throughput of any single node. Furthermore, the throughput of any single node is expected to increase over time as hardware improves.

This goal of maximising the vertical scalability of the chain without requiring sharding infuses every part of Solana’s design. For example:

• The ‘proof of history’ cryptographic clock allows a degree of ‘synchronisation without communication’ across the cluster, enabling semi-autonomous and parallelised operation
• Nodes communicate using a specialised streaming block propagation protocol that significantly reduces overhead of gossiping and downtime between switching leaders
• A runtime engine and programming model that separates state from computation and allows for massively parallel processing and increased program safety

Solana has achieved in excess of 70,000 transactions per second (TPS) in test clusters, with theoretical limits of 710,000 TPS. In the real world, we have already seen speeds of over 2,500 TPS regularly with peaks over 5,000 TPS.

It is our firm belief that these types of core design decisions are nearly impossible to retrofit if not considered upfront. Vertical scaling is very hard.

Complex software is harder to ship then hardware. This is why it’s critical to design up front as much around Moores law as possible. It’s nearly impossible to shoehorn multicore scalability after.

Anatoly Yakovenko

By contrast, horizontal scalability via sharding is always possible. Perhaps the growth of blockchain far exceeds even Solana’s capabilities in future, and some form of sharding may subsequently be needed.

By delaying this question, Solana maximises the creative surface area that web3 can address, and allows developers to treat everything as ‘one chain’. As the dust settles on a web3 future, Solana will be best positioned for its next phase of scaling design.

Affordable always

A Solana transaction currently has a fixed cost of 0.000005 SOL, which translates to 0.025c at today’s prices. This makes it the cheapest blockchain by a long way.

A month ago that would have been the end of the story.

However, in recent months this strength of Solana’s has become its vulnerability. The low cost of transactions has created an economic incentive to spam. That is, in cases where the potential reward (eg an NFT mint) far exceeds the cost (thousands of spam transactions), spam becomes inevitable. Unfortunately, this spam has had a negative impact on Solana’s ability to process transactions. This is being alleviated in two ways:

1. Through the introduction of networking improvements to allow congestion control at the networking level, and
2. The introduction of a local demand driven fee in ‘hotspots’

For deeper coverage of the underlying challenges and their solution, I highly recommend this tweet thread

Demand driven fees are used in most blockchains. How is Solana different then? How can it keep the costs of these fees low?

In other blockchains, the fee is dictated by global demand on the chain. When demand is high for one thing (eg an NFT mint), the cost use the chain is high for everyone temporarily. If global demand becomes high all the time (eg the chain becomes popular in general), then the fees for everyone become high forever.

In Solana, the fee market being introduced is based on local demand. That is, when demand is high for one thing (eg an NFT mint), the cost to submit a transaction for that one thing will become temporarily high. The cost to use the chain for everything else will remain low and fixed.

Solana is able to do this because its underlying runtime requires state and computation to be separated, and allows computation that uses different state to be run in parallel. By creating fee markets based on areas of ‘state contention’, it avoids having a single global fee market dominated by the most demanding use case.

This means that over time the cost to use other chains will be driven by the use cases with the highest economic reward, locking out significant potential usage.

Solana will be able to segregate demand by use case, enabling both high value and low value transactions to co-exist, without impacting each other adversely.

Minimal impact

An under-appreciated aspect of having a chain that is fast and cheap is that it is inherently efficient. Solana’s design and innovation means that it’s profitable to run a validator even when transactions pay only fractions of a cent. This implies that the total compute power needed to process transactions must necessarily be under some threshold to allow the network to exist.

This is confirmed by the excellent Solana Energy Usage Report: a single Solana transaction uses less energy than three Google searches.

As energy usage is a very common criticism of blockchains, some comparisons to popular chains here is relevant. This amount of energy usage is:

• 2,500,000 times less than a single Bitcoin transaction
• 250,000 times less than an Ethereum transaction with its current Proof of Work model
• 45 times less than the expected usage of an Ethereum transaction when it moves to a Proof of Stake model

We believe that sustainability must be a central concern if a blockchain is to have the right impact on our world. Lab Eleven could not in good conscience support a technology that had a material negative impact on our world. We are very proud of both Solana’s footprint and its commitment to tracking and transparency.

Developer momentum

Separate to Solana’s operational characteristics, Solana’s developer momentum is unmatched.

From the excellent Electrical Capital Developer Report, some key items stand out.

Solana has had a 4.9x year increase of developers from Dec 2020 to Dec 2021 - the largest of any chain, and putting it behind only Cosmos and Ethereum in terms of absolute numbers of developers:

There’s no denying Ethereum’s early entry in the market has resulted in the largest number of total developers. However, when normalised for days since first commit, Solana is growing faster in terms of number of developers than Ethereum:

This assessment is consistent with what we’re seeing in terms of real world community participation. The latest Ignition hackathon had 550 projects submitted by over 7,000 participants - an increase of over 760% from Solana’s first hackathon only several years earlier!

What is most impressive, is that Solana has managed to achieve all of the above momentum without any Ethereum compatibility layer.

The vast majority of other chains look to bootstrap initial developers by providing a pathway for existing Ethereum developers to enter their ecosystem. Often this comes in the form of a compatibility layer with Ethereum, allowing existing apps to be more or less ‘copy and pasted’ into the new chain.

This speaks to the inherent pull of Solana’s design - that developers were willing to eat glass and be early adopters because no other chain offers the types of characteristics and future that Solana does.

We believe that this momentum and pull will only continue:

• Solana’s development and test tooling is still reasonably primitive, and will only improve over time (and at Lab Eleven we are committed to contributing to this improvement).
• Solana network performance will continue to improve, with flow on effects to overall user experience and therefore potential use cases.
• Rust continues to be the most loved programming language in the world (for 6 years running!), providing a steady stream of experienced systems programmers ready to dive into the world of web3.

Betting on Solana is betting on the best and most committed developers in the world, which is a bet we make happily!

The future is bright

Solana’s future is very bright indeed. Beyond being fast and cheap, it is designed for the scale of web3’s ambitions, focused on remaining affordable for all, determined to minimise its environmental impact, and is making a lasting impression on developers.

We believe it is best positioned to usher in the new web3 era.

That’s why we are betting the farm on Solana. We are committed to being generous citizens of this amazing technology and community, and are looking forward to being part of bringing its potential to the masses.

My co-founder Yamen Sader described his journey on the Road to the Lab. Dipping his toe in the blockchain world led to a journey of learning, exploration and an intense desire to build.

As his colleague and friend for over 20 years, I watched curiously over his shoulder as I wrangled an engineering practice as a Partner in a Big Four consultancy (which I loved). Yamen’s curiosity was contagious. I felt compelled to take a deep breath and actually try and learn what this blockchain thing is all about. #olddognewtrick

Despite being a fairly technical chap, it had been somewhat convenient for me to dismiss blockchain. I had similar pre-conceptions and pre-fabricated responses. The alternative was to actually learn blockchain and that seemed too much like hard work. Sporadic bursts of investigation had taught me this technology was pretty weird and fundamentally very different from the tech I had leveraged throughout my career.

Eventually, after further polite nudging, I rolled up my sleeves and worked to get my head around the Solana ecosystem. Solana has operational qualities (throughput / latency / cost) that align well with the event-driven architectures I’m familiar with from the enterprise world. It felt like a technology that could solve real-world problems. Today, after a number of (confusing yet fun) months, I’m a converted man. I’ve drunk the crypto Kool-Aid and am very much on board with the spirit of the web3 movement and what modern blockchain technologies can offer.

So like Yamen, I left what was a challenging-yet-enormously-rewarding gig as a consulting Partner and I now find myself building a web3 startup. There’s an enormous amount of energy and momentum in the crypto domain so we are being careful to be respectful and play to our pedigree as founders who have:

• Started and grown significant and highly-respected software engineering teams
• Implemented (literally hundreds) of mission-critical, enterprise-grade distributed systems
• Amassed enormous delivery experience of sophisticated solutions with ambitious clients

Cut out the fluff and we land with our mission statement:

Lab Eleven provides engineering and delivery horsepower to fuel ambitious web3 projects on the Solana blockchain.

Howdy Partner!

When shaping the Lab Eleven vision it was easier for us to agree on what Lab Eleven is not rather than what it is. After 20 years or more in the consulting game, I am hungry to play a role on the product side of the fence. This means pushing beyond the initial project to build sustainable products with ambitious founders with more skin in the game.

We’ve chosen a ‘lab’ dynamic explicitly to build and nurture a team of Solana specialists who will be recognised as masters in their craft. Through focus and engineering precision, we will develop and contribute to the Solana ecosystem whilst partnering with like-minded web3 entrepreneurs who have cool shit to build.

Lab Eleven will operate primarily through an ‘engineering for equity’ partnership model. We will invest our engineering time and delivery expertise (our capital) to ensure innovative web3 ideas we believe in make it to market. We will take ownership of the heavy lifting (blockchain / treasury / infrastructure / security / cloud / automation / quality) to let web3 founders focus on the killer use cases and delightful experiences that captivate users (which isn’t really our thing).

Our confidence in getting web3 solutions to market shapes our partnering preference to share both risk and reward. Rather than a ‘bums on seats’ mentality, Lab Eleven will truly partner with ambitious web3 projects and be rewarded in the same manner as founders and strategic investors. Partners with aligned incentives and shared ambitions quickly find great working dynamics which in turn surface the right engineering behaviours.

We are aiming to be the ‘go to’ partner for Solana development and delivery expertise. With our goal to keep the lab small, we recognise there will be times when we can’t support every clever web3 idea, hence we will prioritise projects which are:

• Solving an interesting problem (that’s not obviously solved)
• Have unique differentiation in the market
• Are aligned with an open-source mindset and the spirit of web3
• Have a valid and thoughtful business model (inc tokenomics)
• Have passionate, genuine and ideally altruistic founders
• Are using interesting technology (ie Solana) in an interesting way

There are plenty of fascinating ideas around and we’re excited about playing a part in them!

What’s in it for us?

Our engineers fundamentally just want to build cool things. Bringing an idea to life is what motivates our founders and developers. We recognise however that whilst we are very good at building engineering teams, those killer product ideas won’t all come from Lab Eleven. Hence we’re playing to our strengths. Bring us your web3 project and we will make it happen!

We aim to partner and co-invest in a handful of web3 projects each year and, as a result, grow a stake in web3 businesses we are proud to have helped build. Obviously, we hope that with Lab Eleven’s engineering all these projects will do well. More obviously we hope some will do very well. Regardless we’ll be along for the ride and we’ll share the risks and rewards with the entrepreneurs and investors who trust us with their idea.

First steps?

Really the first step is to evangelise and grow our engineering team. We’re focusing exclusively on the Solana blockchain. Masters of one trade. We will be training, enabling and certifying talented software engineers to grow the region’s (heck - the globe’s) most highly credentialed Solana Dev Shop. More than a community, more than a DAO (although we do like DAOs), we are a company offering permanent employees an opportunity to join a peloton of exceptional Solana engineers to build a career in web3.

So Solana engineering is the what, but our delivery pedigree underpins the how. We’ll be working more with web3 founders and investors to offer a Solana Delivery Partner that co-invests to bring innovative web3 ideas to market. With the crypto world being a little turbulent, anonymous and unpredictable at times we hope we’ll be seen as a safe, proven and invested pair of hands.

That’s what Lab Eleven is all about! Makes sense? Wish us luck!

After 10 years of co-building one of Australia’s most successful software engineering businesses, I wanted a (questionably) well-earned break. My wife had been presented with an excellent work opportunity, and I offered to take the reins at home with our three kids.

After a few months of adjusting to one of the toughest jobs in the world - raising children - I was ready to seek sanctuary and solitude in my first love: technology. Untethered from any actual users or problems to solve, curiosity was my guide.

Like many people in technology, blockchain had been on my periphery for the last decade. And like almost everyone both in and out of technology, I had been fairly quick to form an opinion and dismiss it.

A solution looking for a problem. Just a database.A haven for Ponzi schemes and scams. A phenomenal waste of energy. Far too slow and expensive for anything useful.

Despite this, the allure remained - the unique melding of computer science and maths intrigued my inner nerd. And it had been many years since I last looked at the space.

Perhaps things had changed?

What I found took me totally by surprise and ignited the passion that eventually lead to the announcement today.

Not your father’s blockchain

I read some blogs, fired up a new wallet, added some funds, and dove into some apps. I signed my first transaction and waited. To my delight, it was confirmed a few seconds later and cost me fractions of a cent. This reasonably pleasant user experience was not the one I was expecting.

What was going on?

When I read popular critical commentary on the blockchain these days, it is dominated by soft and easy targets:

• Too slow

• Too expensive

• Too limited

• Too wasteful

What I learned is that above all else, the blockchain you’re looking at matters. We are in at least the third generation of blockchains, and everything has changed.

• Generation 1 is Bitcoin and its direct derivatives. They introduced the technology to the world, but are absolutely guilty of the above characterisations. I am thankful to Bitcoin for the innovation it unlocked but have little informed opinion on its long term usefulness to society.

• Generation 2 is Ethereum, which introduced programmability and a path to less resource usage but did not solve for speed or cost.

• Generation 3 is Solana, Avalanche, NEAR and various Ethereum ‘Level 2’ scaling solutions that directly tackle speed and cost: the primary impediments to widespread usefulness and adoption.

With these last gaps closed or closing, the obvious criticisms of blockchain begin to fade. But one much more subtle and elusive one remains: why blockchain at all?

For me, there are four core reasons:

1. Extend the lever (of software)

2. Start with open (not closed)

3. Value to users (not platforms)

4. Embrace the bazaar (not the cathedral)

Extend the lever

Give me a lever long enough ... and I shall move the world

-- Archimedes

In 2011, Marc Andreessen argued in Why Software is Eating the World that “all of the technology required to transform industries through software finally works and can be widely delivered at global scale”, and as such software is poised to disrupt all industries.

This article’s prescience is undeniable - since then, the NASDAQ has outperformed the S&P 500 in all years bar one, and the 8 out of the 10 most valuable public companies in the world are focused on technology.

What’s not said but is implied by Andreessen’s argument is that the disruption will continue as more of the world becomes software-enabled. This is a dynamic phenomenon that grows with software’s reach.

The greatest example of this is the cloud. Enterprise-grade hardware, hosting, networks and platforms became available at the click of a mouse for a low pay-as-you-go cost. The hurdle to starting a business, in time, cost and risk, was reduced by several orders of magnitude.

And as a result, a whole generation of new businesses became possible.

Blockchain represents the single biggest increase in the ‘surface area of programmability’ since the cloud. For the first time, money, ownership and contracts could be programmed at a global scale, without being gatekept by intermediaries.

And once again, as a result, a whole generation of new businesses becomes possible.

But why would these businesses be different?

Start with open

I spent a good portion of the last few years of my career working on Open Banking initiatives. The premise here is straightforward: consumers should have control over their financial data and the ability to share that with companies they trust.

After many years and many hundreds of millions of dollars spent globally, the initiative has only had modest success. The standard requires regulation to push through, is not globally adopted, is severely limited in functionality, and often pushes against the moats and business strategies of the banks themselves. Stripe and others like Plaid have built whole businesses on solving this problem.

And this is just to get access to your own data!

The unfortunate reality is that most businesses benefit from starting closed, and getting more closed over time. This often benefits the business but comes at a cost to competitors (barriers to entry) and consumers (increased lock-in and reduced choice).

And trying to open up a business that is already closed is a difficult and often futile proposition.

But a blockchain native business is designed from the ground up to ‘live in the open’. Users always own their data, and a user is always free to move between offerings.

Removing the artificial barriers to competition or portability is a huge boon - it creates incentives to retain users based on the value given to them, rather than by removing their choice to leave.

But this is not the only way in which blockchain native businesses can drive value back to users.

Value to users

The largest technology businesses in the world rely heavily on the contribution of users:

• Google search indexes the content created by people all over the world

• Facebook and Instagram surface content created by the users of the network

• Spotify provides access to music from millions of (often independent) artists

Yet the users of these platforms capture almost none of the value created on them.

You know something is profoundly wrong with our economy when Big Tech has a higher take rate than the mafia.

-- Ritchie Torres, U.S. Congressman representing the South Bronx

And in many cases, the resulting power of these platforms (described brilliantly by Ben Thompson as Aggregators) also extracts the majority of the value from businesses hosted on them as well: CAC becomes the new rent and the value over time accrues to the ‘ad space landowner’.

By enabling simple, cheap and ubiquitous access to primitives such as tokens and ownership, native web3 businesses can create business models where ownership is shared by all participants in the platform, not just its owners.

And these business models are incentivised to maximise the flow of value to users - because the platforms are open and barriers to competition and user portability are so low.

But even if a blockchain native business does represent a far greater value proposition to the consumer - through increased ownership and value - how can they possibly get off the ground given the current power and reach of incumbents?

Embrace the bazaar

In Eric Raymond’s The Cathedral and the Bazaar, one quote always stood out for me:

Who would have thought ... that a world-class operating system could coalesce as if by magic out of part-time hacking by several thousand developers scattered all over the planet, connected only by the tenuous strands of the Internet?

One of the most easily dismissed yet most powerful aspects of blockchain technology is that you can just get started. There are no barriers beyond committing the time - no gatekeepers, no upfront costs.

And you’re never starting with a blank slate - not only is there an open and inviting community connected by the (far less tenuous these days) strands of the internet, but you also have access to all the existing contracts and data on the chain. It is there, ready to be extended and built upon.

Coupled with the extended lever of blockchain, the ability to empower users with economic incentives, and a community that works in the open world of the bazaar: small teams can do more with much, much less on the blockchain.

Time to build

But that is just a lot of words.

When I first started in blockchain, I must admit that I was not building toward the future that I’ve described. I participated in decentralised finance, generating income for myself and providing liquidity through the markets through nothing more than code. I marvelled at the power of a single engineer in blockchain, and at the magic of the technology that I was using. But given what I believed about the blockchain, I did feel very keenly that I was not doing enough.

And that leads us to here. We started Lab Eleven because we believe in the power of the technology, and what it can do to democratise value to users around the world.

So once again, we are bringing together a world-class engineering team, and putting our own efforts behind this movement. If this story has piqued your interest and you want to know more, my co-founder Ben wrote reveals what this whole Lab Eleven thing is all about.

We are thrilled to now be part of accelerating towards a web3 future. Join us!