Bitcoin has changed a lot since its inception, some of the changes are quite big, and others not as much. One of the changes that would fall in the former category would be Segregated Witness, or as most would call it, Segwit.
Segwit was described in BIP 141 in December 2015, and by August 2017 it was activated on Bitcoin mainnet.
But before we speak about what Segwit is and why it was introduced (and dive into the technicals a bit), let us see what Bitcoin looked like pre segwit.
Before Segwit
Before Segwit, transactions were a bit different, simpler in transaction structure. The major things to note were:
- Transactions used the legacy format, where the signature and unlocking data were stored in the input’s scriptSig.
- This data was included in calculating the transaction id.
- Bitcoin had a straightforward 1 MB block size limit, or 1,000,000 bytes.
To illustrate, let me take you through a typical transaction pre segwit.
Say Alice wants to send bitcoin to Bob, spending a pay to public key hash (p2pkh) output. Her transaction has an input pointing to the output she wants to spend, and outputs specifying where the bitcoin goes.
The input references the previous transaction’s id and the output’s index. Its scriptsig contains:
<signature> <public key>
The public key must match the hash in the output Alice is spending, and the signature must be valid for the spend.
The simplified transaction structure looks like this (the input and output lists include their counts):
version, inputs, outputs, locktime
To get the transaction id, or txid, we hash the serialized transaction twice using sha256. So the signature is part of what determines that id.
And the Merkle tree?
The block’s transaction ids, including the coinbase transaction’s id, form the leaves of a Merkle tree.
We take pairs of ids, concatenate them, and hash them twice using sha256. We repeat this with the resulting hashes until there is one hash left, the merkle root. If a level has an odd number of hashes, the last one is duplicated before pairing.
The root goes in the block header. Changing a transaction changes its id, which changes the hashes above it and ultimately the root. This is how the header commits to the transactions in the block.
Before segwit, the signatures were included in that commitment through the transaction ids.
So what was the problem?
One problem was transaction malleability.
In some cases, someone could modify a transaction’s unlocking data while keeping the transaction valid. The bitcoin would still go to the same place, but the transaction id would change.
Now imagine another transaction was already created to spend an output from that transaction. It references the original id. If the modified version confirms instead, that reference no longer points to the confirmed transaction.
This is one of the problems Segwit addresses.
What changed with Segwit?
The idea is to separate the witness, the data used to satisfy the spending conditions, from the part used to calculate the transaction id.
For the native Segwit equivalent of our p2pkh example, called p2wpkh, the signature and public key go in the input’s witness, and the scriptsig is empty:
previous output scriptpubkey: OP_0 <20 byte public key hash>
spending input scriptsig: empty
spending input witness: <signature> <public key>
The signature still has to be checked. We have changed where the data goes.
The full transaction serialization now looks like:
version, marker, flag, inputs, outputs, witnesses, locktime
The marker is 0x00 and the flag is 0x01. Each witness belongs to an input, but the witness stacks are serialized after the outputs, in the same order as the inputs.
There are also two ids:
- txid: calculated without the marker, flag, and witness stacks.
- wtxid: calculated with the full witness inclusive serialization.
Both use double sha256. Changing only the witness changes the wtxid, but leaves the txid unchanged. Transactions still reference previous outputs using the txid.
For appropriately signed transactions whose inputs all use Segwit, this addresses involuntary transaction id malleability. Legacy inputs can still introduce the old problem, and legacy transactions are still valid.
What happens to the Merkle tree now?
The original Merkle tree is still there, built from transaction ids, and its root still goes in the block header.
But these ids no longer include the witness. So how do we make sure the block also commits to that data?
Segwit adds a second tree, the witness Merkle tree, built from the wtxid values. For this tree, the coinbase transaction’s leaf is set to 32 zero bytes.
Its root is used to create a witness commitment:
commitment = sha256(sha256(witness root || witness reserved value))
Here, || means concatenation. The reserved value is a single 32 byte item in the coinbase input’s witness.
The commitment goes in an OP_RETURN output of the coinbase transaction:
OP_RETURN <aa21a9ed followed by the 32 byte commitment>
So the witnesses themselves stay in their respective transactions. The coinbase input holds the reserved value, while a coinbase output holds the commitment.
That output is part of the coinbase’s txid, which is part of the original Merkle tree. Through the coinbase, the block header therefore commits to the witness data too.
Using a zero leaf for the coinbase in the witness tree avoids a circular dependency that is its commitment depends on the witness root, so we cannot also make that root depend on the coinbase’s actual witness transaction id.
This was introduced as a soft fork. Older nodes can check the block without witness data using their existing rules, while upgraded nodes also check the witness commitment and spending rules.
And the 1 MB limit?
Segwit introduced block weight, with a maximum of 4,000,000 weight units:
weight = (stripped size × 3) + total size
Stripped size excludes witness related serialization; total size includes it. Base bytes cost four weight units each, while the additional witness serialization costs one per byte.
For example, a transaction with 100 bytes of stripped data and 210 bytes in total weighs:
(100 × 3) + 210 = 510 weight units
Divide by four and round up, and we get 128 virtual bytes, or vB. This is the size used when expressing a fee rate in satoshis per vB.
Blocks can therefore be larger than 1 MB when they include witness data, but how much larger depends on what is in the block.
So that is the main idea behind Segwit: separate the witness from the transaction id, commit to it through the coinbase, and count it differently when measuring block weight.