Consensus Mechanisms: Detailed Documentation
Overview
The ArthaChain blockchain employs several advanced consensus mechanisms to achieve high throughput, security, and cross-shard capabilities. The primary consensus algorithms include Quantum SVBFT (Scalable Byzantine Fault Tolerance), adaptive consensus, and specialized cross-shard coordination.
Architecture
The consensus module is organized into several specialized components:
blockchain_node/src/consensus/
├── adaptive.rs # Adaptive consensus parameters
├── block_manager.rs # Block creation and management
├── byzantine.rs # Byzantine fault tolerance implementation
├── cross_shard/ # Cross-shard transaction coordination
│ ├── coordinator.rs # Shard coordination logic
│ ├── integration.rs # Integration with main consensus
│ ├── mod.rs # Module definition
│ ├── protocol.rs # Cross-shard protocol implementation
│ ├── resource.rs # Resource management
│ ├── routing.rs # Transaction routing
│ ├── sharding.rs # Sharding logic
│ └── tests/ # Test implementations
├── ltl.rs # Linear Temporal Logic for consensus verification
├── mod.rs # Main module definition
├── parallel_processor.rs # Parallel transaction processing
├── parallel_tx.rs # Parallel transaction validation
├── petri_net.rs # Petri net modeling for consensus
├── quantum_svbft.rs # Quantum-resistant SVBFT implementation
├── reputation.rs # Validator reputation system
├── svcp.rs # Scalable Verifiable Consensus Protocol
└── view_change.rs # View change protocol for leader rotation
Core Consensus Algorithms
1. Quantum SVBFT (quantum_svbft.rs)
The Quantum SVBFT algorithm is an advanced Byzantine Fault Tolerant consensus mechanism designed to be resistant to quantum computing attacks.
pub struct QuantumSVBFT {
// Current validator set
validators: Vec<Validator>,
// Current view (round) number
view: u64,
// Current primary (leader) validator
primary: ValidatorId,
// Quantum-resistant cryptographic components
crypto: QuantumCrypto,
// State of the consensus
state: ConsensusState,
// Configuration parameters
config: QuantumSVBFTConfig,
}
impl QuantumSVBFT {
// Creates a new instance of the Quantum SVBFT
pub fn new(validators: Vec<Validator>, config: QuantumSVBFTConfig) -> Self { ... }
// Processes a consensus message
pub fn process_message(&mut self, message: ConsensusMessage) -> Result<ConsensusAction, ConsensusError> { ... }
// Proposes a new block as the primary validator
pub fn propose_block(&mut self, transactions: Vec<Transaction>) -> Result<Block, ConsensusError> { ... }
// Prepares for a block (prepare phase)
pub fn prepare(&mut self, block_hash: &Hash) -> Result<PrepareMessage, ConsensusError> { ... }
// Commits to a block (commit phase)
pub fn commit(&mut self, block_hash: &Hash) -> Result<CommitMessage, ConsensusError> { ... }
// Finalizes a block, making it part of the chain
pub fn finalize_block(&mut self, block: &Block) -> Result<(), ConsensusError> { ... }
// Initiates a view change when the primary is suspected to be faulty
pub fn initiate_view_change(&mut self) -> Result<ViewChangeMessage, ConsensusError> { ... }
}
Key characteristics:
- Quantum Resistance: Uses post-quantum cryptographic primitives
- High Throughput: Optimized for processing thousands of transactions per second
- Low Latency: Achieves finality in 2-3 seconds under normal conditions
- Byzantine Fault Tolerance: Can tolerate up to f faulty nodes in a 3f+1 system
- Adaptive Difficulty: Adjusts consensus parameters based on network conditions
2. Adaptive Consensus (adaptive.rs)
The adaptive consensus component dynamically adjusts consensus parameters based on network conditions.
pub struct AdaptiveConsensus {
// Network metrics collector
metrics: NetworkMetrics,
// Current parameter set
parameters: ConsensusParameters,
// Parameter adjustment rules
adjustment_rules: Vec<AdjustmentRule>,
// History of parameter changes
parameter_history: VecDeque<ParameterChange>,
}
impl AdaptiveConsensus {
// Creates a new adaptive consensus component
pub fn new(initial_parameters: ConsensusParameters) -> Self { ... }
// Updates network metrics
pub fn update_metrics(&mut self, new_metrics: NetworkMetrics) { ... }
// Computes optimal parameters based on current conditions
pub fn compute_optimal_parameters(&self) -> ConsensusParameters { ... }
// Applies a parameter change
pub fn apply_parameter_change(&mut self, change: ParameterChange) -> Result<(), ConsensusError> { ... }
// Gets the current parameters
pub fn get_current_parameters(&self) -> &ConsensusParameters { ... }
}
Key characteristics:
- Dynamic Timeout Adjustment: Adapts timeout values based on network latency
- Batch Size Optimization: Adjusts transaction batch sizes for optimal throughput
- Validator Set Scaling: Dynamically adjusts the active validator set size
- Progressive Difficulty: Increases security during high-value transaction periods
3. Cross-Shard Consensus (cross_shard/)
The cross-shard consensus enables atomic transactions across multiple shards.
Coordinator (coordinator.rs)
pub struct CrossShardCoordinator {
// Shard this coordinator belongs to
shard_id: ShardId,
// Cross-shard transactions being coordinated
active_transactions: HashMap<TransactionId, CrossShardTransaction>,
// Connections to other shard coordinators
shard_connections: HashMap<ShardId, ShardConnection>,
// Protocol for cross-shard communication
protocol: CrossShardProtocol,
}
impl CrossShardCoordinator {
// Creates a new cross-shard coordinator
pub fn new(shard_id: ShardId, protocol: CrossShardProtocol) -> Self { ... }
// Initiates a cross-shard transaction
pub fn initiate_transaction(&mut self, transaction: Transaction) -> Result<TransactionId, CrossShardError> { ... }
// Prepares resources for a cross-shard transaction
pub fn prepare_resources(&mut self, tx_id: &TransactionId) -> Result<ResourceLockResult, CrossShardError> { ... }
// Commits a cross-shard transaction
pub fn commit_transaction(&mut self, tx_id: &TransactionId) -> Result<(), CrossShardError> { ... }
// Aborts a cross-shard transaction
pub fn abort_transaction(&mut self, tx_id: &TransactionId) -> Result<(), CrossShardError> { ... }
// Processes a message from another shard
pub fn process_message(&mut self, message: CrossShardMessage) -> Result<CrossShardAction, CrossShardError> { ... }
}
Protocol (protocol.rs)
pub struct CrossShardProtocol {
// Protocol version
version: u32,
// Message authentication
authenticator: MessageAuthenticator,
// Timeout settings
timeouts: CrossShardTimeouts,
// Retry policy
retry_policy: RetryPolicy,
}
impl CrossShardProtocol {
// Creates a new cross-shard protocol instance
pub fn new(version: u32, timeouts: CrossShardTimeouts) -> Self { ... }
// Prepares a cross-shard message
pub fn prepare_message(&self, content: CrossShardContent) -> CrossShardMessage { ... }
// Validates a received cross-shard message
pub fn validate_message(&self, message: &CrossShardMessage) -> Result<(), ValidationError> { ... }
// Handles a timeout for a cross-shard operation
pub fn handle_timeout(&self, operation: &CrossShardOperation) -> TimeoutAction { ... }
}
4. Reputation System (reputation.rs)
The reputation system tracks validator performance and reliability.
pub struct ReputationSystem {
// Validator reputation scores
validator_scores: HashMap<ValidatorId, ReputationScore>,
// History of reputation changes
reputation_history: VecDeque<ReputationChange>,
// Reputation update rules
update_rules: Vec<ReputationRule>,
}
impl ReputationSystem {
// Creates a new reputation system
pub fn new(validators: &[ValidatorId]) -> Self { ... }
// Updates a validator's reputation
pub fn update_reputation(&mut self, validator_id: &ValidatorId, event: ReputationEvent) { ... }
// Gets a validator's current reputation score
pub fn get_reputation(&self, validator_id: &ValidatorId) -> Option<ReputationScore> { ... }
// Selects validators for the next consensus round based on reputation
pub fn select_validators(&self, count: usize) -> Vec<ValidatorId> { ... }
// Determines if a validator should be excluded due to poor reputation
pub fn should_exclude(&self, validator_id: &ValidatorId) -> bool { ... }
}
Consensus Flow
1. Block Proposal Phase
- The current primary validator collects transactions from the mempool
- Transactions are validated and organized into a block proposal
- The block proposal is signed using quantum-resistant cryptography
- The proposal is broadcast to all validators
// Primary validator creates a block proposal
let transactions = mempool.get_transactions(max_tx_count);
let block_proposal = BlockProposal::new(
previous_block_hash,
block_height,
transactions,
timestamp,
primary_validator_id,
);
// Sign the proposal with quantum-resistant signature
let signature = quantum_crypto.sign(&block_proposal.hash(), &primary_private_key);
block_proposal.set_signature(signature);
// Broadcast to all validators
network.broadcast(ConsensusMessage::Proposal(block_proposal));
2. Preparation Phase
- Validators receive the block proposal and verify its validity
- Each validator sends a PREPARE message if the proposal is valid
- Validators collect PREPARE messages from other validators
- Once 2f+1 PREPARE messages are collected, the validator moves to the commit phase
// Validator receives a block proposal
fn handle_proposal(&mut self, proposal: BlockProposal) -> Result<(), ConsensusError> {
// Verify proposal validity
if !self.verify_proposal(&proposal) {
return Err(ConsensusError::InvalidProposal);
}
// Create and sign PREPARE message
let prepare_message = PrepareMessage::new(
proposal.hash(),
self.view,
self.validator_id,
);
let signature = self.crypto.sign(&prepare_message.hash(), &self.private_key);
prepare_message.set_signature(signature);
// Broadcast PREPARE message
self.network.broadcast(ConsensusMessage::Prepare(prepare_message));
// Update internal state
self.state = ConsensusState::Preparing(proposal.hash());
Ok(())
}
// Validator collects PREPARE messages
fn handle_prepare(&mut self, prepare: PrepareMessage) -> Result<ConsensusAction, ConsensusError> {
// Verify the PREPARE message
if !self.verify_prepare(&prepare) {
return Err(ConsensusError::InvalidPrepare);
}
// Add to collected PREPAREs
self.prepare_messages.insert(prepare.validator_id(), prepare);
// Check if we have enough PREPARE messages (2f+1)
if self.prepare_messages.len() >= self.quorum_size() {
// Move to COMMIT phase
let commit_message = self.create_commit_message(prepare.block_hash());
self.network.broadcast(ConsensusMessage::Commit(commit_message));
self.state = ConsensusState::Committing(prepare.block_hash());
return Ok(ConsensusAction::MoveToCommit);
}
Ok(ConsensusAction::None)
}
3. Commit Phase
- Once in the commit phase, validators send COMMIT messages
- Validators collect COMMIT messages from other validators
- Once 2f+1 COMMIT messages are collected, the block is finalized
- The block is added to the blockchain and state is updated
// Validator handles COMMIT messages
fn handle_commit(&mut self, commit: CommitMessage) -> Result<ConsensusAction, ConsensusError> {
// Verify the COMMIT message
if !self.verify_commit(&commit) {
return Err(ConsensusError::InvalidCommit);
}
// Add to collected COMMITs
self.commit_messages.insert(commit.validator_id(), commit);
// Check if we have enough COMMIT messages (2f+1)
if self.commit_messages.len() >= self.quorum_size() {
// Finalize the block
let block = self.pending_blocks.get(&commit.block_hash()).unwrap();
self.finalize_block(block);
// Reset consensus state for next round
self.reset_consensus_state();
return Ok(ConsensusAction::FinalizeBlock(block.clone()));
}
Ok(ConsensusAction::None)
}
// Finalizing a block
fn finalize_block(&mut self, block: &Block) -> Result<(), ConsensusError> {
// Add block to blockchain
self.blockchain.add_block(block.clone())?;
// Update state with transactions in the block
for tx in block.transactions() {
self.state_manager.apply_transaction(tx)?;
}
// Update validator set if needed
if block.has_validator_changes() {
self.update_validator_set(block.validator_changes());
}
// Notify other components
self.event_bus.publish(BlockchainEvent::BlockFinalized(block.clone()));
Ok(())
}
4. View Change
If the primary validator is suspected to be faulty (e.g., no proposal for too long), a view change is initiated:
- Validators send VIEW-CHANGE messages
- Once 2f+1 VIEW-CHANGE messages are collected, a new view begins
- The primary for the new view is selected based on a deterministic formula
// Initiating a view change
fn initiate_view_change(&mut self) -> Result<(), ConsensusError> {
// Create VIEW-CHANGE message
let view_change = ViewChangeMessage::new(
self.view,
self.view + 1,
self.validator_id,
self.last_finalized_block_hash,
);
// Sign and broadcast VIEW-CHANGE message
let signature = self.crypto.sign(&view_change.hash(), &self.private_key);
view_change.set_signature(signature);
self.network.broadcast(ConsensusMessage::ViewChange(view_change));
// Update internal state
self.state = ConsensusState::ViewChanging(self.view + 1);
Ok(())
}
// Handling VIEW-CHANGE messages
fn handle_view_change(&mut self, view_change: ViewChangeMessage) -> Result<ConsensusAction, ConsensusError> {
// Verify VIEW-CHANGE message
if !self.verify_view_change(&view_change) {
return Err(ConsensusError::InvalidViewChange);
}
// Add to collected VIEW-CHANGEs
self.view_change_messages.insert(view_change.validator_id(), view_change);
// Check if we have enough VIEW-CHANGE messages (2f+1)
if self.view_change_messages.len() >= self.quorum_size() {
// Execute view change
self.execute_view_change(view_change.new_view());
return Ok(ConsensusAction::ViewChanged(view_change.new_view()));
}
Ok(ConsensusAction::None)
}
// Executing a view change
fn execute_view_change(&mut self, new_view: ViewNumber) -> Result<(), ConsensusError> {
// Update view number
self.view = new_view;
// Select new primary
self.primary = self.select_primary_for_view(new_view);
// Reset consensus state
self.reset_consensus_state();
// If I am the new primary, start a new round
if self.primary == self.validator_id {
self.start_new_round()?;
}
Ok(())
}
Cross-Shard Transaction Flow
Cross-shard transactions follow a two-phase commit protocol:
1. Preparation Phase
- The originating shard coordinator initiates the transaction
- Resource requirements are identified across all involved shards
- Each shard attempts to lock the required resources
- If all resources are successfully locked, the preparation phase is complete
// Initiating a cross-shard transaction
fn initiate_cross_shard_tx(&mut self, tx: Transaction) -> Result<(), CrossShardError> {
// Identify involved shards
let involved_shards = self.identify_involved_shards(&tx);
// Create cross-shard transaction record
let cross_shard_tx = CrossShardTransaction::new(
tx.id(),
self.shard_id,
involved_shards.clone(),
tx.clone(),
);
// Store the transaction
self.active_transactions.insert(tx.id(), cross_shard_tx);
// Send PREPARE messages to all involved shards
for shard_id in involved_shards {
if shard_id != self.shard_id {
let prepare_message = CrossShardMessage::Prepare(
tx.id(),
self.shard_id,
shard_id,
tx.resource_requirements_for_shard(shard_id),
);
self.send_to_shard(shard_id, prepare_message);
}
}
// Lock local resources
self.resource_manager.lock_resources(tx.resource_requirements_for_shard(self.shard_id))?;
Ok(())
}
// Handling a PREPARE message from another shard
fn handle_prepare(&mut self, prepare: CrossShardPrepareMessage) -> Result<(), CrossShardError> {
// Try to lock requested resources
match self.resource_manager.try_lock_resources(prepare.resource_requirements()) {
Ok(_) => {
// Resources locked successfully, send PREPARED response
let prepared_message = CrossShardMessage::Prepared(
prepare.tx_id(),
self.shard_id,
prepare.originating_shard(),
);
self.send_to_shard(prepare.originating_shard(), prepared_message);
// Record the prepared transaction
let tx_record = CrossShardTransactionRecord::new(
prepare.tx_id(),
prepare.originating_shard(),
prepare.resource_requirements(),
);
self.prepared_transactions.insert(prepare.tx_id(), tx_record);
Ok(())
},
Err(e) => {
// Failed to lock resources, send ABORT
let abort_message = CrossShardMessage::Abort(
prepare.tx_id(),
self.shard_id,
prepare.originating_shard(),
format!("Failed to lock resources: {}", e),
);
self.send_to_shard(prepare.originating_shard(), abort_message);
Err(e)
}
}
}
2. Commit Phase
- If preparation is successful, the coordinator sends COMMIT messages
- Each shard executes its portion of the transaction
- Results are reported back to the coordinator
- The coordinator finalizes the transaction
// Coordinator decides to commit after all shards are prepared
fn commit_cross_shard_tx(&mut self, tx_id: &TransactionId) -> Result<(), CrossShardError> {
// Get the cross-shard transaction
let tx = self.active_transactions.get(tx_id)
.ok_or(CrossShardError::TransactionNotFound)?;
// Verify all shards are prepared
if !tx.all_shards_prepared() {
return Err(CrossShardError::NotAllShardsPrepared);
}
// Send COMMIT messages to all involved shards
for shard_id in tx.involved_shards() {
if shard_id != self.shard_id {
let commit_message = CrossShardMessage::Commit(
*tx_id,
self.shard_id,
shard_id,
);
self.send_to_shard(shard_id, commit_message);
}
}
// Execute local portion of the transaction
self.transaction_executor.execute(&tx.transaction())?;
// Update transaction state
tx.set_state(CrossShardTransactionState::Committing);
Ok(())
}
// Handling a COMMIT message from the coordinator
fn handle_commit(&mut self, commit: CrossShardCommitMessage) -> Result<(), CrossShardError> {
// Get the prepared transaction
let tx_record = self.prepared_transactions.get(&commit.tx_id())
.ok_or(CrossShardError::TransactionNotFound)?;
// Execute the transaction
match self.transaction_executor.execute_prepared(tx_record) {
Ok(result) => {
// Send COMMITTED message back to coordinator
let committed_message = CrossShardMessage::Committed(
commit.tx_id(),
self.shard_id,
commit.originating_shard(),
result,
);
self.send_to_shard(commit.originating_shard(), committed_message);
// Clean up resources
self.resource_manager.convert_locks_to_permanent(tx_record.resource_requirements());
self.prepared_transactions.remove(&commit.tx_id());
Ok(())
},
Err(e) => {
// Send FAILED message back to coordinator
let failed_message = CrossShardMessage::Failed(
commit.tx_id(),
self.shard_id,
commit.originating_shard(),
format!("Execution failed: {}", e),
);
self.send_to_shard(commit.originating_shard(), failed_message);
// Release resources
self.resource_manager.release_locks(tx_record.resource_requirements());
self.prepared_transactions.remove(&commit.tx_id());
Err(e.into())
}
}
}
3. Abort Handling
If any shard fails during preparation or commit, the transaction is aborted:
// Coordinator aborts a cross-shard transaction
fn abort_cross_shard_tx(&mut self, tx_id: &TransactionId, reason: &str) -> Result<(), CrossShardError> {
// Get the cross-shard transaction
let tx = self.active_transactions.get(tx_id)
.ok_or(CrossShardError::TransactionNotFound)?;
// Send ABORT messages to all involved shards
for shard_id in tx.involved_shards() {
if shard_id != self.shard_id {
let abort_message = CrossShardMessage::Abort(
*tx_id,
self.shard_id,
shard_id,
reason.to_string(),
);
self.send_to_shard(shard_id, abort_message);
}
}
// Release local resources
self.resource_manager.release_locks(tx.local_resource_requirements());
// Update transaction state and record reason
tx.set_state(CrossShardTransactionState::Aborted(reason.to_string()));
// Clean up after timeout
self.scheduler.schedule_cleanup(*tx_id, Duration::from_secs(60));
Ok(())
}
// Handling an ABORT message
fn handle_abort(&mut self, abort: CrossShardAbortMessage) -> Result<(), CrossShardError> {
// Get the prepared transaction
if let Some(tx_record) = self.prepared_transactions.remove(&abort.tx_id()) {
// Release locked resources
self.resource_manager.release_locks(tx_record.resource_requirements());
// Log the abort
self.logger.info(
"Cross-shard transaction aborted: {} from shard {} reason: {}",
abort.tx_id(),
abort.originating_shard(),
abort.reason(),
);
}
Ok(())
}
Performance Characteristics
The consensus algorithms are optimized for the following performance targets:
- Transaction Throughput: Up to 22,680,000 TPS for small transactions (100 bytes) and 608,000 TPS for large transactions (10KB)
- Block Time: 1-3 seconds under normal network conditions
- Finality: Absolute finality after 2-3 blocks
- Fault Tolerance: Can tolerate up to f Byzantine nodes in a 3f+1 system
- Shard Count: Support for up to 64 independent shards
- Cross-Shard Latency: 2-4 seconds for cross-shard transactions
Security Considerations
- Adaptive Security: Security parameters adjust based on transaction value
- Quantum Resistance: All cryptographic operations use post-quantum algorithms
- Leader Selection: Leader selection uses verifiable random functions to prevent manipulation
- Reputation System: Validators with poor performance or suspicious behavior are excluded
- Formal Verification: Critical consensus paths are formally verified
Configuration Parameters
The consensus mechanisms can be configured through the following parameters:
pub struct ConsensusConfig {
// Maximum block size in bytes
pub max_block_size: usize,
// Maximum number of transactions per block
pub max_transactions_per_block: usize,
// Block time target in milliseconds
pub block_time_ms: u64,
// View change timeout in milliseconds
pub view_change_timeout_ms: u64,
// Minimum number of validators
pub min_validators: usize,
// Maximum number of validators
pub max_validators: usize,
// Number of blocks after which a block is considered final
pub finality_depth: u64,
// Cross-shard transaction timeout in milliseconds
pub cross_shard_timeout_ms: u64,
// Quantum security level (bits)
pub quantum_security_bits: u32,
}
Usage Examples
Basic Consensus Operation
// Initialize consensus with validators
let validators = vec![
Validator::new(ValidatorId(1), PublicKey::from_hex("...")),
Validator::new(ValidatorId(2), PublicKey::from_hex("...")),
Validator::new(ValidatorId(3), PublicKey::from_hex("...")),
Validator::new(ValidatorId(4), PublicKey::from_hex("...")),
];
let config = QuantumSVBFTConfig {
block_time_ms: 2000,
view_change_timeout_ms: 10000,
max_transactions_per_block: 10000,
quantum_security_bits: 256,
};
let mut consensus = QuantumSVBFT::new(validators, config);
// Primary validator proposes a block
if consensus.is_primary() {
let transactions = mempool.get_pending_transactions(1000);
let block = consensus.propose_block(transactions)?;
network.broadcast(ConsensusMessage::Proposal(block));
}
// Non-primary validators process messages
while let Some(message) = network.next_message() {
match message {
ConsensusMessage::Proposal(proposal) => {
consensus.process_proposal(proposal)?;
},
ConsensusMessage::Prepare(prepare) => {
consensus.process_prepare(prepare)?;
},
ConsensusMessage::Commit(commit) => {
consensus.process_commit(commit)?;
},
ConsensusMessage::ViewChange(view_change) => {
consensus.process_view_change(view_change)?;
},
}
}
Cross-Shard Transaction
// Create a cross-shard transaction
let input1 = TransactionInput {
shard_id: ShardId(1),
account: AccountId(0x123),
amount: 100,
};
let output1 = TransactionOutput {
shard_id: ShardId(2),
account: AccountId(0x456),
amount: 100,
};
let cross_shard_tx = Transaction::new(
vec![input1],
vec![output1],
NetworkId(1),
);
// Submit the transaction to the originating shard
let shard1_coordinator = network.get_shard_coordinator(ShardId(1));
shard1_coordinator.submit_transaction(cross_shard_tx)?;
// The coordinator handles the cross-shard protocol
// 1. Preparation phase
shard1_coordinator.prepare_transaction(&cross_shard_tx.id())?;
// 2. Wait for all shards to be prepared
while !shard1_coordinator.all_shards_prepared(&cross_shard_tx.id()) {
thread::sleep(Duration::from_millis(100));
}
// 3. Commit phase
shard1_coordinator.commit_transaction(&cross_shard_tx.id())?;
// 4. Wait for all shards to commit
while !shard1_coordinator.all_shards_committed(&cross_shard_tx.id()) {
thread::sleep(Duration::from_millis(100));
}
println!("Cross-shard transaction completed successfully!");
Future Developments
Planned enhancements for the consensus mechanisms include:
- Parallel Consensus: Running multiple consensus instances in parallel
- Hierarchical Consensus: Consensus at multiple levels for scalability
- Threshold Signatures: Reducing communication complexity with threshold cryptography
- Validator Rotation: Dynamic rotation of validators for improved security
- Deep Sharding: Nested shards for extreme scalability