Common Iterator consumers and adapters used in substrate
Substrate inherits the functional programming nature of rust and therefore makes good use of aspects of the language that underpins the functional paradigm. One of those aspects are iterators. Iterators are powerful tools that can profoundly enhance the efficiency, readability, and flexibility of code. In this guide, we'll explore some of the iterator consumers and adapter commonly used in substrate, as well as examples of where they're used.
Help us measure our progress and improve Substrate in Bits content by filling out our living feedback form. Thank you!
Note: To get the best out of this guide, it's beneficial to have basic knowledge of rust iterators and closures. To gain introductory knowledge of these topics, please check the iterator/closure section of the Rust book.
Before we proceed, let's first take a look at the differences between iterator consumers and adapters.
Consumers vs Adapters in Rust
It's important to understand the difference between iterator consumers and adapters in Rust. An iterator adapter transforms an iterator into another iterator. For example, the filter adapter filters an iterator based on a predicate, and returns another iterator that excludes the filtered items.
On the other hand, an iterator consumer takes an iterator, performs operations on it, and returns a final value. In other words, an iterator consumer doesn't generate new iterators. Rather, it outputs the result of its operation on an iterator.
for_each consumer
The for_each consumer takes an iterator and applies a closure to each item in the iterator. Because it's a consumer, for_each doesn't generate another iterator. Instead, it simply outputs the result of its operation(s) on an iterator.
Here's an example of the for_each consumer in action:
["Name:", "Age:", "Location:"].iter()
.zip(["Abdulbee", "30", "Kano"].iter())
.map(|(fields, details)| { // a tuple, each for one side of the zip
format!("{} {}", fields, details)
}
)
.for_each(|field| {
println!("{}", field);
})
The for_each consumer in the code above applies the println! macro to each item in the iterator it consumes, and return this output:
Name: Abdulbee
Age: 30
Location: Kano
Note that because for_each is a consumer, it doesn't generate a separate iterator. Rather, it consumes the iterator generated by zip adapter, applies a closure on it, and outputs new values.
Where for_each consumer is used in Substrate
Below is an example of how substrate uses the for_each adapter
ChainSpec
In substrate node's chain specification, initial authorities are represented by a vector of accounts. So are initial nominators.
let initial_authorities: Vec<(
AccountId,
AccountId,
GrandpaId,
BabeId,
ImOnlineId,
AuthorityDiscoveryId,
)> = vec![
//account of initial authorities
]
All initial authorities and nominators need to be endowed, therefore, we need to iterate through this list and endow any authority/nominator that has not been endowed.
initial_authorities
.iter() // convert initial_authorities into an iterator
.map(|x| &x.0)
.chain(initial_nominators.iter()) //create an `initial_nominators` iterator and append it with the `initial_authorities iterator`
.for_each(|x| {
// for each account in the iterator, if the account is not in a list of endowed accounts, add the account to the list of endowed accounts
if !endowed_accounts.contains(x) {
endowed_accounts.push(x.clone())
}
});
In the code above, the for_each adapter takes an iterator that contains both initial authorities and initial nominators and checks if each item is endowed. if an item is not endowed, the for_each adapter includes it in the endowed accounts list. Notice that a new iterator isn't generated here. Instead, for_each consumes the previous iterator, and performs an operation that ensures that every item in the iterator is endowed.
try_for_each Consumer
The try_for_each is very similar to the for_each consumer, except that the closure passed to it must return a Result or Option type. Applying the the closure to each item continues until an Err(e) or None is returned, in which case the for_each adapter returns Err(e) or None. This means that try_for_each may not consume all the values as the process of iteration can exit early.
Here's an example:
fn main() {
["Abdulbee", "30", "Kano"].iter()
.try_for_each(|field| -> Result<(), &'static str> {
if field.len() <=10 {Ok(())}
else {Err("At least one field has a length greater than 10")}
});
}
In the example above, we're trying to ensure that each field in the vector has a length that's not more than 10. As long as the length is not more than 10, try_for_each keeps consuming the items. but once an item's length is greater than 10, the consumption stops and an Err type is returned.
Below is an example of where the try_for_each consumer is used in substrate
any consumer
The any consumer takes an iterator and returns true if any of the iterator's items fit a particular condition.
Here's an example:
struct LogEntry {
source_ip: String,
destination_ip: String,
timestamp: u64,
suspicious: bool,
}
fn main() {
let logs = vec![
LogEntry {
source_ip: "192.168.0.1".to_string(),
destination_ip: "10.0.0.2".to_string(),
timestamp: 1636351200,
suspicious: false,
},
LogEntry {
source_ip: "192.168.0.2".to_string(),
destination_ip: "10.0.0.3".to_string(),
timestamp: 1636361200,
suspicious: true,
},
LogEntry {
source_ip: "192.168.0.3".to_string(),
destination_ip: "10.0.0.4".to_string(),
timestamp: 1636381200,
suspicious: false,
},
LogEntry {
source_ip: "192.168.0.7".to_string(),
destination_ip: "10.0.0.7".to_string(),
timestamp: 1636382200,
suspicious: true,
},
];
if logs.iter().any(|entry| entry.suspicious) {
let sus_count = logs.iter().filter(|entry| entry.suspicious).count();
println! ("{} number of suspicious activities have been detected", sus_count);
}
else {
println! ("No suspicious activity detected");
}
}
The program above has a struct that stores details of an IP log, including whether the IP address is suspicious or not. All logs are stored in a vector, and the any consumer is used to determine if any of the logs are suspicious. If any of the logs are suspicious, a warning message is printed, including the total number of suspicious logs
Below is an example of where the any adapter is used in substrate:
impl<T: Config> IsMember<AuthorityId> for Pallet<T> {
fn is_member(authority_id: &AuthorityId) -> bool {
<Pallet<T>>::authorities().iter().any(|id| &id.0 == authority_id)
}
}
The code snippet above comes from BABE module in FRAME. the is_member associated function takes an authority id and determines if the id is a part of the authority set. the any consumer takes the authorities iterator and returns true if any of the ids match the authority id in the function's parameter.
all consumer
The all consumer, unlike the any consumer, only returns true if all the items of the iterator satisfy a given predicate.
Here's an example:
struct LogEntry {
source_ip: String,
destination_ip: String,
timestamp: u64,
suspicious: bool,
}
fn main() {
let logs = vec![
LogEntry {
source_ip: "192.168.0.1".to_string(),
destination_ip: "10.0.0.2".to_string(),
timestamp: 1636351200,
suspicious: false,
},
LogEntry {
source_ip: "192.168.0.2".to_string(),
destination_ip: "10.0.0.3".to_string(),
timestamp: 1636361200,
suspicious: true,
},
LogEntry {
source_ip: "192.168.0.3".to_string(),
destination_ip: "10.0.0.4".to_string(),
timestamp: 1636381200,
suspicious: true,
},
LogEntry {
source_ip: "192.168.0.7".to_string(),
destination_ip: "10.0.0.7".to_string(),
timestamp: 1636382200,
suspicious: true,
},
];
if logs.iter().all(|entry| entry.suspicious == false) {
println! ("No suspicious activity detected");
}
else {
let sus_count = logs.iter().filter(|entry| entry.suspicious).count();
println! ("{} number of suspicious activities have been detected", sus_count);
}
}
This program is similar to a previous one. But in this case, the all consumer is used. the if branch runs if there's any suspicious activity at all. Else, the else branch runs.
Below is an example of where the all adapter is used in substrate
let is_feature_active = pallet_declaration.cfg_pattern.iter().all(|expr| {
expr.eval(|pred| match pred {
Predicate::Feature(f) => feature_set.contains(f),
Predicate::Test => feature_set.contains(&"test"),
_ => false,
})
});
//...
The snippet above is from the construct_runtime macro, and only returns true if the predicates listed in all #[cfg] attributes all matches.
Summary
This tutorial has provided an insightful exploration of common iterator consumers and adapters used in substrate, shedding light on the crucial distinction between the two concepts. Mastering iterators is essential when it comes to the functional programming style used in substrate, and this tutorial serves as a valuable guide for those seeking an intermediate-level grasp of these concepts.
Help us measure our progress and improve Substrate in Bits content by filling out our living feedback form. Thank you!
