Uh oh!
There was an error while loading. Please reload this page.
BREAKING Overlay transaction support. - #3263
Conversation
pepyakin
left a comment
There was a problem hiding this comment.
This looks like a half way through to #2980! (With another part being an runtime instance state/linear memory concerns). But even with that, I think it might be super useful!
We even might be able to construct ext_try from primitives introduced in this PR and with a future introduced primitives for cloning a wasm instance!
Uh oh!
There was an error while loading. Please reload this page.
Uh oh!
There was an error while loading. Please reload this page.
Uh oh!
There was an error while loading. Please reload this page.
Uh oh!
There was an error while loading. Please reload this page.
Uh oh!
There was an error while loading. Please reload this page.
Uh oh!
There was an error while loading. Please reload this page.
Uh oh!
There was an error while loading. Please reload this page.
Uh oh!
There was an error while loading. Please reload this page.
Uh oh!
There was an error while loading. Please reload this page.
Uh oh!
There was an error while loading. Please reload this page.
Uh oh!
There was an error while loading. Please reload this page.
Uh oh!
There was an error while loading. Please reload this page.
Uh oh!
There was an error while loading. Please reload this page.
Uh oh!
There was an error while loading. Please reload this page.
Uh oh!
There was an error while loading. Please reload this page.
Uh oh!
There was an error while loading. Please reload this page.
Remove bench feature, exposing 'set_storage' publicly. Minor fixes.
Uh oh!
There was an error while loading. Please reload this page.
gui1117
commented
Jul 31, 2019
Maybe start_transactional_layer |
I forgot to update my comment, I change it to 'storage_start_transaction' but maybe 'start_transactional_layer' is better? |
Co-Authored-By: Sergei Pepyakin <s.pepyakin@gmail.com>
I am actually not sure about the wording "layer". I.e. transactional layer of what? What if we get another kind of layer? On the other hand, "storage transaction" is unambigious. |
Uh oh!
There was an error while loading. Please reload this page.
Uh oh!
There was an error while loading. Please reload this page.
Uh oh!
There was an error while loading. Please reload this page.
Uh oh!
There was an error while loading. Please reload this page.
Uh oh!
There was an error while loading. Please reload this page.
Uh oh!
There was an error while loading. Please reload this page.
Uh oh!
There was an error while loading. Please reload this page.
Uh oh!
There was an error while loading. Please reload this page.
Uh oh!
There was an error while loading. Please reload this page.
Uh oh!
There was an error while loading. Please reload this page.
| /// Interface for managing transaction within the runtime. | ||
| #[runtime_interface] | ||
| pub trait Transaction { |
There was a problem hiding this comment.
I wonder if StorageTransactions will be a better name?
| // You should have received a copy of the GNU General Public License | ||
| // along with Substrate. If not, see <http://www.gnu.org/licenses/>. | ||
| //! Types and method for managing a stack of transactional values. |
There was a problem hiding this comment.
| //! Types and method for managing a stack of transactional values. | |
| //! Types and methods for managing a stack of transactional values. |
| (_, Some(overlay_key)) => if overlay_key.1.value.is_some() { | ||
| return Some(overlay_key.0.to_vec()) | ||
| } else { | ||
| // TODO make this function non recursive to avoid this clone |
There was a problem hiding this comment.
Does this TODO need to be fixed before landing this? If not, an issue is required.
There was a problem hiding this comment.
No those TODO were just a note to myself, I am removing them.
| }; | ||
| // TODO no need to query child at each recursive iter here | ||
| // and also find a way to remove the clone (non recursive and global mut handle should do the | ||
| // trick). |
There was a problem hiding this comment.
Does this TODO need to be fixed before landing this? If not, an issue is required.
| .map(|&(ref k, ref v)| (k.to_vec(), Some(v.to_vec()))) | ||
| .chain(self.overlay.committed.top.clone().into_iter().map(|(k, v)| (k, v.value))) | ||
| .chain(self.overlay.prospective.top.clone().into_iter().map(|(k, v)| (k, v.value))) | ||
| .chain(self.overlay.changes.iter_values(None).map(|(k, v)| (k.to_vec(), v.map(|s| s.to_vec())))) |
There was a problem hiding this comment.
The line exceeds the recommended limit for 100 characters. Can we wrap this line?
| TxPending, | ||
| /// The transaction has been discarded. | ||
| /// Data from a `LayerEntry` pointing to this layer state should | ||
| /// not be returned and can be remove. |
| /// Get latest prospective value, excludes | ||
| /// committed values. | ||
| pub(crate) fn get_prospective(&self, states: &States) -> Option<&V> { | ||
| let self_len = self.len(); |
There was a problem hiding this comment.
I find it a bit confusing that the definition of len is in another module. Can we move these definitions closer together?
| /// triggering. | ||
| /// | ||
| /// With this default values it should be very unlikelly that gc is needed | ||
| /// during a block processing for most use case. |
| // This information is redundant as it could be | ||
| // calculated by iterating backward over `history` | ||
| // field. | ||
| // Managing this cache allow use to have |
| None | ||
| } | ||
| /// Push a value without checking without transactional layer |
Significant change of the implementation after my look.
cover all case but is a good entry point to read this PR.
cheme
commented
May 14, 2020
This is stale, closing it, @athei will be working on the subject, keeping it open will only create confusion. |
BREAKING: this pr adds externalities, and change mutability of existing one, the corresponding
This PR is a refactor of overlay_change to allow transactional support, it is similar to paritytech/polkadot-sdk#370 goals, but does not stack
Ext, it only stacks storage.It also switches from a stack of hashmap (previously
prospectiveandtop) to a single hashmap (containing history of values) and a transaction global state. Those values with state are managedwith 'historied-data' crate (simple vec of data to query in front of a reference global state).
Under this design access to data is not badly impacted by the number of open transactional layers.
I did fuzz this code a bit against a partial simple layered hashmap implementation.
Usage from a runtime with a function, in a similar way as
ext_tryfrom paritytech/polkadot-sdk#370, there is the very simplewith_transactionfunction: internally it uses three host functionsext_start_transaction,ext_commit_transactionandext_discard_transaction. This does not look as good as the singleext_trybut is clearer: memory mgmt seems way simplier (as there is none).Note that to call global state action, modification of local values need to be synchronize.
eg discard_transaction on states must be follow by apply_discard_transaction for all values related to this state and then follow by ensure_running.
Technically we only maintain a counter of current number of stacked transaction as a global state (this start at 1 and can be 0 only to update some values: case of discarding content, but will then return to 1 through 'finalize_discard' call).
Local state is either committed or the number of stacked transaction when the value was changed, this state is stored with the value history.
polkadot companion: paritytech/polkadot#999