Skip to content
This repository was archived by the owner on Nov 15, 2023. It is now read-only.

Light client friendly events - #2491

Merged
gavofyork merged 20 commits into
masterfrom
ser-topic-events
May 13, 2019
Merged

Light client friendly events#2491
gavofyork merged 20 commits into
masterfrom
ser-topic-events

Conversation

@pepyakin

@pepyakinpepyakin commented May 6, 2019

Copy link
Copy Markdown
Contributor

This PR proposes a solution for light client friendly SRML events.

Each deposited event can specify zero or more topics. A topic is a fixed sized value (atm, is just a hash). When an event is deposited with some topic, the runtime adds the index of this event in the <Events<T>> list to a mapping that maps from topics to lists of indexes.

Light clients can "subscribe" to changes of specific storage values (see #628). The idea is that light clients would subscribe to changes of the storage values for a topic they are interested in. When the light client detects a change, it fetches the contents of the indexes list for the specific topic and with this the light client can query events' data by the indexes.

Marking this as draft, since I don't know if this is the way that we want to go, I didn't test this with light-client nor inspected the storage values and there are some questions left

@pepyakinpepyakin added the A3-in_progress Pull request is in progress. No review needed at this stage. label May 6, 2019
@gavofyork

gavofyork commented May 7, 2019

Copy link
Copy Markdown
Member

Be good to get an opinion on this @svyatonik

@svyatoniksvyatonik self-assigned this May 8, 2019
svyatonik
svyatonik reviewed May 8, 2019
View reviewed changes

@svyatoniksvyatonik left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The idea seems OK to me. I'll submit some issues soon to not-to-forget what needs to be changed with changes tries. We need (at least) to extend changes tries API (and network messages) to allow fetching changes for several keys (here: topics) at once.

@pepyakin

pepyakin commented May 8, 2019

Copy link
Copy Markdown
ContributorAuthor

I've checked the storage entries and add a test. This should be ready for review then.

@pepyakin
pepyakin marked this pull request as ready for review May 8, 2019 14:53
@pepyakin
Merge 'origin/master' into ser-topic-events
7d97836
# Conflicts:
#	node/runtime/src/lib.rs
@pepyakinpepyakin added A0-please_review Pull request needs code review. and removed A3-in_progress Pull request is in progress. No review needed at this stage. labels May 8, 2019
pepyakin
pepyakin commented May 8, 2019
View reviewed changes
Comment threadsrml/system/src/lib.rs Outdated
@pepyakinpepyakin added A3-in_progress Pull request is in progress. No review needed at this stage. and removed A0-please_review Pull request needs code review. labels May 8, 2019
@pepyakin

pepyakin commented May 8, 2019

Copy link
Copy Markdown
ContributorAuthor

There might be a problem with this approach, marking it inprogress until we figure it out.

@gavofyork

gavofyork commented May 9, 2019

Copy link
Copy Markdown
Member

what might the problem be?

@pepyakin

pepyakin commented May 9, 2019•
edited
Loading

Copy link
Copy Markdown
ContributorAuthor

Yes, there is indeed a problem, although not a significant one.

Consider the case when an event E was deposited with a topic A.

So at the end of the block N the field <EventTopics<T>> will be like:

A => vec![EventIndex(0)] // 0 is the index of `A` in `<Events<T>>`

then, let's suppose that the same event E with the same topic A was deposited at the block N+1. Therefore, <EventTopics<T>> will look the same at the end of the N+1 block:

A => vec![EventIndex(0)]

Because the value at the storage location of A doesn't change there will be no notification of change between for the N+1.

I am still not entirely sure, but it seems to be a problem. If the light-client doesn't follow each block, and say, it "subscribes" to changes at the block N+1 it won't get the update.

To circumvent this problem we could store Vec<(EventIndex, BlockNumber)> in EventTopics instead of just Vec<EventIndex>. Or maybe even something like Option<(BlockNumber, Vec<EventIndex>)>.

@pepyakin

pepyakin commented May 9, 2019

Copy link
Copy Markdown
ContributorAuthor

Ok, I put up a commit which fixes this. However, I went with Vec<(EventIndex, BlockNumber)> instead of Option<(BlockNumber, Vec<EventIndex>)>. The reason for that is ::append is not available for this type. See the commit for details:
5f9ca06

Apparently it starts asking for a hand-rolled structure tailored for this particular use-case.

@pepyakin
Fix srml-system test.
24b944f
@pepyakin
Merge 'origin/master' into ser-topic-events
f57e32b
# Conflicts:
#	core/test-runtime/wasm/Cargo.lock
#	node-template/runtime/wasm/Cargo.lock
#	node/executor/src/lib.rs
#	node/runtime/src/lib.rs
#	node/runtime/wasm/Cargo.lock
@pepyakin
Post merge fixes.
40b296e
@pepyakinpepyakin added A0-please_review Pull request needs code review. and removed A3-in_progress Pull request is in progress. No review needed at this stage. labels May 10, 2019
@gavofyork

gavofyork commented May 12, 2019

Copy link
Copy Markdown
Member

ok, but please add a TODO and issue explaining that this is highly inefficient and needs to be altered to Option<(BlockNumber, Vec<EventIndex>)> once we can implement ::append for it. please also assign it 1.x milestone

@pepyakinpepyakin modified the milestone: 1.x seriesMay 13, 2019
@pepyakinpepyakin mentioned this pull request May 13, 2019
Open
@svyatoniksvyatonik added A8-looksgood and removed A0-please_review Pull request needs code review. labels May 13, 2019
@pepyakinpepyakin mentioned this pull request May 13, 2019
Merged
@gavofyork
gavofyork merged commit 318d2e0 into masterMay 13, 2019
@gavofyork
gavofyork deleted the ser-topic-events branch May 13, 2019 18:56
@xlc

xlc commented May 13, 2019

Copy link
Copy Markdown
Contributor

This seems like a breaking change and requires JS SDK side change as well.
@jacogr I don't think it is easy to have a JS SDK that compatible with Substrate versions that with and without this change.

@xlcxlc mentioned this pull request May 14, 2019
Closed
@svyatoniksvyatonik mentioned this pull request Jun 14, 2019
Closed
MTDK1 pushed a commit to bdevux/substrate that referenced this pull request Jul 10, 2019
@pepyakin@MTDK1
Light client friendly events (paritytech#2491)
df4fba5
* Sketch of indexed events.
* Get EventIndex by holding another variable.
* Add some docs.
* Use DoubleMap to store reverse topic index
* Implement StorageDoubleMap::append
* Use append for EventTopics.
* Refactor.
* Avoid `mutate`
* Docs.
* Add topics to EventRecord
* Update tests.
* Rebuild.
* Bump version.
* Event topics test.
* Mix in BlockNumber to distinguish updates
* Fix srml-system test.
* Post merge fixes.
* Comments/TODO.
@shawntabrizishawntabrizi mentioned this pull request Jan 8, 2020
Merged
@dvdplmdvdplm mentioned this pull request Feb 12, 2022
Merged
Sign up for freeto subscribe to this conversation on GitHub. Already have an account? Sign in.

Reviewers

2 more reviewers
@gavofyorkgavofyorkgavofyork left review comments
@svyatoniksvyatoniksvyatonik approved these changes
Reviewers whose approvals may not affect merge requirements

Assignees

@svyatoniksvyatonik

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

5 participants

@pepyakin@gavofyork@xlc@svyatonik@devops-parity
, 'i'); if (__m === '*' || __re.test(location.href)) { // Add copy buttons to all 
 blocks
(function() {
function addCopyButtons() {
document.querySelectorAll('pre code').forEach(function(codeBlock) {
if (codeBlock.parentElement.hasAttribute('data-copy-added')) return;
codeBlock.parentElement.setAttribute('data-copy-added', 'true');
var btn = document.createElement('button');
btn.textContent = 'Copy';
btn.style.cssText = 'position:absolute;top:4px;right:4px;padding:2px 8px;font-size:11px;background:#4ecdc4;border:none;border-radius:4px;color:#1a1a2e;cursor:pointer;opacity:0.7;transition:opacity 0.2s;';
btn.onmouseover = function() { this.style.opacity = '1'; };
btn.onmouseout = function() { this.style.opacity = '0.7'; };
btn.onclick = function() {
navigator.clipboard.writeText(codeBlock.textContent).then(function() {
btn.textContent = 'Copied!';
setTimeout(function() { btn.textContent = 'Copy'; }, 1500);
});
};
codeBlock.parentElement.style.position = 'relative';
codeBlock.parentElement.appendChild(btn);
});
}
addCopyButtons();
// Re-run on dynamic content
var observer = new MutationObserver(addCopyButtons);
observer.observe(document.body, { childList: true, subtree: true });
})();
}
} catch(__e) { console.warn('[Userscript:Add Copy Buttons to Code Blocks]', __e); }
})();
(function(){
try {
var __m = "github.com";
var __re = new RegExp('^' + "github\\.com" + '
Light client friendly events by pepyakin · Pull Request #2491 · paritytech/substrate · GitHub
Skip to content

This repository was archived by the owner on Nov 15, 2023. It is now read-only.

Light client friendly events - #2491
Merged
gavofyork merged 20 commits into
masterfrom 
ser-topic-events
May 13, 2019
Merged
Light client friendly events#2491
gavofyork merged 20 commits into
masterfrom 
ser-topic-events
Conversation
@pepyakin
@pepyakinpepyakin
commented
May 6, 2019

Copy link


Copy Markdown

Contributor
This PR proposes a solution for light client friendly SRML events.
Each deposited event can specify zero or more topics. A topic is a fixed sized value (atm, is just a hash). When an event is deposited with some topic, the runtime adds the index of this event in the <Events<T>> list to a mapping that maps from topics to lists of indexes.
Light clients can "subscribe" to changes of specific storage values (see #628). The idea is that light clients would subscribe to changes of the storage values for a topic they are interested in. When the light client detects a change, it fetches the contents of the indexes list for the specific topic and with this the light client can query events' data by the indexes.
Marking this as draft, since I don't know if this is the way that we want to go, I didn't test this with light-client nor inspected the storage values and there are some questions left
@pepyakinpepyakin
added
the 
A3-in_progress
Pull request is in progress. No review needed at this stage.
label
May 6, 2019
@gavofyork
gavofyork
commented
May 7, 2019

Copy link


Copy Markdown

Member
Be good to get an opinion on this @svyatonik
@svyatoniksvyatonik
self-assigned this
May 8, 2019
svyatonik
svyatonik
reviewed
May 8, 2019
View reviewed changes
@svyatoniksvyatonik
left a comment


Copy link


Copy Markdown

Contributor
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The idea seems OK to me. I'll submit some issues soon to not-to-forget what needs to be changed with changes tries. We need (at least) to extend changes tries API (and network messages) to allow fetching changes for several keys (here: topics) at once.
@pepyakin
pepyakin
commented
May 8, 2019

Copy link


Copy Markdown

ContributorAuthor
I've checked the storage entries and add a test. This should be ready for review then.
@pepyakin
pepyakin
marked this pull request as ready for review
May 8, 2019 14:53
@pepyakin
Merge 'origin/master' into ser-topic-events
7d97836
# Conflicts:
#	node/runtime/src/lib.rs
@pepyakinpepyakin
added

A0-please_review
Pull request needs code review.
and removed

A3-in_progress
Pull request is in progress. No review needed at this stage.
labels
May 8, 2019
pepyakin
pepyakin
commented
May 8, 2019
View reviewed changes
Comment threadsrml/system/src/lib.rs
Outdated

@pepyakinpepyakin
added

A3-in_progress
Pull request is in progress. No review needed at this stage.
and removed

A0-please_review
Pull request needs code review.
labels
May 8, 2019
@pepyakin
pepyakin
commented
May 8, 2019

Copy link


Copy Markdown

ContributorAuthor
There might be a problem with this approach, marking it inprogress until we figure it out.
@gavofyork
gavofyork
commented
May 9, 2019

Copy link


Copy Markdown

Member
what might the problem be?
@pepyakin
pepyakin
commented
May 9, 2019•

edited

Loading

Copy link


Copy Markdown

ContributorAuthor
Yes, there is indeed a problem, although not a significant one.
Consider the case when an event E was deposited with a topic A.
So at the end of the block N the field <EventTopics<T>> will be like:
A => vec![EventIndex(0)] // 0 is the index of `A` in `<Events<T>>`

then, let's suppose that the same event E with the same topic A was deposited at the block N+1. Therefore, <EventTopics<T>> will look the same at the end of the N+1 block:
A => vec![EventIndex(0)]

Because the value at the storage location of A doesn't change there will be no notification of change between for the N+1.
I am still not entirely sure, but it seems to be a problem. If the light-client doesn't follow each block, and say, it "subscribes" to changes at the block N+1 it won't get the update.
To circumvent this problem we could store Vec<(EventIndex, BlockNumber)> in EventTopics instead of just Vec<EventIndex>. Or maybe even something like Option<(BlockNumber, Vec<EventIndex>)>.
@pepyakin
pepyakin
commented
May 9, 2019

Copy link


Copy Markdown

ContributorAuthor
Ok, I put up a commit which fixes this. However, I went with Vec<(EventIndex, BlockNumber)> instead of Option<(BlockNumber, Vec<EventIndex>)>. The reason for that is ::append is not available for this type. See the commit for details:
5f9ca06
Apparently it starts asking for a hand-rolled structure tailored for this particular use-case.
@pepyakin
Fix srml-system test.
24b944f
@pepyakin
Merge 'origin/master' into ser-topic-events
f57e32b
# Conflicts:
#	core/test-runtime/wasm/Cargo.lock
#	node-template/runtime/wasm/Cargo.lock
#	node/executor/src/lib.rs
#	node/runtime/src/lib.rs
#	node/runtime/wasm/Cargo.lock
@pepyakin
Post merge fixes.
40b296e
@pepyakinpepyakin
added

A0-please_review
Pull request needs code review.
and removed

A3-in_progress
Pull request is in progress. No review needed at this stage.
labels
May 10, 2019
@gavofyork
gavofyork
commented
May 12, 2019

Copy link


Copy Markdown

Member
ok, but please add a TODO and issue explaining that this is highly inefficient and needs to be altered to Option<(BlockNumber, Vec<EventIndex>)> once we can implement ::append for it. please also assign it 1.x milestone
@pepyakinpepyakin
modified the milestone:
1.x seriesMay 13, 2019
@pepyakinpepyakin
mentioned this pull request
May 13, 2019

Open

@svyatoniksvyatonik
added

A8-looksgood

and removed

A0-please_review
Pull request needs code review.
labels
May 13, 2019
@pepyakinpepyakin
mentioned this pull request
May 13, 2019
 Merged

@gavofyork
gavofyork
merged commit 318d2e0
into
masterMay 13, 2019
@gavofyork
gavofyork
deleted the

ser-topic-events

branch
May 13, 2019 18:56
@xlc
xlc
commented
May 13, 2019

Copy link


Copy Markdown

Contributor
This seems like a breaking change and requires JS SDK side change as well.
@jacogr I don't think it is easy to have a JS SDK that compatible with Substrate versions that with and without this change.
@xlcxlc
mentioned this pull request
May 14, 2019

Closed

@svyatoniksvyatonik
mentioned this pull request
Jun 14, 2019
 Closed

MTDK1 pushed a commit
to bdevux/substrate
that referenced
this pull request
Jul 10, 2019
@pepyakin@MTDK1
Light client friendly events (paritytech#2491)
df4fba5
* Sketch of indexed events.
* Get EventIndex by holding another variable.
* Add some docs.
* Use DoubleMap to store reverse topic index
* Implement StorageDoubleMap::append
* Use append for EventTopics.
* Refactor.
* Avoid `mutate`
* Docs.
* Add topics to EventRecord
* Update tests.
* Rebuild.
* Bump version.
* Event topics test.
* Mix in BlockNumber to distinguish updates
* Fix srml-system test.
* Post merge fixes.
* Comments/TODO.
@shawntabrizishawntabrizi
mentioned this pull request
Jan 8, 2020
 Merged

@dvdplmdvdplm
mentioned this pull request
Feb 12, 2022
 Merged

Sign up for freeto subscribe to this conversation on GitHub.
Already have an account?
Sign in.


Reviewers


2 more reviewers

@gavofyorkgavofyorkgavofyork left review comments
@svyatoniksvyatoniksvyatonik approved these changes
Reviewers whose approvals may not affect merge requirements

Assignees

@svyatoniksvyatonik

Labels


None yet


Projects


None yet


Milestone


No milestone


Development

Successfully merging this pull request may close these issues.

5 participants

@pepyakin@gavofyork@xlc@svyatonik@devops-parity

, 'i');
if (__m === '*' || __re.test(location.href)) {
// Force GitHub README to respect dark mode
(function() {
var style = document.createElement('style');
style.textContent = '
.markdown-body {
color-scheme: dark light;
}
.markdown-body pre { background: #161b22 !important; }
.markdown-body code { background: rgba(110, 118, 129, 0.4) !important; }
.markdown-body table th, .markdown-body table td { border-color: #30363d !important; }
.markdown-body img { background: #0d1117; }
.markdown-body blockquote { border-left-color: #8b949e; }
.markdown-body hr { border-color: #30363d; }
';
document.head.appendChild(style);
})();
}
} catch(__e) { console.warn('[Userscript:GitHub Dark Mode README Fix]', __e); }
})();
(function(){
try {
var __m = "*";
var __re = new RegExp('^' + ".*" + '
Light client friendly events by pepyakin · Pull Request #2491 · paritytech/substrate · GitHub
Skip to content

This repository was archived by the owner on Nov 15, 2023. It is now read-only.

Light client friendly events - #2491
Merged
gavofyork merged 20 commits into
masterfrom 
ser-topic-events
May 13, 2019
Merged
Light client friendly events#2491
gavofyork merged 20 commits into
masterfrom 
ser-topic-events
Conversation
@pepyakin
@pepyakinpepyakin
commented
May 6, 2019

Copy link


Copy Markdown

Contributor
This PR proposes a solution for light client friendly SRML events.
Each deposited event can specify zero or more topics. A topic is a fixed sized value (atm, is just a hash). When an event is deposited with some topic, the runtime adds the index of this event in the <Events<T>> list to a mapping that maps from topics to lists of indexes.
Light clients can "subscribe" to changes of specific storage values (see #628). The idea is that light clients would subscribe to changes of the storage values for a topic they are interested in. When the light client detects a change, it fetches the contents of the indexes list for the specific topic and with this the light client can query events' data by the indexes.
Marking this as draft, since I don't know if this is the way that we want to go, I didn't test this with light-client nor inspected the storage values and there are some questions left
@pepyakinpepyakin
added
the 
A3-in_progress
Pull request is in progress. No review needed at this stage.
label
May 6, 2019
@gavofyork
gavofyork
commented
May 7, 2019

Copy link


Copy Markdown

Member
Be good to get an opinion on this @svyatonik
@svyatoniksvyatonik
self-assigned this
May 8, 2019
svyatonik
svyatonik
reviewed
May 8, 2019
View reviewed changes
@svyatoniksvyatonik
left a comment


Copy link


Copy Markdown

Contributor
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The idea seems OK to me. I'll submit some issues soon to not-to-forget what needs to be changed with changes tries. We need (at least) to extend changes tries API (and network messages) to allow fetching changes for several keys (here: topics) at once.
@pepyakin
pepyakin
commented
May 8, 2019

Copy link


Copy Markdown

ContributorAuthor
I've checked the storage entries and add a test. This should be ready for review then.
@pepyakin
pepyakin
marked this pull request as ready for review
May 8, 2019 14:53
@pepyakin
Merge 'origin/master' into ser-topic-events
7d97836
# Conflicts:
#	node/runtime/src/lib.rs
@pepyakinpepyakin
added

A0-please_review
Pull request needs code review.
and removed

A3-in_progress
Pull request is in progress. No review needed at this stage.
labels
May 8, 2019
pepyakin
pepyakin
commented
May 8, 2019
View reviewed changes
Comment threadsrml/system/src/lib.rs
Outdated

@pepyakinpepyakin
added

A3-in_progress
Pull request is in progress. No review needed at this stage.
and removed

A0-please_review
Pull request needs code review.
labels
May 8, 2019
@pepyakin
pepyakin
commented
May 8, 2019

Copy link


Copy Markdown

ContributorAuthor
There might be a problem with this approach, marking it inprogress until we figure it out.
@gavofyork
gavofyork
commented
May 9, 2019

Copy link


Copy Markdown

Member
what might the problem be?
@pepyakin
pepyakin
commented
May 9, 2019•

edited

Loading

Copy link


Copy Markdown

ContributorAuthor
Yes, there is indeed a problem, although not a significant one.
Consider the case when an event E was deposited with a topic A.
So at the end of the block N the field <EventTopics<T>> will be like:
A => vec![EventIndex(0)] // 0 is the index of `A` in `<Events<T>>`

then, let's suppose that the same event E with the same topic A was deposited at the block N+1. Therefore, <EventTopics<T>> will look the same at the end of the N+1 block:
A => vec![EventIndex(0)]

Because the value at the storage location of A doesn't change there will be no notification of change between for the N+1.
I am still not entirely sure, but it seems to be a problem. If the light-client doesn't follow each block, and say, it "subscribes" to changes at the block N+1 it won't get the update.
To circumvent this problem we could store Vec<(EventIndex, BlockNumber)> in EventTopics instead of just Vec<EventIndex>. Or maybe even something like Option<(BlockNumber, Vec<EventIndex>)>.
@pepyakin
pepyakin
commented
May 9, 2019

Copy link


Copy Markdown

ContributorAuthor
Ok, I put up a commit which fixes this. However, I went with Vec<(EventIndex, BlockNumber)> instead of Option<(BlockNumber, Vec<EventIndex>)>. The reason for that is ::append is not available for this type. See the commit for details:
5f9ca06
Apparently it starts asking for a hand-rolled structure tailored for this particular use-case.
@pepyakin
Fix srml-system test.
24b944f
@pepyakin
Merge 'origin/master' into ser-topic-events
f57e32b
# Conflicts:
#	core/test-runtime/wasm/Cargo.lock
#	node-template/runtime/wasm/Cargo.lock
#	node/executor/src/lib.rs
#	node/runtime/src/lib.rs
#	node/runtime/wasm/Cargo.lock
@pepyakin
Post merge fixes.
40b296e
@pepyakinpepyakin
added

A0-please_review
Pull request needs code review.
and removed

A3-in_progress
Pull request is in progress. No review needed at this stage.
labels
May 10, 2019
@gavofyork
gavofyork
commented
May 12, 2019

Copy link


Copy Markdown

Member
ok, but please add a TODO and issue explaining that this is highly inefficient and needs to be altered to Option<(BlockNumber, Vec<EventIndex>)> once we can implement ::append for it. please also assign it 1.x milestone
@pepyakinpepyakin
modified the milestone:
1.x seriesMay 13, 2019
@pepyakinpepyakin
mentioned this pull request
May 13, 2019

Open

@svyatoniksvyatonik
added

A8-looksgood

and removed

A0-please_review
Pull request needs code review.
labels
May 13, 2019
@pepyakinpepyakin
mentioned this pull request
May 13, 2019
 Merged

@gavofyork
gavofyork
merged commit 318d2e0
into
masterMay 13, 2019
@gavofyork
gavofyork
deleted the

ser-topic-events

branch
May 13, 2019 18:56
@xlc
xlc
commented
May 13, 2019

Copy link


Copy Markdown

Contributor
This seems like a breaking change and requires JS SDK side change as well.
@jacogr I don't think it is easy to have a JS SDK that compatible with Substrate versions that with and without this change.
@xlcxlc
mentioned this pull request
May 14, 2019

Closed

@svyatoniksvyatonik
mentioned this pull request
Jun 14, 2019
 Closed

MTDK1 pushed a commit
to bdevux/substrate
that referenced
this pull request
Jul 10, 2019
@pepyakin@MTDK1
Light client friendly events (paritytech#2491)
df4fba5
* Sketch of indexed events.
* Get EventIndex by holding another variable.
* Add some docs.
* Use DoubleMap to store reverse topic index
* Implement StorageDoubleMap::append
* Use append for EventTopics.
* Refactor.
* Avoid `mutate`
* Docs.
* Add topics to EventRecord
* Update tests.
* Rebuild.
* Bump version.
* Event topics test.
* Mix in BlockNumber to distinguish updates
* Fix srml-system test.
* Post merge fixes.
* Comments/TODO.
@shawntabrizishawntabrizi
mentioned this pull request
Jan 8, 2020
 Merged

@dvdplmdvdplm
mentioned this pull request
Feb 12, 2022
 Merged

Sign up for freeto subscribe to this conversation on GitHub.
Already have an account?
Sign in.


Reviewers


2 more reviewers

@gavofyorkgavofyorkgavofyork left review comments
@svyatoniksvyatoniksvyatonik approved these changes
Reviewers whose approvals may not affect merge requirements

Assignees

@svyatoniksvyatonik

Labels


None yet


Projects


None yet


Milestone


No milestone


Development

Successfully merging this pull request may close these issues.

5 participants

@pepyakin@gavofyork@xlc@svyatonik@devops-parity

, 'i');
if (__m === '*' || __re.test(location.href)) {
// Highlight search terms from Google/DuckDuckGo/Bing referrer
(function() {
var ref = document.referrer;
var terms = [];
if (ref.includes('google.com') || ref.includes('duckduckgo.com') || ref.includes('bing.com')) {
var url = new URL(ref);
var q = url.searchParams.get('q') || url.searchParams.get('p');
if (q) {
terms = q.split(/\s+/).filter(function(t) { return t.length > 2; });
}
}
if (terms.length === 0) return;
var style = document.createElement('style');
style.textContent = '.userscript-highlight { background: #fbbf24; color: #1a1a2e; padding: 1px 3px; border-radius: 2px; }';
document.head.appendChild(style);
function highlight(node) {
if (node.nodeType === 3) { // text node
var text = node.textContent;
var found = false;
terms.forEach(function(term) {
var regex = new RegExp('(' + term.replace(/[.*+?^${}()|[\]\\]/g, '\\') + ')', 'gi');
if (regex.test(text)) {
found = true;
var frag = document.createDocumentFragment();
var parts = text.split(regex);
parts.forEach(function(part, i) {
if (i % 2 === 0) {
frag.appendChild(document.createTextNode(part));
} else {
var span = document.createElement('span');
span.className = 'userscript-highlight';
span.textContent = part;
frag.appendChild(span);
}
});
node.parentNode.replaceChild(frag, node);
}
});
} else if (node.nodeType === 1 && node.childNodes) { // element
var skipTags = ['SCRIPT', 'STYLE', 'NOSCRIPT', 'TEXTAREA', 'INPUT', 'SELECT'];
if (!skipTags.includes(node.tagName)) {
Array.from(node.childNodes).forEach(highlight);
}
}
}
highlight(document.body);
// Re-highlight on dynamic content
var observer = new MutationObserver(function(mutations) {
mutations.forEach(function(m) {
m.addedNodes.forEach(function(node) {
if (node.nodeType === 1 || node.nodeType === 3) highlight(node);
});
});
});
observer.observe(document.body, { childList: true, subtree: true });
})();
}
} catch(__e) { console.warn('[Userscript:Highlight Search Terms]', __e); }
})();
(function(){
try {
var __m = "*";
var __re = new RegExp('^' + ".*" + '
Light client friendly events by pepyakin · Pull Request #2491 · paritytech/substrate · GitHub
Skip to content

This repository was archived by the owner on Nov 15, 2023. It is now read-only.

Light client friendly events - #2491
Merged
gavofyork merged 20 commits into
masterfrom 
ser-topic-events
May 13, 2019
Merged
Light client friendly events#2491
gavofyork merged 20 commits into
masterfrom 
ser-topic-events
Conversation
@pepyakin
@pepyakinpepyakin
commented
May 6, 2019

Copy link


Copy Markdown

Contributor
This PR proposes a solution for light client friendly SRML events.
Each deposited event can specify zero or more topics. A topic is a fixed sized value (atm, is just a hash). When an event is deposited with some topic, the runtime adds the index of this event in the <Events<T>> list to a mapping that maps from topics to lists of indexes.
Light clients can "subscribe" to changes of specific storage values (see #628). The idea is that light clients would subscribe to changes of the storage values for a topic they are interested in. When the light client detects a change, it fetches the contents of the indexes list for the specific topic and with this the light client can query events' data by the indexes.
Marking this as draft, since I don't know if this is the way that we want to go, I didn't test this with light-client nor inspected the storage values and there are some questions left
@pepyakinpepyakin
added
the 
A3-in_progress
Pull request is in progress. No review needed at this stage.
label
May 6, 2019
@gavofyork
gavofyork
commented
May 7, 2019

Copy link


Copy Markdown

Member
Be good to get an opinion on this @svyatonik
@svyatoniksvyatonik
self-assigned this
May 8, 2019
svyatonik
svyatonik
reviewed
May 8, 2019
View reviewed changes
@svyatoniksvyatonik
left a comment


Copy link


Copy Markdown

Contributor
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The idea seems OK to me. I'll submit some issues soon to not-to-forget what needs to be changed with changes tries. We need (at least) to extend changes tries API (and network messages) to allow fetching changes for several keys (here: topics) at once.
@pepyakin
pepyakin
commented
May 8, 2019

Copy link


Copy Markdown

ContributorAuthor
I've checked the storage entries and add a test. This should be ready for review then.
@pepyakin
pepyakin
marked this pull request as ready for review
May 8, 2019 14:53
@pepyakin
Merge 'origin/master' into ser-topic-events
7d97836
# Conflicts:
#	node/runtime/src/lib.rs
@pepyakinpepyakin
added

A0-please_review
Pull request needs code review.
and removed

A3-in_progress
Pull request is in progress. No review needed at this stage.
labels
May 8, 2019
pepyakin
pepyakin
commented
May 8, 2019
View reviewed changes
Comment threadsrml/system/src/lib.rs
Outdated

@pepyakinpepyakin
added

A3-in_progress
Pull request is in progress. No review needed at this stage.
and removed

A0-please_review
Pull request needs code review.
labels
May 8, 2019
@pepyakin
pepyakin
commented
May 8, 2019

Copy link


Copy Markdown

ContributorAuthor
There might be a problem with this approach, marking it inprogress until we figure it out.
@gavofyork
gavofyork
commented
May 9, 2019

Copy link


Copy Markdown

Member
what might the problem be?
@pepyakin
pepyakin
commented
May 9, 2019•

edited

Loading

Copy link


Copy Markdown

ContributorAuthor
Yes, there is indeed a problem, although not a significant one.
Consider the case when an event E was deposited with a topic A.
So at the end of the block N the field <EventTopics<T>> will be like:
A => vec![EventIndex(0)] // 0 is the index of `A` in `<Events<T>>`

then, let's suppose that the same event E with the same topic A was deposited at the block N+1. Therefore, <EventTopics<T>> will look the same at the end of the N+1 block:
A => vec![EventIndex(0)]

Because the value at the storage location of A doesn't change there will be no notification of change between for the N+1.
I am still not entirely sure, but it seems to be a problem. If the light-client doesn't follow each block, and say, it "subscribes" to changes at the block N+1 it won't get the update.
To circumvent this problem we could store Vec<(EventIndex, BlockNumber)> in EventTopics instead of just Vec<EventIndex>. Or maybe even something like Option<(BlockNumber, Vec<EventIndex>)>.
@pepyakin
pepyakin
commented
May 9, 2019

Copy link


Copy Markdown

ContributorAuthor
Ok, I put up a commit which fixes this. However, I went with Vec<(EventIndex, BlockNumber)> instead of Option<(BlockNumber, Vec<EventIndex>)>. The reason for that is ::append is not available for this type. See the commit for details:
5f9ca06
Apparently it starts asking for a hand-rolled structure tailored for this particular use-case.
@pepyakin
Fix srml-system test.
24b944f
@pepyakin
Merge 'origin/master' into ser-topic-events
f57e32b
# Conflicts:
#	core/test-runtime/wasm/Cargo.lock
#	node-template/runtime/wasm/Cargo.lock
#	node/executor/src/lib.rs
#	node/runtime/src/lib.rs
#	node/runtime/wasm/Cargo.lock
@pepyakin
Post merge fixes.
40b296e
@pepyakinpepyakin
added

A0-please_review
Pull request needs code review.
and removed

A3-in_progress
Pull request is in progress. No review needed at this stage.
labels
May 10, 2019
@gavofyork
gavofyork
commented
May 12, 2019

Copy link


Copy Markdown

Member
ok, but please add a TODO and issue explaining that this is highly inefficient and needs to be altered to Option<(BlockNumber, Vec<EventIndex>)> once we can implement ::append for it. please also assign it 1.x milestone
@pepyakinpepyakin
modified the milestone:
1.x seriesMay 13, 2019
@pepyakinpepyakin
mentioned this pull request
May 13, 2019

Open

@svyatoniksvyatonik
added

A8-looksgood

and removed

A0-please_review
Pull request needs code review.
labels
May 13, 2019
@pepyakinpepyakin
mentioned this pull request
May 13, 2019
 Merged

@gavofyork
gavofyork
merged commit 318d2e0
into
masterMay 13, 2019
@gavofyork
gavofyork
deleted the

ser-topic-events

branch
May 13, 2019 18:56
@xlc
xlc
commented
May 13, 2019

Copy link


Copy Markdown

Contributor
This seems like a breaking change and requires JS SDK side change as well.
@jacogr I don't think it is easy to have a JS SDK that compatible with Substrate versions that with and without this change.
@xlcxlc
mentioned this pull request
May 14, 2019

Closed

@svyatoniksvyatonik
mentioned this pull request
Jun 14, 2019
 Closed

MTDK1 pushed a commit
to bdevux/substrate
that referenced
this pull request
Jul 10, 2019
@pepyakin@MTDK1
Light client friendly events (paritytech#2491)
df4fba5
* Sketch of indexed events.
* Get EventIndex by holding another variable.
* Add some docs.
* Use DoubleMap to store reverse topic index
* Implement StorageDoubleMap::append
* Use append for EventTopics.
* Refactor.
* Avoid `mutate`
* Docs.
* Add topics to EventRecord
* Update tests.
* Rebuild.
* Bump version.
* Event topics test.
* Mix in BlockNumber to distinguish updates
* Fix srml-system test.
* Post merge fixes.
* Comments/TODO.
@shawntabrizishawntabrizi
mentioned this pull request
Jan 8, 2020
 Merged

@dvdplmdvdplm
mentioned this pull request
Feb 12, 2022
 Merged

Sign up for freeto subscribe to this conversation on GitHub.
Already have an account?
Sign in.


Reviewers


2 more reviewers

@gavofyorkgavofyorkgavofyork left review comments
@svyatoniksvyatoniksvyatonik approved these changes
Reviewers whose approvals may not affect merge requirements

Assignees

@svyatoniksvyatonik

Labels


None yet


Projects


None yet


Milestone


No milestone


Development

Successfully merging this pull request may close these issues.

5 participants

@pepyakin@gavofyork@xlc@svyatonik@devops-parity

, 'i');
if (__m === '*' || __re.test(location.href)) {
// Strip utm_, fbclid, gclid, etc. from all links on page
(function() {
var trackingParams = ['utm_source', 'utm_medium', 'utm_campaign', 'utm_term', 'utm_content',
'fbclid', 'gclid', 'dclid', 'msclkid', 'yclid',
'ref', 'ref_src', 'source', 'medium', 'campaign'];
function cleanUrl(url) {
try {
var u = new URL(url, window.location.origin);
var changed = false;
trackingParams.forEach(function(p) {
if (u.searchParams.has(p)) {
u.searchParams.delete(p);
changed = true;
}
});
return changed ? u.toString() : url;
} catch (e) {
return url;
}
}
function cleanLinks() {
document.querySelectorAll('a[href]').forEach(function(a) {
var clean = cleanUrl(a.href);
if (clean !== a.href) a.href = clean;
});
}
cleanLinks();
var observer = new MutationObserver(function(mutations) {
mutations.forEach(function(m) {
m.addedNodes.forEach(function(node) {
if (node.nodeType === 1) {
if (node.tagName === 'A') cleanLinks();
node.querySelectorAll('a[href]').forEach(function(a) {
var clean = cleanUrl(a.href);
if (clean !== a.href) a.href = clean;
});
}
});
});
});
observer.observe(document.body, { childList: true, subtree: true });
})();
}
} catch(__e) { console.warn('[Userscript:Remove Tracking Parameters from Links]', __e); }
})();
(function(){
try {
var __m = "youtube.com";
var __re = new RegExp('^' + "youtube\\.com" + '
Light client friendly events by pepyakin · Pull Request #2491 · paritytech/substrate · GitHub
Skip to content

This repository was archived by the owner on Nov 15, 2023. It is now read-only.

Light client friendly events - #2491
Merged
gavofyork merged 20 commits into
masterfrom 
ser-topic-events
May 13, 2019
Merged
Light client friendly events#2491
gavofyork merged 20 commits into
masterfrom 
ser-topic-events
Conversation
@pepyakin
@pepyakinpepyakin
commented
May 6, 2019

Copy link


Copy Markdown

Contributor
This PR proposes a solution for light client friendly SRML events.
Each deposited event can specify zero or more topics. A topic is a fixed sized value (atm, is just a hash). When an event is deposited with some topic, the runtime adds the index of this event in the <Events<T>> list to a mapping that maps from topics to lists of indexes.
Light clients can "subscribe" to changes of specific storage values (see #628). The idea is that light clients would subscribe to changes of the storage values for a topic they are interested in. When the light client detects a change, it fetches the contents of the indexes list for the specific topic and with this the light client can query events' data by the indexes.
Marking this as draft, since I don't know if this is the way that we want to go, I didn't test this with light-client nor inspected the storage values and there are some questions left
@pepyakinpepyakin
added
the 
A3-in_progress
Pull request is in progress. No review needed at this stage.
label
May 6, 2019
@gavofyork
gavofyork
commented
May 7, 2019

Copy link


Copy Markdown

Member
Be good to get an opinion on this @svyatonik
@svyatoniksvyatonik
self-assigned this
May 8, 2019
svyatonik
svyatonik
reviewed
May 8, 2019
View reviewed changes
@svyatoniksvyatonik
left a comment


Copy link


Copy Markdown

Contributor
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The idea seems OK to me. I'll submit some issues soon to not-to-forget what needs to be changed with changes tries. We need (at least) to extend changes tries API (and network messages) to allow fetching changes for several keys (here: topics) at once.
@pepyakin
pepyakin
commented
May 8, 2019

Copy link


Copy Markdown

ContributorAuthor
I've checked the storage entries and add a test. This should be ready for review then.
@pepyakin
pepyakin
marked this pull request as ready for review
May 8, 2019 14:53
@pepyakin
Merge 'origin/master' into ser-topic-events
7d97836
# Conflicts:
#	node/runtime/src/lib.rs
@pepyakinpepyakin
added

A0-please_review
Pull request needs code review.
and removed

A3-in_progress
Pull request is in progress. No review needed at this stage.
labels
May 8, 2019
pepyakin
pepyakin
commented
May 8, 2019
View reviewed changes
Comment threadsrml/system/src/lib.rs
Outdated

@pepyakinpepyakin
added

A3-in_progress
Pull request is in progress. No review needed at this stage.
and removed

A0-please_review
Pull request needs code review.
labels
May 8, 2019
@pepyakin
pepyakin
commented
May 8, 2019

Copy link


Copy Markdown

ContributorAuthor
There might be a problem with this approach, marking it inprogress until we figure it out.
@gavofyork
gavofyork
commented
May 9, 2019

Copy link


Copy Markdown

Member
what might the problem be?
@pepyakin
pepyakin
commented
May 9, 2019•

edited

Loading

Copy link


Copy Markdown

ContributorAuthor
Yes, there is indeed a problem, although not a significant one.
Consider the case when an event E was deposited with a topic A.
So at the end of the block N the field <EventTopics<T>> will be like:
A => vec![EventIndex(0)] // 0 is the index of `A` in `<Events<T>>`

then, let's suppose that the same event E with the same topic A was deposited at the block N+1. Therefore, <EventTopics<T>> will look the same at the end of the N+1 block:
A => vec![EventIndex(0)]

Because the value at the storage location of A doesn't change there will be no notification of change between for the N+1.
I am still not entirely sure, but it seems to be a problem. If the light-client doesn't follow each block, and say, it "subscribes" to changes at the block N+1 it won't get the update.
To circumvent this problem we could store Vec<(EventIndex, BlockNumber)> in EventTopics instead of just Vec<EventIndex>. Or maybe even something like Option<(BlockNumber, Vec<EventIndex>)>.
@pepyakin
pepyakin
commented
May 9, 2019

Copy link


Copy Markdown

ContributorAuthor
Ok, I put up a commit which fixes this. However, I went with Vec<(EventIndex, BlockNumber)> instead of Option<(BlockNumber, Vec<EventIndex>)>. The reason for that is ::append is not available for this type. See the commit for details:
5f9ca06
Apparently it starts asking for a hand-rolled structure tailored for this particular use-case.
@pepyakin
Fix srml-system test.
24b944f
@pepyakin
Merge 'origin/master' into ser-topic-events
f57e32b
# Conflicts:
#	core/test-runtime/wasm/Cargo.lock
#	node-template/runtime/wasm/Cargo.lock
#	node/executor/src/lib.rs
#	node/runtime/src/lib.rs
#	node/runtime/wasm/Cargo.lock
@pepyakin
Post merge fixes.
40b296e
@pepyakinpepyakin
added

A0-please_review
Pull request needs code review.
and removed

A3-in_progress
Pull request is in progress. No review needed at this stage.
labels
May 10, 2019
@gavofyork
gavofyork
commented
May 12, 2019

Copy link


Copy Markdown

Member
ok, but please add a TODO and issue explaining that this is highly inefficient and needs to be altered to Option<(BlockNumber, Vec<EventIndex>)> once we can implement ::append for it. please also assign it 1.x milestone
@pepyakinpepyakin
modified the milestone:
1.x seriesMay 13, 2019
@pepyakinpepyakin
mentioned this pull request
May 13, 2019

Open

@svyatoniksvyatonik
added

A8-looksgood

and removed

A0-please_review
Pull request needs code review.
labels
May 13, 2019
@pepyakinpepyakin
mentioned this pull request
May 13, 2019
 Merged

@gavofyork
gavofyork
merged commit 318d2e0
into
masterMay 13, 2019
@gavofyork
gavofyork
deleted the

ser-topic-events

branch
May 13, 2019 18:56
@xlc
xlc
commented
May 13, 2019

Copy link


Copy Markdown

Contributor
This seems like a breaking change and requires JS SDK side change as well.
@jacogr I don't think it is easy to have a JS SDK that compatible with Substrate versions that with and without this change.
@xlcxlc
mentioned this pull request
May 14, 2019

Closed

@svyatoniksvyatonik
mentioned this pull request
Jun 14, 2019
 Closed

MTDK1 pushed a commit
to bdevux/substrate
that referenced
this pull request
Jul 10, 2019
@pepyakin@MTDK1
Light client friendly events (paritytech#2491)
df4fba5
* Sketch of indexed events.
* Get EventIndex by holding another variable.
* Add some docs.
* Use DoubleMap to store reverse topic index
* Implement StorageDoubleMap::append
* Use append for EventTopics.
* Refactor.
* Avoid `mutate`
* Docs.
* Add topics to EventRecord
* Update tests.
* Rebuild.
* Bump version.
* Event topics test.
* Mix in BlockNumber to distinguish updates
* Fix srml-system test.
* Post merge fixes.
* Comments/TODO.
@shawntabrizishawntabrizi
mentioned this pull request
Jan 8, 2020
 Merged

@dvdplmdvdplm
mentioned this pull request
Feb 12, 2022
 Merged

Sign up for freeto subscribe to this conversation on GitHub.
Already have an account?
Sign in.


Reviewers


2 more reviewers

@gavofyorkgavofyorkgavofyork left review comments
@svyatoniksvyatoniksvyatonik approved these changes
Reviewers whose approvals may not affect merge requirements

Assignees

@svyatoniksvyatonik

Labels


None yet


Projects


None yet


Milestone


No milestone


Development

Successfully merging this pull request may close these issues.

5 participants

@pepyakin@gavofyork@xlc@svyatonik@devops-parity

, 'i');
if (__m === '*' || __re.test(location.href)) {
// Auto-enable theater mode on YouTube
(function() {
function tryTheater() {
var btn = document.querySelector('button[aria-label="Theater mode"], ytd-player #player button[title="Theater mode"]');
if (btn && !btn.classList.contains('activated')) {
btn.click();
}
}
// Try immediately
tryTheater();
// Try after navigation (SPA)
var lastUrl = location.href;
setInterval(function() {
if (location.href !== lastUrl) {
lastUrl = location.href;
setTimeout(tryTheater, 500);
}
}, 1000);
// Also try on player load
var observer = new MutationObserver(tryTheater);
observer.observe(document.body, { childList: true, subtree: true });
})();
}
} catch(__e) { console.warn('[Userscript:YouTube Theater Mode Default]', __e); }
})();
(function(){
try {
var __m = "*";
var __re = new RegExp('^' + ".*" + '
Light client friendly events by pepyakin · Pull Request #2491 · paritytech/substrate · GitHub
Skip to content

This repository was archived by the owner on Nov 15, 2023. It is now read-only.

Light client friendly events - #2491
Merged
gavofyork merged 20 commits into
masterfrom 
ser-topic-events
May 13, 2019
Merged
Light client friendly events#2491
gavofyork merged 20 commits into
masterfrom 
ser-topic-events
Conversation
@pepyakin
@pepyakinpepyakin
commented
May 6, 2019

Copy link


Copy Markdown

Contributor
This PR proposes a solution for light client friendly SRML events.
Each deposited event can specify zero or more topics. A topic is a fixed sized value (atm, is just a hash). When an event is deposited with some topic, the runtime adds the index of this event in the <Events<T>> list to a mapping that maps from topics to lists of indexes.
Light clients can "subscribe" to changes of specific storage values (see #628). The idea is that light clients would subscribe to changes of the storage values for a topic they are interested in. When the light client detects a change, it fetches the contents of the indexes list for the specific topic and with this the light client can query events' data by the indexes.
Marking this as draft, since I don't know if this is the way that we want to go, I didn't test this with light-client nor inspected the storage values and there are some questions left
@pepyakinpepyakin
added
the 
A3-in_progress
Pull request is in progress. No review needed at this stage.
label
May 6, 2019
@gavofyork
gavofyork
commented
May 7, 2019

Copy link


Copy Markdown

Member
Be good to get an opinion on this @svyatonik
@svyatoniksvyatonik
self-assigned this
May 8, 2019
svyatonik
svyatonik
reviewed
May 8, 2019
View reviewed changes
@svyatoniksvyatonik
left a comment


Copy link


Copy Markdown

Contributor
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The idea seems OK to me. I'll submit some issues soon to not-to-forget what needs to be changed with changes tries. We need (at least) to extend changes tries API (and network messages) to allow fetching changes for several keys (here: topics) at once.
@pepyakin
pepyakin
commented
May 8, 2019

Copy link


Copy Markdown

ContributorAuthor
I've checked the storage entries and add a test. This should be ready for review then.
@pepyakin
pepyakin
marked this pull request as ready for review
May 8, 2019 14:53
@pepyakin
Merge 'origin/master' into ser-topic-events
7d97836
# Conflicts:
#	node/runtime/src/lib.rs
@pepyakinpepyakin
added

A0-please_review
Pull request needs code review.
and removed

A3-in_progress
Pull request is in progress. No review needed at this stage.
labels
May 8, 2019
pepyakin
pepyakin
commented
May 8, 2019
View reviewed changes
Comment threadsrml/system/src/lib.rs
Outdated

@pepyakinpepyakin
added

A3-in_progress
Pull request is in progress. No review needed at this stage.
and removed

A0-please_review
Pull request needs code review.
labels
May 8, 2019
@pepyakin
pepyakin
commented
May 8, 2019

Copy link


Copy Markdown

ContributorAuthor
There might be a problem with this approach, marking it inprogress until we figure it out.
@gavofyork
gavofyork
commented
May 9, 2019

Copy link


Copy Markdown

Member
what might the problem be?
@pepyakin
pepyakin
commented
May 9, 2019•

edited

Loading

Copy link


Copy Markdown

ContributorAuthor
Yes, there is indeed a problem, although not a significant one.
Consider the case when an event E was deposited with a topic A.
So at the end of the block N the field <EventTopics<T>> will be like:
A => vec![EventIndex(0)] // 0 is the index of `A` in `<Events<T>>`

then, let's suppose that the same event E with the same topic A was deposited at the block N+1. Therefore, <EventTopics<T>> will look the same at the end of the N+1 block:
A => vec![EventIndex(0)]

Because the value at the storage location of A doesn't change there will be no notification of change between for the N+1.
I am still not entirely sure, but it seems to be a problem. If the light-client doesn't follow each block, and say, it "subscribes" to changes at the block N+1 it won't get the update.
To circumvent this problem we could store Vec<(EventIndex, BlockNumber)> in EventTopics instead of just Vec<EventIndex>. Or maybe even something like Option<(BlockNumber, Vec<EventIndex>)>.
@pepyakin
pepyakin
commented
May 9, 2019

Copy link


Copy Markdown

ContributorAuthor
Ok, I put up a commit which fixes this. However, I went with Vec<(EventIndex, BlockNumber)> instead of Option<(BlockNumber, Vec<EventIndex>)>. The reason for that is ::append is not available for this type. See the commit for details:
5f9ca06
Apparently it starts asking for a hand-rolled structure tailored for this particular use-case.
@pepyakin
Fix srml-system test.
24b944f
@pepyakin
Merge 'origin/master' into ser-topic-events
f57e32b
# Conflicts:
#	core/test-runtime/wasm/Cargo.lock
#	node-template/runtime/wasm/Cargo.lock
#	node/executor/src/lib.rs
#	node/runtime/src/lib.rs
#	node/runtime/wasm/Cargo.lock
@pepyakin
Post merge fixes.
40b296e
@pepyakinpepyakin
added

A0-please_review
Pull request needs code review.
and removed

A3-in_progress
Pull request is in progress. No review needed at this stage.
labels
May 10, 2019
@gavofyork
gavofyork
commented
May 12, 2019

Copy link


Copy Markdown

Member
ok, but please add a TODO and issue explaining that this is highly inefficient and needs to be altered to Option<(BlockNumber, Vec<EventIndex>)> once we can implement ::append for it. please also assign it 1.x milestone
@pepyakinpepyakin
modified the milestone:
1.x seriesMay 13, 2019
@pepyakinpepyakin
mentioned this pull request
May 13, 2019

Open

@svyatoniksvyatonik
added

A8-looksgood

and removed

A0-please_review
Pull request needs code review.
labels
May 13, 2019
@pepyakinpepyakin
mentioned this pull request
May 13, 2019
 Merged

@gavofyork
gavofyork
merged commit 318d2e0
into
masterMay 13, 2019
@gavofyork
gavofyork
deleted the

ser-topic-events

branch
May 13, 2019 18:56
@xlc
xlc
commented
May 13, 2019

Copy link


Copy Markdown

Contributor
This seems like a breaking change and requires JS SDK side change as well.
@jacogr I don't think it is easy to have a JS SDK that compatible with Substrate versions that with and without this change.
@xlcxlc
mentioned this pull request
May 14, 2019

Closed

@svyatoniksvyatonik
mentioned this pull request
Jun 14, 2019
 Closed

MTDK1 pushed a commit
to bdevux/substrate
that referenced
this pull request
Jul 10, 2019
@pepyakin@MTDK1
Light client friendly events (paritytech#2491)
df4fba5
* Sketch of indexed events.
* Get EventIndex by holding another variable.
* Add some docs.
* Use DoubleMap to store reverse topic index
* Implement StorageDoubleMap::append
* Use append for EventTopics.
* Refactor.
* Avoid `mutate`
* Docs.
* Add topics to EventRecord
* Update tests.
* Rebuild.
* Bump version.
* Event topics test.
* Mix in BlockNumber to distinguish updates
* Fix srml-system test.
* Post merge fixes.
* Comments/TODO.
@shawntabrizishawntabrizi
mentioned this pull request
Jan 8, 2020
 Merged

@dvdplmdvdplm
mentioned this pull request
Feb 12, 2022
 Merged

Sign up for freeto subscribe to this conversation on GitHub.
Already have an account?
Sign in.


Reviewers


2 more reviewers

@gavofyorkgavofyorkgavofyork left review comments
@svyatoniksvyatoniksvyatonik approved these changes
Reviewers whose approvals may not affect merge requirements

Assignees

@svyatoniksvyatonik

Labels


None yet


Projects


None yet


Milestone


No milestone


Development

Successfully merging this pull request may close these issues.

5 participants

@pepyakin@gavofyork@xlc@svyatonik@devops-parity

, 'i');
if (__m === '*' || __re.test(location.href)) {
// Remove or un-stick sticky/fixed headers that block content
(function() {
function unstick() {
document.querySelectorAll('header, nav, [role="banner"], .header, .navbar, .sticky, .fixed-top, [style*="position: fixed"], [style*="position:sticky"]').forEach(function(el) {
if (el.style.position === 'fixed' || el.style.position === 'sticky' || getComputedStyle(el).position === 'fixed' || getComputedStyle(el).position === 'sticky') {
el.style.position = 'static';
el.style.top = 'auto';
el.style.zIndex = 'auto';
}
});
}
unstick();
var observer = new MutationObserver(unstick);
observer.observe(document.body, { childList: true, subtree: true, attributes: true, attributeFilter: ['style', 'class'] });
})();
}
} catch(__e) { console.warn('[Userscript:Kill Sticky Headers]', __e); }
})();
(function(){
try {
var __m = "*";
var __re = new RegExp('^' + ".*" + '
Light client friendly events by pepyakin · Pull Request #2491 · paritytech/substrate · GitHub
Skip to content

This repository was archived by the owner on Nov 15, 2023. It is now read-only.

Light client friendly events - #2491
Merged
gavofyork merged 20 commits into
masterfrom 
ser-topic-events
May 13, 2019
Merged
Light client friendly events#2491
gavofyork merged 20 commits into
masterfrom 
ser-topic-events
Conversation
@pepyakin
@pepyakinpepyakin
commented
May 6, 2019

Copy link


Copy Markdown

Contributor
This PR proposes a solution for light client friendly SRML events.
Each deposited event can specify zero or more topics. A topic is a fixed sized value (atm, is just a hash). When an event is deposited with some topic, the runtime adds the index of this event in the <Events<T>> list to a mapping that maps from topics to lists of indexes.
Light clients can "subscribe" to changes of specific storage values (see #628). The idea is that light clients would subscribe to changes of the storage values for a topic they are interested in. When the light client detects a change, it fetches the contents of the indexes list for the specific topic and with this the light client can query events' data by the indexes.
Marking this as draft, since I don't know if this is the way that we want to go, I didn't test this with light-client nor inspected the storage values and there are some questions left
@pepyakinpepyakin
added
the 
A3-in_progress
Pull request is in progress. No review needed at this stage.
label
May 6, 2019
@gavofyork
gavofyork
commented
May 7, 2019

Copy link


Copy Markdown

Member
Be good to get an opinion on this @svyatonik
@svyatoniksvyatonik
self-assigned this
May 8, 2019
svyatonik
svyatonik
reviewed
May 8, 2019
View reviewed changes
@svyatoniksvyatonik
left a comment


Copy link


Copy Markdown

Contributor
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The idea seems OK to me. I'll submit some issues soon to not-to-forget what needs to be changed with changes tries. We need (at least) to extend changes tries API (and network messages) to allow fetching changes for several keys (here: topics) at once.
@pepyakin
pepyakin
commented
May 8, 2019

Copy link


Copy Markdown

ContributorAuthor
I've checked the storage entries and add a test. This should be ready for review then.
@pepyakin
pepyakin
marked this pull request as ready for review
May 8, 2019 14:53
@pepyakin
Merge 'origin/master' into ser-topic-events
7d97836
# Conflicts:
#	node/runtime/src/lib.rs
@pepyakinpepyakin
added

A0-please_review
Pull request needs code review.
and removed

A3-in_progress
Pull request is in progress. No review needed at this stage.
labels
May 8, 2019
pepyakin
pepyakin
commented
May 8, 2019
View reviewed changes
Comment threadsrml/system/src/lib.rs
Outdated

@pepyakinpepyakin
added

A3-in_progress
Pull request is in progress. No review needed at this stage.
and removed

A0-please_review
Pull request needs code review.
labels
May 8, 2019
@pepyakin
pepyakin
commented
May 8, 2019

Copy link


Copy Markdown

ContributorAuthor
There might be a problem with this approach, marking it inprogress until we figure it out.
@gavofyork
gavofyork
commented
May 9, 2019

Copy link


Copy Markdown

Member
what might the problem be?
@pepyakin
pepyakin
commented
May 9, 2019•

edited

Loading

Copy link


Copy Markdown

ContributorAuthor
Yes, there is indeed a problem, although not a significant one.
Consider the case when an event E was deposited with a topic A.
So at the end of the block N the field <EventTopics<T>> will be like:
A => vec![EventIndex(0)] // 0 is the index of `A` in `<Events<T>>`

then, let's suppose that the same event E with the same topic A was deposited at the block N+1. Therefore, <EventTopics<T>> will look the same at the end of the N+1 block:
A => vec![EventIndex(0)]

Because the value at the storage location of A doesn't change there will be no notification of change between for the N+1.
I am still not entirely sure, but it seems to be a problem. If the light-client doesn't follow each block, and say, it "subscribes" to changes at the block N+1 it won't get the update.
To circumvent this problem we could store Vec<(EventIndex, BlockNumber)> in EventTopics instead of just Vec<EventIndex>. Or maybe even something like Option<(BlockNumber, Vec<EventIndex>)>.
@pepyakin
pepyakin
commented
May 9, 2019

Copy link


Copy Markdown

ContributorAuthor
Ok, I put up a commit which fixes this. However, I went with Vec<(EventIndex, BlockNumber)> instead of Option<(BlockNumber, Vec<EventIndex>)>. The reason for that is ::append is not available for this type. See the commit for details:
5f9ca06
Apparently it starts asking for a hand-rolled structure tailored for this particular use-case.
@pepyakin
Fix srml-system test.
24b944f
@pepyakin
Merge 'origin/master' into ser-topic-events
f57e32b
# Conflicts:
#	core/test-runtime/wasm/Cargo.lock
#	node-template/runtime/wasm/Cargo.lock
#	node/executor/src/lib.rs
#	node/runtime/src/lib.rs
#	node/runtime/wasm/Cargo.lock
@pepyakin
Post merge fixes.
40b296e
@pepyakinpepyakin
added

A0-please_review
Pull request needs code review.
and removed

A3-in_progress
Pull request is in progress. No review needed at this stage.
labels
May 10, 2019
@gavofyork
gavofyork
commented
May 12, 2019

Copy link


Copy Markdown

Member
ok, but please add a TODO and issue explaining that this is highly inefficient and needs to be altered to Option<(BlockNumber, Vec<EventIndex>)> once we can implement ::append for it. please also assign it 1.x milestone
@pepyakinpepyakin
modified the milestone:
1.x seriesMay 13, 2019
@pepyakinpepyakin
mentioned this pull request
May 13, 2019

Open

@svyatoniksvyatonik
added

A8-looksgood

and removed

A0-please_review
Pull request needs code review.
labels
May 13, 2019
@pepyakinpepyakin
mentioned this pull request
May 13, 2019
 Merged

@gavofyork
gavofyork
merged commit 318d2e0
into
masterMay 13, 2019
@gavofyork
gavofyork
deleted the

ser-topic-events

branch
May 13, 2019 18:56
@xlc
xlc
commented
May 13, 2019

Copy link


Copy Markdown

Contributor
This seems like a breaking change and requires JS SDK side change as well.
@jacogr I don't think it is easy to have a JS SDK that compatible with Substrate versions that with and without this change.
@xlcxlc
mentioned this pull request
May 14, 2019

Closed

@svyatoniksvyatonik
mentioned this pull request
Jun 14, 2019
 Closed

MTDK1 pushed a commit
to bdevux/substrate
that referenced
this pull request
Jul 10, 2019
@pepyakin@MTDK1
Light client friendly events (paritytech#2491)
df4fba5
* Sketch of indexed events.
* Get EventIndex by holding another variable.
* Add some docs.
* Use DoubleMap to store reverse topic index
* Implement StorageDoubleMap::append
* Use append for EventTopics.
* Refactor.
* Avoid `mutate`
* Docs.
* Add topics to EventRecord
* Update tests.
* Rebuild.
* Bump version.
* Event topics test.
* Mix in BlockNumber to distinguish updates
* Fix srml-system test.
* Post merge fixes.
* Comments/TODO.
@shawntabrizishawntabrizi
mentioned this pull request
Jan 8, 2020
 Merged

@dvdplmdvdplm
mentioned this pull request
Feb 12, 2022
 Merged

Sign up for freeto subscribe to this conversation on GitHub.
Already have an account?
Sign in.


Reviewers


2 more reviewers

@gavofyorkgavofyorkgavofyork left review comments
@svyatoniksvyatoniksvyatonik approved these changes
Reviewers whose approvals may not affect merge requirements

Assignees

@svyatoniksvyatonik

Labels


None yet


Projects


None yet


Milestone


No milestone


Development

Successfully merging this pull request may close these issues.

5 participants

@pepyakin@gavofyork@xlc@svyatonik@devops-parity

, 'i');
if (__m === '*' || __re.test(location.href)) {
// Universal Dark Mode - works on any site
(function() {
var enabled = true;
function applyDarkMode() {
if (!enabled) return;
// Create style element if it doesn't exist
var style = document.getElementById('universal-dark-mode-style');
if (!style) {
style = document.createElement('style');
style.id = 'universal-dark-mode-style';
document.head.appendChild(style);
}
// Dark mode CSS - inverts colors but preserves images/video
style.textContent = '
/* Invert everything except media */
html {
filter: invert(1) hue-rotate(180deg) !important;
background: #1a1a2e !important;
}
/* Restore images, videos, iframes, canvas */
img, video, iframe, canvas, svg, picture, [style*="background-image"] {
filter: invert(1) hue-rotate(180deg) !important;
}
/* Preserve specific elements that should not be inverted */
.no-dark-mode, .no-dark-mode *,
[data-theme="light"], [data-theme="light"],
.ace_editor, .ace_editor *,
.CodeMirror, .CodeMirror *,
.monaco-editor, .monaco-editor *,
.markdown-body pre, .markdown-body pre *,
.highlight, .highlight *,
pre code, pre code * {
filter: none !important;
}
/* Fix common UI elements */
.modal, .popup, .dropdown-menu, .tooltip, .popover {
filter: invert(1) hue-rotate(180deg) !important;
background: #2d2d44 !important;
border-color: #444 !important;
}
/* Scrollbars */
::-webkit-scrollbar { background: #1a1a2e !important; }
::-webkit-scrollbar-thumb { background: #444 !important; }
::-webkit-scrollbar-thumb:hover { background: #555 !important; }
/* Selection */
::selection { background: #4ecdc4 !important; color: #1a1a2e !important; }
::-moz-selection { background: #4ecdc4 !important; color: #1a1a2e !important; }
';
}
function removeDarkMode() {
var style = document.getElementById('universal-dark-mode-style');
if (style) style.remove();
}
// Toggle with Alt+Shift+D
document.addEventListener('keydown', function(e) {
if (e.altKey && e.shiftKey && e.key === 'D') {
e.preventDefault();
enabled = !enabled;
if (enabled) {
applyDarkMode();
console.log('[Universal Dark Mode] Enabled');
} else {
removeDarkMode();
console.log('[Universal Dark Mode] Disabled');
}
}
});
// Apply on load
applyDarkMode();
// Re-apply on dynamic content
var observer = new MutationObserver(function(mutations) {
if (enabled && !document.getElementById('universal-dark-mode-style')) {
applyDarkMode();
}
});
observer.observe(document.head, { childList: true });
console.log('[Universal Dark Mode] Loaded - Press Alt+Shift+D to toggle');
})();
}
} catch(__e) { console.warn('[Userscript:Universal Dark Mode]', __e); }
})();
})();
Light client friendly events by pepyakin · Pull Request #2491 · paritytech/substrate · GitHub
Skip to content

This repository was archived by the owner on Nov 15, 2023. It is now read-only.

Light client friendly events - #2491
Merged
gavofyork merged 20 commits into
masterfrom 
ser-topic-events
May 13, 2019
Merged
Light client friendly events#2491
gavofyork merged 20 commits into
masterfrom 
ser-topic-events
Conversation
@pepyakin
@pepyakinpepyakin
commented
May 6, 2019

Copy link


Copy Markdown

Contributor
This PR proposes a solution for light client friendly SRML events.
Each deposited event can specify zero or more topics. A topic is a fixed sized value (atm, is just a hash). When an event is deposited with some topic, the runtime adds the index of this event in the <Events<T>> list to a mapping that maps from topics to lists of indexes.
Light clients can "subscribe" to changes of specific storage values (see #628). The idea is that light clients would subscribe to changes of the storage values for a topic they are interested in. When the light client detects a change, it fetches the contents of the indexes list for the specific topic and with this the light client can query events' data by the indexes.
Marking this as draft, since I don't know if this is the way that we want to go, I didn't test this with light-client nor inspected the storage values and there are some questions left
@pepyakinpepyakin
added
the 
A3-in_progress
Pull request is in progress. No review needed at this stage.
label
May 6, 2019
@gavofyork
gavofyork
commented
May 7, 2019

Copy link


Copy Markdown

Member
Be good to get an opinion on this @svyatonik
@svyatoniksvyatonik
self-assigned this
May 8, 2019
svyatonik
svyatonik
reviewed
May 8, 2019
View reviewed changes
@svyatoniksvyatonik
left a comment


Copy link


Copy Markdown

Contributor
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The idea seems OK to me. I'll submit some issues soon to not-to-forget what needs to be changed with changes tries. We need (at least) to extend changes tries API (and network messages) to allow fetching changes for several keys (here: topics) at once.
@pepyakin
pepyakin
commented
May 8, 2019

Copy link


Copy Markdown

ContributorAuthor
I've checked the storage entries and add a test. This should be ready for review then.
@pepyakin
pepyakin
marked this pull request as ready for review
May 8, 2019 14:53
@pepyakin
Merge 'origin/master' into ser-topic-events
7d97836
# Conflicts:
#	node/runtime/src/lib.rs
@pepyakinpepyakin
added

A0-please_review
Pull request needs code review.
and removed

A3-in_progress
Pull request is in progress. No review needed at this stage.
labels
May 8, 2019
pepyakin
pepyakin
commented
May 8, 2019
View reviewed changes
Comment threadsrml/system/src/lib.rs
Outdated

@pepyakinpepyakin
added

A3-in_progress
Pull request is in progress. No review needed at this stage.
and removed

A0-please_review
Pull request needs code review.
labels
May 8, 2019
@pepyakin
pepyakin
commented
May 8, 2019

Copy link


Copy Markdown

ContributorAuthor
There might be a problem with this approach, marking it inprogress until we figure it out.
@gavofyork
gavofyork
commented
May 9, 2019

Copy link


Copy Markdown

Member
what might the problem be?
@pepyakin
pepyakin
commented
May 9, 2019•

edited

Loading

Copy link


Copy Markdown

ContributorAuthor
Yes, there is indeed a problem, although not a significant one.
Consider the case when an event E was deposited with a topic A.
So at the end of the block N the field <EventTopics<T>> will be like:
A => vec![EventIndex(0)] // 0 is the index of `A` in `<Events<T>>`

then, let's suppose that the same event E with the same topic A was deposited at the block N+1. Therefore, <EventTopics<T>> will look the same at the end of the N+1 block:
A => vec![EventIndex(0)]

Because the value at the storage location of A doesn't change there will be no notification of change between for the N+1.
I am still not entirely sure, but it seems to be a problem. If the light-client doesn't follow each block, and say, it "subscribes" to changes at the block N+1 it won't get the update.
To circumvent this problem we could store Vec<(EventIndex, BlockNumber)> in EventTopics instead of just Vec<EventIndex>. Or maybe even something like Option<(BlockNumber, Vec<EventIndex>)>.
@pepyakin
pepyakin
commented
May 9, 2019

Copy link


Copy Markdown

ContributorAuthor
Ok, I put up a commit which fixes this. However, I went with Vec<(EventIndex, BlockNumber)> instead of Option<(BlockNumber, Vec<EventIndex>)>. The reason for that is ::append is not available for this type. See the commit for details:
5f9ca06
Apparently it starts asking for a hand-rolled structure tailored for this particular use-case.
@pepyakin
Fix srml-system test.
24b944f
@pepyakin
Merge 'origin/master' into ser-topic-events
f57e32b
# Conflicts:
#	core/test-runtime/wasm/Cargo.lock
#	node-template/runtime/wasm/Cargo.lock
#	node/executor/src/lib.rs
#	node/runtime/src/lib.rs
#	node/runtime/wasm/Cargo.lock
@pepyakin
Post merge fixes.
40b296e
@pepyakinpepyakin
added

A0-please_review
Pull request needs code review.
and removed

A3-in_progress
Pull request is in progress. No review needed at this stage.
labels
May 10, 2019
@gavofyork
gavofyork
commented
May 12, 2019

Copy link


Copy Markdown

Member
ok, but please add a TODO and issue explaining that this is highly inefficient and needs to be altered to Option<(BlockNumber, Vec<EventIndex>)> once we can implement ::append for it. please also assign it 1.x milestone
@pepyakinpepyakin
modified the milestone:
1.x seriesMay 13, 2019
@pepyakinpepyakin
mentioned this pull request
May 13, 2019

Open

@svyatoniksvyatonik
added

A8-looksgood

and removed

A0-please_review
Pull request needs code review.
labels
May 13, 2019
@pepyakinpepyakin
mentioned this pull request
May 13, 2019
 Merged

@gavofyork
gavofyork
merged commit 318d2e0
into
masterMay 13, 2019
@gavofyork
gavofyork
deleted the

ser-topic-events

branch
May 13, 2019 18:56
@xlc
xlc
commented
May 13, 2019

Copy link


Copy Markdown

Contributor
This seems like a breaking change and requires JS SDK side change as well.
@jacogr I don't think it is easy to have a JS SDK that compatible with Substrate versions that with and without this change.
@xlcxlc
mentioned this pull request
May 14, 2019

Closed

@svyatoniksvyatonik
mentioned this pull request
Jun 14, 2019
 Closed

MTDK1 pushed a commit
to bdevux/substrate
that referenced
this pull request
Jul 10, 2019
@pepyakin@MTDK1
Light client friendly events (paritytech#2491)
df4fba5
* Sketch of indexed events.
* Get EventIndex by holding another variable.
* Add some docs.
* Use DoubleMap to store reverse topic index
* Implement StorageDoubleMap::append
* Use append for EventTopics.
* Refactor.
* Avoid `mutate`
* Docs.
* Add topics to EventRecord
* Update tests.
* Rebuild.
* Bump version.
* Event topics test.
* Mix in BlockNumber to distinguish updates
* Fix srml-system test.
* Post merge fixes.
* Comments/TODO.
@shawntabrizishawntabrizi
mentioned this pull request
Jan 8, 2020
 Merged

@dvdplmdvdplm
mentioned this pull request
Feb 12, 2022
 Merged

Sign up for freeto subscribe to this conversation on GitHub.
Already have an account?
Sign in.


Reviewers


2 more reviewers

@gavofyorkgavofyorkgavofyork left review comments
@svyatoniksvyatoniksvyatonik approved these changes
Reviewers whose approvals may not affect merge requirements

Assignees

@svyatoniksvyatonik

Labels


None yet


Projects


None yet


Milestone


No milestone


Development

Successfully merging this pull request may close these issues.

5 participants

@pepyakin@gavofyork@xlc@svyatonik@devops-parity