feat!: drop legacy storage by benjlevesque · Pull Request #1117 · RequestNetwork/requestNetwork

benjlevesque · 2023-06-20T22:17:30Z

BREAKING CHANGE: Dropping legacy indexing mechanism.

Changes:

enforce usage of TheGraph as a backend for request-node
drop all custom indexing code (mostly DataAccess)
refactor DataAccess to rely on an Indexer
refactor mock to use in-memory indexer
reduce dependency to TheGraph by abstracting most of the code transformation. TheGraph dependency is now down to only the SubgraphClient.

MantisClone · 2023-06-23T19:14:16Z

I recognize that the legacy indexing mechanism doesn't work on many chains, is difficult to maintain, and that no one wants to spend the time to fix it. For those reasons, I agree that removing it now is the right move.

Nonetheless, would it be possible to leave injection points such that someone could plug in their own custom indexer into the protocol?

benjlevesque · 2023-06-26T07:58:18Z

I recognize that the legacy indexing mechanism doesn't work on many chains, is difficult to maintain, and that no one wants to spend the time to fix it. For those reasons, I agree that removing it now is the right move.
Nonetheless, would it be possible to leave injection points such that someone could plug in their own custom indexer into the protocol?

Yes @MantisClone; that would be best indeed! These injection points are still available, but not as the Node level (same as today). I've introduced a new layer called "indexer" that represents better the way things are (it would actually be possible to keep the old code with this indexer layer, but it's not efficient so better remove it).

This PR is still draft because I'm trying to reduce the surface of TheGraph in the code; it should be down to only the Subgraph Client, graphql queries and types. The rest could be generalised to work with any type of indexer.

benjlevesque · 2023-10-06T09:13:12Z

@@ -1,42 +1,29 @@
-#!/usr/bin/env node


this file now only creates the server object, but doesn't start it. startNode is called from bin.ts

benjlevesque · 2023-10-06T09:13:30Z

@@ -1,36 +1,182 @@
-import { DataAccess, TransactionIndex } from '@requestnetwork/data-access';


requestNodeBase.ts moved to requestNode.ts

benjlevesque · 2023-10-06T09:14:03Z

@@ -0,0 +1,194 @@
+import { DataAccessTypes, StorageTypes } from '@requestnetwork/types';


moved from packages/thegraph-data-access/src/data-access.ts

benjlevesque · 2023-10-06T09:14:20Z

@@ -0,0 +1,32 @@
+import { DataAccessTypes } from '@requestnetwork/types';


moved from packages/thegraph-data-access/src/data-access.ts

yomarion

Massive!

MantisClone

I'm about 1/4th of the way through the review.

MantisClone

Impressive refactor! Only a few minor comments.

alexandre-abrioux

👏

benjlevesque · 2023-11-14T09:27:31Z

NB: from now on, the IPFS_URL environment variable replaces IPFS_HOST, IPFS_PORT and IPFS_PROTOCOL

benjlevesque changed the base branch from master to refactor/request-node/configJune 20, 2023 22:17

benjlevesque changed the title ~~feat/drop legacy storage~~feat!: drop legacy storageJun 20, 2023

Base automatically changed from refactor/request-node/config to masterJune 23, 2023 08:43

benjlevesque force-pushed the feat/drop-legacy-storage branch from 36b281d to e339f76Compare June 23, 2023 09:10

benjlevesque force-pushed the feat/drop-legacy-storage branch from d34fb57 to 185f6b2Compare June 26, 2023 08:02

benjlevesque force-pushed the feat/drop-legacy-storage branch from 185f6b2 to ca607aeCompare July 12, 2023 10:39

benjlevesque force-pushed the feat/drop-legacy-storage branch 2 times, most recently from f449cb1 to 70119e9Compare July 25, 2023 15:30

benjlevesque added 20 commits September 15, 2023 09:50

feat!: drop legacy storage
f466c57

fix build + split files
9e69da3

fix keys undefined
41abda8

fix inmemoryindexer
12ec9d4

lint
27dad8e

config cleanup
bdcc101

reorganize node/storage
6dffc82

ipfs
f4a4ab6

remove dependency to currency package
81be91e

remove unnecessary dep
5c60984

self review & fixes
88e1bd2

fix transaction-manager
154ad35

tests and fixes
4a8f5ad

remove duplicate tests CI
6197a3f

tiny test adjustments
6391871

fix CI
70d1f52

ci fix
d6e3750

fix pending store
62cae2f

misc fixes
b5226b5

fix keys field
e9276d2

github-actionsBot requested review from MantisClone, alexandre-abrioux, kevindavee and leoslr October 5, 2023 19:56

benjlevesque requested a review from yomarion October 5, 2023 19:59

benjlevesque added 2 commits October 6, 2023 02:34

Merge branch 'master' into feat/drop-legacy-storage
dc28218

Merge branch 'master' into feat/drop-legacy-storage
4231ec2

benjlevesque commented Oct 6, 2023
View reviewed changes

Comment threadpackages/ethereum-storage/src/ethereum-utils.ts

fix version
5557a89

yomarion approved these changes Oct 6, 2023
View reviewed changes

MantisClone requested a review from skiv71 October 6, 2023 20:20

MantisClone reviewed Oct 9, 2023
View reviewed changes

Comment threadREADME.md

MantisClone approved these changes Oct 10, 2023
View reviewed changes

benjlevesque added 2 commits October 11, 2023 10:16

nullish coalescing operator
4b9d881

Merge branch 'master' into feat/drop-legacy-storage
e3ad460

alexandre-abrioux approved these changes Oct 12, 2023
View reviewed changes

benjlevesque mentioned this pull request Oct 12, 2023
Optimize getTransactionsByTopics#1199
Closed

MantisClone assigned benjlevesque Oct 24, 2023

benjlevesque added 2 commits October 25, 2023 18:42

fix test
21f35fe

revert log
ffb2a23

benjlevesque merged commit 9021cf2 into masterOct 25, 2023

benjlevesque deleted the feat/drop-legacy-storage branch October 25, 2023 16:52

This was referenced Oct 26, 2023
Fix the warnings that occur when importing Request Network SDK in a project #1153
Open
Fix Request Node /waitForConfirmation endpoint. It only checks store, should fallback to query subgraph if confirmed transaction not in store#1145
Closed

benjlevesque mentioned this pull request Nov 13, 2023
refactor: remove axios #1237
Merged

		@@ -1,36 +1,182 @@
		import { DataAccess, TransactionIndex } from '@requestnetwork/data-access';

		@@ -0,0 +1,194 @@
		import { DataAccessTypes, StorageTypes } from '@requestnetwork/types';

		@@ -0,0 +1,32 @@
		import { DataAccessTypes } from '@requestnetwork/types';

Uh oh!

Conversation

benjlevesque commented Jun 20, 2023• edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

MantisClone commented Jun 23, 2023

Uh oh!

benjlevesque commented Jun 26, 2023

Uh oh!

benjlevesqueOct 6, 2023

Choose a reason for hiding this comment

Uh oh!

benjlevesqueOct 6, 2023

Choose a reason for hiding this comment

Uh oh!

benjlevesqueOct 6, 2023

Choose a reason for hiding this comment

Uh oh!

benjlevesqueOct 6, 2023

Choose a reason for hiding this comment

Uh oh!

Uh oh!

yomarion left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

MantisClone left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

MantisClone left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

alexandre-abrioux left a comment

Choose a reason for hiding this comment

Uh oh!

benjlevesque commented Nov 14, 2023• edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

5 participants

benjlevesque commented Jun 20, 2023•
edited
Loading

benjlevesque commented Nov 14, 2023•
edited
Loading