Skip to content

refactor: Split configs in server and service - #671

Merged
flub merged 12 commits into
mainfrom
flub/split-configs
Jan 4, 2023
Merged

refactor: Split configs in server and service#671
flub merged 12 commits into
mainfrom
flub/split-configs

Conversation

@flub

@flub flub commented Jan 3, 2023

Copy link
Copy Markdown
Collaborator

We used to use the same config for the service and server (aka the binary). This is confusing when creating configs to use with e.g. iroh-one, iroh-embed or iroh-share because some fields are not used.

This splits off the config structs to avoid this problem, services now have a Config and binaries a ServerConfig. This allows creating the services standalone without all the baggage a server needs.

While this isn't many fields yet, this will get worse as we add more features (this is split off from another PR where this seemed useful).

Todo

  • iroh-store
  • iroh-p2p
  • iroh-gateway

We used to use the same config for the service and server (aka the
binary).  This is confusing when creating configs to use with
e.g. iroh-one, iroh-embed or iroh-share because some fields are not
used.

This splits off the config structs to avoid this problem, services now
have a Config and binaries a ServerConfig.  This allows creating the
services standalone without all the baggage a server needs.

While this isn't many fields yet, this will get worse as we add more
features (this is split off from another PR where this seemed useful).
Comment thread iroh-store/src/config.rs Outdated
Comment on lines +33 to +34
// TODO: I'd prefer to include [`Config`] under the `store` field like iroh-one does. But
// that's a backwards incompatible change.

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@Arqu I guess you're the main person that uses these config files? What are your thoughts here?

@flub flub Jan 3, 2023

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

For the p2p and gateway services this duplication is painful, it would be a lot nicer if we could break backwards compatibility of the config file formats.

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You should break it, there's no real dependency on them yet. Better break them now than later.

path: db_path.to_path_buf(),
rpc_client: rpc_store_client_config,
metrics: iroh_metrics::config::Config {
tracing: false, // disable tracing by default

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is proof that the current state of things is weird. This did nothing IIUC.

@flub
flub marked this pull request as ready for review January 3, 2023 17:52
@flub
flub requested review from Arqu and ramfox January 3, 2023 17:52
Comment thread iroh-gateway/src/config.rs Outdated
/// This is the configuration which the gateway server binary needs to run. This is a
/// superset from the configuration needed by the gateway service, which can also run
/// integrated into another binary like iroh-one, iroh-share or iroh-embed.
// TODO: I'd prefer to include [`Config`] under the `gateway` field liek iroh-one does. But

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think this should be done before merging, to avoid the duplication, don’t be afraid to break apis

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yeah, feel free to break this, there's nobody really depending on it yet.

Arqu
Arqu previously approved these changes Jan 4, 2023

@Arqu Arqu left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

From a prue code perspective this is 👍
However this "works" for the metrics piece more by coincidence than design at this point.
The reason each had their own metrics directly planted was that different services used to run a different feature set of the metrics crate, which would initialize the metrics just for that service.
This kinda nukes that notion completely. But it was not doing that anyways since the Cargo.toml refactors because it's now from a global workspace and I believe it just builds all the features regardless.

Functionally this is fine, it just wastes a little bit more resources on the metrics side. Not a ton though. I'll spend some time thinking about this.

Comment thread iroh-gateway/src/config.rs Outdated
/// This is the configuration which the gateway server binary needs to run. This is a
/// superset from the configuration needed by the gateway service, which can also run
/// integrated into another binary like iroh-one, iroh-share or iroh-embed.
// TODO: I'd prefer to include [`Config`] under the `gateway` field liek iroh-one does. But

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yeah, feel free to break this, there's nobody really depending on it yet.

Comment thread iroh-store/src/config.rs Outdated
Comment on lines +33 to +34
// TODO: I'd prefer to include [`Config`] under the `store` field like iroh-one does. But
// that's a backwards incompatible change.

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You should break it, there's no real dependency on them yet. Better break them now than later.

@flub

flub commented Jan 4, 2023

Copy link
Copy Markdown
Collaborator Author

From a prue code perspective this is +1 However this "works" for the metrics piece more by coincidence than design at this point. The reason each had their own metrics directly planted was that different services used to run a different feature set of the metrics crate, which would initialize the metrics just for that service. This kinda nukes that notion completely.

But it was not doing that anyways since the Cargo.toml refactors because it's now from a global workspace and I believe it just builds all the features regardless.

Hum, this should not be the case. If you look at iroh-store/Cargo.toml you'll find iroh-metrics = { workspace = true, features = ["store"] }. So if you only build the iroh-store crate the only feature built for iroh-metrics is the store feature. If you however also build other crates at the same time I believe the iroh-metrics crate will be built with the combination of all the features requested. This behaves exactly the same as before the workspace.dependencies refactor.

Functionally this is fine, it just wastes a little bit more resources on the metrics side. Not a ton though. I'll spend some time thinking about this.

This adds a StoreConfig section to remove duplication.
@flub

flub commented Jan 4, 2023

Copy link
Copy Markdown
Collaborator Author

After taking the wrong approach to resolve the TODOs and going astray for a while I've now tiedied this up. So the ServerConfig structs use the Config structs directly which is much neater and easier to follow.

PTAL

@flub
flub requested review from Arqu and dignifiedquire January 4, 2023 18:17
@flub
flub merged commit a0db860 into main Jan 4, 2023
@flub
flub deleted the flub/split-configs branch January 4, 2023 19:07
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants