Add Public facing APIs to display benchmark metrics and list deployme… by makungaj1 · Pull Request #4589 · aws/sagemaker-python-sdk

makungaj1 · 2024-04-17T04:14:09Z

Issue #, if available:

Description of changes:
Introduce ReadOnly APIs, list deployment configurations and display benchmark metrics. These APIs are used to quickly select and try different configuration that best serve the use case.

Sample output for list_deployment_configs api

[{'ConfigName': 'neuron-inference',
'BenchmarkMetrics': [{'name': 'Latency', 'value': '100', 'unit': 'Tokens/S'},
{'name': 'Throughput', 'value': '100', 'unit': 'Tokens/Second'}],
'DeploymentConfig': {'ModelDataDownloadTimeout': None,
'ContainerStartupHealthCheckTimeout': None,
'ImageUri': '763104351884.dkr.ecr.us-west-2.amazonaws.com/pytorch-inference:1.10.0-gpu-py38',
'ModelData': {'S3DataSource': {'S3Uri': 's3://jumpstart-cache-prod-us-west-2/pytorch-ic/pytorch-ic-mobilenet-v2/artifacts/inference-prepack/v1.0.0/',
'S3DataType': 'S3Prefix',
'CompressionType': 'None'}},
'InstanceType': 'ml.g5.2xlarge',
'Environment': {'SAGEMAKER_PROGRAM': 'inference.py',
'ENDPOINT_SERVER_TIMEOUT': '3600',
'MODEL_CACHE_ROOT': '/opt/ml/model',
'SAGEMAKER_ENV': '1',
'SAGEMAKER_MODEL_SERVER_WORKERS': '1'}}},
{'ConfigName': 'neuron-inference-budget',
'BenchmarkMetrics': [{'name': 'Latency', 'value': '100', 'unit': 'Tokens/S'},
{'name': 'Throughput', 'value': '100', 'unit': 'Tokens/Second'}],
'DeploymentConfig': {'ModelDataDownloadTimeout': None,
'ContainerStartupHealthCheckTimeout': None,
'ImageUri': '763104351884.dkr.ecr.us-west-2.amazonaws.com/pytorch-inference:1.10.0-gpu-py38',
'ModelData': {'S3DataSource': {'S3Uri': 's3://jumpstart-cache-prod-us-west-2/pytorch-ic/pytorch-ic-mobilenet-v2/artifacts/inference-prepack/v1.0.0/',
'S3DataType': 'S3Prefix',
'CompressionType': 'None'}},
'InstanceType': 'ml.g5.2xlarge',
'Environment': {'SAGEMAKER_PROGRAM': 'inference.py',
'ENDPOINT_SERVER_TIMEOUT': '3600',
'MODEL_CACHE_ROOT': '/opt/ml/model',
'SAGEMAKER_ENV': '1',
'SAGEMAKER_MODEL_SERVER_WORKERS': '1'}}},
{'ConfigName': 'gpu-inference-budget',
'BenchmarkMetrics': [{'name': 'Latency', 'value': '100', 'unit': 'Tokens/S'},
{'name': 'Throughput', 'value': '100', 'unit': 'Tokens/Second'}],
'DeploymentConfig': {'ModelDataDownloadTimeout': None,
'ContainerStartupHealthCheckTimeout': None,
'ImageUri': '763104351884.dkr.ecr.us-west-2.amazonaws.com/pytorch-inference:1.10.0-gpu-py38',
'ModelData': {'S3DataSource': {'S3Uri': 's3://jumpstart-cache-prod-us-west-2/pytorch-ic/pytorch-ic-mobilenet-v2/artifacts/inference-prepack/v1.0.0/',
'S3DataType': 'S3Prefix',
'CompressionType': 'None'}},
'InstanceType': 'ml.g5.2xlarge',
'Environment': {'SAGEMAKER_PROGRAM': 'inference.py',
'ENDPOINT_SERVER_TIMEOUT': '3600',
'MODEL_CACHE_ROOT': '/opt/ml/model',
'SAGEMAKER_ENV': '1',
'SAGEMAKER_MODEL_SERVER_WORKERS': '1'}}},
{'ConfigName': 'gpu-inference',
'BenchmarkMetrics': [{'name': 'Latency', 'value': '100', 'unit': 'Tokens/S'},
{'name': 'Throughput', 'value': '100', 'unit': 'Tokens/Second'}],
'DeploymentConfig': {'ModelDataDownloadTimeout': None,
'ContainerStartupHealthCheckTimeout': None,
'ImageUri': '763104351884.dkr.ecr.us-west-2.amazonaws.com/pytorch-inference:1.10.0-gpu-py38',
'ModelData': {'S3DataSource': {'S3Uri': 's3://jumpstart-cache-prod-us-west-2/pytorch-ic/pytorch-ic-mobilenet-v2/artifacts/inference-prepack/v1.0.0/',
'S3DataType': 'S3Prefix',
'CompressionType': 'None'}},
'InstanceType': 'ml.g5.2xlarge',
'Environment': {'SAGEMAKER_PROGRAM': 'inference.py',
'ENDPOINT_SERVER_TIMEOUT': '3600',
'MODEL_CACHE_ROOT': '/opt/ml/model',
'SAGEMAKER_ENV': '1',
'SAGEMAKER_MODEL_SERVER_WORKERS': '1'}}}]

Sample output for display_benchmark_metrics api

| | Config Name | Instance Type | Latency (Tokens/S) | Throughput (Tokens/Second) |
|---:|:------------------------|:----------------|---------------------:|-----------------------------:|
| 0 | neuron-inference | ml.g5.2xlarge | 100 | 100 |
| 1 | neuron-inference-budget | ml.g5.2xlarge | 100 | 100 |
| 2 | gpu-inference-budget | ml.g5.2xlarge | 100 | 100 |
| 3 | gpu-inference | ml.g5.2xlarge | 100 | 100 |

Testing done:

Merge Checklist

Put an x in the boxes that apply. You can also fill these out after creating the PR. If you're unsure about any of them, don't hesitate to ask. We're here to help! This is simply a reminder of what we are going to look for before merging your pull request.

General

I have read the CONTRIBUTING doc
I certify that the changes I am introducing will be backward compatible, and I have discussed concerns about this, if any, with the Python SDK team
I used the commit message format described in CONTRIBUTING
I have passed the region in to all S3 and STS clients that I've initialized as part of this change.
I have updated any necessary documentation, including READMEs and API docs (if appropriate)

Tests

I have added tests that prove my fix is effective or that my feature works (if appropriate)
I have added unit and/or integration tests as appropriate to ensure backward compatibility of the changes
I have checked that my tests are not configured for a specific region or account (if appropriate)
I have used unique_name_from_base to create resource names in integ tests (if appropriate)

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.

…nt configs

qiyunz · 2024-04-17T04:24:50Z

+ df["Instance Rate<br>($/Hour)"] = df["Instance Rate<br>($/Hour)"].apply(
+ lambda rate: "${0:.2f}".format(float(rate))
+ )


The instance rate is not available as part of the benchmark metrics data object, we need to use AWS pricing API to query the rate using current region info.

can this be a fast follow to this PR?

jiapinw · 2024-04-17T18:39:17Z


+ def display_benchmark_metrics(self):
+ """Display Benchmark Metrics for deployment configs."""
+ config_names = []


We should consider making this a data class instead of stand along variables

samruds · 2024-04-17T21:17:03Z

+ df["Instance Rate<br>($/Hour)"] = df["Instance Rate<br>($/Hour)"].apply(
+ lambda rate: "${0:.2f}".format(float(rate))
+ )


can this be a fast follow to this PR?

samruds · 2024-04-17T21:29:52Z

+ instance_rates = []
+ for deployment_config in self.list_deployment_configs():
+ config_names.append(deployment_config.get("ConfigName"))
+ instance_types.append(deployment_config.get("BenchmarkMetrics").key()[0])


Is there a chance that a deployment config does not have a benchark metric? In which case do we need validation here?

samruds · 2024-04-17T21:35:03Z

+ config_names.append(deployment_config.get("ConfigName"))
+ instance_types.append(deployment_config.get("BenchmarkMetrics").key()[0])
+
+ for benchmark_metric in deployment_config.get("BenchmarkMetrics").values():


If any of the fields are potentially missing in the deployment configuration, there is a chance we lose order in lists
DC = { {l=A, t=A1, c=A2}, {l=B t=B1, c=}, {l=C, t=C1, c=C2}}
[A, B, C] -> latency
[A1, B1, C1] -> throughput
[A2, C2] -> cost

qiyunz · 2024-04-18T22:14:42Z

+ self.container_startup_health_check_timeout = (
+ deploy_kwargs.container_startup_health_check_timeout
+ )


why not just assign the value?

samruds · 2024-04-19T19:01:02Z

Current structure looks good. Will tests be added in separate PR?

makungaj1 · 2024-04-19T20:39:09Z

Current structure looks good. Will tests be added in separate PR?

Will add tests in the current PR after #4583 is merged as some low levels APIs, bug fixes are not in the main branch yet.

samruds · 2024-04-19T21:03:48Z

Two follow ups, tentative approval for logic. Not ok to merge to master branch.

Add UT
Integrating with Pricing API for cost.

Unit Tests

qiyunz · 2024-04-22T01:48:28Z

+ df.style.set_caption("Benchmark Metrics").set_table_styles([headers]).set_properties(
+ **{"text-align": "left"}
+ )
+ return df


if the function has return value, do you need to update the method signature?

makungaj1 · 2024-04-17T04:14:09Z

Issue #, if available:

Description of changes:
Introduce ReadOnly APIs, list deployment configurations and display benchmark metrics. These APIs are used to quickly select and try different configuration that best serve the use case.

Sample output for list_deployment_configs api

[{'ConfigName': 'neuron-inference',
'BenchmarkMetrics': [{'name': 'Latency', 'value': '100', 'unit': 'Tokens/S'},
{'name': 'Throughput', 'value': '100', 'unit': 'Tokens/Second'}],
'DeploymentConfig': {'ModelDataDownloadTimeout': None,
'ContainerStartupHealthCheckTimeout': None,
'ImageUri': '763104351884.dkr.ecr.us-west-2.amazonaws.com/pytorch-inference:1.10.0-gpu-py38',
'ModelData': {'S3DataSource': {'S3Uri': 's3://jumpstart-cache-prod-us-west-2/pytorch-ic/pytorch-ic-mobilenet-v2/artifacts/inference-prepack/v1.0.0/',
'S3DataType': 'S3Prefix',
'CompressionType': 'None'}},
'InstanceType': 'ml.g5.2xlarge',
'Environment': {'SAGEMAKER_PROGRAM': 'inference.py',
'ENDPOINT_SERVER_TIMEOUT': '3600',
'MODEL_CACHE_ROOT': '/opt/ml/model',
'SAGEMAKER_ENV': '1',
'SAGEMAKER_MODEL_SERVER_WORKERS': '1'}}},
{'ConfigName': 'neuron-inference-budget',
'BenchmarkMetrics': [{'name': 'Latency', 'value': '100', 'unit': 'Tokens/S'},
{'name': 'Throughput', 'value': '100', 'unit': 'Tokens/Second'}],
'DeploymentConfig': {'ModelDataDownloadTimeout': None,
'ContainerStartupHealthCheckTimeout': None,
'ImageUri': '763104351884.dkr.ecr.us-west-2.amazonaws.com/pytorch-inference:1.10.0-gpu-py38',
'ModelData': {'S3DataSource': {'S3Uri': 's3://jumpstart-cache-prod-us-west-2/pytorch-ic/pytorch-ic-mobilenet-v2/artifacts/inference-prepack/v1.0.0/',
'S3DataType': 'S3Prefix',
'CompressionType': 'None'}},
'InstanceType': 'ml.g5.2xlarge',
'Environment': {'SAGEMAKER_PROGRAM': 'inference.py',
'ENDPOINT_SERVER_TIMEOUT': '3600',
'MODEL_CACHE_ROOT': '/opt/ml/model',
'SAGEMAKER_ENV': '1',
'SAGEMAKER_MODEL_SERVER_WORKERS': '1'}}},
{'ConfigName': 'gpu-inference-budget',
'BenchmarkMetrics': [{'name': 'Latency', 'value': '100', 'unit': 'Tokens/S'},
{'name': 'Throughput', 'value': '100', 'unit': 'Tokens/Second'}],
'DeploymentConfig': {'ModelDataDownloadTimeout': None,
'ContainerStartupHealthCheckTimeout': None,
'ImageUri': '763104351884.dkr.ecr.us-west-2.amazonaws.com/pytorch-inference:1.10.0-gpu-py38',
'ModelData': {'S3DataSource': {'S3Uri': 's3://jumpstart-cache-prod-us-west-2/pytorch-ic/pytorch-ic-mobilenet-v2/artifacts/inference-prepack/v1.0.0/',
'S3DataType': 'S3Prefix',
'CompressionType': 'None'}},
'InstanceType': 'ml.g5.2xlarge',
'Environment': {'SAGEMAKER_PROGRAM': 'inference.py',
'ENDPOINT_SERVER_TIMEOUT': '3600',
'MODEL_CACHE_ROOT': '/opt/ml/model',
'SAGEMAKER_ENV': '1',
'SAGEMAKER_MODEL_SERVER_WORKERS': '1'}}},
{'ConfigName': 'gpu-inference',
'BenchmarkMetrics': [{'name': 'Latency', 'value': '100', 'unit': 'Tokens/S'},
{'name': 'Throughput', 'value': '100', 'unit': 'Tokens/Second'}],
'DeploymentConfig': {'ModelDataDownloadTimeout': None,
'ContainerStartupHealthCheckTimeout': None,
'ImageUri': '763104351884.dkr.ecr.us-west-2.amazonaws.com/pytorch-inference:1.10.0-gpu-py38',
'ModelData': {'S3DataSource': {'S3Uri': 's3://jumpstart-cache-prod-us-west-2/pytorch-ic/pytorch-ic-mobilenet-v2/artifacts/inference-prepack/v1.0.0/',
'S3DataType': 'S3Prefix',
'CompressionType': 'None'}},
'InstanceType': 'ml.g5.2xlarge',
'Environment': {'SAGEMAKER_PROGRAM': 'inference.py',
'ENDPOINT_SERVER_TIMEOUT': '3600',
'MODEL_CACHE_ROOT': '/opt/ml/model',
'SAGEMAKER_ENV': '1',
'SAGEMAKER_MODEL_SERVER_WORKERS': '1'}}}]

Sample output for display_benchmark_metrics api

| | Config Name | Instance Type | Latency (Tokens/S) | Throughput (Tokens/Second) |
|---:|:------------------------|:----------------|---------------------:|-----------------------------:|
| 0 | neuron-inference | ml.g5.2xlarge | 100 | 100 |
| 1 | neuron-inference-budget | ml.g5.2xlarge | 100 | 100 |
| 2 | gpu-inference-budget | ml.g5.2xlarge | 100 | 100 |
| 3 | gpu-inference | ml.g5.2xlarge | 100 | 100 |

Testing done:

Merge Checklist

Put an x in the boxes that apply. You can also fill these out after creating the PR. If you're unsure about any of them, don't hesitate to ask. We're here to help! This is simply a reminder of what we are going to look for before merging your pull request.

General

I have read the CONTRIBUTING doc
I certify that the changes I am introducing will be backward compatible, and I have discussed concerns about this, if any, with the Python SDK team
I used the commit message format described in CONTRIBUTING
I have passed the region in to all S3 and STS clients that I've initialized as part of this change.
I have updated any necessary documentation, including READMEs and API docs (if appropriate)

Tests

I have added tests that prove my fix is effective or that my feature works (if appropriate)
I have added unit and/or integration tests as appropriate to ensure backward compatibility of the changes
I have checked that my tests are not configured for a specific region or account (if appropriate)
I have used unique_name_from_base to create resource names in integ tests (if appropriate)

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.

…nt configs

qiyunz · 2024-04-17T04:24:50Z

+ df["Instance Rate<br>($/Hour)"] = df["Instance Rate<br>($/Hour)"].apply(
+ lambda rate: "${0:.2f}".format(float(rate))
+ )


The instance rate is not available as part of the benchmark metrics data object, we need to use AWS pricing API to query the rate using current region info.

can this be a fast follow to this PR?

jiapinw · 2024-04-17T18:39:17Z


+ def display_benchmark_metrics(self):
+ """Display Benchmark Metrics for deployment configs."""
+ config_names = []


We should consider making this a data class instead of stand along variables

samruds · 2024-04-17T21:17:03Z

+ df["Instance Rate<br>($/Hour)"] = df["Instance Rate<br>($/Hour)"].apply(
+ lambda rate: "${0:.2f}".format(float(rate))
+ )


can this be a fast follow to this PR?

samruds · 2024-04-17T21:29:52Z

+ instance_rates = []
+ for deployment_config in self.list_deployment_configs():
+ config_names.append(deployment_config.get("ConfigName"))
+ instance_types.append(deployment_config.get("BenchmarkMetrics").key()[0])


Is there a chance that a deployment config does not have a benchark metric? In which case do we need validation here?

samruds · 2024-04-17T21:35:03Z

+ config_names.append(deployment_config.get("ConfigName"))
+ instance_types.append(deployment_config.get("BenchmarkMetrics").key()[0])
+
+ for benchmark_metric in deployment_config.get("BenchmarkMetrics").values():


If any of the fields are potentially missing in the deployment configuration, there is a chance we lose order in lists
DC = { {l=A, t=A1, c=A2}, {l=B t=B1, c=}, {l=C, t=C1, c=C2}}
[A, B, C] -> latency
[A1, B1, C1] -> throughput
[A2, C2] -> cost

qiyunz · 2024-04-18T22:14:42Z

+ self.container_startup_health_check_timeout = (
+ deploy_kwargs.container_startup_health_check_timeout
+ )


why not just assign the value?

samruds · 2024-04-19T19:01:02Z

Current structure looks good. Will tests be added in separate PR?

makungaj1 · 2024-04-19T20:39:09Z

Current structure looks good. Will tests be added in separate PR?

Will add tests in the current PR after #4583 is merged as some low levels APIs, bug fixes are not in the main branch yet.

samruds · 2024-04-19T21:03:48Z

Two follow ups, tentative approval for logic. Not ok to merge to master branch.

Add UT
Integrating with Pricing API for cost.

Unit Tests

qiyunz · 2024-04-22T01:48:28Z

+ df.style.set_caption("Benchmark Metrics").set_table_styles([headers]).set_properties(
+ **{"text-align": "left"}
+ )
+ return df


if the function has return value, do you need to update the method signature?

makungaj1 · 2024-04-17T04:14:09Z

Issue #, if available:

Description of changes:
Introduce ReadOnly APIs, list deployment configurations and display benchmark metrics. These APIs are used to quickly select and try different configuration that best serve the use case.

Sample output for list_deployment_configs api

[{'ConfigName': 'neuron-inference',
'BenchmarkMetrics': [{'name': 'Latency', 'value': '100', 'unit': 'Tokens/S'},
{'name': 'Throughput', 'value': '100', 'unit': 'Tokens/Second'}],
'DeploymentConfig': {'ModelDataDownloadTimeout': None,
'ContainerStartupHealthCheckTimeout': None,
'ImageUri': '763104351884.dkr.ecr.us-west-2.amazonaws.com/pytorch-inference:1.10.0-gpu-py38',
'ModelData': {'S3DataSource': {'S3Uri': 's3://jumpstart-cache-prod-us-west-2/pytorch-ic/pytorch-ic-mobilenet-v2/artifacts/inference-prepack/v1.0.0/',
'S3DataType': 'S3Prefix',
'CompressionType': 'None'}},
'InstanceType': 'ml.g5.2xlarge',
'Environment': {'SAGEMAKER_PROGRAM': 'inference.py',
'ENDPOINT_SERVER_TIMEOUT': '3600',
'MODEL_CACHE_ROOT': '/opt/ml/model',
'SAGEMAKER_ENV': '1',
'SAGEMAKER_MODEL_SERVER_WORKERS': '1'}}},
{'ConfigName': 'neuron-inference-budget',
'BenchmarkMetrics': [{'name': 'Latency', 'value': '100', 'unit': 'Tokens/S'},
{'name': 'Throughput', 'value': '100', 'unit': 'Tokens/Second'}],
'DeploymentConfig': {'ModelDataDownloadTimeout': None,
'ContainerStartupHealthCheckTimeout': None,
'ImageUri': '763104351884.dkr.ecr.us-west-2.amazonaws.com/pytorch-inference:1.10.0-gpu-py38',
'ModelData': {'S3DataSource': {'S3Uri': 's3://jumpstart-cache-prod-us-west-2/pytorch-ic/pytorch-ic-mobilenet-v2/artifacts/inference-prepack/v1.0.0/',
'S3DataType': 'S3Prefix',
'CompressionType': 'None'}},
'InstanceType': 'ml.g5.2xlarge',
'Environment': {'SAGEMAKER_PROGRAM': 'inference.py',
'ENDPOINT_SERVER_TIMEOUT': '3600',
'MODEL_CACHE_ROOT': '/opt/ml/model',
'SAGEMAKER_ENV': '1',
'SAGEMAKER_MODEL_SERVER_WORKERS': '1'}}},
{'ConfigName': 'gpu-inference-budget',
'BenchmarkMetrics': [{'name': 'Latency', 'value': '100', 'unit': 'Tokens/S'},
{'name': 'Throughput', 'value': '100', 'unit': 'Tokens/Second'}],
'DeploymentConfig': {'ModelDataDownloadTimeout': None,
'ContainerStartupHealthCheckTimeout': None,
'ImageUri': '763104351884.dkr.ecr.us-west-2.amazonaws.com/pytorch-inference:1.10.0-gpu-py38',
'ModelData': {'S3DataSource': {'S3Uri': 's3://jumpstart-cache-prod-us-west-2/pytorch-ic/pytorch-ic-mobilenet-v2/artifacts/inference-prepack/v1.0.0/',
'S3DataType': 'S3Prefix',
'CompressionType': 'None'}},
'InstanceType': 'ml.g5.2xlarge',
'Environment': {'SAGEMAKER_PROGRAM': 'inference.py',
'ENDPOINT_SERVER_TIMEOUT': '3600',
'MODEL_CACHE_ROOT': '/opt/ml/model',
'SAGEMAKER_ENV': '1',
'SAGEMAKER_MODEL_SERVER_WORKERS': '1'}}},
{'ConfigName': 'gpu-inference',
'BenchmarkMetrics': [{'name': 'Latency', 'value': '100', 'unit': 'Tokens/S'},
{'name': 'Throughput', 'value': '100', 'unit': 'Tokens/Second'}],
'DeploymentConfig': {'ModelDataDownloadTimeout': None,
'ContainerStartupHealthCheckTimeout': None,
'ImageUri': '763104351884.dkr.ecr.us-west-2.amazonaws.com/pytorch-inference:1.10.0-gpu-py38',
'ModelData': {'S3DataSource': {'S3Uri': 's3://jumpstart-cache-prod-us-west-2/pytorch-ic/pytorch-ic-mobilenet-v2/artifacts/inference-prepack/v1.0.0/',
'S3DataType': 'S3Prefix',
'CompressionType': 'None'}},
'InstanceType': 'ml.g5.2xlarge',
'Environment': {'SAGEMAKER_PROGRAM': 'inference.py',
'ENDPOINT_SERVER_TIMEOUT': '3600',
'MODEL_CACHE_ROOT': '/opt/ml/model',
'SAGEMAKER_ENV': '1',
'SAGEMAKER_MODEL_SERVER_WORKERS': '1'}}}]

Sample output for display_benchmark_metrics api

| | Config Name | Instance Type | Latency (Tokens/S) | Throughput (Tokens/Second) |
|---:|:------------------------|:----------------|---------------------:|-----------------------------:|
| 0 | neuron-inference | ml.g5.2xlarge | 100 | 100 |
| 1 | neuron-inference-budget | ml.g5.2xlarge | 100 | 100 |
| 2 | gpu-inference-budget | ml.g5.2xlarge | 100 | 100 |
| 3 | gpu-inference | ml.g5.2xlarge | 100 | 100 |

Testing done:

Merge Checklist

Put an x in the boxes that apply. You can also fill these out after creating the PR. If you're unsure about any of them, don't hesitate to ask. We're here to help! This is simply a reminder of what we are going to look for before merging your pull request.

General

I have read the CONTRIBUTING doc
I certify that the changes I am introducing will be backward compatible, and I have discussed concerns about this, if any, with the Python SDK team
I used the commit message format described in CONTRIBUTING
I have passed the region in to all S3 and STS clients that I've initialized as part of this change.
I have updated any necessary documentation, including READMEs and API docs (if appropriate)

Tests

I have added tests that prove my fix is effective or that my feature works (if appropriate)
I have added unit and/or integration tests as appropriate to ensure backward compatibility of the changes
I have checked that my tests are not configured for a specific region or account (if appropriate)
I have used unique_name_from_base to create resource names in integ tests (if appropriate)

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.

…nt configs

qiyunz · 2024-04-17T04:24:50Z

+ df["Instance Rate<br>($/Hour)"] = df["Instance Rate<br>($/Hour)"].apply(
+ lambda rate: "${0:.2f}".format(float(rate))
+ )


The instance rate is not available as part of the benchmark metrics data object, we need to use AWS pricing API to query the rate using current region info.

can this be a fast follow to this PR?

jiapinw · 2024-04-17T18:39:17Z


+ def display_benchmark_metrics(self):
+ """Display Benchmark Metrics for deployment configs."""
+ config_names = []


We should consider making this a data class instead of stand along variables

samruds · 2024-04-17T21:17:03Z

+ df["Instance Rate<br>($/Hour)"] = df["Instance Rate<br>($/Hour)"].apply(
+ lambda rate: "${0:.2f}".format(float(rate))
+ )


can this be a fast follow to this PR?

samruds · 2024-04-17T21:29:52Z

+ instance_rates = []
+ for deployment_config in self.list_deployment_configs():
+ config_names.append(deployment_config.get("ConfigName"))
+ instance_types.append(deployment_config.get("BenchmarkMetrics").key()[0])


Is there a chance that a deployment config does not have a benchark metric? In which case do we need validation here?

samruds · 2024-04-17T21:35:03Z

+ config_names.append(deployment_config.get("ConfigName"))
+ instance_types.append(deployment_config.get("BenchmarkMetrics").key()[0])
+
+ for benchmark_metric in deployment_config.get("BenchmarkMetrics").values():


If any of the fields are potentially missing in the deployment configuration, there is a chance we lose order in lists
DC = { {l=A, t=A1, c=A2}, {l=B t=B1, c=}, {l=C, t=C1, c=C2}}
[A, B, C] -> latency
[A1, B1, C1] -> throughput
[A2, C2] -> cost

qiyunz · 2024-04-18T22:14:42Z

+ self.container_startup_health_check_timeout = (
+ deploy_kwargs.container_startup_health_check_timeout
+ )


why not just assign the value?

samruds · 2024-04-19T19:01:02Z

Current structure looks good. Will tests be added in separate PR?

makungaj1 · 2024-04-19T20:39:09Z

Current structure looks good. Will tests be added in separate PR?

Will add tests in the current PR after #4583 is merged as some low levels APIs, bug fixes are not in the main branch yet.

samruds · 2024-04-19T21:03:48Z

Two follow ups, tentative approval for logic. Not ok to merge to master branch.

Add UT
Integrating with Pricing API for cost.

Unit Tests

qiyunz · 2024-04-22T01:48:28Z

+ df.style.set_caption("Benchmark Metrics").set_table_styles([headers]).set_properties(
+ **{"text-align": "left"}
+ )
+ return df


if the function has return value, do you need to update the method signature?

makungaj1 · 2024-04-17T04:14:09Z

Issue #, if available:

Description of changes:
Introduce ReadOnly APIs, list deployment configurations and display benchmark metrics. These APIs are used to quickly select and try different configuration that best serve the use case.

Sample output for list_deployment_configs api

[{'ConfigName': 'neuron-inference',
'BenchmarkMetrics': [{'name': 'Latency', 'value': '100', 'unit': 'Tokens/S'},
{'name': 'Throughput', 'value': '100', 'unit': 'Tokens/Second'}],
'DeploymentConfig': {'ModelDataDownloadTimeout': None,
'ContainerStartupHealthCheckTimeout': None,
'ImageUri': '763104351884.dkr.ecr.us-west-2.amazonaws.com/pytorch-inference:1.10.0-gpu-py38',
'ModelData': {'S3DataSource': {'S3Uri': 's3://jumpstart-cache-prod-us-west-2/pytorch-ic/pytorch-ic-mobilenet-v2/artifacts/inference-prepack/v1.0.0/',
'S3DataType': 'S3Prefix',
'CompressionType': 'None'}},
'InstanceType': 'ml.g5.2xlarge',
'Environment': {'SAGEMAKER_PROGRAM': 'inference.py',
'ENDPOINT_SERVER_TIMEOUT': '3600',
'MODEL_CACHE_ROOT': '/opt/ml/model',
'SAGEMAKER_ENV': '1',
'SAGEMAKER_MODEL_SERVER_WORKERS': '1'}}},
{'ConfigName': 'neuron-inference-budget',
'BenchmarkMetrics': [{'name': 'Latency', 'value': '100', 'unit': 'Tokens/S'},
{'name': 'Throughput', 'value': '100', 'unit': 'Tokens/Second'}],
'DeploymentConfig': {'ModelDataDownloadTimeout': None,
'ContainerStartupHealthCheckTimeout': None,
'ImageUri': '763104351884.dkr.ecr.us-west-2.amazonaws.com/pytorch-inference:1.10.0-gpu-py38',
'ModelData': {'S3DataSource': {'S3Uri': 's3://jumpstart-cache-prod-us-west-2/pytorch-ic/pytorch-ic-mobilenet-v2/artifacts/inference-prepack/v1.0.0/',
'S3DataType': 'S3Prefix',
'CompressionType': 'None'}},
'InstanceType': 'ml.g5.2xlarge',
'Environment': {'SAGEMAKER_PROGRAM': 'inference.py',
'ENDPOINT_SERVER_TIMEOUT': '3600',
'MODEL_CACHE_ROOT': '/opt/ml/model',
'SAGEMAKER_ENV': '1',
'SAGEMAKER_MODEL_SERVER_WORKERS': '1'}}},
{'ConfigName': 'gpu-inference-budget',
'BenchmarkMetrics': [{'name': 'Latency', 'value': '100', 'unit': 'Tokens/S'},
{'name': 'Throughput', 'value': '100', 'unit': 'Tokens/Second'}],
'DeploymentConfig': {'ModelDataDownloadTimeout': None,
'ContainerStartupHealthCheckTimeout': None,
'ImageUri': '763104351884.dkr.ecr.us-west-2.amazonaws.com/pytorch-inference:1.10.0-gpu-py38',
'ModelData': {'S3DataSource': {'S3Uri': 's3://jumpstart-cache-prod-us-west-2/pytorch-ic/pytorch-ic-mobilenet-v2/artifacts/inference-prepack/v1.0.0/',
'S3DataType': 'S3Prefix',
'CompressionType': 'None'}},
'InstanceType': 'ml.g5.2xlarge',
'Environment': {'SAGEMAKER_PROGRAM': 'inference.py',
'ENDPOINT_SERVER_TIMEOUT': '3600',
'MODEL_CACHE_ROOT': '/opt/ml/model',
'SAGEMAKER_ENV': '1',
'SAGEMAKER_MODEL_SERVER_WORKERS': '1'}}},
{'ConfigName': 'gpu-inference',
'BenchmarkMetrics': [{'name': 'Latency', 'value': '100', 'unit': 'Tokens/S'},
{'name': 'Throughput', 'value': '100', 'unit': 'Tokens/Second'}],
'DeploymentConfig': {'ModelDataDownloadTimeout': None,
'ContainerStartupHealthCheckTimeout': None,
'ImageUri': '763104351884.dkr.ecr.us-west-2.amazonaws.com/pytorch-inference:1.10.0-gpu-py38',
'ModelData': {'S3DataSource': {'S3Uri': 's3://jumpstart-cache-prod-us-west-2/pytorch-ic/pytorch-ic-mobilenet-v2/artifacts/inference-prepack/v1.0.0/',
'S3DataType': 'S3Prefix',
'CompressionType': 'None'}},
'InstanceType': 'ml.g5.2xlarge',
'Environment': {'SAGEMAKER_PROGRAM': 'inference.py',
'ENDPOINT_SERVER_TIMEOUT': '3600',
'MODEL_CACHE_ROOT': '/opt/ml/model',
'SAGEMAKER_ENV': '1',
'SAGEMAKER_MODEL_SERVER_WORKERS': '1'}}}]

Sample output for display_benchmark_metrics api

| | Config Name | Instance Type | Latency (Tokens/S) | Throughput (Tokens/Second) |
|---:|:------------------------|:----------------|---------------------:|-----------------------------:|
| 0 | neuron-inference | ml.g5.2xlarge | 100 | 100 |
| 1 | neuron-inference-budget | ml.g5.2xlarge | 100 | 100 |
| 2 | gpu-inference-budget | ml.g5.2xlarge | 100 | 100 |
| 3 | gpu-inference | ml.g5.2xlarge | 100 | 100 |

Testing done:

Merge Checklist

Put an x in the boxes that apply. You can also fill these out after creating the PR. If you're unsure about any of them, don't hesitate to ask. We're here to help! This is simply a reminder of what we are going to look for before merging your pull request.

General

I have read the CONTRIBUTING doc
I certify that the changes I am introducing will be backward compatible, and I have discussed concerns about this, if any, with the Python SDK team
I used the commit message format described in CONTRIBUTING
I have passed the region in to all S3 and STS clients that I've initialized as part of this change.
I have updated any necessary documentation, including READMEs and API docs (if appropriate)

Tests

I have added tests that prove my fix is effective or that my feature works (if appropriate)
I have added unit and/or integration tests as appropriate to ensure backward compatibility of the changes
I have checked that my tests are not configured for a specific region or account (if appropriate)
I have used unique_name_from_base to create resource names in integ tests (if appropriate)

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.

…nt configs

qiyunz · 2024-04-17T04:24:50Z

+ df["Instance Rate<br>($/Hour)"] = df["Instance Rate<br>($/Hour)"].apply(
+ lambda rate: "${0:.2f}".format(float(rate))
+ )


The instance rate is not available as part of the benchmark metrics data object, we need to use AWS pricing API to query the rate using current region info.

can this be a fast follow to this PR?

jiapinw · 2024-04-17T18:39:17Z


+ def display_benchmark_metrics(self):
+ """Display Benchmark Metrics for deployment configs."""
+ config_names = []


We should consider making this a data class instead of stand along variables

samruds · 2024-04-17T21:17:03Z

+ df["Instance Rate<br>($/Hour)"] = df["Instance Rate<br>($/Hour)"].apply(
+ lambda rate: "${0:.2f}".format(float(rate))
+ )


can this be a fast follow to this PR?

samruds · 2024-04-17T21:29:52Z

+ instance_rates = []
+ for deployment_config in self.list_deployment_configs():
+ config_names.append(deployment_config.get("ConfigName"))
+ instance_types.append(deployment_config.get("BenchmarkMetrics").key()[0])


Is there a chance that a deployment config does not have a benchark metric? In which case do we need validation here?

samruds · 2024-04-17T21:35:03Z

+ config_names.append(deployment_config.get("ConfigName"))
+ instance_types.append(deployment_config.get("BenchmarkMetrics").key()[0])
+
+ for benchmark_metric in deployment_config.get("BenchmarkMetrics").values():


If any of the fields are potentially missing in the deployment configuration, there is a chance we lose order in lists
DC = { {l=A, t=A1, c=A2}, {l=B t=B1, c=}, {l=C, t=C1, c=C2}}
[A, B, C] -> latency
[A1, B1, C1] -> throughput
[A2, C2] -> cost

qiyunz · 2024-04-18T22:14:42Z

+ self.container_startup_health_check_timeout = (
+ deploy_kwargs.container_startup_health_check_timeout
+ )


why not just assign the value?

samruds · 2024-04-19T19:01:02Z

Current structure looks good. Will tests be added in separate PR?

makungaj1 · 2024-04-19T20:39:09Z

Current structure looks good. Will tests be added in separate PR?

Will add tests in the current PR after #4583 is merged as some low levels APIs, bug fixes are not in the main branch yet.

samruds · 2024-04-19T21:03:48Z

Two follow ups, tentative approval for logic. Not ok to merge to master branch.

Add UT
Integrating with Pricing API for cost.

Unit Tests

qiyunz · 2024-04-22T01:48:28Z

+ df.style.set_caption("Benchmark Metrics").set_table_styles([headers]).set_properties(
+ **{"text-align": "left"}
+ )
+ return df


if the function has return value, do you need to update the method signature?

makungaj1 · 2024-04-17T04:14:09Z

Issue #, if available:

Description of changes:
Introduce ReadOnly APIs, list deployment configurations and display benchmark metrics. These APIs are used to quickly select and try different configuration that best serve the use case.

Sample output for list_deployment_configs api

[{'ConfigName': 'neuron-inference',
'BenchmarkMetrics': [{'name': 'Latency', 'value': '100', 'unit': 'Tokens/S'},
{'name': 'Throughput', 'value': '100', 'unit': 'Tokens/Second'}],
'DeploymentConfig': {'ModelDataDownloadTimeout': None,
'ContainerStartupHealthCheckTimeout': None,
'ImageUri': '763104351884.dkr.ecr.us-west-2.amazonaws.com/pytorch-inference:1.10.0-gpu-py38',
'ModelData': {'S3DataSource': {'S3Uri': 's3://jumpstart-cache-prod-us-west-2/pytorch-ic/pytorch-ic-mobilenet-v2/artifacts/inference-prepack/v1.0.0/',
'S3DataType': 'S3Prefix',
'CompressionType': 'None'}},
'InstanceType': 'ml.g5.2xlarge',
'Environment': {'SAGEMAKER_PROGRAM': 'inference.py',
'ENDPOINT_SERVER_TIMEOUT': '3600',
'MODEL_CACHE_ROOT': '/opt/ml/model',
'SAGEMAKER_ENV': '1',
'SAGEMAKER_MODEL_SERVER_WORKERS': '1'}}},
{'ConfigName': 'neuron-inference-budget',
'BenchmarkMetrics': [{'name': 'Latency', 'value': '100', 'unit': 'Tokens/S'},
{'name': 'Throughput', 'value': '100', 'unit': 'Tokens/Second'}],
'DeploymentConfig': {'ModelDataDownloadTimeout': None,
'ContainerStartupHealthCheckTimeout': None,
'ImageUri': '763104351884.dkr.ecr.us-west-2.amazonaws.com/pytorch-inference:1.10.0-gpu-py38',
'ModelData': {'S3DataSource': {'S3Uri': 's3://jumpstart-cache-prod-us-west-2/pytorch-ic/pytorch-ic-mobilenet-v2/artifacts/inference-prepack/v1.0.0/',
'S3DataType': 'S3Prefix',
'CompressionType': 'None'}},
'InstanceType': 'ml.g5.2xlarge',
'Environment': {'SAGEMAKER_PROGRAM': 'inference.py',
'ENDPOINT_SERVER_TIMEOUT': '3600',
'MODEL_CACHE_ROOT': '/opt/ml/model',
'SAGEMAKER_ENV': '1',
'SAGEMAKER_MODEL_SERVER_WORKERS': '1'}}},
{'ConfigName': 'gpu-inference-budget',
'BenchmarkMetrics': [{'name': 'Latency', 'value': '100', 'unit': 'Tokens/S'},
{'name': 'Throughput', 'value': '100', 'unit': 'Tokens/Second'}],
'DeploymentConfig': {'ModelDataDownloadTimeout': None,
'ContainerStartupHealthCheckTimeout': None,
'ImageUri': '763104351884.dkr.ecr.us-west-2.amazonaws.com/pytorch-inference:1.10.0-gpu-py38',
'ModelData': {'S3DataSource': {'S3Uri': 's3://jumpstart-cache-prod-us-west-2/pytorch-ic/pytorch-ic-mobilenet-v2/artifacts/inference-prepack/v1.0.0/',
'S3DataType': 'S3Prefix',
'CompressionType': 'None'}},
'InstanceType': 'ml.g5.2xlarge',
'Environment': {'SAGEMAKER_PROGRAM': 'inference.py',
'ENDPOINT_SERVER_TIMEOUT': '3600',
'MODEL_CACHE_ROOT': '/opt/ml/model',
'SAGEMAKER_ENV': '1',
'SAGEMAKER_MODEL_SERVER_WORKERS': '1'}}},
{'ConfigName': 'gpu-inference',
'BenchmarkMetrics': [{'name': 'Latency', 'value': '100', 'unit': 'Tokens/S'},
{'name': 'Throughput', 'value': '100', 'unit': 'Tokens/Second'}],
'DeploymentConfig': {'ModelDataDownloadTimeout': None,
'ContainerStartupHealthCheckTimeout': None,
'ImageUri': '763104351884.dkr.ecr.us-west-2.amazonaws.com/pytorch-inference:1.10.0-gpu-py38',
'ModelData': {'S3DataSource': {'S3Uri': 's3://jumpstart-cache-prod-us-west-2/pytorch-ic/pytorch-ic-mobilenet-v2/artifacts/inference-prepack/v1.0.0/',
'S3DataType': 'S3Prefix',
'CompressionType': 'None'}},
'InstanceType': 'ml.g5.2xlarge',
'Environment': {'SAGEMAKER_PROGRAM': 'inference.py',
'ENDPOINT_SERVER_TIMEOUT': '3600',
'MODEL_CACHE_ROOT': '/opt/ml/model',
'SAGEMAKER_ENV': '1',
'SAGEMAKER_MODEL_SERVER_WORKERS': '1'}}}]

Sample output for display_benchmark_metrics api

| | Config Name | Instance Type | Latency (Tokens/S) | Throughput (Tokens/Second) |
|---:|:------------------------|:----------------|---------------------:|-----------------------------:|
| 0 | neuron-inference | ml.g5.2xlarge | 100 | 100 |
| 1 | neuron-inference-budget | ml.g5.2xlarge | 100 | 100 |
| 2 | gpu-inference-budget | ml.g5.2xlarge | 100 | 100 |
| 3 | gpu-inference | ml.g5.2xlarge | 100 | 100 |

Testing done:

Merge Checklist

Put an x in the boxes that apply. You can also fill these out after creating the PR. If you're unsure about any of them, don't hesitate to ask. We're here to help! This is simply a reminder of what we are going to look for before merging your pull request.

General

I have read the CONTRIBUTING doc
I certify that the changes I am introducing will be backward compatible, and I have discussed concerns about this, if any, with the Python SDK team
I used the commit message format described in CONTRIBUTING
I have passed the region in to all S3 and STS clients that I've initialized as part of this change.
I have updated any necessary documentation, including READMEs and API docs (if appropriate)

Tests

I have added tests that prove my fix is effective or that my feature works (if appropriate)
I have added unit and/or integration tests as appropriate to ensure backward compatibility of the changes
I have checked that my tests are not configured for a specific region or account (if appropriate)
I have used unique_name_from_base to create resource names in integ tests (if appropriate)

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.

…nt configs

qiyunz · 2024-04-17T04:24:50Z

+ df["Instance Rate<br>($/Hour)"] = df["Instance Rate<br>($/Hour)"].apply(
+ lambda rate: "${0:.2f}".format(float(rate))
+ )


The instance rate is not available as part of the benchmark metrics data object, we need to use AWS pricing API to query the rate using current region info.

can this be a fast follow to this PR?

jiapinw · 2024-04-17T18:39:17Z


+ def display_benchmark_metrics(self):
+ """Display Benchmark Metrics for deployment configs."""
+ config_names = []


We should consider making this a data class instead of stand along variables

samruds · 2024-04-17T21:17:03Z

+ df["Instance Rate<br>($/Hour)"] = df["Instance Rate<br>($/Hour)"].apply(
+ lambda rate: "${0:.2f}".format(float(rate))
+ )


can this be a fast follow to this PR?

samruds · 2024-04-17T21:29:52Z

+ instance_rates = []
+ for deployment_config in self.list_deployment_configs():
+ config_names.append(deployment_config.get("ConfigName"))
+ instance_types.append(deployment_config.get("BenchmarkMetrics").key()[0])


Is there a chance that a deployment config does not have a benchark metric? In which case do we need validation here?

samruds · 2024-04-17T21:35:03Z

+ config_names.append(deployment_config.get("ConfigName"))
+ instance_types.append(deployment_config.get("BenchmarkMetrics").key()[0])
+
+ for benchmark_metric in deployment_config.get("BenchmarkMetrics").values():


If any of the fields are potentially missing in the deployment configuration, there is a chance we lose order in lists
DC = { {l=A, t=A1, c=A2}, {l=B t=B1, c=}, {l=C, t=C1, c=C2}}
[A, B, C] -> latency
[A1, B1, C1] -> throughput
[A2, C2] -> cost

qiyunz · 2024-04-18T22:14:42Z

+ self.container_startup_health_check_timeout = (
+ deploy_kwargs.container_startup_health_check_timeout
+ )


why not just assign the value?

samruds · 2024-04-19T19:01:02Z

Current structure looks good. Will tests be added in separate PR?

makungaj1 · 2024-04-19T20:39:09Z

Current structure looks good. Will tests be added in separate PR?

Will add tests in the current PR after #4583 is merged as some low levels APIs, bug fixes are not in the main branch yet.

samruds · 2024-04-19T21:03:48Z

Two follow ups, tentative approval for logic. Not ok to merge to master branch.

Add UT
Integrating with Pricing API for cost.

Unit Tests

qiyunz · 2024-04-22T01:48:28Z

+ df.style.set_caption("Benchmark Metrics").set_table_styles([headers]).set_properties(
+ **{"text-align": "left"}
+ )
+ return df


if the function has return value, do you need to update the method signature?

makungaj1 · 2024-04-17T04:14:09Z

Issue #, if available:

Description of changes:
Introduce ReadOnly APIs, list deployment configurations and display benchmark metrics. These APIs are used to quickly select and try different configuration that best serve the use case.

Sample output for list_deployment_configs api

[{'ConfigName': 'neuron-inference',
'BenchmarkMetrics': [{'name': 'Latency', 'value': '100', 'unit': 'Tokens/S'},
{'name': 'Throughput', 'value': '100', 'unit': 'Tokens/Second'}],
'DeploymentConfig': {'ModelDataDownloadTimeout': None,
'ContainerStartupHealthCheckTimeout': None,
'ImageUri': '763104351884.dkr.ecr.us-west-2.amazonaws.com/pytorch-inference:1.10.0-gpu-py38',
'ModelData': {'S3DataSource': {'S3Uri': 's3://jumpstart-cache-prod-us-west-2/pytorch-ic/pytorch-ic-mobilenet-v2/artifacts/inference-prepack/v1.0.0/',
'S3DataType': 'S3Prefix',
'CompressionType': 'None'}},
'InstanceType': 'ml.g5.2xlarge',
'Environment': {'SAGEMAKER_PROGRAM': 'inference.py',
'ENDPOINT_SERVER_TIMEOUT': '3600',
'MODEL_CACHE_ROOT': '/opt/ml/model',
'SAGEMAKER_ENV': '1',
'SAGEMAKER_MODEL_SERVER_WORKERS': '1'}}},
{'ConfigName': 'neuron-inference-budget',
'BenchmarkMetrics': [{'name': 'Latency', 'value': '100', 'unit': 'Tokens/S'},
{'name': 'Throughput', 'value': '100', 'unit': 'Tokens/Second'}],
'DeploymentConfig': {'ModelDataDownloadTimeout': None,
'ContainerStartupHealthCheckTimeout': None,
'ImageUri': '763104351884.dkr.ecr.us-west-2.amazonaws.com/pytorch-inference:1.10.0-gpu-py38',
'ModelData': {'S3DataSource': {'S3Uri': 's3://jumpstart-cache-prod-us-west-2/pytorch-ic/pytorch-ic-mobilenet-v2/artifacts/inference-prepack/v1.0.0/',
'S3DataType': 'S3Prefix',
'CompressionType': 'None'}},
'InstanceType': 'ml.g5.2xlarge',
'Environment': {'SAGEMAKER_PROGRAM': 'inference.py',
'ENDPOINT_SERVER_TIMEOUT': '3600',
'MODEL_CACHE_ROOT': '/opt/ml/model',
'SAGEMAKER_ENV': '1',
'SAGEMAKER_MODEL_SERVER_WORKERS': '1'}}},
{'ConfigName': 'gpu-inference-budget',
'BenchmarkMetrics': [{'name': 'Latency', 'value': '100', 'unit': 'Tokens/S'},
{'name': 'Throughput', 'value': '100', 'unit': 'Tokens/Second'}],
'DeploymentConfig': {'ModelDataDownloadTimeout': None,
'ContainerStartupHealthCheckTimeout': None,
'ImageUri': '763104351884.dkr.ecr.us-west-2.amazonaws.com/pytorch-inference:1.10.0-gpu-py38',
'ModelData': {'S3DataSource': {'S3Uri': 's3://jumpstart-cache-prod-us-west-2/pytorch-ic/pytorch-ic-mobilenet-v2/artifacts/inference-prepack/v1.0.0/',
'S3DataType': 'S3Prefix',
'CompressionType': 'None'}},
'InstanceType': 'ml.g5.2xlarge',
'Environment': {'SAGEMAKER_PROGRAM': 'inference.py',
'ENDPOINT_SERVER_TIMEOUT': '3600',
'MODEL_CACHE_ROOT': '/opt/ml/model',
'SAGEMAKER_ENV': '1',
'SAGEMAKER_MODEL_SERVER_WORKERS': '1'}}},
{'ConfigName': 'gpu-inference',
'BenchmarkMetrics': [{'name': 'Latency', 'value': '100', 'unit': 'Tokens/S'},
{'name': 'Throughput', 'value': '100', 'unit': 'Tokens/Second'}],
'DeploymentConfig': {'ModelDataDownloadTimeout': None,
'ContainerStartupHealthCheckTimeout': None,
'ImageUri': '763104351884.dkr.ecr.us-west-2.amazonaws.com/pytorch-inference:1.10.0-gpu-py38',
'ModelData': {'S3DataSource': {'S3Uri': 's3://jumpstart-cache-prod-us-west-2/pytorch-ic/pytorch-ic-mobilenet-v2/artifacts/inference-prepack/v1.0.0/',
'S3DataType': 'S3Prefix',
'CompressionType': 'None'}},
'InstanceType': 'ml.g5.2xlarge',
'Environment': {'SAGEMAKER_PROGRAM': 'inference.py',
'ENDPOINT_SERVER_TIMEOUT': '3600',
'MODEL_CACHE_ROOT': '/opt/ml/model',
'SAGEMAKER_ENV': '1',
'SAGEMAKER_MODEL_SERVER_WORKERS': '1'}}}]

Sample output for display_benchmark_metrics api

| | Config Name | Instance Type | Latency (Tokens/S) | Throughput (Tokens/Second) |
|---:|:------------------------|:----------------|---------------------:|-----------------------------:|
| 0 | neuron-inference | ml.g5.2xlarge | 100 | 100 |
| 1 | neuron-inference-budget | ml.g5.2xlarge | 100 | 100 |
| 2 | gpu-inference-budget | ml.g5.2xlarge | 100 | 100 |
| 3 | gpu-inference | ml.g5.2xlarge | 100 | 100 |

Testing done:

Merge Checklist

Put an x in the boxes that apply. You can also fill these out after creating the PR. If you're unsure about any of them, don't hesitate to ask. We're here to help! This is simply a reminder of what we are going to look for before merging your pull request.

General

I have read the CONTRIBUTING doc
I certify that the changes I am introducing will be backward compatible, and I have discussed concerns about this, if any, with the Python SDK team
I used the commit message format described in CONTRIBUTING
I have passed the region in to all S3 and STS clients that I've initialized as part of this change.
I have updated any necessary documentation, including READMEs and API docs (if appropriate)

Tests

I have added tests that prove my fix is effective or that my feature works (if appropriate)
I have added unit and/or integration tests as appropriate to ensure backward compatibility of the changes
I have checked that my tests are not configured for a specific region or account (if appropriate)
I have used unique_name_from_base to create resource names in integ tests (if appropriate)

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.

…nt configs

qiyunz · 2024-04-17T04:24:50Z

+ df["Instance Rate<br>($/Hour)"] = df["Instance Rate<br>($/Hour)"].apply(
+ lambda rate: "${0:.2f}".format(float(rate))
+ )


The instance rate is not available as part of the benchmark metrics data object, we need to use AWS pricing API to query the rate using current region info.

can this be a fast follow to this PR?

jiapinw · 2024-04-17T18:39:17Z


+ def display_benchmark_metrics(self):
+ """Display Benchmark Metrics for deployment configs."""
+ config_names = []


We should consider making this a data class instead of stand along variables

samruds · 2024-04-17T21:17:03Z

+ df["Instance Rate<br>($/Hour)"] = df["Instance Rate<br>($/Hour)"].apply(
+ lambda rate: "${0:.2f}".format(float(rate))
+ )


can this be a fast follow to this PR?

samruds · 2024-04-17T21:29:52Z

+ instance_rates = []
+ for deployment_config in self.list_deployment_configs():
+ config_names.append(deployment_config.get("ConfigName"))
+ instance_types.append(deployment_config.get("BenchmarkMetrics").key()[0])


Is there a chance that a deployment config does not have a benchark metric? In which case do we need validation here?

samruds · 2024-04-17T21:35:03Z

+ config_names.append(deployment_config.get("ConfigName"))
+ instance_types.append(deployment_config.get("BenchmarkMetrics").key()[0])
+
+ for benchmark_metric in deployment_config.get("BenchmarkMetrics").values():


If any of the fields are potentially missing in the deployment configuration, there is a chance we lose order in lists
DC = { {l=A, t=A1, c=A2}, {l=B t=B1, c=}, {l=C, t=C1, c=C2}}
[A, B, C] -> latency
[A1, B1, C1] -> throughput
[A2, C2] -> cost

qiyunz · 2024-04-18T22:14:42Z

+ self.container_startup_health_check_timeout = (
+ deploy_kwargs.container_startup_health_check_timeout
+ )


why not just assign the value?

samruds · 2024-04-19T19:01:02Z

Current structure looks good. Will tests be added in separate PR?

makungaj1 · 2024-04-19T20:39:09Z

Current structure looks good. Will tests be added in separate PR?

Will add tests in the current PR after #4583 is merged as some low levels APIs, bug fixes are not in the main branch yet.

samruds · 2024-04-19T21:03:48Z

Two follow ups, tentative approval for logic. Not ok to merge to master branch.

Add UT
Integrating with Pricing API for cost.

Unit Tests

qiyunz · 2024-04-22T01:48:28Z

+ df.style.set_caption("Benchmark Metrics").set_table_styles([headers]).set_properties(
+ **{"text-align": "left"}
+ )
+ return df


if the function has return value, do you need to update the method signature?

makungaj1 · 2024-04-17T04:14:09Z

Issue #, if available:

Description of changes:
Introduce ReadOnly APIs, list deployment configurations and display benchmark metrics. These APIs are used to quickly select and try different configuration that best serve the use case.

Sample output for list_deployment_configs api

[{'ConfigName': 'neuron-inference',
'BenchmarkMetrics': [{'name': 'Latency', 'value': '100', 'unit': 'Tokens/S'},
{'name': 'Throughput', 'value': '100', 'unit': 'Tokens/Second'}],
'DeploymentConfig': {'ModelDataDownloadTimeout': None,
'ContainerStartupHealthCheckTimeout': None,
'ImageUri': '763104351884.dkr.ecr.us-west-2.amazonaws.com/pytorch-inference:1.10.0-gpu-py38',
'ModelData': {'S3DataSource': {'S3Uri': 's3://jumpstart-cache-prod-us-west-2/pytorch-ic/pytorch-ic-mobilenet-v2/artifacts/inference-prepack/v1.0.0/',
'S3DataType': 'S3Prefix',
'CompressionType': 'None'}},
'InstanceType': 'ml.g5.2xlarge',
'Environment': {'SAGEMAKER_PROGRAM': 'inference.py',
'ENDPOINT_SERVER_TIMEOUT': '3600',
'MODEL_CACHE_ROOT': '/opt/ml/model',
'SAGEMAKER_ENV': '1',
'SAGEMAKER_MODEL_SERVER_WORKERS': '1'}}},
{'ConfigName': 'neuron-inference-budget',
'BenchmarkMetrics': [{'name': 'Latency', 'value': '100', 'unit': 'Tokens/S'},
{'name': 'Throughput', 'value': '100', 'unit': 'Tokens/Second'}],
'DeploymentConfig': {'ModelDataDownloadTimeout': None,
'ContainerStartupHealthCheckTimeout': None,
'ImageUri': '763104351884.dkr.ecr.us-west-2.amazonaws.com/pytorch-inference:1.10.0-gpu-py38',
'ModelData': {'S3DataSource': {'S3Uri': 's3://jumpstart-cache-prod-us-west-2/pytorch-ic/pytorch-ic-mobilenet-v2/artifacts/inference-prepack/v1.0.0/',
'S3DataType': 'S3Prefix',
'CompressionType': 'None'}},
'InstanceType': 'ml.g5.2xlarge',
'Environment': {'SAGEMAKER_PROGRAM': 'inference.py',
'ENDPOINT_SERVER_TIMEOUT': '3600',
'MODEL_CACHE_ROOT': '/opt/ml/model',
'SAGEMAKER_ENV': '1',
'SAGEMAKER_MODEL_SERVER_WORKERS': '1'}}},
{'ConfigName': 'gpu-inference-budget',
'BenchmarkMetrics': [{'name': 'Latency', 'value': '100', 'unit': 'Tokens/S'},
{'name': 'Throughput', 'value': '100', 'unit': 'Tokens/Second'}],
'DeploymentConfig': {'ModelDataDownloadTimeout': None,
'ContainerStartupHealthCheckTimeout': None,
'ImageUri': '763104351884.dkr.ecr.us-west-2.amazonaws.com/pytorch-inference:1.10.0-gpu-py38',
'ModelData': {'S3DataSource': {'S3Uri': 's3://jumpstart-cache-prod-us-west-2/pytorch-ic/pytorch-ic-mobilenet-v2/artifacts/inference-prepack/v1.0.0/',
'S3DataType': 'S3Prefix',
'CompressionType': 'None'}},
'InstanceType': 'ml.g5.2xlarge',
'Environment': {'SAGEMAKER_PROGRAM': 'inference.py',
'ENDPOINT_SERVER_TIMEOUT': '3600',
'MODEL_CACHE_ROOT': '/opt/ml/model',
'SAGEMAKER_ENV': '1',
'SAGEMAKER_MODEL_SERVER_WORKERS': '1'}}},
{'ConfigName': 'gpu-inference',
'BenchmarkMetrics': [{'name': 'Latency', 'value': '100', 'unit': 'Tokens/S'},
{'name': 'Throughput', 'value': '100', 'unit': 'Tokens/Second'}],
'DeploymentConfig': {'ModelDataDownloadTimeout': None,
'ContainerStartupHealthCheckTimeout': None,
'ImageUri': '763104351884.dkr.ecr.us-west-2.amazonaws.com/pytorch-inference:1.10.0-gpu-py38',
'ModelData': {'S3DataSource': {'S3Uri': 's3://jumpstart-cache-prod-us-west-2/pytorch-ic/pytorch-ic-mobilenet-v2/artifacts/inference-prepack/v1.0.0/',
'S3DataType': 'S3Prefix',
'CompressionType': 'None'}},
'InstanceType': 'ml.g5.2xlarge',
'Environment': {'SAGEMAKER_PROGRAM': 'inference.py',
'ENDPOINT_SERVER_TIMEOUT': '3600',
'MODEL_CACHE_ROOT': '/opt/ml/model',
'SAGEMAKER_ENV': '1',
'SAGEMAKER_MODEL_SERVER_WORKERS': '1'}}}]

Sample output for display_benchmark_metrics api

| | Config Name | Instance Type | Latency (Tokens/S) | Throughput (Tokens/Second) |
|---:|:------------------------|:----------------|---------------------:|-----------------------------:|
| 0 | neuron-inference | ml.g5.2xlarge | 100 | 100 |
| 1 | neuron-inference-budget | ml.g5.2xlarge | 100 | 100 |
| 2 | gpu-inference-budget | ml.g5.2xlarge | 100 | 100 |
| 3 | gpu-inference | ml.g5.2xlarge | 100 | 100 |

Testing done:

Merge Checklist

Put an x in the boxes that apply. You can also fill these out after creating the PR. If you're unsure about any of them, don't hesitate to ask. We're here to help! This is simply a reminder of what we are going to look for before merging your pull request.

General

I have read the CONTRIBUTING doc
I certify that the changes I am introducing will be backward compatible, and I have discussed concerns about this, if any, with the Python SDK team
I used the commit message format described in CONTRIBUTING
I have passed the region in to all S3 and STS clients that I've initialized as part of this change.
I have updated any necessary documentation, including READMEs and API docs (if appropriate)

Tests

I have added tests that prove my fix is effective or that my feature works (if appropriate)
I have added unit and/or integration tests as appropriate to ensure backward compatibility of the changes
I have checked that my tests are not configured for a specific region or account (if appropriate)
I have used unique_name_from_base to create resource names in integ tests (if appropriate)

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.

…nt configs

qiyunz · 2024-04-17T04:24:50Z

+ df["Instance Rate<br>($/Hour)"] = df["Instance Rate<br>($/Hour)"].apply(
+ lambda rate: "${0:.2f}".format(float(rate))
+ )


The instance rate is not available as part of the benchmark metrics data object, we need to use AWS pricing API to query the rate using current region info.

can this be a fast follow to this PR?

jiapinw · 2024-04-17T18:39:17Z


+ def display_benchmark_metrics(self):
+ """Display Benchmark Metrics for deployment configs."""
+ config_names = []


We should consider making this a data class instead of stand along variables

samruds · 2024-04-17T21:17:03Z

+ df["Instance Rate<br>($/Hour)"] = df["Instance Rate<br>($/Hour)"].apply(
+ lambda rate: "${0:.2f}".format(float(rate))
+ )


can this be a fast follow to this PR?

samruds · 2024-04-17T21:29:52Z

+ instance_rates = []
+ for deployment_config in self.list_deployment_configs():
+ config_names.append(deployment_config.get("ConfigName"))
+ instance_types.append(deployment_config.get("BenchmarkMetrics").key()[0])


Is there a chance that a deployment config does not have a benchark metric? In which case do we need validation here?

samruds · 2024-04-17T21:35:03Z

+ config_names.append(deployment_config.get("ConfigName"))
+ instance_types.append(deployment_config.get("BenchmarkMetrics").key()[0])
+
+ for benchmark_metric in deployment_config.get("BenchmarkMetrics").values():


If any of the fields are potentially missing in the deployment configuration, there is a chance we lose order in lists
DC = { {l=A, t=A1, c=A2}, {l=B t=B1, c=}, {l=C, t=C1, c=C2}}
[A, B, C] -> latency
[A1, B1, C1] -> throughput
[A2, C2] -> cost

qiyunz · 2024-04-18T22:14:42Z

+ self.container_startup_health_check_timeout = (
+ deploy_kwargs.container_startup_health_check_timeout
+ )


why not just assign the value?

samruds · 2024-04-19T19:01:02Z

Current structure looks good. Will tests be added in separate PR?

makungaj1 · 2024-04-19T20:39:09Z

Current structure looks good. Will tests be added in separate PR?

Will add tests in the current PR after #4583 is merged as some low levels APIs, bug fixes are not in the main branch yet.

samruds · 2024-04-19T21:03:48Z

Two follow ups, tentative approval for logic. Not ok to merge to master branch.

Add UT
Integrating with Pricing API for cost.

Unit Tests

qiyunz · 2024-04-22T01:48:28Z

+ df.style.set_caption("Benchmark Metrics").set_table_styles([headers]).set_properties(
+ **{"text-align": "left"}
+ )
+ return df


if the function has return value, do you need to update the method signature?

makungaj1 · 2024-04-17T04:14:09Z

Issue #, if available:

Description of changes:
Introduce ReadOnly APIs, list deployment configurations and display benchmark metrics. These APIs are used to quickly select and try different configuration that best serve the use case.

Sample output for list_deployment_configs api

[{'ConfigName': 'neuron-inference',
'BenchmarkMetrics': [{'name': 'Latency', 'value': '100', 'unit': 'Tokens/S'},
{'name': 'Throughput', 'value': '100', 'unit': 'Tokens/Second'}],
'DeploymentConfig': {'ModelDataDownloadTimeout': None,
'ContainerStartupHealthCheckTimeout': None,
'ImageUri': '763104351884.dkr.ecr.us-west-2.amazonaws.com/pytorch-inference:1.10.0-gpu-py38',
'ModelData': {'S3DataSource': {'S3Uri': 's3://jumpstart-cache-prod-us-west-2/pytorch-ic/pytorch-ic-mobilenet-v2/artifacts/inference-prepack/v1.0.0/',
'S3DataType': 'S3Prefix',
'CompressionType': 'None'}},
'InstanceType': 'ml.g5.2xlarge',
'Environment': {'SAGEMAKER_PROGRAM': 'inference.py',
'ENDPOINT_SERVER_TIMEOUT': '3600',
'MODEL_CACHE_ROOT': '/opt/ml/model',
'SAGEMAKER_ENV': '1',
'SAGEMAKER_MODEL_SERVER_WORKERS': '1'}}},
{'ConfigName': 'neuron-inference-budget',
'BenchmarkMetrics': [{'name': 'Latency', 'value': '100', 'unit': 'Tokens/S'},
{'name': 'Throughput', 'value': '100', 'unit': 'Tokens/Second'}],
'DeploymentConfig': {'ModelDataDownloadTimeout': None,
'ContainerStartupHealthCheckTimeout': None,
'ImageUri': '763104351884.dkr.ecr.us-west-2.amazonaws.com/pytorch-inference:1.10.0-gpu-py38',
'ModelData': {'S3DataSource': {'S3Uri': 's3://jumpstart-cache-prod-us-west-2/pytorch-ic/pytorch-ic-mobilenet-v2/artifacts/inference-prepack/v1.0.0/',
'S3DataType': 'S3Prefix',
'CompressionType': 'None'}},
'InstanceType': 'ml.g5.2xlarge',
'Environment': {'SAGEMAKER_PROGRAM': 'inference.py',
'ENDPOINT_SERVER_TIMEOUT': '3600',
'MODEL_CACHE_ROOT': '/opt/ml/model',
'SAGEMAKER_ENV': '1',
'SAGEMAKER_MODEL_SERVER_WORKERS': '1'}}},
{'ConfigName': 'gpu-inference-budget',
'BenchmarkMetrics': [{'name': 'Latency', 'value': '100', 'unit': 'Tokens/S'},
{'name': 'Throughput', 'value': '100', 'unit': 'Tokens/Second'}],
'DeploymentConfig': {'ModelDataDownloadTimeout': None,
'ContainerStartupHealthCheckTimeout': None,
'ImageUri': '763104351884.dkr.ecr.us-west-2.amazonaws.com/pytorch-inference:1.10.0-gpu-py38',
'ModelData': {'S3DataSource': {'S3Uri': 's3://jumpstart-cache-prod-us-west-2/pytorch-ic/pytorch-ic-mobilenet-v2/artifacts/inference-prepack/v1.0.0/',
'S3DataType': 'S3Prefix',
'CompressionType': 'None'}},
'InstanceType': 'ml.g5.2xlarge',
'Environment': {'SAGEMAKER_PROGRAM': 'inference.py',
'ENDPOINT_SERVER_TIMEOUT': '3600',
'MODEL_CACHE_ROOT': '/opt/ml/model',
'SAGEMAKER_ENV': '1',
'SAGEMAKER_MODEL_SERVER_WORKERS': '1'}}},
{'ConfigName': 'gpu-inference',
'BenchmarkMetrics': [{'name': 'Latency', 'value': '100', 'unit': 'Tokens/S'},
{'name': 'Throughput', 'value': '100', 'unit': 'Tokens/Second'}],
'DeploymentConfig': {'ModelDataDownloadTimeout': None,
'ContainerStartupHealthCheckTimeout': None,
'ImageUri': '763104351884.dkr.ecr.us-west-2.amazonaws.com/pytorch-inference:1.10.0-gpu-py38',
'ModelData': {'S3DataSource': {'S3Uri': 's3://jumpstart-cache-prod-us-west-2/pytorch-ic/pytorch-ic-mobilenet-v2/artifacts/inference-prepack/v1.0.0/',
'S3DataType': 'S3Prefix',
'CompressionType': 'None'}},
'InstanceType': 'ml.g5.2xlarge',
'Environment': {'SAGEMAKER_PROGRAM': 'inference.py',
'ENDPOINT_SERVER_TIMEOUT': '3600',
'MODEL_CACHE_ROOT': '/opt/ml/model',
'SAGEMAKER_ENV': '1',
'SAGEMAKER_MODEL_SERVER_WORKERS': '1'}}}]

Sample output for display_benchmark_metrics api

| | Config Name | Instance Type | Latency (Tokens/S) | Throughput (Tokens/Second) |
|---:|:------------------------|:----------------|---------------------:|-----------------------------:|
| 0 | neuron-inference | ml.g5.2xlarge | 100 | 100 |
| 1 | neuron-inference-budget | ml.g5.2xlarge | 100 | 100 |
| 2 | gpu-inference-budget | ml.g5.2xlarge | 100 | 100 |
| 3 | gpu-inference | ml.g5.2xlarge | 100 | 100 |

Testing done:

Merge Checklist

Put an x in the boxes that apply. You can also fill these out after creating the PR. If you're unsure about any of them, don't hesitate to ask. We're here to help! This is simply a reminder of what we are going to look for before merging your pull request.

General

I have read the CONTRIBUTING doc
I certify that the changes I am introducing will be backward compatible, and I have discussed concerns about this, if any, with the Python SDK team
I used the commit message format described in CONTRIBUTING
I have passed the region in to all S3 and STS clients that I've initialized as part of this change.
I have updated any necessary documentation, including READMEs and API docs (if appropriate)

Tests

I have added tests that prove my fix is effective or that my feature works (if appropriate)
I have added unit and/or integration tests as appropriate to ensure backward compatibility of the changes
I have checked that my tests are not configured for a specific region or account (if appropriate)
I have used unique_name_from_base to create resource names in integ tests (if appropriate)

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.

…nt configs

qiyunz · 2024-04-17T04:24:50Z

+ df["Instance Rate<br>($/Hour)"] = df["Instance Rate<br>($/Hour)"].apply(
+ lambda rate: "${0:.2f}".format(float(rate))
+ )


The instance rate is not available as part of the benchmark metrics data object, we need to use AWS pricing API to query the rate using current region info.

can this be a fast follow to this PR?

jiapinw · 2024-04-17T18:39:17Z


+ def display_benchmark_metrics(self):
+ """Display Benchmark Metrics for deployment configs."""
+ config_names = []


We should consider making this a data class instead of stand along variables

samruds · 2024-04-17T21:17:03Z

+ df["Instance Rate<br>($/Hour)"] = df["Instance Rate<br>($/Hour)"].apply(
+ lambda rate: "${0:.2f}".format(float(rate))
+ )


can this be a fast follow to this PR?

samruds · 2024-04-17T21:29:52Z

+ instance_rates = []
+ for deployment_config in self.list_deployment_configs():
+ config_names.append(deployment_config.get("ConfigName"))
+ instance_types.append(deployment_config.get("BenchmarkMetrics").key()[0])


Is there a chance that a deployment config does not have a benchark metric? In which case do we need validation here?

samruds · 2024-04-17T21:35:03Z

+ config_names.append(deployment_config.get("ConfigName"))
+ instance_types.append(deployment_config.get("BenchmarkMetrics").key()[0])
+
+ for benchmark_metric in deployment_config.get("BenchmarkMetrics").values():


If any of the fields are potentially missing in the deployment configuration, there is a chance we lose order in lists
DC = { {l=A, t=A1, c=A2}, {l=B t=B1, c=}, {l=C, t=C1, c=C2}}
[A, B, C] -> latency
[A1, B1, C1] -> throughput
[A2, C2] -> cost

qiyunz · 2024-04-18T22:14:42Z

+ self.container_startup_health_check_timeout = (
+ deploy_kwargs.container_startup_health_check_timeout
+ )


why not just assign the value?

samruds · 2024-04-19T19:01:02Z

Current structure looks good. Will tests be added in separate PR?

makungaj1 · 2024-04-19T20:39:09Z

Current structure looks good. Will tests be added in separate PR?

Will add tests in the current PR after #4583 is merged as some low levels APIs, bug fixes are not in the main branch yet.

samruds · 2024-04-19T21:03:48Z

Two follow ups, tentative approval for logic. Not ok to merge to master branch.

Add UT
Integrating with Pricing API for cost.

Unit Tests

qiyunz · 2024-04-22T01:48:28Z

+ df.style.set_caption("Benchmark Metrics").set_table_styles([headers]).set_properties(
+ **{"text-align": "left"}
+ )
+ return df


if the function has return value, do you need to update the method signature?

Add Public facing APIs to display benchmark metrics and list deployme…
5cff65b
…nt configs

makungaj1 temporarily deployed to auto-approve April 17, 2024 04:14 — with GitHub Actions Inactive

qiyunz reviewed Apr 17, 2024
View reviewed changes

Merge branch 'master' into master
c70ee25

makungaj1 temporarily deployed to auto-approve April 17, 2024 16:15 — with GitHub Actions Inactive

jiapinw reviewed Apr 17, 2024
View reviewed changes

Comment threadsetup.py Outdated

jiapinw reviewed Apr 17, 2024
View reviewed changes

Comment threadsrc/sagemaker/jumpstart/model.py Outdated

qiyunz reviewed Apr 17, 2024
View reviewed changes

Comment threadsetup.py Outdated
Comment threadsrc/sagemaker/jumpstart/model.py Outdated

samruds suggested changes Apr 17, 2024
View reviewed changes

Merge branch 'master' into master
1a080aa

makungaj1 temporarily deployed to auto-approve April 18, 2024 01:04 — with GitHub Actions Inactive

Resolved PR Review comments
1431b5c

makungaj1 temporarily deployed to auto-approve April 18, 2024 05:19 — with GitHub Actions Inactive

Resolve PR Review Comments
3a2547b

makungaj1 temporarily deployed to auto-approve April 18, 2024 05:27 — with GitHub Actions Inactive

Add List and Display APIs in ModelBuilder
1dfa190

makungaj1 temporarily deployed to auto-approve April 18, 2024 05:34 — with GitHub Actions Inactive

Refactoring
9b80d07

makungaj1 temporarily deployed to auto-approve April 18, 2024 06:45 — with GitHub Actions Inactive

Refactoring
4669c9d

makungaj1 temporarily deployed to auto-approve April 18, 2024 07:09 — with GitHub Actions Inactive

Resolved PR review comments
f862797

makungaj1 temporarily deployed to auto-approve April 18, 2024 07:14 — with GitHub Actions Inactive

Merge branch 'master' into master
2b0fc3c

makungaj1 temporarily deployed to auto-approve April 18, 2024 17:23 — with GitHub Actions Inactive

qiyunz reviewed Apr 18, 2024
View reviewed changes

gwang111 reviewed Apr 19, 2024
View reviewed changes

Comment threadsrc/sagemaker/jumpstart/model.py Outdated

gwang111 reviewed Apr 19, 2024
View reviewed changes

Comment threadsrc/sagemaker/jumpstart/model.py Outdated

gwang111 reviewed Apr 19, 2024
View reviewed changes

Comment threadsrc/sagemaker/jumpstart/model.py Outdated

Refactoring
2a1414e

makungaj1 temporarily deployed to auto-approve April 19, 2024 17:05 — with GitHub Actions Inactive

samruds reviewed Apr 19, 2024
View reviewed changes

Comment threadsrc/sagemaker/jumpstart/model.py Outdated
Comment threadsrc/sagemaker/jumpstart/model.py Outdated
Comment threadsrc/sagemaker/jumpstart/types.py
Comment threadsrc/sagemaker/jumpstart/model.py Outdated
Comment threadsrc/sagemaker/jumpstart/types.py

qiyunz reviewed Apr 19, 2024
View reviewed changes

Comment threadsrc/sagemaker/jumpstart/model.py
Comment threadsrc/sagemaker/jumpstart/model.py Outdated

Refactoring
e5ee82e

makungaj1 temporarily deployed to auto-approve April 19, 2024 20:08 — with GitHub Actions Inactive

Resolve PR review comments
edd271d

makungaj1 temporarily deployed to auto-approve April 19, 2024 20:40 — with GitHub Actions Inactive

Resolve PR Review comments
84eece1

makungaj1 temporarily deployed to auto-approve April 19, 2024 20:47 — with GitHub Actions Inactive

Resolve PR review comments
1dd6854

makungaj1 temporarily deployed to auto-approve April 19, 2024 20:54 — with GitHub Actions Inactive

samruds approved these changes Apr 19, 2024
View reviewed changes

Refactoring
08edb27

makungaj1 temporarily deployed to auto-approve April 20, 2024 04:04 — with GitHub Actions Inactive

Add Unit tests
dd1b66e
Unit Tests

makungaj1 temporarily deployed to auto-approve April 21, 2024 18:50 — with GitHub Actions Inactive

Get Instance Rate
e878d75

makungaj1 temporarily deployed to auto-approve April 21, 2024 23:52 — with GitHub Actions Inactive

Refactoring
523e0b9

makungaj1 temporarily deployed to auto-approve April 21, 2024 23:55 — with GitHub Actions Inactive

qiyunz reviewed Apr 22, 2024
View reviewed changes

Jonathan Makungaand others added 2 commits April 22, 2024 09:48

Resolve PR Comments
785063a

Merge branch 'master' into master
2cad435

makungaj1 temporarily deployed to auto-approve April 22, 2024 16:49 — with GitHub Actions Inactive

makungaj1 temporarily deployed to auto-approve April 22, 2024 17:30 — with GitHub Actions Inactive

makungaj1 closed this Apr 22, 2024

Add Public facing APIs to display benchmark metrics and list deployme…
5cff65b
…nt configs

makungaj1 temporarily deployed to auto-approve April 17, 2024 04:14 — with GitHub Actions Inactive

qiyunz reviewed Apr 17, 2024
View reviewed changes

Merge branch 'master' into master
c70ee25

makungaj1 temporarily deployed to auto-approve April 17, 2024 16:15 — with GitHub Actions Inactive

jiapinw reviewed Apr 17, 2024
View reviewed changes

Comment threadsetup.py Outdated

jiapinw reviewed Apr 17, 2024
View reviewed changes

Comment threadsrc/sagemaker/jumpstart/model.py Outdated

qiyunz reviewed Apr 17, 2024
View reviewed changes

Comment threadsetup.py Outdated
Comment threadsrc/sagemaker/jumpstart/model.py Outdated

samruds suggested changes Apr 17, 2024
View reviewed changes

Merge branch 'master' into master
1a080aa

makungaj1 temporarily deployed to auto-approve April 18, 2024 01:04 — with GitHub Actions Inactive

Resolved PR Review comments
1431b5c

makungaj1 temporarily deployed to auto-approve April 18, 2024 05:19 — with GitHub Actions Inactive

Resolve PR Review Comments
3a2547b

makungaj1 temporarily deployed to auto-approve April 18, 2024 05:27 — with GitHub Actions Inactive

Add List and Display APIs in ModelBuilder
1dfa190

makungaj1 temporarily deployed to auto-approve April 18, 2024 05:34 — with GitHub Actions Inactive

Refactoring
9b80d07

makungaj1 temporarily deployed to auto-approve April 18, 2024 06:45 — with GitHub Actions Inactive

Refactoring
4669c9d

makungaj1 temporarily deployed to auto-approve April 18, 2024 07:09 — with GitHub Actions Inactive

Resolved PR review comments
f862797

makungaj1 temporarily deployed to auto-approve April 18, 2024 07:14 — with GitHub Actions Inactive

Merge branch 'master' into master
2b0fc3c

makungaj1 temporarily deployed to auto-approve April 18, 2024 17:23 — with GitHub Actions Inactive

qiyunz reviewed Apr 18, 2024
View reviewed changes

gwang111 reviewed Apr 19, 2024
View reviewed changes

Comment threadsrc/sagemaker/jumpstart/model.py Outdated

gwang111 reviewed Apr 19, 2024
View reviewed changes

Comment threadsrc/sagemaker/jumpstart/model.py Outdated

gwang111 reviewed Apr 19, 2024
View reviewed changes

Comment threadsrc/sagemaker/jumpstart/model.py Outdated

Refactoring
2a1414e

makungaj1 temporarily deployed to auto-approve April 19, 2024 17:05 — with GitHub Actions Inactive

samruds reviewed Apr 19, 2024
View reviewed changes

Comment threadsrc/sagemaker/jumpstart/model.py Outdated
Comment threadsrc/sagemaker/jumpstart/model.py Outdated
Comment threadsrc/sagemaker/jumpstart/types.py
Comment threadsrc/sagemaker/jumpstart/model.py Outdated
Comment threadsrc/sagemaker/jumpstart/types.py

qiyunz reviewed Apr 19, 2024
View reviewed changes

Comment threadsrc/sagemaker/jumpstart/model.py
Comment threadsrc/sagemaker/jumpstart/model.py Outdated

Refactoring
e5ee82e

makungaj1 temporarily deployed to auto-approve April 19, 2024 20:08 — with GitHub Actions Inactive

Resolve PR review comments
edd271d

makungaj1 temporarily deployed to auto-approve April 19, 2024 20:40 — with GitHub Actions Inactive

Resolve PR Review comments
84eece1

makungaj1 temporarily deployed to auto-approve April 19, 2024 20:47 — with GitHub Actions Inactive

Resolve PR review comments
1dd6854

makungaj1 temporarily deployed to auto-approve April 19, 2024 20:54 — with GitHub Actions Inactive

samruds approved these changes Apr 19, 2024
View reviewed changes

Refactoring
08edb27

makungaj1 temporarily deployed to auto-approve April 20, 2024 04:04 — with GitHub Actions Inactive

Add Unit tests
dd1b66e
Unit Tests

makungaj1 temporarily deployed to auto-approve April 21, 2024 18:50 — with GitHub Actions Inactive

Get Instance Rate
e878d75

makungaj1 temporarily deployed to auto-approve April 21, 2024 23:52 — with GitHub Actions Inactive

Refactoring
523e0b9

makungaj1 temporarily deployed to auto-approve April 21, 2024 23:55 — with GitHub Actions Inactive

qiyunz reviewed Apr 22, 2024
View reviewed changes

Jonathan Makungaand others added 2 commits April 22, 2024 09:48

Resolve PR Comments
785063a

Merge branch 'master' into master
2cad435

makungaj1 temporarily deployed to auto-approve April 22, 2024 16:49 — with GitHub Actions Inactive

makungaj1 temporarily deployed to auto-approve April 22, 2024 17:30 — with GitHub Actions Inactive

makungaj1 closed this Apr 22, 2024

Add Public facing APIs to display benchmark metrics and list deployme…
5cff65b
…nt configs

makungaj1 temporarily deployed to auto-approve April 17, 2024 04:14 — with GitHub Actions Inactive

qiyunz reviewed Apr 17, 2024
View reviewed changes

Merge branch 'master' into master
c70ee25

makungaj1 temporarily deployed to auto-approve April 17, 2024 16:15 — with GitHub Actions Inactive

jiapinw reviewed Apr 17, 2024
View reviewed changes

Comment threadsetup.py Outdated

jiapinw reviewed Apr 17, 2024
View reviewed changes

Comment threadsrc/sagemaker/jumpstart/model.py Outdated

qiyunz reviewed Apr 17, 2024
View reviewed changes

Comment threadsetup.py Outdated
Comment threadsrc/sagemaker/jumpstart/model.py Outdated

samruds suggested changes Apr 17, 2024
View reviewed changes

Merge branch 'master' into master
1a080aa

makungaj1 temporarily deployed to auto-approve April 18, 2024 01:04 — with GitHub Actions Inactive

Resolved PR Review comments
1431b5c

makungaj1 temporarily deployed to auto-approve April 18, 2024 05:19 — with GitHub Actions Inactive

Resolve PR Review Comments
3a2547b

makungaj1 temporarily deployed to auto-approve April 18, 2024 05:27 — with GitHub Actions Inactive

Add List and Display APIs in ModelBuilder
1dfa190

makungaj1 temporarily deployed to auto-approve April 18, 2024 05:34 — with GitHub Actions Inactive

Refactoring
9b80d07

makungaj1 temporarily deployed to auto-approve April 18, 2024 06:45 — with GitHub Actions Inactive

Refactoring
4669c9d

makungaj1 temporarily deployed to auto-approve April 18, 2024 07:09 — with GitHub Actions Inactive

Resolved PR review comments
f862797

makungaj1 temporarily deployed to auto-approve April 18, 2024 07:14 — with GitHub Actions Inactive

Merge branch 'master' into master
2b0fc3c

makungaj1 temporarily deployed to auto-approve April 18, 2024 17:23 — with GitHub Actions Inactive

qiyunz reviewed Apr 18, 2024
View reviewed changes

gwang111 reviewed Apr 19, 2024
View reviewed changes

Comment threadsrc/sagemaker/jumpstart/model.py Outdated

gwang111 reviewed Apr 19, 2024
View reviewed changes

Comment threadsrc/sagemaker/jumpstart/model.py Outdated

gwang111 reviewed Apr 19, 2024
View reviewed changes

Comment threadsrc/sagemaker/jumpstart/model.py Outdated

Refactoring
2a1414e

makungaj1 temporarily deployed to auto-approve April 19, 2024 17:05 — with GitHub Actions Inactive

samruds reviewed Apr 19, 2024
View reviewed changes

Comment threadsrc/sagemaker/jumpstart/model.py Outdated
Comment threadsrc/sagemaker/jumpstart/model.py Outdated
Comment threadsrc/sagemaker/jumpstart/types.py
Comment threadsrc/sagemaker/jumpstart/model.py Outdated
Comment threadsrc/sagemaker/jumpstart/types.py

qiyunz reviewed Apr 19, 2024
View reviewed changes

Comment threadsrc/sagemaker/jumpstart/model.py
Comment threadsrc/sagemaker/jumpstart/model.py Outdated

Refactoring
e5ee82e

makungaj1 temporarily deployed to auto-approve April 19, 2024 20:08 — with GitHub Actions Inactive

Resolve PR review comments
edd271d

makungaj1 temporarily deployed to auto-approve April 19, 2024 20:40 — with GitHub Actions Inactive

Resolve PR Review comments
84eece1

makungaj1 temporarily deployed to auto-approve April 19, 2024 20:47 — with GitHub Actions Inactive

Resolve PR review comments
1dd6854

makungaj1 temporarily deployed to auto-approve April 19, 2024 20:54 — with GitHub Actions Inactive

samruds approved these changes Apr 19, 2024
View reviewed changes

Refactoring
08edb27

makungaj1 temporarily deployed to auto-approve April 20, 2024 04:04 — with GitHub Actions Inactive

Add Unit tests
dd1b66e
Unit Tests

makungaj1 temporarily deployed to auto-approve April 21, 2024 18:50 — with GitHub Actions Inactive

Get Instance Rate
e878d75

makungaj1 temporarily deployed to auto-approve April 21, 2024 23:52 — with GitHub Actions Inactive

Refactoring
523e0b9

makungaj1 temporarily deployed to auto-approve April 21, 2024 23:55 — with GitHub Actions Inactive

qiyunz reviewed Apr 22, 2024
View reviewed changes

Jonathan Makungaand others added 2 commits April 22, 2024 09:48

Resolve PR Comments
785063a

Merge branch 'master' into master
2cad435

makungaj1 temporarily deployed to auto-approve April 22, 2024 16:49 — with GitHub Actions Inactive

makungaj1 temporarily deployed to auto-approve April 22, 2024 17:30 — with GitHub Actions Inactive

makungaj1 closed this Apr 22, 2024

Add Public facing APIs to display benchmark metrics and list deployme…
5cff65b
…nt configs

makungaj1 temporarily deployed to auto-approve April 17, 2024 04:14 — with GitHub Actions Inactive

qiyunz reviewed Apr 17, 2024
View reviewed changes

Merge branch 'master' into master
c70ee25

makungaj1 temporarily deployed to auto-approve April 17, 2024 16:15 — with GitHub Actions Inactive

jiapinw reviewed Apr 17, 2024
View reviewed changes

Comment threadsetup.py Outdated

jiapinw reviewed Apr 17, 2024
View reviewed changes

Comment threadsrc/sagemaker/jumpstart/model.py Outdated

qiyunz reviewed Apr 17, 2024
View reviewed changes

Comment threadsetup.py Outdated
Comment threadsrc/sagemaker/jumpstart/model.py Outdated

samruds suggested changes Apr 17, 2024
View reviewed changes

Merge branch 'master' into master
1a080aa

makungaj1 temporarily deployed to auto-approve April 18, 2024 01:04 — with GitHub Actions Inactive

Resolved PR Review comments
1431b5c

makungaj1 temporarily deployed to auto-approve April 18, 2024 05:19 — with GitHub Actions Inactive

Resolve PR Review Comments
3a2547b

makungaj1 temporarily deployed to auto-approve April 18, 2024 05:27 — with GitHub Actions Inactive

Add List and Display APIs in ModelBuilder
1dfa190

makungaj1 temporarily deployed to auto-approve April 18, 2024 05:34 — with GitHub Actions Inactive

Refactoring
9b80d07

makungaj1 temporarily deployed to auto-approve April 18, 2024 06:45 — with GitHub Actions Inactive

Refactoring
4669c9d

makungaj1 temporarily deployed to auto-approve April 18, 2024 07:09 — with GitHub Actions Inactive

Resolved PR review comments
f862797

makungaj1 temporarily deployed to auto-approve April 18, 2024 07:14 — with GitHub Actions Inactive

Merge branch 'master' into master
2b0fc3c

makungaj1 temporarily deployed to auto-approve April 18, 2024 17:23 — with GitHub Actions Inactive

qiyunz reviewed Apr 18, 2024
View reviewed changes

gwang111 reviewed Apr 19, 2024
View reviewed changes

Comment threadsrc/sagemaker/jumpstart/model.py Outdated

gwang111 reviewed Apr 19, 2024
View reviewed changes

Comment threadsrc/sagemaker/jumpstart/model.py Outdated

gwang111 reviewed Apr 19, 2024
View reviewed changes

Comment threadsrc/sagemaker/jumpstart/model.py Outdated

Refactoring
2a1414e

makungaj1 temporarily deployed to auto-approve April 19, 2024 17:05 — with GitHub Actions Inactive

samruds reviewed Apr 19, 2024
View reviewed changes

Comment threadsrc/sagemaker/jumpstart/model.py Outdated
Comment threadsrc/sagemaker/jumpstart/model.py Outdated
Comment threadsrc/sagemaker/jumpstart/types.py
Comment threadsrc/sagemaker/jumpstart/model.py Outdated
Comment threadsrc/sagemaker/jumpstart/types.py

qiyunz reviewed Apr 19, 2024
View reviewed changes

Comment threadsrc/sagemaker/jumpstart/model.py
Comment threadsrc/sagemaker/jumpstart/model.py Outdated

Refactoring
e5ee82e

makungaj1 temporarily deployed to auto-approve April 19, 2024 20:08 — with GitHub Actions Inactive

Resolve PR review comments
edd271d

makungaj1 temporarily deployed to auto-approve April 19, 2024 20:40 — with GitHub Actions Inactive

Resolve PR Review comments
84eece1

makungaj1 temporarily deployed to auto-approve April 19, 2024 20:47 — with GitHub Actions Inactive

Resolve PR review comments
1dd6854

makungaj1 temporarily deployed to auto-approve April 19, 2024 20:54 — with GitHub Actions Inactive

samruds approved these changes Apr 19, 2024
View reviewed changes

Refactoring
08edb27

makungaj1 temporarily deployed to auto-approve April 20, 2024 04:04 — with GitHub Actions Inactive

Add Unit tests
dd1b66e
Unit Tests

makungaj1 temporarily deployed to auto-approve April 21, 2024 18:50 — with GitHub Actions Inactive

Get Instance Rate
e878d75

makungaj1 temporarily deployed to auto-approve April 21, 2024 23:52 — with GitHub Actions Inactive

Refactoring
523e0b9

makungaj1 temporarily deployed to auto-approve April 21, 2024 23:55 — with GitHub Actions Inactive

qiyunz reviewed Apr 22, 2024
View reviewed changes

Jonathan Makungaand others added 2 commits April 22, 2024 09:48

Resolve PR Comments
785063a

Merge branch 'master' into master
2cad435

makungaj1 temporarily deployed to auto-approve April 22, 2024 16:49 — with GitHub Actions Inactive

makungaj1 temporarily deployed to auto-approve April 22, 2024 17:30 — with GitHub Actions Inactive

makungaj1 closed this Apr 22, 2024

Add Public facing APIs to display benchmark metrics and list deployme…
5cff65b
…nt configs

makungaj1 temporarily deployed to auto-approve April 17, 2024 04:14 — with GitHub Actions Inactive

qiyunz reviewed Apr 17, 2024
View reviewed changes

Merge branch 'master' into master
c70ee25

makungaj1 temporarily deployed to auto-approve April 17, 2024 16:15 — with GitHub Actions Inactive

jiapinw reviewed Apr 17, 2024
View reviewed changes

Comment threadsetup.py Outdated

jiapinw reviewed Apr 17, 2024
View reviewed changes

Comment threadsrc/sagemaker/jumpstart/model.py Outdated

qiyunz reviewed Apr 17, 2024
View reviewed changes

Comment threadsetup.py Outdated
Comment threadsrc/sagemaker/jumpstart/model.py Outdated

samruds suggested changes Apr 17, 2024
View reviewed changes

Merge branch 'master' into master
1a080aa

makungaj1 temporarily deployed to auto-approve April 18, 2024 01:04 — with GitHub Actions Inactive

Resolved PR Review comments
1431b5c

makungaj1 temporarily deployed to auto-approve April 18, 2024 05:19 — with GitHub Actions Inactive

Resolve PR Review Comments
3a2547b

makungaj1 temporarily deployed to auto-approve April 18, 2024 05:27 — with GitHub Actions Inactive

Add List and Display APIs in ModelBuilder
1dfa190

makungaj1 temporarily deployed to auto-approve April 18, 2024 05:34 — with GitHub Actions Inactive

Refactoring
9b80d07

makungaj1 temporarily deployed to auto-approve April 18, 2024 06:45 — with GitHub Actions Inactive

Refactoring
4669c9d

makungaj1 temporarily deployed to auto-approve April 18, 2024 07:09 — with GitHub Actions Inactive

Resolved PR review comments
f862797

makungaj1 temporarily deployed to auto-approve April 18, 2024 07:14 — with GitHub Actions Inactive

Merge branch 'master' into master
2b0fc3c

makungaj1 temporarily deployed to auto-approve April 18, 2024 17:23 — with GitHub Actions Inactive

qiyunz reviewed Apr 18, 2024
View reviewed changes

gwang111 reviewed Apr 19, 2024
View reviewed changes

Comment threadsrc/sagemaker/jumpstart/model.py Outdated

gwang111 reviewed Apr 19, 2024
View reviewed changes

Comment threadsrc/sagemaker/jumpstart/model.py Outdated

gwang111 reviewed Apr 19, 2024
View reviewed changes

Comment threadsrc/sagemaker/jumpstart/model.py Outdated

Refactoring
2a1414e

makungaj1 temporarily deployed to auto-approve April 19, 2024 17:05 — with GitHub Actions Inactive

samruds reviewed Apr 19, 2024
View reviewed changes

Comment threadsrc/sagemaker/jumpstart/model.py Outdated
Comment threadsrc/sagemaker/jumpstart/model.py Outdated
Comment threadsrc/sagemaker/jumpstart/types.py
Comment threadsrc/sagemaker/jumpstart/model.py Outdated
Comment threadsrc/sagemaker/jumpstart/types.py

qiyunz reviewed Apr 19, 2024
View reviewed changes

Comment threadsrc/sagemaker/jumpstart/model.py
Comment threadsrc/sagemaker/jumpstart/model.py Outdated

Refactoring
e5ee82e

makungaj1 temporarily deployed to auto-approve April 19, 2024 20:08 — with GitHub Actions Inactive

Resolve PR review comments
edd271d

makungaj1 temporarily deployed to auto-approve April 19, 2024 20:40 — with GitHub Actions Inactive

Resolve PR Review comments
84eece1

makungaj1 temporarily deployed to auto-approve April 19, 2024 20:47 — with GitHub Actions Inactive

Resolve PR review comments
1dd6854

makungaj1 temporarily deployed to auto-approve April 19, 2024 20:54 — with GitHub Actions Inactive

samruds approved these changes Apr 19, 2024
View reviewed changes

Refactoring
08edb27

makungaj1 temporarily deployed to auto-approve April 20, 2024 04:04 — with GitHub Actions Inactive

Add Unit tests
dd1b66e
Unit Tests

makungaj1 temporarily deployed to auto-approve April 21, 2024 18:50 — with GitHub Actions Inactive

Get Instance Rate
e878d75

makungaj1 temporarily deployed to auto-approve April 21, 2024 23:52 — with GitHub Actions Inactive

Refactoring
523e0b9

makungaj1 temporarily deployed to auto-approve April 21, 2024 23:55 — with GitHub Actions Inactive

qiyunz reviewed Apr 22, 2024
View reviewed changes

Jonathan Makungaand others added 2 commits April 22, 2024 09:48

Resolve PR Comments
785063a

Merge branch 'master' into master
2cad435

makungaj1 temporarily deployed to auto-approve April 22, 2024 16:49 — with GitHub Actions Inactive

makungaj1 temporarily deployed to auto-approve April 22, 2024 17:30 — with GitHub Actions Inactive

makungaj1 closed this Apr 22, 2024

Add Public facing APIs to display benchmark metrics and list deployme…
5cff65b
…nt configs

makungaj1 temporarily deployed to auto-approve April 17, 2024 04:14 — with GitHub Actions Inactive

qiyunz reviewed Apr 17, 2024
View reviewed changes

Merge branch 'master' into master
c70ee25

makungaj1 temporarily deployed to auto-approve April 17, 2024 16:15 — with GitHub Actions Inactive

jiapinw reviewed Apr 17, 2024
View reviewed changes

Comment threadsetup.py Outdated

jiapinw reviewed Apr 17, 2024
View reviewed changes

Comment threadsrc/sagemaker/jumpstart/model.py Outdated

qiyunz reviewed Apr 17, 2024
View reviewed changes

Comment threadsetup.py Outdated
Comment threadsrc/sagemaker/jumpstart/model.py Outdated

samruds suggested changes Apr 17, 2024
View reviewed changes

Merge branch 'master' into master
1a080aa

makungaj1 temporarily deployed to auto-approve April 18, 2024 01:04 — with GitHub Actions Inactive

Resolved PR Review comments
1431b5c

makungaj1 temporarily deployed to auto-approve April 18, 2024 05:19 — with GitHub Actions Inactive

Resolve PR Review Comments
3a2547b

makungaj1 temporarily deployed to auto-approve April 18, 2024 05:27 — with GitHub Actions Inactive

Add List and Display APIs in ModelBuilder
1dfa190

makungaj1 temporarily deployed to auto-approve April 18, 2024 05:34 — with GitHub Actions Inactive

Refactoring
9b80d07

makungaj1 temporarily deployed to auto-approve April 18, 2024 06:45 — with GitHub Actions Inactive

Refactoring
4669c9d

makungaj1 temporarily deployed to auto-approve April 18, 2024 07:09 — with GitHub Actions Inactive

Resolved PR review comments
f862797

makungaj1 temporarily deployed to auto-approve April 18, 2024 07:14 — with GitHub Actions Inactive

Merge branch 'master' into master
2b0fc3c

makungaj1 temporarily deployed to auto-approve April 18, 2024 17:23 — with GitHub Actions Inactive

qiyunz reviewed Apr 18, 2024
View reviewed changes

gwang111 reviewed Apr 19, 2024
View reviewed changes

Comment threadsrc/sagemaker/jumpstart/model.py Outdated

gwang111 reviewed Apr 19, 2024
View reviewed changes

Comment threadsrc/sagemaker/jumpstart/model.py Outdated

gwang111 reviewed Apr 19, 2024
View reviewed changes

Comment threadsrc/sagemaker/jumpstart/model.py Outdated

Refactoring
2a1414e

makungaj1 temporarily deployed to auto-approve April 19, 2024 17:05 — with GitHub Actions Inactive

samruds reviewed Apr 19, 2024
View reviewed changes

Comment threadsrc/sagemaker/jumpstart/model.py Outdated
Comment threadsrc/sagemaker/jumpstart/model.py Outdated
Comment threadsrc/sagemaker/jumpstart/types.py
Comment threadsrc/sagemaker/jumpstart/model.py Outdated
Comment threadsrc/sagemaker/jumpstart/types.py

qiyunz reviewed Apr 19, 2024
View reviewed changes

Comment threadsrc/sagemaker/jumpstart/model.py
Comment threadsrc/sagemaker/jumpstart/model.py Outdated

Refactoring
e5ee82e

makungaj1 temporarily deployed to auto-approve April 19, 2024 20:08 — with GitHub Actions Inactive

Resolve PR review comments
edd271d

makungaj1 temporarily deployed to auto-approve April 19, 2024 20:40 — with GitHub Actions Inactive

Resolve PR Review comments
84eece1

makungaj1 temporarily deployed to auto-approve April 19, 2024 20:47 — with GitHub Actions Inactive

Resolve PR review comments
1dd6854

makungaj1 temporarily deployed to auto-approve April 19, 2024 20:54 — with GitHub Actions Inactive

samruds approved these changes Apr 19, 2024
View reviewed changes

Refactoring
08edb27

makungaj1 temporarily deployed to auto-approve April 20, 2024 04:04 — with GitHub Actions Inactive

Add Unit tests
dd1b66e
Unit Tests

makungaj1 temporarily deployed to auto-approve April 21, 2024 18:50 — with GitHub Actions Inactive

Get Instance Rate
e878d75

makungaj1 temporarily deployed to auto-approve April 21, 2024 23:52 — with GitHub Actions Inactive

Refactoring
523e0b9

makungaj1 temporarily deployed to auto-approve April 21, 2024 23:55 — with GitHub Actions Inactive

qiyunz reviewed Apr 22, 2024
View reviewed changes

Jonathan Makungaand others added 2 commits April 22, 2024 09:48

Resolve PR Comments
785063a

Merge branch 'master' into master
2cad435

makungaj1 temporarily deployed to auto-approve April 22, 2024 16:49 — with GitHub Actions Inactive

makungaj1 temporarily deployed to auto-approve April 22, 2024 17:30 — with GitHub Actions Inactive

makungaj1 closed this Apr 22, 2024

Add Public facing APIs to display benchmark metrics and list deployme…
5cff65b
…nt configs

makungaj1 temporarily deployed to auto-approve April 17, 2024 04:14 — with GitHub Actions Inactive

qiyunz reviewed Apr 17, 2024
View reviewed changes

Merge branch 'master' into master
c70ee25

makungaj1 temporarily deployed to auto-approve April 17, 2024 16:15 — with GitHub Actions Inactive

jiapinw reviewed Apr 17, 2024
View reviewed changes

Comment threadsetup.py Outdated

jiapinw reviewed Apr 17, 2024
View reviewed changes

Comment threadsrc/sagemaker/jumpstart/model.py Outdated

qiyunz reviewed Apr 17, 2024
View reviewed changes

Comment threadsetup.py Outdated
Comment threadsrc/sagemaker/jumpstart/model.py Outdated

samruds suggested changes Apr 17, 2024
View reviewed changes

Merge branch 'master' into master
1a080aa

makungaj1 temporarily deployed to auto-approve April 18, 2024 01:04 — with GitHub Actions Inactive

Resolved PR Review comments
1431b5c

makungaj1 temporarily deployed to auto-approve April 18, 2024 05:19 — with GitHub Actions Inactive

Resolve PR Review Comments
3a2547b

makungaj1 temporarily deployed to auto-approve April 18, 2024 05:27 — with GitHub Actions Inactive

Add List and Display APIs in ModelBuilder
1dfa190

makungaj1 temporarily deployed to auto-approve April 18, 2024 05:34 — with GitHub Actions Inactive

Refactoring
9b80d07

makungaj1 temporarily deployed to auto-approve April 18, 2024 06:45 — with GitHub Actions Inactive

Refactoring
4669c9d

makungaj1 temporarily deployed to auto-approve April 18, 2024 07:09 — with GitHub Actions Inactive

Resolved PR review comments
f862797

makungaj1 temporarily deployed to auto-approve April 18, 2024 07:14 — with GitHub Actions Inactive

Merge branch 'master' into master
2b0fc3c

makungaj1 temporarily deployed to auto-approve April 18, 2024 17:23 — with GitHub Actions Inactive

qiyunz reviewed Apr 18, 2024
View reviewed changes

gwang111 reviewed Apr 19, 2024
View reviewed changes

Comment threadsrc/sagemaker/jumpstart/model.py Outdated

gwang111 reviewed Apr 19, 2024
View reviewed changes

Comment threadsrc/sagemaker/jumpstart/model.py Outdated

gwang111 reviewed Apr 19, 2024
View reviewed changes

Comment threadsrc/sagemaker/jumpstart/model.py Outdated

Refactoring
2a1414e

makungaj1 temporarily deployed to auto-approve April 19, 2024 17:05 — with GitHub Actions Inactive

samruds reviewed Apr 19, 2024
View reviewed changes

Comment threadsrc/sagemaker/jumpstart/model.py Outdated
Comment threadsrc/sagemaker/jumpstart/model.py Outdated
Comment threadsrc/sagemaker/jumpstart/types.py
Comment threadsrc/sagemaker/jumpstart/model.py Outdated
Comment threadsrc/sagemaker/jumpstart/types.py

qiyunz reviewed Apr 19, 2024
View reviewed changes

Comment threadsrc/sagemaker/jumpstart/model.py
Comment threadsrc/sagemaker/jumpstart/model.py Outdated

Refactoring
e5ee82e

makungaj1 temporarily deployed to auto-approve April 19, 2024 20:08 — with GitHub Actions Inactive

Resolve PR review comments
edd271d

makungaj1 temporarily deployed to auto-approve April 19, 2024 20:40 — with GitHub Actions Inactive

Resolve PR Review comments
84eece1

makungaj1 temporarily deployed to auto-approve April 19, 2024 20:47 — with GitHub Actions Inactive

Resolve PR review comments
1dd6854

makungaj1 temporarily deployed to auto-approve April 19, 2024 20:54 — with GitHub Actions Inactive

samruds approved these changes Apr 19, 2024
View reviewed changes

Refactoring
08edb27

makungaj1 temporarily deployed to auto-approve April 20, 2024 04:04 — with GitHub Actions Inactive

Add Unit tests
dd1b66e
Unit Tests

makungaj1 temporarily deployed to auto-approve April 21, 2024 18:50 — with GitHub Actions Inactive

Get Instance Rate
e878d75

makungaj1 temporarily deployed to auto-approve April 21, 2024 23:52 — with GitHub Actions Inactive

Refactoring
523e0b9

makungaj1 temporarily deployed to auto-approve April 21, 2024 23:55 — with GitHub Actions Inactive

qiyunz reviewed Apr 22, 2024
View reviewed changes

Jonathan Makungaand others added 2 commits April 22, 2024 09:48

Resolve PR Comments
785063a

Merge branch 'master' into master
2cad435

makungaj1 temporarily deployed to auto-approve April 22, 2024 16:49 — with GitHub Actions Inactive

makungaj1 temporarily deployed to auto-approve April 22, 2024 17:30 — with GitHub Actions Inactive

makungaj1 closed this Apr 22, 2024

Add Public facing APIs to display benchmark metrics and list deployme…
5cff65b
…nt configs

makungaj1 temporarily deployed to auto-approve April 17, 2024 04:14 — with GitHub Actions Inactive

qiyunz reviewed Apr 17, 2024
View reviewed changes

Merge branch 'master' into master
c70ee25

makungaj1 temporarily deployed to auto-approve April 17, 2024 16:15 — with GitHub Actions Inactive

jiapinw reviewed Apr 17, 2024
View reviewed changes

Comment threadsetup.py Outdated

jiapinw reviewed Apr 17, 2024
View reviewed changes

Comment threadsrc/sagemaker/jumpstart/model.py Outdated

qiyunz reviewed Apr 17, 2024
View reviewed changes

Comment threadsetup.py Outdated
Comment threadsrc/sagemaker/jumpstart/model.py Outdated

samruds suggested changes Apr 17, 2024
View reviewed changes

Merge branch 'master' into master
1a080aa

makungaj1 temporarily deployed to auto-approve April 18, 2024 01:04 — with GitHub Actions Inactive

Resolved PR Review comments
1431b5c

makungaj1 temporarily deployed to auto-approve April 18, 2024 05:19 — with GitHub Actions Inactive

Resolve PR Review Comments
3a2547b

makungaj1 temporarily deployed to auto-approve April 18, 2024 05:27 — with GitHub Actions Inactive

Add List and Display APIs in ModelBuilder
1dfa190

makungaj1 temporarily deployed to auto-approve April 18, 2024 05:34 — with GitHub Actions Inactive

Refactoring
9b80d07

makungaj1 temporarily deployed to auto-approve April 18, 2024 06:45 — with GitHub Actions Inactive

Refactoring
4669c9d

makungaj1 temporarily deployed to auto-approve April 18, 2024 07:09 — with GitHub Actions Inactive

Resolved PR review comments
f862797

makungaj1 temporarily deployed to auto-approve April 18, 2024 07:14 — with GitHub Actions Inactive

Merge branch 'master' into master
2b0fc3c

makungaj1 temporarily deployed to auto-approve April 18, 2024 17:23 — with GitHub Actions Inactive

qiyunz reviewed Apr 18, 2024
View reviewed changes

gwang111 reviewed Apr 19, 2024
View reviewed changes

Comment threadsrc/sagemaker/jumpstart/model.py Outdated

gwang111 reviewed Apr 19, 2024
View reviewed changes

Comment threadsrc/sagemaker/jumpstart/model.py Outdated

gwang111 reviewed Apr 19, 2024
View reviewed changes

Comment threadsrc/sagemaker/jumpstart/model.py Outdated

Refactoring
2a1414e

makungaj1 temporarily deployed to auto-approve April 19, 2024 17:05 — with GitHub Actions Inactive

samruds reviewed Apr 19, 2024
View reviewed changes

Comment threadsrc/sagemaker/jumpstart/model.py Outdated
Comment threadsrc/sagemaker/jumpstart/model.py Outdated
Comment threadsrc/sagemaker/jumpstart/types.py
Comment threadsrc/sagemaker/jumpstart/model.py Outdated
Comment threadsrc/sagemaker/jumpstart/types.py

qiyunz reviewed Apr 19, 2024
View reviewed changes

Comment threadsrc/sagemaker/jumpstart/model.py
Comment threadsrc/sagemaker/jumpstart/model.py Outdated

Refactoring
e5ee82e

makungaj1 temporarily deployed to auto-approve April 19, 2024 20:08 — with GitHub Actions Inactive

Resolve PR review comments
edd271d

makungaj1 temporarily deployed to auto-approve April 19, 2024 20:40 — with GitHub Actions Inactive

Resolve PR Review comments
84eece1

makungaj1 temporarily deployed to auto-approve April 19, 2024 20:47 — with GitHub Actions Inactive

Resolve PR review comments
1dd6854

makungaj1 temporarily deployed to auto-approve April 19, 2024 20:54 — with GitHub Actions Inactive

samruds approved these changes Apr 19, 2024
View reviewed changes

Refactoring
08edb27

makungaj1 temporarily deployed to auto-approve April 20, 2024 04:04 — with GitHub Actions Inactive

Add Unit tests
dd1b66e
Unit Tests

makungaj1 temporarily deployed to auto-approve April 21, 2024 18:50 — with GitHub Actions Inactive

Get Instance Rate
e878d75

makungaj1 temporarily deployed to auto-approve April 21, 2024 23:52 — with GitHub Actions Inactive

Refactoring
523e0b9

makungaj1 temporarily deployed to auto-approve April 21, 2024 23:55 — with GitHub Actions Inactive

qiyunz reviewed Apr 22, 2024
View reviewed changes

Jonathan Makungaand others added 2 commits April 22, 2024 09:48

Resolve PR Comments
785063a

Merge branch 'master' into master
2cad435

makungaj1 temporarily deployed to auto-approve April 22, 2024 16:49 — with GitHub Actions Inactive

makungaj1 temporarily deployed to auto-approve April 22, 2024 17:30 — with GitHub Actions Inactive

makungaj1 closed this Apr 22, 2024

Uh oh!

Conversation

makungaj1 commented Apr 17, 2024• edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Merge Checklist

General

Tests

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

samruds commented Apr 19, 2024

Uh oh!

Uh oh!

Uh oh!

makungaj1 commented Apr 19, 2024

Uh oh!

samruds commented Apr 19, 2024• edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

5 participants

Uh oh!

Conversation

makungaj1 commented Apr 17, 2024• edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Merge Checklist

General

Tests

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

makungaj1 commented Apr 17, 2024•
edited
Loading

samruds commented Apr 19, 2024•
edited
Loading

makungaj1 commented Apr 17, 2024•
edited
Loading

samruds commented Apr 19, 2024•
edited
Loading