Executing sagemaker.get_execution_role() locally

[opringle]

Please fill out the form below.

System Information

• Framework (e.g. TensorFlow) / Algorithm (e.g. KMeans): MXNet/None
• Framework Version: 1.1.0
• Python Version: 3.5
• CPU or GPU: CPU
• Python SDK Version: 1.7.0
• Are you using a custom image: No

Describe the problem

• I want to run SageMaker without a notebook instance, from a script on my local machine, for various reasons.
• I can successfully start SageMaker jobs by passing the ARN string from my AWS role to my script
• However, I cannot retrieve the ARN string programatically using sagemaker.get_execution_role(). Instead, I receive a botocore.errorfactory.NoSuchEntityException.

Minimal repro / logs

To reproduce the problem:

Script:

import sagemaker
import boto3

session = boto3.Session(profile_name='personal')
sagemaker_session = sagemaker.Session(boto_session=session)
role = sagemaker.get_execution_role(sagemaker_session=sagemaker_session)

Credentials:

[personal]
aws_secret_access_key = ******************
aws_access_key_id = *******************
region = us-west-2

Error:

Traceback (most recent call last):
 File "mwe.py", line 8, in <module>
 role = sagemaker.get_execution_role(sagemaker_session=sagemaker_session)
 File "/Users/opringle/.virtualenvs/vdcnn/lib/python3.6/site-packages/sagemaker/session.py", line 936, in get_execution_role
 arn = sagemaker_session.get_caller_identity_arn()
 File "/Users/opringle/.virtualenvs/vdcnn/lib/python3.6/site-packages/sagemaker/session.py", line 766, in get_caller_identity_arn
 role = self.boto_session.client('iam').get_role(RoleName=role_name)['Role']['Arn']
 File "/Users/opringle/.virtualenvs/vdcnn/lib/python3.6/site-packages/botocore/client.py", line 314, in _api_call
 return self._make_api_call(operation_name, kwargs)
 File "/Users/opringle/.virtualenvs/vdcnn/lib/python3.6/site-packages/botocore/client.py", line 612, in _make_api_call
 raise error_class(parsed_response, operation_name)
botocore.errorfactory.NoSuchEntityException: An error occurred (NoSuchEntity) when calling the GetRole operation: The user with name oliver_pringle cannot be found.

• Exact command to reproduce: pip install sagemaker && python mwe.py

[yangaws]

Hi @opringle ,

The problem is, the get_execution_role() method is only used on AWS SageMaker notebook instances. So if you use it locally, it won't correctly parse your credential (from your stacktrace, I think you are using IAM user credential).

So if you want to use sagemaker locally, you can create an IAM role with enough SageMaker access permission. Then just directly use that role in your code.

Feel free to reopen this if you have more questions.

Thanks

[leopd]

This is really a pretty bad experience. get_execution_role() sounds like it's going to just figure out all the IAM/role/confusion/whatever to make SageMaker work. And on a notebook instance it does. But if you run that same code on your laptop it fails, sending customers into IAM/role/confusion limbo.

[leopd]

Without this it's basically impossible to write a simple set of code that works both on a SageMaker notebook instance and anywhere else. Which is a real barrier to people who want to build the SageMaker ecosystem.

[laurenyu]

understood. definitely agree that the SDK can do better here. I'll leave this issue open as a feature request, and hopefully we can prioritize this work in the near future. Thanks @leopd!

[thomelane]

Also having issues here, +1 to smoothing it out.

[Soypete]

same

[iluoyi]

A temp solution is re-use the IAM role attached to your notebook (when you create the notebook, you had one there). You can get its arn from IAM console.

[stevehawley]

I think local mode should work offline, what need is there to check credentials when running locally?