Skip to content

Switching datastore cursors to URLsafe base64 encoding. - #1180

Merged
tseaver merged 1 commit into
googleapis:masterfrom
theacodes:master
Oct 14, 2015
Merged

Switching datastore cursors to URLsafe base64 encoding.#1180
tseaver merged 1 commit into
googleapis:masterfrom
theacodes:master

Conversation

@theacodes

Copy link
Copy Markdown
Contributor

@googlebotgooglebot added the cla: yes This human has signed the Contributor License Agreement. label Oct 14, 2015
@tseaver

Copy link
Copy Markdown
Contributor

Given that our code only sends the cursor as a member of a protobuf passed as the body of a POST request (with Content-Type: application/x-protobuf), I don't know why the urlsafe_ encoding variants should matter: the cursor never appears as an item in a URL path or query string, nor even in a body encoded via www-form-urlencoded/multipart-formdata. Do we know whether the back-end uses the urlsafe_ variant to encode the cursor?

@jonparrott Were you able to reproduce GoogleCloudPlatform/getting-started-python#8 before this patch, and then have it not fail with the patch?

@dhermes

Copy link
Copy Markdown
Contributor

I don't know why the urlsafe_ encoding variants should matter

For users creating URLs to page through search results with the cursors


Do we know whether the back-end uses the urlsafe_ variant to encode the cursor?

The backend uses bytes. The only reason we encode bytes as base64 is for our end-users and we decode it when bringing it back in to a protobuf (e.g. if the user gets a base64 encoded cursor from a next_page() and then they use that as an end cursor).

@theacodes

Copy link
Copy Markdown
ContributorAuthor

Inside of gcloud-python, the cursor is never passed around in a URL or query string, but in user applications it's a common pattern:

deflist():
cursor=request.params.get('cursor', None)
query=ds.query(kind='Book', order=['title'])
it=query.fetch(limit=limit, start_cursor=cursor)
entities, more_results, cursor=it.next_page()
return"Next page: {}".format(url_for('list', cursor=cursor))

(abridged)

Returns something like:

/list?cursor=CkUKEwoFdGl0bGUSChoIdGhlb2RvcmUSKmoVc35iaWd0YWJsZS1zZmFuLXRlc3QxchELEgRCb29rGICAgIC+goMKDBgAIAA=

which fails due to the + present.

tseaver added a commit that referenced this pull request Oct 14, 2015
Switching datastore cursors to URLsafe base64 encoding.
@tseaver
tseaver merged commit c148adf into googleapis:masterOct 14, 2015
@theacodes

Copy link
Copy Markdown
ContributorAuthor

Thanks, @tseaver. I'll be on the lookout for the next release so I can update getting-started-python.

@dhermesdhermes mentioned this pull request Jan 25, 2016
parthea pushed a commit that referenced this pull request Nov 24, 2025
* feat: Allow jobs to be run in a different project
* Update test_sqlalchemy_bigquery_remote.py
---------
Co-authored-by: Lingqing Gan <lingqing.gan@gmail.com>
parthea added a commit that referenced this pull request Nov 26, 2025
Source-Link: googleapis/synthtool@e3a1277
Post-Processor: gcr.io/cloud-devrel-public-resources/owlbot-python:latest@sha256:452901c74a22f9b9a3bd02bce780b8e8805c97270d424684bff809ce5be8c2a2
Co-authored-by: Owl Bot <gcf-owl-bot[bot]@users.noreply.github.com>
Co-authored-by: Anthonios Partheniou <partheniou@google.com>
Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Labels

cla: yesThis human has signed the Contributor License Agreement.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants

@theacodes@tseaver@dhermes@googlebot