Arguments for list_entries are confusing

[daniel-sanche]

client.list_entries takes in page_size and page_token arguments. It's not clear to me what a page_token should look like, and page size doesn't seem to have an affect (it says it should determine the "maximum number of entries to return", but that doesn't seem true).

We should look into these and make sure they are well tested and documented

b/174842529

[bbayles]

Another problem: if an error is encountered during paging, there seems to be no way to resume from where you were. list_entries returns a generator now, which doesn't allow access to the page_token.

[daniel-sanche]

What kind of error are you running into? Do you have any code that can replicate the issue?

[bbayles]

Suppose I do:

for flow_entry in logging_client.list_entries(
 filter_='whatever filters',
 resource_names=[f'projects/whatever-project'],
):
 pass

This handles paging for me, but if one of the page calls returns a 429 (slow down), I'm stuck - I can't resume paging from where I was without stating over from the first page. (set a small page size to be more likely to hit the API limit)

Or can I? I would love to be wrong on this. If I am, how would I get the page_token?

[bbayles]

To say it more generally: list_entries does paging for me. Suppose that one of the calls during paging fails for any reason. Does the current implementation of list_entries allow me to recover without starting over? If it does, it's not at all clear from the docs or source.

[daniel-sanche]

That's a good question. I'm going to try to look into improving the way list_entries is handled when I have a chance. unfortunately since it's an API breaking change, any changes will have to wait for the next major release, and likely not this quarter.

If this is causing issues in your work now, you can look into using the low-level auto-generated LoggingServiceV2Client.list_log_entries directly. It's not as idiomatic and harder to work with than the regular logging_client.list_entries, but it should give you more control

[bbayles]

Many thanks. Could you explain succinctly what list_log_entries is vs. list_entries? The latter does not actually call the former, from what I can tell.

For what it's worth, the old version of this library (before v2) worked! The pager returned by list_entries kept track of the next_page_token.

[daniel-sanche]

For context, we use automated code generation to generate the majority of GCP client libraries. But for some like logging, we added an idiomatic layer on top to better integrate with language features (for Python, we want you to be able to send logs using the standard 'logging' library, etc). list_log_entries is the autogenerated layer, and list_entires is a higher-level client built on the other.

Eventually we want the handwritten layer to be used for 100% of tasks, but we have some work to do before it's fully there.

If you follow the call stack, youll see list_enteries calls self.logging_api.list_entries, then logging_api calls _gapic.make_logging_api, which creates a LoggingServiceV2Client instance.

Before 2.0, you were likely using just the auto-generated code, The autogenerated and hand-written code used to live in different folders, but we combined them in the latest release

[bbayles]

Thanks for the help - I see the connection to list_log_entries now. Note that your link goes to some tests, however.

Could I request that the v2 docs be updated to point out that errors during paging are fatal? If this had been in the UPGRADING document I would not have upgraded!