Skip to content

Added chunk_size to Response.iter_bytes() and Response.aiter_bytes() (#394) - #1271

Closed
cdeler wants to merge 2 commits into
encode:masterfrom
cdeler:adding-streaming-by-chunks-to-response
Closed

Added chunk_size to Response.iter_bytes() and Response.aiter_bytes() (#394)#1271
cdeler wants to merge 2 commits into
encode:masterfrom
cdeler:adding-streaming-by-chunks-to-response

Conversation

@cdeler

@cdelercdeler commented Sep 9, 2020

Copy link
Copy Markdown
Contributor

Working on #394 , I added the chunk_size argument to Response.iter_bytes() and Response.aiter_bytes() wrapping these iterators by drain_by_chunks() and async_drain_by_chunks() (I didn't use decorators to make debugging easier). The same way might be applied to (a)iter_text

But I still don't know what the expected behaviour of (a)iter_line and (a)iter_raw (so therefore I omit them in the PR)

For now the code might look like:

>>>importhttpx>>>response=httpx.Response(200, content=b"1234567890")
>>>foritinresponse.iter_bytes(3):
>>>print(it.decode())
1234567890>>># it's done for requests compatibility: >>># https://github.com/encode/httpx/issues/394#issuecomment-598820965>>>foritinresponse.iter_bytes(None):
>>>print(it.decode())
1234567890

@cdeler
cdelerforce-pushed the adding-streaming-by-chunks-to-response branch from e2f38ba to 3aab30bCompareSeptember 9, 2020 11:09
@cdeler
cdeler requested a review from a teamSeptember 9, 2020 12:58
@cdeler
cdelerforce-pushed the adding-streaming-by-chunks-to-response branch 2 times, most recently from b63a642 to aceaa56CompareSeptember 9, 2020 13:45
@lovelydinosaur

Copy link
Copy Markdown
Contributor

This one's a bit fiddly, so I'm going to defer on reviewing it for a little bit.

@cdeler
cdelerforce-pushed the adding-streaming-by-chunks-to-response branch from aceaa56 to 60ea230CompareSeptember 10, 2020 13:18
@lovelydinosaur

Copy link
Copy Markdown
Contributor

So, I'd been planning for us to take an approach more along these lines... #1277

That adds a decoder class,ByteChunker which has the same kind of interface as all the other decoders,
and which doesn't involve having separate sync + async implementations.

I've not quite finished that work off, since it still needs:

  • Using ByteChunker within iter_bytes.
  • Adding a corresponding TextChunker class.
  • Using TextChunker within iter_text.

@cdeler

Copy link
Copy Markdown
ContributorAuthor

@tomchristie
It's better to close this PR?

@lovelydinosaur

Copy link
Copy Markdown
Contributor

@cdeler - I think so, yup - thanks for having taken a crack at it.

@cdeler
cdeler deleted the adding-streaming-by-chunks-to-response branch September 14, 2020 11:30
Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Development

Successfully merging this pull request may close these issues.

2 participants

@cdeler@lovelydinosaur