ext/curl: add CURLOPT_SEEKFUNCTION

[GrahamCampbell]

Today PHP lets you stream a request body from userland through CURLOPT_READFUNCTION, but it never gives libcurl a matching seek callback for that body. That's fine right up until libcurl needs to rewind the upload and send it again, which happens more often than you'd think: on a 307 or 308 redirect, during multi-pass authentication like NTLM or Negotiate, or when a reused keep-alive connection dies after some bytes have already gone out. With no way to rewind, the transfer just fails with CURLE_SEND_FAIL_REWIND (curl error 65). This is the gap behind the very old https://bugs.php.net/bug.php?id=47204, open since 2009, and the more recent https://bugs.php.net/bug.php?id=80518.

libcurl has supported CURLOPT_SEEKFUNCTION since 7.18.0; we just never exposed it for the read-callback body. The only seek callback we register internally is the one on the curl_mime/CURLFile path. So every userland HTTP client has had to work around this itself. Guzzle catches error 65 (and the older errno-0 "silent" variant of the same problem) and rewinds the body in PHP before re-issuing the request, and Symfony's HttpClient forces CURLOPT_FORBID_REUSE for NTLM because, as its own comment puts it, reusing those connections needs seeking capability that only string bodies have. Letting people set a seek callback means libcurl can just rewind and resend on its own.

So this exposes CURLOPT_SEEKFUNCTION. You give it a callable that receives the CurlHandle, the offset and the origin (SEEK_SET, SEEK_CUR or SEEK_END), and returns CURL_SEEKFUNC_OK when it has repositioned the body, CURL_SEEKFUNC_FAIL to fail the transfer, or CURL_SEEKFUNC_CANTSEEK when it can't seek and wants to let libcurl deal with it. In practice it looks like this:

$ch = curl_init('https://example.com/upload');
curl_setopt($ch, CURLOPT_UPLOAD, true);
curl_setopt($ch, CURLOPT_FOLLOWLOCATION, true);
curl_setopt($ch, CURLOPT_READFUNCTION, fn($ch, $fd, $len) => fread($body, $len));
curl_setopt($ch, CURLOPT_SEEKFUNCTION, fn($ch, $offset, $origin) =>
fseek($body, $offset, $origin) === 0 ? CURL_SEEKFUNC_OK : CURL_SEEKFUNC_CANTSEEK);
curl_exec($ch);

The implementation follows the existing callback options as closely as I could. php_curl_handlers gets a seek fcc field next to the other callbacks, a curl_seek trampoline forwards the call into userland and validates the return value the same way curl_prereqfunction does, registration goes through the usual HANDLE_CURL_OPTION_CALLABLE macro (which also points CURLOPT_SEEKDATA at the handle), and the callback is duplicated in curl_copy_handle and released with everything else. When no callback is set, or a callback misbehaves, the trampoline defaults to CURL_SEEKFUNC_CANTSEEK so libcurl never ends up resending from the wrong offset. No version guards are needed, since the option and its return constants have all been around since libcurl 7.18.0, comfortably below our 7.61.0 floor.

Finally, I've implemented a proof of concept cleanup of the relevant Guzzle code that would benefit from this feature at guzzle/guzzle@c2690b5.

[dragoonis]

Hey @adoy, do you have time to take a look at this? If not just say and we can find someone who does. Thanks

[Ayesh]

Looks good to me. There is a merge conflict in the NEWS file, but I think I think a simple git rebase master would solve it.

[GrahamCampbell]

Rebased.

[GrahamCampbell]

@Ayesh@devnexen I've rebased this again. Are we good to merge this now, before there are more conflicts? ❤️

[Ayesh]

Sorry I also probably caused a merge conflict, but thank you for unwrangling it.

I was wondering if @Girgias would get some time to review this, because she worked a lot on FCC macros that SEEKFUNCTION also uses.

I could also test this PR locally; I will merge tomorrow unless there's any further comments.

[GrahamCampbell]

I've re-tested the rebased version of this with some real-world testing beyond the phpt suite, integrating it into Guzzle and comparing behaviour A/B against stock PHP on the same libcurl (8.21.0), using an NTS debug build which stayed quiet throughout.

The two classic failure scenarios both come out as hoped. When a reused keep-alive connection dies mid-upload (bug #47204), libcurl now rewinds a streamed body through the callback and resends it within the same transfer, where stock PHP fails with CURLE_SEND_FAIL_REWIND. An auth challenge arriving after the body has been sent (bug #80518) goes from unrecoverable on stock PHP to completing in a single transfer.

The callback semantics also check out from userland: CURL_SEEKFUNC_OK resends correctly, CURL_SEEKFUNC_CANTSEEK degrades to exactly the pre-PR behaviour, CURL_SEEKFUNC_FAIL fails the transfer cleanly, and clearing the option, curl_reset(), and handle reuse across transfers all behave as expected.

[Ayesh]

Thank you @GrahamCampbell :)

[GrahamCampbell]

Today PHP lets you stream a request body from userland through CURLOPT_READFUNCTION, but it never gives libcurl a matching seek callback for that body. That's fine right up until libcurl needs to rewind the upload and send it again, which happens more often than you'd think: on a 307 or 308 redirect, during multi-pass authentication like NTLM or Negotiate, or when a reused keep-alive connection dies after some bytes have already gone out. With no way to rewind, the transfer just fails with CURLE_SEND_FAIL_REWIND (curl error 65). This is the gap behind the very old https://bugs.php.net/bug.php?id=47204, open since 2009, and the more recent https://bugs.php.net/bug.php?id=80518.

libcurl has supported CURLOPT_SEEKFUNCTION since 7.18.0; we just never exposed it for the read-callback body. The only seek callback we register internally is the one on the curl_mime/CURLFile path. So every userland HTTP client has had to work around this itself. Guzzle catches error 65 (and the older errno-0 "silent" variant of the same problem) and rewinds the body in PHP before re-issuing the request, and Symfony's HttpClient forces CURLOPT_FORBID_REUSE for NTLM because, as its own comment puts it, reusing those connections needs seeking capability that only string bodies have. Letting people set a seek callback means libcurl can just rewind and resend on its own.

So this exposes CURLOPT_SEEKFUNCTION. You give it a callable that receives the CurlHandle, the offset and the origin (SEEK_SET, SEEK_CUR or SEEK_END), and returns CURL_SEEKFUNC_OK when it has repositioned the body, CURL_SEEKFUNC_FAIL to fail the transfer, or CURL_SEEKFUNC_CANTSEEK when it can't seek and wants to let libcurl deal with it. In practice it looks like this:

$ch = curl_init('https://example.com/upload');
curl_setopt($ch, CURLOPT_UPLOAD, true);
curl_setopt($ch, CURLOPT_FOLLOWLOCATION, true);
curl_setopt($ch, CURLOPT_READFUNCTION, fn($ch, $fd, $len) => fread($body, $len));
curl_setopt($ch, CURLOPT_SEEKFUNCTION, fn($ch, $offset, $origin) =>
fseek($body, $offset, $origin) === 0 ? CURL_SEEKFUNC_OK : CURL_SEEKFUNC_CANTSEEK);
curl_exec($ch);

The implementation follows the existing callback options as closely as I could. php_curl_handlers gets a seek fcc field next to the other callbacks, a curl_seek trampoline forwards the call into userland and validates the return value the same way curl_prereqfunction does, registration goes through the usual HANDLE_CURL_OPTION_CALLABLE macro (which also points CURLOPT_SEEKDATA at the handle), and the callback is duplicated in curl_copy_handle and released with everything else. When no callback is set, or a callback misbehaves, the trampoline defaults to CURL_SEEKFUNC_CANTSEEK so libcurl never ends up resending from the wrong offset. No version guards are needed, since the option and its return constants have all been around since libcurl 7.18.0, comfortably below our 7.61.0 floor.

Finally, I've implemented a proof of concept cleanup of the relevant Guzzle code that would benefit from this feature at guzzle/guzzle@c2690b5.

[dragoonis]

Hey @adoy, do you have time to take a look at this? If not just say and we can find someone who does. Thanks

[Ayesh]

Looks good to me. There is a merge conflict in the NEWS file, but I think I think a simple git rebase master would solve it.

[GrahamCampbell]

Rebased.

[GrahamCampbell]

@Ayesh@devnexen I've rebased this again. Are we good to merge this now, before there are more conflicts? ❤️

[Ayesh]

Sorry I also probably caused a merge conflict, but thank you for unwrangling it.

I was wondering if @Girgias would get some time to review this, because she worked a lot on FCC macros that SEEKFUNCTION also uses.

I could also test this PR locally; I will merge tomorrow unless there's any further comments.

[GrahamCampbell]

I've re-tested the rebased version of this with some real-world testing beyond the phpt suite, integrating it into Guzzle and comparing behaviour A/B against stock PHP on the same libcurl (8.21.0), using an NTS debug build which stayed quiet throughout.

The two classic failure scenarios both come out as hoped. When a reused keep-alive connection dies mid-upload (bug #47204), libcurl now rewinds a streamed body through the callback and resends it within the same transfer, where stock PHP fails with CURLE_SEND_FAIL_REWIND. An auth challenge arriving after the body has been sent (bug #80518) goes from unrecoverable on stock PHP to completing in a single transfer.

The callback semantics also check out from userland: CURL_SEEKFUNC_OK resends correctly, CURL_SEEKFUNC_CANTSEEK degrades to exactly the pre-PR behaviour, CURL_SEEKFUNC_FAIL fails the transfer cleanly, and clearing the option, curl_reset(), and handle reuse across transfers all behave as expected.

[Ayesh]

Thank you @GrahamCampbell :)

[GrahamCampbell]

Today PHP lets you stream a request body from userland through CURLOPT_READFUNCTION, but it never gives libcurl a matching seek callback for that body. That's fine right up until libcurl needs to rewind the upload and send it again, which happens more often than you'd think: on a 307 or 308 redirect, during multi-pass authentication like NTLM or Negotiate, or when a reused keep-alive connection dies after some bytes have already gone out. With no way to rewind, the transfer just fails with CURLE_SEND_FAIL_REWIND (curl error 65). This is the gap behind the very old https://bugs.php.net/bug.php?id=47204, open since 2009, and the more recent https://bugs.php.net/bug.php?id=80518.

libcurl has supported CURLOPT_SEEKFUNCTION since 7.18.0; we just never exposed it for the read-callback body. The only seek callback we register internally is the one on the curl_mime/CURLFile path. So every userland HTTP client has had to work around this itself. Guzzle catches error 65 (and the older errno-0 "silent" variant of the same problem) and rewinds the body in PHP before re-issuing the request, and Symfony's HttpClient forces CURLOPT_FORBID_REUSE for NTLM because, as its own comment puts it, reusing those connections needs seeking capability that only string bodies have. Letting people set a seek callback means libcurl can just rewind and resend on its own.

So this exposes CURLOPT_SEEKFUNCTION. You give it a callable that receives the CurlHandle, the offset and the origin (SEEK_SET, SEEK_CUR or SEEK_END), and returns CURL_SEEKFUNC_OK when it has repositioned the body, CURL_SEEKFUNC_FAIL to fail the transfer, or CURL_SEEKFUNC_CANTSEEK when it can't seek and wants to let libcurl deal with it. In practice it looks like this:

$ch = curl_init('https://example.com/upload');
curl_setopt($ch, CURLOPT_UPLOAD, true);
curl_setopt($ch, CURLOPT_FOLLOWLOCATION, true);
curl_setopt($ch, CURLOPT_READFUNCTION, fn($ch, $fd, $len) => fread($body, $len));
curl_setopt($ch, CURLOPT_SEEKFUNCTION, fn($ch, $offset, $origin) =>
fseek($body, $offset, $origin) === 0 ? CURL_SEEKFUNC_OK : CURL_SEEKFUNC_CANTSEEK);
curl_exec($ch);

The implementation follows the existing callback options as closely as I could. php_curl_handlers gets a seek fcc field next to the other callbacks, a curl_seek trampoline forwards the call into userland and validates the return value the same way curl_prereqfunction does, registration goes through the usual HANDLE_CURL_OPTION_CALLABLE macro (which also points CURLOPT_SEEKDATA at the handle), and the callback is duplicated in curl_copy_handle and released with everything else. When no callback is set, or a callback misbehaves, the trampoline defaults to CURL_SEEKFUNC_CANTSEEK so libcurl never ends up resending from the wrong offset. No version guards are needed, since the option and its return constants have all been around since libcurl 7.18.0, comfortably below our 7.61.0 floor.

Finally, I've implemented a proof of concept cleanup of the relevant Guzzle code that would benefit from this feature at guzzle/guzzle@c2690b5.

[dragoonis]

Hey @adoy, do you have time to take a look at this? If not just say and we can find someone who does. Thanks

[Ayesh]

Looks good to me. There is a merge conflict in the NEWS file, but I think I think a simple git rebase master would solve it.

[GrahamCampbell]

Rebased.

[GrahamCampbell]

@Ayesh@devnexen I've rebased this again. Are we good to merge this now, before there are more conflicts? ❤️

[Ayesh]

Sorry I also probably caused a merge conflict, but thank you for unwrangling it.

I was wondering if @Girgias would get some time to review this, because she worked a lot on FCC macros that SEEKFUNCTION also uses.

I could also test this PR locally; I will merge tomorrow unless there's any further comments.

[GrahamCampbell]

I've re-tested the rebased version of this with some real-world testing beyond the phpt suite, integrating it into Guzzle and comparing behaviour A/B against stock PHP on the same libcurl (8.21.0), using an NTS debug build which stayed quiet throughout.

The two classic failure scenarios both come out as hoped. When a reused keep-alive connection dies mid-upload (bug #47204), libcurl now rewinds a streamed body through the callback and resends it within the same transfer, where stock PHP fails with CURLE_SEND_FAIL_REWIND. An auth challenge arriving after the body has been sent (bug #80518) goes from unrecoverable on stock PHP to completing in a single transfer.

The callback semantics also check out from userland: CURL_SEEKFUNC_OK resends correctly, CURL_SEEKFUNC_CANTSEEK degrades to exactly the pre-PR behaviour, CURL_SEEKFUNC_FAIL fails the transfer cleanly, and clearing the option, curl_reset(), and handle reuse across transfers all behave as expected.

[Ayesh]

Thank you @GrahamCampbell :)

[GrahamCampbell]

Today PHP lets you stream a request body from userland through CURLOPT_READFUNCTION, but it never gives libcurl a matching seek callback for that body. That's fine right up until libcurl needs to rewind the upload and send it again, which happens more often than you'd think: on a 307 or 308 redirect, during multi-pass authentication like NTLM or Negotiate, or when a reused keep-alive connection dies after some bytes have already gone out. With no way to rewind, the transfer just fails with CURLE_SEND_FAIL_REWIND (curl error 65). This is the gap behind the very old https://bugs.php.net/bug.php?id=47204, open since 2009, and the more recent https://bugs.php.net/bug.php?id=80518.

libcurl has supported CURLOPT_SEEKFUNCTION since 7.18.0; we just never exposed it for the read-callback body. The only seek callback we register internally is the one on the curl_mime/CURLFile path. So every userland HTTP client has had to work around this itself. Guzzle catches error 65 (and the older errno-0 "silent" variant of the same problem) and rewinds the body in PHP before re-issuing the request, and Symfony's HttpClient forces CURLOPT_FORBID_REUSE for NTLM because, as its own comment puts it, reusing those connections needs seeking capability that only string bodies have. Letting people set a seek callback means libcurl can just rewind and resend on its own.

So this exposes CURLOPT_SEEKFUNCTION. You give it a callable that receives the CurlHandle, the offset and the origin (SEEK_SET, SEEK_CUR or SEEK_END), and returns CURL_SEEKFUNC_OK when it has repositioned the body, CURL_SEEKFUNC_FAIL to fail the transfer, or CURL_SEEKFUNC_CANTSEEK when it can't seek and wants to let libcurl deal with it. In practice it looks like this:

$ch = curl_init('https://example.com/upload');
curl_setopt($ch, CURLOPT_UPLOAD, true);
curl_setopt($ch, CURLOPT_FOLLOWLOCATION, true);
curl_setopt($ch, CURLOPT_READFUNCTION, fn($ch, $fd, $len) => fread($body, $len));
curl_setopt($ch, CURLOPT_SEEKFUNCTION, fn($ch, $offset, $origin) =>
fseek($body, $offset, $origin) === 0 ? CURL_SEEKFUNC_OK : CURL_SEEKFUNC_CANTSEEK);
curl_exec($ch);

The implementation follows the existing callback options as closely as I could. php_curl_handlers gets a seek fcc field next to the other callbacks, a curl_seek trampoline forwards the call into userland and validates the return value the same way curl_prereqfunction does, registration goes through the usual HANDLE_CURL_OPTION_CALLABLE macro (which also points CURLOPT_SEEKDATA at the handle), and the callback is duplicated in curl_copy_handle and released with everything else. When no callback is set, or a callback misbehaves, the trampoline defaults to CURL_SEEKFUNC_CANTSEEK so libcurl never ends up resending from the wrong offset. No version guards are needed, since the option and its return constants have all been around since libcurl 7.18.0, comfortably below our 7.61.0 floor.

Finally, I've implemented a proof of concept cleanup of the relevant Guzzle code that would benefit from this feature at guzzle/guzzle@c2690b5.

[dragoonis]

Hey @adoy, do you have time to take a look at this? If not just say and we can find someone who does. Thanks

[Ayesh]

Looks good to me. There is a merge conflict in the NEWS file, but I think I think a simple git rebase master would solve it.

[GrahamCampbell]

Rebased.

[GrahamCampbell]

@Ayesh@devnexen I've rebased this again. Are we good to merge this now, before there are more conflicts? ❤️

[Ayesh]

Sorry I also probably caused a merge conflict, but thank you for unwrangling it.

I was wondering if @Girgias would get some time to review this, because she worked a lot on FCC macros that SEEKFUNCTION also uses.

I could also test this PR locally; I will merge tomorrow unless there's any further comments.

[GrahamCampbell]

I've re-tested the rebased version of this with some real-world testing beyond the phpt suite, integrating it into Guzzle and comparing behaviour A/B against stock PHP on the same libcurl (8.21.0), using an NTS debug build which stayed quiet throughout.

The two classic failure scenarios both come out as hoped. When a reused keep-alive connection dies mid-upload (bug #47204), libcurl now rewinds a streamed body through the callback and resends it within the same transfer, where stock PHP fails with CURLE_SEND_FAIL_REWIND. An auth challenge arriving after the body has been sent (bug #80518) goes from unrecoverable on stock PHP to completing in a single transfer.

The callback semantics also check out from userland: CURL_SEEKFUNC_OK resends correctly, CURL_SEEKFUNC_CANTSEEK degrades to exactly the pre-PR behaviour, CURL_SEEKFUNC_FAIL fails the transfer cleanly, and clearing the option, curl_reset(), and handle reuse across transfers all behave as expected.

[Ayesh]

Thank you @GrahamCampbell :)

[GrahamCampbell]

Today PHP lets you stream a request body from userland through CURLOPT_READFUNCTION, but it never gives libcurl a matching seek callback for that body. That's fine right up until libcurl needs to rewind the upload and send it again, which happens more often than you'd think: on a 307 or 308 redirect, during multi-pass authentication like NTLM or Negotiate, or when a reused keep-alive connection dies after some bytes have already gone out. With no way to rewind, the transfer just fails with CURLE_SEND_FAIL_REWIND (curl error 65). This is the gap behind the very old https://bugs.php.net/bug.php?id=47204, open since 2009, and the more recent https://bugs.php.net/bug.php?id=80518.

libcurl has supported CURLOPT_SEEKFUNCTION since 7.18.0; we just never exposed it for the read-callback body. The only seek callback we register internally is the one on the curl_mime/CURLFile path. So every userland HTTP client has had to work around this itself. Guzzle catches error 65 (and the older errno-0 "silent" variant of the same problem) and rewinds the body in PHP before re-issuing the request, and Symfony's HttpClient forces CURLOPT_FORBID_REUSE for NTLM because, as its own comment puts it, reusing those connections needs seeking capability that only string bodies have. Letting people set a seek callback means libcurl can just rewind and resend on its own.

So this exposes CURLOPT_SEEKFUNCTION. You give it a callable that receives the CurlHandle, the offset and the origin (SEEK_SET, SEEK_CUR or SEEK_END), and returns CURL_SEEKFUNC_OK when it has repositioned the body, CURL_SEEKFUNC_FAIL to fail the transfer, or CURL_SEEKFUNC_CANTSEEK when it can't seek and wants to let libcurl deal with it. In practice it looks like this:

$ch = curl_init('https://example.com/upload');
curl_setopt($ch, CURLOPT_UPLOAD, true);
curl_setopt($ch, CURLOPT_FOLLOWLOCATION, true);
curl_setopt($ch, CURLOPT_READFUNCTION, fn($ch, $fd, $len) => fread($body, $len));
curl_setopt($ch, CURLOPT_SEEKFUNCTION, fn($ch, $offset, $origin) =>
fseek($body, $offset, $origin) === 0 ? CURL_SEEKFUNC_OK : CURL_SEEKFUNC_CANTSEEK);
curl_exec($ch);

The implementation follows the existing callback options as closely as I could. php_curl_handlers gets a seek fcc field next to the other callbacks, a curl_seek trampoline forwards the call into userland and validates the return value the same way curl_prereqfunction does, registration goes through the usual HANDLE_CURL_OPTION_CALLABLE macro (which also points CURLOPT_SEEKDATA at the handle), and the callback is duplicated in curl_copy_handle and released with everything else. When no callback is set, or a callback misbehaves, the trampoline defaults to CURL_SEEKFUNC_CANTSEEK so libcurl never ends up resending from the wrong offset. No version guards are needed, since the option and its return constants have all been around since libcurl 7.18.0, comfortably below our 7.61.0 floor.

Finally, I've implemented a proof of concept cleanup of the relevant Guzzle code that would benefit from this feature at guzzle/guzzle@c2690b5.

[dragoonis]

Hey @adoy, do you have time to take a look at this? If not just say and we can find someone who does. Thanks

[Ayesh]

Looks good to me. There is a merge conflict in the NEWS file, but I think I think a simple git rebase master would solve it.

[GrahamCampbell]

Rebased.

[GrahamCampbell]

@Ayesh@devnexen I've rebased this again. Are we good to merge this now, before there are more conflicts? ❤️

[Ayesh]

Sorry I also probably caused a merge conflict, but thank you for unwrangling it.

I was wondering if @Girgias would get some time to review this, because she worked a lot on FCC macros that SEEKFUNCTION also uses.

I could also test this PR locally; I will merge tomorrow unless there's any further comments.

[GrahamCampbell]

I've re-tested the rebased version of this with some real-world testing beyond the phpt suite, integrating it into Guzzle and comparing behaviour A/B against stock PHP on the same libcurl (8.21.0), using an NTS debug build which stayed quiet throughout.

The two classic failure scenarios both come out as hoped. When a reused keep-alive connection dies mid-upload (bug #47204), libcurl now rewinds a streamed body through the callback and resends it within the same transfer, where stock PHP fails with CURLE_SEND_FAIL_REWIND. An auth challenge arriving after the body has been sent (bug #80518) goes from unrecoverable on stock PHP to completing in a single transfer.

The callback semantics also check out from userland: CURL_SEEKFUNC_OK resends correctly, CURL_SEEKFUNC_CANTSEEK degrades to exactly the pre-PR behaviour, CURL_SEEKFUNC_FAIL fails the transfer cleanly, and clearing the option, curl_reset(), and handle reuse across transfers all behave as expected.

[Ayesh]

Thank you @GrahamCampbell :)

[GrahamCampbell]

Today PHP lets you stream a request body from userland through CURLOPT_READFUNCTION, but it never gives libcurl a matching seek callback for that body. That's fine right up until libcurl needs to rewind the upload and send it again, which happens more often than you'd think: on a 307 or 308 redirect, during multi-pass authentication like NTLM or Negotiate, or when a reused keep-alive connection dies after some bytes have already gone out. With no way to rewind, the transfer just fails with CURLE_SEND_FAIL_REWIND (curl error 65). This is the gap behind the very old https://bugs.php.net/bug.php?id=47204, open since 2009, and the more recent https://bugs.php.net/bug.php?id=80518.

libcurl has supported CURLOPT_SEEKFUNCTION since 7.18.0; we just never exposed it for the read-callback body. The only seek callback we register internally is the one on the curl_mime/CURLFile path. So every userland HTTP client has had to work around this itself. Guzzle catches error 65 (and the older errno-0 "silent" variant of the same problem) and rewinds the body in PHP before re-issuing the request, and Symfony's HttpClient forces CURLOPT_FORBID_REUSE for NTLM because, as its own comment puts it, reusing those connections needs seeking capability that only string bodies have. Letting people set a seek callback means libcurl can just rewind and resend on its own.

So this exposes CURLOPT_SEEKFUNCTION. You give it a callable that receives the CurlHandle, the offset and the origin (SEEK_SET, SEEK_CUR or SEEK_END), and returns CURL_SEEKFUNC_OK when it has repositioned the body, CURL_SEEKFUNC_FAIL to fail the transfer, or CURL_SEEKFUNC_CANTSEEK when it can't seek and wants to let libcurl deal with it. In practice it looks like this:

$ch = curl_init('https://example.com/upload');
curl_setopt($ch, CURLOPT_UPLOAD, true);
curl_setopt($ch, CURLOPT_FOLLOWLOCATION, true);
curl_setopt($ch, CURLOPT_READFUNCTION, fn($ch, $fd, $len) => fread($body, $len));
curl_setopt($ch, CURLOPT_SEEKFUNCTION, fn($ch, $offset, $origin) =>
fseek($body, $offset, $origin) === 0 ? CURL_SEEKFUNC_OK : CURL_SEEKFUNC_CANTSEEK);
curl_exec($ch);

The implementation follows the existing callback options as closely as I could. php_curl_handlers gets a seek fcc field next to the other callbacks, a curl_seek trampoline forwards the call into userland and validates the return value the same way curl_prereqfunction does, registration goes through the usual HANDLE_CURL_OPTION_CALLABLE macro (which also points CURLOPT_SEEKDATA at the handle), and the callback is duplicated in curl_copy_handle and released with everything else. When no callback is set, or a callback misbehaves, the trampoline defaults to CURL_SEEKFUNC_CANTSEEK so libcurl never ends up resending from the wrong offset. No version guards are needed, since the option and its return constants have all been around since libcurl 7.18.0, comfortably below our 7.61.0 floor.

Finally, I've implemented a proof of concept cleanup of the relevant Guzzle code that would benefit from this feature at guzzle/guzzle@c2690b5.

[dragoonis]

Hey @adoy, do you have time to take a look at this? If not just say and we can find someone who does. Thanks

[Ayesh]

Looks good to me. There is a merge conflict in the NEWS file, but I think I think a simple git rebase master would solve it.

[GrahamCampbell]

Rebased.

[GrahamCampbell]

@Ayesh@devnexen I've rebased this again. Are we good to merge this now, before there are more conflicts? ❤️

[Ayesh]

Sorry I also probably caused a merge conflict, but thank you for unwrangling it.

I was wondering if @Girgias would get some time to review this, because she worked a lot on FCC macros that SEEKFUNCTION also uses.

I could also test this PR locally; I will merge tomorrow unless there's any further comments.

[GrahamCampbell]

I've re-tested the rebased version of this with some real-world testing beyond the phpt suite, integrating it into Guzzle and comparing behaviour A/B against stock PHP on the same libcurl (8.21.0), using an NTS debug build which stayed quiet throughout.

The two classic failure scenarios both come out as hoped. When a reused keep-alive connection dies mid-upload (bug #47204), libcurl now rewinds a streamed body through the callback and resends it within the same transfer, where stock PHP fails with CURLE_SEND_FAIL_REWIND. An auth challenge arriving after the body has been sent (bug #80518) goes from unrecoverable on stock PHP to completing in a single transfer.

The callback semantics also check out from userland: CURL_SEEKFUNC_OK resends correctly, CURL_SEEKFUNC_CANTSEEK degrades to exactly the pre-PR behaviour, CURL_SEEKFUNC_FAIL fails the transfer cleanly, and clearing the option, curl_reset(), and handle reuse across transfers all behave as expected.

[Ayesh]

Thank you @GrahamCampbell :)

[GrahamCampbell]

Today PHP lets you stream a request body from userland through CURLOPT_READFUNCTION, but it never gives libcurl a matching seek callback for that body. That's fine right up until libcurl needs to rewind the upload and send it again, which happens more often than you'd think: on a 307 or 308 redirect, during multi-pass authentication like NTLM or Negotiate, or when a reused keep-alive connection dies after some bytes have already gone out. With no way to rewind, the transfer just fails with CURLE_SEND_FAIL_REWIND (curl error 65). This is the gap behind the very old https://bugs.php.net/bug.php?id=47204, open since 2009, and the more recent https://bugs.php.net/bug.php?id=80518.

libcurl has supported CURLOPT_SEEKFUNCTION since 7.18.0; we just never exposed it for the read-callback body. The only seek callback we register internally is the one on the curl_mime/CURLFile path. So every userland HTTP client has had to work around this itself. Guzzle catches error 65 (and the older errno-0 "silent" variant of the same problem) and rewinds the body in PHP before re-issuing the request, and Symfony's HttpClient forces CURLOPT_FORBID_REUSE for NTLM because, as its own comment puts it, reusing those connections needs seeking capability that only string bodies have. Letting people set a seek callback means libcurl can just rewind and resend on its own.

So this exposes CURLOPT_SEEKFUNCTION. You give it a callable that receives the CurlHandle, the offset and the origin (SEEK_SET, SEEK_CUR or SEEK_END), and returns CURL_SEEKFUNC_OK when it has repositioned the body, CURL_SEEKFUNC_FAIL to fail the transfer, or CURL_SEEKFUNC_CANTSEEK when it can't seek and wants to let libcurl deal with it. In practice it looks like this:

$ch = curl_init('https://example.com/upload');
curl_setopt($ch, CURLOPT_UPLOAD, true);
curl_setopt($ch, CURLOPT_FOLLOWLOCATION, true);
curl_setopt($ch, CURLOPT_READFUNCTION, fn($ch, $fd, $len) => fread($body, $len));
curl_setopt($ch, CURLOPT_SEEKFUNCTION, fn($ch, $offset, $origin) =>
fseek($body, $offset, $origin) === 0 ? CURL_SEEKFUNC_OK : CURL_SEEKFUNC_CANTSEEK);
curl_exec($ch);

The implementation follows the existing callback options as closely as I could. php_curl_handlers gets a seek fcc field next to the other callbacks, a curl_seek trampoline forwards the call into userland and validates the return value the same way curl_prereqfunction does, registration goes through the usual HANDLE_CURL_OPTION_CALLABLE macro (which also points CURLOPT_SEEKDATA at the handle), and the callback is duplicated in curl_copy_handle and released with everything else. When no callback is set, or a callback misbehaves, the trampoline defaults to CURL_SEEKFUNC_CANTSEEK so libcurl never ends up resending from the wrong offset. No version guards are needed, since the option and its return constants have all been around since libcurl 7.18.0, comfortably below our 7.61.0 floor.

Finally, I've implemented a proof of concept cleanup of the relevant Guzzle code that would benefit from this feature at guzzle/guzzle@c2690b5.

[dragoonis]

Hey @adoy, do you have time to take a look at this? If not just say and we can find someone who does. Thanks

[Ayesh]

Looks good to me. There is a merge conflict in the NEWS file, but I think I think a simple git rebase master would solve it.

[GrahamCampbell]

Rebased.

[GrahamCampbell]

@Ayesh@devnexen I've rebased this again. Are we good to merge this now, before there are more conflicts? ❤️

[Ayesh]

Sorry I also probably caused a merge conflict, but thank you for unwrangling it.

I was wondering if @Girgias would get some time to review this, because she worked a lot on FCC macros that SEEKFUNCTION also uses.

I could also test this PR locally; I will merge tomorrow unless there's any further comments.

[GrahamCampbell]

I've re-tested the rebased version of this with some real-world testing beyond the phpt suite, integrating it into Guzzle and comparing behaviour A/B against stock PHP on the same libcurl (8.21.0), using an NTS debug build which stayed quiet throughout.

The two classic failure scenarios both come out as hoped. When a reused keep-alive connection dies mid-upload (bug #47204), libcurl now rewinds a streamed body through the callback and resends it within the same transfer, where stock PHP fails with CURLE_SEND_FAIL_REWIND. An auth challenge arriving after the body has been sent (bug #80518) goes from unrecoverable on stock PHP to completing in a single transfer.

The callback semantics also check out from userland: CURL_SEEKFUNC_OK resends correctly, CURL_SEEKFUNC_CANTSEEK degrades to exactly the pre-PR behaviour, CURL_SEEKFUNC_FAIL fails the transfer cleanly, and clearing the option, curl_reset(), and handle reuse across transfers all behave as expected.

[Ayesh]

Thank you @GrahamCampbell :)

[GrahamCampbell]

Today PHP lets you stream a request body from userland through CURLOPT_READFUNCTION, but it never gives libcurl a matching seek callback for that body. That's fine right up until libcurl needs to rewind the upload and send it again, which happens more often than you'd think: on a 307 or 308 redirect, during multi-pass authentication like NTLM or Negotiate, or when a reused keep-alive connection dies after some bytes have already gone out. With no way to rewind, the transfer just fails with CURLE_SEND_FAIL_REWIND (curl error 65). This is the gap behind the very old https://bugs.php.net/bug.php?id=47204, open since 2009, and the more recent https://bugs.php.net/bug.php?id=80518.

libcurl has supported CURLOPT_SEEKFUNCTION since 7.18.0; we just never exposed it for the read-callback body. The only seek callback we register internally is the one on the curl_mime/CURLFile path. So every userland HTTP client has had to work around this itself. Guzzle catches error 65 (and the older errno-0 "silent" variant of the same problem) and rewinds the body in PHP before re-issuing the request, and Symfony's HttpClient forces CURLOPT_FORBID_REUSE for NTLM because, as its own comment puts it, reusing those connections needs seeking capability that only string bodies have. Letting people set a seek callback means libcurl can just rewind and resend on its own.

So this exposes CURLOPT_SEEKFUNCTION. You give it a callable that receives the CurlHandle, the offset and the origin (SEEK_SET, SEEK_CUR or SEEK_END), and returns CURL_SEEKFUNC_OK when it has repositioned the body, CURL_SEEKFUNC_FAIL to fail the transfer, or CURL_SEEKFUNC_CANTSEEK when it can't seek and wants to let libcurl deal with it. In practice it looks like this:

$ch = curl_init('https://example.com/upload');
curl_setopt($ch, CURLOPT_UPLOAD, true);
curl_setopt($ch, CURLOPT_FOLLOWLOCATION, true);
curl_setopt($ch, CURLOPT_READFUNCTION, fn($ch, $fd, $len) => fread($body, $len));
curl_setopt($ch, CURLOPT_SEEKFUNCTION, fn($ch, $offset, $origin) =>
fseek($body, $offset, $origin) === 0 ? CURL_SEEKFUNC_OK : CURL_SEEKFUNC_CANTSEEK);
curl_exec($ch);

The implementation follows the existing callback options as closely as I could. php_curl_handlers gets a seek fcc field next to the other callbacks, a curl_seek trampoline forwards the call into userland and validates the return value the same way curl_prereqfunction does, registration goes through the usual HANDLE_CURL_OPTION_CALLABLE macro (which also points CURLOPT_SEEKDATA at the handle), and the callback is duplicated in curl_copy_handle and released with everything else. When no callback is set, or a callback misbehaves, the trampoline defaults to CURL_SEEKFUNC_CANTSEEK so libcurl never ends up resending from the wrong offset. No version guards are needed, since the option and its return constants have all been around since libcurl 7.18.0, comfortably below our 7.61.0 floor.

Finally, I've implemented a proof of concept cleanup of the relevant Guzzle code that would benefit from this feature at guzzle/guzzle@c2690b5.

[dragoonis]

Hey @adoy, do you have time to take a look at this? If not just say and we can find someone who does. Thanks

[Ayesh]

Looks good to me. There is a merge conflict in the NEWS file, but I think I think a simple git rebase master would solve it.

[GrahamCampbell]

Rebased.

[GrahamCampbell]

@Ayesh@devnexen I've rebased this again. Are we good to merge this now, before there are more conflicts? ❤️

[Ayesh]

Sorry I also probably caused a merge conflict, but thank you for unwrangling it.

I was wondering if @Girgias would get some time to review this, because she worked a lot on FCC macros that SEEKFUNCTION also uses.

I could also test this PR locally; I will merge tomorrow unless there's any further comments.

[GrahamCampbell]

I've re-tested the rebased version of this with some real-world testing beyond the phpt suite, integrating it into Guzzle and comparing behaviour A/B against stock PHP on the same libcurl (8.21.0), using an NTS debug build which stayed quiet throughout.

The two classic failure scenarios both come out as hoped. When a reused keep-alive connection dies mid-upload (bug #47204), libcurl now rewinds a streamed body through the callback and resends it within the same transfer, where stock PHP fails with CURLE_SEND_FAIL_REWIND. An auth challenge arriving after the body has been sent (bug #80518) goes from unrecoverable on stock PHP to completing in a single transfer.

The callback semantics also check out from userland: CURL_SEEKFUNC_OK resends correctly, CURL_SEEKFUNC_CANTSEEK degrades to exactly the pre-PR behaviour, CURL_SEEKFUNC_FAIL fails the transfer cleanly, and clearing the option, curl_reset(), and handle reuse across transfers all behave as expected.

[Ayesh]

Thank you @GrahamCampbell :)