Fetch Data from URL

fetch is a focused helper package for downloading data from URL endpoints. It wraps common net/http idioms so callers can handle download flows with less boilerplate and safer defaults.

Design goals

• Keep application code simple for one-shot data fetching.
• Hide repetitive net/http handling patterns.
• Keep API surface small and composable.
• Use context.Context based operations as the default style.

Development

Requirements

• Go 1.25 or later
• Task command (local tool for this repository)

Local validation

task test
task govulncheck

Run all maintenance tasks:

task

CI Workflows

• ci: lint (golangci-lint with gosec), tests, and govulncheck
• CodeQL: scheduled and push/PR static analysis

Usage

Install and import

go get github.com/goark/fetch@latest

import"github.com/goark/fetch"

Sample programs

All sample files under sample/ use the run build tag.

go run -tags run ./sample/sample.go

• Basic GET download: sample/sample.go

GET request

package main
import (
"context""fmt""io""os""github.com/goark/fetch"
)
funcmain() {
u, err:=fetch.URL("https://github.com/spiegel-im-spiegel.gpg")
iferr!=nil {
fmt.Fprintln(os.Stderr, err)
return
}
resp, err:=fetch.New().GetWithContext(context.Background(), u)
iferr!=nil {
fmt.Fprintln(os.Stderr, err)
return
}
deferresp.Close()
if_, err:=io.Copy(os.Stdout, resp.Body()); err!=nil {
fmt.Fprintln(os.Stderr, err)
}
}

POST request

payload:=strings.NewReader("name=alice")
resp, err:=fetch.New().PostWithContext(ctx, u, payload,
fetch.WithRequestHeaderSet("Content-Type", "application/x-www-form-urlencoded"),
)
iferr!=nil {
returnerr
}
deferresp.Close()

Public API

• fetch.URL(raw string) parses and validates URL input.
• fetch.New(opts ...fetch.ClientOpts) creates a fetch client.
• Client.GetWithContext(ctx, u, opts...) performs GET request.
• Client.PostWithContext(ctx, u, payload, opts...) performs POST request.
• fetch.WithHTTPClient(cli) injects custom *http.Client.
• fetch.WithRequestHeaderAdd(name, value) and fetch.WithRequestHeaderSet(name, value) apply request headers.

Error handling

The package wraps errors, so use errors.Is for checks against:

• fetch.ErrInvalidURL
• fetch.ErrInvalidRequest
• fetch.ErrHTTPStatus

iferr!=nil {
switch {
caseerrors.Is(err, fetch.ErrInvalidURL):
// invalid URL inputcaseerrors.Is(err, fetch.ErrHTTPStatus):
// non-success HTTP status (>= 400)
}
}

Response lifecycle

Always close the response:

• defer resp.Close() for streaming use.
• Use resp.DumpBodyAndClose() when you need body bytes at once.

Modules Requirement Graph