Transfermarkt API

A transfermarkt scraper that fails on this site usually fails quietly. The cheap request returns HTTP 200, so nothing in your logs looks wrong, and you find out weeks later that half your rows are empty. This repo starts with that failure, shows exactly what it looks like, then covers the data-header selector family that carries every fact on a player page.

Built on ScrapingBee's web scraping API. Verified live on 2026-09-15 against the Erling Haaland profile.

The quiet failure

Same URL, three proxy tiers, all returning HTTP 200:

Configuration	Credits	Bytes	spb-initial-status-code	Usable
mode=auto	1	2,414	202	no
premium_proxy plus country_code=de	25	181,286	200	yes
stealth_proxy	75	252,092	200	yes, and unnecessary

The 1 credit response is 2,414 bytes of interstitial. It carries no title, no player, no market value. The API reports 200 because the fetch itself succeeded, and the real upstream status is in the spb-initial-status-code header, which reads 202 Accepted. A 202 on a page you asked to read is not success, it is a holding response.

So the check is not the HTTP status. It is either the header or a content assertion:

if len(html) < 20000 or "data-header__market-value-wrapper" not in html:
    raise RuntimeError("interstitial, escalate the proxy tier")

Premium is the rung that works. Stealth also works and costs three times more for the same data, so there is no reason to reach for it here.

country_code=de is worth keeping. Transfermarkt is a German site and the German exit returned the page cleanly on every attempt.

The data-header family

Everything on a player profile hangs off one block of classes. These are semantic names, not build hashes, so they survive deploys:

rules = {
    "h1": "h1",
    "market_value": {"selector": "a.data-header__market-value-wrapper", "output": "text"},
    "club":         {"selector": "span.data-header__club a",             "output": "text"},
    "facts":        {"selector": "li.data-header__label", "type": "list"},
}

Live response for transfermarkt.com/erling-haaland/profil/spieler/418560:

{
  "h1": "#9 Erling Haaland",
  "market_value": "€ 220.00 m Last update: 22/07/2026",
  "club": "Man City",
  "facts": [
    "Date of birth/Age: 21/07/2000 (26)",
    "Place of birth: Leeds",
    "Citizenship: Norway",
    "Height: 1,95 m",
    "Position: Centre-Forward",
    "Agent: Rafaela Pimenta",
    "Current international: Norway",
    "Caps/Goals: 55 / 62"
  ]
}

Three things to handle in your parser

The H1 carries the shirt number. It is #9 Erling Haaland, not Erling Haaland. Strip a leading #<digits> before you key anything on the name, or your join against another data source will miss.

Market value is two facts in one string. € 220.00 m Last update: 22/07/2026 holds the valuation and the date it was set. Split on Last update:, then parse the amount. The m suffix is millions and k is thousands, and the currency symbol is euro regardless of the player's league.

Pick the right selector for the facts. Three selectors return overlapping views of the same block, and only one is self describing:

Selector	Returns	Useful
li.data-header__label	9 items, "Label: value"	yes, parse on the first colon
span.data-header__content	10 bare values, no labels	only if you trust positional order
span.data-header__label	3 items	partial, skips most fields

Use li.data-header__label and split each item on the first ": ". The bare content list is tempting because it is clean, but it is position dependent, and a player with no agent shifts every field after it.

Scaling a squad

Player URLs follow transfermarkt.com/<slug>/profil/spieler/<id>, and the numeric id is the stable part. The slug is cosmetic and a wrong slug still resolves, so you can build URLs from ids alone once you have them.

At 25 credits per player, a 25 man squad is 625 credits and a 20 club league is roughly 12,500. The entry paid tier of 250,000 credits covers about 20 full league sweeps a month. Market values move slowly, usually twice a season, so a weekly sweep is wasted spend. Read Last update out of the market value string and skip any player whose valuation has not moved.

ScrapingBee does not cache, so every repeat fetch is billed in full. Store what you pull.

Credit cost

Measured from spb-cost response headers:

Configuration	Credits
mode=auto, returns an interstitial	1
premium_proxy plus country_code	25
stealth_proxy	75
Rejected request	0

The 1 credit call is billed even though it gives you nothing usable, so detect the interstitial and do not retry it at the same tier. mode=auto cannot be combined with premium_proxy or stealth_proxy, and sending both returns HTTP 400 while billing nothing.

Plan tiers are on the pricing page.

Scope

Public player, club and competition pages. Transfermarkt user accounts, forum posts behind a login and anything requiring a signed in session are out of reach, and scraping under login credentials is prohibited by ScrapingBee's terms of service.

Player profiles are personal data about identifiable individuals, including date of birth, place of birth and named agents, so handle them accordingly rather than as plain rows. Market valuations are Transfermarkt's own community driven estimates, not verified transfer fees or salaries, and should be described that way in anything you publish. Transfermarkt's Terms of Use govern use of the data.

Reference: extraction rules, data extraction feature, screenshots for proving what a page showed at fetch time.

Adjacent endpoints: football news API, news results API, review API.

FAQ

Why is my Transfermarkt scrape empty with a 200 status? Because you got the 2,414 byte interstitial. Read spb-initial-status-code, which reads 202, or assert on content length. Escalate to premium_proxy.

Do I need stealth proxies? No. Premium returned the full page every time at 25 credits. Stealth costs 75 for the same result.

Why does my player name have a hash in it? The H1 includes the shirt number, as in #9 Erling Haaland. Strip the prefix before joining on name.

Is there an official Transfermarkt API? Transfermarkt does not publish a public API for this data. Several community projects wrap the site, and this project reads the public pages directly.

How do I get transfer history rather than the profile? The transfer table lives on a separate path under the same player id. The data-header selectors above cover the profile block only.

License

MIT. See LICENSE.