• Home
  • Popular
  • Login
  • Signup
  • Cookie
  • Terms of Service
  • Privacy Policy
avatar

Posted by John Dev


10 Jan, 2025

Updated at 18 Jan, 2025

Reqwest slower than Go in GKE Autopilot cluster

I'm having trouble troubleshooting slow performance with reqwest when running in a Google Kubernetes Engine Autopilot cluster. The exact same code seems to run fine everywhere (even on a Google Compute Engine instance) but the second it is in a GKE Autopilot cluster it is about half as fast as Go.

I'm only testing using http1.1 - I've disabled h2 in my tests.

Here is what I've been testing with: GitHub - bennetthardwick/http-download-testing.

I've put them up as a container - you can test them out like this:

docker run -it bennetthardwick/http-download-testing:ubuntu-24.04

export TEST_URL='some url'

make bench_go # run go | pv > /dev/null
make bench_curl # run curl | pv > /dev/null 
make bench_reqwest # run reqwest | pv > /dev/null 

This is the abridged Go code that I've written:

client := http.Client {
	Transport: &http.Transport {
		TLSNextProto: make(map[string]func(authority string, c *tls.Conn) http.RoundTripper),
	}
}

res, err := http.Get(".. url ..")

io.Copy(os.Stdout, res.Body)

This is the abridged Rust/reqwest version that I've written:

let mut stdout = std::io::stdout().lock();
let client = request::ClientBuilder::new().http1_only().build().unwrap();

let mut bytes = client.get(".. url ..").send().await.unwrap().bytes_stream();

while let Some(next) = bytes.next().await {
	stdout.write_all(&next.unwrap()).unwrap();
}

Running in us-central1 everything is between 200-300MiB/s. But reqwest drops down to half when running in GKE.

Here are the results I see:

GCP instance, g1-small, GCS object same region, https

Lang Speed
Go 230MiB/s
Reqwest 210MiB/s
Reqwest (Rustls) 220MiB/s
Isahc 200MiB/s
Ureq 250MiB/s
Curl 260MiB/s

GCP GKE Autopilot Pod, n1 machine class, GCS object same region, https

Lang Speed
Go 240MiB/s
Reqwest 104MiB/s
Reqwest (Rustls) 108MiB/s
Isahc 140MiB/s
Ureq 230MiB/s
Curl 220MiB/s

One thing to note is Ureq doesn't seem to suffer from the slowdown - which starts to feel like it's some kind of tokio related issue. I've tried changing a bunch of settings and nothing helps. If anyone has any ideas I'd really appreciate it!

1 post - 1 participant

Read full topic