Engineering for Slow Internet

“Extreme congestion, queueing, and dropped packets” ⤴️
“getting a few bytes of text in and out should still be possible!” ⤴️
“Connectivity at the South Pole was never going to meet the performance expectations set by engineers using a robust terrestrial Internet connection. It’s not a good idea to hardcode a single, static, global expectation for how long 20 MB of Javascript should take to download.” ⤴️
“Limited availability, frequent dropouts, and occasional service preemptions.” ⤴️
“an interesting glimpse into the on-the-ground reality of using the Internet in bandwidth-constrained environments.” ⤴️
“access to the Internet only through an extremely limited series of satellite links provided by the United States Antarctic Program.” ⤴️
“less bandwidth shared by everyone combined than what everyone individually can get on a typical 4g cellular network in an American suburb.” ⤴️
“limitations described above,plus there was only connectivity for a few hours a day, when the satellites rose above the horizon and the station was authorized to use them.” ⤴️
“the unavoidable physics of this connectivity. These satellites are in a high orbit, thousands of miles up. This means high latency.” ⤴️
“about thirty times the expected latency of a healthy connection from your home, on a terrestrial cable or fiber connection, to most major content delivery networks (up to 25 ms).” ⤴️
“At the South Pole, the latency was over two hundred and fifty times greater.” ⤴️
“But – a lot of the end-user impact is caused by web and app engineering which fails to take slow/intermittent links into consideration.” ⤴️
“can you tell me, off the top of your head, how your app behaves on a link with 40 kbps available bandwidth, 1,000 ms latency, occasional jitter of up to 2,000 ms, packet loss of 10%, and a complete 15-second connectivity dropout every few minutes?” ⤴️
“Operational needs are aggressively prioritized, and the community soaks up whatever is left.” ⤴️
“the app had been updated since last time I loaded it, so all of my browser’s cached assets were stale and had to be re-downloaded.” ⤴️
“Browsers do a decent job of handling slow Internet. Under the hood, the underlying protocols do a decent job at congestion control. I should get a steady trickle of data. This will be subject to the negotiated send and receive windows between client and server, which are based on the current level of congestion on the link, and which are further influenced by any shaping done by middleware along the way.” ⤴️
“It’s a complex webapp, so the app developer would also need to implement some of their own retry logic. This allows for recovery in the event that individual assets fail, especially for those long, multi-second total connectivity dropouts. But eventually, given enough time, the transfers should complete.” ⤴️
“The developers implemented a global failure trigger somewhere in the app. If the app didn’t fully load within the parameters specified by the developer (time? number of retries? I’m not sure.), then the appstopped, gave up, redirected you to an error page, dropped all the loading progress you’d made, and implemented aggressive cache-busting countermeasures for next time you retried.” ⤴️
“this was primarily a messaging app. The actual content payload here, when the app is running and I’m chatting with my friends, is measured in bytes.” ⤴️
“if I kept reloading the page, and if I kept letting it re-download the same 20 MB of Javascript, and if I kept putting up with the developer’s cache-busting shenanigans, eventually it finished before the artificial failure criteria.” ⤴️
“Does this webapp really need to be 20 MB? What all is being loaded that could be deferred until it is needed, or included in an “optional” add-on bundle? Is there a possibility of a “lite” version, for bandwidth-constrained users?” ⤴️
“There are folks behind misconfigured routers, folks with flaky wifi, folks stuck with fly-by-night WISPs delivering sub-par service. Folks who still use dial-up Internet connections over degraded copper phone lines.” ⤴️
”* If you have the ability to measure whether bytes are flowing, and they are, leave them alone, no matter how slow. Perhaps show some UI indicating what is happening.” ⤴️
“If I can do it, unpaid, working alone, for my silly little hobby blog, in 200 lines of Python… surely your team of engineers can do so for your flagship webapp.” ⤴️
“most of these in-app downloaders are woefully ill-equipped for the task! Many of them lack pause/resume functionality, state notifications, retry logic, and progress tracking. Many of them have frustrating restrictions, such as time limits for downloading the payload. While most of these issues are mere annoyances in the land of fast Internet, at the South Pole, they can make or break the app entirely.” ⤴️
“Let the user break out of your in-app downloader and use their own, if at all possible.” ⤴️
“Apple naively assumed that the 1 GB download would be so fast that they didn’t bother incorporating download speed feedback into the updater’s time estimate. They did not anticipate people installing macOS updates from a location where a gigabyte of downloads can take several hours, if not days.” ⤴️
“Extreme congestion, queueing, and dropped packets” ⤴️
“getting a few bytes of text in and out should still be possible!” ⤴️
“Connectivity at the South Pole was never going to meet the performance expectations set by engineers using a robust terrestrial Internet connection. It’s not a good idea to hardcode a single, static, global expectation for how long 20 MB of Javascript should take to download.” ⤴️
“Limited availability, frequent dropouts, and occasional service preemptions.” ⤴️
“an interesting glimpse into the on-the-ground reality of using the Internet in bandwidth-constrained environments.” ⤴️
“access to the Internet only through an extremely limited series of satellite links provided by the United States Antarctic Program.” ⤴️
“less bandwidth shared by everyone combined than what everyone individually can get on a typical 4g cellular network in an American suburb.” ⤴️
“limitations described above,plus there was only connectivity for a few hours a day, when the satellites rose above the horizon and the station was authorized to use them.” ⤴️
“the unavoidable physics of this connectivity. These satellites are in a high orbit, thousands of miles up. This means high latency.” ⤴️
“about thirty times the expected latency of a healthy connection from your home, on a terrestrial cable or fiber connection, to most major content delivery networks (up to 25 ms).” ⤴️
“At the South Pole, the latency was over two hundred and fifty times greater.” ⤴️
“But – a lot of the end-user impact is caused by web and app engineering which fails to take slow/intermittent links into consideration.” ⤴️
“can you tell me, off the top of your head, how your app behaves on a link with 40 kbps available bandwidth, 1,000 ms latency, occasional jitter of up to 2,000 ms, packet loss of 10%, and a complete 15-second connectivity dropout every few minutes?” ⤴️
“Operational needs are aggressively prioritized, and the community soaks up whatever is left.” ⤴️
“the app had been updated since last time I loaded it, so all of my browser’s cached assets were stale and had to be re-downloaded.” ⤴️
“Browsers do a decent job of handling slow Internet. Under the hood, the underlying protocols do a decent job at congestion control. I should get a steady trickle of data. This will be subject to the negotiated send and receive windows between client and server, which are based on the current level of congestion on the link, and which are further influenced by any shaping done by middleware along the way.” ⤴️
“It’s a complex webapp, so the app developer would also need to implement some of their own retry logic. This allows for recovery in the event that individual assets fail, especially for those long, multi-second total connectivity dropouts. But eventually, given enough time, the transfers should complete.” ⤴️
“The developers implemented a global failure trigger somewhere in the app. If the app didn’t fully load within the parameters specified by the developer (time? number of retries? I’m not sure.), then the appstopped, gave up, redirected you to an error page, dropped all the loading progress you’d made, and implemented aggressive cache-busting countermeasures for next time you retried.” ⤴️
“this was primarily a messaging app. The actual content payload here, when the app is running and I’m chatting with my friends, is measured in bytes.” ⤴️
“if I kept reloading the page, and if I kept letting it re-download the same 20 MB of Javascript, and if I kept putting up with the developer’s cache-busting shenanigans, eventually it finished before the artificial failure criteria.” ⤴️
“Does this webapp really need to be 20 MB? What all is being loaded that could be deferred until it is needed, or included in an “optional” add-on bundle? Is there a possibility of a “lite” version, for bandwidth-constrained users?” ⤴️
“There are folks behind misconfigured routers, folks with flaky wifi, folks stuck with fly-by-night WISPs delivering sub-par service. Folks who still use dial-up Internet connections over degraded copper phone lines.” ⤴️
”* If you have the ability to measure whether bytes are flowing, and they are, leave them alone, no matter how slow. Perhaps show some UI indicating what is happening.” ⤴️
“If I can do it, unpaid, working alone, for my silly little hobby blog, in 200 lines of Python… surely your team of engineers can do so for your flagship webapp.” ⤴️
“most of these in-app downloaders are woefully ill-equipped for the task! Many of them lack pause/resume functionality, state notifications, retry logic, and progress tracking. Many of them have frustrating restrictions, such as time limits for downloading the payload. While most of these issues are mere annoyances in the land of fast Internet, at the South Pole, they can make or break the app entirely.” ⤴️
“Let the user break out of your in-app downloader and use their own, if at all possible.” ⤴️
“Apple naively assumed that the 1 GB download would be so fast that they didn’t bother incorporating download speed feedback into the updater’s time estimate. They did not anticipate people installing macOS updates from a location where a gigabyte of downloads can take several hours, if not days.” ⤴️