I spent a good chunk of my work life this year trying (in collaboration with the amazing Noam Rosenthal) to standardize a new web platform feature: a way to modify the intrinsic size and resolution of images. And hey! We did it! But boy, was it ever a learning experience.
This wasn’t my first standardization rodeo, so many of the issues we ran into, I more-or-less anticipated. Strong negative feedback from browsers. Weird, unforeseen gotchas with the underlying primitives. A complete re-think or two. What I didn’t anticipate though, was that our proposal — which, again, was “only” about modifying the default display size of images — would run afoul of the fundamental privacy and security principles of the web. Because before this year, I didn’t really understand those principles.
Let me set the table a bit. What were we trying to do?
By default, images on the web show up exactly as big as they are. Embedding an 800×600 image? Unless you stretch or shrink that image with CSS or markup, that’s exactly how large it’s going to be: 800 CSS pixels across, and 600 CSS pixels tall. That’s the image’s intrinsic (aka “natural”) size. Another way to put this is that, by default, all images on the web have an intrinsic density of 1×.
That’s all well and good, until you’re trying to serve up high-, low-, or ✨variable✨-density images, without access to CSS or HTML. This is a situation that image hosts like my employer, Cloudinary, find themselves in quite often.
So, we set out to give ourselves and the rest of the web a tool to modify the intrinsic size and resolution of images. After a couple of re-thinks, the solution that we landed on was this:
- Browsers should read and apply metadata contained within image resources themselves, allowing them to declare their own intended display size and resolution.
- Following in the recent footsteps of
image-orientation— by default, browsers would respect and apply this metadata. But you could override it or turn it off with a little CSS (image-resolution), or markup (srcset’sxdescriptors).
We felt pretty good about this. It was flexible, it built on an existing pattern, and it seemed to address all of the issues that had been raised against our previous proposals. Alas, one of the editors of the HTML spec, Anne van Kesteren, said: no. This wasn’t going to work. And image-orientation needed an urgent re-think, too. Because this pattern, where you can turn the effects of EXIF metadata on and off with CSS and HTML, would violate the “Same-Origin Policy.”
Uh… what?
Aren’t we just scaling and rotating images??
Confession time! Before all of this, I’d more or less equated the Same-Origin Policy with CORS errors, and all of the frustration that they’ve caused me over the years. Now, though, the Same-Origin Policy wasn’t just standing between me and handling a fetch, it was holding up a major work initiative. And I had to explain the situation to bosses who knew even less about security and privacy on the web than I did. Time to learn!
Here’s what I learned:
- The Same-Origin Policy isn’t a single, simple, rule. And it certainly isn’t == CORS errors.
- What it is, is a philosophy which has evolved over time, and has been inconsistently implemented across the web platform.
- In general, what it says is: the fundamental security and privacy boundary of the web is origins. Do you share an origin with something else on the web? You can interact with it however you like. If not, though, you might have to jump through some hoops.
- Why “might?” Well, a lot of cross-origin interactions are allowed, by default! Generally, when you’re making a website, you can write across origins (by sending
POSTrequests off to whoever you please, via forms). And you can even embed cross-origin resources (iframes, images, fonts, etc) that your site’s visitors will see, right there on your website. But what you can’t do, is look at those cross-origin resources, yourself. You shouldn’t be able to read anything about a cross-origin resource, in your JavaScript, without specially-granted permission (via our old friend, CORS). - Here’s the thing that blew my mind the most, once I finally understood it: cross-origin reads are forbidden by default because, as end-users, we all see different world-wide webs, and a website shouldn’t be able to see the rest of the web through its visitors’ eyes. Individuals’ varied local browsing contexts – including, but not limited to, cookies — mean that when I go to, say, gmail.com, I’m going to see something different than you, when you enter that same URL into your address bar and hit “return.” If other websites could fire off requests to Gmail from my browser, with my cookies, and read the results, well – that would be very, very bad!
So by default: you can do lots of things with cross-origin resources. But preventing cross-origin reads is kind of the whole ballgame. Those defaults are more-or-less what people are talking about when they talk about the “Same-Origin Policy.”
How does this all relate to the intrinsic size and resolution of images?
Let’s say there’s an image URL – https://coolbank.com/hero.jpg, that happens to return a different resource depending on whether or not a user is currently logged in at coolbank.com. And let’s say that the version that shows up when you’re logged in, has some EXIF resolution info, but the version that shows up when you’re not, doesn’t. Lastly, let’s pretend that I’m an evil phisher-man, trying to figure out which bank you belong to, so I can spoof its homepage and trick you into typing your bank login info into my evil form.
So! I embed https://coolbank.com/hero.jpg on an evil page. I check its intrinsic size. I turn EXIF-sizing off, with image-resolution: none, and then check its size again. Now, even though CORS restrictions are preventing me from looking at any of the image’s pixel data, I know whether or not it contains any EXIF resolution information — I’ve been able to read a little tiny piece of that image, across origins. And now, I know whether or not you’re logged into, and have an account at, coolbank.com.
Far-fetched? Perhaps! But the web is an unimaginably large place. And, as Jen Simmons once put it,
Browsing the web is basically going around running other people’s untrusted and potentially malicious code, willy-nilly, all day long. The principles that underly web security and privacy — including the Same-Origin Policy — enable this safety, and must be defended absolutely. The hole we were unintentionally trying to open in the Same-Origin Policy seemed so small, at first. A few literal bits of seemingly-harmless information. But a cross-origin read, however small, is a cross-origin read, and cross-origin reads are not allowed.
How did we fix our spec? We made EXIF resolution and orientation information un-readable across origins by making it un-turn-off-able: in cross-origin contexts, EXIF modifications are always applied. An 800×600 image whose EXIF says it should be treated as 400×300 will behave exactly like a 400×300 image, would, no matter what. A simple-enough solution — once we understood the problem.
As a bonus, once I really understood the Same-Origin Policy and the whys behind the web’s default security policies, a bunch of other web security pieces started to fall into place for me.
Cross-site request forgery attacks take advantage of the fact that cross-origin writes are allowed, by default. If an API endpoint isn’t careful about how it responds to POST requests, bad things can happen. Likewise, Content Security Policy allows granular control over what sorts of embeds are allowed, because again, by default, they all are, and it turns out, that opens the door to cross-site scripting attacks. And the new alphabet soup of web security features — COOP, COEP, CORP, and CORB — are all about shutting down cross-origin interactions completely, fixing some of the inconsistent ways that the Same-Origin Policy has been implemented over the years and closing down any/all possible cross-origin interaction, to achieve a rarefied state known as “cross-origin isolation”. In a world where Spectre and friends mean that cross-origin loading can be exploited to perform cross-origin reading, full cross-origin isolation is needed to guarantee saftey when doing various, new, powerful things.
In short:
- Security and privacy on the web are actually pretty amazing, when you think about it.
- They’re a product of the platform’s default policies, which are all about restricting interactions across origins.
- By default, the one thing no one should ever be able to do is read data across origins (without special permission).
- The reason reads are forbidden is that we all see different webs, and attackers shouldn’t be able to see the web through potential victims’ eyes.
- No ifs, ands, or buts! Any hole in the Same-Origin Policy, however small, is surface area for abuse.
- In 2020, I tried to open a tiny hole in the Same-Origin Policy (oops), and then got to learn all of the above.
Here’s to a safer and more secure 2021, in every possible sense.
