February 23, 2012
The prevalence of free, open WiFi has made it rather easy for a WiFi eavesdropper to steal your identity cookie for the websites you visit while you're connected to that WiFi access point. This is something I talked about in Breaking the Web's Cookie Jar. It's difficult to fix without making major changes to the web's infrastructure.
In the year since I wrote that, a number of major websites have "solved" the WiFi eavesdropping problem by either making encrypted HTTPS web traffic an account option or mandatory for all logged in users.
For example, I just noticed that Twitter, transparently to me and presumably all other Twitter users, switched to an encrypted web connection by default. You can tell because most modern browsers show the address bar in green when the connection is encrypted.
I initially resisted this as overkill, except for obvious targets like email (the skeleton key to all your online logins) and banking.
Yes, you can naively argue that every website should encrypt all their traffic all the time, but to me that's a "boil the sea" solution. I'd rather see a better, more secure identity protocol than ye olde HTTP cookies. I don't actually care if anyone sees the rest of my public activity on Stack Overflow; it's hardly a secret. But gee, I sure do care if they somehow sniff out my cookie and start running around doing stuff as me! Encrypting everything just to protect that one lousy cookie header seems like a whole lot of overkill to me.
Of course, there's no reason to encrypt traffic for anonymous, not-logged-in users, and Twitter doesn't. You get a plain old HTTP connection until you log in, at which point they automatically switch to HTTPS encryption. Makes sense.
It was totally painless for me, as a user, and it makes stealing my Twitter identity, or eavesdropping on my Twitter activity (as fascinating as I know that must sound), dramatically more difficult. I can't really construct a credible argument against doing this, even for something as relatively trivial as my Twitter account, and it has some definite benefits. So perhaps Twitter has the right idea here; maybe encrypted connections should be the default for all web sites. As tinfoil hat as this seemed to me a year ago, now I'm wondering if that might actually be the right thing to do for the long-term health of the overall web, too.
Why not boil the sea, then? Let us encrypt all the things!
HTTPS isn't (that) expensive any more
Yes, in the hoary old days of the 1999 web, HTTPS was quite computationally expensive. But thanks to 13 years of Moore's Law, that's no longer the case. It's still more work to set up, yes, but consider the real world case of GMail:
In January this year (2010), Gmail switched to using HTTPS for everything by default. Previously it had been introduced as an option, but now all of our users use HTTPS to secure their email between their browsers and Google, all the time. In order to do this we had to deploy no additional machines and no special hardware. On our production frontend machines, SSL/TLS accounts for less than 1% of the CPU load, less than 10KB of memory per connection and less than 2% of network overhead. Many people believe that SSL takes a lot of CPU time and we hope the above numbers (public for the first time) will help to dispel that.
HTTPS means The Man can't spy on your Internet
Since all the traffic between you and the websites you log in to would now be encrypted, the ability of nefarious evildoers to either …
- steal your identity cookie
- peek at what you're doing
- see what you've typed
- interfere with the content you send and receive
… is, if not completely eliminated, drastically limited. Regardless of whether you're on open public WiFi or not.
Personally, I don't care too much if people see what I'm doing online since the whole point of a lot of what I do is to … let people see what I'm doing online. But I certainly don't subscribe to the dangerous idea that "only criminals have things to hide"; everyone deserves the right to personal privacy. And there are lots of repressive governments out there who wouldn't hesitate at the chance to spy on what their citizens do online, or worse. Much, much worse. Why not improve the Internet for all of them at once?
HTTPS goes faster now
Security always comes at a cost, and encrypting a web connection is no different. HTTPS is going to be inevitably slower than a regular HTTP connection. But how much slower? It used to be that encrypted content wouldn't be cached in some browsers, but that's no longer true. And Google's SPDY protocol, intended as a drop-in replacement for HTTP, even goes so far as to bake encryption in by default, and not just for better performance:
[It is a specific technical goal of SPDY to] make SSL the underlying transport protocol, for better security and compatibility with existing network infrastructure. Although SSL does introduce a latency penalty, we believe that the long-term future of the web depends on a secure network connection. In addition, the use of SSL is necessary to ensure that communication across existing proxies is not broken.
There's also SSL False Start which requires a modern browser, but reduces the painful latency inherent in the expensive, but necessary, handshaking required to get encryption going. SSL encryption of HTTP will never be free, exactly, but it's certainly a lot faster than it used to be, and getting faster every year.
Bolting on encryption for logged-in users is by no means an easy thing to accomplish, particularly on large, established websites. You won't see me out there berating every public website for not offering encrypted connections yesterday because I know how much work it takes, and how much additional complexity it can add to an already busy team. Even though HTTPS is way easier now than it was even a few years ago, there are still plenty of tough gotchas: proxy caching, for example, becomes vastly harder when the proxies can no longer "see" what the encrypted traffic they are proxying is doing. Most sites these days are a broad mashup of content from different sources, and technically all of them need to be on HTTPS for a properly encrypted connection. Relatively underpowered and weakly connected mobile devices will pay a much steeper penalty, too.
Maybe not tomorrow, maybe not next year, but over the medium to long term, adopting encrypted web connections as a standard for logged-in users is the healthiest direction for the future of the web. We need to work toward making HTTPS easier, faster, and most of all, the default for logged in users.
[advertisement] What's your next career move? Stack Overflow Careers has the best job listings from great companies, whether you're looking for opportunities at a startup or Fortune 500. You can search our job listings or create a profile and let employers find you.
Posted by Jeff Atwood
In my opinion I think http should be encrypted by default. Security on the Internet is not something to take lightly and should be as wide spread as possible.
I have bookmarked https://tools.ietf.org/html/draft-mbelshe-httpbis-spdy-00 and hope to read through it tomorrow.
Lately I have been thinking a lot about how the infrastructure of the entire Internet is fairly unsustainable, especially the web. SPDY sounds let a step in the right direct for part of the problem, but that is just by reading the abstract and what you have written here.
Another complication with encrypting everything is images - if users can post images (like they can in questions and answers on StackExchange sites) then you run into browsers informing people that some elements on the page haven't been transmitted securely.
Fixing this is possible, but it means that you can just turn on HTTPS and be done. Here's a post from GitHub: Sidejack Prevention Phase 3: SSL Proxied Assets
Just a quick post to link people to this nice little add-on called "HTTPS Everywhere": https://www.eff.org/https-everywhere
Stable for Firefox, alpha version for chrome and chromium.
From their site:
"Our code is partially based on the STS implementation from the groundbreaking NoScript project (there are other STS implementations out there, too). HTTPS Everywhere aims to have a simpler user experience than NoScript, and to support complex rewriting rules that allow services like Google Search and Wikipedia to be redirected to HTTPS without breaking anything."
Being able to safely access Twitter without having your identity revealed is crucial in places like Syria where the government has been known to "disappear" those who speak out against it online. Most of us don't worry if our online identity is known to those around us, but for people in many parts of the world, keeping their online identity secret is a life and death matter.
The problem with enabling SSL by default is that some (disfunctional) "corporate IT" departments block HTTPS traffic through their proxy servers as a control mechanism. We have SSL forced on our clients who are mostly government and semi-government agencies and we often have to ask them to whitelist our sites.
I even asked one client "... so that means I cannot do internet banking while at work" and the response was "management has only authorised internet banking to the accounts team and they are the only ones allowed SSL". Makes you wonder how they then establish an online presence via Facebook/Twitter.
I agree, and encryption should be a toggle on the web server, no certificate, red tape or other unrelated stuff required.
This assumes that the network operator is not a bad actor.
A problem with HTTPS is that it can give you a false sense of security. In an enterprise IT environment, you usually cannot have any confidence that your HTTPS session is terminated at the website that you are visiting.
In a coffeeshop, this is harder, as the snoop needs to have a trusted SSL certificate. But still possible.
"HTTPS means The Man can't spy on your Internet" ... yeah, not really. http://www.schneier.com/blog/archives/2010/04/man-in-the-midd_2.html
Remember that VeriSign sells interception tools to law enforcement: http://www.verisign.com/static/001927.pdf
A quote from: http://forum.icann.org/lists/net-rfp-verisign/msg00008.html
Verisign also operates a 'Lawful Intercept' service called
NetDiscovery . This service is provided to "... [assist]
government agencies with lawful interception and subpoena requests
for subscriber records ."
We believe that under such a service, VeriSign could be required
to issue false certificates, ones _unauthorised_ by the nominal
owner. Such certificates could be employed in an attack on the
user's traffic via the DNS services now under question. Further,
the design of the SSL browser system includes a 'root list' of
trusted issuers, and a breach of _any_ of these means that the
protection afforded by SSL can now be bypassed.
We do not intend to pass comment on the legal issues surrounding
such intercepts. Rather, we wish to draw your attention to the fact
that VeriSign now operates under a conflict of interest. VeriSign
serves both the users of certificates as customers, and also the (legal)
interceptors of same. The certificate owner loses in this battle
due to straightforward economics, and is thus no longer represented.
I got a hardware loadbalancer for my https://clubcompy.com cluster and it offered hardware SSL encryption. So I thought, what the heck, I'll encrypt all the things! It's a site for kids after all, so I had better at least try to keep 'em safe from the bad guys.
Https hasn't slowed the site down one bit, although it took quite a bit of hacking to get every single page to be encrypted. Ended up using a servlet filter to force all requests to redirect to https it rather than configuring anything at the web tier or on the LB.
@Carson: the CA system is far from perfect, but encryption without any kind of identity verification is almost completely useless, because anyone can still MITM you - how would your browser know if they're connecting to the real website or a fake?
@Porges: A simple way of eliminating that threat is to connect to the same website both directly and through a proxy (VPN, Tor, etc) and then compare the certificates you get each time. Unless "The Man" is MITMing that proxy too (which is improbable), the fingerprints won't match.
Perspectives and Convergence are more or less automated versions of the same principle: you ask different computers - called Notaries - to tell you if the certificate they get is the same you got.
@Duffbeer703, how would IT commit a man in the middle account without the user knowing about it? Install a compromised browser or computer spyware?
There's a single point in SSL that makes me shiver. And those are CAs.
I just don't trust the current CAs out there and I think that the model is broken. There should be an public and open CA to issue certificates in a secure and manageable way. Besides that the current model inflate certificate prices.
I sort of like Amazon's approach to this problem. They have a two-stage authentication setup which is a nice compromise between security, performance, and convenience.
Essentially, if you're a registered user and you return to Amazon.com to do some shopping, then you're half-way logged in by default. You can browse your wishlist, or look at Amazon's recommendations for you. You can even look at reviews you may have given.
But if you decide to make changes, view past orders, or do anything that reveals sensitive information, then Amazon forces you to log in for real.
However, after you log in for real, Amazon doesn't switch you over to HTTPS for the entire site. This might be a mistake, and is likely something they'll change in the future.
Why cant webapps just present a challenge on every page? Basically imagine if every request was a post and on the page there was a challenge, and you had to have the correct response in order for your IP to not get banned (or whatever).
Just a thought.
a compelling argument, but what about caching?
bandwidth had gotten cheaper and cheaper, true, but there are still many places where it makes sense. 2nd world universities and large offices, for example, that may have a slower pipe for free browsing.
I was meaning caching proxies, as opposed to local caches.
This is all well and good for desktop browsers, most of the folk reading this blog know how to spot the difference between an encrypted and unencrypted connection in their favorite browser. My real concern is mobile apps, where there really is no way for anyone to tell if Path/twitter/etc is sending our authentication data/address book/etc in cleartext or not. My guess is most startups don't prioritize getting an ssl certificate/connection on the back-end. The good news is most people don't bother connecting their iPhones or Droid devices to public wifi networks when they have a regular phone/data plan in place.
For those who want to see if your mobile apps are broadcasting your personal information, try sniffing you own traffic, you may be suprised how much or how little some mobile applications may broadcast. As for some startups not paying for the initial costs of an ssl certificate, I think it has more to do with not knowing how, or having an api instruction guide explaining how to do it. I have multiple developer licenses for numerous platforms, and I've only come across one api for ssl protocols. Try that on for size. Try blaming those not making the process easier for newer developers.
Step 2 is digitally signing/encrypting all e-mail.
Last time I checked for my own shared hosting, SSL was limited to one domain and one IP, i.e., as long as I use shared hosting, I must decide which one of my domains is used for SSL. Is that still correct?
@Ejc3: Usually companies add their own base certificate to their desktops' browsers and/or OS trusted certificates. Then proxies like Squid can use that base cert to dynamically generated certificates for any domain that are accepted by the browser, therefore MITMing the connection.
The timing on this is pretty pristine. I was just talking about this with a coworker last week, and he cited some of the same reasons for not going to everything over HTTPS that you discussed in your article.
That said, the site I'm developing on my own time is going to utilize external accounts so my visitors can log into their Google, Facebook, OpenId, etc., to post comments and the like.
Well written article, sir. Thank you for discussing this.
I think IPv6 wil reduce some security issues too.
The basic premise of this argument is that SSL is indeed secure. However there are tons of very successful attacks against SSL that range from a university chaining together hundreds of PS3s and generating a legitimate certificate for SSL spoofing, Verisign releasing a couple thousand page document that showed that they were regularly successfully hacked over the past few years (once the CA is compromised, your security no longer exists). The ability to compromise or attack an SSL secured site is not the difficult problem that many may think thanks to easy to use GUI applications such as Cain/Able with built in network sniffing, password crackers, SSL spoofing and tools that can hack into a secured wireless network within minutes or hours (depending on if its WEP or WPA). Also, the idea that The Man can't watch what your doing is also not completely accurate. There are many reports of government agencies going into CAs and demanding legitimate certificates. The NSA loves this system because they can easily spy on whomever they wish simply by forcing CAs to give them a legitimate certificate. Whats more, it used to be there was only Verisign. Now there are hundreds of different CAs that are built into your browser as trusted CAs and they aren't all US based companies, they are located all over the world. There is a tremendous lack of oversight of all of these companies. All that to say, we need to rethink our security strategy. There are many other methods of security data including things like Password Authenticated Key Exchanges which does not require a third party (but it does require a more in depth knowledge of security). This is not something that CAs or the government wants. CAs make money on being a third party and the government likes this type of security because they can continue to be big brother looking over your shoulder.
[[You get a plain old HTTP connection until you log in, at which point they automatically switch to HTTPS encryption. Makes sense.]]
Actually, no, it's completely insecure if the login form is delivered over HTTP. A network based attacker can steal your credentials by changing the form.
[[I agree, and encryption should be a toggle on the web server, no certificate, red tape or other unrelated stuff required.]]
Encryption without authenticity is neither secure or useful.
[[SSL was limited to one domain and one IP]]
That's fixed using SNI, which is available on most platforms, although interestingly not IE6 and some mobile platforms.
Just a small nitpick: The "green bar" you speak of is actually a UI bonus from the use of extended validation certificates -- the certs that are immensely expensive.
The certs /normal/ people buy don't provide that same UI bonus. For example, https://labs.chevronwp7.com is secure but shows no color in IE9 and barely hinted in Chrome.
@Ejc3 At my employer, we issue our own root certificates to authenticate computers. So when we use the proxy to decrypt incoming SSL connections, we re-encrypt the session between the proxy and browser with using an internally trusted certificate.
You should assume that any public computer at a hotel, coffee shop, library, etc is doing this.
To the end user, this is transparent, unless you inspect the SSL data.
For a nefarious network operator (ie. not an IT organization in a company), this is a little harder to do. You either need to compromise a PC and inject a false Root Certificate, or obtain fraudulent certificates from an intermediate CA trusted by your browser already.
Yes, all net traffic should be encrypted.
We can argue about the method, but in a world where you should not only suspect, but expect that your traffic is being intercepted, tracked, compiled, and stored for future reference - either by your ISP, your employer, authorities (yours or others), and who knows else - that there's no compelling reason not to make reading your data as difficult as possible.
This doesn't work well for embedded devices. While I agree that anywhere there needs to be authentication (authentication is NOT encryption), https should be used, but plain HTTP can be done via telnet or netcat type programs. If you have a speedy enough ARM (say, chumby) https isn't as much of a problem. If you are trying to do it with an arduino it won't be possible. This is becoming less of a problem as typically you can go bluetooth or wifi to smartphone and the smartphone is adequate as a platform to use SSL.
I should also note ENCRYPTION USES ENERGY. You complain about battery life now - but you expand by 100x the energy to send the page. Google and Bing have web spiders - what is the carbon footprint if everything is now HTTPS? 1% might be nothing to your laptop, but is it to the search engine spiders?
There is a larger problem of SSL. The CA model is BROKEN! You are encrypting traffic but you might be sending it to a bogus site conducting a MITM. If I disable all CAs in firefox, the web breaks. Mainly when going to abc.xyz, there is a separate cert for static.abc.xyz, serverfarmx.abc.xyz, etc. Try figuring all the individual certs you need to allow for something like github or even so that firefox can update, sync, and load extensions properly.
HTTPS has one significant issue that will spoil the fun. No "host" directive so you need different IPs for different domains just like in good old HTTP 1.0
However it's a good solution for all major players.
I can't rememember when was the last time I accessed a unprotected wireless network without connecting to my VPN account (I use www.happy-vpn.com). VPN, not proxy, where they keep activity logs and many times are used by hackers to steal identity. A VPN is the mother of all protection: don't rely on THEM to protect you ( be it twitter, facebook, so, etc), ACT.
Why is it worth a post that you are finally accepting the value of something that others advocated years ago?
[[[You get a plain old HTTP connection until you log in, at which point they automatically switch to HTTPS encryption. Makes sense.]]]
[Actually, no, it's completely insecure if the login form is delivered over HTTP. A network based attacker can steal your credentials by changing the form.]
@Eric Lawrence, you're absolutely right. But it's actually even worse than that. If your users ever access (or try to access) your site through plain HTTP you have a problem: a man-in-the-middle can intercept this request and prevent the switch to HTTPS. The solution proposed in the OP offers increased security if and only if end-users will notice not being switched over to HTTPS and conclude that they are subject to a MITM attack. Highly unlikely I would say.
Perhaps I'm biased (I work on http://www.anyfinetworks.com ), but I think link level security has its place.
This whole post and not a single word about referral urls? Or the myriad issues associated with SSL certificates?
I don't think so, HTTPS has already filled the security loops.
Security is hard. There's just a whole lot you need to think about. Using HTTPS everywhere means you have one less decision you need to make. It may not be a panacea, but it means you're less likely to accidentally not use it when you should.
Yes, all network traffic should be encrypted by default. Everyone should have a right to privacy as default and then chose to share what they want to share.
I am astonished that anyone has a problem with that.
Should your phone conversations be a matter of public record unless you specify otherwise?
I thought we were transitioning towards IPv6 which by default encrypts all packets sent. This transition in transport protocol will affect all application using TCP/IP, including HTTPS. It is a slow process, but I believe we'll get there someday?
We already have a working alternative to logging in with cookies: HTTP digest authentication. The problem is that (1) the browser vendors do not provide a method to log out, and (2) they present a modal dialog box when authenticating rather than allowing the use of in-page forms.
What is needed is a campaign similar to the campaign for web standards to get the browser vendors to add the ability to forget HTTP authentication credentials and allow in-page forms that can be styled for submitting HTTP authentication.
Oh gosh, you're so wrong!
1. There is absolutely no point in encrypting whole traffic. If content is by definition available to all, what is the point of encryption?
2. It seems like you don't know that, but HTTPS is quite easy to break.
3. It's even easier to buy certificate and use it for evil.
I find it difficult to swallow that some sites sill charge for SSL.
I'm evaluating task managers:
RTM - SSL works with free account (use https:// url)
Nozbe - Requires $89.95 to get a plan with SSL
Toodledo (which for a long time was my favorite) - $14.95 for plan with SSL
Wunderlist - SSL works with free account (use https:// url)
Orchestera - SSL is default !!!!!!!!
Even though I really like Nozbe, the steep cost for SSL (my primary need for upgraded account), really sours my opinion. Toodledo is reasonable, but I've become concerned with providers who feel security is an upcharge.
There are many other services that I haven't mentioned.