programming and human factors
by Jeff Atwood
In Why Isn't My Encryption.. Encrypting? we learned that your encryption is only as good as your understanding of the encryption code. And that the best encryption of all is no encryption, because you kept everything on the server, away from the prying eyes of the client.
In The Bathroom Wall of Code we learned the potential danger of copy-pasting code from the internet, and the continued importance of regular peer review for every line of code that enters your codebase, from whatever source.
I didn't anticipate this series becoming a trilogy, but apparently it has, because Thomas Ptacek of Matsano Security wrote a long blog entry about it. A blog entry masquerading as an overly dramatic college screenplay, but still. These guys, unlike us, are real security experts, so it's worth reading.
But you don't have to read that screenplay, because I'm going to reveal the twist in the final act right here.
Mr. Ptacek is absolutely right. The root problem was that we were working at the wrong layer of abstraction.
Rather than construct code from the low-level cryptography primitives provided in .NET, we should have used a library to handle our encryption needs. I'm reminded of a common Stack Overflow joke:
Q: How do I write this in JavaScript?A: You don't. You use JQuery.
You can save a tremendous amount of time and effort by using the browser-independent framework that JQuery has spent untold man-hours testing, debugging, and proving in the field. While there's nothing wrong with writing JavaScript, why not speed your development time by writing to the library instead? As I've always said, don't reinvent the wheel, unless you plan on learning more about wheels.
Abstractions are important. You could view most of computer programming history as slowly, painfully clawing our way up the evolutionary tree of abstraction -- from assembly language, to C, to Java, to JavaScript, all the way up to JQuery, where the air starts to get pretty darn thin. We've already layered an operating system, web browser, and interpreted scripting language on top of each other to get to this point. It's a testament to the power of abstraction that any of it works at all.
Getting back to specifics: how can you stop programmers from working at the wrong layer of abstraction? One solution would be to disallow the .NET encryption primitives entirely. This is akin to Steve Gibson's holy crusade against raw socket programming in Windows XP. That's one way to do it, I suppose. But putting roadblocks in front of programmers is tantamount to a challenge; why not offer them more attractive alternatives, instead?
Hiding the low-level encryption primitives feels like a temporary solution. That said, I'd strongly recommend marking some of the older encryption methods as deprecated, so programmers who do stumble down some dusty old code path at least have some warning sign that they're using an algorithm with a lot of known vulnerabilities. I'm envisioning a Clippy that pops up with something like:
"Hey! It looks like you're using a method of encryption that's widely regarded as insecure by security experts! Would you like to see alternatives?"
One of those alternatives would be a full-blown library, perhaps something like Bouncy Castle, or Keyczar, or cryptlib. What could be easier than a EncryptStringForBrowser() method which has security and tamper-resistance built in, that's part of a proven, domain-expert-tested set of code that thousands if not millions of developers already rely on?
Using encryption libraries doesn't mean that crucial encryption mistakes will magically disappear overnight. But these libraries, because they force developers to work at a higher level of abstraction, do make it harder to misuse cryptography. And perhaps more importantly, usability improvements to the library can be better handled by the specialists who created the library, rather than the generalists working on the .NET framework itself.
So the next time you set out to write code -- not just encryption code, any code -- ask yourself: am I working at the right level of abstraction?
IMHO it would've been nice to see you comment on the pitfalls of using 3rd party libraries, and how you need to *abstract* those out with this article.
I dunno, I think you could've picked a better word than abstraction to describe implementation specifics in the first place.
Eagan on June 12, 2009 2:41 AMI think abstraction here is a really poor choice of words for what you're describing.
I also think you could've covered some of the pitfalls of using 3rd party libraries in your code, and how you need to *abstract* those out of the rest of your code for maintainability.
I dunno... maybe Coding Horror has jumped the shark. Who are you, and what have you done with Jeff Atwood?
Eagan on June 12, 2009 2:45 AMI think abstraction here is a really poor choice of words for what you're describing.
I also think you could've covered some of the pitfalls of using 3rd party libraries in your code, and how you need to *abstract* those out of the rest of your code for maintainability.
I dunno... maybe Coding Horror has jumped the shark. Who are you, and what have you done with Jeff Atwood?
Eagan on June 12, 2009 2:47 AMI think abstraction here is a really poor choice of words for what you're describing.
I also think you could've covered some of the pitfalls of using 3rd party libraries in your code, and how you need to *abstract* those out of the rest of your code for maintainability.
I dunno... maybe Coding Horror has jumped the shark. Who are you, and what have you done with Jeff Atwood?
Eagius on June 12, 2009 2:47 AMI think abstraction here is a really poor choice of words for what you're describing.
I also think you could've covered some of the pitfalls of using 3rd party libraries in your code, and how you need to *abstract* those out of the rest of your code for maintainability.
I dunno... maybe Coding Horror has jumped the shark. Who are you, and what have you done with Jeff Atwood?
Eagius on June 12, 2009 2:48 AMIt is interesting to see the lengths people will go to in order to justify working with stone knives and bearskins instead of power tools.
I suspect an overriding factor is the lack of experience with anything but the naked programming language they've chosen. These are probably also the same sorts of people who claim migration among programming languages and platforms is trivial. They simply do not have a professional level of experience invested in a given ecosystem.
We used to call these people trainees and junior programmers. But on the Internet nobody knows you're a dog.
DevMan on June 12, 2009 2:48 AMI think abstraction here is a really poor choice of words for what you're describing.
I also think you could've covered some of the pitfalls of using 3rd party libraries in your code, and how you need to *abstract* those out of the rest of your code for maintainability.
I dunno... maybe Coding Horror has jumped the shark. Who are you, and what have you done with Jeff Atwood?
Oh, and I liked "Orange" better.
Eagius on June 12, 2009 2:49 AMOMG they never showed up. Sorry!!!
Eagius on June 12, 2009 2:49 AMThe big problems with this philosophy are:
1) Inter-library dependencies. This is a major problem with a lot of OSS frameworks. These libraries don't worry about these dependencies and leave the worry up to you. So you end up with an encrytpion library that requires JRE 1.4 but some other library that requires JRE 1.6+. And then the headaches begin.
2) Libraries often try to be all things to all people. So you end up with tons of useless code bloat when all you want to do is encrypt a simple string. How many dozens of megabytes of code did you just add to your solution when you only needed one method out of the whole thing? Besides code bloat you are also dealing with learning bloat. You have to dig through all of the countless options to get to what you really want.
3) How do you know that the library you are using is really doing it better than you can do yourself? Unless you take the time to look at the specific implementation details of that EncryptStringforBrowser method, how do you know that it isn't just using the same defaults that you used? You don't. So taking the time to understand it is still required.
4) I don't care what anyone wants to tell you, security through obscurity is not ridiculous. As the sole method of security it is. But as part of a layered approach it just gives one more hurdle that the attackers must crack. And if you use a well known library then they don't have to work very hard to crack your code if there are known exploits. Many will likely beat me up on this but they are probably not understanding my point.
Overall I find libraries to be useful for most things. But their use does have its downsides.
Matt on June 12, 2009 3:12 AM@qwert
"So you do all of your coding in assembly?"
What?! And trust an assembler?!!!
http://cm.bell-labs.com/who/ken/trust.html
yuiop on June 12, 2009 3:49 AMLibraries might be useful in more cases if it didn't take more time to make them work for you than it does to solve the problem yourself.
anonymous on June 12, 2009 4:33 AM>> I believe that the reason that a lot of programmers opt for the "roll your own" method instead of using an existing library is that it makes them feel more like a true programmer, other than just another "code borrower" that they probably went to school with and made fun of going through their CS degree program.
I just felt the need to respond to this.
It's a balancing act.
If you're using a library that's still using a method that's now considered insecure, whose fault is it if your site is insecure? You may not even be fully aware it's happening (that's the point of abstraction isn't it?). If you were down in the details you'd be aware of exactly what was going on, so if you *do* find it's insecure you can both identify that you have a problem, and enable yourself to fix it (not always so with the abstraction).
There's a flip side to that abstraction and sometimes it isn't worth it. In the case of security it can be pretty clear cut many times, but many times you inherit design flaws, maintenance nightmares, etc, for an abstraction that only casually fits your problem. Sometimes it's a win to roll your own, despite the existence of solutions.
Michael Reiland on June 12, 2009 4:51 AMsweet
fishstick_kitty on June 12, 2009 6:31 AMI believe that the reason that a lot of programmers opt for the "roll your own" method instead of using an existing library is that it makes them feel more like a true programmer, other than just another "code borrower" that they probably went to school with and made fun of going through their CS degree program.
None the less, security over ego always wins the fight.
Taylor on June 12, 2009 6:31 AMFirst?
Paul on June 12, 2009 6:32 AMThat way a buffer overflow won't imperil just your site, it will imperil tens of thousands of severs.
(And what it the world does Mitchell hanker after?)
and of course on June 12, 2009 6:35 AMThis is very true.
Now talking about Cryptography, even if you are using a very popular and secure library, there is a chance that you screw things up of you don't know cryptographic primitives.
The biggest factor is that the person writing the code, whether it be from scratch or using some library, should know atleast the basics of the field. Otherwise, you will see people encrypt data with the best encryption algorithm in the most secure chaining mode and then attach the key in plaintext with the data.
I completely agree that we ahould use standrd libraries for these kind of stuff, but I am just adding that to write secure code, get a developer who knows the naunces of security. That is in a way another layer of abstraction, isn't it?
Niyaz PK on June 12, 2009 6:38 AMAnd then there are a bunch of people who don't even use the primitives of the language. They write their own code!: http://www.diovo.com/2009/02/wrote-your-own-encryption-algorithm-duh/
Well, yes. Library code is good, good library code that's being maintained by someone else is even better. We get the strong, debugged, field-tested code just sitting there.
What I wish more people I've worked with would remember is that, if needs be, we can write our own libraries. Finding variations on the same non-circular wheels repeated throughout projects is a waste of developer time, a functionality problem for our users and intensely frustrating for the maintainers. When you then find that a function is written on the assumption that the wheel is basically circular but has this kink under some circumstances...
When we can, use a decent pre-existing library. When we can't, write our own library and use that - most problems aren't unique.
Greg Webb on June 12, 2009 6:53 AM@Taylor
"roll your own" also tends to come about when rolling your own is easier to think about than reading 100 pages of API documentation or guessing when no API documentation is available.
All programmers can think in programming primitives, it's harder to think in someone else's primitives.
Why are there so many web frameworks? Because it's easier to think about parsing a string, and generating another string in response, then it is to learn the required structure of most of these frameworks. So people roll their own.
Jesse McNelis on June 12, 2009 6:54 AMWhenver considering using a library that performs something so non-trivial, remember the Pragmatic warning: "Beware Evil Wizards". Know your library in a truly Heinleinian way* before proceeding, or you'll find yourself looking down the business end.
*"grok" to those who know the word, but not the book.
It's interesting that you say "wrong layer of abstraction" instead of "not enough abstraction".
Not once did you provide an example, or even mention that there are some cases where you want to work with less abstraction (see: performance).
Jeff on June 12, 2009 7:10 AMAnalogy FAIL.
> Getting back to specifics: how can you stop programmers from working
> at the wrong layer of abstraction? One solution would be to disallow
> the .NET encryption primitives entirely. This is akin to Steve
> Gibson's holy crusade against raw socket programming in Windows XP.
Actually, Steve Gibson's objection to the raw socket API was that exposing it would allow malware to be more dangerous. This is totally different to exposing an API which allows naive programmers to make mistakes.
FMark on June 12, 2009 7:14 AMBravo, excellent post.
You're correct: using a library is very DRY, and would have prevented the problems you experienced previously in the trilogy.
Perhaps the most useful thing of all, then, would be some form of central database of code libraries for a variety of languages.
Jeff on June 12, 2009 7:17 AMI found your post on the same page as this link:
http://www.daemonology.net/blog/2009-06-11-cryptographic-right-answers.html
...on Hacker News (http://news.ycombinator.com) - coincidence? :)
Regardless of 3rd party implementation, understand the underlying technology/theory first.
Juan on June 12, 2009 7:23 AMWhilst sometimes the fun is taken out of somebody having already written a good abstraction library, the reality is, is that it's likely your issue has already been encountered and solved many times before.
When building some projects I often feel like I'm playing with Lego. Referencing this library here and that one there. All with their handy API methods like 'EncryptStringForBrowser()'.
I'm just gluing the libraries together to create the right stack that my business model can use to solve the business problem.
Ed Blackburn on June 12, 2009 7:32 AMI'm never working at any level of abstraction other than abstraction I have written myself. I cannot rely on third party code in projects as valuable as the ones I'm using - unless I review all of it. And that will me take much more time than rewriting this from scratch.
Mecki on June 12, 2009 7:36 AM@Taylor
I tend to agree with Jesse, although I think coders 'roll their own' because they don't know enough about 'wheels': they'll just try to solve their problems as they understand it (in their limited way)
Jamie on June 12, 2009 7:40 AMjQuery is probably overkill for most tasks. Hundreds of functions that have to be reparsed by the browser on every page load - in the case of Javascript, writing to jQuery incurs a performance hit. Using a JS library is not analog to using a C library.
Steve on June 12, 2009 7:40 AMThe only problem arises when one million websites build their security using EncryptStringForBrowser() and then someone finds an exploit on that function call.
Since no code is completely safe, it is in fact beneficial to have many languages, and many ways in those languages, to achieve the same goals. This prevents any one layer of abstraction from bringing down the whole series of tubes.
Ian on June 12, 2009 7:57 AMWe have faced a few problem of this nature at work. Working at the wrong level of abstraction is usually a problem that can be attributed to novice users of a programming language. It takes a while to fully understand a language and much more time to find out, try and test libraries which are already available in that language. I know of novice programmers of Python writing code in Python as if they were coding in C++.
Tathagata Chakraborty on June 12, 2009 8:10 AMGood post.
It is not always the high-level abstractions you need to examine, it is often the lower levels too.
Do I really need slick low-profile mag-alloy wheels on my wagon or would a round wooden one do? Do I really need to optimise the hell out of this library fragment I am writing or do I just make it work perfectly?
Fear is the mind killer - don't be afraid of just doing it right.
Paul
Paul on June 12, 2009 8:15 AMYou need Clippy to pop up with a message saying this:
"You look like you're being interrupted by a pop-up paperclip with delusions of Clue! Would you like to get a real development environment?"
And when the user clicks Yes, download and install Emacs...
Oh, and to those "programmers" who roll their own code instead of relying on tested, peer-reviewed libraries used (and debugged) by thousands of people... please post with the name of the company you work for and the products you sell. I need to make sure I never accidentally deal with you, for security's sake.
Eric TF Bat on June 12, 2009 8:22 AM
@Mecki
> I'm never working at any level of abstraction other than abstraction I have written myself.
So you do all of your coding in assembly?
qwert on June 12, 2009 8:24 AMOne big reason people roll their own is both the advantage and disadvantage of third party libraries: somebody else wrote it.
In many cases, this is an advantage because the code has been written by somebody that understands the problem to be solved. The code has undergone testing and has proved to be reliable.
Then there's the other side of the coin. How do I know I can trust other people's code? I have seen some really awful code in 3rd party libraries. How do I know the code is processor and resource-efficient? Bug-free? Secure? Safe? Not filled with back doors or phone-homes? What side effects does it have? Is it supported? What happens if I find a bug?
I remember installing Drupal a couple of years ago, based on its popularity. I couldn't believe the number of errors generated when starting it up the first time. When I examined the source code, it was some of the worst PHP code I had ever seen. It didn't do any error checking and in many cases, errors were being ignored or suppressed. Hopefully, Drupal has improved since then, but you won't find me using it any time soon.
An email parsing library I had been using was extremely memory-intensive because it used recursion inefficiently, resulting in multiple copies of email data being created. With emails containing large attachments, the library crashed with memory errors.
When I write the code myself, I know what I am getting.
@Gary has it right. Use other peoples' code when it doesn't suck. When it does suck, make your own. If you don't know if it sucks, find someone who does.
I sometimes find myself paralyzed for fear of re-inventing a wheel. I'm afraid I'll unknowingly "roll my own" implementation of something that already exists in the .NET Framework.
It means even the simplest tasks have to be started with research of what might already be there, how it works, and how well it may or may not work for my particular situation.
It's like putting on the brakes and pulling off the road to search for a faster route. It doesn't always pay off.
Zack Peterson on June 12, 2009 8:36 AM"The only problem arises when one million websites build their security using EncryptStringForBrowser() and then someone finds an exploit on that function call."
A good counter, but there's a counter to that too - if someone finds an exploit in widespread library code, it is known about quickly* and fixed quickly**. All that is generally required to fix an exploit that you may not even understand is a patch or replacement that someone else provides.
* whether that be because the person who discovers it has the integrity to refrain from abusing it, or because those without that integrity cause problems that are noticed and traced back to the flaw.
** Even if that 'fix' is to advise people not to use it for the time-being.
--
LOL @ one of the re-captcha words being 'orange' :)
Schmoo on June 12, 2009 8:37 AMI have exactly one objection to this (and it doesn't apply all the time). In general, the more abstract a system is, the less efficient it ends up being.
For example, ASM vs anything else; at worst, you can always make the ASM as good as the other option and if you know what you are doing, you can often make it better. (This doesn't always happen though).
The other end is languages like C++ or (IIRC) Lisp where well written code can get processed at "compile time" down thought the layers of abstraction until it's just as good as the lower level option. The trade off here is that the language and libraries require much more work to develop and the tools take longer to run.
BCS on June 12, 2009 8:40 AM@Gary:
"When I write the code myself, I know what I am getting."
When you use open source, you know what you're getting.
Back in the day, when VB6 libraries were compiled from C++ (which the author rarely made available even if a VB6 dev could read C++), you had a point. Nowadays, there are libraries for everything written in your native development environment, that you can include as .Net source code in your projects. If you don't have time to review and evaluate the source for these libraries, you certainly don't have time to re-write them.
Modern libraries are written using popular and proven design patterns (sometimes all of them at once it seems!), in your native language, with source code provided and oftentimes free of charge. Maybe you would have done something differently, but I'm just not as confident in the likelihood of out-thinking the internet community. I'll keep using CodeSmith, Google Charts, CodeSharp, ActiveRecord, NetTiers, etc. My apps are practically generated for me, and I have no problem admitting they're better than I could have written, minus the business-specific knowledge I've gathered working for my organization.
Chris McCall on June 12, 2009 8:44 AM"painfully clawing our way up the evolutionary tree of abstraction...from assembly language"
Which was a pretty high branch compared to machine language....which at least didn't require timing memory drum spinups and flipping toggles. Which were a vastly better IDE than soldering irons and wire strippers I'm sure.
This post is a handy reminder that probably 99% of the world's projects -- even in C -- is a long way from bare metal. Everything we do is standing on the shoulders of giants.
Paul Souders on June 12, 2009 8:50 AMI will just rewrite this part of the post:
"Q: How do I write this in JavaScript?
A: You don't. You use GWT."
jQuery is still JavaScript, GWT is actually one abstraction level up.
Someone told me one day: "There is no problem in IT that can't be solved by adding an extra layer of abstraction", and I must admit he hasn't been proven wrong since...
Olivier Gérardin on June 12, 2009 9:00 AM> The best encryption is no encryption
That might just be the most overly soundbitey misrepresentation of fact I red today.
You might have meant to say something like "The best way to make sure noone will read your message is to make sure no one can access the message". This does not preculde any need to make it (cryptographically) hard to read the message in case access was gained forcefully [1].
For the record, no encryption is actually the worst encryption.
[1] see http://www.vaserv.com/ today: you'll find that 100k websites have been compromised (in this case the (virtual) servers had been _completely deleted_, but they could just as easily have lurked around and harvest all CC, e.g., details on every site, lest it were decently protected by cryptography)
[2] For linguists, pedantists: Except in the strictest sense that it cannot be bad encryption if it isn't encryption in the first place. Either way, you could safely say that no encryption does not encrypt anything very well.
I'd have to agree with Gary. The problem with using a black box is not knowing what lies therein. If the source is provided so you can review it yourself, and it is used widely (such as JQuery), then I would say the concerns are mostly alleviated. I would still be skeptical of using a lesser-known black box for security requirements.
Btw, I just noticed you have an official CAPTCHA now, no more orange. Nice!
Jeromie on June 12, 2009 9:08 AMs/red/read/
seth@mailinator.com on June 12, 2009 9:15 AMThe problem is that people who write encryption code and libraries seem to give zero thought to end users. This isn't limited to encryption, but is a problem related to any intellectually hard problem space.
Harvey on June 12, 2009 9:22 AMI feel a moral obligation to remind people of the best clippy ever made. Bonus points for leaving it running on a co-workers unlocked computer:
http://www.rjlsoftware.com/software/entertainment/clippy/
His text file is also customizable so you could add the "deprecated" classes/methods/members to the phrases that just ooze with cuteness.
Goyuix on June 12, 2009 9:35 AMEven if the source code is available, do you want to spend your time analyzing somebody else's code? Even if you choose to do so, you still would need to become an expert on solving the initial problem in order to determine if the library code is:
- doing it right
- doing it efficiently
- handling errors properly (do you even know what the error cases are?)
- not creating any side effects
If you need to know all these things in order to determine if somebody else's code will work for you, you already know enough to write it yourself.
I'm not proposing that we never use 3rd party libraries. I'm saying that it's not a panacea and there are valid reasons not to do so.
So, are you going to use a library? Which one?
Daniel on June 12, 2009 9:43 AMAmen!
code monkey on June 12, 2009 9:45 AMI won't disagree with using a special purpose library for such things, but I find it funny that in doing so you can (1) not understand the encryption, (2) copy and paste code from the internet and (3) not peer review the code all at the same time.
H on June 12, 2009 10:12 AMOn the 3rd party lib issue; I like my solution best (who doesn't?): find a domain where you don't need any. I, time and again, find that where I most enjoy working is at a level where the only abstractions I deal with are native to the library. It's not a matter of NIH but rather I like working just a smige above the language.
BCS on June 12, 2009 10:18 AMIt is important to have a wide breadth of knowledge in the programming languages that you use, even if it is just knowing that some library or framework exists for some purpose, but not necessary all of the details.
If you do not have a wide breadth of knowledge of the programming languages that you are working with then you need to get out of the hurry up and code mentality and take some time to research the heck out of it so that you can make informed decisions when you sit down to write the code or plot out the design.
You should be able to coherently explain why you coded every piece of code the way that you coded it, and part of that explanation may be why you chose not to use a 3rd party library or existing API.
I cringe when I am working on code that was obviously well written by a smart person who spent a lot of time working on it, but didn't have to because that functionality was already available in a standard library or relatively well known 3rd party library. It was just unknown to them.
Pete on June 12, 2009 10:23 AMApparently some of us need to be reminded of the Open-SSH debacle. It was broken for two years. It's probably still broken on many systems.
One other problem I've seen recently has involved recompiling unchanged (know, proven) libraries with updated compilers. Surprise: the newly compiled library may give very different answers. No rest for the weary.
Jesse McNelis nailed it (or at least one aspect of it). Documentation for third party libraries tends to look like either this:
or this:
about:blank
Neither one of which is useful to anyone.
Daniel Straight on June 12, 2009 10:59 AMI just riffed on this 24 hours ago. The "right" level of abstraction is the one where you no longer need to look under the covers to fix things.
http://clipperhouse.com/blog/post/The-cloud-The-cloud.aspx
I find the article kind of confusing. It talks about the problem that comes up with not using already available and well tested code but writing it from scratch(Reuse),with being on the wrong level of abstraction. BTW i picked it up from here (http://www.reddit.com/r/programming/comments/8rydz/the_wrong_level_of_abstraction/c0a8r6o)
ashish_0x90 on June 12, 2009 11:38 AMOne thing we can notice:
The more the language (or the library) goes towards abstraction, the easier it gets to solve real-world problem.
As programmers, we tend to think that the more we go down to the bits and octets, the more we go towards reality. But for regular people, the reality resides in the analogic macro world, not in the numeric world of micro chips.
Jerome on June 12, 2009 12:03 PMThat sounds like the day i discovered the boost libraries for C++, in a moment I felt how all the code I had written so far should be changed
schizoid on June 12, 2009 12:03 PMAbstraction... You keep using that word. I do not think it means what you think it means. Whether you use a library or not has nothing to do with abstraction. Abstraction has to do with design. You can use a library with poor abstraction and you could have made your encryption method properly abstracted even if you didn't use a library.
tieTYT on June 12, 2009 1:09 PMIt's really hard to avoid reinventing the wheel; there's an anti-pattern I call "stratification".
I had the recent joy of looking at code I wrote 15 years ago. The first thing I noticed was that I made extensive use of linked lists, but I repeated a lot of the code. At the time, it was intentional; function calls were too expensive. But now we have fast computers, so I resolved to write myself a linked-list helper library.
A little further into the project, I was looking through some other code, and - what do you know! - found a linked-list helper library. It seemed like just what I needed. And that's when I realized - it ought to, because *I* wrote that. I just hadn't looked at it in 15 years, and neither had anyone else. So if I can't even remember to use my own abstractions, how can I expect the rest of the company to?
Basically, the more you write high-level libraries that wrap lower-level ones, the less you use the low-level libraries. Thus, the less you understand them. At some point, you forget they even exist. At that point, you will - inevitably - write an even-higher-level library that recreates the lower-level functionality ON TOP OF the high-level library.
If you build a mail system that relies on a database, someone will inevitably decide that a mail system, with its built-in queueing, routing and simplicity, is a great transport for asynchronous replication - which in turn can be used to create a distributed file system. And once you've got a file system, well, wouldn't it be great to get a database running on it?
Jay Levitt on June 12, 2009 1:17 PM>I found your post on the same page as this link:
>http://www.daemonology.net/blog/2009-06-11-cryptographic-right-answers.html
>...on Hacker News (http://news.ycombinator.com) - coincidence? :)
That's some good advice and a lot of dogma. For example "Use RSA-OAEP/-PSS" comes with a corollary of "... and be incompatible with every existing crypto-using application (and most crypto libraries) in existence". "Use Group #14 with a generator of 2" comes with a side-order of "... but be aware that if anyone ever finds a weakness in these monoculture parameters, it's game over for all your deployed apps (and everyone else's too)". "Use AES in CTR (Counter) mode, and append an HMAC" comes with a side-order of "... and then get a cryptographer to go through your code with a fine-toothed comb fixing all the side-channels you've left yourself vulnerable to by doing it yourself". In fact the only advice I agree with completely is "use a crypto library done by someone who knows what they're doing and don't even think about doing any of it yourself".
Dave on June 13, 2009 5:22 AMMy brother always say: "There's no hard work but unappropiate tool"
Rafa on June 13, 2009 7:16 AMI would still say that the root problem was that you were using the wrong solution (i.e. storing and importing data from client), but that's difficult to prove in theory.
Pies on June 13, 2009 12:14 PM>>>ask yourself: am I working at the right level of abstraction?bold and italic!).
RUNTIME is essentially the same as PORTABLE. CONTAINER, as the name suggests contains various portable entities, including PORTABLE.
In the process to build RUNTIME out of PORTABLE, my system typically just pass the data it doesn't understand so the next stage (let's call it "link" stage) has the data in place.
Or maybe not?
I spent a whole day trying to figure out how to properly "link", with code getting grosser and grosser. At the end of the day, it was so magic I had to review the "linking" rules; turns out that the line count goes 400% or so. For something designed for non-programmers this meant sure fail.
The day after, drawing on paper the pipeline giving me PORTABLE, I figured out an important aspect I overlooked:
Just because RUNTIME contains the same data than PORTABLE (+ the runtime optimizations) it doesn't really have to contain the same info! It turns out that the rules are not needed at all if the high-level "link" info is moved back to "low level" PORTABLE representation.
Abstraction may effectively reduce the need for informations. In this case, it increased the need.
Reviewing the whole pipe, I had tracked at least 3 important quirks, 2 of which were design byproducts; the last was an implementative issue.
So, I reworked PORTABLE to include the "link" information (currently, 2 special bytes). Problem solved. No special rules required. No complicated "link" required.
Long story short: abstraction is powerful, and it is both ways. Some apparently difficult problems have surprisingly trivial solutions at a different abstraction level!
>>>No, I wasn't working at the correct one.
Once again, I found a solution coming from LOW abstraction as opposed to higher.
NOTE: the above was meant to be a recent horror story.
I had the unfortunate idea to add a tag to it and it went severed.
When I read "NO HTML" i think than HTML isn't supported because it is escaped. I don't really figured out up to now that it actually causes messages to be severed.
So, no HTML... ok, but at least some input sanification/escaping?
What's wrong with bold and italic really?
MaxDZ8 on June 13, 2009 1:07 PM..........
this is about that blog cliches thing.
i must say i agree with you on some 12 points of yours, but the diary thing was a bit troublesome. of course, the reason why some web space is given to you is undefined, and i am looking for answers still. but you have every right to utilize that webspace of yours as your diary, and when i say diary, i do not mean what you did as a routine, but anything that contains your thoughts.
hope a quick reply
vivek.
ps:i am aware of all the disclaimers you put in that post, but i think i may get a desirable answer from your side. you seem to be an ardent blogger, by the way.
vivek on June 14, 2009 2:01 AM..........
this is about that blog cliches thing.
i must say i agree with you on some 12 points of yours, but the diary thing was a bit troublesome. of course, the reason why some web space is given to you is undefined, and i am looking for answers still. but you have every right to utilize that webspace of yours as your diary, and when i say diary, i do not mean what you did as a routine, but anything that contains your thoughts.
hope a quick reply
vivek.
ps:i am aware of all the disclaimers you put in that post, but i think i may get a desirable answer from your side. you seem to be an ardent blogger, by the way.
vivek on June 14, 2009 2:03 AMOne thing that isn't mentioned enough is the vital question:
"HOW LONG DOES IT NEED TO REMAIN SECURE?"
To me this is more important than whether something should be encrypted or not. A password, for example, may not be stored at all but must still be transmitted in some way, and I for one don't want that done in plain-text.
Counter to popular belief XOR ing or even ROT13 are forms of encryption. VERY basic, and very easy to crack, so now they are only used to post answers on blogs and web pages so that the answer isn't immediately apparent and the reader can have a go at solving the question.
So - in that instance the data doesn't need to be encrypted for very long at all, which is why XOR and ROT13 are appropriate. The longer the data needs to remain private, the longer it should take to unencrypted the data. The time taken to unencrypted given the CORRECT key is directly related to the time taken to crack the encryption with ANY key.
So what about passwords? Well - if updated regularly an encrypted password only has to remain secret for a couple of months - maybe a year or two to be safe. I'm not suggesting they should ever be encrypted - they should be hashed, but even then there are reverse-hash mechanisms, but the variety of outcomes would still take a while to test. So in that case higher level of encryption should be applied - say an encryption of the password, then a hash, then an other encryption.
What about private data like banking details? I believe that this sort of information should remain private for at least the lifetime of the person. Can we really guarantee this type of security using current technology, when computers are roughly doubling in speed every 18 months? What happens when quantum computing comes on line?
Even the current standards don't offer much protection in the long term.
DES is no longer even considered for encryption, but each password had up to 72 thousand billion possible combinations. But DES is now calculated quickly, so it takes very little time to try all of these combinations. DES is about 35 years old, and computers now make short work out of it. Any old data that was encrypted using DES is now insecure.
Triple DES is closer to 10 years and it wasn't long ago that Triple DES was considered rock solid, but now it is considered less than optimal. Yet there are 370 Trillion Trillion Trillion Trillion combinations. And that is for every block - so with appropriate salting it was considered almost impossible crack Triple DES. But with today's computing power, and the ability to grid zombies these encryptions can be cracked.
Triple DES is still too hard to crack quickly, so it is still used in the payment clearing technology, but it can be cracked over time, and so shouldn't be used for stored data.
I think this is one of those areas that you need to have the knowledge to ask the right questions at the outset AND damb well make sure you have the skills to answer them correctly.
To paraphrase Simon above, the biggest lesson is that nothing will help you if you genuinely don't know what you are doing.
http://www.prlsoftware.com/des-encryption.aspx
Your clippy also needs to tell you that on 10.10.2004 you shipped code that uses AES 128bit which was appropriate at the time but is now insecure. There are 214 copies still in the wild using insecure encryption. Obsolete only works when you recompile / rerun static analysis like fxcop.
All of the below is given that a certain developer is even marginally aware of good techniques. That's already cutting out 60% of people who write code.
From my experience, whomever in MS created the API's didn't follow the BCL team recommendations. Have a simple API for basic tasks, and provide a in-depth API in addition to a straightforward transition to using the in-depth API. But as mentioned, very little in crypto is basic, and MS's own implementation and defaults don't help at least for me as I only ever write web farm server software.
MS appeared to create crypto that was heavily tied to the logged-in user, which works great for Windows Forms type software, but not so great otherwise (and I still don't trust the implementation of encrypted files in windows if it ever loses my private key). I would never create a backup of an encrypted file.
The documentation, when you could find it, was horrendous. I remember trying to do some of it back in COM+ days, and lets just say i must be an idiot because i never could figure it out, or at least wasn't confident it was done correctly. Coding by coincedence indeedillydoo.
(In the end I downloaded reference code for a simple symmetric encryption scheme which was _good enough_ when all other factors were weighed in. I could test it. It was straightforward. Job done in less time than I spent reading CryptoAPI docs.)
Playing the devils advocate ...
- Crypto in .net is an API.
- I might not like or be able to understand the decsisions made by BouncyCastle, etc (or have the time to)
- My team/company may forbid external / 3rd party libraries (it is a dependancy)
Too many developers scratch their heads for how to actually get crypto to work and deployed, and when presented with a deadline we have:
1. proper crypto-api. (too) hard to use. don't really understand. my code will probably be buggy anyways
2. use some other library. too long to evaluate. are they secure? can i see the code? What do I do when they update their code. Do I always need to update?
3. do something basic and write your own code (too many XOR or hex-encodings)
4. no crypto
You advocate ditching 1 for 2, but in my experience people typically go for 3 and 4.
Crypto is a moving target, so any decent API needs to take that into account. What happens to old data? Old keys? Revoked keys? Do I care? I don't care. Lets go shopping.
So what do I think? Forget API's.
Encrypting the communication channel is better than manually encrypting the data in the database. If someone has your database or local file access, you're already stuffed.
Protect your browser. If another website can read cookies or know anything about other websites you visit, you're toast.
Protect your OS. If someone can install a keyboard sniffer, you're toast.
Protect your users. If someone can socially engineer details out by calling support staff, you're toast.
Spot on but off the mark.
thx
Phillip, you posit what sounds like an interesting idea at first but upon closer inspection, isn't very interesting at all. Most people are not working on a problem thinking "I need this to be secure for exactly 2 weeks but after that I don't care." Your example of ROT13 is not really secure at any point so I'm not sure what you mean by that. It's more interesting to think about what the sensitive data means to you. If the data is only worth about, say, $1 million (perhaps because of a lawsuit resulting from a breach) then spending $100 million on securing that data may not make sense, depending on other factors as well.
The idea that encryption algorithms become weakened over time is fine but this is only interesting in your example if people built applications with the desire to never change them over 10 years, which is not an interesting case either as most software that goes unchanged for 10 or more years becomes vulnerable for other reasons besides the viability of the encryption algorithm.
Charles on June 14, 2009 7:40 AM@Rodger, your view of not caring about encrypting data because if they can access it, you're already compromised falls down. If some data is accidentally exposed because it was not encrypted, do you really want to pay the data breach penalty just because you have this view? For a trivial example, if you store passwords in a User table, do you really want to risk exposing plaintext passwords because of a bad SELECT statement or some other mistake just because you subscribe this naive view? I don't think its worth it, personally. It's probably easier to just do it properly in first place.
Charles on June 14, 2009 7:46 AMCharles:
I take your point, but I disagree.
I agree that programmers don't think that "this needs to be secure for x amount of time", but they SHOULD. It directly impacts the performance of their program AND the long-term security of their data. The very best security mechanisms are SLOW, and the very worst are fast - it IS a scale. It's not about dollars, it's about time. It takes time to encrypt the data and it takes time to decrypt. The longer that time, the better the encryption but the worse your program performs. That's why the question is so important, and that's why it is important to get it right - especially if you have a lot of data.
Also - this is NOT about updating your processes over time, this is about data you have RIGHT NOW that gets stolen/acquired RIGHT NOW.
I recently went to a fraud presentation by a top police official, so I found out some of the practices of the criminal element in our society.
One of the things the criminal element are doing is collecting information that is available to them now, even though they can't crack it now. Storage space is cheap. They have dedicated computers attempting to crack the information, and as time progresses they are more and more successful at getting historic information.
So when I say "how long do you need to secure it" - I mean, assuming someone gets the encrypted data from your storage NOW (and if that wasn't a possibility the data wouldn't be encrypted) how long before they can get to that data easily.
Cases of sales staff stealing company databases isn't unheard of - and much of that information, such as customer contacts, will still be of interest for at least a decade.
In this country (Australia), CDs and Laptops of top people from government departments, including military, have been stolen/lost. We better hope the developers of those applications DID NOT intend on updating the program and encryption every couple of years. We better hope they made the encryption very rigorous RIGHT NOW.
RE ROT13 - anything that obfuscates the original text is a cipher, even if it is so easy to get back it can be done in your head (think of those code rings).
@Charles,
API's don't protect you from bad programming. It's a red herring and doesn't detract from my position.
My overall point was the individual program is not the right level of caring about encryption.
@Philip,
"The very best security mechanisms are SLOW, and the very worst are fast - it IS a scale."
One of the key factors in the AES selection process was performance in various software and hardware settings. So you want to tell us that they've finally selected the candidate with the worst security for AES?
The only encryption algorithm provably uncrackable (Vernam) is the fastest available -- it takes one operation per byte for encryption/decryption, generation of the key material aside.
Secure on June 14, 2009 11:56 AM@Charles,
API's don't protect you from bad programming. It's a red herring and doesn't detract from my position.
My overall point was the individual program is not the right level of caring about encryption.
Rodger on June 14, 2009 7:36 PM
>> I'm never working at any level of abstraction other than abstraction I have written myself.
>So you do all of your coding in assembly?
Actually assembly is an abstraction for machine code, and after that the question is can you trust your CPU?
Regis on June 15, 2009 7:23 AMThere's nothing worse than working with 3rd party software that doesn't work.
Everything is all well and fine when things go right, but when that 3rd party library f**ks up you'll wish you had written the code for yourself.
Maybe a more instructive blog would have been to write an analysis of how much time it takes to get a 3rd party library working, with all the necessary work arounds, lost development time etc against the time taken just to get straight to the spec, implementation and testing.
Jackie on June 15, 2009 7:38 AMI have to agree with @Jackie.
Even if you do go with a 3rd party you aren't guaranteeing that you are solving the problem. Once you have that dependency then you need to trust them to do the right thing.
At least with an OSS vendor you can tweak it yourself if needed but then you are back right at the start where you have to be able to understand the code yourself.
Marshall on June 15, 2009 11:37 AMGibson's crusade against raw sockets isn't just a wrong analogy, it also wasn't holy - it was crying wolf, overdramatizing, and generally pretty pesky.
Raw sockets required admin privileges. When you have that already, nothing stops you from loading a driver and sending raw packets that way - more work, but no problem at all. If Gibson had had half a clue, he wouldn't have shouted his mouth off to make MS remove raw sockets, but to make the default account in XP non-admin.
f0dder on June 16, 2009 5:29 AMIf I am paid to reinvent the wheel, then I will do so.
AC on June 16, 2009 6:04 AM>Raw sockets required admin privileges.
... which is the default under Windows.
>When you have that already, nothing stops you from loading a driver and sending raw packets that way - more work, but no problem at all.
It's quite a bit more work doing it that way, and given the state of malware when Gibson pointed out the problem (fairly primitive user-mode stuff), this was a very real concern.
>If Gibson had had half a clue, he wouldn't have shouted his mouth off to make MS remove raw sockets, but to make the default account in XP non-admin.
How about you do it for him? Good luck with getting that change made.
JM on June 17, 2009 3:58 AMthanks you..
Okey oyna on June 17, 2009 9:19 AM@Secure
Okay - you are right, I should have been a little more explicit.
I took as given that itterations (key bit size) is the greatest security factor, after that (assuming the same bit size is acheived), then slower algorithms are more secure.
The security of encryptions comes by the number of possible iterations of "keys" multiplied by the time taken to try each iteration. For instance, an encryption that takes 10 mins to try when there are 1,000 possible keys is way more secure than an encryption taking 10 milliseconds to run with 100,000 possible keys.
So you are right in that it isn't "JUST" the algorithm. But given the SAME key length (in bits), then a faster algorithm is less secure.
Let’s say that Triple DES runs slower than AES, then bit for bit it would be more secure than AES.
The benefit of AES is the key size of 192 or 256 bits to DES’s 56 bits or Triple DES’s 168 bits.
That is – there are at least 16,777,216 more combinations to try using AES compared to 3DES. Even if it is faster to encrypt and decrypt, it isn’t 16.8 million times faster. Use the full 256 bits and it leaves 3DES and DES in the dust as far as security goes.
Even so - as a general rule, the implementation of encryption that SLOWS DOWN the encryption/decryption process is more secure given the same number of itterations because... it takes longer to crack...
Triple AES, for instance, will take you three times longer to encrypt AND will increase your bit key length three fold. This will take you longer to process - especially large data files - but depending on your requirements may be what you require.
And that's where it comes back to my original point, you have to first determine how long you need your data to be secure. The encryption process you select should meet that requirement.
Oh my god... What a load of selfinvolved nabcake BS.
I think you should clearly stop coding because you are giving us all a bad name.
Do it. Do it now!
Fred Finkle on June 18, 2009 5:54 AM@Philip,
a very strange point of view, and a bit naive. The fact that your implementation is slow means nothing for the attacker, who probably has $$$ to buy superfast hardware in superlarge amounts and to hire superintelligent programmers who will write a superoptimized implementation fine-tuned to the hardware. Speed is relative. But even given this, it simply doesn't matter.
Let's play the big integer game, and given your 16,777,216 comparison, I know you'll love it. When we have a key size of 128 bits, then we have
340,282,366,920,938,463,463,374,607,431,768,211,456
different keys to test. If we assume that our superattacker is able to test 1,000,000,000 keys per second, which means not only decryption, but includes all necessary tests like character statistics and the search for known file headers to be sure that he has the key, and we further divide by 60 (->minutes), 60 (->hours), 24 (->days) and 365 (->years), he still needs
2^128/(1000000000*60*60*24*365)
= 10,790,283,070,806,014,188,970 years
to test the complete key space of 128 bits. Given that the universe is only somewhat around 15,000,000,000 years old, he has to spend very much more $$$ to test very much more than the 1,000,000,000 keys per second.
Does the speed of the algorithm matter? No. But it matters for me if the decryption of my encrypted hard disk runs with 1 MB per second or with 10 MB per second.
@Secure (good to see you are happy to use your real name with a URL)
Some good calculations, but unfortunately it misses some key points. I feel like I must have glossed over some "assumed" logic so I'll explain more in detail.
Before I start with my rant, I’ll just ask the simple question – are you arguing that a slower encryption IS more secure, but the difference doesn’t matter because it would take so long to crack the “less secure” algorithm that it doesn’t matter? If so then we disagree only on time frame of computers and hacking. If not then we disagree on everything.
First of all the bit I agree with – that big $$ helps, but even then it will only help to an extent. The big $$ is for more processing power by forming a grid of computers, where each computer hacks a different part of the encryption (different key range). Every computer you bring on-line effectively reduces the bit range of the encryption. That is - if you only had 2 computers and 32 passwords to check, 1 computer would check the first 16 while the other would check the next 16.
Therefore we have a distinction between actual bit rate of the encryption and effective bit rate.
1 computer = full effective bit encryption
2 computers = minus 1 effective bit encryption
4 computers = minus 2 effective bit encryption
8 computers = minus 3 effective bit encryption
and so on so that with 1,024 computers the bit range has been reduced by 10, taking the encryption to effectively 118 bit encryption.
Obviously adding somewhere between these values reduces the time, but doesn't calculate as an exact bit reduction.
In historic tests, approximately 100,000 computers were used to crack DES (it took about 22 hours but that was a decade ago).
Let’s round that off to 131,072 as that reduced security bits by 17, taking the 128 bit encryption down to 111 (it reduced DES to effectively 39 bit encryption).
This is the way to speed up encryption “right now”… and this is where either big $$ or zombie PCs would come into the picture. Even then, it is unlikely to be successful with anything but a dictionary or close to dictionary attack.
But secondly, and most importantly, you are still forgetting or leaving out the fact that computers are getting faster. The computer I'm using has 4 cpus, which can process four independent attacks (4 passwords at once) AND computers are faster AND cheaper AND they are bringing in new technology, such as GPU which can offer many-many-many times more parallel hack attempts.
But let’s stick with an approximation that flops double every 18 months… that’s exponential computer speed growth! Exponential growth is scary and should be on the minds of all those who need to protect data. It's scary because it grows really fast constantly making unthinkable processing tasks possible.
If a computer doubles in speed every 18 months, then the effective bit rate of the encryption will decrease by 1 every 18 months. This means that the time taken to crack a 55 bit encryption this year equals the time it will take to crack 56 bit encryption in 18 months time.
Encrypted data that took 24 hours to crack 18 months ago will only take 12 hours to crack today and 6 hours in 18 month’s time. That’s a very sharp reduction in just 3 years, going from 1 day to 6 hours to crack an encryption.
So in 18 months 128 bit encryption becomes effectively 127, in 15 years it becomes 110 bit. In 150 years time it’s effectively 28 bits (comparable to 28 bit encryption by today’s standards). And with my back-of-the-envelope calculations, that would take about 5 seconds to crack. Obviouly that first point is still an issue - crackers with large arrays of computers can still get the data much sooner than 1 computer processing away.
Anyway - it would still be 128 bit encrypted data, but it would only take 5 seconds to decrypt, which is as good as 28 bit encryption on today’s computers. At the same point in time the 192 bit encryption would still take over 8 hours and 256 bit would take until the end of time…. 1.799*10^23 years… well at least for a couple of years until computers are fast enough again.
But all of this is using the DES calculation times, using a faster algorithm reduces the power of the encryption because that 8 hours with 192 bit encryption would be far less. In fact, the faster the encryption, the sooner that it will be viable to crack the encryption cheap and quickly.
That is to say that a slower encryption method IS more secure key bit for key bit, and the results of this security difference WILL be seen within our lifetimes.
To re-iterate my main point, the first question should be how long in time data should remain secret. Keeping in mind that any encryption halves in power ever 18 months or so, calculations should be made as to how long it will take to crack the encryption at various points in time until the privacy of the data is no longer a concern.
To meet that question you need to be aware of the bit rate AND how fast the encryption algorithm itself takes. Yes it DOES mean approximating the future speed of computers, which is NOT going to be accurate, but at least it is better than just wrongly assuming that a fast encryption method is necessarily better than a slow one given the same bit rate.
@Philip,
"@Secure (good to see you are happy to use your real name with a URL)"
and then giving a wrong name and a wrong URL. What does it matter in a forum that doesn't require registration? Since it is not the first time that this typical reaction happens, I guess you felt personally attacked by "strange point of view" and "naive". If you felt insulted I apologize, but it was in no way meant personally and only my honest opinion about your arguments. In fact I don't care who you are, because I want to discuss against your arguments, not against your person.
Besides that anonymity and privacy is strongly related to encryption and security. Isn't it a bit inconsequent to care about the future lifetime of your encryption but not about the present trackability of your person? No one can currently see the encrypted files on your hard disk, but everyone can datamine a lot of information about you if you always use your real name.
Back to the topic. All you say is right and well and I'm well aware of it. It is in fact the exact reason why the average key sizes for encryption are permanently increased over time. But as you may have noticed, I've talked about the naked number of 1,000,000,000 keys per second, without giving any other indications. Yes, if the attacker buys twice the amount of computers, he can test 2,000,000,000 keys per second. If he spends large $$$ 18 months later for a complete new set of computers, he can test 4,000,000,000 keys per second. I didn't deny this.
But the one information you didn't give was any quantification. Slow, fast, what does it mean?
1) How many bits will the slow algorithm be slower than the fast algorithm? You're obviously talking about orders of magnitude here.
2) How is fast and slow defined? I can write an encryption algorithm that uses a lot of fancy maths, squareroots, sin, cos, etc., needing 1 second per byte, while effectively boiling down to ROT13.
3) What implementations are you comparing? Interpreted language, which may be slower than compiled language, which may be slower than handcrafted assembler, which may be slower than a specialized hardware chip? While compiled(A) may be slower than compiled(B), hardware(A) may be faster than compiled(B), even with a highly optimized B.
Does it make sense to look, say, 50 years into the future? What about quantum computers that may cast all the traditional encryption schemes into the void? What if new cryptanalytic methods are invented that break the slower algorithm, but not the faster one? This estimation of encryption lifetime is such a rough estimation that IMHO the speed of the algorithm is the least concern.
And even if all this doesn't matter and the slower algorithm proves to be more secure for the same key bit number, there is one remaining question: So you've estimated that the slower algorithm with N bits will fit your security needs. Why not simply use a faster algorithm with N+64 bits?
Secure on June 22, 2009 2:40 AM@secure - obviously my humour didn't translate, the commont on user name and url was meant to be a joke... no insult was taken about you not using it. I think many names here are fake...
I actually agree with the points you make about your privacy.
You also raise some good points. Some really good points. I'll clarify but from what I can see you already have an excelent grasp on everything.
1: I've only done software benchmarking on various C based encryptions, but I suspect that correctly desinged hardware would provide completely different results. That is - chips that decrypt in parrallel. So I can't give answers for that.
2: I just use the one simple measure - how fast the raw algorithm (in C) takes to decrypt using a test key. This is averaged over several thousand itterations. I mentioned "ecryption" speed several times, but this is actually incorrect unless they encryption is litterally the reverse of the decryption or "symetrical" (which is not the case with private/public key encryption).
That's the point - the time taken to try a key AND the number of combinations of keys is the security strength.
3: Very good point. Going back to 1 - I use software (C with as much optimisation as I can including processor APIs). But your point on hardware is spot on.
RE quantum computers - they may affect more's law but they aren't the leap they were 10-15 years ago. When they come on line they may still be close to in line with More's law. Even so - your point is valid and predicting future computer power is like predicting weather in the future, in the end we are just making rough guesses. We are going to be out but by how much and in which directions?
RE slower vs faster + 64 bits. Yes - agreed and I said that earlier. The difference in keys between 168 3DES and 196 AES is something like 16 million and there's very little chance of AES is 16 million times faster, but if it is twice as fast as DES, then effectively it has 195 bit encryption compared to 3DES. If it is 4 times faster, it has effectively 194 bit encryption. You can adjust DES to go up in 56 bits (i.e. 4DES at 224 Bit)
So for border line cases it isn't a case of just looking at bits.
Here is the equation I would use for strength of encryption:
Strength = TimeTakenToDecrypt * NumberOfKeyCombinations
Where NumberOfKeyCombinations = 2^64 for 64 bit encryption.
So I would forget the focus on bits AND forget the focus on TimeTakenToDecyrpt, it is both that are important.
HOWEVER - this will probably only matter when comparing two algorithms that have similar bits but different speeds - it would affect the security plus or minus 1 to 4 bits.
And back to the original issue - you can do various bit encryption with varying speed (AES can be done multiple times, giving enormous bit length).
I'll throw in two more flies in the ointment. On the plus side we are assuming that the hacker knows the encryption algorithm. This wouldn't normally be known and therefore, the task is multiplied by every possible encryption option... so if they don't know this they are stuffed!
On the down side, probability states that on average only half the keys need to be tried. The above calculations I used are based on that fact. In fact, proability states that half the time you try to crack an encryption you will find the key BEFORE hitting the half way mark.
But many security sites try to show how long it will take to calculate ALL keys. This would only be required if the very last key to be tried was the correct key, which has a probability of 1 in 2^xxx (so inprobable that it would only happen in an infinate improbability machine hooked up to a particularly strong pot of coffee).
Philip on June 22, 2009 8:01 AMOh - one last thing RE aliases. I've always thought it would be a great idea to change my name officially to "The Obviously Innocent Man".
That way if I was ever charged for anything the Judge would have to say "How does the Jury find The Obviously Innocent Man?".
Philip on June 22, 2009 8:36 AMDoes it make sense to look, say, 50 years into the future? What about quantum computers that may cast all the traditional encryption schemes into the void? What if new cryptanalytic methods are invented that break the slower algorithm, but not the faster one? This estimation of encryption lifetime is such a rough estimation that IMHO the speed of the algorithm is the least concern
Oyunlar on June 30, 2009 7:32 AMkoxp
koxp on July 10, 2009 9:07 AMaöf sınav soruları
aöf on July 18, 2009 12:55 PMöaf
aöf on July 18, 2009 12:56 PMI recently did a presentation on web 2.0 cryptography, subtitled "A Study in Failure". Those creating web 2.0 apps may find it instructive:
http://www.subspacefield.org/security/web_20_crypto.pdf
Travis H. on July 21, 2009 1:21 PMI'm really very useful to follow a long-time see this as a blog here Thank you for your valuable information.
OYuN on July 26, 2009 7:54 AMthis is about that blog cliches thing.
i must say i agree with you on some 12 points of yours, but the diary thing was a bit troublesome. of course, the reason why some web space is given to you is undefined, and i am looking for answers still. but you have every right to utilize that webspace of yours as your diary, and when i say diary, i do not mean what you did as a routine, but anything that contains your thoughts.
Before I start with my rant, I’ll just ask the simple question – are you arguing that a slower encryption IS more secure, but the difference doesn’t matter because it would take so long to crack the “less secure” algorithm that it doesn’t matter? If so then we disagree only on time frame of computers and hacking. If not then we disagree on everything.
gögüs estetigi on August 19, 2009 5:01 AMI'm really very useful to follow a long-time see this as a blog here Thank you for your valuable information.
Medyum on August 21, 2009 4:09 AMThe comments to this entry are closed.
Subscribe in a reader
Subscribe via email
|
|
Traffic Stats |