Why is writing your own encryption discouraged?

Your question, Mike Azo's comment, and your reply practically could not be a better example of Schneier's law in practice.

Anyone, from the most clueless amateur to the best cryptographer, can create an algorithm that he himself can't break.

You replied,

How can you break it if I send you this "QTCPIGXKUXTGG" ciphertext encrypted by a merely a simple algorithm which you have no idea about how it was encrypted?

Because even though we might not know exactly what your secret algorithm is, the first thing an attacker is going to reach for are common tools to attack substitution ciphers or polyalphabetic ciphers. Given even a few sentences of ciphertext is likely enough to fully recover every plaintext.

The fact that you don't know how to break it is irrelevant. It's trivial to create a cipher that you yourself can't break, but it's another thing entirely to create a cipher that others can't break. And the odds that you are capable of doing it when you're not aware of even the most basic attacks against ciphers hundreds of years old — not to mention modern concepts like indistinguishability under different attack models — puts you at an insurmountable disadvantage compared to ciphers designed by researchers with decades of experience in the field who are building off of modern notions of security and the discarded remains of thousands of failed ciphers that came before.

As an example, even if your cipher is somehow secure against a ciphertext-only attack (it's not), is it secure if I can trick you into encrypting a message for me? What if I can trick you into decrypting a message for me? What if I know part of or all of one of the messages you send? What if you encrypt multiple messages with the same key?

I'll leave you with another Schneier classic, Memo to the Amateur Cipher Designer:

A cryptographer friend tells the story of an amateur who kept bothering him with the cipher he invented. The cryptographer would break the cipher, the amateur would make a change to "fix" it, and the cryptographer would break it again. This exchange went on a few times until the cryptographer became fed up. When the amateur visited him to hear what the cryptographer thought, the cryptographer put three envelopes face down on the table. "In each of these envelopes is an attack against your cipher. Take one and read it. Don't come back until you've discovered the other two attacks." The amateur was never heard from again.

So here's your first envelope. Given a paragraph or two of ciphertext, your cipher will fail to language-based frequency analysis.

Let me know when you've figured out the other two attacks.

Edit: The comment about indistinguishability under different attack models is one reason why most "decipher this message crypto challenges" are completely bunk. They often simply give an attacker some ciphertext, ask them to decipher it, and declare victory when nobody produces the plaintext after some amount of time. Unfortunately that's not how crypto works in the real world; attackers have many more tricks up their sleeve in practice. They can trick computers into encrypting data of their choosing, they can trick computers into decrypting data of their choosing, and they can usually even do these things thousands, millions, or billions of times. Moxie's post shows how even the most terrible, horribly-designed, and obviously insecure ciphers can be effectively impervious when you restrict an attacker to a single ciphertext-only attack, which aren't representative of attackers' capabilities against ciphers as they're actually deployed in practice.

I actually think this is a really good question. The answer is because cryptography is a skill, and like any skill, it takes time to develop. Additionally, you will be pitting your (in)experience in the skill against the skills of those who would seek to break your algorithm.

This is the real reason why: It's not that you just shouldn't do it, period. It's that if you do write your own algorithms, you need to realize that it will take a long time before you create anything that is actually capable of securing your information against a dedicated adversary in the real world.

Cryptography is like sword fighting. You would not sharpen a bamboo stick, swing it around a few times, and then go challenge a pack of thugs to a fight. The reason why should be really obvious. Especially when the pack of thugs in question could be a 3-Letter entity such as DJB or other Nation State level Adversaries.

I say this as someone who is written too many crypto algorithms to count on both hands - I have written plenty of algorithms, but proposed and used none of them. Until you have something that 1. Is faster then AES or Salsa/ChaCha 2. Provably more secure then AES or Salsa/ChaCha, why should you propose or use the algorithm in question?

Lastly, you will likely enjoy studying information theory. You will come to understand that keeping the algorithm secret is not efficient because the algorithm itself has a minimum number of bits required to represent it. These bits (the algorithm) simply become the key. It is well established that it is simply better to concentrate your secrecy into a proper small key with a public algorithm. If your key becomes compromised, you simply change the key. If the algorithm is the key, well, you'd need a whole new algorithm.

tl;dr

I disagree that you should not write your own algorithms, it's that you need to have a good explicit reason for using/proposing your own algorithms. I personally would encourage you to write your own, as it will teach you to understand what does and what does not work and why. Doing so will help you to understand various sorts of mathematics and information theory (basically all of the math that I know I learned because/for cryptography).

In addition to the great answers on cipher complexity, I think it should be mentioned that the security engineering of the code that implements and utilizes ciphers - things like key exchange, key selection, and the generation of entropy - is very hard as well. Ross Anderson has a terrific book on Security Engineering, but an earlier essay of his (from 1993!) introduces some of the main points specific to cryptography (using old ATM security models as his case study), "Why Cryptosystems Fail" (1st Conf.- Computer and Comm. Security ’93, http://web.archive.org/web/20160125024658/http://www.cl.cam.ac.uk/~rja14/wcf.html).

It's discouraged because (A) it's statistically unlikely that a beginner comes up with a strong cipher and (B) lacking a working knowledge of cryptanalytic techniques, a beginner won't know that their cipher is vulnerable.

With most engineering problems outside of security, it will be obvious when things aren't working. If your video codec doesn't work you'll have strange green lines all over the display; if your stairs sensor doesn't work, the robot falls down the stairs, etc. But if your cipher is cryptographically vulnerable, you will still be able to encrypt and decrypt messages just fine. The only problem will be that people without the key can also decrypt the messages, and they won't necessarily be eager to inform you of this.

It is discouraged, for all the good reasons the people already answered you (so I won't repeat).

There is a twist, however (which is the point of this answer) - no matter how weak your cipher is (and it IS weak), it has two advantages going for it:

• it is more secure than plaintext that you had before (even if only by slight margin, as you're not experienced in crypto).

and (this one is going to be controversial)

• it might actually give you more of a chance of not being spied upon if you're not already interesting target for NSA or some other agency. (Now before you people downvote me to hell with "security though obscurity doesn't work" please allow me to explain :)

As mentioned above, if you're specifically targeted, using toy-cipher won't help you at all and you're practically as unprotected as when you were using unencrypted communications (or even slightly worse, as you might take more risk talking when believing you're protected).

However, beside (or one might argue: before) security of the cipher itself, and security of its implementation (those are two very different things, remember Debian GNU/Linux OpenSSL fiasco for example) there is one other key factor in determining if your communication is going to be de-privatized. And that is, "will anyone try"? And someone will try if any of the 3 things are true:

1. you're interesting target yourself (for example: top politician, financial access / banks, celebrity status, have access to rare information, etc)
2. you're perhaps not interesting yourself, but there is a lot of "you" using same security method. In other words: popularity. Sure, facebook does not contain much of top secret data, but having a way to access ANYONEs facebook account is going to be very good incentive for hackers to try to crack its protections .
3. bad luck (eg. some hacker is bored and just happen to take a pick on your site, transforming it to point 1. above)

Now, sure TLS1.3+AES256-GCM/SHA512 is great thing today. Nobody can spy on you (except NSA and friends, but if you made enemy of them, you're done already). But there is enormous amount of people using it, and thus it becomes enormously profitable to crack it. So it will happen.

Not today, not in 5 years perhaps, but in 50 years random script kiddie would probably be able to decrypt your communications (as you didn't use PFS, doh!) with a simple click.

However, your toy-crypto chat, used in the whole world only by you and your friend to talk about slightly embarrassing fetishes, will likely remain hidden from everyone but legitimate parties for all eternity. Simply because nobody cared enough to give it minimum of effort needed to crack it, and it wasn't popular enough that is was decrypted by default by some PRISM-like mass surveillance.

But the other answers had it correct; best practice for continued secret communications currently is: "a) don't make enemies of various TLA government agencies. b) use secure popular ciphers implemented by competent programmers. c) keep upgrading both ciphers and implementations until at least the day you die" - although it is usually (with perilous results) often shortened just to half of point (b) "use secure popular ciphers".

But I'm wondering if in the long run we wouldn't all be much better if we had billions of obscure and insecure ciphers each used only by few people; as apposed to few supposedly uncrackable ones that everybody uses (but which are secretly flawed - which is known only by 1984-ish goverments and other murky elements.

As (anecdotal) "evidence", every month I help fix dozens of broken wordpress/joomla/etc sites which were "sooo secure" just a few months/years ago. But strangely enough, every now and then I stumble upon properly written (eg. verifies input :) old perl or even shell scripts, using insecure RC2 or even just hackish XOR, which still stand strong after decades. Just because there were different (and not popular) and nobody cared enough to spend time to crack them.

As your biggest problem nowdays is probably not that you will get targeted, but instead that some automated bot will exploit some hole in popular piece of software which you didn't patch quick enough. As there is limited amount of crypto-crackers, and there are most of the time chasing bigger cat than you. So you'd most likely become low-priority ticket and thus fade into oblivion.

I think much of these answers are really derivatives of a much greater and simpler understanding of cryptography.

Besides a one time pad, all cryptography is based on is mathematics and probability.

It's possible someone with no knowledge of anything could break a huge RSA key first cpu cycle, but what is the likelihood? Well we know it, and it's not too good even if an extremely powerful super computer so we trust it. What's the likelihood someone could break the encryption you described? Much much higher.

I think you should negate your question and ask really: Why should I write my own encryption algorithms?

I really don't see a point unless you are developing an algorithm which has some sort of benefit in regards to security or performance.