Why is writing your own encryption discouraged?

Your question, MikeAzo's comment, and your reply practically could not be a better example of Schneier's Law in practice. Schneier stated:

Anyone, from the most clueless amateur to the best cryptographer, can create an algorithm that he himself can't break.

To answer your reply

How can you break it if I send you this "QTCPIGXKUXTGG" ciphertext encrypted by a merely a simple algorithm which you have no idea about how it was encrypted?

Because even though we might not know exactly what your secret algorithm is, the first thing an attacker is going to reach for are common tools to attack substitution ciphers or polyalphabetic ciphers. Given even a few sentences of ciphertext is likely enough to fully recover every plaintext.

The fact that you don't know how to break it is irrelevant. It's trivial to create a cipher that you yourself can't break, but it's another thing entirely to create a cipher that others can't break. And the odds that you are capable of doing it when you're not aware of even the most basic attacks against ciphers hundreds of years old — not to mention modern concepts like indistinguishability under different attack models — puts you at an insurmountable disadvantage compared to ciphers designed by researchers with decades of experience in the field who are building off of modern notions of security and the discarded remains of thousands of failed ciphers that came before.

As an example, even if your cipher is somehow secure against a ciphertext-only attack (it's not), is it secure if I can trick you into encrypting a message for me? What if I can trick you into decrypting a message for me? What if I know part of or all of one of the messages you send? What if you encrypt multiple messages with the same key?

I'll leave you with another Schneier classic, Memo to the Amateur Cipher Designer:

A cryptographer friend tells the story of an amateur who kept bothering him with the cipher he invented. The cryptographer would break the cipher, the amateur would make a change to "fix" it, and the cryptographer would break it again. This exchange went on a few times until the cryptographer became fed up. When the amateur visited him to hear what the cryptographer thought, the cryptographer put three envelopes face down on the table. "In each of these envelopes is an attack against your cipher. Take one and read it. Don't come back until you've discovered the other two attacks." The amateur was never heard from again.

So here's your first envelope. Given a paragraph or two of ciphertext, your cipher will fail to language-based frequency analysis.

Let me know when you've figured out the other two attacks.

Edit: The comment about indistinguishability under different attack models is one reason why most "decipher this message crypto challenges" are completely bunk. They often simply give an attacker some ciphertext, ask them to decipher it, and declare victory when nobody produces the plaintext after some amount of time. Unfortunately that's not how crypto works in the real world; attackers have many more tricks up their sleeve in practice. They can trick computers into encrypting data of their choosing, they can trick computers into decrypting data of their choosing, and they can usually even do these things thousands, millions, or billions of times. Moxie's post shows how even the most terrible, horribly-designed, and obviously insecure ciphers can be effectively impervious when you restrict an attacker to a single ciphertext-only attack, which aren't representative of attackers' capabilities against ciphers as they're actually deployed in practice.

I actually think this is a really good question. The answer is because cryptography is a skill, and like any skill, it takes time to develop. Additionally, you will be pitting your (in)experience in the skill against the skills of those who would seek to break your algorithm.

This is the real reason why: It's not that you just shouldn't do it, period. It's that if you do write your own algorithms, you need to realize that it will take a long time before you create anything that is actually capable of securing your information against a dedicated adversary in the real world.

Cryptography is like sword fighting. You would not sharpen a bamboo stick, swing it around a few times, and then go challenge a pack of thugs to a fight. The reason why should be really obvious. Especially when the pack of thugs in question could be a 3-Letter entity such as DJB or other Nation State level Adversaries.

I say this as someone who is written too many crypto algorithms to count on both hands - I have written plenty of algorithms, but proposed and used none of them. Until you have something that 1. Is faster then AES or Salsa/ChaCha 2. Provably more secure then AES or Salsa/ChaCha, why should you propose or use the algorithm in question?

Lastly, you will likely enjoy studying information theory. You will come to understand that keeping the algorithm secret is not efficient because the algorithm itself has a minimum number of bits required to represent it. These bits (the algorithm) simply become the key. It is well established that it is simply better to concentrate your secrecy into a proper small key with a public algorithm. If your key becomes compromised, you simply change the key. If the algorithm is the key, well, you'd need a whole new algorithm.

tl;dr

I disagree that you should not write your own algorithms, it's that you need to have a good explicit reason for using/proposing your own algorithms. I personally would encourage you to write your own, as it will teach you to understand what does and what does not work and why. Doing so will help you to understand various sorts of mathematics and information theory (basically all of the math that I know I learned because/for cryptography).

Edit

A few comments have been made that I would like to incorporate into the answer.

Using crypto that you wrote yourself is a personal risk, that if you feel if worth taking, nobody can stop you from taking. However, it becomes a problem to others if you start to advertise your algorithms for other people to use. I have seen a few websites not just hosting amateur algorithms, but recommending their use to other people.

This is morally unacceptable, because the people that are likely to use such an algorithm are ignorant of the risks involved (You won't convince me to use your algorithm, for example). Additionally these people are being put at needless risk for the sake of the algorithm author's ego (Why did you want them to use your algorithm specifically, instead of a reviewed one written by a professional with decades of experience?).

It's fine if you're willing to take risks - nobody can stop you - but do not opt other people into the risks you're willing to take. You might not care about what happens to you, but you should care about what happens to those you convinced to count on you for protection.

Also, it was mentioned that before designing your own algorithm(s), you should study currently existing ones and the attacks against them. I consider studying the current body of knowledge to be an essential aspect of self learning. You will get a lot further a lot faster if you build on top of the current level of knowledge as opposed to starting over from the bottom.

However, one of my points is that reading about something is not a replacement for the actual act of doing.

I know a good amount about algorithm design not because I have read every book there is on the subject (though I have read a fair amount), but because I have tried lots of designs and found what does and does not work, and why. "Reading about" is supplementary to "doing" (irrespective of the skill in question). You will never master any skill just by reading about it - It requires practice.

Lastly, in all honesty, symmetric encryption is pretty much a solved problem. This is why we say until you have something: more efficient, provably more secure, or smaller/simpler, there simply is no reason to publishing/using a new design.

It's discouraged because (A) it's statistically unlikely that a beginner comes up with a strong cipher and (B) lacking a working knowledge of cryptanalytic techniques, a beginner won't know that their cipher is vulnerable.

With most engineering problems outside of security, it will be obvious when things aren't working. If your video codec doesn't work you'll have strange green lines all over the display; if your stairs sensor doesn't work, the robot falls down the stairs, etc. But if your cipher is cryptographically vulnerable, you will still be able to encrypt and decrypt messages just fine. The only problem will be that people without the key can also decrypt the messages, and they won't necessarily be eager to inform you of this.

For the same reason that writing your own compiler, framework, or operating system is "discouraged". If you want to do it as a learning experience, for personal satisfaction, or curiosity, go for it. If you want something that actually serves the purpose for which it is intended and you want it sooner than years from now, understand that there are already people who have spend long careers on this and have since made the many, many thousands of mistakes that you will make in the course of producing something much less solid than what they already have.

I would not "discourage" you from writing your own symphony, but if you have not studied music theory, composition and instrumentation, I'm probably not going to be interested in sitting through it, and I doubt if anyone other than your doting mother would be either..

In addition to the great answers on cipher complexity, I think it should be mentioned that the security engineering of the code that implements and utilizes ciphers - things like key exchange, key selection, and the generation of entropy - is very hard as well. Ross Anderson has a terrific book on Security Engineering, but an earlier essay of his (from 1993!) introduces some of the main points specific to cryptography (using old ATM security models as his case study), "Why Cryptosystems Fail" (1st Conf.- Computer and Comm. Security ’93

It is discouraged, for all the good reasons the people already answered you (so I won't repeat).

There is a twist, however (which is the point of this answer) - no matter how weak your cipher is (and it IS weak), it has two advantages going for it:

• it is more secure than plaintext that you had before (even if only by slight margin, as you're not experienced in crypto).

and (this one is going to be controversial)

• it might actually give you more of a chance of not being spied upon if you're not already interesting target for NSA or some other agency. (Now before you people downvote me to hell with "security though obscurity doesn't work" please allow me to explain :)

As mentioned above, if you're specifically targeted, using toy-cipher won't help you at all and you're practically as unprotected as when you were using unencrypted communications (or even slightly worse, as you might take more risk talking when believing you're protected).

However, beside (or one might argue: before) security of the cipher itself, and security of its implementation (those are two very different things, remember Debian GNU/Linux OpenSSL fiasco for example) there is one other key factor in determining if your communication is going to be de-privatized. And that is, "will anyone try"? And someone will try if any of the 3 things are true:

1. you're interesting target yourself (for example: top politician, financial access / banks, celebrity status, have access to rare information, etc)
2. you're perhaps not interesting yourself, but there is a lot of "you" using same security method. In other words: popularity. Sure, facebook does not contain much of top secret data, but having a way to access ANYONEs facebook account is going to be very good incentive for hackers to try to crack its protections .
3. bad luck (eg. some hacker is bored and just happen to take a pick on your site, transforming it to point 1. above)

Now, sure TLS1.3+AES256-GCM/SHA512 is great thing today. Nobody can spy on you (except NSA and friends, but if you made enemy of them, you're done already). But there is enormous amount of people using it, and thus it becomes enormously profitable to crack it. So it will happen.

Not today, not in 5 years perhaps, but in 50 years random script kiddie would probably be able to decrypt your communications (as you didn't use PFS, doh!) with a simple click.

However, your toy-crypto chat, used in the whole world only by you and your friend to talk about slightly embarrassing fetishes, will likely remain hidden from everyone but legitimate parties for all eternity. Simply because nobody cared enough to give it minimum of effort needed to crack it, and it wasn't popular enough that is was decrypted by default by some PRISM-like mass surveillance.

But the other answers had it correct; best practice for continued secret communications currently is: "a) don't make enemies of various TLA government agencies. b) use secure popular ciphers implemented by competent programmers. c) keep upgrading both ciphers and implementations until at least the day you die" - although it is usually (with perilous results) often shortened just to half of point (b) "use secure popular ciphers".

But I'm wondering if in the long run we wouldn't all be much better if we had billions of obscure and insecure ciphers each used only by few people; as apposed to few supposedly uncrackable ones that everybody uses (but which are secretly flawed - which is known only by 1984-ish goverments and other murky elements.

As (anecdotal) "evidence", every month I help fix dozens of broken wordpress/joomla/etc sites which were "sooo secure" just a few months/years ago. But strangely enough, every now and then I stumble upon properly written (eg. verifies input :) old perl or even shell scripts, using insecure RC2 or even just hackish XOR, which still stand strong after decades. Just because there were different (and not popular) and nobody cared enough to spend time to crack them.

As your biggest problem nowdays is probably not that you will get targeted, but instead that some automated bot will exploit some hole in popular piece of software which you didn't patch quick enough. As there is limited amount of crypto-crackers, and there are most of the time chasing bigger cat than you. So you'd most likely become low-priority ticket and thus fade into oblivion.

Obscurity can offer some security, but its usefulness is very limited. Among other things:

1. It is difficulty to quantify how much security it offers. Since human beings make very bad random number generators, it's hard for someone who comes up with a seemingly-original idea to know how likely it is that someone else would think of it.

2. Although it is not difficult to come up with an algorithm which will be secure under the right conditions, maintaining those conditions is difficult. The algorithm and all plaintexts must be forever kept secret from anyone who wouldn't be trusted with the content of all messages that have ever been sent using it. Further, the total volume of ciphertext available must be kept fairly small, but it's difficult to quantify how small.

3. The only way to compartmentalize risk is to use cryptographically-independent algorithms for every different kind of message whose disclosure shouldn't affect any others. Devising truly independent algorithms is difficult for reason #1 above, and most approaches that rely upon security through obscurity fail to properly compartmentalize risk as a result.

Although things like DES, AES, etc. are often described as "algorithms", it would perhaps be more useful to think of them as "algorithm generators". If one has a good source of 256-bit random numbers that have essentially zero probability of matching anything else anywhere in the universe, then "use AES with this number I just generated" can be used as an encryption algorithm which is cryptographically independent from any other algorithm that has ever existed. Consequently, it's much easier to compartmentalize risk using such approaches than by devising one's own encryption schemes.

Crypto is hard, and very easy to get fatally wrong. The chance to win the lottery ten times in a row when you're not even buying tickets is higher then the chance that you will come up with even a halfway passable encryption.

But since crypto is hard, you won't notice that your encryption will take any hobby cryptographer all of half an hour to break. This is not just because algorithms are hard to come up with, there are also modes, key lengths, key exchange and a hundred other things to consider for a full encryption scheme. And even if your scheme is fine on paper, you still need to make an implementation that doesn't add flaws (such as weak IV generation, for example).

Honestly, asking why making your own crypto is discouraged is similar to asking why letting the passengers fly that Airbus 380 is a bad idea. If you actually are a cryptographer, you can play with invention new encryption schemes. If you are not, sit back, enjoy the in-flight entertainment, and let the people who have a couple thousand hours of experience handle the tricky part.

While I agree with the answers already provided, I'd like to add two more points.

The first one is a quote from Bruce Schneier (yeah, him again):

There are two types of encryption: one that will prevent your sister from reading your diary and one that will prevent your government.

So, if confidentiality is actually not a big deal and your only concern is that your texts will not be referenced by Google, you may use your own home-made cryptosystem. I read that some adult newsgroups used ROT13 exactly for this reason: no confidentiality was added, but no one (especially children) would stumble upon the content by chance.

However, if you do so you must assume that someone has broken your system from day 1 and that anyone who wants to break it, actually can. It's all the more true that a cyphertext is similar to a huge billboard reading: "Decrypt me if you can".

If you want real confidentiality, however, then don't use a cryptosystem which has not been tested and approved by experts in the field. See other answers if you're not convinced why.

The second point is related to your comment: "How can you break it if I send you this "QTCPIGXKUXTGG" ciphertext encrypted by a merely a simple algorithm which you have no idea about how it was encrypted?"

This is not compliant with Kerckhoff's principle:

[A cryptosystem's algorithm] should not require secrecy, and it should not be a problem if it falls into enemy hands

Or, more clearly, if your security relies on the fact that your enemy doesn't know the algorithm you used, then your security is crap. The attacker is perfectly able to find a fatal flaw even without knowing your algorithm.

The obvious answer is that nothing is wrong with making a toy cipher to exchange messages with a friend -- just like nothing is wrong with using gets() in a hangman game that you and a friend are going to play. Bruce Schneier isn't going to break down your door and confiscate your computer. What you do with your friends is your own business.

Important qualifications:

1. It would be extremely unwise to think of something like multiplying the value of a letter by 2 and adding 13 as anything more than a toy.

2. What would actually be wrong to the point of professional malpractice would be if you use ad-hoc encryption on a product that you foist on unsuspecting clients (just like it would be wrong if you use gets() in any production code).

3. While there is nothing wrong per se with playing around with toy ciphers or writing private code that is vulnerable to buffer-overflows, doing such things can easily become a habit and habits are hard to break. If you get use to using gets() (a notorious source of security holes but easier to use than secure alternatives) for your own toy projects then you will be tempted to cut corners and use it in production code. Similarly, if you get in the habit of thinking that it is easy to disguise messages then you might be tempted to write your own crypto in production code rather than doing the tedious work of learning how to properly use an industrial-strength crypto API.

To sum up -- nothing is wrong per se with the scenario that you describe (toy crypto between friends) but it nevertheless raises warning flags since it can drift into something which is wrong.

gets API (removed from the latest C specifications and deprecated in C++):

Reads characters from the standard input (stdin) and stores them as a C string into str until a newline character or the end-of-file is reached ... and does not allow to specify a maximum size for str (which can lead to buffer overflows)

Using gets is almost guaranteed to make the system vulnerable to attacks.