How can I replace a newline (\n) using the sed command?
I unsuccessfully tried:
sed 's#\n# #g' file
sed 's#^$# #g' file
How do I fix it?
|
How can I replace a newline (\n) using the sed command? I unsuccessfully tried:
How do I fix it? |
||||
|
Use this solution with GNU
This will read the whole file in a loop, then replaces the newline(s) with a space. Explanation:
Here is cross-platform compatible syntax which works with BSD
|
|||||||||||||||||||||
|
|
Use
or remove the newline characters entirely:
or if you have the GNU version (with it's long options)
|
|||||||||||||||||||||
|
|
Fast answer:
sed will loop through step 1 to 3 until it reach the last line, getting all lines fit in the pattern space where sed will substitute all \n characters Alternatives: All alternatives, unlike sed will not need to reach the last line to begin the process with bash, slow
with perl, sed-like speed
with tr, faster than sed, can replace by one character only
with paste, tr-like speed, can replace by one character only
with awk, tr-like speed
Other alternative like "echo $(< file)" is slow, works only on small files and needs to process the whole file to begin the process. Long answer from the sed FAQ 5.10: 5.10. Why can't I match or delete a newline using the \n escape The \n will never match the newline at the end-of-line because the Sed works like this: sed reads one line at a time, chops off the
will NEVER work, because the trailing newline is removed before
Since versions of sed other than GNU sed have limits to the size of To match a block of two or more lines, there are 3 basic choices: Choices (1) and (2) will put an \n into the pattern space, where it Choice (3) will not put an \n into the pattern space, but it does
in addition to the traditional '/from here/,/to there/{...}' range |
|||||||||||||||||||||
|
|
A shorter awk alternative:
ExplanationAn awk program is built up of rules which consist of conditional code-blocks, i.e.:
If the code-block is omitted, the default is used: When Now, when printing a record, |
|||||||||||||||||
|
|
The Perl version works the way you expected.
As pointed out in the comments, it's worth noting that this edits in place. |
|||||||||||||||||||||
|
|
gnu sed has an option
|
|||||||||||||||||
|
|
Who needs
|
|||||||||||||||||||||
|
|
In order to replace all newlines with spaces using awk, without reading the whole file into memory:
If you want a final newline:
You can use a character other than space:
|
|||||
|
|
Three things.
|
|||||||||||||||||||||
|
is the command. Simple and easy to use. |
|||||
|
|
The answer with the :a label ... How can I replace a newline (\n) using sed? ... does not work in freebsd 7.2 on the command line: ( echo foo ; echo bar ) | sed ':a;N;$!ba;s/\n/ /g' sed: 1: ":a;N;$!ba;s/\n/ /g": unused label 'a;N;$!ba;s/\n/ /g' foo bar But does if you put the sed script in a file or use -e to "build" the sed script... > (echo foo; echo bar) | sed -e :a -e N -e '$!ba' -e 's/\n/ /g' foo bar or ...
Maybe the sed in OS X is similar. |
|||||
|
|
I'm not an expert, but I guess in
From
I've used this to search (multiple) badly formatted log files, in which the search string may be found on an "orphaned" next line. |
|||||||||||||
|
|
|
|||||
|
|
In response to the "tr" solution above, on Windows (probably using the Gnuwin32 version of tr), the proposed solution:
was not working for me, it'd either error or actually replace the \n w/ '' for some reason. Using another feature of tr, the "delete" option -d did work though:
or '\r\n' instead of '\n' |
|||||
|
Easy-to-understand SolutionI had this problem. The kicker was that I needed the solution to work on BSD's (Mac OS X) and GNU's (Linux and Cygwin)
Output:
(has trailing newline) It works on Linux, OS X, and BSD - even without UTF-8 support or with a crappy terminal.
|
|||||
|
|
I used a hybrid approach to get around the newline thing by using tr to replace newlines with tabs, then replacing tabs with whatever I want. In this case, "
|
||||
|
|
|
You could use However, it would have problems if your input has any case of an |
|||||
|
|
In some situations maybe you can change
The power of shell scripting is that if you do not know how to do it in one way you can do it in another way. And many times you have more things to take into account than make a complex solution on a simple problem. Regarding the thing that gawk is slow... and reads the file into memory, I do not know this, but to me gawk seems to work with one line at the time and is very very fast (not that fast as some of the others, but the time to write and test also counts). I process MB and even GB of data, and the only limit I found is line size. |
|||||
|
|
Using Awk:
|
|||||||||||||
|
|
A solution I particularly like is to append all the file in the hold space and replace all newlines at the end of file:
However, someone said me the hold space can be finite in some sed implementations. |
|||||
|
Replace newlines with any string, and replace the last newline tooThe pure
Result:
|
|||||||||||||
|
|
It is sed that introduces the new-lines after "normal" substitution. First, it trims the new-line char, then it processes according to your instructions, then it introduces a new-line. Using sed you can replace "the end" of a line (not the new-line char) after being trimmed, with a string of your choice, for each input line; but, sed will output different lines. For example, suppose you wanted to replace the "end of line" with "===" (more general than a replacing with a single space):
To replace the new-line char with the string, you can, inefficiently though, use tr , as pointed before, to replace the newline-chars with a "special char" and then use sed to replace that special char with the string you want. For example:
|
||||
|
|
Bullet-proof solution. Binary-data-safe and POSIX-compliant, but slow.POSIX sed requires input according to the POSIX text file and POSIX line definitions, so NULL-bytes and too long lines are not allowed and each line must end with a newline (including the last line). This makes it hard to use sed for processing arbitrary input data. The following solution avoids sed and instead converts the input bytes to octal codes and then to bytes again, but intercepts octal code 012 (newline) and outputs the replacement string in place of it. As far as I can tell the solution is POSIX-compliant, so it should work on a wide variety of platforms.
POSIX reference documentation: sh, shell command language, od, tr, grep, read, [, printf. Both Tested on Ubuntu (bash, dash, and busybox), FreeBSD, and OpenBSD. |
|||
|
|
|
@OP, if you want to replace newlines in a file, you can just use dos2unix (or unix2dox)
|
|||||||||
|
|
On Mac OS X (using FreeBSD sed):
|
|||
|
|
|
You can use this method also
Explanation
Flow: |
|||||
|
|
To remove empty lines:
|
|||||
|
|
Another
One would hope y would run faster than s, (perhaps at |
|||||
|
This does not work for huge files (buffer limit), but it is very efficient if there is enough memory to hold the file.
(Correction Another version that change new line while reading (more cpu, less memory)
|
|||||||||||||
|
|
You can also use the Standard Text Editor:
Note: this saves the result back to As with |
|||
|
|
Thank you for your interest in this question.
Because it has attracted low-quality or spam answers that had to be removed, posting an answer now requires 10 reputation on this site (the association bonus does not count).
Would you like to answer one of these unanswered questions instead?
trinstead ofsedas suggested in @dmckee's answer below. It's much simpler and the "right tool for the job." – T. Brian Jones Jan 4 '13 at 6:55tris only the right tool for the job if replace a single character for a single character, while the example above shows replace newline with a space.. So in the above example, tr could work.. But would be limiting later on. – Mayhem Dec 31 '15 at 2:47sed 's/$/ NewDelim/' | tr '\n' ' '. Usesedto append the new delimiter to the end of each line, then remove newlines withtr. Less cryptic than thesedonly way, IMO. – cp.engr Jan 19 '16 at 19:08trin the right tool for the job because the questioner wanted to replace each newline with a space, as shown in his example. The replacement of newlines is uniquely arcane forsedbut easily done bytr. This is a common question. Performing regex replacements is not done bytrbut bysed, which would be the right tool... for a different question. – Mike S Dec 28 '16 at 15:01