How to handle carriage return input in sed?

[geo_c]

I have this script full of a bashrc functions which is currently working well,

But there is a particular function I'm working on, inserting a line of text in a text file from user input using a read line. The insert function works as expected, except if the input is a carriage return in other words ONLY the [enter] key.

My current insert lines are:

Code: Select all

read -p 'Type line number to insert: ' LINENUM
read -p 'Enter text to insert: ' INSERTTEXT
sed -i "${LINENUM}i ${INSERTTEXT}" ${TASKS_DIR}"${*}"

${TASKS_DIR}"${*}" being the file to modify in place. And as I say, it works UNLESS only the [enter] key is the input from the read -p INSERTTEXT which then returns the following error:

sed: -e expression #1, char 3: expected \ after `a', `c' or `i'

Using the expected backslash does result in a successful new line insert from the [enter] key input, BUT the backslash adds a space to all other input.

Is there a way to write the sed expression to either undo the space added to normal character input, or a better way to handle the carriage return?

Or is there a better command to use in this instance?

[geo_c]

Well I thought I had it solved but I realized my print line was filtering the actual sed output.

This line handles the carriage return but adds a slash and space to the beginning of all inserted lines:

Code: Select all

sed -i "${LINENUM}i\/ ${INSERTTEXT}" ${TASKS_DIR}"${*}"

Any other ideas and methods to insert lines of text involving character returns or special characters would be greatly appreciated.

[MochiMoppel]

adds a slash and space to the beginning of all inserted lines:

...because that's what you instructed sed to do.

Try

Code: Select all

sed -i "${LINENUM}i${INSERTTEXT}" ${TASKS_DIR}"${*}"

[Burunduk]

This line handles the carriage return but adds a slash and space to the beginning of all inserted lines:

Code: Select all

sed -i "${LINENUM}i\/ ${INSERTTEXT}" ${TASKS_DIR}"${*}"

A backslash is expected, not a slash. A literal newline is expected too:

Code: Select all

sed -i "${LINENUM}i\\
${INSERTTEXT}" ${TASKS_DIR}"${*}"

[geo_c]

A backslash is expected, not a slash. A literal newline is expected too:

Code: Select all

sed -i "${LINENUM}i\\
${INSERTTEXT}" ${TASKS_DIR}"${*}"

This one works kind of the same way as only using one backslash. One slash prepends the inserted text with one space, and two backslashes prepends the inserted text with two spaces.

Try

Code: Select all

sed -i "${LINENUM}i${INSERTTEXT}" ${TASKS_DIR}"${*}"

And this one throws up the same message when using the [enter] key on the read line.

So the use of the backslash or backslashes successfully inserts a blank line in the text when [enter] is pressed, but they prepend all other text with a space for each slash.

But I'll have to try again tomorrow, as I'm seeing double now.

[williwaw]

or a better way to handle the carriage return?

read -d

-d delim
The first character of delim is used to terminate the input line, rather than newline.

maybe something here?

[MochiMoppel]

@geo_c I think I now understand your problem. Your original code is fine, however when the user, after the 'Enter text to insert' prompt, just pushed the Enter key, without entering any other text, the INSERTTEXT variable remains empty, And when this variable remains empty, then sed just sees a <num>i command followed by a space and the file paths. Sed *assumes* that this space is the text you want to insert and claims that it should be escaped with a backslash. Clearly not what you want.

What should actually happen when no INSERTTEXT is defined? Should the script exit or do you want to insert an empty line?
If the latter then you should test the input. If INSERTTEXT is empty, assign a newline character to it:

Code: Select all

read -p 'Type line number to insert: ' LINENUM
read -p 'Enter text to insert: ' INSERTTEXT
[[ $INSERTTEXT = '' ]] && INSERTTEXT=$'\n'
sed -i "${LINENUM}i ${INSERTTEXT}" ${TASKS_DIR}"${*}"

Note that you will have similar problems when the LINENUM remains empty. In this case INSERTTEXT will be inserted before every line.

[geo_c]

Should the script exit or do you want to insert an empty line?

Yes, exactly! I want to insert an empy line. And your suggestion works perfectly!

And previous to my original posting, I had actually tried applying a test to INSERTTEXT similar to what to you have above, but I didn't know what syntax to use and instead was testing for a 'carriage return' with \r, but in reality INSERTTEXT is empty, and I wasn't comprehending the difference.

Note that you will have similar problems when the LINENUM remains empty. In this case INSERTTEXT will be inserted before every line.

Actually I have experienced the insert on every line of the file result. So when I accidentally hit enter instead of typing a linenumber, then I would ctrl-c and break off the routine to avoid that.

My corrected script looks like this and seems to be working smoothly now:

Code: Select all

read -p "Type line number to insert ('enter' or 'q' to quit'): " LINENUM
    [[ $LINENUM = 'q' ]] && return
    [[ $LINENUM = '' ]] && return
   read -p 'Enter text to insert: ' INSERTTEXT
   [[ $INSERTTEXT = '' ]] && INSERTTEXT=$'\n'
   echo ""
   echo "    Updated:"
   sed -i "${LINENUM}i ${INSERTTEXT}" ${TASKS_DIR}"${*}"

output looks like this:

[rcrsn51]

@MochiMoppel : Ths works for me too but I don't understand the syntax. The sed "i" command inserts a string into the file and ends it with a newline. So shouldn't inserting a newline give you TWO blank lines?

[MochiMoppel]

The sed "i" command inserts a string into the file and ends it with a newline. So shouldn't inserting a newline give you TWO blank lines?

No, it doesn't insert the newline character passed to it with the INSERTTEXT variable. It discards it.

Here a short demo that might help to understand how this works:

Code: Select all

CONTENT='apple
banana
durian'

INSERTTEXT="citron

"
echo "$CONTENT" | sed "3i $INSERTTEXT"

No matter if you add none or many newline characters after 'citron', the result will always be

Code: Select all

apple
banana
citron
durian

This means that sed deletes the first newline it encounters and everthing that follows, and then adds its own newline character to INSERTTEXT. So if INSERTTEXT contains nothing but a newline character, it is deleted, but since sed automatically adds a newline you end up with no text in the third line.

You can add newlines after 'citron', in fact you can add multiline text, but in this case you need to add newline characters in INSERTTEXT like this: INSERTTEXT="citron\n". This also means that you can insert more than one blank line:
1 blank line: INSERTTEXT=$'\n'
2 blank lines: INSERTTEXT='\\n'
3 blank lines: INSERTTEXT='\\n\n'
4 blank lines: INSERTTEXT='\\n\n\n'
etc.

BTW: Funny things happen when you start INSERTTEXT with a newline character :

Code: Select all

INSERTTEXT="
citron"

results in

Code: Select all

itron
itron

itron

[rcrsn51]

But if you just use INSERTTEXT="\n" it fails. Strange.

[geo_c]

BTW: Funny things happen when you start INSERTTEXT with a newline character :

Code: Select all

INSERTTEXT="
citron"

results in

Code: Select all

itron
itron

itron

I was getting some of that kind of thing with my experiments, but I have no idea what combination of backslashes and spaces caused it, becuase I was trying different things rapid fire.

The overarching bashrc file I've been working on is a simple set of todol list functions.

todo.sh.FAKE.gz

(11.66 KiB) Downloaded 6 times

They all seem to be working now. There's only a couple things I'd like to improve. One is the need to use 'return' on the LINENUMBER read if the input is 'enter.' It would be better to throw another test in there and give the user a second chance to enter a number, though the 'enter' to quit is a pretty good option on a second thought decision not to insert a line.

The second thing that is working in this script, but seems too much like a workaround is the todoedit function, (called todoe.) The sed command seems to need an -e option, and I couldn't get it to insert in-place with both the -e and -i option. So I tried piping to cat, but that resulted in a blank file. From what I could tell sed couldn't use the file as an input and pipe to cat simultaneously or something like that.

At any rate for that problem I used a temp file as an intermediary write, and it works like that.
Here is that section of code, and I'm attaching the full bashrc file
also, I don't know how to get this function to replace text with a blank line, hence the instructions to use todoinsert (todoi)

Code: Select all

function todoe {
 if ! [[ -z "$1" ]] && [ -e ${TASKS_DIR}"${*}" ]
 then
   echo ""
   echo "    Current:"
   cat ${TASKS_DIR}"${*}" | nl -ba -s '|  ' | sed "s/.\///"
   echo ""
   read -p "Type line number to replace ('enter' or 'q' to quit'): " LINENUM
    [[ $LINENUM = 'q' ]] && return
    [[ $LINENUM = '' ]] && return
   read -p "Type text to insert ('enter' to quit -- Use todoi for blank lines):" EDITTEXT
   [[ $EDITTEXT = '' ]] && return
   echo ""
   echo "    Updated:"
   sed -e "${LINENUM}s/.*/${EDITTEXT}/" ${TASKS_DIR}"${*}" > ${TODOAPP_DIR}.TEMPfile
   cat ${TODOAPP_DIR}.TEMPfile > ${TASKS_DIR}"${*}"
   cat ${TASKS_DIR}"${*}" | nl -ba -s '|  ' | sed "s/.\///"
   echo ""
 else
   printf "${TODO_LIST_LABEL}"
   find ${TASKS_DIR} -type f -execdir echo '{}' ';' | nl -s '|  ' | sed "s/.\///"
   printf "${TODO_LIST_END}" 
 fi
}

[Burunduk]

Code: Select all

CONTENT='line 1
line 2
line 3'

INSERTTEXT='extra line(s)\

'
echo "$CONTENT" | sed "3i $INSERTTEXT"

results in

Code: Select all

line 1
line 2
extra line(s)

line 3

---

citron means c itron that is change the content of every line to "itron".

[geo_c]

I just updated that todo.sh script in my previous post, because it was missing a couple of the tests on LINENUM. Not testing LINENUM only gives a bad result if the input is 'enter.' Otherwise the funcitions missing the test still worked with all other input.

Should be all correct. Though I'll keep cleaning it up, and probably adding a few more funcitons.

[Doggy]

Just wanted to share a handy tip for handling carriage return input in sed. If you're encountering carriage return characters (\r) in your input and want to remove them, you can use the following sed command:
sed 's/\r//g' input_file > output_file
This command will effectively strip out all carriage returns from the input file and write the result to the output file. Handy for cleaning up text files! Hope this helps someone out there dealing with similar issues.