Unix Sed Tutorial: Advanced Sed Substitution Examples

By Linux Guru 09.12.2021


Sed Substitution Delimiter

As we discussed in our previous post, we can use the different delimiters such as @ % | ; : in sed substitute command. Let us first create path.txt file that will be used in all the Beispielen mentioned below.

cat path.txt

/usr/kbos/bin:/usr/local/bin:/usr/jbin:/usr/bin:/usr/sas/bin /usr/local/sbin:/sbin:/bin/:/usr/sbin:/usr/bin:/opt/omni/bin: /opt/omni/lbin:/opt/omni/sbin:/root/bin


Beispiel 1 sed @ delimiter: Substitute /opt/omni/lbin to /opt/tools/bin

When you substitute a path name which has ‘/’, you can use @ as a delimiter instead of ‘/’. In the sed Beispiel below, in the last line of the input file, /opt/omni/lbin was changed to /opt/tools/bin.

sed '[email protected]/opt/omni/[email protected]/opt/tools/[email protected]' path.txt

/usr/kbos/bin:/usr/local/bin:/usr/jbin/:/usr/bin:/usr/sas/bin /usr/local/sbin:/sbin:/bin/:/usr/sbin:/usr/bin:/opt/omni/bin: /opt/tools/bin:/opt/omni/sbin:/root/bin

Beispiel 2 sed / delimiter: Substitute /opt/omni/lbin to /opt/tools/bin

When you should use ‘/’ in path name related substitution, you have to escape ‘/’ in the substitution data as shown below. In this sed Beispiel, the delimiter ‘/’ was escaped in the REGEXP and REPLACEMENT part.

sed 's/\\/opt\\/omni\\/lbin/\\/opt\\/tools\\/bin/g' path.txt

/usr/kbos/bin:/usr/local/bin:/usr/jbin/:/usr/bin:/usr/sas/bin /usr/local/sbin:/sbin:/bin/:/usr/sbin:/usr/bin:/opt/omni/bin: /opt/tools/bin:/opt/omni/sbin:/root/bin


Sed ‘&’ Get Matched String

The precise part of an input line on which the Regular Expression matches is represented by &, which can then be used in the replacement part.

Beispiel 1 sed & Usage: Substitute /usr/bin/ to /usr/bin/local

sed '[email protected]/usr/[email protected]&/[email protected]' path.txt

/usr/kbos/bin:/usr/local/bin:/usr/jbin/:/usr/bin/local:/usr/sas/bin /usr/local/sbin:/sbin:/bin/:/usr/sbin:/usr/bin/local:/opt/omni/bin: /opt/omni/lbin:/opt/omni/sbin:/root/bin

In the above Beispiel ‘&’ in the replacement part will replace with /usr/bin which is matched pattern and add it with /local. So in the output all the occurrance of /usr/bin will be replaced with /usr/bin/local

Beispiel 2 sed & Usage: Match the whole line

& replaces whatever matches with the given REGEXP.

sed '[email protected]^.\*[email protected]<<<&>>>@g' path.txt

<<</usr/kbos/bin:/usr/local/bin:/usr/jbin/:/usr/bin:/usr/sas/bin>>> <<</usr/local/sbin:/sbin:/bin/:/usr/sbin:/usr/bin:/opt/omni/bin:>>> <<</opt/omni/lbin:/opt/omni/sbin:/root/bin>>>

In the above Beispiel regexp has “^.*$” which matches the whole line. Replacement part «<&»> writes the whole line with «< and »> in the beginning and end of the line respectively.

Grouping and Back-references in Sed

Grouping can be used in sed like normal regular expression. A group is opened with “\(” and closed with “\)”.Grouping can be used in combination with back-referencing.

Back-reference is the re-use of a part of a Regular Expression selected by grouping. Back-references in sed can be used in both a Regular Expression and in the replacement part of the substitute command.

Beispiel 1: Get only the first path in each line

sed 's/\\(\\/\[^:\]\*\\).\*/\\1/g' path.txt

/usr/kbos/bin /usr/local/sbin /opt/omni/lbin 


In the above Beispiel, \(\/[^:]*\) matches the path available before first : comes. \1 replaces the first matched group.


Beispiel 2: Multigrouping

In the file path.txt change the order of field in the last line of the file.

sed '[email protected]\\(\[^:\]\*\\):\\(\[^:\]\*\\):\\(\[^:\]\*\\)@\\3:\\2:\\[email protected]' path.txt

/usr/kbos/bin:/usr/local/bin:/usr/jbin:/usr/bin:/usr/sas/bin /usr/local/sbin:/sbin:/bin:/usr/sbin:/usr/bin:/opt/omni/bin:

In the above command $ specifies substitution to happen only for the last line.Output shows that the order of the path values in the last line has been reversed.

Beispiel 3: Get the list of usernames in /etc/passwd file

This sed Beispiel displays only the first field from the /etc/passwd file.

sed 's/\\(\[^:\]\*\\).\*/\\1/' /etc/passwd

Beispiel 4: Parenthesize first character of each word

This sed Beispiel prints the first character of every word in paranthesis.

echo "Welcome To The Geek Stuff" | sed 's/\\(\\b\[A-Z\]\\)/\\(\\1\\)/g'
(W)elcome (T)o (T)he (G)eek (S)tuff

Beispiel 5: Commify the simple number.

Let us create file called numbers which has list of numbers. The below sed command Beispiel is used to commify the numbers till thousands.

cat  numbers

sed 's/\\(^\\|\[^0-9.\]\\)\\(\[0-9\]\\+\\)\\(\[0-9\]\\{3\\}\\)/\\1\\2,\\3/g' numbers