nth loop

Complex operations in sed

3 min read

Commands in sed can be constructed in sequences and nested in blocks allowing for more complex operation that the built-in commands.

Sequence of commands

A single sed execution can run any number of sed commands on the input text. The commands in the sequence are separated by semi-colons

sed 'command1;command2;command3'

For instance, consider the following sample text:

1 Lorem ipsum dolor sit amet consectetur
2 adipiscing elit. Quisque faucibus ex
3 sapien vitae pellentesque sem placerat.

The following command contains a sequence of 3 sed commands that will delete line 2, substitute some pattern and append a new line after another pattern:

$ sed '2d;s/Lorem/Verbum primum/;/placerat/a Haec est sententia addita.' lorem.txt
 
1 Verbum primum ipsum dolor sit amet consectetur
3 sapien vitae pellentesque sem placerat.
  Haec est sententia addita.

Sequences of commands are independent, each one applies to the complete text input.

Operations within a match

Commands in sed can be grouped in blocks that apply to same the pattern space. Those sequence of commands are separated by semi-colons and encapsulated in curly braces {}.

sed '/pattern/{command1;command2}'

Command blocks in sed allow to perform more complex operations in the matching text than just using the simple built-in commands. We can use different patterns for selecting lines of text and for making substitutions. For instance, consider the following sample text:

1 Lorem ipsum dolor sit amet consectetur
2 adipiscing elit. Quisque faucibus ex
3 sapien vitae pellentesque sem placerat.

The following custom sed command replaces all spaces with underscores (y command) on lines matching the word Quisque:

$ sed '/Quisque/{y/ /_/}' lorem.txt
 
1 Lorem ipsum dolor sit amet consectetur
2 adipiscing_elit._Quisque_faucibus_ex
3 sapien vitae pellentesque sem placerat.

This can be easily expanded with extra commands. The following command adds a substitution of the word elit to rosa:

$ sed '/Quisque/{y/ /_/;s/elit/rosa/}' lorem.txt
 
1 Lorem ipsum dolor sit amet consectetur
2 adipiscing_rosa._Quisque_faucibus_ex
3 sapien vitae pellentesque sem placerat.

Using the insert i and append a commands right in the command-line is a bit more cumbersome as those require using new lines to separate the contents of insert/append from the other commands.

$ sed '/Quisque/{y/ /_/;s/elit/rosa/;i\
  Haec est sententia addita.
  }' lorem.txt
  
1 Lorem ipsum dolor sit amet consectetur
  Haec est sententia addita.
2 adipiscing_rosa._Quisque_faucibus_ex
3 sapien vitae pellentesque sem placerat.

Script files

These operations in sed can become too complicated rather quickly, being impractical to run directly on the command-line. At this point, they can be put in their own file so-called a sed script and be executed from there.

The command in the last example in the previous section that reads

sed '/Quisque/{y/ /_/;s/elit/rosa/;i\
Haec est sententia addita.
}' lorem.txt

Can be written in a sed script as

/Quisque/{
  y/ /_/
  s/elit/rosa/
  i\
Haec est sententia addita
}

Then executed with the command

$ sed -f script.sed lorem.txt
 
1 Lorem ipsum dolor sit amet consectetur
  Haec est sententia addita
2 adipiscing_rosa._Quisque_faucibus_ex
3 sapien vitae pellentesque sem placerat.

Resulting in the same output as the original command.

Example: controlling the installation of JAX in EasyBuild

The installation of JAX v0.3.25 in EasyBuild ensures the use of local sources for TensorFlow by replacing the repository definition in the Bazel build files with a local one.

This is achieved with jaxlib_local-tensorflow-repo.sed, a sed script applied to the WORSKPACE file to match the repository definition of TensorFlow, comment out its contents and inject the local definition instead.

Graph View

Backlinks