Feeds:
Posts
Comments

Archive for the ‘sed’ Category

there have been some situations where i needed a custom grid on a 2-dimensional plot in gnuplot instead of a regularly placed grid. in order for me to set up a customized grid i have to first set up a customised xtics and/or ytics. gnuplot does provide a way of providing user preferred tics by manually typing them in at the gnuplot command prompt:

gnuplot> set xtics (1,2,3,4)

however that might get too tiresome for your digits. naturally one would want to employ the powerful text processors and stream editors that GNU/Linux provides. this is how i went about it:

assume that the data file(data.out, say) is as follows:

# some comments and blank lines

#      #x1	#x2 	  #x3

1      1.1	3.9	  4.5
2      2.7	6.1	  5.6
3      3.9	2.3	  3.4
       
4      3.5	4.3	  34.0
5      12.1	3.4	  15.9

i want to set the xtics to the values in column 2 and ytics to the values in column 3 say. first i conjure up the following shell script (ttf.sh, say):

#!/usr/bin/env bash

# set tic levels from file at the gnuplot prompt
# examples:
# gnuplot> `./tff.sh data.dat 1 xtics`
# gnuplot> `./tff.sh data.dat 2 ytics`

# dataFile from which tic levels will be read
dataFile=$1

# which column to use
column=$2

# xtics OR ytics 
whichTics=$3

sed -e '/^#/d' $dataFile | \
    sed -e '/^$/d' | \
    cut -f $column | \
    tr '\n' ',' | \
    sed -e "s/^/set $whichTics (/" | \
    sed -e 's/,$/)\n/'

and then in gnuplot carry out the following sequence of commands:

gnuplot> `./tff.sh data.out 2 xtics`
gnuplot> `./tff.sh data.out 3 ytics`
gnuplot> plot 'data.out' u 1:2 w p pt 13 ps 2

to produce the following plot:

it should be noted that some of the grid lines are hidden due to alignment with borders.

the shell script needs 3 inputs

  1. datafile name
  2. which column to pull from the datafile
  3. which tics to set the values to

then the whole command is sandwiched between couple of backtics which bring in the command substitution magic. one could very well do the following in bash:

$ ./tff.sh data.out 2 xtics > tmp

and then in gnuplot

gnuplot> load 'tmp'

but i prefer the backtics.

now the explanation of the wonderful sed magic. first we do a little bit of file cleaning/processing like removing comments and blank lines. these operations may not necessarily apply to your case or might need additional operations which is not that difficult a job. anyways on to the explanations now…

sed -e '/^#/d' $dataFile

simply deletes all lines starting with the # symbol. of course if the comment character is different one should change the code appropriately. ^ stands for the beginning of all lines and ^# means all lines starting with a # symbol. the `d’ command instructs sed to delete those lines. we then pipe (|) the output of this command to the next command

 
sed -e '/^$/d' 

which simply deletes all blank lines. since ^ represents the beginning of all lines and $ represents the end of all lines, ^$ then represents those lines with nothing in between ^ and $ i.e. blank lines. as always the `d’ command instructs sed to delete this line.

cut -f $column

next we cut out the required column from the datafile using the cut command. we assume here that the field separater is a TAB character. if some other field separater is being used that can be specified with the -d switch. do a `man cut` to find out more. now comes the all important part of the entire exercise, what we want is to transform the vertical stack of values:

1
2
3

to a horizontal comma separated list of values:

1,2,3,

notice the comma after 3.
we achieve this by using the tr command which we instruct to transform all the newline characters (`\n’) to the comma character (`,’). we do pick up an unwanted comma at the very end of the list which we get rid of and transformed to a right bracket ) like so:

sed -e 's/,$/)\n/'

this time we are using the subsitute command `s’ and asking sed to replace the last comma by a ). as you already know $ represents the end of the line so (,$) represents the last comma we substitute those with a right bracket ) and a newline character `\n’.
of course we had to prepend the “set xtics (” to the beginning of the list and that bit is done by the following:

sed -e "s/^/set $whichTics (/"

notice the double quotes instead of the single quotes. we use double quotes so that $whichTics is automatically expanded to the user supplied tics name.

Read Full Post »

i want to transform this:

# blah
# gen = 0
1 1 1
1 1 1
# gen = 1
2 2 2
2 2 2
# gen = 2
3 3 3 3
3 3
.
.
.
# gen = 10
10 10 10

in to this :

# blah

# gen = 0
1 1 1
1 1 1

# gen = 1
2 2 2
2 2 2

# gen = 2
3 3 3 3
3 3
.
.
.

# gen = 10
10 10 10

that is i want to introduce one or more new line before all lines starting with “# gen = “.  this is how one can do it using sed.

sed 's/^# gen.*/\n\n&/' <a.out  >b.out
  • ‘s/ bit  :: is for the substitute command
  • ^# gen :: is the regular expression which matches all lines starting with a # followed by gen
  • .* ::  matches everything else on the lines starting with a # followed by gen
  • \n\n :: will introduce 2 blank lines
  • & :: simply stands for the matched part that is, it stands for everything matched by ^# gen.*
  • a.out :: input file the file i want to transform
  • b.out :: is the required file..the file i want to have

i wish i could do the same thing sitting in emacs.

Read Full Post »

task at hand

you have two files :: file1.txt and file2.txt

you want to insert the contents of file1.txt into file2.txt starting at line number 2

contents of file1.txt before the process

because i m stupid

contents of file2.txt before the process

why do you want to do a phd ?

ok then, may god rest your spirit in peace.

contents of file2.txt after the process

why do you want to do a phd ?

because i m stupid

ok then, may god rest your spirit in peace.

this is how you do it

sed ‘2r file1.txt’ < file2.txt > tempFile.txt

mv tempFile.txt file2.txt

the r stands for the read FILENAME command and is a GNU extension i believe. so it may not be available on all platforms. to find out more try
info sed
and in particular see the section 3.6 Less Frequently-Used Commands

Read Full Post »