AWK One Liners

AWK One Liners

I have here some of the cool AWK One Liners which I have used and found very helpful.

AWK – Print Line Numbers

– Print the file content with line number against each line of the file.

[[email protected] ~]# awk ‘{print FNR “\t” $0}’ some_file.txt

Output:

1	Line1 value1
2	line 2 value2
3	
4	line4 value4
5	
6	line6
7	
8	line8 value8 value8-1
9	

– Print the file content with line number for an individual file, in case of bunch of files together. In this case I have two files with the same content (some_file.txt & some_file_copy.txt).

[[email protected] ~]# awk ‘{print FNR “\t” $0}’ some_file*

Output:

1	Line1 value1
2	line 2 value2
3	
4	line4 value4
5	
6	line6
7	
8	line8 value8 value8-1
9	
1	Line1 value1
2	line 2 value2
3	
4	line4 value4
5	
6	line6
7	
8	line8 value8 value8-1
9	

– Number each line for file only if the line is not blank.

awk ‘NF{$0=++a ” :” $0};1’

OR

awk ‘{print (NF? ++a ” :” :””) $0}’

Sample Output:

1 :Line1 value1
2 :line 2 value2

3 :line4 value4

4 :line6

5 :line8 value8 value8-1

– Count the lines in a file. This is the samething as wc -l.

awk ‘END{print NR}’

– Print total number of fields in all the lines for a file. This will consider the fields which has a string in them.

awk ‘{ total = total + NF }; END {print total}’ FILE_NAME

– Print the total number of lines which contain a STRING (REGEX).

awk ‘/STRING/{n++}; END {print n+0}’ FILE_NAME

– Print number of fields in each line.

awk ‘{print NF}’ FILE_NAME

– Print number of fields in each line, followed by the line.

awk ‘{print NF ” – ” $0}’ FILE_NAME

AWK One Liners

– Print the last field for each line.

awk ‘{print $NF}’ FILE_NAME

– Print the last field of the last line.

awk ‘{ field = $NF }; END{ print field }’ FILE_NAME

– Print the second last field of the file with same number of fields.

awk ‘{ print ( $ (NF-1) ) }’ FILE_NAME

– Print the second last field of the file with uneven number of fields.

awk ‘NF && NF-1 { print ( $ (NF-1) ) }’ FILE_NAME

– Print the line with more than N number of fields.

awk ‘NF > N’ FILE_NAME

Where, N stands for the number of field. So for instance if we have to print the line which has more than 3 fields, the command will be.

awk ‘NF > 3’ FILE_NAME

AWK One Liners

– Remove leading whitespaces (spaces, tabs) for each line in a file.

awk ‘{sub(/^[ \t]+/, “”)};1’ FILE_NAME

– Remove trailing whitespaces (spaces, tabs) for each line in a file.

awk ‘{sub(/[ \t]+$/, “”)};1’ FILE_NAME

– Remove both leading and trailing whitespaces (spaces, tabs) for each line in a file.

awk ‘{gsub(/^[ \t]+|[ \t]+$/,””)};1’ FILE_NAME

OR

awk ‘{$1=$1};1’ FILE_NAME

AWK One Liners

– Insert STRING / VALUE at the beginning of each line.

awk ‘{sub(/^/, “VALUE”)};1’ FILE_NAME

– The find and replace should work pretty much the same way. Let’s replace ‘foo’ with ‘bar’.

awk ‘{sub(/foo/, “bar”)};1’ FILE_NAME

– Conditional find and replace. Replace ‘foo’ with ‘bar’ only if line contains ‘baz’.

awk ‘/baz/{gsub(/foo/, “bar”)};1’ FILE_NAME

– Conditional find and replace. Replace ‘foo’ with ‘bar’ only if line DOES NOT contains ‘baz’.

awk ‘!/baz/{gsub(/foo/, “bar”)};1’ FILE_NAME

– Find and replace multiple strings with a single string. Change ‘red’ or ‘white’ or ‘green’ with ‘black’.

awk ‘{gsub(/red|white|green/, “black”)}; 1’ FILE_NAME

– Print lines which match the REGEX. Similar to grep REGEX.

awk ‘/REGEX/’ FILE_NAME

– Print lines which DOES NOT match the REGEX. Similar to grep -v REGEX.

awk ‘!/REGEX/’ FILE_NAME

– Print lines where field number N is abc123. We will take an example of field number 2 and REGEX abc123.

awk ‘$2 == “abc123″‘ FILE_NAME

– Print lines where field number N is NOT abc123. We will take an example of field number 2 and REGEX abc123.

awk ‘$2 != “abc123″‘ FILE_NAME

OR

awk ‘!($2 == “abc123”)’ FILE_NAME

– Print lines if field number N MATCHES (~) a regex. We will take an example of field number 2 and REGEX abc123.

awk ‘$2 ~ /abc123/’ FILE_NAME

– Print lines if field number N DOES NOT MATCHES (~) a regex. We will take an example of field number 2 and REGEX abc123.

awk ‘$2 !~ /abc123/’ FILE_NAME

– Print the line immediately after a REGEX, but not the line containing the REGEX.

awk ‘/REGEX/{getline; print}’ FILE_NAME

– Print content between two REGEX / String.

awk ‘/STRING1/,/STRING2/’ FILE_NAME

– Print and sort all the users (usernames) on a Unix/Linux server using passwd file.

awk -F “:” ‘{print $1 | “sort” }’ /etc/passwd

– Print the first 2 fields in the opposite order.

awk ‘{print $2, $1}’ FILE_NAME

– Print all fields except the second field for all lines.

awk ‘{ $2 = “”; print }’ FILE_NAME

– Print first N lines of a file. Similar to head .

awk ‘NR < N' FILE_NAME

OR

awk ‘NR>N{exit};1’ FILE_NAME

Where, in both the cases N stand for the number of lines to print.

– Print last line for a file. Similar to tail -1.

awk ‘END{print}’ FILE_NAME

– Print lines A to C. We will say print from line 11 to 15.

awk ‘NR==11,NR==15’ FILE_NAME

– Print line number N. Where N is the line number. We will print line number 4.

awk ‘NR==4’ FILE_NAME

OR

awk ‘NR==4 {print; exit}; FILE_NAME

AWK One Liners

No Comments

Post a Comment

Time limit is exhausted. Please reload CAPTCHA.