开发者

processing text using awk

开发者 https://www.devze.com 2023-02-05 00:36 出处:网络
I am trying to process a csv file using awk. I have five rows in my csv file. I want my column 1 & column 3 combined to be unique. Using awk I wrote

I am trying to process a csv file using awk. I have five rows in my csv file. I want my column 1 & column 3 combined to be unique. Using awk I wrote

awk -F "," '{pri开发者_如何学Gont $1,",",$2,","$3,",",$4,",",$5}' A.csv|sort|uniq >B.csv

What this code does is to have a unique row with 1 to 5 rows combined. I want my other columns to be in final csv file but sorted and uniq only by 1st and 3rd columns.


assuming you want to keep at least one of the duplicate lines

$ more file
1 2 3 4 5 6
7 4 6 3 5 8
1 43 3 6 5 10
2 3 4 4 4 4

$ awk '(!($1$3 in u)){u[$1$3]=$0}END{for(i in u ) print u[i]}' file
7 4 6 3 5 8
1 2 3 4 5 6
2 3 4 4 4 4

If you don't want to keep duplicate lines

$ awk '($1$3 in u){delete u[$1$3];next}{u[$1$3]=$0}END{for(i in u ) print u[i]}' file
7 4 6 3 5 8
2 3 4 4 4 4
0

精彩评论

暂无评论...
验证码 换一张
取 消