Linux 2 unit6 文本处理工具

###unit6###

diff 命令

diff 命令用于比较两个文件的内容 , 以了解其区别。它还可用于创建补丁文件。补丁文件用于在企业环境的多台计算机之间对相似文件进行更改

patch 命令

patch 采用补丁文件 patchfile ( 包含由 diff 生成的差异列表 ) 并将这些差异应用于生成补丁版的一个或多个原始文件。通常 , 补丁版替换原始文件 , 但当指定 -b 选项时 , 可以制作备份。用 .orig 文件名后缀重命名原始文件

grep 命令

grep 将显示文件中与模式匹配的行。其也可以处理标准输入将显示文件中与模式匹配的行。其也可以处理标准输入

Cut 命令

cut 用于 “ 剪切 ” 文件中的文本字段或列并将其显示到标准输出

sort 命令

sort 用于排序文本数据。该数据可以位于文件中或其他命令输出中。 Sort 通常与管道一起使用

uniq 命令

uniq“ 删除 ” 文件中重复的相邻行。若要只打印文件中出现的唯一行(“ 删除 ” 所有重复行 ), 必须首先对 uniq 的输入进行排序。由于可以为uniq 指定其决策所基于的字段或列 , 因此这些字段或列是对其输入进行排序所必须的字段或列。如果未与选项一起使用 , uniq 会使用整个记录作为决策键 , 删除其输入中的重复行

tr 命令

tr 用于转字符 : 即 , 如果给定了两个字符范围 , 则只要发现某个字符位于第一个范围中 , 就会将其转换为第二个范围中对等的字符。该命令通常在 shell 脚本中使用 , 以按预期情况转换数据

sed 命令

sed 命令是流编辑器 , 用于对文本数据流执行编辑。假定要处理一个文件名 , sed 将对文件中的所有行执行搜索和替换 , 以将修改后的数据发送到标准输出 ; 即 , 其实际上并不修改现有文件。与 grep 一样 , sed 通常在管道中使用

##diff命令的使用##

[root[email protected] ~]# cd /mnt

[[email protected] mnt]# ls

[[email protected] mnt]# vim file

[[email protected] mnt]# cat file

hello xiao

[[email protected] mnt]# vim file1

[[email protected] mnt]# cat file1

hello xiao

123

[[email protected] mnt]# diff file file1  ##查看文件的不同

1a2

> 123

[[email protected] mnt]# diff -c file file1   ##-c显示上下周围的行

*** file2017-04-29 21:08:17.822776336 -0400

--- file12017-04-29 21:08:32.733776336 -0400

***************

*** 1 ****

--- 1,2 ----

hello xiao

+ 123

[[email protected] mnt]# diff -u file file1 ##-u使用统一格式，对生成补丁文件非常有效

--- file2017-04-29 21:08:17.822776336 -0400

+++ file12017-04-29 21:08:32.733776336 -0400

@@ -1 +1,2 @@

hello xiao

+123

[[email protected] mnt]# diff -u file file1 > file.path ##比较两文件的不同，输出补丁文件为file.path

[[email protected] mnt]# ls

file  file1  file.path

[[email protected] mnt]# cat file.path

--- file2017-04-29 21:08:17.822776336 -0400

+++ file12017-04-29 21:08:32.733776336 -0400

@@ -1 +1,2 @@

hello xiao

+123

[[email protected] mnt]# yum install patch -y   ##打补丁插件

Loaded plugins: langpacks

rhel_dvd                                                 | 4.1 kB     00:00

Resolving Dependencies

--> Running transaction check

---> Package patch.x86_64 0:2.7.1-8.el7 will be installed

--> Finished Dependency Resolution

Dependencies Resolved

================================================================================

Package        Arch            Version                 Repository         Size

================================================================================

Installing:

patch          x86_64          2.7.1-8.el7             rhel_dvd          110 k

Transaction Summary

================================================================================

Install  1 Package

Total download size: 110 k

Installed size: 210 k

Downloading packages:

patch-2.7.1-8.el7.x86_64.rpm                               | 110 kB   00:00

Running transaction check

Running transaction test

Transaction test succeeded

Running transaction

Warning: RPMDB altered outside of yum.

Installing : patch-2.7.1-8.el7.x86_64                                     1/1

Verifying  : patch-2.7.1-8.el7.x86_64                                     1/1

Installed:

patch.x86_64 0:2.7.1-8.el7

Complete!

[[email protected] mnt]# patch file file.path  ##给file文件打补丁

patching file file

[[email protected] mnt]# cat file

hello xiao

123

[[email protected] mnt]# ls

file  file1  file.path

[[email protected] mnt]# vim file

[[email protected] mnt]# patch -b file file.path  ##-b备份原文件

patching file file

[[email protected] mnt]# ls

file  file1  file.orig  file.path

[[email protected] mnt]# cat file.orig

hello xiao

[[email protected] mnt]# cat file

hello xiao

123

###grep命令（过滤）###

[[email protected] mnt]# cp /etc/passwd .   ##复制passwd到当前位置

[[email protected] mnt]# ls

file  file1  file.orig  file.path  passwd

[[email protected] mnt]# vim passwd

[[email protected] mnt]# cat passwd

root:x:0:0:root:/root:/bin/bash

bin:x:1:1:bin:/bin:/sbin/nologin

daemon:x:2:2:daemon:/sbin:/sbin/nologin

adm:x:3:4:adm:/var/adm:/sbin/nologin

lp:x:4:7:lp:/var/spool/lpd:/sbin/nologin

sync:x:5:0:sync:/sbin:/bin/sync

shutdown:x:6:0:shutdown:/sbin:/sbin/shutdown

halt:x:7:0:halt:/sbin:/sbin/halt

mail:x:8:12:mail:/var/spool/mail:/sbin/nologin

operator:x:11:0:operator:/root:/sbin/nologin

games:x:12:100:games:/usr/games:/sbin/nologin

ftp:x:14:50:FTP User:/var/ftp:/sbin/nologin

nobody:x:99:99:Nobody:/:/sbin/nologin

test:root:root

test:test:root

root:test:root

root:root:tast

TEST:root:ROOT

[[email protected] mnt]# grep test passwd   ##查看含有test关键字的内容

test:root:root

test:test:root

root:test:root

[[email protected] mnt]# grep test passwd -i   ##模糊查找含有test的文件（忽略大小写）

test:root:root

test:test:root

root:test:root

TEST:root:ROOT

[[email protected] mnt]# grep test passwd -i -v   ##-v反向过滤

root:x:0:0:root:/root:/bin/bash

bin:x:1:1:bin:/bin:/sbin/nologin

daemon:x:2:2:daemon:/sbin:/sbin/nologin

adm:x:3:4:adm:/var/adm:/sbin/nologin

lp:x:4:7:lp:/var/spool/lpd:/sbin/nologin

sync:x:5:0:sync:/sbin:/bin/sync

shutdown:x:6:0:shutdown:/sbin:/sbin/shutdown

halt:x:7:0:halt:/sbin:/sbin/halt

mail:x:8:12:mail:/var/spool/mail:/sbin/nologin

operator:x:11:0:operator:/root:/sbin/nologin

games:x:12:100:games:/usr/games:/sbin/nologin

ftp:x:14:50:FTP User:/var/ftp:/sbin/nologin

nobody:x:99:99:Nobody:/:/sbin/nologin

root:root:tast

[[email protected] mnt]# grep -i -E "test|root" passwd  ##-E过滤多个关键字

root:x:0:0:root:/root:/bin/bash

operator:x:11:0:operator:/root:/sbin/nologin

test:root:root

test:test:root

root:test:root

root:root:tast

TEST:root:ROOT

[[email protected] mnt]# grep -i -E "^test" passwd  ##以test关键字开头的

test:root:root

test:test:root

TEST:root:ROOT

[[email protected] mnt]# grep -i -E "root$" passwd  ##以root关键字结尾的

test:root:root

test:test:root

root:test:root

TEST:root:ROOT

[[email protected] mnt]# grep "test" passwd | grep -E "^test|test$" -v ##反向查找含有test关键字且以其开头和结尾的

root:test:root

[[email protected] mnt]# ls

file  file1  file.orig  file.path  passwd

[[email protected] mnt]# echo westos > file ##将westos输出到file（westos覆盖file内容）

[[email protected] mnt]# cat file

westos

[[email protected] mnt]# grep westos -r /mnt/    ##过滤/mnt下的westos

/mnt/file:westos

[[email protected] mnt]# grep westos -r /mnt/ -n   ##-r对文件实行递归式搜索，从命名目录开始

/mnt/file:1:westos

###cut剪切###

[[email protected] mnt]# ls

passwd

[[email protected] mnt]# cut -d : -f 1 passwd  ##剪切第一列（-d用于指定提取字符段的分隔符，-f指定要从每行中提取的字段）

root

bin

daemon

adm

lp

sync

shutdown

halt

mail

operator

games

ftp

nobody

test

test

root

root

TEST

[[email protected] mnt]# cut -d : -f 1,3 passwd   ##剪切提取第一列和第三列

root:0

bin:1

daemon:2

adm:3

lp:4

sync:5

shutdown:6

halt:7

mail:8

operator:11

games:12

ftp:14

nobody:99

test:root

test:root

root:root

root:tast

TEST:ROOT

[[email protected] mnt]# cut -d : -f 1-3 passwd  ##剪切提取一到三列

root:x:0

bin:x:1

daemon:x:2

adm:x:3

lp:x:4

sync:x:5

shutdown:x:6

halt:x:7

mail:x:8

operator:x:11

games:x:12

ftp:x:14

nobody:x:99

test:root:root

test:test:root

root:test:root

root:root:tast

TEST:root:ROOT

[[email protected] mnt]# cut -c 1-3 passwd   ##剪切提取一到三字符（-c指定要从每行中提取的文本列）

roo

bin

dae

adm

lp:

syn

shu

hal

mai

ope

gam

ftp

nob

tes

tes

roo

roo

TES

实验：

[[email protected] mnt]# ifconfig eth0

eth0: flags=4163<UP,BROADCAST,RUNNING,MULTICAST>  mtu 1500

inet 172.25.254.113  netmask 255.255.255.0  broadcast 172.25.254.255

inet6 fe80::5054:ff:fe00:d0a  prefixlen 64  scopeid 0x20<link>

ether 52:54:00:00:0d:0a  txqueuelen 1000  (Ethernet)

RX packets 7323  bytes 844132 (824.3 KiB)

RX errors 0  dropped 0  overruns 0  frame 0

TX packets 2051  bytes 278634 (272.1 KiB)

TX errors 0  dropped 0 overruns 0  carrier 0  collisions 0

实验要求：提取出其中的ip地址172.25.254.113

1）[[email protected] mnt]# ifconfig eth0 | grep inet

inet 172.25.254.113  netmask 255.255.255.0  broadcast 172.25.254.255

inet6 fe80::5054:ff:fe00:d0a  prefixlen 64  scopeid 0x20<link>

[[email protected] mnt]# ifconfig eth0 | grep inet | inet6 -v

bash: inet6: command not found...

[[email protected] mnt]# ifconfig eth0 | grep inet | grep inet6 -v

inet 172.25.254.113  netmask 255.255.255.0  broadcast 172.25.254.255

[[email protected] mnt]# ifconfig eth0 | grep inet | grep inet6 -v | cut -d " " -f 10

172.25.254.113

2）[[email protected] mnt]# ifconfig eth0 | grep inet | grep inet6 -v | awk -F " " ‘{print $2}‘

172.25.254.113

###sort排序命令与uniq删除命令###

[[email protected] mnt]# vim file

[[email protected] mnt]# sort file   ##直接排序（只排了第一列数字）

1

1

11

13

13

2

2

2

2

22

22

4

4

45

45

5

5

6

6

7

7

76

76

8

8

9

9

[[email protected] mnt]# sort file -n  ##按数值大小排列（-n按数值而非字符排序）

1

1

2

2

2

2

4

4

5

5

6

6

7

7

8

8

9

9

11

13

13

22

22

45

45

76

76

[[email protected] mnt]# sort file -rn  ##-r倒序排列

76

76

45

45

22

22

13

13

11

9

9

8

8

7

7

6

6

5

5

4

4

2

2

2

2

1

1

[[email protected] mnt]# sort file -rnu  ##-u只排列一次，不重复出现

76

45

22

13

11

9

8

7

6

5

4

2

1

[[email protected] mnt]# sort file -rn | uniq -c  ##-c每行显示一次，删除重复行

2 76

2 45

2 22

2 13

1 11

2 9

2 8

2 7

2 6

2 5

2 4

4 2

2 1

[[email protected] mnt]# sort file -rn | uniq -d  ##-d只显示重复行

76

45

22

13

9

8

7

6

5

4

2

1

[[email protected] mnt]# sort file -rn | uniq -u   ##-u仅显示唯一行

11

[[email protected] mnt]# ls

file

[[email protected] mnt]# vim file

[[email protected] mnt]# sort file

1:@:1

1:@:4

14:@:45

23:@:76

2:@:6

2:@:6

3:@:2

3:@:8

4:@:22

4:@:4

5:@:45

6:@:1

6:@:13

6:@:2

6:@:5

65:@:22

6:@:7

6:@:9

7:@:11

7:@:2

7:@:2

7:@:5

7:@:9

9:@:7

9:@:76

9:@:8

99:@:13

[[email protected] mnt]# sort file -n

1:@:1

1:@:4

2:@:6

2:@:6

3:@:2

3:@:8

4:@:22

4:@:4

5:@:45

6:@:1

6:@:13

6:@:2

6:@:5

6:@:7

6:@:9

7:@:11

7:@:2

7:@:2

7:@:5

7:@:9

9:@:7

9:@:76

9:@:8

14:@:45

23:@:76

65:@:22

99:@:13

[[email protected] mnt]# sort -t : -k 3 -n file   ##-t指定分隔符 -k3将第三列按数值排序

1:@:1

6:@:1

3:@:2

6:@:2

7:@:2

7:@:2

1:@:4

4:@:4

6:@:5

7:@:5

2:@:6

2:@:6

6:@:7

9:@:7

3:@:8

9:@:8

6:@:9

7:@:9

7:@:11

6:@:13

99:@:13

4:@:22

65:@:22

14:@:45

5:@:45

23:@:76

9:@:76

[[email protected] mnt]# sort -t : -k 3 -n file | uniq -c  ##每行显示一次，删除重复行

1 1:@:1

1 6:@:1

1 3:@:2

1 6:@:2

2 7:@:2

1 1:@:4

1 4:@:4

1 6:@:5

1 7:@:5

2 2:@:6

1 6:@:7

1 9:@:7

1 3:@:8

1 9:@:8

1 6:@:9

1 7:@:9

1 7:@:11

1 6:@:13

1 99:@:13

1 4:@:22

1 65:@:22

1 14:@:45

1 5:@:45

1 23:@:76

1 9:@:76

实验：

[[email protected] mnt]# ps aux   ##查看各进程运行

USER       PID %CPU %MEM    VSZ   RSS TTY      STAT START   TIME COMMAND

root         1  0.0  0.2  49944  3972 ?        Ss   20:53   0:01 /usr/lib/systemd/systemd --switched-r

root         2  0.0  0.0      0     0 ?        S    20:53   0:00 [kthreadd]

root         3  0.0  0.0      0     0 ?        S    20:53   0:00 [ksoftirqd/0]

root         5  0.0  0.0      0     0 ?        S<   20:53   0:00 [kworker/0:0H]

root      3232  0.0  0.0 123356  1324 pts/1    R+   22:15   0:00 ps aux

[[email protected] mnt]# ps ax -o pid --sort -%mem | grep PID -v | head -n 5   ##只输出pid排名前五个的进程pid

1990

716

1926

2047

602

###sed替换命令###

[[email protected] mnt]# cp /etc/passwd .    ##复制文件到当前位置

[[email protected] mnt]# vim passwd

[[email protected] mnt]# cat passwd

root:x:0:0:root:/root:/bin/bash

bin:x:1:1:bin:/bin:/sbin/nologin

daemon:x:2:2:daemon:/sbin:/sbin/nologin

adm:x:3:4:adm:/var/adm:/sbin/nologin

lp:x:4:7:lp:/var/spool/lpd:/sbin/nologin

sync:x:5:0:sync:/sbin:/bin/sync

shutdown:x:6:0:shutdown:/sbin:/sbin/shutdown

halt:x:7:0:halt:/sbin:/sbin/halt

mail:x:8:12:mail:/var/spool/mail:/sbin/nologin

operator:x:11:0:operator:/root:/sbin/nologin

[[email protected] mnt]# sed ‘s/sbin/westos/g‘ passwd   ##将sbin用westos替换（一定用s开头和g结尾）

root:x:0:0:root:/root:/bin/bash

bin:x:1:1:bin:/bin:/westos/nologin

daemon:x:2:2:daemon:/westos:/westos/nologin

adm:x:3:4:adm:/var/adm:/westos/nologin

lp:x:4:7:lp:/var/spool/lpd:/westos/nologin

sync:x:5:0:sync:/westos:/bin/sync

shutdown:x:6:0:shutdown:/westos:/westos/shutdown

halt:x:7:0:halt:/westos:/westos/halt

mail:x:8:12:mail:/var/spool/mail:/westos/nologin

operator:x:11:0:operator:/root:/westos/nologin

[[email protected] mnt]# sed ‘s/sbin/westos/g‘ -i passwd   ##-i用来改变原文件里面的参数值

[[email protected] mnt]# cat passwd

root:x:0:0:root:/root:/bin/bash

bin:x:1:1:bin:/bin:/westos/nologin

daemon:x:2:2:daemon:/westos:/westos/nologin

adm:x:3:4:adm:/var/adm:/westos/nologin

lp:x:4:7:lp:/var/spool/lpd:/westos/nologin

sync:x:5:0:sync:/westos:/bin/sync

shutdown:x:6:0:shutdown:/westos:/westos/shutdown

halt:x:7:0:halt:/westos:/westos/halt

mail:x:8:12:mail:/var/spool/mail:/westos/nologin

operator:x:11:0:operator:/root:/westos/nologin

[[email protected] mnt]# sed ‘s/westos/sbin/g‘ -i passwd

[[email protected] mnt]# cat passwdroot:x:0:0:root:/root:/bin/bash

bin:x:1:1:bin:/bin:/sbin/nologin

daemon:x:2:2:daemon:/sbin:/sbin/nologin

adm:x:3:4:adm:/var/adm:/sbin/nologin

lp:x:4:7:lp:/var/spool/lpd:/sbin/nologin

sync:x:5:0:sync:/sbin:/bin/sync

shutdown:x:6:0:shutdown:/sbin:/sbin/shutdown

halt:x:7:0:halt:/sbin:/sbin/halt

mail:x:8:12:mail:/var/spool/mail:/sbin/nologin

operator:x:11:0:operator:/root:/sbin/nologin

[[email protected] mnt]# sed -e ‘s/sbin/westos/g‘ -e ‘s/nologin/linux/g‘ -i passwd  ##更改两个字符时记得用-e隔开

[[email protected] mnt]# cat passwdroot:x:0:0:root:/root:/bin/bash

bin:x:1:1:bin:/bin:/westos/linux

daemon:x:2:2:daemon:/westos:/westos/linux

adm:x:3:4:adm:/var/adm:/westos/linux

lp:x:4:7:lp:/var/spool/lpd:/westos/linux

sync:x:5:0:sync:/westos:/bin/sync

shutdown:x:6:0:shutdown:/westos:/westos/shutdown

halt:x:7:0:halt:/westos:/westos/halt

mail:x:8:12:mail:/var/spool/mail:/westos/linux

operator:x:11:0:operator:/root:/westos/linux

[[email protected] mnt]# vim rule

[[email protected] mnt]# sed -f rule passwd

root:x:0:0:root:/root:/bin/bash

bin:x:1:1:bin:/bin:/sbin/nologin

daemon:x:2:2:daemon:/sbin:/sbin/nologin

adm:x:3:4:adm:/var/adm:/sbin/nologin

lp:x:4:7:lp:/var/spool/lpd:/sbin/nologin

sync:x:5:0:sync:/sbin:/bin/sync

shutdown:x:6:0:shutdown:/sbin:/sbin/shutdown

halt:x:7:0:halt:/sbin:/sbin/halt

mail:x:8:12:mail:/var/spool/mail:/sbin/nologin

operator:x:11:0:operator:/root:/sbin/nologin.

[[email protected] mnt]# cat passwd -b > westos  ##-b显示行数

[[email protected] mnt]# cat westos

1root:x:0:0:root:/root:/bin/bash

2bin:x:1:1:bin:/bin:/westos/linux

3daemon:x:2:2:daemon:/westos:/westos/linux

4adm:x:3:4:adm:/var/adm:/westos/linux

5lp:x:4:7:lp:/var/spool/lpd:/westos/linux

6sync:x:5:0:sync:/westos:/bin/sync

7shutdown:x:6:0:shutdown:/westos:/westos/shutdown

8halt:x:7:0:halt:/westos:/westos/halt

9mail:x:8:12:mail:/var/spool/mail:/westos/linux

10operator:x:11:0:operator:/root:/westos/linux

[[email protected] mnt]# sed ‘3,5s/westos/sbin/g‘ passwd   ##更改三到五行

root:x:0:0:root:/root:/bin/bash

bin:x:1:1:bin:/bin:/westos/linux

daemon:x:2:2:daemon:/sbin:/sbin/linux

adm:x:3:4:adm:/var/adm:/sbin/linux

lp:x:4:7:lp:/var/spool/lpd:/sbin/linux

sync:x:5:0:sync:/westos:/bin/sync

shutdown:x:6:0:shutdown:/westos:/westos/shutdown

halt:x:7:0:halt:/westos:/westos/halt

mail:x:8:12:mail:/var/spool/mail:/westos/linux

operator:x:11:0:operator:/root:/westos/linux

[[email protected] mnt]# sed 5x westos   ##屏蔽第五行

1root:x:0:0:root:/root:/bin/bash

2bin:x:1:1:bin:/bin:/westos/linux

3daemon:x:2:2:daemon:/westos:/westos/linux

4adm:x:3:4:adm:/var/adm:/westos/linux

6sync:x:5:0:sync:/westos:/bin/sync

7shutdown:x:6:0:shutdown:/westos:/westos/shutdown

8halt:x:7:0:halt:/westos:/westos/halt

9mail:x:8:12:mail:/var/spool/mail:/westos/linux

10operator:x:11:0:operator:/root:/westos/linux

[[email protected] mnt]# sed 5d westos   ##删除第五行

1root:x:0:0:root:/root:/bin/bash

2bin:x:1:1:bin:/bin:/westos/linux

3daemon:x:2:2:daemon:/westos:/westos/linux

4adm:x:3:4:adm:/var/adm:/westos/linux

6sync:x:5:0:sync:/westos:/bin/sync

7shutdown:x:6:0:shutdown:/westos:/westos/shutdown

8halt:x:7:0:halt:/westos:/westos/halt

9mail:x:8:12:mail:/var/spool/mail:/westos/linux

10operator:x:11:0:operator:/root:/westos/linux

[[email protected] mnt]# sed 5p westos   ##复制第五行

1root:x:0:0:root:/root:/bin/bash

2bin:x:1:1:bin:/bin:/westos/linux

3daemon:x:2:2:daemon:/westos:/westos/linux

4adm:x:3:4:adm:/var/adm:/westos/linux

5lp:x:4:7:lp:/var/spool/lpd:/westos/linux

5lp:x:4:7:lp:/var/spool/lpd:/westos/linux

6sync:x:5:0:sync:/westos:/bin/sync

7shutdown:x:6:0:shutdown:/westos:/westos/shutdown

8halt:x:7:0:halt:/westos:/westos/halt

9mail:x:8:12:mail:/var/spool/mail:/westos/linux

10operator:x:11:0:operator:/root:/westos/linux

[[email protected] mnt]# sed -n 5p westos   ##只显示第五行

5lp:x:4:7:lp:/var/spool/lpd:/westos/linux

###tr 命令####

tr 用于转字符 : 即 , 如果给定了两个字符范围 , 则只要发现

某个字符位于第一个范围中 , 就会将其转换为第二个范围中

对等的字符。该命令通常在 shell 脚本中使用 , 以按预期

情况转换数据

tr ‘A-Z‘ ‘a-z‘ <file