前言

一本关于个人成长、软件开发、系统运维、产品运营、思维、思考的杂记。

贡献名单

按时间先后

  • 王朋

  • 尹磊

1. Docker

基于 CentOS 7

1.1. CentOS7安装配置

1.1.1. 准备分区

# XFS必须启用ftype参数,比如硬盘分区 /dev/sda1
mkfs.xfs -n ftype=1 -f $硬盘分区

# 挂载硬盘分区,用于存储Docker数据
echo `ll /dev/disk/by-uuid/|grep $硬盘分区|awk '{print "UUID="$9" /data                   xfs     defaults        0 0"}'` >> /etc/fstab

# 自动挂载所有分区
mount -a

# 验证挂载
df -h|grep /data

mkdir -p /data/var/lib/docker

1.1.2. 升级内核

由于内核版本太低,Docker 存储方面有限制。需要升级到更版本内核。

每次官方更新了内核版本,grub.cfg会被覆盖。需要重新执行以下步骤。
rpm --import https://www.elrepo.org/RPM-GPG-KEY-elrepo.org
rpm -Uvh http://www.elrepo.org/elrepo-release-7.0-3.el7.elrepo.noarch.rpm

# 稳定版本内核
yum --enablerepo=elrepo-kernel install kernel-ml
# 长期支持版本内核
yum --enablerepo=elrepo-kernel install kernel-lt


# 查找内核启动项
grep "menuentry 'CentOS Linux" /boot/grub2/grub.cfg|awk -F "'" '{print $2}'
grep "menuentry 'CentOS Linux" /boot/efi/EFI/centos/grub.cfg|awk -F "'" '{print $2}'

# 设置默认内核版本
grub2-set-default 'CentOS Linux (4.4.112-1.el7.elrepo.x86_64) 7 (Core)'

cp /boot/grub2/grub.cfg /boot/grub2/grub.cfg.bak
cp /boot/efi/EFI/centos/grub.cfg /boot/efi/EFI/centos/grub.cfg.bak

grub2-mkconfig -o /boot/grub2/grub.cfg
grub2-mkconfig -o /boot/efi/EFI/centos/grub.cfg

重启服务器。

1.1.3. 安装

yum install -y yum-utils \
  device-mapper-persistent-data \
  lvm2

yum-config-manager \
    --add-repo \
    http://mirrors.aliyun.com/docker-ce/linux/centos/docker-ce.repo

yum install -y docker-ce

1.1.4. 配置

modprobe br_netfilter
echo br_netfilter >> /etc/modules-load.d/modules.conf

echo "net.bridge.bridge-nf-call-ip6tables = 1" >> /etc/sysctl.conf
echo "net.bridge.bridge-nf-call-iptables = 1" >> /etc/sysctl.conf
echo "net.ipv4.ip_forward = 1" >> /etc/sysctl.conf

sysctl -p

mkdir -p /etc/docker
cat << EOF > /etc/docker/daemon.json
{
  "registry-mirrors": [
    "https://registry.docker-cn.com",
    "https://hub-mirror.c.163.com"
  ],
  "data-root": "/var/lib/docker",
  "storage-driver": "overlay2",
  "dns" : [
    "223.5.5.5",
    "223.6.6.6"
  ]
}
EOF
mirrors 位置最后一个优先级高
参数说明
registry-mirrors

Docker镜像源

graph

Docker数据目录

storage-driver

Docker 数据文件存储路径

dns

容器默认DNS。如果不设置可能出现 git pull 缓慢

1.1.5. 启动

systemctl enable docker
systemctl start docker

docker info

1.1.6. 新建自定义网桥

docker network create --subnet=10.10.10.0/16 --gateway=10.10.10.1 fifilyu
docker network inspect fifilyu

1.2. CentOS8安装配置

1.2.1. 准备分区

# XFS必须启用ftype参数,比如硬盘分区 /dev/sda1
mkfs.xfs -n ftype=1 -f $硬盘分区

# 挂载硬盘分区,用于存储Docker数据
echo `ll /dev/disk/by-uuid/|grep $硬盘分区|awk '{print "UUID="$9" /data                   xfs     defaults        0 0"}'` >> /etc/fstab

# 自动挂载所有分区
mount -a

# 验证挂载
df -h|grep /data

mkdir -p /data/var/lib/docker

1.2.2. 安装

dnf install -y yum-utils

yum-config-manager \
    --add-repo \
    https://download.docker.com/linux/centos/docker-ce.repo

dnf install -y docker-ce docker-ce-cli containerd.io

systemctl enable docker

1.2.3. 配置

echo "net.bridge.bridge-nf-call-ip6tables = 1" >> /etc/sysctl.conf
echo "net.bridge.bridge-nf-call-iptables = 1" >> /etc/sysctl.conf
echo "net.ipv4.ip_forward = 1" >> /etc/sysctl.conf

sysctl -p

mkdir -p /etc/docker
cat << EOF > /etc/docker/daemon.json
{
  "registry-mirrors": [
    "https://registry.docker-cn.com",
    "https://dockerhub.azk8s.cn",
    "https://hub-mirror.c.163.com"
  ],
  "data-root": "/var/lib/docker",
  "storage-driver": "overlay2",
  "dns" : [
    "223.5.5.5",
    "223.6.6.6"
  ]
}
EOF
mirrors 位置最后一个优先级高
参数说明
registry-mirrors

Docker镜像源

graph

Docker数据目录

storage-driver

Docker 数据文件存储路径

dns

容器默认DNS。如果不设置可能出现 git pull 缓慢

最后,启动或重启docker服务:

systemctl restart docker

1.2.4. 查看Docker运行信息

[root@dell7 ~]# docker info
Client:
 Debug Mode: false

Server:
 Containers: 0
  Running: 0
  Paused: 0
  Stopped: 0
 Images: 1
 Server Version: 19.03.13
 Storage Driver: overlay2
  Backing Filesystem: xfs
  Supports d_type: true
  Native Overlay Diff: true
 Logging Driver: json-file
 Cgroup Driver: cgroupfs
 Plugins:
  Volume: local
  Network: bridge host ipvlan macvlan null overlay
  Log: awslogs fluentd gcplogs gelf journald json-file local logentries splunk syslog
 Swarm: inactive
 Runtimes: runc
 Default Runtime: runc
 Init Binary: docker-init
 containerd version: 8fba4e9a7d01810a393d5d25a3621dc101981175
 runc version: dc9208a3303feef5b3839f4323d9beb36df0a9dd
 init version: fec3683
 Security Options:
  seccomp
   Profile: default
 Kernel Version: 4.18.0-147.8.1.el8_1.x86_64
 Operating System: CentOS Linux 8 (Core)
 OSType: linux
 Architecture: x86_64
 CPUs: 4
 Total Memory: 15.46GiB
 Name: dell7
 ID: WFX3:WXWH:WPML:VRCA:QRTN:LGCB:6UBM:V73O:QEGU:KADY:V3FH:E4Q6
 Docker Root Dir: /var/lib/docker
 Debug Mode: false
 Registry: https://index.docker.io/v1/
 Labels:
 Experimental: false
 Insecure Registries:
  127.0.0.0/8
 Registry Mirrors:
  https://hub-mirror.c.163.com/
 Live Restore Enabled: false

1.2.5. 测试

拉取测试镜像:

docker pull hello-world
控制台输出
[root@dell7 ~]# docker pull hello-world
Using default tag: latest
latest: Pulling from library/hello-world
0e03bdcc26d7: Already exists
Digest: sha256:e7c70bb24b462baa86c102610182e3efcb12a04854e8c582838d92970a09f323
Status: Downloaded newer image for hello-world:latest
docker.io/library/hello-world:latest

创建测试容器:

docker run --name hello hello-world
控制台输出
[root@dell7 ~]# docker run --name hello hello-world

Hello from Docker!
This message shows that your installation appears to be working correctly.

......

查看已经创建的容器:

docker ps -a
控制台输出
[root@dell7 ~]# docker ps -a
CONTAINER ID        IMAGE               COMMAND             CREATED             STATUS                      PORTS               NAMES
59cc42b09fce        hello-world         "/hello"            24 seconds ago      Exited (0) 22 seconds ago                       hello

删除测试容器:

docker rm hello

1.3. 拉取镜像慢的问题

很多时候就算更换了源之后,docker pull 还是很慢时,用拷贝的方法更快。

在有镜像的系统上导出镜像:

docker save -o docker-centos6-php52.tar fifilyu/docker-centos6-php52

scprsyncdocker-centos6-php52.tar 上传到目标系统,然后导入:

docker load -i docker-centos6-php52.tar

1.4. Pure-FTPd in Docker

必须使用 --without-capabilities 参数重新编译 Pure-FTPd,否则会出现以下错误:

[root@690bec85ee11 /]# /usr/sbin/pure-ftpd
421 Unable to switch capabilities : Operation not permitted

1.4.1. 重新打包

准备打包环境
cd ~
yum install yum-utils rpm-build gcc
yumdownloader --source pure-ftpd
rpm -ivh pure-ftpd-1.0.42-3.el7.src.rpm
yum-builddep pure-ftpd
修改包文件
sed -i 's/--with-capabilities /--without-capabilities /g' rpmbuild/SPECS/pure-ftpd.spec
准备源代码
rpmbuild -bp rpmbuild/SPECS/pure-ftpd.spec
重新打包
rpmbuild -ba rpmbuild/SPECS/pure-ftpd.spec
安装测试
rpm -ivh ~/rpmbuild/RPMS/x86_64/pure-ftpd-1.0.42-3.el7.centos.x86_64.rpm
/usr/sbin/pure-config.pl /etc/pure-ftpd/pure-ftpd.conf --daemonize

ps aux|grep ftp

1.5. 容器启动命令

docker run -d --restart=always --name test  centos:7 docker-entrypoint.sh

docker-entrypoint.sh 表示容器中的 /usr/local/bin/docker-entrypoint.sh ,通常内容为:

#!/bin/sh
/usr/bin/rm -f /run/nginx.pid && /usr/sbin/nginx -t && /usr/sbin/nginx

# 始终放到最后,防止以上命令启动失败,导致容器无法启动。
# 容器启动失败的处理流程较复杂,用此代码保证能进入容器处理故障。
while true
do
    sleep 100
done
单行版本
docker run \
	-d \
	--restart always \
	--env LANG=en_US.UTF-8 \
	--env TZ=Asia/Shanghai \
	--name lift360_all_data \
    --mount type=bind,source=/etc/localtime,target=/etc/localtime,readonly \
	--mount source=lift360_all_data,target=/data \
	centos:6 \
	bash -c 'while true; do sleep 100; done;'

1.5.1. 程序启动命令汇总

PSQL
docker pull centos/postgresql-94-centos7

docker run \
    -d \
    --restart always \
    --env LANG=en_US.UTF-8 \
    --env TZ=Asia/Shanghai \
    --mount type=bind,source=/etc/localtime,target=/etc/localtime,readonly \
    -p 18080:8080 \
    --name xxl_job_tomcat \
    xxl_job_tomcat \
    docker-entrypoint.sh

#免密码登录
echo '127.0.0.1:15432:esafesys_com:postgres:GcMuEKzRtvNIAiX1' > ~/.pgpass
chmod 600 ~/.pgpass

psql -h 127.0.0.1 -p 15432 -U postgres -c "create database esafesys_com;"

#导入数据库
psql -h 127.0.0.1 -p 15432 -U postgres esafesys_com -f foo.sql
MySQL5.1
/usr/bin/mysqld_safe

或者

/usr/libexec/mysqld \
    --basedir=/usr \
    --datadir=/var/lib/mysql \
    --user=mysql \
    --log-error=/var/log/mysqld.log \
    --pid-file=/var/run/mysqld/mysqld.pid \
    --socket=/var/lib/mysql/mysql.sock
MySQL5.6+
/usr/sbin/mysqld \
    --basedir=/usr \
    --datadir=/var/lib/mysql \
    --plugin-dir=/usr/lib64/mysql/plugin \
    --user=mysql --log-error=/var/log/mysqld.log \
    --pid-file=/var/run/mysqld/mysqld.pid \
    --socket=/var/lib/mysql/mysql.sock
Tomcat
su tomcat --shell /usr/libexec/tomcat/server start
zookeeper
su - zookeeper -c '/usr/local/zookeeper/bin/zkServer.sh start-foreground'
Nginx
/usr/bin/rm -f /run/nginx.pid && /usr/sbin/nginx -t && /usr/sbin/nginx
Pure-FTPd
/usr/sbin/pure-config.pl /etc/pure-ftpd/pure-ftpd.conf --daemonize
要在 Docker 中使用 Pure-FTPd,请查看 Pure-FTPd in Docker

2. Iptalbes

开启转发:

echo "net.ipv4.ip_forward = 1" >> /etc/sysctl.conf

sysctl -p

2.1. NAT

iptables -t nat -A PREROUTING -i eth0 -p tcp -m tcp --dport 40022 -j DNAT --to-destination 118.262.123.90:22
iptables -t nat -A POSTROUTING -d 118.262.123.90/32 -p tcp -m tcp -j MASQUERADE

iptables -t nat -A PREROUTING -i eth0 -p tcp -m multiport ! --dports 22,34587 -j DNAT --to-destination 118.262.123.90
iptables -t nat -A POSTROUTING -d 18.162.123.80/32 -p tcp -m tcp -j MASQUERADE

个别时候,也可以用python循环实现差不多的效果:

import os
while True:
    os.system('ssh root@125.64.14.218 -p 28529 -N -R 5432:localhost:43306 -v')

2.2. NAT + FTP

Docker 网桥默认转发所有数据包,不限制。

iptables -A FORWARD -i br-da448a6a9576 ! -o br-da448a6a9576 -j ACCEPT
iptables -A FORWARD -i br-da448a6a9576 -o br-da448a6a9576 -j ACCEPT

iptables -t nat -A PREROUTING -i br_wan -p tcp -m tcp -m multiport --dports 21,52000:52050 -j DNAT --to-destination 172.18.0.4
iptables -t nat -A POSTROUTING -s 172.18.0.0/16 ! -o br-da448a6a9576 -j MASQUERADE

KVM 网桥,默认只允许内网发起的连接,但限制外部访问内网服务端口。

必须增加 --dports 21:63535 -j ACCEPT 放行服务端口。

被动模式端口

52000:52050

iptables -A FORWARD -d 192.168.122.0/24 -o virbr0 -p tcp -m multiport --dports 21:63535 -j ACCEPT
iptables -A FORWARD -d 192.168.122.0/24 -o virbr0 -m conntrack --ctstate RELATED,ESTABLISHED -j ACCEPT
iptables -A FORWARD -s 192.168.122.0/24 -i virbr0 -j ACCEPT
iptables -A FORWARD -i virbr0 -o virbr0 -j ACCEPT
iptables -A FORWARD -o virbr0 -j REJECT --reject-with icmp-port-unreachable
iptables -A FORWARD -i virbr0 -j REJECT --reject-with icmp-port-unreachable

iptables -t nat -A PREROUTING -i br_wan -p tcp -m tcp --dport 18421 -j DNAT --to-destination 172.18.0.4:21
iptables -t nat -A PREROUTING -i br_wan -p tcp -m tcp -m multiport --dports 52000:52050 -j DNAT --to-destination 172.18.0.4
iptables -t nat -A POSTROUTING -s 192.168.122.0/24 ! -o virbr0 -j MASQUERADE

3. Nginx

3.1. HTTP基本身份验证

3.1.1. 生成密码文件

安装 htpasswd 命令的软件包和 pwgen 包:

Debian/Ubuntu
apt install apache2-utils pwgen
RHEL/CentOS
yum install -y httpd-tools pwgen
# 或
dnf install -y httpd-tools pwgen
git_password=`pwgen -s 20`
echo $git_password

# 添加第一个用户时,创建 /etc/nginx/.htpasswd  文件
echo $git_password | htpasswd -i -c /etc/nginx/.htpasswd git

添加第二个用户时,不需要 -c 参数(创建密码文件)

echo 'test123456' | htpasswd -i /etc/nginx/.htpasswd test
配置Nginx

在Nginx的Server配置中,加入:

location /api {
    auth_basic           "Administrator’s Area";
    auth_basic_user_file /etc/nginx/.htpasswd;
}

生效新配置:

nginx -t && nginx -s reload

3.2. proxy_pass 传递查询参数

proxy_pass 最后一个 "/" 表示传递查询参数到后端。

location /api-elevator/ {
    proxy_pass http://ttd1_api_elevator:8080/;
}
yum install -y httpd-tools
htpasswd -b -c /etc/nginx/conf.d/dl.2012iot.com.pass admin CaNxR7doJLW76HPXGkrsMmH9epCtSIbn23bz6bOm
server {
    listen    80;
    server_name dl.2012iot.com;
    root /usr/share/nginx/html;
    access_log off;

    auth_basic "auth";
    auth_basic_user_file /etc/nginx/conf.d/dl.2012iot.com.pass;

    charset UTF-8;
    autoindex on;
    autoindex_exact_size off;
    autoindex_localtime on;
}

3.3. 目录浏览

server {
        listen       80;

        server_name dl.cdgeekcamp.com;
        root /data/web/dl.cdgeekcamp.com;

        autoindex on;
        autoindex_localtime on;
        # 设置文件大小显示单位
        autoindex_exact_size off;
}

日志文件设置MIME,支持直接浏览器显示:

yum install -y httpd-tools pwgen

log_password=$(pwgen -s 20)
echo $log_password

htpasswd -b -c /etc/nginx/.htpasswd log $log_password

server {
    listen 34567;
    server_name _;
    root /var/log/nginx;

    autoindex on;
    autoindex_localtime on;
    # 设置文件大小显示单位
    autoindex_exact_size off;

    auth_basic "Administrator’s Area";
    auth_basic_user_file /etc/nginx/.htpasswd;

    location ~ \.log$ {
        default_type text/plain;
    }
}
.gz 文件会提示下载(MIME:application/octet-stream)

3.4. 缓存静态文件

proxy_cache_path /var/lib/nginx/tmp/static_cache levels=1:2 keys_zone=static_cache:100m inactive=1d max_size=10g;

server {
    listen 80;
    server_name chinakaoyan.com www.chinakaoyan.com;
    root /usr/share/nginx/html/;
    access_log  /var/log/nginx/www.log  main;

    location / {
        proxy_pass http://192.168.2.204:80;
    }

    location ~* \.(ico|gif|jpg|jpeg|png|bmp|zip|rar|7z|js|css|docx?|xls|pdf)$ {
        proxy_cache_valid 200 10d;
        proxy_cache_key $uri;
        proxy_cache static_cache;
        proxy_pass http://192.168.2.204:80;
    }
}

3.5. 关于代理链的客户端真实IP地址问题

173.88.111.174(浏览器) → 103.198.200.207(CDN) → 102.237.179.242(网络反代)

常规配置,应用程序只能得到CDN的IP地址,获得真实客户端(浏览器)IP地址需要特殊代码。

启用 ngx_http_realip_module 模块,可以不修改代码获得真实客户端IP地址。

3.5.1. CDN Nginx配置

103.198.200.207 Nginx配置:

server {
    location / {
        # 一旦当前作用域设置proxy_set_header,全局设置即刻失效,注意补全所有需要的Header
        proxy_set_header Host $host;
        proxy_set_header X-Real-IP $remote_addr;
        proxy_set_header X-Forwarded-Proto http;
        proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
        proxy_pass http://127.0.0.1:1034;
    }
}

3.5.2. 应用服务器Nginx配置

102.237.179.242 Nginx配置:

http {
    log_format  main  '$host $server_port $remote_addr - $remote_user [$time_local] "$request" '
                      '$status $request_time $body_bytes_sent "$http_referer" '
                      '"$http_user_agent" "$http_x_forwarded_for" $realip_remote_addr';

    log_format  postdata  '$host $server_port $remote_addr - $remote_user [$time_local] "$request" '
                      '$status $request_time $body_bytes_sent "$http_referer" ""'
                      '"$http_user_agent" "$http_x_forwarded_for" $realip_remote_addr "$request_body"';

    set_real_ip_from 0.0.0.0/0;
    real_ip_header X-Real-IP;
    real_ip_recursive on;
}
启用 ngx_http_realip_module 模块前
foo.com 2081 102.237.179.242 - - [07/Oct/2023:13:05:23 +0800] "GET /api/login HTTP/1.0" 200 0.062 58 "https://www.foo.com/" "Mozilla/5.0 (Linux; Android 6.0; Nexus 5 Build/MRA58N) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/116.0.0.0 Mobile Safari/537.36" "173.88.111.174, 103.198.200.207"
启用 ngx_http_realip_module 模块后
foo.com 2081 173.88.111.174 - - [07/Oct/2023:13:05:23 +0800] "GET /api/login HTTP/1.0" 200 0.062 58 "https://www.foo.com/" "Mozilla/5.0 (Linux; Android 6.0; Nexus 5 Build/MRA58N) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/116.0.0.0 Mobile Safari/537.36" "173.88.111.174"

4. PostgreSQL

4.1. 基础入门

4.1.1. 初始化

mkdir -p /var/lib/postgres/data
chown -R postgres:postgres /var/lib/postgres
su - postgres -c "initdb --locale en_US.UTF-8 -D '/var/lib/postgres/data'"
默认数据库

postgres

默认用户名称

postgres

默认用户密码

4.1.2. 配置文件

主配置文件

/var/lib/postgres/data/postgresql.conf

客户端认证配置文件

/var/lib/postgres/data/pg_hba.conf

pg_hba.conf 中有:

# TYPE  DATABASE        USER            ADDRESS                 METHOD

# "local" is for Unix domain socket connections only
local   all             all                                     trust
# IPv4 local connections:
host    all             all             127.0.0.1/32            trust
socket连接

local 行表示任意用户使用socket(不指定 -h 参数)时连接连接所有数据库,不需要密码。比如,

[postgres@ryzen7 ~]$ psql
psql (13.4)
输入 "help" 来获取帮助信息.

postgres=#

这种连接和监听端口 5432 无关。

连接指定主机(端口5432)

host 行表示任意用户通过 -h 127.0.0.1 连接所有数据库,不需要密码。

[postgres@ryzen7 ~]$ psql -h 127.0.0.1
psql (13.4)
输入 "help" 来获取帮助信息.

postgres=

通常,我会将上面这行修改为:

host    all             all             0.0.0.0/0               password

表示 -h 指定主机时,必须使用密码登录。

4.1.3. 启动服务

Arch Linux
systemctl enable postgresql

systemctl start postgresql

systemctl status postgresql

4.1.4. 通过socket登录数据库

切换到postgres用户:

sudo su - postgres
控制台输出
[fifilyu@ryzen7 ~]$ sudo su - postgres
[sudo] fifilyu 的密码:

[postgres@ryzen7 ~]$ id
用户id=88(postgres) 组id=88(postgres) 组=88(postgres)

直接socket登录pg,不需要 -h 参数:

sudo su - postgres -c psql
控制台输出
[fifilyu@ryzen7 ~]$ sudo su - postgres -c psql
[sudo] fifilyu 的密码:

psql (13.4)
输入 "help" 来获取帮助信息.

postgres=#

4.1.5. psql创建用户

psql 登录pg后,新建用户 foo,同时设置密码:

create user foo with password 'password';
控制台输出
[fifilyu@ryzen7 ~]$ sudo su - postgres -c psql
[sudo] fifilyu 的密码:
psql (13.4)
输入 "help" 来获取帮助信息.

postgres=# create user foo with password 'password';
CREATE ROLE
postgres=#

4.1.6. psql创建数据库

psql 登录pg后,新建数据库 testdb,同时设置数据库主用户:

create database testdb owner foo;
控制台输出
[fifilyu@ryzen7 ~]$ sudo su - postgres -c psql
[sudo] fifilyu 的密码:
psql (13.4)
输入 "help" 来获取帮助信息.

postgres=# create database testdb owner foo;
CREATE DATABASE
postgres=#

4.1.7. 测试通过网络登录数据库

用新建的用户登录数据库 testdb

psql -h 127.0.0.1 -U foo testdb

控制台输出——密码验证成功
[fifilyu@ryzen7 ~]$ psql -h 127.0.0.1 -U foo testdb
用户 foo 的口令:
psql (13.4)
输入 "help" 来获取帮助信息.

testdb=>
控制台输出——密码错误提示
[fifilyu@ryzen7 ~]$ psql -h 127.0.0.1 -U foo testdb
用户 foo 的口令:
psql: 错误: FATAL:  password authentication failed for user "foo"
pg_hba.conf 中必须设置 password 认证密码登录才会有效。

4.1.8. 导入导出

导出指定数据库数据到SQL文件
pg_dump --inserts -h 127.0.0.1 -U foo testdb -f testdb.sql
导入SQL文件到指定数据库
psql -h 127.0.0.1 -U foo testdb -f testdb.sql

4.1.9. pgsql控制台命令

\h

查看SQL命令的解释,比如\h select。

\?

查看psql命令列表。

\l

列出所有数据库。

\c [database_name]

连接其他数据库。

\d

列出当前数据库的所有表格。

\d [table_name]

列出某一张表格的结构。

\du

列出所有用户。

\e

打开文本编辑器。

\conninfo

列出当前数据库和连接的信息。

\password

修改当前用户密码

\q

退出pgsql

4.2. 常用SQL

创建用户
create user dbuser with password 'password';
用户权限设置
grant all privileges on database db to dbuser;
创建数据库
create database db;

create database db owner dbuser;
重命名数据库
alter database db rename to newdb;
创建新表
create table table_name(name varchar(20), signup_date date);
插入数据
insert into table_name(name, signup_date) values('张三', '2013-12-22');
查看记录
select * from table_name;
更新数据
update table_name set name = '李四' where name = '张三';
删除记录
delete from table_name where name = '李四' ;
新增表字段
alter table table_name add email varchar(40);
修改表字段
alter table table_name alter column signup_date set not null;
重命名表字段
alter table table_name rename column signup_date to signup;
删除表字段
alter table table_name drop column email;
重命名表
alter table table_name rename to new_table_name;
删除表
drop table if exists table_name;

4.3. 常用技巧

4.3.1. 查看表大小

select relname, pg_size_pretty(pg_relation_size(relid)) from pg_stat_user_tables where schemaname='public' order by pg_relation_size(relid) desc;
控制台输出
zpmod=> select relname, pg_size_pretty(pg_relation_size(relid)) from pg_stat_user_tables where schemaname='public' order by pg_relation_size(relid) desc;
  relname  | pg_size_pretty
-----------+----------------
 node      | 8192 bytes
 node_attr | 0 bytes
(2 行记录)

5. Linux

5.1. 常用命令

5.1.1. 烤机必备

从0设备到空设备持续写入,会占用单核100%:

dd if=/dev/zero of=/dev/null

如果不够,多来几个就行。

5.1.2. find分别查找目录和文件

修改当前路径所有目录的权限
find . -type d|xargs  -d '\n' chmod 755
修改当前路径所有文件的权限
find . -type f|xargs  -d '\n' chmod 644
由于 xargs 默认使用空格为分隔符,导致目录或文件中有空格时无法正常使用。

5.1.3. 创建用于运行服务的用户

useradd --comment  "Python User" --user-group --no-create-home --shell /usr/bin/nologin  python

5.1.4. 仅修改子目录中的文件权限,忽略目录

find . -type f -exec chmod 644 -- {} +

5.1.5. 跨行替换文件内容

需要将

INSERT INTO `tb1` VALUES (1, 607);
INSERT INTO `tb1` VALUES (2, 1851);

替换为

INSERT INTO `tb1` VALUES (1, 607),(2, 1851);
perl -i -p -E 'BEGIN{undef $/;} s/;\r\nINSERT INTO `[a-zA-Z0-9_]+` VALUES \(/, \(/smg' unilive.sql.test

5.1.6. Nginx日志按日期排序

foo.com 443 38.112.15.172 - - [24/Oct/2023:13:26:53 +0800] "GET / HTTP/1.0" 200 0.001 19023 "https://foo.com/" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/105.0.5195.127 Safari/537.36"
sort -t ' ' \
    -k 6.2,6.3n \
    -k 6.5,6.7M \
    -k 6.9,6.12n \
    -k 6.14,6.15n \
    -k 6.17,6.18n \
    -k 6.20,6.21n \
    access.log
参数说明
-t

以空格分列,针对第六列 [24/Oct/2023:13:26:53 做排序操作

-k

排序关键字,参数值格式:"开始列号[排序规则],结束列号[排序规则]"。 比如,1n,1n 表示从第1列行首开始到第1列行尾结束,都以数字规则 "n" 排序。按前后顺序,参数分别表示:

  • 日期(如 24)按数字排序

  • 月份(如 Oct)按英文月份缩写排序

  • 年份(如 2023)按数字排序

  • 小时(如 13)按数字排序

  • 分钟(如 26)按数字排序

  • 秒(如 53)按数字排序

5.1.7. 双网卡

单网卡系统已经正常工作,配有公网IP地址。

现,需要双公网IP地址(不同网段)上网。

所以,需要设置网卡路由支持第二张网卡IP通外网。

查看路由表
default via 192.168.0.1 dev eth0 proto dhcp metric 100
169.254.169.254 via 192.168.0.1 dev eth0 proto dhcp metric 100
192.168.0.0/24 dev eth0 proto kernel scope link src 192.168.0.68 metric 100
192.168.1.0/24 dev eth1 proto kernel scope link src 192.168.1.26 metric 101
临时有效
ip rule

ip route add default via 192.168.1.1 dev eth1 table 20
ip route add 192.168.1.0/24 dev eth1 table 20
ip rule add from 192.168.1.175 table 20

ip rule

ip route show table 20

ping -I 192.168.1.175 223.5.5.5
重启有效
cat << EOF >> /etc/rc.local
# wait for nics up
sleep 5
# Add v4 routes for eth1
ip route flush table 20
ip route add default via 192.168.1.1 dev eth1 table 20
ip route add 192.168.1.0/24 dev eth1 table 20
ip rule add from 192.168.1.175 table 20
EOF

5.2. NetworkManager

5.2.1. nmcli

配置IP地址
nmcli c modify eth0 ipv4.addresses 192.168.2.12/24 ipv4.method manual
nmcli c modify eth0 ipv4.gateway 192.168.2.1
nmcli c modify eth0 ipv4.dns "223.5.5.5 223.6.6.6"
物理网卡加入网桥
nmcli c add type Ethernet autoconnect yes con-name em1 ifname em1

nmcli c add type bridge autoconnect yes con-name br1 ifname br1
nmcli c modify br1 bridge.stp no
nmcli c modify br1 ipv6.method ignore
nmcli c modify br1 ipv4.addresses 192.168.2.8/24 ipv4.method manual
nmcli c modify br1 ipv4.gateway 192.168.2.1
nmcli c modify br1 ipv4.dns "223.5.5.5 223.6.6.6"

# 以下3条命令必须同时执行,最好放到脚本里面执行。因为,delete网卡后,可能会断网。重启网络服务或启动网桥后才能恢复
cat << EOF > /tmp/set_br1.sh
nmcli c delete em1
nmcli c add type bridge-slave autoconnect yes con-name em1 ifname em1 master br1
nmcli c up br1
EOF

sh /tmp/set_br1.sh &
rm -f /tmp/set_br1.sh

6. Libvirt

6.1. 安装配置

6.1.1. 安装kvm环境

yum install -y virt-manager libvirt virt-install qemu-kvm bridge-utils
# lsmod | grep kvm

kvm_intel       138567  0
kvm             441119  1 kvm_intel
systemctl start libvirtd

systemctl enable libvirtd

echo "net.ipv4.ip_forward = 1" >> /etc/sysctl.conf
sysctl -p

6.1.2. 设置网桥

设置内网网桥和 DHCP

编辑 /etc/libvirt/qemu/networks/default.xml,修改IP段为 192.168.122.0/24,stp='on'

<network>
  <name>default</name>
  <uuid>9719a6f8-7e16-4e7b-a592-909337ddaf73</uuid>
  <forward mode='nat'/>
  <bridge name='virbr0' stp='on' delay='0'/>
  <mac address='52:54:00:dd:9e:e0'/>
  <ip address='192.168.122.1' netmask='255.255.255.0'>
    <dhcp>
      <range start='192.168.122.2' end='192.168.122.254'/>
    </dhcp>
  </ip>
</network>

编辑 /var/lib/libvirt/dnsmasq/default.conf,修改IP段为 192.168.122.0/24

dhcp-range=192.168.122.2,192.168.122.254
重启libvirtd
systemctl restart libvirtd
查看默认网桥virbr0
# ip a
4: virbr0: <NO-CARRIER,BROADCAST,MULTICAST,UP> mtu 1500 qdisc noqueue state DOWN
    link/ether 52:54:00:dd:9e:e0 brd ff:ff:ff:ff:ff:ff
    inet 192.168.122.1/24 brd 192.168.122.255 scope global virbr0
       valid_lft forever preferred_lft forever
5: virbr0-nic: <BROADCAST,MULTICAST> mtu 1500 qdisc pfifo_fast master virbr0 state DOWN qlen 500
    link/ether 52:54:00:dd:9e:e0 brd ff:ff:ff:ff:ff:ff

# brctl show
bridge name	bridge id		STP enabled	interfaces
virbr0		8000.525400dd9ee0	off		virbr0-nic
CentOS 8可以使用 bridge link shownmcli -f bridge c s virbr0

6.1.3. 配置公网网桥,使虚拟机可以绑定公网IP

nmcli依赖于NetworkManager服务
systemctl unmask NetworkManager
systemctl start NetworkManager
systemctl status NetworkManager
添加网桥
nmcli c add type bridge autoconnect yes con-name br1 ifname br1
nmcli c modify br1 bridge.stp no
nmcli c modify br1 ipv6.method ignore
nmcli c modify br1 ipv4.addresses 182.131.21.23/28 ipv4.method manual
nmcli c modify br1 ipv4.gateway 182.131.21.1
nmcli c modify br1 ipv4.dns "223.5.5.5 223.6.6.6"

删除物理网卡em1,将网卡加入网桥。em1的IP地址也自动加入到了网桥。

以下3条命令必须同时执行,最好放到脚本里面执行。因为,delete网卡后,可能会断网。重启网络服务或启动网桥后才能恢复

cat << EOF > /tmp/set_br1.sh
nmcli c delete em1
nmcli c add type bridge-slave autoconnect yes con-name em1 ifname em1 master br1
nmcli c up br1
EOF

chmod 755 /tmp/set_br1.sh
sh /tmp/set_br1.sh &
rm -f /tmp/set_br1.sh
CentOS 7 上意外关闭了 NetworkManager 服务,重启后导致无法访问网络。注意,保持NetworkManager开机运行。

6.1.4. 查看 os-variant 类型

# osinfo-query os|grep -i centos
 centos6.0            | CentOS 6.0                                         | 6.0      | http://centos.org/centos/6.0
 centos6.1            | CentOS 6.1                                         | 6.1      | http://centos.org/centos/6.1
 centos6.2            | CentOS 6.2                                         | 6.2      | http://centos.org/centos/6.2
 centos6.3            | CentOS 6.3                                         | 6.3      | http://centos.org/centos/6.3
 centos6.4            | CentOS 6.4                                         | 6.4      | http://centos.org/centos/6.4
 centos6.5            | CentOS 6.5                                         | 6.5      | http://centos.org/centos/6.5
 centos7.0            | CentOS 7.0                                         | 7.0      | http://centos.org/centos/7.0

# osinfo-query os|grep -i 2008
 mandriva2008.0       | Mandriva Linux 2008                                | 2008.0   | http://mandriva.com/mandriva/2008.0
 mandriva2008.1       | Mandriva Linux 2008 Spring                         | 2008.1   | http://mandriva.com/mandriva/2008.1
 win2k8               | Microsoft Windows Server 2008                      | 6.0      | http://microsoft.com/win/2k8
 win2k8r2             | Microsoft Windows Server 2008 R2                   | 6.1      | http://microsoft.com/win/2k8r2

# osinfo-query os|grep -i 2012
 win2k12              | Microsoft Windows Server 2012                      | 6.3      | http://microsoft.com/win/2k12
 win2k12r2            | Microsoft Windows Server 2012 R2                   | 6.3      | http://microsoft.com/win/2k12r2

6.1.5. 增加虚拟机

qemu-img create -f qcow2 /data/libvirt/images/192168100009.img 30G

virt-install \
--name 192168100009 \
--ram 4096 \
--disk path=/data/libvirt/images/192168100009.img,size=30 \
--vcpus 4 \
--cpu host-model,topology.sockets=1,topology.cores=4,topology.threads=1 \
--os-variant centos6 \
--network bridge=br0 \
--console pty,target_type=serial \
--cdrom=/var/lib/libvirt/images/CentOS-7-x86_64-Minimal-1511.iso \
--graphics vnc,password=aC8W5It9nOyrXchH,port=-1,listen=0.0.0.0

virt-install \
--name tpl_win2k12r2 \
--ram 4048 \
--disk path=/data/libvirt/images/tpl_win2k12r2_hd0.img,size=100 \
--vcpus 4 \
--vcpus 4 \
--cpu host-model,topology.sockets=1,topology.cores=4,topology.threads=1 \
--os-variant win2k12r2 \
--network bridge=virbr0 \
--console pty,target_type=serial \
--cdrom=/data/libvirt/iso/win2k12r2.iso \
--graphics vnc,password=aC8W5It9nOyrXchH,port=-1,listen=0.0.0.0

# 从已经存在的镜像文件(tpl_win2k12r2.img)创建虚拟机
virt-install \
--name tpl_win2k12r2 \
--ram 4048 \
--disk path=/data/libvirt/images/tpl_win2k12r2.img \
--vcpus 4 \
--cpu host-model,topology.sockets=1,topology.cores=4,topology.threads=1 \
--os-variant win2k12r2 \
--network bridge=virbr0 \
--console pty,target_type=serial \
--import \
--graphics vnc,password=aC8W5It9nOyrXchH,port=-1,listen=0.0.0.0

CPU核心数设置:

--vcpus 4 \
--cpu host-model,topology.sockets=1,topology.cores=4,topology.threads=1 \

对应的XML配置:

  <vcpu placement='static'>4</vcpu>
  <cpu mode='host-passthrough' check='none' migratable='on'>
    <topology sockets='2' cores='4' threads='2'/>
  </cpu>
显示启动菜单
  <os>
    <bootmenu enable='yes'/>
  </os>
显示物理CPU
  <cpu mode='host-passthrough' />
错误配置
  <cpu mode='host-passthrough'>
    <feature policy='disable' name='lahf_lm'/>
  </cpu>

会导致windows不停重启。

多网卡并指定网卡名称
    <interface type='bridge'>
      <mac address='52:54:00:3e:61:be'/>
      <source bridge='br1'/>
      <target dev='priweb1_wan'/>
      <model type='rtl8139'/>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x03' function='0x0'/>
    </interface>
    <interface type='bridge'>
      <mac address='52:54:00:3e:61:bf'/>
      <source bridge='virbr0'/>
      <target dev='priweb1_lan'/>
      <model type='rtl8139'/>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x06' function='0x0'/>
    </interface>
    <serial type='pty'>
调试

--debug 参数

指定VNC端口

--graphics vnc,password=cloud,port=5911,listen=0.0.0.0

自动指定VNC端口

--graphics vnc,password=cloud,port=-1,listen=0.0.0.0

查看自动指定的端口

virsh qemu-monitor-command tpl_centos7 --hmp info vnc

查看自动指定的端口

virsh qemu-monitor-command tpl_win2k8r2 --hmp info vnc

从多个硬盘镜像导入
  • --disk path=/data/libvirt/images/primary_website_1_hd0.img

  • --disk path=/data/libvirt/images/primary_website_1_hd1.img

9600.17050.WINBLUE_REFRESH.140317-1640_X64FRE_SERVER_EVAL_ZH-CN-IR3_SSS_X64FREE_ZH-CN_DV9.iso 这种文件名virt-isntall是无法识别的,提示找不到iso

无光驱配置,增加配置
cat << EOF > guest-device.xml
<disk type='block' device='cdrom'>
  <driver name='qemu' type='raw'/>
  <target dev='vdc' bus='sata'/>
  <readonly/>
</disk>
EOF

virsh attach-device --config tpl_win2k12r2 guest-device.xml
有光驱,动态插入ISO
virsh change-media tpl_win2k12r2 vdc /data/libvirt/iso/win2k12r2.iso

注意设置启动菜单:

<os>
  ....
  <boot dev='cdrom'/>
  <boot dev='hd'/>
  <bootmenu enable='yes'/>
</os>
内网NAT
iptables -t nat -A PREROUTING -d 182.131.21.23/32 -p tcp -m tcp --dport 23389 -j DNAT --to-destination 192.168.122.2:3389
iptables -t nat -A PREROUTING -d 182.131.21.23/32 -p tcp -m tcp --dport 322 -j DNAT --to-destination 192.168.122.3:22
iptables -I FORWARD -p tcp -m multiport -d 192.168.122.0/24 -o virbr0 --dports 21,22,3389  -j ACCEPT
service iptables save
修改虚拟机网桥
brctl delif br0 ${vpsname}
brctl addif br1 ${vpsname}
屏蔽外部DHCP
iptables -t nat -A PREROUTING -p udp -m udp --sport 67 -j DROP
iptables -t nat -A FORWARD -o enp5s0f0 -p stp -j DROP
iptables -t nat -A OUTPUT -o enp5s0f0 -p stp -j DROP
iptables -t nat -A POSTROUTING -p udp -m udp --dport 67 -j DROP

firewall-cmd --permanent --direct --passthrough ipv4  -t nat -A POSTROUTING -s 10.8.0.0/24 -j SNAT --to-source 10.162.xxx.xxx
[source, bash]

6.2. 常用命令

qemu-img create -f qcow2 /var/lib/libvirt/images/vm192168122253.img 100G

virt-install \
--name vm192168122253 \
--ram 4096 \
--disk path=/var/lib/libvirt/images/vm192168122253.img,size=100,format=qcow2 \
--vcpus 4 \
--os-type windows \
--os-variant win2k8r2 \
--network bridge=virbr0 \
--console pty,target_type=serial \
--cdrom=/var/lib/libvirt/images/7601.17514.101119-1850_x64fre_server_eval_zh-cn-GRMSXEVAL_CN_DVD.iso \
--graphics vnc,password=YR710pwKX9tGfUOhnRWD,port=-1,listen=0.0.0.0

virt-clone -o centos6 -n master_db -f /data/master_db_sys.img


virsh setmaxmem vm192168122253 12G --config
virsh setmem vm192168122253 12G --config
virsh setvcpus  vm192168122253 8 --maximum --config
virsh setvcpus vm192168122253 8 --config

virsh dominfo vm192168122253


cat <<EOF > NewStorage.xml
<disk type='file' device='disk'>
  <driver name='qemu' type='qcow2' cache='none'/>
  <source file='/var/lib/libvirt/images/vm192168122253_1.img'/>
  <target dev='vdb' bus='sata'/>
</disk>
EOF

virsh attach-device --config vm192168122253 NewStorage.xml
rm -f NewStorage.xml

virsh autostart vm192168122253

!Mwteck.com

7. MySQL

7.1. MySQL57安装

7.1.1. 新增Yum仓库

新增Yum仓库
yum install -y https://dev.mysql.com/get/mysql80-community-release-el7-7.noarch.rpm
导入公钥
rpm --import /etc/pki/rpm-gpg/RPM-GPG-KEY-mysql*
默认禁用MySQL仓库
yum-config-manager --disable mysql-connectors-community | egrep '(\[mysql-connectors-community\])|enabled'
yum-config-manager --disable mysql-tools-community | egrep '(\[mysql-tools-community\])|enabled'
yum-config-manager --disable mysql80-community | egrep '(\[mysql80-community\])|enabled'
按需单独启用MySQL仓库

7.1.2. 安装MySQL57

yum --enablerepo=mysql57-community install -y mysql-community-server

7.1.3. 初始化MySQL57

设置日志
mkdir -p /var/log/mysqld
touch /var/log/mysqld/error.log
chown -R mysql:mysql /var/log/mysqld

crudini --set --existing /etc/my.cnf mysqld log-error /var/log/mysqld/error.log
设置MySQL数据目录
mkdir -p /data/mysql

crudini --set --existing /etc/my.cnf mysqld datadir /data/mysql

7.1.4. 配置MySQL57

crudini --set /etc/my.cnf mysqld default-storage-engine InnoDB
crudini --set /etc/my.cnf mysqld disabled_storage_engines '"MyISAM"'

crudini --set /etc/my.cnf mysqld bind-address 0.0.0.0
crudini --set /etc/my.cnf mysqld max_connections 1000

crudini --set /etc/my.cnf mysqld general_log OFF
crudini --set /etc/my.cnf mysqld general_log_file /var/log/mysqld/general.log

crudini --set /etc/my.cnf mysqld long_query_time 3
crudini --set /etc/my.cnf mysqld slow_query_log ON
crudini --set /etc/my.cnf mysqld slow_query_log_file /var/log/mysqld/slow_query.log

# 开启兼容模式,兼容老MySQL代码,比如使用空字符串代替NULL插入数据
crudini --set /etc/my.cnf mysqld sql_mode '""'

crudini --set /etc/my.cnf mysqld skip-name-resolve 'OFF'

crudini --set /etc/my.cnf mysqldump max_allowed_packet 100M
echo "quick" >> /etc/my.cnf
echo "quote-names" >> /etc/my.cnf
开机启动MySQL
systemctl enable mysqld
启动MySQL服务
systemctl start mysqld
查看MySQL服务状态
systemctl status mysqld
终端输出
● mysqld.service - MySQL Server
   Loaded: loaded (/usr/lib/systemd/system/mysqld.service; enabled; vendor preset: disabled)
   Active: active (running) since Fri 2023-09-15 22:11:05 CST; 9s ago
     Docs: man:mysqld(8)
           http://dev.mysql.com/doc/refman/en/using-systemd.html
  Process: 28577 ExecStart=/usr/sbin/mysqld --daemonize --pid-file=/var/run/mysqld/mysqld.pid $MYSQLD_OPTS (code=exited, status=0/SUCCESS)
  Process: 28540 ExecStartPre=/usr/bin/mysqld_pre_systemd (code=exited, status=0/SUCCESS)
 Main PID: 28580 (mysqld)
   CGroup: /system.slice/mysqld.service
           └─28580 /usr/sbin/mysqld --daemonize --pid-file=/var/run/mysqld/mysqld.pid

Sep 15 22:11:04 gitlab_a_test systemd[1]: Starting MySQL Server...
Sep 15 22:11:05 gitlab_a_test systemd[1]: Started MySQL Server.

7.1.5. 修改密码

临时密码有不常用的特殊字符,不便日常管理。不降低安全性的前提性,更改MySQL密码

MYSQL_TMP_ROOT_PASSWORD=$(grep 'A temporary password' /var/log/mysqld/error.log | tail -n 1 | awk '{print $NF}')

export BY_MYSQL_ROOT_PASSWORD=$(pwgen -csnB 10)_$(pwgen -csnB 10)
# 永久保存临时配置(重新登录或重启都有效)
sed -i '/export BY_/d' ~/.bash_profile && env | grep BY_ | awk '{print "export "$1}' >> ~/.bash_profile

echo -e "  MySQL用户名:root\nMySQL临时密码:${MYSQL_TMP_ROOT_PASSWORD}\n  MySQL新密码:${BY_MYSQL_ROOT_PASSWORD}"

mysqladmin -uroot -p"${MYSQL_TMP_ROOT_PASSWORD}" password ${BY_MYSQL_ROOT_PASSWORD}
终端输出
  MySQL用户名:root
MySQL临时密码:ABCpt*aQ9:k4
  MySQL新密码:sSU9vdfRUP_i7nLFrcYFF

7.1.6. 本机无密码登录设置

脚本无人化配置(自动输入密码)
  1. 执行脚本

    unbuffer expect -c "
    spawn mysql_config_editor set --skip-warn --login-path=client --host=localhost --user=root --password
    expect -nocase \"Enter password:\" {send \"${BY_MYSQL_ROOT_PASSWORD}\n\"; interact}
    "
    终端输出
    spawn mysql_config_editor set --skip-warn --login-path=client --host=localhost --user=root --password (1)
    Enter password:
    1 注意 --login-path 参数值为 client,表示可以这样(mysql --login-path=client)无密码执行MySQL命令
  2. 查看MySQL无密码配置清单

    mysql_config_editor print --all
    终端输出
    [client] (1)
    user = "root"
    password = *****
    host = "localhost"
    1 此处的名称必须与前文的 --login-path=client 一致,都是 client
无密码登录测试

现在,可以不带用户名称及密码在终端执行 mysql 命令了。比如,查看数据库列表:

mysql -e "show databases;"
终端输出
+--------------------+
| Database           |
+--------------------+
| information_schema |
| mysql              |
| performance_schema |
| sys                |
+--------------------+

--login-path 值为 client 时,可以省略此参数。 不省略时,完整命令如下:

mysql --login-path=client -e "show databases;"

7.1.7. 卸载密码插件

默认的密码复杂度要求太高,卸载之:

mysql -e "uninstall plugin validate_password;"

7.2. MySQL57主从

采用冷复制方式

7.2.1. 配置文件

公共配置
mkdir -p /data/mysql_data_log
chown -R mysql:mysql /data/mysql_data_log

crudini --set /etc/my.cnf mysqld expire_logs_days 30
crudini --set /etc/my.cnf mysqld max_binlog_size 1GB
crudini --set /etc/my.cnf mysqld log-bin /data/mysql_data_log/mysql-bin
crudini --set /etc/my.cnf mysqld log_bin_index /data/mysql_data_log/mysql-bin.index
crudini --set /etc/my.cnf mysqld relay-log /data/mysql_data_log/relay-log
crudini --set /etc/my.cnf mysqld relay-log-index /data/mysql_data_log/relay-log-index
crudini --set /etc/my.cnf mysqld relay-log-info-file /data/mysql_data_log/relay-log.info

#强烈建议,其它格式可能造成数据不一致
crudini --set /etc/my.cnf mysqld binlog_format row

crudini --set /etc/my.cnf mysqld gtid-mode on
crudini --set /etc/my.cnf mysqld enforce_gtid_consistency on

crudini --set /etc/my.cnf mysqld default-storage-engine InnoDB
crudini --set /etc/my.cnf mysqld disabled_storage_engines '"MyISAM"'

crudini --set /etc/my.cnf mysqldump max_allowed_packet 100M
echo 'quick' >> /etc/my.cnf
echo 'quote-names' >> /etc/my.cnf
主的配置
crudini --set /etc/my.cnf mysqld master-verify-checksum on
crudini --set /etc/my.cnf mysqld server_id 1

# 主可读可写
crudini --set /etc/my.cnf mysqld read_only off
crudini --set /etc/my.cnf mysqld super_read_only off
从的配置
crudini --set /etc/my.cnf mysqld slave-sql-verify-checksum on
crudini --set /etc/my.cnf mysqld skip-slave-start on
crudini --set /etc/my.cnf mysqld server_id 2

# 从可读不可写
crudini --set /etc/my.cnf mysqld read_only on
crudini --set /etc/my.cnf mysqld super_read_only on

# 从写bin log,做实时备份需要
crudini --set /etc/my.cnf mysqld log-slave-updates on

7.2.2. 主上的操作

mysql -e "update mysql.user set host='%'  where user='root';flush privileges;"
mysql -e "CREATE USER 'repl'@'%' IDENTIFIED BY 'Wielae7eg6Ae9ebayiec';"
mysql -e "GRANT REPLICATION SLAVE ON *.* TO 'repl'@'%' WITH GRANT OPTION;"
mysql -e "show master status;"

systemctl stop mysqld

tar zcf /tmp/mysql.tar.gz -C /data mysql

systemct start mysqld

cd /tmp && python3 -m http.server
将MySQL数据目录打包压缩,上传至从服务器解压,将其作为从服务器的数据目录

7.2.3. 从上的操作

wget x.x.x.x:8000/mysql.tar.gz -O mysql.tar.gz
rm -r /data/mysql
tar xf mysql.tar.gz -C /data

echo -e "[auto]\nserver-uuid=$(uuidgen)" > /data/mysql/auto.cnf
rm -f /data/mysql/*.log

systemctl start mysqld

# 因为是冷复制 master_auto_position=1
mysql -e "change master to master_port=3306, master_host='x.x.x.x', master_user='repl', master_password='Wielae7eg6Ae9ebayiec', master_auto_position=1;"

mysql -e "start slave;"

mysql -e "show slave status\G;"
屏幕输出
*************************** 1. row ***************************
               Slave_IO_State: Waiting for master to send event
                  Master_Host: x.x.x.x
                  Master_User: repl
                  Master_Port: 3306
                Connect_Retry: 60
              Master_Log_File: mysql-bin.000005
          Read_Master_Log_Pos: 353
               Relay_Log_File: relay-log.000004
                Relay_Log_Pos: 454
        Relay_Master_Log_File: mysql-bin.000005
             Slave_IO_Running: Yes (1)
            Slave_SQL_Running: Yes (2)
              Replicate_Do_DB:
          Replicate_Ignore_DB:
           Replicate_Do_Table:
       Replicate_Ignore_Table:
      Replicate_Wild_Do_Table:
  Replicate_Wild_Ignore_Table:
                   Last_Errno: 0
                   Last_Error:
                 Skip_Counter: 0
          Exec_Master_Log_Pos: 353
              Relay_Log_Space: 695
              Until_Condition: None
               Until_Log_File:
                Until_Log_Pos: 0
           Master_SSL_Allowed: No
           Master_SSL_CA_File:
           Master_SSL_CA_Path:
              Master_SSL_Cert:
            Master_SSL_Cipher:
               Master_SSL_Key:
        Seconds_Behind_Master: 0
Master_SSL_Verify_Server_Cert: No
                Last_IO_Errno: 0
                Last_IO_Error:
               Last_SQL_Errno: 0
               Last_SQL_Error:
  Replicate_Ignore_Server_Ids:
             Master_Server_Id: 1
                  Master_UUID: 2a7f13cc-76fb-11ee-8d0e-fa163e0b63c7
             Master_Info_File: /data/mysql/master.info
                    SQL_Delay: 0
          SQL_Remaining_Delay: NULL
      Slave_SQL_Running_State: Slave has read all relay log; waiting for more updates (3)
           Master_Retry_Count: 86400
                  Master_Bind:
      Last_IO_Error_Timestamp:
     Last_SQL_Error_Timestamp:
               Master_SSL_Crl:
           Master_SSL_Crlpath:
           Retrieved_Gtid_Set: 2a7f13cc-76fb-11ee-8d0e-fa163e0b63c7:7
            Executed_Gtid_Set: 2a7f13cc-76fb-11ee-8d0e-fa163e0b63c7:1-7,
fc414685-9b00-4294-a441-750fabb9d141:1-2
                Auto_Position: 1
         Replicate_Rewrite_DB:
                 Channel_Name:
           Master_TLS_Version:
1 主从正在运行
2 主从正在运行
3 主从状态正常,无错误

7.3. 常用命令

7.3.1. MySQL 8.0重置密码

--skip-grant-tables 参数启动MySQL服务
/usr/libexec/mysqld --basedir=/usr --user=mysql --skip-grant-tables
空密码直接登录MySQL
mysql
执行MySQL指令
select host, user, authentication_string from mysql.user;

# 刷新权限后,才能使用password指令
flush privileges;

set password for 'root'@'localhost'='geek';

select host, user, authentication_string from mysql.user;

exit
测试新密码
mysql -uroot -pgeek -e 'show databases;'
重启MySQL

移除 --skip-grant-tables 参数,重启MySQL服务

7.3.2. 卸载密码插件

MySQL5.7+以上的版本安装后必须先改密码,才能进行其它操作:

# 设置临时密码
mysqladmin -uroot -p'旧密码' password "1qaz@2wsX"

# MySQL 5.7
mysql -uroot -p'1qaz@2wsX' -e "uninstall plugin validate_password;"
# MySQL 8.0
mysql -uroot -p'1qaz@2wsX' -e "UNINSTALL COMPONENT 'file://component_validate_password';"

# 设置新密码
mysqladmin -uroot -p'1qaz@2wsX' password '新密码'

7.3.3. 创建数据库时指定字符集

CREATE DATABASE db_name DEFAULT CHARACTER SET utf8mb4 DEFAULT COLLATE utf8mb4_unicode_ci;

7.3.4. 新建或修改密码、开放指定HOST登录

MySQL 5.7及以上,开放指定HOST并设置用户名和密码
CREATE USER 'dummy'@'localhost';

CREATE USER 'finley'@'localhost' IDENTIFIED BY 'password';

GRANT ALL ON *.* TO 'finley'@'localhost' WITH GRANT OPTION;
MySQL 5.7及以上,单独开放指定HOST登录
use mysql
update user set host = '%'  where user ='root';
flush privileges;
MySQL 5.7及以上,修改用户密码
use mysql
ALTER USER 'root'@'%' IDENTIFIED BY 'geek';
flush privileges;
MySQL 5.7以下
use mysql
grant all privileges on *.*  to root@"172.17.%.%" identified by "geek";
update user set Grant_priv='Y' where User='root' and Host='172.17.%.%';
flush privileges;

7.3.5. 本机无密码登录

mysql_config_editor set --login-path=client --host=localhost --user=root --password
mysql_config_editor print
mysql --login-path=client db_name
mysql db_name

7.3.6. 查看字符集

utf8mb4_unicode_ci 是用UNICODE排序,不是中文!

需要中文可以考虑用 utf8mb4_zh_0900_as_cs

修改表排序规则:

ALTER TABLE project CONVERT TO CHARACTER SET utf8mb4 COLLATE utf8mb4_zh_0900_as_cs;

查看表排序规则:

show create table project;

参考:

https://dev.mysql.com/doc/refman/8.0/en/charset-unicode-sets.html
查看数据库字符集1
mysql> show variables like 'collation%';
+----------------------+--------------------+
| Variable_name        | Value              |
+----------------------+--------------------+
| collation_connection | utf8mb4_unicode_ci |
| collation_database   | utf8mb4_unicode_ci |
| collation_server     | utf8mb4_unicode_ci |
+----------------------+--------------------+
3 rows in set (0.00 sec)

mysql> show variables like 'character%';
+--------------------------+----------------------------+
| Variable_name            | Value                      |
+--------------------------+----------------------------+
| character_set_client     | utf8mb4                    |
| character_set_connection | utf8mb4                    |
| character_set_database   | utf8mb4                    |
| character_set_filesystem | binary                     |
| character_set_results    | utf8mb4                    |
| character_set_server     | utf8mb4                    |
| character_set_system     | utf8                       |
| character_sets_dir       | /usr/share/mysql/charsets/ |
+--------------------------+----------------------------+
8 rows in set (0.00 sec)
查看数据库字符集2
mysql> show create database ghibli;
+----------+----------------------------------------------------------------------------------------------------------------------------------+
| Database | Create Database                                                                                                                  |
+----------+----------------------------------------------------------------------------------------------------------------------------------+
| ghibli   | CREATE DATABASE `ghibli` /*!40100 DEFAULT CHARACTER SET utf8mb4 COLLATE utf8mb4_unicode_ci */ /*!80016 DEFAULT ENCRYPTION='N' */ |
+----------+----------------------------------------------------------------------------------------------------------------------------------+
1 row in set (0.00 sec)
查看表字符集
mysql> show create table pre_ucenter_memberfields;
Current database: db_name

+-----------------------------------------------------------------+
| Table                    | Create Table|
+-----------------------------------------------------------------+
| pre_ucenter_memberfields | CREATE TABLE `pre_ucenter_memberfields` (
  `uid` mediumint(8) unsigned NOT NULL DEFAULT '0',
  `blacklist` text NOT NULL,
  PRIMARY KEY (`uid`)
) ENGINE=InnoDB DEFAULT CHARSET=gbk |
+-----------------------------------------------------------------+
1 row in set (0.98 sec)

7.3.7. 修改表字符集

修改表字符集
ALTER TABLE table_name CONVERT TO CHARACTER SET utf8mb4 COLLATE utf8mb4_unicode_ci;
修改数据库字符集
ALTER DATABASE db_name CHARACTER SET = utf8mb4 COLLATE = utf8mb4_unicode_ci;

7.3.8. 备份和还原

# 导出数据库
mysqldump db_name > db_name_dump.sql

# 导出数据库并压缩
mysqldump db_name | gzip > db_name_$(date +%Y_%m_%d_%H_%M_%S).sql.gz

# 导出不带create table的表数据,每行一个INSERT INTO
mysqldump db_name table_name --no-create-info --extended-insert=false

# 每行一个INSERT INTO,替换成INSERT IGNORE INTO,用于合并同表的数据
# 如果表ID存在,则跳过插入
mysqldump db_name table_name --no-create-info --extended-insert=false | grep -E '(INSERT INTO)|LOCK' |sed 's/INSERT INTO /INSERT IGNORE INTO /'

# 导入SQL文件
mysql db_name < db_name_dump.sql

7.3.9. 临时启用性能调优

set profiling=on;
show profiles;
show profile for query query_id;

7.3.10. 查看全局变量

# 开启SQL日志
#
set global general_log = ON;
set gloabl general_log_file = /var/lib/mysql/hostname.log;
show global variables like '%general_log%';

# 开启慢查询
set global long_query_time = 10;
set global slow_query_log = ON;
set gloabl slow_query_log_file = /var/lib/mysql/hostname-slow.log;
show global variables like '%_query_%';
my.cnf
[mysqld]
general_log = ON
general_log_file = /var/lib/mysql/general.log

long-query-time = 10
slow_query_log = ON
slow_query_log_file = /var/lib/mysql/slow.log

7.4. 索引

7.4.1. 查看索引

SHOW INDEX FROM TBLNAME;

7.4.2. 创建索引

CREATE INDEX ev_info_createBy_IDX USING BTREE ON ttd.ev_info (createBy);

7.4.3. 删除索引

DROP INDEX index_name ON talbe_name;

8. ArchLinux

8.1. ArchLinux 安装

Arch Linux安装可以在启动镜像之后,通过SSH连接复制以下命令字符完成安装。步骤如下:

  1. ip a 查看IP地址;

  2. systemctl start sshd 启动SSH服务;

  3. passwd 修改启动镜像root用户密码;

  4. 使用任意SSH客户端软件,连接启动镜像IP地址的22端口即可。

如果是安装中途连接SSH的,请注意适时执行 arch-chroot /mnt 命令进入安装目标硬盘。否则,所有安装操作重启系统后无效(安装到了启动镜像中)。

8.1.1. 建立硬盘分区

cfdisk
按照本教程安装ArchLinux ,硬盘至少需要10G以上。
如果使用UEFI启动方式,需要新建 /dev/sda1 容量200M,并且设置为启动分区,其余分区编号顺延。

创建主分区,此处以7G为例,实际情况至少10G:

archlinux install 01
archlinux install 02
archlinux install 03
archlinux install 04
archlinux install 05

创建交换分区,容量1G:

archlinux install 06
archlinux install 07
archlinux install 08
archlinux install 09

保存修改并退出:

archlinux install 10
archlinux install 11
archlinux install 12
archlinux install 13

8.1.2. 格式化分区

格式化主分区:

mkfs.ext4 /dev/sda1

格式化虚拟分区:

mkswap /dev/sda2

swapon /dev/sda2

如果使用UEFI启动方式,并且 /dev/sda1 是容量200M的启动分区,需要执行以下步骤。

pacman -Syy
pacman -S dosfstools
mkfs.fat -F32 /dev/sda1

8.1.3. 挂载分区

mount /dev/sda1 /mnt

如果使用UEFI启动方式,并且 /dev/sda1 是容量200M的启动分区,需要执行以下步骤。

mkdir -p /mnt/boot/efi
mount /dev/sda1 /mnt/boot/efi

8.1.4. 连接WIFI

执行 iwctl 命令,进入iwd交互Shell:

# 获得WIFI网卡名称 wlan0  和 WIFI硬件名称 phy0
device list

device wlan0 set-property Powered on
adapter phy0 set-property Powered on

# 不会有任何屏幕输出
station wlan0 scan

# 列表WIFI清单
station wlan0 get-networks

# 会提示输入连接密码,其中 GEEKCAMP_5G 是选择的WIFI SSID
station wlan0 connect GEEKCAMP_5G

# 查看WIFI连接
station  wlan0 show

# 退出iwd Shell
exit

测试网络连接:

ping qq.com

8.1.5. 设置国内镜像源

cat << EOF > /etc/pacman.d/mirrorlist
Server = https://mirrors.163.com/archlinux/\$repo/os/\$arch
EOF

8.1.6. 安装基础包

pacstrap /mnt base

8.1.7. 开机挂载分区

genfstab -U /mnt >> /mnt/etc/fstab

编辑 /mnt/etc/fstab 文件,确认分区信息无误。

8.1.8. 切换到全新 Arch Linux 系统

arch-chroot /mnt

8.1.9. 设置国内镜像源

cat << EOF > /etc/pacman.d/mirrorlist
Server = https://mirrors.163.com/archlinux/\$repo/os/\$arch
EOF

8.1.10. 安装vim

pacman -S vim

8.1.11. 设置时区

ln -sf /usr/share/zoneinfo/Asia/Shanghai /etc/localtime

8.1.12. 本地化语言支持

启用语言参数:

echo "en_US.UTF-8 UTF-8" >> /etc/locale.gen
echo "zh_CN.GB18030 GB18030" >> /etc/locale.gen
echo "zh_CN.GBK GBK" >> /etc/locale.gen
echo "zh_CN.UTF-8 UTF-8" >> /etc/locale.gen
echo "zh_CN GB2312" >> /etc/locale.gen

生成locale:

locale-gen

设置系统默认语言:

echo "LANG=en_US.UTF-8" > /etc/locale.conf

全局设置为英文,tty控制台不会乱码,新用户默认目录也是英文名称,方便使用。

新用户登录桌面后,自行找到控制面板中的 “Region and Language”(区域和语言)设置为 汉语 即可。设置后,如果提示更新目录名称为中文,请选择 “保留旧的文件名”,除非你想在终端经常打中文目录名称(累死你~~)。

8.1.13. 键盘布局

echo "KEYMAP=us" > /etc/vconsole.conf

8.1.14. 设置主机名

echo 'archlinux' > /etc/hostname

8.1.15. 本地网络配置

echo '127.0.0.1 localhost' > /etc/hosts
# 添加主机名对应的设置
echo '127.0.0.1 archlinux' >> /etc/hosts

8.1.16. 设置root密码

passwd

8.1.17. 创建用户

新增用户:

useradd -m 你的用户名

设置用户密码:

passwd 你的用户名

添加到sudo列表:

pacman -S sudo

echo '你的用户名   ALL=(ALL:ALL) NOPASSWD: ALL' >> /etc/sudoers

8.1.18. 安装Grub引导

Windows+Linux双引导
pacman -S grub os-prober ntfs-3g
sed -i 's/#GRUB_DISABLE_OS_PROBER=false/GRUB_DISABLE_OS_PROBER=false/g' /etc/default/grub

下次进入GNOME桌面后,执行 grub-mkconfig -o …​ 命令就会自动将Windows系统加入到Grub启动菜单

使用传统方式启动的BIOS
pacman -S grub

grub-install --target=i386-pc --recheck /dev/sda

pacman -S linux linux-headers linux-lts linux-lts-headers

grub-mkconfig -o /boot/grub/grub.cfg
使用UEFI启动的BIOS

对于使用 UEFI 启动的用户,请用以下方式安装和设置Grub引导:

pacman -S grub efibootmgr

grub-install --target=x86_64-efi --efi-directory=/boot/efi --bootloader-id=arch

pacman -S linux linux-headers linux-lts linux-lts-headers

grub-mkconfig -o /boot/grub/grub.cfg

8.1.19. 安装桌面

# 更新软件包索引
pacman -Syy

# 桌面环境
pacman -S gnome vim networkmanager

# WIFI驱动
pacman -S linux-firmware

# 拼音输入法
pacman -S ibus-sunpinyin sunpinyin sunpinyin-data

# 显卡驱动
pacman -S xf86-video-fbdev xf86-video-intel xf86-video-vesa xf86-video-ati xf86-video-amdgpu

# 汉字字体
pacman -S wqy-microhei wqy-zenhei

# 开机启动
systemctl enable NetworkManager
systemctl enable gdm

8.1.20. 重启

exit

reboot

安装完成,重启进入Arch Linux系统。

8.1.21. 设置pacman

增加 Arch Linux 中文社区仓库 的腾讯镜像服务器:

# 进入root
sudo -i

cat << EOF >> /etc/pacman.conf

[archlinuxcn]
Server = https://mirrors.cloud.tencent.com/archlinuxcn/\$arch
SigLevel = Optional TrustAll

EOF

exit

Arch Linux 中文社区仓库 包含深度QQ、深度TIM、WPS、Google Chrome浏览器、WPS等等不开源的软件。

8.1.22. 安装常用软件

在桌面打开终端,复制以下命令一次性安装:

sudo pacman -Syy

sudo pacman -S archlinuxcn-keyring

sudo pacman -S gedit vim screen thunderbird thunderbird-i18n-zh-cn openssh bash-completion cmake git curl wget filezilla gcc make mlocate nginx ntp p7zip rsync virtualbox virtualbox-guest-iso virtualbox-host-dkms file-roller parted sshpass rdesktop qt5-base qt6-base fakeroot yay openssl wireshark-qt base-devel code gnome-terminal os-prober

yay -S google-chrome

8.1.23. 系统设置

在桌面打开终端,执行以下命令:

# GNOME 桌面设置
gsettings set org.gnome.nautilus.preferences always-use-location-entry true
gsettings set org.gnome.nautilus.preferences default-sort-order name
gsettings set org.gtk.Settings.FileChooser sort-directories-first true
gsettings set org.gnome.desktop.interface clock-show-date true
gsettings set org.gnome.desktop.interface clock-show-seconds true
# Ctrl+Shift+Alt+R 录像时,30秒后自动结束。设置为0,不限制
gsettings set org.gnome.settings-daemon.plugins.media-keys max-screencast-length 0
# 禁用最近文件访问记录
gsettings set org.gnome.desktop.privacy remember-recent-files false

# virtualbox 设置
sudo gpasswd -a root vboxusers
sudo gpasswd -a $USER vboxusers
# wireshark 设置
sudo gpasswd -a root wireshark
sudo gpasswd -a $USER wireshark

# 系统日志
sudo gpasswd -a $USER adm
sudo gpasswd -a $USER systemd-journal
sudo gpasswd -a $USER wheel

# docker
#sudo gpasswd -a $USER docker

sudo grpunconv

# 开机启动
sudo systemctl enable systemd-timesyncd
sudo systemctl start systemd-timesyncd
sudo systemctl enable sshd
sudo systemctl mask tmp.mount
默认不能使用 root 用户 SSH 连接 Arch Linux。请修改 /etc/ssh/sshd_config 中的 PermitRootLoginyes
gedit 编辑器设置

安装设置编辑器:

sudo pacman -S dconf-editor

然后,在终端执行 dconf-editor 命令后,依次找到菜单 org.gnome.gedit.preferences.editor → scheme → cobalt,可以设置编辑器背景。

org.gnome.gedit.preferences.editor 中有许多其它设置,根据自己习惯自行设置。

8.1.24. FAQ

E495
  • 禁用安全启动;

  • 进入BIOS,设置:

    • "Startup""UEFI/Legacy Boot""UEFI Only"

    • "Startup""UEFI/Legacy Boot""CMS Support""yes"

如果不设置,安装Linux后显卡硬件不会被识别。在日志文件 "Xorg.0.log" 中会出现 "(EE) open /dev/dri/card0: No such file or directory" 的错误,无法进入桌面。

A485显卡驱动无法加载,导致桌面问题
  • 禁用安全启动;

  • 启用 amdgpu 驱动;

sudo -i

cat << EOF > /etc/modprobe.d/amdgpu.conf
options amdgpu si_support=1
options amdgpu cik_support=1
EOF

cat << EOF > /etc/modprobe.d/radeon.conf
options radeon si_support=0
options radeon cik_support=0
EOF

mkinitcpio -P

重启后,验证:

lspci -k | grep -A 3 -E "(VGA|3D)"

屏幕输出
06:00.0 VGA compatible controller: Advanced Micro Devices, Inc. [AMD/ATI] Raven Ridge [Radeon Vega Series / Radeon Vega Mobile Series] (rev d1)
	Subsystem: Lenovo Raven Ridge [Radeon Vega Series / Radeon Vega Mobile Series]
	Kernel driver in use: amdgpu
	Kernel modules: amdgpu

其中,Kernel driver in use: amdgpu 表示设置已经生效。

8.2. pacman

8.2.1. 用法

安装包
安装指定的包

安装单个包或多个包以及包依赖,执行命令:

pacman -S package_name1 package_name2 ...

有时会出现一个包有多个版本,来自不同的仓库(比如,extratesting)。 以下命令会安装 extra 仓库的包,仓库名称放在包名之前:

pacman -S extra/package_name
安装包组

安装包组下的所有包:

pacman -S gnome

查看 gnome 组下的包:

pacman -Sg gnome
移除包

移除包,但不移除相关依赖包:

pacman -R package_name

移除包及其依赖包:

pacman -Rs package_name
升级包

有时仅仅需要更新软件索引包,使用 Syy

pacman -Syy

pacman 能够使用一个命令升级系统上的所有包。根据系统上次更新时间不同,升级的时间也有所不同。 下面这个命令会同步仓库数据库并更新包。

pacman -Syu
查询包数据库

pacman 使用 -Q 参数查询本地包数据库,使用 -S 参数同步远端数据库,使用 -F 参数同步文件列表数据库。 使用 pacman -Q --helppacman -S --helppacman -F --help 命令可以查看更多信息。

pacman 可以搜索数据库中的包信息,包括包名称和描述:

pacman -Ss string1 string2 ...

搜索已经安装的包:

pacman -Qs string1 string2 ...

根据文件名搜索远端仓库:

pacman -F string1 string2 ...

显示指定包的概要信息:

pacman -Si package_name

显示已经安装的包信息:

pacman -Qi package_name

打印本地包的文件列表:

pacman -Ql package_name

从远端仓库获取包的文件列表:

pacman -Fl package_name

检查本地包文件是否存在:

pacman -Qk package_name
传递两个 k 参数将会执行更多检查。

查询文件所属本地包:

pacman -Qo /path/to/file_name

从远程仓库查询文件所属包:

pacman -F /path/to/file_name
清除包缓存
pacman -Scc
其它命令

仅下载包:

pacman -Sw package_name

从本地文件系统安装包:

pacman -U /path/to/package/package_name-version.pkg.tar.xz
Search for a package that contains a specific file

同步文件数据库:

pacman -Fy

搜索包里面的一个文件,比如:

$ pacman -F pacman
core/pacman 5.2.1-1 (base base-devel) [installed]
    usr/bin/pacman
    usr/share/bash-completion/completions/pacman
extra/xscreensaver 5.43-1
    usr/lib/xscreensaver/pacman

更多高级用法,请安装 pkgfile

8.2.2. Troubleshooting

Signature from "User <email@example.org>" is unknown trust, installation failed

可选方案:

方法一

更新已有密钥:pacman-key --refresh-keys

方法二

手动升级 archlinux-keyring 包:pacman -Sy archlinux-keyring && pacman -Su

方法三

重置所有密钥

  1. 删除 /etc/pacman.d/gnupg 文件夹

  2. pacman-key --init

  3. pacman-key --populate archlinux

8.3. Wireshark

8.3.1. 以普通用户身份抓包

默认情况下,必须使用 sudo -i 等命令切换到 root 用户才能获取到本地接口列表,然后抓包。

如需要使用普通用户抓包,必须将用户加入 wireshark 群组:

gpasswd -a $USERNAME wireshark

重启后生效。

8.4. Sudo

8.4.1. sudo command doesn’t source ~/.bashrc

bashrc is a configuration file of bash, only when it’s executed interactively. It’s only loaded when you start bash, not when you run some other program such as sh (not even if bash is invoked via the name sh). And it’s only loaded when bash is interactive, not when it’s executing a script or a command with -c.

sudo sh -c 'echo $PATH' or sudo bash -c 'echo $PATH' doesn’t invoke an interactive shell, so .bashrc is not involved.

sudo su; echo $PATH runs an interactive instance of root’s shell. If that’s bash, then ~root/.bashrc is loaded. This snippet executes echo $PATH once this interactive shell terminates, so whatever happens in the interactive shell has no influence on what the snippet prints at the end. But if you type echo $PATH at the prompt of the interactive shell started by sudo su, you will see the value set by ~root/.bashrc.

Since .bashrc is invoked in each interactive shell, not by login shells (not even by interactive login shells, which is a design defect in bash), it’s the wrong place to define environment variables. Use .bashrc for interactive bash settings such as key bindings, aliases and completion settings. Set environment variables in files that are loaded when you log in: ~/.pam_environment or ~/.profile.

So set PATH in .profile instead of .bashrc, and either run a login shell with sudo -i 'echo $PATH', or explicitly source .profile with sudo sh -c '. ~/.profile; echo $PATH'.

8.5. 降级包

8.5.1. MySQL 启动失败

分析过程
  1. sudo systemctl status mysqld 无具体错误

  2. journalctl -xe 中没有任何更加详细的 MySQL 错误信息,仅仅提示启动失败

通过分析发现,可能是 MySQL 的依赖出问题导致。使用 ldd /usr/bin/mysqld 查看确认。

$ ldd /usr/bin/mysqld
	linux-vdso.so.1 (0x00007fff4dd95000)
	libjemalloc.so.2 => /usr/lib/libjemalloc.so.2 (0x00007fd61d6d0000)
	libpthread.so.0 => /usr/lib/libpthread.so.0 (0x00007fd61d6ae000)
	libicuuc.so.64 => /usr/lib/libicuuc.so.64 (0x00007fd61d4d6000)
	libicui18n.so.64 => /usr/lib/libicui18n.so.64 (0x00007fd61d1e1000)
	libevent_core-2.1.so.7 => /usr/lib/libevent_core-2.1.so.7 (0x00007fd61d1a8000)
	libprotobuf-lite.so.18 => not found
	librt.so.1 => /usr/lib/librt.so.1 (0x00007fd61d0fe000)

从以上可以看到缺少 libprotobuf-lite.so.18 库文件。

查询库文件所属包

$ pkgfile libprotobuf-lite.so
extra/protobuf

或者

$ sudo pacman -Fy libprotobuf-lite.so
:: 正在同步软件包数据库...
 core 已经是最新版本
 extra 已经是最新版本
 community 已经是最新版本
 multilib 已经是最新版本
 archlinuxcn 已经是最新版本
extra/protobuf 3.10.0-1 [已安装: 3.10.0-1]
    usr/lib/libprotobuf-lite.so

查询库文件最新版本号

$ sudo pacman -Ql protobuf|grep libprotobuf-lite.so
protobuf /usr/lib/libprotobuf-lite.so
protobuf /usr/lib/libprotobuf-lite.so.21
protobuf /usr/lib/libprotobuf-lite.so.21.0.0

或者

$ ldconfig -p|grep libprotobuf-lite.so
	libprotobuf-lite.so.21 (libc6,x86-64) => /usr/lib/libprotobuf-lite.so.21
	libprotobuf-lite.so (libc6,x86-64) => /usr/lib/libprotobuf-lite.so

至此,可以确认是由于 protobuf 包更新导致 MySQL 依赖出现问题。紧急修复的方法只有先降级 protobuf 包。

8.5.2. 降级 protobuf

rm -f protobuf-3.7.0-1-x86_64.pkg.tar.xz
wget https://archive.archlinux.org/packages/p/protobuf/protobuf-3.7.0-1-x86_64.pkg.tar.xz
sudo pacman -U protobuf-3.7.0-1-x86_64.pkg.tar.xz
确认 MySQL 依赖
$ ldconfig -p|grep libprotobuf-lite.so
	libprotobuf-lite.so.18 (libc6,x86-64) => /usr/lib/libprotobuf-lite.so.18
	libprotobuf-lite.so (libc6,x86-64) => /usr/lib/libprotobuf-lite.so

$ ldd /usr/bin/mysqld|grep libprotobuf-lite.so
	libprotobuf-lite.so.18 => /usr/lib/libprotobuf-lite.so.18 (0x00007f209f7b2000)

成功搞定。

没有更快确认 libprotobuf-lite.so.18 库文件属于哪个版本的 protobuf。只有挨个从

下载最近几个版本的 protobuf-3.x.x-x-x86_64.pkg.tar.xz 包测试。

8.5.3. 忽略 protobuf 包的后续更新

/etc/pacman.conf 搜索 IgnorePkg 修改为:

IgnorePkg = protobuf

确认忽略效果
$ sudo pacman -Syu
[sudo] mk 的密码:
:: 正在同步软件包数据库...
 core 已经是最新版本
 extra 已经是最新版本
 community 已经是最新版本
 multilib 已经是最新版本
 archlinuxcn 已经是最新版本
:: 正在进行全面系统更新...
警告:protobuf:忽略软件包升级 (3.7.0-1 => 3.10.0-1)
 今日无事可做

8.6. 视频缩略图

8.6.1. 问题

Nautilus 窗口视频没有缩略图。

8.6.2. 解决方法

rm -rf ~/.cache/thumbnails/fail

sudo pacman -S gst-libav

最后,按 Ctrl-R 刷新 Nautilus 窗口。

8.7. 打印机

8.7.1. 安装配置

systemctl enable cups-browsed
systemctl enable org.cups.cupsd

# 将当前用户加入打印机管理员组,否则无法添加打印机提示 “已禁止”
sed -i "s/^SystemGroup sys root wheel$/SystemGroup sys root wheel $USER/" /etc/cups/cups-files.conf

systemctl restart cups-browsed
systemctl restart org.cups.cupsd

8.7.2. 打印机驱动

惠普打印机(HP Color LaserJet MFP M181fw)
# 安装打印机驱动
pacman -S hplip

# 查看支持的型号
pacman -Ql hplip

8.7.3. 添加打印机

惠普打印机:惠普工具

运行:

hp-toolbox

然后,单击 Setup Device 安装打印机驱动。

Web

访问 http://localhost:631/ ,单击 "Administration""Printer""Add Printer" 添加打印机。

如果提示输入帐号密码,请填写当前登录用户的帐号和密码。

9. XXL-JOB

9.1. 安装配置

9.1.1. 安装依赖

Git
yum install -y git
Maven && Java 8

Maven 包默认依赖 Java 8。以下 Java 将被安装:

  • java-1.8.0-openjdk

  • java-1.8.0-openjdk-devel

  • java-1.8.0-openjdk-headless

安装 Maven

yum install -y maven

查看 Maven 版本

[root@geekcamp ~]# mvn --version
Apache Maven 3.0.5 (Red Hat 3.0.5-17)
Maven home: /usr/share/maven
Java version: 1.8.0_232, vendor: Oracle Corporation
Java home: /usr/lib/jvm/java-1.8.0-openjdk-1.8.0.232.b09-0.el7_7.x86_64/jre
Default locale: en_US, platform encoding: UTF-8
OS name: "linux", version: "3.10.0-1062.el7.x86_64", arch: "amd64", family: "unix"

查看 Java 版本

[root@geekcamp ~]# java -version
openjdk version "1.8.0_232"
OpenJDK Runtime Environment (build 1.8.0_232-b09)
OpenJDK 64-Bit Server VM (build 25.232-b09, mixed mode)

设置 Maven 全局镜像源

mkdir ~/.m2

cat << EOF > ~/.m2/settings.xml
<settings>
  <mirrors>
    <mirror>
      <id>aliyun</id>
      <name>Aliyun Central</name>
        <url>http://maven.aliyun.com/nexus/content/groups/public/</url>
      <mirrorOf>central</mirrorOf>
    </mirror>
  </mirrors>
</settings>
EOF
Java 11(可选)

安装 Java 11:

yum install -y java-11-openjdk java-11-openjdk-devel java-11-openjdk-headless

查看当前默认 Java 版本:

[root@geekcamp ~]# java -version
openjdk version "1.8.0_232"
OpenJDK Runtime Environment (build 1.8.0_232-b09)
OpenJDK 64-Bit Server VM (build 25.232-b09, mixed mode)

[root@geekcamp ~]# alternatives --display java
java - status is auto.
 link currently points to /usr/lib/jvm/java-1.8.0-openjdk-1.8.0.232.b09-0.el7_7.x86_64/jre/bin/java
/usr/lib/jvm/java-1.8.0-openjdk-1.8.0.232.b09-0.el7_7.x86_64/jre/bin/java - family java-1.8.0-openjdk.x86_64 priority 1800232
 slave jre: /usr/lib/jvm/java-1.8.0-openjdk-1.8.0.232.b09-0.el7_7.x86_64/jre
......
Current `best' version is /usr/lib/jvm/java-1.8.0-openjdk-1.8.0.232.b09-0.el7_7.x86_64/jre/bin/java.

设置Java11为默认版本:

alternatives --set java `ls /usr/lib/jvm/java-11-openjdk-*/bin/java`

alternatives --set jre_openjdk `ls -d /usr/lib/jvm/java-11-openjdk-*`

alternatives --set java_sdk_openjdk `ls -d /usr/lib/jvm/java-11-openjdk-*`

查看当前默认 Java 版本:

[root@geekcamp ~]# java -version
openjdk version "11.0.6" 2020-01-14 LTS
OpenJDK Runtime Environment 18.9 (build 11.0.6+10-LTS)
OpenJDK 64-Bit Server VM 18.9 (build 11.0.6+10-LTS, mixed mode, sharing)
MySQL

安装 MySQL

yum install -y mariadb-server mariadb

设置开机启动:

systemctl enable mariadb

启动 MySQL 服务:

systemctl start mariadb

设置密码:

mysqladmin -uroot password "geek"
默认 MySQL 初始化并启动后,可以使用空密码登录。

测试 MySQL

[root@geekcamp ~]# mysql -uroot -pgeek -e "show databases;"
+--------------------+
| Database           |
+--------------------+
| information_schema |
| mysql              |
| performance_schema |
| test               |
+--------------------+

9.1.2. XXL-JOB 安装

构建:

[ ! -d "~/downloads/" ] && mkdir ~/downloads/
cd ~/downloads/
git clone https://github.com/xuxueli/xxl-job.git
cd xxl-job
mvn -T 4 package

安装:

[ ! -d "/data/xxl-job/admin" ] && mkdir -p /data/xxl-job/admin
[ ! -d "/data/xxl-job/executor" ] && mkdir -p /data/xxl-job/executor
[ ! -d "/data/applogs" ] && mkdir -p /data/applogs

echo y | cp xxl-job-admin/target/xxl-job-admin-2.2.0-SNAPSHOT.jar /data/xxl-job/admin
echo y | cp xxl-job-admin/src/main/resources/application.properties /data/xxl-job/admin

echo y | cp xxl-job-executor-samples/xxl-job-executor-sample-springboot/target/xxl-job-executor-sample-springboot-2.2.0-SNAPSHOT.jar /data/xxl-job/executor
echo y | cp xxl-job-executor-samples/xxl-job-executor-sample-springboot/src/main/resources/application.properties /data/xxl-job/executor

确认安装:

[root@geekcamp xxl-job]# find /data/xxl-job/*
/data/xxl-job/admin
/data/xxl-job/admin/xxl-job-admin-2.2.0-SNAPSHOT.jar
/data/xxl-job/admin/application.properties
/data/xxl-job/executor
/data/xxl-job/executor/xxl-job-executor-sample-springboot-2.2.0-SNAPSHOT.jar
/data/xxl-job/executor/application.properties

9.1.3. XXL-JOB 运行前

XXL-JOB 服务端
MySQL配置

创建 xxl_job 数据库:

mysql -uroot -pgeek -e "CREATE DATABASE xxl_job DEFAULT CHARACTER SET utf8mb4 DEFAULT COLLATE utf8mb4_unicode_ci;"

创建 xxl_job 用户:

mysql -uroot -pgeek -e "CREATE USER 'xxl_job'@'localhost' IDENTIFIED BY 'geek';"
mysql -uroot -pgeek -e "GRANT ALL ON xxl_job.* TO 'xxl_job'@'localhost' WITH GRANT OPTION;"

导入 SQL 文件:

mysql -uxxl_job -pgeek xxl_job < doc/db/tables_xxl_job.sql

查看 xxl_job 表:

[root@geekcamp xxl-job]# mysql -uxxl_job -pgeek xxl_job -e "show tables;"
+--------------------+
| Tables_in_xxl_job  |
+--------------------+
| xxl_job_group      |
| xxl_job_info       |
| xxl_job_lock       |
| xxl_job_log        |
| xxl_job_log_report |
| xxl_job_logglue    |
| xxl_job_registry   |
| xxl_job_user       |
+--------------------+
设置配置文件
sed -i '26s/spring.datasource.username=root/spring.datasource.username=xxl_job/' /data/xxl-job/admin/application.properties

sed -i '27s/spring.datasource.password=root_pwd/spring.datasource.password=geek/' /data/xxl-job/admin/application.properties
创建系统用户
useradd --user-group --no-create-home --shell /sbin/nologin --comment "XXL-JOB" xxl_job
chown -R xxl_job:xxl_job /data/applogs /data/xxl-job
增加 systemd 文件
cat << EOF > /usr/lib/systemd/system/xxl_job_admin.service
[Unit]
Description=XXL-JOB Admin Service
After=network.target remote-fs.target nss-lookup.target

[Service]
Type=simple
Environment="PORT=8080"
Environment="CONFIG_FILE=/data/xxl-job/admin/application.properties"
Environment="JAR_FILE=/data/xxl-job/admin/xxl-job-admin-2.2.0-SNAPSHOT.jar"

User=xxl_job
Group=xxl_job
WorkingDirectory=/data/xxl-job/admin
ExecStart=/usr/bin/java -Dserver.port=\${PORT} -Dspring.config.location=\${CONFIG_FILE} -jar \${JAR_FILE}
SuccessExitStatus=143
StandardOutput=null
StandardError=journal

[Install]
WantedBy=multi-user.target
EOF
XXL-JOB 客户端
创建目录
[ ! -d "/data/xxl-job/executor" ] && mkdir -p /data/xxl-job/executor
[ ! -d "/data/applogs" ] && mkdir -p /data/applogs
设置配置文件
sed -i '9s#xxl.job.admin.addresses=http://127.0.0.1:8080/xxl-job-admin#xxl.job.admin.addresses=http://192.168.2.11:8080/xxl-job-admin#' \
/data/xxl-job/executor/application.properties

appname=xxl-job-executor-s01

sed -i "12s/xxl.job.executor.appname=xxl-job-executor-sample/xxl.job.executor.appname=${appname}/" \
/data/xxl-job/executor/application.properties
按需更改参数

xxl.job.admin.addresses=http://127.0.0.1:8080/xxl-job-admin

创建系统用户
useradd --user-group --no-create-home --shell /sbin/nologin --comment "XXL-JOB" xxl_job
chown -R xxl_job:xxl_job /data/applogs /data/xxl-job/executor
增加 systemd 文件
cat << EOF > /usr/lib/systemd/system/xxl_job_executor.service
[Unit]
Description=XXL-JOB Executor Service
After=network.target remote-fs.target nss-lookup.target

[Service]
Type=simple
Environment="PORT=8081"
Environment="CONFIG_FILE=/data/xxl-job/executor/application.properties"
Environment="JAR_FILE=/data/xxl-job/executor/xxl-job-executor-sample-springboot-2.2.0-SNAPSHOT.jar"

User=xxl_job
Group=xxl_job
WorkingDirectory=/data/xxl-job/executor
ExecStart=/usr/bin/java -Dserver.port=\${PORT} -Dspring.config.location=\${CONFIG_FILE} -jar \${JAR_FILE}
SuccessExitStatus=143
StandardOutput=null
StandardError=journal

[Install]
WantedBy=multi-user.target
EOF

9.1.4. 启动 XXL-JOB 服务端

systemctl enable xxl_job_admin
systemctl start xxl_job_admin

确认运行情况:

[root@geekcamp xxl-job]# systemctl status xxl_job_admin
● xxl_job.service - XXL-JOB Service
   Loaded: loaded (/usr/lib/systemd/system/xxl_job.service; enabled; vendor preset: disabled)
   Active: active (running) since Thu 2020-01-16 13:56:05 CST; 1s ago
 Main PID: 3015 (java)
   CGroup: /system.slice/xxl_job.service
           └─3015 /usr/bin/java -Dserver.port=8080 -Dspring.config.location=/data/xxl-job/application.properties -jar /data/xxl-job/xxl-job-admin-2.2.0-SNA...

Jan 16 13:56:05 geekcamp systemd[1]: Started XXL-JOB Service.

[root@geekcamp xxl-job]# ss -antpl|grep 8080
LISTEN     0      100       [::]:8080                  [::]:*                   users:(("java",pid=2288,fd=21))

最后,浏览器访问 http://localhost:8080/xxl-job-admin

XXL-JOB默认帐号:admin
XXL-JOB默认密码:123456

9.1.5. 启动 XXL-JOB 客户端

systemctl enable xxl_job_executor
systemctl start xxl_job_executor

确认运行情况:

[root@localhost admin]# systemctl status xxl_job_executor
● xxl_job_client.service - XXL-JOB Client
   Loaded: loaded (/usr/lib/systemd/system/xxl_job_client.service; enabled; vendor preset: disabled)
   Active: active (running) since Thu 2020-01-16 18:49:43 CST; 4s ago
 Main PID: 2876 (java)
   CGroup: /system.slice/xxl_job_client.service
           └─2876 /usr/bin/java -Dserver.port=8081 -Dspring.config.location=/data/xxl-job/executor/application.properties -jar /data/xxl-job/executor/xxl-j...

Jan 16 18:49:43 localhost.localdomain systemd[1]: Started XXL-JOB Client.

[root@localhost admin]# ss -antpl|grep 8081
LISTEN     0      100       [::]:8081                  [::]:*                   users:(("java",pid=4679,fd=11))

9.1.6. 工作状态确认

访问 http://192.168.2.11:8080/xxl-job-admin/jobgroup 并登录,确认结果与下图一致:

xxl job admin jobgroup
Figure 1. XXL-JOB 执行器界面

10. LVM

10.1. 增加硬盘

pvcreate  /dev/vdb
vgcreate vg_data /dev/vdb
lvcreate -l 100%free -n data vg_data
mkfs.ext4 /dev/vg_data/data
mkdir /data
echo '/dev/mapper/vg_data-data /data                       ext4    defaults        1 1' >> /etc/fstab
mount -a
df -h
ls /data

10.2. 在线扩容

10.2.1. 创建分区

查看硬盘分区
fdisk -l /dev/vda
屏幕输出
Disk /dev/vda: 100 GiB, 107374182400 bytes, 209715200 sectors
Units: sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disklabel type: dos
Disk identifier: 0x9b35f2fd

Device     Boot Start      End  Sectors Size Id Type
/dev/vda1  *     2048 62912511 62910464  30G 8e Linux LVM
创建分区
fdisk /dev/vda
屏幕输出
Welcome to fdisk (util-linux 2.31.1).
Changes will remain in memory only, until you decide to write them.
Be careful before using the write command.


Command (m for help): n
Partition type
   p   primary (1 primary, 0 extended, 3 free)
   e   extended (container for logical partitions)
Select (default p):

Using default response p.
Partition number (2-4, default 2):
First sector (62912512-209715199, default 62912512):
Last sector, +sectors or +size{K,M,G,T,P} (62912512-209715199, default 209715199):

Created a new partition 2 of type 'Linux' and of size 70 GiB.

Command (m for help): w
The partition table has been altered.
Syncing disks.
确认分区
fdisk -l /dev/vda
屏幕输出
Disk /dev/vda: 100 GiB, 107374182400 bytes, 209715200 sectors
Units: sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disklabel type: dos
Disk identifier: 0x9b35f2fd

Device     Boot    Start       End   Sectors Size Id Type
/dev/vda1  *        2048  62912511  62910464  30G 8e Linux LVM
/dev/vda2       62912512 209715199 146802688  70G 83 Linux

10.2.2. 创建物理卷

创建物理卷
pvcreate /dev/vda2
屏幕输出
  Physical volume "/dev/vda2" successfully created.
确认物理卷
pvs
屏幕输出
  PV         VG      Fmt  Attr PSize   PFree
  /dev/vda1  geek-vg lvm2 a--  <30.00g 12.00m
  /dev/vda2          lvm2 ---   70.00g 70.00g

10.2.3. 扩容卷组

查看卷组
vgs
屏幕输出
  VG      #PV #LV #SN Attr   VSize   VFree
  geek-vg   1   2   0 wz--n- <30.00g 12.00m
扩容卷组
vgextend geek-vg /dev/vda2
屏幕输出
  Volume group "geek-vg" successfully extended
确认卷组
vgs
屏幕输出
  VG      #PV #LV #SN Attr   VSize    VFree
  geek-vg   2   2   0 wz--n- <100.00g 70.01g

10.2.4. 扩容逻辑卷

查看逻辑卷
lvs
屏幕输出
  LV     VG      Attr       LSize   Pool Origin Data%  Meta%  Move Log Cpy%Sync Convert
  root   geek-vg -wi-ao---- 29.03g
  swap_1 geek-vg -wi-ao---- 976.00m
扩容逻辑卷
lvextend -L+70G /dev/geek-vg/root
屏幕输出
  Size of logical volume geek-vg/root changed from 29.03 GiB (7432 extents) to 99.03 GiB (25352 extents).
  Logical volume geek-vg/root successfully resized.
确认逻辑卷
lvs
屏幕输出
  LV     VG      Attr       LSize   Pool Origin Data%  Meta%  Move Log Cpy%Sync Convert
  root   geek-vg -wi-ao---- 99.03g
  swap_1 geek-vg -wi-ao---- 976.00m

10.2.5. 在线扩容分区

查看文件系统
df -Th
屏幕输出
Filesystem                Type      Size  Used Avail Use% Mounted on
udev                      devtmpfs  464M     0  464M   0% /dev
tmpfs                     tmpfs      99M  664K   98M   1% /run
/dev/mapper/geek--vg-root ext4       29G   26G  1.2G  96% /
扩容文件系统
resize2fs /dev/mapper/geek--vg-root
屏幕输出
resize2fs 1.44.1 (24-Mar-2018)
Filesystem at /dev/mapper/geek--vg-root is mounted on /; on-line resizing required
old_desc_blocks = 4, new_desc_blocks = 13
The filesystem on /dev/mapper/geek--vg-root is now 25960448 (4k) blocks long.
确认文件系统
df -Th
屏幕输出
Filesystem                Type      Size  Used Avail Use% Mounted on
udev                      devtmpfs  464M     0  464M   0% /dev
tmpfs                     tmpfs      99M  664K   98M   1% /run
/dev/mapper/geek--vg-root ext4       98G   26G   68G  28% /

如果是XFS分区,使用 xfs_growfs / 在线扩容

11. Wireshark

11.1. 技巧汇总

11.1.1. 捕获过滤器

捕获过滤器 不是 应用显示过滤器

捕获过滤器是工作在抓包过程中,此时并未解包,所有数据都是二进制,不能进行高级过滤筛选。比如不支持 http.request.method == POST

捕获过滤器文档

CaptureFilters

字符串匹配-捕获过滤器 生成器

String-Matching Capture Filter Generator

捕获微信回调
捕获包含 POST /pay/callback 的TCP包,保存到 wechat.shark.pcap
tshark -i em1 -n -f 'port 80 and tcp[((tcp[12:1] & 0xf0) >> 2):4] = 0x504f5354 && tcp[((tcp[12:1] & 0xf0) >> 2) + 4:4] = 0x202f7061 && tcp[((tcp[12:1] & 0xf0) >> 2) + 8:4] = 0x792f6361 && tcp[((tcp[12:1] & 0xf0) >> 2) + 12:4] = 0x6c6c6261 && tcp[((tcp[12:1] & 0xf0) >> 2) + 16:2] = 0x636b'  -w wechat.shark.pcap
POST pay callback
Figure 2. POST /pay/callback 字符串匹配-捕获过滤器 生成器截图
捕获过滤器常用表达式
  1. Capture only traffic to or from IP address 172.18.5.4: host 172.18.5.4

  2. Capture traffic to or from a range of IP addresses:
    net 192.168.0.0/24
    or
    net 192.168.0.0 mask 255.255.255.0

  3. Capture traffic from a range of IP addresses:
    src net 192.168.0.0/24
    or
    src net 192.168.0.0 mask 255.255.255.0

  4. Capture traffic to a range of IP addresses:
    dst net 192.168.0.0/24
    or
    dst net 192.168.0.0 mask 255.255.255.0

  5. Capture only DNS (port 53) traffic:
    port 53

  6. Capture non-HTTP and non-SMTP traffic on your server (both are equivalent):
    host www.example.com and not (port 80 or port 25)
    host www.example.com and not port 80 and not port 25

  7. Capture except all ARP and DNS traffic:
    port not 53 and not arp

  8. Capture traffic within a range of ports
    (tcp[0:2] > 1500 and tcp[0:2] < 1550) or (tcp[2:2] > 1500 and tcp[2:2] < 1550)
    or
    tcp portrange 1501-1549

  9. Capture only Ethernet type EAPOL:
    ether proto 0x888e

  10. Reject ethernet frames towards the Link Layer Discovery Protocol Multicast group:
    not ether dst 01:80:c2:00:00:0e

  11. Capture only IPv4 traffic - the shortest filter, but sometimes very useful to get rid of lower layer protocols like ARP and STP:
    ip

  12. Capture HTTP GET requests. This looks for the bytes 'G', 'E', 'T', and ' ' (hex values 47, 45, 54, and 20) just after the TCP header. "tcp[12:1] & 0xf0) >> 2" figures out the TCP header length.
    port 80 and tcp[((tcp[12:1] & 0xf0) >> 2):4] = 0x47455420

更多使用方法,请参考 CaptureFilters

12. Vue.js

12.1. 第一个Vue.js应用

12.1.1. 环境依赖

  • Node.js

12.1.2. 创建应用

npm install --global vue-cli
echo "export PATH=\$PATH:~/.npm-global/bin/" >> ~/.bashrc
source ~/.bashrc
vue init webpack demo_project

12.1.3. 运行

npm run dev

12.1.4. 打包

npm run build

dist 目录中的文件就是构建的静态页面。

$ tree dist/
dist/
├── index.html
└── static
    ├── css
    │   ├── app.30790115300ab27614ce176899523b62.css
    │   └── app.30790115300ab27614ce176899523b62.css.map
    └── js
        ├── app.2b6e1782b7cdd8e0b86c.js
        ├── app.2b6e1782b7cdd8e0b86c.js.map
        ├── manifest.2ae2e69a05c33dfc65f8.js
        ├── manifest.2ae2e69a05c33dfc65f8.js.map
        ├── vendor.dc2a748e035f2d805547.js
        └── vendor.dc2a748e035f2d805547.js.map

13. CentOS

13.1. CentOS6挂载ISO文件做仓库

CentOS6官方已经停止支持,软件仓库被移除(涉及所有仓库镜像,包括国内镜像)。

现在,yum 命令已经不能在线安装任何软件包。

This directory (and version of CentOS) is deprecated. Please see this FAQ concerning the CentOS release scheme:

Please keep in mind that 6.0, 6.1, 6.2, 6.3, 6.4 , 6.5, 6.6, 6.7, 6.8 , 6.9 and 6.10 no longer get any updates, nor any security fix’s. The whole CentOS 6 is dead and shouldn’t be used anywhere at all

生产服务器不能换系统,挂载ISO文件做仓库成唯一快速方法。


另外一种方法是替换 /etc/yum.repos.d/CentOS-Base.repo 中的URL:

baseurl=http://mirror.centos.org/centos/$releasever/

替换为:

baseurl=https://vault.centos.org/6.10/

6.10 注意替换为当前系统的版本号。

替换后下载速度慢,建议加代理使用,比如 all_proxy=http://x.x.x.x:8123


EPEL仓库也已经被移除,官方移动到了 https://archives.fedoraproject.org/pub/archive/epel/6/x86_64/

13.1.1. 下载 CentOS6 ISO镜像

国内CentOS镜像已经无法下载,只能从CentOS官方提供的地址下载。

需要下载 CentOS-6.10-x86_64-bin-DVD1.iso 和 CentOS-6.10-x86_64-bin-DVD2.iso。

推荐用种子下载,更快:

也可以直接下载,只是更慢:

13.1.2. 设置本地软件仓库

创建挂载目录
mkdir -p /media/CentOS-DVD1 /media/CentOS-DVD2
挂载ISO镜像

方法一:从光驱挂载

mount -o loop,ro /dev/cdrom /media/CentOS-DVD1
mount -o loop,ro /dev/cdrom1 /media/CentOS-DVD2

方法二:从ISO文件挂载

mount -o loop,ro CentOS-6.10-x86_64-bin-DVD1.iso /media/CentOS-DVD1
mount -o loop,ro CentOS-6.10-x86_64-bin-DVD2.iso /media/CentOS-DVD2
设置仓库文件

先备份:

cp /etc/yum.repos.d/CentOS-Media.repo /etc/yum.repos.d/CentOS-Media.repo.bak

创建配置文件:

cat << EOF > /etc/yum.repos.d/CentOS-Media.repo
# CentOS-Media.repo
#
#  This repo can be used with mounted DVD media, verify the mount point for
#  CentOS-6.  You can use this repo and yum to install items directly off the
#  DVD ISO that we release.
#
# To use this repo, put in your DVD and use it with the other repos too:
#  yum --enablerepo=c6-media [command]
#
# or for ONLY the media repo, do this:
#
#  yum --disablerepo=\* --enablerepo=c6-media [command]

[c6-media]
name=CentOS-\$releasever - Media
baseurl=file:///media/CentOS-DVD1
        file:///media/CentOS-DVD2
gpgcheck=1
enabled=1
gpgkey=file:///etc/pki/rpm-gpg/RPM-GPG-KEY-CentOS-6
EOF

清除YUM安装包缓存:

yum clean all
禁用在线仓库

因为在线仓库已经不可用,禁用所有在线仓库。

安装 yum-config-manager 命令所属的软件包 yum-utils

yum --disablerepo=\* --enablerepo=c6-media install -y yum-utils

yum-utils 包含很多有用的命令:

[root@localhost ~]# rpm -ql yum-utils|grep bin/
/usr/bin/debuginfo-install
/usr/bin/find-repos-of-install
/usr/bin/needs-restarting
/usr/bin/package-cleanup
/usr/bin/repo-graph
/usr/bin/repo-rss
/usr/bin/repoclosure
/usr/bin/repodiff
/usr/bin/repomanage
/usr/bin/repoquery
/usr/bin/reposync
/usr/bin/repotrack
/usr/bin/show-changed-rco
/usr/bin/show-installed
/usr/bin/verifytree
/usr/bin/yum-builddep
/usr/bin/yum-config-manager
/usr/bin/yum-debug-dump
/usr/bin/yum-debug-restore
/usr/bin/yum-groups-manager
/usr/bin/yumdownloader
/usr/sbin/yum-complete-transaction
/usr/sbin/yumdb

永久禁用默认启用的仓库:

yum-config-manager --disable base updates extras

yum repolist 命令确认只有 "c6-media" 仓库被启用:

控制台输出
[root@localhost ~]# yum repolist
Loaded plugins: fastestmirror
Loading mirror speeds from cached hostfile
repo id                            repo name                                 status
c6-media                           CentOS- - Media                           6,713
repolist: 6,713
确认本地仓库生效
yum list|grep vim
控制台输出
vim-common.x86_64                          2:7.4.629-5.el6_8.1         @c6-media
vim-enhanced.x86_64                        2:7.4.629-5.el6_8.1         @c6-media
vim-filesystem.x86_64                      2:7.4.629-5.el6_8.1         @c6-media
vim-minimal.x86_64                         2:7.4.629-5.el6_8.1         @anaconda-CentOS-201806291108.x86_64/6.10
vim-X11.x86_64                             2:7.4.629-5.el6_8.1         c6-media

现在,可以用 yum install 安装软件包了。

13.2. CentOS6初始化

yum -y erase iscsi* NetworkManager

yum -y install iproute vim-enhanced epel-release chrony wget curl screen sudo rsync tcpdump strace openssh-clients

wget http://mirrors.aliyun.com/repo/epel-6.repo -O /etc/yum.repos.d/epel.repo


chkconfig chronyd on
service chronyd start

#支持gbk文件显示
echo set fencs=utf-8,gbk >>/etc/vimrc

#删除ipv6的localhost配置
#::1         localhost localhost.localdomain localhost6 localhost6.localdomain6
echo "127.0.0.1 localhost $(hostname)" > /etc/hosts

# 禁用ipv6
echo "NETWORKING_IPV6=no" >> /etc/sysconfig/network
echo "net.ipv6.conf.all.disable_ipv6 = 1" >> /etc/sysctl.conf
echo "net.ipv6.conf.default.disable_ipv6 = 1" >> /etc/sysctl.conf
sysctl -p

sed -i "s/HOSTNAME=.*/HOSTNAME=$(hostname)/g"  /etc/sysconfig/network

rm -f ~/anaconda-ks.cfg  ~/install.log  ~/install.log.syslog

#禁用SELINUX,必须重启才能生效
echo SELINUX=disabled>/etc/selinux/config
echo SELINUXTYPE=targeted>>/etc/selinux/config

#清除iptables的默认规则
iptables -F
service iptables save

#最大可以打开的文件
echo "*               soft   nofile            65535" >> /etc/security/limits.conf
echo "*               hard   nofile            65535" >> /etc/security/limits.conf

#增加同时打开文件的数量
#/etc/profile会加载/etc/profile.d下的sh文件

#echo ulimit -n 65535 >/etc/profile.d/ulimit.sh
#chmod 755 /etc/profile.d/ulimit.sh

#禁止关闭显示器 archlinux wiki 提及的方法
echo -ne "\033[9;0]" >> /etc/issue
# 重启,cat /sys/module/kernel/parameters/consoleblank 为空表示生效


# ssh登录时,登录ip被会反向解析为域名,导致ssh登录缓慢
sed -i "s/#UseDNS yes/UseDNS no/" /etc/ssh/sshd_config
sed -i "s/GSSAPIAuthentication yes/GSSAPIAuthentication no/" /etc/ssh/sshd_config
sed -i "s/GSSAPICleanupCredentials yes/GSSAPICleanupCredentials no/" /etc/ssh/sshd_config
sed -i "s/#MaxAuthTries 6/MaxAuthTries 10/" /etc/ssh/sshd_config
# server每隔30秒发送一次请求给client,然后client响应,从而保持连接
sed -i "s/#ClientAliveInterval 0/ClientAliveInterval 30/" /etc/ssh/sshd_config
# server发出请求后,客户端没有响应得次数达到3,就自动断开连接,正常情况下,client不会不响应
sed -i "s/#ClientAliveCountMax 3/ClientAliveCountMax 10/" /etc/ssh/sshd_config

#设定系统时区
yes|cp /usr/share/zoneinfo/Asia/Shanghai /etc/localtime

#如果是x86_64系统,排除32位包
echo "exclude=*.i386 *.i586 *.i686" >> /etc/yum.conf

service_down_list=(lvm2-monitor mdmonitor postfix ip6tables netfs)
service_up_list=(iptables)


#停止服务并取消开机启动
for i in "${service_down_list[@]}"; do
    service $i stop
    chkconfig $i off
done

#开机启动
for i in "${service_up_list[@]}"; do
    service $i start
    chkconfig $i on
done

chkconfig  lvm2-monitor off 1

13.3. CentOS7初始化

13.3.1. 基础设置

rm -f ~/anaconda-ks.cfg  ~/install.log  ~/install.log.syslog

#禁止关闭显示器 archlinux wiki 提及的方法
echo -ne "\033[9;0]" >> /etc/issue
# 重启,cat /sys/module/kernel/parameters/consoleblank 为空表示生效

yum install -y epel-release

yum install -y iproute rsync yum-utils tree pwgen vim-enhanced wget curl screen bzip2 tcpdump unzip tar xz bash-completion-extras mlocate strace telnet iftop iotop

# PHP 7
yum install -y centos-release-scl

#查看主机名
hostnamectl status

#修改主机名
hostnamectl set-hostname 主机名

#删除ipv6的localhost配置
#::1         localhost localhost.localdomain localhost6 localhost6.localdomain6
echo "127.0.0.1 localhost localhost.localdomain localhost4 localhost4.localdomain4 $(hostname)" > /etc/hosts

#禁用SELINUX,必须重启才能生效
echo SELINUX=disabled>/etc/selinux/config
echo SELINUXTYPE=targeted>>/etc/selinux/config

#最大可以打开的文件
echo "*               soft   nofile            65535" >> /etc/security/limits.conf
echo "*               hard   nofile            65535" >> /etc/security/limits.conf

# ssh登录时,登录ip被会反向解析为域名,导致ssh登录缓慢
sed -i "s/#UseDNS yes/UseDNS no/" /etc/ssh/sshd_config
sed -i "s/GSSAPIAuthentication yes/GSSAPIAuthentication no/" /etc/ssh/sshd_config
sed -i "s/GSSAPICleanupCredentials yes/GSSAPICleanupCredentials no/" /etc/ssh/sshd_config
sed -i "s/#MaxAuthTries 6/MaxAuthTries 10/" /etc/ssh/sshd_config
# server每隔30秒发送一次请求给client,然后client响应,从而保持连接
sed -i "s/#ClientAliveInterval 0/ClientAliveInterval 30/" /etc/ssh/sshd_config
# server发出请求后,客户端没有响应得次数达到3,就自动断开连接,正常情况下,client不会不响应
sed -i "s/#ClientAliveCountMax 3/ClientAliveCountMax 10/" /etc/ssh/sshd_config

#支持gbk文件显示
echo "set fencs=utf-8,gbk" >> /etc/vimrc

#设定系统时区
yes|cp /usr/share/zoneinfo/Asia/Shanghai /etc/localtime

#时间同步
yum install -y chrony
systemctl enable chronyd --now

#如果是x86_64系统,排除32位包
echo "exclude=*.i386 *.i586 *.i686" >> /etc/yum.conf

#disable IPv6
echo "net.ipv6.conf.all.disable_ipv6 = 1" >>  /etc/sysctl.conf
echo "net.ipv6.conf.default.disable_ipv6 = 1" >>  /etc/sysctl.conf

#允许网络转发,主要是给iptables使用
echo "net.ipv4.ip_forward = 1" >> /etc/sysctl.conf

sysctl -p

#如果你想使用自己的 iptables 静态防火墙规则, 那么请安装 iptables-services 并且禁用 firewalld ,启用 iptables
yum install -y iptables-services
systemctl stop firewalld
systemctl mask firewalld
systemctl enable iptables --now

iptables -F
iptables-save >/etc/sysconfig/iptables

13.3.2. 常用命令行编辑工具

请先安装新版Python CentOS7 Install Python312

yum install -y xmlstarlet crudini

pip312 install --root-user-action=ignore -U yq toml-cli

wget https://github.com/stedolan/jq/releases/download/jq-1.6/jq-linux64 -O /usr/local/bin/jq
chmod 755 /usr/local/bin/jq

13.4. CentOS8初始化(过时)

yum install -y bash-completion bzip2 curl epel-release iftop iotop iproute lbzip2 mlocate pwgen rsync screen strace tar tcpdump telnet tree unzip vim-enhanced wget xz yum-utils

dnf install -y dnf-plugins-core
dnf config-manager --set-enabled powertools

#禁用SELINUX,必须重启才能生效
echo SELINUX=disabled>/etc/selinux/config
echo SELINUXTYPE=targeted>>/etc/selinux/config

#最大可以打开的文件
echo "*               soft   nofile            65535" >> /etc/security/limits.conf
echo "*               hard   nofile            65535" >> /etc/security/limits.conf

# ssh登录时,登录ip被会反向解析为域名,导致ssh登录缓慢
sed -i "s/#UseDNS yes/UseDNS no/" /etc/ssh/sshd_config
sed -i "s/GSSAPIAuthentication yes/GSSAPIAuthentication no/" /etc/ssh/sshd_config
sed -i "s/GSSAPICleanupCredentials yes/GSSAPICleanupCredentials no/" /etc/ssh/sshd_config
sed -i "s/#MaxAuthTries 6/MaxAuthTries 10/" /etc/ssh/sshd_config
# server每隔30秒发送一次请求给client,然后client响应,从而保持连接
sed -i "s/#ClientAliveInterval 0/ClientAliveInterval 30/" /etc/ssh/sshd_config
# server发出请求后,客户端没有响应得次数达到3,就自动断开连接,正常情况下,client不会不响应
sed -i "s/#ClientAliveCountMax 3/ClientAliveCountMax 10/" /etc/ssh/sshd_config

#支持gbk文件显示
echo "set fencs=utf-8,gbk" >> /etc/vimrc

#设定系统时区
yes|cp /usr/share/zoneinfo/Asia/Chongqing /etc/localtime

#时间同步
dnf install -y systemd-timesyncd
systemctl enable systemd-timesyncd
systemctl start systemd-timesyncd

#如果是x86_64系统,排除32位包
echo "exclude=*.i386 *.i586 *.i686" >> /etc/yum.conf

systemctl stop firewalld
systemctl disable firewalld
systemctl mask firewalld

13.5. CentOS9初始化

13.5.1. 基础设置

rm -f ~/anaconda-ks.cfg

dnf install -y epel-release

dnf install -y iproute rsync yum-utils tree pwgen vim-enhanced wget curl screen lbzip2 tcpdump unzip tar xz bash-completion mlocate strace telnet iftop iotop

dnf install -y dnf-plugins-core
# The CodeReady Linux Builder repository contains additional packages for use by developers.
dnf config-manager --set-enabled crb
# dnf repo-pkgs crb list


#禁用SELINUX,必须重启才能生效
echo SELINUX=disabled>/etc/selinux/config
echo SELINUXTYPE=targeted>>/etc/selinux/config

#最大可以打开的文件
echo "*               soft   nofile            65535" >> /etc/security/limits.conf
echo "*               hard   nofile            65535" >> /etc/security/limits.conf

# ssh登录时,登录ip被会反向解析为域名,导致ssh登录缓慢
sed -i "s/#UseDNS no/UseDNS no/" /etc/ssh/sshd_config
sed -i "s/#GSSAPIAuthentication no/GSSAPIAuthentication no/" /etc/ssh/sshd_config
sed -i "s/#GSSAPICleanupCredentials no/GSSAPICleanupCredentials no/" /etc/ssh/sshd_config
sed -i "s/#MaxAuthTries 6/MaxAuthTries 10/" /etc/ssh/sshd_config
# server每隔30秒发送一次请求给client,然后client响应,从而保持连接
sed -i "s/#ClientAliveInterval 0/ClientAliveInterval 30/" /etc/ssh/sshd_config
# server发出请求后,客户端没有响应得次数达到3,就自动断开连接,正常情况下,client不会不响应
sed -i "s/#ClientAliveCountMax 3/ClientAliveCountMax 10/" /etc/ssh/sshd_config

#支持gbk文件显示
echo "set fencs=utf-8,gbk" >> /etc/vimrc

#设定系统时区
yes|cp /usr/share/zoneinfo/Asia/Shanghai /etc/localtime

#时间同步
dnf install -y systemd-timesyncd
systemctl enable systemd-timesyncd --now

#如果是x86_64系统,排除32位包
echo "exclude=*.i386 *.i586 *.i686" >> /etc/yum.conf

#disable IPv6
echo "net.ipv6.conf.all.disable_ipv6 = 1" >>  /etc/sysctl.conf
echo "net.ipv6.conf.default.disable_ipv6 = 1" >>  /etc/sysctl.conf

#允许网络转发,主要是给iptables使用
echo "net.ipv4.ip_forward = 1" >> /etc/sysctl.conf

sysctl -p

#如果你想使用自己的 iptables 静态防火墙规则, 那么请安装 iptables-services 并且禁用 firewalld ,启用 iptables
dnf install -y iptables-services
systemctl stop firewalld
systemctl mask firewalld
systemctl enable iptables --now

iptables -F
iptables-save >/etc/sysconfig/iptables

13.5.2. 常用命令行编辑工具

dnf install -y xmlstarlet crudini

dnf install -y python3.11 python3.11-pip

pip3.11 install --root-user-action=ignore -U yq toml-cli

wget https://github.com/stedolan/jq/releases/download/jq-1.6/jq-linux64 -O /usr/local/bin/jq
chmod 755 /usr/local/bin/jq

13.6. CentOS7 Install LNMP

13.6.1. 安装 Nginx

增加 Nginx 官方源
cat << EOF > /etc/yum.repos.d/nginx.repo
[nginx-stable]
name=nginx stable repo
baseurl=http://nginx.org/packages/centos/\$releasever/\$basearch/
gpgcheck=1
enabled=1
gpgkey=https://nginx.org/keys/nginx_signing.key
module_hotfixes=true

[nginx-mainline]
name=nginx mainline repo
baseurl=http://nginx.org/packages/mainline/centos/\$releasever/\$basearch/
gpgcheck=1
enabled=0
gpgkey=https://nginx.org/keys/nginx_signing.key
module_hotfixes=true
EOF

EPEL 源中的 nginx.service 由于 KILL 参数问题,启动后无法停止,不建议使用。

安装Nginx
yum install -y nginx
备份Nginx配置文件
echo y|cp /etc/nginx/nginx.conf /etc/nginx/nginx.conf.default
修改 nginx.conf
cat << EOF > /etc/nginx/nginx.conf
# For more information on configuration, see:
#   * Official English Documentation: http://nginx.org/en/docs/
#   * Official Russian Documentation: http://nginx.org/ru/docs/

user nginx;
worker_processes auto;
error_log  /var/log/nginx/error.log warn;
pid        /var/run/nginx.pid;

worker_rlimit_nofile 65535;

events {
    worker_connections 65535;
}

http {
    include             /etc/nginx/mime.types;
    default_type        application/octet-stream;

    log_format  main  '\$host \$server_port \$remote_addr - \$remote_user [\$time_local] "\$request" '
                      '\$status \$request_time \$body_bytes_sent "\$http_referer" '
                      '"\$http_user_agent" "\$http_x_forwarded_for"';

    access_log  /var/log/nginx/access.log  main;

    sendfile            on;
    tcp_nopush          on;
    tcp_nodelay         on;
    keepalive_timeout   65;
    types_hash_max_size 2048;

    server_names_hash_bucket_size 128;
    server_name_in_redirect off;
    client_header_buffer_size 32k;
    large_client_header_buffers 4 32k;

    client_header_timeout  3m;
    client_body_timeout    3m;
    client_max_body_size 50m;
    client_body_buffer_size 256k;
    send_timeout           3m;

    gzip  on;
    gzip_min_length  1k;
    gzip_buffers     4 16k;
    gzip_http_version 1.0;
    gzip_comp_level 2;
    gzip_types image/svg+xml application/x-font-wof text/plain text/xml text/css application/xml application/xhtml+xml application/rss+xml application/javascript application/x-javascript text/javascript;
    gzip_vary on;

    proxy_redirect off;
    proxy_set_header Host \$host;
    proxy_set_header X-Real-IP \$remote_addr;
    proxy_set_header REMOTE-HOST \$remote_addr;
    proxy_set_header X-Forwarded-For \$proxy_add_x_forwarded_for;
    proxy_connect_timeout 60;
    proxy_send_timeout 60;
    proxy_read_timeout 60;
    proxy_buffer_size 256k;
    proxy_buffers 4 256k;
    proxy_busy_buffers_size 256k;
    proxy_temp_file_write_size 256k;
    proxy_next_upstream error timeout invalid_header http_500 http_503 http_404;
    proxy_max_temp_file_size 128m;
    #让代理服务端不要主动关闭客户端的连接,协助处理499返回代码问题
    proxy_ignore_client_abort on;

    fastcgi_buffer_size 64k;
    fastcgi_buffers 4 64k;
    fastcgi_busy_buffers_size 128k;

    index index.html index.htm index.php default.html default.htm default.php;

    # Load modular configuration files from the /etc/nginx/conf.d directory.
    # See http://nginx.org/en/docs/ngx_core_module.html#include
    # for more information.
    include /etc/nginx/conf.d/*.conf;
}
EOF
增加默认Host
mkdir /etc/nginx/conf.d

cat << EOF > /etc/nginx/conf.d/default.conf
server {
    listen       80 default_server;
    listen       [::]:80 default_server;
    server_name  _;
    root         /usr/share/nginx/html;

    # Load configuration files for the default server block.
    include /etc/nginx/default.d/*.conf;

    location / {
    }

    error_page 404 /404.html;
        location = /40x.html {
    }

    error_page 500 502 503 504 /50x.html;
        location = /50x.html {
    }
}
EOF
检测配置
nginx -t && rm -f /var/run/nginx.pid

nginx -t 之后,/var/run/nginx.pid 空文件会一直被保留,而 nginx.service 并不能处理 PIDFile 为空的情况,导致启动失败。

需要手动删除 /var/run/nginx.pid

from nginx/1.16.1

启动Nginx
systemctl start nginx
查看Nginx状态
# systemctl status nginx
● nginx.service - The nginx HTTP and reverse proxy server
   Loaded: loaded (/usr/lib/systemd/system/nginx.service; disabled; vendor preset: disabled)
   Active: active (running) since Fri 2019-11-29 14:02:31 CST; 1h 18min ago
 Main PID: 15759 (nginx)
   CGroup: /system.slice/nginx.service
           ├─15759 nginx: master process /usr/sbin/nginx
           └─17285 nginx: worker process

Nov 29 14:02:31 iZ6weebcmroarpx8rrxscrZ systemd[1]: Starting The nginx HTTP and reverse proxy server...
Nov 29 14:02:31 iZ6weebcmroarpx8rrxscrZ nginx[15753]: nginx: the configuration file /etc/nginx/nginx.conf syntax is ok
Nov 29 14:02:31 iZ6weebcmroarpx8rrxscrZ nginx[15753]: nginx: configuration file /etc/nginx/nginx.conf test is successful
Nov 29 14:02:31 iZ6weebcmroarpx8rrxscrZ systemd[1]: Failed to parse PID from file /run/nginx.pid: Invalid argument
Nov 29 14:02:31 iZ6weebcmroarpx8rrxscrZ systemd[1]: Started The nginx HTTP and reverse proxy server.


# ss -antpl|grep nginx
LISTEN     0      128          *:80                       *:*                   users:(("nginx",pid=17285,fd=6),("nginx",pid=15759,fd=6))
LISTEN     0      128         :::80                      :::*                   users:(("nginx",pid=17285,fd=7),("nginx",pid=15759,fd=7))
增加开机启动
systemctl enable nginx

13.6.2. 安装 MySQL

安装 MySQL
yum install -y mariadb-server
备份 my.cnf
cp /etc/my.cnf /etc/my.cnf.default
修改 my.cnf
cat << EOF > /etc/my.cnf
[mysqld]
datadir=/var/lib/mysql
socket=/var/lib/mysql/mysql.sock
# Disabling symbolic-links is recommended to prevent assorted security risks
symbolic-links=0
# Settings user and group are ignored when systemd is used.
# If you need to run mysqld under a different user or group,
# customize your systemd unit file for mariadb according to the
# instructions in http://fedoraproject.org/wiki/Systemd

max_allowed_packet=20M
max_heap_table_size = 100M
read_buffer_size = 2M
read_rnd_buffer_size = 16M
sort_buffer_size = 8M
join_buffer_size = 8M
tmp_table_size = 100M

# 查询缓存
#query_cache_limit=4M
#query_cache_type=on
#query_cache_size=2G

bind-address = 127.0.0.1
# 跳过主机名解析,比如localhost,foo.com之类,加速访问
skip-name-resolve

# SQL执行日志
general_log=off
general_log_file=/var/log/mariadb/general.log

# SQL慢查询日志
slow_query_log=off
slow_query_log_file=/var/log/mariadb/slowquery.log
long_query_time = 5

max_connections = 1000

# 兼容老MySQL代码,比如使用空字符串代替NULL插入数据
sql_mode = ""

[mysqld_safe]
log-error=/var/log/mariadb/mariadb.log
pid-file=/var/run/mariadb/mariadb.pid

#
# include all files from the config directory
#
!includedir /etc/my.cnf.d
EOF
配置 mysqldump 命令参数
sed -i '16 aquick\nquote-names\nmax_allowed_packet = 100M' /etc/my.cnf.d/mysql-clients.cnf
创建日志文件
touch /var/log/mariadb/general.log /var/log/mariadb/slowquery.log
chown mysql:mysql /var/log/mariadb/general.log /var/log/mariadb/slowquery.log
增加开机启动
systemctl enable mariadb
启动 MySQL 服务
systemctl start mariadb
查看 MySQL 服务状态
# systemctl status mariadb
● mariadb.service - MariaDB database server
   Loaded: loaded (/usr/lib/systemd/system/mariadb.service; enabled; vendor preset: disabled)
   Active: active (running) since Fri 2019-11-29 14:18:12 CST; 1h 7min ago
  Process: 16688 ExecStartPost=/usr/libexec/mariadb-wait-ready $MAINPID (code=exited, status=0/SUCCESS)
  Process: 16653 ExecStartPre=/usr/libexec/mariadb-prepare-db-dir %n (code=exited, status=0/SUCCESS)
 Main PID: 16687 (mysqld_safe)
   CGroup: /system.slice/mariadb.service
           ├─16687 /bin/sh /usr/bin/mysqld_safe --basedir=/usr
           └─17043 /usr/libexec/mysqld --basedir=/usr --datadir=/var/lib/mysql --plugin-dir=/usr/lib64/mysql/plugin --log-error=/var/log/mariadb/mariadb.lo...

Nov 29 14:18:10 iZ6weebcmroarpx8rrxscrZ systemd[1]: Starting MariaDB database server...
Nov 29 14:18:10 iZ6weebcmroarpx8rrxscrZ mariadb-prepare-db-dir[16653]: Database MariaDB is probably initialized in /var/lib/mysql already, nothing is done.
Nov 29 14:18:11 iZ6weebcmroarpx8rrxscrZ mysqld_safe[16687]: 191129 14:18:11 mysqld_safe Logging to '/var/log/mariadb/mariadb.log'.
Nov 29 14:18:11 iZ6weebcmroarpx8rrxscrZ mysqld_safe[16687]: 191129 14:18:11 mysqld_safe Starting mysqld daemon with databases from /var/lib/mysql
Nov 29 14:18:12 iZ6weebcmroarpx8rrxscrZ systemd[1]: Started MariaDB database server.

# ss -antpl|grep mysql
LISTEN     0      50     127.0.0.1:3306                     *:*                   users:(("mysqld",pid=17043,fd=14))
修改root密码
mysqladmin -uroot password "geek"
删除测试数据库和空密码用户
mysql -uroot -pgeek -e 'show databases;'
mysql -uroot -pgeek -e 'drop database test;'
mysql -uroot -pgeek mysql -e 'delete from db;'
mysql -uroot -pgeek mysql -e 'delete from user where Password="";'
mysql -uroot -pgeek -e 'flush privileges;'

13.6.3. 安装 PHP7

增加SCL源
yum install -y centos-release-scl
安装PHP7.2
yum install -y rh-php72 \
    rh-php72-php  \
    rh-php72-php-bcmath \
    rh-php72-php-fpm \
    rh-php72-php-gd \
    rh-php72-php-intl \
    rh-php72-php-mbstring \
    rh-php72-php-mysqlnd \
    rh-php72-php-opcache \
    rh-php72-php-pdo \
    rh-php72-php-pecl-apcu \
    rh-php72-php-xmlrpc \
    rh-php72-php-devel
进入 rh-php72 环境
scl enable rh-php72 bash
确认PHP状态
# php -v
PHP 7.2.24 (cli) (built: Nov  4 2019 10:23:08) ( NTS )
Copyright (c) 1997-2018 The PHP Group
Zend Engine v3.2.0, Copyright (c) 1998-2018 Zend Technologies
    with Zend OPcache v7.2.24, Copyright (c) 1999-2018, by Zend Technologies
备份php.ini
cp /etc/opt/rh/rh-php72/php.ini /etc/opt/rh/rh-php72/php.ini.default
修改php.ini
# 启用 '<? ... ?>' 代码风格
sed -i '197s/short_open_tag = Off/short_open_tag = On/' /etc/opt/rh/rh-php72/php.ini

# 禁止一些危险性高的函数
sed -i '314s/disable_functions =/disable_functions = system,exec,shell_exec,passthru,set_time_limit,ini_alter,dl,openlog,syslog,readlink,symlink,link,leak,popen,escapeshellcmd,virtual,socket_create,mail,eval/' /etc/opt/rh/rh-php72/php.ini

# 配置中国时区
sed -i '902s#;date.timezone =#date.timezone = Asia/Shanghai#' /etc/opt/rh/rh-php72/php.ini
增加开机启动
systemctl enable rh-php72-php-fpm
启动 PHP-FPM 服务
systemctl start rh-php72-php-fpm
查看 PHP-FPM 服务状态
# systemctl status rh-php72-php-fpm
● rh-php72-php-fpm.service - The PHP FastCGI Process Manager
   Loaded: loaded (/usr/lib/systemd/system/rh-php72-php-fpm.service; enabled; vendor preset: disabled)
   Active: active (running) since Fri 2019-11-29 13:36:03 CST; 1h 56min ago
 Main PID: 15360 (php-fpm)
   Status: "Processes active: 0, idle: 6, Requests: 56, slow: 0, Traffic: 0req/sec"
   CGroup: /system.slice/rh-php72-php-fpm.service
           ├─15360 php-fpm: master process (/etc/opt/rh/rh-php72/php-fpm.conf)
           ├─15361 php-fpm: pool www
           ├─15362 php-fpm: pool www
           ├─15363 php-fpm: pool www
           ├─15364 php-fpm: pool www
           ├─15365 php-fpm: pool www
           └─17211 php-fpm: pool www

Nov 29 13:36:03 iZ6weebcmroarpx8rrxscrZ systemd[1]: Starting The PHP FastCGI Process Manager...
Nov 29 13:36:03 iZ6weebcmroarpx8rrxscrZ systemd[1]: Started The PHP FastCGI Process Manager.

# ss -antpl|grep php-fpm
LISTEN     0      128    127.0.0.1:9000                     *:*                   users:(("php-fpm",pid=17211,fd=9),("php-fpm",pid=15365,fd=9),("php-fpm",pid=15364,fd=9),("php-fpm",pid=15363,fd=9),("php-fpm",pid=15362,fd=9),("php-fpm",pid=15361,fd=9),("php-fpm",pid=15360,fd=7))

13.6.4. LNMP 环境测试

增加数据库
mysql -uroot -pgeek -e 'create database drupal;grant all privileges on drupal.*  to drupal@"localhost" identified by "drupal_password";flush privileges;'
增加Nginx Host设置
cat << EOF > /etc/nginx/conf.d/drupal.foo.com.conf
server {
    listen       80;

    server_name  drupal.foo.com;
    root         /data/web/drupal.foo.com;
    error_log /var/log/nginx/drupal.foo.com_error.log;
    access_log /var/log/nginx/drupal.foo.com_access.log  main;

    location / {
        try_files \$uri /index.php\$is_args\$query_string;
    }

    location ~ \.php\$ {
        try_files \$uri \$uri/ 404;

        fastcgi_pass 127.0.0.1:9000;
        fastcgi_param SCRIPT_FILENAME \$document_root\$fastcgi_script_name;
        include fastcgi_params;
    }
}
EOF

# 重载Nginx配置
nginx -t && nginx -s reload
准备 Drupal
mkdir -p /data/web/drupal.foo.com

# 使用 -O 参数指定保存文件名,会强制覆盖已经存在的文件
wget https://ftp.drupal.org/files/projects/drupal-8.7.10.tar.gz -O drupal-8.7.10.tar.gz
tar xf drupal-8.7.10.tar.gz

mv drupal-8.7.10/* /data/web/drupal.foo.com
rm -rf drupal-8.7.10
chown -R apache:nginx /data/web/drupal.foo.com
chmod -R 755 /data/web/drupal.foo.com

drupal-8.7.10/core/INSTALL.txt

Drupal requires:
设置本地解析
echo '47.74.60.161 drupal.foo.com' >> /etc/hosts
  • /etc/hosts (Linux)

  • C:\Windows\System32\drivers\etc\hosts(Windows)

最后,访问 http://drupal.foo.com 完成安装。

Drupal数据库相关信息:

  • 数据库服务器:localhost

  • 数据库端口:3306

  • 数据库名称:drupal

  • 数据库用户名:drupal

  • 数据库密码:drupal_password

13.7. CentOS8配置文档

13.7.1. 安装 Nginx

增加EPEL源
dnf install -y epel-release
安装Nginx
dnf install -y nginx
备份Nginx配置文件
echo y|cp /etc/nginx/nginx.conf /etc/nginx/nginx.conf.defaultv
修改 nginx.conf
cat << EOF > /etc/nginx/nginx.conf
# For more information on configuration, see:
#   * Official English Documentation: http://nginx.org/en/docs/
#   * Official Russian Documentation: http://nginx.org/ru/docs/

user nginx;
worker_processes auto;
error_log /var/log/nginx/error.log;
pid /run/nginx.pid;

# Load dynamic modules. See /usr/share/doc/nginx/README.dynamic.
include /usr/share/nginx/modules/*.conf;

worker_rlimit_nofile 65535;

events {
    worker_connections 65535;
}

http {
    include             /etc/nginx/mime.types;
    default_type        application/octet-stream;

    log_format  main  '\$host \$server_port \$remote_addr - \$remote_user [\$time_local] "\$request" '
                      '\$status \$request_time \$body_bytes_sent "\$http_referer" '
                      '"\$http_user_agent" "\$http_x_forwarded_for"';

    access_log  /var/log/nginx/access.log  main;

    sendfile            on;
    tcp_nopush          on;
    tcp_nodelay         on;
    keepalive_timeout   65;
    types_hash_max_size 2048;

    server_names_hash_bucket_size 128;
    server_name_in_redirect off;
    client_header_buffer_size 32k;
    large_client_header_buffers 4 32k;

    client_header_timeout  3m;
    client_body_timeout    3m;
    client_max_body_size 50m;
    client_body_buffer_size 256k;
    send_timeout           3m;

    gzip  on;
    gzip_min_length  1k;
    gzip_buffers     4 16k;
    gzip_http_version 1.0;
    gzip_comp_level 2;
    gzip_types image/svg+xml application/x-font-wof text/plain text/xml text/css application/xml application/xhtml+xml application/rss+xml application/javascript application/x-javascript text/javascript;
    gzip_vary on;

    proxy_redirect off;
    proxy_set_header Host \$host;
    proxy_set_header X-Real-IP \$remote_addr;
    proxy_set_header REMOTE-HOST \$remote_addr;
    proxy_set_header X-Forwarded-For \$proxy_add_x_forwarded_for;
    proxy_connect_timeout 60;
    proxy_send_timeout 60;
    proxy_read_timeout 60;
    proxy_buffer_size 256k;
    proxy_buffers 4 256k;
    proxy_busy_buffers_size 256k;
    proxy_temp_file_write_size 256k;
    proxy_next_upstream error timeout invalid_header http_500 http_503 http_404;
    proxy_max_temp_file_size 128m;
    #让代理服务端不要主动关闭客户端的连接,协助处理499返回代码问题
    proxy_ignore_client_abort on;

    fastcgi_buffer_size 64k;
    fastcgi_buffers 4 64k;
    fastcgi_busy_buffers_size 128k;

    index index.html index.htm index.php default.html default.htm default.php;

    # Load modular configuration files from the /etc/nginx/conf.d directory.
    # See http://nginx.org/en/docs/ngx_core_module.html#include
    # for more information.
    include /etc/nginx/conf.d/*.conf;
}
EOF
增加默认Host
mkdir /etc/nginx/conf.d

cat << EOF > /etc/nginx/conf.d/default.conf
server {
    listen       80 default_server;
    listen       [::]:80 default_server;
    server_name  _;
    root         /usr/share/nginx/html;

    # Load configuration files for the default server block.
    include /etc/nginx/default.d/*.conf;

    location / {
    }

    error_page 404 /404.html;
        location = /40x.html {
    }

    error_page 500 502 503 504 /50x.html;
        location = /50x.html {
    }
}
EOF
检查配置
nginx -t
启动Nginx
systemctl start nginx
查看Nginx状态
systemctl status nginx
增加开机启动
systemctl enable nginx
查看版本号
nginx -v

13.7.2. 安装 MySQL

安装 MySQL
dnf install -y mariadb-server
备份 my.cnf
cp /etc/my.cnf /etc/my.cnf.default
修改 my.cnf
cat << EOF > /etc/my.cnf
[mysqld]
datadir=/var/lib/mysql
socket=/var/lib/mysql/mysql.sock
# Disabling symbolic-links is recommended to prevent assorted security risks
symbolic-links=0
# Settings user and group are ignored when systemd is used.
# If you need to run mysqld under a different user or group,
# customize your systemd unit file for mariadb according to the
# instructions in http://fedoraproject.org/wiki/Systemd

max_allowed_packet=20M
max_heap_table_size = 100M
read_buffer_size = 2M
read_rnd_buffer_size = 16M
sort_buffer_size = 8M
join_buffer_size = 8M
tmp_table_size = 100M

# 查询缓存
#query_cache_limit=4M
#query_cache_type=on
#query_cache_size=2G

bind-address = 127.0.0.1
# 跳过主机名解析,比如localhost,foo.com之类,加速访问
skip-name-resolve

# SQL执行日志
general_log=off
general_log_file=/var/log/mariadb/general.log

# SQL慢查询日志
slow_query_log=off
slow_query_log_file=/var/log/mariadb/slowquery.log
long_query_time = 5

max_connections = 1000

# 兼容老MySQL代码,比如使用空字符串代替NULL插入数据
sql_mode = ""

[mysqld_safe]
log-error=/var/log/mariadb/mariadb.log
pid-file=/var/run/mariadb/mariadb.pid

#
# include all files from the config directory
#
!includedir /etc/my.cnf.d
EOF
增加开机启动
systemctl enable mariadb
启动MySQL
systemctl start mariadb
查看 MySQL 服务状态
systemctl status mariadb
修改root密码
mysqladmin -uroot password "geek"
删除测试数据库和空密码用户
mysql -uroot -pgeek -e 'show databases;'
mysql -uroot -pgeek -e 'drop database test;'
mysql -uroot -pgeek mysql -e 'delete from db;'
mysql -uroot -pgeek mysql -e 'delete from user where Password="";'
mysql -uroot -pgeek -e 'flush privileges;'

13.7.3. 安装 PHP7

安装PHP7
dnf install -y \
  php  \
  php-bcmath \
  php-fpm \
  php-gd \
  php-intl \
  php-mbstring \
  php-mysqlnd \
  php-opcache \
  php-pdo \
  php-pecl-apcu \
  php-xmlrpc \
  php-devel \
  php-json
修改php.ini
sed -i '197s/short_open_tag = Off/short_open_tag = On/' /etc/php.ini

# 禁止一些危险性高的函数
sed -i '314s/disable_functions =/disable_functions = system,exec,shell_exec,passthru,set_time_limit,ini_alter,dl,openlog,syslog,readlink,symlink,link,leak,popen,escapeshellcmd,virtual,socket_create,mail,eval/' /etc/php.ini

# 配置中国时区
sed -i '902s#;date.timezone =#date.timezone = Asia/Shanghai#' /etc/php.ini
增加开机启动
systemctl enable php-fpm
启动 PHP-FPM 服务
systemctl start php-fpm
查看 PHP-FPM 服务状态
systemctl status php-fpm

13.7.4. LNMP环境测试

增加数据库
mysql -uroot -pgeek -e 'create database drupal;grant all privileges on drupal.* to drupal@"localhost" identified by "drupal_password";flush privileges;'
增加Nginx Host设置
cat << EOF > /etc/nginx/conf.d/drupal.foo.com.conf
server {
    listen       80;

    server_name  drupal.foo.com;
    root         /data/web/drupal.foo.com;
    error_log /var/log/nginx/drupal.foo.com_error.log;
    access_log /var/log/nginx/drupal.foo.com_access.log  main;

    location / {
        try_files \$uri /index.php\$is_args\$query_string;
    }

    location ~ \.php\$ {
        try_files \$uri \$uri/ 404;

        fastcgi_pass unix://run/php-fpm/www.sock;
        include fastcgi.conf;
    }
}
EOF
重载Nginx配置

nginx -s reload

准备 Drupal
mkdir -p /data/web/drupal.foo.com
使用 -O 参数指定保存文件名,会强制覆盖已经存在的文件
wget https://ftp.drupal.org/files/projects/drupal-8.7.10.tar.gz -O drupal-8.7.10.tar.gz
tar xf drupal-8.7.10.tar.gz
mv drupal-8.7.10/* /data/web/drupal.foo.com
rm -rf drupal-8.7.10
chown -R apache:nginx /data/web/drupal.foo.com
chmod -R 755 /data/web/drupal.foo.com
设置本地解析
echo '47.74.60.161 drupal.foo.com' >> /etc/hosts

13.8. CentOS8安装前加载驱动

RedHat7中支持,但是在RedHat8中已经移除的硬件列表。特别是一些老的存储相关硬件。

ELRepo 社区提供一些硬件支持的DUD文件,全文如下:

RHEL 8.0 and support for removed adapters
In RHEL 8.0, support for a good number of hardware devices has been removed. A list of removed adapters with their device IDs can be found in this RHEL documentation[https://access.redhat.com/documentation/en-us/red_hat_enterprise_linux/8/html/considerations_in_adopting_rhel_8/hardware-enablement_considerations-in-adopting-rhel-8#removed-adapters_hardware-enablement]. We provide support for some of those that are still fairly commonly used today. You can check your devices's IDs (as shown by lspci -nn) against our list of supported devices.
(1) Installation of the OS requires a driver for your hardware. We offer driver update disks (DUD). You can download them from here[https://elrepo.org/linux/dud/el8/x86_64/] or the corresponding directory of our mirror sites. Each DUD image contains a driver in the form of a kmod package. The installer is supposed to find the driver. If this does not happen, you need to append the inst.dd option to the boot command line. For details please see Performing an assisted driver update[https://access.redhat.com/documentation/en-us/red_hat_enterprise_linux/8/html/performing_an_advanced_rhel_installation/updating-drivers-during-installation_installing-rhel-as-an-experienced-user#performing-an-assisted-driver-update_updating-drivers-during-installation].
(2) The installation process installs the kmod package for your adapter. Normally, because of the kABI-tracking nature of kmod, there is no need to reinstall the driver upon each new kernel update. However, it was found that the current version of dracut in RHEL 8.0 has a bug and the initramfs image of a new kernel does not contain the kernel module from the installed kmod package. As a result, the new kernel fails to boot.
As an interim solution for the problem in (2), we provide a dracut package with a patch that fixes the bug here:
http://elrepo.org/people/akemi/testing/el8/dracut/
Install this version of dracut and then update the kernel. The system now should boot normally.
[UPDATE]  dracut-049-10.git20190115.el8_0.1 released on Oct 29, 2019 has the patch that fixes the issue.

13.8.1. Dell Precision T3600

Storage Controller

内置:

  • Intel Storage Control Unit (SCU) SATA 3.0Gb/s controller with host based RAID 0, 1, 5, 10.
    (Serial Attached SCSI controller [0300]: Intel Corporation C602 chipset 4-Port SATAStorage Control Unit [8086: 1d6b])

DUD文件 dd-isci-1.2.0-1.el8.elrepo.iso 中,文件路径有内核版本号,需要确认安装镜像中运行的内核版本。

加载DUD文件 dd-isci-1.2.0-1.el8.elrepo.iso 之后,CentOS8 安装界面会出问题,自动重启,无法使用。

  • Dual AHCI 6Gb/s SATA port option (no RAID support)

可选:

  • Dell PERC H310 PCIe host-based RAID card with four usable 6Gb/s ports supporting SATA/SAS/SSD drives and RAID 0, 1, 5, 10 configurations
    (SAS controller: LSI SAS2008)

  • Dell PERC H710P PCIe hardware RAID card with 1GB non-volatile cache memory and four usable 6Gb/s ports supporting SATA/SAS/SSD drives and RAID 0,1, 5, 10 configurations

说明:

Note: In my case, I'm using the card in IT mode; the disks are not set into RAID with the card, so only the mpt3sas DUD is required.
If you have a IR mode (RAID via the controller) configuration, you may need the megaraid-sas DUD too.

H310 加载 megaraid_sas 之后,CentOS 8安装盘识别到了硬盘。

DUD

在USB启动之后,在安装菜单中按 Tab,在启动参数最后追加 inst.dd,进入安装界面之后,会交互式提示选择iso文件。

不过,经过测试发现交互式选择iso,无法挂载。

最终,选择修改USB设备卷标 OEMDRV 方式,成功加载。

如果需要同时加载多个DUD文件,使用 inst.dd=http://192.168.0.2/dd.iso 加上卷标 OEMDRV 方式自动加载,这样会加载两次。

CentOS8 网络安装地址:

H310 不支持SSD启动,建议使用SSD+Sata端口安装和运行。

T3600安装CentOS8时,只有两个盘位。所以,只能接SATA0和SATA1。并且,要保证BIOS中,SATA控制器模式是ACHI。

14. Mosquitto

14.1. 安装

基于CentOS8

官方文档: https://mosquitto.org/

14.1.1. 依赖

系统依赖
dnf install -y openssl-devel gcc make gcc-c++
第三方依赖(可选)

让Mosquitto支持WebSockets客户端需要安装 libwebsockets 依赖。

dnf install -y cmake

wget -O libwebsockets-4.0.1.tar.gz https://github.com/warmcat/libwebsockets/archive/v4.0.1.tar.gz
tar xf libwebsockets-4.0.1.tar.gz
cd libwebsockets-4.0.1
mkdir build
cd build
cmake -DCMAKE_INSTALL_PREFIX=/usr/local/libwebsockets-4.0.1 ..
make -j4 install

echo "/usr/local/libwebsockets-4.0.1/lib" > /etc/ld.so.conf.d/libwebsockets.conf
ldconfig
ldconfig -p|grep libwebsockets

14.1.2. 编译

下载源代码
mkdir ~/downloads
cd ~/downloads
wget -O mosquitto-1.6.9.tar.gz https://mosquitto.org/files/source/mosquitto-1.6.9.tar.gz
tar xf mosquitto-1.6.9.tar.gz
编译:默认配置
cd mosquitto-1.6.9
make -j4
make install prefix=/usr/local/mosquitto-1.6.9
编译:支持WebSockts客户端
export PKG_CONFIG_PATH=$PKG_CONFIG_PATH:/usr/local/libwebsockets-4.0.1/lib/pkgconfig
ln -s /usr/local/libwebsockets-4.0.1/lib/libwebsockets.so /usr/lib/libwebsockets.so

cd mosquitto-1.6.9
make -j4 CFLAGS=`pkg-config --cflags-only-I libwebsockets` WITH_WEBSOCKETS=yes
make install prefix=/usr/local/mosquitto-1.6.9

unlink /usr/lib/libwebsockets.so

14.2. 配置

14.2.1. 基础配置

加入环境变量:

ln -s /usr/local/mosquitto-1.6.9/bin/* /usr/local/bin/
ln -s /usr/local/mosquitto-1.6.9/sbin/* /usr/local/bin/

echo /usr/local/mosquitto-1.6.9/lib/ >> /etc/ld.so.conf
ldconfig

创建用户:

useradd -c "mosquitto server user" -d /var/lib/mosquitto -s /sbin/nologin  mosquitto

创建配置文件:

cp /etc/mosquitto/mosquitto.conf.example /etc/mosquitto/mosquitto.conf
touch /etc/mosquitto/pwfile /etc/mosquitto/aclfile /var/log/mosquitto.log
chown mosquitto:mosquitto /var/log/mosquitto.log

sed -i '588s/#log_dest stderr/log_dest file \/var\/log\/mosquitto.log/' /etc/mosquitto/mosquitto.conf
sed -i '669s/# password_file/password_file \/etc\/mosquitto\/pwfile/' /etc/mosquitto/mosquitto.conf
sed -i '728s/#acl_file/acl_file \/etc\/mosquitto\/aclfile/' /etc/mosquitto/mosquitto.conf
sed -i '651s/#allow_anonymous true/allow_anonymous false/' /etc/mosquitto/mosquitto.conf

14.2.2. 创建系统服务

cat << EOF > /usr/lib/systemd/system/mosquitto.service
[Unit]
Description=Eclipse Mosquitto - An open source MQTT broker
After=network.target remote-fs.target nss-lookup.target

[Service]
Type=forking

User=mosquitto
Group=mosquitto
ExecStart=/usr/local/mosquitto-1.6.9/sbin/mosquitto -d -c /etc/mosquitto/mosquitto.conf -v

[Install]
WantedBy=multi-user.target
EOF

systemctl enable mosquitto

启动服务:

systemctl start mosquitto
systemctl status mosquitto

14.2.3. 权限设置

创建用户 foo:

mosquitto_passwd -b /etc/mosquitto/pwfile foo 'ugnCy55jZpJj0Xnh0ucS'

另外,可以在 '/etc/mosquitto/aclfile' 文件中设置用户可读写的 topic。比如:

cat << EOF > /etc/mosquitto/aclfile
# This affects access control for clients with no username.
topic read \$SYS/#

# This affects all clients.
pattern readwrite #
EOF

以上权限设置表示 $SYS/# 可以被所有客户端只读,pattern readwrite # 表示所有客户端可以访问任何topic。

详情参考 /etc/mosquitto/aclfile.example

在主配置中启用了权限控制(/etc/mosquitto/aclfile )之后,必须配置规则。否则,客户端无法推送消息。

15. Terminal

15.1. 快捷键

Keyboard shortcuts are combinations of keys that allow you to perform actions, such as opening the settings dialog or accessing a feature inside Terminal quickly. These shortcuts can be modified to suit your preferences.

To change a keyboard shortcut:

  1. Press the menu button in the top-right corner of the window and select Preferences.

  2. In the sidebar, select Shortcuts.

  3. Make sure Enable shortcuts is selected.

  4. Select the shortcut that you wish to edit by clicking on it.

  5. Once the shortcut is selected, click on the shortcut key combination to edit it.

  6. Press your desired shortcut key combination as you would enter it to use it. The keys that you can use include Alt, Alt Gr, Ctrl, Shift, Super, number and letter keys.

Mnemonic key combinations, such as Alt+F will not work.
  1. Once you have entered your new shortcut, it will automatically be saved and you should see it in the list next to the corresponding action.

To disable a shortcut, edit it and press the Backspace instead of the new shortcut.

15.1.1. File shortcuts

15.1.2. Edit shortcuts

15.1.3. View shortcuts

15.1.4. Search shortcuts

15.1.5. Tab shortcuts

15.1.6. Other

15.1.7. Bash shortcuts

These are Bash shortcuts. Bash is usually the default shell.

Bash shell specific keyboard shortcuts are:

16. FFmpeg

16.1. 安装最新版FFmpeg

RPM Fusion 仓库的 ffmpeg 版本太老

16.1.1. 安装静态构建版

以下内容根据 Linux Static Builds 整理
mkdir -p ~/downloads
cd ~/downloads

wget https://johnvansickle.com/ffmpeg/releases/ffmpeg-release-amd64-static.tar.xz -O ffmpeg-release-amd64-static.tar.xz
tar xf ffmpeg-release-amd64-static.tar.xz

rm -rf /usr/local/ffmpeg-6.0
mv ffmpeg-6.0-amd64-static /usr/local/ffmpeg-6.0/

mkdir -p /usr/local/ffmpeg-6.0/bin
mv /usr/local/ffmpeg-6.0/ffmpeg /usr/local/ffmpeg-6.0/bin/
mv /usr/local/ffmpeg-6.0/ffprobe /usr/local/ffmpeg-6.0/bin/
mv /usr/local/ffmpeg-6.0/qt-faststart /usr/local/ffmpeg-6.0/bin/

16.1.2. 查看版本

/usr/local/ffmpeg-6.0/bin/ffmpeg -version
屏幕输出
ffmpeg version 6.0-static https://johnvansickle.com/ffmpeg/  Copyright (c) 2000-2023 the FFmpeg developers
built with gcc 8 (Debian 8.3.0-6)
configuration: --enable-gpl --enable-version3 --enable-static --disable-debug --disable-ffplay --disable-indev=sndio --disable-outdev=sndio --cc=gcc --enable-fontconfig --enable-frei0r --enable-gnutls --enable-gmp --enable-libgme --enable-gray --enable-libaom --enable-libfribidi --enable-libass --enable-libvmaf --enable-libfreetype --enable-libmp3lame --enable-libopencore-amrnb --enable-libopencore-amrwb --enable-libopenjpeg --enable-librubberband --enable-libsoxr --enable-libspeex --enable-libsrt --enable-libvorbis --enable-libopus --enable-libtheora --enable-libvidstab --enable-libvo-amrwbenc --enable-libvpx --enable-libwebp --enable-libx264 --enable-libx265 --enable-libxml2 --enable-libdav1d --enable-libxvid --enable-libzvbi --enable-libzimg
libavutil      58.  2.100 / 58.  2.100
libavcodec     60.  3.100 / 60.  3.100
libavformat    60.  3.100 / 60.  3.100
libavdevice    60.  1.100 / 60.  1.100
libavfilter     9.  3.100 /  9.  3.100
libswscale      7.  1.100 /  7.  1.100
libswresample   4. 10.100 /  4. 10.100
libpostproc    57.  1.100 / 57.  1.100

16.2. 命令合集

16.2.1. mp4转换为mp3

ffmpeg -i 'BLACKPINK - Really.mp4'  -vn -ar 44100 -ac 2 -b:a 192k 'BLACKPINK - Really.mp3'

16.2.2. DVD-ISO转换为 matroska 视频格式

在GNOME文件管理窗口选中 BoA_Who’s back 2014 Live Tour_Blu-ray.iso (DVD-ISO)文件,右键选择 “磁盘映像挂载器”,将ISO文件挂载好。

找到挂载目录下的 .m2ts 文件( STREAM 目录下),人工筛选出需要的文件,写入到 m2ts_list.txt 文件。

不是所有 .m2ts 文件都需要,有些是DVD片头或播放菜单
echo "file '/run/media/BOA_LIVE_TOUR2014/BDMV/STREAM/00004.m2ts'" > m2ts_list.txt
echo "file '/run/media/BOA_LIVE_TOUR2014/BDMV/STREAM/00005.m2ts'" >> m2ts_list.txt

ffmpeg -f concat -safe 0 -i m2ts_list.txt  -vcodec copy -acodec pcm_s24le output.mkv
参数解释
pcm_s24le

pcm_bluray 音频编码只能用于m2ts文件,mkv文件格式最多支持到 pcm_s24le

-vcodec copy

保留视频原始比特率,保持画面质量

m2ts_list.txt

多输入文件需要使用到清单文件,官方文档 https://trac.ffmpeg.org/wiki/Concatenate

17. Git

17.1. 配置 Git 客户端

git config …​…​

项目级设置

git config --global …​…​

系统级设置

git config --global core.editor vim

全局默认编辑器

git config --global user.name "foo"

全局用户名称

git config --global user.email foo@example.com

全局Email

git config --global core.quotepath false

git status 显示中文

查看配置列表
$ git config --list
core.packedgitlimit=100m
core.packedgitwindowsize=100m
core.quotepath=false
core.compression=9
core.editor=vim
user.name=foo
user.email=foo@example.com
pack.deltacachesize=100m
pack.packsizelimit=100m
pack.windowmemory=100m
pack.threads=1
pack.window=0
commit.template=/home/mk/.gitmessage.txt

17.2. commit 模板

  1. 设置全局默认编辑器: git config --global core.editor vim

  2. 设置全局 commit 模板: git config --global commit.template ~/.gitmessage.txt

  3. 执行 git commit 即可使用 commit 模板

gitmessage.txt 文件内容
# Type(<scope>): <subject>

# <body>

# <footer>

# ** TYPES **
# feat (new feature)
# fix (bug fix)
# docs (changes to documentation)
# style (formatting, missing semi colons, etc; no code change)
# refactor (refactoring production code)
# perf (improves performance)
# revert (revert commit)
# test (adding missing tests, refactoring tests; no production code change)
# chore (updating grunt tasks etc; no production code change)
#
# ** FOOTERS **
# References #1, #4, and #2.
# Fix #1. note this marks the item as accepted in Sprintly
# Closes #1 and #2. note this marks the item as accepted in Sprintly

# Scope is just the scope of the change. Something like (admin) or (teacher).
# Subject should use impertivite tone and say what you did.
# The body should go into detail about changes made.
# The footer should contain any JIRA (or other tool) issue references or actions.

# e.g.,
# revert: feat(pencil): add 'graphiteWidth' option
# This reverts commit 667ecc1654a317a13331b17617d973392f415f02.

17.3. 修改 commit 邮箱

本方法适用于Python 3.5+

17.3.1. 安装依赖

安装Python插件:

pip install git-filter-repo

ls ~/.local/bin/git-filter-repo

设置环境变量:

echo 'export PATH=$PATH:~/.local/bin' >> ~/.bashrc

生效环境变量:

source ~/.bashrc

17.3.2. 更新commit中的邮箱

# 请自行替换以下信息
echo '作者 <新邮箱> <旧邮箱>' > ~/.mailmap

git filter-repo --mailmap ~/.mailmap --force

git push origin  master --force

17.4. git status 不显示中文

问题描述
$ git status
On branch dev
Untracked files:
  (use "git add <file>..." to include in what will be committed)

    "\346\265\213\350\257\225\344\270\255\346\226\207"

nothing added to commit but untracked files present (use "git add" to track)
解决方法
git config --global core.quotepath false
测试
$ git status
On branch dev
Untracked files:
  (use "git add <file>..." to include in what will be committed)

    测试

nothing added to commit but untracked files present (use "git add" to track)

17.5. 修改远程URL为SSH协议地址

修改前
$ git remote -v
origin  https://github.com/USERNAME/REPOSITORY.git (fetch)
origin  https://github.com/USERNAME/REPOSITORY.git (push)

设置远程仓库: git remote set-url origin git@github.com:USERNAME/OTHERREPOSITORY.git

修改前
$ git remote -v
origin  git@github.com:USERNAME/OTHERREPOSITORY.git (fetch)
origin  git@github.com:USERNAME/OTHERREPOSITORY.git (push)

17.5.1. 延伸

  • git remote add origin git@github.com:USERNAME/OTHERREPOSITORY.git

  • git remote remove origin

  • git remote add test git@github.com:USERNAME/OTHERREPOSITORY.git

  • git remote remove test

以上命令可以增加和移除远程仓库,一个项目可以同时拥有多个远程仓库。仅需要在推送时,指定远程仓库即可,比如

git push test master

18. 字符编码

18.1. 字符编码

18.2. 字符编码实例

18.3. 乱码

18.4. 实用工具

[mk@archlinux test]$ echo 测试测试>out.txt
[mk@archlinux test]$ file out.txt
out.txt: UTF-8 Unicode text

[mk@archlinux test]$ iconv -f utf-8 -t gb2312 out.txt -o out2.txt
[mk@archlinux test]$ file *
out2.txt: ISO-8859 text
out.txt:  UTF-8 Unicode text
[mk@archlinux test]$ hexdump -C out2.txt
00000000  b2 e2 ca d4 0a b2 e2 ca  d4 0a                    |..........|
0000000a


[mk@archlinux test]$ uconv -f utf-8 -t utf-8 --add-signature -o out2.txt out.txt
[mk@archlinux test]$ file *
out2.txt: UTF-8 Unicode (with BOM) text
out.txt:  UTF-8 Unicode text

[mk@archlinux test]$ hexdump -C out2.txt
00000000  ef bb bf e6 b5 8b e8 af  95 0a e6 b5 8b e8 af 95  |................|
00000010  0a                                                |.|
00000011


[mk@archlinux test]$ sudo pacman -F /usr/bin/iconv usr/bin/convmv
[sudo] mk 的密码:
usr/bin/iconv 由 core/glibc 2.30-2 所拥有
usr/bin/convmv 由 extra/convmv 2.05-1 所拥有

[mk@archlinux test]$ sudo pacman -F uconv
core/icu 65.1-2 [已安装]
    usr/bin/uconv


dos2unix out2.txt
file *

unix2dos -m out2.txt
file *

[mk@archlinux test]$ sudo pacman -F /usr/bin/dos2unix /usr/bin/unix2dos
usr/bin/dos2unix 由 community/dos2unix 7.4.1-1 所拥有
usr/bin/unix2dos 由 community/dos2unix 7.4.1-1 所拥有


hexdump

[mk@archlinux test]$ sudo pacman -F base64
[sudo] mk 的密码:
core/coreutils 8.31-3 [已安装]

[mk@archlinux test]$ base64 out.txt
5rWL6K+VCg==

[mk@archlinux test]$ echo ok|base64
b2sK

[mk@archlinux Pictures]$ cat '2019-12-29 00-52-11屏幕截图.png'|more
�PNG
�


���0^L
--更多--


[mk@archlinux Pictures]$ cat '2019-12-29 00-52-11屏幕截图.png'|base64
iVBORw0KGgoAAAANSUhEUgAAAZkAAAJTCAYAAADaPmuBAAAABHNCSVQICAgIfAhkiAAAABl0RVh0
U29mdHdhcmUAZ25vbWUtc2NyZWVuc2hvdO8Dvz4AACAASURBVHic7L3bkiS9jSb4AXSPrJLU3WO7
8wZzuXf7GG17MQ+5ZvNua9ajVutXVWY4CewFCDqdQT/FISt/SSiLikg/kCCdjg8AQZD+llT/3//1
v/Dv//7vGIcB35QATeCkUFZgAISAQQNUFS0R0eJvVS3HiKh7T+8+Ebk5tnZtW9/aNfUxEVkcWytz
jd+6Xb3y6/u2+H0WrdV9D6kqmLn8bsuuz6/V1z73+rpe37W015dH23jvczjC49l6e+W17Xh0rNzT

19. GCC

19.1. 常用技巧

我以CentOS6编译安装mosquitto为例。

CentOS6系统自带的OpenSSL版本太老,需要手动安装新版。
mkdir ~/downloads
cd downloads
wget -c https://www.openssl.org/source/openssl-1.1.1d.tar.gz
tar xf openssl-1.1.1d.tar.gz
cd openssl-1.1.1d
./config --prefix=/usr/local/openssl-1.1.1d
make -j4
make install
配置动态链接运行时(全局有效)
echo "/usr/local/openssl-1.1.1d/lib/" > /etc/ld.so.conf.d/openssl_1.1.1.conf
ldconfig
ldconfig -p|grep ssl
mkdir ~/downloads
cd ~/downloads
wget -O mosquitto-1.6.9.tar.gz https://mosquitto.org/files/source/mosquitto-1.6.9.tar.gz
tar xf mosquitto-1.6.9.tar.gz

export PKG_CONFIG_PATH=$PKG_CONFIG_PATH:/usr/local/openssl-1.1.1d/lib/pkgconfig/

cd mosquitto-1.6.9
CFLAGS=`pkg-config --cflags-only-I openssl` LDFLAGS=`pkg-config --libs-only-L openssl` LIBS=`pkg-config --libs-only-l openssl` make
make install prefix=/usr/local/mosquitto-1.6.9

20. Java

20.1. javac、jar和classpath

20.1.1. 运行jar包

java -jar target/gs-spring-boot-0.1.0.jar

20.1.2. 编译运行java

源代码结构
# tree  statistics/
statistics/
└── src
    ├── connections
    │   ├── ConnectionPool.class
    │   ├── ConnectionPool.java
    │   ├── DBManager.java
    │   └── PooledConnection.java
    ├── jdbc
    │   └── JDBCOperation.java
    ├── module
    │   ├── CompanyInfo.java
    │   ├── Domain.java
    │   └── ResourceLog.java
    └── statistics
        └── Test.java
编译
cd statistics
javac -cp src/:/usr/share/java/postgresql-jdbc-8.4.704.jar -d bin/ src/statistics/Test.java
运行
cd statistics
java -cp ./bin:/usr/share/java/postgresql-jdbc-8.4.704.jar statistics.Test
class结构
# tree bin/
    bin/
    ├── connections
    │   ├── ConnectionPool.class
    │   ├── DBManager.class
    │   └── PooledConnection.class
    ├── jdbc
    │   └── JDBCOperation.class
    ├── module
    │   ├── CompanyInfo.class
    │   ├── Domain.class
    │   └── ResourceLog.class
    └── statistics
        └── Test.class

20.2. Spring

20.3. SpringBoot

20.3.2. 运行时指定配置模式(dev/prod/etc)

通过启用不同的配置文件载入指定的配置参数。这种方式非常适合有多种配置的应用,通过特殊的运行时环境指定配置文件。

在SpringBoot中,你可以在 `application.yml ` 所在目录下创建多个 `application-{profile}.yml ` 文件达到目的。指定的配置文件会覆盖默认设置。如果指定多个配置文件,最后一个有效。

假如我的应用有两个环境,例如 生产环境开发环境。这时,只需要创建两个配置文件即可:

application-dev.yml
logging:
  file: logs/application-debug.log
  pattern:
    console: "%d %-5level %logger : %msg%n"
    file: "%d %-5level [%thread] %logger : %msg%n"
  level:
    org.springframework.web: ERROR
    com.howtodoinjava: DEBUG
    org.hibernate: ERROR
application-prod.yml
logging:
  file: logs/application-debug.log
  pattern:
    console: "%d %-5level %logger : %msg%n"
    file: "%d %-5level [%thread] %logger : %msg%n"
  level:
    org.springframework.web: ERROR
    com.howtodoinjava: INFO
    org.hibernate: ERROR

在运行时增加参数 spring.profiles.active,可以指定启用的配置文件。

$ java -jar -Dspring.profiles.active=prod spring-boot-demo.jar

20.3.3. 开启调试日志

java -jar -Ddebug=true target/server-1.0-SNAPSHOT.jar
java -jar -Dtrace=true target/server-1.0-SNAPSHOT.jar

20.3.4. 开启Web访问日志

java -jar -Dserver.tomcat.basedir=tomcat -Dserver.tomcat.accesslog.enabled=true target/server-1.0-SNAPSHOT.jar

21. GitLab

21.1. CentOS7安装配置

curl -s https://packages.gitlab.com/install/repositories/gitlab/gitlab-ce/script.rpm.sh | bash

EXTERNAL_URL="http://192.168.2.8:8282"  yum install -y gitlab-ce
安装信息
It looks like GitLab has not been configured yet; skipping the upgrade script.

       *.                  *.
      ***                 ***
     *****               *****
    .******             *******
    ********            ********
   ,,,,,,,,,***********,,,,,,,,,
  ,,,,,,,,,,,*********,,,,,,,,,,,
  .,,,,,,,,,,,*******,,,,,,,,,,,,
      ,,,,,,,,,*****,,,,,,,,,.
         ,,,,,,,****,,,,,,
            .,,,***,,,,
                ,*,.



     _______ __  __          __
    / ____(_) /_/ /   ____ _/ /_
   / / __/ / __/ /   / __ `/ __ \
  / /_/ / / /_/ /___/ /_/ / /_/ /
  \____/_/\__/_____/\__,_/_.___/


Thank you for installing GitLab!
GitLab was unable to detect a valid hostname for your instance.
Please configure a URL for your GitLab instance by setting `external_url`
configuration in /etc/gitlab/gitlab.rb file.
Then, you can start your GitLab instance by running the following command:
  sudo gitlab-ctl reconfigure

For a comprehensive list of configuration options please see the Omnibus GitLab readme
https://gitlab.com/gitlab-org/omnibus-gitlab/blob/master/README.md


GitLab now ships with a newer version of PostgreSQL (11.7), but it is not yet
enabled by default. To upgrade, please see:
https://docs.gitlab.com/omnibus/settings/database.html#upgrade-packaged-postgresql-server

  Verifying  : gitlab-ce-12.9.0-ce.0.el7.x86_64                                                                                                           1/1

Installed:
  gitlab-ce.x86_64 0:12.9.0-ce.0.el7

Complete!

重新配置(恢复默认配置):

gitlab-ctl reconfigure

重启服务:

gitlab-ctl restart

安装完毕后,默认访问80端口即可:

默认登录用户 root,密码第一次访问会提示修改。

21.2. CentOS8 安装配置

21.2.1. 安装依赖

dnf install -y curl policycoreutils openssh-server
# Check if opening the firewall is needed with: systemctl status firewalld
firewall-cmd --permanent --add-service=http
firewall-cmd --permanent --add-service=https
systemctl reload firewalld


dnf install -y postfix
systemctl enable postfix
systemctl start postfix

21.2.2. 安装GitLab

curl -s https://packages.gitlab.com/install/repositories/gitlab/gitlab-ce/script.rpm.sh | bash

EXTERNAL_URL="http://0.0.0.0:8000"  dnf install -y gitlab-ce

一般情况下,GitLab的CDN下载会比较慢,可以找出RPM包的URL,单独下载。然后,手动 dnf install RPM包名称

curl -s https://packages.gitlab.com/install/repositories/gitlab/gitlab-ce/script.rpm.sh | bash

yumdownloader --urls gitlab-ce
Repository epel is listed more than once in the configuration
Extra Packages for Enterprise Linux Modular 8 - x86_64                                                                                                        2.7 kB/s |  10 kB     00:03
Extra Packages for Enterprise Linux Modular 8 - x86_64                                                                                                         28 kB/s | 116 kB     00:04
gitlab_gitlab-ce                                                                                                                                               77  B/s | 862  B     00:11
gitlab_gitlab-ce-source                                                                                                                                       100  B/s | 862  B     00:08
https://packages.gitlab.com/gitlab/gitlab-ce/el/8/x86_64/gitlab-ce-12.10.2-ce.0.el8.x86_64.rpm
使用Wget断点续传,可中断后继续下载
wget -c https://packages.gitlab.com/gitlab/gitlab-ce/el/8/x86_64/gitlab-ce-12.10.2-ce.0.el8.x86_64.rpm
如果下载慢,可以多次中断。然后,再执行wget下载。也可以使用 axel -n url 多线程下载。
EXTERNAL_URL="http://0.0.0.0:8000"  dnf install -y gitlab-ce-12.10.2-ce.0.el8.x86_64.rpm
安装信息
Repository epel is listed more than once in the configuration
Last metadata expiration check: 0:00:44 ago on Sat 02 May 2020 01:42:36 PM CST.
Dependencies resolved.
=================================================================================================
 Package          Architecture          Version          Repository          Size
=================================================================================================
Installing:
 gitlab-ce        x86_64                12.10.2-ce.0.el8 @commandline        789 M

Transaction Summary
=================================================================================================
Install  1 Package

Total size: 789 M
Installed size: 1.7 G
Downloading Packages:
Running transaction check
Transaction check succeeded.
Running transaction test
Transaction test succeeded.
Running transaction
  Preparing        :                                                                                                      1/1
  Running scriptlet: gitlab-ce-12.10.2-ce.0.el8.x86_64                                                                    1/1
  Installing       : gitlab-ce-12.10.2-ce.0.el8.x86_64                                                                    1/1
  Running scriptlet: gitlab-ce-12.10.2-ce.0.el8.x86_64                                                                    1/1
It looks like GitLab has not been configured yet; skipping the upgrade script.

       *.                  *.
      ***                 ***
     *****               *****
    .******             *******
    ********            ********
   ,,,,,,,,,***********,,,,,,,,,
  ,,,,,,,,,,,*********,,,,,,,,,,,
  .,,,,,,,,,,,*******,,,,,,,,,,,,
      ,,,,,,,,,*****,,,,,,,,,.
         ,,,,,,,****,,,,,,
            .,,,***,,,,
                ,*,.



     _______ __  __          __
    / ____(_) /_/ /   ____ _/ /_
   / / __/ / __/ /   / __ `/ __ \
  / /_/ / / /_/ /___/ /_/ / /_/ /
  \____/_/\__/_____/\__,_/_.___/


Thank you for installing GitLab!
GitLab was unable to detect a valid hostname for your instance.
Please configure a URL for your GitLab instance by setting `external_url`
configuration in /etc/gitlab/gitlab.rb file.
Then, you can start your GitLab instance by running the following command:
  sudo gitlab-ctl reconfigure

For a comprehensive list of configuration options please see the Omnibus GitLab readme
https://gitlab.com/gitlab-org/omnibus-gitlab/blob/master/README.md


  Verifying        : gitlab-ce-12.10.2-ce.0.el8.x86_64                                                                    1/1

Installed:
  gitlab-ce-12.10.2-ce.0.el8.x86_64

Complete!

重新配置(恢复默认配置):

gitlab-ctl reconfigure

重启服务:

gitlab-ctl restart

安装完毕后,默认访问80端口即可:

默认登录用户 root,密码第一次访问会提示修改。

21.3. Nginx反代GitLab

安装GitLab之后,通过单独的Nginx反代,暴露给外网访问。由于Nginx配置和GitLab内部的访问端口不同,为保证功能正常使用,需要修改GitLab配置。

21.3.1. GitLab 设置

# 禁用默认配置
sed -i 's/^external_url /#external_url /g' /etc/gitlab/gitlab.rb

cat << EOF >> /etc/gitlab/gitlab.rb
# 禁止k8s相关模块
gitlab_kas['enable'] = false

# 禁止默认功能设置
gitlab_rails['gitlab_default_projects_features_wiki'] = false
gitlab_rails['gitlab_default_projects_features_builds'] = false

gitlab_rails['gitlab_host'] = 'git.xxx.com'
gitlab_rails['gitlab_port'] = 443
gitlab_rails['gitlab_https'] = true

gitlab_rails['gitlab_ssh_host'] = 'git.xxx.com'
gitlab_rails['gitlab_shell_ssh_port'] = 22

# 禁用自动签发免费HTTPS证书
letsencrypt['enable'] = false

# 禁用签发证书后,需要显式启用Nginx服务
nginx['enable'] = true
# 禁用Nginx的HTTPS配置
nginx['redirect_http_to_https'] = false
nginx['listen_https'] = false

# Nginx端口监听配置
nginx['listen_addresses'] = ['127.0.0.1']
nginx['listen_port'] = 8000

git_data_dirs({
    "default" => {
        "path" => "/data/gitlab_data"
    }
})
EOF

# 重新生成配置并重启
gitlab-ctl reconfigure && gitlab-ctl restart

# 确认Nginx服务状态
ss -antpl|grep 8000

21.3.2. 设置HTTP Clone相关参数

用管理员帐号登录GitLab,进入以下菜单:

Admin Area > Settings > General > Visibility and access controls

找到 "Custom Git clone URL for HTTP(S) ",根据实际情况填写,比如:https://git.xxx.com:2305

21.3.3. Nginx Server配置

server {
    listen 2035 ssl http2;
    server_name git.xxx.com;

    # certs sent to the client in SERVER HELLO are concatenated in ssl_certificate
    ssl_certificate /etc/letsencrypt/live/xxx.com/fullchain.pem;
    ssl_certificate_key /etc/letsencrypt/live/xxx.com/privkey.pem;
    ssl_session_timeout 1d;
    ssl_session_cache shared:SSL:50m;
    ssl_session_tickets off;

    # Diffie-Hellman parameter for DHE ciphersuites, recommended 2048 bits
    ssl_dhparam /etc/letsencrypt/live/xxx.com/dhparam.pem;

    # intermediate configuration. tweak to your needs.
    ssl_protocols TLSv1 TLSv1.1 TLSv1.2 TLSv1.3;
    ssl_ciphers 'ECDHE-ECDSA-CHACHA20-POLY1305:ECDHE-RSA-CHACHA20-POLY1305:ECDHE-ECDSA-AES128-GCM-SHA256:ECDHE-RSA-AES128-GCM-SHA256:ECDHE-ECDSA-AES256-GCM-SHA384:ECDHE-RSA-AES256-GCM-SHA384:DHE-RSA-AES128-GCM-SHA256:DHE-RSA-AES256-GCM-SHA384:ECDHE-ECDSA-AES128-SHA256:ECDHE-RSA-AES128-SHA256:ECDHE-ECDSA-AES128-SHA:ECDHE-RSA-AES256-SHA384:ECDHE-RSA-AES128-SHA:ECDHE-ECDSA-AES256-SHA384:ECDHE-ECDSA-AES256-SHA:ECDHE-RSA-AES256-SHA:DHE-RSA-AES128-SHA256:DHE-RSA-AES128-SHA:DHE-RSA-AES256-SHA256:DHE-RSA-AES256-SHA:ECDHE-ECDSA-DES-CBC3-SHA:ECDHE-RSA-DES-CBC3-SHA:EDH-RSA-DES-CBC3-SHA:AES128-GCM-SHA256:AES256-GCM-SHA384:AES128-SHA256:AES256-SHA256:AES128-SHA:AES256-SHA:DES-CBC3-SHA:!DSS';
    ssl_prefer_server_ciphers on;

    # HSTS (ngx_http_headers_module is required) (15768000 seconds = 6 months)
    add_header Strict-Transport-Security max-age=15768000;

    # OCSP Stapling ---
    # fetch OCSP records from URL in ssl_certificate and cache them
    ssl_stapling on;
    ssl_stapling_verify on;

    # 替换URL,解决README.md之类无法加载的问题
    sub_filter 'http://git.xxx.com'  'https://git.xxx.com';
    sub_filter_once off;
    sub_filter_last_modified on;
    # 替换 text/html application/json HTTP响应包中的内容
    sub_filter_types application/json;

    location / {
        auth_basic           "Administrator’s Area";
        auth_basic_user_file /etc/nginx/.htpasswd;

        # 一旦当前作用域设置proxy_set_header,全局设置即刻失效,注意补全所有需要的Header
        # 必须设置,否则新建项目后,会302跳转到http的80端口
        proxy_set_header Host git.xxx.com:443;
        proxy_set_header X-Real-IP $remote_addr;
        proxy_set_header X-Forwarded-Proto https;
        proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;

        proxy_pass http://127.0.0.1:1034;
    }

    location ~ ^/\.git/? {
        return 404;
    }
}

生效Nginx配置:

nginx -t && nginx -s reload

21.3.4. 最后

外网GitLab访问地址

https://git.xxx.com

SSH克隆地址

ssh://git@git.xxx.com/用户名/项目名称.git

HTTP克隆地址

https://git.xxx.com/用户名/项目名称.git

22. Redis

22.1. 安装配置

yum install -y redis
systemctl enable redis

systemctl start redis

22.2. 常用命令

22.2.1. 键(key)

set name geek

get name

del name

exists name

# 为给定 key 设置过期时间,以秒计
expire name

# 查找所有符合给定模式( pattern)的 key
keys rts:*

22.2.2. 字符串(String)

# 同时设置一个或多个 key-value 对

mset key1 value1 [key2 value2]

# 获取所有(一个或多个)给定 key 的值
mget key1 [key2]

# 将 key 中储存的数字值增一
incr key
# 将 key 中储存的数字值减一
decr key

# 将 key 所储存的值加上给定的增量值(increment)
incrby key increment
# key 所储存的值减去给定的减量值(decrement)
decrby key decrement

22.2.3. 列表(List)

# 将一个或多个值插入到列表头部
lpush key_list value1 value2
lpush key_list value3

# 获取列表长度
llen key_list


LRANGE runoobkey 0 10

# 移出并获取列表的第一个元素
lpop key_list

22.2.4. 集合(Set)

Redis 的 Set 是 String 类型的无序集合。集合成员是唯一的,这就意味着集合中不能出现重复的数据。

Redis 中集合是通过哈希表实现的,所以添加,删除,查找的复杂度都是 O(1)。

# 向集合添加一个或多个成员
sadd key_set  value1

# 获取集合的成员数
scard key

# 返回集合中的所有成员
smembers key

22.2.5. 有序集合(sorted set)

Redis 有序集合和集合一样也是string类型元素的集合,且不允许重复的成员。

ZADD leaderboard:455 100 user:1
ZADD leaderboard:455 75 user:2
ZADD leaderboard:455 101 user:3
ZADD leaderboard:455 15 user:4
ZADD leaderboard:455 275 user:2


ZRANGE leaderboard:455 0 2 REV WITHSCORES

ZREVRANK leaderboard:455 user:2

22.2.6. 哈希表(Hash)

增加字段或设置字段

向哈希表中增加字段或设置字段值:

hset area_info name "四川"

向哈希表中增加多个字段或设置多个字段值:

hmset area_info name "四川" level 2
获取字段

获取哈希表中指定键的单个字段和值:

hget area_info name

获取所有给定字段的值:

hmget area_info name level

获取哈希表中指定键的所有字段和值:

hgetall area_info
字段是否存在

查看哈希表的指定字段是否存在:

hexists area_info name
删除字段

删除一个或多个哈希表字段:

hdel area_info name level

22.2.7. 删除键

删除一个或多个键:

del area_info_a area_info_b

删除模糊匹配的键:

redis-cli --scan --pattern users:* | xargs redis-cli unlink

23. PHP

23.1. CentOS6使用Yum安装PHP5.3

yum install -y mysql mysql-server httpd php-bcmath php-cli php-common php-gd php-imap php-intl php-mbstring php-mysql php-pdo php-pear php-tidy php-xml php-xmlrpc php-fpm

23.2. CentOS6源码安装PHP5.5

mkdir ~/install
cd ~/install
wget -c http://mirrors.opencas.cn/epel/epel-release-latest-6.noarch.rpm
wget -c http://cn2.php.net/get/php-5.3.29.tar.bz2/from/this/mirror -O php-5.3.29.tar.bz2

yum install -y epel-release

yum install -y gcc zlib-devel freetype-devel gd-devel libmcrypt-devel curl-devel libtool libxml2-devel gdbm-devel libjpeg-devel libpng-devel curl-devel libc-client libtidy pcre-devel openssl-devel net-snmp net-snmp-devel net-snmp-utils net-snmp-libs libevent-devel mhash-devel patch  libtool-ltdl-devel mysql-devel mysql-server

#CentOS7
#yum install -y mariadb mariadb-devel mariadb-libs mariadb-server lbzip2



cd ~/install
tar xf php-5.3.29.tar.bz2
cd php-5.3.29

#Nginx+php-fpm
./configure --prefix=/usr/local/php-5.3.29 --with-libdir=lib64 --with-png-dir --with-jpeg-dir --with-gd --with-freetype-dir --with-mcrypt --with-config-file-path=/etc --disable-debug --enable-sockets --enable-calendar --enable-ftp --enable-gd-native-ttf --with-gdbm --with-gettext --with-iconv --enable-mbstring --with-openssl --with-curl --enable-bcmath --enable-exif --with-pdo-mysql --with-snmp --with-mysqli --enable-zip --with-zlib --with-mhash --enable-xml --with-xmlrpc --with-mysql --enable-fpm --with-fpm-user=nginx --with-fpm-group=nginx

make && make install

cp php.ini-production /usr/local/php/etc/php.ini

cp /usr/local/php/etc/php-fpm.conf.default /usr/local/php/etc/php-fpm.conf

vim +62 /usr/local/php/etc/php-fpm.conf
                        Unix user of processes
                        <value name="user">apache</value>

                        Unix group of processes
                        <value name="group">apache</value>

mkdir /var/log/php

ln -s /usr/local/php/etc/php.ini /etc/
ln -s /usr/local/php/etc/php-fpm.conf /etc/
/etc/init.d/php-fpm  start

23.3. CentOS7 源码安装PHP7-FPM

mkdir ~/install
cd ~/install
wget -c http://php.net/get/php-7.3.7.tar.bz2/from/this/mirror -O php-7.3.7.tar.bz2
wget -c https://libzip.org/download/libzip-1.1.2.tar.gz
wget -c https://www.gnupg.org/ftp/gcrypt/gnutls/v3.5/gnutls-3.5.19.tar.xz
wget -c https://ftp.gnu.org/gnu/nettle/nettle-3.4.tar.gz

tar xf nettle-3.4.tar.gz
cd nettle-3.4/
./configure --prefix=/usr/local/nettle-3.4
make && make install
cd ..

tar xf gnutls-3.5.19.tar.xz
cd gnutls-3.5.19/
export PKG_CONFIG_PATH=/usr/local/nettle-3.4/lib64/pkgconfig/
./configure --prefix=/usr/local/gnutls-3.5.19
make && make install
cd ..

tar xf libzip-1.1.2.tar.gz
cd libzip-1.1.2/
export PKG_CONFIG_PATH=/usr/local/libzip-1.1.2/lib/pkgconfig/
./configure --prefix=/usr/local/libzip-1.1.2
make && make install

yum install -y gcc zlib-devel freetype-devel gd-devel libmcrypt-devel curl-devel libtool libxml2-devel gdbm-devel libjpeg-devel libpng-devel curl-devel libc-client libtidy pcre-devel openssl-devel net-snmp net-snmp-devel net-snmp-utils net-snmp-libs libevent-devel mhash-devel patch mysql-devel libzip-devel systemd-devel

cd ~/install
tar xf php-7.3.7.tar.bz2
cd php-7.3.7

./configure --prefix=/usr/local/php-7.3.7 --with-config-file-path=/usr/local/php-7.3.7/etc --with-config-file-scan-dir=/usr/local/php-7.3.7/etc --with-libdir=lib64 --with-png-dir --with-jpeg-dir --with-gd --with-freetype-dir --with-mcrypt --with-config-file-path=/etc --disable-debug --enable-sockets --enable-calendar --enable-ftp --enable-gd-native-ttf --with-gdbm --with-gettext --with-iconv --enable-mbstring --with-openssl --with-curl --enable-bcmath --enable-exif --with-pdo-mysql --with-snmp --with-mysqli --enable-zip --with-zlib --with-mhash --enable-xml --with-xmlrpc --enable-fpm --with-fpm-user=nginx --with-fpm-group=nginx --with-fpm-systemd

make && make install


ln -s  /usr/local/php-7.3.7/  /usr/local/php

cp php.ini-production /usr/local/php/etc/php.ini

cp /usr/local/php/etc/php-fpm.conf.default /usr/local/php/etc/php-fpm.conf


cat << 'EOF' > /etc/systemd/system/php7-fpm.service
[Unit]
Description=The PHP FastCGI Process Manager
After=network.target

[Service]
Type=notify
PIDFile=/usr/local/php/var/run/php-fpm.pid
ExecStart=/usr/local/php/sbin/php-fpm --nodaemonize --fpm-config /usr/local/php/etc/php-fpm.conf
ExecReload=/bin/kill -USR2 $MAINPID
PrivateTmp=true

[Install]
WantedBy=multi-user.target
EOF

cp /usr/local/php-7.3.7/etc/php-fpm.d/www.conf.default /usr/local/php-7.3.7/etc/php-fpm.d/www.conf

systemctl enable php7-fpm
systemctl start php7-fpm
systemctl status php7-fpm

23.4. PHP5.3使用Pecl安装扩展

配置PECL:

mkdir -p /usr/local/php-5.3.29/modules \
        /usr/local/php-5.3.29/share/doc/pear \
        /usr/local/php-5.3.29/share/pear \
        /usr/local/php-5.3.29/etc/pear \
        /usr/local/php-5.3.29/share/pear-data \
        /usr/local/php-5.3.29/share/tests/pear \
        /usr/local/php-5.3.29/etc/pearkeys

/usr/local/php-5.3.29/bin/pecl config-set bin_dir /usr/local/php-5.3.29/bin
/usr/local/php-5.3.29/bin/pecl config-set ext_dir /usr/local/php-5.3.29/modules
/usr/local/php-5.3.29/bin/pecl config-set doc_dir /usr/local/php-5.3.29/share/doc/pear
/usr/local/php-5.3.29/bin/pecl config-set php_dir /usr/local/php-5.3.29/share/pear
/usr/local/php-5.3.29/bin/pecl config-set cfg_dir /usr/local/php-5.3.29/etc/pear
/usr/local/php-5.3.29/bin/pecl config-set data_dir /usr/local/php-5.3.29/share/pear-data/
/usr/local/php-5.3.29/bin/pecl config-set php_bin /usr/local/php-5.3.29/bin/php
/usr/local/php-5.3.29/bin/pecl config-set test_dir /usr/local/php-5.3.29/share/tests/pear/
/usr/local/php-5.3.29/bin/pecl config-set sig_keydir /usr/local/php-5.3.29/etc/pearkeys
/usr/local/php-5.3.29/bin/pecl config-set Filename /usr/local/php-5.3.29/etc/pear.conf
/usr/local/php-5.3.29/bin/pecl config-set php_ini /usr/local/php-5.3.29/etc/php.ini

# 查看所有PECL配置

# /usr/local/php-5.3.29/bin/pecl config-show

Configuration (channel pecl.php.net):
=====================================
Auto-discover new Channels     auto_discover    <not set>
Default Channel                default_channel  pecl.php.net
HTTP Proxy Server Address      http_proxy       <not set>
PEAR server [DEPRECATED]       master_server    pear.php.net
Default Channel Mirror         preferred_mirror pecl.php.net
Remote Configuration File      remote_config    <not set>
PEAR executables directory     bin_dir          /usr/local/php-5.3.29/bin
PEAR documentation directory   doc_dir          /usr/local/php-5.3.29/share/doc/pear
PHP extension directory        ext_dir          /usr/local/php-5.3.29/modules
PEAR directory                 php_dir          /usr/local/php-5.3.29/share/pear
PEAR Installer cache directory cache_dir        /tmp/pear/cache
PEAR configuration file        cfg_dir          /usr/local/php-5.3.29/etc/pear
directory
PEAR data directory            data_dir         /usr/local/php-5.3.29/share/pear-data/
PEAR Installer download        download_dir     /tmp/pear/download
directory
PHP CLI/CGI binary             php_bin          /usr/local/php-5.3.29/bin/php
php.ini location               php_ini          /usr/local/php-5.3.29/etc/php.ini
--program-prefix passed to     php_prefix       <not set>
PHP's ./configure
--program-suffix passed to     php_suffix       <not set>
PHP's ./configure
PEAR Installer temp directory  temp_dir         /tmp/pear/temp
PEAR test directory            test_dir         /usr/local/php-5.3.29/share/tests/pear/
PEAR www files directory       www_dir          /usr/local/php-5.3.29/lib/php/htdocs
Cache TimeToLive               cache_ttl        3600
Preferred Package State        preferred_state  stable
Unix file mask                 umask            22
Debug Log Level                verbose          1
PEAR password (for             password         <not set>
maintainers)
Signature Handling Program     sig_bin          /usr/bin/gpg
Signature Key Directory        sig_keydir       /usr/local/php-5.3.29/etc/pearkeys
Signature Key Id               sig_keyid        <not set>
Package Signature Type         sig_type         gpg
PEAR username (for             username         <not set>
maintainers)
User Configuration File        Filename         ~/.pearrc
System Configuration File      Filename         /usr/local/php-5.3.29/etc/pear.conf





如果不使用pecl config-set配置bin_dir,会提示找不到phpise命令。

/usr/local/php-5.3.29/bin/pecl install ZendOpcache

echo 'zend_extension=/usr/local/php-5.3.29/lib/php/extensions/no-debug-non-zts-20090626/opcache.so' >> /usr/local/php-5.3.29/etc/php.ini

# 提示输入libmemcached目录时,输入 “/usr”
# memcached-2.2.0 表示老版本,不带版本号表示最新版。最新版需要php7,php5.3只能用2.2.0
yum install -y libmemcached-devel && /usr/local/php-5.3.29/bin/pecl install memcached-2.2.0
echo 'extension=/usr/local/php-5.3.29/lib/php/extensions/no-debug-non-zts-20090626/memcached.so' >> /usr/local/php-5.3.29/etc/php.ini

23.5. PHP5.3 安装 ZendGuardLoader

wget -c http://downloads.zend.com/guard/5.5.0/ZendGuardLoader-php-5.3-linux-glibc23-x86_64.tar.gz
tar xf ZendGuardLoader-php-5.3-linux-glibc23-x86_64.tar.gz
cd ZendGuardLoader-php-5.3-linux-glibc23-x86_64
mv ZendGuardLoader-php-5.3-linux-glibc23-x86_64/php-5.3.x/ZendGuardLoader.so /usr/local/php-5.3.29/lib/php/extensions/no-debug-non-zts-20090626

echo 'zend_extension=/usr/local/php-5.3.29/lib/php/extensions/no-debug-non-zts-20090626/ZendGuardLoader.so'  >> /usr/local/php-5.3.29/etc/php.ini

23.6. Ubuntu18.04源码安装PHP7-FPM

sudo apt-get install aptitude
sudo aptitude install gcc zlib1g-dev libfreetype6-dev libgd-dev libmcrypt-dev libcurl4-openssl-dev libtool libxml2-dev libgdbm-dev libjpeg-dev libpng-dev libc-client2007e libtidy5 libpcre3-dev libssl-dev libevent-dev libmhash-dev patch libltdl-dev libmysqlclient-dev libmysqld-dev mysql-server apache2-dev libsystemd-dev libsnmp-dev libzip-dev


./configure --prefix=/usr/local/php-7.3.7 --with-config-file-path=/usr/local/php-7.3.7/etc --with-config-file-scan-dir=/usr/local/php-7.3.7/etc --with-libdir=lib64 --with-png-dir --with-jpeg-dir --with-gd --with-freetype-dir --with-config-file-path=/etc --disable-debug --enable-sockets --enable-calendar --enable-ftp --with-gdbm --with-gettext --with-iconv --enable-mbstring --with-openssl --with-curl --enable-bcmath --enable-exif --with-pdo-mysql --with-snmp --with-mysqli --enable-zip --with-zlib --with-mhash --enable-xml --with-xmlrpc --enable-fpm --with-fpm-systemd --with-fpm-user=www-data --with-fpm-group=www-data

ake && make install


ln -s  /usr/local/php-7.3.7/  /usr/local/php

cp php.ini-production /usr/local/php/etc/php.ini

cp /usr/local/php/etc/php-fpm.conf.default /usr/local/php/etc/php-fpm.conf


cat << 'EOF' > /etc/systemd/system/php7-fpm.service
[Unit]
Description=The PHP FastCGI Process Manager
After=network.target

[Service]
Type=notify
PIDFile=/usr/local/php/var/run/php-fpm.pid
ExecStart=/usr/local/php/sbin/php-fpm --nodaemonize --fpm-config /usr/local/php/etc/php-fpm.conf
ExecReload=/bin/kill -USR2 $MAINPID
PrivateTmp=true

[Install]
WantedBy=multi-user.target
EOF

cp /usr/local/php-7.3.7/etc/php-fpm.d/www.conf.default /usr/local/php-7.3.7/etc/php-fpm.d/www.conf

systemctl enable php7-fpm
systemctl start php7-fpm
systemctl status php7-fpm

24. Python

24.1. CentOS6 Install Python312

不支持_dbm、tkinter模块

24.1.1. 安装YUM依赖

yum install -y libffi-devel bzip2-devel readline-devel ncurses-devel gdbm-devel tkinter tcl-devel tcl libuuid-devel zlib-devel zlib xz-devel xz tk-devel tk  glibc-devel

24.1.2. 源代码安装

环境准备
mkdir -p ~/downloads
yum install -y gcc gcc-c++ make
编译OpenSSL

CentOS6 YUM源中的OpenSSL版本太老,需要单独安装较新版本。

否则,编译Python3时会出现错误:

Could not build the ssl module!
Python requires an OpenSSL 1.0.2 or 1.1 compatible libssl with X509_VERIFY_PARAM_set1_host().
LibreSSL 2.6.4 and earlier do not provide the necessary APIs, https://github.com/libressl-portable/portable/issues/381
cd  ~/downloads
wget https://www.openssl.org/source/old/1.1.1/openssl-1.1.1n.tar.gz -O openssl-1.1.1n.tar.gz
tar xf openssl-1.1.1n.tar.gz
cd openssl-1.1.1n
./config --prefix=/usr/local/openssl-1.1.1n -fPIC
make
make install

echo '/usr/local/openssl-1.1.1n/lib' > /etc/ld.so.conf.d/openssl-1.1.1n.conf
ldconfig
ldconfig -p|grep openssl-1.1.1n
编译SQLite

同样的,YUM源中的SQLite太老~

cd  ~/downloads
wget https://www.sqlite.org/2020/sqlite-autoconf-3330000.tar.gz -O sqlite-autoconf-3330000.tar.gz
tar xf sqlite-autoconf-3330000.tar.gz
cd sqlite-autoconf-3330000
./configure --prefix=/usr/local/sqlite-3.33
make
make install

echo '/usr/local/sqlite-3.33/lib' > /etc/ld.so.conf.d/sqlite-3.33.conf
ldconfig
ldconfig -p|grep libsqlite3.so
编译Python
cd  ~/downloads
wget https://www.python.org/ftp/python/3.12.2/Python-3.12.2.tar.xz -O Python-3.12.2.tar.xz
tar xf Python-3.12.2.tar.xz
cd Python-3.12.2

CFLAGS='-I/usr/local/sqlite-3.33/include' \
    LDFLAGS='-L/usr/local/sqlite-3.33/lib' \
    ./configure --prefix=/usr/local/python-3.12.2 \
    --enable-optimizations \
    --with-openssl=/usr/local/openssl-1.1.1n \
    --with-ssl-default-suites=openssl \
    --with-ensurepip \
    --enable-loadable-sqlite-extensions
make
make install

24.1.3. 压缩包安装

解压OpenSSL
wget https://dl.cdgeekcamp.com/centos/6/openssl/1.1.1n/openssl-1.1.1n-1.el6.x86_64.tar.gz -O openssl-1.1.1n-1.el6.x86_64.tar.gz
tar xf openssl-1.1.1n-1.el6.x86_64.tar.gz
rm -rf /usr/local/openssl-1.1.1n
mv openssl-1.1.1n/ /usr/local/openssl-1.1.1n

echo '/usr/local/openssl-1.1.1n/lib' > /etc/ld.so.conf.d/openssl-1.1.1n.conf
ldconfig
ldconfig -p|grep openssl-1.1.1n
解压SQLite
wget https://dl.cdgeekcamp.com/centos/6/sqlite/3.33/sqlite-3.33-1.el6.x86_64.tar.gz -O sqlite-3.33-1.el6.x86_64.tar.gz
tar xf sqlite-3.33-1.el6.x86_64.tar.gz
rm -rf /usr/local/sqlite-3.33
mv sqlite-3.33/ /usr/local/sqlite-3.33

echo '/usr/local/sqlite-3.33/lib' > /etc/ld.so.conf.d/sqlite-3.33.conf
ldconfig
ldconfig -p|grep libsqlite3.so
解压Python3
wget https://dl.cdgeekcamp.com/centos/6/python/3/python-3.12.2-1.el6.x86_64.tar.gz -O python-3.12.2-1.el6.x86_64.tar.gz
tar xf python-3.12.2-1.el6.x86_64.tar.gz
rm -rf /usr/local/python-3.12.2
mv python-3.12.2 /usr/local/python-3.12.2

24.1.4. 设置软链接

test -L /usr/local/python3 || ln -s python-3.12.2 /usr/local/python3
test -L /usr/local/python3/bin/pip312 || ln -s pip3 /usr/local/python3/bin/pip312
test -L /usr/local/python3/bin/python312 || ln -s python3 /usr/local/python3/bin/python312

24.1.5. 设置环境变量

通过 pip312 安装的命令包(比如,rain-shell-scripter),命令被安装在 /usr/local/python3/bin 目录下。

为方便,增加环境变量:

echo "export PATH=/usr/local/python3/bin:\${PATH}" > /etc/profile.d/python3.sh

生效环境变量配置,可以退出终端,重新登录即可。

或者,在当前终端执行:

source /etc/profile

确认环境变量:

echo $PATH|sed 's/:/\n/g'|sort -u|grep python

24.1.6. 设置PIP

pip312 install --upgrade -i https://pypi.tuna.tsinghua.edu.cn/simple pip
pip312 config set global.index-url https://pypi.tuna.tsinghua.edu.cn/simple

24.2. CentOS7 Install Python312

不支持tkinter模块

24.2.1. 安装YUM依赖

yum install -y libffi-devel bzip2-devel readline-devel ncurses-devel gdbm-devel tkinter tcl-devel tcl libuuid-devel zlib-devel zlib xz-devel xz tk-devel tk openssl-devel sqlite-devel glibc-devel

24.2.2. 源代码安装

环境准备
mkdir -p ~/downloads
yum install -y gcc gcc-c++ make
编译OpenSSL

CentOS YUM源中的OpenSSL版本太老,需要单独安装较新版本。

cd  ~/downloads
wget https://www.openssl.org/source/old/1.1.1/openssl-1.1.1n.tar.gz -O openssl-1.1.1n.tar.gz
tar xf openssl-1.1.1n.tar.gz
cd openssl-1.1.1n
./config --prefix=/usr/local/openssl-1.1.1n -fPIC
make
make install

echo '/usr/local/openssl-1.1.1n/lib' > /etc/ld.so.conf.d/openssl-1.1.1n.conf
ldconfig
ldconfig -p|grep openssl-1.1.1n
编译Python
cd  ~/downloads
wget https://www.python.org/ftp/python/3.12.2/Python-3.12.2.tar.xz -O Python-3.12.2.tar.xz
tar xf Python-3.12.2.tar.xz
cd Python-3.12.2

./configure --prefix=/usr/local/python-3.12.2 \
    --enable-optimizations \
    --with-openssl=/usr/local/openssl-1.1.1n \
    --with-ssl-default-suites=openssl \
    --with-ensurepip \
    --enable-loadable-sqlite-extensions
make
make install

24.2.3. 压缩包安装

解压OpenSSL
wget https://dl.cdgeekcamp.com/centos/7/openssl/1.1.1n/openssl-1.1.1n-1.el7.x86_64.tar.gz -O openssl-1.1.1n-1.el7.x86_64.tar.gz
tar xf openssl-1.1.1n-1.el7.x86_64.tar.gz
rm -rf /usr/local/openssl-1.1.1n
mv openssl-1.1.1n/ /usr/local/openssl-1.1.1n

echo '/usr/local/openssl-1.1.1n/lib' > /etc/ld.so.conf.d/openssl-1.1.1n.conf
ldconfig
ldconfig -p|grep openssl-1.1.1n
解压Python3
wget https://dl.cdgeekcamp.com/centos/7/python/3/python-3.12.2-1.el7.x86_64.tar.gz -O python-3.12.2-1.el7.x86_64.tar.gz

tar xf python-3.12.2-1.el7.x86_64.tar.gz

rm -rf /usr/local/python-3.12.2
mv python-3.12.2 /usr/local/python-3.12.2

24.2.4. 设置Python

test -L /usr/local/python3 || ln -s python-3.12.2 /usr/local/python3
test -L /usr/local/python3/bin/pip312 || ln -s pip3 /usr/local/python3/bin/pip312
test -L /usr/local/python3/bin/python312 || ln -s python3 /usr/local/python3/bin/python312

echo "export PATH=/usr/local/python3/bin:\${PATH}" > /etc/profile.d/python3.sh
source /etc/profile

pip312 install --upgrade -i https://pypi.tuna.tsinghua.edu.cn/simple pip
pip312 config set global.index-url https://pypi.tuna.tsinghua.edu.cn/simple

24.3. CentOS8 Install Python312

支持tkinter模块

24.3.1. 安装YUM依赖

dnf install -y dnf-plugins-core
# for libnsl2-devel
dnf config-manager --set-enabled powertools
dnf install -y libnsl2-devel

dnf install -y libffi-devel bzip2-devel readline-devel ncurses-devel gdbm-devel tcl-devel tcl libuuid-devel zlib-devel zlib xz-devel xz tk-devel tk openssl-devel sqlite-devel glibc-devel

24.3.2. 源代码安装

mkdir -p ~/downloads
dnf install -y gcc gcc-c++ make

cd  ~/downloads
wget https://www.python.org/ftp/python/3.12.2/Python-3.12.2.tar.xz -O Python-3.12.2.tar.xz
tar xf Python-3.12.2.tar.xz
cd Python-3.12.2

./configure --prefix=/usr/local/python-3.12.2 \
    --enable-optimizations \
    --with-ensurepip \
    --enable-loadable-sqlite-extensions
make
make install

24.3.3. 压缩包安装

wget https://dl.cdgeekcamp.com/centos/8/python/3/python-3.12.2-1.el8.x86_64.tar.gz -O python-3.12.2-1.el8.x86_64.tar.gz

tar xf python-3.12.2-1.el8.x86_64.tar.gz

rm -rf /usr/local/python-3.12.2
mv python-3.12.2 /usr/local/python-3.12.2

24.3.4. 设置Python

test -L /usr/local/python3 || ln -s python-3.12.2 /usr/local/python3
test -L /usr/local/python3/bin/pip312 || ln -s pip3 /usr/local/python3/bin/pip312
test -L /usr/local/python3/bin/python312 || ln -s python3 /usr/local/python3/bin/python312

echo "export PATH=/usr/local/python3/bin:\${PATH}" > /etc/profile.d/python3.sh
source /etc/profile

pip312 install --upgrade -i https://pypi.tuna.tsinghua.edu.cn/simple pip
pip312 config set global.index-url https://pypi.tuna.tsinghua.edu.cn/simple

24.4. CentOS9 Install Python312

支持tkinter模块

24.4.1. 安装YUM依赖

dnf install -y dnf-plugins-core
# for libnsl2-devel and gdbm-devel
dnf config-manager --set-enabled crb
dnf install -y gdbm-devel libnsl2-devel

dnf install -y libffi-devel bzip2-devel readline-devel ncurses-devel tcl-devel tcl libuuid-devel zlib-devel zlib xz-devel xz tk-devel tk openssl-devel sqlite-devel

24.4.2. 源代码安装

mkdir -p ~/downloads
dnf install -y gcc gcc-c++ make

cd  ~/downloads
wget https://www.python.org/ftp/python/3.12.2/Python-3.12.2.tar.xz -O Python-3.12.2.tar.xz
tar xf Python-3.12.2.tar.xz
cd Python-3.12.2

./configure --prefix=/usr/local/python-3.12.2 \
    --enable-optimizations \
    --with-ensurepip \
    --enable-loadable-sqlite-extensions
make
make install

24.4.3. 压缩包安装

wget https://dl.cdgeekcamp.com/centos/9/python/3/python-3.12.2-1.el9.x86_64.tar.gz -O python-3.12.2-1.el9.x86_64.tar.gz

tar xf python-3.12.2-1.el9.x86_64.tar.gz

rm -rf /usr/local/python-3.12.2
mv python-3.12.2 /usr/local/python-3.12.2

24.4.4. 设置Python

test -L /usr/local/python3 || ln -s python-3.12.2 /usr/local/python3
test -L /usr/local/python3/bin/pip312 || ln -s pip3 /usr/local/python3/bin/pip312
test -L /usr/local/python3/bin/python312 || ln -s python3 /usr/local/python3/bin/python312

echo "export PATH=/usr/local/python3/bin:\${PATH}" > /etc/profile.d/python3.sh
source /etc/profile

pip312 install --upgrade -i https://pypi.tuna.tsinghua.edu.cn/simple pip
pip312 config set global.index-url https://pypi.tuna.tsinghua.edu.cn/simple

24.5. Ubuntu Install Python312

支持tkinter模块

24.5.1. 安装DEB依赖

common
apt install -y libssl-dev libffi-dev zlib1g-dev tk-dev libsqlite3-dev libbz2-dev ncurses-dev liblzma-dev uuid-dev libreadline-dev libgdbm-dev
only 18.04/22.04
apt install -y libgdbm-compat-dev

24.5.2. 编译安装

环境准备
mkdir -p ~/downloads
apt install -y gcc make

16.04/18.04系统创建tk-dev包的pkg-config配置文件,解决 _tkinter模块无法安装问题:

mkdir -p ~/downloads/pkgconfig

tk_version=$(dpkg -s tk-dev|grep Version)

cat << EOF > ~/downloads/pkgconfig/tk.pc
# tk pkg-config source file

prefix=/usr
exec_prefix=/usr
libdir=/usr/lib/x86_64-linux-gnu
includedir=/usr/include/tcl8.6

Name: The Tk Toolkit
Description: Tk is a cross-platform graphical user interface toolkit, the standard GUI not only for Tcl, but for many other dynamic languages as well.
URL: https://www.tcl-lang.org/
${tk_version}
Requires: tcl >= 8.6
Libs: -L\${libdir} -ltk8.6 -ltkstub8.6
Libs.private: -lXft -lfontconfig -lfreetype -lfontconfig -lX11 -lXss -lXext
Cflags: -I\${includedir}
EOF

tcl_version=$(dpkg -s tcl-dev|grep Version)

cat << EOF > ~/downloads/pkgconfig/tcl.pc
# tcl pkg-config source file

prefix=/usr
exec_prefix=/usr
libdir=/usr/lib/x86_64-linux-gnu
includedir=/usr/include/tcl8.6

Name: Tool Command Language
Description: Tcl is a powerful, easy-to-learn dynamic programming language, suitable for a wide range of uses.
URL: https://www.tcl-tk.org/
${tcl_version}
Requires.private: zlib >= 1.2.x
Libs: -L\${libdir} -ltcl8.6 -ltclstub8.6
Libs.private: -ldl -lz  -lpthread -lm
Cflags: -I\${includedir}
EOF

export PKG_CONFIG_PATH=$PKG_CONFIG_PATH:~/downloads/pkgconfig
编译Python
cd  ~/downloads
wget https://www.python.org/ftp/python/3.12.2/Python-3.12.2.tar.xz -O Python-3.12.2.tar.xz
tar xf Python-3.12.2.tar.xz
cd Python-3.12.2

./configure --prefix=/usr/local/python-3.12.2 \
    --enable-optimizations \
    --with-openssl=/usr \
    --with-ssl-default-suites=openssl \
    --with-ensurepip \
    --enable-loadable-sqlite-extensions
make
make install

24.5.3. 压缩包安装

# 可选一
wget https://dl.cdgeekcamp.com/ubuntu/16.04/openssl-1.1.1n-1~ubuntu16.04_amd64.tar.gz -O openssl-1.1.1n-1~ubuntu16.04_amd64.tar.gz
tar xf openssl-1.1.1n-1~ubuntu16.04_amd64.tar.gz
rm -rf /usr/local/openssl-1.1.1n
mv openssl-1.1.1n/ /usr/local/openssl-1.1.1n

echo '/usr/local/openssl-1.1.1n/lib' > /etc/ld.so.conf.d/openssl-1.1.1n.conf
ldconfig
ldconfig -p|grep openssl-1.1.1n

wget https://dl.cdgeekcamp.com/ubuntu/16.04/python-3.12.2-1~ubuntu16.04_amd64.tar.gz -O python-3.12.2-1~ubuntu16.04_amd64.tar.gz


# 可选二
wget https://dl.cdgeekcamp.com/ubuntu/18.04/python-3.12.2-1~ubuntu18.04_amd64.tar.gz -O python-3.12.2-1~ubuntu18.04_amd64.tar.gz
# 可选三
wget https://dl.cdgeekcamp.com/ubuntu/22.04/python-3.12.2-1~ubuntu22.04_amd64.tar.gz -O python-3.12.2-1~ubuntu22.04_amd64.tar.gz

tar xf python-3.12.2-1~ubuntu*04_amd64.tar.gz

rm -rf /usr/local/python-3.12.2
mv python-3.12.2 /usr/local/python-3.12.2

24.5.4. 设置Python

test -L /usr/local/python3 || ln -s python-3.12.2 /usr/local/python3
test -L /usr/local/python3/bin/pip312 || ln -s pip3 /usr/local/python3/bin/pip312
test -L /usr/local/python3/bin/python312 || ln -s python3 /usr/local/python3/bin/python312

echo "export PATH=/usr/local/python3/bin:\${PATH}" > /etc/profile.d/python3.sh
source /etc/profile

pip312 install --upgrade -i https://pypi.tuna.tsinghua.edu.cn/simple pip
pip312 config set global.index-url https://pypi.tuna.tsinghua.edu.cn/simple

24.6. PIP常用设置

24.6.1. 设置PIP国内镜像

临时使用
pip install -i https://pypi.tuna.tsinghua.edu.cn/simple 包名称

simple 不能少, 是 https 而不是 http

设为默认
先升级PIP
pip install -i https://pypi.tuna.tsinghua.edu.cn/simple pip -U

手动升级 CentOS 系统 yum 安装的 pip3 时,需要指定安装路径到 /usr ,否则 pip3 会被安装到 /usr/local/bin/pip3

pip install --prefix /usr -i https://pypi.tuna.tsinghua.edu.cn/simple pip -U
设置默认镜像
pip config set global.index-url https://pypi.tuna.tsinghua.edu.cn/simple

24.6.2. 常用设置

# 允许在venv以外对安装包、卸载包
pip config set global.break-system-packages true
# 安装包时,不使用隔离模式。正常情况下,建议使用 `pip install --no-build-isolation xxx` 方式
pip config set global.no-build-isolation true

24.7. 上传 PyPI

官方文档:

24.7.1. 配置文件

~/.pypirc
[distutils]
index-servers=
    pypi
    testpypi

[pypi]
username: <用户名>
password: <密码>

[testpypi]
repository: https://test.pypi.org/legacy/
username: <用户名>
password: <密码>

24.7.2. 打包

安装打包工具
python3 -m pip install -U setuptools wheel
打包
python3 setup.py bdist_wheel
打包后的文件
dist/
  example_pkg_YOUR_USERNAME_HERE-0.0.1-py3-none-any.whl
  example_pkg_YOUR_USERNAME_HERE-0.0.1.tar.gz

24.7.3. 测试上传

安装上传工具
python3 -m pip install -U twine
测试上传
twine upload --repository-url https://test.pypi.org/legacy/ dist/*

或者

twine upload --config-file ~/.pypirc -r testpypi dist/*
测试安装
python3 -m pip install -i https://test.pypi.org/simple/ --no-deps example-pkg-YOUR-USERNAME-HERE
Python 解释器中测试导入包
import example_pkg

24.7.4. 上传

twine upload dist/*

如果使用国内镜像源,需要等一两天国内服务器才会同步官方源。

可以临时指定官方源安装:

python3 -m pip install -i https://pypi.org/simple/ example-pkg

24.8. PIP FAQ

24.8.1. 隔离模式引起的包依赖不存在问题

提前安装指定版本的 jmespath

pip install 'jmespath>=0.9.3,<1.0.0'

pip install alibaba-cloud-python-sdk-v2 时,还是提示找不到 jmespath 模块:

Looking in indexes: https://pypi.tuna.tsinghua.edu.cn/simple
Collecting alibaba-cloud-python-sdk-v2 (from -r requirements.txt (line 2))
  Downloading https://pypi.tuna.tsinghua.edu.cn/packages/6a/67/0244b84f93699e67af65bd18d79c090f09cb1af7c2f363586440190dd6c6/alibaba-cloud-python-sdk-v2-1.0.6.tar.gz (793 kB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 793.4/793.4 kB 2.7 MB/s eta 0:00:00
  Preparing metadata (setup.py) ... erroress-exited-with-er
  error: subprocess-exited-with-error

  × python setup.py egg_info did not run successfully.
  │ exit code: 1
  ╰─> [15 lines of output]
      Traceback (most recent call last):
        File "<string>", line 2, in <module>
        File "<pip-setuptools-caller>", line 34, in <module>
        File "/tmp/pip-install-800r4u6_/alibaba-cloud-python-sdk-v2_e4a92643a9c349fb8482118c5b1dae7e/setup.py", line 19, in <module>
          VERSION = __import__("alibabacloud").__version__
                    ^^^^^^^^^^^^^^^^^^^^^^^^^^
        File "/tmp/pip-install-800r4u6_/alibaba-cloud-python-sdk-v2_e4a92643a9c349fb8482118c5b1dae7e/alibabacloud/__init__.py", line 20, in <module>
          from alibabacloud.client import ClientConfig
        File "/tmp/pip-install-800r4u6_/alibaba-cloud-python-sdk-v2_e4a92643a9c349fb8482118c5b1dae7e/alibabacloud/client.py", line 21, in <module>
          import alibabacloud.retry.retry_policy as retry_policy
        File "/tmp/pip-install-800r4u6_/alibaba-cloud-python-sdk-v2_e4a92643a9c349fb8482118c5b1dae7e/alibabacloud/retry/retry_policy.py", line 15, in <module>
          from alibabacloud.retry.retry_condition import *
        File "/tmp/pip-install-800r4u6_/alibaba-cloud-python-sdk-v2_e4a92643a9c349fb8482118c5b1dae7e/alibabacloud/retry/retry_condition.py", line 15, in <module>
          import jmespath
      ModuleNotFoundError: No module named 'jmespath'
      [end of output]

  note: This error originates from a subprocess, and is likely not a problem with pip.
error: metadata-generation-failed

× Encountered error while generating package metadata.
╰─> See above for output.

note: This is an issue with the package mentioned above, not pip.
hint: See above for details.

确认依赖包的安装和版本无问题后,怀疑 pip 运行在虚拟环境或者隔离环境下,不是本机环境:

$ pip show jmespath
Name: jmespath
Version: 0.10.0
Summary: JSON Matching Expressions
Home-page: https://github.com/jmespath/jmespath.py
Author: James Saryerwinnie
Author-email: js@jamesls.com
License: MIT
Location: /home/lyuqiang/.local/lib/python3.11/site-packages
Requires:
Required-by: alibaba-cloud-python-sdk-v2
$ pip install --help | grep iso
  --no-build-isolation        Disable isolation when building a modern source
  --isolated                  Run pip in an isolated mode, ignoring

重新带 --no-build-isolation 参数安装解决问题:

pip install --no-build-isolation alibaba-cloud-python-sdk-v2

24.9. Pycharm

24.9.1. Disable and suppress inspections

官方文档:

Suppressing comments
# noinspection PyAbstractClass
# noinspection PyArgumentEqualDefault
# noinspection PyArgumentList
# noinspection PyAssignmentToLoopOrWithParameter
# noinspection PyAsyncCall
# noinspection PyAttributeOutsideInit
# noinspection PyAugmentAssignment
# noinspection PyBroadException
# noinspection PyByteLiteral
# noinspection PyCallByClass
# noinspection PyChainedComparsons
# noinspection PyClassHasNoInit
# noinspection PyClassicStyleClass
# noinspection PyComparisonWithNone
# noinspection PyCompatibility
# noinspection PyDecorator
# noinspection PyDefaultArgument
# noinspection PyDictCreation
# noinspection PyDictDuplicateKeys
# noinspection PyDocstringTypes
# noinspection PyExceptClausesOrder
# noinspection PyExceptionInheritance
# noinspection PyFromFutureImport
# noinspection PyGlobalUndefined
# noinspection PyIncorrectDocstring
# noinspection PyInitNewSignature
# noinspection PyInterpreter
# noinspection PyListCreation
# noinspection PyMandatoryEncoding
# noinspection PyMethodFirstArgAssignment
# noinspection PyMethodMayBeStatic
# noinspection PyMethodOverriding
# noinspection PyMethodParameters
# noinspection PyMissingConstructor
# noinspection PyMissingOrEmptyDocstring
# noinspection PyNestedDecorators
# noinspection PynonAsciiChar
# noinspection PyNoneFunctionAssignment
# noinspection PyOldStyleClasses
# noinspection PyPackageRequirements
# noinspection PyPep8
# noinspection PyPep8Naming
# noinspection PyPropertyAccess
# noinspection PyPropertyDefinition
# noinspection PyProtectedMember
# noinspection PyRaisingNewStyleClass
# noinspection PyRedeclaration
# noinspection PyRedundantParentheses
# noinspection PySetFunctionToLiteral
# noinspection PySimplifyBooleanCheck
# noinspection PySingleQuotedDocstring
# noinspection PyStatementEffect
# noinspection PyStringException
# noinspection PyStringFormat
# noinspection PySuperArguments
# noinspection PyTestParametrized
# noinspection PyTrailingSemicolon
# noinspection PyTupleAssignmentBalance
# noinspection PyTupleItemAssignment
# noinspection PyTypeChecker
# noinspection PyUnboundLocalVariable
# noinspection PyUnnecessaryBackslash
# noinspection PyUnreachableCode
# noinspection PyUnresolvedReferences
# noinspection PyUnusedLocal
# noinspection ReturnValueFromInit

25. Axel

有时候,yum/dnf/wget下载国外软件。一开始速度很快,之后速度降了下去。这个时候用多线程下载可以加速下载。

25.1. 安装

25.1.1. Arch Linux

sudo pacman -S axel

25.1.2. CentOS8

wget https://github.com/axel-download-accelerator/axel/releases/download/v2.17.8/axel-2.17.8.tar.gz -O axel-2.17.8.tar.gz
tar xf axel-2.17.8.tar.gz
cd axel-2.17.8

dnf install -y make gcc openssl-devel autoconf

./configure --prefix=/usr/local/axel-2.17.8
make && make install

ln -s /usr/local/axel-2.17.8/bin/axel /usr/local/bin/axel

25.2. 使用

下载GtiLab
axel -n 4 https://packages.gitlab.com/gitlab/gitlab-ce/el/8/x86_64/gitlab-ce-12.10.2-ce.0.el8.x86_64.rpm

26. Let’s Encrypt

Let’s Encrypt 由Google、Apple等大公司支持的免费 HTTPS 证书。所有主流浏览器都支持LE证书。

LE证书的可以通过网站验证或DNS验证方式申请,有效期为三个月。三个月后,可以通过系统计划任务自动续签。

26.1. 安装certbot

pip3 install certbot

python_bin_dir=$(pip3 show certbot | grep 'Location:' | cut -d ' ' -f 2)/../../../bin
python_bin_dir=$(realpath ${python_bin_dir})
ln -s ${python_bin_dir}/certbot /usr/local/bin/certbot

26.2. 签发HTTPS证书之DNS签发通配符域名证书

26.2.1. 签发通配符域名HTTPS证书(一)

certbot certonly  -d *.xxx.com --manual --preferred-challenges dns --server https://acme-v02.api.letsencrypt.org/directory
操作过程
    Saving debug log to /var/log/letsencrypt/letsencrypt.log
    Plugins selected: Authenticator manual, Installer None
    Enter email address (used for urgent renewal and security notices) (Enter 'c' to
    cancel): webmaster@xxx.com

    - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
    Please read the Terms of Service at
    https://letsencrypt.org/documents/LE-SA-v1.2-November-15-2017.pdf. You must
    agree in order to register with the ACME server at
    https://acme-v02.api.letsencrypt.org/directory
    - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
    (A)gree/(C)ancel: A

    - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
    Would you be willing to share your email address with the Electronic Frontier
    Foundation, a founding partner of the Let's Encrypt project and the non-profit
    organization that develops Certbot? We'd like to send you email about our work
    encrypting the web, EFF news, campaigns, and ways to support digital freedom.
    - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
    (Y)es/(N)o: Y
    Obtaining a new certificate
    Performing the following challenges:
    dns-01 challenge for xxx.com

    - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
    NOTE: The IP of this machine will be publicly logged as having requested this
    certificate. If you're running certbot in manual mode on a machine that is not
    your server, please ensure you're okay with that.

    Are you OK with your IP being logged?
    - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
    (Y)es/(N)o: Y

    - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
    Please deploy a DNS TXT record under the name
    _acme-challenge.xxx.com with the following value:

    1EfqEsqE3pbZqiJmxpHvEkM0XdUhvqW5w4lWb46zMjM

    Before continuing, verify the record is deployed.
    - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
    Press Enter to Continue

26.2.2. 设置域名解析

登录域名管理面板,新增 TXT 类型的域名解析记录,主机记录名称: _acme-challenge

如下图所示:

add dns setting

效果如下:

dns settings

26.2.3. 检查域名解析

dig _acme-challenge.xxx.com TXT
终端输出
; <<>> DiG 9.16.2 <<>> _acme-challenge.xxx.com TXT
;; global options: +cmd
;; Got answer:
;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 64914
;; flags: qr rd ra; QUERY: 1, ANSWER: 1, AUTHORITY: 0, ADDITIONAL: 0

;; QUESTION SECTION:
;_acme-challenge.xxx.com.	IN	TXT

;; ANSWER SECTION:
_acme-challenge.xxx.com. 596 IN	TXT	"1EfqEsqE3pbZqiJmxpHvEkM0XdUhvqW5w4lWb46zMjM"

;; Query time: 10 msec
;; SERVER: 223.5.5.5#53(223.5.5.5)
;; WHEN: 六 5月 02 14:59:39 CST 2020
;; MSG SIZE  rcvd: 103

26.2.4. 签发通配符域名HTTPS证书(二)

域名解析完成后,在 Press Enter to Continue 处按 Enter 继续操作…​…​

继续操作过程
    Before continuing, verify the record is deployed.
    - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
    Press Enter to Continue
    Waiting for verification...
    Cleaning up challenges

    IMPORTANT NOTES:
    - Congratulations! Your certificate and chain have been saved at:
    /etc/letsencrypt/live/xxx.com/fullchain.pem
    Your key file has been saved at:
    /etc/letsencrypt/live/xxx.com/privkey.pem
    Your cert will expire on 2020-07-31. To obtain a new or tweaked
    version of this certificate in the future, simply run certbot
    again. To non-interactively renew *all* of your certificates, run
    "certbot renew"
    - If you like Certbot, please consider supporting our work by:

    Donating to ISRG / Let's Encrypt:   https://letsencrypt.org/donate
    Donating to EFF:                    https://eff.org/donate-le
生成交换密钥
openssl dhparam -dsaparam -out /etc/letsencrypt/live/xxx.com/dhparam.pem 2048

26.3. 签发HTTPS证书之标准方式签发单域名证书

标准方式签发HTTPS证书,必须保证 http://www.xxx.com 能够访问

26.3.1. 添加Nginx站点

cat << EOF > /etc/nginx/conf.d/www.xxx.com.conf
server {
    listen 80;
    server_name www.xxx.com;
    root /data/web/www.xxx.com;

    location / {
        return 301 https://www.xxx.com;
    }

    location ^~ /.well-known/acme-challenge/ {
       default_type "text/plain";
    }

    location = /.well-known/acme-challenge/ {
       return 404;
    }
}
EOF

nginx -s reload

26.3.2. 签发单域名HTTPS证书

certbot certonly --email webmaster@xxx.com -w /data/web/www.xxx.com -d www.xxx.com
参数解释
/data/web/www.xxx.com

www.xxx.com.conf 中的 root

www.xxx.com

www.xxx.com.confserver_name

操作过程
Saving debug log to /var/log/letsencrypt/letsencrypt.log

How would you like to authenticate with the ACME CA?
- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
1: Runs an HTTP server locally which serves the necessary validation files under
the /.well-known/acme-challenge/ request path. Suitable if there is no HTTP
server already running. HTTP challenge only (wildcards not supported).
(standalone)
2: Saves the necessary validation files to a .well-known/acme-challenge/
directory within the nominated webroot path. A seperate HTTP server must be
running and serving files from the webroot path. HTTP challenge only (wildcards
not supported). (webroot)
- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
Select the appropriate number [1-2] then [enter] (press 'c' to cancel): 2

- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
Please read the Terms of Service at
https://letsencrypt.org/documents/LE-SA-v1.3-September-21-2022.pdf. You must
agree in order to register with the ACME server. Do you agree?
- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
(Y)es/(N)o: Y

- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
Would you be willing, once your first certificate is successfully issued, to
share your email address with the Electronic Frontier Foundation, a founding
partner of the Let's Encrypt project and the non-profit organization that
develops Certbot? We'd like to send you email about our work encrypting the web,
EFF news, campaigns, and ways to support digital freedom.
- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
(Y)es/(N)o: Y
Account registered.
Requesting a certificate for www.xxx.com

Successfully received certificate.
Certificate is saved at: /etc/letsencrypt/live/www.xxx.com/fullchain.pem
Key is saved at:         /etc/letsencrypt/live/www.xxx.com/privkey.pem
This certificate expires on 2023-05-17.
These files will be updated when the certificate renews.

NEXT STEPS:
- The certificate will need to be renewed before it expires. Certbot can automatically renew the certificate in the background, but you may need to take steps to enable that functionality. See https://certbot.org/renewal-setup for instructions.

- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
If you like Certbot, please consider supporting our work by:
 * Donating to ISRG / Let's Encrypt:   https://letsencrypt.org/donate
 * Donating to EFF:                    https://eff.org/donate-le
- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -

26.3.3. 生成交换密钥

openssl dhparam -dsaparam -out /etc/letsencrypt/live/www.xxx.com/dhparam.pem 2048

26.4. 添加HTTPS站点

26.4.1. 创建配置文件

Nginx会联网检查证书状态,出现警告:

2023/10/15 04:44:09 [warn] 11838#11838: no resolver defined to resolve r3.o.lencr.org while requesting certificate status, responder: r3.o.lencr.org, certificate: "/etc/letsencrypt/live/noby1.cyou/fullchain.pem"

新增Nginx主配置:

http {
    resolver 223.5.5.5:53 ipv6=off; (1)
}
1 在IPv4的系统上,IPv6选项必须关闭
cat << EOF > /etc/nginx/conf.d/xxx.com_ssl.conf
server {
    listen 443 ssl http2;
    listen [::]:443 ssl http2;
    server_name *.xxx.com;
    root /data/web/xxx.com;

    # certs sent to the client in SERVER HELLO are concatenated in ssl_certificate
    ssl_certificate /etc/letsencrypt/live/xxx.com/fullchain.pem;
    ssl_certificate_key /etc/letsencrypt/live/xxx.com/privkey.pem;
    ssl_session_timeout 1d;
    ssl_session_cache shared:SSL:50m;
    ssl_session_tickets off;

    # Diffie-Hellman parameter for DHE ciphersuites, recommended 2048 bits
    ssl_dhparam /etc/letsencrypt/live/xxx.com/dhparam.pem;

    # intermediate configuration. tweak to your needs.
    ssl_protocols TLSv1 TLSv1.1 TLSv1.2 TLSv1.3;
    ssl_ciphers 'ECDHE-ECDSA-CHACHA20-POLY1305:ECDHE-RSA-CHACHA20-POLY1305:ECDHE-ECDSA-AES128-GCM-SHA256:ECDHE-RSA-AES128-GCM-SHA256:ECDHE-ECDSA-AES256-GCM-SHA384:ECDHE-RSA-AES256-GCM-SHA384:DHE-RSA-AES128-GCM-SHA256:DHE-RSA-AES256-GCM-SHA384:ECDHE-ECDSA-AES128-SHA256:ECDHE-RSA-AES128-SHA256:ECDHE-ECDSA-AES128-SHA:ECDHE-RSA-AES256-SHA384:ECDHE-RSA-AES128-SHA:ECDHE-ECDSA-AES256-SHA384:ECDHE-ECDSA-AES256-SHA:ECDHE-RSA-AES256-SHA:DHE-RSA-AES128-SHA256:DHE-RSA-AES128-SHA:DHE-RSA-AES256-SHA256:DHE-RSA-AES256-SHA:ECDHE-ECDSA-DES-CBC3-SHA:ECDHE-RSA-DES-CBC3-SHA:EDH-RSA-DES-CBC3-SHA:AES128-GCM-SHA256:AES256-GCM-SHA384:AES128-SHA256:AES256-SHA256:AES128-SHA:AES256-SHA:DES-CBC3-SHA:!DSS';
    ssl_prefer_server_ciphers on;

    # HSTS (ngx_http_headers_module is required) (15768000 seconds = 6 months)
    add_header Strict-Transport-Security max-age=15768000;

    # OCSP Stapling ---
    # fetch OCSP records from URL in ssl_certificate and cache them
    ssl_stapling on;
    ssl_stapling_verify on;
    # nginx: Specifies a file with trusted CA certificates in the PEM format used to verify client certificates and OCSP responses if ssl_stapling is enabled.
    # certbot: If you’re using OCSP stapling with Nginx >= 1.3.7, chain.pem should be provided as the ssl_trusted_certificate to validate OCSP responses.
    ssl_trusted_certificate /etc/letsencrypt/live/xxx.com/chain.pem;


    location ~ ^/\.git/? {
        return 404;
    }
}
EOF
单域名的站点配置
  • 只需要将上文的 *.xxx.com 替换为 www.xxx.com

  • /etc/letsencrypt/live/xxx.com/ 替换为 /etc/letsencrypt/live/www.xxx.com/

26.4.2. 重载配置

nginx -s reload

26.4.3. SSL安全级别测试

26.5. 自动续签

26.5.1. 创建续签系统服务

通配符域名
cat << EOF > /etc/systemd/system/letsencrypt.service
[Unit]
Description=Let's Encrypt renewal

[Service]
Type=oneshot
# ExecStart=/usr/local/bin/certbot renew

# 自动修改阿里云域名解析,完成证书续签
ExecStart=/usr/local/bin/certbot renew --manual --preferred-challenges=dns --manual-auth-hook '/data/LetsEncryptAliDnsTool/app.py --auth' --manual-cleanup-hook '/data/LetsEncryptAliDnsTool/app.py --cleanup'

ExecStartPost=/usr/sbin/nginx -s reload

[Install]
WantedBy=multi-user.target
EOF
单域名
cat << EOF > /etc/systemd/system/letsencrypt.service
[Unit]
Description=Let's Encrypt renewal

[Service]
Type=oneshot
ExecStart=/usr/local/bin/certbot renew
ExecStartPost=/usr/sbin/nginx -s reload

[Install]
WantedBy=multi-user.target
EOF

26.5.2. 创建续签定时器

cat << EOF > /etc/systemd/system/letsencrypt.timer
[Unit]
Description=Monthly renewal of Let's Encrypt's certificates

[Timer]
OnCalendar=daily
Persistent=true

[Install]
WantedBy=timers.target
EOF

26.5.3. 重载系统服务

systemctl daemon-reload

26.5.4. 设置开机启动

systemctl enable letsencrypt.service
systemctl enable letsencrypt.timer

26.5.5. 启动定时器

systemctl start letsencrypt.timer

26.5.6. 查看定时器

systemctl list-timers letsencrypt.timer

26.6. 日常使用

26.6.1. 查看证书域名列表

方法一:OpenSSL + x509 打印证书信息
openssl x509 -in /etc/letsencrypt/live/foo.com/fullchain.pem -text -noout | grep DNS
屏幕输出
DNS:*.foo.com, DNS:foo.com
方法二:打印指定证书(foo.com)的信息
certbot --help certificates
certbot certificates --cert-name foo.com
屏幕输出
Saving debug log to /var/log/letsencrypt/letsencrypt.log

- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
Found the following matching certs:
  Certificate Name: foo.com
    Serial Number: 3709aba3222e0229f3f9b2e49b1f5664131
    Key Type: ECDSA
    Domains: foo.com *.foo.com
    Expiry Date: 2024-01-03 06:19:50+00:00 (VALID: 89 days)
    Certificate Path: /etc/letsencrypt/live/foo.com/fullchain.pem
    Private Key Path: /etc/letsencrypt/live/foo.com/privkey.pem
- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
方法三:查找同时包括foo.com和*.foo.com域名的证书
certbot certificates -d foo.com -d *.foo.com
屏幕输出
Saving debug log to /var/log/letsencrypt/letsencrypt.log

- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
Found the following matching certs:
  Certificate Name: foo.com
    Serial Number: 3709aba3222e0229f3f9b2e49b1f5664131
    Key Type: ECDSA
    Domains: foo.com *.foo.com
    Expiry Date: 2024-01-03 06:19:50+00:00 (VALID: 89 days)
    Certificate Path: /etc/letsencrypt/live/foo.com/fullchain.pem
    Private Key Path: /etc/letsencrypt/live/foo.com/privkey.pem
- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -

26.6.2. 同时签发根域名(foo.com)和泛域名(*.foo.com)至一个证书

certbot --help certonly
如果域名证书用于CDN,注意使用 certbot certonly --key-type rsa 签发RSA格式私钥才能用
  1. 签发多域名(foo.com、*.foo.com)证书:

    位置:签发界面
    certbot certonly -d foo.com -d *.foo.com --manual --preferred-challenges dns --server https://acme-v02.api.letsencrypt.org/directory
    -d foo.com -d *.foo.com 参数
    顺序不能错乱

    默认用第一个域名作为证书名称,如 /etc/letsencrypt/live/foo.com

    签发界面的屏幕输出(1)
    Saving debug log to /var/log/letsencrypt/letsencrypt.log
    Requesting a certificate for foo.com and *.foo.com
    
    - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
    Please deploy a DNS TXT record under the name:
    
    _acme-challenge.foo.com.
    
    with the following value:
    
    KhfLnoh0x77HK9mdxCZfpWN9RA-J-eXmPvZTpvX9ReA
    
    - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
    Press Enter to Continue (1)
    1 直接按 Enter
    签发界面的屏幕输出(2)
    【接 签发界面的屏幕输出(1) 的内容】
    ......
    ......
    ......
    
    - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
    Please deploy a DNS TXT record under the name:
    
    _acme-challenge.foo.com.
    
    with the following value:
    
    0XfJQBCwpgzREwrxSiBOQRMmn7iNNE3X71QSjhUZbMs
    
    (This must be set up in addition to the previous challenges; do not remove,
    replace, or undo the previous challenge tasks yet. Note that you might be
    asked to create multiple distinct TXT records with the same name. This is
    permitted by DNS standards.)
    
    Before continuing, verify the TXT record has been deployed. Depending on the DNS
    provider, this may take some time, from a few seconds to multiple minutes. You can
    check if it has finished deploying with aid of online tools, such as the Google
    Admin Toolbox: https://toolbox.googleapps.com/apps/dig/#TXT/_acme-challenge.foo.com.
    Look for one or more bolded line(s) below the line ';ANSWER'. It should show the
    value(s) you've just added.
    
    - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
    Press Enter to Continue (1)
    1 解析没生效前,不要按 Enter
    现在,已经拿到两条DNS TXT解析记录字符串,接着去做解析
  1. 登录域名管理界面,新增 两条相同名称的 TXT 解析记录,主机记录名称: _acme-challenge

    如下图所示:

    dns settings two txt
  1. 确认域名解析已经生效

    位置:域名验证界面
    dig _acme-challenge.foo.com TXT
    屏幕输出
    ; <<>> DiG 9.11.4-P2-RedHat-9.11.4-26.P2.el7_9.14 <<>> _acme-challenge.foo.com TXT
    ;; global options: +cmd
    ;; Got answer:
    ;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 57330
    ;; flags: qr rd ra; QUERY: 1, ANSWER: 2, AUTHORITY: 0, ADDITIONAL: 1
    
    ;; OPT PSEUDOSECTION:
    ; EDNS: version: 0, flags:; udp: 512
    ;; QUESTION SECTION:
    ;_acme-challenge.foo.com.	IN	TXT
    
    ;; ANSWER SECTION:
    _acme-challenge.foo.com. 600	IN	TXT	"0XfJQBCwpgzREwrxSiBOQRMmn7iNNE3X71QSjhUZbMs" (1)
    _acme-challenge.foo.com. 600	IN	TXT	"KhfLnoh0x77HK9mdxCZfpWN9RA-J-eXmPvZTpvX9ReA" (2)
    
    ;; Query time: 172 msec
    ;; SERVER: 8.8.8.8#53(8.8.8.8)
    ;; WHEN: Thu Oct 05 16:02:59 CST 2023
    ;; MSG SIZE  rcvd: 165
    1 第一条 _acme-challenge.foo.com DNS TXT解析记录
    2 第二条 _acme-challenge.foo.com DNS TXT解析记录
  1. 继续完成签发多域名(foo.com、*.foo.com)证书:

    签发界面的屏幕输出(3)
    【接 签发界面的屏幕输出(2) 的内容】
    ......
    ......
    ......
    
    Successfully received certificate.
    Certificate is saved at: /etc/letsencrypt/live/foo.com/fullchain.pem
    Key is saved at:         /etc/letsencrypt/live/foo.com/privkey.pem
    This certificate expires on 2024-01-03.
    These files will be updated when the certificate renews.
    
    NEXT STEPS:
    - This certificate will not be renewed automatically. Autorenewal of --manual certificates requires the use of an authentication hook script (--manual-auth-hook) but one was not provided. To renew this certificate, repeat this same certbot command before the certificate's expiry date.
    
    - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
    If you like Certbot, please consider supporting our work by:
     * Donating to ISRG / Let's Encrypt:   https://letsencrypt.org/donate
     * Donating to EFF:                    https://eff.org/donate-le
    - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
关于CDN+域名证书的说明

如果域名证书用于CDN,除了需要RSA格式的私钥外,还需要证书信任链。

本来 fullchain.pem 已经包含了证书信任链,分别是域名证书(cert.pem)、中间证书(chain.pem中第一个证书)和根证书(chain.pem中第二个证书)。

结果,CDN厂商验证信任链是从互联网渠道获得的中间证书和根证书,和 chain.pem 文件中的证书内容不匹配。

手动生成CDN厂商支持的 fullchain.pem 证书文件:

test -f /etc/letsencrypt/isrgrootx1.pem || wget https://letsencrypt.org/certs/isrgrootx1.pem -O /etc/letsencrypt/isrgrootx1.pem
test -f /etc/letsencrypt/lets-encrypt-r3.pem || wget https://letsencrypt.org/certs/isrgrootx1.pem -O /etc/letsencrypt/lets-encrypt-r3.pem

cat /etc/letsencrypt/live/foo.com/cert.pem /etc/letsencrypt/lets-encrypt-r3.pem /etc/letsencrypt/isrgrootx1.pem > /etc/letsencrypt/live/foo.com/fullchain_for_cdn.pem

/etc/letsencrypt/live/foo.com/fullchain_for_cdn.pem 上传到CDN厂商即可。

  1. 生成交换密钥

    位置:签发界面
    openssl dhparam -dsaparam -out /etc/letsencrypt/live/foo.com/dhparam.pem 2048

26.6.3. 前后签发根域名(foo.com)和泛域名(*.foo.com)至一个证书

certbot --help certonly
  1. 已有包含根域名的证书:/etc/letsencrypt/live/foo.com

  2. 在根域名证书中追加新域名(*.foo.com):

    certbot certonly -d foo.com -d *.foo.com --manual --preferred-challenges dns --server https://acme-v02.api.letsencrypt.org/directory
    -d foo.com -d *.foo.com 参数
    顺序不能错乱

    默认用第一个域名作为证书名称,如 /etc/letsencrypt/live/foo.com

    列表不能缺失

    新证书仅签发 -d 指定的域名列表,所以不能有缺失

    签发证书的屏幕输出(1)
    Saving debug log to /var/log/letsencrypt/letsencrypt.log
    
    - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
    You have an existing certificate that contains a portion of the domains you
    requested (ref: /etc/letsencrypt/renewal/foo.com.conf)
    
    It contains these names: foo.com (1)
    
    You requested these names for the new certificate: foo.com, *.foo.com. (2)
    
    Do you want to expand and replace this existing certificate with the new
    certificate?
    - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
    (E)xpand/(C)ancel: E  (3)
    1 根证书包含的域名列表
    2 本次签发指定的域名列表
    3 询问用新生成的域名证书替换老证书,确认请输入 E
  1. 接下来的步骤就非常熟悉了,不再多说:

    签发证书的屏幕输出(2)
    【接 签发证书的屏幕输出(1) 的内容】
    ......
    ......
    ......
    
    (E)xpand/(C)ancel: E
    Renewing an existing certificate for foo.com and *.foo.com
    
    - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
    Please deploy a DNS TXT record under the name:
    
    _acme-challenge.foo.com.
    
    with the following value:
    
    _V7Odzo7QopP9Pi2q_GMbvXyVkcUVhHIn4YVS-4M_Og
    
    Before continuing, verify the TXT record has been deployed. Depending on the DNS
    provider, this may take some time, from a few seconds to multiple minutes. You can
    check if it has finished deploying with aid of online tools, such as the Google
    Admin Toolbox: https://toolbox.googleapps.com/apps/dig/#TXT/_acme-challenge.foo.com.
    Look for one or more bolded line(s) below the line ';ANSWER'. It should show the
    value(s) you've just added.
    
    - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
    Press Enter to Continue
    
    Successfully received certificate.
    Certificate is saved at: /etc/letsencrypt/live/foo.com/fullchain.pem
    Key is saved at:         /etc/letsencrypt/live/foo.com/privkey.pem
    This certificate expires on 2024-01-03.
    These files will be updated when the certificate renews.
    
    NEXT STEPS:
    - This certificate will not be renewed automatically. Autorenewal of --manual certificates requires the use of an authentication hook script (--manual-auth-hook) but one was not provided. To renew this certificate, repeat this same certbot command before the certificate's expiry date.
    
    - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
    If you like Certbot, please consider supporting our work by:
     * Donating to ISRG / Let's Encrypt:   https://letsencrypt.org/donate
     * Donating to EFF:                    https://eff.org/donate-le
    - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
  1. 生成交换密钥

    位置:签发界面
    openssl dhparam -dsaparam -out /etc/letsencrypt/live/foo.com/dhparam.pem 2048

27. GPG

27.1. 生成文件签名

$ gpg --list-signatures
/home/fifilyu/.gnupg/pubring.kbx
--------------------------------
pub   rsa2048 2017-12-14 [SC]
      CD6B852E003F586F3FD6AA6CF5AF9B93E7C36AED
uid           [ 绝对 ] Zhong Jing Wu Lian (release key) <security@mwteck.com>
sig 3        F5AF9B93E7C36AED 2017-12-14  Zhong Jing Wu Lian (release key) <security@mwteck.com>
sub   rsa2048 2017-12-14 [E]
sig          F5AF9B93E7C36AED 2017-12-14  Zhong Jing Wu Lian (release key) <security@mwteck.com>
sha1sum config.properties >config.properties.sha1sum
gpg --armor --digest-algo sha1 --detach-sign --local-user F5AF9B93E7C36AED config.properties.sha1sum
gpg --verify config.properties.sha1sum.asc

28. OpenSSL

28.1. 常用命令

Hmac摘要,输出十六进制字符串
echo -n "abc" | openssl dgst -sha224 -hmac "password"
Hmac摘要,输出Base64字符串
echo -n "abc" | openssl dgst -sha224 -hmac "123456" -binary | base64 -w 0
base64 -w 0 表示不换行
可选算法 sha1/sha224/sha256/sha384/sha512

29. Polipo

29.1. 安装配置

29.1.1. CentOS8

安装依赖
wget http://ftp.gnu.org/gnu/texinfo/texinfo-6.7.tar.xz -O texinfo-6.7.tar.xz
tar xf texinfo-6.7.tar.xz
cd texinfo-6.7
./configure --prefix=/usr/local/texinfo-6.7
make && make install
ln -s /usr/local/texinfo-6.7/bin/makeinfo /usr/local/bin/
安装 Polipo
wget https://github.com/jech/polipo/archive/polipo-1.1.1.tar.gz -O polipo-1.1.1.tar.gz
tar xf polipo-1.1.1.tar.gz
cd polipo-polipo-1.1.1
make all
make install
配置文件 Polipo
mkdir -p /etc/polipo/ /tmp/polipo-cache/

cat << EOF > /etc/polipo/config1
# Sample configuration file for Polipo. -*-sh-*-

# You should not need to use a configuration file; all configuration
# variables have reasonable defaults.  If you want to use one, you
# can copy this to /etc/polipo/config or to ~/.polipo and modify.

# This file only contains some of the configuration variables; see the
# list given by "polipo -v" and the manual for more.


### Basic configuration
### *******************

# Uncomment one of these if you want to allow remote clients to
# connect:

# proxyAddress = "::0"        # both IPv4 and IPv6
proxyAddress = "0.0.0.0"    # IPv4 only

# If you do that, you'll want to restrict the set of hosts allowed to
# connect:

# allowedClients = 127.0.0.1, 134.157.168.57
# allowedClients = 127.0.0.1, 134.157.168.0/24

# Uncomment this if you want your Polipo to identify itself by
# something else than the host name:

# proxyName = "polipo.example.org"

# Uncomment this if there's only one user using this instance of Polipo:

# cacheIsShared = false

# Uncomment this if you want to use a parent proxy:

# parentProxy = "squid.example.org:3128"

# Uncomment this if you want to use a parent SOCKS proxy:

socksParentProxy = "127.0.0.1:1080"
socksProxyType = socks5

# Uncomment this if you want to scrub private information from the log:

# scrubLogs = true


### Memory
### ******

# Uncomment this if you want Polipo to use a ridiculously small amount
# of memory (a hundred C-64 worth or so):

# chunkHighMark = 819200
# objectHighMark = 128

# Uncomment this if you've got plenty of memory:

# chunkHighMark = 50331648
# objectHighMark = 16384


### On-disk data
### ************

# Uncomment this if you want to disable the on-disk cache:

diskCacheRoot = "/tmp/polipo-cache"

# Uncomment this if you want to put the on-disk cache in a
# non-standard location:

# diskCacheRoot = "~/.polipo-cache/"

# Uncomment this if you want to disable the local web server:

# localDocumentRoot = ""

# Uncomment this if you want to enable the pages under /polipo/index?
# and /polipo/servers?.  This is a serious privacy leak if your proxy
# is shared.

# disableIndexing = false
# disableServersList = false


### Domain Name System
### ******************

# Uncomment this if you want to contact IPv4 hosts only (and make DNS
# queries somewhat faster):

# dnsQueryIPv6 = no

# Uncomment this if you want Polipo to prefer IPv4 to IPv6 for
# double-stack hosts:

# dnsQueryIPv6 = reluctantly

# Uncomment this to disable Polipo's DNS resolver and use the system's
# default resolver instead.  If you do that, Polipo will freeze during
# every DNS query:

# dnsUseGethostbyname = yes


### HTTP
### ****

# Uncomment this if you want to enable detection of proxy loops.
# This will cause your hostname (or whatever you put into proxyName
# above) to be included in every request:

# disableVia=false

# Uncomment this if you want to slightly reduce the amount of
# information that you leak about yourself:

# censoredHeaders = from, accept-language
# censorReferer = maybe

# Uncomment this if you're paranoid.  This will break a lot of sites,
# though:

# censoredHeaders = set-cookie, cookie, cookie2, from, accept-language
# censorReferer = true

# Uncomment this if you want to use Poor Man's Multiplexing; increase
# the sizes if you're on a fast line.  They should each amount to a few
# seconds' worth of transfer; if pmmSize is small, you'll want
# pmmFirstSize to be larger.

# Note that PMM is somewhat unreliable.

# pmmFirstSize = 16384
# pmmSize = 8192

# Uncomment this if your user-agent does something reasonable with
# Warning headers (most don't):

# relaxTransparency = maybe

# Uncomment this if you never want to revalidate instances for which
# data is available (this is not a good idea):

# relaxTransparency = yes

# Uncomment this if you have no network:

# proxyOffline = yes

# Uncomment this if you want to avoid revalidating instances with a
# Vary header (this is not a good idea):

# mindlesslyCacheVary = true

# Uncomment this if you want to add a no-transform directive to all
# outgoing requests.

# alwaysAddNoTransform = true
EOF
创建系统服务
cat << EOF > /usr/lib/systemd/system/polipo.service
[Unit]
Description=polipo - a caching web proxy
After=network.target

[Service]
Type=simple
User=nobody
ExecStart=/usr/local/bin/polipo

[Install]
WantedBy=multi-user.target
EOF
启动服务
systemctl enable polipo
systemctl start polipo
systemctl status polipo

ss -antpl|grep 8123

30. Proxychains

30.1. 安装配置

dnf install -y gcc make

wget https://github.com/haad/proxychains/archive/4.3.0.tar.gz -O 4.3.0.tar.gz
tar xf 4.3.0.tar.gz
cd proxychains-4.3.0


./configure --prefix=/usr/local/proxychains-4.3.0
make && make install
cp src/proxychains.conf /usr/local/proxychains-4.3.0/etc

sed -E -i 's/socks4\s+127.0.0.1 9050/socks5 127.0.0.1 1080/' /usr/local/proxychains-4.3.0/etc/proxychains.conf
sed -i 's/#quiet_mode/quiet_mode/g' /usr/local/proxychains-4.3.0/etc/proxychains.conf

echo 'alias p="/usr/local/proxychains-4.3.0/bin/proxychains4"' >> ~/.bashrc
# 不重启终端生效环境配置
source ~/.bashrc

# 测试访问
p curl -I www.youtube.com

31. ImageMagick

yum install -y ImageMagick

31.1. 查看图片元数据

31.1.1. file 命令

PNG图片
# file foo.png
foo.png: PNG image data, 425 x 239, 8-bit/color RGB, non-interlaced
JPG图片
# file bar.jpg
bar.jpg: JPEG image data, JFIF standard 1.01 (1)
1 看不到JPG图片的分辨率
file *.jpg *.png 支持通配符指定多个文件

31.1.2. identify 命令

PNG图片
# identify foo.png
foo.png PNG 425x239 425x239+0+0 8-bit sRGB 143444B 0.000u 0:00.000
JPG图片
# identify bar.jpg
bar.jpg JPEG 540x1080 540x1080+0+0 8-bit sRGB 50405B 0.000u 0:00.000
identify *.jpg *.png 支持通配符指定多个文件

31.2. 转换单张图片

convert 命令可在转换图片格式时,调整图片参数,如分辨率、质量等等。

31.2.1. 直接转换格式

convert foo.png bar.jpg

31.2.2. 转换格式并调整参数

按比例缩放图片(分辨率)
convert foo.png -resize 50% quz.png
按比例调整图片质量(文件大小)
convert foo.png -quality 50% quz.png

31.3. 批量修改图片

mogrify 命令能直接修改原始文件(适合批量操作),而 convert 需要指定新文件名称。

31.3.1. 按比例缩放图片(分辨率)

mogrify -resize 50% *.jpg

31.3.2. 按比例调整图片质量(文件大小)

mogrify -quality 50% foo.jpg

31.4. 批量图片转PDF

[fifilyu@t430-arch ~]$ tree /home/fifilyu/Pictures/hetong/发票合同/2015/发票
/home/fifilyu/Pictures/hetong/发票合同/2015/发票
├── 20-20000
│   └── IMG_20171115_120634.jpg
├── 38
│   └── IMG_20171115_122128.jpg
├── 39
│   └── IMG_20171115_120247.jpg
├── 41-20000
│   └── IMG_20171115_120450.jpg
├── 47
│   └── IMG_20171115_123224.jpg
├── 48
│   └── IMG_20171115_122517.jpg
├── 49
│   └── IMG_20171115_123443.jpg
├── 50
│   ├── IMG_20171115_122619.jpg
│   ├── IMG_20171115_122629.jpg
│   ├── IMG_20171115_122637.jpg
│   ├── IMG_20171115_122645.jpg
│   └── IMG_20171115_122657.jpg
├── 52
│   └── IMG_20171115_122835.jpg
├── 53
│   └── IMG_20171115_122310.jpg
├── 54
│   └── IMG_20171115_122411.jpg
├── 55
│   └── IMG_20171115_122026.jpg
├── 56
│   └── IMG_20171115_122218.jpg
├── 定州-5500
│   └── IMG_20171115_123055.jpg
├── 韩代-6500
│   └── IMG_20171115_123341.jpg
└── 人保-45900
    ├── IMG_20171115_121009.jpg
    └── IMG_20171115_121337.jpg
需求

每个目录都是一份合同(一页或多页)的扫描图片,生成的PDF必须保证同一份合同的图片是连续的。

生成 PDF 的脚本内容:

echo '#!/bin/sh' > make_pdf.sh
echo 'convert \' >> make_pdf.sh
find /home/fifilyu/Pictures/hetong/发票合同/2015/发票 -name '*.jpg' -o -name "*.png" -o -name "*.jpeg"|awk '{print "\""$1"\" \\"}' >> make_pdf.sh
echo 'out.pdf' >> make_pdf.sh

输出最终 PDF 文件:sh make_pdf.sh

convert 命令在批量处理图片时,耗费大量内存。处理300张图片需要25G~30G左右的内存。

32. Rsync

32.1. 使用实例

33. Dante

Dante 是一个SOCKS服务端和客户端。这里,用来作为服务端。

33.1. 安装配置

33.1.1. 安装

mkdir -p ~/downloads
cd ~/downloads

wget -c https://www.inet.no/dante/sslfiles/dante-1.4.2/tgz-prod.dante-1.4.2-rhel72-amd64-64bit-gcc.tar.gz
tar xf tgz-prod.dante-1.4.2-rhel72-amd64-64bit-gcc.tar.gz

cp -r usr/ /

# 测试
sockd --help

33.1.2. 配置

useradd -r -s /bin/false  sockd

cat << EOF > /etc/sockd.conf
# \$Id: sockd.conf,v 1.52.10.2 2014/09/03 14:49:13 michaels Exp \$
#
# A sample sockd.conf
#
#
# The config file is divided into three parts;
#    1) server settings
#    2) rules
#    3) routes
#
# The recommended order is:
#   Server settings:
#               logoutput
#               internal
#               external
#               socksmethod
#               clientmethod
#               users
#               compatibility
#               extension
#               timeout
#               srchost
#
#  Rules:
#        client block/pass
#                from to
#                libwrap
#                log
#
#     block/pass
#                from to
#                socksmethod
#                command
#                libwrap
#                log
#                protocol
#                proxyprotocol
#
#  Routes:

# the server will log both via syslog, to stdout and to /var/log/sockd.log
#logoutput: syslog stdout /var/log/sockd.log
#logoutput: stderr
logoutput: /var/log/sockd.log

# The server will bind to the address 10.1.1.1, port 1080 and will only
# accept connections going to that address.
#internal: 10.1.1.1 port = 1080
# Alternatively, the interface name can be used instead of the address.
#internal: eth0 port = 1080
internal: 0.0.0.0 port = 1080

# all outgoing connections from the server will use the IP address
# 195.168.1.1
#external: 192.168.1.1
external: eth0

# list over acceptable authentication methods, order of preference.
# An authentication method not set here will never be selected.
#
# If the socksmethod field is not set in a rule, the global
# socksmethod is filled in for that rule.
#

# methods for socks-rules.
#socksmethod: username none #rfc931

# methods for client-rules.
# The default of "none" permits anonymous access.
clientmethod: none

#or if you want to allow rfc931 (ident) too
#socksmethod: username rfc931 none

#or for PAM authentication
#socksmethod: pam

# The default of "none" permits anonymous access.
socksmethod: none

#
# User identities, an important section.
#

# when doing something that can require privilege, it will use the
# userid "sockd".
user.privileged: sockd

# when running as usual, it will use the unprivileged userid of "sockd".
user.unprivileged: sockd

# If you are not using libwrap, no need for the below line, so leave
# it commented.
# If you compiled with libwrap support, what userid should it use
# when executing your libwrap commands?  "libwrap".
#user.libwrap: libwrap


#
# Some options to help clients with compatibility:
#

# when a client connection comes in the socks server will try to use
# the same port as the client is using, when the socks server
# goes out on the clients behalf (external: IP address).
# If this option is set, Dante will try to do it for reserved ports as well.
# This will usually require user.privileged to be set to "root".
#compatibility: sameport

# If you are using the Inferno Nettverk bind extension and have trouble
# running servers via the server, you might try setting this.
#compatibility: reuseaddr

#
# The Dante server supports some extensions to the socks protocol.
# These require that the socks client implements the same extension and
# can be enabled using the "extension" keyword.
#
# enable the bind extension.
#extension: bind


#
# Misc options.
#

# how many seconds can pass from when a client connects til it has
# sent us it's request?  Adjust according to your network performance
# and methods supported.
#timeout.negotiate: 30   # on a lan, this should be enough.

# how many seconds can the client and it's peer idle without sending
# any data before we dump it?  Unless you disable tcp keep-alive for
# some reason, it's probably best to set this to 0, which is
# "forever".
#timeout.io: 0 # or perhaps 86400, for a day.

# do you want to accept connections from addresses without
# dns info?  what about addresses having a mismatch in dns info?
#srchost: nounknown nomismatch

#
# The actual rules.  There are two kinds and they work at different levels.
#
# The rules prefixed with "client" are checked first and say who is allowed
# and who is not allowed to speak/connect to the server.  I.e the
# ip range containing possibly valid clients.
# It is especially important that these only use IP addresses, not hostnames,
# for security reasons.
#
# The rules that do not have a "client" prefix are checked later, when the
# client has sent its request and are used to evaluate the actual
# request.
#
# The "to:" in the "client" context gives the address the connection
# is accepted on, i.e the address the socks server is listening on, or
# just "0.0.0.0/0" for any address the server is listening on.
#
# The "to:" in the non-"client" context gives the destination of the clients
# socks request.
#
# "from:" is the source address in both contexts.
#


#
# The "client" rules.  All our clients come from the net 10.0.0.0/8.
#

# Allow our clients, also provides an example of the port range command.
#client pass {
#        from: 10.0.0.0/8 port 1-65535 to: 0.0.0.0/0
#        clientmethod: rfc931 # match all idented users that also are in passwordfile
#}

# This is identical to above, but allows clients without a rfc931 (ident)
# too.  In practice this means the socks server will try to get a rfc931
# reply first (the above rule), if that fails, it tries this rule.
#client pass {
#        from: 10.0.0.0/8 port 1-65535 to: 0.0.0.0/0
#}

client pass {
    from: 0.0.0.0/0 to: 0.0.0.0/0
}


# drop everyone else as soon as we can and log the connect, they are not
# on our net and have no business connecting to us.  This is the default
# but if you give the rule yourself, you can specify details.
#client block {
#        from: 0.0.0.0/0 to: 0.0.0.0/0
#        log: connect error
#}


# the rules controlling what clients are allowed what requests
#

# you probably don't want people connecting to loopback addresses,
# who knows what could happen then.
#socks block {
#        from: 0.0.0.0/0 to: lo0
#        log: connect error
#}

# the people at the 172.16.0.0/12 are bad, no one should talk to them.
# log the connect request and also provide an example on how to
# interact with libwrap.
#socks block {
#        from: 0.0.0.0/0 to: 172.16.0.0/12
#        libwrap: spawn finger @%a
#        log: connect error
#}

# unless you need it, you could block any bind requests.
#socks block {
#        from: 0.0.0.0/0 to: 0.0.0.0/0
#        command: bind
#        log: connect error
#}

# or you might want to allow it, for instance "active" ftp uses it.
# Note that a "bindreply" command must also be allowed, it
# should usually by from "0.0.0.0/0", i.e if a client of yours
# has permission to bind, it will also have permission to accept
# the reply from anywhere.
#socks pass {
#        from: 10.0.0.0/8 to: 0.0.0.0/0
#        command: bind
#        log: connect error
#}

# some connections expect some sort of "reply", this might be
# the reply to a bind request or it may be the reply to a
# udppacket, since udp is packet based.
# Note that nothing is done to verify that it's a "genuine" reply,
# that is in general not possible anyway.  The below will allow
# all "replies" in to your clients at the 10.0.0.0/8 net.
#socks pass {
#        from: 0.0.0.0/0 to: 10.0.0.0/8
#        command: bindreply udpreply
#        log: connect error
#}


# pass any http connects to the example.com domain if they
# authenticate with username.
# This matches "example.com" itself and everything ending in ".example.com".
#socks pass {
#        from: 10.0.0.0/8 to: .example.com port = http
#        log: connect error
#        clientmethod: username
#}


# block any other http connects to the example.com domain.
#socks block {
#        from: 0.0.0.0/0 to: .example.com port = http
#        log: connect error
#}

# everyone from our internal network, 10.0.0.0/8 is allowed to use
# tcp and udp for everything else.
#socks pass {
#        from: 10.0.0.0/8 to: 0.0.0.0/0
#        protocol: tcp udp
#}

socks pass {
    from: 0.0.0.0/0 to: 0.0.0.0/0
}

# last line, block everyone else.  This is the default but if you provide
# one  yourself you can specify your own logging/actions
#socks block {
#        from: 0.0.0.0/0 to: 0.0.0.0/0
#        log: connect error
#}

# route all http connects via an upstream socks server, aka "server-chaining".
#route {
# from: 10.0.0.0/8 to: 0.0.0.0/0 port = http via: socks.example.net port = socks
#}
EOF

33.1.3. 运行

sockd -f /etc/sockd.conf -D

34. frp

34.1. 安装配置

34.1.1. 初始配置

mkdir -p ~/downloads
cd ~/downloads

mkdir -p /usr/local/frp/{bin,etc} /usr/local/frp/etc/conf.d
useradd --no-create-home -s /sbin/nologin frp

wget https://github.com/fatedier/frp/releases/download/v0.52.1/frp_0.52.1_linux_amd64.tar.gz -O frp_0.52.1_linux_amd64.tar.gz

tar xf frp_0.52.1_linux_amd64.tar.gz

rm -rf /usr/local/frp
cd frp_0.52.1_linux_amd64
cp frpc frps /usr/local/frp/bin
cp *.toml /usr/local/frp/etc

find /usr/local/frp/

34.1.2. 服务端

toml set --toml-path /usr/local/frp/etc/frps.toml "bindAddr" 0.0.0.0
toml set --toml-path /usr/local/frp/etc/frps.toml --to-int "bindPort" 1840
toml set --toml-path /usr/local/frp/etc/frps.toml --to-int "kcpBindPort" 1840
toml set --toml-path /usr/local/frp/etc/frps.toml --to-int "vhostHTTPPort" 1080
toml set --toml-path /usr/local/frp/etc/frps.toml --to-int "vhostHTTPSPort" 1443
toml set --toml-path /usr/local/frp/etc/frps.toml "auth.method" token
toml set --toml-path /usr/local/frp/etc/frps.toml "auth.token" eeY8oe2Shaegeivaec5y

toml set --toml-path /usr/local/frp/etc/frps.toml "log.to" /var/log/frps.log
toml set --toml-path /usr/local/frp/etc/frps.toml "log.level" info
toml set --toml-path /usr/local/frp/etc/frps.toml --to-int "log.maxDays" 3

toml set --toml-path /usr/local/frp/etc/frps.toml "webServer.addr" 127.0.0.1
toml set --toml-path /usr/local/frp/etc/frps.toml --to-int "webServer.port" 1940
toml set --toml-path /usr/local/frp/etc/frps.toml "webServer.user" frpletnat
toml set --toml-path /usr/local/frp/etc/frps.toml "webServer.password" aez6uJ9aQuae9saeKi1i

#toml unset --toml-path /usr/local/frp/etc/frps.toml "httpPlugins"


cat << EOF > /usr/lib/systemd/system/frps.service
[Unit]
Description=Frp Server Service
After=network.target

[Service]
Type=simple
User=frp
Restart=on-failure
RestartSec=5s
ExecStart=/usr/local/frp/bin/frps -c /usr/local/frp/etc/frps.toml

[Install]
WantedBy=multi-user.target
EOF

touch /var/log/frps.log
chown frp /var/log/frps.log
systemctl enable frps --now

systemctl status frps

cat /var/log/frps.log

34.1.3. 客户端

toml set --toml-path /usr/local/frp/etc/frpc.toml "user" FrpLessNat
toml set --toml-path /usr/local/frp/etc/frpc.toml "serverAddr" 18.162.123.80
toml set --toml-path /usr/local/frp/etc/frpc.toml --to-int "serverPort" 1840
toml set --toml-path /usr/local/frp/etc/frpc.toml "transport.protocol" kcp
toml set --toml-path /usr/local/frp/etc/frpc.toml "auth.method" token
toml set --toml-path /usr/local/frp/etc/frpc.toml "auth.token" eeY8oe2Shaegeivaec5y

toml set --toml-path /usr/local/frp/etc/frpc.toml "log.to" /var/log/frpc.log
toml set --toml-path /usr/local/frp/etc/frpc.toml "log.level" info
toml set --toml-path /usr/local/frp/etc/frpc.toml --to-int "log.maxDays" 3

toml set --toml-path /usr/local/frp/etc/frpc.toml "webServer.addr" 127.0.0.1
toml set --toml-path /usr/local/frp/etc/frpc.toml --to-int "webServer.port" 1940
toml set --toml-path /usr/local/frp/etc/frpc.toml "webServer.user" frpletnat
toml set --toml-path /usr/local/frp/etc/frpc.toml "webServer.password" aez6uJ9aQuae9saeKi1i

toml set --toml-path /usr/local/frp/etc/frpc.toml --to-array "includes" '["/usr/local/frp/etc/conf.d/*.toml"]'
toml unset --toml-path /usr/local/frp/etc/frpc.toml "proxies"
toml unset --toml-path /usr/local/frp/etc/frpc.toml "visitors"

cat << EOF > /usr/lib/systemd/system/frpc.service
[Unit]
Description=Frp Client Service
After=network.target

[Service]
Type=simple
User=frp
Restart=on-failure
RestartSec=5s
ExecStart=/usr/local/frp/bin/frpc -c /usr/local/frp/etc/frpc.toml
ExecReload=/usr/local/frp/bin/frpc reload -c /usr/local/frp/etc/frpc.toml

[Install]
WantedBy=multi-user.target
EOF

touch /var/log/frpc.log
chown frp /var/log/frpc.log
systemctl enable frpc --now

systemctl status frpc

cat /var/log/frpc.log

新增内网转发配置:

cat << EOF > /usr/local/frp/etc/conf.d/ssh_x.x.x.x_22.toml
[[proxies]]
name = "ssh_x.x.x.x_22"
type = "tcp"
localIP = "x.x.x.x"
localPort = 22
remotePort = 40022
EOF

tail -f  /var/log/frpc.log

35. npm

35.1. 安装配置

35.1.1. 初始配置

npm config set registry https://registry.npm.taobao.org

cd /data/workspace/admin-vue-web
npm install
npm run build

36. lftp

36.1. 安装配置

36.1.1. 初始配置

编辑 /etc/lftp.conf,加入下面内容:

set ssl:verify-certificate no

或直接在lftp命令提示符下输入 :set ssl:verify-certificate no,然后 Enter

37. gradle

37.1. 安装配置

37.1.1. 全局源

cat << EOF > ~/.gradle/init.gradle
allprojects{
    repositories {
        mavenLocal()

        maven { url 'https://maven.aliyun.com/repository/public/' }
        maven { url 'https://maven.aliyun.com/repository/spring/'}
        maven { url 'https://maven.aliyun.com/repository/google/'}
        maven { url 'https://maven.aliyun.com/repository/gradle-plugin/'}
        maven { url 'https://maven.aliyun.com/repository/spring-plugin/'}
        maven { url 'https://maven.aliyun.com/repository/grails-core/'}
        maven { url 'https://maven.aliyun.com/repository/apache-snapshots/'}

        mavenCentral()
    }
}
EOF

37.2. 使用方法

gradle build

构建

gradle bootRun

启动SpringBoot应用

gradle publishToMavenLocal

发布到本地maven仓库

gradle distZip

构建安装包(如,可解压独立运行)

38. uWSGI

基于 CentOS 7

38.1. uWSGI

38.1.1. CentOS6

yum install -y epel
uWSGI
YUM
yum install -y uwsgi

cd /data/project/foo

uwsgi --uwsgi-socket /tmp/foo.sock \
    --uid nginx \
    --gid nginx \
    --manage-script-name \
    --mount /=main:app \
    --plugin python \
    --processes 4 \
    --threads 2 \
    --master

--uwsgi-socket /tmp/foo.sock 可以替换为 --http-socket :9090

Python3
pip3 install -i https://pypi.douban.com/simple/ uwsgi

cd /data/project/foo

/usr/local/python-3.6.2/bin/uwsgi \
    --uid nginx \
    --gid nginx \
    --uwsgi-socket /tmp/foo.sock  \
    --manage-script-name \
    --mount /=main:app \
    --processes 4 \
    --threads 2 \
    --master

--uwsgi-socket /tmp/foo.sock 可以替换为 --http-socket :9090

管理 uWSGI
保存配置
cat < EOF > /data/workspace/ttd-api-mock-server/configs/app-uwsgi.ini
[uwsgi]
uwsgi-socket = /tmp/tams.sock
pidfile = /tmp/tams.pid
stats=/tmp/tams.status
chdir = /data/workspace/ttd-api-mock-server
wsgi-file = main.py
processes = 4
threads = 2
uid = nginx
gid = nginx
manage-script-name = true
mount = /=main:app
master = true
daemonize = true
EOF
启动&停止&重载&查看状态
/usr/local/python3/bin/uwsgi /data/workspace/ttd-api-mock-server/configs/app-uwsgi.ini

/usr/local/python3/bin/uwsgi --stop /tmp/tams.pid

/usr/local/python3/bin/uwsgi --reload /tmp/tams.pid

/usr/local/python3/bin/uwsgi --connect-and-read /tmp/tams.status
Nginx
yum install -y nginx

cat < EOF > /etc/nginx/conf.d/default.conf
server {
    listen       80 default_server;
    server_name  _;

    root         /usr/share/nginx/html/;

    location / {
        try_files $uri @tams;
    }

    location @tams {
        include uwsgi_params;
        uwsgi_pass unix:/tmp/foo.sock;
    }
}
EOF

39. ElasticStack

39.1. CentOS7+ELK环境配置

39.1.1. 安装Java

yum install -y java-11-openjdk java-11-openjdk-devel java-11-openjdk-headless

39.1.2. 增加YUM源

rpm --import https://artifacts.elastic.co/GPG-KEY-elasticsearch

cat << EOF > /etc/yum.repos.d/elasticsearch.repo
[elasticsearch]
name=Elasticsearch repository for 8.x packages
baseurl=https://artifacts.elastic.co/packages/8.x/yum
gpgcheck=1
gpgkey=https://artifacts.elastic.co/GPG-KEY-elasticsearch
enabled=0
autorefresh=1
type=rpm-md
EOF

查看elasticsearch源中的包列表:

yum --disablerepo="*" --enablerepo="elasticsearch" list available
屏幕输出
Available Packages
apm-server.x86_64                              8.10.4-1                          elasticsearch
auditbeat.x86_64                               8.10.4-1                          elasticsearch
elastic-agent.x86_64                           8.10.4-1                          elasticsearch
elasticsearch.x86_64                           8.10.4-1                          elasticsearch
enterprise-search.x86_64                       8.10.4-1                          elasticsearch
filebeat.x86_64                                8.10.4-1                          elasticsearch
heartbeat-elastic.x86_64                       8.10.4-1                          elasticsearch
kibana.x86_64                                  8.10.4-1                          elasticsearch
logstash.x86_64                                1:8.10.4-1                        elasticsearch
metricbeat.x86_64                              8.10.4-1                          elasticsearch
packetbeat.x86_64                              8.10.4-1                          elasticsearch
pf-host-agent.x86_64                           8.10.4-1                          elasticsearch

39.1.3. Elasticsearch

Vo2wa2woo9ThahSh4Ood

安装
export ELASTIC_PASSWORD=$(pwgen 20 1)
echo 'Elasticsearch密码:'$ELASTIC_PASSWORD

yum --disablerepo="*" --enablerepo="elasticsearch" install -y elasticsearch
屏幕输出
--------------------------- Security autoconfiguration information ------------------------------

Authentication and authorization are enabled.
TLS for the transport and HTTP layers is enabled and configured.

The generated password for the elastic built-in superuser is : mo0BMMetyBPrIp_*hX2A

If this node should join an existing cluster, you can reconfigure this with
'/usr/share/elasticsearch/bin/elasticsearch-reconfigure-node --enrollment-token <token-here>'
after creating an enrollment token on your existing cluster.

You can complete the following actions at any time:

Reset the password of the elastic built-in superuser with
'/usr/share/elasticsearch/bin/elasticsearch-reset-password -u elastic'.

Generate an enrollment token for Kibana instances with
 '/usr/share/elasticsearch/bin/elasticsearch-create-enrollment-token -s kibana'.

Generate an enrollment token for Elasticsearch nodes with
'/usr/share/elasticsearch/bin/elasticsearch-create-enrollment-token -s node'.

-------------------------------------------------------------------------------------------------
### NOT starting on installation, please execute the following statements to configure elasticsearch service to start automatically using systemd
 sudo systemctl daemon-reload
 sudo systemctl enable elasticsearch.service
### You can start elasticsearch service by executing
 sudo systemctl start elasticsearch.service
配置
pip312 install --root-user-action=ignore -U yq

yes|cp /etc/elasticsearch/elasticsearch.yml /etc/elasticsearch/elasticsearch.yml.init

yq -yi '."cluster.name"="myapp"' /etc/elasticsearch/elasticsearch.yml
yq -yi '."node.name"="'${HOSTNAME}'"' /etc/elasticsearch/elasticsearch.yml
yq -yi '."network.host"="127.0.0.1"' /etc/elasticsearch/elasticsearch.yml
查看配置
yq -y . /etc/elasticsearch/elasticsearch.yml
屏幕输出
path.data: /var/lib/elasticsearch
path.logs: /var/log/elasticsearch
xpack.security.enabled: true
xpack.security.enrollment.enabled: true
xpack.security.http.ssl:
  enabled: true
  keystore.path: certs/http.p12
xpack.security.transport.ssl:
  enabled: true
  verification_mode: certificate
  keystore.path: certs/transport.p12
  truststore.path: certs/transport.p12
cluster.initial_master_nodes:
  - centos7
http.host: 0.0.0.0
cluster.name: myapp
node.name: centos7
network.host: 127.0.0.1

使用阿里云的ECS服务器配置Elasticsearch时,需要明确指定本机内网IP地址。否则,会出现9200端口已经启动,但无法创建或查询index。

ECS配置示例:

  • network.host: 172.24.109.12

  • discovery.seed_hosts: ["172.24.109.12"]

开机启动
systemctl enable elasticsearch
启动服务
systemctl start elasticsearch
访问
curl -k -u 'elastic:mo0BMMetyBPrIp_*hX2A' https://localhost:9200
{
  "name" : "centos7",
  "cluster_name" : "myapp",
  "cluster_uuid" : "MJ_gAE6wToyeymhvP8RZ-w",
  "version" : {
    "number" : "8.10.4",
    "build_flavor" : "default",
    "build_type" : "rpm",
    "build_hash" : "b4a62ac808e886ff032700c391f45f1408b2538c",
    "build_date" : "2023-10-11T22:04:35.506990650Z",
    "build_snapshot" : false,
    "lucene_version" : "9.7.0",
    "minimum_wire_compatibility_version" : "7.17.0",
    "minimum_index_compatibility_version" : "7.0.0"
  },
  "tagline" : "You Know, for Search"
}

39.1.4. Kibana

安装
yum --disablerepo="*" --enablerepo="elasticsearch" install -y kibana
配置

xpack.security.http.ssl: enabled: true keystore.path: certs/http.p12 xpack.security.transport.ssl: enabled: true verification_mode: certificate keystore.path: certs/transport.p12 truststore.path: certs/transport.p12

bin/elasticsearch-certutil cert -ca elastic-stack-ca.p12 -name kibana-client -dns <your_kibana_hostname> openssl pkcs12 -in kibana-client.p12 -cacerts -nokeys -out kibana-ca.crt

elasticsearch-certutil csr -name kibana-server -dns example.com,www.example.com

bin/elasticsearch-reset-password -u

/usr/share/elasticsearch/bin/elasticsearch-reset-password -u kibana_system --auto --batch

Password for the [kibana_system] user successfully reset. New value: g8mEFUrQclFstSniVqsj

yes|cp /etc/kibana/kibana.yml /etc/kibana/kibana.yml.init

domain=kibana.foo.com
cert_name=foo.com

yq -yi '.server.port=5601' /etc/kibana/kibana.yml
yq -yi '.server.host="0.0.0.0"' /etc/kibana/kibana.yml
yq -yi '.server.name="mykibana"' /etc/kibana/kibana.yml
yq -yi '.server.publicBaseUrl="https://kibana.no1bing.com:5601"' /etc/kibana/kibana.yml
yq -yi '.server.ssl.enabled=true' /etc/kibana/kibana.yml
yq -yi '.server.ssl.certificate="/etc/letsencrypt/live/'${cert_name}'/fullchain.pem"' /etc/kibana/kibana.yml
yq -yi '.server.ssl.key="/etc/letsencrypt/live/'${cert_name}'/privkey.pem"' /etc/kibana/kibana.yml
yq -yi '."i18n.locale"="zh-CN"' /etc/kibana/kibana.yml

yq -yi '.elasticsearch.hosts=["https://localhost:9200"]' /etc/kibana/kibana.yml
yq -yi '.elasticsearch.username="kibana_system"' /etc/kibana/kibana.yml
yq -yi '.elasticsearch.password="g8mEFUrQclFstSniVqsj"' /etc/kibana/kibana.yml

# for read /etc/elasticsearch/certs/http_ca.crt
gpasswd -a kibana elasticsearch
grpunconv

yq -yi '.elasticsearch.ssl.verificationMode="none"' /etc/kibana/kibana.yml
yq -yi '.elasticsearch.ssl.certificateAuthorities=["/etc/elasticsearch/certs/http_ca.crt"]' /etc/kibana/kibana.yml


yq -yi '.elasticsearch.ssl.truststore.path="/etc/elasticsearch/certs/transport.p12"' /etc/kibana/kibana.yml
yq -yi '.elasticsearch.ssl.truststore.password="_5DC1duxS8aXHgtMGjNkWg"' /etc/kibana/kibana.yml
yq -yi '.elasticsearch.ssl.keystore.path="/etc/elasticsearch/certs/transport.p12"' /etc/kibana/kibana.yml
yq -yi '.elasticsearch.ssl.keystore.password="_5DC1duxS8aXHgtMGjNkWg"' /etc/kibana/kibana.yml

yq -yi '.xpack.security.enabled=true' /etc/kibana/kibana.yml
yq -yi '.xpack.security.encryptionKey="'$(pwgen 32 1)'"' /etc/kibana/kibana.yml

yq -yi '.xpack.encryptedSavedObjects.encryptionKey="'$(pwgen 32 1)'"' /etc/kibana/kibana.yml

yq -yi '.xpack.reporting.kibanaServer.hostname="'${domain}'"' /etc/kibana/kibana.yml
yq -yi '.xpack.reporting.encryptionKey="'$(pwgen 32 1)'"' /etc/kibana/kibana.yml
yq -yi '.xpack.reporting.roles.enabled=false' /etc/kibana/kibana.yml

yq -yi '.xpack.screenshotting.browser.chromium.disableSandbox=true' /etc/kibana/kibana.yml


sed -i -E 's/^--openssl-legacy-provider$/#--openssl-legacy-provider/' /etc/kibana/node.options

chmod 755 /etc/letsencrypt/{archive,live}
chmod 644 /etc/letsencrypt/live/${cert_name}/privkey.pem
查看配置
yq -y . /etc/kibana/kibana.yml
屏幕输出
logging:
  appenders:
    file:
      type: file
      fileName: /var/log/kibana/kibana.log
      layout:
        type: json
  root:
    appenders:
      - default
      - file
pid.file: /run/kibana/kibana.pid
server.host: 0.0.0.0
server.name: mykibana
server.ssl.enabled: true
server.ssl.certificate: /etc/letsencrypt/live/kibana.foo.com/fullchain.pem
server.ssl.key: /etc/letsencrypt/live/kibana.foo.com/privkey.pem
i18n.locale: zh-CN
elasticsearch.hosts:
  - https://localhost:9200
elasticsearch.username: elastic
elasticsearch.password: mo0BMMetyBPrIp_*hX2A
xpack.security.enabled: true
xpack.security.encryptionKey: shaireeseimuaphohkaph4queiTh5aJe

如果Elasticsearch的 "network.host" 参数值为具体的IP地址,比如 "172.24.109.12"

那么,Kibana中的 "elasticsearch.hosts" 同样需要设置为 "172.24.109.12",而不能使用 "0.0.0.0"

开机启动
systemctl enable kibana
启动服务
systemctl start kibana
Kibana参数错误导致日志文件没数据,手动排查故障:cd /usr/share/kibana && runuser -u kibana /usr/share/kibana/bin/kibana
访问

使用Elasticsearch随机密码登录 http://172.24.109.12:5601/

安装Logstash
yum --disablerepo="*" --enablerepo="elasticsearch" install -y logstash

39.1.5. 配置

配置文件路径: /etc/elasticsearch/elasticsearch.yml

监听网络IP
network.host: 192.168.0.4
允许的客户端
discovery.seed_hosts: ["192.168.0.0/24"]`

network.host 参数不支持 0.0.0.0

Elasticsearch 在老旧硬件上需要关闭机器学习模块,

在配置文件中增加参数: xpack.ml.enabled: false

39.1.6. logstash

yum --disablerepo="*" --enablerepo="elasticsearch" install -y logstash

openssl x509 -fingerprint -sha256 -noout -in /etc/elasticsearch/certs/http_ca.crt | awk --field-separator="=" '{print $2}' | sed 's/://g'

/usr/share/elasticsearch/bin/elasticsearch-keystore show xpack.security.transport.ssl.keystore.secure_password /usr/share/elasticsearch/bin/elasticsearch-keystore show xpack.security.transport.ssl.truststore.secure_password

yum --disablerepo="*" --enablerepo="elasticsearch" install -y filebeat

yes|cp /etc/filebeat/filebeat.yml /etc/filebeat/filebeat.yml.init

yq -yi '."output.elasticsearch".hosts=["103.231.173.43:9200"]' /etc/filebeat/filebeat.yml yq -yi '."output.elasticsearch".protocol="https"' /etc/filebeat/filebeat.yml yq -yi '."output.elasticsearch".username="elastic"' /etc/filebeat/filebeat.yml yq -yi '."output.elasticsearch".password="mo0BMMetyBPrIp_*hX2A"' /etc/filebeat/filebeat.yml yq -yi '."output.elasticsearch".ssl.enabled=true' /etc/filebeat/filebeat.yml yq -yi '."output.elasticsearch".ssl.ca_trusted_fingerprint="4CE07DBB4193918B332063A51DC0647A0D0FC86CF5DC01AB8CF5FB7459CBA97E"' /etc/filebeat/filebeat.yml

yq -yi '."setup.kibana".host="https://kibana.no1bing.com:5601"' /etc/filebeat/filebeat.yml

filebeat modules list filebeat modules enable nginx

yes|cp /etc/filebeat/modules.d/nginx.yml /etc/filebeat/modules.d/nginx.yml.init

yq -yi '.[0].access.enabled=true' /etc/filebeat/modules.d/nginx.yml yq -yi '.[0].access.var.paths=["/root/downloads/log/jk.no1bing.com_access.log"]' /etc/filebeat/modules.d/nginx.yml access.inputpipeline: filebeat-6.4.3-nginx-access-custom

yq -yi '.[0].error.enabled=true' /etc/filebeat/modules.d/nginx.yml yq -yi '.[0].error.var.paths=["/root/downloads/log/jk.no1bing.com_error.log"]' /etc/filebeat/modules.d/nginx.yml

filebeat test config filebeat test config -e

Config OK

filebeat setup -e

systemctl enable filebeat --now systemctl status filebeat

39.2. CentOS8+ELK环境配置

39.2.1. 安装ELK

39.2.2. Logstash

“如何将Nginx的Web日志导入Elasticsearch?”

日志处理管道

日志记录增加到Elasticsearch时,需要使用管道实时清洗、处理日志格式。

Nginx日志格式
log_format  main  '$host $remote_addr - $remote_user [$time_local] "$request" '
                    '$status $body_bytes_sent "$http_referer" '
                    '"$http_user_agent" "$http_x_forwarded_for"';

执行以下命令,在Elasticsearch中增加管道:

curl -H "Content-Type: application/json" -X PUT http://0.0.0.0:9200/_ingest/pipeline/web_log -d '
{
    "description": "",
    "processors": [
        {
            "dissect": {
                "field": "message",
                "pattern": "%{domain} %{server_port} %{client_ip} - - [%{web_timestamp}] \"%{method} %{uri} HTTP/%{version}\" %{status} %{response_time} %{body_bytes} \"%{referer}\" \"%{user_agent}\" \"%{x_forwarded_for}\" %{server_ip}"
            }
        },
        {
            "grok": {
                "field": "uri",
                "on_failure": [
                    {
                "set": {
                    "field": "ext",
                    "value": ""
                }
                    },
                    {
                "set": {
                    "field": "url_base",
                    "value": "{{uri}}"
                }
                    }
                ],
                "patterns": [
                    "%{GREEDYDATA:url_base}\\.%{WORD:ext}"
                ]
            }
        },
        {
            "geoip": {
                "field": "client_ip",
                "target_field": "client_ip_geo"
            }
        },
        {
            "geoip": {
                "field": "server_ip",
                "target_field": "server_ip_geo"
            }
        },
        {
            "user_agent": {
                "field": "user_agent"
            }
        },
        {
            "date": {
                "field": "web_timestamp",
                "target_field": "@timestamp",
                "formats": [
                    "dd/MMM/yyyy:HH:mm:ss Z"
                ],
                "timezone" : "Asia/Shanghai"
            }
        },
        {
            "convert": {
                "field": "server_port",
                "type": "integer"
            }
        },
        {
            "convert": {
                "field": "status",
                "type": "integer"
            }
        },
        {
            "convert": {
                "field": "response_time",
                "type": "float"
            }
        },
        {
            "convert": {
                "field": "body_bytes",
                "type": "integer"
            }
        },
        {
            "set": {
                "field": "hour_of_day",
                "value": 12
            }
        },
        {
            "grok": {
                "field": "web_timestamp",
                "patterns": [
                    "\\d+/\\w+/\\d+:%{NUMBER:hour_of_day:int}:\\d+:\\d+ \\+\\d+"
                ]
            }
        }
    ]
}
'
准备日志文件
日志文件存放路径

~/es_log/nginx_logs

[root@dell7 nginx_logs]# ls
access.log-20200501  access.log-20200512  access.log-20200523  access.log-20200603
access.log-20200502  access.log-20200513  access.log-20200524  access.log-20200604
access.log-20200503  access.log-20200514  access.log-20200525  access.log-20200605
access.log-20200504  access.log-20200515  access.log-20200526  access.log-20200606
access.log-20200505  access.log-20200516  access.log-20200527  access.log-20200607
access.log-20200506  access.log-20200517  access.log-20200528  access.log-20200608
access.log-20200507  access.log-20200518  access.log-20200529  access.log-20200609
access.log-20200508  access.log-20200519  access.log-20200530  access.log-20200610
access.log-20200509  access.log-20200520  access.log-20200531
access.log-20200510  access.log-20200521  access.log-20200601
access.log-20200511  access.log-20200522  access.log-20200602
配置文件
mkdir -p ~/es_log

cat << EOF > ~/es_log/logstash.conf
input{
  file {
    # https://www.elastic.co/guide/en/logstash/current/plugins-inputs-file.html#plugins-inputs-file-delimiter
    # 默认以"\n"分割,文件必须至少一个换行符
    path => "${HOME}/es_log/nginx_logs/access.log*"
    start_position => "end"
    stat_interval => "1 second"
  }
}
output{
    elasticsearch {
            #这里可以是数组,可以是多个节点的地址,会自动启用负载均衡
            hosts => ["0.0.0.0:9200"]
            index => "foo_com"
            pipeline => "web_log"
    }
    #file  {
    #    path => "/var/log/logstash.log"
    #    codec => json
    #}
}
EOF

如果Elasticsearch的 "network.host" 参数值为具体的IP地址,比如 "172.24.109.12"

那么,Logstash配置中的 output.elasticsearch.hosts 需要设置为 ["172.24.109.12:9200"]

导入Nginx日志到Elasticsearch

导入数据到ES:

/usr/share/logstash/bin/logstash -f ~/es_log/logstash.conf

每次运行的结果会记录到文件:

/usr/share/logstash/data/plugins/inputs/file/.sincedb_100caf6f694120ba2483577719e4a564

文件内容为:

270004749 0 2049 584450300 1592393767.626652 /root/es_log/nginx_logs/access.log-20200501
272281186 0 2049 471367497 1592394059.861428 /root/es_log/nginx_logs/access.log-20200502

如果想重新导入,删除 .sincedb 文件即可。

39.3. ElasticStack使用教程

我们以 wecompany 公司的员工信息管理为例来学习 Elasticsearch 中的基本操作。

39.3.1. 索引文档

向名称为 wecompany 的索引中添加类型为 employee 的3个员工信息的文档
curl -H "Content-Type: application/json" -X PUT 'http://localhost:9200/wecompany/employee/1?pretty' -d '
{
    "first_name" : "John",
    "last_name" :  "Smith",
    "age" :        25,
    "about" :      "I love to go rock climbing",
    "interests": [ "sports", "music" ]
}
'

curl -H "Content-Type: application/json" -X PUT 'http://localhost:9200/wecompany/employee/2?pretty' -d '
{
    "first_name" :  "Jane",
    "last_name" :   "Smith",
    "age" :         32,
    "about" :       "I like to collect rock albums",
    "interests":  [ "music" ]
}
'

curl -H "Content-Type: application/json" -X PUT 'http://localhost:9200/wecompany/employee/3?pretty' -d '
{
    "first_name" :  "Douglas",
    "last_name" :   "Fir",
    "age" :         35,
    "about":        "I like to build cabinets",
    "interests":  [ "forestry" ]
}
'

39.3.2. 搜索

获取ID为1的文档

curl -X GET 'http://localhost:9200/wecompany/employee/1?pretty'

{
  "_index" : "wecompany",
  "_type" : "employee",
  "_id" : "1",
  "_version" : 1,
  "found" : true,
  "_source" : {
    "first_name" : "John",
    "last_name" : "Smith",
    "age" : 25,
    "about" : "I love to go rock climbing",
    "interests" : [
      "sports",
      "music"
    ]
  }
}
搜索姓氏为 Smith 的员工信息

curl -X GET 'http://localhost:9200/wecompany/employee/_search?q=last_name:Smith&pretty'

{
  "took" : 204,
  "timed_out" : false,
  "_shards" : {
    "total" : 5,
    "successful" : 5,
    "skipped" : 0,
    "failed" : 0
  },
  "hits" : {
    "total" : 2,
    "max_score" : 0.2876821,
    "hits" : [
      {
        "_index" : "wecompany",
        "_type" : "employee",
        "_id" : "2",
        "_score" : 0.2876821,
        "_source" : {
          "first_name" : "Jane",
          "last_name" : "Smith",
          "age" : 32,
          "about" : "I like to collect rock albums",
          "interests" : [
            "music"
          ]
        }
      },
      {
        "_index" : "wecompany",
        "_type" : "employee",
        "_id" : "1",
        "_score" : 0.2876821,
        "_source" : {
          "first_name" : "John",
          "last_name" : "Smith",
          "age" : 25,
          "about" : "I love to go rock climbing",
          "interests" : [
            "sports",
            "music"
          ]
        }
      }
    ]
  }
}
使用查询表达式搜索姓氏为Smith的员工信息
curl -H "Content-Type: application/json" -X GET 'http://localhost:9200/wecompany/employee/_search?pretty' -d '
{
    "query" :  {
        "match" : {
            "last_name" : "Smith"
        }
    }
}
'
返回结果同上
姓氏为Smith且年龄大于30的复杂条件搜索员工信息
curl -H "Content-Type: application/json" -X GET 'http://localhost:9200/wecompany/employee/_search?pretty' -d '
{
    "query": {
        "bool": {
            "must": {
                "match": {
                    "last_name": "smith"
                }
            },
            "filter": {
                "range": {
                    "age": {
                        "gt": 30
                    }
                }
            }
        }
    }
}
'
{
  "took" : 39,
  "timed_out" : false,
  "_shards" : {
    "total" : 5,
    "successful" : 5,
    "skipped" : 0,
    "failed" : 0
  },
  "hits" : {
    "total" : 1,
    "max_score" : 0.2876821,
    "hits" : [
      {
        "_index" : "wecompany",
        "_type" : "employee",
        "_id" : "2",
        "_score" : 0.2876821,
        "_source" : {
          "first_name" : "Jane",
          "last_name" : "Smith",
          "age" : 32,
          "about" : "I like to collect rock albums",
          "interests" : [
            "music"
          ]
        }
      }
    ]
  }
}
全文搜索喜欢攀岩(rock climbing)的员工信息
curl -H "Content-Type: application/json" -X GET 'http://localhost:9200/wecompany/employee/_search?pretty' -d '
{
    "query" :  {
        "match" : {
            "about" : "rock climbing"
        }
    }
}
'
{
  "took" : 10,
  "timed_out" : false,
  "_shards" : {
    "total" : 5,
    "successful" : 5,
    "skipped" : 0,
    "failed" : 0
  },
  "hits" : {
    "total" : 2,
    "max_score" : 0.5753642,
    "hits" : [
      {
        "_index" : "wecompany",
        "_type" : "employee",
        "_id" : "1",
        "_score" : 0.5753642,
        "_source" : {
          "first_name" : "John",
          "last_name" : "Smith",
          "age" : 25,
          "about" : "I love to go rock climbing",
          "interests" : [
            "sports",
            "music"
          ]
        }
      },
      {
        "_index" : "wecompany",
        "_type" : "employee",
        "_id" : "2",
        "_score" : 0.2876821,
        "_source" : {
          "first_name" : "Jane",
          "last_name" : "Smith",
          "age" : 32,
          "about" : "I like to collect rock albums",
          "interests" : [
            "music"
          ]
        }
      }
    ]
  }
}

此外,将上述请求中的 match 换成 match_phrase 可以精确匹配短语 rock climbing 的结果。在 query 同级添加 highlight 参数可以在结果中用 <em></em> 标签标注匹配的关键词:

{
"query" :{ ... }
    "highlight" : {
            "fields" : {
                "about" : {}
            }
        }
}
短语搜索
GET /megacorp/employee/_search
{
  "query": {
    "match_phrase": {
      "about": "rock climbing"
    }
  }
}
{
  ...
  "hits": {
    "total": 1,
    "max_score": 0.23013961,
    "hits": [
      {
        ...
        "_score": 0.23013961,
        "_source": {
          "first_name": "John",
          "last_name": "Smith",
          "age": 25,
          "about": "I love to go rock climbing",
          "interests": [
            "sports",
            "music"
          ]
        }
      }
    ]
  }
}
高亮我们的搜索
GET /megacorp/employee/_search
{
  "query": {
    "match_phrase": {
      "about": "rock climbing"
    }
  },
  "highlight": {
    "fields": {
      "about": {}
    }
  }
}
{
  ...
  "hits": {
    "total": 1,
    "max_score": 0.23013961,
    "hits": [
      {
        ...
        "_score": 0.23013961,
        "_source": {
          "first_name": "John",
          "last_name": "Smith",
          "age": 25,
          "about": "I love to go rock climbing",
          "interests": [
            "sports",
            "music"
          ]
        },
        "highlight": {
          "about": [
            "I love to go <em>rock</em> <em>climbing</em>"
          ]
        }
      }
    ]
  }
}

39.3.3. 聚合

查询聚合结果
curl -H "Content-Type: application/json" -X GET 'http://localhost:9200/wecompany/employee/_search?pretty' -d '
{
    "aggs": {
        "all_interests": {
            "terms": { "field": "interests.keyword" }
        }
    }
}
'
{
  "took" : 72,
  "timed_out" : false,
  "_shards" : {
    "total" : 5,
    "successful" : 5,
    "skipped" : 0,
    "failed" : 0
  },
  "hits" : {
    "total" : 3,
    "max_score" : 1.0,
    "hits" : [
      {
        "_index" : "wecompany",
        "_type" : "employee",
        "_id" : "2",
        "_score" : 1.0,
        "_source" : {
          "first_name" : "Jane",
          "last_name" : "Smith",
          "age" : 32,
          "about" : "I like to collect rock albums",
          "interests" : [
            "music"
          ]
        }
      },
      {
        "_index" : "wecompany",
        "_type" : "employee",
        "_id" : "1",
        "_score" : 1.0,
        "_source" : {
          "first_name" : "John",
          "last_name" : "Smith",
          "age" : 25,
          "about" : "I love to go rock climbing",
          "interests" : [
            "sports",
            "music"
          ]
        }
      },
      {
        "_index" : "wecompany",
        "_type" : "employee",
        "_id" : "3",
        "_score" : 1.0,
        "_source" : {
          "first_name" : "Douglas",
          "last_name" : "Fir",
          "age" : 35,
          "about" : "I like to build cabinets",
          "interests" : [
            "forestry"
          ]
        }
      }
    ]
  },
  "aggregations" : {
    "all_interests" : {
      "doc_count_error_upper_bound" : 0,
      "sum_other_doc_count" : 0,
      "buckets" : [
        {
          "key" : "music",
          "doc_count" : 2
        },
        {
          "key" : "forestry",
          "doc_count" : 1
        },
        {
          "key" : "sports",
          "doc_count" : 1
        }
      ]
    }
  }
}

我们可以看到两个职员对音乐有兴趣,一个喜欢林学,一个喜欢运动。这些数据并没有被预先计算好,它们是实时的从匹配查询语句的文档中动态计算生成的。如果我们想知道所有姓"Smith"的人最大的共同点(兴趣爱好),我们只需要增加合适的语句既可:

GET /megacorp/employee/_search
{
  "query": {
    "match": {
      "last_name": "smith"
    }
  },
  "aggs": {
    "all_interests": {
      "terms": {
        "field": "interests.keyword"
      }
    }
  }
}

all_interests 聚合已经变成只包含和查询语句相匹配的文档了:

{
  ...
  "hits": {
    ...
  },
  "aggregations": {
    "all_interests": {
      "buckets": [
        {
          "key": "music",
          "doc_count": 2
        {
          "key": "sports",
          "doc_count": 1
        }
      ]
    }
  }
}

聚合也允许分级汇总。例如,让我们统计每种兴趣下职员的平均年龄:

GET /megacorp/employee/_search
{
  "aggs": {
    "all_interests": {
      "terms": {
        "field": "interests.keyword"
      },
      "aggs": {
        "avg_age": {
          "avg": {
            "field": "age"
          }
        }
      }
    }
  }
}

虽然这次返回的聚合结果有些复杂,但任然很容易理解:

{
  ...
  "hits": {
    ...
  },
  "all_interests": {
    "buckets": [
      {
        "key": "music",
        "doc_count": 2,
        "avg_age": {
          "value": 28.5
        }
      },
      {
        "key": "forestry",
        "doc_count": 1,
        "avg_age": {
          "value": 35
        }
      },
      {
        "key": "sports",
        "doc_count": 1,
        "avg_age": {
          "value": 25
        }
      }
    ]
  }
}

该聚合结果比之前的聚合结果要更加丰富。我们依然得到了兴趣以及数量(指具有该兴趣的员工人数)的列表,但是现在每个兴趣额外拥有 avg_age 字段来显示具有该兴趣员工的平均年龄。

39.3.4. 更新文档

更新ID为2的文档,只需再次PUT即可
curl -H "Content-Type: application/json" -X PUT 'http://localhost:9200/wecompany/employee/2?pretty' -d '
{
    "first_name" :  "Jane",
    "last_name" :   "Smith",
    "age" :         33,
    "about" :       "I like to collect rock albums",
    "interests":  [ "music" ]
}
'

39.3.5. 删除文档

curl -H "Content-Type: application/json" -X DELETE 'http://localhost:9200/wecompany/employee/1?pretty'

39.3.6. 结构化查询

最重要的查询过滤语句
term 查询

term 主要用于精确匹配哪些值,比如数字,日期,布尔值或 not_analyzed的字符串(未经分析的文本数据类型):

{ "term": { "age": 26}}
{ "term": { "date": "2014-09-01" }}
{ "term": { "public": true }}
{ "term": { "tag": "full_text"}}
terms 查询

termsterms 过滤有点类似,但 terms 允许指定多个匹配条件。 如果某个字段指定了多个值,那么文档需要一起去做匹配:

{
  "terms": {
    "tag": [
      "search",
      "full_text",
      "nosql"
    ]
  }
}
range 过滤
{
  "range": {
    "age": {
      "gte": 20,
      "lt": 30
    }
  }
}

范围操作符包含:

gt

大于

gte

大于等于

lt

小于

lte

小于等于

exists 和 missing 过滤

existsmissing 过滤SQL语句中的过滤可以用于查找文档中是否包含指定字段或没有某个字段,类似于 IS_NULL 条件

{
  "exists": {
    "field": "title"
  }
}

这两个过滤只是针对已经查出一批数据来,但是想区分出某个字段是否存在的时候使用。

39.3.7. DSL查询实例

{
  "query": {
    "match": {
      "tweet": "elasticsearch"
    }
  }
}
{
  "bool": {
    "must": {
      "match": {
        "tweet": "elasticsearch"
      }
    },
    "must_not": {
      "match": {
        "name": "mary"
      }
    },
    "should": {
      "match": {
        "tweet": "full text"
      }
    }
  }
}
{
  "bool": {
    "must": {
      "match": {
        "email": "business opportunity"
      }
    },
    "should": [
      {
        "match": {
          "starred": true
        }
      },
      {
        "bool": {
          "must": {
            "folder": "inbox"
          },
          "must_not": {
            "spam": true
          }
        }
      }
    ],
    "minimum_should_match": 1
  }
}

39.4. 从MySQL同步数据到ElasticSearch

参考文档:

/usr/share/logstash/bin/logstash-plugin install logstash-input-jdbc

wget -O /usr/share/logstash/lib/mysql-connector-java-8.0.18.jar https://repo1.maven.org/maven2/mysql/mysql-connector-java/8.0.18/mysql-connector-java-8.0.18.jar

[ ! -d "/data/redas_meta_sync" ] && mkdir /data/redas_meta_sync/logs
chown -R logstash:logstash /usr/share/logstash/data /data/redas_meta_sync

cat << EOF >  /data/redas_meta_sync/logstash_mysql_es_.conf
input{
	jdbc {
		# 驱动方式
		jdbc_driver_library => "/usr/share/logstash/lib/mysql-connector-java-8.0.18.jar"
		# 驱动类名
		jdbc_driver_class => "com.mysql.cj.jdbc.Driver"
		# mysql 数据库链接,blog为数据库名 &useSSL=false 这个是为了防止不支持ssl连接的问题
		jdbc_connection_string => "jdbc:mysql://localhost:3306/redas?serverTimezone=Asia/Shanghai&useUnicode=yes&characterEncoding=UTF-8"
		# 连接数据库用户名
		jdbc_user => "redas"
		# 连接数据库密码
		jdbc_password => "geek"
		# 是否启用分页
		jdbc_paging_enabled => "true"
		jdbc_page_size => "50000"
		# 设置监听间隔  各字段含义(由左至右)分、时、天、月、年,全部为*默认含>义为每分钟都更新
		schedule => "*/1 * * * * "
		# 直接写sql语句用这个
		statement => "select `id`, `content` from `recr_page_storage` where id > :sql_last_value order by id"
		# 是否需要记录某个column 的值,如果 record_last_run 为真,可以自定义我们需要 track 的 column 名称,此时该参数就要为 true. 否则默认 track 的是 timestamp 的值
		use_column_value => true
		# 是否记录上次执行结果, 如果为真,将会把上次执行到的 tracking_column 字段的值记录下来,保存到 last_run_metadata_path 指定的文件中
		record_last_run => true
		# 如果 use_column_value 为真,需配置此参数. track 的数据库 column 名,该 column 必须是递增的.比如:ID.
		tracking_column_type => "numeric"
		tracking_column => "id"
		# 保存上一次运行>的信息(tracking_column)
		last_run_metadata_path => "/data/redas_meta_sync/last_run.txt"
		# 是否清除 last_run_metadata_path 的记录,如果为真那么每次都相当于从头开始查询所有的数据库记录
		clean_run => false
	}
}
filter {
      json {
        source => "content"
        remove_field => "content"
      }
}
output{
	elasticsearch {
		#这里可以是数组,可以是多个节点的地址,会自动启用负载均衡
		hosts => ["127.0.0.1:9200"]
		index => "redas_test"
		# 将表Iid作为ES的主键,防止数据重复
		document_id => "%{id}"
	}
	file {
		path => "/data/redas_meta_sync/logs/%{+YYYY-MM-dd}.log"
		codec => json_lines
	}
}
EOF

cat << EOF > /usr/lib/systemd/system/redas_meta_sync.service
[Unit]
Description=Redas Position Infomation Auto-Sync to ElasticSearch
After=network.target remote-fs.target nss-lookup.target mysqld.service elasticsearch.service

[Service]
Type=simple
User=logstash
Group=logstash
WorkingDirectory=/data/redas_meta_sync
ExecStart=/usr/share/logstash/bin/logstash -f /data/redas_meta_sync/logstash_mysql_es_.conf -l /data/redas_meta_sync/logs --path.settings /etc/logstash
StandardOutput=null
StandardError=null

[Install]
WantedBy=multi-user.target
EOF

systemctl enable redas_meta_sync
systemctl start redas_meta_sync

39.5. ElasticSearch在线迁移

参考文档:

cat << EOF > transfer_es.conf
input {
	elasticsearch {
		hosts => ["http://192.168.2.4:9200"]
		index => "*"
		docinfo => true
	}
}
output {
	elasticsearch {
		hosts => ["http://127.0.0.1:9200"]
		index => "%{[@metadata][_index]}"
	}
}
EOF

/usr/share/logstash/bin/logstash -f transfer_es.conf

40. youtube-dl

40.1. 使用实例

youtube-dl --sub-lang en --write-sub --sub-format srt https://www.youtube.com/watch?v=ptVqRCRPN9I
youtube-dl --limit-rate 10M --ignore-errors --convert-subs srt -f 137+140 --sub-lang en --write-sub 'https://www.youtube.com/watch?v=ptVqRCRPN9I'
youtube-dl --ratelimit 1M --ignore-errors --convert-subs srt -f 137+140 --sub-lang en --write-sub 'https://www.youtube.com/watch?v=ptVqRCRPN9I'
youtube-dl -r 20k --ignore-errors --convert-subs srt -f 137+140 --sub-lang en --write-sub 'https://www.youtube.com/watch?v=ptVqRCRPN9I'
youtube-dl -r 20k --ignore-errors --convert-subs srt -f 137+140 --sub-lang en --write-sub 'https://www.youtube.com/watch?v=ptVqRCRPN9I'
youtube-dl -r 1m --ignore-errors --convert-subs srt -f 137+140 --sub-lang en --write-sub 'https://www.youtube.com/watch?v=ptVqRCRPN9I'
youtube-dl -r 1M --ignore-errors --convert-subs srt -f 137+140 --sub-lang en --write-sub 'https://www.youtube.com/watch?v=ptVqRCRPN9I'
youtube-dl -k  --ignore-errors --convert-subs srt --write-auto-sub --batch-file sub.list
youtube-dl -k  --ignore-errors --convert-subs srt --write-auto-sub -w --batch-file sub.list
youtube-dl -s -k  --ignore-errors --convert-subs srt --write-auto-sub -w --batch-file sub.list
youtube-dl --skip-download -k  --ignore-errors --convert-subs srt --write-auto-sub--batch-file sub.list
youtube-dl --skip-download -k  --ignore-errors --convert-subs srt --write-auto-sub --batch-file sub.list
youtube-dl -k -r 10m --ignore-errors --write-auto-sub --convert-subs srt -f 137+140 'https://www.youtube.com/user/theofficialpeppa/videos'
youtube-dl  --write-auto-sub --convert-subs srt -f 18 'https://www.youtube.com/watch?v=ZczS3DbCDwU'
youtube-dl -F 'https://www.youtube.com/user/theofficialpeppa/videos'
youtube-dl -F 'https://www.youtube.com/watch?v=ZczS3DbCDwU'
youtube-dl  --write-auto-sub --convert-subs srt -f 135+140 'https://www.youtube.com/watch?v=ZczS3DbCDwU'
youtube-dl  --write-auto-sub --convert-subs srt -f 135+140 'https://www.youtube.com/watch?v=ZczS3DbCDwU'
youtube-dl -k --ignore-errors --convert-subs srt -f 137+140 -j --sub-lang en --write-sub 'https://www.youtube.com/user/rachelsenglish/videos'
youtube-dl -F 'https://www.youtube.com/watch?v=qKs6L92qoKo'
youtube-dl -f 22  'https://www.youtube.com/watch?v=qKs6L92qoKo'
youtube-dl -F 'https://www.youtube.com/watch?v=OC6AFSZLtnk&list=PLsVSF-hJhvBKHs3gYB90seTLUwZW0qZUN&index=7'
youtube-dl  -F 'https://www.youtube.com/watch?v=iBKOVdhR22c&list=PLsVSF-hJhvBKHs3gYB90seTLUwZW0qZUN&index=6'
youtube-dl -F 'https://www.youtube.com/watch?v=OC6AFSZLtnk'
youtube-dl -L 'https://www.youtube.com/watch?v=OC6AFSZLtnk'
youtube-dl -F 'https://www.youtube.com/watch?v=OC6AFSZLtnk'
youtube-dl -U
youtube-dl -F 'https://www.youtube.com/watch?v=OC6AFSZLtnk'
youtube-dl -f137+140 'https://www.youtube.com/watch?v=OC6AFSZLtnk'
youtube-dl -F 'https://www.youtube.com/watch?v=lrDlTEbHw3g'
youtube-dl -f 137+140 'https://www.youtube.com/watch?v=lrDlTEbHw3g'
youtube-dl -U  to update
youtube-dl -f 137+140 'https://www.youtube.com/watch?v=lrDlTEbHw3g'

41. maven

41.1. 安装配置

41.1.1. 全局源

cat << EOF > ~/.m2/settings.xml
<settings>
  <mirrors>
    <mirror>
      <id>aliyun</id>
      <name>Aliyun Central</name>
        <url>http://maven.aliyun.com/nexus/content/groups/public/</url>
      <mirrorOf>central</mirrorOf>
    </mirror>
  </mirrors>
</settings>
EOF

41.2. 使用方法

41.2.1. Build Lifecycle

mvn clean

清除

mvn compile

编译,compile the source code of the project

mvn validate

验证mvn配置,validate the project is correct and all necessary information is available

mvn test

运行单元测试,test the compiled source code using a suitable unit testing framework. These tests should not require the code be packaged or deployed

mvn verify

验证编译后的程序,run any checks on results of integration tests to ensure quality criteria are met

mvn install

安装应用到maven目录,install the package into the local repository, for use as a dependency in other projects locally

mvn deploy

编译,done in the build environment, copies the final package to the remote repository for sharing with other developers and projects.

mvn package

打包,take the compiled code and package it in its distributable format, such as a JAR.

mvn compile war:war

Build a WAR file.

mvn war:exploded

Create an exploded webapp in a specified directory.

启动springboot

mvn spring-boot:run

使用Java启动jar包

java -jar target/accessing-data-jpa-0.0.1-SNAPSHOT.jar

42. MongoDB

42.1. 安装配置

基于 CentOS8

42.1.1. 安装

cat > /etc/yum.repos.d/mongodb-org-4.2.repo << EOF
[mongodb-org-4.2]
name=MongoDB Repository
baseurl=https://repo.mongodb.org/yum/redhat/\$releasever/mongodb-org/4.2/x86_64/
gpgcheck=1
enabled=1
gpgkey=https://www.mongodb.org/static/pgp/server-4.2.asc
EOF

dnf install -y mongodb-org
文件目录结构
配置文件

/etc/mongod.conf (YML格式)

配置文件说明
  1. storage.dbPath: 指定数据目录,默认 /var/lib/mongo

  2. systemLog.path: 指定日志目录,默认 /var/log/mongodb

  3. net.port: 指定监听端口,默认 27017

  4. net.bindIp: 指定监听IP,默认 127.0.0.1

默认情况下,只有 mongod 用户有数据目录和日志目录的读写权限。

42.1.2. 配置

禁用SELINUX(重启生效)
echo SELINUX=disabled>/etc/selinux/config
echo SELINUXTYPE=targeted>>/etc/selinux/config
设置 ulimit
echo "*               soft   nofile            65535" >> /etc/security/limits.conf
echo "*               hard   nofile            65535" >> /etc/security/limits.conf
其它优化
echo 'never' > /sys/kernel/mm/transparent_hugepage/enabled

这个设置需要每次开机时设置,需要自行处理。

启用权限验证
echo 'security:' >> /etc/mongod.conf
echo '  authorization: enabled' >> /etc/mongod.conf
开机启动
  1. 开机启动 systemctl enable mongod

  2. 禁用启动 systemctl disable mongod

服务管理
  1. 启动服务 systemctl start mongod

  2. 查看状态 systemctl status mongod

  3. 重启服务 systemctl restart mongod

  4. 停止服务 systemctl stop mongod

验证服务
  1. 查看端口 ss -antpl|grep 27017

  2. 查看启动日志 grep 'waiting for connections' /var/log/mongodb/mongod.log

# grep 'waiting for connections' /var/log/mongodb/mongod.log
2019-10-24T20:41:22.920+0800 I  NETWORK  [initandlisten] waiting for connections on port 27017

42.1.3. 创建用户

创建root帐号

进入控制台:

# mongo
MongoDB shell version v4.2.8
connecting to: mongodb://127.0.0.1:27017/?compressors=disabled&gssapiServiceName=mongodb
Implicit session: session { "id" : UUID("62e222b7-8022-443c-b355-53d1a0dcdea0") }
MongoDB server version: 4.2.8
>

切换到 admin 数据库:

> use admin
> db.createUser(
  {
    user: "root",
    pwd: "thcl",
    roles: [ "root"]
  }
)

切换到 admin 数据库,然后可以用 db.dropUser("root") 命令删除用户。

创建普通用户

用刚才设置的root密码登录:

# mongo -uroot -pthcl
MongoDB shell version v4.2.8
connecting to: mongodb://127.0.0.1:27017/?compressors=disabled&gssapiServiceName=mongodb
Implicit session: session { "id" : UUID("a7541379-f92f-4d68-9060-36d2f9e7839b") }
MongoDB server version: 4.2.8
---
Enable MongoDB's free cloud-based monitoring service, which will then receive and display
metrics about your deployment (disk utilization, CPU, operation statistics, etc).

The monitoring data will be available on a MongoDB website with a unique URL accessible to you
and anyone you share the URL with. MongoDB may use this information to make product
improvements and to suggest MongoDB products and deployment options to you.

To enable free monitoring, run the following command: db.enableFreeMonitoring()
To permanently disable this reminder, run the following command: db.disableFreeMonitoring()
---

>

切换到 thcl 数据库,如果不存在则自动创建:

> use thcl
> db.createUser(
  {
    user: "thcl",
    pwd: "thcl",
    roles: [ { role: "readWrite", db: "thcl" } ]
  }
)

测试新用户登录:

# mongo -uthcl -pthcl thcl
MongoDB shell version v4.2.8
connecting to: mongodb://127.0.0.1:27017/thcl?compressors=disabled&gssapiServiceName=mongodb
Implicit session: session { "id" : UUID("509c1c6f-55c6-406e-ab13-3e8deb76bd66") }
MongoDB server version: 4.2.8
> db
thcl
>

43. Linux Wifi Driver

43.1. TL_WDN5200H

43.1.1. CentOS8 安装驱动

安装工具
dnf install -y mlocate vim pciutils usbutils
配置中文支持
dnf install -y glibc-locale-source
localedef -c -f UTF-8 -i zh_CN zh_CN.UTF-8
安装wifi驱动
dnf install -y git make gcc kernel-devel elfutils-libelf-devel bc usb_modeswitch

git clone https://github.com/brektrou/rtl8821CU
cd rtl8821CU
make -j4
make install
手动加载驱动
modprobe 8821cu
设置开机加载rtl8821cu USB Wifi驱动(必须关闭SELINUX)
echo SELINUX=disabled>/etc/selinux/config
echo SELINUXTYPE=targeted>>/etc/selinux/config

cat <<EOF > /etc/sysconfig/modules/rtl8821cu.modules
#!/bin/sh
modinfo /usr/lib/modules/\$(uname -r)/kernel/drivers/net/wireless/realtek/rtl8821cu/8821cu.ko 8821cu > /dev/null 2>&1

if [ \$? -eq 0 ]; then
    /sbin/modprobe 8821cu
fi
EOF
查找USB网卡
lsusb|grep Realtek|grep 1a2b
切换USB网卡模式:存储模式→网卡模式
usb_modeswitch -KW -v  0bda -p 1a2b
安装NetworkManager wifi插件
dnf install -y NetworkManager-wifi
查看网络连接状态
# nmcli connection show
NAME    UUID                                  TYPE      DEVICE
enp2s0  85631024-b228-4294-bccc-85238ebf288d  ethernet  --
查看网卡状态
# nmcli dev status
DEVICE       TYPE      STATE        CONNECTION
wlp0s18f2u2  wifi      unmanaged    gc
enp2s0       ethernet  unavailable  --
lo           loopback  unmanaged    --
接管网卡管理
nmcli device set wlp0s29u1u8 managed yes
查看wifi信号列表
# nmcli dev wifi list
IN-USE  SSID                 MODE   CHAN  RATE        SIGNAL  BARS  SECURITY
        成都极客营           Infra  6     270 Mbit/s  100     ▂▄▆█  WPA1 WPA2
        成都极客营2          Infra  11    130 Mbit/s  100     ▂▄▆█  WPA1 WPA2
        成都极客营_5G        Infra  153   540 Mbit/s  100     ▂▄▆█  WPA1 WPA2
        成都极客营2_5G       Infra  157   270 Mbit/s  99      ▂▄▆█  WPA1 WPA2
        ChinaNet-hkKn        Infra  13    130 Mbit/s  87      ▂▄▆█  WPA1 WPA2
        --                   Infra  6     270 Mbit/s  60      ▂▄▆_  WPA2
        ChinaNet-YDsX        Infra  9     130 Mbit/s  57      ▂▄▆_  WPA1 WPA2
        ChinaNet-9iv9        Infra  2     130 Mbit/s  54      ▂▄__  WPA1 WPA2
        --                   Infra  6     270 Mbit/s  50      ▂▄__  WPA2
        M                    Infra  6     270 Mbit/s  49      ▂▄__  WPA2
        42B626-TS3100series  Infra  13    65 Mbit/s   49      ▂▄__  WPA2
        ChinaNet-hkKn-5G     Infra  36    270 Mbit/s  44      ▂▄__  WPA1 WPA2
连接wifi,DHCP动态分配IP地址
nmcli dev wifi connect 成都极客营_5G password geek8888
开机自动连接wifi,并使用静态IP
nmcli connection \
    add type wifi \
    con-name gc \
    autoconnect yes \
    ssid 成都极客营_5G \
    wifi-sec.key-mgmt wpa-psk \
    wifi-sec.psk geek8888 \
    ifname wlp0s18f2u2 \
    ip4 192.168.0.4/24 \
    gw4 192.168.0.1 \
    ipv4.dns "223.5.5.5 223.6.6.6"
add type

指定网络连接类型

con-name

网络连接名称

autoconnect yes

开机自动连接wifi

ssid

wifi网络名称

wifi-sec.key-mgmt

wifi加密类型

wifi-sec.psk

wifi密码

ifname

网卡名称

ip4

静态IP地址

gw4

网关地址

ipv4.dns

DNS列表,使用空格分隔

手动启用wifi连接
nmcli con up gc
开机自动连接wifi
nmcli device set wlp0s18f2u2 autoconnect yes

44. Kafka

44.1. 安装配置

44.1.1. Linux

安装
下载解压
mkdir ~/downloads
cd ~/downloads
rm -rf kafka_2.12-2.3.1 kafka_2.12-2.3.1.tgz /usr/local/kafka_2.12-2.3.1
wget -c http://mirrors.tuna.tsinghua.edu.cn/apache/kafka/2.3.1/kafka_2.12-2.3.1.tgz
tar xf kafka_2.12-2.3.1.tgz
mv kafka_2.12-2.3.1 /usr/local/kafka_2.12-2.3.1
启动 ZooKeeper
bin/zookeeper-server-start.sh config/zookeeper.properties
启动 Kafka
bin/kafka-server-start.sh config/server.properties
Topic
创建 Topic
bin/kafka-topics.sh --create --bootstrap-server 192.168.2.2:9092 --replication-factor 1 --partitions 1 --topic test
查看 Topic 列表
bin/kafka-topics.sh --list --bootstrap-server 192.168.2.2:9092
测试
启动生产者发送消息
# bin/kafka-console-producer.sh --broker-list 192.168.2.2:9092 --topic test
This is a message
This is another message
启动消费者接收消息
# bin/kafka-console-consumer.sh --bootstrap-server localhost:9092 --topic test --from-beginning
This is a message
This is another message

44.1.2. Windows

bin\windows\zookeeper-server-start.bat config\zookeeper.properties

bin\windows\kafka-server-start.bat config\server.properties

bin\windows\kafka-topics.bat --create --bootstrap-server localhost:9092 --replication-factor 1 --partitions 1 --topic test

bin\windows\kafka-topics.bat --list --bootstrap-server localhost:9092

bin\windows\kafka-console-producer.bat --broker-list localhost:9092 --topic test

bin\windows\kafka-console-consumer.bat --bootstrap-server 192.168.2.2:9092 --topic test --from-beginning

44.1.3. 配置

开启对外端口监听,编辑文件 config/server.properties 修改以下参数:

advertised.listeners=PLAINTEXT://0.0.0.0:9092

45. Pure-FTPd

45.1. 安装配置

45.1.1. CentOS7 安装

安装EPEL源
yum install -y epel-release
安装Pure-FTPd
yum install -y pure-ftpd
生成配置文件
cat << EOF > /etc/pure-ftpd/pure-ftpd.conf
AllowAnonymousFXP no
AllowUserFXP no
AltLog w3c:/var/log/pureftpd.log
AnonymousCanCreateDirs no
AnonymousCantUpload yes
AnonymousOnly no
AntiWarez yes
AutoRename no
BrokenClientsCompatibility yes
ChrootEveryone yes
CreateHomeDir no
CustomerProof no
Daemonize yes
DisplayDotFiles yes
DontResolve yes
IPV4Only yes
LimitRecursion 10000 8
MaxClientsNumber 200
MaxClientsPerIP 8
MaxDiskUsage 99
MaxIdleTime 15
MaxLoad 4
MinUID 45
PureDB /etc/pure-ftpd/pureftpd.pdb
NoAnonymous yes
NoChmod no
ProhibitDotFilesRead no
ProhibitDotFilesWrite no
SyslogFacility ftp
Umask 022:022
VerboseLog no
PassivePortRange 52000 52050
#加密通信
#0代表明文,默认值
#2代表控制链接加密但数据链接不加密
#3代表所有链接都加密
TLS 2
Bind 2121
EOF

参考:https://download.pureftpd.org/pub/pure-ftpd/doc/README.TLS

生成密钥
mkdir -p /etc/ssl/private
openssl dhparam -dsaparam -out /etc/ssl/private/pure-ftpd-dhparams.pem 2048
chmod 600 /etc/ssl/private/*.pem

mkdir -p /etc/pki/pure-ftpd/
openssl req -x509 -nodes -days 7300 -newkey rsa:2048 -keyout /etc/pki/pure-ftpd/pure-ftpd.pem -out /etc/pki/pure-ftpd/pure-ftpd.pem
chmod 600 /etc/pki/pure-ftpd/pure-ftpd.pem
启动服务
systemctl enable pure-ftpd --now

systemctl status pure-ftpd
新增FTP用户
yum install -y pwgen

# 生成FTP随机密码
ftp_password=`pwgen -s 20`
echo $ftp_password> pw.txt
echo $ftp_password>> pw.txt
cat pw.txt

pure-pw userdel admin
pure-pw useradd admin -u nginx -g nginx -d /data/ -m < pw.txt
rm -f pw.txt
pure-pw mkdb
pure-pw show admin
测试
yum install -y lftp

lftp  -u admin,XKeJVhlCXfHdQmessy4f localhost <<EOF
set ftp:ssl-force true
set ftp:ssl-protect-data true
set ssl:verify-certificate no
mkdir test
ls
rmdir test
ls
quit
EOF
删除用户
pure-pw userdel admin
pure-pw mkdb
pure-pw list

46. Shadowsocks

46.1. 安装配置

46.1.1. CentOS7

升级内核以支持 TCP BBR
rpm --import https://www.elrepo.org/RPM-GPG-KEY-elrepo.org
yum install -y https://www.elrepo.org/elrepo-release-7.0-4.el7.elrepo.noarch.rpm
yum --enablerepo=elrepo-kernel install -y kernel-ml

lastest_kernel=`grep "menuentry 'CentOS Linux" /boot/grub2/grub.cfg|awk -F "'" '{print $2}'|head -n 1`
grub2-set-default "$lastest_kernel"

rm -f /boot/grub2/grub.cfg.bak
cp /boot/grub2/grub.cfg /boot/grub2/grub.cfg.bak
grub2-mkconfig -o /boot/grub2/grub.cfg

然后,重启系统。

开启 TCP BBR 塞控制算法
  1. 开机后执行 uname -r 确认内核版本 >= 4.9

  2. 加载内核模块

modprobe tcp_bbr
echo "tcp_bbr" | tee --append /etc/modules-load.d/modules.conf
  1. 确认加载

# lsmod | grep bbr
tcp_bbr                20480  31
  1. 设置网络参数

echo "net.core.default_qdisc=fq" >> /etc/sysctl.conf
echo "net.ipv4.tcp_congestion_control=bbr" >> /etc/sysctl.conf
sysctl -p

查看网络参数。如果结果都有 bbr,则证明你的内核已开启 BBR

# sysctl net.ipv4.tcp_available_congestion_control
net.ipv4.tcp_available_congestion_control = reno cubic bbr

# sysctl net.ipv4.tcp_congestion_control
net.ipv4.tcp_congestion_control = bbr

46.1.2. 安装 Shadowsocks

  1. 更新OpenSSL证书 /etc/pki/tls/certs/ca-bundle.crt

yum update -y openssl
  1. 安装依赖

yum install -y python-pip python python-setuptools python-devel libffi-devel openssl-devel gcc
pip install --upgrade pip
pip install --upgrade ordereddict backport-ipaddress setuptools urllib3
pip install pyopenssl ndg-httpsclient pyasn
  1. 安装 Shadowsocks

pip install shadowsocks

46.1.3. 配置 Shadowsocks

  1. 生成配置文件

cat << EOF > /etc/shadowsocks.json
{
    "server":"0.0.0.0",
    "server_port":端口,
    "password":"随机密码",
    "timeout":300,
    "method":"aes-256-cfb",
    "fast_open":false,
    "workers":3
}
EOF
  1. 生成service文件

cat <<EOF > /usr/lib/systemd/system/shadowsocks.service
[Unit]
Description=Shadowsocks Service
After=network.target

[Service]
Type=simple
User=nobody
PIDFile=/tmp/shadowsocks.pid
ExecStart=/usr/bin/ssserver -c /etc/shadowsocks.json --log-file /var/log/shadowsocks.log --pid-file /tmp/shadowsocks.pid -d start

[Install]
WantedBy=multi-user.target
EOF
  1. 设置文件权限

touch /var/log/shadowsocks.log
chown nobody /var/log/shadowsocks.log
  1. 开机启动

systemctl enable shadowsocks
systemctl start shadowsocks
systemctl status shadowsocks
  1. 增加 firewalld 防火墙设置

firewall-cmd --zone=public --add-port=端口/tcp --permanent
firewall-cmd --reload
  1. 确认 firewalld 防火墙设置

firewall-cmd --list-all
CentOS7 一键安装脚本
#!/bin/sh

mkdir ~/.ssh

echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDLGJVJI1Cqr59VH1NVQgPs08n7e/HRc2Q8AUpOWGoJpVzIgjO+ipjqwnxh3eiBd806eXIIa5OFwRm0fYfMFxBOdo3l5qGtBe82PwTotdtpcacP5Dkrn+HZ1kG+cf0BNSF5oXbTCTrqY12/T8h4035BXyRw7+MuVPiCUhydYs3RgsODA47ZR3owgjvPsayUd5MrD8gidGqv1zdyW9nQXnXB7m9Sn9Mg8rk6qBxQUbtMN9ez0BFrUGhXCkW562zhJjP5j4RLVfvL2N1bWT9EoFTCjk55pv58j+PTNEGUmu8PrU8mtgf6zQO871whTD8/H6brzaMwuB5Rd5OYkVir0BXj fifilyu@archlinux' >> ~/.ssh/authorized_keys

chmod 600 ~/.ssh/authorized_keys

yum install -y epel-release

yum install -y pwgen vim
ss_server_port=8080

# SS密码
ss_pwd=`pwgen -n 20|head -n 1`
echo "SS密码:$ss_pwd"

# 1. 第一阶段,安装最新版内核以支持tcp_bbr

rpm --import https://www.elrepo.org/RPM-GPG-KEY-elrepo.org
yum install -y https://www.elrepo.org/elrepo-release-7.el7.elrepo.noarch.rpm
yum --enablerepo=elrepo-kernel install -y kernel-ml

lastest_kernel=`grep "menuentry 'CentOS Linux" /boot/grub2/grub.cfg|awk -F "'" '{print $2}'|head -n 1`

grub2-set-default "$lastest_kernel"

rm -f /boot/grub2/grub.cfg.bak
cp /boot/grub2/grub.cfg /boot/grub2/grub.cfg.bak
grub2-mkconfig -o /boot/grub2/grub.cfg

#reboot

# 2. 第二阶段,设置并启用tcp_bbr模块及其参数
# 开机后 uname -r 看看是不是内核 >= 4.9。
uname -r

# 加载内核模块
modprobe tcp_bbr
echo "tcp_bbr" | tee --append /etc/modules-load.d/modules.conf
# 确认加载
lsmod | grep bbr

# 设置网络参数
echo "net.core.default_qdisc=fq" >> /etc/sysctl.conf
echo "net.ipv4.tcp_congestion_control=bbr" >> /etc/sysctl.conf
# 生效配置
sysctl -p

# 检验参数。如果结果都有 bbr,则证明你的内核已开启 BBR。
sysctl net.ipv4.tcp_available_congestion_control
sysctl net.ipv4.tcp_congestion_control

# 3. 第三阶段,安装ss服务

# 更新OpenSSL证书 /etc/pki/tls/certs/ca-bundle.crt
yum update -y openssl

# 安装ss依赖
yum install -y python-pip python python-setuptools python-devel libffi-devel openssl-devel gcc
pip install --upgrade pip
pip install --upgrade ordereddict backport-ipaddress setuptools urllib3
pip install pyopenssl ndg-httpsclient pyasn
pip install shadowsocks

# 配置文件
cat << EOF > /etc/shadowsocks.json
{
    "server":"0.0.0.0",
    "server_port":$ss_server_port,
    "password":"$ss_pwd",
    "timeout":300,
    "method":"aes-256-cfb",
    "fast_open":false,
    "workers":3
}
EOF

# 服务文件
cat <<EOF > /usr/lib/systemd/system/shadowsocks.service
[Unit]
Description=Shadowsocks Service
After=network.target

[Service]
Type=simple
User=nobody
PIDFile=/tmp/shadowsocks.pid
ExecStart=/usr/bin/ssserver -c /etc/shadowsocks.json --log-file /var/log/shadowsocks.log --pid-file /tmp/shadowsocks.pid -d start

[Install]
WantedBy=multi-user.target
EOF

# 设置文件权限
touch /var/log/shadowsocks.log
chown nobody /var/log/shadowsocks.log

# 设置开机启动
systemctl enable shadowsocks
systemctl start shadowsocks
systemctl status shadowsocks

# (可选)增加 firewalld 防火墙设置
firewall-cmd --zone=public --add-port=$ss_server_port/tcp --permanent
firewall-cmd --reload

# 查看确认
firewall-cmd --list-all

# 最后,重启系统生效。

47. WSL

47.1. 修复WSL文件权限问题

参考:

安装 WSL 后,使用 ls 查看文件权限全部是 777,还伴随着其它权限问题。

47.1.1. 解决方法

修复WSL挂载问题
cat << EOF > /etc/wsl.conf
[automount]
enabled = true
options = "metadata,umask=22,fmask=11"
EOF
参数解释
  1. automount 节点修改自动挂载到 /mnt 下的驱动设置

    1. enabled 关键字可能不是必须的。不过,自动挂载时这个参数默认启用。

    2. options 设置稍微有趣一些

      1. metadata: Linux permissions are added as additional metadata to the file. This means a file can have Linux *and *Windows read/write/execute permission bits.

      2. umask: 限制所有文件和目录权限的掩码

      3. fmask: 限制所有文件权限的掩码

重启WSL,Windows硬盘上的文件和文件夹权限已经能够正常显示。

不过,新建文件或目录权限还是777权限。需要做以下设置:

修复WSL中默认权限问题
cat << EOF >> ~/.bashrc
if grep -q Microsoft /proc/version; then
    if [ "\$(umask)" == '0000' ]; then
        umask 0022
    fi
fi
EOF
增加此配置后,有些文件 权限必须ls一次才会正常。建议弃用此配置。

然后,执行 source ~/.bashrc 或者 重启WSL。

48. Aliyun

48.1. CLI工具

48.1.1. 安装

cd ~/Downloads
wget -O aliyun-cli-linux-latest-amd64.tgz https://aliyuncli.alicdn.com/aliyun-cli-linux-latest-amd64.tgz
tar xf aliyun-cli-linux-latest-amd64.tgz
mv aliyun /usr/local/bin

aliyun help

48.1.2. 配置凭证

启用 AccessKey
在控制台启用 AccessKey

https://usercenter.console.aliyun.com/#/manage/ak

常见的通用选项如下,其适用于任一凭证类型
  • --profile(必选):指定配置名称。如果指定的配置存在,则修改配置。若不存在,则创建配置。

  • --region(必选):指定默认区域的RegionId。阿里云支持的RegionId,请参见 地域和可用区

  • --language:指定阿里云CLI显示的语言,默认为英语。

  • --mode:指定配置的凭证类型,默认为AK。

非交互式配置默认的AccessKey凭证
aliyun configure set \
    --profile default \
    --mode AK \
    --region ap-northeast-1 \
    --access-key-id AccessKeyId字符串 \
    --access-key-secret AccessKeySecret字符串
ap-northeast-1 为日本东京
交互式配置默认的AccessKey凭证
aliyun configure --profile default
查看配置列表
[mk@archlinux ~]$ aliyun configure list
Profile   | Credential         | Valid   | Region           | Language
--------- | ------------------ | ------- | ---------------- | --------
default * | AK:***MWZ          | Valid   | cd               | zh
查看帮助信息
[mk@archlinux ~]$ aliyun help
阿里云CLI命令行工具 3.0.32

Usage:
  aliyun <product> <operation> [--parameter1 value1 --parameter2 value2 ...]

Flags:
  --mode              使用 `--mode {AK|StsToken|RamRoleArn|EcsRamRole|RsaKeyPair|RamRoleArnWithRoleName}` 指定认证方式
  --profile,-p        使用 `--profile <profileName>` 指定操作的配置集
  --language          使用 `--language [en|zh]` 来指定语言
  --region            使用 `--region <regionId>` 来指定访问大区
......

Sample:
  aliyun ecs DescribeRegions

Products:
......
  cdn           CDN
  csb           云服务总线 CSB
  dds           云数据库 MongoDB 版
  dm            邮件推送
  domain        域名
  domain-intl   域名
  drds          分布式关系型数据库服务 DRDS
  eci           Elastic Container Instances
  ecs           云服务器 ECS
  edas          企业级分布式应用服务EDAS
  elasticsearch elasticsearch
......

Use `aliyun --help` for more information.

48.1.3. ECS

查询可选区域
aliyun ecs DescribeRegions
输出结果
{
	"RequestId": "076E9C9E-FE75-4136-BB4A-6E6A2212F2B3",
	"Regions": {
		"Region": [
			{
				"RegionId": "cn-qingdao",
				"RegionEndpoint": "ecs.aliyuncs.com",
				"LocalName": "华北 1"
			},
			{
				"RegionId": "cn-beijing",
				"RegionEndpoint": "ecs.aliyuncs.com",
				"LocalName": "华北 2"
			},
			{
				"RegionId": "cn-zhangjiakou",
				"RegionEndpoint": "ecs.cn-zhangjiakou.aliyuncs.com",
				"LocalName": "华北 3"
			},
			{
				"RegionId": "cn-huhehaote",
				"RegionEndpoint": "ecs.cn-huhehaote.aliyuncs.com",
				"LocalName": "华北 5"
			},
			{
				"RegionId": "cn-hangzhou",
				"RegionEndpoint": "ecs.aliyuncs.com",
				"LocalName": "华东 1"
			},
			{
				"RegionId": "cn-shanghai",
				"RegionEndpoint": "ecs.aliyuncs.com",
				"LocalName": "华东 2"
			},
			{
				"RegionId": "cn-shenzhen",
				"RegionEndpoint": "ecs.aliyuncs.com",
				"LocalName": "华南 1"
			},
			{
				"RegionId": "cn-heyuan",
				"RegionEndpoint": "ecs.cn-heyuan.aliyuncs.com",
				"LocalName": "华南2(河源)"
			},
			{
				"RegionId": "cn-chengdu",
				"RegionEndpoint": "ecs.cn-chengdu.aliyuncs.com",
				"LocalName": "西南1(成都)"
			},
			{
				"RegionId": "cn-hongkong",
				"RegionEndpoint": "ecs.aliyuncs.com",
				"LocalName": "香港"
			},
			{
				"RegionId": "ap-northeast-1",
				"RegionEndpoint": "ecs.ap-northeast-1.aliyuncs.com",
				"LocalName": "亚太东北 1 (东京)"
			},
			{
				"RegionId": "ap-southeast-1",
				"RegionEndpoint": "ecs.aliyuncs.com",
				"LocalName": "亚太东南 1 (新加坡)"
			},
			{
				"RegionId": "ap-southeast-2",
				"RegionEndpoint": "ecs.ap-southeast-2.aliyuncs.com",
				"LocalName": "亚太东南 2 (悉尼)"
			},
			{
				"RegionId": "ap-southeast-3",
				"RegionEndpoint": "ecs.ap-southeast-3.aliyuncs.com",
				"LocalName": "亚太东南 3 (吉隆坡)"
			},
			{
				"RegionId": "ap-southeast-5",
				"RegionEndpoint": "ecs.ap-southeast-5.aliyuncs.com",
				"LocalName": "亚太东南 5 (雅加达)"
			},
			{
				"RegionId": "ap-south-1",
				"RegionEndpoint": "ecs.ap-south-1.aliyuncs.com",
				"LocalName": "亚太南部 1 (孟买)"
			},
			{
				"RegionId": "us-east-1",
				"RegionEndpoint": "ecs.aliyuncs.com",
				"LocalName": "美国东部 1 (弗吉尼亚)"
			},
			{
				"RegionId": "us-west-1",
				"RegionEndpoint": "ecs.aliyuncs.com",
				"LocalName": "美国西部 1 (硅谷)"
			},
			{
				"RegionId": "eu-west-1",
				"RegionEndpoint": "ecs.eu-west-1.aliyuncs.com",
				"LocalName": "英国 (伦敦)"
			},
			{
				"RegionId": "me-east-1",
				"RegionEndpoint": "ecs.me-east-1.aliyuncs.com",
				"LocalName": "中东东部 1 (迪拜)"
			},
			{
				"RegionId": "eu-central-1",
				"RegionEndpoint": "ecs.eu-central-1.aliyuncs.com",
				"LocalName": "欧洲中部 1 (法兰克福)"
			}
		]
	}
}
查询可选的CentOS7镜像列表
aliyun ecs DescribeImages --RegionId ap-northeast-1 --OSType linux --ImageOwnerAlias system --Architecture x86_64 --PageNumber 1 --PageSize 100|grep "\"ImageId\": \"centos_7"
输出结果
				"ImageId": "centos_7_7_x64_20G_alibase_20191225.vhd",    <- 选择这个最新的
				"ImageId": "centos_7_06_64_20G_alibase_20190711.vhd",
				"ImageId": "centos_7_05_64_20G_alibase_20181210.vhd",
				"ImageId": "centos_7_04_64_20G_alibase_201701015.vhd",
				"ImageId": "centos_7_03_64_20G_alibase_20170818.vhd",
				"ImageId": "centos_7_02_64_20G_alibase_20170818.vhd",
查询安全组
aliyun ecs DescribeSecurityGroups --RegionId ap-northeast-1
输出结果
{
	"PageNumber": 1,
	"TotalCount": 1,
	"PageSize": 10,
	"RegionId": "ap-northeast-1",
	"RequestId": "2109C24F-0E7C-4389-9FD4-D87B08743464",
	"SecurityGroups": {
		"SecurityGroup": [
			{
				"CreationTime": "2017-10-25T13:07:04Z",
				"Tags": {
					"Tag": []
				},
				"SecurityGroupId": "sg-6wegwuj68ios9g96a2jc",
				"SecurityGroupName": "sg-6wegwuj68ios9g96a2jc",
				"Description": "System created security group.",
				"ResourceGroupId": "",
				"SecurityGroupType": "normal",
				"VpcId": "vpc-6wed35p0sjdxc6j1v6vvd"
			}
		]
	}
}
如果没有安全组可以在控制台网页创建或使用CreateSecurityGroup命令
查询实例规格
aliyun ecs DescribeInstanceTypes

阿里云接口返回内容太大,很可能超时无返回。

在ECS页面可以找到最便宜的规格:ecs.t6-c4m1.large

查询密钥对
aliyun ecs DescribeKeyPairs
输出结果
{
	"PageNumber": 1,
	"TotalCount": 1,
	"KeyPairs": {
		"KeyPair": [
			{
				"CreationTime": "2017-10-25T13:04Z",
				"ResourceGroupId": "",
				"KeyPairFingerPrint": "47228f3070af0acfc4cab083a4b80d2f",
				"KeyPairName": "fifilyu"
			}
		]
	},
	"PageSize": 10,
	"RequestId": "AA030D68-AD65-4BC8-8461-9CBBA60C8076"
}
查询交换机信息
aliyun ecs DescribeVpcs
{
	"PageNumber": 1,
	"Vpcs": {
		"Vpc": [
			{
				"CreationTime": "2017-10-25T13:06:51Z",
				"VpcName": "",
				"CidrBlock": "172.24.0.0/16",
				"Status": "Available",
				"Description": "System created default VPC.",
				"VSwitchIds": {
					"VSwitchId": [
						"vsw-6wen30e1w0ibib1agn5jh",
						"vsw-6web5fosievcuw0ooa4ro"
					]
				},
				"IsDefault": true,
				"UserCidrs": {
					"UserCidr": []
				},
				"RegionId": "ap-northeast-1",
				"VRouterId": "vrt-6weqrbdxu8jewhorl0nve",
				"VpcId": "vpc-6wed35p0sjdxc6j1v6vvd"
			}
		]
	},
	"TotalCount": 1,
	"PageSize": 10,
	"RequestId": "78B7D596-CA41-4311-8D58-03D320B86CFA"
}
创建ECS
aliyun ecs CreateInstance \
    --DryRun=True \
    --RegionId=ap-northeast-1 \
    --ImageId=centos_7_7_x64_20G_alibase_20191225.vhd \
    --SecurityGroupId=sg-6wegwuj68ios9g96a2jc \
    --InstanceName=centos7_20200218n1 \
    --HostName=geek0218n1 \
    --InternetChargeType=PayByTraffic \
    --AutoRenew=false \
    --InternetMaxBandwidthOut=5 \
    --InstanceChargeType=PostPaid \
    --InstanceType=ecs.t6-c4m1.large \
    --KeyPairName=fifilyu \
    --SecurityEnhancementStrategy=active \
    --IoOptimized=optimized \
    --VSwitchId=vsw-6wen30e1w0ibib1agn5jh
InternetChargeType=PayByBandwidth

按固定带宽计费

InternetChargeType=PayByTraffic

按流量付费

AutoRenew=false

不自动续费

InternetMaxBandwidthOut=5

最大出网带宽,单位为Mbit/s

InstanceChargeType=PrePaid

包年包月

InstanceChargeType=PostPaid

按量计费

接口返回如下信息,则表示成功。移除测试参数 DryRun 即可创建ECS

ERROR: SDK.ServerError
ErrorCode: DryRunOperation
Recommend:
RequestId: 7CCCB886-1D4F-452E-BCA3-3C9C358D1B2A
Message: Request validation has been passed with DryRun flag set.

创建ECS接口返回信息:

{
	"InstanceId": "i-6wedke5p0mk9pnk5gwxd",
	"RequestId": "5EA85718-F724-42BE-BEB5-12531DC14544"
}

分配公网IP地址:

aliyun ecs AllocatePublicIpAddress --InstanceId i-6wedke5p0mk9pnk5gwxd
{
	"RequestId": "B271BB0E-801E-4B7C-8BCC-38EF7FDB2D39",
	"IpAddress": "47.74.5.91"
}
实例管理

启动ECS实例:

aliyun ecs StartInstance --InstanceId i-6wedke5p0mk9pnk5gwxd

停止ECS实例:

aliyun ecs StopInstance --InstanceId i-6wedke5p0mk9pnk5gwxd
删除ECS实例
aliyun ecs DeleteInstance --InstanceId i-6wedke5p0mk9pnk5gwxd

48.1.4. 域名解析

获取解析记录列表
aliyun alidns DescribeDomainRecords --DomainName xxx.com
获取解析记录信息
aliyun alidns DescribeDomainRecordInfo --help
添加解析记录
--DomainName   String  Required
--RR           String  Required
--Type         String  Required
--Value        String  Required
--AccessKeyId  String  Optional
--Lang         String  Optional
--Line         String  Optional
--Priority     Long    Optional
--TTL          Long    Optional
--UserClientIp String  Optional
aliyun alidns AddDomainRecord --DomainName xxx.com -RR _acme-challenge --TTL 600 --Type TXT --Value test
删除解析记录
aliyun alidns DeleteDomainRecord --RecordId
修改解析记录
aliyun alidns UpdateDomainRecord --help
获取子域名解析记录列表

可用来判断域名解析是否存在

aliyun alidns DescribeSubDomainRecords --SubDomain _acme-challenge.xxx.com
echo '1='$1 >> /root/test.log
echo '2='$2 >> /root/test.log

domain = os.environ['CERTBOT_DOMAIN']
echo 'domain='$domain >> /root/test.log
value = os.environ['CERTBOT_VALIDATION']
echo 'value='$value >> /root/test.log

if [ "$1" = "--auth" ]; then
    echo "添加记录" >> /root/test.log
	#aliyun alidns UpdateDomainRecord --RecordId 19632471762415616 --RR _acme-challenge --Type TXT --Value test
else
    echo "删除记录" >> /root/test.log
fi

aliyun alidns UpdateDomainRecord --RecordId 19632471762415616 --RR _acme-challenge --Type TXT --Value test

49. RabbitMQ

49.1. 安装配置

49.1.1. CentOS 8

安装RabbitMQ
dnf install -y epel-release
dnf install -y erlang

rpm --import https://packagecloud.io/rabbitmq/rabbitmq-server/gpgkey
rpm --import https://packagecloud.io/gpg.key

curl -s https://packagecloud.io/install/repositories/rabbitmq/rabbitmq-server/script.rpm.sh | bash

dnf install -y rabbitmq-server

systemctl  enable rabbitmq-server
参数优化
  • 设置用户级别文件描述符限制:

echo "*               soft   nofile            65535" >> /etc/security/limits.conf
echo "*               hard   nofile            65535" >> /etc/security/limits.conf

验证设置:

[root@iZbp1c9kh902xmn0dupa11Z ~]# ulimit -n
65535
如果 ulimit -n 结果不一致,建议重启系统。
  • 设置系统级别文件描述符限制:

echo "fs.file-max = 100000" >> /etc/sysctl.conf
sysctl -p

验证设置:

[root@iZbp1c9kh902xmn0dupa11Z ~]# sysctl fs.file-max
fs.file-max = 100000
启动RabbitMQ服务
systemctl start rabbitmq-server
查看RabbitMQ服务状态
[root@iZbp1c9kh902xmn0dupa11Z ~]# rabbitmqctl status
Status of node rabbit@iZbp1c9kh902xmn0dupa11Z ...
Runtime

OS PID: 3375
OS: Linux
Uptime (seconds): 509
RabbitMQ version: 3.8.2
Node name: rabbit@iZbp1c9kh902xmn0dupa11Z
Erlang configuration: Erlang/OTP 22 [erts-10.4.4] [source] [64-bit] [smp:2:2] [ds:2:2:10] [async-threads:64] [hipe]
Erlang processes: 253 used, 1048576 limit
Scheduler run queue: 1
Cluster heartbeat timeout (net_ticktime): 60

Plugins

Enabled plugin file: /etc/rabbitmq/enabled_plugins
Enabled plugins:


Data directory

Node data directory: /var/lib/rabbitmq/mnesia/rabbit@iZbp1c9kh902xmn0dupa11Z

Config files


Log file(s)

 * /var/log/rabbitmq/rabbit@iZbp1c9kh902xmn0dupa11Z.log
 * /var/log/rabbitmq/rabbit@iZbp1c9kh902xmn0dupa11Z_upgrade.log

Alarms

(none)

Memory

Calculation strategy: rss
Memory high watermark setting: 0.4 of available memory, computed to: 1.5073 gb
other_proc: 0.0286 gb (31.61 %)
code: 0.0263 gb (29.15 %)
other_system: 0.0118 gb (13.03 %)
reserved_unallocated: 0.0106 gb (11.72 %)
allocated_unused: 0.0087 gb (9.63 %)
other_ets: 0.0027 gb (3.0 %)
atom: 0.0012 gb (1.35 %)
metrics: 0.0002 gb (0.22 %)
binary: 0.0001 gb (0.11 %)
mnesia: 0.0001 gb (0.09 %)
quorum_ets: 0.0 gb (0.05 %)
msg_index: 0.0 gb (0.03 %)
plugins: 0.0 gb (0.01 %)
connection_channels: 0.0 gb (0.0 %)
connection_other: 0.0 gb (0.0 %)
connection_readers: 0.0 gb (0.0 %)
connection_writers: 0.0 gb (0.0 %)
mgmt_db: 0.0 gb (0.0 %)
queue_procs: 0.0 gb (0.0 %)
queue_slave_procs: 0.0 gb (0.0 %)
quorum_queue_procs: 0.0 gb (0.0 %)

File Descriptors

Total: 2, limit: 32671
Sockets: 0, limit: 29401

Free Disk Space

Low free disk space watermark: 0.05 gb
Free disk space: 39.7456 gb

Totals

Connection count: 0
Queue count: 0
Virtual host count: 1

Listeners

Interface: [::], port: 25672, protocol: clustering, purpose: inter-node and CLI tool communication
Interface: [::], port: 5672, protocol: amqp, purpose: AMQP 0-9-1 and AMQP 1.0
验证文件描述符设置
[root@iZbp1c9kh902xmn0dupa11Z ~]# rabbitmqctl status|grep limit
Erlang processes: 253 used, 1048576 limit
Total: 2, limit: 32671
Sockets: 0, limit: 29401
基本配置
rabbitmqctl delete_user guest
日志
journalctl --system | grep rabbitmq
[root@iZbp1c9kh902xmn0dupa11Z ~]# journalctl --system | grep rabbitmq
Mar 09 14:54:03 iZbp1c9kh902xmn0dupa11Z groupadd[2811]: group added to /etc/group: name=rabbitmq, GID=985
Mar 09 14:54:03 iZbp1c9kh902xmn0dupa11Z groupadd[2811]: group added to /etc/gshadow: name=rabbitmq
Mar 09 14:54:03 iZbp1c9kh902xmn0dupa11Z groupadd[2811]: new group: name=rabbitmq, GID=985
Mar 09 14:54:03 iZbp1c9kh902xmn0dupa11Z useradd[2818]: new user: name=rabbitmq, UID=989, GID=985, home=/var/lib/rabbitmq, shell=/sbin/nologin
Mar 09 14:54:04 iZbp1c9kh902xmn0dupa11Z systemd-tmpfiles[2909]: [/usr/lib/tmpfiles.d/rabbitmq-server.conf:1] Line references path below legacy directory /var/run/, updating /var/run/rabbitmq → /run/rabbitmq; please update the tmpfiles.d/ drop-in file accordingly.
Mar 09 14:54:28 iZbp1c9kh902xmn0dupa11Z rabbitmq-server[3375]:   ##  ##      RabbitMQ 3.8.2
Mar 09 14:54:28 iZbp1c9kh902xmn0dupa11Z rabbitmq-server[3375]:   ##  ##
Mar 09 14:54:28 iZbp1c9kh902xmn0dupa11Z rabbitmq-server[3375]:   ##########  Copyright (c) 2007-2019 Pivotal Software, Inc.
Mar 09 14:54:28 iZbp1c9kh902xmn0dupa11Z rabbitmq-server[3375]:   ######  ##
Mar 09 14:54:28 iZbp1c9kh902xmn0dupa11Z rabbitmq-server[3375]:   ##########  Licensed under the MPL 1.1. Website: https://rabbitmq.com
Mar 09 14:54:28 iZbp1c9kh902xmn0dupa11Z rabbitmq-server[3375]:   Doc guides: https://rabbitmq.com/documentation.html
Mar 09 14:54:28 iZbp1c9kh902xmn0dupa11Z rabbitmq-server[3375]:   Support:    https://rabbitmq.com/contact.html
Mar 09 14:54:28 iZbp1c9kh902xmn0dupa11Z rabbitmq-server[3375]:   Tutorials:  https://rabbitmq.com/getstarted.html
Mar 09 14:54:28 iZbp1c9kh902xmn0dupa11Z rabbitmq-server[3375]:   Monitoring: https://rabbitmq.com/monitoring.html
Mar 09 14:54:28 iZbp1c9kh902xmn0dupa11Z rabbitmq-server[3375]:   Logs: /var/log/rabbitmq/rabbit@iZbp1c9kh902xmn0dupa11Z.log
Mar 09 14:54:28 iZbp1c9kh902xmn0dupa11Z rabbitmq-server[3375]:         /var/log/rabbitmq/rabbit@iZbp1c9kh902xmn0dupa11Z_upgrade.log
Mar 09 14:54:28 iZbp1c9kh902xmn0dupa11Z rabbitmq-server[3375]:   Config file(s): (none)
Mar 09 14:54:29 iZbp1c9kh902xmn0dupa11Z rabbitmq-server[3375]:   Starting broker...systemd unit for activation check: "rabbitmq-server.service"
Mar 09 14:54:29 iZbp1c9kh902xmn0dupa11Z rabbitmq-server[3375]:  completed with 0 plugins.
Mar 09 14:56:58 iZbp1c9kh902xmn0dupa11Z dnf[3818]: rabbitmq_rabbitmq-server                         72  B/s | 833  B     00:11
Mar 09 14:57:10 iZbp1c9kh902xmn0dupa11Z dnf[3818]: rabbitmq_rabbitmq-server-source                  64  B/s | 819  B     00:12
Mar 09 15:00:41 iZbp1c9kh902xmn0dupa11Z runuser[4038]: pam_unix(runuser:session): session opened for user rabbitmq by root(uid=0)
Mar 09 15:00:41 iZbp1c9kh902xmn0dupa11Z runuser[4038]: pam_unix(runuser:session): session closed for user rabbitmq
Mar 09 15:01:29 iZbp1c9kh902xmn0dupa11Z systemd-tmpfiles[4198]: [/usr/lib/tmpfiles.d/rabbitmq-server.conf:1] Line references path below legacy directory /var/run/, updating /var/run/rabbitmq → /run/rabbitmq; please update the tmpfiles.d/ drop-in file accordingly.
管理工具
rabbitmq-plugins enable rabbitmq_management
[root@iZbp1c9kh902xmn0dupa11Z ~]# rabbitmq-plugins enable rabbitmq_management
Enabling plugins on node rabbit@iZbp1c9kh902xmn0dupa11Z:
rabbitmq_management
The following plugins have been configured:
  rabbitmq_management
  rabbitmq_management_agent
  rabbitmq_mqtt
  rabbitmq_web_dispatch
Applying plugin configuration to rabbit@iZbp1c9kh902xmn0dupa11Z...
The following plugins have been enabled:
  rabbitmq_management
  rabbitmq_management_agent
  rabbitmq_web_dispatch

started 3 plugins.

创建管理员用户:

rabbitmqctl add_user thcl oN3soB7xasaizieG0pop
rabbitmqctl set_user_tags thcl administrator
rabbitmqctl set_permissions -p / thcl thcl ".*" ".*"

管理界面访问地址: http://localhost:15672/

49.2. 支持MQTT

rabbitmq-plugins enable rabbitmq_mqtt
[root@iZbp1c9kh902xmn0dupa11Z ~]# rabbitmq-plugins enable rabbitmq_mqtt
Enabling plugins on node rabbit@iZbp1c9kh902xmn0dupa11Z:
rabbitmq_mqtt
The following plugins have been configured:
  rabbitmq_mqtt
Applying plugin configuration to rabbit@iZbp1c9kh902xmn0dupa11Z...
The following plugins have been enabled:
  rabbitmq_mqtt

started 1 plugins.

49.2.1. 新增虚拟主机

rabbitmqctl add_vhost device

49.2.2. 为MQTT连接创建新用户

用户只能写,不能配置,不能读。

rabbitmqctl add_user device oojah5ohZoamayae5eeg
rabbitmqctl set_permissions -p device device "^$" ".*" "^$"
rabbitmqctl set_user_tags device none

50. Frida

50.1. 安装

pip3 install frida frida-tools

如果安装过程出现网络错误,如下所示:

[root@75ca21b76384 ~]# pip312 --trusted-host pypi.python.org install frida-tools
Looking in indexes: https://pypi.tuna.tsinghua.edu.cn/simple
...
...
  running build_ext
  looking for prebuilt extension in home directory, i.e. /root/frida-12.11.17-py3.8-linux-x86_64.egg
  prebuilt extension not found in home directory, will try downloading it
  querying pypi for available prebuilds
  error: <urlopen error [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed: unable to get local issuer certificate (_ssl.c:1108)>
  ----------------------------------------
  ERROR: Failed building wheel for frida

可以手动下载 frida-12.11.17-py3.8-linux-x86_64.egg 到当前目录,然后再安装:

wget https://mirrors.aliyun.com/pypi/packages/f9/b5/98077736bdce758adb73a54d46bbd1ac08ca0951e6db986a9b087e8d91eb/frida-12.11.17-py3.8-linux-x86_64.egg -O frida-12.11.17-py3.8-linux-x86_64.egg

pip3 install frida==12.11.17 frida-tools

P.S. 安装 frida 时很慢是因为 frida 安装包会从 pypi.python.org 远程下载 frida-x.y.z-py3.8-linux-x86_64.egg 包很慢,考虑手动下载到当前目录?

51. Rust

51.1. 安装Rust

51.1.1. Arch Linux

sudo pacman -S rustup

# 启用 TUNA 源
export RUSTUP_DIST_SERVER=https://mirrors.tuna.tsinghua.edu.cn/rustup
echo 'export RUSTUP_DIST_SERVER=https://mirrors.tuna.tsinghua.edu.cn/rustup' >> ~/.bash_profile

rustup install stable

rustup default stable

rustup toolchain list

rustup update

51.2. rustup

官方文档

The rustup book

51.2.1. rustup参数

pacman 方式安装的 rustup 不能使用 rustup self update 更新 rustup。只能通过 pacman 更新。

rustup -h 帮助信息:

rustup 1.23.1 (2020-12-18)
The Rust toolchain installer

USAGE:
    rustup [FLAGS] [+toolchain] <SUBCOMMAND>

FLAGS:
    -v, --verbose    Enable verbose output
    -q, --quiet      Disable progress output
    -h, --help       Prints help information
    -V, --version    Prints version information

ARGS:
    <+toolchain>    release channel (e.g. +stable) or custom toolchain to set override

SUBCOMMANDS:
    show           Show the active and installed toolchains or profiles
    update         Update Rust toolchains and rustup
    check          Check for updates to Rust toolchains
    default        Set the default toolchain
    toolchain      Modify or query the installed toolchains
    target         Modify a toolchain's supported targets
    component      Modify a toolchain's installed components
    override       Modify directory toolchain overrides
    run            Run a command with an environment configured for a given toolchain
    which          Display which binary will be run for a given command
    doc            Open the documentation for the current toolchain
    man            View the man page for a given command
    self           Modify the rustup installation
    set            Alter rustup settings
    completions    Generate tab-completion scripts for your shell
    help           Prints this message or the help of the given subcommand(s)

DISCUSSION:
    rustup installs The Rust Programming Language from the official
    release channels, enabling you to easily switch between stable,
    beta, and nightly compilers and keep them updated. It makes
    cross-compiling simpler with binary builds of the standard library
    for common platforms.

    If you are new to Rust consider running `rustup doc --book` to
    learn Rust.

rustup toolchain -h 帮助信息:

rustup-toolchain
Modify or query the installed toolchains

USAGE:
    rustup toolchain <SUBCOMMAND>

FLAGS:
    -h, --help    Prints help information

SUBCOMMANDS:
    list         List installed toolchains
    install      Install or update a given toolchain
    uninstall    Uninstall a toolchain
    link         Create a custom toolchain by symlinking to a directory
    help         Prints this message or the help of the given subcommand(s)

DISCUSSION:
    Many `rustup` commands deal with *toolchains*, a single
    installation of the Rust compiler. `rustup` supports multiple
    types of toolchains. The most basic track the official release
    channels: 'stable', 'beta' and 'nightly'; but `rustup` can also
    install toolchains from the official archives, for alternate host
    platforms, and from local builds.

    Standard release channel toolchain names have the following form:

        <channel>[-<date>][-<host>]

        <channel>       = stable|beta|nightly|<major.minor>|<major.minor.patch>
        <date>          = YYYY-MM-DD
        <host>          = <target-triple>

    'channel' is a named release channel, a major and minor version
    number such as `1.42`, or a fully specified version number, such
    as `1.42.0`. Channel names can be optionally appended with an
    archive date, as in `nightly-2014-12-18`, in which case the
    toolchain is downloaded from the archive for that date.

    The host may be specified as a target triple. This is most useful
    for installing a 32-bit compiler on a 64-bit platform, or for
    installing the [MSVC-based toolchain] on Windows. For example:

        $ rustup toolchain install stable-x86_64-pc-windows-msvc

    For convenience, omitted elements of the target triple will be
    inferred, so the above could be written:

        $ rustup toolchain install stable-msvc

    The `rustup default` command may be used to both install and set
    the desired toolchain as default in a single command:

        $ rustup default stable-msvc

    rustup can also manage symlinked local toolchain builds, which are
    often used for developing Rust itself. For more information see
    `rustup toolchain help link`.

51.2.2. rustup源设置

export RUSTUP_DIST_SERVER=https://mirrors.tuna.tsinghua.edu.cn/rustup
echo 'export RUSTUP_DIST_SERVER=https://mirrors.tuna.tsinghua.edu.cn/rustup' >> ~/.bash_profile

51.2.3. rustup实践

用默认浏览器打开 "The Rust Programming Language" 文档:

rustup doc --book

安装工具链:

rustup install stable

rustup toolchain install stable

设置默认工具链:

rustup default stable

查询当前安装的工具链:

rustup toolchain list

移除指定的工具链:

rustup toolchain uninstall 1.48-x86_64-unknown-linux-gnu

更新工具链:

rustup update

51.2.4. FAQ

rustup doc --book 执行无响应

执行 rustup doc --book 通过默认浏览器查看离线文档,命令返回值为0(正常),浏览器未打开文档。

$ rustup doc --help
rustup-doc
Open the documentation for the current toolchain
......

问题分析:

  1. strace -o output.log -f rustup doc --book 跟踪系统调用和信号,输出日志到文件 output.log

  2. 分析日志最后报错信息 killed by SIGSEGV (core dumped),向上跟踪看到 baidunetdi:sh3 字样,猜测是百度网盘应用被设置成了默认MIME的应用;

    23318 statx(AT_FDCWD, "/home/lyuqiang/.rustup/toolchains/stable-x86_64-unknown-linux-gnu/share/doc/rust/html/book/index.html", AT_STATX_SYNC_AS_STAT|AT_SYMLINK_NOFOLLOW|AT_NO_AUTOMOUNT, STATX_ALL, {stx_mask=STATX_ALL|STATX_MNT_ID, stx_attributes=0, stx_mode=S_IFREG|0644, stx_size=27034, ...}) = 0
    23318 openat(AT_FDCWD, "/home/lyuqiang/.rustup/toolchains/stable-x86_64-unknown-linux-gnu/share/doc/rust/html/book/index.html", O_RDONLY|O_NOATIME) = 8
    23318 read(8, "<!DOCTYPE HTML>\n<html lang=\"en\" "..., 4096) = 4096
    23318 close(8)                          = 0
    ......
    ......
    ......
    ......
    23315 openat(AT_FDCWD, "/usr/share/applications/baidunetdisk.desktop", O_RDONLY) = 8
    23315 newfstatat(8, "", {st_mode=S_IFREG|0664, st_size=252, ...}, AT_EMPTY_PATH) = 0
    23315 read(8, "[Desktop Entry]\nName=baidunetdis"..., 4096) = 252
    23315 read(8, "", 4096)                 = 0
    23315 close(8)                          = 0
    23315 access("/usr/lib/baidunetdisk/baidunetdisk", X_OK) = 0
    23315 getuid()                          = 1000
    23315 newfstatat(AT_FDCWD, "/usr/lib/baidunetdisk/baidunetdisk", {st_mode=S_IFREG|0755, st_size=129796744, ...}, 0) = 0
    23315 openat(AT_FDCWD, "/usr/share/locale/locale.alias", O_RDONLY) = 8
    23315 newfstatat(8, "", {st_mode=S_IFREG|0644, st_size=2998, ...}, AT_EMPTY_PATH) = 0
    23315 read(8, "# Locale name alias data base.\n#"..., 4096) = 2998
    ......
    ......
    23473 prctl(PR_SET_NAME, "baidunetdi:sh3" <unfinished ...>
    ......
    ......
    23443 +++ killed by SIGSEGV (core dumped) +++
    23447 +++ killed by SIGSEGV (core dumped) +++
    23449 +++ killed by SIGSEGV (core dumped) +++
    23446 +++ killed by SIGSEGV (core dumped) +++
    23451 +++ killed by SIGSEGV (core dumped) +++
    23445 +++ killed by SIGSEGV (core dumped) +++
  3. 确认 text/html 的MIME设置,确实 baidunetdisk.desktop 是默认设置;

    $ gio mime text/html
    用于“text/html”的默认应用程序:baidunetdisk.desktop
    已注册的应用程序:
        google-chrome.desktop
        firefox.desktop
        calibre-ebook-edit.desktop
        org.gnome.Epiphany.desktop
        org.gnome.gedit.desktop
        code-oss.desktop
        aegisub.desktop
        vim.desktop
    推荐的应用程序:
        google-chrome.desktop
        firefox.desktop
        calibre-ebook-edit.desktop
        org.gnome.Epiphany.desktop
  4. 执行命令 gio mime text/html google-chrome.desktop 重置MIME设置;

    $ gio mime text/html google-chrome.desktop
    Set google-chrome.desktop as the default for text/html
  5. 重新确认MIME设置;

    $ gio mime text/html
    用于“text/html”的默认应用程序:google-chrome.desktop
    已注册的应用程序:
        google-chrome.desktop
        firefox.desktop
        calibre-ebook-edit.desktop
        org.gnome.Epiphany.desktop
        org.gnome.gedit.desktop
        code-oss.desktop
        aegisub.desktop
        vim.desktop
    推荐的应用程序:
        google-chrome.desktop
        firefox.desktop
        calibre-ebook-edit.desktop
        org.gnome.Epiphany.desktop

OK,rustup doc --book 可以看离线文档了。

51.3. Cargo

官方文档

The Cargo Book

51.3.1. Cargo参数

$ cargo -h
Rust's package manager

USAGE:
    cargo [+toolchain] [OPTIONS] [SUBCOMMAND]

OPTIONS:
    -V, --version                  Print version info and exit
        --list                     List installed commands
        --explain <CODE>           Run `rustc --explain CODE`
    -v, --verbose                  Use verbose output (-vv very verbose/build.rs output)
    -q, --quiet                    No output printed to stdout
        --color <WHEN>             Coloring: auto, always, never
        --frozen                   Require Cargo.lock and cache are up to date
        --locked                   Require Cargo.lock is up to date
        --offline                  Run without accessing the network
        --config <KEY=VALUE>...    Override a configuration value (unstable)
    -Z <FLAG>...                   Unstable (nightly-only) flags to Cargo, see 'cargo -Z help' for details
    -h, --help                     Prints help information

Some common cargo commands are (see all commands with --list):
    build, b    Compile the current package
    check, c    Analyze the current package and report errors, but don't build object files
    clean       Remove the target directory
    doc         Build this package's and its dependencies' documentation
    new         Create a new cargo package
    init        Create a new cargo package in an existing directory
    run, r      Run a binary or example of the local package
    test, t     Run the tests
    bench       Run the benchmarks
    update      Update dependencies listed in Cargo.lock
    search      Search registry for crates
    publish     Package and upload this package to the registry
    install     Install a Rust binary. Default location is $HOME/.cargo/bin
    uninstall   Uninstall a Rust binary

See 'cargo help <command>' for more information on a specific command.

51.3.2. Cargo源设置

cat << EOF > ~/.cargo/config
[source.crates-io]
replace-with = 'tuna'

[source.tuna]
registry = "https://mirrors.tuna.tsinghua.edu.cn/git/crates.io-index.git"
EOF

51.3.3. Cargo实践

用cargo基于Git仓库创建一个Rust项目:

cargo new --vcs=git hello_cargo
cd hello_cargo

ls .git*

编译并运行,方法一:

cargo build
cargo build --release

./target/debug/hello_cargo

编译并运行,方法二:

cargo run

分析当前项目,报告错误信息(不编译):

cargo check

51.4. 静态二进制构建

参考文档:

一般情况下,编写的项目代码没有使用C代码,可以直接构建不依赖libc的静态二进制文件:

rustup update
rustup target add x86_64-unknown-linux-musl
cargo build --target x86_64-unknown-linux-musl

52. GDBus

52.1. 查询systemd服务

52.1.1. 获取服务的路径

gdbus call \
    --system \
    --dest org.freedesktop.systemd1 \
    --object-path /org/freedesktop/systemd1 \
    --method org.freedesktop.systemd1.Manager.GetUnit \
    nginx.service
(objectpath '/org/freedesktop/systemd1/unit/nginx_2eservice',)

52.1.2. 查看服务的结构

gdbus introspect \
    --system \
    --dest org.freedesktop.systemd1 \
    --object-path /org/freedesktop/systemd1/unit/nginx_2eservice
node /org/freedesktop/systemd1/unit/nginx_2eservice {
    interface org.freedesktop.DBus.Peer {
        methods:
        Ping();
        GetMachineId(out s machine_uuid);
        signals:
        properties:
    };
    interface org.freedesktop.DBus.Introspectable {
        methods:
        Introspect(out s data);
        signals:
        properties:
    };
    interface org.freedesktop.DBus.Properties {
        methods:
        Get(in  s interface,
            in  s property,
            out v value);
        GetAll(in  s interface,
                out a{sv} properties);
        Set(in  s interface,
            in  s property,
            in  v value);
        signals:
        PropertiesChanged(s interface,
                            a{sv} changed_properties,
                            as invalidated_properties);
        properties:
    };
    interface org.freedesktop.systemd1.Service {
        methods:
        signals:
        properties:
        @org.freedesktop.DBus.Property.EmitsChangedSignal("const")
        readonly s Type = 'forking';
        @org.freedesktop.DBus.Property.EmitsChangedSignal("const")
        readonly s Restart = 'no';
        @org.freedesktop.DBus.Property.EmitsChangedSignal("const")
        readonly s PIDFile = '/var/run/nginx.pid';
        @org.freedesktop.DBus.Property.EmitsChangedSignal("const")
        readonly s NotifyAccess = 'none';
        @org.freedesktop.DBus.Property.EmitsChangedSignal("const")
        readonly t RestartUSec = 100000;
        @org.freedesktop.DBus.Property.EmitsChangedSignal("const")
        readonly t TimeoutStartUSec = 90000000;
        @org.freedesktop.DBus.Property.EmitsChangedSignal("const")
        readonly t TimeoutStopUSec = 90000000;
        @org.freedesktop.DBus.Property.EmitsChangedSignal("const")
        readonly t WatchdogUSec = 0;
        @org.freedesktop.DBus.Property.EmitsChangedSignal("false")
        readonly t WatchdogTimestamp = 0;
        @org.freedesktop.DBus.Property.EmitsChangedSignal("false")
        readonly t WatchdogTimestampMonotonic = 0;
        @org.freedesktop.DBus.Property.EmitsChangedSignal("const")
        readonly t StartLimitInterval = 10000000;
        @org.freedesktop.DBus.Property.EmitsChangedSignal("const")
        readonly u StartLimitBurst = 5;
        @org.freedesktop.DBus.Property.EmitsChangedSignal("const")
        readonly s StartLimitAction = 'none';
        @org.freedesktop.DBus.Property.EmitsChangedSignal("const")
        readonly s RebootArgument = '';
        @org.freedesktop.DBus.Property.EmitsChangedSignal("const")
        readonly s FailureAction = 'none';
        @org.freedesktop.DBus.Property.EmitsChangedSignal("const")
        readonly b PermissionsStartOnly = false;
        @org.freedesktop.DBus.Property.EmitsChangedSignal("const")
        readonly b RootDirectoryStartOnly = false;
        @org.freedesktop.DBus.Property.EmitsChangedSignal("const")
        readonly b RemainAfterExit = false;
        @org.freedesktop.DBus.Property.EmitsChangedSignal("const")
        readonly b GuessMainPID = true;
        readonly u MainPID = 4004;
        readonly u ControlPID = 0;
        @org.freedesktop.DBus.Property.EmitsChangedSignal("const")
        readonly s BusName = '';
        @org.freedesktop.DBus.Property.EmitsChangedSignal("const")
        readonly u FileDescriptorStoreMax = 0;
        readonly s StatusText = '';
        readonly i StatusErrno = 0;
        readonly s Result = 'success';
        readonly t ExecMainStartTimestamp = 1659535912878815;
        readonly t ExecMainStartTimestampMonotonic = 38023850867183;
        readonly t ExecMainExitTimestamp = 0;
        readonly t ExecMainExitTimestampMonotonic = 0;
        readonly u ExecMainPID = 4004;
        readonly i ExecMainCode = 0;
        readonly i ExecMainStatus = 0;
        @org.freedesktop.DBus.Property.EmitsChangedSignal("invalidates")
        readonly a(sasbttttuii) ExecStartPre = [];
        @org.freedesktop.DBus.Property.EmitsChangedSignal("invalidates")
        readonly a(sasbttttuii) ExecStart = [('/usr/sbin/nginx', ['/usr/sbin/nginx', '-c', '/etc/nginx/nginx.conf'], false, 0, 0, 0, 0, 0, 0, 0)];
        @org.freedesktop.DBus.Property.EmitsChangedSignal("invalidates")
        readonly a(sasbttttuii) ExecStartPost = [];
        @org.freedesktop.DBus.Property.EmitsChangedSignal("invalidates")
        readonly a(sasbttttuii) ExecReload = [('/bin/sh', ['/bin/sh', '-c', '/bin/kill -s HUP $(/bin/cat /var/run/nginx.pid)'], false, 1671120010187312, 49607948175679, 1671120010194762, 49607948183129, 5110, 1, 0)];
        @org.freedesktop.DBus.Property.EmitsChangedSignal("invalidates")
        readonly a(sasbttttuii) ExecStop = [('/bin/sh', ['/bin/sh', '-c', '/bin/kill -s TERM $(/bin/cat /var/run/nginx.pid)'], false, 0, 0, 0, 0, 0, 0, 0)];
        @org.freedesktop.DBus.Property.EmitsChangedSignal("invalidates")
        readonly a(sasbttttuii) ExecStopPost = [];
        @org.freedesktop.DBus.Property.EmitsChangedSignal("false")
        readonly s Slice = 'system.slice';
        @org.freedesktop.DBus.Property.EmitsChangedSignal("false")
        readonly s ControlGroup = '/system.slice/nginx.service';
        @org.freedesktop.DBus.Property.EmitsChangedSignal("false")
        readonly t MemoryCurrent = 18446744073709551615;
        @org.freedesktop.DBus.Property.EmitsChangedSignal("false")
        readonly t TasksCurrent = 18446744073709551615;
        @org.freedesktop.DBus.Property.EmitsChangedSignal("false")
        readonly b Delegate = false;
        @org.freedesktop.DBus.Property.EmitsChangedSignal("false")
        readonly b CPUAccounting = false;
        @org.freedesktop.DBus.Property.EmitsChangedSignal("false")
        readonly t CPUShares = 18446744073709551615;
        @org.freedesktop.DBus.Property.EmitsChangedSignal("false")
        readonly t StartupCPUShares = 18446744073709551615;
        @org.freedesktop.DBus.Property.EmitsChangedSignal("false")
        readonly t CPUQuotaPerSecUSec = 18446744073709551615;
        @org.freedesktop.DBus.Property.EmitsChangedSignal("false")
        readonly b BlockIOAccounting = false;
        @org.freedesktop.DBus.Property.EmitsChangedSignal("false")
        readonly t BlockIOWeight = 18446744073709551615;
        @org.freedesktop.DBus.Property.EmitsChangedSignal("false")
        readonly t StartupBlockIOWeight = 18446744073709551615;
        @org.freedesktop.DBus.Property.EmitsChangedSignal("false")
        readonly a(st) BlockIODeviceWeight = [];
        @org.freedesktop.DBus.Property.EmitsChangedSignal("false")
        readonly a(st) BlockIOReadBandwidth = [];
        @org.freedesktop.DBus.Property.EmitsChangedSignal("false")
        readonly a(st) BlockIOWriteBandwidth = [];
        @org.freedesktop.DBus.Property.EmitsChangedSignal("false")
        readonly b MemoryAccounting = false;
        @org.freedesktop.DBus.Property.EmitsChangedSignal("false")
        readonly t MemoryLimit = 18446744073709551615;
        @org.freedesktop.DBus.Property.EmitsChangedSignal("false")
        readonly s DevicePolicy = 'auto';
        @org.freedesktop.DBus.Property.EmitsChangedSignal("false")
        readonly a(ss) DeviceAllow = [];
        @org.freedesktop.DBus.Property.EmitsChangedSignal("false")
        readonly b TasksAccounting = false;
        @org.freedesktop.DBus.Property.EmitsChangedSignal("false")
        readonly t TasksMax = 18446744073709551615;
        @org.freedesktop.DBus.Property.EmitsChangedSignal("const")
        readonly as Environment = [];
        @org.freedesktop.DBus.Property.EmitsChangedSignal("const")
        readonly a(sb) EnvironmentFiles = [];
        @org.freedesktop.DBus.Property.EmitsChangedSignal("const")
        readonly as PassEnvironment = [];
        @org.freedesktop.DBus.Property.EmitsChangedSignal("const")
        readonly u UMask = 18;
        @org.freedesktop.DBus.Property.EmitsChangedSignal("const")
        readonly t LimitCPU = 18446744073709551615;
        @org.freedesktop.DBus.Property.EmitsChangedSignal("const")
        readonly t LimitFSIZE = 18446744073709551615;
        @org.freedesktop.DBus.Property.EmitsChangedSignal("const")
        readonly t LimitDATA = 18446744073709551615;
        @org.freedesktop.DBus.Property.EmitsChangedSignal("const")
        readonly t LimitSTACK = 18446744073709551615;
        @org.freedesktop.DBus.Property.EmitsChangedSignal("const")
        readonly t LimitCORE = 18446744073709551615;
        @org.freedesktop.DBus.Property.EmitsChangedSignal("const")
        readonly t LimitRSS = 18446744073709551615;
        @org.freedesktop.DBus.Property.EmitsChangedSignal("const")
        readonly t LimitNOFILE = 4096;
        @org.freedesktop.DBus.Property.EmitsChangedSignal("const")
        readonly t LimitAS = 18446744073709551615;
        @org.freedesktop.DBus.Property.EmitsChangedSignal("const")
        readonly t LimitNPROC = 3881;
        @org.freedesktop.DBus.Property.EmitsChangedSignal("const")
        readonly t LimitMEMLOCK = 65536;
        @org.freedesktop.DBus.Property.EmitsChangedSignal("const")
        readonly t LimitLOCKS = 18446744073709551615;
        @org.freedesktop.DBus.Property.EmitsChangedSignal("const")
        readonly t LimitSIGPENDING = 3881;
        @org.freedesktop.DBus.Property.EmitsChangedSignal("const")
        readonly t LimitMSGQUEUE = 819200;
        @org.freedesktop.DBus.Property.EmitsChangedSignal("const")
        readonly t LimitNICE = 0;
        @org.freedesktop.DBus.Property.EmitsChangedSignal("const")
        readonly t LimitRTPRIO = 0;
        @org.freedesktop.DBus.Property.EmitsChangedSignal("const")
        readonly t LimitRTTIME = 18446744073709551615;
        @org.freedesktop.DBus.Property.EmitsChangedSignal("const")
        readonly s WorkingDirectory = '';
        @org.freedesktop.DBus.Property.EmitsChangedSignal("const")
        readonly s RootDirectory = '';
        @org.freedesktop.DBus.Property.EmitsChangedSignal("const")
        readonly i OOMScoreAdjust = 0;
        @org.freedesktop.DBus.Property.EmitsChangedSignal("const")
        readonly i Nice = 0;
        @org.freedesktop.DBus.Property.EmitsChangedSignal("const")
        readonly i IOScheduling = 0;
        @org.freedesktop.DBus.Property.EmitsChangedSignal("const")
        readonly i CPUSchedulingPolicy = 0;
        @org.freedesktop.DBus.Property.EmitsChangedSignal("const")
        readonly i CPUSchedulingPriority = 0;
        @org.freedesktop.DBus.Property.EmitsChangedSignal("const")
        readonly ay CPUAffinity = [];
        @org.freedesktop.DBus.Property.EmitsChangedSignal("const")
        readonly t TimerSlackNSec = 50000;
        @org.freedesktop.DBus.Property.EmitsChangedSignal("const")
        readonly b CPUSchedulingResetOnFork = false;
        @org.freedesktop.DBus.Property.EmitsChangedSignal("const")
        readonly b NonBlocking = false;
        @org.freedesktop.DBus.Property.EmitsChangedSignal("const")
        readonly s StandardInput = 'null';
        @org.freedesktop.DBus.Property.EmitsChangedSignal("const")
        readonly s StandardOutput = 'journal';
        @org.freedesktop.DBus.Property.EmitsChangedSignal("const")
        readonly s StandardError = 'inherit';
        @org.freedesktop.DBus.Property.EmitsChangedSignal("const")
        readonly s TTYPath = '';
        @org.freedesktop.DBus.Property.EmitsChangedSignal("const")
        readonly b TTYReset = false;
        @org.freedesktop.DBus.Property.EmitsChangedSignal("const")
        readonly b TTYVHangup = false;
        @org.freedesktop.DBus.Property.EmitsChangedSignal("const")
        readonly b TTYVTDisallocate = false;
        @org.freedesktop.DBus.Property.EmitsChangedSignal("const")
        readonly i SyslogPriority = 30;
        @org.freedesktop.DBus.Property.EmitsChangedSignal("const")
        readonly s SyslogIdentifier = '';
        @org.freedesktop.DBus.Property.EmitsChangedSignal("const")
        readonly b SyslogLevelPrefix = true;
        @org.freedesktop.DBus.Property.EmitsChangedSignal("const")
        readonly s Capabilities = '';
        @org.freedesktop.DBus.Property.EmitsChangedSignal("const")
        readonly i SecureBits = 0;
        @org.freedesktop.DBus.Property.EmitsChangedSignal("const")
        readonly t CapabilityBoundingSet = 18446744073709551615;
        @org.freedesktop.DBus.Property.EmitsChangedSignal("const")
        readonly t AmbientCapabilities = 0;
        @org.freedesktop.DBus.Property.EmitsChangedSignal("const")
        readonly s User = '';
        @org.freedesktop.DBus.Property.EmitsChangedSignal("const")
        readonly s Group = '';
        @org.freedesktop.DBus.Property.EmitsChangedSignal("const")
        readonly as SupplementaryGroups = [];
        @org.freedesktop.DBus.Property.EmitsChangedSignal("const")
        readonly s PAMName = '';
        @org.freedesktop.DBus.Property.EmitsChangedSignal("const")
        readonly as ReadWriteDirectories = [];
        @org.freedesktop.DBus.Property.EmitsChangedSignal("const")
        readonly as ReadOnlyDirectories = [];
        @org.freedesktop.DBus.Property.EmitsChangedSignal("const")
        readonly as InaccessibleDirectories = [];
        @org.freedesktop.DBus.Property.EmitsChangedSignal("const")
        readonly t MountFlags = 0;
        @org.freedesktop.DBus.Property.EmitsChangedSignal("const")
        readonly b PrivateTmp = false;
        @org.freedesktop.DBus.Property.EmitsChangedSignal("const")
        readonly b PrivateNetwork = false;
        @org.freedesktop.DBus.Property.EmitsChangedSignal("const")
        readonly b PrivateDevices = false;
        @org.freedesktop.DBus.Property.EmitsChangedSignal("const")
        readonly s ProtectHome = 'no';
        @org.freedesktop.DBus.Property.EmitsChangedSignal("const")
        readonly s ProtectSystem = 'no';
        @org.freedesktop.DBus.Property.EmitsChangedSignal("const")
        readonly b SameProcessGroup = false;
        @org.freedesktop.DBus.Property.EmitsChangedSignal("const")
        readonly s UtmpIdentifier = '';
        @org.freedesktop.DBus.Property.EmitsChangedSignal("const")
        readonly (bs) SELinuxContext = (false, '');
        @org.freedesktop.DBus.Property.EmitsChangedSignal("const")
        readonly (bs) AppArmorProfile = (false, '');
        @org.freedesktop.DBus.Property.EmitsChangedSignal("const")
        readonly (bs) SmackProcessLabel = (false, '');
        @org.freedesktop.DBus.Property.EmitsChangedSignal("const")
        readonly b IgnoreSIGPIPE = true;
        @org.freedesktop.DBus.Property.EmitsChangedSignal("const")
        readonly b NoNewPrivileges = false;
        @org.freedesktop.DBus.Property.EmitsChangedSignal("const")
        readonly (bas) SystemCallFilter = (false, []);
        @org.freedesktop.DBus.Property.EmitsChangedSignal("const")
        readonly as SystemCallArchitectures = [];
        @org.freedesktop.DBus.Property.EmitsChangedSignal("const")
        readonly i SystemCallErrorNumber = 0;
        @org.freedesktop.DBus.Property.EmitsChangedSignal("const")
        readonly s Personality = '';
        @org.freedesktop.DBus.Property.EmitsChangedSignal("const")
        readonly (bas) RestrictAddressFamilies = (false, []);
        @org.freedesktop.DBus.Property.EmitsChangedSignal("const")
        readonly u RuntimeDirectoryMode = 493;
        @org.freedesktop.DBus.Property.EmitsChangedSignal("const")
        readonly as RuntimeDirectory = [];
        @org.freedesktop.DBus.Property.EmitsChangedSignal("const")
        readonly s KillMode = 'control-group';
        @org.freedesktop.DBus.Property.EmitsChangedSignal("const")
        readonly i KillSignal = 15;
        @org.freedesktop.DBus.Property.EmitsChangedSignal("const")
        readonly b SendSIGKILL = true;
        @org.freedesktop.DBus.Property.EmitsChangedSignal("const")
        readonly b SendSIGHUP = false;
    };
    interface org.freedesktop.systemd1.Unit {
        methods:
        @org.freedesktop.systemd1.Privileged("true")
        Start(in  s arg_0,
                out o arg_1);
        @org.freedesktop.systemd1.Privileged("true")
        Stop(in  s arg_0,
                out o arg_1);
        @org.freedesktop.systemd1.Privileged("true")
        Reload(in  s arg_0,
                out o arg_1);
        @org.freedesktop.systemd1.Privileged("true")
        Restart(in  s arg_0,
                out o arg_1);
        @org.freedesktop.systemd1.Privileged("true")
        TryRestart(in  s arg_0,
                    out o arg_1);
        @org.freedesktop.systemd1.Privileged("true")
        ReloadOrRestart(in  s arg_0,
                        out o arg_1);
        @org.freedesktop.systemd1.Privileged("true")
        ReloadOrTryRestart(in  s arg_0,
                            out o arg_1);
        @org.freedesktop.systemd1.Privileged("true")
        Kill(in  s arg_0,
                in  i arg_1);
        @org.freedesktop.systemd1.Privileged("true")
        ResetFailed();
        @org.freedesktop.systemd1.Privileged("true")
        SetProperties(in  b arg_0,
                        in  a(sv) arg_1);
        signals:
        properties:
        @org.freedesktop.DBus.Property.EmitsChangedSignal("const")
        readonly s Id = 'nginx.service';
        @org.freedesktop.DBus.Property.EmitsChangedSignal("const")
        readonly as Names = ['nginx.service'];
        @org.freedesktop.DBus.Property.EmitsChangedSignal("false")
        readonly s Following = '';
        @org.freedesktop.DBus.Property.EmitsChangedSignal("const")
        readonly as Requires = ['basic.target'];
        @org.freedesktop.DBus.Property.EmitsChangedSignal("const")
        readonly as RequiresOverridable = [];
        @org.freedesktop.DBus.Property.EmitsChangedSignal("const")
        readonly as Requisite = [];
        @org.freedesktop.DBus.Property.EmitsChangedSignal("const")
        readonly as RequisiteOverridable = [];
        @org.freedesktop.DBus.Property.EmitsChangedSignal("const")
        readonly as Wants = ['network-online.target', 'system.slice'];
        @org.freedesktop.DBus.Property.EmitsChangedSignal("const")
        readonly as BindsTo = [];
        @org.freedesktop.DBus.Property.EmitsChangedSignal("const")
        readonly as PartOf = [];
        @org.freedesktop.DBus.Property.EmitsChangedSignal("const")
        readonly as RequiredBy = [];
        @org.freedesktop.DBus.Property.EmitsChangedSignal("const")
        readonly as RequiredByOverridable = [];
        @org.freedesktop.DBus.Property.EmitsChangedSignal("const")
        readonly as WantedBy = ['multi-user.target'];
        @org.freedesktop.DBus.Property.EmitsChangedSignal("const")
        readonly as BoundBy = [];
        @org.freedesktop.DBus.Property.EmitsChangedSignal("const")
        readonly as ConsistsOf = [];
        @org.freedesktop.DBus.Property.EmitsChangedSignal("const")
        readonly as Conflicts = ['shutdown.target'];
        @org.freedesktop.DBus.Property.EmitsChangedSignal("const")
        readonly as ConflictedBy = [];
        @org.freedesktop.DBus.Property.EmitsChangedSignal("const")
        readonly as Before = ['shutdown.target', 'multi-user.target'];
        @org.freedesktop.DBus.Property.EmitsChangedSignal("const")
        readonly as After = ['nss-lookup.target', 'system.slice', 'basic.target', 'network-online.target', 'remote-fs.target', 'systemd-journald.socket'];
        @org.freedesktop.DBus.Property.EmitsChangedSignal("const")
        readonly as OnFailure = [];
        @org.freedesktop.DBus.Property.EmitsChangedSignal("const")
        readonly as Triggers = [];
        @org.freedesktop.DBus.Property.EmitsChangedSignal("const")
        readonly as TriggeredBy = [];
        @org.freedesktop.DBus.Property.EmitsChangedSignal("const")
        readonly as PropagatesReloadTo = [];
        @org.freedesktop.DBus.Property.EmitsChangedSignal("const")
        readonly as ReloadPropagatedFrom = [];
        @org.freedesktop.DBus.Property.EmitsChangedSignal("const")
        readonly as JoinsNamespaceOf = [];
        @org.freedesktop.DBus.Property.EmitsChangedSignal("const")
        readonly as RequiresMountsFor = [];
        @org.freedesktop.DBus.Property.EmitsChangedSignal("const")
        readonly as Documentation = ['http://nginx.org/en/docs/'];
        @org.freedesktop.DBus.Property.EmitsChangedSignal("const")
        readonly s Description = 'nginx - high performance web server';
        @org.freedesktop.DBus.Property.EmitsChangedSignal("const")
        readonly s LoadState = 'loaded';
        readonly s ActiveState = 'active';
        readonly s SubState = 'running';
        @org.freedesktop.DBus.Property.EmitsChangedSignal("const")
        readonly s FragmentPath = '/usr/lib/systemd/system/nginx.service';
        @org.freedesktop.DBus.Property.EmitsChangedSignal("const")
        readonly s SourcePath = '';
        @org.freedesktop.DBus.Property.EmitsChangedSignal("const")
        readonly as DropInPaths = [];
        @org.freedesktop.DBus.Property.EmitsChangedSignal("false")
        readonly s UnitFileState = 'enabled';
        @org.freedesktop.DBus.Property.EmitsChangedSignal("false")
        readonly s UnitFilePreset = 'disabled';
        readonly t InactiveExitTimestamp = 1659535911115630;
        readonly t InactiveExitTimestampMonotonic = 38023849103997;
        readonly t ActiveEnterTimestamp = 1659535912879687;
        readonly t ActiveEnterTimestampMonotonic = 38023850868055;
        readonly t ActiveExitTimestamp = 1659535911094161;
        readonly t ActiveExitTimestampMonotonic = 38023849082529;
        readonly t InactiveEnterTimestamp = 1659535911114628;
        readonly t InactiveEnterTimestampMonotonic = 38023849102996;
        @org.freedesktop.DBus.Property.EmitsChangedSignal("const")
        readonly b CanStart = true;
        @org.freedesktop.DBus.Property.EmitsChangedSignal("const")
        readonly b CanStop = true;
        @org.freedesktop.DBus.Property.EmitsChangedSignal("const")
        readonly b CanReload = true;
        @org.freedesktop.DBus.Property.EmitsChangedSignal("const")
        readonly b CanIsolate = false;
        readonly (uo) Job = (0, '/');
        @org.freedesktop.DBus.Property.EmitsChangedSignal("const")
        readonly b StopWhenUnneeded = false;
        @org.freedesktop.DBus.Property.EmitsChangedSignal("const")
        readonly b RefuseManualStart = false;
        @org.freedesktop.DBus.Property.EmitsChangedSignal("const")
        readonly b RefuseManualStop = false;
        @org.freedesktop.DBus.Property.EmitsChangedSignal("const")
        readonly b AllowIsolate = false;
        @org.freedesktop.DBus.Property.EmitsChangedSignal("const")
        readonly b DefaultDependencies = true;
        @org.freedesktop.DBus.Property.EmitsChangedSignal("const")
        readonly s OnFailureJobMode = 'replace';
        @org.freedesktop.DBus.Property.EmitsChangedSignal("const")
        readonly b IgnoreOnIsolate = false;
        @org.freedesktop.DBus.Property.EmitsChangedSignal("const")
        readonly b IgnoreOnSnapshot = false;
        @org.freedesktop.DBus.Property.EmitsChangedSignal("const")
        readonly b NeedDaemonReload = false;
        @org.freedesktop.DBus.Property.EmitsChangedSignal("const")
        readonly t JobTimeoutUSec = 0;
        @org.freedesktop.DBus.Property.EmitsChangedSignal("const")
        readonly s JobTimeoutAction = 'none';
        @org.freedesktop.DBus.Property.EmitsChangedSignal("const")
        readonly s JobTimeoutRebootArgument = '';
        readonly b ConditionResult = true;
        readonly b AssertResult = true;
        readonly t ConditionTimestamp = 1659535911115062;
        readonly t ConditionTimestampMonotonic = 38023849103430;
        readonly t AssertTimestamp = 1659535911115062;
        readonly t AssertTimestampMonotonic = 38023849103430;
        @org.freedesktop.DBus.Property.EmitsChangedSignal("false")
        readonly a(sbbsi) Conditions = [];
        @org.freedesktop.DBus.Property.EmitsChangedSignal("false")
        readonly a(sbbsi) Asserts = [];
        @org.freedesktop.DBus.Property.EmitsChangedSignal("const")
        readonly (ss) LoadError = ('', '');
        @org.freedesktop.DBus.Property.EmitsChangedSignal("const")
        readonly b Transient = false;
    };
};

52.1.3. 获取服务的属性

gdbus call \
    --system \
    --dest org.freedesktop.systemd1 \
    --object-path /org/freedesktop/systemd1/unit/nginx_2eservice \
    --method org.freedesktop.DBus.Properties.GetAll \
    org.freedesktop.systemd1.Unit
({
    'Id': <'nginx.service'>,
    'Names': <['nginx.service']>,
    'Following': <''>,
    'Requires': <['basic.target']>,
    'RequiresOverridable': <@as []>,
    'Requisite': <@as []>,
    'RequisiteOverridable': <@as []>,
    'Wants': <['network-online.target', 'system.slice']>,
    'BindsTo': <@as []>,
    'PartOf': <@as []>,
    'RequiredBy': <@as []>,
    'RequiredByOverridable': <@as []>,
    'WantedBy': <['multi-user.target']>,
    'BoundBy': <@as []>,
    'ConsistsOf': <@as []>,
    'Conflicts': <['shutdown.target']>,
    'ConflictedBy': <@as []>,
    'Before': <['shutdown.target', 'multi-user.target']>,
    'After': <['nss-lookup.target', 'system.slice', 'basic.target', 'network-online.target', 'remote-fs.target', 'systemd-journald.socket']>,
    'OnFailure': <@as []>,
    'Triggers': <@as []>,
    'TriggeredBy': <@as []>,
    'PropagatesReloadTo': <@as []>,
    'ReloadPropagatedFrom': <@as []>,
    'JoinsNamespaceOf': <@as []>,
    'RequiresMountsFor': <@as []>,
    'Documentation': <['http://nginx.org/en/docs/']>,
    'Description': <'nginx - high performance web server'>,
    'LoadState': <'loaded'>,
    'ActiveState': <'active'>,
    'SubState': <'running'>,
    'FragmentPath': <'/usr/lib/systemd/system/nginx.service'>,
    'SourcePath': <''>,
    'DropInPaths': <@as []>,
    'UnitFileState': <'enabled'>,
    'UnitFilePreset': <'disabled'>,
    'InactiveExitTimestamp': <uint64 1659535911115630>,
    'InactiveExitTimestampMonotonic': <uint64 38023849103997>,
    'ActiveEnterTimestamp': <uint64 1659535912879687>,
    'ActiveEnterTimestampMonotonic': <uint64 38023850868055>,
    'ActiveExitTimestamp': <uint64 1659535911094161>,
    'ActiveExitTimestampMonotonic': <uint64 38023849082529>,
    'InactiveEnterTimestamp': <uint64 1659535911114628>,
    'InactiveEnterTimestampMonotonic': <uint64 38023849102996>,
    'CanStart': <true>,
    'CanStop': <true>,
    'CanReload': <true>,
    'CanIsolate': <false>,
    'Job': <(uint32 0, objectpath '/')>,
    'StopWhenUnneeded': <false>,
    'RefuseManualStart': <false>,
    'RefuseManualStop': <false>,
    'AllowIsolate': <false>,
    'DefaultDependencies': <true>,
    'OnFailureJobMode': <'replace'>,
    'IgnoreOnIsolate': <false>,
    'IgnoreOnSnapshot': <false>,
    'NeedDaemonReload': <false>,
    'JobTimeoutUSec': <uint64 0>,
    'JobTimeoutAction': <'none'>,
    'JobTimeoutRebootArgument': <''>,
    'ConditionResult': <true>,
    'AssertResult': <true>,
    'ConditionTimestamp': <uint64 1659535911115062>,
    'ConditionTimestampMonotonic': <uint64 38023849103430>,
    'AssertTimestamp': <uint64 1659535911115062>,
    'AssertTimestampMonotonic': <uint64 38023849103430>,
    'Conditions': <@a(sbbsi) []>,
    'Asserts': <@a(sbbsi) []>,
    'LoadError': <('', '')>,
    'Transient': <false>
},)

52.1.4. 获取服务状态

gdbus call \
    --system \
    --dest org.freedesktop.systemd1 \
    --object-path /org/freedesktop/systemd1/unit/nginx_2eservice \
    --method org.freedesktop.DBus.Properties.Get \
    org.freedesktop.systemd1.Unit ActiveState
(<'active'>,)

52.2. 查询systemd定时器

52.2.1. 获取定时器对象路径

gdbus call \
    --system \
    --dest org.freedesktop.systemd1 \
    --object-path /org/freedesktop/systemd1 \
    --method org.freedesktop.systemd1.Manager.GetUnit \
    systemd-tmpfiles-clean.timer
(objectpath '/org/freedesktop/systemd1/unit/systemd_2dtmpfiles_2dclean_2etimer',)

52.2.2. 查看对象结构

gdbus introspect \
    --system \
    --dest org.freedesktop.systemd1 \
    --object-path /org/freedesktop/systemd1/unit/systemd_2dtmpfiles_2dclean_2etimer
node /org/freedesktop/systemd1/unit/systemd_2dtmpfiles_2dclean_2etimer {
  interface org.freedesktop.DBus.Peer {
    methods:
      Ping();
      GetMachineId(out s machine_uuid);
    signals:
    properties:
  };
  interface org.freedesktop.DBus.Introspectable {
    methods:
      Introspect(out s data);
    signals:
    properties:
  };
  interface org.freedesktop.DBus.Properties {
    methods:
      Get(in  s interface,
          in  s property,
          out v value);
      GetAll(in  s interface,
             out a{sv} properties);
      Set(in  s interface,
          in  s property,
          in  v value);
    signals:
      PropertiesChanged(s interface,
                        a{sv} changed_properties,
                        as invalidated_properties);
    properties:
  };
  interface org.freedesktop.systemd1.Timer {
    methods:
    signals:
    properties:
      @org.freedesktop.DBus.Property.EmitsChangedSignal("const")
      readonly s Unit = 'systemd-tmpfiles-clean.service';
      @org.freedesktop.DBus.Property.EmitsChangedSignal("invalidates")
      readonly a(stt) TimersMonotonic = [('OnUnitActiveUSec', 86400000000, 49682252232703), ('OnBootUSec', 900000000, 900000000)];
      @org.freedesktop.DBus.Property.EmitsChangedSignal("invalidates")
      readonly a(sst) TimersCalendar = [];
      readonly t NextElapseUSecRealtime = 0;
      readonly t NextElapseUSecMonotonic = 49682252232703;
      readonly t LastTriggerUSec = 1671107914243400;
      readonly t LastTriggerUSecMonotonic = 49595852231768;
      readonly s Result = 'success';
      @org.freedesktop.DBus.Property.EmitsChangedSignal("const")
      readonly t AccuracyUSec = 60000000;
      @org.freedesktop.DBus.Property.EmitsChangedSignal("const")
      readonly t RandomizedDelayUSec = 0;
      @org.freedesktop.DBus.Property.EmitsChangedSignal("const")
      readonly b Persistent = false;
      @org.freedesktop.DBus.Property.EmitsChangedSignal("const")
      readonly b WakeSystem = false;
  };
  interface org.freedesktop.systemd1.Unit {
    methods:
      @org.freedesktop.systemd1.Privileged("true")
      Start(in  s arg_0,
            out o arg_1);
      @org.freedesktop.systemd1.Privileged("true")
      Stop(in  s arg_0,
           out o arg_1);
      @org.freedesktop.systemd1.Privileged("true")
      Reload(in  s arg_0,
             out o arg_1);
      @org.freedesktop.systemd1.Privileged("true")
      Restart(in  s arg_0,
              out o arg_1);
      @org.freedesktop.systemd1.Privileged("true")
      TryRestart(in  s arg_0,
                 out o arg_1);
      @org.freedesktop.systemd1.Privileged("true")
      ReloadOrRestart(in  s arg_0,
                      out o arg_1);
      @org.freedesktop.systemd1.Privileged("true")
      ReloadOrTryRestart(in  s arg_0,
                         out o arg_1);
      @org.freedesktop.systemd1.Privileged("true")
      Kill(in  s arg_0,
           in  i arg_1);
      @org.freedesktop.systemd1.Privileged("true")
      ResetFailed();
      @org.freedesktop.systemd1.Privileged("true")
      SetProperties(in  b arg_0,
                    in  a(sv) arg_1);
    signals:
    properties:
      @org.freedesktop.DBus.Property.EmitsChangedSignal("const")
      readonly s Id = 'systemd-tmpfiles-clean.timer';
      @org.freedesktop.DBus.Property.EmitsChangedSignal("const")
      readonly as Names = ['systemd-tmpfiles-clean.timer'];
      @org.freedesktop.DBus.Property.EmitsChangedSignal("false")
      readonly s Following = '';
      @org.freedesktop.DBus.Property.EmitsChangedSignal("const")
      readonly as Requires = ['sysinit.target'];
      @org.freedesktop.DBus.Property.EmitsChangedSignal("const")
      readonly as RequiresOverridable = [];
      @org.freedesktop.DBus.Property.EmitsChangedSignal("const")
      readonly as Requisite = [];
      @org.freedesktop.DBus.Property.EmitsChangedSignal("const")
      readonly as RequisiteOverridable = [];
      @org.freedesktop.DBus.Property.EmitsChangedSignal("const")
      readonly as Wants = [];
      @org.freedesktop.DBus.Property.EmitsChangedSignal("const")
      readonly as BindsTo = [];
      @org.freedesktop.DBus.Property.EmitsChangedSignal("const")
      readonly as PartOf = [];
      @org.freedesktop.DBus.Property.EmitsChangedSignal("const")
      readonly as RequiredBy = [];
      @org.freedesktop.DBus.Property.EmitsChangedSignal("const")
      readonly as RequiredByOverridable = [];
      @org.freedesktop.DBus.Property.EmitsChangedSignal("const")
      readonly as WantedBy = ['timers.target'];
      @org.freedesktop.DBus.Property.EmitsChangedSignal("const")
      readonly as BoundBy = [];
      @org.freedesktop.DBus.Property.EmitsChangedSignal("const")
      readonly as ConsistsOf = [];
      @org.freedesktop.DBus.Property.EmitsChangedSignal("const")
      readonly as Conflicts = ['shutdown.target'];
      @org.freedesktop.DBus.Property.EmitsChangedSignal("const")
      readonly as ConflictedBy = [];
      @org.freedesktop.DBus.Property.EmitsChangedSignal("const")
      readonly as Before = ['timers.target', 'shutdown.target', 'systemd-tmpfiles-clean.service'];
      @org.freedesktop.DBus.Property.EmitsChangedSignal("const")
      readonly as After = ['sysinit.target'];
      @org.freedesktop.DBus.Property.EmitsChangedSignal("const")
      readonly as OnFailure = [];
      @org.freedesktop.DBus.Property.EmitsChangedSignal("const")
      readonly as Triggers = ['systemd-tmpfiles-clean.service'];
      @org.freedesktop.DBus.Property.EmitsChangedSignal("const")
      readonly as TriggeredBy = [];
      @org.freedesktop.DBus.Property.EmitsChangedSignal("const")
      readonly as PropagatesReloadTo = [];
      @org.freedesktop.DBus.Property.EmitsChangedSignal("const")
      readonly as ReloadPropagatedFrom = [];
      @org.freedesktop.DBus.Property.EmitsChangedSignal("const")
      readonly as JoinsNamespaceOf = [];
      @org.freedesktop.DBus.Property.EmitsChangedSignal("const")
      readonly as RequiresMountsFor = [];
      @org.freedesktop.DBus.Property.EmitsChangedSignal("const")
      readonly as Documentation = ['man:tmpfiles.d(5)', 'man:systemd-tmpfiles(8)'];
      @org.freedesktop.DBus.Property.EmitsChangedSignal("const")
      readonly s Description = 'Daily Cleanup of Temporary Directories';
      @org.freedesktop.DBus.Property.EmitsChangedSignal("const")
      readonly s LoadState = 'loaded';
      readonly s ActiveState = 'active';
      readonly s SubState = 'waiting';
      @org.freedesktop.DBus.Property.EmitsChangedSignal("const")
      readonly s FragmentPath = '/usr/lib/systemd/system/systemd-tmpfiles-clean.timer';
      @org.freedesktop.DBus.Property.EmitsChangedSignal("const")
      readonly s SourcePath = '';
      @org.freedesktop.DBus.Property.EmitsChangedSignal("const")
      readonly as DropInPaths = [];
      @org.freedesktop.DBus.Property.EmitsChangedSignal("false")
      readonly s UnitFileState = 'static';
      @org.freedesktop.DBus.Property.EmitsChangedSignal("false")
      readonly s UnitFilePreset = 'disabled';
      readonly t InactiveExitTimestamp = 1621512065994411;
      readonly t InactiveExitTimestampMonotonic = 3982778;
      readonly t ActiveEnterTimestamp = 1621512065994411;
      readonly t ActiveEnterTimestampMonotonic = 3982778;
      readonly t ActiveExitTimestamp = 0;
      readonly t ActiveExitTimestampMonotonic = 0;
      readonly t InactiveEnterTimestamp = 0;
      readonly t InactiveEnterTimestampMonotonic = 0;
      @org.freedesktop.DBus.Property.EmitsChangedSignal("const")
      readonly b CanStart = true;
      @org.freedesktop.DBus.Property.EmitsChangedSignal("const")
      readonly b CanStop = true;
      @org.freedesktop.DBus.Property.EmitsChangedSignal("const")
      readonly b CanReload = false;
      @org.freedesktop.DBus.Property.EmitsChangedSignal("const")
      readonly b CanIsolate = false;
      readonly (uo) Job = (0, '/');
      @org.freedesktop.DBus.Property.EmitsChangedSignal("const")
      readonly b StopWhenUnneeded = false;
      @org.freedesktop.DBus.Property.EmitsChangedSignal("const")
      readonly b RefuseManualStart = false;
      @org.freedesktop.DBus.Property.EmitsChangedSignal("const")
      readonly b RefuseManualStop = false;
      @org.freedesktop.DBus.Property.EmitsChangedSignal("const")
      readonly b AllowIsolate = false;
      @org.freedesktop.DBus.Property.EmitsChangedSignal("const")
      readonly b DefaultDependencies = true;
      @org.freedesktop.DBus.Property.EmitsChangedSignal("const")
      readonly s OnFailureJobMode = 'replace';
      @org.freedesktop.DBus.Property.EmitsChangedSignal("const")
      readonly b IgnoreOnIsolate = false;
      @org.freedesktop.DBus.Property.EmitsChangedSignal("const")
      readonly b IgnoreOnSnapshot = false;
      @org.freedesktop.DBus.Property.EmitsChangedSignal("const")
      readonly b NeedDaemonReload = false;
      @org.freedesktop.DBus.Property.EmitsChangedSignal("const")
      readonly t JobTimeoutUSec = 0;
      @org.freedesktop.DBus.Property.EmitsChangedSignal("const")
      readonly s JobTimeoutAction = 'none';
      @org.freedesktop.DBus.Property.EmitsChangedSignal("const")
      readonly s JobTimeoutRebootArgument = '';
      readonly b ConditionResult = true;
      readonly b AssertResult = true;
      readonly t ConditionTimestamp = 1621512065994401;
      readonly t ConditionTimestampMonotonic = 3982770;
      readonly t AssertTimestamp = 1621512065994403;
      readonly t AssertTimestampMonotonic = 3982770;
      @org.freedesktop.DBus.Property.EmitsChangedSignal("false")
      readonly a(sbbsi) Conditions = [];
      @org.freedesktop.DBus.Property.EmitsChangedSignal("false")
      readonly a(sbbsi) Asserts = [];
      @org.freedesktop.DBus.Property.EmitsChangedSignal("const")
      readonly (ss) LoadError = ('', '');
      @org.freedesktop.DBus.Property.EmitsChangedSignal("const")
      readonly b Transient = false;
  };
};

52.2.3. 获取对象的定时器相关属性

gdbus call \
    --system \
    --dest org.freedesktop.systemd1 \
    --object-path /org/freedesktop/systemd1/unit/systemd_2dtmpfiles_2dclean_2etimer \
    --method org.freedesktop.DBus.Properties.GetAll \
    org.freedesktop.systemd1.Timer
({
    'Unit': <'systemd-tmpfiles-clean.service'>,
    'TimersMonotonic': <[
        ('OnUnitActiveUSec', uint64 86400000000, uint64 49682252232703),
        ('OnBootUSec', 900000000, 900000000)
    ]>,
    'TimersCalendar': <@a(sst) []>,
    'NextElapseUSecRealtime': <uint64 0>,
    'NextElapseUSecMonotonic': <uint64 49682252232703>,
    'LastTriggerUSec': <uint64 1671107914243400>,
    'LastTriggerUSecMonotonic': <uint64 49595852231768>,
    'Result': <'success'>,
    'AccuracyUSec': <uint64 60000000>,
    'RandomizedDelayUSec': <uint64 0>,
    'Persistent': <false>,
    'WakeSystem': <false>
},)

52.2.4. 获取对象定时器上次运行结果

gdbus call \
    --system \
    --dest org.freedesktop.systemd1 \
    --object-path /org/freedesktop/systemd1/unit/systemd_2dtmpfiles_2dclean_2etimer \
    --method org.freedesktop.DBus.Properties.Get \
    org.freedesktop.systemd1.Timer Result
(<'success'>,)

53. Android

53.1. 环境配置

53.1.1. 安装命令行工具

下载地址:

Command line tools onlycommandlinetools-linux-10406996_latest.zip

sudo pacman -S --noconfirm jdk17-openjdk jre17-openjdk jre17-openjdk-headless
sudo archlinux-java set java-17-openjdk

echo "export ANDROID_HOME=${HOME}/Android/Sdk" >> ~/.bashrc
echo "export ANDROID_SDK_HOME=${ANDROID_HOME}" >> ~/.bashrc
source ~/.bashrc

echo "export PATH=${ANDROID_HOME}/cmdline-tools/latest/bin:${ANDROID_HOME}/platform-tools:${ANDROID_HOME}/emulator:\$PATH" >> ~/.bashrc
source ~/.bashrc

mkdir -p ${ANDROID_HOME}

wget https://dl.google.com/android/repository/commandlinetools-linux-10406996_latest.zip -O commandlinetools-linux-10406996_latest.zip
unzip commandlinetools-linux-10406996_latest.zip -d ${ANDROID_HOME}
mkdir -p ${ANDROID_HOME}/cmdline-tools/latest/
mv ${ANDROID_HOME}/cmdline-tools/* ${ANDROID_HOME}/cmdline-tools/latest/

which sdkmanager

sdkmanager --version

sdkmanager --update

53.1.2. 初始化SDK配置

# 逐个同意许可证 'y'
sdkmanager --licenses

53.1.3. 查看包列表

sdkmanager --list

53.1.4. 安装开发工具

sdkmanager --install \
    platform-tools \
    emulator \
    'build-tools;34.0.0' \
    'patcher;v4' \
    'sources;android-34' \
    'platforms;android-33' \
    'platforms;android-34' \
    'system-images;android-34;google_apis;x86_64'

53.1.5. 测试工具

adb version
emulator version

sdkmanager --list_installed

emulator -list-avds

54. Flutter

54.1. 环境配置

echo "export PUB_HOSTED_URL=https://pub.flutter-io.cn" >> ~/.bash_profile
echo "export FLUTTER_STORAGE_BASE_URL=https://storage.flutter-io.cn" >> ~/.bash_profile
echo "export PATH=$PWD/flutter/bin:$PATH" >> ~/.bash_profile
. ~/.bash_profile

git clone https://github.com/flutter/flutter.git

which flutter dart

cd flutter

# ${HOME}/.config/flutter/settings
flutter config --android-sdk=${HOME}/Android/Sdk

# 不加代理 Network resources 检查会失败
export all_proxy="socks5://192.168.2.8:1080"
export http_proxy="http://192.168.2.8:2080"
export https_proxy="http://192.168.2.8:2080"
export no_proxy="localhost,::1,127.0.0.1"

flutter doctor -v

flutter doctor --android-licenses
每次运行 flutter doctor 都会请求网络下载数据

安装指定版本,比如3.10.6。只需要切换分支,然后运行 flutter doctor

git checkout 3.10.6

flutter doctor -v
flutter doctor 使用 proxychains 代理运行时,adb 命令会报错

55. Systemd

55.1. 最佳实践

55.1.1. 保存文本格式日志

标准配置
Service文件配置
cat << EOF > /usr/lib/systemd/system/xxl_job_admin.service
[Unit]
Description=XXL-JOB Admin Service
After=network.target remote-fs.target nss-lookup.target

[Service]
Type=simple
Environment="PORT=8080"
Environment="CONFIG_FILE=/data/xxl-job/admin/application.properties"
Environment="JAR_FILE=/data/xxl-job/admin/xxl-job-admin-2.2.0-SNAPSHOT.jar"

User=xxl_job
Group=xxl_job
WorkingDirectory=/data/xxl-job/admin
ExecStart=/usr/bin/java -Dserver.port=\${PORT} -Dspring.config.location=\${CONFIG_FILE} -jar \${JAR_FILE}
SuccessExitStatus=143
StandardOutput=null (1)
StandardError=journal (2)

[Install]
WantedBy=multi-user.target
EOF
1 标准输出重定向至 /dev/null
2 标准错误重定向至 journaljournal -u xxl_job_admin 会显示服务运行相关错误日志
增强配置

journal 采用非文本格式保存日志,查看不便,增加 syslog 配置,保存副本到文本文件。

重定向到 syslog 后,journal 有同样的日志内容。

Service文件配置:

cat << EOF > /usr/lib/systemd/system/xxl_job_admin.service
[Unit]
Description=XXL-JOB Admin Service
After=network.target remote-fs.target nss-lookup.target

[Service]
Type=simple
Environment="PORT=8080"
Environment="CONFIG_FILE=/data/xxl-job/admin/application.properties"
Environment="JAR_FILE=/data/xxl-job/admin/xxl-job-admin-2.2.0-SNAPSHOT.jar"

User=xxl_job
Group=xxl_job
WorkingDirectory=/data/xxl-job/admin
ExecStart=/usr/bin/java -Dserver.port=\${PORT} -Dspring.config.location=\${CONFIG_FILE} -jar \${JAR_FILE}
SuccessExitStatus=143
StandardOutput=null
StandardError=syslog (1)
SyslogIdentifier=xxl-job-admin (2)

[Install]
WantedBy=multi-user.target
EOF
1 标准错误重定向至 syslog
2 设置 syslog 标识

syslog 配置:

echo ':programname, isequal, "xxl-job-admin" /var/log/xxl_job/admin_error.log' > /etc/rsyslog.d/xxl-job-admin.conf

重启日志服务:

systemctl restart rsyslog

确认日志文件:

tail -f /var/log/xxl_job/admin_error.log

给Nginx配置一个有 HTTP基本身份验证 的站点,可以用浏览器查看日志,方便开发人员调试。

日志轮询归档配置
cat << EOF > /etc/logrotate.d/xxl-job-admin
/var/log/xxl_job/admin_error.log {
        daily
        missingok
        rotate 52
        compress
        delaycompress
        notifempty
        create 640 xxl_job adm
        copytruncate (1)
}
EOF
1 复制日志文件后,再清空旧日志文件。需要程序有 reload 类似支持才能改名后重载
归档时间由 /etc/cron.daily/logrotate 计划任务控制
Nginx配置
cat << EOF > /etc/nginx/conf.d/xxl_admin_log.conf
server {
    listen       34567;

    server_name _;
    root /var/log/xxl_job;
    default_type text/plain; (1)

    autoindex on;
    autoindex_localtime on;
    # 设置文件大小显示单位
    autoindex_exact_size off;

    auth_basic "Administrator’s Area";
    auth_basic_user_file /etc/nginx/.htpasswd_log;

    #location ~ \.log$ {
    #    default_type text/plain; (2)
    #}
}
EOF

nginx -s reload
1 设置 Content-Typetext/plain,浏览器会直接显示而不是提示下载
2 admin_error_log.1 默认配置能直接查看,只想支持 .log 文件可以采用本设置

55.1.2. 清理journal日志

# 查看日志总大小:方法一
du -sh /run/log/journal/

# 查看日志总大小:方法二
journalctl --disk-usage

# 清理前创建新日志文件
journalctl --rotate

# 清理N天前的日志
journalctl --vacuum-time=60d
journalctl --vacuum-time=2m

# 当前保留多大的日志文件总大小,超过大小的将被删除
journalctl --vacuum-size=500M

# 当前保留多少个日志文件,超过数量的将被删除
journalctl --vacuum-files=5
持久化配置
# 设置保存的日志总大小
crudini --set /etc/systemd/journald.conf Journal SystemMaxUse 500M
crudini --set /etc/systemd/journald.conf Journal RuntimeMaxUse 500M

systemctl restart systemd-journald

56. Jenkins

56.1. 最佳实践

56.1.1. Jenkins+HTPPS

cat << EOF > /etc/nginx/conf.d/jenkins.conf
map \$http_upgrade \$connection_upgrade {
  default upgrade;
  '' close;
}

server {
    listen 443 ssl http2;
    server_name jenkins.foo.com;
    root /data/web/jenkins.foo.com;
    access_log  /var/log/nginx/jenkins.foo.com_access.log main;
    error_log  /var/log/nginx/jenkins.foo.com_error.log;

    # certs sent to the client in SERVER HELLO are concatenated in ssl_certificate
    ssl_certificate /etc/letsencrypt/live/foo.com/fullchain.pem;
    ssl_certificate_key /etc/letsencrypt/live/foo.com/privkey.pem;
    ssl_session_timeout 1d;
    ssl_session_cache shared:SSL:50m;
    ssl_session_tickets off;

    # Diffie-Hellman parameter for DHE ciphersuites, recommended 2048 bits
    ssl_dhparam /etc/letsencrypt/live/foo.com/dhparam.pem;

    # intermediate configuration. tweak to your needs.
    ssl_protocols TLSv1 TLSv1.1 TLSv1.2;
    ssl_ciphers 'ECDHE-ECDSA-CHACHA20-POLY1305:ECDHE-RSA-CHACHA20-POLY1305:ECDHE-ECDSA-AES128-GCM-SHA256:ECDHE-RSA-AES128-GCM-SHA256:ECDHE-ECDSA-AES256-GCM-SHA384:ECDHE-RSA-AES256-GCM-SHA384:DHE-RSA-AES128-GCM-SHA256:DHE-RSA-AES256-GCM-SHA384:ECDHE-ECDSA-AES128-SHA256:ECDHE-RSA-AES128-SHA256:ECDHE-ECDSA-AES128-SHA:ECDHE-RSA-AES256-SHA384:ECDHE-RSA-AES128-SHA:ECDHE-ECDSA-AES256-SHA384:ECDHE-ECDSA-AES256-SHA:ECDHE-RSA-AES256-SHA:DHE-RSA-AES128-SHA256:DHE-RSA-AES128-SHA:DHE-RSA-AES256-SHA256:DHE-RSA-AES256-SHA:ECDHE-ECDSA-DES-CBC3-SHA:ECDHE-RSA-DES-CBC3-SHA:EDH-RSA-DES-CBC3-SHA:AES128-GCM-SHA256:AES256-GCM-SHA384:AES128-SHA256:AES256-SHA256:AES128-SHA:AES256-SHA:DES-CBC3-SHA:!DSS';
    ssl_prefer_server_ciphers on;

    # HSTS (ngx_http_headers_module is required) (15768000 seconds = 6 months)
    add_header Strict-Transport-Security max-age=15768000;

    # OCSP Stapling ---
    # fetch OCSP records from URL in ssl_certificate and cache them
    ssl_stapling on;
    ssl_stapling_verify on;
    # nginx: Specifies a file with trusted CA certificates in the PEM format used to verify client certificates and OCSP responses if ssl_stapling is enabled.
    # certbot: If you’re using OCSP stapling with Nginx >= 1.3.7, chain.pem should be provided as the ssl_trusted_certificate to validate OCSP responses.
    ssl_trusted_certificate /etc/letsencrypt/live/foo.com/chain.pem;

    # pass through headers from Jenkins that Nginx considers invalid
    ignore_invalid_headers off;

    location / {
        #this is the maximum upload size
        client_max_body_size       10m;
        client_body_buffer_size    128k;

        proxy_pass http://127.0.0.1:8282;

        proxy_connect_timeout 90;
        proxy_send_timeout 90;
        proxy_read_timeout 90;
        proxy_request_buffering off; # Required for HTTP CLI commands
        proxy_set_header Connection ""; # Clear for keepalive

        # Required for Jenkins websocket agents
        proxy_set_header Connection \$connection_upgrade;
        proxy_set_header Upgrade \$http_upgrade;

        proxy_set_header Host \$http_host;
        proxy_set_header X-Forwarded-For \$proxy_add_x_forwarded_for;
        proxy_set_header X-Forwarded-Proto \$scheme;
        proxy_set_header X-Forwarded-Port \$server_port;
        proxy_set_header X-Real-IP \$remote_addr;
        proxy_max_temp_file_size 0;
        proxy_redirect default;
        proxy_http_version 1.1;
    }

    location /crumbIssuer { (1)
        return 403;
    }
}
EOF
1 恶意程序可能通过 /crumbIssuer/api/json 在未经授权的情况下,安装挖坑程序。默认禁止访问

57. 杂项

57.1. Chrome

57.1.1. 安装 Proxy SwitchyOmega 扩展

安装 Proxy SwitchyOmega 之前,需要访问Google应用商店,让Chrome挂全局代理:

Linux

google-chrome-stable --proxy-server="socks5://192.168.1.5:1080"

Windows

Google Chrome 快捷方式的属性中,追加程序运行参数,得到 "C:\xxxx\chrome.exe" --proxy-server="socks5://192.168.1.5:1080"

然后,安装扩展 Proxy SwitchyOmega

Appendix A: The Extended ASCII Table

A.1. ASCII control characters (character code 0-31)

The first 32 characters in the ASCII-table are unprintable control codes and are used to control peripherals such as printers.

DEC OCT HEX BIN Symbol HTML Number HTML Name Description

0

000

00

00000000

NUL

&#000

Null char

1

001

01

00000001

SOH

&#001

Start of Heading

2

002

02

00000010

STX

&#002

Start of Text

3

003

03

00000011

ETX

&#003

End of Text

4

004

04

00000100

EOT

&#004

End of Transmission

5

005

05

00000101

ENQ

&#005

Enquiry

6

006

06

00000110

ACK

&#006

Acknowledgment

7

007

07

00000111

BEL

&#007

Bell

8

010

08

00001000

BS

&#008

Back Space

9

011

09

00001001

HT

&#009

Horizontal Tab

10

012

0A

00001010

LF

&#010

Line Feed

11

013

0B

00001011

VT

&#011

Vertical Tab

12

014

0C

00001100

FF

&#012

Form Feed

13

015

0D

00001101

CR

&#013

Carriage Return

14

016

0E

00001110

SO

&#014

Shift Out / X-On

15

017

0F

00001111

SI

&#015

Shift In / X-Off

16

020

10

00010000

DLE

&#016

Data Line Escape

17

021

11

00010001

DC1

&#017

Device Control 1 (oft. XON)

18

022

12

00010010

DC2

&#018

Device Control 2

19

023

13

00010011

DC3

&#019

Device Control 3 (oft. XOFF)

20

024

14

00010100

DC4

&#020

Device Control 4

21

025

15

00010101

NAK

&#021

Negative Acknowledgement

22

026

16

00010110

SYN

&#022

Synchronous Idle

23

027

17

00010111

ETB

&#023

End of Transmit Block

24

030

18

00011000

CAN

&#024

Cancel

25

031

19

00011001

EM

&#025

End of Medium

26

032

1A

00011010

SUB

&#026

Substitute

27

033

1B

00011011

ESC

&#027

Escape

28

034

1C

00011100

FS

&#028

File Separator

29

035

1D

00011101

GS

&#029

Group Separator

30

036

1E

00011110

RS

&#030

Record Separator

31

037

1F

00011111

US

&#031

Unit Separator

A.2. ASCII printable characters (character code 32-127)

Codes 32-127 are common for all the different variations of the ASCII table, they are called printable characters, represent letters, digits, punctuation marks, and a few miscellaneous symbols. You will find almost every character on your keyboard. Character 127 represents the command DEL.

DEC OCT HEX BIN Symbol HTML Number HTML Name Description

32

040

20

00100000

 

&#32

Space

33

041

21

00100001

!

&#33

Exclamation mark

34

042

22

00100010

"

&#34;

&quot;

Double quotes (or speech marks)

35

043

23

00100011

#

&#35

Number

36

044

24

00100100

$

&#36

Dollar

37

045

25

00100101

%

&#37

Procenttecken

38

046

26

00100110

&

&#38;

&amp;

Ampersand

39

047

27

00100111

'

&#39

Single quote

40

050

28

00101000

(

&#40

Open parenthesis (or open bracket)

41

051

29

00101001

)

&#41

Close parenthesis (or close bracket)

42

052

2A

00101010

*

&#42

Asterisk

43

053

2B

00101011

+

&#43

Plus

44

054

2C

00101100

,

&#44

Comma

45

055

2D

00101101

-

&#45

Hyphen

46

056

2E

00101110

.

&#46

Period, dot or full stop

47

057

2F

00101111

/

&#47

Slash or divide

48

060

30

00110000

0

&#48

Zero

49

061

31

00110001

1

&#49

One

50

062

32

00110010

2

&#50

Two

51

063

33

00110011

3

&#51

Three

52

064

34

00110100

4

&#52

Four

53

065

35

00110101

5

&#53

Five

54

066

36

00110110

6

&#54

Six

55

067

37

00110111

7

&#55

Seven

56

070

38

00111000

8

&#56

Eight

57

071

39

00111001

9

&#57

Nine

58

072

3A

00111010

:

&#58

Colon

59

073

3B

00111011

;

&#59

Semicolon

60

074

3C

00111100

<

&#60;

&lt;

Less than (or open angled bracket)

61

075

3D

00111101

=

&#61

Equals

62

076

3E

00111110

>

&#62;

&gt;

Greater than (or close angled bracket)

63

077

3F

00111111

?

&#63

Question mark

64

100

40

01000000

@

&#64

At symbol

65

101

41

01000001

A

&#65

Uppercase A

66

102

42

01000010

B

&#66

Uppercase B

67

103

43

01000011

C

&#67

Uppercase C

68

104

44

01000100

D

&#68

Uppercase D

69

105

45

01000101

E

&#69

Uppercase E

70

106

46

01000110

F

&#70

Uppercase F

71

107

47

01000111

G

&#71

Uppercase G

72

110

48

01001000

H

&#72

Uppercase H

73

111

49

01001001

I

&#73

Uppercase I

74

112

4A

01001010

J

&#74

Uppercase J

75

113

4B

01001011

K

&#75

Uppercase K

76

114

4C

01001100

L

&#76

Uppercase L

77

115

4D

01001101

M

&#77

Uppercase M

78

116

4E

01001110

N

&#78

Uppercase N

79

117

4F

01001111

O

&#79

Uppercase O

80

120

50

01010000

P

&#80

Uppercase P

81

121

51

01010001

Q

&#81

Uppercase Q

82

122

52

01010010

R

&#82

Uppercase R

83

123

53

01010011

S

&#83

Uppercase S

84

124

54

01010100

T

&#84

Uppercase T

85

125

55

01010101

U

&#85

Uppercase U

86

126

56

01010110

V

&#86

Uppercase V

87

127

57

01010111

W

&#87

Uppercase W

88

130

58

01011000

X

&#88

Uppercase X

89

131

59

01011001

Y

&#89

Uppercase Y

90

132

5A

01011010

Z

&#90

Uppercase Z

91

133

5B

01011011

[

&#91

Opening bracket

92

134

5C

01011100

\

&#92

Backslash

93

135

5D

01011101

]

&#93

Closing bracket

94

136

5E

01011110

^

&#94

Caret - circumflex

95

137

5F

01011111

_

&#95

Underscore

96

140

60

01100000

`

&#96

Grave accent

97

141

61

01100001

a

&#97

Lowercase a

98

142

62

01100010

b

&#98

Lowercase b

99

143

63

01100011

c

&#99

Lowercase c

100

144

64

01100100

d

&#100

Lowercase d

101

145

65

01100101

e

&#101

Lowercase e

102

146

66

01100110

f

&#102

Lowercase f

103

147

67

01100111

g

&#103

Lowercase g

104

150

68

01101000

h

&#104

Lowercase h

105

151

69

01101001

i

&#105

Lowercase i

106

152

6A

01101010

j

&#106

Lowercase j

107

153

6B

01101011

k

&#107

Lowercase k

108

154

6C

01101100

l

&#108

Lowercase l

109

155

6D

01101101

m

&#109

Lowercase m

110

156

6E

01101110

n

&#110

Lowercase n

111

157

6F

01101111

o

&#111

Lowercase o

112

160

70

01110000

p

&#112

Lowercase p

113

161

71

01110001

q

&#113

Lowercase q

114

162

72

01110010

r

&#114

Lowercase r

115

163

73

01110011

s

&#115

Lowercase s

116

164

74

01110100

t

&#116

Lowercase t

117

165

75

01110101

u

&#117

Lowercase u

118

166

76

01110110

v

&#118

Lowercase v

119

167

77

01110111

w

&#119

Lowercase w

120

170

78

01111000

x

&#120

Lowercase x

121

171

79

01111001

y

&#121

Lowercase y

122

172

7A

01111010

z

&#122

Lowercase z

123

173

7B

01111011

{

&#123

Opening brace

124

174

7C

01111100

|

&#124

Vertical bar

125

175

7D

01111101

}

&#125

Closing brace

126

176

7E

01111110

~

&#126

Equivalency sign - tilde

127

177

7F

01111111

&#127

Delete

A.3. The extended ASCII codes (character code 128-255)

There are several different variations of the 8-bit ASCII table. The table below is according to ISO 8859-1, also called ISO Latin-1. Codes 128-159 contain the Microsoft® Windows Latin-1 extended characters.

DEC OCT HEX BIN Symbol HTML Number HTML Name Description

128

200

80

10000000

&#128;

&euro;

Euro sign

129

201

81

10000001

 

 

 

 

130

202

82

10000010

&#130;

&sbquo;

Single low-9 quotation mark

131

203

83

10000011

ƒ

&#131;

&fnof;

Latin small letter f with hook

132

204

84

10000100

&#132;

&bdquo;

Double low-9 quotation mark

133

205

85

10000101

&#133;

&hellip;

Horizontal ellipsis

134

206

86

10000110

&#134;

&dagger;

Dagger

135

207

87

10000111

&#135;

&Dagger;

Double dagger

136

210

88

10001000

ˆ

&#136;

&circ;

Modifier letter circumflex accent

137

211

89

10001001

&#137;

&permil;

Per mille sign

138

212

8A

10001010

Š

&#138;

&Scaron;

Latin capital letter S with caron

139

213

8B

10001011

&#139;

&lsaquo;

Single left-pointing angle quotation

140

214

8C

10001100

Œ

&#140;

&OElig;

Latin capital ligature OE

141

215

8D

10001101

 

 

 

 

142

216

8E

10001110

Ž

&#142;

 

Latin captial letter Z with caron

143

217

8F

10001111

 

 

 

 

144

220

90

10010000

 

 

 

 

145

221

91

10010001

&#145;

&lsquo;

Left single quotation mark

146

222

92

10010010

&#146;

&rsquo;

Right single quotation mark

147

223

93

10010011

&#147;

&ldquo;

Left double quotation mark

148

224

94

10010100

&#148;

&rdquo;

Right double quotation mark

149

225

95

10010101

&#149;

&bull;

Bullet

150

226

96

10010110

&#150;

&ndash;

En dash

151

227

97

10010111

&#151;

&mdash;

Em dash

152

230

98

10011000

˜

&#152;

&tilde;

Small tilde

153

231

99

10011001

&#153;

&trade;

Trade mark sign

154

232

9A

10011010

š

&#154;

&scaron;

Latin small letter S with caron

155

233

9B

10011011

&#155;

&rsaquo;

Single right-pointing angle quotation mark

156

234

9C

10011100

œ

&#156;

&oelig;

Latin small ligature oe

157

235

9D

10011101

 

 

 

 

158

236

9E

10011110

ž

&#158;

 

Latin small letter z with caron

159

237

9F

10011111

Ÿ

&#159;

&Yuml;

Latin capital letter Y with diaeresis

160

240

A0

10100000

 

&#160;

&nbsp;

Non-breaking space

161

241

A1

10100001

¡

&#161;

&iexcl;

Inverted exclamation mark

162

242

A2

10100010

¢

&#162;

&cent;

Cent sign

163

243

A3

10100011

£

&#163;

&pound;

Pound sign

164

244

A4

10100100

¤

&#164;

&curren;

Currency sign

165

245

A5

10100101

¥

&#165;

&yen;

Yen sign

166

246

A6

10100110

¦

&#166;

&brvbar;

Pipe, Broken vertical bar

167

247

A7

10100111

§

&#167;

&sect;

Section sign

168

250

A8

10101000

¨

&#168;

&uml;

Spacing diaeresis - umlaut

169

251

A9

10101001

©

&#169;

&copy;

Copyright sign

170

252

AA

10101010

ª

&#170;

&ordf;

Feminine ordinal indicator

171

253

AB

10101011

«

&#171;

&laquo;

Left double angle quotes

172

254

AC

10101100

¬

&#172;

&not;

Not sign

173

255

AD

10101101

­

&#173;

&shy;

Soft hyphen

174

256

AE

10101110

®

&#174;

&reg;

Registered trade mark sign

175

257

AF

10101111

¯

&#175;

&macr;

Spacing macron - overline

176

260

B0

10110000

°

&#176;

&deg;

Degree sign

177

261

B1

10110001

±

&#177;

&plusmn;

Plus-or-minus sign

178

262

B2

10110010

²

&#178;

&sup2;

Superscript two - squared

179

263

B3

10110011

³

&#179;

&sup3;

Superscript three - cubed

180

264

B4

10110100

´

&#180;

&acute;

Acute accent - spacing acute

181

265

B5

10110101

µ

&#181;

&micro;

Micro sign

182

266

B6

10110110

&#182;

&para;

Pilcrow sign - paragraph sign

183

267

B7

10110111

·

&#183;

&middot;

Middle dot - Georgian comma

184

270

B8

10111000

¸

&#184;

&cedil;

Spacing cedilla

185

271

B9

10111001

¹

&#185;

&sup1;

Superscript one

186

272

BA

10111010

º

&#186;

&ordm;

Masculine ordinal indicator

187

273

BB

10111011

»

&#187;

&raquo;

Right double angle quotes

188

274

BC

10111100

¼

&#188;

&frac14;

Fraction one quarter

189

275

BD

10111101

½

&#189;

&frac12;

Fraction one half

190

276

BE

10111110

¾

&#190;

&frac34;

Fraction three quarters

191

277

BF

10111111

¿

&#191;

&iquest;

Inverted question mark

192

300

C0

11000000

À

&#192;

&Agrave;

Latin capital letter A with grave

193

301

C1

11000001

Á

&#193;

&Aacute;

Latin capital letter A with acute

194

302

C2

11000010

Â

&#194;

&Acirc;

Latin capital letter A with circumflex

195

303

C3

11000011

Ã

&#195;

&Atilde;

Latin capital letter A with tilde

196

304

C4

11000100

Ä

&#196;

&Auml;

Latin capital letter A with diaeresis

197

305

C5

11000101

Å

&#197;

&Aring;

Latin capital letter A with ring above

198

306

C6

11000110

Æ

&#198;

&AElig;

Latin capital letter AE

199

307

C7

11000111

Ç

&#199;

&Ccedil;

Latin capital letter C with cedilla

200

310

C8

11001000

È

&#200;

&Egrave;

Latin capital letter E with grave

201

311

C9

11001001

É

&#201;

&Eacute;

Latin capital letter E with acute

202

312

CA

11001010

Ê

&#202;

&Ecirc;

Latin capital letter E with circumflex

203

313

CB

11001011

Ë

&#203;

&Euml;

Latin capital letter E with diaeresis

204

314

CC

11001100

Ì

&#204;

&Igrave;

Latin capital letter I with grave

205

315

CD

11001101

Í

&#205;

&Iacute;

Latin capital letter I with acute

206

316

CE

11001110

Î

&#206;

&Icirc;

Latin capital letter I with circumflex

207

317

CF

11001111

Ï

&#207;

&Iuml;

Latin capital letter I with diaeresis

208

320

D0

11010000

Ð

&#208;

&ETH;

Latin capital letter ETH

209

321

D1

11010001

Ñ

&#209;

&Ntilde;

Latin capital letter N with tilde

210

322

D2

11010010

Ò

&#210;

&Ograve;

Latin capital letter O with grave

211

323

D3

11010011

Ó

&#211;

&Oacute;

Latin capital letter O with acute

212

324

D4

11010100

Ô

&#212;

&Ocirc;

Latin capital letter O with circumflex

213

325

D5

11010101

Õ

&#213;

&Otilde;

Latin capital letter O with tilde

214

326

D6

11010110

Ö

&#214;

&Ouml;

Latin capital letter O with diaeresis

215

327

D7

11010111

×

&#215;

&times;

Multiplication sign

216

330

D8

11011000

Ø

&#216;

&Oslash;

Latin capital letter O with slash

217

331

D9

11011001

Ù

&#217;

&Ugrave;

Latin capital letter U with grave

218

332

DA

11011010

Ú

&#218;

&Uacute;

Latin capital letter U with acute

219

333

DB

11011011

Û

&#219;

&Ucirc;

Latin capital letter U with circumflex

220

334

DC

11011100

Ü

&#220;

&Uuml;

Latin capital letter U with diaeresis

221

335

DD

11011101

Ý

&#221;

&Yacute;

Latin capital letter Y with acute

222

336

DE

11011110

Þ

&#222;

&THORN;

Latin capital letter THORN

223

337

DF

11011111

ß

&#223;

&szlig;

Latin small letter sharp s - ess-zed

224

340

E0

11100000

à

&#224;

&agrave;

Latin small letter a with grave

225

341

E1

11100001

á

&#225;

&aacute;

Latin small letter a with acute

226

342

E2

11100010

â

&#226;

&acirc;

Latin small letter a with circumflex

227

343

E3

11100011

ã

&#227;

&atilde;

Latin small letter a with tilde

228

344

E4

11100100

ä

&#228;

&auml;

Latin small letter a with diaeresis

229

345

E5

11100101

å

&#229;

&aring;

Latin small letter a with ring above

230

346

E6

11100110

æ

&#230;

&aelig;

Latin small letter ae

231

347

E7

11100111

ç

&#231;

&ccedil;

Latin small letter c with cedilla

232

350

E8

11101000

è

&#232;

&egrave;

Latin small letter e with grave

233

351

E9

11101001

é

&#233;

&eacute;

Latin small letter e with acute

234

352

EA

11101010

ê

&#234;

&ecirc;

Latin small letter e with circumflex

235

353

EB

11101011

ë

&#235;

&euml;

Latin small letter e with diaeresis

236

354

EC

11101100

ì

&#236;

&igrave;

Latin small letter i with grave

237

355

ED

11101101

í

&#237;

&iacute;

Latin small letter i with acute

238

356

EE

11101110

î

&#238;

&icirc;

Latin small letter i with circumflex

239

357

EF

11101111

ï

&#239;

&iuml;

Latin small letter i with diaeresis

240

360

F0

11110000

ð

&#240;

&eth;

Latin small letter eth

241

361

F1

11110001

ñ

&#241;

&ntilde;

Latin small letter n with tilde

242

362

F2

11110010

ò

&#242;

&ograve;

Latin small letter o with grave

243

363

F3

11110011

ó

&#243;

&oacute;

Latin small letter o with acute

244

364

F4

11110100

ô

&#244;

&ocirc;

Latin small letter o with circumflex

245

365

F5

11110101

õ

&#245;

&otilde;

Latin small letter o with tilde

246

366

F6

11110110

ö

&#246;

&ouml;

Latin small letter o with diaeresis

247

367

F7

11110111

÷

&#247;

&divide;

Division sign

248

370

F8

11111000

ø

&#248;

&oslash;

Latin small letter o with slash

249

371

F9

11111001

ù

&#249;

&ugrave;

Latin small letter u with grave

250

372

FA

11111010

ú

&#250;

&uacute;

Latin small letter u with acute

251

373

FB

11111011

û

&#251;

&ucirc;

Latin small letter u with circumflex

252

374

FC

11111100

ü

&#252;

&uuml;

Latin small letter u with diaeresis

253

375

FD

11111101

ý

&#253;

&yacute;

Latin small letter y with acute

254

376

FE

11111110

þ

&#254;

&thorn;

Latin small letter thorn

255

377

FF

11111111

ÿ

&#255;

&yuml;

Latin small letter y with diaeresis