新增两台联想 ThinkSystem SR860 V3 服务器到虚拟化生产集群

新增两台联想 ThinkSystem SR860 V3 服务器到虚拟化生产集群 Add Two Lenovo ThinkSystem SR860 V3 Servers to the Virtualization Production Cluster
1、将两台 SR860 V3 服务器上架到机柜。确认电源四路接入,并做好标识与基础检查。检查服务器硬件完整性(CPU、内存、RAID 控制器、HBA、网卡)

2、SAN 连接

- 每台服务器至少双 HBA(多路径)。
- 连接到两个独立 Fabric(A/B)SAN 交换机。

3、网络连接

- 每台服务器四网口绑定(链路聚合)。
- 配置管理 VLAN。

4、SAN光交换机配置

#1、登录10.77.77.77、10.77.77.78
telnet 10.77.77.77
telnet 10.77.77.78

#2、检查 zoneset 已有成员
# show zoneset active vsan 10
zoneset name server vsan 10
  zone name zone1 vsan 10
  * fcid 0x880000 [interface fc1/1 swwn 20:00:00:3a:9c:ca:62:70]
  * fcid 0x881500 [interface fc1/2 swwn 20:00:00:3a:9c:ca:62:70]
  * fcid 0x880200 [interface fc1/3 swwn 20:00:00:3a:9c:ca:62:70]

  zone name zone2 vsan 10
  * fcid 0x880000 [interface fc1/1 swwn 20:00:00:3a:9c:ca:62:70]
  * fcid 0x881500 [interface fc1/2 swwn 20:00:00:3a:9c:ca:62:70]
  * fcid 0x880600 [interface fc1/4 swwn 20:00:00:3a:9c:ca:62:70]

  zone name zone3 vsan 10
  * fcid 0x880000 [interface fc1/1 swwn 20:00:00:3a:9c:ca:62:70]
  * fcid 0x881500 [interface fc1/2 swwn 20:00:00:3a:9c:ca:62:70]
  * fcid 0x881600 [interface fc1/5 swwn 20:00:00:3a:9c:ca:62:70]

  zone name zone4 vsan 10
  * fcid 0x880000 [interface fc1/1 swwn 20:00:00:3a:9c:ca:62:70]
  * fcid 0x881500 [interface fc1/2 swwn 20:00:00:3a:9c:ca:62:70]
  * fcid 0x880800 [interface fc1/6 swwn 20:00:00:3a:9c:ca:62:70]

  zone name zone5 vsan 10
  * fcid 0x880000 [interface fc1/1 swwn 20:00:00:3a:9c:ca:62:70]
  * fcid 0x881500 [interface fc1/2 swwn 20:00:00:3a:9c:ca:62:70]
  * fcid 0x880500 [interface fc1/7 swwn 20:00:00:3a:9c:ca:62:70]

  zone name zone6 vsan 10
  * fcid 0x880000 [interface fc1/1 swwn 20:00:00:3a:9c:ca:62:70]
  * fcid 0x881500 [interface fc1/2 swwn 20:00:00:3a:9c:ca:62:70]
  * fcid 0x880400 [interface fc1/8 swwn 20:00:00:3a:9c:ca:62:70]

  zone name desktop01 vsan 10
  * fcid 0x881300 [interface fc1/37 swwn 20:00:00:3a:9c:ca:62:70]
  * fcid 0x880b00 [interface fc1/38 swwn 20:00:00:3a:9c:ca:62:70]
  * fcid 0x880c00 [interface fc1/39 swwn 20:00:00:3a:9c:ca:62:70]

  zone name desktop02 vsan 10
  * fcid 0x881300 [interface fc1/37 swwn 20:00:00:3a:9c:ca:62:70]
  * fcid 0x880b00 [interface fc1/38 swwn 20:00:00:3a:9c:ca:62:70]
  * fcid 0x880d00 [interface fc1/40 swwn 20:00:00:3a:9c:ca:62:70]

  zone name desktop03 vsan 10
  * fcid 0x881300 [interface fc1/37 swwn 20:00:00:3a:9c:ca:62:70]
  * fcid 0x880b00 [interface fc1/38 swwn 20:00:00:3a:9c:ca:62:70]
  * fcid 0x880e00 [interface fc1/41 swwn 20:00:00:3a:9c:ca:62:70]

  zone name desktop04 vsan 10
  * fcid 0x881300 [interface fc1/37 swwn 20:00:00:3a:9c:ca:62:70]
  * fcid 0x880b00 [interface fc1/38 swwn 20:00:00:3a:9c:ca:62:70]
  * fcid 0x880f00 [interface fc1/42 swwn 20:00:00:3a:9c:ca:62:70]

  zone name desktop05 vsan 10
  * fcid 0x881300 [interface fc1/37 swwn 20:00:00:3a:9c:ca:62:70]
  * fcid 0x880b00 [interface fc1/38 swwn 20:00:00:3a:9c:ca:62:70]
  * fcid 0x881000 [interface fc1/43 swwn 20:00:00:3a:9c:ca:62:70]

  zone name desktop06 vsan 10
  * fcid 0x881300 [interface fc1/37 swwn 20:00:00:3a:9c:ca:62:70]
  * fcid 0x880b00 [interface fc1/38 swwn 20:00:00:3a:9c:ca:62:70]
  * fcid 0x881200 [interface fc1/44 swwn 20:00:00:3a:9c:ca:62:70]

#3、确认服务器是否成功连接到SAN交换机
SWFS6700UP(config)# show flogi database detail
--------------------------------------------------------------------------------------------
INTERFACE        VSAN    FCID           PORT NAME               NODE NAME            FLAGS
--------------------------------------------------------------------------------------------
fc1/1            10    0x880000  56:c9:2b:fa:00:21:7e:20 56:c9:2b:fa:00:00:7e:20 P
fc1/2            10    0x881500  56:c9:2b:fa:00:21:7e:21 56:c9:2b:fa:00:00:7e:21 P
fc1/3            10    0x880200  21:00:f4:e9:d4:59:b1:04 20:00:f4:e9:d4:59:b1:04 P
fc1/4            10    0x880600  21:00:f4:e9:d4:59:a6:fa 20:00:f4:e9:d4:59:a6:fa P
fc1/5            10    0x881600  21:00:f4:e9:d4:59:ae:13 20:00:f4:e9:d4:59:ae:13 P
fc1/6            10    0x880800  21:00:f4:e9:d4:59:a6:fe 20:00:f4:e9:d4:59:a6:fe P
fc1/7            10    0x880500  21:00:34:80:0d:6e:21:98 20:00:34:80:0d:6e:21:98 P
fc1/8            10    0x880400  21:00:f4:e9:d4:59:a6:e8 20:00:f4:e9:d4:59:a6:e8 P
fc1/9            10    0x881700  10:00:70:b7:e4:2d:91:a6 20:00:70:b7:e4:2d:91:a6 P
fc1/10           10    0x881800  10:00:70:b7:e4:2d:91:6a 20:00:70:b7:e4:2d:91:6a P
fc1/37           10    0x881300  56:c9:2b:fa:00:21:89:90 56:c9:2b:fa:00:00:89:90 P
fc1/38           10    0x880b00  56:c9:2b:fa:00:22:89:91 56:c9:2b:fa:00:00:89:91 P
fc1/39           10    0x880c00  10:00:00:10:9b:c6:9f:ab 20:00:00:10:9b:c6:9f:ab P
fc1/40           10    0x880d00  10:00:00:10:9b:c6:9f:cf 20:00:00:10:9b:c6:9f:cf P
fc1/41           10    0x880e00  10:00:00:10:9b:c6:9b:45 20:00:00:10:9b:c6:9b:45 P
fc1/42           10    0x880f00  10:00:00:10:9b:c6:b1:3c 20:00:00:10:9b:c6:b1:3c P
fc1/43           10    0x881000  10:00:00:10:9b:c6:9d:f9 20:00:00:10:9b:c6:9d:f9 P
fc1/44           10    0x881200  10:00:00:10:9b:c6:b4:2d 20:00:00:10:9b:c6:b4:2d P

Total number of flogi = 18.

FLAGS:
A area FCID allocation
L loop device
V FDISC
D the wwn matches the default OUI list
O the wwn matches the configured OUI list
P allocation was done based on the persistency table
M indicates a VMID capable FCID
#4、新增zone,添加新服务器接口
#zone name zone7 vsan 10
member interface fc1/1
member interface fc1/2
member interface fc1/9

#zone name zone8 vsan 10
member interface fc1/1
member interface fc1/2
member interface fc1/10

zoneset name server vsan 10
member zone7
member zone8

#5、激活 zoneset
zoneset activate name server vsan 10

show zoneset active vsan 10
show flogi database vsan 10

#6、保存配置
copy running-config startup-config


#7、修改 zone 成员(如有误)(fc1/7 → fc1/9)
进入 zone 配置模式
conf t
zone name zone7 vsan 10
  no member interface fc1/7
  member interface fc1/9


5、网络交换机配置

#
interface Eth-Trunk7
 description LenovoUP
 port link-type trunk
 port trunk allow-pass vlan 2 to 4094
 mode lacp-dynamic
#
interface Eth-Trunk8
 port link-type trunk
 port trunk allow-pass vlan 2 to 4094
 mode lacp-dynamic
#
#
interface 10GE1/0/25
 eth-trunk 7
 device transceiver 10GBASE-FIBER
#
interface 10GE1/0/26
 eth-trunk 7
 device transceiver 10GBASE-FIBER
#
interface 10GE1/0/27
 eth-trunk 7
 device transceiver 10GBASE-FIBER
#
interface 10GE1/0/28
 eth-trunk 7
 device transceiver 10GBASE-FIBER
#
interface 10GE1/0/29
 eth-trunk 8
 device transceiver 10GBASE-FIBER
#
interface 10GE1/0/30
 eth-trunk 8
 device transceiver 10GBASE-FIBER
#               
interface 10GE1/0/31
 eth-trunk 8
 device transceiver 10GBASE-FIBER
#
interface 10GE1/0/32
 eth-trunk 8
 device transceiver 10GBASE-FIBER
#

6、存储添加和卷映射主机


6、服务器安装ESXi7操作系统


7、更新ESXi补丁

# esxcli software sources profile list -d /vmfs/volumes/60071865-c14e10d6-bc6f-b4055d4dc1b4/ISO/VMware-ESXi-7.0U3w-24784741-depot.zip

Name                           Vendor        Acceptance Level  Creation Time        Modification Time
-----------------------------  ------------  ----------------  -------------------  -----------------
ESXi-7.0U3w-24784741-standard  VMware, Inc.  PartnerSupported  2025-07-15T00:00:00  2025-07-15T00:00:00
ESXi-7.0U3w-24784741-no-tools  VMware, Inc.  PartnerSupported  2025-07-15T00:00:00  2025-06-09T07:05:28

# esxcli software profile update -p ESXi-7.0U3w-24784741-standard -d  /vmfs/volumes/60071865-c14e10d6-bc6f-b4055d4dc1b4/ISO/VMware-ESXi-7.0U3w-24784741-depot.zip

8、配置 vSwitch0
负载均衡 修改为 基于 IP 哈希的路由
9、创建 vMotion 网络(vMotion_Network)用于虚拟机迁移。

10、FS6700和博科620SAN交换机配置区别

功能 Cisco(思科 MDS) Brocade(博科) 核心区别 / 注意事项
进入配置模式 conf t configure(很少用在zoning) Cisco 强依赖配置模式;Brocade zoning 多为直接命令
定义逻辑域 vsan database vsan 10 ❌(无 VSAN,单 Fabric) 最大区别:Cisco 支持 VSAN 多租户隔离,Brocade 不支持**
创建别名 fcalias name host1 vsan 10 member pwwn xx alicreate "host1","pwwn" 本质一样(alias),但 Cisco 必须绑定 VSAN
创建zone(两个成员) zone name zone1 vsan 10 member fcalias host1 member fcalias storage1 zonecreate "zone1","host1;storage1" Cisco 多行配置;Brocade 一行完成
添加成员到zone member fcalias host2 zoneadd "zone1","host2" Brocade 支持直接追加;Cisco 在 zone 下操作
创建zone集合 zoneset name zs1 vsan 10 member zone1 cfgcreate "cfg1","zone1" zoneset = cfg(最关键映射)
添加zone到集合 member zone2 cfgadd "cfg1","zone2" Cisco 是上下文命令;Brocade 是独立命令
激活配置 zoneset activate name zs1 vsan 10 cfgenable "cfg1" Cisco 按 VSAN 激活;Brocade 全局激活
保存配置 copy running-config startup-config cfgsave Brocade 不保存会丢;Cisco 也一样但机制不同
查看激活配置 show zoneset active vsan 10 cfgactvshow Cisco 需要指定 VSAN
查看全部配置 show zone cfgshow Brocade 输出更集中(含 alias/zone/cfg)

浪潮 NF5280M5 服务器 CPU 升级 Xeon Gold 6248 实战与经验总结

充分盘活现有硬件资源,本次选取浪潮(Inspur)NF5280M5 服务器进行定向硬件升级,在不更换整机平台的前提下,将原配置的 Xeon Gold 6132 处理器升级为 Xeon Gold 6248

一、升级前确认(非常重要)

1、平台兼容性确认

浪潮 NF280M5 支持:Intel Xeon® Scalable 第一代(Skylake-SP)和 Intel Xeon® Scalable 第二代(Cascade Lake-SP)
- 插槽:LGA3647
- 芯片组:Intel C621

型号 核心 / 线程 基准频率 最大睿频 L3 缓存 TDP
Xeon Gold 6132(当前机型) 14 / 28 2.60 GHz 3.70 GHz 19.25 MB 140 W
Xeon Gold 6134 8 / 16 3.20 GHz 3.70 GHz 24.75 MB 130 W
Xeon Gold 6138 20 / 40 2.00 GHz 3.70 GHz 27.5 MB 125 W
Xeon Gold 6230 20 / 40 2.10 GHz 4.00 GHz 27.5 MB 125 W
Xeon Gold 6230R 26 / 52 2.10 GHz 4.00 GHz 35.75 MB 150 W
Xeon Gold 6248 20 / 40 2.50 GHz 3.90 GHz 27.5 MB 150 W
Xeon Gold 6252 24 / 48 2.10 GHz 3.90 GHz 35.75 MB 150 W
Xeon Platinum 8260 24 / 48 2.40 GHz 3.90 GHz 35.75 MB 205 W
Xeon Platinum 8270 26 / 52 2.70 GHz 4.00 GHz 35.75 MB 205 W
Xeon Platinum 8276L 28 / 56 2.20 GHz 4.00 GHz 38.5 MB 165 W
Xeon Platinum 8280 28 / 56 2.70 GHz 4.00 GHz 38.5 MB 205 W
Xeon Platinum 8280L(大内存版) 28 / 56 2.70 GHz 4.00 GHz 38.5 MB 205 W

2、型号推荐

高性能计算 / 大数据/虚拟化
推荐:Xeon Gold 6230R / Xeon Platinum 8260 / Platinum 8280

通用服务器负载(Web/应用/RDB)
推荐:Xeon Gold 6138 / Gold 6248 / Gold 6252

当前 CPU:Intel Xeon Gold 6132
- 架构:Skylake-SP
- 插槽:LGA3647
- 芯片组:C620 系列
- 核心数:14 核
- 线程数:28 线程
- 基础频率:2.60 GHz
- 最大睿频:3.70 GHz
- L3 缓存:19.25 MB
- TDP:140W

目标 CPU:Intel Xeon Gold 6248
- 架构:Cascade Lake-SP
- 插槽:LGA3647
- 芯片组:C620 系列
- 核心数:20 核
- 线程数:40 线程
- 基础频率:2.50 GHz
- 最大睿频:3.90 GHz
- L3 缓存:27.5 MB
- TDP:150W

3、散热器是否支持 150W

6132 TDP = 140W
6248 TDP = 150W

二、准备工具

- T30梅花内六角螺丝刀(长柄)
- 无水酒精
- 无尘布
- 硅脂(信越7921)
- 防静电措施

三、拆卸步骤(关键)

这种是 四螺丝压框结构,必须按顺序。

第一步:断电

- 关机
- 拔电源
- 按电源键 10 秒放电

第二步:拆散热器

NF5280M5 一般是四颗弹簧螺丝:

按对角线顺序慢慢松:

4 3 2 1

每颗螺丝分几次慢慢松,不要一次拧到底,防止主板受力不均

第三步:拆 CPU 扣具(LGA3647 重点)

- 使用卡片,按照缝隙,一点一点撬起来

四、安装新 CPU(6248)

1、把旧 CPU 从扣具取下

注意方向标识(三角标)

2、放回插槽

轻轻放入定位槽

五、涂硅脂 + 装散热器

- 中间一点或十字涂抹

3、安装CPU扣具和散热

(普通手感:拧紧到停止,不要暴力)

按顺序拧紧:1 2 3 4

六、开机验证

进 BIOS 看:

- CPU 型号
- 核心数
- 频率

七、NF5280M5 升级注意事项

1. 必须双路一致

如果是双路机器:

6132 + 6248 混插是不行的

必须两颗都换成 6248

2. 内存频率变化

6132 最高 2666
6248 支持 2933

如果内存支持 2933,会自动提升

3.Xeon Gold 6132 和 Xeon Gold 6248 参数对比

CPU 型号参数对比

项目 Xeon Gold 6132 Xeon Gold 6248
架构 Skylake Cascade Lake
制程工艺 14 nm 14 nm
核心数 14 核 20 核
线程数 28 线程 40 线程
基础频率 2.60 GHz 2.50 GHz
最大睿频 3.70 GHz 3.90 GHz
L3 缓存 19.25 MB 27.5 MB
TDP 140 W 150 W
内存支持 DDR4-2666 ECC(6 通道) DDR4-2933 ECC(6 通道)
插槽 LGA3647 LGA3647

在VMware ESXi上对三星PCIe 4.0 NVMe SSD 980 PRO硬盘进行直通DirectPath和虚拟化Virtualization的读写测试

在VMware ESXi上对三星PCIe 4.0 NVMe SSD 980 PRO硬盘进行直通DirectPath和虚拟化Virtualization的读写测试
存储虚拟化后,读性能损失约21.5%,写性能损失约18.4%

Read and Write Tests of Samsung PCIe 4.0 NVMe SSD 980 PRO Hard Drive with DirectPath and Virtualization on VMware ESXi

物理服务器 NF5280M5
硬盘 PCIEx4转NMVe
虚拟化软件 ESXi 7.03
虚拟机配置 8核心 16G内存 120G系统盘
操作系统 CentOS7.9
数据盘 三星 PCIe 4.0 NVMe SSD 980 PRO
测试软件 fio

1、软件安装

yum -y install epel-release
yum -y install fio

2、挂载数据目录

sudo vgcreate lvmgroup /dev/vdb
sudo lvcreate -l 100%FREE -n data lvmgroup
sudo mkfs.xfs /dev/lvmgroup/data
sudo mkdir /data
sudo echo "/dev/lvmgroup/data /data xfs defaults 0 0" >>/etc/fstab
sudo mount -av

3、编写 fio 测试配置文件:创建一个用于 fio 测试的配置文件。您可以使用文本编辑器创建一个新文件,例如 fio_test_config.fio

[global]
ioengine=libaio
direct=1
thread=1
rw=randrw
bs=4k
numjobs=4
size=32G
runtime=300
group_reporting

[randwrite]

[global]:这部分指定了全局配置,适用于所有作业。
ioengine=libaio:指定I/O引擎为libaio,表示使用异步I/O。
direct=1:启用直接I/O,绕过操作系统的文件系统缓存。
thread=1:每个作业使用一个线程。
rw=randrw:读写模式为随机读写。
bs=4k:块大小为4KB。
numjobs=4:作业数量为4,表示并发的作业数。
size=32G:每个作业的测试文件大小为32GB。
runtime=300:测试运行时间为300秒。
group_reporting:指定所有线程的统计信息应汇总在一起报告。

直通DirectPath

[root@10-53-220-58 ~]# cd /data/
[root@10-53-220-58 data]# ls
[root@10-53-220-58 data]# sudo dd if=/dev/zero of=/data/fio_test_file bs=1G count=32
32+0 records in
32+0 records out
34359738368 bytes (34 GB) copied, 36.9548 s, 930 MB/s
[root@10-53-220-58 data]# vi fio_test_config.fio
[root@10-53-220-58 data]# sudo fio fio_test_config.fio
randwrite: (g=0): rw=randrw, bs=(R) 4096B-4096B, (W) 4096B-4096B, (T) 4096B-4096B, ioengine=libaio, iodepth=1
...
fio-3.7
Starting 4 threads
randwrite: Laying out IO file (1 file / 32768MiB)
randwrite: Laying out IO file (1 file / 32768MiB)
randwrite: Laying out IO file (1 file / 32768MiB)
randwrite: Laying out IO file (1 file / 32768MiB)
Jobs: 4 (f=4): [m(4)][100.0%][r=102MiB/s,w=103MiB/s][r=26.2k,w=26.3k IOPS][eta 00m:00s] 
randwrite: (groupid=0, jobs=4): err= 0: pid=9048: Fri Apr 12 13:55:32 2024
   read: IOPS=26.0k, BW=102MiB/s (107MB/s)(29.8GiB/300001msec)
    slat (nsec): min=3381, max=69136, avg=4802.53, stdev=1677.41
    clat (usec): min=21, max=19215, avg=113.61, stdev=218.66
     lat (usec): min=34, max=19220, avg=118.47, stdev=218.66
    clat percentiles (usec):
     |  1.00th=[   48],  5.00th=[   51], 10.00th=[   53], 20.00th=[   55],
     | 30.00th=[   57], 40.00th=[   59], 50.00th=[   65], 60.00th=[   74],
     | 70.00th=[   77], 80.00th=[   85], 90.00th=[   93], 95.00th=[  343],
     | 99.00th=[ 1270], 99.50th=[ 1369], 99.90th=[ 1516], 99.95th=[ 1565],
     | 99.99th=[ 1958]
   bw (  KiB/s): min=20680, max=36128, per=24.99%, avg=26004.68, stdev=2194.12, samples=2397
   iops        : min= 5170, max= 9032, avg=6501.10, stdev=548.54, samples=2397
  write: IOPS=25.0k, BW=102MiB/s (106MB/s)(29.7GiB/300001msec)
    slat (usec): min=4, max=135, avg= 6.16, stdev= 1.90
    clat (nsec): min=1292, max=1175.4k, avg=27396.71, stdev=3962.20
     lat (usec): min=19, max=1181, avg=33.62, stdev= 4.09
    clat percentiles (nsec):
     |  1.00th=[18304],  5.00th=[21888], 10.00th=[23680], 20.00th=[24960],
     | 30.00th=[26496], 40.00th=[26752], 50.00th=[27264], 60.00th=[27520],
     | 70.00th=[28288], 80.00th=[29312], 90.00th=[31616], 95.00th=[34048],
     | 99.00th=[39168], 99.50th=[41216], 99.90th=[43776], 99.95th=[45312],
     | 99.99th=[50944]
   bw (  KiB/s): min=20280, max=36008, per=24.99%, avg=25980.38, stdev=2247.11, samples=2397
   iops        : min= 5070, max= 9002, avg=6495.02, stdev=561.78, samples=2397
  lat (usec)   : 2=0.04%, 4=0.01%, 10=0.01%, 20=0.90%, 50=50.88%
  lat (usec)   : 100=44.43%, 250=1.06%, 500=0.46%, 750=0.46%, 1000=0.53%
  lat (msec)   : 2=1.22%, 4=0.01%, 10=0.01%, 20=0.01%
  cpu          : usr=4.49%, sys=11.07%, ctx=15587579, majf=0, minf=10
  IO depths    : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0%
     submit    : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     complete  : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     issued rwts: total=7804560,7797520,0,0 short=0,0,0,0 dropped=0,0,0,0
     latency   : target=0, window=0, percentile=100.00%, depth=1

Run status group 0 (all jobs):
   READ: bw=102MiB/s (107MB/s), 102MiB/s-102MiB/s (107MB/s-107MB/s), io=29.8GiB (31.0GB), run=300001-300001msec
  WRITE: bw=102MiB/s (106MB/s), 102MiB/s-102MiB/s (106MB/s-106MB/s), io=29.7GiB (31.9GB), run=300001-300001msec

Disk stats (read/write):
    dm-2: ios=7801557/7794611, merge=0/0, ticks=862107/186550, in_queue=1048584, util=100.00%, aggrios=7804560/7797562, aggrmerge=0/1, aggrticks=857542/181607, aggrin_queue=1039149, aggrutil=100.00%
  nvme0n1: ios=7804560/7797562, merge=0/1, ticks=857542/181607, in_queue=1039149, util=100.00%
www.zhangfangzhou.cn

虚拟化Virtualization

[root@10-53-220-44 data]# sudo dd if=/dev/zero of=/data/fio_test_file bs=1G count=32
32+0 records in
32+0 records out
34359738368 bytes (34 GB) copied, 71.8294 s, 478 MB/s
[root@10-53-220-44 data]# vi fio_test_config.fio
[root@10-53-220-44 data]# sudo fio fio_test_config.fio
randwrite: (g=0): rw=randrw, bs=(R) 4096B-4096B, (W) 4096B-4096B, (T) 4096B-4096B, ioengine=libaio, iodepth=1
...
fio-3.7
Starting 4 threads
randwrite: Laying out IO file (1 file / 32768MiB)
randwrite: Laying out IO file (1 file / 32768MiB)
randwrite: Laying out IO file (1 file / 32768MiB)
randwrite: Laying out IO file (1 file / 32768MiB)
Jobs: 4 (f=4): [m(4)][100.0%][r=78.2MiB/s,w=77.8MiB/s][r=20.0k,w=19.9k IOPS][eta 00m:00s]
randwrite: (groupid=0, jobs=4): err= 0: pid=9016: Fri Apr 12 14:43:21 2024
   read: IOPS=20.4k, BW=79.7MiB/s (83.6MB/s)(23.4GiB/300001msec)
    slat (usec): min=4, max=114, avg= 6.78, stdev= 1.98
    clat (usec): min=42, max=38534, avg=134.39, stdev=190.16
     lat (usec): min=53, max=38545, avg=141.24, stdev=190.14
    clat percentiles (usec):
     |  1.00th=[   79],  5.00th=[   87], 10.00th=[   89], 20.00th=[   92],
     | 30.00th=[   93], 40.00th=[   96], 50.00th=[   98], 60.00th=[  101],
     | 70.00th=[  104], 80.00th=[  106], 90.00th=[  113], 95.00th=[  161],
     | 99.00th=[ 1205], 99.50th=[ 1319], 99.90th=[ 1467], 99.95th=[ 1516],
     | 99.99th=[ 2057]
   bw (  KiB/s): min=   73, max=24856, per=16.35%, avg=13347.32, stdev=9723.44, samples=2397
   iops        : min=   18, max= 6214, avg=3336.67, stdev=2431.02, samples=2397
  write: IOPS=20.4k, BW=79.6MiB/s (83.5MB/s)(23.3GiB/300001msec)
    slat (usec): min=5, max=115, avg= 8.06, stdev= 1.98
    clat (nsec): min=1403, max=38544k, avg=44914.21, stdev=37185.93
     lat (usec): min=32, max=38560, avg=53.04, stdev=37.23
    clat percentiles (usec):
     |  1.00th=[   36],  5.00th=[   39], 10.00th=[   40], 20.00th=[   42],
     | 30.00th=[   43], 40.00th=[   44], 50.00th=[   45], 60.00th=[   46],
     | 70.00th=[   47], 80.00th=[   48], 90.00th=[   50], 95.00th=[   52],
     | 99.00th=[   57], 99.50th=[   59], 99.90th=[   69], 99.95th=[   87],
     | 99.99th=[  135]
   bw (  KiB/s): min=   71, max=25352, per=16.36%, avg=13341.18, stdev=9726.48, samples=2397
   iops        : min=   17, max= 6338, avg=3335.13, stdev=2431.78, samples=2397
  lat (usec)   : 2=0.01%, 10=0.01%, 20=0.01%, 50=45.37%, 100=33.22%
  lat (usec)   : 250=19.16%, 500=0.43%, 750=0.43%, 1000=0.46%
  lat (msec)   : 2=0.91%, 4=0.01%, 10=0.01%, 20=0.01%, 50=0.01%
  cpu          : usr=3.53%, sys=11.43%, ctx=12237590, majf=0, minf=11
  IO depths    : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0%
     submit    : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     complete  : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     issued rwts: total=6121601,6116013,0,0 short=0,0,0,0 dropped=0,0,0,0
     latency   : target=0, window=0, percentile=100.00%, depth=1

Run status group 0 (all jobs):
   READ: bw=79.7MiB/s (83.6MB/s), 79.7MiB/s-79.7MiB/s (83.6MB/s-83.6MB/s), io=23.4GiB (25.1GB), run=300001-300001msec
  WRITE: bw=79.6MiB/s (83.5MB/s), 79.6MiB/s-79.6MiB/s (83.5MB/s-83.5MB/s), io=23.3GiB (25.1GB), run=300001-300001msec

Disk stats (read/write):
    dm-2: ios=6120646/6115090, merge=0/0, ticks=806337/255779, in_queue=1062184, util=100.00%, aggrios=6121601/6116035, aggrmerge=0/1, aggrticks=801719/251319, aggrin_queue=1051124, aggrutil=100.00%
  sdb: ios=6121601/6116035, merge=0/1, ticks=801719/251319, in_queue=1051124, util=100.00%
www.zhangfangzhou.cn

4、结果
直通 (DirectPath)
read: IOPS=26.0k, BW=102MiB/s (107MB/s)(29.8GiB/300001msec)
write: IOPS=25.0k, BW=102MiB/s (106MB/s)(29.7GiB/300001msec)

虚拟化Virtualization
read: IOPS=20.4k, BW=79.7MiB/s (83.6MB/s)(23.4GiB/300001msec)
write: IOPS=20.4k, BW=79.6MiB/s (83.5MB/s)(23.3GiB/300001msec)

 

读取 (Read) 写入 (Write) 带宽 (Bandwidth)
直通 (DirectPath) 26.0k 25.0k 102MiB/s (107MB/s)
虚拟化 (Virtualization) 20.4k 20.4k 79.7MiB/s (83.6MB/s)

存储虚拟化后,读性能损失约21.5%,写性能损失约18.4%

备份软件之Veeam Backup Replication

备份软件之Veeam Backup Replication
一、为什么要做备份
1、数据对于个人用户和企业来说都是非常宝贵的资产。这些软件提供了可靠的备份机制,可以保护数据免受硬件故障、意外删除、恶意软件和其他灾难性事件的影响。通过定期备份数据,用户可以确保其数据的完整性和可用性。
2、当发生数据丢失、系统崩溃或其他灾难性事件时,可以帮助用户快速恢复数据和系统。这对于保证业务连续性至关重要,尤其对于依赖于数据和系统的企业来说。备份软件提供了灵活的恢复选项,可以按需恢复整个系统、单个文件或文件夹,以满足不同恢复需求。

二、常见的备份软件有哪些
1、安克诺斯
2、Veeam
3、云琪
4、NetBackup & Veritas Backup Exec
5、Zerto

三、Veeam Backup Replication 下载安装
1、Veeam Backup Replication 官方网站ISO文件 点击下载Veeam Backup Replication 12.1.0.2131_20231206
安装要求(推荐)
操作系统推荐 Windows Server 2016-2022
CPU要求至少四个内核、内存建议4GB以上。
存储空间建议100GB以上,备份用空间根据需求来算。
数据库可使用自带的PostgreSQL 15.1 或安装 SQL Server 2012-2022进行使用。
目前支持VMware vSphere 6.0-8.0U2版本

四、Veeam Backup Replication安装

五、激活Veeam Backup Replication(VBR)软件为专业版
替换文件下载
1、替换文件下载1
2、替换文件下载2

VeeamLicense.dll
#替换文件

Veeam_ASv11_1500.lic
Veeam_BackupRep_v12_5000_1.lic
#授权文件

六、停止服务和替换文件
安装完软件之后使用管理员运行PowerShell 执行下面命令:

get-service -displayname Veeam* | stop-service
get-service -displayname Veeam*

七、启动服务

get-service -displayname Veeam* | start-service

八、导入授权

打开Veeam Backup & Replication Console 软件,点击左上角的菜单,选择License选项,
点击install,选择Veeam_BackupRep_v12_5000_1.lic 文件导入安装即可激活。

参考来源
https://www.digiboy.ir/12153/veeam-backup-replication-12-1-0-2131/

使用CLI命令行安装部署vCSA 7 (vCenter Server Appliance 7)

如何使用CLI命令行部署VMware VCSA 7 \ vCSA 7 (vCenter 7)
使用CLI 部署 vCenter Server Appliance 7
使用CLI命令行安装部署 vCenter Server Appliance 7

1、使用CLI命令行安装部署,embedded_vCSA_on_ESXi.json (ESXi 7未加入集群)


# embedded_vCSA_on_ESXi.json ESXi 7 未加入vCenter集群

{
    "__version": "2.13.0",
    "__comments": "Sample template to deploy a vCenter Server Appliance with an embedded Platform Services Controller on an ESXi host.",
    "new_vcsa": {
        "esxi": {
            "hostname": "10.33.201.202",                 #esxi的fqdn或ip
            "username": "root",
            "password": "ESXi的密码",
            "deployment_network": "VM Network",
            "datastore": "Datastore"                     #存放的数据存储
        },
        "appliance": {
            "thin_disk_mode": true,                      #硬盘模式:精简制备
            "deployment_option": "medium",               #部署大小:tiny small medium large,分别为 微、小、中、大
            "name": "www.zhangfangzhou.cn"  #vCenter-Server-Appliance的名字
        },
        "network": {
            "ip_family": "ipv4",
            "mode": "static",
            "ip": "10.33.201.80",
            "prefix": "22",                               #网络前缀24位
            "gateway": "10.33.200.1",
            "dns_servers": [
                "10.33.123.60"
            ]
        },
        "os": {
            "password": "密码",
            "ntp_servers": "ntp.aliyun.com",
            "ssh_enable":  true                            #ssh开启
        },
        "sso": {
            "password": "密码",
            "domain_name": "zhangfangzhou.cn"                     #sso域名
        }
    },
    "ceip": {
        "description": {
            "__comments": [
                "++++VMware Customer Experience Improvement Program (CEIP)++++",
                "--acknowledge-ceip in the command line.",
                "++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++"
            ]
        },
        "settings": {
            "ceip_enabled": false           #不参加(CEIP)客户体验改善计划
        }
    }
}

​注意事项
"system.name": "10.0.0.30"      #系统名称,全域名,例vc.vsphere.local如果FQDN不能解析,请使用IP地址作为系统名称,不然会Firstboot Error

vcsa-deploy.exe install --accept-eula --no-ssl-certificate-verification -t E:\ISO\ESXi\VMware-VCSA-all-7.0.3-18700403\vcsa-cli-installer\templates\install\embedded_vCSA_on_ESXi.json

DRS Warning:  The target ESXi host '10.33.201.202' is managed by vCenter Server
'10.33.201.200'.  If any of these hosts are in a cluster, and DRS is enabled,
vMotion can take effect and adversely impact the installation, upgrade, or
migration processes.  It is recommended that you use the *_on_VC.json template
file for the target ESXi host if it is managed by a vCenter Server, and ensure
the ESXi hosts you have specified are not members of clusters with DRS set to
Fully Automated during the installation, upgrade, or migration processes.

2 、使用CLI命令行安装部署,embedded_vCSA_on_VC.json ESXi 7已经加入vCenter集群

"deployment_option": "medium",               #部署大小:tiny small medium large,分别为 微、小、中、大
#######################################################################################################
例子1
{
    "__version": "2.13.0",
    "__comments": "Sample template to deploy a vCenter Server Appliance with an embedded Platform Services Controller on a vCenter Server instance.",
    "new_vcsa": {
        "vc": {
            "hostname": "10.33.201.66",
            "username": "administrator@zhangfangzhou.cn",
            "password": "vCenter Server 的密码",
            "deployment_network": "VM Network",
            "datacenter": [
                "Datacenter"                    #数据中心名称
            ],
            "datastore": "NFS1",                #存储
            "target": [
            "Cluster1", "10.33.220.99"          #集群名称和目标ESXi主机
            ]
        },
        "appliance": {
            "thin_disk_mode": true,
            "deployment_option": "medium",
            "name": "www.zhangfangzhou.cn"
        },
        "network": {
            "ip_family": "ipv4",
            "mode": "static",
            "system_name": "10.33.220.82",
            "ip": "10.33.220.82",
            "prefix": "24",
            "gateway": "10.33.220.1",
            "dns_servers": [
                "223.5.5.5,114.114.114.114"
            ]
        },
        "os": {
            "password": "密码",
            "ntp_servers": "ntp.aliyun.com",
            "ssh_enable":  true
        },
        "sso": {
            "password": "密码",
            "domain_name": "zhangfangzhou.cn"
        }
    },
    "ceip": {
        "description": {
            "__comments": [
                "++++VMware Customer Experience Improvement Program (CEIP)++++",
                "--acknowledge-ceip in the command line.",
                "++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++"
            ]
        },
        "settings": {
            "ceip_enabled": false
        }
    }
}

#######################################################################################################
C:\Users\Administrator>e:
E:\>cd E:\ISO\ESXi\VMware-VCSA-all-7.0.3-18778458\vcsa-cli-installer\win32

E:\ISO\ESXi\VMware-VCSA-all-7.0.3-18778458\vcsa-cli-installer\win32>
vcsa-deploy.exe install --accept-eula --no-ssl-certificate-verification -t E:\ISO\ESXi\VMware-VCSA-all-7.0.3-18778458\vcsa-cli-installer\templates\install\embedded_vCSA_on_VC.json

vCenter Server Appliance 的 CLI 部署
https://docs.vmware.com/cn/VMware-vSphere/7.0/vsphere-vcenter-server-703-installation-guide.pdf
CLI 部署命令的语法
https://docs.vmware.com/cn/VMware-vSphere/7.0/com.vmware.vcenter.install.doc/GUID-15F4F48B-44D9-4E3C-B9CF-5FFC71515F71.html

--accept-eula   接受最终用户许可协议(最终用户许可协议; End User License Agreement;)。
--no-esx-ssl-verify 跳过 ESXi 连接的 SSL 验证。
--no-ssl-certificate-verification   跳过所有服务器连接的安全证书验证。
-v, --verbose   将调试信息添加到控制台输出。
-t, --terse 隐藏控制台输出。仅显示警告消息和错误消息。
--verify-template-only  对 JSON 文件中的配置参数执行基本模板验证。不部署设备。
--precheck-only 仅执行基本模板验证和 OVF Tool 参数验证。不部署设备。

https://www.zhangfangzhou.cn/cli-install-vcsa.html

基于DHCP服务器使用 PXE 引导和 kickstart (UEFI) 自动安装 ESXi 7

基于DHCP服务器使用 PXE 引导和 kickstart (UEFI) 自动安装 ESXi 7
基于UEFI的PXE安装 ESXi 7
PXE网络引导自动化安装 ESXi 7 详解
PXE+Kickstart 批量安装 ESXi 7
使用PXE 和TFTP 引导ESXi 安装ESXi 7 程序

1 基于DHCP服务器使用 PXE 引导和 kickstart (UEFI) 自动安装 ESXi 7 录制视频

2 浪潮NF5280M5服务器基于DHCP服务器使用 PXE 引导和 kickstart (UEFI) 自动安装 ESXi 7 录制视频

#CentOS 7 系统 安装dhcp服务器 、 tftp-server服务器 、 web服务器
yum install -y dhcp tftp-server httpd

1、配置DHCP服务器

守护进程 :/etc/sbin/dhcpd
脚本:/etc/init.d/dhcpd
端口:67(bootps) 服务端,68(bootpc) 客户端
配置文件:/etc/dhcp/dhcpd.conf
租约信息:/var/lib/dhcpd/dhcpd.leases
配置文件:cp /usr/share/doc/dhcp-4.2.5/dhcpd.conf.example /etc/dhcp/dhcpd.conf
修改DHCP监听特定端口 vi /etc/sysconfig/dhcpd

#编辑DHCP服务器配置文件
vi /etc/dhcp/dhcpd.conf

default-lease-time 600;
max-lease-time 7200;
log-facility local7;

subnet 10.53.220.0 netmask 255.255.255.0 {
  range 10.53.220.41 10.53.220.49;
  option domain-name-servers 10.1.0.1, 114.114.114.114;   #DNS服务器IP地址
  option routers 10.53.220.1;                             #网关地址
  next-server 10.53.220.224;                              # 指定TFTP服务器地址
  filename "/mboot.efi";                                  # 指定网络引导映像文件
}
#绑定IP地址
host zhangfangzhou {                                      #zhangfangzhou 随意指定,但必须唯一
  hardware ethernet 00:50:56:99:06:b7;
  fixed-address 10.53.220.48;
}

#DHCP的服务设置开机启动、启动DHCP服务
systemctl enable dhcpd && systemctl start dhcpd

########防火墙设置
firewall-cmd --zone=public --add-port=67/udp --permanent
firewall-cmd --zone=public --add-port=68/udp --permanent
#重新载入配置
firewall-cmd --reload
firewall-cmd --list-all #查看防火墙规则,只显示/etc/firewalld/zones/public.xml中防火墙策略

2、配置 tftp 服务器

启用 tftp 服务
vi /etc/xinetd.d/tftp

# default: off
# description: The tftp server serves files using the trivial file transfer \
#       protocol.  The tftp protocol is often used to boot diskless \
#       workstations, download configuration files to network-aware printers, \
#       and to start the installation process for some operating systems.
service tftp
{
        socket_type             = dgram
        protocol                = udp
        wait                    = yes
        user                    = root
        server                  = /usr/sbin/in.tftpd
        server_args             = -s /var/lib/tftpboot # TFTP服务器顶级目录
        disable                 = no    # 从yes修改为no
        per_source              = 11
        cps                     = 100 2
        flags                   = IPv4
}

#tftp的服务设置开机启动、启动tftp服务
systemctl enable tftp && systemctl start tftp

########防火墙设置
firewall-cmd --zone=public --add-port=69/tcp --permanent
firewall-cmd --zone=public --add-port=69/udp --permanent
#重新载入配置
firewall-cmd --reload
firewall-cmd --list-all #查看防火墙规则,只显示/etc/firewalld/zones/public.xml中防火墙策略

3、挂载ESXi 7 镜像

#创建文件夹,主要是下载ESXi 7和挂载ESXi 7使用
mkdir -p /var/lib/tftpboot/{iso,ESXi70u2}
wget -P /var/lib/tftpboot/iso  http://10.53.123.144/ISO/ESXi/VMware-VMvisor-Installer-7.0U2a-17867351.x86_64.iso
cd /var/lib/tftpboot
mount /var/lib/tftpboot/iso/VMware-VMvisor-Installer-7.0U2a-17867351.x86_64.iso /var/lib/tftpboot/ESXi70u2     #重启后需要重新mount

#1、UEFI启动 ESXi 7
将上面DHCP服务器指定的启动镜像文件复制到指定目录下。上面的设置示例名为“mboot.efi”,因此将其复制为该名称。
cd /var/lib/tftpboot/
cp -p /var/lib/tftpboot/ESXi70u2/efi/boot/bootx64.efi /var/lib/tftpboot/mboot.efi

直接在 tftpboot 下复制名为 boot.cfg 的文件,该文件描述了引导设置。
cp -p /var/lib/tftpboot/ESXi70u2/efi/boot/boot.cfg /var/lib/tftpboot/boot.cfg
 #2、传统BIOS启动 ESXi 7
如果要使用旧版 BIOS,则需要3.86版的 syslinux 软件包。从 https://www.kernel.org/pub/linux/utils/boot/syslinux/ 下载包
cd /tmp
wget https://mirrors.edge.kernel.org/pub/linux/utils/boot/syslinux/3.xx/syslinux-3.86.tar.gz
tar xvzf syslinux-3.86.tar.gz
cp -p syslinux-3.86/core/mboot.efi /var/lib/tftpboot/
#3、修改boot.cfg
删除 boot.cfg 中文件路径的“/”
sed -i 's|/||g' boot.cfg

4、检查boot.cfg

根据您的环境更改“标题”和“前缀”。为“title”使用一个描述性名称是个好主意,它是自动安装过程中显示的标题字符。“前缀prefix”指定存储 ESXi 安装程序的目录。在这种情况下,它将是“ESXi70u2”。

修改title和refix名称
prefix=ESXi70u2 (ESXi70u2就是上面ESXi ISO文件mount的文件夹)

# vi /var/lib/tftpboot/boot.cfg
bootstate=0
title=Loading ESXi installer www.zhangfangzhou.cn
timeout=5
prefix=ESXi70u2
kernel=b.b00
kernelopt=runweasel cdromBoot
modules=jumpstrt.gz --- useropts.gz --- features.gz --- k.b00 --- uc_intel.b00 --- uc_amd.b00 --- uc_hygon.b00 --- procfs.b00 --- vmx.v00 --- vim.v00 --- tpm.v00 --- sb.v00 --- s.v00 --- atlantic.v00 --- bnxtnet.v00 --- bnxtroce.v00 --- brcmfcoe.v00 --- brcmnvme.v00 --- elxiscsi.v00 --- elxnet.v00 --- i40enu.v00 --- iavmd.v00 --- icen.v00 --- igbn.v00 --- irdman.v00 --- iser.v00 --- ixgben.v00 --- lpfc.v00 --- lpnic.v00 --- lsi_mr3.v00 --- lsi_msgp.v00 --- lsi_msgp.v01 --- lsi_msgp.v02 --- mtip32xx.v00 --- ne1000.v00 --- nenic.v00 --- nfnic.v00 --- nhpsa.v00 --- nmlx4_co.v00 --- nmlx4_en.v00 --- nmlx4_rd.v00 --- nmlx5_co.v00 --- nmlx5_rd.v00 --- ntg3.v00 --- nvme_pci.v00 --- nvmerdma.v00 --- nvmxnet3.v00 --- nvmxnet3.v01 --- pvscsi.v00 --- qcnic.v00 --- qedentv.v00 --- qedrntv.v00 --- qfle3.v00 --- qfle3f.v00 --- qfle3i.v00 --- qflge.v00 --- rste.v00 --- sfvmk.v00 --- smartpqi.v00 --- vmkata.v00 --- vmkfcoe.v00 --- vmkusb.v00 --- vmw_ahci.v00 --- clusters.v00 --- crx.v00 --- elx_esx_.v00 --- btldr.v00 --- esx_dvfi.v00 --- esx_ui.v00 --- esxupdt.v00 --- tpmesxup.v00 --- weaselin.v00 --- loadesx.v00 --- lsuv2_hp.v00 --- lsuv2_in.v00 --- lsuv2_ls.v00 --- lsuv2_nv.v00 --- lsuv2_oe.v00 --- lsuv2_oe.v01 --- lsuv2_oe.v02 --- lsuv2_sm.v00 --- native_m.v00 --- qlnative.v00 --- vdfs.v00 --- vmware_e.v00 --- vsan.v00 --- vsanheal.v00 --- vsanmgmt.v00 --- tools.t00 --- xorg.v00 --- gc.v00 --- imgdb.tgz --- basemisc.tgz --- resvibs.tgz --- imgpayld.tgz
build=7.0.2-0.0.17867351
updated=0

到这一步可以实现通过PXE网络安装ESXi

5、配置web 服务器

#默认网站目录/var/www/html

systemctl enable httpd && systemctl start httpd

########防火墙设置
firewall-cmd --zone=public --add-port=80/tcp --permanent
firewall-cmd --zone=public --add-port=443/tcp --permanent
#重新载入配置
firewall-cmd --reload
firewall-cmd --list-all #查看防火墙规则,只显示/etc/firewalld/zones/public.xml中防火墙策略

6、配置 ESXi 7 的 kickstart

示例 kickstart 文件存储在 ESXi 的 /etc/vmware/weasel/ks.cfg 中。将此复制到 http 服务器上的 /var/www/html

#示例 1 kickstart 文件,给ESXi 7 配置 静态IP地址
vi /var/www/html/ks.cfg
#####################
# Accept the VMware End User License Agreement
vmaccepteula
 
# Set the root password for the DCUI and Tech Support Mode
rootpw P@ssw0rd
 
# The install media is in the CD-ROM drive
install --firstdisk --overwritevmfs
 
# Set the network on the first network adapter
network --bootproto=static --device=vmnic0 --ip=10.53.220.199 --netmask=255.255.255.0 --vlanid=0 --gateway=10.53.220.1 --hostname=10.53.220.199 --nameserver=10.1.0.1

#vmserialnum
vmserialnum --esx=5U4TK-DML1M-M8550-XK1QP-1A052
# reboot after install
reboot
 
# run the following command only on the firstboot
%firstboot --interpreter=busybox

sleep 10
# enable & start remote ESXi Shell (SSH)
vim-cmd hostsvc/enable_ssh
vim-cmd hostsvc/start_ssh
sleep 1
# enable & start ESXi Shell (TSM)
vim-cmd hostsvc/enable_esx_shell
vim-cmd hostsvc/start_esx_shell
sleep 1
# enable High Performance
esxcli system settings advanced set --option=/Power/CpuPolicy --string-value="High Performance"
sleep 1
#Disable ipv6
esxcli network ip set --ipv6-enabled=0
sleep 1
#不加入体验
esxcli system settings advanced set -o /UserVars/HostClientCEIPOptIn -i 2
sleep 1
#enable ntp
/bin/esxcli system ntp set --server=ntp.aliyun.com
sleep 1
/bin/chkconfig ntpd on
sleep 1
#Disable ShellWarning
esxcli system settings advanced set -o /UserVars/SuppressShellWarning -i 1
sleep 1
cat << EOF > /etc/ssh/keys-root/authorized_keys
ssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAgEA6PP8/RDdYIqUq6DE3zj9Qs8AF3uzfYH5lYrB+mMxxfI8kq2NIzQlsW1KDaH/fWYTkkK120lPUu97lWic9Li3Z3iFR6Nh4q6PVTfBNt4xOx754Ipqtpefk+9sLZAYEPK8pnRP0QZv7CtDFv842tfUYIrVNnecRQTFfNtnGDcXnO1u2RE1kq6Tr5N3595PbPLDKczjOFnS+jy0MeKKHPJcZfYz7TUTSzTwTHYbPRRaQ8/0eihUwzpAmXRo9NYNle26qp6+SlRsjGSBcUr0rh0wSe6r/C2btnpOUd//aFvcl8plhyb++nivlhB71v+6I0UcPoSXOIVs/1QuHEMbv7Ircjb2emqHtZDpk8KSYhgV0ZdbAq9XOcux76eok//xtjbleKPAcDMY/KotIEh7QX4NLQxSJOm5gCLkn5kbrHfVx6nWlGzVVds1sDlcnSAWul5lFiI5ZkArXFKcnm+aCnStPpx5SSCpZpMUdZvt8ZA7vLx3xjMDFwv5HTuTFwB9mlZrdfqp5USC2mWC3eAAPE7GxDSfJv9epteIYywIP9NVT3Z4ng9z6jrcFfA4GFlfLrk8J71cnxZ/AWZXXUwp3ooE/Cp3jc473VpK7FZwjQ7Xz9PD8WQgMOO4xnGQhPWlxhRuoTYyQVa0xOBO9gh9Cuc6zq5FQgYQEcSB+/FBJ/YNDIc= www.zhangfangzhou.cn
EOF
sleep 1
date > /finished.stamp

# Restart a last time
reboot


#示例 2 kickstart 文件,ESXi 7 通过 DHCP 服务器获得IP地址
​vi /var/www/html/ks.cfg

# Accept the VMware End User License Agreement
vmaccepteula
 
# Set the root password for the DCUI and Tech Support Mode
rootpw P@ssw0rd
 
# The install media is in the CD-ROM drive
install --firstdisk --overwritevmfs
 
# Set the network on the first network adapter
network --bootproto=dhcp

#vmserialnum
vmserialnum --esx=5U4TK-DML1M-M8550-XK1QP-1A052
# reboot after install
reboot
 
# run the following command only on the firstboot
%firstboot --interpreter=busybox

sleep 10
# enable & start remote ESXi Shell (SSH)
vim-cmd hostsvc/enable_ssh
vim-cmd hostsvc/start_ssh
sleep 1
# enable & start ESXi Shell (TSM)
vim-cmd hostsvc/enable_esx_shell
vim-cmd hostsvc/start_esx_shell
sleep 1
# enable High Performance
esxcli system settings advanced set --option=/Power/CpuPolicy --string-value="High Performance"
sleep 1
#Disable ipv6
esxcli network ip set --ipv6-enabled=0
sleep 1
#不加入体验
esxcli system settings advanced set -o /UserVars/HostClientCEIPOptIn -i 2
sleep 1
#enable ntp
/bin/esxcli system ntp set --server=ntp.aliyun.com
sleep 1
/bin/chkconfig ntpd on
sleep 1
#Disable ShellWarning
esxcli system settings advanced set -o /UserVars/SuppressShellWarning -i 1
sleep 1
cat << EOF > /etc/ssh/keys-root/authorized_keys
ssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAgEA6PP8/RDdYIqUq6DE3zj9Qs8AF3uzfYH5lYrB+mMxxfI8kq2NIzQlsW1KDaH/fWYTkkK120lPUu97lWic9Li3Z3iFR6Nh4q6PVTfBNt4xOx754Ipqtpefk+9sLZAYEPK8pnRP0QZv7CtDFv842tfUYIrVNnecRQTFfNtnGDcXnO1u2RE1kq6Tr5N3595PbPLDKczjOFnS+jy0MeKKHPJcZfYz7TUTSzTwTHYbPRRaQ8/0eihUwzpAmXRo9NYNle26qp6+SlRsjGSBcUr0rh0wSe6r/C2btnpOUd//aFvcl8plhyb++nivlhB71v+6I0UcPoSXOIVs/1QuHEMbv7Ircjb2emqHtZDpk8KSYhgV0ZdbAq9XOcux76eok//xtjbleKPAcDMY/KotIEh7QX4NLQxSJOm5gCLkn5kbrHfVx6nWlGzVVds1sDlcnSAWul5lFiI5ZkArXFKcnm+aCnStPpx5SSCpZpMUdZvt8ZA7vLx3xjMDFwv5HTuTFwB9mlZrdfqp5USC2mWC3eAAPE7GxDSfJv9epteIYywIP9NVT3Z4ng9z6jrcFfA4GFlfLrk8J71cnxZ/AWZXXUwp3ooE/Cp3jc473VpK7FZwjQ7Xz9PD8WQgMOO4xnGQhPWlxhRuoTYyQVa0xOBO9gh9Cuc6zq5FQgYQEcSB+/FBJ/YNDIc= www.zhangfangzhou.cn
EOF
sleep 1
date > /finished.stamp

# Restart a last time
reboot

7、最终boot.cfg配置文件

# vi /var/lib/tftpboot/boot.cfg
bootstate=0
title=Loading ESXi installer www.zhangfangzhou.cn
timeout=5
prefix=ESXi70u2
kernel=b.b00
kernelopt=ks=http://10.53.220.224/ks.cfg
modules=jumpstrt.gz --- useropts.gz --- features.gz --- k.b00 --- uc_intel.b00 --- uc_amd.b00 --- uc_hygon.b00 --- procfs.b00 --- vmx.v00 --- vim.v00 --- tpm.v00 --- sb.v00 --- s.v00 --- atlantic.v00 --- bnxtnet.v00 --- bnxtroce.v00 --- brcmfcoe.v00 --- brcmnvme.v00 --- elxiscsi.v00 --- elxnet.v00 --- i40enu.v00 --- iavmd.v00 --- icen.v00 --- igbn.v00 --- irdman.v00 --- iser.v00 --- ixgben.v00 --- lpfc.v00 --- lpnic.v00 --- lsi_mr3.v00 --- lsi_msgp.v00 --- lsi_msgp.v01 --- lsi_msgp.v02 --- mtip32xx.v00 --- ne1000.v00 --- nenic.v00 --- nfnic.v00 --- nhpsa.v00 --- nmlx4_co.v00 --- nmlx4_en.v00 --- nmlx4_rd.v00 --- nmlx5_co.v00 --- nmlx5_rd.v00 --- ntg3.v00 --- nvme_pci.v00 --- nvmerdma.v00 --- nvmxnet3.v00 --- nvmxnet3.v01 --- pvscsi.v00 --- qcnic.v00 --- qedentv.v00 --- qedrntv.v00 --- qfle3.v00 --- qfle3f.v00 --- qfle3i.v00 --- qflge.v00 --- rste.v00 --- sfvmk.v00 --- smartpqi.v00 --- vmkata.v00 --- vmkfcoe.v00 --- vmkusb.v00 --- vmw_ahci.v00 --- clusters.v00 --- crx.v00 --- elx_esx_.v00 --- btldr.v00 --- esx_dvfi.v00 --- esx_ui.v00 --- esxupdt.v00 --- tpmesxup.v00 --- weaselin.v00 --- loadesx.v00 --- lsuv2_hp.v00 --- lsuv2_in.v00 --- lsuv2_ls.v00 --- lsuv2_nv.v00 --- lsuv2_oe.v00 --- lsuv2_oe.v01 --- lsuv2_oe.v02 --- lsuv2_sm.v00 --- native_m.v00 --- qlnative.v00 --- vdfs.v00 --- vmware_e.v00 --- vsan.v00 --- vsanheal.v00 --- vsanmgmt.v00 --- tools.t00 --- xorg.v00 --- gc.v00 --- imgdb.tgz --- basemisc.tgz --- resvibs.tgz --- imgpayld.tgz
build=7.0.2-0.0.17867351
updated=0

到这里可以基于DHCP实现通过PXE自动安装 ESXi 7、自动设置系统密码、自动接受许可、自动添加产品密钥、自动启用SSH服务、自动启用shell服务、电源模式自动设置为高性能、禁用IPV6、不参加(CEIP)客户体验改善计划、自动配置NTP、自动添加私钥。

https://www.zhangfangzhou.cn/pxe-kickstart-install-uefi-esxi-7.html

ESXi7u1设置NVIDIA GEFORCE RTX 2080TI显卡直通(passthrough)

ESXi7u1设置NVIDIA GEFORCE RTX 2080TI 显卡直通(passthrough),ESXi7.x设置直通NVIDIA GEFORCE RTX 2080TI 显卡,ESXi7.x设置直通 英伟达 RTX 2080TI 显卡,ESXi7.x直通N卡设置。

a、进入到ESXI管理、硬件、PCI设备里面把显卡硬件切换直通(在搜索框里输入需要直通的设备型号)也可以SSH登录到ESXi,把显卡的PCI直通设备添加进去

vi /etc/vmware/esx.conf
/device/显卡设备号/owner = “passthru”
/device/0000:01:00:0/owner = "passthru"
/device/0000:01:00:1/owner = "passthru"


b、创建虚拟机的时候内存需要**设置预留资源**或者**预留所有客户机内存(全部锁定)**,否则会提示失败 - 内存设置无效: 内存预留 (sched.mem.min) 应该等于内存大小 (16384)。

c、创建虚拟机的时候添加PCI 设备

d、编辑虚拟机选项、高级、配置参数,添加如下参数

hypervisor.cpuid.v0 = "FALSE" #hypervisor.cpuid.v0 FALSE 设置成FALSE的意义为欺骗虚拟Windows系统没有运行在虚拟机中
mce.enable = "TRUE" #虚拟化嵌套,启用了嵌套硬件辅助的虚拟化时,无法添加 PCI 直通设备

#屏蔽检测tools
isolation.tools.setVersion.disable = "TRUE"
isolation.tools.getVersion.disable = "TRUE"

#屏蔽检测是否为虚拟机(可以过网易mumu,无法过腾讯 游戏无法运行在虚拟环境中,请更换设备后重试)
monitor_control.restrict_backdoor = "TRUE"
disable_acceleration = "TRUE"



e、如果安装完成vm-tools,需要禁用掉VMware svga 3D,不然每次重启都需要重新安装显卡驱动才能使用显卡,否则设备管理器中显卡提示由于该设备有问题,Windows 已将其停止。 (代码 43)

f、启用了嵌套硬件辅助的虚拟化时,无法添加 PCI 直通设备


U盘安装VMware ESXi 6.7.0.update03的时候出错提示menu.c32 not a com32r image

用U盘给一台电脑安装VMware ESXi 6.7.0.update03的时候出错提示menu.c32 not a com32r image
menu.c32: not a COM32R image
boot:

解决方法1
boot: install
在boot:后面输入install 然后按回车就可以继续安装

解决方法2
电脑支持UEFI BOOT的时候,可以选择UEFI模式从U盘启动,就可以顺利安装VMware ESXi 6.7.0.update03