반응형
Backup Server의 Infiniband Install 방법
Test OS : OEL 7.9
ifconfig 명령어 수행 시 ib0와 ib1이 잡혀 있다. ibstat로 상태를 확인 하려면 command가 없다고 나온다.
[root@node2 ~]# ifconfig
ib0: flags=4099<UP,BROADCAST,MULTICAST> mtu 2044
Infiniband hardware address can be incorrect! Please read BUGS section in ifconfig(8).
infiniband 80:00:02:08:FE:80:00:00:00:00:00:00:00:00:00:00:00:00:00:00 txqueuelen 256 (InfiniBand)
RX packets 0 bytes 0 (0.0 B)
RX errors 0 dropped 0 overruns 0 frame 0
TX packets 0 bytes 0 (0.0 B)
TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0
ib1: flags=4099<UP,BROADCAST,MULTICAST> mtu 2044
Infiniband hardware address can be incorrect! Please read BUGS section in ifconfig(8).
infiniband 80:00:02:09:FE:80:00:00:00:00:00:00:00:00:00:00:00:00:00:00 txqueuelen 256 (InfiniBand)
RX packets 0 bytes 0 (0.0 B)
RX errors 0 dropped 0 overruns 0 frame 0
TX packets 0 bytes 0 (0.0 B)
TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0
[root@node2 ~]# ibstat
bash: ibstat: command not found...
groupinstall 명령어로 infiniband 관련 rpm을 깔아 준다.
[root@node2 /]# rpm -qa | grep infini*
[root@node2 /]# yum groupinstall "Infiniband Support"
Loaded plugins: langpacks, ulninfo
There is no installed groups file.
Maybe run: yum groups mark convert (see man yum)
ol7_kvm_utils | 3.6 kB 00:00:00
Resolving Dependencies
--> Running transaction check
---> Package dapl.x86_64 0:2.1.5-3.el7 will be installed
---> Package glusterfs-rdma.x86_64 0:6.0-37.el7 will be installed
---> Package ibacm.x86_64 0:22.4-5.el7 will be installed
---> Package ibutils.x86_64 0:1.5.7-14.el7 will be installed
--> Processing Dependency: ibutils-libs = 1.5.7-14.el7 for package: ibutils-1.5.7-14.el7.x86_64
--> Processing Dependency: tk for package: ibutils-1.5.7-14.el7.x86_64
--> Processing Dependency: tcl for package: ibutils-1.5.7-14.el7.x86_64
--> Processing Dependency: swig for package: ibutils-1.5.7-14.el7.x86_64
--> Processing Dependency: libosmvendor.so.4(OSMVENDOR_2.0)(64bit) for package: ibutils-1.5.7-14.el7.x86_64
--> Processing Dependency: libosmcomp.so.4(OSMCOMP_2.3)(64bit) for package: ibutils-1.5.7-14.el7.x86_64
--> Processing Dependency: libopensm.so.8(OPENSM_1.5)(64bit) for package: ibutils-1.5.7-14.el7.x86_64
--> Processing Dependency: graphviz-tcl for package: ibutils-1.5.7-14.el7.x86_64
--> Processing Dependency: libtcl8.5.so()(64bit) for package: ibutils-1.5.7-14.el7.x86_64
--> Processing Dependency: libosmvendor.so.4()(64bit) for package: ibutils-1.5.7-14.el7.x86_64
--> Processing Dependency: libosmcomp.so.4()(64bit) for package: ibutils-1.5.7-14.el7.x86_64
--> Processing Dependency: libopensm.so.8()(64bit) for package: ibutils-1.5.7-14.el7.x86_64
--> Processing Dependency: libibdmcom.so.1()(64bit) for package: ibutils-1.5.7-14.el7.x86_64
---> Package infiniband-diags.x86_64 0:2.1.0-1.el7 will be installed
---> Package iwpmd.x86_64 0:22.4-5.el7 will be installed
---> Package libibumad.x86_64 0:22.4-5.el7 will be installed
---> Package libibverbs-utils.x86_64 0:22.4-5.el7 will be installed
---> Package librdmacm-utils.x86_64 0:22.4-5.el7 will be installed
---> Package mstflint.x86_64 0:4.13.3-2.el7 will be installed
--> Processing Dependency: python3 for package: mstflint-4.13.3-2.el7.x86_64
--> Processing Dependency: libboost_regex.so.1.53.0()(64bit) for package: mstflint-4.13.3-2.el7.x86_64
--> Processing Dependency: libboost_filesystem.so.1.53.0()(64bit) for package: mstflint-4.13.3-2.el7.x86_64
---> Package opa-address-resolution.x86_64 1:10.9.0.0.204-1.0.1.el7 will be installed
--> Processing Dependency: opa-basic-tools(x86-64) for package: 1:opa-address-resolution-10.9.0.0.204-1.0.1.el7.x86_64
---> Package opa-fastfabric.x86_64 1:10.9.0.0.204-1.0.1.el7 will be installed
--> Processing Dependency: /usr/bin/expect for package: 1:opa-fastfabric-10.9.0.0.204-1.0.1.el7.x86_64
---> Package opa-libopamgt.x86_64 1:10.9.0.0.204-1.0.1.el7 will be installed
---> Package perftest.x86_64 0:4.2-2.el7 will be installed
---> Package qperf.x86_64 0:0.4.9-3.el7 will be installed
---> Package srp_daemon.x86_64 0:22.4-5.el7 will be installed
--> Running transaction check
---> Package boost-filesystem.x86_64 0:1.53.0-28.el7 will be installed
---> Package boost-regex.x86_64 0:1.53.0-28.el7 will be installed
---> Package expect.x86_64 0:5.45-14.el7_1 will be installed
---> Package graphviz-tcl.x86_64 0:2.30.1-22.el7 will be installed
--> Processing Dependency: graphviz = 2.30.1-22.el7 for package: graphviz-tcl-2.30.1-22.el7.x86_64
--> Processing Dependency: libxdot.so.4()(64bit) for package: graphviz-tcl-2.30.1-22.el7.x86_64
--> Processing Dependency: libpathplan.so.4()(64bit) for package: graphviz-tcl-2.30.1-22.el7.x86_64
--> Processing Dependency: libgvc.so.6()(64bit) for package: graphviz-tcl-2.30.1-22.el7.x86_64
--> Processing Dependency: libcgraph.so.6()(64bit) for package: graphviz-tcl-2.30.1-22.el7.x86_64
--> Processing Dependency: libcdt.so.5()(64bit) for package: graphviz-tcl-2.30.1-22.el7.x86_64
---> Package ibutils-libs.x86_64 0:1.5.7-14.el7 will be installed
---> Package opa-basic-tools.x86_64 1:10.9.0.0.204-1.0.1.el7 will be installed
---> Package opensm-libs.x86_64 0:3.3.21-3.el7 will be installed
---> Package python3.x86_64 0:3.6.8-17.0.1.el7 will be installed
--> Processing Dependency: python3-libs(x86-64) = 3.6.8-17.0.1.el7 for package: python3-3.6.8-17.0.1.el7.x86_64
--> Processing Dependency: python3-setuptools for package: python3-3.6.8-17.0.1.el7.x86_64
--> Processing Dependency: python3-pip for package: python3-3.6.8-17.0.1.el7.x86_64
--> Processing Dependency: libpython3.6m.so.1.0()(64bit) for package: python3-3.6.8-17.0.1.el7.x86_64
---> Package swig.x86_64 0:2.0.10-5.el7 will be installed
---> Package tcl.x86_64 1:8.5.13-8.el7 will be installed
---> Package tk.x86_64 1:8.5.13-6.el7 will be installed
--> Running transaction check
---> Package graphviz.x86_64 0:2.30.1-22.el7 will be installed
--> Processing Dependency: libXaw.so.7()(64bit) for package: graphviz-2.30.1-22.el7.x86_64
---> Package python3-libs.x86_64 0:3.6.8-17.0.1.el7 will be installed
---> Package python3-pip.noarch 0:9.0.3-8.el7 will be installed
---> Package python3-setuptools.noarch 0:39.2.0-10.el7 will be installed
--> Running transaction check
---> Package libXaw.x86_64 0:1.0.13-4.el7 will be installed
--> Finished Dependency Resolution
Dependencies Resolved
====================================================================================================================================================
Package Arch Version Repository Size
====================================================================================================================================================
Installing for group install "Infiniband Support":
dapl x86_64 2.1.5-3.el7 ol7_kvm_utils 222 k
ibacm x86_64 22.4-5.el7 ol7_kvm_utils 82 k
ibutils x86_64 1.5.7-14.el7 ol7_kvm_utils 1.1 M
infiniband-diags x86_64 2.1.0-1.el7 ol7_kvm_utils 306 k
iwpmd x86_64 22.4-5.el7 ol7_kvm_utils 27 k
libibumad x86_64 22.4-5.el7 ol7_kvm_utils 24 k
libibverbs-utils x86_64 22.4-5.el7 ol7_kvm_utils 61 k
librdmacm-utils x86_64 22.4-5.el7 ol7_kvm_utils 84 k
mstflint x86_64 4.13.3-2.el7 ol7_kvm_utils 3.4 M
opa-address-resolution x86_64 1:10.9.0.0.204-1.0.1.el7 ol7_kvm_utils 338 k
opa-fastfabric x86_64 1:10.9.0.0.204-1.0.1.el7 ol7_kvm_utils 1.6 M
opa-libopamgt x86_64 1:10.9.0.0.204-1.0.1.el7 ol7_kvm_utils 106 k
perftest x86_64 4.2-2.el7 ol7_kvm_utils 205 k
qperf x86_64 0.4.9-3.el7 ol7_kvm_utils 58 k
srp_daemon x86_64 22.4-5.el7 ol7_kvm_utils 42 k
Installing:
glusterfs-rdma x86_64 6.0-37.el7 ol7_kvm_utils 41 k
Installing for dependencies:
boost-filesystem x86_64 1.53.0-28.el7 ol7_kvm_utils 68 k
boost-regex x86_64 1.53.0-28.el7 ol7_kvm_utils 295 k
expect x86_64 5.45-14.el7_1 ol7_kvm_utils 262 k
graphviz x86_64 2.30.1-22.el7 ol7_kvm_utils 1.3 M
graphviz-tcl x86_64 2.30.1-22.el7 ol7_kvm_utils 706 k
ibutils-libs x86_64 1.5.7-14.el7 ol7_kvm_utils 690 k
libXaw x86_64 1.0.13-4.el7 ol7_kvm_utils 191 k
opa-basic-tools x86_64 1:10.9.0.0.204-1.0.1.el7 ol7_kvm_utils 406 k
opensm-libs x86_64 3.3.21-3.el7 ol7_kvm_utils 68 k
python3 x86_64 3.6.8-17.0.1.el7 ol7_kvm_utils 70 k
python3-libs x86_64 3.6.8-17.0.1.el7 ol7_kvm_utils 6.9 M
python3-pip noarch 9.0.3-8.el7 ol7_kvm_utils 1.6 M
python3-setuptools noarch 39.2.0-10.el7 ol7_kvm_utils 628 k
swig x86_64 2.0.10-5.el7 ol7_kvm_utils 1.3 M
tcl x86_64 1:8.5.13-8.el7 ol7_kvm_utils 1.9 M
tk x86_64 1:8.5.13-6.el7 ol7_kvm_utils 1.4 M
Transaction Summary
====================================================================================================================================================
Install 16 Packages (+16 Dependent packages)
Total download size: 25 M
Installed size: 112 M
Is this ok [y/d/N]: y
Downloading packages:
----------------------------------------------------------------------------------------------------------------------------------------------------
Total 239 MB/s | 25 MB 00:00:00
Running transaction check
Running transaction test
Transaction test succeeded
Running transaction
Installing : libibumad-22.4-5.el7.x86_64 1/32
Installing : 1:tcl-8.5.13-8.el7.x86_64 2/32
Installing : opensm-libs-3.3.21-3.el7.x86_64 3/32
Installing : 1:tk-8.5.13-6.el7.x86_64 4/32
Installing : 1:opa-basic-tools-10.9.0.0.204-1.0.1.el7.x86_64 5/32
Installing : ibutils-libs-1.5.7-14.el7.x86_64 6/32
Installing : expect-5.45-14.el7_1.x86_64 7/32
Installing : ibacm-22.4-5.el7.x86_64 8/32
Installing : python3-libs-3.6.8-17.0.1.el7.x86_64 9/32
Installing : python3-setuptools-39.2.0-10.el7.noarch 10/32
Installing : python3-3.6.8-17.0.1.el7.x86_64 11/32
Installing : python3-pip-9.0.3-8.el7.noarch 12/32
Installing : boost-filesystem-1.53.0-28.el7.x86_64 13/32
Installing : boost-regex-1.53.0-28.el7.x86_64 14/32
Installing : swig-2.0.10-5.el7.x86_64 15/32
Installing : libXaw-1.0.13-4.el7.x86_64 16/32
Installing : graphviz-2.30.1-22.el7.x86_64 17/32
Installing : graphviz-tcl-2.30.1-22.el7.x86_64 18/32
Installing : ibutils-1.5.7-14.el7.x86_64 19/32
Installing : mstflint-4.13.3-2.el7.x86_64 20/32
Installing : 1:opa-address-resolution-10.9.0.0.204-1.0.1.el7.x86_64 21/32
Installing : 1:opa-fastfabric-10.9.0.0.204-1.0.1.el7.x86_64 22/32
Installing : infiniband-diags-2.1.0-1.el7.x86_64 23/32
Installing : srp_daemon-22.4-5.el7.x86_64 24/32
Installing : perftest-4.2-2.el7.x86_64 25/32
Installing : 1:opa-libopamgt-10.9.0.0.204-1.0.1.el7.x86_64 26/32
Installing : qperf-0.4.9-3.el7.x86_64 27/32
Installing : libibverbs-utils-22.4-5.el7.x86_64 28/32
Installing : dapl-2.1.5-3.el7.x86_64 29/32
Installing : glusterfs-rdma-6.0-37.el7.x86_64 30/32
Installing : librdmacm-utils-22.4-5.el7.x86_64 31/32
Installing : iwpmd-22.4-5.el7.x86_64 32/32
Verifying : iwpmd-22.4-5.el7.x86_64 1/32
Verifying : 1:tcl-8.5.13-8.el7.x86_64 2/32
Verifying : srp_daemon-22.4-5.el7.x86_64 3/32
Verifying : opensm-libs-3.3.21-3.el7.x86_64 4/32
Verifying : python3-pip-9.0.3-8.el7.noarch 5/32
Verifying : python3-libs-3.6.8-17.0.1.el7.x86_64 6/32
Verifying : librdmacm-utils-22.4-5.el7.x86_64 7/32
Verifying : ibutils-1.5.7-14.el7.x86_64 8/32
Verifying : python3-3.6.8-17.0.1.el7.x86_64 9/32
Verifying : expect-5.45-14.el7_1.x86_64 10/32
Verifying : infiniband-diags-2.1.0-1.el7.x86_64 11/32
Verifying : libXaw-1.0.13-4.el7.x86_64 12/32
Verifying : mstflint-4.13.3-2.el7.x86_64 13/32
Verifying : 1:tk-8.5.13-6.el7.x86_64 14/32
Verifying : glusterfs-rdma-6.0-37.el7.x86_64 15/32
Verifying : dapl-2.1.5-3.el7.x86_64 16/32
Verifying : swig-2.0.10-5.el7.x86_64 17/32
Verifying : perftest-4.2-2.el7.x86_64 18/32
Verifying : graphviz-tcl-2.30.1-22.el7.x86_64 19/32
Verifying : ibacm-22.4-5.el7.x86_64 20/32
Verifying : ibutils-libs-1.5.7-14.el7.x86_64 21/32
Verifying : libibverbs-utils-22.4-5.el7.x86_64 22/32
Verifying : python3-setuptools-39.2.0-10.el7.noarch 23/32
Verifying : boost-regex-1.53.0-28.el7.x86_64 24/32
Verifying : qperf-0.4.9-3.el7.x86_64 25/32
Verifying : 1:opa-basic-tools-10.9.0.0.204-1.0.1.el7.x86_64 26/32
Verifying : 1:opa-fastfabric-10.9.0.0.204-1.0.1.el7.x86_64 27/32
Verifying : libibumad-22.4-5.el7.x86_64 28/32
Verifying : 1:opa-libopamgt-10.9.0.0.204-1.0.1.el7.x86_64 29/32
Verifying : 1:opa-address-resolution-10.9.0.0.204-1.0.1.el7.x86_64 30/32
Verifying : graphviz-2.30.1-22.el7.x86_64 31/32
Verifying : boost-filesystem-1.53.0-28.el7.x86_64 32/32
Installed:
dapl.x86_64 0:2.1.5-3.el7 glusterfs-rdma.x86_64 0:6.0-37.el7
ibacm.x86_64 0:22.4-5.el7 ibutils.x86_64 0:1.5.7-14.el7
infiniband-diags.x86_64 0:2.1.0-1.el7 iwpmd.x86_64 0:22.4-5.el7
libibumad.x86_64 0:22.4-5.el7 libibverbs-utils.x86_64 0:22.4-5.el7
librdmacm-utils.x86_64 0:22.4-5.el7 mstflint.x86_64 0:4.13.3-2.el7
opa-address-resolution.x86_64 1:10.9.0.0.204-1.0.1.el7 opa-fastfabric.x86_64 1:10.9.0.0.204-1.0.1.el7
opa-libopamgt.x86_64 1:10.9.0.0.204-1.0.1.el7 perftest.x86_64 0:4.2-2.el7
qperf.x86_64 0:0.4.9-3.el7 srp_daemon.x86_64 0:22.4-5.el7
Dependency Installed:
boost-filesystem.x86_64 0:1.53.0-28.el7 boost-regex.x86_64 0:1.53.0-28.el7 expect.x86_64 0:5.45-14.el7_1
graphviz.x86_64 0:2.30.1-22.el7 graphviz-tcl.x86_64 0:2.30.1-22.el7 ibutils-libs.x86_64 0:1.5.7-14.el7
libXaw.x86_64 0:1.0.13-4.el7 opa-basic-tools.x86_64 1:10.9.0.0.204-1.0.1.el7 opensm-libs.x86_64 0:3.3.21-3.el7
python3.x86_64 0:3.6.8-17.0.1.el7 python3-libs.x86_64 0:3.6.8-17.0.1.el7 python3-pip.noarch 0:9.0.3-8.el7
python3-setuptools.noarch 0:39.2.0-10.el7 swig.x86_64 0:2.0.10-5.el7 tcl.x86_64 1:8.5.13-8.el7
tk.x86_64 1:8.5.13-6.el7
Complete!
[root@node2 /]#
[root@node2 /]#
[root@node2 /]#
[root@node2 /]# rpm -qa | grep infini
infiniband-diags-2.1.0-1.el7.x86_64
[root@node2 /]#
[root@node2 /]#
[root@node2 /]# ibstat
CA 'mlx4_0'
CA type: MT4099
Number of ports: 2
Firmware version: 2.11.1280
Hardware version: 1
Node GUID: 0x0010e00001594228
System image GUID: 0x0010e0000159422b
Port 1:
State: Down
Physical state: Disabled
Rate: 10
Base lid: 0
LMC: 0
SM lid: 0
Capability mask: 0x02594868
Port GUID: 0x0010e00001594229
Link layer: InfiniBand
Port 2:
State: Down
Physical state: Disabled
Rate: 10
Base lid: 0
LMC: 0
SM lid: 0
Capability mask: 0x02594868
Port GUID: 0x0010e0000159422a
Link layer: InfiniBand설치된 rpm 을 확인 할 수 있고, 종속성관련 rpm도 설치가 되었다.
ibstat 명령어가 제대로 수행되는 걸 확인 할 수 있다.
RDMA를 실행을 확인 하고 부팅시 자동으로 시작 되게 설정 한다.
RAMA란?
RDMA(Remote Direct Memory Access)란 컴퓨터 간에 CPU를 거치지 않고 메모리끼리 직접 데이터를 주고받을 수 있게 하는 네트워크 기술을 말합니다. 전송 주체(initiator)에서 대상(target)으로 데이터를 전송할 시, 데이터의 임시복사 과정(NIC에서 애플리케이션 버퍼까지의 복사 과정) 없이 바로 전송을 할 수 있습니다. 이로써 빠른 데이터 처리 속도와 낮은 네트워크 지연시간을 구현할 수 있어, 데이터 센터나 고성능 컴퓨터와 같이 대규모 데이터를 동시에 전송하는 환경에서 활용됩니다. RDMA는 iSCSI나 이더넷의 데이터 경로를 보완해 네트워크와 스토리지의 활용성을 높이기도 합니다.
[root@node2 /]# systemctl status rdma
rdma.service - Initialize the iWARP/InfiniBand/RDMA stack in the kernel
Loaded: loaded (/usr/lib/systemd/system/rdma.service; disabled; vendor preset: disabled)
Active: active (exited) since Thu 2021-09-30 18:12:31 KST; 2 months 13 days ago
Docs: file:/etc/rdma/rdma.conf
Main PID: 950 (code=exited, status=0/SUCCESS)
CGroup: /system.slice/_rdma.service
Sep 30 18:12:30 node2 systemd[1]: Starting Initialize the iWARP/InfiniBand/RDMA stack in the kernel...
Sep 30 18:12:31 node2 systemd[1]: Started Initialize the iWARP/InfiniBand/RDMA stack in the kernel.
[root@node2 /]#
[root@node2 /]# systemctl enable rdma
Created symlink from /etc/systemd/system/sysinit.target.wants/rdma.service to /usr/lib/systemd/system/rdma.service.
[root@node2 /]#
infiniband의 ip설정은 eth0번 ip설정과 동일하다.
기본폼은 아래와 같다.
/etc/sysconfig/network-scripts/ifcfg-ib0
DEVICE=ib0
BOOTPROTO=static
IPADDR=XXX
NETMASK=XXX
NETWORK=XXX
BROADCAST=XXX
ONBOOT=yes
ib0과 ib1을 donding 구성시 현재는 Infiniband Card만 있고, 케이블은 미연결 상태 이다.
[root@em13c network-scripts]# cd /etc/sysconfig/network-scripts/
[root@em13c network-scripts]# cat ifcfg-ib0
DEVICE=ib0
USERCTL=no
ONBOOT=yes
MASTER=bondib0
SLAVE=yes
BOOTPROTO=none
HOTPLUG=no
IPV6INIT=no
[root@em13c network-scripts]# cat ifcfg-ib1
DEVICE=ib1
USERCTL=no
ONBOOT=yes
MASTER=bondib0
SLAVE=yes
BOOTPROTO=none
HOTPLUG=no
IPV6INIT=no
[root@em13c network-scripts]# cat ifcfg-bondib0
DEVICE=bondib0
BOOTPROTO=none
ONBOOT=yes
BONDING_OPTS="mode=active-backup miimon=100 downdelay=2000 updelay=5000 num_grat_arp=100"
IPV6INIT=no
IPADDR=30.30.30.200
NETMASK=255.255.252.0
GATEWAY=30.30.30.1
[root@em13c network-scripts]#
icmp6 ip6_mr_vif ip_tables_matches netfilter packet route snmp6 tcp6 wireless
dev_mcast if_inet6 ip6_tables_matches ip_tables_names netlink protocols rpc sockstat udp xfrm_stat
[root@em13c network-scripts]#
[root@em13c network-scripts]# cd /proc/net/bonding/
[root@em13c bonding]# cat bondib0
Ethernet Channel Bonding Driver: v3.7.1 (April 27, 2011)
Bonding Mode: fault-tolerance (active-backup) (fail_over_mac active)
Primary Slave: None
Currently Active Slave: None
MII Status: down
MII Polling Interval (ms): 100
Up Delay (ms): 5000
Down Delay (ms): 2000
Peer Notification Delay (ms): 0
Slave Interface: ib0
MII Status: down
Speed: Unknown
Duplex: Unknown
Link Failure Count: 0
Permanent HW addr: 80:00:02:08:fe:80:00:00:00:00:00:00:00:10:e0:00:01:48:f3:b9
Slave queue ID: 0
Slave Interface: ib1
MII Status: down
Speed: Unknown
Duplex: Unknown
Link Failure Count: 0
Permanent HW addr: 80:00:02:09:fe:80:00:00:00:00:00:00:00:10:e0:00:01:48:f3:ba
Slave queue ID: 0
[root@em13c bonding]#
반응형
'★━Oracle ExaData〃 > 1. ExaData HW' 카테고리의 다른 글
| Exadata X10M 소개: 최고의 확장성 및 획기적으로 향상된 가격 대비 성능 (0) | 2023.08.05 |
|---|---|
| [ExadataHW] AHF 설치 (0) | 2022.02.14 |
| Exadata Promiscuous Mode messages (0) | 2021.11.05 |
| [ExaData HW] Cellcli command line (0) | 2021.04.28 |
| [Exadata HW] Exadata 장애시 수집 로그 (0) | 2021.04.28 |
댓글