3. Network Devices on master

The master nodes have multiple network devices.

One will be used for internet access. On the Temple HPC Training cluster it is also the network interface which allows to access it.

Another port should be connected to the network for the management network used to control system with IPMI. Finally a third port should be connected to the cluster network which will be used for SSH, network booting and data transfer between compute nodes.

Note

Log in to the master node assigned to you

[student@training1 ~]$ ssh root@masterX
[root@master ~]#

The root password is mhpc2024.

3.1. Identify your available network devices

The ip link command will show you all available network devices and their state.

Note

The interfaces names are assigned automatically by systemd/udev v197 and older, and therefore they might be different in your system. For instance, em[1-4] could be eno[1-4]. For further information about the naming scheme, please follow this link.

[root@master ~]# ip link
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN mode DEFAULT group default qlen 1000
    link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
2: eno1: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc mq state UP mode DEFAULT group default qlen 1000
    link/ether 00:21:9b:9f:7c:df brd ff:ff:ff:ff:ff:ff
3: eno2: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc mq state UP mode DEFAULT group default qlen 1000
    link/ether 00:21:9b:9f:7c:e1 brd ff:ff:ff:ff:ff:ff
4: eno3: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc mq state UP mode DEFAULT group default qlen 1000
    link/ether 00:21:9b:9f:7c:e3 brd ff:ff:ff:ff:ff:ff
5: eno4: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc mq state UP mode DEFAULT group default qlen 1000
    link/ether 00:21:9b:9f:7c:e5 brd ff:ff:ff:ff:ff:ff

Danger

Do not change anything on eno2 on Temple HPC masters. This is the network connection that allows you to access it remotely! Use eno3 and eno4 for the management and cluster network.

Network	Temple	MHPC Lab
Internet	eno2	enp1s1f0
Management (IPMI)	eno3	enp1s1f1
Cluster (SSH)	eno4	enp1s2f0
Not Used	eno1	enp1s2f1

You can change the state of an interface with the ip link set command.

[root@master ~]# ip link set eno3 down
[root@master ~]# ip link set eno3 up

To get more details about your connection, you can use the ethtool command:

[root@master ~]# ethtool eno3
Settings for eno3:
    Supported ports: [ TP ]
    Supported link modes:   10baseT/Half 10baseT/Full
                            100baseT/Half 100baseT/Full
                            1000baseT/Full
    Supported pause frame use: Symmetric Receive-only
    Supports auto-negotiation: Yes
    Supported FEC modes: Not reported
    Advertised link modes:  10baseT/Half 10baseT/Full
                            100baseT/Half 100baseT/Full
                            1000baseT/Full
    Advertised pause frame use: No
    Advertised auto-negotiation: Yes
    Advertised FEC modes: Not reported
    Link partner advertised link modes:  10baseT/Half 10baseT/Full
                                         100baseT/Half 100baseT/Full
                                         1000baseT/Full
    Link partner advertised pause frame use: Symmetric
    Link partner advertised auto-negotiation: Yes
    Link partner advertised FEC modes: Not reported
    Speed: 1000Mb/s
    Duplex: Full
    Port: Twisted Pair
    PHYAD: 19
    Transceiver: internal
    Auto-negotiation: on
    MDI-X: Unknown
    Supports Wake-on: g
    Wake-on: d
    Current message level: 0x00000000 (0)

    Link detected: yes

It will report if it can find a physical link, what transfer rates it supports and its counter-part and what speed was negotiated in the end. In the above example you can see that the connection is running with 1000Mb/s or 1Gb/s.

Having a link does not give you an IP. You either have to request one via DHCP or set it statically.

Our first network device will be connected to the outside world and will get its IP via DHCP. The other two interfaces are connected to isolated networks, for which we will have to set static IPs.