Linux Storage Devices

In Linux we have filesystems (or virtual fs) which is at software layer (and controls how data is stored/ retrieved like xfs, ext4, gfs). Application processes reside in filesystem and interact (read/ write). Storage devices are shown as file in /dev/ directory. Types of Disks Interfaces  – SAS, SATA, Fiber Channel, SCSI, ATA (IDE) etc

InterfaceDescription
ATA/PATA (IDE)AT Attachment, Older form of interface, least expensive and is mostly replaced by SATA
SATASerial AT Attachment, As compared to ATA it has faster speed, reduced cabling, hot swappable, can be used with HDD, Solid State etc.
SCSISCSI has many forms but modern interfaces are serial in Nature like SAS (Serial Attached SCSI) and USB attached SCSI and SCSI over Fiber. Older form of communication was parallel in nature.
HBA/Fibre ChannelHigh Bandwidth adapters commonly used with SAN storage. Each HBA has WWN (World Wide Name) just like MAC address.

When you run df on Unix box you might see a bunch of filesystems like

  • /dev/sda1, /dev/sda2 etc…
  • /dev/xvda* for Xen virtual disks
  • /dev/hda* – for IDE controllers (you will see it in old machines)

/dev/sdx – where x can be ‘a’, ‘b’ etc. It represents raw devices. Like ‘a’ is for hard disk1, b is for hard disk 2 etc. sda1 represents the first partition in hard disk 1.

 

# lsblk
NAME        MAJ:MIN RM   SIZE RO TYPE MOUNTPOINT
sda           8:0    0 238.5G  0 disk 
├─sda1        8:1    0   200M  0 part /boot/efi
├─sda2        8:2    0     1G  0 part /boot
└─sda3        8:3    0 237.3G  0 part 
  ├─cl-root 253:0    0    50G  0 lvm  /
  ├─cl-swap 253:1    0   7.8G  0 lvm  [SWAP]
  └─cl-home 253:2    0 179.5G  0 lvm  /home

from above output you can see the top level disk (raw disk is sda, 3 char name), if there was additional disk it would have been sdb.

sda has 2 partitions (sda1 and sda2) and one lvm (sda3).

 

findmnt

very useful command to find filsystems and partitions. Below is an example from EC2 instance from AWS.

$findmnt
TARGET                                SOURCE         FSTYPE     OPTIONS
/                                     /dev/nvme0n1p1 xfs        rw,relatime,seclabel,attr2,inode64,noquota
├─/sys                                sysfs          sysfs      rw,nosuid,nodev,noexec,relatime,seclabel
│ ├─/sys/kernel/security              securityfs     securityfs rw,nosuid,nodev,noexec,relatime
│ ├─/sys/fs/cgroup                    tmpfs          tmpfs      ro,nosuid,nodev,noexec,seclabel,mode=755
│ │ ├─/sys/fs/cgroup/systemd          cgroup         cgroup     rw,nosuid,nodev,noexec,relatime,seclabel,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd
│ │ ├─/sys/fs/cgroup/devices          cgroup         cgroup     rw,nosuid,nodev,noexec,relatime,seclabel,devices
│ │ ├─/sys/fs/cgroup/cpuset           cgroup         cgroup     rw,nosuid,nodev,noexec,relatime,seclabel,cpuset
│ │ ├─/sys/fs/cgroup/perf_event       cgroup         cgroup     rw,nosuid,nodev,noexec,relatime,seclabel,perf_event
│ │ ├─/sys/fs/cgroup/net_cls,net_prio cgroup         cgroup     rw,nosuid,nodev,noexec,relatime,seclabel,net_prio,net_cls
│ │ ├─/sys/fs/cgroup/cpu,cpuacct      cgroup         cgroup     rw,nosuid,nodev,noexec,relatime,seclabel,cpuacct,cpu
│ │ ├─/sys/fs/cgroup/pids             cgroup         cgroup     rw,nosuid,nodev,noexec,relatime,seclabel,pids
│ │ ├─/sys/fs/cgroup/hugetlb          cgroup         cgroup     rw,nosuid,nodev,noexec,relatime,seclabel,hugetlb
│ │ ├─/sys/fs/cgroup/blkio            cgroup         cgroup     rw,nosuid,nodev,noexec,relatime,seclabel,blkio
│ │ ├─/sys/fs/cgroup/memory           cgroup         cgroup     rw,nosuid,nodev,noexec,relatime,seclabel,memory
│ │ └─/sys/fs/cgroup/freezer          cgroup         cgroup     rw,nosuid,nodev,noexec,relatime,seclabel,freezer
│ ├─/sys/fs/pstore                    pstore         pstore     rw,nosuid,nodev,noexec,relatime
│ ├─/sys/fs/selinux                   selinuxfs      selinuxfs  rw,relatime
│ ├─/sys/kernel/debug                 debugfs        debugfs    rw,relatime
│ └─/sys/kernel/config                configfs       configfs   rw,relatime
├─/proc                               proc           proc       rw,nosuid,nodev,noexec,relatime
│ └─/proc/sys/fs/binfmt_misc          systemd-1      autofs     rw,relatime,fd=28,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=27608
├─/dev                                devtmpfs       devtmpfs   rw,nosuid,seclabel,size=899316k,nr_inodes=224829,mode=755
│ ├─/dev/shm                          tmpfs          tmpfs      rw,nosuid,nodev,seclabel
│ ├─/dev/pts                          devpts         devpts     rw,nosuid,noexec,relatime,seclabel,gid=5,mode=620,ptmxmode=000
│ ├─/dev/mqueue                       mqueue         mqueue     rw,relatime,seclabel
│ └─/dev/hugepages                    hugetlbfs      hugetlbfs  rw,relatime,seclabel
├─/run                                tmpfs          tmpfs      rw,nosuid,nodev,seclabel,mode=755
│ └─/run/user/1001                    tmpfs          tmpfs      rw,nosuid,nodev,relatime,seclabel,size=184384k,mode=700,uid=1001,gid=1001
├─/var/lib/nfs/rpc_pipefs             rpc_pipefs     rpc_pipefs rw,relatime
├─/var/log                            tmpfs          tmpfs      rw,relatime,seclabel,size=3645440k
└─/mnt/www                            /dev/nvme1n1   ext4       rw,relatime,seclabel,data=ordered

 

How to check which disk is on which controller?

run lspci to check controllers

now go to /sys/block/ or /sys/block etc (for that particular disk)

# lspci
00:00.0 Host bridge: Intel Corporation Broadwell-U Host Bridge -OPI (rev 09)
00:02.0 VGA compatible controller: Intel Corporation HD Graphics 5500 (rev 09)
00:03.0 Audio device: Intel Corporation Broadwell-U Audio Controller (rev 09)
00:14.0 USB controller: Intel Corporation Wildcat Point-LP USB xHCI Controller (rev 03)
00:16.0 Communication controller: Intel Corporation Wildcat Point-LP MEI Controller #1 (rev 03)
00:1b.0 Audio device: Intel Corporation Wildcat Point-LP High Definition Audio Controller (rev 03)
00:1c.0 PCI bridge: Intel Corporation Wildcat Point-LP PCI Express Root Port #2 (rev e3)
00:1c.2 PCI bridge: Intel Corporation Wildcat Point-LP PCI Express Root Port #3 (rev e3)
00:1f.0 ISA bridge: Intel Corporation Wildcat Point-LP LPC Controller (rev 03)
00:1f.2 SATA controller: Intel Corporation Wildcat Point-LP SATA Controller [AHCI Mode] (rev 03)
00:1f.3 SMBus: Intel Corporation Wildcat Point-LP SMBus Controller (rev 03)
01:00.0 Unassigned class [ff00]: Realtek Semiconductor Co., Ltd. RTS5227 PCI Express Card Reader (rev 01)
02:00.0 Network controller: Intel Corporation Wireless 7265 (rev 50)

 

# pwd
/sys/block
[root@localhost block]# ls -rplt
total 0
lrwxrwxrwx. 1 root root 0 Dec 26 11:07 sda -> ../devices/pci0000:00/0000:00:1f.2/ata1/host0/target0:0:0/0:0:0:0/block/sda/
lrwxrwxrwx. 1 root root 0 Dec 26 11:07 dm-0 -> ../devices/virtual/block/dm-0/
lrwxrwxrwx. 1 root root 0 Dec 26 11:07 dm-1 -> ../devices/virtual/block/dm-1/
lrwxrwxrwx. 1 root root 0 Dec 26 11:07 dm-2 -> ../devices/virtual/block/dm-2/
[root@localhost block]#

from above you can see sda is using SATA controller (see the bus 00:1f.2).

SATA controller is a hardware interface which connects storage devices like SSF/Hardisk to Motherboard.

Generally, SATA controllers can be SET in three modes

  • IDE – Simplest
  • AHCI – Enables Hot Swapping
  • RAID – RAID + AHCI

lspci -v will give more output on PCI buses.

#lspci -v
00:1f.2 SATA controller: Intel Corporation Wildcat Point-LP SATA Controller [AHCI Mode] (rev 03) (prog-if 01 [AHCI 1.0])
  Subsystem: Hewlett-Packard Company Device 802d
  Flags: bus master, 66MHz, medium devsel, latency 0, IRQ 44
  I/O ports at 30b0 [size=8]
  I/O ports at 30a0 [size=4]
  I/O ports at 3090 [size=8]
  I/O ports at 3080 [size=4]
  I/O ports at 3060 [size=32]
  Memory at b2219000 (32-bit, non-prefetchable) [size=2K]
  Capabilities: [80] MSI: Enable+ Count=1/1 Maskable- 64bit-
  Capabilities: [70] Power Management version 3
  Capabilities: [a8] SATA HBA v1.0
  Kernel driver in use: ahci
  Kernel modules: ahci

lsscsi – list scsi devices and their information.

# lsscsi -v
[0:0:0:0]    disk    ATA      AXNS381E-256GM-B -HP   /dev/sda 
  dir: /sys/bus/scsi/devices/0:0:0:0  [/sys/devices/pci0000:00/0000:00:1f.2/ata1/host0/target0:0:0/0:0:0:0]
[root@localhost abhishekjdl]#

 

SAN Storage Architecture – Server/blade ->HBA->FC Switch->Storage

 

 

image reference: http://www.fujitsu.com/ie/products/computing/servers/primergy/management/primergy-blade-server-io-virtualization.html

Get detailed information from a disk

bitarray.root#hdparm -I /dev/sda|more

/dev/sda:

ATA device, with non-removable media
  Model Number:       AXNS381E-256GM-B                        
  Serial Number:      2F2520037935        
  Firmware Revision:  263j-HP 
  Transport:          Serial, ATA8-AST, SATA 1.0a, SATA II Extensions, SATA Rev 2.5, SATA Rev 2.6, SATA Rev 3.0
Standards:
  Supported: 9 8 7 6 5 
  Likely used: 9
Configuration:
  Logical		max	current
  cylinders	16383	16383
  heads		16	16
  sectors/track	63	63
  --
  CHS current addressable sectors:   16514064
  LBA    user addressable sectors:  268435455
  LBA48  user addressable sectors:  500118192
  Logical  Sector size:                   512 bytes
  Physical Sector size:                   512 bytes
  Logical Sector-0 offset:                  0 bytes
  device size with M = 1024*1024:      244198 MBytes
  device size with M = 1000*1000:      256060 MBytes (256 GB)
...
...
...

 

How to check if you have an SSD drive or Hard Disk?

cat /sys/block/sda/queue/rotational
0

if you see 0 it’s SSD otherwise it’s hard disk. Note the device name “sda” you can check for all disks (if your machine has multiple disks).

 

How to check  RAID settings in Linux?

mdadm is the command to use, see cat /etc/mdadm.conf to get RAID configuration.

Categories

+ There are no comments

Add yours