Contents

Recipes are community-created content. They are neither monitored nor endorsed by IBM. If you find inappropriate content, please use Report Abuse to let us know. For more information on community content, please refer to our Terms of Use.

Overview

Skill Level: Advanced

The user should have basic knowledge on persistent memory technology and device mapper.

This recipe demonstrate the use of device mapper on persistent memory to create larger vol. This is authored by Vaibhav Jain, Harish Sriram,Kalpana Shetty.
A special thanks to Aneesh Veetil for his technical guidance on PMEM technology on Power Platform.

Ingredients

This HowTo document demonstrate how to use the Linux Volume Manager (LVM) to create Device Mapper’s linear logical volumes using persistent memory as the backing storage device on Power 9 system.

 
What is the purpose of using LVM with Persistent Memory?

There are times when applications needs volumes larger than what single dimms or regions/namespace can provide. In such situations, a user can create larger persistent memory volume by using Linux Device Mapper which helps to get a single device persistent device or volume.

This article focuses on using linear device mapper to create LVM volumes on persistent memory devices on Power9 enabled persistent memory systems.

1 

2

Step-by-step

  1. Create Lun of size 16Tb with affinity option

    Create Lun of size 16Tb with affinity option

    Refer to Appendix-A for more details.

    hmc_img1

  2. Activate the partition and ensure OS boots up successfully

    Activate the partition and ensure OS boots up successfully.

    3

    4

    RHEL-8.2

    rh82-ga-1

     

    rh82-ga-3

     

     

  3. Check for regions created by OS for volume size of 16Tb

    For 16Tb volume size, OS created 8 regions. Starting region0 – region7.

    7_1

    7_2

    7_3

    7_4

     

     Note: For presenting purposes, “ndctl list -Ru” has been pasted with multiple separate screenshots. Consider it as a single output with 8 regions.

     

  4. Create namespace each of size equal to the region size

    This can be done by omitting the ‘–size’ argument to the create-namespace command which should create a namespace spanning entire free space available in the region.

     For example following command will create such a namespace on region4 of size 1636.39GiB.

    8

     

    # Create namspace of size equal to region size on all 8 regions.

    This is an example namespace created to demonstrate device mapper creation on persistent device memory. The user can always create multiple namespaces within single region.

    9

    This is the output on screen you will see when executed with above ndctl create-namespace option.

    10_1

    10_2

    10_3

    10_4

    Note: For presenting purposes, “ndctl create-name -r <region#>” ouput has been pasted with multiple separate screenshots. Consider it as a single ouput with 8 namespaces created.

  5. Create LVM Physical Volume on all the persistent memory devices

    # pvcreate /dev/pmem{0..7}

    # Run below script to create physical volume on {0..7} persistent memory devices.

    11

     

    12

    # List newly created Pvs

    13

     

     

     

  6. Add the Physical Volumes to a Volume Group

    # vgcreate PmemVol /dev/pmem{0..7}

    14

    By running pvs the user can verify physical volumes now are associated with the new volume group “PmemVol”.

    15

    16

    17

  7. Create Device Mapper’s Linear Logical Volumes

    To create device mapper’s linear volumes, you can specify with the new volume name using the -n option, a size using the -L option, and the volume group on which to create the logical volume.

    18

  8. Validate Logical Volumes on pmem

    This will validate newly created logical volumes, UUID of pmem namespace, ndctl lists with UUID

    19

    20

    21

     

  9. Create File Systems on Logical Volume

    # Create file system on logical volumes with an ext4 file system

    22

     

    # Finally mount with dax option, and verify the file systems mounted

    23

    24

     #Verify the volume size of 16T with df command

    25

  10. Summary

    Summary
    This article outlined with using Linux Device Mapper how application can create large amounts of persistent memory than a single region/namespace can provide.

    The example demonstrated with 16Tb Lun creation, followed with OS populated multiple regions, and namespace creation on each of those regions. With this setup, it has shown how to use the Linux Device Mapper with Persistent Memory Devices to create a device mapper linear volume of size 16Tb.

  11. Appendix-A

    Appendix-A

    Partition level persistent memory volume aka Lun creation

    hmc_img2

    Select Persistent Memory option.

    hmc_img3

    Click on “Add” to add 16Tb volume or lun, enable affinity option.

    hmc_img4

    After successful Volume creation you should see 16Tb Volume created.

    hmc_img5

  12. Contact Details

    You can contact any of the below people for any questions on PMEM on POWER 9.

    – Aneesh Kk Veetil/India/IBM, Vaibhav Jain21/India/IBM
    – Harish Sriram/India/IBM, Kalpana Shetty/India/IBM

     

Join The Discussion